dsh-line-bot 0.9.7 → 0.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +2 -0
- package/CHANGELOG.md +7 -0
- package/README.en.md +10 -1
- package/README.ja.md +10 -1
- package/README.ko.md +10 -1
- package/README.md +10 -1
- package/README.zh-CN.md +10 -1
- package/TROUBLESHOOTING.md +6 -0
- package/client.js +131 -2
- package/lib/config.mjs +26 -0
- package/lib/host.mjs +177 -6
- package/lib/tts.mjs +540 -0
- package/package.json +3 -2
- package/tools/mlx_synth.py +60 -0
package/AGENTS.md
CHANGED
|
@@ -31,6 +31,7 @@ curl -s http://127.0.0.1:8791/health
|
|
|
31
31
|
| 主動訊息沒出現 | 面板「主動傳訊」 | 每位使用者的倒數還沒到、每日/每月上限、靜音時段、兩則間距冷卻、連續未回覆已被暫停、沒有 session 或不在白名單 | 看 `status.nudge`、使用者表的「下次主動/未回覆」與日誌 |
|
|
32
32
|
| 餘額提醒沒出現 | `balance` | provider 不是 `deepseek-account`、餘額高於門檻、冷卻中、靜音 | 面板「DeepSeek 餘額」看 `applicable`、`lastAskAt` |
|
|
33
33
|
| Windows 安裝失敗 | CLI | 找不到 DSH 內建 pnpm;舊版 spawn `.cmd` 失敗 | 用面板安裝,或升級 CLI;必要時 `npm i -g pnpm` |
|
|
34
|
+
| 語音沒送出或一直回文字 | 面板「語音回覆」、日誌 | 尚未按一鍵配置環境、未確認聲音使用權、沒有 ffmpeg、MLX 載入失敗、reply token 過期 | 面板「語音回覆」按一鍵配置與測試;看日誌 `語音回覆`/`語音合成失敗`;必要時看 `status.voice` |
|
|
34
35
|
| npm `ETARGET` | — | registry/CDN 還沒傳播 | 等 1–5 分鐘再試 |
|
|
35
36
|
|
|
36
37
|
## 2. 圖片沒送出的排查順序
|
|
@@ -50,6 +51,7 @@ curl -s http://127.0.0.1:8791/health
|
|
|
50
51
|
- `lib/meme-source.mjs`:讀 dsh-meme。
|
|
51
52
|
- `lib/chat-import.mjs`:LINE 匯出聊天解析。
|
|
52
53
|
- `lib/diagnostics.mjs`:doctor 用的檢查工具。
|
|
54
|
+
- `lib/tts.mjs`/`tools/mlx_synth.py`:CosyVoice3 語音合成(MLX/GGUF)。
|
|
53
55
|
- `bin/cli.mjs`:安裝/`doctor`。
|
|
54
56
|
- HTTP:`/api/line-bot/status|config|action|doctor`(webServer)、`http://127.0.0.1:<port>/doctor|health|media/<id>|webhook`(外掛自己的伺服器)。
|
|
55
57
|
- 資料:`$DSH_HOME/line-bot/`(config.json 600、state.json、memories.json、logs、personas、cache、users)。
|
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,10 @@
|
|
|
1
|
+
## 0.10.0
|
|
2
|
+
|
|
3
|
+
- 新增語音回覆(CosyVoice3-0.5B):角色可以在回覆最後加 `[[voice]]`,插件會把文字合成語音並用 LINE 語音訊息送出;合成失敗或 reply token 過期時自動改用文字/push。
|
|
4
|
+
- macOS(Apple Silicon)用 MLX 在本機合成;Windows/沒有 MLX 時用 CosyVoice GGUF(CrispASR)。面板可一鍵偵測與配置環境、開關語音、上傳參考音、調整語氣指令與長度、試聽。
|
|
5
|
+
- 參考音預設使用指定音檔的 12–16 秒片段(存在 `<DSH_HOME>/line-bot/tts/ref/default.wav`);可上傳自己的音檔替換。
|
|
6
|
+
- 使用前必須在面板確認「有使用此聲音的權利」;語音檔會存在本機媒體庫,透過 tunnel 的 `/media/<token>` 提供給 LINE。
|
|
7
|
+
|
|
1
8
|
# Changelog
|
|
2
9
|
|
|
3
10
|
## 0.9.7
|
package/README.en.md
CHANGED
|
@@ -99,6 +99,7 @@ At the top of the panel is a **sticky save bar**: "Save changes / Refresh" stays
|
|
|
99
99
|
| Jokes | Text joke library: built-in seeds, Tieba quotes, wanjiqi barrage memes; per-source toggles, per-turn injection count, add/delete entries |
|
|
100
100
|
| Personal memory | Per-user long-term memory, expiry cleanup, daily self-review, manual delete |
|
|
101
101
|
| Memes | Reply with images/stickers from the dsh-meme library, enable/disable, image limit, and sample previews |
|
|
102
|
+
| Voice replies | CosyVoice3 speech synthesis, on/off, reference upload, instruct style, audio preview |
|
|
102
103
|
| Sticker & image vision | Fetch incoming LINE stickers or images and show them to the model; when the model cannot read images, describe them with modlens or a vision model (can be disabled) |
|
|
103
104
|
| DeepSeek balance | Current balance, per-turn context, low-balance proactive reminder, threshold and cooldown (deepseek-account only) |
|
|
104
105
|
| Proactive messages | Toggle, per-user countdown, min/max interval, unanswered limit, daily/monthly caps, minimum gap, quiet hours, idle, target users, prompt, send a test now, statistics (next possible time and paused users) |
|
|
@@ -224,6 +225,14 @@ The persona can attach a sticker to a text reply, or reply with just an image. I
|
|
|
224
225
|
- When a LINE sticker arrives, the bot fetches the image and shows it to the model; vision-capable models read it directly, otherwise the bot asks modlens, then tries another vision model, and only then falls back to the keywords LINE provides. Turn "Understand stickers" off in the LINE credentials card to use keywords only.
|
|
225
226
|
- Regular images work the same way: the bot downloads them through LINE's content API and attaches them to the message; if the model cannot read images, the same modlens / vision-model description path is used. Turn "Understand images" off in the LINE credentials card to skip them.
|
|
226
227
|
- Without dsh-meme installed, the feature stays off and replies are text only.
|
|
228
|
+
## Voice replies (CosyVoice3)
|
|
229
|
+
|
|
230
|
+
- The persona can put `[[voice]]` at the end of a reply; the plugin synthesizes the text with CosyVoice3-0.5B and sends a LINE audio message (m4a).
|
|
231
|
+
- On Apple Silicon macOS it synthesizes locally with MLX; on Windows (or without MLX) it uses CosyVoice GGUF (CrispASR). Press "Set up environment" once in the panel (downloads roughly 1–2 GB).
|
|
232
|
+
- The "Voice replies" card toggles the feature, requires confirming that you have the rights to the voice, lets you upload a reference clip (the default is a 12–16 s cut of the provided sample), choose instruct/zero-shot mode, and preview the audio.
|
|
233
|
+
- Synthesis takes about 20–40 s; if the reply token expires it falls back to push. Audio is stored under `<DSH_HOME>/line-bot/media/` and served at `/media/<token>` through the tunnel.
|
|
234
|
+
- Requires `ffmpeg`; MLX mode needs Python 3.10+.
|
|
235
|
+
|
|
227
236
|
## Diagnostics
|
|
228
237
|
- Panel → Advanced → Run diagnostics checks the server, tunnel, public URL, webhook, dsh-meme library, image converters, model, and recent errors; Safe repair only reconnects the tunnel and re-applies the webhook.
|
|
229
238
|
- CLI: `npx dsh-line-bot doctor`, `doctor --fix`, `doctor --json`.
|
|
@@ -258,4 +267,4 @@ The persona can attach a sticker to a text reply, or reply with just an image. I
|
|
|
258
267
|
|
|
259
268
|
- Host logic: `lib/*.mjs` (`util` / `config` / `line-api` / `persona` / `agent-runner` / `webhook-server` / `cloudflare` / `tunnel` / `nudge` / `tieba-sync` / `host`).
|
|
260
269
|
- Panel: `client.js` (registers `settings.section` via `window.__ModuleLoader__`).
|
|
261
|
-
- Entry point: `index.js` (mounts `/api/line-bot`, dev reload, cleanup on unload).
|
|
270
|
+
- Entry point: `index.js` (mounts `/api/line-bot`, dev reload, cleanup on unload).
|
package/README.ja.md
CHANGED
|
@@ -99,6 +99,7 @@ npx dsh-line-bot install --no-companions
|
|
|
99
99
|
| ジョーク | テキストネタ集:内蔵シード、貼吧名台詞、wanjiqi 弾幕ネタ。ソース別の切り替え、ターンごとの注入数、追加/削除 |
|
|
100
100
|
| 個人メモリ | ユーザーごとの長期記憶、期限切れの自動削除、毎日の自己整理、手動削除 |
|
|
101
101
|
| 表情パック | dsh-meme の画像ライブラリで画像/スタンプを返信、オンオフ、1 回の上限、サンプル表示 |
|
|
102
|
+
| 音声返信 | CosyVoice3 による音声合成、オンオフ、参照音声アップロード、instruct、試聴 |
|
|
102
103
|
| スタンプ・画像理解 | LINE スタンプや通常の画像を受信したら取得してモデルに見せる。画像非対応モデルは modlens や別の視覚モデルで説明(オフ可) |
|
|
103
104
|
| DeepSeek 残高 | 現在の残高、毎ターンの注入、残高が少ないときの主動リマインド、しきい値とクールダウン(deepseek-account のみ) |
|
|
104
105
|
| 自発メッセージ | オン/オフ、ユーザーごとのカウントダウン、最短/最長間隔、連続未返信の上限、1 日/1 か月の上限、最低間隔、静かな時間帯、idle、対象ユーザー、プロンプト、今すぐ試送、統計(次回予定と一時停止中の人数) |
|
|
@@ -224,6 +225,14 @@ Bot がグループ(または複数人のルーム)に追加されたとき
|
|
|
224
225
|
- LINE スタンプを受信すると、スタンプ画像を取得してモデルに見せます。画像対応モデルはそのまま読み、非対応ならまず modlens、次に別の視覚モデルで説明し、最後に LINE のキーワードを使います。「LINE 認証情報」カードの「スタンプを理解する」でオフにできます。
|
|
225
226
|
- 通常の画像も同じです。LINE content API でダウンロードしてモデルに添付し、画像非対応なら同じ modlens/視覚モデルの説明経路を使います。「LINE 認証情報」カードの「画像を理解する」でオフにできます。
|
|
226
227
|
- dsh-meme がインストールされていない場合、この機能は無効になりテキストのみ返します。
|
|
228
|
+
## 音声返信(CosyVoice3)
|
|
229
|
+
|
|
230
|
+
- ペルソナは返信の最後に `[[voice]]` を付けると、CosyVoice3-0.5B で音声合成し、LINE の音声メッセージ(m4a)として送信します。
|
|
231
|
+
- Apple Silicon の macOS では MLX でローカル合成します。Windows や MLX がない環境では CosyVoice GGUF(CrispASR)を使います。初回はパネルの「環境を一括セットアップ」を押してください(モデルを約 1–2GB ダウンロードします)。
|
|
232
|
+
- パネルの「音声返信」でオンオフ、音声使用権の確認、参照音声のアップロード(既定は指定音源の 12–16 秒)、instruct/zero-shot の選択、試聴ができます。
|
|
233
|
+
- 合成には約 20–40 秒かかります。reply token の期限が切れた場合は push に切り替わります。音声は `<DSH_HOME>/line-bot/media/` に保存され、tunnel の `/media/<token>` で配信されます。
|
|
234
|
+
- `ffmpeg` が必要です。MLX モードは Python 3.10+ が必要です。
|
|
235
|
+
|
|
227
236
|
## 診断
|
|
228
237
|
- パネルの「詳細設定 → 診断を実行」でサーバー、Tunnel、公開 URL、webhook、dsh-meme 図庫、画像変換ツール、モデル、最近のエラーを確認できます。「安全な修復」は tunnel 再接続と webhook 再設定だけを行います。
|
|
229
238
|
- CLI:`npx dsh-line-bot doctor`、`doctor --fix`、`doctor --json`。
|
|
@@ -258,4 +267,4 @@ Bot がグループ(または複数人のルーム)に追加されたとき
|
|
|
258
267
|
|
|
259
268
|
- host ロジック:`lib/*.mjs`(`util`/`config`/`line-api`/`persona`/`agent-runner`/`webhook-server`/`cloudflare`/`tunnel`/`nudge`/`tieba-sync`/`host`)。
|
|
260
269
|
- パネル:`client.js`(`window.__ModuleLoader__` で `settings.section` を登録)。
|
|
261
|
-
- エントリーポイント:`index.js`(`/api/line-bot` のマウント、dev reload、アンロード時のクリーンアップ)。
|
|
270
|
+
- エントリーポイント:`index.js`(`/api/line-bot` のマウント、dev reload、アンロード時のクリーンアップ)。
|
package/README.ko.md
CHANGED
|
@@ -99,6 +99,7 @@ npx dsh-line-bot install --no-companions
|
|
|
99
99
|
| 말장난 | 텍스트 밈: 내장 시드, 貼吧 명언, wanjiqi 탄막 밈. 출처별 켜기/끄기, 턴당 주입 수, 추가/삭제 |
|
|
100
100
|
| 개인 기억 | 사용자별 장기 기억, 만료 자동 삭제, 매일 자기 정리, 수동 삭제 |
|
|
101
101
|
| 이모티콘 | dsh-meme 라이브러리로 이미지/스티커 답장, 켜기/끄기, 1회 최대 장수, 샘플 미리보기 |
|
|
102
|
+
| 음성 답장 | CosyVoice3 음성 합성, 켜기/끄기, 참조 음성 업로드, instruct, 미리듣기 |
|
|
102
103
|
| 스티커·이미지 이해 | LINE 스티커나 일반 이미지를 받으면 가져와 모델에 보여 줌. 이미지 미지원 모델은 modlens 또는 다른 비전 모델로 설명 (끌 수 있음) |
|
|
103
104
|
| DeepSeek 잔액 | 현재 잔액, 매 턴 주입, 잔액이 적을 때 캐릭터가 먼저 알림, 기준 금액과 쿨다운(deepseek-account 전용) |
|
|
104
105
|
| 능동 메시지 | 켜기/끄기, 사용자별 카운트다운, 최소/최대 간격, 연속 미응답 상한, 일일/월간 상한, 최소 간격, 조용한 시간대, idle, 대상 사용자, 프롬프트, 지금 시험 발송, 통계(다음 예정 시각과 일시 중지 인원) |
|
|
@@ -222,6 +223,14 @@ Bot이 그룹(또는 다인 룸)에 추가되면:
|
|
|
222
223
|
- LINE 스티커를 받으면 스티커 이미지를 가져와 모델에 보여 줍니다. 이미지 지원 모델은 바로 읽고, 아니면 먼저 modlens, 다음으로 다른 비전 모델로 설명하고, 마지막에 LINE 키워드로 대체합니다. 「LINE 자격 증명」 카드에서 「스티커 이해」를 끌 수 있습니다.
|
|
223
224
|
- 일반 이미지도 같습니다. LINE content API로 내려받아 모델에 첨부하고, 이미지 미지원이면 같은 modlens/비전 모델 설명 경로를 씁니다. 「LINE 자격 증명」 카드에서 「이미지 이해」를 끌 수 있습니다.
|
|
224
225
|
- dsh-meme이 설치되어 있지 않으면 이 기능은 꺼지고 텍스트만 답장합니다.
|
|
226
|
+
## 음성 답장(CosyVoice3)
|
|
227
|
+
|
|
228
|
+
- 페르소나가 답장 끝에 `[[voice]]`를 붙이면 CosyVoice3-0.5B로 음성을 합성해 LINE 음성 메시지(m4a)로 보냅니다.
|
|
229
|
+
- Apple Silicon macOS에서는 MLX로 로컬 합성합니다. Windows이거나 MLX가 없으면 CosyVoice GGUF(CrispASR)를 사용합니다. 처음에는 패널의 「환경 일괄 설정」을 한 번 누르세요(모델 약 1–2GB 다운로드).
|
|
230
|
+
- 패널 「음성 답장」에서 켜기/끄기, 음성 사용 권리 확인, 참조 음성 업로드(기본값은 지정 음원의 12–16초), instruct/zero-shot 선택, 미리듣기를 할 수 있습니다.
|
|
231
|
+
- 합성에는 약 20–40초가 걸립니다. reply token이 만료되면 push로 자동 전환됩니다. 음성은 `<DSH_HOME>/line-bot/media/`에 저장되고 tunnel의 `/media/<token>`으로 제공됩니다.
|
|
232
|
+
- `ffmpeg`가 필요합니다. MLX 모드는 Python 3.10+가 필요합니다.
|
|
233
|
+
|
|
225
234
|
## 진단
|
|
226
235
|
- 패널의 "고급 설정 → 진단 실행"에서 서버, Tunnel, 공개 URL, webhook, dsh-meme 라이브러리, 이미지 변환 도구, 모델, 최근 오류를 확인할 수 있습니다. "안전 복구"는 tunnel 재연결과 webhook 재설정만 수행합니다.
|
|
227
236
|
- CLI: `npx dsh-line-bot doctor`, `doctor --fix`, `doctor --json`.
|
|
@@ -256,4 +265,4 @@ Bot이 그룹(또는 다인 룸)에 추가되면:
|
|
|
256
265
|
|
|
257
266
|
- host 로직: `lib/*.mjs`(`util`/`config`/`line-api`/`persona`/`agent-runner`/`webhook-server`/`cloudflare`/`tunnel`/`nudge`/`tieba-sync`/`host`).
|
|
258
267
|
- 패널: `client.js`(`window.__ModuleLoader__`로 `settings.section` 등록).
|
|
259
|
-
- 진입점: `index.js`(`/api/line-bot` 마운트, dev reload, 언로드 정리).
|
|
268
|
+
- 진입점: `index.js`(`/api/line-bot` 마운트, dev reload, 언로드 정리).
|
package/README.md
CHANGED
|
@@ -99,6 +99,7 @@ npx dsh-line-bot install --no-companions
|
|
|
99
99
|
| 玩梗 | 文字梗庫:內建種子、貼吧金句、wanjiqi 彈幕梗;三個來源可個別開關、每輪注入幾則、新增/刪除梗 |
|
|
100
100
|
| 個人記憶 | 每位使用者分開的長期記憶、到期自動刪、每日請角色整理、手動刪除 |
|
|
101
101
|
| 表情包 | 使用 dsh-meme 圖庫回圖片/貼圖、開關、一次最多幾張、圖庫樣本 |
|
|
102
|
+
| 語音回覆 | CosyVoice3 語音合成、開關、參考音上傳、語氣指令、試聽 |
|
|
102
103
|
| 貼圖與圖片理解 | 收到 LINE 貼圖或一般圖片時抓圖給模型看;模型不吃圖時用 modlens 或另一個視覺模型描述(可關閉) |
|
|
103
104
|
| DeepSeek 餘額 | 目前餘額、每輪注入、低於門檻時由角色主動提醒、提醒門檻與冷卻(僅 deepseek-account) |
|
|
104
105
|
| 主動傳訊 | 開關、每位使用者的倒數、最短/最長間隔、連續未回覆上限、每日/每月上限、兩則間距、安靜時段、idle、指定對象、提示詞、立即試傳、統計(下次可能時間與暫停人數) |
|
|
@@ -224,6 +225,14 @@ Bot 被加進群組(或多人聊天室)後:
|
|
|
224
225
|
- 收到 LINE 貼圖時,會抓貼圖圖片交給模型看;模型支援圖片就直接讀,否則先用 modlens 描述,再不支援就改用另一個視覺模型,最後才退回 LINE 提供的關鍵字。可在「LINE 憑證」卡關閉「看得懂貼圖」。
|
|
225
226
|
- 一般圖片也一樣:透過 LINE content API 下載後直接附給模型;模型不吃圖時走同一套 modlens/視覺模型描述。可在「LINE 憑證」卡關閉「看得懂一般圖片」。
|
|
226
227
|
- 沒安裝 dsh-meme 時這個功能會自動停用,只回文字。
|
|
228
|
+
## 語音回覆(CosyVoice3)
|
|
229
|
+
|
|
230
|
+
- 角色可以在回覆最後加 `[[voice]]`,插件會用 CosyVoice3-0.5B 把文字合成語音,並以 LINE 語音訊息送出(m4a)。
|
|
231
|
+
- macOS(Apple Silicon)在本機用 MLX 合成;Windows/沒有 MLX 時用 CosyVoice GGUF(CrispASR)。第一次要先在面板按「一鍵配置環境」(會下載模型,約 1–2GB)。
|
|
232
|
+
- 面板「語音回覆」可開關、確認聲音使用權、上傳參考音(預設用指定音檔的 12–16 秒片段)、選推理模式與語氣指令、試聽。
|
|
233
|
+
- 合成需要約 20–40 秒;若 reply token 過期會自動改用 push。語音檔存在 `<DSH_HOME>/line-bot/media/`,透過 tunnel 的 `/media/<token>` 提供。
|
|
234
|
+
- 需要 `ffmpeg`;MLX 模式需要 Python 3.10+。
|
|
235
|
+
|
|
227
236
|
## 診斷與修復
|
|
228
237
|
- 面板「進階設定 → 執行診斷」會檢查伺服器、Tunnel、公開網址、webhook、表情包圖庫、轉圖工具、模型與最近錯誤;「自動修復(安全)」只會重連 tunnel、重設 webhook。
|
|
229
238
|
- 終端機:`npx dsh-line-bot doctor`、`npx dsh-line-bot doctor --fix`、`doctor --json`。
|
|
@@ -258,4 +267,4 @@ Bot 被加進群組(或多人聊天室)後:
|
|
|
258
267
|
|
|
259
268
|
- host 邏輯:`lib/*.mjs`(`util`/`config`/`line-api`/`persona`/`agent-runner`/`webhook-server`/`cloudflare`/`tunnel`/`nudge`/`tieba-sync`/`host`)。
|
|
260
269
|
- 面板:`client.js`(`window.__ModuleLoader__` 註冊 `settings.section`)。
|
|
261
|
-
- 進入點:`index.js`(掛 `/api/line-bot`、dev reload、卸載清理)。
|
|
270
|
+
- 進入點:`index.js`(掛 `/api/line-bot`、dev reload、卸載清理)。
|
package/README.zh-CN.md
CHANGED
|
@@ -99,6 +99,7 @@ npx dsh-line-bot install --no-companions
|
|
|
99
99
|
| 玩梗 | 文字梗库:内置种子、贴吧金句、wanjiqi 弹幕梗;三个来源可分别开关、每轮注入几条、新增/删除梗 |
|
|
100
100
|
| 个人记忆 | 每位用户分开的长期记忆、到期自动删除、每日请角色整理、手动删除 |
|
|
101
101
|
| 表情包 | 使用 dsh-meme 图库回图片/贴图、开关、每次最多几张、图库样本 |
|
|
102
|
+
| 语音回复 | CosyVoice3 语音合成、开关、参考音频上传、语气指令、试听 |
|
|
102
103
|
| 贴图与图片理解 | 收到 LINE 贴图或一般图片时抓图给模型看;模型不吃图时用 modlens 或另一个视觉模型描述(可关闭) |
|
|
103
104
|
| DeepSeek 余额 | 当前余额、每轮注入、低于门槛时由角色主动提醒、提醒门槛与冷却(仅 deepseek-account) |
|
|
104
105
|
| 主动传讯 | 开关、每位使用者的倒数、最短/最长间隔、连续未回复上限、每日/每月上限、两则间距、安静时段、idle、指定对象、提示词、立即试传、统计(下次可能时间与暂停人数) |
|
|
@@ -224,6 +225,14 @@ Bot 被加入群组(或多人聊天室)后:
|
|
|
224
225
|
- 收到 LINE 贴图时,会抓贴图图片交给模型看;模型支持图片就直接读,否则先用 modlens 描述,再不支持就改用另一个视觉模型,最后才退回 LINE 提供的关键词。可在「LINE 凭证」卡关闭「看得懂贴图」。
|
|
225
226
|
- 一般图片也一样:通过 LINE content API 下载后直接附给模型;模型不吃图时走同一套 modlens/视觉模型描述。可在「LINE 凭证」卡关闭「看得懂一般图片」。
|
|
226
227
|
- 没安装 dsh-meme 时该功能会自动停用,只回文字。
|
|
228
|
+
## 语音回复(CosyVoice3)
|
|
229
|
+
|
|
230
|
+
- 角色可以在回复最后加 `[[voice]]`,插件会用 CosyVoice3-0.5B 把文字合成语音,并以 LINE 语音消息发送(m4a)。
|
|
231
|
+
- macOS(Apple Silicon)在本机用 MLX 合成;Windows/没有 MLX 时用 CosyVoice GGUF(CrispASR)。第一次要先在面板按「一键配置环境」(会下载模型,约 1–2GB)。
|
|
232
|
+
- 面板「语音回复」可开关、确认声音使用权、上传参考音频(默认用指定音频的 12–16 秒片段)、选择推理模式与语气指令、试听。
|
|
233
|
+
- 合成需要约 20–40 秒;若 reply token 过期会自动改用 push。语音文件存在 `<DSH_HOME>/line-bot/media/`,通过 tunnel 的 `/media/<token>` 提供。
|
|
234
|
+
- 需要 `ffmpeg`;MLX 模式需要 Python 3.10+。
|
|
235
|
+
|
|
227
236
|
## 诊断与修复
|
|
228
237
|
- 面板「高级设置 → 执行诊断」会检查服务器、Tunnel、公开网址、webhook、表情包图库、转图工具、模型与最近错误;「自动修复(安全)」只会重连 tunnel、重设 webhook。
|
|
229
238
|
- 终端:`npx dsh-line-bot doctor`、`npx dsh-line-bot doctor --fix`、`doctor --json`。
|
|
@@ -258,4 +267,4 @@ Bot 被加入群组(或多人聊天室)后:
|
|
|
258
267
|
|
|
259
268
|
- host 逻辑:`lib/*.mjs`(`util`/`config`/`line-api`/`persona`/`agent-runner`/`webhook-server`/`cloudflare`/`tunnel`/`nudge`/`tieba-sync`/`host`)。
|
|
260
269
|
- 面板:`client.js`(`window.__ModuleLoader__` 注册 `settings.section`)。
|
|
261
|
-
- 进入点:`index.js`(挂 `/api/line-bot`、dev reload、卸载清理)。
|
|
270
|
+
- 进入点:`index.js`(挂 `/api/line-bot`、dev reload、卸载清理)。
|
package/TROUBLESHOOTING.md
CHANGED
|
@@ -32,6 +32,12 @@ npx dsh-line-bot doctor --fix # 安全修復:重連 tunnel、重設 webhook
|
|
|
32
32
|
- 模型不支援時,插件會先用 modlens 描述,再改用另一個視覺模型,最後才退回貼圖關鍵字。
|
|
33
33
|
- modlens 可在「進階 → 診斷」看 `vision` check;失敗通常是 provider 額度或 503。一般圖片可在面板「LINE 憑證」關閉「看得懂一般圖片」。
|
|
34
34
|
|
|
35
|
+
### 語音沒送出或一直回文字
|
|
36
|
+
- 面板「語音回覆」要開啟、確認聲音使用權,並按過「一鍵配置環境」(第一次會下載模型)。
|
|
37
|
+
- macOS 需要 Apple Silicon + Python 3.10+;Windows 需要 `ffmpeg`。音檔上傳後可按「測試語音」試聽。
|
|
38
|
+
- 合成約需 20–40 秒;reply token 過期時會自動改用 push。日誌看 `語音回覆`、`語音合成失敗`。
|
|
39
|
+
- 模型只有在語言上下文有語音提示時才會加 `[[voice]]`;可先用「測試 Agent 回覆」確認。
|
|
40
|
+
|
|
35
41
|
### APNG 不會動
|
|
36
42
|
- LINE 不播放 APNG,只顯示第一幀。保持 `stickers.apng=false` 即可。
|
|
37
43
|
|
package/client.js
CHANGED
|
@@ -23,6 +23,7 @@ window.__ModuleLoader__.load({
|
|
|
23
23
|
{ key: 'balance', label: 'DeepSeek 餘額' },
|
|
24
24
|
{ key: 'nudge', label: '主動傳訊' },
|
|
25
25
|
{ key: 'meme', label: '表情包' },
|
|
26
|
+
{ key: 'voice', label: '語音回覆' },
|
|
26
27
|
{ key: 'users', label: '使用者' },
|
|
27
28
|
{ key: 'group', label: '群組' },
|
|
28
29
|
{ key: 'advanced', label: '進階/日誌' },
|
|
@@ -366,6 +367,8 @@ window.__ModuleLoader__.load({
|
|
|
366
367
|
const [modelCatalog, setModelCatalog] = useState(null)
|
|
367
368
|
const [importGroup, setImportGroup] = useState('')
|
|
368
369
|
const [doctorResult, setDoctorResult] = useState(null)
|
|
370
|
+
const [voicePreview, setVoicePreview] = useState(null)
|
|
371
|
+
const voiceInputRef = useRef(null)
|
|
369
372
|
const importInput = useRef(null)
|
|
370
373
|
const [localEdit, setLocalEdit] = useState({ id: '', name: '', text: '' })
|
|
371
374
|
|
|
@@ -697,6 +700,91 @@ window.__ModuleLoader__.load({
|
|
|
697
700
|
await act('tieba-bars-add', {})
|
|
698
701
|
}
|
|
699
702
|
|
|
703
|
+
async function installVoice() {
|
|
704
|
+
setBusy('voice-install')
|
|
705
|
+
setToast(null)
|
|
706
|
+
try {
|
|
707
|
+
await jpost('/action', { action: 'voice-install' })
|
|
708
|
+
setToast({ kind: 'ok', text: '已開始配置語音環境;完成前可以留在這一頁看狀態。' })
|
|
709
|
+
} catch (err) {
|
|
710
|
+
setToast({ kind: 'error', text: '配置失敗:' + (err.message || err) })
|
|
711
|
+
} finally {
|
|
712
|
+
setBusy('')
|
|
713
|
+
}
|
|
714
|
+
}
|
|
715
|
+
|
|
716
|
+
async function detectVoice() {
|
|
717
|
+
setBusy('voice-detect')
|
|
718
|
+
setToast(null)
|
|
719
|
+
try {
|
|
720
|
+
const value = await jpost('/action', { action: 'voice-detect' })
|
|
721
|
+
const result = value?.result || {}
|
|
722
|
+
setToast({ kind: 'ok', text: '偵測完成:' + (result.ready ? '環境就緒' : '尚未就緒(可按一鍵配置)') })
|
|
723
|
+
await refresh()
|
|
724
|
+
} catch (err) {
|
|
725
|
+
setToast({ kind: 'error', text: '偵測失敗:' + (err.message || err) })
|
|
726
|
+
} finally {
|
|
727
|
+
setBusy('')
|
|
728
|
+
}
|
|
729
|
+
}
|
|
730
|
+
|
|
731
|
+
async function testVoice() {
|
|
732
|
+
if (dirty) await persist(false).catch(() => {})
|
|
733
|
+
setBusy('voice-preview')
|
|
734
|
+
setToast(null)
|
|
735
|
+
try {
|
|
736
|
+
const value = await jpost('/action', { action: 'voice-preview', payload: { text: '主人,這是語音測試,聽得到嗎?' } })
|
|
737
|
+
const result = value?.result || null
|
|
738
|
+
setVoicePreview(result)
|
|
739
|
+
setToast({ kind: 'ok', text: result?.url ? '合成完成,可以試聽。' : '合成完成,但沒有公開網址;請確認 tunnel。' })
|
|
740
|
+
} catch (err) {
|
|
741
|
+
setToast({ kind: 'error', text: '語音測試失敗:' + (err.message || err) })
|
|
742
|
+
} finally {
|
|
743
|
+
setBusy('')
|
|
744
|
+
}
|
|
745
|
+
}
|
|
746
|
+
|
|
747
|
+
async function uploadVoice(event) {
|
|
748
|
+
const file = event?.target?.files?.[0]
|
|
749
|
+
if (event?.target) event.target.value = ''
|
|
750
|
+
if (!file) return
|
|
751
|
+
if (file.size > 20 * 1024 * 1024) {
|
|
752
|
+
setToast({ kind: 'error', text: '音檔超過 20MB。' })
|
|
753
|
+
return
|
|
754
|
+
}
|
|
755
|
+
setBusy('voice-save-reference')
|
|
756
|
+
setToast(null)
|
|
757
|
+
try {
|
|
758
|
+
const dataUrl = await new Promise((resolve, reject) => {
|
|
759
|
+
const reader = new FileReader()
|
|
760
|
+
reader.onload = () => resolve(String(reader.result || ''))
|
|
761
|
+
reader.onerror = () => reject(new Error('讀取檔案失敗'))
|
|
762
|
+
reader.readAsDataURL(file)
|
|
763
|
+
})
|
|
764
|
+
await jpost('/action', { action: 'voice-save-reference', payload: { dataUrl, filename: file.name } })
|
|
765
|
+
await refresh()
|
|
766
|
+
setToast({ kind: 'ok', text: '參考音已更新。' })
|
|
767
|
+
} catch (err) {
|
|
768
|
+
setToast({ kind: 'error', text: '上傳失敗:' + (err.message || err) })
|
|
769
|
+
} finally {
|
|
770
|
+
setBusy('')
|
|
771
|
+
}
|
|
772
|
+
}
|
|
773
|
+
|
|
774
|
+
async function resetVoiceReference() {
|
|
775
|
+
setBusy('voice-reset-reference')
|
|
776
|
+
setToast(null)
|
|
777
|
+
try {
|
|
778
|
+
await jpost('/action', { action: 'voice-reset-reference' })
|
|
779
|
+
await refresh()
|
|
780
|
+
setToast({ kind: 'ok', text: '已改用內建預設參考音。' })
|
|
781
|
+
} catch (err) {
|
|
782
|
+
setToast({ kind: 'error', text: '重設失敗:' + (err.message || err) })
|
|
783
|
+
} finally {
|
|
784
|
+
setBusy('')
|
|
785
|
+
}
|
|
786
|
+
}
|
|
787
|
+
|
|
700
788
|
async function refresh() {
|
|
701
789
|
setToast(null)
|
|
702
790
|
try {
|
|
@@ -757,6 +845,9 @@ window.__ModuleLoader__.load({
|
|
|
757
845
|
const memoryStatus = status.memory || {}
|
|
758
846
|
const agentStatus = status.agent || {}
|
|
759
847
|
const modelProviders = Array.isArray(modelCatalog?.providers) ? modelCatalog.providers : []
|
|
848
|
+
const voiceCfg = cfg.voice || {}
|
|
849
|
+
const voiceStatus = status.voice || {}
|
|
850
|
+
const voiceSetup = voiceStatus.setup || {}
|
|
760
851
|
const memoryUsers = memoryData?.users || []
|
|
761
852
|
const syncStats = status.sync || {}
|
|
762
853
|
const runtime = status
|
|
@@ -1317,10 +1408,48 @@ window.__ModuleLoader__.load({
|
|
|
1317
1408
|
),
|
|
1318
1409
|
),
|
|
1319
1410
|
|
|
1411
|
+
// 5.5 語音回覆
|
|
1412
|
+
h(
|
|
1413
|
+
Card,
|
|
1414
|
+
{ id: sectionDomId('voice'), order: 12, title: '語音回覆(CosyVoice3)', desc: '用 CosyVoice3-0.5B 把回覆合成語音;角色可以自己決定要不要用語音,也可以只用文字。第一次使用要先按「一鍵配置環境」。' },
|
|
1415
|
+
h('div', { style: S.row },
|
|
1416
|
+
h(Badge, { ok: voiceStatus.ready === true }, voiceStatus.ready ? '語音環境就緒' : '尚未就緒'),
|
|
1417
|
+
h(Badge, { ok: voiceCfg.consent === true }, voiceCfg.consent ? '已確認使用權' : '未確認使用權'),
|
|
1418
|
+
h(Badge, { ok: true }, voiceStatus.mlx?.ok ? 'macOS MLX' : voiceStatus.gguf?.ok ? 'GGUF' : '未安裝引擎'),
|
|
1419
|
+
h(Badge, { ok: !!voiceStatus.reference?.path }, voiceStatus.reference?.path ? ('參考音 ' + (voiceStatus.reference.seconds || 0) + ' 秒') : '沒有參考音'),
|
|
1420
|
+
),
|
|
1421
|
+
h('div', { style: S.hint }, 'macOS 會用 MLX(較快);Windows 或沒有 MLX 時用 CosyVoice GGUF。合成一次約需 20–40 秒,送出時若 reply token 過期會自動改用 push。'),
|
|
1422
|
+
h(Toggle, { checked: voiceCfg.enabled === true, onChange: (value) => update({ voice: { enabled: value } }), label: '開啟語音回覆', hint: '開啟後角色可以自己決定用語音;關閉時完全不合成。' }),
|
|
1423
|
+
h(Toggle, { checked: voiceCfg.consent === true, onChange: (value) => update({ voice: { consent: value } }), label: '我確認有使用此聲音的權利', hint: '使用他人聲音做克隆前請先取得同意;確認後才會合成。' }),
|
|
1424
|
+
h('div', { style: S.row },
|
|
1425
|
+
h(Btn, { onClick: installVoice, disabled: !!busy }, busy === 'voice-install' ? '安裝中…' : '一鍵配置環境'),
|
|
1426
|
+
h(Btn, { onClick: detectVoice, disabled: !!busy }, busy === 'voice-detect' ? '偵測中…' : '重新偵測'),
|
|
1427
|
+
h(Btn, { onClick: testVoice, disabled: !!busy }, busy === 'voice-preview' ? '合成中…' : '測試語音'),
|
|
1428
|
+
h(Btn, { onClick: () => voiceInputRef.current && voiceInputRef.current.click(), disabled: !!busy }, '上傳參考音'),
|
|
1429
|
+
h('input', { ref: voiceInputRef, type: 'file', accept: 'audio/*', style: { display: 'none' }, onChange: uploadVoice }),
|
|
1430
|
+
),
|
|
1431
|
+
h('div', { style: S.hint }, '目前參考音:' + (voiceStatus.reference?.path || '(尚未設定,預設會用插件內建的 12–16 秒片段)')),
|
|
1432
|
+
voiceStatus.reference?.path ? h('div', { style: S.row }, h(Btn, { onClick: resetVoiceReference, disabled: !!busy }, '改用內建預設參考音')) : null,
|
|
1433
|
+
h('div', { style: S.grid },
|
|
1434
|
+
h(Field, { label: '引擎', hint: 'auto 會依系統自動挑選' }, h('select', { value: voiceCfg.provider || 'auto', onChange: (event) => update({ voice: { provider: event.target.value } }), style: S.input }, ['auto', 'mlx', 'gguf'].map((item) => h('option', { key: item, value: item }, item)))),
|
|
1435
|
+
h(Field, { label: '推理模式' }, h('select', { value: voiceCfg.cloneMode || 'instruct', onChange: (event) => update({ voice: { cloneMode: event.target.value } }), style: S.input }, [['instruct', '自然語言語氣控制'], ['zero-shot', '零樣本(需參考音文字)']].map(([value, label]) => h('option', { key: value, value }, label)))),
|
|
1436
|
+
h(Field, { label: '一次最多幾個字', hint: '太長會截斷;語音訊息上限約 60 秒' }, h(Num, { value: voiceCfg.maxChars, min: 20, max: 300, step: 10, onChange: (value) => update({ voice: { maxChars: value } }) })),
|
|
1437
|
+
),
|
|
1438
|
+
h(Field, { label: '語氣指令(instruct)', hint: '用自然語言描述想要的語氣' }, h(Area, { value: voiceCfg.instructText, rows: 2, onChange: (value) => update({ voice: { instructText: value } }) })),
|
|
1439
|
+
voiceCfg.cloneMode === 'zero-shot' ? h(Field, { label: '參考音文字(zero-shot 用)' }, h(Area, { value: voiceCfg.refText, rows: 2, onChange: (value) => update({ voice: { refText: value } }) })) : null,
|
|
1440
|
+
voiceSetup.phase && voiceSetup.phase !== 'idle'
|
|
1441
|
+
? h('div', { style: voiceSetup.phase === 'error' ? S.warnBox : S.note }, '環境狀態:' + (voiceSetup.message || voiceSetup.phase) + (voiceSetup.error ? '(' + voiceSetup.error + ')' : ''))
|
|
1442
|
+
: null,
|
|
1443
|
+
voiceStatus.lastError ? h('div', { style: S.warnBox }, '最近錯誤:' + voiceStatus.lastError) : null,
|
|
1444
|
+
voicePreview
|
|
1445
|
+
? h('div', { style: S.panelBox }, h('div', { style: S.fieldTitle }, '試聽(' + (voicePreview.provider || '') + ',' + Math.round((voicePreview.durationMs || 0) / 1000) + ' 秒)'), h('audio', { controls: true, src: voicePreview.url || voicePreview.localUrl || '', style: { width: '100%' } }))
|
|
1446
|
+
: null,
|
|
1447
|
+
),
|
|
1448
|
+
|
|
1320
1449
|
// 6. 使用者
|
|
1321
1450
|
h(
|
|
1322
1451
|
Card,
|
|
1323
|
-
{ id: sectionDomId('users'), order:
|
|
1452
|
+
{ id: sectionDomId('users'), order: 13, title: '使用者', desc: '與 bot 互動過的 LINE 使用者。每個人有自己的 DSH session 與工作資料夾(記憶、檔案不互通),人設也可以個別指定。' },
|
|
1324
1453
|
(access.pending || []).length > 0
|
|
1325
1454
|
? h('div', { style: S.panelBox },
|
|
1326
1455
|
h('div', { style: S.fieldTitle }, `待審核(${access.pending.length})`),
|
|
@@ -1370,7 +1499,7 @@ window.__ModuleLoader__.load({
|
|
|
1370
1499
|
// 6.5 群組
|
|
1371
1500
|
h(
|
|
1372
1501
|
Card,
|
|
1373
|
-
{ id: sectionDomId('group'), order:
|
|
1502
|
+
{ id: sectionDomId('group'), order: 14, title: '群組', desc: '被加進群組時,只有被 @、被引用,或依機率擲骰才回覆;每個群組有自己的 session、工作資料夾、人設與記憶;比較晚加入時可以匯入 LINE 匯出的聊天記錄補記憶。' },
|
|
1374
1503
|
h('input', { ref: importInput, type: 'file', accept: '.txt,text/plain', style: { display: 'none' }, onChange: importChatFile }),
|
|
1375
1504
|
h('div', { style: S.grid },
|
|
1376
1505
|
h(Field, { label: '沒被點名時的回覆機率(%)', hint: '每則群組訊息用這個機率決定要不要回;0 就只在被 @ 或引用時回' },
|
package/lib/config.mjs
CHANGED
|
@@ -35,6 +35,19 @@ export const DEFAULT_CONFIG = {
|
|
|
35
35
|
errorReply: '(剛剛卡住了,再說一次好嗎?)',
|
|
36
36
|
emptyReply: '(她沒有說話)',
|
|
37
37
|
},
|
|
38
|
+
voice: {
|
|
39
|
+
enabled: false,
|
|
40
|
+
consent: false,
|
|
41
|
+
provider: 'auto', // auto | mlx | gguf
|
|
42
|
+
cloneMode: 'instruct', // instruct | zero-shot
|
|
43
|
+
instructText: '用可愛、輕快、帶點撒嬌的語氣說,語速稍慢。',
|
|
44
|
+
refText: '',
|
|
45
|
+
maxChars: 80,
|
|
46
|
+
model: '',
|
|
47
|
+
referencePath: '',
|
|
48
|
+
referenceSource: '',
|
|
49
|
+
skipDisclaimer: true,
|
|
50
|
+
},
|
|
38
51
|
server: {
|
|
39
52
|
host: '127.0.0.1',
|
|
40
53
|
port: 8791,
|
|
@@ -245,6 +258,19 @@ export function sanitizePatch(current, patch) {
|
|
|
245
258
|
next.agent.workspaceRoot = String(next.agent.workspaceRoot || '').trim()
|
|
246
259
|
next.agent.tiebaContext = Boolean(next.agent.tiebaContext)
|
|
247
260
|
|
|
261
|
+
const voice = next.voice || (next.voice = {})
|
|
262
|
+
voice.enabled = Boolean(voice.enabled)
|
|
263
|
+
voice.consent = Boolean(voice.consent)
|
|
264
|
+
voice.provider = ['auto', 'mlx', 'gguf'].includes(String(voice.provider)) ? String(voice.provider) : 'auto'
|
|
265
|
+
voice.cloneMode = String(voice.cloneMode) === 'zero-shot' ? 'zero-shot' : 'instruct'
|
|
266
|
+
voice.instructText = String(voice.instructText || '').slice(0, 500)
|
|
267
|
+
voice.refText = String(voice.refText || '').slice(0, 500)
|
|
268
|
+
voice.maxChars = clampInt(voice.maxChars, 20, 300, 80)
|
|
269
|
+
voice.model = String(voice.model || '').slice(0, 200)
|
|
270
|
+
voice.referencePath = String(voice.referencePath || '').trim()
|
|
271
|
+
voice.referenceSource = String(voice.referenceSource || '').trim()
|
|
272
|
+
voice.skipDisclaimer = Boolean(voice.skipDisclaimer)
|
|
273
|
+
|
|
248
274
|
const sync = next.sync
|
|
249
275
|
sync.enabled = Boolean(sync.enabled)
|
|
250
276
|
sync.targetPersonaId = String(sync.targetPersonaId || '').trim() || 'dafeiyu'
|
package/lib/host.mjs
CHANGED
|
@@ -29,6 +29,7 @@ const stickersMod = await import(withVersion('./stickers.mjs'))
|
|
|
29
29
|
const memeSourceMod = await import(withVersion('./meme-source.mjs'))
|
|
30
30
|
const stickerImageMod = await import(withVersion('./sticker-image.mjs'))
|
|
31
31
|
const visionMod = await import(withVersion('./vision.mjs'))
|
|
32
|
+
const ttsMod = await import(withVersion('./tts.mjs'))
|
|
32
33
|
const utilMod = await import(withVersion('./util.mjs'))
|
|
33
34
|
const { createConfigStore, publicConfig, dataDir, dshHome } = configMod
|
|
34
35
|
const { createPersonaClient } = personaMod
|
|
@@ -47,9 +48,10 @@ const { createStickerStore } = stickersMod
|
|
|
47
48
|
const { createMemeSource } = memeSourceMod
|
|
48
49
|
const { createStickerImages } = stickerImageMod
|
|
49
50
|
const { createVision } = visionMod
|
|
51
|
+
const { createTts } = ttsMod
|
|
50
52
|
const { chunkText, clip, inQuietWindow, jsonFail, jsonOk, maskSecret, readJsonBody, safeArray, sleep, readJsonSafe } = utilMod
|
|
51
53
|
|
|
52
|
-
export const VERSION = '0.
|
|
54
|
+
export const VERSION = '0.10.0'
|
|
53
55
|
|
|
54
56
|
// 環境變數提供的 Cloudflare API Token(方便用 shell 注入、不用把 token 寫進設定檔)。
|
|
55
57
|
// 設定檔裡的 token 優先;兩者都沒有時才用環境變數。
|
|
@@ -401,7 +403,77 @@ export async function create({ ctx, config: pluginConfig, logger: upstream }) {
|
|
|
401
403
|
const jokeText = jokes.contextText(() => store.get())
|
|
402
404
|
// 群組不注入貼吧作息,避免在群組裡突然講起刷貼吧的事。
|
|
403
405
|
const tieba = kind === 'user' ? tiebaContextText() : ''
|
|
404
|
-
return [modelContextText(), jokeText, tieba, balanceContextText(), extra, memoryContextText(ownerKey), memeContextText() || stickerContextText()].filter(Boolean).join('\n\n')
|
|
406
|
+
return [modelContextText(), voiceContextText(), jokeText, tieba, balanceContextText(), extra, memoryContextText(ownerKey), memeContextText() || stickerContextText()].filter(Boolean).join('\n\n')
|
|
407
|
+
}
|
|
408
|
+
|
|
409
|
+
const VOICE_MARKER = /\[\[\s*(?:voice|語音|语音|audio)\s*(?:[::]\s*([^\]\n]{0,200}))?\s*\]\]/gi
|
|
410
|
+
let voiceCache = { at: 0, value: null }
|
|
411
|
+
|
|
412
|
+
function voiceStatusCached() {
|
|
413
|
+
if (!voiceCache.value || Date.now() - voiceCache.at > 10000) {
|
|
414
|
+
try {
|
|
415
|
+
voiceCache = { at: Date.now(), value: tts.status() }
|
|
416
|
+
} catch (error) {
|
|
417
|
+
voiceCache = { at: Date.now(), value: { ready: false, lastError: errorText(error) } }
|
|
418
|
+
}
|
|
419
|
+
}
|
|
420
|
+
return voiceCache.value
|
|
421
|
+
}
|
|
422
|
+
|
|
423
|
+
function voiceReady(cfg = store.get()) {
|
|
424
|
+
if (cfg.voice?.enabled !== true || cfg.voice?.consent !== true) return false
|
|
425
|
+
const st = voiceStatusCached()
|
|
426
|
+
return st?.ready === true && Boolean(st?.reference?.path)
|
|
427
|
+
}
|
|
428
|
+
|
|
429
|
+
function voiceContextText() {
|
|
430
|
+
const cfg = store.get()
|
|
431
|
+
if (cfg.voice?.enabled !== true || cfg.voice?.consent !== true) return ''
|
|
432
|
+
if (!voiceReady(cfg)) return ''
|
|
433
|
+
const max = Math.max(20, Math.min(300, Number(cfg.voice?.maxChars) || 80))
|
|
434
|
+
return [
|
|
435
|
+
'【語音】你可以用語音回覆。',
|
|
436
|
+
`想用時把 [[voice]] 放在回覆最後;如果要指定語氣,寫成 [[voice:用溫柔、帶點撒嬌的語氣,語速慢一點]]。`,
|
|
437
|
+
`語氣、情緒、語速、停頓這些「怎麼說」的指示只能放在標記裡;正文只放要念出來的字,不要把語氣描述、括號動作或舞台指示寫進正文(最多約 ${max} 字)。`,
|
|
438
|
+
'一次最多一則語音;不適合用語音(例如內容太長、需要圖片)就不要加標記。',
|
|
439
|
+
].join('\n')
|
|
440
|
+
}
|
|
441
|
+
|
|
442
|
+
function splitVoiceMarker(text) {
|
|
443
|
+
let requested = false
|
|
444
|
+
let instruct = ''
|
|
445
|
+
const cleaned = String(text || '')
|
|
446
|
+
.replace(VOICE_MARKER, (_match, tone) => {
|
|
447
|
+
requested = true
|
|
448
|
+
if (tone && String(tone).trim()) instruct = String(tone).trim().slice(0, 200)
|
|
449
|
+
return ''
|
|
450
|
+
})
|
|
451
|
+
.trim()
|
|
452
|
+
return { text: cleaned, requested, instruct }
|
|
453
|
+
}
|
|
454
|
+
|
|
455
|
+
async function synthesizeVoiceMessage(text, instruct = '') {
|
|
456
|
+
const base = publicBaseSync()
|
|
457
|
+
if (!base) throw new Error('目前沒有公開網址,語音訊息送不出去')
|
|
458
|
+
const result = await tts.synthesize(text, { instructText: instruct })
|
|
459
|
+
const token = registerFile(result.m4a, 'audio/mp4')
|
|
460
|
+
if (!token) throw new Error('無法把語音檔存進媒體庫')
|
|
461
|
+
return {
|
|
462
|
+
message: { type: 'audio', originalContentUrl: `${base}/media/${token}`, duration: Math.max(1000, Math.round(result.durationMs || 0)) },
|
|
463
|
+
token,
|
|
464
|
+
provider: result.provider,
|
|
465
|
+
durationMs: result.durationMs,
|
|
466
|
+
}
|
|
467
|
+
}
|
|
468
|
+
|
|
469
|
+
async function replyMessages(userId, replyToken, messages) {
|
|
470
|
+
if (!Array.isArray(messages) || messages.length === 0) return
|
|
471
|
+
try {
|
|
472
|
+
await withToken((token) => lineApi.replyMessage(token, replyToken, messages))
|
|
473
|
+
} catch (error) {
|
|
474
|
+
logger.warn(`reply 失敗,改用 push:${errorText(error)}`)
|
|
475
|
+
await withToken((token) => lineApi.pushMessage(token, userId, messages))
|
|
476
|
+
}
|
|
405
477
|
}
|
|
406
478
|
|
|
407
479
|
async function refreshBalance() {
|
|
@@ -1426,6 +1498,25 @@ export async function create({ ctx, config: pluginConfig, logger: upstream }) {
|
|
|
1426
1498
|
return token
|
|
1427
1499
|
}
|
|
1428
1500
|
|
|
1501
|
+
function registerFile(filePath, mime, ext = '') {
|
|
1502
|
+
if (!filePath || !fs.existsSync(filePath)) return ''
|
|
1503
|
+
const token = `m${randomBytes(6).toString('hex')}`
|
|
1504
|
+
const suffix = ext || path.extname(String(filePath || '')).toLowerCase() || '.bin'
|
|
1505
|
+
const filename = `${token}${suffix}`
|
|
1506
|
+
const target = path.join(mediaDir, filename)
|
|
1507
|
+
try {
|
|
1508
|
+
fs.copyFileSync(filePath, target)
|
|
1509
|
+
} catch (error) {
|
|
1510
|
+
logger.warn(`複製媒體失敗:${error?.message || error}`)
|
|
1511
|
+
return ''
|
|
1512
|
+
}
|
|
1513
|
+
const item = { file: filename, mime: mime || 'application/octet-stream', at: Date.now(), bytes: fs.statSync(target).size }
|
|
1514
|
+
mediaTokens.set(token, item)
|
|
1515
|
+
mediaIndex[token] = item
|
|
1516
|
+
saveMediaIndex()
|
|
1517
|
+
return token
|
|
1518
|
+
}
|
|
1519
|
+
|
|
1429
1520
|
function getMedia(id) {
|
|
1430
1521
|
const key = String(id || '')
|
|
1431
1522
|
if (!key) return null
|
|
@@ -1804,10 +1895,28 @@ export async function create({ ctx, config: pluginConfig, logger: upstream }) {
|
|
|
1804
1895
|
cwd: ensureUserWorkspace(conversationKey) || cfg.agent.cwd || undefined,
|
|
1805
1896
|
})
|
|
1806
1897
|
let replyText = String(result?.text || '').trim()
|
|
1898
|
+
const voiceDirective = splitVoiceMarker(replyText)
|
|
1899
|
+
replyText = voiceDirective.text
|
|
1807
1900
|
const parsedReply = splitStickerMarkers(replyText)
|
|
1808
|
-
|
|
1809
|
-
|
|
1810
|
-
|
|
1901
|
+
let voiceSent = null
|
|
1902
|
+
if (voiceDirective.requested && voiceReady(cfg)) {
|
|
1903
|
+
const spoken = parsedReply.text || ''
|
|
1904
|
+
if (spoken) {
|
|
1905
|
+
try {
|
|
1906
|
+
voiceSent = await synthesizeVoiceMessage(spoken, voiceDirective.instruct)
|
|
1907
|
+
logger.info(`語音回覆:${spoken.length} 字,${Math.round((voiceSent.durationMs || 0) / 1000)} 秒(${voiceSent.provider};語氣:${voiceDirective.instruct || '預設'})`)
|
|
1908
|
+
} catch (error) {
|
|
1909
|
+
logger.warn(`語音合成失敗,改用文字:${errorText(error)}`)
|
|
1910
|
+
}
|
|
1911
|
+
}
|
|
1912
|
+
}
|
|
1913
|
+
if (voiceSent) {
|
|
1914
|
+
await replyMessages(chatId, event.replyToken, [voiceSent.message])
|
|
1915
|
+
} else {
|
|
1916
|
+
if (!parsedReply.text && parsedReply.names.length === 0) replyText = cfg.line.emptyReply || '(她沒有說話)'
|
|
1917
|
+
await replyOrPush(chatId, event.replyToken, replyText)
|
|
1918
|
+
}
|
|
1919
|
+
runtime.lastReply = { chatId, kind: sourceKind, at: Date.now(), text: clip(voiceSent ? `[語音] ${parsedReply.text || replyText}` : parsedReply.text || (parsedReply.names.length ? `[表情包] ${parsedReply.names.join('、')}` : replyText), 300) }
|
|
1811
1920
|
store.updateState((state) => {
|
|
1812
1921
|
const map = state[bucketName] || {}
|
|
1813
1922
|
const target = map[conversationKey]
|
|
@@ -1834,6 +1943,7 @@ export async function create({ ctx, config: pluginConfig, logger: upstream }) {
|
|
|
1834
1943
|
const memes = createMemeSource({ ctx, logger })
|
|
1835
1944
|
const stickerImages = createStickerImages({ ctx, logger, dir })
|
|
1836
1945
|
const vision = createVision({ logger, cacheDir: path.join(dir, 'cache') })
|
|
1946
|
+
const tts = createTts({ dir, logger, getConfig: () => store.get() })
|
|
1837
1947
|
const mediaTokens = new Map()
|
|
1838
1948
|
let imageSupportCache = { at: 0, key: '', value: false }
|
|
1839
1949
|
|
|
@@ -1949,7 +2059,9 @@ export async function create({ ctx, config: pluginConfig, logger: upstream }) {
|
|
|
1949
2059
|
})
|
|
1950
2060
|
|
|
1951
2061
|
async function sendPush(userId, text) {
|
|
1952
|
-
const
|
|
2062
|
+
const voiceDirective = splitVoiceMarker(text)
|
|
2063
|
+
if (voiceDirective.requested) logger.info('主動訊息中的語音標記已忽略,改用文字')
|
|
2064
|
+
const { messages } = await buildOutgoingMessages(voiceDirective.text)
|
|
1953
2065
|
if (messages.length === 0) return { sent: false, reason: 'empty' }
|
|
1954
2066
|
const imageCount = messages.filter((item) => typeof item !== 'string' && item.type === 'image').length
|
|
1955
2067
|
if (imageCount > 0) logger.info(`主動訊息含 ${imageCount} 張圖片`)
|
|
@@ -2388,6 +2500,18 @@ export async function create({ ctx, config: pluginConfig, logger: upstream }) {
|
|
|
2388
2500
|
imageVision: cfg.line?.imageVision !== false,
|
|
2389
2501
|
stickerVision: cfg.line?.stickerVision !== false,
|
|
2390
2502
|
},
|
|
2503
|
+
voice: {
|
|
2504
|
+
...voiceStatusCached(),
|
|
2505
|
+
enabled: cfg.voice?.enabled === true,
|
|
2506
|
+
consent: cfg.voice?.consent === true,
|
|
2507
|
+
provider: cfg.voice?.provider || 'auto',
|
|
2508
|
+
cloneMode: cfg.voice?.cloneMode || 'instruct',
|
|
2509
|
+
instructText: cfg.voice?.instructText || '',
|
|
2510
|
+
maxChars: Number(cfg.voice?.maxChars) || 80,
|
|
2511
|
+
referencePath: cfg.voice?.referencePath || '',
|
|
2512
|
+
ready: voiceReady(cfg),
|
|
2513
|
+
publicBase: publicBaseSync(),
|
|
2514
|
+
},
|
|
2391
2515
|
memory: {
|
|
2392
2516
|
...memory.stats(),
|
|
2393
2517
|
enabled: cfg.memory?.enabled !== false,
|
|
@@ -2676,6 +2800,53 @@ export async function create({ ctx, config: pluginConfig, logger: upstream }) {
|
|
|
2676
2800
|
const built = await buildOutgoingMessages(String(payload.text))
|
|
2677
2801
|
return { messages: built.messages, parsed: built.parsed }
|
|
2678
2802
|
}
|
|
2803
|
+
case 'voice-detect':
|
|
2804
|
+
return tts.detect()
|
|
2805
|
+
case 'voice-install': {
|
|
2806
|
+
if (tts.status().installing) return { started: false, already: true }
|
|
2807
|
+
tts.install()
|
|
2808
|
+
.then((result) => {
|
|
2809
|
+
if (result?.provider) store.patch({ voice: { provider: result.provider } }, { silent: true })
|
|
2810
|
+
logger.info(`語音環境安裝完成(${result?.provider || 'unknown'})`)
|
|
2811
|
+
voiceCache = { at: 0, value: null }
|
|
2812
|
+
})
|
|
2813
|
+
.catch((error) => logger.error(`語音環境安裝失敗:${errorText(error)}`))
|
|
2814
|
+
return { started: true }
|
|
2815
|
+
}
|
|
2816
|
+
case 'voice-preview':
|
|
2817
|
+
case 'voice-test': {
|
|
2818
|
+
const text = String(payload.text || '主人,這是語音測試,聽得到嗎?').trim()
|
|
2819
|
+
if (!text) throw new Error('缺少文字')
|
|
2820
|
+
const result = await tts.synthesize(text)
|
|
2821
|
+
const token = registerFile(result.m4a, 'audio/mp4')
|
|
2822
|
+
const base = publicBaseSync()
|
|
2823
|
+
voiceCache = { at: 0, value: null }
|
|
2824
|
+
return {
|
|
2825
|
+
provider: result.provider,
|
|
2826
|
+
durationMs: result.durationMs,
|
|
2827
|
+
bytes: result.bytes,
|
|
2828
|
+
chars: result.chars,
|
|
2829
|
+
processingMs: result.processingMs,
|
|
2830
|
+
url: base ? `${base}/media/${token}` : '',
|
|
2831
|
+
localUrl: `/api/line-bot/media/${token}`,
|
|
2832
|
+
token,
|
|
2833
|
+
}
|
|
2834
|
+
}
|
|
2835
|
+
case 'voice-save-reference': {
|
|
2836
|
+
const dataUrl = String(payload.dataUrl || '')
|
|
2837
|
+
const match = /^data:([^;]+);base64,([\s\S]+)$/.exec(dataUrl)
|
|
2838
|
+
if (!match) throw new Error('缺少音檔資料')
|
|
2839
|
+
const buffer = Buffer.from(match[2], 'base64')
|
|
2840
|
+
if (buffer.length > 20 * 1024 * 1024) throw new Error('音檔超過 20MB')
|
|
2841
|
+
const saved = await tts.saveReferenceBuffer(buffer, String(payload.filename || 'reference'), String(payload.ext || '.wav'))
|
|
2842
|
+
store.patch({ voice: { referencePath: saved.path } }, { silent: true })
|
|
2843
|
+
voiceCache = { at: 0, value: null }
|
|
2844
|
+
return { path: saved.path, seconds: saved.seconds }
|
|
2845
|
+
}
|
|
2846
|
+
case 'voice-reset-reference':
|
|
2847
|
+
store.patch({ voice: { referencePath: '' } }, { silent: true })
|
|
2848
|
+
voiceCache = { at: 0, value: null }
|
|
2849
|
+
return { path: tts.referencePath() }
|
|
2679
2850
|
case 'meme-send': {
|
|
2680
2851
|
const userId = String(payload.userId || '').trim()
|
|
2681
2852
|
if (!userId) throw new Error('缺少 userId')
|
package/lib/tts.mjs
ADDED
|
@@ -0,0 +1,540 @@
|
|
|
1
|
+
import fs from 'node:fs'
|
|
2
|
+
import os from 'node:os'
|
|
3
|
+
import path from 'node:path'
|
|
4
|
+
import { execFile, execFileSync } from 'node:child_process'
|
|
5
|
+
import { promisify } from 'node:util'
|
|
6
|
+
import { randomUUID } from 'node:crypto'
|
|
7
|
+
import { fileURLToPath } from 'node:url'
|
|
8
|
+
|
|
9
|
+
const execFileAsync = promisify(execFile)
|
|
10
|
+
|
|
11
|
+
export const MLX_MODEL = 'mlx-community/Fun-CosyVoice3-0.5B-2512-8bit'
|
|
12
|
+
export const GGUF_BACKEND = 'cosyvoice3-tts'
|
|
13
|
+
export const CRISPASR_VERSION = 'v0.8.49'
|
|
14
|
+
export const DEFAULT_INSTRUCT = '用可愛、輕快、帶點撒嬌的語氣說,語速稍慢。'
|
|
15
|
+
|
|
16
|
+
const CRISPASR_MAC_URL = `https://github.com/CrispStrobe/CrispASR/releases/download/${CRISPASR_VERSION}/crispasr-macos-arm64.tar.gz`
|
|
17
|
+
const CRISPASR_WIN_URL = `https://github.com/CrispStrobe/CrispASR/releases/download/${CRISPASR_VERSION}/crispasr-windows-x86_64-cpu.zip`
|
|
18
|
+
|
|
19
|
+
function isWin() {
|
|
20
|
+
return process.platform === 'win32'
|
|
21
|
+
}
|
|
22
|
+
|
|
23
|
+
function which(command) {
|
|
24
|
+
try {
|
|
25
|
+
const out = execFileSync(isWin() ? 'where' : 'which', [command], { encoding: 'utf8', timeout: 4000 })
|
|
26
|
+
return String(out).split(/\r?\n/).map((line) => line.trim()).filter(Boolean)[0] || ''
|
|
27
|
+
} catch {
|
|
28
|
+
return ''
|
|
29
|
+
}
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
function fileExists(file) {
|
|
33
|
+
try {
|
|
34
|
+
return fs.existsSync(file)
|
|
35
|
+
} catch {
|
|
36
|
+
return false
|
|
37
|
+
}
|
|
38
|
+
}
|
|
39
|
+
|
|
40
|
+
function dirExists(dir) {
|
|
41
|
+
try {
|
|
42
|
+
return fs.statSync(dir).isDirectory()
|
|
43
|
+
} catch {
|
|
44
|
+
return false
|
|
45
|
+
}
|
|
46
|
+
}
|
|
47
|
+
|
|
48
|
+
function pythonVersion(python) {
|
|
49
|
+
try {
|
|
50
|
+
const out = execFileSync(python, ['-c', 'import sys; print("%d.%d" % sys.version_info[:2])'], {
|
|
51
|
+
encoding: 'utf8',
|
|
52
|
+
timeout: 6000,
|
|
53
|
+
env: process.env,
|
|
54
|
+
}).trim()
|
|
55
|
+
const [major, minor] = out.split('.').map((value) => Number(value))
|
|
56
|
+
return { text: out, major, minor }
|
|
57
|
+
} catch {
|
|
58
|
+
return { text: '', major: 0, minor: 0 }
|
|
59
|
+
}
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
function parseDurationSeconds(text) {
|
|
63
|
+
const match = /Duration:\s*(\d+):(\d+):(\d+(?:\.\d+)?)/.exec(String(text || ''))
|
|
64
|
+
if (!match) return 0
|
|
65
|
+
return Number(match[1]) * 3600 + Number(match[2]) * 60 + Number(match[3])
|
|
66
|
+
}
|
|
67
|
+
|
|
68
|
+
export function createTts({ dir, logger, getConfig }) {
|
|
69
|
+
const root = path.join(dir, 'tts')
|
|
70
|
+
const paths = {
|
|
71
|
+
root,
|
|
72
|
+
refDir: path.join(root, 'ref'),
|
|
73
|
+
outDir: path.join(root, 'out'),
|
|
74
|
+
jobsDir: path.join(root, 'jobs'),
|
|
75
|
+
hfHome: path.join(root, 'hf'),
|
|
76
|
+
mlxVenv: path.join(root, 'mlx312-venv'),
|
|
77
|
+
mlxPython: path.join(root, 'mlx312-venv', 'bin', 'python'),
|
|
78
|
+
mlxScript: path.join(path.dirname(fileURLToPath(import.meta.url)), '..', 'tools', 'mlx_synth.py'),
|
|
79
|
+
macCrisp: path.join(root, 'crispasr', 'crispasr-macos-arm64', 'crispasr'),
|
|
80
|
+
winCrisp: path.join(root, 'crispasr', 'windows', 'crispasr.exe'),
|
|
81
|
+
winDir: path.join(root, 'crispasr', 'windows'),
|
|
82
|
+
defaultRef: path.join(root, 'ref', 'default.wav'),
|
|
83
|
+
}
|
|
84
|
+
for (const item of [paths.root, paths.refDir, paths.outDir, paths.jobsDir, paths.hfHome]) {
|
|
85
|
+
try {
|
|
86
|
+
fs.mkdirSync(item, { recursive: true })
|
|
87
|
+
} catch {
|
|
88
|
+
/* 權限不足時留給 status 回報 */
|
|
89
|
+
}
|
|
90
|
+
}
|
|
91
|
+
|
|
92
|
+
const state = {
|
|
93
|
+
setup: { phase: 'idle', message: '', error: '', at: 0, startedAt: 0 },
|
|
94
|
+
last: null,
|
|
95
|
+
lastError: '',
|
|
96
|
+
installing: false,
|
|
97
|
+
}
|
|
98
|
+
|
|
99
|
+
function setSetup(phase, message, error = '') {
|
|
100
|
+
state.setup = {
|
|
101
|
+
phase,
|
|
102
|
+
message,
|
|
103
|
+
error: error ? String(error).slice(0, 800) : '',
|
|
104
|
+
at: Date.now(),
|
|
105
|
+
startedAt: state.setup.startedAt || Date.now(),
|
|
106
|
+
}
|
|
107
|
+
if (logger) {
|
|
108
|
+
if (error) logger.warn(`語音環境:${message}(${String(error).slice(0, 200)})`)
|
|
109
|
+
else logger.info(`語音環境:${message}`)
|
|
110
|
+
}
|
|
111
|
+
}
|
|
112
|
+
|
|
113
|
+
function cfgVoice() {
|
|
114
|
+
const cfg = getConfig ? getConfig() : {}
|
|
115
|
+
return cfg.voice || {}
|
|
116
|
+
}
|
|
117
|
+
|
|
118
|
+
function referencePath() {
|
|
119
|
+
const cfg = cfgVoice()
|
|
120
|
+
const configured = String(cfg.referencePath || '').trim()
|
|
121
|
+
if (configured && fileExists(configured)) return configured
|
|
122
|
+
if (fileExists(paths.defaultRef)) return paths.defaultRef
|
|
123
|
+
return ''
|
|
124
|
+
}
|
|
125
|
+
|
|
126
|
+
function referenceInfo() {
|
|
127
|
+
const file = referencePath()
|
|
128
|
+
if (!file) return { path: '', exists: false, seconds: 0 }
|
|
129
|
+
let seconds = 0
|
|
130
|
+
try {
|
|
131
|
+
const out = execFileSync('ffprobe', ['-v', 'error', '-show_entries', 'format=duration', '-of', 'default=nw=1:nk=1', file], {
|
|
132
|
+
encoding: 'utf8',
|
|
133
|
+
timeout: 8000,
|
|
134
|
+
})
|
|
135
|
+
seconds = Number(out.trim()) || 0
|
|
136
|
+
} catch {
|
|
137
|
+
seconds = 0
|
|
138
|
+
}
|
|
139
|
+
return { path: file, exists: true, seconds: Math.round(seconds * 100) / 100 }
|
|
140
|
+
}
|
|
141
|
+
|
|
142
|
+
function mlxModelCached() {
|
|
143
|
+
const hub = path.join(paths.hfHome, 'hub')
|
|
144
|
+
try {
|
|
145
|
+
const dirs = fs.readdirSync(hub).filter((name) => name.startsWith('models--mlx-community--Fun-CosyVoice3'))
|
|
146
|
+
for (const name of dirs) {
|
|
147
|
+
const snapshots = path.join(hub, name, 'snapshots')
|
|
148
|
+
if (!dirExists(snapshots)) continue
|
|
149
|
+
for (const snap of fs.readdirSync(snapshots)) {
|
|
150
|
+
if (fileExists(path.join(snapshots, snap, 'model.safetensors'))) return true
|
|
151
|
+
}
|
|
152
|
+
}
|
|
153
|
+
} catch {
|
|
154
|
+
/* ignore */
|
|
155
|
+
}
|
|
156
|
+
return false
|
|
157
|
+
}
|
|
158
|
+
|
|
159
|
+
let mlxError = ''
|
|
160
|
+
function mlxReady() {
|
|
161
|
+
if (!fileExists(paths.mlxPython)) {
|
|
162
|
+
mlxError = '找不到 MLX Python 環境'
|
|
163
|
+
return false
|
|
164
|
+
}
|
|
165
|
+
if (!fileExists(paths.mlxScript)) {
|
|
166
|
+
mlxError = `找不到 MLX 合成腳本:${paths.mlxScript}`
|
|
167
|
+
return false
|
|
168
|
+
}
|
|
169
|
+
try {
|
|
170
|
+
execFileSync(paths.mlxPython, ['-c', 'import mlx_audio, mlx_lm, librosa'], { stdio: 'ignore', timeout: 30000 })
|
|
171
|
+
mlxError = ''
|
|
172
|
+
return true
|
|
173
|
+
} catch (error) {
|
|
174
|
+
mlxError = String(error?.stderr || error?.message || error).slice(0, 300)
|
|
175
|
+
return false
|
|
176
|
+
}
|
|
177
|
+
}
|
|
178
|
+
|
|
179
|
+
function ggufBinary() {
|
|
180
|
+
const candidates = [isWin() ? paths.winCrisp : paths.macCrisp, which('crispasr')]
|
|
181
|
+
return candidates.find((item) => item && fileExists(item)) || ''
|
|
182
|
+
}
|
|
183
|
+
|
|
184
|
+
function ggufReady() {
|
|
185
|
+
const bin = ggufBinary()
|
|
186
|
+
if (!bin) return false
|
|
187
|
+
try {
|
|
188
|
+
execFileSync(bin, ['--version'], { stdio: 'ignore', timeout: 8000 })
|
|
189
|
+
return true
|
|
190
|
+
} catch {
|
|
191
|
+
return false
|
|
192
|
+
}
|
|
193
|
+
}
|
|
194
|
+
|
|
195
|
+
function detect() {
|
|
196
|
+
const platform = process.platform
|
|
197
|
+
const arch = process.arch
|
|
198
|
+
const preferred = platform === 'darwin' && arch === 'arm64' ? 'mlx' : 'gguf'
|
|
199
|
+
const ffmpeg = which('ffmpeg')
|
|
200
|
+
const ref = referenceInfo()
|
|
201
|
+
return {
|
|
202
|
+
platform,
|
|
203
|
+
arch,
|
|
204
|
+
preferred,
|
|
205
|
+
ffmpeg: { path: ffmpeg, ok: !!ffmpeg },
|
|
206
|
+
mlx: {
|
|
207
|
+
ok: mlxReady(),
|
|
208
|
+
python: fileExists(paths.mlxPython) ? paths.mlxPython : '',
|
|
209
|
+
model: MLX_MODEL,
|
|
210
|
+
modelCached: mlxModelCached(),
|
|
211
|
+
error: mlxError,
|
|
212
|
+
script: paths.mlxScript,
|
|
213
|
+
},
|
|
214
|
+
gguf: {
|
|
215
|
+
ok: ggufReady(),
|
|
216
|
+
binary: ggufBinary(),
|
|
217
|
+
backend: GGUF_BACKEND,
|
|
218
|
+
},
|
|
219
|
+
reference: ref,
|
|
220
|
+
setup: state.setup,
|
|
221
|
+
ready: (preferred === 'mlx' ? mlxReady() : ggufReady()) && !!ref.path,
|
|
222
|
+
}
|
|
223
|
+
}
|
|
224
|
+
|
|
225
|
+
function resolveProvider(cfg = cfgVoice()) {
|
|
226
|
+
const detected = detect()
|
|
227
|
+
const wanted = String(cfg.provider || 'auto')
|
|
228
|
+
if (wanted === 'mlx') {
|
|
229
|
+
if (!detected.mlx.ok) throw new Error('MLX 語音環境還沒安裝完成,請先按「一鍵配置環境」')
|
|
230
|
+
return 'mlx'
|
|
231
|
+
}
|
|
232
|
+
if (wanted === 'gguf') {
|
|
233
|
+
if (!detected.gguf.ok) throw new Error('CosyVoice GGUF 引擎還沒安裝完成,請先按「一鍵配置環境」')
|
|
234
|
+
return 'gguf'
|
|
235
|
+
}
|
|
236
|
+
if (detected.mlx.ok && (detected.platform === 'darwin' || detected.preferred === 'mlx')) return 'mlx'
|
|
237
|
+
if (detected.gguf.ok) return 'gguf'
|
|
238
|
+
if (detected.mlx.ok) return 'mlx'
|
|
239
|
+
throw new Error('還沒有可用的語音引擎,請先按「一鍵配置環境」')
|
|
240
|
+
}
|
|
241
|
+
|
|
242
|
+
async function run(command, args, { env = process.env, timeout = 180000, maxBuffer = 20 * 1024 * 1024 } = {}) {
|
|
243
|
+
return execFileAsync(command, args, { env, timeout, maxBuffer })
|
|
244
|
+
}
|
|
245
|
+
|
|
246
|
+
function findPython() {
|
|
247
|
+
const candidates = [
|
|
248
|
+
process.env.LINEBOT_MLX_PYTHON,
|
|
249
|
+
'/opt/homebrew/opt/python@3.12/bin/python3.12',
|
|
250
|
+
'/opt/homebrew/bin/python3.12',
|
|
251
|
+
'/usr/local/bin/python3.12',
|
|
252
|
+
'/opt/homebrew/bin/python3.11',
|
|
253
|
+
'/usr/local/bin/python3.11',
|
|
254
|
+
].filter(Boolean)
|
|
255
|
+
for (const candidate of candidates) {
|
|
256
|
+
if (!fileExists(candidate)) continue
|
|
257
|
+
const version = pythonVersion(candidate)
|
|
258
|
+
if (version.major > 3 || (version.major === 3 && version.minor >= 10)) return candidate
|
|
259
|
+
}
|
|
260
|
+
const system = which('python3')
|
|
261
|
+
if (system && !system.includes('DeepSeek Harness.app')) {
|
|
262
|
+
const version = pythonVersion(system)
|
|
263
|
+
if (version.major > 3 || (version.major === 3 && version.minor >= 10)) return system
|
|
264
|
+
}
|
|
265
|
+
return ''
|
|
266
|
+
}
|
|
267
|
+
|
|
268
|
+
function uvCommand() {
|
|
269
|
+
const candidates = [path.join(os.homedir(), '.local', 'bin', 'uv'), '/opt/homebrew/bin/uv', which('uv')].filter(Boolean)
|
|
270
|
+
return candidates.find((item) => fileExists(item)) || ''
|
|
271
|
+
}
|
|
272
|
+
|
|
273
|
+
async function installMlx() {
|
|
274
|
+
setSetup('python', '檢查 Python(MLX 需要 3.10 以上)')
|
|
275
|
+
let python = findPython()
|
|
276
|
+
if (!python) {
|
|
277
|
+
const brew = which('brew')
|
|
278
|
+
if (!brew) throw new Error('找不到 Python 3.10 以上,也找不到 Homebrew。請先安裝 Python 3.12。')
|
|
279
|
+
setSetup('python', '用 Homebrew 安裝 Python 3.12(第一次會比較久)')
|
|
280
|
+
await run(brew, ['install', 'python@3.12'], { timeout: 1200000 })
|
|
281
|
+
python = findPython()
|
|
282
|
+
}
|
|
283
|
+
if (!python) throw new Error('找不到可用的 Python 3.10 以上')
|
|
284
|
+
|
|
285
|
+
if (!fileExists(paths.mlxPython)) {
|
|
286
|
+
setSetup('venv', '建立 MLX 專用 Python 環境')
|
|
287
|
+
await run(python, ['-m', 'venv', paths.mlxVenv], { timeout: 120000 })
|
|
288
|
+
} else {
|
|
289
|
+
setSetup('venv', 'MLX Python 環境已存在')
|
|
290
|
+
}
|
|
291
|
+
|
|
292
|
+
const uv = uvCommand()
|
|
293
|
+
const pipBase = uv ? [uv, 'pip', 'install', '--python', paths.mlxPython] : [paths.mlxPython, '-m', 'pip', 'install']
|
|
294
|
+
setSetup('pip', '安裝 MLX 語音套件(mlx-audio-plus)')
|
|
295
|
+
const packages = ['mlx-audio-plus==0.1.7', 'mlx-lm==0.29.1', 'librosa', 'pyloudnorm', 'soundfile']
|
|
296
|
+
await run(pipBase[0], [...pipBase.slice(1), ...packages], { timeout: 1800000, env: { ...process.env, HF_HOME: paths.hfHome } })
|
|
297
|
+
|
|
298
|
+
if (!mlxModelCached()) {
|
|
299
|
+
setSetup('model', '下載 CosyVoice3-0.5B 模型(約 1.4GB,第一次會比較久)')
|
|
300
|
+
const downloader = [
|
|
301
|
+
'from huggingface_hub import snapshot_download',
|
|
302
|
+
`snapshot_download(${JSON.stringify(MLX_MODEL)}, cache_dir=${JSON.stringify(path.join(paths.hfHome, 'hub'))})`,
|
|
303
|
+
].join('\n')
|
|
304
|
+
await run(paths.mlxPython, ['-c', downloader], {
|
|
305
|
+
timeout: 3600000,
|
|
306
|
+
env: { ...process.env, HF_HOME: paths.hfHome },
|
|
307
|
+
})
|
|
308
|
+
} else {
|
|
309
|
+
setSetup('model', 'MLX 模型已下載')
|
|
310
|
+
}
|
|
311
|
+
if (!mlxReady()) throw new Error('MLX 套件安裝後仍無法載入,請查看日誌')
|
|
312
|
+
return 'mlx'
|
|
313
|
+
}
|
|
314
|
+
|
|
315
|
+
async function downloadFile(url, target) {
|
|
316
|
+
const res = await fetch(url, { redirect: 'follow' })
|
|
317
|
+
if (!res.ok) throw new Error(`下載失敗(HTTP ${res.status}):${url}`)
|
|
318
|
+
const bytes = Buffer.from(await res.arrayBuffer())
|
|
319
|
+
fs.writeFileSync(target, bytes)
|
|
320
|
+
}
|
|
321
|
+
|
|
322
|
+
async function installMacGguf() {
|
|
323
|
+
setSetup('engine', '下載 CosyVoice GGUF 引擎(macOS 備援)')
|
|
324
|
+
const target = path.join(root, 'crispasr')
|
|
325
|
+
fs.mkdirSync(target, { recursive: true })
|
|
326
|
+
const tgz = path.join(root, 'crispasr-macos-arm64.tar.gz')
|
|
327
|
+
await downloadFile(CRISPASR_MAC_URL, tgz)
|
|
328
|
+
await run('tar', ['-xzf', tgz, '-C', target], { timeout: 180000 })
|
|
329
|
+
if (!fileExists(paths.macCrisp)) throw new Error('解壓縮後找不到 crispasr 執行檔')
|
|
330
|
+
fs.chmodSync(paths.macCrisp, 0o755)
|
|
331
|
+
return 'gguf'
|
|
332
|
+
}
|
|
333
|
+
|
|
334
|
+
async function installWinGguf() {
|
|
335
|
+
setSetup('engine', '下載 CosyVoice GGUF 引擎(Windows)')
|
|
336
|
+
fs.mkdirSync(paths.winDir, { recursive: true })
|
|
337
|
+
const zip = path.join(root, 'crispasr-windows.zip')
|
|
338
|
+
await downloadFile(CRISPASR_WIN_URL, zip)
|
|
339
|
+
await run('powershell', ['-NoProfile', '-Command', `Expand-Archive -LiteralPath ${JSON.stringify(zip)} -DestinationPath ${JSON.stringify(paths.winDir)} -Force`], { timeout: 300000 })
|
|
340
|
+
const found = []
|
|
341
|
+
for (const name of fs.readdirSync(paths.winDir)) {
|
|
342
|
+
const full = path.join(paths.winDir, name)
|
|
343
|
+
if (/crispasr(\.exe)?$/i.test(name)) found.push(full)
|
|
344
|
+
if (dirExists(full)) {
|
|
345
|
+
for (const sub of fs.readdirSync(full)) {
|
|
346
|
+
if (/^crispasr(\.exe)?$/i.test(sub)) found.push(path.join(full, sub))
|
|
347
|
+
}
|
|
348
|
+
}
|
|
349
|
+
}
|
|
350
|
+
if (!found[0]) throw new Error('解壓縮後找不到 crispasr.exe')
|
|
351
|
+
if (found[0] !== paths.winCrisp) fs.copyFileSync(found[0], paths.winCrisp)
|
|
352
|
+
return 'gguf'
|
|
353
|
+
}
|
|
354
|
+
|
|
355
|
+
async function ensureDefaultReference() {
|
|
356
|
+
setSetup('reference', '準備預設參考音')
|
|
357
|
+
if (fileExists(paths.defaultRef)) return paths.defaultRef
|
|
358
|
+
const cfg = cfgVoice()
|
|
359
|
+
let source = String(cfg.referenceSource || '').trim()
|
|
360
|
+
if (!source || !fileExists(source)) {
|
|
361
|
+
const downloads = path.join(os.homedir(), 'Downloads')
|
|
362
|
+
try {
|
|
363
|
+
const hit = fs.readdirSync(downloads).find((name) => name.includes('本间心铃'))
|
|
364
|
+
if (hit) source = path.join(downloads, hit)
|
|
365
|
+
} catch {
|
|
366
|
+
/* no Downloads */
|
|
367
|
+
}
|
|
368
|
+
}
|
|
369
|
+
if (!source || !fileExists(source)) {
|
|
370
|
+
if (fileExists(paths.defaultRef)) return paths.defaultRef
|
|
371
|
+
throw new Error('找不到預設參考音,請在面板上傳一段聲音檔')
|
|
372
|
+
}
|
|
373
|
+
const ffmpeg = which('ffmpeg')
|
|
374
|
+
if (!ffmpeg) throw new Error('需要 ffmpeg 才能處理參考音,請先安裝 ffmpeg')
|
|
375
|
+
await run(ffmpeg, ['-hide_banner', '-loglevel', 'error', '-y', '-ss', '12', '-t', '4.2', '-i', source, '-ac', '1', '-ar', '16000', '-c:a', 'pcm_s16le', paths.defaultRef], { timeout: 120000 })
|
|
376
|
+
return paths.defaultRef
|
|
377
|
+
}
|
|
378
|
+
|
|
379
|
+
async function install() {
|
|
380
|
+
if (state.installing) return { started: false, already: true }
|
|
381
|
+
state.installing = true
|
|
382
|
+
state.setup.startedAt = Date.now()
|
|
383
|
+
setSetup('detect', '偵測系統環境')
|
|
384
|
+
try {
|
|
385
|
+
const detected = detect()
|
|
386
|
+
let provider = ''
|
|
387
|
+
if (detected.platform === 'darwin' && detected.arch === 'arm64') {
|
|
388
|
+
try {
|
|
389
|
+
provider = await installMlx()
|
|
390
|
+
} catch (error) {
|
|
391
|
+
setSetup('fallback', 'MLX 安裝失敗,改用 GGUF 備援', error)
|
|
392
|
+
provider = await installMacGguf()
|
|
393
|
+
}
|
|
394
|
+
} else if (detected.platform === 'win32') {
|
|
395
|
+
provider = await installWinGguf()
|
|
396
|
+
} else {
|
|
397
|
+
provider = await installMacGguf()
|
|
398
|
+
}
|
|
399
|
+
await ensureDefaultReference()
|
|
400
|
+
const env = { ...process.env, HF_HOME: paths.hfHome }
|
|
401
|
+
setSetup('test', '測試語音合成')
|
|
402
|
+
await synthesize('主人,語音功能測試成功。', { forceProvider: provider, env, skipConsent: true })
|
|
403
|
+
setSetup('done', `環境已就緒(${provider === 'mlx' ? 'macOS MLX' : 'CosyVoice GGUF'})`)
|
|
404
|
+
return { started: true, provider }
|
|
405
|
+
} catch (error) {
|
|
406
|
+
setSetup('error', '環境安裝失敗', error)
|
|
407
|
+
throw error
|
|
408
|
+
} finally {
|
|
409
|
+
state.installing = false
|
|
410
|
+
}
|
|
411
|
+
}
|
|
412
|
+
|
|
413
|
+
function cleanText(text, maxChars) {
|
|
414
|
+
const value = String(text || '')
|
|
415
|
+
.replace(/\[\[\s*(?:voice|語音|语音)\s*\]\]/gi, '')
|
|
416
|
+
.replace(/[*_`#>]/g, '')
|
|
417
|
+
.replace(/\s+/g, ' ')
|
|
418
|
+
.trim()
|
|
419
|
+
return value.slice(0, Math.max(1, maxChars))
|
|
420
|
+
}
|
|
421
|
+
|
|
422
|
+
async function synthesize(text, { forceProvider = '', env = process.env, skipConsent = false, instructText: instructOverride = '' } = {}) {
|
|
423
|
+
const cfg = cfgVoice()
|
|
424
|
+
if (!skipConsent && cfg.consent !== true) throw new Error('尚未確認聲音使用權利(面板:語音回覆 → 我確認有使用權)')
|
|
425
|
+
if (!skipConsent && cfg.enabled === false) throw new Error('語音功能未開啟')
|
|
426
|
+
const provider = forceProvider || resolveProvider(cfg)
|
|
427
|
+
const ref = referencePath()
|
|
428
|
+
if (!ref) throw new Error('尚未設定參考音')
|
|
429
|
+
const maxChars = Math.max(20, Math.min(300, Number(cfg.maxChars) || 80))
|
|
430
|
+
const cleaned = cleanText(text, maxChars)
|
|
431
|
+
if (!cleaned) throw new Error('沒有可合成的文字')
|
|
432
|
+
const ffmpeg = which('ffmpeg')
|
|
433
|
+
if (!ffmpeg) throw new Error('找不到 ffmpeg')
|
|
434
|
+
const id = randomUUID().slice(0, 8)
|
|
435
|
+
const prefix = path.join(paths.outDir, `tts-${id}`)
|
|
436
|
+
const startedAt = Date.now()
|
|
437
|
+
const instructText = String(instructOverride || cfg.instructText || DEFAULT_INSTRUCT).trim()
|
|
438
|
+
const cloneMode = String(cfg.cloneMode || 'instruct')
|
|
439
|
+
let wav = ''
|
|
440
|
+
try {
|
|
441
|
+
if (provider === 'mlx') {
|
|
442
|
+
const jobFile = path.join(paths.jobsDir, `${id}.json`)
|
|
443
|
+
const payload = {
|
|
444
|
+
text: cleaned,
|
|
445
|
+
model: String(cfg.model || MLX_MODEL),
|
|
446
|
+
refAudio: ref,
|
|
447
|
+
prefix,
|
|
448
|
+
instructText: cloneMode === 'instruct' ? instructText : '',
|
|
449
|
+
refText: cloneMode === 'zero-shot' ? String(cfg.refText || '') : '',
|
|
450
|
+
}
|
|
451
|
+
fs.writeFileSync(jobFile, JSON.stringify(payload), { mode: 0o600 })
|
|
452
|
+
const { stdout } = await run(paths.mlxPython, [paths.mlxScript, jobFile], {
|
|
453
|
+
timeout: 600000,
|
|
454
|
+
env: { ...env, HF_HOME: paths.hfHome },
|
|
455
|
+
})
|
|
456
|
+
const line = String(stdout || '').split(/\r?\n/).reverse().find((item) => item.startsWith('RESULT:'))
|
|
457
|
+
if (!line) throw new Error(`MLX 沒有回傳結果:${String(stdout).slice(-300)}`)
|
|
458
|
+
const result = JSON.parse(line.slice('RESULT:'.length))
|
|
459
|
+
if (!result.ok || !result.file) throw new Error('MLX 沒有產生音檔')
|
|
460
|
+
wav = result.file
|
|
461
|
+
} else {
|
|
462
|
+
const bin = ggufBinary()
|
|
463
|
+
if (!bin) throw new Error('找不到 crispasr 執行檔')
|
|
464
|
+
wav = `${prefix}.wav`
|
|
465
|
+
const args = ['--backend', GGUF_BACKEND, '-m', String(cfg.model || 'auto'), '--voice', ref, '--tts', cleaned, '--tts-output', wav, '-tl', 'zh', '--i-have-rights']
|
|
466
|
+
if (cloneMode === 'instruct' && instructText) args.push('--instruct', instructText)
|
|
467
|
+
if (cloneMode === 'zero-shot' && cfg.refText) args.push('--ref-text', String(cfg.refText))
|
|
468
|
+
if (cfg.skipDisclaimer !== false) args.push('--no-spoken-disclaimer', '--accept-marking-responsibility')
|
|
469
|
+
const { stderr } = await run(bin, args, { timeout: 900000, env: { ...env, HF_HOME: paths.hfHome } })
|
|
470
|
+
if (!fileExists(wav)) throw new Error(`GGUF 沒有產生音檔:${String(stderr || '').slice(-300)}`)
|
|
471
|
+
}
|
|
472
|
+
const m4a = `${prefix}.m4a`
|
|
473
|
+
await run(ffmpeg, ['-hide_banner', '-loglevel', 'error', '-y', '-i', wav, '-c:a', 'aac', '-b:a', '96k', '-ar', '44100', '-ac', '1', m4a], { timeout: 120000 })
|
|
474
|
+
const durationMs = await fileDurationMs(m4a)
|
|
475
|
+
const bytes = fs.statSync(m4a).size
|
|
476
|
+
state.last = { provider, at: Date.now(), durationMs, bytes, chars: cleaned.length, processingMs: Date.now() - startedAt }
|
|
477
|
+
state.lastError = ''
|
|
478
|
+
return { provider, wav, m4a, durationMs, bytes, chars: cleaned.length, processingMs: state.last.processingMs }
|
|
479
|
+
} catch (error) {
|
|
480
|
+
state.lastError = String(error?.message || error)
|
|
481
|
+
throw error
|
|
482
|
+
}
|
|
483
|
+
}
|
|
484
|
+
|
|
485
|
+
async function fileDurationMs(file) {
|
|
486
|
+
try {
|
|
487
|
+
const { stdout } = await run('ffprobe', ['-v', 'error', '-show_entries', 'format=duration', '-of', 'default=nw=1:nk=1', file], { timeout: 15000 })
|
|
488
|
+
return Math.max(0, Math.round(Number(String(stdout).trim()) * 1000))
|
|
489
|
+
} catch {
|
|
490
|
+
return 0
|
|
491
|
+
}
|
|
492
|
+
}
|
|
493
|
+
|
|
494
|
+
async function saveReferenceBuffer(buffer, filename, ext = '.wav') {
|
|
495
|
+
const safe = String(filename || 'upload').replace(/[^\w.\-()\u4e00-\u9fff ]+/g, '_').slice(0, 60) || 'upload'
|
|
496
|
+
const src = path.join(paths.refDir, `${Date.now()}-${safe}${safe.endsWith(ext) ? '' : ext}`)
|
|
497
|
+
fs.writeFileSync(src, buffer)
|
|
498
|
+
if (!fileExists(src)) throw new Error('寫入上傳檔案失敗')
|
|
499
|
+
const ffmpeg = which('ffmpeg')
|
|
500
|
+
if (!ffmpeg) throw new Error('需要 ffmpeg 才能轉換音檔')
|
|
501
|
+
const dest = path.join(paths.refDir, `upload-${Date.now()}.wav`)
|
|
502
|
+
await run(ffmpeg, ['-hide_banner', '-loglevel', 'error', '-y', '-i', src, '-ac', '1', '-ar', '16000', '-c:a', 'pcm_s16le', dest], { timeout: 120000 })
|
|
503
|
+
const info = referenceInfoFor(dest)
|
|
504
|
+
return { path: dest, ...info }
|
|
505
|
+
}
|
|
506
|
+
|
|
507
|
+
function referenceInfoFor(file) {
|
|
508
|
+
let seconds = 0
|
|
509
|
+
try {
|
|
510
|
+
const out = execFileSync('ffprobe', ['-v', 'error', '-show_entries', 'format=duration', '-of', 'default=nw=1:nk=1', file], { encoding: 'utf8', timeout: 8000 })
|
|
511
|
+
seconds = Number(out.trim()) || 0
|
|
512
|
+
} catch {
|
|
513
|
+
seconds = 0
|
|
514
|
+
}
|
|
515
|
+
return { seconds: Math.round(seconds * 100) / 100 }
|
|
516
|
+
}
|
|
517
|
+
|
|
518
|
+
function status() {
|
|
519
|
+
const detected = detect()
|
|
520
|
+
return {
|
|
521
|
+
...detected,
|
|
522
|
+
reference: detected.reference,
|
|
523
|
+
setup: state.setup,
|
|
524
|
+
installing: state.installing,
|
|
525
|
+
last: state.last,
|
|
526
|
+
lastError: state.lastError,
|
|
527
|
+
mlxScript: fileExists(paths.mlxScript),
|
|
528
|
+
lastSample: (() => {
|
|
529
|
+
try {
|
|
530
|
+
const files = fs.readdirSync(paths.outDir).filter((name) => name.endsWith('.m4a')).sort().reverse()
|
|
531
|
+
return files[0] ? path.join(paths.outDir, files[0]) : ''
|
|
532
|
+
} catch {
|
|
533
|
+
return ''
|
|
534
|
+
}
|
|
535
|
+
})(),
|
|
536
|
+
}
|
|
537
|
+
}
|
|
538
|
+
|
|
539
|
+
return { detect, install, synthesize, status, saveReferenceBuffer, referencePath, paths }
|
|
540
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dsh-line-bot",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.10.0",
|
|
4
4
|
"description": "把 LINE 官方帳號(Messaging API)接進 DeepSeek Harness 的插件:內建 Cloudflare Tunnel、人設選擇(大肥魚或插件自訂)、主動傳訊與完整設定面板。",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "index.js",
|
|
@@ -17,6 +17,7 @@
|
|
|
17
17
|
"client.js",
|
|
18
18
|
"cordis.patch.yml",
|
|
19
19
|
"lib",
|
|
20
|
+
"tools",
|
|
20
21
|
"locale",
|
|
21
22
|
"icon.svg",
|
|
22
23
|
"README.md",
|
|
@@ -65,7 +66,7 @@
|
|
|
65
66
|
"url": "https://github.com/Hanson0901/dsh-line-bot/issues"
|
|
66
67
|
},
|
|
67
68
|
"bin": {
|
|
68
|
-
"dsh-line-bot": "
|
|
69
|
+
"dsh-line-bot": "bin/cli.mjs"
|
|
69
70
|
},
|
|
70
71
|
"publishConfig": {
|
|
71
72
|
"access": "public"
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""One-shot MLX CosyVoice3 synthesis helper for dsh-line-bot.
|
|
3
|
+
|
|
4
|
+
Reads a JSON job file, synthesizes one WAV, and prints a single
|
|
5
|
+
`RESULT:{...}` line on stdout so the Node side can parse it.
|
|
6
|
+
"""
|
|
7
|
+
import glob
|
|
8
|
+
import json
|
|
9
|
+
import os
|
|
10
|
+
import sys
|
|
11
|
+
import traceback
|
|
12
|
+
|
|
13
|
+
|
|
14
|
+
def emit(payload):
|
|
15
|
+
sys.stdout.write("RESULT:" + json.dumps(payload, ensure_ascii=False) + "\n")
|
|
16
|
+
sys.stdout.flush()
|
|
17
|
+
|
|
18
|
+
|
|
19
|
+
def main():
|
|
20
|
+
if len(sys.argv) < 2:
|
|
21
|
+
emit({"ok": False, "error": "missing job file"})
|
|
22
|
+
return 2
|
|
23
|
+
with open(sys.argv[1], "r", encoding="utf-8") as handle:
|
|
24
|
+
job = json.load(handle)
|
|
25
|
+
|
|
26
|
+
prefix = job.get("prefix") or "tts"
|
|
27
|
+
os.makedirs(os.path.dirname(prefix) or ".", exist_ok=True)
|
|
28
|
+
|
|
29
|
+
try:
|
|
30
|
+
from mlx_audio.tts.generate import generate_audio
|
|
31
|
+
|
|
32
|
+
kwargs = {
|
|
33
|
+
"text": job["text"],
|
|
34
|
+
"model": job.get("model") or "mlx-community/Fun-CosyVoice3-0.5B-2512-8bit",
|
|
35
|
+
"ref_audio": job.get("refAudio"),
|
|
36
|
+
"file_prefix": prefix,
|
|
37
|
+
"audio_format": "wav",
|
|
38
|
+
"verbose": False,
|
|
39
|
+
"join_audio": True,
|
|
40
|
+
}
|
|
41
|
+
if job.get("instructText"):
|
|
42
|
+
kwargs["instruct_text"] = job["instructText"]
|
|
43
|
+
elif job.get("refText"):
|
|
44
|
+
kwargs["ref_text"] = job["refText"]
|
|
45
|
+
generate_audio(**kwargs)
|
|
46
|
+
|
|
47
|
+
candidates = sorted(glob.glob(prefix + "*.wav"), key=os.path.getmtime)
|
|
48
|
+
if not candidates:
|
|
49
|
+
emit({"ok": False, "error": "no output wav produced"})
|
|
50
|
+
return 1
|
|
51
|
+
emit({"ok": True, "file": candidates[-1]})
|
|
52
|
+
return 0
|
|
53
|
+
except Exception as exc: # noqa: BLE001
|
|
54
|
+
traceback.print_exc(file=sys.stderr)
|
|
55
|
+
emit({"ok": False, "error": str(exc)})
|
|
56
|
+
return 1
|
|
57
|
+
|
|
58
|
+
|
|
59
|
+
if __name__ == "__main__":
|
|
60
|
+
raise SystemExit(main())
|