dsh-dafeiyu-emotion 0.3.0 → 0.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,14 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.3.1
4
+
5
+ - Added spoken assistant replies. The host now forwards `assistant/message` text over `/plugins/dsh-dafeiyu-emotion/events`; the client plays it with TTS.
6
+ - Added local Piper TTS (`zh_CN-huayan-medium`) as the default engine, with Edge TTS as an optional online engine and browser `speechSynthesis` as the final fallback.
7
+ - Added `ttsEnabled`, `ttsEngine`, `ttsVoice`, `ttsRate`, `ttsPitch` and `ttsPiperModel` settings, including a voice selector and model path field.
8
+ - Added `POST /plugins/dsh-dafeiyu-emotion/tts`, returning WAV for Piper and MP3 for Edge.
9
+ - Edge TTS failures and missing Piper models fall back to local browser speech instead of breaking the reply.
10
+ - Added TTS handler, assistant-text extraction, SSE broadcast and markdown-cleaning tests.
11
+
3
12
  ## 0.3.0
4
13
 
5
14
  - Added `empathyStyle` with four reply strategies: gentle, direct, humorous and concise. The choice is persisted and injected into the emotion-aware prompt.
package/README.md CHANGED
@@ -76,6 +76,7 @@
76
76
  - 📈 多段语音的情绪时间线,可悬停查看每段文本、情绪和置信度;
77
77
  - 🔐 设置页内的隐私与运行状态:最近音频大小、模型加载、worker PID、临时文件清理;
78
78
  - 🔌 本机 `/context` 接口,其他插件可以读取最近一次情绪上下文;
79
+ - 🔊 大肥鱼回复语音:Piper 本地模型默认朗读,Edge / 浏览器语音可选;
79
80
  - 🐋 大肥鱼主题的“鱼耳”入口,和普通对话共用同一个 composer;
80
81
  - 🧠 情绪标签:喜悦 / 低落 / 生气 / 紧张 / 惊讶 / 厌烦 / 平静;
81
82
  - 📈 额外给出 confidence、valence、arousal 和语气摘要;
@@ -115,13 +116,34 @@ dsh plugin --profile desktop add github:LiDu-china/dsh-dafeiyu-emotion
115
116
  ### 通过 Release tarball
116
117
 
117
118
  ```bash
118
- curl -L -o dsh-dafeiyu-emotion-0.3.0.tgz \
119
- https://github.com/LiDu-china/dsh-dafeiyu-emotion/releases/download/v0.3.0/dsh-dafeiyu-emotion-0.3.0.tgz
120
- dsh plugin --profile desktop add "D:/path/to/dsh-dafeiyu-emotion-0.3.0.tgz"
119
+ curl -L -o dsh-dafeiyu-emotion-0.3.1.tgz \
120
+ https://github.com/LiDu-china/dsh-dafeiyu-emotion/releases/download/v0.3.1/dsh-dafeiyu-emotion-0.3.1.tgz
121
+ dsh plugin --profile desktop add "D:/path/to/dsh-dafeiyu-emotion-0.3.1.tgz"
121
122
  ```
122
123
 
123
124
  > DSH Desktop 的 profile 由 Electron 应用管理;命令行安装请使用桌面应用自带的 `dsh` 命令。
124
125
 
126
+ ## 大肥鱼语音回复
127
+
128
+ 助手回复生成后,插件会通过本机事件流把文本交给 TTS,让大肥鱼把回复读出来。
129
+
130
+ 默认使用 **Piper 本地模型**,音色是 `zh_CN-huayan-medium`。它不联网,延迟低,适合日常对话:
131
+
132
+ ```bash
133
+ python -m pip install piper-tts
134
+ ```
135
+
136
+ 把模型放到默认目录:
137
+
138
+ ```text
139
+ ~/.dsh/dafeiyu-emotion/tts/piper/zh_CN-huayan-medium.onnx
140
+ ~/.dsh/dafeiyu-emotion/tts/piper/zh_CN-huayan-medium.onnx.json
141
+ ```
142
+
143
+ 也可以在设置页填写自定义 Piper 模型路径。模型缺失或合成失败时,会自动回退到浏览器 `speechSynthesis`,不会让回复静默丢失。
144
+
145
+ 如果更喜欢更自然的在线音色,可以把引擎切到 **Edge TTS**,默认音色 `zh-CN-XiaoyiNeural`。Edge 需要网络,且回复文本会发送到 Microsoft;如果网络或服务不可用,同样会回退到浏览器语音。
146
+
125
147
  ## 设置页与侧边栏
126
148
 
127
149
  插件会把自己的设置卡片注册到 DSH 的插件设置页。打开:
@@ -190,6 +212,12 @@ dsh plugin --profile desktop add "D:/path/to/dsh-dafeiyu-emotion-0.3.0.tgz"
190
212
  | `pythonPath` | 空 | 自定义 Python;空则自动寻找 `.venv-ser` |
191
213
  | `language` | `zh` | 识别语言:`zh` / `en` / `auto` |
192
214
  | `empathyStyle` | `gentle` | 共情策略:`gentle` / `direct` / `humorous` / `concise` |
215
+ | `ttsEnabled` | `true` | 是否朗读大肥鱼回复 |
216
+ | `ttsEngine` | `piper` | `piper` 本地 / `edge` 在线 / `browser` 系统语音 |
217
+ | `ttsVoice` | `zh-CN-XiaoyiNeural` | Edge TTS 音色 |
218
+ | `ttsRate` | `+6%` | Piper/Edge 语速 |
219
+ | `ttsPitch` | `+12Hz` | Edge 音调 |
220
+ | `ttsPiperModel` | 空 | 自定义 Piper 模型路径;空则用默认目录 |
193
221
  | `maxAudioBytes` | 12 MB | 单次录音大小上限 |
194
222
  | `startupTimeoutMs` | 180000 | 首次加载增强后端的最长等待 |
195
223
  | `requestTimeoutMs` | 180000 | 单次分析请求超时 |
package/cordis.patch.yml CHANGED
@@ -29,6 +29,14 @@
29
29
  pythonPath: ""
30
30
  language: zh
31
31
  empathyStyle: gentle
32
+ ttsEnabled: true
33
+ ttsEngine: piper
34
+ ttsVoice: zh-CN-XiaoyiNeural
35
+ ttsRate: "+6%"
36
+ ttsPitch: "+12Hz"
37
+ ttsCommand: ""
38
+ ttsPiperModel: ""
39
+ ttsPiperCommand: ""
32
40
  maxAudioBytes: 12582912
33
41
  startupTimeoutMs: 180000
34
42
  requestTimeoutMs: 180000
@@ -72,7 +72,7 @@ https://github.com/LiDu-china/dsh-dafeiyu-emotion/releases/download/v0.2.3/dsh-d
72
72
  dsh plugin --profile desktop add dsh-dafeiyu-emotion
73
73
  ```
74
74
 
75
- Published as `dsh-dafeiyu-emotion@0.3.0` on npm. The 1024Store entry refreshes its one-click install automatically. GitHub-source and Release-tarball installs remain available as fallbacks.
75
+ Published as `dsh-dafeiyu-emotion@0.3.1` on npm. The 1024Store entry refreshes its one-click install automatically. GitHub-source and Release-tarball installs remain available as fallbacks.
76
76
 
77
77
  ## Privacy
78
78
 
@@ -0,0 +1,39 @@
1
+ ## 0.3.1
2
+
3
+ 大肥鱼现在会说话了。
4
+
5
+ ### 语音回复
6
+
7
+ - Host 监听 `assistant/message`,把回复文本通过 `/plugins/dsh-dafeiyu-emotion/events` 推给客户端。
8
+ - 客户端收到后朗读回复,默认使用 **Piper 本地模型** `zh_CN-huayan-medium`。
9
+ - 可选 **Edge TTS**(`zh-CN-XiaoyiNeural`,更自然,但需要网络),以及 **浏览器 speechSynthesis** 兜底。
10
+ - Piper/Edge 失败时自动回退浏览器语音,不会让回复静默丢失。
11
+
12
+ ### 设置
13
+
14
+ - 朗读开关、合成引擎、Edge 音色、语速、音调、Piper 模型路径。
15
+ - 默认 `ttsEnabled: true`,Piper 模型不存在时自动使用浏览器语音。
16
+
17
+ ### 本地模型
18
+
19
+ ```bash
20
+ python -m pip install piper-tts
21
+ ```
22
+
23
+ 默认模型目录:
24
+
25
+ ```text
26
+ ~/.dsh/dafeiyu-emotion/tts/piper/zh_CN-huayan-medium.onnx
27
+ ~/.dsh/dafeiyu-emotion/tts/piper/zh_CN-huayan-medium.onnx.json
28
+ ```
29
+
30
+ 可以在设置页填写自定义路径。
31
+
32
+ ### 接口
33
+
34
+ ```text
35
+ POST /plugins/dsh-dafeiyu-emotion/tts
36
+ GET /plugins/dsh-dafeiyu-emotion/events
37
+ ```
38
+
39
+ `/tts` 返回 WAV(Piper)或 MP3(Edge),只允许本机访问。
package/lib/client.js CHANGED
@@ -10,6 +10,8 @@ window.__ModuleLoader__.load({
10
10
  const ANALYZE_ENDPOINT = '/plugins/dsh-dafeiyu-emotion/analyze'
11
11
  const CONFIG_ENDPOINT = '/plugins/dsh-dafeiyu-emotion/config'
12
12
  const HEALTH_ENDPOINT = '/plugins/dsh-dafeiyu-emotion/health'
13
+ const TTS_ENDPOINT = '/plugins/dsh-dafeiyu-emotion/tts'
14
+ const EVENTS_ENDPOINT = '/plugins/dsh-dafeiyu-emotion/events'
13
15
  const MAX_RECORDING_MS = 30000
14
16
  const PERMISSION_TIMEOUT_MS = 20000
15
17
  const EMOTION_LABELS = {
@@ -823,6 +825,12 @@ window.__ModuleLoader__.load({
823
825
  || draft.pythonPath !== config.pythonPath
824
826
  || draft.language !== config.language
825
827
  || draft.empathyStyle !== config.empathyStyle
828
+ || draft.ttsEnabled !== config.ttsEnabled
829
+ || draft.ttsEngine !== config.ttsEngine
830
+ || draft.ttsVoice !== config.ttsVoice
831
+ || draft.ttsRate !== config.ttsRate
832
+ || draft.ttsPitch !== config.ttsPitch
833
+ || draft.ttsPiperModel !== config.ttsPiperModel
826
834
  || draft.maxAudioBytes !== config.maxAudioBytes
827
835
  || draft.startupTimeoutMs !== config.startupTimeoutMs
828
836
  || draft.requestTimeoutMs !== config.requestTimeoutMs
@@ -831,7 +839,7 @@ window.__ModuleLoader__.load({
831
839
  const save = async () => {
832
840
  if (!config || !draft || saving) return
833
841
  const patch = {}
834
- for (const key of ['enabled', 'backendRoot', 'pythonPath', 'language', 'empathyStyle', 'maxAudioBytes', 'startupTimeoutMs', 'requestTimeoutMs']) {
842
+ for (const key of ['enabled', 'backendRoot', 'pythonPath', 'language', 'empathyStyle', 'ttsEnabled', 'ttsEngine', 'ttsVoice', 'ttsRate', 'ttsPitch', 'ttsPiperModel', 'maxAudioBytes', 'startupTimeoutMs', 'requestTimeoutMs']) {
835
843
  if (draft[key] !== config[key]) patch[key] = draft[key]
836
844
  }
837
845
  if (Object.keys(patch).length === 0) return
@@ -957,6 +965,7 @@ window.__ModuleLoader__.load({
957
965
  event.currentTarget.style.borderColor = T.border
958
966
  }
959
967
  const numberStyle = { ...inputStyle, width: 96, textAlign: 'right', fontVariantNumeric: 'tabular-nums' }
968
+ const selectStyle = { ...inputStyle, width: isSidebar ? '100%' : 220, paddingRight: 26 }
960
969
  const secondaryButton = {
961
970
  height: 32,
962
971
  padding: '0 11px',
@@ -994,6 +1003,15 @@ window.__ModuleLoader__.load({
994
1003
  },
995
1004
  }, value),
996
1005
  )
1006
+ const voiceOptions = [
1007
+ ['zh-CN-XiaoyiNeural', '小艺 · 活泼'],
1008
+ ['zh-CN-XiaoxiaoNeural', '晓晓 · 温柔'],
1009
+ ['zh-CN-YunxiNeural', '云希 · 男声'],
1010
+ ['zh-CN-liaoning-XiaobeiNeural', '小北 · 东北'],
1011
+ ]
1012
+ if (draft.ttsVoice && !voiceOptions.some(([value]) => value === draft.ttsVoice)) {
1013
+ voiceOptions.push([draft.ttsVoice, draft.ttsVoice])
1014
+ }
997
1015
 
998
1016
  return h(Root, {
999
1017
  'data-testid': 'dsh-dafeiyu-emotion-settings',
@@ -1235,6 +1253,84 @@ window.__ModuleLoader__.load({
1235
1253
  onChange: (value) => update('empathyStyle', value),
1236
1254
  })),
1237
1255
  ),
1256
+ h(SettingsSection, { title: '语音回复' },
1257
+ h(SettingsField, {
1258
+ icon: 'spark',
1259
+ label: '朗读大肥鱼回复',
1260
+ hint: '开启后,助手回复会自动朗读。Piper 本地模型最稳;Edge 需要网络;浏览器引擎留在本机。',
1261
+ }, h(SettingsSwitch, {
1262
+ checked: draft.ttsEnabled === true,
1263
+ disabled,
1264
+ label: '朗读大肥鱼回复',
1265
+ onChange: (value) => update('ttsEnabled', value),
1266
+ })),
1267
+ h(SettingsField, {
1268
+ icon: 'spark',
1269
+ label: '合成引擎',
1270
+ hint: 'Piper 使用本地中文模型;Edge 音色更自然但可能被网络限制;浏览器使用系统语音。',
1271
+ }, h(SettingsSegmented, {
1272
+ value: draft.ttsEngine || 'piper',
1273
+ label: '合成引擎',
1274
+ disabled,
1275
+ options: [
1276
+ { value: 'piper', label: 'Piper' },
1277
+ { value: 'edge', label: 'Edge' },
1278
+ { value: 'browser', label: '浏览器' },
1279
+ ],
1280
+ onChange: (value) => update('ttsEngine', value),
1281
+ })),
1282
+ h(SettingsField, {
1283
+ icon: 'spark',
1284
+ label: '音色',
1285
+ hint: 'Edge TTS 的中文女声/男声;浏览器引擎使用系统默认音色。',
1286
+ wide: true,
1287
+ }, h('select', {
1288
+ value: draft.ttsVoice || 'zh-CN-XiaoyiNeural',
1289
+ disabled: disabled || draft.ttsEngine !== 'edge',
1290
+ 'aria-label': 'Edge TTS 音色',
1291
+ onChange: (event) => update('ttsVoice', event.target.value),
1292
+ style: selectStyle,
1293
+ }, ...voiceOptions.map(([value, label]) => h('option', { key: value, value }, label)))),
1294
+ draft.ttsEngine === 'piper' || draft.ttsEngine === 'edge'
1295
+ ? h(SettingsField, {
1296
+ icon: 'folder',
1297
+ label: 'Piper 模型',
1298
+ hint: '留空使用 ~/.dsh/dafeiyu-emotion/tts/piper/zh_CN-huayan-medium.onnx。',
1299
+ wide: true,
1300
+ }, h('input', {
1301
+ type: 'text',
1302
+ value: draft.ttsPiperModel || '',
1303
+ disabled: disabled || draft.ttsEngine !== 'piper',
1304
+ spellCheck: false,
1305
+ placeholder: '留空使用默认 Piper 模型',
1306
+ 'aria-label': 'Piper 模型路径',
1307
+ onChange: (event) => update('ttsPiperModel', event.target.value),
1308
+ style: { ...inputStyle, width: '100%' },
1309
+ }))
1310
+ : null,
1311
+ h(SettingsField, {
1312
+ icon: 'spark',
1313
+ label: '语速 / 音调',
1314
+ hint: '语速 +6% 适用于 Piper/Edge;音调 +12Hz 仅 Edge。',
1315
+ }, h('div', { style: { display: 'flex', alignItems: 'center', gap: 6, flexWrap: 'wrap' } },
1316
+ h('input', {
1317
+ type: 'text',
1318
+ value: draft.ttsRate || '+6%',
1319
+ disabled: disabled || draft.ttsEngine === 'browser',
1320
+ 'aria-label': '语速',
1321
+ onChange: (event) => update('ttsRate', event.target.value),
1322
+ style: { ...inputStyle, width: 76 },
1323
+ }),
1324
+ h('input', {
1325
+ type: 'text',
1326
+ value: draft.ttsPitch || '+12Hz',
1327
+ disabled: disabled || draft.ttsEngine !== 'edge',
1328
+ 'aria-label': '音调',
1329
+ onChange: (event) => update('ttsPitch', event.target.value),
1330
+ style: { ...inputStyle, width: 86 },
1331
+ }),
1332
+ )),
1333
+ ),
1238
1334
  h(SettingsSection, { title: '增强后端', hint: '可选' },
1239
1335
  h(SettingsField, {
1240
1336
  icon: 'folder',
@@ -1449,6 +1545,13 @@ window.__ModuleLoader__.load({
1449
1545
  const [sending, setSending] = useState(false)
1450
1546
  const [pluginEnabled, setPluginEnabled] = useState(null)
1451
1547
  const [language, setLanguage] = useState('zh')
1548
+ const [ttsConfig, setTtsConfig] = useState({
1549
+ enabled: false,
1550
+ engine: 'piper',
1551
+ voice: 'zh-CN-XiaoyiNeural',
1552
+ rate: '+6%',
1553
+ pitch: '+12Hz',
1554
+ })
1452
1555
  const recorder = useRef(null)
1453
1556
  const abort = useRef(null)
1454
1557
  const timer = useRef(null)
@@ -1458,6 +1561,10 @@ window.__ModuleLoader__.load({
1458
1561
  const levelsRef = useRef([])
1459
1562
  const shortcutAction = useRef(null)
1460
1563
  const shortcutRegistration = useRef(null)
1564
+ const ttsRef = useRef(ttsConfig)
1565
+ const audioRef = useRef(null)
1566
+ const speechGeneration = useRef(0)
1567
+ ttsRef.current = ttsConfig
1461
1568
  const deliveryGate = useRef(null)
1462
1569
  if (!deliveryGate.current) deliveryGate.current = createDeliveryGate()
1463
1570
  const useInputSnapshot = typeof useInput === 'function' ? useInput : useNoInput
@@ -1734,12 +1841,95 @@ window.__ModuleLoader__.load({
1734
1841
  shortcutRegistration.current?.touch?.()
1735
1842
  }
1736
1843
 
1844
+ function speakBrowser(text) {
1845
+ const synth = window.speechSynthesis
1846
+ if (!synth || typeof SpeechSynthesisUtterance === 'undefined') return false
1847
+ synth.cancel()
1848
+ const utterance = new SpeechSynthesisUtterance(text)
1849
+ utterance.lang = 'zh-CN'
1850
+ utterance.rate = 1.06
1851
+ utterance.pitch = 1.08
1852
+ synth.speak(utterance)
1853
+ return true
1854
+ }
1855
+
1856
+ async function speakReply(text) {
1857
+ const config = ttsRef.current
1858
+ if (!config?.enabled || !text) return false
1859
+ const run = ++speechGeneration.current
1860
+ if (config.engine === 'browser') return speakBrowser(text)
1861
+ try {
1862
+ const response = await fetch(TTS_ENDPOINT, {
1863
+ method: 'POST',
1864
+ headers: { 'content-type': 'application/json' },
1865
+ body: JSON.stringify({
1866
+ text,
1867
+ voice: config.voice,
1868
+ rate: config.rate,
1869
+ pitch: config.pitch,
1870
+ }),
1871
+ })
1872
+ if (!response.ok) {
1873
+ const payload = await response.json().catch(() => ({}))
1874
+ throw new Error(payload.error || `TTS 失败(${response.status})`)
1875
+ }
1876
+ const blob = await response.blob()
1877
+ if (run !== speechGeneration.current) return false
1878
+ const url = URL.createObjectURL(blob)
1879
+ audioRef.current?.pause?.()
1880
+ const audio = new Audio(url)
1881
+ audioRef.current = audio
1882
+ audio.onended = () => {
1883
+ URL.revokeObjectURL(url)
1884
+ if (audioRef.current === audio) audioRef.current = null
1885
+ }
1886
+ try {
1887
+ await audio.play()
1888
+ return true
1889
+ } catch {
1890
+ URL.revokeObjectURL(url)
1891
+ return speakBrowser(text)
1892
+ }
1893
+ } catch {
1894
+ return speakBrowser(text)
1895
+ }
1896
+ }
1897
+
1898
+ useEffect(() => {
1899
+ if (typeof EventSource === 'undefined') return undefined
1900
+ const source = new EventSource(EVENTS_ENDPOINT)
1901
+ source.onmessage = (event) => {
1902
+ try {
1903
+ const message = JSON.parse(event.data)
1904
+ if (message?.kind !== 'assistant' || !message.text) return
1905
+ if (message.sessionId && sessionId && message.sessionId !== sessionId) return
1906
+ void speakReply(message.text)
1907
+ } catch {}
1908
+ }
1909
+ return () => source.close()
1910
+ }, [sessionId])
1911
+
1912
+ useEffect(() => () => {
1913
+ speechGeneration.current += 1
1914
+ audioRef.current?.pause?.()
1915
+ window.speechSynthesis?.cancel?.()
1916
+ }, [])
1917
+
1737
1918
  useEffect(() => {
1738
1919
  let active = true
1739
1920
  const applyConfig = (next) => {
1740
1921
  if (!active || !next) return
1741
1922
  setPluginEnabled(next.enabled !== false)
1742
1923
  if (next.language) setLanguage(next.language)
1924
+ setTtsConfig({
1925
+ enabled: next.ttsEnabled === true,
1926
+ engine: next.ttsEngine === 'browser'
1927
+ ? 'browser'
1928
+ : next.ttsEngine === 'piper' ? 'piper' : 'edge',
1929
+ voice: next.ttsVoice || 'zh-CN-XiaoyiNeural',
1930
+ rate: next.ttsRate || '+6%',
1931
+ pitch: next.ttsPitch || '+12Hz',
1932
+ })
1743
1933
  }
1744
1934
  fetch(CONFIG_ENDPOINT, { cache: 'no-store' })
1745
1935
  .then((response) => (response.ok ? response.json() : null))
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "dsh-dafeiyu-emotion",
3
- "version": "0.3.0",
3
+ "version": "0.3.1",
4
4
  "description": "大肥鱼语音情绪耳朵:听语气、识别情绪,并让 DSH 按对方当下的状态自然回复。 · Voice + emotion-aware companion for DeepSeek Harness.",
5
5
  "type": "module",
6
6
  "main": "src/index.js",
@@ -61,7 +61,7 @@
61
61
  },
62
62
  "scripts": {
63
63
  "check": "node --check src/index.js && node --check lib/client.js",
64
- "test": "node --check src/index.js && node --check lib/client.js && node --test scripts/test-config-schema.mjs scripts/test-config-handler.mjs scripts/test-analyze-handler.mjs scripts/test-backend-cancel.mjs scripts/test-delivery-policy.mjs scripts/test-shortcut-registration.mjs scripts/test-client-registration.mjs"
64
+ "test": "node --check src/index.js && node --check lib/client.js && node --test scripts/test-config-schema.mjs scripts/test-config-handler.mjs scripts/test-analyze-handler.mjs scripts/test-backend-cancel.mjs scripts/test-tts-handler.mjs scripts/test-delivery-policy.mjs scripts/test-shortcut-registration.mjs scripts/test-client-registration.mjs"
65
65
  },
66
66
  "dependencies": {
67
67
  "@deepseek-ai/schemastery": "3.18.5",
package/src/index.js CHANGED
@@ -1,6 +1,6 @@
1
1
  import { spawn } from 'node:child_process'
2
2
  import { existsSync } from 'node:fs'
3
- import { mkdtemp, rm, writeFile } from 'node:fs/promises'
3
+ import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'
4
4
  import { createInterface } from 'node:readline'
5
5
  import { tmpdir } from 'node:os'
6
6
  import { dirname, join, resolve } from 'node:path'
@@ -14,6 +14,8 @@ const ANALYZE_ENDPOINT = '/plugins/dsh-dafeiyu-emotion/analyze'
14
14
  const HEALTH_ENDPOINT = '/plugins/dsh-dafeiyu-emotion/health'
15
15
  const CONFIG_ENDPOINT = '/plugins/dsh-dafeiyu-emotion/config'
16
16
  const CONTEXT_ENDPOINT = '/plugins/dsh-dafeiyu-emotion/context'
17
+ const TTS_ENDPOINT = '/plugins/dsh-dafeiyu-emotion/tts'
18
+ const EVENTS_ENDPOINT = '/plugins/dsh-dafeiyu-emotion/events'
17
19
  const SETTINGS_NAMESPACE = 'dsh-dafeiyu-emotion'
18
20
 
19
21
  const here = dirname(fileURLToPath(import.meta.url))
@@ -28,6 +30,14 @@ const DEFAULTS = Object.freeze({
28
30
  startupTimeoutMs: 180000,
29
31
  requestTimeoutMs: 180000,
30
32
  empathyStyle: 'gentle',
33
+ ttsEnabled: true,
34
+ ttsEngine: 'piper',
35
+ ttsVoice: 'zh-CN-XiaoyiNeural',
36
+ ttsRate: '+6%',
37
+ ttsPitch: '+12Hz',
38
+ ttsCommand: '',
39
+ ttsPiperModel: '',
40
+ ttsPiperCommand: '',
31
41
  })
32
42
 
33
43
  const CONFIG_FIELDS = new Set(Object.keys(DEFAULTS))
@@ -69,6 +79,18 @@ export const Config = Schema.object({
69
79
  Schema.const('humorous').description('幽默'),
70
80
  Schema.const('concise').description('简洁'),
71
81
  ]).default('gentle').description('共情策略').volatile(),
82
+ ttsEnabled: Schema.boolean().default(true).description('朗读大肥鱼回复').volatile(),
83
+ ttsEngine: Schema.union([
84
+ Schema.const('piper').description('Piper 本地模型(推荐)'),
85
+ Schema.const('edge').description('Edge TTS(更好听,需要网络)'),
86
+ Schema.const('browser').description('浏览器本地语音'),
87
+ ]).default('piper').description('语音合成引擎').volatile(),
88
+ ttsVoice: Schema.string().default('zh-CN-XiaoyiNeural').description('Edge TTS 音色').volatile(),
89
+ ttsRate: Schema.string().default('+6%').description('语速').volatile(),
90
+ ttsPitch: Schema.string().default('+12Hz').description('音调').volatile(),
91
+ ttsCommand: Schema.string().default('').description('edge-tts 可执行文件路径').volatile(),
92
+ ttsPiperModel: Schema.string().default('').description('Piper 模型路径').volatile(),
93
+ ttsPiperCommand: Schema.string().default('').description('Piper 可执行文件路径').volatile(),
72
94
  }).description('大肥鱼语音情绪耳朵')
73
95
 
74
96
  const EMOTION_ZH = Object.freeze({
@@ -125,6 +147,266 @@ function jsonResponse(res, status, body) {
125
147
  res.end(payload)
126
148
  }
127
149
 
150
+ function resolveEdgeTts(options) {
151
+ const explicit = String(options?.ttsCommand || '').trim()
152
+ if (explicit) return explicit
153
+ const fromEnv = String(process.env.EDGE_TTS_PATH || '').trim()
154
+ if (fromEnv) return fromEnv
155
+ const local = process.env.LOCALAPPDATA
156
+ const roaming = process.env.APPDATA
157
+ const candidates = [
158
+ local && join(local, 'Programs', 'Python', 'Python313', 'Scripts', 'edge-tts.exe'),
159
+ local && join(local, 'Programs', 'Python', 'Python312', 'Scripts', 'edge-tts.exe'),
160
+ roaming && join(roaming, 'Python', 'Python313', 'Scripts', 'edge-tts.exe'),
161
+ 'edge-tts',
162
+ ].filter(Boolean)
163
+ return candidates.find((candidate) => candidate === 'edge-tts' || existsSync(candidate)) || 'edge-tts'
164
+ }
165
+
166
+ function defaultPiperModel() {
167
+ const home = process.env.USERPROFILE || process.env.HOME || ''
168
+ return join(home, '.dsh', 'dafeiyu-emotion', 'tts', 'piper', 'zh_CN-huayan-medium.onnx')
169
+ }
170
+
171
+ function resolvePiperModel(options) {
172
+ return String(options?.ttsPiperModel || process.env.DSH_DAFEIYU_PIPER_MODEL || defaultPiperModel()).trim()
173
+ }
174
+
175
+ function resolvePiperCommand(options) {
176
+ const explicit = String(options?.ttsPiperCommand || process.env.PIPER_PATH || '').trim()
177
+ if (explicit) return explicit
178
+ const local = process.env.LOCALAPPDATA
179
+ const candidates = [
180
+ local && join(local, 'Programs', 'Python', 'Python313', 'Scripts', 'piper.exe'),
181
+ local && join(local, 'Programs', 'Python', 'Python312', 'Scripts', 'piper.exe'),
182
+ 'piper',
183
+ ].filter(Boolean)
184
+ return candidates.find((candidate) => candidate === 'piper' || existsSync(candidate)) || 'piper'
185
+ }
186
+
187
+ async function synthesizePiper(text, options) {
188
+ const model = resolvePiperModel(options)
189
+ const command = resolvePiperCommand(options)
190
+ if (!existsSync(model)) throw new Error(`Piper 模型不存在:${model}`)
191
+ const config = `${model}.json`
192
+ if (!existsSync(config)) throw new Error(`Piper 模型配置不存在:${config}`)
193
+ const dir = await mkdtemp(join(tmpdir(), 'dsh-dafeiyu-tts-'))
194
+ const input = join(dir, 'input.txt')
195
+ const out = join(dir, 'reply.wav')
196
+ try {
197
+ await writeFile(input, text, 'utf8')
198
+ const speed = Number.parseFloat(String(options.ttsRate || '+0%').replace('%', ''))
199
+ const lengthScale = Number.isFinite(speed) && speed !== 0 ? 1 / (1 + speed / 100) : 1
200
+ await new Promise((resolvePromise, rejectPromise) => {
201
+ const proc = spawn(command, [
202
+ '-m', model,
203
+ '-c', config,
204
+ '-i', input,
205
+ '-f', out,
206
+ '--length-scale', String(Math.max(0.7, Math.min(1.4, lengthScale))),
207
+ ], {
208
+ windowsHide: true,
209
+ env: { ...process.env, PYTHONUTF8: '1', PYTHONIOENCODING: 'utf-8' },
210
+ })
211
+ let stderr = ''
212
+ let settled = false
213
+ const timer = setTimeout(() => {
214
+ if (settled) return
215
+ settled = true
216
+ try { proc.kill() } catch {}
217
+ rejectPromise(new Error('Piper 合成超时'))
218
+ }, 30000)
219
+ proc.stderr?.on('data', (chunk) => { stderr += chunk.toString('utf8') })
220
+ proc.on('error', (error) => {
221
+ if (settled) return
222
+ settled = true
223
+ clearTimeout(timer)
224
+ rejectPromise(error)
225
+ })
226
+ proc.on('exit', (code) => {
227
+ if (settled) return
228
+ settled = true
229
+ clearTimeout(timer)
230
+ if (code === 0) resolvePromise()
231
+ else rejectPromise(new Error(stderr.trim() || `piper 退出(code=${code})`))
232
+ })
233
+ })
234
+ return await readFile(out)
235
+ } finally {
236
+ await rm(dir, { recursive: true, force: true }).catch(() => {})
237
+ }
238
+ }
239
+
240
+ function ttsReady(options) {
241
+ if (!options.ttsEnabled) return false
242
+ if (options.ttsEngine === 'browser') return true
243
+ if (options.ttsEngine === 'piper') return existsSync(resolvePiperModel(options))
244
+ return existsSync(resolveEdgeTts(options))
245
+ }
246
+
247
+ function assistantTextFromEvent(event) {
248
+ const content = event?.data?.message?.content
249
+ if (!Array.isArray(content)) return ''
250
+ return content
251
+ .filter((block) => block?.type === 'text' && typeof block.text === 'string')
252
+ .map((block) => block.text)
253
+ .join('\n')
254
+ }
255
+
256
+ function cleanForSpeech(text) {
257
+ return String(text || '')
258
+ .replace(/```[\s\S]*?```/g, ' ')
259
+ .replace(/`[^`]*`/g, ' ')
260
+ .replace(/!\[[^\]]*\]\([^)]*\)/g, ' ')
261
+ .replace(/\[([^\]]+)\]\([^)]*\)/g, '$1')
262
+ .replace(/^#{1,6}\s+/gm, '')
263
+ .replace(/[*_~>#]/g, ' ')
264
+ .replace(/\s+/g, ' ')
265
+ .trim()
266
+ .slice(0, 800)
267
+ }
268
+
269
+ async function synthesizeEdge(text, options) {
270
+ const command = resolveEdgeTts(options)
271
+ const dir = await mkdtemp(join(tmpdir(), 'dsh-dafeiyu-tts-'))
272
+ const out = join(dir, 'reply.mp3')
273
+ try {
274
+ await new Promise((resolvePromise, rejectPromise) => {
275
+ const proc = spawn(command, [
276
+ '--voice', options.ttsVoice,
277
+ '--rate', options.ttsRate,
278
+ '--pitch', options.ttsPitch,
279
+ '--text', text,
280
+ '--write-media', out,
281
+ ], { windowsHide: true })
282
+ let stderr = ''
283
+ let settled = false
284
+ const timer = setTimeout(() => {
285
+ if (settled) return
286
+ settled = true
287
+ try { proc.kill() } catch {}
288
+ rejectPromise(new Error('TTS 合成超时'))
289
+ }, 25000)
290
+ proc.stderr?.on('data', (chunk) => { stderr += chunk.toString('utf8') })
291
+ proc.on('error', (error) => {
292
+ if (settled) return
293
+ settled = true
294
+ clearTimeout(timer)
295
+ rejectPromise(error)
296
+ })
297
+ proc.on('exit', (code) => {
298
+ if (settled) return
299
+ settled = true
300
+ clearTimeout(timer)
301
+ if (code === 0) resolvePromise()
302
+ else rejectPromise(new Error(stderr.trim() || `edge-tts 退出(code=${code})`))
303
+ })
304
+ })
305
+ return await readFile(out)
306
+ } finally {
307
+ await rm(dir, { recursive: true, force: true }).catch(() => {})
308
+ }
309
+ }
310
+
311
+ function createTtsHandler(runtime) {
312
+ return async (req, res) => {
313
+ if (!isLocalRequest(req)) {
314
+ jsonResponse(res, 403, { error: '只允许本机访问' })
315
+ return
316
+ }
317
+ if (req.method !== 'POST') {
318
+ jsonResponse(res, 405, { error: 'method not allowed' })
319
+ return
320
+ }
321
+ if (!runtime.options.ttsEnabled) {
322
+ jsonResponse(res, 503, { error: '语音回复已关闭' })
323
+ return
324
+ }
325
+ try {
326
+ const body = await readJsonBody(req, 16 * 1024)
327
+ const text = cleanForSpeech(body?.text)
328
+ if (!text) throw new HttpError(400, '没有可朗读的文本')
329
+ if (runtime.options.ttsEngine === 'browser') {
330
+ throw new HttpError(400, '浏览器语音引擎在客户端本地运行,不需要调用 Host TTS')
331
+ }
332
+ const voice = String(body?.voice || runtime.options.ttsVoice)
333
+ const rate = String(body?.rate || runtime.options.ttsRate)
334
+ const pitch = String(body?.pitch || runtime.options.ttsPitch)
335
+ if (!/^[+-]\d{1,3}%$/.test(rate)) throw new HttpError(400, '语速格式不合法')
336
+ let audio
337
+ let contentType
338
+ if (runtime.options.ttsEngine === 'piper') {
339
+ audio = await synthesizePiper(text, { ...runtime.options, ttsRate: rate })
340
+ contentType = 'audio/wav'
341
+ } else {
342
+ if (!/^[A-Za-z]{2,3}-[A-Za-z0-9-]+Neural$/.test(voice)) throw new HttpError(400, '音色不合法')
343
+ if (!/^[+-]\d{1,3}Hz$/.test(pitch)) throw new HttpError(400, '音调格式不合法')
344
+ audio = await synthesizeEdge(text, {
345
+ ...runtime.options,
346
+ ttsVoice: voice,
347
+ ttsRate: rate,
348
+ ttsPitch: pitch,
349
+ })
350
+ contentType = 'audio/mpeg'
351
+ }
352
+ res.writeHead(200, {
353
+ 'content-type': contentType,
354
+ 'cache-control': 'no-store',
355
+ 'content-length': audio.length,
356
+ })
357
+ res.end(audio)
358
+ } catch (error) {
359
+ const status = error instanceof HttpError ? error.status : 502
360
+ jsonResponse(res, status, {
361
+ error: error instanceof Error ? error.message : String(error),
362
+ })
363
+ }
364
+ }
365
+ }
366
+
367
+ function createSpeechStream() {
368
+ const clients = new Set()
369
+ return {
370
+ add(res) {
371
+ clients.add(res)
372
+ res.writeHead(200, {
373
+ 'content-type': 'text/event-stream; charset=utf-8',
374
+ 'cache-control': 'no-store',
375
+ connection: 'keep-alive',
376
+ })
377
+ res.write(': connected\n\n')
378
+ const timer = setInterval(() => {
379
+ try {
380
+ res.write(': ping\n\n')
381
+ } catch {
382
+ clearInterval(timer)
383
+ clients.delete(res)
384
+ }
385
+ }, 15000)
386
+ res.on('close', () => {
387
+ clearInterval(timer)
388
+ clients.delete(res)
389
+ })
390
+ },
391
+ broadcast(message) {
392
+ const payload = `data: ${JSON.stringify(message)}\n\n`
393
+ for (const res of clients) {
394
+ try {
395
+ res.write(payload)
396
+ } catch {
397
+ clients.delete(res)
398
+ }
399
+ }
400
+ },
401
+ close() {
402
+ for (const res of clients) {
403
+ try { res.end() } catch {}
404
+ }
405
+ clients.clear()
406
+ },
407
+ }
408
+ }
409
+
128
410
  async function readJsonBody(req, limit) {
129
411
  const chunks = []
130
412
  let bytes = 0
@@ -181,6 +463,14 @@ function resolveOptions(raw = {}) {
181
463
  empathyStyle: ['gentle', 'direct', 'humorous', 'concise'].includes(value.empathyStyle)
182
464
  ? value.empathyStyle
183
465
  : DEFAULTS.empathyStyle,
466
+ ttsEnabled: value.ttsEnabled === true,
467
+ ttsEngine: ['piper', 'edge', 'browser'].includes(value.ttsEngine) ? value.ttsEngine : DEFAULTS.ttsEngine,
468
+ ttsVoice: String(value.ttsVoice || DEFAULTS.ttsVoice).trim(),
469
+ ttsRate: String(value.ttsRate || DEFAULTS.ttsRate).trim(),
470
+ ttsPitch: String(value.ttsPitch || DEFAULTS.ttsPitch).trim(),
471
+ ttsCommand: String(value.ttsCommand || '').trim(),
472
+ ttsPiperModel: String(value.ttsPiperModel || '').trim(),
473
+ ttsPiperCommand: String(value.ttsPiperCommand || '').trim(),
184
474
  maxAudioBytes: Math.min(32 * 1024 * 1024, Math.max(256 * 1024, asNumber(value.maxAudioBytes, DEFAULTS.maxAudioBytes))),
185
475
  startupTimeoutMs: Math.min(600000, Math.max(10000, asNumber(value.startupTimeoutMs, DEFAULTS.startupTimeoutMs))),
186
476
  requestTimeoutMs: Math.min(600000, Math.max(10000, asNumber(value.requestTimeoutMs, DEFAULTS.requestTimeoutMs))),
@@ -194,6 +484,14 @@ function publicConfig(options, writable = false) {
194
484
  pythonPath: options.pythonPath || '',
195
485
  language: options.language || DEFAULTS.language,
196
486
  empathyStyle: options.empathyStyle || DEFAULTS.empathyStyle,
487
+ ttsEnabled: options.ttsEnabled === true,
488
+ ttsEngine: options.ttsEngine || DEFAULTS.ttsEngine,
489
+ ttsVoice: options.ttsVoice || DEFAULTS.ttsVoice,
490
+ ttsRate: options.ttsRate || DEFAULTS.ttsRate,
491
+ ttsPitch: options.ttsPitch || DEFAULTS.ttsPitch,
492
+ ttsCommand: options.ttsCommand || '',
493
+ ttsPiperModel: options.ttsPiperModel || '',
494
+ ttsPiperCommand: options.ttsPiperCommand || '',
197
495
  maxAudioBytes: options.maxAudioBytes,
198
496
  startupTimeoutMs: options.startupTimeoutMs,
199
497
  requestTimeoutMs: options.requestTimeoutMs,
@@ -227,6 +525,54 @@ function normalizeConfigPatch(value) {
227
525
  patch.empathyStyle = style
228
526
  continue
229
527
  }
528
+ if (key === 'ttsEnabled') {
529
+ if (typeof next !== 'boolean') throw new HttpError(400, 'ttsEnabled 必须是布尔值')
530
+ patch.ttsEnabled = next
531
+ continue
532
+ }
533
+ if (key === 'ttsEngine') {
534
+ const engine = String(next || '').trim().toLowerCase()
535
+ if (!['piper', 'edge', 'browser'].includes(engine)) {
536
+ throw new HttpError(400, 'ttsEngine 只能是 piper、edge 或 browser')
537
+ }
538
+ patch.ttsEngine = engine
539
+ continue
540
+ }
541
+ if (key === 'ttsVoice') {
542
+ const voice = String(next || '').trim()
543
+ if (!/^[A-Za-z]{2,3}-[A-Za-z0-9-]+Neural$/.test(voice)) {
544
+ throw new HttpError(400, 'ttsVoice 不是合法的 Edge TTS 音色')
545
+ }
546
+ patch.ttsVoice = voice
547
+ continue
548
+ }
549
+ if (key === 'ttsRate' || key === 'ttsPitch') {
550
+ const value = String(next || '').trim()
551
+ const pattern = key === 'ttsRate' ? /^[+-]\d{1,3}%$/ : /^[+-]\d{1,3}Hz$/
552
+ if (!pattern.test(value)) {
553
+ throw new HttpError(400, `${key} 格式应为 ${key === 'ttsRate' ? '+6%' : '+12Hz'}`)
554
+ }
555
+ patch[key] = value
556
+ continue
557
+ }
558
+ if (key === 'ttsCommand') {
559
+ if (typeof next !== 'string') throw new HttpError(400, 'ttsCommand 必须是字符串')
560
+ const text = next.trim()
561
+ if (text.length > 500 || /[\u0000-\u001f]/.test(text)) {
562
+ throw new HttpError(400, 'ttsCommand 不是合法的路径')
563
+ }
564
+ patch.ttsCommand = text
565
+ continue
566
+ }
567
+ if (key === 'ttsPiperModel' || key === 'ttsPiperCommand') {
568
+ if (typeof next !== 'string') throw new HttpError(400, `${key} 必须是字符串`)
569
+ const text = next.trim()
570
+ if (text.length > 500 || /[\u0000-\u001f]/.test(text)) {
571
+ throw new HttpError(400, `${key} 不是合法的路径`)
572
+ }
573
+ patch[key] = text
574
+ continue
575
+ }
230
576
  if (key === 'backendRoot' || key === 'pythonPath') {
231
577
  if (typeof next !== 'string') throw new HttpError(400, `${key} 必须是字符串`)
232
578
  const text = next.trim()
@@ -1049,6 +1395,7 @@ export function apply(ctx, rawConfig = {}) {
1049
1395
  lastCleanupAt: null,
1050
1396
  }
1051
1397
  const backend = new VoiceBackend(runtime.options, logger)
1398
+ const speechStream = createSpeechStream()
1052
1399
  let speechToText
1053
1400
  ctx.inject(['settings'], (settingsCtx) => {
1054
1401
  settingsService = settingsCtx.settings
@@ -1060,6 +1407,25 @@ export function apply(ctx, rawConfig = {}) {
1060
1407
  })
1061
1408
  })
1062
1409
  ctx.effect(() => () => backend.close())
1410
+ ctx.effect(() => () => speechStream.close())
1411
+
1412
+ ctx.inject(['sessions'], (sessionCtx) => {
1413
+ try {
1414
+ const off = sessionCtx.on?.('session/event', (session, event) => {
1415
+ if (event?.type !== 'assistant/message' || !runtime.options.ttsEnabled) return
1416
+ const text = cleanForSpeech(assistantTextFromEvent(event))
1417
+ if (!text) return
1418
+ speechStream.broadcast({
1419
+ kind: 'assistant',
1420
+ sessionId: String(session?.header?.id ?? session?.id ?? ''),
1421
+ text,
1422
+ })
1423
+ })
1424
+ if (typeof off === 'function') ctx.effect(() => off)
1425
+ } catch (error) {
1426
+ logger.warn?.('[dsh-dafeiyu-emotion] failed to subscribe to session events:', error)
1427
+ }
1428
+ })
1063
1429
 
1064
1430
  ctx.inject(['webServer'], (httpCtx) => {
1065
1431
  httpCtx.effect(
@@ -1086,6 +1452,10 @@ export function apply(ctx, rawConfig = {}) {
1086
1452
  enabled: options.enabled,
1087
1453
  language: options.language,
1088
1454
  empathyStyle: options.empathyStyle,
1455
+ ttsEnabled: options.ttsEnabled,
1456
+ ttsEngine: options.ttsEngine,
1457
+ ttsVoice: options.ttsVoice,
1458
+ ttsReady: ttsReady(options),
1089
1459
  engine: backendAvailable ? 'SenseVoice + emotion2vec' : 'DSH Speech + prosody',
1090
1460
  mode: !backendConfigured
1091
1461
  ? 'dsh-speech-prosody'
@@ -1154,19 +1524,52 @@ export function apply(ctx, rawConfig = {}) {
1154
1524
  }),
1155
1525
  'dsh-dafeiyu-emotion: 最近情绪上下文',
1156
1526
  )
1527
+ httpCtx.effect(
1528
+ () => httpCtx.webServer.register({
1529
+ kind: 'exact',
1530
+ path: TTS_ENDPOINT,
1531
+ handler: createTtsHandler(runtime),
1532
+ }),
1533
+ 'dsh-dafeiyu-emotion: 语音合成',
1534
+ )
1535
+ httpCtx.effect(
1536
+ () => httpCtx.webServer.register({
1537
+ kind: 'exact',
1538
+ path: EVENTS_ENDPOINT,
1539
+ handler(req, res) {
1540
+ if (!isLocalRequest(req)) {
1541
+ jsonResponse(res, 403, { error: '只允许本机访问' })
1542
+ return
1543
+ }
1544
+ if (req.method !== 'GET') {
1545
+ jsonResponse(res, 405, { error: 'method not allowed' })
1546
+ return
1547
+ }
1548
+ speechStream.add(res)
1549
+ },
1550
+ }),
1551
+ 'dsh-dafeiyu-emotion: 回复语音事件',
1552
+ )
1157
1553
  })
1158
1554
 
1159
1555
  }
1160
1556
 
1161
1557
  export {
1162
1558
  VoiceBackend,
1559
+ assistantTextFromEvent,
1163
1560
  buildFallbackAnalysis,
1164
1561
  buildPrompt,
1562
+ cleanForSpeech,
1165
1563
  createConfigHandler,
1166
1564
  decorateAnalysis,
1167
1565
  normalizeConfigPatch,
1168
1566
  plainConfig,
1169
1567
  publicConfig,
1568
+ resolveEdgeTts,
1569
+ resolvePiperCommand,
1570
+ resolvePiperModel,
1170
1571
  resolveOptions,
1572
+ synthesizePiper,
1573
+ ttsReady,
1171
1574
  transcribeWithSpeech,
1172
1575
  }