dsh-dafeiyu-emotion 0.3.0 → 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +9 -0
- package/README.md +31 -3
- package/cordis.patch.yml +8 -0
- package/docs/MARKETPLACE.md +1 -1
- package/docs/RELEASE-0.3.1.md +39 -0
- package/lib/client.js +191 -1
- package/package.json +2 -2
- package/src/index.js +404 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,14 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.3.1
|
|
4
|
+
|
|
5
|
+
- Added spoken assistant replies. The host now forwards `assistant/message` text over `/plugins/dsh-dafeiyu-emotion/events`; the client plays it with TTS.
|
|
6
|
+
- Added local Piper TTS (`zh_CN-huayan-medium`) as the default engine, with Edge TTS as an optional online engine and browser `speechSynthesis` as the final fallback.
|
|
7
|
+
- Added `ttsEnabled`, `ttsEngine`, `ttsVoice`, `ttsRate`, `ttsPitch` and `ttsPiperModel` settings, including a voice selector and model path field.
|
|
8
|
+
- Added `POST /plugins/dsh-dafeiyu-emotion/tts`, returning WAV for Piper and MP3 for Edge.
|
|
9
|
+
- Edge TTS failures and missing Piper models fall back to local browser speech instead of breaking the reply.
|
|
10
|
+
- Added TTS handler, assistant-text extraction, SSE broadcast and markdown-cleaning tests.
|
|
11
|
+
|
|
3
12
|
## 0.3.0
|
|
4
13
|
|
|
5
14
|
- Added `empathyStyle` with four reply strategies: gentle, direct, humorous and concise. The choice is persisted and injected into the emotion-aware prompt.
|
package/README.md
CHANGED
|
@@ -76,6 +76,7 @@
|
|
|
76
76
|
- 📈 多段语音的情绪时间线,可悬停查看每段文本、情绪和置信度;
|
|
77
77
|
- 🔐 设置页内的隐私与运行状态:最近音频大小、模型加载、worker PID、临时文件清理;
|
|
78
78
|
- 🔌 本机 `/context` 接口,其他插件可以读取最近一次情绪上下文;
|
|
79
|
+
- 🔊 大肥鱼回复语音:Piper 本地模型默认朗读,Edge / 浏览器语音可选;
|
|
79
80
|
- 🐋 大肥鱼主题的“鱼耳”入口,和普通对话共用同一个 composer;
|
|
80
81
|
- 🧠 情绪标签:喜悦 / 低落 / 生气 / 紧张 / 惊讶 / 厌烦 / 平静;
|
|
81
82
|
- 📈 额外给出 confidence、valence、arousal 和语气摘要;
|
|
@@ -115,13 +116,34 @@ dsh plugin --profile desktop add github:LiDu-china/dsh-dafeiyu-emotion
|
|
|
115
116
|
### 通过 Release tarball
|
|
116
117
|
|
|
117
118
|
```bash
|
|
118
|
-
curl -L -o dsh-dafeiyu-emotion-0.3.
|
|
119
|
-
https://github.com/LiDu-china/dsh-dafeiyu-emotion/releases/download/v0.3.
|
|
120
|
-
dsh plugin --profile desktop add "D:/path/to/dsh-dafeiyu-emotion-0.3.
|
|
119
|
+
curl -L -o dsh-dafeiyu-emotion-0.3.1.tgz \
|
|
120
|
+
https://github.com/LiDu-china/dsh-dafeiyu-emotion/releases/download/v0.3.1/dsh-dafeiyu-emotion-0.3.1.tgz
|
|
121
|
+
dsh plugin --profile desktop add "D:/path/to/dsh-dafeiyu-emotion-0.3.1.tgz"
|
|
121
122
|
```
|
|
122
123
|
|
|
123
124
|
> DSH Desktop 的 profile 由 Electron 应用管理;命令行安装请使用桌面应用自带的 `dsh` 命令。
|
|
124
125
|
|
|
126
|
+
## 大肥鱼语音回复
|
|
127
|
+
|
|
128
|
+
助手回复生成后,插件会通过本机事件流把文本交给 TTS,让大肥鱼把回复读出来。
|
|
129
|
+
|
|
130
|
+
默认使用 **Piper 本地模型**,音色是 `zh_CN-huayan-medium`。它不联网,延迟低,适合日常对话:
|
|
131
|
+
|
|
132
|
+
```bash
|
|
133
|
+
python -m pip install piper-tts
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
把模型放到默认目录:
|
|
137
|
+
|
|
138
|
+
```text
|
|
139
|
+
~/.dsh/dafeiyu-emotion/tts/piper/zh_CN-huayan-medium.onnx
|
|
140
|
+
~/.dsh/dafeiyu-emotion/tts/piper/zh_CN-huayan-medium.onnx.json
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
也可以在设置页填写自定义 Piper 模型路径。模型缺失或合成失败时,会自动回退到浏览器 `speechSynthesis`,不会让回复静默丢失。
|
|
144
|
+
|
|
145
|
+
如果更喜欢更自然的在线音色,可以把引擎切到 **Edge TTS**,默认音色 `zh-CN-XiaoyiNeural`。Edge 需要网络,且回复文本会发送到 Microsoft;如果网络或服务不可用,同样会回退到浏览器语音。
|
|
146
|
+
|
|
125
147
|
## 设置页与侧边栏
|
|
126
148
|
|
|
127
149
|
插件会把自己的设置卡片注册到 DSH 的插件设置页。打开:
|
|
@@ -190,6 +212,12 @@ dsh plugin --profile desktop add "D:/path/to/dsh-dafeiyu-emotion-0.3.0.tgz"
|
|
|
190
212
|
| `pythonPath` | 空 | 自定义 Python;空则自动寻找 `.venv-ser` |
|
|
191
213
|
| `language` | `zh` | 识别语言:`zh` / `en` / `auto` |
|
|
192
214
|
| `empathyStyle` | `gentle` | 共情策略:`gentle` / `direct` / `humorous` / `concise` |
|
|
215
|
+
| `ttsEnabled` | `true` | 是否朗读大肥鱼回复 |
|
|
216
|
+
| `ttsEngine` | `piper` | `piper` 本地 / `edge` 在线 / `browser` 系统语音 |
|
|
217
|
+
| `ttsVoice` | `zh-CN-XiaoyiNeural` | Edge TTS 音色 |
|
|
218
|
+
| `ttsRate` | `+6%` | Piper/Edge 语速 |
|
|
219
|
+
| `ttsPitch` | `+12Hz` | Edge 音调 |
|
|
220
|
+
| `ttsPiperModel` | 空 | 自定义 Piper 模型路径;空则用默认目录 |
|
|
193
221
|
| `maxAudioBytes` | 12 MB | 单次录音大小上限 |
|
|
194
222
|
| `startupTimeoutMs` | 180000 | 首次加载增强后端的最长等待 |
|
|
195
223
|
| `requestTimeoutMs` | 180000 | 单次分析请求超时 |
|
package/cordis.patch.yml
CHANGED
|
@@ -29,6 +29,14 @@
|
|
|
29
29
|
pythonPath: ""
|
|
30
30
|
language: zh
|
|
31
31
|
empathyStyle: gentle
|
|
32
|
+
ttsEnabled: true
|
|
33
|
+
ttsEngine: piper
|
|
34
|
+
ttsVoice: zh-CN-XiaoyiNeural
|
|
35
|
+
ttsRate: "+6%"
|
|
36
|
+
ttsPitch: "+12Hz"
|
|
37
|
+
ttsCommand: ""
|
|
38
|
+
ttsPiperModel: ""
|
|
39
|
+
ttsPiperCommand: ""
|
|
32
40
|
maxAudioBytes: 12582912
|
|
33
41
|
startupTimeoutMs: 180000
|
|
34
42
|
requestTimeoutMs: 180000
|
package/docs/MARKETPLACE.md
CHANGED
|
@@ -72,7 +72,7 @@ https://github.com/LiDu-china/dsh-dafeiyu-emotion/releases/download/v0.2.3/dsh-d
|
|
|
72
72
|
dsh plugin --profile desktop add dsh-dafeiyu-emotion
|
|
73
73
|
```
|
|
74
74
|
|
|
75
|
-
Published as `dsh-dafeiyu-emotion@0.3.
|
|
75
|
+
Published as `dsh-dafeiyu-emotion@0.3.1` on npm. The 1024Store entry refreshes its one-click install automatically. GitHub-source and Release-tarball installs remain available as fallbacks.
|
|
76
76
|
|
|
77
77
|
## Privacy
|
|
78
78
|
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
## 0.3.1
|
|
2
|
+
|
|
3
|
+
大肥鱼现在会说话了。
|
|
4
|
+
|
|
5
|
+
### 语音回复
|
|
6
|
+
|
|
7
|
+
- Host 监听 `assistant/message`,把回复文本通过 `/plugins/dsh-dafeiyu-emotion/events` 推给客户端。
|
|
8
|
+
- 客户端收到后朗读回复,默认使用 **Piper 本地模型** `zh_CN-huayan-medium`。
|
|
9
|
+
- 可选 **Edge TTS**(`zh-CN-XiaoyiNeural`,更自然,但需要网络),以及 **浏览器 speechSynthesis** 兜底。
|
|
10
|
+
- Piper/Edge 失败时自动回退浏览器语音,不会让回复静默丢失。
|
|
11
|
+
|
|
12
|
+
### 设置
|
|
13
|
+
|
|
14
|
+
- 朗读开关、合成引擎、Edge 音色、语速、音调、Piper 模型路径。
|
|
15
|
+
- 默认 `ttsEnabled: true`,Piper 模型不存在时自动使用浏览器语音。
|
|
16
|
+
|
|
17
|
+
### 本地模型
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
python -m pip install piper-tts
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
默认模型目录:
|
|
24
|
+
|
|
25
|
+
```text
|
|
26
|
+
~/.dsh/dafeiyu-emotion/tts/piper/zh_CN-huayan-medium.onnx
|
|
27
|
+
~/.dsh/dafeiyu-emotion/tts/piper/zh_CN-huayan-medium.onnx.json
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
可以在设置页填写自定义路径。
|
|
31
|
+
|
|
32
|
+
### 接口
|
|
33
|
+
|
|
34
|
+
```text
|
|
35
|
+
POST /plugins/dsh-dafeiyu-emotion/tts
|
|
36
|
+
GET /plugins/dsh-dafeiyu-emotion/events
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
`/tts` 返回 WAV(Piper)或 MP3(Edge),只允许本机访问。
|
package/lib/client.js
CHANGED
|
@@ -10,6 +10,8 @@ window.__ModuleLoader__.load({
|
|
|
10
10
|
const ANALYZE_ENDPOINT = '/plugins/dsh-dafeiyu-emotion/analyze'
|
|
11
11
|
const CONFIG_ENDPOINT = '/plugins/dsh-dafeiyu-emotion/config'
|
|
12
12
|
const HEALTH_ENDPOINT = '/plugins/dsh-dafeiyu-emotion/health'
|
|
13
|
+
const TTS_ENDPOINT = '/plugins/dsh-dafeiyu-emotion/tts'
|
|
14
|
+
const EVENTS_ENDPOINT = '/plugins/dsh-dafeiyu-emotion/events'
|
|
13
15
|
const MAX_RECORDING_MS = 30000
|
|
14
16
|
const PERMISSION_TIMEOUT_MS = 20000
|
|
15
17
|
const EMOTION_LABELS = {
|
|
@@ -823,6 +825,12 @@ window.__ModuleLoader__.load({
|
|
|
823
825
|
|| draft.pythonPath !== config.pythonPath
|
|
824
826
|
|| draft.language !== config.language
|
|
825
827
|
|| draft.empathyStyle !== config.empathyStyle
|
|
828
|
+
|| draft.ttsEnabled !== config.ttsEnabled
|
|
829
|
+
|| draft.ttsEngine !== config.ttsEngine
|
|
830
|
+
|| draft.ttsVoice !== config.ttsVoice
|
|
831
|
+
|| draft.ttsRate !== config.ttsRate
|
|
832
|
+
|| draft.ttsPitch !== config.ttsPitch
|
|
833
|
+
|| draft.ttsPiperModel !== config.ttsPiperModel
|
|
826
834
|
|| draft.maxAudioBytes !== config.maxAudioBytes
|
|
827
835
|
|| draft.startupTimeoutMs !== config.startupTimeoutMs
|
|
828
836
|
|| draft.requestTimeoutMs !== config.requestTimeoutMs
|
|
@@ -831,7 +839,7 @@ window.__ModuleLoader__.load({
|
|
|
831
839
|
const save = async () => {
|
|
832
840
|
if (!config || !draft || saving) return
|
|
833
841
|
const patch = {}
|
|
834
|
-
for (const key of ['enabled', 'backendRoot', 'pythonPath', 'language', 'empathyStyle', 'maxAudioBytes', 'startupTimeoutMs', 'requestTimeoutMs']) {
|
|
842
|
+
for (const key of ['enabled', 'backendRoot', 'pythonPath', 'language', 'empathyStyle', 'ttsEnabled', 'ttsEngine', 'ttsVoice', 'ttsRate', 'ttsPitch', 'ttsPiperModel', 'maxAudioBytes', 'startupTimeoutMs', 'requestTimeoutMs']) {
|
|
835
843
|
if (draft[key] !== config[key]) patch[key] = draft[key]
|
|
836
844
|
}
|
|
837
845
|
if (Object.keys(patch).length === 0) return
|
|
@@ -957,6 +965,7 @@ window.__ModuleLoader__.load({
|
|
|
957
965
|
event.currentTarget.style.borderColor = T.border
|
|
958
966
|
}
|
|
959
967
|
const numberStyle = { ...inputStyle, width: 96, textAlign: 'right', fontVariantNumeric: 'tabular-nums' }
|
|
968
|
+
const selectStyle = { ...inputStyle, width: isSidebar ? '100%' : 220, paddingRight: 26 }
|
|
960
969
|
const secondaryButton = {
|
|
961
970
|
height: 32,
|
|
962
971
|
padding: '0 11px',
|
|
@@ -994,6 +1003,15 @@ window.__ModuleLoader__.load({
|
|
|
994
1003
|
},
|
|
995
1004
|
}, value),
|
|
996
1005
|
)
|
|
1006
|
+
const voiceOptions = [
|
|
1007
|
+
['zh-CN-XiaoyiNeural', '小艺 · 活泼'],
|
|
1008
|
+
['zh-CN-XiaoxiaoNeural', '晓晓 · 温柔'],
|
|
1009
|
+
['zh-CN-YunxiNeural', '云希 · 男声'],
|
|
1010
|
+
['zh-CN-liaoning-XiaobeiNeural', '小北 · 东北'],
|
|
1011
|
+
]
|
|
1012
|
+
if (draft.ttsVoice && !voiceOptions.some(([value]) => value === draft.ttsVoice)) {
|
|
1013
|
+
voiceOptions.push([draft.ttsVoice, draft.ttsVoice])
|
|
1014
|
+
}
|
|
997
1015
|
|
|
998
1016
|
return h(Root, {
|
|
999
1017
|
'data-testid': 'dsh-dafeiyu-emotion-settings',
|
|
@@ -1235,6 +1253,84 @@ window.__ModuleLoader__.load({
|
|
|
1235
1253
|
onChange: (value) => update('empathyStyle', value),
|
|
1236
1254
|
})),
|
|
1237
1255
|
),
|
|
1256
|
+
h(SettingsSection, { title: '语音回复' },
|
|
1257
|
+
h(SettingsField, {
|
|
1258
|
+
icon: 'spark',
|
|
1259
|
+
label: '朗读大肥鱼回复',
|
|
1260
|
+
hint: '开启后,助手回复会自动朗读。Piper 本地模型最稳;Edge 需要网络;浏览器引擎留在本机。',
|
|
1261
|
+
}, h(SettingsSwitch, {
|
|
1262
|
+
checked: draft.ttsEnabled === true,
|
|
1263
|
+
disabled,
|
|
1264
|
+
label: '朗读大肥鱼回复',
|
|
1265
|
+
onChange: (value) => update('ttsEnabled', value),
|
|
1266
|
+
})),
|
|
1267
|
+
h(SettingsField, {
|
|
1268
|
+
icon: 'spark',
|
|
1269
|
+
label: '合成引擎',
|
|
1270
|
+
hint: 'Piper 使用本地中文模型;Edge 音色更自然但可能被网络限制;浏览器使用系统语音。',
|
|
1271
|
+
}, h(SettingsSegmented, {
|
|
1272
|
+
value: draft.ttsEngine || 'piper',
|
|
1273
|
+
label: '合成引擎',
|
|
1274
|
+
disabled,
|
|
1275
|
+
options: [
|
|
1276
|
+
{ value: 'piper', label: 'Piper' },
|
|
1277
|
+
{ value: 'edge', label: 'Edge' },
|
|
1278
|
+
{ value: 'browser', label: '浏览器' },
|
|
1279
|
+
],
|
|
1280
|
+
onChange: (value) => update('ttsEngine', value),
|
|
1281
|
+
})),
|
|
1282
|
+
h(SettingsField, {
|
|
1283
|
+
icon: 'spark',
|
|
1284
|
+
label: '音色',
|
|
1285
|
+
hint: 'Edge TTS 的中文女声/男声;浏览器引擎使用系统默认音色。',
|
|
1286
|
+
wide: true,
|
|
1287
|
+
}, h('select', {
|
|
1288
|
+
value: draft.ttsVoice || 'zh-CN-XiaoyiNeural',
|
|
1289
|
+
disabled: disabled || draft.ttsEngine !== 'edge',
|
|
1290
|
+
'aria-label': 'Edge TTS 音色',
|
|
1291
|
+
onChange: (event) => update('ttsVoice', event.target.value),
|
|
1292
|
+
style: selectStyle,
|
|
1293
|
+
}, ...voiceOptions.map(([value, label]) => h('option', { key: value, value }, label)))),
|
|
1294
|
+
draft.ttsEngine === 'piper' || draft.ttsEngine === 'edge'
|
|
1295
|
+
? h(SettingsField, {
|
|
1296
|
+
icon: 'folder',
|
|
1297
|
+
label: 'Piper 模型',
|
|
1298
|
+
hint: '留空使用 ~/.dsh/dafeiyu-emotion/tts/piper/zh_CN-huayan-medium.onnx。',
|
|
1299
|
+
wide: true,
|
|
1300
|
+
}, h('input', {
|
|
1301
|
+
type: 'text',
|
|
1302
|
+
value: draft.ttsPiperModel || '',
|
|
1303
|
+
disabled: disabled || draft.ttsEngine !== 'piper',
|
|
1304
|
+
spellCheck: false,
|
|
1305
|
+
placeholder: '留空使用默认 Piper 模型',
|
|
1306
|
+
'aria-label': 'Piper 模型路径',
|
|
1307
|
+
onChange: (event) => update('ttsPiperModel', event.target.value),
|
|
1308
|
+
style: { ...inputStyle, width: '100%' },
|
|
1309
|
+
}))
|
|
1310
|
+
: null,
|
|
1311
|
+
h(SettingsField, {
|
|
1312
|
+
icon: 'spark',
|
|
1313
|
+
label: '语速 / 音调',
|
|
1314
|
+
hint: '语速 +6% 适用于 Piper/Edge;音调 +12Hz 仅 Edge。',
|
|
1315
|
+
}, h('div', { style: { display: 'flex', alignItems: 'center', gap: 6, flexWrap: 'wrap' } },
|
|
1316
|
+
h('input', {
|
|
1317
|
+
type: 'text',
|
|
1318
|
+
value: draft.ttsRate || '+6%',
|
|
1319
|
+
disabled: disabled || draft.ttsEngine === 'browser',
|
|
1320
|
+
'aria-label': '语速',
|
|
1321
|
+
onChange: (event) => update('ttsRate', event.target.value),
|
|
1322
|
+
style: { ...inputStyle, width: 76 },
|
|
1323
|
+
}),
|
|
1324
|
+
h('input', {
|
|
1325
|
+
type: 'text',
|
|
1326
|
+
value: draft.ttsPitch || '+12Hz',
|
|
1327
|
+
disabled: disabled || draft.ttsEngine !== 'edge',
|
|
1328
|
+
'aria-label': '音调',
|
|
1329
|
+
onChange: (event) => update('ttsPitch', event.target.value),
|
|
1330
|
+
style: { ...inputStyle, width: 86 },
|
|
1331
|
+
}),
|
|
1332
|
+
)),
|
|
1333
|
+
),
|
|
1238
1334
|
h(SettingsSection, { title: '增强后端', hint: '可选' },
|
|
1239
1335
|
h(SettingsField, {
|
|
1240
1336
|
icon: 'folder',
|
|
@@ -1449,6 +1545,13 @@ window.__ModuleLoader__.load({
|
|
|
1449
1545
|
const [sending, setSending] = useState(false)
|
|
1450
1546
|
const [pluginEnabled, setPluginEnabled] = useState(null)
|
|
1451
1547
|
const [language, setLanguage] = useState('zh')
|
|
1548
|
+
const [ttsConfig, setTtsConfig] = useState({
|
|
1549
|
+
enabled: false,
|
|
1550
|
+
engine: 'piper',
|
|
1551
|
+
voice: 'zh-CN-XiaoyiNeural',
|
|
1552
|
+
rate: '+6%',
|
|
1553
|
+
pitch: '+12Hz',
|
|
1554
|
+
})
|
|
1452
1555
|
const recorder = useRef(null)
|
|
1453
1556
|
const abort = useRef(null)
|
|
1454
1557
|
const timer = useRef(null)
|
|
@@ -1458,6 +1561,10 @@ window.__ModuleLoader__.load({
|
|
|
1458
1561
|
const levelsRef = useRef([])
|
|
1459
1562
|
const shortcutAction = useRef(null)
|
|
1460
1563
|
const shortcutRegistration = useRef(null)
|
|
1564
|
+
const ttsRef = useRef(ttsConfig)
|
|
1565
|
+
const audioRef = useRef(null)
|
|
1566
|
+
const speechGeneration = useRef(0)
|
|
1567
|
+
ttsRef.current = ttsConfig
|
|
1461
1568
|
const deliveryGate = useRef(null)
|
|
1462
1569
|
if (!deliveryGate.current) deliveryGate.current = createDeliveryGate()
|
|
1463
1570
|
const useInputSnapshot = typeof useInput === 'function' ? useInput : useNoInput
|
|
@@ -1734,12 +1841,95 @@ window.__ModuleLoader__.load({
|
|
|
1734
1841
|
shortcutRegistration.current?.touch?.()
|
|
1735
1842
|
}
|
|
1736
1843
|
|
|
1844
|
+
function speakBrowser(text) {
|
|
1845
|
+
const synth = window.speechSynthesis
|
|
1846
|
+
if (!synth || typeof SpeechSynthesisUtterance === 'undefined') return false
|
|
1847
|
+
synth.cancel()
|
|
1848
|
+
const utterance = new SpeechSynthesisUtterance(text)
|
|
1849
|
+
utterance.lang = 'zh-CN'
|
|
1850
|
+
utterance.rate = 1.06
|
|
1851
|
+
utterance.pitch = 1.08
|
|
1852
|
+
synth.speak(utterance)
|
|
1853
|
+
return true
|
|
1854
|
+
}
|
|
1855
|
+
|
|
1856
|
+
async function speakReply(text) {
|
|
1857
|
+
const config = ttsRef.current
|
|
1858
|
+
if (!config?.enabled || !text) return false
|
|
1859
|
+
const run = ++speechGeneration.current
|
|
1860
|
+
if (config.engine === 'browser') return speakBrowser(text)
|
|
1861
|
+
try {
|
|
1862
|
+
const response = await fetch(TTS_ENDPOINT, {
|
|
1863
|
+
method: 'POST',
|
|
1864
|
+
headers: { 'content-type': 'application/json' },
|
|
1865
|
+
body: JSON.stringify({
|
|
1866
|
+
text,
|
|
1867
|
+
voice: config.voice,
|
|
1868
|
+
rate: config.rate,
|
|
1869
|
+
pitch: config.pitch,
|
|
1870
|
+
}),
|
|
1871
|
+
})
|
|
1872
|
+
if (!response.ok) {
|
|
1873
|
+
const payload = await response.json().catch(() => ({}))
|
|
1874
|
+
throw new Error(payload.error || `TTS 失败(${response.status})`)
|
|
1875
|
+
}
|
|
1876
|
+
const blob = await response.blob()
|
|
1877
|
+
if (run !== speechGeneration.current) return false
|
|
1878
|
+
const url = URL.createObjectURL(blob)
|
|
1879
|
+
audioRef.current?.pause?.()
|
|
1880
|
+
const audio = new Audio(url)
|
|
1881
|
+
audioRef.current = audio
|
|
1882
|
+
audio.onended = () => {
|
|
1883
|
+
URL.revokeObjectURL(url)
|
|
1884
|
+
if (audioRef.current === audio) audioRef.current = null
|
|
1885
|
+
}
|
|
1886
|
+
try {
|
|
1887
|
+
await audio.play()
|
|
1888
|
+
return true
|
|
1889
|
+
} catch {
|
|
1890
|
+
URL.revokeObjectURL(url)
|
|
1891
|
+
return speakBrowser(text)
|
|
1892
|
+
}
|
|
1893
|
+
} catch {
|
|
1894
|
+
return speakBrowser(text)
|
|
1895
|
+
}
|
|
1896
|
+
}
|
|
1897
|
+
|
|
1898
|
+
useEffect(() => {
|
|
1899
|
+
if (typeof EventSource === 'undefined') return undefined
|
|
1900
|
+
const source = new EventSource(EVENTS_ENDPOINT)
|
|
1901
|
+
source.onmessage = (event) => {
|
|
1902
|
+
try {
|
|
1903
|
+
const message = JSON.parse(event.data)
|
|
1904
|
+
if (message?.kind !== 'assistant' || !message.text) return
|
|
1905
|
+
if (message.sessionId && sessionId && message.sessionId !== sessionId) return
|
|
1906
|
+
void speakReply(message.text)
|
|
1907
|
+
} catch {}
|
|
1908
|
+
}
|
|
1909
|
+
return () => source.close()
|
|
1910
|
+
}, [sessionId])
|
|
1911
|
+
|
|
1912
|
+
useEffect(() => () => {
|
|
1913
|
+
speechGeneration.current += 1
|
|
1914
|
+
audioRef.current?.pause?.()
|
|
1915
|
+
window.speechSynthesis?.cancel?.()
|
|
1916
|
+
}, [])
|
|
1917
|
+
|
|
1737
1918
|
useEffect(() => {
|
|
1738
1919
|
let active = true
|
|
1739
1920
|
const applyConfig = (next) => {
|
|
1740
1921
|
if (!active || !next) return
|
|
1741
1922
|
setPluginEnabled(next.enabled !== false)
|
|
1742
1923
|
if (next.language) setLanguage(next.language)
|
|
1924
|
+
setTtsConfig({
|
|
1925
|
+
enabled: next.ttsEnabled === true,
|
|
1926
|
+
engine: next.ttsEngine === 'browser'
|
|
1927
|
+
? 'browser'
|
|
1928
|
+
: next.ttsEngine === 'piper' ? 'piper' : 'edge',
|
|
1929
|
+
voice: next.ttsVoice || 'zh-CN-XiaoyiNeural',
|
|
1930
|
+
rate: next.ttsRate || '+6%',
|
|
1931
|
+
pitch: next.ttsPitch || '+12Hz',
|
|
1932
|
+
})
|
|
1743
1933
|
}
|
|
1744
1934
|
fetch(CONFIG_ENDPOINT, { cache: 'no-store' })
|
|
1745
1935
|
.then((response) => (response.ok ? response.json() : null))
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dsh-dafeiyu-emotion",
|
|
3
|
-
"version": "0.3.
|
|
3
|
+
"version": "0.3.1",
|
|
4
4
|
"description": "大肥鱼语音情绪耳朵:听语气、识别情绪,并让 DSH 按对方当下的状态自然回复。 · Voice + emotion-aware companion for DeepSeek Harness.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "src/index.js",
|
|
@@ -61,7 +61,7 @@
|
|
|
61
61
|
},
|
|
62
62
|
"scripts": {
|
|
63
63
|
"check": "node --check src/index.js && node --check lib/client.js",
|
|
64
|
-
"test": "node --check src/index.js && node --check lib/client.js && node --test scripts/test-config-schema.mjs scripts/test-config-handler.mjs scripts/test-analyze-handler.mjs scripts/test-backend-cancel.mjs scripts/test-delivery-policy.mjs scripts/test-shortcut-registration.mjs scripts/test-client-registration.mjs"
|
|
64
|
+
"test": "node --check src/index.js && node --check lib/client.js && node --test scripts/test-config-schema.mjs scripts/test-config-handler.mjs scripts/test-analyze-handler.mjs scripts/test-backend-cancel.mjs scripts/test-tts-handler.mjs scripts/test-delivery-policy.mjs scripts/test-shortcut-registration.mjs scripts/test-client-registration.mjs"
|
|
65
65
|
},
|
|
66
66
|
"dependencies": {
|
|
67
67
|
"@deepseek-ai/schemastery": "3.18.5",
|
package/src/index.js
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import { spawn } from 'node:child_process'
|
|
2
2
|
import { existsSync } from 'node:fs'
|
|
3
|
-
import { mkdtemp, rm, writeFile } from 'node:fs/promises'
|
|
3
|
+
import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'
|
|
4
4
|
import { createInterface } from 'node:readline'
|
|
5
5
|
import { tmpdir } from 'node:os'
|
|
6
6
|
import { dirname, join, resolve } from 'node:path'
|
|
@@ -14,6 +14,8 @@ const ANALYZE_ENDPOINT = '/plugins/dsh-dafeiyu-emotion/analyze'
|
|
|
14
14
|
const HEALTH_ENDPOINT = '/plugins/dsh-dafeiyu-emotion/health'
|
|
15
15
|
const CONFIG_ENDPOINT = '/plugins/dsh-dafeiyu-emotion/config'
|
|
16
16
|
const CONTEXT_ENDPOINT = '/plugins/dsh-dafeiyu-emotion/context'
|
|
17
|
+
const TTS_ENDPOINT = '/plugins/dsh-dafeiyu-emotion/tts'
|
|
18
|
+
const EVENTS_ENDPOINT = '/plugins/dsh-dafeiyu-emotion/events'
|
|
17
19
|
const SETTINGS_NAMESPACE = 'dsh-dafeiyu-emotion'
|
|
18
20
|
|
|
19
21
|
const here = dirname(fileURLToPath(import.meta.url))
|
|
@@ -28,6 +30,14 @@ const DEFAULTS = Object.freeze({
|
|
|
28
30
|
startupTimeoutMs: 180000,
|
|
29
31
|
requestTimeoutMs: 180000,
|
|
30
32
|
empathyStyle: 'gentle',
|
|
33
|
+
ttsEnabled: true,
|
|
34
|
+
ttsEngine: 'piper',
|
|
35
|
+
ttsVoice: 'zh-CN-XiaoyiNeural',
|
|
36
|
+
ttsRate: '+6%',
|
|
37
|
+
ttsPitch: '+12Hz',
|
|
38
|
+
ttsCommand: '',
|
|
39
|
+
ttsPiperModel: '',
|
|
40
|
+
ttsPiperCommand: '',
|
|
31
41
|
})
|
|
32
42
|
|
|
33
43
|
const CONFIG_FIELDS = new Set(Object.keys(DEFAULTS))
|
|
@@ -69,6 +79,18 @@ export const Config = Schema.object({
|
|
|
69
79
|
Schema.const('humorous').description('幽默'),
|
|
70
80
|
Schema.const('concise').description('简洁'),
|
|
71
81
|
]).default('gentle').description('共情策略').volatile(),
|
|
82
|
+
ttsEnabled: Schema.boolean().default(true).description('朗读大肥鱼回复').volatile(),
|
|
83
|
+
ttsEngine: Schema.union([
|
|
84
|
+
Schema.const('piper').description('Piper 本地模型(推荐)'),
|
|
85
|
+
Schema.const('edge').description('Edge TTS(更好听,需要网络)'),
|
|
86
|
+
Schema.const('browser').description('浏览器本地语音'),
|
|
87
|
+
]).default('piper').description('语音合成引擎').volatile(),
|
|
88
|
+
ttsVoice: Schema.string().default('zh-CN-XiaoyiNeural').description('Edge TTS 音色').volatile(),
|
|
89
|
+
ttsRate: Schema.string().default('+6%').description('语速').volatile(),
|
|
90
|
+
ttsPitch: Schema.string().default('+12Hz').description('音调').volatile(),
|
|
91
|
+
ttsCommand: Schema.string().default('').description('edge-tts 可执行文件路径').volatile(),
|
|
92
|
+
ttsPiperModel: Schema.string().default('').description('Piper 模型路径').volatile(),
|
|
93
|
+
ttsPiperCommand: Schema.string().default('').description('Piper 可执行文件路径').volatile(),
|
|
72
94
|
}).description('大肥鱼语音情绪耳朵')
|
|
73
95
|
|
|
74
96
|
const EMOTION_ZH = Object.freeze({
|
|
@@ -125,6 +147,266 @@ function jsonResponse(res, status, body) {
|
|
|
125
147
|
res.end(payload)
|
|
126
148
|
}
|
|
127
149
|
|
|
150
|
+
function resolveEdgeTts(options) {
|
|
151
|
+
const explicit = String(options?.ttsCommand || '').trim()
|
|
152
|
+
if (explicit) return explicit
|
|
153
|
+
const fromEnv = String(process.env.EDGE_TTS_PATH || '').trim()
|
|
154
|
+
if (fromEnv) return fromEnv
|
|
155
|
+
const local = process.env.LOCALAPPDATA
|
|
156
|
+
const roaming = process.env.APPDATA
|
|
157
|
+
const candidates = [
|
|
158
|
+
local && join(local, 'Programs', 'Python', 'Python313', 'Scripts', 'edge-tts.exe'),
|
|
159
|
+
local && join(local, 'Programs', 'Python', 'Python312', 'Scripts', 'edge-tts.exe'),
|
|
160
|
+
roaming && join(roaming, 'Python', 'Python313', 'Scripts', 'edge-tts.exe'),
|
|
161
|
+
'edge-tts',
|
|
162
|
+
].filter(Boolean)
|
|
163
|
+
return candidates.find((candidate) => candidate === 'edge-tts' || existsSync(candidate)) || 'edge-tts'
|
|
164
|
+
}
|
|
165
|
+
|
|
166
|
+
function defaultPiperModel() {
|
|
167
|
+
const home = process.env.USERPROFILE || process.env.HOME || ''
|
|
168
|
+
return join(home, '.dsh', 'dafeiyu-emotion', 'tts', 'piper', 'zh_CN-huayan-medium.onnx')
|
|
169
|
+
}
|
|
170
|
+
|
|
171
|
+
function resolvePiperModel(options) {
|
|
172
|
+
return String(options?.ttsPiperModel || process.env.DSH_DAFEIYU_PIPER_MODEL || defaultPiperModel()).trim()
|
|
173
|
+
}
|
|
174
|
+
|
|
175
|
+
function resolvePiperCommand(options) {
|
|
176
|
+
const explicit = String(options?.ttsPiperCommand || process.env.PIPER_PATH || '').trim()
|
|
177
|
+
if (explicit) return explicit
|
|
178
|
+
const local = process.env.LOCALAPPDATA
|
|
179
|
+
const candidates = [
|
|
180
|
+
local && join(local, 'Programs', 'Python', 'Python313', 'Scripts', 'piper.exe'),
|
|
181
|
+
local && join(local, 'Programs', 'Python', 'Python312', 'Scripts', 'piper.exe'),
|
|
182
|
+
'piper',
|
|
183
|
+
].filter(Boolean)
|
|
184
|
+
return candidates.find((candidate) => candidate === 'piper' || existsSync(candidate)) || 'piper'
|
|
185
|
+
}
|
|
186
|
+
|
|
187
|
+
async function synthesizePiper(text, options) {
|
|
188
|
+
const model = resolvePiperModel(options)
|
|
189
|
+
const command = resolvePiperCommand(options)
|
|
190
|
+
if (!existsSync(model)) throw new Error(`Piper 模型不存在:${model}`)
|
|
191
|
+
const config = `${model}.json`
|
|
192
|
+
if (!existsSync(config)) throw new Error(`Piper 模型配置不存在:${config}`)
|
|
193
|
+
const dir = await mkdtemp(join(tmpdir(), 'dsh-dafeiyu-tts-'))
|
|
194
|
+
const input = join(dir, 'input.txt')
|
|
195
|
+
const out = join(dir, 'reply.wav')
|
|
196
|
+
try {
|
|
197
|
+
await writeFile(input, text, 'utf8')
|
|
198
|
+
const speed = Number.parseFloat(String(options.ttsRate || '+0%').replace('%', ''))
|
|
199
|
+
const lengthScale = Number.isFinite(speed) && speed !== 0 ? 1 / (1 + speed / 100) : 1
|
|
200
|
+
await new Promise((resolvePromise, rejectPromise) => {
|
|
201
|
+
const proc = spawn(command, [
|
|
202
|
+
'-m', model,
|
|
203
|
+
'-c', config,
|
|
204
|
+
'-i', input,
|
|
205
|
+
'-f', out,
|
|
206
|
+
'--length-scale', String(Math.max(0.7, Math.min(1.4, lengthScale))),
|
|
207
|
+
], {
|
|
208
|
+
windowsHide: true,
|
|
209
|
+
env: { ...process.env, PYTHONUTF8: '1', PYTHONIOENCODING: 'utf-8' },
|
|
210
|
+
})
|
|
211
|
+
let stderr = ''
|
|
212
|
+
let settled = false
|
|
213
|
+
const timer = setTimeout(() => {
|
|
214
|
+
if (settled) return
|
|
215
|
+
settled = true
|
|
216
|
+
try { proc.kill() } catch {}
|
|
217
|
+
rejectPromise(new Error('Piper 合成超时'))
|
|
218
|
+
}, 30000)
|
|
219
|
+
proc.stderr?.on('data', (chunk) => { stderr += chunk.toString('utf8') })
|
|
220
|
+
proc.on('error', (error) => {
|
|
221
|
+
if (settled) return
|
|
222
|
+
settled = true
|
|
223
|
+
clearTimeout(timer)
|
|
224
|
+
rejectPromise(error)
|
|
225
|
+
})
|
|
226
|
+
proc.on('exit', (code) => {
|
|
227
|
+
if (settled) return
|
|
228
|
+
settled = true
|
|
229
|
+
clearTimeout(timer)
|
|
230
|
+
if (code === 0) resolvePromise()
|
|
231
|
+
else rejectPromise(new Error(stderr.trim() || `piper 退出(code=${code})`))
|
|
232
|
+
})
|
|
233
|
+
})
|
|
234
|
+
return await readFile(out)
|
|
235
|
+
} finally {
|
|
236
|
+
await rm(dir, { recursive: true, force: true }).catch(() => {})
|
|
237
|
+
}
|
|
238
|
+
}
|
|
239
|
+
|
|
240
|
+
function ttsReady(options) {
|
|
241
|
+
if (!options.ttsEnabled) return false
|
|
242
|
+
if (options.ttsEngine === 'browser') return true
|
|
243
|
+
if (options.ttsEngine === 'piper') return existsSync(resolvePiperModel(options))
|
|
244
|
+
return existsSync(resolveEdgeTts(options))
|
|
245
|
+
}
|
|
246
|
+
|
|
247
|
+
function assistantTextFromEvent(event) {
|
|
248
|
+
const content = event?.data?.message?.content
|
|
249
|
+
if (!Array.isArray(content)) return ''
|
|
250
|
+
return content
|
|
251
|
+
.filter((block) => block?.type === 'text' && typeof block.text === 'string')
|
|
252
|
+
.map((block) => block.text)
|
|
253
|
+
.join('\n')
|
|
254
|
+
}
|
|
255
|
+
|
|
256
|
+
function cleanForSpeech(text) {
|
|
257
|
+
return String(text || '')
|
|
258
|
+
.replace(/```[\s\S]*?```/g, ' ')
|
|
259
|
+
.replace(/`[^`]*`/g, ' ')
|
|
260
|
+
.replace(/!\[[^\]]*\]\([^)]*\)/g, ' ')
|
|
261
|
+
.replace(/\[([^\]]+)\]\([^)]*\)/g, '$1')
|
|
262
|
+
.replace(/^#{1,6}\s+/gm, '')
|
|
263
|
+
.replace(/[*_~>#]/g, ' ')
|
|
264
|
+
.replace(/\s+/g, ' ')
|
|
265
|
+
.trim()
|
|
266
|
+
.slice(0, 800)
|
|
267
|
+
}
|
|
268
|
+
|
|
269
|
+
async function synthesizeEdge(text, options) {
|
|
270
|
+
const command = resolveEdgeTts(options)
|
|
271
|
+
const dir = await mkdtemp(join(tmpdir(), 'dsh-dafeiyu-tts-'))
|
|
272
|
+
const out = join(dir, 'reply.mp3')
|
|
273
|
+
try {
|
|
274
|
+
await new Promise((resolvePromise, rejectPromise) => {
|
|
275
|
+
const proc = spawn(command, [
|
|
276
|
+
'--voice', options.ttsVoice,
|
|
277
|
+
'--rate', options.ttsRate,
|
|
278
|
+
'--pitch', options.ttsPitch,
|
|
279
|
+
'--text', text,
|
|
280
|
+
'--write-media', out,
|
|
281
|
+
], { windowsHide: true })
|
|
282
|
+
let stderr = ''
|
|
283
|
+
let settled = false
|
|
284
|
+
const timer = setTimeout(() => {
|
|
285
|
+
if (settled) return
|
|
286
|
+
settled = true
|
|
287
|
+
try { proc.kill() } catch {}
|
|
288
|
+
rejectPromise(new Error('TTS 合成超时'))
|
|
289
|
+
}, 25000)
|
|
290
|
+
proc.stderr?.on('data', (chunk) => { stderr += chunk.toString('utf8') })
|
|
291
|
+
proc.on('error', (error) => {
|
|
292
|
+
if (settled) return
|
|
293
|
+
settled = true
|
|
294
|
+
clearTimeout(timer)
|
|
295
|
+
rejectPromise(error)
|
|
296
|
+
})
|
|
297
|
+
proc.on('exit', (code) => {
|
|
298
|
+
if (settled) return
|
|
299
|
+
settled = true
|
|
300
|
+
clearTimeout(timer)
|
|
301
|
+
if (code === 0) resolvePromise()
|
|
302
|
+
else rejectPromise(new Error(stderr.trim() || `edge-tts 退出(code=${code})`))
|
|
303
|
+
})
|
|
304
|
+
})
|
|
305
|
+
return await readFile(out)
|
|
306
|
+
} finally {
|
|
307
|
+
await rm(dir, { recursive: true, force: true }).catch(() => {})
|
|
308
|
+
}
|
|
309
|
+
}
|
|
310
|
+
|
|
311
|
+
function createTtsHandler(runtime) {
|
|
312
|
+
return async (req, res) => {
|
|
313
|
+
if (!isLocalRequest(req)) {
|
|
314
|
+
jsonResponse(res, 403, { error: '只允许本机访问' })
|
|
315
|
+
return
|
|
316
|
+
}
|
|
317
|
+
if (req.method !== 'POST') {
|
|
318
|
+
jsonResponse(res, 405, { error: 'method not allowed' })
|
|
319
|
+
return
|
|
320
|
+
}
|
|
321
|
+
if (!runtime.options.ttsEnabled) {
|
|
322
|
+
jsonResponse(res, 503, { error: '语音回复已关闭' })
|
|
323
|
+
return
|
|
324
|
+
}
|
|
325
|
+
try {
|
|
326
|
+
const body = await readJsonBody(req, 16 * 1024)
|
|
327
|
+
const text = cleanForSpeech(body?.text)
|
|
328
|
+
if (!text) throw new HttpError(400, '没有可朗读的文本')
|
|
329
|
+
if (runtime.options.ttsEngine === 'browser') {
|
|
330
|
+
throw new HttpError(400, '浏览器语音引擎在客户端本地运行,不需要调用 Host TTS')
|
|
331
|
+
}
|
|
332
|
+
const voice = String(body?.voice || runtime.options.ttsVoice)
|
|
333
|
+
const rate = String(body?.rate || runtime.options.ttsRate)
|
|
334
|
+
const pitch = String(body?.pitch || runtime.options.ttsPitch)
|
|
335
|
+
if (!/^[+-]\d{1,3}%$/.test(rate)) throw new HttpError(400, '语速格式不合法')
|
|
336
|
+
let audio
|
|
337
|
+
let contentType
|
|
338
|
+
if (runtime.options.ttsEngine === 'piper') {
|
|
339
|
+
audio = await synthesizePiper(text, { ...runtime.options, ttsRate: rate })
|
|
340
|
+
contentType = 'audio/wav'
|
|
341
|
+
} else {
|
|
342
|
+
if (!/^[A-Za-z]{2,3}-[A-Za-z0-9-]+Neural$/.test(voice)) throw new HttpError(400, '音色不合法')
|
|
343
|
+
if (!/^[+-]\d{1,3}Hz$/.test(pitch)) throw new HttpError(400, '音调格式不合法')
|
|
344
|
+
audio = await synthesizeEdge(text, {
|
|
345
|
+
...runtime.options,
|
|
346
|
+
ttsVoice: voice,
|
|
347
|
+
ttsRate: rate,
|
|
348
|
+
ttsPitch: pitch,
|
|
349
|
+
})
|
|
350
|
+
contentType = 'audio/mpeg'
|
|
351
|
+
}
|
|
352
|
+
res.writeHead(200, {
|
|
353
|
+
'content-type': contentType,
|
|
354
|
+
'cache-control': 'no-store',
|
|
355
|
+
'content-length': audio.length,
|
|
356
|
+
})
|
|
357
|
+
res.end(audio)
|
|
358
|
+
} catch (error) {
|
|
359
|
+
const status = error instanceof HttpError ? error.status : 502
|
|
360
|
+
jsonResponse(res, status, {
|
|
361
|
+
error: error instanceof Error ? error.message : String(error),
|
|
362
|
+
})
|
|
363
|
+
}
|
|
364
|
+
}
|
|
365
|
+
}
|
|
366
|
+
|
|
367
|
+
function createSpeechStream() {
|
|
368
|
+
const clients = new Set()
|
|
369
|
+
return {
|
|
370
|
+
add(res) {
|
|
371
|
+
clients.add(res)
|
|
372
|
+
res.writeHead(200, {
|
|
373
|
+
'content-type': 'text/event-stream; charset=utf-8',
|
|
374
|
+
'cache-control': 'no-store',
|
|
375
|
+
connection: 'keep-alive',
|
|
376
|
+
})
|
|
377
|
+
res.write(': connected\n\n')
|
|
378
|
+
const timer = setInterval(() => {
|
|
379
|
+
try {
|
|
380
|
+
res.write(': ping\n\n')
|
|
381
|
+
} catch {
|
|
382
|
+
clearInterval(timer)
|
|
383
|
+
clients.delete(res)
|
|
384
|
+
}
|
|
385
|
+
}, 15000)
|
|
386
|
+
res.on('close', () => {
|
|
387
|
+
clearInterval(timer)
|
|
388
|
+
clients.delete(res)
|
|
389
|
+
})
|
|
390
|
+
},
|
|
391
|
+
broadcast(message) {
|
|
392
|
+
const payload = `data: ${JSON.stringify(message)}\n\n`
|
|
393
|
+
for (const res of clients) {
|
|
394
|
+
try {
|
|
395
|
+
res.write(payload)
|
|
396
|
+
} catch {
|
|
397
|
+
clients.delete(res)
|
|
398
|
+
}
|
|
399
|
+
}
|
|
400
|
+
},
|
|
401
|
+
close() {
|
|
402
|
+
for (const res of clients) {
|
|
403
|
+
try { res.end() } catch {}
|
|
404
|
+
}
|
|
405
|
+
clients.clear()
|
|
406
|
+
},
|
|
407
|
+
}
|
|
408
|
+
}
|
|
409
|
+
|
|
128
410
|
async function readJsonBody(req, limit) {
|
|
129
411
|
const chunks = []
|
|
130
412
|
let bytes = 0
|
|
@@ -181,6 +463,14 @@ function resolveOptions(raw = {}) {
|
|
|
181
463
|
empathyStyle: ['gentle', 'direct', 'humorous', 'concise'].includes(value.empathyStyle)
|
|
182
464
|
? value.empathyStyle
|
|
183
465
|
: DEFAULTS.empathyStyle,
|
|
466
|
+
ttsEnabled: value.ttsEnabled === true,
|
|
467
|
+
ttsEngine: ['piper', 'edge', 'browser'].includes(value.ttsEngine) ? value.ttsEngine : DEFAULTS.ttsEngine,
|
|
468
|
+
ttsVoice: String(value.ttsVoice || DEFAULTS.ttsVoice).trim(),
|
|
469
|
+
ttsRate: String(value.ttsRate || DEFAULTS.ttsRate).trim(),
|
|
470
|
+
ttsPitch: String(value.ttsPitch || DEFAULTS.ttsPitch).trim(),
|
|
471
|
+
ttsCommand: String(value.ttsCommand || '').trim(),
|
|
472
|
+
ttsPiperModel: String(value.ttsPiperModel || '').trim(),
|
|
473
|
+
ttsPiperCommand: String(value.ttsPiperCommand || '').trim(),
|
|
184
474
|
maxAudioBytes: Math.min(32 * 1024 * 1024, Math.max(256 * 1024, asNumber(value.maxAudioBytes, DEFAULTS.maxAudioBytes))),
|
|
185
475
|
startupTimeoutMs: Math.min(600000, Math.max(10000, asNumber(value.startupTimeoutMs, DEFAULTS.startupTimeoutMs))),
|
|
186
476
|
requestTimeoutMs: Math.min(600000, Math.max(10000, asNumber(value.requestTimeoutMs, DEFAULTS.requestTimeoutMs))),
|
|
@@ -194,6 +484,14 @@ function publicConfig(options, writable = false) {
|
|
|
194
484
|
pythonPath: options.pythonPath || '',
|
|
195
485
|
language: options.language || DEFAULTS.language,
|
|
196
486
|
empathyStyle: options.empathyStyle || DEFAULTS.empathyStyle,
|
|
487
|
+
ttsEnabled: options.ttsEnabled === true,
|
|
488
|
+
ttsEngine: options.ttsEngine || DEFAULTS.ttsEngine,
|
|
489
|
+
ttsVoice: options.ttsVoice || DEFAULTS.ttsVoice,
|
|
490
|
+
ttsRate: options.ttsRate || DEFAULTS.ttsRate,
|
|
491
|
+
ttsPitch: options.ttsPitch || DEFAULTS.ttsPitch,
|
|
492
|
+
ttsCommand: options.ttsCommand || '',
|
|
493
|
+
ttsPiperModel: options.ttsPiperModel || '',
|
|
494
|
+
ttsPiperCommand: options.ttsPiperCommand || '',
|
|
197
495
|
maxAudioBytes: options.maxAudioBytes,
|
|
198
496
|
startupTimeoutMs: options.startupTimeoutMs,
|
|
199
497
|
requestTimeoutMs: options.requestTimeoutMs,
|
|
@@ -227,6 +525,54 @@ function normalizeConfigPatch(value) {
|
|
|
227
525
|
patch.empathyStyle = style
|
|
228
526
|
continue
|
|
229
527
|
}
|
|
528
|
+
if (key === 'ttsEnabled') {
|
|
529
|
+
if (typeof next !== 'boolean') throw new HttpError(400, 'ttsEnabled 必须是布尔值')
|
|
530
|
+
patch.ttsEnabled = next
|
|
531
|
+
continue
|
|
532
|
+
}
|
|
533
|
+
if (key === 'ttsEngine') {
|
|
534
|
+
const engine = String(next || '').trim().toLowerCase()
|
|
535
|
+
if (!['piper', 'edge', 'browser'].includes(engine)) {
|
|
536
|
+
throw new HttpError(400, 'ttsEngine 只能是 piper、edge 或 browser')
|
|
537
|
+
}
|
|
538
|
+
patch.ttsEngine = engine
|
|
539
|
+
continue
|
|
540
|
+
}
|
|
541
|
+
if (key === 'ttsVoice') {
|
|
542
|
+
const voice = String(next || '').trim()
|
|
543
|
+
if (!/^[A-Za-z]{2,3}-[A-Za-z0-9-]+Neural$/.test(voice)) {
|
|
544
|
+
throw new HttpError(400, 'ttsVoice 不是合法的 Edge TTS 音色')
|
|
545
|
+
}
|
|
546
|
+
patch.ttsVoice = voice
|
|
547
|
+
continue
|
|
548
|
+
}
|
|
549
|
+
if (key === 'ttsRate' || key === 'ttsPitch') {
|
|
550
|
+
const value = String(next || '').trim()
|
|
551
|
+
const pattern = key === 'ttsRate' ? /^[+-]\d{1,3}%$/ : /^[+-]\d{1,3}Hz$/
|
|
552
|
+
if (!pattern.test(value)) {
|
|
553
|
+
throw new HttpError(400, `${key} 格式应为 ${key === 'ttsRate' ? '+6%' : '+12Hz'}`)
|
|
554
|
+
}
|
|
555
|
+
patch[key] = value
|
|
556
|
+
continue
|
|
557
|
+
}
|
|
558
|
+
if (key === 'ttsCommand') {
|
|
559
|
+
if (typeof next !== 'string') throw new HttpError(400, 'ttsCommand 必须是字符串')
|
|
560
|
+
const text = next.trim()
|
|
561
|
+
if (text.length > 500 || /[\u0000-\u001f]/.test(text)) {
|
|
562
|
+
throw new HttpError(400, 'ttsCommand 不是合法的路径')
|
|
563
|
+
}
|
|
564
|
+
patch.ttsCommand = text
|
|
565
|
+
continue
|
|
566
|
+
}
|
|
567
|
+
if (key === 'ttsPiperModel' || key === 'ttsPiperCommand') {
|
|
568
|
+
if (typeof next !== 'string') throw new HttpError(400, `${key} 必须是字符串`)
|
|
569
|
+
const text = next.trim()
|
|
570
|
+
if (text.length > 500 || /[\u0000-\u001f]/.test(text)) {
|
|
571
|
+
throw new HttpError(400, `${key} 不是合法的路径`)
|
|
572
|
+
}
|
|
573
|
+
patch[key] = text
|
|
574
|
+
continue
|
|
575
|
+
}
|
|
230
576
|
if (key === 'backendRoot' || key === 'pythonPath') {
|
|
231
577
|
if (typeof next !== 'string') throw new HttpError(400, `${key} 必须是字符串`)
|
|
232
578
|
const text = next.trim()
|
|
@@ -1049,6 +1395,7 @@ export function apply(ctx, rawConfig = {}) {
|
|
|
1049
1395
|
lastCleanupAt: null,
|
|
1050
1396
|
}
|
|
1051
1397
|
const backend = new VoiceBackend(runtime.options, logger)
|
|
1398
|
+
const speechStream = createSpeechStream()
|
|
1052
1399
|
let speechToText
|
|
1053
1400
|
ctx.inject(['settings'], (settingsCtx) => {
|
|
1054
1401
|
settingsService = settingsCtx.settings
|
|
@@ -1060,6 +1407,25 @@ export function apply(ctx, rawConfig = {}) {
|
|
|
1060
1407
|
})
|
|
1061
1408
|
})
|
|
1062
1409
|
ctx.effect(() => () => backend.close())
|
|
1410
|
+
ctx.effect(() => () => speechStream.close())
|
|
1411
|
+
|
|
1412
|
+
ctx.inject(['sessions'], (sessionCtx) => {
|
|
1413
|
+
try {
|
|
1414
|
+
const off = sessionCtx.on?.('session/event', (session, event) => {
|
|
1415
|
+
if (event?.type !== 'assistant/message' || !runtime.options.ttsEnabled) return
|
|
1416
|
+
const text = cleanForSpeech(assistantTextFromEvent(event))
|
|
1417
|
+
if (!text) return
|
|
1418
|
+
speechStream.broadcast({
|
|
1419
|
+
kind: 'assistant',
|
|
1420
|
+
sessionId: String(session?.header?.id ?? session?.id ?? ''),
|
|
1421
|
+
text,
|
|
1422
|
+
})
|
|
1423
|
+
})
|
|
1424
|
+
if (typeof off === 'function') ctx.effect(() => off)
|
|
1425
|
+
} catch (error) {
|
|
1426
|
+
logger.warn?.('[dsh-dafeiyu-emotion] failed to subscribe to session events:', error)
|
|
1427
|
+
}
|
|
1428
|
+
})
|
|
1063
1429
|
|
|
1064
1430
|
ctx.inject(['webServer'], (httpCtx) => {
|
|
1065
1431
|
httpCtx.effect(
|
|
@@ -1086,6 +1452,10 @@ export function apply(ctx, rawConfig = {}) {
|
|
|
1086
1452
|
enabled: options.enabled,
|
|
1087
1453
|
language: options.language,
|
|
1088
1454
|
empathyStyle: options.empathyStyle,
|
|
1455
|
+
ttsEnabled: options.ttsEnabled,
|
|
1456
|
+
ttsEngine: options.ttsEngine,
|
|
1457
|
+
ttsVoice: options.ttsVoice,
|
|
1458
|
+
ttsReady: ttsReady(options),
|
|
1089
1459
|
engine: backendAvailable ? 'SenseVoice + emotion2vec' : 'DSH Speech + prosody',
|
|
1090
1460
|
mode: !backendConfigured
|
|
1091
1461
|
? 'dsh-speech-prosody'
|
|
@@ -1154,19 +1524,52 @@ export function apply(ctx, rawConfig = {}) {
|
|
|
1154
1524
|
}),
|
|
1155
1525
|
'dsh-dafeiyu-emotion: 最近情绪上下文',
|
|
1156
1526
|
)
|
|
1527
|
+
httpCtx.effect(
|
|
1528
|
+
() => httpCtx.webServer.register({
|
|
1529
|
+
kind: 'exact',
|
|
1530
|
+
path: TTS_ENDPOINT,
|
|
1531
|
+
handler: createTtsHandler(runtime),
|
|
1532
|
+
}),
|
|
1533
|
+
'dsh-dafeiyu-emotion: 语音合成',
|
|
1534
|
+
)
|
|
1535
|
+
httpCtx.effect(
|
|
1536
|
+
() => httpCtx.webServer.register({
|
|
1537
|
+
kind: 'exact',
|
|
1538
|
+
path: EVENTS_ENDPOINT,
|
|
1539
|
+
handler(req, res) {
|
|
1540
|
+
if (!isLocalRequest(req)) {
|
|
1541
|
+
jsonResponse(res, 403, { error: '只允许本机访问' })
|
|
1542
|
+
return
|
|
1543
|
+
}
|
|
1544
|
+
if (req.method !== 'GET') {
|
|
1545
|
+
jsonResponse(res, 405, { error: 'method not allowed' })
|
|
1546
|
+
return
|
|
1547
|
+
}
|
|
1548
|
+
speechStream.add(res)
|
|
1549
|
+
},
|
|
1550
|
+
}),
|
|
1551
|
+
'dsh-dafeiyu-emotion: 回复语音事件',
|
|
1552
|
+
)
|
|
1157
1553
|
})
|
|
1158
1554
|
|
|
1159
1555
|
}
|
|
1160
1556
|
|
|
1161
1557
|
export {
|
|
1162
1558
|
VoiceBackend,
|
|
1559
|
+
assistantTextFromEvent,
|
|
1163
1560
|
buildFallbackAnalysis,
|
|
1164
1561
|
buildPrompt,
|
|
1562
|
+
cleanForSpeech,
|
|
1165
1563
|
createConfigHandler,
|
|
1166
1564
|
decorateAnalysis,
|
|
1167
1565
|
normalizeConfigPatch,
|
|
1168
1566
|
plainConfig,
|
|
1169
1567
|
publicConfig,
|
|
1568
|
+
resolveEdgeTts,
|
|
1569
|
+
resolvePiperCommand,
|
|
1570
|
+
resolvePiperModel,
|
|
1170
1571
|
resolveOptions,
|
|
1572
|
+
synthesizePiper,
|
|
1573
|
+
ttsReady,
|
|
1171
1574
|
transcribeWithSpeech,
|
|
1172
1575
|
}
|