dsh-audiogen 0.4.19 → 0.4.22

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "dsh-audiogen",
3
3
  "description": "AI audio generation plugin for the dsh web GUI: multi-vendor TTS/music/sound-effect channels (OpenAI-compatible, ElevenLabs, MiniMax, Stability AI and custom), per-channel model/voice catalogs, Agent tool and a sidebar AI 音频 panel.",
4
- "version": "0.4.19",
4
+ "version": "0.4.22",
5
5
  "type": "module",
6
6
  "main": "lib/index.js",
7
7
  "exports": {
@@ -0,0 +1,151 @@
1
+ ---
2
+ name: dsh-audiogen-voice-cast
3
+ description: 角色音色选角(casting)——把角色画像(JSON 或文本)交给 DSH AI 音频插件,manage_audio_voices 的 action=cast 做确定性硬过滤(性别/年龄/用途严格、accent 可放松)得到每个角色的候选音色池,Agent 在上下文中全局权衡主音色(lead/major 不复用)+ 备用音色,再用 action=save_cast 校验落盘并生成 TTS。当用户提到「为角色选音色 / 角色配音 / 给每个角色分配声音 / 选角 / 角色音色表 / 给小说或游戏角色挑声线」时触发。
4
+ whenToUse: 用户有一批角色(小说/游戏/剧本),需要用 MiniMax 或 ElevenLabs 音色为每个角色分配主音色(可带备用音色),且输入可能是结构化 JSON 或自然语言文本;或想把选角结果保存下来再逐角色生成 TTS 时。
5
+ ---
6
+
7
+ # 角色音色选角(dsh-audiogen voice cast)
8
+
9
+ 选角 = **角色画像 → 确定性硬过滤(工具)→ 全局权衡(Agent 推理)→ 校验落盘(工具)→ TTS 生成**。
10
+ 工具不做「选谁」的推理(那是本 Agent 的工作,保持整组阵容的全局视野);工具只做可校验的过滤与防幻觉校验。
11
+
12
+ ## 第一步:整理输入(JSON 或文本都行,但工具只接受 JSON 结构)
13
+
14
+ `manage_audio_voices` 的 `characters` 参数接受:JSON 数组 / 单个对象 / JSON 字符串。
15
+ 如果用户给的是自然语言文本(如「男主慕声,少年,声音低沉阴郁;女主慕瑶,青年,清冷……」),
16
+ **先把它整理成下面的结构**再调用:
17
+
18
+ ```json
19
+ {
20
+ "action": "cast",
21
+ "channel": "MiniMax",
22
+ "language": "zh",
23
+ "characters": [
24
+ {
25
+ "character_id": "char_musheng",
26
+ "character_name": "慕声",
27
+ "gender": "男性",
28
+ "age_stage": "少年",
29
+ "voice_traits": ["声音低沉轻柔", "冷淡"],
30
+ "personality_traits": ["阴郁", "狠厉", "占有欲强"],
31
+ "appearance": ["少年;高马尾;黑发微卷;身形单薄"],
32
+ "sample_lines": [{"text": "……", "emotion_hint": "冷淡"}],
33
+ "dialogue_count": 300
34
+ },
35
+ {
36
+ "character_id": "char_muyao",
37
+ "character_name": "慕瑶",
38
+ "gender": "女性",
39
+ "age_stage": "青年",
40
+ "voice_traits": ["清冽动听", "清冷"],
41
+ "personality_traits": ["清冷", "孤傲", "坚强"],
42
+ "dialogue_count": 120
43
+ }
44
+ ]
45
+ }
46
+ ```
47
+
48
+ 字段说明:
49
+
50
+ | 字段 | 必填 | 说明 |
51
+ |---|---|---|
52
+ | `character_id` | 否 | 缺省按角色名自动生成(`char_<slug>`) |
53
+ | `character_name` | 是 | 角色名(缺失会报 `cast-character-missing-name`) |
54
+ | `gender` | 否 | 男/女、male/female(自动归一) |
55
+ | `age_stage` | 否 | 少年/青年/中年/老年 或数组;可加 `age_stage_source`(explicit/appearance_inferred/unknown)表示年龄字段可信度 |
56
+ | `voice_traits` / `personality_traits` / `appearance` | 否 | 字符串或数组,自动按 `;;、,,/|` 拆分去重 |
57
+ | `sample_lines` | 否 | 样例台词(voice_design 的 preview_text 来源,最多 3 句) |
58
+ | `dialogue_count` | 否 | 台词量:≥200 → lead,≥50 → major,其余 supporting(决定「主音色不复用」的约束) |
59
+ | `language` / `use_case` | 否 | 语言(如 zh/en)与用途(如 characters_animation);也可在工具顶层传 |
60
+
61
+ ## 第二步:action=cast 拿每角色的候选池(工具做确定性硬过滤)
62
+
63
+ ```json
64
+ {
65
+ "action": "cast",
66
+ "channel": "ElevenLabs",
67
+ "language": "en",
68
+ "use_case": "characters_animation",
69
+ "accent": "british",
70
+ "characters": [……]
71
+ }
72
+ ```
73
+
74
+ 返回每个角色的 `character`(含 `importance_tier`)、`mapped_filters`(映射后的过滤条件 + notes)与
75
+ `candidate_voices`(每条含 voice_id/name/source/gender/age/accent/use_case/description/preview_url)。
76
+
77
+ **过滤规则(与音频工作室项目一致)**:
78
+
79
+ - `gender`:男→male / 女→female;映射出来后是**硬过滤**,绝不放松。
80
+ - `age`:少年/少女/青年/成年→young;中年→middle_aged;老年→old;**硬过滤**。
81
+ - `use_case`:**硬过滤**(ElevenLabs 默认 `characters_animation`;传 `""` 关闭)。
82
+ - `accent`:**只是偏好**——严格候选为空时才放松(重新从全池过滤)。
83
+ - MiniMax 系统/自建音色没有性别/年龄/用途元数据:候选池按语言(voice_id 前缀)本地过滤,
84
+ 排名完全靠名称/描述,需要 Agent 结合声音描述判断。
85
+ - 候选最多 60 条/角色,超限自动截断并在 `note` 标注。
86
+
87
+ ## 第三步:全局选角(Agent 推理,这是 LLM 的活)
88
+
89
+ 拿到每角色候选池后,在上下文里统一权衡(不要逐角色独立选):
90
+
91
+ 1. 先看 `voice_traits`(声线质感),用 `personality_traits` 修正气质(清冷/温润/阴郁/活泼/胆怯/威严)。
92
+ 2. 用 `gender` + `age_stage` 排除明显不合的候选(注意 `age_stage_source`:explicit 时年龄是唯一依据,
93
+ appearance 里的年龄词只能作辅助)。
94
+ 3. 主音色要能承担长台词:清晰度、稳定音量、中性/紧张/温柔/愤怒都要可行;
95
+ 很强"效果音"质感的只适合备用槽,除非那正是角色日常说话身份。
96
+ 4. **lead / major 角色主音色不得复用**;supporting 只有无替代候选时才可复用并在 reason 说明。
97
+ 5. 每个角色主音色 + 最多 2 个备用(`backup_voice_ids`);`reason` 简短中文理由。
98
+
99
+ ## 第四步:action=save_cast 校验落盘
100
+
101
+ ```json
102
+ {
103
+ "action": "save_cast",
104
+ "channel": "ElevenLabs",
105
+ "characters": [……同 cast……],
106
+ "selections": [
107
+ {"character_id": "char_musheng", "voice_id": "bjoUrk7s2fY9cu2u67KF", "backup_voice_ids": ["q1h5HGdnfVxp4TXTJRNN"], "reason": "低沉阴郁的少年音…"}
108
+ ]
109
+ }
110
+ ```
111
+
112
+ 工具会:
113
+ - 重新按相同规则算候选池,校验 `voice_id` 属于该角色候选(**编造/过期 id 会被兜底并标记 `tool_fallback`**);
114
+ - 自动补齐 `backup_voice_ids`(最多 2 个,不重复、不等于主音色);
115
+ - 检查 lead/major 主音色复用(`issues` 里给 `primary_voice_reused` 警告,需你下一轮修正后重存);
116
+ - 持久化到 `~/.dsh/dsh-audiogen/cast-selections.json`(按渠道 + character_id 覆盖保存)。
117
+
118
+ 返回 `selections`(含 `selection_status`/`issues`)与 `issues`;有 `primary_voice_reused` 时修一下再存。
119
+
120
+ ## 第五步:按选定音色生成 TTS
121
+
122
+ ```json
123
+ {"mode": "tts", "channel": "ElevenLabs", "voice": "<voice_id>", "prompt": "对白文本"}
124
+ ```
125
+
126
+ - MiniMax 的 `voice` 必须是官方 voice_id(如 `Chinese (Mandarin)_Gentleman` 或 `voice_xxx`)。
127
+ - 若字符没有合适现成音色,先用 **voice_design 创作专属音色**(见 dsh-audiogen-voice-design 技能):
128
+
129
+ ```json
130
+ {"mode": "voice_design", "prompt": "一位清冷孤傲却带柔情的青年女声,音色清冽,气质疏离……", "preview_text": "从 sample_lines 里选一句,或写一句自然对白"}
131
+ ```
132
+
133
+ 生成的 `voiceId` 可直接作为该角色的主音色 `voice_id` 写入 save_cast。
134
+
135
+ ## 常见错误
136
+
137
+ - `cast-characters-parse-failed`:characters 不是合法 JSON → 先整理成数组/对象再调用。
138
+ - `cast-character-missing-name`:某个角色缺 `character_name`。
139
+ - `cast-selections-required`:save_cast 忘了传 `selections`。
140
+ - `character_count=0` 或显示候选为空:检查语言/use_case 太严(use_case 传 `""` 再试),或渠道网关不支持音色库。
141
+ - `channel-choice-required`:多渠道未指定 → 显式传 `channel`(如 "ElevenLabs")。
142
+ - 候选池只有 shared/system(只读)没关系:cast 只需要 voice_id,生成时照常可用。
143
+ - 网关渠道无音色库端点时:cast 候选池会回退为该渠道配置的音色目录(`source=渠道模型`,无性别/年龄
144
+ 元数据),此时硬过滤不适用,只能按名称/描述判断;需要完整候选池请配置官方 API 地址的渠道。
145
+
146
+ ## 与推荐的区别
147
+
148
+ - `action=recommend`:一句需求文本(如「17岁清亮甜美少女音」)→ 插件 LLM 给 top-k,
149
+ 适合**单个声线的快速推荐**(无角色结构)。
150
+ - `action=cast` + `save_cast`:**整组角色** → 工具硬过滤 + Agent 全局选角 + 校验落盘,
151
+ 适合**小说/游戏配音的正式选角**(JSON 或文本角色清单)。
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  name: dsh-audiogen-voice-management
3
- description: 浏览/筛选/删除厂商音色库(MiniMax、ElevenLabs)并复用选定音色生成 TTS。当用户提到「音色管理 / 音色列表 / 查看有哪些音色 / 删除音色 / 筛选音色 / 用某个音色 TTS / 声音列表」时触发。与资源库(本地生成结果)无关,请勿混淆。
4
- whenToUse: 需要查看某个渠道(MiniMax / ElevenLabs)有哪些可用音色、按语言/关键词筛选音色、删除某个自建音色,或拿到音色 voice_id 后用它生成语音时。
3
+ description: 浏览/筛选/删除厂商音色库(MiniMax、ElevenLabs)并按需求描述(prompt)推荐音色,复用选定音色生成 TTS。当用户提到「音色管理 / 音色列表 / 查看有哪些音色 / 删除音色 / 筛选音色 / 推荐音色 / 选个合适的音色 / 用某个音色 TTS / 声音列表」时触发。与资源库(本地生成结果)无关,请勿混淆。
4
+ whenToUse: 需要查看某个渠道(MiniMax / ElevenLabs)有哪些可用音色、按语言/关键词筛选音色、按一段描述(如「清亮甜美的少女音」)让模型推荐音色、删除某个自建音色,或拿到音色 voice_id 后用它生成语音时。
5
5
  ---
6
6
 
7
7
  # 厂商音色管理(dsh-audiogen)
@@ -45,6 +45,19 @@ whenToUse: 需要查看某个渠道(MiniMax / ElevenLabs)有哪些可用音
45
45
 
46
46
  这些参数只对 ElevenLabs 共享库起作用(服务端筛选),自有音色与其他渠道按同名字段本地兜底;MiniMax 无服务端筛选端点,会在返回 `note` 中说明。
47
47
 
48
+ ## 按需求描述推荐音色(action=recommend)
49
+
50
+ ```json
51
+ {"action": "recommend", "channel": "ElevenLabs", "language": "en", "requirement": "17岁清亮甜美的少女音,适合活泼女主角,英式口音", "top_k": 5}
52
+ ```
53
+
54
+ - `requirement`(必填):自然语言需求描述。模型按语言/性别/年龄感/气质/用途综合打分。
55
+ - `top_k`:推荐条数(1-10,默认 5)。
56
+ - 候选池 = 当前渠道 + 传入的筛选条件(language/keyword/source + 共享库服务端参数),默认拉宽到上限 500 条;结果中的 `voice_id` 均校验为候选池真实成员(模型编造的 id 会被丢弃)。
57
+ - 返回 `recommendations`:每条含 `voice_id` / `name` / `source` / `deletable` / `preview_url` 等音色字段 + `reason`(中文推荐理由)。
58
+ - 机制:复用 Agent 默认模型(「设置 → 模型」的 agent-default-model),**不新增 API key**;模型未配置时提示先设置默认模型。
59
+ - 推荐后再 `generate_audio(mode="tts", voice="<voice_id>")` 闭环生成。
60
+
48
61
  ## 删除音色(action=delete)
49
62
 
50
63
  ```json
@@ -64,5 +77,27 @@ MiniMax TTS 的 `voice` 参数需要官方 voice_id;ElevenLabs 的 `voice` 参
64
77
  ## 常见错误
65
78
 
66
79
  - `channel-choice-required`:多个渠道未指定 → 先 `manage_audio_voices(list, channel=...)` 或询问用户。
80
+ - `recommend-requirement-required`:推荐时缺 `requirement` → 补上自然语言描述。
81
+ - `recommend-no-candidates`:候选池为空 → 确认渠道音色库可用,或放宽筛选。
82
+ - `recommend-parse-failed`:模型返回的推荐未命中候选池 → 重试,或用语言/关键词先缩小候选集。
67
83
  - 删除被拒(`system`/`shared` 只读):换个 `source=custom`/`owned` 的音色。
68
84
  - 拉取失败:渠道 API 地址/密钥错误,或网关不支持音色管理端点(Stability、自定义 OpenAI 兼容渠道无此能力,会明确提示不支持)。
85
+ - 网关渠道(如 new-api 中转站)未映射 ElevenLabs 的 `/v1/voices`、`/v1/shared-voices`:列表与 AI 推荐会**自动回退**
86
+ 为该渠道「配置的音色目录」(`source=渠道模型`,只有名称、无试听/语言/性别/年龄元数据),仍可推荐与生成;
87
+ 需要完整音色库(试听、描述、性别年龄筛选)时请添加一个官方 API 地址(https://api.elevenlabs.io)的渠道。
88
+
89
+ ## 角色选角(多角色分配音色)
90
+
91
+ 上面是「一个需求 → 一个音色」。如果用户有一整组角色(小说/游戏配音)要为每个角色分配
92
+ 主音色(+备用音色),使用工具的 **action=cast / action=save_cast** 两个动作:
93
+
94
+ 1. `action=cast`:传 `characters`(角色画像 JSON 数组/对象,含 character_name/gender/age_stage/
95
+ voice_traits/personality_traits/appearance/sample_lines/dialogue_count),工具按性别/年龄/用途做
96
+ 确定性硬过滤(accent 为偏好、候选为空才放松),返回每个角色的候选池。
97
+ 2. Agent 在上下文中全局选角(lead/major 角色主音色不要复用;每角色主音色 + 最多 2 个备用)。
98
+ 3. `action=save_cast`:传同样的 `characters` + `selections`,工具校验 voice_id 属于候选池、
99
+ 补齐备份、标记复用警告并持久化到 `~/.dsh/dsh-audiogen/cast-selections.json`。
100
+ 4. 用选定 `voice_id` 调 `generate_audio` 生成 TTS;无合适音色时先用
101
+ `generate_audio(mode="voice_design")` 创作专属音色。
102
+
103
+ 完整流程、输入结构示例与约束见 **dsh-audiogen-voice-cast** 技能。
@@ -12,9 +12,19 @@ import { generateAudio, AudioGenError } from './audio-engine.ts'
12
12
  import type { GenerationBudget } from './audio-scheduler.ts'
13
13
  import { appendHistory, saveAudioFile, saveToLibrary, listLibrary } from './audio-store.ts'
14
14
  import {
15
- listVendorVoices,
15
+ listVendorVoicesWithFallback,
16
16
  deleteVendorVoice,
17
+ isElevenLabs,
18
+ type VendorVoiceEntry,
17
19
  } from './voice-manager.ts'
20
+ import type { VoiceRecommendation } from './voice-recommend.ts'
21
+ import { appendVoiceRecommendRecord } from './voice-recommend.ts'
22
+ import {
23
+ parseCharacterProfiles,
24
+ prepareVoiceCast,
25
+ saveVoiceCast,
26
+ type CastSelectionInput,
27
+ } from './voice-cast.ts'
18
28
  import type { AudioMode, GenerateAudioRequest, LibraryType } from './protocol.ts'
19
29
 
20
30
  export interface AgentAudioToolConfig {
@@ -27,6 +37,8 @@ export interface AgentAudioToolConfig {
27
37
  budget?: GenerationBudget
28
38
  /** 提示词增强(复用 Agent 默认模型);enhance_prompt=true 时在生成前调用。 */
29
39
  enhance?: (prompt: string, mode: AudioMode) => Promise<string>
40
+ /** 音色推荐(复用 Agent 默认模型):需求描述 + 候选池 → top-k 推荐。 */
41
+ recommend?: (requirement: string, candidates: VendorVoiceEntry[], topK: number) => Promise<VoiceRecommendation[]>
30
42
  }
31
43
 
32
44
  interface AgentAudioRef {
@@ -553,19 +565,44 @@ export function registerAgentAudioTools(ctx: Context, resolve: () => AgentAudioT
553
565
  },
554
566
  }))
555
567
 
556
- /** 厂商音色管理:浏览/筛选 + 删除(仅自建)。删除不可逆,必须 confirm=true。 */
568
+ /** 单条音色条目 schema(voices / candidate_voices 共用)。 */
569
+ const voiceItemSchema = {
570
+ type: 'object',
571
+ additionalProperties: false,
572
+ properties: {
573
+ voice_id: { type: 'string', required: true },
574
+ name: { type: 'string', required: true },
575
+ source: { type: 'string', required: true },
576
+ deletable: { type: 'boolean', required: true },
577
+ language: { type: 'string' },
578
+ locale: { type: 'string' },
579
+ accent: { type: 'string' },
580
+ gender: { type: 'string' },
581
+ age: { type: 'string' },
582
+ use_case: { type: 'string' },
583
+ category: { type: 'string' },
584
+ descriptive: { type: 'string' },
585
+ labels: { type: 'object', additionalProperties: true },
586
+ description: { type: 'string' },
587
+ preview_url: { type: 'string' },
588
+ },
589
+ } as const
590
+
591
+ /** 厂商音色管理:浏览/筛选 + 按需求描述推荐 + 删除(仅自建)+ 角色选角(cast)。删除不可逆,必须 confirm=true。 */
557
592
  const managementDisposer = ctx.tools.register(defineTool({
558
593
  name: 'manage_audio_voices',
559
- description: 'Manage vendor voice libraries (MiniMax / ElevenLabs). action=list: browse available TTS voices of a channel — official/shared voices plus voices designed/cloned by the account; filtering supports the official /v1/shared-voices server-side filters (search/use_case/accent/gender/age/locale/category/sort/featured/free_users_allowed/descriptive) for the ElevenLabs shared library, plus local language/keyword/source filtering everywhere; returns voice_id/name/source/description/preview_url and whether each voice is deletable. action=delete: delete one OWNED voice (custom/owned only; official/shared/system voices are read-only and refused) — irreversible, so confirm must be true (pass the exact voice_id from action=list). Use the returned voice_id with generate_audio (mode=tts, voice=<voice_id>) to speak with the selected voice.',
594
+ description: 'Manage vendor voice libraries (MiniMax / ElevenLabs). action=list: browse available TTS voices of a channel — official/shared voices plus voices designed/cloned by the account; filtering supports the official /v1/shared-voices server-side filters (search/use_case/accent/gender/age/locale/category/sort/featured/free_users_allowed/descriptive) for the ElevenLabs shared library, plus local language/keyword/source filtering everywhere; returns voice_id/name/source/description/preview_url and whether each voice is deletable. Channels whose gateway lacks the vendor voice-library endpoints (e.g. new-api relays) automatically fall back to the configured model-alias catalog as the candidate pool (source=configured, name only). action=recommend: let the agent default model pick the top-k voices for a natural-language requirement (e.g. "17岁清亮甜美的少女音,适合活泼女主角,英式口音") from the same candidate pool — pass requirement (required) and optional top_k (1-10, default 5); returns ranked voices with a short reason each; voice_ids are validated against the pool (hallucinated ids are dropped). action=cast: 角色音色选角第一步 — pass characters (角色画像 JSON 数组/对象/JSON 字符串:character_id, character_name, gender 男/女, age_stage 少年/青年/中年/老年, voice_traits, personality_traits, appearance, sample_lines, dialogue_count, language, use_case) plus optional language/use_case/accent; the tool applies deterministic hard filters per character (gender/age/use_case strict; accent is a preference and is relaxed only when the strict pool is empty) and returns each character\u2019s mapped filters + filtered candidate_voices (primary + backup slots). Then select voices globally in-context (lead/major 角色主音色不要复用), and call action=save_cast with the same characters plus selections [{character_id, voice_id, character_name?, backup_voice_ids?, reason?}] to validate membership, auto-fill backups, flag primary reuse and persist the cast plan to ~/.dsh/dsh-audiogen/cast-selections.json. action=delete: delete one OWNED voice (custom/owned only; official/shared/system voices are read-only and refused) — irreversible, so confirm must be true (pass the exact voice_id from action=list). Use the returned voice_id with generate_audio (mode=tts, voice=<voice_id>) to speak with the selected voice.',
560
595
  parameters: {
561
- action: { type: 'string', enum: ['list', 'delete'], required: true, description: 'list = browse/filter voices; delete = remove an owned voice.' },
596
+ action: { type: 'string', enum: ['list', 'recommend', 'delete', 'cast', 'save_cast'], required: true, description: 'list = browse/filter voices; recommend = pick voices for a requirement with the agent default model; cast = prepare per-character filtered candidate pools (deterministic hard filters, no LLM); save_cast = validate + persist a cast plan made in-context; delete = remove an owned voice.' },
562
597
  channel: { type: 'string', description: 'Channel name or id (e.g. the channel shown in settings). Defaults to the default channel; required when more than one channel is configured.' },
563
- language: { type: 'string', description: 'Filter for list: language substring (ISO code like en/zh/ja, or a label like Chinese (Mandarin)).' },
564
- keyword: { type: 'string', description: 'Filter for list: free text over voice name/description/accent/use_case.' },
565
- source: { type: 'string', enum: ['system', 'custom', 'owned', 'shared'], description: 'Filter for list: MiniMax system/custom; ElevenLabs owned (account) / shared (community).' },
598
+ characters: { type: 'json', description: 'Required for cast/save_cast: 角色画像 — JSON array of profile objects, a single profile object, or a JSON string. Each profile: character_id (optional, auto-derived from character_name), character_name (required), gender (男/女 or male/female), age_stage (少年/青年/中年/老年…, string or array), age_stage_source (explicit/appearance_inferred/unknown), voice_traits, personality_traits, appearance (string or string[]), sample_lines ([{text, emotion_hint?}] or text), dialogue_count (number), language (en/zh/ja…), use_case. Convert free-text character descriptions into this structure before calling.' },
599
+ selections: { type: 'json', description: 'Required for save_cast: the cast plan made by you — JSON array of {character_id, voice_id, character_name?, backup_voice_ids?, reason?}. voice_id must come from the cast action\u2019s candidate_voices; the tool validates membership, fills missing backups, flags lead/major primary reuse and persists to ~/.dsh/dsh-audiogen/cast-selections.json.' },
600
+ language: { type: 'string', description: 'Filter for list/recommend: language substring (ISO code like en/zh/ja, or a label like Chinese (Mandarin)). For cast: pool language for the candidates (per-character language overrides it).' },
601
+ keyword: { type: 'string', description: 'Filter for list/recommend: free text over voice name/description/accent/use_case.' },
602
+ source: { type: 'string', enum: ['system', 'custom', 'owned', 'shared'], description: 'Filter for list/recommend: MiniMax system/custom; ElevenLabs owned (account) / shared (community).' },
566
603
  search: { type: 'string', description: 'Official /v1/shared-voices filter: free-text search over the ElevenLabs shared voice library (ElevenLabs only; local fallback elsewhere).' },
567
- use_case: { type: 'string', description: 'Official filter: use case, e.g. characters_animation / conversational / narration / gaming (ElevenLabs shared library).' },
568
- accent: { type: 'string', description: 'Official filter: accent, e.g. british / american / australian.' },
604
+ use_case: { type: 'string', description: 'Official filter: use case, e.g. characters_animation / conversational / narration / gaming (ElevenLabs shared library). For cast: hard filter per character (default characters_animation for ElevenLabs; pass "" to disable).' },
605
+ accent: { type: 'string', description: 'Official filter: accent, e.g. british / american / australian. For cast: accent is a preference (strict first, relaxed when the strict pool is empty).' },
569
606
  gender: { type: 'string', description: 'Official filter: male / female.' },
570
607
  age: { type: 'string', description: 'Official filter: age bracket, e.g. adult / young / middle_aged.' },
571
608
  locale: { type: 'string', description: 'Official filter: language locale, e.g. en-us / en-gb.' },
@@ -574,6 +611,8 @@ export function registerAgentAudioTools(ctx: Context, resolve: () => AgentAudioT
574
611
  featured: { type: 'boolean', description: 'Official filter: featured shared voices only (true only).' },
575
612
  free_users_allowed: { type: 'boolean', description: 'Official filter: voices allowed for free users only (true only).' },
576
613
  descriptive: { type: 'boolean', description: 'Official filter: voices with descriptions only (true only).' },
614
+ requirement: { type: 'string', description: 'Required for recommend: the natural-language voice requirement, e.g. "低沉磁性的中年男声,适合沉稳旁白".' },
615
+ top_k: { type: 'integer', description: 'Optional for recommend: how many voices to return (1-10, default 5).' },
577
616
  voice_id: { type: 'string', description: 'Required for delete: the exact voice_id from action=list.' },
578
617
  confirm: { type: 'boolean', description: 'Required for delete: must be true; deletion is irreversible.' },
579
618
  },
@@ -583,12 +622,18 @@ export function registerAgentAudioTools(ctx: Context, resolve: () => AgentAudioT
583
622
  additionalProperties: false,
584
623
  properties: {
585
624
  status: { type: 'string', required: true, enum: ['ok'] },
586
- kind: { type: 'string', required: true, enum: ['list', 'delete'] },
625
+ kind: { type: 'string', required: true, enum: ['list', 'recommend', 'delete', 'cast', 'save_cast'] },
587
626
  vendor: { type: 'string', required: true },
588
627
  channel: { type: 'string', required: true },
589
628
  count: { type: 'integer' },
590
629
  truncated: { type: 'boolean' },
630
+ requirement: { type: 'string' },
631
+ candidate_count: { type: 'integer' },
591
632
  voices: {
633
+ type: 'array',
634
+ items: voiceItemSchema,
635
+ },
636
+ recommendations: {
592
637
  type: 'array',
593
638
  items: {
594
639
  type: 'object',
@@ -599,15 +644,113 @@ export function registerAgentAudioTools(ctx: Context, resolve: () => AgentAudioT
599
644
  source: { type: 'string', required: true },
600
645
  deletable: { type: 'boolean', required: true },
601
646
  language: { type: 'string' },
647
+ locale: { type: 'string' },
602
648
  accent: { type: 'string' },
603
649
  gender: { type: 'string' },
604
650
  age: { type: 'string' },
605
651
  use_case: { type: 'string' },
652
+ category: { type: 'string' },
653
+ descriptive: { type: 'string' },
654
+ labels: { type: 'object', additionalProperties: true },
606
655
  description: { type: 'string' },
607
656
  preview_url: { type: 'string' },
657
+ reason: { type: 'string', required: true },
658
+ },
659
+ },
660
+ },
661
+ // ---- cast / save_cast ----
662
+ pool_size: { type: 'integer' },
663
+ use_case_filter: { type: 'string' },
664
+ accent_preference: { type: 'string' },
665
+ character_count: { type: 'integer' },
666
+ characters: {
667
+ type: 'array',
668
+ items: {
669
+ type: 'object',
670
+ additionalProperties: false,
671
+ properties: {
672
+ character: {
673
+ type: 'object',
674
+ additionalProperties: false,
675
+ properties: {
676
+ character_id: { type: 'string', required: true },
677
+ character_name: { type: 'string', required: true },
678
+ gender: { type: 'string' },
679
+ age_stage: { type: 'array', items: { type: 'string' } },
680
+ age_stage_source: { type: 'string' },
681
+ voice_traits: { type: 'array', items: { type: 'string' } },
682
+ personality_traits: { type: 'array', items: { type: 'string' } },
683
+ appearance: { type: 'array', items: { type: 'string' } },
684
+ sample_lines: {
685
+ type: 'array',
686
+ items: {
687
+ type: 'object',
688
+ additionalProperties: false,
689
+ properties: {
690
+ text: { type: 'string', required: true },
691
+ emotion_hint: { type: 'string' },
692
+ },
693
+ },
694
+ },
695
+ dialogue_count: { type: 'integer', required: true },
696
+ importance_tier: { type: 'string', required: true },
697
+ language: { type: 'string' },
698
+ use_case: { type: 'string' },
699
+ },
700
+ },
701
+ mapped_filters: {
702
+ type: 'object',
703
+ additionalProperties: false,
704
+ properties: {
705
+ gender: { type: 'string' },
706
+ age: { type: 'array', items: { type: 'string' } },
707
+ fallback_age: { type: 'array', items: { type: 'string' } },
708
+ accent: { type: 'string' },
709
+ use_case: { type: 'string' },
710
+ language: { type: 'string' },
711
+ notes: { type: 'string', required: true },
712
+ },
713
+ },
714
+ candidate_count: { type: 'integer', required: true },
715
+ candidate_voices: { type: 'array', required: true, items: voiceItemSchema },
716
+ note: { type: 'string' },
608
717
  },
609
718
  },
610
719
  },
720
+ selections: {
721
+ type: 'array',
722
+ items: {
723
+ type: 'object',
724
+ additionalProperties: false,
725
+ properties: {
726
+ character_id: { type: 'string', required: true },
727
+ character_name: { type: 'string', required: true },
728
+ voice_id: { type: 'string', required: true },
729
+ voice_name: { type: 'string', required: true },
730
+ backup_voice_ids: { type: 'array', items: { type: 'string' }, required: true },
731
+ reason: { type: 'string', required: true },
732
+ dialogue_count: { type: 'integer' },
733
+ importance_tier: { type: 'string' },
734
+ selection_status: { type: 'string', required: true },
735
+ issues: { type: 'array', items: { type: 'string' }, required: true },
736
+ selected_at: { type: 'string' },
737
+ },
738
+ },
739
+ },
740
+ issues: {
741
+ type: 'array',
742
+ items: {
743
+ type: 'object',
744
+ additionalProperties: false,
745
+ properties: {
746
+ character_id: { type: 'string', required: true },
747
+ character_name: { type: 'string', required: true },
748
+ issue: { type: 'string', required: true },
749
+ detail: { type: 'string', required: true },
750
+ },
751
+ },
752
+ },
753
+ store_path: { type: 'string' },
611
754
  voice_id: { type: 'string' },
612
755
  deleted: { type: 'boolean' },
613
756
  message: { type: 'string' },
@@ -624,8 +767,57 @@ export function registerAgentAudioTools(ctx: Context, resolve: () => AgentAudioT
624
767
  throw new AudioGenError('AI audio is disabled. Open Settings > Plugins > AI Audio and enable it.', 'plugin-disabled')
625
768
  }
626
769
  const channel = resolveVoiceManagementChannel(config, args.channel)
627
- const action = args.action === 'delete' ? 'delete' : 'list'
628
- if (action === 'list') {
770
+ const action = args.action === 'delete' ? 'delete'
771
+ : args.action === 'recommend' ? 'recommend'
772
+ : args.action === 'cast' ? 'cast'
773
+ : args.action === 'save_cast' ? 'save_cast'
774
+ : 'list'
775
+ // ---- 角色选角:先过滤(cast),后校验落盘(save_cast)——LLM 推理在中间由 Agent 完成。 ----
776
+ if (action === 'cast' || action === 'save_cast') {
777
+ const characters = parseCharacterProfiles(args.characters)
778
+ // ElevenLabs 选角默认硬过滤用途 characters_animation(传 "" 可关闭);accent 仅是偏好。
779
+ const defaultUseCase = isElevenLabs(channel) && typeof args.use_case !== 'string' ? 'characters_animation' : undefined
780
+ const castOptions = {
781
+ ...(typeof args.use_case === 'string' ? { use_case: args.use_case.trim() } : defaultUseCase === undefined ? {} : { use_case: defaultUseCase }),
782
+ ...(typeof args.language === 'string' && args.language.trim() !== '' ? { language: args.language.trim() } : {}),
783
+ ...(typeof args.accent === 'string' && args.accent.trim() !== '' ? { accent: args.accent.trim() } : {}),
784
+ }
785
+ if (action === 'cast') {
786
+ const prepared = await prepareVoiceCast(channel, characters, castOptions)
787
+ return {
788
+ status: 'ok' as const,
789
+ kind: 'cast' as const,
790
+ vendor: prepared.vendor,
791
+ channel: prepared.channel,
792
+ pool_size: prepared.pool_size,
793
+ use_case_filter: prepared.use_case_filter,
794
+ accent_preference: prepared.accent_preference,
795
+ character_count: prepared.character_count,
796
+ characters: prepared.characters,
797
+ ...(prepared.note === undefined ? {} : { note: prepared.note }),
798
+ }
799
+ }
800
+ const rawSelections = args.selections
801
+ const selectionInputs: CastSelectionInput[] = Array.isArray(rawSelections) ? rawSelections as unknown as CastSelectionInput[] : []
802
+ if (selectionInputs.length === 0) {
803
+ throw new AudioGenError(
804
+ 'save_cast requires selections — an array of {character_id, voice_id, character_name?, backup_voice_ids?, reason?} matching the cast result',
805
+ 'cast-selections-required',
806
+ )
807
+ }
808
+ const saved = await saveVoiceCast(channel, characters, selectionInputs, castOptions)
809
+ return {
810
+ status: 'ok' as const,
811
+ kind: 'save_cast' as const,
812
+ vendor: saved.vendor,
813
+ channel: saved.channel,
814
+ store_path: saved.store_path,
815
+ count: saved.count,
816
+ selections: saved.entries,
817
+ issues: saved.issues,
818
+ }
819
+ }
820
+ if (action === 'list' || action === 'recommend') {
629
821
  const source = typeof args.source === 'string' && ['system', 'custom', 'owned', 'shared'].includes(args.source) ? args.source : undefined
630
822
  const pick = (value: unknown): string | undefined => typeof value === 'string' && value.trim() !== '' ? value.trim() : undefined
631
823
  const serverFilters = {
@@ -641,12 +833,68 @@ export function registerAgentAudioTools(ctx: Context, resolve: () => AgentAudioT
641
833
  ...(args.free_users_allowed === true ? { free_users_allowed: true } : {}),
642
834
  ...(args.descriptive === true ? { descriptive: true } : {}),
643
835
  }
644
- const result = await listVendorVoices(channel, {
836
+ const result = await listVendorVoicesWithFallback(channel, {
645
837
  ...(typeof args.language === 'string' && args.language.trim() !== '' ? { language: args.language.trim() } : {}),
646
838
  ...(typeof args.keyword === 'string' && args.keyword.trim() !== '' ? { keyword: args.keyword.trim() } : {}),
647
839
  ...(source === undefined ? {} : { source }),
648
840
  ...(Object.keys(serverFilters).length === 0 ? {} : { serverFilters }),
841
+ // 推荐用宽候选池(最高上限);浏览保持默认窗口更快。
842
+ ...(action === 'recommend' ? { limit: 500 } : {}),
649
843
  })
844
+ if (action === 'recommend') {
845
+ const requirement = typeof args.requirement === 'string' ? args.requirement.trim() : ''
846
+ if (requirement === '') throw new AudioGenError('recommend requires requirement (a natural-language voice description).', 'recommend-requirement-required')
847
+ const rawTopK = args.top_k
848
+ const topK = typeof rawTopK === 'number' && Number.isFinite(rawTopK)
849
+ ? Math.max(1, Math.min(10, Math.floor(rawTopK)))
850
+ : 5
851
+ if (config.recommend === undefined) throw new AudioGenError('Voice recommendation is unavailable (LLM service not wired).', 'recommend-unavailable')
852
+ const recommendations = await config.recommend(requirement, result.voices, topK)
853
+ // 与面板共用同一份推荐记录(最近 50 条,失败不影响结果)。
854
+ void appendVoiceRecommendRecord({
855
+ channel: channel.name,
856
+ vendor: result.vendor,
857
+ requirement,
858
+ candidate_count: result.voices.length,
859
+ top_k: topK,
860
+ channel_id: channel.id,
861
+ filters: {
862
+ ...(typeof args.language === 'string' && args.language.trim() !== '' ? { language: args.language.trim() } : {}),
863
+ ...(typeof args.keyword === 'string' && args.keyword.trim() !== '' ? { keyword: args.keyword.trim() } : {}),
864
+ ...(source === undefined ? {} : { source }),
865
+ ...(typeof args.use_case === 'string' && args.use_case.trim() !== '' ? { use_case: args.use_case.trim() } : {}),
866
+ ...(typeof args.accent === 'string' && args.accent.trim() !== '' ? { accent: args.accent.trim() } : {}),
867
+ },
868
+ recommendations: recommendations.map(item => ({
869
+ voice_id: item.voice_id,
870
+ name: item.name,
871
+ source: item.source,
872
+ deletable: item.deletable,
873
+ ...(item.language === undefined ? {} : { language: item.language }),
874
+ ...(item.locale === undefined ? {} : { locale: item.locale }),
875
+ ...(item.accent === undefined ? {} : { accent: item.accent }),
876
+ ...(item.gender === undefined ? {} : { gender: item.gender }),
877
+ ...(item.age === undefined ? {} : { age: item.age }),
878
+ ...(item.use_case === undefined ? {} : { use_case: item.use_case }),
879
+ ...(item.category === undefined ? {} : { category: item.category }),
880
+ ...(item.labels === undefined ? {} : { labels: item.labels }),
881
+ ...(item.descriptive === undefined ? {} : { descriptive: item.descriptive }),
882
+ ...(item.description === undefined ? {} : { description: item.description }),
883
+ ...(item.preview_url === undefined ? {} : { preview_url: item.preview_url }),
884
+ reason: item.reason,
885
+ })),
886
+ }).catch(() => { /* best-effort */ })
887
+ return {
888
+ status: 'ok' as const,
889
+ kind: 'recommend' as const,
890
+ vendor: result.vendor,
891
+ channel: channel.name,
892
+ requirement,
893
+ candidate_count: result.voices.length,
894
+ recommendations,
895
+ ...(result.note === undefined ? {} : { note: result.note }),
896
+ }
897
+ }
650
898
  return {
651
899
  status: 'ok' as const,
652
900
  kind: 'list' as const,
@@ -67,7 +67,7 @@ export function AudioGenPanel(props: { api: AudiogenApi; scope: AudiogenScope })
67
67
  data-active={tab === 'voices' ? 'true' : 'false'}
68
68
  onClick={() => setTab('voices')}
69
69
  >
70
- <MicIcon /> {tt('tab.voices')}
70
+ <MicIcon /> 音色管理
71
71
  </button>
72
72
  </div>
73
73
  </header>