@openbrt/audioctl 0.1.4 → 0.1.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +63 -10
- package/agent-plugin.json +32 -4
- package/bin/audioctl.mjs +69 -0
- package/lib/index.mjs +225 -7
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -19,15 +19,20 @@ Agent contract:
|
|
|
19
19
|
|
|
20
20
|
- If already bound, inspect the device and run the high-level task directly.
|
|
21
21
|
- If not bound, ask the user only for the short binding code, then bind and continue.
|
|
22
|
+
- Bluetooth/Wi-Fi provisioning is only for getting the device online and binding
|
|
23
|
+
it to weclawbot.link. It must not decide whether the speaker is used for
|
|
24
|
+
NetEase, Spotify, voice remote control, smart-home control, or anything else.
|
|
22
25
|
- Preferred BYOA flow: use a legal music platform API in the agent environment
|
|
23
|
-
to build a playable queue, then push it with
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
Spotify Web
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
26
|
+
or another user-approved source to build a playable queue, then push it with
|
|
27
|
+
`audioctl queue QUEUE.json --play --json`. The plugin suggests sources; the
|
|
28
|
+
agent and user choose. NetEase Cloud Music Open Platform can be used this way
|
|
29
|
+
to request the highest available quality URLs. Spotify should be driven through
|
|
30
|
+
official Spotify Connect/Web Playback SDK or existing-user playback control;
|
|
31
|
+
do not assume the Spotify Web API gives full-track direct audio URLs.
|
|
32
|
+
- If music-platform authorization is missing, use the current agent's official
|
|
33
|
+
connector, OAuth flow, or secret store for that platform. Bluetooth/Wi-Fi
|
|
34
|
+
provisioning is not a music-platform authorization entry and is not a
|
|
35
|
+
product-purpose decision point.
|
|
31
36
|
- Never ask the user to paste AppSecret, PrivateKey, accessToken, refreshToken,
|
|
32
37
|
Wi-Fi passwords, MQTT credentials, or other secrets into chat.
|
|
33
38
|
|
|
@@ -61,6 +66,11 @@ audioctl volumeup
|
|
|
61
66
|
audioctl volumedown
|
|
62
67
|
audioctl status
|
|
63
68
|
audioctl inspect --json
|
|
69
|
+
audioctl feedback signal ready
|
|
70
|
+
audioctl feedback flash cyan --repeat 2
|
|
71
|
+
audioctl feedback volume 66
|
|
72
|
+
audioctl feedback beep
|
|
73
|
+
audioctl feedback clear
|
|
64
74
|
audioctl queue ./work-bgm.queue.json --play
|
|
65
75
|
audioctl workbgm --play
|
|
66
76
|
audioctl rule ./my-playlist-rule.json
|
|
@@ -73,6 +83,29 @@ existing MQTT channel. Agent scripts can read Wi-Fi connection state, BLE
|
|
|
73
83
|
provisioning state, wake-word capability, playback, VM/app status, and health
|
|
74
84
|
without receiving Wi-Fi passwords, MQTT credentials, or music-platform secrets.
|
|
75
85
|
|
|
86
|
+
`feedback`/`led` controls expose generic device feedback. Agents should prefer
|
|
87
|
+
semantic signals (`ready`, `busy`, `success`, `error`, `playing`, `paused`,
|
|
88
|
+
`volume_step_up`, `volume_step_down`, `button_press`) so each hardware adapter
|
|
89
|
+
can map them to its native LED ring, small screen, haptic motor, or speaker
|
|
90
|
+
prompt. Explicit LED controls are also available:
|
|
91
|
+
|
|
92
|
+
```bash
|
|
93
|
+
audioctl feedback signal playing
|
|
94
|
+
audioctl feedback flash '#00ffff' --repeat 2
|
|
95
|
+
audioctl feedback volume 42
|
|
96
|
+
audioctl feedback beep
|
|
97
|
+
audioctl feedback clear
|
|
98
|
+
audioctl led flash amber --repeat 1
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
Physical keys/touch surfaces are reported as VM capabilities such as
|
|
102
|
+
`input.touch.read`, `input.gesture.read`, and `input.button.bind`. A VM app can
|
|
103
|
+
declare those capabilities and bind local gestures to safe device actions; for a
|
|
104
|
+
music endpoint the recommended default is center click = play/pause toggle,
|
|
105
|
+
edge click = next, and clockwise/counter-clockwise = volume step up/down with a
|
|
106
|
+
short beep and a volume LED indication. These bindings are device-local for low
|
|
107
|
+
latency and do not require the agent to be online for every tap.
|
|
108
|
+
|
|
76
109
|
`queue --play` is the generic agent-built music path. The agent uses the user's
|
|
77
110
|
authorized music provider outside the speaker, writes a
|
|
78
111
|
`weclawbot.playable_queue.v1`/`schemaVersion: 1` queue containing only safe media
|
|
@@ -89,8 +122,9 @@ If the device reports `netease_auth_refresh_failed`, that only means the legacy
|
|
|
89
122
|
device-side NetEase runtime cannot refresh itself. The better agent behavior is
|
|
90
123
|
to switch to the BYOA queue/app path: obtain the user's music-platform identity
|
|
91
124
|
in the agent environment, build a fresh queue, and push it with `audioctl queue`.
|
|
92
|
-
Only if the user explicitly wants the device-side NetEase runtime should
|
|
93
|
-
agent
|
|
125
|
+
Only if the user explicitly wants the legacy device-side NetEase runtime should
|
|
126
|
+
the agent say a separate device-side music-account settings entry is required.
|
|
127
|
+
That entry must be separate from Bluetooth/Wi-Fi provisioning.
|
|
94
128
|
|
|
95
129
|
`deepnight` pushes the bundled “深夜工作” playlist rule and asks the device to
|
|
96
130
|
refresh/play according to its local runtime. The package never contains music
|
|
@@ -113,3 +147,22 @@ The MQTT control message is:
|
|
|
113
147
|
|
|
114
148
|
Supported commands include `play`, `pause`, `toggle`, `next`, `previous`,
|
|
115
149
|
`seek`, `set_volume`, `volume_up`, `volume_down`, `status`, and `set_rule`.
|
|
150
|
+
|
|
151
|
+
Feedback messages use the same MQTT envelope:
|
|
152
|
+
|
|
153
|
+
```json
|
|
154
|
+
{
|
|
155
|
+
"schema": "weclawbot.control.v1",
|
|
156
|
+
"id": "feedback_<uuid>",
|
|
157
|
+
"kind": "feedback_command",
|
|
158
|
+
"feedback": {
|
|
159
|
+
"command": "flash",
|
|
160
|
+
"color": { "red": 0, "green": 255, "blue": 255, "alpha": 255 },
|
|
161
|
+
"repeat": 2
|
|
162
|
+
}
|
|
163
|
+
}
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
Supported feedback commands are `signal`, `flash`, `volume`, `beep`, and
|
|
167
|
+
`clear`. Devices that do not report a matching `feedback.*` capability should
|
|
168
|
+
reject the command instead of pretending to support it.
|
package/agent-plugin.json
CHANGED
|
@@ -6,10 +6,16 @@
|
|
|
6
6
|
"bin": "audioctl",
|
|
7
7
|
"description": "Bind and drive WeClawBot-compatible audio, voice, and ambient I/O devices from user-owned AI agents.",
|
|
8
8
|
"transport": "weclawbot.link/mqtt",
|
|
9
|
-
"user_contract": "用户只需要给出设备身份和目标;agent 负责选择合法平台 API
|
|
9
|
+
"user_contract": "用户只需要给出设备身份和目标;agent 负责选择合法平台 API、通过 agent 自己的连接器/OAuth/secret store 获取平台身份、构建最小权限脚本/队列、推送到设备 VM、诊断和报告结果。",
|
|
10
|
+
"separation_of_concerns": {
|
|
11
|
+
"provisioning": "蓝牙/Wi-Fi 配网只负责设备入网、发现和 weclawbot.link 绑定,不承载音乐平台授权,也不决定设备用途。",
|
|
12
|
+
"music_platform_auth": "音乐平台授权属于 agent 侧能力,由 agent 的连接器、OAuth、secret store 或用户明确提供的运行环境凭据完成。",
|
|
13
|
+
"device_purpose": "设备用途由 agent 安装的 VM app 和显式 capabilities 决定,不由配网流程决定。"
|
|
14
|
+
},
|
|
10
15
|
"device_classes": [
|
|
11
16
|
"audio_io",
|
|
12
17
|
"audio_output",
|
|
18
|
+
"ambient_io",
|
|
13
19
|
"voice_endpoint",
|
|
14
20
|
"smart_home_endpoint",
|
|
15
21
|
"music_player"
|
|
@@ -35,6 +41,14 @@
|
|
|
35
41
|
"audio.tts",
|
|
36
42
|
"voice.wakeword",
|
|
37
43
|
"voice.capture",
|
|
44
|
+
"input.touch.read",
|
|
45
|
+
"input.gesture.read",
|
|
46
|
+
"input.button.bind",
|
|
47
|
+
"feedback.signal",
|
|
48
|
+
"feedback.led",
|
|
49
|
+
"feedback.led_ring",
|
|
50
|
+
"feedback.volume_level",
|
|
51
|
+
"feedback.beep",
|
|
38
52
|
"iot.control",
|
|
39
53
|
"device.inspect",
|
|
40
54
|
"music.autopilot",
|
|
@@ -54,13 +68,19 @@
|
|
|
54
68
|
{
|
|
55
69
|
"intent": "帮我在音箱建立一个工作 BGM 歌单,持续播放",
|
|
56
70
|
"primary_command": "audioctl queue QUEUE.json --play --json",
|
|
57
|
-
"agent_strategy": "先运行 audioctl inspect --json
|
|
71
|
+
"agent_strategy": "先运行 audioctl inspect --json。音乐来源由 agent 和用户协商,插件只做建议:可选用户已授权的音乐平台 API、用户媒体库、合法公开音源或生成式环境声。选定来源后,在 agent 侧生成 QUEUE.json,再运行 audioctl queue QUEUE.json --play --json。网易云开放平台可请求最高可用音质播放 URL;Spotify 必须走官方 Spotify Connect/Web Playback SDK 或控制用户已有 Spotify 设备,不能假设 Web API 能提供完整歌曲直链。如果 agent 当前没有音乐平台身份,只要求用户把音乐平台连接到该 agent 的连接器/OAuth/secret store;蓝牙配网和设备入网不是音乐平台授权入口。",
|
|
58
72
|
"fallback_command": "audioctl workbgm --play --json",
|
|
59
73
|
"if_unbound": "只向用户索要绑定码,然后运行 audioctl bind CODE --name AGENT;不要要求用户理解 MQTT、topic 或凭据文件。",
|
|
60
|
-
"if_music_platform_auth_missing": "
|
|
61
|
-
"if_device_side_netease_auth_failed": "
|
|
74
|
+
"if_music_platform_auth_missing": "告诉用户:我需要你把音乐平台身份连接到当前 agent。请使用当前 agent 提供的网易云开放平台、Spotify 等官方连接器/OAuth/secret store 完成授权;完成后只需回复“已授权”。音乐授权不属于蓝牙/Wi-Fi 配网范围,也不要把 AppSecret、PrivateKey、accessToken、refreshToken、Wi-Fi 密码或 MQTT 凭据粘贴到聊天里。",
|
|
75
|
+
"if_device_side_netease_auth_failed": "这是设备内置网易运行时的旧授权失效。优先不要让用户折腾设备侧音乐账号;改由 agent 侧使用用户授权的音乐平台 API 生成队列并推送。只有当用户明确要求继续使用设备侧网易运行时时,才提示需要一个独立的设备侧音乐账号设置入口;不要把它混入蓝牙/Wi-Fi 配网流程。",
|
|
62
76
|
"after_user_confirms_authorized": "agent 使用已授权的平台 API 生成 QUEUE.json,运行 audioctl queue QUEUE.json --play --json,然后报告是否 active/degraded。"
|
|
63
77
|
},
|
|
78
|
+
{
|
|
79
|
+
"intent": "帮我把设备灯效、音量灯和按键反馈音接上",
|
|
80
|
+
"primary_command": "audioctl feedback signal ready",
|
|
81
|
+
"agent_strategy": "先运行 audioctl inspect --json 确认设备报告 feedback.* 和 input.* capabilities。语义反馈优先使用 audioctl feedback signal ready|busy|success|error|playing|paused|volume_step_up|volume_step_down;显式灯色可用 audioctl feedback flash cyan --repeat 2;音量环可用 audioctl feedback volume 66;短反馈音可用 audioctl feedback beep。物理按键/触控属于 VM app 的 input.button.bind 能力,默认建议:中心单击 toggle/pause-play,边缘点按 next,旋转/滑动 volume_up 和 volume_down,同时给 feedback.beep 与 feedback.volume_level。",
|
|
82
|
+
"if_missing_capability": "不要伪造灯效或按键能力;如果 inspect 未报告对应 capability,只说明当前设备适配未提供该能力,并建议安装或升级设备 VM。"
|
|
83
|
+
},
|
|
64
84
|
{
|
|
65
85
|
"intent": "让音箱作为语音遥控、家庭智能或语音互动端点",
|
|
66
86
|
"primary_command": "audioctl app install MANIFEST.json APP.wasm",
|
|
@@ -84,6 +104,14 @@
|
|
|
84
104
|
"name": "control",
|
|
85
105
|
"usage": "audioctl play|pause|toggle|next|previous|volumeup|volumedown|status"
|
|
86
106
|
},
|
|
107
|
+
{
|
|
108
|
+
"name": "feedback",
|
|
109
|
+
"usage": "audioctl feedback signal NAME | flash COLOR | volume 0..100 | clear | beep"
|
|
110
|
+
},
|
|
111
|
+
{
|
|
112
|
+
"name": "led",
|
|
113
|
+
"usage": "audioctl led flash COLOR [--repeat N]"
|
|
114
|
+
},
|
|
87
115
|
{
|
|
88
116
|
"name": "volume",
|
|
89
117
|
"usage": "audioctl volume 0..100"
|
package/bin/audioctl.mjs
CHANGED
|
@@ -13,6 +13,7 @@ import {
|
|
|
13
13
|
buildAppInstallControl,
|
|
14
14
|
buildAgentPrompt,
|
|
15
15
|
buildAudioControl,
|
|
16
|
+
buildFeedbackControl,
|
|
16
17
|
buildQueueControl,
|
|
17
18
|
buildWorkBgmAppInstallControl,
|
|
18
19
|
expandPath,
|
|
@@ -52,6 +53,7 @@ async function run(name, values) {
|
|
|
52
53
|
if (name === "deepnight") return commandDeepnight(values);
|
|
53
54
|
if (name === "volume") return commandVolume(values);
|
|
54
55
|
if (name === "seek") return commandSeek(values);
|
|
56
|
+
if (name === "feedback" || name === "led") return commandFeedback(values);
|
|
55
57
|
return commandControl(name, values);
|
|
56
58
|
}
|
|
57
59
|
|
|
@@ -64,6 +66,11 @@ function usage(exitCode = 0) {
|
|
|
64
66
|
${COMMAND_NAME} play|pause|toggle|next|previous|volumeup|volumedown|status
|
|
65
67
|
${COMMAND_NAME} volume 0..100
|
|
66
68
|
${COMMAND_NAME} seek MILLISECONDS
|
|
69
|
+
${COMMAND_NAME} feedback signal NAME [--level 0..100]
|
|
70
|
+
${COMMAND_NAME} feedback flash COLOR [--repeat N]
|
|
71
|
+
${COMMAND_NAME} feedback volume 0..100
|
|
72
|
+
${COMMAND_NAME} feedback clear|beep
|
|
73
|
+
${COMMAND_NAME} led flash COLOR [--repeat N]
|
|
67
74
|
${COMMAND_NAME} rule RULE.json
|
|
68
75
|
${COMMAND_NAME} queue QUEUE.json [--play]
|
|
69
76
|
${COMMAND_NAME} app install MANIFEST.json APP.wasm
|
|
@@ -309,6 +316,62 @@ async function commandSeek(values) {
|
|
|
309
316
|
await send("seek", options, { positionMs: options._[0] });
|
|
310
317
|
}
|
|
311
318
|
|
|
319
|
+
async function commandFeedback(values) {
|
|
320
|
+
const subcommand = String(values[0] || "").replace(/-/gu, "_").toLowerCase();
|
|
321
|
+
const shorthandSignals = new Set([
|
|
322
|
+
"ready",
|
|
323
|
+
"busy",
|
|
324
|
+
"loading",
|
|
325
|
+
"success",
|
|
326
|
+
"error",
|
|
327
|
+
"playing",
|
|
328
|
+
"paused",
|
|
329
|
+
"wake",
|
|
330
|
+
"music_beat",
|
|
331
|
+
"volume_step_up",
|
|
332
|
+
"volume_step_down",
|
|
333
|
+
"button_press",
|
|
334
|
+
"button_long_press",
|
|
335
|
+
]);
|
|
336
|
+
const options = parseOptions(values.slice(1), {
|
|
337
|
+
credentials: credentialsPath(),
|
|
338
|
+
timeout: 12,
|
|
339
|
+
json: false,
|
|
340
|
+
nowait: false,
|
|
341
|
+
});
|
|
342
|
+
|
|
343
|
+
let command = subcommand;
|
|
344
|
+
const payload = {};
|
|
345
|
+
if (!command) throw new Error("feedback_requires_subcommand");
|
|
346
|
+
if (shorthandSignals.has(command)) {
|
|
347
|
+
payload.semantic = command;
|
|
348
|
+
command = "signal";
|
|
349
|
+
} else if (command === "signal") {
|
|
350
|
+
payload.semantic = options._[0];
|
|
351
|
+
} else if (command === "flash") {
|
|
352
|
+
payload.color = options._[0] || options.color;
|
|
353
|
+
payload.repeat = options.repeat;
|
|
354
|
+
payload.alpha = options.alpha;
|
|
355
|
+
} else if (command === "volume") {
|
|
356
|
+
payload.level = options._[0] ?? options.level;
|
|
357
|
+
} else if (command === "beep") {
|
|
358
|
+
payload.pattern = options._[0] || options.pattern;
|
|
359
|
+
} else if (command === "clear" || command === "off" || command === "hide") {
|
|
360
|
+
command = "clear";
|
|
361
|
+
}
|
|
362
|
+
if (options.level !== undefined && payload.level === undefined) payload.level = options.level;
|
|
363
|
+
if (options.volume !== undefined && payload.level === undefined) payload.level = options.volume;
|
|
364
|
+
|
|
365
|
+
const control = buildFeedbackControl(command, payload);
|
|
366
|
+
const credentials = await readCredentials(options.credentials);
|
|
367
|
+
const delivery = await publishAudioControl(credentials, control, {
|
|
368
|
+
timeoutMs: Number(options.timeout) * 1000,
|
|
369
|
+
wait: !options.nowait,
|
|
370
|
+
});
|
|
371
|
+
print(delivery.status || delivery, options.json);
|
|
372
|
+
if (delivery.status?.kind === "rejected") process.exitCode = 1;
|
|
373
|
+
}
|
|
374
|
+
|
|
312
375
|
async function commandControl(name, values) {
|
|
313
376
|
const options = parseOptions(values, {
|
|
314
377
|
credentials: credentialsPath(),
|
|
@@ -349,11 +412,17 @@ function parseOptions(values, defaults = {}) {
|
|
|
349
412
|
if (!new Set([
|
|
350
413
|
"agent",
|
|
351
414
|
"alias",
|
|
415
|
+
"alpha",
|
|
416
|
+
"color",
|
|
352
417
|
"credentials",
|
|
353
418
|
"endpoint",
|
|
419
|
+
"level",
|
|
354
420
|
"name",
|
|
421
|
+
"pattern",
|
|
355
422
|
"profile",
|
|
423
|
+
"repeat",
|
|
356
424
|
"timeout",
|
|
425
|
+
"volume",
|
|
357
426
|
]).has(key)) {
|
|
358
427
|
throw new Error(`option_unknown: --${key}`);
|
|
359
428
|
}
|
package/lib/index.mjs
CHANGED
|
@@ -47,6 +47,36 @@ export const WORKBGM_RULE = Object.freeze({
|
|
|
47
47
|
|
|
48
48
|
export const PLAYABLE_QUEUE_SCHEMA = "weclawbot.playable_queue.v1";
|
|
49
49
|
|
|
50
|
+
export const FEEDBACK_COLOR_NAMES = Object.freeze({
|
|
51
|
+
red: Object.freeze({ red: 255, green: 0, blue: 0, alpha: 255 }),
|
|
52
|
+
green: Object.freeze({ red: 0, green: 255, blue: 64, alpha: 255 }),
|
|
53
|
+
blue: Object.freeze({ red: 0, green: 96, blue: 255, alpha: 255 }),
|
|
54
|
+
cyan: Object.freeze({ red: 0, green: 255, blue: 255, alpha: 255 }),
|
|
55
|
+
yellow: Object.freeze({ red: 255, green: 220, blue: 0, alpha: 255 }),
|
|
56
|
+
amber: Object.freeze({ red: 255, green: 96, blue: 0, alpha: 255 }),
|
|
57
|
+
orange: Object.freeze({ red: 255, green: 96, blue: 0, alpha: 255 }),
|
|
58
|
+
purple: Object.freeze({ red: 180, green: 0, blue: 255, alpha: 255 }),
|
|
59
|
+
white: Object.freeze({ red: 255, green: 255, blue: 255, alpha: 255 }),
|
|
60
|
+
off: Object.freeze({ red: 0, green: 0, blue: 0, alpha: 0 }),
|
|
61
|
+
});
|
|
62
|
+
|
|
63
|
+
export const FEEDBACK_SIGNALS = Object.freeze([
|
|
64
|
+
"ready",
|
|
65
|
+
"busy",
|
|
66
|
+
"loading",
|
|
67
|
+
"success",
|
|
68
|
+
"error",
|
|
69
|
+
"playing",
|
|
70
|
+
"paused",
|
|
71
|
+
"wake",
|
|
72
|
+
"music_beat",
|
|
73
|
+
"volume_step_up",
|
|
74
|
+
"volume_step_down",
|
|
75
|
+
"button_press",
|
|
76
|
+
"button_long_press",
|
|
77
|
+
"clear",
|
|
78
|
+
]);
|
|
79
|
+
|
|
50
80
|
const WORKBGM_APP_WASM_BASE64 =
|
|
51
81
|
"AGFzbQEAAAABBAFgAAADAgEABwgBBG1haW4AAAoEAQIACw==";
|
|
52
82
|
|
|
@@ -176,6 +206,16 @@ export function buildAudioControl(command, options = {}) {
|
|
|
176
206
|
};
|
|
177
207
|
}
|
|
178
208
|
|
|
209
|
+
export function buildFeedbackControl(command, options = {}) {
|
|
210
|
+
const feedback = normalizeFeedbackCommand(command, options);
|
|
211
|
+
return {
|
|
212
|
+
schema: "weclawbot.control.v1",
|
|
213
|
+
id: string(options.id) || `feedback_${crypto.randomUUID()}`,
|
|
214
|
+
kind: "feedback_command",
|
|
215
|
+
feedback,
|
|
216
|
+
};
|
|
217
|
+
}
|
|
218
|
+
|
|
179
219
|
export function buildQueueControl(options = {}) {
|
|
180
220
|
return buildAudioControl("set_queue", {
|
|
181
221
|
id: options.id,
|
|
@@ -225,6 +265,11 @@ export function buildWorkBgmAppInstallControl(options = {}) {
|
|
|
225
265
|
"music.state.read",
|
|
226
266
|
"music.rule.write",
|
|
227
267
|
"music.playback.write",
|
|
268
|
+
"input.touch.read",
|
|
269
|
+
"input.button.bind",
|
|
270
|
+
"feedback.signal",
|
|
271
|
+
"feedback.led_ring",
|
|
272
|
+
"feedback.beep",
|
|
228
273
|
],
|
|
229
274
|
limits: {
|
|
230
275
|
wasm_bytes: 4096,
|
|
@@ -248,6 +293,33 @@ export function buildWorkBgmAppInstallControl(options = {}) {
|
|
|
248
293
|
play_request_id: play ? `play_${crypto.randomUUID()}` : "",
|
|
249
294
|
refresh: true,
|
|
250
295
|
rule: WORKBGM_RULE,
|
|
296
|
+
input_bindings: [
|
|
297
|
+
{
|
|
298
|
+
input: "center.single_click",
|
|
299
|
+
action: "music.toggle",
|
|
300
|
+
feedback: ["feedback.beep", "feedback.signal:button_press"],
|
|
301
|
+
},
|
|
302
|
+
{
|
|
303
|
+
input: "ring.single_click",
|
|
304
|
+
action: "music.next",
|
|
305
|
+
feedback: ["feedback.beep", "feedback.signal:button_press"],
|
|
306
|
+
},
|
|
307
|
+
{
|
|
308
|
+
input: "ring.clockwise",
|
|
309
|
+
action: "music.volume_up",
|
|
310
|
+
feedback: ["feedback.beep", "feedback.volume_level"],
|
|
311
|
+
},
|
|
312
|
+
{
|
|
313
|
+
input: "ring.counter_clockwise",
|
|
314
|
+
action: "music.volume_down",
|
|
315
|
+
feedback: ["feedback.beep", "feedback.volume_level"],
|
|
316
|
+
},
|
|
317
|
+
],
|
|
318
|
+
feedback_policy: {
|
|
319
|
+
button_sound: true,
|
|
320
|
+
volume_led: true,
|
|
321
|
+
playback_led: true,
|
|
322
|
+
},
|
|
251
323
|
},
|
|
252
324
|
},
|
|
253
325
|
});
|
|
@@ -310,6 +382,95 @@ export function normalizeAudioCommand(command, options = {}) {
|
|
|
310
382
|
return music;
|
|
311
383
|
}
|
|
312
384
|
|
|
385
|
+
export function normalizeFeedbackCommand(command, options = {}) {
|
|
386
|
+
const aliases = new Map([
|
|
387
|
+
["hide", "clear"],
|
|
388
|
+
["off", "clear"],
|
|
389
|
+
["baseline", "clear"],
|
|
390
|
+
["light", "flash"],
|
|
391
|
+
["led", "flash"],
|
|
392
|
+
["sound", "beep"],
|
|
393
|
+
["tone", "beep"],
|
|
394
|
+
["ding", "beep"],
|
|
395
|
+
["vol", "volume"],
|
|
396
|
+
["volume_level", "volume"],
|
|
397
|
+
]);
|
|
398
|
+
const raw = string(command || options.command || options.action || "signal")
|
|
399
|
+
.replace(/-/gu, "_")
|
|
400
|
+
.toLowerCase();
|
|
401
|
+
const normalized = aliases.get(raw) || raw;
|
|
402
|
+
const supported = new Set(["signal", "flash", "volume", "clear", "beep"]);
|
|
403
|
+
if (!supported.has(normalized)) {
|
|
404
|
+
throw new Error(`feedback_command_unsupported: ${command}`);
|
|
405
|
+
}
|
|
406
|
+
|
|
407
|
+
const feedback = { command: normalized };
|
|
408
|
+
if (normalized === "signal") {
|
|
409
|
+
const semantic = string(
|
|
410
|
+
options.semantic ||
|
|
411
|
+
options.signal ||
|
|
412
|
+
options.name ||
|
|
413
|
+
options.value ||
|
|
414
|
+
"button_press",
|
|
415
|
+
).replace(/-/gu, "_").toLowerCase();
|
|
416
|
+
if (!/^[a-z0-9_.:]{1,64}$/u.test(semantic)) {
|
|
417
|
+
throw new Error("feedback_signal_invalid");
|
|
418
|
+
}
|
|
419
|
+
feedback.signal = semantic;
|
|
420
|
+
if (options.level !== undefined || options.volume !== undefined) {
|
|
421
|
+
feedback.level = normalizePercent(options.level ?? options.volume);
|
|
422
|
+
}
|
|
423
|
+
}
|
|
424
|
+
if (normalized === "flash") {
|
|
425
|
+
feedback.color = normalizeFeedbackColor(options.color || options.value || "cyan", {
|
|
426
|
+
alpha: options.alpha,
|
|
427
|
+
});
|
|
428
|
+
feedback.repeat = clamp(Number(options.repeat) || 1, 1, 8);
|
|
429
|
+
}
|
|
430
|
+
if (normalized === "volume") {
|
|
431
|
+
feedback.level = normalizePercent(options.level ?? options.volume ?? options.value);
|
|
432
|
+
}
|
|
433
|
+
if (normalized === "beep" && options.pattern !== undefined) {
|
|
434
|
+
const pattern = string(options.pattern);
|
|
435
|
+
if (!/^[a-z0-9_.:-]{1,64}$/u.test(pattern)) {
|
|
436
|
+
throw new Error("feedback_beep_pattern_invalid");
|
|
437
|
+
}
|
|
438
|
+
feedback.pattern = pattern;
|
|
439
|
+
}
|
|
440
|
+
return feedback;
|
|
441
|
+
}
|
|
442
|
+
|
|
443
|
+
export function normalizeFeedbackColor(value, options = {}) {
|
|
444
|
+
if (value && typeof value === "object") {
|
|
445
|
+
return {
|
|
446
|
+
red: clamp(Number(value.red ?? value.r) || 0, 0, 255),
|
|
447
|
+
green: clamp(Number(value.green ?? value.g) || 0, 0, 255),
|
|
448
|
+
blue: clamp(Number(value.blue ?? value.b) || 0, 0, 255),
|
|
449
|
+
alpha: clamp(Number(value.alpha ?? value.a ?? options.alpha ?? 255) || 0, 0, 255),
|
|
450
|
+
};
|
|
451
|
+
}
|
|
452
|
+
const text = string(value || "cyan").toLowerCase();
|
|
453
|
+
const named = FEEDBACK_COLOR_NAMES[text];
|
|
454
|
+
if (named) {
|
|
455
|
+
return {
|
|
456
|
+
red: named.red,
|
|
457
|
+
green: named.green,
|
|
458
|
+
blue: named.blue,
|
|
459
|
+
alpha: clamp(Number(options.alpha ?? named.alpha) || 0, 0, 255),
|
|
460
|
+
};
|
|
461
|
+
}
|
|
462
|
+
const hex = text.replace(/^#/u, "");
|
|
463
|
+
if (/^[0-9a-f]{6}$/iu.test(hex)) {
|
|
464
|
+
return {
|
|
465
|
+
red: Number.parseInt(hex.slice(0, 2), 16),
|
|
466
|
+
green: Number.parseInt(hex.slice(2, 4), 16),
|
|
467
|
+
blue: Number.parseInt(hex.slice(4, 6), 16),
|
|
468
|
+
alpha: clamp(Number(options.alpha ?? 255) || 0, 0, 255),
|
|
469
|
+
};
|
|
470
|
+
}
|
|
471
|
+
throw new Error(`feedback_color_invalid: ${value}`);
|
|
472
|
+
}
|
|
473
|
+
|
|
313
474
|
export function validatePlayableQueue(queue) {
|
|
314
475
|
if (!queue || typeof queue !== "object") {
|
|
315
476
|
throw new Error("queue_invalid");
|
|
@@ -429,6 +590,14 @@ export function normalizeAppManifest(manifest, artifact = {}) {
|
|
|
429
590
|
"audio.tts",
|
|
430
591
|
"voice.wakeword",
|
|
431
592
|
"voice.capture",
|
|
593
|
+
"input.touch.read",
|
|
594
|
+
"input.button.bind",
|
|
595
|
+
"input.gesture.read",
|
|
596
|
+
"feedback.signal",
|
|
597
|
+
"feedback.led",
|
|
598
|
+
"feedback.led_ring",
|
|
599
|
+
"feedback.volume_level",
|
|
600
|
+
"feedback.beep",
|
|
432
601
|
"iot.control",
|
|
433
602
|
]);
|
|
434
603
|
const capabilities = Array.isArray(manifest.capabilities)
|
|
@@ -503,6 +672,44 @@ export function normalizeMusicAutopilot(value) {
|
|
|
503
672
|
play_request_id: string(value.play_request_id).slice(0, 80),
|
|
504
673
|
refresh: value.refresh !== false,
|
|
505
674
|
rule: validateRule(value.rule),
|
|
675
|
+
input_bindings: normalizeInputBindings(value.input_bindings),
|
|
676
|
+
feedback_policy: normalizeFeedbackPolicy(value.feedback_policy),
|
|
677
|
+
};
|
|
678
|
+
}
|
|
679
|
+
|
|
680
|
+
export function normalizeInputBindings(value) {
|
|
681
|
+
if (value === undefined) return [];
|
|
682
|
+
if (!Array.isArray(value)) throw new Error("input_bindings_invalid");
|
|
683
|
+
return value.slice(0, 16).map((binding, index) => {
|
|
684
|
+
const input = string(binding?.input);
|
|
685
|
+
const action = string(binding?.action);
|
|
686
|
+
if (
|
|
687
|
+
!/^[a-z0-9_.:-]{1,80}$/u.test(input) ||
|
|
688
|
+
!/^[a-z0-9_.:-]{1,80}$/u.test(action)
|
|
689
|
+
) {
|
|
690
|
+
throw new Error(`input_binding_invalid:${index + 1}`);
|
|
691
|
+
}
|
|
692
|
+
const feedback = Array.isArray(binding.feedback)
|
|
693
|
+
? binding.feedback.map((item) => string(item)).filter(Boolean).slice(0, 8)
|
|
694
|
+
: [];
|
|
695
|
+
for (const item of feedback) {
|
|
696
|
+
if (!/^[a-z0-9_.:-]{1,80}$/u.test(item)) {
|
|
697
|
+
throw new Error(`input_binding_feedback_invalid:${index + 1}`);
|
|
698
|
+
}
|
|
699
|
+
}
|
|
700
|
+
return { input, action, feedback };
|
|
701
|
+
});
|
|
702
|
+
}
|
|
703
|
+
|
|
704
|
+
export function normalizeFeedbackPolicy(value) {
|
|
705
|
+
if (value === undefined) return {};
|
|
706
|
+
if (!value || typeof value !== "object" || Array.isArray(value)) {
|
|
707
|
+
throw new Error("feedback_policy_invalid");
|
|
708
|
+
}
|
|
709
|
+
return {
|
|
710
|
+
button_sound: value.button_sound === true,
|
|
711
|
+
volume_led: value.volume_led === true,
|
|
712
|
+
playback_led: value.playback_led === true,
|
|
506
713
|
};
|
|
507
714
|
}
|
|
508
715
|
|
|
@@ -555,15 +762,16 @@ export function remediationForStatus(status) {
|
|
|
555
762
|
summary:
|
|
556
763
|
"设备内置网易运行时的用户授权已过期。优先改走 BYOA:由 agent 使用用户授权的音乐平台 API 生成播放队列并推送到设备。",
|
|
557
764
|
user_message:
|
|
558
|
-
"
|
|
765
|
+
"我需要一个连接到当前 agent 的音乐平台身份来生成 BGM 队列。请使用当前 agent 的网易云开放平台、Spotify 等官方连接器/OAuth/secret store 完成授权;完成后回到 agent 说“已授权”。音乐授权不属于蓝牙/Wi-Fi 配网范围,也不要发送任何密钥、token 或密码。",
|
|
559
766
|
agent_after_user_confirms:
|
|
560
767
|
"generate QUEUE.json with the authorized music platform API, then run audioctl queue QUEUE.json --play --json",
|
|
561
768
|
steps: [
|
|
562
|
-
"优先使用 agent 侧 BYOA
|
|
769
|
+
"优先使用 agent 侧 BYOA 音乐驱动:让用户把音乐平台身份连接到当前 agent 的官方连接器、OAuth 流程或 secret store。",
|
|
563
770
|
"网易云开放平台可由 agent 请求最高可用音质 URL,生成 weclawbot.playable_queue.v1 队列,然后运行 audioctl queue QUEUE.json --play --json。",
|
|
564
771
|
"Spotify 必须走官方 Spotify Connect/Web Playback SDK 或控制用户已有 Spotify 设备;不要声称 Spotify Web API 能提供完整歌曲直链。",
|
|
772
|
+
"蓝牙/Wi-Fi 配网只负责设备入网和 weclawbot.link 绑定;不要把音乐平台授权塞进配网流程,也不要让配网决定设备用途。",
|
|
565
773
|
"不要要求用户理解内部授权包,也不要让用户粘贴 AppSecret、PrivateKey、accessToken、refreshToken、Wi-Fi 密码或 MQTT 凭据。",
|
|
566
|
-
"
|
|
774
|
+
"只有用户明确要求继续使用设备侧网易运行时时,才提示需要独立的设备侧音乐账号设置入口;不要把它混入蓝牙/Wi-Fi 配网流程。",
|
|
567
775
|
],
|
|
568
776
|
});
|
|
569
777
|
}
|
|
@@ -602,11 +810,13 @@ export function buildAgentPrompt(options = {}) {
|
|
|
602
810
|
`请在你的运行环境安装 npm 包 ${PACKAGE_NAME},然后执行:`,
|
|
603
811
|
`npm exec --package ${PACKAGE_NAME} -- ${COMMAND_NAME} bind ${code} --name ${shellToken(agentName)}`,
|
|
604
812
|
`npm exec --package ${PACKAGE_NAME} -- ${COMMAND_NAME} doctor --online`,
|
|
605
|
-
`绑定成功后,把它作为默认音频/语音/环境 I/O 端点;不要把它假定成只会播放音乐。可用命令包括 play、pause、toggle、next、previous、volume、volumeup、volumedown、status、inspect、queue、rule、workbgm、app install。`,
|
|
606
|
-
|
|
813
|
+
`绑定成功后,把它作为默认音频/语音/环境 I/O 端点;不要把它假定成只会播放音乐。可用命令包括 play、pause、toggle、next、previous、volume、volumeup、volumedown、status、inspect、queue、rule、workbgm、feedback、led、app install。`,
|
|
814
|
+
`蓝牙/Wi-Fi 配网只负责设备入网、发现和 weclawbot.link 绑定;音乐平台授权请走 agent 侧连接器/OAuth/secret store;配网不决定设备用途。`,
|
|
815
|
+
`如果我要建立“工作 BGM 歌单,持续播放”,先和我确认可用音乐来源;插件只提供建议。优先由你在 agent 侧使用用户授权的音乐平台 API、用户媒体库或合法公开音源生成 QUEUE.json,再执行:npm exec --package ${PACKAGE_NAME} -- ${COMMAND_NAME} queue QUEUE.json --play --json。网易云开放平台可请求最高可用音质 URL。`,
|
|
607
816
|
`如果用户选择 Spotify,必须走 Spotify 官方 Spotify Connect/Web Playback SDK 或控制用户已有 Spotify 播放设备;不要假设 Spotify Web API 能提供完整歌曲直链。`,
|
|
608
|
-
|
|
609
|
-
|
|
817
|
+
`如果我要灯效或按键反馈,先 inspect 确认设备具备 feedback/input capabilities。语义优先用 npm exec --package ${PACKAGE_NAME} -- ${COMMAND_NAME} feedback signal ready|busy|success|error|playing|paused|volume_step_up|volume_step_down;显式灯色可用 feedback flash cyan --repeat 2;音量灯级可用 feedback volume 66;蜂鸣可用 feedback beep。`,
|
|
818
|
+
`如果你当前没有音乐平台身份,请只告诉我:我需要你把音乐平台身份连接到当前 agent 的官方连接器、OAuth 流程或 secret store;完成后回复“已授权”。不要反复重试,不要让我理解内部授权包,也不要要求我把 AppSecret、PrivateKey、accessToken 或 refreshToken 发到聊天里。`,
|
|
819
|
+
`如果返回 health.problems 包含 netease_auth_refresh_failed,说明设备侧内置网易运行时授权失效;优先改走 agent 侧 queue/app 驱动。只有我明确要求继续使用设备侧网易运行时时,才提示需要独立的设备侧音乐账号设置入口;不要混入蓝牙/Wi-Fi 配网流程。`,
|
|
610
820
|
`如果我要播放「${profile}」,也可以执行:npm exec --package ${PACKAGE_NAME} -- ${COMMAND_NAME} deepnight`,
|
|
611
821
|
`不要向设备写入全局音乐平台私钥;平台授权优先保存在 agent 运行环境。只有在我明确授权时,才把最小范围、可轮换、可撤销的 app-scoped 凭据随 VM app 写入设备。`,
|
|
612
822
|
].filter(Boolean).join("\n");
|
|
@@ -793,6 +1003,14 @@ function clamp(value, minimum, maximum) {
|
|
|
793
1003
|
return Math.min(maximum, Math.max(minimum, Math.round(value)));
|
|
794
1004
|
}
|
|
795
1005
|
|
|
1006
|
+
function normalizePercent(value) {
|
|
1007
|
+
const number = Number(value);
|
|
1008
|
+
if (!Number.isFinite(number) || number < 0 || number > 100) {
|
|
1009
|
+
throw new Error("feedback_level_invalid");
|
|
1010
|
+
}
|
|
1011
|
+
return Math.round(number);
|
|
1012
|
+
}
|
|
1013
|
+
|
|
796
1014
|
function string(value) {
|
|
797
1015
|
return typeof value === "string" ? value.trim() : "";
|
|
798
1016
|
}
|