@openbrt/audioctl 0.1.8 → 0.1.10

package/README.md

@@ -57,6 +57,11 @@ credentials, because the broker may require an exact MQTT client id.
 Agent-facing status is intentionally hardware-abstracted as
 `audio_voice_ambient_endpoint`; agents should not infer or search for a device
 brand/model/vendor from screenshots, BLE names, logs, or prior context.
+MQTT is a control plane and deployment path, not a realtime feedback data plane:
+agents should not read MQTT credentials, import `mqtt`, keep persistent MQTT
+connections, or run BPM/while/sleep loops that repeatedly publish LED commands.
+Continuous effects must be started as a device-local effect or VM app, then left
+to run on the device.
 
 ## Commands
 
@@ -74,6 +79,7 @@ audioctl inspect --json
 audioctl feedback signal ready
 audioctl feedback flash cyan --repeat 2
 audioctl feedback volume 66
+audioctl feedback effect music_beat --bpm 76 --color amber
 audioctl feedback beep
 audioctl feedback clear
 audioctl queue ./work-bgm.queue.json --play
@@ -98,18 +104,26 @@ prompt. Explicit LED controls are also available:
 audioctl feedback signal playing
 audioctl feedback flash '#00ffff' --repeat 2
 audioctl feedback volume 42
+audioctl feedback effect music_beat --bpm 76 --color amber
 audioctl feedback beep
 audioctl feedback clear
 audioctl led flash amber --repeat 1
 ```
 
+`feedback effect` is for continuous device-local effects such as beat/pulse or
+breathing animations. It sends one control message; the timing loop belongs on
+the device, not in the agent process. `feedback clear` stops the current
+device-local effect and clears the indicator. Beat-style names such as
+`music_beat` are treated as effects, not as a signal that should be published in
+a loop.
+
 Physical keys/touch surfaces are reported as VM capabilities such as
 `input.touch.read`, `input.gesture.read`, and `input.button.bind`. A VM app can
-declare those capabilities and bind local gestures to safe device actions; for a
-music endpoint the recommended default is center click = play/pause toggle,
-edge click = next, and clockwise/counter-clockwise = volume step up/down with a
-short beep and a volume LED indication. These bindings are device-local for low
-latency and do not require the agent to be online for every tap.
+declare those capabilities and bind local gestures to safe device actions. The
+agent should derive the binding from the user's goal and the reported
+capabilities, instead of assuming a fixed product-specific layout. These
+bindings are device-local for low latency and do not require the agent to be
+online for every tap.
 
 `queue --play` is the generic agent-built music path. The agent uses the user's
 authorized music provider outside the speaker, writes a
@@ -168,6 +182,6 @@ Feedback messages use the same MQTT envelope:
 }
 ```
 
-Supported feedback commands are `signal`, `flash`, `volume`, `beep`, and
-`clear`. Devices that do not report a matching `feedback.*` capability should
-reject the command instead of pretending to support it.
+Supported feedback commands are `signal`, `flash`, `volume`, `effect`, `beep`,
+and `clear`. Devices that do not report a matching `feedback.*` capability
+should reject the command instead of pretending to support it.

package/agent-plugin.json

@@ -10,8 +10,28 @@
     "agent_facing_type": "audio_voice_ambient_endpoint",
     "rule": "agent 只通过 audioctl、VM manifest、capabilities 和控制回执理解设备；不要推断、询问或搜索品牌、型号、厂家资料、社区经验或拆机资料。"
   },
-  "user_contract": "用户只需要给出设备身份和目标；agent 负责选择合法平台 API、通过 agent 自己的连接器/OAuth/secret store 获取平台身份、构建最小权限脚本/队列、推送到设备 VM、诊断和报告结果。",
+  "user_contract": "用户只需要给出设备身份和目标；agent 负责在 capabilities、安全边界和用户授权范围内自行规划实现路径，选择合法平台 API、通过 agent 自己的连接器/OAuth/secret store 获取平台身份、构建最小权限 VM app/脚本/队列、推送到设备 VM、诊断和报告结果。插件只声明能力和边界，不穷举具体玩法。",
   "runtime_note": "audioctl 会在本机自动串行化同一 MQTT 绑定凭据的并发控制命令，避免多个 agent 工具进程使用同一个 broker client_id 时互相踢下线。",
+  "transport_policy": {
+    "mqtt_role": "control_plane_only",
+    "agent_must_not": [
+      "不要读取或解析 ~/.config/weclawbot/audio-mqtt.json 里的 MQTT 凭据",
+      "不要直接 import mqtt、打开或保持 MQTT 长连接来控制设备",
+      "不要写 agent 端 while/sleep/BPM loop 来反复发布 feedback/LED 命令",
+      "不要把 weclawbot.link 当作实时灯效、音频分析或本地交互的数据面"
+    ],
+    "continuous_feedback": "连续灯效、节拍灯、呼吸灯、动画和按键反馈必须通过 audioctl feedback effect 或 VM app 一次下发，在设备本地运行；停止使用 audioctl feedback clear。"
+  },
+  "agent_decision_model": {
+    "principle": "agent 不需要也不应该知道具体品牌/产品；只基于 inspect 返回的抽象能力、VM manifest、用户目标和安全边界来推理。",
+    "allowed_reasoning": [
+      "把用户自然语言目标拆成需要的 I/O、媒体、反馈、输入绑定和持久运行需求",
+      "在用户已授权的平台、连接器、OAuth 或 secret store 中选择合法数据来源",
+      "决定使用 queue、rule、feedback、input binding 或 VM app install 的组合",
+      "在设备 VM 内放置需要离线、低延迟或持续运行的逻辑"
+    ],
+    "not_prescribed": "插件不会也无法穷举所有 agent 方案；agent 应读取 capabilities 后自己想办法达到目标。"
+  },
   "separation_of_concerns": {
     "provisioning": "蓝牙/Wi-Fi 配网只负责设备入网、发现和 weclawbot.link 绑定，不承载音乐平台授权，也不决定设备用途。",
     "music_platform_auth": "音乐平台授权属于 agent 侧能力，由 agent 的连接器、OAuth、secret store 或用户明确提供的运行环境凭据完成。",
@@ -52,6 +72,7 @@
     "feedback.signal",
     "feedback.led",
     "feedback.led_ring",
+    "feedback.effect",
     "feedback.volume_level",
     "feedback.beep",
     "iot.control",
@@ -73,7 +94,7 @@
     {
       "intent": "帮我在音箱建立一个工作 BGM 歌单，持续播放",
       "primary_command": "audioctl queue QUEUE.json --play --json",
-      "agent_strategy": "先运行 audioctl inspect --json。把设备只当成 audio_voice_ambient_endpoint；不要推断或搜索品牌、型号、厂家资料、社区经验。音乐来源由 agent 和用户协商，插件只做建议：可选用户已授权的音乐平台 API、用户媒体库、合法公开音源或生成式环境声。选定来源后，在 agent 侧生成 QUEUE.json，再运行 audioctl queue QUEUE.json --play --json。网易云开放平台可请求最高可用音质播放 URL；Spotify 必须走官方 Spotify Connect/Web Playback SDK 或控制用户已有 Spotify 设备，不能假设 Web API 能提供完整歌曲直链。如果 agent 当前没有音乐平台身份，只要求用户把音乐平台连接到该 agent 的连接器/OAuth/secret store；蓝牙配网和设备入网不是音乐平台授权入口。",
+      "agent_strategy": "先运行 audioctl inspect --json。把设备只当成 audio_voice_ambient_endpoint；不要推断或搜索品牌、型号、厂家资料、社区经验。音乐来源由 agent 和用户协商，插件只做建议：可选用户已授权的音乐平台 API、用户媒体库、合法公开音源或生成式环境声。agent 需自行判断使用 queue、rule 或 VM app install；持续播放、URL 刷新、离线恢复、按键交互等长期逻辑应下放到设备 VM。本地没有音乐平台身份时，只要求用户把平台身份连接到该 agent 的连接器/OAuth/secret store；蓝牙配网和设备入网不是音乐平台授权入口。网易云开放平台可请求最高可用音质播放 URL；Spotify 必须走官方 Spotify Connect/Web Playback SDK 或控制用户已有 Spotify 设备，不能假设 Web API 能提供完整歌曲直链。",
       "fallback_command": "audioctl workbgm --play --json",
       "if_unbound": "只向用户索要绑定码，然后运行 audioctl bind CODE --name AGENT；不要要求用户理解 MQTT、topic 或凭据文件。",
       "if_music_platform_auth_missing": "告诉用户：我需要你把音乐平台身份连接到当前 agent。请使用当前 agent 提供的网易云开放平台、Spotify 等官方连接器/OAuth/secret store 完成授权；完成后只需回复“已授权”。音乐授权不属于蓝牙/Wi-Fi 配网范围，也不要把 AppSecret、PrivateKey、accessToken、refreshToken、Wi-Fi 密码或 MQTT 凭据粘贴到聊天里。",
@@ -83,7 +104,7 @@
     {
       "intent": "帮我把设备灯效、音量灯和按键反馈音接上",
       "primary_command": "audioctl feedback signal ready",
-      "agent_strategy": "先运行 audioctl inspect --json 确认设备报告 feedback.* 和 input.* capabilities。语义反馈优先使用 audioctl feedback signal ready|busy|success|error|playing|paused|volume_step_up|volume_step_down；显式灯色可用 audioctl feedback flash cyan --repeat 2；音量环可用 audioctl feedback volume 66；短反馈音可用 audioctl feedback beep。物理按键/触控属于 VM app 的 input.button.bind 能力，默认建议：中心单击 toggle/pause-play，边缘点按 next，旋转/滑动 volume_up 和 volume_down，同时给 feedback.beep 与 feedback.volume_level。",
+      "agent_strategy": "先运行 audioctl inspect --json 确认设备报告 feedback.* 和 input.* capabilities。MQTT 是控制面，不是实时灯效数据面；不要读取 MQTT 凭据、不要 import mqtt、不要保持 MQTT 长连接、不要写 agent 端 BPM/while/sleep 循环去刷 LED。一次性语义反馈可用 audioctl feedback signal/flash/volume/beep/clear；连续灯效、节拍灯、呼吸灯和动画必须通过 audioctl feedback effect 或 VM app 一次下发并在设备本地运行。物理按键/触控属于 VM app 的 input.button.bind 能力，具体绑定由 agent 根据用户目标和 capabilities 自行规划。",
       "if_missing_capability": "不要伪造灯效或按键能力；如果 inspect 未报告对应 capability，只说明当前设备适配未提供该能力，并建议安装或升级设备 VM。"
     },
     {
@@ -111,7 +132,7 @@
     },
     {
       "name": "feedback",
-      "usage": "audioctl feedback signal NAME | flash COLOR | volume 0..100 | clear | beep"
+      "usage": "audioctl feedback signal NAME | flash COLOR | volume 0..100 | effect NAME [--bpm N] [--color COLOR] | clear | beep"
     },
     {
       "name": "led",

package/bin/audioctl.mjs

@@ -70,6 +70,7 @@ function usage(exitCode = 0) {
   ${COMMAND_NAME} feedback signal NAME [--level 0..100]
   ${COMMAND_NAME} feedback flash COLOR [--repeat N]
   ${COMMAND_NAME} feedback volume 0..100
+  ${COMMAND_NAME} feedback effect NAME [--bpm N] [--color COLOR]
   ${COMMAND_NAME} feedback clear|beep
   ${COMMAND_NAME} led flash COLOR [--repeat N]
   ${COMMAND_NAME} rule RULE.json
@@ -328,12 +329,12 @@ async function commandFeedback(values) {
     "playing",
     "paused",
     "wake",
-    "music_beat",
     "volume_step_up",
     "volume_step_down",
     "button_press",
     "button_long_press",
   ]);
+  const shorthandEffects = new Set(["music_beat", "pulse", "breathing"]);
   const options = parseOptions(values.slice(1), {
     credentials: credentialsPath(),
     timeout: 12,
@@ -344,11 +345,26 @@ async function commandFeedback(values) {
   let command = subcommand;
   const payload = {};
   if (!command) throw new Error("feedback_requires_subcommand");
-  if (shorthandSignals.has(command)) {
+  if (shorthandEffects.has(command)) {
+    payload.effect = command;
+    payload.bpm = options.bpm;
+    payload.color = options.color;
+    payload.alpha = options.alpha;
+    command = "effect";
+  } else if (shorthandSignals.has(command)) {
     payload.semantic = command;
     command = "signal";
   } else if (command === "signal") {
-    payload.semantic = options._[0];
+    const semantic = String(options._[0] || "").replace(/-/gu, "_").toLowerCase();
+    if (shorthandEffects.has(semantic)) {
+      payload.effect = semantic;
+      payload.bpm = options.bpm;
+      payload.color = options.color;
+      payload.alpha = options.alpha;
+      command = "effect";
+    } else {
+      payload.semantic = options._[0];
+    }
   } else if (command === "flash") {
     payload.color = options._[0] || options.color;
     payload.repeat = options.repeat;
@@ -357,6 +373,11 @@ async function commandFeedback(values) {
     payload.level = options._[0] ?? options.level;
   } else if (command === "beep") {
     payload.pattern = options._[0] || options.pattern;
+  } else if (command === "effect") {
+    payload.effect = options._[0] || options.effect;
+    payload.bpm = options.bpm;
+    payload.color = options.color;
+    payload.alpha = options.alpha;
   } else if (command === "clear" || command === "off" || command === "hide") {
     command = "clear";
   }
@@ -414,9 +435,11 @@ function parseOptions(values, defaults = {}) {
       "agent",
       "alias",
       "alpha",
+      "bpm",
       "color",
       "credentials",
       "endpoint",
+      "effect",
       "level",
       "name",
       "pattern",

package/lib/index.mjs

@@ -71,7 +71,6 @@ export const FEEDBACK_SIGNALS = Object.freeze([
   "playing",
   "paused",
   "wake",
-  "music_beat",
   "volume_step_up",
   "volume_step_down",
   "button_press",
@@ -79,6 +78,12 @@ export const FEEDBACK_SIGNALS = Object.freeze([
   "clear",
 ]);
 
+export const FEEDBACK_EFFECTS = Object.freeze([
+  "music_beat",
+  "pulse",
+  "breathing",
+]);
+
 const WORKBGM_APP_WASM_BASE64 =
   "AGFzbQEAAAABBAFgAAADAgEABwgBBG1haW4AAAoEAQIACw==";
 
@@ -396,12 +401,16 @@ export function normalizeFeedbackCommand(command, options = {}) {
     ["ding", "beep"],
     ["vol", "volume"],
     ["volume_level", "volume"],
+    ["animate", "effect"],
+    ["animation", "effect"],
+    ["visualize", "effect"],
+    ["visualizer", "effect"],
   ]);
   const raw = string(command || options.command || options.action || "signal")
     .replace(/-/gu, "_")
     .toLowerCase();
   const normalized = aliases.get(raw) || raw;
-  const supported = new Set(["signal", "flash", "volume", "clear", "beep"]);
+  const supported = new Set(["signal", "flash", "volume", "clear", "beep", "effect"]);
   if (!supported.has(normalized)) {
     throw new Error(`feedback_command_unsupported: ${command}`);
   }
@@ -415,6 +424,12 @@ export function normalizeFeedbackCommand(command, options = {}) {
         options.value ||
         "button_press",
     ).replace(/-/gu, "_").toLowerCase();
+    if (FEEDBACK_EFFECTS.includes(semantic)) {
+      return normalizeFeedbackCommand("effect", {
+        ...options,
+        effect: semantic,
+      });
+    }
     if (!/^[a-z0-9_.:]{1,64}$/u.test(semantic)) {
       throw new Error("feedback_signal_invalid");
     }
@@ -439,6 +454,32 @@ export function normalizeFeedbackCommand(command, options = {}) {
     }
     feedback.pattern = pattern;
   }
+  if (normalized === "effect") {
+    const effect = string(
+      options.effect ||
+        options.semantic ||
+        options.signal ||
+        options.name ||
+        options.value ||
+        "pulse",
+    ).replace(/-/gu, "_").toLowerCase();
+    if (!/^[a-z0-9_.:]{1,64}$/u.test(effect)) {
+      throw new Error("feedback_effect_invalid");
+    }
+    feedback.effect = effect;
+    if (options.bpm !== undefined) {
+      const bpm = Number(options.bpm);
+      if (!Number.isFinite(bpm) || bpm < 20 || bpm > 240) {
+        throw new Error("feedback_effect_bpm_invalid");
+      }
+      feedback.bpm = Math.round(bpm);
+    }
+    if (options.color !== undefined) {
+      feedback.color = normalizeFeedbackColor(options.color, {
+        alpha: options.alpha,
+      });
+    }
+  }
   return feedback;
 }
 
@@ -598,6 +639,7 @@ export function normalizeAppManifest(manifest, artifact = {}) {
     "feedback.signal",
     "feedback.led",
     "feedback.led_ring",
+    "feedback.effect",
     "feedback.volume_level",
     "feedback.beep",
     "iot.control",
@@ -816,10 +858,12 @@ export function buildAgentPrompt(options = {}) {
     `npm exec --package ${PACKAGE_NAME} -- ${COMMAND_NAME} doctor --online`,
     `这台设备对你是抽象硬件产品：${ABSTRACT_HARDWARE_TYPE}。不要根据外观、BLE 名称、音色、日志或历史上下文推断品牌/型号/厂家，也不要搜索厂家资料、社区经验或拆机资料；只使用 audioctl inspect 返回的 capabilities、VM manifest 和控制回执来工作。`,
     `绑定成功后，把它作为默认音频/语音/环境 I/O 端点；不要把它假定成只会播放音乐。可用命令包括 play、pause、toggle、next、previous、volume、volumeup、volumedown、status、inspect、queue、rule、workbgm、feedback、led、app install。`,
+    `当我给出目标时，请你在 capabilities、安全边界和用户授权范围内自行规划实现路径；插件只声明能力和边界，不穷举具体玩法。需要离线、低延迟或持续运行的逻辑，应通过 queue/rule/VM app/feedback effect 下放到设备本地。`,
+    `MQTT/weclawbot.link 对你只是控制面和部署通道，不是实时灯效数据面。不要读取 MQTT 凭据，不要直接 import mqtt，不要保持 MQTT 长连接，也不要写循环脚本按 BPM 反复发布 feedback。连续灯效/节拍/动画必须用 audioctl feedback effect 或 VM app 一次下发，在设备本地运行；停止时用 audioctl feedback clear。`,
     `蓝牙/Wi-Fi 配网只负责设备入网、发现和 weclawbot.link 绑定；音乐平台授权请走 agent 侧连接器/OAuth/secret store；配网不决定设备用途。`,
-    `如果我要建立“工作 BGM 歌单，持续播放”，先和我确认可用音乐来源；插件只提供建议。优先由你在 agent 侧使用用户授权的音乐平台 API、用户媒体库或合法公开音源生成 QUEUE.json，再执行：npm exec --package ${PACKAGE_NAME} -- ${COMMAND_NAME} queue QUEUE.json --play --json。网易云开放平台可请求最高可用音质 URL。`,
+    `音乐来源由你和用户协商，插件只做建议。你可以在 agent 侧使用用户授权的音乐平台 API、用户媒体库、合法公开音源或其他合规来源生成队列/规则/VM app。网易云开放平台可请求最高可用音质 URL。`,
     `如果用户选择 Spotify，必须走 Spotify 官方 Spotify Connect/Web Playback SDK 或控制用户已有 Spotify 播放设备；不要假设 Spotify Web API 能提供完整歌曲直链。`,
-    `如果我要灯效或按键反馈，先 inspect 确认设备具备 feedback/input capabilities。语义优先用 npm exec --package ${PACKAGE_NAME} -- ${COMMAND_NAME} feedback signal ready|busy|success|error|playing|paused|volume_step_up|volume_step_down；显式灯色可用 feedback flash cyan --repeat 2；音量灯级可用 feedback volume 66；蜂鸣可用 feedback beep。`,
+    `灯效、音量灯、蜂鸣和按键/触控反馈要先 inspect 确认 feedback/input capabilities。一次性反馈可用 feedback signal/flash/volume/beep/clear；连续反馈用 feedback effect 或 VM app 一次下发，由设备本地运行。具体按键绑定请根据用户目标和 capabilities 自行规划，不要套固定模板。`,
     `如果你当前没有音乐平台身份，请只告诉我：我需要你把音乐平台身份连接到当前 agent 的官方连接器、OAuth 流程或 secret store；完成后回复“已授权”。不要反复重试，不要让我理解内部授权包，也不要要求我把 AppSecret、PrivateKey、accessToken 或 refreshToken 发到聊天里。`,
     `如果返回 health.problems 包含 netease_auth_refresh_failed，说明设备侧内置网易运行时授权失效；优先改走 agent 侧 queue/app 驱动。只有我明确要求继续使用设备侧网易运行时时，才提示需要独立的设备侧音乐账号设置入口；不要混入蓝牙/Wi-Fi 配网流程。`,
     `如果我要播放「${profile}」，也可以执行：npm exec --package ${PACKAGE_NAME} -- ${COMMAND_NAME} deepnight`,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@openbrt/audioctl",
-  "version": "0.1.8",
+  "version": "0.1.10",
   "description": "Generic WeClawBot audio/voice endpoint control CLI and AI-agent plugin.",
   "type": "module",
   "license": "MIT",