dsh-theme-endfield 1.1.9 → 1.2.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +163 -118
- package/client.js +10186 -10063
- package/docs/PR-description.md +141 -141
- package/docs/audio-notifications.md +133 -133
- package/docs/contour-scan-bounds.md +24 -24
- package/docs/contour-trail.md +61 -61
- package/docs/contour-worker.md +38 -38
- package/docs/design-language.md +212 -212
- package/docs/engineering-notes.md +1138 -1147
- package/docs/features.md +298 -295
- package/docs/glass.md +32 -32
- package/docs/testing.md +427 -427
- package/index.js +1403 -1403
- package/lib/audio.js +426 -426
- package/lib/slots.js +97 -97
- package/lib/tone.js +216 -216
- package/locale/en.json +6 -6
- package/locale/zh.json +6 -6
- package/package.json +113 -105
package/docs/PR-description.md
CHANGED
|
@@ -1,141 +1,141 @@
|
|
|
1
|
-
# PR: 音频通知(Audio notifications)
|
|
2
|
-
|
|
3
|
-
> 这份文件是给上游作者的 PR 描述草稿,可直接粘贴到 GitHub。分支:`feat/audio-notifications`
|
|
4
|
-
|
|
5
|
-
## What this adds
|
|
6
|
-
|
|
7
|
-
Four audio notifications for the web profile, played **by the host process** (not the browser),
|
|
8
|
-
so a minimized window or another app in front still gets the sound:
|
|
9
|
-
|
|
10
|
-
| Slot | Moment | Trigger |
|
|
11
|
-
| --- | --- | --- |
|
|
12
|
-
| `boot` | the boot plate plays | client reports the once-per-page-load run of `runLoader()` |
|
|
13
|
-
| `turn-start` | the user submits from the composer | `agent/inbox/claimed` where the message carries a composer submission credential (`source.rpcId`) and the agent is a root |
|
|
14
|
-
| `turn-done` | the assistant produced a final answer | `agent/turn-stopping` where that turn's last assistant message had visible text |
|
|
15
|
-
| `attention` | a confirmation box that needs the user appears | the page observes the three panels, **and** the host-side `approval/request` / `user-questions/request` waterfalls |
|
|
16
|
-
| `turn-fail` | — | **deliberately unwired**: an error that needs no human decision stays silent |
|
|
17
|
-
|
|
18
|
-
Plus a `05 音频` settings section (master switch, per-slot switches, volume, four preview buttons,
|
|
19
|
-
diagnostics), a loopback bridge (`/theme-endfield/audio/*`), and a zero-dependency tone synthesizer
|
|
20
|
-
so the package ships no binary audio assets.
|
|
21
|
-
|
|
22
|
-
## Why the host plays the sound
|
|
23
|
-
|
|
24
|
-
A browser `AudioContext` is suspended until a user gesture and dies with the tab, so a task that
|
|
25
|
-
finishes while the window is minimized stays silent — which is exactly the case worth notifying.
|
|
26
|
-
`lib/audio.js` hands the file to the platform player instead (`powershell.exe` / `afplay` /
|
|
27
|
-
`paplay`), rescales PCM for volume without touching the system volume, and degrades to a no-op when
|
|
28
|
-
the `subprocess` service is absent.
|
|
29
|
-
|
|
30
|
-
## Two things worth a reviewer's attention
|
|
31
|
-
|
|
32
|
-
### 1. The page observes the UI for `attention` — and only via `data-*` attributes
|
|
33
|
-
|
|
34
|
-
In *some* deployments (including the one this was developed against) `ask_user_question` is provided
|
|
35
|
-
outside the profile's plugin stack, so `dsh-tool-ask-user` never runs and `user-questions/request` is
|
|
36
|
-
never raised. Measured: a question was answered while the host-side counter stayed at `0`.
|
|
37
|
-
|
|
38
|
-
So the client watches for the three confirmation panels. The anchors are the panels' own data
|
|
39
|
-
attributes — `[data-approval-key]`, `[data-plan-review-key]`, `[data-question-key]`.
|
|
40
|
-
|
|
41
|
-
A class-based first attempt was **removed**: `[class*='_card']` matches 15 different components in the
|
|
42
|
-
installed client packages and `[class*='_frame']` matches 8, so opening an unrelated card rang the
|
|
43
|
-
sound. A test now asserts **no marker contains `class`**.
|
|
44
|
-
|
|
45
|
-
Detection is a `MutationObserver` coalesced onto the next animation frame, with a 1 s poll as the
|
|
46
|
-
backstop (an observer bound to a container the app later replaces stops delivering silently; a poll
|
|
47
|
-
cannot). The host still owns the switch, the volume and the debounce; the page only reports "a box is
|
|
48
|
-
on screen".
|
|
49
|
-
|
|
50
|
-
### 2. `tools/execute` is a waterfall — do not listen to it naively
|
|
51
|
-
|
|
52
|
-
An earlier revision of this branch added a `tools/execute` listener as a redundant `ask_user_question`
|
|
53
|
-
trigger. It broke **every tool call in the profile** (`Cannot read properties of undefined (reading
|
|
54
|
-
'isError')`) because `tools/execute` is a waterfall:
|
|
55
|
-
|
|
56
|
-
```js
|
|
57
|
-
// @deepseek-ai/dsh-tools
|
|
58
|
-
const result = await this.ctx.waterfall(carrier, 'tools/execute', mutableExec, () => this.dispatchToolBody(mutableExec));
|
|
59
|
-
```
|
|
60
|
-
|
|
61
|
-
A listener that inspects the call and returns without calling `next()` reports "no result", so the
|
|
62
|
-
tool body never runs. The listener was removed and a structural test now guards it:
|
|
63
|
-
`the plugin never subscribes to the tools/execute waterfall`.
|
|
64
|
-
|
|
65
|
-
## Duration handling
|
|
66
|
-
|
|
67
|
-
The host-derived "still speaking" gate (`playUntil` = the file's own duration + 250 ms) means a long
|
|
68
|
-
user-supplied announcement is never clipped and a short one is not held back — no per-slot setting to
|
|
69
|
-
keep in sync with whatever audio is dropped in. Volume scaling is channel-aware (the first version
|
|
70
|
-
assumed mono and would have left the right channel untouched on stereo input).
|
|
71
|
-
|
|
72
|
-
## Tests
|
|
73
|
-
|
|
74
|
-
```bash
|
|
75
|
-
node test/audio-notify.test.js # host half
|
|
76
|
-
node test/audio-attention-watch.test.js # client half
|
|
77
|
-
node scripts/build-sounds.js --check # generated tones match their definitions
|
|
78
|
-
```
|
|
79
|
-
|
|
80
|
-
The suite covers: composer-credential classification, "only a turn that produced text ends with a
|
|
81
|
-
sound", subagents staying silent, switch/volume gating, silent degradation without `subprocess`,
|
|
82
|
-
user-file override, the volume cache, the duration gate, and `agent/error` staying silent.
|
|
83
|
-
|
|
84
|
-
Full run: 14/14 test files pass, plus `node check.js` and `node selftest.js`.
|
|
85
|
-
**One pre-existing failure is untouched by this branch**: `test/contour-perf.test.js` reports
|
|
86
|
-
`p95 over 120fps budget` on the pristine upstream tree too (verified by stashing this branch).
|
|
87
|
-
|
|
88
|
-
## A note on the commit history
|
|
89
|
-
|
|
90
|
-
Two pairs of commits in this branch show their own development and cancel out:
|
|
91
|
-
|
|
92
|
-
- `290e5dc` adds a `tools/execute` listener, `f77beda` removes it (the waterfall regression explained above)
|
|
93
|
-
- `8d9aaf3` introduces the notification feature and the later commits refine the same code
|
|
94
|
-
|
|
95
|
-
The **aggregate diff is what matters** and is clean; the branch was left linear and honest rather
|
|
96
|
-
than rewritten, so the reasoning is auditable. Squash on merge if you prefer.
|
|
97
|
-
|
|
98
|
-
### Re-authoring the commits
|
|
99
|
-
|
|
100
|
-
Everything in this branch was committed under a placeholder identity (`dsh agent <agent@local>`),
|
|
101
|
-
because the work went through an assistant session. Two ranges need different authors — the
|
|
102
|
-
upstream-only baseline, and this contribution:
|
|
103
|
-
|
|
104
|
-
```bash
|
|
105
|
-
# the two baseline commits contain ONLY upstream code -> keep the upstream author
|
|
106
|
-
git rebase --exec 'git commit --amend --no-edit --author="没用的小废鼠 <107793048+ymh0000123@users.noreply.github.com>"' 6375836
|
|
107
|
-
# then the audio commits
|
|
108
|
-
git rebase --exec 'git commit --amend --no-edit --reset-author --ignore-date' 292e8a0
|
|
109
|
-
```
|
|
110
|
-
|
|
111
|
-
`C:\TestBox\reauthor-and-push.ps1` does exactly this — one amend per commit, because amending a
|
|
112
|
-
commit twice makes the second amend's parent the first one's result and git aborts the rebase
|
|
113
|
-
mid-way — then force-pushes with `--force-with-lease` and verifies the tree hash is unchanged, so the
|
|
114
|
-
rewrite can never silently alter file content.
|
|
115
|
-
|
|
116
|
-
## One unrelated fix included
|
|
117
|
-
|
|
118
|
-
`contourRenderer` was written and read by the settings panel but was **never declared** in
|
|
119
|
-
`FIELD_DEFAULTS`, so it lived only in the page and `test/settings-namespace.test.js` failed on it
|
|
120
|
-
(`client maps keys onto fields the host does NOT declare`). This branch declares it
|
|
121
|
-
(`contourRenderer: 'canvas'`), which is what made that test pass. Happy to split it into its own
|
|
122
|
-
commit/PR if you prefer.
|
|
123
|
-
|
|
124
|
-
## Files
|
|
125
|
-
|
|
126
|
-
| Path | What |
|
|
127
|
-
| --- | --- |
|
|
128
|
-
| `lib/tone.js` | PCM WAV synthesizer + WAV parsing + volume rescale (no deps) |
|
|
129
|
-
| `lib/slots.js` | the four slot definitions (notes, envelopes, durations) |
|
|
130
|
-
| `lib/audio.js` | resolution order, volume cache, platform players, debounce + duration gate |
|
|
131
|
-
| `scripts/build-sounds.js` | regenerates `sounds/*.wav` from `lib/slots.js` (`--check` verifies) |
|
|
132
|
-
| `index.js` | slots/fields/settings registration, host event wiring, `/theme-endfield/audio/*` routes |
|
|
133
|
-
| `client.js` | `05 音频` panel (switches, volume, preview, source read-out) |
|
|
134
|
-
| `docs/audio-notifications.md` | the feature's own documentation |
|
|
135
|
-
|
|
136
|
-
## Not included on purpose
|
|
137
|
-
|
|
138
|
-
The voice-pack wav used in development is **not** committed (1.3 MB of generated binary). Users point
|
|
139
|
-
`audioSoundDir` at their own directory, or drop `<slot>.wav` into the workspace; the resolution order
|
|
140
|
-
is custom dir → workspace → Desktop → bundled tone. This keeps the repository free of binary assets
|
|
141
|
-
while making replacement a one-file drop.
|
|
1
|
+
# PR: 音频通知(Audio notifications)
|
|
2
|
+
|
|
3
|
+
> 这份文件是给上游作者的 PR 描述草稿,可直接粘贴到 GitHub。分支:`feat/audio-notifications`
|
|
4
|
+
|
|
5
|
+
## What this adds
|
|
6
|
+
|
|
7
|
+
Four audio notifications for the web profile, played **by the host process** (not the browser),
|
|
8
|
+
so a minimized window or another app in front still gets the sound:
|
|
9
|
+
|
|
10
|
+
| Slot | Moment | Trigger |
|
|
11
|
+
| --- | --- | --- |
|
|
12
|
+
| `boot` | the boot plate plays | client reports the once-per-page-load run of `runLoader()` |
|
|
13
|
+
| `turn-start` | the user submits from the composer | `agent/inbox/claimed` where the message carries a composer submission credential (`source.rpcId`) and the agent is a root |
|
|
14
|
+
| `turn-done` | the assistant produced a final answer | `agent/turn-stopping` where that turn's last assistant message had visible text |
|
|
15
|
+
| `attention` | a confirmation box that needs the user appears | the page observes the three panels, **and** the host-side `approval/request` / `user-questions/request` waterfalls |
|
|
16
|
+
| `turn-fail` | — | **deliberately unwired**: an error that needs no human decision stays silent |
|
|
17
|
+
|
|
18
|
+
Plus a `05 音频` settings section (master switch, per-slot switches, volume, four preview buttons,
|
|
19
|
+
diagnostics), a loopback bridge (`/theme-endfield/audio/*`), and a zero-dependency tone synthesizer
|
|
20
|
+
so the package ships no binary audio assets.
|
|
21
|
+
|
|
22
|
+
## Why the host plays the sound
|
|
23
|
+
|
|
24
|
+
A browser `AudioContext` is suspended until a user gesture and dies with the tab, so a task that
|
|
25
|
+
finishes while the window is minimized stays silent — which is exactly the case worth notifying.
|
|
26
|
+
`lib/audio.js` hands the file to the platform player instead (`powershell.exe` / `afplay` /
|
|
27
|
+
`paplay`), rescales PCM for volume without touching the system volume, and degrades to a no-op when
|
|
28
|
+
the `subprocess` service is absent.
|
|
29
|
+
|
|
30
|
+
## Two things worth a reviewer's attention
|
|
31
|
+
|
|
32
|
+
### 1. The page observes the UI for `attention` — and only via `data-*` attributes
|
|
33
|
+
|
|
34
|
+
In *some* deployments (including the one this was developed against) `ask_user_question` is provided
|
|
35
|
+
outside the profile's plugin stack, so `dsh-tool-ask-user` never runs and `user-questions/request` is
|
|
36
|
+
never raised. Measured: a question was answered while the host-side counter stayed at `0`.
|
|
37
|
+
|
|
38
|
+
So the client watches for the three confirmation panels. The anchors are the panels' own data
|
|
39
|
+
attributes — `[data-approval-key]`, `[data-plan-review-key]`, `[data-question-key]`.
|
|
40
|
+
|
|
41
|
+
A class-based first attempt was **removed**: `[class*='_card']` matches 15 different components in the
|
|
42
|
+
installed client packages and `[class*='_frame']` matches 8, so opening an unrelated card rang the
|
|
43
|
+
sound. A test now asserts **no marker contains `class`**.
|
|
44
|
+
|
|
45
|
+
Detection is a `MutationObserver` coalesced onto the next animation frame, with a 1 s poll as the
|
|
46
|
+
backstop (an observer bound to a container the app later replaces stops delivering silently; a poll
|
|
47
|
+
cannot). The host still owns the switch, the volume and the debounce; the page only reports "a box is
|
|
48
|
+
on screen".
|
|
49
|
+
|
|
50
|
+
### 2. `tools/execute` is a waterfall — do not listen to it naively
|
|
51
|
+
|
|
52
|
+
An earlier revision of this branch added a `tools/execute` listener as a redundant `ask_user_question`
|
|
53
|
+
trigger. It broke **every tool call in the profile** (`Cannot read properties of undefined (reading
|
|
54
|
+
'isError')`) because `tools/execute` is a waterfall:
|
|
55
|
+
|
|
56
|
+
```js
|
|
57
|
+
// @deepseek-ai/dsh-tools
|
|
58
|
+
const result = await this.ctx.waterfall(carrier, 'tools/execute', mutableExec, () => this.dispatchToolBody(mutableExec));
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
A listener that inspects the call and returns without calling `next()` reports "no result", so the
|
|
62
|
+
tool body never runs. The listener was removed and a structural test now guards it:
|
|
63
|
+
`the plugin never subscribes to the tools/execute waterfall`.
|
|
64
|
+
|
|
65
|
+
## Duration handling
|
|
66
|
+
|
|
67
|
+
The host-derived "still speaking" gate (`playUntil` = the file's own duration + 250 ms) means a long
|
|
68
|
+
user-supplied announcement is never clipped and a short one is not held back — no per-slot setting to
|
|
69
|
+
keep in sync with whatever audio is dropped in. Volume scaling is channel-aware (the first version
|
|
70
|
+
assumed mono and would have left the right channel untouched on stereo input).
|
|
71
|
+
|
|
72
|
+
## Tests
|
|
73
|
+
|
|
74
|
+
```bash
|
|
75
|
+
node test/audio-notify.test.js # host half
|
|
76
|
+
node test/audio-attention-watch.test.js # client half
|
|
77
|
+
node scripts/build-sounds.js --check # generated tones match their definitions
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
The suite covers: composer-credential classification, "only a turn that produced text ends with a
|
|
81
|
+
sound", subagents staying silent, switch/volume gating, silent degradation without `subprocess`,
|
|
82
|
+
user-file override, the volume cache, the duration gate, and `agent/error` staying silent.
|
|
83
|
+
|
|
84
|
+
Full run: 14/14 test files pass, plus `node check.js` and `node selftest.js`.
|
|
85
|
+
**One pre-existing failure is untouched by this branch**: `test/contour-perf.test.js` reports
|
|
86
|
+
`p95 over 120fps budget` on the pristine upstream tree too (verified by stashing this branch).
|
|
87
|
+
|
|
88
|
+
## A note on the commit history
|
|
89
|
+
|
|
90
|
+
Two pairs of commits in this branch show their own development and cancel out:
|
|
91
|
+
|
|
92
|
+
- `290e5dc` adds a `tools/execute` listener, `f77beda` removes it (the waterfall regression explained above)
|
|
93
|
+
- `8d9aaf3` introduces the notification feature and the later commits refine the same code
|
|
94
|
+
|
|
95
|
+
The **aggregate diff is what matters** and is clean; the branch was left linear and honest rather
|
|
96
|
+
than rewritten, so the reasoning is auditable. Squash on merge if you prefer.
|
|
97
|
+
|
|
98
|
+
### Re-authoring the commits
|
|
99
|
+
|
|
100
|
+
Everything in this branch was committed under a placeholder identity (`dsh agent <agent@local>`),
|
|
101
|
+
because the work went through an assistant session. Two ranges need different authors — the
|
|
102
|
+
upstream-only baseline, and this contribution:
|
|
103
|
+
|
|
104
|
+
```bash
|
|
105
|
+
# the two baseline commits contain ONLY upstream code -> keep the upstream author
|
|
106
|
+
git rebase --exec 'git commit --amend --no-edit --author="没用的小废鼠 <107793048+ymh0000123@users.noreply.github.com>"' 6375836
|
|
107
|
+
# then the audio commits
|
|
108
|
+
git rebase --exec 'git commit --amend --no-edit --reset-author --ignore-date' 292e8a0
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
`C:\TestBox\reauthor-and-push.ps1` does exactly this — one amend per commit, because amending a
|
|
112
|
+
commit twice makes the second amend's parent the first one's result and git aborts the rebase
|
|
113
|
+
mid-way — then force-pushes with `--force-with-lease` and verifies the tree hash is unchanged, so the
|
|
114
|
+
rewrite can never silently alter file content.
|
|
115
|
+
|
|
116
|
+
## One unrelated fix included
|
|
117
|
+
|
|
118
|
+
`contourRenderer` was written and read by the settings panel but was **never declared** in
|
|
119
|
+
`FIELD_DEFAULTS`, so it lived only in the page and `test/settings-namespace.test.js` failed on it
|
|
120
|
+
(`client maps keys onto fields the host does NOT declare`). This branch declares it
|
|
121
|
+
(`contourRenderer: 'canvas'`), which is what made that test pass. Happy to split it into its own
|
|
122
|
+
commit/PR if you prefer.
|
|
123
|
+
|
|
124
|
+
## Files
|
|
125
|
+
|
|
126
|
+
| Path | What |
|
|
127
|
+
| --- | --- |
|
|
128
|
+
| `lib/tone.js` | PCM WAV synthesizer + WAV parsing + volume rescale (no deps) |
|
|
129
|
+
| `lib/slots.js` | the four slot definitions (notes, envelopes, durations) |
|
|
130
|
+
| `lib/audio.js` | resolution order, volume cache, platform players, debounce + duration gate |
|
|
131
|
+
| `scripts/build-sounds.js` | regenerates `sounds/*.wav` from `lib/slots.js` (`--check` verifies) |
|
|
132
|
+
| `index.js` | slots/fields/settings registration, host event wiring, `/theme-endfield/audio/*` routes |
|
|
133
|
+
| `client.js` | `05 音频` panel (switches, volume, preview, source read-out) |
|
|
134
|
+
| `docs/audio-notifications.md` | the feature's own documentation |
|
|
135
|
+
|
|
136
|
+
## Not included on purpose
|
|
137
|
+
|
|
138
|
+
The voice-pack wav used in development is **not** committed (1.3 MB of generated binary). Users point
|
|
139
|
+
`audioSoundDir` at their own directory, or drop `<slot>.wav` into the workspace; the resolution order
|
|
140
|
+
is custom dir → workspace → Desktop → bundled tone. This keeps the repository free of binary assets
|
|
141
|
+
while making replacement a one-file drop.
|
|
@@ -1,133 +1,133 @@
|
|
|
1
|
-
# 音频通知(fork 增量)— 实现说明
|
|
2
|
-
|
|
3
|
-
本文件描述 `dsh-theme-endfield` 这个工作树相对上游 `b3e60dc` 新增的**音频通知**功能。
|
|
4
|
-
上游原版的说明在 `README.md`(主题本身的用法)。
|
|
5
|
-
|
|
6
|
-
> 播报词(Terminal 语音文案)见工作区外层的 `endfield-audio-voice-lines.md`;
|
|
7
|
-
> 试玩与还原步骤见 `endfield-audio-tryout.md`。
|
|
8
|
-
|
|
9
|
-
---
|
|
10
|
-
|
|
11
|
-
## 1. 它做什么
|
|
12
|
-
|
|
13
|
-
在四个**真实发生**的时刻各播一声,声音由**宿主进程**播放(不是浏览器),因此页面最小化、切到别的应用、甚至关掉标签页都还能听见:
|
|
14
|
-
|
|
15
|
-
| 槽位 | 时刻 | 判据 |
|
|
16
|
-
| --- | --- | --- |
|
|
17
|
-
| `boot` | 页面加载播放 ENDFIELD 启动加载板 | 客户端在 `runLoader()` 里报告一次,**每次页面加载只报一次**(点「预览」重播不报) |
|
|
18
|
-
| `turn-start` | 用户从会话框提交指令 | 宿主 `agent/inbox/claimed`,且消息带**会话框提交凭据**(`source.rpcId`)且发起者是根 agent |
|
|
19
|
-
| `turn-done` | 助手产出最终结果 | 宿主 `agent/turn-stopping`,且该回合最后一条助手消息**含可见文本** |
|
|
20
|
-
| `attention` | **出现需要人点一下的确认框** | 客户端观察界面上的三个面板(见下),或宿主侧的 `approval/request` / `user-questions/request` 瀑布 |
|
|
21
|
-
| `turn-fail` | ⛔ **不接任何事件** | 故意的:不需要人工干预的错误保持静音 |
|
|
22
|
-
|
|
23
|
-
## 2. 为什么 `attention` 要观察界面
|
|
24
|
-
|
|
25
|
-
在**本部署**里,`ask_user_question` 由 harness 提供,**不在 profile 的插件栈里**(`dsh-tool-ask-user` 未被组合),
|
|
26
|
-
因此 `ctx.userQuestions.ask()` 从不执行、`user-questions/request` 瀑布从不派发。
|
|
27
|
-
|
|
28
|
-
实测证据:确认框在屏幕上并被回答的同时,宿主的 `question` 计数**始终为 0**。
|
|
29
|
-
|
|
30
|
-
所以改成从界面观察——它不关心是谁发起的,只要"需要你点一下"的东西出现就算。
|
|
31
|
-
宿主侧那两条瀑布监听**仍然保留**(在真实 dsh profile 里它们是对的),两条路都汇到同一个槽位,
|
|
32
|
-
宿主侧防抖会把重复上报合并成一声。
|
|
33
|
-
|
|
34
|
-
**锚点只用面板专属的 data 属性,不用类名**:
|
|
35
|
-
|
|
36
|
-
| 面板 | 锚点 |
|
|
37
|
-
| --- | --- |
|
|
38
|
-
| 审批 | `[data-approval-key]` |
|
|
39
|
-
| 计划求批 | `[data-plan-review-key]` |
|
|
40
|
-
| 提问 | `[data-question-key]` |
|
|
41
|
-
|
|
42
|
-
> 类名方案试过并已废弃:`[class*='_card']` 在已安装的客户端包里命中 **15 个**不同组件、`[class*='_frame']` 命中 **8 个**,
|
|
43
|
-
> 打开任意一张卡片都会误响。现在有一条测试断言**任何锚点都不得包含 `class`**。
|
|
44
|
-
|
|
45
|
-
检测方式:`MutationObserver` 即时触发 + 合并到下一帧;**1000ms 轮询作为兜底**
|
|
46
|
-
(观察者若绑定在被替换的容器上会静默失效,轮询不会)。
|
|
47
|
-
|
|
48
|
-
## 3. 防噪规则
|
|
49
|
-
|
|
50
|
-
- **同一槽位 2.5 秒防抖**(`audioDebounceMs`),由宿主独家裁决;
|
|
51
|
-
- **仅根 agent**:子代理、后台驱动不单独发声;
|
|
52
|
-
- **边沿触发**:确认框一直开着不会反复响;
|
|
53
|
-
- 流式输出、每次工具调用、状态栏变化一律不响。
|
|
54
|
-
|
|
55
|
-
## 4. 配置
|
|
56
|
-
|
|
57
|
-
设置页 **设置 › 终末地主题设置 › 05 音频**(字段在 `dsh-theme-endfield` 命名空间;0.1.7 起持久化在 profile 的 `cordis.patch.yml`,旧版宿主回落 `<dshHome>/settings.yaml`):
|
|
58
|
-
|
|
59
|
-
**总开关 `audioEnabled` 默认关闭**:音效是选择加入(opt-in)的功能,升级到带本功能的版本不会自己开始出声。
|
|
60
|
-
总开关打开后,下面的槽位开关(默认开启)决定具体哪几种情形响。
|
|
61
|
-
|
|
62
|
-
| 字段 | 默认 | 含义 |
|
|
63
|
-
| --- | --- | --- |
|
|
64
|
-
| `audioEnabled` | `'0'` | 总开关(**默认关闭**,需手动开启) |
|
|
65
|
-
| `audioVolume` | `'100'` | 音量,对 PCM 采样缩放,**不改系统音量** |
|
|
66
|
-
| `audioBoot` | `'1'` | 启动加载动画音 |
|
|
67
|
-
| `audioTurnStart` | `'1'` | 任务开始音 |
|
|
68
|
-
| `audioTurnDone` | `'1'` | 任务结束音 |
|
|
69
|
-
| `audioAttention` | `'1'` | 需要你回应 |
|
|
70
|
-
| `audioTurnFail` | `'1'` | 出错音(**不接事件**) |
|
|
71
|
-
| `audioDebounceMs` | `'2500'` | 同槽位最小间隔 |
|
|
72
|
-
| `audioSoundDir` | `''` | 自定义音效目录 |
|
|
73
|
-
| `audioHumanOnly` | `'1'` | 开始音只认会话框提交(关掉=宽松模式,调试用) |
|
|
74
|
-
| `audioDiag` | `'0'` | 诊断日志(宿主控制台 + `/state`) |
|
|
75
|
-
|
|
76
|
-
## 5. 自定义音效(覆盖内置电子音)
|
|
77
|
-
|
|
78
|
-
把 wav 按槽位名放进 **自定义目录 → 工作区根目录 → 桌面 → 内置合成音** 的第一处命中位置:
|
|
79
|
-
|
|
80
|
-
```
|
|
81
|
-
boot.wav turn-start.wav turn-done.wav attention.wav turn-fail.wav
|
|
82
|
-
```
|
|
83
|
-
|
|
84
|
-
要求 **16-bit PCM WAV**、44.1kHz、单声道或立体声;推荐 0.5–1.5 秒,**上限 2.5 秒**(防抖窗)。
|
|
85
|
-
设置页「当前音源」行显示每个槽位实际解析到的文件路径,可确认是否已替换成功。
|
|
86
|
-
|
|
87
|
-
内置音不是二进制素材,而是由 `lib/tone.js` 按 `lib/slots.js` 的音符定义**算出来**的:
|
|
88
|
-
|
|
89
|
-
```bash
|
|
90
|
-
node scripts/build-sounds.js # 重新生成 sounds/*.wav
|
|
91
|
-
node scripts/build-sounds.js --check # 校验生成物与定义一致
|
|
92
|
-
```
|
|
93
|
-
|
|
94
|
-
## 6. 桥接路由(宿主提供,页面调用)
|
|
95
|
-
|
|
96
|
-
| 方法 | 路径 | 作用 |
|
|
97
|
-
| --- | --- | --- |
|
|
98
|
-
| `GET` | `/theme-endfield/audio/state` | 槽位快照、解析到的文件、计数、诊断环形日志 |
|
|
99
|
-
| `GET`/`POST` | `/theme-endfield/audio/preview?slot=<id>` | 试听:**强制播放**(跳过防抖),但仍尊重槽位开关 |
|
|
100
|
-
| `POST` | `/theme-endfield/audio/attention` | 页面报告"看到确认框了":**不强制**,尊重开关/音量/防抖 |
|
|
101
|
-
|
|
102
|
-
## 7. 平台与性能
|
|
103
|
-
|
|
104
|
-
| 平台 | 播放器 |
|
|
105
|
-
| --- | --- |
|
|
106
|
-
| Windows | `pwsh.exe` → `powershell.exe`(`-EncodedCommand` + `Media.SoundPlayer`) |
|
|
107
|
-
| macOS | `afplay` |
|
|
108
|
-
| Linux | `paplay` → `aplay` |
|
|
109
|
-
|
|
110
|
-
**实测(同一台开发机,已扣除音频时长)**:`powershell.exe` 5.1 启动 **0.55–0.8 s**;
|
|
111
|
-
`pwsh` 7.6.6(MSI 版)**0.92–0.96 s**;`pwsh` 7(Store 版)**约 1.33 s**。
|
|
112
|
-
即 **PowerShell 7 反而更慢**,`playerCommands()` 的回退顺序在只有 5.1 的机器上正好落到最快的那个。
|
|
113
|
-
要再压缩只能上"常驻播放器",收益(约 0.6 s)不值得其复杂度与静默失效风险。
|
|
114
|
-
|
|
115
|
-
## 8. 不要碰 `tools/execute`
|
|
116
|
-
|
|
117
|
-
`tools/execute` 是 **waterfall**(`dsh-tools`: `await this.ctx.waterfall(carrier, 'tools/execute', mutableExec, () => this.dispatchToolBody(mutableExec))`)。
|
|
118
|
-
一个"检查参数后直接 `return`"的监听器**不调用 `next()`** 就等于宣布"这次工具调用没有结果":
|
|
119
|
-
工具体不执行,失败以空结果回传,**会话里所有后续工具调用都会坏**(表现为 `Cannot read properties of undefined (reading 'isError')`)。
|
|
120
|
-
|
|
121
|
-
本插件因此**不监听**该事件,并有结构性断言守着:`the plugin never subscribes to the tools/execute waterfall`。
|
|
122
|
-
|
|
123
|
-
## 9. 测试
|
|
124
|
-
|
|
125
|
-
```bash
|
|
126
|
-
node test/audio-notify.test.js # 宿主半边:判据、门控、防抖、降级、路由
|
|
127
|
-
node test/audio-attention-watch.test.js # 客户端:锚点、即时检测、边沿、开关
|
|
128
|
-
node scripts/build-sounds.js --check # 内置音与定义一致
|
|
129
|
-
```
|
|
130
|
-
|
|
131
|
-
`test/audio-notify.test.js` 覆盖的关键断言:会话框凭据判定、只有"含可见文本的回合"才播结束音、
|
|
132
|
-
子代理不响、开关与音量 0 静音、无 `subprocess` 时静默降级、自定义文件覆盖、音量缩放走缓存副本、
|
|
133
|
-
以及 `agent/error` **必须不发声**(用户的规则:不需要人工干预的错误静音)。
|
|
1
|
+
# 音频通知(fork 增量)— 实现说明
|
|
2
|
+
|
|
3
|
+
本文件描述 `dsh-theme-endfield` 这个工作树相对上游 `b3e60dc` 新增的**音频通知**功能。
|
|
4
|
+
上游原版的说明在 `README.md`(主题本身的用法)。
|
|
5
|
+
|
|
6
|
+
> 播报词(Terminal 语音文案)见工作区外层的 `endfield-audio-voice-lines.md`;
|
|
7
|
+
> 试玩与还原步骤见 `endfield-audio-tryout.md`。
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
## 1. 它做什么
|
|
12
|
+
|
|
13
|
+
在四个**真实发生**的时刻各播一声,声音由**宿主进程**播放(不是浏览器),因此页面最小化、切到别的应用、甚至关掉标签页都还能听见:
|
|
14
|
+
|
|
15
|
+
| 槽位 | 时刻 | 判据 |
|
|
16
|
+
| --- | --- | --- |
|
|
17
|
+
| `boot` | 页面加载播放 ENDFIELD 启动加载板 | 客户端在 `runLoader()` 里报告一次,**每次页面加载只报一次**(点「预览」重播不报) |
|
|
18
|
+
| `turn-start` | 用户从会话框提交指令 | 宿主 `agent/inbox/claimed`,且消息带**会话框提交凭据**(`source.rpcId`)且发起者是根 agent |
|
|
19
|
+
| `turn-done` | 助手产出最终结果 | 宿主 `agent/turn-stopping`,且该回合最后一条助手消息**含可见文本** |
|
|
20
|
+
| `attention` | **出现需要人点一下的确认框** | 客户端观察界面上的三个面板(见下),或宿主侧的 `approval/request` / `user-questions/request` 瀑布 |
|
|
21
|
+
| `turn-fail` | ⛔ **不接任何事件** | 故意的:不需要人工干预的错误保持静音 |
|
|
22
|
+
|
|
23
|
+
## 2. 为什么 `attention` 要观察界面
|
|
24
|
+
|
|
25
|
+
在**本部署**里,`ask_user_question` 由 harness 提供,**不在 profile 的插件栈里**(`dsh-tool-ask-user` 未被组合),
|
|
26
|
+
因此 `ctx.userQuestions.ask()` 从不执行、`user-questions/request` 瀑布从不派发。
|
|
27
|
+
|
|
28
|
+
实测证据:确认框在屏幕上并被回答的同时,宿主的 `question` 计数**始终为 0**。
|
|
29
|
+
|
|
30
|
+
所以改成从界面观察——它不关心是谁发起的,只要"需要你点一下"的东西出现就算。
|
|
31
|
+
宿主侧那两条瀑布监听**仍然保留**(在真实 dsh profile 里它们是对的),两条路都汇到同一个槽位,
|
|
32
|
+
宿主侧防抖会把重复上报合并成一声。
|
|
33
|
+
|
|
34
|
+
**锚点只用面板专属的 data 属性,不用类名**:
|
|
35
|
+
|
|
36
|
+
| 面板 | 锚点 |
|
|
37
|
+
| --- | --- |
|
|
38
|
+
| 审批 | `[data-approval-key]` |
|
|
39
|
+
| 计划求批 | `[data-plan-review-key]` |
|
|
40
|
+
| 提问 | `[data-question-key]` |
|
|
41
|
+
|
|
42
|
+
> 类名方案试过并已废弃:`[class*='_card']` 在已安装的客户端包里命中 **15 个**不同组件、`[class*='_frame']` 命中 **8 个**,
|
|
43
|
+
> 打开任意一张卡片都会误响。现在有一条测试断言**任何锚点都不得包含 `class`**。
|
|
44
|
+
|
|
45
|
+
检测方式:`MutationObserver` 即时触发 + 合并到下一帧;**1000ms 轮询作为兜底**
|
|
46
|
+
(观察者若绑定在被替换的容器上会静默失效,轮询不会)。
|
|
47
|
+
|
|
48
|
+
## 3. 防噪规则
|
|
49
|
+
|
|
50
|
+
- **同一槽位 2.5 秒防抖**(`audioDebounceMs`),由宿主独家裁决;
|
|
51
|
+
- **仅根 agent**:子代理、后台驱动不单独发声;
|
|
52
|
+
- **边沿触发**:确认框一直开着不会反复响;
|
|
53
|
+
- 流式输出、每次工具调用、状态栏变化一律不响。
|
|
54
|
+
|
|
55
|
+
## 4. 配置
|
|
56
|
+
|
|
57
|
+
设置页 **设置 › 终末地主题设置 › 05 音频**(字段在 `dsh-theme-endfield` 命名空间;0.1.7 起持久化在 profile 的 `cordis.patch.yml`,旧版宿主回落 `<dshHome>/settings.yaml`):
|
|
58
|
+
|
|
59
|
+
**总开关 `audioEnabled` 默认关闭**:音效是选择加入(opt-in)的功能,升级到带本功能的版本不会自己开始出声。
|
|
60
|
+
总开关打开后,下面的槽位开关(默认开启)决定具体哪几种情形响。
|
|
61
|
+
|
|
62
|
+
| 字段 | 默认 | 含义 |
|
|
63
|
+
| --- | --- | --- |
|
|
64
|
+
| `audioEnabled` | `'0'` | 总开关(**默认关闭**,需手动开启) |
|
|
65
|
+
| `audioVolume` | `'100'` | 音量,对 PCM 采样缩放,**不改系统音量** |
|
|
66
|
+
| `audioBoot` | `'1'` | 启动加载动画音 |
|
|
67
|
+
| `audioTurnStart` | `'1'` | 任务开始音 |
|
|
68
|
+
| `audioTurnDone` | `'1'` | 任务结束音 |
|
|
69
|
+
| `audioAttention` | `'1'` | 需要你回应 |
|
|
70
|
+
| `audioTurnFail` | `'1'` | 出错音(**不接事件**) |
|
|
71
|
+
| `audioDebounceMs` | `'2500'` | 同槽位最小间隔 |
|
|
72
|
+
| `audioSoundDir` | `''` | 自定义音效目录 |
|
|
73
|
+
| `audioHumanOnly` | `'1'` | 开始音只认会话框提交(关掉=宽松模式,调试用) |
|
|
74
|
+
| `audioDiag` | `'0'` | 诊断日志(宿主控制台 + `/state`) |
|
|
75
|
+
|
|
76
|
+
## 5. 自定义音效(覆盖内置电子音)
|
|
77
|
+
|
|
78
|
+
把 wav 按槽位名放进 **自定义目录 → 工作区根目录 → 桌面 → 内置合成音** 的第一处命中位置:
|
|
79
|
+
|
|
80
|
+
```
|
|
81
|
+
boot.wav turn-start.wav turn-done.wav attention.wav turn-fail.wav
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
要求 **16-bit PCM WAV**、44.1kHz、单声道或立体声;推荐 0.5–1.5 秒,**上限 2.5 秒**(防抖窗)。
|
|
85
|
+
设置页「当前音源」行显示每个槽位实际解析到的文件路径,可确认是否已替换成功。
|
|
86
|
+
|
|
87
|
+
内置音不是二进制素材,而是由 `lib/tone.js` 按 `lib/slots.js` 的音符定义**算出来**的:
|
|
88
|
+
|
|
89
|
+
```bash
|
|
90
|
+
node scripts/build-sounds.js # 重新生成 sounds/*.wav
|
|
91
|
+
node scripts/build-sounds.js --check # 校验生成物与定义一致
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
## 6. 桥接路由(宿主提供,页面调用)
|
|
95
|
+
|
|
96
|
+
| 方法 | 路径 | 作用 |
|
|
97
|
+
| --- | --- | --- |
|
|
98
|
+
| `GET` | `/theme-endfield/audio/state` | 槽位快照、解析到的文件、计数、诊断环形日志 |
|
|
99
|
+
| `GET`/`POST` | `/theme-endfield/audio/preview?slot=<id>` | 试听:**强制播放**(跳过防抖),但仍尊重槽位开关 |
|
|
100
|
+
| `POST` | `/theme-endfield/audio/attention` | 页面报告"看到确认框了":**不强制**,尊重开关/音量/防抖 |
|
|
101
|
+
|
|
102
|
+
## 7. 平台与性能
|
|
103
|
+
|
|
104
|
+
| 平台 | 播放器 |
|
|
105
|
+
| --- | --- |
|
|
106
|
+
| Windows | `pwsh.exe` → `powershell.exe`(`-EncodedCommand` + `Media.SoundPlayer`) |
|
|
107
|
+
| macOS | `afplay` |
|
|
108
|
+
| Linux | `paplay` → `aplay` |
|
|
109
|
+
|
|
110
|
+
**实测(同一台开发机,已扣除音频时长)**:`powershell.exe` 5.1 启动 **0.55–0.8 s**;
|
|
111
|
+
`pwsh` 7.6.6(MSI 版)**0.92–0.96 s**;`pwsh` 7(Store 版)**约 1.33 s**。
|
|
112
|
+
即 **PowerShell 7 反而更慢**,`playerCommands()` 的回退顺序在只有 5.1 的机器上正好落到最快的那个。
|
|
113
|
+
要再压缩只能上"常驻播放器",收益(约 0.6 s)不值得其复杂度与静默失效风险。
|
|
114
|
+
|
|
115
|
+
## 8. 不要碰 `tools/execute`
|
|
116
|
+
|
|
117
|
+
`tools/execute` 是 **waterfall**(`dsh-tools`: `await this.ctx.waterfall(carrier, 'tools/execute', mutableExec, () => this.dispatchToolBody(mutableExec))`)。
|
|
118
|
+
一个"检查参数后直接 `return`"的监听器**不调用 `next()`** 就等于宣布"这次工具调用没有结果":
|
|
119
|
+
工具体不执行,失败以空结果回传,**会话里所有后续工具调用都会坏**(表现为 `Cannot read properties of undefined (reading 'isError')`)。
|
|
120
|
+
|
|
121
|
+
本插件因此**不监听**该事件,并有结构性断言守着:`the plugin never subscribes to the tools/execute waterfall`。
|
|
122
|
+
|
|
123
|
+
## 9. 测试
|
|
124
|
+
|
|
125
|
+
```bash
|
|
126
|
+
node test/audio-notify.test.js # 宿主半边:判据、门控、防抖、降级、路由
|
|
127
|
+
node test/audio-attention-watch.test.js # 客户端:锚点、即时检测、边沿、开关
|
|
128
|
+
node scripts/build-sounds.js --check # 内置音与定义一致
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
`test/audio-notify.test.js` 覆盖的关键断言:会话框凭据判定、只有"含可见文本的回合"才播结束音、
|
|
132
|
+
子代理不响、开关与音量 0 静音、无 `subprocess` 时静默降级、自定义文件覆盖、音量缩放走缓存副本、
|
|
133
|
+
以及 `agent/error` **必须不发声**(用户的规则:不需要人工干预的错误静音)。
|
|
@@ -1,24 +1,24 @@
|
|
|
1
|
-
# Exact bounds for contour extraction
|
|
2
|
-
|
|
3
|
-
Each row is divided into blocks of 16 cells. A block records the exact minimum
|
|
4
|
-
and maximum of its two vertex rows, including its shared right boundary. One
|
|
5
|
-
range pass is reused for all contour levels; blocks wholly on one side of a
|
|
6
|
-
level skip the original per-cell scan. Retained cells keep their original order,
|
|
7
|
-
interpolation, saddle handling, path stitching and filters. The grid, level
|
|
8
|
-
count, spatial smoothing and drawing functions are unchanged (with one
|
|
9
|
-
exception, noted after the buffer paragraph below).
|
|
10
|
-
|
|
11
|
-
The range buffers are allocated with the field and reused each frame. Calling
|
|
12
|
-
`contourEvaluate()` invalidates the ranges; direct level extraction falls back
|
|
13
|
-
to the full scan until `contourExtract()` prepares matching bounds.
|
|
14
|
-
|
|
15
|
-
> Since v1.1.6 the sampling STEP may coarsen on very large canvases
|
|
16
|
-
> (`contourStepFor()` / `CONTOUR_MAX_CELLS`), and the block layout follows
|
|
17
|
-
> `contourGeom.step` rather than a hard-coded 6. Every ordinary window still
|
|
18
|
-
> samples at the shipped 6px grid, which is what the comparison below covers.
|
|
19
|
-
|
|
20
|
-
Run `npm run test:bounds` for complete coordinate-array comparisons against a
|
|
21
|
-
forced full scan, including partial blocks, equality, plateaus and saddle cells.
|
|
22
|
-
The existing browser render, smoothness, coverage and performance runners remain
|
|
23
|
-
applicable. CPU extraction measurements do not establish GPU presentation FPS or
|
|
24
|
-
video dropped-frame improvements.
|
|
1
|
+
# Exact bounds for contour extraction
|
|
2
|
+
|
|
3
|
+
Each row is divided into blocks of 16 cells. A block records the exact minimum
|
|
4
|
+
and maximum of its two vertex rows, including its shared right boundary. One
|
|
5
|
+
range pass is reused for all contour levels; blocks wholly on one side of a
|
|
6
|
+
level skip the original per-cell scan. Retained cells keep their original order,
|
|
7
|
+
interpolation, saddle handling, path stitching and filters. The grid, level
|
|
8
|
+
count, spatial smoothing and drawing functions are unchanged (with one
|
|
9
|
+
exception, noted after the buffer paragraph below).
|
|
10
|
+
|
|
11
|
+
The range buffers are allocated with the field and reused each frame. Calling
|
|
12
|
+
`contourEvaluate()` invalidates the ranges; direct level extraction falls back
|
|
13
|
+
to the full scan until `contourExtract()` prepares matching bounds.
|
|
14
|
+
|
|
15
|
+
> Since v1.1.6 the sampling STEP may coarsen on very large canvases
|
|
16
|
+
> (`contourStepFor()` / `CONTOUR_MAX_CELLS`), and the block layout follows
|
|
17
|
+
> `contourGeom.step` rather than a hard-coded 6. Every ordinary window still
|
|
18
|
+
> samples at the shipped 6px grid, which is what the comparison below covers.
|
|
19
|
+
|
|
20
|
+
Run `npm run test:bounds` for complete coordinate-array comparisons against a
|
|
21
|
+
forced full scan, including partial blocks, equality, plateaus and saddle cells.
|
|
22
|
+
The existing browser render, smoothness, coverage and performance runners remain
|
|
23
|
+
applicable. CPU extraction measurements do not establish GPU presentation FPS or
|
|
24
|
+
video dropped-frame improvements.
|