agentus 0.0.0-stage → 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (49) hide show
  1. package/LICENSE +202 -0
  2. package/NOTICE +11 -0
  3. package/README.md +459 -2
  4. package/bin/agentus.mjs +147 -0
  5. package/docs/android-notify-contract.md +658 -0
  6. package/docs/refs/aionui-acp-rules.md +46 -0
  7. package/docs/refs/hermes-studio-ui-ideas.md +191 -0
  8. package/docs/transcript-folding.md +139 -0
  9. package/package.json +75 -5
  10. package/packages/server/mcp/plan-server.mjs +217 -0
  11. package/packages/server/mock/agent.mjs +658 -0
  12. package/packages/server/package.json +22 -0
  13. package/packages/server/src/acp/backends.ts +196 -0
  14. package/packages/server/src/acp/errors.ts +46 -0
  15. package/packages/server/src/acp/registry.ts +579 -0
  16. package/packages/server/src/acp/session-manager.ts +1868 -0
  17. package/packages/server/src/auth.ts +688 -0
  18. package/packages/server/src/dashscope.ts +235 -0
  19. package/packages/server/src/fs.ts +165 -0
  20. package/packages/server/src/hotwords.ts +132 -0
  21. package/packages/server/src/index.ts +1404 -0
  22. package/packages/server/src/notify/center.ts +994 -0
  23. package/packages/server/src/notify/types.ts +222 -0
  24. package/packages/server/src/plan/plan.ts +126 -0
  25. package/packages/server/src/settings.ts +469 -0
  26. package/packages/server/src/store/export.ts +209 -0
  27. package/packages/server/src/store/store.ts +1238 -0
  28. package/packages/server/src/term.ts +160 -0
  29. package/packages/server/src/title.ts +109 -0
  30. package/packages/server/src/voice.ts +251 -0
  31. package/packages/shared/package.json +14 -0
  32. package/packages/shared/src/index.ts +271 -0
  33. package/packages/web/dist/assets/index-B52l55_4.css +1 -0
  34. package/packages/web/dist/assets/index-BRgC7oYt.js +246 -0
  35. package/packages/web/dist/coding-agents/hermes.png +0 -0
  36. package/packages/web/dist/coding-agents/qoder.svg +5 -0
  37. package/packages/web/dist/icons/agentus-icon-inverse.svg +301 -0
  38. package/packages/web/dist/icons/agentus-icon.svg +301 -0
  39. package/packages/web/dist/icons/agentus-mark-inverse.svg +301 -0
  40. package/packages/web/dist/icons/agentus-mark.svg +301 -0
  41. package/packages/web/dist/icons/apple-touch-icon-180.png +0 -0
  42. package/packages/web/dist/icons/favicon.ico +0 -0
  43. package/packages/web/dist/icons/icon-192.png +0 -0
  44. package/packages/web/dist/icons/icon-512.png +0 -0
  45. package/packages/web/dist/icons/icon-maskable-512.png +0 -0
  46. package/packages/web/dist/index.html +30 -0
  47. package/packages/web/dist/manifest.webmanifest +17 -0
  48. package/packages/web/dist/sw.js +61 -0
  49. package/scripts/make-cert.sh +101 -0
@@ -0,0 +1,658 @@
1
+ # Notify channel contract (v0.2, as built)
2
+
3
+ The interface between an Agentus server and a phone app, designed so that **adding a
4
+ notification or an interaction never requires a new APK**.
5
+
6
+ | Piece | Where |
7
+ |---|---|
8
+ | Server side (pairing, fan-out, cursors, callbacks, ACP→activity mapping) | `packages/server/src/notify/{types,center}.ts` |
9
+ | Standalone test bed (same code, own ports) | `scripts/notify-testbed.mts` |
10
+ | Contract smoke test (real HTTP + real WS) | `scripts/notify-smoke.mts` → `npm run notify-smoke` |
11
+ | Android client | `android/` (see `android/README.md`) |
12
+
13
+ Design record and the decision trail: workspace `tasks/20261001-agentus/track.md` §34 and
14
+ `android-notify-contract.md` (v0.1 draft).
15
+
16
+ ---
17
+
18
+ ## 1. Shape: the contract is server ↔ app, not page ↔ app
19
+
20
+ - **Server = the only source of truth.** Activities are derived from the *same* `ServerEvent`
21
+ stream the browser already receives (`emit()` in `index.ts`), so "what is happening" has one
22
+ definition.
23
+ - **App = a generic renderer.** It draws `title/progress/actions/input/deeplink` and interprets
24
+ nothing.
25
+ - **Phone behind NAT ⇒ the phone opens the socket.** The server can only push back over a
26
+ connection the phone itself keeps open. One websocket, held by a foreground service.
27
+ - **A page cannot notify.** It cannot outlive its own tab, and a WebView has neither
28
+ `Notification` nor `PushManager`. The web UI's job is to *configure* and *preview*; the path
29
+ is always page → server → phone.
30
+
31
+ ## 2. Contract A — pairing
32
+
33
+ ```http
34
+ POST /api/notify/pair # no operator session needed: this is how a phone gets one
35
+ { "code": "L84JRD5W", # rotating 8-char code (or null + operator credential)
36
+ "deviceName": "Xiaomi 15", "platform": "android", "sdkInt": 36,
37
+ "appVersion": "0.1.0", "schema": 1,
38
+ "capabilities": ["progress","actions","remote_input","channels","deeplink","live_update"] }
39
+
40
+ 200 { "deviceId": "dev_7f3a…", "deviceToken": "…", "baseUrl": "https://host:port",
41
+ "wsUrl": "wss://host:port/api/notify/ws", "serverTime": 1791110000 }
42
+ ```
43
+
44
+ - The code is accepted only from `GET /api/notify/pair-code` (operator credential) and is
45
+ brute-force limited (10 tries / 10 min / IP). Re-pairing with the same device name replaces
46
+ the old row.
47
+ - `capabilities` is **the only version switch**. The server never compares `appVersion`.
48
+ - Device tokens are per device, revocable (`POST /api/notify/devices/revoke`).
49
+
50
+ ## 3. Contract B — ActivityObject (server → app, over the socket)
51
+
52
+ ```json
53
+ { "schema": 1, "op": "upsert", "activityId": "perm:req_9", "revision": 1,
54
+ "kind": "approval", "priority": "high", "ongoing": true, "promotable": true,
55
+ "title": "待你确认:写文件", "body": "会话名 · /etc/hosts",
56
+ "progress": { "value": 0.6, "segments": [ { "length": 6, "color": "#4f8" } ] },
57
+ "channel": { "id": "agent_approval", "name": "权限请求", "importance": "high", "sound": true },
58
+ "actions": [ { "id": "allow_once", "label": "仅此次", "style": "primary" } ],
59
+ "input": { "enabled": true, "placeholder": "或直接回一句" },
60
+ "visibility": "private", "smallIcon": "agentus",
61
+ "open": { "url": "/?session=…", "prefer": "app" }, // §5.2; `deeplink` stays as its old alias
62
+ "deeplink": "/?session=…" }
63
+ ```
64
+
65
+ Rules that make the "no new APK" promise hold:
66
+
67
+ - `op` ∈ `upsert | dismiss | clear_all`; every op is idempotent on `activityId + revision`.
68
+ - A **dismissal carries no title** — the route validates accordingly.
69
+ - **The server defines channels** (`channel.id/name/importance/sound/vibration`); the app creates
70
+ them on demand. Changing importance or sound is therefore a server-side change.
71
+ - Unknown fields/actions/ops must be ignored by the app, never fatal.
72
+ - `revision` guards ordering; the app drops anything at or below the cursor it already applied.
73
+ - **`title` says what happened; `body` carries `<会话名> · <detail>`; `subtitle` stays unused.**
74
+ Android draws `contentText` (our `body`) on the **collapsed** row and `subText` (our `subtitle`) only
75
+ on the **expanded** card, so a name parked in subtitle is a name the operator never sees when they
76
+ glance at the shade — measured, §11.1.7. One shape for every kind: 运行中 / 已完成 / 执行失败 /
77
+ 待你确认:<工具>.
78
+ - **No protocol tokens on a card.** The completion line is 「用时 1m12s」 (the real turn duration,
79
+ `turn-end.durationMs`), never `stopReason: end_turn` — a card is read by a human, not a debugger.
80
+ An approval splits the agent's own tool title (`Approve edit: /var/…` → title `待你确认:Approve edit`,
81
+ body `… · /var/…`) so neither the state line nor the target has to be guessed at.
82
+ - Frames carry a monotonic `seq`; the app acks with `{t:"ack",cursor:N}` and reconnects with
83
+ `?since=N`, so a lock screen that missed an hour still learns the outcome.
84
+
85
+ ## 4. Contract C — the button callback (app → server)
86
+
87
+ ```http
88
+ POST /api/notify/actions # device token (Bearer)
89
+ { "activityId": "perm:req_9", "revision": 1, "actionId": "allow_once",
90
+ "input": null, "ts": 1791110000 }
91
+ 200 { "ok": true, "note": "received" }
92
+ ```
93
+
94
+ - 200 means *received*: the result comes back later as a new ActivityObject (`permission-resolved`
95
+ ⇒ the app dismisses the card).
96
+ - **Replay guard:** the same `(activityId, actionId)` fires once; a second attempt is `409`.
97
+ Re-publishing the activity re-arms its buttons. (This replaced the nonce idea from v0.1 —
98
+ idempotency at the server is easier to get right than a nonce the app must store.)
99
+ - Unknown activity or unknown action ⇒ `409`; bad token ⇒ `401`.
100
+ - Server side, an interaction is one row: `actionRefs[actionId] = { type, … }` handed to
101
+ `NotifyCenter.onAction`. In the cockpit that handler is `notify.onAction(...)` in `index.ts`
102
+ (`ref.type === "permission"` → `mgr.respondPermission(sessionId, requestId, {outcome:"selected",
103
+ optionId})`; `"open"` → nothing to do, the app already navigated). **The lock screen can answer an
104
+ agent's permission prompt** — and that is verified live, not just in theory (§11).
105
+
106
+ ## 5. Contract D — the page-side half (built, without a native bridge)
107
+
108
+ The web cockpit is a first-class participant, and it needed no JS bridge at all — it talks to the
109
+ same HTTP surface the app does:
110
+
111
+ | The cockpit does | Endpoint | Why it is not a "bridge" |
112
+ |---|---|---|
113
+ | shows the pairing string, device list, APK size | `GET /api/notify/settings` | read-only projection of server state |
114
+ | edits the push rules | `POST /api/notify/settings` | server-side state, so a rule change needs **no APK rebuild** |
115
+ | says which session is on screen (30s keep-alive) | `POST /api/notify/presence` | drives the "quiet while I am watching" rule |
116
+ | pushes the current session to the phone | `POST /api/notify/push` | one activity through the same `publish()` |
117
+ | sends one island-shaped frame down one promotion path | `POST /api/notify/push` with `{kind:"island", path:"aosp"|"xiaomi"}` | the same route; `path` is a contract field (§6), not a test hook |
118
+
119
+ Notifications still need **no** JS bridge, and that has not changed. The one bridge the app does inject is
120
+ for the microphone: `window.AgentusMic` (`available()` / `start(rate)` / `stop()`), which the page uses
121
+ **instead of** `getUserMedia` when the WebView's own capture refuses to start (see §11.1.5). It is gated:
122
+ the app only allows it while the loaded page is one of the operator's saved servers, so a page that ends
123
+ up in the WebView cannot quietly record. A `window.AgentusNative.publish()` bridge for foreground-only
124
+ niceties stays reserved and unused; nothing depends on it, and the app still decides nothing on the
125
+ page's behalf.
126
+
127
+ ### 5.1 The rules, and the one deliberate asymmetry
128
+
129
+ | Rule | Off ⇒ | While the operator is watching that session | While a VOICE CALL is up on that session |
130
+ |---|---|---|---|
131
+ | `turnStart` | no "running" card at all | suppressed (a card for the page in front of me is noise) | suppressed (same) |
132
+ | `completion` | no done/failed notification | suppressed | suppressed — the call is already reading that reply out loud |
133
+ | `approval` | no approval notification | **never suppressed** — a decision waiting on a human is the whole reason the phone is in the loop | **never suppressed** either, but delivered **silently** |
134
+ | `quietWhenWatching` | presence is ignored entirely | — | — (a call implies watching) |
135
+
136
+ Two details that are easy to get wrong and are therefore explicit here:
137
+
138
+ - the `dismiss` of the running card on turn-end is **unconditional**, even when that turn-start was
139
+ suppressed — otherwise the phone keeps a zombie "still running" card forever;
140
+ - presence is **in-memory with a 90s TTL**, never persisted: "who is looking" is a fact about right
141
+ now, and a stale one that survived a restart would silently silence the phone.
142
+
143
+ #### 5.1.1 A call is not "watching" — it is stricter, and it is reported as such
144
+
145
+ The operator reported this from a real call: the turn ended, the call read the answer out loud, and the
146
+ phone **dinged** with a 「跑完了」 card. Two independent causes, both fixed server-side:
147
+
148
+ 1. `completion` was persisted but **never read** — the 完成 switch did nothing, and presence was only
149
+ consulted for `turnStart`, so the done card fired even while the operator watched that very session.
150
+ 2. **A call is not "watching".** "Watching" needs an interaction within 120s (`presence.ts`), and on a
151
+ call the hands are free — presence went stale mid-conversation and the phone started notifying again.
152
+
153
+ So presence carries an explicit `call: boolean` (page-reported, written the moment `CallMode` mounts and
154
+ cleared when it hangs up, not on the next 30s tick). The server then:
155
+
156
+ - counts a call as watching for that session (no duplicate cards), and
157
+ - **sends everything else silently**: the frame is never dropped (the operator must be able to see and
158
+ act on it), but its channel loses sound and vibration and drops to low importance.
159
+
160
+ Silence is not cosmetic here. The call's capture is the **native bridge** (`AudioRecord`, §11.1.5) with no
161
+ echo cancellation, so a notification tone played on the phone's speaker is recorded by the very
162
+ microphone the agent is listening on — the workaround for one bug would otherwise feed the other.
163
+
164
+ "What I am watching" is deliberately **not** just "the tab is visible" (`packages/web/src/presence.ts`):
165
+ it also requires an interaction (pointer/key/wheel/touch) within 120s, because a laptop left open with
166
+ the cockpit on screen is nobody watching — and calling that "watching" would swallow exactly the
167
+ notifications the phone is there for. The page reports every 30s while alive; the "I am leaving" write
168
+ on navigate-away is best-effort (a cross-origin unload drops it), which is fine precisely because the
169
+ 90s TTL, not that write, is what guarantees a stale presence cannot keep the phone silent.
170
+
171
+ ### 5.2 The tap target, in the abstract (`open`): the sender names the destination, the device keeps the policy
172
+
173
+ The operator caught the first version of this: 「消息提醒打开后为什么是浏览器呀?应该打开我们应用,然后对应的会话」.
174
+ It was right, and it was a contract bug, not an Android bug: the activity carried a bare `deeplink`, the app
175
+ turned it into an implicit `ACTION_VIEW` on an `https://` URL, and the system handed it to whatever browser
176
+ claims http(s). Nothing in that chain ever said "this belongs to *our* app, and to *this* session".
177
+
178
+ **The sender must not decide where a tap lands, and the receiver must not have to guess.** So the activity
179
+ carries a target object:
180
+
181
+ ```jsonc
182
+ "open": {
183
+ "url": "https://host:port/?session=57f078d2", // absolute, or relative to the server it came from
184
+ "prefer": "app" // "app" | "web" | "auto" (default)
185
+ }
186
+ ```
187
+
188
+ - `url` may be **relative** (`/?session=<id>`): a self-hosted server that sits behind a tunnel does not
189
+ know its own public address, and the one party that *does* is the app holding the connection. A receiver
190
+ resolves a relative url against the server the activity arrived from, and uses an absolute one as-is.
191
+ - `prefer` is a **hint about the sender's intent**, never a command:
192
+ - `app` — the sender asserts an app on this device handles this URL (our cockpit);
193
+ - `web` — always hand it to the platform's web handler (a third-party sender pushing a plain link);
194
+ - `auto` (default) — the receiver's own rule, spelled out below.
195
+
196
+ **The receiver's policy** (the only party that knows what is installed and which origins it trusts):
197
+
198
+ | `prefer` | url origin is one of MY saved servers | url origin is foreign |
199
+ |---|---|---|
200
+ | `app` | explicit intent into this app, which then opens the page and the session | **not** this app — hand it to the platform (chooser) |
201
+ | `web` | the platform's web handler (the app deliberately declines) | the platform's web handler |
202
+ | `auto` | same as `app` | the platform's web handler |
203
+ | *(no `open`, no `deeplink`)* | tapping only brings the app to the front | — |
204
+
205
+ The allow-list is the security boundary, and it is why "the sender decides" alone would be wrong: this app's
206
+ WebView holds an **authenticated** cockpit session. If a payload could name an arbitrary target for in-app
207
+ navigation, any sender — today our own server, tomorrow somebody's webhook — would gain a navigation
208
+ primitive inside the operator's session. A foreign origin is therefore never rendered inside the app; it
209
+ goes to the platform, where the user (not the payload) chooses. Implementation:
210
+ `Notifier.openFor()` / `ownOrigin()` (Android), with the cockpit-side half being `?session=<id>` on the
211
+ web app (`packages/web/src/state.ts#openTarget`), read once and then dropped from the URL so a later
212
+ reconnect cannot override the operator's own choice of session.
213
+
214
+ `deeplink` remains emitted as an alias (`openFor()` in `center.ts` writes both from one place so they can
215
+ never disagree) purely so an app built before `open` keeps a working tap; one capability (`deeplink`) gates
216
+ both spellings. It goes away once no such app is in the field.
217
+
218
+ **Where this is going for third parties** (the reason this is worth abstracting now): the same activity
219
+ object, pushed by anyone. Today the sender is our own server; the missing pieces for a webhook sender are
220
+ (a) a per-sender token + a `source` label so several systems can push into the same app without sharing the
221
+ operator's credentials, and (b) the reverse direction — Agentus POSTing its own events out to a URL. With
222
+ `open` in place, (a) needs no change to the renderer or the tap policy: a sender that has never heard of
223
+ Agentus sends `{title, body, open: {url, prefer: "web"}}` and gets exactly the behaviour it expects.
224
+
225
+ ## 6. Degradation: the server gates on capabilities, the app tries and falls back
226
+
227
+ | Reported capability missing | The server sends | The app does |
228
+ |---|---|---|
229
+ | `progress` | `progress: null` | plain ongoing notification, no bar |
230
+ | `actions` | `actions: []` | tappable card only (deeplink) |
231
+ | `remote_input` | `input: null` | no inline reply |
232
+ | `channels` | `channel: null` | the default Agentus channel |
233
+ | `icon_url` | `iconUrl: null` | (v0.1 does not claim it: no download implemented) |
234
+ | `live_update` | `promotable: false` | — |
235
+ | `xiaomi_focus` | `promotable: false` | — (Xiaomi's own island, see below) |
236
+
237
+ Independently, on the device: API < 36, or the user switched Live Updates off, or the promotion
238
+ is refused ⇒ **silently** the same notification is posted as a normal ongoing one.
239
+
240
+ The two island paths are probed on the device, never assumed, and **the vendor one is asked first**:
241
+
242
+ | Order | Path | When | Guard |
243
+ |---|---|---|---|
244
+ | 1 | Xiaomi HyperOS 焦点通知/超级岛 (`notification.extras["miui.focus.param"]`) | any HyperOS/MIUI; **the payload depends on the ROM's generation** — OS2 (protocol 2) takes `ticker`/`aodTitle`/`baseInfo`/`hintInfo`, OS3 (protocol 3) adds the island node `param_island` (`islandProperty` + `smallIslandArea` + `bigIslandArea{imageTextInfoLeft, imageTextInfoRight{textInfo}}`) | `content://miui.statusbar.notification.public` → `canShowFocus`; generation from `Settings.System["notification_focus_protocol"]` (0/1/2/3) and `persist.sys.feature.island` |
245
+ | 2 | AOSP promoted (`POST_PROMOTED_NOTIFICATIONS`, `ProgressStyle`, `setRequestPromotedOngoing`) | `sdkInt >= 36` **and path 1 declined** | `LiveUpdate.available()` + the user's system switch |
246
+
247
+ Two traps cost a round each, and both fail **silently** (no exception, no log on the system side, just
248
+ "nothing happens"):
249
+
250
+ 1. **Sending one generation's template to the other.** The island node only exists from OS3 on; OS2's
251
+ SystemUI has no template for it. Pick the template from the ROM's reported protocol, not from
252
+ `Build.VERSION`.
253
+ 2. **A field name that looked like the docs.** The 大岛 slot's text node is `textInfo`; an earlier build
254
+ wrote `miui.focus.paramtextInfo`, which is what the published guide renders when `miui.focus.param`
255
+ (the extras key) sits immediately before `textInfo` in a table cell. Cross-checked against a
256
+ community SDK that actually runs on the OS (`HyperNotification`'s Kotlin models).
257
+
258
+ Why the order is part of the logic, not a detail: on a Xiaomi phone the island *is* the vendor
259
+ feature. Asking AOSP first (as v0.3.2 did) handed HyperOS a promoted notification it does not render
260
+ as an island — the notification looked fine and simply never appeared at the top. One path owns the
261
+ notification, and the app's log says which one.
262
+
263
+ **`path` overrides the order, and that is how the unanswerable question gets answered.** The order above
264
+ cannot tell whether HyperOS 3 renders Android 16's *own* Live Update — it never asks. So an activity may
265
+ carry `path: "aosp"` (vendor call skipped) or `path: "xiaomi"` (AOSP call skipped); anything else keeps
266
+ the order. It is a plain field on the activity object, so the cockpit can send one frame down each path
267
+ from 设置 → 手机通知 → 试上岛 (自动通道 / 原生实时动态 / 小米岛) and the app's 「原生实时动态」 button does
268
+ the same locally without a server. The point of putting it in the contract rather than in a test build:
269
+ "does this phone show the island" is now a button, not an APK.
270
+
271
+ ### 6.1 Xiaomi's island is whitelist-only, and the app says so instead of pretending
272
+
273
+ 焦点通知/超级岛 is **not a permission an app can request**. Xiaomi grants it per app on review
274
+ (apply by mail — app name/package/appid, channel, scenario description; the enterprise route also
275
+ wants a company entity), and the OS enables it platform-side. Until then `canShowFocus` answers
276
+ false and the island never appears — while the ordinary notification keeps working
277
+ (`param_v2.filterWhenNoPermission` defaults to false, i.e. nothing is filtered). Consequences:
278
+
279
+ - The app never guesses: `XiaomiFocus.canShowFocus` gates the param blob, and `xiaomi_focus` is
280
+ reported as a capability **only when true** (same for `live_update`: it now requires
281
+ `canPostPromotedNotifications()`, not merely `sdkInt >= 36`).
282
+ - 「上岛自检」 posts a real ongoing/promotable notification through the real renderer, prints the
283
+ full device diagnosis (model, Android release, HyperOS version, `canShowFocus`, a probe of the
284
+ SystemUI methods, AOSP availability) into the app log, ticks for 12 s and cleans up.
285
+ - Where the honest fallback is: an ordinary ongoing notification with progress still shows in the
286
+ shade, lock screen and AOD. To force an island on a phone Xiaomi has not granted, the only route is
287
+ a rooted device with an island module (e.g. the `io.github.hyperisland` LSPosed module, which
288
+ re-renders any app's notifications as 超级岛).
289
+
290
+ As built, the app reports: `progress, actions, remote_input, channels, deeplink`
291
+ (+ `live_update` when API ≥ 36 *and* the platform would promote; + `xiaomi_focus` when HyperOS
292
+ grants focus permission).
293
+
294
+ ## 7. Version policy
295
+
296
+ 1. This file only grows: add fields, never repurpose them.
297
+ 2. Clients ignore what they do not know.
298
+ 3. New behaviour ⇒ new capability string, not a version comparison.
299
+ 4. Every op idempotent; nothing assumes messages arrive (replay by cursor instead).
300
+ 5. An APK rebuild is needed only for (a) a new Android platform API, (b) a capability that
301
+ only native code can provide (widgets, shortcuts), or (c) a defect **in the native renderer** —
302
+ copy, wording and field mapping are server-side and must never require a rebuild, but a renderer
303
+ bug ships as a new APK (§11.1.7 is the worked example: an action with a `RemoteInput` needs a
304
+ mutable `PendingIntent`, and the card it dropped was the approval one).
305
+
306
+ ## 8. What the native side has to own
307
+
308
+ | # | Native surface | As built (v0.1) |
309
+ |---|---|---|
310
+ | 1 | Background connection (WS + cursor + backoff + FGS) | ✅ `NotifyService`, `specialUse` FGS, network callback, 2s→60s backoff |
311
+ | 2 | Notification renderer (channels on demand, icons, idempotent upsert) | ✅ `Notifier` + `LiveUpdate` (API 36 promotion) |
312
+ | 3 | Action callback (receiver + WorkManager-class durability) | ✅ `ActionReceiver` via `goAsync()` + inline reply |
313
+ | 4 | Open the cockpit | ✅ deeplink → browser (`ACTION_VIEW`) |
314
+ | 5 | Permission guidance (notifications, Live Updates, battery) | ⚠️ notifications + the Live Update settings intent; battery-optimisation prompt still missing |
315
+ | 6 | Device registration + token storage | ⚠️ pairing done (code, machine token, or username+password); token in app-private prefs, not the Keystore |
316
+ | 7 | Saved servers (multiple, switchable, password kept) | ✅ `Profiles` + `SecretBox` (AES-256/GCM, Android Keystore) — switching is one tap and re-pairs itself if the token was revoked |
317
+ | 8 | Vendor island (Xiaomi HyperOS 焦点通知/超级岛) | ✅ `XiaomiFocus` — `miui.focus.param` + `canShowFocus` probe; capability reported only when the OS grants it |
318
+ | 9 | Foldable layout (fold/unfold without losing the form) | ✅ `resizeableActivity` + continuity meta-data + `onSaveInstanceState`; two panes at ≥ 720dp, one below |
319
+ | 10 | The cockpit itself, in the app | ✅ a plain `WebView` on the active server, and the app **opens there**; the server list + notification settings sit behind the toolbar's 服务器 button |
320
+ | 11 | An answer to "can this phone show the island at all" | ✅ 「上岛自检」: posts a real ongoing/promotable notification through the real renderer, logs the full device diagnosis, ticks 12 s, cleans up |
321
+ | 12 | Keyboard must shrink the page, not sit on top of it | ✅ ime() inset folded into the root padding (see P13) |
322
+ | 13 | The page's microphone (dictation / call mode) | ✅ two gates, both closed — `PermissionRequest` granted in `onPermissionRequest` **and** the app's own `RECORD_AUDIO` runtime grant (the held request is granted once the system dialog is allowed), plus a warning when the address is http, where `getUserMedia` does not exist at all |
323
+
324
+ ## 9. Android constraints this design answers
325
+
326
+ | # | Constraint | Handling |
327
+ |---|---|---|
328
+ | P1 | Android 15 caps `dataSync` FGS at 6h/24h | `specialUse` + `FOREGROUND_SERVICE_SPECIAL_USE`, with the reason declared in `PROPERTY_SPECIAL_USE_FGS_SUBTYPE` |
329
+ | P2 | `POST_NOTIFICATIONS` is runtime; Live Updates are a user switch | app asks on first run, and offers a button straight to `MANAGE_APP_PROMOTED_NOTIFICATIONS` |
330
+ | P3 | Aggressive OEM background kills | exponential backoff + cursor replay (so being killed only delays, never loses) |
331
+ | P4 | Android 7+ ignores user-installed CAs; the server cert is a self-signed DDNS leaf | `res/raw/agentus_ca.pem` + a composite trust manager (system ∪ our root) and a waived hostname **for that root** |
332
+ | P5 | Cleartext is blocked since Android 9 | `network_security_config`: cleartext only for LAN test addresses, everything else https/wss |
333
+ | P6 | Locked screen loses events | monotonic `seq` + `?since=` + ack (verified in the smoke test) |
334
+ | P7 | The cockpit had no notification concept | activity centre derives activities from the existing event stream |
335
+ | P8 | A notification button is a remote execution path | device token per device + single-use action + `visibility: private` + `VISIBILITY_PRIVATE`; the LAN/pin design keeps the transport from leaking a bearer token in clear |
336
+ | P9 | Streaming agents update constantly | the server throttles/decides; `revision` + `setOnlyAlertOnce` keep updates from re-alerting |
337
+ | P10 | Distribution | one static URL (`/agentus-companion.apk` on the cockpit, `/dl/…` on the test bed), **behind the operator login** — a phone-pairing page and an installable APK are not public material; schema compatibility keeps rebuilds rare |
338
+ | P11 | Xiaomi HyperOS ignores an app's `configChanges` and force-relaunches the activity on fold/unfold | do not fight it: declare `resizeableActivity` + continuity meta-data, accept the recreate, restore the half-typed form from `onSaveInstanceState`, and derive the layout from the current `screenWidthDp` |
339
+ | P12 | The vendor island is gated by a HyperOS-side permission that has no runtime API | probe `canShowFocus`; attach `miui.focus.param` only when it says yes; report the outcome as a capability and in the UI instead of silently doing nothing (Xiaomi grants it per app on review — see §6.1) |
340
+ | P13 | Since Android 15 / targetSdk 35, `windowSoftInputMode="adjustResize"` no longer resizes the window when the keyboard appears | handle `WindowInsets.Type.ime()` in the same listener that handles the system bars and pad the root by `max(bars, ime)` — padding the root is what makes a WebView (and any form) genuinely shrink, instead of the keyboard covering the page (verified: WebView 1688 px → 848 px → 1688 px around the IME on the emulator) |
341
+ | P14 | A page in a WebView needs TWO microphone gates, and denying the page to ask the system guarantees the first press fails | grant the `PermissionRequest` **and** hold the app's own `RECORD_AUDIO`; never `deny()` first — hold the request and grant that same object once the system dialog is allowed (`NotReadableError: Could not start audio source` is what a missing app-side grant looks like, not a "permission denied") |
342
+ | P15 | `getUserMedia` only exists in a secure context | an `http://<LAN-IP>` address has no `navigator.mediaDevices` at all (measured in the WebView: the page reports none), so voice needs the https address or `localhost`; the app states this in its own status panel instead of letting the page fail cryptically |
343
+ | P16 | The WebView's own capture can refuse to start even when every gate is green (secure origin, `RECORD_AUDIO` granted, 18/18 page requests granted) | this is Chromium's pipeline, not the app's permissions. Two answers, in order: declare `MODIFY_AUDIO_SETTINGS` (Chromium's audio input switches the AudioManager mode before it opens the source, and several reports of this exact error name that permission as the fix), and give the page a native `AudioRecord` bridge to fall back on — the app can record where the WebView cannot (see §11.1.5) |
344
+ | P17 | A `@JavascriptInterface` method runs on the JavaBridge thread, and any WebView call from there throws | never touch the `WebView` inside the bridge: compute what it needs on the UI thread (`onPageStarted` / `onPageFinished`) and only read a `volatile` flag there. Also wrap the whole method body — an escaping `RuntimeException` reaches the page as the useless `Java exception was raised during method invocation`, and wrap every frame-posting loop in a log, or "0 frames arrived" has no cause anywhere |
345
+ | P18 | `View.post()` on a DETACHED view queues the runnable until the view is attached | the app shows its config screen by detaching the WebView on purpose (so going back does not reload the cockpit), which silently starved the microphone bridge: `送出帧=60 · 收到帧=0`. Post to `new Handler(Looper.getMainLooper())` instead, and keep **two** counters ("sent" and "received") — they are different bugs |
346
+ | P19 | A notification channel's sound and importance are **immutable after creation** (Android 8+), and the app deliberately returns early for an existing channel id | "send the same channel with `sound:false`" is a silent no-op: the phone keeps dinging. A muted frame therefore carries `channel.muted: true` and the app routes it to **one** dedicated quiet channel (`agentus-quiet`, 通话中(静音), importance low, no sound) — one extra channel beats a channel-setting mutation dance that would also clear the notifications already in it |
347
+ | P21 | An implicit `ACTION_VIEW` on an `https://` URL is the **browser's** notification to handle, not yours — the operator tapped a message reminder and got Chrome | the tap intent must be **explicit** (`Intent(ctx, MainActivity.class)`) whenever the target is one of the operator's own servers, and the rest of the time it must go to the platform on purpose (`open.prefer`, §5.2). "Open the URL" is not a behaviour a notification can own; "open it here, or hand it over" is |
348
+ | P20 | Presence that requires "an interaction within 120s" is wrong during a call: the hands are free | report `call` explicitly and write it the moment the call mounts / hangs up (not on the 30s tick), treat a call as watching for that session, and silence everything else. Verified on the emulator: the same push lands in `agent_done` (importance 4, system sound) with no call and in `agentus-quiet` (importance 2, `mSound=null`, `SILENT`) during one |
349
+ | P22 | An action that carries a `RemoteInput` must be posted with a **mutable** `PendingIntent` on Android 12+ (`FLAG_IMMUTABLE` there makes SystemUI drop the WHOLE notification, not just the action) | the inline-reply action gets `FLAG_MUTABLE` (`Build.VERSION.SDK_INT >= 31` guard; minSdk is 26), every other action stays immutable. How it fails when wrong, measured: logcat `Not posted. PendingIntents attached to actions with remote inputs must be mutable`, `dumpsys notification \| grep -c channel=agent_approval` ⇒ `0`, while the running/done cards post normally — a card that never appears and never errors in our own code (§11.1.7) |
350
+
351
+ ## 10. Endpoints as built
352
+
353
+ | Method | Path | Auth | Purpose |
354
+ |---|---|---|---|
355
+ | GET | `/api/notify/health` | none | liveness; echoes the code only to an operator |
356
+ | GET | `/api/notify/pair-code` | operator | the code + a ready-to-paste `agentus://pair?u=…&c=…` |
357
+ | POST | `/api/notify/pair-code/rotate` | operator | mint a new code |
358
+ | POST | `/api/notify/pair` | code *or* operator | register a device, get its token |
359
+ | GET | `/api/notify/devices` | operator | list (with `online`) |
360
+ | POST | `/api/notify/devices/revoke` | operator | drop a device and close its socket |
361
+ | POST | `/api/notify/activities` | operator | publish any payload (the preview/debug hook) |
362
+ | GET | `/api/notify/settings` | operator | rules + pairing string + presence + device list in one call (what 设置 → 手机通知 renders) |
363
+ | POST | `/api/notify/settings` | operator | change the rules (`{rules:{turnStart,approval,completion,quietWhenWatching}}`), persisted |
364
+ | POST | `/api/notify/presence` | operator | which session the cockpit is showing (`{sessionId, visible}`), in-memory + 90s TTL. `{call:true}` adds "and I am on a voice call with it": the server then suppresses that session's turn cards and sends every other frame **silently** (`channel.muted`) — a notification tone would be recorded by the call's own microphone |
365
+ | POST | `/api/notify/push` | operator | "push this session to my phone": one activity through the normal `publish()`. Options: `kind: "note"|"approval"|"island"` (`island` = an ongoing promotable card, the only shape either path can promote), `path: "aosp"|"xiaomi"` to force one island API, and `open: {prefer}` to say how a tap should land (§5.2 — `prefer:"web"` is what a third-party sender wants) |
366
+ | POST | `/api/notify/probe` | operator *or* device | canned sequence: running → progress → approval → done |
367
+ | GET | `/api/notify/events?since=N` | operator | replay log as JSON |
368
+ | POST | `/api/notify/actions` | device | a button or a reply |
369
+ | POST | `/api/notify/diag` | device | the device's own report (version, ROM probes, the payload it would attach, mic state); stored on the device record and readable via `/api/notify/devices` — the only way to diagnose a phone that cannot be plugged in |
370
+ | WS | `/api/notify/ws?token=…&since=…&v=<appVersion>` | device | `hello` → replay → live `activity` frames; `v` refreshes the recorded build on every connect |
371
+
372
+ In the cockpit these run **ahead of the operator gate** (a phone has no session cookie); the
373
+ operator-only routes inside re-check their own credential.
374
+
375
+ ## 11. Verification
376
+
377
+ - `npm run notify-smoke` — **37/37**, over real HTTP + real websocket: wrong code refused, both
378
+ pairing paths, socket auth, fan-out, revision/dismiss frames, capability degradation (keeps
379
+ what it reported, nulls what it did not), cursor replay (`since=0` replays, a later cursor
380
+ skips), button accepted once then `409`, unknown action refused, bad token `401`, re-publish
381
+ re-arms.
382
+ - `npx tsx scripts/qa/notify-cockpit-e2e.mts` — **15/15 against the LIVE cockpit** (manual, needs a
383
+ running server + a real agent): pairs a simulated device, creates a real session, sends a prompt.
384
+ The agent's `turn-start` arrived as an ongoing, promotable activity whose deeplink points at that
385
+ session; its **permission request arrived as an approval activity whose buttons are the real ACP
386
+ options** (`allow_once` / `deny`, title `Approve edit: …`); pressing `allow_once` over the device
387
+ token made the cockpit resolve the permission and the agent continue (the card was dismissed by
388
+ `permission-resolved`); `turn-end` arrived as a **non-ongoing** completion activity. The scratch
389
+ session is closed at the end.
390
+ Note: the trigger matters — a prompt that makes the agent use the write tool asks; asking it to `rm`
391
+ something outside the workspace did not.
392
+ - `auth-smoke` PASS and `workspace-smoke` 66/66 — the routing change (`/api/notify/*` ahead of the
393
+ operator gate) did not disturb the existing server.
394
+ - `npm run typecheck -w @agentus/server` clean.
395
+ - APK (v0.3.0): `assembleRelease` OK; `aapt dump badging` shows `versionCode 3` / `targetSdk 36` + all
396
+ permissions; `apksigner verify` OK (v2, debug key); `dexdump` confirms the shipped bytecode calls
397
+ `Notification$Builder.setRequestPromotedOngoing(Z)` and `Notification$ProgressStyle`; the CA is
398
+ packaged as `raw/agentus_ca` → `res/9n.pem`; `strings` on `classes.dex` finds the Xiaomi island
399
+ keys (`miui.focus.param`, `canShowFocus`) and both `resizeableActivity` flags survive into the
400
+ merged manifest.
401
+ - Artifact delivery: `GET /agentus-companion.apk` answers **302 → `/` without a login** and **200
402
+ with the operator token**, streaming bytes byte-identical to `android/artifacts/agentus-companion.apk`
403
+ (sha256 `defc3a48…`, 1048446 bytes) — compared by hashing the served stream, not by trusting the 200.
404
+ `/notify` (the onboarding page with the pairing string + device table) is behind the same login.
405
+ - Username+password pairing: wrong password → `403 wrong username or password`, right one → `200` with a
406
+ device token, neither code nor login → `403`.
407
+ - **A3 (the operator's switches, 57 checks in `notify-smoke`)**: rules read/write/persist; a rule switched
408
+ off publishes nothing; presence suppresses that session's `turn-start` and the very same event arrives
409
+ once presence is cleared; **an approval is never suppressed**; the one-tap push reaches a real socket;
410
+ and all three new routes answer `401` without the operator credential. The test bed grew a
411
+ `/test/observe` route for this — it feeds synthetic events into the *real* `observe()` (the cockpit has
412
+ no such route), so the rules are exercised end to end instead of unit-mocked.
413
+ - **The cockpit panel itself, driven in a real browser**: 设置 → 手机通知 renders the pairing string, the
414
+ device list (with a live online dot) and the four switches; flipping 任务开始 in the UI turned the
415
+ server's stored rule into `turnStart:false`; clicking 把当前会话推到手机 made the server publish
416
+ `push:…` with `deeplink=/?session=<id>`; and selecting a session flipped presence to
417
+ `{sessionId, visible:true}, watching:true` with the panel showing 「你正看着 …」.
418
+ Driving an authenticated SPA without a password: CDP `Network.setExtraHTTPHeaders` with
419
+ `Authorization: Bearer <machine token>` (a session cookie minted by a *separate* process is refused —
420
+ the session registry lives in the running server's memory).
421
+ - **Device side, on a real Android 16 emulator**: see §11.1.
422
+ - **Still unverified**: whether *HyperOS* grants this app 焦点通知 (the operator's own MIX Fold 4 is the
423
+ only place that can answer); the code path is gated on `canShowFocus` and reports the answer in the UI.
424
+
425
+ ### 11.1 Android 16 emulator run (API 36, arm64)
426
+
427
+ The earlier "1.4 GB is not worth ~7 hours at 56 KB/s" verdict was wrong — `dl.google.com` serves at
428
+ **1.89 MB/s direct** and the proxy only adds ~15% (2.22 MB/s), so the 1.87 GB image landed in ~20
429
+ minutes and the app was verified on a real device instead of by inspection. Measured on the emulator
430
+ (API 36, `sdk_gphone64_arm64`):
431
+
432
+ | Check | Method | Result |
433
+ |---|---|---|
434
+ | installs and launches | `adb install -r` + `am start` | ✅ v0.3.2 up, nothing in `logcat -b crash` |
435
+ | a notification really appears | paired to the test bed (`10.0.2.2:8790`), posted the canned sequence | ✅ shade shows `跑完了 / 探针序列结束(end_turn)` with a 「查看」 action; the FGS notification `Agentus · 已连接 · 10.0.2.2:8790` is there too |
436
+ | promotion (AOSP path) | `NotificationManager.canPostPromotedNotifications()` | ⚠️ false on this image (the system has it off), and the app says so: 「实时动态: 被系统/用户关闭」。A real Android 16 phone is still the judge |
437
+ | foldable one-pane / two-pane | `adb shell wm size 1080x2520` (cover, 411dp) vs `2224x2488` | ✅ 「单栏(折叠态)」 vs 「双栏(展开态)」, servers left / form + status right |
438
+ | fold/unfold keeps the form | typed `https://fold4.test:8443`, unfolded, folded back | ✅ the activity was recreated and the typed text **survived** — `onSaveInstanceState` genuinely works |
439
+ | capability handshake | the app's real pair request | ✅ server recorded `[progress,actions,remote_input,channels,deeplink,live_update]` |
440
+ | password sealing | in-app status line | ✅ 「密码存储: Keystore 加密」 (falls back to plaintext *and says so* without a Keystore) |
441
+ | service self-heals | `force-stop`, then open the app only | ✅ device goes offline → online without pressing anything |
442
+
443
+ Two real defects the emulator caught (both looked fine in code and in a screenshot):
444
+
445
+ 1. **The four form inputs were never `addView`ed** (the bug predates this round; the rewrite copied
446
+ it). The screen "looked plausible" while there was no way to type an address, a username or a
447
+ password — i.e. the whole saved-servers feature had no entry point. Caught by counting widgets in
448
+ the accessibility tree (`uiautomator dump` → `EditText: 0`), not by looking at a screenshot.
449
+ 2. **The service never came back on its own**: after `force-stop`, opening the app left the channel
450
+ dead until the operator pressed something. Fixed by starting it from `MainActivity.onCreate` when
451
+ the active profile has a token (`onStartCommand` is idempotent — it only connects with no socket).
452
+
453
+ #### 11.1.1 Rounds 2–3: v0.4.1 → v0.5.1 (the app IS the cockpit, the toolbar, the keyboard, the self-test)
454
+
455
+ | Check | Method | Result |
456
+ |---|---|---|
457
+ | The app opens on the cockpit | launch with a saved server | ✅ toolbar + WebView, and the page really renders (the Agentus login screen — so the SPA's JS runs, it is not a white box); 服务器 → settings, 进驾驶舱 → back to the cockpit **without reloading** |
458
+ | Toolbar no longer under the status bar | `uiautomator dump`, the 服务器 node's `y` | ✅ y≈10 (under the bar, taps eaten by SystemUI) → y≈116 |
459
+ | Toolbar buttons were oversized | screenshot | ✅ replaced the platform `Button` (48dp minimum height, wide padding, raised gradient) with small rounded pills |
460
+ | Keyboard shrinks the page | tap the page's 用户名 field; `dumpsys input_method` + the WebView's bounds | ✅ IME `mInputShown=true`, WebView **1688 px → 848 px**, and back to **1688 px** after `keyevent 4` (which dismisses the IME only — the app stayed in the foreground) |
461
+ | 上岛自检 | tap it; read `logcat -s AgentusNotify` | ✅ first the device diagnosis (model / Android / HyperOS version / `canShowFocus` / SystemUI probe / AOSP availability), then 8 frames of a real ongoing notification (same id updated in place, 12.5% → 100%), auto-cleaned after 12 s. On the emulator (not Xiaomi, no promotion) the log says exactly that instead of pretending |
462
+
463
+ **Evidence from the real phone** (in the cockpit's own device table / `notify/state.json`): the
464
+ operator's MIX Fold 4 (`24072PX77C`, `sdkInt` 36, appVersion 0.3.2) paired with
465
+ `capabilities=[…, live_update, xiaomi_focus]` — and `xiaomi_focus` is only reported when
466
+ `canShowFocus` is true, i.e. **that phone already grants this app focus notifications**. v0.3.2 asked
467
+ AOSP first, so the island parameters never got attached; v0.5.1 reverses the order. That is the most
468
+ likely reason the island was never seen, and the app's own 「上岛自检」 is what settles it.
469
+
470
+ #### 11.1.2 Rounds 4–5: v0.5.2 / v0.5.3 (the island template per generation, the microphone)
471
+
472
+ | Check | Method | Result |
473
+ |---|---|---|
474
+ | the island payload matches OS3 | `adb shell settings put system notification_focus_protocol 3`, then 上岛自检 and read the log | ✅ the log's 「会发出的焦点通知参数(岛模板=true)」 prints exactly `param_v2{…, islandFirstFloat, param_island{islandProperty, islandTimeout, smallIslandArea{picInfo}, bigIslandArea{imageTextInfoLeft{picInfo}, imageTextInfoRight{type:2, textInfo}}}}` — the OS3 template, chosen from the ROM's protocol rather than from `Build.VERSION` |
475
+ | the diagnostic is readable on the phone | same run | ✅ 协议版本 3 / 超级岛特性(`persist.sys.feature.island`)/ canShowFocus / SystemUI probe / platform path — so the next "上岛没反应" is answered from the device, not from a doc |
476
+ | the page's mic request reaches the app | `WebView` on `http://localhost:8787` (a secure origin) calling `getUserMedia` | ✅ app log: 「页面要麦克风:系统权限没有,弹系统授权(请求先挂住)」 |
477
+ | the OS dialog appears and the grant lands | tap the dialog | ✅ 「Allow Agentus to record audio?」 → `RECORD_AUDIO: granted=true` (flags `USER_SET`) |
478
+ | an http LAN origin cannot do media at all | the same probe page over `http://10.0.2.2:8787` | ✅ the page itself reports no `navigator.mediaDevices` → hence the app's own warning line (P15) |
479
+ | ⚠️ not provable on the emulator | the page's promise after the grant | the emulator has no usable audio device, so the `getUserMedia` promise never settles there; "and then the page really gets the microphone" needs the real phone |
480
+
481
+ The emulator was reached over CDP for this (`adb forward` → the WebView's devtools socket → `Runtime.evaluate`),
482
+ which is also how the http-vs-localhost difference was measured without touching the app's UI.
483
+
484
+ #### 11.1.3 Round 6: the device reports on itself (v0.5.4)
485
+
486
+ Repeated "still not working" from a phone nobody can plug in is a signal to **stop changing behaviour and
487
+ build the observation path first**. `POST /api/notify/diag` (device token) stores the app's own account of
488
+ itself on the device record, and the WS handshake now carries `v=<appVersion>` because the version written
489
+ at pairing time goes stale after the first sideload — the operator could not even tell which build was
490
+ being tested.
491
+
492
+ Verified end to end on the emulator (tap 上岛自检 → upload → read `notify/state.json`):
493
+
494
+ ```
495
+ App 0.5.4(code 12) 服务器: http://10.0.2.2:8787
496
+ 设备: Google sdk_gphone64_arm64 系统: Android 16(API 36)
497
+ isXiaomi: 否 canShowFocus: 关 焦点通知协议版本: 3 超级岛特性: 无
498
+ 这条要不要按岛模板发: 否(没权限) 麦克风: 已授权
499
+ 会发出的焦点通知参数(岛模板=true): {"param_v2":{… "param_island":{…}}}
500
+ ```
501
+
502
+ and the device table's `appVersion` flipped `0.4.0` → `0.5.4` on the next connect ✓. The app also offers
503
+ 「复制诊断」 (clipboard, when the server is unreachable) and 「系统设置」 (the place a denied 录音 gets fixed).
504
+
505
+ #### 11.1.4 The real phone answers (v0.5.4 diagnosis from the MIX Fold 4)
506
+
507
+ The diagnostic pipe paid for itself on the first try. The phone's own report:
508
+
509
+ ```
510
+ App 0.5.4(code 12) 服务器: https://i207f47592.wicp.vip:38787
511
+ 设备: Xiaomi 24072PX77C 系统: Android 16(API 36) 小米系统版本: V816(code 816)
512
+ isXiaomi: 是 canShowFocus(焦点通知开关): 开
513
+ 焦点通知协议版本: 3 超级岛特性: 有(persist.sys.feature.island)
514
+ 这条要不要按岛模板发: 是(OS3 岛模板)
515
+ 实时动态(Android 16 原生提升): 可用
516
+ 麦克风: 已授权
517
+ ```
518
+
519
+ Reading: the ROM is an island-capable OS3, the app holds the focus switch, the template choice is the OS3
520
+ one and the payload matches the community SDK's OS3 example field for field. **Nothing left to change in
521
+ this codebase** — the remaining gate is Xiaomi's platform-side per-app grant (§6.1). Note also
522
+ `canPostPromotedNotifications()` is true there, so both island paths are live on that device and the
523
+ vendor path wins by design (§6).
524
+
525
+ For the microphone the same report narrows it to "granted + secure origin, still refuses", so the app
526
+ grew a 「语音自检」 that splits the question in half: a native `AudioRecord` open (can this app record at
527
+ all, WebView out of the picture?) and an injected `getUserMedia` (what does the page actually get?).
528
+ Both halves are part of the uploaded diagnosis. Verified on the emulator over an **http** LAN origin,
529
+ where it reproduces the insecure-context judgement exactly:
530
+
531
+ ```
532
+ 语音自检: 原生 AudioRecord=可以录(读到 512 帧) / 页面 getUserMedia={"secure":false,"md":false,
533
+ "perm":"prompt","err":"TypeError :: Cannot read properties of undefined (reading 'getUserMedia')"}
534
+ ```
535
+
536
+ Implementation trap worth remembering: `WebView.evaluateJavascript` does **not** await promises — an
537
+ async IIFE comes back as `{}`, which looks exactly like "the probe returned an empty object". Park the
538
+ result on `window` and read it back with a second injection.
539
+
540
+ #### 11.1.5 v0.5.7: the native microphone bridge and the two island paths (emulator, API 36)
541
+
542
+ The phone's v0.5.5 self-test settled the question — `原生 AudioRecord=可以录(读到 640 帧)` against
543
+ `页面 getUserMedia=NotReadableError :: Could not start audio source`, `secure:true, md:true` — and its
544
+ v0.5.6 counters added the last piece: **页面请求过 18 次(放行 18 / 挂起 0)**, i.e. every page request was
545
+ granted and Chromium still could not start the source. So the failure is inside Chromium's capture
546
+ pipeline, not in the app's permission plumbing. v0.5.6 also showed the probe stalling: the read-back said
547
+ `(跑着…)` because a `getUserMedia` that neither resolves nor rejects was never given a deadline.
548
+
549
+ v0.5.7 answers with three things and this run verifies each on the emulator:
550
+
551
+ | Change | Evidence |
552
+ |---|---|
553
+ | `MODIFY_AUDIO_SETTINGS` declared (the fix several reports name for exactly this error: Chromium's audio input switches the AudioManager mode before it opens the source) | `改动音频设置权限=有` in the diagnosis |
554
+ | The probe writes its state **after every step** and every step has a timeout | `{"step":"done", …}` instead of「跑着…」; each variant then reports its own `a_default` / `b_noProc` / `c_16k` outcome |
555
+ | A native microphone bridge (`window.AgentusMic`, `AudioRecord` 16 kHz mono s16le → base64 → `window.__asMic`), used by the page **instead of** `getUserMedia`, and probed as its own stage | `语音自检(原生桥): ok · 收到帧=51` and `原生桥=有 · 送出帧=51` — the two numbers agree, so frames really cross the bridge |
556
+ | The page's real dictation path (设置 → 语音 → 开麦说一句) picking the bridge | `原生麦克风:已开始` → `已送出 100 帧` in ~6 s, status `listening`, engine `stream` |
557
+ | `path` on the activity object, and a 「原生实时动态」 button that forces the AOSP path | `自检:通道=aosp` → `路径:aosp → 小米=跳过 · 原生实时动态=没用` |
558
+
559
+ Two traps this run cost a build each, both about the bridge being called from the wrong thread:
560
+
561
+ - **`View.post()` on a DETACHED view parks the runnable in the view's run queue.** The app shows its
562
+ config screen by detaching the WebView (it must not be destroyed, or going back reloads the cockpit),
563
+ so every frame posted from that screen waited for a re-attach: `送出帧=60 · 收到帧=0`. Use a
564
+ `Handler(Looper.getMainLooper())` instead — it runs whether or not the view is attached.
565
+ - **A `@JavascriptInterface` method must never touch the WebView.** `webView.getUrl()` from the
566
+ JavaBridge thread throws *"All WebView methods must be called on the same thread"*, and the page sees
567
+ it as `Java exception was raised during method invocation` with no clue what happened. The saved-host
568
+ check is therefore computed on the UI thread (`onPageStarted` / `onPageFinished`) and only *read* by
569
+ the bridge.
570
+
571
+ Caveat on this evidence: the emulator's dictation run ends with `the streaming socket failed — is the
572
+ server up?`. That is an artifact of driving the WebView headless with `Authorization: Bearer` (CDP
573
+ `Network.setExtraHTTPHeaders`): the Bearer header authenticates the page's `fetch` calls, but the
574
+ `/ws/asr` **WebSocket upgrade carries no header** and needs the session cookie, which this drive never
575
+ has. On the phone the page is logged in normally, so the socket has its cookie.
576
+
577
+ #### 11.1.6 v0.5.8: a call must be silent (server + app, verified on the emulator)
578
+
579
+ Reported from a real call: the turn ended, the call read the reply out loud, and the phone **dinged** with
580
+ a 「跑完了」 card. Fixed in three places, each verified:
581
+
582
+ | Fix | Where | Evidence |
583
+ |---|---|---|
584
+ | `completion` is actually read now, and the done card honours presence | `center.ts` `observe()` | smoke `rules: completion off ⇒ no done card (the switch used to be stored and never read)`, `presence: watching ⇒ the completion card is suppressed too` |
585
+ | presence carries `call`, written the instant `CallMode` mounts / hangs up | `CallMode.tsx`, `App.tsx`, `types.ts`, `center.ts` | smoke `presence: a call is reported as such`; `call: the called session gets neither the running card nor the completion card` |
586
+ | every other frame is sent **silently** while a call is live | `center.ts` `publish()` → `silentChannel`, `ChannelSpec.muted`, `Notifier.apply` (quiet channel) | smoke `call: another session's completion still arrives (never dropped)` + `…but silently`; `call: an approval still arrives`; `call over ⇒ the completion is loud again` |
587
+ | the app routes muted frames to one quiet channel | `Notifier.java` | `adb shell dumpsys notification --noredact` on API 36: during a call the same push lands in `mId='agentus-quiet'`, `mImportance=2`, `mSound=null`, `flags=…|SILENT`; with no call it lands in `mId='agent_done'`, `mImportance=4`, `mSound=content://settings/system/notification_sound` |
588
+
589
+ `notify-smoke` went 57 → **65 checks** (all green) with the call cases; `typecheck` clean.
590
+
591
+ #### 11.1.7 v0.5.10: 一张从来没发出过的审批卡,和让会话名出现在收起那一行
592
+
593
+ 两件事,同一轮里发现并修完(服务端文案 + 一个 App 侧的平台约束)。
594
+
595
+ **(1)审批卡在 Android 12+ 从来没投递成功过。** 发探针时 logcat 里出现
596
+ `Not posted. PendingIntents attached to actions with remote inputs must be mutable`:带 `RemoteInput`
597
+ (那个「或直接回一句」输入框)的 action,其 `PendingIntent` 必须是 **mutable**,而代码写的是
598
+ `FLAG_IMMUTABLE`。SystemUI 因此**整条通知都不投递**(不是砍掉按钮),我们的代码里不抛异常、不返回非 0。
599
+
600
+ | 证据 | 数值 |
601
+ |---|---|
602
+ | `dumpsys notification \| grep -c channel=agent_approval`(修前,连发两次探针) | **0** |
603
+ | 同一句(修后,同一条通知 id `83539`) | **1**,`actions=3`,`ONGOING_EVENT` |
604
+ | 运行中 / 完成两张卡(修前修后) | 都正常出现 —— 所以"其它通知都在"完全不能说明这张也在 |
605
+
606
+ 修法:那个 action 用 `Build.VERSION.SDK_INT >= 31 ? FLAG_MUTABLE : 0`(minSdk 26),其它 action 保持
607
+ immutable(没有人往里写东西)。顺带把 `NotifyService` 里"解析 JSON 失败"和"应用一帧失败"拆成两句话——
608
+ 原来共用「无法解析帧」,把一次平台拒收写成了 JSON 问题,正是它把这轮排查带偏过。
609
+
610
+ **(2)会话名从 `subtitle` 挪到 `body` 的最前面。** Android 的 `setContentText`(我们的 `body`)画在
611
+ **收起**的那一行,`setSubText`(我们的 `subtitle`)只在**展开**才画。改前实测 `android.title=跑完了` /
612
+ `android.subText=hello`,唯一可见的那行是 `stopReason: end_turn`——既没有会话名,也是给调试器看的字符串。
613
+
614
+ | kind | 标题 | 正文 |
615
+ |---|---|---|
616
+ | 运行中 | `Agentus · 运行中` | `<会话名> · <模型> · <思考深度>` |
617
+ | 完成 | `已完成` | `<会话名> · 用时 10s`(`turn-end.durationMs` 真量出来的) |
618
+ | 失败 | `执行失败` | `<会话名> · <错误摘要>` |
619
+ | 审批 | `待你确认:<工具短名>` | `<会话名> · <工具目标>` |
620
+
621
+ 审批那一行来自 agent 自己的 `toolCallTitle`,只在它自己的第一个 `": "` 处切成两半
622
+ (`Approve edit: /var/…` → 标题 `待你确认:Approve edit`、正文 `… · /var/…`):不翻译、不补全语义。
623
+
624
+ 真机实测(模拟器 API 36):分组收起形态里三行**每一行都带会话名**——
625
+ `已完成 在的 #2 · 用时 10s` / `已完成 探针会话 · 探针序列结束` / `待你确认:写文件 探针会话 · edit /etc/hosts —…`,
626
+ 审批那条下方是「仅此次 / 总是允许 / 拒绝」。断言落在 `scripts/notify-smoke.mts`(5 条,包括"三张卡都不再有
627
+ `subtitle`"和"卡上不出现 `stopReason`")。
628
+
629
+ ## 12. Run it
630
+
631
+ ```bash
632
+ # 1. server half, without touching the running cockpit
633
+ npm run notify-testbed # http :8790, https :8791 (test bed state is its own dir)
634
+
635
+ # 2. the phone app
636
+ npm run apk # -> android/artifacts/agentus-companion.apk
637
+
638
+ # 3. on the phone: open the test bed page, copy the pairing string, paste it into the app
639
+ # http://<mac-lan-ip>:8790/ (or https://<mac-lan-ip>:8791/ and accept the cert once)
640
+ # 4. press "发一条探针通知" (or POST /api/notify/probe) and watch the phone
641
+ # 5. 设置 → 上岛自检 # 10 s, no server: does THIS phone show the island, and via which path
642
+ ```
643
+
644
+ The test bed serves its own state (`packages/server/.data-notify-testbed`) precisely so it can run
645
+ while the cockpit is up. Once the cockpit is restarted it owns the channel itself
646
+ (`/api/notify/*`, `/agentus-companion.apk`) and has its own pairing code.
647
+
648
+ ## 13. What is left
649
+
650
+ | Item | Note |
651
+ |---|---|
652
+ | A2 island polish | Segments/points are passed through and drawn; tuning the tracker icon and testing the actual chip needs the phone |
653
+ | 上岛的真机结论 | Runs on the phone: 设置 → 上岛自检 — it now prints the exact payload plus the device's protocol generation. The vendor path is whitelist-only (§6.1) and the island has its own package whitelist beyond `canShowFocus`: if Xiaomi has not granted this app, no code change produces an island — the honest fallbacks are the ordinary ongoing notification, or a rooted device with an LSPosed island module |
654
+ | Token storage | Device token still in app-private prefs (the *passwords* are already Keystore-sealed via `SecretBox`) |
655
+ | Battery optimisation | The "don't kill my service" prompt/whitelist flow is still missing (`REQUEST_IGNORE_BATTERY_OPTIMIZATIONS`, sideload only) |
656
+ | `icon_url` | Bitmap download + cache, then claim the capability |
657
+ | Per-session mute | The rules are global today; "quiet for this session only" would be the next switch |
658
+ | iOS | Deliberately out of scope for now (APNs + ActivityKit + a widget extension) |