dsh-mobilecode 0.8.0 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +22 -9
- package/lib/android-stream.js +204 -133
- package/lib/client.js +136 -48
- package/lib/index.js +32 -25
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -95,23 +95,29 @@ drawer:
|
|
|
95
95
|
- `device_stream` — drive the live screen stream the panel shows: `start` an
|
|
96
96
|
online device (returns a signed `streamUrl` **and** an `android-stream`
|
|
97
97
|
`presentationMeta` that renders a compact conversation card and auto-opens the
|
|
98
|
-
panel), `status`, or `stop`.
|
|
99
|
-
|
|
98
|
+
panel), `status`, or `stop`. Streams are **per-serial and independent** since
|
|
99
|
+
0.9.0 — start both players of a co-op game and `status` lists every live
|
|
100
|
+
stream (`serials`); `stop` ends one (with `serial`) or all of them. Agents
|
|
101
|
+
that just need to see the screen should still prefer `device_screen` /
|
|
102
|
+
`device_ui_tree`.
|
|
100
103
|
|
|
101
104
|
**Live device stream (the panel)**
|
|
102
105
|
|
|
103
106
|
The Devices pane shows a real-time mirror of the attached device. It is produced
|
|
104
107
|
**in-process** — no inner loopback port, no external helper:
|
|
105
108
|
|
|
106
|
-
- ONE persistent `adb exec-out "while :; do screencap -p; done"` child
|
|
107
|
-
~8 fps with zero per-frame process cost (spawning
|
|
108
|
-
and 100% churn). A quote-safe PNG splitter cuts
|
|
109
|
-
frames by walking chunk headers (no marker
|
|
109
|
+
- ONE persistent `adb exec-out "while :; do screencap -p; done"` child **per
|
|
110
|
+
streamed device** runs at ~8 fps with zero per-frame process cost (spawning
|
|
111
|
+
adb per frame caps at ~5 fps and 100% churn). A quote-safe PNG splitter cuts
|
|
112
|
+
the concatenated output into frames by walking chunk headers (no marker
|
|
113
|
+
scanning, no false positives).
|
|
110
114
|
- The browser `<img>` reads a `multipart/x-mixed-replace` body served straight
|
|
111
115
|
from the latest-frame buffer. Backpressure is **latest-wins**: a slow tab skips
|
|
112
116
|
frames instead of building an unbounded queue or watching a growing delay.
|
|
113
|
-
-
|
|
114
|
-
|
|
117
|
+
- Each stream owns its consumer refcount + idle timeout (nobody watching that
|
|
118
|
+
device → its child stops) and crash keep-alive. Since 0.9.0 starting device B
|
|
119
|
+
**never** retires device A's child — co-op streams are fully independent, and
|
|
120
|
+
a frame from one device can never leak into the other viewer's pipe.
|
|
115
121
|
- **Tap or drag directly on the screen** to drive the device: a short press is a
|
|
116
122
|
tap, a long drag becomes a swipe carrying the real press duration (clamped
|
|
117
123
|
0–5000 ms), coalesced into one control call. A floating **pill toolbar** of SVG
|
|
@@ -126,6 +132,12 @@ The Devices pane shows a real-time mirror of the attached device. It is produced
|
|
|
126
132
|
- **Screenshot** captures a still via `POST /stream/still` (a real `screencap`,
|
|
127
133
|
embedded as a data URL up to 4 MB) and flips the stage to a still view with a
|
|
128
134
|
"back to Live" link.
|
|
135
|
+
- **Co-op split view (⧉, v0.9.0)**: with two or more online devices the card
|
|
136
|
+
header grows a ⧉ toggle that docks a compact **Player B** pane beside it —
|
|
137
|
+
its own device select, its own live `<img>`, and its own tap/drag control, so
|
|
138
|
+
a human watches both players fight at the same instant. Each pane grants its
|
|
139
|
+
own HMAC capability; closing one pane never stalls or disturbs the other
|
|
140
|
+
stream.
|
|
129
141
|
- **Security**: every stream route sits behind a loopback + trusted-browser
|
|
130
142
|
transport fence (peer address, loopback `Host`, `Sec-Fetch-Site` / `Origin` —
|
|
131
143
|
so a LAN client cannot spoof localhost and a DNS-rebinding `Host` is rejected),
|
|
@@ -252,7 +264,8 @@ join it and talk through it, and the AI model reads and steers the same wire.
|
|
|
252
264
|
Typical loop: `device_avd_create` + `device_boot` a clone → both apps
|
|
253
265
|
`join` → `mesh_link` → `device_batch` inputs at both → `device_pair_capture`
|
|
254
266
|
to watch → `mesh_log` to see the traffic → `mesh_tune` to inject real-world
|
|
255
|
-
network pain.
|
|
267
|
+
network pain. For the human at the keyboard: open the panel and hit ⧉ — the
|
|
268
|
+
Devices pane mirrors **both** devices live, side by side, tappable (v0.9.0).
|
|
256
269
|
|
|
257
270
|
**Conversation surface (v0.7.0)** — the transcript integration ported from
|
|
258
271
|
dsh-android's UI/UX:
|
package/lib/android-stream.js
CHANGED
|
@@ -1,10 +1,14 @@
|
|
|
1
1
|
/**
|
|
2
|
-
* dsh-mobilecode — host-side lifecycle manager for
|
|
2
|
+
* dsh-mobilecode — host-side lifecycle manager for live Android streams.
|
|
3
3
|
*
|
|
4
|
-
* Ported (lean) from ZSeven-W/dsh-android (android-host.ts, MIT)
|
|
5
|
-
*
|
|
6
|
-
*
|
|
7
|
-
*
|
|
4
|
+
* Ported (lean) from ZSeven-W/dsh-android (android-host.ts, MIT), widened in
|
|
5
|
+
* 0.9.0 for co-op testing: the host keeps one INDEPENDENT frame loop per
|
|
6
|
+
* serial (Map<serial, state>) instead of a single global loop, so two devices
|
|
7
|
+
* stream at once and the panel can show them side by side. Each stream owns
|
|
8
|
+
* its consumer refcount, idle timer, crash bookkeeping and frame subscribers;
|
|
9
|
+
* the "primary" stream (most recently ensured; the first live one as a
|
|
10
|
+
* fallback) preserves every single-stream default the agent tools and the
|
|
11
|
+
* control surface already rely on.
|
|
8
12
|
*
|
|
9
13
|
* The control surface takes NORMALIZED 0..1 coordinates of the streamed frame
|
|
10
14
|
* (the panel's <img> is scaled, so the browser knows fractions, not pixels) and
|
|
@@ -47,98 +51,126 @@ function requireNormalized(x, y) {
|
|
|
47
51
|
}
|
|
48
52
|
}
|
|
49
53
|
|
|
50
|
-
/** Lifecycle manager for
|
|
54
|
+
/** Lifecycle manager for live Android device streams (one loop per serial). */
|
|
51
55
|
export class AndroidStreamHost {
|
|
52
56
|
#options
|
|
53
|
-
|
|
54
|
-
#
|
|
55
|
-
|
|
56
|
-
#
|
|
57
|
+
/** @type {Map<string, object>} serial → per-stream state. */
|
|
58
|
+
#streams = new Map()
|
|
59
|
+
/** Most recently ensured serial; the anchor for single-stream defaults. */
|
|
60
|
+
#primary = undefined
|
|
57
61
|
#keepAliveTimer
|
|
58
|
-
#idleTimer
|
|
59
|
-
#restarts = 0
|
|
60
|
-
#startedAt
|
|
61
|
-
#exitAt
|
|
62
|
-
#lastError
|
|
63
|
-
#lastSerial
|
|
64
|
-
#intentionalStop = false
|
|
65
62
|
#disposed = false
|
|
66
|
-
#frameSubscribers = new Set()
|
|
67
63
|
|
|
68
64
|
constructor(options = {}) {
|
|
69
65
|
this.#options = {
|
|
70
66
|
restartDelayMs: options.restartDelayMs ?? DEFAULT_RESTART_DELAY_MS,
|
|
71
67
|
idleTimeoutMs: options.idleTimeoutMs ?? DEFAULT_IDLE_TIMEOUT_MS,
|
|
72
68
|
firstFrameTimeoutMs: options.firstFrameTimeoutMs ?? FIRST_FRAME_TIMEOUT_MS,
|
|
69
|
+
// Test seam: the offline suite injects a fake loop factory + device list.
|
|
70
|
+
loopFactory: options.loopFactory ?? ((serial, handlers) => new AdbFrameLoop(serial, handlers)),
|
|
71
|
+
devicesFn: options.devicesFn ?? (() => DeviceBuild.devices()),
|
|
73
72
|
}
|
|
74
73
|
}
|
|
75
74
|
|
|
76
|
-
|
|
77
|
-
|
|
75
|
+
// ── registry ───────────────────────────────────────────────────────────────
|
|
76
|
+
|
|
77
|
+
#state(serial) {
|
|
78
|
+
if (typeof serial !== 'string' || serial === '') {
|
|
79
|
+
throw new TypeError('dsh-mobilecode: stream operations require a non-empty serial')
|
|
80
|
+
}
|
|
81
|
+
let state = this.#streams.get(serial)
|
|
82
|
+
if (state === undefined) {
|
|
83
|
+
state = {
|
|
84
|
+
serial,
|
|
85
|
+
loop: undefined,
|
|
86
|
+
starting: undefined,
|
|
87
|
+
queue: Promise.resolve(),
|
|
88
|
+
consumers: 0,
|
|
89
|
+
restarts: 0,
|
|
90
|
+
startedAt: undefined,
|
|
91
|
+
exitAt: undefined,
|
|
92
|
+
lastError: undefined,
|
|
93
|
+
intentionalStop: false,
|
|
94
|
+
idleTimer: undefined,
|
|
95
|
+
subscribers: new Set(),
|
|
96
|
+
}
|
|
97
|
+
this.#streams.set(serial, state)
|
|
98
|
+
}
|
|
99
|
+
return state
|
|
78
100
|
}
|
|
79
101
|
|
|
80
|
-
|
|
81
|
-
|
|
102
|
+
/** Serials with a live loop, in start order. */
|
|
103
|
+
runningSerials() {
|
|
104
|
+
return [...this.#streams.values()].filter((s) => s.loop?.running === true).map((s) => s.serial)
|
|
82
105
|
}
|
|
83
106
|
|
|
84
|
-
|
|
85
|
-
return
|
|
107
|
+
isStreaming(serial) {
|
|
108
|
+
if (typeof serial !== 'string' || serial === '') return false
|
|
109
|
+
return this.#streams.get(serial)?.loop?.running === true
|
|
86
110
|
}
|
|
87
111
|
|
|
88
|
-
/**
|
|
89
|
-
|
|
90
|
-
this
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
112
|
+
/** The primary serial: a live stream if any (first started), else the last ensured. */
|
|
113
|
+
get streamedSerial() {
|
|
114
|
+
const running = this.runningSerials()
|
|
115
|
+
if (running.length > 0) return running[0]
|
|
116
|
+
return this.#primary !== undefined && this.#streams.has(this.#primary) ? this.#primary : undefined
|
|
117
|
+
}
|
|
118
|
+
|
|
119
|
+
get running() {
|
|
120
|
+
return this.runningSerials().length > 0
|
|
94
121
|
}
|
|
95
122
|
|
|
123
|
+
// ── lifecycle ──────────────────────────────────────────────────────────────
|
|
124
|
+
|
|
96
125
|
/**
|
|
97
|
-
* Make sure
|
|
98
|
-
*
|
|
99
|
-
*
|
|
126
|
+
* Make sure a frame loop is live for `serial`. Concurrent callers of the same
|
|
127
|
+
* serial share one launch; every serial gets its OWN loop — starting a second
|
|
128
|
+
* device never retires the first (co-op streams are independent).
|
|
100
129
|
*/
|
|
101
130
|
async ensureStreaming({ serial }) {
|
|
131
|
+
const state = this.#state(serial)
|
|
132
|
+
this.#primary = serial
|
|
133
|
+
return this.#ensureStream(state)
|
|
134
|
+
}
|
|
135
|
+
|
|
136
|
+
async #ensureStream(state) {
|
|
102
137
|
if (this.#disposed) throw new Error('dsh-mobilecode: the stream host is disposed')
|
|
103
|
-
if (
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
if (current !== undefined && current.running && current.serial === serial) return this.#infoOf(current)
|
|
107
|
-
if (current !== undefined && current.serial !== serial) {
|
|
108
|
-
current.stop()
|
|
109
|
-
this.#loop = undefined
|
|
110
|
-
}
|
|
111
|
-
return this.#startFor(serial)
|
|
138
|
+
if (state.loop !== undefined && state.loop.running) {
|
|
139
|
+
this.#armIdle(state)
|
|
140
|
+
return this.#infoOf(state.loop)
|
|
112
141
|
}
|
|
113
|
-
|
|
114
|
-
if (starting !== undefined) {
|
|
142
|
+
if (state.starting !== undefined) {
|
|
115
143
|
try {
|
|
116
|
-
await starting
|
|
144
|
+
await state.starting
|
|
117
145
|
} catch {
|
|
118
146
|
// A failed shared launch is settled; retry below.
|
|
119
147
|
}
|
|
120
|
-
if (
|
|
121
|
-
this.#armIdle()
|
|
122
|
-
return this.#infoOf(
|
|
148
|
+
if (state.loop !== undefined && state.loop.running) {
|
|
149
|
+
this.#armIdle(state)
|
|
150
|
+
return this.#infoOf(state.loop)
|
|
123
151
|
}
|
|
124
|
-
starting = undefined
|
|
125
152
|
}
|
|
126
|
-
|
|
127
|
-
|
|
153
|
+
const launch = async () => {
|
|
154
|
+
if (state.loop !== undefined && state.loop.running) return this.#infoOf(state.loop)
|
|
155
|
+
return this.#startFor(state)
|
|
156
|
+
}
|
|
157
|
+
const launching = state.queue.then(launch, launch)
|
|
158
|
+
state.queue = launching.then(() => undefined, () => undefined)
|
|
159
|
+
state.starting = launching
|
|
128
160
|
try {
|
|
129
|
-
const info = await
|
|
130
|
-
|
|
131
|
-
this.#armIdle()
|
|
161
|
+
const info = await launching
|
|
162
|
+
state.lastError = undefined
|
|
163
|
+
this.#armIdle(state)
|
|
132
164
|
return info
|
|
133
165
|
} catch (error) {
|
|
134
|
-
|
|
166
|
+
state.lastError = errorMessage(error)
|
|
135
167
|
throw error
|
|
136
168
|
} finally {
|
|
137
|
-
if (
|
|
169
|
+
if (state.starting === launching) state.starting = undefined
|
|
138
170
|
}
|
|
139
171
|
}
|
|
140
172
|
|
|
141
|
-
/** Start the crash keep-alive loop (restarts
|
|
173
|
+
/** Start the crash keep-alive loop (restarts any unintentionally dead loop). */
|
|
142
174
|
startKeepAlive() {
|
|
143
175
|
if (this.#keepAliveTimer !== undefined || this.#disposed) return
|
|
144
176
|
this.#keepAliveTimer = setInterval(() => {
|
|
@@ -152,55 +184,97 @@ export class AndroidStreamHost {
|
|
|
152
184
|
this.#keepAliveTimer = undefined
|
|
153
185
|
}
|
|
154
186
|
|
|
155
|
-
/** Stop
|
|
156
|
-
async stop() {
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
187
|
+
/** Stop one stream intentionally (defaults to the primary). */
|
|
188
|
+
async stop(serial = this.streamedSerial) {
|
|
189
|
+
if (serial === undefined) return
|
|
190
|
+
const state = this.#streams.get(serial)
|
|
191
|
+
if (state !== undefined) await this.#stopStream(state)
|
|
192
|
+
}
|
|
193
|
+
|
|
194
|
+
/** Stop every stream (panel teardown, device_stream stop without a serial). */
|
|
195
|
+
async stopAll() {
|
|
196
|
+
for (const state of [...this.#streams.values()]) await this.#stopStream(state)
|
|
197
|
+
this.#primary = undefined
|
|
198
|
+
}
|
|
199
|
+
|
|
200
|
+
async #stopStream(state) {
|
|
201
|
+
this.#streams.delete(state.serial)
|
|
202
|
+
if (this.#primary === state.serial) this.#primary = undefined
|
|
203
|
+
state.intentionalStop = true
|
|
204
|
+
this.#clearIdle(state)
|
|
205
|
+
state.exitAt = undefined
|
|
206
|
+
const loop = state.loop
|
|
207
|
+
state.loop = undefined
|
|
208
|
+
state.startedAt = undefined
|
|
163
209
|
loop?.stop()
|
|
164
|
-
await
|
|
165
|
-
const landed =
|
|
210
|
+
await state.starting?.catch(() => {})
|
|
211
|
+
const landed = state.loop
|
|
166
212
|
if (landed !== undefined) {
|
|
167
|
-
|
|
213
|
+
state.loop = undefined
|
|
168
214
|
landed.stop()
|
|
169
215
|
}
|
|
170
216
|
}
|
|
171
217
|
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
this
|
|
175
|
-
this.#
|
|
218
|
+
dispose() {
|
|
219
|
+
this.#disposed = true
|
|
220
|
+
this.stopKeepAlive()
|
|
221
|
+
for (const state of this.#streams.values()) {
|
|
222
|
+
state.subscribers.clear()
|
|
223
|
+
void this.#stopStream(state)
|
|
224
|
+
}
|
|
225
|
+
this.#streams.clear()
|
|
226
|
+
this.#primary = undefined
|
|
227
|
+
}
|
|
228
|
+
|
|
229
|
+
// ── accessors (primary unless a serial is given) ───────────────────────────
|
|
230
|
+
|
|
231
|
+
/** Hold a stream alive for one consumer; release exactly once. */
|
|
232
|
+
acquire(serial = this.streamedSerial) {
|
|
233
|
+
const state = this.#state(serial)
|
|
234
|
+
state.consumers += 1
|
|
235
|
+
this.#armIdle(state)
|
|
176
236
|
let released = false
|
|
177
237
|
return () => {
|
|
178
238
|
if (released) return
|
|
179
239
|
released = true
|
|
180
|
-
|
|
181
|
-
this.#armIdle()
|
|
240
|
+
state.consumers = Math.max(0, state.consumers - 1)
|
|
241
|
+
this.#armIdle(state)
|
|
242
|
+
}
|
|
243
|
+
}
|
|
244
|
+
|
|
245
|
+
/**
|
|
246
|
+
* Observe every decoded frame of ONE serial; the returned function
|
|
247
|
+
* unsubscribes. Serial-scoped so two panels never cross frames.
|
|
248
|
+
*/
|
|
249
|
+
subscribeFrames(serial, subscriber) {
|
|
250
|
+
const state = this.#state(serial)
|
|
251
|
+
state.subscribers.add(subscriber)
|
|
252
|
+
return () => {
|
|
253
|
+
state.subscribers.delete(subscriber)
|
|
182
254
|
}
|
|
183
255
|
}
|
|
184
256
|
|
|
185
|
-
status() {
|
|
186
|
-
const
|
|
187
|
-
|
|
257
|
+
status(serial = this.streamedSerial) {
|
|
258
|
+
const state = serial === undefined ? undefined : this.#streams.get(serial)
|
|
259
|
+
if (state === undefined) {
|
|
260
|
+
return { running: false, restarts: 0, consumers: 0, stderr: [] }
|
|
261
|
+
}
|
|
262
|
+
const frame = state.loop?.latestFrame
|
|
188
263
|
return {
|
|
189
|
-
running:
|
|
190
|
-
|
|
191
|
-
restarts:
|
|
192
|
-
...(
|
|
193
|
-
consumers:
|
|
264
|
+
running: state.loop?.running === true,
|
|
265
|
+
serial: state.serial,
|
|
266
|
+
restarts: state.restarts,
|
|
267
|
+
...(state.lastError === undefined ? {} : { lastError: state.lastError }),
|
|
268
|
+
consumers: state.consumers,
|
|
194
269
|
...(frame === undefined ? {} : { frameSequence: frame.sequence, lastFrameAt: frame.at, width: frame.width, height: frame.height }),
|
|
195
|
-
stderr: loop?.stderrLines ?? [],
|
|
270
|
+
stderr: state.loop?.stderrLines ?? [],
|
|
196
271
|
}
|
|
197
272
|
}
|
|
198
273
|
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
this.#
|
|
203
|
-
return this.stop()
|
|
274
|
+
/** Latest frame of one stream (primary by default). */
|
|
275
|
+
latestFrame(serial = this.streamedSerial) {
|
|
276
|
+
if (serial === undefined) return undefined
|
|
277
|
+
return this.#streams.get(serial)?.loop?.latestFrame
|
|
204
278
|
}
|
|
205
279
|
|
|
206
280
|
// ── control surface (normalized 0..1 of the streamed frame) ────────────────
|
|
@@ -262,29 +336,33 @@ export class AndroidStreamHost {
|
|
|
262
336
|
.map((row) => ({ serial: row.serial, kind: row.serial.startsWith('emulator-') ? 'emulator' : 'physical' }))
|
|
263
337
|
}
|
|
264
338
|
|
|
339
|
+
// ── internals ──────────────────────────────────────────────────────────────
|
|
340
|
+
|
|
265
341
|
async #shell(serial, shell) {
|
|
266
342
|
await DeviceBuild.adbRun(serial, ['shell', ...shell], { timeoutMs: CONTROL_TIMEOUT_MS })
|
|
267
343
|
}
|
|
268
344
|
|
|
269
345
|
async #keepAliveTick() {
|
|
270
|
-
if (this.#disposed
|
|
271
|
-
const
|
|
272
|
-
const
|
|
273
|
-
|
|
274
|
-
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
|
|
346
|
+
if (this.#disposed) return
|
|
347
|
+
const now = Date.now()
|
|
348
|
+
for (const state of [...this.#streams.values()]) {
|
|
349
|
+
if (state.intentionalStop || state.exitAt === undefined) continue
|
|
350
|
+
if (now - state.exitAt < this.#options.restartDelayMs) continue
|
|
351
|
+
state.exitAt = undefined
|
|
352
|
+
state.restarts += 1
|
|
353
|
+
try {
|
|
354
|
+
await this.#ensureStream(state)
|
|
355
|
+
} catch (error) {
|
|
356
|
+
state.lastError = errorMessage(error)
|
|
357
|
+
if (state.exitAt === undefined) state.exitAt = Date.now()
|
|
358
|
+
}
|
|
282
359
|
}
|
|
283
360
|
}
|
|
284
361
|
|
|
285
|
-
async #startFor(
|
|
362
|
+
async #startFor(state) {
|
|
286
363
|
if (this.#disposed) throw new Error('dsh-mobilecode: the stream host is disposed')
|
|
287
|
-
const
|
|
364
|
+
const { serial } = state
|
|
365
|
+
const online = await this.#options.devicesFn()
|
|
288
366
|
if (!online.some((row) => row.serial === serial && row.state === 'device')) {
|
|
289
367
|
const known = online.find((row) => row.serial === serial)
|
|
290
368
|
throw new Error(
|
|
@@ -293,9 +371,9 @@ export class AndroidStreamHost {
|
|
|
293
371
|
: `device ${serial} is ${known.state}, not ready to stream`,
|
|
294
372
|
)
|
|
295
373
|
}
|
|
296
|
-
const loop =
|
|
374
|
+
const loop = this.#options.loopFactory(serial, {
|
|
297
375
|
onFrame: (frame) => {
|
|
298
|
-
for (const subscriber of
|
|
376
|
+
for (const subscriber of state.subscribers) {
|
|
299
377
|
try {
|
|
300
378
|
subscriber(frame)
|
|
301
379
|
} catch {
|
|
@@ -304,27 +382,26 @@ export class AndroidStreamHost {
|
|
|
304
382
|
}
|
|
305
383
|
},
|
|
306
384
|
onExit: (detail) => {
|
|
307
|
-
if (
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
385
|
+
if (state.loop !== loop) return
|
|
386
|
+
state.loop = undefined
|
|
387
|
+
state.startedAt = undefined
|
|
388
|
+
state.lastError = `the screencap loop for ${serial} died (${detail})`
|
|
389
|
+
state.exitAt = Date.now()
|
|
312
390
|
},
|
|
313
391
|
})
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
this.#intentionalStop = false
|
|
392
|
+
state.loop = loop
|
|
393
|
+
state.intentionalStop = false
|
|
317
394
|
loop.reset()
|
|
318
395
|
loop.start()
|
|
319
396
|
const frame = await loop.waitForFrame(this.#options.firstFrameTimeoutMs)
|
|
320
397
|
if (frame === undefined) {
|
|
321
398
|
const stderr = loop.stderrLines.join('\n')
|
|
322
399
|
loop.stop()
|
|
323
|
-
if (
|
|
400
|
+
if (state.loop === loop) state.loop = undefined
|
|
324
401
|
throw new Error(`no frame arrived from ${serial} within ${this.#options.firstFrameTimeoutMs} ms${stderr === '' ? '' : `: ${stderr}`}`)
|
|
325
402
|
}
|
|
326
|
-
|
|
327
|
-
|
|
403
|
+
state.startedAt = Date.now()
|
|
404
|
+
state.exitAt = undefined
|
|
328
405
|
return this.#infoOf(loop)
|
|
329
406
|
}
|
|
330
407
|
|
|
@@ -335,36 +412,30 @@ export class AndroidStreamHost {
|
|
|
335
412
|
|
|
336
413
|
/** Normalized frame coordinates → `input` pixels via the live frame size. */
|
|
337
414
|
async #pixels(serial, x, y) {
|
|
338
|
-
const frame = this.
|
|
415
|
+
const frame = this.#streams.get(serial)?.loop?.latestFrame
|
|
339
416
|
if (frame !== undefined) return { x: Math.round(x * frame.width), y: Math.round(y * frame.height) }
|
|
340
417
|
const size = await screenSize(serial)
|
|
341
418
|
return { x: Math.round(x * size.width), y: Math.round(y * size.height) }
|
|
342
419
|
}
|
|
343
420
|
|
|
344
|
-
#armIdle() {
|
|
345
|
-
this.#clearIdle()
|
|
421
|
+
#armIdle(state) {
|
|
422
|
+
this.#clearIdle(state)
|
|
346
423
|
const idleMs = this.#options.idleTimeoutMs
|
|
347
424
|
if (idleMs <= 0) return
|
|
348
|
-
|
|
349
|
-
|
|
350
|
-
if (
|
|
351
|
-
this.#armIdle()
|
|
425
|
+
state.idleTimer = setTimeout(() => {
|
|
426
|
+
state.idleTimer = undefined
|
|
427
|
+
if (state.consumers > 0) {
|
|
428
|
+
this.#armIdle(state)
|
|
352
429
|
return
|
|
353
430
|
}
|
|
354
|
-
void this
|
|
431
|
+
void this.#stopStream(state)
|
|
355
432
|
}, idleMs)
|
|
356
|
-
|
|
357
|
-
}
|
|
358
|
-
|
|
359
|
-
#clearIdle() {
|
|
360
|
-
if (this.#idleTimer !== undefined) clearTimeout(this.#idleTimer)
|
|
361
|
-
this.#idleTimer = undefined
|
|
433
|
+
state.idleTimer.unref?.()
|
|
362
434
|
}
|
|
363
435
|
|
|
364
|
-
#
|
|
365
|
-
|
|
366
|
-
|
|
367
|
-
return run
|
|
436
|
+
#clearIdle(state) {
|
|
437
|
+
if (state.idleTimer !== undefined) clearTimeout(state.idleTimer)
|
|
438
|
+
state.idleTimer = undefined
|
|
368
439
|
}
|
|
369
440
|
}
|
|
370
441
|
|
package/lib/client.js
CHANGED
|
@@ -117,6 +117,12 @@ window.__ModuleLoader__.load({
|
|
|
117
117
|
.mc-live-stage { position: relative; align-self: center; background: #000; border-radius: 10px; overflow: hidden; line-height: 0; box-shadow: inset 0 0 0 1px light-dark(rgba(0,0,0,.12), rgba(255,255,255,.14)); }
|
|
118
118
|
.mc-live-stage img { display: block; max-width: 100%; max-height: 60vh; width: auto; height: auto; cursor: crosshair; touch-action: none; user-select: none; }
|
|
119
119
|
.mc-live-stage .mc-live-off { display: flex; align-items: center; justify-content: center; width: 260px; height: 460px; color: #9aa0a8; font-size: 13px; line-height: 1.5; text-align: center; }
|
|
120
|
+
/* 0.9.0 co-op: card + second pane share the panel width side by side. */
|
|
121
|
+
.mc-live-section { display: flex; flex-direction: column; gap: 10px; }
|
|
122
|
+
.mc-live-section.coop { flex-direction: row; flex-wrap: wrap; align-items: flex-start; }
|
|
123
|
+
.mc-live-section.coop > .mc-card { flex: 1 1 320px; min-width: 300px; }
|
|
124
|
+
.mc-live-section.coop .mc-live-stage img { max-height: 48vh; }
|
|
125
|
+
.mc-coop-select { width: auto; padding: 3px 6px; font-size: 12px; }
|
|
120
126
|
.mc-live-nav { display: flex; align-items: center; justify-content: center; gap: 8px; }
|
|
121
127
|
.mc-live-nav button { border: 1px solid light-dark(rgba(0,0,0,.16), rgba(255,255,255,.18)); background: light-dark(#fff, #20242b); color: inherit; border-radius: 999px; width: 40px; height: 32px; cursor: pointer; font-size: 14px; }
|
|
122
128
|
.mc-live-nav button:hover { background: light-dark(rgba(0,0,0,.06), rgba(255,255,255,.1)); }
|
|
@@ -332,7 +338,7 @@ window.__ModuleLoader__.load({
|
|
|
332
338
|
);
|
|
333
339
|
}),
|
|
334
340
|
),
|
|
335
|
-
h(
|
|
341
|
+
h(LiveStreamSection, null),
|
|
336
342
|
platforms.map((platform) => {
|
|
337
343
|
const srv = server(platform);
|
|
338
344
|
const bld = build(platform);
|
|
@@ -486,69 +492,45 @@ window.__ModuleLoader__.load({
|
|
|
486
492
|
);
|
|
487
493
|
}
|
|
488
494
|
|
|
489
|
-
/**
|
|
490
|
-
*
|
|
491
|
-
|
|
492
|
-
|
|
493
|
-
const [avds, setAvds] = useState([]);
|
|
494
|
-
const [serial, setSerial] = useState("");
|
|
495
|
+
/** Shared stream lifecycle (0.9.0 co-op): HMAC-granted frame URL with a
|
|
496
|
+
* re-mint timer, plus tap/drag pointer mapping for the <img>. Used by
|
|
497
|
+
* the full card and by the second co-op pane. */
|
|
498
|
+
function useLiveStream({ serial, onError }) {
|
|
495
499
|
const [streamUrl, setStreamUrl] = useState("");
|
|
496
|
-
const [error, setError] = useState("");
|
|
497
|
-
const [pickerOpen, setPickerOpen] = useState(false);
|
|
498
|
-
const [menuOpen, setMenuOpen] = useState(false);
|
|
499
|
-
const [still, setStill] = useState(null);
|
|
500
|
-
const [view, setView] = useState("live"); // 'live' | 'still'
|
|
501
|
-
const [sizeMode, setSizeMode] = useState({ mode: "fit" }); // {mode:'fit'|'pct'|'px', value?}
|
|
502
|
-
const [frame, setFrame] = useState("none"); // 'none' | 'bezel' | 'device'
|
|
503
500
|
const imgRef = useRef(null);
|
|
504
501
|
const dragRef = useRef(null);
|
|
505
502
|
const grantTimer = useRef(null);
|
|
503
|
+
const serialRef = useRef(serial);
|
|
504
|
+
serialRef.current = serial;
|
|
505
|
+
const onErrorRef = useRef(onError);
|
|
506
|
+
onErrorRef.current = onError;
|
|
506
507
|
|
|
507
508
|
const grant = async (device) => {
|
|
508
|
-
if (!device) return;
|
|
509
|
+
if (!device) return false;
|
|
509
510
|
try {
|
|
510
511
|
const r = await streamPost("/stream/grant", { device });
|
|
511
512
|
// Absolute URL: the plugin card can be hosted under a different
|
|
512
513
|
// origin/path than the API, and <img> src ignores fetch()'s base.
|
|
513
514
|
setStreamUrl(location.origin + r.streamUrl);
|
|
514
|
-
setSerial(r.device);
|
|
515
|
-
setError("");
|
|
516
515
|
clearTimeout(grantTimer.current);
|
|
517
516
|
// Re-mint a minute before the 10-minute capability expires.
|
|
518
517
|
const ms = Math.max(30000, (r.expiresAt - Date.now()) - 60000);
|
|
519
518
|
grantTimer.current = setTimeout(() => { grant(r.device).catch(() => {}); }, ms);
|
|
519
|
+
return true;
|
|
520
520
|
} catch (err) {
|
|
521
|
-
|
|
521
|
+
onErrorRef.current(err instanceof Error ? err.message : String(err));
|
|
522
522
|
setStreamUrl("");
|
|
523
|
+
return false;
|
|
523
524
|
}
|
|
524
525
|
};
|
|
525
|
-
|
|
526
|
-
|
|
527
|
-
try {
|
|
528
|
-
const r = await streamPost("/stream/devices", {});
|
|
529
|
-
setDevices(r.devices);
|
|
530
|
-
setAvds(r.avds ?? []);
|
|
531
|
-
setSerial((cur) => cur || (r.devices.find((d) => d.streaming)?.serial ?? r.devices[0]?.serial ?? ""));
|
|
532
|
-
} catch (err) {
|
|
533
|
-
setError(err instanceof Error ? err.message : String(err));
|
|
534
|
-
}
|
|
535
|
-
};
|
|
536
|
-
|
|
537
|
-
useEffect(() => {
|
|
538
|
-
refreshDevices();
|
|
539
|
-
const timer = setInterval(refreshDevices, 5000);
|
|
540
|
-
return () => { clearInterval(timer); clearTimeout(grantTimer.current); };
|
|
541
|
-
}, []);
|
|
542
|
-
|
|
543
|
-
// Auto-grant once a serial is known, no stream, and we're in live view.
|
|
544
|
-
useEffect(() => { if (serial !== "" && streamUrl === "" && view === "live") grant(serial).catch(() => {}); }, [serial, streamUrl, view]);
|
|
526
|
+
const reset = () => { clearTimeout(grantTimer.current); setStreamUrl(""); };
|
|
527
|
+
useEffect(() => () => clearTimeout(grantTimer.current), []);
|
|
545
528
|
|
|
546
529
|
const control = async (action) => {
|
|
547
|
-
if (
|
|
548
|
-
try { await streamPost("/stream/control", { device:
|
|
549
|
-
catch (err) {
|
|
530
|
+
if (serialRef.current === "") return;
|
|
531
|
+
try { await streamPost("/stream/control", { device: serialRef.current, action }); }
|
|
532
|
+
catch (err) { onErrorRef.current(err instanceof Error ? err.message : String(err)); }
|
|
550
533
|
};
|
|
551
|
-
|
|
552
534
|
const norm = (clientX, clientY) => {
|
|
553
535
|
const el = imgRef.current;
|
|
554
536
|
if (!el) return null;
|
|
@@ -574,6 +556,46 @@ window.__ModuleLoader__.load({
|
|
|
574
556
|
control({ kind: "tap", x: end.x, y: end.y });
|
|
575
557
|
}
|
|
576
558
|
};
|
|
559
|
+
return { streamUrl, grant, reset, control, imgProps: { ref: imgRef, draggable: false, onPointerDown: onDown, onPointerUp: onUp } };
|
|
560
|
+
}
|
|
561
|
+
|
|
562
|
+
/** Live device stream: picker-popover header, SVG toolbar pill, device
|
|
563
|
+
* menu, quick sizes + frame styles, and tap/drag with duration. */
|
|
564
|
+
function LiveDeviceCard({ coopOn, onToggleCoop, onSerial }) {
|
|
565
|
+
const [devices, setDevices] = useState([]);
|
|
566
|
+
const [avds, setAvds] = useState([]);
|
|
567
|
+
const [serial, setSerial] = useState("");
|
|
568
|
+
const [error, setError] = useState("");
|
|
569
|
+
const [pickerOpen, setPickerOpen] = useState(false);
|
|
570
|
+
const [menuOpen, setMenuOpen] = useState(false);
|
|
571
|
+
const [still, setStill] = useState(null);
|
|
572
|
+
const [view, setView] = useState("live"); // 'live' | 'still'
|
|
573
|
+
const [sizeMode, setSizeMode] = useState({ mode: "fit" }); // {mode:'fit'|'pct'|'px', value?}
|
|
574
|
+
const [frame, setFrame] = useState("none"); // 'none' | 'bezel' | 'device'
|
|
575
|
+
const live = useLiveStream({ serial, onError: setError });
|
|
576
|
+
const grant = (device) => live.grant(device).then((ok) => { if (ok) setError(""); });
|
|
577
|
+
const control = live.control;
|
|
578
|
+
useEffect(() => { if (onSerial) onSerial(serial); }, [serial]);
|
|
579
|
+
|
|
580
|
+
const refreshDevices = async () => {
|
|
581
|
+
try {
|
|
582
|
+
const r = await streamPost("/stream/devices", {});
|
|
583
|
+
setDevices(r.devices);
|
|
584
|
+
setAvds(r.avds ?? []);
|
|
585
|
+
setSerial((cur) => cur || (r.devices.find((d) => d.streaming)?.serial ?? r.devices[0]?.serial ?? ""));
|
|
586
|
+
} catch (err) {
|
|
587
|
+
setError(err instanceof Error ? err.message : String(err));
|
|
588
|
+
}
|
|
589
|
+
};
|
|
590
|
+
|
|
591
|
+
useEffect(() => {
|
|
592
|
+
refreshDevices();
|
|
593
|
+
const timer = setInterval(refreshDevices, 5000);
|
|
594
|
+
return () => clearInterval(timer);
|
|
595
|
+
}, []);
|
|
596
|
+
|
|
597
|
+
// Auto-grant once a serial is known, no stream, and we're in live view.
|
|
598
|
+
useEffect(() => { if (serial !== "" && live.streamUrl === "" && view === "live") grant(serial); }, [serial, live.streamUrl, view]);
|
|
577
599
|
|
|
578
600
|
const capture = async () => {
|
|
579
601
|
if (serial === "") return;
|
|
@@ -599,8 +621,8 @@ window.__ModuleLoader__.load({
|
|
|
599
621
|
catch (err) { setError(err instanceof Error ? err.message : String(err)); }
|
|
600
622
|
};
|
|
601
623
|
|
|
602
|
-
const pick = (device) => { setSerial(device);
|
|
603
|
-
const showLive = () => { setView("live"); if (serial !== "" && streamUrl === "") grant(serial)
|
|
624
|
+
const pick = (device) => { setSerial(device); live.reset(); setStill(null); setView("live"); setError(""); setPickerOpen(false); };
|
|
625
|
+
const showLive = () => { setView("live"); if (serial !== "" && live.streamUrl === "") grant(serial); };
|
|
604
626
|
|
|
605
627
|
const sizeStyle = sizeMode.mode === "pct"
|
|
606
628
|
? { width: sizeMode.value + "%", maxWidth: "100%", maxHeight: "60vh" }
|
|
@@ -614,7 +636,7 @@ window.__ModuleLoader__.load({
|
|
|
614
636
|
{ title: "Recents", icon: "M12 3l9 5-9 5-9-5 9-5zM3 13l9 5 9-5", run: () => control({ kind: "button", name: "recents" }) },
|
|
615
637
|
{ title: "Screenshot", icon: "M4 7h4l2-3h4l2 3h4v13H4zM12 17a3 3 0 1 0 0-6 3 3 0 0 0 0 6z", run: () => capture() },
|
|
616
638
|
{ title: "Rotate", icon: "M21 12a9 9 0 1 1-2.6-6.3M21 3v6h-6", run: () => control({ kind: "rotate" }) },
|
|
617
|
-
{ title: "Refresh", icon: "M21 12a9 9 0 1 1-2.6-6.3M21 3v6h-6", run: () => { if (serial !== "") grant(serial)
|
|
639
|
+
{ title: "Refresh", icon: "M21 12a9 9 0 1 1-2.6-6.3M21 3v6h-6", run: () => { if (serial !== "") grant(serial); } },
|
|
618
640
|
];
|
|
619
641
|
const menuItems = [
|
|
620
642
|
{ action: "notifications", label: "Notifications" },
|
|
@@ -635,7 +657,7 @@ window.__ModuleLoader__.load({
|
|
|
635
657
|
return h("div", { className: "mc-card mc-live" },
|
|
636
658
|
h("div", { className: "mc-card-head" },
|
|
637
659
|
h("span", null, "Live device"),
|
|
638
|
-
streamUrl !== "" && h("span", { className: "mc-live-badge" }, h("span", { className: "mc-dot on" }), "Live"),
|
|
660
|
+
live.streamUrl !== "" && h("span", { className: "mc-live-badge" }, h("span", { className: "mc-dot on" }), "Live"),
|
|
639
661
|
h("span", { className: "sp" }),
|
|
640
662
|
h("div", { className: "mc-picker" },
|
|
641
663
|
h("button", { className: "mc-btn", onClick: () => setPickerOpen((v) => !v) }, serial || "no device", " ▾"),
|
|
@@ -662,6 +684,8 @@ window.__ModuleLoader__.load({
|
|
|
662
684
|
menuItems.map((item) => h("button", { key: item.action, onClick: () => menuAction(item.action) }, item.label)),
|
|
663
685
|
),
|
|
664
686
|
),
|
|
687
|
+
// 0.9.0 co-op: side-by-side second pane (needs two online devices).
|
|
688
|
+
devices.length >= 2 && h("button", { className: "mc-btn", "data-on": coopOn || undefined, title: "Co-op: watch two devices side by side", onClick: onToggleCoop }, "⧉"),
|
|
665
689
|
),
|
|
666
690
|
error !== "" && h("div", { className: "mc-error" }, error),
|
|
667
691
|
h("div", { className: "mc-toolbar" },
|
|
@@ -674,8 +698,8 @@ window.__ModuleLoader__.load({
|
|
|
674
698
|
h("div", { className: stageClass },
|
|
675
699
|
view === "still" && still?.dataUrl
|
|
676
700
|
? h("img", { src: still.dataUrl, alt: "still capture", draggable: false })
|
|
677
|
-
: streamUrl !== ""
|
|
678
|
-
? h("img", {
|
|
701
|
+
: live.streamUrl !== ""
|
|
702
|
+
? h("img", { ...live.imgProps, src: live.streamUrl, alt: "device screen", style: sizeStyle, onError: () => grant(serial) })
|
|
679
703
|
: h("div", { className: "mc-live-off" }, devices.length === 0
|
|
680
704
|
? "No device attached. Boot one with device_boot, or press Run app."
|
|
681
705
|
: "Connecting to " + serial + "..."),
|
|
@@ -711,6 +735,70 @@ window.__ModuleLoader__.load({
|
|
|
711
735
|
),
|
|
712
736
|
);
|
|
713
737
|
}
|
|
738
|
+
/** Co-op second pane (0.9.0): a compact stream view for a second device —
|
|
739
|
+
* device select, live image, tap/drag. No toolbar/menu/size knobs; the
|
|
740
|
+
* full card next to it carries those for whichever device it shows. */
|
|
741
|
+
function CoopPane({ excludeSerial }) {
|
|
742
|
+
const [devices, setDevices] = useState([]);
|
|
743
|
+
const [serial, setSerial] = useState("");
|
|
744
|
+
const [error, setError] = useState("");
|
|
745
|
+
const live = useLiveStream({ serial, onError: setError });
|
|
746
|
+
const grant = (device) => live.grant(device).then((ok) => { if (ok) setError(""); });
|
|
747
|
+
|
|
748
|
+
const refresh = async () => {
|
|
749
|
+
try {
|
|
750
|
+
const r = await streamPost("/stream/devices", {});
|
|
751
|
+
setDevices(r.devices);
|
|
752
|
+
// Keep my pick unless it vanished or became the other pane's device.
|
|
753
|
+
setSerial((cur) => {
|
|
754
|
+
if (cur !== "" && cur !== excludeSerial && r.devices.some((d) => d.serial === cur)) return cur;
|
|
755
|
+
const others = r.devices.filter((d) => d.serial !== excludeSerial);
|
|
756
|
+
return (others.find((d) => d.streaming) ?? others[0])?.serial ?? "";
|
|
757
|
+
});
|
|
758
|
+
} catch {
|
|
759
|
+
/* the header keeps the last device list */
|
|
760
|
+
}
|
|
761
|
+
};
|
|
762
|
+
|
|
763
|
+
useEffect(() => {
|
|
764
|
+
refresh();
|
|
765
|
+
const timer = setInterval(refresh, 5000);
|
|
766
|
+
return () => clearInterval(timer);
|
|
767
|
+
}, [excludeSerial]);
|
|
768
|
+
|
|
769
|
+
useEffect(() => { if (serial !== "" && live.streamUrl === "") grant(serial); }, [serial, live.streamUrl]);
|
|
770
|
+
|
|
771
|
+
return h("div", { className: "mc-card mc-live mc-coop-pane" },
|
|
772
|
+
h("div", { className: "mc-card-head" },
|
|
773
|
+
h("span", null, "Player B"),
|
|
774
|
+
live.streamUrl !== "" && h("span", { className: "mc-live-badge" }, h("span", { className: "mc-dot on" }), "Live"),
|
|
775
|
+
h("span", { className: "sp" }),
|
|
776
|
+
h("select", { className: "mc-input mc-coop-select", value: serial, onChange: (e) => { setSerial(e.target.value); live.reset(); } },
|
|
777
|
+
devices.length === 0 && h("option", { value: "" }, "no device"),
|
|
778
|
+
devices.map((d) => h("option", { key: d.serial, value: d.serial }, d.serial + (d.serial === excludeSerial ? " (A)" : d.streaming ? " ●" : ""))),
|
|
779
|
+
),
|
|
780
|
+
),
|
|
781
|
+
error !== "" && h("div", { className: "mc-error" }, error),
|
|
782
|
+
h("div", { className: "mc-live-stage" },
|
|
783
|
+
live.streamUrl !== ""
|
|
784
|
+
? h("img", { ...live.imgProps, src: live.streamUrl, alt: "co-op device screen", onError: () => grant(serial) })
|
|
785
|
+
: h("div", { className: "mc-live-off" }, devices.length === 0 ? "No second device — boot another emulator." : "Connecting to " + serial + "..."),
|
|
786
|
+
),
|
|
787
|
+
h("div", { className: "mc-live-cap" }, "tap or drag on the screen to drive this device"),
|
|
788
|
+
);
|
|
789
|
+
}
|
|
790
|
+
|
|
791
|
+
/** Stream section: the classic single card, plus a co-op mode that docks
|
|
792
|
+
* a second pane next to it for two-player testing. */
|
|
793
|
+
function LiveStreamSection() {
|
|
794
|
+
const [coop, setCoop] = useState(false);
|
|
795
|
+
const [serialA, setSerialA] = useState("");
|
|
796
|
+
return h("div", { className: "mc-live-section" + (coop ? " coop" : "") },
|
|
797
|
+
h(LiveDeviceCard, { coopOn: coop, onToggleCoop: () => setCoop((v) => !v), onSerial: setSerialA }),
|
|
798
|
+
coop && h(CoopPane, { excludeSerial: serialA }),
|
|
799
|
+
);
|
|
800
|
+
}
|
|
801
|
+
|
|
714
802
|
//#region setup UI (welcome + settings)
|
|
715
803
|
|
|
716
804
|
/** PaddleOCR status card: fetch /ocr, Install button, log tail. */
|
package/lib/index.js
CHANGED
|
@@ -421,8 +421,9 @@ function makeRoutes(engine, config, stream) {
|
|
|
421
421
|
const token = new URL(req.url ?? '/', 'http://localhost').searchParams.get('token') ?? ''
|
|
422
422
|
const payload = await stream.access.verifyStreamToken(token)
|
|
423
423
|
if (payload === undefined) { writeJson(res, 403, { code: 'token_invalid', error: 'the stream token is invalid or expired' }); return }
|
|
424
|
-
|
|
425
|
-
|
|
424
|
+
// 0.9.0: streams are per-serial and independent (co-op) — a token for
|
|
425
|
+
// device B never shares a pipe with device A's writer.
|
|
426
|
+
const release = stream.host.acquire(payload.serial)
|
|
426
427
|
try {
|
|
427
428
|
await stream.host.ensureStreaming({ serial: payload.serial })
|
|
428
429
|
} catch (error) {
|
|
@@ -439,15 +440,10 @@ function makeRoutes(engine, config, stream) {
|
|
|
439
440
|
writer.close()
|
|
440
441
|
release()
|
|
441
442
|
}
|
|
442
|
-
const unsubscribe = stream.host.subscribeFrames((frame) =>
|
|
443
|
-
// Frames for a different serial (after a device switch) must not leak
|
|
444
|
-
// into a capability minted for the old device.
|
|
445
|
-
if (stream.host.streamedSerial === payload.serial) writer.writeFrame(frame)
|
|
446
|
-
else teardown()
|
|
447
|
-
})
|
|
443
|
+
const unsubscribe = stream.host.subscribeFrames(payload.serial, (frame) => writer.writeFrame(frame))
|
|
448
444
|
res.on('error', teardown)
|
|
449
445
|
res.on('close', teardown)
|
|
450
|
-
const latest = stream.host.latestFrame
|
|
446
|
+
const latest = stream.host.latestFrame(payload.serial)
|
|
451
447
|
if (latest !== undefined) writer.writeFrame(latest)
|
|
452
448
|
},
|
|
453
449
|
},
|
|
@@ -465,7 +461,7 @@ function makeRoutes(engine, config, stream) {
|
|
|
465
461
|
const serial = typeof body.device === 'string' && body.device !== '' ? body.device : stream.host.streamedSerial
|
|
466
462
|
if (!serial) { writeJson(res, 409, { error: 'no device is streaming; pass a serial' }); return }
|
|
467
463
|
if (!StreamAccess.SERIAL_PATTERN.test(serial)) { writeJson(res, 400, { error: 'device must be an adb device serial' }); return }
|
|
468
|
-
if (stream.host.
|
|
464
|
+
if (!stream.host.isStreaming(serial)) {
|
|
469
465
|
const online = await stream.host.listDevices()
|
|
470
466
|
if (!online.some((device) => device.serial === serial)) { writeJson(res, 409, { error: `device ${serial} is not online` }); return }
|
|
471
467
|
}
|
|
@@ -486,9 +482,9 @@ function makeRoutes(engine, config, stream) {
|
|
|
486
482
|
if (!fence(req, res, true) || !isPost(req, res)) return
|
|
487
483
|
const body = await readBody(req, res)
|
|
488
484
|
if (body === undefined) return
|
|
489
|
-
const
|
|
490
|
-
const
|
|
491
|
-
const running = status.running && status.serial !== undefined
|
|
485
|
+
const filter = typeof body.device === 'string' && body.device !== '' ? body.device : undefined
|
|
486
|
+
const status = stream.host.status(filter)
|
|
487
|
+
const running = status.running && status.serial !== undefined
|
|
492
488
|
if (!running) { writeJson(res, 200, { ok: true, running: false }); return }
|
|
493
489
|
writeJson(res, 200, { ok: true, running: true, serial: status.serial, width: status.width, height: status.height })
|
|
494
490
|
},
|
|
@@ -506,11 +502,11 @@ function makeRoutes(engine, config, stream) {
|
|
|
506
502
|
stream.host.listDevices(),
|
|
507
503
|
DeviceBuild.androidAvds().catch(() => []),
|
|
508
504
|
])
|
|
509
|
-
const
|
|
505
|
+
const streaming = new Set(stream.host.runningSerials())
|
|
510
506
|
writeJson(res, 200, {
|
|
511
507
|
ok: true,
|
|
512
508
|
avds,
|
|
513
|
-
devices: devices.map((device) => ({ ...device, ...(device.serial
|
|
509
|
+
devices: devices.map((device) => ({ ...device, ...(streaming.has(device.serial) ? { streaming: true } : {}) })),
|
|
514
510
|
})
|
|
515
511
|
} catch (error) {
|
|
516
512
|
writeJson(res, 503, { code: 'devices_unavailable', error: error instanceof Error ? error.message : String(error) })
|
|
@@ -564,7 +560,7 @@ function makeRoutes(engine, config, stream) {
|
|
|
564
560
|
const argv = DEVICE_ACTIONS[action]
|
|
565
561
|
if (argv === undefined) { writeJson(res, 400, { code: 'unknown_action', error: `unknown device action "${action}"` }); return }
|
|
566
562
|
if (!StreamAccess.SERIAL_PATTERN.test(serial)) { writeJson(res, 400, { code: 'bad_request', error: 'device must be an adb device serial' }); return }
|
|
567
|
-
if (
|
|
563
|
+
if (!stream.host.isStreaming(serial)) {
|
|
568
564
|
const online = await stream.host.listDevices()
|
|
569
565
|
if (!online.some((device) => device.serial === serial)) { writeJson(res, 409, { code: 'device_not_found', error: `device ${serial} is not online` }); return }
|
|
570
566
|
}
|
|
@@ -594,11 +590,11 @@ function makeRoutes(engine, config, stream) {
|
|
|
594
590
|
if (action.kind === 'drag' && !(point(action.fromX, action.fromY) && point(action.toX, action.toY))) { writeJson(res, 400, { code: 'bad_request', error: 'drag needs normalized fromX,fromY,toX,toY in 0..1' }); return }
|
|
595
591
|
if (action.kind === 'button' && (typeof action.name !== 'string' || action.name === '')) { writeJson(res, 400, { code: 'bad_request', error: 'button requires a non-empty name' }); return }
|
|
596
592
|
if (action.kind === 'type' && (typeof action.text !== 'string' || action.text === '')) { writeJson(res, 400, { code: 'bad_request', error: 'type requires a non-empty text' }); return }
|
|
597
|
-
if (stream.host.
|
|
593
|
+
if (!stream.host.isStreaming(serial)) {
|
|
598
594
|
const online = await stream.host.listDevices()
|
|
599
595
|
if (!online.some((device) => device.serial === serial)) { writeJson(res, 409, { code: 'device_offline', error: `device ${serial} is not online` }); return }
|
|
600
596
|
}
|
|
601
|
-
const release = stream.host.acquire()
|
|
597
|
+
const release = stream.host.acquire(serial)
|
|
602
598
|
try {
|
|
603
599
|
let result = { ok: true }
|
|
604
600
|
switch (action.kind) {
|
|
@@ -2120,7 +2116,11 @@ function deviceAvdCreateTool() {
|
|
|
2120
2116
|
if (images.length === 0) throw new Error('no system images installed — install one first: sdkmanager --install "system-images;android-35;google_apis;x86_64".')
|
|
2121
2117
|
imageId = newestImage(images)
|
|
2122
2118
|
}
|
|
2123
|
-
|
|
2119
|
+
// Always pass -d: without a device profile avdmanager prompts "Do you
|
|
2120
|
+
// wish to create a custom hardware profile?" and crashes on EOF stdin
|
|
2121
|
+
// (Range [0, 0 + -1) out of bounds). Clone parity comes from the
|
|
2122
|
+
// config.ini merge below, not from the base profile.
|
|
2123
|
+
const created = await DeviceBuild.createAvd({ name, imageId, deviceProfile: args.device ?? 'pixel_7', cloneConfigText: cloneConfig })
|
|
2124
2124
|
return { name, imageId, configPath: created.configPath, ...(args.clone_from ? { clonedFrom: args.clone_from } : {}) }
|
|
2125
2125
|
},
|
|
2126
2126
|
})
|
|
@@ -2773,8 +2773,10 @@ function deviceStreamTool(host, access) {
|
|
|
2773
2773
|
return defineTool({
|
|
2774
2774
|
name: 'device_stream',
|
|
2775
2775
|
description: 'Drive the live device screen stream the Devices panel shows. action=start begins the frame loop for an ' +
|
|
2776
|
-
'online device and returns a signed streamUrl;
|
|
2777
|
-
'
|
|
2776
|
+
'online device and returns a signed streamUrl; streams are PER-DEVICE and independent, so a co-op test can start ' +
|
|
2777
|
+
'two emulators at once and the panel shows them side by side. status reports the streams; stop tears down one ' +
|
|
2778
|
+
'(with serial) or all of them (without). This is a human-panel feature — agents that just need to see the screen ' +
|
|
2779
|
+
'should use device_screen or device_ui_tree instead.',
|
|
2778
2780
|
parameters: {
|
|
2779
2781
|
action: { type: 'string', enum: ['status', 'start', 'stop'], description: 'What to do (default status).' },
|
|
2780
2782
|
serial: { type: 'string', description: 'Device serial (start needs an online device; omit to use the first attached one).' },
|
|
@@ -2790,6 +2792,7 @@ function deviceStreamTool(host, access) {
|
|
|
2790
2792
|
streamUrl: { type: 'string' },
|
|
2791
2793
|
width: { type: 'integer' },
|
|
2792
2794
|
height: { type: 'integer' },
|
|
2795
|
+
serials: { type: 'array', description: 'All live stream serials (only when more than one device streams).' },
|
|
2793
2796
|
presentationMeta: {
|
|
2794
2797
|
type: 'object',
|
|
2795
2798
|
additionalProperties: true,
|
|
@@ -2799,21 +2802,25 @@ function deviceStreamTool(host, access) {
|
|
|
2799
2802
|
},
|
|
2800
2803
|
render: (_args, value) => {
|
|
2801
2804
|
const v = value ?? { action: 'status', running: false }
|
|
2805
|
+
const multi = Array.isArray(v.serials) && v.serials.length > 1 ? ` (+${v.serials.length - 1} more: ${v.serials.filter((s) => s !== v.serial).join(', ')})` : ''
|
|
2802
2806
|
const text = v.action === 'start' && v.streamUrl
|
|
2803
|
-
? `Streaming ${v.serial} (${v.width}x${v.height}) — ${v.streamUrl}`
|
|
2804
|
-
: `Stream ${v.action}: running=${v.running}${v.serial ? ` (${v.serial})` : ''}`
|
|
2807
|
+
? `Streaming ${v.serial} (${v.width}x${v.height})${multi} — ${v.streamUrl}`
|
|
2808
|
+
: `Stream ${v.action}: running=${v.running}${v.serial ? ` (${v.serial})` : ''}${v.action === 'status' && Array.isArray(v.serials) && v.serials.length > 1 ? ` streams: ${v.serials.join(', ')}` : ''}`
|
|
2805
2809
|
return [{ type: 'text', text }]
|
|
2806
2810
|
},
|
|
2807
2811
|
},
|
|
2808
2812
|
async execute(args) {
|
|
2809
2813
|
const action = args.action ?? 'status'
|
|
2810
2814
|
if (action === 'stop') {
|
|
2811
|
-
|
|
2815
|
+
// 0.9.0 co-op: a serial stops one stream; without one, tear down all.
|
|
2816
|
+
if (typeof args.serial === 'string' && args.serial !== '') await host.stop(args.serial)
|
|
2817
|
+
else await host.stopAll()
|
|
2812
2818
|
return { action, running: false }
|
|
2813
2819
|
}
|
|
2814
2820
|
if (action === 'status') {
|
|
2815
2821
|
const s = host.status()
|
|
2816
|
-
|
|
2822
|
+
const serials = host.runningSerials()
|
|
2823
|
+
return { action, running: s.running, ...(s.serial !== undefined ? { serial: s.serial } : {}), ...(s.width !== undefined ? { width: s.width, height: s.height } : {}), ...(serials.length > 1 ? { serials } : {}) }
|
|
2817
2824
|
}
|
|
2818
2825
|
const serial = await requireAndroidDevice(args.serial)
|
|
2819
2826
|
const online = await host.listDevices()
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dsh-mobilecode",
|
|
3
3
|
"description": "MobileCode for the dsh web GUI: detect iOS/Android projects, run preview servers, and drive the simulator/emulator from the session — 37 agent tools (device_run, device_screen, device_ui_tree, device_ui_rows, device_tap_row, device_tap_element, device_wait_for, device_scroll_to, device_input, device_batch, device_intent, device_connect, device_pair_qr, device_perf, device_meminfo, device_backtrace, device_display, device_avd_create, device_pair_capture, device_app_info, device_install, device_uninstall, device_reboot, device_log, live screen stream, multimodal screenshots) plus a host-mediated co-op mesh hub (random callsigns, JSON pub/sub, tunable latency/jitter/drop/dup/throttle) so two virtual devices — and the AI watching them — can talk in real time. One classified adb boundary and a Wi-Fi connect/pair QR flow in the Connection settings tab. Hot-pluggable — mounted via the profile bundle list + cordis.patch.yml, no dsh source changes.",
|
|
4
|
-
"version": "0.
|
|
4
|
+
"version": "0.9.0",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"packageManager": "pnpm@11.22.0",
|
|
7
7
|
"engines": {
|