simframe 0.7.0 → 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -4,11 +4,11 @@
4
4
  [![npm](https://img.shields.io/npm/v/simframe.svg)](https://www.npmjs.com/package/simframe)
5
5
  [![license](https://img.shields.io/npm/l/simframe.svg)](./LICENSE)
6
6
 
7
- **Eyes, hands and memory for an agent driving the iOS Simulator.**
7
+ **Eyes, hands and memory for an agent driving the iOS Simulator or an Android emulator.**
8
8
 
9
9
  [Website](https://lvlrsajjad.github.io/simframe/) · [npm](https://www.npmjs.com/package/simframe)
10
10
 
11
- An agent driving the iOS Simulator is slow for three reasons, and only the first
11
+ An agent driving a simulator is slow for three reasons, and only the first
12
12
  one is obvious:
13
13
 
14
14
  1. **Every look is a wait.** `simctl io screenshot` costs ~130 ms of blocking
@@ -37,6 +37,7 @@ Same four-tab navigation flow, on a real production app:
37
37
  | | Before | With simframe |
38
38
  | --- | --- | --- |
39
39
  | Look at the screen | ~130–400 ms, blocking | **~20 ms**, already captured |
40
+
40
41
  | "Did anything change?" | a full image | **~2 ms**, text only |
41
42
  | Finding a control | read tree (~570 ms) + reason | **~1 ms** from memory |
42
43
  | A 4-step flow, verified | 4+ model round trips | **1 call**, 3.6 s |
@@ -44,6 +45,13 @@ Same four-tab navigation flow, on a real production app:
44
45
  | A 10-step flow | 10 turns, 10 images (~16,000 tokens at best) | **1 turn, 0 images, ~1,650 characters** |
45
46
  | Reading a screen | an image: ~1,600 tokens, no tap points | **~330 tokens** of text, with tap points |
46
47
 
48
+ Every figure above is the cost inside a live process — the MCP server, or the
49
+ daemon answering a socket — which is how an agent actually uses simframe. A
50
+ one-shot `simframe` command from a shell pays about 200 ms of Node startup on
51
+ top, and a frame sitting on an idle screen can be older than 20 ms because the
52
+ capture loop throttles when nothing moves. `~20 ms` is the read, not the
53
+ process.
54
+
47
55
  The four-tab tour, three times back to back from a cleared memory:
48
56
 
49
57
  | Pass | Wall clock | Steps verified | Controls from memory |
@@ -67,10 +75,16 @@ npm install -g simframe
67
75
  simframe doctor
68
76
  ```
69
77
 
70
- The first `simframe start` builds a small Swift daemon from source — a few
71
- seconds, once. It needs the Xcode command line tools, which you already have if
72
- you have a simulator. Without them simframe falls back to the original
73
- `simctl` loop and says so.
78
+ **Whichever of those two commands you run first** builds a small Swift daemon
79
+ from source — including `doctor`, which is why a cold `doctor` takes around 15
80
+ seconds and every later one takes two. It needs the Xcode command line tools,
81
+ which you already have if you have a simulator. Without them simframe falls
82
+ back to the original `simctl` loop and says so.
83
+
84
+ If more than one simulator is booted, name the one you mean — `--device=<udid>`,
85
+ or `export SIMFRAME_DEVICE=<udid>` once per shell. simframe refuses to choose
86
+ for you, because the first booted device is nobody's idea of "yours" and the
87
+ command that would act on it is a tap.
74
88
 
75
89
  `doctor` checks each capability separately and tells you what you have:
76
90
 
@@ -143,7 +157,7 @@ boundary hands it frames and nothing above it knows what a simulator is.
143
157
  | Read labels + coordinates from pixels | yes | the same Vision OCR + CV, off the same PNG |
144
158
  | Screen map, refs, screen memory, the graph | yes | unchanged above the boundary |
145
159
  | Tap, type, swipe, keys | yes | the console's `event mouse` as a real down/move/up, `event text` for characters, `input keyevent` for keys |
146
- | Clipboard | yes | the emulator's gRPC `setClipboard`, over `node:http2`, no dependency |
160
+ | Clipboard, and `paste` into a field | yes | the emulator's gRPC `setClipboard`, over `node:http2`, no dependency, then `KEYCODE_PASTE` to deliver it |
147
161
  | List/resolve devices, launch, terminate, open a URL, permissions | yes | `adb`, with the permission state read back off the device |
148
162
  | Accessibility tree | **not available (OCR + CV only)** | `uiautomator dump` costs **2,012 ms** a read, against 45 ms for the iOS tree. See [`docs/DEFERRED.md`](docs/DEFERRED.md) |
149
163
 
@@ -154,6 +168,17 @@ simframe ui --device=emulator-5554
154
168
  simframe do --device=emulator-5554 flow.json
155
169
  ```
156
170
 
171
+ A host with a booted simulator **and** a booted emulator has no default, and
172
+ simframe will not pick one for you: preferring iOS because it came first would
173
+ tap a simulator while you were driving an emulator, and acting on the wrong
174
+ device is worse than refusing. So a command with no device names both and stops.
175
+ `--device` answers it per command; `SIMFRAME_DEVICE` answers it per shell:
176
+
177
+ ```bash
178
+ export SIMFRAME_DEVICE=emulator-5554
179
+ simframe ui # the emulator, without saying so every time
180
+ ```
181
+
157
182
  The tree is a deliberate omission, not an oversight. Making it fast needs a
158
183
  resident instrumentation APK on the device — the shape uiautomator2, Maestro and
159
184
  Appium all converged on — and that would be simframe's first runtime artifact
@@ -517,8 +542,14 @@ simframe frame --out=now.png # newest frame, native resolution, to a file
517
542
  simframe strip --count=6 # contact sheet, for an animation
518
543
  simframe doctor --strict # any degraded layer is a non-zero exit
519
544
  simframe start / status / stop [--force] / devices
545
+ simframe ui --device=emulator-5554 # or export SIMFRAME_DEVICE once
520
546
  ```
521
547
 
548
+ `--device` takes `--device=X` and `--device X` alike. It used to take only the
549
+ first: the space form set the flag to `true` and then resolved a device named
550
+ "true", which is a poor answer to a flag `doctor`'s own advice tells you to
551
+ type.
552
+
522
553
  ### The Claude Code skill
523
554
 
524
555
  [`skills/simframe/SKILL.md`](skills/simframe/SKILL.md) teaches the CLI path
package/package.json CHANGED
@@ -1,11 +1,13 @@
1
1
  {
2
2
  "name": "simframe",
3
- "version": "0.7.0",
3
+ "version": "0.8.0",
4
4
  "mcpName": "io.github.lvlrSajjad/simframe",
5
- "description": "Always-warm iOS Simulator frames: agents read the screen in ~20ms instead of waiting on screenshots. MCP server + CLI.",
5
+ "description": "Always-warm iOS Simulator and Android emulator frames: agents read the screen in ~20ms instead of waiting on screenshots. MCP server + CLI.",
6
6
  "keywords": [
7
7
  "ios",
8
8
  "simulator",
9
+ "android",
10
+ "emulator",
9
11
  "mcp",
10
12
  "model-context-protocol",
11
13
  "claude",
@@ -123,4 +123,19 @@ if (missing.length) {
123
123
  );
124
124
  process.exit(1);
125
125
  }
126
+ // The MCP Registry rejects a description over 100 characters, and it does so at
127
+ // `mcp-publisher validate` — after the tag is pushed and the release has begun.
128
+ // A 0.7.1 release found that out the hard way with a 113-character one. The
129
+ // constraint is the registry's; discovering it locally is this script's job.
130
+ const DESCRIPTION_MAX = 100;
131
+ const server = JSON.parse(fs.readFileSync(path.join(ROOT, 'server.json'), 'utf8'));
132
+ if (server.description.length > DESCRIPTION_MAX) {
133
+ console.error(
134
+ `server.json description is ${server.description.length} characters; the MCP Registry accepts ` +
135
+ `${DESCRIPTION_MAX}. It would fail at validate, with the tag already pushed.`,
136
+ );
137
+ process.exit(1);
138
+ }
139
+ console.log(`server.json description ${server.description.length}/${DESCRIPTION_MAX} chars`);
140
+
126
141
  console.log('ok — every build input ships');
@@ -257,7 +257,14 @@ if (first) {
257
257
  // to defend — and it is how this check has failed twice.
258
258
  if (moved) {
259
259
  const stale = await cli(['find', `#${first.ref}`], { expectFail: true });
260
- check(/different screen|read the screen/i.test(stale),
260
+ // Matched on prose, which is this check's weakness: simframe refused
261
+ // correctly with "#1 cannot be trusted here — simframe does not recognise
262
+ // this screen. Read it again", and the check failed because that wording was
263
+ // not one of the two it knew. The refusal is what matters, so the
264
+ // alternation covers how a refusal is actually phrased; the durable fix is a
265
+ // machine-readable reason on the failure, which `find --json` does not yet
266
+ // carry.
267
+ check(/different screen|read the screen|read it again|does not recognise this screen|cannot be trusted/i.test(stale),
261
268
  'a ref numbered on another screen refuses instead of tapping those coordinates',
262
269
  stale.trim().split('\n')[0]?.slice(0, 90));
263
270
  }
@@ -398,7 +405,13 @@ check(Array.isArray(nonsense.known) && nonsense.known.length > 0,
398
405
 
399
406
  const target = screens[0];
400
407
  const walked = await jsonRetry(['goto', target.hash], { allowFail: true });
401
- const outcomes = ['no-route', 'unreplayable-edge', 'ambiguous', 'unknown-screen'];
408
+ // `no-identity` is new in 0.7.2 and belongs here: `goto` used to *throw*
409
+ // `Cannot read properties of null (reading 'slice')` on a screen whose token set
410
+ // was empty, because `hashTokens` returns null for one on purpose. A crash is
411
+ // neither walking there nor naming why it cannot, so this check failed with an
412
+ // empty detail — the reason was `undefined` — and the check was right to fail.
413
+ // Now there is a name for it, and the list has to know the name.
414
+ const outcomes = ['no-route', 'unreplayable-edge', 'ambiguous', 'unknown-screen', 'no-identity'];
402
415
  check(walked.ok === true || outcomes.includes(walked.reason),
403
416
  'and asked for a screen it knows, it either walks there or names why it cannot',
404
417
  walked.ok ? (walked.already ? 'already there' : `walked ${walked.ranSteps} step(s)`) : walked.reason);
package/src/actions.js CHANGED
@@ -6,7 +6,7 @@ import * as api from './index.js';
6
6
  import * as graph from './graph.js';
7
7
  import * as input from './input.js';
8
8
  import * as intent from './intent.js';
9
- import { launchApp, openUrl, setPasteboard, setPermission, terminateApp } from './platform/index.js';
9
+ import { launchApp, openUrl, setPermission, terminateApp } from './platform/index.js';
10
10
 
11
11
  const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
12
12
  const MAX_PAUSE_MS = 5000;
@@ -272,6 +272,33 @@ export async function runScript(
272
272
  };
273
273
  }
274
274
 
275
+ /**
276
+ * Tap a field and wait for it to take focus, for the steps that then put text
277
+ * in it.
278
+ *
279
+ * `locate`, not `tapLabel`: tapLabel asks the accessibility tree directly, so a
280
+ * field only OCR can see was untypeable and a selector (`#4`, `@x,y`) meant
281
+ * nothing here. The returned `quiet` says when the field never visibly took
282
+ * focus — usually fine, since a field that already had focus does not move, but
283
+ * also exactly what a tap that missed looks like, so the caller should be told.
284
+ */
285
+ async function focusField(deviceQuery, udid, step, ctx) {
286
+ const found = await api.locate(deviceQuery, step.into, { index: step.index, refresh: step.refresh });
287
+ await input.tapPoint(udid, found.target.x, found.target.y);
288
+ const focused = await api.waitFor(deviceQuery, {
289
+ mode: 'settle',
290
+ stableMs: FOCUS_STABLE_MS,
291
+ reactionMs: FOCUS_REACTION_MS,
292
+ timeoutMs: FOCUS_TIMEOUT_MS,
293
+ options: ctx.options,
294
+ });
295
+ return {
296
+ found,
297
+ where: `"${found.target.label}" at ${found.target.x},${found.target.y}`,
298
+ quiet: focused.satisfied ? '' : ' [the field did not visibly take focus]',
299
+ };
300
+ }
301
+
275
302
  async function runStep(deviceQuery, udid, step, ctx) {
276
303
  switch (step.action) {
277
304
  case 'tap': {
@@ -298,33 +325,25 @@ async function runStep(deviceQuery, udid, step, ctx) {
298
325
  }
299
326
  case 'type': {
300
327
  if (step.into) {
301
- // locate, not tapLabel: tapLabel asks the accessibility tree directly,
302
- // so a field that only OCR can see was untypeable, and a selector
303
- // (`#4`, `@x,y`) meant nothing here.
304
- const found = await api.locate(deviceQuery, step.into, { index: step.index, refresh: step.refresh });
305
- await input.tapPoint(udid, found.target.x, found.target.y);
306
- const focused = await api.waitFor(deviceQuery, {
307
- mode: 'settle',
308
- stableMs: FOCUS_STABLE_MS,
309
- reactionMs: FOCUS_REACTION_MS,
310
- timeoutMs: FOCUS_TIMEOUT_MS,
311
- options: ctx.options,
312
- });
328
+ const field = await focusField(deviceQuery, udid, step, ctx);
313
329
  await input.typeText(udid, step.text ?? step.value);
314
- // Say when the field never visibly took focus. It is usually fine — a
315
- // field that was already focused does not move — but it is also what a
316
- // tap that missed looks like, and the caller should be able to tell.
317
- const quiet = focused.satisfied ? '' : ' [the field did not visibly take focus]';
318
- return `typed into "${found.target.label}" at ${found.target.x},${found.target.y}${quiet}`;
330
+ return `typed into ${field.where}${field.quiet}`;
319
331
  }
320
332
  await input.typeText(udid, step.text ?? step.value);
321
333
  return 'typed text';
322
334
  }
323
335
  case 'paste': {
324
- // Long strings are much faster on the pasteboard than through the keyboard.
325
- await setPasteboard(udid, step.text ?? step.value);
326
- if (step.into) await input.tapLabel(udid, step.into, { index: step.index, durationMs: 900 });
327
- return 'placed text on the pasteboard';
336
+ // Long strings are much faster on the pasteboard than through the
337
+ // keyboard. `pasteText` delivers the keystroke as well as setting the
338
+ // pasteboard, and throws if it cannot — this step used to do neither and
339
+ // report success anyway.
340
+ if (step.into) {
341
+ const field = await focusField(deviceQuery, udid, step, ctx);
342
+ await input.pasteText(udid, step.text ?? step.value);
343
+ return `pasted into ${field.where}${field.quiet}`;
344
+ }
345
+ await input.pasteText(udid, step.text ?? step.value);
346
+ return 'pasted into the focused field';
328
347
  }
329
348
  case 'swipe': {
330
349
  const from = { x: step.from?.[0] ?? step.from?.x, y: step.from?.[1] ?? step.from?.y };
package/src/cli.js CHANGED
@@ -86,14 +86,39 @@ The reliable pattern around an action is:
86
86
  simframe state --since=$H
87
87
  `;
88
88
 
89
+ /**
90
+ * Flags that take a value, so `--flag value` can mean what it looks like.
91
+ *
92
+ * Deliberately not every flag: `--json`, `--refresh` and friends have a bare
93
+ * form, and letting those swallow the next argument would turn
94
+ * `simframe tap --refresh Save` into a tap on nothing.
95
+ */
96
+ const VALUE_FLAGS = new Set([
97
+ 'ago', 'count', 'detail', 'device', 'durationMs', 'engine', 'filter', 'fps', 'index', 'maxDim',
98
+ 'mode', 'out', 'ringSize', 'since', 'spanMs', 'stableMs', 'timeoutMs',
99
+ ]);
100
+
89
101
  function parseArgs(argv) {
90
102
  const flags = {};
91
103
  const positional = [];
92
- for (const arg of argv) {
104
+ for (let i = 0; i < argv.length; i += 1) {
105
+ const arg = argv[i];
93
106
  if (arg.startsWith('--')) {
94
107
  const [key, value] = arg.slice(2).split('=');
95
108
  const camel = key.replace(/-([a-z])/g, (_, c) => c.toUpperCase());
96
- flags[camel] = value === undefined ? true : value;
109
+ if (value !== undefined) {
110
+ flags[camel] = value;
111
+ } else if (VALUE_FLAGS.has(camel) && argv[i + 1] != null && !argv[i + 1].startsWith('--')) {
112
+ // `--device X` as well as `--device=X`. Only for flags whose bare form
113
+ // means nothing: `simframe doctor --device B55AB0AE` used to set
114
+ // `device` to `true`, push the udid to positional, and resolve the
115
+ // literal string "true" — and doctor's own advice is to name a device
116
+ // with --device, which is exactly how someone would write it.
117
+ flags[camel] = argv[i + 1];
118
+ i += 1;
119
+ } else {
120
+ flags[camel] = true;
121
+ }
97
122
  } else {
98
123
  positional.push(arg);
99
124
  }
@@ -218,6 +243,11 @@ async function main() {
218
243
  `stopped ${stopped} daemon${stopped === 1 ? '' : 's'}` +
219
244
  (inUse ? `; left ${inUse} in use by another client (pass --force to stop anyway)` : ''),
220
245
  );
246
+ // Asked to stop one device, refused, and exited 0 — which is a success
247
+ // code for work not done, and a script checking `$?` could not tell the
248
+ // difference. `--all` is informational by nature, so it keeps exiting 0
249
+ // when it skips a device somebody else holds.
250
+ if (!flags.all && inUse && !stopped) process.exitCode = 1;
221
251
  return;
222
252
  }
223
253
 
@@ -800,6 +830,25 @@ async function doctor({ json = false, strict = false, device } = {}) {
800
830
  const wanted = await resolveDevice(device);
801
831
  booted = booted.filter((d) => d.udid === wanted.udid);
802
832
  }
833
+ // Which devices get *probed*, as opposed to listed. The probes below start
834
+ // a capture loop and read frames, and doctor used to do that to every
835
+ // booted device on the host. On a shared machine that means starting a
836
+ // daemon on a colleague's simulator and capturing their screen to answer a
837
+ // question about this one. Listing is free and stays; probing is not, so
838
+ // without --device it goes to a device already running its own capture loop
839
+ // (nothing new is started), or to the only booted device, and otherwise to
840
+ // none, with a line saying which flag would pick one.
841
+ let probed = booted;
842
+ if (!device && booted.length > 1) {
843
+ probed = booted.filter((d) => engineModule.runningEngine(d.udid));
844
+ if (probed.length !== 1) {
845
+ probed = [];
846
+ add('device probes', 'warn',
847
+ `${booted.length} devices are booted and none is clearly yours — name one with --device ` +
848
+ 'to check its capture, input and accessibility layers',
849
+ { key: 'probes.skipped', value: booted.length });
850
+ }
851
+ }
803
852
  // `deviceNoun` earns its place here: one platform's devices are called by
804
853
  // its own word, and a mixed set by the neutral one. An emulator reported as
805
854
  // a "booted simulator" is the same small lie as an emulator reported as
@@ -807,7 +856,7 @@ async function doctor({ json = false, strict = false, device } = {}) {
807
856
  const nouns = [...new Set(booted.map((d) => capabilitiesFor(d.udid) && PLATFORMS[d.platform].deviceNoun))];
808
857
  add(`booted ${nouns.length === 1 ? nouns[0] : 'device'}`, booted.length ? 'ok' : 'warn',
809
858
  booted.map((d) => `${d.name} (${d.runtime})`).join(', ') || 'none');
810
- for (const d of booted) {
859
+ for (const d of probed) {
811
860
  const input = await import('./input.js');
812
861
  const control = await import('./control.js');
813
862
  // What this device's platform can do at all. Without asking, doctor
@@ -871,9 +920,9 @@ async function doctor({ json = false, strict = false, device } = {}) {
871
920
  ax.available ? `${ax.name}: ${ax.version}` : `unavailable: ${ax.reason}`,
872
921
  { key: 'ax.driver', value: ax.name });
873
922
  }
874
- if (booted.length) {
923
+ if (probed.length) {
875
924
  const t0 = Date.now();
876
- const res = await api.getFrame(booted[0].udid);
925
+ const res = await api.getFrame(probed[0].udid);
877
926
  add('capture', 'ok',
878
927
  `frame #${res.state.seq} ${res.width}x${res.height} in ${Date.now() - t0}ms (age ${res.ageMs}ms)`,
879
928
  { key: 'capture.frames', value: res.state.seq });
@@ -881,7 +930,7 @@ async function doctor({ json = false, strict = false, device } = {}) {
881
930
  // to ask the capture loop rather than look at the frames. `fail`, not
882
931
  // `warn`: nothing here is degraded-but-working, and the cure is a device
883
932
  // restart that simframe deliberately does not perform.
884
- for (const d of booted) {
933
+ for (const d of probed) {
885
934
  const live = api.liveness(d.udid, (await api.getState(d.udid)).state);
886
935
  if (live.stalled) add(`capture health (${d.name})`, 'fail', live.note, { key: 'capture.stalled', value: true });
887
936
  }
@@ -893,8 +942,21 @@ async function doctor({ json = false, strict = false, device } = {}) {
893
942
  // doctor is a diagnostic, not a way to start things. If it had to start a
894
943
  // daemon to answer "which engine is in use", it stops it again rather than
895
944
  // leaving a detached process behind.
945
+ //
946
+ // And it says when it could not. This was `catch { /* best effort */ }`, and
947
+ // best effort silently failed: a stop refused because another client holds
948
+ // the device left a capture loop running on a machine somebody else was
949
+ // using, with doctor reporting a clean bill of health. A diagnostic that
950
+ // leaves something behind has to name it.
896
951
  for (const udid of startedHere) {
897
- try { await api.stopDaemon(udid); } catch { /* best effort */ }
952
+ try {
953
+ await api.stopDaemon(udid);
954
+ } catch (err) {
955
+ add('cleanup', 'warn',
956
+ `started a capture loop on ${udid} to answer a question and could not stop it again ` +
957
+ `(${err.message}) — stop it with: simframe stop --device=${udid}`,
958
+ { key: 'cleanup.left', value: udid });
959
+ }
898
960
  }
899
961
 
900
962
  const failed = checks.filter((c) => c.level === 'fail');
package/src/daemon.js CHANGED
@@ -159,6 +159,12 @@ export async function runDaemon(device, options = {}) {
159
159
  store.writeCaptureHealth(udid, null);
160
160
  }
161
161
  consecutiveErrors = 0;
162
+ // And the clock, which it did not. A second stall then kept the first
163
+ // one's timestamp — `stalledSince ??=` only fills a null — and reported
164
+ // "unreadable for 4 hours" about a wedge four seconds old. The Swift loop
165
+ // resets it (main.swift), which is what made this a missed line rather
166
+ // than a difference of opinion between the two engines.
167
+ stalledSince = null;
162
168
 
163
169
  const hash = frameHash(bmp);
164
170
  const layout = layoutHash(bmp);
package/src/index.js CHANGED
@@ -223,9 +223,19 @@ function spawnNodeDaemon(udid, options) {
223
223
  for (const key of ['fps', 'maxDim', 'ringSize', 'idleExitMs']) {
224
224
  if (options[key] != null) args.push(`--${key}=${options[key]}`);
225
225
  }
226
+ // stderr goes to the device's own log, the way the Swift daemon's does
227
+ // (engine.js). It was `stdio: 'ignore'`, so anything this loop said about
228
+ // itself went to /dev/null — including the Android backend's notice that the
229
+ // console capture path had failed and it had fallen back to adb at five times
230
+ // the latency. A silent fallback is the failure mode this project has been
231
+ // bitten by twice; it does not get a third time for want of a file handle.
232
+ // The directory may not exist yet on a first start, and `writeCaptureHealth`
233
+ // already taught this project that an unwritable path here fails silently.
234
+ fs.mkdirSync(store.paths(udid).dir, { recursive: true });
235
+ const log = fs.openSync(store.paths(udid).log, 'a');
226
236
  const child = spawn(process.execPath, args, {
227
237
  detached: true,
228
- stdio: 'ignore',
238
+ stdio: ['ignore', log, log],
229
239
  env: process.env,
230
240
  });
231
241
  child.unref();
package/src/input.js CHANGED
@@ -3,7 +3,7 @@
3
3
  // even available?" before it answers anything else.
4
4
  import { execFile } from 'node:child_process';
5
5
  import * as control from './control.js';
6
- import { geometryFor, inputDriverFor } from './platform/index.js';
6
+ import { capabilitiesFor, geometryFor, inputDriverFor, setPasteboard } from './platform/index.js';
7
7
  import { promisify } from 'node:util';
8
8
 
9
9
  const run = promisify(execFile);
@@ -207,6 +207,14 @@ export async function describeAll(udid) {
207
207
  /* daemon went away mid-call; fall through to idb */
208
208
  }
209
209
  }
210
+ // A platform with no accessibility tree is not a machine missing idb.
211
+ // `doctor` learned that when it reported "input driver: idb" for an emulator;
212
+ // this path had not, so every Android screen map carried a note telling the
213
+ // reader to brew-install a tool that has never spoken to an Android device —
214
+ // and on a machine where idb *is* installed, spawned it against an emulator
215
+ // serial on every map build.
216
+ const ax = capabilitiesFor(udid).ax;
217
+ if (!ax.supported) throw new Error(`no accessibility tree on this device — ${ax.note}`);
210
218
  // Passing --json here yields empty output; the default already emits JSON.
211
219
  const out = await idb(['ui', 'describe-all', '--udid', udid]);
212
220
  const nodes = [];
@@ -357,6 +365,39 @@ export async function typeText(udid, value) {
357
365
  await idb(['ui', 'text', '--udid', udid, String(value)]);
358
366
  }
359
367
 
368
+ /**
369
+ * Put text on the pasteboard **and deliver it** into the focused field.
370
+ *
371
+ * The delivery is the whole point, and it is what was missing: the `paste` step
372
+ * used to set the pasteboard, long-press the field, and report success while
373
+ * the field stayed empty — on both platforms, not just the one it was filed
374
+ * against. iOS had the mechanism and did not use it (the daemon's `paste` is
375
+ * pbcopy *plus* Cmd-V) and Android had the keycode sitting unused in `KEYS`.
376
+ *
377
+ * When nothing can deliver the keystroke this throws rather than returning a
378
+ * cheerful description of half the job. A step that cannot do what it says has
379
+ * to say so; `type` still works on every device.
380
+ */
381
+ export async function pasteText(udid, value) {
382
+ const own = inputDriverFor(udid);
383
+ if (own?.key) {
384
+ // The clipboard goes over gRPC; KEYCODE_PASTE is what puts it in the field.
385
+ await setPasteboard(udid, String(value));
386
+ await own.key(udid, 'paste');
387
+ return;
388
+ }
389
+ if (control.available(udid)) {
390
+ // One round trip: the daemon copies and presses Cmd-V.
391
+ await control.paste(udid, String(value));
392
+ return;
393
+ }
394
+ await setPasteboard(udid, String(value));
395
+ throw new Error(
396
+ 'the text is on the pasteboard but nothing here can paste it: the keystroke needs the simframe ' +
397
+ 'daemon (start it with `simframe start`), or use `type`, which carries the characters itself',
398
+ );
399
+ }
400
+
360
401
  /** Key events rather than text: for shortcuts and search-as-you-type. */
361
402
  export async function typeKeys(udid, value) {
362
403
  const own = inputDriverFor(udid);
package/src/navigate.js CHANGED
@@ -51,6 +51,13 @@ export async function goto(deviceQuery, target, { options, ...runOptions } = {})
51
51
  return { ok: true, already: true, screen: found.name, steps: [] };
52
52
  }
53
53
 
54
+ // `hashTokens` returns null for an empty token set on purpose — a constant
55
+ // hash for "I could read nothing" is the self-confirming-emptiness bug. So a
56
+ // screen with no identity has to be reported, not sliced: this threw
57
+ // `Cannot read properties of null (reading 'slice')` instead of answering.
58
+ // Not hypothetical on Android, where README's own table puts the launcher at
59
+ // one token.
60
+ if (!here.hash) return { ok: false, reason: 'no-identity', to: found.name };
54
61
  const path_ = graph.route(udid, { hash: here.hash, tokens: here.tokens }, found.node.hash);
55
62
  if (!path_) return { ok: false, reason: 'no-route', from: here.hash.slice(0, 8), to: found.name };
56
63
 
@@ -65,7 +72,7 @@ export async function goto(deviceQuery, target, { options, ...runOptions } = {})
65
72
  steps,
66
73
  ranSteps: result.ranSteps,
67
74
  results: result.results,
68
- arrived: arrived.hash.slice(0, 8),
75
+ arrived: arrived.hash ? arrived.hash.slice(0, 8) : null,
69
76
  };
70
77
  }
71
78
 
@@ -10,6 +10,10 @@
10
10
  // below, and the reason two of them are not the obvious command:
11
11
  //
12
12
  // emulator console `screenrecord screenshot <dir>` 20 ms, written host-side
13
+ //
14
+ // Those are screenshot costs. A *frame* — screenshot plus the resize that makes
15
+ // it readable — measures 41 ms, which is the number README quotes; the two were
16
+ // quoted interchangeably in three files and agreed with each other in none.
13
17
  // adb exec-out screencap -p 113 ms, 9 KB over adb
14
18
  // adb exec-out screencap (raw RGBA, 3.7 MB) 218 ms — the transfer, not the encode
15
19
  // adb shell getprop x4 in one hop 28 ms
@@ -107,7 +111,14 @@ async function fetchDevices() {
107
111
  const serials = [];
108
112
  for (const line of stdout.split('\n').slice(1)) {
109
113
  const [serial, state] = line.trim().split(/\s+/);
110
- if (serial && state) serials.push({ serial, adbState: state });
114
+ // Only emulators. Physical devices are a stated non-goal, and this list is
115
+ // "devices simframe can drive" — `ownsUdid` claims emulator serials and
116
+ // nothing else, so listing a plugged-in phone here made listing and routing
117
+ // disagree: `devices` offered it, a bare `resolveDevice` could return it,
118
+ // and the tap that followed failed with a complaint about the serial rather
119
+ // than an honest "not supported". A unit test asserts the two agree; it
120
+ // passed only because nobody had a phone attached.
121
+ if (serial && state && ownsUdid(serial)) serials.push({ serial, adbState: state });
111
122
  }
112
123
  const out = [];
113
124
  for (const { serial, adbState } of serials) {
@@ -165,6 +176,18 @@ async function resolveDevice(query, opts) {
165
176
  : 'no booted emulator (start one with `emulator -avd <name>`)',
166
177
  );
167
178
  }
179
+ // See the same guard in ios.js: an arbitrary pick is a tap on the wrong
180
+ // device, and this backend can act too.
181
+ if (booted.length > 1) {
182
+ throw Object.assign(
183
+ new Error(
184
+ `${booted.length} emulators are running and none was named: ` +
185
+ `${booted.map((d) => `${d.name} (${d.udid})`).join(', ')} — name one with --device, ` +
186
+ 'or set SIMFRAME_DEVICE to pick a default for this shell',
187
+ ),
188
+ { ambiguous: true },
189
+ );
190
+ }
168
191
  return booted[0];
169
192
  }
170
193
  const q = loose(query);
@@ -556,6 +579,12 @@ async function screenshot(udid, outFile, { mask: _mask = 'ignored' } = {}) {
556
579
  try {
557
580
  const { stdout } = await adb(udid, ['exec-out', 'screencap', '-p'], { encoding: 'buffer', timeout: 20_000 });
558
581
  fs.writeFileSync(outFile, stdout);
582
+ // stderr, not a store write: a backend below the boundary has no business
583
+ // knowing where simframe keeps its files. The capture loop is now spawned
584
+ // with its stderr pointed at `daemon.log` (src/index.js), which is what
585
+ // makes this line readable at all — it used to go to a daemon started
586
+ // `stdio: 'ignore'`, so a five-times-slower capture path could be in
587
+ // effect for a whole session with no trace of it anywhere.
559
588
  process.env.SIMFRAME_QUIET === '1' ||
560
589
  process.stderr.write(`simframe: emulator console unavailable (${consoleErr.message}); used adb screencap\n`);
561
590
  } catch (adbErr) {
@@ -796,7 +825,14 @@ async function launchApp(udid, bundleId, { args = [], env = {}, terminateFirst =
796
825
  }
797
826
 
798
827
  async function terminateApp(udid, bundleId) {
799
- await adb(udid, ['shell', 'am', 'force-stop', bundleId]);
828
+ // `am` reports failure on stdout and still exits 0 — the same trap launchApp
829
+ // and openUrl already check for. Without this, terminating a package that is
830
+ // not installed answered "terminated com.typo.app".
831
+ const { stdout, stderr } = await adb(udid, ['shell', 'am', 'force-stop', bundleId]);
832
+ const error = /^Error:.*$/m.exec(`${stdout}${stderr}`);
833
+ if (error) {
834
+ throw new Error(`could not terminate ${bundleId}: ${error[0].replace(/^Error:\s*/, '')}`);
835
+ }
800
836
  }
801
837
 
802
838
  async function openUrl(udid, url) {
@@ -928,10 +964,15 @@ function toolchain() {
928
964
  * There is no framebuffer engine, so `screenshot` is not a downgrade on this
929
965
  * platform and doctor must not report it as one.
930
966
  *
931
- * Input and the accessibility tree: not yet, and said so rather than answered
932
- * with the iOS driver's name. Both paths are measured and unwired — the console's
933
- * `event mouse` puts a real down/move/up on the touch screen in ~20 ms, and
934
- * `uiautomator dump` costs 2 s a read, which is the interesting problem.
967
+ * Input: yes, through the emulator console — `event mouse` puts a real
968
+ * down/move/up on the touch screen, and `event text` carries characters rather
969
+ * than key positions, so a non-Latin host layout cannot reinterpret them. This
970
+ * comment said "not yet" for a release after it shipped; a stale comment above
971
+ * a live value is worse than no comment, because the next reader believes it.
972
+ *
973
+ * The accessibility tree: no, and said so rather than answered with the iOS
974
+ * driver's name. `uiautomator dump` costs 2,012 ms a read, which is why it is
975
+ * not the answer; see docs/DEFERRED.md for the shape of the one that would be.
935
976
  */
936
977
  function capabilities() {
937
978
  return {
@@ -147,18 +147,30 @@ export async function bootedDevices(opts) {
147
147
  * device is the wrong-device bug wearing a different hat.
148
148
  *
149
149
  * Cross-platform ambiguity — one name matching a simulator and an emulator — is
150
- * reported rather than guessed at. Which of them a bare query should prefer is
151
- * a question for the step that adds the second backend, not one to invent an
152
- * answer to here.
150
+ * reported rather than guessed at, and so is a *bare* query on a host with a
151
+ * booted device on both platforms. That case is not exotic: it is the mixed
152
+ * setup the second backend exists for, and it used to break every command that
153
+ * did not name a device, with the word "undefined" standing in for the query.
154
+ *
155
+ * There is no defensible way to pick for you. Preferring iOS because it came
156
+ * first would silently tap a simulator while you were driving an emulator,
157
+ * which is the one failure this project will not ship. So a bare query on a
158
+ * mixed host says so and names both devices — and `SIMFRAME_DEVICE` exists so
159
+ * the answer can be given once per shell instead of on every command.
153
160
  *
154
161
  * @returns {Promise<Device>}
155
162
  */
156
163
  export async function resolveDevice(query, opts) {
157
- return resolveAcross(query, opts, backends());
164
+ return resolveAcross(query ?? process.env.SIMFRAME_DEVICE ?? undefined, opts, backends());
158
165
  }
159
166
 
160
167
  /** As above, over a given set of backends — the testable half. @returns {Promise<Device>} */
161
168
  export async function resolveAcross(query, opts, all) {
169
+ // A backend is entitled to assume a query is a string. `--device X` used to
170
+ // parse as `device: true`, which reached ios.js as `query.toLowerCase` and
171
+ // crashed with a TypeError where an unmatched device should have been a
172
+ // sentence. The parser no longer produces that, and this makes it unable to.
173
+ if (query != null && typeof query !== 'string') query = String(query);
162
174
  const hits = [];
163
175
  const misses = [];
164
176
  for (const backend of all) {
@@ -171,9 +183,12 @@ export async function resolveAcross(query, opts, all) {
171
183
  }
172
184
  if (hits.length === 1) return hits[0];
173
185
  if (hits.length > 1) {
186
+ const named = hits.map((d) => `${d.name} (${d.platform}, ${d.udid})`).join(', ');
174
187
  throw new Error(
175
- `"${query}" matches a device on more than one platform: ` +
176
- `${hits.map((d) => `${d.name} (${d.platform})`).join(', ')} — name one by its id`,
188
+ (query == null || query === ''
189
+ ? 'no device named, and there is a booted device on more than one platform'
190
+ : `"${query}" matches a device on more than one platform`) +
191
+ `: ${named} — name one by its id, or set SIMFRAME_DEVICE to pick a default`,
177
192
  );
178
193
  }
179
194
  // One backend: its own message, unchanged. It is the better message, because
@@ -66,6 +66,29 @@ async function resolveDevice(query, opts) {
66
66
  const booted = all.filter((d) => d.state === 'Booted');
67
67
  if (!query) {
68
68
  if (booted.length === 0) throw new Error('no booted simulator (open Simulator.app or run `xcrun simctl boot <udid>`)');
69
+ // `booted[0]` was the wrong-device bug, and it was worse than it looked.
70
+ // simctl's order is not "yours" by any definition, so on a machine with
71
+ // more than one booted simulator a bare `simframe ui` read whichever came
72
+ // first — and a bare `simframe tap` would have *injected input* into it. A
73
+ // reviewer reproduced it deterministically against a colleague's simulator.
74
+ //
75
+ // `doctor` got a guard for its own fan-out and this default did not, which
76
+ // is how the same command set could pick two different devices in one
77
+ // moment. Refusing is the only safe answer here: the seam cannot see which
78
+ // device simframe is already driving (that is store state, above the
79
+ // boundary), and a backend must never guess when the cost of guessing wrong
80
+ // is a tap on somebody else's screen. `ambiguous` so that a second platform
81
+ // matching cleanly cannot override this — see resolveAcross.
82
+ if (booted.length > 1) {
83
+ throw Object.assign(
84
+ new Error(
85
+ `${booted.length} simulators are booted and none was named: ` +
86
+ `${booted.map((d) => `${d.name} (${d.udid})`).join(', ')} — name one with --device, ` +
87
+ 'or set SIMFRAME_DEVICE to pick a default for this shell',
88
+ ),
89
+ { ambiguous: true },
90
+ );
91
+ }
69
92
  return booted[0];
70
93
  }
71
94
  const q = query.toLowerCase();