mobitru-mobile-cli 1.0.0-beta.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json ADDED
@@ -0,0 +1,36 @@
1
+ {
2
+ "name": "mobitru-mobile-cli",
3
+ "version": "1.0.0-beta.0",
4
+ "description": "Mobitru CLI (mobile) — command-line tool for AI agents to automate mobile testing on Mobitru cloud (Appium-backed)",
5
+ "keywords": [
6
+ "mobitru",
7
+ "cli",
8
+ "mobile",
9
+ "automation",
10
+ "testing",
11
+ "appium",
12
+ "device-farm",
13
+ "ai-agent"
14
+ ],
15
+ "engines": {
16
+ "node": ">=20"
17
+ },
18
+ "license": "ISC",
19
+ "author": "Mobitru",
20
+ "bin": {
21
+ "mobitru-mobile-cli": "./mobitru-mobile-cli.js"
22
+ },
23
+ "dependencies": {
24
+ "commander": "^14.0.3",
25
+ "fastify": "^5.8.4",
26
+ "webdriverio": "^9.0.0",
27
+ "webdriver": "^9.19.2"
28
+ },
29
+ "files": [
30
+ "*.js",
31
+ "skills/**",
32
+ "README.md",
33
+ "LICENSE",
34
+ "CHANGELOG.md"
35
+ ]
36
+ }
@@ -0,0 +1,650 @@
1
+ ---
2
+ name: mobitru-mobile-cli
3
+ description: Automate mobile testing on Mobitru cloud against real Android or iOS devices. Use when an agent needs to book a remote device (immediate or future), drive a native app or device browser via taps and accessibility snapshots, install and exercise APK/IPA artifacts, capture screen recordings or device logs, simulate network conditions (throttling, IP geolocation, GPS), capture HTTP traffic to HAR, or rotate the screen — all against a cloud-leased phone or tablet.
4
+ allowed-tools: Bash(mobitru-mobile-cli:*) Bash(npx:*) Bash(npm:*)
5
+ ---
6
+
7
+ # Mobile Automation with mobitru-mobile-cli
8
+
9
+ This CLI drives a **real device leased from Mobitru cloud**. Every
10
+ command operates on a remote phone or tablet — there is no local emulator.
11
+ See [references/mobile-sessions.md](references/mobile-sessions.md) for the
12
+ booking lifecycle, credentials setup, and cleanup window.
13
+
14
+ ## Quick start
15
+
16
+ ```bash
17
+ # Find an available device for the platform you want
18
+ mobitru-mobile-cli device-list android
19
+
20
+ # Book one by its serial (default lease is 180 minutes)
21
+ mobitru-mobile-cli device-use <DEVICE_SERIAL>
22
+
23
+ # Capture the screen — refs (e1..eN) come back for every interactable element
24
+ mobitru-mobile-cli snapshot
25
+
26
+ # Tap by ref, type into the focused field, submit with --submit
27
+ mobitru-mobile-cli tap e3
28
+ mobitru-mobile-cli type "mobitru.com" --submit
29
+
30
+ # Release the device when done (frees the cloud slot)
31
+ mobitru-mobile-cli device-release
32
+ ```
33
+
34
+ ## Commands
35
+
36
+ ### Device lifecycle
37
+
38
+ ```bash
39
+ mobitru-mobile-cli device-list android # platform: android or ios
40
+ mobitru-mobile-cli device-use <serial> # native session, 180 min lease
41
+ mobitru-mobile-cli device-use <serial> --duration=30 # custom lease in minutes
42
+ mobitru-mobile-cli device-use <serial> --web # browser (web) session
43
+ mobitru-mobile-cli device-release
44
+ ```
45
+
46
+ ### Bookings (future reservations)
47
+
48
+ **Distinct from `device-use`.** `device-use` takes an immediately available
49
+ device for the current session. **Bookings reserve a device for a future time
50
+ window** without starting any session. CRUD against the workspace, never the
51
+ active device.
52
+
53
+ ```bash
54
+ mobitru-mobile-cli bookings-list # all bookings (ISO dates)
55
+ mobitru-mobile-cli bookings-list --till=2026-06-01T00:00:00Z # filter: only those starting before <iso>
56
+ mobitru-mobile-cli bookings-info <booking-id> # full record
57
+
58
+ mobitru-mobile-cli booking-create \
59
+ --name="regression run" \
60
+ --start=2026-05-20T09:00:00Z \
61
+ --end=2026-05-20T10:00:00Z \
62
+ --devices=<DEVICE_SERIAL> # comma-separate for multi-device
63
+ mobitru-mobile-cli booking-create ... --private # mark booking as private
64
+
65
+ mobitru-mobile-cli booking-cancel <booking-id> # → { cancelled: true, id }
66
+ ```
67
+
68
+ > ⚠️ **`booking-create` response has empty `startISO`/`endISO`.** A Mobitru
69
+ > server quirk — the ISO fields populate on `bookings-list` / `bookings-info`
70
+ > but come back empty on the create response. Re-fetch via `bookings-info`
71
+ > if you need ISO timestamps right after creation. Epoch-seconds fields
72
+ > (`start`, `end`) are always populated.
73
+
74
+ ## Native vs Web sessions
75
+
76
+ Each booking is **either** a native session (default — drives the installed app)
77
+ **or** a web session (drives the device's browser — Chrome on Android, Safari on
78
+ iOS — with DOM-level operations on a web page).
79
+
80
+ Pick at booking time:
81
+
82
+ ```bash
83
+ mobitru-mobile-cli device-use <serial> # native (default)
84
+ mobitru-mobile-cli device-use <serial> --web # web
85
+ ```
86
+
87
+ Switching session type on the **same device** is fast (~5-10s) — the booking is
88
+ preserved, only the underlying session restarts. Switching to a **different
89
+ device** still requires `device-release` and incurs the 2-5min cloud cleanup
90
+ window.
91
+
92
+ ```bash
93
+ mobitru-mobile-cli device-use X # native on X
94
+ mobitru-mobile-cli device-use X --web # in-place switch to web on X — fast
95
+ mobitru-mobile-cli device-use X # in-place switch back to native — fast
96
+ mobitru-mobile-cli device-release && mobitru-mobile-cli device-use Y --web # different device — slow
97
+ ```
98
+
99
+ | Command | Native | Web |
100
+ |---|---|---|
101
+ | `screenshot`, `snapshot`, `tap`, `tap-at`, `type`, `swipe`, `continuous-swipe` | ✅ | ✅ (acts on browser viewport / DOM) |
102
+ | `screen-size`, `get-orientation`, `set-orientation` | ✅ | ✅ |
103
+ | `recording-*`, `logs-*`, `crashlogs`, `artifacts-*` | ✅ | ✅ (cloud-side, session-agnostic) |
104
+ | `geolocation-*`, `ip-location-*`, `throttling-*`, `http-inspection-*`, `har-download`, `profiler-*`, `inject-*` | ✅ | ✅ (cloud-side, session-agnostic) |
105
+ | `bookings-*`, `booking-*`, `espresso-*`, `xcuitest-*` | ✅ | ✅ (workspace-level, no device required) |
106
+ | `launch`, `terminate`, `is-installed`, `install-app`, `install-app-ota`, `uninstall-app` | ✅ | ❌ |
107
+ | `press` (HOME/BACK/etc.) | ✅ | ❌ |
108
+ | `open-url` | ❌ | ✅ |
109
+ | `click-web-element <selector>` | ❌ | ✅ |
110
+
111
+ > ⚠️ Calling a native-only command in a web session (or vice-versa) surfaces a
112
+ > clean error like `Cannot open URL in non-web session`. Switch session type
113
+ > instead of retrying.
114
+
115
+ Snapshot refs are session-type-specific (native AX tree ≠ web DOM tree).
116
+ Re-snapshot after switching. Recording and logs survive a switch unchanged —
117
+ the cloud captures the device's screen regardless of which session is active.
118
+
119
+ ### Observation
120
+
121
+ ```bash
122
+ # PNG screenshot saved to disk; stdout returns { path, bytes }
123
+ mobitru-mobile-cli screenshot
124
+ mobitru-mobile-cli screenshot --output=./screen.png
125
+
126
+ # Device screen dimensions in device pixels
127
+ mobitru-mobile-cli screen-size
128
+
129
+ # Accessibility tree of interactable elements; full tree saved to disk,
130
+ # stdout returns { path, bytes, refCount }. Refs are e1..eN.
131
+ mobitru-mobile-cli snapshot
132
+ mobitru-mobile-cli snapshot --output=./tree.json
133
+ ```
134
+
135
+ ### App management
136
+
137
+ ```bash
138
+ # Check whether an app is installed (use before launch to fail fast)
139
+ mobitru-mobile-cli is-installed com.android.chrome
140
+ # → { "appId": "com.android.chrome", "installed": true }
141
+
142
+ # Launch an app by package name (Android) or bundle ID (iOS)
143
+ mobitru-mobile-cli launch com.android.chrome
144
+
145
+ # Stop a running app (no-op if not running)
146
+ mobitru-mobile-cli terminate com.android.chrome
147
+ ```
148
+
149
+ ### Artifacts & install (bring your own app)
150
+
151
+ ```bash
152
+ # List artifacts uploaded to the workspace
153
+ mobitru-mobile-cli artifacts-list
154
+
155
+ # Inspect one (status, package name from manifest, target platform, …)
156
+ mobitru-mobile-cli artifacts-info c829dd49-...
157
+
158
+ # Upload an APK or IPA. Blocks until the cloud finishes processing.
159
+ # Target is auto-detected by extension (.apk → android, .ipa → ios).
160
+ mobitru-mobile-cli artifacts-upload ./app.apk
161
+ mobitru-mobile-cli artifacts-upload ./app.ipa --alias="login-flow-v2"
162
+
163
+ # Install onto the active device. Blocks until done.
164
+ mobitru-mobile-cli install-app c829dd49-...
165
+
166
+ # iOS only: install from an OTA manifest URL (skips the artifacts step entirely)
167
+ mobitru-mobile-cli install-app-ota https://example.com/app.plist
168
+
169
+ # Uninstall by package name / bundle ID
170
+ mobitru-mobile-cli uninstall-app com.epam.mobitru
171
+
172
+ # Download an artifact (logs zip, uploaded APK, etc.) by id
173
+ mobitru-mobile-cli artifacts-download 03e71031-c92e-4e87-8d8a-9fe4804a1d06
174
+ mobitru-mobile-cli artifacts-download <id> --output=./logs.zip
175
+ ```
176
+
177
+ ### Recording (screen video)
178
+
179
+ ```bash
180
+ # Start a screen recording. Returns max duration in seconds.
181
+ mobitru-mobile-cli recording-start
182
+ # → { "limitSeconds": 900 }
183
+
184
+ # Stop the recording. Returns recordingId for download.
185
+ mobitru-mobile-cli recording-stop
186
+ # → { "status": "recording has stopped", "recordingId": "6635051b-..." }
187
+
188
+ # Download the MP4. Blocks until the cloud finishes finalizing the file —
189
+ # no waiting/polling needed on your side. Default path is
190
+ # <CWD>/.mobitru-mobile-cli/recording-<ts>.mp4.
191
+ mobitru-mobile-cli recording-download 6635051b-...
192
+ mobitru-mobile-cli recording-download <id> --output=./demo.mp4
193
+ ```
194
+
195
+ > ⚠️ **Very short recordings may be discarded by the cloud.** If you call
196
+ > `recording-stop` within a second or two of `recording-start`, the
197
+ > `recordingId` may never become available and `recording-download` will
198
+ > error out after ~30s. Record long enough for visible activity.
199
+
200
+ > 💡 **Safety net: auto-stop on release.** If `device-release` or `stop`
201
+ > is called while a recording is still in flight, the daemon auto-stops
202
+ > it first and includes the resulting `recordingId` in the release
203
+ > response:
204
+ >
205
+ > ```json
206
+ > { "released": true, "serial": "...", "result": {...}, "recordingId": "..." }
207
+ > ```
208
+ >
209
+ > You can then `recording-download <recordingId>` before the daemon
210
+ > exits. Explicit `recording-stop` is still the preferred flow — it
211
+ > returns the `recordingId` immediately and lets you download before
212
+ > teardown. The auto-stop is a recovery for forgotten-stop cases. If the
213
+ > auto-stop itself fails (e.g., cloud-side state mismatch), the release
214
+ > response will lack `recordingId` and the recording is lost.
215
+
216
+ ### Logs & crashlogs
217
+
218
+ ```bash
219
+ # Start / stop device logs (logcat on Android, syslog on iOS).
220
+ # logs-stop returns the artifactId — pass it to artifacts-download.
221
+ mobitru-mobile-cli logs-start
222
+ mobitru-mobile-cli logs-stop
223
+ # → { "artifactId": "03e71031-..." }
224
+ mobitru-mobile-cli artifacts-download 03e71031-...
225
+
226
+ # One-shot dump of device crashlogs (zip of crash reports). Empty/small
227
+ # zip is normal if no crashes occurred.
228
+ mobitru-mobile-cli crashlogs
229
+ mobitru-mobile-cli crashlogs --output=./crash.zip
230
+ ```
231
+
232
+ ### Network simulation (throttling)
233
+
234
+ Constrain the booked device's bandwidth and latency to test how an app behaves
235
+ on slow networks. Effects persist until `throttling-disable` or device release.
236
+
237
+ ```bash
238
+ mobitru-mobile-cli throttling-presets # list named presets (3g, 4g, edge, …)
239
+ mobitru-mobile-cli throttling-status # current state
240
+ mobitru-mobile-cli throttling-enable 3g # apply a named preset
241
+ mobitru-mobile-cli throttling-enable custom \
242
+ --download 500 --upload 250 --latency 200 # kbps / kbps / ms
243
+ mobitru-mobile-cli throttling-disable
244
+ ```
245
+
246
+ > 💡 `throttling-presets` is global (no booked device required) — list options
247
+ > before booking if you want to plan ahead.
248
+
249
+ ### Location & IP geolocation
250
+
251
+ Two **independent** dimensions on the same device:
252
+
253
+ - **GPS** — the location reported to apps via the OS location service.
254
+ - **IP location** — the outbound network egress (proxy-based). Affects
255
+ IP-geolocation services (e.g. `https://ip-api.com/json`).
256
+
257
+ ```bash
258
+ # Device GPS (lat/lng, WGS84 — values clamped to [-90,90] / [-180,180])
259
+ mobitru-mobile-cli geolocation-get
260
+ mobitru-mobile-cli geolocation-set 48.8566 2.3522 # Paris
261
+
262
+ # Outbound IP location — list available country proxies, then set/reset
263
+ mobitru-mobile-cli ip-location-list # global, no device needed
264
+ mobitru-mobile-cli ip-location-set us # ISO-3166-1 alpha-2
265
+ mobitru-mobile-cli ip-location-set DEFAULT # reset to data-center IP
266
+ ```
267
+
268
+ > ⚠️ GPS lat/lng may round-trip with **minor float drift** (e.g. `48.8566` →
269
+ > `48.85660171508789`) because of the cloud's float storage precision.
270
+ > Harmless; don't treat it as a mismatch.
271
+
272
+ ### HTTP traffic capture — HAR (Android only)
273
+
274
+ Capture all device-originating HTTP traffic into a HAR file. Useful for
275
+ debugging network calls, verifying API contracts, or inspecting third-party
276
+ SDK chatter.
277
+
278
+ ```bash
279
+ mobitru-mobile-cli http-inspection-start # → { "harId": "..." }
280
+ # Exercise the app: launch, snapshot, tap, type — traffic accumulates server-side
281
+ mobitru-mobile-cli http-inspection-stop # → { "harId": "..." } (same id)
282
+ mobitru-mobile-cli har-download <har-id> # default: <CWD>/.mobitru-mobile-cli/har-<ts>.har
283
+ mobitru-mobile-cli har-download <har-id> --output=./traffic.har
284
+ ```
285
+
286
+ Flags on `http-inspection-start`:
287
+
288
+ ```bash
289
+ --capture-binary # include binary payloads (default: off)
290
+ --no-capture-response-content # skip textual response bodies (default: capture)
291
+ ```
292
+
293
+ > ⚠️ **Android only.** Calling `http-inspection-start` against an iOS device
294
+ > returns a clean cloud-side error. iOS HTTP capture is not exposed by
295
+ > Mobitru today.
296
+
297
+ > 💡 Stopping before any traffic occurred still produces a structurally
298
+ > valid (but empty) HAR (~300 bytes of wrapper metadata).
299
+
300
+ ### App profiler (CPU/memory)
301
+
302
+ Capture CPU and memory samples while exercising a running app. Output is a
303
+ JSON file with device, app, and sampling metadata.
304
+
305
+ ```bash
306
+ mobitru-mobile-cli launch com.android.chrome # the target MUST be running
307
+ mobitru-mobile-cli profiler-start com.android.chrome # → { appPackage, started: true }
308
+ # Exercise the app — taps, navigation, scrolling…
309
+ mobitru-mobile-cli profiler-stop # default: <CWD>/.mobitru-mobile-cli/profiler-<ts>.profile
310
+ mobitru-mobile-cli profiler-stop --output=./run.profile
311
+ ```
312
+
313
+ > ⚠️ `profiler-start` returns **400 if the target app isn't running**.
314
+ > Launch (or otherwise foreground) the app first, then start the profiler.
315
+
316
+ > 💡 Despite the `.profile` extension, the output is JSON — open with any
317
+ > text editor or pipe through `jq`.
318
+
319
+ ### Mock injection (camera image, biometric touch)
320
+
321
+ For testing flows that depend on camera input (KYC scans, document capture,
322
+ profile-photo upload) or biometric authentication (fingerprint, Touch ID),
323
+ inject the desired result directly instead of relying on a real sensor.
324
+
325
+ ```bash
326
+ # Camera mock — feeds an image as the next camera frame the app receives
327
+ mobitru-mobile-cli inject-image ./id-card.png
328
+ mobitru-mobile-cli inject-image photo.jpg --content-type=image/jpeg --name=passport.jpg
329
+
330
+ # Biometric mock — simulates a successful or failed fingerprint/Face ID
331
+ mobitru-mobile-cli inject-touch valid # successful auth
332
+ mobitru-mobile-cli inject-touch invalid # failed auth
333
+ ```
334
+
335
+ > ⚠️ **Requires app cooperation.** Both commands fail with **412 Precondition
336
+ > Failed** ("Check if application supports the injection feature") unless the
337
+ > target app was installed with `doInjection=true` AND is currently in the
338
+ > matching state — i.e. requesting a camera frame for `inject-image`, or
339
+ > awaiting biometric auth for `inject-touch`. Standard installs (via
340
+ > `install-app`) do not enable injection.
341
+
342
+ Inferred defaults for `inject-image`:
343
+
344
+ - **Content type** by extension: `.png` → `image/png`, `.jpg`/`.jpeg` → `image/jpeg`, `.gif` → `image/gif`, `.webp` → `image/webp`, anything else → `image/png`. Override with `--content-type`.
345
+ - **File name** = basename of the path. Override with `--name`.
346
+ - **Max 5 MB** per the cloud limit (returned as a clean error if exceeded).
347
+
348
+ ### Espresso / XCUITest test runs
349
+
350
+ Submit an instrumentation test run to the cloud and poll for status. Distinct
351
+ from interactive `device-use` — the cloud-side runner allocates the device,
352
+ installs both APKs/IPAs, and manages the run. No active `device-use` booking
353
+ is required.
354
+
355
+ ```bash
356
+ # Espresso (Android) — submit a run; cloud picks the device unless pinned
357
+ mobitru-mobile-cli espresso-run \
358
+ --app=<app-artifact-id> \
359
+ --test=<test-artifact-id> \
360
+ --runner=androidx.test.runner.AndroidJUnitRunner
361
+ mobitru-mobile-cli espresso-run ... --serial=<serial> # pin to a device
362
+ mobitru-mobile-cli espresso-run ... --filter="class:com.foo.LoginTests"
363
+ mobitru-mobile-cli espresso-run ... --shards=4 --duration=60
364
+
365
+ # XCUITest (iOS) — serial is required (cloud-side allocation isn't auto-pooled)
366
+ mobitru-mobile-cli xcuitest-run \
367
+ --serial=<serial> \
368
+ --app=<app-artifact-id> \
369
+ --test=<test-artifact-id>
370
+ mobitru-mobile-cli xcuitest-run ... --no-resign # skip Mobitru profile resign
371
+
372
+ # List, status, cancel — both engines share the same shape
373
+ mobitru-mobile-cli espresso-list
374
+ mobitru-mobile-cli espresso-status <run-id>
375
+ mobitru-mobile-cli espresso-cancel <run-id>
376
+
377
+ mobitru-mobile-cli xcuitest-list
378
+ mobitru-mobile-cli xcuitest-status <run-id>
379
+ mobitru-mobile-cli xcuitest-cancel <run-id>
380
+ ```
381
+
382
+ > 💡 **Test runs are asynchronous.** `*-run` returns immediately with a `pending`
383
+ > status. Poll `*-status <id>` until the status transitions to `completed`,
384
+ > `failed`, or `canceled`. Final artifacts (logs, reports) appear in the
385
+ > `artifacts` field of the status response — use `artifacts-download` to fetch
386
+ > them by id.
387
+
388
+ > ⚠️ **Espresso supports `--serial` / `--device-name` / `--platform-version` as
389
+ > optional;** XCUITest requires `--serial`. The asymmetry comes from how
390
+ > Mobitru allocates iOS vs Android — the iOS pool doesn't auto-select.
391
+
392
+ > 💡 **Full bring-your-own-app loop:**
393
+ > `artifacts-upload app.apk` → grab `id` from output → `install-app <id>` →
394
+ > `launch <package>` → exercise the app via Tier 1 commands →
395
+ > `uninstall-app <package>` (optional) → `device-release`.
396
+ > Both upload and install **block** until the cloud confirms success — no
397
+ > polling logic needed on your side.
398
+
399
+ > 💡 **Prefer `launch` over tapping a home-screen icon** when the goal is "open
400
+ > app X". `launch` works regardless of which screen the device is on, doesn't
401
+ > depend on the icon being visible (or not buried in an app drawer), and
402
+ > won't tap the wrong same-labeled icon. Use snapshot+tap only when you
403
+ > need to interact with something already on-screen.
404
+
405
+ ### Appium scripts (custom WebDriver tests)
406
+
407
+ Submit a Node-style Appium test script (mocha bodies — `it(...)`, `describe(...)`) to run against the booked device. The script runs in the background on the daemon; you can keep doing other work and poll for status.
408
+
409
+ ```bash
410
+ mobitru-mobile-cli device-use <serial> # booking required first
411
+ mobitru-mobile-cli appium-run path/to/test.js # submit; returns immediately
412
+ mobitru-mobile-cli appium-run path/to/test.js --timeout 600000 # wall-clock cap (ms)
413
+ mobitru-mobile-cli appium-run path/to/test.js \
414
+ --capabilities '{"appium:noReset": true}' # extra/override caps
415
+
416
+ mobitru-mobile-cli appium-status # state of current/last job
417
+ mobitru-mobile-cli appium-logs # combined stdout+stderr
418
+ mobitru-mobile-cli appium-cancel # SIGTERM the child
419
+ ```
420
+
421
+ **Script shape.** Write mocha-style `it(...)` / `describe(...)` bodies. A `driver` global is in scope — a WebDriverIO session bound to the booked device. Don't set `platformName`, `udid`, or `automationName` capabilities yourself; they're set for you. Pass extra/override caps via `--capabilities`.
422
+
423
+ ```js
424
+ it("logs in", async () => {
425
+ await driver.$("~loginButton").click();
426
+ const text = await driver.$("~welcomeLabel").getText();
427
+ if (text !== "Welcome") { throw new Error("got: " + text); }
428
+ });
429
+ ```
430
+
431
+ **Statuses** (terminal): `succeeded`, `failed`, `cancelled`, `timed-out`. While running: `status === "running"` with a live `durationMs`. `appium-logs` returns the current/last run's combined stdout+stderr.
432
+
433
+ > ⚠️ **Single-job-at-a-time.** While an `appium-run` job is in flight, a second `appium-run` is rejected, interactive verbs (`tap`, `swipe`, `screenshot`, `snapshot`, `type`, `press`, orientation, web) are rejected, and `device-use` / `device-release` are rejected. Cloud-side verbs (`artifacts-*`, `throttling-*`, `recording-*`, `crashlogs`, etc.) are still allowed. Wait for terminal status or call `appium-cancel`.
434
+
435
+ > 💡 **If the CLI dies mid-run**, the orphan mocha self-exits within ~5s, but the cloud booking still holds the device until its lease expires. Pass a tight `--duration` to `device-use` so the worst-case lock window is bounded.
436
+
437
+ ### Interaction
438
+
439
+ ```bash
440
+ # Tap on an element by ref (taps the center of the element's rect)
441
+ mobitru-mobile-cli tap e3
442
+
443
+ # Tap at raw pixel coordinates — for elements invisible to snapshot
444
+ # (most commonly: empty Material text fields). See "Empty Material fields"
445
+ # below for the gap-inference pattern that produces the coords.
446
+ mobitru-mobile-cli tap-at 540 601
447
+
448
+ # Type into the currently focused input. Tap the input first to give it focus.
449
+ mobitru-mobile-cli type "user@example.com"
450
+ mobitru-mobile-cli type "query" --submit # presses ENTER after
451
+
452
+ # Hardware buttons — case-insensitive
453
+ mobitru-mobile-cli press HOME
454
+ mobitru-mobile-cli press BACK
455
+ mobitru-mobile-cli press VOLUME_UP
456
+ mobitru-mobile-cli press VOLUME_DOWN
457
+ mobitru-mobile-cli press ENTER
458
+ mobitru-mobile-cli press DPAD_CENTER # plus DPAD_UP/DOWN/LEFT/RIGHT
459
+
460
+ # Directional swipe; default force is 0.5 (half-screen distance)
461
+ mobitru-mobile-cli swipe up
462
+ mobitru-mobile-cli swipe down --force=0.8
463
+
464
+ # Multi-point drag (2..20 coordinate pairs). Useful for gestures the
465
+ # direction-based swipe can't express — diagonals, signatures, paths.
466
+ mobitru-mobile-cli continuous-swipe 100,400 100,1200 # straight drag
467
+ mobitru-mobile-cli continuous-swipe 100,400 800,400 100,1200 # L-shape
468
+ ```
469
+
470
+ > 💡 **`swipe` vs `continuous-swipe` — pick by scope.**
471
+ > - `swipe <direction>` is a **full-screen** gesture. Use it for
472
+ > dismissing notification shades, pulling drawer menus, system-level
473
+ > interactions where you want the gesture to reach the OS.
474
+ > - `continuous-swipe x,y x,y` runs **between exact coordinates**. Use
475
+ > it for **all in-app scrolling** (catalog lists, modals, inner
476
+ > regions). Read the visible content rect from `snapshot` and drag
477
+ > between two points well inside it.
478
+ >
479
+ > ⚠️ **`swipe up` is unsafe inside an app on Android 11+.** The OS's
480
+ > gesture-navigation system reserves the bottom edge for "swipe up =
481
+ > home", and the default `swipe up` originates in the lower third of
482
+ > the screen — close enough that the OS grabs the gesture and exits
483
+ > your app. For scrolling the current screen's content downward, use
484
+ > `continuous-swipe` with both points inside the app's content area
485
+ > (e.g. `continuous-swipe 540,1600 540,800` — starts well above the
486
+ > gesture zone). `swipe down` from the top is safe (no system gesture
487
+ > there). `swipe left/right` is also generally safe.
488
+
489
+ ### Orientation
490
+
491
+ ```bash
492
+ mobitru-mobile-cli get-orientation
493
+ mobitru-mobile-cli set-orientation portrait
494
+ mobitru-mobile-cli set-orientation landscape
495
+ ```
496
+
497
+ ## Snapshots and refs
498
+
499
+ `snapshot` returns a flat list of interactable elements with rects, labels,
500
+ and text. Each element gets a ref like `e1`, `e2`, `e3` — numbered in the
501
+ order they appear on screen. Pass that ref to `tap`.
502
+
503
+ ```bash
504
+ mobitru-mobile-cli snapshot
505
+ # → { "path": ".../snapshot-1778515305673.json", "bytes": 12480, "refCount": 32 }
506
+
507
+ # Read the file to find what you want, e.g. an omnibox:
508
+ # {
509
+ # "ref": "e3",
510
+ # "type": "android.widget.EditText",
511
+ # "label": "Search Google or type URL",
512
+ # "rect": { "x": 60, "y": 220, "width": 920, "height": 56 }
513
+ # }
514
+
515
+ mobitru-mobile-cli tap e3
516
+ ```
517
+
518
+ Refs are valid only until the next `snapshot` (which renumbers from scratch).
519
+ Re-snapshot after any action that changes the screen — opening a menu,
520
+ navigating, scrolling — before tapping again.
521
+
522
+ > 💡 The mobile accessibility tree is **not** the DOM. Labels come from
523
+ > Android's `contentDescription` / iOS's `accessibilityLabel`. Elements
524
+ > without semantic information may not appear. If you can't find what you
525
+ > need by `snapshot`, fall back to `screenshot` + visual reasoning.
526
+
527
+ ### Empty input fields and the snapshot filter (Android-only)
528
+
529
+ **Android.** `snapshot` only includes elements that carry at least one of:
530
+ `text`, `content-desc`, or `hint`. An empty `EditText` with no
531
+ `contentDescription` and no `android:hint` (common with Material
532
+ `TextInputEditText`, where the floating label is rendered as a separate
533
+ `TextView`) carries none of those and is filtered out — even when focused.
534
+
535
+ **iOS.** Empty `XCUIElementTypeTextField` / `TextView` widgets are reported
536
+ in the snapshot regardless of content. `tap eN` works directly; the
537
+ gap-inference pattern below is not needed.
538
+
539
+ Consequence for the agent: empty input fields are not addressable by ref.
540
+ You won't find an `EditText` entry to feed to `tap eN`. Use **`tap-at`
541
+ with coordinates inferred from neighboring anchors** instead, then type.
542
+ Once the field has content, the next snapshot includes it with `focused:
543
+ true` and the typed text.
544
+
545
+ Inferring coordinates — three cases, in decreasing reliability:
546
+
547
+ | Layout | Coordinate strategy |
548
+ |---|---|
549
+ | Label above + another visible anchor below the input (next label, button, divider) | Tap at the midpoint of the gap between the two anchors |
550
+ | Label above, nothing visible below | Tap at `(label_x_center, label_y_end + 60)` — an offset that empirically lands inside the input |
551
+ | No label nearby | Take a `screenshot`, identify the field visually, pick coords by sight |
552
+
553
+ Example for the gap case:
554
+
555
+ ```bash
556
+ mobitru-mobile-cli snapshot
557
+ # Labels: First name at y=448..501, Last name at y=702..755
558
+ # Empty First Name input lives in the gap y ∈ [501, 702]
559
+ # Midpoint y = 601, horizontal center of the form x = 540
560
+ mobitru-mobile-cli tap-at 540 601
561
+ mobitru-mobile-cli type "Alice"
562
+ mobitru-mobile-cli snapshot
563
+ # First Name now appears: { ref: "eN", text: "Alice", focused: true, rect: ... }
564
+ ```
565
+
566
+ > ⚠️ Don't rely on the snapshot's `focused` field to verify focus *before*
567
+ > typing into an empty field — the field isn't in the snapshot to report
568
+ > focus. Verify by either: (a) the next `type` succeeding (it errors with
569
+ > `no such element` when no input is focused), or (b) inspecting a
570
+ > `screenshot` (caret visible, soft keyboard open).
571
+
572
+ ## The agent loop
573
+
574
+ ```
575
+ snapshot → identify the target by label/text/type → tap ref → action → snapshot
576
+ ```
577
+
578
+ After every screen-changing action (`tap`, `tap-at`, `type`, `swipe`,
579
+ `continuous-swipe`, `press`, `set-orientation`, `launch`, `terminate`),
580
+ call `snapshot` again before the next ref-based command.
581
+
582
+ ## Common errors
583
+
584
+ | Error | Cause | Recovery |
585
+ |---|---|---|
586
+ | `no active device — run device-use first` | No device booked in this daemon | `device-use <serial>` |
587
+ | `Failed to send keys: ... no such element` | `type` called with nothing focused | Tap a text input first, then retry |
588
+ | `ref "eN" not found. Run snapshot first.` | Stale ref after the screen changed | Re-`snapshot`, get the new ref |
589
+ | `device "<serial>" is currently in use. Run device-release first.` | Trying to switch devices without releasing | `device-release` then `device-use <new-serial>` |
590
+ | `Failed to take device: 404` on `device-use` | Device is in post-release cleanup | Wait 30-60s and retry; or pick a different serial |
591
+ | `another command is in flight; CLI calls must be sequential` | An earlier command is still running | Wait — most commands finish in seconds; `install-app` ~20-30s; `artifacts-upload` scales with file size. If it persists past a minute, run `mobitru-mobile-cli stop` to force-reset (kills any in-flight upload/download). |
592
+
593
+ ## Artifact output
594
+
595
+ Screenshots, snapshots, and other generated files land under
596
+ `<CWD>/.mobitru-mobile-cli/` by default, where `<CWD>` is the directory the
597
+ command runs in. Override per-command with `--output=<path>`. The override
598
+ is resolved relative to the current shell CWD, not the artifacts dir — pass
599
+ an absolute path to be unambiguous.
600
+
601
+ Run `mobitru-mobile-cli config` to see the resolved `artifactDir`.
602
+ Set `MOBILE_CLI_OUTPUT_DIR=<dir>` to override the default location across
603
+ all commands.
604
+
605
+ > ⚠️ The artifact dir is resolved **per-invocation** from the shell's CWD,
606
+ > not stamped once at booking. If you `cd` between commands, downloads
607
+ > land relative to the new CWD. Stay in one directory for the duration of
608
+ > a session, or pass `--output=<absolute-path>` for downloads.
609
+
610
+ ## Session state
611
+
612
+ ```bash
613
+ mobitru-mobile-cli status # show current state — { running, activeDevice, sessionType, recordingActive, currentJob }
614
+ mobitru-mobile-cli stop # release the device and reset session state
615
+ ```
616
+
617
+ > ⚠️ **Always `stop` (or `device-release`) when done.** A leaked session
618
+ > holds the cloud booking until the device lease expires.
619
+
620
+ ## Installation
621
+
622
+ ```bash
623
+ # Install globally
624
+ npm install -g mobitru-mobile-cli
625
+
626
+ # Or use without installing
627
+ npx mobitru-mobile-cli --version
628
+ ```
629
+
630
+ First-time credentials setup is required before `device-use` works — see
631
+ [references/mobile-sessions.md](references/mobile-sessions.md).
632
+
633
+ ## Example: Web search via Chrome
634
+
635
+ ```bash
636
+ mobitru-mobile-cli device-use <DEVICE_SERIAL>
637
+ mobitru-mobile-cli launch com.android.chrome
638
+ mobitru-mobile-cli snapshot
639
+ # Find the omnibox (android.widget.EditText with label "Search Google or type URL")
640
+
641
+ mobitru-mobile-cli tap e3
642
+ mobitru-mobile-cli type "mobitru.com" --submit
643
+ mobitru-mobile-cli screenshot
644
+
645
+ mobitru-mobile-cli device-release
646
+ ```
647
+
648
+ ## Specific tasks
649
+
650
+ * **Booking lifecycle, credentials, cleanup window** — [references/mobile-sessions.md](references/mobile-sessions.md)