device-devtools-mcp 0.1.0

package/README.md

@@ -0,0 +1,491 @@
+# DeviceTools for agents
+
+DeviceTools for agents (`device-devtools-mcp`) lets your coding agent (such as
+Claude, Codex, Cursor or Copilot) control and inspect a real iOS or Android app
+running on a phone plugged into your Mac. A CLI is also provided for use without
+MCP.
+
+Chrome DevTools MCP gives an agent eyes and hands on a web page. This is the
+same thing for a native app, on hardware you can hold — and for a lot of teams
+the phone is the only place the app builds at all.
+
+## Key features
+
+- **Reading, not looking**: `snapshot` returns every element that can be acted
+  on or carries text, with its rectangle. About a third the cost of a
+  screenshot, and unlike a screenshot it can be measured.
+- **Layout faults, computed**: overlap, escaping a parent, off screen, hit areas
+  below the platform minimum, a control whose centre lands under the status bar,
+  text the OS truncated. Every number is a subtraction between two rectangles.
+- **Testability, linted**: a control with no accessibility identifier is named,
+  because the only selector left for it is its label — and a label changes when
+  the copy is rewritten or the app is read in another language.
+- **Refusal over guessing**: a selector matching two controls, or a uid that no
+  longer names what it named, stops the action instead of tapping something.
+- **Installing a build**: `app install` puts a bundle on the device this config
+  selects and no other, and says why iOS would refuse it before iOS does.
+- **Several steps, one call**: `run` takes a list — tap, type, tap, wait — and
+  `flow` saves that list in your repository under a name. Logging in stops
+  being twelve calls and becomes one word.
+
+## Disclaimers
+
+DeviceTools exposes the screen and the logs of the app under test to the MCP
+client, allowing it to inspect and modify anything the app shows. Screenshots
+are sent to your model provider unmasked.
+
+Control comes from XCUITest (iOS) and UiAutomator2 (Android). Nothing is added
+to your app: no debug server, no `#if DEBUG` hook, no test-only endpoint, and no
+jailbreak. There is no `evaluate_script` — a native app is not a scripting host,
+so state that is not in the accessibility tree is not observable from here.
+
+Android is written and unit-checked but has never been run against a physical
+device. iOS is proven on an iPhone 13 Pro Max.
+
+## Usage statistics
+
+None are collected. DeviceTools makes no network request of its own; everything
+it talks to is on this machine or on the cable.
+
+## Update checks
+
+None are automatic. `devicetools update` fetches, reports how many commits
+behind you are, and refuses to pull over uncommitted changes.
+
+## Requirements
+
+- macOS for iOS; macOS or Linux for Android
+- `jq` and `curl`
+- iOS: Xcode, an Apple Developer account, `libimobiledevice`, `ios-deploy`
+- Android: `adb`
+- A device attached over USB, unlocked, with the app already installed
+- Node 18+ **only if you install with `npx`** — it starts the server and does
+  nothing else. The tool itself is bash: no runtime dependency, no npm packages,
+  and the same bytes run from a clone with no Node at all.
+
+## Getting started
+
+Add this to your MCP client:
+
+```json
+{
+  "mcpServers": {
+    "devicetools": {
+      "command": "npx",
+      "args": ["-y", "device-devtools-mcp@latest"],
+      "env": {
+        "DEVICETOOLS_PROJECT": "/Users/you/work/my-app"
+      }
+    }
+  }
+}
+```
+
+> [!NOTE]
+> `device-devtools-mcp@latest` keeps your client on the newest version. While
+> the repository is private and the package is unpublished, use
+> `["-y", "github:hung977/device-devtools-mcp"]` instead — it works the same
+> way for anyone with access to the repository.
+> `DEVICETOOLS_PROJECT` is the repository holding the app under test: its
+> `config.json` is the config, and flows you save land in its `flows/`
+> directory. An MCP server has no working directory, so it cannot find that for
+> itself.
+
+Then, once per app:
+
+```bash
+npx device-devtools-mcp@latest init ~/work/my-app   # writes config.json there
+$EDITOR ~/work/my-app/config.json               # the udid, the bundle id, your team
+npx device-devtools-mcp@latest doctor               # says what is still missing
+```
+
+The same binary is the server (no arguments) and the command line (any verb), so
+there is nothing else to install.
+
+### iOS needs a driver built once
+
+`doctor` will tell you if it is not there. This is the one step `npx` cannot do
+for you — XCUITest has to be signed with your Apple team, on your machine:
+
+```bash
+npx device-devtools-mcp@latest setup
+```
+
+It clones WebDriverAgent, rewrites its bundle identifiers, signs and builds it.
+Four things only a person can do, and it prints whichever are missing: sign in
+to Xcode, register the device UDID with your team, trust the developer
+certificate on the phone, and turn on **Settings → Developer → Enable UI
+Automation**.
+
+Android needs `adb` and nothing else.
+
+### From a clone instead
+
+```bash
+git clone https://github.com/hung977/device-devtools-mcp ~/tools/devicetools
+cd ~/tools/devicetools
+./install.sh --ios --app com.yourcompany.app        # or --android
+```
+
+`install.sh` installs missing dependencies with Homebrew, puts `devicetools` on
+your PATH, detects the attached device, writes `config.json`, builds the driver,
+and runs `doctor`. Running it again is safe: an existing `config.json` is filled
+in, never overwritten. More than one phone attached, or more than one Apple team
+on the keychain, and it lists them and asks.
+
+### MCP client configuration
+
+<details>
+  <summary>Claude Code</summary>
+
+```bash
+claude mcp add devicetools --scope user \
+  --env DEVICETOOLS_PROJECT=/Users/you/work/my-app \
+  -- npx -y device-devtools-mcp@latest
+```
+
+</details>
+
+<details>
+  <summary>Codex</summary>
+
+In `~/.codex/config.toml`:
+
+```toml
+[mcp_servers.devicetools]
+command = "npx"
+args = ["-y", "device-devtools-mcp@latest"]
+env = { DEVICETOOLS_PROJECT = "/Users/you/work/my-app" }
+```
+
+</details>
+
+<details>
+  <summary>Cursor, Windsurf, Copilot, and anything else that reads mcpServers</summary>
+
+Use the JSON at the top of this section.
+
+</details>
+
+<details>
+  <summary>From a clone, without npx</summary>
+
+```json
+{
+  "mcpServers": {
+    "devicetools": {
+      "command": "/Users/you/tools/devicetools/integrations/mcp/server.sh",
+      "env": {
+        "DEVICETOOLS_CONFIG": "/Users/you/tools/devicetools/config.json"
+      }
+    }
+  }
+}
+```
+
+</details>
+
+### Without MCP
+
+An agent that can run a shell needs no server. Drop a pointer to `AGENTS.md`
+into your repository:
+
+```bash
+integrations/agent-pointer.sh ~/work/my-app claude   # cursor | gemini | codex | mcp
+```
+
+### Your first prompt
+
+> Open the transfer screen and tell me if anything is laid out wrong.
+
+## Tool reference
+
+See [Tool reference](docs/tool-reference.md), generated from the schemas the
+server sends. The contract an agent reads is [`AGENTS.md`](AGENTS.md).
+
+## Tools
+
+- **Reading the screen** (3 tools)
+  - [`snapshot`](docs/tool-reference.md#snapshot)
+  - [`screenshot`](docs/tool-reference.md#screenshot)
+  - [`measure`](docs/tool-reference.md#measure)
+- **Input automation** (7 tools)
+  - [`tap`](docs/tool-reference.md#tap)
+  - [`type`](docs/tool-reference.md#type)
+  - [`swipe`](docs/tool-reference.md#swipe)
+  - [`scroll`](docs/tool-reference.md#scroll)
+  - [`back`](docs/tool-reference.md#back)
+  - [`key`](docs/tool-reference.md#key)
+  - [`wait`](docs/tool-reference.md#wait)
+- **Batches and flows** (2 tools)
+  - [`run`](docs/tool-reference.md#run)
+  - [`flow`](docs/tool-reference.md#flow)
+- **The app** (2 tools)
+  - [`app`](docs/tool-reference.md#app)
+  - [`open`](docs/tool-reference.md#open)
+- **The device** (5 tools)
+  - [`doctor`](docs/tool-reference.md#doctor)
+  - [`devices`](docs/tool-reference.md#devices)
+  - [`logs`](docs/tool-reference.md#logs)
+  - [`settings`](docs/tool-reference.md#settings)
+  - [`permission`](docs/tool-reference.md#permission)
+- **The project** (1 tool)
+  - [`init`](docs/tool-reference.md#init)
+- **Getting back** (1 tool)
+  - [`waypoint`](docs/tool-reference.md#waypoint)
+
+The CLI is the same program with positional arguments: `devicetools snapshot`,
+`devicetools tap 19`, `devicetools logs start`. It also carries `find`, `tree`
+and `controls`, which are not exposed over MCP.
+
+## Configuration
+
+Everything lives in `config.json` beside the checkout. Edit it without opening
+it:
+
+```bash
+devicetools config                                  # print it
+devicetools config get app.bundle_id
+devicetools config set app.bundle_id com.other.app
+```
+
+- **`platform`**
+  Which adapter runs. The device, `wda` and `ua2` blocks belonging to the other
+  platform are ignored, not wrong.
+  - **Type:** string, `ios` or `android`
+- **`device.udid` / `device.serial`**
+  The phone. `install.sh` fills this in from what is attached.
+  - **Type:** string
+- **`device.kind`**
+  - **Type:** string, one of `auto`, `device`, `simulator`, `emulator`
+  - **Default:** `auto`
+- **`app.bundle_id` / `app.package`**
+  The app under test. Every verb acts on this one unless told otherwise.
+  - **Type:** string
+- **`wda.team_id`**
+  The Apple team that signs WebDriverAgent. Not the app's team, and not the ten
+  characters in parentheses in `security find-identity` — it is the certificate's
+  `OU`.
+  - **Type:** string
+- **`timeouts.wait_seconds`**
+  How long `wait` waits when not told.
+  - **Type:** number
+  - **Default:** `15`
+- **`logs.network_pattern`**
+  What counts as a request-shaped log line.
+  - **Type:** string, an extended regular expression
+- **`scroll.max_swipes`**
+  The ceiling on `scroll --until`.
+  - **Type:** number
+  - **Default:** `15`
+- **`paths.flows_dir`**
+  Where `flow save` writes. Relative paths resolve against the project, not the
+  checkout, because a flow belongs to the app it describes.
+  - **Type:** string
+  - **Default:** `flows`
+- **`vars`**
+  Values for the `{{ name }}` placeholders in a flow. Anything secret is better
+  passed as `DEVICETOOLS_VAR_<NAME>` in the environment — `config.json` is a
+  file, and a file gets copied.
+  - **Type:** object
+
+A key that does not exist is treated as a typo: `config set` lists the real keys
+and asks for `--new`. The type comes from the value already there, so a number
+stays a number. Point the server at a different file with `DEVICETOOLS_CONFIG`;
+that is also how you drive two apps, one config and one server each.
+
+## Concepts
+
+### uids die at the next snapshot
+
+The number in brackets is valid until the next `snapshot` and no longer. Acting
+on one re-reads the screen and requires the element to still be exactly what was
+numbered — same type, identifier, label, value and rectangle, give or take two
+points. If it is not, the action is refused and says what changed. Nothing is
+scored, and there is no runner-up.
+
+Every action takes `--snapshot`, which prints the screen it left behind and
+renumbers the uids in the same call. That is the cheapest thing in here: half
+the calls in a measured session were snapshots taken only to see what the
+previous action had produced.
+
+### Two points is nothing
+
+Both drivers report rectangles as floating point and this pipeline floors them,
+so any pair of edges carries up to two points of error. Differences at or below
+that are not reported. On a real screen this took sixty-three warnings down to
+eight, and the eight were true.
+
+### The screen hash, and what it does not cover
+
+`SCREEN "Auth.LoginView" #6360` names the screen by a short hash over the type
+and label of everything shown. It answers "am I still on the same screen", so a
+field filling up hashes the same. A second digest covers values, which is why an
+action can report `SCREEN same — #6360, contents changed`.
+
+### Selectors match whole
+
+`id:`, `label:`, `text:`, `kind:` and a plain string all match a value **whole**.
+`text:OTP` does not find a label reading "Vui lòng nhập mã OTP…". `snapshot
+--grep OTP` does, because reading may guess and acting may not — a substring
+match is how a tap lands on the caption instead of the button.
+
+`kind:<Type>` names the element's class, the word `snapshot` prints first on the
+line. It is the escape hatch for the control `snapshot` warns about: no
+identifier, and no label either. Usually written with `--index`:
+
+```bash
+devicetools tap kind:SecureTextField --index 0
+```
+
+The alternative is a raw coordinate, and coordinates break — on this app, once
+the keyboard went down the sheet holding that field moved from y=507 to y=774.
+
+### Batches, and flows
+
+An agent that taps, reads, taps, reads spends most of its calls reading, and
+every call re-sends the whole conversation. `run` collapses that:
+
+```bash
+devicetools run --steps '[
+  {"tap":  "label:Đăng nhập"},
+  {"type": "{{ password }}"},
+  {"key":  "hide"},
+  {"tap":  "label:Xác nhận"},
+  {"wait": "screen:Home"}
+]' --var password="$PASSWORD"
+```
+
+A step is an object with exactly one verb in it; any other key is that verb's
+flag. It stops at the first failing step and prints the screen it stopped on —
+fail-fast with the scene, because a batch that says only "step 3 failed" has
+traded a legible error for a black box.
+
+uids are refused inside a batch. A uid names a position on a screen the caller
+was shown before the batch began, and by step three that screen is two actions
+old. Selectors resolve against what is actually there.
+
+`flow` gives a batch a name and puts it in `flows/<name>.json` in the project —
+run `devicetools init` in your app's repository first, or it lands in the
+DeviceTools checkout, which every app on the machine shares. `flows/` is
+deliberately not gitignored: it is the reason to have a project at all.
+
+```bash
+devicetools flow save login --steps '[...]'
+devicetools flow run login --var password="$PASSWORD"
+devicetools flow list
+```
+
+Credentials stay as `{{ name }}` and are filled at run time from `--var`, from
+`DEVICETOOLS_VAR_<NAME>`, or from `.vars` in the config — and the substituted
+value is masked back to `{{ name }}` in everything a run prints. The variables a
+flow needs are read out of its own steps when it is saved, including those of
+any flow it includes.
+
+A step of `{"flow": "login"}` runs a saved flow inside another, so `activate` is
+`login` plus the OTP rather than a second copy of logging in.
+
+`{"expect": "screen:Auth.LoginView"}` is how a flow says where it expects to
+start — a `wait` with a three-second deadline, so it fails at the door rather
+than tapping into a screen it was not written for. On a real app "log in" has
+three different routes depending on whether the device is activated.
+
+`{"snapshot": true}` is a flag on any step, not a step of its own:
+`{"tap": "label:X", "snapshot": true}` acts and hands back the new screen.
+
+Logs are attached only to the step that **failed**. In a batch of eight, the
+system's own log lines outweighed everything the batch said; `--logs` keeps
+them all.
+
+There is no `flow record`. Watching somebody tap produces a list of coordinates,
+which is the one spelling guaranteed to break when the layout moves.
+
+### Logs have to be started
+
+`logs action=start`, once per session. An action must not spawn a background
+collector as a side effect of touching a button, and the device under test
+produced twenty-one thousand log lines in thirty seconds.
+
+Only the app's own lines are kept. The OS logs a great deal on an app's behalf
+under its process name — hit testing, the accelerometer, lifecycle — and on a
+real 40,775-line capture every one of the 162 lines carrying the app's process
+name came from a system framework, while the app's own logging was zero lines.
+`logs.exclude_subsystems` is that list; `logs --raw` shows everything.
+
+`logs --network` filters the app's own output. It is not a proxy — a proxy needs
+a CA certificate on the device and an app that pins defeats one — so an app that
+does not print its traffic shows nothing.
+
+### Installing a build
+
+DeviceTools does not build. It installs what your build produced, onto the
+device this config selects and no other:
+
+```bash
+devicetools app install ~/DerivedData/Build/Products/Debug-iphoneos/My.app --launch
+```
+
+```
+OK installed com.example.my
+   version  3.4.1 (1207)
+   signed   signed, team ABCDE12345
+   onto     00008030-…  (Work iPhone)
+```
+
+A bundle built with `CODE_SIGNING_ALLOWED=NO` is refused before it reaches the
+phone, with that named as the likely cause. iOS calls it
+`ApplicationVerificationFailed`, which sends people to read profiles instead.
+
+`doctor` does not look at certificates or provisioning profiles. Whose account
+signs the app is the build's business; a check standing outside the build can
+only guess, and on a machine with fifteen teams on the keychain it guessed
+wrong. The signature `app install` reads comes from the bundle in front of it.
+
+### The two directories, and how MCP names one
+
+`DT_HOME` is the checkout — scripts, adapters, `AGENTS.md`. `DT_ROOT` is where
+`config.json`, `flows/`, `.state/` and `.runs/` live, which by default is the
+checkout too. `devicetools init <dir>` marks a different directory with a
+`.devicetools` file, and every relative path in a config resolves against it.
+
+From a shell, the project is found by walking up from where you are standing,
+the way git does. **An MCP server has no such thing**: the host sets its working
+directory when it spawns the process and it never changes, so every walk-up
+lands in the checkout no matter which repository you are working in. So every
+verb takes `--project <dir>` — `project` in the arguments over MCP — and that
+names the project for that call:
+
+```bash
+devicetools flow save login --project ~/work/my-app --steps '[...]'
+```
+
+Its `config.json` becomes the config for that call too, ahead of anything in the
+environment: a flag on this call is more specific than a variable set when the
+server started.
+
+## Known limitations
+
+| You see | Do this |
+| --- | --- |
+| `Timed out while enabling automation mode` | Settings → Developer → Enable UI Automation, unlock the screen |
+| `the runner is not signed for this device` | Register the UDID with your team, then `devicetools setup --force` |
+| `N teams on this machine` (no terminal) | `./install.sh --ios --team <ID>` |
+| `N iPhones attached` (no terminal) | `./install.sh --ios --udid <UDID>` |
+| `Unable to find a destination matching` | Reconnect the cable, unlock, trust this computer |
+| `bundle is unsigned` | Rebuild with signing on; a Makefile probably sets `CODE_SIGNING_ALLOWED=NO` |
+| exit 3 mid-session | `devicetools doctor --recover` |
+| `uid 7 is stale — …` | `snapshot` again |
+| `logs` returns nothing | `logs start` was never called |
+
+Exit codes are uniform: `0` done, `1` usage, `2` environment or configuration,
+`3` device unreachable, `4` the command ran and the screen did not satisfy it.
+
+## Uninstalling
+
+```bash
+./uninstall.sh              # symlink, MCP registration, .state/, background processes
+./uninstall.sh --all        # also the driver on the phone, its build, and config.json
+./uninstall.sh --dry-run    # preview
+```
+
+Homebrew packages and the checkout itself are left alone.

package/VERSION

@@ -0,0 +1 @@
+0.1.0

package/bin/device-devtools-mcp.js

@@ -0,0 +1,106 @@
+#!/usr/bin/env node
+// device-devtools-mcp — the npx entry point.
+//
+//   npx device-devtools-mcp@latest              the MCP server, on stdin/stdout
+//   npx device-devtools-mcp@latest doctor       any CLI verb, same as `devicetools`
+//
+// WHY THERE IS JAVASCRIPT IN A PROJECT THAT SAYS "NO NODE.JS".
+//
+// There still is none in the tool. Every verb is bash, every adapter is bash,
+// and the MCP server is bash; this file starts one of them and then does
+// nothing. It exists because `npx <name>@latest` is how an MCP server gets
+// installed in practice — the host configuration people paste is `npx`, and a
+// `git clone` plus a `./install.sh` is a different conversation with whoever is
+// setting it up. Node is the delivery mechanism, not a dependency of the work.
+//
+// It has no dependencies of its own, deliberately: a supply chain is a strange
+// thing to accept in return for one spawn call.
+//
+// STDIO IS INHERITED, NOT PIPED. The MCP protocol is JSON-RPC on stdin and
+// stdout, one message per line. Anything this file printed to stdout would be
+// read by the host as a malformed message, so it prints nothing there ever —
+// diagnostics go to stderr, which the host shows in its log.
+//
+// bash IS INVOKED EXPLICITLY rather than executing the script by its own
+// shebang. npm preserves the executable bit in a tarball, but a package
+// extracted by something else may not, and "permission denied" from inside an
+// npx cache is a confusing place to start debugging.
+
+'use strict';
+
+const { spawn, spawnSync } = require('node:child_process');
+const { join } = require('node:path');
+const { existsSync } = require('node:fs');
+
+const root = join(__dirname, '..');
+const args = process.argv.slice(2);
+
+// No arguments means the server, which is what a host's `npx` line sends.
+// Anything else is the command line, so that `npx device-devtools-mcp doctor` works
+// without a second package.
+const server = args.length === 0;
+const target = server
+  ? join(root, 'integrations', 'mcp', 'server.sh')
+  : join(root, 'bin', 'devicetools');
+
+if (!existsSync(target)) {
+  process.stderr.write(
+    `device-devtools-mcp: ${target} is missing — this package did not unpack completely\n`);
+  process.exit(2);
+}
+
+// THE ONE THING npm BREAKS THAT NOTHING ELSE HERE COULD SURVIVE.
+//
+// Every verb in scripts/ is a symlink to dispatch.sh, and every adapter's copy
+// of a shared verb is a symlink into scripts/common/ whose own directory is
+// what selects the adapter. `npm pack` carries none of them — measured: 63
+// files in the tarball instead of 110, and zero symlinks, not dereferenced but
+// simply absent. Installed like that the server answers `unknown tool` to all
+// twenty-one of them while looking perfectly fine.
+//
+// So they are put back here, from a manifest generated out of the repository.
+// Not in a postinstall script: `npm install --ignore-scripts` is common and
+// some organisations enforce it, and a tool that is silently broken under a
+// security setting is worse than one that is loudly broken. This runs from the
+// entry point, where nothing can switch it off, and on a git checkout it finds
+// nothing to do and says nothing.
+if (!existsSync(join(root, 'scripts', 'dispatch.sh'))) {
+  process.stderr.write(
+    'device-devtools-mcp: scripts/dispatch.sh is missing — this package did not unpack completely\n');
+  process.exit(2);
+}
+if (!existsSync(join(root, 'scripts', 'tap.sh'))) {
+  const relink = spawnSync('/bin/bash', [join(root, 'scripts', 'relink.sh')],
+                           { stdio: ['ignore', 'pipe', 'inherit'] });
+  if (relink.status !== 0) {
+    process.stderr.write('device-devtools-mcp: the verb surface could not be restored\n');
+    process.exit(2);
+  }
+  // stdout is the JSON-RPC channel in server mode, so this goes to the log.
+  process.stderr.write(`device-devtools-mcp: ${String(relink.stdout).trim()}\n`);
+}
+
+const child = spawn('/bin/bash', [target, ...args], {
+  stdio: 'inherit',
+  env: { ...process.env, DEVICETOOLS_VIA_NPX: '1' },
+});
+
+// Passed on rather than swallowed: the host stops an MCP server by signalling
+// it, and a wrapper that ignored that would leave the real server running with
+// a device port held open.
+for (const sig of ['SIGINT', 'SIGTERM', 'SIGHUP']) {
+  process.on(sig, () => { try { child.kill(sig); } catch { /* already gone */ } });
+}
+
+child.on('error', (err) => {
+  process.stderr.write(`device-devtools-mcp: could not start bash — ${err.message}\n`);
+  process.exit(2);
+});
+
+// The child's exit code is the contract every verb here documents: 0 done,
+// 1 usage, 2 environment, 3 device unreachable, 4 the screen did not satisfy.
+// Reporting anything else would break the one thing a caller can rely on.
+child.on('exit', (code, signal) => {
+  if (signal) { process.kill(process.pid, signal); return; }
+  process.exit(code === null ? 1 : code);
+});