simtap 0.0.0-stage → 0.6.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 the simtap authors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,187 @@
-# Temporary Holding Version
+# simtap
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Give your agent a phone. simtap lets Claude Code, Codex, Cursor and any other MCP client read and drive the iOS Simulator on your Mac, with a small set of tools and a compact view of the screen.
+
+It builds, runs and tests Xcode projects, reads and uses the screen of any app (SwiftUI, UIKit, React Native), answers iOS's permission prompts, reads the app's logs and crashes, and records video. Tested on Xcode 26.5 and 27 with iOS 26.5 and 27, on iPhone and iPad.
+
+## What the agent sees
+
+```
+Demo · Sign In · iPhone 17 Pro (iOS 26.5)
+textfield "Email" #email @201,194
+securefield "Password" #password @201,246
+switch "Remember me" #remember off @341,298
+button "Sign In" #signIn @201,350
+text "Not signed in" #status @201,402
+tabs: Login (selected) @158,822, Items @244,822
+```
+
+- One line per element: kind, label, `#id` (the app's accessibility identifier), state and `@x,y`, the center in points.
+- Groups, icons inside buttons, scroll bars, the status bar and the keyboard's keys are folded away. The tab bar, scrolling and the keyboard are one line each.
+- Only what can be tapped is listed. An alert, a sheet, a menu or a full-screen view in front hides the screen behind it, and iOS's own permission prompts read as alerts over the app. Rows under the navigation bar, the tab bar or the keyboard are counted as hidden (above, below, to the left or right), not listed.
+- Web views (Capacitor, Expo DOM components), maps, games and iOS's out-of-process pickers have nothing for accessibility to read. They come back as one `unreadable` line with their place, so the agent knows to take a screenshot and tap by point.
+- A screenshot, when asked for, is sized to points, so a point in the picture is a point to tap.
+- The sign-in screen above is about 70 tokens. The full accessibility tree it comes from is about 40,000.
+
+## Using the screen
+
+`ui_act` takes steps in the same words `ui_look` prints, runs them in order, and answers with what changed:
+
+```
+ui_act steps: ["fill #email gary@example.com", "fill #password secret1", "tap #signIn", "wait \"Welcome\""]
+
+4 steps done in 4.5 s
+changed: #email "gary@example.com" · #password "•••••••" · #status "Welcome, gary@example.com"
+```
+
+Steps:
+
+| Step | Does |
+|---|---|
+| `tap #id`, `tap "label"`, `tap button "label"`, `tap @x,y` | Taps; a switch gets a real touch |
+| `fill #id text` | Focuses the field, clears it, types, and reads the text back |
+| `type text` | Types into the field that has focus |
+| `scroll down`, `scroll right @201,402` | Scrolls the screen, or one carousel |
+| `swipe left`, `swipe left "Row 3"` | Swipes the screen, or one row (to reveal Delete) |
+| `hold "Row 5" [seconds]` | Long press, for context menus |
+| `set #volume 75` | Sets a slider to a percent |
+| `back`, `home`, `key return` | The back button (or edge swipe), the home button, a key |
+| `wait "Welcome" [seconds]`, `gone "Loading"` | Waits for something to appear or go |
+
+Every step is checked, so nothing succeeds silently:
+- Typed text is read back from the field. When iOS's autocorrect or smart punctuation changes it (`--` to `—`, `'` to `’`), the text is pasted instead, which iOS takes as is. A field that still shows something else stops the call and says what it shows.
+- Text in any language (Café, 日本語, emoji) can be typed: what a US keyboard lacks goes through the Simulator's clipboard.
+- `fill` refuses anything that isn't a text field, so keys never land in whichever field still has focus.
+- A tap that changes nothing is reported, after waiting 2.5 s for late answers such as a permission prompt. A tap inside a web view says it can't be checked.
+- A target that isn't on screen yet gets two seconds to appear, then fails with the next step: "2 elements are below: scroll down", or, when an alert came up in front, "Answer it first: tap "Cancel" or "Open"".
+- The first failing step stops the rest and returns the screen as it is now.
+- A new screen comes back whole; otherwise only what changed.
+
+## Tools
+
+| Tool | CLI | Does |
+|---|---|---|
+| `app_run` | `simtap run` | Builds the Xcode project here, installs and launches it, answers with the first screen; errors as `file:line` |
+| `devices` | `simtap devices` | Lists the Simulators on this Mac with their runtime, UDID and state |
+| `device_use` | `simtap use "iPhone 17 Pro"` | Chooses the Simulator the other tools use, booting it (one boot at a time); dark mode, a clean status bar |
+| `app_control` | `simtap app launch dev.example.app` | `install`, `launch` (answers with the first screen), `terminate`, `uninstall`, `open_url`; `grant`, `revoke` and `reset_permissions`; `location`; `push` |
+| `ui_look` | `simtap look` | Reads the screen; `find` keeps matching lines, `screenshot` adds a picture, `full` saves the whole tree |
+| `ui_act` | `simtap act "tap #signIn" ...` | Does steps in order and answers with what changed |
+| `app_logs` | `simtap logs` | The app's print output and its own system log lines; `grep`, `wait_for`, `crashes` |
+| `tests_run` | `simtap test` | Runs the unit and UI tests; counts, and each failure as `file:line`; "0 tests ran" is an error |
+| `screen_record` | `simtap record start` | Starts or stops a video of the screen; stop answers with the MP4 |
+| `job_status` | `simtap job` | Waits up to 40 s for a long build and answers with its result, or its progress |
+| `job_cancel` | `simtap cancel` | Stops a job |
+
+The MCP server and the CLI are built from one list, so they always match. The tool list and instructions stay under 4,000 tokens; a test enforces it.
+
+## Build and run
+
+`app_run` does the whole loop in one call: it finds the project (a workspace first, then a project, an XcodeGen spec, or `ios/`), picks the scheme, builds for the Simulator, installs, launches, and answers with the first screen.
+
+```
+Built Demo in 4.2 s, installed dev.simtap.demo.
+Launched Demo in 3.9 s.
+Demo · Sign In · iPhone 17 Pro (iOS 26.5)
+textfield "Email" #email @201,194
+...
+```
+
+- A failed build answers with the errors only, as `Sources/App.swift:49:37: error: ...`, and the path to the full log.
+- Each folder (each git worktree) builds into its own DerivedData, so rebuilds are incremental and worktrees never share products.
+- An XcodeGen project is regenerated from `project.yml` before each build (skipped when nothing changed), so new files are always in it.
+- The next call in the same folder needs no arguments.
+- An MCP call answers within 40 s; a longer build answers with a job, and `job_status` waits for it. From the CLI, `simtap run` simply waits.
+- Expo, React Native, Flutter and Capacitor builds are not done yet; build them with their own tools, then `app_control install`.
+
+## Tests
+
+`tests_run` runs the scheme's tests and reads the answer from Xcode's result bundle, not from the scrolling output:
+
+```
+Tests failed: 1 passed, 2 failed (3 in 35.3 s). Result bundle: .../tests-1792370825323.xcresult
+DemoTests/SignInTests/testRejectsAShortPassword(): Tests/SignInTests.swift:12: XCTAssertEqual failed: ("Welcome, gary@example.com") is not equal to ("Invalid email or password")
+DemoTests/rejectsAnEmailWithoutAt(): Tests/SignInTests.swift:18: Expectation failed: ...
+```
+
+- XCTest and Swift Testing both.
+- Each failing test is named the way `only` takes it, to rerun just that one.
+- "0 tests ran" (no test target, or a filter that matched nothing) is an error, never a pass.
+
+## Logs and crashes
+
+`app_logs` shows what the app itself said, not the thousands of lines Apple's frameworks log from inside it:
+- Its `print()` output, kept from the moment `app_control` launches it.
+- Its own system log lines (its subsystem, or logged by its own code), streamed from launch. macOS keeps info lines only in memory and drops them within seconds when the Simulator is busy, so reading the log store afterwards misses them.
+- `grep` also searches Apple's lines, for things like a missing image or a layout warning.
+- `wait_for` waits up to 25 seconds for a line, for example after a tap that starts a network call.
+- `crashes` lists crash reports with the crashing thread down to your code (`ItemsView.body.getter (App.swift:64)`), next to the app's last output, which holds Swift's own message (`App.swift:64: Fatal error: ...`). macOS can take up to a minute to write a report.
+
+## Several agents, one Mac
+
+- Each agent runs its own MCP server and holds its own Simulator through `device_use`.
+- Boots never overlap, across every simtap process on the Mac: two Simulators booting at once on an 8 GB Mac pushed it into swap for minutes.
+- `SIMTAP_MAX_BOOTED` caps how many run at once; asking for another shuts down the others first.
+- Calls on one Simulator run one at a time, so two agents' steps never interleave.
+
+## What it can't do yet
+
+- Read inside web views, Flutter or game canvases: it says so, and a screenshot plus tap by point works.
+- Real devices, Android, rotating to landscape, pinch and multi-touch.
+- Reading text an app hides from accessibility: some React Native apps set a test id as the label, and the read says when labels look like that.
+
+## Install
+
+Needs macOS with Xcode and a Simulator runtime, and Node 20 or newer.
+
+```sh
+# Claude Code
+claude mcp add simtap -- npx -y simtap mcp
+
+# Codex
+codex mcp add simtap -- npx -y simtap mcp
+```
+
+Cursor, Windsurf, VS Code and other clients that take a JSON config:
+
+```json
+{
+  "mcpServers": {
+    "simtap": { "command": "npx", "args": ["-y", "simtap", "mcp"] }
+  }
+}
+```
+
+Any other client: run `npx -y simtap mcp` as a stdio server. It speaks MCP 2024-11-05 through 2026-07-28.
+
+Agents that work from a shell can use the CLI instead; `skill/SKILL.md` in the package teaches it in under 2 KB.
+
+From a shell: `npx simtap look`.
+
+## Privacy
+
+simtap sends nothing anywhere. There is no telemetry and no network access at runtime: it runs `xcrun simctl`, the bundled AXe binary and `sips`, all on your Mac.
+
+## How it works
+
+simtap reads the Simulator through [AXe](https://github.com/cameroncooke/AXe) (MIT), bundled at a pinned version with its checksum checked at packaging, and through `xcrun simctl`. There is no test runner to build, so the first call is as fast as the rest.
+
+## Develop
+
+```sh
+npm install
+npm run axe        # fetch the pinned AXe into vendor/axe
+npm test           # node --test, no build needed
+npm run build      # dist/ for the npm package
+node bin/simtap.js look
+```
+
+`examples/swiftui-demo` is the sample app the tests' screens come from. `scripts/golden.sh` builds it, runs it on a Simulator and checks 19 golden tasks: build and launch in one call, sign in, toggle, open an item, back and scroll, the app's logs, a crash, a build error and a failing test as `file:line`, a video, and a wrong password that must fail loudly.
+
+`examples/edge-lab` is an app with every awkward screen: segments, steppers, sliders, menus, alerts, sheets, full-screen covers, permission prompts, text in any language, autocorrect, late content, toasts, swipe-to-delete, long press, a web view, UIKit and a deep link. `scripts/lab.sh` builds it and runs 26 checks on it.
+
+`SIMTAP_DEBUG=1` prints every step and every program run with its time.
+
+## License
+
+MIT. AXe and the IDB frameworks it includes are MIT too; see `THIRD_PARTY_LICENSES`.

package/THIRD_PARTY_LICENSES

@@ -0,0 +1,61 @@
+simtap ships a copy of AXe (vendor/axe), which includes frameworks from Facebook IDB.
+
+---------------------------------------------------------------------
+
+AXe 1.8.0
+https://github.com/cameroncooke/AXe
+
+MIT License
+
+Copyright (c) 2025 Cameron Cooke
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+---------------------------------------------------------------------
+
+AXe's own third-party notice:
+
+This project depends on third-party open source software.
+
+---------------------------------------------------------------------
+
+Facebook IDB
+https://github.com/facebook/idb
+
+Copyright (c) Meta Platforms, Inc. and affiliates.
+
+Licensed under the MIT License:
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/bin/simtap.js

@@ -0,0 +1,4 @@
+#!/usr/bin/env node
+import { main } from "../dist/cli.js";
+
+process.exitCode = await main(process.argv.slice(2));

package/dist/act/diff.js

@@ -0,0 +1,67 @@
+import { line, quote, ref, render } from "../look/snapshot.js";
+/**
+ * Pairs each element before with itself after: by id, then by kind and label, then a field by its place
+ * (a field with no label is named by its text, which is what changed). Ids can appear late: right after a
+ * launch the tab buttons had none, a second later they did.
+ */
+export function pair(before, after) {
+    const left = new Set(before);
+    const right = new Set(after);
+    const pairs = [];
+    const match = (same) => {
+        for (const b of right) {
+            const a = [...left].find((x) => same(x, b));
+            if (!a)
+                continue;
+            pairs.push([a, b]);
+            left.delete(a);
+            right.delete(b);
+        }
+    };
+    match((a, b) => Boolean(a.id) && a.id === b.id && a.kind === b.kind);
+    match((a, b) => a.kind === b.kind && a.label !== undefined && a.label === b.label);
+    match((a, b) => a.kind === b.kind && a.frame.x === b.frame.x && a.frame.y === b.frame.y && a.frame.width === b.frame.width);
+    return { pairs, added: [...right], gone: [...left] };
+}
+/** Short form for a changed element: `#status "Welcome, gary@example.com"`. */
+function brief(e) {
+    let out = e.id && e.id !== e.label ? ref(e.id) : e.kind;
+    if (e.label !== undefined)
+        out += ` ${quote(e.label)}`;
+    if (e.value)
+        out += ` =${quote(e.value)}`;
+    if (e.state)
+        out += ` ${e.state}`;
+    return out;
+}
+/**
+ * What an action changed, in as few tokens as possible. A new screen (another title or app, or most lines
+ * different) comes back whole; otherwise only the changed, new and gone elements, and the keyboard if it moved.
+ */
+export function diff(before, after, device) {
+    const all = (s) => [...s.elements, ...s.tabs];
+    const { pairs, added, gone } = pair(all(before), all(after));
+    // What the agent cares about is what an element says; an id showing up late is not a change.
+    const says = (e) => `${e.kind}|${e.label ?? ""}|${e.value ?? ""}|${e.state ?? ""}`;
+    const edited = pairs.filter(([a, b]) => says(a) !== says(b)).map(([, b]) => b);
+    // The same element somewhere else: the content moved (a scroll, a banner pushing rows down).
+    const moved = pairs.filter(([a, b]) => says(a) === says(b) && (a.x !== b.x || a.y !== b.y)).map(([, b]) => b);
+    const size = all(after).length;
+    const newScreen = before.app !== after.app || before.title !== after.title || added.length + gone.length > Math.max(4, size / 2) || moved.length > 3;
+    if (newScreen)
+        return `screen:\n${render(after, { device })}`;
+    const out = [];
+    if (edited.length)
+        out.push(`changed: ${edited.map(brief).join(" · ")}`);
+    if (added.length)
+        out.push(`new: ${added.map(line).join(" · ")}`);
+    if (gone.length)
+        out.push(`gone: ${gone.map(brief).join(" · ")}`);
+    if (moved.length)
+        out.push(`moved: ${moved.map(line).join(" · ")}`);
+    if (before.scroll !== after.scroll && after.scroll)
+        out.push(`scroll: ${after.scroll}`);
+    if (before.keyboard !== after.keyboard)
+        out.push(after.keyboard ? `keyboard: shown, covering y ${after.keyboardTop ?? "?"} and below` : "keyboard: hidden");
+    return out.length ? out.join("\n") : "no visible change";
+}

package/dist/act/resolve.js

@@ -0,0 +1,49 @@
+import { line } from "../look/snapshot.js";
+import { describeTarget } from "./steps.js";
+/** Kinds a person taps, preferred over plain text with the same label. */
+const TAPPABLE = new Set(["button", "back", "link", "cell", "tab", "switch", "checkbox", "radio", "textfield", "securefield", "searchfield", "textview", "popup", "menu", "segmented", "picker", "slider", "stepper"]);
+/**
+ * Finds a target on the screen: by #id exactly; by label exactly (any case), then by a unique partial match;
+ * or a point as given. Ties go to tappable elements; a tie that remains is an error listing the candidates.
+ */
+export function resolve(snap, target) {
+    if (target.by === "point")
+        return { x: Math.round(target.x), y: Math.round(target.y) };
+    const all = [...snap.elements, ...snap.tabs];
+    let found;
+    if (target.by === "id") {
+        found = all.filter((e) => e.id === target.id);
+    }
+    else {
+        const want = target.label.toLowerCase();
+        const ofKind = target.kind ? all.filter((e) => e.kind === target.kind) : all;
+        found = ofKind.filter((e) => e.label?.toLowerCase() === want);
+        if (!found.length)
+            found = ofKind.filter((e) => e.label?.toLowerCase().includes(want) || e.value?.toLowerCase() === want);
+    }
+    if (found.length > 1) {
+        const tappable = found.filter((e) => TAPPABLE.has(e.kind));
+        if (tappable.length)
+            found = tappable;
+    }
+    // One id on several elements (every row of a list): the same point twice is one element, else it's ambiguous.
+    if (target.by === "id" && found.length > 1 && found.every((e) => e.x === found[0].x && e.y === found[0].y))
+        found = [found[0]];
+    if (found.length === 1) {
+        const element = found[0];
+        return { element, x: element.x, y: element.y };
+    }
+    if (found.length > 1) {
+        return {
+            error: `${found.length} elements match ${describeTarget(target)}: ${found.slice(0, 5).map(line).join(" · ")}`,
+            next: "Use a #id, a quoted exact label, kind plus label (button \"Save\"), or @x,y.",
+            ambiguous: true,
+        };
+    }
+    const hidden = snap.hidden.below + snap.hidden.above;
+    return {
+        error: `No ${describeTarget(target)} on screen.`,
+        next: hidden ? `${snap.hidden.below ? `${snap.hidden.below} elements are below: scroll down` : `${snap.hidden.above} elements are above: scroll up`}, or call ui_look.` : "Call ui_look to see what is on screen.",
+    };
+}
+export const isResolved = (r) => !("error" in r);

package/dist/act/steps.js

@@ -0,0 +1,188 @@
+/**
+ * The steps ui_act takes, one short line each, in the same words ui_look prints:
+ *
+ *   tap #signIn            tap "Sign In"           tap button "Sign In"     tap @201,350
+ *   fill #email gary@example.com                   type hello
+ *   scroll down            swipe left              swipe left "Row 3"       scroll right @201,402
+ *   hold "Row 3"           hold #photo 2           set #volume 75           back        home
+ *   wait "Welcome"         wait #status 10         gone "Loading"           key return
+ */
+/** HID keycodes AXe sends. */
+export const KEYS = { return: 40, escape: 41, delete: 42, tab: 43, space: 44 };
+export const DEFAULT_WAIT_SECONDS = 5;
+export const MAX_WAIT_SECONDS = 30;
+export const DEFAULT_HOLD_SECONDS = 1;
+export const MAX_HOLD_SECONDS = 10;
+const DIRECTIONS = new Set(["up", "down", "left", "right"]);
+/** Reads a target from the start of `rest`. Returns it and what follows, or an error message. */
+export function parseTarget(rest) {
+    const s = rest.trimStart();
+    const point = /^@(-?\d+(?:\.\d+)?),\s*(-?\d+(?:\.\d+)?)(?=\s|$)/.exec(s);
+    if (point)
+        return { target: { by: "point", x: Number(point[1]), y: Number(point[2]) }, after: s.slice(point[0].length) };
+    if (s.startsWith("#")) {
+        if (s[1] === '"') {
+            const quoted = readQuoted(s.slice(1));
+            if (typeof quoted === "string")
+                return quoted;
+            return { target: { by: "id", id: quoted.value }, after: quoted.after };
+        }
+        const id = /^#(\S+)/.exec(s)?.[1];
+        if (!id)
+            return "Expected an id after #.";
+        return { target: { by: "id", id }, after: s.slice(id.length + 1) };
+    }
+    // `button "Sign In"`: a kind, then a quoted label.
+    const kinded = /^([a-z]+)\s+(?=")/.exec(s);
+    const kind = kinded?.[1];
+    const labelPart = kinded ? s.slice(kinded[0].length) : s;
+    if (labelPart.startsWith('"')) {
+        const quoted = readQuoted(labelPart);
+        if (typeof quoted === "string")
+            return quoted;
+        return { target: { by: "label", label: quoted.value, ...(kind ? { kind } : {}) }, after: quoted.after };
+    }
+    const word = /^\S+/.exec(s)?.[0];
+    if (!word)
+        return "Expected a target: #id, \"label\" or @x,y.";
+    return { target: { by: "label", label: word }, after: s.slice(word.length) };
+}
+function readQuoted(s) {
+    // A JSON string, so \" and \n work as they do in ui_look's output.
+    for (let end = 1; end < s.length; end++) {
+        if (s[end] === "\\") {
+            end++;
+            continue;
+        }
+        if (s[end] === '"') {
+            try {
+                return { value: JSON.parse(s.slice(0, end + 1)), after: s.slice(end + 1) };
+            }
+            catch {
+                return `Bad quoted text: ${s.slice(0, end + 1)}`;
+            }
+        }
+    }
+    return `Missing closing quote: ${s}`;
+}
+/** Text to type: the rest of the line, or a quoted string to keep spaces at its ends. */
+function readText(rest) {
+    const s = rest.replace(/^\s/, "");
+    if (s.startsWith('"') && s.endsWith('"') && s.length >= 2) {
+        try {
+            return JSON.parse(s);
+        }
+        catch {
+            return s;
+        }
+    }
+    return s;
+}
+function seconds(rest, fallback = DEFAULT_WAIT_SECONDS, max = MAX_WAIT_SECONDS) {
+    const s = rest.trim();
+    if (!s)
+        return fallback;
+    const n = Number(s.replace(/s$/, ""));
+    if (!Number.isFinite(n) || n <= 0)
+        return `Expected seconds, got "${s}".`;
+    return Math.min(n, max);
+}
+/** A target, then an optional number: `wait Welcome back 10`, `hold "Row 3" 2`. An unquoted label may have spaces. */
+function targetThenNumber(rest) {
+    const bare = /^(?![#"@])(.+?)(?:\s+(\d+(?:\.\d+)?)(?:s|%)?)?$/.exec(rest.trim());
+    if (bare?.[1])
+        return { target: { by: "label", label: bare[1] }, number: bare[2] ?? "" };
+    const t = parseTarget(rest);
+    if (typeof t === "string")
+        return t;
+    return { target: t.target, number: t.after.trim().replace(/(s|%)$/, "") };
+}
+export function parseStep(line) {
+    const trimmed = line.trim();
+    const [, verb = "", rest = ""] = /^(\S+)\s*([\s\S]*)$/.exec(trimmed) ?? [];
+    switch (verb.toLowerCase()) {
+        case "tap": {
+            const t = parseTarget(rest);
+            if (typeof t === "string")
+                return t;
+            if (t.after.trim() && t.target.by === "label" && !rest.trimStart().startsWith('"'))
+                return { op: "tap", target: { by: "label", label: rest.trim() } };
+            if (t.after.trim())
+                return `Unexpected "${t.after.trim()}" after the target.`;
+            return { op: "tap", target: t.target };
+        }
+        case "fill": {
+            const t = parseTarget(rest);
+            if (typeof t === "string")
+                return t;
+            if (!t.after.trim())
+                return "fill needs a target and the text: fill #email gary@example.com";
+            return { op: "fill", target: t.target, text: readText(t.after) };
+        }
+        case "type":
+            if (!rest)
+                return "type needs the text: type hello";
+            return { op: "type", text: readText(` ${rest}`) };
+        case "scroll":
+        case "swipe": {
+            // `swipe left` sweeps the screen; `swipe left "Row 3"` sweeps that row (to delete it, for example).
+            const [, word = "", on = ""] = /^(\S*)\s*([\s\S]*)$/.exec(rest.trim()) ?? [];
+            const direction = word.toLowerCase();
+            if (!DIRECTIONS.has(direction))
+                return `${verb} needs a direction: up, down, left or right.`;
+            const op = verb.toLowerCase();
+            if (!on.trim())
+                return { op, direction: direction };
+            const t = parseTarget(on.trim().startsWith('"') || on.trim().startsWith("#") || on.trim().startsWith("@") ? on : `"${on.trim()}"`);
+            if (typeof t === "string")
+                return t;
+            return { op, direction: direction, target: t.target };
+        }
+        case "wait":
+        case "gone":
+        case "hold": {
+            const t = targetThenNumber(rest);
+            if (typeof t === "string")
+                return t;
+            if (verb.toLowerCase() === "hold") {
+                const s = seconds(t.number, DEFAULT_HOLD_SECONDS, MAX_HOLD_SECONDS);
+                if (typeof s === "string")
+                    return s;
+                return { op: "hold", target: t.target, seconds: s };
+            }
+            const s = seconds(t.number);
+            if (typeof s === "string")
+                return s;
+            return { op: verb.toLowerCase(), target: t.target, seconds: s };
+        }
+        case "set": {
+            const t = targetThenNumber(rest);
+            if (typeof t === "string")
+                return t;
+            const percent = Number(t.number);
+            if (t.number === "" || !Number.isFinite(percent) || percent < 0 || percent > 100)
+                return "set takes a slider and a percent from 0 to 100: set #volume 75";
+            return { op: "set", target: t.target, percent };
+        }
+        case "key": {
+            const key = rest.trim().toLowerCase();
+            if (!(key in KEYS))
+                return `key takes one of: ${Object.keys(KEYS).join(", ")}.`;
+            return { op: "key", key: key };
+        }
+        case "back":
+        case "home":
+            if (rest.trim())
+                return `${verb} takes nothing after it.`;
+            return { op: verb.toLowerCase() };
+        default:
+            return `Unknown step "${verb}". Steps: tap, fill, type, scroll, swipe, hold, set, wait, gone, key, back, home.`;
+    }
+}
+export function describeTarget(t) {
+    if (t.by === "id")
+        return `#${t.id}`;
+    if (t.by === "point")
+        return `@${t.x},${t.y}`;
+    return `${t.kind ? `${t.kind} ` : ""}${JSON.stringify(t.label)}`;
+}