sim-mirror 2.0.1__tar.gz → 2.2.0__tar.gz

{sim_mirror-2.0.1 → sim_mirror-2.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: sim-mirror
-Version: 2.0.1
+Version: 2.2.0
 Summary: Mirror and drive iOS Simulators and real iPhones from Claude Code, CLI agents and the browser.
 Project-URL: Homepage, https://github.com/AndrewKochulab/sim-mirror
 Project-URL: Documentation, https://github.com/AndrewKochulab/sim-mirror/tree/main/docs
@@ -39,7 +39,7 @@ Description-Content-Type: text/markdown
 
 # SimMirror
 
-**Mirror and drive iOS Simulators and real iPhones from Claude Code, CLI agents and the browser.**
+**Mirror and drive iOS Simulators, real iPhones and Android devices from Claude Code, CLI agents and the browser.**
 
 [![CI](https://github.com/AndrewKochulab/sim-mirror/actions/workflows/ci.yml/badge.svg)](https://github.com/AndrewKochulab/sim-mirror/actions/workflows/ci.yml)
 [![License: Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
@@ -52,15 +52,18 @@ through its notes, then fills in and saves a new one, the agent's cursor gliding
 *A real session, sped up four times. [The whole recording](https://github.com/AndrewKochulab/sim-mirror/releases/download/v1.2.0/sim-mirror-plugin-demo.mp4)
 runs 1m 39s.*
 
-A live iOS Simulator -- or your own iPhone, over its cable -- in any browser tab or web page, an animated cursor that
+A live iOS Simulator -- or your own iPhone, over its cable, or an Android emulator or phone -- in any browser tab or
+web page, an animated cursor that
 shows exactly what your AI agent is about to tap, and token-efficient UI snapshots so agents read the screen as compact
 text instead of screenshots. Record any screen as an MP4 or a GIF for a pull request. Works with Claude Code, Codex,
 Cursor and any other MCP client.
 
-> **Status: 2.0.** The agent tools, the viewer, the protocol and the embedding API are stable, and a check holds every
-> release to [what SimMirror promises not to break](docs/stability.md). 2.0 brings real iPhones and iPads beside the
-> simulators -- live over the cable, driven through WebDriverAgent -- and records any screen as an MP4 or a GIF. It
-> removes nothing 1.x promised.
+> **Status: 2.2.** The agent tools, the viewer, the protocol and the embedding API are stable, and a check holds every
+> release to [what SimMirror promises not to break](docs/stability.md). 2.0 brought real iPhones and iPads beside the
+> simulators -- live over the cable, driven through WebDriverAgent -- and records any screen as an MP4 or a GIF. 2.1
+> reads and touches apps turned to landscape, keeps taps off what a tab bar covers, builds the worktree the agent works
+> in, and adds observe mode for walkthroughs on accounts an agent must not change. 2.2 adds Android emulators and
+> phones, with the same viewer, cursor and tools. It removes nothing 1.x promised.
 
 ## Why SimMirror
 
@@ -88,7 +91,7 @@ Cursor and any other MCP client.
 **Any MCP client**, as a stdio server:
 
 ```sh
-claude mcp add sim-mirror -- uvx --from sim-mirror==2.0.1 sim-mirror mcp
+claude mcp add sim-mirror -- uvx --from sim-mirror==2.2.0 sim-mirror mcp
 ```
 
 Configurations for [Codex](docs/clients/codex.md), [Cursor](docs/clients/cursor.md) and
@@ -97,7 +100,7 @@ Configurations for [Codex](docs/clients/codex.md), [Cursor](docs/clients/cursor.
 **The command**, for `sim-mirror open`, `doctor` and the rest:
 
 ```sh
-uv tool install sim-mirror==2.0.1
+uv tool install sim-mirror==2.2.0
 # or
 brew install andrewkochulab/tap/sim-mirror
 ```
@@ -141,8 +144,13 @@ the viewer's own pointer glides there first (`agent.cursor_lead_ms`). Never the
 
 **Screen understanding.** A snapshot is a few lines -- `e2 button "General" (201,319)` -- with refs that stay with their
 elements across scrolls, a digest, and diffs. `sim_act` plays up to 20 steps in one call and waits for text, its
-absence, or the screen to settle -- past a spinner or a pulsing dot that never stops. See
-[Screen understanding](docs/screen-understanding.md).
+absence, or the screen to settle -- past a spinner or a pulsing dot that never stops. An app turned to landscape is
+read and touched in its own points, `sim_device rotate` turns a simulator, and a tap on what a tab bar, a toolbar or
+the keyboard covers is refused with how to reach it. See [Screen understanding](docs/screen-understanding.md).
+
+**Observe mode.** For a walkthrough on an account the agent must not change, `agent.observe` refuses a tap on what
+reads like Delete, Save, Send or Sign out -- in English and a dozen other languages, or styled destructive by its app --
+unless the call confirms it. See [Security](docs/security.md).
 
 **Reading pixels.** A screen whose accessibility says nothing -- a game, a canvas, an app still loading -- reads as the
 text macOS's Vision finds in it, with refs to tap and text to wait for, and the viewer can outline what was read
@@ -159,8 +167,14 @@ location through Xcode's `devicectl`; and touching, typing and its element tree
 with a click and signed by your project's own team -- a tap lands in about a third of a second. It is reached only over
 the cable, and what SimMirror changes on it is put back when it is let go. See [Real devices](docs/real-devices.md).
 
+**Android.** Android emulators and phones sit beside the simulators, with the same viewer, cursor and tools: adb
+lists, starts and drives them, scrcpy's server -- set up once with a click -- shows the screen live, UI Automator reads
+it for snapshots, and `sim_build_run` and `sim_test` build a Gradle project with its own wrapper. A phone is never
+started or shut down, and what SimMirror changes on it is put back. See [Android](docs/android.md).
+
 **Recording.** The viewer's Record button, `sim_record` or `sim-mirror record` keep a device's screen as an MP4, a GIF
-or both, with each touch drawn where it landed and a 9:41 status bar. See [Recording](docs/recording.md).
+or both, with each touch drawn where it landed and a 9:41 status bar, and `preset: "pr"` keeps one small enough for
+a pull request. See [Recording](docs/recording.md).
 
 **Connectors.** SimMirror's own native helper gives full control with nothing to install; idb_companion still can,
 and the simctl connector mirrors the screen view-only when neither is there, and says why. `auto` uses the fastest that
@@ -183,7 +197,9 @@ scoped tokens, and one-shot codes in URL fragments. See [Security](docs/security
 
 **Build and test.** `sim_build_run` and `sim_test` build with xcodebuild for the agent's simulator, install and
 launch, and answer with only what failed and where -- each failing test named so it can be run again alone, and on
-another simulator when asked. Off by default, since they run commands: `sim-mirror config set build.tools true`.
+another simulator when asked -- and with the folder, branch and commit they built. They build the git worktree the
+agent works in, with the configuration its scheme runs. Off by default, since they run commands: `sim-mirror config
+set build.tools true`.
 
 ## Measure the token cost
 
@@ -210,6 +226,7 @@ an image at about 750 pixels a token).
 | Xcode | 26 and 27 |
 | Connectors | native helper (full control), idb_companion 1.5 (full control), simctl (view-only) |
 | Real devices | an iPhone or iPad that Xcode's `devicectl` reaches, by cable (view-only without WebDriverAgent) |
+| Android | emulators and phones adb reaches, with the Android SDK; scrcpy 5.0.1 for the live screen; Gradle builds |
 | App SDK | Debug builds on the iOS Simulator, iOS 16 or later |
 | Clients | Claude Code, Codex, Cursor, any stdio MCP client |
 | Browsers | Chrome, Safari, Firefox, Edge |
@@ -234,8 +251,9 @@ the [command line](docs/reference/cli.md) and the [protocol](docs/reference/prot
 | **v0.2** | Build, run and test tools, a settings panel, Xcode 27's UI hierarchy through `mcpbridge`, one shared daemon for many hosts |
 | **v1.0** | Stable protocol and embedding API, held by a check; PyPI, npm and the MCP Registry; a cursor that stays while the agent works |
 | **v1.2** | SimMirror's own native helper instead of idb_companion, screens read from their pixels and settling past endless animations, an optional in-app debug SDK, and a Homebrew tap |
-| **v2.0** (now) | Real iPhones and iPads: the live screen over the cable, apps, logs, look and location without signing; full control through WebDriverAgent; recording a screen as an MP4 or GIF |
-| **Android** | Android emulator support with the same viewer, cursor and tools |
+| **v2.0** | Real iPhones and iPads: the live screen over the cable, apps, logs, look and location without signing; full control through WebDriverAgent; recording a screen as an MP4 or GIF |
+| **v2.1** | Apps turned to landscape and `sim_device rotate`, taps kept off what bars cover, builds of the agent's worktree, observe mode, `sim-mirror stop`, and recordings sized for a pull request |
+| **v2.2** (now) | Android emulators and phones with the same viewer, cursor and tools: adb, scrcpy's live screen, and Gradle builds |
 | **Website & launch** | Landing page, video tutorials and guides |
 
 See [ROADMAP.md](ROADMAP.md) and the [milestones](https://github.com/AndrewKochulab/sim-mirror/milestones). How

{sim_mirror-2.0.1 → sim_mirror-2.2.0}/README.md

@@ -2,7 +2,7 @@
 
 # SimMirror
 
-**Mirror and drive iOS Simulators and real iPhones from Claude Code, CLI agents and the browser.**
+**Mirror and drive iOS Simulators, real iPhones and Android devices from Claude Code, CLI agents and the browser.**
 
 [![CI](https://github.com/AndrewKochulab/sim-mirror/actions/workflows/ci.yml/badge.svg)](https://github.com/AndrewKochulab/sim-mirror/actions/workflows/ci.yml)
 [![License: Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
@@ -15,15 +15,18 @@ through its notes, then fills in and saves a new one, the agent's cursor gliding
 *A real session, sped up four times. [The whole recording](https://github.com/AndrewKochulab/sim-mirror/releases/download/v1.2.0/sim-mirror-plugin-demo.mp4)
 runs 1m 39s.*
 
-A live iOS Simulator -- or your own iPhone, over its cable -- in any browser tab or web page, an animated cursor that
+A live iOS Simulator -- or your own iPhone, over its cable, or an Android emulator or phone -- in any browser tab or
+web page, an animated cursor that
 shows exactly what your AI agent is about to tap, and token-efficient UI snapshots so agents read the screen as compact
 text instead of screenshots. Record any screen as an MP4 or a GIF for a pull request. Works with Claude Code, Codex,
 Cursor and any other MCP client.
 
-> **Status: 2.0.** The agent tools, the viewer, the protocol and the embedding API are stable, and a check holds every
-> release to [what SimMirror promises not to break](docs/stability.md). 2.0 brings real iPhones and iPads beside the
-> simulators -- live over the cable, driven through WebDriverAgent -- and records any screen as an MP4 or a GIF. It
-> removes nothing 1.x promised.
+> **Status: 2.2.** The agent tools, the viewer, the protocol and the embedding API are stable, and a check holds every
+> release to [what SimMirror promises not to break](docs/stability.md). 2.0 brought real iPhones and iPads beside the
+> simulators -- live over the cable, driven through WebDriverAgent -- and records any screen as an MP4 or a GIF. 2.1
+> reads and touches apps turned to landscape, keeps taps off what a tab bar covers, builds the worktree the agent works
+> in, and adds observe mode for walkthroughs on accounts an agent must not change. 2.2 adds Android emulators and
+> phones, with the same viewer, cursor and tools. It removes nothing 1.x promised.
 
 ## Why SimMirror
 
@@ -51,7 +54,7 @@ Cursor and any other MCP client.
 **Any MCP client**, as a stdio server:
 
 ```sh
-claude mcp add sim-mirror -- uvx --from sim-mirror==2.0.1 sim-mirror mcp
+claude mcp add sim-mirror -- uvx --from sim-mirror==2.2.0 sim-mirror mcp
 ```
 
 Configurations for [Codex](docs/clients/codex.md), [Cursor](docs/clients/cursor.md) and
@@ -60,7 +63,7 @@ Configurations for [Codex](docs/clients/codex.md), [Cursor](docs/clients/cursor.
 **The command**, for `sim-mirror open`, `doctor` and the rest:
 
 ```sh
-uv tool install sim-mirror==2.0.1
+uv tool install sim-mirror==2.2.0
 # or
 brew install andrewkochulab/tap/sim-mirror
 ```
@@ -104,8 +107,13 @@ the viewer's own pointer glides there first (`agent.cursor_lead_ms`). Never the
 
 **Screen understanding.** A snapshot is a few lines -- `e2 button "General" (201,319)` -- with refs that stay with their
 elements across scrolls, a digest, and diffs. `sim_act` plays up to 20 steps in one call and waits for text, its
-absence, or the screen to settle -- past a spinner or a pulsing dot that never stops. See
-[Screen understanding](docs/screen-understanding.md).
+absence, or the screen to settle -- past a spinner or a pulsing dot that never stops. An app turned to landscape is
+read and touched in its own points, `sim_device rotate` turns a simulator, and a tap on what a tab bar, a toolbar or
+the keyboard covers is refused with how to reach it. See [Screen understanding](docs/screen-understanding.md).
+
+**Observe mode.** For a walkthrough on an account the agent must not change, `agent.observe` refuses a tap on what
+reads like Delete, Save, Send or Sign out -- in English and a dozen other languages, or styled destructive by its app --
+unless the call confirms it. See [Security](docs/security.md).
 
 **Reading pixels.** A screen whose accessibility says nothing -- a game, a canvas, an app still loading -- reads as the
 text macOS's Vision finds in it, with refs to tap and text to wait for, and the viewer can outline what was read
@@ -122,8 +130,14 @@ location through Xcode's `devicectl`; and touching, typing and its element tree
 with a click and signed by your project's own team -- a tap lands in about a third of a second. It is reached only over
 the cable, and what SimMirror changes on it is put back when it is let go. See [Real devices](docs/real-devices.md).
 
+**Android.** Android emulators and phones sit beside the simulators, with the same viewer, cursor and tools: adb
+lists, starts and drives them, scrcpy's server -- set up once with a click -- shows the screen live, UI Automator reads
+it for snapshots, and `sim_build_run` and `sim_test` build a Gradle project with its own wrapper. A phone is never
+started or shut down, and what SimMirror changes on it is put back. See [Android](docs/android.md).
+
 **Recording.** The viewer's Record button, `sim_record` or `sim-mirror record` keep a device's screen as an MP4, a GIF
-or both, with each touch drawn where it landed and a 9:41 status bar. See [Recording](docs/recording.md).
+or both, with each touch drawn where it landed and a 9:41 status bar, and `preset: "pr"` keeps one small enough for
+a pull request. See [Recording](docs/recording.md).
 
 **Connectors.** SimMirror's own native helper gives full control with nothing to install; idb_companion still can,
 and the simctl connector mirrors the screen view-only when neither is there, and says why. `auto` uses the fastest that
@@ -146,7 +160,9 @@ scoped tokens, and one-shot codes in URL fragments. See [Security](docs/security
 
 **Build and test.** `sim_build_run` and `sim_test` build with xcodebuild for the agent's simulator, install and
 launch, and answer with only what failed and where -- each failing test named so it can be run again alone, and on
-another simulator when asked. Off by default, since they run commands: `sim-mirror config set build.tools true`.
+another simulator when asked -- and with the folder, branch and commit they built. They build the git worktree the
+agent works in, with the configuration its scheme runs. Off by default, since they run commands: `sim-mirror config
+set build.tools true`.
 
 ## Measure the token cost
 
@@ -173,6 +189,7 @@ an image at about 750 pixels a token).
 | Xcode | 26 and 27 |
 | Connectors | native helper (full control), idb_companion 1.5 (full control), simctl (view-only) |
 | Real devices | an iPhone or iPad that Xcode's `devicectl` reaches, by cable (view-only without WebDriverAgent) |
+| Android | emulators and phones adb reaches, with the Android SDK; scrcpy 5.0.1 for the live screen; Gradle builds |
 | App SDK | Debug builds on the iOS Simulator, iOS 16 or later |
 | Clients | Claude Code, Codex, Cursor, any stdio MCP client |
 | Browsers | Chrome, Safari, Firefox, Edge |
@@ -197,8 +214,9 @@ the [command line](docs/reference/cli.md) and the [protocol](docs/reference/prot
 | **v0.2** | Build, run and test tools, a settings panel, Xcode 27's UI hierarchy through `mcpbridge`, one shared daemon for many hosts |
 | **v1.0** | Stable protocol and embedding API, held by a check; PyPI, npm and the MCP Registry; a cursor that stays while the agent works |
 | **v1.2** | SimMirror's own native helper instead of idb_companion, screens read from their pixels and settling past endless animations, an optional in-app debug SDK, and a Homebrew tap |
-| **v2.0** (now) | Real iPhones and iPads: the live screen over the cable, apps, logs, look and location without signing; full control through WebDriverAgent; recording a screen as an MP4 or GIF |
-| **Android** | Android emulator support with the same viewer, cursor and tools |
+| **v2.0** | Real iPhones and iPads: the live screen over the cable, apps, logs, look and location without signing; full control through WebDriverAgent; recording a screen as an MP4 or GIF |
+| **v2.1** | Apps turned to landscape and `sim_device rotate`, taps kept off what bars cover, builds of the agent's worktree, observe mode, `sim-mirror stop`, and recordings sized for a pull request |
+| **v2.2** (now) | Android emulators and phones with the same viewer, cursor and tools: adb, scrcpy's live screen, and Gradle builds |
 | **Website & launch** | Landing page, video tutorials and guides |
 
 See [ROADMAP.md](ROADMAP.md) and the [milestones](https://github.com/AndrewKochulab/sim-mirror/milestones). How

{sim_mirror-2.0.1 → sim_mirror-2.2.0}/THIRD_PARTY_LICENSES.md

@@ -127,5 +127,32 @@ ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 ```
 
+## scrcpy server
+
+- **What:** the server SimMirror runs on an Android device to stream its screen as H.264 and take its touches, keys
+  and text ([Android](docs/android.md)). It is **not** shipped: a person's setup (Set up touch, or `sim-mirror android
+  setup`) fetches release 5.0.1's `scrcpy-server-v5.0.1`, checks its size and SHA-256
+  (`764eb6f79811d5211fe9df341120882ba9994c7a61b897d7bf3fb662e53bc536`), and keeps it on your Mac; it is pushed to a
+  device only while SimMirror shows it, and is unchanged.
+- **Source:** [Genymobile/scrcpy](https://github.com/Genymobile/scrcpy)
+- **License:** Apache-2.0
+
+```
+Copyright (C) 2018 Genymobile
+Copyright (C) 2018-2026 Romain Vimont
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+```
+
 Runtime dependencies installed from PyPI and npm (FastAPI, Uvicorn, grpclib, protobuf, tomlkit, Pillow and others) are
 not redistributed here; each carries its own license in its package.

{sim_mirror-2.0.1 → sim_mirror-2.2.0}/docs/README.md

@@ -13,10 +13,11 @@
 - [Cursor](clients/cursor.md)
 - [Any other MCP client](clients/other-mcp.md)
 
-## Use it with a real device, and record
+## Use it with a real device or Android, and record
 
 - [Real devices](real-devices.md) -- a cabled iPhone or iPad: its live screen, apps and logs, and WebDriverAgent for
   touch, typing and the element tree
+- [Android](android.md) -- Android emulators and phones: adb, scrcpy's live screen, and Gradle builds and tests
 - [Recording](recording.md) -- a screen as an MP4 or a GIF, with each touch drawn, and a demo in a pull request
 
 ## Change its settings

{sim_mirror-2.0.1 → sim_mirror-2.2.0}/examples/README.md

@@ -14,7 +14,9 @@ Each folder is one way to use SimMirror, small enough to copy.
 | [custom-connector](custom-connector/) | A connector of your own, installed through the `sim_mirror.connectors` entry point, with contract tests |
 | [token-budget](token-budget/) | Measuring what each tool's answer costs an agent |
 | [app-sdk](app-sdk/) | An iOS app sharing its own views through SimMirrorKit: icon buttons, a tap gesture, a hand-drawn control with a describer, `.simMirror` tags |
+| [android-app](android-app/) | An Android app and its instrumented tests, which `sim_build_run` and `sim_test` build with its Gradle wrapper |
 
-Every example assumes a Mac with Xcode and a booted-or-bootable simulator, and `sim-mirror doctor` passing. The
+Every example assumes a Mac with Xcode and a booted-or-bootable simulator -- android-app, the Android SDK and an
+emulator instead -- and `sim-mirror doctor` passing. The
 examples are tested with SimMirror's own suite (`tests/examples/`), without starting a simulator; app-sdk's hosted
 tests run on a simulator with `make sdk-app-test`.

sim_mirror-2.2.0/examples/android-app/README.md

@@ -0,0 +1,37 @@
+# Android app
+
+A grocery list in one Kotlin activity, with two instrumented tests: the smallest Android project `sim_build_run` and
+`sim_test` build, install and test on an Android emulator or phone.
+
+```
+examples/android-app/
+├── settings.gradle.kts         the Gradle build SimMirror finds
+├── gradlew, gradle/wrapper/    its own wrapper: Gradle 9.3.1, checked by its SHA-256
+└── app/                        the one app module (AGP 9.1, Kotlin built in)
+    ├── src/main/               MainActivity: type an item, press Add
+    └── src/androidTest/        MainActivityTest: startsEmpty, addsAnItem
+```
+
+## Try it
+
+With an Android emulator chosen (`sim_device list`, then `choose`) and `build.tools` on, point the build at this
+folder and ask the agent to
+
+- build and run it: `sim_build_run` with `folder: "examples/android-app"` builds `:app:assembleDebug`, installs
+  `app-debug.apk` and launches `dev.simmirror.example`;
+- run its tests: `sim_test` runs `:app:connectedDebugAndroidTest` on the device and answers with what failed and where;
+  `only_testing: ["dev.simmirror.example.MainActivityTest/addsAnItem"]` runs one;
+- launch it with an item already on the list: `sim_app` `launch` with `args: ["item=Milk"]` -- each launch argument is
+  a string extra of the launch intent.
+
+Gradle needs a JDK (17 or later): Android Studio's own is found, or name one with `android.java_home`. The first build
+fetches Gradle and the Android Gradle plugin, which takes a minute; later builds take seconds.
+
+From a shell, the same build is
+
+```sh
+cd examples/android-app
+./gradlew :app:assembleDebug
+```
+
+SimMirror's own suite never builds this project; `make live-android` does, on an emulator.

{sim_mirror-2.0.1 → sim_mirror-2.2.0}/examples/claude-code-cli/README.md

@@ -16,7 +16,7 @@ Its tools are named `mcp__plugin_sim-mirror_sim-mirror__sim_snapshot` and so on.
 ## Without the plugin
 
 ```sh
-claude mcp add sim-mirror -- uvx --from sim-mirror==2.0.1 sim-mirror mcp
+claude mcp add sim-mirror -- uvx --from sim-mirror==2.2.0 sim-mirror mcp
 ```
 
 Add `--scope project` before the name to write it to the project's `.mcp.json` for everyone working on it. Its tools
@@ -30,7 +30,7 @@ simulator (its scope) and this folder as the only place builds and installs may
 ## Watch it
 
 ```sh
-uvx --from sim-mirror==2.0.1 sim-mirror open
+uvx --from sim-mirror==2.2.0 sim-mirror open
 ```
 
 opens this folder's viewer in your browser. Every gesture the agent makes is drawn there as a cursor just before it

{sim_mirror-2.0.1 → sim_mirror-2.2.0}/examples/codex/README.md

@@ -6,7 +6,7 @@ Add [config.toml](config.toml)'s table to `~/.codex/config.toml`, then start Cod
   and installs may reach. The commented `args` pin a project by name and path instead.
 - `tool_timeout_sec` is raised because `sim_build_run` and `sim_test` can take minutes (they are there only when build
   tools are switched on).
-- Watch and take over in a browser: `uvx --from sim-mirror==2.0.1 sim-mirror open`
+- Watch and take over in a browser: `uvx --from sim-mirror==2.2.0 sim-mirror open`
   in the same folder.
 
 The tools are the same for every client; see [claude-code-cli](../claude-code-cli/) for a first prompt to try.

{sim_mirror-2.0.1 → sim_mirror-2.2.0}/examples/cursor/README.md

@@ -6,7 +6,7 @@ one), and enable the server in Cursor's MCP settings.
 - `sim-mirror mcp` gives the agent the simulator of the folder it starts in, and that folder as the only place builds
   and installs may reach. If the tools act on the wrong project, append `"--scope", "my-app", "--root",
   "/absolute/path/to/my-app"` to `args`.
-- Watch and take over in a browser: `uvx --from sim-mirror==2.0.1 sim-mirror open`
+- Watch and take over in a browser: `uvx --from sim-mirror==2.2.0 sim-mirror open`
   in the same folder.
 
 The tools are the same for every client; see [claude-code-cli](../claude-code-cli/) for a first prompt to try.

{sim_mirror-2.0.1 → sim_mirror-2.2.0}/examples/fastapi-embed/README.md

@@ -5,7 +5,7 @@ A Python application with SimMirror inside it: [app.py](app.py). No daemon runs;
 
 ```sh
 cd examples/fastapi-embed
-EXAMPLE_KEY=change-me uv run --with "sim-mirror==2.0.1" \
+EXAMPLE_KEY=change-me uv run --with "sim-mirror==2.2.0" \
   uvicorn --factory app:create_app --port 7484
 ```
 

{sim_mirror-2.0.1 → sim_mirror-2.2.0}/helper/Sources/HelperCore/Accessibility.swift

@@ -36,12 +36,17 @@ public enum AccessibilityValue: Equatable, Sendable {
 /// One element of a screen's accessibility tree, read from the device.
 ///
 /// Only what SimMirror reads of an element is kept. Each attribute is a round trip to the simulator, and a snapshot
-/// reads every element, so an attribute nothing reads -- a role's description, help text, custom actions -- is not
-/// asked for at all.
+/// reads every element, so an attribute nothing reads -- help text, custom actions -- is not asked for at all. A role's
+/// description is asked of a group only: it is how a navigation bar says what it is.
 public struct AXNode: Equatable, Sendable {
+    /// The role of an element that only holds others, as a navigation bar, a tab bar and a toolbar are listed.
+    public static let groupRole = "AXGroup"
+
     /// Its role as the accessibility API says it, ``AXButton``.
     public var role: String?
     public var subrole: String?
+    /// What its role is called, in the device's language: ``Nav bar`` for a navigation bar UIKit lists as a group.
+    public var roleDescription: String?
     public var label: String?
     public var title: String?
     public var identifier: String?
@@ -53,12 +58,14 @@ public struct AXNode: Equatable, Sendable {
     public var children: [AXNode]
 
     public init(
-        role: String? = nil, subrole: String? = nil, label: String? = nil, title: String? = nil,
-        identifier: String? = nil, value: AccessibilityValue? = nil, frame: Rect = Rect(x: 0, y: 0, width: 0, height: 0),
-        traits: UInt64? = nil, enabled: Bool = true, children: [AXNode] = []
+        role: String? = nil, subrole: String? = nil, roleDescription: String? = nil, label: String? = nil,
+        title: String? = nil, identifier: String? = nil, value: AccessibilityValue? = nil,
+        frame: Rect = Rect(x: 0, y: 0, width: 0, height: 0), traits: UInt64? = nil, enabled: Bool = true,
+        children: [AXNode] = []
     ) {
         self.role = role
         self.subrole = subrole
+        self.roleDescription = roleDescription
         self.label = label
         self.title = title
         self.identifier = identifier
@@ -111,12 +118,15 @@ public enum AXDocument {
         "Tab", "Key", "DisclosureTriangle",
     ]
 
-    public static func make(root: AXNode, screen: ScreenGeometry) -> JSONValue {
-        let elements = kept(root)
+    /// The document of a tree. Its frames are the interface's points, and its ``orientation`` says how that interface
+    /// lies on the screen, whose portrait size ``screen`` is.
+    public static func make(_ tree: AXTree, screen: ScreenGeometry) -> JSONValue {
+        let elements = kept(tree.root)
         return .object([
             "backend": .string("native"),
             "elements": .array(elements),
             "modal": .null,
+            "orientation": .string(tree.orientation.rawValue),
             "screen": .object([
                 "coordinate_space": .string("screen"),
                 "width": .int(screen.widthPt),
@@ -126,6 +136,11 @@ public enum AXDocument {
         ])
     }
 
+    /// What was found at points of the screen: each element alone, as the document writes one, or null.
+    public static func found(_ nodes: [AXNode?]) -> JSONValue {
+        .object(["elements": .array(nodes.map { $0.map { element($0, children: []) } ?? .null })])
+    }
+
     /// A role without its ``AX`` prefix: ``AXButton`` is ``Button``.
     public static func type(of role: String?) -> String? {
         guard let role else { return nil }
@@ -148,6 +163,7 @@ public enum AXDocument {
         .object([
             "type": .text(type(of: node.role)),
             "subrole": .text(node.subrole),
+            "role_description": .text(node.roleDescription),
             "label": .text(node.label),
             "title": .text(node.title),
             "identifier": .text(node.identifier),

{sim_mirror-2.0.1 → sim_mirror-2.2.0}/helper/Sources/HelperCore/Arguments.swift

@@ -142,7 +142,7 @@ public struct ServeOptions: Equatable, Sendable {
 /// What `sim-mirror-helper version` prints.
 public struct VersionReport: Equatable, Sendable, Encodable {
     /// What this helper can do besides serving a simulator, so SimMirror asks only a helper that can.
-    public static let features = ["render", "capture"]
+    public static let features = ["render", "capture", "rotate", "at"]
 
     public let version: String
     public let wire: Int
@@ -217,7 +217,7 @@ public struct SelfCheckReport: Equatable, Sendable, Encodable {
             return "\(jpeg.width)x\(jpeg.height), \(jpeg.data.count) bytes"
         }
         await check("input") { "through \(try device.input().transport.name)" }
-        await check("element tree") { "\(count(try await device.tree())) elements" }
+        await check("element tree") { "\(count(try await device.tree().root)) elements" }
         return SelfCheckReport(parts: parts)
     }
 

{sim_mirror-2.0.1 → sim_mirror-2.2.0}/helper/Sources/HelperCore/Input.swift

@@ -2,7 +2,7 @@
 import Foundation
 
 /// One input event, as SimMirror sends it: a finger at a point, a hardware button or a keyboard key, down or up.
-/// Points are the device's points, portrait.
+/// Points are the screen's own portrait points, whichever way an app's interface is turned: so is a simulator's digitizer.
 public struct HIDEvent: Equatable, Sendable, Codable {
     public enum Kind: String, Sendable, Codable { case touch, button, key }
     public enum Phase: String, Sendable, Codable { case down, up }

sim_mirror-2.2.0/helper/Sources/HelperCore/Orientation.swift

@@ -0,0 +1,117 @@
+// SPDX-License-Identifier: Apache-2.0
+import Foundation
+
+/// How an app's interface lies on the device's screen, as UIKit names the four ways.
+///
+/// A simulator's screen never turns: its framebuffer, its touches and `ScreenGeometry` are the device's portrait points
+/// whichever way the interface is drawn. Accessibility answers in the interface's own points, so a tree read from an app
+/// turned to landscape is 874 wide on a screen 402 wide. The orientation is what turns one into the other. The device
+/// turned in Simulator.app is no different: measured on iOS 26.5, its framebuffer stays portrait too.
+public enum InterfaceOrientation: String, Sendable, CaseIterable {
+    case portrait
+    case portraitUpsideDown = "portrait_upside_down"
+    case landscapeLeft = "landscape_left"
+    case landscapeRight = "landscape_right"
+
+    /// Whether the interface is the screen turned a quarter, so its width is the screen's height.
+    public var sideways: Bool {
+        self == .landscapeLeft || self == .landscapeRight
+    }
+
+    /// Where a point on the screen, in the screen's portrait points, is in the interface's points.
+    ///
+    /// Measured on an iPhone 17 Pro simulator: an app turned to landscape right is drawn with its top along the
+    /// screen's right edge, so the screen's top right corner is the interface's top left.
+    public func interfacePoint(x: Double, y: Double, on screen: ScreenGeometry) -> (x: Double, y: Double) {
+        let width = Double(screen.widthPt)
+        let height = Double(screen.heightPt)
+        switch self {
+        case .portrait: return (x, y)
+        case .portraitUpsideDown: return (width - x, height - y)
+        case .landscapeRight: return (y, width - x)
+        case .landscapeLeft: return (height - y, x)
+        }
+    }
+
+    /// Where a point of the interface, in its points, is on the screen, in the screen's portrait points.
+    public func screenPoint(x: Double, y: Double, on screen: ScreenGeometry) -> (x: Double, y: Double) {
+        let width = Double(screen.widthPt)
+        let height = Double(screen.heightPt)
+        switch self {
+        case .portrait: return (x, y)
+        case .portraitUpsideDown: return (width - x, height - y)
+        case .landscapeRight: return (width - y, x)
+        case .landscapeLeft: return (y, height - x)
+        }
+    }
+
+    /// How many of a tree's elements are looked for on the screen before points of it are.
+    static let elementsAsked = 4
+
+    /// The share of the screen probed for which way the interface is turned, across and down: inside the edges, and
+    /// nowhere two orientations put the same interface point.
+    static let probes: [(Double, Double)] = [
+        (0.3, 0.25), (0.7, 0.25), (0.3, 0.75), (0.7, 0.75), (0.5, 0.15), (0.5, 0.85), (0.2, 0.5), (0.8, 0.5),
+    ]
+
+    /// Which way a tree's interface is turned, from the size of its root -- the frontmost application -- and what is
+    /// found at points of the screen.
+    ///
+    /// An interface as wide as the screen is portrait or upside down, and one as wide as the screen is tall is one of the
+    /// landscapes. Which of the two is told by asking what is at a point of the screen (`hit`, answering the frame of
+    /// the element there in the interface's points, as the tree's are). The middle of one of the tree's small elements
+    /// is asked for where each of the two would put it on the screen: only the right one finds an element there. Then
+    /// points spread over the screen are asked: only the right one puts the point inside what is found. A point that
+    /// finds nothing, or an element both would put it in, tells nothing, and the next is asked. When none tells, the
+    /// first of the two: portrait, or landscape right -- where UIKit puts an app that takes either landscape.
+    public static func find(
+        _ root: AXNode, screen: ScreenGeometry, hit: (_ x: Double, _ y: Double) -> Rect?
+    ) -> InterfaceOrientation {
+        let interface = root.frame
+        let wideScreen = screen.widthPt > screen.heightPt
+        let quarter = interface.width > 0 && interface.height > 0 && (interface.width > interface.height) != wideScreen
+        let candidates: [InterfaceOrientation] = quarter ? [.landscapeRight, .landscapeLeft] : [.portrait, .portraitUpsideDown]
+        let most = Double(screen.widthPt * screen.heightPt) / 4
+        let small = leaves(root).filter { $0.width >= 4 && $0.height >= 4 && $0.width * $0.height <= most }
+        for frame in small.prefix(elementsAsked) {
+            let middle = (x: frame.x + frame.width / 2, y: frame.y + frame.height / 2)
+            let fitting = candidates.filter { candidate in
+                let point = candidate.screenPoint(x: middle.x, y: middle.y, on: screen)
+                return hit(point.x, point.y)?.contains(point: middle) ?? false
+            }
+            if fitting.count == 1 { return fitting[0] }
+        }
+        for (across, down) in probes {
+            let x = across * Double(screen.widthPt)
+            let y = down * Double(screen.heightPt)
+            guard let found = hit(x, y), found.width > 0, found.height > 0 else { continue }
+            let fitting = candidates.filter { found.contains(point: $0.interfacePoint(x: x, y: y, on: screen)) }
+            if fitting.count == 1 { return fitting[0] }
+        }
+        return candidates[0]
+    }
+
+    /// The frames of the elements with nothing inside them, in the tree's order.
+    static func leaves(_ node: AXNode) -> [Rect] {
+        node.children.isEmpty ? [node.frame] : node.children.flatMap(leaves)
+    }
+}
+
+extension Rect {
+    /// Whether a point is inside, edges included, give or take half a point.
+    func contains(point: (x: Double, y: Double)) -> Bool {
+        let slack = 0.5
+        return point.x >= x - slack && point.x <= x + width + slack && point.y >= y - slack && point.y <= y + height + slack
+    }
+}
+
+/// A screen's accessibility tree and the way its interface is turned, which its frames are measured in.
+public struct AXTree: Equatable, Sendable {
+    public let root: AXNode
+    public let orientation: InterfaceOrientation
+
+    public init(root: AXNode, orientation: InterfaceOrientation = .portrait) {
+        self.root = root
+        self.orientation = orientation
+    }
+}