sim-mirror 2.0.0__tar.gz → 2.1.0__tar.gz

{sim_mirror-2.0.0 → sim_mirror-2.1.0}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.5
 Name: sim-mirror
-Version: 2.0.0
-Summary: Mirror and drive the iOS Simulator from Claude Code, CLI agents and the browser.
+Version: 2.1.0
+Summary: Mirror and drive iOS Simulators and real iPhones from Claude Code, CLI agents and the browser.
 Project-URL: Homepage, https://github.com/AndrewKochulab/sim-mirror
 Project-URL: Documentation, https://github.com/AndrewKochulab/sim-mirror/tree/main/docs
 Project-URL: Issues, https://github.com/AndrewKochulab/sim-mirror/issues
@@ -39,7 +39,7 @@ Description-Content-Type: text/markdown
 
 # SimMirror
 
-**Mirror and drive the iOS Simulator from Claude Code, CLI agents and the browser.**
+**Mirror and drive iOS Simulators and real iPhones from Claude Code, CLI agents and the browser.**
 
 [![CI](https://github.com/AndrewKochulab/sim-mirror/actions/workflows/ci.yml/badge.svg)](https://github.com/AndrewKochulab/sim-mirror/actions/workflows/ci.yml)
 [![License: Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
@@ -52,14 +52,16 @@ through its notes, then fills in and saves a new one, the agent's cursor gliding
 *A real session, sped up four times. [The whole recording](https://github.com/AndrewKochulab/sim-mirror/releases/download/v1.2.0/sim-mirror-plugin-demo.mp4)
 runs 1m 39s.*
 
-A live iOS Simulator in any browser tab or web page, an animated cursor that shows exactly what your AI agent is about
-to tap, and token-efficient UI snapshots so agents read the screen as compact text instead of screenshots. Works with
-Claude Code, Codex, Cursor and any other MCP client.
+A live iOS Simulator -- or your own iPhone, over its cable -- in any browser tab or web page, an animated cursor that
+shows exactly what your AI agent is about to tap, and token-efficient UI snapshots so agents read the screen as compact
+text instead of screenshots. Record any screen as an MP4 or a GIF for a pull request. Works with Claude Code, Codex,
+Cursor and any other MCP client.
 
-> **Status: 2.0.** The agent tools, the viewer, the protocol and the embedding API are stable, and a check holds every
-> release to [what SimMirror promises not to break](docs/stability.md). 2.0 brings real iPhones and iPads beside the
-> simulators -- live over the cable, driven through WebDriverAgent -- and records any screen as an MP4 or a GIF. It
-> removes nothing 1.x promised.
+> **Status: 2.1.** The agent tools, the viewer, the protocol and the embedding API are stable, and a check holds every
+> release to [what SimMirror promises not to break](docs/stability.md). 2.0 brought real iPhones and iPads beside the
+> simulators -- live over the cable, driven through WebDriverAgent -- and records any screen as an MP4 or a GIF. 2.1
+> reads and touches apps turned to landscape, keeps taps off what a tab bar covers, builds the worktree the agent works
+> in, and adds observe mode for walkthroughs on accounts an agent must not change. It removes nothing 1.x promised.
 
 ## Why SimMirror
 
@@ -71,6 +73,9 @@ Claude Code, Codex, Cursor and any other MCP client.
   in batches with waits, and reach for a screenshot only when a question is visual.
 - **Read screens accessibility cannot.** A game or a canvas reads from its pixels, and an app under development can
   share its own views, so icon-only buttons and hand-drawn controls read by name.
+- **Test on a real iPhone.** Plug it in and it sits beside the simulators: live over the cable at up to 60 fps, touched
+  and typed on from the browser or by an agent, and given back everything SimMirror changed on it.
+- **Show what happened.** Record a flow as an MP4 or a GIF with every touch drawn in, and put it in a pull request.
 
 ## Install
 
@@ -84,7 +89,7 @@ Claude Code, Codex, Cursor and any other MCP client.
 **Any MCP client**, as a stdio server:
 
 ```sh
-claude mcp add sim-mirror -- uvx --from sim-mirror==2.0.0 sim-mirror mcp
+claude mcp add sim-mirror -- uvx --from sim-mirror==2.1.0 sim-mirror mcp
 ```
 
 Configurations for [Codex](docs/clients/codex.md), [Cursor](docs/clients/cursor.md) and
@@ -93,7 +98,7 @@ Configurations for [Codex](docs/clients/codex.md), [Cursor](docs/clients/cursor.
 **The command**, for `sim-mirror open`, `doctor` and the rest:
 
 ```sh
-uv tool install sim-mirror==2.0.0
+uv tool install sim-mirror==2.1.0
 # or
 brew install andrewkochulab/tap/sim-mirror
 ```
@@ -103,8 +108,8 @@ brew install andrewkochulab/tap/sim-mirror
 [on PyPI](https://pypi.org/project/sim-mirror/) and in a [Homebrew tap](https://github.com/AndrewKochulab/homebrew-tap).
 
 **Requirements:** an Apple Silicon Mac, Xcode 26 or later, and [uv](https://docs.astral.sh/uv/) or Homebrew. Nothing
-else: SimMirror drives a simulator with its own helper, which comes in the package. Details in
-[Installation](docs/installation.md).
+else: SimMirror drives a simulator with its own helper, which comes in the package. A real iPhone needs Developer Mode,
+and an Apple development team to be touched. Details in [Installation](docs/installation.md).
 
 ## Quickstart
 
@@ -118,6 +123,12 @@ Then ask your agent:
 > Open Settings → General → About and tell me which iOS version this simulator runs. Use sim_snapshot rather than
 > screenshots.
 
+With an iPhone plugged in, unlocked and in Developer Mode:
+
+> Switch to my iPhone, build and run the app on it, walk through sign-up and record it as a GIF.
+
+The first time, press **Set up touch** in the viewer: it builds WebDriverAgent with your project's own signing team.
+
 [Getting started](docs/getting-started.md) explains what happens along the way.
 
 ## Features
@@ -131,8 +142,13 @@ the viewer's own pointer glides there first (`agent.cursor_lead_ms`). Never the
 
 **Screen understanding.** A snapshot is a few lines -- `e2 button "General" (201,319)` -- with refs that stay with their
 elements across scrolls, a digest, and diffs. `sim_act` plays up to 20 steps in one call and waits for text, its
-absence, or the screen to settle -- past a spinner or a pulsing dot that never stops. See
-[Screen understanding](docs/screen-understanding.md).
+absence, or the screen to settle -- past a spinner or a pulsing dot that never stops. An app turned to landscape is
+read and touched in its own points, `sim_device rotate` turns a simulator, and a tap on what a tab bar, a toolbar or
+the keyboard covers is refused with how to reach it. See [Screen understanding](docs/screen-understanding.md).
+
+**Observe mode.** For a walkthrough on an account the agent must not change, `agent.observe` refuses a tap on what
+reads like Delete, Save, Send or Sign out -- in English and a dozen other languages, or styled destructive by its app --
+unless the call confirms it. See [Security](docs/security.md).
 
 **Reading pixels.** A screen whose accessibility says nothing -- a game, a canvas, an app still loading -- reads as the
 text macOS's Vision finds in it, with refs to tap and text to wait for, and the viewer can outline what was read
@@ -143,13 +159,15 @@ in a Debug build on the simulator it shares its own UIKit and SwiftUI views, so
 gesture or a hand-drawn control reads by name, and `.simMirror("Settings")` names a SwiftUI view. Optional, and nothing
 of it is in a Release build. See [The app SDK](docs/app-sdk.md).
 
-**Real devices.** A cabled iPhone or iPad shows up beside the simulators: its live screen over the cable, its apps,
-logs, look and location through Xcode's `devicectl`, and touching, typing and its element tree through a WebDriverAgent
-you build with your own team. What SimMirror changes on it is put back when it is let go. See
-[Real devices](docs/real-devices.md).
+**Real devices.** A cabled iPhone or iPad shows up beside the simulators, and an agent can switch to it
+(`sim_device list` and `choose`): its live screen over the cable, at about 55 frames a second; its apps, logs, look and
+location through Xcode's `devicectl`; and touching, typing and its element tree through WebDriverAgent, set up once
+with a click and signed by your project's own team -- a tap lands in about a third of a second. It is reached only over
+the cable, and what SimMirror changes on it is put back when it is let go. See [Real devices](docs/real-devices.md).
 
 **Recording.** The viewer's Record button, `sim_record` or `sim-mirror record` keep a device's screen as an MP4, a GIF
-or both, with each touch drawn where it landed and a 9:41 status bar. See [Recording](docs/recording.md).
+or both, with each touch drawn where it landed and a 9:41 status bar, and `preset: "pr"` keeps one small enough for
+a pull request. See [Recording](docs/recording.md).
 
 **Connectors.** SimMirror's own native helper gives full control with nothing to install; idb_companion still can,
 and the simctl connector mirrors the screen view-only when neither is there, and says why. `auto` uses the fastest that
@@ -172,7 +190,9 @@ scoped tokens, and one-shot codes in URL fragments. See [Security](docs/security
 
 **Build and test.** `sim_build_run` and `sim_test` build with xcodebuild for the agent's simulator, install and
 launch, and answer with only what failed and where -- each failing test named so it can be run again alone, and on
-another simulator when asked. Off by default, since they run commands: `sim-mirror config set build.tools true`.
+another simulator when asked -- and with the folder, branch and commit they built. They build the git worktree the
+agent works in, with the configuration its scheme runs. Off by default, since they run commands: `sim-mirror config
+set build.tools true`.
 
 ## Measure the token cost
 
@@ -206,8 +226,8 @@ an image at about 750 pixels a token).
 Rows checked on real hardware are marked verified, with the date, in [Compatibility](docs/compatibility.md).
 
 From 1.0 the tools, the protocol, the embedding API, the viewer package, settings and the command line are held to
-[what SimMirror promises not to break](docs/stability.md): a release adds, and never breaks; 2.0 broke nothing. Connectors are the
-exception, and pin a minor version.
+[what SimMirror promises not to break](docs/stability.md): a release adds, and never breaks; 2.0 broke nothing.
+Connectors are the exception, and pin a minor version.
 
 ## Documentation
 
@@ -223,7 +243,8 @@ the [command line](docs/reference/cli.md) and the [protocol](docs/reference/prot
 | **v0.2** | Build, run and test tools, a settings panel, Xcode 27's UI hierarchy through `mcpbridge`, one shared daemon for many hosts |
 | **v1.0** | Stable protocol and embedding API, held by a check; PyPI, npm and the MCP Registry; a cursor that stays while the agent works |
 | **v1.2** | SimMirror's own native helper instead of idb_companion, screens read from their pixels and settling past endless animations, an optional in-app debug SDK, and a Homebrew tap |
-| **v2.0** (now) | Real iPhones and iPads: the live screen over the cable, apps, logs, look and location without signing; full control through WebDriverAgent; recording a screen as an MP4 or GIF |
+| **v2.0** | Real iPhones and iPads: the live screen over the cable, apps, logs, look and location without signing; full control through WebDriverAgent; recording a screen as an MP4 or GIF |
+| **v2.1** (now) | Apps turned to landscape and `sim_device rotate`, taps kept off what bars cover, builds of the agent's worktree, observe mode, `sim-mirror stop`, and recordings sized for a pull request |
 | **Android** | Android emulator support with the same viewer, cursor and tools |
 | **Website & launch** | Landing page, video tutorials and guides |
 

{sim_mirror-2.0.0 → sim_mirror-2.1.0}/README.md

@@ -2,7 +2,7 @@
 
 # SimMirror
 
-**Mirror and drive the iOS Simulator from Claude Code, CLI agents and the browser.**
+**Mirror and drive iOS Simulators and real iPhones from Claude Code, CLI agents and the browser.**
 
 [![CI](https://github.com/AndrewKochulab/sim-mirror/actions/workflows/ci.yml/badge.svg)](https://github.com/AndrewKochulab/sim-mirror/actions/workflows/ci.yml)
 [![License: Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
@@ -15,14 +15,16 @@ through its notes, then fills in and saves a new one, the agent's cursor gliding
 *A real session, sped up four times. [The whole recording](https://github.com/AndrewKochulab/sim-mirror/releases/download/v1.2.0/sim-mirror-plugin-demo.mp4)
 runs 1m 39s.*
 
-A live iOS Simulator in any browser tab or web page, an animated cursor that shows exactly what your AI agent is about
-to tap, and token-efficient UI snapshots so agents read the screen as compact text instead of screenshots. Works with
-Claude Code, Codex, Cursor and any other MCP client.
+A live iOS Simulator -- or your own iPhone, over its cable -- in any browser tab or web page, an animated cursor that
+shows exactly what your AI agent is about to tap, and token-efficient UI snapshots so agents read the screen as compact
+text instead of screenshots. Record any screen as an MP4 or a GIF for a pull request. Works with Claude Code, Codex,
+Cursor and any other MCP client.
 
-> **Status: 2.0.** The agent tools, the viewer, the protocol and the embedding API are stable, and a check holds every
-> release to [what SimMirror promises not to break](docs/stability.md). 2.0 brings real iPhones and iPads beside the
-> simulators -- live over the cable, driven through WebDriverAgent -- and records any screen as an MP4 or a GIF. It
-> removes nothing 1.x promised.
+> **Status: 2.1.** The agent tools, the viewer, the protocol and the embedding API are stable, and a check holds every
+> release to [what SimMirror promises not to break](docs/stability.md). 2.0 brought real iPhones and iPads beside the
+> simulators -- live over the cable, driven through WebDriverAgent -- and records any screen as an MP4 or a GIF. 2.1
+> reads and touches apps turned to landscape, keeps taps off what a tab bar covers, builds the worktree the agent works
+> in, and adds observe mode for walkthroughs on accounts an agent must not change. It removes nothing 1.x promised.
 
 ## Why SimMirror
 
@@ -34,6 +36,9 @@ Claude Code, Codex, Cursor and any other MCP client.
   in batches with waits, and reach for a screenshot only when a question is visual.
 - **Read screens accessibility cannot.** A game or a canvas reads from its pixels, and an app under development can
   share its own views, so icon-only buttons and hand-drawn controls read by name.
+- **Test on a real iPhone.** Plug it in and it sits beside the simulators: live over the cable at up to 60 fps, touched
+  and typed on from the browser or by an agent, and given back everything SimMirror changed on it.
+- **Show what happened.** Record a flow as an MP4 or a GIF with every touch drawn in, and put it in a pull request.
 
 ## Install
 
@@ -47,7 +52,7 @@ Claude Code, Codex, Cursor and any other MCP client.
 **Any MCP client**, as a stdio server:
 
 ```sh
-claude mcp add sim-mirror -- uvx --from sim-mirror==2.0.0 sim-mirror mcp
+claude mcp add sim-mirror -- uvx --from sim-mirror==2.1.0 sim-mirror mcp
 ```
 
 Configurations for [Codex](docs/clients/codex.md), [Cursor](docs/clients/cursor.md) and
@@ -56,7 +61,7 @@ Configurations for [Codex](docs/clients/codex.md), [Cursor](docs/clients/cursor.
 **The command**, for `sim-mirror open`, `doctor` and the rest:
 
 ```sh
-uv tool install sim-mirror==2.0.0
+uv tool install sim-mirror==2.1.0
 # or
 brew install andrewkochulab/tap/sim-mirror
 ```
@@ -66,8 +71,8 @@ brew install andrewkochulab/tap/sim-mirror
 [on PyPI](https://pypi.org/project/sim-mirror/) and in a [Homebrew tap](https://github.com/AndrewKochulab/homebrew-tap).
 
 **Requirements:** an Apple Silicon Mac, Xcode 26 or later, and [uv](https://docs.astral.sh/uv/) or Homebrew. Nothing
-else: SimMirror drives a simulator with its own helper, which comes in the package. Details in
-[Installation](docs/installation.md).
+else: SimMirror drives a simulator with its own helper, which comes in the package. A real iPhone needs Developer Mode,
+and an Apple development team to be touched. Details in [Installation](docs/installation.md).
 
 ## Quickstart
 
@@ -81,6 +86,12 @@ Then ask your agent:
 > Open Settings → General → About and tell me which iOS version this simulator runs. Use sim_snapshot rather than
 > screenshots.
 
+With an iPhone plugged in, unlocked and in Developer Mode:
+
+> Switch to my iPhone, build and run the app on it, walk through sign-up and record it as a GIF.
+
+The first time, press **Set up touch** in the viewer: it builds WebDriverAgent with your project's own signing team.
+
 [Getting started](docs/getting-started.md) explains what happens along the way.
 
 ## Features
@@ -94,8 +105,13 @@ the viewer's own pointer glides there first (`agent.cursor_lead_ms`). Never the
 
 **Screen understanding.** A snapshot is a few lines -- `e2 button "General" (201,319)` -- with refs that stay with their
 elements across scrolls, a digest, and diffs. `sim_act` plays up to 20 steps in one call and waits for text, its
-absence, or the screen to settle -- past a spinner or a pulsing dot that never stops. See
-[Screen understanding](docs/screen-understanding.md).
+absence, or the screen to settle -- past a spinner or a pulsing dot that never stops. An app turned to landscape is
+read and touched in its own points, `sim_device rotate` turns a simulator, and a tap on what a tab bar, a toolbar or
+the keyboard covers is refused with how to reach it. See [Screen understanding](docs/screen-understanding.md).
+
+**Observe mode.** For a walkthrough on an account the agent must not change, `agent.observe` refuses a tap on what
+reads like Delete, Save, Send or Sign out -- in English and a dozen other languages, or styled destructive by its app --
+unless the call confirms it. See [Security](docs/security.md).
 
 **Reading pixels.** A screen whose accessibility says nothing -- a game, a canvas, an app still loading -- reads as the
 text macOS's Vision finds in it, with refs to tap and text to wait for, and the viewer can outline what was read
@@ -106,13 +122,15 @@ in a Debug build on the simulator it shares its own UIKit and SwiftUI views, so
 gesture or a hand-drawn control reads by name, and `.simMirror("Settings")` names a SwiftUI view. Optional, and nothing
 of it is in a Release build. See [The app SDK](docs/app-sdk.md).
 
-**Real devices.** A cabled iPhone or iPad shows up beside the simulators: its live screen over the cable, its apps,
-logs, look and location through Xcode's `devicectl`, and touching, typing and its element tree through a WebDriverAgent
-you build with your own team. What SimMirror changes on it is put back when it is let go. See
-[Real devices](docs/real-devices.md).
+**Real devices.** A cabled iPhone or iPad shows up beside the simulators, and an agent can switch to it
+(`sim_device list` and `choose`): its live screen over the cable, at about 55 frames a second; its apps, logs, look and
+location through Xcode's `devicectl`; and touching, typing and its element tree through WebDriverAgent, set up once
+with a click and signed by your project's own team -- a tap lands in about a third of a second. It is reached only over
+the cable, and what SimMirror changes on it is put back when it is let go. See [Real devices](docs/real-devices.md).
 
 **Recording.** The viewer's Record button, `sim_record` or `sim-mirror record` keep a device's screen as an MP4, a GIF
-or both, with each touch drawn where it landed and a 9:41 status bar. See [Recording](docs/recording.md).
+or both, with each touch drawn where it landed and a 9:41 status bar, and `preset: "pr"` keeps one small enough for
+a pull request. See [Recording](docs/recording.md).
 
 **Connectors.** SimMirror's own native helper gives full control with nothing to install; idb_companion still can,
 and the simctl connector mirrors the screen view-only when neither is there, and says why. `auto` uses the fastest that
@@ -135,7 +153,9 @@ scoped tokens, and one-shot codes in URL fragments. See [Security](docs/security
 
 **Build and test.** `sim_build_run` and `sim_test` build with xcodebuild for the agent's simulator, install and
 launch, and answer with only what failed and where -- each failing test named so it can be run again alone, and on
-another simulator when asked. Off by default, since they run commands: `sim-mirror config set build.tools true`.
+another simulator when asked -- and with the folder, branch and commit they built. They build the git worktree the
+agent works in, with the configuration its scheme runs. Off by default, since they run commands: `sim-mirror config
+set build.tools true`.
 
 ## Measure the token cost
 
@@ -169,8 +189,8 @@ an image at about 750 pixels a token).
 Rows checked on real hardware are marked verified, with the date, in [Compatibility](docs/compatibility.md).
 
 From 1.0 the tools, the protocol, the embedding API, the viewer package, settings and the command line are held to
-[what SimMirror promises not to break](docs/stability.md): a release adds, and never breaks; 2.0 broke nothing. Connectors are the
-exception, and pin a minor version.
+[what SimMirror promises not to break](docs/stability.md): a release adds, and never breaks; 2.0 broke nothing.
+Connectors are the exception, and pin a minor version.
 
 ## Documentation
 
@@ -186,7 +206,8 @@ the [command line](docs/reference/cli.md) and the [protocol](docs/reference/prot
 | **v0.2** | Build, run and test tools, a settings panel, Xcode 27's UI hierarchy through `mcpbridge`, one shared daemon for many hosts |
 | **v1.0** | Stable protocol and embedding API, held by a check; PyPI, npm and the MCP Registry; a cursor that stays while the agent works |
 | **v1.2** | SimMirror's own native helper instead of idb_companion, screens read from their pixels and settling past endless animations, an optional in-app debug SDK, and a Homebrew tap |
-| **v2.0** (now) | Real iPhones and iPads: the live screen over the cable, apps, logs, look and location without signing; full control through WebDriverAgent; recording a screen as an MP4 or GIF |
+| **v2.0** | Real iPhones and iPads: the live screen over the cable, apps, logs, look and location without signing; full control through WebDriverAgent; recording a screen as an MP4 or GIF |
+| **v2.1** (now) | Apps turned to landscape and `sim_device rotate`, taps kept off what bars cover, builds of the agent's worktree, observe mode, `sim-mirror stop`, and recordings sized for a pull request |
 | **Android** | Android emulator support with the same viewer, cursor and tools |
 | **Website & launch** | Landing page, video tutorials and guides |
 

{sim_mirror-2.0.0 → sim_mirror-2.1.0}/examples/claude-code-cli/README.md

@@ -16,7 +16,7 @@ Its tools are named `mcp__plugin_sim-mirror_sim-mirror__sim_snapshot` and so on.
 ## Without the plugin
 
 ```sh
-claude mcp add sim-mirror -- uvx --from sim-mirror==2.0.0 sim-mirror mcp
+claude mcp add sim-mirror -- uvx --from sim-mirror==2.1.0 sim-mirror mcp
 ```
 
 Add `--scope project` before the name to write it to the project's `.mcp.json` for everyone working on it. Its tools
@@ -30,7 +30,7 @@ simulator (its scope) and this folder as the only place builds and installs may
 ## Watch it
 
 ```sh
-uvx --from sim-mirror==2.0.0 sim-mirror open
+uvx --from sim-mirror==2.1.0 sim-mirror open
 ```
 
 opens this folder's viewer in your browser. Every gesture the agent makes is drawn there as a cursor just before it

{sim_mirror-2.0.0 → sim_mirror-2.1.0}/examples/codex/README.md

@@ -6,7 +6,7 @@ Add [config.toml](config.toml)'s table to `~/.codex/config.toml`, then start Cod
   and installs may reach. The commented `args` pin a project by name and path instead.
 - `tool_timeout_sec` is raised because `sim_build_run` and `sim_test` can take minutes (they are there only when build
   tools are switched on).
-- Watch and take over in a browser: `uvx --from sim-mirror==2.0.0 sim-mirror open`
+- Watch and take over in a browser: `uvx --from sim-mirror==2.1.0 sim-mirror open`
   in the same folder.
 
 The tools are the same for every client; see [claude-code-cli](../claude-code-cli/) for a first prompt to try.

{sim_mirror-2.0.0 → sim_mirror-2.1.0}/examples/cursor/README.md

@@ -6,7 +6,7 @@ one), and enable the server in Cursor's MCP settings.
 - `sim-mirror mcp` gives the agent the simulator of the folder it starts in, and that folder as the only place builds
   and installs may reach. If the tools act on the wrong project, append `"--scope", "my-app", "--root",
   "/absolute/path/to/my-app"` to `args`.
-- Watch and take over in a browser: `uvx --from sim-mirror==2.0.0 sim-mirror open`
+- Watch and take over in a browser: `uvx --from sim-mirror==2.1.0 sim-mirror open`
   in the same folder.
 
 The tools are the same for every client; see [claude-code-cli](../claude-code-cli/) for a first prompt to try.

{sim_mirror-2.0.0 → sim_mirror-2.1.0}/examples/fastapi-embed/README.md

@@ -5,7 +5,7 @@ A Python application with SimMirror inside it: [app.py](app.py). No daemon runs;
 
 ```sh
 cd examples/fastapi-embed
-EXAMPLE_KEY=change-me uv run --with "sim-mirror==2.0.0" \
+EXAMPLE_KEY=change-me uv run --with "sim-mirror==2.1.0" \
   uvicorn --factory app:create_app --port 7484
 ```
 

{sim_mirror-2.0.0 → sim_mirror-2.1.0}/helper/Sources/HelperCore/Accessibility.swift

@@ -36,12 +36,17 @@ public enum AccessibilityValue: Equatable, Sendable {
 /// One element of a screen's accessibility tree, read from the device.
 ///
 /// Only what SimMirror reads of an element is kept. Each attribute is a round trip to the simulator, and a snapshot
-/// reads every element, so an attribute nothing reads -- a role's description, help text, custom actions -- is not
-/// asked for at all.
+/// reads every element, so an attribute nothing reads -- help text, custom actions -- is not asked for at all. A role's
+/// description is asked of a group only: it is how a navigation bar says what it is.
 public struct AXNode: Equatable, Sendable {
+    /// The role of an element that only holds others, as a navigation bar, a tab bar and a toolbar are listed.
+    public static let groupRole = "AXGroup"
+
     /// Its role as the accessibility API says it, ``AXButton``.
     public var role: String?
     public var subrole: String?
+    /// What its role is called, in the device's language: ``Nav bar`` for a navigation bar UIKit lists as a group.
+    public var roleDescription: String?
     public var label: String?
     public var title: String?
     public var identifier: String?
@@ -53,12 +58,14 @@ public struct AXNode: Equatable, Sendable {
     public var children: [AXNode]
 
     public init(
-        role: String? = nil, subrole: String? = nil, label: String? = nil, title: String? = nil,
-        identifier: String? = nil, value: AccessibilityValue? = nil, frame: Rect = Rect(x: 0, y: 0, width: 0, height: 0),
-        traits: UInt64? = nil, enabled: Bool = true, children: [AXNode] = []
+        role: String? = nil, subrole: String? = nil, roleDescription: String? = nil, label: String? = nil,
+        title: String? = nil, identifier: String? = nil, value: AccessibilityValue? = nil,
+        frame: Rect = Rect(x: 0, y: 0, width: 0, height: 0), traits: UInt64? = nil, enabled: Bool = true,
+        children: [AXNode] = []
     ) {
         self.role = role
         self.subrole = subrole
+        self.roleDescription = roleDescription
         self.label = label
         self.title = title
         self.identifier = identifier
@@ -111,12 +118,15 @@ public enum AXDocument {
         "Tab", "Key", "DisclosureTriangle",
     ]
 
-    public static func make(root: AXNode, screen: ScreenGeometry) -> JSONValue {
-        let elements = kept(root)
+    /// The document of a tree. Its frames are the interface's points, and its ``orientation`` says how that interface
+    /// lies on the screen, whose portrait size ``screen`` is.
+    public static func make(_ tree: AXTree, screen: ScreenGeometry) -> JSONValue {
+        let elements = kept(tree.root)
         return .object([
             "backend": .string("native"),
             "elements": .array(elements),
             "modal": .null,
+            "orientation": .string(tree.orientation.rawValue),
             "screen": .object([
                 "coordinate_space": .string("screen"),
                 "width": .int(screen.widthPt),
@@ -126,6 +136,11 @@ public enum AXDocument {
         ])
     }
 
+    /// What was found at points of the screen: each element alone, as the document writes one, or null.
+    public static func found(_ nodes: [AXNode?]) -> JSONValue {
+        .object(["elements": .array(nodes.map { $0.map { element($0, children: []) } ?? .null })])
+    }
+
     /// A role without its ``AX`` prefix: ``AXButton`` is ``Button``.
     public static func type(of role: String?) -> String? {
         guard let role else { return nil }
@@ -148,6 +163,7 @@ public enum AXDocument {
         .object([
             "type": .text(type(of: node.role)),
             "subrole": .text(node.subrole),
+            "role_description": .text(node.roleDescription),
             "label": .text(node.label),
             "title": .text(node.title),
             "identifier": .text(node.identifier),

{sim_mirror-2.0.0 → sim_mirror-2.1.0}/helper/Sources/HelperCore/Arguments.swift

@@ -142,7 +142,7 @@ public struct ServeOptions: Equatable, Sendable {
 /// What `sim-mirror-helper version` prints.
 public struct VersionReport: Equatable, Sendable, Encodable {
     /// What this helper can do besides serving a simulator, so SimMirror asks only a helper that can.
-    public static let features = ["render", "capture"]
+    public static let features = ["render", "capture", "rotate", "at"]
 
     public let version: String
     public let wire: Int
@@ -217,7 +217,7 @@ public struct SelfCheckReport: Equatable, Sendable, Encodable {
             return "\(jpeg.width)x\(jpeg.height), \(jpeg.data.count) bytes"
         }
         await check("input") { "through \(try device.input().transport.name)" }
-        await check("element tree") { "\(count(try await device.tree())) elements" }
+        await check("element tree") { "\(count(try await device.tree().root)) elements" }
         return SelfCheckReport(parts: parts)
     }
 

{sim_mirror-2.0.0 → sim_mirror-2.1.0}/helper/Sources/HelperCore/Input.swift

@@ -2,7 +2,7 @@
 import Foundation
 
 /// One input event, as SimMirror sends it: a finger at a point, a hardware button or a keyboard key, down or up.
-/// Points are the device's points, portrait.
+/// Points are the screen's own portrait points, whichever way an app's interface is turned: so is a simulator's digitizer.
 public struct HIDEvent: Equatable, Sendable, Codable {
     public enum Kind: String, Sendable, Codable { case touch, button, key }
     public enum Phase: String, Sendable, Codable { case down, up }

sim_mirror-2.1.0/helper/Sources/HelperCore/Orientation.swift

@@ -0,0 +1,117 @@
+// SPDX-License-Identifier: Apache-2.0
+import Foundation
+
+/// How an app's interface lies on the device's screen, as UIKit names the four ways.
+///
+/// A simulator's screen never turns: its framebuffer, its touches and `ScreenGeometry` are the device's portrait points
+/// whichever way the interface is drawn. Accessibility answers in the interface's own points, so a tree read from an app
+/// turned to landscape is 874 wide on a screen 402 wide. The orientation is what turns one into the other. The device
+/// turned in Simulator.app is no different: measured on iOS 26.5, its framebuffer stays portrait too.
+public enum InterfaceOrientation: String, Sendable, CaseIterable {
+    case portrait
+    case portraitUpsideDown = "portrait_upside_down"
+    case landscapeLeft = "landscape_left"
+    case landscapeRight = "landscape_right"
+
+    /// Whether the interface is the screen turned a quarter, so its width is the screen's height.
+    public var sideways: Bool {
+        self == .landscapeLeft || self == .landscapeRight
+    }
+
+    /// Where a point on the screen, in the screen's portrait points, is in the interface's points.
+    ///
+    /// Measured on an iPhone 17 Pro simulator: an app turned to landscape right is drawn with its top along the
+    /// screen's right edge, so the screen's top right corner is the interface's top left.
+    public func interfacePoint(x: Double, y: Double, on screen: ScreenGeometry) -> (x: Double, y: Double) {
+        let width = Double(screen.widthPt)
+        let height = Double(screen.heightPt)
+        switch self {
+        case .portrait: return (x, y)
+        case .portraitUpsideDown: return (width - x, height - y)
+        case .landscapeRight: return (y, width - x)
+        case .landscapeLeft: return (height - y, x)
+        }
+    }
+
+    /// Where a point of the interface, in its points, is on the screen, in the screen's portrait points.
+    public func screenPoint(x: Double, y: Double, on screen: ScreenGeometry) -> (x: Double, y: Double) {
+        let width = Double(screen.widthPt)
+        let height = Double(screen.heightPt)
+        switch self {
+        case .portrait: return (x, y)
+        case .portraitUpsideDown: return (width - x, height - y)
+        case .landscapeRight: return (width - y, x)
+        case .landscapeLeft: return (y, height - x)
+        }
+    }
+
+    /// How many of a tree's elements are looked for on the screen before points of it are.
+    static let elementsAsked = 4
+
+    /// The share of the screen probed for which way the interface is turned, across and down: inside the edges, and
+    /// nowhere two orientations put the same interface point.
+    static let probes: [(Double, Double)] = [
+        (0.3, 0.25), (0.7, 0.25), (0.3, 0.75), (0.7, 0.75), (0.5, 0.15), (0.5, 0.85), (0.2, 0.5), (0.8, 0.5),
+    ]
+
+    /// Which way a tree's interface is turned, from the size of its root -- the frontmost application -- and what is
+    /// found at points of the screen.
+    ///
+    /// An interface as wide as the screen is portrait or upside down, and one as wide as the screen is tall is one of the
+    /// landscapes. Which of the two is told by asking what is at a point of the screen (`hit`, answering the frame of
+    /// the element there in the interface's points, as the tree's are). The middle of one of the tree's small elements
+    /// is asked for where each of the two would put it on the screen: only the right one finds an element there. Then
+    /// points spread over the screen are asked: only the right one puts the point inside what is found. A point that
+    /// finds nothing, or an element both would put it in, tells nothing, and the next is asked. When none tells, the
+    /// first of the two: portrait, or landscape right -- where UIKit puts an app that takes either landscape.
+    public static func find(
+        _ root: AXNode, screen: ScreenGeometry, hit: (_ x: Double, _ y: Double) -> Rect?
+    ) -> InterfaceOrientation {
+        let interface = root.frame
+        let wideScreen = screen.widthPt > screen.heightPt
+        let quarter = interface.width > 0 && interface.height > 0 && (interface.width > interface.height) != wideScreen
+        let candidates: [InterfaceOrientation] = quarter ? [.landscapeRight, .landscapeLeft] : [.portrait, .portraitUpsideDown]
+        let most = Double(screen.widthPt * screen.heightPt) / 4
+        let small = leaves(root).filter { $0.width >= 4 && $0.height >= 4 && $0.width * $0.height <= most }
+        for frame in small.prefix(elementsAsked) {
+            let middle = (x: frame.x + frame.width / 2, y: frame.y + frame.height / 2)
+            let fitting = candidates.filter { candidate in
+                let point = candidate.screenPoint(x: middle.x, y: middle.y, on: screen)
+                return hit(point.x, point.y)?.contains(point: middle) ?? false
+            }
+            if fitting.count == 1 { return fitting[0] }
+        }
+        for (across, down) in probes {
+            let x = across * Double(screen.widthPt)
+            let y = down * Double(screen.heightPt)
+            guard let found = hit(x, y), found.width > 0, found.height > 0 else { continue }
+            let fitting = candidates.filter { found.contains(point: $0.interfacePoint(x: x, y: y, on: screen)) }
+            if fitting.count == 1 { return fitting[0] }
+        }
+        return candidates[0]
+    }
+
+    /// The frames of the elements with nothing inside them, in the tree's order.
+    static func leaves(_ node: AXNode) -> [Rect] {
+        node.children.isEmpty ? [node.frame] : node.children.flatMap(leaves)
+    }
+}
+
+extension Rect {
+    /// Whether a point is inside, edges included, give or take half a point.
+    func contains(point: (x: Double, y: Double)) -> Bool {
+        let slack = 0.5
+        return point.x >= x - slack && point.x <= x + width + slack && point.y >= y - slack && point.y <= y + height + slack
+    }
+}
+
+/// A screen's accessibility tree and the way its interface is turned, which its frames are measured in.
+public struct AXTree: Equatable, Sendable {
+    public let root: AXNode
+    public let orientation: InterfaceOrientation
+
+    public init(root: AXNode, orientation: InterfaceOrientation = .portrait) {
+        self.root = root
+        self.orientation = orientation
+    }
+}