noracu 0.1.0 → 0.1.1

package/CHANGELOG.md

@@ -1,5 +1,39 @@
 # Changelog
 
+## [0.1.1] — 2026-11-02
+
+### Added
+- **iPhone driver bundled.** imctl (the Python driver behind the 14 `phone_*`
+  tools) now ships inside the npm package at `imctl/` (MIT, ~1,000 lines).
+  No companion checkout is needed; `NORACU_IMCTL_DIR` still overrides the
+  location for development. The layer stays EXPERIMENTAL.
+- `NORACU_PYTHON`: pick the python3 that runs imctl. Without it, noracu
+  probes `/opt/homebrew/bin/python3`, `/usr/local/bin/python3`, `python3`
+  and `/usr/bin/python3` for one that imports pyobjc (Quartz, Vision,
+  AppKit) — probed once, lazily, on the first `phone_*` call. When none
+  qualifies the tool returns one actionable line with the `pip install`
+  command.
+- `phone_status` now reports `window_id`, `onscreen`, `frontmost` and
+  `paused` (true when the Mirroring window shows "Connection Paused").
+- `phone_launch` resumes a paused session (clicks Resume) after bringing the
+  window to the front. imctl gains `screenshot [PATH]` and `resume`
+  commands.
+- e2e: 35 checks (new CI-safe guard that the bundled imctl is present).
+
+### Fixed
+- `phone_screenshot` / `phone_ocr` / `phone_find` captured whatever was on
+  screen at the window's rectangle, so a covered or background Mirroring
+  window produced screenshots and OCR of other apps. Capture is now by
+  CGWindow id (`screencapture -l`), which returns the window's own pixels
+  even when it is covered; region capture remains as a fallback.
+- imctl no longer writes `__pycache__` into the package directory
+  (`PYTHONDONTWRITEBYTECODE`).
+
+### Notes
+- Mac and browser layers are unchanged.
+- The iPhone layer needs macOS 15+, iPhone Mirroring connected, and a
+  python3 with pyobjc; Apple's `/usr/bin/python3` usually lacks pyobjc.
+
 ## [0.1.0] — 2026-11-02
 
 ### Added
@@ -36,8 +70,8 @@
 
 ### Notes
 - macOS 15+ required for Mac/iPhone layers; Browser layer is cross-platform.
-- imctl is a companion checkout (sibling directory or `NORACU_IMCTL_DIR`);
-  phone tools degrade gracefully when absent.
+- imctl was a companion checkout in 0.1.0 (sibling directory or `NORACU_IMCTL_DIR`);
+  phone tools degrade gracefully when absent. Bundled since 0.1.1.
 - Tool descriptions are English; every JSON-schema property is single-typed
   (no union/`oneOf`) for reliable LLM tool-calling across clients.
 - `mac_click` / `mac_drag` refuse an `element_index` when a different app is

package/README.md

@@ -23,9 +23,9 @@ One MCP server, three control surfaces:
 |---|---|---|---|
 | **Mac** | 9 (`mac_*`) | cliclick, screencapture, JXA System Events | stable |
 | **Browser** | 12 (`browser_*`) | Playwright (persistent profile) | stable |
-| **iPhone** | 14 (`phone_*`) | iPhone Mirroring via imctl (not public; 2 tools work without it) | experimental |
+| **iPhone** | 14 (`phone_*`) | iPhone Mirroring via the bundled imctl driver (Python + pyobjc) | experimental |
 
-The iPhone layer is experimental and gated by imctl, a companion tool that is not yet public. Without imctl, most `phone_*` tools return a structured error; the Mac and browser layers are unaffected (`phone_home` / `phone_app_switcher` still work through the iPhone Mirroring menu when a phone is connected). The observe/act vocabulary follows the action/observation split popularized by ChatGPT's computer-use, re-implemented on open local tooling.
+The iPhone layer is experimental and ships with its driver, imctl (bundled in `imctl/`). It needs an active iPhone Mirroring session on this Mac and a python3 with pyobjc; when no such python is found, `phone_*` tools return a one-line `imctl unavailable` error with the install command, and the Mac and browser layers are unaffected (`phone_home` / `phone_app_switcher` work through the iPhone Mirroring menu even then). The observe/act vocabulary follows the action/observation split popularized by ChatGPT's computer-use, re-implemented on open local tooling.
 
 ## Why noracu
 
@@ -38,7 +38,7 @@ noracu is the local hands, not the model. Your MCP client (Claude Desktop, Claud
 - **Off switches that are actually off.** `NORACU_READ_ONLY=1` (observation-only, default-deny), `NORACU_DISABLE_EVAL=1`, and `file://` blocked by default. See [Security](#security).
 - **Verifiable.** 34 end-to-end checks (`npm test`), CI on macos-latest, MIT, and the whole server is one file: `src/server.mjs`.
 
-**Not a good fit if** you need multiple browser tabs or sessions (single session for now), you're not on macOS (the Mac and iPhone layers are macOS-only; the browser layer is cross-platform), or you need the iPhone layer today. If you only need browser automation, playwright-mcp has tab management and selector APIs that noracu v0.1 lacks; noracu is for when one agent also has to leave the browser and drive Mac apps.
+**Not a good fit if** you need multiple browser tabs or sessions (single session for now), you're not on macOS (the Mac and iPhone layers are macOS-only; the browser layer is cross-platform), or you need a finished iPhone layer today (it ships, but experimental). If you only need browser automation, playwright-mcp has tab management and selector APIs that noracu v0.1 lacks; noracu is for when one agent also has to leave the browser and drive Mac apps.
 
 ## Quickstart
 
@@ -87,7 +87,7 @@ Then **quit the host app completely (including any menu-bar icon) and reopen it.
 | `ENOENT` mentioning cliclick | `brew install cliclick` (or set `NORACU_CLICLICK`) |
 | `spawn node` / `npx` `ENOENT` from a GUI host | use an absolute `node` path in the config — nvm/volta/homebrew shims are not on the GUI `PATH` |
 | `browser_launch` hangs on first use | one-time Chromium download; or auto-install disabled (`NORACU_DISABLE_AUTO_INSTALL=1`) → run `npx playwright install chromium` |
-| `phone_*` says `imctl unavailable … missing Quartz` | expected — imctl is not public; ignore the phone tools (two menu tools still work, see below) |
+| `phone_*` says `imctl unavailable: no python3 with pyobjc found` | install pyobjc for a python3 (`python3 -m pip install pyobjc-framework-Quartz pyobjc-framework-Vision pyobjc-framework-AppKit`; Homebrew python3 is the usual choice) or set `NORACU_PYTHON` — see [iPhone layer](#iphone-layer-experimental) |
 | Typed text comes out re-spaced (e.g. "no ra cu") | a CJK input method is active — use `mac_paste` instead of `mac_type` |
 | `element_no_longer_valid` | the snapshot is older than 30 s or the UI changed — re-run `mac_ax_state` / `browser_snapshot` and use a fresh index |
 
@@ -97,12 +97,15 @@ Full worked examples: [docs/usage.md](docs/usage.md).
 
 ## iPhone layer (experimental)
 
-The iPhone layer has 14 `phone_*` tools that drive iPhone Mirroring through **imctl**, a companion tool that is not yet public.
+The 14 `phone_*` tools drive iPhone Mirroring through **imctl**, a small Python driver that ships inside this package (`imctl/`, MIT, about 1,000 lines). Nothing else to install on noracu's side; the layer stays experimental because it depends on a live mirror and an unlocked phone.
 
-- **With imctl** installed and an iPhone mirrored to this Mac: all 14 tools are available.
-- **Without imctl:** most `phone_*` tools return a structured `imctl unavailable` error. Two tools still work when the mirror is connected, because they go through the iPhone Mirroring menu: `phone_home` (Home Screen) and `phone_app_switcher` (App Switcher).
+**Requirements:** macOS 15+ · iPhone Mirroring connected (same Apple Account, Bluetooth + Wi-Fi on) · Accessibility + Screen Recording granted to the host app · a python3 that imports pyobjc (Quartz, Vision, AppKit).
 
-No `phone_*` tool runs without an active iPhone Mirroring session on this Mac. Tool names and behavior may change before imctl is released. Do not set `NORACU_IMCTL_DIR` unless you already have imctl from its author.
+**Python:** set `NORACU_PYTHON=/path/to/python3`, or let noracu probe `/opt/homebrew/bin/python3`, `/usr/local/bin/python3`, `python3`, `/usr/bin/python3` once, on the first `phone_*` call. Apple's `/usr/bin/python3` usually lacks pyobjc; the common working setup is Homebrew's python3 plus `python3 -m pip install pyobjc-framework-Quartz pyobjc-framework-Vision pyobjc-framework-AppKit`. When nothing qualifies, every `phone_*` tool returns one line — `imctl unavailable: no python3 with pyobjc found (tried …) — run …` — and the Mac and browser layers are unaffected.
+
+**First success:** `phone_status` → if `paused` is `true`, `phone_launch` (brings the mirror to the front and clicks Resume) → `phone_screenshot` → `phone_ocr` → `phone_find` / `phone_tap`. Screenshots and OCR capture the Mirroring window by window id, so they are the phone's own pixels even when another window covers the mirror.
+
+**Known limits:** the iPhone must be unlocked (unlocking cannot be driven from the Mac); `paused` reads `null` on non-English macOS; `phone_open` relies on Spotlight + OCR and can miss apps whose labels are truncated. `phone_home` / `phone_app_switcher` go through the iPhone Mirroring menu and work without python when the mirror is connected.
 
 ## Environment variables
 
@@ -113,7 +116,8 @@ No `phone_*` tool runs without an active iPhone Mirroring session on this Mac. T
 | `NORACU_ALLOW_FILE_URL` | unset | `=1` lets `browser_navigate` open `file://` URLs (blocked by default) |
 | `NORACU_DISABLE_AUTO_INSTALL` | unset | `=1` skips the Chromium auto-install on first `browser_launch` |
 | `NORACU_CLICLICK` | `/opt/homebrew/bin/cliclick` | cliclick binary path |
-| `NORACU_IMCTL_DIR` | sibling `../imctl` | imctl checkout location (do not set unless you have imctl) |
+| `NORACU_PYTHON` | auto-probe | python3 used for the bundled iPhone driver; must import pyobjc (Quartz, Vision, AppKit). Unset → first working of `/opt/homebrew/bin/python3`, `/usr/local/bin/python3`, `python3`, `/usr/bin/python3` |
+| `NORACU_IMCTL_DIR` | bundled `imctl/` | override the driver location (development only) |
 
 ## Security
 

package/README.zh-CN.md

@@ -23,9 +23,9 @@ For English, see [README.md](README.md).
 |---|---|---|---|
 | **Mac** | 9 个（`mac_*`） | cliclick、screencapture、JXA System Events | stable |
 | **Browser** | 12 个（`browser_*`） | Playwright（持久化 profile） | stable |
-| **iPhone** | 14 个（`phone_*`） | 经 imctl 驱动 iPhone Mirroring（尚未公开；2 个工具无需它也能用） | experimental |
+| **iPhone** | 14 个（`phone_*`） | 经随包附带的 imctl 驱动 iPhone Mirroring（Python + pyobjc） | experimental |
 
-iPhone 层是实验性的，依赖 imctl——一个尚未公开的伴生工具。没有 imctl 时，多数 `phone_*` 工具返回结构化错误；Mac 层和 Browser 层不受影响（`phone_home` / `phone_app_switcher` 在镜像已连接时仍可经 iPhone Mirroring 菜单工作）。observe/act 词汇表沿用 ChatGPT computer-use 推广的 action/observation 拆分思路，在开放的本地工具上重新实现。
+iPhone 层是实验性的，驱动 imctl 随包附带（在 `imctl/`）。它需要本 Mac 上有活动的 iPhone Mirroring 会话和一个带 pyobjc 的 python3；找不到这样的 python 时，`phone_*` 工具返回一行带安装命令的 `imctl unavailable` 错误，Mac 层和 Browser 层不受影响（`phone_home` / `phone_app_switcher` 即便如此仍可经 iPhone Mirroring 菜单工作）。observe/act 词汇表沿用 ChatGPT computer-use 推广的 action/observation 拆分思路，在开放的本地工具上重新实现。
 
 ## 为什么用 noracu
 
@@ -38,7 +38,7 @@ noracu 是本地的"手"，不是"脑"。决策由你的 MCP client（Claude Des
 - **真正能关掉的开关。** `NORACU_READ_ONLY=1`（只观察，默认拒绝）、`NORACU_DISABLE_EVAL=1`、以及默认封禁 `file://`。见 [安全](#安全)。
 - **可验证。** 34 项端到端检查（`npm test`）、CI 跑在 macos-latest、MIT 许可，整个 server 就一个文件：`src/server.mjs`。
 
-**不适合的场景：** 你需要多个浏览器标签页或会话（当前仅单会话）、你不在 macOS 上（Mac 与 iPhone 层仅限 macOS；Browser 层跨平台）、或你现在就要用 iPhone 层。如果你只需要浏览器自动化，playwright-mcp 有 noracu v0.1 还没有的 tab 管理和 selector API；noracu 的价值在于：同一个 agent 还得离开浏览器去操作 Mac 应用。
+**不适合的场景：** 你需要多个浏览器标签页或会话（当前仅单会话）、你不在 macOS 上（Mac 与 iPhone 层仅限 macOS；Browser 层跨平台）、或你现在就需要一个成品级的 iPhone 层（它已随包附带，但仍是实验性）。如果你只需要浏览器自动化，playwright-mcp 有 noracu v0.1 还没有的 tab 管理和 selector API；noracu 的价值在于：同一个 agent 还得离开浏览器去操作 Mac 应用。
 
 ## 快速开始
 
@@ -87,7 +87,7 @@ Claude Code：`claude mcp add noracu -- /绝对路径/TO/node /绝对路径/nora
 | `ENOENT` 提到 cliclick | `brew install cliclick`（或设 `NORACU_CLICLICK`） |
 | GUI 宿主报 `spawn node` / `npx` `ENOENT` | 配置里改用绝对 `node` 路径——nvm/volta/homebrew 的 shim 不在 GUI `PATH` 上 |
 | 首次 `browser_launch` 卡住 | 一次性 Chromium 下载；或自动安装被关闭（`NORACU_DISABLE_AUTO_INSTALL=1`）→ 执行 `npx playwright install chromium` |
-| `phone_*` 报 `imctl unavailable … missing Quartz` | 正常——imctl 未公开；忽略 phone 工具（两个菜单工具仍可用，见下） |
+| `phone_*` 报 `imctl unavailable: no python3 with pyobjc found` | 给某个 python3 装 pyobjc（`python3 -m pip install pyobjc-framework-Quartz pyobjc-framework-Vision pyobjc-framework-AppKit`，通常选 Homebrew 的 python3）或设 `NORACU_PYTHON`——见[iPhone 层](#iphone-层实验性) |
 | 输入的文本被重新空格化（如 "no ra cu"） | 有 CJK 输入法在激活——用 `mac_paste` 代替 `mac_type` |
 | `element_no_longer_valid` | 快照超过 30 秒或界面变了——重跑 `mac_ax_state` / `browser_snapshot` 并用新索引 |
 
@@ -97,12 +97,15 @@ Claude Code：`claude mcp add noracu -- /绝对路径/TO/node /绝对路径/nora
 
 ## iPhone 层（实验性）
 
-iPhone 层有 14 个 `phone_*` 工具，经 **imctl**（尚未公开的伴生工具）驱动 iPhone Mirroring。
+14 个 `phone_*` 工具经 **imctl** 驱动 iPhone Mirroring——一个随本包附带的小型 Python 驱动（`imctl/`，MIT，约 1,000 行）。noracu 这边不需要再装别的；该层仍为实验性，因为它依赖在线的镜像会话和已解锁的手机。
 
-- **装了 imctl** 且 iPhone 已镜像到本 Mac 时：14 个工具全部可用。
-- **没有 imctl 时：** 多数 `phone_*` 返回结构化的 `imctl unavailable` 错误。有两个工具在镜像已连接时仍可用，因为它们走 iPhone Mirroring 菜单：`phone_home`（回主屏）和 `phone_app_switcher`（打开 App 切换器）。
+**要求：** macOS 15+ · iPhone Mirroring 已连接（同一 Apple 账户，蓝牙 + Wi-Fi 打开）· Accessibility + Screen Recording 授给宿主应用 · 一个能导入 pyobjc（Quartz、Vision、AppKit）的 python3。
 
-任何 `phone_*` 工具都需要本 Mac 上有活动的 iPhone Mirroring 会话。imctl 发布前，工具名和行为可能变化。除非你已从作者处拿到 imctl，否则不要设 `NORACU_IMCTL_DIR`。
+**Python：** 设置 `NORACU_PYTHON=/path/to/python3`，或由 noracu 在首次调用 `phone_*` 时依次探测一次 `/opt/homebrew/bin/python3`、`/usr/local/bin/python3`、`python3`、`/usr/bin/python3`。Apple 自带的 `/usr/bin/python3` 通常没有 pyobjc；常见可用配置是 Homebrew 的 python3 加 `python3 -m pip install pyobjc-framework-Quartz pyobjc-framework-Vision pyobjc-framework-AppKit`。都不可用时，每个 `phone_*` 工具返回一行错误——`imctl unavailable: no python3 with pyobjc found (tried …) — run …`——Mac 层与 Browser 层不受影响。
+
+**首次成功路径：** `phone_status` → 若 `paused` 为 `true`，先 `phone_launch`（置前并点击 Resume）→ `phone_screenshot` → `phone_ocr` → `phone_find` / `phone_tap`。截图与 OCR 按窗口 id 截取镜像窗口，即使被其它窗口遮挡，拿到的也是手机自身的画面。
+
+**已知限制：** 手机须处于解锁状态（解锁无法从 Mac 侧代做）；非英文 macOS 下 `paused` 读为 `null`；`phone_open` 依赖 Spotlight + OCR，标签被截断的 app 可能找不到。`phone_home` / `phone_app_switcher` 走 iPhone Mirroring 菜单，镜像已连接时无需 python 也能用。
 
 ## 环境变量
 
@@ -113,7 +116,8 @@ iPhone 层有 14 个 `phone_*` 工具，经 **imctl**（尚未公开的伴生工
 | `NORACU_ALLOW_FILE_URL` | 未设 | `=1` 允许 `browser_navigate` 打开 `file://` URL（默认封禁） |
 | `NORACU_DISABLE_AUTO_INSTALL` | 未设 | `=1` 跳过首次 `browser_launch` 的 Chromium 自动安装 |
 | `NORACU_CLICLICK` | `/opt/homebrew/bin/cliclick` | cliclick 二进制路径 |
-| `NORACU_IMCTL_DIR` | 同级 `../imctl` | imctl 检出位置（没有 imctl 就别设） |
+| `NORACU_PYTHON` | 自动探测 | 运行随包 iPhone 驱动的 python3；须能导入 pyobjc（Quartz、Vision、AppKit）。未设置时依次尝试 `/opt/homebrew/bin/python3`、`/usr/local/bin/python3`、`python3`、`/usr/bin/python3` |
+| `NORACU_IMCTL_DIR` | 包内 `imctl/` | 覆盖驱动位置（仅开发用） |
 
 ## 安全
 

package/docs/architecture.md

@@ -39,17 +39,18 @@ A single persistent Playwright 1.58 session (one browser, one context), held in
 
 ### iPhone layer (14 tools)
 
-imctl subprocess (`python3 -m imctl.cli`, PYTHONPATH pointing at imctl/src) drives iPhone Mirroring. **Experimental**: when imctl or its Python dependencies (pyobjc/Quartz) are absent, most tools return a structured `imctl unavailable` error rather than crashing the server (`phone_home`/`phone_app_switcher` still work via the iPhone Mirroring menu).
+The bundled imctl driver (`imctl/`, Python) runs as a subprocess (`<python3> -m imctl.cli`, PYTHONPATH = `imctl/src`, no bytecode written). The interpreter is `NORACU_PYTHON` or the first of `/opt/homebrew/bin/python3`, `/usr/local/bin/python3`, `python3`, `/usr/bin/python3` that imports pyobjc (Quartz, Vision, AppKit); probed once, lazily, on the first `phone_*` call. **Experimental**: when no such python exists, tools return a structured one-line `imctl unavailable` error with the install command rather than crashing the server (`phone_home`/`phone_app_switcher` still work via the iPhone Mirroring menu).
 
-- Observe: `phone_ocr` (coordinates + text), `phone_screenshot` (captures the Mirroring window region), `phone_find` (locates text)
+- Observe: `phone_ocr` (coordinates + text), `phone_screenshot` (captures the Mirroring window by CGWindow id, so a covered window still yields its own pixels; region capture is the fallback), `phone_find` (locates text)
 - Act: `phone_tap/longpress/swipe` (normalized 0-1 coordinates), `phone_home` / `phone_app_switcher` (iPhone Mirroring menu first, imctl gesture fallback), `phone_scroll`, `phone_type` (clipboard-based, IME-safe), `phone_key`
-- Session: `phone_status`, `phone_launch`, `phone_open` (Spotlight + OCR + tap)
+- Session: `phone_status` (running, window, window_id, onscreen, frontmost, paused), `phone_launch` (ensure running + frontmost, clicks Resume when the session shows "Connection Paused"), `phone_open` (Spotlight + OCR + tap)
 
 ## Configuration
 
 | Environment variable | Default | Description |
 |---|---|---|
-| `NORACU_IMCTL_DIR` | `../imctl` beside the repo root (`src/../../imctl`) | imctl checkout location |
+| `NORACU_PYTHON` | auto-probe (see iPhone layer) | python3 that runs the bundled driver; must import pyobjc |
+| `NORACU_IMCTL_DIR` | bundled `imctl/` (`src/../imctl`) | override the driver location (development only) |
 | `NORACU_CLICLICK` | `/opt/homebrew/bin/cliclick` | path to cliclick |
 | `NORACU_READ_ONLY` | unset (full control) | `=1` observation-only mode: a default-deny allowlist of 12 tools — the observe tools (`mac_screenshot`, `mac_ax_state`, `browser_snapshot`, `browser_screenshot`, `browser_read`, `phone_status`, `phone_screenshot`, `phone_ocr`, `phone_find`) plus browser session lifecycle (`browser_launch`, `browser_navigate`, `browser_close`). `browser_navigate` still issues a page-load request. All input-synthesis tools (click/drag/type/key/paste/tap/swipe/evaluate/…) are rejected |
 | `NORACU_DISABLE_EVAL` | unset (enabled) | `=1` makes `browser_evaluate` return an error on every call (kill switch for arbitrary JS execution; it stays listed) |
@@ -61,7 +62,7 @@ imctl subprocess (`python3 -m imctl.cli`, PYTHONPATH pointing at imctl/src) driv
 - Mac-layer actions spawn subprocesses (cliclick/osascript); per-action latency is tens of milliseconds (subprocess spawn; not benchmarked in this release) — a resident FFI process could cut it (see roadmap)
 - No axState diffing (every observation is a full snapshot)
 - Single browser session (no tab-level management)
-- The iPhone layer requires iPhone Mirroring to be online; OCR accuracy depends on screen content
+- The iPhone layer requires iPhone Mirroring to be online and the phone unlocked; OCR accuracy depends on screen content; `paused` detection reads the English "Connection Paused" label
 - macOS-only (Mac/iPhone layers); the Browser layer is cross-platform
 
 ## Roadmap (v2 candidates)

package/docs/tools.md

@@ -1,7 +1,7 @@
 # Tool reference
 
 > Auto-generated from the running server (`node scripts/gen-tools.mjs`) — always in sync with the implementation.
-> noracu v0.1.0 · 35 tools
+> noracu v0.1.1 · 35 tools
 
 ## Mac layer (computer-use)
 
@@ -44,18 +44,18 @@
 
 | Tool | Arguments | Description |
 |---|---|---|
-| `phone_status` | — | EXPERIMENTAL. Requires imctl (not yet public). Return the iPhone Mirroring session status as JSON. |
-| `phone_launch` | — | EXPERIMENTAL. Requires imctl (not yet public). Ensure iPhone Mirroring is running and brought to the front. |
-| `phone_screenshot` | — | EXPERIMENTAL. Requires imctl (not yet public). Screenshot the iPhone screen (the Mirroring window region). |
-| `phone_ocr` | — | EXPERIMENTAL. Requires imctl (not yet public). OCR the iPhone screen into text with coordinates. |
-| `phone_find` | text:string? | EXPERIMENTAL. Requires imctl (not yet public). Locate text on the iPhone screen. |
-| `phone_tap` | x:number?, y:number? | EXPERIMENTAL. Requires imctl (not yet public). Tap at {x, y} in normalized 0-1 coordinates. |
-| `phone_longpress` | x:number?, y:number?, duration:number? | EXPERIMENTAL. Requires imctl (not yet public). Long-press at {x, y} for an optional duration in seconds. |
-| `phone_swipe` | from:array?, to:array?, ms:number? | EXPERIMENTAL. Requires imctl (not yet public). Swipe from [x,y] to [x,y] over an optional duration in ms. |
-| `phone_home` | — | EXPERIMENTAL. Go to the iPhone Home Screen via the iPhone Mirroring menu (works without imctl when the mirror is connected); falls back to an imctl swipe. |
-| `phone_app_switcher` | — | EXPERIMENTAL. Open the iPhone App Switcher via the iPhone Mirroring menu (works without imctl when the mirror is connected); falls back to an imctl gesture. |
-| `phone_scroll` | direction:string?, distance:number? | EXPERIMENTAL. Requires imctl (not yet public). Scroll up or down by a distance. |
-| `phone_type` | text:string? | EXPERIMENTAL. Requires imctl (not yet public). Type text via the clipboard (IME-safe). |
-| `phone_key` | key:string? | EXPERIMENTAL. Requires imctl (not yet public). Press a key: return, delete, up, down, left, or right. |
-| `phone_open` | app:string? | EXPERIMENTAL. Requires imctl (not yet public). Open an app via Spotlight search plus OCR-guided tap. |
+| `phone_status` | — | EXPERIMENTAL. Needs iPhone Mirroring and a python3 with pyobjc. Return the iPhone Mirroring session status as JSON (running, window, onscreen, frontmost, paused). |
+| `phone_launch` | — | EXPERIMENTAL. Needs iPhone Mirroring and a python3 with pyobjc. Ensure iPhone Mirroring is running, brought to the front, and resumed if the connection is paused. |
+| `phone_screenshot` | — | EXPERIMENTAL. Needs iPhone Mirroring and a python3 with pyobjc. Screenshot the iPhone screen (the Mirroring window, captured by window id so it works even when covered). |
+| `phone_ocr` | — | EXPERIMENTAL. Needs iPhone Mirroring and a python3 with pyobjc. OCR the iPhone screen into text with coordinates. |
+| `phone_find` | text:string? | EXPERIMENTAL. Needs iPhone Mirroring and a python3 with pyobjc. Locate text on the iPhone screen. |
+| `phone_tap` | x:number?, y:number? | EXPERIMENTAL. Needs iPhone Mirroring and a python3 with pyobjc. Tap at {x, y} in normalized 0-1 coordinates. |
+| `phone_longpress` | x:number?, y:number?, duration:number? | EXPERIMENTAL. Needs iPhone Mirroring and a python3 with pyobjc. Long-press at {x, y} for an optional duration in seconds. |
+| `phone_swipe` | from:array?, to:array?, ms:number? | EXPERIMENTAL. Needs iPhone Mirroring and a python3 with pyobjc. Swipe from [x,y] to [x,y] over an optional duration in ms. |
+| `phone_home` | — | EXPERIMENTAL. Go to the iPhone Home Screen via the iPhone Mirroring menu (works without python/pyobjc when the mirror is connected); falls back to an imctl swipe. |
+| `phone_app_switcher` | — | EXPERIMENTAL. Open the iPhone App Switcher via the iPhone Mirroring menu (works without python/pyobjc when the mirror is connected); falls back to an imctl gesture. |
+| `phone_scroll` | direction:string?, distance:number? | EXPERIMENTAL. Needs iPhone Mirroring and a python3 with pyobjc. Scroll up or down by a distance. |
+| `phone_type` | text:string? | EXPERIMENTAL. Needs iPhone Mirroring and a python3 with pyobjc. Type text via the clipboard (IME-safe). |
+| `phone_key` | key:string? | EXPERIMENTAL. Needs iPhone Mirroring and a python3 with pyobjc. Press a key: return, delete, up, down, left, or right. |
+| `phone_open` | app:string? | EXPERIMENTAL. Needs iPhone Mirroring and a python3 with pyobjc. Open an app via Spotlight search plus OCR-guided tap. |
 

package/docs/usage.md

@@ -161,7 +161,8 @@ model — don't let it override the task.
 | `NORACU_ALLOW_FILE_URL` | unset | `=1` allows `browser_navigate` to open `file://` |
 | `NORACU_DISABLE_AUTO_INSTALL` | unset | `=1` skips Chromium auto-install on first `browser_launch` |
 | `NORACU_CLICLICK` | `/opt/homebrew/bin/cliclick` | cliclick binary path |
-| `NORACU_IMCTL_DIR` | sibling `../imctl` | imctl location (leave unset unless you have imctl) |
+| `NORACU_PYTHON` | auto-probe | python3 for the bundled iPhone driver (must import pyobjc: Quartz, Vision, AppKit) |
+| `NORACU_IMCTL_DIR` | bundled `imctl/` | override the driver location (development only) |
 | `NORACU_CI` | unset | `=1` only affects `npm test` (skips hardware-gated checks) |
 
 ## 8. Error reference
@@ -174,10 +175,39 @@ model — don't let it override the task.
 | `ENOENT` cliclick | `brew install cliclick` or set `NORACU_CLICLICK` |
 | `… blocked by NORACU_READ_ONLY=1` | you're in observation-only mode — unset it to act |
 | `browser_evaluate disabled by NORACU_DISABLE_EVAL=1` | eval kill-switch is on |
-| `imctl unavailable …` | expected — imctl is not public; the iPhone layer is experimental |
+| `imctl unavailable: no python3 with pyobjc found …` | install pyobjc for a python3 (`python3 -m pip install pyobjc-framework-Quartz pyobjc-framework-Vision pyobjc-framework-AppKit`) or set `NORACU_PYTHON` |
 
 ## 9. iPhone layer
 
-Experimental and gated by imctl (not public). There are no worked examples
-here; see the [iPhone layer section in the README](../README.md#iphone-layer-experimental)
-for exactly which tools work without imctl and what the structured error means.
+Experimental. The driver (imctl, Python) ships inside the package; the tools
+need iPhone Mirroring connected and a python3 with pyobjc (`NORACU_PYTHON`, or
+the automatic probe described in the README). A typical session:
+
+```
+phone_status
+→ {"running": true, "window": {...}, "window_id": 275104, "onscreen": true,
+   "frontmost": false, "paused": true}
+
+phone_launch          # brings the mirror to the front and clicks Resume
+→ iPhone Mirroring ready
+
+phone_status
+→ {... "paused": false ...}
+
+phone_ocr             # OCR of the Mirroring window (captured by window id)
+→ (0.50,0.06) 'Settings' [1.00]
+  (0.18,0.31) 'FaceTime' [1.00]
+  -- 2 hits
+
+phone_find {"text": "Settings"}
+→ (0.50,0.06) 'Settings' [1.00]
+
+phone_tap {"x": 0.50, "y": 0.06}
+→ tapped (0.5, 0.06)
+```
+
+Coordinates are normalized 0–1 over the mirror window. `phone_screenshot`
+returns a PNG of the window's own pixels even when another window covers it.
+If the phone is locked, unlock it on the phone; that step cannot be driven
+from the Mac. `phone_home` / `phone_app_switcher` go through the iPhone
+Mirroring menu and work without python when the mirror is connected.

package/imctl/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 imctl contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/imctl/README.md

@@ -0,0 +1,140 @@
+# imctl — Control your iPhone from macOS
+
+> Ships inside [noracu](https://github.com/sgInnora/noracu) (`imctl/`) as the driver behind its
+> `phone_*` MCP tools; it also works standalone from this directory.
+
+`imctl` drives the **iPhone Mirroring** app on macOS (Sequoia and later)
+from the command line: tap, swipe, type, OCR the screen, and open apps —
+all locally, no network, no API keys.
+
+```
+$ imctl status
+$ imctl tap 0.5 0.5
+$ imctl home
+$ imctl type "hello"
+$ imctl ocr
+(0.50,0.06) 'Settings'
+(0.18,0.31) 'FaceTime'
+...
+$ imctl open Photos
+```
+
+## How it works
+
+```
+┌────────────┐   CGEvent (HID tap)   ┌───────────────────┐
+│   imctl    │ ────────────────────▶ │ iPhone Mirroring  │
+│            │                       │  (video stream)   │
+│  Vision ◀──┼─── screencapture ─────│                   │
+│   OCR      │                       └─────────┬─────────┘
+└────────────┘                                 │ Continuity
+                                               ▼
+                                          ┌─────────┐
+                                          │ iPhone  │
+                                          └─────────┘
+```
+
+- **Input** — synthesized CoreGraphics mouse/keyboard events posted to
+  the HID event tap. iPhone Mirroring translates them into touches.
+- **Vision** — the macOS Vision framework runs OCR locally on a
+  window screenshot (captured by CGWindow id, so it is the window's own
+  pixels even when covered) and returns text with normalized coordinates.
+- **Verification** — every gesture can be followed by a downsampled
+  pixel-hash comparison to confirm the screen actually changed.
+
+## Why these specific parameters
+
+Every constant in `input.py` is backed by empirical testing against
+the mirroring gesture recognizer:
+
+| Parameter | Value | Why |
+|-----------|-------|-----|
+| tap down-up interval | 50 ms | shorter intervals are sometimes dropped |
+| swipe steps | 20 | single-shot drags are **not** recognized as swipes |
+| initial nudge | 3 px | disambiguates swipe from icon long-press-drag |
+| swipe curve | quadratic ease-in-out | stays inside the recognizer's velocity window |
+| home gesture start | 0.95 height | higher start points are treated as in-app scrolls |
+| text input | clipboard + Cmd+V | direct keystroke synthesis breaks under CJK IMEs |
+
+## Requirements
+
+- macOS 15 (Sequoia) or later with **iPhone Mirroring** configured
+  (iPhone signed into the same Apple Account, Bluetooth + Wi-Fi on)
+- Python 3.10+ with pyobjc: `python3 -m pip install pyobjc-framework-Quartz pyobjc-framework-Vision pyobjc-framework-AppKit`
+  (Apple's `/usr/bin/python3` usually lacks it; Homebrew's `python3` is the common working choice)
+- **Accessibility** and **Screen Recording** permissions granted to
+  your terminal app (System Settings → Privacy & Security)
+
+## Install
+
+```bash
+cd imctl
+pip install -e .          # or: pip install -e ".[pyobjc]" if pyobjc missing
+imctl status
+```
+
+## Commands
+
+| Command | Description |
+|---------|-------------|
+| `status` | session + window info (JSON: running, window, window_id, onscreen, frontmost, paused) |
+| `screenshot [PATH]` | capture the mirroring window to PNG by window id (works when covered) |
+| `resume` | click Resume when the window shows Connection Paused |
+| `launch` | ensure mirroring is running and frontmost |
+| `tap X Y` | tap at normalized coords (0-1) |
+| `longpress X Y [D]` | long-press (default 0.8s) |
+| `swipe FX FY TX TY [--ms]` | eased swipe between two points |
+| `home` | home gesture (bottom-edge swipe up) |
+| `app-switcher` | swipe up and hold |
+| `scroll up/down [D]` | scroll by normalized distance |
+| `page left/right` | home-screen paging |
+| `type TEXT` | type text (clipboard-based, IME-safe) |
+| `key K` | return / delete / up / down / left / right |
+| `ocr` | OCR the screen with coordinates |
+| `find TEXT` | locate text on screen |
+| `open APP` | open an app via Spotlight + OCR + tap |
+| `verify ACTION` | run a gesture and verify the screen changed |
+
+## Python API
+
+```python
+from imctl import gestures, input, vision
+
+gestures.launch()
+gestures.home()
+
+input.tap(0.5, 0.5)
+input.swipe(0.5, 0.8, 0.5, 0.2)
+input.type_text("hello")
+
+hits = vision.read_screen()
+for h in hits:
+    print(h.nx, h.ny, h.text)
+```
+
+## Design notes
+
+**Dynamic window tracking.** Window bounds are re-resolved before every
+gesture — the window can move, close, or be re-created between actions.
+A cached position means tapping the wrong screen region.
+
+**Observe → act → re-observe.** `verify.verify_action()` implements the
+observe → act → re-observe discipline: an action is only considered
+successful when the screen content actually changed. Dropped events are
+common enough with synthesized input that silent failure is the default
+assumption.
+
+**Local-only.** No telemetry, no network calls, no API keys. OCR runs
+through the Vision framework on-device.
+
+## Limitations
+
+- The iPhone must stay unlocked and connected; lock-screen state is
+  surfaced by `status` only indirectly (window content freezes).
+- Drag-and-drop between Mac and iPhone is not supported.
+- Multi-display setups: the window is located by AX query, so it works
+  on any display, but coordinates are always screen-global.
+
+## License
+
+MIT — see [LICENSE](LICENSE).

package/imctl/pyproject.toml

@@ -0,0 +1,37 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "imctl"
+version = "0.1.0"
+description = "Control iPhone Mirroring on macOS from the CLI — CGEvent input, Vision OCR, pixel verification"
+readme = "README.md"
+license = { text = "MIT" }
+requires-python = ">=3.10"
+authors = [{ name = "imctl contributors" }]
+keywords = ["iphone", "mirroring", "macos", "automation", "cgevent", "vision-ocr"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Environment :: MacOS X",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Topic :: Utilities",
+]
+# pyobjc comes with macOS system Python; declare as optional to avoid
+# clobbering a user's pyobjc installation.
+dependencies = []
+
+[project.optional-dependencies]
+pyobjc = [
+    "pyobjc-framework-Quartz>=10.0",
+    "pyobjc-framework-Vision>=10.0",
+    "pyobjc-framework-AppKit>=10.0",
+]
+
+[project.scripts]
+imctl = "imctl.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]

package/imctl/src/imctl/__init__.py

@@ -0,0 +1,31 @@
+"""imctl — Control your iPhone from macOS via iPhone Mirroring.
+
+A CLI tool that drives the iPhone Mirroring app on macOS with
+CGEvent input, Vision OCR, and pixel-level verification.
+
+Architecture:
+
+- window.py  — AX-based dynamic window tracking (never assume position)
+- input.py   — CGEvent engine: tap / long-press / swipe / scroll / key
+- vision.py  — macOS Vision framework OCR with normalized coordinates
+- verify.py  — pixel-hash diff to confirm actions took effect
+- gestures.py — composed high-level gestures (home, app switcher, ...)
+- cli.py     — argparse CLI entry point
+
+Key design decisions (each backed by empirical testing):
+
+1. Swipe uses 20-step ease-in-out interpolation with an initial 3px
+   nudge. Single-shot drags are dropped by iPhone Mirroring's gesture
+   recognizer; stepped drags are recognized reliably.
+2. Text input goes through the clipboard (Cmd+V) to bypass IME
+   interference, restoring the user's clipboard afterwards.
+3. Tap uses a 50ms down-up interval (empirically the minimum that
+   iPhone Mirroring registers reliably).
+4. OCR runs locally via the Vision framework — no network, no API
+   keys, and returns normalized (0-1) coordinates ready for tap().
+5. Every window lookup is dynamic: the window may move or close
+   between actions, so we re-resolve bounds before each gesture.
+"""
+
+__version__ = "0.1.0"
+__all__ = ["cli", "window", "input", "vision", "verify", "gestures"]