noracu 0.0.0-stage → 0.1.0

package/CHANGELOG.md

@@ -0,0 +1,52 @@
+# Changelog
+
+## [0.1.0] — 2026-11-02
+
+### Added
+- **Mac layer (9 tools)**: `mac_screenshot` (region crop; returns main-display
+  scale + logical size), `mac_ax_state` (JXA Accessibility tree with element
+  indices + frames), `mac_click` (dual-mode: `[x,y]` or `element_index`),
+  `mac_drag` (index-only: `from_index`/`to_index`), `mac_scroll`, `mac_key`,
+  `mac_type`, `mac_paste`, `mac_menu_command` (localized-safe menu bar
+  traversal).
+- **Browser layer (12 tools)**: Playwright session model — `browser_launch` /
+  `navigate` / `snapshot` (ARIA) / `screenshot` (format/quality/clip + DPR
+  metadata) / `click` / `fill` / `press_key` / `type` / `scroll` / `read` /
+  `evaluate` / `close`.
+- **iPhone layer (14 tools)**: iPhone Mirroring driver (imctl) — `phone_status`
+  / `launch` / `screenshot` / `ocr` / `find` / `tap` / `longpress` / `swipe`
+  / `home` / `app_switcher` / `scroll` / `type` / `key` / `open`.
+- **Security**: argv-parameterized JXA (string params never in script source), `validateArgs`
+  on every tool call, `NORACU_DISABLE_EVAL` kill switch,
+  `NORACU_READ_ONLY=1` observation mode (default-deny allowlist),
+  `file://` navigation blocked by default (`NORACU_ALLOW_FILE_URL=1`
+  opt-in), injection regression tests.
+- **CI**: GitHub Actions (macos-latest) with `NORACU_CI` e2e mode.
+- **Docs**: English and Chinese READMEs, architecture, a usage guide, and an auto-generated tool reference; SECURITY.md with per-threat status.
+- Focus guard: `mac_type` / `mac_paste` / `mac_key` accept `app` and refuse to type when that app is not frontmost; `mac_ax_state` reports `ax_empty` instead of an empty tree.
+- `mac_key` accepts human key names (cmd-shift-v, esc, return, arrow-up) with validation.
+
+### Security
+- Fixed JXA injection in `mac_ax_state` / `mac_menu_command` (static script
+  + `osascript` argv — parameters are data, never script source).
+- `mac_paste` restores the previous text clipboard after pasting
+  (non-text clipboard contents are not preserved).
+- element_index cache is never overwritten by an empty AX snapshot.
+- `mac_paste` and modifier chords in `mac_key` now use cliclick key-down/up sequences (previously failed on `kp:cmd-v`).
+
+### Notes
+- macOS 15+ required for Mac/iPhone layers; Browser layer is cross-platform.
+- imctl is a companion checkout (sibling directory or `NORACU_IMCTL_DIR`);
+  phone tools degrade gracefully when absent.
+- Tool descriptions are English; every JSON-schema property is single-typed
+  (no union/`oneOf`) for reliable LLM tool-calling across clients.
+- `mac_click` / `mac_drag` refuse an `element_index` when a different app is
+  frontmost than the one the snapshot came from (prevents clicking the wrong
+  window within the 30s cache window).
+- iPhone-layer failures (imctl missing, Mirroring window not frontmost) are reported as tool errors instead of plain text.
+- When imctl's python3 lacks pyobjc, `phone_*` returns a one-line "imctl unavailable" message instead of a raw Python traceback.
+- `mac_ax_state` retries once after ~350ms when the tree is empty, so an app just brought to the front (whose windows haven't attached to the AX tree yet) isn't falsely reported as `ax_empty`.
+- `browser_launch` reports a clean "profile in use" error when another noracu instance holds the persistent browser profile, instead of crashing.
+- `phone_home` / `phone_app_switcher` use the iPhone Mirroring menu (Home Screen / App Switcher) first — reliable without the window being frontmost and without imctl — and fall back to imctl gestures on a non-English menu or when the mirror is not connected.
+
+History note: the repository history was reset before this release; earlier unpublished `noracu-mcp` versions are superseded.

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 noracu contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,153 @@
-# Temporary Holding Version
+# noracu
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+**Local, no-cloud MCP server for computer-use on macOS apps, a Playwright browser, and (experimentally) a mirrored iPhone — the agent reads an accessibility/ARIA tree and acts by `element_index`, not pixels. No API key, no telemetry.**
+
+[![CI](https://github.com/sgInnora/noracu/actions/workflows/ci.yml/badge.svg)](https://github.com/sgInnora/noracu/actions/workflows/ci.yml)
+[![npm](https://img.shields.io/npm/v/noracu.svg)](https://www.npmjs.com/package/noracu)
+[![node](https://img.shields.io/badge/node-%3E%3D20-brightgreen.svg)](https://nodejs.org)
+[![license](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+
+中文说明见 [README.zh-CN.md](README.zh-CN.md)。
+
+One stdio MCP server. The agent snapshots a tree, gets an indexed element list, then clicks or types by `element_index` instead of raw pixels — the same observe/act split on the Mac and in the browser.
+
+![browser element_index demo](https://raw.githubusercontent.com/sgInnora/noracu/v0.1.0/docs/demo-browser.gif)
+
+*`browser_snapshot` returns an indexed ARIA tree; `browser_click` acts by `element_index`. The Mac layer runs the same loop (`mac_ax_state` → `mac_click`). Highlight boxes are overlays from `demo/record-browser.mjs`, not a product feature. The iPhone layer is not in this demo.*
+
+## What it does
+
+One MCP server, three control surfaces:
+
+| Layer | Tools | Drives | Status |
+|---|---|---|---|
+| **Mac** | 9 (`mac_*`) | cliclick, screencapture, JXA System Events | stable |
+| **Browser** | 12 (`browser_*`) | Playwright (persistent profile) | stable |
+| **iPhone** | 14 (`phone_*`) | iPhone Mirroring via imctl (not public; 2 tools work without it) | experimental |
+
+The iPhone layer is experimental and gated by imctl, a companion tool that is not yet public. Without imctl, most `phone_*` tools return a structured error; the Mac and browser layers are unaffected (`phone_home` / `phone_app_switcher` still work through the iPhone Mirroring menu when a phone is connected). The observe/act vocabulary follows the action/observation split popularized by ChatGPT's computer-use, re-implemented on open local tooling.
+
+## Why noracu
+
+noracu is the local hands, not the model. Your MCP client (Claude Desktop, Claude Code, Cursor) decides what to do; this server only observes and acts on this machine.
+
+- **Runs entirely on your machine.** No cloud, no API key, no telemetry, no postinstall scripts. Sole runtime dependency: Playwright. noracu makes no network calls of its own; the only automatic one is the one-time Chromium download on the first `browser_launch` (`NORACU_DISABLE_AUTO_INSTALL=1` turns that off). The browser still fetches the pages you point it at.
+- **Acts on semantics, not pixels.** Observation returns the accessibility tree (Mac) or ARIA tree (browser) as an indexed list — text, not screenshots. Actions reference `element_index`; indices expire after 30 s, and clicks re-verify the target (tag/name/href/position in the browser, frontmost app on the Mac) before firing. A stale reference returns `element_no_longer_valid` instead of clicking the wrong thing. Screenshots are there when the tree is not enough.
+- **One protocol, three surfaces — not three scripts.** The same observe/act loop drives native Mac apps, a persistent-profile browser, and (experimentally) a mirrored iPhone. The pattern is identical across surfaces; only the tool names differ (`mac_*` vs `browser_*`).
+- **Uses your real sessions.** Your logged-in browser profile, your Mac apps — not a hosted VM. Compared with cloud computer-use: neither is universally more reliable; they fail differently. noracu keeps the screen on your machine.
+- **Off switches that are actually off.** `NORACU_READ_ONLY=1` (observation-only, default-deny), `NORACU_DISABLE_EVAL=1`, and `file://` blocked by default. See [Security](#security).
+- **Verifiable.** 34 end-to-end checks (`npm test`), CI on macos-latest, MIT, and the whole server is one file: `src/server.mjs`.
+
+**Not a good fit if** you need multiple browser tabs or sessions (single session for now), you're not on macOS (the Mac and iPhone layers are macOS-only; the browser layer is cross-platform), or you need the iPhone layer today. If you only need browser automation, playwright-mcp has tab management and selector APIs that noracu v0.1 lacks; noracu is for when one agent also has to leave the browser and drive Mac apps.
+
+## Quickstart
+
+First success is `mac_ax_state` (or a browser snapshot) — not a click, and not a `phone_*` tool. The biggest day-one failure is granting macOS permissions to `node` instead of to the host app; grants apply only after a full quit and reopen.
+
+**1. Install** (Node ≥ 20; no postinstall script):
+
+```bash
+npm install noracu          # server at node_modules/noracu/src/server.mjs
+brew install cliclick       # required for Mac input (mac_click/type/key/paste)
+```
+
+Or from source: `git clone https://github.com/sgInnora/noracu.git && cd noracu && npm install && npm test` (add `NORACU_CI=1` to skip hardware-gated checks).
+
+**2. Add to your MCP client — use an absolute path to `src/server.mjs`.** GUI apps launched from Finder/Dock do not inherit your shell `PATH`, so give an absolute path to the `node` binary as well (`which node`) — a bare `node`, `npx`, or `noracu` needs Node on the GUI `PATH` and otherwise fails with `ENOENT`.
+
+Claude Desktop (`~/Library/Application Support/Claude/claude_desktop_config.json`) / Cursor (`~/.cursor/mcp.json`):
+
+```json
+{
+  "mcpServers": {
+    "noracu": {
+      "command": "/opt/homebrew/bin/node",
+      "args": ["/ABSOLUTE/PATH/TO/noracu/src/server.mjs"]
+    }
+  }
+}
+```
+
+Claude Code: `claude mcp add noracu -- /ABSOLUTE/PATH/TO/node /ABSOLUTE/PATH/TO/noracu/src/server.mjs`
+
+Find your node path with `which node` (Apple Silicon: `/opt/homebrew/bin/node`; Intel: `/usr/local/bin/node`; nvm: the versioned path). Convenience, only if Node is already on the GUI `PATH`: `"command": "npx", "args": ["-y", "noracu"]`.
+
+Then **quit the host app completely (including any menu-bar icon) and reopen it.** A loaded tool list means the transport works — it does not need any permission yet.
+
+**3. Grant macOS permissions to the host app, not to `node`.** System Settings → Privacy & Security → **Accessibility** and **Screen Recording** (macOS 15: "Screen & System Audio Recording") → enable the host (Claude Desktop / Cursor / your terminal). noracu never appears here. Quit and reopen the host after granting.
+
+**4. Verify.** With Finder in front, call `mac_ax_state` (no args) — it returns an indexed accessibility tree; indices last 30 s. Browser smoke test (no Accessibility needed): `browser_launch` (first run downloads Chromium unless `NORACU_DISABLE_AUTO_INSTALL=1`) → `browser_navigate` → `browser_snapshot` → `browser_click` by index.
+
+### Troubleshooting
+
+| Symptom | Cause / fix |
+|---|---|
+| `mac_*` fails with error **-1719** | Accessibility not granted to the **host** app — grant it, then quit + reopen the host |
+| Screenshots come back all black | Screen Recording not granted to the host — grant it, then quit + reopen |
+| `ENOENT` mentioning cliclick | `brew install cliclick` (or set `NORACU_CLICLICK`) |
+| `spawn node` / `npx` `ENOENT` from a GUI host | use an absolute `node` path in the config — nvm/volta/homebrew shims are not on the GUI `PATH` |
+| `browser_launch` hangs on first use | one-time Chromium download; or auto-install disabled (`NORACU_DISABLE_AUTO_INSTALL=1`) → run `npx playwright install chromium` |
+| `phone_*` says `imctl unavailable … missing Quartz` | expected — imctl is not public; ignore the phone tools (two menu tools still work, see below) |
+| Typed text comes out re-spaced (e.g. "no ra cu") | a CJK input method is active — use `mac_paste` instead of `mac_type` |
+| `element_no_longer_valid` | the snapshot is older than 30 s or the UI changed — re-run `mac_ax_state` / `browser_snapshot` and use a fresh index |
+
+Node 18 is rejected at startup — use Node ≥ 20. Claude Desktop may start one noracu process per surface (e.g. chat and a second surface); they share the browser profile, so launch the browser from only one at a time.
+
+Full worked examples: [docs/usage.md](docs/usage.md).
+
+## iPhone layer (experimental)
+
+The iPhone layer has 14 `phone_*` tools that drive iPhone Mirroring through **imctl**, a companion tool that is not yet public.
+
+- **With imctl** installed and an iPhone mirrored to this Mac: all 14 tools are available.
+- **Without imctl:** most `phone_*` tools return a structured `imctl unavailable` error. Two tools still work when the mirror is connected, because they go through the iPhone Mirroring menu: `phone_home` (Home Screen) and `phone_app_switcher` (App Switcher).
+
+No `phone_*` tool runs without an active iPhone Mirroring session on this Mac. Tool names and behavior may change before imctl is released. Do not set `NORACU_IMCTL_DIR` unless you already have imctl from its author.
+
+## Environment variables
+
+| Variable | Default | Purpose |
+|---|---|---|
+| `NORACU_READ_ONLY` | unset | `=1` observation-only: a default-deny allowlist of 12 observe + browser-lifecycle tools; every click/type/paste/key/eval tool is refused |
+| `NORACU_DISABLE_EVAL` | unset | `=1` makes `browser_evaluate` (arbitrary JS) return an error when called (it stays listed) |
+| `NORACU_ALLOW_FILE_URL` | unset | `=1` lets `browser_navigate` open `file://` URLs (blocked by default) |
+| `NORACU_DISABLE_AUTO_INSTALL` | unset | `=1` skips the Chromium auto-install on first `browser_launch` |
+| `NORACU_CLICLICK` | `/opt/homebrew/bin/cliclick` | cliclick binary path |
+| `NORACU_IMCTL_DIR` | sibling `../imctl` | imctl checkout location (do not set unless you have imctl) |
+
+## Security
+
+noracu runs with the privileges of your MCP client. It clicks, types, pastes, and drives a real browser on this machine — there is no sandbox and no remote approval prompt. Set these on the server entry, then quit the host completely and reopen it:
+
+```json
+"env": { "NORACU_READ_ONLY": "1", "NORACU_DISABLE_EVAL": "1" }
+```
+
+- **`NORACU_READ_ONLY=1`** is the safe first-run mode. It refuses every tool not on a 12-name allowlist (the observe tools plus `browser_launch`/`browser_navigate`/`browser_close`); every click, type, paste, and eval tool is denied. `browser_navigate` is on the allowlist, so it can still load a URL.
+- **`NORACU_DISABLE_EVAL=1`** makes `browser_evaluate` return an error when called (it stays listed in `tools/list`). Snapshot, click, and type still work.
+- **`file://` is refused** unless `NORACU_ALLOW_FILE_URL=1`, so a browser agent can't read `~/Documents` by navigating to it. (The initial navigation URL is scheme-checked; redirects are not — see SECURITY.md.)
+- **Focus guard:** pass `app` to `mac_type` / `mac_paste` / `mac_key` and input is refused unless that app is frontmost. Omit `app` and the guard is off.
+- **Stale-element protection:** an `element_index` is not a coordinate. It expires after 30 s (`element_no_longer_valid`); a browser click re-checks tag/name/href/position, and a Mac click is refused if the frontmost app changed since the snapshot.
+- **Injection-safe JXA:** string parameters are passed to `osascript` as argv data, never built into the script; the one numeric value (AX read depth) is clamped to 1–10 before use. Regression-tested.
+
+These switches do not make observed content trustworthy. Screenshot text, accessibility names, and page content are untrusted input to your model — do not let them override the task. Full threat model and per-threat status: [SECURITY.md](SECURITY.md).
+
+## Tools & docs
+
+- Full tool reference: [docs/tools.md](docs/tools.md) (auto-generated from the running server)
+- Usage walkthroughs: [docs/usage.md](docs/usage.md)
+- Architecture: [docs/architecture.md](docs/architecture.md)
+
+## Roadmap
+
+- Browser: tab management (list / select / close)
+- Act vocabulary (browser): `hover`, `set_value`, `wait_for`
+- Snapshot diffing to cut token use — pending measurement
+
+## Maintenance
+
+Solo-maintained, no SLA. macOS 15+ for the Mac and iPhone layers; the browser layer runs wherever Playwright does.
+
+## License
+
+MIT — see [LICENSE](LICENSE).

package/README.zh-CN.md

@@ -0,0 +1,153 @@
+# noracu
+
+**本地、无云的 computer-use MCP server：在 macOS 原生应用、Playwright 浏览器、以及（实验性的）镜像 iPhone 上操作——agent 读取 accessibility / ARIA 树，按 `element_index` 动作，而非像素坐标。无 API key，无遥测。**
+
+[![CI](https://github.com/sgInnora/noracu/actions/workflows/ci.yml/badge.svg)](https://github.com/sgInnora/noracu/actions/workflows/ci.yml)
+[![npm](https://img.shields.io/npm/v/noracu.svg)](https://www.npmjs.com/package/noracu)
+[![node](https://img.shields.io/badge/node-%3E%3D20-brightgreen.svg)](https://nodejs.org)
+[![license](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+
+For English, see [README.md](README.md).
+
+一个 stdio MCP server。agent 对界面做一次快照、拿到带索引的元素列表，然后按 `element_index` 点击或输入，而不是用原始像素坐标——Mac 层和 Browser 层是同一套 observe/act 协议。
+
+![browser element_index demo](https://raw.githubusercontent.com/sgInnora/noracu/v0.1.0/docs/demo-browser.gif)
+
+*`browser_snapshot` 返回带索引的 ARIA 树；`browser_click` 按 `element_index` 动作。Mac 层跑同一个循环（`mac_ax_state` → `mac_click`）。高亮框是 `demo/record-browser.mjs` 录制时的叠加层，不是产品功能。iPhone 层不在此 demo 中。*
+
+## 它做什么
+
+一个 MCP server，三个控制面：
+
+| 层 | 工具 | 底层 | 状态 |
+|---|---|---|---|
+| **Mac** | 9 个（`mac_*`） | cliclick、screencapture、JXA System Events | stable |
+| **Browser** | 12 个（`browser_*`） | Playwright（持久化 profile） | stable |
+| **iPhone** | 14 个（`phone_*`） | 经 imctl 驱动 iPhone Mirroring（尚未公开；2 个工具无需它也能用） | experimental |
+
+iPhone 层是实验性的，依赖 imctl——一个尚未公开的伴生工具。没有 imctl 时，多数 `phone_*` 工具返回结构化错误；Mac 层和 Browser 层不受影响（`phone_home` / `phone_app_switcher` 在镜像已连接时仍可经 iPhone Mirroring 菜单工作）。observe/act 词汇表沿用 ChatGPT computer-use 推广的 action/observation 拆分思路，在开放的本地工具上重新实现。
+
+## 为什么用 noracu
+
+noracu 是本地的"手"，不是"脑"。决策由你的 MCP client（Claude Desktop、Claude Code、Cursor）做；本 server 只在这台机器上观察与动作。
+
+- **完全在本机运行。** 无云、无 API key、无遥测、无 postinstall 脚本。唯一运行期依赖是 Playwright。noracu 自身不发任何网络请求；唯一的自动网络行为是首次 `browser_launch` 时一次性下载 Chromium（`NORACU_DISABLE_AUTO_INSTALL=1` 可关闭）。你让浏览器打开的页面当然会被抓取。
+- **按语义动作，而非像素。** 观察返回 accessibility 树（Mac）或 ARIA 树（browser）的带索引列表——是文本，不是截图。动作引用 `element_index`；索引 30 秒后过期，点击前会重新校验目标（browser 校验 tag/name/href/位置，Mac 校验 frontmost app）。索引失效时返回 `element_no_longer_valid`，而不是点错位置。树不够用时才用截图。
+- **一套协议，三个面——不是三套脚本。** 同一个 observe/act 循环驱动 Mac 原生应用、持久化 profile 的浏览器、以及（实验性的）镜像 iPhone。模式跨面一致，只是工具名不同（`mac_*` vs `browser_*`）。
+- **用你真实的会话。** 你已登录的浏览器 profile、你的 Mac 应用——不是托管的 VM。与云端 computer-use 相比：两者没有谁普遍更可靠，只是失败方式不同。noracu 把屏幕留在你本机。
+- **真正能关掉的开关。** `NORACU_READ_ONLY=1`（只观察，默认拒绝）、`NORACU_DISABLE_EVAL=1`、以及默认封禁 `file://`。见 [安全](#安全)。
+- **可验证。** 34 项端到端检查（`npm test`）、CI 跑在 macos-latest、MIT 许可，整个 server 就一个文件：`src/server.mjs`。
+
+**不适合的场景：** 你需要多个浏览器标签页或会话（当前仅单会话）、你不在 macOS 上（Mac 与 iPhone 层仅限 macOS；Browser 层跨平台）、或你现在就要用 iPhone 层。如果你只需要浏览器自动化，playwright-mcp 有 noracu v0.1 还没有的 tab 管理和 selector API；noracu 的价值在于：同一个 agent 还得离开浏览器去操作 Mac 应用。
+
+## 快速开始
+
+第一个该成功的调用是 `mac_ax_state`（或一次 browser 快照）——不是点击，也不是 `phone_*`。上手第一天最常见的失败是：把 macOS 权限授给了 `node` 而不是宿主应用；权限要在**完全退出并重开**宿主后才生效。
+
+**1. 安装**（Node ≥ 20；无 postinstall 脚本）：
+
+```bash
+npm install noracu          # server 位于 node_modules/noracu/src/server.mjs
+brew install cliclick       # Mac 输入所需（mac_click/type/key/paste）
+```
+
+或从源码：`git clone https://github.com/sgInnora/noracu.git && cd noracu && npm install && npm test`（加 `NORACU_CI=1` 可跳过依赖硬件的检查）。
+
+**2. 接入你的 MCP client——用 `src/server.mjs` 的绝对路径。** 从 Finder/Dock 启动的 GUI 应用不继承你的 shell `PATH`，所以 `node` 二进制也要给绝对路径（`which node`）——裸 `node`、`npx` 或 `noracu` 都需要 Node 在 GUI `PATH` 上，否则会 `ENOENT` 失败。
+
+Claude Desktop（`~/Library/Application Support/Claude/claude_desktop_config.json`）/ Cursor（`~/.cursor/mcp.json`）：
+
+```json
+{
+  "mcpServers": {
+    "noracu": {
+      "command": "/opt/homebrew/bin/node",
+      "args": ["/绝对路径/TO/noracu/src/server.mjs"]
+    }
+  }
+}
+```
+
+Claude Code：`claude mcp add noracu -- /绝对路径/TO/node /绝对路径/noracu/src/server.mjs`
+
+用 `which node` 找到你的 node 路径（Apple Silicon：`/opt/homebrew/bin/node`；Intel：`/usr/local/bin/node`；nvm：对应版本路径）。便捷写法，仅当 Node 已在 GUI `PATH` 上时：`"command": "npx", "args": ["-y", "noracu"]`。
+
+然后**完全退出宿主应用（含菜单栏图标）并重开**。工具列表能加载出来，说明 transport 通了——此时还不需要任何权限。
+
+**3. 把 macOS 权限授给宿主应用，而不是 `node`。** 系统设置 → 隐私与安全性 → **辅助功能（Accessibility）** 和 **屏幕录制（Screen Recording）**（macOS 15 为"屏幕与系统音频录制"）→ 勾选宿主（Claude Desktop / Cursor / 你的终端）。noracu 不会出现在这里。授权后退出并重开宿主。
+
+**4. 验证。** 让 Finder 在最前，调用 `mac_ax_state`（无参数）——它返回带索引的 accessibility 树；索引有效 30 秒。浏览器冒烟测试（不需要 Accessibility）：`browser_launch`（首次运行会下载 Chromium，除非 `NORACU_DISABLE_AUTO_INSTALL=1`）→ `browser_navigate` → `browser_snapshot` → 按索引 `browser_click`。
+
+### 排障
+
+| 现象 | 原因 / 解决 |
+|---|---|
+| `mac_*` 报错 **-1719** | Accessibility 没授给**宿主**应用——授权后退出并重开宿主 |
+| 截图全黑 | Screen Recording 没授给宿主——授权后退出并重开 |
+| `ENOENT` 提到 cliclick | `brew install cliclick`（或设 `NORACU_CLICLICK`） |
+| GUI 宿主报 `spawn node` / `npx` `ENOENT` | 配置里改用绝对 `node` 路径——nvm/volta/homebrew 的 shim 不在 GUI `PATH` 上 |
+| 首次 `browser_launch` 卡住 | 一次性 Chromium 下载；或自动安装被关闭（`NORACU_DISABLE_AUTO_INSTALL=1`）→ 执行 `npx playwright install chromium` |
+| `phone_*` 报 `imctl unavailable … missing Quartz` | 正常——imctl 未公开；忽略 phone 工具（两个菜单工具仍可用，见下） |
+| 输入的文本被重新空格化（如 "no ra cu"） | 有 CJK 输入法在激活——用 `mac_paste` 代替 `mac_type` |
+| `element_no_longer_valid` | 快照超过 30 秒或界面变了——重跑 `mac_ax_state` / `browser_snapshot` 并用新索引 |
+
+启动时拒绝 Node 18——请用 Node ≥ 20。Claude Desktop 可能为每个使用面各起一个 noracu 进程；它们共享浏览器 profile，所以同一时间只从一个面启动浏览器。
+
+完整实操示例见 [docs/usage.md](docs/usage.md)（英文）。
+
+## iPhone 层（实验性）
+
+iPhone 层有 14 个 `phone_*` 工具，经 **imctl**（尚未公开的伴生工具）驱动 iPhone Mirroring。
+
+- **装了 imctl** 且 iPhone 已镜像到本 Mac 时：14 个工具全部可用。
+- **没有 imctl 时：** 多数 `phone_*` 返回结构化的 `imctl unavailable` 错误。有两个工具在镜像已连接时仍可用，因为它们走 iPhone Mirroring 菜单：`phone_home`（回主屏）和 `phone_app_switcher`（打开 App 切换器）。
+
+任何 `phone_*` 工具都需要本 Mac 上有活动的 iPhone Mirroring 会话。imctl 发布前，工具名和行为可能变化。除非你已从作者处拿到 imctl，否则不要设 `NORACU_IMCTL_DIR`。
+
+## 环境变量
+
+| 变量 | 默认 | 作用 |
+|---|---|---|
+| `NORACU_READ_ONLY` | 未设 | `=1` 只观察模式：默认拒绝的白名单仅 12 个观察 + 浏览器生命周期工具；所有 click/type/paste/key/eval 类工具被拒 |
+| `NORACU_DISABLE_EVAL` | 未设 | `=1` 让 `browser_evaluate`（任意 JS）被调用时报错（仍在列表中） |
+| `NORACU_ALLOW_FILE_URL` | 未设 | `=1` 允许 `browser_navigate` 打开 `file://` URL（默认封禁） |
+| `NORACU_DISABLE_AUTO_INSTALL` | 未设 | `=1` 跳过首次 `browser_launch` 的 Chromium 自动安装 |
+| `NORACU_CLICLICK` | `/opt/homebrew/bin/cliclick` | cliclick 二进制路径 |
+| `NORACU_IMCTL_DIR` | 同级 `../imctl` | imctl 检出位置（没有 imctl 就别设） |
+
+## 安全
+
+noracu 以你 MCP client 的权限运行。它会在本机点击、输入、粘贴、驱动真实浏览器——没有沙箱，没有远程确认弹窗。请在 server 条目上设好下面这些，然后完全退出宿主并重开：
+
+```json
+"env": { "NORACU_READ_ONLY": "1", "NORACU_DISABLE_EVAL": "1" }
+```
+
+- **`NORACU_READ_ONLY=1`** 是安全的首次运行模式。它拒绝不在 12 项白名单内的一切工具（观察类 + `browser_launch`/`browser_navigate`/`browser_close`）；所有 click/type/paste/eval 被拒。`browser_navigate` 在白名单内，仍会加载 URL。
+- **`NORACU_DISABLE_EVAL=1`** 让 `browser_evaluate` 被调用时报错（仍列在 `tools/list` 中）。snapshot、click、type 保留。
+- **`file://` 被拒**，除非 `NORACU_ALLOW_FILE_URL=1`，这样浏览器 agent 无法通过导航到 `~/Documents` 来读本地文件。（仅校验初始导航 URL；重定向不校验——见 SECURITY.md。）
+- **焦点守卫：** 给 `mac_type` / `mac_paste` / `mac_key` 传 `app`，当该应用不在最前时输入被拒。不传 `app` 则守卫关闭。
+- **失效元素保护：** `element_index` 不是坐标。它 30 秒后过期（`element_no_longer_valid`）；browser 点击会重校验 tag/name/href/位置，Mac 点击在 frontmost app 自快照以来变化时拒绝执行。
+- **防注入 JXA：** 字符串参数以 argv 数据传给 `osascript`，绝不拼进脚本；唯一的数值参数（AX 读取 depth）在使用前 clamp 到 1–10。有回归测试覆盖。
+
+这些开关不会让被观察的内容变得可信。截图文字、accessibility 名称、页面内容都是喂给你模型的**不可信输入**——别让它们覆盖任务指令。完整威胁模型与逐条状态见 [SECURITY.md](SECURITY.md)（英文）。
+
+## 工具与文档
+
+- 完整工具参考：[docs/tools.md](docs/tools.md)（由运行中的 server 自动生成）
+- 使用实操：[docs/usage.md](docs/usage.md)
+- 架构：[docs/architecture.md](docs/architecture.md)
+
+## 路线图
+
+- Browser：tab 管理（list / select / close）
+- 动作词汇（browser）：`hover`、`set_value`、`wait_for`
+- 快照 diffing 以降低 token 消耗——待测量
+
+## 维护
+
+个人维护，无 SLA。Mac 与 iPhone 层需 macOS 15+；Browser 层随 Playwright 跨平台。
+
+## 许可
+
+MIT——见 [LICENSE](LICENSE)。

package/SECURITY.md

@@ -0,0 +1,41 @@
+# Security Policy
+
+noracu gives an MCP client control over your **real machine** — mouse,
+keyboard, clipboard, browser, and (via iPhone Mirroring) your phone.
+Treat every tool call as privileged.
+
+## Threat model
+
+| # | Threat | Vector | Status | Mitigation / notes |
+|---|---|---|---|---|
+| 1 | **Prompt injection → arbitrary code execution** | Observed content (web page text, AX tree titles, OCR output) contains instructions that trick the agent into calling tools with malicious arguments | **Mitigated for JXA** | `mac_ax_state` / `mac_menu_command` use static scripts + `osascript` argv — string parameters are data, never script source; the one numeric parameter (AX read depth) is clamped before use (regression-tested in `test/e2e.mjs`) |
+| 2 | **Arbitrary local control** | `mac_click` / `mac_type` / `mac_paste` can operate password managers, banking pages, terminals | **Inherent** | Use client-side confirmation gates for high-stakes targets; see recommendations below. Optional `app` focus guard on `mac_type` / `mac_paste` / `mac_key` refuses input when another app is frontmost; `element_index` clicks additionally refuse (`element_no_longer_valid`) if the frontmost app changed since the snapshot the index came from. |
+| 3 | **Arbitrary JS in browser** | `browser_evaluate` executes any JavaScript in the page context | **Mitigated (kill switch)** | Set `NORACU_DISABLE_EVAL=1`; `browser_evaluate` then returns an error on every call (it stays listed in `tools/list`) |
+| 4 | **Input validation gaps** | Malformed arguments (NaN coordinates, oversized strings, `javascript:` URLs) | **Mitigated** | `validateArgs` runs on every tool call; tools with coordinate/enum/length/URL constraints are checked (others pass through) |
+| 5 | **Screenshot privacy** | `mac_screenshot` / `phone_screenshot` capture the full screen / phone display | **Inherent** | Run in a dedicated macOS user account or virtual display for untrusted automation |
+| 6 | **iPhone operations** | `phone_tap` / `phone_type` can interact with payment and SMS-verification UIs | **Inherent** | Consider disconnecting Apple Pay / keeping Mirroring off for untrusted sessions |
+| 7 | **Supply chain** | malicious PRs / dependency poisoning after open-sourcing | **Partially mitigated** | Review all PRs touching `src/server.mjs` argument handling; lockfile committed; no `postinstall` scripts; the only runtime download is the one-time Playwright `chromium` install triggered by `browser_launch`, which can be disabled with `NORACU_DISABLE_AUTO_INSTALL=1` |
+| 8 | **Local file read via browser** | `browser_navigate` to a `file://` URL, then `browser_read` exfiltrates local files | **Mitigated (default)** | `file://` blocked at the `browser_navigate` argument unless `NORACU_ALLOW_FILE_URL=1` (initial URL only; redirects fall under threat 9) |
+| 9 | **HTTP GET side effects** | navigation triggers logout links, one-time tokens, private-network / cloud-metadata endpoints | **Not mitigated** | No URL blocklist (redirects and DNS rebinding defeat pre-navigation checks); client-side policy |
+| 10 | **Clipboard clobbering by mac_paste** | `mac_paste` overwrites the clipboard with the pasted text | **Partially mitigated** | Previous text content is restored after ~150ms; non-text clipboard contents are lost; async paste targets may race |
+| 11 | **Untrusted observation content** | ARIA snapshots, AX trees, OCR, page text contain attacker-controlled text | **Not mitigated** | The MCP client must treat tool results as data, never as instructions |
+
+## Hardening recommendations
+
+- Set `NORACU_DISABLE_EVAL=1` unless you actively need `browser_evaluate`.
+- Gate `mac_type` / `mac_paste` behind human confirmation when the frontmost
+  app is a password manager or banking site (client-side policy).
+- Run untrusted automation in a dedicated macOS user account with
+  Accessibility + Screen Recording granted only to that account's client.
+- Audit tool calls in your MCP client's log — every action is visible there.
+- The persistent browser profile at `~/.noracu/browser-profile` keeps cookies
+  across sessions — clear it between untrusted runs.
+- `NORACU_READ_ONLY=1` does not block the one-time `playwright install
+  chromium` download triggered by `browser_launch`; set
+  `NORACU_DISABLE_AUTO_INSTALL=1` to require a manual install instead.
+
+## Reporting a vulnerability
+
+Please report security issues privately through a GitHub Security Advisory
+(repo → **Security** → **Advisories** → **Report a vulnerability**). Do not
+open public issues for exploitable bugs. We aim to respond within 72 hours.

package/docs/architecture.md

@@ -0,0 +1,74 @@
+# noracu Architecture
+
+> A unified three-layer computer-use control plane — Mac / Browser / iPhone — behind a single MCP server (stdio JSON-RPC 2.0).
+
+## Design principles
+
+1. **Observe/act separation**:
+   - Observe: `ax_state` (Accessibility/ARIA tree, token-cheap) or `screenshot` (visual), chosen per step
+   - Act: click / drag / scroll / key / type / paste / evaluate, by coordinates or element reference
+2. **Local-first**: no cloud, no API key — every capability comes from local tools
+3. **Layer autonomy**: each layer is largely independent; a missing dependency (e.g. imctl) disables only that layer. The one deliberate cross-layer path is the iPhone layer's Home / App Switcher, which prefer the iPhone Mirroring menu (driven through the Mac layer) and fall back to imctl gestures.
+
+The observe/act vocabulary follows the action/observation split popularized by ChatGPT's computer-use, re-implemented on open local tooling.
+
+## Layer implementations
+
+### Mac layer (9 tools)
+
+| Tool | Implementation |
+|---|---|
+| `mac_screenshot` | `screencapture -x` (region crop via `-R`; returns scale + logical-size + `screen:'main'` metadata) |
+| `mac_ax_state` | `osascript` JXA → System Events AX tree (role/title/frame, depth control 1-10) |
+| `mac_click/drag/scroll` | `cliclick` (c:/dd:dm:du:/sc:); click takes `[x,y]` or `element_index`, drag is index-only (`from_index`/`to_index`) — indexes come from the most recent `mac_ax_state` (TTL 30s; stale references raise `element_no_longer_valid`, as does a frontmost-app change since the snapshot) |
+| `mac_key` | `cliclick` — human key names parsed to an arg sequence: a chord like `cmd-shift-v` → `kd:cmd,shift t:v ku:cmd,shift`, a named key like `return` → `kp:return` |
+| `mac_type` | `cliclick t:<text>` (literal text) |
+| `mac_paste` | `pbcopy` writes the clipboard, then `cliclick kd:cmd t:v ku:cmd` (⌘V), then the previous clipboard is restored |
+| `mac_menu_command` | JXA traverses the target app's menu bar → `click` on the menu item (localization-safe) |
+
+`mac_key` / `mac_type` / `mac_paste` accept an optional `app`: when set, the tool verifies that app is frontmost before synthesizing input and returns `focus_lost` (sending nothing) if it is not.
+
+### Browser layer (12 tools)
+
+A single persistent Playwright 1.58 session (one browser, one context), held in a `Map` keyed by `browser_id` that carries one active entry at a time:
+
+- Session: `browser_launch` (headless optional) → `browser_navigate` → `browser_close`; `browser_snapshot` accepts an optional `browser_id`, but only one session is live at a time (true multi-session `browser_id` + `tab_id` is a roadmap item)
+- Observe: `browser_snapshot` (`page.ariaSnapshot()` ARIA tree YAML + list of interactive element indices), `browser_screenshot` (fullPage / format png|jpeg / quality / clip; returns DPR + viewport metadata)
+- Act: `browser_click` (element_index / selector / xy; element_index clicks run a tag/name/href/position TOCTOU check first), `browser_fill`, `browser_press_key`, `browser_type`, `browser_scroll`, `browser_evaluate`
+- Read: `browser_read` (page text, optional selector)
+
+### iPhone layer (14 tools)
+
+imctl subprocess (`python3 -m imctl.cli`, PYTHONPATH pointing at imctl/src) drives iPhone Mirroring. **Experimental**: when imctl or its Python dependencies (pyobjc/Quartz) are absent, most tools return a structured `imctl unavailable` error rather than crashing the server (`phone_home`/`phone_app_switcher` still work via the iPhone Mirroring menu).
+
+- Observe: `phone_ocr` (coordinates + text), `phone_screenshot` (captures the Mirroring window region), `phone_find` (locates text)
+- Act: `phone_tap/longpress/swipe` (normalized 0-1 coordinates), `phone_home` / `phone_app_switcher` (iPhone Mirroring menu first, imctl gesture fallback), `phone_scroll`, `phone_type` (clipboard-based, IME-safe), `phone_key`
+- Session: `phone_status`, `phone_launch`, `phone_open` (Spotlight + OCR + tap)
+
+## Configuration
+
+| Environment variable | Default | Description |
+|---|---|---|
+| `NORACU_IMCTL_DIR` | `../imctl` beside the repo root (`src/../../imctl`) | imctl checkout location |
+| `NORACU_CLICLICK` | `/opt/homebrew/bin/cliclick` | path to cliclick |
+| `NORACU_READ_ONLY` | unset (full control) | `=1` observation-only mode: a default-deny allowlist of 12 tools — the observe tools (`mac_screenshot`, `mac_ax_state`, `browser_snapshot`, `browser_screenshot`, `browser_read`, `phone_status`, `phone_screenshot`, `phone_ocr`, `phone_find`) plus browser session lifecycle (`browser_launch`, `browser_navigate`, `browser_close`). `browser_navigate` still issues a page-load request. All input-synthesis tools (click/drag/type/key/paste/tap/swipe/evaluate/…) are rejected |
+| `NORACU_DISABLE_EVAL` | unset (enabled) | `=1` makes `browser_evaluate` return an error on every call (kill switch for arbitrary JS execution; it stays listed) |
+| `NORACU_DISABLE_AUTO_INSTALL` | unset (auto-install) | `=1` stops `browser_launch` from auto-running `playwright install chromium` on a clean machine; returns an error prompting a manual install instead |
+| `NORACU_ALLOW_FILE_URL` | unset (blocked) | `=1` permits `browser_navigate` to open `file://` URLs (blocked by default to avoid local-file exfiltration via the browser layer) |
+
+## Known limitations (v0.1)
+
+- Mac-layer actions spawn subprocesses (cliclick/osascript); per-action latency is tens of milliseconds (subprocess spawn; not benchmarked in this release) — a resident FFI process could cut it (see roadmap)
+- No axState diffing (every observation is a full snapshot)
+- Single browser session (no tab-level management)
+- The iPhone layer requires iPhone Mirroring to be online; OCR accuracy depends on screen content
+- macOS-only (Mac/iPhone layers); the Browser layer is cross-platform
+
+## Roadmap (v2 candidates)
+
+- ~~element_index coordinate system~~ (landed in v0.1.0, both Mac and Browser layers); next: monotonic index reuse across snapshots (prerequisite for diffing)
+- axState diffing (incremental observation, saves tokens)
+- Browser tool gaps (select_text/set_value/hover/wait_for…) + tab management
+- koffi FFI resident process (direct CGEvent/AXUIElement calls)
+- Multi-session architecture (browser_id + tab_id)
+- Security hardening: sensitive-app foreground detection, action audit log

package/docs/tools.md

@@ -0,0 +1,61 @@
+# Tool reference
+
+> Auto-generated from the running server (`node scripts/gen-tools.mjs`) — always in sync with the implementation.
+> noracu v0.1.0 · 35 tools
+
+## Mac layer (computer-use)
+
+*cliclick · screencapture · JXA System Events*
+
+| Tool | Arguments | Description |
+|---|---|---|
+| `mac_screenshot` | region:object? | Capture the Mac screen as PNG base64, optionally cropped to region {x,y,w,h}. Returns {scale, logical_size, screen:'main'} for the main display; click coordinates are in points (pixels / scale). |
+| `mac_ax_state` | app:string?, depth:number? | Read the foreground app's accessibility tree (element index/role/title/value/frame; indexes feed mac_click/mac_drag for 30s); menu bar pruned by default (use mac_menu_command); optional app name and depth (default 3, clamped 1-10). Returns ax_empty if the app has no AX-visible window (not frontmost / another Space). |
+| `mac_click` | x:number?, y:number?, element_index:integer?, button:string?, count:number? | Click at {x,y} or on a UI element, with optional button (left\|right\|middle) and count. Use element_index from the latest mac_ax_state (valid 30s); returns element_no_longer_valid when stale. |
+| `mac_drag` | **from_index**:integer, **to_index**:integer | Drag from one UI element to another (move windows, sliders, selections). Use from_index/to_index from the latest mac_ax_state (valid 30s); returns element_no_longer_valid when stale. |
+| `mac_scroll` | direction:string?, pages:number? | Scroll the frontmost window up/down/left/right by a number of pages (default 1); moves the pointer over that window first. |
+| `mac_key` | key:string?, app:string? | Press a key or modifier chord, e.g. 'return', 'esc', 'cmd-c', 'cmd-shift-v', 'arrow-up'. Pass app to refuse input unless that app is frontmost (focus guard). |
+| `mac_type` | text:string?, app:string? | Type text at the current focus without modifier keys. Pass app to refuse input unless that app is frontmost (focus guard). Not IME-safe: with a CJK input method active, keystrokes may be re-segmented — use mac_paste for reliable text entry. |
+| `mac_paste` | text:string?, app:string? | Write text to the clipboard and paste it with Cmd+V. Pass app to refuse input unless that app is frontmost (focus guard). |
+| `mac_menu_command` | app:string?, command:string? | Run an app's menu bar command via accessibility traversal (locale-safe). |
+
+## Browser layer (browser-use)
+
+*Playwright*
+
+| Tool | Arguments | Description |
+|---|---|---|
+| `browser_launch` | headless:boolean? | Launch the browser with a persistent profile (headed by default; pass headless:true for headless). |
+| `browser_navigate` | url:string? | Navigate the current tab to an http(s) URL. file:// is disabled unless NORACU_ALLOW_FILE_URL=1. |
+| `browser_snapshot` | browser_id:string? | Return the page's ARIA accessibility snapshot plus a clickable element index list (indexes feed browser_click). |
+| `browser_screenshot` | fullPage:boolean?, format:string?, quality:number?, clip:object? | Screenshot the page with optional fullPage, format png\|jpeg, quality (1-100, jpeg) and clip {x,y,width,height}; returns the image plus DPR/viewport metadata. |
+| `browser_click` | element_index:integer?, selector:string?, xy:array?, button:string? | Click a page element, preferring element_index; selector is a fallback, xy pixel coordinates not recommended. Use element_index from the latest browser_snapshot (valid 30s); returns element_no_longer_valid when stale. |
+| `browser_fill` | selector:string?, value:string? | Fill an input field matched by selector; fill replaces the whole value (browser_type appends keystrokes). |
+| `browser_press_key` | key:string? | Press a key or chord, e.g. 'Enter' or 'Control+A'. |
+| `browser_type` | text:string? | Type text via the keyboard into the focused element; type appends keystrokes (browser_fill replaces the whole value). |
+| `browser_scroll` | direction:string?, pages:number? | Scroll the page up or down by a number of pages. |
+| `browser_read` | selector:string? | Read the page's visible text, optionally scoped to a selector. |
+| `browser_evaluate` | js:string? | Evaluate JavaScript in the page and return the result. |
+| `browser_close` | — | Close the browser. |
+
+## iPhone layer (device-use)
+
+*imctl / iPhone Mirroring*
+
+| Tool | Arguments | Description |
+|---|---|---|
+| `phone_status` | — | EXPERIMENTAL. Requires imctl (not yet public). Return the iPhone Mirroring session status as JSON. |
+| `phone_launch` | — | EXPERIMENTAL. Requires imctl (not yet public). Ensure iPhone Mirroring is running and brought to the front. |
+| `phone_screenshot` | — | EXPERIMENTAL. Requires imctl (not yet public). Screenshot the iPhone screen (the Mirroring window region). |
+| `phone_ocr` | — | EXPERIMENTAL. Requires imctl (not yet public). OCR the iPhone screen into text with coordinates. |
+| `phone_find` | text:string? | EXPERIMENTAL. Requires imctl (not yet public). Locate text on the iPhone screen. |
+| `phone_tap` | x:number?, y:number? | EXPERIMENTAL. Requires imctl (not yet public). Tap at {x, y} in normalized 0-1 coordinates. |
+| `phone_longpress` | x:number?, y:number?, duration:number? | EXPERIMENTAL. Requires imctl (not yet public). Long-press at {x, y} for an optional duration in seconds. |
+| `phone_swipe` | from:array?, to:array?, ms:number? | EXPERIMENTAL. Requires imctl (not yet public). Swipe from [x,y] to [x,y] over an optional duration in ms. |
+| `phone_home` | — | EXPERIMENTAL. Go to the iPhone Home Screen via the iPhone Mirroring menu (works without imctl when the mirror is connected); falls back to an imctl swipe. |
+| `phone_app_switcher` | — | EXPERIMENTAL. Open the iPhone App Switcher via the iPhone Mirroring menu (works without imctl when the mirror is connected); falls back to an imctl gesture. |
+| `phone_scroll` | direction:string?, distance:number? | EXPERIMENTAL. Requires imctl (not yet public). Scroll up or down by a distance. |
+| `phone_type` | text:string? | EXPERIMENTAL. Requires imctl (not yet public). Type text via the clipboard (IME-safe). |
+| `phone_key` | key:string? | EXPERIMENTAL. Requires imctl (not yet public). Press a key: return, delete, up, down, left, or right. |
+| `phone_open` | app:string? | EXPERIMENTAL. Requires imctl (not yet public). Open an app via Spotlight search plus OCR-guided tap. |
+