@anionex/dsh-computer-use 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 anionex
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.i18n.yaml

@@ -0,0 +1,6 @@
+# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
+# side as of the last confirmed-consistent state. Both languages carry equal authority;
+# after editing either side, bring the other along and re-record with:
+#   pnpm run verify-translation-pairing --write dsh-computer-use/README.md
+README.md: 0447b8679a6278ad42fbcf05beb10b66bf32fea8
+README.zh.md: 87a5b1556abd96fdf8250401b27dd4ee8a3a127e

package/README.md

@@ -0,0 +1,307 @@
+# DSH Computer Use
+
+[![X (Twitter)](https://img.shields.io/badge/-@anion__ex-000000?style=flat-square&logo=x&logoColor=white)](https://x.com/anion_ex)
+[![License: MIT](https://img.shields.io/badge/license-MIT-2f855a.svg)](LICENSE)
+![macOS](https://img.shields.io/badge/macOS-14%2B-111827.svg)
+![Universal binary](https://img.shields.io/badge/native-arm64%20%2B%20x86__64-2563eb.svg)
+![DeepSeek Harness](https://img.shields.io/badge/DeepSeek%20Harness-Bundle-5b50ed.svg)
+
+**Native macOS control for DeepSeek Harness that keeps your real cursor and foreground application alone by default; the Bundle may bring the target app forward before keyboard input for reliable typing.**
+
+DSH Computer Use gives an Agent fresh Accessibility observations, exact process/window targeting, stale-state rejection, scoped application access, and verified post-action state. Semantic Accessibility comes first; mouse, drag, wheel, and keyboard fallback are routed to the selected process instead of the global desktop.
+
+English | [中文](README.zh.md)
+
+## Why it is different
+
+Accessibility permission lets a process inspect and operate macOS UI elements, but the permission itself does not prevent focus stealing or cursor movement. Those behaviors depend on the input route.
+
+The default DSH Computer Use route is deliberately non-interfering:
+
+- **No system-cursor movement:** the helper contains no cursor-warp path.
+- **No global pointer injection:** click, scroll, and drag fallback use a pid/window-targeted SkyLight route, not the global HID event stream.
+- **No pointer-triggered activation:** semantic Accessibility, process-targeted pointer input, and `keyboardPolicy: preserve` run without activation; `keyboardPolicy: activate` (Bundle default) brings the target app forward before keyboard fallback, matching Codex Computer Use.
+- **A separate Agent cursor:** click, scroll, and drag actions animate a click-through, nonactivating software cursor while the macOS system cursor remains untouched. It is visible by default and stays at the action position until the bound window changes or a hide command; `cursorAutoHideMs` can opt into timed auto-hide.
+- **No blind replay:** every action is tied to an exact, unexpired observation and returns fresh state.
+
+The result is a native action layer that can operate many background applications while the user continues working in the current foreground application.
+
+## What it adds
+
+- **Observe before acting.** Return a bounded Accessibility tree, indexed elements, exact app/process/window metadata, permission state, and an optional screenshot Artifact.
+- **Bind actions to state.** Every element exposes an observation-local index and opaque `targetHandle`; exact lookup remains compatible, while explicitly allowed rebinding accepts only a unique native or semantic identity inside the same process and window.
+- **Prefer semantic input.** Use `AXPress`, editable values, selected-text assignment, and advertised Accessibility actions before pointer fallback.
+- **Route fallback to the target.** Keyboard input goes to the selected pid; pointer input goes to the selected pid and `CGWindowID` with window-local coordinates, resolving the app window under the point so arbitrary screen coordinates work.
+- **Return fresh evidence.** Every successful action settles for a bounded interval and returns a new full or diff observation.
+- **Scope application access.** Read and control leases are separated by Agent, Session, turn, and exact bundle id; high-impact actions require one-use confirmation.
+- **Keep the model surface focused.** Execution Tools appear only after the current Agent loads the Computer Use Skill.
+
+## Proof: a never-active background fixture
+
+The repository includes a deterministic AppKit fixture and a universal native helper. Release tests start the fixture with `open -g` in background mode, then use the same protocol exposed to the Agent.
+
+```text
+observe exact bundle id + pid
+-> element: "Targeted pointer probe", no AXPress action
+-> computer_click with observationId + element index + allowCoordinateFallback
+-> fresh observation
+-> activation "not-requested"; pointerRouting "target-process"
+-> status "Status: pointer click"
+```
+
+<p align="center">
+  <img src="assets/computer-use-fixture.png" width="760" alt="The never-active deterministic native fixture before target-process pointer input, showing the dedicated pointer probe and ready status." />
+</p>
+
+The fixture records every `applicationDidBecomeActive` callback. An independent native monitor also samples the system cursor and frontmost pid every millisecond throughout click, scroll, and drag. The default release path must not increase `activationCount`; it also requires unchanged cursor coordinates, an unchanged frontmost pid, exact click/scroll counts, and one complete down/up drag gesture.
+
+See [Foreground-safe input policy](docs/interaction-policy.md) for the requirements, architecture, decisions, evidence, and compatibility limits.
+
+## Scope
+
+`dsh-computer-use` is the native **action layer**. It does not replace narrower interfaces:
+
+- browser tasks should use browser automation and DOM/CDP state;
+- APIs, CLIs, and purpose-built application plugins remain preferable when available;
+- OCR, visual grounding, and pixel interpretation should use the separately installed `dsh-vision-toolkit`: load the `vision-tools` Skill and pass the exact screenshot Artifact path to `vision_glance`, `vision_ground`, `vision_detect`, `vision_crop`, or `vision_long_screenshot_ocr`; do not replace those tools with shell-driven `tesseract`, `screencapture`, or ad hoc Swift/Python OCR;
+- domain bundles such as `dsh-design` can compose Computer Use when a workflow crosses into a native application.
+
+## Quick start
+
+### Prerequisites
+
+- macOS 14 or newer.
+- DeepSeek Harness with a Web or Headless Profile and the Skill Tool mounted.
+- macOS Accessibility permission for observation and native actions.
+- macOS Screen Recording permission only when a screenshot is requested.
+- Node.js `^22.19.0` or `>=24.0.0` when building this repository.
+
+Install the Web and Headless bundles directly from npm:
+
+```sh
+dsh plugin --profile web add @anionex/dsh-computer-use
+dsh plugin --profile headless add @anionex/dsh-computer-use
+
+dsh --profile web --dump-config | grep computer-use
+dsh --profile headless --dump-config | grep computer-use
+```
+
+For local development, replace the package name with an absolute checkout path.
+
+Restart a running `dsh web` host after changing the installed plugin, then start a new Session so the host reloads the Bundle and Skill catalog.
+
+Load the Skill in that Session:
+
+```text
+/computer-use
+```
+
+Then try:
+
+> Use Computer Use to inspect the running DSH Computer Use Fixture, enable its deterministic option, and report the fresh status. Prefer Accessibility elements and do not reuse an old observation.
+
+## How it works
+
+```mermaid
+flowchart LR
+    A["Select exact bundle id and pid"] --> B["Acquire scoped read access"]
+    B --> C["Observe AX tree and optional screenshot"]
+    C --> D["Choose target handle, index, or window-relative point"]
+    D --> E["Acquire control and optional one-use confirmation"]
+    E --> F["Re-observe and validate exact target"]
+    F --> G{"Input route"}
+    G -->|"Semantic"| H["Accessibility action or value"]
+    G -->|"Keyboard"| I["Post to target pid"]
+    G -->|"Pointer"| J["Post to target pid + window"]
+    H --> K["Wait for settlement"]
+    I --> K
+    J --> K
+    K --> L["Return fresh full or diff observation"]
+```
+
+Every observed element has an observation-local compatibility index and an opaque `targetHandle`. Index-only actions retain exact locator behavior. A low-risk element action may pass `targetHandle` with `allowRebind: true`; immediately before input, the provider-independent resolver obtains fresh Accessibility state and checks, in order, the original locator, a unique provider-native identifier such as macOS `AXIdentifier`, then one unique semantic match over role, accessible name, advertised actions, and stable ancestor fingerprint. The resolver keeps the exact bundle id, pid, and selected-window identity, and fails closed with `COMPUTER_TARGET_AMBIGUOUS` or `COMPUTER_TARGET_LOW_CONFIDENCE` instead of guessing. Coordinate actions still require the complete referenced window state to remain current.
+
+Successful element actions report `resolution.mode`, `confidence`, `candidateCount`, and `targetChanged`. A sensitive target that needs rebinding invalidates the prior one-use confirmation and returns `COMPUTER_TARGET_REBIND_REQUIRES_CONFIRMATION`; the caller must observe the current UI and confirm the newly selected handle. Visual coordinates are not target handles and never authorize sensitive rebinding. Provider-native visual hit-testing is not part of this foundation release and remains follow-up work.
+
+The default interaction policy is:
+
+```yaml
+interaction:
+  focusPolicy: preserve
+  keyboardPolicy: activate
+  pointerInputPolicy: targeted
+  cursorVisualization: visible
+  cursorMotionMs: 180
+  cursorAutoHideMs: 0
+```
+
+`cursorVisualization: visible` displays the Agent's own non-interactive cursor for click, scroll, and drag. It never replaces or moves the macOS system cursor. Set it to `hidden` when visual feedback is unwanted. `pointerInputPolicy: deny` disables coordinate click/fallback, scroll, and drag. `keyboardPolicy: activate` (Bundle default) makes `type-text` keyboard fallback and `press-key` reliable by activating the target app first; `focusPolicy: activate` is the broader compatibility mode that also activates before pointer input. After activation, the helper re-observes and revalidates the exact target before input.
+
+The cursor is a 28x28 transparent whole-image cursor (Cursor arrow plus DeepSeek whale, `assets/cursor.png`) with the hotspot at the image's top-left corner. It is a separate process, click-through, nonactivating, and bound to the exact observed pid, window, and frame so it disappears if the target window closes, moves, resizes, or is minimized.
+
+The helper executable is an internal DSH transport rather than a public authorization API. It requires an isolated process group plus parent-owned standard transports, so ordinary shell redirection fails closed before command parsing. This is defense in depth, not authentication against arbitrary code running as the same macOS user: a deliberately constructed detached parent can reproduce that transport topology. Use the registered Tools so application leases, sensitive-action confirmation, and host policy checks remain in force; `danger-full-access` must not be treated as protection against direct native invocation.
+
+Successful action results include:
+
+```ts
+activation: 'not-requested' | 'already-frontmost' | 'activated'
+pointerInput: boolean
+pointerRouting: 'none' | 'target-process'
+resolution?: {
+  mode: 'exact-locator' | 'native-identifier' | 'semantic-rebind'
+  confidence: number
+  candidateCount: number
+  targetChanged: boolean
+}
+```
+
+The model cannot override these host policies through Tool arguments.
+
+## Model Tools
+
+The Bundle initially contributes only `computer_use_activate`. Loading the Skill exposes the focused execution vocabulary for that Agent.
+
+<details>
+<summary>Show the complete Tool vocabulary</summary>
+
+| Tool | Purpose |
+|---|---|
+| `computer_list_apps` | List bounded user-facing applications with bundle id, pid, frontmost state, and permission diagnostics |
+| `computer_observe` | Return a fresh full/diff Accessibility observation and optional screenshot Artifact |
+| `computer_click` | Prefer `AXPress`; accept an exact index or opaque target handle, with optional safe rebinding, before target-process coordinate fallback |
+| `computer_set_value` | Set or clear an editable Accessibility value through an exact index or opaque target handle without using the clipboard |
+| `computer_type_text` | Insert Unicode through Accessibility when supported, with a process-targeted keyboard fallback |
+| `computer_press_key` | Send one key from a finite vocabulary to the selected process, with optional modifiers |
+| `computer_scroll` | Send bounded directional scrolling to the selected process and window at a resolved element or window/screen coordinate |
+| `computer_drag` | Drag between two window/screen points in the referenced observation |
+| `computer_perform_action` | Execute one Accessibility action advertised by an exact or safely rebound selected element |
+| `computer_wait` | Poll one bounded text/role/title condition and return fresh state without modifying the app |
+| `computer_confirm` | Obtain a one-use token bound to one exact sensitive action |
+
+No Tool accepts AppleScript, JXA, shell, Swift, Objective-C, native selectors, arbitrary Accessibility constants, or source code.
+
+</details>
+
+## Observation, permissions, and sensitive actions
+
+An observation contains an opaque id and expiry, exact app identity, frontmost/window metadata, bounded tree text, current elements with opaque target handles, optional screenshot metadata, and permission state. Target handles expose no provider object reference or native identifier. Secure text values are emitted as `[secure]`; they do not enter target descriptors, tree text, Tool results, screenshot metadata, or native errors. A screenshot can still contain other visible application data and should be treated as sensitive.
+
+The technical access model has two exact-bundle-id leases:
+
+- `read`: inspect Accessibility state and a requested screenshot;
+- `control`: send UI input to the selected application.
+
+Without a configured grant, DSH asks for approval. Read approval lasts for the Session; control approval lasts for the current turn. A user rejection is final for that app and scope for the rest of the Session.
+
+The Bundle keeps Session-wide read grants and rejected app/scope decisions in its own `computer_use_state` storage-domain sidecar, fenced by the Session header's `createdAt` and `cwd`. It does not add Computer Use events to the official Session log or modify DSH Core. The Web Profile already composes `@deepseek-ai/dsh-storage-domain`; a custom Profile must compose it before this Bundle if interactive read grants or durable rejections are needed. Exact grants configured in Settings remain available without storage-domain, and an allowed control decision remains process-local for the current turn. When a durable interactive decision cannot be stored, the operation fails clearly instead of silently weakening its lifetime.
+
+The DSH `danger-full-access` preset uses `approval/policy: never`, so an ungranted app is policy-blocked before any prompt. The plugin reports an actionable `COMPUTER_PERMISSION_REQUIRED` error and does not record that outcome as a user rejection. Add the exact bundle id in Computer Use Settings or use a preset whose approval policy is `ask`.
+
+High-impact communication, sensitive-data transmission, irreversible deletion, account/security/privacy changes, unrequested installation, legal acceptance, and financial completion beyond explicit authorization require `computer_confirm` immediately before execution. The token is short-lived, one-use, and bound to the exact app, process, observation, target handle, and action. Grants do not bypass it. If resolution moves beyond the exact locator, the token is invalidated and a fresh observation plus confirmation is required.
+
+## macOS permissions and native integrity
+
+The Web Settings section reports helper integrity, Accessibility and Screen Recording status, active generation, interaction policy, limits, and exact application grants. Its buttons can open the relevant macOS privacy pane after a user click; the plugin cannot grant TCC permission itself.
+
+Accessibility and Screen Recording are UI permissions, not filesystem permissions. Normal use stays under DSH `workspace-write`: screenshots remain in the Session workspace, transient files use Session-private temporary storage, and the Bundle does not require `danger-full-access`.
+
+The committed helper is an ad-hoc-signed universal `arm64` + `x86_64` binary targeting macOS 14 or newer. [`native/macos/manifest.json`](native/macos/manifest.json) pins its SHA-256, source digest, architectures, and deployment target. `pnpm run check:native` also checks the target-process-only pointer route and rejects system-cursor warp or global pointer-post symbols.
+
+## Configuration
+
+<details>
+<summary>Show Bundle configuration fields</summary>
+
+| Field | Purpose |
+|---|---|
+| `observationTtlMs` | Lifetime of an observation before reuse is rejected; default `0` disables expiry, or set any value up to `86400000` ms (24 hours) |
+| `confirmationTtlMs` | Lifetime of a one-use sensitive-action confirmation |
+| `actionTimeoutMs` | Hard native action timeout from `1000` to `120000` ms |
+| `settleMs` | Interval between post-action state checks from `0` to `10000` ms |
+| `maxSettleMs` | Maximum post-action settlement budget from `100` to `60000` ms |
+| `maxNodes` / `maxDepth` / `maxTextBytes` | Accessibility traversal and model-visible text bounds |
+| `maxScreenshotBytes` | Maximum PNG Artifact size |
+| `artifactRoot` | Workspace-relative screenshot directory |
+| `helper.path` | Optional explicit external helper executable |
+| `helper.allowSourceBuild` | Permit an explicit managed-source rebuild when the committed helper is absent; default `false` |
+| `interaction.focusPolicy` | `preserve` (default) avoids target-app activation; `activate` explicitly permits it and requires re-observation/revalidation |
+| `interaction.keyboardPolicy` | `preserve` keeps keyboard events routed without activation; `activate` (Bundle default) activates the target app before keyboard fallback |
+| `interaction.pointerInputPolicy` | `targeted` (default) permits pid/window-targeted pointer input; `deny` disables click fallback, scroll, and drag |
+| `interaction.cursorVisualization` | `visible` (default) shows the separate Agent cursor; `hidden` disables only the overlay |
+| `interaction.cursorMotionMs` | Animated Agent-cursor travel duration, default `180` ms |
+| `interaction.cursorAutoHideMs` | Idle time before the Agent cursor hides; default `0` keeps it visible until the bound window changes or a hide command, or set a finite value up to `30000` ms |
+| `allowAllApps` | Grant `read` and `control` to every running app; default `false`. When enabled, exact `grants` are ignored |
+| `grants` | Exact non-wildcard bundle-id read/control policy; `control: true` implies read |
+
+</details>
+
+Settings updates replace the active provider generation only after validation and health checks pass. Replacement invalidates existing observations and pending confirmations.
+
+## Status and limitations
+
+- Status: early `0.1.0`; model-facing and provider behavior may change before a stable release.
+- The current provider is macOS-only. Windows UI Automation and Linux providers are not implemented.
+- Target-process pointer delivery uses dynamically resolved SkyLight SPI. If it is unavailable, pointer fallback fails closed rather than switching to global input.
+- The clicked point must fall inside an on-screen window of the selected app; the helper resolves the topmost matching window so ambiguous frame/title matches no longer block coordinate actions. Minimized, hidden, or windowless targets fail closed.
+- Custom canvases, games, hardened input surfaces, and future macOS releases may reject target-process pointer or keyboard events. Prefer semantic Accessibility whenever possible.
+- `focusPolicy: activate` and `keyboardPolicy: activate` are intentionally disruptive and exist as operator-selected compatibility modes.
+- A target application may change its own activation or focus as a side effect of an accepted action.
+- The package captures requested discrete observations, not a live desktop feed.
+- Browser work should continue to use browser automation because DOM/CDP state is narrower and more precise.
+- The public npm package installs into both Web and Headless profiles as `@anionex/dsh-computer-use`.
+
+## Development and release verification
+
+Place this repository beside a DeepSeek Harness checkout so TypeScript and Vitest resolve the exact DSH peer declarations and runtime modules:
+
+```text
+workspace/
+├── packages/
+├── vendor/
+└── dsh-computer-use/
+```
+
+Then run:
+
+```sh
+pnpm install --frozen-lockfile
+pnpm run build
+DSH_COMPUTER_USE_REQUIRE_TCC=1 pnpm test
+pnpm run check:native
+pnpm pack --dry-run
+pnpm run validate
+```
+
+`pnpm run validate` runs the keyless local and clean Web/Headless Profile lanes. The real-model release lane needs `DEEPSEEK_API_KEY` and accepts an optional `DEEPSEEK_BASE_URL`:
+
+```sh
+pnpm run validate:model
+# or keyless validation followed by the real-model lane
+pnpm run validate:release
+```
+
+## Removal
+
+```sh
+dsh plugin --profile web remove @anionex/dsh-computer-use
+dsh plugin --profile headless remove @anionex/dsh-computer-use
+```
+
+Removing or disabling the Bundle unregisters the Skill and Tools, cancels helper work, releases process-local Agent observations, turn control grants, and confirmations, closes its storage-domain handle, and removes Web contributions. Existing screenshot files and the plugin-owned `computer_use_state` sidecar remain for explicit user cleanup.
+
+## Security, community, and support
+
+- Report suspected vulnerabilities privately through [SECURITY.md](SECURITY.md).
+- Read [CONTRIBUTING.md](CONTRIBUTING.md) before changing code or documentation.
+- Use [SUPPORT.md](SUPPORT.md) for installation, permission, configuration, and workflow questions.
+- Follow the [Code of Conduct](CODE_OF_CONDUCT.md) in project spaces.
+- See [CHANGELOG.md](CHANGELOG.md) for release history.
+- See [FUNDING.md](FUNDING.md) to support maintenance without purchasing roadmap control or private support.
+
+## About
+
+DSH Computer Use is maintained by [anionex](https://anionex.me/). If you would like to follow my future work, [follow me on X](https://x.com/anion_ex) or [GitHub](https://github.com/Anionex).
+
+## License
+
+[MIT](LICENSE) © 2026 anionex.

package/README.zh.md

@@ -0,0 +1,307 @@
+# DSH Computer Use
+
+[![X (Twitter)](https://img.shields.io/badge/-@anion__ex-000000?style=flat-square&logo=x&logoColor=white)](https://x.com/anion_ex)
+[![MIT License](https://img.shields.io/badge/license-MIT-2f855a.svg)](LICENSE)
+![macOS](https://img.shields.io/badge/macOS-14%2B-111827.svg)
+![Universal binary](https://img.shields.io/badge/native-arm64%20%2B%20x86__64-2563eb.svg)
+![DeepSeek Harness](https://img.shields.io/badge/DeepSeek%20Harness-Bundle-5b50ed.svg)
+
+**为 DeepSeek Harness 提供原生 macOS 控制能力，默认不碰你的真实光标，也不因指针动作抢占前台；Bundle 可以在键盘输入前把目标应用带到前台，保证输入可靠。**
+
+DSH Computer Use 为 Agent 提供新鲜的 Accessibility observation、准确进程/窗口定向、stale state 拒绝、按应用限制的访问，以及动作后的可验证状态。语义化 Accessibility 始终优先；鼠标、拖拽、滚轮与键盘 fallback 会投递给选定进程，而不是全局桌面。
+
+[English](README.md) | 中文
+
+## 为什么它不同
+
+Accessibility 权限允许进程读取和操作 macOS UI 元素，但这个权限本身并不会自动防止抢焦点或移动光标。是否干扰用户，取决于输入事件走哪条路由。
+
+DSH Computer Use 的默认路由有意避免干扰：
+
+- **不移动系统光标：** helper 中没有 cursor warp 路径。
+- **不做全局指针注入：** click、scroll 与 drag fallback 使用 pid/window 定向的 SkyLight 路由，不进入全局 HID 事件流。
+- **指针动作不激活应用：** 语义化 Accessibility、目标进程指针输入与 `keyboardPolicy: preserve` 都不激活；`keyboardPolicy: activate`（Bundle 默认）在键盘 fallback 前把目标应用带到前台，与 Codex Computer Use 对齐。
+- **独立 Agent 光标：** click、scroll 与 drag 动作会移动一个点击穿透、不会激活应用的软件光标，同时保持系统真实光标不变。默认显示，并停留在动作落点，直到绑定窗口变化或收到 hide 命令；`cursorAutoHideMs` 可开启定时自动隐藏。
+- **不盲目重放：** 每个动作都绑定准确、未过期的 observation，并返回新鲜状态。
+
+因此，这个原生动作层可以在用户继续使用当前前台应用时，操作许多后台应用。
+
+## 它补充了什么
+
+- **先观察再动作。** 返回有界 Accessibility tree、带 index 的元素、准确 app/process/window metadata、权限状态和可选截图 Artifact。
+- **把动作绑定到状态。** 每个元素都带有 observation-local index 与 opaque `targetHandle`；准确 locator 继续兼容，而显式允许的重绑定只接受同一进程和窗口内唯一的 native 或 semantic 身份。
+- **优先语义输入。** 先使用 `AXPress`、可编辑 value、selected-text 赋值和元素声明的 Accessibility action，再考虑指针 fallback。
+- **把 fallback 投递给目标。** 键盘输入发给选定 pid；指针输入携带窗口本地坐标，发给选定 pid 和 `CGWindowID`，并通过点解析应用窗口，任意屏幕坐标都可以点击。
+- **返回新鲜证据。** 每个成功动作都会经过有界 settle，并返回新的完整或差分 observation。
+- **按应用限制访问。** read/control lease 按 Agent、Session、turn 与准确 bundle id 分离；高影响动作另需一次性确认。
+- **保持模型表面聚焦。** 只有当前 Agent 加载 Computer Use Skill 后才暴露执行 Tool。
+
+## 证据：从未激活的后台 fixture
+
+仓库包含确定性 AppKit fixture 和 universal native helper。发布测试会通过 `open -g` 以后台模式启动 fixture，再通过 Agent 使用的同一协议完成操作。
+
+```text
+observe exact bundle id + pid
+-> element: "Targeted pointer probe", no AXPress action
+-> computer_click with observationId + element index + allowCoordinateFallback
+-> fresh observation
+-> activation "not-requested"; pointerRouting "target-process"
+-> status "Status: pointer click"
+```
+
+<p align="center">
+  <img src="assets/computer-use-fixture.png" width="760" alt="目标进程指针输入前、从未激活的确定性原生 fixture，显示专用 pointer probe 与 ready 状态。" />
+</p>
+
+Fixture 会记录每次 `applicationDidBecomeActive` 回调。独立 native monitor 还会在 click、scroll 与 drag 整个动作期间每毫秒采样系统光标和前台 pid。默认发布路径不得增加 `activationCount`，并要求光标坐标不变、前台 pid 不变、click/scroll 精确计数，并且 drag 只有一组完整 down/up gesture。
+
+需求、架构、关键决策、验证证据和兼容性边界见[前台安全输入策略](docs/interaction-policy.md)。
+
+## 范围
+
+`dsh-computer-use` 是原生**动作层**，不会取代更窄的接口：
+
+- 浏览器任务应继续使用 browser automation 和 DOM/CDP 状态；
+- 有 API、CLI 或专用应用插件时仍应优先使用；
+- OCR、视觉 grounding 与像素理解应交给独立安装的 `dsh-vision-toolkit`：加载 `vision-tools` Skill，把准确的截图 Artifact 路径传给 `vision_glance`、`vision_ground`、`vision_detect`、`vision_crop` 或 `vision_long_screenshot_ocr`；不要用 shell 驱动的 `tesseract`、`screencapture` 或临时 Swift/Python OCR 取代这些工具；
+- `dsh-design` 等领域 Bundle 可以在工作流跨入原生应用时组合 Computer Use。
+
+## 快速开始
+
+### 前置条件
+
+- macOS 14 或更新版本。
+- 已安装 Web 或 Headless Profile、并挂载 Skill Tool 的 DeepSeek Harness。
+- 用于观察和原生动作的 macOS Accessibility 权限。
+- 只有请求截图时才需要 macOS Screen Recording 权限。
+- 构建本仓库时需要 Node.js `^22.19.0` 或 `>=24.0.0`。
+
+直接从 npm 一键安装 Web 与 Headless Bundle：
+
+```sh
+dsh plugin --profile web add @anionex/dsh-computer-use
+dsh plugin --profile headless add @anionex/dsh-computer-use
+
+dsh --profile web --dump-config | grep computer-use
+dsh --profile headless --dump-config | grep computer-use
+```
+
+本地开发时，把包名替换为 checkout 的绝对路径即可。
+
+修改已安装插件后，需要重启正在运行的 `dsh web` host，再创建一个新 Session，让 host 重新载入 Bundle 与 Skill catalog。
+
+在新 Session 中加载 Skill：
+
+```text
+/computer-use
+```
+
+然后可以尝试：
+
+> 使用 Computer Use 检查正在运行的 DSH Computer Use Fixture，启用 deterministic option，并根据动作后返回的新状态报告结果。优先使用 Accessibility 元素，不要复用旧 observation。
+
+## 工作原理
+
+```mermaid
+flowchart LR
+    A["Select exact bundle id and pid"] --> B["Acquire scoped read access"]
+    B --> C["Observe AX tree and optional screenshot"]
+    C --> D["Choose target handle, index, or window-relative point"]
+    D --> E["Acquire control and optional one-use confirmation"]
+    E --> F["Re-observe and validate exact target"]
+    F --> G{"Input route"}
+    G -->|"Semantic"| H["Accessibility action or value"]
+    G -->|"Keyboard"| I["Post to target pid"]
+    G -->|"Pointer"| J["Post to target pid + window"]
+    H --> K["Wait for settlement"]
+    I --> K
+    J --> K
+    K --> L["Return fresh full or diff observation"]
+```
+
+每个已观察元素都有 observation-local 兼容 index 和 opaque `targetHandle`。仅传 index 的动作保留准确 locator 行为。低风险元素动作可以传入 `targetHandle` 与 `allowRebind: true`；输入前，provider-independent resolver 会获取新鲜 Accessibility 状态，并依次检查原 locator、唯一 provider-native identifier（例如 macOS `AXIdentifier`），以及基于 role、accessible name、已声明 action 和稳定 ancestor fingerprint 的唯一 semantic match。Resolver 始终保留准确 bundle id、pid 与选定窗口身份；遇到歧义或低置信度时返回 `COMPUTER_TARGET_AMBIGUOUS` 或 `COMPUTER_TARGET_LOW_CONFIDENCE`，不会猜测。坐标动作仍要求引用窗口的完整状态保持当前。
+
+成功的元素动作会报告 `resolution.mode`、`confidence`、`candidateCount` 与 `targetChanged`。敏感目标一旦需要 rebind，旧的一次性 confirmation 会失效，并返回 `COMPUTER_TARGET_REBIND_REQUIRES_CONFIRMATION`；调用方必须观察当前 UI，并对新选择的 handle 再次确认。视觉坐标不是 target handle，不能授权敏感 rebind。本基础版本不包含 provider-native visual hit-test，该能力留待后续实现。
+
+默认 interaction policy 为：
+
+```yaml
+interaction:
+  focusPolicy: preserve
+  keyboardPolicy: activate
+  pointerInputPolicy: targeted
+  cursorVisualization: visible
+  cursorMotionMs: 180
+  cursorAutoHideMs: 0
+```
+
+`cursorVisualization: visible` 会在 click、scroll 与 drag 时显示 Agent 自己的非交互光标；它不会替代或移动 macOS 系统光标。不需要视觉反馈时可设为 `hidden`。`pointerInputPolicy: deny` 会禁用坐标点击/fallback、滚动与拖拽。`keyboardPolicy: activate`（Bundle 默认）会先激活目标应用，让 `type-text` 键盘 fallback 与 `press-key` 可靠工作；`focusPolicy: activate` 是更宽的兼容模式，还会在指针输入前激活。激活后 helper 会重新观察并校验准确目标，之后才发出输入。
+
+Agent 光标是 28x28 的透明整图光标（Cursor 箭头加 DeepSeek 鲸鱼，`assets/cursor.png`），热点位于图片左上角。它由独立进程运行，点击穿透、不激活应用，并绑定准确已观察的 pid、窗口与 frame；目标窗口关闭、移动、缩放或最小化时会自动隐藏。
+
+Helper executable 是 DSH 内部传输实现，不是公共授权 API。它要求独立进程组和父进程持有的标准传输，因此普通 shell 重定向会在解析命令前 fail closed。这个检查只属于纵深防御，不会认证同一 macOS 用户下运行的任意代码：专门构造的 detached 父进程仍能复现这类传输拓扑。应通过已注册 Tool 使用该能力，以保留应用 lease、敏感动作 confirmation 与宿主策略检查；不能把 `danger-full-access` 当作阻止直接 native 调用的保护。
+
+成功动作结果包括：
+
+```ts
+activation: 'not-requested' | 'already-frontmost' | 'activated'
+pointerInput: boolean
+pointerRouting: 'none' | 'target-process'
+resolution?: {
+  mode: 'exact-locator' | 'native-identifier' | 'semantic-rebind'
+  confidence: number
+  candidateCount: number
+  targetChanged: boolean
+}
+```
+
+模型不能通过 Tool 参数覆盖这些宿主策略。
+
+## 模型 Tool
+
+Bundle 初始只贡献 `computer_use_activate`。加载 Skill 后，才为当前 Agent 暴露聚焦的执行 vocabulary。
+
+<details>
+<summary>查看完整 Tool 列表</summary>
+
+| Tool | 用途 |
+|---|---|
+| `computer_list_apps` | 列出有界用户应用及 bundle id、pid、前台状态和权限诊断 |
+| `computer_observe` | 返回新鲜的 full/diff Accessibility observation 与可选截图 Artifact |
+| `computer_click` | 优先使用 `AXPress`；接受准确 index 或 opaque target handle，并可在目标进程坐标 fallback 前执行安全 rebind |
+| `computer_set_value` | 通过准确 index 或 opaque target handle 设置或清空可编辑 Accessibility value，不使用剪贴板 |
+| `computer_type_text` | 支持时通过 Accessibility 插入 Unicode，否则使用进程定向键盘 fallback |
+| `computer_press_key` | 向选定进程发送有限词表中的按键，并支持可选 modifier |
+| `computer_scroll` | 在已解析元素或窗口/屏幕坐标处向选定进程与窗口发送有界方向滚动 |
+| `computer_drag` | 在引用 observation 的窗口/屏幕两点之间拖拽 |
+| `computer_perform_action` | 执行准确或安全 rebind 后的选定元素所声明的 Accessibility action |
+| `computer_wait` | 轮询一个有界 text/role/title 条件，不修改应用并返回新鲜状态 |
+| `computer_confirm` | 获取绑定准确敏感动作的一次性 token |
+
+任何 Tool 都不接受 AppleScript、JXA、shell、Swift、Objective-C、native selector、任意 Accessibility constant 或源码。
+
+</details>
+
+## Observation、权限与敏感动作
+
+Observation 包含 opaque id 与过期时间、准确 app 身份、frontmost/window metadata、有界 tree text、带 opaque target handle 的当前元素、可选截图 metadata 和权限状态。Target handle 不暴露 provider object reference 或 native identifier。Secure text value 以 `[secure]` 输出，不会进入 target descriptor、tree text、Tool result、截图 metadata 或 native error。截图仍可能包含应用中其他可见数据，应按敏感数据处理。
+
+技术访问模型包含两类准确 bundle-id lease：
+
+- `read`：读取 Accessibility 状态和请求的截图；
+- `control`：向选定应用发送 UI 输入。
+
+没有配置 grant 时，DSH 会请求 approval。Read approval 在 Session 内有效，control approval 只在当前 turn 有效。用户拒绝后，该 app/scope 在当前 Session 内保持最终结果。
+
+Bundle 把 Session 级 read grant 和被拒绝的 app/scope 决定保存在自己拥有的 `computer_use_state` storage-domain sidecar 中，并用 Session header 的 `createdAt` 与 `cwd` 绑定准确生命周期；它不会向官方 Session 日志新增 Computer Use event，也不会修改 DSH Core。Web Profile 已经组合 `@deepseek-ai/dsh-storage-domain`；自定义 Profile 如果需要交互式 read grant 或持久拒绝状态，必须在本 Bundle 之前组合它。Settings 中配置的准确 grant 在没有 storage-domain 时仍可使用；获准的 control 决定只保存在当前进程的当前 turn。需要持久化的交互决定如果无法保存，操作会明确失败，不会静默缩短或削弱授权生命周期。
+
+DSH `danger-full-access` preset 使用 `approval/policy: never`，因此未授权应用会在弹窗前被策略阻断。插件返回可操作的 `COMPUTER_PERMISSION_REQUIRED` 错误，并且不会把它记录成用户拒绝。请在 Computer Use Settings 中添加准确 bundle id，或改用 approval policy 为 `ask` 的 preset。
+
+高影响外部通信、敏感数据传输、不可逆删除、账户/安全/隐私变更、未经请求的安装、法律条款接受，以及超出明确授权的财务完成动作，都需要在执行前立即调用 `computer_confirm`。Token 有短 TTL、只能使用一次，并绑定准确 app、process、observation、target handle 与 action；grant 不能绕过它。只要 resolution 不再使用原 locator，该 token 就会失效，必须重新观察并再次确认。
+
+## macOS 权限与 native 完整性
+
+Web Settings 分区展示 helper 完整性、Accessibility 与 Screen Recording 状态、当前 generation、interaction policy、限制和准确应用 grant。只有用户点击后，按钮才会打开相关 macOS 隐私页面；插件不能自行授予 TCC 权限。
+
+Accessibility 与 Screen Recording 是 UI 权限，不是文件系统权限。正常使用保持在 DSH `workspace-write` 下：截图留在 Session workspace，临时文件使用 Session 私有临时目录，Bundle 不要求 `danger-full-access`。
+
+仓库提交的 helper 是最低支持 macOS 14、ad-hoc 签名的 `arm64` + `x86_64` universal binary。[`native/macos/manifest.json`](native/macos/manifest.json) 固定其 SHA-256、源码 digest、架构与 deployment target。`pnpm run check:native` 还会检查只有目标进程指针路由，并拒绝系统光标 warp 或全局指针 post symbol。
+
+## 配置
+
+<details>
+<summary>查看 Bundle 配置字段</summary>
+
+| 字段 | 用途 |
+|---|---|
+| `observationTtlMs` | observation 允许复用的生命周期；默认 `0` 关闭过期，也可设为最大 `86400000` ms（24 小时）内的任意值 |
+| `confirmationTtlMs` | 一次性敏感动作 confirmation 的生命周期 |
+| `actionTimeoutMs` | `1000` 到 `120000` ms 的 native action 硬超时 |
+| `settleMs` | `0` 到 `10000` ms 的动作后状态检查间隔 |
+| `maxSettleMs` | `100` 到 `60000` ms 的动作后 settle 最大预算 |
+| `maxNodes` / `maxDepth` / `maxTextBytes` | Accessibility 遍历与模型可见文本上限 |
+| `maxScreenshotBytes` | PNG Artifact 最大字节数 |
+| `artifactRoot` | workspace 内的相对截图目录 |
+| `helper.path` | 可选的显式外部 helper executable |
+| `helper.allowSourceBuild` | 提交 helper 缺失时允许显式托管源码重建；默认 `false` |
+| `interaction.focusPolicy` | `preserve`（默认）避免激活目标应用；`activate` 显式允许激活，并要求重新观察/校验 |
+| `interaction.keyboardPolicy` | `preserve` 不激活地把键盘事件定向投递；`activate`（Bundle 默认）在键盘 fallback 前激活目标应用 |
+| `interaction.pointerInputPolicy` | `targeted`（默认）允许 pid/window 定向指针输入；`deny` 禁用 click fallback、scroll 和 drag |
+| `interaction.cursorVisualization` | `visible`（默认）显示独立 Agent 光标；`hidden` 只关闭 overlay，不影响输入 |
+| `interaction.cursorMotionMs` | Agent 光标移动动画时长，默认 `180` 毫秒 |
+| `interaction.cursorAutoHideMs` | Agent 光标空闲后隐藏时间；默认 `0` 表示保持显示，直到绑定窗口变化或收到 hide 命令，也可设为最大 `30000` ms 内的有限值 |
+| `allowAllApps` | 向所有运行中的应用授予 `read` 与 `control`；默认 `false`。开启后忽略精确 `grants` |
+| `grants` | 准确、无通配符的 bundle-id read/control policy；`control: true` 隐含 read |
+
+</details>
+
+Settings 更新只有在校验与健康检查通过后才替换当前 provider generation；替换会使已有 observation 与待用 confirmation 失效。
+
+## 状态与限制
+
+- 状态：早期 `0.1.0`；稳定版本发布前，模型可见和 provider 行为仍可能变化。
+- 当前 provider 只支持 macOS；Windows UI Automation 和 Linux provider 尚未实现。
+- 目标进程指针投递使用动态解析的 SkyLight SPI。该路由不可用时，pointer fallback 会 fail closed，不会切换到全局输入。
+- 点击点必须落在选定应用的某个屏幕内窗口中；helper 会解析该点下最上层的匹配窗口，frame/title 匹配有歧义不再阻塞坐标动作。最小化、隐藏或无窗口目标会 fail closed。
+- 自定义 canvas、游戏、强化输入 surface 与未来 macOS 版本可能拒绝目标进程指针或键盘事件。应尽量优先使用语义化 Accessibility。
+- `focusPolicy: activate` 与 `keyboardPolicy: activate` 会有意打断前台工作，只作为操作方显式选择的兼容模式。
+- 目标应用可能因接受动作而自行改变 activation 或 focus。
+- 软件包按请求捕获离散 observation，不提供实时桌面流。
+- 浏览器工作应继续使用 browser automation，因为 DOM/CDP 状态更窄、更精确。
+- 公共 npm 包以 `@anionex/dsh-computer-use` 安装，可同时挂载到 Web 与 Headless Profile。
+
+## 开发与发布验收
+
+请把仓库放在 DeepSeek Harness checkout 旁边，让 TypeScript 与 Vitest 解析准确的 DSH peer declaration 和 runtime module：
+
+```text
+workspace/
+├── packages/
+├── vendor/
+└── dsh-computer-use/
+```
+
+随后运行：
+
+```sh
+pnpm install --frozen-lockfile
+pnpm run build
+DSH_COMPUTER_USE_REQUIRE_TCC=1 pnpm test
+pnpm run check:native
+pnpm pack --dry-run
+pnpm run validate
+```
+
+`pnpm run validate` 会运行 keyless local lane 和干净 Web/Headless Profile lane。真实模型发布 lane 需要 `DEEPSEEK_API_KEY`，并支持可选的 `DEEPSEEK_BASE_URL`：
+
+```sh
+pnpm run validate:model
+# or keyless validation followed by the real-model lane
+pnpm run validate:release
+```
+
+## 移除
+
+```sh
+dsh plugin --profile web remove @anionex/dsh-computer-use
+dsh plugin --profile headless remove @anionex/dsh-computer-use
+```
+
+移除或禁用 Bundle 会注销 Skill 与 Tool、取消 helper 工作、释放进程内 Agent observation、turn control grant 与 confirmation、关闭 storage-domain handle，并移除 Web contribution。已经生成的截图文件和插件自有的 `computer_use_state` sidecar 会保留，供用户显式清理。
+
+## 安全、社区与支持
+
+- 潜在漏洞按 [SECURITY.md](SECURITY.md) 私下报告。
+- 修改代码或文档前请阅读 [CONTRIBUTING.md](CONTRIBUTING.md)。
+- 安装、权限、配置和工作流问题见 [SUPPORT.md](SUPPORT.md)。
+- 在项目空间中遵守 [Code of Conduct](CODE_OF_CONDUCT.md)。
+- 版本记录见 [CHANGELOG.md](CHANGELOG.md)。
+- 维护支持方式见 [FUNDING.md](FUNDING.md)，赞助不购买 roadmap 控制权或私有支持。
+
+## 关于
+
+DSH Computer Use 由 [anionex](https://anionex.me/) 维护。如果你想了解我后续的更多工作，欢迎在 [X](https://x.com/anion_ex) 或 [GitHub](https://github.com/Anionex) 关注我。
+
+## 许可证
+
+[MIT](LICENSE) © 2026 anionex。