@amaster.ai/pi-computer-use 0.1.0

package/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

package/README.md

@@ -0,0 +1,136 @@
+# @amaster.ai/pi-computer-use
+
+![pi-computer-use preview](https://raw.githubusercontent.com/TGYD-helige/pi/master/packages/pi-computer-use/preview.png)
+
+pi-coding-agent extension that wraps [cua-driver-rs](https://github.com/trycua/cua/), exposing desktop automation tools with a `computer_use_` prefix.
+
+## Features
+
+- **Zero external dependencies** — pre-compiled cua-driver-rs binaries bundled for all platforms
+- **MCP stdio communication** — spawns `cua-driver mcp` via `StdioClientTransport`, JSON-RPC over stdio
+- **Dynamic tool discovery** — auto-discovers upstream MCP tools and registers with `computer_use_` prefix; falls back to a built-in tool list when cua-driver fails to start
+- **Smart tool filtering** — excludes non-essential tools (agent cursor, recording, config, raw screenshot), exposes 17 action tools + 1 vision tool
+- **Optional visual analysis** — `computer_use_analyze_screenshot` via configurable vision model
+- **Cross-platform permission handling** — detects platform-specific permission issues (macOS TCC, Windows UAC, Linux display server access) and returns actionable guidance
+- **Graceful degradation** — tools are always registered even when cua-driver cannot connect; lazy reconnect is attempted on each tool call
+
+## Install
+
+```bash
+bun add @amaster.ai/pi-computer-use
+```
+
+Requires Node.js >= 20 and `@earendil-works/pi-coding-agent >= 0.74.0`.
+
+## Usage
+
+Install the package and pi-coding-agent will automatically discover and load the extension. All tools are registered on `session_start`.
+
+Configure via `.pi/settings.json` (project-level) or `~/.pi/agent/settings.json` (user-level) under the `"pi-computer-use"` key:
+
+```json
+{
+  "pi-computer-use": {
+    "mode": "bundled"
+  }
+}
+```
+
+## Configuration
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `mode` | `'bundled' \| 'path'` | `'bundled'` | Binary resolution strategy |
+| `binaryPath` | `string` | — | Custom cua-driver binary path (requires `mode: 'path'`) |
+| `extraArgs` | `string[]` | — | Extra CLI arguments passed to cua-driver |
+| `visionModel` | `VisionModelConfig` | — | Enable visual screenshot analysis |
+
+### Vision Model (Optional)
+
+Enable `computer_use_analyze_screenshot` by referencing a model already configured in Pi's model registry (`models.json`):
+
+```json
+{
+  "pi-computer-use": {
+    "visionModel": {
+      "provider": "openai",
+      "model": "gpt-4o"
+    }
+  }
+}
+```
+
+The extension resolves API key, base URL, and headers from the model registry automatically — no need to duplicate credentials here.
+
+## Exposed Tools (17 + 1 vision)
+
+### Input
+
+| Tool | Description |
+|------|-------------|
+| `computer_use_click` | Left-click via element_index or x/y coordinates |
+| `computer_use_double_click` | Double-click at x/y or on an AX element |
+| `computer_use_right_click` | Right-click (context menu) |
+| `computer_use_type_text` | Insert text via AX or CGEvent fallback |
+| `computer_use_press_key` | Press and release a single key |
+| `computer_use_hotkey` | Press a key combination (e.g. Cmd+C) |
+| `computer_use_scroll` | Scroll by line or page in a direction |
+| `computer_use_drag` | Press-drag-release gesture between two points |
+| `computer_use_set_value` | Set value on UI elements (popups, sliders, steppers) |
+
+### Query
+
+| Tool | Description |
+|------|-------------|
+| `computer_use_get_screen_size` | Get display dimensions and scale factor |
+| `computer_use_get_cursor_position` | Get current mouse cursor position |
+| `computer_use_get_accessibility_tree` | Lightweight desktop snapshot (apps, windows, bounds) |
+| `computer_use_get_window_state` | Full AX tree of a window with actionable element indices |
+| `computer_use_list_windows` | List all top-level windows with bounds and z-order |
+| `computer_use_list_apps` | List running and installed apps with state flags |
+
+### App Lifecycle
+
+| Tool | Description |
+|------|-------------|
+| `computer_use_launch_app` | Launch an app in the background without focus steal |
+| `computer_use_kill_app` | Force-terminate a process by pid |
+
+### Vision (requires `visionModel` config)
+
+| Tool | Description |
+|------|-------------|
+| `computer_use_analyze_screenshot` | Take a screenshot and analyze it with a vision model |
+
+## Excluded Tools (16)
+
+Agent cursor styling, recording/replay, config management, zoom, raw screenshot (use `analyze_screenshot` instead), and browser-specific operations are filtered out.
+
+## Permissions
+
+On `session_start`, the extension checks permissions via cua-driver's `check_permissions` tool. Platform-specific guidance is provided:
+
+| Platform | Accessibility | Screen Capture |
+|----------|--------------|----------------|
+| macOS | System Settings → Privacy & Security → Accessibility | System Settings → Privacy & Security → Screen & System Audio Recording |
+| Windows | Run as Administrator / UI Automation access | Check DRM or security policy |
+| Linux | AT-SPI accessibility service | PipeWire portal or X11 access |
+
+When cua-driver fails to connect (missing permissions, binary not found, etc.):
+1. User is notified with a platform-appropriate warning
+2. Tools are still registered using a built-in fallback schema
+3. On each tool call, lazy reconnect is attempted; if it still fails, a friendly error with permission instructions is returned
+
+## Supported Platforms
+
+| Platform | Binary |
+|----------|--------|
+| macOS ARM64 | `bin/darwin-arm64/cua-driver` |
+| macOS x64 | `bin/darwin-x64/cua-driver` |
+| Linux x64 | `bin/linux-x64/cua-driver` |
+| Windows x64 | `bin/win32-x64/cua-driver.exe` |
+| Windows ARM64 | `bin/win32-arm64/cua-driver.exe` |
+
+## License
+
+Apache-2.0

package/bin/darwin-arm64/.version

@@ -0,0 +1,2 @@
+cua-driver-v0.2.0
+swift

package/bin/darwin-arm64/CuaDriver.app/Contents/Info.plist

@@ -0,0 +1,32 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+    <key>CFBundleIdentifier</key>
+    <string>com.trycua.driver</string>
+    <key>CFBundleName</key>
+    <string>Cua Driver</string>
+    <key>CFBundleDisplayName</key>
+    <string>Cua Driver</string>
+    <key>CFBundleExecutable</key>
+    <string>cua-driver</string>
+    <key>CFBundleIconFile</key>
+    <string>AppIcon</string>
+    <key>CFBundleIconName</key>
+    <string>AppIcon</string>
+    <key>CFBundlePackageType</key>
+    <string>APPL</string>
+    <key>CFBundleShortVersionString</key>
+    <string>0.2.0</string>
+    <key>CFBundleVersion</key>
+    <string>1</string>
+    <key>LSMinimumSystemVersion</key>
+    <string>14.0</string>
+    <key>LSUIElement</key>
+    <true/>
+    <key>NSHighResolutionCapable</key>
+    <true/>
+    <key>NSSupportsAutomaticTermination</key>
+    <true/>
+</dict>
+</plist>

package/bin/darwin-arm64/CuaDriver.app/Contents/Resources/Skills/cua-driver/README.md

@@ -0,0 +1,140 @@
+# cua-driver — Claude Code skill
+
+A [Claude Code](https://code.claude.com) skill that teaches Claude to
+drive native macOS apps via the
+[`cua-driver`](https://github.com/trycua/cua/tree/main/libs/cua-driver)
+CLI — snapshot an app's accessibility tree, click/type/scroll by
+`element_index`, and verify via re-snapshot. Backgrounded-first: no
+focus steal, no cursor warp, no Space follow.
+
+## What the skill covers
+
+- The snapshot-before-AND-after invariant that keeps the agent honest
+  about whether an action actually landed.
+- The backgrounded-click recipe (yabai focus-without-raise + stamped
+  SLEventPostToPid) that lets synthetic clicks land on Chrome web
+  content without raising the window or pulling the user across Spaces.
+- Web-app quirks (`WEB_APPS.md`) — Chromium/WebKit/Electron/Tauri,
+  including the minimized-Chrome keyboard-commit caveat and the
+  `set_value` workaround.
+- Trajectory recording (`RECORDING.md`) — optional per-session
+  recording + replay for demos and regressions.
+- Canvas/viewport apps (Blender, Unity, GHOST, Qt, wxWidgets) —
+  HID-tap fallback when AX is empty.
+
+See `SKILL.md` for the main body.
+
+## Prerequisites
+
+1. **macOS 14 or newer** — the driver depends on SkyLight private SPIs
+   that were stabilized in Sonoma.
+2. **`cua-driver` CLI + `CuaDriver.app`** — installable one-liner:
+   ```bash
+   /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh)"
+   ```
+   Or from a clone of `trycua/cua`:
+   ```bash
+   cd libs/cua-driver
+   scripts/install-local.sh   # builds + installs + symlinks for dev use
+   ```
+   The driver runs as an `.app` bundle because macOS TCC grants are
+   tied to a stable bundle id (`com.trycua.driver`). The CLI symlink
+   lets Claude invoke tools via plain shell.
+3. **TCC grants on `CuaDriver.app`** — **Accessibility** and
+   **Screen Recording** in System Settings → Privacy & Security.
+   Verify with:
+   ```bash
+   cua-driver check_permissions
+   ```
+   Both fields must be `true`. If not, the app appears in the
+   relevant panes of System Settings after first use; toggle it on
+   there.
+
+## Install
+
+The skill is two drop-in directories.
+
+**Personal scope** (all Claude Code sessions on your machine):
+
+```bash
+mkdir -p ~/.claude/skills
+cp -R Skills/cua-driver ~/.claude/skills/
+```
+
+Or symlink if you want edits-in-place:
+
+```bash
+ln -s "$PWD/Skills/cua-driver" ~/.claude/skills/cua-driver
+```
+
+**Project scope** (committed alongside a specific repo):
+
+```bash
+mkdir -p .claude/skills
+cp -R /path/to/cua/libs/cua-driver/Skills/cua-driver .claude/skills/
+```
+
+## Invoking the skill
+
+Claude Code auto-invokes the skill when you ask for macOS GUI
+automation — e.g. "open the Downloads folder in Finder", "click the
+Save button in Numbers", "navigate to trycua.com in Chrome". You can
+also invoke it explicitly:
+
+```
+/cua-driver
+```
+
+## Claude Code MCP compatibility mode
+
+For normal skill-driven use, prefer the CLI or the standard MCP server. If you want Claude Code's vision/computer-use-style flow to ground on CuaDriver screenshots, register the compatibility server:
+
+```bash
+claude mcp add --transport stdio cua-computer-use -- cua-driver mcp --claude-code-computer-use-compat
+```
+
+This mode exposes the normal CuaDriver tools and changes only `screenshot`. The compatibility screenshot requires `pid` and `window_id`, captures that window only, and establishes a window-local pixel coordinate frame. It does not call Anthropic APIs or expose Anthropic's native computer-use API tool.
+
+Use MCP for this Claude Code vision/computer-use-style path. CLI screenshots still work as CuaDriver calls, but they do not expose the `mcp__cua-computer-use__screenshot` tool name that Claude Code appears to use as the image-grounding cue.
+
+## Files
+
+- `SKILL.md` — the main skill body (~500 lines). Loaded on first
+  invocation; stays in context for the session.
+- `WEB_APPS.md` — browsers, Electron, Tauri (Chromium + WebKit). Loaded
+  on demand when SKILL.md's pointer is followed.
+- `RECORDING.md` — trajectory recording / replay. Loaded on demand.
+- `TESTS.md` — manual test scripts for end-to-end skill verification.
+
+## Troubleshooting
+
+- `cua-driver: command not found` → re-run the installer or add
+  `.build/CuaDriver.app/Contents/MacOS/` to `$PATH`.
+- `No cached AX state for pid X window_id W` → element_index was
+  reused across turns, or across different windows of the same app.
+  Call `get_window_state({pid, window_id})` first in the same turn,
+  with the same window_id you're about to act against.
+- Empty `tree_markdown` → `capture_mode` is set to `vision`, which
+  skips the AX walk by design. Flip back to the default `som`
+  (`cua-driver config set capture_mode som`) to get the tree.
+  Tiny screenshot → likely a stale window capture. See "Behavior
+  matrix" in SKILL.md for the full mode table.
+- System-alert beep when pressing Return on a minimized Chrome
+  omnibox → the keyboard-commit-on-minimized limitation. Use
+  `set_value` on the field instead, or AX-click a Go/Submit button.
+  See `WEB_APPS.md`.
+
+## Updates
+
+The skill evolves alongside the driver. To update:
+
+```bash
+cd /path/to/cua && git pull
+# if you copied: re-copy
+cp -R libs/cua-driver/Skills/cua-driver ~/.claude/skills/
+# if you symlinked: nothing needed
+```
+
+## License
+
+MIT. Same license as the parent `trycua/cua` repo.

package/bin/darwin-arm64/CuaDriver.app/Contents/Resources/Skills/cua-driver/RECORDING.md

@@ -0,0 +1,113 @@
+# Recording & replaying trajectories
+
+Session-scoped capture of action sequences + pre/post state, suitable
+for demos, regression diffs, and training data. Invoked only when the
+user explicitly asks to record — the skill does not auto-enable this.
+
+`set_recording` turns on a session-scoped trajectory recorder. While
+enabled, every action-tool call (`click`, `right_click`, `scroll`,
+`type_text`, `press_key`, `hotkey`, `set_value`)
+writes a numbered turn folder under a caller-chosen output
+directory. Read-only tools (`get_window_state`, `list_windows`,
+`screenshot`, `list_apps`, permission probes, agent-cursor getters /
+setters, and `set_recording` itself) are not recorded.
+
+## Enable / disable
+
+Two equivalent surfaces: the `set_recording` MCP tool, or the
+friendlier `cua-driver recording` subcommand group (wraps
+`set_recording` + `get_recording_state` with human-readable output).
+
+```
+cua-driver recording start ~/cua-trajectories/run-1
+# … run the workflow …
+cua-driver recording status    # -> enabled / disabled, next_turn, output_dir
+cua-driver recording stop      # -> "Recording disabled (N turns captured in …)"
+```
+
+Raw-tool equivalent:
+
+```
+cua-driver set_recording '{"enabled":true,"output_dir":"~/cua-trajectories/run-1"}'
+cua-driver get_recording_state
+cua-driver set_recording '{"enabled":false}'
+```
+
+The `recording` subcommands require a running daemon (`cua-driver
+serve &`) because recording state is per-process. `output_dir` expands
+`~` and is created (with intermediates) if missing. Turn numbering
+starts at `1` every time recording is (re-)enabled, regardless of any
+existing contents in the directory. State lives in memory only — a
+daemon restart resets to disabled.
+
+## What each turn folder contains
+
+Each action writes to `turn-NNNNN/` (five-digit zero-padded counter):
+
+- `app_state.json` — post-action AX snapshot for the target pid, same
+  shape `get_window_state` returns (tree_markdown, element_count,
+  turn_id, etc.) minus the screenshot fields. The recorder resolves a
+  frontmost window internally (visible + on-current-Space preferred,
+  max-area fallback) since individual action tools carry a
+  window_id but the recorder has no caller-supplied anchor.
+- `screenshot.png` — post-action capture of the same window the
+  recorder just snapshotted. Omitted when the pid has no visible
+  window.
+- `action.json` — the tool name, full input arguments, result
+  summary, pid, click point (when applicable), ISO-8601 timestamp.
+- `click.png` — only for click-family actions (`click`,
+  `right_click`): a copy of `screenshot.png` with a red dot drawn at
+  the click point (screen-absolute point → window-local pixels via
+  the screenshot's `scale_factor`). Absent for other tools and for
+  clicks whose point falls outside the captured window.
+
+## When to use it
+
+- Demos and screen recordings — play the turn folder back to show
+  exactly what the agent saw and what it did.
+- Replay for regression — re-run the same sequence against a future
+  build and diff the new trajectory against the saved one.
+- Training data collection — each turn is a
+  `(state, action, next_state)` triple ready for offline learning.
+
+## When to invoke it
+
+This skill does **not** auto-enable recording. The client invokes
+`set_recording` explicitly when the user asks to capture a session.
+If the user says "record this session" or similar, call
+`set_recording({enabled:true, output_dir:…})` before the first
+action, and `set_recording({enabled:false})` when done.
+
+## Replaying a recorded trajectory
+
+`replay_trajectory({dir})` walks `<dir>/turn-NNNNN/` folders in
+lexical order, reads each `action.json`, and re-invokes the recorded
+tool with its recorded `arguments`. Optional knobs: `delay_ms`
+(pacing between turns, default 500) and `stop_on_error` (halt on
+first failure, default true).
+
+```
+cua-driver recording start ~/cua-trajectories/demo1
+# … run the workflow …
+cua-driver recording stop
+# Later: replay against a new build.
+cua-driver replay_trajectory '{"dir":"~/cua-trajectories/demo1","delay_ms":500}'
+```
+
+Important caveat: **element_index doesn't survive across sessions**.
+Indices are assigned fresh on every `get_window_state` snapshot,
+keyed on `(pid, window_id)`, so a recorded
+`click({pid, window_id, element_index: 14})` from yesterday won't
+resolve today — the pid is usually different, the window_id always
+is. The call returns `Invalid element_index` or `No cached AX
+state`. Pixel clicks (`click({pid, x, y})`) and keyboard tools
+(`press_key`, `hotkey`, `type_text` without element_index) replay cleanly; element-indexed actions require a
+live snapshot that replay doesn't currently re-emit (read-only tools
+like `get_window_state` aren't recorded). For a reliable replay, either
+compose the trajectory from pixel + keyboard primitives, or capture
+it as a regression artifact (compare the failure/success pattern
+across builds) rather than a re-driving script.
+
+If recording is still enabled while replay runs, the replay is
+itself recorded into the current output directory — that's the
+intended regression-diff workflow.