physiclaw 0.0.1__tar.gz

physiclaw-0.0.1/.claude/settings.local.json

@@ -0,0 +1,23 @@
+{
+  "permissions": {
+    "allow": [
+      "mcp__physiclaw__screenshot",
+      "mcp__physiclaw__move",
+      "mcp__physiclaw__swipe",
+      "mcp__physiclaw__park",
+      "mcp__physiclaw__bbox_target",
+      "mcp__physiclaw__confirm_bbox",
+      "mcp__physiclaw__tap",
+      "mcp__physiclaw__long_press",
+      "mcp__physiclaw__grid_overlay",
+      "Bash(uv run:*)",
+      "Bash(git mv:*)",
+      "Bash(git rm:*)",
+      "WebFetch(domain:raw.githubusercontent.com)",
+      "WebSearch",
+      "WebFetch(domain:www.anthropic.com)",
+      "Bash(python3 -c \"import json,sys; d=json.load\\(sys.stdin\\); [print\\(i['path']\\) for i in d.get\\('tree',[]\\) if 'prompt' in i['path'].lower\\(\\) or 'system' in i['path'].lower\\(\\) or 'instruct' in i['path'].lower\\(\\)]\")",
+      "Bash(uv pip *)"
+    ]
+  }
+}

physiclaw-0.0.1/.claude/skills/calibrate-keyboard/SKILL.md

@@ -0,0 +1,137 @@
+---
+name: calibrate-keyboard
+description: Detect keyboard keys from phone screenshots and generate UI preset for typing. Use when setting up a new phone or the keyboard layout has changed.
+allowed-tools: Bash, Read, Edit, Write
+---
+
+# Keyboard Calibration
+
+Guide the user through calibrating the on-screen keyboard for PhysiClaw.
+
+## Step 1: Collect screenshots
+
+Ask the user to take two phone screenshots with the keyboard visible:
+
+1. **Alpha keyboard** (default QWERTY layout)
+2. **Numeric keyboard** (tap the 123 key first)
+
+Save both as PNG/JPG in `data/image/keyboard/`. Remove any old images from that directory first.
+
+## Step 2: Check images and run detection
+
+Run this check first:
+
+```bash
+uv run python -c "
+import cv2
+from pathlib import Path
+imgs = sorted(Path('data/image/keyboard').glob('*.*'))
+imgs = [p for p in imgs if p.suffix.lower() in ('.png', '.jpg', '.jpeg')]
+print(f'{len(imgs)} images found')
+sizes = set()
+for p in imgs:
+    img = cv2.imread(str(p))
+    if img is not None:
+        sizes.add((img.shape[1], img.shape[0]))
+        print(f'  {p.name}: {img.shape[1]}x{img.shape[0]}')
+if len(sizes) == 1:
+    print('All same size')
+elif len(sizes) > 1:
+    print(f'ERROR: different sizes: {sizes}')
+if len(imgs) < 2:
+    print('ERROR: need at least 2 images (alpha + numeric)')
+"
+```
+
+Verify:
+
+1. At least 2 images (one alpha, one numeric)
+2. All images have the same width and height (same phone, same orientation)
+3. The keyboard background is clean and uniform (no custom themes, no wallpaper keyboards)
+4. The keyboard is the system default (Gboard, iOS keyboard, etc.) -- not a third-party keyboard
+5. Original images, no resize, no editing
+
+If any check fails, ask the user to fix and re-screenshot.
+
+Then run detection:
+
+```bash
+uv run python scripts/calibrate_keyboard.py
+```
+
+This detects key bounding boxes and generates:
+
+- Bounding box images in `data/image/keyboard/bbox/`
+- A preset template at `.claude/ui-presets/system-keyboard.md` with positions filled in
+
+Check the output: it should report 4 rows per keyboard, ~33 keys for alpha, ~35 for numeric.
+If detection fails or key counts are wrong, ask the user to retake screenshots (original image, no resize, no editing).
+
+## Step 3: Fill ??? labels
+
+Keys marked ??? need to be identified. After filling, tell the user:
+"You can open `.claude/ui-presets/system-keyboard.md` to check my editing."
+
+For each keyboard page:
+
+1. Tell the user: "Please open `{bbox image path}` to verify my labels." (the path is listed under each page heading)
+2. Read the bounding box image yourself to identify keys
+3. Fill ??? entries one page at a time
+4. Always refer to keys by bbox index: "bbox 11: @, bbox 12: #"
+5. After filling each page, list your guesses and ask the user to confirm
+6. Punctuation/symbols are hard to distinguish -- always ask the user to verify these
+
+### Symbol reference for punctuation
+
+When identifying symbol keys, select from this list.
+Note: Chinese Pinyin keyboards have no straight quotes (' "). Only curly quotes and enumeration comma.
+If a symbol looks like ' or `, it is most likely 、 (enumeration comma).
+
+| Symbol | Name | Type |
+| -- | -------------- | ---- |
+| @ | at sign | English |
+| # | hash / pound | English |
+| $ | dollar sign | English |
+| _ | underscore | English |
+| & | ampersand | English |
+| * | asterisk | English |
+| - | hyphen | English |
+| + | plus | English |
+| / | slash | English |
+| , | comma | English |
+| . | period | English |
+| : | colon | English |
+| ; | semicolon | English |
+| ! | exclamation mark | English |
+| ? | question mark | English |
+| ( | left parenthesis | English |
+| ) | right parenthesis | English |
+| （ | left parenthesis (Chinese) | Chinese |
+| ） | right parenthesis (Chinese) | Chinese |
+| 、 | enumeration comma | Chinese |
+| " | left double quote (Chinese) | Chinese |
+| " | right double quote (Chinese) | Chinese |
+| ： | colon (Chinese) | Chinese |
+| ； | semicolon (Chinese) | Chinese |
+| ！ | exclamation mark (Chinese) | Chinese |
+| ？ | question mark (Chinese) | Chinese |
+| ， | comma (Chinese) | Chinese |
+| 。 | period (Chinese) | Chinese |
+| ¥ | yen / RMB sign | Currency |
+
+## Step 4: Verify positions unchanged
+
+After filling all ???, verify that no Position values were accidentally modified by comparing
+the Position columns of the filled file against the reference copy:
+
+```bash
+diff <(grep -oP '\[[\d., ]+\]' .claude/ui-presets/system-keyboard.md) \
+     <(grep -oP '\[[\d., ]+\]' data/image/keyboard/bbox/system-keyboard.ref.md)
+```
+
+If no output, all positions match. If there are differences, the Position column was accidentally
+edited — restore those rows from the reference file.
+
+Then ask the user to review the final `.claude/ui-presets/system-keyboard.md`.
+
+Done. The AI agent can now use the keyboard preset for typing.

physiclaw-0.0.1/.claude/skills/calibrate-keyboard/template.md

@@ -0,0 +1,8 @@
+# System Keyboard
+
+To type uppercase: tap ⇧ Shift first, then tap the letter. Shift auto-reverts to lowercase after one letter.
+To type a digit or symbol: tap ?123 to switch to Numeric Keyboard, type, then tap 返回(ABC) to switch back.
+To type Chinese (Pinyin): type the pinyin letters, then tap Space to confirm the first candidate. E.g. to type "你好", tap n-i-h-a-o then tap Space.
+To hide the keyboard: tap the back button or tap outside the text field.
+
+{{pages}}

physiclaw-0.0.1/.claude/skills/cron/SKILL.md

@@ -0,0 +1,115 @@
+---
+name: cron
+description: Manage scheduled jobs in jobs/jobs.md — list, add, or cancel cron entries. Verifies the file parses before and after every change.
+allowed-tools: Bash, Read, Edit, Write
+---
+
+# Cron Job Management
+
+The runtime checks `jobs/jobs.md` every minute. Each `## <id>` section
+is a scheduled job. When a job's `Schedule:` matches the current minute
+and its `Status:` is `pend`, the hook fires a `Trigger` and the
+runtime spawns Claude Code with the job's context.
+
+**Jobs are never deleted.** They transition through statuses:
+`pend` → `fired` (hook) → `pend` (periodic done/fail) or `done`/`fail` (one-time).
+`pend` → `cancel` (user).
+
+## CLI tools
+
+```bash
+uv run python -m physiclaw.agent.hooks.cron verify          # parse + list every job
+uv run python -m physiclaw.agent.hooks.cron jobs-to-do      # list fired jobs awaiting agent execution
+uv run python -m physiclaw.agent.hooks.cron done <id> <result>  # agent: mark complete with result
+uv run python -m physiclaw.agent.hooks.cron fail <id> <reason>  # agent: mark failed with reason
+uv run python -m physiclaw.agent.hooks.cron cancel <id>     # cancel a job
+```
+
+Always run `verify` first, and again after any change.
+
+### Layout
+
+1. `## <id>` heading — lowercase kebab-case
+2. **Description** — one plain line below the heading (not a list item).
+   For the user: "what is this job?"
+3. **Fields** — markdown list (`- Key: value`).
+
+**Description vs Context**: Description is for the user scanning the
+file. Context is for the agent executing the job — include everything
+Claude needs: which app to open, what to do, edge cases.
+
+### Fields
+
+| Field              | Set by   | Notes                                            |
+|--------------------|----------|--------------------------------------------------|
+| description line   | skill    | One line under heading (required)                |
+| `Type:`            | skill    | `periodic` or `one-time` (required)              |
+| `Schedule:`        | skill    | 5-field cron in backticks (required)             |
+| `Create time:`     | skill    | ISO timestamp when the job was added             |
+| `Next fire time:`  | skill    | Computed after each fire                         |
+| `Last fire time:`  | hook     | When the hook last fired this job                |
+| `Execution time:`  | agent    | When Claude finished — set via `done`/`fail` CLI |
+| `Execution result:`| agent    | One-line result summary — set via `done`/`fail`  |
+| `Status:`          | mixed    | `pend`, `fired`, `cancel`, `done`, or `fail`     |
+| `Context:`         | skill    | Full agent instructions (required, last field)   |
+
+**Status lifecycle:**
+
+- `pend` — waiting for next fire time to arrive
+- `fired` — hook fired the job, awaiting agent execution
+- `cancel` — user cancelled; never fires
+- `done` — agent finished a one-time job (`done <id>`)
+- `fail` — agent failed a job (`fail <id>`)
+
+**Transitions:**
+`pend` → `fired` (hook fires) → `pend` (periodic done/fail) or
+`done`/`fail` (one-time). `pend` → `cancel` (user).
+
+**Time format:** ISO 8601 truncated to minute, naive local time.
+Example: `2026-04-09T09:00`. Unfired fields use `(never)`.
+
+## Workflows
+
+### List jobs
+
+1. Run verify
+2. Show each job's id, description, type, schedule, status, and times
+
+### Add a job
+
+1. Run verify
+2. Based on user request, determine: **id**, **description**, **type**,
+   **schedule**, **context**
+3. Create `jobs/jobs.md` if it doesn't exist:
+
+   ```markdown
+   # Cron Jobs
+   ```
+
+4. Append using this template:
+
+   ```markdown
+   ## <id (e.g. `weather-check`, `mom-birthday`)>
+
+   <one-line description>
+
+   - Type: <periodic|one-time>
+   - Schedule: `<cron expression>`
+   - Create time: <now as ISO>
+   - Next fire time: <compute from schedule and current time, fill as ISO>
+   - Last fire time: (never)
+   - Execution time: (never)
+   - Execution result: (never)
+   - Status: pend
+   - Context: <full instructions for the agent>
+   ```
+
+5. Run verify (this also validates Next fire time matches the schedule)
+6. Tell the user when the job will next fire
+
+### Cancel a job
+
+1. Run verify
+2. Read `jobs/jobs.md`, confirm which job to cancel with the user
+3. Edit the `Status:` field to `cancel`
+4. Run verify

physiclaw-0.0.1/.claude/skills/jd/SKILL.md

@@ -0,0 +1,38 @@
+---
+name: jd
+description: Use when the task is grocery / fresh-food shopping via JD / 京东 / 七鲜 / 7Fresh — owner asks to buy 零食, 水果, 蔬菜, 日用品, or to place an order on 京东. NOT for clothing, electronics, or non-grocery shopping, NOT for other grocery apps (Meituan / 盒马 / Dingdong).
+---
+
+# JD (京东) — Grocery shopping
+
+Use **京东七鲜** (JD 7Fresh) for groceries. Other JD categories need explicit owner ask.
+
+## Tool choice
+
+Prefer `peek()` — cheap, no app-side reactions. Reach for `screenshot()` only when the target is icon-only (no text label). **`screenshot()` on a commodity detail page triggers the share overlay** — see Gotcha.
+
+## Flow
+
+1. `/open-app 京东` (or `JD`). Make sure you're on the **首页** tab (top nav), then tap into 京东七鲜.
+2. Tap the search box. If it has stale text, tap **backspace** (bbox in PHYSICLAW.md "iPhone keyboard bboxes") until empty — see also `Skill(name="search-in-app")` for the full clear-paste-submit flow. Type/paste the item, open its shop page.
+3. Tap 加入购物车 (Add to cart).
+   - **A spec-selection sheet often slides up** (size, brand, weight). Pick the right spec, then tap **确定** to confirm. The sheet dismisses, the item is added, and you land back on the product page.
+   - Items without variants skip the sheet — one tap on 加入购物车 adds directly.
+   - **NEVER tap 加入购物车 again on the product page.** The product page looks the same before and after add — you cannot tell from the layout alone whether the add succeeded. Re-tapping just re-opens the spec sheet, you re-confirm, and you've now added the item TWICE. Trust that 确定 worked; verify by checking the cart-icon badge count (top-right corner of the page) or by going to the cart in step 4.
+4. Tap the cart icon, review line items, tap 去结算. **Always review the cart here** — if quantities look wrong (item appearing twice when you only meant once), that's a sign step 3 was tapped twice; remove the dup before checkout.
+5. Send the owner: item, qty, price, address, fees, ETA. Wait for explicit OK.
+6. Tap 提交订单 / 立即支付.
+
+## Gotcha — screenshot triggers share popup
+
+JD intercepts the iOS screenshot gesture and overlays a 分享截屏 menu (朋友圈 / QQ / 微信好友 / 保存图片 / 搜问题). The screenshot captured the real page first; recover without re-shooting:
+
+1. `screenshot()` — pixel-perfect bboxes, listing captured.
+2. `peek()` — if it shows share-sheet text + dim area, the popup is covering the page.
+3. Dismiss the popup (see below).
+4. `peek()` — confirm the overlay is gone.
+5. **Act on labelled targets from the screenshot's text rows.** After the peek stubs the screenshot, icon rows are dropped but text rows (`加入购物车`, product title, price, `去结算`) survive in the stubbed listing — tap those. For the `+` add-to-cart button on list rows, it's typically detected as a `[text]` row too (OCR reads the `+` glyph), so it survives. If what you need is purely icon-only (e.g. the cart-badge icon in the top-right), re-`screenshot` — that's rare.
+
+## Dismiss popup / share sheet / bottom sheet
+
+Tap the **dimmed area** above the popup. Safe target: `[0.05, 0.40, 0.15, 0.60]` (center ≈ x=0.10, y=0.50 — left edge, vertical middle). The mirror bbox on the right edge `[0.85, 0.40, 0.95, 0.60]` works too.

physiclaw-0.0.1/.claude/skills/open-app/SKILL.md

@@ -0,0 +1,64 @@
+---
+name: open-app
+description: Use when you need to launch an app that is NOT on the current screen and NOT in the dock — any time "open <app>" is the next step and you can't see the icon. Use FIRST before tapping blindly. NOT needed when the target app's icon is already visible.
+---
+
+# Open App via Spotlight
+
+**Argument:** App name (e.g., "美团", "WeChat", "Safari").
+
+## Steps
+
+Each step is one `[note, one-other]` turn. `<name>` placeholders
+refer to the **Fixed elements** table at the bottom.
+
+1. `send_to_clipboard(text="<app name>")`
+   — copy the app name (use the exact text the owner asked for, e.g.
+   `"美团"`, `"WeChat"`).
+2. `home_screen()` — return to a clean launch pad. **Skip if you're
+   already on the home screen** (peek shows app-icon grid + dock,
+   no in-app chrome) — slow physical motion (~2s); don't waste it.
+3. `swipe(bbox=<spotlight-pull>, direction="down", size="l")`
+   — open Spotlight. Bbox is mid-screen (NOT the top edge — that
+   opens Notification Center). `size="l"` (~4cm) avoids overshoot.
+4. `peek()` — refresh listing; the search field and keyboard should
+   be visible.
+5. **If the field has stale text** (no "搜索"/"Search" placeholder),
+   clear it via tap+backspace. Chain with ONE `sequence` call (still
+   one tool call, so the turn remains `[note, one-other]`):
+
+   ```python
+   sequence(
+     step1={"tool_name": "tap", "arg": <search-field>},
+     step2={"tool_name": "tap", "arg": <backspace>},
+     step3={"tool_name": "tap", "arg": <backspace>},
+     step4={"tool_name": "tap", "arg": <backspace>},
+     step5={"tool_name": "tap", "arg": <backspace>},
+   )
+   ```
+   Tap 1 focuses; taps 2-5 each delete one char. Over-tapping an
+   empty field is a no-op — safe to over-estimate. `peek` to verify;
+   if text remains, run another sequence. **Skip when the field is
+   empty.** Prefer tap-backspace over `long_press(backspace)` —
+   per-tap deletions are deterministic.
+6. `long_press(bbox=<search-field>)` — opens the Paste popover above
+   the search field. If no popover appears after long-press, you
+   tapped the wrong element — re-`peek` and pick again.
+7. `peek()` — refresh listing; "Paste" / "粘贴" appears in the popover.
+8. `tap(bbox=<paste-button>)` — paste the app name into the field.
+9. `peek()` — refresh listing; search results render below the field.
+10. `tap(bbox=<app-icon>)` — launch the app. (See `<app-icon>` row
+    below for decoy warnings.)
+
+## Fixed elements
+
+Typical bboxes as priors — per CONVENTION.md, copy verbatim from the
+latest `peek` / `screenshot` listing before tapping.
+
+| Name | Typical bbox | How to find it / decoys |
+|---|---|---|
+| `<spotlight-pull>` | `[0.3, 0.4, 0.7, 0.6]` | Mid-screen rectangle. Swipe-down anchor; not a peek target. |
+| `<search-field>` | `[0.11, 0.60, 0.99, 0.66]` | The **focused input** at the BOTTOM, just above the keyboard, y≈0.62. Full-width field with mic icon on right. |
+| `<backspace>` | see PHYSICLAW.md "iPhone keyboard bboxes" | — |
+| `<paste-button>` | `[0.09, 0.56, 0.19, 0.58]` | Row labeled `Paste` / `粘贴` in the pill popover just ABOVE the focused field at y≈0.56. Often next to an `AutoFill` button (same y, x≈0.26-0.38) — pick `Paste`, not `AutoFill`. Dismisses if you tap elsewhere first. |
+| `<app-icon>` | varies — in search results below the field | Row whose label matches the app name **exactly**. Skip rows with App Store badges or "in Safari" / web hits. |

physiclaw-0.0.1/.claude/skills/phone-setup/SKILL.md

@@ -0,0 +1,117 @@
+---
+name: phone-setup
+description: Guide user step-by-step to set up their iPhone for PhysiClaw — enable AssistiveTouch, create three iOS Shortcuts (take screenshot, upload latest, clipboard sync).
+allowed-tools: Bash, Read
+---
+
+# iPhone Setup for PhysiClaw
+
+Set up three iOS Shortcuts (take screenshot · upload latest · clipboard sync) and bind them to AssistiveTouch taps. The server must be running.
+
+## Step 1: Check `<name>.local` hostname
+
+Shortcuts that embed the LAN IP break when DHCP changes it. `<name>.local` survives IP changes on the same Wi-Fi.
+
+Check the current name resolves:
+
+```bash
+CUR=$(scutil --get LocalHostName) && echo "$CUR" && ping -c 1 -W 1000 "${CUR}.local"
+```
+
+Use whatever the current name is — do **not** rename automatically. If the user wants a more stable or unique name (e.g. to avoid macOS numeric collision suffixes like `-2`), they can rename it themselves with:
+
+```bash
+sudo scutil --set LocalHostName "new-name" && dscacheutil -flushcache
+```
+
+Shortcut URLs use the lowercase form (`<name>.local`).
+
+## Step 2: Server URLs
+
+```bash
+uv run python -c "
+from physiclaw.core.bridge import bridge_base_urls
+p, f = bridge_base_urls(8048)
+if p != f: print(f'Recommended: {p}')
+print(f'Fallback (IP): {f}')
+"
+```
+
+Use `Recommended` in the Shortcuts. Fallback to the IP only if the network blocks mDNS.
+
+## Step 3: "PhysiClaw Tap" Shortcut (take screenshot)
+
+Tell the user:
+
+> Shortcuts app → **+**:
+>
+> 1. Add **"Take Screenshot"**
+> 2. Rename to **"PhysiClaw Tap"** → Done
+
+Wait for confirmation.
+
+## Step 4: "PhysiClaw Screenshot" Shortcut (upload latest)
+
+Tell the user (replace `HOST` with the URL from Step 2):
+
+> Shortcuts app → **+**:
+>
+> 1. Add **"Get Latest Screenshots"**
+> 2. Add **"Get Contents of URL"**:
+>    - URL: `http://HOST/api/bridge/screenshot`
+>    - Show More → Method **POST**, Request Body **File**, File → **Screenshots** variable
+> 3. Rename to **"PhysiClaw Screenshot"** → Done
+
+Wait for confirmation.
+
+## Step 5: "PhysiClaw Clipboard" Shortcut (sync clipboard)
+
+Tell the user (same `HOST`):
+
+> Shortcuts app → **+**:
+>
+> 1. Add **"Get Contents of URL"** → URL `http://HOST/api/bridge/clipboard` (GET)
+> 2. Add **"Copy to Clipboard"** → input = **Contents of URL**
+> 3. Rename to **"PhysiClaw Clipboard"** → Done
+
+Wait for confirmation.
+
+## Step 6: AssistiveTouch (skip if configured)
+
+Tell the user:
+
+> Settings → Accessibility → Touch → **AssistiveTouch ON**. Custom Actions:
+>
+> - **Single-Tap** → Shortcut → PhysiClaw Tap
+> - **Double-Tap** → Shortcut → PhysiClaw Screenshot
+> - **Long Press** → Shortcut → PhysiClaw Clipboard
+
+Wait for confirmation.
+
+## Step 7: Test
+
+Tell the user:
+
+> Tap AssistiveTouch **once** (take screenshot), then **twice** (upload latest).
+
+Record the baseline, then poll for a new file:
+
+```bash
+baseline=$(ls -t data/phone/screenshot/ 2>/dev/null | head -1)
+for i in $(seq 1 25); do
+  newest=$(ls -t data/phone/screenshot/ 2>/dev/null | head -1)
+  [ -n "$newest" ] && [ "$newest" != "$baseline" ] && { echo "✓ $newest"; open "data/phone/screenshot/$newest"; exit 0; }
+  sleep 1
+done
+echo "✗ no upload in 25s"
+```
+
+If nothing arrives:
+
+- Run the Shortcut manually (Shortcuts app → ▶) — check for errors.
+- From the Mac, `ping "$(scutil --get LocalHostName).local"` — if it fails from the phone's network, mDNS is blocked; swap both Shortcut URLs to the IP fallback from Step 2.
+- Ensure at least one screenshot exists in Photos.
+
+## Done
+
+> AssistiveTouch: single tap = take screenshot, double tap = upload latest, long press = clipboard fetch. If a Shortcut breaks after a network change, rerun this setup to get the new URL and paste it into each Shortcut's "Get Contents of URL" step.

physiclaw-0.0.1/.claude/skills/setup/SKILL.md

@@ -0,0 +1,15 @@
+---
+name: setup
+description: Connect the robotic arm and camera, then calibrate. Required before using any PhysiClaw MCP tools.
+allowed-tools: Bash
+---
+
+# Setup
+
+```bash
+uv run python scripts/setup.py           # interactive, default
+uv run python scripts/setup.py -y        # auto mode, skip prompts
+uv run python scripts/setup.py --trace   # add edge-trace visual check at end
+```
+
+Fails with non-zero exit and prints which step failed. Fix the physical setup and rerun.

physiclaw-0.0.1/.claude/skills/setup-vision-models/SKILL.md

@@ -0,0 +1,38 @@
+---
+name: setup-vision-models
+description: Download and prepare OmniParser icon detection model for the screenshot() tool. One-time setup — installs temporary deps, converts model, then cleans up.
+allowed-tools: Bash, Read
+---
+
+# Setup Vision Models
+
+One-time setup for icon detection and OCR in `screenshot()`. RapidOCR and onnxruntime are already in project dependencies — this only sets up the OmniParser ONNX model.
+
+## Step 1: Check
+
+```bash
+ls data/model/omniparser_icon_detect/model.onnx 2>/dev/null && echo "OK" || echo "MISSING"
+```
+
+If OK, tell the user it's already set up and stop.
+
+## Step 2: Install, convert, clean up
+
+```bash
+uv sync --group convert
+uv run python scripts/download_omniparser.py
+uv sync
+```
+
+## Step 3: Verify
+
+```bash
+uv run python -c "
+from physiclaw.core.vision.icon_detect import IconDetector
+from physiclaw.core.vision.ocr import OCRReader
+IconDetector(); OCRReader()
+print('OK')
+"
+```
+
+Tell the user setup is complete.

physiclaw-0.0.1/.claude/skills/wechat/SKILL.md

@@ -0,0 +1,100 @@
+---
+name: wechat
+description: Use when the task involves WeChat / 微信 — reading owner IM, sending a chat reply, finding a contact by name, or any "check messages" / "reply to <name>" request. NOT for Messages / SMS / Telegram / Signal, not for phone calls or FaceTime.
+---
+
+# WeChat (微信)
+
+The owner's primary IM.
+
+## Tool choice
+
+High-frequency app. Use `peek()` to check chat status — the camera view reliably reads Chinese text bubbles.
+
+**Never act on a chat-list preview.** Rows on the Chats tab show only the most recent message per contact, and it's truncated. To read the owner's actual messages, you MUST tap the row and enter the thread, then peek there.
+
+## Flow — read messages
+
+1. If already in the right chat, skip to step 5.
+2. From Home, tap the WeChat dock icon.
+3. If not on the **Chats** tab, tap it.
+4. **Tap the target 1:1 contact's row** — this step is non-skippable. A chat-list preview is truncated and hides earlier messages if the owner sent multiple since your last reply.
+5. Read the new messages. The thread opens at the bottom; if the topmost visible bubble doesn't connect to your last reply (or looks like a new unrelated message), swipe down on the bubble area to scroll up until you see the first bubble after your last reply.
+
+## Flow — voice message
+
+Use the convert-to-text option, then `peek()` the transcript that renders under the bubble.
+
+Reply with your reading + planned action and wait for OK before executing. ASR mishears names, amounts, addresses — never act on a voice instruction unconfirmed.
+
+## Flow — send a message
+
+Two states: **keyboard hidden** (input bar at bottom) and **keyboard visible** (shifted up, send key on keyboard).
+
+1. Confirm right contact in the chat header.
+2. If keyboard is hidden, tap the input box (keyboard-hidden bbox) — the keyboard opens and the input shifts to the keyboard-visible row.
+3. If the input has stale text, tap **backspace** until empty.
+4. `send_to_clipboard(text)`, then long-press the input box (keyboard-visible bbox).
+5. Tap **Paste** in the popup.
+6. Tap the keyboard's **send** key — **not** the (+) on the input bar.
+7. Hide the keyboard (see Hide keyboard below).
+8. Confirm the bubble appeared in the chat.
+9. Tap the back arrow `<` (top-left) to return to the Chats list — leaves WeChat in its main state for the next wake.
+
+### Fast path — `sequence` (5 steps)
+
+When you're already on the right 1:1 chat, the input is clean, and the keyboard is hidden, collapse steps 2–6 into one `sequence` call:
+
+```python
+sequence(
+    step1 = {"tool_name": "tap",               "arg": [0.100, 0.910, 0.700, 0.960]},
+    step2 = {"tool_name": "send_to_clipboard", "arg": "<your text>"},
+    step3 = {"tool_name": "long_press",        "arg": [0.100, 0.575, 0.700, 0.625]},
+    step4 = {"tool_name": "tap",               "arg": [0.050, 0.530, 0.220, 0.570]},
+    step5 = {"tool_name": "tap",               "arg": [0.752, 0.864, 0.992, 0.917]},
+)
+```
+
+After the sequence: hide keyboard, confirm bubble, tap back arrow (steps 7–9 above).
+
+## Fixed elements
+
+Bboxes `[left, top, right, bottom]` in 0-1 screen coords. Re-peek if a row looks off — banners shift the layout.
+
+**WeChat dock icon (Home Screen):** `[0.294, 0.891, 0.474, 0.967]`
+
+### Bottom nav
+
+| Tab               | Bbox                            |
+| ----------------- | ------------------------------- |
+| Chats (微信)      | `[0.070, 0.945, 0.190, 0.965]`  |
+| Contacts (通讯录) | `[0.324, 0.947, 0.437, 0.962]`  |
+| Discover (发现)   | `[0.578, 0.947, 0.684, 0.963]`  |
+| Me (我)           | `[0.840, 0.945, 0.940, 0.965]`  |
+
+### Chats list rows
+
+Each row is **0.08 tall**. Derive the Nth row from the 1st by adding `0.08 × (N-1)` to the y values.
+
+| Row | Bbox                            |
+| --- | ------------------------------- |
+| 1st | `[0.160, 0.180, 0.940, 0.260]`  |
+| 2nd | `[0.160, 0.260, 0.940, 0.340]`  |
+
+### Chat page
+
+| Element         | When                | Bbox                            |
+| --------------- | ------------------- | ------------------------------- |
+| Back arrow `<`  | always              | `[0.016, 0.066, 0.082, 0.102]`  |
+| Contact name    | always              | `[0.400, 0.060, 0.600, 0.100]`  |
+| Text input area | keyboard hidden     | `[0.100, 0.910, 0.700, 0.960]`  |
+| Text input area | keyboard visible    | `[0.100, 0.575, 0.700, 0.625]`  |
+| Paste button    | long-pressing input | `[0.050, 0.530, 0.220, 0.570]`  |
+
+The send key on the keyboard and the backspace key are standard iPhone keyboard positions — see PHYSICLAW.md "iPhone keyboard bboxes".
+
+### Hide keyboard
+
+A small upward swipe in the empty chat scroll area dismisses the keyboard without scrolling the just-sent bubble off-screen.
+
+`swipe(bbox=[0.300, 0.300, 0.700, 0.500], direction="up", size="s")`