@swmansion/argent 0.32.0 → 0.32.1-next.1

package/skills/argent-device-interact/SKILL.md

@@ -1,401 +1,99 @@
 ---
 name: argent-device-interact
-description: Interact with an iOS simulator, Android emulator, or Chromium (CDP) app using argent MCP tools. Use when tapping UI elements, performing gestures, scrolling/swiping, typing text, pressing hardware buttons, launching apps, opening URLs, taking screenshots, waiting for an element to appear or disappear, or checking visible app state after interactions. Not for TV targets.
+description: Interact with an iOS simulator or iPhone, Android emulator or phone, or Chromium (CDP) app using argent MCP tools. Use when tapping UI elements, performing gestures, scrolling/swiping, typing text, pressing hardware buttons, launching apps, opening URLs, taking screenshots, waiting for an element to appear or disappear, or checking visible app state after interactions. Not for TV targets.
 ---
 
-## Unified tool surface
-
-All interaction tools below accept a `udid` parameter and auto-dispatch iOS vs Android based on its shape (UUID → iOS simulator, `chromium-cdp-<port>` → Chromium (CDP) app, anything else → Android adb serial). You use the same tool names on every platform.
-
-**Chromium (CDP) app** = an Electron app or a Chromium-family browser (Chrome/Brave/Edge) exposing a Chrome DevTools Protocol endpoint. The same describe/tap/keyboard/screenshot surface drives it, but scrolling, tabs, cookies and storage differ: **read `references/chromium.md` before driving a `chromium` target.**
+Each tool's own description carries its parameters, defaults, return shape and examples. This skill covers choosing between tools and the gotchas those descriptions don't state.
 
 ## 1. Before You Start
 
-If you delegate simulator tasks to sub-agents, make sure they have MCP permissions.
-
-Use `list-devices` to get a target id. Results are tagged with `platform` (`ios`, `android`, or `chromium`); booted/ready devices come first. For a `harmony` target, read `references/harmonyos.md` before driving it. Pick the first entry that matches the platform you need; if none are ready, call `boot-device` with `udid` (iOS), `avdName` (Android), or `electronAppPath` (boots an Electron app as a `chromium` device). A Chromium browser already running with a CDP port shows up directly; no `boot-device` needed. See `argent-ios-simulator-setup` / `argent-android-emulator-setup` for full setup flow.
-
-**Load tool schemas before first use.** Gesture tools (`gesture-tap`, `gesture-swipe`, `gesture-pinch`, `gesture-rotate`, `gesture-custom`) may be deferred: their parameter schemas are not loaded until fetched. Always use ToolSearch to load the schemas of all gesture tools you plan to use **before** calling any of them. If you skip this step, parameters may be coerced to strings instead of numbers, causing validation errors.
-
-## 2. Best Practices
-
-1. **Always refer to tapping_rule** from your argent.md rule before tapping.
-2. Before performing interactions, consider whether they can be **dispatched sequentially** - more on that in `run-sequence`.
-3. **Use `gesture-swipe` for lists/scrolling**, not `gesture-custom`, unless you need non-linear movement. On Chromium use `gesture-scroll` instead; `gesture-swipe` is touch-only. Consider whether you need multiple swipes, if yes - use `run-sequence`. Pass `momentum: false` when the swipe should decelerate before ending for a precise movement.
-4. **Tap a text field before typing**, then use `keyboard` to enter text.
-5. **Coordinates are normalized**: always 0.0–1.0, not pixels.
-6. **For app navigation, use the element tree returned after each action** (`--- Elements after action (describe) ---`); call `describe` only when no fresh tree is available for the current screen. It works on any screen without app restart. Do not navigate from screenshot pixels on regular in-app screens unless the tree failed to expose a reliable target. Use `native-describe-screen` only when you need app-scoped UIKit properties.
-
-## 3. Opening Apps
-
-**Never navigate to an app by tapping home-screen icons.** Use `launch-app` or `open-url`: they are instant and reliable.
-
-### launch-app: by bundle ID
-
-```json
-{ "udid": "<UDID>", "bundleId": "com.apple.MobileSMS" }
-```
-
-Common IDs: `com.apple.MobileSMS` (Messages), `com.apple.mobilesafari` (Safari), `com.apple.Preferences` (Settings), `com.apple.Maps`, `com.apple.mobileslideshow` (Photos), `com.apple.MobileAddressBook` (Contacts)
-
-### open-url: by URL scheme
-
-```json
-{ "udid": "<UDID>", "url": "messages://" }
-```
-
-Common schemes: `messages://`, `settings://`, `maps://?q=<query>`, `tel://<number>`, `mailto:<address>`, `https://...` (Safari)
-
-## 4. Choosing the Right Tool
-
-| Action            | Tool                | Notes                                                             |
-| ----------------- | ------------------- | ----------------------------------------------------------------- |
-| Multiple actions  | `run-sequence`      | Batch steps in one call (no intermediate screenshots)             |
-| Open an app       | `launch-app`        | **Always: never tap home-screen icons**                           |
-| Restart an app    | `restart-app`       | Terminate and relaunch by bundle ID                               |
-| Open URL/scheme   | `open-url`          | Web pages, deep links, URL schemes                                |
-| Single tap        | `gesture-tap`       | Buttons, links, checkboxes                                        |
-| Scroll/swipe      | `gesture-swipe`     | Straight-line scroll or swipe                                     |
-| Scroll (Chromium) | `gesture-scroll`    | Wheel-based; deltas are window fractions, positive deltaY = down  |
-| Drag (Chromium)   | `gesture-drag`      | Sliders, drag-and-drop, text selection                            |
-| Long press        | `gesture-custom`    | Context menus, drag start                                         |
-| Drag & drop       | `gesture-custom`    | Complex drag interactions                                         |
-| Pinch/zoom        | `gesture-pinch`     | Two-finger pinch with auto-interpolation                          |
-| Rotation          | `gesture-rotate`    | Two-finger rotation with auto-interpolation                       |
-| Custom gesture    | `gesture-custom`    | Arbitrary touch sequences, optional interpolation                 |
-| Hardware key      | `button`            | Home, back, power, volume, appSwitch, actionButton                |
-| Type text         | `keyboard`          | Every platform. Text or one named key per call, never both        |
-| Paste text        | `paste`             | Only where a user would paste (OTP code, long link). Sim/emu only |
-| Rotate device     | `rotate`            | Orientation changes                                               |
-| Fold device       | `fold`              | Foldable simulator/emulator: closed / half-open / open, or angle  |
-| Shake device      | `shake`             | Shake handlers (sim/emu only), Undo-typing prompt, RN dev menu    |
-| Wait for UI       | `await-ui-element`  | Block until an element is visible/hidden/exists/contains text     |
-| Wait for idle     | `await-screen-idle` | Block until a non-empty screen tree stops changing                |
-
-## 5. Finding Tap Targets
-
-IMPORTANT. When moved to a different screen after an action or do not know the coordinates of component, **always** perform proper discovery first.
-
-| App type                          | Discovery tool            | What it returns                                                                                                                                                                                                                                    |
-| --------------------------------- | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| Target app discovery              | `describe`                | Accessibility element tree for the current device screen (iOS AX-service, Android uiautomator, or Chromium DOM walker) with normalized frame coordinates. Works on any app, system dialogs, and Home screen; no app restart or `bundleId` required |
-| React Native                      | `debugger-component-tree` | React component tree with names, text, testID, and (tap: x,y)                                                                                                                                                                                      |
-| App-scoped native                 | `native-describe-screen`  | Low-level app-scoped accessibility elements with normalized and raw coordinates; requires `bundleId`                                                                                                                                               |
-| Permission / system modal overlay | `describe`                | `describe` detects system dialogs automatically and returns dialog buttons with tap coordinates. Fall back to `screenshot` only if `describe` does not expose the controls                                                                         |
-| Final visual fallback             | `screenshot`              | Use only when discovery tools cannot inspect the current UI reliably. Do not derive routine in-app navigation targets from screenshots                                                                                                             |
-
-Point follow-up native diagnostics after you already have a candidate point:
-
-- `native-user-interactable-view-at-point`: deepest native view that would receive touch at a known raw iOS point; requires `bundleId`
-- `native-view-at-point`: deepest visible native view at a known raw iOS point; requires `bundleId`
-
-### If `describe` Tool Fails
-
-Read the exact error and choose the action that matches it:
-
-- Error mentions `ax-service` not available or daemon startup failure:
-  the ax-service daemon could not start. Check that the simulator is booted. Use `screenshot` as a temporary fallback, or use `native-describe-screen` with an explicit `bundleId` if the app has native devtools injected.
-- `describe` returns an empty element list:
-  the screen may be blank, loading, or showing content without accessibility labels. Use `screenshot` to see what is visible, then retry after the content has loaded.
-- `describe` succeeds but is not detailed enough for a React Native app:
-  use `debugger-component-tree` next.
-- You need app-scoped inspection with full UIKit properties (`accessibilityIdentifier`, `viewClassName`):
-  use `native-describe-screen` with an explicit `bundleId`. This requires native devtools (dylib) injection.
-- You already have a candidate point and want to confirm what would actually receive touch:
-  use `native-user-interactable-view-at-point`. Use `native-view-at-point` when you want the visually deepest view instead of the hit-test target.
-
-## 6. Tool Usage
-
-### gesture-tap: Single tap at a point
-
-```json
-{ "udid": "<UDID>", "x": 0.5, "y": 0.5 }
-```
-
-Coordinates: `0.0` = left/top, `1.0` = right/bottom.
-
-Before tapping near the bottom of the screen in React Native apps, check that "Open Debugger to View Warnings" banners are not visible: tapping them breaks the debugger connection. Close them with the X icon if present.
-
-### gesture-swipe: Straight-line gesture
-
-```json
-{ "udid": "<UDID>", "fromX": 0.5, "fromY": 0.7, "toX": 0.5, "toY": 0.3 }
-```
-
-Swipe **up** (`fromY > toY`) = scroll content **down**. Default duration: 300ms. Optional: `"durationMs": 500` for slower swipe.
-
-`"momentum"` defaults to `true` (a natural flinging swipe). Pass `"momentum": false` for a momentum-free swipe: the finger decelerates into the end point, resulting in little to no fling. It needs `durationMs` of at least 150 and is rejected below it.
-
-### gesture-pinch: Two-finger pinch
-
-```json
-{ "udid": "<UDID>", "centerX": 0.5, "centerY": 0.5, "startDistance": 0.2, "endDistance": 0.6 }
-```
-
-All values are normalized 0.0–1.0 (fractions of screen, not pixels), same as all other gesture tools. `startDistance: 0.2` means fingers start 20% of the screen apart; `endDistance: 0.6` means they end 60% apart. `startDistance < endDistance` = pinch out (zoom in). `startDistance > endDistance` = pinch in (zoom out). Defaults: `angle: 0` (horizontal), `durationMs: 300`. Optional: `"angle": 90` for vertical axis, `"durationMs": 500` for slower pinch, `"endCenterX"`/`"endCenterY"` to let the centroid drift to a new center over the gesture (omitted = fixed center).
-
-### gesture-rotate: Two-finger rotation
-
-```json
-{
-  "udid": "<UDID>",
-  "centerX": 0.5,
-  "centerY": 0.5,
-  "radius": 0.15,
-  "startAngle": 0,
-  "endAngle": 90
-}
-```
-
-All positions and radii are normalized 0.0–1.0 (fractions of screen, not pixels). `radius: 0.15` means each finger is 15% of the screen away from center. `endAngle > startAngle` = clockwise. Default duration: 300ms. Optional: `"durationMs": 500` for slower rotation, and `"radiusX"`/`"radiusY"` (fractions of screen width/height; give both: they override `radius`) with `radiusX·width = radiusY·height` for a physically circular orbit; a single `radius` traces a physical ellipse on a non-square screen, coupling a slight pinch into the turn.
-
-### gesture-custom: Custom touch sequence
-
-For long-press, drag-and-drop, and other complex sequences, see `references/gesture-examples.md`. Set `"interpolate": 10` to auto-generate smooth intermediate Move events between keyframes.
-
-### button: Hardware button press
-
-```json
-{ "udid": "<UDID>", "button": "home" }
-```
-
-Values: `home`, `back`, `power`, `volumeUp`, `volumeDown`, `appSwitch`, `actionButton`
-
-### keyboard: Type text or press special keys
-
-```json
-{ "udid": "<UDID>", "text": "search query" }
-```
-
-One call does one action. `text` and `key` are mutually exclusive, and a call that carries both is rejected with nothing typed. To type and then submit, send two `keyboard` steps in one `run-sequence` (§ 8): `{ "text": "search query" }`, then `{ "key": "enter" }`. Two separate calls do the same work, but cost an extra round-trip.
-
-**React Native: the first tap after typing may only close the keyboard.** Inside a scroll view with the default `keyboardShouldPersistTaps="never"`, React Native uses a tap outside the focused field to dismiss the keyboard and does not pass it on (with `"handled"` or `"always"` the tap reaches the button), so a button tapped right after typing may need a second tap; check the returned tree. When the field submits on return, `{ "key": "enter" }` avoids the extra tap.
-
-Special keys: `enter`, `escape`, `backspace`, `tab`, `space`, `arrow-up`, `arrow-down`, `arrow-left`, `arrow-right`, `f1`–`f12`. Optional: `"delayMs": 100` between keystrokes (default 50ms); applies to the iOS simulator and Chromium; it is ignored on Android phones/tablets (typed via `adb input text`, no per-key cadence), on Vega, and on TV targets.
-
-**Typing secrets.** To enter a credential without its plaintext ever entering your context, transcript, or logs, use a secret placeholder in `text` (works in `keyboard`, `paste`, `run-sequence` keyboard steps, and e2e `type` steps):
+- Get a target id from `list-devices`; if none is ready, `boot-device` one. Full setup flow: `argent-ios-simulator-setup` / `argent-ios-device-setup` / `argent-android-emulator-setup`. The same tool names drive iOS, Android and Chromium; each takes the `udid` from `list-devices`.
+- Physical iPhone (`kind: "device"`): read `argent-ios-device-interact` first. Only the tools it lists exist there, and `describe` sees only the app `launch-app` registered.
+- Read the matching reference before driving these targets: `references/chromium.md` for a `chromium` target, `references/harmonyos.md` for a `harmony` target, `references/foldables.md` for a device `list-devices` marks `foldable: true` (or before calling `fold`).
+- Load a deferred tool's schema with ToolSearch before its first call (e.g. `gesture-pinch`, `gesture-rotate`, `gesture-custom`); without it, numbers go out as strings and fail validation.
+- If you delegate device tasks to sub-agents, make sure they have MCP permissions.
 
-```json
-{ "udid": "<UDID>", "text": "{{secret:APP_PASSWORD}}" }
-```
+## 2. Choosing the Right Tool
 
-Where the value is read from, and the rules for using a placeholder (including not screenshotting the field afterwards) are in `references/secrets.md`. Read it before typing any credential.
+| Action                         | Tool                | Notes                                                          |
+| ------------------------------ | ------------------- | -------------------------------------------------------------- |
+| Several known steps            | `run-sequence`      | One call, one screenshot at the end (§5)                       |
+| Open an app                    | `launch-app`        | **Never tap home-screen icons**                                |
+| Restart an app                 | `restart-app`       | Clean in-memory state                                          |
+| Open URL / deep link           | `open-url`          | Web pages, deep links, URL schemes                             |
+| Tap                            | `gesture-tap`       | Buttons, links, checkboxes                                     |
+| Scroll / swipe                 | `gesture-swipe`     | Touch targets; not Chromium                                    |
+| Scroll (Chromium)              | `gesture-scroll`    | Mouse wheel                                                    |
+| Drag (Chromium)                | `gesture-drag`      | Sliders, drag-and-drop, text selection                         |
+| Long press, drag & drop        | `gesture-custom`    | Also any other non-linear touch sequence                       |
+| Pinch / zoom                   | `gesture-pinch`     |                                                                |
+| Two-finger rotation            | `gesture-rotate`    |                                                                |
+| Hardware key                   | `button`            | Home, back, power, volume, appSwitch, actionButton             |
+| Type text or press a named key | `keyboard`          | Every platform; text or one key per call (§6)                  |
+| Paste text                     | `paste`             | Only where a user would paste (§6). Sim/emu only               |
+| Rotate device                  | `rotate`            | Orientation changes                                            |
+| Fold device                    | `fold`              | Foldables only: `references/foldables.md`                      |
+| Shake device                   | `shake`             | Shake handlers (sim/emu only), Undo-typing prompt, RN dev menu |
+| Wait for UI                    | `await-ui-element`  | Block until an element is visible/hidden/exists/contains text  |
+| Wait for idle                  | `await-screen-idle` | Block until a non-empty screen tree stops changing (§4)        |
 
-### paste: Paste text into the focused field
+## 3. Finding Tap Targets
 
-```json
-{ "udid": "<UDID>", "text": "482913" }
-```
+After an action moves you to a different screen, or whenever you don't know an element's coordinates, discover before tapping. Interaction results already carry the element tree (`--- Elements after action (describe) ---`): use it, and call a discovery tool only when no fresh tree covers the current screen.
 
-Puts `text` on the **device** clipboard and triggers the platform's paste shortcut. iOS simulator and Android emulator only; a TV target, a physical device, Chromium and Vega are rejected.
+| Need                                           | Tool                                                                                                                           |
+| ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
+| Any screen: app, system dialog, Home screen    | `describe`                                                                                                                     |
+| App-scoped UIKit properties (iOS simulator)    | `native-describe-screen` (needs `bundleId`)                                                                                    |
+| What sits at a known raw point (iOS simulator) | `native-user-interactable-view-at-point` (the view that receives touch) or `native-view-at-point` (the visually deepest view)  |
+| Final visual fallback                          | `screenshot`, only when the tools above cannot expose the control. Do not derive routine in-app navigation targets from pixels |
 
-`paste` is **not** a faster `keyboard`. `keyboard` types the way a user types and stays the default for every text entry: a search query, a login, a form field. Reach for `paste` only where a real user would paste: a 2FA / OTP code copied from another app, a long link or token, or to test how the app handles pasted input. It also carries what `keyboard` can't type on a given platform (multi-line text, non-ASCII on Android), but that alone is not a reason to paste; ask whether the user would.
+If `describe` returns an empty tree, follow its `hint`; with no hint, the screen is blank, loading or unlabeled: check the attached screenshot and retry once content has loaded.
 
-Tap the field first so it has focus; pasting with no focused field is a silent no-op, as with `keyboard`. `text` accepts the same `{{secret:<NAME>}}` placeholders as `keyboard`, with the same auto-screenshot skip.
+For a React Native testID or component name that `describe` does not show, use `debugger-component-tree`; take tap points from `describe`.
 
-### rotate: Change orientation
+**React Native:** LogBox toasts sit over the bottom of the screen and catch taps meant for what is under them. Argent hides them when a `debugger-*` tool connects and after `debugger-reload-metro`; after `restart-app`, call a `debugger-*` tool again. After any other reload they come back: close each by tapping the right end of its `describe` frame (the X), not its middle.
 
-```json
-{ "udid": "<UDID>", "orientation": "LandscapeLeft" }
-```
+**System and permission dialogs:** on a simulator or emulator, `describe` returns a visible dialog's buttons with tap coordinates; keep using it for in-app modals too. Fall back to `screenshot` only if it does not expose the controls: then tap one obvious button (`Allow`, `OK`, `Don't Allow`, `Not Now`, `Continue`) at a time, check the returned screenshot, and go back to `describe` once the dialog is gone.
 
-Values: `Portrait`, `LandscapeLeft`, `LandscapeRight`, `PortraitUpsideDown`
+> **Prefer the dialog over the Settings tool.** When the app triggers its own permission prompt, answering it is the real user path. Use `settings-permissions` only when you can't make the change through the app: set a permission _before_ the app asks, re-enable one the user denied (iOS won't re-prompt), or reset it so the prompt reappears. See `argent-settings-permissions`.
 
-On an unfolded foldable simulator, the value sets the orientation of the device, not of the UI. `Portrait` gives a landscape UI. `LandscapeLeft` gives a portrait UI.
+## 4. Waiting: `await-ui-element` and `await-screen-idle`
 
-### fold: Fold or unfold a foldable device
+**Never poll `screenshot`/`describe` in a loop to wait for something.** `await-ui-element` blocks server-side on the same tree `describe` reads. It has no bare-timer mode; for a plain pause, use your own harness sleep. Its description covers conditions, selector matching and defaults. What it does not tell you:
 
-```json
-{ "udid": "<UDID>", "posture": "open" }
-```
-
-Give `posture` (`closed`, `half-open` or `open`) or `angle` (0–180). Do not give both.
-
-Use `fold` only on a foldable iOS simulator (the iPhone Duo) or Android emulator (a Pixel Fold or generic foldable AVD). `list-devices` marks them with `foldable: true`. Other devices reject the call.
-
-Closed, the cover panel shows the UI. Half-open and open, the inner panel shows the UI. All tools use the active panel (the panel that shows the UI), also after a fold made outside argent. `fold` returns when the device accepts input again, so the next tap lands. This is also true for the next step in `run-sequence`.
-
-Rules:
-
-- The coordinates change with the panel. Before you tap, read the element tree in the `fold` result, or run `describe` again. Do not use frames from before the fold.
-- The screenshot size changes with the panel. Keep one screenshot-diff baseline for each posture.
-- iOS: unfolded, the UI is landscape. The screenshot shows the UI turned 90 degrees, and the `describe` frames use the same axes, as on a rotated iPhone.
-- iOS: a fold between two angles that are not 0 or 180 can keep the current panel. The result names the active panel. To change panels, fold to `closed` or `open`.
-- Fold between gestures, not during a gesture. A gesture stays on the panel where it started.
-- Android: the result gives `deviceState` (`CLOSED`, `HALF_OPENED`, `OPENED`). A lock screen that the fold to closed shows is dismissed; a `warning` says when it cannot be, or when a lock screen is up after the fold. Unlock the device before the next action.
-- Android: only a Pixel Fold AVD has a cover panel. When a generic foldable AVD is closed, the emulator shows only the left part of the display. `describe` frames do not match tap coordinates. Argent can reliably tap only an element that covers the visible part, for example a full-width row. Use a Pixel Fold AVD to test the cover panel.
-- iOS: if argent cannot find the active panel, it uses the cover panel. The tool result then has a `warning`, and a screenshot-diff summary has a `panel:` line. Take a screenshot to see what the device shows, then do the check that the warning gives.
-
-### await-ui-element: Block until a UI element reaches a state
-
-**Never poll `screenshot`/`describe` in a loop to wait for something.** Use `await-ui-element`: it blocks server-side on the same tree `describe` reads. It has no bare-timer mode by design; for a plain pause, use your own harness sleep.
-
-```json
-{ "udid": "<UDID>", "condition": "visible", "selector": { "text": "Continue" } }
-```
-
-The tool's own description carries the conditions, selector matching, defaults and return shape. What it does not tell you:
-
-- A `hidden` check that succeeds **immediately** may be a false pass: its `note` then says the selector never matched anything at all. Treat that as a failed check and fix the selector; do not read it as "the element went away".
+- A `hidden` check that succeeds **immediately** may be a false pass: its `note` then says the selector never matched anything. Treat that as a failed check and fix the selector; it does not mean "the element went away".
 - The synthetic `ROOT` container `describe` prints is never matched, so a `role` like `AXGroup`/`html` won't trivially "match the screen".
-- To disambiguate a loose selector, pin the `role` to a text role like `StaticText`; that skips a same-named button.
-- On a `text` timeout the `note` quotes the text of the element the check actually read, so you can see which match it landed on.
-
-### await-screen-idle: Block until the screen stops changing
-
-Use after launch/navigation and before a raw tap, when an early-painted element may still be moving:
-
-```json
-{ "udid": "<UDID>", "timeoutMs": 3000, "minStableMs": 250 }
-```
-
-On local iOS, Android, and Chromium, the tool waits for a non-empty `describe` tree to stop changing. Continue only when `settled: true`. Pair it with a destination-specific `await-ui-element`; stillness does not identify a screen.
-
-Use it only for live diagnosis. Do not record it or put it in `run-sequence`. E2E tests use `await: { idle: true }`, which also compares pixels. This live tool can return during a presentation-layer animation.
-
----
-
-## 7. Screenshots
-
-Use the explicit `screenshot` tool only when:
-
-- You need the initial screen state before any action.
-- You are about to edit visible UI and need a baseline capture before making changes.
-- The auto-attached screenshot shows a transitional or loading frame.
-- You require extra context.
-- You want to check state after a delay (e.g. waiting for a network response).
-- A permission dialog, system alert, or native modal overlay is visible and `describe` did not expose reliable targets.
-
-When using `screenshot` for permission or native modal navigation:
-
-- Do not switch to screenshot-driven navigation just because a modal is visible. On regular app screens and in-app modals, keep using `describe`.
-- Prefer obvious, centered alert buttons such as `Allow`, `OK`, `Don't Allow`, `Not Now`, or `Continue`.
-- Tap one control at a time and inspect the returned auto-screenshot before doing anything else.
-- After the modal is dismissed, return to normal discovery with `describe`, `native-describe-screen`, or `debugger-component-tree`.
-
-> **Prefer the dialog over the Settings tool.** When the app triggers its own permission prompt, answering it here is the real user path; do that. Reach for the `settings-permissions` tool only when you can't get to the change through the app: pre-authorize/deny a permission _before_ the app asks, re-enable one the user already denied (iOS won't re-prompt), or reset it so the prompt reappears. See the `argent-settings-permissions` skill.
-
-Optional rotation parameter: `{ "udid": "<UDID>", "rotation": "LandscapeLeft" }`: rotates the capture without changing simulator orientation.
+- To disambiguate a loose selector, pin `role` to a text role like `StaticText`; that skips a same-named button.
 
-Screenshots are downscaled by default (30% of original resolution) to reduce context size. Use the normal downscaled screenshot for UI context and state checks. `scale` accepts values from 0.01 to 1.0, but do not use `scale: 1.0` as a general readability or tapping aid.
+`await-screen-idle` is for live diagnosis after a launch or navigation, before a raw tap on an element that may still be moving. Continue only when `settled: true`, and pair it with an `await-ui-element` for the destination: stillness does not identify a screen, and the tool can return during a presentation-layer animation. Do not record it in an e2e test; tests use `await: { idle: true }`, which also compares pixels.
 
-Use full-resolution screenshots only when saving baseline/current PNG files for comparison. In that case, suppress the image block so the full-size PNG is not loaded into agent context:
+## 5. Action Sequencing with `run-sequence`
 
-```json
-{ "udid": "<UDID>", "scale": 1.0, "includeImageInContext": false }
-```
+Use `run-sequence` when every step is known in advance; if a step depends on what the previous one shows, use separate calls. Its description lists the allowed tools and their arguments. Typical batches:
 
-For visual regression checks, before/after screenshot comparisons, and detailed `screenshot-diff` parameter guidance, use the `argent-screenshot-diff` skill. Keep this skill focused on device interaction mechanics and screenshot capture.
+- "scroll to bottom", "scroll until X": 3-5 swipes
+- "clear and retype a field": triple-tap to select all, then type the new value
+- typing then submitting: a `keyboard` text step, then a `keyboard` `enter` step (one call cannot carry both)
+- "submit form": fill every field, then tap submit
+- "go back to X": a known tap path
 
-### Troubleshooting
+Gate a step that depends on a screen transition with an `await-ui-element` step rather than a fixed `delayMs`. If its condition is not met before the timeout, the sequence **stops there** and the later steps do not run, so a tap can't fire against a screen that never appeared. Any failing step also stops the sequence, with partial results.
 
-| Problem                 | Solution                                                      |
-| ----------------------- | ------------------------------------------------------------- |
-| Screenshot times out    | Restart the simulator-server via `stop-simulator-server` tool |
-| No booted iOS simulator | Call `boot-device` with the iOS `udid`                        |
-| No ready Android device | Call `boot-device` with `avdName`                             |
+## 6. Text Entry and Secrets
 
----
-
-## 8. Action Sequencing with `run-sequence`
-
-Use `run-sequence` to batch multiple interaction steps into **a single tool call**. Only one screenshot is returned, after all steps complete.
-
-Do **not** use `run-sequence` when any step depends on observing the result of a previous step.
-
-### Use cases
-
-- "scroll to bottom", "scroll to top", "scroll until X" -> sequence 3-5 scrolls
-- form interactions, "clear and retype field" -> triple-tap to select all, then type the new value
-- "submit form" → fill all fields in sequence, tap submit
-- "go back to X" → defined tap sequence for the navigation
-
-### Allowed tools inside `run-sequence`
-
-`gesture-tap`, `gesture-swipe`, `gesture-scroll`, `gesture-drag`, `gesture-custom`, `gesture-pinch`, `gesture-rotate`, `button`, `keyboard`, `paste`, `rotate`, `shake`, `fold`, `tv-remote`, `await-ui-element`
-
-The `udid` is shared; do **not** include it in each step's `args`. Optional `delayMs` per step (default 100ms).
-
-Add an `await-ui-element` step to gate a later tap on a screen transition (e.g. tap → wait for the next screen's button → tap it). If its condition is **not** met before the timeout, the sequence stops at that step and the following steps do **not** run, so a mistimed tap can't fire against a screen that never settled.
-
-### Examples
-
-Scroll down three times:
-
-```json
-{
-  "udid": "<UDID>",
-  "steps": [
-    { "tool": "gesture-swipe", "args": { "fromX": 0.5, "fromY": 0.7, "toX": 0.5, "toY": 0.3 } },
-    { "tool": "gesture-swipe", "args": { "fromX": 0.5, "fromY": 0.7, "toX": 0.5, "toY": 0.3 } },
-    { "tool": "gesture-swipe", "args": { "fromX": 0.5, "fromY": 0.7, "toX": 0.5, "toY": 0.3 } }
-  ]
-}
-```
+- Tap the field first. Typing or pasting with no focused field is a silent no-op.
+- `paste` is **not** a faster `keyboard`. `keyboard` types the way a user types and stays the default for every text entry. Use `paste` only where a real user would paste: a 2FA / OTP code from another app, a long link or token, or testing how the app handles pasted input. That it can carry text `keyboard` can't type on a platform (multi-line text, non-ASCII on Android) is not a reason on its own.
+- **Credentials:** type them as a `{{secret:<NAME>}}` placeholder in `text` (`keyboard`, `paste`, `run-sequence` keyboard steps, e2e `type` steps), never as plaintext, and never ask the user for the value. Do **not** `describe` or `screenshot` a non-secure field you just filled with a secret: submit or navigate away first, then verify the resulting screen.
 
-Type into a focused field and submit. This is the only way to mix text and a key, because one `keyboard` call cannot carry both:
-
-```json
-{
-  "udid": "<UDID>",
-  "steps": [
-    { "tool": "keyboard", "args": { "text": "hello world" } },
-    { "tool": "keyboard", "args": { "key": "enter" } }
-  ]
-}
-```
-
-Tap a known button, then scroll down:
-
-```json
-{
-  "udid": "<UDID>",
-  "steps": [
-    { "tool": "gesture-tap", "args": { "x": 0.5, "y": 0.15 } },
-    {
-      "tool": "gesture-swipe",
-      "args": { "fromX": 0.5, "fromY": 0.7, "toX": 0.5, "toY": 0.3 },
-      "delayMs": 300
-    }
-  ]
-}
-```
-
-Tap, wait for the next screen, then act on it: the `await-ui-element` step **gates** the tap after it:
-
-```json
-{
-  "udid": "<UDID>",
-  "steps": [
-    { "tool": "gesture-tap", "args": { "x": 0.5, "y": 0.9 } },
-    {
-      "tool": "await-ui-element",
-      "args": { "condition": "visible", "selector": { "text": "Continue" } }
-    },
-    { "tool": "gesture-tap", "args": { "x": 0.5, "y": 0.5 } }
-  ]
-}
-```
-
-Prefer this over a fixed `delayMs` when a step depends on a screen transition: it adapts to real load time, and if the condition is not met before the timeout the sequence **stops there** so the next tap can't fire against a screen that never settled.
-
-Stops on the first error (or unmet `await-ui-element` condition) and returns partial results.
-
----
-
-## 9. Platform-specific notes
-
-### Android
+## 7. Screenshots
 
-- **Metro reachability**: run `adb reverse tcp:8081 tcp:8081` on the device before the RN app starts, or Metro won't be reachable from the device. See `argent-metro-debugger` for the full workflow. Re-run if the device restarts. If `list-devices` returns a `hint` about the adb server, prefix the command as the hint shows.
-- **First-launch permission prompts**: `reinstall-app` on Android always installs with `-g` so runtime permissions are pre-granted on first launch; no flag to pass.
-- **Locked screen / secure surfaces**: `describe` throws a clear error if it can't capture (keyguard, DRM, Play Integrity). Unlock the device or fall back to `screenshot`.
-- **APK vs .app in `reinstall-app`**: pass `.apk` absolute path on Android; `.app` directory on iOS.
+Interaction tools already attach a screenshot. Call `screenshot` for the initial state, for a baseline before you change visible UI, to check state after a delay, or when the attached capture shows a transitional or loading frame.
 
-### Chromium
+Keep the default downscaled capture for UI context and state checks; do not use `scale: 1.0` as a readability or tapping aid. Full resolution is only for saving baseline/current PNGs for comparison (`scale: 1.0` with `includeImageInContext: false`); for visual regression checks use the `argent-screenshot-diff` skill.
 
-See `references/chromium.md` (tabs, cookies/storage).
+If `screenshot` times out, restart the device transport with `stop-simulator-server`.
 
-### iOS
+## 8. Android notes
 
-_(no iOS-only gotchas collected here yet; add them as they come up)_
+- **First-launch permission prompts**: `reinstall-app` on Android installs with `-g`, so runtime permissions are pre-granted on first launch.

package/skills/argent-device-interact/references/chromium.md

@@ -2,23 +2,8 @@
 
 Everything in the related SKILL.md still applies; this page covers what differs.
 
-## Multi-tab / windows
+A Chromium (CDP) target is an Electron app or a Chromium-family browser (Chrome/Brave/Edge) exposing a Chrome DevTools Protocol endpoint. `describe`, `gesture-tap`, `keyboard` and `screenshot` work as on a phone. What differs:
 
-A Chromium device may have several tabs / BrowserWindows. Use `chromium-tabs` to `list` them (stable ids `t1`, `t2`, …, optional labels), open a `new` one, `select` which is active, or `close` one.
-
-Every other tool (`describe`, `gesture-tap`, `screenshot`, `debugger-evaluate`, `open-url`, …) acts on the **active** tab, so `chromium-tabs action=select` before driving a different tab.
-
-Note: a cross-process navigation (some redirects) can swap a tab's underlying CDP target; re-run `chromium-tabs action=list` to pick it up under a fresh id.
-
-## Cookies & storage
-
-`chromium-cookies` reads/writes cookies via the Network domain (so HttpOnly cookies are visible):
-
-- `action=get` (optionally scoped by `url`)
-- `set` (`name`, `value`, + `url`/`domain`, optional `secure`/`httpOnly`/`sameSite`/`expires`)
-- `delete` (`name`)
-- `clear` (all)
-
-`chromium-storage` reads/writes Web Storage for the active page: `store=local|session`, `action=get` (one `key` or all entries), `set`, `remove`, `clear`.
-
-Both are per-origin / active-tab. Handy for seeding auth before an e2e test or asserting app state after one.
+- Scroll with `gesture-scroll` and drag with `gesture-drag`; `gesture-swipe` is touch-only.
+- Tabs and windows: `chromium-tabs`. Every other tool acts on the **active** tab, so `select` before driving a different one. A cross-process navigation (some redirects) can swap a tab's CDP target; `list` again to pick it up under a fresh id.
+- Cookies, HttpOnly included: `chromium-cookies`. Web Storage: `chromium-storage`. Use them to seed auth before an e2e test or assert app state after one.

package/skills/argent-device-interact/references/foldables.md

@@ -0,0 +1,8 @@
+# Foldable Devices
+
+Read this for a device `list-devices` marks `foldable: true`: a foldable iOS simulator (the iPhone Duo) or Android emulator. Everything in the related SKILL.md still applies, and the `fold` description covers postures, panels and the result fields. What it does not say:
+
+- Keep one screenshot-diff baseline per posture: the screenshot size changes with the panel.
+- Fold between gestures, not during one. A gesture stays on the panel where it started.
+- iOS: if argent cannot find the active panel, it uses the cover panel and sets `warning` (a screenshot-diff summary gets a `panel:` line). Take a screenshot to see what the device shows, then do the check the warning names.
+- Android: on a closed generic foldable AVD, tap only full-width rows; test the cover panel on a Pixel Fold AVD.