@swmansion/argent 0.12.1 → 0.13.1-next.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Binary file
Binary file
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@swmansion/argent",
3
- "version": "0.12.1",
3
+ "version": "0.13.1-next.0",
4
4
  "description": "MCP server for iOS Simulator and Android Emulator control",
5
5
  "license": "Apache-2.0",
6
6
  "repository": {
@@ -41,12 +41,14 @@
41
41
  "scripts/postinstall.cjs"
42
42
  ],
43
43
  "dependencies": {
44
+ "@fails-components/webtransport": "^1.6.3",
45
+ "@fails-components/webtransport-transport-http3-quiche": "^1.6.3",
44
46
  "@modelcontextprotocol/sdk": "^1.20.0",
45
47
  "tree-sitter": "^0.21.1",
46
48
  "tree-sitter-typescript": "^0.23.2"
47
49
  },
48
50
  "optionalDependencies": {
49
- "electron": "^42.4.1"
51
+ "electron": "^42.5.0"
50
52
  },
51
53
  "devDependencies": {
52
54
  "@argent/cli": "file:../argent-cli",
@@ -54,14 +56,14 @@
54
56
  "@argent/mcp": "file:../argent-mcp",
55
57
  "@argent/telemetry": "file:../telemetry",
56
58
  "@argent/tools-client": "file:../argent-tools-client",
57
- "@clack/prompts": "^1.5.1",
58
- "@types/node": "^25.9.3",
59
+ "@clack/prompts": "^1.6.0",
60
+ "@types/node": "^26.0.1",
59
61
  "@types/semver": "^7.7.1",
60
62
  "esbuild": "^0.28.1",
61
63
  "picocolors": "^1.1.1",
62
- "posthog-node": "5.35.0",
63
- "semver": "^7.8.4",
64
- "smol-toml": "^1.6.1",
64
+ "posthog-node": "5.38.6",
65
+ "semver": "^7.8.5",
66
+ "smol-toml": "^1.7.0",
65
67
  "typescript": "^6.0.3",
66
68
  "vitest": "^4.1.9",
67
69
  "yaml": "^2.8.3"
package/rules/argent.md CHANGED
@@ -4,7 +4,7 @@ alwaysApply: true
4
4
  ---
5
5
 
6
6
  <description>
7
- If argent is installed and configured in this environment, its MCP tools are the preferred form of interaction with the application for iOS simulator, Android emulator, and Chromium (CDP) app control; otherwise see `<availability_check>` below before attempting any argent workflow. A "Chromium (CDP) app" is any Chromium runtime exposing a Chrome DevTools Protocol endpoint — an Electron app, or any Chromium-family browser (Chrome/Brave/Edge) launched with `--remote-debugging-port`; all are driven through the same tool surface and tagged `platform: "chromium"`.
7
+ If argent is installed and configured in this environment, its MCP tools are the preferred form of interaction with the application for iOS simulator, Android emulator, Chromium (CDP) app, and Vega (Amazon Fire TV) device control; otherwise see `<availability_check>` below before attempting any argent workflow. A "Chromium (CDP) app" is any Chromium runtime exposing a Chrome DevTools Protocol endpoint — an Electron app, or any Chromium-family browser (Chrome/Brave/Edge) launched with `--remote-debugging-port`; all are driven through the same tool surface and tagged `platform: "chromium"`. A "Vega device" is a virtual device (VVD) or physical unit — driven by tv-remote (D-pad) and tagged `platform: "vega"`.
8
8
  Running MCP server and managing the Argent toolkit utilises `argent` command - if asked use `argent --help` for reference.
9
9
  To check current version of MCP server run `argent --version` command.
10
10
 
@@ -15,7 +15,7 @@ Use cases:
15
15
  - Any tapping, swiping, typing, screenshotting, or inspecting a running app
16
16
  - Any code change that affects visible mobile UI, layout, styling, copy, navigation, or screen composition
17
17
  - Any request to execute manual QA, UI QA, or visual behavior validation for a mobile app
18
- - Running, debugging, or testing a React Native app (iOS or Android)
18
+ - Running, debugging, or testing a React Native app (iOS, Android or Vega)
19
19
  - Profiling performance or diagnosing re-renders in a React Native app (iOS or Android)
20
20
  - Running, debugging, or testing a Chromium (CDP) app — an Electron app (boot with `boot-device` + `electronAppPath`) or a Chromium browser exposing CDP (auto-discovered on port `9222` / `ARGENT_CHROMIUM_PORTS`); on Chromium scroll with `gesture-scroll` and drag with `gesture-drag` — `gesture-swipe` is touch-only
21
21
  </description>
@@ -80,7 +80,7 @@ Decision order:
80
80
  - When the session ends or the user says they are done: call `stop-all-simulator-servers`.
81
81
  If the user started Metro separately, ask whether to call `stop-metro` (specify the port if not 8081).
82
82
  - If tools provided by mcp-server are not sufficient and action can be done using `xcrun`, `adb`, or other commands, use the command. Examples: changing device options, performing a device action such as lock, shake, etc.
83
- - When waiting for an action, do not call `screenshot` repeatedly without a proper wait mechanism. For example, six consecutive `screenshot` calls with no adequate delay between them will cause context bloat.
83
+ - When waiting for an action, do not call `screenshot` repeatedly without a proper wait mechanism. Use the `await-ui-element` tool to block until the UI settles (e.g. wait for an element to become `visible`/`hidden`, or to contain expected `text`) instead of polling.
84
84
  </general_rules>
85
85
 
86
86
  <react_native_detection>
@@ -98,7 +98,7 @@ Load the matching skill before starting work and executing tools from argent-mcp
98
98
  procedure and edge-case handling for each workflow.
99
99
 
100
100
  PLATFORM DETECTION
101
- If the user did not specify a platform, call `list-devices` first and pick the booted target — do not default to iOS.
101
+ If the user did not specify a platform, call `list-devices` first and pick the booted target — do not default to iOS. Vega (Amazon Fire TV) devices appear as `platform:"vega"`, when present load `argent-tv-interact`
102
102
 
103
103
  iOS SIMULATOR SETUP
104
104
  Skill: `argent-ios-simulator-setup`
@@ -110,7 +110,12 @@ When: Beginning a task that involves the Android emulator, no emulator running y
110
110
 
111
111
  TAPPING, SWIPING, TYPING, GESTURES, SCREENSHOTS, SCROLLING
112
112
  Skill: `argent-device-interact`
113
- When: Performing touch interactions, typing, pressing hardware buttons, launching/restarting apps, opening URLs, rotating device, taking standalone screenshots, or verifying a visible UI code change.
113
+ When: Performing touch interactions, typing, pressing hardware buttons, launching/restarting apps, opening URLs, rotating device, taking standalone screenshots, or verifying a visible UI code change. Phone/tablet iOS and Android only — for any TV target use the TV skill below.
114
+
115
+ TV INTERACTION (APPLE TV / ANDROID TV / FIRE TV)
116
+ Skill: `argent-tv-interact`
117
+ When: Any TV target — a `list-devices` entry with `runtimeKind: "tv"` (Apple TV simulator or Android TV emulator) or `platform:"vega"` / `kind:"vvd"` (Amazon Fire TV / VVD), or the user mentions Apple TV / tvOS / Android TV / leanback / Vega / Fire TV. A TV UI is focus-driven, not touch-driven: drive it with `describe` (read focus) + `tv-remote` (D-pad presses) + `keyboard` (type); `gesture-*` tools do NOT apply. Covers booting the target, app lifecycle, focus navigation, typing, screenshots, and (Vega) VVD lifecycle + Fast Refresh.
118
+ Prompt keywords: apple tv, tvos, android tv, leanback, vega, fire tv, vvd, d-pad
114
119
 
115
120
  SCREENSHOT DIFF & VISUAL REGRESSION
116
121
  Skill: `argent-screenshot-diff`
@@ -23,6 +23,7 @@ Pass the Android serial as `udid` to the unified interaction tools — `gesture-
23
23
 
24
24
  ## 4. Notes
25
25
 
26
+ - **Android TV / leanback AVDs** boot through this exact flow (same `boot-device` + `avdName`), but they are **focus-driven, not touch-driven** — do not use `gesture-tap`/`gesture-swipe` on them. `list-devices` tags a leanback device with `runtimeKind: "tv"` (detected via the system feature list, not the serial — a TV AVD's serial looks just like a phone's). When you see `runtimeKind: "tv"`, drive it with the focus-driven tools (`describe` / `tv-remote` / `keyboard`) and the `argent-tv-interact` skill (it covers Android TV as well as Apple TV, including the full TV setup flow).
26
27
  - Serials are the adb device id. iOS UDIDs and Android serials are not interchangeable, but you do NOT need to tell the tools which platform — dispatch is automatic.
27
28
  - `describe` on Android returns a shallower tree than iOS (no accessibility-service equivalent), but covers most tap-target discovery.
28
29
  - `reinstall-app` on Android always installs with `-g` so first-launch runtime permissions are pre-granted.
@@ -58,6 +58,13 @@ command: "screenshot"
58
58
  args: "{\"udid\": \"<UDID>\"}"
59
59
  ```
60
60
 
61
+ ```
62
+ command: "await-ui-element"
63
+ args: "{\"udid\": \"<UDID>\", \"condition\": \"visible\", \"selector\": {\"text\": \"Continue\"}}"
64
+ ```
65
+
66
+ Record an `await-ui-element` step to **gate** the next step on a screen transition — it blocks until the element is `visible`/`hidden` (or contains `text`), so the following step runs only once the screen has actually settled. If its condition is not met before the timeout, replay **stops at that step** (the steps after it assume the transition happened). Prefer this over a fixed `delayMs`. See the `await-ui-element` section of `argent-device-interact` for the full condition/selector reference.
67
+
61
68
  For tools with no arguments, omit `args` entirely.
62
69
 
63
70
  ## 5. Important Rules
@@ -99,7 +106,7 @@ flow-execute { name: "open-settings", project_root: "/Users/dev/MyApp", prereq
99
106
  Flow files use YAML. The top-level is an object with `executionPrerequisite` (describes required state) and `steps` (array of actions):
100
107
 
101
108
  - `- echo: <message>` — a label
102
- - `- tool: <name>` with optional `args:` — a tool call. Add `delayMs: <ms>` to sleep that long before the step runs (use sparingly — only when the app needs a fixed wait between actions).
109
+ - `- tool: <name>` with optional `args:` — a tool call. A tool step may also carry `delayMs: <ms>` to sleep that long before it runs. (`await-ui-element` is an ordinary tool step; see §4 and §10.5 for when to gate a transition with one.)
103
110
 
104
111
  Example `.yaml` file:
105
112
 
@@ -111,6 +118,13 @@ steps:
111
118
  args:
112
119
  udid: ABC
113
120
  bundleId: com.apple.Preferences
121
+ - echo: Wait for the Settings list to render
122
+ - tool: await-ui-element
123
+ args:
124
+ udid: ABC
125
+ condition: visible
126
+ selector:
127
+ text: General
114
128
  - echo: Tap General
115
129
  - tool: gesture-tap
116
130
  args:
@@ -207,6 +221,7 @@ After applying a correction, re-run `flow-execute` to verify.
207
221
  Apply these when recording new flows to reduce future breakage:
208
222
 
209
223
  - **Echo expected state, not just actions.** Write `"On Settings > General screen, about to tap About"` not `"Tap About"`. During diagnosis these tell you what the screen _should_ look like.
224
+ - **Gate transitions with `await-ui-element`, not fixed delays.** After a tap that triggers a navigation, record an `await-ui-element` step that waits for the next screen's element to be `visible` (or a spinner to be `hidden`) before the following step. This removes the **Timing** failure mode in §10.2 (the element is in the tree but the tap fired before the screen settled) and is more reliable than `delayMs` or an extra `screenshot`. An unmet wait stops replay at that step, so a mistimed step can never run blind.
210
225
  - **Add screenshot steps after critical navigation.** Insert `screenshot` steps after screen transitions. These produce images in the flow result you can inspect during diagnosis.
211
226
  - **Write specific executionPrerequisites.** `"App on home tab, user logged in, simulator UDID is <X>"` — not `"App running"`. Verify with `screenshot` + `describe` before acknowledging.
212
227
  - **Prefer launch-app / open-url over navigation chains.** Deep links are more resilient to layout changes than tap sequences.
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: argent-device-interact
3
- description: Interact with an iOS simulator, Android emulator, or Chromium (CDP) app using argent MCP tools. Use when tapping UI elements, performing gestures, scrolling/swiping, typing text, pressing hardware buttons, launching apps, opening URLs, taking screenshots, or checking visible app state after interactions.
3
+ description: Interact with an iOS simulator, Android emulator, or Chromium (CDP) app using argent MCP tools. Use when tapping UI elements, performing gestures, scrolling/swiping, typing text, pressing hardware buttons, launching apps, opening URLs, taking screenshots, waiting for an element to appear or disappear, or checking visible app state after interactions.
4
4
  ---
5
5
 
6
6
  ## Unified tool surface
@@ -13,6 +13,8 @@ All interaction tools below accept a `udid` parameter and auto-dispatch iOS vs A
13
13
 
14
14
  **Cookies & storage (Chromium only):** `chromium-cookies` reads/writes cookies via the Network domain (so HttpOnly cookies are visible): `action=get` (optionally scoped by `url`), `set` (`name`, `value`, + `url`/`domain`, optional `secure`/`httpOnly`/`sameSite`/`expires`), `delete` (`name`), `clear` (all). `chromium-storage` reads/writes Web Storage for the active page: `store=local|session`, `action=get` (one `key` or all entries), `set`, `remove`, `clear`. Both are per-origin / active-tab. Handy for seeding auth before a flow or asserting app state after one.
15
15
 
16
+ > **TV targets (Apple TV / Android TV) are not covered by this skill.** A TV target is **focus-driven, not touch-driven** — the `gesture-*` tools are the wrong tools for it. This applies to both Apple TV simulators (UUID-shaped, identical to iOS) and Android TV / leanback devices (serial-shaped, identical to a phone emulator). If `list-devices` tags your target `runtimeKind: "tv"`, stop and use the `argent-tv-interact` skill: `describe` to read focus, `tv-remote` for remote / D-pad presses, and `keyboard` to type.
17
+
16
18
  For platform-specific caveats (Metro `adb reverse`, locked-screen describe errors, etc.), see § 9 Platform-specific notes at the bottom.
17
19
 
18
20
  ## 1. Before You Start
@@ -28,7 +30,7 @@ Use `list-devices` to get a target id. Results are tagged with `platform` (`ios`
28
30
  1. **Always refer to tapping_rule** from your argent.md rule before tapping.
29
31
  2. Before performing interactions, consider whether they can be **dispatched sequentially** - more on that in `run-sequence`.
30
32
  3. **Use `gesture-swipe` for lists/scrolling**, not `gesture-custom`, unless you need non-linear movement. On Chromium use `gesture-scroll` instead — `gesture-swipe` is touch-only. Consider whether you need multiple swipes, if yes - use `run-sequence`.
31
- 4. **Tap a text field before typing** — on iOS try `paste` first then fall back to `keyboard`; on Android use `keyboard` directly (`paste` is iOS-only).
33
+ 4. **Tap a text field before typing**, then use `keyboard` to enter text.
32
34
  5. **Coordinates are normalized** — always 0.0–1.0, not pixels.
33
35
  6. **For app navigation, prefer `describe` first.** It works on any screen without app restart. Do not navigate from screenshots on regular in-app screens unless `describe` failed to expose a reliable target. Use `native-describe-screen` only when you need app-scoped UIKit properties.
34
36
 
@@ -54,25 +56,25 @@ Common schemes: `messages://`, `settings://`, `maps://?q=<query>`, `tel://<numbe
54
56
 
55
57
  ## 4. Choosing the Right Tool
56
58
 
57
- | Action | Tool | Notes |
58
- | ----------------- | ---------------- | ---------------------------------------------------------------------- |
59
- | Multiple actions | `run-sequence` | Batch steps in one call (no intermediate screenshots) |
60
- | Open an app | `launch-app` | **Always — never tap home-screen icons** |
61
- | Restart an app | `restart-app` | Terminate and relaunch by bundle ID |
62
- | Open URL/scheme | `open-url` | Web pages, deep links, URL schemes |
63
- | Single tap | `gesture-tap` | Buttons, links, checkboxes |
64
- | Scroll/swipe | `gesture-swipe` | Straight-line scroll or swipe |
65
- | Scroll (Chromium) | `gesture-scroll` | Wheel-based; deltas are window fractions, positive deltaY = down |
66
- | Drag (Chromium) | `gesture-drag` | Sliders, drag-and-drop, text selection |
67
- | Long press | `gesture-custom` | Context menus, drag start |
68
- | Drag & drop | `gesture-custom` | Complex drag interactions |
69
- | Pinch/zoom | `gesture-pinch` | Two-finger pinch with auto-interpolation |
70
- | Rotation | `gesture-rotate` | Two-finger rotation with auto-interpolation |
71
- | Custom gesture | `gesture-custom` | Arbitrary touch sequences, optional interpolation |
72
- | Hardware key | `button` | Home, back, power, volume, appSwitch, actionButton |
73
- | Type text (fast) | `paste` | iOS only. Form fields — uses clipboard |
74
- | Type text | `keyboard` | iOS+Android. Fallback when paste fails; supports Enter, Escape, arrows |
75
- | Rotate device | `rotate` | Orientation changes |
59
+ | Action | Tool | Notes |
60
+ | ----------------- | ------------------ | ---------------------------------------------------------------- |
61
+ | Multiple actions | `run-sequence` | Batch steps in one call (no intermediate screenshots) |
62
+ | Open an app | `launch-app` | **Always — never tap home-screen icons** |
63
+ | Restart an app | `restart-app` | Terminate and relaunch by bundle ID |
64
+ | Open URL/scheme | `open-url` | Web pages, deep links, URL schemes |
65
+ | Single tap | `gesture-tap` | Buttons, links, checkboxes |
66
+ | Scroll/swipe | `gesture-swipe` | Straight-line scroll or swipe |
67
+ | Scroll (Chromium) | `gesture-scroll` | Wheel-based; deltas are window fractions, positive deltaY = down |
68
+ | Drag (Chromium) | `gesture-drag` | Sliders, drag-and-drop, text selection |
69
+ | Long press | `gesture-custom` | Context menus, drag start |
70
+ | Drag & drop | `gesture-custom` | Complex drag interactions |
71
+ | Pinch/zoom | `gesture-pinch` | Two-finger pinch with auto-interpolation |
72
+ | Rotation | `gesture-rotate` | Two-finger rotation with auto-interpolation |
73
+ | Custom gesture | `gesture-custom` | Arbitrary touch sequences, optional interpolation |
74
+ | Hardware key | `button` | Home, back, power, volume, appSwitch, actionButton |
75
+ | Type text | `keyboard` | iOS+Android. Supports Enter, Escape, arrows |
76
+ | Rotate device | `rotate` | Orientation changes |
77
+ | Wait for UI | `await-ui-element` | Block until an element is visible/hidden/exists/contains text |
76
78
 
77
79
  ## 5. Finding Tap Targets
78
80
 
@@ -161,14 +163,6 @@ For long-press, drag-and-drop, and other complex sequences, see `references/gest
161
163
 
162
164
  Values: `home`, `back`, `power`, `volumeUp`, `volumeDown`, `appSwitch`, `actionButton`
163
165
 
164
- ### paste — Type text into focused field (iOS only)
165
-
166
- ```json
167
- { "udid": "<UDID>", "text": "Hello, world!" }
168
- ```
169
-
170
- Tap the field first, then paste. Fall back to `keyboard` if it doesn't work. On Android the call is rejected by the capability gate ("Tool 'paste' is not supported on android") — use `keyboard` directly.
171
-
172
166
  ### keyboard — Type text or press special keys
173
167
 
174
168
  ```json
@@ -185,6 +179,23 @@ Special keys: `enter`, `escape`, `backspace`, `tab`, `space`, `arrow-up`, `arrow
185
179
 
186
180
  Values: `Portrait`, `LandscapeLeft`, `LandscapeRight`, `PortraitUpsideDown`
187
181
 
182
+ ### await-ui-element — Block until a UI element reaches a state
183
+
184
+ Instead of polling `screenshot`/`describe` in a loop, use `await-ui-element` to block server-side until an element reaches an expected state (or `timeoutMs`, default 5000ms, elapses). It polls the same accessibility/DOM tree as `describe`. (For a plain pause, use your own harness sleep — this tool deliberately has no bare-timer mode.)
185
+
186
+ ```json
187
+ { "udid": "<UDID>", "condition": "visible", "selector": { "text": "Continue" } }
188
+ ```
189
+
190
+ - `condition`: `exists`, `visible`, `hidden`, or `text`.
191
+ - `selector`: `{ text?, identifier?, role? }` — every provided field must match (case-insensitive substring). `text` matches the element's label or value; `identifier` matches its accessibility id / resource-id / testID; `role` matches its element role (e.g. `AXButton`, `button`, `TextView`, `StaticText`). The synthetic `ROOT` container `describe` prints is never matched, so a `role` like `AXGroup`/`html` won't trivially "match the screen".
192
+ - Prefer a **specific** selector. A loose substring can match several elements, and the tool may then key off one you didn't mean: `text` reads the **first** match in **reading order** (top-to-bottom, left-to-right — the same order `describe` lists them, so it's the one you saw first), while `visible`/`exists` are satisfied by **any** match. Disambiguate with a longer or more exact string, an `identifier`, or a `role` (e.g. pin to a text role like `StaticText` to skip a same-named button). On a `text` timeout the `note` quotes the matched element's text, so you can see which one it landed on.
193
+ - `text` condition also needs `expectedText` (substring the matched element must contain).
194
+ - `hidden` treats a selector that matches **nothing** as already-hidden, so a typo'd selector returns an instant (false) success. Double-check the selector for `hidden` waits — the result `note` flags when the selector never matched any element. (On iOS, if the accessibility backend is down the tree comes back empty; the tool will **not** report `hidden` success off such a degraded read and the `note` surfaces the boot hint instead.)
195
+ - Optional `timeoutMs` (default 5000) and `pollIntervalMs` (default 400).
196
+
197
+ Returns `{ success, elapsed }`; on a timeout `success` is `false` and a `note` explains what was seen.
198
+
188
199
  ---
189
200
 
190
201
  ## 7. Screenshots
@@ -246,10 +257,12 @@ Use the sequencing when:
246
257
 
247
258
  ### Allowed tools inside `run-sequence`
248
259
 
249
- `gesture-tap`, `gesture-swipe`, `gesture-custom`, `gesture-pinch`, `gesture-rotate`, `button`, `keyboard`, `rotate`
260
+ `gesture-tap`, `gesture-swipe`, `gesture-scroll`, `gesture-drag`, `gesture-custom`, `gesture-pinch`, `gesture-rotate`, `button`, `keyboard`, `rotate`, `await-ui-element`
250
261
 
251
262
  The `udid` is shared — do **not** include it in each step's `args`. Optional `delayMs` per step (default 100ms).
252
263
 
264
+ Add an `await-ui-element` step to gate a later tap on a screen transition (e.g. tap → wait for the next screen's button → tap it). If its condition is **not** met before the timeout, the sequence stops at that step and the following steps do **not** run — so a mistimed tap can't fire against a screen that never settled.
265
+
253
266
  ### Examples
254
267
 
255
268
  Scroll down three times:
@@ -293,7 +306,25 @@ Tap a known button, then scroll down:
293
306
  }
294
307
  ```
295
308
 
296
- Stops on the first error and returns partial results.
309
+ Tap, wait for the next screen, then act on it — the `await-ui-element` step **gates** the tap after it:
310
+
311
+ ```json
312
+ {
313
+ "udid": "<UDID>",
314
+ "steps": [
315
+ { "tool": "gesture-tap", "args": { "x": 0.5, "y": 0.9 } },
316
+ {
317
+ "tool": "await-ui-element",
318
+ "args": { "condition": "visible", "selector": { "text": "Continue" } }
319
+ },
320
+ { "tool": "gesture-tap", "args": { "x": 0.5, "y": 0.5 } }
321
+ ]
322
+ }
323
+ ```
324
+
325
+ Prefer this over a fixed `delayMs` when a step depends on a screen transition: it adapts to real load time, and if the condition is not met before the timeout the sequence **stops there** so the next tap can't fire against a screen that never settled.
326
+
327
+ Stops on the first error (or unmet `await-ui-element` condition) and returns partial results.
297
328
 
298
329
  ---
299
330
 
@@ -87,7 +87,7 @@ To revisit a previous trace:
87
87
  Bottlenecks are categorized by severity:
88
88
 
89
89
  - **RED**: CPU functions taking >15% of total time, all UI hangs, and **attributed** memory leaks (those with a resolved responsible frame). These require immediate attention.
90
- - **YELLOW**: CPU functions taking 5-15% of total time, and **unattributed** memory leaks (`<Call stack limit reached>`, no library — see the memory-leaks caveat below). Worth investigating but may be acceptable.
90
+ - **YELLOW**: CPU functions taking 3-15% of total time, and **unattributed** memory leaks (`<Call stack limit reached>`, no library — see the memory-leaks caveat below). Worth investigating but may be acceptable.
91
91
 
92
92
  Each bottleneck type indicates a different class of problem:
93
93
 
@@ -29,7 +29,7 @@ For implementation tasks that modify visible UI, this workflow can also serve as
29
29
  - **Permission prompts / system modal overlays**: try `describe` first. Fall back to `screenshot` only if the overlay is not exposed reliably.
30
30
  - **Fallback**: use `screenshot` to estimate where the desired component is, then verify immediately after the action.
31
31
  3. **Interact**: Perform the action (`gesture-tap`, `gesture-swipe`, `keyboard`, `button`, ...) — you receive a screenshot automatically.
32
- 4. **Verify**: Check the returned screenshot for expected results. If it shows a loading/transitional state, retake with normal downscaled `screenshot`. Pick evidence by what's being asserted:
32
+ 4. **Verify**: Check the returned screenshot for expected results. If it shows a loading/transitional state, prefer blocking until it settles with `await-ui-element` (expected element `visible`, or a spinner `hidden`) over a guessed delay — but only with a selector you can trust (`text`/`identifier`/`role`) that the screen is known to have or that you saw in a prior `describe`; a guessed one just times out. Otherwise use a short fixed wait. Pick evidence by what's being asserted:
33
33
  - **Visual** (layout, spacing, color, typography, image/icon rendering, clipping, overflow, text rendering): prefer `screenshot-diff` against the baseline captured in step 1 — it surfaces pixel-visible changes the auto-screenshot might miss. Fall back to visual inspection of the auto-screenshot only when a stable baseline isn't available.
34
34
  - **Structural** (navigation state, element existence, accessibility labels/values, selection, hierarchy, route): verify with `describe`, `debugger-component-tree`, or `native-describe-screen`.
35
35
  - **Runtime / log / network** (console errors, API calls, persistence, timing): verify with `view-network-logs`, `debugger-log-registry`, `debugger-evaluate`, or targeted tests.
@@ -58,9 +58,9 @@ Steps:
58
58
  ```
59
59
  1. screenshot → see login screen
60
60
  2. gesture-tap { x: 0.5, y: 0.4 } → tap email field
61
- 3. paste { text: "user@example.com" }
61
+ 3. keyboard { text: "user@example.com" }
62
62
  4. gesture-tap { x: 0.5, y: 0.55 } → tap password field
63
- 5. paste { text: "password123" }
63
+ 5. keyboard { text: "password123" }
64
64
  6. gesture-tap { x: 0.5, y: 0.7 } → tap Login button
65
65
  7. screenshot → verify home screen appeared
66
66
  ```
@@ -89,11 +89,20 @@ Steps:
89
89
  8. Report combined verdict from expected behavior, visual inspection, diff summary, and structural evidence.
90
90
  ```
91
91
 
92
+ ### Wait for a loading spinner
93
+
94
+ ```
95
+ 1. gesture-tap { x: 0.5, y: 0.7 } → trigger an action that fetches data
96
+ 2. screenshot → loading spinner is showing
97
+ 3. await-ui-element { condition: hidden, selector: { text: "Loading" } } → block until the fetch finishes and the spinner disappears
98
+ 4. describe / screenshot → verify the fetched content rendered
99
+ ```
100
+
92
101
  ---
93
102
 
94
103
  ## 4. Recovery Pattern
95
104
 
96
- - If screenshot shows loading/transition: wait 500ms, retake with `screenshot`.
105
+ - If a screen is mid-transition or loading: block until it settles with `await-ui-element` (wait for the target element to be `visible`, or the spinner/placeholder to be `hidden`) instead of a blind fixed delay, then re-check. Fall back to a fixed wait + `screenshot` only when no element reliably marks the transition.
97
106
  - If tap misses target: re-run discovery tool (`describe` / `debugger-component-tree`), retry once with new coordinates.
98
107
  - If a permission dialog or modal is visible: re-run `describe` first. Stay in screenshot-driven navigation only when the overlay is not exposed reliably, then switch back to `describe` / `debugger-component-tree` as soon as it is dismissed.
99
108
  - If tap fails twice at same coordinates: stop, re-discover, report if element not found.
@@ -101,7 +110,7 @@ Steps:
101
110
 
102
111
  ## Tips
103
112
 
104
- - **Use `paste` for text entry on iOS** — faster and more reliable than key-by-key `keyboard`. `paste` is iOS-only; on Android use `keyboard` instead.
113
+ - **Wait on the UI, don't poll.** When a step needs the screen to change first, gate it with `await-ui-element` (block until an element is `visible`/`hidden` or contains `text`) rather than repeated `screenshot` calls with fixed sleeps. See the `await-ui-element` section of `argent-device-interact`.
105
114
  - **Use `gesture-custom` for long-press** context menus (800ms hold).
106
115
  - **Report clearly**: state what you expected, what you saw, and the verdict.
107
116
  - **Permission modals**: try `describe` first. Use `screenshot` only as fallback, tap one visible button at a time, and verify with the returned screenshot before continuing.
@@ -0,0 +1,60 @@
1
+ ---
2
+ name: argent-tv-interact
3
+ description: Control and inspect TV apps via argent — Apple TV (tvOS), Android TV (leanback), and Amazon Fire TV (Vega). Boot the target, read focus, navigate with the D-pad remote, type, and screenshot. Use when a task targets a TV (runtimeKind "tv", or platform "vega"), or mentions Apple TV / tvOS / Android TV / leanback / Vega / Fire TV / VVD.
4
+ ---
5
+
6
+ # Argent TV (Apple TV + Android TV + Fire TV)
7
+
8
+ ## Critical
9
+
10
+ - A TV is **focus-driven, not touch-driven.** Drive every interaction with `describe` + `tv-remote` + `keyboard`; never use `gesture-*` / coordinate taps — they don't apply on any TV platform.
11
+ - **Always `describe` before navigating** to find the live cursor and your target — never guess focus from a screenshot. The cursor is the focused element; on **Vega** the toolkit often leaves `focused` false and marks the highlighted item `[selected]`, so treat `[selected]` as the cursor when nothing reports `[focused]`.
12
+ - Pass the `udid` from `list-devices` — an Apple TV simulator UDID or an Android TV / Vega `serial`. Dispatch is automatic from the id; the same tools drive all three.
13
+
14
+ ## The navigation loop
15
+
16
+ 1. `describe` — find the cursor and your target (returns the focused element + all focusable ones, not a tap tree).
17
+ 2. `tv-remote` — move focus toward the target. Prefer **one** call with a path ending in `select`, e.g. `{button:["down","right","select"]}`; count rows/columns from the frames to build the path.
18
+ 3. `describe` again to confirm. On a miss, repeat.
19
+
20
+ ## Tools
21
+
22
+ - `describe {udid}` — focus view: the focused / `[selected]` element + focusable elements with labels and normalized frames. The discovery tool — call before and after navigating. Empty tree → see the per-platform notes.
23
+ - `tv-remote {udid, button}` — D-pad / remote. `button` is one key **or a whole path** (run in one call). Keys: `up`/`down`/`left`/`right`, `select`, `back`, `menu`, `home`, `playPause`, plus media keys `rewind`/`fastForward`/`next`/`previous`/`volumeUp`/`volumeDown`/`mute`. Single: `{button:"down"}`; repeat: `{button:"down", repeat:3}`; path: `{button:["up","right","select"]}`.
24
+ - `keyboard {udid, text}` — type into the focused field (focus it with `tv-remote` first). Named `key` presses (e.g. `{key:"enter"}`) work on Vega; on Apple TV / Android TV move focus with `tv-remote` instead.
25
+ - `launch-app` / `restart-app` / `reinstall-app {udid, bundleId}` — `bundleId` from the app manifest. Vega `reinstall-app` takes `appPath` = a `.vpkg`.
26
+ - `screenshot {udid, scale?}` — Apple TV via `xcrun simctl io` (downscaled); Android TV / Vega host-side via `adb` / `screencap`.
27
+
28
+ ## Per-platform
29
+
30
+ ### Apple TV (tvOS simulator)
31
+
32
+ - Boot like any iOS sim (`boot-device`); the AX + HID daemons auto-start on the first `describe` / `tv-remote` (first call may take a few seconds). Give the RN bundle a few seconds to render before the first `describe`.
33
+ - Media-transport / volume keys are **rejected** — the sim's HID stack ignores them (they work on Android TV / Vega).
34
+ - Dev build: `open-url {udid, url:"<scheme>://expo-development-client/?url=http%3A%2F%2F<HOST_IP>%3A8081"}` (`<HOST_IP>` = your Mac's LAN IP, shown on the launcher).
35
+
36
+ ### Android TV (leanback emulator)
37
+
38
+ - Boot the leanback AVD like any emulator — see `argent-android-emulator-setup`.
39
+ - **`describe` may report zero focusables on a screen with visible tiles**: many `react-native-tvos` screens use RN's own focus engine, invisible to the OS accessibility tree. `describe` auto-falls-back to the full UI tree (and says so in the hint); `tv-remote` still moves focus, so drive blind + `screenshot` to confirm.
40
+ - Dev build: `adb -s <serial> reverse tcp:8081 tcp:8081`, deep-link `<pkg>://expo-development-client/?url=http%3A%2F%2F10.0.2.2%3A8081`, dismiss the first dev-menu with `adb shell input keyevent KEYCODE_DPAD_CENTER` (not Back — Back exits the app).
41
+
42
+ ### Fire TV (Vega / VVD)
43
+
44
+ - `list-devices` shows a `serial` (use as `udid`) and a `vvdImage`. `boot-device {vvdImage}` (e.g. `"tv"`) starts the single SDK-managed VVD; skip if one already runs.
45
+ - **Stop the VVD** with `vega virtual-device stop` in your shell. The CLI only tracks VVDs it started in the foreground, so it may report "not running" for one started via `boot-device`; to restart that one use `boot-device {vvdImage, force:true}` (stops then re-boots).
46
+ - Empty `describe` tree → `restart-app` (the automation toolkit attaches at launch), then retry. Input ignored → enable developer mode in the VVD: `vsm developer-mode enable`.
47
+ - Editing `node_modules` has no effect on a Release build — only Debug `.vpkg` builds load patchable JS.
48
+ - Profiling / crashes → `amazon-devices-buildertools-mcp` server (`analyze_perfetto_traces`, `get_app_hot_functions`, `symbolicate_acr`); docs via its `search_documentation` tool.
49
+
50
+ ## Common gotchas
51
+
52
+ - **Empty focus right after `launch-app` / `restart-app`** is the splash / loading window — `describe` retries internally; wait ~2-3s and retry on a cold start.
53
+ - Passing a phone/tablet (`runtimeKind: "mobile"`) udid to `tv-remote` fails with a clear "tvOS-only" / "Android-TV-only" error — pick a TV target from `list-devices`.
54
+
55
+ ## Fast Refresh (dev builds)
56
+
57
+ Needs a Debug build + Metro running. argent only _connects_ to Metro — start Metro and port-forward yourself (any platform). Metro is fixed on **:8081**.
58
+
59
+ - **Apple TV / Android TV:** use the dev-build deep-links above; `npm start` for Metro.
60
+ - **Vega:** build/install a Debug `.vpkg` (`vega device install-app -p <path>`), `npm start`, `vega device start-port-forwarding --port 8081 --forward false`, then `vega device launch-app -a <appId>`. Confirm `http://localhost:8081/json/list` shows a `Hermes React Native` target; `.tsx` edits then hot-reload.