@swmansion/argent 0.8.1 → 0.9.0

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@swmansion/argent",
-  "version": "0.8.1",
+  "version": "0.9.0",
   "description": "MCP server for iOS Simulator and Android Emulator control",
   "license": "Apache-2.0",
   "repository": {

package/rules/argent.md

@@ -13,6 +13,8 @@ Use cases:
 - User mentions iOS simulator, Android emulator, device, or app interaction
 - The app user is working with is a mobile application which can be run in a simulator/emulator
 - Any tapping, swiping, typing, screenshotting, or inspecting a running app
+- Any code change that affects visible mobile UI, layout, styling, copy, navigation, or screen composition
+- Any request to execute manual QA, UI QA, or visual behavior validation for a mobile app
 - Running, debugging, or testing a React Native app (iOS or Android)
 - Profiling performance or diagnosing re-renders in a React Native app (iOS or Android)
   </description>
@@ -92,7 +94,11 @@ When: Beginning a task that involves the Android emulator, no emulator running y
 
 TAPPING, SWIPING, TYPING, GESTURES, SCREENSHOTS, SCROLLING
 Skill: `argent-device-interact`
-When: Performing touch interactions, typing, pressing hardware buttons, launching/restarting apps, opening URLs, rotating device, or taking standalone screenshots.
+When: Performing touch interactions, typing, pressing hardware buttons, launching/restarting apps, opening URLs, rotating device, taking standalone screenshots, or verifying a visible UI code change.
+
+SCREENSHOT DIFF & VISUAL REGRESSION
+Skill: `argent-screenshot-diff`
+When: Explicit visual regression, screenshot diff, compare screenshots, before/after visual comparison requests, or visible UI changes where stable pixel comparison would add useful evidence.
 
 RUNNING / BUILDING / DEBUGGING REACT NATIVE APP
 Skill: `argent-react-native-app-workflow`
@@ -116,7 +122,7 @@ When: App feels slow, user asks to optimize, reducing bundle size, improving sta
 
 END-TO-END UI TESTING
 Skill: `argent-test-ui-flow`
-When: Verifying complete user flows, running interact → screenshot → verify loops, testing features by using the app.
+When: Verifying complete user flows, running interact → screenshot → verify loops, testing features by using the app, executing manual QA steps, or validating visible UI changes or visual behavior after implementation.
 
 RECORDING & REPLAYING FLOWS
 Use skill: `argent-create-flow`

package/skills/argent-device-interact/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: argent-device-interact
-description: Interact with an iOS simulator or Android emulator using argent MCP tools. Use when tapping UI elements, performing gestures, scrolling, typing text, pressing hardware buttons, launching apps, opening URLs, taking screenshots.
+description: Interact with an iOS simulator or Android emulator using argent MCP tools. Use when tapping UI elements, performing gestures, scrolling/swiping, typing text, pressing hardware buttons, launching apps, opening URLs, taking screenshots, or checking visible app state after interactions.
 ---
 
 ## Unified tool surface
@@ -184,6 +184,7 @@ Values: `Portrait`, `LandscapeLeft`, `LandscapeRight`, `PortraitUpsideDown`
 Use the explicit `screenshot` tool only when:
 
 - You need the initial screen state before any action.
+- You are about to edit visible UI and need a baseline capture before making changes.
 - The auto-attached screenshot shows a transitional or loading frame.
 - You require extra context.
 - You want to check state after a delay (e.g. waiting for a network response).
@@ -198,7 +199,15 @@ When using `screenshot` for permission or native modal navigation:
 
 Optional rotation parameter: `{ "udid": "<UDID>", "rotation": "LandscapeLeft" }` — rotates the capture without changing simulator orientation.
 
-Screenshots are downscaled by default (30% of original resolution) to reduce context size. `scale` accepts values from 0.01 to 1.0. If UI elements are hard to read or you need to inspect fine detail, pass `scale: 1.0` to get full resolution: `{ "udid": "<UDID>", "scale": 1.0 }`.
+Screenshots are downscaled by default (30% of original resolution) to reduce context size. Use the normal downscaled screenshot for UI context and state checks. `scale` accepts values from 0.01 to 1.0, but do not use `scale: 1.0` as a general readability or tapping aid.
+
+Use full-resolution screenshots only when saving baseline/current PNG files for comparison. In that case, suppress the image block so the full-size PNG is not loaded into agent context:
+
+```json
+{ "udid": "<UDID>", "scale": 1.0, "includeImageInContext": false }
+```
+
+For visual regression checks, before/after screenshot comparisons, and detailed `screenshot-diff` parameter guidance, use the `argent-screenshot-diff` skill. Keep this skill focused on device interaction mechanics and screenshot capture.
 
 ### Troubleshooting
 

package/skills/argent-react-native-app-workflow/SKILL.md

@@ -186,6 +186,7 @@ Check the `argent-environment-inspector` result for test commands. For interacti
 
 - **Unit tests**: Look for Jest in `package.json` (`"test": "jest"`, `jest` config). Run: `npm test` or `yarn test`.
 - **E2E**: Look for Detox (`.detoxrc.js` or similar), or other E2E config. Dependencies: `detox`, `detox-cli`, and for iOS often `applesimutils`.
+- **Visible UI changes**: Use `argent-test-ui-flow` for manual QA. For `screenshot-diff` rules and parameters, follow the `argent-screenshot-diff` skill. Use it when stable before/after screenshots add meaningful pixel-visible evidence.
 - **UI flow testing**: For interactive UI testing with automatic screenshot verification, refer to the `argent-test-ui-flow` skill.
 
 ### 5.2 Running Tests (Typical)
@@ -219,6 +220,7 @@ If the user's intent is ambiguous (run existing tests, write new tests, or find
 | List devices                 | `list-devices` tool (iOS + Android)                                                                                                      |
 | Boot a device                | `boot-device` tool (pass `udid` for iOS or `avdName` for Android)                                                                        |
 | Take screenshot              | `screenshot` tool                                                                                                                        |
+| Compare visible UI changes   | `screenshot-diff` tool; follow the `argent-screenshot-diff` skill for baseline/current capture choices                                   |
 | Describe screen (a11y tree)  | `describe` tool for normal app screens and in-app modals; use `screenshot` only when permission/system overlays are not exposed reliably |
 | Read JS console logs         | `debugger-log-registry` tool                                                                                                             |
 | Reload JS bundle             | `debugger-reload-metro` tool                                                                                                             |

package/skills/argent-screenshot-diff/SKILL.md

@@ -0,0 +1,67 @@
+---
+name: argent-screenshot-diff
+description: Compare saved or live app screenshots with the argent screenshot-diff tool. Use when testing visual regressions, before/after UI comparisons, screenshot diff checks, visible layout, spacing, color, typography, clipping, overflow, text rendering, or image/icon rendering changes.
+---
+
+## 1. Role
+
+Use `screenshot-diff` as supporting visual evidence for UI QA and visual regression checks. It highlights pixel-visible change or stability; it does not replace visual inspection, accessibility/component-tree state, frame/attribute checks, logs, network evidence, or app behavior.
+
+Do not use screenshot diffing for tap-coordinate discovery. Use `describe`, `debugger-component-tree`, or `native-describe-screen` to find targets first.
+
+## 2. When To Use
+
+Use `screenshot-diff` when pixel comparison can answer the verification question:
+
+- Required for explicit "UI regression test", "visual regression test", "screenshot diff", "compare screenshots", or "before/after visual comparison" requests, unless stable comparable screenshots cannot be produced.
+- Good fit when the affected screen has stable before/after states and the expected result is pixel-visible: layout, position, size, spacing, color, typography, image/icon rendering, clipping, overflow, or text rendering.
+- Good fit when the risk is unintended visual regression outside the exact element changed.
+- Poor fit when the result is better verified structurally: state changes, navigation existence, accessibility tree contents, console/network behavior, or unit tests.
+- Poor fit when dynamic content, unpausable animation, timestamps, ads, random data, or missing baseline/current screenshots would make the comparison noisy or meaningless.
+
+## 3. Capture Rules
+
+Use normal downscaled `screenshot` calls for UI context and state checks. Use full-resolution screenshots only when saving baseline/current PNG files for visual regression comparison. Suppress the image block so the full-size PNG is not loaded into context:
+
+```json
+{ "udid": "<UDID>", "scale": 1.0, "includeImageInContext": false }
+```
+
+Capture the stable baseline before the relevant interaction or before editing whenever feasible. Compare it to the post-change or post-interaction screen after the app reloads, rebuilds, or reaches the state under test.
+
+## 4. Parameters
+
+Provide `udid` and exactly one input for the baseline side and exactly one input for the current side:
+
+- Common UI regression flow: saved baseline plus live current -> `baselinePath`, `captureCurrent: true`, `udid`, `outputDir`.
+- Both screenshots already saved -> `baselinePath`, `currentPath`, `udid`, `outputDir`.
+- Rare fixture flow: live baseline plus saved current -> `captureBaseline: true`, `currentPath`, `udid`, `outputDir`.
+- Do not combine `captureBaseline: true` with `captureCurrent: true`, or provide both a path and live capture flag for the same side.
+
+## 5. Deterministic Flow
+
+1. Navigate to the known-good state.
+2. Capture a baseline PNG with `screenshot` using `scale: 1.0` and `includeImageInContext: false`; keep the returned `path`.
+3. Perform the interaction, apply the code change and navigate to the state under test.
+4. Call `screenshot-diff` with the saved `baselinePath`, `captureCurrent: true`, `udid`, and `outputDir`.
+5. Inspect the summary and artifact paths, then combine the diff with normal visual inspection and any structural/runtime evidence needed for the assertion.
+
+```json
+{
+  "baselinePath": "/tmp/baseline.png",
+  "captureCurrent": true,
+  "udid": "<UDID>",
+  "outputDir": "/tmp/argent-diff"
+}
+```
+
+If both images are already saved, use file paths for both sides:
+
+```json
+{
+  "baselinePath": "/tmp/baseline.png",
+  "currentPath": "/tmp/current.png",
+  "udid": "<UDID>",
+  "outputDir": "/tmp/argent-diff"
+}
+```

package/skills/argent-test-ui-flow/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: argent-test-ui-flow
-description: Autonomously test an app UI (iOS or Android) by running interact-screenshot-verify loops using argent MCP tools. Use when testing a UI flow, verifying login works, testing navigation, or running an end-to-end UI test scenario.
+description: Autonomously test an app UI (iOS or Android) by running interact-screenshot-verify loops using argent MCP tools. Use when testing UI flows, verifying login works, testing navigation, running end-to-end UI test scenarios, manual QA steps, visible UI changes, or visual behavior.
 ---
 
 ## Platform-agnostic
@@ -20,14 +20,21 @@ Once a platform is chosen, the per-platform setup skill takes over:
 
 All interactions go through argent MCP tools. Ensure the simulator/emulator is ready before starting.
 
-1. **Baseline screenshot**: Call `screenshot` to see the current UI state.
+For implementation tasks that modify visible UI, this workflow can also serve as a visual acceptance path.
+
+1. **Baseline screenshot**: Call `screenshot` to see the current UI state. For visual regression comparison or UI change verification, capture the baseline at `scale: 1.0` with `includeImageInContext: false` and keep the returned `path` before editing whenever feasible.
 2. **Find target**: Before tapping, use a discovery tool to get element coordinates:
    - **React Native apps**: use `debugger-component-tree` — it returns component names with (tap: x,y) coordinates. This is the preferred tool for RN apps on either platform. To use it, resolve the `argent-react-native-app-workflow` skill for setup; on Android you must also run `adb -s <serial> reverse tcp:8081 tcp:8081` so Metro is reachable from the device.
    - **Standard app screens and in-app modals**: use `describe`. On iOS this returns the AX tree (falls back to native-devtools when AX is empty); on Android it returns the uiautomator tree in the same DescribeNode shape.
    - **Permission prompts / system modal overlays**: try `describe` first. Fall back to `screenshot` only if the overlay is not exposed reliably.
    - **Fallback**: use `screenshot` to estimate where the desired component is, then verify immediately after the action.
 3. **Interact**: Perform the action (`gesture-tap`, `gesture-swipe`, `keyboard`, `button`, ...) — you receive a screenshot automatically.
-4. **Verify**: Check the returned screenshot for expected results. If it shows a loading/transitional state, retake with `screenshot`.
+4. **Verify**: Check the returned screenshot for expected results. If it shows a loading/transitional state, retake with normal downscaled `screenshot`. Pick evidence by what's being asserted:
+   - **Visual** (layout, spacing, color, typography, image/icon rendering, clipping, overflow, text rendering): prefer `screenshot-diff` against the baseline captured in step 1 — it surfaces pixel-visible changes the auto-screenshot might miss. Fall back to visual inspection of the auto-screenshot only when a stable baseline isn't available.
+   - **Structural** (navigation state, element existence, accessibility labels/values, selection, hierarchy, route): verify with `describe`, `debugger-component-tree`, or `native-describe-screen`.
+   - **Runtime / log / network** (console errors, API calls, persistence, timing): verify with `view-network-logs`, `debugger-log-registry`, `debugger-evaluate`, or targeted tests.
+   - **Mixed**: collect evidence for each relevant class.
+   - Report the combined verdict: expected behavior, observed behavior, evidence used, and any blocker for requested visual diffing.
 5. **Repeat** for each step in the flow.
 
 ## 2. Template
@@ -36,10 +43,12 @@ All interactions go through argent MCP tools. Ensure the simulator/emulator is r
 Goal: Test [feature name]
 
 Steps:
-1. screenshot → see current state (baseline)
-2. [Navigate / tap / type to reach starting point] → verify auto-screenshot
-3. [Perform the action to test] → verify auto-screenshot
-4. Report: pass / fail with details
+1. Classify expected result: visual / structural / runtime-log-network / mixed → choose evidence
+2. [Navigate / tap / type to reach stable comparable starting point] → verify auto-screenshot
+3. screenshot { scale: 1.0, includeImageInContext: false } → save baseline path when visual or mixed evidence needs diffing
+4. [Perform the action to test] → verify auto-screenshot
+5. Use screenshot-diff when requested or when comparable images add useful visual evidence
+6. Report: pass / fail with combined visual, structural, runtime/log/network evidence as applicable
 ```
 
 ## 3. Examples
@@ -67,6 +76,19 @@ Steps:
 6. screenshot → verify returned to list
 ```
 
+### Visual behavior check
+
+```
+1. Classify expected result as visual or mixed.
+2. Navigate to the stable starting state.
+3. screenshot { scale: 1.0, includeImageInContext: false } → save baseline path.
+4. describe / debugger-component-tree → find the control and use its returned tap coordinates.
+5. gesture-tap → perform the visual behavior under test.
+6. screenshot-diff { baselinePath, captureCurrent: true, udid, outputDir } → inspect visible change or stability.
+7. describe / debugger-component-tree → verify selected state, label, route, or attributes if relevant.
+8. Report combined verdict from expected behavior, visual inspection, diff summary, and structural evidence.
+```
+
 ---
 
 ## 4. Recovery Pattern
@@ -82,17 +104,17 @@ Steps:
 - **Use `paste` for text entry on iOS** — faster and more reliable than key-by-key `keyboard`. `paste` is iOS-only; on Android use `keyboard` instead.
 - **Use `gesture-custom` for long-press** context menus (800ms hold).
 - **Report clearly**: state what you expected, what you saw, and the verdict.
-- **Coordinate estimation**: center = 0.5, 0.5; top-third ~ 0.2; bottom-third ~ 0.8.
 - **Permission modals**: try `describe` first. Use `screenshot` only as fallback, tap one visible button at a time, and verify with the returned screenshot before continuing.
 - **Record for replay**: If a tested flow is likely to be repeated, use the `argent-create-flow` skill to record it as a `.yaml` script. This lets you replay the entire sequence later with a single `flow-execute` call instead of re-running each step manually.
 
 ## Related Skills
 
-| Skill                              | When to use                                             |
-| ---------------------------------- | ------------------------------------------------------- |
-| `argent-device-interact`           | Tool usage for tapping, swiping, typing (iOS + Android) |
-| `argent-ios-simulator-setup`       | Booting and connecting an iOS simulator                 |
-| `argent-android-emulator-setup`    | Booting and connecting an Android emulator              |
-| `argent-react-native-app-workflow` | Starting the app, Metro, build issues                   |
-| `argent-metro-debugger`            | Breakpoints, console logs, JS evaluation                |
-| `argent-create-flow`               | Record a test sequence as a replayable flow             |
+| Skill                              | When to use                                              |
+| ---------------------------------- | -------------------------------------------------------- |
+| `argent-device-interact`           | Tool usage for tapping, swiping, typing (iOS + Android)  |
+| `argent-screenshot-diff`           | Visual regression and before/after screenshot comparison |
+| `argent-ios-simulator-setup`       | Booting and connecting an iOS simulator                  |
+| `argent-android-emulator-setup`    | Booting and connecting an Android emulator               |
+| `argent-react-native-app-workflow` | Starting the app, Metro, build issues                    |
+| `argent-metro-debugger`            | Breakpoints, console logs, JS evaluation                 |
+| `argent-create-flow`               | Record a test sequence as a replayable flow              |