@loadmill/droid-cua 2.43.0 → 2.45.0

package/skill/droid-cua/SKILL.md

@@ -2,7 +2,7 @@
 name: droid-cua
 description: Operate and improve Droid CUA `.dcua` test projects. Use when the user explicitly names Droid CUA or @loadmill/droid-cua, or asks to author, configure, run, diagnose, heal, stabilize, or speed up `.dcua` tests with the Droid CUA CLI or desktop app. Do not use for general web browsing, website development, browser automation, generic mobile work, or non-Droid test tasks.
 metadata:
-  version: "1.0.17"
+  version: "1.1.17"
 ---
 
 # Droid CUA
@@ -13,7 +13,7 @@ Use the existing `droid-cua` CLI and repository files. Do not invent commands, f
 
 1. Inspect the active test project before creating files. Find existing `.dcua` tests, `context.md`, `test-data.md`, `.secrets`, `.env`, and `ci/droid-cua.json` without reading secret values. Do not search other projects or the user's home directory for credentials.
 2. Determine the requested platform and target from the task or project. Ask the user only when the choice is unclear.
-3. Check authentication and environment readiness before the first run. Read [project-and-auth.md](references/project-and-auth.md) for project boundaries and token handling. Read [onboarding-and-troubleshooting.md](references/onboarding-and-troubleshooting.md) when setup, dependency, device, simulator, browser, or connection work is needed.
+3. Check authentication and environment readiness before the first run. Read [project-and-auth.md](references/project-and-auth.md) for project boundaries and token handling. Read [onboarding-and-troubleshooting.md](references/onboarding-and-troubleshooting.md) when setup, dependency, device, simulator, USB iPhone, browser, or connection work is needed.
 4. Identify the intended coverage, starting state, scenario inputs, and cleanup; author or update the smallest useful test that preserves them. Read [authoring-tests.md](references/authoring-tests.md) before changing `.dcua`, context, test data, secrets, or config.
 5. Before execution, present the proposed tests and target to the user, resolve any questions, and get their confirmation. Read [running-and-healing.md](references/running-and-healing.md) for the required pre-run check.
 6. Run the confirmed, narrowest relevant test and monitor it to completion. Keep the user oriented during the run by surfacing meaningful live screenshots, not just terminal output.

package/skill/droid-cua/references/authoring-tests.md

@@ -108,7 +108,7 @@ Droid searches for the best matching Loadmill flow, starts it, and waits for it
 
 ## Validate captured network traffic
 
-For local Chrome, Edge, iOS Simulator or Android Emulator runs with Network Capture enabled, describe HTTP expectations in natural language. Keep the action and its network verification distinct, and name enough of the endpoint and method to identify the intended traffic:
+For local Chrome, Edge, iPhone, iOS Simulator, Android phone or Android Emulator runs with Network Capture enabled, describe HTTP expectations in natural language. Keep the action and its network verification distinct, and name enough of the endpoint and method to identify the intended traffic:
 
 ```text
 Open the Home page.

package/skill/droid-cua/references/onboarding-and-troubleshooting.md

@@ -50,7 +50,7 @@ adb devices
 
 Use `--avd adb:<serial>` for an attached target and `--avd avd:<name>` to launch an AVD.
 
-For network capture, use **Tools → Manage Certificates** in the desktop app to inspect the active CA, upload one `.pem` with a CA certificate and matching unencrypted private key, generate a new CA, or remove it. Connect a local Android Emulator and use **Tools → Install Network Certificate**; it installs the active CA or generates one if none exists. Installation requires a rootable image with working certificate-store mount capabilities and supports Android through version 16. Google Play images without root access and Android 17 are outside this setup path. Configure the emulator's system proxy manually, then enable desktop **Network Capture** or run a local mobile CLI test with `--capture-network`. `--network-proxy` enables forwarding without capture. Follow the approval rule above before changing device trust on the user's behalf.
+For network capture, use **Settings → Network capture → Manage certificates** in the desktop app to inspect the active CA, upload one `.pem` with a CA certificate and matching unencrypted private key, generate a new CA, or remove it. Connect a local Android Emulator and use **Settings → Network capture → Install on connected device**; it installs the active CA or generates one if none exists. Installation requires a rootable image with working certificate-store mount capabilities and supports Android through version 16. Google Play images without root access and Android 17 are outside this setup path. Configure the emulator's system proxy manually, then enable desktop **Network Capture** or run a local mobile CLI test with `--capture-network`. `--network-proxy` enables forwarding without capture. Follow the approval rule above before changing device trust on the user's behalf.
 
 ## iOS
 
@@ -82,7 +82,38 @@ appium driver install xcuitest
 
 Use `--platform ios --avd "<simulator name>"` for the run.
 
-For network capture, use **Tools → Manage Certificates** to select the active CA, connect the simulator, and use **Tools → Install Network Certificate**. If no CA exists, installation generates a Droid CA. The CLI assumes the simulator already trusts the active CA. Configure the Mac's proxy manually, then enable desktop **Network Capture** or run a local mobile CLI test with `--capture-network`. Follow the approval rule above before changing simulator trust on the user's behalf.
+For network capture, use **Settings → Network capture → Manage certificates** to select the active CA, connect the simulator, and use **Settings → Network capture → Install on connected device**. If no CA exists, installation generates a Droid CA. The CLI assumes the simulator already trusts the active CA. Configure the Mac's proxy manually, then enable desktop **Network Capture** or run a local mobile CLI test with `--capture-network`. Follow the approval rule above before changing simulator trust on the user's behalf.
+
+## iPhone over USB
+
+A real iPhone connected to the Mac by USB is a local, attended target: the user must keep it nearby and unlocked. It needs macOS, Xcode that supports the iPhone's iOS, the XCUITest driver, and the user's Apple ID in Xcode on their company's paid Apple Developer team (Developer role with Access to Certificates, Identifiers & Profiles). Free Apple IDs are not supported. Simulators are not needed.
+
+Check readiness without starting a test:
+
+```sh
+droid-cua ios-device check
+```
+
+Each line is one check: `✔` passed, `✖` failed, `·` not checked yet because an earlier check failed, and `!` a non-blocking reminder (Auto-Lock, test-only iPhone). The `Fix:` line under a `✖` is the exact remediation. Work on the first `✖` only, then rerun the check. Exit `0` means ready, and the last line gives the `--avd ios-device:<udid>` token to run with. Add `--udid <udid>` when several iPhones are connected.
+
+Interpret failures:
+
+- **Xcode installed** or **Device components**: the user must install or update the Xcode named in the message and open it once (it asks for a Mac admin password). When the message says Xcode isn't in the Applications folder, the user must install it or move it to `/Applications`. When it says the Command Line Tools are selected or the license isn't accepted, the `Fix:` line gives the exact `sudo xcode-select -s` or `sudo xcodebuild -license accept` command; get approval before running either.
+- **iOS platform installed**: Xcode lacks the iOS platform for its SDK (Xcode 26 ships without it), so it can't build for an iPhone or fetch the user's Apple teams. The user installs it from Xcode > Settings > Components, or runs `xcodebuild -downloadPlatform iOS`. It's about 10 GB, so get approval before running the command and expect it to take a while.
+- **Apple team access**: on a fresh Mac the check fetches the user's teams from Apple with a short Xcode build within the same run. "No Apple account is signed into Xcode" means the user must sign in (Xcode > Settings > Apple Accounts). "Signed into Xcode, but none of your accounts is in a paid Apple Developer team" means the user must ask their Apple team admin for the Developer role with Access to Certificates, Identifiers & Profiles. With several teams, ask the user which team to use and pass `--ios-team <TEAMID>`; never choose a team for them.
+- **iPhone connected**, **iPhone trusts this Mac**, **Developer Mode**, **iPhone unlocked**: steps only the user can do on the phone. Relay the `Fix:` text, wait for them to confirm, then rerun the check. Turning on Developer Mode restarts the iPhone.
+- **Signing ready** and **Helper app (WebDriverAgent)**: droid-cua fixes these itself. Get approval before the first `droid-cua ios-device check --fix`: it runs a one-time Xcode signing build (about a minute) that can create the user's Apple Development certificate and register the iPhone on their company team. A test run also applies these fixes automatically.
+- **Signing ready** failing with "There's no usable Apple Development certificate for your team on this Mac": a rerun won't help. Relay the `Fix:` line: the user creates an Apple Development certificate for the team in Xcode (Xcode > Settings > Apple Accounts > select the account > select the team > Manage Certificates > + > Apple Development) or on the Apple Developer website (developer.apple.com > Certificates, Identifiers & Profiles > Certificates), or asks their Apple Developer team admin for one if they don't have permission, then reruns the fix. Never revoke a certificate yourself or suggest revoking one.
+
+Never restart, reset, or change settings on the user's iPhone yourself. Do not run `xcodebuild`, `codesign`, or `devicectl` to work around a failing check; use `ios-device check`.
+
+### iPhone network capture
+
+Network capture on an iPhone needs manual setup on the phone, done by the user. The Mac and iPhone must share a Wi-Fi network that allows device-to-device traffic (guest and many corporate networks block it). With an iPhone connected, desktop **Settings → Network capture → Install on connected device** shows the steps instead of installing anything: its **Send to iPhone** button opens the macOS share menu with a `.cer` copy of the CA (named after the certificate's common name, such as `Droid CUA Root fbd877ab.cer`, in `~/.droid-cua/network/iphone/`) to AirDrop to the iPhone; the user installs it from Settings → Profile Downloaded, then turns on full trust for that common name in Settings → General → About → Certificate Trust Settings. The user then sets Wi-Fi → Configure Proxy → Manual to the Mac address and port `8765` that Droid prints, and turns the proxy off after testing, since the iPhone loses internet access through it while Droid is not running. Apps that pin certificates still work through the proxy, but their HTTPS is not decrypted. Relay these steps; never change the iPhone's certificate, trust, or proxy settings yourself.
+
+### Android phone network capture
+
+A physical Android phone connected over ADB also needs manual setup by the user, on a Wi-Fi network shared with the Mac that allows device-to-device traffic. With the phone connected, desktop **Settings → Network capture → Install on connected device** writes a `.crt` copy of the CA named after its common name (such as `Droid CUA Root fbd877ab.crt`, in `~/.droid-cua/network/android/`), copies it to the phone's Downloads folder, and shows the steps: Settings → Security → Encryption & credentials → Install a certificate → CA certificate (names vary by vendor), set Wi-Fi → Edit network → Proxy → Manual to the Mac address and port `8765` that Droid prints, then restart the tested app. If the copy fails, the dialog names the local file for the user to copy. Since Android 7, apps trust user CAs only when their network security config allows it (`<certificates src="user"/>` or debug overrides), so debug and QA builds usually work and Play Store builds usually do not; Chrome trusts user CAs. Rooted-phone system trust is not supported. Relay these steps; never change the phone's certificates or settings yourself, and remind the user to turn the proxy off after testing.
 
 ## Web
 

package/skill/droid-cua/references/running-and-healing.md

@@ -2,6 +2,22 @@
 
 ## Construct the run
 
+### Step Watchdog
+
+Step Watchdog is experimental and defaults to `off`. Add `--step-watchdog on` to check for failures or blockers after each completed instruction and stop on a verdict, or `--step-watchdog off` to disable checks. The headless config field is `stepWatchdogMode`; only `on` and `off` are accepted, and the CLI flag overrides it. Desktop users toggle **Settings → Agent → Verify Step Results**: **On** enables checks; **Off** disables checks. Unrecognized saved desktop values reset to `off`; enable the toggle again to use the watchdog. Preserve the project's selected mode and provider when diagnosing a run.
+
+Checks run silently when they pass. Keep validation details out of user-facing progress updates and result summaries unless the user explicitly asks to inspect them. A failure appears in desktop timelines and HTML reports as a **Step Failed** card with the expected instruction, the actor's latest message from that instruction (or a generic failure message), and a screenshot. This is the actor's explanation, not a written Decisions explanation. Interactive runs offer **Skip** and **Stop**, like assertions: Skip records the choice and advances without replaying actions, while Stop fails the test. A skipped step keeps its failure evidence even if the remaining run passes; do not report it as successful or choose Skip merely to get a pass. Headless runs fail automatically without waiting for input (`failureKind: step_watchdog`). A timeout, refusal, invalid response, or unavailable endpoint is an evaluator error (`failureKind: step_watchdog_error`) and cannot be skipped. These do not increment assertion counts. A low stop probability is not proof of success; keep explicit assertions.
+
+The evaluator judges the outcome after the actor has finished the current instruction, before the next one starts. An unfinished required action or destination should fail at that boundary even if another action could recover it later; attempt limits do not make the outcome optional. When diagnosing a missing-page failure, check the attempt limit and destination required by that step. Treat an actor's admission that it could not finish as evidence unless other evidence shows success, and compare tool success messages with the screenshot when they conflict. Expected negative-test outcomes and work explicitly assigned to later instructions remain valid exceptions.
+
+For persisted Decisions diagnostics, enable CLI `--debug` or desktop **Settings → Debug Mode** and inspect `step_watchdog.result` in the run's `execution-<runId>-<timestamp>.jsonl`. CLI logs are in `./logs/`; desktop logs are in the app's user-data `logs/` directory. Match `ids.runId`, `ids.stepId`, and `ids.instructionIndex`. The entry includes the API answers and usage, probability, threshold, mode, resulting action, duration, available response/request IDs, and screenshot path. It also records evaluator errors and cancellations. Input prompts, credentials, test text, and inline images are omitted from this entry. Only failed-check screenshots appear in normal timelines and reports. For probabilities, passing-check screenshots, and evaluator diagnostics, use the debug log and its screenshot artifacts for diagnosis.
+
+The watchdog may also receive the latest non-empty reasoning summary from the current instruction, capped at 2,000 characters and redacted like other text evidence. It never reuses summaries from earlier instructions or makes an extra model call to generate one. Treat this summary as the actor's interpretation, not proof of success or failure; prioritize the original requirements, screenshot, and recorded actions. A missing summary is normal when the actor returns none.
+
+The check uses the selected provider's Decisions endpoint and `gpt-6-luna`. Loadmill Smart and Pulse use `/llm/v1/decisions` with the existing Loadmill sign-in; no client OpenAI key is needed. The gateway must have Decisions support deployed and its OpenAI account must have Decisions access. Team overrides of the acting model or reasoning effort do not change the evaluator. Do not silently switch providers or turn the check off to get a pass. Distinguish service/setup problems from application failures. For a likely false stop, compare the original scenario, current step, expected negative outcomes, later form/submission steps, and screenshot. Preserve the original inputs and requirements; do not change them merely to bypass a blocker. Review labeled runs with `on` before enabling widely; the `0.65` threshold is provisional.
+
+### Target and configuration
+
 Prefer a single test before a directory, multi-path selection, or scenario run. Use `droid-cua run <paths...>` for saved tests. It accepts one or more `.dcua` files or directories, including files from different folders. `run` must be the first argument. Keep all selected paths contiguous and place options after the complete path list. Inputs retain command-line order, directories use natural test-name ordering, and overlapping files run only once. The legacy `--instructions <file-or-directory>` form remains supported, but prefer `run` for new commands.
 
 A single target, config, context, test-data file, and secrets source applies to the entire command. Combine tests only when they share those run-level settings; otherwise use separate commands. Multi-path runs execute sequentially and produce an aggregate **Selected tests** report in addition to individual test reports.
@@ -22,7 +38,11 @@ Include `--report reports/<test-name>.html` for normal test runs unless the user
 
 For macOS and Windows desktop tests in application mode, the desktop app shows a system notice for three seconds before Droid takes control. The three-second delay is fixed; once the test starts, use Stop in Droid to end it. Avoid using the mouse or keyboard during the run. Droid brings the selected target window to the foreground before the first capture. When the run ends, the target stays in front and a second system notice says Droid has finished controlling the desktop. Open Droid to review the result. Some Windows apps return an all-black window capture; Droid brings the target back to the foreground before cropping the visible desktop to its visible frame, excluding the invisible Windows resize border. The fallback logs an all-black capture warning and crops before resizing to preserve detail. A visible-frame lookup warning means Droid is using the outer window bounds, which may include invisible resize borders. Keep the target visible and unobstructed on the primary display. In accessibility input mode, bring the target to the foreground yourself if the fallback requests it; Droid does not focus it for screenshots. If a screenshot shows another app or black off-screen edges, check whether the target could be focused and fully displayed before changing the `.dcua` instructions.
 
-The local network proxy is off by default. Use `--network-proxy` only when the user asks for local mobile proxy forwarding and has prepared device routing. This flag starts an idle forwarding listener; it does not capture traffic or enable network assertions. Use `--capture-network` only when the user asks for captured traffic or network assertions for local Chrome, Edge, iOS Simulator or Android Emulator tests. It writes one JSONL log per test, beside `--report` output when enabled or under `logs/` otherwise. For mobile capture it starts the proxy; the active CA must exist and the device must trust it. Configure mobile proxy routing manually before the run. Manual setup and cleanup are the accepted V1 workflow. Clear that routing after use; if Droid exits or crashes, restart it or clear the proxy to restore connectivity. Captures stop recording at 100 MiB with a CAPTURE_LIMIT marker while traffic continues forwarding. Treat that partial artifact as incomplete evidence; live queries cannot establish absence after the limit. Capture is best effort and WebSocket messages are unsupported. Captures may contain credentials or sensitive data despite partial filtering, so do not print their contents or share the artifact without the user's direction. Report the printed log path and capture health. Network-enabled runs expose `query_network` for focused questions and cross-request assertions; its helper answers include flow references. Distinguish an application mismatch from an assertion that could not be established because capture evidence was insufficient. Absence checks need a completed observation window and working capture evidence; never claim that no request occurred solely because none was captured.
+The local network proxy is off by default. Use `--network-proxy` only when the user asks for local mobile proxy forwarding and has prepared device routing. This flag starts an idle forwarding listener; it does not capture traffic or enable network assertions. Use `--capture-network` only when the user asks for captured traffic or network assertions for local Chrome, Edge, iPhone, iOS Simulator, Android phone or Android Emulator tests. It writes one JSONL log per test, beside `--report` output when enabled or under `logs/` otherwise. For mobile capture it starts the proxy; the active CA must exist and the device must trust it. Configure mobile proxy routing manually before the run. Manual setup and cleanup are the accepted V1 workflow. Clear that routing after use; if Droid exits or crashes, restart it or clear the proxy to restore connectivity. Captures stop recording at 100 MiB with a CAPTURE_LIMIT marker while traffic continues forwarding. Treat that partial artifact as incomplete evidence; live queries cannot establish absence after the limit. Capture is best effort and WebSocket messages are unsupported. Captures may contain credentials or sensitive data despite partial filtering, so do not print their contents or share the artifact without the user's direction. Report the printed log path and capture health. Network-enabled runs expose `query_network` for focused questions and cross-request assertions; its helper answers include flow references. Distinguish an application mismatch from an assertion that could not be established because capture evidence was insufficient. Absence checks need a completed observation window and working capture evidence; never claim that no request occurred solely because none was captured.
+
+For a local iPhone (`--avd ios-device:<udid|auto>`), the proxy listens on all Mac addresses and the CLI prints the line to relay, such as `On the iPhone, set Wi-Fi → Configure Proxy → Manual: server 192.168.1.23, port 8765.` If it fails because the Mac has no network address an iPhone can reach, ask the user to join the Mac and iPhone to the same Wi-Fi network. "No traffic observed" on an iPhone usually means its proxy is not set, or the Wi-Fi blocks device-to-device traffic; "no requests were decrypted" means the Droid CA is not installed with full trust on the iPhone or the app pins its certificates. Pinned apps cannot be decrypted; report that rather than retrying.
+
+A physical Android phone connected over ADB (any serial other than `emulator-<port>`) works the same way: the CLI prints a line such as `On the Android phone, set Wi-Fi → Edit network → Proxy → Manual: hostname 192.168.1.23, port 8765.` to relay. "No requests were decrypted" on an Android phone usually means the Droid CA is not installed as a user CA certificate, or the app's network security config does not trust user CAs (common for Play Store builds; debug and QA builds usually opt in). Report that rather than retrying.
 
 Local Chrome and Edge capture through Playwright; no certificate or proxy setup is needed. Keep the same assertions, query tool and report workflow. Capture begins at run start and includes frames and popups in that browser context. CLI `--url` navigation is captured. Earlier traffic from an already-open page is not recovered: include navigation in the test when its initial load matters, rather than silently reloading or changing the browser profile. Browser capture observes normal cache and service-worker behavior. Worker-originated requests and responses served by a worker are identified in query metadata; an observed exchange is not proof of a new server request. Pending or unavailable bodies remain incomplete evidence. Cloud browsers, WebSockets, streaming-message inspection and HAR export are outside this capture support. A browser run does not depend on the mobile proxy being available.
 
@@ -44,6 +64,9 @@ droid-cua run tests/example.dcua --avd avd:<avd-name> --config ci/droid-cua.json
 # iOS Simulator
 droid-cua run tests/example.dcua --llm-provider loadmill --cua-model loadmill-pulse --platform ios --avd "<simulator-name>" --debug --report reports/example.html
 
+# iPhone connected by USB (run `droid-cua ios-device check` first)
+droid-cua run tests/example.dcua --llm-provider loadmill --cua-model loadmill-pulse --avd ios-device:<udid> --debug --report reports/example.html
+
 # Local web
 droid-cua run tests/example.dcua --llm-provider loadmill --cua-model loadmill-pulse --target web --browser chrome --session-mode fresh --debug --report reports/example.html
 
@@ -62,6 +85,8 @@ droid-cua run tests/auth/login.dcua tests/checkout/purchase.dcua admin/refund.dc
 
 Use `.ipa` and `--platform ios` for supported cloud iOS runs. Preserve existing config flags rather than replacing them with defaults.
 
+For an iPhone connected by USB, confirm `droid-cua ios-device check` passes before the first run (see [onboarding-and-troubleshooting.md](onboarding-and-troubleshooting.md)). Use `--avd ios-device:<udid>` from the check's last line, or `ios-device:auto` when only one iPhone is connected; `--platform` is not needed. Add `--ios-team <TEAMID>` only when the user belongs to several paid Apple teams. There is no `--app` for a local iPhone: the test opens an app already installed on it. In the run plan, tell the user the test runs on their own iPhone with the apps and data already on it, and that they need to keep it unlocked and nearby. During the run, `Enter your passcode on the iPhone to allow automation` and `iPhone locked — unlock to continue` are waits for the user, not failures: tell the user right away and keep monitoring.
+
 ## Confirm the run with the user
 
 Before starting any test command, send the user a short, plain-language run plan and wait for explicit confirmation. Treat this as a product conversation, not a command preview. Do not infer that a request to run a test also confirms an automatically selected device, browser, or batch of tests. For an explicitly requested author-run-improve workflow, the plan may ask approval for the baseline and the evidence-driven edit/rerun and confidence runs needed to reach a readiness decision; follow [iterative-improvement.md](iterative-improvement.md).
@@ -138,7 +163,7 @@ Keep this retrospective concise and lead with the result and improvement. Do not
 ## Classify the first divergence
 
 1. **Setup or authentication**: missing/invalid provider credential, dependency, license, browser, or executable. Return to onboarding and rerun only the failed check.
-2. **Target connectivity**: no device, unauthorized/offline ADB state, unavailable simulator, cloud credential/device/app problem, or lost session. Restore the same target, then rerun.
+2. **Target connectivity**: no device, unauthorized/offline ADB state, unavailable simulator, iPhone unplugged/untrusted/locked or its helper app not starting, cloud credential/device/app problem, or lost session. Restore the same target (for an iPhone, rerun `droid-cua ios-device check`), then rerun.
 3. **Test ambiguity or stale test knowledge**: instruction has multiple interpretations, relies on an unstable label, lacks a starting condition, or context contradicts the visible product. A minimal test/context correction is allowed.
 4. **Application regression**: the app crashes, displays an error, violates an explicit requirement, or consistently fails a stable assertion after correct setup. Stop and report a bug with evidence.
 5. **Transient execution**: temporary network, service, model, or timing failure without evidence of a deterministic product/test issue. Retry the unchanged targeted test once.