qualiow-exploratory-testing 1.0.0 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +85 -0
- package/bin/mcli +16 -0
- package/bin/mobile-cli.mjs +573 -0
- package/bin/wadb +6 -0
- package/bin/wk-ios +58 -0
- package/bin/wkeval.mjs +44 -0
- package/data/knowledge/learned-patterns.md +85 -0
- package/data/knowledge/manifest.yml +2 -1
- package/data/targets/_example-mobile-emulation.yml +52 -0
- package/data/targets/_example-native-mobile.yml +61 -0
- package/data/targets/_example-sim-android-chrome.yml +81 -0
- package/data/targets/_example-sim-ios-safari.yml +80 -0
- package/dist/cli/index.js +156 -33
- package/dist/cli/index.js.map +1 -1
- package/dist/index.d.ts +260 -527
- package/dist/index.js +142 -12
- package/dist/index.js.map +1 -1
- package/docs/MOBILE-SETUP.md +92 -0
- package/package.json +33 -11
- package/scripts/doctor-mobile.sh +143 -0
- package/scripts/setup-mobile.sh +213 -0
- package/skills/qa-explore/SKILL.md +2 -1
- package/skills/qa-explore/phases/00-setup.md +22 -1
- package/skills/qa-explore/phases/01-auth.md +4 -0
- package/skills/qa-explore/phases/03-discovery.md +14 -0
- package/skills/qa-explore/phases/05-features.md +20 -0
- package/skills/qa-explore/phases/06-edge-cases.md +2 -0
- package/skills/qa-explore/phases/07-reporting.md +11 -0
- package/skills/qa-explore/references/playwright-agents-integration.md +126 -0
- package/skills/qa-explore-mobile/SKILL.md +146 -0
- package/skills/qa-explore-mobile/phases/00-setup.md +203 -0
- package/skills/qa-explore-mobile/phases/01-auth.md +127 -0
- package/skills/qa-explore-mobile/phases/02-charter.md +70 -0
- package/skills/qa-explore-mobile/phases/03-discovery.md +79 -0
- package/skills/qa-explore-mobile/phases/04-journeys.md +49 -0
- package/skills/qa-explore-mobile/phases/05-features.md +72 -0
- package/skills/qa-explore-mobile/phases/06-edge-cases.md +157 -0
- package/skills/qa-explore-mobile/phases/07-reporting.md +123 -0
- package/skills/qa-explore-mobile/references/mobile-edges.md +117 -0
- package/skills/qa-explore-quick/SKILL.md +7 -0
- package/skills/qa-target-setup/SKILL.md +12 -0
|
@@ -54,6 +54,8 @@ playwright-cli snapshot
|
|
|
54
54
|
playwright-cli resize 1280 720 # restore
|
|
55
55
|
```
|
|
56
56
|
|
|
57
|
+
> **Note on `page.accessibility` (Playwright 1.57 deprecation):** Upstream Playwright deprecated the `page.accessibility` snapshot API in 1.57. The replacement, available as of Playwright 1.59, is `page.ariaSnapshot()` — an aria-tree-based accessibility snapshot that is both more accurate and directly usable as a locator base. If a session upgrades to deeper a11y checks than the quick-pass above, prefer `page.ariaSnapshot()` over `page.accessibility`. The CLI checks above are unaffected and remain the primary path. See `../references/playwright-agents-integration.md` for a feature cross-reference.
|
|
58
|
+
|
|
57
59
|
## Empty States
|
|
58
60
|
|
|
59
61
|
- What does the cart look like with no items?
|
|
@@ -7,6 +7,17 @@ playwright-cli tracing-stop
|
|
|
7
7
|
playwright-cli close
|
|
8
8
|
```
|
|
9
9
|
|
|
10
|
+
### Trace Analysis (Playwright 1.58–1.59, optional)
|
|
11
|
+
|
|
12
|
+
Once tracing is stopped, the captured trace can be explored from the command line or the HTML reporter:
|
|
13
|
+
|
|
14
|
+
- **`npx playwright trace <trace.zip>`** *(Playwright 1.59)* — opens the trace from the CLI without launching the full UI reporter. Useful for agent-driven debugging of a specific step or failure.
|
|
15
|
+
- **HTML reporter "Speedboard" timeline** *(Playwright 1.58)* — when the session generated a Playwright HTML report, the "Speedboard" tab shows a timeline visualization of steps useful for spotting latency anomalies that correlate with the bugs you filed.
|
|
16
|
+
- **Trace Viewer themes** *(Playwright 1.58)* — the Trace Viewer now honors system theme; no action required, just mentioned here so the reviewer knows the option exists.
|
|
17
|
+
- **`page.screencast()`** *(Playwright 1.59, optional)* — if the session wants to attach an annotated video of the reproduction to the bug report, `page.screencast()` captures it with action annotations and chapter overlays. Treat it as premium evidence for Critical / High bugs; a screenshot remains sufficient for Medium / Low.
|
|
18
|
+
|
|
19
|
+
These helpers are additive — the existing `screenshots/bug-NNN.png` workflow below is unchanged.
|
|
20
|
+
|
|
10
21
|
## Reflect Before Writing
|
|
11
22
|
|
|
12
23
|
Before writing the report, answer these questions:
|
|
@@ -0,0 +1,126 @@
|
|
|
1
|
+
# Playwright Test Agents Integration (opt-in)
|
|
2
|
+
|
|
3
|
+
> **This file is opt-in reference material.** Read it only when the session has access to `@playwright/test` and the consumer project has a `playwright.config.*` file. Otherwise skip this file — `qa-explore` works end-to-end without it.
|
|
4
|
+
|
|
5
|
+
## Purpose
|
|
6
|
+
|
|
7
|
+
`qa-explore` is an **exploratory** skill: its job is blind (or context-enriched) discovery of bugs, risks, and missing functionality in a web application, bounded by a 45-minute session.
|
|
8
|
+
|
|
9
|
+
**Playwright Test Agents** (introduced in Playwright 1.56) is a different, complementary capability: a set of three LLM-driven agents — **planner**, **generator**, **healer** — that author and maintain executable Playwright test specs.
|
|
10
|
+
|
|
11
|
+
These two are complementary, not overlapping:
|
|
12
|
+
|
|
13
|
+
| Concern | `qa-explore` | Playwright Test Agents |
|
|
14
|
+
|---|---|---|
|
|
15
|
+
| Goal | Find unknown bugs and risks | Generate and maintain regression tests |
|
|
16
|
+
| Output | Bug reports, session report, coverage map | `tests/*.spec.ts` files |
|
|
17
|
+
| When to use | New feature, new app, post-incident, pre-release audit | Known behavior you want locked in with a test |
|
|
18
|
+
| Input required | Just a URL (context optional) | A plan / spec / reproducible behavior |
|
|
19
|
+
|
|
20
|
+
Use `qa-explore` to discover; use Test Agents to codify a discovery into a regression test.
|
|
21
|
+
|
|
22
|
+
## When to use this handoff
|
|
23
|
+
|
|
24
|
+
You are in the right file if ANY of these are true:
|
|
25
|
+
|
|
26
|
+
- Phase 5 or 6 of `qa-explore` produced a **reproducible** bug that deserves a regression test.
|
|
27
|
+
- The user completed a `qa-explore` session and explicitly asked "can you write a test for this bug?"
|
|
28
|
+
- The user wants to seed a new test project from a qa-explore charter.
|
|
29
|
+
|
|
30
|
+
You are in the WRONG file if:
|
|
31
|
+
|
|
32
|
+
- You are mid-session. Finish `qa-explore` first; Test Agents run after reporting.
|
|
33
|
+
- `@playwright/test` is not a resolved dependency in the consumer project (the peer dep is optional — consumers who didn't install it cannot use this file).
|
|
34
|
+
- The bug is not reproducible. Test Agents need deterministic input; flaky observations go in the bug report only.
|
|
35
|
+
|
|
36
|
+
## Prerequisites
|
|
37
|
+
|
|
38
|
+
1. `@playwright/test ^1.59.1` installed in the consumer project (declared as optional peer dep in this package).
|
|
39
|
+
2. A `playwright.config.ts` or `playwright.config.js` present in the consumer project. If absent, run `npm init playwright@latest` first.
|
|
40
|
+
3. The `qa-explore` session has completed at least Phase 5 (features) and produced a bug report with exact reproduction steps.
|
|
41
|
+
|
|
42
|
+
## Bootstrap
|
|
43
|
+
|
|
44
|
+
From the consumer project root:
|
|
45
|
+
|
|
46
|
+
```bash
|
|
47
|
+
npx playwright init-agents --loop=claude
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
This generates the following (as of Playwright 1.59):
|
|
51
|
+
|
|
52
|
+
- Agent definition files for the selected loop (`claude`, `vscode`, or `opencode` are supported)
|
|
53
|
+
- A minimal `tests/agents/` scaffolding directory
|
|
54
|
+
- Updates to `playwright.config.ts` to wire agent reporters
|
|
55
|
+
|
|
56
|
+
The `--loop=claude` flag optimizes the agent definitions for Claude Code. `--loop=vscode` optimizes them for VS Code's built-in Copilot integration. Pick the one that matches the consumer's environment.
|
|
57
|
+
|
|
58
|
+
This command is idempotent — re-running it updates definitions without deleting committed tests.
|
|
59
|
+
|
|
60
|
+
## The three agents
|
|
61
|
+
|
|
62
|
+
### Planner
|
|
63
|
+
|
|
64
|
+
**Role:** Given a description of desired behavior (from a charter, a PRD, or a bug repro), produce an ordered plan of tests to write.
|
|
65
|
+
|
|
66
|
+
**Hand-off from qa-explore:** After a session, if the charter identified P0 journeys that have NO automated coverage, feed the charter and coverage map to the planner. The output is a list of test files the team should author.
|
|
67
|
+
|
|
68
|
+
**Output:** Markdown plan with one bullet per proposed test spec.
|
|
69
|
+
|
|
70
|
+
### Generator
|
|
71
|
+
|
|
72
|
+
**Role:** Given a plan item (or a single reproducible bug), emit a working `.spec.ts` file.
|
|
73
|
+
|
|
74
|
+
**Hand-off from qa-explore:** This is the most common handoff. When Phase 5 or 6 produces a reproducible bug, pass the bug's repro steps + expected vs actual to the generator. The generator writes `tests/repro-<bug-id>.spec.ts`, typically a 20-40 line spec.
|
|
75
|
+
|
|
76
|
+
**Output:** A test spec file that fails initially (since the bug is not yet fixed) and will pass once the fix lands.
|
|
77
|
+
|
|
78
|
+
### Healer
|
|
79
|
+
|
|
80
|
+
**Role:** When an existing spec becomes flaky — for example, because a locator shifted — the healer rewrites the failing parts of the spec to match the current DOM, using `locator.normalize()` and `page.pickLocator()` (Playwright 1.59) under the hood.
|
|
81
|
+
|
|
82
|
+
**Hand-off from qa-explore:** Not a direct handoff. Used by the consumer's team later when a generated spec starts failing due to non-bug changes.
|
|
83
|
+
|
|
84
|
+
## Bug → regression test workflow
|
|
85
|
+
|
|
86
|
+
1. **In Phase 5 or 6 of `qa-explore`**, confirm the bug is reproducible: trigger it twice in a row with the same steps and record identical results.
|
|
87
|
+
2. **In Phase 7 (reporting)**, write the bug report with exact steps-to-reproduce and business impact. Place the generated `bugs/BUG-NNN.md` in the session output directory as usual.
|
|
88
|
+
3. **After `qa-explore` finishes**, invoke the generator agent (via the loop configured at bootstrap) with the bug-report markdown as input. The generator produces `tests/repro-BUG-NNN.spec.ts`.
|
|
89
|
+
4. **Run the new spec once** with `npx playwright test tests/repro-BUG-NNN.spec.ts` to confirm it fails (the bug is still present). Commit the failing test alongside the bug report; when the fix lands, the same test will pass.
|
|
90
|
+
|
|
91
|
+
This workflow treats `qa-explore` as the discovery phase and Test Agents as the codification phase.
|
|
92
|
+
|
|
93
|
+
## Complementary tooling note
|
|
94
|
+
|
|
95
|
+
The official **Playwright MCP server** (see `github.com/microsoft/playwright-mcp`) is an alternative runtime harness for LLM-driven browser automation via accessibility trees. It overlaps with `playwright-cli` but targets long-lived agent sessions with persistent browser state. It is NOT required for Test Agents or `qa-explore` — mentioned here only so the consumer knows it exists.
|
|
96
|
+
|
|
97
|
+
## Limitations
|
|
98
|
+
|
|
99
|
+
- **Opt-in only.** This file, the `Bash(npx playwright:*)` permission, and the `@playwright/test` peer dep are all optional. Consumers on Playwright 1.0–1.55 or without `@playwright/test` are unaffected — `qa-explore` works identically for them.
|
|
100
|
+
- **Not a replacement for the charter.** Test Agents turn known behavior into specs. They do not replace exploratory discovery; a planner fed only a charter will miss what the charter itself missed.
|
|
101
|
+
- **Requires a reproducible bug.** Flaky observations, race conditions that only triggered once, and "it felt wrong" findings are valuable in the bug report but cannot be codified by the generator.
|
|
102
|
+
- **Peer dep floor is 1.59.1.** This is deliberate: the APIs referenced here (`ariaSnapshot`, `screencast`, `pickLocator`, `normalize`, `trace` CLI) are all 1.59 features. Earlier Playwright versions will either omit these methods or expose older variants that do not match the guidance below.
|
|
103
|
+
|
|
104
|
+
## Feature cross-reference
|
|
105
|
+
|
|
106
|
+
Playwright 1.56–1.59 additions and the qa-explore files that reference them:
|
|
107
|
+
|
|
108
|
+
| Feature | Playwright version | Referenced in |
|
|
109
|
+
|---|---|---|
|
|
110
|
+
| Test Agents framework (planner / generator / healer) | 1.56 | `SKILL.md`, `phases/00-setup.md`, `phases/05-features.md`, this file |
|
|
111
|
+
| `npx playwright init-agents` | 1.56 | `phases/00-setup.md`, `phases/05-features.md`, this file |
|
|
112
|
+
| `page.consoleMessages()` | 1.56 | `phases/03-discovery.md` |
|
|
113
|
+
| `page.pageErrors()` | 1.56 | `phases/03-discovery.md` |
|
|
114
|
+
| `page.requests()` | 1.56 | `phases/03-discovery.md` |
|
|
115
|
+
| Service Worker network routing | 1.56 | `phases/03-discovery.md` |
|
|
116
|
+
| `--test-list` / `--test-list-invert` | 1.56 | (used by Test Agents internally) |
|
|
117
|
+
| `page.accessibility` deprecation | 1.57 | `phases/06-edge-cases.md` |
|
|
118
|
+
| HTML reporter "Speedboard" timeline | 1.58 | `phases/07-reporting.md` |
|
|
119
|
+
| Trace Viewer themes | 1.58 | `phases/07-reporting.md` |
|
|
120
|
+
| `page.ariaSnapshot()` | 1.59 | `phases/06-edge-cases.md` |
|
|
121
|
+
| `page.pickLocator()` | 1.59 | `phases/05-features.md`, `qa-explore-quick/SKILL.md` |
|
|
122
|
+
| `locator.normalize()` | 1.59 | `phases/05-features.md`, `qa-explore-quick/SKILL.md` |
|
|
123
|
+
| `npx playwright trace` CLI | 1.59 | `phases/07-reporting.md` |
|
|
124
|
+
| `page.screencast()` | 1.59 | `phases/07-reporting.md` |
|
|
125
|
+
| `browser.bind()` | 1.59 | (reference only, not in phase guidance) |
|
|
126
|
+
| `--debug=cli` flag | 1.59 | (reference only, not in phase guidance) |
|
|
@@ -0,0 +1,146 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: qa-explore-mobile
|
|
3
|
+
description: >
|
|
4
|
+
Run a full AI-driven exploratory testing session on a simulator/emulator — works for
|
|
5
|
+
BOTH a NATIVE app (the installed app is the system under test) and a WEB app loaded in
|
|
6
|
+
the real device browser (iOS Simulator Safari / Android emulator Chrome). The mode is
|
|
7
|
+
selected from the target config. Same Principal-QA mindset as /qa-explore, but the driver
|
|
8
|
+
is Maestro (via the mobile-cli shim) instead of Playwright, and the phases are adapted for
|
|
9
|
+
mobile concerns (app/browser lifecycle, rotation, on-screen keyboard, deep links, offline,
|
|
10
|
+
permissions, native a11y tree). Use when user says: "test on the simulator/emulator",
|
|
11
|
+
"explore the app on iOS/Android", "find bugs on mobile", "QA the mobile build", or provides
|
|
12
|
+
a mobile-sim target (native or web).
|
|
13
|
+
allowed-tools: Bash(node:*), Bash(adb:*), Bash(xcrun:*), Bash(maestro:*), Read, Write, Glob, Grep
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
# Mobile Exploratory Testing Session
|
|
17
|
+
|
|
18
|
+
You are a **Principal QA Engineer** running an exploratory session on a simulator/emulator.
|
|
19
|
+
This skill works in **two modes, both co-equal, selected from the target config**:
|
|
20
|
+
|
|
21
|
+
- **NATIVE mode** — the target declares an installable app (`app.bundle_id`/`app.package` +
|
|
22
|
+
`app.app_paths`/`apk_paths`, optionally a `source_repo` with `build_commands`). The
|
|
23
|
+
installed app is the system under test; you drive it directly.
|
|
24
|
+
- **WEB mode** — the target declares a browser as the app (`app: com.apple.mobilesafari` /
|
|
25
|
+
`com.android.chrome`) plus a `web.base_url`. The browser is just the driver; the web URL
|
|
26
|
+
(the app's mobile site, e.g. `https://staging.m.example.com`) is the system under test.
|
|
27
|
+
|
|
28
|
+
The mindset and reporting standards are identical to `/qa-explore` — what changes is the
|
|
29
|
+
driver (Maestro via `mobile-cli`) and the failure modes (app/browser lifecycle, on-screen
|
|
30
|
+
keyboard, rotation, permissions, native a11y tree instead of console/network).
|
|
31
|
+
|
|
32
|
+
**Mode detection** is done in `phases/00-setup.md`: if the target's `app` id is a known
|
|
33
|
+
browser bundle AND a `web.base_url` is present → WEB mode; otherwise (non-browser app id
|
|
34
|
+
with installable artifacts) → NATIVE mode. Everything downstream (the snapshot→click→fill
|
|
35
|
+
loop, charter, discovery, journeys, features, edges, reporting) is shared and generic; each
|
|
36
|
+
phase notes the few places native vs web diverge.
|
|
37
|
+
|
|
38
|
+
Why Maestro/`mobile-cli` instead of Playwright? Playwright **cannot** drive a native app at
|
|
39
|
+
all, and it **cannot** drive iOS Simulator Safari (its "webkit" is desktop macOS WebKit, not
|
|
40
|
+
the simulator's engine). The Maestro/simctl/adb route used by `mobile-cli` is the only way to
|
|
41
|
+
exercise either a real native app or the genuine mobile-browser engine.
|
|
42
|
+
|
|
43
|
+
## The Driver — mobile-cli
|
|
44
|
+
|
|
45
|
+
A single shim wraps Maestro and `adb` / `xcrun simctl` to present a playwright-cli–shaped
|
|
46
|
+
command surface. Call it through the `bin/mcli` wrapper (it sets `JAVA_HOME` / `ANDROID_HOME`
|
|
47
|
+
/ `PATH` for you, so you don't need an export block):
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
# Isolate this session's device/app state so you don't clobber another target's state.
|
|
51
|
+
export MOBILE_CLI_STATE=/tmp/<target>-state.json
|
|
52
|
+
MCLI="$PWD/bin/mcli" # run from the repo root
|
|
53
|
+
# Then:
|
|
54
|
+
$MCLI snapshot
|
|
55
|
+
$MCLI click e7
|
|
56
|
+
$MCLI fill e9 "user@example.com"
|
|
57
|
+
# For raw adb (logcat, am start, pm) that mcli doesn't wrap, use the bin/wadb wrapper.
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
Command surface (mirrors playwright-cli where it makes sense):
|
|
61
|
+
|
|
62
|
+
| Command | Purpose |
|
|
63
|
+
|---|---|
|
|
64
|
+
| `set-device <udid>` | `emulator-5554` for Android, iOS sim UDID for iOS — sets platform |
|
|
65
|
+
| `set-app <id>` | NATIVE: the app's bundle id / package. WEB: the browser bundle `com.apple.mobilesafari` / `com.android.chrome`. |
|
|
66
|
+
| `launch` / `stop` / `relaunch-clean` | App/browser lifecycle. `relaunch-clean` is Android-only (force-stop + pm clear) — fine for a cold native start, but **avoid it in WEB mode, it wipes the logged-in browser profile.** |
|
|
67
|
+
| `open-url <url>` / `deep-link <url>` | `simctl openurl` / `am start -a VIEW -d`. WEB mode: your "address bar" to load `base_url`. NATIVE mode: fire the app's deep links. |
|
|
68
|
+
| `snapshot [--full]` | Parses Maestro's a11y hierarchy, assigns refs `e1…eN`, prints a tree. In WEB mode the browser chrome (URL bar, tabs, toolbar, keyboard) is auto-filtered. `--full` includes non-interactive nodes. KNOWN QUIRK (Android Chrome): after keyboard show/dismiss the a11y export can go stale — the WebView shows as one empty "Web View" group. Recover with `screenshot` + a coordinate tap (`bin/wadb shell input tap <x> <y>`), or `open-url` the page again to refresh the tree. |
|
|
69
|
+
| `click <ref>` | Tap by ref. Uses `adb input tap x y` on Android, `maestro tap --point` on iOS. |
|
|
70
|
+
| `fill <ref> <text>` | Tap then `adb input text` / `maestro input`. Appends to existing text — `clear` first if the field is pre-filled. Filling opens the on-screen keyboard, which SHIFTS the layout — refs cached before the keyboard are stale afterwards. Always re-`snapshot` after the last `fill` before clicking a submit control (and `press BACK` on Android first to dismiss the keyboard — with the keyboard closed BACK becomes page-back/history, so only use it while the keyboard is up). |
|
|
71
|
+
| `clear <ref>` | Tap then erase the field's existing text (Maestro `eraseText`). |
|
|
72
|
+
| `press <BACK\|ENTER\|HOME\|TAB\|ESCAPE>` | Android keyevents. iOS supports ENTER + HOME (via Maestro `pressKey`); iOS has no hardware BACK — use an in-app/on-page back control via snapshot. |
|
|
73
|
+
| `screenshot <path>` | Save PNG. |
|
|
74
|
+
| `logs [--since <sec>] [--errors]` | logcat tail (Android) / `simctl spawn log show` (iOS), filtered to the app/browser process. Replaces playwright-cli `console error`. In WEB mode it is the BROWSER's process log, not the page's JS console. |
|
|
75
|
+
| `record-start <path>` / `record-stop` | Video. Replaces playwright-cli tracing. |
|
|
76
|
+
| `running` | Foreground app — sanity check the app/browser is frontmost. |
|
|
77
|
+
|
|
78
|
+
### Safari DOM bridge — `bin/wk-ios` (iOS WEB mode only)
|
|
79
|
+
|
|
80
|
+
For iOS Safari web targets, `bin/wk-ios` restores real JS/DOM access via WebKit Remote
|
|
81
|
+
Inspector (through `ios-webkit-debug-proxy`, installed by `scripts/setup-mobile.sh`):
|
|
82
|
+
|
|
83
|
+
```bash
|
|
84
|
+
bin/wk-ios 'document.title' # eval any JS in the sim's Safari page
|
|
85
|
+
bin/wk-ios 'document.querySelectorAll(".cart_item").length' # exact-DOM assertions
|
|
86
|
+
bin/wk-ios --url # print the page's WebSocket debugger URL
|
|
87
|
+
bin/wk-ios --stop # stop the background proxy when the session ends
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
Use it whenever the a11y snapshot is not enough: exact-DOM verification (counts, attribute
|
|
91
|
+
values, hidden state), reading values the a11y tree collapses, or checking a suspected
|
|
92
|
+
rendering/JS failure. It is the same power as `playwright-cli eval`, but on the REAL iOS
|
|
93
|
+
engine. Requirements: a booted simulator with Safari on a page (`open-url` first). There is
|
|
94
|
+
no Android equivalent wired up in v1 — on Android, drive by a11y refs and `logs`.
|
|
95
|
+
|
|
96
|
+
## What's Dropped vs Playwright
|
|
97
|
+
|
|
98
|
+
Both modes are driven through the device **accessibility tree, not a DOM** — native views and
|
|
99
|
+
web content alike are surfaced to a11y (headings, links, buttons, inputs and visible text
|
|
100
|
+
become tappable refs). Versus Playwright you LOSE, in BOTH modes:
|
|
101
|
+
|
|
102
|
+
- **CSS / DOM selectors** — you drive purely by visible/accessible labels and tap coordinates (native has no DOM at all; WEB mode has one but `snapshot` doesn't read it). EXCEPTION: iOS Safari web targets can READ the DOM via `bin/wk-ios` (see above) — driving/tapping still goes through refs.
|
|
103
|
+
- **eval / JS injection** — no script context on native or Android Chrome. EXCEPTION: iOS Safari web targets get full JS eval via `bin/wk-ios`.
|
|
104
|
+
- **console & network introspection** — `logs` is the app/browser process log, not a page console; there is no network panel. (iOS WEB: `bin/wk-ios` can read `window`-level state, but there is still no console/network history.)
|
|
105
|
+
- **route / network mocking** — would need mitmproxy. Out of scope for v1; flag in the report if a test required it.
|
|
106
|
+
- **state-save / state-load (storage_state)** — no transferable cookie jar. Auth happens on the device (NATIVE: in-app login/signup; WEB: a one-time interactive login the browser profile persists — see Phase 1).
|
|
107
|
+
- **resize** — sim viewport is fixed. Use rotation instead.
|
|
108
|
+
- **multi-session race conditions** — would need two emulators. Defer to a future enhancement.
|
|
109
|
+
- **XSS payload assertions (WEB)** — without DOM/console you can't reliably confirm execution; you can still observe gross misbehavior (crash, broken render). Generic input edges (long strings, emoji, paste, RTL, control chars) still apply in both modes.
|
|
110
|
+
|
|
111
|
+
## Mobile-Specific Heuristics (added on top of FEW HICCUPPS / SFDIPOT)
|
|
112
|
+
|
|
113
|
+
When charter-selecting heuristics in Phase 2, consider these mobile dimensions (most apply to
|
|
114
|
+
both modes; mode-specific ones are tagged):
|
|
115
|
+
|
|
116
|
+
- **Lifecycle:** background the app/browser → foreground after 10 min, kill → relaunch, OS-initiated suspend mid-flow. Does state survive? Is the session still valid? (WEB: does the page reload and lose input?)
|
|
117
|
+
- **Permissions (mostly NATIVE):** deny camera / notifications / contacts / location / biometrics on first prompt; revoke and re-grant. In WEB mode this surfaces as the browser's permission prompts (getUserMedia, geolocation, notifications).
|
|
118
|
+
- **Connectivity:** airplane mode mid-action, slow 3G, captive-portal Wi-Fi, IPv6-only.
|
|
119
|
+
- **Hardware / display:** rotation, dark mode, dynamic type / large font, low-battery (≤15%), on-screen keyboard show/hide jank and input occlusion.
|
|
120
|
+
- **Deep links / URLs:** legitimate inbound link/intent, malformed scheme or query, foreign-app intent injection, very long URL.
|
|
121
|
+
- **Native a11y:** VoiceOver/TalkBack focus order, contrast, dynamic type, RTL layout — driven through the device a11y tree.
|
|
122
|
+
- **Storage / profile:** NATIVE — fresh install vs upgrade, force-clear app data mid-flow; WEB — fresh profile vs returning profile, cleared cookies, first-run interstitials.
|
|
123
|
+
|
|
124
|
+
## Session Phases
|
|
125
|
+
|
|
126
|
+
Execute each phase in order. Each phase doc points at the platform-agnostic guidance in
|
|
127
|
+
`../qa-explore/phases/` and lists the mobile-specific replacements/additions.
|
|
128
|
+
|
|
129
|
+
| Phase | File | Summary |
|
|
130
|
+
|-------|------|---------|
|
|
131
|
+
| **Setup** | `phases/00-setup.md` | Resolve target + **detect mode**, boot sim; NATIVE: verify/install/launch the app; WEB: set browser, open base_url; init session dir |
|
|
132
|
+
| **Auth** | `phases/01-auth.md` | NATIVE: in-app login/signup (static OTP from config on staging). WEB: one-time interactive SSO/MFA the profile persists — never automate MFA |
|
|
133
|
+
| **Charter** | `phases/02-charter.md` | Business context + journeys + risk ranking + heuristic selection |
|
|
134
|
+
| **Discovery** | `phases/03-discovery.md` | Map screens/views (not raw DOM); app/browser logs instead of console; flag a11y aggregation (iOS) |
|
|
135
|
+
| **Journeys** | `phases/04-journeys.md` | E2E user journeys with data verification across views |
|
|
136
|
+
| **Features** | `phases/05-features.md` | Deep feature testing — same SFDIPOT, no eval-based tampering |
|
|
137
|
+
| **Edge Cases** | `phases/06-edge-cases.md` | Mobile edges: lifecycle, permissions, rotation, keyboard, deep links, offline (mode-tagged) |
|
|
138
|
+
| **Reporting** | `phases/07-reporting.md` | Same report format. Use video instead of trace. |
|
|
139
|
+
|
|
140
|
+
## References
|
|
141
|
+
|
|
142
|
+
- `references/mobile-edges.md` — checklist of mobile-only edge cases, tagged by mode (rotation, keyboard, deep links, offline, permissions, biometrics, a11y).
|
|
143
|
+
- Content is matched by **visible text / accessibility labels** from `snapshot` (and tap coordinates when a11y aggregates nodes). There is no fixed selector/testID catalog — read the live snapshot tree each time.
|
|
144
|
+
- `../qa-explore/references/security-rules.md` — applies as-is.
|
|
145
|
+
- `../qa-explore/references/severity-guide.md` — applies as-is.
|
|
146
|
+
- `../qa-explore/references/session-rules.md` — applies as-is (the 45-min cap matters even more on mobile because emulator/sim latency adds up).
|
|
@@ -0,0 +1,203 @@
|
|
|
1
|
+
# Phase 0: Mobile Session Setup (mode-aware)
|
|
2
|
+
|
|
3
|
+
Follow `../../qa-explore/phases/00-setup.md` for the general structure (parse input, load
|
|
4
|
+
context, load knowledge, create session dir, init progress.json). The mobile differences,
|
|
5
|
+
and the native-vs-web branch, are below.
|
|
6
|
+
|
|
7
|
+
## Step 0 — Preflight (fail fast on a fresh machine)
|
|
8
|
+
|
|
9
|
+
Before anything else, verify the toolchain exists — the mobile path needs Maestro plus the
|
|
10
|
+
iOS and/or Android SDKs, which are NOT installed by default and are easy to assume present:
|
|
11
|
+
|
|
12
|
+
```bash
|
|
13
|
+
scripts/doctor-mobile.sh # or --ios / --android to scope to one platform
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
If it prints **NOT READY**, STOP: surface the `✗` items to the user and point them at
|
|
17
|
+
`scripts/setup-mobile.sh` (auto-installs the scriptable prerequisites; Xcode + the iOS
|
|
18
|
+
runtime are guided-manual). Do NOT start a session on a half-provisioned machine — a missing
|
|
19
|
+
`adb` / `maestro` / runtime fails cryptically deep in the run. Full guide: `docs/MOBILE-SETUP.md`.
|
|
20
|
+
|
|
21
|
+
## Step 1 — Parse target + DETECT MODE
|
|
22
|
+
|
|
23
|
+
From the user message or the target config in `data/targets/`, extract:
|
|
24
|
+
|
|
25
|
+
- **platform**: `android` | `ios` (required).
|
|
26
|
+
- **device**: emulator/sim identifier.
|
|
27
|
+
- Android: AVD name (from `device.avd`) or `emulator-5554` if already booted.
|
|
28
|
+
- iOS: simulator name (e.g. `iPhone 17`) or UDID.
|
|
29
|
+
- **app id**: `app.bundle_id` (iOS) / `app.package` (Android).
|
|
30
|
+
- **mode** — derive it:
|
|
31
|
+
- **WEB mode** if the app id is a known browser bundle (`com.apple.mobilesafari` /
|
|
32
|
+
`com.android.chrome`) AND the config has a `web.base_url`. The browser is the driver; the
|
|
33
|
+
web URL is the SUT. Templates: `data/targets/_example-sim-ios-safari.yml`,
|
|
34
|
+
`data/targets/_example-sim-android-chrome.yml` (read the matching one — their `notes:`
|
|
35
|
+
blocks document the web setup).
|
|
36
|
+
- **NATIVE mode** otherwise — i.e. a non-browser app id with installable artifacts
|
|
37
|
+
(`app.app_paths` / `app.apk_paths`), optionally a `source_repo` with `build_commands`.
|
|
38
|
+
The installed app is the SUT. Template: `data/targets/_example-native-mobile.yml`.
|
|
39
|
+
- **WEB only**: `web.base_url` / `web.start_url`.
|
|
40
|
+
- **NATIVE only**: `app.app_paths`/`apk_paths` (artifact to verify/install), and optional
|
|
41
|
+
`source_repo.path` + `source_repo.build_commands`.
|
|
42
|
+
|
|
43
|
+
Record `mode` — every later step branches on it.
|
|
44
|
+
|
|
45
|
+
## Step 2 — Boot the simulator/emulator if needed (shared)
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
# Android — check first, only boot if no device attached
|
|
49
|
+
bin/wadb devices
|
|
50
|
+
# If empty:
|
|
51
|
+
# export ANDROID_HOME=$HOME/Library/Android/sdk
|
|
52
|
+
# export PATH=$ANDROID_HOME/platform-tools:$ANDROID_HOME/emulator:$PATH
|
|
53
|
+
# nohup emulator -avd <avd-name> -no-snapshot -no-boot-anim -memory 4096 -gpu auto & disown
|
|
54
|
+
# adb wait-for-device
|
|
55
|
+
|
|
56
|
+
# iOS — check first, only boot if no sim booted
|
|
57
|
+
xcrun simctl list devices booted
|
|
58
|
+
# If empty:
|
|
59
|
+
# xcrun simctl boot <UDID-or-name>
|
|
60
|
+
# open -a Simulator
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
**Do NOT reboot a sim/emulator that's already running** — preserve installed apps and (WEB
|
|
64
|
+
mode) the persisted browser profile that holds the logged-in session.
|
|
65
|
+
|
|
66
|
+
Set the driver state for the rest of the phase:
|
|
67
|
+
|
|
68
|
+
```bash
|
|
69
|
+
export MOBILE_CLI_STATE=/tmp/<target>-state.json # isolate this session's device/app state
|
|
70
|
+
MCLI="$PWD/bin/mcli" # run from the repo root
|
|
71
|
+
$MCLI set-device <emulator-5554 OR ios-udid>
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Use a dedicated `MOBILE_CLI_STATE` file per target so concurrent/sequential sessions don't
|
|
75
|
+
clobber each other's device/app state.
|
|
76
|
+
|
|
77
|
+
---
|
|
78
|
+
|
|
79
|
+
## Step 3 — NATIVE branch: verify / install / (optional rebuild) / launch
|
|
80
|
+
|
|
81
|
+
Skip this whole step in WEB mode.
|
|
82
|
+
|
|
83
|
+
### 3a. Verify the app is installed
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
# Android
|
|
87
|
+
bin/wadb shell pm list packages | grep <app.package>
|
|
88
|
+
# iOS
|
|
89
|
+
xcrun simctl get_app_container <udid> <app.bundle_id> 2>/dev/null
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
### 3b. Install if missing
|
|
93
|
+
|
|
94
|
+
If not installed and the config provides an artifact path, install from it:
|
|
95
|
+
- Android: `bin/wadb install -r <app.apk_paths[...]>`
|
|
96
|
+
- iOS: `xcrun simctl install <udid> <app.app_paths[...]>`
|
|
97
|
+
|
|
98
|
+
If not installed and no artifact path is configured, **stop and ask the user** how to obtain
|
|
99
|
+
a build (point them at the target's `source_repo.build_commands` if defined). Do NOT invent a
|
|
100
|
+
build command.
|
|
101
|
+
|
|
102
|
+
### 3c. OPTIONAL refresh against latest source
|
|
103
|
+
|
|
104
|
+
**Skip unless the user passed `--rebuild` OR the session goal explicitly mentions "latest" /
|
|
105
|
+
"fresh" / "newest" build.** Otherwise run against whatever is installed (faster, preserves
|
|
106
|
+
state). This skill does **not** assume any build framework — it runs the EXACT commands the
|
|
107
|
+
target config provides:
|
|
108
|
+
|
|
109
|
+
```bash
|
|
110
|
+
# 1. Drift check (using the target's source_repo.build_commands.refresh, if defined)
|
|
111
|
+
cd <source_repo.path>
|
|
112
|
+
git status -sb && git fetch && git log HEAD..@{u} --oneline
|
|
113
|
+
|
|
114
|
+
# 2. Decide WITH the user before pulling:
|
|
115
|
+
# clean + behind → safe to pull then rebuild; dirty → ask (stash/commit/build-as-is);
|
|
116
|
+
# intentionally pinned branch → do NOT switch branches.
|
|
117
|
+
|
|
118
|
+
# 3. Run the platform build command verbatim from source_repo.build_commands
|
|
119
|
+
# (e.g. build_commands.android_release / build_commands.ios_debug). These are
|
|
120
|
+
# project-defined; the skill does not hardcode them. Long builds: run in the
|
|
121
|
+
# background and PID-wait so you get notified on completion:
|
|
122
|
+
# nohup <the configured build command> > /tmp/mobile-build.log 2>&1 &
|
|
123
|
+
# BUILD_PID=$!; until ! kill -0 $BUILD_PID 2>/dev/null; do sleep 15; done
|
|
124
|
+
|
|
125
|
+
# 4. After install, cold-start fresh: $MCLI relaunch-clean (Android) or stop+launch (iOS)
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
**Always log the source state in the report** (branch + sha + clean/dirty) so readers know
|
|
129
|
+
what code the session ran against.
|
|
130
|
+
|
|
131
|
+
### 3d. Launch the app
|
|
132
|
+
|
|
133
|
+
```bash
|
|
134
|
+
$MCLI set-app <app.bundle_id OR app.package>
|
|
135
|
+
$MCLI launch
|
|
136
|
+
$MCLI snapshot # sanity check — first screen renders
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
---
|
|
140
|
+
|
|
141
|
+
## Step 4 — WEB branch: set browser, open URL, snapshot
|
|
142
|
+
|
|
143
|
+
Skip this step in NATIVE mode.
|
|
144
|
+
|
|
145
|
+
On Android, confirm the browser exists first (Safari ships with every iOS runtime; Chrome is
|
|
146
|
+
only guaranteed on **Google-APIs/Play** images — stock AOSP images may lack it):
|
|
147
|
+
|
|
148
|
+
```bash
|
|
149
|
+
bin/wadb shell pm list packages | grep chrome # expect: package:com.android.chrome
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
If `com.android.chrome` is absent, use a Play-image AVD (or fall back to the iOS Safari
|
|
153
|
+
target) rather than assuming Chrome is installed. Do not attempt to install a browser.
|
|
154
|
+
|
|
155
|
+
```bash
|
|
156
|
+
$MCLI set-app <com.apple.mobilesafari OR com.android.chrome>
|
|
157
|
+
$MCLI launch
|
|
158
|
+
$MCLI open-url <web.base_url or web.start_url>
|
|
159
|
+
$MCLI snapshot # sanity check — the page renders
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
On a fresh Chrome profile you may see a one-time first-run / "Open tabs" promo bubble —
|
|
163
|
+
`mobile-cli` filters the routine promo out of snapshots, but if a full-screen
|
|
164
|
+
first-run/sign-in-to-Chrome interstitial appears, it must be dismissed once by hand.
|
|
165
|
+
|
|
166
|
+
---
|
|
167
|
+
|
|
168
|
+
## Step 5 — Session directory (shared)
|
|
169
|
+
|
|
170
|
+
Same as qa-explore but include `platform` in the dir name:
|
|
171
|
+
|
|
172
|
+
```
|
|
173
|
+
output/sessions/<YYYY-MM-DD-HHmm>-<target>-<platform>/
|
|
174
|
+
charter.md
|
|
175
|
+
session-log.md
|
|
176
|
+
screenshots/
|
|
177
|
+
bugs/
|
|
178
|
+
videos/ # mobile-only — for record-start output
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
## Step 6 — progress.json additions (shared)
|
|
182
|
+
|
|
183
|
+
Same base schema as qa-explore. Add a `platform` key including the resolved `mode`:
|
|
184
|
+
|
|
185
|
+
```json
|
|
186
|
+
"platform": {
|
|
187
|
+
"mode": "native",
|
|
188
|
+
"os": "android",
|
|
189
|
+
"device": "emulator-5554",
|
|
190
|
+
"app_id": "com.example.myapp",
|
|
191
|
+
"app_version": "<from `adb shell dumpsys package <pkg> | grep versionName`>",
|
|
192
|
+
"target_url": null
|
|
193
|
+
}
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
For WEB mode, set `"mode": "web"`, `app_id` to the browser bundle, and `target_url` to the
|
|
197
|
+
`base_url` (leave `app_version` null — it's the browser's, not the SUT's).
|
|
198
|
+
|
|
199
|
+
The `console_errors` counter from qa-explore becomes `app_log_errors` (app/browser-process
|
|
200
|
+
ERROR-level log matches — coarser than a real JS console, see Phase 3).
|
|
201
|
+
|
|
202
|
+
**Skip qa-explore's Step 6 (Playwright Test Agents bootstrap) entirely — not applicable when
|
|
203
|
+
the driver is mobile-cli/Maestro.**
|
|
@@ -0,0 +1,127 @@
|
|
|
1
|
+
# Phase 1: Authentication (mode-aware)
|
|
2
|
+
|
|
3
|
+
In both modes there is **NO transferable `storage_state` you can inject** — auth happens on
|
|
4
|
+
the device. Pick the branch by the target's `auth.strategy`:
|
|
5
|
+
|
|
6
|
+
- `auth.strategy: in-app` → **NATIVE branch** (account creation / login inside the app).
|
|
7
|
+
- `auth.strategy: interactive-sso` (or any interactive web SSO) → **WEB branch** (one-time
|
|
8
|
+
human login the browser profile persists).
|
|
9
|
+
- `auth.strategy: none` → skip this phase.
|
|
10
|
+
|
|
11
|
+
Set up once per Bash block:
|
|
12
|
+
|
|
13
|
+
```bash
|
|
14
|
+
export MOBILE_CLI_STATE=/tmp/<target>-state.json
|
|
15
|
+
MCLI="$PWD/bin/mcli" # run from the repo root
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## NATIVE branch — in-app login / signup
|
|
21
|
+
|
|
22
|
+
Drive the app's own auth screens via the snapshot→click→fill loop, following the steps the
|
|
23
|
+
target's `auth.strategy` describes. There is no shared session, so you authenticate inside
|
|
24
|
+
the app each run (or reuse an existing account already on the device).
|
|
25
|
+
|
|
26
|
+
### Account creation (when the charter targets signup, or as one-time setup)
|
|
27
|
+
|
|
28
|
+
```bash
|
|
29
|
+
# Optional cold start (Android): relaunch-clean wipes app data, then cold-starts.
|
|
30
|
+
$MCLI relaunch-clean
|
|
31
|
+
|
|
32
|
+
$MCLI snapshot # welcome / landing
|
|
33
|
+
$MCLI click <create-account / sign-up ref>
|
|
34
|
+
$MCLI snapshot # email / details screen
|
|
35
|
+
$MCLI fill <email input ref> "<config test_email_pattern, e.g. qa+$(date +%s)@example.com>"
|
|
36
|
+
$MCLI click <continue ref>
|
|
37
|
+
$MCLI snapshot # OTP / verification screen
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
**OTP on staging:** if the target config records a fixed code (`auth.static_otp`), use it
|
|
41
|
+
instead of waiting for a real OTP. Multi-cell OTP inputs often **do not accept a single
|
|
42
|
+
`fill`** — tap each cell and send one digit at a time:
|
|
43
|
+
|
|
44
|
+
```bash
|
|
45
|
+
$MCLI snapshot # find the per-cell OTP refs
|
|
46
|
+
for d in <digits of static_otp>; do
|
|
47
|
+
$MCLI click <next-cell-ref>
|
|
48
|
+
$MCLI fill <that-ref> "$d"
|
|
49
|
+
done
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
### Existing-user login (faster for post-auth flows)
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
$MCLI launch # do NOT relaunch-clean — preserves the existing account
|
|
56
|
+
$MCLI snapshot # if on landing, tap Log In
|
|
57
|
+
$MCLI click <login ref>
|
|
58
|
+
$MCLI fill <email/username input ref> "<existing test account>"
|
|
59
|
+
$MCLI click <continue ref>
|
|
60
|
+
# OTP same as above (static_otp on staging)
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
Generalize email/validation rules from the target config or prior findings (e.g. some backends
|
|
64
|
+
reject disposable-email domains, or client vs server validation diverges) — do not assume any
|
|
65
|
+
specific provider's quirks.
|
|
66
|
+
|
|
67
|
+
---
|
|
68
|
+
|
|
69
|
+
## WEB branch — one-time interactive SSO/MFA login
|
|
70
|
+
|
|
71
|
+
The browser keeps its own profile (cookies/session) on the device, persisted across `launch`
|
|
72
|
+
/ `open-url`. Do the login **once per device per session-lifetime; a HUMAN performs the MFA
|
|
73
|
+
step — never script it.** This applies to any interactive IdP (Microsoft Entra, Okta,
|
|
74
|
+
Google Workspace, Auth0, …) — see the web target templates (`_example-sim-ios-safari.yml`,
|
|
75
|
+
`_example-sim-android-chrome.yml`).
|
|
76
|
+
|
|
77
|
+
```bash
|
|
78
|
+
$MCLI launch # do NOT relaunch-clean — that wipes the profile
|
|
79
|
+
$MCLI open-url <web.start_url e.g. .../auth/login>
|
|
80
|
+
$MCLI snapshot # login surface / SSO button
|
|
81
|
+
$MCLI click <sign-in / SSO button ref>
|
|
82
|
+
# → hands off to the identity provider (e.g. login.microsoftonline.com, your Okta org, …).
|
|
83
|
+
# A HUMAN completes username + password + MFA by hand in the sim/emulator UI.
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
Once login completes, subsequent `launch` / `open-url` / exploration reuse the session until
|
|
87
|
+
it expires or the profile is wiped. **Never automate the MFA challenge.** If the IdP domain
|
|
88
|
+
reappears mid-session, the session lapsed — hand back to a human; do not fill credentials.
|
|
89
|
+
|
|
90
|
+
### Returning session (already logged in)
|
|
91
|
+
|
|
92
|
+
```bash
|
|
93
|
+
$MCLI launch
|
|
94
|
+
$MCLI open-url <web.base_url>
|
|
95
|
+
$MCLI snapshot # expect an authenticated landing view, not login
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
If you see the login screen or IdP domain, re-do the interactive login above.
|
|
99
|
+
|
|
100
|
+
### Profile hygiene (WEB) — what NOT to do
|
|
101
|
+
|
|
102
|
+
- **Do not run `relaunch-clean`** on a browser target — on Android it `pm clear`s the browser and forces a fresh SSO+MFA login.
|
|
103
|
+
- **Do not** inject cookies / storage_state — no supported path on a real sim browser.
|
|
104
|
+
- Boot, don't erase, the device — erasing the sim wipes the profile.
|
|
105
|
+
|
|
106
|
+
---
|
|
107
|
+
|
|
108
|
+
## Verify Auth (shared)
|
|
109
|
+
|
|
110
|
+
After auth, `$MCLI snapshot` and confirm authenticated indicators are present:
|
|
111
|
+
- NATIVE: the app's post-login home/landing controls.
|
|
112
|
+
- WEB: a known post-login heading, nav, or the user's name.
|
|
113
|
+
|
|
114
|
+
If stuck on a login/IdP screen, capture the log:
|
|
115
|
+
|
|
116
|
+
```bash
|
|
117
|
+
$MCLI logs --errors --since 60
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
**Auth issues to flag as bugs (not session blockers):**
|
|
121
|
+
- OTP rejected even with the documented static/staging code (NATIVE).
|
|
122
|
+
- A valid session dropped far sooner than expected (forced re-login mid-flow).
|
|
123
|
+
- A broken/blank state instead of redirecting to login when the session expires.
|
|
124
|
+
- Login surface renders but inputs / SSO button are unreachable in the a11y tree (blocks automation AND assistive tech).
|
|
125
|
+
- Client-side input validation (email format, required fields) diverges from server-side behavior.
|
|
126
|
+
|
|
127
|
+
**Update progress:** mark auth phase complete with `branch: native|web`, `user: <redacted>`.
|