qualiow-exploratory-testing 1.0.0 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +85 -0
- package/bin/mcli +16 -0
- package/bin/mobile-cli.mjs +573 -0
- package/bin/wadb +6 -0
- package/bin/wk-ios +58 -0
- package/bin/wkeval.mjs +44 -0
- package/data/knowledge/learned-patterns.md +85 -0
- package/data/knowledge/manifest.yml +2 -1
- package/data/targets/_example-mobile-emulation.yml +52 -0
- package/data/targets/_example-native-mobile.yml +61 -0
- package/data/targets/_example-sim-android-chrome.yml +81 -0
- package/data/targets/_example-sim-ios-safari.yml +80 -0
- package/dist/cli/index.js +156 -33
- package/dist/cli/index.js.map +1 -1
- package/dist/index.d.ts +260 -527
- package/dist/index.js +142 -12
- package/dist/index.js.map +1 -1
- package/docs/MOBILE-SETUP.md +92 -0
- package/package.json +33 -11
- package/scripts/doctor-mobile.sh +143 -0
- package/scripts/setup-mobile.sh +213 -0
- package/skills/qa-explore/SKILL.md +2 -1
- package/skills/qa-explore/phases/00-setup.md +22 -1
- package/skills/qa-explore/phases/01-auth.md +4 -0
- package/skills/qa-explore/phases/03-discovery.md +14 -0
- package/skills/qa-explore/phases/05-features.md +20 -0
- package/skills/qa-explore/phases/06-edge-cases.md +2 -0
- package/skills/qa-explore/phases/07-reporting.md +11 -0
- package/skills/qa-explore/references/playwright-agents-integration.md +126 -0
- package/skills/qa-explore-mobile/SKILL.md +146 -0
- package/skills/qa-explore-mobile/phases/00-setup.md +203 -0
- package/skills/qa-explore-mobile/phases/01-auth.md +127 -0
- package/skills/qa-explore-mobile/phases/02-charter.md +70 -0
- package/skills/qa-explore-mobile/phases/03-discovery.md +79 -0
- package/skills/qa-explore-mobile/phases/04-journeys.md +49 -0
- package/skills/qa-explore-mobile/phases/05-features.md +72 -0
- package/skills/qa-explore-mobile/phases/06-edge-cases.md +157 -0
- package/skills/qa-explore-mobile/phases/07-reporting.md +123 -0
- package/skills/qa-explore-mobile/references/mobile-edges.md +117 -0
- package/skills/qa-explore-quick/SKILL.md +7 -0
- package/skills/qa-target-setup/SKILL.md +12 -0
|
@@ -0,0 +1,117 @@
|
|
|
1
|
+
# Mobile Edge Cases — Checklist (native + web)
|
|
2
|
+
|
|
3
|
+
Treat as a thinking tool, not a checkbox sheet. Pick what matches the feature and the resolved
|
|
4
|
+
**mode**. Each entry is tagged **[both]**, **[native]**, or **[web]**, with the adb / simctl
|
|
5
|
+
trigger where possible. `<app>` = the app under test's package/bundle in NATIVE mode, or the
|
|
6
|
+
browser bundle (`com.android.chrome` / `com.apple.mobilesafari`) in WEB mode. `$MCLI` is the
|
|
7
|
+
`bin/mcli` wrapper; use `bin/wadb` for raw adb.
|
|
8
|
+
|
|
9
|
+
## Lifecycle [both]
|
|
10
|
+
|
|
11
|
+
| Edge | Android trigger | iOS sim trigger |
|
|
12
|
+
|---|---|---|
|
|
13
|
+
| Background app/browser | `bin/wadb shell input keyevent KEYCODE_HOME` | home gesture/button in Simulator |
|
|
14
|
+
| Kill app/browser | `bin/wadb shell am force-stop <app>` | `xcrun simctl terminate <udid> <app>` |
|
|
15
|
+
| Cold relaunch | `$MCLI launch` (WEB: then `$MCLI open-url <url>`) | `$MCLI launch` (WEB: then `$MCLI open-url <url>`) |
|
|
16
|
+
| Clear data | NATIVE: `$MCLI relaunch-clean` (= force-stop + pm clear) — clean-state test. WEB: same command **wipes the logged-in profile; avoid unless intended** | NATIVE: `xcrun simctl uninstall` + reinstall. WEB: clear Safari data in sim Settings (drops the login) |
|
|
17
|
+
| Screen off / on | `bin/wadb shell input keyevent KEYCODE_POWER` (toggle) | n/a (sim doesn't lock realistically) |
|
|
18
|
+
|
|
19
|
+
**Ask:** does state survive? Does an in-progress action complete? Does auth/session persist, or is everything reloaded?
|
|
20
|
+
|
|
21
|
+
## Permissions [native] (and browser prompts in [web])
|
|
22
|
+
|
|
23
|
+
NATIVE: the app requests OS runtime permissions directly. WEB: the *page* requests them via
|
|
24
|
+
the BROWSER, so grant/revoke on the browser package controls `getUserMedia` camera,
|
|
25
|
+
geolocation, notifications. Use `<app>` = app package (native) or browser (web).
|
|
26
|
+
|
|
27
|
+
| Permission | Android revoke | Android grant | iOS sim |
|
|
28
|
+
|---|---|---|---|
|
|
29
|
+
| Camera | `bin/wadb shell pm revoke <app> android.permission.CAMERA` | `pm grant` | `xcrun simctl privacy <udid> reset camera <app>` (then re-prompt) |
|
|
30
|
+
| Location | `pm revoke <app> android.permission.ACCESS_FINE_LOCATION` | `pm grant` | `xcrun simctl privacy <udid> reset location <app>` |
|
|
31
|
+
| Notifications | `pm revoke <app> android.permission.POST_NOTIFICATIONS` | `pm grant` | `xcrun simctl privacy <udid> reset notifications <app>` |
|
|
32
|
+
| Contacts [native] | `pm revoke <app> android.permission.READ_CONTACTS` | `pm grant` | `xcrun simctl privacy <udid> reset contacts <app>` |
|
|
33
|
+
|
|
34
|
+
**Ask:** does the app explain WHY before prompting? Does denial crash or degrade gracefully? Can the user recover/re-grant?
|
|
35
|
+
|
|
36
|
+
## Biometrics [native]
|
|
37
|
+
|
|
38
|
+
App-level biometric auth only (a real sim browser doesn't expose a biometric flow to web).
|
|
39
|
+
Approve → succeeds; deny/cancel → password fallback; disable in OS mid-flow → graceful
|
|
40
|
+
fallback. Android emulator can simulate a fingerprint: `bin/wadb -e emu finger touch 1`.
|
|
41
|
+
|
|
42
|
+
## Push notifications [native]
|
|
43
|
+
|
|
44
|
+
If the app registers for push: deny the prompt and confirm the app still works; deliver a
|
|
45
|
+
notification and tap it — does it deep-link to the right screen with context?
|
|
46
|
+
|
|
47
|
+
## Connectivity [both]
|
|
48
|
+
|
|
49
|
+
| Edge | Android | iOS sim |
|
|
50
|
+
|---|---|---|
|
|
51
|
+
| Airplane mode | `bin/wadb shell svc wifi disable && bin/wadb shell svc data disable` (re-enable to undo) | Network Link Conditioner (manual) |
|
|
52
|
+
| Slow network | telnet to emulator console: `telnet localhost 5554`, then `network speed gsm` | Network Link Conditioner |
|
|
53
|
+
| Captive portal | hard — skip for v1 | hard — skip |
|
|
54
|
+
|
|
55
|
+
**Ask:** does the app surface offline cleanly? Does it queue actions or fail loudly? Does it
|
|
56
|
+
auto-retry? Does it leak retries (battery drain)?
|
|
57
|
+
|
|
58
|
+
## Hardware / display [both]
|
|
59
|
+
|
|
60
|
+
| Edge | Android | iOS sim |
|
|
61
|
+
|---|---|---|
|
|
62
|
+
| Rotation portrait/landscape | `bin/wadb shell settings put system user_rotation 0/1/2/3` | Cmd+Right/Left Arrow in Simulator |
|
|
63
|
+
| Dark mode | `bin/wadb shell "cmd uimode night yes"` (and `no`) | `xcrun simctl ui <udid> appearance dark/light` |
|
|
64
|
+
| Dynamic type / large font | `bin/wadb shell settings put system font_scale 1.5` | Settings app inside sim |
|
|
65
|
+
| Low battery | `bin/wadb shell dumpsys battery set level 10` (and `reset`) | n/a |
|
|
66
|
+
| Split-screen | manual via Recents | n/a |
|
|
67
|
+
|
|
68
|
+
**Ask:** does the layout reflow, clip, or hide content? Does it honor the OS dark-mode /
|
|
69
|
+
font-scale, or break? **[web]** also: address-bar show/hide on scroll changing the viewport
|
|
70
|
+
height (`100vh` jumps), momentum/overscroll bounce, and whether the page respects the
|
|
71
|
+
`viewport` meta (no desktop-width zoom-out, no horizontal scroll).
|
|
72
|
+
|
|
73
|
+
## Deep links / URL handling [both]
|
|
74
|
+
|
|
75
|
+
```bash
|
|
76
|
+
# NATIVE — the app's own scheme/intent
|
|
77
|
+
$MCLI deep-link "<app-scheme>://<a real path>"
|
|
78
|
+
$MCLI deep-link "<app-scheme>://<path>?id=&n=NaN"
|
|
79
|
+
bin/wadb shell am start -W -a android.intent.action.VIEW -d "<app-scheme>://<path>" -f 0x10000000
|
|
80
|
+
|
|
81
|
+
# WEB — the target URL
|
|
82
|
+
$MCLI open-url "<base_url>/<a real deep view>"
|
|
83
|
+
$MCLI open-url "<base_url>/<view>?id=&n=NaN"
|
|
84
|
+
$MCLI open-url "<base_url>/<view>?next=javascript:alert(1)"
|
|
85
|
+
$MCLI open-url "<base_url>/<view>?q=$(python3 -c 'print("A"*5000)')"
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
**Ask:** crash, freeze, blank screen, open-redirect, or unsanitized reflection?
|
|
89
|
+
|
|
90
|
+
## Privacy / leak surface [both]
|
|
91
|
+
|
|
92
|
+
- **Recents preview:** background while on a sensitive screen, screencap Recents — is personal data / a token-in-URL visible? A native app can mask its preview (FLAG_SECURE); web pages generally can't.
|
|
93
|
+
- **Process log:** `$MCLI logs --errors` — grep for PII or credentials (emails, tokens, secrets) leaking into the app/browser process log.
|
|
94
|
+
- **URL leakage [web]:** sensitive values in query strings end up in history and Recents — flag where they shouldn't be.
|
|
95
|
+
- **Clipboard:** if the app copies sensitive text, does it auto-clear/expire?
|
|
96
|
+
|
|
97
|
+
## Storage / state [both]
|
|
98
|
+
|
|
99
|
+
- NATIVE: force-clear app data mid-flow (`pm clear`) — clean restart or stuck intermediate screen? Fresh-install vs upgrade: did state migrate?
|
|
100
|
+
- WEB: clearing browser cookies/data drops the login session — test the cleared-state behavior knowingly, not by accident.
|
|
101
|
+
- Returning vs fresh state: correct first-run interstitials, empty states, onboarding?
|
|
102
|
+
- Long-lived session: does a session that expires while the app/page is open prompt a graceful re-auth, or break into a blank/broken state?
|
|
103
|
+
|
|
104
|
+
## Native a11y [both] (driven through the device a11y tree)
|
|
105
|
+
|
|
106
|
+
- TalkBack (Android): `bin/wadb shell settings put secure enabled_accessibility_services com.google.android.marvin.talkback/.TalkBackService`
|
|
107
|
+
- VoiceOver (iOS sim): mostly limited on sim — manual via the accessibility menu
|
|
108
|
+
- **Focus order:** swipe right repeatedly — does focus land on every interactive element in a sensible order?
|
|
109
|
+
- **Aggregation:** does each control announce its own label, or does the view dump many fields into one comma-joined label? (Known iOS issue — see `phases/03-discovery.md`.)
|
|
110
|
+
- **Contrast:** spot-check critical text — sufficient contrast in both light and dark mode?
|
|
111
|
+
- **Touch targets:** are tap targets finger-sized, or desktop-sized and hard to hit on a phone?
|
|
112
|
+
|
|
113
|
+
## Empty / boundary states [both]
|
|
114
|
+
|
|
115
|
+
- Zero items / one item / many items in lists.
|
|
116
|
+
- First-ever visit vs returning visit.
|
|
117
|
+
- Formatter extremes: very large numbers, long names, long localized strings at mobile width — truncation, overflow, `NaN`/`undefined`?
|
|
@@ -19,6 +19,12 @@ You are a **Principal QA Engineer** doing a focused 15-minute exploratory sessio
|
|
|
19
19
|
## Flow
|
|
20
20
|
|
|
21
21
|
### 1. Setup (~1 min)
|
|
22
|
+
|
|
23
|
+
Before opening the browser, check for project-local config:
|
|
24
|
+
1. If `qa/target.yml` exists in CWD, read it for base URL, auth strategy, and scope hints
|
|
25
|
+
2. If `qa/.env` exists in CWD, load credentials from it (never log these values)
|
|
26
|
+
3. If the target requires auth (storage_state or token strategy), apply it before navigating
|
|
27
|
+
|
|
22
28
|
```bash
|
|
23
29
|
playwright-cli open <url>
|
|
24
30
|
playwright-cli snapshot
|
|
@@ -56,6 +62,7 @@ Based on what's on the page, apply relevant heuristics:
|
|
|
56
62
|
- `playwright-cli network` — failed requests
|
|
57
63
|
- `playwright-cli press Tab` (x5) — focus order and visibility
|
|
58
64
|
- Responsive mobile: `playwright-cli resize 375 667`
|
|
65
|
+
- **Locator stability (optional, Playwright 1.59+):** if a target element is ambiguous, `page.pickLocator()` and `locator.normalize()` help produce a portable locator for the bug report. Only available when the consumer has installed the `@playwright/test` peer dep.
|
|
59
66
|
|
|
60
67
|
For each bug: `playwright-cli screenshot` and document immediately.
|
|
61
68
|
|
|
@@ -104,3 +104,15 @@ Present the saved config to user:
|
|
|
104
104
|
- Auth strategy configured
|
|
105
105
|
- Scope defined
|
|
106
106
|
- How to use: `/qa-explore --target <target-name>`
|
|
107
|
+
|
|
108
|
+
### 9. Optional — Offer Playwright Test Agents Bootstrap
|
|
109
|
+
|
|
110
|
+
If the user plans to author full Playwright tests against this target later (not just exploratory sessions), offer to scaffold the Test Agents workspace now while you already have credentials and scope loaded:
|
|
111
|
+
|
|
112
|
+
```bash
|
|
113
|
+
npx playwright init-agents --loop=claude
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
This creates the planner / generator / healer agent definitions in the target project (Playwright 1.56+). It is **strictly optional** — declining does not affect the target config just written. If the user accepts, note in the target config's `notes` field that Test Agents are scaffolded; if they decline, continue normally.
|
|
117
|
+
|
|
118
|
+
This step requires `@playwright/test` to be installed in the target project. If it is not, skip this step silently.
|