qualiow-exploratory-testing 1.3.0 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +22 -0
- package/.env.example +27 -0
- package/CHANGELOG.md +194 -0
- package/README.md +362 -205
- package/agents/qa-gather-agent.md +24 -0
- package/{scripts → bin}/doctor-mobile.sh +57 -18
- package/bin/mcli +15 -9
- package/bin/mobile-cli.mjs +574 -286
- package/{scripts → bin}/setup-mobile.sh +56 -19
- package/bin/wadb +10 -4
- package/bin/wk-ios +29 -17
- package/bin/wkeval.mjs +7 -1
- package/data/domains/_default.yml +1 -1
- package/data/knowledge/changelog.yml +30 -12
- package/data/knowledge/manifest.yml +333 -127
- package/data/knowledge/releases/v0.1.0/entries/technique-business-logic-race-conditions.yml +1 -1
- package/data/knowledge/releases/v0.1.0/release.yml +8 -1
- package/data/knowledge/releases/v0.6.0/entries/technique-async-callback-contracts.yml +98 -0
- package/data/knowledge/releases/v0.6.0/entries/technique-exactly-once-verification.yml +116 -0
- package/data/knowledge/releases/v0.6.0/release.yml +30 -0
- package/data/security/SECURITY-POLICY.md +273 -0
- package/data/targets/_default.yml +1 -1
- package/data/targets/_example-api-only.yml +80 -0
- package/data/targets/_example-mobile-emulation.yml +2 -2
- package/data/targets/_example-native-mobile.yml +2 -2
- package/data/targets/_example-sim-android-chrome.yml +1 -1
- package/data/targets/_example-sim-ios-safari.yml +1 -1
- package/data/templates/api-probe-matrix.md +3 -3
- package/data/templates/bug-report.md +29 -151
- package/data/templates/coverage-map.md +9 -6
- package/data/templates/expected-behaviour.md +3 -2
- package/data/templates/session-charter.md +3 -0
- package/data/templates/session-report.md +57 -80
- package/dist/cli/index.js +1452 -700
- package/dist/index.d.ts +589 -81
- package/dist/index.js +1612 -227
- package/docs/BACKEND-VERIFICATION.md +39 -9
- package/docs/GETTING-STARTED.md +188 -128
- package/docs/MOBILE-SETUP.md +149 -18
- package/package.json +32 -18
- package/skills/qa-explore/SKILL.md +31 -25
- package/skills/qa-explore/phases/00-setup.md +67 -41
- package/skills/qa-explore/phases/01-auth.md +32 -15
- package/skills/qa-explore/phases/02-charter.md +16 -5
- package/skills/qa-explore/phases/03-discovery.md +50 -26
- package/skills/qa-explore/phases/04-journeys.md +2 -2
- package/skills/qa-explore/phases/05-features.md +39 -24
- package/skills/qa-explore/phases/06-edge-cases.md +24 -13
- package/skills/qa-explore/phases/07-reporting.md +79 -140
- package/skills/qa-explore/references/context-integration.md +1 -1
- package/skills/qa-explore/references/output-contract.md +187 -0
- package/skills/qa-explore/references/paths.md +84 -0
- package/skills/qa-explore/references/playwright-agents-integration.md +61 -84
- package/skills/qa-explore/references/security-rules.md +66 -25
- package/skills/qa-explore/references/severity-guide.md +31 -4
- package/skills/qa-explore-cleanup/SKILL.md +17 -6
- package/skills/qa-explore-feedback/SKILL.md +18 -5
- package/skills/qa-explore-mobile/SKILL.md +86 -64
- package/skills/qa-explore-mobile/phases/00-setup.md +71 -62
- package/skills/qa-explore-mobile/phases/01-auth.md +28 -31
- package/skills/qa-explore-mobile/phases/02-charter.md +8 -4
- package/skills/qa-explore-mobile/phases/03-discovery.md +17 -11
- package/skills/qa-explore-mobile/phases/04-journeys.md +6 -6
- package/skills/qa-explore-mobile/phases/05-features.md +9 -9
- package/skills/qa-explore-mobile/phases/06-edge-cases.md +35 -40
- package/skills/qa-explore-mobile/phases/07-reporting.md +82 -56
- package/skills/qa-explore-mobile/references/mobile-edges.md +26 -26
- package/skills/qa-explore-quick/SKILL.md +60 -25
- package/skills/qa-explore-report/SKILL.md +40 -34
- package/skills/qa-gather/SKILL.md +17 -2
- package/skills/qa-knowledge-add/SKILL.md +20 -9
- package/skills/qa-knowledge-list/SKILL.md +12 -3
- package/skills/qa-target-setup/SKILL.md +63 -41
- package/skills/qa-verify-backend/SKILL.md +26 -17
- package/skills/qa-verify-backend/phases/00-setup.md +57 -12
- package/skills/qa-verify-backend/phases/01-ac-decomposition.md +2 -2
- package/skills/qa-verify-backend/phases/03-live-verification.md +3 -3
- package/skills/qa-verify-backend/phases/03b-api-verification.md +22 -9
- package/skills/qa-verify-backend/phases/04-e2e-trigger.md +18 -2
- package/skills/qa-verify-backend/phases/05-reporting.md +38 -10
- package/skills/qa-verify-backend/references/api-probes.md +28 -1
- package/skills/qa-verify-backend/references/payload-verification.md +1 -1
- package/skills/qa-verify-backend/references/release-readiness.md +1 -1
- package/skills/qa-verify-backend/references/safety-rules.md +13 -8
- package/data/domains/_default.md +0 -181
- package/data/domains/ecommerce.md +0 -192
- package/data/domains/fintech.md +0 -200
- package/data/domains/identity.md +0 -88
- package/data/domains/marketing.md +0 -194
- package/data/domains/saas.md +0 -197
- package/dist/cli/index.d.ts +0 -1
- package/dist/cli/index.js.map +0 -1
- package/dist/index.js.map +0 -1
|
@@ -38,10 +38,19 @@ implements it.
|
|
|
38
38
|
### 2. Create a target config
|
|
39
39
|
|
|
40
40
|
```bash
|
|
41
|
-
cp data/targets/_example-backend.yml data/targets/local-my-service.yml
|
|
41
|
+
cp data/targets/_example-backend.yml data/targets/local-my-service.yml # service behind a UI
|
|
42
|
+
cp data/targets/_example-api-only.yml data/targets/local-my-api.yml # HTTP API, no UI
|
|
42
43
|
```
|
|
43
44
|
|
|
45
|
+
Both templates ship with the package, so the `cp` works from an installed copy as well as
|
|
46
|
+
from a checkout.
|
|
47
|
+
|
|
44
48
|
Fill in `base_url`, `environment.kind`, the `backend.resources` names, and `source.branch`.
|
|
49
|
+
For an API-only service most of that is empty: what matters is the `api:` block — its
|
|
50
|
+
`auth` mode, the env var holding the key, and the two allowlists. The `domain:` field names a
|
|
51
|
+
profile in `data/domains/` (`_default.yml`, `fintech.yml`, `identity.yml`, `saas.yml`,
|
|
52
|
+
`ecommerce.yml`, `marketing.yml`) — YAML only since 2.0.0 — which supplies the risk ranking
|
|
53
|
+
and integrity checks the session applies.
|
|
45
54
|
|
|
46
55
|
Two rules:
|
|
47
56
|
|
|
@@ -57,10 +66,14 @@ Two rules:
|
|
|
57
66
|
/qa-verify-backend --target local-my-service --context output/context/TICKET-123-context.md
|
|
58
67
|
```
|
|
59
68
|
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
69
|
+
All four flags:
|
|
70
|
+
|
|
71
|
+
| Flag | Effect |
|
|
72
|
+
|------|--------|
|
|
73
|
+
| `--static-only` | Branch review only — skip the live, API and end-to-end lanes. Use when you have no cloud credentials |
|
|
74
|
+
| `--api-only` | Skip the cloud lane and verify the HTTP surface |
|
|
75
|
+
| `--no-e2e` | Run everything except the write-path phase |
|
|
76
|
+
| `--parity <target-id>` | Run the same API matrix in a second environment and compare the two column by column |
|
|
64
77
|
|
|
65
78
|
### 4. Explore the UI, if the ticket has one
|
|
66
79
|
|
|
@@ -109,6 +122,20 @@ playwright-cli -s=my-env open https://service.example.com/ --headed --profile .a
|
|
|
109
122
|
playwright-cli -s=my-env eval "$(cat probes/search-cases.js)" --raw
|
|
110
123
|
```
|
|
111
124
|
|
|
125
|
+
**When the service has no UI**, none of that applies: there is no page to run inside. Declare
|
|
126
|
+
`api.auth: api-key-env` with `token_env` naming the variable and `header_name` naming the
|
|
127
|
+
header, and the lane runs out of browser, reading the key from the environment at run time.
|
|
128
|
+
Two obligations come with it. Prove the credential is doing something before the first probe —
|
|
129
|
+
one call without the header must be rejected, one with it must succeed, both recorded — or a
|
|
130
|
+
`401` rendered as JSON will read like a passing case all session. And skip the differential
|
|
131
|
+
pass with the reason stated: with no second surface every finding is "API only", which is not
|
|
132
|
+
a downgrade but the whole contract, since there is no client guard in front of the endpoint
|
|
133
|
+
for anyone.
|
|
134
|
+
|
|
135
|
+
State-changing calls are limited to `api.write_allowlist` and belong to phase 4. Where a
|
|
136
|
+
service offers a workspace reset, it stays deliberate: it is the one call that erases the
|
|
137
|
+
evidence for everything already observed.
|
|
138
|
+
|
|
112
139
|
Cases come from the ACs plus the standard families — length boundaries, tokenisation,
|
|
113
140
|
metacharacters of whatever query language sits underneath, the four different kinds of
|
|
114
141
|
nothing (`""`, `null`, whitespace-only, field absent), enum values valid and invalid,
|
|
@@ -137,7 +164,7 @@ And the payload is held next to the screen, because a correct response can still
|
|
|
137
164
|
user as a wrong number — a formatter that guesses what a value is, a unit applied twice,
|
|
138
165
|
rounding that crosses a threshold, a truncated figure shown as a total. That defect is
|
|
139
166
|
invisible from either surface alone, and it usually belongs to a different change than the one
|
|
140
|
-
under test. See
|
|
167
|
+
under test. See `skills/qa-verify-backend/references/payload-verification.md`.
|
|
141
168
|
|
|
142
169
|
## Verdicts
|
|
143
170
|
|
|
@@ -176,7 +203,7 @@ Each AC is routed to the channel that can actually falsify it
|
|
|
176
203
|
## Safety
|
|
177
204
|
|
|
178
205
|
The full policy is in
|
|
179
|
-
|
|
206
|
+
`skills/qa-verify-backend/references/safety-rules.md`. The parts worth knowing before
|
|
180
207
|
your first session:
|
|
181
208
|
|
|
182
209
|
- **Production hard stop.** If the account, profile, resource name or URL contains `prod`,
|
|
@@ -224,13 +251,16 @@ Knowledge base entries loaded by the skill (`data/knowledge/`):
|
|
|
224
251
|
| `technique-expected-behaviour-specification` | The session found behaviour no AC covers |
|
|
225
252
|
| `technique-config-surface-verification` | The change under test *is* the configuration mechanism |
|
|
226
253
|
| `technique-release-readiness-verification` | The ask is "is this release good to go" |
|
|
254
|
+
| `technique-exactly-once-verification` | Money moves, or any write that must land exactly once — assert the balance delta, not the status field, and attack redelivery and identity collision |
|
|
255
|
+
| `technique-async-callback-contracts` | A webhook or callback endpoint: what it owes its caller, and what it does with an event it cannot match |
|
|
227
256
|
|
|
228
|
-
Browse them with `/qa-knowledge-list`.
|
|
257
|
+
Browse them with `/qa-knowledge-list`. The knowledge base ships 29 entries across releases
|
|
258
|
+
v0.1.0–v0.6.0.
|
|
229
259
|
|
|
230
260
|
## Verifying a release rather than a ticket
|
|
231
261
|
|
|
232
262
|
When the ask is "is this release good to go" instead of "does this ticket meet its ACs", follow
|
|
233
|
-
|
|
263
|
+
`skills/qa-verify-backend/references/release-readiness.md`. It changes the shape of
|
|
234
264
|
the session:
|
|
235
265
|
|
|
236
266
|
- **The build table comes first, for everything.** Per ticket, is the commit it depends on
|
package/docs/GETTING-STARTED.md
CHANGED
|
@@ -2,123 +2,179 @@
|
|
|
2
2
|
|
|
3
3
|
Step-by-step guide to running your first AI-powered exploratory testing session.
|
|
4
4
|
|
|
5
|
-
## 1.
|
|
5
|
+
## 1. Install
|
|
6
6
|
|
|
7
7
|
### System requirements
|
|
8
8
|
|
|
9
|
-
- Node.js >=
|
|
10
|
-
-
|
|
11
|
-
-
|
|
9
|
+
- **Node.js >= 22.4** (`node --version`)
|
|
10
|
+
- **Claude Code**, installed and configured — the skills run as slash commands inside it
|
|
11
|
+
- `playwright-cli` arrives as a dependency of this package. There is nothing else to install
|
|
12
|
+
for the web path.
|
|
12
13
|
|
|
13
14
|
### Install the package
|
|
14
15
|
|
|
15
16
|
```bash
|
|
16
|
-
npm install -g
|
|
17
|
+
npm install -g qualiow-exploratory-testing
|
|
18
|
+
qualiow --version # prints the installed package version, e.g. 2.0.0
|
|
17
19
|
```
|
|
18
20
|
|
|
19
|
-
###
|
|
20
|
-
|
|
21
|
-
Playwright CLI is the browser control layer. Install it as a project dependency or globally.
|
|
21
|
+
### Initialise a project
|
|
22
22
|
|
|
23
23
|
```bash
|
|
24
|
-
|
|
24
|
+
cd my-project
|
|
25
|
+
qualiow init
|
|
25
26
|
```
|
|
26
27
|
|
|
27
|
-
|
|
28
|
+
That writes:
|
|
29
|
+
|
|
30
|
+
| What | Where |
|
|
31
|
+
|------|-------|
|
|
32
|
+
| the 11 skills | `.claude/skills/qa-*/` |
|
|
33
|
+
| the `qa-gather-agent` sub-agent | `.claude/agents/qa-gather-agent.md` |
|
|
34
|
+
| knowledge base, domain profiles, templates, security policy | `data/knowledge/`, `data/domains/`, `data/templates/`, `data/security/` |
|
|
35
|
+
| the `_default` target (add `--include-examples` for the `_example-*` templates and `testers-ai.yml`) | `data/targets/` |
|
|
36
|
+
| the mobile driver plus `setup-mobile.sh` and `doctor-mobile.sh`, executable | `qa/bin/` |
|
|
37
|
+
| the credential template | `qa/.env.example` |
|
|
38
|
+
| empty session/bug/context trees and the auth directory | `output/sessions/`, `output/bugs/`, `output/context/`, `.auth/` |
|
|
39
|
+
| a `# Qualiow` block appended to `.gitignore` | `.gitignore` |
|
|
40
|
+
|
|
41
|
+
Re-running `qualiow init` changes nothing unless you pass `--force`. Use `--dry-run` to see
|
|
42
|
+
the plan first.
|
|
28
43
|
|
|
29
44
|
```bash
|
|
30
|
-
|
|
45
|
+
cp qa/.env.example qa/.env # credentials go here, never in a YAML file
|
|
46
|
+
npx playwright-cli install --skills # optional: the official Playwright skill alongside
|
|
31
47
|
```
|
|
32
48
|
|
|
33
|
-
|
|
49
|
+
Alternative install paths — the Claude Code plugin (`claude --plugin-dir <repo>`, skills
|
|
50
|
+
appear as `/qualiow:qa-explore`) and a plain git checkout — are described in the project
|
|
51
|
+
README.
|
|
34
52
|
|
|
35
|
-
|
|
53
|
+
## 2. Run your first session
|
|
36
54
|
|
|
37
|
-
|
|
55
|
+
### Option A: explore a public site
|
|
38
56
|
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
| SauceDemo | https://www.saucedemo.com | ecommerce |
|
|
42
|
-
| ParaBank | https://parabank.parasoft.com/parabank/index.htm | fintech |
|
|
43
|
-
| DemoQA | https://demoqa.com | general |
|
|
44
|
-
|
|
45
|
-
Start a session inside Claude Code:
|
|
57
|
+
Point `/qa-explore` at any public site. A good first target is
|
|
58
|
+
<https://testers.ai/testing/>, which ships as the `testers-ai` target config.
|
|
46
59
|
|
|
47
60
|
```bash
|
|
48
|
-
/qa-explore https://
|
|
61
|
+
/qa-explore https://testers.ai/testing/
|
|
49
62
|
```
|
|
50
63
|
|
|
51
64
|
Claude will:
|
|
52
|
-
1. Launch a browser via Playwright CLI
|
|
53
|
-
2. Read the knowledge base for applicable heuristics
|
|
54
|
-
3. Explore the site across 3-4 phases (discovery, functional, edge cases, synthesis)
|
|
55
|
-
4. File individual bug reports in `output/sessions/<session-id>/bugs/`
|
|
56
|
-
5. Write a session report at `output/sessions/<session-id>/session-report.md`
|
|
57
|
-
|
|
58
|
-
### Option B: Quick check on a specific page
|
|
59
65
|
|
|
60
|
-
|
|
66
|
+
1. Resolve the target config and credentials, and create the session directory
|
|
67
|
+
2. Authenticate if the target requires it
|
|
68
|
+
3. Write a charter — what is being tested, why, and what "done" looks like
|
|
69
|
+
4. Read the knowledge base for the heuristics that apply to this domain
|
|
70
|
+
5. Explore across eight phases, saving findings to disk between each
|
|
71
|
+
6. File one bug report per finding in `bugs/`
|
|
72
|
+
7. Write `session-report.md` and update the session index
|
|
73
|
+
|
|
74
|
+
The eight phases and their share of the 45-minute cap:
|
|
75
|
+
|
|
76
|
+
| # | Phase | Minutes | What happens |
|
|
77
|
+
|---|-------|---------|--------------|
|
|
78
|
+
| 0 | Setup | 2 | Resolve config, create session dir, open the browser session |
|
|
79
|
+
| 1 | Auth | 2 | Log in (storage state or adaptive login) |
|
|
80
|
+
| 2 | Charter | 5 | Understand the business, risk-rank features P0–P3 |
|
|
81
|
+
| 3 | Discovery | 6 | Map the app: pages, forms, roles, console and network health |
|
|
82
|
+
| 4 | Journeys | 10 | End-to-end user journeys with data-integrity checks |
|
|
83
|
+
| 5 | Features | 10 | Deep testing of the highest-risk features |
|
|
84
|
+
| 6 | Edge cases | 6 | Boundaries, negative input, security, "what is missing?" |
|
|
85
|
+
| 7 | Reporting | 4 | Reflection, coverage map, session report, cleanup |
|
|
86
|
+
|
|
87
|
+
### Option B: quick check on one page
|
|
61
88
|
|
|
62
89
|
```bash
|
|
63
90
|
/qa-explore-quick https://app.example.com/checkout
|
|
64
91
|
```
|
|
65
92
|
|
|
66
|
-
|
|
93
|
+
A 15-minute focused session — no full site mapping. It produces the same artefacts as a full
|
|
94
|
+
session, so reports, feedback and cleanup all work on it.
|
|
95
|
+
|
|
96
|
+
### Option C: give it the requirements first
|
|
97
|
+
|
|
98
|
+
```bash
|
|
99
|
+
/qa-gather
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
Point it at a ticket, a PR, a design doc, a URL or pasted text. It writes
|
|
103
|
+
`output/context/<TICKET>-context.md`, which you then pass to a session:
|
|
104
|
+
|
|
105
|
+
```bash
|
|
106
|
+
/qa-explore --target my-app --context output/context/TICKET-123-context.md
|
|
107
|
+
```
|
|
67
108
|
|
|
68
|
-
## 3.
|
|
109
|
+
## 3. Understand the output
|
|
69
110
|
|
|
70
|
-
|
|
111
|
+
Every session — web, quick, mobile or backend — lands in
|
|
112
|
+
`output/sessions/<YYYY-MM-DD-HHmm>-<kind>-<slug>/`, where `kind` is `explore`, `quick`,
|
|
113
|
+
`mobile` or `backend`:
|
|
71
114
|
|
|
72
115
|
```
|
|
73
|
-
output/sessions
|
|
74
|
-
charter.md # what
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
phase-3-
|
|
78
|
-
|
|
116
|
+
output/sessions/2026-09-08-1813-explore-parabank/
|
|
117
|
+
charter.md # what is being tested and why
|
|
118
|
+
session-log.md # real-time findings, with the WHY not just the WHAT
|
|
119
|
+
progress.json
|
|
120
|
+
phase-3-discovery.md # phase artefacts — the file number is the phase number
|
|
121
|
+
phase-4-journeys.md
|
|
122
|
+
phase-5-features.md
|
|
123
|
+
phase-6-edge-cases.md
|
|
79
124
|
bugs/
|
|
80
|
-
BUG-001.md #
|
|
125
|
+
BUG-001.md # one bug = one report
|
|
81
126
|
BUG-002.md
|
|
82
|
-
...
|
|
83
127
|
screenshots/
|
|
84
|
-
|
|
85
|
-
|
|
128
|
+
BUG-001.png
|
|
129
|
+
videos/
|
|
130
|
+
session-report.md # the deliverable
|
|
131
|
+
stats.json # machine-readable session metrics
|
|
86
132
|
```
|
|
87
133
|
|
|
134
|
+
Plus, at the top level:
|
|
135
|
+
|
|
136
|
+
- `output/sessions/INDEX.md` — one row per session
|
|
137
|
+
(`| Date | Kind | Target | Bugs | Duration | Status | Report |`)
|
|
138
|
+
- `output/bugs/all-bugs.md` — every bug across every session
|
|
139
|
+
|
|
88
140
|
### Bug report structure
|
|
89
141
|
|
|
90
|
-
Each
|
|
142
|
+
Each report opens with the confidentiality blockquote, then:
|
|
91
143
|
|
|
92
|
-
- **Title**
|
|
93
|
-
- **Severity**
|
|
94
|
-
|
|
95
|
-
- **Expected
|
|
96
|
-
- **
|
|
97
|
-
- **
|
|
144
|
+
- **Title** — `# BUG-NNN: [Component] fails [Condition] causing [Impact]`
|
|
145
|
+
- **Severity** — Critical, High, Medium or Low (conservative by design) — plus Priority,
|
|
146
|
+
Component, URL, Environment and reproduction rate
|
|
147
|
+
- **Summary**, **Expected Behavior**, **Actual Behavior**
|
|
148
|
+
- **Steps to Reproduce** — numbered, specific enough for someone else to follow
|
|
149
|
+
- **Business Impact** — revenue, trust, regulatory, data and scale. Mandatory
|
|
150
|
+
- **Evidence** — screenshot, video, logs, console errors, network failures
|
|
151
|
+
- **Recommended Fix Priority**
|
|
98
152
|
|
|
99
153
|
### Session report structure
|
|
100
154
|
|
|
101
|
-
|
|
155
|
+
- Session metadata and executive summary
|
|
156
|
+
- Summary stats and the bug table
|
|
157
|
+
- **Coverage map** (`| Area | Risk | Status | Bugs | Notes |`) — including what was *not* tested
|
|
158
|
+
- Observations that are not bugs but are worth discussing
|
|
159
|
+
- Recommendations, reflection, session stats
|
|
102
160
|
|
|
103
|
-
|
|
104
|
-
- Bug table with severity distribution
|
|
105
|
-
- Coverage map showing which areas were tested
|
|
106
|
-
- Observations (not bugs, but worth discussing)
|
|
107
|
-
- Recommendations for follow-up testing
|
|
161
|
+
## 4. Configure a target
|
|
108
162
|
|
|
109
|
-
|
|
163
|
+
For repeated testing of the same application, save a target config.
|
|
110
164
|
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
### Generate a target config interactively
|
|
165
|
+
### Interactively
|
|
114
166
|
|
|
115
167
|
```bash
|
|
116
168
|
/qa-target-setup
|
|
117
169
|
```
|
|
118
170
|
|
|
119
|
-
|
|
171
|
+
It navigates to the URL, detects the auth requirement, walks you through the login flow, and
|
|
172
|
+
writes `qa/target.yml` — the project-local target. Pass `--shared` to write
|
|
173
|
+
`data/targets/<id>.yml` instead, for a config the whole team uses.
|
|
174
|
+
|
|
175
|
+
### Or by hand
|
|
120
176
|
|
|
121
|
-
|
|
177
|
+
`qa/target.yml`:
|
|
122
178
|
|
|
123
179
|
```yaml
|
|
124
180
|
id: my-app
|
|
@@ -147,93 +203,84 @@ scope:
|
|
|
147
203
|
- /api/*
|
|
148
204
|
```
|
|
149
205
|
|
|
150
|
-
|
|
206
|
+
Validate it before you use it:
|
|
151
207
|
|
|
152
208
|
```bash
|
|
153
|
-
|
|
209
|
+
qualiow validate --all
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
### Resolution order
|
|
213
|
+
|
|
214
|
+
```
|
|
215
|
+
--target <name> → qa/target.yml → data/targets/_default.yml
|
|
154
216
|
```
|
|
155
217
|
|
|
218
|
+
Domains come from `data/domains/*.yml` (`_default`, `ecommerce`, `fintech`, `identity`,
|
|
219
|
+
`marketing`, `saas`), and drive the risk ranking, completeness checklist and data-integrity
|
|
220
|
+
checks a session applies.
|
|
221
|
+
|
|
156
222
|
### Authentication strategies
|
|
157
223
|
|
|
158
224
|
| Strategy | When to use | Setup |
|
|
159
225
|
|----------|-------------|-------|
|
|
160
226
|
| `none` | Public sites, no login required | Default |
|
|
161
|
-
| `storage_state` |
|
|
162
|
-
| `credentials` | Username/password login |
|
|
163
|
-
| `token` | API token or bearer auth |
|
|
227
|
+
| `storage_state` | Cookie/session auth | Log in once by hand, then `playwright-cli state-save .auth/<id>.json` |
|
|
228
|
+
| `credentials` | Username/password login | Values in `qa/.env`, referenced by variable name |
|
|
229
|
+
| `token` | API token or bearer auth | Value in `qa/.env`, referenced by variable name |
|
|
164
230
|
|
|
165
|
-
|
|
231
|
+
### Credentials
|
|
166
232
|
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
MY_APP_USERNAME=testuser
|
|
170
|
-
MY_APP_PASSWORD=testpass123
|
|
171
|
-
```
|
|
172
|
-
|
|
173
|
-
## 5. How to Add Knowledge
|
|
174
|
-
|
|
175
|
-
The knowledge base drives Claude's testing strategy. Add domain-specific heuristics, techniques, or checklists.
|
|
176
|
-
|
|
177
|
-
### Add knowledge interactively
|
|
233
|
+
Secrets live in `qa/.env` (gitignored), falling back to `.env`. YAML files reference them **by
|
|
234
|
+
variable name only** — never by value. The names the shipped configs use:
|
|
178
235
|
|
|
179
236
|
```bash
|
|
180
|
-
|
|
237
|
+
# qa/.env
|
|
238
|
+
QA_USER=testuser
|
|
239
|
+
QA_PASS=...
|
|
240
|
+
QA_TOKEN=...
|
|
241
|
+
|
|
242
|
+
# /qa-verify-backend
|
|
243
|
+
QA_AWS_PROFILE=... # read-only profile. Never point this at production
|
|
244
|
+
QA_AWS_REGION=eu-west-1
|
|
245
|
+
QA_API_TOKEN=... # only for a target declaring api.auth: bearer-env
|
|
246
|
+
EXAMPLE_API_KEY=... # a no-UI service names its own variable; see _example-api-only.yml
|
|
181
247
|
```
|
|
182
248
|
|
|
183
|
-
|
|
249
|
+
## 5. Add knowledge
|
|
250
|
+
|
|
251
|
+
The knowledge base drives Claude's testing strategy: 29 entries across releases v0.1.0–v0.6.0,
|
|
252
|
+
indexed by `data/knowledge/manifest.yml`.
|
|
184
253
|
|
|
185
254
|
```bash
|
|
186
|
-
/qa-knowledge-
|
|
255
|
+
/qa-knowledge-add # add a heuristic, technique, checklist or reference
|
|
256
|
+
/qa-knowledge-list # browse what is there, by type, domain or tag
|
|
187
257
|
```
|
|
188
258
|
|
|
189
|
-
### Knowledge types
|
|
190
|
-
|
|
191
259
|
| Type | Purpose | Example |
|
|
192
260
|
|------|---------|---------|
|
|
193
261
|
| `heuristic` | Mental model or framework for exploration | SFDIPOT, FEW HICCUPPS |
|
|
194
|
-
| `technique` |
|
|
262
|
+
| `technique` | A specific testing method | boundary testing, authenticated API probing |
|
|
195
263
|
| `checklist` | Verification list for a specific area | WCAG accessibility checks |
|
|
196
264
|
| `reference` | Background material | exploratory testing type definitions |
|
|
197
265
|
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
## 6. How to Use CLI Commands
|
|
266
|
+
Entries are YAML files in `data/knowledge/releases/<version>/entries/`. After editing by hand,
|
|
267
|
+
run `qualiow kb sync` to regenerate the manifest and `qualiow kb check` to verify it.
|
|
201
268
|
|
|
202
|
-
|
|
269
|
+
## 6. CLI commands
|
|
203
270
|
|
|
204
271
|
```bash
|
|
205
|
-
qualiow
|
|
272
|
+
qualiow init --dry-run # preview what init would write
|
|
273
|
+
qualiow validate --all # targets + qa/target.yml + domains + knowledge base
|
|
274
|
+
qualiow list sessions # also: knowledge | targets | domains
|
|
275
|
+
qualiow report -s latest -f html -o report.html
|
|
276
|
+
qualiow kb check
|
|
206
277
|
```
|
|
207
278
|
|
|
208
|
-
|
|
279
|
+
`qualiow explore [url]` is **pre-flight only**: it validates the inputs, creates the session
|
|
280
|
+
directory, and prints the `/qa-explore … --session <dir>` command to run inside Claude Code.
|
|
281
|
+
It does not drive a browser itself.
|
|
209
282
|
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
```bash
|
|
213
|
-
# list all sessions
|
|
214
|
-
qualiow list sessions
|
|
215
|
-
|
|
216
|
-
# list knowledge entries
|
|
217
|
-
qualiow list knowledge
|
|
218
|
-
|
|
219
|
-
# list configured targets
|
|
220
|
-
qualiow list targets
|
|
221
|
-
|
|
222
|
-
# list domain profiles
|
|
223
|
-
qualiow list domains
|
|
224
|
-
```
|
|
225
|
-
|
|
226
|
-
### Generate reports from an existing session
|
|
227
|
-
|
|
228
|
-
```bash
|
|
229
|
-
qualiow report
|
|
230
|
-
```
|
|
231
|
-
|
|
232
|
-
Or use the skill for more control:
|
|
233
|
-
|
|
234
|
-
```bash
|
|
235
|
-
/qa-explore-report output/sessions/2026-03-28-parabank
|
|
236
|
-
```
|
|
283
|
+
There is no `qualiow gather` — it was removed in 2.0.0. Use the `/qa-gather` skill.
|
|
237
284
|
|
|
238
285
|
### Export session data programmatically
|
|
239
286
|
|
|
@@ -242,24 +289,37 @@ import {
|
|
|
242
289
|
generateHtmlReport,
|
|
243
290
|
generateJsonReport,
|
|
244
291
|
generateJiraExport,
|
|
245
|
-
} from '
|
|
292
|
+
} from 'qualiow-exploratory-testing';
|
|
293
|
+
|
|
294
|
+
const session = 'output/sessions/2026-09-08-1813-explore-parabank';
|
|
246
295
|
|
|
247
296
|
// HTML report (standalone, dark-mode, print-friendly)
|
|
248
|
-
const html = await generateHtmlReport(
|
|
297
|
+
const html = await generateHtmlReport(session);
|
|
249
298
|
|
|
250
299
|
// JSON report (structured data for dashboards and CI)
|
|
251
|
-
const json = await generateJsonReport(
|
|
300
|
+
const json = await generateJsonReport(session);
|
|
252
301
|
|
|
253
302
|
// Jira CSV (bulk import into Jira)
|
|
254
|
-
const csv = await generateJiraExport(
|
|
303
|
+
const csv = await generateJiraExport(session);
|
|
255
304
|
```
|
|
256
305
|
|
|
257
|
-
|
|
306
|
+
Every export carries the confidentiality header and runs through the same redaction list the
|
|
307
|
+
skills apply.
|
|
308
|
+
|
|
309
|
+
## 7. Beyond the desktop browser
|
|
310
|
+
|
|
311
|
+
- **Mobile** — `/qa-explore-mobile` runs a session on an iOS Simulator or Android Emulator,
|
|
312
|
+
against a native app or a web app in the real device browser. The machine needs the mobile
|
|
313
|
+
toolchain first: run `qa/bin/setup-mobile.sh`, then `qa/bin/doctor-mobile.sh` until it says
|
|
314
|
+
READY. Full guide: **MOBILE-SETUP.md**.
|
|
315
|
+
- **Backend, API and infrastructure** — `/qa-verify-backend` verifies acceptance criteria that
|
|
316
|
+
never reach a screen and produces an AC traceability matrix. Full guide:
|
|
317
|
+
**BACKEND-VERIFICATION.md**.
|
|
258
318
|
|
|
259
|
-
|
|
319
|
+
## What to do next
|
|
260
320
|
|
|
261
|
-
1. Review the session report and bug reports
|
|
262
|
-
2.
|
|
263
|
-
3. Configure a target for your own application
|
|
264
|
-
4. Add domain-specific knowledge
|
|
321
|
+
1. Review the session report and the bug reports
|
|
322
|
+
2. Run `/qa-explore-feedback` to mark false positives or note bugs the session missed
|
|
323
|
+
3. Configure a target for your own application with `/qa-target-setup`
|
|
324
|
+
4. Add domain-specific knowledge so the next session starts smarter
|
|
265
325
|
5. Run `/qa-explore-cleanup` to archive completed sessions
|