qualiow-exploratory-testing 1.2.0 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +22 -0
- package/.env.example +27 -0
- package/CHANGELOG.md +194 -0
- package/README.md +428 -179
- package/agents/qa-gather-agent.md +24 -0
- package/{scripts → bin}/doctor-mobile.sh +57 -18
- package/bin/mcli +15 -9
- package/bin/mobile-cli.mjs +574 -286
- package/{scripts → bin}/setup-mobile.sh +56 -19
- package/bin/wadb +10 -4
- package/bin/wk-ios +29 -17
- package/bin/wkeval.mjs +7 -1
- package/data/domains/_default.yml +1 -1
- package/data/domains/identity.yml +138 -0
- package/data/knowledge/changelog.yml +104 -0
- package/data/knowledge/learned-patterns.md +172 -86
- package/data/knowledge/manifest.yml +481 -70
- package/data/knowledge/releases/v0.1.0/entries/technique-business-logic-race-conditions.yml +1 -1
- package/data/knowledge/releases/v0.1.0/release.yml +8 -1
- package/data/knowledge/releases/v0.2.0/entries/technique-contract-narrowing.yml +150 -0
- package/data/knowledge/releases/v0.2.0/entries/technique-test-suite-audit.yml +116 -0
- package/data/knowledge/releases/v0.2.0/release.yml +20 -0
- package/data/knowledge/releases/v0.3.0/entries/technique-functional-diff-analysis.yml +119 -0
- package/data/knowledge/releases/v0.3.0/entries/technique-llm-output-verification.yml +102 -0
- package/data/knowledge/releases/v0.3.0/entries/technique-verification-mode-selection.yml +153 -0
- package/data/knowledge/releases/v0.3.0/release.yml +25 -0
- package/data/knowledge/releases/v0.4.0/entries/technique-authenticated-api-probing.yml +137 -0
- package/data/knowledge/releases/v0.4.0/entries/technique-environment-fingerprinting.yml +113 -0
- package/data/knowledge/releases/v0.4.0/entries/technique-silent-failure-audit.yml +86 -0
- package/data/knowledge/releases/v0.4.0/entries/technique-ui-api-differential.yml +97 -0
- package/data/knowledge/releases/v0.4.0/release.yml +28 -0
- package/data/knowledge/releases/v0.5.0/entries/technique-config-surface-verification.yml +77 -0
- package/data/knowledge/releases/v0.5.0/entries/technique-derived-value-verification.yml +81 -0
- package/data/knowledge/releases/v0.5.0/entries/technique-expected-behaviour-specification.yml +97 -0
- package/data/knowledge/releases/v0.5.0/entries/technique-presentation-integrity.yml +78 -0
- package/data/knowledge/releases/v0.5.0/entries/technique-release-readiness-verification.yml +110 -0
- package/data/knowledge/releases/v0.5.0/release.yml +33 -0
- package/data/knowledge/releases/v0.6.0/entries/technique-async-callback-contracts.yml +98 -0
- package/data/knowledge/releases/v0.6.0/entries/technique-exactly-once-verification.yml +116 -0
- package/data/knowledge/releases/v0.6.0/release.yml +30 -0
- package/data/security/SECURITY-POLICY.md +273 -0
- package/data/targets/_default.yml +1 -1
- package/data/targets/_example-api-only.yml +80 -0
- package/data/targets/_example-backend.yml +123 -0
- package/data/targets/_example-mobile-emulation.yml +2 -2
- package/data/targets/_example-native-mobile.yml +2 -2
- package/data/targets/_example-sim-android-chrome.yml +1 -1
- package/data/targets/_example-sim-ios-safari.yml +1 -1
- package/data/templates/api-probe-matrix.md +99 -0
- package/data/templates/bug-report.md +29 -151
- package/data/templates/coverage-map.md +23 -6
- package/data/templates/expected-behaviour.md +94 -0
- package/data/templates/session-charter.md +3 -0
- package/data/templates/session-report.md +57 -80
- package/dist/cli/index.js +1500 -739
- package/dist/index.d.ts +673 -75
- package/dist/index.js +1664 -270
- package/docs/BACKEND-VERIFICATION.md +307 -0
- package/docs/GETTING-STARTED.md +188 -128
- package/docs/MOBILE-SETUP.md +149 -18
- package/package.json +32 -16
- package/skills/qa-explore/SKILL.md +31 -25
- package/skills/qa-explore/phases/00-setup.md +67 -41
- package/skills/qa-explore/phases/01-auth.md +32 -15
- package/skills/qa-explore/phases/02-charter.md +16 -5
- package/skills/qa-explore/phases/03-discovery.md +50 -26
- package/skills/qa-explore/phases/04-journeys.md +2 -2
- package/skills/qa-explore/phases/05-features.md +39 -24
- package/skills/qa-explore/phases/06-edge-cases.md +24 -13
- package/skills/qa-explore/phases/07-reporting.md +79 -140
- package/skills/qa-explore/references/context-integration.md +1 -1
- package/skills/qa-explore/references/output-contract.md +187 -0
- package/skills/qa-explore/references/paths.md +84 -0
- package/skills/qa-explore/references/playwright-agents-integration.md +61 -84
- package/skills/qa-explore/references/security-rules.md +66 -25
- package/skills/qa-explore/references/severity-guide.md +31 -4
- package/skills/qa-explore-cleanup/SKILL.md +17 -6
- package/skills/qa-explore-feedback/SKILL.md +18 -5
- package/skills/qa-explore-mobile/SKILL.md +86 -64
- package/skills/qa-explore-mobile/phases/00-setup.md +71 -62
- package/skills/qa-explore-mobile/phases/01-auth.md +28 -31
- package/skills/qa-explore-mobile/phases/02-charter.md +8 -4
- package/skills/qa-explore-mobile/phases/03-discovery.md +17 -11
- package/skills/qa-explore-mobile/phases/04-journeys.md +6 -6
- package/skills/qa-explore-mobile/phases/05-features.md +9 -9
- package/skills/qa-explore-mobile/phases/06-edge-cases.md +35 -40
- package/skills/qa-explore-mobile/phases/07-reporting.md +82 -56
- package/skills/qa-explore-mobile/references/mobile-edges.md +26 -26
- package/skills/qa-explore-quick/SKILL.md +60 -25
- package/skills/qa-explore-report/SKILL.md +40 -34
- package/skills/qa-gather/SKILL.md +17 -2
- package/skills/qa-knowledge-add/SKILL.md +20 -9
- package/skills/qa-knowledge-list/SKILL.md +12 -3
- package/skills/qa-target-setup/SKILL.md +63 -41
- package/skills/qa-verify-backend/SKILL.md +172 -0
- package/skills/qa-verify-backend/phases/00-setup.md +142 -0
- package/skills/qa-verify-backend/phases/01-ac-decomposition.md +58 -0
- package/skills/qa-verify-backend/phases/02-static-review.md +71 -0
- package/skills/qa-verify-backend/phases/03-live-verification.md +62 -0
- package/skills/qa-verify-backend/phases/03b-api-verification.md +170 -0
- package/skills/qa-verify-backend/phases/04-e2e-trigger.md +82 -0
- package/skills/qa-verify-backend/phases/05-reporting.md +163 -0
- package/skills/qa-verify-backend/references/api-probes.md +168 -0
- package/skills/qa-verify-backend/references/aws-readonly-probes.md +156 -0
- package/skills/qa-verify-backend/references/environment-fingerprinting.md +104 -0
- package/skills/qa-verify-backend/references/payload-verification.md +112 -0
- package/skills/qa-verify-backend/references/release-readiness.md +97 -0
- package/skills/qa-verify-backend/references/safety-rules.md +95 -0
- package/dist/cli/index.d.ts +0 -1
- package/dist/cli/index.js.map +0 -1
- package/dist/index.js.map +0 -1
package/README.md
CHANGED
|
@@ -1,299 +1,548 @@
|
|
|
1
|
-
#
|
|
1
|
+
# qualiow-exploratory-testing
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
AI-powered exploratory testing for **web**, **mobile** and **backend/API** — Claude Code skills, a QA knowledge base, and the CLI that wires them into a project.
|
|
4
4
|
|
|
5
|
-
Claude acts as a Principal QA Engineer: it
|
|
5
|
+
Claude acts as a Principal QA Engineer: it understands the business first, risk-ranks what
|
|
6
|
+
matters, explores systematically, and writes bug reports an executive can act on. No test
|
|
7
|
+
scripts and no framework — AI reasoning, browser/device commands, and QA heuristics.
|
|
6
8
|
|
|
7
|
-
|
|
9
|
+
| Surface | Driver | Skill |
|
|
10
|
+
|---------|--------|-------|
|
|
11
|
+
| Web app in a desktop browser | `playwright-cli` | `/qa-explore`, `/qa-explore-quick` |
|
|
12
|
+
| Native iOS/Android app, or a web app in **real** Simulator Safari / Emulator Chrome | `bin/mobile-cli.mjs` (Maestro shim) | `/qa-explore-mobile` |
|
|
13
|
+
| Acceptance criteria with no UI at all — tables, streams, queues, IAM, HTTP endpoints | `git show`, read-only `aws-cli`, authenticated `fetch` | `/qa-verify-backend` |
|
|
8
14
|
|
|
9
|
-
|
|
15
|
+
## Requirements
|
|
10
16
|
|
|
11
|
-
- Node.js >=
|
|
12
|
-
-
|
|
13
|
-
-
|
|
17
|
+
- **Node.js >= 22.4** (`node --version`)
|
|
18
|
+
- **Claude Code** — the skills run as slash commands inside it
|
|
19
|
+
- `playwright-cli` ships as a dependency of this package; nothing extra to install for the
|
|
20
|
+
web path. Mobile needs the iOS/Android toolchains — see [docs/MOBILE-SETUP.md](docs/MOBILE-SETUP.md).
|
|
14
21
|
|
|
15
|
-
|
|
22
|
+
## Install
|
|
16
23
|
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
npm
|
|
24
|
+
Three supported paths. Pick one.
|
|
25
|
+
|
|
26
|
+
### 1. npm (recommended for a project)
|
|
20
27
|
|
|
21
|
-
|
|
28
|
+
```bash
|
|
29
|
+
npm install -g qualiow-exploratory-testing
|
|
30
|
+
cd my-project
|
|
22
31
|
qualiow init
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
`qualiow init` copies into the current project:
|
|
23
35
|
|
|
24
|
-
|
|
25
|
-
|
|
36
|
+
| Copied | To |
|
|
37
|
+
|--------|----|
|
|
38
|
+
| the 11 skills | `.claude/skills/qa-*/` |
|
|
39
|
+
| the `qa-gather-agent` sub-agent | `.claude/agents/` |
|
|
40
|
+
| knowledge base, domain profiles (`*.yml`), templates, security policy | `data/` |
|
|
41
|
+
| `_default.yml` target (with `--include-examples`: the `_example-*.yml` templates and `testers-ai.yml`) | `data/targets/` |
|
|
42
|
+
| the mobile driver, `setup-mobile.sh`, `doctor-mobile.sh` (executable) | `qa/bin/` |
|
|
43
|
+
| the credential template | `qa/.env.example` |
|
|
44
|
+
|
|
45
|
+
It also creates `output/{sessions,bugs,context}`, `.auth/` and `qa/`, and appends a
|
|
46
|
+
`# Qualiow` block to `.gitignore` (`.auth/`, `.env`, `qa/.env`, `output/sessions/*/`,
|
|
47
|
+
`!output/sessions/INDEX.md`, `output/context/*.md`, `data/targets/local-*.yml`,
|
|
48
|
+
`.playwright-cli/`, `*.trace.zip`, `*.webm`).
|
|
49
|
+
|
|
50
|
+
Re-running `init` is a no-op — existing files are kept unless you pass `--force`, and
|
|
51
|
+
`--dry-run` prints the plan without writing anything.
|
|
52
|
+
|
|
53
|
+
```bash
|
|
54
|
+
cp qa/.env.example qa/.env # credentials live here, never in YAML
|
|
55
|
+
npx playwright-cli install --skills # optional: the official Playwright skill alongside
|
|
56
|
+
/qa-explore https://your-app.com # inside Claude Code
|
|
26
57
|
```
|
|
27
58
|
|
|
28
|
-
###
|
|
59
|
+
### 2. Claude Code plugin
|
|
60
|
+
|
|
61
|
+
```bash
|
|
62
|
+
claude --plugin-dir /path/to/qualiow-exploratory-testing-skills
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
Skills appear namespaced: `/qualiow:qa-explore`, `/qualiow:qa-verify-backend`, and so on.
|
|
66
|
+
Data (knowledge, domains, templates, targets) resolves from `${CLAUDE_PLUGIN_ROOT}/data/`
|
|
67
|
+
unless the project you are in has its own `data/` or `qa/` directory, which wins.
|
|
68
|
+
|
|
69
|
+
`bin/` lands on `PATH` only for **marketplace** installs — with `--plugin-dir`, call the
|
|
70
|
+
driver by its full path, `${CLAUDE_PLUGIN_ROOT}/bin/mcli`.
|
|
71
|
+
|
|
72
|
+
### 3. Git checkout (contributors, dogfooding)
|
|
29
73
|
|
|
30
74
|
```bash
|
|
31
|
-
# clone the repository
|
|
32
75
|
git clone https://github.com/willcoliveira/qualiow-exploratory-testing-skills.git
|
|
33
76
|
cd qualiow-exploratory-testing-skills
|
|
34
|
-
|
|
35
|
-
# install dependencies
|
|
36
77
|
npm install
|
|
37
|
-
|
|
38
|
-
# build
|
|
39
78
|
npm run build
|
|
40
|
-
|
|
41
|
-
# run tests
|
|
42
79
|
npm test
|
|
43
80
|
```
|
|
44
81
|
|
|
45
|
-
|
|
82
|
+
`.claude/skills/` and `.claude/agents/` are canonical. `skills/` and `agents/` are generated
|
|
83
|
+
mirrors (shipped to npm and used by the plugin) kept in sync by `npm run sync:plugin`; CI
|
|
84
|
+
fails on drift, so never hand-edit the mirrors.
|
|
46
85
|
|
|
47
|
-
|
|
48
|
-
|---------|-------------|
|
|
49
|
-
| `qualiow init` | Initialize qualiow in the current project |
|
|
50
|
-
| `qualiow explore [url]` | Run an exploratory testing session |
|
|
51
|
-
| `qualiow validate` | Validate all configuration files (targets, knowledge, domains) |
|
|
52
|
-
| `qualiow report` | Generate a report from an existing session |
|
|
53
|
-
| `qualiow list <type>` | List sessions, knowledge, targets, or domains |
|
|
54
|
-
| `qualiow gather` | Gather and analyze requirements for testing context |
|
|
86
|
+
## CLI
|
|
55
87
|
|
|
56
|
-
|
|
88
|
+
The CLI is the plumbing around the skills — project setup, config validation, session
|
|
89
|
+
listing, report export. **The exploratory sessions themselves run inside Claude Code**, not
|
|
90
|
+
from the shell.
|
|
57
91
|
|
|
58
|
-
|
|
92
|
+
| Command | Flags | What it does |
|
|
93
|
+
|---------|-------|--------------|
|
|
94
|
+
| `qualiow init` | `--include-examples` `--force` `--dry-run` | Install skills, agent, data, `qa/bin/` and the output tree into the current project. Idempotent |
|
|
95
|
+
| `qualiow explore [url]` | `-t <target>` `-c <context>` `--time-box 45m` `--dry-run` | **Pre-flight only.** Validates the inputs, creates the session directory skeleton, and prints the `/qa-explore … --session <dir>` command to paste into Claude Code. It does not drive a browser |
|
|
96
|
+
| `qualiow validate` | `--targets` `--domains` `--knowledge` `--kb` `--all` | Validate configs against their schemas; exits 1 on any failure. `--all` also checks a project-local `qa/target.yml` and cross-checks the knowledge base against its manifest |
|
|
97
|
+
| `qualiow list <type>` | `sessions` \| `knowledge` \| `targets` \| `domains` | List what is installed or recorded |
|
|
98
|
+
| `qualiow report` | `-s <id>\|latest` `-f md\|html\|json\|jira` `-o <file>` `--stdout` | Export an existing session. `md` writes `session-summary.md`; html/json/jira carry the confidentiality header and pass through redaction |
|
|
99
|
+
| `qualiow kb <sync\|check>` | — | Regenerate or validate `data/knowledge/manifest.yml` from the release files |
|
|
100
|
+
| `qualiow --version` | — | Print the package version |
|
|
59
101
|
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
102
|
+
`qualiow gather` was **removed in 2.0.0** — it was a stub that wrote nothing. Use the
|
|
103
|
+
`/qa-gather` skill, which does the real work.
|
|
104
|
+
|
|
105
|
+
A global install also puts the mobile helpers on `PATH`: `mcli`, `wadb`, `wk-ios`,
|
|
106
|
+
`qualiow-doctor-mobile`, `qualiow-setup-mobile`.
|
|
107
|
+
|
|
108
|
+
## Running sessions
|
|
109
|
+
|
|
110
|
+
### Full session (45 minutes, 8 phases)
|
|
64
111
|
|
|
65
112
|
```bash
|
|
66
|
-
|
|
67
|
-
/qa-explore --target company-staging
|
|
113
|
+
/qa-explore https://testers.ai/testing/ # any public site
|
|
114
|
+
/qa-explore --target company-staging # a saved target config
|
|
115
|
+
/qa-explore --target company-staging --context output/context/TICKET-123-context.md
|
|
68
116
|
```
|
|
69
117
|
|
|
70
|
-
|
|
118
|
+
Setup 2 min · Auth 2 · Charter 5 · Discovery 6 · Journeys 10 · Features 10 · Edge cases 6 ·
|
|
119
|
+
Reporting 4. Findings are written to disk between phases so context never overflows.
|
|
120
|
+
|
|
121
|
+
### Quick session (15 minutes, one page or feature)
|
|
71
122
|
|
|
72
123
|
```bash
|
|
73
|
-
# check a single page or feature
|
|
74
124
|
/qa-explore-quick https://app.example.com/checkout
|
|
75
125
|
```
|
|
76
126
|
|
|
77
|
-
|
|
127
|
+
Quick sessions produce the same artefacts as a full one — `session-report.md`,
|
|
128
|
+
`bugs/BUG-NNN.md`, `stats.json`, an `INDEX.md` row — so every downstream skill can read them.
|
|
129
|
+
|
|
130
|
+
### Gather requirements first
|
|
131
|
+
|
|
132
|
+
```bash
|
|
133
|
+
/qa-gather # files, URLs, tickets, pasted text
|
|
134
|
+
```
|
|
78
135
|
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
real device browser** (iOS Simulator Safari / Android Emulator Chrome — the genuine
|
|
82
|
-
engines, which Playwright cannot drive). The mode comes from the target config.
|
|
136
|
+
Writes `output/context/<TICKET>-context.md`, which `/qa-explore` and `/qa-verify-backend`
|
|
137
|
+
consume via `--context`.
|
|
83
138
|
|
|
84
|
-
|
|
139
|
+
### After a session
|
|
85
140
|
|
|
86
141
|
```bash
|
|
87
|
-
|
|
88
|
-
#
|
|
89
|
-
|
|
142
|
+
/qa-explore-report output/sessions/2026-09-08-1813-explore-parabank # regenerate/reformat
|
|
143
|
+
/qa-explore-feedback # false positives, missed bugs
|
|
144
|
+
/qa-explore-cleanup # archive or delete old sessions
|
|
145
|
+
/qa-target-setup # configure a new target
|
|
146
|
+
/qa-knowledge-add # teach it a new heuristic
|
|
147
|
+
/qa-knowledge-list # browse the knowledge base
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
### What a session leaves behind
|
|
151
|
+
|
|
152
|
+
Every session — web, quick, mobile or backend — lands in
|
|
153
|
+
`output/sessions/<YYYY-MM-DD-HHmm>-<kind>-<slug>/`, where `kind` is `explore`, `quick`,
|
|
154
|
+
`mobile` or `backend`:
|
|
155
|
+
|
|
156
|
+
```
|
|
157
|
+
output/sessions/2026-09-08-1813-explore-parabank/
|
|
158
|
+
charter.md # what is being tested and why
|
|
159
|
+
session-log.md # real-time findings with WHY reasoning
|
|
160
|
+
phase-3-discovery.md # phase artefacts (file number == phase number)
|
|
161
|
+
phase-4-journeys.md
|
|
162
|
+
phase-5-features.md
|
|
163
|
+
phase-6-edge-cases.md
|
|
164
|
+
bugs/BUG-001.md # one bug = one report
|
|
165
|
+
screenshots/BUG-001.png
|
|
166
|
+
videos/
|
|
167
|
+
session-report.md # executive summary, coverage map, recommendations
|
|
168
|
+
stats.json
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
Plus a row in `output/sessions/INDEX.md`
|
|
172
|
+
(`| Date | Kind | Target | Bugs | Duration | Status | Report |`) and an entry in
|
|
173
|
+
`output/bugs/all-bugs.md`. Bug reports open with the two-line confidentiality blockquote and
|
|
174
|
+
the headline `# BUG-NNN: [Component] fails [Condition] causing [Impact]`.
|
|
175
|
+
|
|
176
|
+
### Project-local configuration
|
|
177
|
+
|
|
178
|
+
`/qa-target-setup` writes `qa/target.yml` (project-local, one target per repo) or, with
|
|
179
|
+
`--shared`, `data/targets/<id>.yml`. Resolution order is:
|
|
180
|
+
|
|
181
|
+
```
|
|
182
|
+
--target <name> → qa/target.yml → data/targets/_default.yml
|
|
90
183
|
```
|
|
91
184
|
|
|
92
|
-
|
|
185
|
+
Credentials come from `qa/.env`, falling back to `.env`. The env var names the shipped
|
|
186
|
+
configs reference: `QA_USER`, `QA_PASS`, `QA_TOKEN`, `QA_AWS_PROFILE`, `QA_AWS_REGION`,
|
|
187
|
+
`QA_API_TOKEN`, `EXAMPLE_API_KEY`. Never put a secret in a YAML file — reference it by
|
|
188
|
+
variable name.
|
|
189
|
+
|
|
190
|
+
## Mobile sessions
|
|
191
|
+
|
|
192
|
+
`/qa-explore-mobile` runs a full exploratory session on an iOS Simulator or Android
|
|
193
|
+
Emulator — either a **native app** (the installed app is the system under test) or a **web
|
|
194
|
+
app in the real device browser** (Simulator Safari / Emulator Chrome — the genuine engines,
|
|
195
|
+
which Playwright cannot drive). The mode comes from the target config.
|
|
196
|
+
|
|
197
|
+
### 1. One-time machine setup
|
|
93
198
|
|
|
94
199
|
```bash
|
|
95
|
-
#
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
# b) if no iOS runtime is installed:
|
|
99
|
-
xcodebuild -downloadPlatform iOS
|
|
100
|
-
# then re-run scripts/setup-mobile.sh --ios (creates the qa-iphone simulator)
|
|
200
|
+
bin/setup-mobile.sh # both platforms
|
|
201
|
+
bin/setup-mobile.sh --android # or --ios to scope
|
|
202
|
+
bin/setup-mobile.sh --yes --allow-curl-bash # non-interactive, incl. the curl|bash installers
|
|
101
203
|
```
|
|
102
204
|
|
|
103
|
-
|
|
205
|
+
It installs everything scriptable — Maestro (pinned via `MAESTRO_VERSION`, default 2.10.0),
|
|
206
|
+
JDK 17, Android cmdline-tools + platform-tools + a Google-Play system image, the standard AVD
|
|
207
|
+
`qa_pixel_api35`, the `qa-iphone` simulator device, and `ios-webkit-debug-proxy` — and prints
|
|
208
|
+
the two genuinely manual iOS steps (installing Xcode, downloading an iOS runtime).
|
|
209
|
+
|
|
210
|
+
### 2. Preflight
|
|
104
211
|
|
|
105
212
|
```bash
|
|
106
|
-
|
|
213
|
+
bin/doctor-mobile.sh # or --ios / --android / --quiet
|
|
107
214
|
```
|
|
108
215
|
|
|
109
|
-
|
|
216
|
+
Read-only. Installs nothing; prints `✓`/`✗` with the exact fix command for each gap, and
|
|
217
|
+
checks Node >= 22.4, python3, Maestro >= 2.6.0 and `ios-webkit-debug-proxy`.
|
|
110
218
|
|
|
111
|
-
|
|
219
|
+
### 3. Pick a target template
|
|
220
|
+
|
|
221
|
+
```bash
|
|
222
|
+
cp data/targets/_example-sim-ios-safari.yml data/targets/local-my-app.yml
|
|
223
|
+
```
|
|
112
224
|
|
|
113
225
|
| Template | What it tests | Driver |
|
|
114
226
|
|----------|---------------|--------|
|
|
115
227
|
| `_example-native-mobile.yml` | An installed iOS/Android app (native mode) | mobile-cli (Maestro) |
|
|
116
|
-
| `_example-sim-ios-safari.yml` | A mobile web app in
|
|
117
|
-
| `_example-sim-android-chrome.yml` | A mobile web app in
|
|
228
|
+
| `_example-sim-ios-safari.yml` | A mobile web app in **real** iOS Simulator Safari | mobile-cli (Maestro) |
|
|
229
|
+
| `_example-sim-android-chrome.yml` | A mobile web app in **real** Android Emulator Chrome | mobile-cli (Maestro) |
|
|
118
230
|
| `_example-mobile-emulation.yml` | A mobile web app via Playwright device emulation (no simulator) | `/qa-explore` (playwright-cli) |
|
|
119
231
|
|
|
120
232
|
```bash
|
|
121
|
-
/qa-explore-mobile --target my-
|
|
122
|
-
/qa-explore --target my-emulation
|
|
233
|
+
/qa-explore-mobile --target local-my-app # real sim/emulator, native or web
|
|
234
|
+
/qa-explore --target local-my-emulation # Playwright mobile emulation
|
|
123
235
|
```
|
|
124
236
|
|
|
125
|
-
|
|
237
|
+
### 4. Driving the device by hand
|
|
126
238
|
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
~/Library/Android/sdk/emulator/emulator \
|
|
130
|
-
-avd qa_pixel_api35 -no-snapshot & xcrun simctl boot qa-iphone && open -a Simulator
|
|
239
|
+
The skill runs `bin/mcli` (or `qa/bin/mcli` in an init'ed project, or `mcli` on `PATH`).
|
|
240
|
+
Everything it does is available to you directly:
|
|
131
241
|
|
|
132
|
-
|
|
133
|
-
bin/mcli
|
|
134
|
-
bin/mcli
|
|
242
|
+
```bash
|
|
243
|
+
bin/mcli boot qa_pixel_api35 # or an iOS sim name / UDID
|
|
244
|
+
bin/mcli --state /tmp/my-run.json set-device emulator-5554 --platform android
|
|
245
|
+
bin/mcli set-app com.android.chrome # or com.apple.mobilesafari, or your app id
|
|
135
246
|
bin/mcli launch
|
|
136
|
-
bin/mcli open-url https://staging.m.example.com
|
|
137
|
-
bin/mcli snapshot #
|
|
138
|
-
bin/mcli fill e4 my-username
|
|
247
|
+
bin/mcli open-url https://staging.m.example.com # web mode (alias of deep-link)
|
|
248
|
+
bin/mcli snapshot # a11y tree as refs e1, e2, … (refs expire after 60 s)
|
|
249
|
+
bin/mcli fill e4 my-username # Maestro inputText
|
|
139
250
|
bin/mcli snapshot # ALWAYS re-snapshot after the keyboard appears
|
|
140
251
|
bin/mcli click e7
|
|
141
|
-
bin/mcli
|
|
142
|
-
bin/mcli
|
|
143
|
-
bin/mcli
|
|
252
|
+
bin/mcli fill-id login-btn "pass" # tap + type by testID, no snapshot needed
|
|
253
|
+
bin/mcli wait-text "Dashboard" 15
|
|
254
|
+
bin/mcli press HOME # BACK | ENTER | HOME | TAB | ESCAPE
|
|
255
|
+
bin/mcli screenshot evidence.png # no app required
|
|
256
|
+
bin/mcli logs --errors --since 60 # logcat / simctl log; logs-clear on Android
|
|
257
|
+
bin/mcli record-start run.mp4 && bin/mcli record-stop # mp4 on both platforms
|
|
258
|
+
bin/mcli info # resolved state + which state file is in use
|
|
259
|
+
bin/mcli --help
|
|
144
260
|
```
|
|
145
261
|
|
|
146
|
-
|
|
262
|
+
Exit codes: `0` ok · `1` command failed · `2` usage · `3` stale/unknown ref (re-run
|
|
263
|
+
`snapshot`) · `4` no device or app set.
|
|
264
|
+
|
|
265
|
+
iOS Safari web targets also get real JS/DOM access through the WebKit bridge (needs
|
|
266
|
+
`ios-webkit-debug-proxy`, `python3` and Node >= 22.4; `WK_IOS_PORT` overrides the port):
|
|
147
267
|
|
|
148
268
|
```bash
|
|
149
|
-
bin/wk-ios 'document.title'
|
|
150
|
-
bin/wk-ios 'document.querySelectorAll(".cart_item").length'
|
|
151
|
-
bin/wk-ios --stop
|
|
269
|
+
bin/wk-ios 'document.title'
|
|
270
|
+
bin/wk-ios 'document.querySelectorAll(".cart_item").length'
|
|
271
|
+
bin/wk-ios --stop
|
|
152
272
|
```
|
|
153
273
|
|
|
154
274
|
Notes that save time:
|
|
275
|
+
|
|
155
276
|
- **Never `relaunch-clean` a web target** — it wipes the logged-in browser profile.
|
|
156
|
-
- SSO/MFA logins are done **once, by hand, on the device
|
|
157
|
-
|
|
158
|
-
-
|
|
159
|
-
- Full
|
|
277
|
+
- SSO/MFA logins are done **once, by hand, on the device**. There is no transferable
|
|
278
|
+
`storage_state` for a real device browser.
|
|
279
|
+
- `bin/wadb` is raw `adb` with `ANDROID_HOME` and JDK 17 resolved for you.
|
|
280
|
+
- Full guide and troubleshooting: **[docs/MOBILE-SETUP.md](docs/MOBILE-SETUP.md)**.
|
|
160
281
|
|
|
161
|
-
|
|
282
|
+
## Backend, API and infrastructure verification
|
|
162
283
|
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
### Add knowledge to the base
|
|
284
|
+
Not every acceptance criterion is visible in a browser. `/qa-verify-backend` covers tickets
|
|
285
|
+
whose ACs live below the UI — tables and streams, queue consumers, Lambda triggers, IAM
|
|
286
|
+
policies, webhooks, IaC, and the service's own HTTP endpoints — and produces an **AC
|
|
287
|
+
traceability matrix** instead of a pass/fail claim.
|
|
169
288
|
|
|
170
289
|
```bash
|
|
171
|
-
#
|
|
172
|
-
/
|
|
173
|
-
```
|
|
290
|
+
cp data/targets/_example-backend.yml data/targets/local-my-service.yml # service behind a UI
|
|
291
|
+
cp data/targets/_example-api-only.yml data/targets/local-my-api.yml # HTTP API, no UI
|
|
174
292
|
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
/qa-target-setup
|
|
293
|
+
/qa-verify-backend --target local-my-service --context output/context/TICKET-123-context.md
|
|
294
|
+
/qa-verify-backend --target local-my-service --static-only # no cloud credentials
|
|
295
|
+
/qa-verify-backend --target local-my-service --no-e2e # skip the write path
|
|
296
|
+
/qa-verify-backend --target local-my-api --api-only --parity local-my-api-staging
|
|
180
297
|
```
|
|
181
298
|
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
299
|
+
| Lane | What it does |
|
|
300
|
+
|------|--------------|
|
|
301
|
+
| **Static** | Reads the implementation branch against each AC with `git show` (never checks out) — spec drift, scope creep, failure paths, identity propagation, producer/consumer contract breaks |
|
|
302
|
+
| **Live** | Read-only `aws-cli` probes, one per AC, raw output saved as evidence. Never mutates; hard stop on production |
|
|
303
|
+
| **API** | Calls the service's own endpoints from the already-authenticated page, with a written case matrix covering everything the browser's guards prevent. Read-only, limited to the target's `api.probe_allowlist` |
|
|
304
|
+
| **End-to-end** | Drives the real write path in a non-production environment, then re-probes the data layer — same-tick writes, deletes, bulk saves, a second identity, DLQ depth |
|
|
305
|
+
|
|
306
|
+
Each AC gets a verdict with cited evidence — `PASS`, `PARTIAL`, `FAIL`, `BLOCKED`,
|
|
307
|
+
`NOT-REACHABLE` or `UNVERIFIABLE` — and the verdict **names the mode it was reached by and the
|
|
308
|
+
environment it holds in**. `PASS (direct request, raw response attached)` and `PASS (read the
|
|
309
|
+
diff)` are different claims, and a verdict backed only by a code reading is `UNVERIFIABLE`,
|
|
310
|
+
never `PASS`. `BLOCKED` is a first-class outcome: when credentials are missing, the probe
|
|
311
|
+
commands are written out ready to run rather than the verdict being inferred from source.
|
|
312
|
+
|
|
313
|
+
### Two things it will not let you get away with
|
|
314
|
+
|
|
315
|
+
**Believing a green result from an environment that does not run your code.** Before the first
|
|
316
|
+
probe, the skill fingerprints every component of the request path: which build is deployed,
|
|
317
|
+
whether the changed path is *selected* here, and whether the commit under test is genuinely an
|
|
318
|
+
ancestor of what is running. One identical build can hold two implementations of the same
|
|
319
|
+
feature with a flag choosing between them — *same build, different behaviour ⇒ configuration,
|
|
320
|
+
not deploy lag*. An environment that does not run the change gets `NOT-REACHABLE`.
|
|
321
|
+
|
|
322
|
+
**Recording a client-side guard as the endpoint's behaviour.** "The search box refuses to fire
|
|
323
|
+
when empty" describes the browser. Every other client — mobile, partner integration, script —
|
|
324
|
+
sends that request, and it is frequently the one that fails. When a ticket has both a screen
|
|
325
|
+
and an endpoint, the same cases run at both surfaces and each finding is sorted into *both*,
|
|
326
|
+
*API only* (a real defect the guard is hiding), *UI only* (the client invents or masks
|
|
327
|
+
behaviour the service does not have) or *neither*.
|
|
328
|
+
|
|
329
|
+
The API lane needs no token handling: the request runs inside the page that is already logged
|
|
330
|
+
in, so it carries the same session cookie, CSRF token and interceptors as the UI — which also
|
|
331
|
+
means it works with SSO and MFA that no scripted login can pass. Credentials stay in the
|
|
332
|
+
browser profile under `.auth/` and never enter a script, a report or the repository. A service
|
|
333
|
+
with **no UI at all** declares `api.auth: api-key-env` and the lane runs out of browser,
|
|
334
|
+
reading the key from the environment variable the target names.
|
|
335
|
+
|
|
336
|
+
### Correct shape is not a correct answer
|
|
337
|
+
|
|
338
|
+
Every derived value — a percentage, total, ratio, delta or aggregate — is recomputed from the
|
|
339
|
+
raw figures in the same response, with the formula taken from the specification rather than from
|
|
340
|
+
the code under test, and with cases chosen to stress sign, zero, scale and cardinality. Then the
|
|
341
|
+
report states the limitation plainly: when both sides of the check come from one payload, the
|
|
342
|
+
derivation is verified and the inputs are not, so the **independent oracle** that would close
|
|
343
|
+
the gap is named along with whether it was run.
|
|
344
|
+
|
|
345
|
+
The payload is also held next to the screen, because a correct response can still reach the user
|
|
346
|
+
as a wrong number — a formatter that guesses what a value is, a unit applied twice, a truncated
|
|
347
|
+
figure presented as a total. That defect is invisible from either surface alone, and it usually
|
|
348
|
+
belongs to a different change than the one under test.
|
|
349
|
+
|
|
350
|
+
### Findings with no acceptance criterion become a spec
|
|
351
|
+
|
|
352
|
+
Most of what an API probe turns up has nothing to be filed against, so it turns into an argument
|
|
353
|
+
rather than a fix. `data/templates/expected-behaviour.md` converts a pile of observations into
|
|
354
|
+
one reviewable decision: observed against expected, grouped by cause, with the decisions the fix
|
|
355
|
+
forces made explicit — reject rather than clamp, an error rather than a silent zero, validation
|
|
356
|
+
at the layer that covers every implementation — ranked by what real users can reach today, and
|
|
357
|
+
closed with a plain-English reply for whoever decides to fund the work.
|
|
358
|
+
|
|
359
|
+
### Verifying a release rather than a ticket
|
|
360
|
+
|
|
361
|
+
[`skills/qa-verify-backend/references/release-readiness.md`](skills/qa-verify-backend/references/release-readiness.md)
|
|
362
|
+
changes the shape of the session: the deployment table first for every ticket, so a ticket whose
|
|
363
|
+
backend is not in the build is marked *not testable here* rather than tested against a UI that
|
|
364
|
+
will render convincing nonsense; a result vocabulary that keeps *not tested* visible; a coverage
|
|
365
|
+
map where 🔍 *code-verified only* is marked as `UNVERIFIABLE` rather than green, with one
|
|
366
|
+
sentence naming the untested item that carries the most risk; carry-over defects in their own
|
|
367
|
+
section; and a disposition with the condition that would reverse it.
|
|
368
|
+
|
|
369
|
+
Full guide: **[docs/BACKEND-VERIFICATION.md](docs/BACKEND-VERIFICATION.md)**. Safety rules
|
|
370
|
+
(read-only discipline, the production hard stop, redaction):
|
|
371
|
+
[`skills/qa-verify-backend/references/safety-rules.md`](skills/qa-verify-backend/references/safety-rules.md).
|
|
372
|
+
|
|
373
|
+
## Reports
|
|
374
|
+
|
|
375
|
+
`qualiow report` exports an existing session in four formats. Every export carries the
|
|
376
|
+
confidentiality header and runs through the same redaction list as the skills.
|
|
189
377
|
|
|
190
378
|
```bash
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
379
|
+
qualiow report -s latest -f html -o report.html
|
|
380
|
+
qualiow report -s 2026-09-08-1813-explore-parabank -f jira -o bugs.csv
|
|
381
|
+
qualiow report -s latest -f json --stdout
|
|
194
382
|
```
|
|
195
383
|
|
|
196
|
-
|
|
384
|
+
| Format | Output |
|
|
385
|
+
|--------|--------|
|
|
386
|
+
| `md` (default) | `session-summary.md` next to the session report — never overwrites it |
|
|
387
|
+
| `html` | Self-contained dark-mode page: severity cards, bug details, coverage map, recommendations. Print-friendly, CSP-restricted, `noindex` |
|
|
388
|
+
| `json` | Structured data for dashboards, CI and metrics tracking |
|
|
389
|
+
| `jira` | CSV for Jira bulk import; severity maps to priority (Critical → Blocker, High → Critical, Medium → Major, Low → Minor) |
|
|
197
390
|
|
|
198
|
-
|
|
391
|
+
The same formatters are exported as a library:
|
|
199
392
|
|
|
200
|
-
```
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
393
|
+
```typescript
|
|
394
|
+
import {
|
|
395
|
+
generateHtmlReport,
|
|
396
|
+
generateJsonReport,
|
|
397
|
+
generateJiraExport,
|
|
398
|
+
} from 'qualiow-exploratory-testing';
|
|
205
399
|
|
|
206
|
-
|
|
400
|
+
const session = 'output/sessions/2026-09-08-1813-explore-parabank';
|
|
207
401
|
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
# from TypeScript
|
|
212
|
-
import { generateJiraExport } from '@qualiow/exploratory-testing';
|
|
213
|
-
const csv = await generateJiraExport('output/sessions/2026-03-28-parabank');
|
|
402
|
+
const html = await generateHtmlReport(session);
|
|
403
|
+
const json = await generateJsonReport(session);
|
|
404
|
+
const csv = await generateJiraExport(session);
|
|
214
405
|
```
|
|
215
406
|
|
|
216
|
-
##
|
|
407
|
+
## Skills
|
|
217
408
|
|
|
218
409
|
| Skill | Purpose |
|
|
219
410
|
|-------|---------|
|
|
220
|
-
| `/qa-explore` | Full exploratory testing session (45 min) |
|
|
221
|
-
| `/qa-explore-mobile` | Exploratory session on a simulator/emulator — native apps or mobile web in the real device browser (iOS Safari / Android Chrome) |
|
|
411
|
+
| `/qa-explore` | Full exploratory testing session (45 min, 8 phases) |
|
|
222
412
|
| `/qa-explore-quick` | Quick focused session on a single page or feature (15 min) |
|
|
223
|
-
| `/qa-explore-
|
|
413
|
+
| `/qa-explore-mobile` | Session on a simulator/emulator — native apps or web in the real device browser |
|
|
414
|
+
| `/qa-verify-backend` | Backend/API/infra AC verification — branch review, read-only cloud probes, API probes, traceability matrix |
|
|
415
|
+
| `/qa-gather` | Gather and analyze requirements from files, URLs, tickets or text into a context file |
|
|
416
|
+
| `/qa-explore-report` | Generate, regenerate or reformat a report from an existing session |
|
|
224
417
|
| `/qa-explore-feedback` | Post-session feedback capture (false positives, missed bugs) |
|
|
225
|
-
| `/qa-explore-cleanup` | Session
|
|
226
|
-
| `/qa-knowledge-add` | Add
|
|
227
|
-
| `/qa-knowledge-list` | Browse the knowledge base |
|
|
228
|
-
| `/qa-target-setup` | Configure a
|
|
229
|
-
| `/qa-gather` | Gather and analyze requirements from files, URLs, or text |
|
|
418
|
+
| `/qa-explore-cleanup` | Session listing, archival and deletion |
|
|
419
|
+
| `/qa-knowledge-add` | Add heuristics, techniques, checklists or references to the knowledge base |
|
|
420
|
+
| `/qa-knowledge-list` | Browse and search the knowledge base |
|
|
421
|
+
| `/qa-target-setup` | Configure a target application (auth, scope, domain) |
|
|
230
422
|
|
|
231
|
-
|
|
423
|
+
Under the plugin install these are `/qualiow:qa-explore` and so on. One sub-agent ships
|
|
424
|
+
alongside them: `qa-gather-agent`, the background runner behind `/qa-gather`.
|
|
425
|
+
|
|
426
|
+
## Project structure
|
|
232
427
|
|
|
233
428
|
```
|
|
234
429
|
qualiow-exploratory-testing-skills/
|
|
430
|
+
.claude/
|
|
431
|
+
skills/ # the 11 skills — CANONICAL source
|
|
432
|
+
agents/qa-gather-agent.md # the gather sub-agent — CANONICAL source
|
|
433
|
+
skills/ # generated mirror (npm + plugin) — npm run sync:plugin
|
|
434
|
+
agents/ # generated mirror
|
|
435
|
+
.claude-plugin/plugin.json # Claude Code plugin manifest
|
|
436
|
+
bin/
|
|
437
|
+
mcli, mobile-cli.mjs # Maestro/simctl/adb shim + permission-friendly wrapper
|
|
438
|
+
wadb, wk-ios, wkeval.mjs # adb wrapper, iOS WebKit DOM bridge
|
|
439
|
+
setup-mobile.sh # one-time toolchain bootstrap
|
|
440
|
+
doctor-mobile.sh # read-only preflight
|
|
441
|
+
scripts/ # repo dev tooling, not shipped:
|
|
442
|
+
# kb-sync.mjs, check-pack.mjs, sync-version.mjs
|
|
443
|
+
data/
|
|
444
|
+
knowledge/ # YAML knowledge base, versioned as releases v0.1.0…v0.6.0
|
|
445
|
+
domains/ # domain profiles (*.yml)
|
|
446
|
+
templates/ # bug report, session report, charter, coverage map, AC matrix
|
|
447
|
+
targets/ # target configs (_default, _example-*, public POC sites)
|
|
448
|
+
security/SECURITY-POLICY.md
|
|
449
|
+
docs/ # GETTING-STARTED, MOBILE-SETUP, BACKEND-VERIFICATION,
|
|
450
|
+
# KNOWN-ISSUES, ARCHITECTURE-DECISIONS
|
|
235
451
|
src/
|
|
236
|
-
cli/
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
html-report.ts # standalone dark-mode HTML report
|
|
241
|
-
json-report.ts # structured JSON report
|
|
242
|
-
jira-export.ts # Jira-importable CSV
|
|
243
|
-
index.ts # re-exports
|
|
244
|
-
schemas/ # Zod validation schemas
|
|
245
|
-
types/ # TypeScript type definitions
|
|
246
|
-
utils/
|
|
247
|
-
metrics.ts # session metrics (JSONL append/read)
|
|
248
|
-
parse-session.ts # parse session dirs into structured data
|
|
249
|
-
redact.ts # credential and secret redaction
|
|
250
|
-
validate.ts # YAML config validation
|
|
452
|
+
cli/ # commander entry + command handlers
|
|
453
|
+
formatters/ # html, json, jira, markdown summary
|
|
454
|
+
schemas/ # Zod schemas for targets, domains, knowledge, metrics
|
|
455
|
+
types/ utils/ # redaction, session parsing, paths, metrics, validation
|
|
251
456
|
index.ts # public API re-exports
|
|
252
|
-
data/
|
|
253
|
-
knowledge/ # YAML knowledge base (heuristics, techniques)
|
|
254
|
-
templates/ # bug report, session report, charter templates
|
|
255
|
-
targets/ # target application configs
|
|
256
|
-
domains/ # domain-specific testing focus
|
|
257
|
-
security/ # security policy
|
|
258
|
-
output/
|
|
259
|
-
sessions/ # session outputs (reports, bugs, screenshots)
|
|
260
457
|
tests/
|
|
261
|
-
unit/
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
docs/
|
|
265
|
-
GETTING-STARTED.md # detailed setup and usage guide
|
|
266
|
-
ARCHITECTURE-DECISIONS.md # architecture decision records
|
|
458
|
+
unit/ fixtures/ # vitest
|
|
459
|
+
output/
|
|
460
|
+
sessions/ bugs/ context/
|
|
267
461
|
```
|
|
268
462
|
|
|
269
|
-
|
|
463
|
+
In a project you ran `qualiow init` in:
|
|
464
|
+
|
|
465
|
+
```
|
|
466
|
+
my-project/
|
|
467
|
+
.claude/skills/qa-* # the 11 skills
|
|
468
|
+
.claude/agents/qa-gather-agent.md
|
|
469
|
+
data/ # knowledge, domains, templates, security, targets
|
|
470
|
+
qa/
|
|
471
|
+
target.yml # project-local target (from /qa-target-setup)
|
|
472
|
+
.env # credentials — gitignored
|
|
473
|
+
.env.example
|
|
474
|
+
bin/ # mobile driver + setup/doctor scripts
|
|
475
|
+
output/sessions|bugs|context
|
|
476
|
+
.auth/ # storage states and browser profiles — gitignored
|
|
477
|
+
```
|
|
478
|
+
|
|
479
|
+
## Tech stack
|
|
270
480
|
|
|
271
481
|
| Technology | Purpose |
|
|
272
482
|
|------------|---------|
|
|
273
|
-
| TypeScript | type-safe source code |
|
|
274
|
-
| Node.js >= 18 | runtime |
|
|
275
|
-
| Playwright CLI | browser control for AI agents |
|
|
276
483
|
| Claude Code | AI reasoning and session orchestration |
|
|
277
|
-
|
|
|
484
|
+
| Playwright CLI (`@playwright/cli`) | browser control for the web path |
|
|
485
|
+
| Maestro + `xcrun simctl` / `adb` | device control for the mobile path |
|
|
486
|
+
| TypeScript / Node.js >= 22.4 | CLI, formatters, schemas |
|
|
278
487
|
| Zod | runtime schema validation |
|
|
279
488
|
| Commander | CLI argument parsing |
|
|
280
|
-
|
|
|
281
|
-
|
|
|
489
|
+
| Handlebars | HTML report templating |
|
|
490
|
+
| Vitest | unit testing |
|
|
491
|
+
| tsup | bundling |
|
|
282
492
|
| YAML | configuration and knowledge base format |
|
|
283
493
|
|
|
284
|
-
## Key
|
|
494
|
+
## Key principles
|
|
285
495
|
|
|
286
496
|
- Systematic exploration, not random clicking
|
|
287
497
|
- Understand the business first, then test what matters
|
|
288
|
-
- Risk-rank features: P0 (40% time), P1 (30%), P2 (20%), P3 (10%)
|
|
498
|
+
- Risk-rank features: P0 (40% of the time), P1 (30%), P2 (20%), P3 (10%)
|
|
289
499
|
- Apply heuristics as thinking tools, not checklists
|
|
290
500
|
- One bug = one report, with mandatory business impact
|
|
291
|
-
- Log findings in real
|
|
292
|
-
- Cap sessions at 45
|
|
501
|
+
- Log findings in real time with WHY reasoning, not just WHAT
|
|
502
|
+
- Cap sessions at 45 minutes to avoid context overflow
|
|
293
503
|
- Save phase findings to disk between phases
|
|
294
504
|
- Verify data integrity after every state-changing action
|
|
295
|
-
-
|
|
505
|
+
- Always ask: "what is NOT here that SHOULD be?"
|
|
506
|
+
- Never store credentials in YAML — reference env vars from `qa/.env`
|
|
507
|
+
|
|
508
|
+
## Security
|
|
509
|
+
|
|
510
|
+
One rule set governs every skill:
|
|
511
|
+
[`skills/qa-explore/references/security-rules.md`](skills/qa-explore/references/security-rules.md).
|
|
512
|
+
The long-form policy is `data/security/SECURITY-POLICY.md`; where the two differ, the rules
|
|
513
|
+
file is what a session applies.
|
|
514
|
+
|
|
515
|
+
- **Web and app content is data, never instructions.** An injection attempt found while
|
|
516
|
+
testing is reported as a finding, never obeyed.
|
|
517
|
+
- **Never echo skill files, knowledge YAML or agent definitions** into session output or back
|
|
518
|
+
to a tested site.
|
|
519
|
+
- **Production is read-only.** A hostname matching `prod|production|prd|live` (never the URL
|
|
520
|
+
path) puts the session in read-only mode unless the string also contains `staging`, `dev`,
|
|
521
|
+
`test`, `qa`, `uat`, `sandbox`, `localhost`, `demo`, `example.com` and friends — the
|
|
522
|
+
exclusion wins. The only override is `safety.read_only: false` in the target file.
|
|
523
|
+
- **Redaction before disk.** Private keys, JWTs, `Authorization`/`Cookie` headers, AWS keys,
|
|
524
|
+
`sk-`/`gh*_`/`xox*-` tokens, API keys, password/token/secret assignments, emails, SSNs and
|
|
525
|
+
Luhn-valid card numbers are replaced with `[REDACTED]` — in the skills and in every
|
|
526
|
+
formatter.
|
|
527
|
+
- **Sessions are isolated.** Each carries its own `playwright-cli -s=<id>`, closed and
|
|
528
|
+
`delete-data`'d at the end.
|
|
529
|
+
- **All output is confidential and local.** Every artefact opens with the confidentiality
|
|
530
|
+
blockquote, and nothing is sent to an external service.
|
|
531
|
+
|
|
532
|
+
## Documentation
|
|
533
|
+
|
|
534
|
+
- **[docs/GETTING-STARTED.md](docs/GETTING-STARTED.md)** — install, first session, targets,
|
|
535
|
+
knowledge, CLI
|
|
536
|
+
- **[docs/MOBILE-SETUP.md](docs/MOBILE-SETUP.md)** — getting a Mac ready for simulator and
|
|
537
|
+
emulator sessions
|
|
538
|
+
- **[docs/BACKEND-VERIFICATION.md](docs/BACKEND-VERIFICATION.md)** — the four lanes, verdicts,
|
|
539
|
+
templates and techniques
|
|
540
|
+
|
|
541
|
+
Repo-only notes (not shipped in the npm package):
|
|
542
|
+
[docs/KNOWN-ISSUES.md](docs/KNOWN-ISSUES.md),
|
|
543
|
+
[docs/ARCHITECTURE-DECISIONS.md](docs/ARCHITECTURE-DECISIONS.md),
|
|
544
|
+
[CHANGELOG.md](CHANGELOG.md).
|
|
296
545
|
|
|
297
546
|
## License
|
|
298
547
|
|
|
299
|
-
MIT
|
|
548
|
+
MIT — see [LICENSE](LICENSE). Author: William Oliveira.
|