qualiow-exploratory-testing 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +214 -0
- package/data/domains/_default.yml +167 -0
- package/data/domains/ecommerce.yml +198 -0
- package/data/domains/fintech.yml +207 -0
- package/data/domains/marketing.yml +202 -0
- package/data/domains/saas.yml +206 -0
- package/data/knowledge/changelog.yml +75 -0
- package/data/knowledge/custom/.gitkeep +0 -0
- package/data/knowledge/learned-patterns.md +40 -0
- package/data/knowledge/manifest.yml +141 -0
- package/data/knowledge/releases/v0.1.0/entries/checklist-accessibility-wcag.yml +326 -0
- package/data/knowledge/releases/v0.1.0/entries/heuristic-few-hiccupps.yml +194 -0
- package/data/knowledge/releases/v0.1.0/entries/heuristic-goldilocks.yml +143 -0
- package/data/knowledge/releases/v0.1.0/entries/heuristic-rcrcrc.yml +207 -0
- package/data/knowledge/releases/v0.1.0/entries/heuristic-sfdipot.yml +171 -0
- package/data/knowledge/releases/v0.1.0/entries/heuristic-test-tours.yml +253 -0
- package/data/knowledge/releases/v0.1.0/entries/reference-exploratory-testing-types.yml +84 -0
- package/data/knowledge/releases/v0.1.0/entries/reference-heuristic-praxis.yml +95 -0
- package/data/knowledge/releases/v0.1.0/entries/technique-boundary-testing.yml +203 -0
- package/data/knowledge/releases/v0.1.0/entries/technique-business-logic-race-conditions.yml +146 -0
- package/data/knowledge/releases/v0.1.0/entries/technique-data-integrity-testing.yml +269 -0
- package/data/knowledge/releases/v0.1.0/entries/technique-error-guessing.yml +216 -0
- package/data/knowledge/releases/v0.1.0/entries/technique-risk-based-testing.yml +298 -0
- package/data/knowledge/releases/v0.1.0/release.yml +22 -0
- package/data/targets/_default.yml +11 -0
- package/data/targets/testers-ai.yml +14 -0
- package/data/templates/bug-report.md +178 -0
- package/data/templates/coverage-map.md +70 -0
- package/data/templates/session-charter.md +77 -0
- package/data/templates/session-report.md +112 -0
- package/dist/cli/index.d.ts +1 -0
- package/dist/cli/index.js +1757 -0
- package/dist/cli/index.js.map +1 -0
- package/dist/index.d.ts +899 -0
- package/dist/index.js +1133 -0
- package/dist/index.js.map +1 -0
- package/docs/GETTING-STARTED.md +265 -0
- package/package.json +70 -0
- package/skills/qa-explore/SKILL.md +70 -0
- package/skills/qa-explore/phases/00-setup.md +96 -0
- package/skills/qa-explore/phases/01-auth.md +34 -0
- package/skills/qa-explore/phases/02-charter.md +112 -0
- package/skills/qa-explore/phases/03-discovery.md +49 -0
- package/skills/qa-explore/phases/04-journeys.md +51 -0
- package/skills/qa-explore/phases/05-features.md +55 -0
- package/skills/qa-explore/phases/06-edge-cases.md +69 -0
- package/skills/qa-explore/phases/07-reporting.md +162 -0
- package/skills/qa-explore/references/context-integration.md +44 -0
- package/skills/qa-explore/references/security-rules.md +43 -0
- package/skills/qa-explore/references/session-rules.md +33 -0
- package/skills/qa-explore/references/severity-guide.md +14 -0
- package/skills/qa-explore-cleanup/SKILL.md +39 -0
- package/skills/qa-explore-feedback/SKILL.md +73 -0
- package/skills/qa-explore-quick/SKILL.md +80 -0
- package/skills/qa-explore-report/SKILL.md +62 -0
- package/skills/qa-gather/SKILL.md +198 -0
- package/skills/qa-knowledge-add/SKILL.md +91 -0
- package/skills/qa-knowledge-list/SKILL.md +54 -0
- package/skills/qa-target-setup/SKILL.md +106 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 William Oliveira
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,214 @@
|
|
|
1
|
+
# @qualiow/exploratory-testing
|
|
2
|
+
|
|
3
|
+
qualiow -- AI-powered exploratory testing skills using Claude Code + Playwright CLI.
|
|
4
|
+
|
|
5
|
+
Claude acts as a Principal QA Engineer: it explores web applications systematically, finds bugs, and generates structured reports. No test scripts, no framework -- just AI reasoning, browser commands, and QA heuristics.
|
|
6
|
+
|
|
7
|
+
## How to Install This Project
|
|
8
|
+
|
|
9
|
+
### Prerequisites
|
|
10
|
+
|
|
11
|
+
- Node.js >= 18
|
|
12
|
+
- Playwright CLI (`npm install @playwright/cli`)
|
|
13
|
+
- Claude Code (for running exploratory sessions)
|
|
14
|
+
|
|
15
|
+
### Quick Start
|
|
16
|
+
|
|
17
|
+
```bash
|
|
18
|
+
# install globally
|
|
19
|
+
npm install -g @qualiow/exploratory-testing
|
|
20
|
+
|
|
21
|
+
# initialize in your project
|
|
22
|
+
qualiow init
|
|
23
|
+
|
|
24
|
+
# run your first session
|
|
25
|
+
/qa-explore https://your-app.com
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
### Local Development
|
|
29
|
+
|
|
30
|
+
```bash
|
|
31
|
+
# clone the repository
|
|
32
|
+
git clone https://github.com/willcoliveira/qualiow-exploratory-testing-skills.git
|
|
33
|
+
cd qualiow-exploratory-testing-skills
|
|
34
|
+
|
|
35
|
+
# install dependencies
|
|
36
|
+
npm install
|
|
37
|
+
|
|
38
|
+
# build
|
|
39
|
+
npm run build
|
|
40
|
+
|
|
41
|
+
# run tests
|
|
42
|
+
npm test
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
## How to Run the CLI
|
|
46
|
+
|
|
47
|
+
| Command | Description |
|
|
48
|
+
|---------|-------------|
|
|
49
|
+
| `qualiow init` | Initialize qualiow in the current project |
|
|
50
|
+
| `qualiow explore [url]` | Run an exploratory testing session |
|
|
51
|
+
| `qualiow validate` | Validate all configuration files (targets, knowledge, domains) |
|
|
52
|
+
| `qualiow report` | Generate a report from an existing session |
|
|
53
|
+
| `qualiow list <type>` | List sessions, knowledge, targets, or domains |
|
|
54
|
+
| `qualiow gather` | Gather and analyze requirements for testing context |
|
|
55
|
+
|
|
56
|
+
## How to Run Exploratory Sessions
|
|
57
|
+
|
|
58
|
+
### Full session (45 minutes)
|
|
59
|
+
|
|
60
|
+
```bash
|
|
61
|
+
# explore a public site by URL
|
|
62
|
+
/qa-explore https://testers.ai/testing/
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
```bash
|
|
66
|
+
# explore using a saved target configuration
|
|
67
|
+
/qa-explore --target company-staging
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
### Quick focused session (15 minutes)
|
|
71
|
+
|
|
72
|
+
```bash
|
|
73
|
+
# check a single page or feature
|
|
74
|
+
/qa-explore-quick https://app.example.com/checkout
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
### Generate or regenerate a report
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
# regenerate report for an existing session
|
|
81
|
+
/qa-explore-report output/sessions/2026-03-28-parabank
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
### Add knowledge to the base
|
|
85
|
+
|
|
86
|
+
```bash
|
|
87
|
+
# add new heuristics, techniques, or documentation
|
|
88
|
+
/qa-knowledge-add
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
### Set up a new target
|
|
92
|
+
|
|
93
|
+
```bash
|
|
94
|
+
# configure authentication, scope, and domain for a target app
|
|
95
|
+
/qa-target-setup
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
## How to Generate Reports
|
|
99
|
+
|
|
100
|
+
qualiow produces three report formats from session data.
|
|
101
|
+
|
|
102
|
+
### HTML report
|
|
103
|
+
|
|
104
|
+
Self-contained dark-mode HTML with severity cards, bug details, coverage map, and recommendations. Print-friendly.
|
|
105
|
+
|
|
106
|
+
```bash
|
|
107
|
+
# from TypeScript
|
|
108
|
+
import { generateHtmlReport } from '@qualiow/exploratory-testing';
|
|
109
|
+
const html = await generateHtmlReport('output/sessions/2026-03-28-parabank');
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
### JSON report
|
|
113
|
+
|
|
114
|
+
Structured JSON for programmatic consumption -- dashboards, CI integrations, metrics tracking.
|
|
115
|
+
|
|
116
|
+
```bash
|
|
117
|
+
# from TypeScript
|
|
118
|
+
import { generateJsonReport } from '@qualiow/exploratory-testing';
|
|
119
|
+
const data = await generateJsonReport('output/sessions/2026-03-28-parabank');
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
### Jira CSV export
|
|
123
|
+
|
|
124
|
+
CSV importable via Jira bulk import. Maps severity to Jira priority (Critical -> Blocker, High -> Critical, Medium -> Major, Low -> Minor).
|
|
125
|
+
|
|
126
|
+
```bash
|
|
127
|
+
# from TypeScript
|
|
128
|
+
import { generateJiraExport } from '@qualiow/exploratory-testing';
|
|
129
|
+
const csv = await generateJiraExport('output/sessions/2026-03-28-parabank');
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
## How to Use Skills in Claude Code
|
|
133
|
+
|
|
134
|
+
| Skill | Purpose |
|
|
135
|
+
|-------|---------|
|
|
136
|
+
| `/qa-explore` | Full exploratory testing session (45 min) |
|
|
137
|
+
| `/qa-explore-quick` | Quick focused session on a single page or feature (15 min) |
|
|
138
|
+
| `/qa-explore-report` | Generate or regenerate report from existing session |
|
|
139
|
+
| `/qa-explore-feedback` | Post-session feedback capture (false positives, missed bugs) |
|
|
140
|
+
| `/qa-explore-cleanup` | Session cleanup and archival |
|
|
141
|
+
| `/qa-knowledge-add` | Add new heuristics, techniques, docs to knowledge base |
|
|
142
|
+
| `/qa-knowledge-list` | Browse the knowledge base |
|
|
143
|
+
| `/qa-target-setup` | Configure a new target application (auth, scope, domain) |
|
|
144
|
+
| `/qa-gather` | Gather and analyze requirements from files, URLs, or text |
|
|
145
|
+
|
|
146
|
+
## Project Structure
|
|
147
|
+
|
|
148
|
+
```
|
|
149
|
+
qualiow-exploratory-testing-skills/
|
|
150
|
+
src/
|
|
151
|
+
cli/
|
|
152
|
+
index.ts # CLI entry point (commander)
|
|
153
|
+
commands/ # CLI command handlers
|
|
154
|
+
formatters/
|
|
155
|
+
html-report.ts # standalone dark-mode HTML report
|
|
156
|
+
json-report.ts # structured JSON report
|
|
157
|
+
jira-export.ts # Jira-importable CSV
|
|
158
|
+
index.ts # re-exports
|
|
159
|
+
schemas/ # Zod validation schemas
|
|
160
|
+
types/ # TypeScript type definitions
|
|
161
|
+
utils/
|
|
162
|
+
metrics.ts # session metrics (JSONL append/read)
|
|
163
|
+
parse-session.ts # parse session dirs into structured data
|
|
164
|
+
redact.ts # credential and secret redaction
|
|
165
|
+
validate.ts # YAML config validation
|
|
166
|
+
index.ts # public API re-exports
|
|
167
|
+
data/
|
|
168
|
+
knowledge/ # YAML knowledge base (heuristics, techniques)
|
|
169
|
+
templates/ # bug report, session report, charter templates
|
|
170
|
+
targets/ # target application configs
|
|
171
|
+
domains/ # domain-specific testing focus
|
|
172
|
+
security/ # security policy
|
|
173
|
+
output/
|
|
174
|
+
sessions/ # session outputs (reports, bugs, screenshots)
|
|
175
|
+
tests/
|
|
176
|
+
unit/ # unit tests (vitest)
|
|
177
|
+
fixtures/ # test fixture data
|
|
178
|
+
benchmarks/ # performance benchmarks
|
|
179
|
+
docs/
|
|
180
|
+
GETTING-STARTED.md # detailed setup and usage guide
|
|
181
|
+
ARCHITECTURE-DECISIONS.md # architecture decision records
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
## Tech Stack
|
|
185
|
+
|
|
186
|
+
| Technology | Purpose |
|
|
187
|
+
|------------|---------|
|
|
188
|
+
| TypeScript | type-safe source code |
|
|
189
|
+
| Node.js >= 18 | runtime |
|
|
190
|
+
| Playwright CLI | browser control for AI agents |
|
|
191
|
+
| Claude Code | AI reasoning and session orchestration |
|
|
192
|
+
| Handlebars | HTML report templating |
|
|
193
|
+
| Zod | runtime schema validation |
|
|
194
|
+
| Commander | CLI argument parsing |
|
|
195
|
+
| Vitest | unit and integration testing |
|
|
196
|
+
| tsup | TypeScript bundling |
|
|
197
|
+
| YAML | configuration and knowledge base format |
|
|
198
|
+
|
|
199
|
+
## Key Principles
|
|
200
|
+
|
|
201
|
+
- Systematic exploration, not random clicking
|
|
202
|
+
- Understand the business first, then test what matters
|
|
203
|
+
- Risk-rank features: P0 (40% time), P1 (30%), P2 (20%), P3 (10%)
|
|
204
|
+
- Apply heuristics as thinking tools, not checklists
|
|
205
|
+
- One bug = one report, with mandatory business impact
|
|
206
|
+
- Log findings in real-time with WHY reasoning, not just WHAT
|
|
207
|
+
- Cap sessions at 45 min to avoid context overflow
|
|
208
|
+
- Save phase findings to disk between phases
|
|
209
|
+
- Verify data integrity after every state-changing action
|
|
210
|
+
- Never store credentials in YAML -- use env vars via .env
|
|
211
|
+
|
|
212
|
+
## License
|
|
213
|
+
|
|
214
|
+
MIT -- see [LICENSE](LICENSE) for details.
|
|
@@ -0,0 +1,167 @@
|
|
|
1
|
+
id: default
|
|
2
|
+
name: "Generic Web Application"
|
|
3
|
+
|
|
4
|
+
risk_ranking:
|
|
5
|
+
p0: ["authentication & authorization", "core form submissions", "payment/revenue flows"]
|
|
6
|
+
p1: ["primary navigation", "user registration", "data creation/modification"]
|
|
7
|
+
p2: ["search & filtering", "settings", "responsive design", "error handling"]
|
|
8
|
+
p3: ["about page", "help docs", "static content", "SEO meta tags"]
|
|
9
|
+
|
|
10
|
+
completeness_checklist:
|
|
11
|
+
- "Logo links to homepage"
|
|
12
|
+
- "Primary navigation is present and functional on all pages"
|
|
13
|
+
- "Footer is present with expected links (privacy, terms, contact)"
|
|
14
|
+
- "Breadcrumbs reflect the current location (if applicable)"
|
|
15
|
+
- "404 page exists and is user-friendly (not a raw server error)"
|
|
16
|
+
- "500 error page exists and is user-friendly"
|
|
17
|
+
- "Registration flow completes end to end"
|
|
18
|
+
- "Login flow works with valid credentials"
|
|
19
|
+
- "Logout clears session and redirects appropriately"
|
|
20
|
+
- "Password reset flow sends email and allows reset"
|
|
21
|
+
- "Protected pages redirect to login when unauthenticated"
|
|
22
|
+
- "Empty list/table shows a helpful message (not a blank page)"
|
|
23
|
+
- "Empty search results suggest alternatives or show a clear message"
|
|
24
|
+
- "New user with no data sees onboarding or guidance, not emptiness"
|
|
25
|
+
- "No placeholder text (lorem ipsum, TODO, TBD)"
|
|
26
|
+
- "No broken images or missing media"
|
|
27
|
+
- "Copyright year is current"
|
|
28
|
+
- "Legal pages are linked and accessible (privacy policy, terms of service)"
|
|
29
|
+
- "All forms submit and provide feedback (success or error)"
|
|
30
|
+
- "All buttons and links lead somewhere (no dead ends)"
|
|
31
|
+
- "Loading states exist for async operations (no frozen UI)"
|
|
32
|
+
- "Back button behavior is consistent and does not break state"
|
|
33
|
+
|
|
34
|
+
data_integrity_checks:
|
|
35
|
+
- "Persistence -- Reload the page. Does the change survive? Is the data still correct?"
|
|
36
|
+
- "Reflection -- Does the change appear everywhere it should? (List views, detail views, summaries, exports.)"
|
|
37
|
+
- "Consistency -- Do calculated values (totals, counts, timestamps) update correctly?"
|
|
38
|
+
- "Reversibility -- If the action can be undone (delete, cancel), does the system return to a clean state?"
|
|
39
|
+
- "Concurrency -- If two users (or two tabs) act on the same data, does the system handle it without silent data loss?"
|
|
40
|
+
- "Cascade -- If the changed item is referenced elsewhere (e.g., a category that contains products), are related items updated or handled?"
|
|
41
|
+
|
|
42
|
+
journeys:
|
|
43
|
+
- name: "New User Onboarding"
|
|
44
|
+
steps:
|
|
45
|
+
- "Register"
|
|
46
|
+
- "Verify Email"
|
|
47
|
+
- "Login"
|
|
48
|
+
- "Complete Profile"
|
|
49
|
+
- "Use Primary Feature"
|
|
50
|
+
- "Logout"
|
|
51
|
+
- "Verify: session persists correctly through each step, data entered in earlier steps is visible later, logout clears everything"
|
|
52
|
+
- name: "Content Lifecycle"
|
|
53
|
+
steps:
|
|
54
|
+
- "Create Item"
|
|
55
|
+
- "View in List"
|
|
56
|
+
- "Edit Item"
|
|
57
|
+
- "Verify Changes"
|
|
58
|
+
- "Delete Item"
|
|
59
|
+
- "Verify Removal"
|
|
60
|
+
- "Verify: item appears in all relevant views after creation, edits persist after reload, deletion removes item from all views"
|
|
61
|
+
- name: "Error Recovery"
|
|
62
|
+
steps:
|
|
63
|
+
- "Start Action"
|
|
64
|
+
- "Force Error (disconnect network, submit invalid data)"
|
|
65
|
+
- "See Error State"
|
|
66
|
+
- "Retry"
|
|
67
|
+
- "Complete Successfully"
|
|
68
|
+
- "Verify: error messages are clear, no data is lost or duplicated on retry, system returns to a consistent state"
|
|
69
|
+
|
|
70
|
+
must_test_patterns:
|
|
71
|
+
forms:
|
|
72
|
+
- "Required field validation (submit empty, partial, complete)"
|
|
73
|
+
- "Input boundary values (min/max length, special characters, Unicode)"
|
|
74
|
+
- "Email and URL format validation"
|
|
75
|
+
- "Date and number field constraints"
|
|
76
|
+
- "Form submission with network interruption"
|
|
77
|
+
- "Double-submit prevention"
|
|
78
|
+
- "Tab order through form fields"
|
|
79
|
+
- "Error message clarity and placement"
|
|
80
|
+
navigation:
|
|
81
|
+
- "All internal links resolve (no 404s)"
|
|
82
|
+
- "Browser back/forward behavior is consistent"
|
|
83
|
+
- "Deep linking and bookmarking work"
|
|
84
|
+
- "Breadcrumbs reflect actual location"
|
|
85
|
+
- "Menu states (open, close, hover, focus)"
|
|
86
|
+
- "Navigation on mobile (hamburger menu, touch targets)"
|
|
87
|
+
authentication:
|
|
88
|
+
- "Login with valid and invalid credentials"
|
|
89
|
+
- "Password reset flow end to end"
|
|
90
|
+
- "Session timeout behavior"
|
|
91
|
+
- "Concurrent session handling"
|
|
92
|
+
- "Remember me functionality"
|
|
93
|
+
- "Logout clears session data"
|
|
94
|
+
- "Protected pages redirect to login"
|
|
95
|
+
error_handling:
|
|
96
|
+
- "404 page for invalid URLs"
|
|
97
|
+
- "Server error pages (500) are user-friendly"
|
|
98
|
+
- "Network timeout behavior"
|
|
99
|
+
- "Invalid input error messages are helpful"
|
|
100
|
+
- "Form validation errors do not lose user input"
|
|
101
|
+
- "API failure graceful degradation"
|
|
102
|
+
responsive:
|
|
103
|
+
- "Layout at 320px, 768px, 1024px, 1440px widths"
|
|
104
|
+
- "Images scale without distortion"
|
|
105
|
+
- "Text remains readable at all sizes"
|
|
106
|
+
- "Touch targets are at least 44x44px on mobile"
|
|
107
|
+
- "No horizontal scrolling on mobile"
|
|
108
|
+
- "Modals and popups work on small screens"
|
|
109
|
+
accessibility:
|
|
110
|
+
- "All images have alt text"
|
|
111
|
+
- "Form fields have associated labels"
|
|
112
|
+
- "Focus indicators are visible"
|
|
113
|
+
- "Color is not the only indicator of state"
|
|
114
|
+
- "Page has a logical heading hierarchy (h1-h6)"
|
|
115
|
+
- "Skip navigation link is present"
|
|
116
|
+
- "ARIA roles are used correctly where needed"
|
|
117
|
+
performance:
|
|
118
|
+
- "Initial page load under 3 seconds"
|
|
119
|
+
- "Time to interactive is reasonable"
|
|
120
|
+
- "Large images are optimized"
|
|
121
|
+
- "No excessive DOM size"
|
|
122
|
+
- "No memory leaks on repeated interactions"
|
|
123
|
+
- "Lazy loading for below-the-fold content"
|
|
124
|
+
seo:
|
|
125
|
+
- "Each page has a unique title tag"
|
|
126
|
+
- "Meta descriptions are present and descriptive"
|
|
127
|
+
- "Heading hierarchy is semantic"
|
|
128
|
+
- "Canonical URLs are set"
|
|
129
|
+
- "Robots.txt and sitemap.xml are accessible"
|
|
130
|
+
- "Open Graph and Twitter Card meta tags present"
|
|
131
|
+
|
|
132
|
+
common_bugs:
|
|
133
|
+
- "Broken links after content updates"
|
|
134
|
+
- "Form submission succeeds silently without confirmation"
|
|
135
|
+
- "Session expires during long form entry, losing data"
|
|
136
|
+
- "Mobile menu overlaps content"
|
|
137
|
+
- "Inconsistent error message styles"
|
|
138
|
+
- "Missing loading states causing double clicks"
|
|
139
|
+
- "Stale cache showing outdated content"
|
|
140
|
+
- "Z-index issues with overlapping modals and dropdowns"
|
|
141
|
+
- "Autofill conflicts with custom input components"
|
|
142
|
+
- "Scroll position not restored on back navigation"
|
|
143
|
+
|
|
144
|
+
compliance:
|
|
145
|
+
- "WCAG 2.1 Level AA (baseline accessibility)"
|
|
146
|
+
- "GDPR cookie consent if applicable"
|
|
147
|
+
- "HTTPS everywhere"
|
|
148
|
+
- "Privacy policy and terms of service links accessible"
|
|
149
|
+
|
|
150
|
+
guidance: |
|
|
151
|
+
This is the default domain config for generic web applications. Use it when
|
|
152
|
+
no specific domain (ecommerce, fintech, saas, marketing) applies.
|
|
153
|
+
|
|
154
|
+
Priority areas in order:
|
|
155
|
+
1. Core Functionality -- Forms, navigation, and primary user flows work correctly
|
|
156
|
+
2. Authentication & Authorization -- Login, logout, session management, protected routes
|
|
157
|
+
3. Error Handling -- Graceful failures, meaningful error messages, recovery paths
|
|
158
|
+
4. Responsive Design -- Layout integrity across viewport sizes
|
|
159
|
+
5. Accessibility -- Keyboard navigation, screen reader basics, color contrast
|
|
160
|
+
6. Performance -- Page load times, interaction responsiveness
|
|
161
|
+
7. SEO Fundamentals -- Meta tags, heading structure, crawlability
|
|
162
|
+
|
|
163
|
+
Feature risk ranking guide:
|
|
164
|
+
- P0: "If this feature is broken, does the business lose money or get sued?" If yes, P0.
|
|
165
|
+
- P1: "If this feature is broken, do users fail their primary task?" If yes, P1.
|
|
166
|
+
- P2: "If this feature is broken, do users notice and complain?" If yes, P2.
|
|
167
|
+
- P3: Everything else.
|
|
@@ -0,0 +1,198 @@
|
|
|
1
|
+
id: ecommerce
|
|
2
|
+
name: "E-Commerce"
|
|
3
|
+
|
|
4
|
+
risk_ranking:
|
|
5
|
+
p0: ["checkout", "payment processing", "cart", "pricing/totals"]
|
|
6
|
+
p1: ["product catalog", "search", "authentication"]
|
|
7
|
+
p2: ["reviews & ratings", "wishlist", "filters & sort"]
|
|
8
|
+
p3: ["about page", "contact page", "social media links"]
|
|
9
|
+
|
|
10
|
+
completeness_checklist:
|
|
11
|
+
- "Cart displays items with name, image, price, and quantity"
|
|
12
|
+
- "Cart quantity can be increased, decreased, and item removed"
|
|
13
|
+
- "Empty cart shows a helpful message and a link to continue shopping"
|
|
14
|
+
- "Delivery/shipping address form with validation"
|
|
15
|
+
- "Payment method selector (at least one method functional)"
|
|
16
|
+
- "Order summary visible before final confirmation with itemized totals"
|
|
17
|
+
- "Order confirmation page with a unique order ID"
|
|
18
|
+
- "Confirmation email or receipt sent after purchase"
|
|
19
|
+
- "Order history accessible from user account"
|
|
20
|
+
- "Guest checkout option (or clear account creation prompt)"
|
|
21
|
+
|
|
22
|
+
data_integrity_checks:
|
|
23
|
+
- "Item price x quantity = line total -- Check for every line item in the cart and in the order confirmation. Watch for rounding issues with discounts."
|
|
24
|
+
- "All line totals + shipping + tax - discounts = order total -- Verify on the cart page, checkout summary, confirmation page, and confirmation email. All four should match."
|
|
25
|
+
- "Inventory decremented after purchase -- If stock count is visible, verify it decreases by the purchased quantity. Verify out-of-stock messaging appears when inventory hits zero."
|
|
26
|
+
- "Order appears in history -- After completing checkout, the order should immediately appear in the user's order history with correct items, quantities, totals, and status."
|
|
27
|
+
- "Coupon/discount applied correctly -- Discount shows as a line item, subtotal adjusts, and the discount does not apply to items it should not (e.g., excluded categories)."
|
|
28
|
+
- "Price consistency -- The price on the product page, in the cart, at checkout, and on the confirmation must all match (accounting for any discounts applied)."
|
|
29
|
+
|
|
30
|
+
journeys:
|
|
31
|
+
- name: "Browse to Purchase"
|
|
32
|
+
steps:
|
|
33
|
+
- "Browse Catalog"
|
|
34
|
+
- "Add to Cart"
|
|
35
|
+
- "Proceed to Checkout"
|
|
36
|
+
- "Enter Shipping"
|
|
37
|
+
- "Select Payment"
|
|
38
|
+
- "Confirm Order"
|
|
39
|
+
- "View Order History"
|
|
40
|
+
- "Verify: item details persist from catalog through confirmation, totals are correct at every step, order appears in history with correct status"
|
|
41
|
+
- name: "Search to Purchase"
|
|
42
|
+
steps:
|
|
43
|
+
- "Search for Product"
|
|
44
|
+
- "Apply Filters"
|
|
45
|
+
- "Open Product Detail"
|
|
46
|
+
- "Select Variant"
|
|
47
|
+
- "Add to Cart"
|
|
48
|
+
- "Checkout"
|
|
49
|
+
- "Verify: search results match filters, selected variant (size/color) carries into cart correctly, price reflects variant choice"
|
|
50
|
+
- name: "Full Account Lifecycle"
|
|
51
|
+
steps:
|
|
52
|
+
- "Register"
|
|
53
|
+
- "Login"
|
|
54
|
+
- "Browse"
|
|
55
|
+
- "Add to Cart"
|
|
56
|
+
- "Purchase"
|
|
57
|
+
- "Logout"
|
|
58
|
+
- "Login Again"
|
|
59
|
+
- "View Order History"
|
|
60
|
+
- "Verify: registration completes, cart persists across login, order history is accessible after re-login, all data is consistent"
|
|
61
|
+
|
|
62
|
+
must_test_patterns:
|
|
63
|
+
product_catalog:
|
|
64
|
+
- "Search returns relevant results for exact and partial matches"
|
|
65
|
+
- "Filters narrow results correctly (price range, category, size, color)"
|
|
66
|
+
- "Sort options work (price low-high, high-low, newest, rating)"
|
|
67
|
+
- "Product images load and zoom functions work"
|
|
68
|
+
- "Product variants (size, color) update price and availability"
|
|
69
|
+
- "Out-of-stock items display clear messaging"
|
|
70
|
+
- "Pagination or infinite scroll loads correctly"
|
|
71
|
+
- "Category navigation reflects accurate product counts"
|
|
72
|
+
- "Recently viewed and related products display correctly"
|
|
73
|
+
search:
|
|
74
|
+
- "Exact product name search"
|
|
75
|
+
- "Misspelled search terms (fuzzy matching)"
|
|
76
|
+
- "Search with special characters"
|
|
77
|
+
- "Empty search submission"
|
|
78
|
+
- "Search suggestions and autocomplete"
|
|
79
|
+
- "No results page with helpful alternatives"
|
|
80
|
+
- "Search within category or filtered results"
|
|
81
|
+
cart:
|
|
82
|
+
- "Add single and multiple items"
|
|
83
|
+
- "Update quantity (increase, decrease, zero, negative, decimal)"
|
|
84
|
+
- "Remove items from cart"
|
|
85
|
+
- "Cart persists across sessions (logged in)"
|
|
86
|
+
- "Cart persists across sessions (guest via cookie)"
|
|
87
|
+
- "Cart reflects current prices (not stale cached prices)"
|
|
88
|
+
- "Cart handles out-of-stock items added before stock depletion"
|
|
89
|
+
- "Maximum quantity limits enforced"
|
|
90
|
+
- "Cart total updates in real time"
|
|
91
|
+
- "Empty cart state displays correctly"
|
|
92
|
+
- "Cart icon badge count updates immediately"
|
|
93
|
+
checkout:
|
|
94
|
+
- "Guest checkout flow end to end"
|
|
95
|
+
- "Registered user checkout with saved details"
|
|
96
|
+
- "Address validation and autocomplete"
|
|
97
|
+
- "Shipping method selection updates total"
|
|
98
|
+
- "Order summary accuracy before payment"
|
|
99
|
+
- "Back button behavior during checkout"
|
|
100
|
+
- "Checkout with expired session"
|
|
101
|
+
- "Checkout with empty cart (direct URL access)"
|
|
102
|
+
- "Order confirmation page with correct details"
|
|
103
|
+
- "Confirmation email sent with correct content"
|
|
104
|
+
payments:
|
|
105
|
+
- "Credit/debit card processing (valid card)"
|
|
106
|
+
- "Credit card with invalid number, expired date, wrong CVV"
|
|
107
|
+
- "PayPal or third-party payment redirect and return"
|
|
108
|
+
- "Payment failure and retry flow"
|
|
109
|
+
- "Insufficient funds handling"
|
|
110
|
+
- "3D Secure authentication flow"
|
|
111
|
+
- "Saved payment methods selection"
|
|
112
|
+
- "Payment timeout handling (gateway slow response)"
|
|
113
|
+
- "Refund to original payment method"
|
|
114
|
+
- "Currency display matches user locale"
|
|
115
|
+
shipping:
|
|
116
|
+
- "Shipping rate calculation by weight, destination, method"
|
|
117
|
+
- "Free shipping threshold display and application"
|
|
118
|
+
- "Multiple shipping addresses (split shipment)"
|
|
119
|
+
- "International shipping with customs information"
|
|
120
|
+
- "Estimated delivery date accuracy"
|
|
121
|
+
- "Shipping to PO boxes and military addresses"
|
|
122
|
+
- "Express vs standard shipping option differences"
|
|
123
|
+
- "Shipping cost updates when address changes"
|
|
124
|
+
returns:
|
|
125
|
+
- "Return request initiation within policy window"
|
|
126
|
+
- "Return request outside policy window (should be denied)"
|
|
127
|
+
- "Return label generation"
|
|
128
|
+
- "Refund processing and confirmation"
|
|
129
|
+
- "Partial returns from multi-item orders"
|
|
130
|
+
- "Exchange flow for different size or color"
|
|
131
|
+
- "Return status tracking"
|
|
132
|
+
pricing:
|
|
133
|
+
- "Regular price display"
|
|
134
|
+
- "Sale price with original price strikethrough"
|
|
135
|
+
- "Percentage and fixed amount discount application"
|
|
136
|
+
- "Coupon code entry (valid, invalid, expired, case sensitivity)"
|
|
137
|
+
- "Stacking rules for multiple discounts"
|
|
138
|
+
- "Tax calculation by jurisdiction"
|
|
139
|
+
- "Bundle pricing accuracy"
|
|
140
|
+
- "Bulk pricing tiers"
|
|
141
|
+
- "Currency conversion if multi-currency supported"
|
|
142
|
+
- "Price updates when variant is changed"
|
|
143
|
+
user_accounts:
|
|
144
|
+
- "Registration with email verification"
|
|
145
|
+
- "Social login (Google, Facebook, Apple)"
|
|
146
|
+
- "Profile update (name, email, password)"
|
|
147
|
+
- "Address book management (add, edit, delete, set default)"
|
|
148
|
+
- "Order history with status and details"
|
|
149
|
+
- "Reorder from previous order"
|
|
150
|
+
- "Wishlist add, remove, move to cart"
|
|
151
|
+
- "Account deletion and data removal"
|
|
152
|
+
reviews:
|
|
153
|
+
- "Submit a review with star rating and text"
|
|
154
|
+
- "Review with photo upload"
|
|
155
|
+
- "Review moderation (pending state)"
|
|
156
|
+
- "Sort reviews by date, rating, helpfulness"
|
|
157
|
+
- "Average rating calculation accuracy"
|
|
158
|
+
- "Review filtering by star count"
|
|
159
|
+
- "Verified purchase badge display"
|
|
160
|
+
security:
|
|
161
|
+
- "PCI DSS -- Payment card data handled securely, no card numbers in logs or URLs"
|
|
162
|
+
|
|
163
|
+
common_bugs:
|
|
164
|
+
- "Cart calculation errors -- Rounding issues with discounts, tax computed on wrong subtotal, quantity multiplier off by one"
|
|
165
|
+
- "Checkout flow breaks -- Back button causes duplicate order, session timeout loses cart, address validation rejects valid addresses"
|
|
166
|
+
- "Payment gateway timeouts -- Order created but payment status unknown, user charged but order not confirmed, retry creates duplicate charge"
|
|
167
|
+
- "Inventory sync issues -- Item shows in stock but fails at checkout, overselling due to race condition, stock not restored on cancelled order"
|
|
168
|
+
- "Coupon stacking exploits -- Applying multiple exclusive coupons, coupon applied after order total goes negative, expired coupon still works via direct URL"
|
|
169
|
+
- "Price display mismatches -- Cart shows different price than product page, currency rounding differs between pages, tax-inclusive vs exclusive confusion"
|
|
170
|
+
- "Search relevance issues -- Exact match not in top results, filters reset after pagination, zero results for valid product names"
|
|
171
|
+
- "Account edge cases -- Guest checkout email conflicts with existing account, social login does not merge with email account, order history missing after password reset"
|
|
172
|
+
- "Mobile checkout -- Payment form keyboard covers input, autofill breaks card number formatting, touch targets too small on shipping options"
|
|
173
|
+
|
|
174
|
+
compliance:
|
|
175
|
+
- "PCI DSS -- Payment card data handled securely, no card numbers in logs or URLs"
|
|
176
|
+
- "GDPR -- Cookie consent, data export, right to deletion, privacy policy"
|
|
177
|
+
- "Consumer protection -- Clear return policy, accurate product descriptions, transparent pricing"
|
|
178
|
+
- "Accessibility (WCAG 2.1 AA) -- Full checkout flow usable by keyboard and screen reader"
|
|
179
|
+
- "Tax compliance -- Correct tax rates by jurisdiction, tax displayed before final purchase"
|
|
180
|
+
- "Email compliance -- CAN-SPAM Act for marketing emails, transactional email opt-out separation"
|
|
181
|
+
|
|
182
|
+
guidance: |
|
|
183
|
+
E-commerce testing priority: the money trail. Follow every dollar from product page
|
|
184
|
+
through cart, checkout, payment, and confirmation. Every number must match at every step.
|
|
185
|
+
|
|
186
|
+
Priority areas in order:
|
|
187
|
+
1. Checkout Flow -- Cart to payment confirmation must be rock solid
|
|
188
|
+
2. Payment Processing -- All payment methods work, edge cases handled
|
|
189
|
+
3. Product Catalog -- Search, filter, sort, and display accuracy
|
|
190
|
+
4. Pricing & Calculations -- Discounts, coupons, tax, and totals are precise
|
|
191
|
+
5. Inventory Management -- Stock levels, availability, and oversell prevention
|
|
192
|
+
6. User Accounts -- Registration, profiles, order history, saved addresses
|
|
193
|
+
7. Shipping & Returns -- Rate calculation, tracking, return/refund flows
|
|
194
|
+
|
|
195
|
+
Key heuristic focus for e-commerce:
|
|
196
|
+
- SFDIPOT: Function (buy flow), Data (prices/inventory), Interfaces (payment gateway)
|
|
197
|
+
- Test Tours: Money Tour, Bad Neighborhood
|
|
198
|
+
- FEW HICCUPPS: focus on Users (would a shopper expect this?) and Claims (does it match the listing?)
|