qualiow-exploratory-testing 1.2.0 → 2.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (111) hide show
  1. package/.claude-plugin/plugin.json +22 -0
  2. package/.env.example +27 -0
  3. package/CHANGELOG.md +194 -0
  4. package/README.md +428 -179
  5. package/agents/qa-gather-agent.md +24 -0
  6. package/{scripts → bin}/doctor-mobile.sh +57 -18
  7. package/bin/mcli +15 -9
  8. package/bin/mobile-cli.mjs +574 -286
  9. package/{scripts → bin}/setup-mobile.sh +56 -19
  10. package/bin/wadb +10 -4
  11. package/bin/wk-ios +29 -17
  12. package/bin/wkeval.mjs +7 -1
  13. package/data/domains/_default.yml +1 -1
  14. package/data/domains/identity.yml +138 -0
  15. package/data/knowledge/changelog.yml +104 -0
  16. package/data/knowledge/learned-patterns.md +172 -86
  17. package/data/knowledge/manifest.yml +481 -70
  18. package/data/knowledge/releases/v0.1.0/entries/technique-business-logic-race-conditions.yml +1 -1
  19. package/data/knowledge/releases/v0.1.0/release.yml +8 -1
  20. package/data/knowledge/releases/v0.2.0/entries/technique-contract-narrowing.yml +150 -0
  21. package/data/knowledge/releases/v0.2.0/entries/technique-test-suite-audit.yml +116 -0
  22. package/data/knowledge/releases/v0.2.0/release.yml +20 -0
  23. package/data/knowledge/releases/v0.3.0/entries/technique-functional-diff-analysis.yml +119 -0
  24. package/data/knowledge/releases/v0.3.0/entries/technique-llm-output-verification.yml +102 -0
  25. package/data/knowledge/releases/v0.3.0/entries/technique-verification-mode-selection.yml +153 -0
  26. package/data/knowledge/releases/v0.3.0/release.yml +25 -0
  27. package/data/knowledge/releases/v0.4.0/entries/technique-authenticated-api-probing.yml +137 -0
  28. package/data/knowledge/releases/v0.4.0/entries/technique-environment-fingerprinting.yml +113 -0
  29. package/data/knowledge/releases/v0.4.0/entries/technique-silent-failure-audit.yml +86 -0
  30. package/data/knowledge/releases/v0.4.0/entries/technique-ui-api-differential.yml +97 -0
  31. package/data/knowledge/releases/v0.4.0/release.yml +28 -0
  32. package/data/knowledge/releases/v0.5.0/entries/technique-config-surface-verification.yml +77 -0
  33. package/data/knowledge/releases/v0.5.0/entries/technique-derived-value-verification.yml +81 -0
  34. package/data/knowledge/releases/v0.5.0/entries/technique-expected-behaviour-specification.yml +97 -0
  35. package/data/knowledge/releases/v0.5.0/entries/technique-presentation-integrity.yml +78 -0
  36. package/data/knowledge/releases/v0.5.0/entries/technique-release-readiness-verification.yml +110 -0
  37. package/data/knowledge/releases/v0.5.0/release.yml +33 -0
  38. package/data/knowledge/releases/v0.6.0/entries/technique-async-callback-contracts.yml +98 -0
  39. package/data/knowledge/releases/v0.6.0/entries/technique-exactly-once-verification.yml +116 -0
  40. package/data/knowledge/releases/v0.6.0/release.yml +30 -0
  41. package/data/security/SECURITY-POLICY.md +273 -0
  42. package/data/targets/_default.yml +1 -1
  43. package/data/targets/_example-api-only.yml +80 -0
  44. package/data/targets/_example-backend.yml +123 -0
  45. package/data/targets/_example-mobile-emulation.yml +2 -2
  46. package/data/targets/_example-native-mobile.yml +2 -2
  47. package/data/targets/_example-sim-android-chrome.yml +1 -1
  48. package/data/targets/_example-sim-ios-safari.yml +1 -1
  49. package/data/templates/api-probe-matrix.md +99 -0
  50. package/data/templates/bug-report.md +29 -151
  51. package/data/templates/coverage-map.md +23 -6
  52. package/data/templates/expected-behaviour.md +94 -0
  53. package/data/templates/session-charter.md +3 -0
  54. package/data/templates/session-report.md +57 -80
  55. package/dist/cli/index.js +1500 -739
  56. package/dist/index.d.ts +673 -75
  57. package/dist/index.js +1664 -270
  58. package/docs/BACKEND-VERIFICATION.md +307 -0
  59. package/docs/GETTING-STARTED.md +188 -128
  60. package/docs/MOBILE-SETUP.md +149 -18
  61. package/package.json +32 -16
  62. package/skills/qa-explore/SKILL.md +31 -25
  63. package/skills/qa-explore/phases/00-setup.md +67 -41
  64. package/skills/qa-explore/phases/01-auth.md +32 -15
  65. package/skills/qa-explore/phases/02-charter.md +16 -5
  66. package/skills/qa-explore/phases/03-discovery.md +50 -26
  67. package/skills/qa-explore/phases/04-journeys.md +2 -2
  68. package/skills/qa-explore/phases/05-features.md +39 -24
  69. package/skills/qa-explore/phases/06-edge-cases.md +24 -13
  70. package/skills/qa-explore/phases/07-reporting.md +79 -140
  71. package/skills/qa-explore/references/context-integration.md +1 -1
  72. package/skills/qa-explore/references/output-contract.md +187 -0
  73. package/skills/qa-explore/references/paths.md +84 -0
  74. package/skills/qa-explore/references/playwright-agents-integration.md +61 -84
  75. package/skills/qa-explore/references/security-rules.md +66 -25
  76. package/skills/qa-explore/references/severity-guide.md +31 -4
  77. package/skills/qa-explore-cleanup/SKILL.md +17 -6
  78. package/skills/qa-explore-feedback/SKILL.md +18 -5
  79. package/skills/qa-explore-mobile/SKILL.md +86 -64
  80. package/skills/qa-explore-mobile/phases/00-setup.md +71 -62
  81. package/skills/qa-explore-mobile/phases/01-auth.md +28 -31
  82. package/skills/qa-explore-mobile/phases/02-charter.md +8 -4
  83. package/skills/qa-explore-mobile/phases/03-discovery.md +17 -11
  84. package/skills/qa-explore-mobile/phases/04-journeys.md +6 -6
  85. package/skills/qa-explore-mobile/phases/05-features.md +9 -9
  86. package/skills/qa-explore-mobile/phases/06-edge-cases.md +35 -40
  87. package/skills/qa-explore-mobile/phases/07-reporting.md +82 -56
  88. package/skills/qa-explore-mobile/references/mobile-edges.md +26 -26
  89. package/skills/qa-explore-quick/SKILL.md +60 -25
  90. package/skills/qa-explore-report/SKILL.md +40 -34
  91. package/skills/qa-gather/SKILL.md +17 -2
  92. package/skills/qa-knowledge-add/SKILL.md +20 -9
  93. package/skills/qa-knowledge-list/SKILL.md +12 -3
  94. package/skills/qa-target-setup/SKILL.md +63 -41
  95. package/skills/qa-verify-backend/SKILL.md +172 -0
  96. package/skills/qa-verify-backend/phases/00-setup.md +142 -0
  97. package/skills/qa-verify-backend/phases/01-ac-decomposition.md +58 -0
  98. package/skills/qa-verify-backend/phases/02-static-review.md +71 -0
  99. package/skills/qa-verify-backend/phases/03-live-verification.md +62 -0
  100. package/skills/qa-verify-backend/phases/03b-api-verification.md +170 -0
  101. package/skills/qa-verify-backend/phases/04-e2e-trigger.md +82 -0
  102. package/skills/qa-verify-backend/phases/05-reporting.md +163 -0
  103. package/skills/qa-verify-backend/references/api-probes.md +168 -0
  104. package/skills/qa-verify-backend/references/aws-readonly-probes.md +156 -0
  105. package/skills/qa-verify-backend/references/environment-fingerprinting.md +104 -0
  106. package/skills/qa-verify-backend/references/payload-verification.md +112 -0
  107. package/skills/qa-verify-backend/references/release-readiness.md +97 -0
  108. package/skills/qa-verify-backend/references/safety-rules.md +95 -0
  109. package/dist/cli/index.d.ts +0 -1
  110. package/dist/cli/index.js.map +0 -1
  111. package/dist/index.js.map +0 -1
package/package.json CHANGED
@@ -1,63 +1,79 @@
1
1
  {
2
2
  "name": "qualiow-exploratory-testing",
3
- "version": "1.2.0",
4
- "description": "qualiow — AI-powered exploratory testing skills using Claude Code + Playwright CLI (web) and Maestro/mobile-cli (iOS/Android simulators)",
3
+ "version": "2.0.0",
4
+ "description": "qualiow — AI-powered exploratory testing skills for Claude Code: web via playwright-cli, iOS/Android simulators via Maestro/mobile-cli, and backend/API acceptance-criteria verification",
5
5
  "type": "module",
6
6
  "main": "dist/index.js",
7
7
  "types": "dist/index.d.ts",
8
8
  "bin": {
9
- "qualiow": "dist/cli/index.js"
9
+ "qualiow": "dist/cli/index.js",
10
+ "mcli": "bin/mcli",
11
+ "wadb": "bin/wadb",
12
+ "wk-ios": "bin/wk-ios",
13
+ "qualiow-doctor-mobile": "bin/doctor-mobile.sh",
14
+ "qualiow-setup-mobile": "bin/setup-mobile.sh"
10
15
  },
11
16
  "files": [
12
17
  "dist/",
13
18
  "skills/",
19
+ "agents/",
14
20
  "bin/",
15
- "scripts/setup-mobile.sh",
16
- "scripts/doctor-mobile.sh",
21
+ ".claude-plugin/",
17
22
  "data/domains/*.yml",
18
23
  "data/knowledge/",
19
24
  "data/templates/",
20
- "data/targets/_default.yml",
21
- "data/targets/_example-native-mobile.yml",
22
- "data/targets/_example-sim-ios-safari.yml",
23
- "data/targets/_example-sim-android-chrome.yml",
24
- "data/targets/_example-mobile-emulation.yml",
25
+ "data/security/",
26
+ "data/targets/_*.yml",
25
27
  "data/targets/testers-ai.yml",
26
28
  "docs/GETTING-STARTED.md",
27
- "docs/MOBILE-SETUP.md"
29
+ "docs/MOBILE-SETUP.md",
30
+ "docs/BACKEND-VERIFICATION.md",
31
+ ".env.example",
32
+ "CHANGELOG.md"
28
33
  ],
29
34
  "scripts": {
30
- "sync:skills": "rm -rf skills && cp -R .claude/skills skills",
31
- "prebuild": "npm run sync:skills",
35
+ "sync:plugin": "rm -rf skills agents && cp -R .claude/skills skills && cp -R .claude/agents agents",
36
+ "prebuild": "npm run sync:plugin",
32
37
  "build": "tsup",
33
38
  "dev": "tsup --watch",
34
39
  "test": "vitest run",
35
40
  "test:watch": "vitest",
36
41
  "test:coverage": "vitest run --coverage",
37
42
  "validate": "node dist/cli/index.js validate --all",
43
+ "kb:sync": "node dist/cli/index.js kb sync",
44
+ "kb:check": "node dist/cli/index.js kb check",
45
+ "check:mirror": "diff -rq .claude/skills skills && diff -rq .claude/agents agents",
46
+ "check:pack": "node scripts/check-pack.mjs",
38
47
  "lint": "tsc --noEmit",
39
- "prepublishOnly": "npm run build && npm test"
48
+ "version": "node scripts/sync-version.mjs && git add .claude-plugin/plugin.json",
49
+ "prepublishOnly": "npm run build && npm run lint && npm test && npm run check:pack"
40
50
  },
41
51
  "keywords": [
42
52
  "exploratory-testing",
43
53
  "qa",
44
54
  "testing",
45
55
  "playwright",
56
+ "playwright-cli",
46
57
  "maestro",
47
58
  "mobile-testing",
48
59
  "ios-simulator",
49
60
  "android-emulator",
61
+ "api-testing",
62
+ "acceptance-criteria",
50
63
  "ai",
51
64
  "claude",
65
+ "claude-code",
52
66
  "skills",
67
+ "plugin",
53
68
  "heuristics",
54
69
  "bug-detection"
55
70
  ],
56
71
  "author": "William Oliveira",
57
72
  "license": "MIT",
58
73
  "engines": {
59
- "node": ">=18.0.0"
74
+ "node": ">=22.4.0"
60
75
  },
76
+ "packageManager": "npm@10.9.2",
61
77
  "repository": {
62
78
  "type": "git",
63
79
  "url": "git+https://github.com/willcoliveira/qualiow-exploratory-testing-skills.git"
@@ -67,7 +83,7 @@
67
83
  "url": "https://github.com/willcoliveira/qualiow-exploratory-testing-skills/issues"
68
84
  },
69
85
  "dependencies": {
70
- "@playwright/cli": "^0.1.15",
86
+ "@playwright/cli": "^0.1.19",
71
87
  "chalk": "^5.3.0",
72
88
  "commander": "^15.0.0",
73
89
  "glob": "^13.0.6",
@@ -7,10 +7,11 @@ description: >
7
7
  and writes bug reports that executives act on.
8
8
  Use when user says: "explore", "test this site", "find bugs", "QA check", "exploratory session",
9
9
  or provides a URL to test.
10
- allowed-tools: Bash(playwright-cli:*), Bash(npx playwright-cli:*), Bash(npx playwright:*), Read, Write, Glob, Grep
10
+ argument-hint: "<url> [--target <id>] [--context <file>] [--focus <area>] [--session <dir>]"
11
+ allowed-tools: Bash(playwright-cli:*), Bash(npx playwright-cli:*), Bash(npx playwright:*), Bash(diff:*), Read, Write, Glob, Grep
11
12
  ---
12
13
 
13
- # Exploratory Testing Session (v2)
14
+ # Exploratory Testing Session
14
15
 
15
16
  ## Your Identity
16
17
 
@@ -29,43 +30,48 @@ You are a **Principal QA Engineer** with 20+ years of experience. You don't just
29
30
  # Mode 1: Blind exploration (default — always works, no prior info needed)
30
31
  /qa-explore https://example.com
31
32
 
32
- # Mode 2: Context-enriched (requirements file from tool-qa-workflow or manual)
33
- /qa-explore https://example.com --context requirements-SI-2305.md
33
+ # Mode 2: Context-enriched (a /qa-gather context file or any requirements document)
34
+ /qa-explore https://example.com --context output/context/checkout-context.md
34
35
 
35
36
  # Mode 3: Inline context (user provides context in their message)
36
37
  /qa-explore https://example.com
37
38
  > "This is a new checkout flow. Should support Visa/MC, require delivery address,
38
39
  > send confirmation email. Team just refactored the payment service."
39
40
 
40
- # With target config
41
- /qa-explore --target company-staging --context requirements-SI-2305.md
41
+ # With a saved target config, or reusing a `qualiow explore` pre-flight directory
42
+ /qa-explore --target company-staging --context output/context/SI-2305-context.md
43
+ /qa-explore https://example.com --session output/sessions/2026-09-08-1420-explore-example-com
42
44
  ```
43
45
 
44
- ---
46
+ ## Before You Start
45
47
 
46
- ## Session Phases
48
+ - Paths, target resolution, the session-directory scheme and the `-s=<sid>` session id are defined once in **`${CLAUDE_SKILL_DIR}/references/paths.md`**. Read it first.
49
+ - Every artefact you write follows **`${CLAUDE_SKILL_DIR}/references/output-contract.md`** (bug report, session report, `stats.json`, confidentiality header). `qualiow report` and the feedback skills only understand that format.
50
+ - **`${CLAUDE_SKILL_DIR}/references/security-rules.md`** is absolute: production detection, redaction before disk, session isolation, output classification.
47
51
 
48
- Execute each phase in order. Read and follow the linked file for full instructions.
52
+ ## Session Phases
49
53
 
50
- | Phase | File | Summary |
51
- |-------|------|---------|
52
- | **Setup** | Read and follow: `phases/00-setup.md` | Parse input, resolve target, load context, load knowledge, create session directory |
53
- | **Auth** | Read and follow: `phases/01-auth.md` | Authenticate via storage state, credentials, or token; verify login |
54
- | **Charter** | Read and follow: `phases/02-charter.md` | Use app as real user, identify journeys, risk-rank features, select heuristics, write charter |
55
- | **Discovery** | Read and follow: `phases/03-discovery.md` | Map the site, check console/network, enforce scope, find what's MISSING |
56
- | **Journeys** | Read and follow: `phases/04-journeys.md` | End-to-end user journeys, data integrity verification, cross-page consistency |
57
- | **Features** | Read and follow: `phases/05-features.md` | Deep feature testing with SFDIPOT, business logic stress, negative space, FEW HICCUPPS |
58
- | **Edge Cases** | Read and follow: `phases/06-edge-cases.md` | Input attacks, race conditions, state manipulation, security, accessibility, empty states |
59
- | **Reporting** | Read and follow: `phases/07-reporting.md` | Stop recording, reflect, write bug reports with business impact, session report, coverage map |
54
+ Execute each phase in order. Read and follow the linked file. The budget sums to 45 minutes; write to disk after each phase and carry only summaries in context.
60
55
 
61
- ---
56
+ | Phase | File | Budget | Summary |
57
+ |-------|------|--------|---------|
58
+ | **Setup** | `${CLAUDE_SKILL_DIR}/phases/00-setup.md` | 2 min | Parse input, resolve target and data paths, load context and knowledge, create the session directory |
59
+ | **Auth** | `${CLAUDE_SKILL_DIR}/phases/01-auth.md` | 2 min | Authenticate via storage state, credentials, or token; verify login |
60
+ | **Charter** | `${CLAUDE_SKILL_DIR}/phases/02-charter.md` | 5 min | Use the app as a real user, identify journeys, risk-rank features, select heuristics, write the charter |
61
+ | **Discovery** | `${CLAUDE_SKILL_DIR}/phases/03-discovery.md` | 6 min | Map the site, check console and requests, enforce scope, find what's MISSING |
62
+ | **Journeys** | `${CLAUDE_SKILL_DIR}/phases/04-journeys.md` | 10 min | End-to-end user journeys, data-integrity verification, cross-page consistency |
63
+ | **Features** | `${CLAUDE_SKILL_DIR}/phases/05-features.md` | 10 min | Deep feature testing with SFDIPOT, business-logic stress, negative space, FEW HICCUPPS |
64
+ | **Edge Cases** | `${CLAUDE_SKILL_DIR}/phases/06-edge-cases.md` | 6 min | Input attacks, race conditions, state manipulation, security, accessibility, empty states |
65
+ | **Reporting** | `${CLAUDE_SKILL_DIR}/phases/07-reporting.md` | 4 min | Stop recording, reflect, write bug reports with business impact, session report, stats, close the browser session |
62
66
 
63
67
  ## References
64
68
 
65
69
  Consult these as needed throughout the session:
66
70
 
67
- - **`references/security-rules.md`** — Prompt injection resistance, credential protection, session isolation, production safety, output classification
68
- - **`references/severity-guide.md`** — Severity definitions (Critical/High/Medium/Low) and the "when in doubt go LOWER" rule
69
- - **`references/session-rules.md`** — 20 testing discipline rules: 45-min cap, one bug one report, evidence, risk-proportional time, adapt mid-session, log WHY, AI bias avoidance
70
- - **`references/context-integration.md`** — How `--context` changes the session (context file, inline context, blind mode)
71
- - **`references/playwright-agents-integration.md`** — Opt-in handoff to Playwright Test Agents (planner / generator / healer, 1.56+) when a reproducible bug should become a regression test. Requires the `@playwright/test` peer dependency.
71
+ - **`${CLAUDE_SKILL_DIR}/references/paths.md`** — resolution order for targets, data, credentials and output; session-directory scheme; session id; index rows
72
+ - **`${CLAUDE_SKILL_DIR}/references/output-contract.md`** — the one bug-report and session-report format, `stats.json`, confidentiality header
73
+ - **`${CLAUDE_SKILL_DIR}/references/security-rules.md`** — prompt-injection resistance, production rule, redaction list, session isolation, output classification
74
+ - **`${CLAUDE_SKILL_DIR}/references/severity-guide.md`** — severity definitions, the "when in doubt go LOWER" rule, risk matrix, priority table
75
+ - **`${CLAUDE_SKILL_DIR}/references/session-rules.md`** — 20 testing-discipline rules: 45-min cap, one bug one report, evidence, risk-proportional time, adapt mid-session, log WHY, AI-bias avoidance
76
+ - **`${CLAUDE_SKILL_DIR}/references/context-integration.md`** — how `--context` changes the session (context file, inline context, blind mode)
77
+ - **`${CLAUDE_SKILL_DIR}/references/playwright-agents-integration.md`** — opt-in handoff to Playwright Test Agents (planner / generator / healer, Playwright 1.56+) when a reproducible bug should become a regression test; requires the optional `@playwright/test` peer dependency
@@ -1,44 +1,53 @@
1
- # Phase 0: Session Setup
1
+ # Phase 0: Session Setup (2 min)
2
2
 
3
3
  ## Step 1: Parse Input & Resolve Target
4
4
 
5
- Extract from user message:
6
- - **target_url**: The URL to test (required)
7
- - **domain**: Domain type (optional -- auto-detect if not specified)
8
- - **focus**: Specific area or feature (optional)
9
- - **time_box**: Session duration (default: 45 min, max: 45 min)
10
- - **context**: Path to requirements file OR inline context text (optional)
11
-
12
- Resolve target config in this order:
13
- 1. If `--target <name>` given → load from skill `data/targets/<name>.yml`
14
- 2. Else if CWD contains `qa/target.yml` → load it as the target config
15
- 3. Else fall back to `data/targets/_default.yml`
16
-
17
- If loaded from `qa/target.yml`, also check `qa/.env` in CWD for credential env var values before falling back to the skill root `.env`.
18
-
19
- Read domain config from `data/domains/`.
5
+ Extract from the user message:
6
+ - **target_url**: the URL to test (required unless `--target` is given)
7
+ - **--target <id>**: a saved target config (optional)
8
+ - **--context <file>** or inline context text (optional)
9
+ - **--focus <area>**: a specific area or feature (optional)
10
+ - **--session <dir>**: an existing pre-flight directory created by `qualiow explore` (optional)
11
+ - **domain**: domain type (optional; auto-detect from the app if not given)
12
+ - **time_box**: 45 min (default and maximum)
13
+
14
+ Resolve the target config, the data directory (`<data>`) and the credentials file in the
15
+ order defined in `${CLAUDE_SKILL_DIR}/references/paths.md`: `--target` → `data/targets/<id>.yml`
16
+ in the project, else the plugin's copy; no `--target` → `qa/target.yml` in the project, else
17
+ `_default.yml`; credentials from `qa/.env`, else `.env`. Then read
18
+ `<data>/domains/<domain>.yml` for the domain's completeness checklist, data-integrity checks
19
+ and journeys.
20
+
21
+ Apply the **production rule** from `${CLAUDE_SKILL_DIR}/references/security-rules.md` to the
22
+ resolved hostname now. If it fires, log `[SAFETY] Production detected -- running in read-only mode`
23
+ and carry read-only mode through every later phase.
20
24
 
21
25
  ## Step 2: Load Context (if provided)
22
26
 
23
- **If `--context <file>` provided:** Read the file. It may be a `tool-qa-workflow` output (structured markdown with Jira data, Confluence specs, GitLab MR diffs, Figma design notes) or any document describing what to test.
27
+ **If `--context <file>` provided:** read the file. It may be a `/qa-gather` context file
28
+ (structured markdown with ticket data, specs, MR diffs, design notes) or any document
29
+ describing what to test.
24
30
 
25
- **If inline context provided:** Extract from the user's message any descriptions of: what the feature does, acceptance criteria, what changed, what the design should look like, known risks, user personas.
31
+ **If inline context provided:** extract from the user's message any descriptions of: what
32
+ the feature does, acceptance criteria, what changed, what the design should look like,
33
+ known risks, user personas.
26
34
 
27
- **If no context provided:** That's fine -- run in blind exploration mode. The agent discovers everything on its own.
35
+ **If no context provided:** that's fine; run in blind exploration mode.
28
36
 
29
37
  **When context IS available, extract and use:**
30
38
 
31
39
  | Context Element | How It Changes the Session |
32
40
  |----------------|---------------------------|
33
- | **Acceptance criteria** | Add as explicit checkpoints -- verify each AC during testing |
34
- | **What changed** (MR diff, commits) | Focus testing on changed areas -- highest regression risk |
35
- | **Design specs** (Figma, mockups) | Compare live app against intended design -- spot deviations |
41
+ | **Acceptance criteria** | Add as explicit checkpoints; verify each AC during testing |
42
+ | **What changed** (MR diff, commits) | Focus testing on changed areas; highest regression risk |
43
+ | **Design specs** (Figma, mockups) | Compare the live app against the intended design |
36
44
  | **User stories / requirements** | Test against INTENT, not just what's visible |
37
- | **Known risks** (from risk analysis) | Prioritize testing on flagged risk areas |
38
- | **API contracts** (OpenAPI, endpoints) | Verify UI matches API behavior |
39
- | **Previous test results** | Don't re-test what passed -- focus on gaps |
45
+ | **Known risks** | Prioritize testing on flagged risk areas |
46
+ | **API contracts** (OpenAPI, endpoints) | Verify the UI matches API behaviour |
47
+ | **Previous test results** | Don't re-test what passed; focus on gaps |
40
48
 
41
- **Context makes exploration sharper but NEVER blocks it.** A QA walking up to a random app with zero documentation should still get a full, valuable session. Context is a boost, not a gate.
49
+ **Context makes exploration sharper but NEVER blocks it.** A QA walking up to a random app
50
+ with zero documentation should still get a full, valuable session.
42
51
 
43
52
  When context is loaded, note in the charter:
44
53
  ```
@@ -49,17 +58,24 @@ CHANGE SCOPE: [what changed or "unknown -- full exploration"]
49
58
 
50
59
  ## Step 3: Load Knowledge
51
60
 
52
- Read `data/knowledge/manifest.yml`. Load:
61
+ Read `<data>/knowledge/manifest.yml`. Load:
53
62
  1. Core heuristics (SFDIPOT, FEW HICCUPPS, Test Tours)
54
- 2. Domain-specific entries
55
- 3. `data/knowledge/learned-patterns.md` -- false-positive skip list + must-check patterns
56
- 4. Business logic & race condition techniques (if app has transactions/state changes)
63
+ 2. Domain-specific entries from `loading_strategy.by_domain`
64
+ 3. `<data>/knowledge/learned-patterns.md`: the false-positive skip list and must-check patterns
65
+ 4. Business-logic and race-condition techniques (if the app has transactions or state changes)
66
+
67
+ Keep brief summaries only; don't fill context with full entries.
57
68
 
58
- Keep brief summaries only -- don't fill context with full entries.
69
+ ## Step 4: Create the Session Directory
59
70
 
60
- ## Step 4: Create Session Directory
71
+ Unless `--session <dir>` was given, create
72
+ `output/sessions/<YYYY-MM-DD-HHmm>-explore-<slug>/` (scheme in `paths.md`; `<slug>` is the
73
+ target id without a leading `_`, or the hostname with dots replaced by dashes) containing
74
+ `screenshots/`, `bugs/`, `videos/`, and an empty `session-log.md`. With `--session`, reuse
75
+ that directory and append to its `session-log.md`.
61
76
 
62
- Create `output/sessions/<YYYY-MM-DD-HHmm-target>/` with: `charter.md`, `session-log.md`, `screenshots/`, `bugs/`
77
+ Choose the playwright-cli session id `-s=explore-<HHmm>-<slug>`. **Every `playwright-cli`
78
+ call in this session carries `-s=<sid>`; the examples in the phase files omit the prefix.**
63
79
 
64
80
  ## Step 5: Initialize Progress Tracking
65
81
 
@@ -67,6 +83,7 @@ Write `output/sessions/<session-dir>/progress.json`:
67
83
  ```json
68
84
  {
69
85
  "session_id": "<session-dir>",
86
+ "kind": "explore",
70
87
  "target": "<target_url>",
71
88
  "domain": "<domain>",
72
89
  "started_at": "<ISO timestamp>",
@@ -91,27 +108,36 @@ Write `output/sessions/<session-dir>/progress.json`:
91
108
 
92
109
  Append to `session-log.md`:
93
110
  ```
94
- [<timestamp>] [PHASE] Setup complete — session initialized
111
+ [<timestamp>] [PHASE] Setup complete — session <session-dir>, sid <sid>, read_only: <true|false>
95
112
  ```
96
113
 
97
114
  ## Step 6: Optional — Bootstrap Playwright Test Agents (opt-in)
98
115
 
99
- **Skip this step entirely if `@playwright/test` is not installed as a peer dep OR `playwright.config.*` is not present.** This is an additive capability for consumers who want to pair an exploratory session with the Playwright Test Agents framework (planner / generator / healer, Playwright 1.56+).
116
+ **Skip this step entirely if `@playwright/test` is not installed in the consumer project OR
117
+ no `playwright.config.*` is present.** This is an additive capability for teams that want to
118
+ turn reproducible findings into regression tests with the Playwright Test Agents framework
119
+ (planner / generator / healer, Playwright 1.56+).
100
120
 
101
- If both prerequisites are present AND the user has opted in (via target config or inline context), you MAY offer to scaffold the agent workspace:
121
+ If both prerequisites are present AND the user has opted in (via the target config or the
122
+ context), you MAY offer to scaffold the agent workspace:
102
123
 
103
124
  ```bash
104
125
  npx playwright init-agents --loop=claude
105
126
  ```
106
127
 
107
- This generates agent definition files that later phases can hand off to — most usefully in phase 5 (features), where a reproducible bug can be converted into a regression test by the generator agent. See `references/playwright-agents-integration.md` for the full workflow.
128
+ With `--loop=claude` this writes the three agent definitions into `.claude/agents/`, a
129
+ `.mcp.json` registering the `playwright-test` server, and a `specs/` scaffold with a seed
130
+ spec if the project has none; other loops: `codex`, `copilot`, `opencode`, `vscode`,
131
+ `vscode-legacy`. It does not touch `playwright.config.*`. See
132
+ `${CLAUDE_SKILL_DIR}/references/playwright-agents-integration.md` for the hand-off after the
133
+ session.
108
134
 
109
- **Never block the session on this step.** If the user declines, if the prerequisites are missing, or if the command fails, log a note in `session-log.md` and continue to Phase 1 normally.
135
+ **Never block the session on this step.** If the user declines, the prerequisites are
136
+ missing, or the command fails, log a note in `session-log.md` and continue.
110
137
 
111
- **IMPORTANT: After EVERY phase completion, update progress.json and append to session-log.md.** This is how the user monitors session progress. Use this pattern at the end of each phase:
138
+ **IMPORTANT: After EVERY phase completion, update `progress.json` and append to
139
+ `session-log.md`.** This is how the user monitors progress:
112
140
 
113
141
  ```
114
- # Update progress.json — set current phase complete, next phase as current
115
- # Append to session-log.md:
116
142
  [<timestamp>] [PHASE] <phase_name> complete — <pages> pages, <bugs> bugs, <key finding summary>
117
143
  ```
@@ -1,6 +1,6 @@
1
- # Phase 1: Authentication
1
+ # Phase 1: Authentication (2 min)
2
2
 
3
- Skip if `auth.strategy: none`. Otherwise:
3
+ Skip if `auth.strategy: none`. Every command below carries the session id `-s=<sid>` (omitted here).
4
4
 
5
5
  ## Storage State (fastest)
6
6
 
@@ -9,30 +9,47 @@ playwright-cli state-load .auth/<target>.json
9
9
  playwright-cli goto <base_url>
10
10
  ```
11
11
 
12
- ## Credentials (adaptive -- uses snapshot to find form fields)
12
+ If the state file is older than 24 hours, expect it to have expired and fall through to
13
+ credentials when the login page appears.
13
14
 
14
- Before filling credentials, resolve their values:
15
- 1. If `qa/.env` exists in CWD, read it — use the values for the env var names specified in `auth.credentials.username` / `auth.credentials.password` in the target config
16
- 2. Otherwise read from the skill root `.env`
15
+ ## Credentials (adaptive: uses snapshot to find form fields)
16
+
17
+ Resolve the values named by `auth.credentials.username` / `auth.credentials.password` in the
18
+ order from `${CLAUDE_SKILL_DIR}/references/paths.md` (`qa/.env`, then `.env`). Read the value,
19
+ pass it to the command, and never echo it into the log or the report.
17
20
 
18
21
  ```bash
19
22
  playwright-cli open <login_url>
20
23
  playwright-cli snapshot
21
- playwright-cli fill <email_ref> "$QA_USER"
22
- playwright-cli fill <pass_ref> "$QA_PASS"
24
+ playwright-cli fill <email_ref> "<username value>"
25
+ playwright-cli fill <pass_ref> "<password value>"
23
26
  playwright-cli click <submit_ref>
24
27
  playwright-cli state-save .auth/<target>.json
25
28
  ```
26
29
 
30
+ ## Token (`auth.strategy: token`)
31
+
32
+ Inject the token named by `auth.token` where the app expects it, then reload:
33
+
34
+ ```bash
35
+ playwright-cli open <base_url>
36
+ playwright-cli cookie-set <cookie-name> "<token value>" --domain=<host> --httpOnly --secure # cookie-based
37
+ playwright-cli localstorage-set <key> "<token value>" # localStorage-based
38
+ playwright-cli reload
39
+ ```
40
+
27
41
  ## Verify Login
28
42
 
29
- After authentication, verify login via `playwright-cli snapshot` -- confirm user indicators are present (e.g., username display, avatar, dashboard access).
43
+ After authentication, `playwright-cli snapshot` and confirm user indicators are present
44
+ (username display, avatar, dashboard access).
30
45
 
31
46
  If login fails:
32
- 1. Check console for errors: `playwright-cli console error`
33
- 2. Verify credentials are correct
34
- 3. Check for MFA requirements
35
- 4. Log the failure and attempt alternative auth strategy if available
36
-
37
- **Update progress:** Set auth phase complete (or failed) in progress.json. Append to session-log.md:
47
+ 1. Check the console: `playwright-cli console error`
48
+ 2. Check the last requests: `playwright-cli requests --filter="login|auth|session"`
49
+ 3. Verify the credentials are the ones the target names
50
+ 4. Check for MFA. Never automate an MFA challenge; hand it to the user in a headed browser
51
+ (`playwright-cli open <login_url> --headed`) and continue once they confirm
52
+ 5. Log the failure and attempt an alternative strategy if the target defines one
53
+
54
+ **Update progress:** set the auth phase complete (or failed) in `progress.json`. Append to `session-log.md`:
38
55
  `[<timestamp>] [PHASE] Auth complete — strategy: <strategy>, result: <success/failed>, user: <redacted>`
@@ -1,10 +1,10 @@
1
- # Phase 2: Business Context & Charter (~5 min)
1
+ # Phase 2: Business Context & Charter (5 min)
2
2
 
3
3
  **This is the most important phase. Skip it and everything else is shallow.**
4
4
 
5
5
  ## Step 1: Use the App as a Real User
6
6
 
7
- Before any testing, spend 2-3 minutes USING the app the way a customer would. Not testing -- USING.
7
+ Before any testing, spend 2 minutes USING the app the way a customer would. Not testing -- USING.
8
8
 
9
9
  ```bash
10
10
  playwright-cli open <url>
@@ -62,9 +62,12 @@ Also select 2 Test Tours:
62
62
 
63
63
  ## Step 5: Write Charter
64
64
 
65
- Save to `output/sessions/<session-dir>/charter.md`:
65
+ Save to `output/sessions/<session-dir>/charter.md` (confidentiality header first, per `${CLAUDE_SKILL_DIR}/references/output-contract.md`):
66
66
 
67
67
  ```markdown
68
+ > CONFIDENTIAL: This report may contain internal URLs, security vulnerabilities,
69
+ > and application details. Do not share outside your organization without review.
70
+
68
71
  # Session Charter
69
72
 
70
73
  ## Context Source
@@ -101,12 +104,20 @@ Save to `output/sessions/<session-dir>/charter.md`:
101
104
  - FEW HICCUPPS oracles: Focus on [Users, Claims, Product]
102
105
 
103
106
  ## Session Parameters
104
- - Time box: [duration]
107
+ - Time box: 45 min
105
108
  - Domain: [domain]
106
109
  - Focus: [focus area or "full exploration"]
110
+ - Read-only: [true | false]
111
+ ```
112
+
113
+ Start evidence capture (both carry `-s=<sid>`):
114
+
115
+ ```bash
116
+ playwright-cli tracing-start
117
+ playwright-cli video-start output/sessions/<session-dir>/videos/session.webm
107
118
  ```
108
119
 
109
- Start trace: `playwright-cli tracing-start`
120
+ Add `playwright-cli video-chapter "<phase or feature>"` at each phase boundary so the recording is navigable.
110
121
 
111
122
  **Update progress:** Set charter phase complete in progress.json. Append to session-log.md:
112
123
  `[<timestamp>] [PHASE] Charter complete — risk ranking defined, <N> journeys planned, heuristics: <selected>`
@@ -1,6 +1,6 @@
1
- # Phase 3: Discovery & Mapping (~15% of time)
1
+ # Phase 3: Discovery & Mapping (6 min)
2
2
 
3
- **Goal:** Map the app structure and refine your risk ranking.
3
+ **Goal:** Map the app structure and refine your risk ranking. Every command carries `-s=<sid>` (omitted here).
4
4
 
5
5
  ## Site Mapping
6
6
 
@@ -8,56 +8,80 @@
8
8
  playwright-cli goto <start_url>
9
9
  playwright-cli snapshot
10
10
  playwright-cli console error
11
- playwright-cli network
11
+ playwright-cli requests
12
12
  ```
13
13
 
14
14
  For each page:
15
- 1. `playwright-cli snapshot` -- note structure, forms, interactive elements
16
- 2. Click through navigation to discover all sections
15
+ 1. `playwright-cli snapshot` — note structure, forms, interactive elements. Use `snapshot --depth=3` on very large pages, then `snapshot <ref>` for the region you care about.
16
+ 2. Click through navigation to discover all sections.
17
17
  3. Log: `[DISCOVERED] <url> -- <description> -- Risk: <P0/P1/P2/P3>`
18
- 4. Check `playwright-cli console error` on each page
18
+ 4. Check `playwright-cli console error` and `playwright-cli requests` on each page.
19
+
20
+ Useful helpers:
21
+ - `playwright-cli find "<text>"` (or `find --regex "<re>"`) — locate a label or message in the snapshot without reading the whole tree.
22
+ - `playwright-cli --raw snapshot > before.yml`, act, `playwright-cli --raw snapshot > after.yml`, then `diff before.yml after.yml` — see exactly what an action changed.
23
+ - `playwright-cli requests --filter="/api/"` — only the calls that matter; `playwright-cli request <n>` for headers and body of one of them.
19
24
 
20
25
  ## Scope Enforcement
21
26
 
22
- Stay within the target domain. Do not follow links to external sites unless they are part of the application's workflow (e.g., OAuth redirects, payment gateways).
27
+ Stay within the target domain. Do not follow links to external sites unless they are part of
28
+ the application's workflow (OAuth redirects, payment gateways).
23
29
 
24
30
  ## What's MISSING Check
25
31
 
26
- **Critical during discovery -- look for what's MISSING:**
27
- - Read the domain completeness checklist from `data/domains/<domain>.md` (or the corresponding `.yml`)
28
- - Compare what the app HAS vs what it SHOULD have
29
- - E-commerce missing payment method? Log it now.
32
+ **Critical during discovery — look for what's MISSING:**
33
+ - Read the domain completeness checklist from `<data>/domains/<domain>.yml` (`completeness_checklist`)
34
+ - Compare what the app HAS against what it SHOULD have
35
+ - E-commerce missing a payment method? Log it now.
30
36
  - Banking app missing transaction history? Flag it immediately.
31
37
  - No error messages on empty states? Note it.
32
38
 
33
39
  ## Console & Network Checks
34
40
 
35
41
  On every page during discovery:
36
- - `playwright-cli console error` -- log any JavaScript errors
37
- - `playwright-cli network` -- note any failed requests (4xx, 5xx)
38
- - Watch for mixed content warnings (HTTP resources on HTTPS pages)
42
+ - `playwright-cli console error` — JavaScript errors
43
+ - `playwright-cli requests` — failed requests (4xx, 5xx); `requests --static` to include assets
44
+ - Watch for mixed-content warnings (HTTP resources on HTTPS pages)
45
+
46
+ ### Cross-navigation observation via `run-code` (Playwright 1.59+)
39
47
 
40
- ### API-level observation helpers (optional, Playwright 1.56+)
48
+ The CLI's `console` and `requests` show only the tail since the last navigation. To reach
49
+ messages from *before* it, run Playwright's retrieval methods inside the session with
50
+ `run-code` (the callback receives the live `page`) and pass `filter: 'all'` — **without it
51
+ they default to `since-navigation` and return exactly the same window as the CLI**. The
52
+ buffers are capped: 200 console messages, 200 page errors, 100 requests.
41
53
 
42
- If the session has access to the `@playwright/test` peer dep AND is running inside a Playwright test context (rare for qa-explore, common for bug-reproduction follow-ups), the following 1.56 retrieval methods can be used alongside the CLI commands above:
54
+ ```bash
55
+ # every console message still buffered, across navigations
56
+ playwright-cli run-code "async page => (await page.consoleMessages({ filter: 'all' })).map(m => m.type() + ': ' + m.text()).join('\n')"
43
57
 
44
- - `await page.consoleMessages()` — returns all console messages captured so far on the page. Useful when you need the full history rather than the most-recent tail.
45
- - `await page.pageErrors()` — returns runtime page errors (uncaught exceptions). More reliable than filtering `console error` output.
46
- - `await page.requests()` — returns all network requests issued by the page, with status, resource type, and timing. Use this for a single network snapshot instead of parsing `playwright-cli network` output twice.
58
+ # uncaught exceptions only — more reliable than filtering console output
59
+ playwright-cli run-code "async page => (await page.pageErrors({ filter: 'all' })).map(e => e.message).join('\n')"
60
+
61
+ # failures and error statuses among the last 100 requests (page.requests() takes no filter)
62
+ playwright-cli run-code "async page => { const out = []; for (const r of await page.requests()) { const res = await r.response(); const f = r.failure(); if (f) out.push('FAILED ' + f.errorText + ' ' + r.url()); else if (res && res.status() >= 400) out.push(res.status() + ' ' + r.url()); } return out.join('\n'); }"
63
+ ```
47
64
 
48
- These are OPTIONAL and only apply when exploratory testing is done from a Playwright test harness. The bash `playwright-cli` commands above remain the primary path for the skill.
65
+ These are optional; the bash commands above remain the primary path.
49
66
 
50
- ### Service Workers (Playwright 1.56+)
67
+ ### Service Workers (Playwright 1.61+)
51
68
 
52
- Playwright 1.56 added Service Worker network routing. If `playwright-cli network` shows requests you cannot account for on the main document, assume a Service Worker is intercepting them and flag this as a high-risk area worth re-testing with the route filter.
69
+ Service-worker requests are reported and routable at the **BrowserContext** level from 1.61.
70
+ `playwright-cli route` maps to `page.route()`, which per the API docs still does **not**
71
+ intercept service-worker traffic. So if `playwright-cli requests` shows calls you cannot
72
+ account for on the main document, assume a service worker is serving them: flag the area as
73
+ high-risk, and note in the report that mocking it needs service workers blocked
74
+ (`serviceWorkers: 'block'`), not `route`.
53
75
 
54
76
  ## After Discovery
55
77
 
56
- 1. Write findings to `phase-1-discovery.md`
78
+ 1. Write findings to `phase-3-discovery.md` (confidentiality header first)
57
79
  2. **Update your risk ranking** if discovery changed your understanding
58
- 3. Carry forward: site map summary, risk ranking, list of what's missing
80
+ 3. Carry forward: site-map summary, risk ranking, list of what's missing
59
81
 
60
- **Log your reasoning:** "I expected a payment method selector in checkout because this is e-commerce, but it's missing. I'll flag this as a medium bug."
82
+ **Log your reasoning:** "I expected a payment-method selector in checkout because this is
83
+ e-commerce, but it's missing. I'll flag this as a medium bug."
61
84
 
62
- **Update progress:** Set discovery phase complete in progress.json with pages_explored count. Append to session-log.md:
85
+ **Update progress:** set the discovery phase complete in `progress.json` with the
86
+ `pages_explored` count. Append to `session-log.md`:
63
87
  `[<timestamp>] [PHASE] Discovery complete — <N> pages mapped, <N> console errors, <N> missing checklist items, <N> bugs so far`
@@ -1,4 +1,4 @@
1
- # Phase 4: End-to-End User Journeys (~25% of time)
1
+ # Phase 4: End-to-End User Journeys (10 min)
2
2
 
3
3
  **Goal:** Test the critical user journeys you defined in the charter -- end to end, crossing multiple features.
4
4
 
@@ -43,7 +43,7 @@ Do things change? Are order IDs unique? Are timestamps correct?
43
43
 
44
44
  ## After Journeys
45
45
 
46
- 1. Write findings to `phase-2-user-journeys.md`
46
+ 1. Write findings to `phase-4-journeys.md` (confidentiality header first)
47
47
  2. Include data integrity results
48
48
  3. Note any cross-feature inconsistencies
49
49