qa-engineer 0.9.1 → 0.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (109) hide show
  1. package/COMPATIBILITY.md +50 -26
  2. package/README.md +40 -32
  3. package/package.json +12 -20
  4. package/packages/engine/bin/qa-engine.mjs +336 -0
  5. package/packages/engine/lib/analysis/branding.mjs +187 -0
  6. package/packages/engine/lib/analysis/context.mjs +294 -0
  7. package/packages/{installer/lib/core/schema-validate.mjs → engine/lib/analysis/contracts.mjs} +11 -4
  8. package/packages/engine/lib/analysis/diff-guard.mjs +425 -0
  9. package/packages/engine/lib/analysis/discovery.mjs +220 -0
  10. package/packages/engine/lib/analysis/evidence.mjs +116 -0
  11. package/packages/engine/lib/analysis/har.mjs +90 -0
  12. package/packages/engine/lib/analysis/junit.mjs +126 -0
  13. package/packages/engine/lib/analysis/redaction.mjs +127 -0
  14. package/packages/engine/lib/analysis/report-html.mjs +763 -0
  15. package/packages/engine/lib/analysis/taxonomy.mjs +109 -0
  16. package/packages/engine/lib/analysis/xml.mjs +153 -0
  17. package/packages/engine/lib/analysis/zip.mjs +107 -0
  18. package/packages/engine/lib/diagnostics/engine.mjs +165 -0
  19. package/packages/engine/lib/diagnostics/internal-contracts.mjs +50 -0
  20. package/packages/engine/lib/diagnostics/prioritization.mjs +99 -0
  21. package/packages/engine/lib/diagnostics/repair.mjs +73 -0
  22. package/packages/engine/lib/diagnostics/root-cause.mjs +92 -0
  23. package/packages/engine/lib/diagnostics/timeline.mjs +101 -0
  24. package/packages/engine/lib/frameworks/junit-frameworks.mjs +48 -0
  25. package/packages/engine/lib/frameworks/playwright.mjs +158 -0
  26. package/packages/engine/package.json +14 -0
  27. package/packages/installer/lib/agents/registry.mjs +46 -0
  28. package/packages/installer/lib/commands/doctor.mjs +7 -16
  29. package/packages/installer/lib/commands/install.mjs +59 -4
  30. package/packages/installer/lib/core/bundle.mjs +69 -92
  31. package/packages/installer/lib/core/config.mjs +1 -1
  32. package/packages/installer/lib/core/lockfile.mjs +1 -1
  33. package/packages/installer/lib/core/manifest.mjs +48 -54
  34. package/packages/installer/lib/core/validate-install.mjs +12 -19
  35. package/packages/installer/package.json +1 -1
  36. package/shared/tooling/qa-tool.mjs +125 -0
  37. package/skills/qa-api/SKILL.md +5 -5
  38. package/skills/qa-api/references/deterministic-tooling.md +34 -30
  39. package/skills/qa-api/references/evidence-and-reporting.md +29 -4
  40. package/skills/qa-api/scripts/qa-tool.mjs +126 -0
  41. package/skills/qa-audit/SKILL.md +5 -5
  42. package/skills/qa-audit/references/deterministic-tooling.md +34 -30
  43. package/skills/qa-audit/references/evidence-and-reporting.md +29 -4
  44. package/skills/qa-audit/scripts/qa-tool.mjs +126 -0
  45. package/skills/qa-debug/SKILL.md +6 -6
  46. package/skills/qa-debug/references/deterministic-tooling.md +34 -30
  47. package/skills/qa-debug/references/diagnostic-engine.md +3 -3
  48. package/skills/qa-debug/references/evidence-and-reporting.md +29 -4
  49. package/skills/qa-debug/scripts/qa-tool.mjs +126 -0
  50. package/skills/qa-explore/SKILL.md +22 -5
  51. package/skills/qa-explore/contracts/explore-result.schema.json +26 -2
  52. package/skills/qa-explore/references/deterministic-tooling.md +132 -0
  53. package/skills/qa-explore/references/evidence-and-reporting.md +29 -4
  54. package/skills/qa-explore/references/report-pipeline.md +89 -8
  55. package/skills/qa-explore/scripts/qa-tool.mjs +126 -0
  56. package/skills/qa-fix/SKILL.md +4 -4
  57. package/skills/qa-fix/references/deterministic-tooling.md +34 -30
  58. package/skills/qa-fix/references/diagnostic-engine.md +3 -3
  59. package/skills/qa-fix/references/evidence-and-reporting.md +29 -4
  60. package/skills/qa-fix/scripts/qa-tool.mjs +126 -0
  61. package/skills/qa-flaky/SKILL.md +4 -4
  62. package/skills/qa-flaky/references/deterministic-tooling.md +34 -30
  63. package/skills/qa-flaky/references/evidence-and-reporting.md +29 -4
  64. package/skills/qa-flaky/scripts/qa-tool.mjs +126 -0
  65. package/skills/qa-generate/references/evidence-and-reporting.md +29 -4
  66. package/skills/qa-init/SKILL.md +4 -4
  67. package/skills/qa-init/references/deterministic-tooling.md +34 -30
  68. package/skills/qa-init/references/evidence-and-reporting.md +29 -4
  69. package/skills/qa-init/scripts/qa-tool.mjs +126 -0
  70. package/skills/qa-report/SKILL.md +8 -7
  71. package/skills/qa-report/references/deterministic-tooling.md +34 -30
  72. package/skills/qa-report/references/diagnostic-engine.md +3 -3
  73. package/skills/qa-report/references/evidence-and-reporting.md +29 -4
  74. package/skills/qa-report/scripts/qa-tool.mjs +126 -0
  75. package/skills/qa-review/references/evidence-and-reporting.md +29 -4
  76. package/skills/qa-run/SKILL.md +6 -6
  77. package/skills/qa-run/references/deterministic-tooling.md +34 -30
  78. package/skills/qa-run/references/evidence-and-reporting.md +29 -4
  79. package/skills/qa-run/scripts/qa-tool.mjs +126 -0
  80. package/shared/analysis/lib/qa_analysis/__init__.py +0 -11
  81. package/shared/analysis/lib/qa_analysis/branding.py +0 -175
  82. package/shared/analysis/lib/qa_analysis/cli.py +0 -129
  83. package/shared/analysis/lib/qa_analysis/context.py +0 -233
  84. package/shared/analysis/lib/qa_analysis/contracts.py +0 -158
  85. package/shared/analysis/lib/qa_analysis/diff_guard.py +0 -327
  86. package/shared/analysis/lib/qa_analysis/discovery.py +0 -113
  87. package/shared/analysis/lib/qa_analysis/evidence.py +0 -128
  88. package/shared/analysis/lib/qa_analysis/har.py +0 -58
  89. package/shared/analysis/lib/qa_analysis/junit.py +0 -80
  90. package/shared/analysis/lib/qa_analysis/redaction.py +0 -99
  91. package/shared/analysis/lib/qa_analysis/taxonomy.py +0 -98
  92. package/shared/diagnostics/lib/qa_diagnostics/__init__.py +0 -13
  93. package/shared/diagnostics/lib/qa_diagnostics/cli.py +0 -124
  94. package/shared/diagnostics/lib/qa_diagnostics/engine.py +0 -143
  95. package/shared/diagnostics/lib/qa_diagnostics/internal_contracts.py +0 -75
  96. package/shared/diagnostics/lib/qa_diagnostics/prioritization.py +0 -101
  97. package/shared/diagnostics/lib/qa_diagnostics/repair.py +0 -72
  98. package/shared/diagnostics/lib/qa_diagnostics/root_cause.py +0 -89
  99. package/shared/diagnostics/lib/qa_diagnostics/timeline.py +0 -71
  100. package/shared/frameworks/cypress/lib/cypress_analysis.py +0 -27
  101. package/shared/frameworks/playwright/lib/playwright_analysis.py +0 -167
  102. package/shared/frameworks/selenium/lib/selenium_analysis.py +0 -28
  103. package/shared/frameworks/webdriverio/lib/webdriverio_analysis.py +0 -24
  104. package/shared/tooling/qa_tool.py +0 -127
  105. /package/{shared/analysis/lib/qa_analysis → packages/engine/lib/analysis}/branding.json +0 -0
  106. /package/{shared → packages/engine/lib}/analysis/schemas/context.schema.json +0 -0
  107. /package/{shared → packages/engine/lib}/diagnostics/schemas/internal/analysis-result.schema.json +0 -0
  108. /package/{shared → packages/engine/lib}/diagnostics/schemas/internal/diagnosis.schema.json +0 -0
  109. /package/{shared → packages/engine/lib}/diagnostics/schemas/internal/execution-result-min.schema.json +0 -0
@@ -0,0 +1,126 @@
1
+ // synced-from: shared/tooling/qa-tool.mjs — do not edit; edit the source and run: node scripts/sync-shared.mjs --write
2
+ // The launcher every skill invokes to reach the deterministic engine.
3
+ //
4
+ // A skill's SKILL.md documents exactly one command shape:
5
+ //
6
+ // node <SKILL_DIR>/scripts/qa-tool.mjs <tool> <subcommand> [args]
7
+ //
8
+ // and this file finds the engine, wherever it happens to be. That indirection
9
+ // exists because the pack is installed three different ways and the engine lands in
10
+ // a different place each time:
11
+ //
12
+ // 1. `qa install` bundles the engine into the skill, at ./lib/. Offline, fastest,
13
+ // and pinned to the version that was installed.
14
+ // 2. `npx skills add <owner>/<repo>` — or any generic file copier — copies the
15
+ // skill directory out of git and bundles nothing. The engine is then resolved
16
+ // from node_modules if the project happens to depend on the pack.
17
+ // 3. Neither: fall back to `npx qa-engineer`, which fetches the published package
18
+ // on first use and is served from the npm cache afterwards.
19
+ //
20
+ // Because this file is committed rather than generated, path 2 works at all — which
21
+ // is what makes the pack installable by the wider Agent Skills ecosystem. The
22
+ // command a skill runs never changes; only where the engine came from does.
23
+ //
24
+ // Exit codes pass through unchanged: 0 success, 1 an invalid contract, 2 unreadable
25
+ // input or bad usage.
26
+ //
27
+ // No shebang: the synced copies carry a provenance marker on line one, which would
28
+ // sit above it and stop the kernel seeing it anyway. Every documented invocation is
29
+ // `node qa-tool.mjs …`, which needs none.
30
+
31
+ import fs from 'node:fs';
32
+ import path from 'node:path';
33
+ import { spawnSync } from 'node:child_process';
34
+ import { fileURLToPath } from 'node:url';
35
+
36
+ const here = path.dirname(fileURLToPath(import.meta.url));
37
+
38
+ const USAGE = `usage: node qa-tool.mjs <tool> <subcommand> [args]
39
+
40
+ analysis parse artifacts, classify errors, validate contracts, diff-guard,
41
+ read .qa/context.md, render an HTML report, print the footer
42
+ diagnostics root cause, timeline, priority, repair plans, release readiness
43
+ playwright normalize a Playwright report or summarize a trace
44
+
45
+ --where print how the engine was resolved, and stop
46
+
47
+ examples:
48
+ node qa-tool.mjs analysis junit test-results/results.xml
49
+ node qa-tool.mjs analysis report-html qa-artifacts/explore-result.json --out report.html
50
+ node qa-tool.mjs diagnostics report --execution-result qa-artifacts/run.json
51
+ `;
52
+
53
+ /**
54
+ * Where the engine is, and how we found it.
55
+ *
56
+ * Ordered by cost: a bundled copy needs no resolution and no network, a
57
+ * node_modules copy needs no network, and npx needs both on first use. Reporting
58
+ * *which* one answered matters when a skill degrades — "the engine is missing" and
59
+ * "the engine is being fetched" are different problems.
60
+ */
61
+ function resolveEngine() {
62
+ const bundled = path.join(here, 'lib', 'bin', 'qa-engine.mjs');
63
+ if (fs.existsSync(bundled)) return { kind: 'bundled', command: [process.execPath, bundled] };
64
+
65
+ for (const base of candidateModuleRoots()) {
66
+ const installed = path.join(base, 'qa-engineer', 'packages', 'engine', 'bin', 'qa-engine.mjs');
67
+ if (fs.existsSync(installed)) {
68
+ return { kind: 'node_modules', command: [process.execPath, installed] };
69
+ }
70
+ }
71
+
72
+ return {
73
+ kind: 'npx',
74
+ command: [npxCommand(), '--yes', 'qa-engineer', 'engine'],
75
+ };
76
+ }
77
+
78
+ /** node_modules directories worth checking, nearest first. */
79
+ function candidateModuleRoots() {
80
+ const roots = [];
81
+ let dir = here;
82
+ for (let depth = 0; depth < 12; depth += 1) {
83
+ roots.push(path.join(dir, 'node_modules'));
84
+ const parent = path.dirname(dir);
85
+ if (parent === dir) break;
86
+ dir = parent;
87
+ }
88
+ if (process.cwd() !== here) roots.push(path.join(process.cwd(), 'node_modules'));
89
+ return roots;
90
+ }
91
+
92
+ // `npx` is a shell script on POSIX and a .cmd shim on Windows; spawnSync needs the
93
+ // exact name, and `shell: true` would put user-supplied arguments through a shell.
94
+ function npxCommand() {
95
+ return process.platform === 'win32' ? 'npx.cmd' : 'npx';
96
+ }
97
+
98
+ function main(argv) {
99
+ if (argv.length === 0 || argv[0] === '--help' || argv[0] === '-h') {
100
+ process.stdout.write(USAGE);
101
+ return argv.length === 0 ? 2 : 0;
102
+ }
103
+
104
+ const engine = resolveEngine();
105
+
106
+ if (argv[0] === '--where') {
107
+ process.stdout.write(`${JSON.stringify({ resolved: engine.kind, command: engine.command }, null, 2)}\n`);
108
+ return 0;
109
+ }
110
+
111
+ const [program, ...prefix] = engine.command;
112
+ const run = spawnSync(program, [...prefix, ...argv], { stdio: 'inherit' });
113
+
114
+ if (run.error) {
115
+ // Say which path was tried and what to do, because a skill's fallback prose
116
+ // cannot diagnose this and the user is the one who has to fix it.
117
+ const advice = engine.kind === 'npx'
118
+ ? 'the engine is not bundled and npx is unavailable — run `npx qa-engineer install` in this project, or install Node 18+'
119
+ : `could not execute ${program}`;
120
+ process.stderr.write(`qa-tool: ${advice}\n${run.error.message}\n`);
121
+ return 2;
122
+ }
123
+ return run.status ?? 2;
124
+ }
125
+
126
+ process.exitCode = main(process.argv.slice(2));
@@ -41,7 +41,7 @@ Do not investigate failures here (that is `/qa-debug`) or plan repairs (`/qa-fix
41
41
  2. **Aggregate deterministically.** Run the engine's summarize step (see Tooling) to compute the test totals, the by-classification breakdown, the ranked findings and recommendations, and the release-readiness verdict.
42
42
  3. **Frame for audiences.** Write the executive summary (shippability in a paragraph and a verdict) and the engineering summary (what broke, why, who owns it, in priority order) from the same aggregated data.
43
43
  4. **Detail.** Fill the test, failure, coverage, and risk sections; mark coverage unavailable rather than inventing it when there is no coverage data.
44
- 5. **Render.** Produce the report in Markdown, an HTML-ready structure, and the JSON result the same content in three forms. Append the rendered attribution footer (see Tooling) to the Markdown and HTML renderings only; the JSON result is an interface and carries no footer.
44
+ 5. **Render.** Write the JSON result first and validate it; the Markdown and HTML are renderings of it, not parallel drafts. Render the HTML with the report renderer (see Tooling) rather than typing it. Append the rendered attribution footer to the Markdown; the renderer embeds it in the HTML, and the JSON result is an interface and carries no footer.
45
45
  6. **Report.** Emit the report result and present the Markdown.
46
46
 
47
47
  ## Guardrails
@@ -54,16 +54,17 @@ Do not investigate failures here (that is `/qa-debug`) or plan repairs (`/qa-fix
54
54
 
55
55
  ## Tooling
56
56
 
57
- Invoke the bundled engine through its launcher, as documented in [references/deterministic-tooling.md](references/deterministic-tooling.md). `SKILL_DIR` below is this skill's own directory — `.agents/skills/qa-report` or `.claude/skills/qa-report`, whichever exists. The command shape is the same in bash, zsh, PowerShell, and cmd.exe; on Windows use `python` if `python3` is not on PATH.
57
+ Invoke the bundled engine through its launcher, as documented in [references/deterministic-tooling.md](references/deterministic-tooling.md). `SKILL_DIR` below is this skill's own directory — `.agents/skills/qa-report` or `.claude/skills/qa-report`, whichever exists. The command shape is the same in bash, zsh, PowerShell, and cmd.exe, and it runs under the same Node that installed the pack — there is no second runtime to find.
58
58
 
59
59
  | Tool | Invocation | Output | Fallback |
60
60
  | --- | --- | --- | --- |
61
- | Report aggregator | `python3 <SKILL_DIR>/scripts/qa_tool.py diagnostics summarize --execution-result <path> --diagnosis <path>` | Totals, by-classification breakdown, top-priority findings, release-readiness verdict | Aggregate the structured results manually per the report-aggregation module and mark the report degraded |
62
- | One-shot pipeline | `python3 <SKILL_DIR>/scripts/qa_tool.py diagnostics report --execution-result <path>` | Diagnosis, plans, and summary in a single call when no diagnosis exists yet | Run `diagnose` then `summarize` separately |
63
- | Contract self-check | `python3 <SKILL_DIR>/scripts/qa_tool.py analysis validate <report.json> <schema.json>` | `{valid, errors}` before the report is declared complete | None: an unvalidated report is not complete |
64
- | Attribution footer | `python3 <SKILL_DIR>/scripts/qa_tool.py analysis branding --format markdown` (or `html`) | The exact footer bytes to append to the **rendered** report | Omit the footer; never retype it |
61
+ | Report aggregator | `node <SKILL_DIR>/scripts/qa-tool.mjs diagnostics summarize --execution-result <path> --diagnosis <path>` | Totals, by-classification breakdown, top-priority findings, release-readiness verdict | Aggregate the structured results manually per the report-aggregation module and mark the report degraded |
62
+ | One-shot pipeline | `node <SKILL_DIR>/scripts/qa-tool.mjs diagnostics report --execution-result <path>` | Diagnosis, plans, and summary in a single call when no diagnosis exists yet | Run `diagnose` then `summarize` separately |
63
+ | Contract self-check | `node <SKILL_DIR>/scripts/qa-tool.mjs analysis validate <report.json> <schema.json>` | `{valid, errors}` before the report is declared complete | None: an unvalidated report is not complete |
64
+ | HTML report renderer | `node <SKILL_DIR>/scripts/qa-tool.mjs analysis report-html <report-result.json> --out report.html` | The self-contained HTML rendering of the validated result, footer included | Write the HTML by hand from the contract fields and say it was not machine-rendered |
65
+ | Attribution footer | `node <SKILL_DIR>/scripts/qa-tool.mjs analysis branding --format markdown` | The exact footer bytes to append to the **Markdown** rendering (`report-html` already embeds it) | Omit the footer; never retype it |
65
66
 
66
- A missing `qa_tool.py` means the engine is not installed.
67
+ A missing `qa-tool.mjs` means the engine is not installed.
67
68
 
68
69
  The release verdict is the engine's, not a judgment call: `releaseReadiness` comes from `summarize`. The contract rejects `ready` over any failing test, so a green verdict must be backed by zero failures.
69
70
 
@@ -17,7 +17,7 @@ resolves its own location. There is nothing to set up and no shell features are
17
17
  involved:
18
18
 
19
19
  ```bash
20
- python3 <skill-dir>/scripts/qa_tool.py <tool> <subcommand> [args]
20
+ node <skill-dir>/scripts/qa-tool.mjs <tool> <subcommand> [args]
21
21
  ```
22
22
 
23
23
  `<skill-dir>` is wherever the host installed this skill — usually
@@ -25,19 +25,21 @@ python3 <skill-dir>/scripts/qa_tool.py <tool> <subcommand> [args]
25
25
  whichever exists.
26
26
 
27
27
  ```bash
28
- python3 .agents/skills/qa-run/scripts/qa_tool.py analysis junit test-results/results.xml
28
+ node .agents/skills/qa-run/scripts/qa-tool.mjs analysis junit test-results/results.xml
29
29
  ```
30
30
 
31
- That line is identical in bash, zsh, PowerShell, and cmd.exe. **On Windows, use
32
- `python` if `python3` is not on PATH** that is the only platform difference.
31
+ That line is identical in bash, zsh, PowerShell, and cmd.exe, with no platform
32
+ difference at all: it runs under the same Node the user already had to install the
33
+ pack.
33
34
 
34
- An earlier version of this contract used a shell recipe
35
- (`QA_LIB="$(ls -d … | head -1)"` with a `PYTHONPATH=` prefix). It was POSIX-only,
36
- so on Windows every deterministic call failed, each skill fell back to its manual
37
- path, and the user silently got guesswork while believing the tooling had run.
38
- Never reintroduce a shell-dependent invocation.
35
+ Two earlier versions of this contract failed the same way. The first was a shell
36
+ recipe (`QA_LIB="$(ls -d … | head -1)"` with a `PYTHONPATH=` prefix), POSIX-only, so
37
+ on Windows every deterministic call failed and each skill fell back to guesswork
38
+ while appearing to have run its tooling. The second was portable but needed a Python
39
+ interpreter the user never agreed to install. Never reintroduce a shell-dependent
40
+ invocation, and never add a runtime the install did not already require.
39
41
 
40
- If `qa_tool.py` is missing, the engine is not installed: say so, recommend
42
+ If `qa-tool.mjs` is missing, the engine is not installed: say so, recommend
41
43
  `qa repair`, use the skill's documented fallback, and mark the result degraded.
42
44
 
43
45
  Every tool writes JSON to stdout. Exit `0` means success; exit `1` means an
@@ -45,33 +47,35 @@ invalid contract; exit `2` means unreadable input, a malformed artifact, or a
45
47
  payload that failed its seam contract, and the JSON body carries `error` and
46
48
  `detail`. Treat a non-zero exit as missing evidence, never as a value to guess.
47
49
 
48
- Standard-library Python 3.8+ only — nothing to install.
50
+ Dependency-free Node — nothing to install beyond the Node that ran `npx`.
49
51
 
50
- ## 2. Analysis core — `qa_tool.py analysis`
52
+ ## 2. Analysis core — `qa-tool.mjs analysis`
51
53
 
52
54
  Framework-agnostic parsing, redaction, and validation.
53
55
 
54
56
  | Subcommand | Invocation | Returns |
55
57
  | --- | --- | --- |
56
- | `junit` | `python3 <skill-dir>/scripts/qa_tool.py analysis junit <report.xml>` | `{tests: {...}, executed: [...]}` normalized counts and per-test outcomes |
57
- | `har` | `python3 <skill-dir>/scripts/qa_tool.py analysis har <file.har> [--slow-ms N]` | Redacted request/response summary, failures, slow calls |
58
- | `discover` | `python3 <skill-dir>/scripts/qa_tool.py analysis discover [--root DIR] [--path P]` | Artifacts found, by type, with presence flags |
59
- | `diff-guard` | `python3 <skill-dir>/scripts/qa_tool.py analysis diff-guard <diff-file>` | `{issues: [...], safe: bool}` — `safe:false` blocks the change |
60
- | `redact` | `python3 <skill-dir>/scripts/qa_tool.py analysis redact <file>` | The file's text with credentials masked |
61
- | `validate` | `python3 <skill-dir>/scripts/qa_tool.py analysis validate <instance.json> <schema.json>` | `{valid: bool, errors: [...]}`; exit 1 when invalid |
62
- | `classify` | `python3 <skill-dir>/scripts/qa_tool.py analysis classify "<error message>" [--http-status N]` | `{classification, confidence, reason}` from the shared taxonomy |
63
- | `context` | `python3 <skill-dir>/scripts/qa_tool.py analysis context [--root DIR] [--path .qa/context.md]` | The parsed, schema-validated project context as JSON |
64
-
65
- ## 3. Diagnostic engine `qa_tool.py diagnostics`
58
+ | `junit` | `node <skill-dir>/scripts/qa-tool.mjs analysis junit <report.xml>` | `{tests: {...}, executed: [...]}` normalized counts and per-test outcomes |
59
+ | `har` | `node <skill-dir>/scripts/qa-tool.mjs analysis har <file.har> [--slow-ms N]` | Redacted request/response summary, failures, slow calls |
60
+ | `discover` | `node <skill-dir>/scripts/qa-tool.mjs analysis discover [--root DIR] [--path P]` | Artifacts found, by type, with presence flags |
61
+ | `diff-guard` | `node <skill-dir>/scripts/qa-tool.mjs analysis diff-guard <diff-file>` | `{issues: [...], safe: bool}` — `safe:false` blocks the change |
62
+ | `redact` | `node <skill-dir>/scripts/qa-tool.mjs analysis redact <file>` | The file's text with credentials masked |
63
+ | `validate` | `node <skill-dir>/scripts/qa-tool.mjs analysis validate <instance.json> <schema.json>` | `{valid: bool, errors: [...]}`; exit 1 when invalid |
64
+ | `classify` | `node <skill-dir>/scripts/qa-tool.mjs analysis classify "<error message>" [--http-status N]` | `{classification, confidence, reason}` from the shared taxonomy |
65
+ | `context` | `node <skill-dir>/scripts/qa-tool.mjs analysis context [--root DIR] [--path .qa/context.md]` | The parsed, schema-validated project context as JSON |
66
+ | `report-html` | `node <skill-dir>/scripts/qa-tool.mjs analysis report-html <result.json> [--out report.html]` | The result rendered as one self-contained HTML report — every required field, footer included. Run it instead of writing HTML |
67
+ | `branding` | `node <skill-dir>/scripts/qa-tool.mjs analysis branding --format markdown\|html\|text` | The exact attribution footer bytes for a **rendered** report (never for a JSON artifact) |
68
+
69
+ ## 3. Diagnostic engine — `qa-tool.mjs diagnostics`
66
70
 
67
71
  One engine, consumed by the diagnostic skills. Reasoning lives here once.
68
72
 
69
73
  | Subcommand | Invocation | Returns |
70
74
  | --- | --- | --- |
71
- | `diagnose` | `python3 <skill-dir>/scripts/qa_tool.py diagnostics diagnose --execution-result <path> [--analysis-result <path>]` | `{entries: [...], timeline: [...], recommendations: [...]}` |
72
- | `plan-repairs` | `python3 <skill-dir>/scripts/qa_tool.py diagnostics plan-repairs --diagnosis <path>` | `{plans: [...]}` — one plan per entry, escalations included |
73
- | `summarize` | `python3 <skill-dir>/scripts/qa_tool.py diagnostics summarize --execution-result <path> --diagnosis <path>` | `{totals, byClassification, topPriority, releaseReadiness}` |
74
- | `report` | `python3 <skill-dir>/scripts/qa_tool.py diagnostics report --execution-result <path> [--analysis-result <path>]` | `{diagnosis, plans, summary}` — all three in one call |
75
+ | `diagnose` | `node <skill-dir>/scripts/qa-tool.mjs diagnostics diagnose --execution-result <path> [--analysis-result <path>]` | `{entries: [...], timeline: [...], recommendations: [...]}` |
76
+ | `plan-repairs` | `node <skill-dir>/scripts/qa-tool.mjs diagnostics plan-repairs --diagnosis <path>` | `{plans: [...]}` — one plan per entry, escalations included |
77
+ | `summarize` | `node <skill-dir>/scripts/qa-tool.mjs diagnostics summarize --execution-result <path> --diagnosis <path>` | `{totals, byClassification, topPriority, releaseReadiness}` |
78
+ | `report` | `node <skill-dir>/scripts/qa-tool.mjs diagnostics report --execution-result <path> [--analysis-result <path>]` | `{diagnosis, plans, summary}` — all three in one call |
75
79
 
76
80
  **Inputs.** `--execution-result` takes a `qa-run` execution result, or the minimal
77
81
  subset (`tests` counts plus `executed[]` entries carrying `status`).
@@ -99,7 +103,7 @@ contract names:
99
103
 
100
104
  This mapping is not busywork: the strictness is what stops a skill from shipping a
101
105
  result whose shape nobody checked. Validate before completion —
102
- `python3 <skill-dir>/scripts/qa_tool.py analysis validate <result.json> <schema.json>` — and fix the
106
+ `node <skill-dir>/scripts/qa-tool.mjs analysis validate <result.json> <schema.json>` — and fix the
103
107
  result, never the claim.
104
108
 
105
109
  ## 4. Framework adapters
@@ -109,11 +113,11 @@ a `--framework` flag.
109
113
 
110
114
  | Adapter | Invocation | Returns |
111
115
  | --- | --- | --- |
112
- | Playwright report | `python3 <skill-dir>/scripts/qa_tool.py playwright report <results.json>` | The same `{tests, executed}` shape as `junit` |
113
- | Playwright trace | `python3 <skill-dir>/scripts/qa_tool.py playwright trace <trace.zip>` | Actions, console/network counts, errors, classification |
116
+ | Playwright report | `node <skill-dir>/scripts/qa-tool.mjs playwright report <results.json>` | The same `{tests, executed}` shape as `junit` |
117
+ | Playwright trace | `node <skill-dir>/scripts/qa-tool.mjs playwright trace <trace.zip>` | Actions, console/network counts, errors, classification |
114
118
 
115
119
  For Selenium, Cypress, and WebdriverIO, normalize through
116
- `qa_tool.py analysis junit` — those adapters have no richer artifact than JUnit, and
120
+ `qa-tool.mjs analysis junit` — those adapters have no richer artifact than JUnit, and
117
121
  the skill says so rather than implying trace-grade depth.
118
122
 
119
123
  ## 5. Reporting what ran
@@ -5,11 +5,11 @@ How the one engine composes the other platforms into a diagnosis, and what lives
5
5
 
6
6
  ## What the engine does, in code
7
7
 
8
- The `qa_diagnostics` package (in lib/) implements the deterministic reasoning, reusing `qa_analysis`:
8
+ The `diagnostics` package (in lib/) implements the deterministic reasoning, reusing `analysis`:
9
9
 
10
10
  | Step | Module | Reuses |
11
11
  | --- | --- | --- |
12
- | Classify the failure into a root cause | `root_cause` | `qa_analysis.taxonomy` |
12
+ | Classify the failure into a root cause | `root_cause` | `the shared taxonomy` |
13
13
  | Assign severity, priority, impacts, owner, effort | `prioritization` | the taxonomy classes |
14
14
  | Reconstruct the ordered timeline | `timeline` | the evidence in findings |
15
15
  | Plan a repair (never code) | `repair` | the taxonomy; the diff guard's guarantees |
@@ -33,4 +33,4 @@ The engine calls the analysis platform for classification and evidence, the exec
33
33
 
34
34
  ## Bundling
35
35
 
36
- Because the diagnostic skills run the engine in a consumer's repository, the `qa_analysis` and `qa_diagnostics` packages are bundled into each skill's `scripts/lib/` from their canonical source in `shared/`. The bundle is a build artifact produced by the bundler; the source of truth is `shared/`. See ADR-0011.
36
+ Because the diagnostic skills run the engine in a consumer's repository, the `analysis` and `diagnostics` packages are bundled into each skill's `scripts/lib/` from their canonical source in `shared/`. The bundle is a build artifact produced by the bundler; the source of truth is `shared/`. See ADR-0011.
@@ -27,17 +27,42 @@ Any skill that reaches a conclusion — a detected fact, a chosen strategy, a cl
27
27
  | Confidence | Calibrated per rule 4; omitted rather than invented when the skill did not weigh alternatives |
28
28
  | Recommendations | The next action, named concretely — often the next command and the artifact to feed it |
29
29
 
30
+ ## Rendering the HTML report
31
+
32
+ **Never type the HTML.** When a skill's result conforms to a contract the renderer
33
+ supports, the HTML report is generated from that artifact:
34
+
35
+ ```bash
36
+ node <SKILL_DIR>/scripts/qa-tool.mjs analysis report-html <result.json> --out <report.html>
37
+ ```
38
+
39
+ The reason is not tidiness. A hand-written report is a second, lossy copy of the
40
+ artifact, and it loses exactly what the reader needs: the first live `/qa-explore`
41
+ run wrote findings whose `actual`, `expected`, and `fixDirection` were all required,
42
+ all present in the JSON, and all absent from the page — collapsed into one sentence
43
+ that left the reader to guess what correct behaviour would have been. The renderer
44
+ cannot make that mistake, because the fields are in its template.
45
+
46
+ What the renderer guarantees per finding: severity, the defect, **what happens
47
+ now**, **what should happen instead**, how to reproduce it, the fix direction, and
48
+ every evidence entry — plus the attribution footer, in one self-contained file with
49
+ no external assets.
50
+
51
+ Run `qa-tool.mjs analysis report-html --help` for the supported contracts. For an
52
+ artifact it does not support, write the HTML by hand from the contract's fields —
53
+ and render **every** field a reader needs, in that order.
54
+
30
55
  ## Attribution on rendered reports
31
56
 
32
57
  A report a person opens carries a product attribution footer, the way a Lighthouse
33
58
  or Allure report does. It is rendered, never typed: the exact bytes come from the
34
59
  bundled analysis toolkit, so every report is identical and a wording change is a
35
- one-file edit.
60
+ one-file edit. `report-html` already embeds it; for any other rendering, ask for it:
36
61
 
37
62
  ```bash
38
- PYTHONPATH="$QA_LIB" python3 -m qa_analysis.cli branding --format html
39
- PYTHONPATH="$QA_LIB" python3 -m qa_analysis.cli branding --format markdown
40
- PYTHONPATH="$QA_LIB" python3 -m qa_analysis.cli branding --format text
63
+ node <SKILL_DIR>/scripts/qa-tool.mjs analysis branding --format markdown
64
+ node <SKILL_DIR>/scripts/qa-tool.mjs analysis branding --format html
65
+ node <SKILL_DIR>/scripts/qa-tool.mjs analysis branding --format text
41
66
  ```
42
67
 
43
68
  Append the output as the last element of the rendered document: `html` inside
@@ -0,0 +1,126 @@
1
+ // synced-from: shared/tooling/qa-tool.mjs — do not edit; edit the source and run: node scripts/sync-shared.mjs --write
2
+ // The launcher every skill invokes to reach the deterministic engine.
3
+ //
4
+ // A skill's SKILL.md documents exactly one command shape:
5
+ //
6
+ // node <SKILL_DIR>/scripts/qa-tool.mjs <tool> <subcommand> [args]
7
+ //
8
+ // and this file finds the engine, wherever it happens to be. That indirection
9
+ // exists because the pack is installed three different ways and the engine lands in
10
+ // a different place each time:
11
+ //
12
+ // 1. `qa install` bundles the engine into the skill, at ./lib/. Offline, fastest,
13
+ // and pinned to the version that was installed.
14
+ // 2. `npx skills add <owner>/<repo>` — or any generic file copier — copies the
15
+ // skill directory out of git and bundles nothing. The engine is then resolved
16
+ // from node_modules if the project happens to depend on the pack.
17
+ // 3. Neither: fall back to `npx qa-engineer`, which fetches the published package
18
+ // on first use and is served from the npm cache afterwards.
19
+ //
20
+ // Because this file is committed rather than generated, path 2 works at all — which
21
+ // is what makes the pack installable by the wider Agent Skills ecosystem. The
22
+ // command a skill runs never changes; only where the engine came from does.
23
+ //
24
+ // Exit codes pass through unchanged: 0 success, 1 an invalid contract, 2 unreadable
25
+ // input or bad usage.
26
+ //
27
+ // No shebang: the synced copies carry a provenance marker on line one, which would
28
+ // sit above it and stop the kernel seeing it anyway. Every documented invocation is
29
+ // `node qa-tool.mjs …`, which needs none.
30
+
31
+ import fs from 'node:fs';
32
+ import path from 'node:path';
33
+ import { spawnSync } from 'node:child_process';
34
+ import { fileURLToPath } from 'node:url';
35
+
36
+ const here = path.dirname(fileURLToPath(import.meta.url));
37
+
38
+ const USAGE = `usage: node qa-tool.mjs <tool> <subcommand> [args]
39
+
40
+ analysis parse artifacts, classify errors, validate contracts, diff-guard,
41
+ read .qa/context.md, render an HTML report, print the footer
42
+ diagnostics root cause, timeline, priority, repair plans, release readiness
43
+ playwright normalize a Playwright report or summarize a trace
44
+
45
+ --where print how the engine was resolved, and stop
46
+
47
+ examples:
48
+ node qa-tool.mjs analysis junit test-results/results.xml
49
+ node qa-tool.mjs analysis report-html qa-artifacts/explore-result.json --out report.html
50
+ node qa-tool.mjs diagnostics report --execution-result qa-artifacts/run.json
51
+ `;
52
+
53
+ /**
54
+ * Where the engine is, and how we found it.
55
+ *
56
+ * Ordered by cost: a bundled copy needs no resolution and no network, a
57
+ * node_modules copy needs no network, and npx needs both on first use. Reporting
58
+ * *which* one answered matters when a skill degrades — "the engine is missing" and
59
+ * "the engine is being fetched" are different problems.
60
+ */
61
+ function resolveEngine() {
62
+ const bundled = path.join(here, 'lib', 'bin', 'qa-engine.mjs');
63
+ if (fs.existsSync(bundled)) return { kind: 'bundled', command: [process.execPath, bundled] };
64
+
65
+ for (const base of candidateModuleRoots()) {
66
+ const installed = path.join(base, 'qa-engineer', 'packages', 'engine', 'bin', 'qa-engine.mjs');
67
+ if (fs.existsSync(installed)) {
68
+ return { kind: 'node_modules', command: [process.execPath, installed] };
69
+ }
70
+ }
71
+
72
+ return {
73
+ kind: 'npx',
74
+ command: [npxCommand(), '--yes', 'qa-engineer', 'engine'],
75
+ };
76
+ }
77
+
78
+ /** node_modules directories worth checking, nearest first. */
79
+ function candidateModuleRoots() {
80
+ const roots = [];
81
+ let dir = here;
82
+ for (let depth = 0; depth < 12; depth += 1) {
83
+ roots.push(path.join(dir, 'node_modules'));
84
+ const parent = path.dirname(dir);
85
+ if (parent === dir) break;
86
+ dir = parent;
87
+ }
88
+ if (process.cwd() !== here) roots.push(path.join(process.cwd(), 'node_modules'));
89
+ return roots;
90
+ }
91
+
92
+ // `npx` is a shell script on POSIX and a .cmd shim on Windows; spawnSync needs the
93
+ // exact name, and `shell: true` would put user-supplied arguments through a shell.
94
+ function npxCommand() {
95
+ return process.platform === 'win32' ? 'npx.cmd' : 'npx';
96
+ }
97
+
98
+ function main(argv) {
99
+ if (argv.length === 0 || argv[0] === '--help' || argv[0] === '-h') {
100
+ process.stdout.write(USAGE);
101
+ return argv.length === 0 ? 2 : 0;
102
+ }
103
+
104
+ const engine = resolveEngine();
105
+
106
+ if (argv[0] === '--where') {
107
+ process.stdout.write(`${JSON.stringify({ resolved: engine.kind, command: engine.command }, null, 2)}\n`);
108
+ return 0;
109
+ }
110
+
111
+ const [program, ...prefix] = engine.command;
112
+ const run = spawnSync(program, [...prefix, ...argv], { stdio: 'inherit' });
113
+
114
+ if (run.error) {
115
+ // Say which path was tried and what to do, because a skill's fallback prose
116
+ // cannot diagnose this and the user is the one who has to fix it.
117
+ const advice = engine.kind === 'npx'
118
+ ? 'the engine is not bundled and npx is unavailable — run `npx qa-engineer install` in this project, or install Node 18+'
119
+ : `could not execute ${program}`;
120
+ process.stderr.write(`qa-tool: ${advice}\n${run.error.message}\n`);
121
+ return 2;
122
+ }
123
+ return run.status ?? 2;
124
+ }
125
+
126
+ process.exitCode = main(process.argv.slice(2));
@@ -27,17 +27,42 @@ Any skill that reaches a conclusion — a detected fact, a chosen strategy, a cl
27
27
  | Confidence | Calibrated per rule 4; omitted rather than invented when the skill did not weigh alternatives |
28
28
  | Recommendations | The next action, named concretely — often the next command and the artifact to feed it |
29
29
 
30
+ ## Rendering the HTML report
31
+
32
+ **Never type the HTML.** When a skill's result conforms to a contract the renderer
33
+ supports, the HTML report is generated from that artifact:
34
+
35
+ ```bash
36
+ node <SKILL_DIR>/scripts/qa-tool.mjs analysis report-html <result.json> --out <report.html>
37
+ ```
38
+
39
+ The reason is not tidiness. A hand-written report is a second, lossy copy of the
40
+ artifact, and it loses exactly what the reader needs: the first live `/qa-explore`
41
+ run wrote findings whose `actual`, `expected`, and `fixDirection` were all required,
42
+ all present in the JSON, and all absent from the page — collapsed into one sentence
43
+ that left the reader to guess what correct behaviour would have been. The renderer
44
+ cannot make that mistake, because the fields are in its template.
45
+
46
+ What the renderer guarantees per finding: severity, the defect, **what happens
47
+ now**, **what should happen instead**, how to reproduce it, the fix direction, and
48
+ every evidence entry — plus the attribution footer, in one self-contained file with
49
+ no external assets.
50
+
51
+ Run `qa-tool.mjs analysis report-html --help` for the supported contracts. For an
52
+ artifact it does not support, write the HTML by hand from the contract's fields —
53
+ and render **every** field a reader needs, in that order.
54
+
30
55
  ## Attribution on rendered reports
31
56
 
32
57
  A report a person opens carries a product attribution footer, the way a Lighthouse
33
58
  or Allure report does. It is rendered, never typed: the exact bytes come from the
34
59
  bundled analysis toolkit, so every report is identical and a wording change is a
35
- one-file edit.
60
+ one-file edit. `report-html` already embeds it; for any other rendering, ask for it:
36
61
 
37
62
  ```bash
38
- PYTHONPATH="$QA_LIB" python3 -m qa_analysis.cli branding --format html
39
- PYTHONPATH="$QA_LIB" python3 -m qa_analysis.cli branding --format markdown
40
- PYTHONPATH="$QA_LIB" python3 -m qa_analysis.cli branding --format text
63
+ node <SKILL_DIR>/scripts/qa-tool.mjs analysis branding --format markdown
64
+ node <SKILL_DIR>/scripts/qa-tool.mjs analysis branding --format html
65
+ node <SKILL_DIR>/scripts/qa-tool.mjs analysis branding --format text
41
66
  ```
42
67
 
43
68
  Append the output as the last element of the rendered document: `html` inside
@@ -71,16 +71,16 @@ Follow the execution lifecycle. Perform each phase in order; stop and explain th
71
71
 
72
72
  ## Tooling
73
73
 
74
- Invoke the bundled engine through its launcher, as documented in [references/deterministic-tooling.md](references/deterministic-tooling.md). `SKILL_DIR` below is this skill's own directory — `.agents/skills/qa-run` or `.claude/skills/qa-run`, whichever exists. The command shape is the same in bash, zsh, PowerShell, and cmd.exe; on Windows use `python` if `python3` is not on PATH.
74
+ Invoke the bundled engine through its launcher, as documented in [references/deterministic-tooling.md](references/deterministic-tooling.md). `SKILL_DIR` below is this skill's own directory — `.agents/skills/qa-run` or `.claude/skills/qa-run`, whichever exists. The command shape is the same in bash, zsh, PowerShell, and cmd.exe, and it runs under the same Node that installed the pack — there is no second runtime to find.
75
75
 
76
76
  | Tool | Invocation | Output | Fallback |
77
77
  | --- | --- | --- | --- |
78
- | Playwright report normalizer | `python3 <SKILL_DIR>/scripts/qa_tool.py playwright report <results.json>` | `{tests, executed}` — the counts and per-test outcomes the result requires | None: without it, classify `errored` and say the reporter could not be normalized |
79
- | JUnit normalizer | `python3 <SKILL_DIR>/scripts/qa_tool.py analysis junit <report.xml>` | The same shape from a JUnit reporter | None: as above |
80
- | Artifact discovery | `python3 <SKILL_DIR>/scripts/qa_tool.py analysis discover --root <dir>` | Which artifacts the run produced, by type, with presence flags | List artifacts from the reporter's own paths only |
81
- | Contract self-check | `python3 <SKILL_DIR>/scripts/qa_tool.py analysis validate <result.json> <schema.json>` | `{valid, errors}` before the result is declared complete | None: an unvalidated result is not complete |
78
+ | Playwright report normalizer | `node <SKILL_DIR>/scripts/qa-tool.mjs playwright report <results.json>` | `{tests, executed}` — the counts and per-test outcomes the result requires | None: without it, classify `errored` and say the reporter could not be normalized |
79
+ | JUnit normalizer | `node <SKILL_DIR>/scripts/qa-tool.mjs analysis junit <report.xml>` | The same shape from a JUnit reporter | None: as above |
80
+ | Artifact discovery | `node <SKILL_DIR>/scripts/qa-tool.mjs analysis discover --root <dir>` | Which artifacts the run produced, by type, with presence flags | List artifacts from the reporter's own paths only |
81
+ | Contract self-check | `node <SKILL_DIR>/scripts/qa-tool.mjs analysis validate <result.json> <schema.json>` | `{valid, errors}` before the result is declared complete | None: an unvalidated result is not complete |
82
82
 
83
- A missing `qa_tool.py` means the engine is not installed. A run whose numbers could not be normalized is never reported `passed`.
83
+ A missing `qa-tool.mjs` means the engine is not installed. A run whose numbers could not be normalized is never reported `passed`.
84
84
 
85
85
  ## Output
86
86