qa-engineer 0.10.0 → 0.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +167 -59
- package/package.json +1 -1
- package/packages/engine/bin/qa-engine.mjs +232 -8
- package/packages/engine/lib/analysis/har.mjs +34 -0
- package/packages/engine/lib/analysis/network.mjs +237 -0
- package/packages/engine/lib/analysis/report-html.mjs +47 -734
- package/packages/engine/lib/artifacts/manager.mjs +453 -0
- package/packages/engine/lib/artifacts/mime.mjs +109 -0
- package/packages/engine/lib/artifacts/zip-write.mjs +143 -0
- package/packages/engine/lib/report/components/charts.mjs +424 -0
- package/packages/engine/lib/report/components/evidence.mjs +207 -0
- package/packages/engine/lib/report/components/findings.mjs +258 -0
- package/packages/engine/lib/report/components/nav.mjs +99 -0
- package/packages/engine/lib/report/components/primitives.mjs +246 -0
- package/packages/engine/lib/report/components/runtime.mjs +246 -0
- package/packages/engine/lib/report/components/timeline.mjs +65 -0
- package/packages/engine/lib/report/core/model.mjs +270 -0
- package/packages/engine/lib/report/core/normalize.mjs +226 -0
- package/packages/engine/lib/report/core/sections.mjs +978 -0
- package/packages/engine/lib/report/export/bundle.mjs +293 -0
- package/packages/engine/lib/report/export/html.mjs +183 -0
- package/packages/engine/lib/report/export/machine.mjs +290 -0
- package/packages/engine/lib/report/export/markdown.mjs +323 -0
- package/packages/engine/lib/report/schemas/qa-report.schema.json +555 -0
- package/packages/engine/lib/report/theme/css.mjs +529 -0
- package/packages/engine/lib/report/theme/tokens.mjs +137 -0
- package/packages/engine/lib/report/version.mjs +78 -0
- package/packages/engine/package.json +2 -2
- package/packages/installer/lib/agents/targets.mjs +90 -0
- package/packages/installer/lib/agents/user-level.mjs +80 -0
- package/packages/installer/lib/cli/flags.mjs +20 -2
- package/packages/installer/lib/commands/doctor.mjs +6 -4
- package/packages/installer/lib/commands/install.mjs +134 -93
- package/packages/installer/lib/commands/repair.mjs +10 -6
- package/packages/installer/lib/commands/self-test.mjs +4 -3
- package/packages/installer/lib/commands/uninstall.mjs +14 -8
- package/packages/installer/lib/commands/update.mjs +9 -4
- package/packages/installer/lib/commands/verify.mjs +13 -12
- package/packages/installer/lib/constants.mjs +13 -0
- package/packages/installer/lib/core/conflict.mjs +5 -4
- package/packages/installer/lib/core/fs-safe.mjs +146 -6
- package/packages/installer/lib/core/integrity.mjs +59 -0
- package/packages/installer/lib/core/lockfile.mjs +19 -3
- package/packages/installer/lib/core/plan.mjs +213 -0
- package/packages/installer/lib/core/qa-home.mjs +145 -0
- package/packages/installer/lib/core/scope.mjs +274 -0
- package/packages/installer/lib/core/validate-install.mjs +37 -12
- package/packages/installer/package.json +1 -1
- package/packages/installer/schemas/qa-lock.schema.json +119 -21
- package/shared/tooling/qa-tool.mjs +41 -5
- package/skills/qa-api/scripts/qa-tool.mjs +41 -5
- package/skills/qa-audit/scripts/qa-tool.mjs +41 -5
- package/skills/qa-debug/SKILL.md +3 -0
- package/skills/qa-debug/references/failure-handoff.md +44 -0
- package/skills/qa-debug/scripts/qa-tool.mjs +41 -5
- package/skills/qa-explore/SKILL.md +26 -10
- package/skills/qa-explore/contracts/explore-result.schema.json +517 -11
- package/skills/qa-explore/references/api-replay.md +40 -1
- package/skills/qa-explore/references/report-pipeline.md +265 -95
- package/skills/qa-explore/scripts/qa-tool.mjs +41 -5
- package/skills/qa-fix/scripts/qa-tool.mjs +41 -5
- package/skills/qa-flaky/scripts/qa-tool.mjs +41 -5
- package/skills/qa-init/scripts/qa-tool.mjs +41 -5
- package/skills/qa-report/scripts/qa-tool.mjs +41 -5
- package/skills/qa-run/README.md +7 -0
- package/skills/qa-run/SKILL.md +12 -7
- package/skills/qa-run/contracts/execution-result.schema.json +77 -0
- package/skills/qa-run/examples/execute-playwright.md +42 -10
- package/skills/qa-run/examples/plan-a-run.md +5 -3
- package/skills/qa-run/references/artifact-collector.md +2 -1
- package/skills/qa-run/references/command-builder.md +18 -1
- package/skills/qa-run/references/execution-strategy.md +2 -0
- package/skills/qa-run/references/failure-handoff.md +44 -0
- package/skills/qa-run/references/playwright-artifacts.md +3 -1
- package/skills/qa-run/references/playwright-execution.md +14 -1
- package/skills/qa-run/scripts/qa-tool.mjs +41 -5
package/README.md
CHANGED
|
@@ -2,13 +2,13 @@
|
|
|
2
2
|
|
|
3
3
|
> Teach your AI coding assistant to work like a senior QA engineer — and stop it from telling you tests pass when they don't.
|
|
4
4
|
|
|
5
|
-
[](CHANGELOG.md)
|
|
6
6
|
[](docs/release/v0.9-release-checklist.md)
|
|
7
7
|
[](LICENSE)
|
|
8
8
|
[](#step-1--check-your-prerequisite)
|
|
9
9
|
[](https://agentskills.io)
|
|
10
10
|
[](https://github.com/abisheik88/qa-engineer/actions/workflows/ci.yml)
|
|
11
|
-
[](#how-this-is-verified)
|
|
12
12
|
|
|
13
13
|
**This page is everything you need.** Install, first run, every command, and what to do when something breaks — no other document required.
|
|
14
14
|
|
|
@@ -45,13 +45,13 @@ If you use an **AI coding assistant** — Claude Code, Cursor, GitHub Copilot, C
|
|
|
45
45
|
|
|
46
46
|
That last one is the dangerous part. A green test suite that checks nothing is worse than a red one, because you stop looking.
|
|
47
47
|
|
|
48
|
-
**This project installs
|
|
48
|
+
**This project installs thirteen QA skills** — twelve commands plus a worked example — that your AI assistant reads and follows. Think of it as handing your assistant a senior QA engineer's playbook, plus a set of tools it must actually run — so its answers come from real measurements instead of guesses.
|
|
49
49
|
|
|
50
50
|
<details>
|
|
51
51
|
<summary><b>New to this? Three terms explained</b></summary>
|
|
52
52
|
|
|
53
53
|
- **AI coding assistant / agent** — a tool where you chat with an AI inside your codebase, and it can read and edit files. Claude Code, Cursor, and GitHub Copilot are examples.
|
|
54
|
-
- **Skill** — a Markdown file with instructions your assistant reads when a task matches. It's like a checklist the AI follows. This project installs
|
|
54
|
+
- **Skill** — a Markdown file with instructions your assistant reads when a task matches. It's like a checklist the AI follows. This project installs thirteen of them — twelve commands you use, plus one worked example.
|
|
55
55
|
- **Slash command** — how you trigger a skill: you type `/qa-run` in your assistant's chat. If your assistant doesn't support slash commands, plain English works too ("run my tests and report the result").
|
|
56
56
|
|
|
57
57
|
You do not need to learn a new language, framework, or config file. You install once and then talk to your assistant normally.
|
|
@@ -84,7 +84,7 @@ that runs on the same Node you just checked.
|
|
|
84
84
|
<details>
|
|
85
85
|
<summary><b>"command not found" — how to install Node.js</b></summary>
|
|
86
86
|
|
|
87
|
-
Node.js gives you the `
|
|
87
|
+
Node.js gives you the `npm` command used to install this pack, and it runs the
|
|
88
88
|
analysis tools afterwards.
|
|
89
89
|
|
|
90
90
|
- **macOS** — `brew install node` (needs [Homebrew](https://brew.sh)), or download from [nodejs.org](https://nodejs.org).
|
|
@@ -99,69 +99,133 @@ You also need **an AI coding assistant** open on your project — Claude Code, C
|
|
|
99
99
|
|
|
100
100
|
## Step 2 — Install it
|
|
101
101
|
|
|
102
|
-
|
|
102
|
+
**Install once for your whole machine.** Every project then works with no further setup:
|
|
103
103
|
|
|
104
104
|
```bash
|
|
105
|
-
|
|
105
|
+
npm install -g qa-engineer
|
|
106
|
+
qa-engineer install --global
|
|
106
107
|
```
|
|
107
108
|
|
|
108
|
-
That's the
|
|
109
|
+
That's it. Open any project, and the QA commands are already there.
|
|
109
110
|
|
|
110
|
-
|
|
111
|
-
| --- | --- |
|
|
112
|
-
| `npx` | Comes with Node. Downloads and runs a tool without installing it permanently. |
|
|
113
|
-
| `qa-engineer` | The name of this package. |
|
|
114
|
-
| `--yes` | Don't ask me questions, just use sensible defaults. |
|
|
115
|
-
| `--project .` | Install into the current folder (`.` means "here"). |
|
|
111
|
+
What those two lines do, one at a time:
|
|
116
112
|
|
|
117
|
-
|
|
113
|
+
| Line | What it does |
|
|
114
|
+
| --- | --- |
|
|
115
|
+
| `npm install -g qa-engineer` | Puts the `qa-engineer` command on your system, so you can type it anywhere. `-g` means "global" — available everywhere, not just in one folder. |
|
|
116
|
+
| `qa-engineer install --global` | Installs the QA skills your assistant reads, once, for every project on this machine. |
|
|
118
117
|
|
|
119
|
-
|
|
118
|
+
Here is what the second command prints:
|
|
120
119
|
|
|
121
120
|
```text
|
|
122
|
-
›
|
|
123
|
-
› agents:
|
|
121
|
+
› scope: global (/Users/you/.qa-engineer)
|
|
122
|
+
› agents: claude-code, antigravity
|
|
124
123
|
› skills: 13
|
|
124
|
+
› shared engine: /Users/you/.qa-engineer/engine
|
|
125
125
|
› ██████████████████ 100% Configuration complete
|
|
126
|
-
✓ installed
|
|
126
|
+
✓ installed 214 file(s) and 26 link(s); lockfile /Users/you/.qa-engineer/qa-lock.json
|
|
127
|
+
→ in Claude Code: type /qa-explore, or just describe the task — skills auto-activate
|
|
127
128
|
```
|
|
128
129
|
|
|
130
|
+
**Everything lives in one folder, `~/.qa-engineer/`.** Nothing is scattered around your
|
|
131
|
+
home directory, so removing it later is one clean step.
|
|
132
|
+
|
|
133
|
+
> **On "not installed for cursor, codex, …"** — you may see a note listing assistants
|
|
134
|
+
> that were skipped, each with a reason. That is expected and honest: those assistants
|
|
135
|
+
> only look for skills *inside a project*, so there is no machine-wide folder to put
|
|
136
|
+
> them in. For those, use a project install (below). Claude Code and Antigravity do have
|
|
137
|
+
> a machine-wide folder, so they are set up for you.
|
|
138
|
+
|
|
129
139
|
<details>
|
|
130
|
-
<summary><b>
|
|
140
|
+
<summary><b>Installing from a clone instead</b></summary>
|
|
131
141
|
|
|
132
|
-
|
|
142
|
+
To run the newest code, or to work on the pack itself, install from a clone rather than
|
|
143
|
+
from npm:
|
|
133
144
|
|
|
134
145
|
```bash
|
|
135
|
-
git clone
|
|
146
|
+
git clone https://github.com/abisheik88/qa-engineer.git
|
|
136
147
|
cd qa-engineer
|
|
137
148
|
npm install
|
|
138
|
-
npm
|
|
149
|
+
npm install -g . # puts `qa-engineer` on your PATH
|
|
150
|
+
qa-engineer install --global
|
|
139
151
|
```
|
|
140
152
|
|
|
141
|
-
|
|
153
|
+
Everything else in this README works exactly the same afterwards. Use `npm link` in
|
|
154
|
+
place of `npm install -g .` if you are editing the pack — it symlinks, so your changes
|
|
155
|
+
take effect without reinstalling.
|
|
156
|
+
|
|
157
|
+
</details>
|
|
158
|
+
|
|
159
|
+
### Three ways to install — pick one
|
|
160
|
+
|
|
161
|
+
Most people want the first. The other two exist for real situations, not as clutter.
|
|
162
|
+
|
|
163
|
+
| Command | Installs | Choose it when |
|
|
164
|
+
| --- | --- | --- |
|
|
165
|
+
| `qa-engineer install --global` | Once per machine, in `~/.qa-engineer/` | **The usual choice.** You want every project to just work. |
|
|
166
|
+
| `qa-engineer install --workspace` | Once at the top of a monorepo | You have many packages in one repository and want them to share an install. |
|
|
167
|
+
| `qa-engineer install --project .` | Into the current project only | The install should be committed to the repository and shared with your team. |
|
|
168
|
+
|
|
169
|
+
**They can coexist.** If a project has its own install, that one is used; otherwise the
|
|
170
|
+
machine-wide one is. This is the same rule you already rely on: a package installed in
|
|
171
|
+
your project beats the same package installed globally. So a repository can pin the
|
|
172
|
+
version it wants, and nothing else on your machine changes.
|
|
173
|
+
|
|
174
|
+
<details>
|
|
175
|
+
<summary><b>What a project install is for</b></summary>
|
|
176
|
+
|
|
177
|
+
A project install is **self-contained**: the analysis engine is copied inside each
|
|
178
|
+
skill, so the repository works on a machine where nothing else is installed. That costs
|
|
179
|
+
more disk, and it is exactly what you want when the install is committed to git and a
|
|
180
|
+
colleague clones the repository.
|
|
181
|
+
|
|
182
|
+
```bash
|
|
183
|
+
cd /path/to/your-project
|
|
184
|
+
qa-engineer install --project .
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
A machine-wide install shares one copy of the engine instead — about 2 MB total,
|
|
188
|
+
compared with 13 MB per project — which is why it is the default recommendation.
|
|
189
|
+
|
|
190
|
+
</details>
|
|
191
|
+
|
|
192
|
+
<details>
|
|
193
|
+
<summary><b>Extra options you may want later</b></summary>
|
|
194
|
+
|
|
195
|
+
| Option | What it does |
|
|
196
|
+
| --- | --- |
|
|
197
|
+
| `--all-agents` | Set up every assistant with a machine-wide skills folder, not only the ones detected here. |
|
|
198
|
+
| `--yes` | Don't ask any questions; use sensible defaults. Useful in scripts and CI. |
|
|
199
|
+
| `--dry-run` | Show what would change and write nothing. |
|
|
200
|
+
| `--force` | Overwrite files a previous install doesn't own. Backs them up first. |
|
|
201
|
+
| `QA_ENGINEER_HOME` | An environment variable that moves `~/.qa-engineer/` somewhere else entirely. |
|
|
142
202
|
|
|
143
203
|
</details>
|
|
144
204
|
|
|
145
205
|
## Step 3 — Confirm it worked
|
|
146
206
|
|
|
147
207
|
```bash
|
|
148
|
-
|
|
208
|
+
qa-engineer self-test --global
|
|
149
209
|
```
|
|
150
210
|
|
|
151
|
-
Real output from a healthy install:
|
|
211
|
+
Real output from a healthy machine-wide install:
|
|
152
212
|
|
|
153
213
|
```text
|
|
154
|
-
✓ [PASS] lockfile: qa-lock.json present (
|
|
214
|
+
✓ [PASS] lockfile: .qa-engineer/qa-lock.json present (240 files, pack 0.10.0)
|
|
155
215
|
✓ [PASS] integrity: all installed files match lockfile hashes
|
|
156
216
|
✓ [PASS] skills: 13 skill(s) installed
|
|
157
217
|
✓ [PASS] contracts: 12 contract schema(s) present
|
|
158
|
-
✓ [PASS] engine: deterministic engine
|
|
159
|
-
✓ [PASS] engine-runs:
|
|
218
|
+
✓ [PASS] engine: deterministic engine shared at .qa-engineer/engine
|
|
219
|
+
✓ [PASS] engine-runs: shared engine runs (node 24.18.0)
|
|
160
220
|
✓ [PASS] node: Node v24.18.0
|
|
161
221
|
✓ self-test PASSED
|
|
162
222
|
```
|
|
163
223
|
|
|
164
|
-
Every line `[PASS]`? You're done installing. If anything says `[FAIL]`, jump to
|
|
224
|
+
Every line `[PASS]`? You're done installing. If anything says `[FAIL]`, jump to
|
|
225
|
+
[When something goes wrong](#when-something-goes-wrong).
|
|
226
|
+
|
|
227
|
+
> Checking a **project** install instead? Drop the `--global`: run
|
|
228
|
+
> `qa-engineer self-test` from inside the project, and it finds the right one on its own.
|
|
165
229
|
|
|
166
230
|
## Step 4 — Your first real task
|
|
167
231
|
|
|
@@ -223,29 +287,51 @@ That's the core loop: **`/qa-init` once, then `/qa-run` and `/qa-debug` whenever
|
|
|
223
287
|
|
|
224
288
|
## What got installed
|
|
225
289
|
|
|
226
|
-
|
|
290
|
+
**With a machine-wide install**, everything lives in one folder:
|
|
291
|
+
|
|
292
|
+
```text
|
|
293
|
+
~/.qa-engineer/
|
|
294
|
+
├── engine/ ← the analysis tools, one shared copy
|
|
295
|
+
├── skills/ ← the thirteen skills your assistant reads
|
|
296
|
+
└── qa-lock.json ← every installed file and its checksum
|
|
297
|
+
```
|
|
298
|
+
|
|
299
|
+
Your assistant is pointed at those skills by a shortcut in its own folder
|
|
300
|
+
(`~/.claude/skills/` for Claude Code). **Your projects get nothing at all** until you
|
|
301
|
+
actually run a command — at which point you'll see:
|
|
227
302
|
|
|
228
303
|
```text
|
|
229
304
|
your-app/
|
|
230
|
-
├── .
|
|
231
|
-
|
|
232
|
-
│ ├── qa-debug/
|
|
233
|
-
│ └── …
|
|
234
|
-
├── .claude/skills/ ← same files again, only if you use Claude Code
|
|
235
|
-
├── qa-lock.json ← a list of every installed file and its checksum
|
|
236
|
-
└── (later) .qa/context.md and qa-artifacts/ ← created when you run the commands
|
|
305
|
+
├── .qa/context.md ← what this project is, written by /qa-init
|
|
306
|
+
└── qa-artifacts/ ← reports and evidence from the commands you run
|
|
237
307
|
```
|
|
238
308
|
|
|
239
|
-
|
|
309
|
+
<details>
|
|
310
|
+
<summary><b>What a project install puts in your repository instead</b></summary>
|
|
311
|
+
|
|
312
|
+
```text
|
|
313
|
+
your-app/
|
|
314
|
+
├── .agents/skills/ ← the thirteen skills your assistant reads
|
|
315
|
+
├── .claude/skills/ ← the same skills again, only if you use Claude Code
|
|
316
|
+
├── qa-lock.json ← every installed file and its checksum
|
|
317
|
+
└── (later) .qa/context.md and qa-artifacts/
|
|
318
|
+
```
|
|
240
319
|
|
|
241
|
-
Worth adding to your `.gitignore` if you don't want the
|
|
320
|
+
Worth adding to your `.gitignore` if you don't want the outputs committed:
|
|
242
321
|
|
|
243
322
|
```gitignore
|
|
244
323
|
qa-artifacts/
|
|
245
324
|
.qa/backups/
|
|
246
325
|
```
|
|
247
326
|
|
|
248
|
-
Most teams **do** commit `.agents/skills/` and `qa-lock.json`, so everyone
|
|
327
|
+
Most teams **do** commit `.agents/skills/` and `qa-lock.json`, so everyone gets the same
|
|
328
|
+
skills from a single `git clone`.
|
|
329
|
+
|
|
330
|
+
</details>
|
|
331
|
+
|
|
332
|
+
**Nothing else is touched.** Your source code, your tests, and your config are left
|
|
333
|
+
exactly as they were. `qa-lock.json` records precisely what was written, so `uninstall`
|
|
334
|
+
removes that and nothing more.
|
|
249
335
|
|
|
250
336
|
## How you type a command in your assistant
|
|
251
337
|
|
|
@@ -314,24 +400,24 @@ reaches `/qa-explore` in all of them. Full details and the source for each path:
|
|
|
314
400
|
Start here. It fixes most problems:
|
|
315
401
|
|
|
316
402
|
```bash
|
|
317
|
-
|
|
403
|
+
qa-engineer doctor
|
|
318
404
|
```
|
|
319
405
|
|
|
320
406
|
`doctor` checks your environment and your install and prints a hint for anything it finds. Items marked `warn` with an "optional" hint are fine to ignore.
|
|
321
407
|
|
|
322
408
|
### The commands don't appear in my assistant
|
|
323
409
|
|
|
324
|
-
1. **
|
|
410
|
+
1. **Is it actually installed?** Run `qa-engineer self-test`. If it reports a missing lockfile, the install did not land where you think — re-run `qa-engineer install --global`.
|
|
325
411
|
2. **Restart your assistant.** Most only look for new skills at startup.
|
|
326
412
|
3. **Try plain English.** Instead of `/qa-run`, ask *"run my tests and report the result"*. Slash-command support varies by assistant; the skills work either way.
|
|
327
|
-
4. **Check your assistant reads the standard path.** Most read `.agents/skills/`. Claude Code uses `.claude/skills/`, which the installer also writes when it detects Claude. To force it: `
|
|
413
|
+
4. **Check your assistant reads the standard path.** Most read `.agents/skills/`. Claude Code uses `.claude/skills/`, which the installer also writes when it detects Claude. To force it: `qa-engineer install --global --agent claude-code`
|
|
328
414
|
|
|
329
415
|
### "refusing to overwrite N file(s) not owned by a previous install"
|
|
330
416
|
|
|
331
417
|
You already have a file where the pack wants to write one. It will not silently overwrite your work. Either move your file, or overwrite deliberately:
|
|
332
418
|
|
|
333
419
|
```bash
|
|
334
|
-
|
|
420
|
+
qa-engineer install --force
|
|
335
421
|
```
|
|
336
422
|
|
|
337
423
|
`--force` backs everything up to `.qa/backups/<timestamp>/` first.
|
|
@@ -341,8 +427,8 @@ npx qa-engineer install --yes --force --project .
|
|
|
341
427
|
The deterministic engine isn't reachable. Fix in this order:
|
|
342
428
|
|
|
343
429
|
```bash
|
|
344
|
-
|
|
345
|
-
|
|
430
|
+
qa-engineer repair # reinstall the analysis tools
|
|
431
|
+
qa-engineer doctor # should now report the engine runs
|
|
346
432
|
```
|
|
347
433
|
|
|
348
434
|
"Degraded" is not a bug — it is the skill telling you it could not run a tool and therefore trusts its own answer less. That's the honest behaviour.
|
|
@@ -352,7 +438,7 @@ npx qa-engineer doctor --project . # should now say "bundled engine runs
|
|
|
352
438
|
Someone edited an installed skill file. Restore them:
|
|
353
439
|
|
|
354
440
|
```bash
|
|
355
|
-
|
|
441
|
+
qa-engineer repair
|
|
356
442
|
```
|
|
357
443
|
|
|
358
444
|
If the edit was deliberate, note that `update` will overwrite it again — keep customisations outside `.agents/skills/`.
|
|
@@ -364,7 +450,7 @@ That is the exact failure this project exists to prevent, and it has three defen
|
|
|
364
450
|
### Still stuck
|
|
365
451
|
|
|
366
452
|
```bash
|
|
367
|
-
|
|
453
|
+
qa-engineer doctor --json
|
|
368
454
|
```
|
|
369
455
|
|
|
370
456
|
Open an issue and paste that output. It answers most questions before they're asked.
|
|
@@ -372,12 +458,34 @@ Open an issue and paste that output. It answers most questions before they're as
|
|
|
372
458
|
## Updating and uninstalling
|
|
373
459
|
|
|
374
460
|
```bash
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
|
|
461
|
+
qa-engineer update # refresh to the current version
|
|
462
|
+
qa-engineer verify # check nothing was corrupted
|
|
463
|
+
qa-engineer doctor # diagnose problems and suggest fixes
|
|
464
|
+
qa-engineer repair # reinstall anything missing or damaged
|
|
465
|
+
qa-engineer uninstall # remove everything it installed
|
|
378
466
|
```
|
|
379
467
|
|
|
380
|
-
|
|
468
|
+
**You don't have to say which install you mean.** Each command looks for a project
|
|
469
|
+
install where you're standing, and falls back to the machine-wide one. Add `--global` or
|
|
470
|
+
`--project .` when you want to be explicit.
|
|
471
|
+
|
|
472
|
+
`uninstall` removes exactly the files listed in `qa-lock.json`, backs each one up first,
|
|
473
|
+
and leaves everything else alone. If you edited an installed file it stops and tells you
|
|
474
|
+
rather than destroying your change; `--force` proceeds anyway. Add `--dry-run` to any
|
|
475
|
+
command to see what it *would* do without doing it.
|
|
476
|
+
|
|
477
|
+
<details>
|
|
478
|
+
<summary><b>Removing it completely</b></summary>
|
|
479
|
+
|
|
480
|
+
```bash
|
|
481
|
+
qa-engineer uninstall --global # the machine-wide install
|
|
482
|
+
npm uninstall -g qa-engineer # the qa-engineer command itself
|
|
483
|
+
```
|
|
484
|
+
|
|
485
|
+
Run `qa-engineer uninstall --project .` inside any repository that has its own install
|
|
486
|
+
first — those are separate and are not removed by the global uninstall.
|
|
487
|
+
|
|
488
|
+
</details>
|
|
381
489
|
|
|
382
490
|
## How it works
|
|
383
491
|
|
|
@@ -439,7 +547,7 @@ Stated plainly, so nothing surprises you:
|
|
|
439
547
|
- **It is not a test runner.** It drives Playwright; it doesn't replace it.
|
|
440
548
|
- **It doesn't run your tests in CI by itself.** The skills need an AI assistant to read them. The installer works fine in CI, and `verify` makes a good pipeline check.
|
|
441
549
|
- **It can't stop an assistant that ignores it.** The defences make dishonest answers *fail loudly* rather than pass quietly — they can't force a model to read the skill.
|
|
442
|
-
- **It has not been benchmarked across AI models.** The tooling is thoroughly tested (
|
|
550
|
+
- **It has not been benchmarked across AI models.** The tooling is thoroughly tested (245 automated tests, 27 evaluation cases). How faithfully each assistant follows the skills is measured for one model, in one session, and [documented as such](docs/release/v1-excellence-audit.md). If you need a published accuracy number before adopting a tool, this one doesn't have it yet.
|
|
443
551
|
- **It sends nothing anywhere.** No telemetry, no network calls, no accounts.
|
|
444
552
|
- **It is version 0.9.0** — a public preview. Solid and heavily tested, still pre-1.0.
|
|
445
553
|
|
|
@@ -449,10 +557,10 @@ Everything below is reproducible from a clone with the command beside it.
|
|
|
449
557
|
|
|
450
558
|
| Evidence | Command |
|
|
451
559
|
| --- | --- |
|
|
452
|
-
|
|
|
560
|
+
| 151 engine tests: analysis, diagnostics, reporting, artifacts, frameworks, seams | `node --test packages/engine/test/*.test.mjs` |
|
|
453
561
|
| The recorded corpus, proven against a second implementation | `node --test packages/engine/test/corpus.test.mjs` |
|
|
454
|
-
|
|
|
455
|
-
|
|
|
562
|
+
| 94 installer tests — installation scopes, security, and repeated-use stress | `npm test` |
|
|
563
|
+
| 27 evaluation cases, including deliberately dishonest outputs the scorer must reject | `npm run validate:evals` |
|
|
456
564
|
| 4 real AI-produced results, scored | `node tests/evals/run-live.mjs --captures claude-opus-5` |
|
|
457
565
|
| 17 repository checks — including "documentation matches implementation" | `npm run validate:skills` … |
|
|
458
566
|
|
|
@@ -470,7 +578,7 @@ Nothing below is required to use the pack.
|
|
|
470
578
|
| [docs/report-format.md](docs/report-format.md) | Consuming the JSON output from your own tooling |
|
|
471
579
|
| [docs/installation/](docs/installation/README.md) | Per-assistant installation guides |
|
|
472
580
|
| [COMPATIBILITY.md](COMPATIBILITY.md) | Supported assistants, runtimes, and frameworks |
|
|
473
|
-
| [docs/architecture/README.md](docs/architecture/README.md) |
|
|
581
|
+
| [docs/architecture/README.md](docs/architecture/README.md) | 17 decision records explaining every major choice |
|
|
474
582
|
| [docs/release/v0.9-release-checklist.md](docs/release/v0.9-release-checklist.md) | Exactly what was verified for this release, and what wasn't |
|
|
475
583
|
| [CHANGELOG.md](CHANGELOG.md) | What changed, including what regressed |
|
|
476
584
|
|
|
@@ -502,7 +610,7 @@ Start with [CONTRIBUTING.md](CONTRIBUTING.md). Everyone is bound by the [Code of
|
|
|
502
610
|
|
|
503
611
|
| Need | Where |
|
|
504
612
|
| --- | --- |
|
|
505
|
-
| Something is broken | Run `
|
|
613
|
+
| Something is broken | Run `qa-engineer doctor`, then read [When something goes wrong](#when-something-goes-wrong) |
|
|
506
614
|
| A question | GitHub Discussions — see [SUPPORT.md](SUPPORT.md) |
|
|
507
615
|
| A bug | GitHub Issues, with your `doctor --json` output |
|
|
508
616
|
| A security problem | **Privately** — see [SECURITY.md](SECURITY.md). Never a public issue. |
|
package/package.json
CHANGED