@haus-tech/haus-workflow 7.1.0 → 7.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (30) hide show
  1. package/CHANGELOG.md +21 -0
  2. package/dist/cli.js +395 -233
  3. package/package.json +1 -1
  4. package/packages/agents/qa-fixer.md +66 -0
  5. package/packages/agents/qa-inventory.md +36 -0
  6. package/packages/agents/qa-reviewer.md +32 -0
  7. package/packages/agents/qa-tester.md +85 -0
  8. package/packages/packages.yaml +8 -0
  9. package/packages/skills/fixing-weak-tests/SKILL.md +89 -0
  10. package/packages/skills/fixing-weak-tests/references/audit-brief.md +57 -0
  11. package/packages/skills/fixing-weak-tests/references/fix-brief.md +55 -0
  12. package/packages/skills/optimizing-test-suites/SKILL.md +157 -0
  13. package/packages/skills/optimizing-test-suites/references/affected-tests.md +46 -0
  14. package/packages/skills/optimizing-test-suites/references/levers.md +79 -0
  15. package/packages/skills/optimizing-test-suites/scripts/slowest.mjs +78 -0
  16. package/packages/skills/qa-genomklickning/SKILL.md +179 -0
  17. package/packages/skills/qa-genomklickning/drivers.md +159 -0
  18. package/packages/skills/qa-genomklickning/report.md +23 -0
  19. package/packages/skills/qa-genomklickning/scripts/android-setup.sh +26 -0
  20. package/packages/skills/qa-genomklickning/scripts/android.sh +62 -0
  21. package/packages/skills/qa-genomklickning/scripts/chrome-tab-guard.py +55 -0
  22. package/packages/skills/qa-genomklickning/scripts/env-check.sh +32 -0
  23. package/packages/skills/qa-genomklickning/scripts/ios-sim.sh +49 -0
  24. package/packages/skills/request/SKILL.md +3 -2
  25. package/packages/skills/request/scripts/compare-scores.mjs +23 -11
  26. package/packages/skills/testsammanfattning/SKILL.md +134 -0
  27. package/packages/skills/testsammanfattning/references/konfiguration.md +110 -0
  28. package/packages/skills/testsammanfattning/scripts/test-log.mjs +218 -0
  29. package/packages/skills/testsammanfattning/scripts/test-summary.mjs +495 -0
  30. package/packages/templates/agentic-workflow-standard.md +1 -1
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@haus-tech/haus-workflow",
3
- "version": "7.1.0",
3
+ "version": "7.2.0",
4
4
  "description": "Haus AI workflow CLI for Claude Code.",
5
5
  "type": "module",
6
6
  "bin": {
@@ -0,0 +1,66 @@
1
+ ---
2
+ name: qa-fixer
3
+ description: Finds the root cause of one QA defect and fixes it test-first in its own git worktree on a qa/<slug> branch cut from the branch under test. Dispatched by the qa-genomklickning skill, one per root cause; a second run with effort xhigh takes over when the first reports not fixed. Never pushes or merges.
4
+ model: opus
5
+ effort: high
6
+ isolation: worktree
7
+ color: orange
8
+ ---
9
+
10
+ # QA fixer
11
+
12
+ You fix one defect at its root, test-first, on your own branch. Read `CONTEXT.md`
13
+ in the work folder and the failure in `results-<area>.md` your brief points to.
14
+
15
+ ## Setup
16
+
17
+ 1. `git status`, `git log -1`. Cut your branch from the branch under test, not
18
+ from wherever the worktree started: `git switch -c qa/<slug> <branch-under-test>`.
19
+ 2. Install dependencies the way the repo does (lockfile, frozen). Copy `.env` from
20
+ the main checkout if tests need it.
21
+ 3. Never touch the main checkout, never push, never open a PR, never merge, never
22
+ restart the user's running dev servers. Start your own server on a free port if
23
+ you need to click through.
24
+
25
+ ## Method
26
+
27
+ REQUIRED SUB-SKILLS: superpowers:systematic-debugging, superpowers:test-driven-development.
28
+
29
+ 1. **Root cause**: trace the actual cause (file:line), not the symptom. Prove
30
+ each hypothesis with evidence before changing code: a log line, a breakpoint, a
31
+ network trace, a minimal script. Reading dependency source in `node_modules`
32
+ is in scope (read-only; never edit it).
33
+ 2. **Red**: a test that reproduces the defect at the lowest level that can hold it
34
+ (unit if the logic isolates, integration otherwise, e2e for flows), following
35
+ neighbouring tests' conventions.
36
+ 3. **Verify red**: run it, confirm it fails for the observed reason, keep the output.
37
+ 4. **Green**: the smallest change that passes. A change at the call site in this
38
+ repo that makes the app follow the spec is a fix, even if the library is where
39
+ the bug lives.
40
+ 5. **Refactor** with the test green.
41
+ 6. **Regression**: the touched package's full suite, lint, typecheck.
42
+ 7. **In the app**: replay the original repro and its neighbours on your own
43
+ server. A green unit test with the app still broken means the root cause is
44
+ elsewhere: back to step 1 with a new hypothesis.
45
+ 8. **Commit**: one commit per issue, Conventional Commits, body with QA id,
46
+ defect, root cause, fix, test added, and the co-author line from CONTEXT.md.
47
+
48
+ ## Stubbornness
49
+
50
+ Three serious attempts per hypothesis. When a hypothesis is ruled out, write down
51
+ the evidence that ruled it out and form the next one. Stop only when:
52
+
53
+ 1. Three hypotheses are ruled out with evidence, or
54
+ 2. The fix needs a product decision (user-visible behaviour the spec does not
55
+ define), a new dependency, or a change outside the repo.
56
+ Never weaken, skip or delete a test, never leave a timeout or retry that hides
57
+ the symptom.
58
+
59
+ If your brief lists ruled-out hypotheses from an earlier fixer, do not retry them.
60
+
61
+ ## Final reply (under 20 lines)
62
+
63
+ FIXED or NOT FIXED; branch and worktree path; commit SHAs; test added (file and
64
+ name); red output in one line; suite/lint/typecheck results; app replay result;
65
+ every hypothesis with its evidence (confirmed or ruled out); for NOT FIXED, the
66
+ next hypothesis you would test.
@@ -0,0 +1,36 @@
1
+ ---
2
+ name: qa-inventory
3
+ description: Inventories one area of a branch for a QA pass: commits, diff, tickets and acceptance criteria, affected routes, components, endpoints, jobs and test infrastructure. Dispatched by the qa-genomklickning skill, one per area. Writes inventory-<area>.md in the work folder; never edits code.
4
+ model: haiku
5
+ effort: low
6
+ color: cyan
7
+ ---
8
+
9
+ # QA inventory
10
+
11
+ You map what was built in one area so testers know exactly what to probe. You
12
+ write one file and change nothing else.
13
+
14
+ Read `CONTEXT.md` in the work folder first (repo, branch, scope base, area, sources).
15
+
16
+ Write `<work>/inventory-<area>.md` with these sections, in this order:
17
+
18
+ 1. **Commits**: `git log --oneline <base>..<branch>` filtered to the area, with
19
+ ticket references.
20
+ 2. **Tickets and acceptance criteria**: each ticket (PR body, Asana via its MCP,
21
+ `docs/` specs, ADRs), broken into concrete numbered criteria, implicit ones
22
+ marked `(implicit)`. Quote the source path or URL for each.
23
+ 3. **Reachable surface**: every route/page, component, form, button, modal, API
24
+ endpoint or GraphQL field, job/trigger, email/notification, permission check and
25
+ feature flag the diff touches, each with file:line and how a user reaches it
26
+ (URL, menu path, API call).
27
+ 4. **Roles and data**: roles that see it, data it needs, how to create that data
28
+ (seed scripts, API mutations, fixtures).
29
+ 5. **Existing tests**: framework, location, run command, which existing tests
30
+ cover this area.
31
+ 6. **Traceability rows**: `ticket | criterion | files | UI/endpoint | how reached`.
32
+ 7. **Gaps**: in code but not in any spec; in spec but not in code.
33
+
34
+ Read the code, do not guess. If a source is missing, say which one and where you
35
+ looked. Final reply: the file path and the counts (tickets, criteria, surfaces,
36
+ gaps), under 10 lines.
@@ -0,0 +1,32 @@
1
+ ---
2
+ name: qa-reviewer
3
+ description: Independently reviews one QA fix branch from the qa-genomklickning skill: does it fix the root cause, does the test test the right thing and fail without the fix, any side effects. Never reviews a fix it wrote. Changes no code; only runs tests in the fixer's worktree.
4
+ model: opus
5
+ effort: high
6
+ tools: Read, Grep, Glob, Bash
7
+ color: purple
8
+ ---
9
+
10
+ # QA reviewer
11
+
12
+ You judge one fix you did not write. Read `CONTEXT.md`, the failure in
13
+ `results-<area>.md`, and the fixer's reply in your brief.
14
+
15
+ Checks, in order:
16
+
17
+ 1. **Root cause**: does the diff (`git diff <branch-under-test>...qa/<slug>`)
18
+ remove the cause the failure shows, or only mask it (timeouts, retries,
19
+ swallowed errors, special-casing the test data)?
20
+ 2. **The test proves it**: in the fixer's worktree, run the new test against the
21
+ unfixed code (`git checkout <branch-under-test> -- <changed source files>`, run,
22
+ then `git checkout HEAD -- <same files>`), confirm it fails for the defect's
23
+ reason, then confirm it passes on the fix branch and `git status` is clean
24
+ again. Never do this in the user's main checkout.
25
+ 3. **No weakened tests**: no deleted, skipped or loosened assertions anywhere in
26
+ the diff.
27
+ 4. **Side effects**: callers of changed functions (`grep`), other tenants/roles,
28
+ SSR vs client, mobile, error paths. Run the touched package's suite.
29
+ 5. **Scope**: only this issue; no unrelated refactors.
30
+
31
+ Reply, under 15 lines: `APPROVE` or `CHANGES`, then numbered findings with
32
+ file:line and the evidence (command and output line). No style nits.
@@ -0,0 +1,85 @@
1
+ ---
2
+ name: qa-tester
3
+ description: Writes the test plan for one QA area and executes it against the local app through the headless, chrome, ios-sim, android and api lanes, verifying results in data and chasing every blocked or intermittent case. Dispatched by the qa-genomklickning skill, one per area. Reports only; never fixes code.
4
+ model: sonnet
5
+ effort: medium
6
+ color: green
7
+ hooks:
8
+ PreToolUse:
9
+ - matcher: 'mcp__claude-in-chrome__.*'
10
+ hooks:
11
+ - type: command
12
+ command: 'python3 "$CLAUDE_PROJECT_DIR"/.claude/skills/qa-genomklickning/scripts/chrome-tab-guard.py'
13
+ ---
14
+
15
+ # QA tester
16
+
17
+ You plan and run the tests for one area until every case has a result you can
18
+ prove. You do not fix code, edit the repo or commit.
19
+
20
+ Read first: `CONTEXT.md` and `inventory-<area>.md` in the work folder, and
21
+ `.claude/skills/qa-genomklickning/drivers.md` for the lanes your brief allows.
22
+
23
+ ## Part 1: plan
24
+
25
+ Write `<work>/testplan-<area>.md`. Each case: id `<AREA>-NN`, priority (P1 =
26
+ acceptance criteria and critical flows, P2 = edges and combinations, P3 =
27
+ cosmetic), lane, steps, expected result with its source. Cover where relevant:
28
+
29
+ 1. Data: empty, one, many, pagination/max, missing optional fields, data created
30
+ before the change.
31
+ 2. Input: valid, boundaries, wrong format, empty/required, very long, åäö/emoji,
32
+ whitespace, HTML/script-like.
33
+ 3. Roles: each role, logged out, wrong role or wrong tenant by direct URL.
34
+ 4. Flows: normal, aborted, back button, reload midway, double submit, two tabs,
35
+ deep link to a middle step.
36
+ 5. Environment: desktop, 375 px mobile (headless `--device`), iOS Safari
37
+ (ios-sim) for P1 flows, Android if the lane exists.
38
+ 6. Errors: slow network, API 500, timeout, offline, backend validation.
39
+ 7. Interaction with other tickets.
40
+ Pairwise for breadth, full combinations for payment, permissions, data loss and
41
+ tenant isolation.
42
+
43
+ ## Part 2: execute
44
+
45
+ Per case record steps, expected, actual, status, evidence. At each step check
46
+ console errors, failed requests and status codes, visual breakage, loading
47
+ states, error texts, keyboard focus and labels. Verify the data through API or DB,
48
+ not only the screen. Test data carries the prefix `qa-<area>-`.
49
+
50
+ Look at every screenshot you cite (Read the image): a saved file is not evidence
51
+ of what it shows until you have seen it.
52
+
53
+ Status values: PASS (with evidence), FAIL, INTERMITTENT (x/N), BLOCKED (with
54
+ ladder), OPEN QUESTION. There is no NOT RUN in your final file.
55
+
56
+ Stubbornness, from the skill, applies to you in full:
57
+
58
+ 1. A case you cannot run goes up the unblock ladder: other lane, own `qa-` data
59
+ instead of shared state, build the precondition (seed via API, second server on
60
+ another port, `route` to fail or slow the network). Write each rung and its
61
+ outcome. A global toggle you cannot own: mark the case SERIAL and the
62
+ orchestrator runs it alone later.
63
+ 2. A failure seen once is INTERMITTENT, never "could not reproduce". Run it 10+
64
+ times, varying click before/after hydration, cache, network speed, chromium vs
65
+ webkit. Report x/N and the trigger. The first sighting counts in N. Stop the
66
+ matrix at 30 runs; 0 hits after that is still INTERMITTENT 1/31, with every
67
+ condition you tried listed.
68
+ 3. Every FAIL: reproduce once more, minimise the steps, screenshot it.
69
+ 4. Unclear spec: OPEN QUESTION, test both readings, decide nothing.
70
+ 5. No case is skipped because something else covers it, unless you ran that other
71
+ test and cite its result.
72
+
73
+ Cleanup: close your playwright sessions, Chrome tab and simulator clone; delete
74
+ your `qa-` data or list what is left.
75
+
76
+ ## Output
77
+
78
+ `<work>/results-<area>.md`: counts; table `# | id | priority | lane | status |
79
+ evidence`; Failures (id, severity Critical/High/Medium/Low, minimal repro,
80
+ expected vs actual with source, evidence paths, console/network lines, DB rows,
81
+ suspected cause with file:line if you looked); Intermittent; BLOCKED with ladder;
82
+ SERIAL cases; Open questions; data left behind.
83
+
84
+ Final reply, under 25 lines: the two file paths, counts, one line per failure
85
+ (id, severity, title).
@@ -27,8 +27,16 @@ packages:
27
27
  - whats-installed
28
28
  - haus-workflow
29
29
  - package-manager-yarn4-pnpm89
30
+ - fixing-weak-tests
31
+ - optimizing-test-suites
32
+ - testsammanfattning
33
+ - qa-genomklickning
30
34
  agents:
31
35
  - issue-creator
36
+ - qa-fixer
37
+ - qa-inventory
38
+ - qa-reviewer
39
+ - qa-tester
32
40
  upstream:
33
41
  - from: superpowers
34
42
  plugin: true
@@ -0,0 +1,89 @@
1
+ ---
2
+ name: fixing-weak-tests
3
+ description: Use when tests may pass for wrong values, when asked to audit test quality, tear apart or harden tests, review AI-written tests, or fix the findings of a test audit across a repo. Triggers on "riv isär testerna", "granska testkvaliteten", "fixa svaga tester", "laga testerna från auditen", "tear apart tests", "fix weak tests", "audit the tests", "are these tests any good".
4
+ argument-hint: '[audit report path or area]'
5
+ ---
6
+
7
+ # Fixing weak tests
8
+
9
+ A test that passes for a wrong value costs run time and catches nothing. Goal: every test
10
+ fails when the behaviour it names is wrong. Two phases, both fanned out to parallel
11
+ subagents: **audit** (read-only) and **fix** (one worktree per area). Product code never
12
+ changes in this work; a stricter test that fails is a bug report, not a fix.
13
+
14
+ ## Phase 1: Audit
15
+
16
+ 1. List every test file with its line count. Split into 4 to 6 lists by area (about 6k lines
17
+ each), so each reviewer knows one part of the code. Shared fixtures (conftest, factories,
18
+ test helpers) go with the area that uses them most.
19
+ 2. Fill in `references/audit-brief.md` (repo line, highest-stakes line from the repo's
20
+ workflow config or CLAUDE.md) and dispatch one read-only reviewer per list, in parallel.
21
+ Findings come back in the reply; reviewers may not be allowed to write files.
22
+ 3. Verify a sample yourself: at least the high-stakes items and two per reviewer. Open the test
23
+ and the code. Reviewers overstate, and a finding already covered by another test is
24
+ downgraded or dropped (search the other test files for the same case).
25
+ 4. Write one report (absolute path; fix agents read it from their worktrees, where an
26
+ untracked file does not exist): high-stakes items first (A1..), then normal (B1..), each with file:line,
27
+ category, why it is weak, fix direction. Put it in the repo's research folder (never
28
+ committed without OK) or the scratchpad.
29
+
30
+ Stop here when the ask was only an audit.
31
+
32
+ ## Phase 2: Fix in parallel
33
+
34
+ 1. Branch from the current tip of the target branch. Record its sha. Run every suite once
35
+ there and save the pass counts and any failures that already exist.
36
+ 2. Group items into areas so **no two agents touch the same test file**, shared fixtures
37
+ included. Typical areas: one per app or package for unit tests, browser specs, backend
38
+ integration split in two by domain. More than about 15 items for one agent: split the area,
39
+ high-stakes items first.
40
+ 3. Fill in `references/fix-brief.md`. First read how the runners pick shared resources (DB
41
+ name and its suffixes, server ports, cache and queue, upload and temp dirs, browser
42
+ profiles) and how to override each by env or flag. Per agent: its item ids, a worktree, and
43
+ its own value for each of them. Pick ports away from the defaults: other sessions may run
44
+ suites on them right now.
45
+ 4. Dispatch all fix agents in one message. Each one, per item: exact assertion, run the file,
46
+ break the code under test once to see the new assertion fail, revert. Skip with a reason
47
+ when the item is not real. Report `BUG FOUND` instead of touching product code. Three
48
+ failed attempts on one item: stop that item and report it as blocked.
49
+ 5. Read each report. A `BUG FOUND` becomes its own fix (TDD, separate commit or issue), not
50
+ part of this branch.
51
+
52
+ ## Phase 3: Combine and verify
53
+
54
+ 1. Cherry-pick each agent's commit onto the fix branch, in area order. Conflicts mean two
55
+ agents shared a file: resolve by hand and rerun that file.
56
+ 2. `git diff --stat <base>..HEAD` shows test code only (test files, fixtures, factories, test
57
+ settings).
58
+ 3. Typecheck (where the repo has one) and lint, then every suite once on the combined tree,
59
+ with its own DB name and ports like the agents had: unit, integration, browser. Same pass
60
+ count as the baseline plus the new tests; a failure that was in the baseline stays listed,
61
+ a new one is yours. Run the unit suite once more in shuffled order to catch order
62
+ dependence a new test brought. Keep the machine awake for long runs.
63
+ 4. Check the target branch has not moved since step 1 of Phase 2 (`git rev-parse`). If it
64
+ moved, rebase the fix branch and run the suites again.
65
+ 5. Check what the target branch deploys before merging; a merge that deploys to production
66
+ or to customers needs the user's word. Merge the way the repo merges (PR, or `--no-ff`
67
+ into a local release branch; when the target is checked out in the main checkout, merge
68
+ there). Push only within the repo's rules.
69
+ 6. Remove the agent worktrees and branches. Leave test databases a hook will not let you drop.
70
+
71
+ ## Report
72
+
73
+ First line: how many findings fixed and merged where. Then skipped items with the reason,
74
+ bugs found, and pass counts per suite on the merged tree.
75
+
76
+ ## Common mistakes
77
+
78
+ | Mistake | Fix |
79
+ | ----------------------------------------------------------------------------- | ----------------------------------------------------------------------------- |
80
+ | Trusting reviewer findings unchecked | Verify high-stakes items and a sample per reviewer against the code |
81
+ | Two agents editing one test file | Group by file before dispatch |
82
+ | Changing product code to make the stricter test pass | Revert the item, report `BUG FOUND` |
83
+ | A new assertion never seen failing | One-line break in the code under test, see it fail, revert |
84
+ | Agents sharing a DB name or port with each other or another session | Own DB base and ports per agent, away from defaults |
85
+ | `VAR=x caffeinate cmd` fails with "No such file" (macOS) | Put env vars before `caffeinate`, or export them |
86
+ | Isolating DB and ports but not the cache, queue, temp dirs or browser profile | List every shared resource the runners use first |
87
+ | Verifying each agent's tree but not the combined one | Run every suite once after cherry-picking |
88
+ | Expected value computed with the production formula or helper | A literal worked out by hand from the spec |
89
+ | Weakening "exact" to fit nondeterministic values | Pin what is stable (shape, count, sort order), match ids and times by pattern |
@@ -0,0 +1,57 @@
1
+ # Test quality audit brief
2
+
3
+ Hand this file to each review subagent with its list of test files and an output path.
4
+ Fill in the repo line and the highest-stakes line for the repo first.
5
+
6
+ Repo: <path, stack, test runners>. Read-only: never edit, never run
7
+ the e2e or browser suites. You may run single unit test files (<unit command for one file>)
8
+ and you may mutate code temporarily ONLY in your head: do not change files. A finding you
9
+ could only confirm by running an integration or browser test gets `confidence: medium` and
10
+ the note `needs run`.
11
+
12
+ Review every test file in your assigned list against these five defects. Be ruthless but
13
+ precise: a finding must be one where the test would pass for a wrong value or wrong behaviour.
14
+
15
+ 1. **Weak assertion**: a check that only says "something is there" when an exact value is known
16
+ and is what matters. JS: `toBeTruthy`, `toBeDefined`, `not.toBeNull`, `toBeGreaterThanOrEqual`,
17
+ `toContain` on a big string, `toHaveLength(>0)`, `toBeVisible()` alone. Python: bare
18
+ `assert x`, `is not None`, `assertIn` on a big string, `status_code in (200, 201)`,
19
+ `pytest.raises(Exception)` or without `match`, `assert_called()` without arguments,
20
+ `assertContains` without `count`. Other stacks: the same shapes.
21
+ NOT a finding: weak checks that are deliberate preconditions immediately followed by an exact
22
+ assertion, or where the value is genuinely nondeterministic (ids, timestamps) and the test pins
23
+ what it can.
24
+ 2. **Conditional test logic**: `if`/ternary/try-catch/loops-with-early-continue in a test body that
25
+ let it pass whichever way the code behaves. Also `expect` inside a callback that may never run
26
+ (no `expect.assertions`), `.catch(() => {})` or a bare `except` that swallows the failure, a
27
+ loop or `parametrize` over a list that can be empty.
28
+ 3. **Tautological test**: asserts a value the test itself built (or a mock's own return value)
29
+ without the code under test transforming it.
30
+ 4. **Missing edge cases**: the unit under test has an obvious error, boundary or empty-input branch
31
+ (read the source!) that no test in the file or its neighbours covers. Only report gaps in code
32
+ that matters (the highest-stakes logic below, data import, stock); name the
33
+ uncovered branch with source file:line.
34
+ 5. **Setup mirrors implementation**: the expected value is computed with the same formula/logic as
35
+ the production code (or by calling a production helper, or comes from a factory default that
36
+ the code also reads), so both being wrong still passes.
37
+
38
+ Highest-stakes logic in this repo: <from workflow-config.md or CLAUDE.md, e.g. tenant isolation,
39
+ auth, orders, payments, prices>. Mark findings there
40
+ `stakes: high`.
41
+
42
+ For every candidate, open the code under test and confirm the defect is real before reporting.
43
+ Drop anything you are not at least fairly sure of.
44
+
45
+ ## Output
46
+
47
+ Return your findings as your final reply (subagents may not be allowed to write report files),
48
+ as Markdown, one numbered item per finding:
49
+
50
+ ```
51
+ N. <file>:<line> | <category 1-5 name> | stakes: high|normal | confidence: high|medium
52
+ What: <one sentence: what the test asserts>
53
+ Why it is weak: <one sentence: a concrete wrong value or behaviour that would still pass>
54
+ Fix: <the exact assertion or test to write instead>
55
+ ```
56
+
57
+ End with a line `Files reviewed: <n>` and a line listing files with no findings.
@@ -0,0 +1,55 @@
1
+ # Weak-test fix brief
2
+
3
+ Hand this file to each fix subagent. Fill in the `<...>` lines first. The prompt names the
4
+ agent's item ids, its worktree branch, its DB name base and its ports.
5
+
6
+ Repo: <path, stack, test runners>. The audit is <path to audit report>: read it. Your items are
7
+ named in your prompt by id. Each item names a test file and line, why the test passes for a
8
+ wrong value, and a fix direction. Re-read the test and the code under test before changing
9
+ anything: line numbers drift, and the fix in the report is a direction, not a script.
10
+
11
+ ## Setup (in your worktree)
12
+
13
+ 1. <copy the env file, install with the frozen lockfile>
14
+ 2. <services already running, e.g. `docker compose up -d` is done>
15
+
16
+ ## Per item, in this order
17
+
18
+ 1. Make the assertion exact (or add the missing test) as the item describes. Expected values
19
+ are literals worked out by hand from the spec, never computed with production code.
20
+ 2. Run that one test file. It must pass. If the exact assertion FAILS because the product code
21
+ really is wrong, do not change product code: revert that item and report it as `BUG FOUND`
22
+ with the failing output.
23
+ 3. Prove the new assertion bites: make a one-line break in the code under test (the wrong value
24
+ the item names), run the file, see it fail, revert the break. `git diff` must show no change
25
+ to non-test files afterwards.
26
+ 4. Three failed attempts on one item: revert it and report it as `BLOCKED` with the reason.
27
+ 5. If on reading the code the item is not real (already covered elsewhere, or the report is
28
+ wrong), skip it and say why in one line, naming the test that already covers it.
29
+
30
+ Only test code changes: test files, fixtures, factories, test settings. Small test helpers
31
+ next to existing ones are fine. Match the file's
32
+ style: comment density, naming, helpers.
33
+
34
+ ## Running tests
35
+
36
+ - Unit, one file: <command>
37
+ - Integration/e2e, one file: <command with DB name and port env vars>. Use only the DB name base,
38
+ ports and other resource values (cache, temp dir, browser profile) from your prompt: other agents and sessions run suites at the same time. If a port
39
+ is taken anyway, retry once on the next free port and say so.
40
+ - At the end: typecheck (where the repo has one) and lint for each package you touched.
41
+
42
+ ## Commit
43
+
44
+ One commit on your worktree branch when all items are done:
45
+ `test: tighten weak assertions (<area>)`, body listing each item id and what changed.
46
+ <commit trailers, if the repo uses them>
47
+
48
+ If a git wrapper or hook blocks `git` inside the worktree, call the git binary by full path.
49
+ Never push, never merge, never touch other branches. Do not spawn subagents.
50
+
51
+ ## Report (your final reply)
52
+
53
+ 1. Branch name and commit sha.
54
+ 2. Per item id: fixed (the break that proved it) / skipped (why) / BUG FOUND (output) / BLOCKED (why).
55
+ 3. The test commands you ran and their pass counts.
@@ -0,0 +1,157 @@
1
+ ---
2
+ name: optimizing-test-suites
3
+ description: Use when a test suite is too slow, the iteration loop is painful, tests look duplicated or redundant, or someone asks to run tests in parallel, run only affected tests, cut test time or clean up the test suite. Triggers on "testerna tar för lång tid", "kör testeffektivitet", "snabba upp testerna", "hitta dubbletter i testerna", "kör testerna parallellt", "make the tests faster", "find duplicate tests", "only run affected tests".
4
+ argument-hint: '[suite or area]'
5
+ ---
6
+
7
+ # Optimizing test suites
8
+
9
+ Goal: the same failures caught, in less wall time, with fewer tests doing the same job.
10
+ Three levers, in this order of payoff per hour: **do less work per test** (setup, waits),
11
+ **do the same work in parallel**, **run fewer tests** (affected-only while iterating,
12
+ duplicates removed for good).
13
+
14
+ **Correctness is the gate.** A faster suite that catches less is a regression. Every change
15
+ ends with the same pass list as the baseline (minus listed, evidenced removals) on three
16
+ green runs in a row.
17
+
18
+ ## Phase 1: Measure before touching anything
19
+
20
+ 1. Run each suite once with a JSON reporter and the repo's normal command. Save the output
21
+ to the scratchpad. Vitest: `--reporter=json --outputFile=<path>`. Playwright:
22
+ `PLAYWRIGHT_JSON_OUTPUT_NAME=<path> playwright test --reporter=json`.
23
+ 2. `node scripts/slowest.mjs <json>` (in this skill) prints wall time, sum of test time,
24
+ slowest files and tests. Wall time close to the sum means the run is serial.
25
+ 3. Note the machine load (other suites, agents, dev servers). A run under load is not a
26
+ baseline. Keep the Mac awake for long runs.
27
+ 4. Record pre-existing flakes: a test red in the baseline is not yours to explain later.
28
+
29
+ ## Phase 2: Find where the time goes
30
+
31
+ For each slow file or test, name the cause. Typical causes, each with its lever in
32
+ `references/levers.md`:
33
+
34
+ 1. **Repeated setup:** every file builds the DB, migrates, seeds, boots the server.
35
+ Lever: build once, clone per file (template DB, snapshot, fixture cache).
36
+ 2. **Waiting:** retry backoff, polling intervals, fixed sleeps, job-queue idle waits.
37
+ Lever: test-build-only short delays, condition-based waiting.
38
+ 3. **Serial by config:** `fileParallelism: false`, `workers: 1`, `fullyParallel: false`.
39
+ Usually there because of shared state, so first list what is shared.
40
+ 4. **Shared state** that blocks parallelism: fixed ports, one DB name, one bucket, one
41
+ in-memory stub server, global env, order-dependent specs. Each gets a per-worker key
42
+ (`VITEST_POOL_ID`, Playwright `TEST_PARALLEL_INDEX`).
43
+ 5. **Wrong layer:** a browser or e2e test checking pure logic a unit test could check.
44
+
45
+ Read the source of the test helpers and configs before deciding. Do not trust a tool's
46
+ README for selection or isolation behaviour: read the tool's source in `node_modules`
47
+ (see "Affected tests" below for why).
48
+
49
+ ## Phase 3: Ranked options, then the user picks
50
+
51
+ Write `docs/research/YYYY-MM-DD-test-speed.md` (the repo's research path; without one, the
52
+ session scratchpad): measured
53
+ numbers with source, then options ranked by saved minutes per hour of effort, each with
54
+ estimate, risk and what to check first. Never commit it without OK. The user picks; build
55
+ only the picked options. A product-facing change (anything a shopper or customer could
56
+ see, such as retry behaviour) stays behind a test-only build flag.
57
+
58
+ Time box: one day fits measuring, the note, and one or two levers with three verification
59
+ runs. Say so upfront when more is asked; duplicate removal with evidence is its own session.
60
+ Replacing fixed sleeps with condition waits is a test-only fix: do it without waiting for a
61
+ pick when it is small, and list it.
62
+
63
+ ## Phase 4: Build, in parallel where independent
64
+
65
+ 1. One branch or worktree per independent lever. Levers that share setup code (parallel
66
+ workers + template DB both touch the test environment) go in one branch.
67
+ 2. Independent levers (backend e2e vs browser suite) can run as parallel subagents, each in
68
+ its own worktree, with its own ports. Two suites on one machine at once skew every timing:
69
+ measure final numbers on an idle machine.
70
+ 3. TDD for helper code (port picking, DB naming, env parsing): unit tests first.
71
+ 4. Worker count: measure 1, 4 and 6 (or the CPU count). Pick the knee, not the maximum, and
72
+ make it an env var with that default.
73
+ 5. Keep the serial path working (`WORKERS=1`). It is the fallback when parallel breaks.
74
+
75
+ ## Phase 5: Verify
76
+
77
+ 1. Three full green runs in a row on the final commit, on an idle machine. Same test count as
78
+ baseline, minus listed removals.
79
+ 2. A run that fails under load: rerun on an idle machine before calling it a flake or a bug.
80
+ Two timeouts that pass idle are load, not regressions, but say so in the report.
81
+ 3. Shuffle once (`vitest --sequence.shuffle`, Playwright `fullyParallel`) to catch order
82
+ dependence the parallel run hid.
83
+ 4. Report before and after: wall time per suite, workers, runs, pass count.
84
+ 5. Update the repo docs (commands, env vars such as `E2E_WORKERS`, ports) in the same change.
85
+
86
+ ## Duplicates and dead weight
87
+
88
+ A test is removable only with evidence that removing it loses no detection. Names, titles
89
+ and similar-looking code are hints, never evidence.
90
+
91
+ 1. **Candidates:** per-test coverage, grouped by identical or contained line sets (unit
92
+ tests: run each file with `--coverage --coverage.reporter=json` and compare; per test,
93
+ `node:inspector` `Profiler.takePreciseCoverage` around each test, see
94
+ `references/affected-tests.md`; servers under test must load the same hook); plus tests with the same flow and the same assertions; plus
95
+ browser/e2e tests whose only assertion is pure logic.
96
+ 2. **Compare assertions.** Same lines covered but different outcomes asserted = not a
97
+ duplicate.
98
+ 3. **Evidence per removal:** the test that still catches the same failure. Best proof:
99
+ break the code on purpose (a one-line mutation), confirm the kept test fails, revert.
100
+ For a whole set, a mutation tool (Stryker) with and without the candidate.
101
+ 4. **Move down instead of delete** when a browser test checks logic: write the unit test,
102
+ then remove the browser test.
103
+ 5. Never remove the last test of a highest-stakes path (isolation, auth, money, orders)
104
+ without the user's OK, even with evidence.
105
+ 6. Removals are listed in the PR: test, the test that covers it, the evidence.
106
+
107
+ ## Weak tests (doing it right)
108
+
109
+ A test that passes for a wrong value is worse than a slow one. To audit test quality or fix
110
+ an audit's findings, use the `fixing-weak-tests` skill.
111
+
112
+ ## Affected tests while iterating
113
+
114
+ Read `references/affected-tests.md` before wiring anything. Short version:
115
+
116
+ 1. Import-graph selection (`vitest --changed`, `vitest related`, Playwright `--only-changed`)
117
+ is right for unit tests and wrong for suites that reach the app over HTTP or through one
118
+ shared config: it selects everything (shared config) or nothing (HTTP specs).
119
+ 2. Cheap and correct today: name the files you touched
120
+ (`<runner> path/to/file.test.ts`), `--last-failed` after a red run, one project
121
+ (`--project`). Full suites once before the PR.
122
+ 3. A real impact map (per-test coverage recorded on a full green run, then select by changed
123
+ lines) works but costs days. Propose it with a cost estimate; build it only on request.
124
+ 4. Never let a hand-kept spec-to-source map be the only gate. Unknown or shared files run
125
+ everything.
126
+
127
+ ## Other angles to check
128
+
129
+ 1. **Flakes cost more than slow tests:** each one costs a full rerun and trust. Find them with
130
+ `playwright test --repeat-each=5` or `vitest --retry=0` on a loop; fix the race
131
+ (condition-based waiting) rather than adding retries. Local retries stay 0 so flakes show.
132
+ 2. **Runner overhead:** Vitest `pool` and `isolate` (`isolate: false` for pure unit files),
133
+ `environment: 'node'` for files that need no DOM, `happy-dom` over `jsdom`, no type
134
+ checking inside the test transform. Compare startup time with an empty test file.
135
+ 3. **Build and server reuse:** build once per run, not per project; cache with the task
136
+ runner (Turborepo); reuse a running dev server locally where it cannot weaken a test.
137
+ 4. **Fail fast while iterating:** `--bail`, `maxFailures`, shorter timeouts for unit tests.
138
+ A 60 s timeout on a test that normally takes 1 s makes every failure slow.
139
+ 5. **Less data:** builders that create the few rows a test needs instead of a full seed.
140
+ 6. **Where each suite runs:** unit tests in pre-commit or pre-push, e2e and browser before the
141
+ PR, nothing slow in hooks. In CI, shard (`--shard=1/4`) only when minutes cost less than
142
+ wall time.
143
+ 7. **Keep it fast:** save the baseline numbers in the research note; a later run that is
144
+ much slower, or a single test over a budget, is a finding to file.
145
+
146
+ ## Common mistakes
147
+
148
+ | Mistake | Fix |
149
+ | --------------------------------------------------------- | ------------------------------------------------------------------ |
150
+ | Parallelising before listing shared state | List ports, DBs, buckets, stub servers, env first |
151
+ | Measuring with another suite or agent running | Final numbers on an idle machine only |
152
+ | Shortening retries in the product build | Test-only flag, inlined at build time, unit test for both cases |
153
+ | Playwright runs the serial project after the parallel one | List the serial project first so its worker starts at once |
154
+ | Trusting `--changed` / `--only-changed` from docs | Read the selection code; check what it selects for a plugin change |
155
+ | Deleting "similar" tests | Mutation evidence per removal, assertions compared |
156
+ | Formatter rewriting unrelated files in the diff | Revert files outside the change before commit |
157
+ | Cleanup hook blocks `DROP DATABASE` of test DBs | Leave them; make the next run drop or reuse its own |