@1aboveio/engineering-skills 0.1.32 → 0.1.34
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +2 -2
- package/skills/better-review/SKILL.md +1 -1
- package/skills/e2e-test/SKILL.md +1 -1
- package/skills/engineering/SKILL.md +1 -1
- package/skills/engineering-runtime/SKILL.md +1 -1
- package/skills/engineering-runtime/coherence/workflow.json +13 -13
- package/skills/ensure-coverage/SKILL.md +1 -1
- package/skills/golden-path/SKILL.md +1 -1
- package/skills/harness-runtime/SKILL.md +1 -1
- package/skills/resolve-issues/SKILL.md +1 -1
- package/skills/resolve-release/SKILL.md +1 -1
- package/skills/rush-issues/SKILL.md +1 -1
- package/skills/rush-release/SKILL.md +1 -1
- package/skills/test-quality/SKILL.md +3 -2
- package/skills/test-quality/references/audit.md +30 -12
- package/skills/test-quality/references/speed.md +15 -0
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@1aboveio/engineering-skills",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.34",
|
|
4
4
|
"description": "Issue-to-PR and release orchestrators, review, coverage, golden paths and test quality.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
@@ -27,7 +27,7 @@
|
|
|
27
27
|
"id": "engineering",
|
|
28
28
|
"core": "@1aboveio/core-skills",
|
|
29
29
|
"coreSkills": [],
|
|
30
|
-
"digest": "
|
|
30
|
+
"digest": "31a5d3f233a23d00ed1ac08345219b56665cc80dd4e3bb19e57dfa28743f611a"
|
|
31
31
|
},
|
|
32
32
|
"publishConfig": {
|
|
33
33
|
"registry": "https://registry.npmjs.org/",
|
|
@@ -4,7 +4,7 @@ description: "Slash/explicit-only (/better-review). Single-agent adversarial rev
|
|
|
4
4
|
disable-model-invocation: true
|
|
5
5
|
dependencies:
|
|
6
6
|
- engineering-runtime
|
|
7
|
-
release-version: 0.50.
|
|
7
|
+
release-version: 0.50.14
|
|
8
8
|
---
|
|
9
9
|
**Updates:** Once/session run `node <skill-base-dir>/release-check.mjs`.
|
|
10
10
|
# Better Review
|
package/skills/e2e-test/SKILL.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: e2e-test
|
|
3
3
|
description: "Playwright end-to-end guidance for affected browser journeys, render health and presentation checks, with route manifests and sweeps for broader audits. Use when browser routes or user flows ship or change, or a Playwright test flakes. NOT for non-browser tests, defining what to test (ensure-coverage), the curated ALIVE/DEAD golden-path gate (golden-path), or CI workflow YAML (github-actions)."
|
|
4
|
-
release-version: 0.50.
|
|
4
|
+
release-version: 0.50.14
|
|
5
5
|
---
|
|
6
6
|
**Updates:** Once/session run `node <skill-base-dir>/release-check.mjs`.
|
|
7
7
|
# E2E Testing Standard
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: engineering
|
|
3
3
|
description: "Engineering conventions: feature-delivery priorities, focused verification and review, operational boundaries, and tracker routing. Standalone /engineering offers Global, Project, or Session setup and targeted resets for configured scopes; /engineering init keeps explicit project setup. Use when setting up AGENTS.md or CONTRIBUTING.md, saving or resetting engineering defaults, or before creating an engineering issue, spec, epic, ticket, or sub-issue on GitHub or Linear. Reads existing policy without writing during ordinary work. NOT the issue-to-PR workflow (resolve-issues), release execution (resolve-release), Git delivery (github-flow), or CI setup (github-actions)."
|
|
4
|
-
release-version: 0.50.
|
|
4
|
+
release-version: 0.50.14
|
|
5
5
|
---
|
|
6
6
|
**Updates:** Once/session run `node <skill-base-dir>/release-check.mjs`.
|
|
7
7
|
# Engineering
|
|
@@ -3,7 +3,7 @@ name: engineering-runtime
|
|
|
3
3
|
description: Internal support package for shared Engineering Workflow executable behavior. Never select or invoke this package as a user capability.
|
|
4
4
|
disable-model-invocation: true
|
|
5
5
|
user-invocable: false
|
|
6
|
-
release-version: 0.50.
|
|
6
|
+
release-version: 0.50.14
|
|
7
7
|
---
|
|
8
8
|
**Updates:** Once/session run `node <skill-base-dir>/release-check.mjs`.
|
|
9
9
|
# Engineering Runtime
|
|
@@ -369,7 +369,7 @@
|
|
|
369
369
|
"repository": "1aboveio/ai",
|
|
370
370
|
"revision": "main"
|
|
371
371
|
},
|
|
372
|
-
"contentDigest": "
|
|
372
|
+
"contentDigest": "752b804d780a38b1d9bd0e9d72ea1b5da72c57f0780c2bf1c12e14f1bf6e64f1",
|
|
373
373
|
"digestExcludes": []
|
|
374
374
|
},
|
|
375
375
|
{
|
|
@@ -382,7 +382,7 @@
|
|
|
382
382
|
"repository": "1aboveio/ai",
|
|
383
383
|
"revision": "main"
|
|
384
384
|
},
|
|
385
|
-
"contentDigest": "
|
|
385
|
+
"contentDigest": "f33ad00b81ed30dc790bb98cd7a5f8847e0b740242e90782ed62c714469c22c6",
|
|
386
386
|
"digestExcludes": []
|
|
387
387
|
},
|
|
388
388
|
{
|
|
@@ -395,7 +395,7 @@
|
|
|
395
395
|
"repository": "1aboveio/ai",
|
|
396
396
|
"revision": "main"
|
|
397
397
|
},
|
|
398
|
-
"contentDigest": "
|
|
398
|
+
"contentDigest": "beaf102a6fd0cd61d161840cd714ba666c9624630d9f036faf632ba87edf0a10",
|
|
399
399
|
"digestExcludes": []
|
|
400
400
|
},
|
|
401
401
|
{
|
|
@@ -408,7 +408,7 @@
|
|
|
408
408
|
"repository": "1aboveio/ai",
|
|
409
409
|
"revision": "main"
|
|
410
410
|
},
|
|
411
|
-
"contentDigest": "
|
|
411
|
+
"contentDigest": "2f14b74693c26eac70c66c395324bfc66d0a875a4e5cdab806bbc444ce3d266e",
|
|
412
412
|
"digestExcludes": []
|
|
413
413
|
},
|
|
414
414
|
{
|
|
@@ -421,7 +421,7 @@
|
|
|
421
421
|
"repository": "1aboveio/ai",
|
|
422
422
|
"revision": "main"
|
|
423
423
|
},
|
|
424
|
-
"contentDigest": "
|
|
424
|
+
"contentDigest": "d7f6366ceffa696bb27c22ecb1e36aae497a59a9a823bbcfdc7e5e4589bf5d5a",
|
|
425
425
|
"digestExcludes": []
|
|
426
426
|
},
|
|
427
427
|
{
|
|
@@ -434,7 +434,7 @@
|
|
|
434
434
|
"repository": "1aboveio/ai",
|
|
435
435
|
"revision": "main"
|
|
436
436
|
},
|
|
437
|
-
"contentDigest": "
|
|
437
|
+
"contentDigest": "590f366d88d51c1af4b55f73352de84ab7c11e7c3694c3faeced6aa86d6b1eab",
|
|
438
438
|
"digestExcludes": []
|
|
439
439
|
},
|
|
440
440
|
{
|
|
@@ -447,7 +447,7 @@
|
|
|
447
447
|
"repository": "1aboveio/ai",
|
|
448
448
|
"revision": "main"
|
|
449
449
|
},
|
|
450
|
-
"contentDigest": "
|
|
450
|
+
"contentDigest": "c669052d7b3b5aa78ae0f196cc2c8144a5a4abf9a4bfb6215231979d0258d8d0",
|
|
451
451
|
"digestExcludes": []
|
|
452
452
|
},
|
|
453
453
|
{
|
|
@@ -460,7 +460,7 @@
|
|
|
460
460
|
"repository": "1aboveio/ai",
|
|
461
461
|
"revision": "main"
|
|
462
462
|
},
|
|
463
|
-
"contentDigest": "
|
|
463
|
+
"contentDigest": "bd68c893af54b78bed446cb5cbd315f35db7678fb3a6ac0ae8086fb955980862",
|
|
464
464
|
"digestExcludes": []
|
|
465
465
|
},
|
|
466
466
|
{
|
|
@@ -473,7 +473,7 @@
|
|
|
473
473
|
"repository": "1aboveio/ai",
|
|
474
474
|
"revision": "main"
|
|
475
475
|
},
|
|
476
|
-
"contentDigest": "
|
|
476
|
+
"contentDigest": "1d36c6b8cadc5bbf7db837327896c538eed5a4118ef00fdc9a347532d514d1e3",
|
|
477
477
|
"digestExcludes": []
|
|
478
478
|
},
|
|
479
479
|
{
|
|
@@ -486,7 +486,7 @@
|
|
|
486
486
|
"repository": "1aboveio/ai",
|
|
487
487
|
"revision": "main"
|
|
488
488
|
},
|
|
489
|
-
"contentDigest": "
|
|
489
|
+
"contentDigest": "584b292f9a8bd228e90f42872f094d4cd80fb5cace439a19bc515c713b0b9656",
|
|
490
490
|
"digestExcludes": []
|
|
491
491
|
},
|
|
492
492
|
{
|
|
@@ -499,7 +499,7 @@
|
|
|
499
499
|
"repository": "1aboveio/ai",
|
|
500
500
|
"revision": "main"
|
|
501
501
|
},
|
|
502
|
-
"contentDigest": "
|
|
502
|
+
"contentDigest": "b0c1ddcf1c23e9a954678b5025235bbbdad41ff49a8495d905f598a9aac628e3",
|
|
503
503
|
"digestExcludes": []
|
|
504
504
|
},
|
|
505
505
|
{
|
|
@@ -512,13 +512,13 @@
|
|
|
512
512
|
"repository": "1aboveio/ai",
|
|
513
513
|
"revision": "main"
|
|
514
514
|
},
|
|
515
|
-
"contentDigest": "
|
|
515
|
+
"contentDigest": "a87001b2051f68b6e1ffbf2a076d08797538828c6310bf5318bd4d0b13900b59",
|
|
516
516
|
"digestExcludes": [
|
|
517
517
|
"coherence/workflow.json"
|
|
518
518
|
]
|
|
519
519
|
}
|
|
520
520
|
],
|
|
521
|
-
"releaseIdentity": "
|
|
521
|
+
"releaseIdentity": "7997c2e323a245373531ad4683ebf7f465bd0fb3254a3b665a67abc033b81c36",
|
|
522
522
|
"lifecycleAuthority": "native-skills-cli",
|
|
523
523
|
"repairRecipe": {
|
|
524
524
|
"id": "engineering-workflow-dependency-first",
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
name: ensure-coverage
|
|
3
3
|
description: "Define or audit behavior-first test criteria — ACs, required test levels, mock/integration/E2E policy — across three axes: breadth (Coverage Ledger), depth (over-mocking), presentation (perceivability of browser surfaces). NOT judging whether a single test is any good (test-quality), authoring browser tests (e2e-test), reviewing a PR (better-review), or delivering issues (resolve-issues)."
|
|
4
4
|
disable-model-invocation: true
|
|
5
|
-
release-version: 0.50.
|
|
5
|
+
release-version: 0.50.14
|
|
6
6
|
---
|
|
7
7
|
**Updates:** Once/session run `node <skill-base-dir>/release-check.mjs`.
|
|
8
8
|
# Ensure Coverage
|
|
@@ -3,7 +3,7 @@ name: golden-path
|
|
|
3
3
|
description: "Golden-path E2E: the short list of user journeys whose failure blocks a release, kept in a repo as one contract (golden-paths.json) with a generated GOLDEN_PATHS.md and a generated GitHub Actions workflow, and run against a local build or a deployed URL for an ALIVE/DEAD verdict. Use to set golden paths up in a repo ('init golden paths', 'add golden-path CI'), to migrate an older smoke.manifest.json or a hand-built smoke setup ('audit', 'migrate smoke'), to add, change, or remove a golden path, to ask 'is the build alive?' or run them before a PR or release, or when a change touches a flow listed in GOLDEN_PATHS.md. NOT for writing the Playwright specs themselves (e2e-test), deciding what a change must prove (ensure-coverage), or general workflow YAML (github-actions)."
|
|
4
4
|
dependencies:
|
|
5
5
|
- e2e-test
|
|
6
|
-
release-version: 0.50.
|
|
6
|
+
release-version: 0.50.14
|
|
7
7
|
---
|
|
8
8
|
**Updates:** Once/session run `node <skill-base-dir>/release-check.mjs`.
|
|
9
9
|
# Golden path
|
|
@@ -3,7 +3,7 @@ name: harness-runtime
|
|
|
3
3
|
description: Internal import-only library shared by first-party engineering packages. Not a workflow skill; never select or invoke it as a user capability.
|
|
4
4
|
disable-model-invocation: true
|
|
5
5
|
user-invocable: false
|
|
6
|
-
release-version: 0.50.
|
|
6
|
+
release-version: 0.50.14
|
|
7
7
|
---
|
|
8
8
|
**Updates:** Once/session run `node <skill-base-dir>/release-check.mjs`.
|
|
9
9
|
# Harness Runtime
|
|
@@ -4,7 +4,7 @@ description: "Slash/explicit-only (/resolve-release). Full GitHub Flow release f
|
|
|
4
4
|
disable-model-invocation: true
|
|
5
5
|
metadata:
|
|
6
6
|
workflow-role: coordinator
|
|
7
|
-
release-version: 0.50.
|
|
7
|
+
release-version: 0.50.14
|
|
8
8
|
---
|
|
9
9
|
**Updates:** Once/session run `node <skill-base-dir>/release-check.mjs`.
|
|
10
10
|
# Resolve release
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: test-quality
|
|
3
3
|
description: "Seven principles for tests that catch real bugs, and an audit that reports weak, redundant, and not-coverage tests without fixing them. Use when writing, changing, or reviewing tests, or when asked whether tests are useful, redundant, over-mocked, flaky, only assert static text, or test config files, or asked to make a slow test suite faster without losing coverage, or when the user asks to write the testing conventions into a repo's CONTRIBUTING.md. NOT defining what a spec must prove (ensure-coverage) or authoring browser tests (e2e-test)."
|
|
4
|
-
release-version: 0.50.
|
|
4
|
+
release-version: 0.50.14
|
|
5
5
|
---
|
|
6
6
|
**Updates:** Once/session run `node <skill-base-dir>/release-check.mjs`.
|
|
7
7
|
# Test Quality
|
|
@@ -10,6 +10,7 @@ A test is worth keeping when it fails for a real bug and stays green through a h
|
|
|
10
10
|
|
|
11
11
|
- **Write mode**: judge the tests needed for the requested behavior using the principles and the one question. Run `node "$SKILL_DIR/scripts/lint-tests.mjs" --diff <base>` when requested or required by the repository ([lint](#the-lint)); it covers uncommitted and untracked tests. An optional lint result does not create a new acceptance gate.
|
|
12
12
|
- **Audit mode**: reviewer subagents sweep a suite or a diff and report findings, quick by default ([audit](#audit-mode)). The audit **never edits, rewrites, or deletes a test**. It names the principle broken and the evidence; the fix belongs to the implementer.
|
|
13
|
+
- **Speed mode**: make a suite faster without losing coverage: measure durations first, remove shared overhead, then simplify slow tests by code path and verify ([speed](references/speed.md)). This mode changes tests; audit alone only reports.
|
|
13
14
|
- **Persist mode**: only when the user asks, write the principles into the repo's `CONTRIBUTING.md` ([persist](#persist-mode)).
|
|
14
15
|
|
|
15
16
|
| # | Principle | The failure it names | Quick tell |
|
|
@@ -58,7 +59,7 @@ A good assertion has a concrete answer to (1) and none to (2). An assertion with
|
|
|
58
59
|
|
|
59
60
|
## Audit mode
|
|
60
61
|
|
|
61
|
-
Reviewer subagents audit and never fix. The **quick audit** is the default and finishes in under 10 minutes: `lint-tests.mjs --shares 4` splits the flagged files, at most 4 reviewers (or the harness's parallel limit) read only their flagged spots on a 6-minute budget, each writes its report to a file and replies in one line, and the lead merges once and posts once, with no per-reviewer updates. A **deep audit**
|
|
62
|
+
Reviewer subagents audit and never fix. The **quick audit** is the default and finishes in under 10 minutes: `lint-tests.mjs --shares 4` splits the flagged files, at most 4 reviewers (or the harness's parallel limit) read only their flagged spots on a 6-minute budget, each writes its report to a file and replies in one line, and the lead merges once and posts once, with no per-reviewer updates. A **deep audit** runs only when the user asks for one: reviewers read every test file and explicitly answer all seven [principle questions](references/audit.md#principle-questions) in their reports. Any request that ends in changed tests ("audit and fix", "clean up the tests", "improve the suite") then runs **fix** (implementer subagents, disjoint files; each opens a `Redundant` test's twin and searches for what depends on a test before deleting it) → **verify** (lint on the diff, fresh reviewer on the fix). Steps, budgets, and the report template: [references/audit.md](references/audit.md).
|
|
62
63
|
|
|
63
64
|
### Labels
|
|
64
65
|
|
|
@@ -30,6 +30,36 @@ The lead runs it; the times are wall-clock budgets, not estimates.
|
|
|
30
30
|
|
|
31
31
|
The same flow with three changes: shares are packages or areas instead of lint shares (merge small packages so there are no more shares than the parallel limit), reviewers read **every** test file in their share, and there is no time budget. Reviewers use a frontier-tier model. Files, silence, and the single merge still apply.
|
|
32
32
|
|
|
33
|
+
Continue even when lint finds nothing; lint is a lead source, not the reading list. The quick audit's 8-minute reviewer cutoff does not apply. Each reviewer reads [principles.md](principles.md) in full before reviewing.
|
|
34
|
+
|
|
35
|
+
### Principle questions
|
|
36
|
+
|
|
37
|
+
Include these questions explicitly in every deep-audit reviewer's prompt, with the resolved path to `principles.md`. Ask the reviewer, not the user. The principles and their look-alikes decide the verdict; these questions make their application explicit.
|
|
38
|
+
|
|
39
|
+
| Principle | Question the reviewer must answer |
|
|
40
|
+
|---|---|
|
|
41
|
+
| 5 | Are any tests duplicates, or testing code that no longer runs? Name the equally strong twin or the evidence of dead code. |
|
|
42
|
+
| 1 | Are any tests checking only checked-in text instead of executing the behavior they claim to prove? |
|
|
43
|
+
| 6 | Are any tests asserting a config file's contents or format, as text or parsed JSON/YAML/TOML, instead of testing code that consumes it? Distinguish app data and test inputs from config. |
|
|
44
|
+
| 4 | Are any tests always successful, assertion-free, self-skipping, or swallowing failures? Can each assertion fail for a realistic bug, and does it check what the test's name promises? |
|
|
45
|
+
| 2 | Do expected values come from the code under test or copied literals rather than an independent contract or input? |
|
|
46
|
+
| 3 | Do mocks hide the first-party behavior the test claims to prove? |
|
|
47
|
+
| 7 | Can randomness, time, timing, run order, network, or shared files change the result for the same code? |
|
|
48
|
+
|
|
49
|
+
Apply all seven questions across every test file in the assigned share, including unflagged files. For duplication, compare tests within and across files; the lead also checks candidates across shares by opening the named twins. A clean lint or a search with no hits is not an answer.
|
|
50
|
+
|
|
51
|
+
Each reviewer adds a `Principle answers` section to its report, with one row per principle:
|
|
52
|
+
|
|
53
|
+
```markdown
|
|
54
|
+
## Principle answers
|
|
55
|
+
| principle | result | evidence and scope | unchecked scope and reason |
|
|
56
|
+
|---|---|---|---|
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Use `Found`, `None found`, or `Unverified`. Link `Found` to findings or unresolved Leads with real `file:line` evidence. Support `None found` with what was read and why it satisfies the principle. If any relevant scope could not be checked, record it in the last column and under `Not audited`; use `Unverified` when no problem is established and the check is incomplete. Keep any confirmed findings even when part of the scope is unchecked.
|
|
60
|
+
|
|
61
|
+
The lead preserves all seven answers and every unchecked scope in the merged report. Apply the same questions and reporting requirement when auditing a handful of tests without subagents. Deep audit remains read-only.
|
|
62
|
+
|
|
33
63
|
## Fix and verify
|
|
34
64
|
|
|
35
65
|
After the report, and only when the request ends in changed tests:
|
|
@@ -41,18 +71,6 @@ After the report, and only when the request ends in changed tests:
|
|
|
41
71
|
- **Run what you touched for real.** Run edited files with the lane's real dependencies (start a disposable database for DB-backed tests). A test that could not run is reported unverified, never "unrelated".
|
|
42
72
|
2. **Verify.** Run the lint on the diff, then have a fresh reviewer check the fix diff against the principles. Ask it to compare each deletion with its twin and to look for over-deletion; the fix step errs toward deleting too much, not too little.
|
|
43
73
|
|
|
44
|
-
## When the goal is speed
|
|
45
|
-
|
|
46
|
-
A request to make a suite faster "without losing coverage" is a fix request with a time target. The principles still decide what may go; measurement decides what is worth touching.
|
|
47
|
-
|
|
48
|
-
- **Budget, unless the repo sets its own.** Measured on the CI runner, test execution only (checkout, install, database start and migrations excluded): the unit and integration lanes together take **3 minutes or less**, and the integration lane alone **under 2 minutes**. Neither lane drives a real browser; browser journeys belong to the E2E lane, which has its own budget of **5 minutes or less**. Over budget is a finding even when every test is sound.
|
|
49
|
-
- **Measure before cutting.** Get per-file and per-test durations from the runner (Vitest's JSON reporter or a small reporter over `TestModule.diagnostic()`, `pytest --durations=0`, `go test -json`). For CI, measure on the CI runner: a laptop is often faster and less CPU-bound, so local timings mislead.
|
|
50
|
-
- **Find per-test overhead before cutting tests.** Durations say which files are slow, not why. Trace what the slow tests start and wait on (child processes, containers, database setup, network, file system) and count it per test. A cost every test pays, such as a subprocess each write triggers or setup repeated per test instead of per file, usually outweighs every duplicate in the suite. Remove that cost first, then prove the behavior it guarded still has a test that fails without it.
|
|
51
|
-
- **Expect little from deleting fast tests.** Mocked unit tests run in milliseconds; removing duplicates is worth it for refactor noise, rarely for time. A few slow files usually dominate. Look in them for real-time waits ([principle 7](principles.md#7-same-code-same-result)) and combinatorial matrices.
|
|
52
|
-
- **Collapse a matrix by code path, then prove it.** Map each case to the production branch it exercises (which lock, which recheck, which input class). Keep one case per distinct path and drop cases that differ only in data that takes the same path. Prove each kept case by disabling its path in production code, watching that case fail, and restoring the code. End with no production diff.
|
|
53
|
-
- **Shorten a wait only when it stays deterministic.** Prefer faking the clock. When real elapsed time is the behavior, follow principle 7's look-alike for it instead of trimming the margin.
|
|
54
|
-
- **Leave runner tuning to CI work.** Shard counts, worker counts, and test pools change no test and belong to the CI workflow (the `github-actions` skill), measured on the runner's real CPU quota.
|
|
55
|
-
|
|
56
74
|
## Reviewer steps
|
|
57
75
|
|
|
58
76
|
1. **Start from the lint.** Your share's findings are in `lint.json`. Every hit, gate or lead, is a candidate, not yet a finding. In a deep audit, also grep your files for the quick tells in [SKILL.md](../SKILL.md)'s principle table.
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# Speed mode
|
|
2
|
+
|
|
3
|
+
> Read when: making a test suite faster without losing coverage.
|
|
4
|
+
|
|
5
|
+
A request to make a suite faster "without losing coverage" is a fix request with a time target, not a read-only audit. The [principles](principles.md) still decide what may go; measurement decides what is worth touching. Measure first, use the [audit](audit.md) to assess the slow tests, then follow its [fix and verify](audit.md#fix-and-verify) safeguards. A timing target does not authorize deleting unique coverage.
|
|
6
|
+
|
|
7
|
+
- **Budget, unless the repo sets its own.** Measured on the CI runner, test execution only (checkout, install, database start and migrations excluded): the unit and integration lanes together take **3 minutes or less**, and the integration lane alone **under 2 minutes**. Neither lane drives a real browser; browser journeys belong to the E2E lane, which has its own budget of **5 minutes or less**. Over budget is a finding even when every test is sound.
|
|
8
|
+
- **Measure before cutting.** Get per-file and per-test durations from the runner (Vitest's JSON reporter or a small reporter over `TestModule.diagnostic()`, `pytest --durations=0`, `go test -json`). For CI, measure on the CI runner: a laptop is often faster and less CPU-bound, so local timings mislead.
|
|
9
|
+
- **Find per-test overhead before cutting tests.** Durations say which files are slow, not why. Trace what the slow tests start and wait on (child processes, containers, database setup, network, file system) and count it per test. A cost every test pays, such as a subprocess each write triggers or setup repeated per test instead of per file, usually outweighs every duplicate in the suite. Remove that cost first, then prove the behavior it guarded still has a test that fails without it.
|
|
10
|
+
- **Expect little from deleting fast tests.** Mocked unit tests run in milliseconds; removing duplicates is worth it for refactor noise, rarely for time. A few slow files usually dominate. Look in them for real-time waits ([principle 7](principles.md#7-same-code-same-result)) and combinatorial matrices.
|
|
11
|
+
- **Collapse a matrix by code path, then prove it.** Map each case to the production branch it exercises (which lock, which recheck, which input class). Keep one case per distinct path and drop cases that differ only in data that takes the same path. Prove each kept case by disabling its path in production code, watching that case fail, and restoring the code. End with no production diff.
|
|
12
|
+
- **Shorten a wait only when it stays deterministic.** Prefer faking the clock. When real elapsed time is the behavior, follow principle 7's look-alike for it instead of trimming the margin.
|
|
13
|
+
- **Leave runner tuning to CI work.** Shard counts, worker counts, and test pools change no test and belong to the CI workflow (the `github-actions` skill), measured on the runner's real CPU quota.
|
|
14
|
+
|
|
15
|
+
Report before/after timings from the same environment, the changes made, and the behavior checks that passed or remain unverified.
|