@avi2dg/checks 0.13.0 → 0.14.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +124 -0
- package/CONTRIBUTING.md +89 -0
- package/README.md +131 -1216
- package/docs/configs/commit-messages.md +43 -0
- package/docs/configs/dependency-rules.md +62 -0
- package/docs/configs/effect-rules.md +48 -0
- package/docs/configs/quality-file.md +99 -0
- package/docs/design.md +77 -0
- package/docs/gates/checks-backtest.md +50 -0
- package/docs/gates/checks-ci-wiring.md +122 -0
- package/docs/gates/checks-comment-gate.md +58 -0
- package/docs/gates/checks-commit-identity.md +63 -0
- package/docs/gates/checks-docs.md +98 -0
- package/docs/gates/checks-feature-owners.md +113 -0
- package/docs/gates/checks-flake.md +88 -0
- package/docs/gates/checks-lint-coverage.md +49 -0
- package/docs/gates/checks-lint.md +136 -0
- package/docs/gates/checks-mutation-compare.md +84 -0
- package/docs/gates/checks-quality.md +94 -0
- package/docs/gates/checks-size-budget.md +64 -0
- package/docs/gates/checks-suppressions-ratchet.md +51 -0
- package/docs/gates/checks-test-layout.md +77 -0
- package/docs/gates/checks-test.md +75 -0
- package/package.json +7 -2
- package/scripts/doc-rules.ts +4 -3
- package/scripts/doc-templates.ts +10 -5
- package/scripts/quality-file.ts +1 -1
- package/templates/changelog.md +5 -5
- package/templates/claude.md +1 -1
- package/templates/readme.md +3 -1
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
# checks-docs
|
|
2
|
+
|
|
3
|
+
`checks-docs` is the gate that holds each doc file a change touches to the template for its kind, and a reader looks it up to learn which template a file answers to.
|
|
4
|
+
|
|
5
|
+
## What it checks
|
|
6
|
+
|
|
7
|
+
It holds each doc file a change touches to the template for its kind, and lists every other doc file that does not conform yet without failing.
|
|
8
|
+
The package ships one template per kind under `templates/`, and a repository starts a new doc file by copying one:
|
|
9
|
+
|
|
10
|
+
```sh
|
|
11
|
+
cp node_modules/@avi2dg/checks/templates/how-to.md docs/add-a-supplier.md
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
<!-- generated doc-kinds: bun run build writes it from scripts/doc-rules.ts, scripts/quality-file.ts, scripts/doc-templates.ts and scripts/doc-blocks.ts -->
|
|
15
|
+
|
|
16
|
+
| File | Kind | Template |
|
|
17
|
+
| --- | --- | --- |
|
|
18
|
+
| `README.md` | readme | `templates/readme.md` |
|
|
19
|
+
| `CHANGELOG.md` | changelog | `templates/changelog.md` |
|
|
20
|
+
| `AGENTS.md` | agents | `templates/agents.md` |
|
|
21
|
+
| `CLAUDE.md` | claude | `templates/claude.md` |
|
|
22
|
+
| `CONTRIBUTING.md` | how-to | `templates/how-to.md` |
|
|
23
|
+
| each file in `docs/adr/` but its generated index, `README.md` | adr | `templates/adr.md` |
|
|
24
|
+
| a page `docs.pages` declares | tutorial, how-to, reference or explanation | `templates/<mode>.md` |
|
|
25
|
+
|
|
26
|
+
<!-- end generated doc-kinds -->
|
|
27
|
+
|
|
28
|
+
A file the table names on its own, such as `README.md`, sits at the repository root.
|
|
29
|
+
No other Markdown file is judged, save a page under `docs/`, which needs a mode.
|
|
30
|
+
Which Diátaxis mode a page is written in is a judgment, so `quality.json` declares it:
|
|
31
|
+
|
|
32
|
+
```json
|
|
33
|
+
"docs": {
|
|
34
|
+
"pages": {
|
|
35
|
+
"reference": ["docs/gates/*.md"],
|
|
36
|
+
"explanation": ["docs/design.md"]
|
|
37
|
+
}
|
|
38
|
+
}
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
A template decides a file's structure, and the template file itself is the reference for each kind:
|
|
42
|
+
|
|
43
|
+
- A file opens with one `# ` title on its first line and has text before its first section.
|
|
44
|
+
It skips no heading level, and no heading is Overview, Introduction or How it works.
|
|
45
|
+
- Its sections are the template's headings in the template's order.
|
|
46
|
+
A heading in angle brackets is one the writer names.
|
|
47
|
+
One marked verb first is left to review, since no program tells a verb from a noun there.
|
|
48
|
+
A heading the template does not have, in that place, is refused.
|
|
49
|
+
- A record in `docs/adr/` is named for its four-digit number, and its title opens with the same number.
|
|
50
|
+
A `Date: YYYY-MM-DD` line follows the title, the first word under Status is Proposed, Accepted, Rejected, Deprecated, Superseded or Retired, and no other record holds its number.
|
|
51
|
+
- A changelog lists its releases newest first, each opening with a `Released YYYY-MM-DD.` line.
|
|
52
|
+
- A how-to or tutorial page numbers its steps.
|
|
53
|
+
- `CLAUDE.md` is its template word for word.
|
|
54
|
+
It is a fixed agent pointer rather than a people doc, so the separator ban does not apply to it: people docs take no em dash, en dash, parenthesis or hyphen used as a dash, and no semicolon, and `checks-docs` does not check separators.
|
|
55
|
+
|
|
56
|
+
## What it reads
|
|
57
|
+
|
|
58
|
+
It reads each Markdown file from the head commit, and `docs.pages` from `quality.json`.
|
|
59
|
+
A file the range adds, changes or renames is held to its template, and a file it deletes is not.
|
|
60
|
+
|
|
61
|
+
## Arguments
|
|
62
|
+
|
|
63
|
+
```sh
|
|
64
|
+
checks-docs <base-ref> <head-ref>
|
|
65
|
+
checks-docs <ref>
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
With two arguments the range starts where the head branched from the base, at their merge-base.
|
|
69
|
+
With one it is that commit against its parent, or against the empty tree for a repository's first commit.
|
|
70
|
+
|
|
71
|
+
## Exit codes
|
|
72
|
+
|
|
73
|
+
| Code | When |
|
|
74
|
+
| --- | --- |
|
|
75
|
+
| 0 | every doc file the range touches holds to its template |
|
|
76
|
+
| 1 | a doc file the range touches does not |
|
|
77
|
+
| 2 | `quality.json` does not decode, or a ref does not resolve |
|
|
78
|
+
|
|
79
|
+
## Sample output
|
|
80
|
+
|
|
81
|
+
```
|
|
82
|
+
docs: 2 violation(s) in the doc files the range touches:
|
|
83
|
+
README.md:1: lacks `## Where things are`
|
|
84
|
+
docs/parts.md: is a page under docs/ with no mode; declare it under docs.pages in quality.json as tutorial, how-to, reference, explanation
|
|
85
|
+
docs: advisory, 1 doc file(s) the range leaves alone do not hold to their templates yet:
|
|
86
|
+
docs/adr/0001-quality-gates.md: 5 violation(s)
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
## Opting out
|
|
90
|
+
|
|
91
|
+
It applies to every repository, so no selection leaves it out.
|
|
92
|
+
A file the range leaves alone is only listed as advisory, so a repository adopts the templates as its files change.
|
|
93
|
+
`checks-lint` runs it over each pull request's range, as [checks-lint](checks-lint.md) says.
|
|
94
|
+
|
|
95
|
+
## Related topics
|
|
96
|
+
|
|
97
|
+
- [Why it is shaped this way](../design.md)
|
|
98
|
+
- [checks-lint](checks-lint.md)
|
|
@@ -0,0 +1,113 @@
|
|
|
1
|
+
# checks-feature-owners
|
|
2
|
+
|
|
3
|
+
`checks-feature-owners` is the gate that holds each declared feature to a runnable proof and lists the feature owners a change touches, and a reader looks it up to declare a feature.
|
|
4
|
+
|
|
5
|
+
## What it checks
|
|
6
|
+
|
|
7
|
+
A repository opts a feature in by declaring, in `quality.json`, the directory it owns, the files code outside it imports it through, the files that may reach past those, and the end-to-end test that proves it runs:
|
|
8
|
+
|
|
9
|
+
```json
|
|
10
|
+
"features": [
|
|
11
|
+
{
|
|
12
|
+
"name": "billing",
|
|
13
|
+
"root": "src/billing",
|
|
14
|
+
"entries": ["src/billing/index.ts"],
|
|
15
|
+
"allowFrom": ["src/main.ts", "src/cli/*.ts"],
|
|
16
|
+
"proof": "tests/e2e/billing.test.ts"
|
|
17
|
+
}
|
|
18
|
+
],
|
|
19
|
+
"changeSignal": "advisory"
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
`root` is a directory and `entries` are files under it, both without globs.
|
|
23
|
+
`allowFrom` holds globs of the same shape as `sources`.
|
|
24
|
+
`proof` is a `.test.ts` or `.test.tsx` file under `tests/e2e/`.
|
|
25
|
+
Nothing moves, since a root is wherever the feature already lives.
|
|
26
|
+
`quality.json` refuses an entry outside its root, a name used twice, and two features sharing a root or one root inside another, so a file has at most one owner.
|
|
27
|
+
|
|
28
|
+
The gate fails when a feature's proof cannot prove it:
|
|
29
|
+
|
|
30
|
+
- the proof or an entry is not in the head commit
|
|
31
|
+
- the proof does not parse
|
|
32
|
+
- the proof imports none of the feature's entries
|
|
33
|
+
|
|
34
|
+
An import counts when it is a runtime `import`, `export ... from` or `export * from` of a relative path that names an entry.
|
|
35
|
+
It names the entry by its own name, by the `.js`, `.jsx`, `.mjs` or `.cjs` spelling of it, or without an extension, the way a directory `index` is imported.
|
|
36
|
+
`.js` names a `.tsx` entry as well as a `.ts` one.
|
|
37
|
+
`import type` does not count, and neither does a path alias.
|
|
38
|
+
The proof runs in `bun run test` like any end-to-end test, which is what shows it passes.
|
|
39
|
+
|
|
40
|
+
With `changeSignal` set to `advisory` it also lists each owner the range touches, with the paths it touched under the owner's root or at its proof, and still passes.
|
|
41
|
+
Whether a change that spans owners is one coherent slice is for a reviewer to judge.
|
|
42
|
+
A rename counts at both of its paths.
|
|
43
|
+
|
|
44
|
+
## What it reads
|
|
45
|
+
|
|
46
|
+
It reads `features` and `changeSignal` from `quality.json`, each proof and entry from the head commit, and the paths the range touches.
|
|
47
|
+
|
|
48
|
+
## Arguments
|
|
49
|
+
|
|
50
|
+
```sh
|
|
51
|
+
checks-feature-owners <base-ref> <head-ref>
|
|
52
|
+
checks-feature-owners <ref>
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
With two arguments the range starts where the head branched from the base, at their merge-base.
|
|
56
|
+
With one it is that commit against its parent, or against the empty tree for a repository's first commit.
|
|
57
|
+
|
|
58
|
+
## Exit codes
|
|
59
|
+
|
|
60
|
+
| Code | When |
|
|
61
|
+
| --- | --- |
|
|
62
|
+
| 0 | every feature's proof imports one of its entries |
|
|
63
|
+
| 1 | a feature's proof cannot prove it |
|
|
64
|
+
| 2 | `quality.json` does not decode, which is where a proof outside `tests/e2e/` is refused, or a ref does not resolve |
|
|
65
|
+
|
|
66
|
+
## Sample output
|
|
67
|
+
|
|
68
|
+
```
|
|
69
|
+
feature-owners: 1 problem(s) with the features' runnable proofs:
|
|
70
|
+
billing: proof tests/e2e/billing.test.ts imports none of its entries, src/billing/index.ts
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
With `changeSignal` set to `advisory`:
|
|
74
|
+
|
|
75
|
+
```
|
|
76
|
+
feature-owners: advisory, the range touches 2 feature owner(s); a reviewer judges whether they make one slice:
|
|
77
|
+
billing: src/billing/charge.ts, src/billing/tax.ts
|
|
78
|
+
invoices: src/invoices/tax.ts, tests/e2e/invoices.test.ts
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
## Opting out
|
|
82
|
+
|
|
83
|
+
A repository that declares no feature passes.
|
|
84
|
+
`quality.json` refuses a `changeSignal` without features, which would map a change to no owner.
|
|
85
|
+
A repository that tracks no `.ts` or `.tsx` file leaves it out of `gates.lint`, as [Gate selection](checks-lint.md#gate-selection) says.
|
|
86
|
+
|
|
87
|
+
## Import boundary
|
|
88
|
+
|
|
89
|
+
`dist/feature-rules.js` compiles `features` into one dependency-cruiser rule per feature, which `.dependency-cruiser.cjs` spreads beside its own:
|
|
90
|
+
|
|
91
|
+
```js
|
|
92
|
+
const { featureRules } = require("@avi2dg/checks/dist/feature-rules.js");
|
|
93
|
+
|
|
94
|
+
module.exports = {
|
|
95
|
+
extends: "./node_modules/@avi2dg/checks/dependency-cruiser.config.js",
|
|
96
|
+
forbidden: [...featureRules(require("./quality.json"))],
|
|
97
|
+
};
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
A module outside a feature's root that imports a file inside it must import one of the feature's `entries`.
|
|
101
|
+
Modules under `tests/` and the files `allowFrom` matches, such as a CLI or a harness, may import any file in it:
|
|
102
|
+
|
|
103
|
+
```
|
|
104
|
+
error feature-billing-entries: src/report.ts → src/billing/charge.ts
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
`featureRules` decodes its argument with the schema the bins use and throws the schema's refusal when it does not decode, which stops the cruise.
|
|
108
|
+
It is an ES module, as `effect` is, so a `.cjs` config loads it through `require`, which needs node 20.19, 22.12 or later.
|
|
109
|
+
|
|
110
|
+
## Related topics
|
|
111
|
+
|
|
112
|
+
- [The dependency rules](../configs/dependency-rules.md)
|
|
113
|
+
- [The quality file](../configs/quality-file.md)
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
# checks-flake
|
|
2
|
+
|
|
3
|
+
`checks-flake` is the scheduled run that finds flaky tests and records the seeds each one fails with, and a reader looks it up to reproduce a flaky failure.
|
|
4
|
+
|
|
5
|
+
## What it checks
|
|
6
|
+
|
|
7
|
+
A green run proves nothing failed in that run, not that no test is flaky.
|
|
8
|
+
`checks-flake` runs the whole suite several times, each with its own `--seed`, and records per failing test the seeds it failed with.
|
|
9
|
+
`bun test --randomize --seed=<seed>` puts the suite in the same order, so a seed reproduces a failure that hangs on order.
|
|
10
|
+
A run that fails with no failing test, such as a test file that throws while loading, is listed with its seed on its own line.
|
|
11
|
+
|
|
12
|
+
## What it reads
|
|
13
|
+
|
|
14
|
+
It reads bun's JUnit report of each run.
|
|
15
|
+
Under GitHub Actions it appends its summary to the file `GITHUB_STEP_SUMMARY` names, which is the job summary.
|
|
16
|
+
|
|
17
|
+
## Arguments
|
|
18
|
+
|
|
19
|
+
```sh
|
|
20
|
+
checks-flake [--runs <count> | --seed <seed>...] [--report <file>]
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
`--runs` sets the number of runs on random seeds, 10 when absent.
|
|
24
|
+
`--seed`, given once per run, replays chosen seeds, such as the ones a report recorded.
|
|
25
|
+
`--report` writes the record as JSON, holding every run's seed and failing tests and every failing test's seeds.
|
|
26
|
+
|
|
27
|
+
## Exit codes
|
|
28
|
+
|
|
29
|
+
| Code | When |
|
|
30
|
+
| --- | --- |
|
|
31
|
+
| 0 | every run passed |
|
|
32
|
+
| 1 | a run failed |
|
|
33
|
+
| 2 | an argument does not parse, or bun passed without writing its report |
|
|
34
|
+
|
|
35
|
+
## Sample output
|
|
36
|
+
|
|
37
|
+
```
|
|
38
|
+
checks-flake: 3 of 10 run(s) failed, 1 test(s) failing in them
|
|
39
|
+
|
|
40
|
+
| Test | Failed | Seeds |
|
|
41
|
+
| --- | --- | --- |
|
|
42
|
+
| tests/cache.test.ts:6 reads the cache | 3 of 10 runs | 2170533150, 4046124386, 180394251 |
|
|
43
|
+
|
|
44
|
+
Reproduce a failing run with bun test --randomize --seed=<seed>.
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
## Opting out
|
|
48
|
+
|
|
49
|
+
Nothing runs it but a schedule the repository writes.
|
|
50
|
+
A repository that stops scheduling it also drops it from `gates.scheduled`, which `checks-ci-wiring` otherwise fails on.
|
|
51
|
+
|
|
52
|
+
## Running it on a schedule
|
|
53
|
+
|
|
54
|
+
A repository runs it on a schedule and keeps the record as an artifact:
|
|
55
|
+
|
|
56
|
+
```yaml
|
|
57
|
+
on:
|
|
58
|
+
schedule:
|
|
59
|
+
- cron: "17 5 * * *"
|
|
60
|
+
workflow_dispatch:
|
|
61
|
+
jobs:
|
|
62
|
+
flake:
|
|
63
|
+
runs-on: ubuntu-latest
|
|
64
|
+
steps:
|
|
65
|
+
- uses: actions/checkout@v5
|
|
66
|
+
- uses: oven-sh/setup-bun@v2
|
|
67
|
+
- run: bun install --frozen-lockfile
|
|
68
|
+
- run: bunx checks-flake --runs 10 --report flake-report.json
|
|
69
|
+
- uses: actions/upload-artifact@v4
|
|
70
|
+
if: always()
|
|
71
|
+
with:
|
|
72
|
+
name: flake-report
|
|
73
|
+
path: flake-report.json
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
It declares the step in `gates.scheduled`, so `checks-ci-wiring` fails once the schedule stops running it:
|
|
77
|
+
|
|
78
|
+
```json
|
|
79
|
+
"gates": {
|
|
80
|
+
"ci": ["bun run lint", "bun run typecheck", "bun run test"],
|
|
81
|
+
"scheduled": ["bunx checks-flake --runs 10 --report flake-report.json"]
|
|
82
|
+
}
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
## Related topics
|
|
86
|
+
|
|
87
|
+
- [checks-test](checks-test.md)
|
|
88
|
+
- [checks-ci-wiring](checks-ci-wiring.md)
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
# checks-lint-coverage
|
|
2
|
+
|
|
3
|
+
`checks-lint-coverage` is the gate that fails when oxlint silently skips a tracked TypeScript file, and a reader looks it up when a file seems never to be linted.
|
|
4
|
+
|
|
5
|
+
## What it checks
|
|
6
|
+
|
|
7
|
+
It fails when oxlint skips a tracked `.ts` or `.tsx` file, for example through a stray `.gitignore` entry.
|
|
8
|
+
It compares `git ls-files` against oxlint's own file walk and names the missing files.
|
|
9
|
+
|
|
10
|
+
## What it reads
|
|
11
|
+
|
|
12
|
+
It reads the working tree: the `*.ts` and `*.tsx` files `git ls-files` lists, and the files `oxlint --debug=files` walks.
|
|
13
|
+
It walks without naming a path, since an explicit path bypasses the ignore files whose skips it looks for.
|
|
14
|
+
oxlint must be on `PATH`, as it is under a package script.
|
|
15
|
+
|
|
16
|
+
## Arguments
|
|
17
|
+
|
|
18
|
+
It takes none.
|
|
19
|
+
|
|
20
|
+
## Exit codes
|
|
21
|
+
|
|
22
|
+
| Code | When |
|
|
23
|
+
| --- | --- |
|
|
24
|
+
| 0 | oxlint walks every tracked `.ts` and `.tsx` file, or the repository tracks none |
|
|
25
|
+
| 1 | oxlint skips a tracked file |
|
|
26
|
+
| 2 | oxlint cannot walk the tree, as when it is not on `PATH` or its config does not parse |
|
|
27
|
+
|
|
28
|
+
## Sample output
|
|
29
|
+
|
|
30
|
+
```
|
|
31
|
+
lint-coverage: oxlint skips 1/3 tracked .ts/.tsx files; missing:
|
|
32
|
+
ignored/b.ts
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
A passing run counts the files:
|
|
36
|
+
|
|
37
|
+
```
|
|
38
|
+
lint-coverage: 71/71 tracked .ts/.tsx files
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
## Opting out
|
|
42
|
+
|
|
43
|
+
A repository that tracks no `.ts` or `.tsx` file leaves it out of `gates.lint`, as [Gate selection](checks-lint.md#gate-selection) says.
|
|
44
|
+
A repository that tracks one keeps it.
|
|
45
|
+
|
|
46
|
+
## Related topics
|
|
47
|
+
|
|
48
|
+
- [checks-lint](checks-lint.md)
|
|
49
|
+
- [The Effect rules](../configs/effect-rules.md)
|
|
@@ -0,0 +1,136 @@
|
|
|
1
|
+
# checks-lint
|
|
2
|
+
|
|
3
|
+
`checks-lint` is the entry point that runs every lint gate of the kit over one range, and a reader looks it up to learn which range it resolves and how a repository selects its gates.
|
|
4
|
+
|
|
5
|
+
## What it checks
|
|
6
|
+
|
|
7
|
+
It runs each gate the table under [What runs](../../README.md#what-runs) lists, in that table's order, and names every one that fails rather than stopping at the first.
|
|
8
|
+
Each gate runs as its own bin in a child process, with its output passed straight through, so a gate behaves the same called alone or through `checks-lint`.
|
|
9
|
+
`checks-ci-wiring` always runs, so a repository on `checks-lint` declares `gates.ci`, as [checks-ci-wiring](checks-ci-wiring.md) says.
|
|
10
|
+
It holds the test layout unless the selection leaves out `checks-test-layout`.
|
|
11
|
+
|
|
12
|
+
## What it reads
|
|
13
|
+
|
|
14
|
+
It reads `quality.json` for `defaultBranch` and `gates.lint`, and resolves the range once, handing the same one to every range gate.
|
|
15
|
+
A tree gate reads the working tree and is handed no range.
|
|
16
|
+
|
|
17
|
+
Locally, and on any event other than a pull request, the range ends at `HEAD` and starts where `HEAD` branched from the origin default branch.
|
|
18
|
+
That branch is `origin/HEAD`.
|
|
19
|
+
When `origin/HEAD` is not set, as in an `actions/checkout` clone, it is `origin/<defaultBranch>` from `quality.json`, and `origin/main` when that is not declared.
|
|
20
|
+
|
|
21
|
+
In a GitHub Actions pull request, where `GITHUB_EVENT_NAME` is `pull_request`, the range ends at the event's head sha and starts where that branched from `origin/<base branch>`, so GitHub's merge commit is never in it.
|
|
22
|
+
The base branch is read from the fetch, not from the event's recorded base sha, which GitHub leaves stale once the base branch advances after the pull request opens.
|
|
23
|
+
|
|
24
|
+
The range always starts at the merge base, never at the base branch's tip, since commits the base branch gained after the head branched off would otherwise count against the head.
|
|
25
|
+
When the head is the merge base, as on a push to the default branch or a local run on it, the range would be empty.
|
|
26
|
+
Each range gate is then handed that tip commit alone, and checks it against its parent, or against the empty tree when it is a repository's first commit:
|
|
27
|
+
|
|
28
|
+
```
|
|
29
|
+
checks-lint: tip 10ba7d8935b73ed72624120a1542e51bd21ca7c7 from HEAD against origin/main
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
A clone with no remote-tracking refs at all has no default branch to start from, such as a freshly initialised repository with no remote or one whose remote was never fetched.
|
|
33
|
+
Each range gate is then handed `HEAD` alone the same way:
|
|
34
|
+
|
|
35
|
+
```
|
|
36
|
+
checks-lint: tip 10ba7d8935b73ed72624120a1542e51bd21ca7c7 from HEAD alone, as the clone has no remote-tracking refs
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
Once any ref sits under `refs/remotes/`, a missing `origin/<default branch>` exits 2 instead.
|
|
40
|
+
That is the shape of a shallow CI checkout, where `HEAD` alone would leave the commits before it unchecked.
|
|
41
|
+
|
|
42
|
+
## Arguments
|
|
43
|
+
|
|
44
|
+
```sh
|
|
45
|
+
checks-lint
|
|
46
|
+
checks-lint <base-ref> <head-ref>
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
With no arguments it resolves the range as What it reads says.
|
|
50
|
+
A base and a head override that resolution.
|
|
51
|
+
|
|
52
|
+
## Exit codes
|
|
53
|
+
|
|
54
|
+
| Code | When |
|
|
55
|
+
| --- | --- |
|
|
56
|
+
| 0 | every gate passed |
|
|
57
|
+
| 1 | a gate found a violation |
|
|
58
|
+
| 2 | the range or the selection does not resolve, or no failing gate could decide |
|
|
59
|
+
|
|
60
|
+
## Sample output
|
|
61
|
+
|
|
62
|
+
It prints the range, the declared selection if there is one, each gate's own report, then its verdict:
|
|
63
|
+
|
|
64
|
+
```
|
|
65
|
+
checks-lint: range 2504acf098d120e73a8ece3c96f22b934f35c6a8..10ba7d8935b73ed72624120a1542e51bd21ca7c7 from HEAD against origin/main
|
|
66
|
+
...
|
|
67
|
+
checks-lint: 3 of 10 gate(s) failed: checks-commit-identity, checks-comment-gate, checks-suppressions-ratchet
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
## Opting out
|
|
71
|
+
|
|
72
|
+
A repository leaves out a gate that does not apply to it, as Gate selection says.
|
|
73
|
+
A repository that runs its gates without `checks-lint` calls each gate's bin in its own CI step and declares each command in `gates.ci`.
|
|
74
|
+
|
|
75
|
+
## Gate selection
|
|
76
|
+
|
|
77
|
+
A repository with no TypeScript source gives `checks-lint-coverage`, `checks-test-layout`, `checks-size-budget` and `checks-feature-owners` nothing to check, and test-layout would still refuse its missing `bun test` script and `bunfig.toml`.
|
|
78
|
+
It declares the gates `checks-lint` runs as `gates.lint`:
|
|
79
|
+
|
|
80
|
+
```json
|
|
81
|
+
"gates": {
|
|
82
|
+
"ci": ["bun run lint"],
|
|
83
|
+
"lint": [
|
|
84
|
+
"checks-commit-identity",
|
|
85
|
+
"checks-comment-gate",
|
|
86
|
+
"checks-suppressions-ratchet",
|
|
87
|
+
"checks-ci-wiring",
|
|
88
|
+
"checks-docs",
|
|
89
|
+
"checks-quality"
|
|
90
|
+
]
|
|
91
|
+
}
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
`checks-lint` runs exactly those, in the order of the table under What runs, and every gate when `gates.lint` is absent.
|
|
95
|
+
A selection in `quality.json` always keeps `checks-quality`, since the file it sits in is what makes that gate apply.
|
|
96
|
+
A step running `checks-lint` then counts only for a declared gate that `gates.lint` keeps.
|
|
97
|
+
|
|
98
|
+
A selection may leave out only a gate that does not apply, and the Runs in column of the table under What runs says where each gate applies.
|
|
99
|
+
Both `checks-lint` and `checks-ci-wiring` exit 2 on a `gates.lint` that names an unknown gate or leaves out one that applies to every repository.
|
|
100
|
+
`checks-ci-wiring` exits 1 when the selection leaves out a gate the repository's tracked files make applicable, and names the gate and the files:
|
|
101
|
+
|
|
102
|
+
```
|
|
103
|
+
ci-wiring: quality.json gates.lint leaves out 4 gate(s) this repository's contents make applicable:
|
|
104
|
+
checks-lint-coverage: the repository tracks TypeScript source (src/widget.ts)
|
|
105
|
+
checks-test-layout: the repository tracks TypeScript source (src/widget.ts)
|
|
106
|
+
checks-size-budget: the repository tracks TypeScript source (src/widget.ts)
|
|
107
|
+
checks-feature-owners: the repository tracks TypeScript source (src/widget.ts)
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
It reads the files tracked at the checkout, so the pull request that adds the first TypeScript file is the one refused.
|
|
111
|
+
|
|
112
|
+
## Running it in CI
|
|
113
|
+
|
|
114
|
+
CI runs it through `lint`.
|
|
115
|
+
The checkout fetches the whole history, which the merge base needs:
|
|
116
|
+
|
|
117
|
+
```yaml
|
|
118
|
+
on:
|
|
119
|
+
pull_request:
|
|
120
|
+
jobs:
|
|
121
|
+
lint:
|
|
122
|
+
runs-on: ubuntu-latest
|
|
123
|
+
steps:
|
|
124
|
+
- uses: actions/checkout@v5
|
|
125
|
+
with:
|
|
126
|
+
fetch-depth: 0
|
|
127
|
+
- uses: oven-sh/setup-bun@v2
|
|
128
|
+
- run: bun install --frozen-lockfile
|
|
129
|
+
- run: bun run lint
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
## Related topics
|
|
133
|
+
|
|
134
|
+
- [What runs](../../README.md#what-runs)
|
|
135
|
+
- [checks-ci-wiring](checks-ci-wiring.md)
|
|
136
|
+
- [The quality file](../configs/quality-file.md)
|
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
# checks-mutation-compare
|
|
2
|
+
|
|
3
|
+
`checks-mutation-compare` is the gate that holds a pull request's mutation score to no regression rather than an absolute threshold, and a reader looks it up to wire mutation testing into CI.
|
|
4
|
+
|
|
5
|
+
## What it checks
|
|
6
|
+
|
|
7
|
+
The head mutation score may not fall below the score at the merge-base.
|
|
8
|
+
The score is Stryker's, `Killed` and `Timeout` over those plus `Survived` and `NoCoverage`, so `CompileError`, `RuntimeError`, `Ignored` and `Pending` mutants leave it.
|
|
9
|
+
Every file in a report counts toward that report's score, including files present in only one of the two.
|
|
10
|
+
|
|
11
|
+
## What it reads
|
|
12
|
+
|
|
13
|
+
It reads two Stryker `mutation.json` reports, one built at the merge-base and one at the head.
|
|
14
|
+
The shared Stryker preset's `json` reporter writes `reports/mutation/mutation.json` in each worktree.
|
|
15
|
+
|
|
16
|
+
A repository that runs mutation testing installs `@stryker-mutator/core` and `@hughescr/stryker-bun-runner`, then spreads the shipped preset in `stryker.conf.mjs`:
|
|
17
|
+
|
|
18
|
+
```js
|
|
19
|
+
import preset from "@avi2dg/checks/stryker.preset.js";
|
|
20
|
+
|
|
21
|
+
export default {
|
|
22
|
+
...preset,
|
|
23
|
+
};
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
Keys the repository sets after the spread win.
|
|
27
|
+
|
|
28
|
+
## Arguments
|
|
29
|
+
|
|
30
|
+
```sh
|
|
31
|
+
checks-mutation-compare [--advisory] <base-report> <head-report>
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
`--advisory`, anywhere among the arguments, prints the same verdict and always exits 0.
|
|
35
|
+
|
|
36
|
+
## Exit codes
|
|
37
|
+
|
|
38
|
+
| Code | When |
|
|
39
|
+
| --- | --- |
|
|
40
|
+
| 0 | the head score is not below the base score, or `--advisory` is given |
|
|
41
|
+
| 1 | the head score is below the base score |
|
|
42
|
+
| 2 | a report is not a Stryker mutation report, or the arguments do not parse |
|
|
43
|
+
|
|
44
|
+
## Sample output
|
|
45
|
+
|
|
46
|
+
It prints the overall score of each report and the per-file scores that differ:
|
|
47
|
+
|
|
48
|
+
```
|
|
49
|
+
mutation-compare: base 83.33% (5/6) head 66.67% (4/6) delta -16.67pp
|
|
50
|
+
src/billing.ts: 75.00% (3/4) -> 50.00% (2/4)
|
|
51
|
+
1 unchanged file(s)
|
|
52
|
+
mutation-compare: REGRESSION (-16.67pp)
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
## Opting out
|
|
56
|
+
|
|
57
|
+
Nothing runs it but a CI step the repository writes.
|
|
58
|
+
A repository runs it with `--advisory` for its first month, then drops the flag so it blocks.
|
|
59
|
+
|
|
60
|
+
## Running it in CI
|
|
61
|
+
|
|
62
|
+
It runs on pull requests, comparing the head report against a report built at the merge-base:
|
|
63
|
+
|
|
64
|
+
```yaml
|
|
65
|
+
jobs:
|
|
66
|
+
mutation-compare:
|
|
67
|
+
runs-on: ubuntu-latest
|
|
68
|
+
steps:
|
|
69
|
+
- uses: actions/checkout@v5
|
|
70
|
+
with:
|
|
71
|
+
fetch-depth: 0
|
|
72
|
+
- uses: oven-sh/setup-bun@v2
|
|
73
|
+
- run: bun install --frozen-lockfile
|
|
74
|
+
- run: bunx stryker run
|
|
75
|
+
- run: |
|
|
76
|
+
base="$(git merge-base HEAD origin/main)"
|
|
77
|
+
git worktree add /tmp/mutation-base "$base"
|
|
78
|
+
(cd /tmp/mutation-base && bun install --frozen-lockfile && bunx stryker run)
|
|
79
|
+
- run: bun run checks-mutation-compare --advisory /tmp/mutation-base/reports/mutation/mutation.json reports/mutation/mutation.json
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
## Related topics
|
|
83
|
+
|
|
84
|
+
- [checks-test-layout](checks-test-layout.md)
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
# checks-quality
|
|
2
|
+
|
|
3
|
+
`checks-quality` is the bin that writes what `quality.json` declares for oxlint and tsc into generated fragments and checks them, and a reader looks it up when a fragment is stale.
|
|
4
|
+
|
|
5
|
+
## What it checks
|
|
6
|
+
|
|
7
|
+
oxlint and tsc read their own JSON and nothing else, so `checks-quality generate` writes what `sources.effect` declares into two fragments at the repository root, and the hand-written configs extend them.
|
|
8
|
+
Both fragments are committed.
|
|
9
|
+
|
|
10
|
+
`oxlintrc.quality.json` holds one override: the declared paths as `files`, the exempt ones as `excludeFiles`, and the kit's Effect rule block, `presets/effect.oxlint.json`.
|
|
11
|
+
`tsconfig.quality.json` holds the language-service override: the same paths as `include`, the exempt ones as `exclude`, and `presets/effect.language-service.json`.
|
|
12
|
+
A kit release that changes a preset reaches the repository through its next `generate`.
|
|
13
|
+
A rule only this repository needs stays in its own `.oxlintrc.json`, whose overrides come after the fragment's and so win.
|
|
14
|
+
|
|
15
|
+
`.oxlintrc.json` extends its fragment:
|
|
16
|
+
|
|
17
|
+
```json
|
|
18
|
+
{
|
|
19
|
+
"extends": ["./node_modules/@avi2dg/checks/oxlintrc.json", "./oxlintrc.quality.json"],
|
|
20
|
+
"plugins": ["typescript", "oxc", "eslint", "import"]
|
|
21
|
+
}
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
`tsconfig.json` extends its fragment:
|
|
25
|
+
|
|
26
|
+
```json
|
|
27
|
+
{
|
|
28
|
+
"extends": ["@avi2dg/checks/tsconfig.effect.json", "./tsconfig.quality.json"]
|
|
29
|
+
}
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
`checks-quality --check` fails when any of these holds:
|
|
33
|
+
|
|
34
|
+
- A fragment is missing, or differs from what `generate` would write from `quality.json` and the installed kit's presets.
|
|
35
|
+
- A fragment is left over once `quality.json` stops declaring `sources.effect`.
|
|
36
|
+
- `.oxlintrc.json` or `tsconfig.json` does not list its fragment in `extends`, so the tool never reads it.
|
|
37
|
+
- A `sources.effect.paths` glob, or a `sources.production` glob while `size` is declared, matches no tracked or untracked file, so it holds nothing.
|
|
38
|
+
|
|
39
|
+
Two details of the fragments are easy to get wrong, so the kit's tests pin both:
|
|
40
|
+
|
|
41
|
+
- A fragment sits at the repository root.
|
|
42
|
+
oxlint resolves an override's `files`, and the language service an override's `include`, against the directory of the config holding it.
|
|
43
|
+
From `.quality/` the language service reports no error at all on an `async function` planted under a declared path.
|
|
44
|
+
- The oxlint fragment always sets `plugins`, to the kit's.
|
|
45
|
+
A config in `extends` that sets none brings in oxlint's default plugins, whose category rules then fire across the whole tree.
|
|
46
|
+
The override names the kit's plugins beside the preset's `node`, `promise` and `unicorn`, because one that leaves any of the kit's out turns on the category rules of the plugins it adds under every declared path.
|
|
47
|
+
|
|
48
|
+
## What it reads
|
|
49
|
+
|
|
50
|
+
It reads the working tree: `quality.json`, the presets of the installed kit, `.oxlintrc.json`, `tsconfig.json` and the two fragments.
|
|
51
|
+
It lists the tracked and untracked files to see what each declared glob matches.
|
|
52
|
+
|
|
53
|
+
## Arguments
|
|
54
|
+
|
|
55
|
+
```sh
|
|
56
|
+
checks-quality generate
|
|
57
|
+
checks-quality --check
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
`generate` writes the fragments, removes a left-over one, then runs the same check as `--check`.
|
|
61
|
+
`--check` writes nothing.
|
|
62
|
+
|
|
63
|
+
## Exit codes
|
|
64
|
+
|
|
65
|
+
| Code | When |
|
|
66
|
+
| --- | --- |
|
|
67
|
+
| 0 | the fragments hold what `quality.json` declares |
|
|
68
|
+
| 1 | a fragment is stale, missing, left over or not extended, or a declared glob matches no file |
|
|
69
|
+
| 2 | `quality.json` does not decode, or the arguments are neither `generate` nor `--check` |
|
|
70
|
+
|
|
71
|
+
## Sample output
|
|
72
|
+
|
|
73
|
+
```
|
|
74
|
+
checks-quality: 2 problem(s) with what quality.json declares:
|
|
75
|
+
oxlintrc.quality.json is stale against quality.json and the kit's presets; run checks-quality generate
|
|
76
|
+
tsconfig.json does not extend ./tsconfig.quality.json, so the language service never reads it
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
A passing run says what the fragments hold:
|
|
80
|
+
|
|
81
|
+
```
|
|
82
|
+
checks-quality: oxlintrc.quality.json and tsconfig.quality.json hold what quality.json declares
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
## Opting out
|
|
86
|
+
|
|
87
|
+
It runs only in a repository that tracks `quality.json`, and a selection in `quality.json` always keeps it, since the file it sits in is what makes it apply.
|
|
88
|
+
A repository that declares no `sources.effect` gets no fragment, and the check then only refuses a left-over one.
|
|
89
|
+
|
|
90
|
+
## Related topics
|
|
91
|
+
|
|
92
|
+
- [The quality file](../configs/quality-file.md)
|
|
93
|
+
- [The Effect rules](../configs/effect-rules.md)
|
|
94
|
+
- [checks-lint](checks-lint.md)
|