@avi2dg/checks 0.12.0 → 0.14.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +124 -0
- package/CONTRIBUTING.md +89 -0
- package/README.md +137 -1230
- package/dist/feature-rules.js +14 -1
- package/docs/configs/commit-messages.md +43 -0
- package/docs/configs/dependency-rules.md +62 -0
- package/docs/configs/effect-rules.md +48 -0
- package/docs/configs/quality-file.md +99 -0
- package/docs/design.md +77 -0
- package/docs/gates/checks-backtest.md +50 -0
- package/docs/gates/checks-ci-wiring.md +122 -0
- package/docs/gates/checks-comment-gate.md +58 -0
- package/docs/gates/checks-commit-identity.md +63 -0
- package/docs/gates/checks-docs.md +98 -0
- package/docs/gates/checks-feature-owners.md +113 -0
- package/docs/gates/checks-flake.md +88 -0
- package/docs/gates/checks-lint-coverage.md +49 -0
- package/docs/gates/checks-lint.md +136 -0
- package/docs/gates/checks-mutation-compare.md +84 -0
- package/docs/gates/checks-quality.md +94 -0
- package/docs/gates/checks-size-budget.md +64 -0
- package/docs/gates/checks-suppressions-ratchet.md +51 -0
- package/docs/gates/checks-test-layout.md +77 -0
- package/docs/gates/checks-test.md +75 -0
- package/package.json +24 -3
- package/quality.schema.json +48 -0
- package/scripts/doc-outline.ts +215 -0
- package/scripts/doc-rules.ts +177 -0
- package/scripts/doc-templates.ts +208 -0
- package/scripts/docs.ts +90 -0
- package/scripts/gates.ts +1 -0
- package/scripts/quality-file.ts +22 -1
- package/templates/adr.md +29 -0
- package/templates/agents.md +17 -0
- package/templates/changelog.md +37 -0
- package/templates/claude.md +2 -0
- package/templates/explanation.md +15 -0
- package/templates/how-to.md +33 -0
- package/templates/readme.md +45 -0
- package/templates/reference.md +15 -0
- package/templates/tutorial.md +31 -0
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
# checks-commit-identity
|
|
2
|
+
|
|
3
|
+
`checks-commit-identity` is the gate that refuses a commit carrying an identity other than the repository owner's, and a reader looks it up to allow another author.
|
|
4
|
+
|
|
5
|
+
## What it checks
|
|
6
|
+
|
|
7
|
+
It walks every commit in a range and fails when one carries an identity outside the allowlist, naming the offending commit and the reason.
|
|
8
|
+
It refuses a commit whose author or committer is outside the allowlist.
|
|
9
|
+
It refuses a commit whose trailer block carries a `Co-authored-by:` trailer, as git parses it.
|
|
10
|
+
Other trailers, and prose in the body that mentions an address, are left alone.
|
|
11
|
+
`GitHub <noreply@github.com>` is allowed as committer only, since that is who writes a squash merge.
|
|
12
|
+
|
|
13
|
+
## What it reads
|
|
14
|
+
|
|
15
|
+
It reads the author, the committer and the trailers of each commit in the range, and `commitIdentity.authors` from `quality.json`.
|
|
16
|
+
The allowlist is `avi2d <avi2dg@gmail.com>` when `quality.json` declares none.
|
|
17
|
+
|
|
18
|
+
## Arguments
|
|
19
|
+
|
|
20
|
+
```sh
|
|
21
|
+
checks-commit-identity <base-ref> <head-ref>
|
|
22
|
+
checks-commit-identity <ref>
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
With two arguments it checks every commit the head holds and the base does not.
|
|
26
|
+
With one it checks that commit alone.
|
|
27
|
+
|
|
28
|
+
## Exit codes
|
|
29
|
+
|
|
30
|
+
| Code | When |
|
|
31
|
+
| --- | --- |
|
|
32
|
+
| 0 | every commit carries only allowed identities |
|
|
33
|
+
| 1 | a commit carries a foreign identity |
|
|
34
|
+
| 2 | a ref does not resolve, the arguments do not parse, or `quality.json` does not decode |
|
|
35
|
+
|
|
36
|
+
## Sample output
|
|
37
|
+
|
|
38
|
+
```
|
|
39
|
+
commit-identity: 1 of 1 commit(s) in HEAD carry a foreign identity:
|
|
40
|
+
79fa94f9f027 feat: foreign
|
|
41
|
+
author someone <someone@example.com>
|
|
42
|
+
committer someone <someone@example.com>
|
|
43
|
+
allowed: avi2d <avi2dg@gmail.com>
|
|
44
|
+
allowed as committer only: GitHub <noreply@github.com>
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
## Opting out
|
|
48
|
+
|
|
49
|
+
It applies to every repository, so no selection leaves it out.
|
|
50
|
+
A repository with other owners restates the allowlist in `quality.json`:
|
|
51
|
+
|
|
52
|
+
```json
|
|
53
|
+
"commitIdentity": {
|
|
54
|
+
"authors": [{ "name": "avi2d", "email": "avi2dg@gmail.com" }]
|
|
55
|
+
}
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
`checks-lint` runs it over each pull request's range, as [checks-lint](checks-lint.md) says.
|
|
59
|
+
|
|
60
|
+
## Related topics
|
|
61
|
+
|
|
62
|
+
- [The commit message lint](../configs/commit-messages.md)
|
|
63
|
+
- [checks-lint](checks-lint.md)
|
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
# checks-docs
|
|
2
|
+
|
|
3
|
+
`checks-docs` is the gate that holds each doc file a change touches to the template for its kind, and a reader looks it up to learn which template a file answers to.
|
|
4
|
+
|
|
5
|
+
## What it checks
|
|
6
|
+
|
|
7
|
+
It holds each doc file a change touches to the template for its kind, and lists every other doc file that does not conform yet without failing.
|
|
8
|
+
The package ships one template per kind under `templates/`, and a repository starts a new doc file by copying one:
|
|
9
|
+
|
|
10
|
+
```sh
|
|
11
|
+
cp node_modules/@avi2dg/checks/templates/how-to.md docs/add-a-supplier.md
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
<!-- generated doc-kinds: bun run build writes it from scripts/doc-rules.ts, scripts/quality-file.ts, scripts/doc-templates.ts and scripts/doc-blocks.ts -->
|
|
15
|
+
|
|
16
|
+
| File | Kind | Template |
|
|
17
|
+
| --- | --- | --- |
|
|
18
|
+
| `README.md` | readme | `templates/readme.md` |
|
|
19
|
+
| `CHANGELOG.md` | changelog | `templates/changelog.md` |
|
|
20
|
+
| `AGENTS.md` | agents | `templates/agents.md` |
|
|
21
|
+
| `CLAUDE.md` | claude | `templates/claude.md` |
|
|
22
|
+
| `CONTRIBUTING.md` | how-to | `templates/how-to.md` |
|
|
23
|
+
| each file in `docs/adr/` but its generated index, `README.md` | adr | `templates/adr.md` |
|
|
24
|
+
| a page `docs.pages` declares | tutorial, how-to, reference or explanation | `templates/<mode>.md` |
|
|
25
|
+
|
|
26
|
+
<!-- end generated doc-kinds -->
|
|
27
|
+
|
|
28
|
+
A file the table names on its own, such as `README.md`, sits at the repository root.
|
|
29
|
+
No other Markdown file is judged, save a page under `docs/`, which needs a mode.
|
|
30
|
+
Which Diátaxis mode a page is written in is a judgment, so `quality.json` declares it:
|
|
31
|
+
|
|
32
|
+
```json
|
|
33
|
+
"docs": {
|
|
34
|
+
"pages": {
|
|
35
|
+
"reference": ["docs/gates/*.md"],
|
|
36
|
+
"explanation": ["docs/design.md"]
|
|
37
|
+
}
|
|
38
|
+
}
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
A template decides a file's structure, and the template file itself is the reference for each kind:
|
|
42
|
+
|
|
43
|
+
- A file opens with one `# ` title on its first line and has text before its first section.
|
|
44
|
+
It skips no heading level, and no heading is Overview, Introduction or How it works.
|
|
45
|
+
- Its sections are the template's headings in the template's order.
|
|
46
|
+
A heading in angle brackets is one the writer names.
|
|
47
|
+
One marked verb first is left to review, since no program tells a verb from a noun there.
|
|
48
|
+
A heading the template does not have, in that place, is refused.
|
|
49
|
+
- A record in `docs/adr/` is named for its four-digit number, and its title opens with the same number.
|
|
50
|
+
A `Date: YYYY-MM-DD` line follows the title, the first word under Status is Proposed, Accepted, Rejected, Deprecated, Superseded or Retired, and no other record holds its number.
|
|
51
|
+
- A changelog lists its releases newest first, each opening with a `Released YYYY-MM-DD.` line.
|
|
52
|
+
- A how-to or tutorial page numbers its steps.
|
|
53
|
+
- `CLAUDE.md` is its template word for word.
|
|
54
|
+
It is a fixed agent pointer rather than a people doc, so the separator ban does not apply to it: people docs take no em dash, en dash, parenthesis or hyphen used as a dash, and no semicolon, and `checks-docs` does not check separators.
|
|
55
|
+
|
|
56
|
+
## What it reads
|
|
57
|
+
|
|
58
|
+
It reads each Markdown file from the head commit, and `docs.pages` from `quality.json`.
|
|
59
|
+
A file the range adds, changes or renames is held to its template, and a file it deletes is not.
|
|
60
|
+
|
|
61
|
+
## Arguments
|
|
62
|
+
|
|
63
|
+
```sh
|
|
64
|
+
checks-docs <base-ref> <head-ref>
|
|
65
|
+
checks-docs <ref>
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
With two arguments the range starts where the head branched from the base, at their merge-base.
|
|
69
|
+
With one it is that commit against its parent, or against the empty tree for a repository's first commit.
|
|
70
|
+
|
|
71
|
+
## Exit codes
|
|
72
|
+
|
|
73
|
+
| Code | When |
|
|
74
|
+
| --- | --- |
|
|
75
|
+
| 0 | every doc file the range touches holds to its template |
|
|
76
|
+
| 1 | a doc file the range touches does not |
|
|
77
|
+
| 2 | `quality.json` does not decode, or a ref does not resolve |
|
|
78
|
+
|
|
79
|
+
## Sample output
|
|
80
|
+
|
|
81
|
+
```
|
|
82
|
+
docs: 2 violation(s) in the doc files the range touches:
|
|
83
|
+
README.md:1: lacks `## Where things are`
|
|
84
|
+
docs/parts.md: is a page under docs/ with no mode; declare it under docs.pages in quality.json as tutorial, how-to, reference, explanation
|
|
85
|
+
docs: advisory, 1 doc file(s) the range leaves alone do not hold to their templates yet:
|
|
86
|
+
docs/adr/0001-quality-gates.md: 5 violation(s)
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
## Opting out
|
|
90
|
+
|
|
91
|
+
It applies to every repository, so no selection leaves it out.
|
|
92
|
+
A file the range leaves alone is only listed as advisory, so a repository adopts the templates as its files change.
|
|
93
|
+
`checks-lint` runs it over each pull request's range, as [checks-lint](checks-lint.md) says.
|
|
94
|
+
|
|
95
|
+
## Related topics
|
|
96
|
+
|
|
97
|
+
- [Why it is shaped this way](../design.md)
|
|
98
|
+
- [checks-lint](checks-lint.md)
|
|
@@ -0,0 +1,113 @@
|
|
|
1
|
+
# checks-feature-owners
|
|
2
|
+
|
|
3
|
+
`checks-feature-owners` is the gate that holds each declared feature to a runnable proof and lists the feature owners a change touches, and a reader looks it up to declare a feature.
|
|
4
|
+
|
|
5
|
+
## What it checks
|
|
6
|
+
|
|
7
|
+
A repository opts a feature in by declaring, in `quality.json`, the directory it owns, the files code outside it imports it through, the files that may reach past those, and the end-to-end test that proves it runs:
|
|
8
|
+
|
|
9
|
+
```json
|
|
10
|
+
"features": [
|
|
11
|
+
{
|
|
12
|
+
"name": "billing",
|
|
13
|
+
"root": "src/billing",
|
|
14
|
+
"entries": ["src/billing/index.ts"],
|
|
15
|
+
"allowFrom": ["src/main.ts", "src/cli/*.ts"],
|
|
16
|
+
"proof": "tests/e2e/billing.test.ts"
|
|
17
|
+
}
|
|
18
|
+
],
|
|
19
|
+
"changeSignal": "advisory"
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
`root` is a directory and `entries` are files under it, both without globs.
|
|
23
|
+
`allowFrom` holds globs of the same shape as `sources`.
|
|
24
|
+
`proof` is a `.test.ts` or `.test.tsx` file under `tests/e2e/`.
|
|
25
|
+
Nothing moves, since a root is wherever the feature already lives.
|
|
26
|
+
`quality.json` refuses an entry outside its root, a name used twice, and two features sharing a root or one root inside another, so a file has at most one owner.
|
|
27
|
+
|
|
28
|
+
The gate fails when a feature's proof cannot prove it:
|
|
29
|
+
|
|
30
|
+
- the proof or an entry is not in the head commit
|
|
31
|
+
- the proof does not parse
|
|
32
|
+
- the proof imports none of the feature's entries
|
|
33
|
+
|
|
34
|
+
An import counts when it is a runtime `import`, `export ... from` or `export * from` of a relative path that names an entry.
|
|
35
|
+
It names the entry by its own name, by the `.js`, `.jsx`, `.mjs` or `.cjs` spelling of it, or without an extension, the way a directory `index` is imported.
|
|
36
|
+
`.js` names a `.tsx` entry as well as a `.ts` one.
|
|
37
|
+
`import type` does not count, and neither does a path alias.
|
|
38
|
+
The proof runs in `bun run test` like any end-to-end test, which is what shows it passes.
|
|
39
|
+
|
|
40
|
+
With `changeSignal` set to `advisory` it also lists each owner the range touches, with the paths it touched under the owner's root or at its proof, and still passes.
|
|
41
|
+
Whether a change that spans owners is one coherent slice is for a reviewer to judge.
|
|
42
|
+
A rename counts at both of its paths.
|
|
43
|
+
|
|
44
|
+
## What it reads
|
|
45
|
+
|
|
46
|
+
It reads `features` and `changeSignal` from `quality.json`, each proof and entry from the head commit, and the paths the range touches.
|
|
47
|
+
|
|
48
|
+
## Arguments
|
|
49
|
+
|
|
50
|
+
```sh
|
|
51
|
+
checks-feature-owners <base-ref> <head-ref>
|
|
52
|
+
checks-feature-owners <ref>
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
With two arguments the range starts where the head branched from the base, at their merge-base.
|
|
56
|
+
With one it is that commit against its parent, or against the empty tree for a repository's first commit.
|
|
57
|
+
|
|
58
|
+
## Exit codes
|
|
59
|
+
|
|
60
|
+
| Code | When |
|
|
61
|
+
| --- | --- |
|
|
62
|
+
| 0 | every feature's proof imports one of its entries |
|
|
63
|
+
| 1 | a feature's proof cannot prove it |
|
|
64
|
+
| 2 | `quality.json` does not decode, which is where a proof outside `tests/e2e/` is refused, or a ref does not resolve |
|
|
65
|
+
|
|
66
|
+
## Sample output
|
|
67
|
+
|
|
68
|
+
```
|
|
69
|
+
feature-owners: 1 problem(s) with the features' runnable proofs:
|
|
70
|
+
billing: proof tests/e2e/billing.test.ts imports none of its entries, src/billing/index.ts
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
With `changeSignal` set to `advisory`:
|
|
74
|
+
|
|
75
|
+
```
|
|
76
|
+
feature-owners: advisory, the range touches 2 feature owner(s); a reviewer judges whether they make one slice:
|
|
77
|
+
billing: src/billing/charge.ts, src/billing/tax.ts
|
|
78
|
+
invoices: src/invoices/tax.ts, tests/e2e/invoices.test.ts
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
## Opting out
|
|
82
|
+
|
|
83
|
+
A repository that declares no feature passes.
|
|
84
|
+
`quality.json` refuses a `changeSignal` without features, which would map a change to no owner.
|
|
85
|
+
A repository that tracks no `.ts` or `.tsx` file leaves it out of `gates.lint`, as [Gate selection](checks-lint.md#gate-selection) says.
|
|
86
|
+
|
|
87
|
+
## Import boundary
|
|
88
|
+
|
|
89
|
+
`dist/feature-rules.js` compiles `features` into one dependency-cruiser rule per feature, which `.dependency-cruiser.cjs` spreads beside its own:
|
|
90
|
+
|
|
91
|
+
```js
|
|
92
|
+
const { featureRules } = require("@avi2dg/checks/dist/feature-rules.js");
|
|
93
|
+
|
|
94
|
+
module.exports = {
|
|
95
|
+
extends: "./node_modules/@avi2dg/checks/dependency-cruiser.config.js",
|
|
96
|
+
forbidden: [...featureRules(require("./quality.json"))],
|
|
97
|
+
};
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
A module outside a feature's root that imports a file inside it must import one of the feature's `entries`.
|
|
101
|
+
Modules under `tests/` and the files `allowFrom` matches, such as a CLI or a harness, may import any file in it:
|
|
102
|
+
|
|
103
|
+
```
|
|
104
|
+
error feature-billing-entries: src/report.ts → src/billing/charge.ts
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
`featureRules` decodes its argument with the schema the bins use and throws the schema's refusal when it does not decode, which stops the cruise.
|
|
108
|
+
It is an ES module, as `effect` is, so a `.cjs` config loads it through `require`, which needs node 20.19, 22.12 or later.
|
|
109
|
+
|
|
110
|
+
## Related topics
|
|
111
|
+
|
|
112
|
+
- [The dependency rules](../configs/dependency-rules.md)
|
|
113
|
+
- [The quality file](../configs/quality-file.md)
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
# checks-flake
|
|
2
|
+
|
|
3
|
+
`checks-flake` is the scheduled run that finds flaky tests and records the seeds each one fails with, and a reader looks it up to reproduce a flaky failure.
|
|
4
|
+
|
|
5
|
+
## What it checks
|
|
6
|
+
|
|
7
|
+
A green run proves nothing failed in that run, not that no test is flaky.
|
|
8
|
+
`checks-flake` runs the whole suite several times, each with its own `--seed`, and records per failing test the seeds it failed with.
|
|
9
|
+
`bun test --randomize --seed=<seed>` puts the suite in the same order, so a seed reproduces a failure that hangs on order.
|
|
10
|
+
A run that fails with no failing test, such as a test file that throws while loading, is listed with its seed on its own line.
|
|
11
|
+
|
|
12
|
+
## What it reads
|
|
13
|
+
|
|
14
|
+
It reads bun's JUnit report of each run.
|
|
15
|
+
Under GitHub Actions it appends its summary to the file `GITHUB_STEP_SUMMARY` names, which is the job summary.
|
|
16
|
+
|
|
17
|
+
## Arguments
|
|
18
|
+
|
|
19
|
+
```sh
|
|
20
|
+
checks-flake [--runs <count> | --seed <seed>...] [--report <file>]
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
`--runs` sets the number of runs on random seeds, 10 when absent.
|
|
24
|
+
`--seed`, given once per run, replays chosen seeds, such as the ones a report recorded.
|
|
25
|
+
`--report` writes the record as JSON, holding every run's seed and failing tests and every failing test's seeds.
|
|
26
|
+
|
|
27
|
+
## Exit codes
|
|
28
|
+
|
|
29
|
+
| Code | When |
|
|
30
|
+
| --- | --- |
|
|
31
|
+
| 0 | every run passed |
|
|
32
|
+
| 1 | a run failed |
|
|
33
|
+
| 2 | an argument does not parse, or bun passed without writing its report |
|
|
34
|
+
|
|
35
|
+
## Sample output
|
|
36
|
+
|
|
37
|
+
```
|
|
38
|
+
checks-flake: 3 of 10 run(s) failed, 1 test(s) failing in them
|
|
39
|
+
|
|
40
|
+
| Test | Failed | Seeds |
|
|
41
|
+
| --- | --- | --- |
|
|
42
|
+
| tests/cache.test.ts:6 reads the cache | 3 of 10 runs | 2170533150, 4046124386, 180394251 |
|
|
43
|
+
|
|
44
|
+
Reproduce a failing run with bun test --randomize --seed=<seed>.
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
## Opting out
|
|
48
|
+
|
|
49
|
+
Nothing runs it but a schedule the repository writes.
|
|
50
|
+
A repository that stops scheduling it also drops it from `gates.scheduled`, which `checks-ci-wiring` otherwise fails on.
|
|
51
|
+
|
|
52
|
+
## Running it on a schedule
|
|
53
|
+
|
|
54
|
+
A repository runs it on a schedule and keeps the record as an artifact:
|
|
55
|
+
|
|
56
|
+
```yaml
|
|
57
|
+
on:
|
|
58
|
+
schedule:
|
|
59
|
+
- cron: "17 5 * * *"
|
|
60
|
+
workflow_dispatch:
|
|
61
|
+
jobs:
|
|
62
|
+
flake:
|
|
63
|
+
runs-on: ubuntu-latest
|
|
64
|
+
steps:
|
|
65
|
+
- uses: actions/checkout@v5
|
|
66
|
+
- uses: oven-sh/setup-bun@v2
|
|
67
|
+
- run: bun install --frozen-lockfile
|
|
68
|
+
- run: bunx checks-flake --runs 10 --report flake-report.json
|
|
69
|
+
- uses: actions/upload-artifact@v4
|
|
70
|
+
if: always()
|
|
71
|
+
with:
|
|
72
|
+
name: flake-report
|
|
73
|
+
path: flake-report.json
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
It declares the step in `gates.scheduled`, so `checks-ci-wiring` fails once the schedule stops running it:
|
|
77
|
+
|
|
78
|
+
```json
|
|
79
|
+
"gates": {
|
|
80
|
+
"ci": ["bun run lint", "bun run typecheck", "bun run test"],
|
|
81
|
+
"scheduled": ["bunx checks-flake --runs 10 --report flake-report.json"]
|
|
82
|
+
}
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
## Related topics
|
|
86
|
+
|
|
87
|
+
- [checks-test](checks-test.md)
|
|
88
|
+
- [checks-ci-wiring](checks-ci-wiring.md)
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
# checks-lint-coverage
|
|
2
|
+
|
|
3
|
+
`checks-lint-coverage` is the gate that fails when oxlint silently skips a tracked TypeScript file, and a reader looks it up when a file seems never to be linted.
|
|
4
|
+
|
|
5
|
+
## What it checks
|
|
6
|
+
|
|
7
|
+
It fails when oxlint skips a tracked `.ts` or `.tsx` file, for example through a stray `.gitignore` entry.
|
|
8
|
+
It compares `git ls-files` against oxlint's own file walk and names the missing files.
|
|
9
|
+
|
|
10
|
+
## What it reads
|
|
11
|
+
|
|
12
|
+
It reads the working tree: the `*.ts` and `*.tsx` files `git ls-files` lists, and the files `oxlint --debug=files` walks.
|
|
13
|
+
It walks without naming a path, since an explicit path bypasses the ignore files whose skips it looks for.
|
|
14
|
+
oxlint must be on `PATH`, as it is under a package script.
|
|
15
|
+
|
|
16
|
+
## Arguments
|
|
17
|
+
|
|
18
|
+
It takes none.
|
|
19
|
+
|
|
20
|
+
## Exit codes
|
|
21
|
+
|
|
22
|
+
| Code | When |
|
|
23
|
+
| --- | --- |
|
|
24
|
+
| 0 | oxlint walks every tracked `.ts` and `.tsx` file, or the repository tracks none |
|
|
25
|
+
| 1 | oxlint skips a tracked file |
|
|
26
|
+
| 2 | oxlint cannot walk the tree, as when it is not on `PATH` or its config does not parse |
|
|
27
|
+
|
|
28
|
+
## Sample output
|
|
29
|
+
|
|
30
|
+
```
|
|
31
|
+
lint-coverage: oxlint skips 1/3 tracked .ts/.tsx files; missing:
|
|
32
|
+
ignored/b.ts
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
A passing run counts the files:
|
|
36
|
+
|
|
37
|
+
```
|
|
38
|
+
lint-coverage: 71/71 tracked .ts/.tsx files
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
## Opting out
|
|
42
|
+
|
|
43
|
+
A repository that tracks no `.ts` or `.tsx` file leaves it out of `gates.lint`, as [Gate selection](checks-lint.md#gate-selection) says.
|
|
44
|
+
A repository that tracks one keeps it.
|
|
45
|
+
|
|
46
|
+
## Related topics
|
|
47
|
+
|
|
48
|
+
- [checks-lint](checks-lint.md)
|
|
49
|
+
- [The Effect rules](../configs/effect-rules.md)
|
|
@@ -0,0 +1,136 @@
|
|
|
1
|
+
# checks-lint
|
|
2
|
+
|
|
3
|
+
`checks-lint` is the entry point that runs every lint gate of the kit over one range, and a reader looks it up to learn which range it resolves and how a repository selects its gates.
|
|
4
|
+
|
|
5
|
+
## What it checks
|
|
6
|
+
|
|
7
|
+
It runs each gate the table under [What runs](../../README.md#what-runs) lists, in that table's order, and names every one that fails rather than stopping at the first.
|
|
8
|
+
Each gate runs as its own bin in a child process, with its output passed straight through, so a gate behaves the same called alone or through `checks-lint`.
|
|
9
|
+
`checks-ci-wiring` always runs, so a repository on `checks-lint` declares `gates.ci`, as [checks-ci-wiring](checks-ci-wiring.md) says.
|
|
10
|
+
It holds the test layout unless the selection leaves out `checks-test-layout`.
|
|
11
|
+
|
|
12
|
+
## What it reads
|
|
13
|
+
|
|
14
|
+
It reads `quality.json` for `defaultBranch` and `gates.lint`, and resolves the range once, handing the same one to every range gate.
|
|
15
|
+
A tree gate reads the working tree and is handed no range.
|
|
16
|
+
|
|
17
|
+
Locally, and on any event other than a pull request, the range ends at `HEAD` and starts where `HEAD` branched from the origin default branch.
|
|
18
|
+
That branch is `origin/HEAD`.
|
|
19
|
+
When `origin/HEAD` is not set, as in an `actions/checkout` clone, it is `origin/<defaultBranch>` from `quality.json`, and `origin/main` when that is not declared.
|
|
20
|
+
|
|
21
|
+
In a GitHub Actions pull request, where `GITHUB_EVENT_NAME` is `pull_request`, the range ends at the event's head sha and starts where that branched from `origin/<base branch>`, so GitHub's merge commit is never in it.
|
|
22
|
+
The base branch is read from the fetch, not from the event's recorded base sha, which GitHub leaves stale once the base branch advances after the pull request opens.
|
|
23
|
+
|
|
24
|
+
The range always starts at the merge base, never at the base branch's tip, since commits the base branch gained after the head branched off would otherwise count against the head.
|
|
25
|
+
When the head is the merge base, as on a push to the default branch or a local run on it, the range would be empty.
|
|
26
|
+
Each range gate is then handed that tip commit alone, and checks it against its parent, or against the empty tree when it is a repository's first commit:
|
|
27
|
+
|
|
28
|
+
```
|
|
29
|
+
checks-lint: tip 10ba7d8935b73ed72624120a1542e51bd21ca7c7 from HEAD against origin/main
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
A clone with no remote-tracking refs at all has no default branch to start from, such as a freshly initialised repository with no remote or one whose remote was never fetched.
|
|
33
|
+
Each range gate is then handed `HEAD` alone the same way:
|
|
34
|
+
|
|
35
|
+
```
|
|
36
|
+
checks-lint: tip 10ba7d8935b73ed72624120a1542e51bd21ca7c7 from HEAD alone, as the clone has no remote-tracking refs
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
Once any ref sits under `refs/remotes/`, a missing `origin/<default branch>` exits 2 instead.
|
|
40
|
+
That is the shape of a shallow CI checkout, where `HEAD` alone would leave the commits before it unchecked.
|
|
41
|
+
|
|
42
|
+
## Arguments
|
|
43
|
+
|
|
44
|
+
```sh
|
|
45
|
+
checks-lint
|
|
46
|
+
checks-lint <base-ref> <head-ref>
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
With no arguments it resolves the range as What it reads says.
|
|
50
|
+
A base and a head override that resolution.
|
|
51
|
+
|
|
52
|
+
## Exit codes
|
|
53
|
+
|
|
54
|
+
| Code | When |
|
|
55
|
+
| --- | --- |
|
|
56
|
+
| 0 | every gate passed |
|
|
57
|
+
| 1 | a gate found a violation |
|
|
58
|
+
| 2 | the range or the selection does not resolve, or no failing gate could decide |
|
|
59
|
+
|
|
60
|
+
## Sample output
|
|
61
|
+
|
|
62
|
+
It prints the range, the declared selection if there is one, each gate's own report, then its verdict:
|
|
63
|
+
|
|
64
|
+
```
|
|
65
|
+
checks-lint: range 2504acf098d120e73a8ece3c96f22b934f35c6a8..10ba7d8935b73ed72624120a1542e51bd21ca7c7 from HEAD against origin/main
|
|
66
|
+
...
|
|
67
|
+
checks-lint: 3 of 10 gate(s) failed: checks-commit-identity, checks-comment-gate, checks-suppressions-ratchet
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
## Opting out
|
|
71
|
+
|
|
72
|
+
A repository leaves out a gate that does not apply to it, as Gate selection says.
|
|
73
|
+
A repository that runs its gates without `checks-lint` calls each gate's bin in its own CI step and declares each command in `gates.ci`.
|
|
74
|
+
|
|
75
|
+
## Gate selection
|
|
76
|
+
|
|
77
|
+
A repository with no TypeScript source gives `checks-lint-coverage`, `checks-test-layout`, `checks-size-budget` and `checks-feature-owners` nothing to check, and test-layout would still refuse its missing `bun test` script and `bunfig.toml`.
|
|
78
|
+
It declares the gates `checks-lint` runs as `gates.lint`:
|
|
79
|
+
|
|
80
|
+
```json
|
|
81
|
+
"gates": {
|
|
82
|
+
"ci": ["bun run lint"],
|
|
83
|
+
"lint": [
|
|
84
|
+
"checks-commit-identity",
|
|
85
|
+
"checks-comment-gate",
|
|
86
|
+
"checks-suppressions-ratchet",
|
|
87
|
+
"checks-ci-wiring",
|
|
88
|
+
"checks-docs",
|
|
89
|
+
"checks-quality"
|
|
90
|
+
]
|
|
91
|
+
}
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
`checks-lint` runs exactly those, in the order of the table under What runs, and every gate when `gates.lint` is absent.
|
|
95
|
+
A selection in `quality.json` always keeps `checks-quality`, since the file it sits in is what makes that gate apply.
|
|
96
|
+
A step running `checks-lint` then counts only for a declared gate that `gates.lint` keeps.
|
|
97
|
+
|
|
98
|
+
A selection may leave out only a gate that does not apply, and the Runs in column of the table under What runs says where each gate applies.
|
|
99
|
+
Both `checks-lint` and `checks-ci-wiring` exit 2 on a `gates.lint` that names an unknown gate or leaves out one that applies to every repository.
|
|
100
|
+
`checks-ci-wiring` exits 1 when the selection leaves out a gate the repository's tracked files make applicable, and names the gate and the files:
|
|
101
|
+
|
|
102
|
+
```
|
|
103
|
+
ci-wiring: quality.json gates.lint leaves out 4 gate(s) this repository's contents make applicable:
|
|
104
|
+
checks-lint-coverage: the repository tracks TypeScript source (src/widget.ts)
|
|
105
|
+
checks-test-layout: the repository tracks TypeScript source (src/widget.ts)
|
|
106
|
+
checks-size-budget: the repository tracks TypeScript source (src/widget.ts)
|
|
107
|
+
checks-feature-owners: the repository tracks TypeScript source (src/widget.ts)
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
It reads the files tracked at the checkout, so the pull request that adds the first TypeScript file is the one refused.
|
|
111
|
+
|
|
112
|
+
## Running it in CI
|
|
113
|
+
|
|
114
|
+
CI runs it through `lint`.
|
|
115
|
+
The checkout fetches the whole history, which the merge base needs:
|
|
116
|
+
|
|
117
|
+
```yaml
|
|
118
|
+
on:
|
|
119
|
+
pull_request:
|
|
120
|
+
jobs:
|
|
121
|
+
lint:
|
|
122
|
+
runs-on: ubuntu-latest
|
|
123
|
+
steps:
|
|
124
|
+
- uses: actions/checkout@v5
|
|
125
|
+
with:
|
|
126
|
+
fetch-depth: 0
|
|
127
|
+
- uses: oven-sh/setup-bun@v2
|
|
128
|
+
- run: bun install --frozen-lockfile
|
|
129
|
+
- run: bun run lint
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
## Related topics
|
|
133
|
+
|
|
134
|
+
- [What runs](../../README.md#what-runs)
|
|
135
|
+
- [checks-ci-wiring](checks-ci-wiring.md)
|
|
136
|
+
- [The quality file](../configs/quality-file.md)
|
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
# checks-mutation-compare
|
|
2
|
+
|
|
3
|
+
`checks-mutation-compare` is the gate that holds a pull request's mutation score to no regression rather than an absolute threshold, and a reader looks it up to wire mutation testing into CI.
|
|
4
|
+
|
|
5
|
+
## What it checks
|
|
6
|
+
|
|
7
|
+
The head mutation score may not fall below the score at the merge-base.
|
|
8
|
+
The score is Stryker's, `Killed` and `Timeout` over those plus `Survived` and `NoCoverage`, so `CompileError`, `RuntimeError`, `Ignored` and `Pending` mutants leave it.
|
|
9
|
+
Every file in a report counts toward that report's score, including files present in only one of the two.
|
|
10
|
+
|
|
11
|
+
## What it reads
|
|
12
|
+
|
|
13
|
+
It reads two Stryker `mutation.json` reports, one built at the merge-base and one at the head.
|
|
14
|
+
The shared Stryker preset's `json` reporter writes `reports/mutation/mutation.json` in each worktree.
|
|
15
|
+
|
|
16
|
+
A repository that runs mutation testing installs `@stryker-mutator/core` and `@hughescr/stryker-bun-runner`, then spreads the shipped preset in `stryker.conf.mjs`:
|
|
17
|
+
|
|
18
|
+
```js
|
|
19
|
+
import preset from "@avi2dg/checks/stryker.preset.js";
|
|
20
|
+
|
|
21
|
+
export default {
|
|
22
|
+
...preset,
|
|
23
|
+
};
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
Keys the repository sets after the spread win.
|
|
27
|
+
|
|
28
|
+
## Arguments
|
|
29
|
+
|
|
30
|
+
```sh
|
|
31
|
+
checks-mutation-compare [--advisory] <base-report> <head-report>
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
`--advisory`, anywhere among the arguments, prints the same verdict and always exits 0.
|
|
35
|
+
|
|
36
|
+
## Exit codes
|
|
37
|
+
|
|
38
|
+
| Code | When |
|
|
39
|
+
| --- | --- |
|
|
40
|
+
| 0 | the head score is not below the base score, or `--advisory` is given |
|
|
41
|
+
| 1 | the head score is below the base score |
|
|
42
|
+
| 2 | a report is not a Stryker mutation report, or the arguments do not parse |
|
|
43
|
+
|
|
44
|
+
## Sample output
|
|
45
|
+
|
|
46
|
+
It prints the overall score of each report and the per-file scores that differ:
|
|
47
|
+
|
|
48
|
+
```
|
|
49
|
+
mutation-compare: base 83.33% (5/6) head 66.67% (4/6) delta -16.67pp
|
|
50
|
+
src/billing.ts: 75.00% (3/4) -> 50.00% (2/4)
|
|
51
|
+
1 unchanged file(s)
|
|
52
|
+
mutation-compare: REGRESSION (-16.67pp)
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
## Opting out
|
|
56
|
+
|
|
57
|
+
Nothing runs it but a CI step the repository writes.
|
|
58
|
+
A repository runs it with `--advisory` for its first month, then drops the flag so it blocks.
|
|
59
|
+
|
|
60
|
+
## Running it in CI
|
|
61
|
+
|
|
62
|
+
It runs on pull requests, comparing the head report against a report built at the merge-base:
|
|
63
|
+
|
|
64
|
+
```yaml
|
|
65
|
+
jobs:
|
|
66
|
+
mutation-compare:
|
|
67
|
+
runs-on: ubuntu-latest
|
|
68
|
+
steps:
|
|
69
|
+
- uses: actions/checkout@v5
|
|
70
|
+
with:
|
|
71
|
+
fetch-depth: 0
|
|
72
|
+
- uses: oven-sh/setup-bun@v2
|
|
73
|
+
- run: bun install --frozen-lockfile
|
|
74
|
+
- run: bunx stryker run
|
|
75
|
+
- run: |
|
|
76
|
+
base="$(git merge-base HEAD origin/main)"
|
|
77
|
+
git worktree add /tmp/mutation-base "$base"
|
|
78
|
+
(cd /tmp/mutation-base && bun install --frozen-lockfile && bunx stryker run)
|
|
79
|
+
- run: bun run checks-mutation-compare --advisory /tmp/mutation-base/reports/mutation/mutation.json reports/mutation/mutation.json
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
## Related topics
|
|
83
|
+
|
|
84
|
+
- [checks-test-layout](checks-test-layout.md)
|