@avi2dg/checks 0.21.0 → 0.22.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +58 -40
- package/CONTRIBUTING.md +11 -8
- package/README.md +15 -24
- package/bunfig.toml +1 -1
- package/docs/configs/commit-messages.md +5 -1
- package/docs/configs/dependency-rules.md +5 -2
- package/docs/configs/effect-rules.md +32 -33
- package/docs/configs/native-settings.md +74 -0
- package/docs/configs/typescript-rules.md +4 -0
- package/docs/design.md +24 -25
- package/docs/gates/checks-backtest.md +4 -0
- package/docs/gates/checks-ci-wiring.md +30 -93
- package/docs/gates/checks-comment-gate.md +4 -0
- package/docs/gates/checks-commit-identity.md +23 -27
- package/docs/gates/checks-docs.md +23 -21
- package/docs/gates/checks-flake.md +4 -10
- package/docs/gates/checks-lint-coverage.md +5 -1
- package/docs/gates/checks-lint.md +22 -104
- package/docs/gates/checks-mutation-compare.md +4 -0
- package/docs/gates/checks-quarantine-clock.md +4 -0
- package/docs/gates/checks-repetition.md +26 -41
- package/docs/gates/checks-subsumed-tests.md +4 -0
- package/docs/gates/checks-suppressions-ratchet.md +4 -0
- package/docs/gates/checks-test-layout.md +16 -8
- package/docs/gates/checks-test.md +68 -36
- package/docs/gates/checks-vendor.md +20 -13
- package/package.json +8 -21
- package/scripts/ci-wiring.ts +28 -97
- package/scripts/commit-identity.ts +32 -4
- package/scripts/doc-rules.ts +26 -11
- package/scripts/doc-templates.ts +2 -1
- package/scripts/docs.ts +4 -7
- package/scripts/gates.ts +0 -29
- package/scripts/git.ts +24 -1
- package/scripts/lint.ts +15 -34
- package/scripts/range-gate.ts +1 -2
- package/scripts/repetition.ts +40 -40
- package/scripts/shell-command.ts +7 -1
- package/scripts/swc.ts +46 -0
- package/scripts/test-layout.ts +40 -56
- package/scripts/test-skips.ts +180 -0
- package/scripts/test.ts +33 -38
- package/scripts/vendor.ts +55 -11
- package/dist/feature-rules.js +0 -354
- package/docs/configs/quality-file.md +0 -103
- package/docs/gates/checks-feature-owners.md +0 -113
- package/docs/gates/checks-quality.md +0 -111
- package/docs/gates/checks-size-budget.md +0 -107
- package/quality.schema.json +0 -514
- package/scripts/feature-owners.ts +0 -139
- package/scripts/quality-file.ts +0 -353
- package/scripts/quality.ts +0 -363
- package/scripts/size-budget.ts +0 -285
- package/scripts/size-rules.ts +0 -126
|
@@ -1,43 +1,25 @@
|
|
|
1
|
+
---
|
|
2
|
+
kind: reference
|
|
3
|
+
audience: consumers
|
|
4
|
+
---
|
|
1
5
|
# checks-lint
|
|
2
6
|
|
|
3
|
-
`checks-lint`
|
|
7
|
+
`checks-lint` runs every applicable kit lint gate over one range and reports each failure.
|
|
4
8
|
|
|
5
9
|
## What it checks
|
|
6
10
|
|
|
7
|
-
It runs
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
It holds the test layout unless the selection leaves out `checks-test-layout`.
|
|
11
|
+
It runs the gates under [What runs](../../README.md#what-runs) in table order, each in its own process.
|
|
12
|
+
Gates requiring tracked TypeScript files begin running when the repository tracks TypeScript.
|
|
13
|
+
All other gates run for every repository.
|
|
11
14
|
|
|
12
15
|
## What it reads
|
|
13
16
|
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
In a GitHub Actions pull request, where `GITHUB_EVENT_NAME` is `pull_request`, the range ends at the event's head sha and starts where that branched from `origin/<base branch>`, so GitHub's merge commit is never in it.
|
|
22
|
-
The base branch is read from the fetch, not from the event's recorded base sha, which GitHub leaves stale once the base branch advances after the pull request opens.
|
|
23
|
-
|
|
24
|
-
The range always starts at the merge base, never at the base branch's tip, since commits the base branch gained after the head branched off would otherwise count against the head.
|
|
25
|
-
When the head is the merge base, as on a push to the default branch or a local run on it, the range would be empty.
|
|
26
|
-
Each range gate is then handed that tip commit alone, and checks it against its parent, or against the empty tree when it is a repository's first commit:
|
|
27
|
-
|
|
28
|
-
```
|
|
29
|
-
checks-lint: tip 10ba7d8935b73ed72624120a1542e51bd21ca7c7 from HEAD against origin/main
|
|
30
|
-
```
|
|
31
|
-
|
|
32
|
-
A clone with no remote-tracking refs at all has no default branch to start from, such as a freshly initialised repository with no remote or one whose remote was never fetched.
|
|
33
|
-
Each range gate is then handed `HEAD` alone the same way:
|
|
34
|
-
|
|
35
|
-
```
|
|
36
|
-
checks-lint: tip 10ba7d8935b73ed72624120a1542e51bd21ca7c7 from HEAD alone, as the clone has no remote-tracking refs
|
|
37
|
-
```
|
|
38
|
-
|
|
39
|
-
Once any ref sits under `refs/remotes/`, a missing `origin/<default branch>` exits 2 instead.
|
|
40
|
-
That is the shape of a shallow CI checkout, where `HEAD` alone would leave the commits before it unchecked.
|
|
17
|
+
The range ends at `HEAD` and starts at the merge base with `origin/HEAD` for a local run.
|
|
18
|
+
If remote HEAD is absent, the range starts at the default branch GitHub's event names in CI, or at `origin/main`.
|
|
19
|
+
A local run with `GITHUB_BASE_REF` set starts at that branch on `origin` instead.
|
|
20
|
+
A clone with no remote tracking refs checks `HEAD` alone.
|
|
21
|
+
On a pull request the range ends at the event's head commit and starts at its merge base with the event's base branch.
|
|
22
|
+
Tree gates read the working tree rather than the range.
|
|
41
23
|
|
|
42
24
|
## Arguments
|
|
43
25
|
|
|
@@ -46,93 +28,29 @@ checks-lint
|
|
|
46
28
|
checks-lint <base-ref> <head-ref>
|
|
47
29
|
```
|
|
48
30
|
|
|
49
|
-
With
|
|
50
|
-
A base and a head override that resolution.
|
|
31
|
+
With two arguments, the base and head override range discovery.
|
|
51
32
|
|
|
52
33
|
## Exit codes
|
|
53
34
|
|
|
54
|
-
| Code |
|
|
35
|
+
| Code | Result |
|
|
55
36
|
| --- | --- |
|
|
56
|
-
| 0 |
|
|
57
|
-
| 1 |
|
|
58
|
-
| 2 |
|
|
37
|
+
| 0 | Every applicable gate passed. |
|
|
38
|
+
| 1 | At least one gate found a violation. |
|
|
39
|
+
| 2 | The range could not resolve or no failed gate could decide. |
|
|
59
40
|
|
|
60
41
|
## Sample output
|
|
61
42
|
|
|
62
|
-
It prints the range, the declared selection if there is one, each gate's own report, then its verdict:
|
|
63
|
-
|
|
64
43
|
```
|
|
65
44
|
checks-lint: range 2504acf098d120e73a8ece3c96f22b934f35c6a8..10ba7d8935b73ed72624120a1542e51bd21ca7c7 from HEAD against origin/main
|
|
66
|
-
|
|
67
|
-
checks-lint: 3 of 12 gate(s) failed: checks-commit-identity, checks-comment-gate, checks-suppressions-ratchet
|
|
45
|
+
checks-lint: 1 of 9 gate(s) failed: checks-comment-gate
|
|
68
46
|
```
|
|
69
47
|
|
|
70
48
|
## Opting out
|
|
71
49
|
|
|
72
|
-
A repository
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
## Gate selection
|
|
76
|
-
|
|
77
|
-
A repository with no TypeScript source gives `checks-lint-coverage`, `checks-test-layout`, `checks-size-budget`, `checks-repetition` and `checks-feature-owners` nothing to check, and test-layout would still refuse its missing `bun test` script and `bunfig.toml`.
|
|
78
|
-
It declares the gates `checks-lint` runs as `gates.lint`:
|
|
79
|
-
|
|
80
|
-
```json
|
|
81
|
-
"gates": {
|
|
82
|
-
"ci": ["bun run lint"],
|
|
83
|
-
"lint": [
|
|
84
|
-
"checks-commit-identity",
|
|
85
|
-
"checks-comment-gate",
|
|
86
|
-
"checks-suppressions-ratchet",
|
|
87
|
-
"checks-ci-wiring",
|
|
88
|
-
"checks-docs",
|
|
89
|
-
"checks-quarantine-clock",
|
|
90
|
-
"checks-quality"
|
|
91
|
-
]
|
|
92
|
-
}
|
|
93
|
-
```
|
|
94
|
-
|
|
95
|
-
`checks-lint` runs exactly those, in the order of the table under What runs, and every gate when `gates.lint` is absent.
|
|
96
|
-
A selection in `quality.json` always keeps `checks-quality`, since the file it sits in is what makes that gate apply.
|
|
97
|
-
A step running `checks-lint` then counts only for a declared gate that `gates.lint` keeps.
|
|
98
|
-
|
|
99
|
-
A selection may leave out only a gate that does not apply, and the Runs in column of the table under What runs says where each gate applies.
|
|
100
|
-
Both `checks-lint` and `checks-ci-wiring` exit 2 on a `gates.lint` that names an unknown gate or leaves out one that applies to every repository.
|
|
101
|
-
`checks-ci-wiring` exits 1 when the selection leaves out a gate the repository's tracked files make applicable, and names the gate and the files:
|
|
102
|
-
|
|
103
|
-
```
|
|
104
|
-
ci-wiring: quality.json gates.lint leaves out 5 gate(s) this repository's contents make applicable:
|
|
105
|
-
checks-lint-coverage: the repository tracks TypeScript source (src/widget.ts)
|
|
106
|
-
checks-test-layout: the repository tracks TypeScript source (src/widget.ts)
|
|
107
|
-
checks-size-budget: the repository tracks TypeScript source (src/widget.ts)
|
|
108
|
-
checks-repetition: the repository tracks TypeScript source (src/widget.ts)
|
|
109
|
-
checks-feature-owners: the repository tracks TypeScript source (src/widget.ts)
|
|
110
|
-
```
|
|
111
|
-
|
|
112
|
-
It reads the files tracked at the checkout, so the pull request that adds the first TypeScript file is the one refused.
|
|
113
|
-
|
|
114
|
-
## Running it in CI
|
|
115
|
-
|
|
116
|
-
CI runs it through `lint`.
|
|
117
|
-
The checkout fetches the whole history, which the merge base needs:
|
|
118
|
-
|
|
119
|
-
```yaml
|
|
120
|
-
on:
|
|
121
|
-
pull_request:
|
|
122
|
-
jobs:
|
|
123
|
-
lint:
|
|
124
|
-
runs-on: ubuntu-latest
|
|
125
|
-
steps:
|
|
126
|
-
- uses: actions/checkout@v5
|
|
127
|
-
with:
|
|
128
|
-
fetch-depth: 0
|
|
129
|
-
- uses: oven-sh/setup-bun@v2
|
|
130
|
-
- run: bun install --frozen-lockfile
|
|
131
|
-
- run: bun run lint
|
|
132
|
-
```
|
|
50
|
+
A repository runs `checks-lint` in a pull request workflow with the full git history fetched.
|
|
51
|
+
The gate automatically omits TypeScript gates when no TypeScript file is tracked.
|
|
133
52
|
|
|
134
53
|
## Related topics
|
|
135
54
|
|
|
136
55
|
- [What runs](../../README.md#what-runs)
|
|
137
56
|
- [checks-ci-wiring](checks-ci-wiring.md)
|
|
138
|
-
- [The quality file](../configs/quality-file.md)
|
|
@@ -1,3 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
kind: reference
|
|
3
|
+
audience: consumers
|
|
4
|
+
---
|
|
1
5
|
# checks-mutation-compare
|
|
2
6
|
|
|
3
7
|
`checks-mutation-compare` is the gate that holds every mutant in a pull request to no regression rather than an absolute score, and a reader looks it up to wire mutation testing into CI.
|
|
@@ -1,32 +1,29 @@
|
|
|
1
|
+
---
|
|
2
|
+
kind: reference
|
|
3
|
+
audience: consumers
|
|
4
|
+
---
|
|
1
5
|
# checks-repetition
|
|
2
6
|
|
|
3
|
-
`checks-repetition`
|
|
7
|
+
`checks-repetition` refuses an increase in repeated lines across a commit range, in the files `.jscpd.json` holds.
|
|
4
8
|
|
|
5
9
|
## What it checks
|
|
6
10
|
|
|
7
|
-
|
|
11
|
+
The gate runs jscpd at 50 tokens and 5 lines against the base and head revisions.
|
|
12
|
+
It compares repeated lines for each file, so a decrease in another file never offsets a rise.
|
|
13
|
+
It follows an edited rename back to the original file.
|
|
14
|
+
A file that repeats lines without a rise is advisory.
|
|
8
15
|
|
|
9
|
-
|
|
10
|
-
"sources": { "production": ["src/**/*.ts"] }
|
|
11
|
-
```
|
|
12
|
-
|
|
13
|
-
jscpd finds each block of at least 50 tokens and 5 lines that appears twice, within one file or across two.
|
|
14
|
-
A repeated line is a line of a file inside such a block.
|
|
15
|
-
The gate counts the repeated lines of each production file at the head and where the range starts, and fails when a file counts more at the head.
|
|
16
|
-
A renamed file is compared with its count under the old path.
|
|
17
|
-
A block counts only when both copies are in production files, so a test that copies production code changes no count.
|
|
18
|
-
Each file whose count rose is listed with the blocks it repeats and where the other copy is.
|
|
16
|
+
## What it reads
|
|
19
17
|
|
|
20
|
-
|
|
21
|
-
The advisory list names the production files whose count did not rise, and every other tracked `.ts` or `.tsx` file that repeats a block within itself or from another such file.
|
|
22
|
-
`.d.ts` files are not measured.
|
|
23
|
-
Nothing is committed as a baseline, because each run measures where the range starts as well as the head.
|
|
18
|
+
jscpd reads the head's `.jscpd.json` at both ends, so its `path` and `ignore` globs select the files to measure.
|
|
24
19
|
|
|
25
|
-
|
|
20
|
+
```json
|
|
21
|
+
{ "path": ["src"], "format": ["typescript"], "ignore": ["**/*.d.ts"] }
|
|
22
|
+
```
|
|
26
23
|
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
24
|
+
The gate sets the 50 tokens and 5 lines itself, over any the file names.
|
|
25
|
+
A `threshold` the file names never fails the gate, which reads jscpd's report once the scan finishes.
|
|
26
|
+
The bin reads the base and head commits rather than the working tree's file contents.
|
|
30
27
|
|
|
31
28
|
## Arguments
|
|
32
29
|
|
|
@@ -35,38 +32,26 @@ checks-repetition <base-ref> <head-ref>
|
|
|
35
32
|
checks-repetition <ref>
|
|
36
33
|
```
|
|
37
34
|
|
|
38
|
-
With two arguments the range starts where the head branched from the base, at their merge-base.
|
|
39
|
-
With one it is that commit against its parent, or against the empty tree for a repository's first commit.
|
|
40
|
-
|
|
41
35
|
## Exit codes
|
|
42
36
|
|
|
43
|
-
| Code |
|
|
37
|
+
| Code | Result |
|
|
44
38
|
| --- | --- |
|
|
45
|
-
| 0 |
|
|
46
|
-
| 1 |
|
|
47
|
-
| 2 |
|
|
39
|
+
| 0 | No measured file repeated more lines. |
|
|
40
|
+
| 1 | At least one measured file repeated more lines. |
|
|
41
|
+
| 2 | A config or ref is invalid, or jscpd cannot run. |
|
|
48
42
|
|
|
49
43
|
## Sample output
|
|
50
44
|
|
|
51
45
|
```
|
|
52
|
-
repetition: 1
|
|
53
|
-
src/
|
|
54
|
-
src/billing/refund.ts:1-11 repeats src/billing/legacy.ts:1-11
|
|
55
|
-
repetition: advisory, 4 file(s) repeat lines the hold does not fail:
|
|
56
|
-
src/billing/ledger.ts: 11 repeated line(s)
|
|
57
|
-
src/billing/legacy.ts: 21 repeated line(s)
|
|
58
|
-
tests/one.test.ts: 11 repeated line(s)
|
|
59
|
-
tests/two.test.ts: 11 repeated line(s)
|
|
46
|
+
repetition: 1 file(s) .jscpd.json holds repeat more lines than where the range starts, at 50 tokens and 5 lines:
|
|
47
|
+
src/copy.ts: 10 repeated line(s), up from 0
|
|
60
48
|
```
|
|
61
49
|
|
|
62
50
|
## Opting out
|
|
63
51
|
|
|
64
|
-
|
|
65
|
-
`checks-quality` refuses a `sources.production` glob that matches no file.
|
|
66
|
-
A repository that tracks no `.ts` or `.tsx` file leaves it out of `gates.lint`, as [Gate selection](checks-lint.md#gate-selection) says.
|
|
52
|
+
When the head holds no `.jscpd.json`, the gate reports that no file was measured.
|
|
67
53
|
|
|
68
54
|
## Related topics
|
|
69
55
|
|
|
70
|
-
- [
|
|
71
|
-
- [checks-
|
|
72
|
-
- [checks-lint](checks-lint.md)
|
|
56
|
+
- [Native settings](../configs/native-settings.md)
|
|
57
|
+
- [checks-suppressions-ratchet](checks-suppressions-ratchet.md)
|
|
@@ -1,3 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
kind: reference
|
|
3
|
+
audience: consumers
|
|
4
|
+
---
|
|
1
5
|
# checks-subsumed-tests
|
|
2
6
|
|
|
3
7
|
`checks-subsumed-tests` is the report that lists each test another test subsumes in a Stryker mutation run, and a reader looks it up to judge whether the suite carries tests it no longer needs.
|
|
@@ -1,3 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
kind: reference
|
|
3
|
+
audience: consumers
|
|
4
|
+
---
|
|
1
5
|
# checks-test-layout
|
|
2
6
|
|
|
3
7
|
`checks-test-layout` is the gate that holds a repository's tests to one layout, and a reader looks it up to learn where a test file goes and what it may import.
|
|
@@ -10,16 +14,20 @@ It fails unless the repository holds this shape, and names the file and the path
|
|
|
10
14
|
A `*.test.ts`, `*.spec.ts` or `*_test.ts` under `src/`, `test/`, `__tests__/` or the repository root fails.
|
|
11
15
|
- `tests/lib/**` holds helpers and `tests/fixtures/**` holds data, and neither may hold a test file.
|
|
12
16
|
Every other directory directly under `tests/` is a test group and may nest as deep as it likes.
|
|
17
|
+
- `tests/live/**` holds tests that need a live machine, and `tests/pixel/**` holds tests that need a display.
|
|
18
|
+
Bun ignores both directories in the default suite.
|
|
19
|
+
A live tier with test files requires `test:live` set to `checks-test --tier=live`.
|
|
20
|
+
A pixel tier with test files requires `test:pixel` set to `checks-test --tier=pixel`.
|
|
13
21
|
- A test runs at one of two levels.
|
|
14
|
-
A test outside `tests/e2e/` runs in-process, so it may not import `node:child_process`, `net`, `http`, `https`, `http2`, `tls` or `dgram`.
|
|
22
|
+
A test outside `tests/e2e/`, `tests/live/` and `tests/pixel/` runs in-process, so it may not import `node:child_process`, `net`, `http`, `https`, `http2`, `tls` or `dgram`.
|
|
15
23
|
It may not import `$`, `spawn`, `spawnSync`, `connect`, `serve` or `listen` from `bun`, may not touch `Bun.$` or `Bun.spawn`, and may not call `fetch`.
|
|
16
|
-
A test inside `tests/e2e/` may do all of it.
|
|
24
|
+
A test inside `tests/e2e/`, `tests/live/` or `tests/pixel/` may do all of it.
|
|
17
25
|
Helpers in `tests/lib/**` answer to the same rule, since an in-process test reaches them.
|
|
18
26
|
- `scripts.test` is exactly `checks-test`, which runs `bun test --randomize` as [checks-test](checks-test.md) says.
|
|
19
27
|
- `scripts.lint` runs this check, itself or through `checks-lint` called by its bare bin name.
|
|
20
28
|
- `bunfig.toml` carries every `[test]` key of the shipped preset with the same value.
|
|
21
|
-
`[test].pathIgnorePatterns` is the preset's `["**/tests/quarantine/**", "repos/**"]`, which the check pins itself, so the kit's own repository, whose bunfig is the preset, cannot drift it either.
|
|
22
|
-
|
|
29
|
+
`[test].pathIgnorePatterns` is the preset's `["**/tests/quarantine/**", "**/tests/live/**", "**/tests/pixel/**", "repos/**"]`, which the check pins itself, so the kit's own repository, whose bunfig is the preset, cannot drift it either.
|
|
30
|
+
The check also accepts the list without `repos/**`, so a repository that links no library under `repos/` may drop it.
|
|
23
31
|
Other tables, and extra `[test]` keys, are the repository's own.
|
|
24
32
|
|
|
25
33
|
The in-process half is what a mutation run can mutate.
|
|
@@ -41,8 +49,8 @@ It reads the working tree.
|
|
|
41
49
|
It scans the tracked and untracked files that `git ls-files --exclude-standard` reports, so `node_modules/` and every gitignored tree are out of reach, and a local run agrees with CI before `git add`.
|
|
42
50
|
It parses each test and helper with swc and reads import specifiers and identifier use, so a test that only carries `"node:child_process"` as a string is not a violation.
|
|
43
51
|
`tests/fixtures/**` is data and is not parsed.
|
|
44
|
-
It reads `package.json` for `scripts.test` and `scripts.lint
|
|
45
|
-
It
|
|
52
|
+
It reads `package.json` for `scripts.test` and `scripts.lint`.
|
|
53
|
+
It also checks `test:live` and `test:pixel` when their directories contain test files, and compares `bunfig.toml` with the preset the installed kit ships.
|
|
46
54
|
|
|
47
55
|
## Arguments
|
|
48
56
|
|
|
@@ -58,7 +66,7 @@ It checks the directory it runs in, or the directory it is given.
|
|
|
58
66
|
| --- | --- |
|
|
59
67
|
| 0 | the repository holds the layout |
|
|
60
68
|
| 1 | a file breaks the layout |
|
|
61
|
-
| 2 | a test, a helper
|
|
69
|
+
| 2 | a test, a helper or `package.json` does not parse |
|
|
62
70
|
|
|
63
71
|
## Sample output
|
|
64
72
|
|
|
@@ -72,7 +80,7 @@ test-layout: 4 violation(s)
|
|
|
72
80
|
|
|
73
81
|
## Opting out
|
|
74
82
|
|
|
75
|
-
|
|
83
|
+
`checks-lint` runs this gate when the repository tracks TypeScript, so a repository without TypeScript needs neither the test script nor bunfig.
|
|
76
84
|
A repository that tracks one keeps it.
|
|
77
85
|
|
|
78
86
|
## Related topics
|
|
@@ -1,64 +1,96 @@
|
|
|
1
|
+
---
|
|
2
|
+
kind: reference
|
|
3
|
+
audience: consumers
|
|
4
|
+
---
|
|
1
5
|
# checks-test
|
|
2
6
|
|
|
3
|
-
`checks-test`
|
|
7
|
+
`checks-test` runs the test suite and rejects skips without a reason at the test site.
|
|
4
8
|
|
|
5
9
|
## What it checks
|
|
6
10
|
|
|
7
|
-
It runs the
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
A test counts as skipped through `test.skip`, `test.skipIf`, `test.if`, `describe.skip` or `test.todo`.
|
|
11
|
+
It runs the default suite with `bun test --randomize` and reads Bun's JUnit report from that run.
|
|
12
|
+
Bun exits zero when tests skip, so `checks-test` checks every skipped test against its source declaration.
|
|
13
|
+
A test that `test.skip`, `test.skipIf`, `test.if`, `test.todo` or an enclosing `describe.skip` skips fails unless the test declares its reason.
|
|
11
14
|
|
|
12
|
-
|
|
15
|
+
Import `skipReason` from `@avi2dg/checks/scripts/test-skips.ts` beside the native Bun test call:
|
|
13
16
|
|
|
14
|
-
```
|
|
15
|
-
"
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
}
|
|
22
|
-
]
|
|
17
|
+
```ts
|
|
18
|
+
import { skipReason } from "@avi2dg/checks/scripts/test-skips.ts";
|
|
19
|
+
|
|
20
|
+
test.skipIf(!hasNix)(
|
|
21
|
+
skipReason("Nix is unavailable", "loads the theme"),
|
|
22
|
+
() => loadTheme(),
|
|
23
|
+
);
|
|
23
24
|
```
|
|
24
25
|
|
|
25
|
-
`
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
26
|
+
`skipReason` requires a literal reason and a literal test name.
|
|
27
|
+
It returns the test name unchanged, and `checks-test` reads the reason, the name and the source line from the test file.
|
|
28
|
+
The Bun call stays at the test site so the JUnit report points to the line where its first argument starts.
|
|
29
|
+
Use `test.skip(skipReason(reason, name), fn)` for an unconditional skip and `test.todo(skipReason(reason, name))` for a todo.
|
|
30
|
+
A skip is declared on the test itself and only there.
|
|
31
|
+
`checks-test` refuses `skipReason` on `describe.skip`, `describe.skipIf` or `describe.if`, so declare each test inside the describe instead.
|
|
32
|
+
|
|
33
|
+
Add `"ci"` or `"local"` as the third `skipReason` argument when a declaration applies to one environment.
|
|
34
|
+
Omit the third argument when it applies in both environments.
|
|
35
|
+
A declaration for the other environment is not judged in the current run.
|
|
29
36
|
|
|
30
|
-
A declaration
|
|
31
|
-
A local run
|
|
32
|
-
|
|
37
|
+
A declaration whose test passes or does not register fails a CI run.
|
|
38
|
+
A local run warns about the same declaration because a condition can depend on the machine.
|
|
39
|
+
A skipped test or a todo without `skipReason` fails in every environment.
|
|
33
40
|
|
|
34
41
|
## What it reads
|
|
35
42
|
|
|
36
|
-
|
|
37
|
-
It
|
|
43
|
+
`checks-test` reads test source files and the JUnit report written by its own run.
|
|
44
|
+
It does not read a skip list from `package.json`.
|
|
45
|
+
It counts a run as `ci` when `CI` is true and as `local` otherwise.
|
|
46
|
+
|
|
47
|
+
The command takes no arguments for the default suite.
|
|
48
|
+
Use `checks-test --tier=live` or `checks-test --tier=pixel` for a named test tier.
|
|
49
|
+
A tier run clears Bun's ignored paths, runs only `./tests/live` or `./tests/pixel`, and still checks each skip.
|
|
50
|
+
|
|
51
|
+
Files under `tests/quarantine/` are not run or judged, as [checks-test-layout](checks-test-layout.md) says.
|
|
52
|
+
|
|
53
|
+
## Test tiers
|
|
54
|
+
|
|
55
|
+
Tests that need a live machine go in `tests/live/`, and tests that need a screen go in `tests/pixel/`.
|
|
56
|
+
For example, a test under `tests/e2e/stack/` that skips when Nix is missing moves to the same path under `tests/live/stack/`.
|
|
57
|
+
Its skip declares `skipReason("Nix is unavailable", name)` inside its `test.skipIf` call.
|
|
58
|
+
Add the matching package scripts when either directory contains tests:
|
|
59
|
+
|
|
60
|
+
```json
|
|
61
|
+
{
|
|
62
|
+
"scripts": {
|
|
63
|
+
"test:live": "checks-test --tier=live",
|
|
64
|
+
"test:pixel": "checks-test --tier=pixel"
|
|
65
|
+
}
|
|
66
|
+
}
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
Bun ignores both directories during the default run.
|
|
70
|
+
`checks-test-layout` requires each tier script when its directory contains a test file.
|
|
38
71
|
|
|
39
72
|
## Arguments
|
|
40
73
|
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
74
|
+
The default command takes no arguments.
|
|
75
|
+
Its named tier options are `--tier=live` and `--tier=pixel`.
|
|
76
|
+
It refuses test filters because every test excluded by a filter would look skipped.
|
|
44
77
|
|
|
45
78
|
## Exit codes
|
|
46
79
|
|
|
47
80
|
| Code | When |
|
|
48
81
|
| --- | --- |
|
|
49
|
-
| 0 | every test
|
|
50
|
-
| 1 | a test failed, a skip
|
|
51
|
-
| 2 |
|
|
82
|
+
| 0 | every test passed and every skip has a matching site reason |
|
|
83
|
+
| 1 | a test failed, a skip lacks a site reason or a declaration is stale in CI |
|
|
84
|
+
| 2 | a declaration cannot be read, `CI` is not a boolean or Bun wrote no report |
|
|
52
85
|
|
|
53
86
|
## Sample output
|
|
54
87
|
|
|
55
88
|
```
|
|
56
|
-
checks-test: 1 skipped test(s) undeclared
|
|
57
|
-
tests/pricing.test.ts:12
|
|
58
|
-
tests/e2e/docker.test.ts > images > builds the release image: declared, but no such test skipped; delete the declaration
|
|
89
|
+
checks-test: 1 skipped test(s) undeclared in this local run:
|
|
90
|
+
tests/pricing.test.ts:12 rounds half to even: skipped with no reason at its test site; use test.skipIf(condition)(skipReason(reason, name), fn)
|
|
59
91
|
```
|
|
60
92
|
|
|
61
|
-
A run with
|
|
93
|
+
A run with no skipped tests ends with:
|
|
62
94
|
|
|
63
95
|
```
|
|
64
96
|
checks-test: no test skipped
|
|
@@ -66,10 +98,10 @@ checks-test: no test skipped
|
|
|
66
98
|
|
|
67
99
|
## Opting out
|
|
68
100
|
|
|
69
|
-
A
|
|
70
|
-
A repository that tracks TypeScript source runs `checks-test` as `scripts.test`, since [checks-test-layout](checks-test-layout.md) requires it.
|
|
101
|
+
A repository that tracks no TypeScript source does not need `checks-test`, as [checks-test-layout](checks-test-layout.md) says.
|
|
71
102
|
|
|
72
103
|
## Related topics
|
|
73
104
|
|
|
74
105
|
- [checks-test-layout](checks-test-layout.md)
|
|
75
106
|
- [checks-flake](checks-flake.md)
|
|
107
|
+
- [checks-quarantine-clock](checks-quarantine-clock.md)
|
|
@@ -1,11 +1,15 @@
|
|
|
1
|
+
---
|
|
2
|
+
kind: reference
|
|
3
|
+
audience: consumers
|
|
4
|
+
---
|
|
1
5
|
# checks-vendor
|
|
2
6
|
|
|
3
|
-
`checks-vendor` pins each library
|
|
7
|
+
`checks-vendor` pins each library its arguments name to one shared read-only clone and links it under `repos/`.
|
|
4
8
|
|
|
5
9
|
## What it checks
|
|
6
10
|
|
|
7
|
-
Each
|
|
8
|
-
Its
|
|
11
|
+
Each `--library` names an npm package, a git remote and a tag template holding `{version}`.
|
|
12
|
+
Its name is the link under `repos/`, and its optional `--path` is the manifest inside the clone that holds the version, `package.json` when absent.
|
|
9
13
|
`checks-vendor` reads the installed version from `node_modules/<package>/package.json` and resolves the template to one tag.
|
|
10
14
|
It clones that tag once into a cache shared across repositories, records the landed commit in `<tag>.commit` beside the tree, strips every write bit and links `repos/<name>` to the tree.
|
|
11
15
|
The clone is staged beside its cache entry and moves into place only once it is recorded, checked and read only, so a concurrent or killed first run never leaves a half built tree there.
|
|
@@ -26,7 +30,7 @@ A read through the link resolves outside the checkout, so a reader that must sta
|
|
|
26
30
|
## What it reads
|
|
27
31
|
|
|
28
32
|
It reads the working tree.
|
|
29
|
-
That is `
|
|
33
|
+
That is `node_modules/<package>/package.json` for each named library and the `repos/` links.
|
|
30
34
|
It reads the shared cache outside the checkout.
|
|
31
35
|
That is each tag tree, its recorded commit and its manifest.
|
|
32
36
|
It contacts a remote only when a tag is not cached yet, to confirm the tag exists and to clone it.
|
|
@@ -34,18 +38,21 @@ It contacts a remote only when a tag is not cached yet, to confirm the tag exist
|
|
|
34
38
|
## Arguments
|
|
35
39
|
|
|
36
40
|
```sh
|
|
37
|
-
checks-vendor
|
|
41
|
+
checks-vendor [--library <name> --package <package> --repository <remote> --tag <template> [--path <manifest>]]...
|
|
38
42
|
```
|
|
39
43
|
|
|
40
|
-
|
|
44
|
+
Each `--library` opens one library, and the flags after it up to the next `--library` describe it.
|
|
45
|
+
`--package`, `--repository` and `--tag` are required, and `--path` is optional.
|
|
46
|
+
A name is lowercase words joined by hyphens, and no two libraries share one.
|
|
47
|
+
With no arguments it pins nothing.
|
|
41
48
|
|
|
42
49
|
## Exit codes
|
|
43
50
|
|
|
44
51
|
| Code | When |
|
|
45
52
|
| --- | --- |
|
|
46
|
-
| 0 | every
|
|
53
|
+
| 0 | every named library links a verified tree, its remote could not be reached for a first fetch, or no git checkout holds the run |
|
|
47
54
|
| 1 | a tag is missing or lands elsewhere than its record, an installed version is unknown, a tree was written to, a record disagrees with its tree, a version disagrees or a link is blocked |
|
|
48
|
-
| 2 |
|
|
55
|
+
| 2 | an argument is unknown, missing, repeated or malformed, or `HOME` is unset |
|
|
49
56
|
|
|
50
57
|
## Sample output
|
|
51
58
|
|
|
@@ -66,7 +73,7 @@ A later run reports the link it kept.
|
|
|
66
73
|
A consuming repository runs it from its `prepare` script, so every install pins and verifies the trees.
|
|
67
74
|
|
|
68
75
|
```json
|
|
69
|
-
{ "scripts": { "prepare": "checks-vendor" } }
|
|
76
|
+
{ "scripts": { "prepare": "checks-vendor --library effect --package effect --repository https://github.com/Effect-TS/effect.git --tag 'effect@{version}' --path packages/effect/package.json" } }
|
|
70
77
|
```
|
|
71
78
|
|
|
72
79
|
An install offline still passes.
|
|
@@ -88,10 +95,10 @@ A consumer whose `tsconfig.json` has no explicit `include` keeps the trees out w
|
|
|
88
95
|
|
|
89
96
|
## Opting out
|
|
90
97
|
|
|
91
|
-
It
|
|
92
|
-
A
|
|
93
|
-
A repository that
|
|
98
|
+
It pins sources only for the libraries its arguments name.
|
|
99
|
+
A run with no arguments reports nothing to pin and changes nothing.
|
|
100
|
+
A repository that pins no library needs no `prepare` entry for it.
|
|
94
101
|
|
|
95
102
|
## Related topics
|
|
96
103
|
|
|
97
|
-
- [
|
|
104
|
+
- [Native settings](../configs/native-settings.md)
|