@avi2dg/checks 0.43.0 → 0.45.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +20 -0
- package/README.md +9 -4
- package/{commitlint.config.js → commitlint.config.ts} +1 -1
- package/dist/mutation/mutation-guard-plugin.js +40 -0
- package/dist/mutation/mutation-refusal.js +23 -0
- package/dist/presets/stylelint.d.ts +10 -0
- package/dist/presets/stylelint.js +73 -0
- package/docs/configs/commit-messages.md +2 -2
- package/docs/configs/native-settings.md +15 -2
- package/docs/design.md +26 -1
- package/docs/gates/checks-ci-wiring.md +3 -2
- package/docs/gates/checks-comment-gate.md +2 -0
- package/docs/gates/checks-copy.md +113 -0
- package/docs/gates/checks-docs-only.md +3 -2
- package/docs/gates/checks-docs.md +3 -0
- package/docs/gates/checks-flake.md +49 -1
- package/docs/gates/checks-frontend-syntax.md +87 -12
- package/docs/gates/checks-lint.md +2 -1
- package/docs/gates/checks-mutation-baseline.md +4 -2
- package/docs/gates/checks-mutation-compare.md +33 -8
- package/docs/gates/checks-mutation-scope.md +140 -0
- package/docs/gates/checks-mutation.md +11 -7
- package/docs/gates/checks-no-plain-js.md +60 -0
- package/docs/gates/checks-repetition.md +3 -0
- package/docs/gates/checks-suppressions-ratchet.md +11 -9
- package/docs/gates/checks-test.md +5 -2
- package/package.json +33 -6
- package/src/complexity/suppressions-ratchet.ts +37 -25
- package/src/core/declared-inputs.ts +77 -0
- package/src/core/gates.ts +4 -0
- package/src/core/git.ts +9 -0
- package/src/delivery/ci-wiring.ts +25 -11
- package/src/delivery/commit-title.ts +2 -2
- package/src/docs/copy-html.ts +110 -0
- package/src/docs/copy-text.ts +70 -0
- package/src/docs/copy.ts +70 -0
- package/src/docs/prose-matchers.ts +14 -14
- package/src/quality/comment-matchers.ts +4 -1
- package/src/quality/frontend-syntax.ts +114 -71
- package/src/quality/no-plain-js.ts +71 -0
- package/src/quality/presets/stylelint.ts +91 -0
- package/src/testing/flake-preload.ts +4 -0
- package/src/testing/flake.ts +142 -16
- package/src/testing/{mutation-scope.js → mutation-refusal.ts} +11 -9
- package/src/testing/mutation-scope.ts +354 -0
- package/src/testing/mutation.ts +1 -1
- package/src/testing/test-layout.ts +4 -20
- package/src/testing/test-skips.ts +1 -1
- package/src/testing/test.ts +19 -3
- package/stryker.preset.js +3 -3
- package/src/testing/mutation-guard-plugin.js +0 -13
|
@@ -4,11 +4,18 @@ audience: consumers
|
|
|
4
4
|
---
|
|
5
5
|
# checks-frontend-syntax
|
|
6
6
|
|
|
7
|
-
`checks-frontend-syntax` is the opt-in gate that
|
|
7
|
+
`checks-frontend-syntax` is the opt-in gate that holds a repository's CSS to stylelint, under the kit's two rules of its own and the rule families of `@avi2dg/checks/stylelint`.
|
|
8
8
|
|
|
9
9
|
## What it checks
|
|
10
10
|
|
|
11
|
-
It runs stylelint
|
|
11
|
+
It runs stylelint twice over each declared input.
|
|
12
|
+
The repository run holds the families the repository's `stylelint.config.ts` sets, and applies `--fix`.
|
|
13
|
+
The kit run then holds the kit's two rules over each file the repository run matched.
|
|
14
|
+
It reads no repository config and no count in `stylelint-suppressions.json`, so neither turns those rules off.
|
|
15
|
+
|
|
16
|
+
### The kit's two rules
|
|
17
|
+
|
|
18
|
+
The kit run refuses two CSS constraints a program can decide, a transition over `all` and a body-wide `user-select`:
|
|
12
19
|
|
|
13
20
|
| Rule | What it refuses | Why |
|
|
14
21
|
| --- | --- | --- |
|
|
@@ -19,7 +26,37 @@ The second rule refuses any value on those selectors, so `body { user-select: te
|
|
|
19
26
|
It matches each item of the selector list as a whole.
|
|
20
27
|
So it does not follow `:is()`, `:where()` or `:not()`, nor a compound such as `html body` or `body *`.
|
|
21
28
|
A nested rule is judged by its own selector, so `.card { * { user-select: none; } }` fails as `*`.
|
|
22
|
-
A control that needs selection off sets it on its own selector, such as `.drag-handle { user-select: none; }`, which passes.
|
|
29
|
+
A control that needs selection off sets it on its own selector, such as `.drag-handle { user-select: none; }`, which this rule passes and the Baseline family reports.
|
|
30
|
+
|
|
31
|
+
### The rule families
|
|
32
|
+
|
|
33
|
+
`defineConfig` from `@avi2dg/checks/stylelint` builds the config the repository run holds:
|
|
34
|
+
|
|
35
|
+
| Family | Rules | Default |
|
|
36
|
+
| --- | --- | --- |
|
|
37
|
+
| G1 correctness | `stylelint-config-recommended`, the stylelint team's rules for likely mistakes, such as `font-variant-numeric: oldstyle-num` for `oldstyle-nums` | on |
|
|
38
|
+
| G2 conventions | `stylelint-config-standard`, which extends G1 with the stylelint team's notation and spacing rules | on |
|
|
39
|
+
| G3 selector complexity | `selector-max-specificity` at `0,3,2` and `selector-max-compound-selectors` at 3 | on |
|
|
40
|
+
| G4 design tokens | `color-no-hex`, `color-named: "never"`, `function-disallowed-list` over `rgb`, `rgba`, `hsl`, `hsla`, `hwb`, `lab`, `lch`, `oklab`, `oklch` and `color`, and `font-size` only as one `var(--…)` with no fallback, or `inherit` | off until `tokens` names the token files, and off in those files |
|
|
41
|
+
| G5 Baseline target | `plugin/use-baseline` from `stylelint-plugin-use-baseline`, which judges each property, value, function, at-rule, selector and unit against the Baseline data, and passes a property inside an `@supports` block | on, at the target `baseline` sets |
|
|
42
|
+
|
|
43
|
+
The builder changes three of the shared config's rules:
|
|
44
|
+
|
|
45
|
+
- `no-descending-specificity` is off, since it compares only the last compound selector and has no access to the DOM.
|
|
46
|
+
It reports components that never match one element.
|
|
47
|
+
- `property-no-vendor-prefix` keeps reporting without a fix, and passes `-webkit-user-select`, since its fix deletes the prefix Safari still needs.
|
|
48
|
+
- In an `.html` or `.astro` file, `declaration-block-single-line-max-declarations` is off, since a `style` attribute is one line.
|
|
49
|
+
|
|
50
|
+
CSS repetition is not a family of this gate: a repository adds `"css"` to `format` in `.jscpd.json`, and [checks-repetition](checks-repetition.md) then fails when a stylesheet's repeated lines grow.
|
|
51
|
+
|
|
52
|
+
### What no comment turns off
|
|
53
|
+
|
|
54
|
+
It sets stylelint's `ignoreDisables`, so a `stylelint-disable` comment silences nothing in either run, and [checks-comment-gate](checks-comment-gate.md) refuses one on the line that adds it.
|
|
55
|
+
A repository adopts a family over existing problems by counting them in `stylelint-suppressions.json` with `--suppress`.
|
|
56
|
+
stylelint lets a file and rule through while its problems stay within the count, and [checks-suppressions-ratchet](checks-suppressions-ratchet.md) fails a change that raises a count.
|
|
57
|
+
`--suppress` counts only what the repository run reports, and a file that does not parse still fails in the kit run.
|
|
58
|
+
|
|
59
|
+
### Files
|
|
23
60
|
|
|
24
61
|
It judges each `.css` file, and each `<style>` block and `style` attribute in an `.html`, `.astro`, `.vue` or `.svelte` file, through `postcss-html`.
|
|
25
62
|
It reports a file that does not parse, since it cannot judge one.
|
|
@@ -28,7 +65,7 @@ It fails on each declared input that matches no file, so an input that a renamed
|
|
|
28
65
|
These are outside what it decides:
|
|
29
66
|
|
|
30
67
|
- A `transition` shorthand that names no property, such as `transition: 200ms`, which the browser reads as `all`.
|
|
31
|
-
- A value it cannot see, such as `transition: var(--motion)
|
|
68
|
+
- A value it cannot see, such as `transition: var(--motion)`, a style a script sets, or a colour a build writes into CSS outside the declared inputs.
|
|
32
69
|
- `user-select: none` in a `style` attribute on `<body>`, which has no selector to judge.
|
|
33
70
|
- A viewport that disables zoom, since stylelint reads only CSS.
|
|
34
71
|
A repository that runs only this gate gets no viewport check, and [checks-browser](checks-browser.md) refuses one on the rendered page through axe's `meta-viewport` rule.
|
|
@@ -49,35 +86,69 @@ A glob over built output, such as `dist/**/*.html`, judges the styles a framewor
|
|
|
49
86
|
stylelint resolves each glob from that directory, skips `node_modules/` and honours `.stylelintignore`.
|
|
50
87
|
It reads the working tree, so an uncommitted file is judged like a committed one.
|
|
51
88
|
|
|
52
|
-
|
|
89
|
+
It imports `stylelint.config.ts` from the same directory, and passes its default export to stylelint:
|
|
90
|
+
|
|
91
|
+
```ts
|
|
92
|
+
import { defineConfig } from "@avi2dg/checks/stylelint";
|
|
93
|
+
|
|
94
|
+
export default defineConfig({
|
|
95
|
+
baseline: "widely",
|
|
96
|
+
tokens: ["src/styles/tokens.css"],
|
|
97
|
+
});
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
`baseline` and `tokens` have no default, since only the repository knows them, so `tsc` refuses a config that leaves either out.
|
|
101
|
+
The builder throws when the gate loads such a config anyway.
|
|
102
|
+
|
|
103
|
+
| Key | Takes |
|
|
104
|
+
| --- | --- |
|
|
105
|
+
| `baseline` | `"widely"`, `"newly"` or a year such as `2024`, which allows what became Baseline newly available in that year or earlier |
|
|
106
|
+
| `tokens` | the globs of the files that declare the design tokens, which turns G4 on in every other file, or `false` |
|
|
107
|
+
|
|
108
|
+
`rules`, `overrides`, `ignoreFiles` and the other keys mean what they mean in stylelint's own docs, and land after the kit's.
|
|
109
|
+
An accepted exception lives there rather than in a comment, such as a property the Baseline rule passes:
|
|
110
|
+
|
|
111
|
+
```ts
|
|
112
|
+
rules: { "plugin/use-baseline": [true, { available: "widely", ignoreProperties: { "user-select": ["none"] } }] },
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
The builder owns `extends` and `plugins`.
|
|
116
|
+
It resolves the shared config, the plugin and `postcss-html` beside the kit, so they need not sit under the directory the gate runs in.
|
|
117
|
+
stylelint cannot load `stylelint.config.ts` itself under TypeScript 7, so a bare `stylelint --fix` finds no config, and the gate's `--fix` stands in for it.
|
|
118
|
+
A config not built with `defineConfig` names its own `customSyntax` for markup, or each `.html` or `.astro` file fails to parse in the repository run.
|
|
119
|
+
|
|
120
|
+
The repository installs the optional peers the gate loads, at the versions the kit pins:
|
|
53
121
|
|
|
54
122
|
```sh
|
|
55
|
-
bun add -d stylelint@17.16.0 postcss-html@2.0.0
|
|
123
|
+
bun add -d stylelint@17.16.0 postcss-html@2.0.0 stylelint-config-standard@40.0.0 stylelint-plugin-use-baseline@1.4.7
|
|
56
124
|
```
|
|
57
125
|
|
|
58
126
|
## Arguments
|
|
59
127
|
|
|
60
128
|
```sh
|
|
61
|
-
checks-frontend-syntax [<directory>]
|
|
129
|
+
checks-frontend-syntax [--fix] [--suppress] [<directory>]
|
|
62
130
|
```
|
|
63
131
|
|
|
64
132
|
It checks the directory it runs in, or the directory it is given.
|
|
133
|
+
`--fix` rewrites what a rule of the config fixes, such as `#ffffff` to `#fff`, and reports what is left.
|
|
134
|
+
`--suppress` writes each problem the config reports into `stylelint-suppressions.json`, and drops a count whose problems are gone.
|
|
65
135
|
|
|
66
136
|
## Exit codes
|
|
67
137
|
|
|
68
138
|
| Code | When |
|
|
69
139
|
| --- | --- |
|
|
70
|
-
| 0 | every declared input matches a file, and no file holds a
|
|
71
|
-
| 1 | a file holds a
|
|
72
|
-
| 2 | `frontend-syntax.json` is missing or does not decode, its input list is empty, or stylelint
|
|
140
|
+
| 0 | every declared input matches a file, and no file holds a problem beyond its suppressed count |
|
|
141
|
+
| 1 | a file holds a problem or does not parse, or a declared input matches no file |
|
|
142
|
+
| 2 | `frontend-syntax.json` is missing or does not decode, its input list is empty, `stylelint.config.ts` is missing or does not load, stylelint refuses an option, a peer cannot load, or an argument is unknown |
|
|
73
143
|
|
|
74
144
|
## Sample output
|
|
75
145
|
|
|
76
146
|
```
|
|
77
|
-
frontend-syntax:
|
|
147
|
+
frontend-syntax: 4 problem(s) in 2 file(s) from 3 declared input(s):
|
|
78
148
|
dist/**/*.html: the declared input matches no file
|
|
79
149
|
src/layouts/Base.astro:4:15 Disallowed property "user-select" for selector "body". Leave text selectable across the page, and turn selection off only on the control that needs it.
|
|
80
150
|
src/styles/site.css:1:17 Disallowed value "all 200ms" for property "transition". Name the properties the transition animates.
|
|
151
|
+
src/styles/site.css:9:1 Too high specificity in ".nav .item.is-open > a:hover", maximum "0,3,2" (selector-max-specificity)
|
|
81
152
|
```
|
|
82
153
|
|
|
83
154
|
A passing run counts what it judged:
|
|
@@ -89,9 +160,13 @@ frontend-syntax: no violation in 2 file(s) from 2 declared input(s)
|
|
|
89
160
|
## When it runs
|
|
90
161
|
|
|
91
162
|
`checks-lint` runs it when the repository tracks `frontend-syntax.json` at its root.
|
|
92
|
-
A repository without that file never runs it and needs neither stylelint nor
|
|
163
|
+
A repository without that file never runs it and needs neither `stylelint.config.ts` nor the peers.
|
|
93
164
|
|
|
94
165
|
## Related topics
|
|
95
166
|
|
|
96
167
|
- [checks-browser](checks-browser.md)
|
|
97
168
|
- [checks-lint](checks-lint.md)
|
|
169
|
+
- [checks-comment-gate](checks-comment-gate.md)
|
|
170
|
+
- [checks-suppressions-ratchet](checks-suppressions-ratchet.md)
|
|
171
|
+
- [checks-repetition](checks-repetition.md)
|
|
172
|
+
- [Native settings](../configs/native-settings.md)
|
|
@@ -13,6 +13,7 @@ Gates requiring tracked TypeScript files begin running when the repository track
|
|
|
13
13
|
`checks-lint-coverage` and `checks-unused` also begin running when the repository tracks an `.astro` file.
|
|
14
14
|
`checks-advisories` begins running when the repository tracks `bun.lock`.
|
|
15
15
|
`checks-frontend-syntax` begins running when the repository tracks `frontend-syntax.json` at its root.
|
|
16
|
+
`checks-copy` begins running when the repository tracks `copy.json` at its root.
|
|
16
17
|
All other gates run for every repository.
|
|
17
18
|
|
|
18
19
|
## What it reads
|
|
@@ -47,7 +48,7 @@ With two arguments, the base and head override range discovery.
|
|
|
47
48
|
|
|
48
49
|
```
|
|
49
50
|
checks-lint: range 2504acf098d120e73a8ece3c96f22b934f35c6a8..10ba7d8935b73ed72624120a1542e51bd21ca7c7 from HEAD against origin/main
|
|
50
|
-
checks-lint: 1 of
|
|
51
|
+
checks-lint: 1 of 18 gate(s) failed: checks-comment-gate
|
|
51
52
|
```
|
|
52
53
|
|
|
53
54
|
<!-- end generated lint-sample -->
|
|
@@ -181,13 +181,15 @@ It slims the report first, so the artifact holds about 0.5 MB in place of about
|
|
|
181
181
|
The full upload keeps 30 days, so the newest full baseline outlives the gap between full runs.
|
|
182
182
|
A pull request's scope step loses its coverage when no full baseline outlives the gap.
|
|
183
183
|
|
|
184
|
-
A pull request's scope step restores the full report with `--full
|
|
184
|
+
A pull request's scope step restores the full report with `--full`, and [checks-mutation-scope](checks-mutation-scope.md) reads it:
|
|
185
185
|
|
|
186
186
|
```yaml
|
|
187
187
|
- name: Select mutation scope
|
|
188
188
|
env:
|
|
189
189
|
GH_TOKEN: ${{ github.token }}
|
|
190
|
-
run:
|
|
190
|
+
run: |
|
|
191
|
+
bunx checks-mutation-baseline --full "$RUNNER_TEMP/baseline-full/mutation.json"
|
|
192
|
+
bunx checks-mutation-scope "$(git merge-base HEAD origin/main)" "$RUNNER_TEMP/baseline-full/mutation.json" >> "$GITHUB_ENV"
|
|
191
193
|
```
|
|
192
194
|
|
|
193
195
|
## Related topics
|
|
@@ -78,7 +78,7 @@ The shared preset refuses a full `stryker run` outside CI and names that workflo
|
|
|
78
78
|
|
|
79
79
|
## Running it in CI
|
|
80
80
|
|
|
81
|
-
It runs on pull requests from a workflow named `mutation-compare`, comparing the head report against a report built at the merge-base:
|
|
81
|
+
It runs on pull requests from a workflow named `mutation-compare`, comparing the head report against a report built at the merge-base over the same scope:
|
|
82
82
|
|
|
83
83
|
```yaml
|
|
84
84
|
name: mutation-compare
|
|
@@ -87,6 +87,9 @@ on:
|
|
|
87
87
|
jobs:
|
|
88
88
|
mutation-compare:
|
|
89
89
|
runs-on: ubuntu-latest
|
|
90
|
+
permissions:
|
|
91
|
+
contents: read
|
|
92
|
+
actions: read
|
|
90
93
|
steps:
|
|
91
94
|
- uses: actions/checkout@v5
|
|
92
95
|
with:
|
|
@@ -95,28 +98,50 @@ jobs:
|
|
|
95
98
|
with:
|
|
96
99
|
bun-version-file: .bun-version
|
|
97
100
|
- run: bun install --frozen-lockfile
|
|
98
|
-
-
|
|
99
|
-
|
|
101
|
+
- name: Select mutation scope
|
|
102
|
+
env:
|
|
103
|
+
GH_TOKEN: ${{ github.token }}
|
|
104
|
+
run: |
|
|
105
|
+
base="$(git merge-base HEAD "origin/$GITHUB_BASE_REF")"
|
|
106
|
+
echo "BASE_SHA=$base" >> "$GITHUB_ENV"
|
|
107
|
+
bunx checks-mutation-baseline --full "$RUNNER_TEMP/baseline-full/mutation.json"
|
|
108
|
+
bunx checks-mutation-scope "$base" "$RUNNER_TEMP/baseline-full/mutation.json" >> "$GITHUB_ENV"
|
|
109
|
+
- if: env.SCOPE == ''
|
|
110
|
+
run: echo "::notice::mutation-compare: nothing in mutation scope changed, so neither Stryker run starts"
|
|
111
|
+
- if: env.SCOPE != ''
|
|
112
|
+
run: bunx stryker run --mutate "$SCOPE"
|
|
113
|
+
- if: env.SCOPE != ''
|
|
114
|
+
run: |
|
|
100
115
|
base_worktree="$RUNNER_TEMP/mutation-base-$GITHUB_RUN_ID-$GITHUB_RUN_ATTEMPT"
|
|
101
116
|
echo "BASE_WORKTREE=$base_worktree" >> "$GITHUB_ENV"
|
|
102
117
|
git worktree prune
|
|
103
|
-
git worktree add "$base_worktree" "$
|
|
104
|
-
(cd "$base_worktree" && bun install --frozen-lockfile && bunx stryker run)
|
|
105
|
-
-
|
|
118
|
+
git worktree add "$base_worktree" "$BASE_SHA"
|
|
119
|
+
(cd "$base_worktree" && bun install --frozen-lockfile && bunx stryker run --mutate "$SCOPE")
|
|
120
|
+
- if: env.SCOPE != ''
|
|
121
|
+
run: bun run checks-mutation-compare --advisory "$BASE_WORKTREE/reports/mutation/mutation.json" reports/mutation/mutation.json
|
|
106
122
|
- if: always() && env.BASE_WORKTREE != ''
|
|
107
123
|
run: |
|
|
108
124
|
git worktree remove --force "$BASE_WORKTREE"
|
|
109
125
|
git worktree prune
|
|
110
126
|
```
|
|
111
127
|
|
|
112
|
-
|
|
128
|
+
The scope step picks the files the pull request changes and the sources its changed unit tests cover or kill, as [checks-mutation-scope](checks-mutation-scope.md) says.
|
|
129
|
+
Neither run mutates a file outside that scope, so the job's time follows the size of the change and not the size of the repository.
|
|
130
|
+
Both runs take the same `--mutate "$SCOPE"`, so the two reports hold the same files.
|
|
131
|
+
A scope file the merge base lacks matches nothing there, and Stryker still writes a report.
|
|
132
|
+
An empty scope skips both runs and the comparison, and the notice names the skip on the run's summary.
|
|
133
|
+
A changed unit test with no full baseline on `main` fails the scope step, since no report then says which sources the test covers.
|
|
134
|
+
A changed unit test that `main` edited after that baseline widens the scope to every file the `mutate` globs match.
|
|
135
|
+
Start a full baseline with `gh workflow run mutation`, as [checks-mutation](checks-mutation.md#running-it-in-ci) shows.
|
|
136
|
+
A run with `--mutate` is scoped, so the preset lets it through without `CI=true`.
|
|
113
137
|
The base worktree's path carries the run's id and attempt, and the last step removes it even when a run fails, so a runner kept between jobs starts each job clean.
|
|
114
138
|
A public repository keeps `runs-on: ubuntu-latest`, because a pull request from a fork runs its own code on the runner.
|
|
115
|
-
A private repository sets `runs-on: [self-hosted, Linux, X64, winbox]` instead, which is the fleet's self-hosted Linux runner where it sends its
|
|
139
|
+
A private repository sets `runs-on: [self-hosted, Linux, X64, winbox]` instead, which is the fleet's self-hosted Linux runner where it sends its mutation runs.
|
|
116
140
|
The job never reads `CI_RUNS_ON`, so an override that moves a repository's other jobs during a runner outage leaves the comparison waiting for `winbox`.
|
|
117
141
|
[checks-ci-wiring](checks-ci-wiring.md#runners) refuses a mutation job that names it in `runs-on`.
|
|
118
142
|
|
|
119
143
|
## Related topics
|
|
120
144
|
|
|
121
145
|
- [checks-mutation](checks-mutation.md)
|
|
146
|
+
- [checks-mutation-scope](checks-mutation-scope.md)
|
|
122
147
|
- [checks-test-layout](checks-test-layout.md)
|
|
@@ -0,0 +1,140 @@
|
|
|
1
|
+
---
|
|
2
|
+
kind: reference
|
|
3
|
+
audience: consumers
|
|
4
|
+
---
|
|
5
|
+
# checks-mutation-scope
|
|
6
|
+
|
|
7
|
+
`checks-mutation-scope` turns a pull request's diff into the files its mutation runs mutate, and prints them as lines for `GITHUB_ENV`.
|
|
8
|
+
|
|
9
|
+
## What it checks
|
|
10
|
+
|
|
11
|
+
It picks a scope rather than judging one, so it fails only when it cannot pick one.
|
|
12
|
+
It reads `git diff --name-status -M` from the merge base of `<base>` and `HEAD` to `HEAD`.
|
|
13
|
+
A changed source enters the scope when the repository's own Stryker `mutate` globs match it, with each negated glob applied in order.
|
|
14
|
+
It resolves each glob and each path against the working directory, as Stryker does, so an absolute glob matches too.
|
|
15
|
+
A `mutate` entry with a mutation range, such as `src/a.ts:1-10`, fails the run, since a scope of whole files cannot keep the range.
|
|
16
|
+
`--mutate` replaces those globs, so a path the config excludes, such as `packages/*/src/live/**` or `packages/*/src/bin.ts`, stays out of the scope.
|
|
17
|
+
A renamed source enters under both names, and a deleted source does not enter.
|
|
18
|
+
|
|
19
|
+
A unit test is a `.test.ts` or `.test.tsx` file under `tests/unit/`, the files the preset's `unitTestFiles` lists.
|
|
20
|
+
A unit test the pull request adds, edits, deletes or renames is changed.
|
|
21
|
+
A unit test that reads a changed file under `tests/` is changed too.
|
|
22
|
+
It reads a file when it imports it, directly or through another file under `tests/`.
|
|
23
|
+
An import resolves as Bun resolves it, to the file it names, then with an extension added, and only then to the directory's `index`.
|
|
24
|
+
A unit test also reads a changed fixture under `tests/fixtures/` when it names the fixture's path or a directory above it.
|
|
25
|
+
Bun loads each `[test] preload` in `bunfig.toml` before every test file.
|
|
26
|
+
So every unit test is changed when the preload list differs between the merge base and `HEAD`.
|
|
27
|
+
The same holds when the pull request changes a preload or a file a preload imports, wherever that file lives.
|
|
28
|
+
|
|
29
|
+
Each source whose mutants a changed unit test covers or kills in the full baseline report then enters the scope, when it still exists at `HEAD`.
|
|
30
|
+
So a kill lost through a test change alone still shows in the comparison, in a source the pull request leaves alone.
|
|
31
|
+
A changed unit test with no baseline report fails the run rather than reading incomplete coverage.
|
|
32
|
+
Stryker records each test file's source in the report's `testFiles`.
|
|
33
|
+
A changed unit test the merge base holds must have that recorded source match its content at the merge base.
|
|
34
|
+
Otherwise the report ran an older test, so its coverage cannot say what the test covers now.
|
|
35
|
+
The scope then widens to every file at `HEAD` the `mutate` globs match.
|
|
36
|
+
A unit test the pull request adds is new to every report, so it needs no entry.
|
|
37
|
+
Stryker writes no source without mutants into the report, so a mutated source the report lacks may be newer than it.
|
|
38
|
+
When a unit test changed, each such source that exists at `HEAD` enters the scope.
|
|
39
|
+
A change under `tests/e2e/` changes no unit test, so it needs no baseline report.
|
|
40
|
+
The scope trusts the coverage edges of the latest full baseline.
|
|
41
|
+
So a helper, fixture or preload changed on `main` since that baseline can hide a lost kill until the scheduled full baseline refreshes.
|
|
42
|
+
|
|
43
|
+
## What it reads
|
|
44
|
+
|
|
45
|
+
- The diff from the merge base to `HEAD`, and the files git tracks at both.
|
|
46
|
+
- The Stryker config file `stryker run` loads from the working directory, such as `stryker.conf.mjs`, for its `mutate` globs.
|
|
47
|
+
It imports a JavaScript config, so the step runs after `bun install`.
|
|
48
|
+
With no config file, or a config with no `mutate`, it takes Stryker's default globs.
|
|
49
|
+
- `bunfig.toml` at the merge base and at `HEAD`, for its `[test] preload` entries.
|
|
50
|
+
- Each preload at `HEAD`, and each file it imports through a relative path, for the files the preloads load.
|
|
51
|
+
- Each tracked `.ts` and `.tsx` file under `tests/` from the working tree, for its relative imports and the fixture paths it names.
|
|
52
|
+
- Each changed unit test at the merge base, to match it against the source the report recorded.
|
|
53
|
+
- The full baseline report that `checks-mutation-baseline --full` restores, for each mutant's `coveredBy` and `killedBy` and the `testFiles` that own those tests.
|
|
54
|
+
|
|
55
|
+
Stryker leaves `coveredBy` empty on a static mutant, so only `killedBy` names its tests.
|
|
56
|
+
The report must come from a run that restored no state and passed `--disableBail`, which lists every killing test.
|
|
57
|
+
|
|
58
|
+
## Arguments
|
|
59
|
+
|
|
60
|
+
```sh
|
|
61
|
+
checks-mutation-scope <base> [<full-baseline-report>]
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
`<base>` is the branch the pull request merges into, such as `origin/main`, or the merge base itself.
|
|
65
|
+
`<full-baseline-report>` is the path the full baseline was restored to, and a path with no file reads as no report.
|
|
66
|
+
|
|
67
|
+
## Output
|
|
68
|
+
|
|
69
|
+
It prints one line for each list, as `NAME=a,b`, with each list sorted by path:
|
|
70
|
+
|
|
71
|
+
| Line | What it lists |
|
|
72
|
+
| --- | --- |
|
|
73
|
+
| `SCOPE` | the files the base and head runs both mutate, empty when nothing in mutation scope changed |
|
|
74
|
+
| `STALE_TESTS` | the unit tests that count as changed only because they read a changed file |
|
|
75
|
+
|
|
76
|
+
Stryker reads each `--mutate` entry as a glob and turns each backslash into a `/`.
|
|
77
|
+
So `SCOPE` wraps each glob character in a path in a class of its own.
|
|
78
|
+
`app/routes/[id].ts` prints as `app/routes/[[]id[]].ts`, which matches that file and not `app/routes/i.ts`.
|
|
79
|
+
`STALE_TESTS` names each test by its literal path, as the incremental state keys it.
|
|
80
|
+
|
|
81
|
+
Stryker compares only a test file's own source when it resumes from incremental state.
|
|
82
|
+
A run that resumes from the merge base's state drops each `STALE_TESTS` entry from that state first, so Stryker retests the mutants those tests cover.
|
|
83
|
+
|
|
84
|
+
On stderr it names each unit test that reads a changed file, and each source a changed unit test brings in.
|
|
85
|
+
It also names each stale test that widens the scope.
|
|
86
|
+
|
|
87
|
+
## Exit codes
|
|
88
|
+
|
|
89
|
+
| Code | When |
|
|
90
|
+
| --- | --- |
|
|
91
|
+
| 0 | it printed the scope, an empty one included |
|
|
92
|
+
| 2 | the arguments do not parse, git fails, the Stryker config, `bunfig.toml` or the baseline report does not load, a `mutate` entry names a mutation range, or a unit test changed with no baseline report |
|
|
93
|
+
|
|
94
|
+
## Sample output
|
|
95
|
+
|
|
96
|
+
A helper change that reaches a source the pull request leaves alone:
|
|
97
|
+
|
|
98
|
+
```
|
|
99
|
+
mutation-scope: tests/unit/dashboard/snapshot.test.ts reads tests/lib/session-fixture.ts
|
|
100
|
+
mutation-scope: packages/ui/src/layout.ts enters through tests/unit/dashboard/snapshot.test.ts
|
|
101
|
+
SCOPE=packages/session-quota/src/notes.ts,packages/ui/src/layout.ts
|
|
102
|
+
STALE_TESTS=tests/unit/dashboard/snapshot.test.ts
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
A changed unit test with no full baseline on `main`, on stderr:
|
|
106
|
+
|
|
107
|
+
```
|
|
108
|
+
mutation-scope: tests/unit/dashboard/snapshot.test.ts changed, but no baseline report at /runner/temp/baseline-full/mutation.json says which sources they cover
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
A changed unit test that `main` edited after its newest full baseline, on stderr:
|
|
112
|
+
|
|
113
|
+
```
|
|
114
|
+
mutation-scope: the baseline report did not run tests/unit/dashboard/snapshot.test.ts as the merge base holds it, so every mutated source enters
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
## When it runs
|
|
118
|
+
|
|
119
|
+
Only a CI step the repository writes runs it, in the pull request's mutation comparison, before either Stryker run:
|
|
120
|
+
|
|
121
|
+
```yaml
|
|
122
|
+
- name: Select mutation scope
|
|
123
|
+
env:
|
|
124
|
+
GH_TOKEN: ${{ github.token }}
|
|
125
|
+
run: |
|
|
126
|
+
base="$(git merge-base HEAD "origin/$GITHUB_BASE_REF")"
|
|
127
|
+
echo "BASE_SHA=$base" >> "$GITHUB_ENV"
|
|
128
|
+
bunx checks-mutation-baseline --full "$RUNNER_TEMP/baseline-full/mutation.json"
|
|
129
|
+
bunx checks-mutation-scope "$base" "$RUNNER_TEMP/baseline-full/mutation.json" >> "$GITHUB_ENV"
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
Each later step reads `$SCOPE` from the environment, and a failed scope step adds nothing to it.
|
|
133
|
+
[checks-mutation-compare](checks-mutation-compare.md#running-it-in-ci) shows the whole job.
|
|
134
|
+
The full baseline comes from the `mutation` workflow, as [checks-mutation](checks-mutation.md#running-it-in-ci) shows.
|
|
135
|
+
|
|
136
|
+
## Related topics
|
|
137
|
+
|
|
138
|
+
- [checks-mutation-compare](checks-mutation-compare.md)
|
|
139
|
+
- [checks-mutation](checks-mutation.md)
|
|
140
|
+
- [checks-mutation-baseline](checks-mutation-baseline.md)
|
|
@@ -96,17 +96,20 @@ jobs:
|
|
|
96
96
|
with:
|
|
97
97
|
bun-version-file: .bun-version
|
|
98
98
|
- run: bun install --frozen-lockfile
|
|
99
|
-
- run: bunx stryker run
|
|
100
|
-
-
|
|
101
|
-
run: bunx checks-mutation-baseline --slim reports/mutation/mutation.json "$RUNNER_TEMP/report/mutation.json"
|
|
99
|
+
- run: bunx stryker run --disableBail
|
|
100
|
+
- run: bunx checks-mutation-baseline --slim reports/mutation/mutation.json "$RUNNER_TEMP/full/mutation.json"
|
|
102
101
|
- uses: actions/upload-artifact@v4
|
|
103
|
-
if: always()
|
|
104
102
|
with:
|
|
105
|
-
name: mutation-
|
|
106
|
-
path: ${{ runner.temp }}/
|
|
107
|
-
retention-days:
|
|
103
|
+
name: mutation-baseline-full
|
|
104
|
+
path: ${{ runner.temp }}/full/mutation.json
|
|
105
|
+
retention-days: 30
|
|
108
106
|
```
|
|
109
107
|
|
|
108
|
+
The run restores no state and passes `--disableBail`, so its report names every test that covers or kills each mutant.
|
|
109
|
+
The upload is the `mutation-baseline-full` artifact that a pull request's scope step restores with `checks-mutation-baseline --full`.
|
|
110
|
+
[checks-mutation-scope](checks-mutation-scope.md) reads it to bring in the sources a changed unit test covers or kills.
|
|
111
|
+
The restore reads only a successful run of `mutation.yml` on `main`, so a sweep that fails publishes nothing a pull request reads.
|
|
112
|
+
The artifact keeps 30 days, so the newest full baseline outlives the week between full runs.
|
|
110
113
|
The upload holds the report without each mutant's `statusReason`, as [checks-mutation-baseline](checks-mutation-baseline.md#the-saved-baselines) says, since the full report of a large repository passes 100 MB as an artifact.
|
|
111
114
|
|
|
112
115
|
The job runs on the fleet's self-hosted Linux runner labelled `winbox`, which is where a repository sends its full sweeps.
|
|
@@ -118,4 +121,5 @@ GitHub sets `CI=true` on every runner, so the preset lets the full run through t
|
|
|
118
121
|
## Related topics
|
|
119
122
|
|
|
120
123
|
- [checks-mutation-compare](checks-mutation-compare.md)
|
|
124
|
+
- [checks-mutation-scope](checks-mutation-scope.md)
|
|
121
125
|
- [checks-subsumed-tests](checks-subsumed-tests.md)
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
---
|
|
2
|
+
kind: reference
|
|
3
|
+
audience: consumers
|
|
4
|
+
---
|
|
5
|
+
# checks-no-plain-js
|
|
6
|
+
|
|
7
|
+
`checks-no-plain-js` is the gate that refuses a plain JavaScript source file in a repository that tracks TypeScript.
|
|
8
|
+
|
|
9
|
+
## What it checks
|
|
10
|
+
|
|
11
|
+
It fails on every `.js` or `.mjs` file in the working tree that none of the exemptions below covers.
|
|
12
|
+
Each line names the file and tells its writer to convert it to TypeScript.
|
|
13
|
+
`dist/` holds the committed build output, so the check passes over it.
|
|
14
|
+
`dependency-cruiser.config.js` at the repository root is written by `bun run build` from the TypeScript builder, so the check passes over that one path.
|
|
15
|
+
`node_modules/` and `repos/` hold installed and vendored trees, so the check passes over both.
|
|
16
|
+
`public/` holds assets the server sends verbatim, which no compiler reads, so the check passes over it.
|
|
17
|
+
`tests/fixtures/` holds data the suite never parses, so the check passes over it.
|
|
18
|
+
Stryker loads its config only as JSON or JavaScript, so the check passes over a config file carrying a Stryker config name, such as `stryker.conf.mjs`.
|
|
19
|
+
The Stryker preset ships as `stryker.preset.js` under its documented import path, so the check passes over that name.
|
|
20
|
+
A JavaScript config whose tool also loads TypeScript fails, such as a commitlint or dependency-cruiser config.
|
|
21
|
+
|
|
22
|
+
## What it reads
|
|
23
|
+
|
|
24
|
+
It reads the working tree.
|
|
25
|
+
It scans the tracked and untracked files that `git ls-files --exclude-standard` reports, so `node_modules/` and every gitignored tree are out of reach.
|
|
26
|
+
A local run agrees with CI before `git add`.
|
|
27
|
+
|
|
28
|
+
## Arguments
|
|
29
|
+
|
|
30
|
+
```sh
|
|
31
|
+
checks-no-plain-js [<directory>]
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
It checks the directory it runs in, or the directory it is given.
|
|
35
|
+
|
|
36
|
+
## Exit codes
|
|
37
|
+
|
|
38
|
+
| Code | When |
|
|
39
|
+
| --- | --- |
|
|
40
|
+
| 0 | the working tree holds no plain JavaScript source |
|
|
41
|
+
| 1 | a file in the working tree is plain JavaScript |
|
|
42
|
+
| 2 | git does not list the working tree |
|
|
43
|
+
|
|
44
|
+
## Sample output
|
|
45
|
+
|
|
46
|
+
```
|
|
47
|
+
no-plain-js: 2 violation(s)
|
|
48
|
+
src/legacy.js: plain JavaScript source in a TypeScript repository; convert it to TypeScript
|
|
49
|
+
tools/build.mjs: plain JavaScript source in a TypeScript repository; convert it to TypeScript
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
## When it runs
|
|
53
|
+
|
|
54
|
+
`checks-lint` runs it when the repository tracks TypeScript.
|
|
55
|
+
|
|
56
|
+
## Related topics
|
|
57
|
+
|
|
58
|
+
- [checks-lint](checks-lint.md)
|
|
59
|
+
- [checks-test-layout](checks-test-layout.md)
|
|
60
|
+
- [checks-unused](checks-unused.md)
|
|
@@ -23,6 +23,8 @@ A `path` entry the base revision lacks reads as empty there, so a newly measured
|
|
|
23
23
|
{ "path": ["src"], "format": ["typescript"], "ignore": ["**/*.d.ts"] }
|
|
24
24
|
```
|
|
25
25
|
|
|
26
|
+
A repository that lists `"css"` in `format` measures its stylesheets the same way, and [checks-frontend-syntax](checks-frontend-syntax.md) leaves CSS repetition to this gate.
|
|
27
|
+
|
|
26
28
|
The gate sets the 50 tokens and 5 lines itself, over any the file names.
|
|
27
29
|
A `threshold` the file names never fails the gate, which reads jscpd's report once the scan finishes.
|
|
28
30
|
The bin reads the base and head commits rather than the working tree's file contents.
|
|
@@ -56,5 +58,6 @@ When the head holds no `.jscpd.json`, the gate reports that no file was measured
|
|
|
56
58
|
|
|
57
59
|
## Related topics
|
|
58
60
|
|
|
61
|
+
- [checks-frontend-syntax](checks-frontend-syntax.md)
|
|
59
62
|
- [Native settings](../configs/native-settings.md)
|
|
60
63
|
- [checks-suppressions-ratchet](checks-suppressions-ratchet.md)
|
|
@@ -4,18 +4,19 @@ audience: consumers
|
|
|
4
4
|
---
|
|
5
5
|
# checks-suppressions-ratchet
|
|
6
6
|
|
|
7
|
-
`checks-suppressions-ratchet` is the gate that fails when a change raises a count in `oxlint-suppressions.json`.
|
|
7
|
+
`checks-suppressions-ratchet` is the gate that fails when a change raises a count in `oxlint-suppressions.json` or `stylelint-suppressions.json`.
|
|
8
8
|
|
|
9
9
|
## What it checks
|
|
10
10
|
|
|
11
11
|
oxlint accepts whatever `oxlint --suppress-all` writes to `oxlint-suppressions.json`, so raising a count to let a new site through passes the lint.
|
|
12
|
-
|
|
12
|
+
stylelint accepts `stylelint-suppressions.json` the same way, and writes it in the same shape, a count per rule per file.
|
|
13
|
+
The ratchet fails naming the suppressions file, the file and the rule of each count that rose or appeared.
|
|
13
14
|
A count that fell and an entry that went both pass.
|
|
14
15
|
|
|
15
16
|
## What it reads
|
|
16
17
|
|
|
17
|
-
It reads `oxlint-suppressions.json` at a base and at a head, from the directory the command runs in, which is where oxlint
|
|
18
|
-
A commit without
|
|
18
|
+
It reads `oxlint-suppressions.json` and `stylelint-suppressions.json` at a base and at a head, from the directory the command runs in, which is where oxlint and stylelint write them.
|
|
19
|
+
A commit without a file counts it as empty, so the commit that first adds a baseline fails with every entry appearing.
|
|
19
20
|
|
|
20
21
|
## Arguments
|
|
21
22
|
|
|
@@ -33,22 +34,23 @@ With one it compares that commit with its parent, or with the empty tree for a r
|
|
|
33
34
|
| --- | --- |
|
|
34
35
|
| 0 | no count rose and no entry appeared |
|
|
35
36
|
| 1 | a count rose or an entry appeared |
|
|
36
|
-
| 2 | a ref does not resolve, the parent exists but is not in the clone, or
|
|
37
|
+
| 2 | a ref does not resolve, the parent exists but is not in the clone, or a file is not a count per rule per file |
|
|
37
38
|
|
|
38
39
|
## Sample output
|
|
39
40
|
|
|
40
41
|
```
|
|
41
|
-
suppressions-ratchet: 2 count(s)
|
|
42
|
-
src/
|
|
43
|
-
src/
|
|
42
|
+
suppressions-ratchet: 2 count(s) rose or appeared; fix the site instead of suppressing it:
|
|
43
|
+
oxlint-suppressions.json src/dispatch.ts typescript/no-non-null-assertion rose from 12 to 13
|
|
44
|
+
stylelint-suppressions.json src/styles.css selector-max-specificity appeared with 1
|
|
44
45
|
```
|
|
45
46
|
|
|
46
47
|
## When it runs
|
|
47
48
|
|
|
48
49
|
`checks-lint` runs it over each pull request's range in every repository, as [checks-lint](checks-lint.md) says.
|
|
49
|
-
A repository with
|
|
50
|
+
A repository with neither file passes, since both ends count as empty.
|
|
50
51
|
|
|
51
52
|
## Related topics
|
|
52
53
|
|
|
53
54
|
- [The Effect rules](../configs/effect-rules.md)
|
|
55
|
+
- [checks-frontend-syntax](checks-frontend-syntax.md)
|
|
54
56
|
- [checks-lint](checks-lint.md)
|
|
@@ -49,7 +49,9 @@ It counts a run as `ci` when `CI` is true and as `local` otherwise.
|
|
|
49
49
|
|
|
50
50
|
The command takes no arguments for the default suite, except the worker count of [Parallel runs](#parallel-runs).
|
|
51
51
|
Use `checks-test --tier=live` or `checks-test --tier=pixel` for a named test tier.
|
|
52
|
-
|
|
52
|
+
Use `checks-test --tier=unit` or `checks-test --tier=e2e` to run only one level of the default suite.
|
|
53
|
+
A live or pixel run clears Bun's ignored paths, runs only `./tests/live` or `./tests/pixel`, and still checks each skip.
|
|
54
|
+
A unit or e2e run keeps Bun's ignored paths, runs only `./tests/unit` or `./tests/e2e`, and still checks each skip.
|
|
53
55
|
|
|
54
56
|
Files under `tests/quarantine/` are not run or judged, as [checks-test-layout](checks-test-layout.md) says.
|
|
55
57
|
|
|
@@ -90,6 +92,7 @@ The count is a whole number from 1 up, and without the option the suite runs ser
|
|
|
90
92
|
Tests still run in random order, and `test.only` still fails the run.
|
|
91
93
|
Bun merges every worker's results into the one JUnit report, so each skipped test and each todo is judged as in a serial run.
|
|
92
94
|
A tier script takes the same option, as in `checks-test --tier=live --parallel=4`.
|
|
95
|
+
[checks-flake](checks-flake.md#parallel-hunts) hunts with the worker count the `test` script sets.
|
|
93
96
|
|
|
94
97
|
A file whose worker crashes, for example when a test calls `process.exit`, fails the run.
|
|
95
98
|
Bun reports no result from that file, so its skips are not judged and its declarations are not reported as stale.
|
|
@@ -100,7 +103,7 @@ A timings option waits for Bun to honour timings under random order.
|
|
|
100
103
|
## Arguments
|
|
101
104
|
|
|
102
105
|
The default command takes no arguments except `--parallel=<workers>`, as [Parallel runs](#parallel-runs) says.
|
|
103
|
-
Its named tier options are `--tier=live` and `--tier=
|
|
106
|
+
Its named tier options are `--tier=live`, `--tier=pixel`, `--tier=unit` and `--tier=e2e`, and a tier run takes `--parallel=<workers>` too.
|
|
104
107
|
Each option appears at most once.
|
|
105
108
|
It refuses test filters because every test excluded by a filter would look skipped.
|
|
106
109
|
|