@avi2dg/checks 0.43.0 → 0.45.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +20 -0
- package/README.md +9 -4
- package/{commitlint.config.js → commitlint.config.ts} +1 -1
- package/dist/mutation/mutation-guard-plugin.js +40 -0
- package/dist/mutation/mutation-refusal.js +23 -0
- package/dist/presets/stylelint.d.ts +10 -0
- package/dist/presets/stylelint.js +73 -0
- package/docs/configs/commit-messages.md +2 -2
- package/docs/configs/native-settings.md +15 -2
- package/docs/design.md +26 -1
- package/docs/gates/checks-ci-wiring.md +3 -2
- package/docs/gates/checks-comment-gate.md +2 -0
- package/docs/gates/checks-copy.md +113 -0
- package/docs/gates/checks-docs-only.md +3 -2
- package/docs/gates/checks-docs.md +3 -0
- package/docs/gates/checks-flake.md +49 -1
- package/docs/gates/checks-frontend-syntax.md +87 -12
- package/docs/gates/checks-lint.md +2 -1
- package/docs/gates/checks-mutation-baseline.md +4 -2
- package/docs/gates/checks-mutation-compare.md +33 -8
- package/docs/gates/checks-mutation-scope.md +140 -0
- package/docs/gates/checks-mutation.md +11 -7
- package/docs/gates/checks-no-plain-js.md +60 -0
- package/docs/gates/checks-repetition.md +3 -0
- package/docs/gates/checks-suppressions-ratchet.md +11 -9
- package/docs/gates/checks-test.md +5 -2
- package/package.json +33 -6
- package/src/complexity/suppressions-ratchet.ts +37 -25
- package/src/core/declared-inputs.ts +77 -0
- package/src/core/gates.ts +4 -0
- package/src/core/git.ts +9 -0
- package/src/delivery/ci-wiring.ts +25 -11
- package/src/delivery/commit-title.ts +2 -2
- package/src/docs/copy-html.ts +110 -0
- package/src/docs/copy-text.ts +70 -0
- package/src/docs/copy.ts +70 -0
- package/src/docs/prose-matchers.ts +14 -14
- package/src/quality/comment-matchers.ts +4 -1
- package/src/quality/frontend-syntax.ts +114 -71
- package/src/quality/no-plain-js.ts +71 -0
- package/src/quality/presets/stylelint.ts +91 -0
- package/src/testing/flake-preload.ts +4 -0
- package/src/testing/flake.ts +142 -16
- package/src/testing/{mutation-scope.js → mutation-refusal.ts} +11 -9
- package/src/testing/mutation-scope.ts +354 -0
- package/src/testing/mutation.ts +1 -1
- package/src/testing/test-layout.ts +4 -20
- package/src/testing/test-skips.ts +1 -1
- package/src/testing/test.ts +19 -3
- package/stryker.preset.js +3 -3
- package/src/testing/mutation-guard-plugin.js +0 -13
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,26 @@
|
|
|
2
2
|
|
|
3
3
|
Every release of `@avi2dg/checks`, newest first, written by the release from its conventional commits.
|
|
4
4
|
|
|
5
|
+
## 0.45.0
|
|
6
|
+
|
|
7
|
+
Released 2026-10-18.
|
|
8
|
+
|
|
9
|
+
### Breaking changes
|
|
10
|
+
|
|
11
|
+
- **quality:** hold CSS to the stylelint rule families set in stylelint.config.ts [#167](https://github.com/avi2d/checks/pull/167)
|
|
12
|
+
|
|
13
|
+
## 0.44.0
|
|
14
|
+
|
|
15
|
+
Released 2026-10-18.
|
|
16
|
+
|
|
17
|
+
### Features
|
|
18
|
+
|
|
19
|
+
- split the test suite into parallel unit and e2e CI tiers [#162](https://github.com/avi2d/checks/pull/162)
|
|
20
|
+
- **testing:** hunt flakes with parallel test workers and replay failing workers [#164](https://github.com/avi2d/checks/pull/164)
|
|
21
|
+
- **mutation:** add checks-mutation-scope to limit PR mutation runs to the change [#161](https://github.com/avi2d/checks/pull/161)
|
|
22
|
+
- **quality:** add checks-no-plain-js gate refusing plain JavaScript sources [#163](https://github.com/avi2d/checks/pull/163)
|
|
23
|
+
- **docs:** add checks-copy gate to refuse dashes in declared copy [#160](https://github.com/avi2d/checks/pull/160)
|
|
24
|
+
|
|
5
25
|
## 0.43.0
|
|
6
26
|
|
|
7
27
|
Released 2026-10-17.
|
package/README.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# checks
|
|
2
2
|
|
|
3
3
|
`@avi2dg/checks` is the kit of deterministic checks a TypeScript repository installs to hold its code, tests, commits, CI wiring and docs to one shared standard.
|
|
4
|
-
It ships the lint gates `checks-lint` runs over each pull request, the test runners, and a `defineConfig` for oxlint, Knip
|
|
4
|
+
It ships the lint gates `checks-lint` runs over each pull request, the test runners, and a `defineConfig` for oxlint, Knip, dependency-cruiser and stylelint.
|
|
5
5
|
It also ships the configs a repository extends for tsc, commitlint, bun and Stryker.
|
|
6
6
|
Each repository owns its workflows and native tool configs, as [Native settings](docs/configs/native-settings.md) maps.
|
|
7
7
|
|
|
@@ -26,6 +26,8 @@ Each repository owns its workflows and native tool configs, as [Native settings]
|
|
|
26
26
|
- `playwright-core` 1.63.0
|
|
27
27
|
- `postcss-html` 2.0.0
|
|
28
28
|
- `stylelint` 17.16.0
|
|
29
|
+
- `stylelint-config-standard` 40.0.0
|
|
30
|
+
- `stylelint-plugin-use-baseline` 1.4.7
|
|
29
31
|
|
|
30
32
|
<!-- end generated prerequisites -->
|
|
31
33
|
|
|
@@ -170,9 +172,11 @@ The table groups the gates by vector, the part of a repository each one judges.
|
|
|
170
172
|
| quality | [`checks-comment-gate`](docs/gates/checks-comment-gate.md) | the range | every repository |
|
|
171
173
|
| quality | [`checks-effect-scope`](docs/gates/checks-effect-scope.md) | the working tree | a repository tracking `oxlint.config.ts` |
|
|
172
174
|
| quality | [`checks-frontend-syntax`](docs/gates/checks-frontend-syntax.md) | the working tree | a repository tracking `frontend-syntax.json` |
|
|
175
|
+
| quality | [`checks-no-plain-js`](docs/gates/checks-no-plain-js.md) | the working tree | a repository tracking `*.ts` or `*.tsx` |
|
|
173
176
|
| testing | [`checks-test-layout`](docs/gates/checks-test-layout.md) | the working tree | a repository tracking `*.ts` or `*.tsx` |
|
|
174
177
|
| testing | [`checks-quarantine-clock`](docs/gates/checks-quarantine-clock.md) | the range | every repository |
|
|
175
178
|
| docs | [`checks-docs`](docs/gates/checks-docs.md) | the range, and every agent file at the head commit | every repository |
|
|
179
|
+
| docs | [`checks-copy`](docs/gates/checks-copy.md) | the working tree | a repository tracking `copy.json` |
|
|
176
180
|
| delivery | [`checks-commit-identity`](docs/gates/checks-commit-identity.md) | the range | every repository |
|
|
177
181
|
| delivery | [`checks-ci-wiring`](docs/gates/checks-ci-wiring.md) | the working tree | every repository |
|
|
178
182
|
| delivery | [`checks-secrets`](docs/gates/checks-secrets.md) | the range | every repository |
|
|
@@ -188,6 +192,7 @@ These bins run on their own:
|
|
|
188
192
|
- [`checks-mutation`](docs/gates/checks-mutation.md) runs Stryker for scoped checks and refuses a full run outside CI.
|
|
189
193
|
- [`checks-mutation-compare`](docs/gates/checks-mutation-compare.md) compares the mutants of two reports and lists each one that regresses.
|
|
190
194
|
- [`checks-mutation-baseline`](docs/gates/checks-mutation-baseline.md) restores the newest mutation baseline from `main`, saves it on a self-hosted runner's disk in place of an artifact, and deletes superseded baselines after a `main` run.
|
|
195
|
+
- [`checks-mutation-scope`](docs/gates/checks-mutation-scope.md) turns a pull request's diff into the files its mutation runs mutate.
|
|
191
196
|
- [`checks-subsumed-tests`](docs/gates/checks-subsumed-tests.md) lists each test another test subsumes in a mutation run.
|
|
192
197
|
- [`checks-changelog`](docs/gates/checks-changelog.md) writes the pending release into `CHANGELOG.md` from the conventional commits since the last release.
|
|
193
198
|
- [`checks-release-notes`](docs/gates/checks-release-notes.md) writes one `CHANGELOG.md` section to a file for a GitHub release.
|
|
@@ -207,7 +212,7 @@ oxlint runs through its own tool, as the pages under Related topics say.
|
|
|
207
212
|
To move a repository to a newer release of the kit:
|
|
208
213
|
|
|
209
214
|
1. Run the install line again, which moves the kit to its newest release and the peers to the versions it pins.
|
|
210
|
-
A repository that opted in to `checks-browser
|
|
215
|
+
A repository that opted in to `checks-browser`, `checks-frontend-syntax` or `checks-copy` also reruns the `bun add` line on its page.
|
|
211
216
|
1. Run `bun run build`, whose `checks-effect-scope` rewrites the Effect override and the kit's severities in `tsconfig.json` when a release changes them.
|
|
212
217
|
A severity key the release drops stays in `tsconfig.json` until the repository removes it.
|
|
213
218
|
1. Copy `node_modules/@avi2dg/checks/bunfig.toml` over `bunfig.toml` again, since `checks-test-layout` compares the copy with the installed preset.
|
|
@@ -227,11 +232,11 @@ Every path is relative to the installed package, `node_modules/@avi2dg/checks/`.
|
|
|
227
232
|
| `CHANGELOG.md` | every release, and what it changed |
|
|
228
233
|
| `docs/` | a reference page per bin and per shared config, and why the kit is shaped this way |
|
|
229
234
|
| `bunfig.toml` | the bunfig preset a repository copies |
|
|
230
|
-
| `commitlint.config.
|
|
235
|
+
| `commitlint.config.ts` | the shared commitlint config |
|
|
231
236
|
| `dependency-cruiser.config.js` | the kit's dependency-cruiser rules as a base a `.dependency-cruiser.cjs` extends by path, which the build writes |
|
|
232
237
|
| `knip-base.json` | the Knip `include` setting, for a configuration that spreads it |
|
|
233
238
|
| `src/` | every bin, which a package script calls by its `checks-` name, the modules the bins import, and the Effect language service severities under `src/quality/presets/` |
|
|
234
|
-
| `dist/` | the compiled oxlint plugins, the config builders `@avi2dg/checks/oxlint`, `@avi2dg/checks/knip
|
|
239
|
+
| `dist/` | the compiled oxlint plugins, the config builders `@avi2dg/checks/oxlint`, `@avi2dg/checks/knip`, `@avi2dg/checks/dependency-cruiser` and `@avi2dg/checks/stylelint` resolve to, and the doc templates, one template per kind of doc file |
|
|
235
240
|
| `oxlintrc.json` | the oxlint `base` as a config a `.oxlintrc.json` extends by path, which the build writes |
|
|
236
241
|
| `stryker.preset.js` | the Stryker mutation-testing preset, which refuses a full run outside CI |
|
|
237
242
|
| `tsconfig.effect.json` | the tsconfig fragment with the shared compiler options and the Effect language-service block |
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
// src/testing/mutation-guard-plugin.ts
|
|
2
|
+
import { existsSync } from "node:fs";
|
|
3
|
+
|
|
4
|
+
// src/testing/mutation-refusal.ts
|
|
5
|
+
var NON_RUNS = ["--help", "-h", "--version"];
|
|
6
|
+
var MUTATE_FLAGS = ["--mutate", "-m"];
|
|
7
|
+
function namesGlob(args) {
|
|
8
|
+
return args.some((arg, index) => MUTATE_FLAGS.includes(arg) && (args[index + 1] ?? "") !== "" || arg.startsWith("--mutate=") && arg !== "--mutate=");
|
|
9
|
+
}
|
|
10
|
+
function allowedAnyway(args, ci) {
|
|
11
|
+
return ci === "true" || args.some((arg) => NON_RUNS.includes(arg)) || namesGlob(args);
|
|
12
|
+
}
|
|
13
|
+
function fullRunRefusal(args, ci) {
|
|
14
|
+
if (allowedAnyway(args, ci) || args.includes("--incremental") && !args.includes("--force"))
|
|
15
|
+
return;
|
|
16
|
+
return "refusing a full mutation run outside CI; start the same run in CI with `gh workflow run mutation`, or scope this run with `--mutate <glob>` or `--incremental`";
|
|
17
|
+
}
|
|
18
|
+
function missingReportRefusal(args, ci, incrementalFile, reportExists) {
|
|
19
|
+
if (allowedAnyway(args, ci) || reportExists(incrementalFile))
|
|
20
|
+
return;
|
|
21
|
+
return `refusing a full mutation run outside CI: \`--incremental\` finds no report at ${incrementalFile} to reuse; scope this run with \`--mutate <glob>\`, or start the full baseline in CI with \`gh workflow run mutation\``;
|
|
22
|
+
}
|
|
23
|
+
|
|
24
|
+
// src/testing/mutation-guard-plugin.ts
|
|
25
|
+
var GUARD_IGNORER = "checks-incremental-report-guard";
|
|
26
|
+
function incrementalReportGuard(options) {
|
|
27
|
+
const refusal = missingReportRefusal(process.argv.slice(3), process.env.CI ?? "", options.incrementalFile, existsSync);
|
|
28
|
+
if (refusal !== undefined)
|
|
29
|
+
throw new Error(refusal);
|
|
30
|
+
return { shouldIgnore: () => {
|
|
31
|
+
return;
|
|
32
|
+
} };
|
|
33
|
+
}
|
|
34
|
+
var strykerPlugins = [
|
|
35
|
+
{ kind: "Ignore", name: GUARD_IGNORER, factory: Object.assign(incrementalReportGuard, { inject: ["options"] }) }
|
|
36
|
+
];
|
|
37
|
+
export {
|
|
38
|
+
GUARD_IGNORER,
|
|
39
|
+
strykerPlugins
|
|
40
|
+
};
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
// src/testing/mutation-refusal.ts
|
|
2
|
+
var NON_RUNS = ["--help", "-h", "--version"];
|
|
3
|
+
var MUTATE_FLAGS = ["--mutate", "-m"];
|
|
4
|
+
function namesGlob(args) {
|
|
5
|
+
return args.some((arg, index) => MUTATE_FLAGS.includes(arg) && (args[index + 1] ?? "") !== "" || arg.startsWith("--mutate=") && arg !== "--mutate=");
|
|
6
|
+
}
|
|
7
|
+
function allowedAnyway(args, ci) {
|
|
8
|
+
return ci === "true" || args.some((arg) => NON_RUNS.includes(arg)) || namesGlob(args);
|
|
9
|
+
}
|
|
10
|
+
function fullRunRefusal(args, ci) {
|
|
11
|
+
if (allowedAnyway(args, ci) || args.includes("--incremental") && !args.includes("--force"))
|
|
12
|
+
return;
|
|
13
|
+
return "refusing a full mutation run outside CI; start the same run in CI with `gh workflow run mutation`, or scope this run with `--mutate <glob>` or `--incremental`";
|
|
14
|
+
}
|
|
15
|
+
function missingReportRefusal(args, ci, incrementalFile, reportExists) {
|
|
16
|
+
if (allowedAnyway(args, ci) || reportExists(incrementalFile))
|
|
17
|
+
return;
|
|
18
|
+
return `refusing a full mutation run outside CI: \`--incremental\` finds no report at ${incrementalFile} to reuse; scope this run with \`--mutate <glob>\`, or start the full baseline in CI with \`gh workflow run mutation\``;
|
|
19
|
+
}
|
|
20
|
+
export {
|
|
21
|
+
fullRunRefusal,
|
|
22
|
+
missingReportRefusal
|
|
23
|
+
};
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
import type { Config } from "stylelint";
|
|
2
|
+
type Globs = readonly [string, ...string[]];
|
|
3
|
+
export type BaselineTarget = "widely" | "newly" | number;
|
|
4
|
+
export type ChecksStylelintConfig = Omit<Config, "extends" | "plugins"> & {
|
|
5
|
+
readonly baseline: BaselineTarget;
|
|
6
|
+
readonly tokens: Globs | false;
|
|
7
|
+
};
|
|
8
|
+
export declare const MARKUP: readonly ["**/*.html", "**/*.astro", "**/*.vue", "**/*.svelte"];
|
|
9
|
+
export declare function defineConfig({ baseline, tokens, rules, overrides, ...rest }: ChecksStylelintConfig): Config;
|
|
10
|
+
export {};
|
|
@@ -0,0 +1,73 @@
|
|
|
1
|
+
// src/quality/presets/stylelint.ts
|
|
2
|
+
import { fileURLToPath } from "node:url";
|
|
3
|
+
var HTML_AND_ASTRO = ["**/*.html", "**/*.astro"];
|
|
4
|
+
var MARKUP = [...HTML_AND_ASTRO, "**/*.vue", "**/*.svelte"];
|
|
5
|
+
function peer(name) {
|
|
6
|
+
return fileURLToPath(import.meta.resolve(name));
|
|
7
|
+
}
|
|
8
|
+
var CONVENTION_CHANGES = {
|
|
9
|
+
"no-descending-specificity": null,
|
|
10
|
+
"property-no-vendor-prefix": [true, { disableFix: true, ignoreProperties: ["-webkit-user-select"] }]
|
|
11
|
+
};
|
|
12
|
+
var SELECTOR_COMPLEXITY = {
|
|
13
|
+
"selector-max-specificity": "0,3,2",
|
|
14
|
+
"selector-max-compound-selectors": 3
|
|
15
|
+
};
|
|
16
|
+
var COLOUR_FUNCTIONS = ["rgb", "rgba", "hsl", "hsla", "hwb", "lab", "lch", "oklab", "oklch", "color"];
|
|
17
|
+
var DESIGN_TOKENS = {
|
|
18
|
+
"color-no-hex": true,
|
|
19
|
+
"color-named": "never",
|
|
20
|
+
"function-disallowed-list": COLOUR_FUNCTIONS.map((name) => `/^${name}$/i`),
|
|
21
|
+
"declaration-property-value-allowed-list": { "/^font-size$/i": ["/^var\\(\\s*--[^\\s,()]+\\s*\\)$/i", "/^inherit$/i"] }
|
|
22
|
+
};
|
|
23
|
+
var REFUSED_BASELINE = '@avi2dg/checks/stylelint: set baseline to "widely", "newly" or a year such as 2024';
|
|
24
|
+
var REFUSED_TOKENS = "@avi2dg/checks/stylelint: set tokens to the globs of the token files, or false to leave the design token rules off";
|
|
25
|
+
function baselineTarget(baseline) {
|
|
26
|
+
if (baseline === "widely" || baseline === "newly")
|
|
27
|
+
return baseline;
|
|
28
|
+
if (typeof baseline === "number" && Number.isInteger(baseline))
|
|
29
|
+
return baseline;
|
|
30
|
+
throw new TypeError(REFUSED_BASELINE);
|
|
31
|
+
}
|
|
32
|
+
function isGlobList(value) {
|
|
33
|
+
return Array.isArray(value) && value.every((glob) => typeof glob === "string" && glob !== "");
|
|
34
|
+
}
|
|
35
|
+
function tokenFiles(tokens) {
|
|
36
|
+
if (tokens === false)
|
|
37
|
+
return [];
|
|
38
|
+
if (isGlobList(tokens) && tokens.length > 0)
|
|
39
|
+
return tokens;
|
|
40
|
+
throw new TypeError(REFUSED_TOKENS);
|
|
41
|
+
}
|
|
42
|
+
function designTokens(files) {
|
|
43
|
+
if (files.length === 0)
|
|
44
|
+
return { rules: {}, overrides: [] };
|
|
45
|
+
const off = Object.fromEntries(Object.keys(DESIGN_TOKENS).map((rule) => [rule, null]));
|
|
46
|
+
return { rules: DESIGN_TOKENS, overrides: [{ files: [...files], rules: off }] };
|
|
47
|
+
}
|
|
48
|
+
function defineConfig({ baseline, tokens, rules, overrides = [], ...rest }) {
|
|
49
|
+
const available = baselineTarget(baseline);
|
|
50
|
+
const tokenRules = designTokens(tokenFiles(tokens));
|
|
51
|
+
return {
|
|
52
|
+
...rest,
|
|
53
|
+
extends: [peer("stylelint-config-standard")],
|
|
54
|
+
plugins: [peer("stylelint-plugin-use-baseline")],
|
|
55
|
+
rules: {
|
|
56
|
+
...CONVENTION_CHANGES,
|
|
57
|
+
...SELECTOR_COMPLEXITY,
|
|
58
|
+
"plugin/use-baseline": [true, { available }],
|
|
59
|
+
...tokenRules.rules,
|
|
60
|
+
...rules
|
|
61
|
+
},
|
|
62
|
+
overrides: [
|
|
63
|
+
{ files: [...MARKUP], customSyntax: peer("postcss-html") },
|
|
64
|
+
{ files: [...HTML_AND_ASTRO], rules: { "declaration-block-single-line-max-declarations": null } },
|
|
65
|
+
...tokenRules.overrides,
|
|
66
|
+
...overrides
|
|
67
|
+
]
|
|
68
|
+
};
|
|
69
|
+
}
|
|
70
|
+
export {
|
|
71
|
+
MARKUP,
|
|
72
|
+
defineConfig
|
|
73
|
+
};
|
|
@@ -8,14 +8,14 @@ The shared commitlint config holds each pull request title to the conventional c
|
|
|
8
8
|
|
|
9
9
|
## Config
|
|
10
10
|
|
|
11
|
-
Commits follow `@commitlint/config-conventional` plus the house prefixes `commitlint.config.
|
|
11
|
+
Commits follow `@commitlint/config-conventional` plus the house prefixes `commitlint.config.ts` lists, shared from `node_modules/@avi2dg/checks/commitlint.config.ts`.
|
|
12
12
|
It arrives with the kit, since `@commitlint/cli` and `@commitlint/config-conventional` are dependencies, not peers.
|
|
13
13
|
|
|
14
14
|
## Workflow
|
|
15
15
|
|
|
16
16
|
The lint runs in CI on pull requests, because `jj` never fires a git hook.
|
|
17
17
|
Each repository lints the title in one step of its `ci` job with `bunx checks-commit-title`.
|
|
18
|
-
The workflow lints with the installed kit's `commitlint.config.
|
|
18
|
+
The workflow lints with the installed kit's `commitlint.config.ts`, so every repository holds titles to the same rules.
|
|
19
19
|
The `ci` job reports the title step's outcome as a `commitlint` status on every run, so a required `commitlint` status stays while `commitlint.yml` goes away.
|
|
20
20
|
That post needs the workflow's `statuses: write` permission.
|
|
21
21
|
The title lint runs before the docs-only filter, so a change to top-level Markdown and `docs/` Markdown alone still lints its title.
|
|
@@ -13,9 +13,11 @@ A consuming repository puts each setting in the file its tool reads, and no mani
|
|
|
13
13
|
| `.github/workflows/*.yml` | Pull request commands, scheduled commands, runner labels | GitHub Actions and `checks-ci-wiring` |
|
|
14
14
|
| `oxlint.config.ts` | Effect paths and their exemptions, ignore patterns, and any rule or budget the repository sets over the kit's defaults | oxlint and `checks-effect-scope` |
|
|
15
15
|
| `oxlint-suppressions.json` | The existing violations oxlint suppresses, per file and rule | oxlint and `checks-suppressions-ratchet` |
|
|
16
|
+
| `stylelint-suppressions.json` | The existing violations stylelint suppresses, per file and rule | stylelint, `checks-frontend-syntax` and `checks-suppressions-ratchet` |
|
|
16
17
|
| `exports-baseline.json` | The existing unused exports and types, per file, kind and name | `checks-exports` |
|
|
17
18
|
| `.jscpd.json` | The `path` and `ignore` globs of the files repetition is measured in | jscpd and `checks-repetition` |
|
|
18
19
|
| `knip.config.ts` | The `entry` globs Knip traces unreferenced files from | Knip, `checks-unused` and `checks-exports` |
|
|
20
|
+
| `stylelint.config.ts` | The Baseline target, the token files, and any stylelint rule the repository sets over the kit's families | `checks-frontend-syntax` |
|
|
19
21
|
| `dependency-cruiser.config.ts` | Entry points nothing imports, paths that may import dev dependencies, and import boundaries, when the kit's defaults do not fit | `checks-imports` |
|
|
20
22
|
| `tsconfig.json` | Compiler options, and the Effect language service plugin whose `overrides` and kit severities `checks-effect-scope` writes, keeping every other key | TypeScript and Effect language service |
|
|
21
23
|
| `package.json` | `scripts` with the `checks-vendor` arguments in `prepare`, `author` and `contributors` | Bun, `checks-commit-identity` and `checks-vendor` |
|
|
@@ -29,7 +31,7 @@ A page whose front matter sets `audience: consumers` names commands that a consu
|
|
|
29
31
|
|
|
30
32
|
## The config builders
|
|
31
33
|
|
|
32
|
-
The kit ships a `defineConfig` for oxlint, Knip
|
|
34
|
+
The kit ships a `defineConfig` for oxlint, Knip, dependency-cruiser and stylelint.
|
|
33
35
|
Each takes the tool's own config type, less the keys the kit owns, and returns the whole config with a default for every kit rule.
|
|
34
36
|
A key only the repository knows has no default, so `tsc` refuses a config that leaves it out.
|
|
35
37
|
The builder throws when the tool loads such a config anyway.
|
|
@@ -39,6 +41,7 @@ The builder throws when the tool loads such a config anyway.
|
|
|
39
41
|
| `@avi2dg/checks/oxlint` | `effect` | `extends`, `plugins`, `jsPlugins` and `categories` |
|
|
40
42
|
| `@avi2dg/checks/knip` | `entry`, where `[]` means the package scripts and tests name every entry | `include` |
|
|
41
43
|
| `@avi2dg/checks/dependency-cruiser` | nothing | `extends` |
|
|
44
|
+
| `@avi2dg/checks/stylelint` | `baseline` and `tokens`, where `false` leaves the design token rules off | `extends` and `plugins` |
|
|
42
45
|
|
|
43
46
|
A repository whose sources are Effect programs writes `oxlint.config.ts`:
|
|
44
47
|
|
|
@@ -75,7 +78,16 @@ export default defineConfig({ devOnly: ["^(?:evals|tests)/"] });
|
|
|
75
78
|
```
|
|
76
79
|
|
|
77
80
|
[The dependency rules](dependency-rules.md) lists its keys.
|
|
78
|
-
|
|
81
|
+
|
|
82
|
+
A repository that declares `frontend-syntax.json` writes `stylelint.config.ts`, which [checks-frontend-syntax](../gates/checks-frontend-syntax.md) loads and describes:
|
|
83
|
+
|
|
84
|
+
```ts
|
|
85
|
+
import { defineConfig } from "@avi2dg/checks/stylelint";
|
|
86
|
+
|
|
87
|
+
export default defineConfig({ baseline: "widely", tokens: false });
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
`tsconfig.json` keeps the config files in its program, so `bun run typecheck` holds them to the builders' types.
|
|
79
91
|
|
|
80
92
|
## Size limits
|
|
81
93
|
|
|
@@ -108,4 +120,5 @@ overrides: [{ files: ["src/lib/server.ts"], rules: { "max-lines": ["error", { ma
|
|
|
108
120
|
- [The dependency rules](dependency-rules.md)
|
|
109
121
|
- [checks-ci-wiring](../gates/checks-ci-wiring.md)
|
|
110
122
|
- [checks-suppressions-ratchet](../gates/checks-suppressions-ratchet.md)
|
|
123
|
+
- [checks-frontend-syntax](../gates/checks-frontend-syntax.md)
|
|
111
124
|
- [checks-docs](../gates/checks-docs.md)
|
package/docs/design.md
CHANGED
|
@@ -98,6 +98,7 @@ npm adds `package.json`, `README.md` and `LICENSE` whatever `files` says.
|
|
|
98
98
|
`bun pm pack` builds the same tarball the registry serves, and the consumer e2e test installs that tarball.
|
|
99
99
|
|
|
100
100
|
Each oxlint plugin and each config builder ships compiled under `dist/`, because Node refuses to strip types from a `.ts` file under `node_modules`.
|
|
101
|
+
Stryker loads its config under Node too, so the guard plugin and run-scope helpers the Stryker preset imports ship compiled under `dist/mutation/`.
|
|
101
102
|
oxlint loads `oxlint.config.ts` under Node, so the builders ship as JavaScript beside a `.d.ts` that `tsc` reads.
|
|
102
103
|
`@oxlint/plugins` ships no RuleTester, so each `effect-channel`, `readability` and `data-shape` rule is proven red and green against an installed consumer in `tests/e2e/consumer.test.ts`, except `readability/thin-astro`, which `tests/e2e/astro-consumer.test.ts` proves against a consumer tree linked to the checkout.
|
|
103
104
|
`dist/` is committed, with the doc templates in `dist/templates/`, and so is `CHANGELOG.md`, which the same build writes.
|
|
@@ -301,7 +302,12 @@ So a product opts in by declaring `browser-checks.json`, `checks-lint` never sta
|
|
|
301
302
|
`checks-frontend-syntax` needs no browser and runs inside `checks-lint`, but only where `frontend-syntax.json` declares its inputs, since a repository with no interface has no CSS to hold to it.
|
|
302
303
|
|
|
303
304
|
Each check wraps a detector that already works rather than a rule of the kit's own.
|
|
304
|
-
The syntax gate runs two of stylelint's built-in rules, `axe` runs axe-core and `nesting` runs one html-validate rule.
|
|
305
|
+
The syntax gate runs two of stylelint's built-in rules and the stylelint team's shared configs, `axe` runs axe-core and `nesting` runs one html-validate rule.
|
|
306
|
+
Its Baseline family runs `stylelint-plugin-use-baseline`, the one stylelint rule that judges against the Baseline data rather than a browser list.
|
|
307
|
+
Its design token family is stylelint's own colour and value rules, applied outside the files a repository names as its tokens.
|
|
308
|
+
A check of the rendered page against declared tokens would need a token format of the kit's own.
|
|
309
|
+
The repository's `stylelint.config.ts` sets the Baseline target and the token files, since only the repository knows either.
|
|
310
|
+
The gate imports that file itself, because stylelint's config loader calls `typescript.findConfigFile`, which TypeScript 7 lacks, and a JavaScript config is what [checks-no-plain-js](gates/checks-no-plain-js.md) refuses.
|
|
305
311
|
The `layout`, `keyboard` and `motion` probes are the website's own browser tests, generalized after they held on its English and Russian pages.
|
|
306
312
|
A zoom-disabling viewport is judged by axe on the rendered page, because stylelint reads only CSS and html-validate has no rule for the viewport's content.
|
|
307
313
|
|
|
@@ -315,6 +321,25 @@ So the runner calls the product's hooks at each visit and holds them to the same
|
|
|
315
321
|
The runner drives the Chrome a machine already has through `playwright-core`, since Playwright's own browser builds are a separate download of their own.
|
|
316
322
|
GitHub's hosted Ubuntu runners carry Chrome, and `CHROME_PATH` names any other build.
|
|
317
323
|
|
|
324
|
+
## Copy is judged in the files a site renders
|
|
325
|
+
|
|
326
|
+
`checks-copy` takes the three separator rules from `src/docs/prose-matchers.ts`, so a doc and a page refuse the same characters in the same words.
|
|
327
|
+
Its Markdown goes through the scanner `checks-docs` uses, so the two gates never disagree about what is code.
|
|
328
|
+
Its HTML goes through html-validate, which the `nesting` check already runs, with one rule of the kit's own.
|
|
329
|
+
No rule html-validate or markdownlint ships refuses a spaced hyphen between words.
|
|
330
|
+
textlint and Vale could each carry the rule, but each adds a package or a binary.
|
|
331
|
+
Each would also hold a second copy of patterns the kit already holds.
|
|
332
|
+
|
|
333
|
+
The gate reads files rather than strings in source.
|
|
334
|
+
An oxlint rule over string literals and JSX text was run across the repositories that consume the kit.
|
|
335
|
+
It found mostly code inside a string, test fixtures and error messages.
|
|
336
|
+
The few hits that were copy, such as a line a CLI prints, look to a program like `scrollHeight - innerHeight` in a test script.
|
|
337
|
+
A site's strings all reach its built pages, so a glob over the built HTML judges each one where a visitor reads it.
|
|
338
|
+
|
|
339
|
+
Only a repository knows which of its files are copy, so it opts in by declaring them in `copy.json`.
|
|
340
|
+
The gate judges every line of each declared file, unlike the prose rules in `checks-docs`, which judge only the lines a change edits.
|
|
341
|
+
A visitor reads the whole page, so a dash on a line the change left alone is as wrong as one it added.
|
|
342
|
+
|
|
318
343
|
## Related topics
|
|
319
344
|
|
|
320
345
|
- [checks](../README.md)
|
|
@@ -11,6 +11,7 @@ audience: consumers
|
|
|
11
11
|
The kit requires `bunx checks-commit-title` on every pull request.
|
|
12
12
|
A step that calls `bun src/delivery/commit-title.ts` or `./node_modules/.bin/commitlint` directly still satisfies the check.
|
|
13
13
|
It requires `bun run lint`, `bun run build`, `bun run typecheck` and `bun run test` for each of those scripts that `package.json` defines.
|
|
14
|
+
When `package.json` defines both `test:unit` and `test:e2e`, a step for each of `bun run test:unit` and `bun run test:e2e` also satisfies `bun run test`.
|
|
14
15
|
A `build` script also requires `git diff --exit-code`.
|
|
15
16
|
The target branch is the pull request base in CI, else the branch `refs/remotes/origin/HEAD` names, else the default branch of the repository in GitHub's event, else `main`.
|
|
16
17
|
A `pull_request` trigger without a branch filter covers every target branch.
|
|
@@ -43,7 +44,7 @@ A file counts as a shell script when its name ends in `.sh` or its first line na
|
|
|
43
44
|
The call counts as a command of its own or inside a command substitution, never as an argument to another command.
|
|
44
45
|
A here-document that `sh`, `bash`, `dash`, `ksh` or `zsh` reads is a script, and its calls count.
|
|
45
46
|
The text of any other here-document is data, so a call there counts only inside a command substitution when the delimiter is unquoted.
|
|
46
|
-
A mutation job never names `CI_RUNS_ON` directly in `runs-on`, as `vars.CI_RUNS_ON` or `vars['CI_RUNS_ON']`, so an override for an outage cannot send its
|
|
47
|
+
A mutation job never names `CI_RUNS_ON` directly in `runs-on`, as `vars.CI_RUNS_ON` or `vars['CI_RUNS_ON']`, so an override for an outage cannot send its mutation runs to hosted runners.
|
|
47
48
|
Any other job that names `CI_RUNS_ON` in `runs-on` carries exactly the hosted-default expression `${{ vars.CI_RUNS_ON || 'ubuntu-latest' }}`.
|
|
48
49
|
In a private repository each mutation job carries exactly the labels `[self-hosted, Linux, X64, winbox]` and every other job exactly the hosted-default expression.
|
|
49
50
|
A public repository may keep `runs-on: ubuntu-latest` on every job, because a pull request from a fork runs its own code on the runner.
|
|
@@ -106,7 +107,7 @@ A mutation job that names `CI_RUNS_ON` in `runs-on` produces a report like this:
|
|
|
106
107
|
|
|
107
108
|
```
|
|
108
109
|
ci-wiring: 1 job(s) run on the wrong runner:
|
|
109
|
-
.github/workflows/mutation-compare.yml job mutation-compare: a mutation job reads CI_RUNS_ON, so an override moves its
|
|
110
|
+
.github/workflows/mutation-compare.yml job mutation-compare: a mutation job reads CI_RUNS_ON, so an override moves its mutation runs off winbox; set runs-on: [self-hosted, Linux, X64, winbox]
|
|
110
111
|
```
|
|
111
112
|
|
|
112
113
|
A reference to a missing workflow produces a report like this:
|
|
@@ -10,6 +10,8 @@ audience: consumers
|
|
|
10
10
|
|
|
11
11
|
It fails when an added line carries a banned comment.
|
|
12
12
|
It refuses a machine-read directive, a record or ticket pointer, a doc block, and a file opening with a rationale block over three lines, licence headers excepted.
|
|
13
|
+
The directives are `@ts-expect-error`, `@ts-ignore`, `biome-ignore`, `prettier-ignore`, and every directive that begins `eslint-disable` or `stylelint-disable`, such as `stylelint-disable-next-line`.
|
|
14
|
+
A stylesheet keeps its count of existing stylelint problems in `stylelint-suppressions.json` instead, as [checks-frontend-syntax](checks-frontend-syntax.md) says.
|
|
13
15
|
Each refusal says what to write instead.
|
|
14
16
|
Only added lines are checked, so a violation in a file the diff never touches stays silent.
|
|
15
17
|
A refusal counts when any line of the comment carrying it was added.
|
|
@@ -0,0 +1,113 @@
|
|
|
1
|
+
---
|
|
2
|
+
kind: reference
|
|
3
|
+
audience: consumers
|
|
4
|
+
---
|
|
5
|
+
# checks-copy
|
|
6
|
+
|
|
7
|
+
`checks-copy` is the opt-in gate that refuses an em dash, an en dash or a hyphen used as a dash in copy a person reads.
|
|
8
|
+
|
|
9
|
+
## What it checks
|
|
10
|
+
|
|
11
|
+
It judges every line of each Markdown and HTML file a repository declares as its copy, in any language.
|
|
12
|
+
It holds them to the three separator rules of [The prose rules](checks-docs.md#the-prose-rules):
|
|
13
|
+
|
|
14
|
+
| Refused | For example | Passes |
|
|
15
|
+
| --- | --- | --- |
|
|
16
|
+
| an em dash | `hexn.io — платформа` | `hexn.io, платформа` |
|
|
17
|
+
| an en dash | `pages 1–5` | `pages 1 to 5` |
|
|
18
|
+
| a hyphen used as a dash | `hexn.io - платформа` or `a -- b` | `well-known`, `kebab-case`, `1-5` and `--flag` |
|
|
19
|
+
|
|
20
|
+
A hyphen used as a dash is a run of one to three hyphens with white space on both sides and text on both sides.
|
|
21
|
+
So a hyphen inside a word or a number, a flag and a list marker all pass.
|
|
22
|
+
It reads text as a reader sees it, so a line break, a non-breaking space or a run of spaces counts as one space.
|
|
23
|
+
A hyphen that ends one line of a paragraph and is followed by a word on the next still reads as spaced.
|
|
24
|
+
Copy in a table passes a hyphen standing alone in a cell.
|
|
25
|
+
|
|
26
|
+
In a Markdown file it reads what [checks-docs](checks-docs.md) reads, with the same scanner.
|
|
27
|
+
It skips fenced code, inline code, link destinations, URLs, HTML comments and front matter, and keeps the text of a link.
|
|
28
|
+
Each paragraph and each list item is read whole across its lines, while a heading and a table row are each read on their own.
|
|
29
|
+
|
|
30
|
+
In an HTML file it reads the page through html-validate, with one rule of the kit's own named `copy/separators`.
|
|
31
|
+
It reads the text of the page, with `<title>` included, and each `alt`, `title`, `placeholder`, `aria-label` and `aria-description` attribute.
|
|
32
|
+
It reads the `value` of an `<input>` whose `type` is `submit`, `button` or `reset`, and the `label` of an `<option>` or an `<optgroup>`, since each shows as a control's label.
|
|
33
|
+
It does not read the `value` of any other `<input>`, which holds what a visitor types.
|
|
34
|
+
It reads the `content` of a `<meta>` whose `name` or `property` is `description`, `og:title`, `og:description`, `twitter:title` or `twitter:description`.
|
|
35
|
+
It decodes a numeric character reference and `—`, `–` and ` `, so `a — b` fails as an em dash.
|
|
36
|
+
Text runs across an inline element such as `<a>` or `<strong>`, as `<a href="https://example.com">Example</a> - about` shows, and breaks at a block such as `<p>` or `<li>` and at `<br>`.
|
|
37
|
+
Each of `<code>`, `<kbd>`, `<samp>` and `<var>` stands in as one character the rules never refuse, so a hyphen used as a dash after it still fails.
|
|
38
|
+
It does not read `<script>`, `<style>`, `<template>` or `<pre>`, and reads the text of `<textarea>`, `<noscript>`, `<svg>` and `<math>` like any other.
|
|
39
|
+
It reports a file that does not parse as HTML, since it cannot judge one.
|
|
40
|
+
It fails on each declared input that matches no Markdown or HTML file.
|
|
41
|
+
So an input that a renamed directory or a skipped build leaves empty never passes.
|
|
42
|
+
It also fails on each file an input matches that is neither Markdown nor HTML.
|
|
43
|
+
|
|
44
|
+
These are outside what it decides:
|
|
45
|
+
|
|
46
|
+
- A string in TypeScript source, since no program tells copy from code in a string.
|
|
47
|
+
A site's UI strings reach its built pages, so a glob over the built HTML judges them.
|
|
48
|
+
- What CSS hides or a script changes, since it reads the HTML source rather than a rendered page.
|
|
49
|
+
- A dash in front matter, such as a page title, which a glob over the built HTML judges on the page.
|
|
50
|
+
- Parentheses, semicolons and the other prose rules, which judge only the docs [checks-docs](checks-docs.md) holds.
|
|
51
|
+
|
|
52
|
+
## What it reads
|
|
53
|
+
|
|
54
|
+
It reads `copy.json` in the directory it runs in, which lists the inputs as globs:
|
|
55
|
+
|
|
56
|
+
```json
|
|
57
|
+
{
|
|
58
|
+
"inputs": ["src/content/**/*.md", "dist/**/*.html"]
|
|
59
|
+
}
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
A glob over built output, such as `dist/**/*.html`, judges every string a site renders, and needs the build to run first.
|
|
63
|
+
Each glob resolves from that directory, matches only files and skips `node_modules/`.
|
|
64
|
+
It reads each file ending in `.md` or `.html` that a glob matches, and a file two globs match is judged once.
|
|
65
|
+
It reads the working tree, so an uncommitted file is judged like a committed one.
|
|
66
|
+
|
|
67
|
+
A repository that declares an HTML file installs the optional peer the gate loads for it, at the version the kit pins:
|
|
68
|
+
|
|
69
|
+
```sh
|
|
70
|
+
bun add -d html-validate@11.16.0
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
## Arguments
|
|
74
|
+
|
|
75
|
+
```sh
|
|
76
|
+
checks-copy [<directory>]
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
It checks the directory it runs in, or the directory it is given.
|
|
80
|
+
|
|
81
|
+
## Exit codes
|
|
82
|
+
|
|
83
|
+
| Code | When |
|
|
84
|
+
| --- | --- |
|
|
85
|
+
| 0 | every declared input matches a Markdown or HTML file, and no file carries a refused separator |
|
|
86
|
+
| 1 | a file carries a refused separator or does not parse, a declared input matches no Markdown or HTML file, or an input matches a file of another kind |
|
|
87
|
+
| 2 | `copy.json` is missing or does not decode, its input list is empty, or html-validate cannot load |
|
|
88
|
+
|
|
89
|
+
## Sample output
|
|
90
|
+
|
|
91
|
+
```
|
|
92
|
+
copy: 3 problem(s) in 2 file(s) from 3 declared input(s):
|
|
93
|
+
public/**/*.html: the declared input matches no Markdown or HTML file
|
|
94
|
+
dist/index.html:102:57 carries `-`, a hyphen used as a dash. End the sentence, or use a comma
|
|
95
|
+
src/content/home.ru.md:26:33 carries `-`, a hyphen used as a dash. End the sentence, or use a comma
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
A passing run counts what it judged:
|
|
99
|
+
|
|
100
|
+
```
|
|
101
|
+
copy: no violation in 2 file(s) from 2 declared input(s)
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
## When it runs
|
|
105
|
+
|
|
106
|
+
`checks-lint` runs it when the repository tracks `copy.json` at its root.
|
|
107
|
+
A repository without that file never runs it and needs no html-validate.
|
|
108
|
+
|
|
109
|
+
## Related topics
|
|
110
|
+
|
|
111
|
+
- [checks-docs](checks-docs.md)
|
|
112
|
+
- [checks-browser](checks-browser.md)
|
|
113
|
+
- [checks-lint](checks-lint.md)
|
|
@@ -41,9 +41,10 @@ docs-only=true
|
|
|
41
41
|
|
|
42
42
|
## When it runs
|
|
43
43
|
|
|
44
|
-
|
|
44
|
+
Each of the `static`, `unit` and `e2e` jobs of the `ci` workflow appends the verdict to `GITHUB_OUTPUT` in a filter step.
|
|
45
|
+
Their typecheck and test steps skip when the verdict reads `true`.
|
|
45
46
|
The build, its `git diff --exit-code` and `bun run lint` still run, so `checks-docs` and the generated blocks hold on a docs-only change.
|
|
46
|
-
|
|
47
|
+
Every job still runs and reports, so every required check stays green on a docs-only change.
|
|
47
48
|
|
|
48
49
|
## Related topics
|
|
49
50
|
|
|
@@ -113,6 +113,8 @@ Readability scores and word choice, such as easy, are not checked.
|
|
|
113
113
|
A sentence over the trial length cap is listed as advisory and never fails the run.
|
|
114
114
|
The cap is 20 words in an ordered list item and 25 words elsewhere.
|
|
115
115
|
|
|
116
|
+
[checks-copy](checks-copy.md) holds every line of the copy a repository declares to the three separator rules.
|
|
117
|
+
|
|
116
118
|
`src/docs/prose-matchers.ts` holds the rules and a synchronous `proseRefused()`, and imports nothing.
|
|
117
119
|
The package exports it as `@avi2dg/checks/scripts/prose-matchers.ts`.
|
|
118
120
|
A host such as a write-time hook can copy that one file into a directory with no `node_modules`, and refuse the same lines.
|
|
@@ -225,4 +227,5 @@ A name the range removes fails wherever a doc still carries it, because the remo
|
|
|
225
227
|
## Related topics
|
|
226
228
|
|
|
227
229
|
- [Why it is shaped this way](../design.md)
|
|
230
|
+
- [checks-copy](checks-copy.md)
|
|
228
231
|
- [checks-lint](checks-lint.md)
|
|
@@ -13,9 +13,38 @@ A green run proves nothing failed in that run, not that no test is flaky.
|
|
|
13
13
|
`bun test --randomize --seed=<seed>` puts the suite in the same order, so a seed reproduces a failure that hangs on order.
|
|
14
14
|
A run that fails with no failing test, such as a test file that throws while loading, is listed with its seed on its own line.
|
|
15
15
|
|
|
16
|
+
## Parallel hunts
|
|
17
|
+
|
|
18
|
+
A repository whose `test` script passes `--parallel=<workers>` to [checks-test](checks-test.md#parallel-runs) is hunted with the same worker count, so each run is `bun test --randomize --seed=<seed> --parallel=<workers>`, the mode CI runs in.
|
|
19
|
+
A `test` script without the option keeps the serial hunt.
|
|
20
|
+
|
|
21
|
+
Under workers a seed fixes the order Bun hands out the files.
|
|
22
|
+
Which worker takes each file is a race between the workers.
|
|
23
|
+
Rerunning a failing seed with the same worker count therefore reproduces the failure only some of the time.
|
|
24
|
+
A failure that hangs on which files share a worker comes back every time from that worker's files.
|
|
25
|
+
They run again in one process, in the order the worker ran them.
|
|
26
|
+
|
|
27
|
+
So `checks-flake` passes Bun a preload that records which worker ran each file, after the repository's own `[test]` preload in `bunfig.toml`.
|
|
28
|
+
For each failing worker in a failed run, the summary prints a replay command.
|
|
29
|
+
It runs the files that worker ran, up to the last failing one, with the run's seed:
|
|
30
|
+
|
|
31
|
+
```sh
|
|
32
|
+
bun test --isolate --randomize --seed=2170533150 ./tests/unit/cache.test.ts ./tests/unit/session.test.ts
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
`--isolate` gives each file a fresh global in one process, as a worker does.
|
|
36
|
+
The seed orders the tests inside each file as the worker ran them, whichever other files run with it.
|
|
37
|
+
Bun also shuffles the files it lists, by a permutation of their positions that the seed and their count fix.
|
|
38
|
+
So `checks-flake` learns that permutation from a run of placeholder files and lists the files so the seed runs them in the worker's order.
|
|
39
|
+
On more than one worker the summary also says that rerunning a seed with the worker count reproduces only some of the time.
|
|
40
|
+
A run whose worker crashed gets a replay of that worker's files up to the crashed one.
|
|
41
|
+
|
|
16
42
|
## What it reads
|
|
17
43
|
|
|
18
44
|
It reads bun's JUnit report of each run.
|
|
45
|
+
It reads the worker count from the `test` script in `package.json`, as [checks-test-layout](checks-test-layout.md) validates that script, and hunts serially when the file is missing.
|
|
46
|
+
Under workers it reads the record its preload writes of which worker ran each file.
|
|
47
|
+
For each replay it reads the order Bun ran its placeholder files.
|
|
19
48
|
Under GitHub Actions it appends its summary to the file `GITHUB_STEP_SUMMARY` names, which is the job summary.
|
|
20
49
|
|
|
21
50
|
## Arguments
|
|
@@ -27,6 +56,7 @@ checks-flake [--runs <count> | --seed <seed>...] [--report <file>]
|
|
|
27
56
|
`--runs` sets the number of runs on random seeds, 10 when absent.
|
|
28
57
|
`--seed`, given once per run, replays chosen seeds, such as the ones a report recorded.
|
|
29
58
|
`--report` writes the record as JSON, holding every run's seed and failing tests and every failing test's seeds.
|
|
59
|
+
Under workers the record also holds `parallel`, the worker count and each replay's seed, files in the order the worker ran them, and command.
|
|
30
60
|
|
|
31
61
|
## Exit codes
|
|
32
62
|
|
|
@@ -34,7 +64,7 @@ checks-flake [--runs <count> | --seed <seed>...] [--report <file>]
|
|
|
34
64
|
| --- | --- |
|
|
35
65
|
| 0 | every run passed |
|
|
36
66
|
| 1 | a run failed |
|
|
37
|
-
| 2 | an argument does not parse, or bun passed without writing its report |
|
|
67
|
+
| 2 | an argument does not parse, `package.json` or the worker record does not decode, bun passed without writing its report, or a replay's placeholder files did not each run once |
|
|
38
68
|
|
|
39
69
|
## Sample output
|
|
40
70
|
|
|
@@ -48,6 +78,23 @@ checks-flake: 3 of 10 run(s) failed, 1 test(s) failing in them
|
|
|
48
78
|
Reproduce a failing run with bun test --randomize --seed=<seed>.
|
|
49
79
|
```
|
|
50
80
|
|
|
81
|
+
A hunt on workers prints a replay for each failed run:
|
|
82
|
+
|
|
83
|
+
```
|
|
84
|
+
checks-flake: 2 of 10 run(s) failed on 4 worker(s), 1 test(s) failing in them
|
|
85
|
+
|
|
86
|
+
| Test | Failed | Seeds |
|
|
87
|
+
| --- | --- | --- |
|
|
88
|
+
| tests/unit/cache.test.ts:6 reads the cache | 2 of 10 runs | 2170533150, 180394251 |
|
|
89
|
+
|
|
90
|
+
Reproduce a failing run with the files its failing worker ran, listed so the seed runs them in that order:
|
|
91
|
+
|
|
92
|
+
- seed 2170533150: `bun test --isolate --randomize --seed=2170533150 ./tests/unit/cache.test.ts ./tests/unit/session.test.ts`
|
|
93
|
+
- seed 180394251: `bun test --isolate --randomize --seed=180394251 ./tests/unit/session.test.ts ./tests/unit/cache.test.ts ./tests/unit/pricing.test.ts`
|
|
94
|
+
|
|
95
|
+
Rerunning a seed with bun test --randomize --seed=<seed> --parallel=4 reproduces a failing run only some of the time, because which worker takes each file is a race.
|
|
96
|
+
```
|
|
97
|
+
|
|
51
98
|
## When it runs
|
|
52
99
|
|
|
53
100
|
Only a schedule the repository writes runs it.
|
|
@@ -88,4 +135,5 @@ A private repository sets `runs-on: ${{ vars.CI_RUNS_ON || 'ubuntu-latest' }}` o
|
|
|
88
135
|
## Related topics
|
|
89
136
|
|
|
90
137
|
- [checks-test](checks-test.md)
|
|
138
|
+
- [checks-test-layout](checks-test-layout.md)
|
|
91
139
|
- [checks-ci-wiring](checks-ci-wiring.md)
|