@avi2dg/checks 0.42.0 → 0.44.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +20 -0
- package/README.md +5 -2
- package/{commitlint.config.js → commitlint.config.ts} +1 -1
- package/dist/mutation/mutation-guard-plugin.js +40 -0
- package/dist/mutation/mutation-refusal.js +23 -0
- package/docs/configs/commit-messages.md +2 -2
- package/docs/design.md +20 -0
- package/docs/gates/checks-ci-wiring.md +3 -2
- package/docs/gates/checks-copy.md +113 -0
- package/docs/gates/checks-docs-only.md +3 -2
- package/docs/gates/checks-docs.md +3 -0
- package/docs/gates/checks-flake.md +49 -1
- package/docs/gates/checks-lint.md +2 -1
- package/docs/gates/checks-mutation-baseline.md +4 -2
- package/docs/gates/checks-mutation-compare.md +33 -8
- package/docs/gates/checks-mutation-scope.md +140 -0
- package/docs/gates/checks-mutation.md +11 -7
- package/docs/gates/checks-no-plain-js.md +60 -0
- package/docs/gates/checks-test-layout.md +3 -2
- package/docs/gates/checks-test.md +35 -6
- package/package.json +18 -6
- package/src/core/declared-inputs.ts +76 -0
- package/src/core/gates.ts +4 -0
- package/src/core/git.ts +9 -0
- package/src/delivery/ci-wiring.ts +25 -11
- package/src/delivery/commit-title.ts +2 -2
- package/src/docs/copy-html.ts +110 -0
- package/src/docs/copy-text.ts +70 -0
- package/src/docs/copy.ts +70 -0
- package/src/docs/prose-matchers.ts +14 -14
- package/src/quality/frontend-syntax.ts +9 -52
- package/src/quality/no-plain-js.ts +71 -0
- package/src/testing/flake-preload.ts +4 -0
- package/src/testing/flake.ts +142 -16
- package/src/testing/{mutation-scope.js → mutation-refusal.ts} +11 -9
- package/src/testing/mutation-scope.ts +354 -0
- package/src/testing/mutation.ts +1 -1
- package/src/testing/test-layout.ts +8 -15
- package/src/testing/test-report.ts +20 -5
- package/src/testing/test-skips.ts +1 -1
- package/src/testing/test.ts +53 -11
- package/stryker.preset.js +3 -3
- package/src/testing/mutation-guard-plugin.js +0 -13
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,26 @@
|
|
|
2
2
|
|
|
3
3
|
Every release of `@avi2dg/checks`, newest first, written by the release from its conventional commits.
|
|
4
4
|
|
|
5
|
+
## 0.44.0
|
|
6
|
+
|
|
7
|
+
Released 2026-10-18.
|
|
8
|
+
|
|
9
|
+
### Features
|
|
10
|
+
|
|
11
|
+
- split the test suite into parallel unit and e2e CI tiers [#162](https://github.com/avi2d/checks/pull/162)
|
|
12
|
+
- **testing:** hunt flakes with parallel test workers and replay failing workers [#164](https://github.com/avi2d/checks/pull/164)
|
|
13
|
+
- **mutation:** add checks-mutation-scope to limit PR mutation runs to the change [#161](https://github.com/avi2d/checks/pull/161)
|
|
14
|
+
- **quality:** add checks-no-plain-js gate refusing plain JavaScript sources [#163](https://github.com/avi2d/checks/pull/163)
|
|
15
|
+
- **docs:** add checks-copy gate to refuse dashes in declared copy [#160](https://github.com/avi2d/checks/pull/160)
|
|
16
|
+
|
|
17
|
+
## 0.43.0
|
|
18
|
+
|
|
19
|
+
Released 2026-10-17.
|
|
20
|
+
|
|
21
|
+
### Features
|
|
22
|
+
|
|
23
|
+
- **testing:** run checks-test files in parallel on opt-in [#158](https://github.com/avi2d/checks/pull/158)
|
|
24
|
+
|
|
5
25
|
## 0.42.0
|
|
6
26
|
|
|
7
27
|
Released 2026-10-14.
|
package/README.md
CHANGED
|
@@ -170,9 +170,11 @@ The table groups the gates by vector, the part of a repository each one judges.
|
|
|
170
170
|
| quality | [`checks-comment-gate`](docs/gates/checks-comment-gate.md) | the range | every repository |
|
|
171
171
|
| quality | [`checks-effect-scope`](docs/gates/checks-effect-scope.md) | the working tree | a repository tracking `oxlint.config.ts` |
|
|
172
172
|
| quality | [`checks-frontend-syntax`](docs/gates/checks-frontend-syntax.md) | the working tree | a repository tracking `frontend-syntax.json` |
|
|
173
|
+
| quality | [`checks-no-plain-js`](docs/gates/checks-no-plain-js.md) | the working tree | a repository tracking `*.ts` or `*.tsx` |
|
|
173
174
|
| testing | [`checks-test-layout`](docs/gates/checks-test-layout.md) | the working tree | a repository tracking `*.ts` or `*.tsx` |
|
|
174
175
|
| testing | [`checks-quarantine-clock`](docs/gates/checks-quarantine-clock.md) | the range | every repository |
|
|
175
176
|
| docs | [`checks-docs`](docs/gates/checks-docs.md) | the range, and every agent file at the head commit | every repository |
|
|
177
|
+
| docs | [`checks-copy`](docs/gates/checks-copy.md) | the working tree | a repository tracking `copy.json` |
|
|
176
178
|
| delivery | [`checks-commit-identity`](docs/gates/checks-commit-identity.md) | the range | every repository |
|
|
177
179
|
| delivery | [`checks-ci-wiring`](docs/gates/checks-ci-wiring.md) | the working tree | every repository |
|
|
178
180
|
| delivery | [`checks-secrets`](docs/gates/checks-secrets.md) | the range | every repository |
|
|
@@ -188,6 +190,7 @@ These bins run on their own:
|
|
|
188
190
|
- [`checks-mutation`](docs/gates/checks-mutation.md) runs Stryker for scoped checks and refuses a full run outside CI.
|
|
189
191
|
- [`checks-mutation-compare`](docs/gates/checks-mutation-compare.md) compares the mutants of two reports and lists each one that regresses.
|
|
190
192
|
- [`checks-mutation-baseline`](docs/gates/checks-mutation-baseline.md) restores the newest mutation baseline from `main`, saves it on a self-hosted runner's disk in place of an artifact, and deletes superseded baselines after a `main` run.
|
|
193
|
+
- [`checks-mutation-scope`](docs/gates/checks-mutation-scope.md) turns a pull request's diff into the files its mutation runs mutate.
|
|
191
194
|
- [`checks-subsumed-tests`](docs/gates/checks-subsumed-tests.md) lists each test another test subsumes in a mutation run.
|
|
192
195
|
- [`checks-changelog`](docs/gates/checks-changelog.md) writes the pending release into `CHANGELOG.md` from the conventional commits since the last release.
|
|
193
196
|
- [`checks-release-notes`](docs/gates/checks-release-notes.md) writes one `CHANGELOG.md` section to a file for a GitHub release.
|
|
@@ -207,7 +210,7 @@ oxlint runs through its own tool, as the pages under Related topics say.
|
|
|
207
210
|
To move a repository to a newer release of the kit:
|
|
208
211
|
|
|
209
212
|
1. Run the install line again, which moves the kit to its newest release and the peers to the versions it pins.
|
|
210
|
-
A repository that opted in to `checks-browser
|
|
213
|
+
A repository that opted in to `checks-browser`, `checks-frontend-syntax` or `checks-copy` also reruns the `bun add` line on its page.
|
|
211
214
|
1. Run `bun run build`, whose `checks-effect-scope` rewrites the Effect override and the kit's severities in `tsconfig.json` when a release changes them.
|
|
212
215
|
A severity key the release drops stays in `tsconfig.json` until the repository removes it.
|
|
213
216
|
1. Copy `node_modules/@avi2dg/checks/bunfig.toml` over `bunfig.toml` again, since `checks-test-layout` compares the copy with the installed preset.
|
|
@@ -227,7 +230,7 @@ Every path is relative to the installed package, `node_modules/@avi2dg/checks/`.
|
|
|
227
230
|
| `CHANGELOG.md` | every release, and what it changed |
|
|
228
231
|
| `docs/` | a reference page per bin and per shared config, and why the kit is shaped this way |
|
|
229
232
|
| `bunfig.toml` | the bunfig preset a repository copies |
|
|
230
|
-
| `commitlint.config.
|
|
233
|
+
| `commitlint.config.ts` | the shared commitlint config |
|
|
231
234
|
| `dependency-cruiser.config.js` | the kit's dependency-cruiser rules as a base a `.dependency-cruiser.cjs` extends by path, which the build writes |
|
|
232
235
|
| `knip-base.json` | the Knip `include` setting, for a configuration that spreads it |
|
|
233
236
|
| `src/` | every bin, which a package script calls by its `checks-` name, the modules the bins import, and the Effect language service severities under `src/quality/presets/` |
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
// src/testing/mutation-guard-plugin.ts
|
|
2
|
+
import { existsSync } from "node:fs";
|
|
3
|
+
|
|
4
|
+
// src/testing/mutation-refusal.ts
|
|
5
|
+
var NON_RUNS = ["--help", "-h", "--version"];
|
|
6
|
+
var MUTATE_FLAGS = ["--mutate", "-m"];
|
|
7
|
+
function namesGlob(args) {
|
|
8
|
+
return args.some((arg, index) => MUTATE_FLAGS.includes(arg) && (args[index + 1] ?? "") !== "" || arg.startsWith("--mutate=") && arg !== "--mutate=");
|
|
9
|
+
}
|
|
10
|
+
function allowedAnyway(args, ci) {
|
|
11
|
+
return ci === "true" || args.some((arg) => NON_RUNS.includes(arg)) || namesGlob(args);
|
|
12
|
+
}
|
|
13
|
+
function fullRunRefusal(args, ci) {
|
|
14
|
+
if (allowedAnyway(args, ci) || args.includes("--incremental") && !args.includes("--force"))
|
|
15
|
+
return;
|
|
16
|
+
return "refusing a full mutation run outside CI; start the same run in CI with `gh workflow run mutation`, or scope this run with `--mutate <glob>` or `--incremental`";
|
|
17
|
+
}
|
|
18
|
+
function missingReportRefusal(args, ci, incrementalFile, reportExists) {
|
|
19
|
+
if (allowedAnyway(args, ci) || reportExists(incrementalFile))
|
|
20
|
+
return;
|
|
21
|
+
return `refusing a full mutation run outside CI: \`--incremental\` finds no report at ${incrementalFile} to reuse; scope this run with \`--mutate <glob>\`, or start the full baseline in CI with \`gh workflow run mutation\``;
|
|
22
|
+
}
|
|
23
|
+
|
|
24
|
+
// src/testing/mutation-guard-plugin.ts
|
|
25
|
+
var GUARD_IGNORER = "checks-incremental-report-guard";
|
|
26
|
+
function incrementalReportGuard(options) {
|
|
27
|
+
const refusal = missingReportRefusal(process.argv.slice(3), process.env.CI ?? "", options.incrementalFile, existsSync);
|
|
28
|
+
if (refusal !== undefined)
|
|
29
|
+
throw new Error(refusal);
|
|
30
|
+
return { shouldIgnore: () => {
|
|
31
|
+
return;
|
|
32
|
+
} };
|
|
33
|
+
}
|
|
34
|
+
var strykerPlugins = [
|
|
35
|
+
{ kind: "Ignore", name: GUARD_IGNORER, factory: Object.assign(incrementalReportGuard, { inject: ["options"] }) }
|
|
36
|
+
];
|
|
37
|
+
export {
|
|
38
|
+
GUARD_IGNORER,
|
|
39
|
+
strykerPlugins
|
|
40
|
+
};
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
// src/testing/mutation-refusal.ts
|
|
2
|
+
var NON_RUNS = ["--help", "-h", "--version"];
|
|
3
|
+
var MUTATE_FLAGS = ["--mutate", "-m"];
|
|
4
|
+
function namesGlob(args) {
|
|
5
|
+
return args.some((arg, index) => MUTATE_FLAGS.includes(arg) && (args[index + 1] ?? "") !== "" || arg.startsWith("--mutate=") && arg !== "--mutate=");
|
|
6
|
+
}
|
|
7
|
+
function allowedAnyway(args, ci) {
|
|
8
|
+
return ci === "true" || args.some((arg) => NON_RUNS.includes(arg)) || namesGlob(args);
|
|
9
|
+
}
|
|
10
|
+
function fullRunRefusal(args, ci) {
|
|
11
|
+
if (allowedAnyway(args, ci) || args.includes("--incremental") && !args.includes("--force"))
|
|
12
|
+
return;
|
|
13
|
+
return "refusing a full mutation run outside CI; start the same run in CI with `gh workflow run mutation`, or scope this run with `--mutate <glob>` or `--incremental`";
|
|
14
|
+
}
|
|
15
|
+
function missingReportRefusal(args, ci, incrementalFile, reportExists) {
|
|
16
|
+
if (allowedAnyway(args, ci) || reportExists(incrementalFile))
|
|
17
|
+
return;
|
|
18
|
+
return `refusing a full mutation run outside CI: \`--incremental\` finds no report at ${incrementalFile} to reuse; scope this run with \`--mutate <glob>\`, or start the full baseline in CI with \`gh workflow run mutation\``;
|
|
19
|
+
}
|
|
20
|
+
export {
|
|
21
|
+
fullRunRefusal,
|
|
22
|
+
missingReportRefusal
|
|
23
|
+
};
|
|
@@ -8,14 +8,14 @@ The shared commitlint config holds each pull request title to the conventional c
|
|
|
8
8
|
|
|
9
9
|
## Config
|
|
10
10
|
|
|
11
|
-
Commits follow `@commitlint/config-conventional` plus the house prefixes `commitlint.config.
|
|
11
|
+
Commits follow `@commitlint/config-conventional` plus the house prefixes `commitlint.config.ts` lists, shared from `node_modules/@avi2dg/checks/commitlint.config.ts`.
|
|
12
12
|
It arrives with the kit, since `@commitlint/cli` and `@commitlint/config-conventional` are dependencies, not peers.
|
|
13
13
|
|
|
14
14
|
## Workflow
|
|
15
15
|
|
|
16
16
|
The lint runs in CI on pull requests, because `jj` never fires a git hook.
|
|
17
17
|
Each repository lints the title in one step of its `ci` job with `bunx checks-commit-title`.
|
|
18
|
-
The workflow lints with the installed kit's `commitlint.config.
|
|
18
|
+
The workflow lints with the installed kit's `commitlint.config.ts`, so every repository holds titles to the same rules.
|
|
19
19
|
The `ci` job reports the title step's outcome as a `commitlint` status on every run, so a required `commitlint` status stays while `commitlint.yml` goes away.
|
|
20
20
|
That post needs the workflow's `statuses: write` permission.
|
|
21
21
|
The title lint runs before the docs-only filter, so a change to top-level Markdown and `docs/` Markdown alone still lints its title.
|
package/docs/design.md
CHANGED
|
@@ -98,6 +98,7 @@ npm adds `package.json`, `README.md` and `LICENSE` whatever `files` says.
|
|
|
98
98
|
`bun pm pack` builds the same tarball the registry serves, and the consumer e2e test installs that tarball.
|
|
99
99
|
|
|
100
100
|
Each oxlint plugin and each config builder ships compiled under `dist/`, because Node refuses to strip types from a `.ts` file under `node_modules`.
|
|
101
|
+
Stryker loads its config under Node too, so the guard plugin and run-scope helpers the Stryker preset imports ship compiled under `dist/mutation/`.
|
|
101
102
|
oxlint loads `oxlint.config.ts` under Node, so the builders ship as JavaScript beside a `.d.ts` that `tsc` reads.
|
|
102
103
|
`@oxlint/plugins` ships no RuleTester, so each `effect-channel`, `readability` and `data-shape` rule is proven red and green against an installed consumer in `tests/e2e/consumer.test.ts`, except `readability/thin-astro`, which `tests/e2e/astro-consumer.test.ts` proves against a consumer tree linked to the checkout.
|
|
103
104
|
`dist/` is committed, with the doc templates in `dist/templates/`, and so is `CHANGELOG.md`, which the same build writes.
|
|
@@ -315,6 +316,25 @@ So the runner calls the product's hooks at each visit and holds them to the same
|
|
|
315
316
|
The runner drives the Chrome a machine already has through `playwright-core`, since Playwright's own browser builds are a separate download of their own.
|
|
316
317
|
GitHub's hosted Ubuntu runners carry Chrome, and `CHROME_PATH` names any other build.
|
|
317
318
|
|
|
319
|
+
## Copy is judged in the files a site renders
|
|
320
|
+
|
|
321
|
+
`checks-copy` takes the three separator rules from `src/docs/prose-matchers.ts`, so a doc and a page refuse the same characters in the same words.
|
|
322
|
+
Its Markdown goes through the scanner `checks-docs` uses, so the two gates never disagree about what is code.
|
|
323
|
+
Its HTML goes through html-validate, which the `nesting` check already runs, with one rule of the kit's own.
|
|
324
|
+
No rule html-validate or markdownlint ships refuses a spaced hyphen between words.
|
|
325
|
+
textlint and Vale could each carry the rule, but each adds a package or a binary.
|
|
326
|
+
Each would also hold a second copy of patterns the kit already holds.
|
|
327
|
+
|
|
328
|
+
The gate reads files rather than strings in source.
|
|
329
|
+
An oxlint rule over string literals and JSX text was run across the repositories that consume the kit.
|
|
330
|
+
It found mostly code inside a string, test fixtures and error messages.
|
|
331
|
+
The few hits that were copy, such as a line a CLI prints, look to a program like `scrollHeight - innerHeight` in a test script.
|
|
332
|
+
A site's strings all reach its built pages, so a glob over the built HTML judges each one where a visitor reads it.
|
|
333
|
+
|
|
334
|
+
Only a repository knows which of its files are copy, so it opts in by declaring them in `copy.json`.
|
|
335
|
+
The gate judges every line of each declared file, unlike the prose rules in `checks-docs`, which judge only the lines a change edits.
|
|
336
|
+
A visitor reads the whole page, so a dash on a line the change left alone is as wrong as one it added.
|
|
337
|
+
|
|
318
338
|
## Related topics
|
|
319
339
|
|
|
320
340
|
- [checks](../README.md)
|
|
@@ -11,6 +11,7 @@ audience: consumers
|
|
|
11
11
|
The kit requires `bunx checks-commit-title` on every pull request.
|
|
12
12
|
A step that calls `bun src/delivery/commit-title.ts` or `./node_modules/.bin/commitlint` directly still satisfies the check.
|
|
13
13
|
It requires `bun run lint`, `bun run build`, `bun run typecheck` and `bun run test` for each of those scripts that `package.json` defines.
|
|
14
|
+
When `package.json` defines both `test:unit` and `test:e2e`, a step for each of `bun run test:unit` and `bun run test:e2e` also satisfies `bun run test`.
|
|
14
15
|
A `build` script also requires `git diff --exit-code`.
|
|
15
16
|
The target branch is the pull request base in CI, else the branch `refs/remotes/origin/HEAD` names, else the default branch of the repository in GitHub's event, else `main`.
|
|
16
17
|
A `pull_request` trigger without a branch filter covers every target branch.
|
|
@@ -43,7 +44,7 @@ A file counts as a shell script when its name ends in `.sh` or its first line na
|
|
|
43
44
|
The call counts as a command of its own or inside a command substitution, never as an argument to another command.
|
|
44
45
|
A here-document that `sh`, `bash`, `dash`, `ksh` or `zsh` reads is a script, and its calls count.
|
|
45
46
|
The text of any other here-document is data, so a call there counts only inside a command substitution when the delimiter is unquoted.
|
|
46
|
-
A mutation job never names `CI_RUNS_ON` directly in `runs-on`, as `vars.CI_RUNS_ON` or `vars['CI_RUNS_ON']`, so an override for an outage cannot send its
|
|
47
|
+
A mutation job never names `CI_RUNS_ON` directly in `runs-on`, as `vars.CI_RUNS_ON` or `vars['CI_RUNS_ON']`, so an override for an outage cannot send its mutation runs to hosted runners.
|
|
47
48
|
Any other job that names `CI_RUNS_ON` in `runs-on` carries exactly the hosted-default expression `${{ vars.CI_RUNS_ON || 'ubuntu-latest' }}`.
|
|
48
49
|
In a private repository each mutation job carries exactly the labels `[self-hosted, Linux, X64, winbox]` and every other job exactly the hosted-default expression.
|
|
49
50
|
A public repository may keep `runs-on: ubuntu-latest` on every job, because a pull request from a fork runs its own code on the runner.
|
|
@@ -106,7 +107,7 @@ A mutation job that names `CI_RUNS_ON` in `runs-on` produces a report like this:
|
|
|
106
107
|
|
|
107
108
|
```
|
|
108
109
|
ci-wiring: 1 job(s) run on the wrong runner:
|
|
109
|
-
.github/workflows/mutation-compare.yml job mutation-compare: a mutation job reads CI_RUNS_ON, so an override moves its
|
|
110
|
+
.github/workflows/mutation-compare.yml job mutation-compare: a mutation job reads CI_RUNS_ON, so an override moves its mutation runs off winbox; set runs-on: [self-hosted, Linux, X64, winbox]
|
|
110
111
|
```
|
|
111
112
|
|
|
112
113
|
A reference to a missing workflow produces a report like this:
|
|
@@ -0,0 +1,113 @@
|
|
|
1
|
+
---
|
|
2
|
+
kind: reference
|
|
3
|
+
audience: consumers
|
|
4
|
+
---
|
|
5
|
+
# checks-copy
|
|
6
|
+
|
|
7
|
+
`checks-copy` is the opt-in gate that refuses an em dash, an en dash or a hyphen used as a dash in copy a person reads.
|
|
8
|
+
|
|
9
|
+
## What it checks
|
|
10
|
+
|
|
11
|
+
It judges every line of each Markdown and HTML file a repository declares as its copy, in any language.
|
|
12
|
+
It holds them to the three separator rules of [The prose rules](checks-docs.md#the-prose-rules):
|
|
13
|
+
|
|
14
|
+
| Refused | For example | Passes |
|
|
15
|
+
| --- | --- | --- |
|
|
16
|
+
| an em dash | `hexn.io — платформа` | `hexn.io, платформа` |
|
|
17
|
+
| an en dash | `pages 1–5` | `pages 1 to 5` |
|
|
18
|
+
| a hyphen used as a dash | `hexn.io - платформа` or `a -- b` | `well-known`, `kebab-case`, `1-5` and `--flag` |
|
|
19
|
+
|
|
20
|
+
A hyphen used as a dash is a run of one to three hyphens with white space on both sides and text on both sides.
|
|
21
|
+
So a hyphen inside a word or a number, a flag and a list marker all pass.
|
|
22
|
+
It reads text as a reader sees it, so a line break, a non-breaking space or a run of spaces counts as one space.
|
|
23
|
+
A hyphen that ends one line of a paragraph and is followed by a word on the next still reads as spaced.
|
|
24
|
+
Copy in a table passes a hyphen standing alone in a cell.
|
|
25
|
+
|
|
26
|
+
In a Markdown file it reads what [checks-docs](checks-docs.md) reads, with the same scanner.
|
|
27
|
+
It skips fenced code, inline code, link destinations, URLs, HTML comments and front matter, and keeps the text of a link.
|
|
28
|
+
Each paragraph and each list item is read whole across its lines, while a heading and a table row are each read on their own.
|
|
29
|
+
|
|
30
|
+
In an HTML file it reads the page through html-validate, with one rule of the kit's own named `copy/separators`.
|
|
31
|
+
It reads the text of the page, with `<title>` included, and each `alt`, `title`, `placeholder`, `aria-label` and `aria-description` attribute.
|
|
32
|
+
It reads the `value` of an `<input>` whose `type` is `submit`, `button` or `reset`, and the `label` of an `<option>` or an `<optgroup>`, since each shows as a control's label.
|
|
33
|
+
It does not read the `value` of any other `<input>`, which holds what a visitor types.
|
|
34
|
+
It reads the `content` of a `<meta>` whose `name` or `property` is `description`, `og:title`, `og:description`, `twitter:title` or `twitter:description`.
|
|
35
|
+
It decodes a numeric character reference and `—`, `–` and ` `, so `a — b` fails as an em dash.
|
|
36
|
+
Text runs across an inline element such as `<a>` or `<strong>`, as `<a href="https://example.com">Example</a> - about` shows, and breaks at a block such as `<p>` or `<li>` and at `<br>`.
|
|
37
|
+
Each of `<code>`, `<kbd>`, `<samp>` and `<var>` stands in as one character the rules never refuse, so a hyphen used as a dash after it still fails.
|
|
38
|
+
It does not read `<script>`, `<style>`, `<template>` or `<pre>`, and reads the text of `<textarea>`, `<noscript>`, `<svg>` and `<math>` like any other.
|
|
39
|
+
It reports a file that does not parse as HTML, since it cannot judge one.
|
|
40
|
+
It fails on each declared input that matches no Markdown or HTML file.
|
|
41
|
+
So an input that a renamed directory or a skipped build leaves empty never passes.
|
|
42
|
+
It also fails on each file an input matches that is neither Markdown nor HTML.
|
|
43
|
+
|
|
44
|
+
These are outside what it decides:
|
|
45
|
+
|
|
46
|
+
- A string in TypeScript source, since no program tells copy from code in a string.
|
|
47
|
+
A site's UI strings reach its built pages, so a glob over the built HTML judges them.
|
|
48
|
+
- What CSS hides or a script changes, since it reads the HTML source rather than a rendered page.
|
|
49
|
+
- A dash in front matter, such as a page title, which a glob over the built HTML judges on the page.
|
|
50
|
+
- Parentheses, semicolons and the other prose rules, which judge only the docs [checks-docs](checks-docs.md) holds.
|
|
51
|
+
|
|
52
|
+
## What it reads
|
|
53
|
+
|
|
54
|
+
It reads `copy.json` in the directory it runs in, which lists the inputs as globs:
|
|
55
|
+
|
|
56
|
+
```json
|
|
57
|
+
{
|
|
58
|
+
"inputs": ["src/content/**/*.md", "dist/**/*.html"]
|
|
59
|
+
}
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
A glob over built output, such as `dist/**/*.html`, judges every string a site renders, and needs the build to run first.
|
|
63
|
+
Each glob resolves from that directory, matches only files and skips `node_modules/`.
|
|
64
|
+
It reads each file ending in `.md` or `.html` that a glob matches, and a file two globs match is judged once.
|
|
65
|
+
It reads the working tree, so an uncommitted file is judged like a committed one.
|
|
66
|
+
|
|
67
|
+
A repository that declares an HTML file installs the optional peer the gate loads for it, at the version the kit pins:
|
|
68
|
+
|
|
69
|
+
```sh
|
|
70
|
+
bun add -d html-validate@11.16.0
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
## Arguments
|
|
74
|
+
|
|
75
|
+
```sh
|
|
76
|
+
checks-copy [<directory>]
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
It checks the directory it runs in, or the directory it is given.
|
|
80
|
+
|
|
81
|
+
## Exit codes
|
|
82
|
+
|
|
83
|
+
| Code | When |
|
|
84
|
+
| --- | --- |
|
|
85
|
+
| 0 | every declared input matches a Markdown or HTML file, and no file carries a refused separator |
|
|
86
|
+
| 1 | a file carries a refused separator or does not parse, a declared input matches no Markdown or HTML file, or an input matches a file of another kind |
|
|
87
|
+
| 2 | `copy.json` is missing or does not decode, its input list is empty, or html-validate cannot load |
|
|
88
|
+
|
|
89
|
+
## Sample output
|
|
90
|
+
|
|
91
|
+
```
|
|
92
|
+
copy: 3 problem(s) in 2 file(s) from 3 declared input(s):
|
|
93
|
+
public/**/*.html: the declared input matches no Markdown or HTML file
|
|
94
|
+
dist/index.html:102:57 carries `-`, a hyphen used as a dash. End the sentence, or use a comma
|
|
95
|
+
src/content/home.ru.md:26:33 carries `-`, a hyphen used as a dash. End the sentence, or use a comma
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
A passing run counts what it judged:
|
|
99
|
+
|
|
100
|
+
```
|
|
101
|
+
copy: no violation in 2 file(s) from 2 declared input(s)
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
## When it runs
|
|
105
|
+
|
|
106
|
+
`checks-lint` runs it when the repository tracks `copy.json` at its root.
|
|
107
|
+
A repository without that file never runs it and needs no html-validate.
|
|
108
|
+
|
|
109
|
+
## Related topics
|
|
110
|
+
|
|
111
|
+
- [checks-docs](checks-docs.md)
|
|
112
|
+
- [checks-browser](checks-browser.md)
|
|
113
|
+
- [checks-lint](checks-lint.md)
|
|
@@ -41,9 +41,10 @@ docs-only=true
|
|
|
41
41
|
|
|
42
42
|
## When it runs
|
|
43
43
|
|
|
44
|
-
|
|
44
|
+
Each of the `static`, `unit` and `e2e` jobs of the `ci` workflow appends the verdict to `GITHUB_OUTPUT` in a filter step.
|
|
45
|
+
Their typecheck and test steps skip when the verdict reads `true`.
|
|
45
46
|
The build, its `git diff --exit-code` and `bun run lint` still run, so `checks-docs` and the generated blocks hold on a docs-only change.
|
|
46
|
-
|
|
47
|
+
Every job still runs and reports, so every required check stays green on a docs-only change.
|
|
47
48
|
|
|
48
49
|
## Related topics
|
|
49
50
|
|
|
@@ -113,6 +113,8 @@ Readability scores and word choice, such as easy, are not checked.
|
|
|
113
113
|
A sentence over the trial length cap is listed as advisory and never fails the run.
|
|
114
114
|
The cap is 20 words in an ordered list item and 25 words elsewhere.
|
|
115
115
|
|
|
116
|
+
[checks-copy](checks-copy.md) holds every line of the copy a repository declares to the three separator rules.
|
|
117
|
+
|
|
116
118
|
`src/docs/prose-matchers.ts` holds the rules and a synchronous `proseRefused()`, and imports nothing.
|
|
117
119
|
The package exports it as `@avi2dg/checks/scripts/prose-matchers.ts`.
|
|
118
120
|
A host such as a write-time hook can copy that one file into a directory with no `node_modules`, and refuse the same lines.
|
|
@@ -225,4 +227,5 @@ A name the range removes fails wherever a doc still carries it, because the remo
|
|
|
225
227
|
## Related topics
|
|
226
228
|
|
|
227
229
|
- [Why it is shaped this way](../design.md)
|
|
230
|
+
- [checks-copy](checks-copy.md)
|
|
228
231
|
- [checks-lint](checks-lint.md)
|
|
@@ -13,9 +13,38 @@ A green run proves nothing failed in that run, not that no test is flaky.
|
|
|
13
13
|
`bun test --randomize --seed=<seed>` puts the suite in the same order, so a seed reproduces a failure that hangs on order.
|
|
14
14
|
A run that fails with no failing test, such as a test file that throws while loading, is listed with its seed on its own line.
|
|
15
15
|
|
|
16
|
+
## Parallel hunts
|
|
17
|
+
|
|
18
|
+
A repository whose `test` script passes `--parallel=<workers>` to [checks-test](checks-test.md#parallel-runs) is hunted with the same worker count, so each run is `bun test --randomize --seed=<seed> --parallel=<workers>`, the mode CI runs in.
|
|
19
|
+
A `test` script without the option keeps the serial hunt.
|
|
20
|
+
|
|
21
|
+
Under workers a seed fixes the order Bun hands out the files.
|
|
22
|
+
Which worker takes each file is a race between the workers.
|
|
23
|
+
Rerunning a failing seed with the same worker count therefore reproduces the failure only some of the time.
|
|
24
|
+
A failure that hangs on which files share a worker comes back every time from that worker's files.
|
|
25
|
+
They run again in one process, in the order the worker ran them.
|
|
26
|
+
|
|
27
|
+
So `checks-flake` passes Bun a preload that records which worker ran each file, after the repository's own `[test]` preload in `bunfig.toml`.
|
|
28
|
+
For each failing worker in a failed run, the summary prints a replay command.
|
|
29
|
+
It runs the files that worker ran, up to the last failing one, with the run's seed:
|
|
30
|
+
|
|
31
|
+
```sh
|
|
32
|
+
bun test --isolate --randomize --seed=2170533150 ./tests/unit/cache.test.ts ./tests/unit/session.test.ts
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
`--isolate` gives each file a fresh global in one process, as a worker does.
|
|
36
|
+
The seed orders the tests inside each file as the worker ran them, whichever other files run with it.
|
|
37
|
+
Bun also shuffles the files it lists, by a permutation of their positions that the seed and their count fix.
|
|
38
|
+
So `checks-flake` learns that permutation from a run of placeholder files and lists the files so the seed runs them in the worker's order.
|
|
39
|
+
On more than one worker the summary also says that rerunning a seed with the worker count reproduces only some of the time.
|
|
40
|
+
A run whose worker crashed gets a replay of that worker's files up to the crashed one.
|
|
41
|
+
|
|
16
42
|
## What it reads
|
|
17
43
|
|
|
18
44
|
It reads bun's JUnit report of each run.
|
|
45
|
+
It reads the worker count from the `test` script in `package.json`, as [checks-test-layout](checks-test-layout.md) validates that script, and hunts serially when the file is missing.
|
|
46
|
+
Under workers it reads the record its preload writes of which worker ran each file.
|
|
47
|
+
For each replay it reads the order Bun ran its placeholder files.
|
|
19
48
|
Under GitHub Actions it appends its summary to the file `GITHUB_STEP_SUMMARY` names, which is the job summary.
|
|
20
49
|
|
|
21
50
|
## Arguments
|
|
@@ -27,6 +56,7 @@ checks-flake [--runs <count> | --seed <seed>...] [--report <file>]
|
|
|
27
56
|
`--runs` sets the number of runs on random seeds, 10 when absent.
|
|
28
57
|
`--seed`, given once per run, replays chosen seeds, such as the ones a report recorded.
|
|
29
58
|
`--report` writes the record as JSON, holding every run's seed and failing tests and every failing test's seeds.
|
|
59
|
+
Under workers the record also holds `parallel`, the worker count and each replay's seed, files in the order the worker ran them, and command.
|
|
30
60
|
|
|
31
61
|
## Exit codes
|
|
32
62
|
|
|
@@ -34,7 +64,7 @@ checks-flake [--runs <count> | --seed <seed>...] [--report <file>]
|
|
|
34
64
|
| --- | --- |
|
|
35
65
|
| 0 | every run passed |
|
|
36
66
|
| 1 | a run failed |
|
|
37
|
-
| 2 | an argument does not parse, or bun passed without writing its report |
|
|
67
|
+
| 2 | an argument does not parse, `package.json` or the worker record does not decode, bun passed without writing its report, or a replay's placeholder files did not each run once |
|
|
38
68
|
|
|
39
69
|
## Sample output
|
|
40
70
|
|
|
@@ -48,6 +78,23 @@ checks-flake: 3 of 10 run(s) failed, 1 test(s) failing in them
|
|
|
48
78
|
Reproduce a failing run with bun test --randomize --seed=<seed>.
|
|
49
79
|
```
|
|
50
80
|
|
|
81
|
+
A hunt on workers prints a replay for each failed run:
|
|
82
|
+
|
|
83
|
+
```
|
|
84
|
+
checks-flake: 2 of 10 run(s) failed on 4 worker(s), 1 test(s) failing in them
|
|
85
|
+
|
|
86
|
+
| Test | Failed | Seeds |
|
|
87
|
+
| --- | --- | --- |
|
|
88
|
+
| tests/unit/cache.test.ts:6 reads the cache | 2 of 10 runs | 2170533150, 180394251 |
|
|
89
|
+
|
|
90
|
+
Reproduce a failing run with the files its failing worker ran, listed so the seed runs them in that order:
|
|
91
|
+
|
|
92
|
+
- seed 2170533150: `bun test --isolate --randomize --seed=2170533150 ./tests/unit/cache.test.ts ./tests/unit/session.test.ts`
|
|
93
|
+
- seed 180394251: `bun test --isolate --randomize --seed=180394251 ./tests/unit/session.test.ts ./tests/unit/cache.test.ts ./tests/unit/pricing.test.ts`
|
|
94
|
+
|
|
95
|
+
Rerunning a seed with bun test --randomize --seed=<seed> --parallel=4 reproduces a failing run only some of the time, because which worker takes each file is a race.
|
|
96
|
+
```
|
|
97
|
+
|
|
51
98
|
## When it runs
|
|
52
99
|
|
|
53
100
|
Only a schedule the repository writes runs it.
|
|
@@ -88,4 +135,5 @@ A private repository sets `runs-on: ${{ vars.CI_RUNS_ON || 'ubuntu-latest' }}` o
|
|
|
88
135
|
## Related topics
|
|
89
136
|
|
|
90
137
|
- [checks-test](checks-test.md)
|
|
138
|
+
- [checks-test-layout](checks-test-layout.md)
|
|
91
139
|
- [checks-ci-wiring](checks-ci-wiring.md)
|
|
@@ -13,6 +13,7 @@ Gates requiring tracked TypeScript files begin running when the repository track
|
|
|
13
13
|
`checks-lint-coverage` and `checks-unused` also begin running when the repository tracks an `.astro` file.
|
|
14
14
|
`checks-advisories` begins running when the repository tracks `bun.lock`.
|
|
15
15
|
`checks-frontend-syntax` begins running when the repository tracks `frontend-syntax.json` at its root.
|
|
16
|
+
`checks-copy` begins running when the repository tracks `copy.json` at its root.
|
|
16
17
|
All other gates run for every repository.
|
|
17
18
|
|
|
18
19
|
## What it reads
|
|
@@ -47,7 +48,7 @@ With two arguments, the base and head override range discovery.
|
|
|
47
48
|
|
|
48
49
|
```
|
|
49
50
|
checks-lint: range 2504acf098d120e73a8ece3c96f22b934f35c6a8..10ba7d8935b73ed72624120a1542e51bd21ca7c7 from HEAD against origin/main
|
|
50
|
-
checks-lint: 1 of
|
|
51
|
+
checks-lint: 1 of 18 gate(s) failed: checks-comment-gate
|
|
51
52
|
```
|
|
52
53
|
|
|
53
54
|
<!-- end generated lint-sample -->
|
|
@@ -181,13 +181,15 @@ It slims the report first, so the artifact holds about 0.5 MB in place of about
|
|
|
181
181
|
The full upload keeps 30 days, so the newest full baseline outlives the gap between full runs.
|
|
182
182
|
A pull request's scope step loses its coverage when no full baseline outlives the gap.
|
|
183
183
|
|
|
184
|
-
A pull request's scope step restores the full report with `--full
|
|
184
|
+
A pull request's scope step restores the full report with `--full`, and [checks-mutation-scope](checks-mutation-scope.md) reads it:
|
|
185
185
|
|
|
186
186
|
```yaml
|
|
187
187
|
- name: Select mutation scope
|
|
188
188
|
env:
|
|
189
189
|
GH_TOKEN: ${{ github.token }}
|
|
190
|
-
run:
|
|
190
|
+
run: |
|
|
191
|
+
bunx checks-mutation-baseline --full "$RUNNER_TEMP/baseline-full/mutation.json"
|
|
192
|
+
bunx checks-mutation-scope "$(git merge-base HEAD origin/main)" "$RUNNER_TEMP/baseline-full/mutation.json" >> "$GITHUB_ENV"
|
|
191
193
|
```
|
|
192
194
|
|
|
193
195
|
## Related topics
|
|
@@ -78,7 +78,7 @@ The shared preset refuses a full `stryker run` outside CI and names that workflo
|
|
|
78
78
|
|
|
79
79
|
## Running it in CI
|
|
80
80
|
|
|
81
|
-
It runs on pull requests from a workflow named `mutation-compare`, comparing the head report against a report built at the merge-base:
|
|
81
|
+
It runs on pull requests from a workflow named `mutation-compare`, comparing the head report against a report built at the merge-base over the same scope:
|
|
82
82
|
|
|
83
83
|
```yaml
|
|
84
84
|
name: mutation-compare
|
|
@@ -87,6 +87,9 @@ on:
|
|
|
87
87
|
jobs:
|
|
88
88
|
mutation-compare:
|
|
89
89
|
runs-on: ubuntu-latest
|
|
90
|
+
permissions:
|
|
91
|
+
contents: read
|
|
92
|
+
actions: read
|
|
90
93
|
steps:
|
|
91
94
|
- uses: actions/checkout@v5
|
|
92
95
|
with:
|
|
@@ -95,28 +98,50 @@ jobs:
|
|
|
95
98
|
with:
|
|
96
99
|
bun-version-file: .bun-version
|
|
97
100
|
- run: bun install --frozen-lockfile
|
|
98
|
-
-
|
|
99
|
-
|
|
101
|
+
- name: Select mutation scope
|
|
102
|
+
env:
|
|
103
|
+
GH_TOKEN: ${{ github.token }}
|
|
104
|
+
run: |
|
|
105
|
+
base="$(git merge-base HEAD "origin/$GITHUB_BASE_REF")"
|
|
106
|
+
echo "BASE_SHA=$base" >> "$GITHUB_ENV"
|
|
107
|
+
bunx checks-mutation-baseline --full "$RUNNER_TEMP/baseline-full/mutation.json"
|
|
108
|
+
bunx checks-mutation-scope "$base" "$RUNNER_TEMP/baseline-full/mutation.json" >> "$GITHUB_ENV"
|
|
109
|
+
- if: env.SCOPE == ''
|
|
110
|
+
run: echo "::notice::mutation-compare: nothing in mutation scope changed, so neither Stryker run starts"
|
|
111
|
+
- if: env.SCOPE != ''
|
|
112
|
+
run: bunx stryker run --mutate "$SCOPE"
|
|
113
|
+
- if: env.SCOPE != ''
|
|
114
|
+
run: |
|
|
100
115
|
base_worktree="$RUNNER_TEMP/mutation-base-$GITHUB_RUN_ID-$GITHUB_RUN_ATTEMPT"
|
|
101
116
|
echo "BASE_WORKTREE=$base_worktree" >> "$GITHUB_ENV"
|
|
102
117
|
git worktree prune
|
|
103
|
-
git worktree add "$base_worktree" "$
|
|
104
|
-
(cd "$base_worktree" && bun install --frozen-lockfile && bunx stryker run)
|
|
105
|
-
-
|
|
118
|
+
git worktree add "$base_worktree" "$BASE_SHA"
|
|
119
|
+
(cd "$base_worktree" && bun install --frozen-lockfile && bunx stryker run --mutate "$SCOPE")
|
|
120
|
+
- if: env.SCOPE != ''
|
|
121
|
+
run: bun run checks-mutation-compare --advisory "$BASE_WORKTREE/reports/mutation/mutation.json" reports/mutation/mutation.json
|
|
106
122
|
- if: always() && env.BASE_WORKTREE != ''
|
|
107
123
|
run: |
|
|
108
124
|
git worktree remove --force "$BASE_WORKTREE"
|
|
109
125
|
git worktree prune
|
|
110
126
|
```
|
|
111
127
|
|
|
112
|
-
|
|
128
|
+
The scope step picks the files the pull request changes and the sources its changed unit tests cover or kill, as [checks-mutation-scope](checks-mutation-scope.md) says.
|
|
129
|
+
Neither run mutates a file outside that scope, so the job's time follows the size of the change and not the size of the repository.
|
|
130
|
+
Both runs take the same `--mutate "$SCOPE"`, so the two reports hold the same files.
|
|
131
|
+
A scope file the merge base lacks matches nothing there, and Stryker still writes a report.
|
|
132
|
+
An empty scope skips both runs and the comparison, and the notice names the skip on the run's summary.
|
|
133
|
+
A changed unit test with no full baseline on `main` fails the scope step, since no report then says which sources the test covers.
|
|
134
|
+
A changed unit test that `main` edited after that baseline widens the scope to every file the `mutate` globs match.
|
|
135
|
+
Start a full baseline with `gh workflow run mutation`, as [checks-mutation](checks-mutation.md#running-it-in-ci) shows.
|
|
136
|
+
A run with `--mutate` is scoped, so the preset lets it through without `CI=true`.
|
|
113
137
|
The base worktree's path carries the run's id and attempt, and the last step removes it even when a run fails, so a runner kept between jobs starts each job clean.
|
|
114
138
|
A public repository keeps `runs-on: ubuntu-latest`, because a pull request from a fork runs its own code on the runner.
|
|
115
|
-
A private repository sets `runs-on: [self-hosted, Linux, X64, winbox]` instead, which is the fleet's self-hosted Linux runner where it sends its
|
|
139
|
+
A private repository sets `runs-on: [self-hosted, Linux, X64, winbox]` instead, which is the fleet's self-hosted Linux runner where it sends its mutation runs.
|
|
116
140
|
The job never reads `CI_RUNS_ON`, so an override that moves a repository's other jobs during a runner outage leaves the comparison waiting for `winbox`.
|
|
117
141
|
[checks-ci-wiring](checks-ci-wiring.md#runners) refuses a mutation job that names it in `runs-on`.
|
|
118
142
|
|
|
119
143
|
## Related topics
|
|
120
144
|
|
|
121
145
|
- [checks-mutation](checks-mutation.md)
|
|
146
|
+
- [checks-mutation-scope](checks-mutation-scope.md)
|
|
122
147
|
- [checks-test-layout](checks-test-layout.md)
|