@avi2dg/checks 0.9.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +376 -92
- package/package.json +23 -14
- package/presets/effect.language-service.json +8 -0
- package/presets/effect.oxlint.json +11 -0
- package/quality.schema.json +197 -0
- package/scripts/ci-wiring.ts +74 -58
- package/scripts/commit-identity.ts +4 -33
- package/scripts/flake.ts +153 -0
- package/scripts/gates.ts +17 -15
- package/scripts/lint.ts +7 -17
- package/scripts/quality-file.ts +188 -0
- package/scripts/quality.ts +172 -0
- package/scripts/test-layout.ts +4 -4
- package/scripts/test-report.ts +115 -0
- package/scripts/test.ts +153 -0
package/README.md
CHANGED
|
@@ -2,13 +2,17 @@
|
|
|
2
2
|
|
|
3
3
|
Deterministic checks shared across my TypeScript repos. One package,
|
|
4
4
|
`@avi2dg/checks`: the `checks-lint` entry point that runs every lint gate
|
|
5
|
-
below over a range it resolves itself, the
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
5
|
+
below over a range it resolves itself, the `checks-test` entry point
|
|
6
|
+
that runs the suite and refuses an undeclared skip, the `checks-flake`
|
|
7
|
+
run that records the seeds a failing test fails with, the oxlint base
|
|
8
|
+
config, the tsconfig fragment with the Effect language-service block,
|
|
9
|
+
the `quality.json` schema with the generator that turns its Effect paths
|
|
10
|
+
into oxlint and tsconfig fragments, the shared commitlint config, the
|
|
11
|
+
shared dependency-cruiser base, the test-layout check with its bunfig preset, the commit-identity check, the
|
|
12
|
+
comment gate with its backtest, the oxlint suppressions ratchet, the
|
|
13
|
+
Stryker mutation-testing preset with its no-regression comparator, the
|
|
14
|
+
CI-wiring check, and the Effect error-channel plugin compiled to
|
|
15
|
+
JavaScript.
|
|
12
16
|
|
|
13
17
|
Published as `@avi2dg/checks` on the public npm registry.
|
|
14
18
|
|
|
@@ -43,6 +47,17 @@ node_modules/
|
|
|
43
47
|
}
|
|
44
48
|
```
|
|
45
49
|
|
|
50
|
+
`quality.json` at the repository root declares what the repository
|
|
51
|
+
opts into, starting with the commands its CI runs; see "Quality file"
|
|
52
|
+
below:
|
|
53
|
+
|
|
54
|
+
```json
|
|
55
|
+
{
|
|
56
|
+
"$schema": "./node_modules/@avi2dg/checks/quality.schema.json",
|
|
57
|
+
"gates": { "ci": ["bun run lint", "bun run typecheck", "bun run test"] }
|
|
58
|
+
}
|
|
59
|
+
```
|
|
60
|
+
|
|
46
61
|
`bunfig.toml` is a copy of the shipped preset:
|
|
47
62
|
|
|
48
63
|
```sh
|
|
@@ -54,9 +69,12 @@ cp node_modules/@avi2dg/checks/bunfig.toml bunfig.toml
|
|
|
54
69
|
```json
|
|
55
70
|
"lint": "oxlint --type-aware && checks-lint",
|
|
56
71
|
"typecheck": "tsc --noEmit && effect-tsgo diagnostics --project tsconfig.json --format text --strict",
|
|
57
|
-
"test": "
|
|
72
|
+
"test": "checks-test"
|
|
58
73
|
```
|
|
59
74
|
|
|
75
|
+
`checks-test` runs `bun test --randomize` and fails on a skip the
|
|
76
|
+
repository has not declared; see "Test entry point" below.
|
|
77
|
+
|
|
60
78
|
`checks-lint` runs every kit gate a lint needs; see "Lint entry point"
|
|
61
79
|
below. Three of them:
|
|
62
80
|
|
|
@@ -86,12 +104,148 @@ export default {
|
|
|
86
104
|
The registry version is pinned by the consumer's lockfile; bump
|
|
87
105
|
`@avi2dg/checks` to adopt a new release.
|
|
88
106
|
|
|
107
|
+
## Quality file
|
|
108
|
+
|
|
109
|
+
`quality.json` at the repository root says what the repository has
|
|
110
|
+
opted into. The kit's bins find it at the git root and read it there:
|
|
111
|
+
|
|
112
|
+
```json
|
|
113
|
+
{
|
|
114
|
+
"$schema": "./node_modules/@avi2dg/checks/quality.schema.json",
|
|
115
|
+
"defaultBranch": "main",
|
|
116
|
+
"gates": {
|
|
117
|
+
"ci": ["bun run lint", "bun run typecheck", "bun run test"],
|
|
118
|
+
"scheduled": ["bunx checks-flake --runs 10 --report flake-report.json"]
|
|
119
|
+
},
|
|
120
|
+
"commitIdentity": { "authors": [{ "name": "avi2d", "email": "avi2dg@gmail.com" }] },
|
|
121
|
+
"sources": {
|
|
122
|
+
"production": ["src/**/*.ts"],
|
|
123
|
+
"effect": { "paths": ["src/**/*.ts"], "exempt": ["src/host/*.ts"] }
|
|
124
|
+
},
|
|
125
|
+
"agentRules": { "on": [], "off": [] }
|
|
126
|
+
}
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
| Key | Read by | Holds |
|
|
130
|
+
| --- | --- | --- |
|
|
131
|
+
| `defaultBranch` | `checks-lint`, `checks-ci-wiring` | the branch pull requests merge into, `main` when absent |
|
|
132
|
+
| `gates.ci` | `checks-ci-wiring` | the commands CI runs on every pull request; see "CI wiring" |
|
|
133
|
+
| `gates.scheduled` | `checks-ci-wiring` | the commands a schedule runs |
|
|
134
|
+
| `gates.lint` | `checks-lint`, `checks-ci-wiring` | the gates `checks-lint` runs when not all apply; see "Gate selection" |
|
|
135
|
+
| `commitIdentity.authors` | `checks-commit-identity` | the identities allowed to author and commit; see "Commit identity" |
|
|
136
|
+
| `sources.effect` | `checks-quality` | the paths held to the Effect rules, and the files under them that are not; see "Effect rules" |
|
|
137
|
+
| `sources.production` | no kit gate yet | the source the repository ships |
|
|
138
|
+
| `agentRules.on`, `agentRules.off` | agent Rule selection, not the kit | catalogued Rules switched on or off for this repository |
|
|
139
|
+
|
|
140
|
+
Every key is optional. The bins decode the file with one Effect
|
|
141
|
+
`Schema`, and the package ships `quality.schema.json` emitted from that
|
|
142
|
+
schema, so the `$schema` line gives an editor the verdict the bins
|
|
143
|
+
reach, save one: a Rule switched both on and off, which the bins refuse
|
|
144
|
+
and no JSON Schema can express across two lists. A key the schema does
|
|
145
|
+
not name is refused, not ignored, so a misspelt `sources` cannot switch
|
|
146
|
+
the Effect rules off unnoticed. A glob in `sources`
|
|
147
|
+
starts at the repository root, names a directory first, uses `*`
|
|
148
|
+
only within a segment and `**` only as a whole one, and ends in a file
|
|
149
|
+
name with an extension, which are the globs oxlint, the language
|
|
150
|
+
service and git all read alike. oxlint matches `*.ts` at any depth
|
|
151
|
+
where the other two match it at the root alone, so the schema refuses
|
|
152
|
+
it; `**/*.ts` means every depth to all three. The language service
|
|
153
|
+
matches nothing for `src/**` and oxlint nothing for `src/lib`, so the
|
|
154
|
+
schema refuses both; `src/**/*.ts` and `src/lib/*.ts` say it to all
|
|
155
|
+
three.
|
|
156
|
+
|
|
157
|
+
Until a later minor release, a repository with no `quality.json` still
|
|
158
|
+
has `ciWiring` and `commitIdentity` read from `package.json`, with a
|
|
159
|
+
notice on each read. One with both exits 2 until `package.json` drops
|
|
160
|
+
them. The keys map one for one:
|
|
161
|
+
|
|
162
|
+
| `package.json` | `quality.json` |
|
|
163
|
+
| --- | --- |
|
|
164
|
+
| `ciWiring.gates` | `gates.ci` |
|
|
165
|
+
| `ciWiring.scheduled` | `gates.scheduled` |
|
|
166
|
+
| `ciWiring.lintGates` | `gates.lint` |
|
|
167
|
+
| `ciWiring.defaultBranch` | `defaultBranch` |
|
|
168
|
+
| `commitIdentity` | `commitIdentity` |
|
|
169
|
+
|
|
170
|
+
### Generated fragments
|
|
171
|
+
|
|
172
|
+
oxlint and tsc read their own JSON and nothing else, so
|
|
173
|
+
`checks-quality generate` writes what `sources.effect` declares into two
|
|
174
|
+
fragments at the repository root, and the hand-written configs extend
|
|
175
|
+
them. Both fragments are committed:
|
|
176
|
+
|
|
177
|
+
```sh
|
|
178
|
+
checks-quality generate
|
|
179
|
+
checks-quality --check
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
`.oxlintrc.json`:
|
|
183
|
+
|
|
184
|
+
```json
|
|
185
|
+
{
|
|
186
|
+
"extends": ["./node_modules/@avi2dg/checks/oxlintrc.json", "./oxlintrc.quality.json"],
|
|
187
|
+
"plugins": ["typescript", "oxc", "eslint", "import"]
|
|
188
|
+
}
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
`tsconfig.json`:
|
|
192
|
+
|
|
193
|
+
```json
|
|
194
|
+
{
|
|
195
|
+
"extends": ["@avi2dg/checks/tsconfig.effect.json", "./tsconfig.quality.json"]
|
|
196
|
+
}
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
`oxlintrc.quality.json` holds one override: the declared paths as
|
|
200
|
+
`files`, the exempt ones as `excludeFiles`, and the kit's Effect rule
|
|
201
|
+
block, `presets/effect.oxlint.json`. `tsconfig.quality.json` holds the
|
|
202
|
+
language-service override: the same paths as `include`, the exempt ones
|
|
203
|
+
as `exclude`, and `presets/effect.language-service.json`. A kit release
|
|
204
|
+
that changes a preset reaches the repository through its next
|
|
205
|
+
`generate`. A rule only this repository needs stays in its own
|
|
206
|
+
`.oxlintrc.json`, whose overrides come after the fragment's and so win.
|
|
207
|
+
|
|
208
|
+
`checks-lint` runs `checks-quality --check`, which exits 1 when:
|
|
209
|
+
|
|
210
|
+
- a fragment is missing, or differs from what `generate` would write
|
|
211
|
+
from `quality.json` and the installed kit's presets;
|
|
212
|
+
- a fragment is left over once `quality.json` stops declaring
|
|
213
|
+
`sources.effect`;
|
|
214
|
+
- `.oxlintrc.json` or `tsconfig.json` does not list its fragment in
|
|
215
|
+
`extends`, so the tool never reads it;
|
|
216
|
+
- a `sources.effect.paths` glob matches no tracked or untracked file,
|
|
217
|
+
so it holds nothing to the rules.
|
|
218
|
+
|
|
219
|
+
It exits 2 when `quality.json` does not decode. `generate` writes the
|
|
220
|
+
fragments, removes a left-over one, then runs the same check.
|
|
221
|
+
|
|
222
|
+
```
|
|
223
|
+
checks-quality: 2 problem(s) with what quality.json declares:
|
|
224
|
+
oxlintrc.quality.json is stale against quality.json and the kit's presets; run checks-quality generate
|
|
225
|
+
tsconfig.json does not extend ./tsconfig.quality.json, so the language service never reads it
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
Two details of the fragments are easy to get wrong, so the kit's tests
|
|
229
|
+
pin both:
|
|
230
|
+
|
|
231
|
+
- A fragment sits at the repository root. oxlint resolves an
|
|
232
|
+
override's `files`, and the language service an override's `include`,
|
|
233
|
+
against the directory of the config holding it. From `.quality/` the
|
|
234
|
+
language service reports no error at all on an `async function`
|
|
235
|
+
planted under a declared path.
|
|
236
|
+
- The oxlint fragment always sets `plugins`, to the kit's. A config in
|
|
237
|
+
`extends` that sets none brings in oxlint's default plugins, whose
|
|
238
|
+
category rules then fire across the whole tree. The override names
|
|
239
|
+
the kit's plugins beside the preset's `node`, `promise` and `unicorn`,
|
|
240
|
+
because one that leaves any of the kit's out turns on the category
|
|
241
|
+
rules of the plugins it adds under every declared path.
|
|
242
|
+
|
|
89
243
|
## Lint entry point
|
|
90
244
|
|
|
91
245
|
`checks-lint` runs each of the kit's lint gates in turn and names every
|
|
92
246
|
one that fails, rather than stopping at the first. A repository whose
|
|
93
247
|
tracked files give a gate nothing to check can leave it out through
|
|
94
|
-
`
|
|
248
|
+
`gates.lint`; see "Gate selection" under "CI wiring".
|
|
95
249
|
|
|
96
250
|
| Gate | Reads |
|
|
97
251
|
| --- | --- |
|
|
@@ -101,6 +255,7 @@ tracked files give a gate nothing to check can leave it out through
|
|
|
101
255
|
| `checks-comment-gate` | the range |
|
|
102
256
|
| `checks-suppressions-ratchet` | the range |
|
|
103
257
|
| `checks-ci-wiring` | the working tree |
|
|
258
|
+
| `checks-quality` | the working tree |
|
|
104
259
|
|
|
105
260
|
```sh
|
|
106
261
|
checks-lint
|
|
@@ -111,8 +266,8 @@ It resolves the range once and hands the same one to every range gate.
|
|
|
111
266
|
Locally, and on any event other than a pull request, the range ends at
|
|
112
267
|
`HEAD` and starts where `HEAD` branched from the origin default branch:
|
|
113
268
|
`origin/HEAD`, or when `origin/HEAD` is not set, as in an
|
|
114
|
-
`actions/checkout` clone, `origin/<
|
|
115
|
-
repository's `
|
|
269
|
+
`actions/checkout` clone, `origin/<defaultBranch>` from the
|
|
270
|
+
repository's `quality.json` (see "Quality file"), and `origin/main` when
|
|
116
271
|
that is not declared. In a GitHub Actions pull request, where
|
|
117
272
|
`GITHUB_EVENT_NAME` is `pull_request`, it ends
|
|
118
273
|
at the event's head sha and starts where that branched from
|
|
@@ -153,12 +308,12 @@ gate's own report, then its verdict:
|
|
|
153
308
|
```
|
|
154
309
|
checks-lint: range 2504acf098d120e73a8ece3c96f22b934f35c6a8..10ba7d8935b73ed72624120a1542e51bd21ca7c7 from HEAD against origin/main
|
|
155
310
|
...
|
|
156
|
-
checks-lint: 3 of
|
|
311
|
+
checks-lint: 3 of 7 gate(s) failed: checks-commit-identity, checks-comment-gate, checks-suppressions-ratchet
|
|
157
312
|
```
|
|
158
313
|
|
|
159
314
|
It exits 1 when any gate found a violation, and 2 when the range or the
|
|
160
315
|
selection does not resolve, or no failing gate could decide. ci-wiring
|
|
161
|
-
always runs, so a repository on `checks-lint` declares `
|
|
316
|
+
always runs, so a repository on `checks-lint` declares `gates.ci`
|
|
162
317
|
(see "CI wiring"), and it holds the test layout unless its selection
|
|
163
318
|
leaves out `checks-test-layout`.
|
|
164
319
|
|
|
@@ -199,69 +354,43 @@ Each refusal says what to write instead: a `Schema.TaggedError` failed
|
|
|
199
354
|
through `Effect.fail`, a throwing call wrapped in `Effect.try` or
|
|
200
355
|
`Effect.tryPromise`, and recovery by tag with `Effect.catchTag`.
|
|
201
356
|
|
|
202
|
-
A repository turns them on for the paths it writes in Effect
|
|
203
|
-
`
|
|
357
|
+
A repository turns them on for the paths it writes in Effect by
|
|
358
|
+
declaring those paths in `quality.json` and extending the generated
|
|
359
|
+
fragments; see "Generated fragments" under "Quality file":
|
|
204
360
|
|
|
205
361
|
```json
|
|
206
|
-
{
|
|
207
|
-
"
|
|
208
|
-
"plugins": ["typescript", "oxc", "eslint", "import"],
|
|
209
|
-
"overrides": [
|
|
210
|
-
{
|
|
211
|
-
"files": ["src/**"],
|
|
212
|
-
"rules": {
|
|
213
|
-
"effect-channel/no-throw": "error",
|
|
214
|
-
"effect-channel/no-try-catch": "error"
|
|
215
|
-
}
|
|
216
|
-
}
|
|
217
|
-
]
|
|
362
|
+
"sources": {
|
|
363
|
+
"effect": { "paths": ["src/**/*.ts"], "exempt": ["src/host/*.ts"] }
|
|
218
364
|
}
|
|
219
365
|
```
|
|
220
366
|
|
|
367
|
+
The fragment's override carries the kit's Effect rule block,
|
|
368
|
+
`presets/effect.oxlint.json`: the two rules above, plus `node/no-sync`,
|
|
369
|
+
`oxc/no-async-await`, `promise/avoid-new` and `unicorn/no-process-exit`.
|
|
370
|
+
Files under `exempt` answer to none of them.
|
|
371
|
+
|
|
221
372
|
oxlint resolves `files` against the directory of the config that holds
|
|
222
373
|
the override, so a config passed with `-c` from outside the repository
|
|
223
374
|
matches nothing and reports nothing.
|
|
224
375
|
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
a
|
|
230
|
-
|
|
231
|
-
baseline, `oxlint --suppress-all`,
|
|
232
|
-
|
|
233
|
-
|
|
376
|
+
`unicorn/no-process-exit` passes over any file that opens with a
|
|
377
|
+
shebang, so a repository whose bins open with one bans `process.exit`
|
|
378
|
+
itself with `no-restricted-properties` in its own `.oxlintrc.json`, as
|
|
379
|
+
this one does. The preset leaves that rule out because
|
|
380
|
+
a repository's own `no-restricted-properties` list for the same files
|
|
381
|
+
would replace it, or be replaced by it. Sites standing when the
|
|
382
|
+
declaration lands go in oxlint's own baseline, `oxlint --suppress-all`,
|
|
383
|
+
so their count can only fall. This repository's own `.oxlintrc.json`
|
|
384
|
+
also lifts `effect-channel/no-throw` from `scripts/comment-matchers.ts`,
|
|
385
|
+
the one file under its Effect path that a host loads without
|
|
386
|
+
`node_modules`.
|
|
234
387
|
|
|
235
388
|
The language service holds the same paths to Effect-native IO through
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
{
|
|
242
|
-
"extends": "@avi2dg/checks/tsconfig.effect.json",
|
|
243
|
-
"compilerOptions": {
|
|
244
|
-
"plugins": [
|
|
245
|
-
{
|
|
246
|
-
"name": "@effect/language-service",
|
|
247
|
-
"overrides": [
|
|
248
|
-
{
|
|
249
|
-
"include": ["src/**/*.ts"],
|
|
250
|
-
"options": {
|
|
251
|
-
"diagnosticSeverity": {
|
|
252
|
-
"nodeBuiltinImport": "error",
|
|
253
|
-
"asyncFunction": "error",
|
|
254
|
-
"newPromise": "error",
|
|
255
|
-
"extendsNativeError": "error"
|
|
256
|
-
}
|
|
257
|
-
}
|
|
258
|
-
}
|
|
259
|
-
]
|
|
260
|
-
}
|
|
261
|
-
]
|
|
262
|
-
}
|
|
263
|
-
}
|
|
264
|
-
```
|
|
389
|
+
the tsconfig fragment's override, whose severities are
|
|
390
|
+
`presets/effect.language-service.json`: `nodeBuiltinImport`,
|
|
391
|
+
`asyncFunction`, `newPromise` and `extendsNativeError`, all errors.
|
|
392
|
+
effect-tsgo keeps the severities `tsconfig.effect.json` sets when a later
|
|
393
|
+
config in `extends` restates the plugin with only its overrides.
|
|
265
394
|
|
|
266
395
|
## Test layout
|
|
267
396
|
|
|
@@ -285,8 +414,9 @@ file and the path to move it to when it does not:
|
|
|
285
414
|
them; `tests/fixtures/**` is data and is not parsed. Detection parses
|
|
286
415
|
with swc and reads import specifiers and identifier use, so a test that
|
|
287
416
|
only carries `"node:child_process"` as a string is not a violation.
|
|
288
|
-
- `scripts.test` is exactly `bun test --randomize`
|
|
289
|
-
|
|
417
|
+
- `scripts.test` is exactly `checks-test`, which runs `bun test --randomize`
|
|
418
|
+
(see "Test entry point"), and `scripts.lint` runs this check, itself or
|
|
419
|
+
through `checks-lint` called by its bare bin name.
|
|
290
420
|
- `bunfig.toml` carries every `[test]` key of the shipped preset with the
|
|
291
421
|
same value, and `[test].pathIgnorePatterns` is always
|
|
292
422
|
`["**/tests/quarantine/**"]`: the check pins it itself, so this repo,
|
|
@@ -305,6 +435,109 @@ still run on demand:
|
|
|
305
435
|
bun test --path-ignore-patterns='' tests/quarantine
|
|
306
436
|
```
|
|
307
437
|
|
|
438
|
+
## Test entry point
|
|
439
|
+
|
|
440
|
+
`checks-test` runs the whole suite with `bun test --randomize`, passes
|
|
441
|
+
bun's output through, and then reads bun's JUnit report of the same run.
|
|
442
|
+
bun exits 0 with tests skipped, so a green run says nothing about the
|
|
443
|
+
tests that never ran. `checks-test` fails when a test was skipped, by
|
|
444
|
+
`test.skip`, `test.skipIf`, `test.if`, `describe.skip` or `test.todo`,
|
|
445
|
+
without a declaration in `package.json`:
|
|
446
|
+
|
|
447
|
+
```json
|
|
448
|
+
"testSkips": [
|
|
449
|
+
{
|
|
450
|
+
"file": "tests/e2e/docker.test.ts",
|
|
451
|
+
"test": "images > builds the release image",
|
|
452
|
+
"reason": "the runner has no docker daemon",
|
|
453
|
+
"when": "ci"
|
|
454
|
+
}
|
|
455
|
+
]
|
|
456
|
+
```
|
|
457
|
+
|
|
458
|
+
`file` is the path bun reports, relative to the package root, and `test`
|
|
459
|
+
is the name bun's console prints: the describe blocks and the test name
|
|
460
|
+
joined by ` > `. `reason` is required. `when` is `ci` or `local` for a
|
|
461
|
+
test skipped only there, and a declaration without it holds in both;
|
|
462
|
+
`checks-test` counts a run as `ci` when `CI` is set true, as GitHub
|
|
463
|
+
Actions sets it. A declaration that holds for the run but matches no
|
|
464
|
+
skipped test fails a ci run too, so a fixed or renamed test takes its
|
|
465
|
+
declaration with it. A local run only warns about it, because whether a
|
|
466
|
+
test skips there can hang on the machine, such as a docker daemon being
|
|
467
|
+
up:
|
|
468
|
+
|
|
469
|
+
```
|
|
470
|
+
checks-test: 1 skipped test(s) undeclared and 1 declaration(s) matching no skipped test in this ci run:
|
|
471
|
+
tests/pricing.test.ts:12 pricing > rounds half to even: skipped with no declaration; run it, or declare it in package.json testSkips with its reason
|
|
472
|
+
tests/e2e/docker.test.ts > images > builds the release image: declared, but no such test skipped; delete the declaration
|
|
473
|
+
```
|
|
474
|
+
|
|
475
|
+
It exits 1 when a test failed or a skip is undeclared or, in a ci run,
|
|
476
|
+
a declaration stale, and 2 when `testSkips` does not parse or bun
|
|
477
|
+
passed without writing its report. It takes no arguments: a `-t`
|
|
478
|
+
filter reports every test it leaves out as skipped and a path filter
|
|
479
|
+
drops files a declaration names, so a narrowed run is plain
|
|
480
|
+
`bun test --randomize` with the arguments. Files under
|
|
481
|
+
`tests/quarantine/` are never run and so never reported; see "Test
|
|
482
|
+
layout".
|
|
483
|
+
|
|
484
|
+
## Flake run
|
|
485
|
+
|
|
486
|
+
A green run proves nothing failed in that run, not that no test is
|
|
487
|
+
flaky. `checks-flake` runs the whole suite several times, each with its
|
|
488
|
+
own `--seed`, and records per failing test the seeds it failed with:
|
|
489
|
+
|
|
490
|
+
```sh
|
|
491
|
+
checks-flake [--runs <count> | --seed <seed>...] [--report <file>]
|
|
492
|
+
```
|
|
493
|
+
|
|
494
|
+
`--runs` defaults to 10 runs on random seeds, and `--seed`, given once
|
|
495
|
+
per run, replays chosen seeds, such as the ones a report recorded.
|
|
496
|
+
`bun test --randomize --seed=<seed>` puts the suite in the same order,
|
|
497
|
+
so a seed reproduces a failure that hangs on order. `--report` writes the
|
|
498
|
+
record as JSON, every run's seed and failing tests and every failing
|
|
499
|
+
test's seeds, and under GitHub Actions the summary below is appended to
|
|
500
|
+
the job summary:
|
|
501
|
+
|
|
502
|
+
```
|
|
503
|
+
checks-flake: 3 of 10 run(s) failed, 1 test(s) failing in them
|
|
504
|
+
|
|
505
|
+
| Test | Failed | Seeds |
|
|
506
|
+
| --- | --- | --- |
|
|
507
|
+
| tests/cache.test.ts:6 reads the cache | 3 of 10 runs | 2170533150, 4046124386, 180394251 |
|
|
508
|
+
|
|
509
|
+
Reproduce a failing run with bun test --randomize --seed=<seed>.
|
|
510
|
+
```
|
|
511
|
+
|
|
512
|
+
A run that fails with no failing test, such as a test file that throws
|
|
513
|
+
while loading, is listed with its seed on its own line. It exits 1 when
|
|
514
|
+
any run failed and 2 when bun passed without writing its report.
|
|
515
|
+
|
|
516
|
+
A consumer runs it on a schedule and keeps the record as an artifact:
|
|
517
|
+
|
|
518
|
+
```yaml
|
|
519
|
+
on:
|
|
520
|
+
schedule:
|
|
521
|
+
- cron: "17 5 * * *"
|
|
522
|
+
workflow_dispatch:
|
|
523
|
+
jobs:
|
|
524
|
+
flake:
|
|
525
|
+
runs-on: ubuntu-latest
|
|
526
|
+
steps:
|
|
527
|
+
- uses: actions/checkout@v5
|
|
528
|
+
- uses: oven-sh/setup-bun@v2
|
|
529
|
+
- run: bun install --frozen-lockfile
|
|
530
|
+
- run: bunx checks-flake --runs 10 --report flake-report.json
|
|
531
|
+
- uses: actions/upload-artifact@v4
|
|
532
|
+
if: always()
|
|
533
|
+
with:
|
|
534
|
+
name: flake-report
|
|
535
|
+
path: flake-report.json
|
|
536
|
+
```
|
|
537
|
+
|
|
538
|
+
and declares the step in `gates.scheduled`, so `checks-ci-wiring`
|
|
539
|
+
fails once the schedule stops running it; see "CI wiring".
|
|
540
|
+
|
|
308
541
|
## Dependency rules
|
|
309
542
|
|
|
310
543
|
`.dependency-cruiser.cjs` extends the shared base, which carries
|
|
@@ -410,7 +643,7 @@ body are left alone. `GitHub <noreply@github.com>` is allowed as
|
|
|
410
643
|
committer only, since that is who writes a squash merge.
|
|
411
644
|
|
|
412
645
|
The allowlist defaults to `avi2d <avi2dg@gmail.com>`. A repo with other
|
|
413
|
-
owners restates it in `
|
|
646
|
+
owners restates it in `quality.json`:
|
|
414
647
|
|
|
415
648
|
```json
|
|
416
649
|
"commitIdentity": {
|
|
@@ -491,13 +724,17 @@ longer runs on pull requests to the default branch. No local check sees
|
|
|
491
724
|
that: a workflow whose lint step became a no-op leaves `bun run lint`
|
|
492
725
|
green.
|
|
493
726
|
|
|
494
|
-
The repository declares its gates once, in `
|
|
495
|
-
|
|
727
|
+
The repository declares its gates once, in `quality.json`:
|
|
728
|
+
|
|
729
|
+
```json
|
|
730
|
+
"gates": {
|
|
731
|
+
"ci": ["bun run lint", "bun run typecheck", "bun run test"]
|
|
732
|
+
}
|
|
733
|
+
```
|
|
734
|
+
|
|
735
|
+
and `package.json` runs the check through `checks-lint`:
|
|
496
736
|
|
|
497
737
|
```json
|
|
498
|
-
"ciWiring": {
|
|
499
|
-
"gates": ["bun run lint", "bun run typecheck", "bun run test"]
|
|
500
|
-
},
|
|
501
738
|
"scripts": {
|
|
502
739
|
"lint": "oxlint --type-aware && checks-lint"
|
|
503
740
|
}
|
|
@@ -539,11 +776,11 @@ one of the gates `checks-lint` runs by its bare bin name, when the step
|
|
|
539
776
|
calls `checks-lint` the same way: `bunx checks-lint` counts for
|
|
540
777
|
`bunx checks-comment-gate "origin/$BASE_REF" "$HEAD_SHA"`. A step running `bun run lint` counts only for the `bun run lint` gate,
|
|
541
778
|
since the check never reads what a package script runs. So once `lint`
|
|
542
|
-
runs `checks-lint`, the per-gate entries can leave `
|
|
779
|
+
runs `checks-lint`, the per-gate entries can leave `gates.ci`
|
|
543
780
|
along with the workflows that ran them.
|
|
544
781
|
|
|
545
782
|
The default branch is `main`; a repo with another one sets
|
|
546
|
-
`"defaultBranch"`
|
|
783
|
+
`"defaultBranch"` in `quality.json`, which `checks-lint` also reads when
|
|
547
784
|
`origin/HEAD` is not set.
|
|
548
785
|
|
|
549
786
|
It exits 1 naming each gap, with every step that runs the gate and why
|
|
@@ -555,32 +792,57 @@ ci-wiring: 1 of 8 gate(s) do not run on pull requests to main:
|
|
|
555
792
|
.github/workflows/release.yml job publish step 7: .github/workflows/release.yml does not trigger on pull_request
|
|
556
793
|
```
|
|
557
794
|
|
|
558
|
-
It exits 2 when `
|
|
559
|
-
|
|
560
|
-
|
|
795
|
+
It exits 2 when `quality.json` declares no `gates.ci` or does not
|
|
796
|
+
decode, a gate or scheduled command is not one plain command, or a
|
|
797
|
+
workflow does not parse. Whether a workflow is well formed is
|
|
798
|
+
actionlint's question, not this one's.
|
|
799
|
+
|
|
800
|
+
A command a schedule must run, such as the flake run, goes in
|
|
801
|
+
`gates.scheduled`:
|
|
802
|
+
|
|
803
|
+
```json
|
|
804
|
+
"gates": {
|
|
805
|
+
"ci": ["bun run lint", "bun run typecheck", "bun run test"],
|
|
806
|
+
"scheduled": ["bunx checks-flake --runs 10 --report flake-report.json"]
|
|
807
|
+
}
|
|
808
|
+
```
|
|
809
|
+
|
|
810
|
+
Each counts only as a step of the same plain shape in a workflow whose
|
|
811
|
+
`on` carries `schedule` with at least one `cron`, under the same
|
|
812
|
+
`if: false`, `continue-on-error: true` and `needs` rules as a gate. It
|
|
813
|
+
exits 1 naming each one no schedule runs:
|
|
814
|
+
|
|
815
|
+
```
|
|
816
|
+
ci-wiring: 1 of 1 scheduled command(s) do not run on a schedule:
|
|
817
|
+
bunx checks-flake --runs 10 --report flake-report.json
|
|
818
|
+
.github/workflows/ci.yml job checks step 5: .github/workflows/ci.yml does not trigger on a schedule
|
|
819
|
+
```
|
|
561
820
|
|
|
562
821
|
### Gate selection
|
|
563
822
|
|
|
564
823
|
A repository with no TypeScript source gives `checks-lint-coverage` and
|
|
565
824
|
`checks-test-layout` nothing to check, and test-layout still refuses its
|
|
566
825
|
missing `bun test` script and `bunfig.toml`. It declares the gates
|
|
567
|
-
`checks-lint` runs as `
|
|
826
|
+
`checks-lint` runs as `gates.lint`:
|
|
568
827
|
|
|
569
828
|
```json
|
|
570
|
-
"
|
|
571
|
-
"
|
|
572
|
-
"
|
|
829
|
+
"gates": {
|
|
830
|
+
"ci": ["bun run lint"],
|
|
831
|
+
"lint": [
|
|
573
832
|
"checks-commit-identity",
|
|
574
833
|
"checks-comment-gate",
|
|
575
834
|
"checks-suppressions-ratchet",
|
|
576
|
-
"checks-ci-wiring"
|
|
835
|
+
"checks-ci-wiring",
|
|
836
|
+
"checks-quality"
|
|
577
837
|
]
|
|
578
838
|
}
|
|
579
839
|
```
|
|
580
840
|
|
|
581
841
|
`checks-lint` runs exactly those, in the "Lint entry point" table's
|
|
582
|
-
order, and all
|
|
583
|
-
`
|
|
842
|
+
order, and all seven when `gates.lint` is absent. A selection in
|
|
843
|
+
`quality.json` always keeps `checks-quality`, since the file it sits in
|
|
844
|
+
is what makes that gate apply. A step running
|
|
845
|
+
`checks-lint` then counts only for a declared gate that `gates.lint`
|
|
584
846
|
keeps.
|
|
585
847
|
|
|
586
848
|
A selection may leave out only a gate that does not apply:
|
|
@@ -593,14 +855,15 @@ A selection may leave out only a gate that does not apply:
|
|
|
593
855
|
| `checks-comment-gate` | always |
|
|
594
856
|
| `checks-suppressions-ratchet` | always |
|
|
595
857
|
| `checks-ci-wiring` | always |
|
|
858
|
+
| `checks-quality` | tracks a `quality.json` |
|
|
596
859
|
|
|
597
|
-
Both bins exit 2 on a `
|
|
860
|
+
Both bins exit 2 on a `gates.lint` that names an unknown gate or leaves
|
|
598
861
|
out one that always applies. ci-wiring exits 1 when the
|
|
599
862
|
selection leaves out a gate the repository's tracked files make
|
|
600
863
|
applicable, and names the gate and the files:
|
|
601
864
|
|
|
602
865
|
```
|
|
603
|
-
ci-wiring:
|
|
866
|
+
ci-wiring: quality.json gates.lint leaves out 2 gate(s) this repository's contents make applicable:
|
|
604
867
|
checks-lint-coverage: the repository tracks TypeScript source (src/widget.ts)
|
|
605
868
|
checks-test-layout: the repository tracks TypeScript source (src/widget.ts)
|
|
606
869
|
```
|
|
@@ -687,9 +950,12 @@ The shared Stryker preset's `json` reporter writes
|
|
|
687
950
|
|
|
688
951
|
## Why it is shaped this way
|
|
689
952
|
|
|
690
|
-
-
|
|
691
|
-
|
|
692
|
-
|
|
953
|
+
- Every config in an oxlint `extends` chain brings its own `plugins`,
|
|
954
|
+
and one that sets none brings oxlint's default plugins, whose
|
|
955
|
+
category rules the base's `categories` then turn on across the tree.
|
|
956
|
+
`rules`, `categories` and `jsPlugins` inherit as expected. That is why
|
|
957
|
+
the consumer snippet restates `plugins` and nothing else, and why the
|
|
958
|
+
generated fragment always sets them.
|
|
693
959
|
- `node_modules/` is excluded through the consumer's `.gitignore`, not
|
|
694
960
|
`ignorePatterns`: oxlint still walks the installed package when only
|
|
695
961
|
`ignorePatterns` names it.
|
|
@@ -705,7 +971,19 @@ The shared Stryker preset's `json` reporter writes
|
|
|
705
971
|
- `dist/` is committed. No `prepack` or `prepublishOnly` builds it, so a
|
|
706
972
|
publish ships whatever bundle the publishing worktree holds. Rebuild it
|
|
707
973
|
after pulling with `bun run build`; CI fails when the committed bundle
|
|
708
|
-
is stale.
|
|
974
|
+
is stale. `bun run build` also emits `quality.schema.json`, which is
|
|
975
|
+
committed the same way, and a test fails when it differs from what
|
|
976
|
+
the schema emits.
|
|
977
|
+
- `quality.json` is JSON, not TOML or a TypeScript module: a bun bin, a
|
|
978
|
+
hook running without `node_modules`, a `.cjs` or `.mjs` config and
|
|
979
|
+
`jq` all parse it with nothing installed, and nobody runs a
|
|
980
|
+
repository's own code to learn its policy. It holds declarations
|
|
981
|
+
only. The kit's bins read it directly; oxlint and tsc read nothing but
|
|
982
|
+
their own JSON, so they extend generated fragments, which
|
|
983
|
+
`checks-quality --check` holds to the declarations.
|
|
984
|
+
- `quality.json` refuses a key its schema does not name, so a kit that
|
|
985
|
+
cannot enforce a newer key refuses it rather than let the repository
|
|
986
|
+
believe it enforced.
|
|
709
987
|
- The base parses with swc because typescript 7 (tsgo) has no compiler
|
|
710
988
|
API for dependency-cruiser to use. Without `@swc/core` installed the
|
|
711
989
|
cruise silently skips every `.ts` file, so this repo's test asserts its
|
|
@@ -741,6 +1019,11 @@ The shared Stryker preset's `json` reporter writes
|
|
|
741
1019
|
that check, which is why a selection without it, or without another
|
|
742
1020
|
gate that applies everywhere, is refused as `checks-lint` reads it:
|
|
743
1021
|
nothing would check the selection otherwise.
|
|
1022
|
+
- `checks-test` runs bun itself rather than reading a report some other
|
|
1023
|
+
run left: a skip taken only on CI is visible only in CI's own run, and
|
|
1024
|
+
an earlier run's report may be stale or narrowed. It reads the JUnit
|
|
1025
|
+
report bun writes to a temporary directory, since bun has no other
|
|
1026
|
+
per-test output meant for a program.
|
|
744
1027
|
- `checks-ci-wiring` runs inside `lint`, not in a workflow of its own:
|
|
745
1028
|
deleting the step that runs a check is the violation it catches, so the
|
|
746
1029
|
local `lint` is where it has to fail.
|
|
@@ -781,7 +1064,8 @@ a tag off `main` and reruns the build, `dist/` check, lint, typecheck
|
|
|
781
1064
|
and tests before it publishes:
|
|
782
1065
|
|
|
783
1066
|
```sh
|
|
784
|
-
|
|
1067
|
+
tag="v$(bun -p 'require("./package.json").version')"
|
|
1068
|
+
git tag "$tag" && git push origin "$tag"
|
|
785
1069
|
```
|
|
786
1070
|
|
|
787
1071
|
The `release` workflow publishes the tagged version through npm
|