@avi2dg/checks 0.13.0 → 0.14.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,1267 +1,182 @@
1
1
  # checks
2
2
 
3
- Deterministic checks shared across my TypeScript repos. One package,
4
- `@avi2dg/checks`: the `checks-lint` entry point that runs every lint gate
5
- below over a range it resolves itself, the `checks-test` entry point
6
- that runs the suite and refuses an undeclared skip, the `checks-flake`
7
- run that records the seeds a failing test fails with, the oxlint base
8
- config, the tsconfig fragment with the Effect language-service block,
9
- the `quality.json` schema with the generator that turns its Effect paths
10
- into oxlint and tsconfig fragments, the shared commitlint config, the
11
- shared dependency-cruiser base with the feature-owner rules `quality.json`
12
- compiles into it, the test-layout check with its bunfig preset, the commit-identity check, the
13
- comment gate with its backtest, the oxlint suppressions ratchet, the
14
- size budget, the feature-owner change signal and proof check, the
15
- Stryker mutation-testing preset with its no-regression comparator, the
16
- CI-wiring check, a template for each kind of doc file with the gate that
17
- holds each doc file to its template, and the Effect error-channel plugin
18
- compiled to JavaScript.
19
-
20
- Published as `@avi2dg/checks` on the public npm registry.
3
+ `@avi2dg/checks` is the kit of deterministic checks a TypeScript repository installs to hold its code, tests, commits, CI wiring and docs to one shared standard.
4
+ It ships the lint gates `checks-lint` runs over each pull request, the test runners, and the configs a repository extends for oxlint, tsc, dependency-cruiser, commitlint, bun and Stryker.
5
+ A repository declares what it opts into once, in `quality.json`, and each check decides its constraint the same way on every run.
21
6
 
22
7
  ## Before you begin
23
8
 
24
- - Bun, which runs every bin.
25
- The kit is tested on the version its own `.bun-version` pins.
9
+ <!-- generated prerequisites: bun run build writes it from package.json, .bun-version and scripts/doc-blocks.ts -->
10
+
26
11
  - A git repository, whose history the range gates read.
27
- - The peer versions the install line below pins, which `peerDependencies` in `package.json` holds.
12
+ - Bun 1.3.13, which runs every bin.
13
+ - TypeScript 7.0.2, whose `tsc` the `typecheck` script runs.
14
+ - The peer dependencies, at the exact versions the kit pins:
15
+ - `@effect/tsgo` 0.45.0
16
+ - `@swc/core` 1.16.2
17
+ - `dependency-cruiser` 18.4.0
18
+ - `effect` 4.0.0-rc.115
19
+ - `oxlint` 1.83.0
20
+ - `oxlint-tsgolint` 7.0.2002
21
+
22
+ <!-- end generated prerequisites -->
28
23
 
29
24
  ## Install
30
25
 
31
- From the consuming repo:
32
-
33
- ```sh
34
- bun add -d @avi2dg/checks effect@4.0.0-rc.115 oxlint@1.83.0 oxlint-tsgolint@7.0.2002 @effect/tsgo@0.45.0 typescript@7.0.2 dependency-cruiser@18.4.0 @swc/core@1.16.2
35
- ```
36
-
37
- `.oxlintrc.json`:
38
-
39
- ```json
40
- {
41
- "extends": ["./node_modules/@avi2dg/checks/oxlintrc.json"],
42
- "plugins": ["typescript", "oxc", "eslint", "import"]
43
- }
44
- ```
45
-
46
- `.gitignore` keeps oxlint out of `node_modules/`:
47
-
48
- ```
49
- node_modules/
50
- ```
51
-
52
- `tsconfig.json` gains one line:
53
-
54
- ```json
55
- {
56
- "extends": "@avi2dg/checks/tsconfig.effect.json"
57
- }
58
- ```
59
-
60
- `quality.json` at the repository root declares what the repository
61
- opts into, starting with the commands its CI runs. See "Declare policy in the quality file"
62
- below:
63
-
64
- ```json
65
- {
66
- "$schema": "./node_modules/@avi2dg/checks/quality.schema.json",
67
- "gates": { "ci": ["bun run lint", "bun run typecheck", "bun run test"] }
68
- }
69
- ```
70
-
71
- `bunfig.toml` is a copy of the shipped preset:
72
-
73
- ```sh
74
- cp node_modules/@avi2dg/checks/bunfig.toml bunfig.toml
75
- ```
76
-
77
- `package.json` gains three scripts:
78
-
79
- ```json
80
- "lint": "oxlint --type-aware && checks-lint",
81
- "typecheck": "tsc --noEmit && effect-tsgo diagnostics --project tsconfig.json --format text --strict",
82
- "test": "checks-test"
83
- ```
84
-
85
- `checks-test` runs `bun test --randomize` and fails on a skip the
86
- repository has not declared. See "Run the suite" below.
87
-
88
- `checks-lint` runs every kit gate a lint needs. See "Run every lint gate"
89
- below. Three of them:
90
-
91
- `lint-coverage.sh` fails when oxlint silently skips a tracked `.ts` or
92
- `.tsx` file, for example through a stray `.gitignore` entry. It compares
93
- `git ls-files` against oxlint's own file walk and names the missing files.
94
- It exits 2 when oxlint cannot walk the tree, as when it is not on `PATH`
95
- or its config does not parse.
96
-
97
- `test-layout.ts` decides the test layout described below.
98
-
99
- `commit-identity.ts` refuses a commit with an author other than the
100
- repository owner. See "Check commit identities" below.
101
-
102
- A repo that runs mutation testing installs `@stryker-mutator/core` and
103
- `@hughescr/stryker-bun-runner`, then spreads the shipped preset in
104
- `stryker.conf.mjs`:
105
-
106
- ```js
107
- import preset from "@avi2dg/checks/stryker.preset.js";
108
-
109
- export default {
110
- ...preset,
111
- };
112
- ```
113
-
114
- The registry version is pinned by the consumer's lockfile; bump
115
- `@avi2dg/checks` to adopt a new release.
116
-
117
- ## Declare policy in the quality file
118
-
119
- `quality.json` at the repository root says what the repository has
120
- opted into. The kit's bins find it at the git root and read it there:
121
-
122
- ```json
123
- {
124
- "$schema": "./node_modules/@avi2dg/checks/quality.schema.json",
125
- "defaultBranch": "main",
126
- "gates": {
127
- "ci": ["bun run lint", "bun run typecheck", "bun run test"],
128
- "scheduled": ["bunx checks-flake --runs 10 --report flake-report.json"]
129
- },
130
- "commitIdentity": { "authors": [{ "name": "avi2d", "email": "avi2dg@gmail.com" }] },
131
- "sources": {
132
- "production": ["src/**/*.ts"],
133
- "effect": { "paths": ["src/**/*.ts"], "exempt": ["src/host/*.ts"] }
134
- },
135
- "size": { "fileLines": 400, "functionLines": 100, "applies": "changed" },
136
- "features": [
137
- {
138
- "name": "billing",
139
- "root": "src/billing",
140
- "entries": ["src/billing/index.ts"],
141
- "allowFrom": ["src/main.ts"],
142
- "proof": "tests/e2e/billing.test.ts"
143
- }
144
- ],
145
- "changeSignal": "advisory",
146
- "agentRules": { "on": [], "off": [] },
147
- "docs": { "pages": { "reference": ["docs/gates/*.md"], "explanation": ["docs/design.md"] } }
148
- }
149
- ```
150
-
151
- | Key | Read by | Holds |
152
- | --- | --- | --- |
153
- | `defaultBranch` | `checks-lint`, `checks-ci-wiring` | the branch pull requests merge into, `main` when absent |
154
- | `gates.ci` | `checks-ci-wiring` | the commands CI runs on every pull request. See "Check the CI wiring" |
155
- | `gates.scheduled` | `checks-ci-wiring` | the commands a schedule runs |
156
- | `gates.lint` | `checks-lint`, `checks-ci-wiring` | the gates `checks-lint` runs when not all apply. See "Gate selection" |
157
- | `commitIdentity.authors` | `checks-commit-identity` | the identities allowed to author and commit. See "Check commit identities" |
158
- | `sources.effect` | `checks-quality` | the paths held to the Effect rules, and the files under them that are not. See "Enforce the Effect rules" |
159
- | `sources.production` | `checks-size-budget`, `checks-quality` | the source the repository ships. See "Hold files to a size budget" |
160
- | `size` | `checks-size-budget` | the line budget, and which production files it holds. See "Hold files to a size budget" |
161
- | `features` | `featureRules`, `checks-feature-owners` | each feature's root, entries, exempt importers and proof. See "Declare feature owners" |
162
- | `changeSignal` | `checks-feature-owners` | `advisory` to list the feature owners a change touches. See "Declare feature owners" |
163
- | `agentRules.on`, `agentRules.off` | agent Rule selection, not the kit | catalogued Rules switched on or off for this repository |
164
- | `docs.pages` | `checks-docs` | the Diátaxis mode of each page, by glob. See "Hold docs to their templates" |
165
-
166
- Every key is optional. The bins decode the file with one Effect
167
- `Schema`, and the package ships `quality.schema.json` emitted from that
168
- schema, so the `$schema` line gives an editor the verdict the bins
169
- reach, save what no JSON Schema can express across two values, which
170
- the bins refuse: a Rule switched both on and off, a feature entry
171
- outside its root, and two features with one name or sharing a root. A key the schema does
172
- not name is refused, not ignored, so a misspelt `sources` cannot switch
173
- the Effect rules off unnoticed. A glob in `sources`
174
- starts at the repository root, names a directory first, uses `*`
175
- only within a segment and `**` only as a whole one, and ends in a file
176
- name with an extension, which are the globs oxlint, the language
177
- service and git all read alike. oxlint matches `*.ts` at any depth
178
- where the other two match it at the root alone, so the schema refuses
179
- it; `**/*.ts` means every depth to all three. The language service
180
- matches nothing for `src/**` and oxlint nothing for `src/lib`, so the
181
- schema refuses both; `src/**/*.ts` and `src/lib/*.ts` say it to all
182
- three.
183
-
184
- Until a later minor release, a repository with no `quality.json` still
185
- has `ciWiring` and `commitIdentity` read from `package.json`, with a
186
- notice on each read. One with both exits 2 until `package.json` drops
187
- them. The keys map one for one:
188
-
189
- | `package.json` | `quality.json` |
190
- | --- | --- |
191
- | `ciWiring.gates` | `gates.ci` |
192
- | `ciWiring.scheduled` | `gates.scheduled` |
193
- | `ciWiring.lintGates` | `gates.lint` |
194
- | `ciWiring.defaultBranch` | `defaultBranch` |
195
- | `commitIdentity` | `commitIdentity` |
196
-
197
- ### Generated fragments
198
-
199
- oxlint and tsc read their own JSON and nothing else, so
200
- `checks-quality generate` writes what `sources.effect` declares into two
201
- fragments at the repository root, and the hand-written configs extend
202
- them. Both fragments are committed:
203
-
204
- ```sh
205
- checks-quality generate
206
- checks-quality --check
207
- ```
208
-
209
- `.oxlintrc.json`:
210
-
211
- ```json
212
- {
213
- "extends": ["./node_modules/@avi2dg/checks/oxlintrc.json", "./oxlintrc.quality.json"],
214
- "plugins": ["typescript", "oxc", "eslint", "import"]
215
- }
216
- ```
217
-
218
- `tsconfig.json`:
219
-
220
- ```json
221
- {
222
- "extends": ["@avi2dg/checks/tsconfig.effect.json", "./tsconfig.quality.json"]
223
- }
224
- ```
225
-
226
- `oxlintrc.quality.json` holds one override: the declared paths as
227
- `files`, the exempt ones as `excludeFiles`, and the kit's Effect rule
228
- block, `presets/effect.oxlint.json`. `tsconfig.quality.json` holds the
229
- language-service override: the same paths as `include`, the exempt ones
230
- as `exclude`, and `presets/effect.language-service.json`. A kit release
231
- that changes a preset reaches the repository through its next
232
- `generate`. A rule only this repository needs stays in its own
233
- `.oxlintrc.json`, whose overrides come after the fragment's and so win.
234
-
235
- `checks-lint` runs `checks-quality --check`, which exits 1 when:
236
-
237
- - a fragment is missing, or differs from what `generate` would write
238
- from `quality.json` and the installed kit's presets;
239
- - a fragment is left over once `quality.json` stops declaring
240
- `sources.effect`;
241
- - `.oxlintrc.json` or `tsconfig.json` does not list its fragment in
242
- `extends`, so the tool never reads it;
243
- - a `sources.effect.paths` glob, or a `sources.production` glob while
244
- `size` is declared, matches no tracked or untracked file, so it holds
245
- nothing.
246
-
247
- It exits 2 when `quality.json` does not decode. `generate` writes the
248
- fragments, removes a left-over one, then runs the same check.
249
-
250
- ```
251
- checks-quality: 2 problem(s) with what quality.json declares:
252
- oxlintrc.quality.json is stale against quality.json and the kit's presets; run checks-quality generate
253
- tsconfig.json does not extend ./tsconfig.quality.json, so the language service never reads it
254
- ```
255
-
256
- Two details of the fragments are easy to get wrong, so the kit's tests
257
- pin both:
258
-
259
- - A fragment sits at the repository root. oxlint resolves an
260
- override's `files`, and the language service an override's `include`,
261
- against the directory of the config holding it. From `.quality/` the
262
- language service reports no error at all on an `async function`
263
- planted under a declared path.
264
- - The oxlint fragment always sets `plugins`, to the kit's. A config in
265
- `extends` that sets none brings in oxlint's default plugins, whose
266
- category rules then fire across the whole tree. The override names
267
- the kit's plugins beside the preset's `node`, `promise` and `unicorn`,
268
- because one that leaves any of the kit's out turns on the category
269
- rules of the plugins it adds under every declared path.
270
-
271
- ## Run every lint gate
26
+ To consume the kit from a repository:
272
27
 
273
- `checks-lint` runs each of the kit's lint gates in turn and names every
274
- one that fails, rather than stopping at the first. A repository whose
275
- tracked files give a gate nothing to check can leave it out through
276
- `gates.lint`. See "Gate selection" under "Check the CI wiring".
28
+ 1. Add the kit and its peers:
277
29
 
278
- | Gate | Reads |
279
- | --- | --- |
280
- | `checks-lint-coverage` | the working tree |
281
- | `checks-test-layout` | the working tree |
282
- | `checks-commit-identity` | the range |
283
- | `checks-comment-gate` | the range |
284
- | `checks-suppressions-ratchet` | the range |
285
- | `checks-ci-wiring` | the working tree |
286
- | `checks-docs` | the range |
287
- | `checks-quality` | the working tree |
288
- | `checks-size-budget` | the range |
289
- | `checks-feature-owners` | the range |
290
-
291
- ```sh
292
- checks-lint
293
- checks-lint <base-ref> <head-ref>
294
- ```
295
-
296
- It resolves the range once and hands the same one to every range gate.
297
- Locally, and on any event other than a pull request, the range ends at
298
- `HEAD` and starts where `HEAD` branched from the origin default branch:
299
- `origin/HEAD`, or when `origin/HEAD` is not set, as in an
300
- `actions/checkout` clone, `origin/<defaultBranch>` from the
301
- repository's `quality.json`, as "Declare policy in the quality file" says, and `origin/main` when
302
- that is not declared. In a GitHub Actions pull request, where
303
- `GITHUB_EVENT_NAME` is `pull_request`, it ends
304
- at the event's head sha and starts where that branched from
305
- `origin/<base branch>`, so GitHub's merge commit is never in it. The
306
- base branch is read from the fetch, not from the event's recorded base
307
- sha, which GitHub leaves stale once the base branch advances after the
308
- pull request opens. Explicit base and head arguments override both.
309
-
310
- The range always starts at the merge base, never at the base branch's
311
- tip: commits the base branch gained after the head branched off would
312
- otherwise count against the head. When the head is the merge base, as
313
- on a push to the default branch or a local run on it, the range would be
314
- empty, so each range gate is handed that tip commit alone and checks it
315
- against its parent, or against the empty tree when it is a repository's
316
- first commit:
317
-
318
- ```
319
- checks-lint: tip 10ba7d8935b73ed72624120a1542e51bd21ca7c7 from HEAD against origin/main
320
- ```
321
-
322
- A clone with no remote-tracking refs at all, such as a freshly
323
- initialised repository with no remote or one whose remote was never
324
- fetched, has no default branch to start from, so each range gate is
325
- handed `HEAD` alone the same way:
326
-
327
- ```
328
- checks-lint: tip 10ba7d8935b73ed72624120a1542e51bd21ca7c7 from HEAD alone, as the clone has no remote-tracking refs
329
- ```
330
-
331
- Once any ref sits under `refs/remotes/`, a missing
332
- `origin/<default branch>` exits 2 instead, since that is the shape of a
333
- shallow CI checkout, where `HEAD` alone would leave the commits before
334
- it unchecked.
335
-
336
- It prints the range, the declared selection if there is one, each
337
- gate's own report, then its verdict:
338
-
339
- ```
340
- checks-lint: range 2504acf098d120e73a8ece3c96f22b934f35c6a8..10ba7d8935b73ed72624120a1542e51bd21ca7c7 from HEAD against origin/main
341
- ...
342
- checks-lint: 3 of 9 gate(s) failed: checks-commit-identity, checks-comment-gate, checks-suppressions-ratchet
343
- ```
344
-
345
- It exits 1 when any gate found a violation, and 2 when the range or the
346
- selection does not resolve, or no failing gate could decide. ci-wiring
347
- always runs, so a repository on `checks-lint` declares `gates.ci`,
348
- as "Check the CI wiring" says, and it holds the test layout unless its selection
349
- leaves out `checks-test-layout`.
350
-
351
- CI runs it through `lint`. The checkout fetches the whole history, which
352
- the merge base needs:
30
+ <!-- generated install: bun run build writes it from package.json and scripts/doc-blocks.ts -->
353
31
 
354
- ```yaml
355
- on:
356
- pull_request:
357
- jobs:
358
- lint:
359
- runs-on: ubuntu-latest
360
- steps:
361
- - uses: actions/checkout@v5
362
- with:
363
- fetch-depth: 0
364
- - uses: oven-sh/setup-bun@v2
365
- - run: bun install --frozen-lockfile
366
- - run: bun run lint
367
- ```
32
+ ```sh
33
+ bun add -d @avi2dg/checks @effect/tsgo@0.45.0 @swc/core@1.16.2 dependency-cruiser@18.4.0 effect@4.0.0-rc.115 oxlint@1.83.0 oxlint-tsgolint@7.0.2002 typescript@7.0.2
34
+ ```
368
35
 
369
- ## Enforce the Effect rules
36
+ <!-- end generated install -->
370
37
 
371
- The base config loads the `effect-channel` plugin and turns on
372
- `effect-channel/no-error-channel-escape`, which refuses `Effect.ignore`,
373
- `Effect.ignoreCause`, the `Effect.catchCause` family, and an
374
- `Effect.catch` whose handler takes no error or names it `_`.
38
+ 1. Extend the oxlint base in `.oxlintrc.json`, restating `plugins`:
375
39
 
376
- Two more rules ship off, because a repository writes only some of its
377
- paths in Effect: code a host loads without `node_modules`, such as a
378
- hook bundle or this oxlint plugin, cannot import it.
40
+ ```json
41
+ {
42
+ "extends": ["./node_modules/@avi2dg/checks/oxlintrc.json"],
43
+ "plugins": ["typescript", "oxc", "eslint", "import"]
44
+ }
45
+ ```
379
46
 
380
- - `effect-channel/no-throw` refuses a `throw` statement.
381
- - `effect-channel/no-try-catch` refuses a `try` statement with a
382
- `catch` clause. `try`/`finally` stays allowed.
47
+ 1. Keep oxlint out of `node_modules/` through `.gitignore`:
383
48
 
384
- Each refusal says what to write instead: a `Schema.TaggedError` failed
385
- through `Effect.fail`, a throwing call wrapped in `Effect.try` or
386
- `Effect.tryPromise`, and recovery by tag with `Effect.catchTag`.
49
+ ```
50
+ node_modules/
51
+ ```
387
52
 
388
- A repository turns them on for the paths it writes in Effect by
389
- declaring those paths in `quality.json` and extending the generated
390
- fragments. See "Generated fragments" under "Declare policy in the quality file":
53
+ 1. Extend the tsconfig fragment in `tsconfig.json`:
391
54
 
392
- ```json
393
- "sources": {
394
- "effect": { "paths": ["src/**/*.ts"], "exempt": ["src/host/*.ts"] }
395
- }
396
- ```
55
+ ```json
56
+ {
57
+ "extends": "@avi2dg/checks/tsconfig.effect.json"
58
+ }
59
+ ```
397
60
 
398
- The fragment's override carries the kit's Effect rule block,
399
- `presets/effect.oxlint.json`: the two rules above, plus `node/no-sync`,
400
- `oxc/no-async-await`, `promise/avoid-new` and `unicorn/no-process-exit`.
401
- Files under `exempt` answer to none of them.
61
+ 1. Declare the commands CI runs in `quality.json` at the repository root:
402
62
 
403
- oxlint resolves `files` against the directory of the config that holds
404
- the override, so a config passed with `-c` from outside the repository
405
- matches nothing and reports nothing.
63
+ ```json
64
+ {
65
+ "$schema": "./node_modules/@avi2dg/checks/quality.schema.json",
66
+ "gates": { "ci": ["bun run lint", "bun run typecheck", "bun run test"] }
67
+ }
68
+ ```
406
69
 
407
- `unicorn/no-process-exit` passes over any file that opens with a
408
- shebang, so a repository whose bins open with one bans `process.exit`
409
- itself with `no-restricted-properties` in its own `.oxlintrc.json`, as
410
- this one does. The preset leaves that rule out because
411
- a repository's own `no-restricted-properties` list for the same files
412
- would replace it, or be replaced by it. Sites standing when the
413
- declaration lands go in oxlint's own baseline, `oxlint --suppress-all`,
414
- so their count can only fall. This repository's own `.oxlintrc.json`
415
- also lifts `effect-channel/no-throw` from `scripts/comment-matchers.ts`,
416
- the one file under its Effect path that a host loads without
417
- `node_modules`.
70
+ 1. Copy the bunfig preset:
418
71
 
419
- The language service holds the same paths to Effect-native IO through
420
- the tsconfig fragment's override, whose severities are
421
- `presets/effect.language-service.json`: `nodeBuiltinImport`,
422
- `asyncFunction`, `newPromise` and `extendsNativeError`, all errors.
423
- effect-tsgo keeps the severities `tsconfig.effect.json` sets when a later
424
- config in `extends` restates the plugin with only its overrides.
72
+ ```sh
73
+ cp node_modules/@avi2dg/checks/bunfig.toml bunfig.toml
74
+ ```
425
75
 
426
- ## Lay out the tests
76
+ 1. Add three scripts to `package.json`:
427
77
 
428
- `checks-test-layout` fails unless the repo holds this shape, and names the
429
- file and the path to move it to when it does not:
78
+ ```json
79
+ "lint": "oxlint --type-aware && checks-lint",
80
+ "typecheck": "tsc --noEmit && effect-tsgo diagnostics --project tsconfig.json --format text --strict",
81
+ "test": "checks-test"
82
+ ```
430
83
 
431
- - Every test file is `tests/**/*.test.ts` or `.tsx`. A `*.test.ts`, `*.spec.ts` or
432
- `*_test.ts` under `src/`, `test/`, `__tests__/` or the repo root fails.
433
- Tracked and untracked files that `git ls-files --exclude-standard`
434
- reports are scanned, so `node_modules/` and every gitignored tree are
435
- out of reach, and a local run agrees with CI before `git add`.
436
- - `tests/lib/**` holds helpers and `tests/fixtures/**` holds data; neither
437
- may hold a test file. Every other directory directly under `tests/` is a
438
- test group and may nest as deep as it likes.
439
- - Two levels. A test outside `tests/e2e/` runs in-process, so it may not
440
- import `node:child_process`, `net`, `http`, `https`, `http2`, `tls` or
441
- `dgram`, may not import `$`, `spawn`, `spawnSync`, `connect`, `serve` or
442
- `listen` from `bun`, may not touch `Bun.$` or `Bun.spawn`, and may not
443
- call `fetch`. A test inside `tests/e2e/` may do all of it. Helpers in
444
- `tests/lib/**` answer to the same rule, since an in-process test reaches
445
- them; `tests/fixtures/**` is data and is not parsed. Detection parses
446
- with swc and reads import specifiers and identifier use, so a test that
447
- only carries `"node:child_process"` as a string is not a violation.
448
- - `scripts.test` is exactly `checks-test`, which runs `bun test --randomize`
449
- as "Run the suite" says, and `scripts.lint` runs this check, itself or
450
- through `checks-lint` called by its bare bin name.
451
- - `bunfig.toml` carries every `[test]` key of the shipped preset with the
452
- same value, and `[test].pathIgnorePatterns` is always
453
- `["**/tests/quarantine/**"]`: the check pins it itself, so this repo,
454
- whose bunfig is the preset, cannot drift it either. Other tables, and
455
- extra `[test]` keys, are the repo's own.
84
+ 1. Run the three on every pull request in `.github/workflows/ci.yml`, fetching the whole history the range needs:
456
85
 
457
- The in-process half is what a mutation run can mutate; `tests/e2e/**` is
458
- excluded from a mutate scope by construction, because a subprocess kills
459
- both the speed and the coverage signal a mutant needs.
86
+ ```yaml
87
+ on:
88
+ pull_request:
89
+ jobs:
90
+ checks:
91
+ runs-on: ubuntu-latest
92
+ steps:
93
+ - uses: actions/checkout@v5
94
+ with:
95
+ fetch-depth: 0
96
+ - uses: oven-sh/setup-bun@v2
97
+ - run: bun install --frozen-lockfile
98
+ - run: bun run lint
99
+ - run: bun run typecheck
100
+ - run: bun run test
101
+ ```
460
102
 
461
- The preset also skips `tests/quarantine/**` on a default run. A test that
462
- turns flaky moves there, so the suite stays trustworthy, and the flake is
463
- still run on demand:
103
+ `bun run lint` then ends with `checks-lint: <count> gate(s) pass`.
464
104
 
465
- ```sh
466
- bun test --path-ignore-patterns='' tests/quarantine
467
- ```
105
+ ## What runs
468
106
 
469
- ## Run the suite
107
+ `checks-lint` runs these gates in this order, each over the working tree or over the range it resolves, and names every one that fails.
108
+ A repository leaves out a gate that does not apply to it through `gates.lint`, as [Gate selection](docs/gates/checks-lint.md#gate-selection) says.
470
109
 
471
- `checks-test` runs the whole suite with `bun test --randomize`, passes
472
- bun's output through, and then reads bun's JUnit report of the same run.
473
- bun exits 0 with tests skipped, so a green run says nothing about the
474
- tests that never ran. `checks-test` fails when a test was skipped, by
475
- `test.skip`, `test.skipIf`, `test.if`, `describe.skip` or `test.todo`,
476
- without a declaration in `package.json`:
110
+ <!-- generated gates: bun run build writes it from KIT_GATES in scripts/gates.ts and scripts/doc-blocks.ts -->
477
111
 
478
- ```json
479
- "testSkips": [
480
- {
481
- "file": "tests/e2e/docker.test.ts",
482
- "test": "images > builds the release image",
483
- "reason": "the runner has no docker daemon",
484
- "when": "ci"
485
- }
486
- ]
487
- ```
488
-
489
- `file` is the path bun reports, relative to the package root, and `test`
490
- is the name bun's console prints: the describe blocks and the test name
491
- joined by ` > `. `reason` is required. `when` is `ci` or `local` for a
492
- test skipped only there, and a declaration without it holds in both;
493
- `checks-test` counts a run as `ci` when `CI` is set true, as GitHub
494
- Actions sets it. A declaration that holds for the run but matches no
495
- skipped test fails a ci run too, so a fixed or renamed test takes its
496
- declaration with it. A local run only warns about it, because whether a
497
- test skips there can hang on the machine, such as a docker daemon being
498
- up:
499
-
500
- ```
501
- checks-test: 1 skipped test(s) undeclared and 1 declaration(s) matching no skipped test in this ci run:
502
- tests/pricing.test.ts:12 pricing > rounds half to even: skipped with no declaration; run it, or declare it in package.json testSkips with its reason
503
- tests/e2e/docker.test.ts > images > builds the release image: declared, but no such test skipped; delete the declaration
504
- ```
505
-
506
- It exits 1 when a test failed or a skip is undeclared or, in a ci run,
507
- a declaration stale, and 2 when `testSkips` does not parse or bun
508
- passed without writing its report. It takes no arguments: a `-t`
509
- filter reports every test it leaves out as skipped and a path filter
510
- drops files a declaration names, so a narrowed run is plain
511
- `bun test --randomize` with the arguments. Files under
512
- `tests/quarantine/` are never run and so never reported. See "Test
513
- layout".
514
-
515
- ## Find flaky tests
516
-
517
- A green run proves nothing failed in that run, not that no test is
518
- flaky. `checks-flake` runs the whole suite several times, each with its
519
- own `--seed`, and records per failing test the seeds it failed with:
520
-
521
- ```sh
522
- checks-flake [--runs <count> | --seed <seed>...] [--report <file>]
523
- ```
524
-
525
- `--runs` defaults to 10 runs on random seeds, and `--seed`, given once
526
- per run, replays chosen seeds, such as the ones a report recorded.
527
- `bun test --randomize --seed=<seed>` puts the suite in the same order,
528
- so a seed reproduces a failure that hangs on order. `--report` writes the
529
- record as JSON, every run's seed and failing tests and every failing
530
- test's seeds, and under GitHub Actions the summary below is appended to
531
- the job summary:
532
-
533
- ```
534
- checks-flake: 3 of 10 run(s) failed, 1 test(s) failing in them
535
-
536
- | Test | Failed | Seeds |
112
+ | Gate | Reads | Runs in |
537
113
  | --- | --- | --- |
538
- | tests/cache.test.ts:6 reads the cache | 3 of 10 runs | 2170533150, 4046124386, 180394251 |
539
-
540
- Reproduce a failing run with bun test --randomize --seed=<seed>.
541
- ```
542
-
543
- A run that fails with no failing test, such as a test file that throws
544
- while loading, is listed with its seed on its own line. It exits 1 when
545
- any run failed and 2 when bun passed without writing its report.
546
-
547
- A consumer runs it on a schedule and keeps the record as an artifact:
548
-
549
- ```yaml
550
- on:
551
- schedule:
552
- - cron: "17 5 * * *"
553
- workflow_dispatch:
554
- jobs:
555
- flake:
556
- runs-on: ubuntu-latest
557
- steps:
558
- - uses: actions/checkout@v5
559
- - uses: oven-sh/setup-bun@v2
560
- - run: bun install --frozen-lockfile
561
- - run: bunx checks-flake --runs 10 --report flake-report.json
562
- - uses: actions/upload-artifact@v4
563
- if: always()
564
- with:
565
- name: flake-report
566
- path: flake-report.json
567
- ```
568
-
569
- and declares the step in `gates.scheduled`, so `checks-ci-wiring`
570
- fails once the schedule stops running it. See "Check the CI wiring".
571
-
572
- ## Enforce dependency rules
573
-
574
- `.dependency-cruiser.cjs` extends the shared base, which carries
575
- `no-circular`, `no-orphans`, `not-to-dev-dep` (shipped source importing
576
- a dev-only package, which a package listed in `peerDependencies` too is
577
- not), `not-to-unresolvable` (nothing installed answers the
578
- specifier), and `no-deep-imports` (a subpath the package's exports map
579
- does not publish):
580
-
581
- ```js
582
- module.exports = {
583
- extends: "./node_modules/@avi2dg/checks/dependency-cruiser.config.js",
584
- forbidden: [
585
- {
586
- name: "ui-cannot-reach-server",
587
- severity: "error",
588
- from: { path: "^src/ui" },
589
- to: { path: "^src/server" },
590
- },
591
- ],
592
- };
593
- ```
594
-
595
- That last block is the layer-boundary recipe: append a named rule per
596
- boundary you own. A rule that restates a base name overrides it field
597
- by field, which is how an entry point stops being an orphan:
598
- redeclare `no-orphans` with your entry added to its `pathNot`.
599
- This repo's own `.dependency-cruiser.cjs` does that for the plugin
600
- entry. A repository that declares feature owners spreads the rules
601
- `quality.json` compiles to into the same `forbidden`. See "Feature
602
- owners".
603
-
604
- `package.json` gains the script:
605
-
606
- ```json
607
- "lint:deps": "depcruise --config .dependency-cruiser.cjs src"
608
- ```
609
-
610
- Enforcement runs in CI beside the other checks:
611
-
612
- ```yaml
613
- jobs:
614
- lint:
615
- steps:
616
- - uses: actions/checkout@v5
617
- - uses: oven-sh/setup-bun@v2
618
- - run: bun install --frozen-lockfile
619
- - run: bun run lint:deps
620
- ```
621
-
622
- ## Lint commit messages
623
-
624
- Commits follow `@commitlint/config-conventional` plus the house
625
- prefixes listed in `commitlint.config.js`, shared from
626
- `@avi2dg/checks/commitlint.config.js`. It arrives with the registry
627
- dependency above, since `@commitlint/cli` and
628
- `@commitlint/config-conventional` are dependencies, not peers.
629
- Enforcement runs in CI on pull requests, because `jj` never fires a
630
- git hook. Add this workflow to the consuming repo:
631
-
632
- ```yaml
633
- on:
634
- pull_request:
635
- types: [opened, edited, synchronize, reopened]
636
- jobs:
637
- commitlint:
638
- runs-on: ubuntu-latest
639
- steps:
640
- - uses: actions/checkout@v5
641
- - uses: oven-sh/setup-bun@v2
642
- - run: bun install --frozen-lockfile
643
- - run: printf '%s' "$PR_TITLE (#0000)" > "$RUNNER_TEMP/pr-title"
644
- env:
645
- PR_TITLE: ${{ github.event.pull_request.title }}
646
- - run: ./node_modules/.bin/commitlint --config ./node_modules/@avi2dg/checks/commitlint.config.js --edit "$RUNNER_TEMP/pr-title"
647
- ```
648
-
649
- It lints the pull request title and nothing else. The title is the
650
- enforced subject because squash merges use the PR title as the main
651
- commit subject; per-commit messages are not linted. GitHub appends
652
- ` (#N)` to the squashed subject, so the workflow lints the title with
653
- that suffix attached and the header length limit applies to the landed
654
- subject, not the bare title.
655
-
656
- It never sees a commit's author or committer fields, nor the
657
- `Co-authored-by` trailer GitHub writes from a foreign author when it
658
- squashes, so it cannot enforce who a commit belongs to. The
659
- commit-identity check below is the enforcement.
660
-
661
- ## Check commit identities
662
-
663
- `scripts/commit-identity.ts` walks every commit in a range and fails when
664
- one carries an identity other than the repository owner's:
665
-
666
- ```sh
667
- bun run checks-commit-identity <base-ref> <head-ref>
668
- bun run checks-commit-identity <ref>
669
- ```
670
-
671
- With one argument it checks that commit alone. It refuses a commit
672
- whose author or committer is outside the allowlist, and one whose
673
- trailer block carries a `Co-authored-by:` trailer as git parses it, and
674
- it names the offending commit and reason. Other trailers and prose mentioning an address in the
675
- body are left alone. `GitHub <noreply@github.com>` is allowed as
676
- committer only, since that is who writes a squash merge.
677
-
678
- The allowlist defaults to `avi2d <avi2dg@gmail.com>`. A repo with other
679
- owners restates it in `quality.json`:
680
-
681
- ```json
682
- "commitIdentity": {
683
- "authors": [{ "name": "avi2d", "email": "avi2dg@gmail.com" }]
684
- }
685
- ```
686
-
687
- `checks-lint` runs it over each pull request's range.
688
- See "Run every lint gate".
689
-
690
- ## Refuse banned comments
691
-
692
- `scripts/comment-gate.ts` runs the comment check over a diff and fails
693
- when an added line carries a banned comment:
114
+ | [`checks-lint-coverage`](docs/gates/checks-lint-coverage.md) | the working tree | a repository tracking `*.ts` or `*.tsx` |
115
+ | [`checks-test-layout`](docs/gates/checks-test-layout.md) | the working tree | a repository tracking `*.ts` or `*.tsx` |
116
+ | [`checks-commit-identity`](docs/gates/checks-commit-identity.md) | the range | every repository |
117
+ | [`checks-comment-gate`](docs/gates/checks-comment-gate.md) | the range | every repository |
118
+ | [`checks-suppressions-ratchet`](docs/gates/checks-suppressions-ratchet.md) | the range | every repository |
119
+ | [`checks-ci-wiring`](docs/gates/checks-ci-wiring.md) | the working tree | every repository |
120
+ | [`checks-docs`](docs/gates/checks-docs.md) | the range | every repository |
121
+ | [`checks-quality`](docs/gates/checks-quality.md) | the working tree | a repository tracking `quality.json` |
122
+ | [`checks-size-budget`](docs/gates/checks-size-budget.md) | the range | a repository tracking `*.ts` or `*.tsx` |
123
+ | [`checks-feature-owners`](docs/gates/checks-feature-owners.md) | the range | a repository tracking `*.ts` or `*.tsx` |
694
124
 
695
- ```sh
696
- bun run checks-comment-gate <base-ref> <head-ref>
697
- bun run checks-comment-gate <ref>
698
- ```
125
+ <!-- end generated gates -->
699
126
 
700
- With two arguments it diffs the base against the head. With one it diffs
701
- that commit against its parent, or against the empty tree for a
702
- repository's first commit, which has none. It exits 2 when the parent
703
- exists but is not in the clone rather than widening to the whole tree,
704
- so a shallow checkout running the one-argument form needs
705
- `fetch-depth: 2`. Only added lines
706
- are checked, so a violation in a file the diff never touches stays
707
- silent, and a refusal counts when any line of the comment carrying it was
708
- added. The check refuses a machine-read directive, a record or ticket
709
- pointer, a doc block, and a file opening with a rationale block over
710
- three lines, licence headers excepted.
127
+ These bins run on their own:
711
128
 
712
- `scripts/comment-matchers.ts` holds the scanner, the comment syntaxes
713
- and a synchronous `refused()`, and imports nothing, so a host such as a
714
- hook bundle can copy it alone into a directory with no `node_modules`
715
- and import it as `@avi2dg/checks/scripts/comment-matchers.ts`. The repo's
716
- cruise fails when it gains an import. `scripts/comments.ts` wraps the
717
- same matchers in Effect for the gate and the backtest.
129
+ - [`checks-test`](docs/gates/checks-test.md) runs the suite as `scripts.test` and refuses a skip the repository has not declared.
130
+ - [`checks-flake`](docs/gates/checks-flake.md) runs the suite on a schedule and records the seeds a flaky test fails with.
131
+ - [`checks-mutation-compare`](docs/gates/checks-mutation-compare.md) holds a pull request's mutation score to no regression.
132
+ - [`checks-backtest`](docs/gates/checks-backtest.md) reports what the comment check would have refused in recent history.
718
133
 
719
- `checks-lint` runs it over each pull request's range.
720
- See "Run every lint gate".
134
+ `checks-lint` has [its own page](docs/gates/checks-lint.md), which says which range it resolves.
135
+ The oxlint base, the dependency-cruiser base and the commitlint config run through their own tools, as the pages under Related topics say.
721
136
 
722
- ## Ratchet the suppressions
137
+ ## Upgrade
723
138
 
724
- `scripts/suppressions-ratchet.ts` holds oxlint's bulk-suppression
725
- baseline, `oxlint-suppressions.json`, to counts that only fall. oxlint
726
- accepts whatever `oxlint --suppress-all` writes, so raising a count to
727
- let a new site through passes the lint. The ratchet reads the file at a
728
- base and a head and fails naming each file and rule whose count rose or
729
- that appeared:
139
+ To move a repository to a newer release of the kit:
730
140
 
731
- ```sh
732
- bun run checks-suppressions-ratchet <base-ref> <head-ref>
733
- bun run checks-suppressions-ratchet <ref>
734
- ```
141
+ 1. Run the install line again, which moves the kit to its newest release and the peers to the versions it pins.
142
+ 1. Run `bun run checks-quality generate` when `quality.json` declares `sources.effect`, since a release that changes a preset reaches the fragments only through it.
143
+ 1. Copy `node_modules/@avi2dg/checks/bunfig.toml` over `bunfig.toml` again, since `checks-test-layout` compares the copy with the installed preset.
144
+ 1. Run `bun run lint`, `bun run typecheck` and `bun run test`.
735
145
 
736
- With one argument it compares that commit with its parent, or with the
737
- empty tree for a repository's first commit, and exits 2 when the parent
738
- exists but is not in the clone. With two it reads the base where
739
- the head branched off, at their merge-base, so a count the base branch
740
- lowered since does not read as a rise at the head. A count that fell and
741
- an entry that went both pass. A commit without the file counts as empty,
742
- so the commit that first adds a baseline fails with every entry
743
- appearing. The file is read from the directory the command runs in,
744
- which is where oxlint writes it. It exits 2 when a ref does not resolve
745
- or the file is not oxlint's count per rule per file:
146
+ The repository's lockfile pins the kit, so a repository moves only when it runs these steps.
147
+ [CHANGELOG.md](CHANGELOG.md), shipped in the package, lists what each release changed.
148
+ A repository that still declares `ciWiring` or `commitIdentity` in `package.json` moves them into `quality.json`, as [Keys moved from package.json](docs/configs/quality-file.md#keys-moved-from-packagejson) maps.
746
149
 
747
- ```
748
- suppressions-ratchet: 2 count(s) in oxlint-suppressions.json rose or appeared; fix the site instead of suppressing it:
749
- src/added.ts typescript/no-unsafe-type-assertion appeared with 1
750
- src/dispatch.ts typescript/no-non-null-assertion rose from 12 to 13
751
- ```
752
-
753
- `checks-lint` runs it over each pull request's range.
754
- See "Run every lint gate".
755
-
756
- ## Hold files to a size budget
757
-
758
- `checks-size-budget` holds production files to the line budget
759
- `quality.json` declares, and lists every other file over it without
760
- failing:
761
-
762
- ```json
763
- "sources": { "production": ["src/**/*.ts"] },
764
- "size": { "fileLines": 400, "functionLines": 100, "applies": "changed" }
765
- ```
766
-
767
- ```sh
768
- checks-size-budget <base-ref> <head-ref>
769
- checks-size-budget <ref>
770
- ```
771
-
772
- It runs oxlint with a configuration of two rules and nothing else:
773
- `max-lines` at `fileLines` and `max-lines-per-function` at
774
- `functionLines`, both counting blank and comment lines. With `applies`
775
- set to `changed` it holds the files under `sources.production` that the
776
- range adds or changes, a rename that edits the file included. With
777
- `all` it holds every file under `sources.production`. A file the range
778
- deletes or only renames is not held. Every other tracked `.ts` or
779
- `.tsx` file over the budget, tests and unchanged production files alike,
780
- is listed as advisory and never fails the gate; `.d.ts` files are not
781
- measured.
782
-
783
- It reads each file from the head commit rather than the working tree,
784
- so an uncommitted edit neither fails nor passes a range, and a pull
785
- request's merge checkout measures what the pull request holds. With two
786
- arguments the range starts where the head branched from the base, at
787
- their merge-base. With one it is that commit against its parent, or
788
- against the empty tree for a repository's first commit. oxlint must be
789
- on `PATH`, as it is under a package script.
790
-
791
- ```
792
- size-budget: 1 overrun(s) of 400 lines per file and 100 per function in the production files the range adds or changes:
793
- src/billing/ledger.ts:12: The function `settle` has too many lines (131). Maximum allowed is 100.
794
- size-budget: advisory, 1 overrun(s) where the budget does not hold yet:
795
- tests/e2e/billing.test.ts: File has too many lines (512).
796
- ```
797
-
798
- It exits 1 on an overrun in a file it holds, and 2 when `quality.json`
799
- does not decode, a ref does not resolve or oxlint cannot run. A
800
- repository that declares no `size` passes. `quality.json` refuses a
801
- `size` without `sources.production`, which would hold nothing, and
802
- with `size` declared `checks-quality` refuses a `sources.production`
803
- glob that matches no file. Moving `applies` from `changed` to `all`
804
- tightens the budget to every production file, once the advisory list
805
- names none.
806
-
807
- `checks-lint` runs it over each pull request's range.
808
- See "Run every lint gate".
809
-
810
- ## Declare feature owners
811
-
812
- A repository opts a feature in by declaring, in `quality.json`, the
813
- directory it owns, the files code outside it imports it through, the
814
- files that may reach past those, and the end-to-end test that proves it
815
- runs:
816
-
817
- ```json
818
- "features": [
819
- {
820
- "name": "billing",
821
- "root": "src/billing",
822
- "entries": ["src/billing/index.ts"],
823
- "allowFrom": ["src/main.ts", "src/cli/*.ts"],
824
- "proof": "tests/e2e/billing.test.ts"
825
- }
826
- ],
827
- "changeSignal": "advisory"
828
- ```
829
-
830
- `root` is a directory and `entries` are files under it, both without
831
- globs. `allowFrom` holds globs of the same shape as `sources`. `proof`
832
- is a `.test.ts` or `.test.tsx` file under `tests/e2e/`. Nothing moves:
833
- a root is wherever the feature already lives. `quality.json` refuses an
834
- entry outside its root, a name used twice, and two features sharing a
835
- root or one root inside another, so a file has at most one owner.
836
-
837
- ### Import boundary
838
-
839
- `dist/feature-rules.js` compiles `features` into one dependency-cruiser
840
- rule per feature, which `.dependency-cruiser.cjs` spreads beside its
841
- own:
842
-
843
- ```js
844
- const { featureRules } = require("@avi2dg/checks/dist/feature-rules.js");
845
-
846
- module.exports = {
847
- extends: "./node_modules/@avi2dg/checks/dependency-cruiser.config.js",
848
- forbidden: [...featureRules(require("./quality.json"))],
849
- };
850
- ```
851
-
852
- A module outside a feature's root that imports a file inside it must
853
- import one of the feature's `entries`. Modules under `tests/` and the
854
- files `allowFrom` matches, such as a CLI or a harness, may import any
855
- file in it:
856
-
857
- ```
858
- error feature-billing-entries: src/report.ts → src/billing/charge.ts
859
- ```
860
-
861
- `featureRules` decodes its argument with the schema the bins use and
862
- throws the schema's refusal when it does not decode, which stops the
863
- cruise. It is an ES module, as `effect` is, so a `.cjs` config loads it
864
- through `require`, which needs node 20.19, 22.12 or later.
865
-
866
- ### Change signal and proof
867
-
868
- `checks-feature-owners` reads the same declaration over a range:
869
-
870
- ```sh
871
- checks-feature-owners <base-ref> <head-ref>
872
- checks-feature-owners <ref>
873
- ```
874
-
875
- It exits 1 when a feature's proof cannot prove it: the proof or an
876
- entry is not in the head commit, the proof does not parse, or it
877
- imports none of the feature's entries. An import counts when it is a
878
- runtime `import`, `export ... from` or `export * from` of a relative
879
- path that names an entry: by its own name, by the `.js`, `.jsx`, `.mjs`
880
- or `.cjs` spelling of it, `.js` naming a `.tsx` entry as well as a
881
- `.ts` one, or without an extension, the way a directory `index` is
882
- imported. `import type` does not count, and neither does a
883
- path alias. The proof runs in `bun run test` like any end-to-end test,
884
- which is what shows it passes.
885
-
886
- ```
887
- feature-owners: 1 problem(s) with the features' runnable proofs:
888
- billing: proof tests/e2e/billing.test.ts imports none of its entries, src/billing/index.ts
889
- ```
890
-
891
- With `changeSignal` set to `advisory` it also lists each owner the
892
- range touches, with the paths it touched under the owner's root or at
893
- its proof, and still exits 0. Whether a change that spans owners is one
894
- coherent slice is for a reviewer to judge. A rename counts at both of
895
- its paths:
896
-
897
- ```
898
- feature-owners: advisory, the range touches 2 feature owner(s); a reviewer judges whether they make one slice:
899
- billing: src/billing/charge.ts, src/billing/tax.ts
900
- invoices: src/invoices/tax.ts, tests/e2e/invoices.test.ts
901
- ```
902
-
903
- It exits 2 when `quality.json` does not decode, which is where a proof
904
- outside `tests/e2e/` is refused, or a ref does not resolve. A
905
- repository that declares no feature passes, and `quality.json` refuses
906
- a `changeSignal` without features, which would map a change to no
907
- owner.
908
-
909
- `checks-lint` runs it over each pull request's range.
910
- See "Run every lint gate".
911
-
912
- ## Hold docs to their templates
913
-
914
- `checks-docs` holds each doc file a change touches to the template for its kind, and lists every other doc file that does not conform yet without failing.
915
- The package ships one template per kind under `templates/`, and a repository starts a new doc file by copying one:
916
-
917
- ```sh
918
- cp node_modules/@avi2dg/checks/templates/how-to.md docs/add-a-supplier.md
919
- ```
920
-
921
- | File | Kind | Template |
922
- | --- | --- | --- |
923
- | `README.md` | readme | `templates/readme.md` |
924
- | `CHANGELOG.md` | changelog | `templates/changelog.md` |
925
- | `AGENTS.md` | agents | `templates/agents.md` |
926
- | `CLAUDE.md` | claude | `templates/claude.md` |
927
- | each file in `docs/adr/` but its generated index, `README.md` | adr | `templates/adr.md` |
928
- | a page `docs.pages` declares | tutorial, how-to, reference or explanation | `templates/<mode>.md` |
929
-
930
- The first four are the files at the repository root.
931
- No other Markdown file is judged, save a page under `docs/`, which needs a mode.
932
- Which Diátaxis mode a page is written in is a judgment, so `quality.json` declares it:
933
-
934
- ```json
935
- "docs": {
936
- "pages": {
937
- "reference": ["docs/gates/*.md"],
938
- "explanation": ["docs/design.md"]
939
- }
940
- }
941
- ```
942
-
943
- A template decides a file's structure, and the template file itself is the reference for each kind:
944
-
945
- - A file opens with one `# ` title on its first line and has text before its first section.
946
- It skips no heading level, and no heading is Overview, Introduction or How it works.
947
- - Its sections are the template's headings in the template's order.
948
- A heading in angle brackets is one the writer names.
949
- One marked verb first is left to review, since no program tells a verb from a noun there.
950
- A heading the template does not have, in that place, is refused.
951
- - A record in `docs/adr/` is named for its four-digit number, and its title opens with the same number.
952
- A `Date: YYYY-MM-DD` line follows the title, the first word under Status is Proposed, Accepted, Rejected, Deprecated, Superseded or Retired, and no other record holds its number.
953
- - A changelog lists its releases newest first, each opening with a `Released YYYY-MM-DD.` line.
954
- - A how-to or tutorial page numbers its steps.
955
- - `CLAUDE.md` is its template word for word.
956
-
957
- ```sh
958
- checks-docs <base-ref> <head-ref>
959
- checks-docs <ref>
960
- ```
961
-
962
- It reads each file from the head commit.
963
- With two arguments the range starts where the head branched from the base, at their merge-base.
964
- With one it is that commit against its parent, or against the empty tree for a repository's first commit.
965
- A file the range adds, changes or renames is held to its template, and a file it deletes is not.
966
-
967
- ```
968
- docs: 2 violation(s) in the doc files the range touches:
969
- README.md:1: lacks `## Where things are`
970
- docs/parts.md: is a page under docs/ with no mode; declare it under docs.pages in quality.json as tutorial, how-to, reference, explanation
971
- docs: advisory, 1 doc file(s) the range leaves alone do not hold to their templates yet:
972
- docs/adr/0001-quality-gates.md: 5 violation(s)
973
- ```
974
-
975
- It exits 1 on a violation in a file the range touches, and 2 when `quality.json` does not decode or a ref does not resolve.
976
-
977
- `checks-lint` runs it over each pull request's range.
978
- See "Run every lint gate".
979
-
980
- ## Check the CI wiring
981
-
982
- `checks-ci-wiring` fails when a command the repository's CI must run no
983
- longer runs on pull requests to the default branch. No local check sees
984
- that: a workflow whose lint step became a no-op leaves `bun run lint`
985
- green.
986
-
987
- The repository declares its gates once, in `quality.json`:
988
-
989
- ```json
990
- "gates": {
991
- "ci": ["bun run lint", "bun run typecheck", "bun run test"]
992
- }
993
- ```
994
-
995
- and `package.json` runs the check through `checks-lint`:
996
-
997
- ```json
998
- "scripts": {
999
- "lint": "oxlint --type-aware && checks-lint"
1000
- }
1001
- ```
1002
-
1003
- It parses every `.github/workflows/*.yml` and `*.yaml` and looks, for
1004
- each gate, for a `run:` step that is the gate command alone on one line,
1005
- optionally followed by plain arguments: words, quoted strings, and
1006
- `$VAR` or `${VAR}` expansions. `bun run lint --quiet` and
1007
- `bunx checks-comment-gate "origin/$BASE_REF" "$HEAD_SHA"` count;
1008
- `bun run lint:deps`, `echo bun run lint` and a step `name:` do not. A
1009
- step whose script has a second line, or any `|`, `||`, `&&`, `;`, `&`,
1010
- `$(...)`, backticks, `<` or `>` redirection, a comment or a leading
1011
- `NAME=value`, never counts, because each can run the gate without its
1012
- failure failing the step; the report names the gate and says to give it
1013
- its own step with nothing else in it. A gate step counts only when:
1014
-
1015
- - its workflow triggers on `pull_request`, any `branches` or
1016
- `branches-ignore` filter there keeps the default branch, any `types`
1017
- filter keeps `opened` and `synchronize`, and it sets no `paths` or
1018
- `paths-ignore` filter, which lets some pull requests skip the gate;
1019
- - neither the step nor its job sets `if: false` or
1020
- `continue-on-error: true`, bare or as `${{ false }}` and `${{ true }}`;
1021
- - its job needs no job, directly or through a chain, that sets
1022
- `if: false`, unless a job on that chain has an `if:` calling
1023
- `always()`, `failure()` or `cancelled()`. GitHub prefixes every other
1024
- `if:`, including `true` and `success()`, with `success()`, so a job
1025
- whose needed job was skipped is skipped too.
1026
-
1027
- A job calling a local reusable workflow (`uses: ./.github/workflows/x.yml`)
1028
- passes its own trigger and `if:` down to the called workflow's steps.
1029
- A remote reusable workflow (`uses: owner/repo/...@ref`) is not a
1030
- supported way to wire a gate: it is not read, so a gate must run as a
1031
- `run:` step, such as `bunx checks-comment-gate`, in the repo's own
1032
- workflows.
1033
-
1034
- A step running `checks-lint` also counts for a declared gate that calls
1035
- one of the gates `checks-lint` runs by its bare bin name, when the step
1036
- calls `checks-lint` the same way: `bunx checks-lint` counts for
1037
- `bunx checks-comment-gate "origin/$BASE_REF" "$HEAD_SHA"`. A step running `bun run lint` counts only for the `bun run lint` gate,
1038
- since the check never reads what a package script runs. So once `lint`
1039
- runs `checks-lint`, the per-gate entries can leave `gates.ci`
1040
- along with the workflows that ran them.
1041
-
1042
- The default branch is `main`; a repo with another one sets
1043
- `"defaultBranch"` in `quality.json`, which `checks-lint` also reads when
1044
- `origin/HEAD` is not set.
1045
-
1046
- It exits 1 naming each gap, with every step that runs the gate and why
1047
- that step does not count:
1048
-
1049
- ```
1050
- ci-wiring: 1 of 8 gate(s) do not run on pull requests to main:
1051
- bun run lint
1052
- .github/workflows/release.yml job publish step 7: .github/workflows/release.yml does not trigger on pull_request
1053
- ```
1054
-
1055
- It exits 2 when `quality.json` declares no `gates.ci` or does not
1056
- decode, a gate or scheduled command is not one plain command, or a
1057
- workflow does not parse. Whether a workflow is well formed is
1058
- actionlint's question, not this one's.
1059
-
1060
- A command a schedule must run, such as the flake run, goes in
1061
- `gates.scheduled`:
1062
-
1063
- ```json
1064
- "gates": {
1065
- "ci": ["bun run lint", "bun run typecheck", "bun run test"],
1066
- "scheduled": ["bunx checks-flake --runs 10 --report flake-report.json"]
1067
- }
1068
- ```
1069
-
1070
- Each counts only as a step of the same plain shape in a workflow whose
1071
- `on` carries `schedule` with at least one `cron`, under the same
1072
- `if: false`, `continue-on-error: true` and `needs` rules as a gate. It
1073
- exits 1 naming each one no schedule runs:
1074
-
1075
- ```
1076
- ci-wiring: 1 of 1 scheduled command(s) do not run on a schedule:
1077
- bunx checks-flake --runs 10 --report flake-report.json
1078
- .github/workflows/ci.yml job checks step 5: .github/workflows/ci.yml does not trigger on a schedule
1079
- ```
1080
-
1081
- ### Gate selection
1082
-
1083
- A repository with no TypeScript source gives `checks-lint-coverage`,
1084
- `checks-test-layout`, `checks-size-budget` and `checks-feature-owners`
1085
- nothing to check, and test-layout still refuses its
1086
- missing `bun test` script and `bunfig.toml`. It declares the gates
1087
- `checks-lint` runs as `gates.lint`:
1088
-
1089
- ```json
1090
- "gates": {
1091
- "ci": ["bun run lint"],
1092
- "lint": [
1093
- "checks-commit-identity",
1094
- "checks-comment-gate",
1095
- "checks-suppressions-ratchet",
1096
- "checks-ci-wiring",
1097
- "checks-docs",
1098
- "checks-quality"
1099
- ]
1100
- }
1101
- ```
1102
-
1103
- `checks-lint` runs exactly those, in the "Run every lint gate" table's
1104
- order, and all ten when `gates.lint` is absent. A selection in
1105
- `quality.json` always keeps `checks-quality`, since the file it sits in
1106
- is what makes that gate apply. A step running
1107
- `checks-lint` then counts only for a declared gate that `gates.lint`
1108
- keeps.
1109
-
1110
- A selection may leave out only a gate that does not apply:
1111
-
1112
- | Gate | Applies when the repository |
1113
- | --- | --- |
1114
- | `checks-lint-coverage` | tracks a `.ts` or `.tsx` file |
1115
- | `checks-test-layout` | tracks a `.ts` or `.tsx` file |
1116
- | `checks-commit-identity` | always |
1117
- | `checks-comment-gate` | always |
1118
- | `checks-suppressions-ratchet` | always |
1119
- | `checks-ci-wiring` | always |
1120
- | `checks-docs` | always |
1121
- | `checks-quality` | tracks a `quality.json` |
1122
- | `checks-size-budget` | tracks a `.ts` or `.tsx` file |
1123
- | `checks-feature-owners` | tracks a `.ts` or `.tsx` file |
1124
-
1125
- Both bins exit 2 on a `gates.lint` that names an unknown gate or leaves
1126
- out one that always applies. ci-wiring exits 1 when the
1127
- selection leaves out a gate the repository's tracked files make
1128
- applicable, and names the gate and the files:
1129
-
1130
- ```
1131
- ci-wiring: quality.json gates.lint leaves out 4 gate(s) this repository's contents make applicable:
1132
- checks-lint-coverage: the repository tracks TypeScript source (src/widget.ts)
1133
- checks-test-layout: the repository tracks TypeScript source (src/widget.ts)
1134
- checks-size-budget: the repository tracks TypeScript source (src/widget.ts)
1135
- checks-feature-owners: the repository tracks TypeScript source (src/widget.ts)
1136
- ```
1137
-
1138
- It reads the files tracked at the checkout, so the pull request that
1139
- adds the first TypeScript file is the one refused.
1140
-
1141
- ### Limits
1142
-
1143
- The check reads workflow files and never runs them, so it deliberately
1144
- does not evaluate:
1145
-
1146
- - an `if:` expression other than a constant `true` or `false`, which
1147
- counts as running;
1148
- - a `strategy.matrix` `include` or `exclude`, so a matrix that drops
1149
- every combination still counts as running its steps;
1150
- - a remote reusable workflow (`uses: owner/repo/...@ref`), whose steps
1151
- are never read;
1152
- - anything that happens at run time on the runner: what the gate
1153
- command itself does, the shell's options, and a step or job that
1154
- fails or times out before the gate step.
1155
-
1156
- ## Backtest the comment check
1157
-
1158
- `scripts/backtest.ts` reports what the comment check would have refused
1159
- at each recent commit, so a repository can measure its own history:
1160
-
1161
- ```sh
1162
- bun run checks-backtest [commit-count]
1163
- ```
1164
-
1165
- It walks the last `commit-count` first-parent commits, 60 by default,
1166
- prints a row per commit that touches code and then the totals,
1167
- attributes only the refusals each commit introduced, and counts new
1168
- comment text as a share of added lines.
1169
- `generated/`, `vendor/`, `repos/`, `node_modules/` and `dist/` are out
1170
- of reach, so the figures are authored code.
1171
-
1172
- ## Compare mutation scores
1173
-
1174
- `checks-mutation-compare` gates a pull request on no-regression rather
1175
- than an absolute threshold: the head mutation score may not fall below
1176
- the score at the merge-base.
1177
-
1178
- ```sh
1179
- checks-mutation-compare <base-report> <head-report> [--advisory]
1180
- ```
1181
-
1182
- Both reports are Stryker `mutation.json` files. It prints the overall
1183
- score of each report and the per-file scores that differ, and exits 1
1184
- when the head score is below the base score. The score is Stryker's:
1185
- `Killed` and `Timeout` over those plus `Survived` and `NoCoverage`, so
1186
- `CompileError`, `RuntimeError`, `Ignored` and `Pending` mutants leave
1187
- it. Every file in a report counts toward that report's score, including
1188
- files present in only one of the two.
1189
-
1190
- `--advisory` prints the same verdict and always exits 0. That is how
1191
- consumers run it for the first month; after that they drop the flag and
1192
- it blocks.
1193
-
1194
- Enforcement runs on pull requests, comparing the head report against a
1195
- report built at the merge-base:
1196
-
1197
- ```yaml
1198
- jobs:
1199
- mutation-compare:
1200
- runs-on: ubuntu-latest
1201
- steps:
1202
- - uses: actions/checkout@v5
1203
- with:
1204
- fetch-depth: 0
1205
- - uses: oven-sh/setup-bun@v2
1206
- - run: bun install --frozen-lockfile
1207
- - run: bunx stryker run
1208
- - run: |
1209
- base="$(git merge-base HEAD origin/main)"
1210
- git worktree add /tmp/mutation-base "$base"
1211
- (cd /tmp/mutation-base && bun install --frozen-lockfile && bunx stryker run)
1212
- - run: bun run checks-mutation-compare --advisory /tmp/mutation-base/reports/mutation/mutation.json reports/mutation/mutation.json
1213
- ```
1214
-
1215
- The shared Stryker preset's `json` reporter writes
1216
- `reports/mutation/mutation.json` in each worktree.
1217
-
1218
- ## Develop
1219
-
1220
- ```sh
1221
- bun install
1222
- bun run build
1223
- bun run lint
1224
- bun run typecheck
1225
- bun run test
1226
- ```
1227
-
1228
- Release by tagging a commit on `main`. The `release` workflow refuses
1229
- a tag off `main` and reruns the build, `dist/` check, lint, typecheck
1230
- and tests before it publishes:
1231
-
1232
- ```sh
1233
- tag="v$(bun -p 'require("./package.json").version')"
1234
- git tag "$tag" && git push origin "$tag"
1235
- ```
1236
-
1237
- The `release` workflow publishes the tagged version through npm
1238
- trusted publishing, so the tag and `package.json` must agree. No
1239
- token is stored anywhere: GitHub mints the publish credential for
1240
- each run.
1241
-
1242
- npm attaches a trusted publisher only to a package that already
1243
- exists, so the first version goes out by hand: `npm publish` from the
1244
- tagged commit as `avi2dg`, then add the trusted publisher (repository
1245
- `avi2d/checks`, workflow `release.yml`) in the package's npm settings.
1246
- That first tag's `release` run fails on the already-published version;
1247
- every later tag publishes through the workflow.
150
+ ## Where things are
1248
151
 
1249
- `publishConfig.access` in package.json is what makes the scoped package
1250
- public.
152
+ Every path is relative to the installed package, `node_modules/@avi2dg/checks/`.
1251
153
 
1252
- ## Where things are
154
+ <!-- generated shipped: bun run build writes it from package.json and scripts/doc-blocks.ts -->
1253
155
 
1254
156
  | Path | What it holds |
1255
157
  | --- | --- |
1256
- | `scripts/` | every bin, and the modules they share |
1257
- | `effect-channel/` | the Effect error-channel oxlint plugin |
1258
- | `dist/` | the committed bundles of the plugin and of `featureRules` |
1259
- | `presets/` | the Effect presets `checks-quality` builds its fragments from |
1260
- | `templates/` | one template per kind of doc file, which `bun run build` renders |
1261
- | `tests/` | the suite, with the tests that spawn a process under `tests/e2e/` |
1262
- | `docs/` | pages for whoever develops the kit |
1263
- | the root configs | `oxlintrc.json`, `tsconfig.effect.json`, `bunfig.toml`, `commitlint.config.js`, `dependency-cruiser.config.js`, `stryker.preset.js` and `quality.schema.json`, which a consuming repository extends or copies |
158
+ | `CHANGELOG.md` | every release, and what it changed |
159
+ | `CONTRIBUTING.md` | how this repository is developed and released |
160
+ | `docs/` | a reference page per bin and per shared config, and why the kit is shaped this way |
161
+ | `bunfig.toml` | the bunfig preset a repository copies |
162
+ | `commitlint.config.js` | the shared commitlint config |
163
+ | `dependency-cruiser.config.js` | the shared dependency-cruiser base |
164
+ | `scripts/` | every bin, which a package script calls by its `checks-` name |
165
+ | `templates/` | one template per kind of doc file, which a new doc file starts from |
166
+ | `presets/` | the Effect rule blocks `checks-quality generate` writes into the fragments |
167
+ | `quality.schema.json` | the schema of `quality.json`, which its `$schema` line names |
168
+ | `oxlintrc.json` | the oxlint base config `.oxlintrc.json` extends |
169
+ | `stryker.preset.js` | the Stryker mutation-testing preset |
170
+ | `tsconfig.effect.json` | the tsconfig fragment with the Effect language-service block |
171
+ | `dist/` | the compiled Effect error-channel plugin and `featureRules` |
172
+
173
+ <!-- end generated shipped -->
1264
174
 
1265
175
  ## Related topics
1266
176
 
177
+ - [The quality file](docs/configs/quality-file.md)
178
+ - [The Effect rules](docs/configs/effect-rules.md)
179
+ - [The dependency rules](docs/configs/dependency-rules.md)
180
+ - [The commit message lint](docs/configs/commit-messages.md)
1267
181
  - [Why it is shaped this way](docs/design.md)
182
+ - [Contribute to checks](CONTRIBUTING.md)