@avi2dg/checks 0.42.0 → 0.43.0

package/CHANGELOG.md

@@ -2,6 +2,14 @@
 
 Every release of `@avi2dg/checks`, newest first, written by the release from its conventional commits.
 
+## 0.43.0
+
+Released 2026-10-17.
+
+### Features
+
+- **testing:** run checks-test files in parallel on opt-in [#158](https://github.com/avi2d/checks/pull/158)
+
 ## 0.42.0
 
 Released 2026-10-14.

package/docs/gates/checks-test-layout.md

@@ -20,12 +20,13 @@ It fails unless the repository holds this shape, and names the file and the path
   Bun ignores both directories in the default suite.
   A live tier with test files requires `test:live` set to `checks-test --tier=live`.
   A pixel tier with test files requires `test:pixel` set to `checks-test --tier=pixel`.
+  Either tier script may add `--parallel=<workers>`.
 - A test runs in-process or in a process of its own.
   A test outside the `e2e`, `live` and `pixel` levels runs in-process, quarantined or not, so it may not import `node:child_process`, `net`, `http`, `https`, `http2`, `tls` or `dgram`.
   It may not import `$`, `spawn`, `spawnSync`, `connect`, `serve` or `listen` from `bun`, may not touch `Bun.$` or `Bun.spawn`, and may not call `fetch`.
   A test at the `e2e`, `live` or `pixel` level may do all of it, under `tests/quarantine/` as well.
   Helpers in `tests/lib/**` answer to the same rule, since an in-process test reaches them.
-- `scripts.test` is exactly `checks-test`, which runs `bun test --randomize` as [checks-test](checks-test.md) says.
+- `scripts.test` is `checks-test`, optionally with `--parallel=<workers>`, which runs `bun test --randomize` as [checks-test](checks-test.md) says.
 - `scripts.lint` runs this check, itself or through `checks-lint` called by its bare bin name.
 - `bunfig.toml` carries every `[test]` key of the shipped preset with the same value.
   `[test].pathIgnorePatterns` is the preset's `["**/tests/quarantine/**", "**/tests/live/**", "**/tests/pixel/**", "repos/**"]`, which the check pins itself, so the kit's own repository, whose bunfig is the preset, cannot drift it either.
@@ -78,7 +79,7 @@ It checks the directory it runs in, or the directory it is given.
 ```
 test-layout: 4 violation(s)
   src/a.test.ts: a test file must live at tests/<level>/**/*.test.ts, or tests/quarantine/<level>/**/*.test.ts while quarantined, with unit, e2e, live, or pixel as the level; move it to tests/unit/a.test.ts
-  package.json: scripts.test must be exactly "checks-test", which runs bun test --randomize and judges its skips, found "bun test"
+  package.json: scripts.test must be "checks-test", optionally with --parallel=<workers>, which runs bun test --randomize and judges its skips, found "bun test"
   package.json: scripts.lint must run the layout check: add "checks-lint"
   bunfig.toml: bunfig.toml is missing; bun has no bunfig extends, so copy node_modules/@avi2dg/checks/bunfig.toml
 ```

package/docs/gates/checks-test.md

@@ -47,7 +47,7 @@ A skipped test or a todo without `skipReason` fails in every environment.
 It does not read a skip list from `package.json`.
 It counts a run as `ci` when `CI` is true and as `local` otherwise.
 
-The command takes no arguments for the default suite.
+The command takes no arguments for the default suite, except the worker count of [Parallel runs](#parallel-runs).
 Use `checks-test --tier=live` or `checks-test --tier=pixel` for a named test tier.
 A tier run clears Bun's ignored paths, runs only `./tests/live` or `./tests/pixel`, and still checks each skip.
 
@@ -72,10 +72,36 @@ Add the matching package scripts when either directory contains tests:
 Bun ignores both directories during the default run.
 `checks-test-layout` requires each tier script when its directory contains a test file.
 
+## Parallel runs
+
+A repository runs its test files in parallel by passing a worker count in its `test` script:
+
+```json
+{
+  "scripts": {
+    "test": "checks-test --parallel=4"
+  }
+}
+```
+
+`checks-test --parallel=<workers>` runs `bun test --randomize --parallel=<workers>`, so Bun hands each test file to one of that many worker processes.
+Each file then runs in a fresh global, so a file cannot lean on state another file left behind.
+The count is a whole number from 1 up, and without the option the suite runs serially in one process.
+Tests still run in random order, and `test.only` still fails the run.
+Bun merges every worker's results into the one JUnit report, so each skipped test and each todo is judged as in a serial run.
+A tier script takes the same option, as in `checks-test --tier=live --parallel=4`.
+
+A file whose worker crashes, for example when a test calls `process.exit`, fails the run.
+Bun reports no result from that file, so its skips are not judged and its declarations are not reported as stale.
+
+`checks-test` takes no timings file, because Bun ignores the order a timings file sets whenever `--randomize` is on.
+A timings option waits for Bun to honour timings under random order.
+
 ## Arguments
 
-The default command takes no arguments.
-Its named tier options are `--tier=live` and `--tier=pixel`.
+The default command takes no arguments except `--parallel=<workers>`, as [Parallel runs](#parallel-runs) says.
+Its named tier options are `--tier=live` and `--tier=pixel`, and a tier run takes `--parallel=<workers>` too.
+Each option appears at most once.
 It refuses test filters because every test excluded by a filter would look skipped.
 
 ## Exit codes
@@ -83,8 +109,8 @@ It refuses test filters because every test excluded by a filter would look skipp
 | Code | When |
 | --- | --- |
 | 0 | every test passed and every skip has a matching site reason |
-| 1 | a test failed, a skip lacks a site reason or a declaration is stale in CI |
-| 2 | a declaration cannot be read, `CI` is not a boolean or Bun wrote no report |
+| 1 | a test failed, a parallel worker crashed, a skip lacks a site reason or a declaration is stale in CI |
+| 2 | an argument is not one it takes, a declaration cannot be read, `CI` is not a boolean or Bun wrote no report |
 
 ## Sample output
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@avi2dg/checks",
-  "version": "0.42.0",
+  "version": "0.43.0",
   "description": "Deterministic checks shared across a set of TypeScript repositories",
   "license": "MIT",
   "repository": {

package/src/testing/flake.ts

@@ -75,7 +75,7 @@ const runOnce = Effect.fn("runOnce")(function* (seed: number, outfile: string) {
     if (passed) return yield* new ReportError({ message: `bun test passed with seed ${seed} but wrote no report` });
     return { seed, passed, tests: 0, failed: [] } satisfies Run;
   }
-  const results = yield* parseReport(yield* fs.readFileString(outfile));
+  const { results } = yield* parseReport(yield* fs.readFileString(outfile));
   return { seed, passed, tests: results.length, failed: results.filter((result) => result.outcome === "failed") } satisfies Run;
 });
 

package/src/testing/test-layout.ts

@@ -4,6 +4,7 @@ import { git } from "../core/git.ts";
 import { runMain } from "../core/main.ts";
 import { ENTRY_POINT, TEST_ENTRY_POINT } from "../core/gates.ts";
 import { identifierName, isRecord, lineOf, parseTypeScript, spanStart, stringValue } from "../core/swc.ts";
+import { testOptions, type TestOptions } from "./test.ts";
 
 export type Violation = {
   readonly file: string;
@@ -61,6 +62,7 @@ export const LAYOUT_CHECK_MARK = "src/testing/test-layout.ts";
 export const LAYOUT_CHECK_BIN = "checks-test-layout";
 const OWN_ENTRY_POINT = `src/${ENTRY_POINT.script}`;
 const TEST_SCRIPTS: readonly string[] = [TEST_ENTRY_POINT.bin, `bun src/${TEST_ENTRY_POINT.script}`];
+const PARALLEL_OPTION = "optionally with --parallel=<workers>";
 
 export class LayoutError extends Schema.TaggedError<LayoutError>()("LayoutError", {
   message: Schema.String,
@@ -229,6 +231,12 @@ export const isolationViolations = Effect.fn("isolationViolations")(function* (f
   return violations;
 });
 
+function entryOptions(script: unknown): TestOptions | undefined {
+  if (typeof script !== "string") return undefined;
+  const entry = TEST_SCRIPTS.find((candidate) => script === candidate || script.startsWith(`${candidate} `));
+  return entry === undefined ? undefined : testOptions(script.slice(entry.length).split(" ").filter((arg) => arg !== ""));
+}
+
 function tierScriptViolation(scripts: unknown, files: readonly string[], tier: (typeof TEST_TIERS)[number]): Violation | undefined {
   const file = "package.json";
   const key = `test:${tier}`;
@@ -236,11 +244,11 @@ function tierScriptViolation(scripts: unknown, files: readonly string[], tier: (
   const expected = `${TEST_ENTRY_POINT.bin} --tier=${tier}`;
   const hasTests = files.some((candidate) => candidate.startsWith(`tests/${tier}/`) && TEST_FILE.test(candidate));
   if (!hasTests && found === undefined) return undefined;
-  if (found === expected) return undefined;
+  if (entryOptions(found)?.tier === tier) return undefined;
   return {
     file,
     line: undefined,
-    message: `${key} must be "${expected}"${hasTests ? " when its test tier has files" : " when declared"}, found ${JSON.stringify(found ?? null)}`,
+    message: `${key} must be "${expected}", ${PARALLEL_OPTION},${hasTests ? " when its test tier has files" : " when declared"}, found ${JSON.stringify(found ?? null)}`,
   };
 }
 
@@ -250,11 +258,12 @@ export function scriptViolations(manifest: unknown, files: readonly string[] = [
   const test = isRecord(scripts) ? scripts["test"] : undefined;
   const lint = isRecord(scripts) ? scripts["lint"] : undefined;
   const violations: Violation[] = [];
-  if (typeof test !== "string" || !TEST_SCRIPTS.includes(test)) {
+  const options = entryOptions(test);
+  if (options === undefined || options.tier !== undefined) {
     violations.push({
       file,
       line: undefined,
-      message: `scripts.test must be exactly "${TEST_ENTRY_POINT.bin}", which runs bun test --randomize and judges its skips, found ${JSON.stringify(test ?? null)}`,
+      message: `scripts.test must be "${TEST_ENTRY_POINT.bin}", ${PARALLEL_OPTION}, which runs bun test --randomize and judges its skips, found ${JSON.stringify(test ?? null)}`,
     });
   }
   if (typeof lint !== "string" || !runsLayoutCheck(lint)) {

package/src/testing/test-report.ts

@@ -9,6 +9,11 @@ export type TestResult = {
   readonly outcome: Outcome;
 };
 
+type Report = {
+  readonly results: readonly TestResult[];
+  readonly crashedFiles: readonly string[];
+};
+
 export class ReportError extends Schema.TaggedError<ReportError>()("ReportError", {
   message: Schema.String,
 }) {}
@@ -36,6 +41,7 @@ const decodeSuite = Schema.decodeUnknownEffect(Schema.Struct({ name: Schema.Stri
 const decodeCase = Schema.decodeUnknownEffect(
   Schema.Struct({ name: Schema.String, file: Schema.String, line: Schema.NumberFromString.check(Schema.isInt()) }),
 );
+const decodeCrash = Schema.decodeUnknownEffect(Schema.Struct({ file: Schema.String }));
 
 function unescape(text: string): string {
   return text.replace(ENTITY, (_, named: string | undefined, decimal: string | undefined, hex: string | undefined) => {
@@ -91,20 +97,28 @@ function outcomeOf(testcase: Element): Outcome {
   return testcase.children.some((child) => child.tag === "failure" || child.tag === "error") ? "failed" : "passed";
 }
 
+// bun's parallel coordinator replaces every result of a file whose worker crashed with one failure that has no line.
+function crashed(testcase: Element): boolean {
+  return testcase.attributes["line"] === undefined && outcomeOf(testcase) === "failed";
+}
+
 const collect = Effect.fnUntraced(function* (
   suite: Element,
   describes: readonly string[],
-  results: TestResult[],
+  report: { readonly results: TestResult[]; readonly crashedFiles: string[] },
 ): Effect.fn.Return<void, ReportError> {
   const unreadable = (tag: string) => (cause: { readonly message: string }) =>
     new ReportError({ message: `the junit report holds a <${tag}> bun never writes: ${cause.message}` });
   for (const child of suite.children) {
     if (child.tag === "testsuite") {
       const { name } = yield* decodeSuite(child.attributes).pipe(Effect.mapError(unreadable(child.tag)));
-      yield* collect(child, [...describes, name], results);
+      yield* collect(child, [...describes, name], report);
+    } else if (child.tag === "testcase" && crashed(child)) {
+      const { file } = yield* decodeCrash(child.attributes).pipe(Effect.mapError(unreadable(child.tag)));
+      report.crashedFiles.push(file);
     } else if (child.tag === "testcase") {
       const { name, file, line } = yield* decodeCase(child.attributes).pipe(Effect.mapError(unreadable(child.tag)));
-      results.push({ file, name: [...describes, name].join(NAME_SEPARATOR), line, outcome: outcomeOf(child) });
+      report.results.push({ file, name: [...describes, name].join(NAME_SEPARATOR), line, outcome: outcomeOf(child) });
     }
   }
 });
@@ -113,8 +127,9 @@ const collect = Effect.fnUntraced(function* (
 export const parseReport = Effect.fn("parseReport")(function* (xml: string) {
   const root = yield* parseElements(xml);
   const results: TestResult[] = [];
+  const crashedFiles: string[] = [];
   for (const fileSuite of root.children) {
-    if (fileSuite.tag === "testsuite") yield* collect(fileSuite, [], results);
+    if (fileSuite.tag === "testsuite") yield* collect(fileSuite, [], { results, crashedFiles });
   }
-  return results;
+  return { results, crashedFiles } satisfies Report;
 });

package/src/testing/test.ts

@@ -20,12 +20,35 @@ export class SkipGateError extends Schema.TaggedError<SkipGateError>()("SkipGate
   message: Schema.String,
 }) {}
 
+export type TestOptions = {
+  readonly tier?: TestTier;
+  readonly workers?: number;
+};
+
 const NAME = TEST_ENTRY_POINT.bin;
-const USAGE = `usage: ${NAME} takes no arguments except --tier=live or --tier=pixel`;
+const USAGE = `usage: ${NAME} [--tier=live | --tier=pixel] [--parallel=<workers>], and it refuses test filters`;
 const TIERS = new Map<string, TestTier>([
   ["--tier=live", "live"],
   ["--tier=pixel", "pixel"],
 ]);
+const PARALLEL = /^--parallel=([1-9][0-9]*)$/;
+
+function optionOf(arg: string): TestOptions | undefined {
+  const tier = TIERS.get(arg);
+  if (tier !== undefined) return { tier };
+  const workers = PARALLEL.exec(arg)?.[1];
+  return workers === undefined ? undefined : { workers: Number(workers) };
+}
+
+export function testOptions(args: readonly string[]): TestOptions | undefined {
+  let options: TestOptions = {};
+  for (const arg of args) {
+    const option = optionOf(arg);
+    if (option === undefined || Object.keys(option).some((key) => key in options)) return undefined;
+    options = { ...options, ...option };
+  }
+  return options;
+}
 
 function matches(declaration: SkipDeclaration, result: TestResult): boolean {
   const { name } = declaration;
@@ -113,10 +136,11 @@ const environmentOf = Config.Boolean("CI").pipe(
   Effect.mapError((cause) => new SkipGateError({ message: `cannot tell a ci run from a local one: ${cause.message}` })),
 );
 
-const runSuite = Effect.fn("runSuite")(function* (outfile: string, tier: TestTier | undefined) {
+const runSuite = Effect.fn("runSuite")(function* (outfile: string, { tier, workers }: TestOptions) {
   const spawner = yield* ChildProcessSpawner.ChildProcessSpawner;
   const tierArgs = tier === undefined ? [] : ["--path-ignore-patterns", "", `./tests/${tier}`];
-  const args = ["test", "--randomize", ...tierArgs, ...reporterArgs(outfile)];
+  const parallelArgs = workers === undefined ? [] : [`--parallel=${workers}`];
+  const args = ["test", "--randomize", ...parallelArgs, ...tierArgs, ...reporterArgs(outfile)];
   return yield* spawner.exitCode(
     ChildProcess.make(process.execPath, args, {
       stdin: "ignore",
@@ -129,15 +153,14 @@ const runSuite = Effect.fn("runSuite")(function* (outfile: string, tier: TestTie
 });
 
 const testEntry = Effect.gen(function* () {
-  const args = process.argv.slice(2);
-  const tier = args.length === 0 ? undefined : args.length === 1 ? TIERS.get(args[0] ?? "") : undefined;
-  if (args.length > 0 && tier === undefined) return yield* new Usage({ message: USAGE });
+  const options = testOptions(process.argv.slice(2));
+  if (options === undefined) return yield* new Usage({ message: USAGE });
   const fs = yield* FileSystem.FileSystem;
-  const declarations = yield* readSkipDeclarations(process.cwd(), tier);
+  const declarations = yield* readSkipDeclarations(process.cwd(), options.tier);
   const environment = yield* environmentOf;
 
   const outfile = (yield* Path.Path).join(yield* fs.makeTempDirectoryScoped({ prefix: "checks-test-" }), "junit.xml");
-  const suitePassed = (yield* runSuite(outfile, tier)) === ChildProcessSpawner.ExitCode(0);
+  const suitePassed = (yield* runSuite(outfile, options)) === ChildProcessSpawner.ExitCode(0);
   if (!(yield* fs.exists(outfile))) {
     if (!suitePassed) {
       yield* Console.error(`${NAME}: bun test failed before it wrote a report, so no skip was judged`);
@@ -146,9 +169,12 @@ const testEntry = Effect.gen(function* () {
     return yield* new ReportError({ message: `bun test passed but wrote no report to ${outfile}` });
   }
 
-  const verdict = judgeSkips(yield* parseReport(yield* fs.readFileString(outfile)), declarations, environment);
+  const { results, crashedFiles } = yield* parseReport(yield* fs.readFileString(outfile));
+  for (const file of crashedFiles) yield* Console.error(`${NAME}: the worker running ${file} crashed, so its skips were not judged`);
+  const judged = declarations.filter((declaration) => !crashedFiles.includes(declaration.file));
+  const verdict = judgeSkips(results, judged, environment);
   yield* passes(verdict) ? Console.log(report(verdict)) : Console.error(report(verdict));
-  return suitePassed && passes(verdict);
+  return suitePassed && crashedFiles.length === 0 && passes(verdict);
 }).pipe(Effect.scoped);
 
 if (import.meta.main) runMain(NAME, testEntry);