@avi2dg/checks 0.37.0 → 0.38.0

package/CHANGELOG.md

@@ -2,6 +2,14 @@
 
 Every release of `@avi2dg/checks`, newest first, written by the release from its conventional commits.
 
+## 0.38.0
+
+Released 2026-10-10.
+
+### Features
+
+- **testing:** add checks-mutation-baseline to restore the baseline from a runner cache [#140](https://github.com/avi2d/checks/pull/140)
+
 ## 0.37.0
 
 Released 2026-10-09.

package/README.md

@@ -157,6 +157,7 @@ These bins run on their own:
 - [`checks-flake`](docs/gates/checks-flake.md) runs the suite on a schedule and records the seeds a flaky test fails with.
 - [`checks-mutation`](docs/gates/checks-mutation.md) runs Stryker for scoped checks and refuses a full run outside CI.
 - [`checks-mutation-compare`](docs/gates/checks-mutation-compare.md) compares the mutants of two reports and lists each one that regresses.
+- [`checks-mutation-baseline`](docs/gates/checks-mutation-baseline.md) restores the newest mutation baseline from `main`, through a cache on a self-hosted runner.
 - [`checks-subsumed-tests`](docs/gates/checks-subsumed-tests.md) lists each test another test subsumes in a mutation run.
 - [`checks-changelog`](docs/gates/checks-changelog.md) writes the pending release into `CHANGELOG.md` from the conventional commits since the last release.
 - [`checks-release-notes`](docs/gates/checks-release-notes.md) writes one `CHANGELOG.md` section to a file for a GitHub release.

package/docs/gates/checks-mutation-baseline.md

@@ -0,0 +1,99 @@
+---
+kind: reference
+audience: consumers
+---
+# checks-mutation-baseline
+
+`checks-mutation-baseline` restores the newest mutation baseline that `main` published, and on a self-hosted runner it keeps each baseline it downloads so the next job copies it instead.
+
+## What it checks
+
+It reads the successful runs of `mutation.yml` on `main`.
+Without `--full` it takes the newest of the 20 newest of those runs that a pull request did not start.
+It restores that run's `mutation-baseline` artifact, and takes the run whether or not it holds one.
+It tries no older run, so a newest run without the artifact restores nothing.
+An artifact without `stryker-incremental.json` counts as no baseline.
+It copies `stryker-incremental.json` to the first destination, and `mutation/mutation.json` to the second destination when the artifact holds it.
+
+With `--full` it lists up to 50 `schedule` runs and up to 50 `workflow_dispatch` runs apart, so pushes cannot crowd the full runs out of one list.
+It tries those runs newest first, and restores the first `mutation-baseline-full` artifact that has not expired, downloads and holds the report.
+It tries no run after that one.
+It takes the report from `mutation.json` at the artifact's root, or else from `mutation/mutation.json`, and copies it to the destination.
+
+It creates the directories each destination needs.
+
+## What it reads
+
+It reads the runs and their artifacts through `gh`, which needs a token that reads the repository's Actions, such as `GH_TOKEN: ${{ github.token }}` with `actions: read`.
+The `mutation-baseline` artifact holds `stryker-incremental.json` and `mutation/mutation.json`, which an upload of `reports/stryker-incremental.json` and `reports/mutation/mutation.json` writes.
+The `mutation-baseline-full` artifact holds `mutation.json` at its root, which an upload of `reports/mutation/mutation.json` alone writes.
+It reads `RUNNER_ENVIRONMENT` and `HOME` to place the runner cache.
+
+## The runner cache
+
+The cache sits under `$HOME/.cache/avi2dg-checks/mutation-baseline/`.
+It lies outside the job's workspace and `RUNNER_TEMP`, so it outlives the job on a self-hosted runner.
+Each entry sits at `<repository id>/<artifact name>/<artifact id>/`.
+GitHub gives every upload a new artifact id, so an entry under a listed id is never stale.
+On a hit the bin copies the files from the entry and downloads nothing.
+On a miss it downloads into a staging directory beside the entry and renames it into place.
+A job sharing the runner sees an entry whole or not at all.
+If the entry already exists, because another job placed it or it lost a file, the bin keeps it and restores from its own download.
+After each download it keeps the 2 highest artifact ids for that repository and artifact name, plus the entry it restores from.
+It removes the rest.
+A 100 MB baseline zip can unpack to about 500 MB, so a repository restoring both artifacts holds about 2 GB.
+
+When `RUNNER_ENVIRONMENT` is `github-hosted`, it downloads into a temporary directory and writes no cache, since a hosted runner starts every job on a fresh machine.
+When `HOME` is unset it works the same way, and prints to stderr that the runner cache is disabled.
+
+## Arguments
+
+```sh
+checks-mutation-baseline <incremental-dest> [mutation-json-dest]
+checks-mutation-baseline --full <mutation-json-dest>
+```
+
+`--full` restores only the report of a run that started with no state.
+
+## Exit codes
+
+| Code | When |
+| --- | --- |
+| 0 | it restored a baseline, or found none to restore |
+| 2 | the arguments do not parse, `gh` cannot list the runs, the cache cannot be written, or another job removed the entry before it was copied |
+
+A failed artifact lookup or download prints the `gh` error and moves on, as a missing artifact does.
+
+## Sample output
+
+```
+mutation-baseline: restored it from the runner's cache, mutation-baseline artifact 11666313211 from run 38042411822
+```
+
+A miss prints `downloaded it into the runner's cache` instead, and a hosted runner prints `downloaded it`.
+
+## When it runs
+
+Only a CI step the repository writes runs it.
+A baseline job restores the state the previous `main` run left before an incremental Stryker run:
+
+```yaml
+- name: Restore previous baseline state
+  env:
+    GH_TOKEN: ${{ github.token }}
+  run: bunx checks-mutation-baseline reports/stryker-incremental.json
+```
+
+A pull request's scope step restores the full report with `--full`:
+
+```yaml
+- name: Select mutation scope
+  env:
+    GH_TOKEN: ${{ github.token }}
+  run: bunx checks-mutation-baseline --full "$RUNNER_TEMP/baseline-full/mutation.json"
+```
+
+## Related topics
+
+- [checks-mutation](checks-mutation.md)
+- [checks-mutation-compare](checks-mutation-compare.md)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@avi2dg/checks",
-  "version": "0.37.0",
+  "version": "0.38.0",
   "description": "Deterministic checks shared across a set of TypeScript repositories",
   "license": "MIT",
   "repository": {
@@ -44,6 +44,7 @@
     "src/testing/mutation-scope.js",
     "src/testing/mutation-guard-plugin.js",
     "src/testing/mutation-compare.ts",
+    "src/testing/mutation-baseline.ts",
     "src/testing/subsumed-tests.ts",
     "src/delivery/ci-wiring.ts",
     "src/delivery/shell-command.ts",
@@ -133,6 +134,7 @@
     "checks-commit-identity": "src/delivery/commit-identity.ts",
     "checks-mutation": "src/testing/mutation.ts",
     "checks-mutation-compare": "src/testing/mutation-compare.ts",
+    "checks-mutation-baseline": "src/testing/mutation-baseline.ts",
     "checks-subsumed-tests": "src/testing/subsumed-tests.ts",
     "checks-ci-wiring": "src/delivery/ci-wiring.ts",
     "checks-comment-gate": "src/quality/comment-gate.ts",

package/src/testing/mutation-baseline.ts

@@ -0,0 +1,211 @@
+#!/usr/bin/env bun
+import { Config, Console, Effect, FileSystem, Option, Path, Schema } from "effect";
+import { ChildProcessSpawner } from "effect/process";
+import { collect } from "../core/git.ts";
+import { runMain, Usage } from "../core/main.ts";
+import { cacheRoot } from "../dependencies/cache-root.ts";
+
+class BaselineFailure extends Schema.TaggedError<BaselineFailure>()("BaselineFailure", {
+  message: Schema.String,
+}) {}
+
+const NAME = "checks-mutation-baseline";
+const USAGE = "usage: checks-mutation-baseline <incremental-dest> [mutation-json-dest] | --full <mutation-json-dest>";
+const WORKFLOW = "mutation.yml";
+const BRANCH = "main";
+const INCREMENTAL = "stryker-incremental.json";
+const REPORT = "mutation/mutation.json";
+const REPORT_ALONE = "mutation.json";
+const KEPT_PER_ARTIFACT = 2;
+const LATEST_ARTIFACT = "mutation-baseline";
+const FULL_ARTIFACT = "mutation-baseline-full";
+const LATEST_LIMIT = 20;
+const FULL_LIMIT = 50;
+const FULL_EVENTS = ["schedule", "workflow_dispatch"];
+
+// The first of `from` the artifact holds is copied to `to`.
+type Copy = { readonly from: readonly string[]; readonly to: string };
+
+const Run = Schema.Struct({ databaseId: Schema.Int, event: Schema.String, createdAt: Schema.String });
+type Run = typeof Run.Type;
+
+const Artifact = Schema.Struct({
+  id: Schema.Int,
+  name: Schema.String,
+  expired: Schema.Boolean,
+  workflow_run: Schema.Struct({ repository_id: Schema.Int }),
+});
+type Artifact = typeof Artifact.Type;
+
+const decodeRuns = Schema.decodeUnknownEffect(Schema.fromJsonString(Schema.Array(Run)));
+const decodeArtifacts = Schema.decodeUnknownEffect(Schema.fromJsonString(Schema.Struct({ artifacts: Schema.Array(Artifact) })));
+
+const gh = Effect.fn("gh")(function* (args: readonly string[]) {
+  const failed = (reason: string): BaselineFailure => new BaselineFailure({ message: `gh ${args.join(" ")}: ${reason.trim()}` });
+  const { stdout, stderr, exitCode } = yield* collect("gh", args).pipe(Effect.mapError((cause) => failed(cause.message)));
+  if (exitCode !== ChildProcessSpawner.ExitCode(0)) return yield* failed(stderr);
+  return stdout;
+});
+
+const warned = <A>(fallback: A) => (failure: BaselineFailure) => Console.error(`mutation-baseline: ${failure.message}`).pipe(Effect.as(fallback));
+
+const listed = Effect.fn("listed")(function* (event: string | undefined, limit: number) {
+  const filter = event === undefined ? [] : ["--event", event];
+  const args = ["run", "list", "--workflow", WORKFLOW, "--branch", BRANCH, "--status", "success", ...filter, "--limit", String(limit), "--json", "databaseId,event,createdAt"];
+  return yield* gh(args).pipe(
+    Effect.flatMap(decodeRuns),
+    Effect.mapError((cause) => new BaselineFailure({ message: cause.message })),
+  );
+});
+
+export function newestFirst(runs: readonly Run[]): readonly number[] {
+  return [...runs].sort((one, other) => other.createdAt.localeCompare(one.createdAt)).map((run) => run.databaseId);
+}
+
+const latestCandidates = Effect.gen(function* () {
+  const newest = (yield* listed(undefined, LATEST_LIMIT)).find((run) => run.event !== "pull_request");
+  return newest === undefined ? [] : [newest.databaseId];
+});
+
+// Each full event is listed on its own, because one shared window lets push runs crowd out every full run.
+const fullCandidates = Effect.gen(function* () {
+  const perEvent = yield* Effect.forEach(FULL_EVENTS, (event) => listed(event, FULL_LIMIT));
+  return newestFirst(perEvent.flat());
+});
+
+const artifactOf = Effect.fn("artifactOf")(function* (run: number, name: string) {
+  const found = yield* gh(["api", `repos/{owner}/{repo}/actions/runs/${run}/artifacts?name=${name}`]).pipe(
+    Effect.flatMap(decodeArtifacts),
+    Effect.mapError((cause) => new BaselineFailure({ message: cause.message })),
+  );
+  return Option.fromNullishOr(found.artifacts.find((artifact) => artifact.name === name && !artifact.expired));
+});
+
+const held = Effect.fn("held")(function* (dir: string, names: readonly string[]) {
+  const fs = yield* FileSystem.FileSystem;
+  const path = yield* Path.Path;
+  for (const name of names) {
+    const file = path.join(dir, name);
+    if (yield* fs.exists(file)) return Option.some(file);
+  }
+  return Option.none<string>();
+});
+
+const holds = (dir: string, needed: Copy) => held(dir, needed.from).pipe(Effect.map(Option.isSome));
+
+const downloaded = Effect.fn("downloaded")(function* (run: number, artifact: Artifact, dir: string, needed: Copy) {
+  const fetched = yield* gh(["run", "download", String(run), "--name", artifact.name, "--dir", dir]).pipe(Effect.as(true), Effect.catch(warned(false)));
+  return fetched && (yield* holds(dir, needed));
+});
+
+export function pruned(entries: readonly string[], kept: number, restoring: string): readonly string[] {
+  return entries
+    .filter((entry) => /^\d+$/.test(entry))
+    .sort((one, other) => Number(other) - Number(one))
+    .slice(kept)
+    .filter((entry) => entry !== restoring);
+}
+
+const prune = Effect.fn("prune")(function* (shelf: string, restoring: string) {
+  const fs = yield* FileSystem.FileSystem;
+  const path = yield* Path.Path;
+  for (const entry of pruned(yield* fs.readDirectory(shelf), KEPT_PER_ARTIFACT, restoring)) {
+    yield* fs.remove(path.join(shelf, entry), { recursive: true, force: true });
+  }
+});
+
+type Fetched = { readonly dir: string; readonly how: string };
+
+// The entry lands through a rename beside it, so a job sharing the runner sees it whole or not at all.
+const throughCache = Effect.fn("throughCache")(function* (run: number, artifact: Artifact, root: string, needed: Copy) {
+  const fs = yield* FileSystem.FileSystem;
+  const path = yield* Path.Path;
+  const shelf = path.join(root, "mutation-baseline", String(artifact.workflow_run.repository_id), artifact.name);
+  const entry = path.join(shelf, String(artifact.id));
+  if (yield* holds(entry, needed)) return Option.some<Fetched>({ dir: entry, how: "restored it from the runner's cache" });
+  yield* fs.makeDirectory(shelf, { recursive: true });
+  const staged = path.join(yield* fs.makeTempDirectoryScoped({ directory: shelf, prefix: ".staging-" }), "artifact");
+  if (!(yield* downloaded(run, artifact, staged, needed))) return Option.none<Fetched>();
+  // An entry already there stays, since a job sharing the runner may be copying from it, and the staged copy is whole.
+  const placed = yield* fs.rename(staged, entry).pipe(
+    Effect.as(entry),
+    Effect.orElseSucceed(() => staged),
+  );
+  yield* prune(shelf, String(artifact.id));
+  return Option.some<Fetched>({ dir: placed, how: "downloaded it into the runner's cache" });
+});
+
+const uncached = Effect.fn("uncached")(function* (run: number, artifact: Artifact, needed: Copy) {
+  const dir = yield* (yield* FileSystem.FileSystem).makeTempDirectoryScoped({ prefix: "mutation-baseline-" });
+  return (yield* downloaded(run, artifact, dir, needed)) ? Option.some<Fetched>({ dir, how: "downloaded it" }) : Option.none<Fetched>();
+});
+
+const restore = Effect.fn("restore")(function* (from: string, needed: Copy, optional: readonly Copy[]) {
+  const fs = yield* FileSystem.FileSystem;
+  const path = yield* Path.Path;
+  const copy = Effect.fn("copy")(function* (file: string, to: string) {
+    yield* fs.makeDirectory(path.dirname(to), { recursive: true });
+    yield* fs.copyFile(file, to);
+  });
+  const source = yield* held(from, needed.from);
+  if (Option.isNone(source)) return yield* new BaselineFailure({ message: `${from} lost ${needed.from.join(" and ")} before it was copied` });
+  yield* copy(source.value, needed.to);
+  for (const wanted of optional) {
+    const file = yield* held(from, wanted.from);
+    if (Option.isSome(file)) yield* copy(file.value, wanted.to);
+  }
+});
+
+const restoredFrom = Effect.fn("restoredFrom")(
+  function* (run: number, artifact: Artifact, root: Option.Option<string>, request: Request) {
+    const fetched = Option.isSome(root) ? yield* throughCache(run, artifact, root.value, request.needed) : yield* uncached(run, artifact, request.needed);
+    if (Option.isNone(fetched)) return false;
+    yield* restore(fetched.value.dir, request.needed, request.optional);
+    yield* Console.log(`mutation-baseline: ${fetched.value.how}, ${artifact.name} artifact ${artifact.id} from run ${run}`);
+    return true;
+  },
+  Effect.scoped,
+);
+
+// A hosted runner is a fresh machine every job, so a cache there only costs a copy.
+const runnerCache = Effect.gen(function* () {
+  const environment = yield* Config.String("RUNNER_ENVIRONMENT").pipe(Config.withDefault(""));
+  if (environment === "github-hosted") return Option.none<string>();
+  return yield* cacheRoot().pipe(
+    Effect.asSome,
+    Effect.catchTag("CacheUnrooted", () =>
+      Console.error("mutation-baseline: the runner cache is disabled because HOME is unset, so the baseline is downloaded").pipe(Effect.as(Option.none<string>())),
+    ),
+  );
+});
+
+// An artifact that lacks `needed` counts as no baseline, and `optional` copies only what the artifact holds.
+type Request = { readonly full: boolean; readonly needed: Copy; readonly optional: readonly Copy[] };
+
+const isDestination = (arg: string | undefined): arg is string => arg !== undefined && !arg.startsWith("-");
+
+function parsed(args: readonly string[]): Request | undefined {
+  if (args[0] === "--full") {
+    const [report, ...extra] = args.slice(1);
+    if (!isDestination(report) || extra.length > 0) return undefined;
+    return { full: true, needed: { from: [REPORT_ALONE, REPORT], to: report }, optional: [] };
+  }
+  const [incremental, report, ...extra] = args;
+  if (!isDestination(incremental) || extra.length > 0) return undefined;
+  return { full: false, needed: { from: [INCREMENTAL], to: incremental }, optional: report === undefined ? [] : [{ from: [REPORT], to: report }] };
+}
+
+const mutationBaseline = Effect.gen(function* () {
+  const request = parsed(process.argv.slice(2));
+  if (request === undefined) return yield* new Usage({ message: USAGE });
+  const name = request.full ? FULL_ARTIFACT : LATEST_ARTIFACT;
+  const root = yield* runnerCache;
+  for (const run of yield* request.full ? fullCandidates : latestCandidates) {
+    const artifact = yield* artifactOf(run, name).pipe(Effect.catch(warned(Option.none<Artifact>())));
+    if (Option.isSome(artifact) && (yield* restoredFrom(run, artifact.value, root, request))) return true;
+  }
+  yield* Console.log(`mutation-baseline: no successful ${BRANCH} run holds a ${name} artifact, so nothing was restored`);
+  return true;
+});
+
+if (import.meta.main) runMain(NAME, mutationBaseline);