coverage-check 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Jonathan Ong
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,152 @@
+# coverage-check
+
+Patch-coverage gate for CI: checks that newly added lines meet per-path coverage thresholds using LCOV reports and `git diff`. Supports per-suite LCOV accumulation for conditional CI pipelines.
+
+## Install
+
+```sh
+npm install coverage-check
+```
+
+## Usage
+
+### Basic (single run)
+
+```sh
+coverage-check check \
+  --rules .coverage-rules.yml \
+  --artifacts ./coverage-artifacts \
+  --base origin/main \
+  --head HEAD
+```
+
+Exits `0` on pass, `1` on failure, `2` on configuration error.
+
+### Suite store (conditional CI)
+
+When only some CI suites run per PR (e.g. backend tests only when backend files change), store each suite's LCOV on every run and merge them when checking:
+
+```sh
+# After backend tests run — store this suite's coverage
+coverage-check store-put \
+  --suite backend \
+  --store ./coverage-store \
+  --artifacts ./coverage-artifacts \
+  --sha "$GITHUB_SHA" \
+  --ref "$GITHUB_REF"
+
+# Sync the store directory to persistent storage (e.g. S3, git orphan branch)
+aws s3 sync ./coverage-store s3://my-bucket/coverage-store/
+
+# --- On the next PR that runs only frontend tests ---
+
+# Pull the store
+aws s3 sync s3://my-bucket/coverage-store/ ./coverage-store
+
+# Store-put the current suite
+coverage-check store-put --suite frontend --store ./coverage-store --artifacts ./coverage-artifacts
+
+# Check: merges all stored suites (backend from baseline + frontend from this run)
+coverage-check check \
+  --rules .coverage-rules.yml \
+  --artifacts ./coverage-artifacts \
+  --store ./coverage-store \
+  --suite frontend \
+  --base origin/main \
+  --head HEAD
+```
+
+The `--suite` flag on `check` tells the tool to replace the same-named suite in the store with the fresh `--artifacts` (so you always see this PR's coverage for the suite that ran, and historical coverage for suites that didn't).
+
+### GitHub PR sticky comment
+
+Pass `--pr` and `--repo` to post (or update) a sticky comment on a pull request. Requires the `gh` CLI and `GH_TOKEN`/`GITHUB_TOKEN`.
+
+```sh
+coverage-check check \
+  --rules .coverage-rules.yml \
+  --artifacts ./coverage-artifacts \
+  --pr "${{ github.event.pull_request.number }}" \
+  --repo "${{ github.repository }}"
+```
+
+## Rules file
+
+```yaml
+# .coverage-rules.yml
+rules:
+  - paths: backend/**
+    patch_coverage_min: 90
+  - paths: web/lib/api/**
+    patch_coverage_min: 100
+  - paths: web/**
+    patch_coverage_min: 5
+```
+
+Rules are matched in order; the first match wins. Files in the diff not matched by any rule are reported as informational (not gated).
+
+## CLI reference
+
+### `coverage-check check`
+
+| Flag             | Default                | Description                                                                  |
+| ---------------- | ---------------------- | ---------------------------------------------------------------------------- |
+| `--rules`        | `.coverage-rules.yml`  | Path to YAML rules file                                                      |
+| `--artifacts`    | `./coverage-artifacts` | Directory to scan for `lcov.info` files                                      |
+| `--base`         | `origin/main`          | Base git ref for `git diff`                                                  |
+| `--head`         | `HEAD`                 | Head git ref for `git diff`                                                  |
+| `--store`        | —                      | Path to a suite store directory                                              |
+| `--suite`        | —                      | Name of the current suite (fresh artifacts override this suite in the store) |
+| `--strip-prefix` | —                      | Extra path prefix to strip from LCOV `SF:` lines (repeatable)                |
+| `--pr`           | —                      | Pull request number for sticky comment                                       |
+| `--repo`         | `$GITHUB_REPOSITORY`   | `owner/repo` for sticky comment                                              |
+| `--json`         | —                      | Write JSON result to this path                                               |
+
+### `coverage-check store-put`
+
+| Flag             | Default                | Description                             |
+| ---------------- | ---------------------- | --------------------------------------- |
+| `--suite`        | required               | Suite name to store                     |
+| `--store`        | required               | Path to the suite store directory       |
+| `--artifacts`    | `./coverage-artifacts` | Directory to scan for `lcov.info` files |
+| `--strip-prefix` | —                      | Extra path prefix to strip (repeatable) |
+| `--sha`          | —                      | Git SHA to record in metadata           |
+| `--ref`          | —                      | Git ref to record in metadata           |
+
+## Programmatic API
+
+```ts
+import { runCheck, runStorePut, FileSystemSuiteStore } from "coverage-check";
+
+// Custom store adapter (e.g. S3)
+import { SuiteStore } from "coverage-check";
+
+class S3SuiteStore implements SuiteStore {
+  async list() {
+    /* ... */
+  }
+  async get(suite: string) {
+    /* ... */
+  }
+  async put(suite: string, lcov: Buffer, meta?: SuiteMeta) {
+    /* ... */
+  }
+}
+
+await runCheck({
+  rules: ".coverage-rules.yml",
+  artifacts: "./coverage",
+  base: "origin/main",
+  head: "HEAD",
+  pr: null,
+  repo: "",
+  json: null,
+  stripPrefixes: [],
+  store: new S3SuiteStore(),
+  suite: "backend",
+});
+```
+
+## License
+
+[MIT](LICENSE)

package/bin/coverage-check.mts

@@ -0,0 +1,6 @@
+#!/usr/bin/env -S node --experimental-strip-types --no-warnings
+
+import { main } from "../src/cli.mts";
+
+/* c8 ignore next */
+process.exit(await main(process.argv.slice(2)));

package/package.json

@@ -0,0 +1,59 @@
+{
+  "name": "coverage-check",
+  "version": "0.1.0",
+  "description": "Patch-coverage gate: checks that newly added lines meet per-path coverage thresholds. Supports per-suite LCOV accumulation for conditional CI.",
+  "license": "MIT",
+  "author": "Jonathan Ong",
+  "type": "module",
+  "bin": {
+    "coverage-check": "./bin/coverage-check.mts"
+  },
+  "files": [
+    "bin/**",
+    "src/**",
+    "README.md",
+    "LICENSE"
+  ],
+  "engines": {
+    "node": ">=22.0.0"
+  },
+  "dependencies": {
+    "js-yaml": "^4.1.1"
+  },
+  "devDependencies": {
+    "@types/js-yaml": "^4.0.9",
+    "@types/node": "^25.8.0",
+    "@vitest/coverage-v8": "^4.1.6",
+    "husky": "^9.1.7",
+    "oxfmt": "^0.50.0",
+    "oxlint": "^1.65.0",
+    "typescript": "^6.0.3",
+    "vitest": "^4.1.6"
+  },
+  "scripts": {
+    "prepare": "husky",
+    "prepublishOnly": "npm run typecheck && npm test",
+    "typecheck": "tsc --noEmit",
+    "lint": "oxlint src/ bin/",
+    "format": "oxfmt src/ bin/ README.md",
+    "format:check": "oxfmt --check src/ bin/ README.md",
+    "test": "vitest run",
+    "test:coverage": "vitest run --coverage"
+  },
+  "keywords": [
+    "coverage",
+    "lcov",
+    "patch-coverage",
+    "github",
+    "ci",
+    "test-coverage"
+  ],
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/jonathanong/coverage-check.git"
+  },
+  "bugs": {
+    "url": "https://github.com/jonathanong/coverage-check/issues"
+  },
+  "homepage": "https://github.com/jonathanong/coverage-check"
+}

package/src/cli.mts

@@ -0,0 +1,15 @@
+import { main as checkMain } from "./commands/check.mts";
+import { main as storePutMain } from "./commands/store-put.mts";
+
+const stderr = (msg: string) => process.stderr.write(`${msg}\n`);
+
+export async function main(argv: string[]): Promise<number> {
+  const sub = argv[0];
+
+  if (!sub || sub.startsWith("-")) return checkMain(argv);
+  if (sub === "check") return checkMain(argv.slice(1));
+  if (sub === "store-put") return storePutMain(argv.slice(1));
+
+  stderr(`coverage-check: unknown subcommand: ${JSON.stringify(sub)}`);
+  return 2;
+}

package/src/cli.test.mts

@@ -0,0 +1,45 @@
+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { join } from "node:path";
+import { tmpdir } from "node:os";
+import { afterEach, beforeEach, describe, expect, it } from "vitest";
+import { main } from "./cli.mts";
+
+describe("cli subcommand dispatch", () => {
+  let tmpDir: string;
+  let rulesPath: string;
+  let artifactsDir: string;
+
+  beforeEach(() => {
+    tmpDir = mkdtempSync(join(tmpdir(), "cli-dispatch-"));
+    rulesPath = join(tmpDir, "rules.yml");
+    artifactsDir = join(tmpDir, "artifacts");
+    mkdirSync(artifactsDir);
+    writeFileSync(rulesPath, "rules:\n  - paths: backend/**\n    patch_coverage_min: 90\n");
+  });
+
+  afterEach(() => {
+    rmSync(tmpDir, { recursive: true, force: true });
+  });
+
+  it("defaults to check when no args given", async () => {
+    // No args → check → no lcov files → returns 0
+    expect(await main(["--rules", rulesPath, "--artifacts", artifactsDir])).toBe(0);
+  });
+
+  it("explicit check subcommand works", async () => {
+    expect(await main(["check", "--rules", rulesPath, "--artifacts", artifactsDir])).toBe(0);
+  });
+
+  it("explicit store-put subcommand returns 2 when --suite is missing", async () => {
+    expect(await main(["store-put", "--store", "/tmp/store"])).toBe(2);
+  });
+
+  it("returns 2 for unknown subcommand", async () => {
+    expect(await main(["unknown-command"])).toBe(2);
+  });
+
+  it("flags-first argument (starting with --) goes to check", async () => {
+    // '--rules' starts with '-', so dispatch goes to check
+    expect(await main(["--rules", rulesPath, "--artifacts", artifactsDir])).toBe(0);
+  });
+});

package/src/commands/check.mts

@@ -0,0 +1,200 @@
+import { readFileSync, writeFileSync } from "node:fs";
+import { parseLcov } from "../lcov-parser.mts";
+import { mergeLcov } from "../lcov-merge.mts";
+import { getChangedLines } from "../diff-parser.mts";
+import { loadRules } from "../rules.mts";
+import { computePatchCoverage } from "../patch-coverage.mts";
+import { collapseRanges, renderFailureComment, renderPassComment } from "../report.mts";
+import { upsertComment } from "../github-comment.mts";
+import { collectLcovFiles, buildStripPrefixes } from "../load-artifacts.mts";
+import { FileSystemSuiteStore } from "../suite-store.mts";
+import type { LcovData } from "../types.mts";
+import type { SuiteStore } from "../suite-store.mts";
+import type { GhRunner } from "../github-comment.mts";
+
+const stdout = (msg: string) => process.stdout.write(`${msg}\n`);
+const stderr = (msg: string) => process.stderr.write(`${msg}\n`);
+
+export type CheckArgs = {
+  rules: string;
+  artifacts: string;
+  base: string;
+  head: string;
+  pr: number | null;
+  repo: string;
+  json: string | null;
+  stripPrefixes: string[];
+  store: SuiteStore | null;
+  suite: string | null;
+  gh?: GhRunner;
+};
+
+function parseArgs(argv: string[]): CheckArgs {
+  const args: CheckArgs = {
+    rules: ".coverage-rules.yml",
+    artifacts: "./coverage-artifacts",
+    base: "origin/main",
+    head: "HEAD",
+    pr: null,
+    repo: process.env["GITHUB_REPOSITORY"] ?? "",
+    json: null,
+    stripPrefixes: [],
+    store: null,
+    suite: null,
+  };
+
+  for (let i = 0; i < argv.length; i++) {
+    const flag = argv[i]!;
+    const next = argv[i + 1];
+    const val = (): string => {
+      if (next === undefined) throw new Error(`${flag} requires a value`);
+      i++;
+      return next;
+    };
+    switch (flag) {
+      case "--rules":
+        args.rules = val();
+        break;
+      case "--artifacts":
+        args.artifacts = val();
+        break;
+      case "--base":
+        args.base = val();
+        break;
+      case "--head":
+        args.head = val();
+        break;
+      case "--pr": {
+        const raw = val();
+        if (!/^\d+$/.test(raw) || raw === "0")
+          throw new Error(`--pr must be a positive integer, got: ${JSON.stringify(raw)}`);
+        args.pr = parseInt(raw, 10);
+        break;
+      }
+      case "--repo":
+        args.repo = val();
+        break;
+      case "--json":
+        args.json = val();
+        break;
+      case "--strip-prefix":
+        args.stripPrefixes.push(val());
+        break;
+      case "--store":
+        args.store = new FileSystemSuiteStore(val());
+        break;
+      case "--suite":
+        args.suite = val();
+        break;
+      default:
+        throw new Error(`unknown flag: ${flag}`);
+    }
+  }
+
+  return args;
+}
+
+export async function main(argv: string[]): Promise<number> {
+  let args: CheckArgs;
+  try {
+    args = parseArgs(argv);
+  } catch (err) {
+    /* c8 ignore next */
+    stderr(`coverage-check: ${err instanceof Error ? err.message : err}`);
+    return 2;
+  }
+  return runCheck(args);
+}
+
+export async function runCheck(args: CheckArgs): Promise<number> {
+  let rules;
+  try {
+    rules = loadRules(args.rules);
+  } catch (err) {
+    stderr(`coverage-check: failed to load rules: ${err}`);
+    return 2;
+  }
+
+  const stripPrefixes = buildStripPrefixes(args.stripPrefixes);
+  const reports: LcovData[] = [];
+
+  // Merge in suites from the store (skip the current suite — fresh artifacts take precedence)
+  if (args.store !== null) {
+    const suites = await args.store.list();
+    for (const suite of suites) {
+      if (suite === args.suite) continue;
+      const buf = await args.store.get(suite);
+      if (buf !== null) {
+        reports.push(parseLcov(buf.toString("utf8"), stripPrefixes));
+      }
+    }
+  }
+
+  // Add current run's lcov files
+  const lcovFiles = collectLcovFiles(args.artifacts);
+  for (const f of lcovFiles) {
+    reports.push(parseLcov(readFileSync(f, "utf8"), stripPrefixes));
+  }
+
+  if (reports.length === 0) {
+    stderr(`coverage-check: no coverage data found — skipping`);
+    return 0;
+  }
+
+  const lcov = mergeLcov(reports);
+
+  let diff;
+  try {
+    diff = await getChangedLines(args.base, args.head);
+  } catch (err) {
+    stderr(`coverage-check: git diff failed: ${err}`);
+    return 2;
+  }
+
+  const { buckets, informational } = computePatchCoverage(diff, lcov, rules);
+  const passed = buckets.every((b) => b.passed);
+  const result = { buckets, informational, passed };
+
+  if (args.json) {
+    writeFileSync(args.json, JSON.stringify(result, null, 2));
+  }
+
+  const runUrl =
+    process.env["GITHUB_SERVER_URL"] && process.env["GITHUB_RUN_ID"]
+      ? `${process.env["GITHUB_SERVER_URL"]}/${args.repo}/actions/runs/${process.env["GITHUB_RUN_ID"]}`
+      : "N/A";
+
+  if (!passed) {
+    stdout("\ncoverage-check: FAILED\n");
+    for (const bucket of buckets.filter((b) => !b.passed)) {
+      /* c8 ignore next -- buckets always have coverable>0 by construction */
+      const pct =
+        bucket.coverable > 0 ? `${((bucket.hit / bucket.coverable) * 100).toFixed(1)}%` : "—";
+      stdout(
+        `  ${bucket.rule}: ${pct} (${bucket.hit}/${bucket.coverable}) — threshold ${bucket.threshold}%`,
+      );
+      for (const file of bucket.files.filter((f) => f.uncoveredLines.length > 0)) {
+        stdout(`    ${file.file}: ${collapseRanges(file.uncoveredLines)}`);
+      }
+    }
+  } else {
+    stdout("\ncoverage-check: PASSED\n");
+    for (const bucket of buckets) {
+      /* c8 ignore next -- buckets always have coverable>0 by construction */
+      const pct =
+        bucket.coverable > 0 ? `${((bucket.hit / bucket.coverable) * 100).toFixed(1)}%` : "—";
+      stdout(`  ${bucket.rule}: ${pct} ✓`);
+    }
+  }
+
+  if (args.pr !== null && args.repo) {
+    const body = passed ? renderPassComment(runUrl) : renderFailureComment(result, runUrl);
+    try {
+      await upsertComment(body, args.repo, args.pr, passed, args.gh);
+    } catch (err) {
+      stderr(`coverage-check: failed to post PR comment: ${err}`);
+    }
+  }
+
+  return passed ? 0 : 1;
+}