@geonosis/ratchet 0.1.1 → 0.2.0

package/README.md

@@ -18,6 +18,22 @@ geonosis-ratchet --cwd apps/web
 
 Exit 1 when a number grew, 2 when a counter could not measure at all, 0 otherwise.
 
+### A baseline is not proof the gate still gates
+
+The ratchet compares a number against a number. Swapping the tool that produces it — a vendored lint
+plugin for a published one, one version for the next — can hold every number and still have stopped
+firing, because a rule that fires nowhere counts zero exactly like a rule with nothing to find. Check
+that separately, with [`@geonosis/lint-parity`](https://www.npmjs.com/package/@geonosis/lint-parity):
+
+```bash
+npx geonosis-lint-parity \
+  --corpus node_modules/@geonosis/oxlint-plugin-biological-architecture/corpus \
+  --a oxlintrc.old.json --b oxlintrc.new.json --out proofs/reach
+npx geonosis-lint-parity --a oxlintrc.old.json --b oxlintrc.new.json --out proofs/parity -- apps packages
+```
+
+Reach first — parity between two configs that both fire nothing is perfect parity — then the tree.
+
 ## Config
 
 `geonosis.ratchet.json` at the repo root names the counters and how to run them:
@@ -57,10 +73,64 @@ A counter whose command cannot run is a hard error, never a silent zero. A gate
 has not passed. Nor is a number believed on its own: `oxlintErrors` reads the finding lines AND the
 summary oxlint printed, and raises when they disagree.
 
+### The warnings-only window, and `expectFormat`
+
+The last-resort backstop under the oxlint counters — output in no shape we know *and* a non-zero
+exit means the format moved, so refuse — has one gap, and it is worth naming rather than pretending
+otherwise. **oxlint exits 0 when a run found only warnings.** On such a run the backstop cannot
+fire: an unrecognised format reads as zero warnings, and a zero that only shrinks is a zero the
+ratchet banks by rewriting the baseline down.
+
+`--deny-warnings` is **not** the fix. It makes warnings exit non-zero by making them errors, which
+moves every warning into `oxlintErrors` and changes what both numbers mean — a bigger lie than the
+one it patches, told to the whole baseline.
+
+The fix is to say which shape you asked for:
+
+```jsonc
+{ "counter": "oxlintWarnings", "command": "npx oxlint --format=unix .", "expectFormat": "unix" }
+```
+
+With `expectFormat` set, output that matches none of the three shapes and is not empty once npm's
+own chatter is stripped (`> …` script echoes, `npm warn …`, `npm notice …` update nags, blank
+lines) is refused **even on a clean exit**. `npm ERR!` is not stripped — that is npm saying the
+command never ran, and it must never read as zero. Unset is the default and changes nothing, so a repo that never opts in keeps exactly
+the behaviour it had. Valid values: `unix`, `agent`, `default`.
+
 When a number **grew**, the last ten lines of that counter's command output print under the
 `<-- REGRESSED` line, indented — so the report says what grew, not only that something did. A
 counter that reads a file rather than running a command (`lawLineCount`) prints nothing extra.
 
+### `testFailures` reads the runner's summary, never the exit code
+
+The counter looks for the runner's own count — `N failed` — and **refuses when it cannot parse
+one**. It never reads the process exit code, because a test runner exiting 0 over a red suite is
+commoner than anyone expects: `@cloudflare/vitest-pool-workers` 0.22 on vitest 4.1 exited 0 with
+failing tests on every workerd suite of a consumer, and every gate that trusted the exit code
+reported green over red for weeks. A refusal is loud and stops the run; a trusted 0 is silent and
+banks the red as a win.
+
+For the same reason, **give every test package its own `testFailures` entry**, each with its own
+`key`. One entry over one workspace measures one workspace; the suites it does not run are not zero
+failures, they are unmeasured — and unmeasured reads exactly like green.
+
+```jsonc
+{ "counter": "testFailures", "key": "testFailuresApi", "command": "bun --cwd apps/api test", "tiers": ["full"] }
+```
+
+`examples/during-day.ratchet.json` is four test entries for that reason: `apps/web` plus the three
+workerd packages that were previously outside every gate.
+
+### `oxlintRule` counts a warned rule twice, on purpose
+
+A rule parked at `"warn"` as ratcheted debt appears in two numbers: once inside `oxlintWarnings`,
+and once under its own `oxlintRule` key, which measures it at `"error"` through a strict temp copy of
+the config. So arming a rule at warn raises **both**, and fixing one finding lowers **both**.
+
+That is not double-counting the total; the second key exists so the debt is visible per rule instead
+of hidden inside a lump sum that a different rule's warning could mask. dielime today: `oxlintWarnings`
+12 → 135 when `no-raw-html-atoms` was armed, with its own key at 123.
+
 ## Running it where it will actually run
 
 - **Counters run in the caller's environment.** They inherit the shell the ratchet was started in,
@@ -76,6 +146,12 @@ counter that reads a file rather than running a command (`lawLineCount`) prints
   `minimumReleaseAgeExclude` into `pnpm-workspace.yaml` — rather than lowering the cooldown. The
   cooldown is protecting every other dependency in the tree; the exclusion is scoped to the one you
   chose to trust.
+- **Bumping to a new version: keep both exclusions until the install is done, then drop the old one
+  in the same commit.** `minimumReleaseAgeExclude` is matched against what the lockfile is being
+  asked to resolve, so removing the old pair first makes the resolver walk back through the version
+  it is replacing and refuse it — an install that fails for the version you are leaving, not the one
+  you are taking. Old and new together, install, then delete the old line before committing: an
+  exclusion left behind is a cooldown quietly off for a package nobody is watching any more.
 
 ## Counters
 
@@ -83,8 +159,8 @@ Every counter takes its `command` from the config, so the toolchain stays the re
 
 | id | counts | params |
 | --- | --- | --- |
-| `oxlintErrors` / `oxlintWarnings` | findings under any shape oxlint prints — `--format=unix`, the compact `agent` format, the graphical `default` — cross-checked against the tool's own summary | `command` |
-| `oxlintRule` | one named rule's findings; with `config`, after forcing it to `error` in a temp copy, so debt cannot grow behind a downgrade | `rule`, `config`, `command` |
+| `oxlintErrors` / `oxlintWarnings` | findings under any shape oxlint prints — `--format=unix`, the compact `agent` format, the graphical `default` — cross-checked against the tool's own summary | `command`, `expectFormat` |
+| `oxlintRule` | one named rule's findings, counted only on lines the run reported as findings and refused when none of them attributes itself readably; with `config`, after forcing the rule to `error` in a temp copy — `"warn"`, `"off"` and the `["off", { … }]` array form alike — so debt cannot grow behind a downgrade | `rule`, `config`, `command`, `expectFormat` |
 | `typecheckErrors` | `error TS` occurrences | `command` |
 | `testFailures` | the runner's own failure summary; throws when neither a pass nor a fail count is readable | `command` |
 | `unformattedFiles` | paths `--list-different` names that exist on disk | `command` |

package/dist/{chunk-PUJWT3QM.js → chunk-BTWIR7DK.js}

@@ -112,6 +112,25 @@ var UNIX_FINDING = /^\S[^\n]*:\d+:\d+: .*\[(Error|Warning)\/[^\]\n]+\]$/gm;
 var UNIX_SUMMARY = /^(\d+) problems?$/m;
 var AGENT_FINDING = /^\S[^\n]*:\d+:\d+: (error|warning) /gm;
 var DEFAULT_SUMMARY = /^Found (\d+) warnings? and (\d+) errors?\.$/m;
+var DEFAULT_FINDING = /^\s*[x!] [^\s(]+\([^)\n]+\): /m;
+var FINDING_SHAPES = [UNIX_FINDING, AGENT_FINDING, DEFAULT_FINDING].map(
+  (shape) => new RegExp(shape.source)
+);
+var RULE_TOKEN = /\([^()\n]+\)/;
+var findingLinesOf = (output) => output.split("\n").filter((line) => FINDING_SHAPES.some((shape) => shape.test(line)));
+var FORMATS = ["agent", "default", "unix"];
+var isFormat = (value) => typeof value === "string" && FORMATS.some((one) => one === value);
+var expectedFormat = (counter, params) => {
+  const value = params.expectFormat;
+  if (value === void 0) return void 0;
+  if (isFormat(value)) return value;
+  throw new CounterError(
+    counter,
+    `expectFormat must be one of ${FORMATS.join(", ")} \u2014 got ${JSON.stringify(value)}`
+  );
+};
+var NOISE = /^(> .*|npm warn .*|npm notice.*|\s*)$/;
+var spokenByTheTool = (output) => output.split("\n").filter((line) => !NOISE.test(line)).join("\n");
 var total = (reading) => reading.errors + reading.warnings;
 var countFindings = (output, pattern, errorToken) => {
   let errors = 0;
@@ -122,7 +141,7 @@ var countFindings = (output, pattern, errorToken) => {
   }
   return { errors, warnings };
 };
-var readFindings = (counter, { code, output }) => {
+var readFindings = (counter, { code, output }, expect) => {
   const unix = countFindings(output, UNIX_FINDING, "Error");
   const agent = countFindings(output, AGENT_FINDING, "error");
   const problems = UNIX_SUMMARY.exec(output);
@@ -148,22 +167,42 @@ ${output.trim()}`);
     return summary;
   }
   if (total(agent) > 0) return agent;
-  if (code === 0) return { errors: 0, warnings: 0 };
+  if (code === 0) {
+    if (expect !== void 0 && spokenByTheTool(output) !== "") {
+      return refuse(`asked for --format=${expect} and got output in no shape this counter reads`);
+    }
+    return { errors: 0, warnings: 0 };
+  }
   return refuse(`the tool exited ${code} and printed no findings and no summary`);
 };
 var oxlintErrors = {
   id: "oxlintErrors",
   run: async ({ params, run }) => {
     const command = stringParam("oxlintErrors", params, "command", DEFAULT_COMMAND);
-    return readFindings("oxlintErrors", run(command)).errors;
+    const expect = expectedFormat("oxlintErrors", params);
+    return readFindings("oxlintErrors", run(command), expect).errors;
   }
 };
 var oxlintWarnings = {
   id: "oxlintWarnings",
   run: async ({ params, run }) => {
     const command = stringParam("oxlintWarnings", params, "command", DEFAULT_COMMAND);
-    return readFindings("oxlintWarnings", run(command)).warnings;
+    const expect = expectedFormat("oxlintWarnings", params);
+    return readFindings("oxlintWarnings", run(command), expect).warnings;
+  }
+};
+var countRule = (result, rule, expect) => {
+  const reading = readFindings("oxlintRule", result, expect);
+  const lines = findingLinesOf(result.output);
+  if (total(reading) > 0 && !lines.some((line) => RULE_TOKEN.test(line))) {
+    throw new CounterError(
+      "oxlintRule",
+      `${total(reading)} findings, not one of them attributed to a "plugin(rule)" this counter can read \u2014 oxlint's output changed:
+${result.output.trim()}`
+    );
   }
+  const named = new RegExp(`\\(${escapeForRegex(rule)}\\)`);
+  return lines.filter((line) => named.test(line)).length;
 };
 var oxlintRule = {
   id: "oxlintRule",
@@ -171,19 +210,18 @@ var oxlintRule = {
     const rule = stringParam("oxlintRule", params, "rule");
     const command = stringParam("oxlintRule", params, "command", DEFAULT_COMMAND);
     const config = typeof params.config === "string" ? params.config : "";
-    if (config === "")
-      return countMatches(run(command).output, new RegExp(`\\(${escapeForRegex(rule)}\\)`));
+    const expect = expectedFormat("oxlintRule", params);
+    if (config === "") return countRule(run(command), rule, expect);
     const source = resolve3(cwd, config);
     if (!existsSync3(source)) throw new CounterError("oxlintRule", `no config at ${config}`);
     const strictName = `.oxlintrc.ratchet-${key}.json`;
     const strict = readFileSync2(source, "utf8").replace(
-      new RegExp(`("[^"]*${escapeForRegex(rule)}"\\s*:\\s*)"(warn|off)"`),
+      new RegExp(`("[^"]*${escapeForRegex(rule)}"\\s*:\\s*\\[?\\s*)"(warn|off)"`),
       '$1"error"'
     );
     writeFileSync(resolve3(cwd, strictName), strict);
     try {
-      const output = run(command.replace("{config}", strictName)).output;
-      return countMatches(output, new RegExp(`\\(${escapeForRegex(rule)}\\)`));
+      return countRule(run(command.replace("{config}", strictName)), rule, expect);
     } finally {
       rmSync(resolve3(cwd, strictName), { force: true });
     }

package/dist/cli.js

@@ -2,7 +2,7 @@ import {
   COUNTERS,
   formatReport,
   runRatchet
-} from "./chunk-PUJWT3QM.js";
+} from "./chunk-BTWIR7DK.js";
 
 // src/cli.ts
 var cwdFlag = process.argv.indexOf("--cwd");

package/dist/index.js

@@ -8,7 +8,7 @@ import {
   loadConfig,
   runCommand,
   runRatchet
-} from "./chunk-PUJWT3QM.js";
+} from "./chunk-BTWIR7DK.js";
 export {
   CONFIG_FILE,
   COUNTERS,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@geonosis/ratchet",
-  "version": "0.1.1",
+  "version": "0.2.0",
   "description": "Debt as a number that may only shrink — one ratchet, pluggable counters.",
   "keywords": [
     "ratchet",