agent-roast 0.8.1 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -17,36 +17,75 @@
17
17
 
18
18
  </div>
19
19
 
20
- `agent-roast` is an offline review aid for skipped tests, fake assertions, swallowed
21
- errors, unfinished stubs and type-check bypasses. Start with the code you have now;
22
- no AI commit tags, cloud account, API key or commit history are required for `check`.
23
- A Git repository is required; `git init` is sufficient.
20
+ **Did the AI fix the code, or just make the checks pass?**
21
+
22
+ Coding agents (and tired humans) often get a red build green by changing the *checks*
23
+ instead of the *code*: deleting the failing assertion, weakening it to `toBeDefined()`,
24
+ removing the test, lowering the coverage threshold, switching off `strict`, or letting
25
+ CI steps fail. A linter reading the new file cannot see that, because nothing in the new
26
+ file looks wrong. `agent-roast review` compares the **before and after** of a change and
27
+ flags exactly these moves. It is offline, runs no project code, and needs only Git.
24
28
 
25
29
  ```bash
26
- # Inspect current tracked + non-ignored untracked source files
27
- npx agent-roast check
30
+ npx agent-roast review --base origin/main # a pull request
31
+ npx agent-roast --staged # what you are about to commit
32
+ ```
28
33
 
29
- # Understand a finding and what to do next
34
+ | Rule | Fires when | Blocks by default |
35
+ | --- | --- | --- |
36
+ | `assertion-weakened` | a test keeps its name but loses assertions, or an exact check becomes `toBeDefined`/`toBeTruthy`/`assert x` | yes |
37
+ | `guard-weakened` | coverage threshold lowered, `strict`/`noImplicitAny` off, lint rule downgraded, CI step may fail or no longer runs tests, test script cannot fail | yes |
38
+ | `test-removed` | a test (or test file) disappears and no identical body appears elsewhere | no, reported |
39
+ | `expectation-rewritten` | an expected value changed, test inputs did not, and production code changed in the same PR | no, reported |
40
+ | `test-skip`, `test-focus`, `trivial-assert`, `swallowed-error`, `type-escape`, `stub-abandonment` | the classic shortcuts, compared before/after so old debt does not block | skip/focus/trivial: yes |
41
+
42
+ Renames, moved tests, parametrization, and formatter rewraps (prettier, black) are
43
+ recognized and not reported. Intentional cases can be accepted one by one with an
44
+ expiring, reviewed exception (see below). Policy is read from the base branch, so a PR
45
+ cannot switch the rules off for itself.
46
+
47
+ ### How well does it work?
48
+
49
+ Measured, not promised (`npm run build`, then `npm run bench` and `npm run bench:noise`;
50
+ method and raw numbers are in [docs/benchmarks.md](docs/benchmarks.md)):
51
+
52
+ - **Labeled changes** (`benchmarks/cases.mjs`): 32 cheating edits across JS/TS, Python, Go
53
+ and CI/config files are caught, and 15 harmless look-alikes (reformat, rename, move,
54
+ parametrize, snapshot, raised thresholds, CI step split) stay silent. The cases were
55
+ written together with the detector, so they guard against regressions; they do not prove
56
+ accuracy. Three known gaps are listed in the file and still open: mocking the module
57
+ under test, hardcoding the tested value, and vacuous matchers such as `expect.anything()`.
58
+ - **Noise on real human history** (the last 300 commits of commander.js, execa, click,
59
+ requests and zod; 606 of them touched tests or CI/config): the two blocking rules fired on
60
+ 7 commits (1.2%). Each was read by hand: one was a real defect (a refactor in `requests`
61
+ dropped the `assert` keyword, leaving a comparison with no effect); the other six were
62
+ deliberate rewrites a reviewer should still see, such as a test-framework migration,
63
+ dropping Python 2, or a documented `continue-on-error`. `test-removed` fired on 6.1% and
64
+ `expectation-rewritten` on 2.8% of those commits, which is why they only report.
65
+ - **Not measured yet:** accuracy on real AI-agent pull requests. The sample above is human
66
+ history, and it is small. Treat the numbers as a floor for noise, not a guarantee.
67
+
68
+ It is a **review aid**. It cannot prove who wrote a change or that code is correct, and it
69
+ reads text, not program semantics. A green result is not a quality certificate.
70
+
71
+ ### Quick start
72
+
73
+ ```bash
74
+ # Everything that is true of the code you have now
75
+ npx agent-roast check
30
76
  npx agent-roast explain swallowed-error
31
77
 
32
- # Adopt it in an existing project without blocking on all historical debt
78
+ # Adopt in an existing project without blocking on historical debt
33
79
  npx agent-roast check --write-baseline .agentroast-baseline.json
34
- # Review the findings and commit this policy file, then gate only new problems
35
80
  npx agent-roast check --baseline .agentroast-baseline.json
36
81
 
37
- # Verify precisely what you are about to commit
38
- npx agent-roast --staged
39
-
40
- # Audit pull request diffs (CI gate)
41
- npx agent-roast review --base origin/main --head HEAD --format github
42
-
43
- # Optional historical audit
82
+ # Optional: score a repository's history
44
83
  npx agent-roast --all --verbose
45
84
  ```
46
85
 
47
- A finding includes its file and line, the source snippet, and a concrete next step.
48
- To see the complete machine-readable result, use `check --format json`; terminal
49
- output shows the first 20 findings. `check --format markdown` produces a review table.
86
+ A finding includes its file and line, the evidence, and a concrete next step.
87
+ `--format json|markdown|sarif|github` are available; terminal output shows the first
88
+ 20 findings.
50
89
 
51
90
  ### Context review for pull requests
52
91
 
@@ -106,12 +145,12 @@ regressions) may appear in the report/summary rather than as an inline PR annota
106
145
  Use a separate SARIF category for introduced-findings review; it does not represent
107
146
  a full scan of all outstanding alerts.
108
147
 
109
- Install 0.8.1: `npx agent-roast@0.8.1 review --base origin/main`. For local development,
148
+ Install 0.9.0: `npx agent-roast@0.9.0 review --base origin/main`. For local development,
110
149
  run `npm run build` and use `node dist/cli.js`.
111
150
 
112
151
  ### Precise exceptions (since 0.8.0)
113
152
 
114
- Use `npx agent-roast@0.8.1 check --format json`, or build this checkout with
153
+ Use `npx agent-roast@0.9.0 check --format json`, or build this checkout with
115
154
  `npm run build` and run `node dist/cli.js`.
116
155
 
117
156
  Some skipped integrations or abstract stubs are intentional. In 0.8.0 you can accept
@@ -347,7 +386,7 @@ jobs:
347
386
 
348
387
  To pilot the Action without blocking on findings, set `advisory: true`. It passes
349
388
  an explicit empty fail-on list; invalid configuration, Git errors and incomplete
350
- scans still fail. The `v1` Action installs 0.8.1. The CLI equivalent is `--fail-on ''`.
389
+ scans still fail. The `v1` Action installs 0.9.0. The CLI equivalent is `--fail-on ''`.
351
390
 
352
391
  > **Security Note:** `agent-roast` runs in an isolated runner directory (`$RUNNER_TEMP`) with `--ignore-scripts` to neutralize malicious `.npmrc` files in untrusted pull requests. `pull_request_target` is not supported.
353
392
 
@@ -435,7 +474,7 @@ node scripts/verify-user-workflows.mjs dist/cli.js
435
474
  against current code. Regression tests use disposable Git repositories. The programmatic
436
475
  `auditPullRequest({ staged: true })` API retains the legacy added-lines contract;
437
476
  use `reviewStagedChanges()` for the new context comparison and `ReviewResult`.
438
- The composite Action installs exactly 0.8.1. For local development, use
477
+ The composite Action installs exactly 0.9.0. For local development, use
439
478
  `node dist/cli.js --staged` after building.
440
479
 
441
480
  ### Author & Support
@@ -786,6 +786,30 @@ var RULE_GUIDES = {
786
786
  suggestion: "Implement the path or make its unsupported state explicit to callers. Check whether this is an intentional abstract contract.",
787
787
  example: "throw new Error('Not implemented') \u2192 implement the operation and add a behavior test"
788
788
  },
789
+ "assertion-weakened": {
790
+ title: "Test checks less than before",
791
+ why: "A test that keeps its name but loses assertions, or has one replaced by a weaker check, still passes while verifying less behavior.",
792
+ suggestion: "Restore the removed assertion or explain in the PR why that behavior is no longer required. Fix the code under test instead of weakening its check.",
793
+ example: "expect(total).toBe(42) \u2192 expect(total).toBeDefined() (restore the exact check)"
794
+ },
795
+ "test-removed": {
796
+ title: "Test removed",
797
+ why: "Deleting a failing test makes a suite green without fixing the behavior it covered. Removals are legitimate when the feature itself is gone.",
798
+ suggestion: "Confirm the behavior was intentionally removed. If it still exists, restore the test and fix the code.",
799
+ example: "it('rejects expired tokens', ...) deleted while token validation remains in src/"
800
+ },
801
+ "expectation-rewritten": {
802
+ title: "Expected value edited alongside production code",
803
+ why: "Changing an expected value while the test inputs stay the same can mean the test was adjusted to match new output instead of the output being corrected.",
804
+ suggestion: "Verify the new expectation against the requirement, not against the program's current output. Intentional behavior changes should say so in the PR.",
805
+ example: "expect(price(cart)).toBe(100) \u2192 toBe(90) with price() changed in the same PR"
806
+ },
807
+ "guard-weakened": {
808
+ title: "Quality gate made easier to pass",
809
+ why: "Lower coverage thresholds, disabled compiler strictness, downgraded lint rules, tolerated CI failures or an always-passing test script let defects through without touching the code.",
810
+ suggestion: "Keep the gate and fix the underlying failure. If the change is deliberate, record the reason and a plan to restore it.",
811
+ example: 'coverageThreshold lines: 80 \u2192 60, "strict": true \u2192 false, continue-on-error: true'
812
+ },
789
813
  "panic-loop": {
790
814
  title: "Rapid overlapping fix commits",
791
815
  why: "Repeated short fix cycles can suggest a missing reproduction or feedback loop; timing alone does not prove poor work.",
@@ -806,7 +830,11 @@ var VALID_DETECTORS = [
806
830
  "panic-loop",
807
831
  "trivial-assert",
808
832
  "stub-abandonment",
809
- "test-focus"
833
+ "test-focus",
834
+ "assertion-weakened",
835
+ "test-removed",
836
+ "expectation-rewritten",
837
+ "guard-weakened"
810
838
  ];
811
839
 
812
840
  // src/file-read.ts
@@ -839,13 +867,13 @@ function readRegularFile(filename, limit) {
839
867
  import { createHash, randomUUID as randomUUID2 } from "crypto";
840
868
  import * as fs2 from "fs";
841
869
  import path from "path";
842
- function findingFingerprint(finding) {
843
- return createHash("sha256").update(JSON.stringify([finding.type, finding.file, finding.lineSnippet?.trim()])).digest("hex");
870
+ function findingFingerprint(finding2) {
871
+ return createHash("sha256").update(JSON.stringify([finding2.type, finding2.file, finding2.lineSnippet?.trim()])).digest("hex");
844
872
  }
845
873
  function createBaseline(findings) {
846
874
  const counts = /* @__PURE__ */ new Map();
847
- for (const finding of findings) {
848
- const fingerprint = findingFingerprint(finding);
875
+ for (const finding2 of findings) {
876
+ const fingerprint = findingFingerprint(finding2);
849
877
  counts.set(fingerprint, (counts.get(fingerprint) || 0) + 1);
850
878
  }
851
879
  return {
@@ -877,14 +905,14 @@ function applyBaseline(findings, baseline) {
877
905
  const remaining = new Map(baseline.entries.map((entry) => [entry.fingerprint, entry.count]));
878
906
  const fresh = [];
879
907
  let baselineCount = 0;
880
- for (const finding of findings) {
881
- const fingerprint = findingFingerprint(finding);
908
+ for (const finding2 of findings) {
909
+ const fingerprint = findingFingerprint(finding2);
882
910
  const count = remaining.get(fingerprint) || 0;
883
911
  if (count > 0) {
884
912
  remaining.set(fingerprint, count - 1);
885
913
  baselineCount++;
886
914
  } else {
887
- fresh.push(finding);
915
+ fresh.push(finding2);
888
916
  }
889
917
  }
890
918
  return { findings: fresh, baselineCount, staleBaselineCount: [...remaining.values()].reduce((a, b) => a + b, 0) };
@@ -933,11 +961,11 @@ function applyExceptions(findings, exceptions = [], now = /* @__PURE__ */ new Da
933
961
  const occurrences = new Map(priorOccurrences);
934
962
  const actionable = [];
935
963
  const suppressedFindings = [];
936
- for (const finding of findings) {
937
- const fingerprint = findingFingerprint(finding);
964
+ for (const finding2 of findings) {
965
+ const fingerprint = findingFingerprint(finding2);
938
966
  const occurrence = (occurrences.get(fingerprint) ?? 0) + 1;
939
967
  occurrences.set(fingerprint, occurrence);
940
- const enriched = { ...finding, fingerprint, occurrence };
968
+ const enriched = { ...finding2, fingerprint, occurrence };
941
969
  const allowance = active.get(fingerprint);
942
970
  if (allowance && allowance.remaining > 0) {
943
971
  allowance.remaining--;
@@ -950,7 +978,7 @@ function applyExceptions(findings, exceptions = [], now = /* @__PURE__ */ new Da
950
978
  // src/config.ts
951
979
  var MAX_CONFIG_BYTES = 64 * 1024;
952
980
  var DEFAULT_CONFIG = {
953
- failOn: ["test-skip", "trivial-assert", "test-focus"],
981
+ failOn: ["test-skip", "trivial-assert", "test-focus", "assertion-weakened", "guard-weakened"],
954
982
  ignorePaths: [
955
983
  "**/fixtures/**",
956
984
  "**/__fixtures__/**",
@@ -966,6 +994,10 @@ var DEFAULT_CONFIG = {
966
994
  "test-skip": "error",
967
995
  "trivial-assert": "error",
968
996
  "test-focus": "error",
997
+ "assertion-weakened": "error",
998
+ "guard-weakened": "error",
999
+ "test-removed": "warn",
1000
+ "expectation-rewritten": "warn",
969
1001
  "swallowed-error": "warn",
970
1002
  "stub-abandonment": "warn",
971
1003
  "type-escape": "warn"
@@ -1395,7 +1427,7 @@ function auditFileDiffs(files, commitHash = "HEAD", commitDate = (/* @__PURE__ *
1395
1427
  });
1396
1428
  }
1397
1429
  const ordered = infractions.splice(firstFinding).sort((left, right) => (left.lineNumber || 0) - (right.lineNumber || 0));
1398
- for (const finding of ordered) infractions.push(finding);
1430
+ for (const finding2 of ordered) infractions.push(finding2);
1399
1431
  }
1400
1432
  return { infractions: infractions.filter((inf) => options.rules?.[inf.type] !== "off").map((inf) => ({ ...inf, suggestion: RULE_GUIDES[inf.type].suggestion })), hasSupportedChanges, filesInspected, linesAdded };
1401
1433
  }
@@ -1530,7 +1562,7 @@ var MetricsCollector = class {
1530
1562
  const counts = {};
1531
1563
  for (const infraction of audit.infractions) {
1532
1564
  const count = counts[infraction.type] || 0;
1533
- if (count < MAX_INFRACTIONS_PER_COMMIT) buckets[infraction.type].push(infraction);
1565
+ if (count < MAX_INFRACTIONS_PER_COMMIT) buckets[infraction.type]?.push(infraction);
1534
1566
  counts[infraction.type] = count + 1;
1535
1567
  }
1536
1568
  }
@@ -1976,8 +2008,8 @@ function exceptionTerminal(result) {
1976
2008
  const accepted = result.suppressedFindings || [];
1977
2009
  if (accepted.length) {
1978
2010
  lines.push(` ${accepted.length} findings accepted by explicit exceptions (not fixed)`);
1979
- for (const finding of accepted.slice(0, 20)) lines.push(
1980
- ` ACCEPTED [${finding.type}] ${terminalText(finding.file || "unknown")}:${finding.lineNumber} \u2014 ${terminalText(finding.suppression.reason)} (expires ${terminalText(finding.suppression.expires)})`
2011
+ for (const finding2 of accepted.slice(0, 20)) lines.push(
2012
+ ` ACCEPTED [${finding2.type}] ${terminalText(finding2.file || "unknown")}:${finding2.lineNumber} \u2014 ${terminalText(finding2.suppression.reason)} (expires ${terminalText(finding2.suppression.expires)})`
1981
2013
  );
1982
2014
  if (accepted.length > 20) lines.push(` ${accepted.length - 20} more accepted findings; use --format json.`);
1983
2015
  }
@@ -1997,7 +2029,7 @@ function exceptionMarkdown(result) {
1997
2029
  "| Rule | Location | Reason | Expires |",
1998
2030
  "| --- | --- | --- | --- |"
1999
2031
  );
2000
- for (const finding of accepted) lines.push(`| ${finding.type} | ${markdownCell(`${finding.file}:${finding.lineNumber}`)} | ${markdownCell(finding.suppression.reason)} | ${markdownCell(finding.suppression.expires)} |`);
2032
+ for (const finding2 of accepted) lines.push(`| ${finding2.type} | ${markdownCell(`${finding2.file}:${finding2.lineNumber}`)} | ${markdownCell(finding2.suppression.reason)} | ${markdownCell(finding2.suppression.expires)} |`);
2001
2033
  }
2002
2034
  if (result.expiredExceptionCount) lines.push("", `${result.expiredExceptionCount} expired exceptions were not applied; review their findings.`);
2003
2035
  return lines;
@@ -2054,12 +2086,12 @@ function formatReviewTerminal(result) {
2054
2086
  lines.push(...exceptionTerminal(result));
2055
2087
  if (result.policyChanges.length) lines.push(" Candidate policy changes were ignored; adopt them separately after review.");
2056
2088
  if (result.excludedFiles || result.unsupportedFiles) lines.push(` ${result.excludedFiles} excluded \xB7 ${result.unsupportedFiles} unsupported/non-source changes`);
2057
- for (const finding of result.infractions.slice(0, 20)) {
2089
+ for (const finding2 of result.infractions.slice(0, 20)) {
2058
2090
  lines.push(
2059
2091
  "",
2060
- ` ${result.failedRules.includes(finding.type) ? "FAIL" : "WARN"} [${finding.type}] ${terminalText(finding.file || "unknown")}:${finding.lineNumber}`,
2061
- ` ${terminalText(finding.lineSnippet || finding.reason)}`,
2062
- ` Next: ${terminalText(finding.suggestion || "Review this occurrence.")}`
2092
+ ` ${result.failedRules.includes(finding2.type) ? "FAIL" : "WARN"} [${finding2.type}] ${terminalText(finding2.file || "unknown")}:${finding2.lineNumber}`,
2093
+ ` ${terminalText(finding2.lineSnippet || finding2.reason)}`,
2094
+ ` Next: ${terminalText(finding2.suggestion || "Review this occurrence.")}`
2063
2095
  );
2064
2096
  }
2065
2097
  if (result.infractions.length > 20) lines.push(` ${result.infractions.length - 20} more new findings; use --format json for all results.`);
@@ -2080,7 +2112,7 @@ function formatReviewMarkdown(result) {
2080
2112
  ];
2081
2113
  if (result.policyChanges.length) lines.push("Candidate policy changes were ignored. This review uses the supplied base commit's policy.", "");
2082
2114
  lines.push("| New finding | Head location | Evidence | Next step |", "| --- | --- | --- | --- |");
2083
- for (const finding of result.infractions) lines.push(`| ${finding.type} | ${markdownCell(`${finding.file}:${finding.lineNumber}`)} | ${markdownCell(finding.lineSnippet || finding.reason)} | ${markdownCell(finding.suggestion || "Review this occurrence.")} |`);
2115
+ for (const finding2 of result.infractions) lines.push(`| ${finding2.type} | ${markdownCell(`${finding2.file}:${finding2.lineNumber}`)} | ${markdownCell(finding2.lineSnippet || finding2.reason)} | ${markdownCell(finding2.suggestion || "Review this occurrence.")} |`);
2084
2116
  lines.push(...exceptionMarkdown(result));
2085
2117
  if (result.removedFindings.length) {
2086
2118
  lines.push(
@@ -2092,7 +2124,7 @@ function formatReviewMarkdown(result) {
2092
2124
  "| Rule | Previous location | Previous evidence |",
2093
2125
  "| --- | --- | --- |"
2094
2126
  );
2095
- for (const finding of result.removedFindings) lines.push(`| ${finding.type} | ${markdownCell(`${finding.file}:${finding.lineNumber}`)} | ${markdownCell(finding.lineSnippet || finding.reason)} |`);
2127
+ for (const finding2 of result.removedFindings) lines.push(`| ${finding2.type} | ${markdownCell(`${finding2.file}:${finding2.lineNumber}`)} | ${markdownCell(finding2.lineSnippet || finding2.reason)} |`);
2096
2128
  }
2097
2129
  for (const skipped of result.skippedFiles) lines.push("", `Not compared: ${markdownCell(skipped.file)} \u2014 ${markdownCell(skipped.reason)}`);
2098
2130
  lines.push("", `Scope: changed supported files only; ${result.excludedFiles} excluded and ${result.unsupportedFiles} unsupported/non-source changes.`);
@@ -2338,26 +2370,26 @@ var version = require2("../package.json").version;
2338
2370
  function formatSarif(report) {
2339
2371
  const occurrences = /* @__PURE__ */ new Map();
2340
2372
  const allFindings = [
2341
- ...report.infractions.map((finding) => ({ finding, suppression: void 0 })),
2342
- ...(report.suppressedFindings ?? []).map((finding) => ({ finding, suppression: finding.suppression }))
2373
+ ...report.infractions.map((finding2) => ({ finding: finding2, suppression: void 0 })),
2374
+ ...(report.suppressedFindings ?? []).map((finding2) => ({ finding: finding2, suppression: finding2.suppression }))
2343
2375
  ];
2344
2376
  allFindings.sort((a, b) => (a.finding.file ?? "").localeCompare(b.finding.file ?? "") || (a.finding.lineNumber ?? 1) - (b.finding.lineNumber ?? 1) || a.finding.type.localeCompare(b.finding.type));
2345
- const results = allFindings.map(({ finding, suppression }) => {
2346
- const filename = finding.file;
2377
+ const results = allFindings.map(({ finding: finding2, suppression }) => {
2378
+ const filename = finding2.file;
2347
2379
  if (!filename || filename.startsWith("/") || filename.split("/").includes("..") || /^[a-z]:/i.test(filename)) {
2348
2380
  throw new ConfigError("SARIF needs repository-relative finding paths.");
2349
2381
  }
2350
- const fingerprint = findingFingerprint(finding);
2351
- const occurrence = finding.occurrence ?? (occurrences.get(fingerprint) || 0) + 1;
2382
+ const fingerprint = findingFingerprint(finding2);
2383
+ const occurrence = finding2.occurrence ?? (occurrences.get(fingerprint) || 0) + 1;
2352
2384
  occurrences.set(fingerprint, occurrence);
2353
2385
  return {
2354
- ruleId: finding.type,
2355
- ruleIndex: VALID_DETECTORS.indexOf(finding.type),
2356
- level: report.failedRules.includes(finding.type) ? "error" : "warning",
2357
- message: { text: `${finding.reason}. ${finding.suggestion || RULE_GUIDES[finding.type].suggestion}` },
2386
+ ruleId: finding2.type,
2387
+ ruleIndex: VALID_DETECTORS.indexOf(finding2.type),
2388
+ level: report.failedRules.includes(finding2.type) ? "error" : "warning",
2389
+ message: { text: `${finding2.reason}. ${finding2.suggestion || RULE_GUIDES[finding2.type].suggestion}` },
2358
2390
  locations: [{ physicalLocation: {
2359
2391
  artifactLocation: { uri: filename.split("/").map((segment) => encodeURIComponent(segment)).join("/") },
2360
- region: { startLine: Math.max(1, finding.lineNumber || 1) }
2392
+ region: { startLine: Math.max(1, finding2.lineNumber || 1) }
2361
2393
  } }],
2362
2394
  partialFingerprints: { "agentRoastFinding/v1": `${fingerprint}:${occurrence}` },
2363
2395
  ...suppression ? { suppressions: [{
@@ -2662,7 +2694,7 @@ async function checkProject(options = {}) {
2662
2694
  const cwd = path5.resolve(options.cwd || process.cwd());
2663
2695
  const root = await fs6.realpath(gitOutputPath(await git(cwd, ["rev-parse", "--show-toplevel"], options.signal)));
2664
2696
  const config = loadConfig(root);
2665
- const failOn = options.failOn ?? config.failOn ?? ["test-skip", "trivial-assert", "test-focus"];
2697
+ const failOn = options.failOn ?? config.failOn ?? ["test-skip", "trivial-assert", "test-focus", "assertion-weakened", "guard-weakened"];
2666
2698
  for (const rule of failOn) if (!VALID_DETECTORS.includes(rule)) throw new ConfigError(`Unknown rule '${rule}'.`);
2667
2699
  const format = options.format || "terminal";
2668
2700
  if (!["terminal", "json", "github", "markdown", "sarif"].includes(format)) throw new ConfigError("Unsupported check output format.");
@@ -2699,14 +2731,14 @@ async function checkProject(options = {}) {
2699
2731
  const applied = baseline ? applyBaseline(infractions, baseline) : { findings: infractions, baselineCount: 0, staleBaselineCount: 0 };
2700
2732
  const notInBaseline = new Set(applied.findings);
2701
2733
  const acceptedCounts = /* @__PURE__ */ new Map();
2702
- for (const finding of infractions) {
2703
- if (notInBaseline.has(finding)) continue;
2704
- const fingerprint = findingFingerprint(finding);
2734
+ for (const finding2 of infractions) {
2735
+ if (notInBaseline.has(finding2)) continue;
2736
+ const fingerprint = findingFingerprint(finding2);
2705
2737
  acceptedCounts.set(fingerprint, (acceptedCounts.get(fingerprint) ?? 0) + 1);
2706
2738
  }
2707
2739
  const exceptions = applyExceptions(applied.findings, options.ignoreExceptions ? [] : config.exceptions, /* @__PURE__ */ new Date(), acceptedCounts);
2708
2740
  const findings = exceptions.findings;
2709
- const failedRules = [...new Set(findings.filter((finding) => failOn.includes(finding.type)).map((finding) => finding.type))];
2741
+ const failedRules = [...new Set(findings.filter((finding2) => failOn.includes(finding2.type)).map((finding2) => finding2.type))];
2710
2742
  const complete = skippedFiles.length === 0;
2711
2743
  const passed = complete && failedRules.length === 0;
2712
2744
  const result = {
@@ -2738,24 +2770,355 @@ async function checkProject(options = {}) {
2738
2770
 
2739
2771
  // src/review.ts
2740
2772
  import path6 from "path";
2773
+
2774
+ // src/tampering.ts
2775
+ import { createHash as createHash3 } from "crypto";
2776
+ var EMPTY = { immediate: [], removedTests: [], addedHashes: [], rewritten: [] };
2777
+ function languageOf(file) {
2778
+ if (/\.(?:[cm]?[jt]sx?)$/i.test(file)) return "js";
2779
+ if (/\.py$/i.test(file)) return "py";
2780
+ if (/_test\.go$/i.test(file)) return "go";
2781
+ return null;
2782
+ }
2783
+ function codeLines(text, language) {
2784
+ let block = false;
2785
+ return text.split(/\r?\n/).map((line) => {
2786
+ const trimmed = line.trim();
2787
+ if (language !== "py") {
2788
+ if (block) {
2789
+ if (trimmed.includes("*/")) block = false;
2790
+ return "";
2791
+ }
2792
+ if (trimmed.startsWith("/*")) {
2793
+ if (!trimmed.includes("*/")) block = true;
2794
+ return "";
2795
+ }
2796
+ if (trimmed.startsWith("//")) return "";
2797
+ } else if (trimmed.startsWith("#")) return "";
2798
+ return line;
2799
+ });
2800
+ }
2801
+ var JS_TEST_START = /(?<![\w$.])(?:it|test)(?:\s*\.\s*(?:only|skip|concurrent|serial))*\s*\(\s*(['"`])((?:\\.|(?!\1)[^\\\n])*)\1/;
2802
+ var PY_TEST_START = /^\s*(?:async\s+)?def\s+(test\w*)\s*\(/;
2803
+ var GO_TEST_START = /^func\s+(Test\w+)\s*\(/;
2804
+ var JS_ASSERTION = /(?<![\w$.])(?:expect|assert|expectTypeOf)\b(?!\s*\.\s*(?:assertions|hasAssertions|extend|anything|any|objectContaining|arrayContaining|stringContaining|stringMatching)\b)(?:\s*\.\s*\w+)*\s*\(|\.should\b/;
2805
+ var PY_ASSERTION = /^\s*assert\b|\bself\s*\.\s*assert\w*\s*\(|\bpytest\s*\.\s*raises\b|\bassertRaises\b/;
2806
+ var GO_ASSERTION = /\bt\s*\.\s*(?:Error|Errorf|Fatal|Fatalf|Fail|FailNow)\b|\b(?:assert|require)\s*\.\s*\w+\s*\(/;
2807
+ function callEnd(text, from, limit) {
2808
+ let depth = 1, quote = "";
2809
+ for (let i = from; i < limit; i++) {
2810
+ const char = text[i];
2811
+ if (quote) {
2812
+ if (char === "\\") i++;
2813
+ else if (char === quote || char === "\n" && quote !== "`") quote = "";
2814
+ } else if (char === '"' || char === "'" || char === "`") quote = char;
2815
+ else if (char === "/" && text[i + 1] === "/" && (i === from || /\s/.test(text[i - 1]))) {
2816
+ const newline = text.indexOf("\n", i);
2817
+ i = newline < 0 || newline > limit ? limit : newline;
2818
+ } else if (char === "(") depth++;
2819
+ else if (char === ")" && --depth === 0) return i;
2820
+ }
2821
+ return limit;
2822
+ }
2823
+ function testBodies(text, language) {
2824
+ const lines = codeLines(text, language);
2825
+ if (language === "js") {
2826
+ const code = lines.join("\n");
2827
+ const matches = [...code.matchAll(new RegExp(JS_TEST_START.source, "g"))];
2828
+ let line = 1, cursor = 0;
2829
+ return matches.map((match, i) => {
2830
+ const at = match.index ?? 0;
2831
+ for (; cursor < at; cursor++) if (code.charCodeAt(cursor) === 10) line++;
2832
+ const from = at + match[0].length;
2833
+ const limit = i + 1 < matches.length ? matches[i + 1].index ?? code.length : code.length;
2834
+ return { name: match[2], line, body: code.slice(from, Math.min(callEnd(code, from, limit) + 1, limit)), parametrized: false };
2835
+ });
2836
+ }
2837
+ const startPattern = language === "py" ? PY_TEST_START : GO_TEST_START;
2838
+ const starts = [];
2839
+ lines.forEach((line, index) => {
2840
+ const match = startPattern.exec(line);
2841
+ if (match) starts.push({ index, name: match[1], length: match[0].length });
2842
+ });
2843
+ const indentOf = (line) => /^\s*/.exec(line)[0].length;
2844
+ return starts.map((start, n) => {
2845
+ const limit = n + 1 < starts.length ? starts[n + 1].index : lines.length;
2846
+ const header = lines[start.index], indent = indentOf(header);
2847
+ let end = start.index + 1, depth = balance(header);
2848
+ while (depth > 0 && end < limit) depth += balance(lines[end++]);
2849
+ while (end < limit && (language === "go" ? lines[end - 1] !== "}" : !lines[end].trim() || indentOf(lines[end]) > indent)) end++;
2850
+ let top = start.index;
2851
+ while (top > 0 && start.index - top < 200 && lines[top - 1].trim() && !/^\s*(?:async\s+)?(?:def|class)\b/.test(lines[top - 1])) top--;
2852
+ return {
2853
+ name: start.name,
2854
+ line: start.index + 1,
2855
+ body: [header.slice(start.length), ...lines.slice(start.index + 1, end)].join("\n"),
2856
+ parametrized: /parametrize/.test(lines.slice(top, start.index).join("\n"))
2857
+ };
2858
+ });
2859
+ }
2860
+ function balance(line) {
2861
+ const bare = line.replace(/(['"`])(?:\\.|(?!\1)[^\\])*\1/g, "");
2862
+ let net = 0;
2863
+ for (const char of bare) net += "([{".includes(char) ? 1 : ")]}".includes(char) ? -1 : 0;
2864
+ return net;
2865
+ }
2866
+ function normalize(statement) {
2867
+ return statement.replace(/\s+/g, "").replace(/'/g, '"').replace(/,([)\]}])/g, "$1").replace(/;$/, "");
2868
+ }
2869
+ function splitStatements(text) {
2870
+ const pieces = [];
2871
+ let quote = "", start = 0;
2872
+ for (let i = 0; i < text.length; i++) {
2873
+ const char = text[i];
2874
+ if (quote) {
2875
+ if (char === "\\") i++;
2876
+ else if (char === quote) quote = "";
2877
+ } else if (char === '"' || char === "'" || char === "`") quote = char;
2878
+ else if (char === ";") {
2879
+ pieces.push(text.slice(start, i));
2880
+ start = i + 1;
2881
+ }
2882
+ }
2883
+ pieces.push(text.slice(start));
2884
+ return pieces.map((piece) => piece.trim()).filter(Boolean);
2885
+ }
2886
+ var CALLBACK_HEADER = /^[,\s]*(?:async\s*)?(?:\([^)]*\)|[\w$]+)\s*=>\s*\{\s*|^[,\s]*(?:async\s+)?function\s*\w*\s*\([^)]*\)\s*\{\s*/;
2887
+ function parseTests(text, language) {
2888
+ const assertionPattern = language === "js" ? JS_ASSERTION : language === "py" ? PY_ASSERTION : GO_ASSERTION;
2889
+ const seen = /* @__PURE__ */ new Map();
2890
+ return testBodies(text, language).map((test) => {
2891
+ const body = test.body.split("\n");
2892
+ const assertions = [], raw = [], setupLines = [];
2893
+ for (let i = 0; i < body.length; i++) {
2894
+ if (!assertionPattern.test(body[i])) {
2895
+ setupLines.push(body[i]);
2896
+ continue;
2897
+ }
2898
+ let statement = body[i].trim(), depth = balance(body[i]);
2899
+ while (depth > 0 && i + 1 < body.length) {
2900
+ i++;
2901
+ statement += " " + body[i].trim();
2902
+ depth += balance(body[i]);
2903
+ }
2904
+ for (const piece of language === "js" ? splitStatements(statement) : [statement]) {
2905
+ const clean = piece.replace(CALLBACK_HEADER, "");
2906
+ if (assertionPattern.test(clean)) {
2907
+ raw.push(clean);
2908
+ assertions.push(normalize(clean));
2909
+ } else setupLines.push(clean);
2910
+ }
2911
+ }
2912
+ const setup = setupLines.join("\n").replace(/\s+/g, "");
2913
+ const bodyHash = createHash3("sha1").update(test.body.replace(/\s+/g, "")).digest("hex");
2914
+ const count = (seen.get(test.name) ?? 0) + 1;
2915
+ seen.set(test.name, count);
2916
+ return { key: `${test.name}#${count}`, parametrized: test.parametrized, name: test.name, line: test.line, assertions, raw, setup, bodyHash };
2917
+ });
2918
+ }
2919
+ function skeleton(assertion) {
2920
+ return assertion.replace(/(['"`])(?:\\.|(?!\1)[^\\])*\1/g, "S").replace(/\b\d+(?:\.\d+)?\b/g, "N").replace(/\b(?:true|false|null|undefined|None|True|False|nil)\b/g, "V").replace(/\s+/g, "");
2921
+ }
2922
+ var WEAK_JS = /\.(?:toBeDefined|toBeTruthy|toBeFalsy|toBeUndefined)\s*\(|\.not\s*\.\s*(?:toBeNull|toBeUndefined|toThrow)\s*\(|\.toBeGreaterThanOrEqual\s*\(\s*0\s*\)/;
2923
+ var WEAK_PY = /^\s*assert\s+[\w.()\[\]]+\s*$|^\s*assert\s+.+\sis\s+not\s+None\s*$|^\s*assert\s+True\s*$/;
2924
+ function isWeaker(oldAssertion, newAssertion, language) {
2925
+ if (language === "js") return WEAK_JS.test(newAssertion) && !WEAK_JS.test(oldAssertion);
2926
+ if (language === "py") return WEAK_PY.test(newAssertion) && !WEAK_PY.test(oldAssertion);
2927
+ return false;
2928
+ }
2929
+ function finding(type, file, line, snippet, reason, commit) {
2930
+ return { type, file, lineNumber: Math.max(1, line), lineSnippet: snippet, commitHash: commit, commitDate: "", reason, suggestion: RULE_GUIDES[type].suggestion };
2931
+ }
2932
+ function analyzeTestFile(oldPath, newPath, oldText, newText, commit) {
2933
+ const language = languageOf(newText === null ? oldPath : newPath);
2934
+ if (!language || !(isTestFile(newPath) || isTestFile(oldPath))) return EMPTY;
2935
+ const before = oldText === null ? [] : parseTests(oldText, language);
2936
+ const after = newText === null ? [] : parseTests(newText, language);
2937
+ const afterByKey = new Map(after.map((test) => [test.key, test]));
2938
+ const newCode = newText === null ? "" : codeLines(newText, language).join("\n");
2939
+ const tally = (tests) => {
2940
+ const totals = /* @__PURE__ */ new Map();
2941
+ for (const test of tests) for (const assertion of test.assertions) totals.set(assertion, (totals.get(assertion) ?? 0) + 1);
2942
+ return totals;
2943
+ };
2944
+ const oldTotals = tally(before), newTotals = tally(after);
2945
+ const beforeKeys = new Set(before.map((test) => test.key));
2946
+ const result = { immediate: [], removedTests: [], addedHashes: [], rewritten: [] };
2947
+ for (const test of after) if (!beforeKeys.has(test.key)) result.addedHashes.push(test.bodyHash);
2948
+ for (const old of before) {
2949
+ const current = afterByKey.get(old.key);
2950
+ if (!current) {
2951
+ if (newCode && ['"', "'", "`"].some((quote) => newCode.includes(quote + old.name + quote))) continue;
2952
+ result.removedTests.push({
2953
+ bodyHash: old.bodyHash,
2954
+ finding: finding(
2955
+ "test-removed",
2956
+ newPath,
2957
+ 1,
2958
+ `${old.name}: test no longer present`,
2959
+ newText === null ? "Test file deleted" : "Test removed",
2960
+ commit
2961
+ )
2962
+ });
2963
+ continue;
2964
+ }
2965
+ if (current.parametrized && !old.parametrized) continue;
2966
+ const snapshot = (test) => test.raw.some((statement) => /\.toMatch(?:Inline)?Snapshot\b|\.matchSnapshot\b|\bsnapshot\s*\(/.test(statement));
2967
+ if (snapshot(current) && !snapshot(old)) continue;
2968
+ if (current.assertions.length < old.assertions.length) {
2969
+ const budget = tally([current]);
2970
+ const lost = old.assertions.filter((assertion) => {
2971
+ const left = budget.get(assertion) ?? 0;
2972
+ if (left > 0) {
2973
+ budget.set(assertion, left - 1);
2974
+ return false;
2975
+ }
2976
+ return true;
2977
+ });
2978
+ if (lost.every((assertion) => (newTotals.get(assertion) ?? 0) >= (oldTotals.get(assertion) ?? 0))) continue;
2979
+ result.immediate.push(finding(
2980
+ "assertion-weakened",
2981
+ newPath,
2982
+ current.line,
2983
+ `${old.name}: ${old.assertions.length} \u2192 ${current.assertions.length} assertions`,
2984
+ "Test checks less than before",
2985
+ commit
2986
+ ));
2987
+ continue;
2988
+ }
2989
+ if (current.assertions.length === old.assertions.length && old.assertions.length > 0) {
2990
+ const changed = old.assertions.map((assertion, i) => ({ before: assertion, after: current.assertions[i], rawBefore: old.raw[i], rawAfter: current.raw[i] })).filter((pair) => pair.before !== pair.after);
2991
+ const weaker = changed.find((pair) => isWeaker(pair.rawBefore, pair.rawAfter, language));
2992
+ if (weaker) {
2993
+ result.immediate.push(finding(
2994
+ "assertion-weakened",
2995
+ newPath,
2996
+ current.line,
2997
+ `${old.name}: ${weaker.rawBefore.slice(0, 80)} \u2192 ${weaker.rawAfter.slice(0, 80)}`,
2998
+ "Assertion replaced by a weaker check",
2999
+ commit
3000
+ ));
3001
+ } else if (changed.length && old.setup === current.setup && changed.every((pair) => skeleton(pair.before) === skeleton(pair.after))) {
3002
+ result.rewritten.push(finding(
3003
+ "expectation-rewritten",
3004
+ newPath,
3005
+ current.line,
3006
+ `${old.name}: ${changed[0].rawBefore.slice(0, 80)} \u2192 ${changed[0].rawAfter.slice(0, 80)}`,
3007
+ "Expected value edited while the test inputs stayed the same",
3008
+ commit
3009
+ ));
3010
+ }
3011
+ }
3012
+ }
3013
+ return result;
3014
+ }
3015
+ function resolveTampering(files, productionChanged) {
3016
+ const added = new Set(files.flatMap((file) => file.addedHashes));
3017
+ return [
3018
+ ...files.flatMap((file) => file.immediate),
3019
+ ...files.flatMap((file) => file.removedTests).filter((removed) => !added.has(removed.bodyHash)).map((removed) => removed.finding),
3020
+ ...productionChanged ? files.flatMap((file) => file.rewritten) : []
3021
+ ];
3022
+ }
3023
+ var GUARD_FILE = /(?:^|\/)(?:package\.json|tsconfig[^/]*\.json|jsconfig\.json|(?:jest|vitest|vite)\.config\.[cm]?[jt]s|\.eslintrc(?:\.[a-z]+)?|eslint\.config\.[cm]?[jt]s|pyproject\.toml|setup\.cfg|pytest\.ini|tox\.ini|\.coveragerc|codecov\.ya?ml|\.gitlab-ci\.ya?ml|\.github\/workflows\/[^/]+\.ya?ml)$/i;
3024
+ function isGuardFile(file) {
3025
+ return GUARD_FILE.test(file);
3026
+ }
3027
+ function lineOf(text, index) {
3028
+ let line = 1;
3029
+ for (let i = 0; i < index && i < text.length; i++) if (text.charCodeAt(i) === 10) line++;
3030
+ return line;
3031
+ }
3032
+ function numbers(text, pattern) {
3033
+ const seen = /* @__PURE__ */ new Map();
3034
+ return [...text.matchAll(pattern)].map((match) => {
3035
+ const label = match[1] ?? "value";
3036
+ const count = (seen.get(label) ?? 0) + 1;
3037
+ seen.set(label, count);
3038
+ return { key: `${label}#${count}`, value: Number(match[match.length - 1]), index: match.index ?? 0 };
3039
+ });
3040
+ }
3041
+ var THRESHOLD_PATTERNS = [
3042
+ /\b(lines|branches|functions|statements)\b["']?\s*[:=]\s*(\d+(?:\.\d+)?)/g,
3043
+ /\b(fail_under)\s*[:=]\s*(\d+(?:\.\d+)?)/g,
3044
+ /(--cov-fail-under)[= ](\d+(?:\.\d+)?)/g
3045
+ ];
3046
+ var SWITCHES = ["strict", "noImplicitAny", "strictNullChecks", "noImplicitReturns", "useUnknownInCatchVariables"];
3047
+ var TEST_COMMAND = /\brun:.*\b(?:tests?|unit|pytest|vitest|jest|mocha|ava|tap|tox|nox|e2e|spec)\b|^\s*-?\s*(?:npm|yarn|pnpm|go|cargo|mvn|gradle|make)\s+(?:run\s+)?(?:\w+\s+)?tests?\b/;
3048
+ function severityOf(value) {
3049
+ return /^(?:error|2)$/.test(value) ? 2 : /^(?:warn|1)$/.test(value) ? 1 : 0;
3050
+ }
3051
+ function analyzeGuardFile(file, oldText, newText, commit) {
3052
+ if (oldText === null || newText === null) return [];
3053
+ const out = [];
3054
+ const add = (index, snippet, reason) => out.push(finding("guard-weakened", file, lineOf(newText, index), snippet, reason, commit));
3055
+ if (/threshold|fail_under|cov-fail-under|coverage/i.test(newText)) {
3056
+ for (const pattern of THRESHOLD_PATTERNS) {
3057
+ const old = new Map(numbers(oldText, pattern).map((item) => [item.key, item.value]));
3058
+ for (const item of numbers(newText, pattern)) {
3059
+ const previous = old.get(item.key);
3060
+ if (previous !== void 0 && item.value < previous) {
3061
+ add(item.index, `${item.key.split("#")[0]}: ${previous} \u2192 ${item.value}`, "Coverage threshold lowered");
3062
+ }
3063
+ }
3064
+ }
3065
+ }
3066
+ if (/(?:^|\/)(?:tsconfig|jsconfig)[^/]*\.json$/i.test(file)) {
3067
+ for (const name of SWITCHES) {
3068
+ const pattern = new RegExp(`"${name}"\\s*:\\s*(true|false)`);
3069
+ const before = pattern.exec(oldText), after = pattern.exec(newText);
3070
+ if (before?.[1] === "true" && after?.[1] === "false") add(after.index, `${name}: true \u2192 false`, "Compiler strictness switched off");
3071
+ }
3072
+ }
3073
+ if (/eslint/i.test(file)) {
3074
+ const rules = (text) => new Map([...text.matchAll(/["']([@\w/.-]+)["']\s*:\s*(?:\[\s*)?["']?(error|warn|off|[012])["']?/g)].map((match) => [match[1], { level: severityOf(match[2]), index: match.index ?? 0 }]));
3075
+ const before = rules(oldText);
3076
+ for (const [name, now] of rules(newText)) {
3077
+ const previous = before.get(name);
3078
+ if (previous && previous.level === 2 && now.level < 2) add(now.index, `${name}: error \u2192 ${now.level ? "warn" : "off"}`, "Lint rule downgraded");
3079
+ }
3080
+ }
3081
+ if (/(?:\.github\/workflows\/|\.gitlab-ci)/.test(file)) {
3082
+ const count = (text, pattern) => text.split(/\r?\n/).filter((line) => !line.trim().startsWith("#") && pattern.test(line)).length;
3083
+ const lenient = /continue-on-error:\s*true|allow_failure:\s*true/;
3084
+ if (count(newText, lenient) > count(oldText, lenient)) {
3085
+ add(newText.search(lenient), "continue-on-error: true", "CI step is now allowed to fail");
3086
+ }
3087
+ if (count(newText, TEST_COMMAND) < count(oldText, TEST_COMMAND)) {
3088
+ add(0, "test command no longer runs in CI", "Test step removed from CI");
3089
+ }
3090
+ }
3091
+ if (/(?:^|\/)package\.json$/i.test(file)) {
3092
+ try {
3093
+ const before = JSON.parse(oldText)?.scripts?.test, after = JSON.parse(newText)?.scripts?.test;
3094
+ if (typeof before === "string" && typeof after === "string" && before !== after && (/^\s*(?:echo\b|true\b|exit\s+0|:)/.test(after) || /\|\|\s*(?:true|exit\s+0)\b/.test(after) || /--passWithNoTests/.test(after) && !/--passWithNoTests/.test(before))) {
3095
+ add(Math.max(0, newText.indexOf('"test"')), `test: ${after}`, "Test script can no longer fail");
3096
+ }
3097
+ } catch {
3098
+ }
3099
+ }
3100
+ return out;
3101
+ }
3102
+
3103
+ // src/review.ts
2741
3104
  var GENERATED_PATHS2 = ["**/node_modules/**", "**/dist/**", "**/build/**", "**/vendor/**", "**/*.min.js", "**/*.map"];
2742
3105
  var MAX_SCAN_BYTES2 = 50 * 1024 * 1024;
2743
3106
  function compareFindings(before, after, newPath) {
2744
3107
  const old = /* @__PURE__ */ new Map();
2745
- for (const finding of before) {
2746
- const key = findingFingerprint({ ...finding, file: newPath });
3108
+ for (const finding2 of before) {
3109
+ const key = findingFingerprint({ ...finding2, file: newPath });
2747
3110
  const bucket = old.get(key) || { findings: [], matched: 0 };
2748
- bucket.findings.push(finding);
3111
+ bucket.findings.push(finding2);
2749
3112
  old.set(key, bucket);
2750
3113
  }
2751
3114
  const introduced = [];
2752
3115
  let unchanged = 0;
2753
- for (const finding of after) {
2754
- const bucket = old.get(findingFingerprint(finding));
3116
+ for (const finding2 of after) {
3117
+ const bucket = old.get(findingFingerprint(finding2));
2755
3118
  if (bucket && bucket.matched < bucket.findings.length) {
2756
3119
  bucket.matched++;
2757
3120
  unchanged++;
2758
- } else introduced.push(finding);
3121
+ } else introduced.push(finding2);
2759
3122
  }
2760
3123
  return { introduced, removed: [...old.values()].flatMap((bucket) => bucket.findings.slice(bucket.matched)), unchanged };
2761
3124
  }
@@ -2773,7 +3136,7 @@ async function compareSourceChanges(options) {
2773
3136
  const policy = refs.baseSha === "EMPTY" ? { config: configFromText("{}", "default first-commit policy"), filename: null } : await reviewPolicy(cwd, refs.baseSha, options.signal);
2774
3137
  const format = options.format ?? "terminal";
2775
3138
  if (!["terminal", "json", "github", "markdown", "sarif"].includes(format)) throw new ConfigError("Unsupported review format.");
2776
- const failOn = options.failOn ?? policy.config.failOn ?? ["test-skip", "trivial-assert", "test-focus"];
3139
+ const failOn = options.failOn ?? policy.config.failOn ?? ["test-skip", "trivial-assert", "test-focus", "assertion-weakened", "guard-weakened"];
2777
3140
  for (const rule of failOn) if (!VALID_DETECTORS.includes(rule)) throw new ConfigError(`Unknown review rule '${rule}'.`);
2778
3141
  const changes = staged?.changes ?? await changedTreeFiles(cwd, refs.mergeBaseSha, refs.headSha, options.signal);
2779
3142
  const policyChanges = changes.filter((change) => CONFIG_FILENAMES.includes(change.oldPath) || CONFIG_FILENAMES.includes(change.newPath)).map((change) => change.newPath);
@@ -2805,13 +3168,39 @@ async function compareSourceChanges(options) {
2805
3168
  output: ""
2806
3169
  };
2807
3170
  let totalBytes = 0;
3171
+ const tamperingFiles = [];
3172
+ const guardFindings = [];
3173
+ let productionChanged = false;
3174
+ const readText = async (mode, oid) => {
3175
+ if (mode === "000000") return null;
3176
+ if (!["100644", "100755"].includes(mode)) throw new ConfigError("Changed source is a symlink, submodule or unsupported Git mode; comparison is incomplete.");
3177
+ const content = await readBlob(cwd, oid, options.signal);
3178
+ totalBytes += content.length;
3179
+ if (totalBytes > MAX_SCAN_BYTES2) throw new ConfigError("Review exceeds the 50 MiB source comparison limit.");
3180
+ if (content.includes(0)) throw new ConfigError("Binary content in a source comparison.");
3181
+ return new TextDecoder("utf-8", { fatal: true }).decode(content);
3182
+ };
2808
3183
  const existingCounts = /* @__PURE__ */ new Map();
2809
3184
  for (const change of changes) {
2810
3185
  assertNotAborted(options.signal);
2811
3186
  const oldSupported = change.oldMode !== "000000" && isCodeFile(change.oldPath);
2812
3187
  const newSupported = change.newMode !== "000000" && isCodeFile(change.newPath);
2813
3188
  if (!oldSupported && !newSupported) {
2814
- result.unsupportedFiles++;
3189
+ const guarded = (isGuardFile(change.newPath) || isGuardFile(change.oldPath)) && isCodeFile(change.newPath + ".ts", ignores);
3190
+ if (!guarded) {
3191
+ result.unsupportedFiles++;
3192
+ continue;
3193
+ }
3194
+ try {
3195
+ const oldText = await readText(change.oldMode, change.oldOid);
3196
+ const newText = await readText(change.newMode, change.newOid);
3197
+ guardFindings.push(...analyzeGuardFile(change.newPath, oldText, newText, refs.headSha));
3198
+ result.filesInspected++;
3199
+ } catch (error) {
3200
+ assertNotAborted(options.signal);
3201
+ result.skippedFiles.push({ file: change.newPath, reason: error instanceof Error ? error.message : String(error) });
3202
+ if (totalBytes > MAX_SCAN_BYTES2) break;
3203
+ }
2815
3204
  continue;
2816
3205
  }
2817
3206
  const oldInScope = oldSupported && isCodeFile(change.oldPath, ignores);
@@ -2822,14 +3211,8 @@ async function compareSourceChanges(options) {
2822
3211
  }
2823
3212
  try {
2824
3213
  let inspectedLines = 0;
2825
- const scan = async (mode, oid, filename, commit) => {
2826
- if (mode === "000000") return [];
2827
- if (!["100644", "100755"].includes(mode)) throw new ConfigError("Changed source is a symlink, submodule or unsupported Git mode; comparison is incomplete.");
2828
- const content = await readBlob(cwd, oid, options.signal);
2829
- totalBytes += content.length;
2830
- if (totalBytes > MAX_SCAN_BYTES2) throw new ConfigError("Review exceeds the 50 MiB source comparison limit.");
2831
- if (content.includes(0)) throw new ConfigError("Binary content in a source comparison.");
2832
- const text = new TextDecoder("utf-8", { fatal: true }).decode(content);
3214
+ const scan = (text, filename, commit) => {
3215
+ if (text === null) return [];
2833
3216
  const lines = text.split(/\r?\n/);
2834
3217
  if (lines.at(-1) === "") lines.pop();
2835
3218
  inspectedLines += lines.length;
@@ -2838,13 +3221,18 @@ async function compareSourceChanges(options) {
2838
3221
  if (oldSupported && !newSupported && change.newMode !== "000000") {
2839
3222
  throw new ConfigError("Source moved to an unsupported extension; cannot claim its findings were resolved.");
2840
3223
  }
2841
- const before = await scan(change.oldMode, change.oldOid, change.oldPath, refs.mergeBaseSha);
2842
- const after = await scan(change.newMode, change.newOid, change.newPath, refs.headSha);
3224
+ const oldText = await readText(change.oldMode, change.oldOid);
3225
+ const newText = await readText(change.newMode, change.newOid);
3226
+ const before = scan(oldText, change.oldPath, refs.mergeBaseSha);
3227
+ const after = scan(newText, change.newPath, refs.headSha);
3228
+ if (isTestFile(change.oldPath) || isTestFile(change.newPath)) tamperingFiles.push(analyzeTestFile(change.oldPath, change.newPath, oldText, newText, refs.headSha));
3229
+ else if (change.oldOid !== change.newOid) productionChanged = true;
3230
+ if ((isGuardFile(change.newPath) || isGuardFile(change.oldPath)) && oldText !== null && newText !== null) guardFindings.push(...analyzeGuardFile(change.newPath, oldText, newText, refs.headSha));
2843
3231
  const compared = compareFindings(before, after, change.newPath);
2844
3232
  const introduced = new Set(compared.introduced);
2845
- for (const finding of after) {
2846
- if (introduced.has(finding)) continue;
2847
- const fingerprint = findingFingerprint(finding);
3233
+ for (const finding2 of after) {
3234
+ if (introduced.has(finding2)) continue;
3235
+ const fingerprint = findingFingerprint(finding2);
2848
3236
  existingCounts.set(fingerprint, (existingCounts.get(fingerprint) ?? 0) + 1);
2849
3237
  }
2850
3238
  result.infractions.push(...compared.introduced);
@@ -2863,11 +3251,13 @@ async function compareSourceChanges(options) {
2863
3251
  assertNotAborted(options.signal);
2864
3252
  await staged?.verify();
2865
3253
  result.complete = result.skippedFiles.length === 0;
3254
+ const rules = policy.config.rules ?? {};
3255
+ result.infractions.push(...[...resolveTampering(tamperingFiles, productionChanged), ...guardFindings].filter((finding2) => rules[finding2.type] !== "off"));
2866
3256
  const exceptions = applyExceptions(result.infractions, options.ignoreExceptions ? [] : policy.config.exceptions, /* @__PURE__ */ new Date(), existingCounts);
2867
3257
  result.infractions = exceptions.findings;
2868
3258
  result.suppressedFindings = exceptions.suppressedFindings;
2869
3259
  result.expiredExceptionCount = exceptions.expiredExceptionCount;
2870
- result.failedRules = [...new Set(result.infractions.filter((finding) => failOn.includes(finding.type)).map((finding) => finding.type))];
3260
+ result.failedRules = [...new Set(result.infractions.filter((finding2) => failOn.includes(finding2.type)).map((finding2) => finding2.type))];
2871
3261
  result.passed = result.complete && result.failedRules.length === 0;
2872
3262
  result.exitCode = !result.complete ? 2 : result.passed ? 0 : 1;
2873
3263
  if (format === "json") result.output = JSON.stringify(result, null, 2);
package/dist/cli.js CHANGED
@@ -22,7 +22,7 @@ import {
22
22
  roastRepository,
23
23
  terminalText,
24
24
  writeBaseline
25
- } from "./chunk-U7OCVA3D.js";
25
+ } from "./chunk-GMBQNL6Z.js";
26
26
 
27
27
  // src/cli.ts
28
28
  import { Command } from "commander";
package/dist/index.d.ts CHANGED
@@ -32,7 +32,7 @@ interface CommitInfo {
32
32
  isAiAttributed: boolean;
33
33
  aiSignatures: string[];
34
34
  }
35
- type DetectorType = "test-skip" | "type-escape" | "swallowed-error" | "panic-loop" | "trivial-assert" | "stub-abandonment" | "test-focus";
35
+ type DetectorType = "test-skip" | "type-escape" | "swallowed-error" | "panic-loop" | "trivial-assert" | "stub-abandonment" | "test-focus" | "assertion-weakened" | "test-removed" | "expectation-rewritten" | "guard-weakened";
36
36
  declare const VALID_DETECTORS: readonly DetectorType[];
37
37
  interface Infraction {
38
38
  fingerprint?: string;
package/dist/index.js CHANGED
@@ -63,7 +63,7 @@ import {
63
63
  unquoteGitPath,
64
64
  validateExceptions,
65
65
  writeBaseline
66
- } from "./chunk-U7OCVA3D.js";
66
+ } from "./chunk-GMBQNL6Z.js";
67
67
  export {
68
68
  CONFIG_FILENAMES,
69
69
  ConfigError,
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "agent-roast",
3
- "version": "0.8.1",
4
- "description": "Offline code checks for skipped tests, swallowed errors, unfinished stubs and AI coding shortcuts",
3
+ "version": "0.9.0",
4
+ "description": "Offline review for AI-assisted changes: catches weakened tests, lowered quality gates, skipped tests and swallowed errors",
5
5
  "type": "module",
6
6
  "main": "./dist/index.js",
7
7
  "types": "./dist/index.d.ts",
@@ -20,16 +20,22 @@
20
20
  "test": "vitest run",
21
21
  "prepublishOnly": "npm run build",
22
22
  "pretest": "npm run build",
23
- "typecheck": "tsc --noEmit"
23
+ "typecheck": "tsc --noEmit",
24
+ "bench": "node scripts/benchmark-recall.mjs",
25
+ "bench:noise": "node scripts/benchmark-noise.mjs"
24
26
  },
25
27
  "keywords": [
26
28
  "ai",
27
29
  "agents",
28
- "cursor",
30
+ "ai-agents",
29
31
  "claude-code",
30
- "roast",
32
+ "cursor",
33
+ "codex",
34
+ "code-review",
35
+ "test-quality",
36
+ "ci",
37
+ "sarif",
31
38
  "git",
32
- "code-quality",
33
39
  "cli"
34
40
  ],
35
41
  "author": "Nari-Ab",
@@ -47,9 +53,7 @@
47
53
  "picocolors": "^1.1.0"
48
54
  },
49
55
  "devDependencies": {
50
- "@napi-rs/canvas": "^1.0.10",
51
56
  "@types/node": "^20.0.0",
52
- "gifenc": "^1.0.3",
53
57
  "tsup": "^8.0.0",
54
58
  "typescript": "^5.0.0",
55
59
  "vitest": "^4.1.12"
package/schema.json CHANGED
@@ -20,7 +20,11 @@
20
20
  "panic-loop",
21
21
  "trivial-assert",
22
22
  "stub-abandonment",
23
- "test-focus"
23
+ "test-focus",
24
+ "assertion-weakened",
25
+ "test-removed",
26
+ "expectation-rewritten",
27
+ "guard-weakened"
24
28
  ]
25
29
  }
26
30
  },
@@ -94,6 +98,34 @@
94
98
  "warn",
95
99
  "off"
96
100
  ]
101
+ },
102
+ "assertion-weakened": {
103
+ "enum": [
104
+ "error",
105
+ "warn",
106
+ "off"
107
+ ]
108
+ },
109
+ "test-removed": {
110
+ "enum": [
111
+ "error",
112
+ "warn",
113
+ "off"
114
+ ]
115
+ },
116
+ "expectation-rewritten": {
117
+ "enum": [
118
+ "error",
119
+ "warn",
120
+ "off"
121
+ ]
122
+ },
123
+ "guard-weakened": {
124
+ "enum": [
125
+ "error",
126
+ "warn",
127
+ "off"
128
+ ]
97
129
  }
98
130
  }
99
131
  },