agent-roast 0.18.0 → 0.20.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -78,7 +78,7 @@ Policy is read from the **base** branch, so a pull request cannot switch off the
78
78
 
79
79
  | Rule | Fires when | By default |
80
80
  | --- | --- | :-: |
81
- | `assertion-weakened` | an exact check is replaced by a weaker one: `toBe(42)` → `toBeDefined()`, `toBe(2)` → `toBeLessThanOrEqual(2)`, `toEqual(x)` → `toMatchObject(x)`, `assert x == 1` → `assert x` | blocks |
81
+ | `assertion-weakened` | an exact check is replaced by a weaker one: `toBe(42)` → `toBeDefined()`, `toBe(2)` → `toBeLessThanOrEqual(2)`, `toBe(2)` → `not.toBe(0)`, `toEqual(x)` → `toMatchObject(x)`, `assert x == 1` → `assert x` | blocks |
82
82
  | `guard-weakened` | a coverage threshold is lowered, `strict` is switched off, a lint rule is downgraded, a CI step may fail, a workflow stops running tests, a `conftest.py` hook sets a report to `passed` | blocks |
83
83
  | `test-skip` | a test that was already there is switched off or expected to fail: `it.skip`, `@pytest.mark.skip`, `@pytest.mark.xfail`, `it.fails`, `test.fixme`, `t.Skip`, `#[ignore]`, `@Disabled`, `[Ignore]`, `xit`, ... (a new test that is skipped is not reported) | blocks |
84
84
  | `test-focus`, `trivial-assert` | `.only` / `fit`; `expect(true).toBe(true)` and its relatives in ten languages; a test that asserts on a function it mocked itself | blocks |
@@ -109,23 +109,27 @@ old debt with a baseline: [docs/reference.md](docs/reference.md#configuration).
109
109
  ## Does it work?
110
110
 
111
111
  Measured, with the denominators and the caveats in [docs/benchmarks.md](docs/benchmarks.md). **Weakened checks are rare, mostly
112
- deliberate, and not specific to AI. agent-roast is a tripwire that asks about them. On repositories it had not seen, about half of its
113
- blocks were questions with a harmless answer.**
112
+ deliberate, and not specific to AI. agent-roast is a tripwire that asks about them. On repositories it had not seen, between a quarter and
113
+ two thirds of its blocks were questions with a harmless answer, depending on the sample.**
114
114
 
115
- - **131 of 131** hand-labeled cheating edits are caught and **110 of 110** harmless look-alikes (reformats, moves, raised
115
+ - **152 of 152** hand-labeled cheating edits are caught and **136 of 136** harmless look-alikes (reformats, moves, raised
116
116
  thresholds, environment-gated skips) stay silent. The cases were written with the detector, so they guard against regressions;
117
117
  they do not prove accuracy. One gap is documented and still open.
118
118
  - **12,645 commits** that touch tests or CI/config in 166 public repositories: blocking rules fired on 0.3% of AI-signed commits
119
119
  (Claude, Copilot, Cursor, Codex, Devin, aider) and on 0.3% of the others. Inside the same repositories the odds for an
120
120
  AI-signed commit were 0.81 times those of another commit (95% CI 0.40–1.67). There is no sign that AI-signed commits weaken
121
121
  checks more often.
122
- - **On repositories it was not written from, about half of the blocked commits were false alarms.** Four samples of 39 or 40 public
123
- repositories (158 in all, 44,878 commits) were replayed and every blocked commit was read. 0.17.0 blocked 74 of 11,604 commits in the
124
- first sample and 65 of 10,726 in the fourth, and 69% of both were false alarms, mostly a skip on a test that the same commit adds.
125
- This version, frozen before the fourth sample was drawn, blocked 41 of its 10,726 commits: 20 changed a check a reviewer should see
126
- (an assertion that got weaker, a gate that was relaxed, a test switched off or left focused) and 21 were false alarms, nine of them in
127
- one repository. Most of the rest, in the earlier measurements too, were deliberate and explained. One reader (an AI assistant) sorted
128
- them, so treat the figures as a first estimate ([section 9](docs/benchmarks.md#9-repositories-nobody-had-tuned-on)).
122
+ - **On repositories it was not written from, between a quarter and two thirds of the blocked commits were false alarms.** Seven samples of 39
123
+ or 40 public repositories (278 in all, 79,843 commits) were replayed and every blocked commit was read. The share was 69% for 0.17.0 on the
124
+ first sample and on the fourth, 51% for 0.18.0 on the fourth and 27% on the fifth, which it had not seen either. This version, frozen before the
125
+ sixth sample was drawn, blocked 100 of its 11,335 commits: 34 changed a check a reviewer should see (an assertion that got weaker, a gate
126
+ that was relaxed, a test switched off or left focused) and 66 were false alarms, but 51 of the 66 were one defect in one repository, an array of
127
+ mypy overrides read as one table, which is fixed in this release; the released build blocks 44 of those commits, 34 and 10. A new
128
+ test that asserts nothing counts as a block for a reviewer; counted as false alarms, 13 of 33 on the fifth sample and 72 of 100 on the sixth. Most of the rest, in the
129
+ earlier measurements too, were deliberate and explained. One reader (an AI assistant) sorted them, and the fixes were made from the same
130
+ sorting, so treat the figures as a first estimate. The published 0.19.0 blocked 31 of the 12,000 commits of the seventh sample: 16 for a reviewer and 15 false alarms (48%, 6 of them one
131
+ repository's lint configuration); this release fixes the kinds of false alarm that reading it showed, and blocks 22 of those commits ([section 9](docs/benchmarks.md#9-repositories-nobody-had-tuned-on),
132
+ [section 10](docs/benchmarks.md#10-two-more-samples-and-what-the-fixes-from-sample-4-did), [section 11](docs/benchmarks.md#11-a-seventh-sample-and-two-things-that-were-missing)).
129
133
  - **9,732 commits compared in full** from 164 public repositories that use coding agents (5,454 of them AI-signed) were reviewed again for the
130
134
  last two releases, and every new finding was read. 0.15.0's rules reported 30 new findings there and 7 were false alarms (fixed in 0.16.0);
131
135
  0.17.0's rule for an exact check turned into a bound reported 18 in 14 commits, all of them loosened checks, after two false alarms were
@@ -158,7 +162,7 @@ It is a review aid. It cannot prove who wrote a change or that code is correct,
158
162
 
159
163
  [Rules](docs/rules.md) · [Integrations](docs/integrations.md) · [Reference](docs/reference.md) (commands, exit codes,
160
164
  configuration, exceptions, baselines, limits) · [Comparison](docs/comparison.md) · [Measurements](docs/benchmarks.md) ·
161
- [Try it on your repository](docs/pilot.md) · [Contributing](CONTRIBUTING.md) · [Changelog](CHANGELOG.md)
165
+ [Real blocks](docs/examples.md) · [Try it on your repository](docs/pilot.md) · [Contributing](CONTRIBUTING.md) · [Changelog](CHANGELOG.md)
162
166
 
163
167
  Found a finding that was wrong, or a shortcut it missed? That is the most useful report there is:
164
168
  [open an issue](https://github.com/Nari-Ab/agent-roast/issues/new/choose).
@@ -1166,7 +1166,8 @@ var PYTHON_TEST_SKIP_PATTERNS = [
1166
1166
  { regex: /\bself\s*\.\s*skipTest\s*\(/, name: "imperative unittest skip (self.skipTest)", imperative: true },
1167
1167
  // A test that is expected to fail cannot fail the run, which is what a skip does too.
1168
1168
  { regex: /(?<![\w.])pytest\s*\.\s*xfail\s*\(/, name: "imperative pytest xfail (pytest.xfail)", imperative: true },
1169
- { regex: /@(?:unittest\s*\.\s*)?expectedFailure\b/, name: "test expected to fail (@unittest.expectedFailure)", imperative: false }
1169
+ // `@unittest.expectedFailure`, and the same decorator under another module's name (`@test.expectedFailure` of a test framework that re-exports it).
1170
+ { regex: /@(?:[A-Za-z_]\w*\s*\.\s*)*expectedFailure\b/, name: "test expected to fail (@unittest.expectedFailure)", imperative: false }
1170
1171
  ];
1171
1172
  var PYTHON_MARKER = /(@\s*|(?<![\w.])marks\s*=\s*\[?\s*|^[ \t]*pytestmark\s*=\s*\[?\s*)pytest\s*\.\s*mark\s*\.\s*(skip|xfail)\b/gm;
1172
1173
  function isConditionalXfail(args) {
@@ -1317,13 +1318,55 @@ function isGuardedBlockSkip(lines, profile) {
1317
1318
  return compiledConditionally(i + 1);
1318
1319
  }
1319
1320
  var JS_TEST_API = /(?<![\w$.])(?:(?:it|test|describe|suite)(?:\s*\.\s*(?:concurrent|serial|parallel|describe|failing|sequential))*\s*\.\s*(skip|todo|only|fixme|failing|fails|fail)|((?:xit|xtest|xdescribe|fit|fdescribe)))\b((?:\s*\.\s*(?:each|failing|concurrent|serial|parallel|sequential))*)\s*(?=\(|\u0001)/g;
1320
- function isConditionalSkip(source, raw, open2) {
1321
+ var MENTIONS_CI = /\bCI\b|\bisCI\b|GITHUB_ACTIONS/;
1322
+ function isConditionalSkip(source, raw, open2, start) {
1321
1323
  const args = callArguments(source, raw, open2);
1322
1324
  if (!args) return false;
1323
1325
  const isString = (arg) => /^(['"`])[\s\S]*\1$/.test(arg);
1324
- if (!args.length || isString(args[0]) || args[0] === "true") return false;
1326
+ if (!args.length || args[0] === "true") return isUnderRuntimeCondition(source, raw, start);
1327
+ if (isString(args[0])) return false;
1325
1328
  if (args.length > 1 && !isString(args.at(-1))) return false;
1326
- return !/\bCI\b|\bisCI\b|GITHUB_ACTIONS/.test(args[0]);
1329
+ return !MENTIONS_CI.test(args[0]);
1330
+ }
1331
+ var TEST_REGISTRATION = /^(?:await\s+)?(?:test|it|describe|suite|beforeAll|beforeEach|afterAll|afterEach)\b/;
1332
+ function isUnderRuntimeCondition(source, raw, start) {
1333
+ const alwaysTrue = (text) => /^\s*(?:true|1|!\s*0|!\s*false)\s*$/.test(text);
1334
+ const indentOf = (text) => text.length - text.trimStart().length;
1335
+ const lineStartOf = (offset) => Math.max(source.lastIndexOf("\n", offset - 1), source.lastIndexOf("\0", offset - 1)) + 1;
1336
+ const mentionsCi = (from, to) => MENTIONS_CI.test(raw.slice(from, to));
1337
+ const lineStart = lineStartOf(start);
1338
+ const head = source.slice(lineStart, start).trimEnd();
1339
+ if (head.endsWith(")")) {
1340
+ let depth = 0, opening = -1;
1341
+ for (let i = head.length - 1; i >= 0; i--) {
1342
+ if (head[i] === ")") depth++;
1343
+ else if (head[i] === "(" && --depth === 0) {
1344
+ opening = i;
1345
+ break;
1346
+ }
1347
+ }
1348
+ if (opening >= 0 && /(?:^|[\s}])if\s*$/.test(head.slice(0, opening))) {
1349
+ return !mentionsCi(lineStart + opening + 1, lineStart + head.length - 1) && !alwaysTrue(head.slice(opening + 1, -1));
1350
+ }
1351
+ } else if (/(?:^|[\s}])else$/.test(head)) return true;
1352
+ else if (head.endsWith("&&")) return !mentionsCi(lineStart, start) && !alwaysTrue(head.slice(0, -2).replace(/^[\s{}]+/, ""));
1353
+ const lineEnd = source.slice(start).search(/[\n\0]/);
1354
+ let limit = indentOf(source.slice(lineStart, lineEnd < 0 ? void 0 : start + lineEnd)), at = lineStart;
1355
+ for (let steps = 0; steps < 200 && at > 0; steps++) {
1356
+ if (source[at - 1] === "\0") return false;
1357
+ const previous = lineStartOf(at - 1), line = source.slice(previous, at - 1);
1358
+ const lineRaw = raw.slice(previous, at - 1);
1359
+ at = previous;
1360
+ if (!line.trim() || indentOf(line) >= limit) continue;
1361
+ const header = line.trim();
1362
+ if (/^(?:\}\s*)?(?:else\s+)?if\s*\(/.test(header)) {
1363
+ return !MENTIONS_CI.test(lineRaw) && !alwaysTrue(/\bif\s*\((.*)\)\s*\{?\s*$/.exec(header)?.[1] ?? "");
1364
+ }
1365
+ if (/^(?:\}\s*)?(?:else|catch)\b|^(?:case\b|default\s*:)|^\)\s*\{?\s*$/.test(header)) return true;
1366
+ if (TEST_REGISTRATION.test(header)) return false;
1367
+ limit = indentOf(line);
1368
+ }
1369
+ return false;
1327
1370
  }
1328
1371
  function callArguments(source, raw, open2) {
1329
1372
  if (source[open2] !== "(") return null;
@@ -1624,10 +1667,14 @@ function auditFileDiffs(files, commitHash = "HEAD", commitDate = (/* @__PURE__ *
1624
1667
  if (source[previous] === ".") continue;
1625
1668
  if (match[2] && !isTestFile(file.path)) continue;
1626
1669
  if (match[2] && callArguments(source, raw, match.index + match[0].length)?.length === 0) continue;
1670
+ if ((match[2] === "fit" || match[2] === "fdescribe") && !match[3]) {
1671
+ const first = callArguments(source, raw, match.index + match[0].length)?.[0];
1672
+ if (first !== void 0 && !/^[`'"]/.test(first)) continue;
1673
+ }
1627
1674
  if (match[2] && previous >= 0 && (/[(,=:?[&|!+\-*%<]/.test(source[previous]) || /(?:await|return|yield|typeof|void)$/.test(source.slice(Math.max(0, previous - 6), previous + 1)))) continue;
1628
1675
  if (/\bfunction$/.test(source.slice(Math.max(0, previous - 8), previous + 1))) continue;
1629
1676
  if (source[match.index + match[0].length] === "" && !/\.\s*each\s*$/.test(match[3])) continue;
1630
- if ((match[1] === "skip" || match[1] === "fixme" || match[1] === "fail") && !match[3] && isConditionalSkip(source, raw, match.index + match[0].length)) continue;
1677
+ if ((match[1] === "skip" || match[1] === "fixme" || match[1] === "fail") && !match[3] && isConditionalSkip(source, raw, match.index + match[0].length, match.index)) continue;
1631
1678
  const focus = match[1] === "only" || match[2] === "fit" || match[2] === "fdescribe";
1632
1679
  const expectedToFail = match[1] === "failing" || match[1] === "fails" || match[1] === "fail";
1633
1680
  infractions.push({
@@ -2859,6 +2906,14 @@ async function readBlob(cwd, oid, signal, limit = 2 * 1024 * 1024) {
2859
2906
  if (buffer.length !== size) throw new ConfigError("Git blob size differs from the recorded object.");
2860
2907
  return buffer;
2861
2908
  }
2909
+ async function ciFilesAt(cwd, sha, signal) {
2910
+ if (!/^[a-f0-9]{40,64}$/.test(sha)) return [];
2911
+ const raw = await readGit(cwd, ["ls-tree", "-r", "-z", "--full-tree", sha, "--", ".github/workflows", ".gitlab-ci.yml"], signal, 1024 * 1024);
2912
+ return raw.toString().split("\0").filter(Boolean).flatMap((entry) => {
2913
+ const match = /^100(?:644|755) blob ([a-f0-9]{40,64})\t(.+\.ya?ml)$/.exec(entry);
2914
+ return match ? [{ path: match[2], oid: match[1] }] : [];
2915
+ });
2916
+ }
2862
2917
  async function reviewPolicy(cwd, baseSha, signal) {
2863
2918
  const raw = await readGit(cwd, ["ls-tree", "--full-tree", "-z", baseSha, "--", ...CONFIG_FILENAMES], signal, 8192);
2864
2919
  const entries = raw.toString().split("\0").filter(Boolean);
@@ -3208,6 +3263,12 @@ function javascript(text) {
3208
3263
  const end = closing(code, open2);
3209
3264
  if (end > 0) callbacks.push([open2, end]);
3210
3265
  }
3266
+ const tests = [];
3267
+ for (const match of code.matchAll(/(?<![\w$.])(?:it|test|describe)(?:\s*\.\s*\w+)*\s*\(/g)) {
3268
+ const open2 = match.index + match[0].length - 1;
3269
+ const end = closing(code, open2);
3270
+ if (end > 0) tests.push([open2, end]);
3271
+ }
3211
3272
  const programLevel = (index) => {
3212
3273
  let depth = 0;
3213
3274
  for (let i = 0; i < index; i++) {
@@ -3226,8 +3287,9 @@ function javascript(text) {
3226
3287
  const end = closing(code, open2);
3227
3288
  const args = end > 0 ? raw.slice(open2, end) : "";
3228
3289
  if (/requireActual|importActual|importOriginal/.test(args)) continue;
3229
- const loader = /^\(\s*(['"`])[^'"`\n]+\1\s*,\s*(?:async\s+)?(?:function\s*[\w$]*\s*)?\(\s*([A-Za-z_$][\w$]*)\s*[,)]/.exec(args)?.[2];
3230
- if (loader && new RegExp(`(?<![\\w$.])${loader.replace(/\$/g, "\\$")}\\s*(?:<[^\\n]*?>)?\\s*\\(`).test(args.slice(args.indexOf(loader) + loader.length))) continue;
3290
+ const factoryHead = /^\(\s*(['"`])[^'"`\n]+\1\s*,\s*(?:async\s+)?(?:function\s*[\w$]*\s*)?(?:\(\s*([A-Za-z_$][\w$]*)\s*[,)]|([A-Za-z_$][\w$]*)\s*=>)/.exec(args);
3291
+ const loader = factoryHead?.[2] ?? factoryHead?.[3];
3292
+ if (loader && new RegExp(`(?<![\\w$])(?:(?<=\\.\\.\\.)|(?<!\\.))${loader.replace(/\$/g, "\\$")}\\s*(?:<[^\\n]*?>)?\\s*\\(`).test(args.slice(args.indexOf(loader) + loader.length))) continue;
3231
3293
  const factory = /^\(\s*(['"`])[^'"`\n]+\1\s*,/.test(args) && !/,\s*\{\s*spy\s*:\s*true\b/.test(args);
3232
3294
  if (!mocked.has(specifier)) mocked.set(specifier, { index: match.index, fake: factory });
3233
3295
  }
@@ -3256,7 +3318,8 @@ function javascript(text) {
3256
3318
  if (match[1].startsWith("{")) bind(match[1], specifier);
3257
3319
  else bindings.set(match[1], { specifier, name: "*", namespace: true });
3258
3320
  }
3259
- const found = /* @__PURE__ */ new Map();
3321
+ const candidates = [];
3322
+ const mockedNames = [...bindings].filter(([, binding]) => mocked.has(binding.specifier)).map(([local]) => local.replace(/\$/g, "\\$"));
3260
3323
  for (const match of code.matchAll(/(?<![\w$.])expect\s*\(/g)) {
3261
3324
  const open2 = match.index + match[0].length - 1;
3262
3325
  const end = closing(code, open2);
@@ -3277,18 +3340,32 @@ function javascript(text) {
3277
3340
  const name = binding.namespace ? call[2] : call[2] ? void 0 : binding.name;
3278
3341
  if (!name) continue;
3279
3342
  const identifier = call[1].replace(/\$/g, "\\$");
3280
- const configured = new RegExp(`(?<![\\w$.])${identifier}\\b[^;]{0,80}?\\.\\s*mock(?:Return|Resolved|Rejected|Implementation)\\w*\\s*\\(`).test(raw);
3343
+ const configures = new RegExp(`(?<![\\w$.])${identifier}\\b[^;]{0,80}?\\.\\s*mock(?:Return|Resolved|Rejected|Implementation)\\w*\\s*\\(`, "g");
3344
+ const configured = [...raw.matchAll(configures)].some((config) => tests.every((range) => !within(range, config.index) || within(range, match.index)));
3281
3345
  if (!mock.fake && !configured) continue;
3282
3346
  const key = `${binding.specifier}::${name}`;
3283
- if (found.has(key)) continue;
3284
3347
  const line = lineOf2(mock.index);
3285
- found.set(key, {
3348
+ candidates.push({
3286
3349
  key,
3287
- line,
3288
- snippet: lines[line - 1].trim().slice(0, 120),
3289
- reason: `assertion on the output of ${name}(), which this test replaces with a mock of "${binding.specifier}"`
3350
+ at: match.index,
3351
+ range: innermost(callbacks, match.index),
3352
+ subject: {
3353
+ key,
3354
+ line,
3355
+ snippet: lines[line - 1].trim().slice(0, 120),
3356
+ reason: `assertion on the output of ${name}(), which this test replaces with a mock of "${binding.specifier}"`
3357
+ }
3290
3358
  });
3291
3359
  }
3360
+ const candidateAt = new Set(candidates.map((candidate) => candidate.at));
3361
+ const mentionsMock = mockedNames.length ? new RegExp(`(?<![\\w$.])(?:${mockedNames.join("|")})(?![\\w$])`) : null;
3362
+ const checksSomethingElse = (range) => !!range && [...code.matchAll(/(?<![\w$.])expect\s*\(/g)].some((other) => {
3363
+ if (!within(range, other.index) || candidateAt.has(other.index)) return false;
3364
+ const open2 = other.index + other[0].length - 1, end = closing(code, open2);
3365
+ return end > 0 && !mentionsMock?.test(code.slice(open2 + 1, end));
3366
+ });
3367
+ const found = /* @__PURE__ */ new Map();
3368
+ for (const candidate of candidates) if (!found.has(candidate.key) && !checksSomethingElse(candidate.range)) found.set(candidate.key, candidate.subject);
3292
3369
  return [...found.values()];
3293
3370
  }
3294
3371
  var DOTTED = /^[A-Za-z_][\w.]*$/;
@@ -3398,6 +3475,7 @@ function python(text) {
3398
3475
  const receiver = new RegExp(`(?<![\\w.])${patch.object.replace(/[.$]/g, "\\$&")}\\s*\\.\\s*(?!${attribute}\\b)\\w+\\s*\\(`);
3399
3476
  if (receiver.test(code.slice(patch.end + 1, match.index))) continue;
3400
3477
  }
3478
+ if (/(?<![\w$])(?:[A-Za-z_]\w*\s*\.\s*)*_*(?:install|wrap|instrument|inject|hook|swap|rewire|reload)\w*\s*\(/.test(code.slice(patch.end + 1, match.index))) continue;
3401
3479
  const line = lineOf2(patch.index);
3402
3480
  found.set(key, {
3403
3481
  key,
@@ -3428,13 +3506,15 @@ function openBrackets(text) {
3428
3506
  function configEntries(text, toml) {
3429
3507
  const lines = text.split(/\r?\n/);
3430
3508
  const entries = [];
3431
- let section = "";
3509
+ const elements = /* @__PURE__ */ new Map();
3510
+ let section = "", block = 0;
3432
3511
  for (let i = 0; i < lines.length; i++) {
3433
3512
  const trimmed = lines[i].trim();
3434
3513
  if (!trimmed || trimmed.startsWith("#") || trimmed.startsWith(";")) continue;
3435
- const header = /^\[\[?\s*([^\]]+?)\s*\]\]?\s*(?:[#;].*)?$/.exec(trimmed);
3514
+ const header = /^(\[\[?)\s*([^\]]+?)\s*\]\]?\s*(?:[#;].*)?$/.exec(trimmed);
3436
3515
  if (header) {
3437
- section = header[1].replace(/["']/g, "");
3516
+ section = header[2].replace(/["']/g, "");
3517
+ block = header[1] === "[[" ? (elements.set(section, (elements.get(section) ?? 0) + 1), elements.get(section)) : 0;
3438
3518
  continue;
3439
3519
  }
3440
3520
  const pair = /^("[^"]*"|'[^']*'|[\w.-]+)\s*=\s*(.*)$/.exec(trimmed);
@@ -3446,14 +3526,16 @@ function configEntries(text, toml) {
3446
3526
  } else {
3447
3527
  while (i + 1 < lines.length && /^\s+\S/.test(lines[i + 1]) && !/^\s*[#;]/.test(lines[i + 1])) value += "\n" + lines[++i].trim();
3448
3528
  }
3449
- entries.push({ section, key: pair[1].replace(/["']/g, ""), value, line: start + 1 });
3529
+ entries.push({ section, key: pair[1].replace(/["']/g, ""), value, line: start + 1, block });
3450
3530
  }
3451
3531
  return entries;
3452
3532
  }
3533
+ var withoutComments = (value) => value.replace(/"(?:\\.|[^"\\\n])*"|'[^'\n]*'|[#;][^\n]*/g, (found) => found[0] === "#" || found[0] === ";" ? "" : found);
3453
3534
  function members(value) {
3454
- const quoted = [...value.matchAll(/"((?:\\.|[^"\\\n])*)"|'([^'\n]*)'/g)].map((match) => (match[1] ?? match[2]).trim()).filter(Boolean);
3535
+ const code = withoutComments(value);
3536
+ const quoted = [...code.matchAll(/"((?:\\.|[^"\\\n])*)"|'([^'\n]*)'/g)].map((match) => (match[1] ?? match[2]).trim()).filter(Boolean);
3455
3537
  if (quoted.length) return quoted;
3456
- return value.replace(/[#;].*$/gm, "").split(/[\s,]+/).map((word) => word.trim()).filter(Boolean);
3538
+ return code.split(/[\s,]+/).map((word) => word.trim()).filter(Boolean);
3457
3539
  }
3458
3540
  function covers(selector, code) {
3459
3541
  if (selector === "ALL") return true;
@@ -3504,8 +3586,31 @@ function analyzePythonConfig(file, oldText, newText, report) {
3504
3586
  const base = explicit.length ? explicit : tool === "ruff" ? ["E4", "E7", "E9", "F"] : ["E", "F", "W", "C90"];
3505
3587
  return [...base, ...extra].some((selected2) => covers(selected2, code) || covers(code, selected2));
3506
3588
  };
3589
+ const versionParts = (text) => /^\d{2,}$/.test(text) ? [Number(text[0]), Number(text.slice(1))] : text.split(".").map(Number);
3590
+ const higher = (was2, now) => {
3591
+ for (let index = 0; index < Math.max(was2.length, now.length); index++) if ((now[index] ?? 0) !== (was2[index] ?? 0)) return (now[index] ?? 0) > (was2[index] ?? 0);
3592
+ return false;
3593
+ };
3594
+ const newest = (text, pattern) => {
3595
+ let best = null;
3596
+ for (const match of text.matchAll(pattern)) {
3597
+ const parts = versionParts(match[1]);
3598
+ if (!best || higher(best, parts)) best = parts;
3599
+ }
3600
+ return best;
3601
+ };
3602
+ const versions = [
3603
+ /^\s*requires-python\s*=\s*["'][<>=!~^\s]*(\d+(?:\.\d+)*)/gm,
3604
+ /^\s*target-version\s*=\s*["']?(?:py)?(\d+(?:\.\d+)*)/gm,
3605
+ /^\s*python_version\s*=\s*["']?(\d+(?:\.\d+)*)/gm,
3606
+ /\bruff\s*(?:(?:===|==|~=|>=|>)\s*["']?|=\s*["'][\^~>=]*)(\d+(?:\.\d+)*)/g
3607
+ ];
3608
+ const bumped = versions.some((pattern) => {
3609
+ const was2 = newest(oldText, pattern), now = newest(newText, pattern);
3610
+ return !!was2 && !!now && higher(was2, now);
3611
+ });
3507
3612
  for (const [tool, bucket] of ignored(after)) {
3508
- if (!configured.has(tool)) continue;
3613
+ if (!configured.has(tool) || tool === "ruff" && bumped) continue;
3509
3614
  const old = [...was.get(tool)?.keys() ?? []];
3510
3615
  const fresh = [...bucket].filter(([code]) => !old.some((known) => covers(known, code)) && enforced(tool, code));
3511
3616
  if (fresh.length) report(Math.min(...fresh.map(([, line]) => line)), `${tool} ignore: ${fresh.slice(0, 4).map(([code]) => "+" + code).join(", ")}${fresh.length > 4 ? ", \u2026" : ""}`, "Lint or type checks switched off in the configuration");
@@ -3520,10 +3625,16 @@ function analyzePythonConfig(file, oldText, newText, report) {
3520
3625
  report(line, `${tool} select: ${dropped.slice(0, 4).map((code) => "-" + code).join(", ")}${dropped.length > 4 ? ", \u2026" : ""}`, "Lint rules removed from the selected set");
3521
3626
  }
3522
3627
  }
3523
- const switches = new Map(before.filter((entry) => toolOf(file, entry.section) === "mypy" && MYPY_SWITCHES.test(entry.key)).map((entry) => [`${entry.section}|${entry.key}`, truthy(entry.value)]));
3628
+ const tableOf = (entries) => {
3629
+ const modules = /* @__PURE__ */ new Map();
3630
+ for (const entry of entries) if (entry.block && entry.key === "module") modules.set(`${entry.section}#${entry.block}`, members(entry.value).join(","));
3631
+ return (entry) => entry.block ? `${entry.section}#${modules.get(`${entry.section}#${entry.block}`) ?? entry.block}` : entry.section;
3632
+ };
3633
+ const tableBefore = tableOf(before), tableAfter = tableOf(after);
3634
+ const switches = new Map(before.filter((entry) => toolOf(file, entry.section) === "mypy" && MYPY_SWITCHES.test(entry.key)).map((entry) => [`${tableBefore(entry)}|${entry.key}`, truthy(entry.value)]));
3524
3635
  for (const entry of after) {
3525
3636
  if (toolOf(file, entry.section) !== "mypy" || !MYPY_SWITCHES.test(entry.key)) continue;
3526
- if (switches.get(`${entry.section}|${entry.key}`) === true && !truthy(entry.value)) report(entry.line, `mypy ${entry.key}: true \u2192 false`, "Type checker strictness switched off");
3637
+ if (switches.get(`${tableAfter(entry)}|${entry.key}`) === true && !truthy(entry.value)) report(entry.line, `mypy ${entry.key}: true \u2192 false`, "Type checker strictness switched off");
3527
3638
  }
3528
3639
  const ignoring = (entries) => entries.filter((entry) => toolOf(file, entry.section) === "mypy" && entry.key === "ignore_errors" && truthy(entry.value));
3529
3640
  const before_ = ignoring(before), after_ = ignoring(after);
@@ -3796,6 +3907,12 @@ var EMPTY_PY = /==\s*(?:0(?:\.0+)?(?![\w.])|\[\s*\]|\{\s*\}|""|''|\(\s*\)|(?:set
3796
3907
  var PY_SPECIFIC = /==|!=|<=|>=|\bin\b|\bis\b|\bisinstance\s*\(|\.startswith\s*\(|\.endswith\s*\(|\bcallable\s*\(/;
3797
3908
  var JS_SPECIFIC = /\.(?:toBe|toEqual|toStrictEqual|toHaveLength|toContain|toContainEqual|toMatch|toMatchObject|toHaveProperty|toHaveBeenCalledWith|toHaveBeenLastCalledWith|toHaveBeenNthCalledWith|toBeCalledWith|toHaveBeenCalledTimes|toHaveBeenCalledOnce|toHaveTextContent|toHaveAttribute|toHaveValue|toThrow\w*|toBeGreaterThan\w*|toBeLessThan\w*)\s*\(/;
3798
3909
  var THROWING_QUERY = /\b(?:get|find)(?:All)?By\w*\s*\(/;
3910
+ var PRESENCE_PREDICATE = /\.(?:exists|isVisible|isExisting|isDisplayed)\(\)$/;
3911
+ function samePresence(oldAssertion, newAssertion) {
3912
+ if (!/\.toBe\s*\(\s*true\s*\)\s*;?\s*$/.test(oldAssertion) || !/\.(?:not\s*\.\s*toBeNull|toBeTruthy|toBeDefined|toBeInTheDocument)\s*\(\s*\)\s*;?\s*$/.test(newAssertion)) return false;
3913
+ const was = checkedExpression(oldAssertion, "js"), now = checkedExpression(newAssertion, "js");
3914
+ return !!was && !!now && PRESENCE_PREDICATE.test(was) && (PRESENCE_PREDICATE.test(now) || /(?:querySelector|queryBy\w*|getBy\w*|findBy\w*|\.find|\.get|\$)\(/.test(now));
3915
+ }
3799
3916
  function checkedExpression(assertion, language) {
3800
3917
  const squash = (text) => text.replace(/\s+/g, "").replace(/^len\((.*)\)$/, "$1").replace(/\.length$/, "");
3801
3918
  if (language === "py") {
@@ -3846,12 +3963,33 @@ function admits(value, operator, limit) {
3846
3963
  return true;
3847
3964
  }
3848
3965
  }
3966
+ var EXACT_MATCHERS = ["toBe", "toEqual", "toStrictEqual", "toHaveLength"];
3967
+ function comparedValue(assertion, language, negated) {
3968
+ if (language === "js") {
3969
+ for (const matcher of EXACT_MATCHERS) {
3970
+ const value = matcherArgument(assertion, negated ? `not\\s*\\.\\s*${matcher}` : matcher);
3971
+ if (value !== null) return value;
3972
+ }
3973
+ return null;
3974
+ }
3975
+ const call = new RegExp(`\\bself\\s*\\.\\s*assert${negated ? "NotEqual" : "Equal"}\\s*\\(([\\s\\S]*)\\)\\s*;?\\s*$`).exec(assertion);
3976
+ if (call) return splitArguments(call[1])[1]?.replace(/\s+/g, "") ?? null;
3977
+ const at = assertion.indexOf(negated ? "!=" : "==");
3978
+ return at < 0 ? null : splitArguments(assertion.slice(at + 2))[0]?.replace(/\s+/g, "") ?? null;
3979
+ }
3980
+ var NOTHING_OR_BOOLEAN = /^(?:true|false|True|False|null|undefined|None|NaN|0(?:\.0+)?|""|''|``|\[\]|\{\}|\(\))$/;
3981
+ var LITERAL_VALUE = /^(?:-?\d+(?:\.\d+)?|"(?:[^"\\]|\\.)*"|'(?:[^'\\]|\\.)*'|null|undefined|None|\[\]|\{\}|\(\))$/;
3982
+ function exactBecameExclusion(oldAssertion, newAssertion, language) {
3983
+ const pinned = comparedValue(oldAssertion, language, false), ruledOut = comparedValue(newAssertion, language, true);
3984
+ return pinned !== null && ruledOut !== null && pinned !== "" && !NOTHING_OR_BOOLEAN.test(pinned) && LITERAL_VALUE.test(ruledOut) && pinned !== ruledOut;
3985
+ }
3849
3986
  function relaxes(oldAssertion, newAssertion, language, siblings = []) {
3850
3987
  const subject = checkedExpression(oldAssertion, language);
3851
3988
  if (!subject || subject !== checkedExpression(newAssertion, language)) return false;
3852
3989
  if (siblings.some((other) => isExact(other, language) && checkedExpression(other, language) === subject)) return false;
3853
3990
  if (language === "js") {
3854
- if (/\.\s*not\s*\./.test(oldAssertion) || /\.\s*not\s*\./.test(newAssertion)) return false;
3991
+ const negated = (text) => /\.\s*not\s*\./.test(text);
3992
+ if (negated(oldAssertion) || negated(newAssertion)) return !negated(oldAssertion) && exactBecameExclusion(oldAssertion, newAssertion, language);
3855
3993
  const exact = /\.(?:toBe|toEqual|toStrictEqual|toHaveLength)\s*\(\s*(-?\d+(?:\.\d+)?)\s*\)/.exec(oldAssertion)?.[1] ?? /assert\s*\.\s*(?:equal|strictEqual|deepEqual|deepStrictEqual)\s*\([^,]+,\s*(-?\d+(?:\.\d+)?)\s*[,)]/.exec(oldAssertion)?.[1];
3856
3994
  const bound = /\.(?:toBeGreaterThan|toBeGreaterThanOrEqual|toBeLessThan|toBeLessThanOrEqual)\s*\(/.test(newAssertion) || /assert\s*\.\s*ok\s*\([^;]*(?:>=|<=|>|<)/.test(newAssertion);
3857
3995
  if (exact !== void 0 && Number(exact) !== 0 && bound) {
@@ -3871,6 +4009,8 @@ function relaxes(oldAssertion, newAssertion, language, siblings = []) {
3871
4009
  const operator = (text) => /^\s*assert\s+(?:not\s+)?.+?\s*(==|!=|>=|<=|>|<|\bin\b|\bis\b)/.exec(text)?.[1] ?? "";
3872
4010
  const exact = operator(oldAssertion) === "==" && plain(oldAssertion) || /\bself\s*\.\s*assertEqual\s*\(/.test(oldAssertion);
3873
4011
  const bound = [">=", "<=", ">", "<"].includes(operator(newAssertion)) && plain(newAssertion) || /\bself\s*\.\s*assert(?:Greater|Less)(?:Equal)?\s*\(/.test(newAssertion);
4012
+ const unequal = operator(newAssertion) === "!=" && plain(newAssertion) && !/^\s*assert\s+not\b/.test(newAssertion) || /\bself\s*\.\s*assertNotEqual\s*\(/.test(newAssertion);
4013
+ if (exact && unequal) return exactBecameExclusion(oldAssertion, newAssertion, language);
3874
4014
  if (!exact || !bound) return false;
3875
4015
  const value = (text) => /==\s*(-?\d+(?:\.\d+)?|[\w.]+(?:\([^)]*\))?)\s*(?:,|$)/.exec(text)?.[1] ?? /\bassertEqual\s*\([^,]+,\s*(-?\d+(?:\.\d+)?|[\w.]+)\s*[,)]/.exec(text)?.[1];
3876
4016
  const limit = (text) => {
@@ -3887,6 +4027,34 @@ function relaxes(oldAssertion, newAssertion, language, siblings = []) {
3887
4027
  }
3888
4028
  return false;
3889
4029
  }
4030
+ function splitArguments(text) {
4031
+ const parts = [];
4032
+ let depth = 0, quote = "", start = 0;
4033
+ for (let i = 0; i < text.length; i++) {
4034
+ const char = text[i];
4035
+ if (quote) {
4036
+ if (char === "\\") i++;
4037
+ else if (char === quote) quote = "";
4038
+ } else if (char === '"' || char === "'") quote = char;
4039
+ else if ("([{".includes(char)) depth++;
4040
+ else if (")]}".includes(char)) depth--;
4041
+ else if (char === "," && depth === 0) {
4042
+ parts.push(text.slice(start, i).trim());
4043
+ start = i + 1;
4044
+ }
4045
+ }
4046
+ parts.push(text.slice(start).trim());
4047
+ return parts.filter(Boolean);
4048
+ }
4049
+ function unittestAsAssert(statement) {
4050
+ const call = /^\s*self\s*\.\s*assert(True|IsNotNone|GreaterEqual|Greater)\s*\(([\s\S]*)\)\s*;?\s*$/.exec(statement);
4051
+ if (!call) return null;
4052
+ const [first, second] = splitArguments(call[2]);
4053
+ if (!first) return null;
4054
+ if (call[1] === "True") return `assert ${first}`;
4055
+ if (call[1] === "IsNotNone") return `assert ${first} is not None`;
4056
+ return second === void 0 ? null : `assert ${first} ${call[1] === "Greater" ? ">" : ">="} ${second}`;
4057
+ }
3890
4058
  function pinsLess(oldAssertion, newAssertion, language) {
3891
4059
  if (language === "js") {
3892
4060
  const was = /\.toThrow(?:Error)?\s*\(\s*([^)\s][^)]*)\)/.exec(oldAssertion);
@@ -3909,6 +4077,7 @@ function pinsLess(oldAssertion, newAssertion, language) {
3909
4077
  }
3910
4078
  function isWeaker(rawOld, rawNew, language) {
3911
4079
  const oldAssertion = language === "py" ? withoutOuterParens(rawOld) : rawOld, newAssertion = language === "py" ? withoutOuterParens(rawNew) : rawNew;
4080
+ if (language === "js" && samePresence(oldAssertion, newAssertion)) return false;
3912
4081
  if (pinsLess(oldAssertion, newAssertion, language) || relaxes(oldAssertion, newAssertion, language)) return true;
3913
4082
  if (language === "js") {
3914
4083
  if (THROWING_QUERY.test(newAssertion) || !JS_SPECIFIC.test(oldAssertion) || WEAK_JS.test(oldAssertion) || BOUND_JS.test(oldAssertion)) return false;
@@ -3916,7 +4085,8 @@ function isWeaker(rawOld, rawNew, language) {
3916
4085
  return WEAK_JS.test(newAssertion) || BOUND_JS.test(newAssertion) && !ZERO_JS.test(oldAssertion) && (limit === void 0 || /^-?\d+(?:\.\d+)?$/.test(limit.trim()));
3917
4086
  }
3918
4087
  if (language === "py") {
3919
- if (WEAK_PY.test(oldAssertion) || BOUND_PY.test(oldAssertion) || !PY_SPECIFIC.test(oldAssertion)) return false;
4088
+ const oldBare = unittestAsAssert(oldAssertion) ?? oldAssertion;
4089
+ if (WEAK_PY.test(oldAssertion) || BOUND_PY.test(oldAssertion) || WEAK_PY.test(oldBare) || BOUND_PY.test(oldBare) || !PY_SPECIFIC.test(oldAssertion)) return false;
3920
4090
  const limit = /(?:>=|<=|>|<)\s*([^\s,)]+)/.exec(oldAssertion)?.[1];
3921
4091
  return WEAK_PY.test(newAssertion) || BOUND_PY.test(newAssertion) && !EMPTY_PY.test(oldAssertion) && /==|[<>]=?/.test(oldAssertion) && (limit === void 0 || /^-?\d+(?:\.\d+)?$/.test(limit));
3922
4092
  }
@@ -4105,10 +4275,10 @@ function markedTest(finding2, lines) {
4105
4275
  if (/\.py$/i.test(finding2.file ?? "")) {
4106
4276
  if (/^\s*pytestmark\b/.test(snippet)) return null;
4107
4277
  if (/imperative/.test(finding2.reason)) {
4108
- const indent2 = (lines[index] ?? "").search(/\S/);
4278
+ const indent = (lines[index] ?? "").search(/\S/);
4109
4279
  for (let i = index - 1; i >= 0 && i > index - 200; i--) {
4110
4280
  const def = /^(\s*)(?:async\s+)?def\s+(\w+)\s*\(/.exec(lines[i]);
4111
- if (def && def[1].length < indent2) return { key: `py:${def[2]}`, title: def[2] };
4281
+ if (def && def[1].length < indent) return { key: `py:${def[2]}`, title: def[2] };
4112
4282
  }
4113
4283
  return null;
4114
4284
  }
@@ -4133,18 +4303,25 @@ function markedTest(finding2, lines) {
4133
4303
  return null;
4134
4304
  }
4135
4305
  if (/\.each\b/.test(snippet)) return null;
4136
- const title = /\b(?:it|test|describe|suite|xit|xtest|xdescribe|fit|fdescribe)\b[\w$.\s]*\(\s*(['"`])((?:\\.|(?!\1)[^\\\n])+)\1\s*[,)]/.exec(snippet);
4306
+ const title = /\b(?:it|test|describe|suite|xit|xtest|xdescribe|fit|fdescribe)\b[\w$.\s]*\(\s*(['"`])((?:\\.|(?!\1)[^\\\n])+)\1\s*(?:[,)]|$)/.exec(titleOnNextLine(snippet, lines, index));
4137
4307
  if (title) return { key: `js:${title[1]}${title[2]}${title[1]}`, title: `${title[1]}${title[2]}${title[1]}` };
4138
- const indent = (lines[index] ?? "").search(/\S/);
4139
- if (indent > 0 && /\b(?:test|it)\s*\.\s*(?:skip|fixme|fail)\s*\(/.test(snippet)) {
4140
- for (let i = index - 1; i >= 0 && i > index - 200; i--) {
4141
- if (!lines[i].trim() || lines[i].search(/\S/) >= indent) continue;
4142
- const owner = /^\s*(?:test|it)(?:\s*\.\s*\w+)*\s*\(\s*(['"`])((?:\\.|(?!\1)[^\\\n])+)\1/.exec(lines[i]);
4143
- return owner ? { key: `js:${owner[1]}${owner[2]}${owner[1]}`, title: `${owner[1]}${owner[2]}${owner[1]}` } : null;
4308
+ let limit = (lines[index] ?? "").search(/\S/);
4309
+ if (limit > 0 && /\b(?:test|it)\s*\.\s*(?:skip|fixme|fail)\s*\(/.test(snippet)) {
4310
+ for (let i = index - 1; i >= 0 && i > index - 200 && limit > 0; i--) {
4311
+ if (!lines[i].trim() || lines[i].search(/\S/) >= limit) continue;
4312
+ limit = lines[i].search(/\S/);
4313
+ const owner = /^\s*(?:test|it)(?:\s*\.\s*(?!describe\b|step\b)\w+)*\s*\(\s*(['"`])((?:\\.|(?!\1)[^\\\n])+)\1/.exec(titleOnNextLine(lines[i], lines, i));
4314
+ if (owner) return { key: `js:${owner[1]}${owner[2]}${owner[1]}`, title: `${owner[1]}${owner[2]}${owner[1]}` };
4315
+ if (/^\s*(?:await\s+)?(?:test|it)\s*\.\s*describe\b|^\s*(?:describe|suite)\b/.test(lines[i])) return null;
4144
4316
  }
4145
4317
  }
4146
4318
  return null;
4147
4319
  }
4320
+ function titleOnNextLine(text, lines, index) {
4321
+ if (!/\(\s*$/.test(text)) return text;
4322
+ const next = (lines[index + 1] ?? "").trim();
4323
+ return /^(['"`])/.test(next) ? `${text} ${next}` : text;
4324
+ }
4148
4325
  function newTestMarkers(before, after, oldText, newText) {
4149
4326
  const markers = after.filter(isTestMarker);
4150
4327
  if (!markers.length || oldText === null) return [];
@@ -4200,32 +4377,111 @@ var THRESHOLD_PATTERNS = [
4200
4377
  var SWITCHES = ["strict", "noImplicitAny", "strictNullChecks", "noImplicitReturns", "useUnknownInCatchVariables"];
4201
4378
  var GATE_SCRIPT = /^(?:pre|post)?(?:test|lint|typecheck|type-check|check|ci|verify|validate)(?::[\w:.-]+)?$/;
4202
4379
  var CI_RUNNER = new RegExp(
4203
- String.raw`(?<![\w./-])(?:pytest|vitest|jest|mocha|tox|nox)(?![\w/-])` + String.raw`|(?<![\w./-])(?:npm|yarn|pnpm|bun|go|cargo|mvn|gradle|make|dotnet|flutter)\s+(?:run\s+)?(?:-\S+\s+)*test(?![\w/])` + // Monorepo runners: `nx affected --target=test`, `turbo run test`, `lerna run test`, `bazel test //...`.
4204
- String.raw`|(?<![\w./-])(?:nx|turbo|lerna|rush)\s+(?:run-many|affected|run|test)\b[^\n]*(?<![\w-])(?:test|tests)(?![\w-])` + String.raw`|(?<![\w./-])(?:bazel\s+test|ctest|phpunit|rspec|deno\s+test)(?![\w/-])` + String.raw`|\brun:.*(?<=\s)(?:tests?|unit|e2e)(?![\w/.-])`
4380
+ String.raw`(?<![\w./-])(?:pytest|vitest|jest|mocha|tox|nox)(?![\w/-])` + String.raw`|(?<![\w./-])(?:npm|yarn|pnpm|bun|go|cargo|mvn|gradle|make|dotnet|flutter)\s+(?:run\s+)?(?:-\S+\s+)*test(?![\w/])` + // A package script that runs the tests with coverage: `yarn coverage`, `npm run coverage`.
4381
+ String.raw`|(?<![\w./-])(?:npm|yarn|pnpm|bun)\s+(?:run\s+)?coverage(?![\w/-])` + // Monorepo runners: `nx affected --target=test`, `turbo run test`, `lerna run test`, `bazel test //...`.
4382
+ String.raw`|(?<![\w./-])(?:nx|turbo|lerna|rush)\s+(?:run-many|affected|run|test)\b[^\n]*(?<![\w-])(?:test|tests)(?![\w-])` + String.raw`|(?<![\w./-])(?:bazel\s+test|ctest|phpunit|rspec|deno\s+test)(?![\w/-])` + // Python's own runners and the browser runners: `python3 -m unittest`, `nosetests`, `playwright test`, `cypress run`, `karma start`.
4383
+ String.raw`|(?<![\w./-])python[\d.]*\s+-m\s+(?:unittest|pytest|nose2?)(?![\w/-])` + String.raw`|(?<![\w./-])(?:nosetests|nose2|py\.test|stestr|behave|jasmine|playwright\s+test|cypress\s+run|karma\s+start|hatch\s+(?:run\s+)?test|swift\s+test)(?![\w/-])` + // Task runners: `deno task test`, `just test`, `rake test`, `composer test`, `mix test`, `sbt test`, `./gradlew test`, and Node's own `node --test`.
4384
+ String.raw`|(?<![\w./-])(?:deno\s+(?:task|run)|just|task|poe|rake|composer(?:\s+run)?|mix|sbt|lein|stack|cabal|(?:\./)?(?:gradlew|mvnw)(?:\.bat|\.cmd)?)\s+(?:-\S+\s+)*test(?![\w/-])` + String.raw`|(?<![\w./-])(?:node|tsx)(?:\s+-\S+)*\s+--test(?![\w-])` + // A project's own test script, run through its interpreter or by path: `python scripts/run_tests.py`, `./scripts/test.sh`, `python tests/test_cli.py`.
4385
+ String.raw`|(?<![\w./-])(?:(?:python[\d.]*|node|bash|sh|zsh|ruby|pwsh|tsx|ts-node)(?:\s+-\S+)*\s+|\./)(?:[\w.-]+/)*(?:(?:run[_-]?)?tests?(?:[_-](?:all|suite|runner|unit|ci))?|test_[\w-]+|[\w-]+_test)\.(?:py|sh|bash|js|mjs|cjs|ts|rb|ps1)(?![\w-])` + String.raw`|\brun:.*(?<=\s)(?:tests?|unit|e2e)(?![\w/.-])`
4205
4386
  );
4387
+ var SHELL_TEST = /(?:^|run:\s*|[;&|({]\s*|\b(?:if|then|do|else|elif|while|until)\s+|!\s+)test\s+!?\s*-[bcdefghkLnOGprsStuwxzN](?=\s)/g;
4388
+ var ciRuns = (line) => CI_RUNNER.test(line.replace(SHELL_TEST, " ").replace(/\s#.*$/, ""));
4206
4389
  var CI_FILE = /(?:\.github\/workflows\/|\.gitlab-ci)/;
4207
4390
  var CI_REUSABLE = /^\s*-?\s*uses:\s*\S*\.github\/workflows\/\S+/;
4208
4391
  var CI_GATE = /^\s*(?:pull_request|pull_request_target|push|merge_group)\s*:|^\s*on\s*:\s*(?:\[[^\]]*\b(?:push|pull_request|pull_request_target|merge_group)\b[^\]]*\]|(?:push|pull_request|pull_request_target|merge_group)\s*$)|^\s*-\s*(?:push|pull_request|pull_request_target|merge_group)\s*$/m;
4209
- function gatingTestLines(file, text) {
4210
- return /\.gitlab-ci/.test(file) || CI_GATE.test(text) ? matchingLines(text, CI_RUNNER) : 0;
4392
+ var CI_NON_GATE = /^\s*(?:schedule|release|workflow_dispatch|workflow_run|repository_dispatch)\s*:/m;
4393
+ function gatesChanges(file, text) {
4394
+ const section = triggerSection(text), triggers = gatingTriggers(text) ?? text;
4395
+ if (/\.gitlab-ci/.test(file) || CI_GATE.test(triggers) || /\bworkflow_call\b/.test(triggers)) return true;
4396
+ return section !== null && triggers !== section ? false : !CI_NON_GATE.test(triggers);
4397
+ }
4398
+ function gatingTriggers(text) {
4399
+ const section = triggerSection(text);
4400
+ if (section === null) return null;
4401
+ const lines = section.split(/\r?\n/), kept = [];
4402
+ for (let i = 0; i < lines.length; i++) {
4403
+ const head = /^(\s*)push\s*:\s*(?:#.*)?$/.exec(lines[i]);
4404
+ if (!head) {
4405
+ kept.push(lines[i]);
4406
+ continue;
4407
+ }
4408
+ let end = i + 1;
4409
+ while (end < lines.length && (!lines[end].trim() || lines[end].search(/\S/) > head[1].length)) end++;
4410
+ const block = lines.slice(i + 1, end).join("\n");
4411
+ if (!(/^\s*tags\s*:/m.test(block) && !/^\s*branches(?:-ignore)?\s*:/m.test(block))) kept.push(...lines.slice(i, end));
4412
+ i = end - 1;
4413
+ }
4414
+ return kept.join("\n");
4415
+ }
4416
+ function triggerSection(text) {
4417
+ const lines = text.split(/\r?\n/);
4418
+ const start = lines.findIndex((line) => /^["']?on["']?\s*:/.test(line));
4419
+ if (start < 0) return null;
4420
+ let end = start + 1;
4421
+ while (end < lines.length && (!lines[end].trim() || /^[\s#-]/.test(lines[end]))) end++;
4422
+ return lines.slice(start, end).join("\n");
4423
+ }
4424
+ function gatingTestCommands(file, text) {
4425
+ if (!(/\.gitlab-ci/.test(file) || CI_GATE.test(gatingTriggers(text) ?? text))) return [];
4426
+ const directories = [...new Set([...text.matchAll(/^\s*(?:-\s+)?working-directory\s*:\s*["']?([^\s"'#]+)/gm)].map((match) => match[1]))].sort().join(",");
4427
+ return text.split(/\r?\n/).filter((line) => !line.trim().startsWith("#") && ciRuns(line)).map((line) => `${line.replace(/\s#.*$/, "").replace(/^\s*-?\s*(?:run\s*:\s*)?/, "").replace(/\s+/g, " ").trim()} @${directories}`);
4211
4428
  }
4212
4429
  function matchingLines(text, pattern) {
4213
- return text.split(/\r?\n/).filter((line) => !line.trim().startsWith("#") && pattern.test(line)).length;
4430
+ const matches = typeof pattern === "function" ? pattern : (line) => pattern.test(line);
4431
+ return text.split(/\r?\n/).filter((line) => !line.trim().startsWith("#") && matches(line)).length;
4214
4432
  }
4215
4433
  function ciTestSteps(file, oldText, newText) {
4216
4434
  if (!CI_FILE.test(file) || oldText === null && newText === null) return null;
4217
4435
  const was = oldText ?? "", now = newText ?? "";
4218
- const after = gatingTestLines(file, now);
4219
- return { file, before: gatingTestLines(file, was), after, reusableAfter: matchingLines(now, CI_REUSABLE), ungatedAfter: after === 0 ? matchingLines(now, CI_RUNNER) : 0 };
4436
+ const commandsBefore = gatingTestCommands(file, was), commandsAfter = gatingTestCommands(file, now);
4437
+ const after = commandsAfter.length;
4438
+ return { file, before: commandsBefore.length, after, reusableAfter: matchingLines(now, CI_REUSABLE), ungatedAfter: after === 0 ? matchingLines(now, ciRuns) : 0, commandsBefore, commandsAfter };
4220
4439
  }
4221
- function resolveCiTestSteps(counts, commit) {
4440
+ function resolveCiTestSteps(counts, commit, elsewhere = /* @__PURE__ */ new Set()) {
4222
4441
  const before = counts.reduce((sum, item) => sum + item.before, 0);
4223
4442
  const after = counts.reduce((sum, item) => sum + item.after + item.reusableAfter, 0);
4224
4443
  if (after >= before) return [];
4225
- return counts.filter((item) => item.before > 0 && item.after === 0 && item.reusableAfter === 0).map((item) => item.ungatedAfter > 0 ? finding("guard-weakened", item.file, 1, "tests no longer run on push or pull_request", "Workflow keeps its tests but no longer runs on pushes or pull requests", commit) : finding("guard-weakened", item.file, 1, "test command no longer runs in CI", "Test step removed from CI", commit));
4444
+ const running = /* @__PURE__ */ new Set([...counts.flatMap((item) => item.commandsAfter ?? []), ...elsewhere]);
4445
+ return counts.filter((item) => item.before > 0 && item.after === 0 && item.reusableAfter === 0).filter((item) => item.commandsBefore === void 0 || item.commandsBefore.some((command) => !running.has(command))).map((item) => item.ungatedAfter > 0 ? finding("guard-weakened", item.file, 1, "tests no longer run on push or pull_request", "Workflow keeps its tests but no longer runs on pushes or pull requests", commit) : finding("guard-weakened", item.file, 1, "test command no longer runs in CI", "Test step removed from CI", commit));
4226
4446
  }
4227
4447
  var LENIENT = /continue-on-error:\s*true|allow_failure:\s*true|^\s*(?:-\s+)?if:\s*(?:\$\{\{\s*)?false\s*(?:\}\})?\s*(?:#.*)?$/;
4228
4448
  var SHELL_SWALLOW = /\|\|\s*(?:true|exit\s+0|:|echo\b)|;\s*(?:true|exit\s+0)\b/;
4449
+ var NON_CHECK_ACTION = new RegExp(String.raw`^\s*(?:-\s+)?uses:\s*["']?(?:` + [
4450
+ String.raw`actions/(?:upload-artifact|download-artifact|upload-pages-artifact|deploy-pages|cache|attest[\w-]*|checkout|setup-[\w-]+)`,
4451
+ String.raw`docker/(?:login-action|metadata-action|build-push-action|setup-[\w-]+)`,
4452
+ String.raw`azure/login`,
4453
+ String.raw`aws-actions/(?:configure-aws-credentials|amazon-ecr-login)`,
4454
+ String.raw`google-github-actions/(?:auth|setup-gcloud)`,
4455
+ String.raw`softprops/action-gh-release`,
4456
+ String.raw`ncipollo/release-action`,
4457
+ String.raw`pypa/gh-action-pypi-publish`,
4458
+ String.raw`JS-DevTools/npm-publish`,
4459
+ String.raw`peaceiris/actions-gh-pages`,
4460
+ String.raw`peter-evans/[\w-]+`,
4461
+ String.raw`marocchino/sticky-pull-request-comment`,
4462
+ String.raw`slackapi/slack-github-action`,
4463
+ String.raw`codecov/codecov-action`,
4464
+ String.raw`coverallsapp/github-action`
4465
+ ].join("|") + String.raw`)(?:/[\w./-]+)?@`, "i");
4466
+ function scopeCommands(scope) {
4467
+ const found = [];
4468
+ scope.forEach((line, index) => {
4469
+ const head = /^(\s*(?:-\s+)?)(?:run|uses)\s*:\s*(.*)$/.exec(line);
4470
+ if (!head || line.trim().startsWith("#")) return;
4471
+ const text = head[2].replace(/\s#.*$/, "").trim();
4472
+ if (text && !/^[|>][+-]?$/.test(text)) {
4473
+ found.push(text.replace(/\s+/g, " "));
4474
+ return;
4475
+ }
4476
+ const keyAt = head[1].length;
4477
+ for (let next = index + 1; next < scope.length; next++) {
4478
+ if (!scope[next].trim()) continue;
4479
+ if (scope[next].length - scope[next].trimStart().length <= keyAt) break;
4480
+ if (!scope[next].trim().startsWith("#")) found.push(scope[next].trim().replace(/\s+/g, " "));
4481
+ }
4482
+ });
4483
+ return found;
4484
+ }
4229
4485
  function lenientSteps(text) {
4230
4486
  const lines = text.split(/\r?\n/);
4231
4487
  const indentOf = (line) => line.length - line.trimStart().length;
@@ -4242,9 +4498,20 @@ function lenientSteps(text) {
4242
4498
  while (j >= 0 && (!lines[j].trim() || lines[j].trim().startsWith("#") || indentOf(lines[j]) >= indentOf(line))) j--;
4243
4499
  identity = j >= 0 ? lines[j].trim() : "";
4244
4500
  }
4501
+ const start = /^\s*-\s+if:/.test(line) ? i : Math.max(j, 0), level = indentOf(lines[start]);
4502
+ let end = start + 1;
4503
+ while (end < lines.length && (!lines[end].trim() || lines[end].trim().startsWith("#") || indentOf(lines[end]) > level)) end++;
4504
+ const scope = lines.slice(start, end), runsChecks = matchingLines(scope.join("\n"), ciRuns) > 0;
4505
+ const uses = scope.filter((item) => /^\s*(?:-\s+)?uses:/.test(item));
4506
+ const nonCheck = !runsChecks && uses.length === 1 && NON_CHECK_ACTION.test(uses[0]) && !scope.some((item) => /^\s*(?:-\s+)?run:/.test(item));
4507
+ const commands = scopeCommands(scope);
4245
4508
  const entry = found.get(identity);
4246
- if (entry) entry.count++;
4247
- else found.set(identity, { count: 1, line: i + 1 });
4509
+ if (entry) {
4510
+ entry.count++;
4511
+ entry.runsChecks ||= runsChecks;
4512
+ entry.nonCheck &&= nonCheck;
4513
+ entry.commands.push(...commands);
4514
+ } else found.set(identity, { count: 1, line: i + 1, runsChecks, nonCheck, commands });
4248
4515
  });
4249
4516
  return found;
4250
4517
  }
@@ -4306,16 +4573,22 @@ function analyzeGuardFile(file, oldText, newText, commit) {
4306
4573
  });
4307
4574
  }
4308
4575
  if (CI_FILE.test(file)) {
4576
+ const gating = gatesChanges(file, oldText) || gatesChanges(file, newText);
4309
4577
  const before = lenientSteps(oldText), existing = new Set(oldText.split(/\r?\n/).flatMap((line) => [line.trim(), line.trim().replace(/^-\s+/, "")]));
4310
4578
  const newLines = newText.split(/\r?\n/);
4579
+ const toleratedPlaces = (text) => text.split(/\r?\n/).filter((line) => !line.trim().startsWith("#") && /continue-on-error:\s*true|allow_failure:\s*true/.test(line)).length;
4580
+ const toleratedBefore = new Set([...before.values()].flatMap((step) => step.commands));
4581
+ const moved = toleratedPlaces(newText) <= toleratedPlaces(oldText);
4311
4582
  for (const [identity, now2] of lenientSteps(newText)) {
4583
+ if (now2.nonCheck || !gating && !now2.runsChecks) continue;
4584
+ if (moved && now2.commands.length > 0 && now2.commands.every((command) => toleratedBefore.has(command))) continue;
4312
4585
  if (identity && existing.has(identity) && now2.count > (before.get(identity)?.count ?? 0)) {
4313
4586
  const marker = (newLines[now2.line - 1] ?? "").trim();
4314
4587
  out.push(finding("guard-weakened", file, now2.line, marker, /^(?:-\s+)?if:/.test(marker) ? "CI step is now switched off" : "CI step is now allowed to fail", commit));
4315
4588
  break;
4316
4589
  }
4317
4590
  }
4318
- const swallowing = (text) => text.split(/\r?\n/).map((line, index) => ({ line: line.trim(), index })).filter((item) => !item.line.startsWith("#") && CI_RUNNER.test(item.line) && SHELL_SWALLOW.test(item.line));
4591
+ const swallowing = (text) => text.split(/\r?\n/).map((line, index) => ({ line: line.trim(), index })).filter((item) => !item.line.startsWith("#") && ciRuns(item.line) && SHELL_SWALLOW.test(item.line));
4319
4592
  const was = swallowing(oldText), now = swallowing(newText);
4320
4593
  const tolerance = (text) => text.split(/\r?\n/).filter((line) => !line.trim().startsWith("#") && /continue-on-error:\s*true|allow_failure:\s*true/.test(line)).length;
4321
4594
  const traded = Math.max(0, tolerance(oldText) - tolerance(newText));
@@ -4520,7 +4793,23 @@ async function compareSnapshot(options, cwd, snapshot) {
4520
4793
  await snapshot?.verify();
4521
4794
  result.complete = result.skippedFiles.length === 0;
4522
4795
  const rules = policy.config.rules ?? {};
4523
- result.infractions.push(...[...resolveTampering(tamperingFiles, productionChanged), ...guardFindings, ...resolveCiTestSteps(ciCounts, refs.headSha)].filter((finding2) => rules[finding2.type] !== "off"));
4796
+ const elsewhere = /* @__PURE__ */ new Set();
4797
+ if (ciCounts.some((item) => item.before > 0 && item.after === 0 && item.reusableAfter === 0)) {
4798
+ const touched = new Set(changes.flatMap((change) => [change.oldPath, change.newPath]));
4799
+ const tree = /^[a-f0-9]{40,64}$/.test(refs.headSha) ? refs.headSha : refs.baseSha;
4800
+ try {
4801
+ for (const file of (await ciFilesAt(cwd, tree, options.signal)).filter((candidate) => !touched.has(candidate.path)).slice(0, 100)) {
4802
+ try {
4803
+ for (const command of gatingTestCommands(file.path, new TextDecoder().decode(await readBlob(cwd, file.oid, options.signal, 512 * 1024)))) elsewhere.add(command);
4804
+ } catch (error) {
4805
+ assertNotAborted(options.signal);
4806
+ }
4807
+ }
4808
+ } catch (error) {
4809
+ assertNotAborted(options.signal);
4810
+ }
4811
+ }
4812
+ result.infractions.push(...[...resolveTampering(tamperingFiles, productionChanged), ...guardFindings, ...resolveCiTestSteps(ciCounts, refs.headSha, elsewhere)].filter((finding2) => rules[finding2.type] !== "off"));
4524
4813
  const exceptions = applyExceptions(result.infractions, options.ignoreExceptions ? [] : policy.config.exceptions, /* @__PURE__ */ new Date(), existingCounts);
4525
4814
  result.infractions = exceptions.findings;
4526
4815
  result.suppressedFindings = exceptions.suppressedFindings;
package/dist/cli.js CHANGED
@@ -38,7 +38,7 @@ import {
38
38
  terminalText,
39
39
  uninstallHooks,
40
40
  writeBaseline
41
- } from "./chunk-2KWC2IAP.js";
41
+ } from "./chunk-BNDNXKBM.js";
42
42
 
43
43
  // src/cli.ts
44
44
  import { Command } from "commander";
package/dist/index.js CHANGED
@@ -91,7 +91,7 @@ import {
91
91
  unquoteGitPath,
92
92
  validateExceptions,
93
93
  writeBaseline
94
- } from "./chunk-2KWC2IAP.js";
94
+ } from "./chunk-BNDNXKBM.js";
95
95
  export {
96
96
  AGENT_LABELS,
97
97
  CONFIG_FILENAMES,
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "agent-roast",
3
- "version": "0.18.0",
3
+ "version": "0.20.0",
4
4
  "description": "Stops AI coding agents (and rushed humans) from turning a build green by weakening tests, coverage gates or CI. Hook for Claude Code, Codex and Cursor, pre-commit, GitHub Action. Offline.",
5
5
  "mcpName": "io.github.Nari-Ab/agent-roast",
6
6
  "type": "module",