agent-roast 0.18.0 → 0.19.0

package/README.md

@@ -109,23 +109,26 @@ old debt with a baseline: [docs/reference.md](docs/reference.md#configuration).
 ## Does it work?
 
 Measured, with the denominators and the caveats in [docs/benchmarks.md](docs/benchmarks.md). **Weakened checks are rare, mostly
-deliberate, and not specific to AI. agent-roast is a tripwire that asks about them. On repositories it had not seen, about half of its
-blocks were questions with a harmless answer.**
+deliberate, and not specific to AI. agent-roast is a tripwire that asks about them. On repositories it had not seen, between a quarter and
+two thirds of its blocks were questions with a harmless answer, depending on the sample.**
 
-- **131 of 131** hand-labeled cheating edits are caught and **110 of 110** harmless look-alikes (reformats, moves, raised
+- **145 of 145** hand-labeled cheating edits are caught and **127 of 127** harmless look-alikes (reformats, moves, raised
   thresholds, environment-gated skips) stay silent. The cases were written with the detector, so they guard against regressions;
   they do not prove accuracy. One gap is documented and still open.
 - **12,645 commits** that touch tests or CI/config in 166 public repositories: blocking rules fired on 0.3% of AI-signed commits
   (Claude, Copilot, Cursor, Codex, Devin, aider) and on 0.3% of the others. Inside the same repositories the odds for an
   AI-signed commit were 0.81 times those of another commit (95% CI 0.40–1.67). There is no sign that AI-signed commits weaken
   checks more often.
-- **On repositories it was not written from, about half of the blocked commits were false alarms.** Four samples of 39 or 40 public
-  repositories (158 in all, 44,878 commits) were replayed and every blocked commit was read. 0.17.0 blocked 74 of 11,604 commits in the
-  first sample and 65 of 10,726 in the fourth, and 69% of both were false alarms, mostly a skip on a test that the same commit adds.
-  This version, frozen before the fourth sample was drawn, blocked 41 of its 10,726 commits: 20 changed a check a reviewer should see
-  (an assertion that got weaker, a gate that was relaxed, a test switched off or left focused) and 21 were false alarms, nine of them in
-  one repository. Most of the rest, in the earlier measurements too, were deliberate and explained. One reader (an AI assistant) sorted
-  them, so treat the figures as a first estimate ([section 9](docs/benchmarks.md#9-repositories-nobody-had-tuned-on)).
+- **On repositories it was not written from, between a quarter and two thirds of the blocked commits were false alarms.** Six samples of 39
+  or 40 public repositories (238 in all, 67,843 commits) were replayed and every blocked commit was read. The share was 69% for 0.17.0 on the
+  first sample and on the fourth, 51% for 0.18.0 on the fourth and 27% on the fifth, which it had not seen either. This version, frozen before the
+  sixth sample was drawn, blocked 100 of its 11,335 commits: 34 changed a check a reviewer should see (an assertion that got weaker, a gate
+  that was relaxed, a test switched off or left focused) and 66 were false alarms, but 51 of the 66 were one defect in one repository, an array of
+  mypy overrides read as one table, which is fixed in this release; the released build blocks 44 of those commits, 34 and 10. A new
+  test that asserts nothing counts as a block for a reviewer; counted as false alarms, 13 of 33 on the fifth sample and 72 of 100 on the sixth. Most of the rest, in the
+  earlier measurements too, were deliberate and explained. One reader (an AI assistant) sorted them, and the fixes were made from the same
+  sorting, so treat the figures as a first estimate ([section 9](docs/benchmarks.md#9-repositories-nobody-had-tuned-on),
+  [section 10](docs/benchmarks.md#10-two-more-samples-and-what-the-fixes-from-sample-4-did)).
 - **9,732 commits compared in full** from 164 public repositories that use coding agents (5,454 of them AI-signed) were reviewed again for the
   last two releases, and every new finding was read. 0.15.0's rules reported 30 new findings there and 7 were false alarms (fixed in 0.16.0);
   0.17.0's rule for an exact check turned into a bound reported 18 in 14 commits, all of them loosened checks, after two false alarms were

package/dist/{chunk-2KWC2IAP.js → chunk-OAMVMB5N.js}

@@ -1166,7 +1166,8 @@ var PYTHON_TEST_SKIP_PATTERNS = [
   { regex: /\bself\s*\.\s*skipTest\s*\(/, name: "imperative unittest skip (self.skipTest)", imperative: true },
   // A test that is expected to fail cannot fail the run, which is what a skip does too.
   { regex: /(?<![\w.])pytest\s*\.\s*xfail\s*\(/, name: "imperative pytest xfail (pytest.xfail)", imperative: true },
-  { regex: /@(?:unittest\s*\.\s*)?expectedFailure\b/, name: "test expected to fail (@unittest.expectedFailure)", imperative: false }
+  // `@unittest.expectedFailure`, and the same decorator under another module's name (`@test.expectedFailure` of a test framework that re-exports it).
+  { regex: /@(?:[A-Za-z_]\w*\s*\.\s*)*expectedFailure\b/, name: "test expected to fail (@unittest.expectedFailure)", imperative: false }
 ];
 var PYTHON_MARKER = /(@\s*|(?<![\w.])marks\s*=\s*\[?\s*|^[ \t]*pytestmark\s*=\s*\[?\s*)pytest\s*\.\s*mark\s*\.\s*(skip|xfail)\b/gm;
 function isConditionalXfail(args) {
@@ -1317,13 +1318,55 @@ function isGuardedBlockSkip(lines, profile) {
   return compiledConditionally(i + 1);
 }
 var JS_TEST_API = /(?<![\w$.])(?:(?:it|test|describe|suite)(?:\s*\.\s*(?:concurrent|serial|parallel|describe|failing|sequential))*\s*\.\s*(skip|todo|only|fixme|failing|fails|fail)|((?:xit|xtest|xdescribe|fit|fdescribe)))\b((?:\s*\.\s*(?:each|failing|concurrent|serial|parallel|sequential))*)\s*(?=\(|\u0001)/g;
-function isConditionalSkip(source, raw, open2) {
+var MENTIONS_CI = /\bCI\b|\bisCI\b|GITHUB_ACTIONS/;
+function isConditionalSkip(source, raw, open2, start) {
   const args = callArguments(source, raw, open2);
   if (!args) return false;
   const isString = (arg) => /^(['"`])[\s\S]*\1$/.test(arg);
-  if (!args.length || isString(args[0]) || args[0] === "true") return false;
+  if (!args.length || args[0] === "true") return isUnderRuntimeCondition(source, raw, start);
+  if (isString(args[0])) return false;
   if (args.length > 1 && !isString(args.at(-1))) return false;
-  return !/\bCI\b|\bisCI\b|GITHUB_ACTIONS/.test(args[0]);
+  return !MENTIONS_CI.test(args[0]);
+}
+var TEST_REGISTRATION = /^(?:await\s+)?(?:test|it|describe|suite|beforeAll|beforeEach|afterAll|afterEach)\b/;
+function isUnderRuntimeCondition(source, raw, start) {
+  const alwaysTrue = (text) => /^\s*(?:true|1|!\s*0|!\s*false)\s*$/.test(text);
+  const indentOf = (text) => text.length - text.trimStart().length;
+  const lineStartOf = (offset) => Math.max(source.lastIndexOf("\n", offset - 1), source.lastIndexOf("\0", offset - 1)) + 1;
+  const mentionsCi = (from, to) => MENTIONS_CI.test(raw.slice(from, to));
+  const lineStart = lineStartOf(start);
+  const head = source.slice(lineStart, start).trimEnd();
+  if (head.endsWith(")")) {
+    let depth = 0, opening = -1;
+    for (let i = head.length - 1; i >= 0; i--) {
+      if (head[i] === ")") depth++;
+      else if (head[i] === "(" && --depth === 0) {
+        opening = i;
+        break;
+      }
+    }
+    if (opening >= 0 && /(?:^|[\s}])if\s*$/.test(head.slice(0, opening))) {
+      return !mentionsCi(lineStart + opening + 1, lineStart + head.length - 1) && !alwaysTrue(head.slice(opening + 1, -1));
+    }
+  } else if (/(?:^|[\s}])else$/.test(head)) return true;
+  else if (head.endsWith("&&")) return !mentionsCi(lineStart, start) && !alwaysTrue(head.slice(0, -2).replace(/^[\s{}]+/, ""));
+  const lineEnd = source.slice(start).search(/[\n\0]/);
+  let limit = indentOf(source.slice(lineStart, lineEnd < 0 ? void 0 : start + lineEnd)), at = lineStart;
+  for (let steps = 0; steps < 200 && at > 0; steps++) {
+    if (source[at - 1] === "\0") return false;
+    const previous = lineStartOf(at - 1), line = source.slice(previous, at - 1);
+    const lineRaw = raw.slice(previous, at - 1);
+    at = previous;
+    if (!line.trim() || indentOf(line) >= limit) continue;
+    const header = line.trim();
+    if (/^(?:\}\s*)?(?:else\s+)?if\s*\(/.test(header)) {
+      return !MENTIONS_CI.test(lineRaw) && !alwaysTrue(/\bif\s*\((.*)\)\s*\{?\s*$/.exec(header)?.[1] ?? "");
+    }
+    if (/^(?:\}\s*)?(?:else|catch)\b|^(?:case\b|default\s*:)|^\)\s*\{?\s*$/.test(header)) return true;
+    if (TEST_REGISTRATION.test(header)) return false;
+    limit = indentOf(line);
+  }
+  return false;
 }
 function callArguments(source, raw, open2) {
   if (source[open2] !== "(") return null;
@@ -1627,7 +1670,7 @@ function auditFileDiffs(files, commitHash = "HEAD", commitDate = (/* @__PURE__ *
         if (match[2] && previous >= 0 && (/[(,=:?[&|!+\-*%<]/.test(source[previous]) || /(?:await|return|yield|typeof|void)$/.test(source.slice(Math.max(0, previous - 6), previous + 1)))) continue;
         if (/\bfunction$/.test(source.slice(Math.max(0, previous - 8), previous + 1))) continue;
         if (source[match.index + match[0].length] === "
" && !/\.\s*each\s*$/.test(match[3])) continue;
-        if ((match[1] === "skip" || match[1] === "fixme" || match[1] === "fail") && !match[3] && isConditionalSkip(source, raw, match.index + match[0].length)) continue;
+        if ((match[1] === "skip" || match[1] === "fixme" || match[1] === "fail") && !match[3] && isConditionalSkip(source, raw, match.index + match[0].length, match.index)) continue;
         const focus = match[1] === "only" || match[2] === "fit" || match[2] === "fdescribe";
         const expectedToFail = match[1] === "failing" || match[1] === "fails" || match[1] === "fail";
         infractions.push({
@@ -3208,6 +3251,12 @@ function javascript(text) {
     const end = closing(code, open2);
     if (end > 0) callbacks.push([open2, end]);
   }
+  const tests = [];
+  for (const match of code.matchAll(/(?<![\w$.])(?:it|test|describe)(?:\s*\.\s*\w+)*\s*\(/g)) {
+    const open2 = match.index + match[0].length - 1;
+    const end = closing(code, open2);
+    if (end > 0) tests.push([open2, end]);
+  }
   const programLevel = (index) => {
     let depth = 0;
     for (let i = 0; i < index; i++) {
@@ -3226,8 +3275,9 @@ function javascript(text) {
     const end = closing(code, open2);
     const args = end > 0 ? raw.slice(open2, end) : "";
     if (/requireActual|importActual|importOriginal/.test(args)) continue;
-    const loader = /^\(\s*(['"`])[^'"`\n]+\1\s*,\s*(?:async\s+)?(?:function\s*[\w$]*\s*)?\(\s*([A-Za-z_$][\w$]*)\s*[,)]/.exec(args)?.[2];
-    if (loader && new RegExp(`(?<![\\w$.])${loader.replace(/\$/g, "\\$")}\\s*(?:<[^\\n]*?>)?\\s*\\(`).test(args.slice(args.indexOf(loader) + loader.length))) continue;
+    const factoryHead = /^\(\s*(['"`])[^'"`\n]+\1\s*,\s*(?:async\s+)?(?:function\s*[\w$]*\s*)?(?:\(\s*([A-Za-z_$][\w$]*)\s*[,)]|([A-Za-z_$][\w$]*)\s*=>)/.exec(args);
+    const loader = factoryHead?.[2] ?? factoryHead?.[3];
+    if (loader && new RegExp(`(?<![\\w$])(?:(?<=\\.\\.\\.)|(?<!\\.))${loader.replace(/\$/g, "\\$")}\\s*(?:<[^\\n]*?>)?\\s*\\(`).test(args.slice(args.indexOf(loader) + loader.length))) continue;
     const factory = /^\(\s*(['"`])[^'"`\n]+\1\s*,/.test(args) && !/,\s*\{\s*spy\s*:\s*true\b/.test(args);
     if (!mocked.has(specifier)) mocked.set(specifier, { index: match.index, fake: factory });
   }
@@ -3256,7 +3306,8 @@ function javascript(text) {
     if (match[1].startsWith("{")) bind(match[1], specifier);
     else bindings.set(match[1], { specifier, name: "*", namespace: true });
   }
-  const found = /* @__PURE__ */ new Map();
+  const candidates = [];
+  const mockedNames = [...bindings].filter(([, binding]) => mocked.has(binding.specifier)).map(([local]) => local.replace(/\$/g, "\\$"));
   for (const match of code.matchAll(/(?<![\w$.])expect\s*\(/g)) {
     const open2 = match.index + match[0].length - 1;
     const end = closing(code, open2);
@@ -3277,18 +3328,32 @@ function javascript(text) {
     const name = binding.namespace ? call[2] : call[2] ? void 0 : binding.name;
     if (!name) continue;
     const identifier = call[1].replace(/\$/g, "\\$");
-    const configured = new RegExp(`(?<![\\w$.])${identifier}\\b[^;]{0,80}?\\.\\s*mock(?:Return|Resolved|Rejected|Implementation)\\w*\\s*\\(`).test(raw);
+    const configures = new RegExp(`(?<![\\w$.])${identifier}\\b[^;]{0,80}?\\.\\s*mock(?:Return|Resolved|Rejected|Implementation)\\w*\\s*\\(`, "g");
+    const configured = [...raw.matchAll(configures)].some((config) => tests.every((range) => !within(range, config.index) || within(range, match.index)));
     if (!mock.fake && !configured) continue;
     const key = `${binding.specifier}::${name}`;
-    if (found.has(key)) continue;
     const line = lineOf2(mock.index);
-    found.set(key, {
+    candidates.push({
       key,
-      line,
-      snippet: lines[line - 1].trim().slice(0, 120),
-      reason: `assertion on the output of ${name}(), which this test replaces with a mock of "${binding.specifier}"`
+      at: match.index,
+      range: innermost(callbacks, match.index),
+      subject: {
+        key,
+        line,
+        snippet: lines[line - 1].trim().slice(0, 120),
+        reason: `assertion on the output of ${name}(), which this test replaces with a mock of "${binding.specifier}"`
+      }
     });
   }
+  const candidateAt = new Set(candidates.map((candidate) => candidate.at));
+  const mentionsMock = mockedNames.length ? new RegExp(`(?<![\\w$.])(?:${mockedNames.join("|")})(?![\\w$])`) : null;
+  const checksSomethingElse = (range) => !!range && [...code.matchAll(/(?<![\w$.])expect\s*\(/g)].some((other) => {
+    if (!within(range, other.index) || candidateAt.has(other.index)) return false;
+    const open2 = other.index + other[0].length - 1, end = closing(code, open2);
+    return end > 0 && !mentionsMock?.test(code.slice(open2 + 1, end));
+  });
+  const found = /* @__PURE__ */ new Map();
+  for (const candidate of candidates) if (!found.has(candidate.key) && !checksSomethingElse(candidate.range)) found.set(candidate.key, candidate.subject);
   return [...found.values()];
 }
 var DOTTED = /^[A-Za-z_][\w.]*$/;
@@ -3398,6 +3463,7 @@ function python(text) {
       const receiver = new RegExp(`(?<![\\w.])${patch.object.replace(/[.$]/g, "\\$&")}\\s*\\.\\s*(?!${attribute}\\b)\\w+\\s*\\(`);
       if (receiver.test(code.slice(patch.end + 1, match.index))) continue;
     }
+    if (/(?<![\w$])(?:[A-Za-z_]\w*\s*\.\s*)*_*(?:install|wrap|instrument|inject|hook|swap|rewire|reload)\w*\s*\(/.test(code.slice(patch.end + 1, match.index))) continue;
     const line = lineOf2(patch.index);
     found.set(key, {
       key,
@@ -3428,13 +3494,15 @@ function openBrackets(text) {
 function configEntries(text, toml) {
   const lines = text.split(/\r?\n/);
   const entries = [];
-  let section = "";
+  const elements = /* @__PURE__ */ new Map();
+  let section = "", block = 0;
   for (let i = 0; i < lines.length; i++) {
     const trimmed = lines[i].trim();
     if (!trimmed || trimmed.startsWith("#") || trimmed.startsWith(";")) continue;
-    const header = /^\[\[?\s*([^\]]+?)\s*\]\]?\s*(?:[#;].*)?$/.exec(trimmed);
+    const header = /^(\[\[?)\s*([^\]]+?)\s*\]\]?\s*(?:[#;].*)?$/.exec(trimmed);
     if (header) {
-      section = header[1].replace(/["']/g, "");
+      section = header[2].replace(/["']/g, "");
+      block = header[1] === "[[" ? (elements.set(section, (elements.get(section) ?? 0) + 1), elements.get(section)) : 0;
       continue;
     }
     const pair = /^("[^"]*"|'[^']*'|[\w.-]+)\s*=\s*(.*)$/.exec(trimmed);
@@ -3446,7 +3514,7 @@ function configEntries(text, toml) {
     } else {
       while (i + 1 < lines.length && /^\s+\S/.test(lines[i + 1]) && !/^\s*[#;]/.test(lines[i + 1])) value += "\n" + lines[++i].trim();
     }
-    entries.push({ section, key: pair[1].replace(/["']/g, ""), value, line: start + 1 });
+    entries.push({ section, key: pair[1].replace(/["']/g, ""), value, line: start + 1, block });
   }
   return entries;
 }
@@ -3520,10 +3588,16 @@ function analyzePythonConfig(file, oldText, newText, report) {
       report(line, `${tool} select: ${dropped.slice(0, 4).map((code) => "-" + code).join(", ")}${dropped.length > 4 ? ", \u2026" : ""}`, "Lint rules removed from the selected set");
     }
   }
-  const switches = new Map(before.filter((entry) => toolOf(file, entry.section) === "mypy" && MYPY_SWITCHES.test(entry.key)).map((entry) => [`${entry.section}|${entry.key}`, truthy(entry.value)]));
+  const tableOf = (entries) => {
+    const modules = /* @__PURE__ */ new Map();
+    for (const entry of entries) if (entry.block && entry.key === "module") modules.set(`${entry.section}#${entry.block}`, members(entry.value).join(","));
+    return (entry) => entry.block ? `${entry.section}#${modules.get(`${entry.section}#${entry.block}`) ?? entry.block}` : entry.section;
+  };
+  const tableBefore = tableOf(before), tableAfter = tableOf(after);
+  const switches = new Map(before.filter((entry) => toolOf(file, entry.section) === "mypy" && MYPY_SWITCHES.test(entry.key)).map((entry) => [`${tableBefore(entry)}|${entry.key}`, truthy(entry.value)]));
   for (const entry of after) {
     if (toolOf(file, entry.section) !== "mypy" || !MYPY_SWITCHES.test(entry.key)) continue;
-    if (switches.get(`${entry.section}|${entry.key}`) === true && !truthy(entry.value)) report(entry.line, `mypy ${entry.key}: true \u2192 false`, "Type checker strictness switched off");
+    if (switches.get(`${tableAfter(entry)}|${entry.key}`) === true && !truthy(entry.value)) report(entry.line, `mypy ${entry.key}: true \u2192 false`, "Type checker strictness switched off");
   }
   const ignoring = (entries) => entries.filter((entry) => toolOf(file, entry.section) === "mypy" && entry.key === "ignore_errors" && truthy(entry.value));
   const before_ = ignoring(before), after_ = ignoring(after);
@@ -3796,6 +3870,12 @@ var EMPTY_PY = /==\s*(?:0(?:\.0+)?(?![\w.])|\[\s*\]|\{\s*\}|""|''|\(\s*\)|(?:set
 var PY_SPECIFIC = /==|!=|<=|>=|\bin\b|\bis\b|\bisinstance\s*\(|\.startswith\s*\(|\.endswith\s*\(|\bcallable\s*\(/;
 var JS_SPECIFIC = /\.(?:toBe|toEqual|toStrictEqual|toHaveLength|toContain|toContainEqual|toMatch|toMatchObject|toHaveProperty|toHaveBeenCalledWith|toHaveBeenLastCalledWith|toHaveBeenNthCalledWith|toBeCalledWith|toHaveBeenCalledTimes|toHaveBeenCalledOnce|toHaveTextContent|toHaveAttribute|toHaveValue|toThrow\w*|toBeGreaterThan\w*|toBeLessThan\w*)\s*\(/;
 var THROWING_QUERY = /\b(?:get|find)(?:All)?By\w*\s*\(/;
+var PRESENCE_PREDICATE = /\.(?:exists|isVisible|isExisting|isDisplayed)\(\)$/;
+function samePresence(oldAssertion, newAssertion) {
+  if (!/\.toBe\s*\(\s*true\s*\)\s*;?\s*$/.test(oldAssertion) || !/\.(?:not\s*\.\s*toBeNull|toBeTruthy|toBeDefined|toBeInTheDocument)\s*\(\s*\)\s*;?\s*$/.test(newAssertion)) return false;
+  const was = checkedExpression(oldAssertion, "js"), now = checkedExpression(newAssertion, "js");
+  return !!was && !!now && PRESENCE_PREDICATE.test(was) && (PRESENCE_PREDICATE.test(now) || /(?:querySelector|queryBy\w*|getBy\w*|findBy\w*|\.find|\.get|\$)\(/.test(now));
+}
 function checkedExpression(assertion, language) {
   const squash = (text) => text.replace(/\s+/g, "").replace(/^len\((.*)\)$/, "$1").replace(/\.length$/, "");
   if (language === "py") {
@@ -3887,6 +3967,34 @@ function relaxes(oldAssertion, newAssertion, language, siblings = []) {
   }
   return false;
 }
+function splitArguments(text) {
+  const parts = [];
+  let depth = 0, quote = "", start = 0;
+  for (let i = 0; i < text.length; i++) {
+    const char = text[i];
+    if (quote) {
+      if (char === "\\") i++;
+      else if (char === quote) quote = "";
+    } else if (char === '"' || char === "'") quote = char;
+    else if ("([{".includes(char)) depth++;
+    else if (")]}".includes(char)) depth--;
+    else if (char === "," && depth === 0) {
+      parts.push(text.slice(start, i).trim());
+      start = i + 1;
+    }
+  }
+  parts.push(text.slice(start).trim());
+  return parts.filter(Boolean);
+}
+function unittestAsAssert(statement) {
+  const call = /^\s*self\s*\.\s*assert(True|IsNotNone|GreaterEqual|Greater)\s*\(([\s\S]*)\)\s*;?\s*$/.exec(statement);
+  if (!call) return null;
+  const [first, second] = splitArguments(call[2]);
+  if (!first) return null;
+  if (call[1] === "True") return `assert ${first}`;
+  if (call[1] === "IsNotNone") return `assert ${first} is not None`;
+  return second === void 0 ? null : `assert ${first} ${call[1] === "Greater" ? ">" : ">="} ${second}`;
+}
 function pinsLess(oldAssertion, newAssertion, language) {
   if (language === "js") {
     const was = /\.toThrow(?:Error)?\s*\(\s*([^)\s][^)]*)\)/.exec(oldAssertion);
@@ -3909,6 +4017,7 @@ function pinsLess(oldAssertion, newAssertion, language) {
 }
 function isWeaker(rawOld, rawNew, language) {
   const oldAssertion = language === "py" ? withoutOuterParens(rawOld) : rawOld, newAssertion = language === "py" ? withoutOuterParens(rawNew) : rawNew;
+  if (language === "js" && samePresence(oldAssertion, newAssertion)) return false;
   if (pinsLess(oldAssertion, newAssertion, language) || relaxes(oldAssertion, newAssertion, language)) return true;
   if (language === "js") {
     if (THROWING_QUERY.test(newAssertion) || !JS_SPECIFIC.test(oldAssertion) || WEAK_JS.test(oldAssertion) || BOUND_JS.test(oldAssertion)) return false;
@@ -3916,7 +4025,8 @@ function isWeaker(rawOld, rawNew, language) {
     return WEAK_JS.test(newAssertion) || BOUND_JS.test(newAssertion) && !ZERO_JS.test(oldAssertion) && (limit === void 0 || /^-?\d+(?:\.\d+)?$/.test(limit.trim()));
   }
   if (language === "py") {
-    if (WEAK_PY.test(oldAssertion) || BOUND_PY.test(oldAssertion) || !PY_SPECIFIC.test(oldAssertion)) return false;
+    const oldBare = unittestAsAssert(oldAssertion) ?? oldAssertion;
+    if (WEAK_PY.test(oldAssertion) || BOUND_PY.test(oldAssertion) || WEAK_PY.test(oldBare) || BOUND_PY.test(oldBare) || !PY_SPECIFIC.test(oldAssertion)) return false;
     const limit = /(?:>=|<=|>|<)\s*([^\s,)]+)/.exec(oldAssertion)?.[1];
     return WEAK_PY.test(newAssertion) || BOUND_PY.test(newAssertion) && !EMPTY_PY.test(oldAssertion) && /==|[<>]=?/.test(oldAssertion) && (limit === void 0 || /^-?\d+(?:\.\d+)?$/.test(limit));
   }
@@ -4105,10 +4215,10 @@ function markedTest(finding2, lines) {
   if (/\.py$/i.test(finding2.file ?? "")) {
     if (/^\s*pytestmark\b/.test(snippet)) return null;
     if (/imperative/.test(finding2.reason)) {
-      const indent2 = (lines[index] ?? "").search(/\S/);
+      const indent = (lines[index] ?? "").search(/\S/);
       for (let i = index - 1; i >= 0 && i > index - 200; i--) {
         const def = /^(\s*)(?:async\s+)?def\s+(\w+)\s*\(/.exec(lines[i]);
-        if (def && def[1].length < indent2) return { key: `py:${def[2]}`, title: def[2] };
+        if (def && def[1].length < indent) return { key: `py:${def[2]}`, title: def[2] };
       }
       return null;
     }
@@ -4133,18 +4243,25 @@ function markedTest(finding2, lines) {
     return null;
   }
   if (/\.each\b/.test(snippet)) return null;
-  const title = /\b(?:it|test|describe|suite|xit|xtest|xdescribe|fit|fdescribe)\b[\w$.\s]*\(\s*(['"`])((?:\\.|(?!\1)[^\\\n])+)\1\s*[,)]/.exec(snippet);
+  const title = /\b(?:it|test|describe|suite|xit|xtest|xdescribe|fit|fdescribe)\b[\w$.\s]*\(\s*(['"`])((?:\\.|(?!\1)[^\\\n])+)\1\s*(?:[,)]|$)/.exec(titleOnNextLine(snippet, lines, index));
   if (title) return { key: `js:${title[1]}${title[2]}${title[1]}`, title: `${title[1]}${title[2]}${title[1]}` };
-  const indent = (lines[index] ?? "").search(/\S/);
-  if (indent > 0 && /\b(?:test|it)\s*\.\s*(?:skip|fixme|fail)\s*\(/.test(snippet)) {
-    for (let i = index - 1; i >= 0 && i > index - 200; i--) {
-      if (!lines[i].trim() || lines[i].search(/\S/) >= indent) continue;
-      const owner = /^\s*(?:test|it)(?:\s*\.\s*\w+)*\s*\(\s*(['"`])((?:\\.|(?!\1)[^\\\n])+)\1/.exec(lines[i]);
-      return owner ? { key: `js:${owner[1]}${owner[2]}${owner[1]}`, title: `${owner[1]}${owner[2]}${owner[1]}` } : null;
+  let limit = (lines[index] ?? "").search(/\S/);
+  if (limit > 0 && /\b(?:test|it)\s*\.\s*(?:skip|fixme|fail)\s*\(/.test(snippet)) {
+    for (let i = index - 1; i >= 0 && i > index - 200 && limit > 0; i--) {
+      if (!lines[i].trim() || lines[i].search(/\S/) >= limit) continue;
+      limit = lines[i].search(/\S/);
+      const owner = /^\s*(?:test|it)(?:\s*\.\s*(?!describe\b|step\b)\w+)*\s*\(\s*(['"`])((?:\\.|(?!\1)[^\\\n])+)\1/.exec(titleOnNextLine(lines[i], lines, i));
+      if (owner) return { key: `js:${owner[1]}${owner[2]}${owner[1]}`, title: `${owner[1]}${owner[2]}${owner[1]}` };
+      if (/^\s*(?:await\s+)?(?:test|it)\s*\.\s*describe\b|^\s*(?:describe|suite)\b/.test(lines[i])) return null;
     }
   }
   return null;
 }
+function titleOnNextLine(text, lines, index) {
+  if (!/\(\s*$/.test(text)) return text;
+  const next = (lines[index + 1] ?? "").trim();
+  return /^(['"`])/.test(next) ? `${text} ${next}` : text;
+}
 function newTestMarkers(before, after, oldText, newText) {
   const markers = after.filter(isTestMarker);
   if (!markers.length || oldText === null) return [];
@@ -4200,23 +4317,60 @@ var THRESHOLD_PATTERNS = [
 var SWITCHES = ["strict", "noImplicitAny", "strictNullChecks", "noImplicitReturns", "useUnknownInCatchVariables"];
 var GATE_SCRIPT = /^(?:pre|post)?(?:test|lint|typecheck|type-check|check|ci|verify|validate)(?::[\w:.-]+)?$/;
 var CI_RUNNER = new RegExp(
-  String.raw`(?<![\w./-])(?:pytest|vitest|jest|mocha|tox|nox)(?![\w/-])` + String.raw`|(?<![\w./-])(?:npm|yarn|pnpm|bun|go|cargo|mvn|gradle|make|dotnet|flutter)\s+(?:run\s+)?(?:-\S+\s+)*test(?![\w/])` + // Monorepo runners: `nx affected --target=test`, `turbo run test`, `lerna run test`, `bazel test //...`.
-  String.raw`|(?<![\w./-])(?:nx|turbo|lerna|rush)\s+(?:run-many|affected|run|test)\b[^\n]*(?<![\w-])(?:test|tests)(?![\w-])` + String.raw`|(?<![\w./-])(?:bazel\s+test|ctest|phpunit|rspec|deno\s+test)(?![\w/-])` + String.raw`|\brun:.*(?<=\s)(?:tests?|unit|e2e)(?![\w/.-])`
+  String.raw`(?<![\w./-])(?:pytest|vitest|jest|mocha|tox|nox)(?![\w/-])` + String.raw`|(?<![\w./-])(?:npm|yarn|pnpm|bun|go|cargo|mvn|gradle|make|dotnet|flutter)\s+(?:run\s+)?(?:-\S+\s+)*test(?![\w/])` + // A package script that runs the tests with coverage: `yarn coverage`, `npm run coverage`.
+  String.raw`|(?<![\w./-])(?:npm|yarn|pnpm|bun)\s+(?:run\s+)?coverage(?![\w/-])` + // Monorepo runners: `nx affected --target=test`, `turbo run test`, `lerna run test`, `bazel test //...`.
+  String.raw`|(?<![\w./-])(?:nx|turbo|lerna|rush)\s+(?:run-many|affected|run|test)\b[^\n]*(?<![\w-])(?:test|tests)(?![\w-])` + String.raw`|(?<![\w./-])(?:bazel\s+test|ctest|phpunit|rspec|deno\s+test)(?![\w/-])` + // Python's own runners and the browser runners: `python3 -m unittest`, `nosetests`, `playwright test`, `cypress run`, `karma start`.
+  String.raw`|(?<![\w./-])python[\d.]*\s+-m\s+(?:unittest|pytest|nose2?)(?![\w/-])` + String.raw`|(?<![\w./-])(?:nosetests|nose2|py\.test|stestr|behave|jasmine|playwright\s+test|cypress\s+run|karma\s+start|hatch\s+(?:run\s+)?test|swift\s+test)(?![\w/-])` + String.raw`|\brun:.*(?<=\s)(?:tests?|unit|e2e)(?![\w/.-])`
 );
+var SHELL_TEST = /(?:^|run:\s*|[;&|({]\s*|\b(?:if|then|do|else|elif|while|until)\s+|!\s+)test\s+!?\s*-[bcdefghkLnOGprsStuwxzN](?=\s)/g;
+var ciRuns = (line) => CI_RUNNER.test(line.replace(SHELL_TEST, " ").replace(/\s#.*$/, ""));
 var CI_FILE = /(?:\.github\/workflows\/|\.gitlab-ci)/;
 var CI_REUSABLE = /^\s*-?\s*uses:\s*\S*\.github\/workflows\/\S+/;
 var CI_GATE = /^\s*(?:pull_request|pull_request_target|push|merge_group)\s*:|^\s*on\s*:\s*(?:\[[^\]]*\b(?:push|pull_request|pull_request_target|merge_group)\b[^\]]*\]|(?:push|pull_request|pull_request_target|merge_group)\s*$)|^\s*-\s*(?:push|pull_request|pull_request_target|merge_group)\s*$/m;
+var CI_NON_GATE = /^\s*(?:schedule|release|workflow_dispatch|workflow_run|repository_dispatch)\s*:/m;
+function gatesChanges(file, text) {
+  const section = triggerSection(text), triggers = gatingTriggers(text) ?? text;
+  if (/\.gitlab-ci/.test(file) || CI_GATE.test(triggers) || /\bworkflow_call\b/.test(triggers)) return true;
+  return section !== null && triggers !== section ? false : !CI_NON_GATE.test(triggers);
+}
+function gatingTriggers(text) {
+  const section = triggerSection(text);
+  if (section === null) return null;
+  const lines = section.split(/\r?\n/), kept = [];
+  for (let i = 0; i < lines.length; i++) {
+    const head = /^(\s*)push\s*:\s*(?:#.*)?$/.exec(lines[i]);
+    if (!head) {
+      kept.push(lines[i]);
+      continue;
+    }
+    let end = i + 1;
+    while (end < lines.length && (!lines[end].trim() || lines[end].search(/\S/) > head[1].length)) end++;
+    const block = lines.slice(i + 1, end).join("\n");
+    if (!(/^\s*tags\s*:/m.test(block) && !/^\s*branches(?:-ignore)?\s*:/m.test(block))) kept.push(...lines.slice(i, end));
+    i = end - 1;
+  }
+  return kept.join("\n");
+}
+function triggerSection(text) {
+  const lines = text.split(/\r?\n/);
+  const start = lines.findIndex((line) => /^["']?on["']?\s*:/.test(line));
+  if (start < 0) return null;
+  let end = start + 1;
+  while (end < lines.length && (!lines[end].trim() || /^[\s#-]/.test(lines[end]))) end++;
+  return lines.slice(start, end).join("\n");
+}
 function gatingTestLines(file, text) {
-  return /\.gitlab-ci/.test(file) || CI_GATE.test(text) ? matchingLines(text, CI_RUNNER) : 0;
+  return /\.gitlab-ci/.test(file) || CI_GATE.test(gatingTriggers(text) ?? text) ? matchingLines(text, ciRuns) : 0;
 }
 function matchingLines(text, pattern) {
-  return text.split(/\r?\n/).filter((line) => !line.trim().startsWith("#") && pattern.test(line)).length;
+  const matches = typeof pattern === "function" ? pattern : (line) => pattern.test(line);
+  return text.split(/\r?\n/).filter((line) => !line.trim().startsWith("#") && matches(line)).length;
 }
 function ciTestSteps(file, oldText, newText) {
   if (!CI_FILE.test(file) || oldText === null && newText === null) return null;
   const was = oldText ?? "", now = newText ?? "";
   const after = gatingTestLines(file, now);
-  return { file, before: gatingTestLines(file, was), after, reusableAfter: matchingLines(now, CI_REUSABLE), ungatedAfter: after === 0 ? matchingLines(now, CI_RUNNER) : 0 };
+  return { file, before: gatingTestLines(file, was), after, reusableAfter: matchingLines(now, CI_REUSABLE), ungatedAfter: after === 0 ? matchingLines(now, ciRuns) : 0 };
 }
 function resolveCiTestSteps(counts, commit) {
   const before = counts.reduce((sum, item) => sum + item.before, 0);
@@ -4226,6 +4380,23 @@ function resolveCiTestSteps(counts, commit) {
 }
 var LENIENT = /continue-on-error:\s*true|allow_failure:\s*true|^\s*(?:-\s+)?if:\s*(?:\$\{\{\s*)?false\s*(?:\}\})?\s*(?:#.*)?$/;
 var SHELL_SWALLOW = /\|\|\s*(?:true|exit\s+0|:|echo\b)|;\s*(?:true|exit\s+0)\b/;
+var NON_CHECK_ACTION = new RegExp(String.raw`^\s*(?:-\s+)?uses:\s*["']?(?:` + [
+  String.raw`actions/(?:upload-artifact|download-artifact|upload-pages-artifact|deploy-pages|cache|attest[\w-]*|checkout|setup-[\w-]+)`,
+  String.raw`docker/(?:login-action|metadata-action|build-push-action|setup-[\w-]+)`,
+  String.raw`azure/login`,
+  String.raw`aws-actions/(?:configure-aws-credentials|amazon-ecr-login)`,
+  String.raw`google-github-actions/(?:auth|setup-gcloud)`,
+  String.raw`softprops/action-gh-release`,
+  String.raw`ncipollo/release-action`,
+  String.raw`pypa/gh-action-pypi-publish`,
+  String.raw`JS-DevTools/npm-publish`,
+  String.raw`peaceiris/actions-gh-pages`,
+  String.raw`peter-evans/[\w-]+`,
+  String.raw`marocchino/sticky-pull-request-comment`,
+  String.raw`slackapi/slack-github-action`,
+  String.raw`codecov/codecov-action`,
+  String.raw`coverallsapp/github-action`
+].join("|") + String.raw`)(?:/[\w./-]+)?@`, "i");
 function lenientSteps(text) {
   const lines = text.split(/\r?\n/);
   const indentOf = (line) => line.length - line.trimStart().length;
@@ -4242,9 +4413,18 @@ function lenientSteps(text) {
       while (j >= 0 && (!lines[j].trim() || lines[j].trim().startsWith("#") || indentOf(lines[j]) >= indentOf(line))) j--;
       identity = j >= 0 ? lines[j].trim() : "";
     }
+    const start = /^\s*-\s+if:/.test(line) ? i : Math.max(j, 0), level = indentOf(lines[start]);
+    let end = start + 1;
+    while (end < lines.length && (!lines[end].trim() || lines[end].trim().startsWith("#") || indentOf(lines[end]) > level)) end++;
+    const scope = lines.slice(start, end), runsChecks = matchingLines(scope.join("\n"), ciRuns) > 0;
+    const uses = scope.filter((item) => /^\s*(?:-\s+)?uses:/.test(item));
+    const nonCheck = !runsChecks && uses.length === 1 && NON_CHECK_ACTION.test(uses[0]) && !scope.some((item) => /^\s*(?:-\s+)?run:/.test(item));
     const entry = found.get(identity);
-    if (entry) entry.count++;
-    else found.set(identity, { count: 1, line: i + 1 });
+    if (entry) {
+      entry.count++;
+      entry.runsChecks ||= runsChecks;
+      entry.nonCheck &&= nonCheck;
+    } else found.set(identity, { count: 1, line: i + 1, runsChecks, nonCheck });
   });
   return found;
 }
@@ -4306,16 +4486,18 @@ function analyzeGuardFile(file, oldText, newText, commit) {
     });
   }
   if (CI_FILE.test(file)) {
+    const gating = gatesChanges(file, oldText) || gatesChanges(file, newText);
     const before = lenientSteps(oldText), existing = new Set(oldText.split(/\r?\n/).flatMap((line) => [line.trim(), line.trim().replace(/^-\s+/, "")]));
     const newLines = newText.split(/\r?\n/);
     for (const [identity, now2] of lenientSteps(newText)) {
+      if (now2.nonCheck || !gating && !now2.runsChecks) continue;
       if (identity && existing.has(identity) && now2.count > (before.get(identity)?.count ?? 0)) {
         const marker = (newLines[now2.line - 1] ?? "").trim();
         out.push(finding("guard-weakened", file, now2.line, marker, /^(?:-\s+)?if:/.test(marker) ? "CI step is now switched off" : "CI step is now allowed to fail", commit));
         break;
       }
     }
-    const swallowing = (text) => text.split(/\r?\n/).map((line, index) => ({ line: line.trim(), index })).filter((item) => !item.line.startsWith("#") && CI_RUNNER.test(item.line) && SHELL_SWALLOW.test(item.line));
+    const swallowing = (text) => text.split(/\r?\n/).map((line, index) => ({ line: line.trim(), index })).filter((item) => !item.line.startsWith("#") && ciRuns(item.line) && SHELL_SWALLOW.test(item.line));
     const was = swallowing(oldText), now = swallowing(newText);
     const tolerance = (text) => text.split(/\r?\n/).filter((line) => !line.trim().startsWith("#") && /continue-on-error:\s*true|allow_failure:\s*true/.test(line)).length;
     const traded = Math.max(0, tolerance(oldText) - tolerance(newText));

package/dist/cli.js

@@ -38,7 +38,7 @@ import {
   terminalText,
   uninstallHooks,
   writeBaseline
-} from "./chunk-2KWC2IAP.js";
+} from "./chunk-OAMVMB5N.js";
 
 // src/cli.ts
 import { Command } from "commander";

package/dist/index.js

@@ -91,7 +91,7 @@ import {
   unquoteGitPath,
   validateExceptions,
   writeBaseline
-} from "./chunk-2KWC2IAP.js";
+} from "./chunk-OAMVMB5N.js";
 export {
   AGENT_LABELS,
   CONFIG_FILENAMES,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agent-roast",
-  "version": "0.18.0",
+  "version": "0.19.0",
   "description": "Stops AI coding agents (and rushed humans) from turning a build green by weakening tests, coverage gates or CI. Hook for Claude Code, Codex and Cursor, pre-commit, GitHub Action. Offline.",
   "mcpName": "io.github.Nari-Ab/agent-roast",
   "type": "module",