agent-roast 0.15.0 → 0.17.0

package/README.md

@@ -78,10 +78,10 @@ Policy is read from the **base** branch, so a pull request cannot switch off the
 
 | Rule | Fires when | By default |
 | --- | --- | :-: |
-| `assertion-weakened` | an exact check is replaced by a weaker one: `toBe(42)` → `toBeDefined()`, `assert x == 1` → `assert x` | blocks |
+| `assertion-weakened` | an exact check is replaced by a weaker one: `toBe(42)` → `toBeDefined()`, `toBe(2)` → `toBeLessThanOrEqual(2)`, `toEqual(x)` → `toMatchObject(x)`, `assert x == 1` → `assert x` | blocks |
 | `guard-weakened` | a coverage threshold is lowered, `strict` is switched off, a lint rule is downgraded, a CI step may fail, a workflow stops running tests, a `conftest.py` hook sets a report to `passed` | blocks |
 | `test-skip` | a test is switched off or expected to fail: `it.skip`, `@pytest.mark.skip`, `@pytest.mark.xfail`, `it.fails`, `test.fixme`, `t.Skip`, `#[ignore]`, `@Disabled`, `[Ignore]`, `xit`, ... | blocks |
-| `test-focus`, `trivial-assert` | `.only` / `fit`; `expect(true).toBe(true)` and its relatives in ten languages | blocks |
+| `test-focus`, `trivial-assert` | `.only` / `fit`; `expect(true).toBe(true)` and its relatives in ten languages; a test that asserts on a function it mocked itself | blocks |
 | `assertion-removed`, `test-removed`, `expectation-rewritten` | a test loses assertions, disappears, or its expected value is edited next to production code | warns |
 | `swallowed-error`, `type-escape`, `stub-abandonment` | `catch {}` / `except: pass`, `as any` / `# type: ignore` / `# pragma: no cover` / `istanbul ignore`, `throw new Error("Not implemented")` | warns |
 
@@ -111,15 +111,20 @@ old debt with a baseline: [docs/reference.md](docs/reference.md#configuration).
 Measured, with the denominators and the caveats in [docs/benchmarks.md](docs/benchmarks.md). **Weakened checks are rare, mostly
 deliberate, and not specific to AI. agent-roast is a tripwire that stays silent until a check gets weaker.**
 
-- **114 of 114** hand-labeled cheating edits are caught and **80 of 80** harmless look-alikes (reformats, moves, raised
+- **126 of 126** hand-labeled cheating edits are caught and **90 of 90** harmless look-alikes (reformats, moves, raised
   thresholds, environment-gated skips) stay silent. The cases were written with the detector, so they guard against regressions;
-  they do not prove accuracy. Two gaps are documented and still open.
+  they do not prove accuracy. One gap is documented and still open.
 - **12,645 commits** that touch tests or CI/config in 166 public repositories: blocking rules fired on 0.3% of AI-signed commits
   (Claude, Copilot, Cursor, Codex, Devin, aider) and on 0.3% of the others. Inside the same repositories the odds for an
   AI-signed commit were 0.81 times those of another commit (95% CI 0.40–1.67). There is no sign that AI-signed commits weaken
   checks more often.
 - **About one finding in five was a false alarm**, and most of the rest were deliberate, explained changes a reviewer should
   see rather than mistakes. The labels come from one reader (an AI assistant), so treat them as a first estimate.
+- **9,732 commits compared in full** from 164 public repositories that use coding agents (5,454 of them AI-signed) were reviewed again for the
+  last two releases, and every new finding was read. 0.15.0's rules reported 30 new findings there and 7 were false alarms (fixed in 0.16.0);
+  0.17.0's rule for an exact check turned into a bound reported 18 in 14 commits, all of them loosened checks, after two false alarms were
+  fixed. One reader, and a convenience sample ([section 7](docs/benchmarks.md#7-ai-signed-commits-and-a-test-that-checks-its-own-mock),
+  [section 8](docs/benchmarks.md#8-an-exact-check-turned-into-a-bound)).
 - `replay` ran on 179 public repositories that use coding agents: no commit would have been blocked in 149 of them.
 - The rules for **eight more languages** (Go, Rust, Java, Kotlin, C#, Ruby, PHP, Swift) ran on the newest 300 commits of 39
   repositories and every finding was read: 26 of 99 were skips that depend on the environment, now fixed; of the 73 left, 56 are

package/dist/{chunk-BWGYT2YZ.js → chunk-IKXGHCWB.js}

@@ -764,9 +764,9 @@ var RULE_GUIDES = {
   },
   "trivial-assert": {
     title: "Assertion that always passes",
-    why: "Comparing a constant to itself does not check the implementation and cannot catch regressions.",
-    suggestion: "Call the real behavior and assert its output or observable side effect instead of comparing constants.",
-    example: "expect(true).toBe(true) \u2192 expect(canSignIn(validUser)).toBe(true)"
+    why: "Comparing a constant to itself, or checking the output of a function the test itself replaces with a mock, does not check the implementation and cannot catch regressions.",
+    suggestion: "Call the real behavior and assert its output or observable side effect instead of comparing constants. Mock the dependencies of the code under test, never the code under test.",
+    example: "expect(true).toBe(true) \u2192 expect(canSignIn(validUser)).toBe(true); jest.mock('./price', ...) then expect(price(cart)).toBe(42) \u2192 mock the price list and assert total(cart)"
   },
   "type-escape": {
     title: "Type, lint or coverage check bypass",
@@ -788,9 +788,9 @@ var RULE_GUIDES = {
   },
   "assertion-weakened": {
     title: "Exact check replaced by a weaker one",
-    why: "Swapping an exact assertion for toBeDefined, a truthiness check or `is not None` keeps the test green while it verifies far less.",
+    why: "Swapping an exact assertion for toBeDefined, a truthiness check, `is not None`, a bound (`>= 2` in place of `== 2`) or toMatchObject in place of toEqual keeps the test green while it verifies far less.",
     suggestion: "Restore the exact check, or assert the specific value the code should produce.",
-    example: "expect(total).toBe(42) \u2192 expect(total).toBeDefined()  (restore the exact check)"
+    example: "expect(total).toBe(42) \u2192 expect(total).toBeDefined()  (restore the exact check); expect(cost).toBe(2) \u2192 expect(cost).toBeLessThanOrEqual(2)  (pin the value again)"
   },
   "assertion-removed": {
     title: "Test checks less than before",
@@ -1453,13 +1453,13 @@ function auditFileDiffs(files, commitHash = "HEAD", commitDate = (/* @__PURE__ *
     hasSupportedChanges = true;
     filesInspected++;
     linesAdded += file.addedLines.length;
-    const javascript = /\.(?:[cm]?[jt]sx?|vue|svelte)$/i.test(file.path);
+    const javascript2 = /\.(?:[cm]?[jt]sx?|vue|svelte)$/i.test(file.path);
     const typescript = /\.(?:[cm]?tsx?|vue|svelte)$/i.test(file.path);
-    const python = /\.py$/i.test(file.path);
+    const python2 = /\.py$/i.test(file.path);
     const rust = /\.rs$/i.test(file.path);
-    const braceCatches = javascript || /\.(?:java|kt|php|cs|cpp)$/i.test(file.path);
-    const directives = javascript ? JS_COMMENT_DIRECTIVE_PATTERNS : python ? PYTHON_COMMENT_DIRECTIVE_PATTERNS : [];
-    const stubs = javascript ? JS_STUB_PATTERNS : python ? PYTHON_STUB_PATTERNS : rust ? RUST_STUB_PATTERNS : [];
+    const braceCatches = javascript2 || /\.(?:java|kt|php|cs|cpp)$/i.test(file.path);
+    const directives = javascript2 ? JS_COMMENT_DIRECTIVE_PATTERNS : python2 ? PYTHON_COMMENT_DIRECTIVE_PATTERNS : [];
+    const stubs = javascript2 ? JS_STUB_PATTERNS : python2 ? PYTHON_STUB_PATTERNS : rust ? RUST_STUB_PATTERNS : [];
     const firstFinding = infractions.length;
     const languageRules = LANGUAGE_LINE_RULES.find((entry) => entry.files.test(file.path))?.rules;
     const masker = new SourceMasker(file.path);
@@ -1472,7 +1472,7 @@ function auditFileDiffs(files, commitHash = "HEAD", commitDate = (/* @__PURE__ *
       for (const start of masked.templateStarts) testCode[start] = "
";
       maskedLines.push({ code: cleanedLine, testCode: testCode.join(""), raw: rawLine, number: added.lineNumber });
       const isComment = !cleanedLine.trim();
-      if (python && !isComment) {
+      if (python2 && !isComment) {
         for (const rule of PYTHON_TEST_SKIP_PATTERNS) {
           if (rule.regex.test(cleanedLine)) {
             if (rule.imperative && isGuardedSkip(maskedLines)) break;
@@ -1513,12 +1513,12 @@ function auditFileDiffs(files, commitHash = "HEAD", commitDate = (/* @__PURE__ *
           break;
         }
       }
-      const blockDirective = javascript && !foundEscape ? JS_BLOCK_DIRECTIVE_PATTERNS.find((rule) => masked.blockStarts.some((start) => rule.regex.test(rawLine.slice(start)))) : void 0;
+      const blockDirective = javascript2 && !foundEscape ? JS_BLOCK_DIRECTIVE_PATTERNS.find((rule) => masked.blockStarts.some((start) => rule.regex.test(rawLine.slice(start)))) : void 0;
       if (blockDirective) {
         infractions.push({ type: "type-escape", file: file.path, lineNumber: added.lineNumber, lineSnippet: rawLine.trim(), commitHash, commitDate, reason: blockDirective.name });
         foundEscape = true;
       }
-      if (python && !isComment && isTestFile(file.path)) {
+      if (python2 && !isComment && isTestFile(file.path)) {
         for (const rule of PYTHON_TRIVIAL_ASSERT_PATTERNS) {
           if (rule.regex.test(cleanedLine)) {
             infractions.push({
@@ -1578,7 +1578,7 @@ function auditFileDiffs(files, commitHash = "HEAD", commitDate = (/* @__PURE__ *
         infractions.push({ type: "stub-abandonment", file: file.path, ...location, commitHash, commitDate, reason: rule.name });
       }
     }
-    if (python) {
+    if (python2) {
       const reportedSkips = new Set(infractions.slice(firstFinding).filter((f) => f.type === "test-skip").map((f) => f.lineNumber));
       for (const match of source.matchAll(PYTHON_MARKER)) {
         const xfail = match[2] === "xfail";
@@ -1617,7 +1617,7 @@ function auditFileDiffs(files, commitHash = "HEAD", commitDate = (/* @__PURE__ *
         }
       }
     }
-    if (javascript) {
+    if (javascript2) {
       for (const match of source.matchAll(JS_TEST_API)) {
         let previous = match.index - 1;
         while (previous >= 0 && /\s/.test(source[previous])) previous--;
@@ -1659,9 +1659,9 @@ function auditFileDiffs(files, commitHash = "HEAD", commitDate = (/* @__PURE__ *
     }
     for (let index = 0; index < maskedLines.length; index++) {
       const start = maskedLines[index];
-      const inlineExcept = python && /^\s*except(?:\*?\s+[^:]+)?:\s*pass\s*;?\s*$/.test(start.code);
+      const inlineExcept = python2 && /^\s*except(?:\*?\s+[^:]+)?:\s*pass\s*;?\s*$/.test(start.code);
       if (inlineExcept) infractions.push({ type: "swallowed-error", file: file.path, lineNumber: start.number, lineSnippet: start.raw.trim(), commitHash, commitDate, reason: "silent python error swallow (except: pass)" });
-      const isExcept = python && /^\s*except(?:\*?\s+[^:]+)?:\s*$/.test(start.code);
+      const isExcept = python2 && /^\s*except(?:\*?\s+[^:]+)?:\s*$/.test(start.code);
       if (!isExcept) continue;
       let next = index + 1;
       while (next < maskedLines.length && maskedLines[next].number === maskedLines[next - 1].number + 1 && !maskedLines[next].code.trim()) next++;
@@ -3165,6 +3165,230 @@ import path6 from "path";
 // src/tampering.ts
 import { createHash as createHash3 } from "crypto";
 
+// src/mocks.ts
+function views(text, language) {
+  const masker = new SourceMasker(language === "js" ? "file.ts" : "file.py");
+  const lines = text.split(/\r?\n/);
+  const code = lines.map((line, index) => masker.mask(line, index + 1).code).join("\n");
+  const starts = [];
+  let offset = 0;
+  for (const line of lines) {
+    starts.push(offset);
+    offset += line.length + 1;
+  }
+  const lineOf2 = (index) => {
+    let low = 0, high = starts.length - 1;
+    while (low < high) {
+      const middle = low + high + 1 >>> 1;
+      if (starts[middle] <= index) low = middle;
+      else high = middle - 1;
+    }
+    return low + 1;
+  };
+  return { raw: lines.join("\n"), code, lines, starts, lineOf: lineOf2 };
+}
+function closing(code, open2) {
+  let depth = 0;
+  for (let i = open2; i < code.length && i - open2 < 2e4; i++) {
+    if (code[i] === "(") depth++;
+    else if (code[i] === ")" && --depth === 0) return i;
+  }
+  return -1;
+}
+var STRING_AFTER = /^\s*(['"`])([^'"`\n]+)\1/;
+var within = (range, index) => !range || index >= range[0] && index <= range[1];
+var innermost = (ranges, index) => ranges.filter((range) => index >= range[0] && index <= range[1]).sort((a, b) => a[1] - a[0] - (b[1] - b[0]))[0] ?? null;
+function javascript(text) {
+  const { raw, code, lines, lineOf: lineOf2 } = views(text, "js");
+  const callbacks = [];
+  for (const match of code.matchAll(/(?<![\w$.])(?:it|test|beforeEach|beforeAll|afterEach|afterAll)(?:\s*\.\s*\w+)*\s*\(/g)) {
+    const open2 = match.index + match[0].length - 1;
+    const end = closing(code, open2);
+    if (end > 0) callbacks.push([open2, end]);
+  }
+  const programLevel = (index) => {
+    let depth = 0;
+    for (let i = 0; i < index; i++) {
+      const char = code[i];
+      if (char === "(" || char === "{" || char === "[") depth++;
+      else if (char === ")" || char === "}" || char === "]") depth--;
+    }
+    return depth === 0;
+  };
+  const mocked = /* @__PURE__ */ new Map();
+  for (const match of code.matchAll(/(?<![\w$.])(?:jest|vi)\s*\.\s*mock\s*\(/g)) {
+    if (!programLevel(match.index)) continue;
+    const open2 = match.index + match[0].length - 1;
+    const specifier = STRING_AFTER.exec(raw.slice(open2 + 1))?.[2];
+    if (!specifier) continue;
+    const end = closing(code, open2);
+    const args = end > 0 ? raw.slice(open2, end) : "";
+    if (/requireActual|importActual|importOriginal/.test(args)) continue;
+    const factory = /^\(\s*(['"`])[^'"`\n]+\1\s*,/.test(args) && !/,\s*\{\s*spy\s*:\s*true\b/.test(args);
+    if (!mocked.has(specifier)) mocked.set(specifier, { index: match.index, fake: factory });
+  }
+  if (!mocked.size) return [];
+  const bindings = /* @__PURE__ */ new Map();
+  const bind = (clause, specifier) => {
+    const named = /\{([^}]*)\}/.exec(clause)?.[1];
+    for (const part of (named ?? "").split(",")) {
+      const item = part.trim().replace(/^type\s+/, "");
+      if (!item) continue;
+      const [imported, local] = item.includes(" as ") ? item.split(/\s+as\s+/) : item.includes(":") ? item.split(/\s*:\s*/) : [item, item];
+      if (/^[\w$]+$/.test(imported) && /^[\w$]+$/.test(local)) bindings.set(local, { specifier, name: imported, namespace: false });
+    }
+    const namespace = /\*\s*as\s+([\w$]+)/.exec(clause)?.[1];
+    if (namespace) bindings.set(namespace, { specifier, name: "*", namespace: true });
+    const standalone = /^\s*([\w$]+)\s*(?:,|$)/.exec(clause.replace(/\{[^}]*\}/, ""))?.[1];
+    if (standalone && standalone !== "type" && !/^\s*\{/.test(clause)) bindings.set(standalone, { specifier, name: "default", namespace: false });
+  };
+  for (const match of code.matchAll(/\bimport\s+([^;'"`]*?)\bfrom\b/g)) {
+    const specifier = STRING_AFTER.exec(raw.slice(match.index + match[0].length))?.[2];
+    if (specifier) bind(match[1], specifier);
+  }
+  for (const match of code.matchAll(/\b(?:const|let|var)\s+(\{[^}]*\}|[\w$]+)\s*=\s*(?:await\s+)?require\s*\(/g)) {
+    const specifier = STRING_AFTER.exec(raw.slice(match.index + match[0].length))?.[2];
+    if (!specifier) continue;
+    if (match[1].startsWith("{")) bind(match[1], specifier);
+    else bindings.set(match[1], { specifier, name: "*", namespace: true });
+  }
+  const found = /* @__PURE__ */ new Map();
+  for (const match of code.matchAll(/(?<![\w$.])expect\s*\(/g)) {
+    const open2 = match.index + match[0].length - 1;
+    const end = closing(code, open2);
+    if (end < 0) continue;
+    const checked = code.slice(open2 + 1, end);
+    let call = /^\s*(?:await\s+)?([A-Za-z_$][\w$]*)\s*(?:\.\s*([A-Za-z_$][\w$]*)\s*)?\(/.exec(checked);
+    const variable = /^\s*([A-Za-z_$][\w$]*)\s*$/.exec(checked)?.[1];
+    if (!call && variable) {
+      const from = innermost(callbacks, match.index)?.[0] ?? 0;
+      const name2 = variable.replace(/\$/g, "\\$");
+      const assigned = [...code.slice(from, match.index).matchAll(new RegExp(`(?<![\\w$.])${name2}\\s*=(?!=)\\s*(?:await\\s+)?([A-Za-z_$][\\w$]*)\\s*(?:\\.\\s*([A-Za-z_$][\\w$]*)\\s*)?\\(`, "g"))].at(-1);
+      if (assigned) call = assigned;
+    }
+    if (!call) continue;
+    const binding = bindings.get(call[1]);
+    const mock = binding && mocked.get(binding.specifier);
+    if (!binding || !mock) continue;
+    const name = binding.namespace ? call[2] : call[2] ? void 0 : binding.name;
+    if (!name) continue;
+    const identifier = call[1].replace(/\$/g, "\\$");
+    const configured = new RegExp(`(?<![\\w$.])${identifier}\\b[^;]{0,80}?\\.\\s*mock(?:Return|Resolved|Rejected|Implementation)\\w*\\s*\\(`).test(raw);
+    if (!mock.fake && !configured) continue;
+    const key = `${binding.specifier}::${name}`;
+    if (found.has(key)) continue;
+    const line = lineOf2(mock.index);
+    found.set(key, {
+      key,
+      line,
+      snippet: lines[line - 1].trim().slice(0, 120),
+      reason: `assertion on the output of ${name}(), which this test replaces with a mock of "${binding.specifier}"`
+    });
+  }
+  return [...found.values()];
+}
+var DOTTED = /^[A-Za-z_][\w.]*$/;
+function python(text) {
+  const { raw, code, lines, starts, lineOf: lineOf2 } = views(text, "py");
+  const alias = /* @__PURE__ */ new Map();
+  for (const match of code.matchAll(/^[ \t]*import\s+([\w.]+)\s+as\s+(\w+)/gm)) alias.set(match[2], match[1]);
+  for (const match of code.matchAll(/^[ \t]*from\s+([\w.]+)\s+import\s+([^\n(]+)/gm)) {
+    for (const part of match[2].split(",")) {
+      const [name, local] = part.trim().split(/\s+as\s+/);
+      if (/^\w+$/.test(name)) alias.set(local || name, `${match[1]}.${name}`);
+    }
+  }
+  const expand = (dotted) => {
+    const [head, ...rest] = dotted.split(".");
+    return rest.length && alias.has(head) ? [alias.get(head), ...rest].join(".") : dotted;
+  };
+  const expandObject = (dotted) => {
+    const [head, ...rest] = dotted.split(".");
+    return alias.has(head) ? [alias.get(head), ...rest].join(".") : dotted;
+  };
+  const codeLines2 = code.split("\n");
+  const indent = (line) => line.length - line.trimStart().length;
+  const functions = [];
+  codeLines2.forEach((line, i) => {
+    if (!/^\s*(?:async\s+)?def\s+\w+/.test(line)) return;
+    let end = i;
+    for (let j = i + 1; j < codeLines2.length; j++) {
+      if (!codeLines2[j].trim()) continue;
+      if (indent(codeLines2[j]) <= indent(line)) break;
+      end = j;
+    }
+    functions.push({ range: [i, end], header: i });
+  });
+  const lineIndex = (offset) => lineOf2(offset) - 1;
+  const scopeOf = (line) => {
+    if (/^\s*@/.test(codeLines2[line] ?? "")) {
+      for (let j = line; j < codeLines2.length && j < line + 40; j++) {
+        const owner = functions.find((fn) => fn.header === j);
+        if (owner) return owner.range;
+      }
+    }
+    return innermost(functions.map((fn) => fn.range), line);
+  };
+  const blockOf = (line) => {
+    if (!/^\s*with\b/.test(codeLines2[line] ?? "")) return null;
+    let end = line;
+    for (let j = line + 1; j < codeLines2.length; j++) {
+      if (!codeLines2[j].trim()) continue;
+      if (indent(codeLines2[j]) <= indent(codeLines2[line])) break;
+      end = j;
+    }
+    return [line, end];
+  };
+  const patched = /* @__PURE__ */ new Map();
+  const patchCall = /(?<![\w.])(?:(?:mock|unittest\s*\.\s*mock)\s*\.\s*)?patch(\s*\.\s*object)?\s*\(|(?<![\w.])mocker\s*\.\s*patch(\s*\.\s*object)?\s*\(|(?<![\w.])monkeypatch\s*\.\s*setattr\s*\(/g;
+  for (const match of code.matchAll(patchCall)) {
+    const open2 = match.index + match[0].length - 1;
+    const end = closing(code, open2);
+    if (end < 0) continue;
+    const args = raw.slice(open2 + 1, end);
+    if (/\bwraps\s*=/.test(args)) continue;
+    const target = STRING_AFTER.exec(args);
+    const parts = target ? null : /^\s*([A-Za-z_][\w.]*)\s*,\s*(['"])(\w+)\2/.exec(args);
+    const key = target ? target[2] : parts ? `${expandObject(parts[1])}.${parts[3]}` : void 0;
+    if (!key || !DOTTED.test(key)) continue;
+    const line = lineIndex(match.index);
+    patched.set(key, [...patched.get(key) ?? [], { index: match.index, scope: blockOf(line) ?? scopeOf(line) }]);
+  }
+  if (!patched.size) return [];
+  const found = /* @__PURE__ */ new Map();
+  const assertion = /^[ \t]*assert\s+(?:not\s+)?([A-Za-z_][\w.]*)\s*(\()?|\bself\s*\.\s*assert\w*\s*\(\s*([A-Za-z_][\w.]*)\s*(\()?/gm;
+  for (const match of code.matchAll(assertion)) {
+    let callee = (match[1] ?? match[3]) && (match[2] ?? match[4]) ? match[1] ?? match[3] : void 0;
+    const variable = !callee ? match[1] ?? match[3] : void 0;
+    if (!callee && variable && /^[A-Za-z_]\w*$/.test(variable)) {
+      const owner = scopeOf(lineIndex(match.index));
+      const from = owner ? starts[owner[0]] ?? 0 : 0;
+      const assigned = [...code.slice(from, match.index).matchAll(new RegExp(`^[ \\t]*${variable}\\s*=(?!=)\\s*(?:await\\s+)?([A-Za-z_][\\w.]*)\\s*\\(`, "gm"))].at(-1);
+      callee = assigned?.[1];
+    }
+    if (!callee) continue;
+    const key = expand(callee);
+    if (found.has(key)) continue;
+    const at = lineIndex(match.index);
+    const patch = patched.get(key)?.find((item) => item.scope && within(item.scope, at) && (at >= lineIndex(item.index) || /^\s*@/.test(codeLines2[lineIndex(item.index)] ?? "")));
+    if (!patch) continue;
+    const line = lineOf2(patch.index);
+    found.set(key, {
+      key,
+      line,
+      snippet: lines[line - 1].trim().slice(0, 120),
+      reason: `assertion on the output of ${callee}(), which this test replaces with a mock`
+    });
+  }
+  return [...found.values()];
+}
+function mockedSubjects(text, file) {
+  if (/mock|monkeypatch/i.test(file)) return [];
+  if (/\.[cm]?[jt]sx?$/i.test(file)) return javascript(text);
+  if (/\.py$/i.test(file)) return python(text);
+  return [];
+}
+
 // src/pyconfig.ts
 function openBrackets(text) {
   const bare = text.replace(/"(?:\\.|[^"\\\n])*"|'[^'\n]*'/g, '""').replace(/#.*$/gm, "");
@@ -3205,6 +3429,13 @@ function members(value) {
   if (quoted.length) return quoted;
   return value.replace(/[#;].*$/gm, "").split(/[\s,]+/).map((word) => word.trim()).filter(Boolean);
 }
+function covers(selector, code) {
+  if (selector === "ALL") return true;
+  const wanted = /^([A-Za-z]+)(\d*)$/.exec(selector), actual = /^([A-Za-z]+)(\d*)$/.exec(code);
+  if (!wanted || !actual) return code.startsWith(selector);
+  const linter = (letters) => letters.toUpperCase().replace(/^TCH$/, "TC");
+  return linter(wanted[1]) === linter(actual[1]) && actual[2].startsWith(wanted[2]);
+}
 var truthy = (value) => /^(?:true|1|yes|on)$/i.test(value.replace(/\s*[#;].*$/, "").trim());
 function toolOf(file, section) {
   const name = section.toLowerCase();
@@ -3240,17 +3471,24 @@ function analyzePythonConfig(file, oldText, newText, report) {
     return found;
   };
   const was = ignored(before);
+  const enforced = (tool, code) => {
+    if (tool !== "ruff" && tool !== "flake8") return true;
+    const chosen = (key) => before.filter((entry) => toolOf(file, entry.section) === tool && key.test(entry.key)).flatMap((entry) => members(entry.value));
+    const explicit = chosen(/^select$/), extra = chosen(/^extend[-_]select$/);
+    const base = explicit.length ? explicit : tool === "ruff" ? ["E4", "E7", "E9", "F"] : ["E", "F", "W", "C90"];
+    return [...base, ...extra].some((selected2) => covers(selected2, code) || covers(code, selected2));
+  };
   for (const [tool, bucket] of ignored(after)) {
     if (!configured.has(tool)) continue;
     const old = [...was.get(tool)?.keys() ?? []];
-    const fresh = [...bucket].filter(([code]) => !old.some((known) => code.startsWith(known)));
+    const fresh = [...bucket].filter(([code]) => !old.some((known) => covers(known, code)) && enforced(tool, code));
     if (fresh.length) report(Math.min(...fresh.map(([, line]) => line)), `${tool} ignore: ${fresh.slice(0, 4).map(([code]) => "+" + code).join(", ")}${fresh.length > 4 ? ", \u2026" : ""}`, "Lint or type checks switched off in the configuration");
   }
   const selected = (entries, tool) => entries.filter((entry) => toolOf(file, entry.section) === tool && entry.key === "select").flatMap((entry) => members(entry.value));
   for (const tool of ["ruff", "flake8"]) {
     const old = selected(before, tool), now = selected(after, tool);
     if (!old.length || !now.length || now.includes("ALL")) continue;
-    const dropped = old.filter((code) => !now.some((kept) => code.startsWith(kept)));
+    const dropped = old.filter((code) => !now.some((kept) => covers(kept, code)));
     if (dropped.length) {
       const line = after.find((entry) => toolOf(file, entry.section) === tool && entry.key === "select")?.line ?? 1;
       report(line, `${tool} select: ${dropped.slice(0, 4).map((code) => "-" + code).join(", ")}${dropped.length > 4 ? ", \u2026" : ""}`, "Lint rules removed from the selected set");
@@ -3430,15 +3668,71 @@ function parseTests(text, language) {
 function skeleton(assertion) {
   return assertion.replace(/(['"`])(?:\\.|(?!\1)[^\\])*\1/g, "S").replace(/\b\d+(?:\.\d+)?\b/g, "N").replace(/\b(?:true|false|null|undefined|None|True|False|nil)\b/g, "V").replace(/\s+/g, "");
 }
-var WEAK_JS = /\.(?:toBeDefined|toBeTruthy|toBeFalsy|toBeUndefined)\s*\(|\.not\s*\.\s*(?:toBeNull|toBeUndefined|toThrow)\s*\(|\.toBeGreaterThanOrEqual\s*\(\s*0\s*\)|\.(?:toHaveBeenCalled|toBeCalled)\s*\(\s*\)|\.(?:toBe|toEqual|toStrictEqual)\s*\(\s*expect\s*\.\s*(?:anything\s*\(\s*\)|any\s*\(\s*\w+\s*\))\s*\)/;
+var WEAK_JS = /\.(?:toBeDefined|toBeTruthy|toBeFalsy|toBeUndefined)\s*\(|\.not\s*\.\s*(?:toBeNull|toBeUndefined|toThrow)\s*\(|\.toBeGreaterThanOrEqual\s*\(\s*0\s*\)|(?<!\.not)\.(?:toHaveBeenCalled|toBeCalled)\s*\(\s*\)|\.(?:toBe|toEqual|toStrictEqual)\s*\(\s*expect\s*\.\s*(?:anything\s*\(\s*\)|any\s*\(\s*\w+\s*\))\s*\)/;
 var BOUND_JS = /\.toBeGreaterThan\s*\(\s*(?:0|-1)\s*\)|\.toBeGreaterThanOrEqual\s*\(\s*[01]\s*\)/;
 var ZERO_JS = /\.(?:toBe|toEqual|toStrictEqual|toHaveLength|toHaveBeenCalledTimes)\s*\(\s*(?:0|\[\s*\]|\{\s*\}|null|undefined|false|"")\s*\)/;
 var WEAK_PY = /^\s*assert\s+[\w.\[\]]+\s*(?:,.*)?$|^\s*assert\s+.+\bis\s+not\s+None\s*(?:,.*)?$|^\s*assert\s+True\b/;
 var BOUND_PY = /^\s*assert\s+.+(?:>\s*0|>=\s*[01])\s*(?:,.*)?$/;
-var EMPTY_PY = /==\s*(?:0|\[\s*\]|\{\s*\}|""|''|None|False)\b|\bis\s+None\b|\bnot\b|!=\s*(?:0|None)\b/;
+var EMPTY_PY = /==\s*(?:0(?:\.0+)?(?![\w.])|\[\s*\]|\{\s*\}|""|''|\(\s*\)|(?:set|dict|list|tuple|frozenset)\(\s*\)|None\b|False\b)|\bis\s+None\b|\bnot\b|!=\s*(?:0|None)\b/;
 var PY_SPECIFIC = /==|!=|<=|>=|\bin\b|\bis\b|\bisinstance\s*\(|\.startswith\s*\(|\.endswith\s*\(|\bcallable\s*\(/;
 var JS_SPECIFIC = /\.(?:toBe|toEqual|toStrictEqual|toHaveLength|toContain|toContainEqual|toMatch|toMatchObject|toHaveProperty|toHaveBeenCalledWith|toHaveBeenLastCalledWith|toHaveBeenNthCalledWith|toBeCalledWith|toHaveBeenCalledTimes|toHaveBeenCalledOnce|toHaveTextContent|toHaveAttribute|toHaveValue|toThrow\w*|toBeGreaterThan\w*|toBeLessThan\w*)\s*\(/;
 var THROWING_QUERY = /\b(?:get|find)(?:All)?By\w*\s*\(/;
+function checkedExpression(assertion, language) {
+  const squash = (text) => text.replace(/\s+/g, "").replace(/^len\((.*)\)$/, "$1").replace(/\.length$/, "");
+  if (language === "py") {
+    const operand = /^\s*assert\s+(?:not\s+)?(.+?)\s*(?:==|!=|>=|<=|>|<|\bin\b|\bis\b)/.exec(assertion)?.[1] ?? /\bself\s*\.\s*assert\w*\s*\(\s*([^,]+?)\s*,/.exec(assertion)?.[1];
+    return operand ? squash(operand) : null;
+  }
+  const start = assertion.search(/(?<![\w$.])expect\s*\(/);
+  if (start >= 0) {
+    const open2 = assertion.indexOf("(", start);
+    let depth = 0;
+    for (let i = open2; i < assertion.length; i++) {
+      if (assertion[i] === "(") depth++;
+      else if (assertion[i] === ")" && --depth === 0) return squash(assertion.slice(open2 + 1, i));
+    }
+    return null;
+  }
+  const node = /assert\s*\.\s*(?:equal|strictEqual|deepEqual|deepStrictEqual|ok)\s*\(\s*([^,)]+?)\s*(?:,|\)|>=|<=|>|<)/.exec(assertion)?.[1];
+  return node ? squash(node) : null;
+}
+function matcherArgument(assertion, name) {
+  const open2 = new RegExp(`\\.\\s*${name}\\s*\\(`).exec(assertion);
+  if (!open2) return null;
+  let depth = 1;
+  for (let i = open2.index + open2[0].length; i < assertion.length; i++) {
+    if ("([{".includes(assertion[i])) depth++;
+    else if (")]}".includes(assertion[i]) && --depth === 0) return assertion.slice(open2.index + open2[0].length, i).replace(/\s+/g, "");
+  }
+  return null;
+}
+function isExact(assertion, language) {
+  if (language === "js") {
+    return !/\.\s*not\s*\./.test(assertion) && (/\.(?:toBe|toEqual|toStrictEqual|toHaveLength)\s*\(/.test(assertion) || /assert\s*\.\s*(?:equal|strictEqual|deepEqual|deepStrictEqual)\s*\(/.test(assertion));
+  }
+  if (language === "py") return /^\s*assert\s+.+?\s*==\s*\S/.test(assertion) || /\bself\s*\.\s*assertEqual\s*\(/.test(assertion);
+  return false;
+}
+function relaxes(oldAssertion, newAssertion, language, siblings = []) {
+  const subject = checkedExpression(oldAssertion, language);
+  if (!subject || subject !== checkedExpression(newAssertion, language)) return false;
+  if (siblings.some((other) => isExact(other, language) && checkedExpression(other, language) === subject)) return false;
+  if (language === "js") {
+    if (/\.\s*not\s*\./.test(oldAssertion) || /\.\s*not\s*\./.test(newAssertion)) return false;
+    const exact = /\.(?:toBe|toEqual|toStrictEqual|toHaveLength)\s*\(\s*(-?\d+(?:\.\d+)?)\s*\)/.exec(oldAssertion)?.[1] ?? /assert\s*\.\s*(?:equal|strictEqual|deepEqual|deepStrictEqual)\s*\([^,]+,\s*(-?\d+(?:\.\d+)?)\s*[,)]/.exec(oldAssertion)?.[1];
+    const bound = /\.(?:toBeGreaterThan|toBeGreaterThanOrEqual|toBeLessThan|toBeLessThanOrEqual)\s*\(/.test(newAssertion) || /assert\s*\.\s*ok\s*\([^;]*(?:>=|<=|>|<)/.test(newAssertion);
+    if (exact !== void 0 && Number(exact) !== 0 && bound) return true;
+    const same = matcherArgument(oldAssertion, "toEqual") ?? matcherArgument(oldAssertion, "toStrictEqual");
+    return same !== null && same !== "" && same === matcherArgument(newAssertion, "toMatchObject");
+  }
+  if (language === "py") {
+    if (EMPTY_PY.test(oldAssertion)) return false;
+    const exact = /^\s*assert\s+.+?\s*==\s*\S/.test(oldAssertion) || /\bself\s*\.\s*assertEqual\s*\(/.test(oldAssertion);
+    const bound = /^\s*assert\s+.+?\s*(?:>=|<=|>|<)\s*\S/.test(newAssertion) && !/==/.test(newAssertion.replace(/[<>]=/g, "")) || /\bself\s*\.\s*assert(?:Greater|Less)(?:Equal)?\s*\(/.test(newAssertion);
+    return exact && bound;
+  }
+  return false;
+}
 function pinsLess(oldAssertion, newAssertion, language) {
   if (language === "js") {
     const was = /\.toThrow(?:Error)?\s*\(\s*([^)\s][^)]*)\)/.exec(oldAssertion);
@@ -3447,26 +3741,27 @@ function pinsLess(oldAssertion, newAssertion, language) {
   }
   if (language === "py") {
     const raised = (text) => {
+      const captured = /\)\s*as\s+\w+/.test(text);
       const marker = /\bpytest\s*\.\s*raises\s*\(\s*([\w.]+|\([^)]*\))([^)]*)\)/.exec(text);
-      if (marker) return { type: marker[1], message: /\bmatch\s*=/.test(marker[2]) };
+      if (marker) return { type: marker[1], message: /\bmatch\s*=/.test(marker[2]), captured };
       const unit = /\bassertRaises(Regex)?\s*\(\s*([\w.]+|\([^)]*\))/.exec(text);
-      return unit ? { type: unit[2], message: !!unit[1] } : null;
+      return unit ? { type: unit[2], message: !!unit[1], captured } : null;
     };
     const was = raised(oldAssertion), now = raised(newAssertion);
     const broad = (type) => /^(?:builtins\.)?(?:Base)?Exception$/.test(type);
-    return !!was && !!now && (was.message && !now.message || !broad(was.type) && broad(now.type));
+    return !!was && !!now && (was.message && !now.message && !now.captured || !broad(was.type) && broad(now.type));
   }
   return false;
 }
 function isWeaker(oldAssertion, newAssertion, language) {
-  if (pinsLess(oldAssertion, newAssertion, language)) return true;
+  if (pinsLess(oldAssertion, newAssertion, language) || relaxes(oldAssertion, newAssertion, language)) return true;
   if (language === "js") {
     if (THROWING_QUERY.test(newAssertion) || !JS_SPECIFIC.test(oldAssertion) || WEAK_JS.test(oldAssertion) || BOUND_JS.test(oldAssertion)) return false;
     return WEAK_JS.test(newAssertion) || BOUND_JS.test(newAssertion) && !ZERO_JS.test(oldAssertion);
   }
   if (language === "py") {
     if (WEAK_PY.test(oldAssertion) || BOUND_PY.test(oldAssertion) || !PY_SPECIFIC.test(oldAssertion)) return false;
-    return WEAK_PY.test(newAssertion) || BOUND_PY.test(newAssertion) && !EMPTY_PY.test(oldAssertion);
+    return WEAK_PY.test(newAssertion) || BOUND_PY.test(newAssertion) && !EMPTY_PY.test(oldAssertion) && /==|\bin\b|[<>]=?/.test(oldAssertion);
   }
   return false;
 }
@@ -3485,6 +3780,34 @@ var HELPER_CALL = /(?<![\w$])((?:expect|assert|verify|check|require)[A-Z]\w*|_{0
 function finding(type, file, line, snippet, reason, commit) {
   return { type, file, lineNumber: Math.max(1, line), lineSnippet: snippet, commitHash: commit, commitDate: "", reason, suggestion: RULE_GUIDES[type].suggestion };
 }
+function alignAssertions(before, after) {
+  const table = Array.from({ length: before.length + 1 }, () => new Array(after.length + 1).fill(0));
+  for (let i = before.length - 1; i >= 0; i--) {
+    for (let j = after.length - 1; j >= 0; j--) table[i][j] = before[i] === after[j] ? table[i + 1][j + 1] + 1 : Math.max(table[i + 1][j], table[i][j + 1]);
+  }
+  const pairs = /* @__PURE__ */ new Map();
+  for (let i = 0, j = 0; i < before.length && j < after.length; ) {
+    if (before[i] === after[j]) {
+      pairs.set(i, j);
+      i++;
+      j++;
+    } else if (table[i + 1][j] >= table[i][j + 1]) i++;
+    else j++;
+  }
+  return pairs;
+}
+function unmatchedIndexes(from, against) {
+  const budget = /* @__PURE__ */ new Map();
+  for (const assertion of against) budget.set(assertion, (budget.get(assertion) ?? 0) + 1);
+  return from.map((assertion, i) => ({ assertion, i })).filter(({ assertion }) => {
+    const available = budget.get(assertion) ?? 0;
+    if (available > 0) {
+      budget.set(assertion, available - 1);
+      return false;
+    }
+    return true;
+  }).map(({ i }) => i);
+}
 function analyzeTestFile(oldPath, newPath, oldText, newText, commit) {
   const language = languageOf(newText === null ? oldPath : newPath);
   if (!language || !(isTestFile(newPath) || isTestFile(oldPath))) return EMPTY;
@@ -3502,6 +3825,10 @@ function analyzeTestFile(oldPath, newPath, oldText, newText, commit) {
   const addedText = after.filter((test) => !beforeKeys.has(test.key)).map((test) => test.text).join("\n");
   const result = { immediate: [], removedTests: [], addedHashes: [], rewritten: [] };
   for (const test of after) if (!beforeKeys.has(test.key)) result.addedHashes.push(test.bodyHash);
+  const mockedBefore = new Set((oldText === null ? [] : mockedSubjects(oldText, oldPath)).map((item) => item.key));
+  for (const item of newText === null ? [] : mockedSubjects(newText, newPath)) {
+    if (!mockedBefore.has(item.key)) result.immediate.push(finding("trivial-assert", newPath, item.line, item.snippet, item.reason, commit));
+  }
   for (const old of before) {
     const current = afterByKey.get(old.key);
     if (!current) {
@@ -3522,6 +3849,34 @@ function analyzeTestFile(oldPath, newPath, oldText, newText, commit) {
     if (current.parametrized && !old.parametrized) continue;
     const snapshot = (test) => test.raw.some((statement) => /\.toMatch(?:Inline)?Snapshot\b|\.matchSnapshot\b|\bsnapshot\s*\(/.test(statement));
     if (snapshot(current) && !snapshot(old)) continue;
+    if (old.assertions.length && current.assertions.length && current.assertions.length !== old.assertions.length && old.assertions.length * current.assertions.length <= 25e4) {
+      const aligned = alignAssertions(old.assertions, current.assertions);
+      const fresh = unmatchedIndexes(current.assertions, old.assertions);
+      const gone = unmatchedIndexes(old.assertions, current.assertions).filter((index) => (newTotals.get(old.assertions[index]) ?? 0) < (oldTotals.get(old.assertions[index]) ?? 0));
+      for (const index of gone) {
+        const was = checkedExpression(old.raw[index], language);
+        if (was === null) continue;
+        let low = -1, high = current.assertions.length;
+        for (const [from, to] of aligned) {
+          if (from < index) low = Math.max(low, to);
+          else if (from > index) high = Math.min(high, to);
+        }
+        const successors = fresh.filter((candidate) => candidate > low && candidate < high);
+        const siblings = successors.map((candidate) => current.raw[candidate]);
+        const partner = successors.find((candidate) => checkedExpression(current.raw[candidate], language) === was && relaxes(old.raw[index], current.raw[candidate], language, siblings));
+        if (partner !== void 0) {
+          result.immediate.push(finding(
+            "assertion-weakened",
+            newPath,
+            current.line,
+            `${old.name}: ${old.raw[index].slice(0, 80)} \u2192 ${current.raw[partner].slice(0, 80)}`,
+            "Assertion replaced by a weaker check",
+            commit
+          ));
+          break;
+        }
+      }
+    }
     if (current.assertions.length < old.assertions.length) {
       if ([...current.text.matchAll(HELPER_CALL)].some((call) => !(oldText ?? "").includes(call[1]))) continue;
       const budget = tally([current]);
@@ -3550,7 +3905,8 @@ ${addedText}`;
       continue;
     }
     if (current.assertions.length === old.assertions.length && old.assertions.length > 0) {
-      const changed = old.assertions.map((assertion, i) => ({ before: assertion, after: current.assertions[i], rawBefore: old.raw[i], rawAfter: current.raw[i] })).filter((pair) => pair.before !== pair.after);
+      const goneIndexes = unmatchedIndexes(old.assertions, current.assertions), freshIndexes = unmatchedIndexes(current.assertions, old.assertions);
+      const changed = goneIndexes.map((index, k) => ({ before: old.assertions[index], after: current.assertions[freshIndexes[k]], rawBefore: old.raw[index], rawAfter: current.raw[freshIndexes[k]] })).filter((pair) => pair.after !== void 0);
       const weaker = changed.find((pair) => isWeaker(pair.rawBefore, pair.rawAfter, language));
       if (weaker) {
         result.immediate.push(finding(
@@ -3779,7 +4135,9 @@ function analyzeGuardFile(file, oldText, newText, commit) {
     }
     const swallowing = (text) => text.split(/\r?\n/).map((line, index) => ({ line: line.trim(), index })).filter((item) => !item.line.startsWith("#") && CI_RUNNER.test(item.line) && SHELL_SWALLOW.test(item.line));
     const was = swallowing(oldText), now = swallowing(newText);
-    if (now.length > was.length) {
+    const tolerance = (text) => text.split(/\r?\n/).filter((line) => !line.trim().startsWith("#") && /continue-on-error:\s*true|allow_failure:\s*true/.test(line)).length;
+    const traded = Math.max(0, tolerance(oldText) - tolerance(newText));
+    if (now.length > was.length + traded) {
       const known = new Set(was.map((item) => item.line));
       const fresh = now.find((item) => !known.has(item.line)) ?? now[0];
       out.push(finding("guard-weakened", file, fresh.index + 1, fresh.line.slice(0, 80), "Test command can no longer fail the CI step", commit));

package/dist/cli.js

@@ -38,7 +38,7 @@ import {
   terminalText,
   uninstallHooks,
   writeBaseline
-} from "./chunk-BWGYT2YZ.js";
+} from "./chunk-IKXGHCWB.js";
 
 // src/cli.ts
 import { Command } from "commander";

package/dist/index.js

@@ -91,7 +91,7 @@ import {
   unquoteGitPath,
   validateExceptions,
   writeBaseline
-} from "./chunk-BWGYT2YZ.js";
+} from "./chunk-IKXGHCWB.js";
 export {
   AGENT_LABELS,
   CONFIG_FILENAMES,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agent-roast",
-  "version": "0.15.0",
+  "version": "0.17.0",
   "description": "Stops AI coding agents (and rushed humans) from turning a build green by weakening tests, coverage gates or CI. Hook for Claude Code, Codex and Cursor, pre-commit, GitHub Action. Offline.",
   "mcpName": "io.github.Nari-Ab/agent-roast",
   "type": "module",