agent-roast 0.18.0 → 0.20.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +16 -12
- package/dist/{chunk-2KWC2IAP.js → chunk-BNDNXKBM.js} +336 -47
- package/dist/cli.js +1 -1
- package/dist/index.js +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -78,7 +78,7 @@ Policy is read from the **base** branch, so a pull request cannot switch off the
|
|
|
78
78
|
|
|
79
79
|
| Rule | Fires when | By default |
|
|
80
80
|
| --- | --- | :-: |
|
|
81
|
-
| `assertion-weakened` | an exact check is replaced by a weaker one: `toBe(42)` → `toBeDefined()`, `toBe(2)` → `toBeLessThanOrEqual(2)`, `toEqual(x)` → `toMatchObject(x)`, `assert x == 1` → `assert x` | blocks |
|
|
81
|
+
| `assertion-weakened` | an exact check is replaced by a weaker one: `toBe(42)` → `toBeDefined()`, `toBe(2)` → `toBeLessThanOrEqual(2)`, `toBe(2)` → `not.toBe(0)`, `toEqual(x)` → `toMatchObject(x)`, `assert x == 1` → `assert x` | blocks |
|
|
82
82
|
| `guard-weakened` | a coverage threshold is lowered, `strict` is switched off, a lint rule is downgraded, a CI step may fail, a workflow stops running tests, a `conftest.py` hook sets a report to `passed` | blocks |
|
|
83
83
|
| `test-skip` | a test that was already there is switched off or expected to fail: `it.skip`, `@pytest.mark.skip`, `@pytest.mark.xfail`, `it.fails`, `test.fixme`, `t.Skip`, `#[ignore]`, `@Disabled`, `[Ignore]`, `xit`, ... (a new test that is skipped is not reported) | blocks |
|
|
84
84
|
| `test-focus`, `trivial-assert` | `.only` / `fit`; `expect(true).toBe(true)` and its relatives in ten languages; a test that asserts on a function it mocked itself | blocks |
|
|
@@ -109,23 +109,27 @@ old debt with a baseline: [docs/reference.md](docs/reference.md#configuration).
|
|
|
109
109
|
## Does it work?
|
|
110
110
|
|
|
111
111
|
Measured, with the denominators and the caveats in [docs/benchmarks.md](docs/benchmarks.md). **Weakened checks are rare, mostly
|
|
112
|
-
deliberate, and not specific to AI. agent-roast is a tripwire that asks about them. On repositories it had not seen,
|
|
113
|
-
blocks were questions with a harmless answer.**
|
|
112
|
+
deliberate, and not specific to AI. agent-roast is a tripwire that asks about them. On repositories it had not seen, between a quarter and
|
|
113
|
+
two thirds of its blocks were questions with a harmless answer, depending on the sample.**
|
|
114
114
|
|
|
115
|
-
- **
|
|
115
|
+
- **152 of 152** hand-labeled cheating edits are caught and **136 of 136** harmless look-alikes (reformats, moves, raised
|
|
116
116
|
thresholds, environment-gated skips) stay silent. The cases were written with the detector, so they guard against regressions;
|
|
117
117
|
they do not prove accuracy. One gap is documented and still open.
|
|
118
118
|
- **12,645 commits** that touch tests or CI/config in 166 public repositories: blocking rules fired on 0.3% of AI-signed commits
|
|
119
119
|
(Claude, Copilot, Cursor, Codex, Devin, aider) and on 0.3% of the others. Inside the same repositories the odds for an
|
|
120
120
|
AI-signed commit were 0.81 times those of another commit (95% CI 0.40–1.67). There is no sign that AI-signed commits weaken
|
|
121
121
|
checks more often.
|
|
122
|
-
- **On repositories it was not written from,
|
|
123
|
-
repositories (
|
|
124
|
-
first sample and
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
122
|
+
- **On repositories it was not written from, between a quarter and two thirds of the blocked commits were false alarms.** Seven samples of 39
|
|
123
|
+
or 40 public repositories (278 in all, 79,843 commits) were replayed and every blocked commit was read. The share was 69% for 0.17.0 on the
|
|
124
|
+
first sample and on the fourth, 51% for 0.18.0 on the fourth and 27% on the fifth, which it had not seen either. This version, frozen before the
|
|
125
|
+
sixth sample was drawn, blocked 100 of its 11,335 commits: 34 changed a check a reviewer should see (an assertion that got weaker, a gate
|
|
126
|
+
that was relaxed, a test switched off or left focused) and 66 were false alarms, but 51 of the 66 were one defect in one repository, an array of
|
|
127
|
+
mypy overrides read as one table, which is fixed in this release; the released build blocks 44 of those commits, 34 and 10. A new
|
|
128
|
+
test that asserts nothing counts as a block for a reviewer; counted as false alarms, 13 of 33 on the fifth sample and 72 of 100 on the sixth. Most of the rest, in the
|
|
129
|
+
earlier measurements too, were deliberate and explained. One reader (an AI assistant) sorted them, and the fixes were made from the same
|
|
130
|
+
sorting, so treat the figures as a first estimate. The published 0.19.0 blocked 31 of the 12,000 commits of the seventh sample: 16 for a reviewer and 15 false alarms (48%, 6 of them one
|
|
131
|
+
repository's lint configuration); this release fixes the kinds of false alarm that reading it showed, and blocks 22 of those commits ([section 9](docs/benchmarks.md#9-repositories-nobody-had-tuned-on),
|
|
132
|
+
[section 10](docs/benchmarks.md#10-two-more-samples-and-what-the-fixes-from-sample-4-did), [section 11](docs/benchmarks.md#11-a-seventh-sample-and-two-things-that-were-missing)).
|
|
129
133
|
- **9,732 commits compared in full** from 164 public repositories that use coding agents (5,454 of them AI-signed) were reviewed again for the
|
|
130
134
|
last two releases, and every new finding was read. 0.15.0's rules reported 30 new findings there and 7 were false alarms (fixed in 0.16.0);
|
|
131
135
|
0.17.0's rule for an exact check turned into a bound reported 18 in 14 commits, all of them loosened checks, after two false alarms were
|
|
@@ -158,7 +162,7 @@ It is a review aid. It cannot prove who wrote a change or that code is correct,
|
|
|
158
162
|
|
|
159
163
|
[Rules](docs/rules.md) · [Integrations](docs/integrations.md) · [Reference](docs/reference.md) (commands, exit codes,
|
|
160
164
|
configuration, exceptions, baselines, limits) · [Comparison](docs/comparison.md) · [Measurements](docs/benchmarks.md) ·
|
|
161
|
-
[Try it on your repository](docs/pilot.md) · [Contributing](CONTRIBUTING.md) · [Changelog](CHANGELOG.md)
|
|
165
|
+
[Real blocks](docs/examples.md) · [Try it on your repository](docs/pilot.md) · [Contributing](CONTRIBUTING.md) · [Changelog](CHANGELOG.md)
|
|
162
166
|
|
|
163
167
|
Found a finding that was wrong, or a shortcut it missed? That is the most useful report there is:
|
|
164
168
|
[open an issue](https://github.com/Nari-Ab/agent-roast/issues/new/choose).
|
|
@@ -1166,7 +1166,8 @@ var PYTHON_TEST_SKIP_PATTERNS = [
|
|
|
1166
1166
|
{ regex: /\bself\s*\.\s*skipTest\s*\(/, name: "imperative unittest skip (self.skipTest)", imperative: true },
|
|
1167
1167
|
// A test that is expected to fail cannot fail the run, which is what a skip does too.
|
|
1168
1168
|
{ regex: /(?<![\w.])pytest\s*\.\s*xfail\s*\(/, name: "imperative pytest xfail (pytest.xfail)", imperative: true },
|
|
1169
|
-
|
|
1169
|
+
// `@unittest.expectedFailure`, and the same decorator under another module's name (`@test.expectedFailure` of a test framework that re-exports it).
|
|
1170
|
+
{ regex: /@(?:[A-Za-z_]\w*\s*\.\s*)*expectedFailure\b/, name: "test expected to fail (@unittest.expectedFailure)", imperative: false }
|
|
1170
1171
|
];
|
|
1171
1172
|
var PYTHON_MARKER = /(@\s*|(?<![\w.])marks\s*=\s*\[?\s*|^[ \t]*pytestmark\s*=\s*\[?\s*)pytest\s*\.\s*mark\s*\.\s*(skip|xfail)\b/gm;
|
|
1172
1173
|
function isConditionalXfail(args) {
|
|
@@ -1317,13 +1318,55 @@ function isGuardedBlockSkip(lines, profile) {
|
|
|
1317
1318
|
return compiledConditionally(i + 1);
|
|
1318
1319
|
}
|
|
1319
1320
|
var JS_TEST_API = /(?<![\w$.])(?:(?:it|test|describe|suite)(?:\s*\.\s*(?:concurrent|serial|parallel|describe|failing|sequential))*\s*\.\s*(skip|todo|only|fixme|failing|fails|fail)|((?:xit|xtest|xdescribe|fit|fdescribe)))\b((?:\s*\.\s*(?:each|failing|concurrent|serial|parallel|sequential))*)\s*(?=\(|\u0001)/g;
|
|
1320
|
-
|
|
1321
|
+
var MENTIONS_CI = /\bCI\b|\bisCI\b|GITHUB_ACTIONS/;
|
|
1322
|
+
function isConditionalSkip(source, raw, open2, start) {
|
|
1321
1323
|
const args = callArguments(source, raw, open2);
|
|
1322
1324
|
if (!args) return false;
|
|
1323
1325
|
const isString = (arg) => /^(['"`])[\s\S]*\1$/.test(arg);
|
|
1324
|
-
if (!args.length ||
|
|
1326
|
+
if (!args.length || args[0] === "true") return isUnderRuntimeCondition(source, raw, start);
|
|
1327
|
+
if (isString(args[0])) return false;
|
|
1325
1328
|
if (args.length > 1 && !isString(args.at(-1))) return false;
|
|
1326
|
-
return
|
|
1329
|
+
return !MENTIONS_CI.test(args[0]);
|
|
1330
|
+
}
|
|
1331
|
+
var TEST_REGISTRATION = /^(?:await\s+)?(?:test|it|describe|suite|beforeAll|beforeEach|afterAll|afterEach)\b/;
|
|
1332
|
+
function isUnderRuntimeCondition(source, raw, start) {
|
|
1333
|
+
const alwaysTrue = (text) => /^\s*(?:true|1|!\s*0|!\s*false)\s*$/.test(text);
|
|
1334
|
+
const indentOf = (text) => text.length - text.trimStart().length;
|
|
1335
|
+
const lineStartOf = (offset) => Math.max(source.lastIndexOf("\n", offset - 1), source.lastIndexOf("\0", offset - 1)) + 1;
|
|
1336
|
+
const mentionsCi = (from, to) => MENTIONS_CI.test(raw.slice(from, to));
|
|
1337
|
+
const lineStart = lineStartOf(start);
|
|
1338
|
+
const head = source.slice(lineStart, start).trimEnd();
|
|
1339
|
+
if (head.endsWith(")")) {
|
|
1340
|
+
let depth = 0, opening = -1;
|
|
1341
|
+
for (let i = head.length - 1; i >= 0; i--) {
|
|
1342
|
+
if (head[i] === ")") depth++;
|
|
1343
|
+
else if (head[i] === "(" && --depth === 0) {
|
|
1344
|
+
opening = i;
|
|
1345
|
+
break;
|
|
1346
|
+
}
|
|
1347
|
+
}
|
|
1348
|
+
if (opening >= 0 && /(?:^|[\s}])if\s*$/.test(head.slice(0, opening))) {
|
|
1349
|
+
return !mentionsCi(lineStart + opening + 1, lineStart + head.length - 1) && !alwaysTrue(head.slice(opening + 1, -1));
|
|
1350
|
+
}
|
|
1351
|
+
} else if (/(?:^|[\s}])else$/.test(head)) return true;
|
|
1352
|
+
else if (head.endsWith("&&")) return !mentionsCi(lineStart, start) && !alwaysTrue(head.slice(0, -2).replace(/^[\s{}]+/, ""));
|
|
1353
|
+
const lineEnd = source.slice(start).search(/[\n\0]/);
|
|
1354
|
+
let limit = indentOf(source.slice(lineStart, lineEnd < 0 ? void 0 : start + lineEnd)), at = lineStart;
|
|
1355
|
+
for (let steps = 0; steps < 200 && at > 0; steps++) {
|
|
1356
|
+
if (source[at - 1] === "\0") return false;
|
|
1357
|
+
const previous = lineStartOf(at - 1), line = source.slice(previous, at - 1);
|
|
1358
|
+
const lineRaw = raw.slice(previous, at - 1);
|
|
1359
|
+
at = previous;
|
|
1360
|
+
if (!line.trim() || indentOf(line) >= limit) continue;
|
|
1361
|
+
const header = line.trim();
|
|
1362
|
+
if (/^(?:\}\s*)?(?:else\s+)?if\s*\(/.test(header)) {
|
|
1363
|
+
return !MENTIONS_CI.test(lineRaw) && !alwaysTrue(/\bif\s*\((.*)\)\s*\{?\s*$/.exec(header)?.[1] ?? "");
|
|
1364
|
+
}
|
|
1365
|
+
if (/^(?:\}\s*)?(?:else|catch)\b|^(?:case\b|default\s*:)|^\)\s*\{?\s*$/.test(header)) return true;
|
|
1366
|
+
if (TEST_REGISTRATION.test(header)) return false;
|
|
1367
|
+
limit = indentOf(line);
|
|
1368
|
+
}
|
|
1369
|
+
return false;
|
|
1327
1370
|
}
|
|
1328
1371
|
function callArguments(source, raw, open2) {
|
|
1329
1372
|
if (source[open2] !== "(") return null;
|
|
@@ -1624,10 +1667,14 @@ function auditFileDiffs(files, commitHash = "HEAD", commitDate = (/* @__PURE__ *
|
|
|
1624
1667
|
if (source[previous] === ".") continue;
|
|
1625
1668
|
if (match[2] && !isTestFile(file.path)) continue;
|
|
1626
1669
|
if (match[2] && callArguments(source, raw, match.index + match[0].length)?.length === 0) continue;
|
|
1670
|
+
if ((match[2] === "fit" || match[2] === "fdescribe") && !match[3]) {
|
|
1671
|
+
const first = callArguments(source, raw, match.index + match[0].length)?.[0];
|
|
1672
|
+
if (first !== void 0 && !/^[`'"]/.test(first)) continue;
|
|
1673
|
+
}
|
|
1627
1674
|
if (match[2] && previous >= 0 && (/[(,=:?[&|!+\-*%<]/.test(source[previous]) || /(?:await|return|yield|typeof|void)$/.test(source.slice(Math.max(0, previous - 6), previous + 1)))) continue;
|
|
1628
1675
|
if (/\bfunction$/.test(source.slice(Math.max(0, previous - 8), previous + 1))) continue;
|
|
1629
1676
|
if (source[match.index + match[0].length] === "" && !/\.\s*each\s*$/.test(match[3])) continue;
|
|
1630
|
-
if ((match[1] === "skip" || match[1] === "fixme" || match[1] === "fail") && !match[3] && isConditionalSkip(source, raw, match.index + match[0].length)) continue;
|
|
1677
|
+
if ((match[1] === "skip" || match[1] === "fixme" || match[1] === "fail") && !match[3] && isConditionalSkip(source, raw, match.index + match[0].length, match.index)) continue;
|
|
1631
1678
|
const focus = match[1] === "only" || match[2] === "fit" || match[2] === "fdescribe";
|
|
1632
1679
|
const expectedToFail = match[1] === "failing" || match[1] === "fails" || match[1] === "fail";
|
|
1633
1680
|
infractions.push({
|
|
@@ -2859,6 +2906,14 @@ async function readBlob(cwd, oid, signal, limit = 2 * 1024 * 1024) {
|
|
|
2859
2906
|
if (buffer.length !== size) throw new ConfigError("Git blob size differs from the recorded object.");
|
|
2860
2907
|
return buffer;
|
|
2861
2908
|
}
|
|
2909
|
+
async function ciFilesAt(cwd, sha, signal) {
|
|
2910
|
+
if (!/^[a-f0-9]{40,64}$/.test(sha)) return [];
|
|
2911
|
+
const raw = await readGit(cwd, ["ls-tree", "-r", "-z", "--full-tree", sha, "--", ".github/workflows", ".gitlab-ci.yml"], signal, 1024 * 1024);
|
|
2912
|
+
return raw.toString().split("\0").filter(Boolean).flatMap((entry) => {
|
|
2913
|
+
const match = /^100(?:644|755) blob ([a-f0-9]{40,64})\t(.+\.ya?ml)$/.exec(entry);
|
|
2914
|
+
return match ? [{ path: match[2], oid: match[1] }] : [];
|
|
2915
|
+
});
|
|
2916
|
+
}
|
|
2862
2917
|
async function reviewPolicy(cwd, baseSha, signal) {
|
|
2863
2918
|
const raw = await readGit(cwd, ["ls-tree", "--full-tree", "-z", baseSha, "--", ...CONFIG_FILENAMES], signal, 8192);
|
|
2864
2919
|
const entries = raw.toString().split("\0").filter(Boolean);
|
|
@@ -3208,6 +3263,12 @@ function javascript(text) {
|
|
|
3208
3263
|
const end = closing(code, open2);
|
|
3209
3264
|
if (end > 0) callbacks.push([open2, end]);
|
|
3210
3265
|
}
|
|
3266
|
+
const tests = [];
|
|
3267
|
+
for (const match of code.matchAll(/(?<![\w$.])(?:it|test|describe)(?:\s*\.\s*\w+)*\s*\(/g)) {
|
|
3268
|
+
const open2 = match.index + match[0].length - 1;
|
|
3269
|
+
const end = closing(code, open2);
|
|
3270
|
+
if (end > 0) tests.push([open2, end]);
|
|
3271
|
+
}
|
|
3211
3272
|
const programLevel = (index) => {
|
|
3212
3273
|
let depth = 0;
|
|
3213
3274
|
for (let i = 0; i < index; i++) {
|
|
@@ -3226,8 +3287,9 @@ function javascript(text) {
|
|
|
3226
3287
|
const end = closing(code, open2);
|
|
3227
3288
|
const args = end > 0 ? raw.slice(open2, end) : "";
|
|
3228
3289
|
if (/requireActual|importActual|importOriginal/.test(args)) continue;
|
|
3229
|
-
const
|
|
3230
|
-
|
|
3290
|
+
const factoryHead = /^\(\s*(['"`])[^'"`\n]+\1\s*,\s*(?:async\s+)?(?:function\s*[\w$]*\s*)?(?:\(\s*([A-Za-z_$][\w$]*)\s*[,)]|([A-Za-z_$][\w$]*)\s*=>)/.exec(args);
|
|
3291
|
+
const loader = factoryHead?.[2] ?? factoryHead?.[3];
|
|
3292
|
+
if (loader && new RegExp(`(?<![\\w$])(?:(?<=\\.\\.\\.)|(?<!\\.))${loader.replace(/\$/g, "\\$")}\\s*(?:<[^\\n]*?>)?\\s*\\(`).test(args.slice(args.indexOf(loader) + loader.length))) continue;
|
|
3231
3293
|
const factory = /^\(\s*(['"`])[^'"`\n]+\1\s*,/.test(args) && !/,\s*\{\s*spy\s*:\s*true\b/.test(args);
|
|
3232
3294
|
if (!mocked.has(specifier)) mocked.set(specifier, { index: match.index, fake: factory });
|
|
3233
3295
|
}
|
|
@@ -3256,7 +3318,8 @@ function javascript(text) {
|
|
|
3256
3318
|
if (match[1].startsWith("{")) bind(match[1], specifier);
|
|
3257
3319
|
else bindings.set(match[1], { specifier, name: "*", namespace: true });
|
|
3258
3320
|
}
|
|
3259
|
-
const
|
|
3321
|
+
const candidates = [];
|
|
3322
|
+
const mockedNames = [...bindings].filter(([, binding]) => mocked.has(binding.specifier)).map(([local]) => local.replace(/\$/g, "\\$"));
|
|
3260
3323
|
for (const match of code.matchAll(/(?<![\w$.])expect\s*\(/g)) {
|
|
3261
3324
|
const open2 = match.index + match[0].length - 1;
|
|
3262
3325
|
const end = closing(code, open2);
|
|
@@ -3277,18 +3340,32 @@ function javascript(text) {
|
|
|
3277
3340
|
const name = binding.namespace ? call[2] : call[2] ? void 0 : binding.name;
|
|
3278
3341
|
if (!name) continue;
|
|
3279
3342
|
const identifier = call[1].replace(/\$/g, "\\$");
|
|
3280
|
-
const
|
|
3343
|
+
const configures = new RegExp(`(?<![\\w$.])${identifier}\\b[^;]{0,80}?\\.\\s*mock(?:Return|Resolved|Rejected|Implementation)\\w*\\s*\\(`, "g");
|
|
3344
|
+
const configured = [...raw.matchAll(configures)].some((config) => tests.every((range) => !within(range, config.index) || within(range, match.index)));
|
|
3281
3345
|
if (!mock.fake && !configured) continue;
|
|
3282
3346
|
const key = `${binding.specifier}::${name}`;
|
|
3283
|
-
if (found.has(key)) continue;
|
|
3284
3347
|
const line = lineOf2(mock.index);
|
|
3285
|
-
|
|
3348
|
+
candidates.push({
|
|
3286
3349
|
key,
|
|
3287
|
-
|
|
3288
|
-
|
|
3289
|
-
|
|
3350
|
+
at: match.index,
|
|
3351
|
+
range: innermost(callbacks, match.index),
|
|
3352
|
+
subject: {
|
|
3353
|
+
key,
|
|
3354
|
+
line,
|
|
3355
|
+
snippet: lines[line - 1].trim().slice(0, 120),
|
|
3356
|
+
reason: `assertion on the output of ${name}(), which this test replaces with a mock of "${binding.specifier}"`
|
|
3357
|
+
}
|
|
3290
3358
|
});
|
|
3291
3359
|
}
|
|
3360
|
+
const candidateAt = new Set(candidates.map((candidate) => candidate.at));
|
|
3361
|
+
const mentionsMock = mockedNames.length ? new RegExp(`(?<![\\w$.])(?:${mockedNames.join("|")})(?![\\w$])`) : null;
|
|
3362
|
+
const checksSomethingElse = (range) => !!range && [...code.matchAll(/(?<![\w$.])expect\s*\(/g)].some((other) => {
|
|
3363
|
+
if (!within(range, other.index) || candidateAt.has(other.index)) return false;
|
|
3364
|
+
const open2 = other.index + other[0].length - 1, end = closing(code, open2);
|
|
3365
|
+
return end > 0 && !mentionsMock?.test(code.slice(open2 + 1, end));
|
|
3366
|
+
});
|
|
3367
|
+
const found = /* @__PURE__ */ new Map();
|
|
3368
|
+
for (const candidate of candidates) if (!found.has(candidate.key) && !checksSomethingElse(candidate.range)) found.set(candidate.key, candidate.subject);
|
|
3292
3369
|
return [...found.values()];
|
|
3293
3370
|
}
|
|
3294
3371
|
var DOTTED = /^[A-Za-z_][\w.]*$/;
|
|
@@ -3398,6 +3475,7 @@ function python(text) {
|
|
|
3398
3475
|
const receiver = new RegExp(`(?<![\\w.])${patch.object.replace(/[.$]/g, "\\$&")}\\s*\\.\\s*(?!${attribute}\\b)\\w+\\s*\\(`);
|
|
3399
3476
|
if (receiver.test(code.slice(patch.end + 1, match.index))) continue;
|
|
3400
3477
|
}
|
|
3478
|
+
if (/(?<![\w$])(?:[A-Za-z_]\w*\s*\.\s*)*_*(?:install|wrap|instrument|inject|hook|swap|rewire|reload)\w*\s*\(/.test(code.slice(patch.end + 1, match.index))) continue;
|
|
3401
3479
|
const line = lineOf2(patch.index);
|
|
3402
3480
|
found.set(key, {
|
|
3403
3481
|
key,
|
|
@@ -3428,13 +3506,15 @@ function openBrackets(text) {
|
|
|
3428
3506
|
function configEntries(text, toml) {
|
|
3429
3507
|
const lines = text.split(/\r?\n/);
|
|
3430
3508
|
const entries = [];
|
|
3431
|
-
|
|
3509
|
+
const elements = /* @__PURE__ */ new Map();
|
|
3510
|
+
let section = "", block = 0;
|
|
3432
3511
|
for (let i = 0; i < lines.length; i++) {
|
|
3433
3512
|
const trimmed = lines[i].trim();
|
|
3434
3513
|
if (!trimmed || trimmed.startsWith("#") || trimmed.startsWith(";")) continue;
|
|
3435
|
-
const header =
|
|
3514
|
+
const header = /^(\[\[?)\s*([^\]]+?)\s*\]\]?\s*(?:[#;].*)?$/.exec(trimmed);
|
|
3436
3515
|
if (header) {
|
|
3437
|
-
section = header[
|
|
3516
|
+
section = header[2].replace(/["']/g, "");
|
|
3517
|
+
block = header[1] === "[[" ? (elements.set(section, (elements.get(section) ?? 0) + 1), elements.get(section)) : 0;
|
|
3438
3518
|
continue;
|
|
3439
3519
|
}
|
|
3440
3520
|
const pair = /^("[^"]*"|'[^']*'|[\w.-]+)\s*=\s*(.*)$/.exec(trimmed);
|
|
@@ -3446,14 +3526,16 @@ function configEntries(text, toml) {
|
|
|
3446
3526
|
} else {
|
|
3447
3527
|
while (i + 1 < lines.length && /^\s+\S/.test(lines[i + 1]) && !/^\s*[#;]/.test(lines[i + 1])) value += "\n" + lines[++i].trim();
|
|
3448
3528
|
}
|
|
3449
|
-
entries.push({ section, key: pair[1].replace(/["']/g, ""), value, line: start + 1 });
|
|
3529
|
+
entries.push({ section, key: pair[1].replace(/["']/g, ""), value, line: start + 1, block });
|
|
3450
3530
|
}
|
|
3451
3531
|
return entries;
|
|
3452
3532
|
}
|
|
3533
|
+
var withoutComments = (value) => value.replace(/"(?:\\.|[^"\\\n])*"|'[^'\n]*'|[#;][^\n]*/g, (found) => found[0] === "#" || found[0] === ";" ? "" : found);
|
|
3453
3534
|
function members(value) {
|
|
3454
|
-
const
|
|
3535
|
+
const code = withoutComments(value);
|
|
3536
|
+
const quoted = [...code.matchAll(/"((?:\\.|[^"\\\n])*)"|'([^'\n]*)'/g)].map((match) => (match[1] ?? match[2]).trim()).filter(Boolean);
|
|
3455
3537
|
if (quoted.length) return quoted;
|
|
3456
|
-
return
|
|
3538
|
+
return code.split(/[\s,]+/).map((word) => word.trim()).filter(Boolean);
|
|
3457
3539
|
}
|
|
3458
3540
|
function covers(selector, code) {
|
|
3459
3541
|
if (selector === "ALL") return true;
|
|
@@ -3504,8 +3586,31 @@ function analyzePythonConfig(file, oldText, newText, report) {
|
|
|
3504
3586
|
const base = explicit.length ? explicit : tool === "ruff" ? ["E4", "E7", "E9", "F"] : ["E", "F", "W", "C90"];
|
|
3505
3587
|
return [...base, ...extra].some((selected2) => covers(selected2, code) || covers(code, selected2));
|
|
3506
3588
|
};
|
|
3589
|
+
const versionParts = (text) => /^\d{2,}$/.test(text) ? [Number(text[0]), Number(text.slice(1))] : text.split(".").map(Number);
|
|
3590
|
+
const higher = (was2, now) => {
|
|
3591
|
+
for (let index = 0; index < Math.max(was2.length, now.length); index++) if ((now[index] ?? 0) !== (was2[index] ?? 0)) return (now[index] ?? 0) > (was2[index] ?? 0);
|
|
3592
|
+
return false;
|
|
3593
|
+
};
|
|
3594
|
+
const newest = (text, pattern) => {
|
|
3595
|
+
let best = null;
|
|
3596
|
+
for (const match of text.matchAll(pattern)) {
|
|
3597
|
+
const parts = versionParts(match[1]);
|
|
3598
|
+
if (!best || higher(best, parts)) best = parts;
|
|
3599
|
+
}
|
|
3600
|
+
return best;
|
|
3601
|
+
};
|
|
3602
|
+
const versions = [
|
|
3603
|
+
/^\s*requires-python\s*=\s*["'][<>=!~^\s]*(\d+(?:\.\d+)*)/gm,
|
|
3604
|
+
/^\s*target-version\s*=\s*["']?(?:py)?(\d+(?:\.\d+)*)/gm,
|
|
3605
|
+
/^\s*python_version\s*=\s*["']?(\d+(?:\.\d+)*)/gm,
|
|
3606
|
+
/\bruff\s*(?:(?:===|==|~=|>=|>)\s*["']?|=\s*["'][\^~>=]*)(\d+(?:\.\d+)*)/g
|
|
3607
|
+
];
|
|
3608
|
+
const bumped = versions.some((pattern) => {
|
|
3609
|
+
const was2 = newest(oldText, pattern), now = newest(newText, pattern);
|
|
3610
|
+
return !!was2 && !!now && higher(was2, now);
|
|
3611
|
+
});
|
|
3507
3612
|
for (const [tool, bucket] of ignored(after)) {
|
|
3508
|
-
if (!configured.has(tool)) continue;
|
|
3613
|
+
if (!configured.has(tool) || tool === "ruff" && bumped) continue;
|
|
3509
3614
|
const old = [...was.get(tool)?.keys() ?? []];
|
|
3510
3615
|
const fresh = [...bucket].filter(([code]) => !old.some((known) => covers(known, code)) && enforced(tool, code));
|
|
3511
3616
|
if (fresh.length) report(Math.min(...fresh.map(([, line]) => line)), `${tool} ignore: ${fresh.slice(0, 4).map(([code]) => "+" + code).join(", ")}${fresh.length > 4 ? ", \u2026" : ""}`, "Lint or type checks switched off in the configuration");
|
|
@@ -3520,10 +3625,16 @@ function analyzePythonConfig(file, oldText, newText, report) {
|
|
|
3520
3625
|
report(line, `${tool} select: ${dropped.slice(0, 4).map((code) => "-" + code).join(", ")}${dropped.length > 4 ? ", \u2026" : ""}`, "Lint rules removed from the selected set");
|
|
3521
3626
|
}
|
|
3522
3627
|
}
|
|
3523
|
-
const
|
|
3628
|
+
const tableOf = (entries) => {
|
|
3629
|
+
const modules = /* @__PURE__ */ new Map();
|
|
3630
|
+
for (const entry of entries) if (entry.block && entry.key === "module") modules.set(`${entry.section}#${entry.block}`, members(entry.value).join(","));
|
|
3631
|
+
return (entry) => entry.block ? `${entry.section}#${modules.get(`${entry.section}#${entry.block}`) ?? entry.block}` : entry.section;
|
|
3632
|
+
};
|
|
3633
|
+
const tableBefore = tableOf(before), tableAfter = tableOf(after);
|
|
3634
|
+
const switches = new Map(before.filter((entry) => toolOf(file, entry.section) === "mypy" && MYPY_SWITCHES.test(entry.key)).map((entry) => [`${tableBefore(entry)}|${entry.key}`, truthy(entry.value)]));
|
|
3524
3635
|
for (const entry of after) {
|
|
3525
3636
|
if (toolOf(file, entry.section) !== "mypy" || !MYPY_SWITCHES.test(entry.key)) continue;
|
|
3526
|
-
if (switches.get(`${entry
|
|
3637
|
+
if (switches.get(`${tableAfter(entry)}|${entry.key}`) === true && !truthy(entry.value)) report(entry.line, `mypy ${entry.key}: true \u2192 false`, "Type checker strictness switched off");
|
|
3527
3638
|
}
|
|
3528
3639
|
const ignoring = (entries) => entries.filter((entry) => toolOf(file, entry.section) === "mypy" && entry.key === "ignore_errors" && truthy(entry.value));
|
|
3529
3640
|
const before_ = ignoring(before), after_ = ignoring(after);
|
|
@@ -3796,6 +3907,12 @@ var EMPTY_PY = /==\s*(?:0(?:\.0+)?(?![\w.])|\[\s*\]|\{\s*\}|""|''|\(\s*\)|(?:set
|
|
|
3796
3907
|
var PY_SPECIFIC = /==|!=|<=|>=|\bin\b|\bis\b|\bisinstance\s*\(|\.startswith\s*\(|\.endswith\s*\(|\bcallable\s*\(/;
|
|
3797
3908
|
var JS_SPECIFIC = /\.(?:toBe|toEqual|toStrictEqual|toHaveLength|toContain|toContainEqual|toMatch|toMatchObject|toHaveProperty|toHaveBeenCalledWith|toHaveBeenLastCalledWith|toHaveBeenNthCalledWith|toBeCalledWith|toHaveBeenCalledTimes|toHaveBeenCalledOnce|toHaveTextContent|toHaveAttribute|toHaveValue|toThrow\w*|toBeGreaterThan\w*|toBeLessThan\w*)\s*\(/;
|
|
3798
3909
|
var THROWING_QUERY = /\b(?:get|find)(?:All)?By\w*\s*\(/;
|
|
3910
|
+
var PRESENCE_PREDICATE = /\.(?:exists|isVisible|isExisting|isDisplayed)\(\)$/;
|
|
3911
|
+
function samePresence(oldAssertion, newAssertion) {
|
|
3912
|
+
if (!/\.toBe\s*\(\s*true\s*\)\s*;?\s*$/.test(oldAssertion) || !/\.(?:not\s*\.\s*toBeNull|toBeTruthy|toBeDefined|toBeInTheDocument)\s*\(\s*\)\s*;?\s*$/.test(newAssertion)) return false;
|
|
3913
|
+
const was = checkedExpression(oldAssertion, "js"), now = checkedExpression(newAssertion, "js");
|
|
3914
|
+
return !!was && !!now && PRESENCE_PREDICATE.test(was) && (PRESENCE_PREDICATE.test(now) || /(?:querySelector|queryBy\w*|getBy\w*|findBy\w*|\.find|\.get|\$)\(/.test(now));
|
|
3915
|
+
}
|
|
3799
3916
|
function checkedExpression(assertion, language) {
|
|
3800
3917
|
const squash = (text) => text.replace(/\s+/g, "").replace(/^len\((.*)\)$/, "$1").replace(/\.length$/, "");
|
|
3801
3918
|
if (language === "py") {
|
|
@@ -3846,12 +3963,33 @@ function admits(value, operator, limit) {
|
|
|
3846
3963
|
return true;
|
|
3847
3964
|
}
|
|
3848
3965
|
}
|
|
3966
|
+
var EXACT_MATCHERS = ["toBe", "toEqual", "toStrictEqual", "toHaveLength"];
|
|
3967
|
+
function comparedValue(assertion, language, negated) {
|
|
3968
|
+
if (language === "js") {
|
|
3969
|
+
for (const matcher of EXACT_MATCHERS) {
|
|
3970
|
+
const value = matcherArgument(assertion, negated ? `not\\s*\\.\\s*${matcher}` : matcher);
|
|
3971
|
+
if (value !== null) return value;
|
|
3972
|
+
}
|
|
3973
|
+
return null;
|
|
3974
|
+
}
|
|
3975
|
+
const call = new RegExp(`\\bself\\s*\\.\\s*assert${negated ? "NotEqual" : "Equal"}\\s*\\(([\\s\\S]*)\\)\\s*;?\\s*$`).exec(assertion);
|
|
3976
|
+
if (call) return splitArguments(call[1])[1]?.replace(/\s+/g, "") ?? null;
|
|
3977
|
+
const at = assertion.indexOf(negated ? "!=" : "==");
|
|
3978
|
+
return at < 0 ? null : splitArguments(assertion.slice(at + 2))[0]?.replace(/\s+/g, "") ?? null;
|
|
3979
|
+
}
|
|
3980
|
+
var NOTHING_OR_BOOLEAN = /^(?:true|false|True|False|null|undefined|None|NaN|0(?:\.0+)?|""|''|``|\[\]|\{\}|\(\))$/;
|
|
3981
|
+
var LITERAL_VALUE = /^(?:-?\d+(?:\.\d+)?|"(?:[^"\\]|\\.)*"|'(?:[^'\\]|\\.)*'|null|undefined|None|\[\]|\{\}|\(\))$/;
|
|
3982
|
+
function exactBecameExclusion(oldAssertion, newAssertion, language) {
|
|
3983
|
+
const pinned = comparedValue(oldAssertion, language, false), ruledOut = comparedValue(newAssertion, language, true);
|
|
3984
|
+
return pinned !== null && ruledOut !== null && pinned !== "" && !NOTHING_OR_BOOLEAN.test(pinned) && LITERAL_VALUE.test(ruledOut) && pinned !== ruledOut;
|
|
3985
|
+
}
|
|
3849
3986
|
function relaxes(oldAssertion, newAssertion, language, siblings = []) {
|
|
3850
3987
|
const subject = checkedExpression(oldAssertion, language);
|
|
3851
3988
|
if (!subject || subject !== checkedExpression(newAssertion, language)) return false;
|
|
3852
3989
|
if (siblings.some((other) => isExact(other, language) && checkedExpression(other, language) === subject)) return false;
|
|
3853
3990
|
if (language === "js") {
|
|
3854
|
-
|
|
3991
|
+
const negated = (text) => /\.\s*not\s*\./.test(text);
|
|
3992
|
+
if (negated(oldAssertion) || negated(newAssertion)) return !negated(oldAssertion) && exactBecameExclusion(oldAssertion, newAssertion, language);
|
|
3855
3993
|
const exact = /\.(?:toBe|toEqual|toStrictEqual|toHaveLength)\s*\(\s*(-?\d+(?:\.\d+)?)\s*\)/.exec(oldAssertion)?.[1] ?? /assert\s*\.\s*(?:equal|strictEqual|deepEqual|deepStrictEqual)\s*\([^,]+,\s*(-?\d+(?:\.\d+)?)\s*[,)]/.exec(oldAssertion)?.[1];
|
|
3856
3994
|
const bound = /\.(?:toBeGreaterThan|toBeGreaterThanOrEqual|toBeLessThan|toBeLessThanOrEqual)\s*\(/.test(newAssertion) || /assert\s*\.\s*ok\s*\([^;]*(?:>=|<=|>|<)/.test(newAssertion);
|
|
3857
3995
|
if (exact !== void 0 && Number(exact) !== 0 && bound) {
|
|
@@ -3871,6 +4009,8 @@ function relaxes(oldAssertion, newAssertion, language, siblings = []) {
|
|
|
3871
4009
|
const operator = (text) => /^\s*assert\s+(?:not\s+)?.+?\s*(==|!=|>=|<=|>|<|\bin\b|\bis\b)/.exec(text)?.[1] ?? "";
|
|
3872
4010
|
const exact = operator(oldAssertion) === "==" && plain(oldAssertion) || /\bself\s*\.\s*assertEqual\s*\(/.test(oldAssertion);
|
|
3873
4011
|
const bound = [">=", "<=", ">", "<"].includes(operator(newAssertion)) && plain(newAssertion) || /\bself\s*\.\s*assert(?:Greater|Less)(?:Equal)?\s*\(/.test(newAssertion);
|
|
4012
|
+
const unequal = operator(newAssertion) === "!=" && plain(newAssertion) && !/^\s*assert\s+not\b/.test(newAssertion) || /\bself\s*\.\s*assertNotEqual\s*\(/.test(newAssertion);
|
|
4013
|
+
if (exact && unequal) return exactBecameExclusion(oldAssertion, newAssertion, language);
|
|
3874
4014
|
if (!exact || !bound) return false;
|
|
3875
4015
|
const value = (text) => /==\s*(-?\d+(?:\.\d+)?|[\w.]+(?:\([^)]*\))?)\s*(?:,|$)/.exec(text)?.[1] ?? /\bassertEqual\s*\([^,]+,\s*(-?\d+(?:\.\d+)?|[\w.]+)\s*[,)]/.exec(text)?.[1];
|
|
3876
4016
|
const limit = (text) => {
|
|
@@ -3887,6 +4027,34 @@ function relaxes(oldAssertion, newAssertion, language, siblings = []) {
|
|
|
3887
4027
|
}
|
|
3888
4028
|
return false;
|
|
3889
4029
|
}
|
|
4030
|
+
function splitArguments(text) {
|
|
4031
|
+
const parts = [];
|
|
4032
|
+
let depth = 0, quote = "", start = 0;
|
|
4033
|
+
for (let i = 0; i < text.length; i++) {
|
|
4034
|
+
const char = text[i];
|
|
4035
|
+
if (quote) {
|
|
4036
|
+
if (char === "\\") i++;
|
|
4037
|
+
else if (char === quote) quote = "";
|
|
4038
|
+
} else if (char === '"' || char === "'") quote = char;
|
|
4039
|
+
else if ("([{".includes(char)) depth++;
|
|
4040
|
+
else if (")]}".includes(char)) depth--;
|
|
4041
|
+
else if (char === "," && depth === 0) {
|
|
4042
|
+
parts.push(text.slice(start, i).trim());
|
|
4043
|
+
start = i + 1;
|
|
4044
|
+
}
|
|
4045
|
+
}
|
|
4046
|
+
parts.push(text.slice(start).trim());
|
|
4047
|
+
return parts.filter(Boolean);
|
|
4048
|
+
}
|
|
4049
|
+
function unittestAsAssert(statement) {
|
|
4050
|
+
const call = /^\s*self\s*\.\s*assert(True|IsNotNone|GreaterEqual|Greater)\s*\(([\s\S]*)\)\s*;?\s*$/.exec(statement);
|
|
4051
|
+
if (!call) return null;
|
|
4052
|
+
const [first, second] = splitArguments(call[2]);
|
|
4053
|
+
if (!first) return null;
|
|
4054
|
+
if (call[1] === "True") return `assert ${first}`;
|
|
4055
|
+
if (call[1] === "IsNotNone") return `assert ${first} is not None`;
|
|
4056
|
+
return second === void 0 ? null : `assert ${first} ${call[1] === "Greater" ? ">" : ">="} ${second}`;
|
|
4057
|
+
}
|
|
3890
4058
|
function pinsLess(oldAssertion, newAssertion, language) {
|
|
3891
4059
|
if (language === "js") {
|
|
3892
4060
|
const was = /\.toThrow(?:Error)?\s*\(\s*([^)\s][^)]*)\)/.exec(oldAssertion);
|
|
@@ -3909,6 +4077,7 @@ function pinsLess(oldAssertion, newAssertion, language) {
|
|
|
3909
4077
|
}
|
|
3910
4078
|
function isWeaker(rawOld, rawNew, language) {
|
|
3911
4079
|
const oldAssertion = language === "py" ? withoutOuterParens(rawOld) : rawOld, newAssertion = language === "py" ? withoutOuterParens(rawNew) : rawNew;
|
|
4080
|
+
if (language === "js" && samePresence(oldAssertion, newAssertion)) return false;
|
|
3912
4081
|
if (pinsLess(oldAssertion, newAssertion, language) || relaxes(oldAssertion, newAssertion, language)) return true;
|
|
3913
4082
|
if (language === "js") {
|
|
3914
4083
|
if (THROWING_QUERY.test(newAssertion) || !JS_SPECIFIC.test(oldAssertion) || WEAK_JS.test(oldAssertion) || BOUND_JS.test(oldAssertion)) return false;
|
|
@@ -3916,7 +4085,8 @@ function isWeaker(rawOld, rawNew, language) {
|
|
|
3916
4085
|
return WEAK_JS.test(newAssertion) || BOUND_JS.test(newAssertion) && !ZERO_JS.test(oldAssertion) && (limit === void 0 || /^-?\d+(?:\.\d+)?$/.test(limit.trim()));
|
|
3917
4086
|
}
|
|
3918
4087
|
if (language === "py") {
|
|
3919
|
-
|
|
4088
|
+
const oldBare = unittestAsAssert(oldAssertion) ?? oldAssertion;
|
|
4089
|
+
if (WEAK_PY.test(oldAssertion) || BOUND_PY.test(oldAssertion) || WEAK_PY.test(oldBare) || BOUND_PY.test(oldBare) || !PY_SPECIFIC.test(oldAssertion)) return false;
|
|
3920
4090
|
const limit = /(?:>=|<=|>|<)\s*([^\s,)]+)/.exec(oldAssertion)?.[1];
|
|
3921
4091
|
return WEAK_PY.test(newAssertion) || BOUND_PY.test(newAssertion) && !EMPTY_PY.test(oldAssertion) && /==|[<>]=?/.test(oldAssertion) && (limit === void 0 || /^-?\d+(?:\.\d+)?$/.test(limit));
|
|
3922
4092
|
}
|
|
@@ -4105,10 +4275,10 @@ function markedTest(finding2, lines) {
|
|
|
4105
4275
|
if (/\.py$/i.test(finding2.file ?? "")) {
|
|
4106
4276
|
if (/^\s*pytestmark\b/.test(snippet)) return null;
|
|
4107
4277
|
if (/imperative/.test(finding2.reason)) {
|
|
4108
|
-
const
|
|
4278
|
+
const indent = (lines[index] ?? "").search(/\S/);
|
|
4109
4279
|
for (let i = index - 1; i >= 0 && i > index - 200; i--) {
|
|
4110
4280
|
const def = /^(\s*)(?:async\s+)?def\s+(\w+)\s*\(/.exec(lines[i]);
|
|
4111
|
-
if (def && def[1].length <
|
|
4281
|
+
if (def && def[1].length < indent) return { key: `py:${def[2]}`, title: def[2] };
|
|
4112
4282
|
}
|
|
4113
4283
|
return null;
|
|
4114
4284
|
}
|
|
@@ -4133,18 +4303,25 @@ function markedTest(finding2, lines) {
|
|
|
4133
4303
|
return null;
|
|
4134
4304
|
}
|
|
4135
4305
|
if (/\.each\b/.test(snippet)) return null;
|
|
4136
|
-
const title = /\b(?:it|test|describe|suite|xit|xtest|xdescribe|fit|fdescribe)\b[\w$.\s]*\(\s*(['"`])((?:\\.|(?!\1)[^\\\n])+)\1\s*[,)]/.exec(snippet);
|
|
4306
|
+
const title = /\b(?:it|test|describe|suite|xit|xtest|xdescribe|fit|fdescribe)\b[\w$.\s]*\(\s*(['"`])((?:\\.|(?!\1)[^\\\n])+)\1\s*(?:[,)]|$)/.exec(titleOnNextLine(snippet, lines, index));
|
|
4137
4307
|
if (title) return { key: `js:${title[1]}${title[2]}${title[1]}`, title: `${title[1]}${title[2]}${title[1]}` };
|
|
4138
|
-
|
|
4139
|
-
if (
|
|
4140
|
-
for (let i = index - 1; i >= 0 && i > index - 200; i--) {
|
|
4141
|
-
if (!lines[i].trim() || lines[i].search(/\S/) >=
|
|
4142
|
-
|
|
4143
|
-
|
|
4308
|
+
let limit = (lines[index] ?? "").search(/\S/);
|
|
4309
|
+
if (limit > 0 && /\b(?:test|it)\s*\.\s*(?:skip|fixme|fail)\s*\(/.test(snippet)) {
|
|
4310
|
+
for (let i = index - 1; i >= 0 && i > index - 200 && limit > 0; i--) {
|
|
4311
|
+
if (!lines[i].trim() || lines[i].search(/\S/) >= limit) continue;
|
|
4312
|
+
limit = lines[i].search(/\S/);
|
|
4313
|
+
const owner = /^\s*(?:test|it)(?:\s*\.\s*(?!describe\b|step\b)\w+)*\s*\(\s*(['"`])((?:\\.|(?!\1)[^\\\n])+)\1/.exec(titleOnNextLine(lines[i], lines, i));
|
|
4314
|
+
if (owner) return { key: `js:${owner[1]}${owner[2]}${owner[1]}`, title: `${owner[1]}${owner[2]}${owner[1]}` };
|
|
4315
|
+
if (/^\s*(?:await\s+)?(?:test|it)\s*\.\s*describe\b|^\s*(?:describe|suite)\b/.test(lines[i])) return null;
|
|
4144
4316
|
}
|
|
4145
4317
|
}
|
|
4146
4318
|
return null;
|
|
4147
4319
|
}
|
|
4320
|
+
function titleOnNextLine(text, lines, index) {
|
|
4321
|
+
if (!/\(\s*$/.test(text)) return text;
|
|
4322
|
+
const next = (lines[index + 1] ?? "").trim();
|
|
4323
|
+
return /^(['"`])/.test(next) ? `${text} ${next}` : text;
|
|
4324
|
+
}
|
|
4148
4325
|
function newTestMarkers(before, after, oldText, newText) {
|
|
4149
4326
|
const markers = after.filter(isTestMarker);
|
|
4150
4327
|
if (!markers.length || oldText === null) return [];
|
|
@@ -4200,32 +4377,111 @@ var THRESHOLD_PATTERNS = [
|
|
|
4200
4377
|
var SWITCHES = ["strict", "noImplicitAny", "strictNullChecks", "noImplicitReturns", "useUnknownInCatchVariables"];
|
|
4201
4378
|
var GATE_SCRIPT = /^(?:pre|post)?(?:test|lint|typecheck|type-check|check|ci|verify|validate)(?::[\w:.-]+)?$/;
|
|
4202
4379
|
var CI_RUNNER = new RegExp(
|
|
4203
|
-
String.raw`(?<![\w./-])(?:pytest|vitest|jest|mocha|tox|nox)(?![\w/-])` + String.raw`|(?<![\w./-])(?:npm|yarn|pnpm|bun|go|cargo|mvn|gradle|make|dotnet|flutter)\s+(?:run\s+)?(?:-\S+\s+)*test(?![\w/])` + //
|
|
4204
|
-
String.raw`|(?<![\w./-])(?:
|
|
4380
|
+
String.raw`(?<![\w./-])(?:pytest|vitest|jest|mocha|tox|nox)(?![\w/-])` + String.raw`|(?<![\w./-])(?:npm|yarn|pnpm|bun|go|cargo|mvn|gradle|make|dotnet|flutter)\s+(?:run\s+)?(?:-\S+\s+)*test(?![\w/])` + // A package script that runs the tests with coverage: `yarn coverage`, `npm run coverage`.
|
|
4381
|
+
String.raw`|(?<![\w./-])(?:npm|yarn|pnpm|bun)\s+(?:run\s+)?coverage(?![\w/-])` + // Monorepo runners: `nx affected --target=test`, `turbo run test`, `lerna run test`, `bazel test //...`.
|
|
4382
|
+
String.raw`|(?<![\w./-])(?:nx|turbo|lerna|rush)\s+(?:run-many|affected|run|test)\b[^\n]*(?<![\w-])(?:test|tests)(?![\w-])` + String.raw`|(?<![\w./-])(?:bazel\s+test|ctest|phpunit|rspec|deno\s+test)(?![\w/-])` + // Python's own runners and the browser runners: `python3 -m unittest`, `nosetests`, `playwright test`, `cypress run`, `karma start`.
|
|
4383
|
+
String.raw`|(?<![\w./-])python[\d.]*\s+-m\s+(?:unittest|pytest|nose2?)(?![\w/-])` + String.raw`|(?<![\w./-])(?:nosetests|nose2|py\.test|stestr|behave|jasmine|playwright\s+test|cypress\s+run|karma\s+start|hatch\s+(?:run\s+)?test|swift\s+test)(?![\w/-])` + // Task runners: `deno task test`, `just test`, `rake test`, `composer test`, `mix test`, `sbt test`, `./gradlew test`, and Node's own `node --test`.
|
|
4384
|
+
String.raw`|(?<![\w./-])(?:deno\s+(?:task|run)|just|task|poe|rake|composer(?:\s+run)?|mix|sbt|lein|stack|cabal|(?:\./)?(?:gradlew|mvnw)(?:\.bat|\.cmd)?)\s+(?:-\S+\s+)*test(?![\w/-])` + String.raw`|(?<![\w./-])(?:node|tsx)(?:\s+-\S+)*\s+--test(?![\w-])` + // A project's own test script, run through its interpreter or by path: `python scripts/run_tests.py`, `./scripts/test.sh`, `python tests/test_cli.py`.
|
|
4385
|
+
String.raw`|(?<![\w./-])(?:(?:python[\d.]*|node|bash|sh|zsh|ruby|pwsh|tsx|ts-node)(?:\s+-\S+)*\s+|\./)(?:[\w.-]+/)*(?:(?:run[_-]?)?tests?(?:[_-](?:all|suite|runner|unit|ci))?|test_[\w-]+|[\w-]+_test)\.(?:py|sh|bash|js|mjs|cjs|ts|rb|ps1)(?![\w-])` + String.raw`|\brun:.*(?<=\s)(?:tests?|unit|e2e)(?![\w/.-])`
|
|
4205
4386
|
);
|
|
4387
|
+
var SHELL_TEST = /(?:^|run:\s*|[;&|({]\s*|\b(?:if|then|do|else|elif|while|until)\s+|!\s+)test\s+!?\s*-[bcdefghkLnOGprsStuwxzN](?=\s)/g;
|
|
4388
|
+
var ciRuns = (line) => CI_RUNNER.test(line.replace(SHELL_TEST, " ").replace(/\s#.*$/, ""));
|
|
4206
4389
|
var CI_FILE = /(?:\.github\/workflows\/|\.gitlab-ci)/;
|
|
4207
4390
|
var CI_REUSABLE = /^\s*-?\s*uses:\s*\S*\.github\/workflows\/\S+/;
|
|
4208
4391
|
var CI_GATE = /^\s*(?:pull_request|pull_request_target|push|merge_group)\s*:|^\s*on\s*:\s*(?:\[[^\]]*\b(?:push|pull_request|pull_request_target|merge_group)\b[^\]]*\]|(?:push|pull_request|pull_request_target|merge_group)\s*$)|^\s*-\s*(?:push|pull_request|pull_request_target|merge_group)\s*$/m;
|
|
4209
|
-
|
|
4210
|
-
|
|
4392
|
+
var CI_NON_GATE = /^\s*(?:schedule|release|workflow_dispatch|workflow_run|repository_dispatch)\s*:/m;
|
|
4393
|
+
function gatesChanges(file, text) {
|
|
4394
|
+
const section = triggerSection(text), triggers = gatingTriggers(text) ?? text;
|
|
4395
|
+
if (/\.gitlab-ci/.test(file) || CI_GATE.test(triggers) || /\bworkflow_call\b/.test(triggers)) return true;
|
|
4396
|
+
return section !== null && triggers !== section ? false : !CI_NON_GATE.test(triggers);
|
|
4397
|
+
}
|
|
4398
|
+
function gatingTriggers(text) {
|
|
4399
|
+
const section = triggerSection(text);
|
|
4400
|
+
if (section === null) return null;
|
|
4401
|
+
const lines = section.split(/\r?\n/), kept = [];
|
|
4402
|
+
for (let i = 0; i < lines.length; i++) {
|
|
4403
|
+
const head = /^(\s*)push\s*:\s*(?:#.*)?$/.exec(lines[i]);
|
|
4404
|
+
if (!head) {
|
|
4405
|
+
kept.push(lines[i]);
|
|
4406
|
+
continue;
|
|
4407
|
+
}
|
|
4408
|
+
let end = i + 1;
|
|
4409
|
+
while (end < lines.length && (!lines[end].trim() || lines[end].search(/\S/) > head[1].length)) end++;
|
|
4410
|
+
const block = lines.slice(i + 1, end).join("\n");
|
|
4411
|
+
if (!(/^\s*tags\s*:/m.test(block) && !/^\s*branches(?:-ignore)?\s*:/m.test(block))) kept.push(...lines.slice(i, end));
|
|
4412
|
+
i = end - 1;
|
|
4413
|
+
}
|
|
4414
|
+
return kept.join("\n");
|
|
4415
|
+
}
|
|
4416
|
+
function triggerSection(text) {
|
|
4417
|
+
const lines = text.split(/\r?\n/);
|
|
4418
|
+
const start = lines.findIndex((line) => /^["']?on["']?\s*:/.test(line));
|
|
4419
|
+
if (start < 0) return null;
|
|
4420
|
+
let end = start + 1;
|
|
4421
|
+
while (end < lines.length && (!lines[end].trim() || /^[\s#-]/.test(lines[end]))) end++;
|
|
4422
|
+
return lines.slice(start, end).join("\n");
|
|
4423
|
+
}
|
|
4424
|
+
function gatingTestCommands(file, text) {
|
|
4425
|
+
if (!(/\.gitlab-ci/.test(file) || CI_GATE.test(gatingTriggers(text) ?? text))) return [];
|
|
4426
|
+
const directories = [...new Set([...text.matchAll(/^\s*(?:-\s+)?working-directory\s*:\s*["']?([^\s"'#]+)/gm)].map((match) => match[1]))].sort().join(",");
|
|
4427
|
+
return text.split(/\r?\n/).filter((line) => !line.trim().startsWith("#") && ciRuns(line)).map((line) => `${line.replace(/\s#.*$/, "").replace(/^\s*-?\s*(?:run\s*:\s*)?/, "").replace(/\s+/g, " ").trim()} @${directories}`);
|
|
4211
4428
|
}
|
|
4212
4429
|
function matchingLines(text, pattern) {
|
|
4213
|
-
|
|
4430
|
+
const matches = typeof pattern === "function" ? pattern : (line) => pattern.test(line);
|
|
4431
|
+
return text.split(/\r?\n/).filter((line) => !line.trim().startsWith("#") && matches(line)).length;
|
|
4214
4432
|
}
|
|
4215
4433
|
function ciTestSteps(file, oldText, newText) {
|
|
4216
4434
|
if (!CI_FILE.test(file) || oldText === null && newText === null) return null;
|
|
4217
4435
|
const was = oldText ?? "", now = newText ?? "";
|
|
4218
|
-
const
|
|
4219
|
-
|
|
4436
|
+
const commandsBefore = gatingTestCommands(file, was), commandsAfter = gatingTestCommands(file, now);
|
|
4437
|
+
const after = commandsAfter.length;
|
|
4438
|
+
return { file, before: commandsBefore.length, after, reusableAfter: matchingLines(now, CI_REUSABLE), ungatedAfter: after === 0 ? matchingLines(now, ciRuns) : 0, commandsBefore, commandsAfter };
|
|
4220
4439
|
}
|
|
4221
|
-
function resolveCiTestSteps(counts, commit) {
|
|
4440
|
+
function resolveCiTestSteps(counts, commit, elsewhere = /* @__PURE__ */ new Set()) {
|
|
4222
4441
|
const before = counts.reduce((sum, item) => sum + item.before, 0);
|
|
4223
4442
|
const after = counts.reduce((sum, item) => sum + item.after + item.reusableAfter, 0);
|
|
4224
4443
|
if (after >= before) return [];
|
|
4225
|
-
|
|
4444
|
+
const running = /* @__PURE__ */ new Set([...counts.flatMap((item) => item.commandsAfter ?? []), ...elsewhere]);
|
|
4445
|
+
return counts.filter((item) => item.before > 0 && item.after === 0 && item.reusableAfter === 0).filter((item) => item.commandsBefore === void 0 || item.commandsBefore.some((command) => !running.has(command))).map((item) => item.ungatedAfter > 0 ? finding("guard-weakened", item.file, 1, "tests no longer run on push or pull_request", "Workflow keeps its tests but no longer runs on pushes or pull requests", commit) : finding("guard-weakened", item.file, 1, "test command no longer runs in CI", "Test step removed from CI", commit));
|
|
4226
4446
|
}
|
|
4227
4447
|
var LENIENT = /continue-on-error:\s*true|allow_failure:\s*true|^\s*(?:-\s+)?if:\s*(?:\$\{\{\s*)?false\s*(?:\}\})?\s*(?:#.*)?$/;
|
|
4228
4448
|
var SHELL_SWALLOW = /\|\|\s*(?:true|exit\s+0|:|echo\b)|;\s*(?:true|exit\s+0)\b/;
|
|
4449
|
+
var NON_CHECK_ACTION = new RegExp(String.raw`^\s*(?:-\s+)?uses:\s*["']?(?:` + [
|
|
4450
|
+
String.raw`actions/(?:upload-artifact|download-artifact|upload-pages-artifact|deploy-pages|cache|attest[\w-]*|checkout|setup-[\w-]+)`,
|
|
4451
|
+
String.raw`docker/(?:login-action|metadata-action|build-push-action|setup-[\w-]+)`,
|
|
4452
|
+
String.raw`azure/login`,
|
|
4453
|
+
String.raw`aws-actions/(?:configure-aws-credentials|amazon-ecr-login)`,
|
|
4454
|
+
String.raw`google-github-actions/(?:auth|setup-gcloud)`,
|
|
4455
|
+
String.raw`softprops/action-gh-release`,
|
|
4456
|
+
String.raw`ncipollo/release-action`,
|
|
4457
|
+
String.raw`pypa/gh-action-pypi-publish`,
|
|
4458
|
+
String.raw`JS-DevTools/npm-publish`,
|
|
4459
|
+
String.raw`peaceiris/actions-gh-pages`,
|
|
4460
|
+
String.raw`peter-evans/[\w-]+`,
|
|
4461
|
+
String.raw`marocchino/sticky-pull-request-comment`,
|
|
4462
|
+
String.raw`slackapi/slack-github-action`,
|
|
4463
|
+
String.raw`codecov/codecov-action`,
|
|
4464
|
+
String.raw`coverallsapp/github-action`
|
|
4465
|
+
].join("|") + String.raw`)(?:/[\w./-]+)?@`, "i");
|
|
4466
|
+
function scopeCommands(scope) {
|
|
4467
|
+
const found = [];
|
|
4468
|
+
scope.forEach((line, index) => {
|
|
4469
|
+
const head = /^(\s*(?:-\s+)?)(?:run|uses)\s*:\s*(.*)$/.exec(line);
|
|
4470
|
+
if (!head || line.trim().startsWith("#")) return;
|
|
4471
|
+
const text = head[2].replace(/\s#.*$/, "").trim();
|
|
4472
|
+
if (text && !/^[|>][+-]?$/.test(text)) {
|
|
4473
|
+
found.push(text.replace(/\s+/g, " "));
|
|
4474
|
+
return;
|
|
4475
|
+
}
|
|
4476
|
+
const keyAt = head[1].length;
|
|
4477
|
+
for (let next = index + 1; next < scope.length; next++) {
|
|
4478
|
+
if (!scope[next].trim()) continue;
|
|
4479
|
+
if (scope[next].length - scope[next].trimStart().length <= keyAt) break;
|
|
4480
|
+
if (!scope[next].trim().startsWith("#")) found.push(scope[next].trim().replace(/\s+/g, " "));
|
|
4481
|
+
}
|
|
4482
|
+
});
|
|
4483
|
+
return found;
|
|
4484
|
+
}
|
|
4229
4485
|
function lenientSteps(text) {
|
|
4230
4486
|
const lines = text.split(/\r?\n/);
|
|
4231
4487
|
const indentOf = (line) => line.length - line.trimStart().length;
|
|
@@ -4242,9 +4498,20 @@ function lenientSteps(text) {
|
|
|
4242
4498
|
while (j >= 0 && (!lines[j].trim() || lines[j].trim().startsWith("#") || indentOf(lines[j]) >= indentOf(line))) j--;
|
|
4243
4499
|
identity = j >= 0 ? lines[j].trim() : "";
|
|
4244
4500
|
}
|
|
4501
|
+
const start = /^\s*-\s+if:/.test(line) ? i : Math.max(j, 0), level = indentOf(lines[start]);
|
|
4502
|
+
let end = start + 1;
|
|
4503
|
+
while (end < lines.length && (!lines[end].trim() || lines[end].trim().startsWith("#") || indentOf(lines[end]) > level)) end++;
|
|
4504
|
+
const scope = lines.slice(start, end), runsChecks = matchingLines(scope.join("\n"), ciRuns) > 0;
|
|
4505
|
+
const uses = scope.filter((item) => /^\s*(?:-\s+)?uses:/.test(item));
|
|
4506
|
+
const nonCheck = !runsChecks && uses.length === 1 && NON_CHECK_ACTION.test(uses[0]) && !scope.some((item) => /^\s*(?:-\s+)?run:/.test(item));
|
|
4507
|
+
const commands = scopeCommands(scope);
|
|
4245
4508
|
const entry = found.get(identity);
|
|
4246
|
-
if (entry)
|
|
4247
|
-
|
|
4509
|
+
if (entry) {
|
|
4510
|
+
entry.count++;
|
|
4511
|
+
entry.runsChecks ||= runsChecks;
|
|
4512
|
+
entry.nonCheck &&= nonCheck;
|
|
4513
|
+
entry.commands.push(...commands);
|
|
4514
|
+
} else found.set(identity, { count: 1, line: i + 1, runsChecks, nonCheck, commands });
|
|
4248
4515
|
});
|
|
4249
4516
|
return found;
|
|
4250
4517
|
}
|
|
@@ -4306,16 +4573,22 @@ function analyzeGuardFile(file, oldText, newText, commit) {
|
|
|
4306
4573
|
});
|
|
4307
4574
|
}
|
|
4308
4575
|
if (CI_FILE.test(file)) {
|
|
4576
|
+
const gating = gatesChanges(file, oldText) || gatesChanges(file, newText);
|
|
4309
4577
|
const before = lenientSteps(oldText), existing = new Set(oldText.split(/\r?\n/).flatMap((line) => [line.trim(), line.trim().replace(/^-\s+/, "")]));
|
|
4310
4578
|
const newLines = newText.split(/\r?\n/);
|
|
4579
|
+
const toleratedPlaces = (text) => text.split(/\r?\n/).filter((line) => !line.trim().startsWith("#") && /continue-on-error:\s*true|allow_failure:\s*true/.test(line)).length;
|
|
4580
|
+
const toleratedBefore = new Set([...before.values()].flatMap((step) => step.commands));
|
|
4581
|
+
const moved = toleratedPlaces(newText) <= toleratedPlaces(oldText);
|
|
4311
4582
|
for (const [identity, now2] of lenientSteps(newText)) {
|
|
4583
|
+
if (now2.nonCheck || !gating && !now2.runsChecks) continue;
|
|
4584
|
+
if (moved && now2.commands.length > 0 && now2.commands.every((command) => toleratedBefore.has(command))) continue;
|
|
4312
4585
|
if (identity && existing.has(identity) && now2.count > (before.get(identity)?.count ?? 0)) {
|
|
4313
4586
|
const marker = (newLines[now2.line - 1] ?? "").trim();
|
|
4314
4587
|
out.push(finding("guard-weakened", file, now2.line, marker, /^(?:-\s+)?if:/.test(marker) ? "CI step is now switched off" : "CI step is now allowed to fail", commit));
|
|
4315
4588
|
break;
|
|
4316
4589
|
}
|
|
4317
4590
|
}
|
|
4318
|
-
const swallowing = (text) => text.split(/\r?\n/).map((line, index) => ({ line: line.trim(), index })).filter((item) => !item.line.startsWith("#") &&
|
|
4591
|
+
const swallowing = (text) => text.split(/\r?\n/).map((line, index) => ({ line: line.trim(), index })).filter((item) => !item.line.startsWith("#") && ciRuns(item.line) && SHELL_SWALLOW.test(item.line));
|
|
4319
4592
|
const was = swallowing(oldText), now = swallowing(newText);
|
|
4320
4593
|
const tolerance = (text) => text.split(/\r?\n/).filter((line) => !line.trim().startsWith("#") && /continue-on-error:\s*true|allow_failure:\s*true/.test(line)).length;
|
|
4321
4594
|
const traded = Math.max(0, tolerance(oldText) - tolerance(newText));
|
|
@@ -4520,7 +4793,23 @@ async function compareSnapshot(options, cwd, snapshot) {
|
|
|
4520
4793
|
await snapshot?.verify();
|
|
4521
4794
|
result.complete = result.skippedFiles.length === 0;
|
|
4522
4795
|
const rules = policy.config.rules ?? {};
|
|
4523
|
-
|
|
4796
|
+
const elsewhere = /* @__PURE__ */ new Set();
|
|
4797
|
+
if (ciCounts.some((item) => item.before > 0 && item.after === 0 && item.reusableAfter === 0)) {
|
|
4798
|
+
const touched = new Set(changes.flatMap((change) => [change.oldPath, change.newPath]));
|
|
4799
|
+
const tree = /^[a-f0-9]{40,64}$/.test(refs.headSha) ? refs.headSha : refs.baseSha;
|
|
4800
|
+
try {
|
|
4801
|
+
for (const file of (await ciFilesAt(cwd, tree, options.signal)).filter((candidate) => !touched.has(candidate.path)).slice(0, 100)) {
|
|
4802
|
+
try {
|
|
4803
|
+
for (const command of gatingTestCommands(file.path, new TextDecoder().decode(await readBlob(cwd, file.oid, options.signal, 512 * 1024)))) elsewhere.add(command);
|
|
4804
|
+
} catch (error) {
|
|
4805
|
+
assertNotAborted(options.signal);
|
|
4806
|
+
}
|
|
4807
|
+
}
|
|
4808
|
+
} catch (error) {
|
|
4809
|
+
assertNotAborted(options.signal);
|
|
4810
|
+
}
|
|
4811
|
+
}
|
|
4812
|
+
result.infractions.push(...[...resolveTampering(tamperingFiles, productionChanged), ...guardFindings, ...resolveCiTestSteps(ciCounts, refs.headSha, elsewhere)].filter((finding2) => rules[finding2.type] !== "off"));
|
|
4524
4813
|
const exceptions = applyExceptions(result.infractions, options.ignoreExceptions ? [] : policy.config.exceptions, /* @__PURE__ */ new Date(), existingCounts);
|
|
4525
4814
|
result.infractions = exceptions.findings;
|
|
4526
4815
|
result.suppressedFindings = exceptions.suppressedFindings;
|
package/dist/cli.js
CHANGED
package/dist/index.js
CHANGED
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-roast",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.20.0",
|
|
4
4
|
"description": "Stops AI coding agents (and rushed humans) from turning a build green by weakening tests, coverage gates or CI. Hook for Claude Code, Codex and Cursor, pre-commit, GitHub Action. Offline.",
|
|
5
5
|
"mcpName": "io.github.Nari-Ab/agent-roast",
|
|
6
6
|
"type": "module",
|