agent-roast 0.8.0 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +75 -23
- package/dist/{chunk-C423TSZF.js → chunk-GMBQNL6Z.js} +544 -128
- package/dist/cli.js +2 -2
- package/dist/index.d.ts +3 -1
- package/dist/index.js +1 -1
- package/package.json +12 -8
- package/schema.json +33 -1
package/README.md
CHANGED
|
@@ -17,36 +17,75 @@
|
|
|
17
17
|
|
|
18
18
|
</div>
|
|
19
19
|
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
20
|
+
**Did the AI fix the code, or just make the checks pass?**
|
|
21
|
+
|
|
22
|
+
Coding agents (and tired humans) often get a red build green by changing the *checks*
|
|
23
|
+
instead of the *code*: deleting the failing assertion, weakening it to `toBeDefined()`,
|
|
24
|
+
removing the test, lowering the coverage threshold, switching off `strict`, or letting
|
|
25
|
+
CI steps fail. A linter reading the new file cannot see that, because nothing in the new
|
|
26
|
+
file looks wrong. `agent-roast review` compares the **before and after** of a change and
|
|
27
|
+
flags exactly these moves. It is offline, runs no project code, and needs only Git.
|
|
24
28
|
|
|
25
29
|
```bash
|
|
26
|
-
|
|
27
|
-
npx agent-roast
|
|
30
|
+
npx agent-roast review --base origin/main # a pull request
|
|
31
|
+
npx agent-roast --staged # what you are about to commit
|
|
32
|
+
```
|
|
28
33
|
|
|
29
|
-
|
|
34
|
+
| Rule | Fires when | Blocks by default |
|
|
35
|
+
| --- | --- | --- |
|
|
36
|
+
| `assertion-weakened` | a test keeps its name but loses assertions, or an exact check becomes `toBeDefined`/`toBeTruthy`/`assert x` | yes |
|
|
37
|
+
| `guard-weakened` | coverage threshold lowered, `strict`/`noImplicitAny` off, lint rule downgraded, CI step may fail or no longer runs tests, test script cannot fail | yes |
|
|
38
|
+
| `test-removed` | a test (or test file) disappears and no identical body appears elsewhere | no, reported |
|
|
39
|
+
| `expectation-rewritten` | an expected value changed, test inputs did not, and production code changed in the same PR | no, reported |
|
|
40
|
+
| `test-skip`, `test-focus`, `trivial-assert`, `swallowed-error`, `type-escape`, `stub-abandonment` | the classic shortcuts, compared before/after so old debt does not block | skip/focus/trivial: yes |
|
|
41
|
+
|
|
42
|
+
Renames, moved tests, parametrization, and formatter rewraps (prettier, black) are
|
|
43
|
+
recognized and not reported. Intentional cases can be accepted one by one with an
|
|
44
|
+
expiring, reviewed exception (see below). Policy is read from the base branch, so a PR
|
|
45
|
+
cannot switch the rules off for itself.
|
|
46
|
+
|
|
47
|
+
### How well does it work?
|
|
48
|
+
|
|
49
|
+
Measured, not promised (`npm run build`, then `npm run bench` and `npm run bench:noise`;
|
|
50
|
+
method and raw numbers are in [docs/benchmarks.md](docs/benchmarks.md)):
|
|
51
|
+
|
|
52
|
+
- **Labeled changes** (`benchmarks/cases.mjs`): 32 cheating edits across JS/TS, Python, Go
|
|
53
|
+
and CI/config files are caught, and 15 harmless look-alikes (reformat, rename, move,
|
|
54
|
+
parametrize, snapshot, raised thresholds, CI step split) stay silent. The cases were
|
|
55
|
+
written together with the detector, so they guard against regressions; they do not prove
|
|
56
|
+
accuracy. Three known gaps are listed in the file and still open: mocking the module
|
|
57
|
+
under test, hardcoding the tested value, and vacuous matchers such as `expect.anything()`.
|
|
58
|
+
- **Noise on real human history** (the last 300 commits of commander.js, execa, click,
|
|
59
|
+
requests and zod; 606 of them touched tests or CI/config): the two blocking rules fired on
|
|
60
|
+
7 commits (1.2%). Each was read by hand: one was a real defect (a refactor in `requests`
|
|
61
|
+
dropped the `assert` keyword, leaving a comparison with no effect); the other six were
|
|
62
|
+
deliberate rewrites a reviewer should still see, such as a test-framework migration,
|
|
63
|
+
dropping Python 2, or a documented `continue-on-error`. `test-removed` fired on 6.1% and
|
|
64
|
+
`expectation-rewritten` on 2.8% of those commits, which is why they only report.
|
|
65
|
+
- **Not measured yet:** accuracy on real AI-agent pull requests. The sample above is human
|
|
66
|
+
history, and it is small. Treat the numbers as a floor for noise, not a guarantee.
|
|
67
|
+
|
|
68
|
+
It is a **review aid**. It cannot prove who wrote a change or that code is correct, and it
|
|
69
|
+
reads text, not program semantics. A green result is not a quality certificate.
|
|
70
|
+
|
|
71
|
+
### Quick start
|
|
72
|
+
|
|
73
|
+
```bash
|
|
74
|
+
# Everything that is true of the code you have now
|
|
75
|
+
npx agent-roast check
|
|
30
76
|
npx agent-roast explain swallowed-error
|
|
31
77
|
|
|
32
|
-
# Adopt
|
|
78
|
+
# Adopt in an existing project without blocking on historical debt
|
|
33
79
|
npx agent-roast check --write-baseline .agentroast-baseline.json
|
|
34
|
-
# Review the findings and commit this policy file, then gate only new problems
|
|
35
80
|
npx agent-roast check --baseline .agentroast-baseline.json
|
|
36
81
|
|
|
37
|
-
#
|
|
38
|
-
npx agent-roast --staged
|
|
39
|
-
|
|
40
|
-
# Audit pull request diffs (CI gate)
|
|
41
|
-
npx agent-roast review --base origin/main --head HEAD --format github
|
|
42
|
-
|
|
43
|
-
# Optional historical audit
|
|
82
|
+
# Optional: score a repository's history
|
|
44
83
|
npx agent-roast --all --verbose
|
|
45
84
|
```
|
|
46
85
|
|
|
47
|
-
A finding includes its file and line, the
|
|
48
|
-
|
|
49
|
-
|
|
86
|
+
A finding includes its file and line, the evidence, and a concrete next step.
|
|
87
|
+
`--format json|markdown|sarif|github` are available; terminal output shows the first
|
|
88
|
+
20 findings.
|
|
50
89
|
|
|
51
90
|
### Context review for pull requests
|
|
52
91
|
|
|
@@ -106,12 +145,12 @@ regressions) may appear in the report/summary rather than as an inline PR annota
|
|
|
106
145
|
Use a separate SARIF category for introduced-findings review; it does not represent
|
|
107
146
|
a full scan of all outstanding alerts.
|
|
108
147
|
|
|
109
|
-
Install 0.
|
|
148
|
+
Install 0.9.0: `npx agent-roast@0.9.0 review --base origin/main`. For local development,
|
|
110
149
|
run `npm run build` and use `node dist/cli.js`.
|
|
111
150
|
|
|
112
151
|
### Precise exceptions (since 0.8.0)
|
|
113
152
|
|
|
114
|
-
Use `npx agent-roast@0.
|
|
153
|
+
Use `npx agent-roast@0.9.0 check --format json`, or build this checkout with
|
|
115
154
|
`npm run build` and run `node dist/cli.js`.
|
|
116
155
|
|
|
117
156
|
Some skipped integrations or abstract stubs are intentional. In 0.8.0 you can accept
|
|
@@ -278,6 +317,11 @@ The JSON schema version for context reviews is now `1.1.0`.
|
|
|
278
317
|
The generated hook uses the locally installed CLI (`--no-install`) and blocks on configured error rules instead of scoring
|
|
279
318
|
old commits. It preserves an existing pre-commit hook.
|
|
280
319
|
|
|
320
|
+
`init` resolves the repository root even when run from a package/subdirectory, so
|
|
321
|
+
its policy is used by `check` and its workflow can be discovered by GitHub. Outside
|
|
322
|
+
a Git repository, initialization uses the supplied directory. Its output shows the
|
|
323
|
+
actual destination and existing policy/hooks are preserved.
|
|
324
|
+
|
|
281
325
|
### Configure what blocks a change
|
|
282
326
|
|
|
283
327
|
Create `.agentroastrc.json` (or run `npx agent-roast init`):
|
|
@@ -342,7 +386,7 @@ jobs:
|
|
|
342
386
|
|
|
343
387
|
To pilot the Action without blocking on findings, set `advisory: true`. It passes
|
|
344
388
|
an explicit empty fail-on list; invalid configuration, Git errors and incomplete
|
|
345
|
-
scans still fail. The `v1` Action installs 0.
|
|
389
|
+
scans still fail. The `v1` Action installs 0.9.0. The CLI equivalent is `--fail-on ''`.
|
|
346
390
|
|
|
347
391
|
> **Security Note:** `agent-roast` runs in an isolated runner directory (`$RUNNER_TEMP`) with `--ignore-scripts` to neutralize malicious `.npmrc` files in untrusted pull requests. `pull_request_target` is not supported.
|
|
348
392
|
|
|
@@ -393,6 +437,11 @@ Rust raw strings/lifetimes and Java/Kotlin text blocks. Contiguous comments, Pyt
|
|
|
393
437
|
docstrings and multiline empty handlers are recognized. Template interpolation, complex language syntax and context outside
|
|
394
438
|
the added diff in legacy audit modes are not fully parsed. Custom test API aliases,
|
|
395
439
|
computed property calls and type/import resolution are not handled. Panic loops are an observational history heuristic.
|
|
440
|
+
Test registration options such as Node's `test('name', { skip: true }, fn)` and
|
|
441
|
+
`{ only: true }` are currently not inspected; use explicit `.skip`/`.only` APIs and
|
|
442
|
+
the runner's CI safeguards. Regexp masking is contextual, not a full JavaScript
|
|
443
|
+
parser. Formatting across contiguous lines is supported for casts, empty catches
|
|
444
|
+
and placeholder errors, but legacy added-line audits cannot recover missing hunks.
|
|
396
445
|
The discipline score summarizes added history; it does not certify current code quality.
|
|
397
446
|
`check` reports actionable findings without assigning a personality or quality score.
|
|
398
447
|
History attribution is based on explicit text/identity heuristics; it does not prove
|
|
@@ -416,13 +465,16 @@ are exercised on Node 18/20/22 using `AGENT_ROAST_TEST_NODE`.
|
|
|
416
465
|
npm ci
|
|
417
466
|
npm run typecheck
|
|
418
467
|
npm test
|
|
468
|
+
|
|
469
|
+
# Optional end-to-end CLI checks in disposable Git repositories
|
|
470
|
+
node scripts/verify-user-workflows.mjs dist/cli.js
|
|
419
471
|
```
|
|
420
472
|
|
|
421
473
|
`npm test` builds the distribution first so CLI integration and bundle checks run
|
|
422
474
|
against current code. Regression tests use disposable Git repositories. The programmatic
|
|
423
475
|
`auditPullRequest({ staged: true })` API retains the legacy added-lines contract;
|
|
424
476
|
use `reviewStagedChanges()` for the new context comparison and `ReviewResult`.
|
|
425
|
-
The composite Action installs exactly 0.
|
|
477
|
+
The composite Action installs exactly 0.9.0. For local development, use
|
|
426
478
|
`node dist/cli.js --staged` after building.
|
|
427
479
|
|
|
428
480
|
### Author & Support
|
|
@@ -576,6 +576,8 @@ var SourceMasker = class {
|
|
|
576
576
|
triple = false;
|
|
577
577
|
verbatim = false;
|
|
578
578
|
previousLine = 0;
|
|
579
|
+
tokenPrefix = "";
|
|
580
|
+
parentheses = [];
|
|
579
581
|
python;
|
|
580
582
|
javascript;
|
|
581
583
|
rust;
|
|
@@ -602,12 +604,14 @@ var SourceMasker = class {
|
|
|
602
604
|
this.rawEnd = "";
|
|
603
605
|
this.triple = false;
|
|
604
606
|
this.verbatim = false;
|
|
607
|
+
this.tokenPrefix = "";
|
|
608
|
+
this.parentheses.length = 0;
|
|
605
609
|
}
|
|
606
610
|
this.previousLine = number;
|
|
607
611
|
const code = Array(line.length).fill(" ");
|
|
608
612
|
const comments = Array(line.length).fill(" ");
|
|
609
613
|
let i = 0;
|
|
610
|
-
let prefix =
|
|
614
|
+
let prefix = this.tokenPrefix;
|
|
611
615
|
const appendPrefix = (text) => {
|
|
612
616
|
for (const char of text) {
|
|
613
617
|
if (/\s/.test(char)) {
|
|
@@ -706,7 +710,7 @@ var SourceMasker = class {
|
|
|
706
710
|
appendPrefix(" literal ");
|
|
707
711
|
continue;
|
|
708
712
|
}
|
|
709
|
-
if (this.javascript && char === "/" && (/(
|
|
713
|
+
if (this.javascript && char === "/" && (/(?:=>|[=(:,[!&|?{;]|\breturn|\byield|\bthrow)\s*$/.test(prefix) || !prefix.trim())) {
|
|
710
714
|
let j = i + 1;
|
|
711
715
|
let inClass = false;
|
|
712
716
|
for (; j < line.length; j++) {
|
|
@@ -727,10 +731,19 @@ var SourceMasker = class {
|
|
|
727
731
|
}
|
|
728
732
|
}
|
|
729
733
|
code[i] = char;
|
|
734
|
+
if (this.javascript && char === "(") {
|
|
735
|
+
const control = /(?<![\w$])(?:if|while|for|with|switch|catch)\s*$/.exec(prefix);
|
|
736
|
+
this.parentheses.push(Boolean(control && !/\.\s*$/.test(prefix.slice(0, control.index))));
|
|
737
|
+
} else if (this.javascript && char === ")" && this.parentheses.pop()) {
|
|
738
|
+
prefix = "; ";
|
|
739
|
+
i++;
|
|
740
|
+
continue;
|
|
741
|
+
}
|
|
730
742
|
appendPrefix(char);
|
|
731
743
|
i++;
|
|
732
744
|
}
|
|
733
745
|
if (this.quote !== "`" && !this.triple && !this.verbatim && !continued) this.quote = "";
|
|
746
|
+
this.tokenPrefix = this.javascript ? (prefix + " ").slice(-64) : "";
|
|
734
747
|
return { code: code.join(""), comments: comments.join(""), templateStarts, blockStarts };
|
|
735
748
|
}
|
|
736
749
|
};
|
|
@@ -773,6 +786,30 @@ var RULE_GUIDES = {
|
|
|
773
786
|
suggestion: "Implement the path or make its unsupported state explicit to callers. Check whether this is an intentional abstract contract.",
|
|
774
787
|
example: "throw new Error('Not implemented') \u2192 implement the operation and add a behavior test"
|
|
775
788
|
},
|
|
789
|
+
"assertion-weakened": {
|
|
790
|
+
title: "Test checks less than before",
|
|
791
|
+
why: "A test that keeps its name but loses assertions, or has one replaced by a weaker check, still passes while verifying less behavior.",
|
|
792
|
+
suggestion: "Restore the removed assertion or explain in the PR why that behavior is no longer required. Fix the code under test instead of weakening its check.",
|
|
793
|
+
example: "expect(total).toBe(42) \u2192 expect(total).toBeDefined() (restore the exact check)"
|
|
794
|
+
},
|
|
795
|
+
"test-removed": {
|
|
796
|
+
title: "Test removed",
|
|
797
|
+
why: "Deleting a failing test makes a suite green without fixing the behavior it covered. Removals are legitimate when the feature itself is gone.",
|
|
798
|
+
suggestion: "Confirm the behavior was intentionally removed. If it still exists, restore the test and fix the code.",
|
|
799
|
+
example: "it('rejects expired tokens', ...) deleted while token validation remains in src/"
|
|
800
|
+
},
|
|
801
|
+
"expectation-rewritten": {
|
|
802
|
+
title: "Expected value edited alongside production code",
|
|
803
|
+
why: "Changing an expected value while the test inputs stay the same can mean the test was adjusted to match new output instead of the output being corrected.",
|
|
804
|
+
suggestion: "Verify the new expectation against the requirement, not against the program's current output. Intentional behavior changes should say so in the PR.",
|
|
805
|
+
example: "expect(price(cart)).toBe(100) \u2192 toBe(90) with price() changed in the same PR"
|
|
806
|
+
},
|
|
807
|
+
"guard-weakened": {
|
|
808
|
+
title: "Quality gate made easier to pass",
|
|
809
|
+
why: "Lower coverage thresholds, disabled compiler strictness, downgraded lint rules, tolerated CI failures or an always-passing test script let defects through without touching the code.",
|
|
810
|
+
suggestion: "Keep the gate and fix the underlying failure. If the change is deliberate, record the reason and a plan to restore it.",
|
|
811
|
+
example: 'coverageThreshold lines: 80 \u2192 60, "strict": true \u2192 false, continue-on-error: true'
|
|
812
|
+
},
|
|
776
813
|
"panic-loop": {
|
|
777
814
|
title: "Rapid overlapping fix commits",
|
|
778
815
|
why: "Repeated short fix cycles can suggest a missing reproduction or feedback loop; timing alone does not prove poor work.",
|
|
@@ -793,7 +830,11 @@ var VALID_DETECTORS = [
|
|
|
793
830
|
"panic-loop",
|
|
794
831
|
"trivial-assert",
|
|
795
832
|
"stub-abandonment",
|
|
796
|
-
"test-focus"
|
|
833
|
+
"test-focus",
|
|
834
|
+
"assertion-weakened",
|
|
835
|
+
"test-removed",
|
|
836
|
+
"expectation-rewritten",
|
|
837
|
+
"guard-weakened"
|
|
797
838
|
];
|
|
798
839
|
|
|
799
840
|
// src/file-read.ts
|
|
@@ -826,13 +867,13 @@ function readRegularFile(filename, limit) {
|
|
|
826
867
|
import { createHash, randomUUID as randomUUID2 } from "crypto";
|
|
827
868
|
import * as fs2 from "fs";
|
|
828
869
|
import path from "path";
|
|
829
|
-
function findingFingerprint(
|
|
830
|
-
return createHash("sha256").update(JSON.stringify([
|
|
870
|
+
function findingFingerprint(finding2) {
|
|
871
|
+
return createHash("sha256").update(JSON.stringify([finding2.type, finding2.file, finding2.lineSnippet?.trim()])).digest("hex");
|
|
831
872
|
}
|
|
832
873
|
function createBaseline(findings) {
|
|
833
874
|
const counts = /* @__PURE__ */ new Map();
|
|
834
|
-
for (const
|
|
835
|
-
const fingerprint = findingFingerprint(
|
|
875
|
+
for (const finding2 of findings) {
|
|
876
|
+
const fingerprint = findingFingerprint(finding2);
|
|
836
877
|
counts.set(fingerprint, (counts.get(fingerprint) || 0) + 1);
|
|
837
878
|
}
|
|
838
879
|
return {
|
|
@@ -864,14 +905,14 @@ function applyBaseline(findings, baseline) {
|
|
|
864
905
|
const remaining = new Map(baseline.entries.map((entry) => [entry.fingerprint, entry.count]));
|
|
865
906
|
const fresh = [];
|
|
866
907
|
let baselineCount = 0;
|
|
867
|
-
for (const
|
|
868
|
-
const fingerprint = findingFingerprint(
|
|
908
|
+
for (const finding2 of findings) {
|
|
909
|
+
const fingerprint = findingFingerprint(finding2);
|
|
869
910
|
const count = remaining.get(fingerprint) || 0;
|
|
870
911
|
if (count > 0) {
|
|
871
912
|
remaining.set(fingerprint, count - 1);
|
|
872
913
|
baselineCount++;
|
|
873
914
|
} else {
|
|
874
|
-
fresh.push(
|
|
915
|
+
fresh.push(finding2);
|
|
875
916
|
}
|
|
876
917
|
}
|
|
877
918
|
return { findings: fresh, baselineCount, staleBaselineCount: [...remaining.values()].reduce((a, b) => a + b, 0) };
|
|
@@ -920,11 +961,11 @@ function applyExceptions(findings, exceptions = [], now = /* @__PURE__ */ new Da
|
|
|
920
961
|
const occurrences = new Map(priorOccurrences);
|
|
921
962
|
const actionable = [];
|
|
922
963
|
const suppressedFindings = [];
|
|
923
|
-
for (const
|
|
924
|
-
const fingerprint = findingFingerprint(
|
|
964
|
+
for (const finding2 of findings) {
|
|
965
|
+
const fingerprint = findingFingerprint(finding2);
|
|
925
966
|
const occurrence = (occurrences.get(fingerprint) ?? 0) + 1;
|
|
926
967
|
occurrences.set(fingerprint, occurrence);
|
|
927
|
-
const enriched = { ...
|
|
968
|
+
const enriched = { ...finding2, fingerprint, occurrence };
|
|
928
969
|
const allowance = active.get(fingerprint);
|
|
929
970
|
if (allowance && allowance.remaining > 0) {
|
|
930
971
|
allowance.remaining--;
|
|
@@ -937,7 +978,7 @@ function applyExceptions(findings, exceptions = [], now = /* @__PURE__ */ new Da
|
|
|
937
978
|
// src/config.ts
|
|
938
979
|
var MAX_CONFIG_BYTES = 64 * 1024;
|
|
939
980
|
var DEFAULT_CONFIG = {
|
|
940
|
-
failOn: ["test-skip", "trivial-assert", "test-focus"],
|
|
981
|
+
failOn: ["test-skip", "trivial-assert", "test-focus", "assertion-weakened", "guard-weakened"],
|
|
941
982
|
ignorePaths: [
|
|
942
983
|
"**/fixtures/**",
|
|
943
984
|
"**/__fixtures__/**",
|
|
@@ -953,6 +994,10 @@ var DEFAULT_CONFIG = {
|
|
|
953
994
|
"test-skip": "error",
|
|
954
995
|
"trivial-assert": "error",
|
|
955
996
|
"test-focus": "error",
|
|
997
|
+
"assertion-weakened": "error",
|
|
998
|
+
"guard-weakened": "error",
|
|
999
|
+
"test-removed": "warn",
|
|
1000
|
+
"expectation-rewritten": "warn",
|
|
956
1001
|
"swallowed-error": "warn",
|
|
957
1002
|
"stub-abandonment": "warn",
|
|
958
1003
|
"type-escape": "warn"
|
|
@@ -1139,7 +1184,7 @@ var PYTHON_TRIVIAL_ASSERT_PATTERNS = [
|
|
|
1139
1184
|
{ regex: /\bself\s*\.\s*assertEqual\s*\(\s*(?:1\s*,\s*1|True\s*,\s*True)\s*\)/, name: "trivial assert (self.assertEqual(1, 1))" }
|
|
1140
1185
|
];
|
|
1141
1186
|
var JS_STUB_PATTERNS = [
|
|
1142
|
-
{ regex: /\bthrow\s+new\s+Error\s*\(\s*["'](?:TODO\b|Not implemented\b|to be implemented\b|stub\b|implement me\b)/i, name: "unimplemented stub (throw new Error('Not implemented'))" }
|
|
1187
|
+
{ regex: /\bthrow\s+new\s+Error\s*\(\s*["'`](?:TODO(?=\s*(?:["'`:]|implement\b))|Not implemented\b|to be implemented\b|stub\b|implement me\b)/i, name: "unimplemented stub (throw new Error('Not implemented'))" }
|
|
1143
1188
|
];
|
|
1144
1189
|
var PYTHON_STUB_PATTERNS = [
|
|
1145
1190
|
{ regex: /\braise\s+NotImplementedError\b/, name: "unimplemented stub (raise NotImplementedError)" },
|
|
@@ -1251,22 +1296,6 @@ function auditFileDiffs(files, commitHash = "HEAD", commitDate = (/* @__PURE__ *
|
|
|
1251
1296
|
infractions.push({ type: "type-escape", file: file.path, lineNumber: added.lineNumber, lineSnippet: rawLine.trim(), commitHash, commitDate, reason: "linter suppression (eslint-disable)" });
|
|
1252
1297
|
foundEscape = true;
|
|
1253
1298
|
}
|
|
1254
|
-
if (typescript && !foundEscape && !isComment) {
|
|
1255
|
-
for (const rule of CODE_TYPE_ESCAPE_PATTERNS) {
|
|
1256
|
-
if (rule.regex.test(cleanedLine)) {
|
|
1257
|
-
infractions.push({
|
|
1258
|
-
type: "type-escape",
|
|
1259
|
-
file: file.path,
|
|
1260
|
-
lineNumber: added.lineNumber,
|
|
1261
|
-
lineSnippet: rawLine.trim(),
|
|
1262
|
-
commitHash,
|
|
1263
|
-
commitDate,
|
|
1264
|
-
reason: rule.name
|
|
1265
|
-
});
|
|
1266
|
-
break;
|
|
1267
|
-
}
|
|
1268
|
-
}
|
|
1269
|
-
}
|
|
1270
1299
|
if (python && !isComment && isTestFile(file.path)) {
|
|
1271
1300
|
for (const rule of PYTHON_TRIVIAL_ASSERT_PATTERNS) {
|
|
1272
1301
|
if (rule.regex.test(cleanedLine)) {
|
|
@@ -1283,40 +1312,55 @@ function auditFileDiffs(files, commitHash = "HEAD", commitDate = (/* @__PURE__ *
|
|
|
1283
1312
|
}
|
|
1284
1313
|
}
|
|
1285
1314
|
}
|
|
1286
|
-
|
|
1287
|
-
|
|
1288
|
-
|
|
1289
|
-
|
|
1290
|
-
|
|
1291
|
-
|
|
1292
|
-
|
|
1293
|
-
|
|
1294
|
-
|
|
1295
|
-
|
|
1296
|
-
|
|
1297
|
-
|
|
1298
|
-
|
|
1299
|
-
|
|
1300
|
-
|
|
1301
|
-
|
|
1315
|
+
}
|
|
1316
|
+
let source = "";
|
|
1317
|
+
let code = "";
|
|
1318
|
+
let raw = "";
|
|
1319
|
+
const offsets = [];
|
|
1320
|
+
for (let i = 0; i < maskedLines.length; i++) {
|
|
1321
|
+
const separator = i ? maskedLines[i].number === maskedLines[i - 1].number + 1 ? "\n" : "\0" : "";
|
|
1322
|
+
source += separator;
|
|
1323
|
+
code += separator;
|
|
1324
|
+
raw += separator;
|
|
1325
|
+
offsets.push(source.length);
|
|
1326
|
+
source += maskedLines[i].testCode;
|
|
1327
|
+
code += maskedLines[i].code;
|
|
1328
|
+
raw += maskedLines[i].raw;
|
|
1329
|
+
}
|
|
1330
|
+
const evidence = (index, length) => {
|
|
1331
|
+
let low = 0;
|
|
1332
|
+
let high = offsets.length;
|
|
1333
|
+
while (low + 1 < high) {
|
|
1334
|
+
const middle = low + high >>> 1;
|
|
1335
|
+
if (offsets[middle] <= index) low = middle;
|
|
1336
|
+
else high = middle;
|
|
1337
|
+
}
|
|
1338
|
+
let end = low;
|
|
1339
|
+
while (end + 1 < offsets.length && offsets[end + 1] < index + length) end++;
|
|
1340
|
+
return { lineNumber: maskedLines[low].number, lineSnippet: maskedLines.slice(low, end + 1).map((line) => line.raw.trim()).join(" ") };
|
|
1341
|
+
};
|
|
1342
|
+
const reportedStubs = /* @__PURE__ */ new Set();
|
|
1343
|
+
for (const rule of stubs) {
|
|
1344
|
+
for (const match of raw.matchAll(new RegExp(rule.regex.source, rule.regex.flags + "g"))) {
|
|
1345
|
+
if (code[match.index] !== raw[match.index]) continue;
|
|
1346
|
+
const location = evidence(match.index, match[0].length);
|
|
1347
|
+
if (reportedStubs.has(location.lineNumber)) continue;
|
|
1348
|
+
reportedStubs.add(location.lineNumber);
|
|
1349
|
+
infractions.push({ type: "stub-abandonment", file: file.path, ...location, commitHash, commitDate, reason: rule.name });
|
|
1350
|
+
}
|
|
1351
|
+
}
|
|
1352
|
+
if (typescript) {
|
|
1353
|
+
const reported = new Set(infractions.slice(firstFinding).filter((f) => f.type === "type-escape").map((f) => f.lineNumber));
|
|
1354
|
+
for (const rule of CODE_TYPE_ESCAPE_PATTERNS) {
|
|
1355
|
+
for (const match of code.matchAll(new RegExp(rule.regex.source, "g"))) {
|
|
1356
|
+
const location = evidence(match.index, match[0].length);
|
|
1357
|
+
if (reported.has(location.lineNumber)) continue;
|
|
1358
|
+
reported.add(location.lineNumber);
|
|
1359
|
+
infractions.push({ type: "type-escape", file: file.path, ...location, commitHash, commitDate, reason: rule.name });
|
|
1302
1360
|
}
|
|
1303
1361
|
}
|
|
1304
1362
|
}
|
|
1305
1363
|
if (javascript) {
|
|
1306
|
-
let source = "";
|
|
1307
|
-
const offsets = [];
|
|
1308
|
-
for (let i = 0; i < maskedLines.length; i++) {
|
|
1309
|
-
if (i) source += maskedLines[i].number === maskedLines[i - 1].number + 1 ? "\n" : "\0";
|
|
1310
|
-
offsets.push(source.length);
|
|
1311
|
-
source += maskedLines[i].testCode;
|
|
1312
|
-
}
|
|
1313
|
-
let row = 0;
|
|
1314
|
-
const evidence = (index, length) => {
|
|
1315
|
-
while (row + 1 < offsets.length && offsets[row + 1] <= index) row++;
|
|
1316
|
-
let end = row;
|
|
1317
|
-
while (end + 1 < offsets.length && offsets[end + 1] < index + length) end++;
|
|
1318
|
-
return { lineNumber: maskedLines[row].number, lineSnippet: maskedLines.slice(row, end + 1).map((line) => line.raw.trim()).join(" ") };
|
|
1319
|
-
};
|
|
1320
1364
|
for (const match of source.matchAll(JS_TEST_API)) {
|
|
1321
1365
|
let previous = match.index - 1;
|
|
1322
1366
|
while (previous >= 0 && /\s/.test(source[previous])) previous--;
|
|
@@ -1335,34 +1379,35 @@ function auditFileDiffs(files, commitHash = "HEAD", commitDate = (/* @__PURE__ *
|
|
|
1335
1379
|
});
|
|
1336
1380
|
}
|
|
1337
1381
|
if (isTestFile(file.path)) {
|
|
1338
|
-
row = 0;
|
|
1339
1382
|
for (const rule of JS_TRIVIAL_ASSERT_PATTERNS) {
|
|
1340
1383
|
for (const match of source.matchAll(new RegExp(rule.regex.source, "g"))) {
|
|
1341
1384
|
infractions.push({ type: "trivial-assert", file: file.path, ...evidence(match.index, match[0].length), commitHash, commitDate, reason: rule.name });
|
|
1342
1385
|
}
|
|
1343
|
-
row = 0;
|
|
1344
1386
|
}
|
|
1345
1387
|
}
|
|
1346
1388
|
}
|
|
1389
|
+
if (braceCatches) {
|
|
1390
|
+
for (const match of code.matchAll(/(?<![\w$.])catch\s*(?:\(\s*[^()\s\0][^()\0]*\))?\s*\{\s*\}/g)) {
|
|
1391
|
+
infractions.push({
|
|
1392
|
+
type: "swallowed-error",
|
|
1393
|
+
file: file.path,
|
|
1394
|
+
...evidence(match.index, match[0].length),
|
|
1395
|
+
commitHash,
|
|
1396
|
+
commitDate,
|
|
1397
|
+
reason: match[0].includes("\n") ? "empty multi-line catch block" : "empty catch block (catch {})"
|
|
1398
|
+
});
|
|
1399
|
+
}
|
|
1400
|
+
}
|
|
1347
1401
|
for (let index = 0; index < maskedLines.length; index++) {
|
|
1348
1402
|
const start = maskedLines[index];
|
|
1349
|
-
const catches = braceCatches ? [...start.code.matchAll(/(?<![\w$.])catch\s*(?:\(\s*[^)\s][^)]*\))?\s*\{/g)] : [];
|
|
1350
|
-
for (const match of catches) {
|
|
1351
|
-
if (/^\s*}/.test(start.code.slice(match.index + match[0].length))) {
|
|
1352
|
-
infractions.push({ type: "swallowed-error", file: file.path, lineNumber: start.number, lineSnippet: start.raw.trim(), commitHash, commitDate, reason: "empty catch block (catch {})" });
|
|
1353
|
-
}
|
|
1354
|
-
}
|
|
1355
|
-
const lastCatch = catches.at(-1);
|
|
1356
|
-
const isCatch = lastCatch && /^\s*$/.test(start.code.slice(lastCatch.index + lastCatch[0].length));
|
|
1357
1403
|
const inlineExcept = python && /^\s*except(?:\*?\s+[^:]+)?:\s*pass\s*;?\s*$/.test(start.code);
|
|
1358
1404
|
if (inlineExcept) infractions.push({ type: "swallowed-error", file: file.path, lineNumber: start.number, lineSnippet: start.raw.trim(), commitHash, commitDate, reason: "silent python error swallow (except: pass)" });
|
|
1359
1405
|
const isExcept = python && /^\s*except(?:\*?\s+[^:]+)?:\s*$/.test(start.code);
|
|
1360
|
-
if (!
|
|
1406
|
+
if (!isExcept) continue;
|
|
1361
1407
|
let next = index + 1;
|
|
1362
1408
|
while (next < maskedLines.length && maskedLines[next].number === maskedLines[next - 1].number + 1 && !maskedLines[next].code.trim()) next++;
|
|
1363
1409
|
const end = maskedLines[next];
|
|
1364
1410
|
if (!end || end.number !== maskedLines[next - 1].number + 1) continue;
|
|
1365
|
-
const closesCatch = isCatch && /^\s*}/.test(end.code);
|
|
1366
1411
|
let closesExcept = isExcept && /^\s*pass\s*;?\s*$/.test(end.code);
|
|
1367
1412
|
if (closesExcept) {
|
|
1368
1413
|
const indent = start.raw.search(/\S/);
|
|
@@ -1371,18 +1416,18 @@ function auditFileDiffs(files, commitHash = "HEAD", commitDate = (/* @__PURE__ *
|
|
|
1371
1416
|
while (after < maskedLines.length && maskedLines[after].number === maskedLines[after - 1].number + 1 && !maskedLines[after].code.trim()) after++;
|
|
1372
1417
|
if (after < maskedLines.length && maskedLines[after].number === maskedLines[after - 1].number + 1 && maskedLines[after].raw.search(/\S/) > indent) closesExcept = false;
|
|
1373
1418
|
}
|
|
1374
|
-
if (
|
|
1419
|
+
if (closesExcept) infractions.push({
|
|
1375
1420
|
type: "swallowed-error",
|
|
1376
1421
|
file: file.path,
|
|
1377
1422
|
lineNumber: start.number,
|
|
1378
1423
|
lineSnippet: maskedLines.slice(index, next + 1).map((line) => line.raw.trim()).join(" "),
|
|
1379
1424
|
commitHash,
|
|
1380
1425
|
commitDate,
|
|
1381
|
-
reason:
|
|
1426
|
+
reason: "silent multi-line python error swallow"
|
|
1382
1427
|
});
|
|
1383
1428
|
}
|
|
1384
1429
|
const ordered = infractions.splice(firstFinding).sort((left, right) => (left.lineNumber || 0) - (right.lineNumber || 0));
|
|
1385
|
-
for (const
|
|
1430
|
+
for (const finding2 of ordered) infractions.push(finding2);
|
|
1386
1431
|
}
|
|
1387
1432
|
return { infractions: infractions.filter((inf) => options.rules?.[inf.type] !== "off").map((inf) => ({ ...inf, suggestion: RULE_GUIDES[inf.type].suggestion })), hasSupportedChanges, filesInspected, linesAdded };
|
|
1388
1433
|
}
|
|
@@ -1517,7 +1562,7 @@ var MetricsCollector = class {
|
|
|
1517
1562
|
const counts = {};
|
|
1518
1563
|
for (const infraction of audit.infractions) {
|
|
1519
1564
|
const count = counts[infraction.type] || 0;
|
|
1520
|
-
if (count < MAX_INFRACTIONS_PER_COMMIT) buckets[infraction.type]
|
|
1565
|
+
if (count < MAX_INFRACTIONS_PER_COMMIT) buckets[infraction.type]?.push(infraction);
|
|
1521
1566
|
counts[infraction.type] = count + 1;
|
|
1522
1567
|
}
|
|
1523
1568
|
}
|
|
@@ -1963,8 +2008,8 @@ function exceptionTerminal(result) {
|
|
|
1963
2008
|
const accepted = result.suppressedFindings || [];
|
|
1964
2009
|
if (accepted.length) {
|
|
1965
2010
|
lines.push(` ${accepted.length} findings accepted by explicit exceptions (not fixed)`);
|
|
1966
|
-
for (const
|
|
1967
|
-
` ACCEPTED [${
|
|
2011
|
+
for (const finding2 of accepted.slice(0, 20)) lines.push(
|
|
2012
|
+
` ACCEPTED [${finding2.type}] ${terminalText(finding2.file || "unknown")}:${finding2.lineNumber} \u2014 ${terminalText(finding2.suppression.reason)} (expires ${terminalText(finding2.suppression.expires)})`
|
|
1968
2013
|
);
|
|
1969
2014
|
if (accepted.length > 20) lines.push(` ${accepted.length - 20} more accepted findings; use --format json.`);
|
|
1970
2015
|
}
|
|
@@ -1984,7 +2029,7 @@ function exceptionMarkdown(result) {
|
|
|
1984
2029
|
"| Rule | Location | Reason | Expires |",
|
|
1985
2030
|
"| --- | --- | --- | --- |"
|
|
1986
2031
|
);
|
|
1987
|
-
for (const
|
|
2032
|
+
for (const finding2 of accepted) lines.push(`| ${finding2.type} | ${markdownCell(`${finding2.file}:${finding2.lineNumber}`)} | ${markdownCell(finding2.suppression.reason)} | ${markdownCell(finding2.suppression.expires)} |`);
|
|
1988
2033
|
}
|
|
1989
2034
|
if (result.expiredExceptionCount) lines.push("", `${result.expiredExceptionCount} expired exceptions were not applied; review their findings.`);
|
|
1990
2035
|
return lines;
|
|
@@ -2041,12 +2086,12 @@ function formatReviewTerminal(result) {
|
|
|
2041
2086
|
lines.push(...exceptionTerminal(result));
|
|
2042
2087
|
if (result.policyChanges.length) lines.push(" Candidate policy changes were ignored; adopt them separately after review.");
|
|
2043
2088
|
if (result.excludedFiles || result.unsupportedFiles) lines.push(` ${result.excludedFiles} excluded \xB7 ${result.unsupportedFiles} unsupported/non-source changes`);
|
|
2044
|
-
for (const
|
|
2089
|
+
for (const finding2 of result.infractions.slice(0, 20)) {
|
|
2045
2090
|
lines.push(
|
|
2046
2091
|
"",
|
|
2047
|
-
` ${result.failedRules.includes(
|
|
2048
|
-
` ${terminalText(
|
|
2049
|
-
` Next: ${terminalText(
|
|
2092
|
+
` ${result.failedRules.includes(finding2.type) ? "FAIL" : "WARN"} [${finding2.type}] ${terminalText(finding2.file || "unknown")}:${finding2.lineNumber}`,
|
|
2093
|
+
` ${terminalText(finding2.lineSnippet || finding2.reason)}`,
|
|
2094
|
+
` Next: ${terminalText(finding2.suggestion || "Review this occurrence.")}`
|
|
2050
2095
|
);
|
|
2051
2096
|
}
|
|
2052
2097
|
if (result.infractions.length > 20) lines.push(` ${result.infractions.length - 20} more new findings; use --format json for all results.`);
|
|
@@ -2067,7 +2112,7 @@ function formatReviewMarkdown(result) {
|
|
|
2067
2112
|
];
|
|
2068
2113
|
if (result.policyChanges.length) lines.push("Candidate policy changes were ignored. This review uses the supplied base commit's policy.", "");
|
|
2069
2114
|
lines.push("| New finding | Head location | Evidence | Next step |", "| --- | --- | --- | --- |");
|
|
2070
|
-
for (const
|
|
2115
|
+
for (const finding2 of result.infractions) lines.push(`| ${finding2.type} | ${markdownCell(`${finding2.file}:${finding2.lineNumber}`)} | ${markdownCell(finding2.lineSnippet || finding2.reason)} | ${markdownCell(finding2.suggestion || "Review this occurrence.")} |`);
|
|
2071
2116
|
lines.push(...exceptionMarkdown(result));
|
|
2072
2117
|
if (result.removedFindings.length) {
|
|
2073
2118
|
lines.push(
|
|
@@ -2079,7 +2124,7 @@ function formatReviewMarkdown(result) {
|
|
|
2079
2124
|
"| Rule | Previous location | Previous evidence |",
|
|
2080
2125
|
"| --- | --- | --- |"
|
|
2081
2126
|
);
|
|
2082
|
-
for (const
|
|
2127
|
+
for (const finding2 of result.removedFindings) lines.push(`| ${finding2.type} | ${markdownCell(`${finding2.file}:${finding2.lineNumber}`)} | ${markdownCell(finding2.lineSnippet || finding2.reason)} |`);
|
|
2083
2128
|
}
|
|
2084
2129
|
for (const skipped of result.skippedFiles) lines.push("", `Not compared: ${markdownCell(skipped.file)} \u2014 ${markdownCell(skipped.reason)}`);
|
|
2085
2130
|
lines.push("", `Scope: changed supported files only; ${result.excludedFiles} excluded and ${result.unsupportedFiles} unsupported/non-source changes.`);
|
|
@@ -2184,11 +2229,24 @@ var hasEntry = (filename) => {
|
|
|
2184
2229
|
}
|
|
2185
2230
|
};
|
|
2186
2231
|
async function initProject(options = {}) {
|
|
2187
|
-
|
|
2232
|
+
let cwd = fs4.realpathSync(path3.resolve(options.cwd || process.cwd()));
|
|
2233
|
+
const root = spawnSync2("git", [...HERMETIC_GIT_FLAGS, "rev-parse", "--show-toplevel"], {
|
|
2234
|
+
cwd,
|
|
2235
|
+
env: { ...getHermeticGitEnv(), LC_ALL: "C" },
|
|
2236
|
+
encoding: "utf8",
|
|
2237
|
+
timeout: 3e4,
|
|
2238
|
+
maxBuffer: 8192,
|
|
2239
|
+
windowsHide: true
|
|
2240
|
+
});
|
|
2241
|
+
if (root.error) throw new ConfigError("Cannot resolve the repository root for initialization.");
|
|
2242
|
+
if (root.status === 0) cwd = fs4.realpathSync(gitOutputPath(root.stdout));
|
|
2243
|
+
else if (root.status !== 128 || !root.stderr.includes("not a git repository")) {
|
|
2244
|
+
throw new ConfigError("Cannot resolve a working repository root for initialization.");
|
|
2245
|
+
}
|
|
2188
2246
|
const created = [];
|
|
2189
2247
|
const skipped = [];
|
|
2190
|
-
const publish = (
|
|
2191
|
-
(createNewFile(
|
|
2248
|
+
const publish = (root2, relative2, label, content, mode) => {
|
|
2249
|
+
(createNewFile(root2, relative2, content, mode) ? created : skipped).push(label);
|
|
2192
2250
|
};
|
|
2193
2251
|
if ([".agentroastrc.json", "agent-roast.config.json", ".agentroastrc"].some((name) => hasEntry(path3.join(cwd, name)))) {
|
|
2194
2252
|
skipped.push(".agentroastrc.json");
|
|
@@ -2220,7 +2278,7 @@ async function initProject(options = {}) {
|
|
|
2220
2278
|
} else publish(commonDir, relative2, ".git/hooks/pre-commit", PRE_COMMIT_HOOK_TEMPLATE, 493);
|
|
2221
2279
|
}
|
|
2222
2280
|
}
|
|
2223
|
-
return { created, skipped };
|
|
2281
|
+
return { directory: cwd, created, skipped };
|
|
2224
2282
|
}
|
|
2225
2283
|
|
|
2226
2284
|
// src/badge.ts
|
|
@@ -2312,26 +2370,26 @@ var version = require2("../package.json").version;
|
|
|
2312
2370
|
function formatSarif(report) {
|
|
2313
2371
|
const occurrences = /* @__PURE__ */ new Map();
|
|
2314
2372
|
const allFindings = [
|
|
2315
|
-
...report.infractions.map((
|
|
2316
|
-
...(report.suppressedFindings ?? []).map((
|
|
2373
|
+
...report.infractions.map((finding2) => ({ finding: finding2, suppression: void 0 })),
|
|
2374
|
+
...(report.suppressedFindings ?? []).map((finding2) => ({ finding: finding2, suppression: finding2.suppression }))
|
|
2317
2375
|
];
|
|
2318
2376
|
allFindings.sort((a, b) => (a.finding.file ?? "").localeCompare(b.finding.file ?? "") || (a.finding.lineNumber ?? 1) - (b.finding.lineNumber ?? 1) || a.finding.type.localeCompare(b.finding.type));
|
|
2319
|
-
const results = allFindings.map(({ finding, suppression }) => {
|
|
2320
|
-
const filename =
|
|
2377
|
+
const results = allFindings.map(({ finding: finding2, suppression }) => {
|
|
2378
|
+
const filename = finding2.file;
|
|
2321
2379
|
if (!filename || filename.startsWith("/") || filename.split("/").includes("..") || /^[a-z]:/i.test(filename)) {
|
|
2322
2380
|
throw new ConfigError("SARIF needs repository-relative finding paths.");
|
|
2323
2381
|
}
|
|
2324
|
-
const fingerprint = findingFingerprint(
|
|
2325
|
-
const occurrence =
|
|
2382
|
+
const fingerprint = findingFingerprint(finding2);
|
|
2383
|
+
const occurrence = finding2.occurrence ?? (occurrences.get(fingerprint) || 0) + 1;
|
|
2326
2384
|
occurrences.set(fingerprint, occurrence);
|
|
2327
2385
|
return {
|
|
2328
|
-
ruleId:
|
|
2329
|
-
ruleIndex: VALID_DETECTORS.indexOf(
|
|
2330
|
-
level: report.failedRules.includes(
|
|
2331
|
-
message: { text: `${
|
|
2386
|
+
ruleId: finding2.type,
|
|
2387
|
+
ruleIndex: VALID_DETECTORS.indexOf(finding2.type),
|
|
2388
|
+
level: report.failedRules.includes(finding2.type) ? "error" : "warning",
|
|
2389
|
+
message: { text: `${finding2.reason}. ${finding2.suggestion || RULE_GUIDES[finding2.type].suggestion}` },
|
|
2332
2390
|
locations: [{ physicalLocation: {
|
|
2333
2391
|
artifactLocation: { uri: filename.split("/").map((segment) => encodeURIComponent(segment)).join("/") },
|
|
2334
|
-
region: { startLine: Math.max(1,
|
|
2392
|
+
region: { startLine: Math.max(1, finding2.lineNumber || 1) }
|
|
2335
2393
|
} }],
|
|
2336
2394
|
partialFingerprints: { "agentRoastFinding/v1": `${fingerprint}:${occurrence}` },
|
|
2337
2395
|
...suppression ? { suppressions: [{
|
|
@@ -2636,7 +2694,7 @@ async function checkProject(options = {}) {
|
|
|
2636
2694
|
const cwd = path5.resolve(options.cwd || process.cwd());
|
|
2637
2695
|
const root = await fs6.realpath(gitOutputPath(await git(cwd, ["rev-parse", "--show-toplevel"], options.signal)));
|
|
2638
2696
|
const config = loadConfig(root);
|
|
2639
|
-
const failOn = options.failOn ?? config.failOn ?? ["test-skip", "trivial-assert", "test-focus"];
|
|
2697
|
+
const failOn = options.failOn ?? config.failOn ?? ["test-skip", "trivial-assert", "test-focus", "assertion-weakened", "guard-weakened"];
|
|
2640
2698
|
for (const rule of failOn) if (!VALID_DETECTORS.includes(rule)) throw new ConfigError(`Unknown rule '${rule}'.`);
|
|
2641
2699
|
const format = options.format || "terminal";
|
|
2642
2700
|
if (!["terminal", "json", "github", "markdown", "sarif"].includes(format)) throw new ConfigError("Unsupported check output format.");
|
|
@@ -2673,14 +2731,14 @@ async function checkProject(options = {}) {
|
|
|
2673
2731
|
const applied = baseline ? applyBaseline(infractions, baseline) : { findings: infractions, baselineCount: 0, staleBaselineCount: 0 };
|
|
2674
2732
|
const notInBaseline = new Set(applied.findings);
|
|
2675
2733
|
const acceptedCounts = /* @__PURE__ */ new Map();
|
|
2676
|
-
for (const
|
|
2677
|
-
if (notInBaseline.has(
|
|
2678
|
-
const fingerprint = findingFingerprint(
|
|
2734
|
+
for (const finding2 of infractions) {
|
|
2735
|
+
if (notInBaseline.has(finding2)) continue;
|
|
2736
|
+
const fingerprint = findingFingerprint(finding2);
|
|
2679
2737
|
acceptedCounts.set(fingerprint, (acceptedCounts.get(fingerprint) ?? 0) + 1);
|
|
2680
2738
|
}
|
|
2681
2739
|
const exceptions = applyExceptions(applied.findings, options.ignoreExceptions ? [] : config.exceptions, /* @__PURE__ */ new Date(), acceptedCounts);
|
|
2682
2740
|
const findings = exceptions.findings;
|
|
2683
|
-
const failedRules = [...new Set(findings.filter((
|
|
2741
|
+
const failedRules = [...new Set(findings.filter((finding2) => failOn.includes(finding2.type)).map((finding2) => finding2.type))];
|
|
2684
2742
|
const complete = skippedFiles.length === 0;
|
|
2685
2743
|
const passed = complete && failedRules.length === 0;
|
|
2686
2744
|
const result = {
|
|
@@ -2712,24 +2770,355 @@ async function checkProject(options = {}) {
|
|
|
2712
2770
|
|
|
2713
2771
|
// src/review.ts
|
|
2714
2772
|
import path6 from "path";
|
|
2773
|
+
|
|
2774
|
+
// src/tampering.ts
|
|
2775
|
+
import { createHash as createHash3 } from "crypto";
|
|
2776
|
+
var EMPTY = { immediate: [], removedTests: [], addedHashes: [], rewritten: [] };
|
|
2777
|
+
function languageOf(file) {
|
|
2778
|
+
if (/\.(?:[cm]?[jt]sx?)$/i.test(file)) return "js";
|
|
2779
|
+
if (/\.py$/i.test(file)) return "py";
|
|
2780
|
+
if (/_test\.go$/i.test(file)) return "go";
|
|
2781
|
+
return null;
|
|
2782
|
+
}
|
|
2783
|
+
function codeLines(text, language) {
|
|
2784
|
+
let block = false;
|
|
2785
|
+
return text.split(/\r?\n/).map((line) => {
|
|
2786
|
+
const trimmed = line.trim();
|
|
2787
|
+
if (language !== "py") {
|
|
2788
|
+
if (block) {
|
|
2789
|
+
if (trimmed.includes("*/")) block = false;
|
|
2790
|
+
return "";
|
|
2791
|
+
}
|
|
2792
|
+
if (trimmed.startsWith("/*")) {
|
|
2793
|
+
if (!trimmed.includes("*/")) block = true;
|
|
2794
|
+
return "";
|
|
2795
|
+
}
|
|
2796
|
+
if (trimmed.startsWith("//")) return "";
|
|
2797
|
+
} else if (trimmed.startsWith("#")) return "";
|
|
2798
|
+
return line;
|
|
2799
|
+
});
|
|
2800
|
+
}
|
|
2801
|
+
var JS_TEST_START = /(?<![\w$.])(?:it|test)(?:\s*\.\s*(?:only|skip|concurrent|serial))*\s*\(\s*(['"`])((?:\\.|(?!\1)[^\\\n])*)\1/;
|
|
2802
|
+
var PY_TEST_START = /^\s*(?:async\s+)?def\s+(test\w*)\s*\(/;
|
|
2803
|
+
var GO_TEST_START = /^func\s+(Test\w+)\s*\(/;
|
|
2804
|
+
var JS_ASSERTION = /(?<![\w$.])(?:expect|assert|expectTypeOf)\b(?!\s*\.\s*(?:assertions|hasAssertions|extend|anything|any|objectContaining|arrayContaining|stringContaining|stringMatching)\b)(?:\s*\.\s*\w+)*\s*\(|\.should\b/;
|
|
2805
|
+
var PY_ASSERTION = /^\s*assert\b|\bself\s*\.\s*assert\w*\s*\(|\bpytest\s*\.\s*raises\b|\bassertRaises\b/;
|
|
2806
|
+
var GO_ASSERTION = /\bt\s*\.\s*(?:Error|Errorf|Fatal|Fatalf|Fail|FailNow)\b|\b(?:assert|require)\s*\.\s*\w+\s*\(/;
|
|
2807
|
+
function callEnd(text, from, limit) {
|
|
2808
|
+
let depth = 1, quote = "";
|
|
2809
|
+
for (let i = from; i < limit; i++) {
|
|
2810
|
+
const char = text[i];
|
|
2811
|
+
if (quote) {
|
|
2812
|
+
if (char === "\\") i++;
|
|
2813
|
+
else if (char === quote || char === "\n" && quote !== "`") quote = "";
|
|
2814
|
+
} else if (char === '"' || char === "'" || char === "`") quote = char;
|
|
2815
|
+
else if (char === "/" && text[i + 1] === "/" && (i === from || /\s/.test(text[i - 1]))) {
|
|
2816
|
+
const newline = text.indexOf("\n", i);
|
|
2817
|
+
i = newline < 0 || newline > limit ? limit : newline;
|
|
2818
|
+
} else if (char === "(") depth++;
|
|
2819
|
+
else if (char === ")" && --depth === 0) return i;
|
|
2820
|
+
}
|
|
2821
|
+
return limit;
|
|
2822
|
+
}
|
|
2823
|
+
function testBodies(text, language) {
|
|
2824
|
+
const lines = codeLines(text, language);
|
|
2825
|
+
if (language === "js") {
|
|
2826
|
+
const code = lines.join("\n");
|
|
2827
|
+
const matches = [...code.matchAll(new RegExp(JS_TEST_START.source, "g"))];
|
|
2828
|
+
let line = 1, cursor = 0;
|
|
2829
|
+
return matches.map((match, i) => {
|
|
2830
|
+
const at = match.index ?? 0;
|
|
2831
|
+
for (; cursor < at; cursor++) if (code.charCodeAt(cursor) === 10) line++;
|
|
2832
|
+
const from = at + match[0].length;
|
|
2833
|
+
const limit = i + 1 < matches.length ? matches[i + 1].index ?? code.length : code.length;
|
|
2834
|
+
return { name: match[2], line, body: code.slice(from, Math.min(callEnd(code, from, limit) + 1, limit)), parametrized: false };
|
|
2835
|
+
});
|
|
2836
|
+
}
|
|
2837
|
+
const startPattern = language === "py" ? PY_TEST_START : GO_TEST_START;
|
|
2838
|
+
const starts = [];
|
|
2839
|
+
lines.forEach((line, index) => {
|
|
2840
|
+
const match = startPattern.exec(line);
|
|
2841
|
+
if (match) starts.push({ index, name: match[1], length: match[0].length });
|
|
2842
|
+
});
|
|
2843
|
+
const indentOf = (line) => /^\s*/.exec(line)[0].length;
|
|
2844
|
+
return starts.map((start, n) => {
|
|
2845
|
+
const limit = n + 1 < starts.length ? starts[n + 1].index : lines.length;
|
|
2846
|
+
const header = lines[start.index], indent = indentOf(header);
|
|
2847
|
+
let end = start.index + 1, depth = balance(header);
|
|
2848
|
+
while (depth > 0 && end < limit) depth += balance(lines[end++]);
|
|
2849
|
+
while (end < limit && (language === "go" ? lines[end - 1] !== "}" : !lines[end].trim() || indentOf(lines[end]) > indent)) end++;
|
|
2850
|
+
let top = start.index;
|
|
2851
|
+
while (top > 0 && start.index - top < 200 && lines[top - 1].trim() && !/^\s*(?:async\s+)?(?:def|class)\b/.test(lines[top - 1])) top--;
|
|
2852
|
+
return {
|
|
2853
|
+
name: start.name,
|
|
2854
|
+
line: start.index + 1,
|
|
2855
|
+
body: [header.slice(start.length), ...lines.slice(start.index + 1, end)].join("\n"),
|
|
2856
|
+
parametrized: /parametrize/.test(lines.slice(top, start.index).join("\n"))
|
|
2857
|
+
};
|
|
2858
|
+
});
|
|
2859
|
+
}
|
|
2860
|
+
function balance(line) {
|
|
2861
|
+
const bare = line.replace(/(['"`])(?:\\.|(?!\1)[^\\])*\1/g, "");
|
|
2862
|
+
let net = 0;
|
|
2863
|
+
for (const char of bare) net += "([{".includes(char) ? 1 : ")]}".includes(char) ? -1 : 0;
|
|
2864
|
+
return net;
|
|
2865
|
+
}
|
|
2866
|
+
function normalize(statement) {
|
|
2867
|
+
return statement.replace(/\s+/g, "").replace(/'/g, '"').replace(/,([)\]}])/g, "$1").replace(/;$/, "");
|
|
2868
|
+
}
|
|
2869
|
+
function splitStatements(text) {
|
|
2870
|
+
const pieces = [];
|
|
2871
|
+
let quote = "", start = 0;
|
|
2872
|
+
for (let i = 0; i < text.length; i++) {
|
|
2873
|
+
const char = text[i];
|
|
2874
|
+
if (quote) {
|
|
2875
|
+
if (char === "\\") i++;
|
|
2876
|
+
else if (char === quote) quote = "";
|
|
2877
|
+
} else if (char === '"' || char === "'" || char === "`") quote = char;
|
|
2878
|
+
else if (char === ";") {
|
|
2879
|
+
pieces.push(text.slice(start, i));
|
|
2880
|
+
start = i + 1;
|
|
2881
|
+
}
|
|
2882
|
+
}
|
|
2883
|
+
pieces.push(text.slice(start));
|
|
2884
|
+
return pieces.map((piece) => piece.trim()).filter(Boolean);
|
|
2885
|
+
}
|
|
2886
|
+
var CALLBACK_HEADER = /^[,\s]*(?:async\s*)?(?:\([^)]*\)|[\w$]+)\s*=>\s*\{\s*|^[,\s]*(?:async\s+)?function\s*\w*\s*\([^)]*\)\s*\{\s*/;
|
|
2887
|
+
function parseTests(text, language) {
|
|
2888
|
+
const assertionPattern = language === "js" ? JS_ASSERTION : language === "py" ? PY_ASSERTION : GO_ASSERTION;
|
|
2889
|
+
const seen = /* @__PURE__ */ new Map();
|
|
2890
|
+
return testBodies(text, language).map((test) => {
|
|
2891
|
+
const body = test.body.split("\n");
|
|
2892
|
+
const assertions = [], raw = [], setupLines = [];
|
|
2893
|
+
for (let i = 0; i < body.length; i++) {
|
|
2894
|
+
if (!assertionPattern.test(body[i])) {
|
|
2895
|
+
setupLines.push(body[i]);
|
|
2896
|
+
continue;
|
|
2897
|
+
}
|
|
2898
|
+
let statement = body[i].trim(), depth = balance(body[i]);
|
|
2899
|
+
while (depth > 0 && i + 1 < body.length) {
|
|
2900
|
+
i++;
|
|
2901
|
+
statement += " " + body[i].trim();
|
|
2902
|
+
depth += balance(body[i]);
|
|
2903
|
+
}
|
|
2904
|
+
for (const piece of language === "js" ? splitStatements(statement) : [statement]) {
|
|
2905
|
+
const clean = piece.replace(CALLBACK_HEADER, "");
|
|
2906
|
+
if (assertionPattern.test(clean)) {
|
|
2907
|
+
raw.push(clean);
|
|
2908
|
+
assertions.push(normalize(clean));
|
|
2909
|
+
} else setupLines.push(clean);
|
|
2910
|
+
}
|
|
2911
|
+
}
|
|
2912
|
+
const setup = setupLines.join("\n").replace(/\s+/g, "");
|
|
2913
|
+
const bodyHash = createHash3("sha1").update(test.body.replace(/\s+/g, "")).digest("hex");
|
|
2914
|
+
const count = (seen.get(test.name) ?? 0) + 1;
|
|
2915
|
+
seen.set(test.name, count);
|
|
2916
|
+
return { key: `${test.name}#${count}`, parametrized: test.parametrized, name: test.name, line: test.line, assertions, raw, setup, bodyHash };
|
|
2917
|
+
});
|
|
2918
|
+
}
|
|
2919
|
+
function skeleton(assertion) {
|
|
2920
|
+
return assertion.replace(/(['"`])(?:\\.|(?!\1)[^\\])*\1/g, "S").replace(/\b\d+(?:\.\d+)?\b/g, "N").replace(/\b(?:true|false|null|undefined|None|True|False|nil)\b/g, "V").replace(/\s+/g, "");
|
|
2921
|
+
}
|
|
2922
|
+
var WEAK_JS = /\.(?:toBeDefined|toBeTruthy|toBeFalsy|toBeUndefined)\s*\(|\.not\s*\.\s*(?:toBeNull|toBeUndefined|toThrow)\s*\(|\.toBeGreaterThanOrEqual\s*\(\s*0\s*\)/;
|
|
2923
|
+
var WEAK_PY = /^\s*assert\s+[\w.()\[\]]+\s*$|^\s*assert\s+.+\sis\s+not\s+None\s*$|^\s*assert\s+True\s*$/;
|
|
2924
|
+
function isWeaker(oldAssertion, newAssertion, language) {
|
|
2925
|
+
if (language === "js") return WEAK_JS.test(newAssertion) && !WEAK_JS.test(oldAssertion);
|
|
2926
|
+
if (language === "py") return WEAK_PY.test(newAssertion) && !WEAK_PY.test(oldAssertion);
|
|
2927
|
+
return false;
|
|
2928
|
+
}
|
|
2929
|
+
function finding(type, file, line, snippet, reason, commit) {
|
|
2930
|
+
return { type, file, lineNumber: Math.max(1, line), lineSnippet: snippet, commitHash: commit, commitDate: "", reason, suggestion: RULE_GUIDES[type].suggestion };
|
|
2931
|
+
}
|
|
2932
|
+
function analyzeTestFile(oldPath, newPath, oldText, newText, commit) {
|
|
2933
|
+
const language = languageOf(newText === null ? oldPath : newPath);
|
|
2934
|
+
if (!language || !(isTestFile(newPath) || isTestFile(oldPath))) return EMPTY;
|
|
2935
|
+
const before = oldText === null ? [] : parseTests(oldText, language);
|
|
2936
|
+
const after = newText === null ? [] : parseTests(newText, language);
|
|
2937
|
+
const afterByKey = new Map(after.map((test) => [test.key, test]));
|
|
2938
|
+
const newCode = newText === null ? "" : codeLines(newText, language).join("\n");
|
|
2939
|
+
const tally = (tests) => {
|
|
2940
|
+
const totals = /* @__PURE__ */ new Map();
|
|
2941
|
+
for (const test of tests) for (const assertion of test.assertions) totals.set(assertion, (totals.get(assertion) ?? 0) + 1);
|
|
2942
|
+
return totals;
|
|
2943
|
+
};
|
|
2944
|
+
const oldTotals = tally(before), newTotals = tally(after);
|
|
2945
|
+
const beforeKeys = new Set(before.map((test) => test.key));
|
|
2946
|
+
const result = { immediate: [], removedTests: [], addedHashes: [], rewritten: [] };
|
|
2947
|
+
for (const test of after) if (!beforeKeys.has(test.key)) result.addedHashes.push(test.bodyHash);
|
|
2948
|
+
for (const old of before) {
|
|
2949
|
+
const current = afterByKey.get(old.key);
|
|
2950
|
+
if (!current) {
|
|
2951
|
+
if (newCode && ['"', "'", "`"].some((quote) => newCode.includes(quote + old.name + quote))) continue;
|
|
2952
|
+
result.removedTests.push({
|
|
2953
|
+
bodyHash: old.bodyHash,
|
|
2954
|
+
finding: finding(
|
|
2955
|
+
"test-removed",
|
|
2956
|
+
newPath,
|
|
2957
|
+
1,
|
|
2958
|
+
`${old.name}: test no longer present`,
|
|
2959
|
+
newText === null ? "Test file deleted" : "Test removed",
|
|
2960
|
+
commit
|
|
2961
|
+
)
|
|
2962
|
+
});
|
|
2963
|
+
continue;
|
|
2964
|
+
}
|
|
2965
|
+
if (current.parametrized && !old.parametrized) continue;
|
|
2966
|
+
const snapshot = (test) => test.raw.some((statement) => /\.toMatch(?:Inline)?Snapshot\b|\.matchSnapshot\b|\bsnapshot\s*\(/.test(statement));
|
|
2967
|
+
if (snapshot(current) && !snapshot(old)) continue;
|
|
2968
|
+
if (current.assertions.length < old.assertions.length) {
|
|
2969
|
+
const budget = tally([current]);
|
|
2970
|
+
const lost = old.assertions.filter((assertion) => {
|
|
2971
|
+
const left = budget.get(assertion) ?? 0;
|
|
2972
|
+
if (left > 0) {
|
|
2973
|
+
budget.set(assertion, left - 1);
|
|
2974
|
+
return false;
|
|
2975
|
+
}
|
|
2976
|
+
return true;
|
|
2977
|
+
});
|
|
2978
|
+
if (lost.every((assertion) => (newTotals.get(assertion) ?? 0) >= (oldTotals.get(assertion) ?? 0))) continue;
|
|
2979
|
+
result.immediate.push(finding(
|
|
2980
|
+
"assertion-weakened",
|
|
2981
|
+
newPath,
|
|
2982
|
+
current.line,
|
|
2983
|
+
`${old.name}: ${old.assertions.length} \u2192 ${current.assertions.length} assertions`,
|
|
2984
|
+
"Test checks less than before",
|
|
2985
|
+
commit
|
|
2986
|
+
));
|
|
2987
|
+
continue;
|
|
2988
|
+
}
|
|
2989
|
+
if (current.assertions.length === old.assertions.length && old.assertions.length > 0) {
|
|
2990
|
+
const changed = old.assertions.map((assertion, i) => ({ before: assertion, after: current.assertions[i], rawBefore: old.raw[i], rawAfter: current.raw[i] })).filter((pair) => pair.before !== pair.after);
|
|
2991
|
+
const weaker = changed.find((pair) => isWeaker(pair.rawBefore, pair.rawAfter, language));
|
|
2992
|
+
if (weaker) {
|
|
2993
|
+
result.immediate.push(finding(
|
|
2994
|
+
"assertion-weakened",
|
|
2995
|
+
newPath,
|
|
2996
|
+
current.line,
|
|
2997
|
+
`${old.name}: ${weaker.rawBefore.slice(0, 80)} \u2192 ${weaker.rawAfter.slice(0, 80)}`,
|
|
2998
|
+
"Assertion replaced by a weaker check",
|
|
2999
|
+
commit
|
|
3000
|
+
));
|
|
3001
|
+
} else if (changed.length && old.setup === current.setup && changed.every((pair) => skeleton(pair.before) === skeleton(pair.after))) {
|
|
3002
|
+
result.rewritten.push(finding(
|
|
3003
|
+
"expectation-rewritten",
|
|
3004
|
+
newPath,
|
|
3005
|
+
current.line,
|
|
3006
|
+
`${old.name}: ${changed[0].rawBefore.slice(0, 80)} \u2192 ${changed[0].rawAfter.slice(0, 80)}`,
|
|
3007
|
+
"Expected value edited while the test inputs stayed the same",
|
|
3008
|
+
commit
|
|
3009
|
+
));
|
|
3010
|
+
}
|
|
3011
|
+
}
|
|
3012
|
+
}
|
|
3013
|
+
return result;
|
|
3014
|
+
}
|
|
3015
|
+
function resolveTampering(files, productionChanged) {
|
|
3016
|
+
const added = new Set(files.flatMap((file) => file.addedHashes));
|
|
3017
|
+
return [
|
|
3018
|
+
...files.flatMap((file) => file.immediate),
|
|
3019
|
+
...files.flatMap((file) => file.removedTests).filter((removed) => !added.has(removed.bodyHash)).map((removed) => removed.finding),
|
|
3020
|
+
...productionChanged ? files.flatMap((file) => file.rewritten) : []
|
|
3021
|
+
];
|
|
3022
|
+
}
|
|
3023
|
+
var GUARD_FILE = /(?:^|\/)(?:package\.json|tsconfig[^/]*\.json|jsconfig\.json|(?:jest|vitest|vite)\.config\.[cm]?[jt]s|\.eslintrc(?:\.[a-z]+)?|eslint\.config\.[cm]?[jt]s|pyproject\.toml|setup\.cfg|pytest\.ini|tox\.ini|\.coveragerc|codecov\.ya?ml|\.gitlab-ci\.ya?ml|\.github\/workflows\/[^/]+\.ya?ml)$/i;
|
|
3024
|
+
function isGuardFile(file) {
|
|
3025
|
+
return GUARD_FILE.test(file);
|
|
3026
|
+
}
|
|
3027
|
+
function lineOf(text, index) {
|
|
3028
|
+
let line = 1;
|
|
3029
|
+
for (let i = 0; i < index && i < text.length; i++) if (text.charCodeAt(i) === 10) line++;
|
|
3030
|
+
return line;
|
|
3031
|
+
}
|
|
3032
|
+
function numbers(text, pattern) {
|
|
3033
|
+
const seen = /* @__PURE__ */ new Map();
|
|
3034
|
+
return [...text.matchAll(pattern)].map((match) => {
|
|
3035
|
+
const label = match[1] ?? "value";
|
|
3036
|
+
const count = (seen.get(label) ?? 0) + 1;
|
|
3037
|
+
seen.set(label, count);
|
|
3038
|
+
return { key: `${label}#${count}`, value: Number(match[match.length - 1]), index: match.index ?? 0 };
|
|
3039
|
+
});
|
|
3040
|
+
}
|
|
3041
|
+
var THRESHOLD_PATTERNS = [
|
|
3042
|
+
/\b(lines|branches|functions|statements)\b["']?\s*[:=]\s*(\d+(?:\.\d+)?)/g,
|
|
3043
|
+
/\b(fail_under)\s*[:=]\s*(\d+(?:\.\d+)?)/g,
|
|
3044
|
+
/(--cov-fail-under)[= ](\d+(?:\.\d+)?)/g
|
|
3045
|
+
];
|
|
3046
|
+
var SWITCHES = ["strict", "noImplicitAny", "strictNullChecks", "noImplicitReturns", "useUnknownInCatchVariables"];
|
|
3047
|
+
var TEST_COMMAND = /\brun:.*\b(?:tests?|unit|pytest|vitest|jest|mocha|ava|tap|tox|nox|e2e|spec)\b|^\s*-?\s*(?:npm|yarn|pnpm|go|cargo|mvn|gradle|make)\s+(?:run\s+)?(?:\w+\s+)?tests?\b/;
|
|
3048
|
+
function severityOf(value) {
|
|
3049
|
+
return /^(?:error|2)$/.test(value) ? 2 : /^(?:warn|1)$/.test(value) ? 1 : 0;
|
|
3050
|
+
}
|
|
3051
|
+
function analyzeGuardFile(file, oldText, newText, commit) {
|
|
3052
|
+
if (oldText === null || newText === null) return [];
|
|
3053
|
+
const out = [];
|
|
3054
|
+
const add = (index, snippet, reason) => out.push(finding("guard-weakened", file, lineOf(newText, index), snippet, reason, commit));
|
|
3055
|
+
if (/threshold|fail_under|cov-fail-under|coverage/i.test(newText)) {
|
|
3056
|
+
for (const pattern of THRESHOLD_PATTERNS) {
|
|
3057
|
+
const old = new Map(numbers(oldText, pattern).map((item) => [item.key, item.value]));
|
|
3058
|
+
for (const item of numbers(newText, pattern)) {
|
|
3059
|
+
const previous = old.get(item.key);
|
|
3060
|
+
if (previous !== void 0 && item.value < previous) {
|
|
3061
|
+
add(item.index, `${item.key.split("#")[0]}: ${previous} \u2192 ${item.value}`, "Coverage threshold lowered");
|
|
3062
|
+
}
|
|
3063
|
+
}
|
|
3064
|
+
}
|
|
3065
|
+
}
|
|
3066
|
+
if (/(?:^|\/)(?:tsconfig|jsconfig)[^/]*\.json$/i.test(file)) {
|
|
3067
|
+
for (const name of SWITCHES) {
|
|
3068
|
+
const pattern = new RegExp(`"${name}"\\s*:\\s*(true|false)`);
|
|
3069
|
+
const before = pattern.exec(oldText), after = pattern.exec(newText);
|
|
3070
|
+
if (before?.[1] === "true" && after?.[1] === "false") add(after.index, `${name}: true \u2192 false`, "Compiler strictness switched off");
|
|
3071
|
+
}
|
|
3072
|
+
}
|
|
3073
|
+
if (/eslint/i.test(file)) {
|
|
3074
|
+
const rules = (text) => new Map([...text.matchAll(/["']([@\w/.-]+)["']\s*:\s*(?:\[\s*)?["']?(error|warn|off|[012])["']?/g)].map((match) => [match[1], { level: severityOf(match[2]), index: match.index ?? 0 }]));
|
|
3075
|
+
const before = rules(oldText);
|
|
3076
|
+
for (const [name, now] of rules(newText)) {
|
|
3077
|
+
const previous = before.get(name);
|
|
3078
|
+
if (previous && previous.level === 2 && now.level < 2) add(now.index, `${name}: error \u2192 ${now.level ? "warn" : "off"}`, "Lint rule downgraded");
|
|
3079
|
+
}
|
|
3080
|
+
}
|
|
3081
|
+
if (/(?:\.github\/workflows\/|\.gitlab-ci)/.test(file)) {
|
|
3082
|
+
const count = (text, pattern) => text.split(/\r?\n/).filter((line) => !line.trim().startsWith("#") && pattern.test(line)).length;
|
|
3083
|
+
const lenient = /continue-on-error:\s*true|allow_failure:\s*true/;
|
|
3084
|
+
if (count(newText, lenient) > count(oldText, lenient)) {
|
|
3085
|
+
add(newText.search(lenient), "continue-on-error: true", "CI step is now allowed to fail");
|
|
3086
|
+
}
|
|
3087
|
+
if (count(newText, TEST_COMMAND) < count(oldText, TEST_COMMAND)) {
|
|
3088
|
+
add(0, "test command no longer runs in CI", "Test step removed from CI");
|
|
3089
|
+
}
|
|
3090
|
+
}
|
|
3091
|
+
if (/(?:^|\/)package\.json$/i.test(file)) {
|
|
3092
|
+
try {
|
|
3093
|
+
const before = JSON.parse(oldText)?.scripts?.test, after = JSON.parse(newText)?.scripts?.test;
|
|
3094
|
+
if (typeof before === "string" && typeof after === "string" && before !== after && (/^\s*(?:echo\b|true\b|exit\s+0|:)/.test(after) || /\|\|\s*(?:true|exit\s+0)\b/.test(after) || /--passWithNoTests/.test(after) && !/--passWithNoTests/.test(before))) {
|
|
3095
|
+
add(Math.max(0, newText.indexOf('"test"')), `test: ${after}`, "Test script can no longer fail");
|
|
3096
|
+
}
|
|
3097
|
+
} catch {
|
|
3098
|
+
}
|
|
3099
|
+
}
|
|
3100
|
+
return out;
|
|
3101
|
+
}
|
|
3102
|
+
|
|
3103
|
+
// src/review.ts
|
|
2715
3104
|
var GENERATED_PATHS2 = ["**/node_modules/**", "**/dist/**", "**/build/**", "**/vendor/**", "**/*.min.js", "**/*.map"];
|
|
2716
3105
|
var MAX_SCAN_BYTES2 = 50 * 1024 * 1024;
|
|
2717
3106
|
function compareFindings(before, after, newPath) {
|
|
2718
3107
|
const old = /* @__PURE__ */ new Map();
|
|
2719
|
-
for (const
|
|
2720
|
-
const key = findingFingerprint({ ...
|
|
3108
|
+
for (const finding2 of before) {
|
|
3109
|
+
const key = findingFingerprint({ ...finding2, file: newPath });
|
|
2721
3110
|
const bucket = old.get(key) || { findings: [], matched: 0 };
|
|
2722
|
-
bucket.findings.push(
|
|
3111
|
+
bucket.findings.push(finding2);
|
|
2723
3112
|
old.set(key, bucket);
|
|
2724
3113
|
}
|
|
2725
3114
|
const introduced = [];
|
|
2726
3115
|
let unchanged = 0;
|
|
2727
|
-
for (const
|
|
2728
|
-
const bucket = old.get(findingFingerprint(
|
|
3116
|
+
for (const finding2 of after) {
|
|
3117
|
+
const bucket = old.get(findingFingerprint(finding2));
|
|
2729
3118
|
if (bucket && bucket.matched < bucket.findings.length) {
|
|
2730
3119
|
bucket.matched++;
|
|
2731
3120
|
unchanged++;
|
|
2732
|
-
} else introduced.push(
|
|
3121
|
+
} else introduced.push(finding2);
|
|
2733
3122
|
}
|
|
2734
3123
|
return { introduced, removed: [...old.values()].flatMap((bucket) => bucket.findings.slice(bucket.matched)), unchanged };
|
|
2735
3124
|
}
|
|
@@ -2747,7 +3136,7 @@ async function compareSourceChanges(options) {
|
|
|
2747
3136
|
const policy = refs.baseSha === "EMPTY" ? { config: configFromText("{}", "default first-commit policy"), filename: null } : await reviewPolicy(cwd, refs.baseSha, options.signal);
|
|
2748
3137
|
const format = options.format ?? "terminal";
|
|
2749
3138
|
if (!["terminal", "json", "github", "markdown", "sarif"].includes(format)) throw new ConfigError("Unsupported review format.");
|
|
2750
|
-
const failOn = options.failOn ?? policy.config.failOn ?? ["test-skip", "trivial-assert", "test-focus"];
|
|
3139
|
+
const failOn = options.failOn ?? policy.config.failOn ?? ["test-skip", "trivial-assert", "test-focus", "assertion-weakened", "guard-weakened"];
|
|
2751
3140
|
for (const rule of failOn) if (!VALID_DETECTORS.includes(rule)) throw new ConfigError(`Unknown review rule '${rule}'.`);
|
|
2752
3141
|
const changes = staged?.changes ?? await changedTreeFiles(cwd, refs.mergeBaseSha, refs.headSha, options.signal);
|
|
2753
3142
|
const policyChanges = changes.filter((change) => CONFIG_FILENAMES.includes(change.oldPath) || CONFIG_FILENAMES.includes(change.newPath)).map((change) => change.newPath);
|
|
@@ -2779,13 +3168,39 @@ async function compareSourceChanges(options) {
|
|
|
2779
3168
|
output: ""
|
|
2780
3169
|
};
|
|
2781
3170
|
let totalBytes = 0;
|
|
3171
|
+
const tamperingFiles = [];
|
|
3172
|
+
const guardFindings = [];
|
|
3173
|
+
let productionChanged = false;
|
|
3174
|
+
const readText = async (mode, oid) => {
|
|
3175
|
+
if (mode === "000000") return null;
|
|
3176
|
+
if (!["100644", "100755"].includes(mode)) throw new ConfigError("Changed source is a symlink, submodule or unsupported Git mode; comparison is incomplete.");
|
|
3177
|
+
const content = await readBlob(cwd, oid, options.signal);
|
|
3178
|
+
totalBytes += content.length;
|
|
3179
|
+
if (totalBytes > MAX_SCAN_BYTES2) throw new ConfigError("Review exceeds the 50 MiB source comparison limit.");
|
|
3180
|
+
if (content.includes(0)) throw new ConfigError("Binary content in a source comparison.");
|
|
3181
|
+
return new TextDecoder("utf-8", { fatal: true }).decode(content);
|
|
3182
|
+
};
|
|
2782
3183
|
const existingCounts = /* @__PURE__ */ new Map();
|
|
2783
3184
|
for (const change of changes) {
|
|
2784
3185
|
assertNotAborted(options.signal);
|
|
2785
3186
|
const oldSupported = change.oldMode !== "000000" && isCodeFile(change.oldPath);
|
|
2786
3187
|
const newSupported = change.newMode !== "000000" && isCodeFile(change.newPath);
|
|
2787
3188
|
if (!oldSupported && !newSupported) {
|
|
2788
|
-
|
|
3189
|
+
const guarded = (isGuardFile(change.newPath) || isGuardFile(change.oldPath)) && isCodeFile(change.newPath + ".ts", ignores);
|
|
3190
|
+
if (!guarded) {
|
|
3191
|
+
result.unsupportedFiles++;
|
|
3192
|
+
continue;
|
|
3193
|
+
}
|
|
3194
|
+
try {
|
|
3195
|
+
const oldText = await readText(change.oldMode, change.oldOid);
|
|
3196
|
+
const newText = await readText(change.newMode, change.newOid);
|
|
3197
|
+
guardFindings.push(...analyzeGuardFile(change.newPath, oldText, newText, refs.headSha));
|
|
3198
|
+
result.filesInspected++;
|
|
3199
|
+
} catch (error) {
|
|
3200
|
+
assertNotAborted(options.signal);
|
|
3201
|
+
result.skippedFiles.push({ file: change.newPath, reason: error instanceof Error ? error.message : String(error) });
|
|
3202
|
+
if (totalBytes > MAX_SCAN_BYTES2) break;
|
|
3203
|
+
}
|
|
2789
3204
|
continue;
|
|
2790
3205
|
}
|
|
2791
3206
|
const oldInScope = oldSupported && isCodeFile(change.oldPath, ignores);
|
|
@@ -2796,14 +3211,8 @@ async function compareSourceChanges(options) {
|
|
|
2796
3211
|
}
|
|
2797
3212
|
try {
|
|
2798
3213
|
let inspectedLines = 0;
|
|
2799
|
-
const scan =
|
|
2800
|
-
if (
|
|
2801
|
-
if (!["100644", "100755"].includes(mode)) throw new ConfigError("Changed source is a symlink, submodule or unsupported Git mode; comparison is incomplete.");
|
|
2802
|
-
const content = await readBlob(cwd, oid, options.signal);
|
|
2803
|
-
totalBytes += content.length;
|
|
2804
|
-
if (totalBytes > MAX_SCAN_BYTES2) throw new ConfigError("Review exceeds the 50 MiB source comparison limit.");
|
|
2805
|
-
if (content.includes(0)) throw new ConfigError("Binary content in a source comparison.");
|
|
2806
|
-
const text = new TextDecoder("utf-8", { fatal: true }).decode(content);
|
|
3214
|
+
const scan = (text, filename, commit) => {
|
|
3215
|
+
if (text === null) return [];
|
|
2807
3216
|
const lines = text.split(/\r?\n/);
|
|
2808
3217
|
if (lines.at(-1) === "") lines.pop();
|
|
2809
3218
|
inspectedLines += lines.length;
|
|
@@ -2812,13 +3221,18 @@ async function compareSourceChanges(options) {
|
|
|
2812
3221
|
if (oldSupported && !newSupported && change.newMode !== "000000") {
|
|
2813
3222
|
throw new ConfigError("Source moved to an unsupported extension; cannot claim its findings were resolved.");
|
|
2814
3223
|
}
|
|
2815
|
-
const
|
|
2816
|
-
const
|
|
3224
|
+
const oldText = await readText(change.oldMode, change.oldOid);
|
|
3225
|
+
const newText = await readText(change.newMode, change.newOid);
|
|
3226
|
+
const before = scan(oldText, change.oldPath, refs.mergeBaseSha);
|
|
3227
|
+
const after = scan(newText, change.newPath, refs.headSha);
|
|
3228
|
+
if (isTestFile(change.oldPath) || isTestFile(change.newPath)) tamperingFiles.push(analyzeTestFile(change.oldPath, change.newPath, oldText, newText, refs.headSha));
|
|
3229
|
+
else if (change.oldOid !== change.newOid) productionChanged = true;
|
|
3230
|
+
if ((isGuardFile(change.newPath) || isGuardFile(change.oldPath)) && oldText !== null && newText !== null) guardFindings.push(...analyzeGuardFile(change.newPath, oldText, newText, refs.headSha));
|
|
2817
3231
|
const compared = compareFindings(before, after, change.newPath);
|
|
2818
3232
|
const introduced = new Set(compared.introduced);
|
|
2819
|
-
for (const
|
|
2820
|
-
if (introduced.has(
|
|
2821
|
-
const fingerprint = findingFingerprint(
|
|
3233
|
+
for (const finding2 of after) {
|
|
3234
|
+
if (introduced.has(finding2)) continue;
|
|
3235
|
+
const fingerprint = findingFingerprint(finding2);
|
|
2822
3236
|
existingCounts.set(fingerprint, (existingCounts.get(fingerprint) ?? 0) + 1);
|
|
2823
3237
|
}
|
|
2824
3238
|
result.infractions.push(...compared.introduced);
|
|
@@ -2837,11 +3251,13 @@ async function compareSourceChanges(options) {
|
|
|
2837
3251
|
assertNotAborted(options.signal);
|
|
2838
3252
|
await staged?.verify();
|
|
2839
3253
|
result.complete = result.skippedFiles.length === 0;
|
|
3254
|
+
const rules = policy.config.rules ?? {};
|
|
3255
|
+
result.infractions.push(...[...resolveTampering(tamperingFiles, productionChanged), ...guardFindings].filter((finding2) => rules[finding2.type] !== "off"));
|
|
2840
3256
|
const exceptions = applyExceptions(result.infractions, options.ignoreExceptions ? [] : policy.config.exceptions, /* @__PURE__ */ new Date(), existingCounts);
|
|
2841
3257
|
result.infractions = exceptions.findings;
|
|
2842
3258
|
result.suppressedFindings = exceptions.suppressedFindings;
|
|
2843
3259
|
result.expiredExceptionCount = exceptions.expiredExceptionCount;
|
|
2844
|
-
result.failedRules = [...new Set(result.infractions.filter((
|
|
3260
|
+
result.failedRules = [...new Set(result.infractions.filter((finding2) => failOn.includes(finding2.type)).map((finding2) => finding2.type))];
|
|
2845
3261
|
result.passed = result.complete && result.failedRules.length === 0;
|
|
2846
3262
|
result.exitCode = !result.complete ? 2 : result.passed ? 0 : 1;
|
|
2847
3263
|
if (format === "json") result.output = JSON.stringify(result, null, 2);
|
package/dist/cli.js
CHANGED
|
@@ -22,7 +22,7 @@ import {
|
|
|
22
22
|
roastRepository,
|
|
23
23
|
terminalText,
|
|
24
24
|
writeBaseline
|
|
25
|
-
} from "./chunk-
|
|
25
|
+
} from "./chunk-GMBQNL6Z.js";
|
|
26
26
|
|
|
27
27
|
// src/cli.ts
|
|
28
28
|
import { Command } from "commander";
|
|
@@ -254,7 +254,7 @@ program.command("init").description("Initialize .agentroastrc.json and CI/hook c
|
|
|
254
254
|
hook: opts.hook
|
|
255
255
|
});
|
|
256
256
|
if (res.created.length > 0) {
|
|
257
|
-
console.log(`Initialized agent-roast in ${terminalText(
|
|
257
|
+
console.log(`Initialized agent-roast in ${terminalText(res.directory)}:`);
|
|
258
258
|
for (const file of res.created) {
|
|
259
259
|
console.log(` + Created ${terminalText(file)}`);
|
|
260
260
|
}
|
package/dist/index.d.ts
CHANGED
|
@@ -32,7 +32,7 @@ interface CommitInfo {
|
|
|
32
32
|
isAiAttributed: boolean;
|
|
33
33
|
aiSignatures: string[];
|
|
34
34
|
}
|
|
35
|
-
type DetectorType = "test-skip" | "type-escape" | "swallowed-error" | "panic-loop" | "trivial-assert" | "stub-abandonment" | "test-focus";
|
|
35
|
+
type DetectorType = "test-skip" | "type-escape" | "swallowed-error" | "panic-loop" | "trivial-assert" | "stub-abandonment" | "test-focus" | "assertion-weakened" | "test-removed" | "expectation-rewritten" | "guard-weakened";
|
|
36
36
|
declare const VALID_DETECTORS: readonly DetectorType[];
|
|
37
37
|
interface Infraction {
|
|
38
38
|
fingerprint?: string;
|
|
@@ -366,6 +366,8 @@ interface InitOptions {
|
|
|
366
366
|
hook?: boolean;
|
|
367
367
|
}
|
|
368
368
|
interface InitResult {
|
|
369
|
+
/** Actual project root containing the initialized files. */
|
|
370
|
+
directory: string;
|
|
369
371
|
created: string[];
|
|
370
372
|
skipped: string[];
|
|
371
373
|
}
|
package/dist/index.js
CHANGED
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-roast",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"description": "Offline
|
|
3
|
+
"version": "0.9.0",
|
|
4
|
+
"description": "Offline review for AI-assisted changes: catches weakened tests, lowered quality gates, skipped tests and swallowed errors",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "./dist/index.js",
|
|
7
7
|
"types": "./dist/index.d.ts",
|
|
@@ -20,16 +20,22 @@
|
|
|
20
20
|
"test": "vitest run",
|
|
21
21
|
"prepublishOnly": "npm run build",
|
|
22
22
|
"pretest": "npm run build",
|
|
23
|
-
"typecheck": "tsc --noEmit"
|
|
23
|
+
"typecheck": "tsc --noEmit",
|
|
24
|
+
"bench": "node scripts/benchmark-recall.mjs",
|
|
25
|
+
"bench:noise": "node scripts/benchmark-noise.mjs"
|
|
24
26
|
},
|
|
25
27
|
"keywords": [
|
|
26
28
|
"ai",
|
|
27
29
|
"agents",
|
|
28
|
-
"
|
|
30
|
+
"ai-agents",
|
|
29
31
|
"claude-code",
|
|
30
|
-
"
|
|
32
|
+
"cursor",
|
|
33
|
+
"codex",
|
|
34
|
+
"code-review",
|
|
35
|
+
"test-quality",
|
|
36
|
+
"ci",
|
|
37
|
+
"sarif",
|
|
31
38
|
"git",
|
|
32
|
-
"code-quality",
|
|
33
39
|
"cli"
|
|
34
40
|
],
|
|
35
41
|
"author": "Nari-Ab",
|
|
@@ -47,9 +53,7 @@
|
|
|
47
53
|
"picocolors": "^1.1.0"
|
|
48
54
|
},
|
|
49
55
|
"devDependencies": {
|
|
50
|
-
"@napi-rs/canvas": "^1.0.10",
|
|
51
56
|
"@types/node": "^20.0.0",
|
|
52
|
-
"gifenc": "^1.0.3",
|
|
53
57
|
"tsup": "^8.0.0",
|
|
54
58
|
"typescript": "^5.0.0",
|
|
55
59
|
"vitest": "^4.1.12"
|
package/schema.json
CHANGED
|
@@ -20,7 +20,11 @@
|
|
|
20
20
|
"panic-loop",
|
|
21
21
|
"trivial-assert",
|
|
22
22
|
"stub-abandonment",
|
|
23
|
-
"test-focus"
|
|
23
|
+
"test-focus",
|
|
24
|
+
"assertion-weakened",
|
|
25
|
+
"test-removed",
|
|
26
|
+
"expectation-rewritten",
|
|
27
|
+
"guard-weakened"
|
|
24
28
|
]
|
|
25
29
|
}
|
|
26
30
|
},
|
|
@@ -94,6 +98,34 @@
|
|
|
94
98
|
"warn",
|
|
95
99
|
"off"
|
|
96
100
|
]
|
|
101
|
+
},
|
|
102
|
+
"assertion-weakened": {
|
|
103
|
+
"enum": [
|
|
104
|
+
"error",
|
|
105
|
+
"warn",
|
|
106
|
+
"off"
|
|
107
|
+
]
|
|
108
|
+
},
|
|
109
|
+
"test-removed": {
|
|
110
|
+
"enum": [
|
|
111
|
+
"error",
|
|
112
|
+
"warn",
|
|
113
|
+
"off"
|
|
114
|
+
]
|
|
115
|
+
},
|
|
116
|
+
"expectation-rewritten": {
|
|
117
|
+
"enum": [
|
|
118
|
+
"error",
|
|
119
|
+
"warn",
|
|
120
|
+
"off"
|
|
121
|
+
]
|
|
122
|
+
},
|
|
123
|
+
"guard-weakened": {
|
|
124
|
+
"enum": [
|
|
125
|
+
"error",
|
|
126
|
+
"warn",
|
|
127
|
+
"off"
|
|
128
|
+
]
|
|
97
129
|
}
|
|
98
130
|
}
|
|
99
131
|
},
|