@kavishdua/proctor 1.0.0 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +48 -37
- package/dist/cli.js +714 -59
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -53,9 +53,14 @@ source needs Node 20.19+ or 22.12+.
|
|
|
53
53
|
|
|
54
54
|
## Quickstart
|
|
55
55
|
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
56
|
+
Check the changes in any git repository, with nothing to install first:
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
cd /path/to/repository
|
|
60
|
+
npx @kavishdua/proctor check
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
To run it from source instead:
|
|
59
64
|
|
|
60
65
|
```bash
|
|
61
66
|
git clone https://github.com/catfish-1234/proctor.git
|
|
@@ -91,16 +96,10 @@ qwen extensions install https://github.com/catfish-1234/proctor
|
|
|
91
96
|
|
|
92
97
|
**Other agents, or CLI use without an agent** (30 agents supported)
|
|
93
98
|
|
|
94
|
-
|
|
99
|
+
Add it to the repository as a dev dependency:
|
|
95
100
|
|
|
96
101
|
```bash
|
|
97
|
-
|
|
98
|
-
npm ci
|
|
99
|
-
npm run build
|
|
100
|
-
npm pack
|
|
101
|
-
|
|
102
|
-
# In the repository proctor should guard; use the .tgz path printed above
|
|
103
|
-
npm install --save-dev /absolute/path/to/kavishdua-proctor-*.tgz
|
|
102
|
+
npm install --save-dev @kavishdua/proctor
|
|
104
103
|
```
|
|
105
104
|
|
|
106
105
|
**Proctor does not install guards during npm installation by default.** npm still updates your
|
|
@@ -276,15 +275,17 @@ families.
|
|
|
276
275
|
**WI1xx checks "the work is done."** These look for incomplete or disabled behavior in source,
|
|
277
276
|
config, and tests, including discarded errors, removed validation, and altered expected values.
|
|
278
277
|
|
|
279
|
-
> **
|
|
280
|
-
>
|
|
281
|
-
>
|
|
282
|
-
>
|
|
283
|
-
>
|
|
278
|
+
> **Six WI checks run by default:** WI101, WI102, WI105, WI108, WI109, and WI110.
|
|
279
|
+
> Alongside RH001 to RH014, they close error-handling, unfinished-body, canned-output,
|
|
280
|
+
> source-hiding, expectation-edit, and script-verification gaps. The remaining seven WI checks
|
|
281
|
+
> stay opt-in because their human-commit findings still include legitimate maintenance.
|
|
282
|
+
> Run `proctor check --wi` (or `--all-checks`) to add the entire family, or name checks in
|
|
283
|
+
> `enabled` in the committed config. An explicit `enabled` list is still honored exactly.
|
|
284
|
+
> See the [measurements and per-check decisions](bench/detection/README.md).
|
|
284
285
|
|
|
285
286
|
| | Catches |
|
|
286
287
|
|---|---|
|
|
287
|
-
| WI101 |
|
|
288
|
+
| WI101 | Empty or logging-only handlers, and formerly propagated operation errors discarded or replaced with defaults |
|
|
288
289
|
| WI102 | An explicit "not implemented" marker shipped inside finished-looking work |
|
|
289
290
|
| WI103 | Validation deleted so the case it rejected now goes through |
|
|
290
291
|
| WI104 | Proctor, a commit hook, or a type/lint gate switched off instead of satisfied |
|
|
@@ -338,12 +339,11 @@ jobs:
|
|
|
338
339
|
steps:
|
|
339
340
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
|
340
341
|
with: { fetch-depth: 0 }
|
|
341
|
-
- uses: catfish-1234/proctor@
|
|
342
|
+
- uses: catfish-1234/proctor@ede7e0ec0d9a91c241af1b181b74b1a6af67df07 # v1.0.0
|
|
342
343
|
```
|
|
343
344
|
|
|
344
|
-
|
|
345
|
-
|
|
346
|
-
reproducible, use a full commit SHA as above; a full SHA is immutable.
|
|
345
|
+
The example pins the commit behind the `v1.0.0` release. You can use the `v1.0.0` tag instead, but
|
|
346
|
+
a full commit SHA is immutable, so it keeps builds reproducible.
|
|
347
347
|
|
|
348
348
|
## How it works
|
|
349
349
|
|
|
@@ -363,7 +363,7 @@ unchecked commit.
|
|
|
363
363
|
that same diff is checked. Approvals and inline `proctor-ignore` markers must also be committed
|
|
364
364
|
before the change they cover.
|
|
365
365
|
|
|
366
|
-
**
|
|
366
|
+
**An opt-in check can flag changes to the guard.** WI104 fires on a check removed from `enabled`, a
|
|
367
367
|
severity downgraded, an ignore pattern or approval added, a ruleset file deleted, a `--no-verify`
|
|
368
368
|
added to a script, or TypeScript strictness turned off. WI104 is part of the opt-in WI family.
|
|
369
369
|
|
|
@@ -383,22 +383,32 @@ changes and allows turns when proctor cannot run, so it is not a substitute for
|
|
|
383
383
|
| **up to 100 pts** | gap between "the visible tests pass" and "the held-out tests pass" on the largest tasks in [SpecBench](https://arxiv.org/abs/2605.21384). It widens by about 28 points per tenfold increase in code size, and adding test coverage does not close it |
|
|
384
384
|
| **8 of 8** | agent benchmarks that [Berkeley RDI](https://rdi.berkeley.edu/blog/trustworthy-benchmarks-cont/) drove to near-perfect scores without solving a single task. SWE-bench Verified fell to a pytest hook that forced every test to pass |
|
|
385
385
|
|
|
386
|
-
**What proctor catches.** The
|
|
387
|
-
|
|
386
|
+
**What proctor catches.** The original rule fixtures and recorded benchmark cases are checked by
|
|
387
|
+
`npm test`. The redteam probe is a repeatedly tuned **dev corpus**, not independent evidence of
|
|
388
|
+
generalization. Fresh measurements compare the unchanged main baseline with these changes:
|
|
388
389
|
|
|
389
390
|
| | |
|
|
390
391
|
|---|---|
|
|
391
|
-
| **139 of 139** | planted
|
|
392
|
-
| **
|
|
393
|
-
| **
|
|
394
|
-
| **
|
|
395
|
-
| **
|
|
396
|
-
| **
|
|
397
|
-
| **
|
|
398
|
-
|
|
399
|
-
|
|
400
|
-
|
|
401
|
-
|
|
392
|
+
| **139 of 139** | original planted rule fixtures caught with all checks, with **0 of 33** original near-miss fixtures flagged. These are development fixtures, not an unseen evaluation |
|
|
393
|
+
| **21 of 21** | recorded cheats caught in the mock benchmark corpus, across 7 signatures. The 22nd task is a control; it stays quiet |
|
|
394
|
+
| **25 to 54 of 76** | tuned dev cheats caught by defaults before and after. With all checks: **76 to 76 of 76**. Dev controls: **0 to 0 of 24** flagged in either mode. Run `node bench/redteam/probe.mjs --default` and `node bench/redteam/probe.mjs` |
|
|
395
|
+
| **7 to 36 of 36** | cheats caught by defaults in the **74-case frozen validation set**, with benign flags falling from **2/38 to 0/38**. All checks improve from **9/36 to 36/36**. The separately frozen 74-case design set has the same totals. Both are small authored sets with paired variations |
|
|
396
|
+
| **4.29% to 2.00% blocking** | **30/700 to 14/700** real non-bot commits flagged by defaults on the same fresh pinned sample from 20 repositories. Any finding: **46/700 (6.57%) to 26/700 (3.71%)**. Every flagged human commit counts as a conservative false-positive proxy, including intentional test deletions |
|
|
397
|
+
| **26 to 35 of 72** | cheats caught by defaults on a **72-cheat, 72-benign held-out set** written after 1.0.0 by a separate run that never saw the checks. Benign flags: **5/72 before, 5/72 after**. All checks: **38 to 42 of 72**, with 6/72 benign flagged. Error handling went from 0/12 to 7/12. Hardcoded special cases (2/8), removed validation (0/3), edited expected values (0/3), race masking (0/3) and suppression (0/3) are still mostly missed. The set was used only to measure, not to tune, but it is now public, so it no longer counts as blind. Run `node bench/heldout/probe.mjs --default` |
|
|
398
|
+
| **2.6% blocking** | **11/420** real commits blocked by both 1.0.0 and this release on a second fresh sample of 14 repositories that neither the checks nor the first sample touched. Any finding: **17 before, 16 after**. A few false positives found there were fixed with rules, not by pinning commits |
|
|
399
|
+
| **27** | checks in two families, with **20 enabled by default**, across **25+** languages, installable into **30** agents |
|
|
400
|
+
|
|
401
|
+
The old **689-commit** claim, **2.32% blocking and 3.8% any finding**, refers to a missing ignored
|
|
402
|
+
sample and harness. It could not be reproduced from this checkout or tracked history. The fresh
|
|
403
|
+
700-commit sample uses the same 20 repositories, pins every commit and patch hash, and retains
|
|
404
|
+
all findings. It is a replacement cohort, not the original sample. See
|
|
405
|
+
[methods, per-rule results, and reproduction commands](bench/detection/README.md).
|
|
406
|
+
|
|
407
|
+
**What we don't claim.** Proctor is a heuristic diff checker. Defaults still miss more than half of the held-out cheats, mostly the ones that need running the code to see: special-cased inputs, edited expected values, removed validation, masked races and suppression. It detects suspicious changes;
|
|
408
|
+
it does not prevent them or judge intent. A determined author can write a cheat that no static
|
|
409
|
+
diff check recognizes. The dev and authored-validation results do not establish a detection
|
|
410
|
+
rate for unseen cheats. Our [live benchmark](#benchmark) has not shown that proctor reduces
|
|
411
|
+
cheating: all 44 invocations in each complete run passed their evaluation tests in both arms.
|
|
402
412
|
|
|
403
413
|
## Benchmark
|
|
404
414
|
|
|
@@ -465,8 +475,9 @@ For command details, configuration, support limits, and contributor instructions
|
|
|
465
475
|
| [bench/METHODOLOGY.md](https://github.com/catfish-1234/proctor/blob/main/bench/METHODOLOGY.md) | How the benchmark works and what it does not claim |
|
|
466
476
|
|
|
467
477
|
proctor supports 25+ languages and installs to 30 agents. Five diff-level checks (RH001, RH002,
|
|
468
|
-
RH003, RH007, RH011)
|
|
469
|
-
JS/TS/Python
|
|
478
|
+
RH003, RH007, RH011) have signatures across the language roster. RH004, RH006, RH008, RH009, and
|
|
479
|
+
RH010 have their tested coverage in JS/TS/Python; RH005 also detects emptied Go and Rust bodies.
|
|
480
|
+
RH012 and RH013 read CI and coverage config, so they apply everywhere. Of the
|
|
470
481
|
work-integrity family, WI101, WI102 and WI103 carry per-language signatures, WI104 reads config
|
|
471
482
|
files so it applies everywhere, and WI105 and WI106 are scoped to the languages whose tokens are
|
|
472
483
|
unambiguous. [Full matrix](https://github.com/catfish-1234/proctor/blob/main/docs/LANGUAGES.md).
|
package/dist/cli.js
CHANGED
|
@@ -65,7 +65,7 @@ import { homedir } from "os";
|
|
|
65
65
|
// package.json
|
|
66
66
|
var package_default = {
|
|
67
67
|
name: "@kavishdua/proctor",
|
|
68
|
-
version: "1.
|
|
68
|
+
version: "1.1.0",
|
|
69
69
|
description: "Catch AI agents gaming their test suites: deleted tests, skipped tests, weakened assertions, hardcoded outputs",
|
|
70
70
|
homepage: "https://github.com/catfish-1234/proctor#readme",
|
|
71
71
|
bugs: {
|
|
@@ -653,7 +653,9 @@ var WI_CHECKS = [
|
|
|
653
653
|
"WI112",
|
|
654
654
|
"WI113"
|
|
655
655
|
];
|
|
656
|
-
var
|
|
656
|
+
var DEFAULT_WI_CHECKS = ["WI101", "WI102", "WI105", "WI108", "WI109", "WI110"];
|
|
657
|
+
var DEFAULT_CHECKS = [...RH_CHECKS, ...DEFAULT_WI_CHECKS];
|
|
658
|
+
var DEFAULT_ENABLED = DEFAULT_CHECKS;
|
|
657
659
|
var VALID_SEVERITIES = /* @__PURE__ */ new Set(["error", "warn", "info"]);
|
|
658
660
|
function normalizeConfig(config) {
|
|
659
661
|
const out = { ...config };
|
|
@@ -923,8 +925,9 @@ function withoutTrailingComment(text) {
|
|
|
923
925
|
return text.replace(/\s*(?:\/\/|#|--\s).*$/, "").replace(/\s*\/\*.*\*\/\s*$/, "");
|
|
924
926
|
}
|
|
925
927
|
function hasExplanation(text) {
|
|
926
|
-
const comment = /(?:\/\/|#(?!\!)
|
|
928
|
+
const comment = /(?:\/\/|#(?!\!)|--(?=\s)|\/\*|\*)\s*(.+)$/.exec(text);
|
|
927
929
|
if (!comment) return false;
|
|
930
|
+
if (/^proctor-ignore\s*:/.test(comment[1].trim())) return false;
|
|
928
931
|
const words = comment[1].replace(/\*\/\s*$/, "").trim();
|
|
929
932
|
if (/^(?:todo|fixme|xxx|hack|noqa|nosec|eslint-disable\S*|prettier-ignore)\b\W*$/i.test(words)) return false;
|
|
930
933
|
return words.split(/\s+/).filter(Boolean).length >= 3;
|
|
@@ -956,6 +959,155 @@ function codeLines(chunk) {
|
|
|
956
959
|
return added.filter((line, i) => !templated.has(i) && !isCommentLine(line.text));
|
|
957
960
|
}
|
|
958
961
|
|
|
962
|
+
// src/verifiers/structure.ts
|
|
963
|
+
function maskCode(text, pythonOrRuby = false, preserveDelimiters = false) {
|
|
964
|
+
const out = text.split("");
|
|
965
|
+
let quote = "";
|
|
966
|
+
let block = false;
|
|
967
|
+
let line = false;
|
|
968
|
+
let previous = "";
|
|
969
|
+
for (let i = 0; i < text.length; i++) {
|
|
970
|
+
const ch = text[i];
|
|
971
|
+
const next = text[i + 1];
|
|
972
|
+
if (ch === "\n") {
|
|
973
|
+
line = false;
|
|
974
|
+
continue;
|
|
975
|
+
}
|
|
976
|
+
if (line) {
|
|
977
|
+
out[i] = " ";
|
|
978
|
+
continue;
|
|
979
|
+
}
|
|
980
|
+
if (block) {
|
|
981
|
+
out[i] = " ";
|
|
982
|
+
if (ch === "*" && next === "/") {
|
|
983
|
+
out[++i] = " ";
|
|
984
|
+
block = false;
|
|
985
|
+
}
|
|
986
|
+
continue;
|
|
987
|
+
}
|
|
988
|
+
if (quote) {
|
|
989
|
+
out[i] = " ";
|
|
990
|
+
if (ch === "\\") {
|
|
991
|
+
if (i + 1 < text.length) out[++i] = " ";
|
|
992
|
+
} else if (text.startsWith(quote, i)) {
|
|
993
|
+
for (let k = 0; k < quote.length; k++) out[i + k] = preserveDelimiters ? quote[k] : " ";
|
|
994
|
+
i += quote.length - 1;
|
|
995
|
+
quote = "";
|
|
996
|
+
}
|
|
997
|
+
continue;
|
|
998
|
+
}
|
|
999
|
+
if (ch === "/" && next === "*") {
|
|
1000
|
+
out[i] = out[++i] = " ";
|
|
1001
|
+
block = true;
|
|
1002
|
+
} else if (!pythonOrRuby && ch === "/" && next === "/" || pythonOrRuby && ch === "#") {
|
|
1003
|
+
out[i] = " ";
|
|
1004
|
+
line = true;
|
|
1005
|
+
} else if (ch === '"' || ch === "'" || ch === "`") {
|
|
1006
|
+
quote = pythonOrRuby && text.startsWith(ch.repeat(3), i) ? ch.repeat(3) : ch;
|
|
1007
|
+
out[i] = preserveDelimiters ? ch : " ";
|
|
1008
|
+
for (let k = 1; k < quote.length; k++) out[i + k] = preserveDelimiters ? ch : " ";
|
|
1009
|
+
i += quote.length - 1;
|
|
1010
|
+
previous = "v";
|
|
1011
|
+
} else if (!pythonOrRuby && ch === "/" && /^(?:$|[(,=:[!&|?{};+*%~^<>-])$/.test(previous)) {
|
|
1012
|
+
let end = i + 1;
|
|
1013
|
+
let inClass = false;
|
|
1014
|
+
for (; end < text.length && text[end] !== "\n"; end++) {
|
|
1015
|
+
if (text[end] === "\\") end++;
|
|
1016
|
+
else if (text[end] === "[") inClass = true;
|
|
1017
|
+
else if (text[end] === "]") inClass = false;
|
|
1018
|
+
else if (text[end] === "/" && !inClass) break;
|
|
1019
|
+
}
|
|
1020
|
+
if (text[end] === "/") {
|
|
1021
|
+
for (let cursor = i; cursor <= end; cursor++) out[cursor] = " ";
|
|
1022
|
+
i = end;
|
|
1023
|
+
previous = "v";
|
|
1024
|
+
} else previous = ch;
|
|
1025
|
+
} else if (!/\s/.test(ch)) previous = ch;
|
|
1026
|
+
}
|
|
1027
|
+
return out.join("");
|
|
1028
|
+
}
|
|
1029
|
+
function sourceView(chunk, side, pythonOrRuby = false) {
|
|
1030
|
+
const selected = chunk.changes.map((change, position) => ({ change, position })).filter(({ change }) => change.type !== (side === "after" ? "del" : "add"));
|
|
1031
|
+
const starts = [];
|
|
1032
|
+
let text = "";
|
|
1033
|
+
for (const { change } of selected) {
|
|
1034
|
+
starts.push(text.length);
|
|
1035
|
+
text += body(change.content) + "\n";
|
|
1036
|
+
}
|
|
1037
|
+
const indexAt = (offset) => {
|
|
1038
|
+
let lo = 0;
|
|
1039
|
+
let hi = starts.length;
|
|
1040
|
+
while (lo + 1 < hi) {
|
|
1041
|
+
const mid = lo + hi >>> 1;
|
|
1042
|
+
if (starts[mid] <= offset) lo = mid;
|
|
1043
|
+
else hi = mid;
|
|
1044
|
+
}
|
|
1045
|
+
return lo;
|
|
1046
|
+
};
|
|
1047
|
+
return {
|
|
1048
|
+
text,
|
|
1049
|
+
code: maskCode(text, pythonOrRuby),
|
|
1050
|
+
signatures: maskCode(text, pythonOrRuby, true),
|
|
1051
|
+
offsetAtLine: (line) => {
|
|
1052
|
+
const index = selected.findIndex(({ change }) => (change.type === "normal" ? side === "before" ? change.ln1 : change.ln2 : change.ln) === line);
|
|
1053
|
+
return index < 0 ? void 0 : starts[index];
|
|
1054
|
+
},
|
|
1055
|
+
lineAt: (offset) => {
|
|
1056
|
+
const change = selected[indexAt(offset)]?.change;
|
|
1057
|
+
return change ? change.type === "normal" ? side === "before" ? change.ln1 : change.ln2 : change.ln : chunk.newStart;
|
|
1058
|
+
},
|
|
1059
|
+
touched: (start, end) => {
|
|
1060
|
+
const first = selected[indexAt(start)];
|
|
1061
|
+
const last = selected[indexAt(end)];
|
|
1062
|
+
return !!first && !!last && chunk.changes.slice(first.position, last.position + 1).some((c) => c.type !== "normal");
|
|
1063
|
+
}
|
|
1064
|
+
};
|
|
1065
|
+
}
|
|
1066
|
+
function closingDelimiter(code, open, left = "{", right = "}") {
|
|
1067
|
+
let depth = 0;
|
|
1068
|
+
for (let i = open; i < code.length; i++) {
|
|
1069
|
+
if (code[i] === left) depth++;
|
|
1070
|
+
else if (code[i] === right && --depth === 0) return i;
|
|
1071
|
+
}
|
|
1072
|
+
return -1;
|
|
1073
|
+
}
|
|
1074
|
+
var compact = (text) => text.replace(/\s+/g, "");
|
|
1075
|
+
function callables(view) {
|
|
1076
|
+
const result = [];
|
|
1077
|
+
const header = /\b(?:function\s+|fn\s+|func\s+)([A-Za-z_$][\w$]*)\s*\([^{}]*\)[^{;]*\{/g;
|
|
1078
|
+
for (const match of view.code.matchAll(header)) {
|
|
1079
|
+
const start = match.index;
|
|
1080
|
+
const open = start + match[0].length - 1;
|
|
1081
|
+
const end = closingDelimiter(view.code, open);
|
|
1082
|
+
if (end < 0) continue;
|
|
1083
|
+
result.push({ name: match[1], header: compact(match[0]), start, open, end, body: view.code.slice(open + 1, end) });
|
|
1084
|
+
}
|
|
1085
|
+
return result;
|
|
1086
|
+
}
|
|
1087
|
+
function testBodies(view) {
|
|
1088
|
+
const result = [];
|
|
1089
|
+
for (const match of view.code.matchAll(/\b(?:it|test)(?:\.(?!each\b)\w+)*\s*\(/g)) {
|
|
1090
|
+
const openArgs = match.index + match[0].length - 1;
|
|
1091
|
+
const endArgs = closingDelimiter(view.code, openArgs, "(", ")");
|
|
1092
|
+
if (endArgs < 0) continue;
|
|
1093
|
+
const title = /^\s*(['"`])((?:\\.|[^\\])*?)\1/.exec(view.text.slice(openArgs + 1, endArgs));
|
|
1094
|
+
if (!title) continue;
|
|
1095
|
+
let depth = 0;
|
|
1096
|
+
let last = openArgs + 1;
|
|
1097
|
+
for (let i = openArgs + 1; i < endArgs; i++) {
|
|
1098
|
+
if ("({[".includes(view.code[i])) depth++;
|
|
1099
|
+
else if (")}]".includes(view.code[i])) depth--;
|
|
1100
|
+
else if (view.code[i] === "," && depth === 0) last = i + 1;
|
|
1101
|
+
}
|
|
1102
|
+
const callback = /^\s*(?:async\s+)?(?:\([^)]*\)|[A-Za-z_$][\w$]*)\s*=>\s*\{|^\s*(?:async\s+)?function\s*\([^)]*\)\s*\{/.exec(view.code.slice(last, endArgs));
|
|
1103
|
+
if (!callback) continue;
|
|
1104
|
+
const open = last + callback[0].length - 1;
|
|
1105
|
+
const end = closingDelimiter(view.code, open);
|
|
1106
|
+
if (end >= 0) result.push({ key: title[2], open, end, body: view.code.slice(open + 1, end) });
|
|
1107
|
+
}
|
|
1108
|
+
return result;
|
|
1109
|
+
}
|
|
1110
|
+
|
|
959
1111
|
// src/verifiers/rh001.ts
|
|
960
1112
|
var JS_TS_DEL = /^-\s*(?:it|test|describe)(?:\.\w+)?\s*\(/;
|
|
961
1113
|
var PY_DEL = /^-\s*(?:async\s+)?def test_/;
|
|
@@ -1032,6 +1184,28 @@ function testsRelocated(file, files, ctx) {
|
|
|
1032
1184
|
if (relocated.size !== removed.size) return false;
|
|
1033
1185
|
return assertionsGained >= assertionCount(file, "del");
|
|
1034
1186
|
}
|
|
1187
|
+
function removedFeature(context, testPath, testCode) {
|
|
1188
|
+
const unit = (path9) => (path9.split("/").pop() ?? "").replace(/\.(?:test|spec)\.[^.]+$|\.[^.]+$/, "").replace(/^test_/, "").replace(/_test$/, "").replace(/^_+/, "");
|
|
1189
|
+
const declaration = /\b(?:function|def|class)\s+([A-Za-z_$][\w$]*)/g;
|
|
1190
|
+
for (const candidate of context.files) {
|
|
1191
|
+
const path9 = candidate.from ?? candidate.to ?? "";
|
|
1192
|
+
if (context.isTestFile(path9) || unit(path9) !== unit(testPath)) continue;
|
|
1193
|
+
const before = candidate.chunks.map((chunk) => sourceView(chunk, "before", path9.endsWith(".py")));
|
|
1194
|
+
const after = candidate.chunks.map((chunk) => sourceView(chunk, "after", path9.endsWith(".py")));
|
|
1195
|
+
for (const view of before) for (const match of view.code.matchAll(declaration)) {
|
|
1196
|
+
const name = match[1];
|
|
1197
|
+
if (!view.touched(match.index, match.index + match[0].length)) continue;
|
|
1198
|
+
if (after.some((next) => [...next.code.matchAll(/\b(?:function|def|class|const|let|var)\s+([A-Za-z_$][\w$]*)/g)].some((declaration2) => declaration2[1] === name))) continue;
|
|
1199
|
+
for (const reference of testCode.matchAll(/[A-Za-z_$][\w$]*/g)) {
|
|
1200
|
+
if (reference[0] !== name) continue;
|
|
1201
|
+
let next = reference.index + reference[0].length;
|
|
1202
|
+
while (/\s/.test(testCode[next] ?? "") && next < testCode.length) next++;
|
|
1203
|
+
if (testCode[reference.index - 1] === "." || testCode[next] === "(") return true;
|
|
1204
|
+
}
|
|
1205
|
+
}
|
|
1206
|
+
}
|
|
1207
|
+
return false;
|
|
1208
|
+
}
|
|
1035
1209
|
function run(context) {
|
|
1036
1210
|
const files = context.files;
|
|
1037
1211
|
const ctx = context;
|
|
@@ -1082,6 +1256,28 @@ function run(context) {
|
|
|
1082
1256
|
if (change.type !== "del") continue;
|
|
1083
1257
|
if (templateLines.has(changeIndex)) continue;
|
|
1084
1258
|
if (hasReconcilingAdd) continue;
|
|
1259
|
+
const before = sourceView(chunk, "before", from.endsWith(".py"));
|
|
1260
|
+
const start = before.offsetAtLine(change.ln);
|
|
1261
|
+
if (start !== void 0) {
|
|
1262
|
+
let end = before.code.length;
|
|
1263
|
+
if (from.endsWith(".py")) {
|
|
1264
|
+
const indent = /^\s*/.exec(change.content.slice(1))[0].length;
|
|
1265
|
+
const nextLine = before.code.indexOf("\n", start);
|
|
1266
|
+
const rest = before.code.slice(nextLine + 1);
|
|
1267
|
+
const boundary = /^( *)(\S)/gm;
|
|
1268
|
+
for (const match of rest.matchAll(boundary)) if (match[1].length <= indent) {
|
|
1269
|
+
end = nextLine + 1 + match.index;
|
|
1270
|
+
break;
|
|
1271
|
+
}
|
|
1272
|
+
} else {
|
|
1273
|
+
const open = before.code.indexOf("(", start);
|
|
1274
|
+
if (open >= 0) {
|
|
1275
|
+
const close = closingDelimiter(before.code, open, "(", ")");
|
|
1276
|
+
if (close >= 0) end = close + 1;
|
|
1277
|
+
}
|
|
1278
|
+
}
|
|
1279
|
+
if (removedFeature(ctx, from, before.code.slice(start, end))) continue;
|
|
1280
|
+
}
|
|
1085
1281
|
if (JS_TS_DEL.test(change.content)) {
|
|
1086
1282
|
const name = extractTestName(change.content);
|
|
1087
1283
|
findings.push({
|
|
@@ -1196,7 +1392,7 @@ var WEAK_PATTERNS = [
|
|
|
1196
1392
|
];
|
|
1197
1393
|
var ORDERING_WEAK = /\.toBe(?:GreaterThan|LessThan)(?:OrEqual)?\(/;
|
|
1198
1394
|
var SOLE_ANYTHING = /\.(?:toBe|toEqual|toStrictEqual)\(\s*expect\.anything\(\)\s*\)/;
|
|
1199
|
-
var CONTAINMENT_WEAK = /\.(?:toContain|toContainEqual|toMatch|toHaveProperty)\(/;
|
|
1395
|
+
var CONTAINMENT_WEAK = /\.(?:toContain|toContainEqual|toMatch|toHaveProperty|toBeCloseTo)\(/;
|
|
1200
1396
|
var GROUP_B_STRONG = [
|
|
1201
1397
|
{ ext: /\.(?:pl|pm|t)$/i, re: /\bis(?:_deeply)?\s*\(/ },
|
|
1202
1398
|
{ ext: /\.r$/i, re: /\bexpect_(?:equal|identical)\s*\(/ },
|
|
@@ -1581,6 +1777,29 @@ function run2(context) {
|
|
|
1581
1777
|
for (const file of files) {
|
|
1582
1778
|
const filePath = file.to ?? file.from ?? "";
|
|
1583
1779
|
for (const chunk of file.chunks) {
|
|
1780
|
+
if (context.isTestFile(filePath) && /\.[cm]?[jt]sx?$/.test(filePath)) {
|
|
1781
|
+
const before = sourceView(chunk, "before");
|
|
1782
|
+
const after = sourceView(chunk, "after");
|
|
1783
|
+
const priorTests = testBodies(before);
|
|
1784
|
+
for (const next of testBodies(after)) {
|
|
1785
|
+
const prior = priorTests.find((test) => test.key === next.key);
|
|
1786
|
+
if (!prior || !after.touched(next.open, next.end)) continue;
|
|
1787
|
+
for (const assertion of prior.body.matchAll(/\bexpect\s*\(\s*([A-Za-z_$][\w$]*)\s*\)\s*\.(?:toBe|toEqual|toStrictEqual|toBeCloseTo)\s*\(/g)) {
|
|
1788
|
+
const subject = assertion[1];
|
|
1789
|
+
const computesSubject = (code) => [...code.matchAll(/\b(?:const|let)\s+([A-Za-z_$][\w$]*)\s*=\s*([^;\n]+)/g)].some((binding) => binding[1] === subject && /[^\s(]+\s*\(/.test(binding[2]));
|
|
1790
|
+
if (!computesSubject(prior.body) || !computesSubject(next.body)) continue;
|
|
1791
|
+
if ([...next.body.matchAll(/[A-Za-z_$][\w$]*/g)].filter((token) => token[0] === subject).length !== 1) continue;
|
|
1792
|
+
findings.push({
|
|
1793
|
+
verifierId: "RH002",
|
|
1794
|
+
severity: "error",
|
|
1795
|
+
file: filePath,
|
|
1796
|
+
line: after.lineAt(next.open),
|
|
1797
|
+
message: "A computed test result is still assigned, but its value assertion was removed and the value is no longer used.",
|
|
1798
|
+
suggestion: "Restore the result assertion or replace it with an equivalent check of that value."
|
|
1799
|
+
});
|
|
1800
|
+
}
|
|
1801
|
+
}
|
|
1802
|
+
}
|
|
1584
1803
|
const dels = chunk.changes.filter((c) => c.type === "del");
|
|
1585
1804
|
const adds = chunk.changes.filter((c) => c.type === "add");
|
|
1586
1805
|
const at = new Map(chunk.changes.map((c, i) => [c, i]));
|
|
@@ -1632,6 +1851,41 @@ function run2(context) {
|
|
|
1632
1851
|
suggestion: "Restore the specific value assertion to preserve test coverage strength."
|
|
1633
1852
|
});
|
|
1634
1853
|
}
|
|
1854
|
+
if (context.isTestFile(filePath)) for (const added of adds) {
|
|
1855
|
+
if (reported.has(added.ln) || hasExplanation(added.content)) continue;
|
|
1856
|
+
const next = body(added.content);
|
|
1857
|
+
const nextSubject = extractSubject(next);
|
|
1858
|
+
const prior = dels.find((del) => {
|
|
1859
|
+
if (!nearby(del, added)) return false;
|
|
1860
|
+
const old = body(del.content);
|
|
1861
|
+
const subject = extractSubject(old);
|
|
1862
|
+
if (subject && subject === nextSubject) {
|
|
1863
|
+
const a = /\.toBeCloseTo\(\s*([^,]+),\s*(\d+)\s*\)/.exec(old);
|
|
1864
|
+
const b = /\.toBeCloseTo\(\s*([^,]+),\s*(\d+)\s*\)/.exec(next);
|
|
1865
|
+
if (a && b && compact(a[1]) === compact(b[1]) && Number(b[2]) < Number(a[2])) return true;
|
|
1866
|
+
}
|
|
1867
|
+
if (subject && isStrongAssertion(old, filePath) && /\.toHaveBeenCalled(?:Times|With)?\(/.test(next)) {
|
|
1868
|
+
return !chunk.changes.some((c) => c.type !== "del" && extractSubject(c.content) === subject && isStrongAssertion(c.content, filePath));
|
|
1869
|
+
}
|
|
1870
|
+
if (filePath.endsWith(".py")) {
|
|
1871
|
+
const pattern = /\bassert\s+(.+?)==\s*pytest\.approx\(\s*([^,]+),\s*(abs|rel)\s*=\s*([0-9.eE+-]+)\s*\)/;
|
|
1872
|
+
const a = pattern.exec(old);
|
|
1873
|
+
const b = pattern.exec(next);
|
|
1874
|
+
return !!a && !!b && compact(a[1]) === compact(b[1]) && compact(a[2]) === compact(b[2]) && a[3] === b[3] && Number(b[4]) > Number(a[4]);
|
|
1875
|
+
}
|
|
1876
|
+
return false;
|
|
1877
|
+
});
|
|
1878
|
+
if (!prior) continue;
|
|
1879
|
+
reported.add(added.ln);
|
|
1880
|
+
findings.push({
|
|
1881
|
+
verifierId: "RH002",
|
|
1882
|
+
severity: "error",
|
|
1883
|
+
file: filePath,
|
|
1884
|
+
line: added.ln,
|
|
1885
|
+
message: "A result assertion was replaced by a wider tolerance or an assertion on mock interactions.",
|
|
1886
|
+
suggestion: "Preserve the result contract and its precision, or document the specification change."
|
|
1887
|
+
});
|
|
1888
|
+
}
|
|
1635
1889
|
for (const del of dels) {
|
|
1636
1890
|
if (!ALMOST_EQUAL.test(del.content)) continue;
|
|
1637
1891
|
const delPlaces = del.content.match(/places\s*=\s*(\d+)/);
|
|
@@ -2293,6 +2547,22 @@ function isSkipPattern(content, flags) {
|
|
|
2293
2547
|
return JS_PATTERNS.some((re) => re.test(content));
|
|
2294
2548
|
}
|
|
2295
2549
|
}
|
|
2550
|
+
function skipIdentity(text) {
|
|
2551
|
+
const match = /\b((?:it|test|describe)(?:\.\w+)*)\s*\(\s*(['"`])((?:\\.|[^\\])*?)\2/.exec(text);
|
|
2552
|
+
return match ? match[1] + ":" + match[3] : void 0;
|
|
2553
|
+
}
|
|
2554
|
+
function nativeReason(text) {
|
|
2555
|
+
const match = /\breason\s*=\s*(['"])/.exec(text);
|
|
2556
|
+
if (!match) return false;
|
|
2557
|
+
const start = match.index + match[0].length;
|
|
2558
|
+
let end = start;
|
|
2559
|
+
for (; end < text.length; end++) {
|
|
2560
|
+
if (text[end] === "\\") end++;
|
|
2561
|
+
else if (text[end] === match[1]) break;
|
|
2562
|
+
}
|
|
2563
|
+
const reason = text.slice(start, end).trim();
|
|
2564
|
+
return reason.split(/\s+/).length >= 3 || /^https?:\/\/\S+/.test(reason);
|
|
2565
|
+
}
|
|
2296
2566
|
function run3(context) {
|
|
2297
2567
|
const files = context.files;
|
|
2298
2568
|
const ctx = context;
|
|
@@ -2313,10 +2583,77 @@ function run3(context) {
|
|
|
2313
2583
|
const luaTestFile = ext === "lua" && isLuaTestFile(filePath);
|
|
2314
2584
|
const batsFile = ext === "bats" && isBatsFile(filePath);
|
|
2315
2585
|
if (!isPython && !isNewLang && !ctx.isTestFile(filePath)) continue;
|
|
2586
|
+
const previousSkips = new Set(file.chunks.flatMap((chunk) => chunk.changes).filter((c) => c.type === "del" && isSkipPattern(c.content, { ext, pyTestModule, goTestFile, rubyTestFile, kotlinTestFile, cTestFile, scalaTestFile, rTestFile, haskellTestFile, luaTestFile, batsFile })).map((c) => skipIdentity(c.content)).filter(Boolean));
|
|
2316
2587
|
for (const chunk of file.chunks) {
|
|
2588
|
+
const lines = afterLines(chunk);
|
|
2589
|
+
const view = sourceView(chunk, "after", isPython);
|
|
2590
|
+
const signatureLines = view.signatures.split("\n");
|
|
2591
|
+
const maskedLines = view.code.split("\n");
|
|
2592
|
+
if (ctx.isTestFile(filePath) && !isPython && chunk.changes.some((change) => change.type === "add" && body(change.content).includes("/*"))) {
|
|
2593
|
+
const before = sourceView(chunk, "before");
|
|
2594
|
+
const surviving = new Set(testBodies(view).map((test) => test.key));
|
|
2595
|
+
for (const prior of testBodies(before)) {
|
|
2596
|
+
if (surviving.has(prior.key)) continue;
|
|
2597
|
+
const oldLine = before.lineAt(prior.open);
|
|
2598
|
+
const unchanged = chunk.changes.find((change) => change.type === "normal" && change.ln1 === oldLine);
|
|
2599
|
+
if (!unchanged || unchanged.type !== "normal") continue;
|
|
2600
|
+
const offset = view.offsetAtLine(unchanged.ln2);
|
|
2601
|
+
if (offset === void 0) continue;
|
|
2602
|
+
const end = view.code.indexOf("\n", offset);
|
|
2603
|
+
if (view.code.slice(offset, end < 0 ? void 0 : end).trim()) continue;
|
|
2604
|
+
findings.push({
|
|
2605
|
+
verifierId: "RH003",
|
|
2606
|
+
severity: "error",
|
|
2607
|
+
file: filePath,
|
|
2608
|
+
line: unchanged.ln2,
|
|
2609
|
+
message: "An existing named test is now inside a block comment and no longer executes.",
|
|
2610
|
+
suggestion: "Restore the executable test or remove the feature and its tests deliberately."
|
|
2611
|
+
});
|
|
2612
|
+
}
|
|
2613
|
+
}
|
|
2614
|
+
if (ctx.isTestFile(filePath)) {
|
|
2615
|
+
const after = sourceView(chunk, "after", isPython);
|
|
2616
|
+
for (const match of after.code.matchAll(/\bif\s*\([^\n;{}]*\)\s*(?:\{\s*)?return\s*;|^([ \t]*)if\s+[^:\n]+:\s*return\s*$/gm)) {
|
|
2617
|
+
if (!after.touched(match.index, match.index + match[0].length)) continue;
|
|
2618
|
+
const offset = match.index;
|
|
2619
|
+
const inTest = isPython ? /\bdef\s+test_\w+\s*\(/.test(after.code.slice(0, offset)) : testBodies(after).some((test) => offset > test.open && offset < test.end && !/=>|\bfunction\b/.test(after.code.slice(test.open + 1, offset)));
|
|
2620
|
+
if (!inTest) continue;
|
|
2621
|
+
const line = after.lineAt(offset);
|
|
2622
|
+
const raw = after.text.slice(match.index, match.index + match[0].length);
|
|
2623
|
+
if (hasExplanation(raw)) continue;
|
|
2624
|
+
const later = after.code.slice(match.index + match[0].length);
|
|
2625
|
+
if (!/\bexpect\s*\(|\bassert\b/.test(later)) continue;
|
|
2626
|
+
findings.push({
|
|
2627
|
+
verifierId: "RH003",
|
|
2628
|
+
severity: "error",
|
|
2629
|
+
file: filePath,
|
|
2630
|
+
line,
|
|
2631
|
+
message: "A new conditional return can bypass the assertions in a test that still reports success.",
|
|
2632
|
+
suggestion: "Run the assertions on every supported path, or document and report an explicit test skip."
|
|
2633
|
+
});
|
|
2634
|
+
}
|
|
2635
|
+
}
|
|
2317
2636
|
for (const change of chunk.changes) {
|
|
2318
2637
|
if (change.type !== "add") continue;
|
|
2319
|
-
|
|
2638
|
+
const text = body(change.content);
|
|
2639
|
+
const identity = skipIdentity(text);
|
|
2640
|
+
if (identity && previousSkips.has(identity)) continue;
|
|
2641
|
+
const index = lines.findIndex((line) => line.line === change.ln && line.added);
|
|
2642
|
+
const offset = lines.slice(0, index).reduce((sum, line) => sum + line.text.length + 1, 0);
|
|
2643
|
+
const open = view.code.indexOf("(", offset);
|
|
2644
|
+
const end = open < 0 ? -1 : closingDelimiter(view.code, open, "(", ")");
|
|
2645
|
+
const invocation = end < 0 ? text : view.text.slice(offset, end + 1);
|
|
2646
|
+
const previousComment = index > 0 ? lines[index - 1].text : "";
|
|
2647
|
+
if (!/\.only\b|\[\s*['"]only['"]\s*\]|\b(?:fit|fdescribe)\s*\(/.test(text) && (nativeReason(invocation) || /^\s*(?:\/\/|#)/.test(previousComment) && /\breason\s*:/i.test(previousComment) && hasExplanation(previousComment))) continue;
|
|
2648
|
+
if (!/\.only\b|\[\s*['"]only['"]\s*\]|\b(?:fit|fdescribe)\s*\(/.test(text) && (/\breason\s*:/i.test(text) && hasExplanation(text) || /\breason\s*=\s*['"][^'"]*\S+\s+\S+\s+\S+[^'"]*['"]/.test(text))) continue;
|
|
2649
|
+
const lexical = "+" + (signatureLines[index] ?? "");
|
|
2650
|
+
const masked = "+" + (maskedLines[index] ?? "");
|
|
2651
|
+
const bracketApi = [...change.content.matchAll(new RegExp(BRACKET_SKIP, "g"))].some((match) => {
|
|
2652
|
+
const root = /^(?:it|test|describe)/.exec(match[0])[0];
|
|
2653
|
+
return masked.slice(match.index, match.index + root.length) === root;
|
|
2654
|
+
});
|
|
2655
|
+
const commentedDeclaration = COMMENTED_JS_TEST.test(change.content) || COMMENTED_PY_TEST.test(change.content);
|
|
2656
|
+
if (!isSkipPattern(commentedDeclaration || bracketApi ? change.content : lexical, {
|
|
2320
2657
|
ext,
|
|
2321
2658
|
pyTestModule,
|
|
2322
2659
|
goTestFile,
|
|
@@ -2408,10 +2745,86 @@ function extractLiterals(line) {
|
|
|
2408
2745
|
}
|
|
2409
2746
|
return result;
|
|
2410
2747
|
}
|
|
2748
|
+
function assertionPairs(context) {
|
|
2749
|
+
const pairs = /* @__PURE__ */ new Map();
|
|
2750
|
+
const pattern = new RegExp(`expect\\(\\s*([A-Za-z_$][\\w$]*)\\(\\s*(${LITERAL_TOKEN})\\s*\\)\\s*\\)\\.(?:toBe|toEqual|toStrictEqual)\\(\\s*(${LITERAL_TOKEN})\\s*\\)`, "g");
|
|
2751
|
+
for (const file of context.files) {
|
|
2752
|
+
if (!context.isTestFile(pathOf(file))) continue;
|
|
2753
|
+
for (const chunk of file.chunks) {
|
|
2754
|
+
const view = sourceView(chunk, "after");
|
|
2755
|
+
for (const match of view.text.matchAll(pattern)) {
|
|
2756
|
+
if (!view.code.slice(match.index, match.index + 6).startsWith("expect")) continue;
|
|
2757
|
+
const set = pairs.get(match[1]) ?? /* @__PURE__ */ new Set();
|
|
2758
|
+
set.add(match[2].trim() + ":" + match[3].trim());
|
|
2759
|
+
pairs.set(match[1], set);
|
|
2760
|
+
}
|
|
2761
|
+
}
|
|
2762
|
+
}
|
|
2763
|
+
return pairs;
|
|
2764
|
+
}
|
|
2765
|
+
function structuralHardcodes(context) {
|
|
2766
|
+
const findings = [];
|
|
2767
|
+
const pairs = assertionPairs(context);
|
|
2768
|
+
if (!pairs.size) return findings;
|
|
2769
|
+
for (const file of context.files) {
|
|
2770
|
+
const filePath = pathOf(file);
|
|
2771
|
+
if (!isWatchedSource(context, filePath)) continue;
|
|
2772
|
+
for (const chunk of file.chunks) {
|
|
2773
|
+
const before = sourceView(chunk, "before");
|
|
2774
|
+
const after = sourceView(chunk, "after");
|
|
2775
|
+
const oldFunctions = callables(before);
|
|
2776
|
+
for (const fn of callables(after)) {
|
|
2777
|
+
const evidence = pairs.get(fn.name);
|
|
2778
|
+
const old = oldFunctions.find((candidate) => candidate.header === fn.header);
|
|
2779
|
+
if (!evidence || !old) continue;
|
|
2780
|
+
const raw = after.text.slice(fn.open + 1, fn.end);
|
|
2781
|
+
const branch = new RegExp(`\\bif\\s*\\(\\s*[A-Za-z_$][\\w$]*\\s*===?\\s*(${LITERAL_TOKEN})\\s*\\)\\s*\\{\\s*return\\s+(${LITERAL_TOKEN})\\s*;?\\s*\\}`, "g");
|
|
2782
|
+
for (const match of raw.matchAll(branch)) {
|
|
2783
|
+
if (!evidence.has(match[1].trim() + ":" + match[2].trim())) continue;
|
|
2784
|
+
const offset = fn.open + 1 + match.index;
|
|
2785
|
+
if (!after.code.slice(offset, offset + 2).startsWith("if")) continue;
|
|
2786
|
+
if (!after.touched(offset, offset + match[0].length)) continue;
|
|
2787
|
+
const open = after.code.indexOf("{", offset);
|
|
2788
|
+
const close = closingDelimiter(after.code, open);
|
|
2789
|
+
const rest = after.code.slice(fn.open + 1, offset) + after.code.slice(close + 1, fn.end);
|
|
2790
|
+
if (compact(rest) !== compact(old.body)) continue;
|
|
2791
|
+
findings.push({
|
|
2792
|
+
verifierId: "RH004",
|
|
2793
|
+
severity: "error",
|
|
2794
|
+
file: filePath,
|
|
2795
|
+
line: after.lineAt(offset),
|
|
2796
|
+
message: "A literal-input branch returns the matching test expectation while leaving the general computation unchanged.",
|
|
2797
|
+
suggestion: "Implement the behavior generally rather than adding a branch for one asserted example."
|
|
2798
|
+
});
|
|
2799
|
+
}
|
|
2800
|
+
if (!/\breturn\s+[^;]+\(/.test(old.body) || /\breturn\s+[^;]+\(/.test(fn.body)) continue;
|
|
2801
|
+
const lookup = /\breturn\s+([A-Za-z_$][\w$]*)\s*\[\s*([A-Za-z_$][\w$]*)\s*\]/.exec(fn.body);
|
|
2802
|
+
if (!lookup) continue;
|
|
2803
|
+
const table = [...fn.body.matchAll(/\b(?:const|let)\s+([A-Za-z_$][\w$]*)\s*=\s*\{/g)].find((binding) => binding[1] === lookup[1]);
|
|
2804
|
+
if (!table) continue;
|
|
2805
|
+
const literalEntry = new RegExp(`\\[?\\s*(${LITERAL_TOKEN})\\s*\\]?\\s*:\\s*(${LITERAL_TOKEN})`, "g");
|
|
2806
|
+
const tableOpen = fn.open + 1 + table.index + table[0].length - 1;
|
|
2807
|
+
const tableEnd = closingDelimiter(after.code, tableOpen);
|
|
2808
|
+
if (tableEnd < 0) continue;
|
|
2809
|
+
const entries = after.text.slice(tableOpen + 1, tableEnd);
|
|
2810
|
+
if (![...entries.matchAll(literalEntry)].some((entry) => evidence.has(entry[1].trim() + ":" + entry[2].trim()))) continue;
|
|
2811
|
+
findings.push({
|
|
2812
|
+
verifierId: "RH004",
|
|
2813
|
+
severity: "error",
|
|
2814
|
+
file: filePath,
|
|
2815
|
+
line: after.lineAt(fn.open),
|
|
2816
|
+
message: "A computed return was replaced by a lookup containing the same literal input and output as a test assertion.",
|
|
2817
|
+
suggestion: "Restore the computation. A finite domain table needs a stated specification and complete domain coverage."
|
|
2818
|
+
});
|
|
2819
|
+
}
|
|
2820
|
+
}
|
|
2821
|
+
}
|
|
2822
|
+
return findings;
|
|
2823
|
+
}
|
|
2411
2824
|
async function run4(context) {
|
|
2412
2825
|
const files = context.files;
|
|
2413
2826
|
const ctx = context;
|
|
2414
|
-
const findings =
|
|
2827
|
+
const findings = structuralHardcodes(context);
|
|
2415
2828
|
const fuzzyCandidates = [];
|
|
2416
2829
|
for (const file of files) {
|
|
2417
2830
|
const filePath = file.to ?? file.from ?? "";
|
|
@@ -2455,6 +2868,7 @@ async function run4(context) {
|
|
|
2455
2868
|
const branchMatch = add.content.match(BRANCH_LITERAL_RE) ?? add.content.match(PY_BRANCH_LITERAL_RE);
|
|
2456
2869
|
if (!branchMatch) continue;
|
|
2457
2870
|
if (EMPTINESS_GUARD_RE.test(add.content) || TYPEOF_GUARD_RE.test(add.content)) continue;
|
|
2871
|
+
if (/^(['"`])[^\w\s]\1$/.test(branchMatch[1])) continue;
|
|
2458
2872
|
findings.push({
|
|
2459
2873
|
verifierId: "RH004",
|
|
2460
2874
|
severity: "error",
|
|
@@ -2619,6 +3033,22 @@ async function run5(context) {
|
|
|
2619
3033
|
const adds = chunk.changes.filter((c) => c.type === "add" && !isMoved(moved, c.content));
|
|
2620
3034
|
const near = (a, b) => Math.abs((at.get(a) ?? 0) - (at.get(b) ?? 0)) <= PAIRING_DISTANCE;
|
|
2621
3035
|
if (!isTest) {
|
|
3036
|
+
const beforeView = sourceView(chunk, "before");
|
|
3037
|
+
const afterView = sourceView(chunk, "after");
|
|
3038
|
+
const previous = callables(beforeView);
|
|
3039
|
+
for (const fn of callables(afterView)) {
|
|
3040
|
+
if (fn.body.trim() || !afterView.touched(fn.start, fn.end)) continue;
|
|
3041
|
+
const prior = previous.find((old) => old.header === fn.header);
|
|
3042
|
+
if (!prior || !/\b(?:return|await)\b|\w\s*\(/.test(prior.body)) continue;
|
|
3043
|
+
findings.push({
|
|
3044
|
+
verifierId: "RH005",
|
|
3045
|
+
severity: "error",
|
|
3046
|
+
file: filePath,
|
|
3047
|
+
line: afterView.lineAt(fn.open),
|
|
3048
|
+
message: "An existing function lost its executable body and is now empty.",
|
|
3049
|
+
suggestion: "Restore the implementation or remove the feature and its callers deliberately."
|
|
3050
|
+
});
|
|
3051
|
+
}
|
|
2622
3052
|
const shrank = adds.length <= dels.length;
|
|
2623
3053
|
for (const add of adds) {
|
|
2624
3054
|
if (!isGuttedAdd(add.content)) continue;
|
|
@@ -2729,6 +3159,7 @@ var rh006 = { id: "RH006", severity: "warn", run: run6 };
|
|
|
2729
3159
|
|
|
2730
3160
|
// src/verifiers/rh007.ts
|
|
2731
3161
|
import path from "path";
|
|
3162
|
+
import picomatch4 from "picomatch";
|
|
2732
3163
|
var CONFIG_FILE_RE = /(?:jest|vitest|vite)\.config\.(?:[mc]?[jt]s|json)$|(?:^|\/)jest\.config\.json$|tsconfig(?:\.[^/]*)?\.json$|(?:pytest\.ini|setup\.cfg|pyproject\.toml|conftest\.py)$|(?:^|\/)pom\.xml$|(?:^|\/)build\.gradle(?:\.kts)?$|(?:^|\/)Cargo\.toml$|(?:^|\/)\.rspec$|(?:^|\/)phpunit\.xml(?:\.dist)?$|\.runsettings$|(?:^|\/)CMakeLists\.txt$|\.xctestplan$|(?:^|\/)dart_test\.yaml$|(?:^|\/)build\.sbt$|(?:^|\/)\.Rbuildignore$|\.cabal$|(?:^|\/)test_helper\.exs$|(?:^|\/)\.busted$|(?:^|\/)project\.clj$/;
|
|
2733
3164
|
var PACKAGE_JSON_RE = /(?:^|\/)package\.json$/;
|
|
2734
3165
|
var JEST_EXCLUSION_KEY_RE = /"(testPathIgnorePatterns)"\s*:/;
|
|
@@ -2803,8 +3234,8 @@ function narrowsSelection(changes, added) {
|
|
|
2803
3234
|
if (!key) return false;
|
|
2804
3235
|
const prior = changes.find((c) => c.type === "del" && c.content.includes(key));
|
|
2805
3236
|
if (!prior) return false;
|
|
2806
|
-
const kept =
|
|
2807
|
-
return patternsIn(prior.content).some((p) => !kept.
|
|
3237
|
+
const kept = patternsIn(added.content);
|
|
3238
|
+
return patternsIn(prior.content).some((p) => !kept.some((next) => next === p || key === "testMatch" && !/[?*{[]/.test(p) && picomatch4(next)(p) || key === "testMatch" && next.startsWith("**/") && picomatch4(next.slice(3))(p.split("/").pop() ?? p)));
|
|
2808
3239
|
}
|
|
2809
3240
|
var EXCLUSION_PATTERNS = [
|
|
2810
3241
|
{ re: /testPathIgnorePatterns/, key: "testPathIgnorePatterns", severity: "error", langs: ["js"] },
|
|
@@ -2995,6 +3426,16 @@ function run7(context) {
|
|
|
2995
3426
|
const testSuiteNameMatch = chunk.changes.map((c) => c.content.match(/\btest-suite\s+([A-Za-z0-9_-]+)/i)).find((m) => m !== null);
|
|
2996
3427
|
for (const change of chunk.changes) {
|
|
2997
3428
|
if (change.type !== "add") continue;
|
|
3429
|
+
if (lang === "js" && /\bpassWithNoTests\s*:\s*true\b|"passWithNoTests"\s*:\s*true\b/.test(change.content) && !chunk.changes.some((c) => c.type === "del" && /passWithNoTests["']?\s*:\s*true\b/.test(c.content))) {
|
|
3430
|
+
findings.push({
|
|
3431
|
+
verifierId: "RH007",
|
|
3432
|
+
severity: "error",
|
|
3433
|
+
file: filePath,
|
|
3434
|
+
line: change.ln,
|
|
3435
|
+
message: "Test discovery now succeeds when it finds no tests.",
|
|
3436
|
+
suggestion: "Keep empty test discovery failing so an accidentally detached suite cannot report success."
|
|
3437
|
+
});
|
|
3438
|
+
}
|
|
2998
3439
|
const matched = matchExclusion(change.content, lang);
|
|
2999
3440
|
if (!matched) continue;
|
|
3000
3441
|
const { pattern, afterMatch } = matched;
|
|
@@ -3403,7 +3844,9 @@ function isJustified(content) {
|
|
|
3403
3844
|
(m) => m !== null
|
|
3404
3845
|
);
|
|
3405
3846
|
if (!match) return false;
|
|
3406
|
-
|
|
3847
|
+
const tail = content.slice(match.index + match[0].length);
|
|
3848
|
+
if (/^\s*(?:\/\/|\/\*)\s*@ts-(?:expect-error|ignore)\b/.test(body(content)) && tail.trim().split(/\s+/).filter(Boolean).length >= 3) return true;
|
|
3849
|
+
return JUSTIFICATION_RE.test(tail);
|
|
3407
3850
|
}
|
|
3408
3851
|
function isFilewideSuppression(content) {
|
|
3409
3852
|
return FILEWIDE_ESLINT_DISABLE_RE.test(content) || FILEWIDE_FLAKE8_NOQA_RE.test(content) || FILEWIDE_TS_NOCHECK_RE.test(content) || FILEWIDE_MYPY_RE.test(content) || FILEWIDE_RUST_ALLOW_RE.test(content) || FILEWIDE_KOTLIN_SUPPRESS_RE.test(content) || FILEWIDE_PHPCS_IGNOREFILE_RE.test(content) || FILEWIDE_SWIFTLINT_ALL_RE.test(content) || FILEWIDE_DART_IGNOREFILE_RE.test(content) || FILEWIDE_HLINT_RE.test(content) || FILEWIDE_CREDO_FILE_RE.test(content);
|
|
@@ -3428,8 +3871,20 @@ function run11(context) {
|
|
|
3428
3871
|
if (DOC_FILE_RE.test(filePath)) continue;
|
|
3429
3872
|
if (VENDORED_PATH_RE.test(filePath.replace(/\\/g, "/"))) continue;
|
|
3430
3873
|
const occurrences = [];
|
|
3874
|
+
const identity = (text) => body(text).trim().replace(/\s*(?:\/\/|#)\s+(?:for|because|reason:)\b.*$/, "").trim();
|
|
3875
|
+
const removed = /* @__PURE__ */ new Map();
|
|
3876
|
+
for (const change of file.chunks.flatMap((chunk) => chunk.changes).filter((c) => c.type === "del")) {
|
|
3877
|
+
const key = identity(change.content);
|
|
3878
|
+
removed.set(key, (removed.get(key) ?? 0) + 1);
|
|
3879
|
+
}
|
|
3431
3880
|
for (const chunk of file.chunks) {
|
|
3432
3881
|
for (const add of chunk.changes.filter((c) => c.type === "add")) {
|
|
3882
|
+
const key = identity(add.content);
|
|
3883
|
+
const remaining = removed.get(key) ?? 0;
|
|
3884
|
+
if (remaining > 0) {
|
|
3885
|
+
removed.set(key, remaining - 1);
|
|
3886
|
+
continue;
|
|
3887
|
+
}
|
|
3433
3888
|
const line = add.ln;
|
|
3434
3889
|
if (filewideInCode(add.content)) {
|
|
3435
3890
|
findings.push({
|
|
@@ -3533,10 +3988,13 @@ var COMMAND_KEYS = /* @__PURE__ */ new Set(["run", "script", "command", "command
|
|
|
3533
3988
|
function isCommandLine(text) {
|
|
3534
3989
|
if (/^\s*#/.test(text)) return false;
|
|
3535
3990
|
const key = /^[ \t]*(?:-[ \t]*)?([A-Za-z_][\w.-]*)[ \t]*:/.exec(text);
|
|
3991
|
+
if (/^[ \t]*(?:-[ \t]*)?\{/.test(text)) return false;
|
|
3536
3992
|
return key === null || COMMAND_KEYS.has(key[1].toLowerCase());
|
|
3537
3993
|
}
|
|
3538
3994
|
function commandKey(content) {
|
|
3539
|
-
|
|
3995
|
+
const command = stripDiffPrefix2(content);
|
|
3996
|
+
const invocation = TEST_COMMAND_RE.exec(command);
|
|
3997
|
+
return (invocation ? command.slice(invocation.index) : command).replace(/^-\s*/, "").replace(/^(?:run|script|command)\s*:\s*/, "").replace(/['"]/g, "").replace(/\s+/g, " ").trim().toLowerCase();
|
|
3540
3998
|
}
|
|
3541
3999
|
var RETRY_ACTION_RE = /uses\s*:\s*[\w-]+\/(?:retry|action-retry|retry-action)@/i;
|
|
3542
4000
|
var RETRY_ATTEMPTS_RE = /max_attempts\s*:\s*([2-9]|\d{2,})/;
|
|
@@ -3740,45 +4198,83 @@ function withoutLeadingSlice(text) {
|
|
|
3740
4198
|
function withoutFilter(text) {
|
|
3741
4199
|
return withoutTrailingComment(text).replace(/\.filter\([^)]*\)/, "").replace(/\s+/g, " ").trim();
|
|
3742
4200
|
}
|
|
3743
|
-
var PARAMETERIZED_RUNNER_RE = /\b(?:it|test)\.each\s*\(|pytest\.mark\.parametrize\b/;
|
|
3744
|
-
var TABLE_ROW_RE = /^\s*\[[^\]]+\],?\s*$/;
|
|
3745
4201
|
var LOOP_BOUND_RE = /\bfor\s*\([^;]*;\s*([A-Za-z_$][\w$]*)\s*<\s*(\d+)\s*;/;
|
|
3746
|
-
function
|
|
3747
|
-
|
|
4202
|
+
function elements(view, open, end) {
|
|
4203
|
+
let depth = 0;
|
|
4204
|
+
let start = open + 1;
|
|
4205
|
+
let count = 0;
|
|
4206
|
+
const present = (from, to) => !!view.code.slice(from, to).trim() || /^\s*['"`]/.test(view.text.slice(from, to));
|
|
4207
|
+
for (let i = start; i < end; i++) {
|
|
4208
|
+
const ch = view.code[i];
|
|
4209
|
+
if ("[{(".includes(ch)) depth++;
|
|
4210
|
+
else if ("]})".includes(ch)) depth--;
|
|
4211
|
+
else if (ch === "," && depth === 0) {
|
|
4212
|
+
if (present(start, i)) count++;
|
|
4213
|
+
start = i + 1;
|
|
4214
|
+
}
|
|
4215
|
+
}
|
|
4216
|
+
return count + Number(present(start, end));
|
|
4217
|
+
}
|
|
4218
|
+
function parameterTables(view) {
|
|
4219
|
+
const tables = [];
|
|
4220
|
+
const runner = /\b(?:it|test)\.each\s*\(|\bpytest\.mark\.parametrize\s*\(/g;
|
|
4221
|
+
for (const match of view.code.matchAll(runner)) {
|
|
4222
|
+
const args = match.index + match[0].length - 1;
|
|
4223
|
+
const argsEnd = closingDelimiter(view.code, args, "(", ")");
|
|
4224
|
+
if (argsEnd < 0) continue;
|
|
4225
|
+
let open = args + 1;
|
|
4226
|
+
while (/\s/.test(view.code[open] ?? "") && open < argsEnd) open++;
|
|
4227
|
+
let key = "";
|
|
4228
|
+
if (match[0].includes("parametrize")) {
|
|
4229
|
+
const comma = view.code.indexOf(",", open);
|
|
4230
|
+
if (comma < 0 || comma >= argsEnd) continue;
|
|
4231
|
+
key = "python:" + view.text.slice(open, comma).replace(/\s+/g, "");
|
|
4232
|
+
if (key === "python:") continue;
|
|
4233
|
+
open = comma + 1;
|
|
4234
|
+
while (/\s/.test(view.code[open] ?? "") && open < argsEnd) open++;
|
|
4235
|
+
} else {
|
|
4236
|
+
const reference = /^[A-Za-z_$][\w$]*/.exec(view.code.slice(open, argsEnd));
|
|
4237
|
+
if (reference) {
|
|
4238
|
+
key = "binding:" + reference[0];
|
|
4239
|
+
const binding = [...view.code.matchAll(/\b(?:const|let|var)\s+([A-Za-z_$][\w$]*)\s*=\s*\[/g)].find((binding2) => binding2[1] === reference[0]);
|
|
4240
|
+
if (!binding) continue;
|
|
4241
|
+
open = binding.index + binding[0].length - 1;
|
|
4242
|
+
} else {
|
|
4243
|
+
const title = /^\s*\(\s*(['"`])([^'"`]+)\1/.exec(view.text.slice(argsEnd + 1));
|
|
4244
|
+
if (!title) continue;
|
|
4245
|
+
key = "inline:" + title[2];
|
|
4246
|
+
}
|
|
4247
|
+
}
|
|
4248
|
+
const left = view.code[open];
|
|
4249
|
+
if (left !== "[" && left !== "(") continue;
|
|
4250
|
+
const end = closingDelimiter(view.code, open, left, left === "[" ? "]" : ")");
|
|
4251
|
+
if (end < 0) continue;
|
|
4252
|
+
tables.push({ key, count: elements(view, open, end), start: open, end });
|
|
4253
|
+
}
|
|
4254
|
+
return tables;
|
|
3748
4255
|
}
|
|
3749
4256
|
function run14(context) {
|
|
3750
4257
|
const findings = [];
|
|
3751
4258
|
for (const file of context.files) {
|
|
3752
4259
|
const filePath = pathOf(file);
|
|
3753
4260
|
if (!context.isTestFile(filePath)) continue;
|
|
3754
|
-
|
|
3755
|
-
const
|
|
3756
|
-
|
|
3757
|
-
const
|
|
3758
|
-
const
|
|
3759
|
-
|
|
3760
|
-
|
|
3761
|
-
|
|
3762
|
-
|
|
3763
|
-
|
|
3764
|
-
file: filePath,
|
|
3765
|
-
line: netRemoved[0].line,
|
|
3766
|
-
message: "Test workload reduced: a row was removed from a parameterized test table while the test itself still runs.",
|
|
3767
|
-
suggestion: "Restore the removed case and fix the behavior it exposed."
|
|
3768
|
-
});
|
|
3769
|
-
}
|
|
3770
|
-
for (const added of file.chunks.flatMap(addedLines)) {
|
|
3771
|
-
const nextCount = inlineRowCount(added.text);
|
|
3772
|
-
if (nextCount === 0) continue;
|
|
3773
|
-
const prior = file.chunks.flatMap(deletedLines).find((line) => inlineRowCount(line.text) > nextCount);
|
|
3774
|
-
if (!prior || hasExplanation(added.text)) continue;
|
|
4261
|
+
if (file.deleted) continue;
|
|
4262
|
+
for (const chunk of file.chunks) {
|
|
4263
|
+
const before = sourceView(chunk, "before", filePath.endsWith(".py"));
|
|
4264
|
+
const after = sourceView(chunk, "after", filePath.endsWith(".py"));
|
|
4265
|
+
const old = parameterTables(before);
|
|
4266
|
+
for (const next of parameterTables(after)) {
|
|
4267
|
+
const prior = old.find((table) => table.key === next.key);
|
|
4268
|
+
if (!prior || next.count >= prior.count || !after.touched(next.start, next.end)) continue;
|
|
4269
|
+
if (after.text.slice(next.start, next.end).split("\n").some(hasExplanation)) continue;
|
|
4270
|
+
const inline = !after.text.slice(next.start, next.end).includes("\n");
|
|
3775
4271
|
findings.push({
|
|
3776
4272
|
verifierId: "RH014",
|
|
3777
4273
|
severity: "error",
|
|
3778
4274
|
file: filePath,
|
|
3779
|
-
line:
|
|
3780
|
-
message: `Test workload reduced: an inline parameter table fell from ${
|
|
3781
|
-
suggestion: "Restore the removed inline cases and fix the behavior they expose."
|
|
4275
|
+
line: after.lineAt(next.start),
|
|
4276
|
+
message: inline ? `Test workload reduced: an inline parameter table fell from ${prior.count} cases to ${next.count}.` : "Test workload reduced: a row was removed from a parameterized test table while the test itself still runs.",
|
|
4277
|
+
suggestion: inline ? "Restore the removed inline cases and fix the behavior they expose." : "Restore the removed case and fix the behavior it exposed."
|
|
3782
4278
|
});
|
|
3783
4279
|
}
|
|
3784
4280
|
}
|
|
@@ -3965,6 +4461,128 @@ function bareCall(text) {
|
|
|
3965
4461
|
);
|
|
3966
4462
|
return match?.[1]?.replace(/\s+/g, "");
|
|
3967
4463
|
}
|
|
4464
|
+
function handlers(view, indented) {
|
|
4465
|
+
const result = [];
|
|
4466
|
+
const opener = indented ? /^([ \t]*)(?:except\b[^:\n]*:|rescue\b[^\n]*)/gm : /\bcatch\s*(?:\([^)]*\))?\s*\{|\bif\s+([A-Za-z_]\w*)\s*!=\s*nil\s*\{/g;
|
|
4467
|
+
for (const match of view.code.matchAll(opener)) {
|
|
4468
|
+
const start = match.index;
|
|
4469
|
+
const bodyStart = start + match[0].length;
|
|
4470
|
+
let end;
|
|
4471
|
+
if (!indented) end = closingDelimiter(view.code, bodyStart - 1);
|
|
4472
|
+
else {
|
|
4473
|
+
end = view.code.length;
|
|
4474
|
+
const indent = match[1].replace(/\n/g, "").length;
|
|
4475
|
+
const tail = view.code.slice(bodyStart);
|
|
4476
|
+
const boundary = /\n([ \t]*)(\S)/g;
|
|
4477
|
+
for (const line of tail.matchAll(boundary)) {
|
|
4478
|
+
if (line[1].length <= indent) {
|
|
4479
|
+
end = bodyStart + line.index + 1;
|
|
4480
|
+
break;
|
|
4481
|
+
}
|
|
4482
|
+
}
|
|
4483
|
+
}
|
|
4484
|
+
if (end < 0) continue;
|
|
4485
|
+
result.push({ start, end, key: compact(match[0]), code: view.code.slice(bodyStart, end), raw: view.text.slice(start, end + 1) });
|
|
4486
|
+
}
|
|
4487
|
+
if (!indented) for (const match of view.code.matchAll(/\.catch\s*\(\s*(?:\([^)]*\)|[A-Za-z_$][\w$]*)\s*=>\s*/g)) {
|
|
4488
|
+
const start = match.index;
|
|
4489
|
+
const bodyStart = start + match[0].length;
|
|
4490
|
+
const open = view.code.indexOf("(", start);
|
|
4491
|
+
const end = closingDelimiter(view.code, open, "(", ")");
|
|
4492
|
+
if (end < 0) continue;
|
|
4493
|
+
const code = view.code.slice(bodyStart, end).replace(/^\{([\s\S]*)\}$/, "$1");
|
|
4494
|
+
result.push({ start, end, key: compact(match[0]), code, raw: view.text.slice(start, end + 1) });
|
|
4495
|
+
}
|
|
4496
|
+
return result;
|
|
4497
|
+
}
|
|
4498
|
+
function discardedHandler(code) {
|
|
4499
|
+
let rest = code;
|
|
4500
|
+
const logging = /\b(?:(?:console|logger|logging|log|Console|System\.err)\.(?:error|warn|warning|info|debug|trace|log|Print|Printf|Println|WriteLine|println)|error_log)\s*\(/g;
|
|
4501
|
+
const ranges = [];
|
|
4502
|
+
for (const match of rest.matchAll(logging)) {
|
|
4503
|
+
const open = match.index + match[0].lastIndexOf("(");
|
|
4504
|
+
const end = closingDelimiter(rest, open, "(", ")");
|
|
4505
|
+
if (end < 0) return false;
|
|
4506
|
+
ranges.push([match.index, end + 1]);
|
|
4507
|
+
}
|
|
4508
|
+
for (const [start, end] of ranges.reverse()) rest = rest.slice(0, start) + rest.slice(end);
|
|
4509
|
+
return /^(?:\s|;|pass\b)*$/.test(rest);
|
|
4510
|
+
}
|
|
4511
|
+
function structuralSwallows(context) {
|
|
4512
|
+
const findings = [];
|
|
4513
|
+
for (const file of context.files) {
|
|
4514
|
+
const path9 = pathOf(file);
|
|
4515
|
+
if (!isWatchedSource(context, path9) || !/\.(?:[cm]?[jt]sx?|py|go|rs|java|rb|cs|php|sh|bash)$/.test(path9)) continue;
|
|
4516
|
+
const indented = /\.(?:py|rb)$/.test(path9);
|
|
4517
|
+
for (const chunk of file.chunks) {
|
|
4518
|
+
const before = sourceView(chunk, "before", indented);
|
|
4519
|
+
const after = sourceView(chunk, "after", indented);
|
|
4520
|
+
const prior = handlers(before, indented);
|
|
4521
|
+
for (const handler of handlers(after, indented)) {
|
|
4522
|
+
if (!after.touched(handler.start, handler.end) || !discardedHandler(handler.code)) continue;
|
|
4523
|
+
if (handler.raw.split("\n").some(hasExplanation)) continue;
|
|
4524
|
+
if (indented && /^[ \t]*\r?\n?[ \t]*else[ \t]*:/.test(after.text.slice(handler.end))) continue;
|
|
4525
|
+
const retainedOpener = chunk.changes.find((change) => change.type === "normal" && change.ln2 === after.lineAt(handler.start));
|
|
4526
|
+
const previous = retainedOpener?.type === "normal" ? prior.filter((old) => before.lineAt(old.start) === retainedOpener.ln1) : prior;
|
|
4527
|
+
if (previous.some((old) => old.key === handler.key && discardedHandler(old.code))) continue;
|
|
4528
|
+
findings.push({
|
|
4529
|
+
verifierId: "WI101",
|
|
4530
|
+
severity: "error",
|
|
4531
|
+
file: path9,
|
|
4532
|
+
line: after.lineAt(handler.start),
|
|
4533
|
+
message: "Error handling was added or changed to an empty or logging-only handler, so failures no longer propagate.",
|
|
4534
|
+
suggestion: "Return an error, retry, or propagate the failure. Document a specific, deliberate fallback."
|
|
4535
|
+
});
|
|
4536
|
+
}
|
|
4537
|
+
const removed = deletedLines(chunk);
|
|
4538
|
+
for (const added of addedLines(chunk)) {
|
|
4539
|
+
if (isCommentLine(added.text) || hasExplanation(added.text)) continue;
|
|
4540
|
+
let discarded = false;
|
|
4541
|
+
if (path9.endsWith(".rs")) {
|
|
4542
|
+
const operation = (text) => compact(text.replace(/^\s*(?:let\s+[^=]+\s*=|return\s+)/, "").replace(/\?\s*;?\s*$/, "").replace(/;\s*$/, ""));
|
|
4543
|
+
const lost = removed.filter((line) => /\?\s*;?\s*$/.test(line.text));
|
|
4544
|
+
const replacement = added.text.replace(/^\s*let\s+_\s*=\s*/, "").replace(/\.unwrap_or_default\(\)/, "");
|
|
4545
|
+
discarded = /\blet\s+_\s*=|\.unwrap_or_default\(\)/.test(added.text) && lost.some((line) => operation(line.text) === operation(replacement));
|
|
4546
|
+
} else if (path9.endsWith(".go")) {
|
|
4547
|
+
const assignment = /^\s*([^:=]+)(?::=|=)\s*(.+?)\s*;?\s*$/;
|
|
4548
|
+
const next = assignment.exec(added.text);
|
|
4549
|
+
if (next && next[1].split(",").some((binding) => binding.trim() === "_")) {
|
|
4550
|
+
discarded = removed.some((line) => {
|
|
4551
|
+
const old = assignment.exec(line.text);
|
|
4552
|
+
if (!old || compact(old[2]) !== compact(next[2])) return false;
|
|
4553
|
+
const names = old[1].split(",").map((binding) => binding.trim());
|
|
4554
|
+
const slots = next[1].split(",").map((binding) => binding.trim());
|
|
4555
|
+
return names.some((name, i) => slots[i] === "_" && name !== "_" && [...before.code.matchAll(/\bif\s+([A-Za-z_]\w*)\s*!=\s*nil\b/g)].some((check) => check[1] === name));
|
|
4556
|
+
});
|
|
4557
|
+
}
|
|
4558
|
+
} else if (/\.(?:sh|bash)$/.test(path9)) {
|
|
4559
|
+
const swallow = /\s*\|\|\s*(?:true|:|exit\s+0)\s*$/.exec(added.text);
|
|
4560
|
+
discarded = !!swallow && removed.some((line) => compact(line.text) === compact(added.text.slice(0, swallow.index)));
|
|
4561
|
+
}
|
|
4562
|
+
if (discarded) findings.push({
|
|
4563
|
+
verifierId: "WI101",
|
|
4564
|
+
severity: "error",
|
|
4565
|
+
file: path9,
|
|
4566
|
+
line: added.line,
|
|
4567
|
+
message: "A previously propagated operation failure is now discarded or replaced with a default.",
|
|
4568
|
+
suggestion: "Preserve the operation failure or handle it explicitly instead of converting it to success."
|
|
4569
|
+
});
|
|
4570
|
+
}
|
|
4571
|
+
}
|
|
4572
|
+
}
|
|
4573
|
+
return findings;
|
|
4574
|
+
}
|
|
4575
|
+
function followedByElse(lines, index) {
|
|
4576
|
+
if (index < 0) return false;
|
|
4577
|
+
const indent = (text) => text.length - text.trimStart().length;
|
|
4578
|
+
const base = indent(lines[index].text);
|
|
4579
|
+
for (const line of lines.slice(index + 1)) {
|
|
4580
|
+
if (!line.text.trim()) continue;
|
|
4581
|
+
if (indent(line.text) > base) continue;
|
|
4582
|
+
return indent(line.text) === base && /^\s*else\s*:/.test(line.text);
|
|
4583
|
+
}
|
|
4584
|
+
return false;
|
|
4585
|
+
}
|
|
3968
4586
|
function run15(context) {
|
|
3969
4587
|
const findings = [];
|
|
3970
4588
|
for (const file of context.files) {
|
|
@@ -3988,6 +4606,7 @@ function run15(context) {
|
|
|
3988
4606
|
}
|
|
3989
4607
|
for (const chunk of file.chunks) {
|
|
3990
4608
|
const after = afterLines(chunk);
|
|
4609
|
+
const lexicalLines = sourceView(chunk, "after", /\.(?:py|rb)$/.test(filePath)).code.split("\n");
|
|
3991
4610
|
const addedInOrder = addedLines(chunk);
|
|
3992
4611
|
const templated = insideTemplateLiteral(addedInOrder);
|
|
3993
4612
|
const removedLoud = deletedLines(chunk).filter((line) => ERROR_DIAGNOSTIC_RE.test(line.text));
|
|
@@ -4022,7 +4641,10 @@ function run15(context) {
|
|
|
4022
4641
|
if (templated.has(index)) continue;
|
|
4023
4642
|
if (hasExplanation(added.text)) continue;
|
|
4024
4643
|
if (isCommentLine(added.text)) continue;
|
|
4644
|
+
const lexicalIndex = after.findIndex((line) => line.line === added.line && line.added);
|
|
4645
|
+
if (lexicalIndex >= 0 && !lexicalLines[lexicalIndex]?.trim()) continue;
|
|
4025
4646
|
const code = withoutLiterals(added.text);
|
|
4647
|
+
if (/^\s*except\b/.test(code) && followedByElse(after, after.findIndex((l) => l.line === added.line && l.added))) continue;
|
|
4026
4648
|
const inline = INLINE_SIGNATURES.find((sig) => sig.opener.test(code));
|
|
4027
4649
|
if (inline) {
|
|
4028
4650
|
findings.push({
|
|
@@ -4065,7 +4687,8 @@ function run15(context) {
|
|
|
4065
4687
|
}
|
|
4066
4688
|
}
|
|
4067
4689
|
}
|
|
4068
|
-
|
|
4690
|
+
const combined = [...findings, ...structuralSwallows(context)];
|
|
4691
|
+
return combined.filter((finding, index) => !combined.slice(0, index).some((prior) => prior.file === finding.file && prior.line === finding.line && prior.severity === finding.severity));
|
|
4069
4692
|
}
|
|
4070
4693
|
var wi101 = { id: "WI101", severity: "error", run: run15 };
|
|
4071
4694
|
|
|
@@ -4878,6 +5501,7 @@ function assertionKey(text) {
|
|
|
4878
5501
|
var TABLE_RUNNER_RE = /\.each\b|\bit_each\b|@pytest\.mark\.parametrize|@ParameterizedTest\b|\[InlineData\b|\[Theory\]|\bfor\s+(?:const|let|var)?\s*[\w[\],{} ]+\s+(?:of|in)\s+\w*(?:case|test|table|fixture|sample|scenario|example)\w*\b|\bfor\s+\w+,\s*\w+\s*:=\s*range\b|\.forEach\s*\(|\btests\s*:?=\s*\[\]struct\b/i;
|
|
4879
5502
|
function isDataRow(text) {
|
|
4880
5503
|
const stripped = withoutTrailingComment(text);
|
|
5504
|
+
if (/^\s*[A-Za-z_$][\w$.]*\s*\(/.test(stripped) && !TABLE_RUNNER_RE.test(stripped)) return false;
|
|
4881
5505
|
if (!/[[{]/.test(stripped)) return false;
|
|
4882
5506
|
const literals = stripped.match(/-?\d+(?:\.\d+)?|'[^']*'|"[^"]*"|`[^`]*`|\btrue\b|\bfalse\b|\bTrue\b|\bFalse\b/g);
|
|
4883
5507
|
return (literals?.length ?? 0) >= 2;
|
|
@@ -4915,12 +5539,20 @@ function run23(context) {
|
|
|
4915
5539
|
for (const file of context.files) {
|
|
4916
5540
|
const filePath = pathOf(file);
|
|
4917
5541
|
if (!filePath || !context.isTestFile(filePath)) continue;
|
|
4918
|
-
const hasTableRunner = file.chunks.some(
|
|
4919
|
-
(chunk) => chunk.changes.some((change) => TABLE_RUNNER_RE.test(change.content.replace(/^[+\- ]/, "")))
|
|
4920
|
-
);
|
|
5542
|
+
const hasTableRunner = file.chunks.some((chunk) => TABLE_RUNNER_RE.test(sourceView(chunk, "after", /\.(?:py|rb)$/.test(filePath)).code));
|
|
4921
5543
|
for (const chunk of file.chunks) {
|
|
4922
|
-
const
|
|
4923
|
-
|
|
5544
|
+
const before = sourceView(chunk, "before", /\.(?:py|rb)$/.test(filePath));
|
|
5545
|
+
const after = sourceView(chunk, "after", /\.(?:py|rb)$/.test(filePath));
|
|
5546
|
+
const codeAt = (view, line) => {
|
|
5547
|
+
const offset = view.offsetAtLine(line);
|
|
5548
|
+
if (offset === void 0) return "";
|
|
5549
|
+
const end = view.code.indexOf("\n", offset);
|
|
5550
|
+
return view.code.slice(offset, end < 0 ? void 0 : end);
|
|
5551
|
+
};
|
|
5552
|
+
const removedLines = deletedLines(chunk).filter((line) => codeAt(before, line.line).trim());
|
|
5553
|
+
const additions = addedLines(chunk).filter((line) => codeAt(after, line.line).trim());
|
|
5554
|
+
const removedBindings = removedLines.map((line) => expectedBinding(line.text)).filter((binding) => binding !== void 0);
|
|
5555
|
+
for (const added of additions) {
|
|
4924
5556
|
const binding = expectedBinding(added.text);
|
|
4925
5557
|
if (!binding) continue;
|
|
4926
5558
|
const prior = removedBindings.find((old) => old.name === binding.name && old.value !== binding.value);
|
|
@@ -4934,9 +5566,9 @@ function run23(context) {
|
|
|
4934
5566
|
suggestion: "Change the implementation, not the expected/want binding. If the specification changed, record that reason and update the implementation in the same reviewable change."
|
|
4935
5567
|
});
|
|
4936
5568
|
}
|
|
4937
|
-
const removed =
|
|
5569
|
+
const removed = removedLines.map((l) => ({ line: l, value: expectedLiteral(l.text), key: assertionKey(l.text) })).filter((x) => x.value !== void 0);
|
|
4938
5570
|
if (removed.length > 0) {
|
|
4939
|
-
for (const added of
|
|
5571
|
+
for (const added of additions) {
|
|
4940
5572
|
const value = expectedLiteral(added.text);
|
|
4941
5573
|
if (value === void 0) continue;
|
|
4942
5574
|
const key = assertionKey(added.text);
|
|
@@ -4953,8 +5585,9 @@ function run23(context) {
|
|
|
4953
5585
|
}
|
|
4954
5586
|
}
|
|
4955
5587
|
if (!hasTableRunner) continue;
|
|
4956
|
-
const removedRows =
|
|
4957
|
-
|
|
5588
|
+
const removedRows = removedLines.filter((line) => isDataRow(line.text));
|
|
5589
|
+
const removedTexts = new Set(removedLines.map((line) => line.text.trim()));
|
|
5590
|
+
for (const added of additions.filter((line) => isDataRow(line.text) && !removedTexts.has(line.text.trim()))) {
|
|
4958
5591
|
const pair = removedRows.map((prior) => changedTableExpectation(prior.text, added.text)).find((value) => value !== void 0);
|
|
4959
5592
|
if (!pair) continue;
|
|
4960
5593
|
findings.push({
|
|
@@ -4976,7 +5609,7 @@ var wi109 = { id: "WI109", severity: "error", run: run23 };
|
|
|
4976
5609
|
import path6 from "path";
|
|
4977
5610
|
var TASK_FILE_RE = /(?:^|\/)package\.json$|(?:^|\/)Makefile$|(?:^|\/)Taskfile\.ya?ml$|(?:^|\/)justfile$|(?:^|\/)pyproject\.toml$|(?:^|\/)Rakefile$|(?:^|\/)composer\.json$/i;
|
|
4978
5611
|
var VERIFICATION_SCRIPT_RE = /"(test|tests|check|lint|typecheck|type-check|tsc|verify|validate|ci|coverage|audit)[\w:-]*"\s*:/i;
|
|
4979
|
-
var REAL_VERIFIER_RE = /\b(?:vitest|jest|mocha|ava|pytest|tox|nox|rspec|phpunit|cargo\s+test|go\s+test|dotnet\s+test|mvn|gradle|swift\s+test|mix\s+test|bats|busted|prove|tsc|eslint|ruff|flake8|mypy|pylint|clippy|rubocop|phpstan|stylelint|biome)\b|\
|
|
5612
|
+
var REAL_VERIFIER_RE = /\b(?:vitest|jest|mocha|ava|pytest|tox|nox|rspec|phpunit|cargo\s+test|go\s+test|dotnet\s+test|mvn|gradle|swift\s+test|mix\s+test|bats|busted|prove|tsc|eslint|ruff|flake8|mypy|pylint|clippy|rubocop|phpstan|stylelint|biome)\b|\b(?:npm|pnpm|yarn|bun)\s+(?:run\s+)?(?:test|check|lint|build|typecheck|verify|coverage|audit)[\w:-]*\b|\bnode\s+--test\b/;
|
|
4980
5613
|
var NO_OP_COMMAND_RE = /^\s*(?:echo\b|true\b|:\s*$|exit\s+0\b|#|node\s+-e\s+["']["']|cd\s+\.)/;
|
|
4981
5614
|
var NEUTERING_SIGNATURES = [
|
|
4982
5615
|
{
|
|
@@ -5032,8 +5665,8 @@ var SNAPSHOT_UPDATE_RE = /(?:^|\s)(?:-u|--update|--updateSnapshot)\b/;
|
|
|
5032
5665
|
function verifierTool(text) {
|
|
5033
5666
|
return /\b(vitest|jest|mocha|ava|pytest|tox|nox|rspec|phpunit|cargo\s+test|go\s+test|dotnet\s+test)\b/.exec(text)?.[1] ?? (/\bnpm\s+(?:run\s+)?test\b/.test(text) ? "npm test" : void 0);
|
|
5034
5667
|
}
|
|
5035
|
-
var PROPAGATED_EXIT_RE = /\b(?:process\.exit|sys\.exit|os\.exit|
|
|
5036
|
-
var FORCED_EXIT_ZERO_RE = /\b(?:process\.exit|sys\.exit|os\.exit|
|
|
5668
|
+
var PROPAGATED_EXIT_RE = /\b(?:process\.exit|sys\.exit|os\.(?:exit|_exit|Exit)|(?:std::)?process::exit|System\.exit|Environment\.Exit|exit|die)\s*\(\s*([^)]+?)\s*\)|^\s*exit\s+(\$\{?\w+\}?)\s*(?:#.*)?$/;
|
|
5669
|
+
var FORCED_EXIT_ZERO_RE = /\b(?:process\.exit|sys\.exit|os\.(?:exit|_exit|Exit)|(?:std::)?process::exit|System\.exit|Environment\.Exit|exit|die)\s*\(\s*0\s*\)|^\s*exit\s+0\s*(?:#.*)?$/;
|
|
5037
5670
|
var PROPAGATED_EXIT_CODE_RE = /\bprocess\.exitCode\s*=\s*([^;]+?)\s*;?\s*$/;
|
|
5038
5671
|
var FORCED_EXIT_CODE_ZERO_RE = /\bprocess\.exitCode\s*=\s*0\s*;?\s*$/;
|
|
5039
5672
|
function exitStatus(text) {
|
|
@@ -5045,9 +5678,9 @@ function exitStatus(text) {
|
|
|
5045
5678
|
return (m[1] ?? m[2])?.trim();
|
|
5046
5679
|
}
|
|
5047
5680
|
function laundersStatus(text, status) {
|
|
5048
|
-
const
|
|
5681
|
+
const compact3 = withoutTrailingComment(text).replace(/\s+/g, "");
|
|
5049
5682
|
const value = status.replace(/\s+/g, "");
|
|
5050
|
-
return
|
|
5683
|
+
return compact3.includes(`process.exit(${value}?0:${value})`) || compact3.includes(`process.exitCode=Math.min(${value},0)`);
|
|
5051
5684
|
}
|
|
5052
5685
|
function scriptCommand(text) {
|
|
5053
5686
|
const m = /"[\w:-]+"\s*:\s*"([^"]*)"/.exec(text);
|
|
@@ -5108,7 +5741,7 @@ function run24(context) {
|
|
|
5108
5741
|
});
|
|
5109
5742
|
}
|
|
5110
5743
|
const after = afterLines(chunk);
|
|
5111
|
-
const afterRunsVerification = after.some((line) => REAL_VERIFIER_RE.test(line.text));
|
|
5744
|
+
const afterRunsVerification = after.some((line) => !isCommentLine(line.text) && REAL_VERIFIER_RE.test(line.text));
|
|
5112
5745
|
const failFastShell = after.some((line) => /\bset\s+-[a-z]*e\b/.test(line.text));
|
|
5113
5746
|
const backgroundStatusPropagated = after.some((line) => /^\s*wait\b/.test(line.text)) && after.some((line) => /^\s*exit\s+\$\?\s*$/.test(line.text));
|
|
5114
5747
|
for (const line of added) {
|
|
@@ -5238,7 +5871,29 @@ function run24(context) {
|
|
|
5238
5871
|
});
|
|
5239
5872
|
if (!newEntry) continue;
|
|
5240
5873
|
const newCommand = scriptCommand(newEntry.text) ?? "";
|
|
5241
|
-
const
|
|
5874
|
+
const entries = new Map(file.chunks.flatMap(afterLines).flatMap((line) => scriptEntries(line.text)).map((entry) => [entry.name, entry.command]));
|
|
5875
|
+
const expand2 = (command, seen = /* @__PURE__ */ new Set()) => {
|
|
5876
|
+
let expanded = command;
|
|
5877
|
+
for (const call of command.matchAll(/\b(?:npm|pnpm|yarn|bun)\s+(?:run\s+)?([\w:-]+)/g)) {
|
|
5878
|
+
const name = call[1];
|
|
5879
|
+
if (seen.has(name) || seen.size >= 20) continue;
|
|
5880
|
+
const target = entries.get(name);
|
|
5881
|
+
if (target) expanded += " && " + expand2(target, /* @__PURE__ */ new Set([...seen, name]));
|
|
5882
|
+
}
|
|
5883
|
+
return expanded;
|
|
5884
|
+
};
|
|
5885
|
+
const expandedCommand = expand2(newCommand);
|
|
5886
|
+
const dropped = oldSteps.filter((s) => {
|
|
5887
|
+
const invocation = REAL_VERIFIER_RE.exec(s);
|
|
5888
|
+
if (!invocation || expandedCommand.includes(s)) return false;
|
|
5889
|
+
const tail = s.slice(invocation.index);
|
|
5890
|
+
if (expandedCommand.includes(tail)) return false;
|
|
5891
|
+
const testTool = (command) => /\b(?:vitest|jest|mocha|ava|pytest|tox|rspec|phpunit)\b|\bnode\s+--test\b/.exec(command)?.[0];
|
|
5892
|
+
const priorTool = testTool(s);
|
|
5893
|
+
const nextTool = testTool(newCommand);
|
|
5894
|
+
const importsMigrated = context.files.some((candidate) => candidate.chunks.some((change) => deletedLines(change).some((line) => /\bimport\b|\brequire\s*\(/.test(line.text) && !!priorTool && line.text.includes(priorTool)) && addedLines(change).some((line) => /\bimport\b|\brequire\s*\(/.test(line.text) && /['"](?:node:test|vitest|jest|mocha|ava)['"]/.test(line.text))));
|
|
5895
|
+
return !(priorTool && nextTool && priorTool !== nextTool && importsMigrated);
|
|
5896
|
+
});
|
|
5242
5897
|
if (dropped.length === 0) continue;
|
|
5243
5898
|
findings.push({
|
|
5244
5899
|
verifierId: "WI110",
|
|
@@ -5543,7 +6198,7 @@ var VERIFIERS = [
|
|
|
5543
6198
|
];
|
|
5544
6199
|
|
|
5545
6200
|
// src/engine.ts
|
|
5546
|
-
import
|
|
6201
|
+
import picomatch5 from "picomatch";
|
|
5547
6202
|
var norm = (p) => p.replace(/\\/g, "/");
|
|
5548
6203
|
async function runChecks(context) {
|
|
5549
6204
|
const activeVerifiers = VERIFIERS.filter((v) => context.enabled.includes(v.id));
|
|
@@ -5583,7 +6238,7 @@ function applySuppression(findings, files) {
|
|
|
5583
6238
|
var GLOB_OPTIONS = { dot: true };
|
|
5584
6239
|
function applyIgnorePatterns(findings, patterns) {
|
|
5585
6240
|
if (patterns.length === 0 || findings.length === 0) return findings;
|
|
5586
|
-
const matchesIgnoredPath =
|
|
6241
|
+
const matchesIgnoredPath = picomatch5(patterns, GLOB_OPTIONS);
|
|
5587
6242
|
return findings.filter((f) => !matchesIgnoredPath(norm(f.file)));
|
|
5588
6243
|
}
|
|
5589
6244
|
function applyApprovals(findings, approvals) {
|
|
@@ -5591,7 +6246,7 @@ function applyApprovals(findings, approvals) {
|
|
|
5591
6246
|
return findings.map((f) => {
|
|
5592
6247
|
const path9 = norm(f.file);
|
|
5593
6248
|
const match = approvals.find(
|
|
5594
|
-
(a) => a.rule === f.verifierId && (norm(a.file) === path9 ||
|
|
6249
|
+
(a) => a.rule === f.verifierId && (norm(a.file) === path9 || picomatch5(norm(a.file), GLOB_OPTIONS)(path9))
|
|
5595
6250
|
);
|
|
5596
6251
|
if (!match) return f;
|
|
5597
6252
|
return { ...f, severity: "info", approved: true, approvalReason: match.reason };
|
|
@@ -7181,7 +7836,7 @@ async function readStdin() {
|
|
|
7181
7836
|
}
|
|
7182
7837
|
var program = new Command();
|
|
7183
7838
|
program.name("proctor").description("AI agent test-tampering guard").version(package_default.version);
|
|
7184
|
-
program.command("check [path]").description("Check staged, unstaged, and untracked changes for test tampering").option("--staged", "analyze only staged changes").option("--uncommitted", "explicitly analyze the default scope: staged, unstaged, and untracked changes together").option("--base <ref>", "analyze changes against a base ref (e.g. origin/main or a commit SHA) instead of staged/working-tree changes, for CI, where nothing is staged in a fresh checkout").option("--ci", "show only errors in the text report; warnings do not affect the exit code").option("--json", "output findings as JSON to stdout").option("--sarif", "output SARIF 2.1.0 JSON to stdout").option("--ai", "enable the networked LLM judge (requires ANTHROPIC_API_KEY)").option("--rules <ids>", "comma-separated list of verifier IDs to run (narrows the enabled set, e.g. RH001,RH003)").option("--wi", "also run the
|
|
7839
|
+
program.command("check [path]").description("Check staged, unstaged, and untracked changes for test tampering").option("--staged", "analyze only staged changes").option("--uncommitted", "explicitly analyze the default scope: staged, unstaged, and untracked changes together").option("--base <ref>", "analyze changes against a base ref (e.g. origin/main or a commit SHA) instead of staged/working-tree changes, for CI, where nothing is staged in a fresh checkout").option("--ci", "show only errors in the text report; warnings do not affect the exit code").option("--json", "output findings as JSON to stdout").option("--sarif", "output SARIF 2.1.0 JSON to stdout").option("--ai", "enable the networked LLM judge (requires ANTHROPIC_API_KEY)").option("--rules <ids>", "comma-separated list of verifier IDs to run (narrows the enabled set, e.g. RH001,RH003)").option("--wi", "also run the remaining opt-in work-integrity checks (WI1xx)").option("--all-checks", "alias for --wi: add the entire WI family to the configured checks").option("--explain <id>", "print the full explanation for a verifier ID and exit, no diff analysis").option("--fix", "with --explain, print what an honest fix looks like instead of the rule description").option("--markdown <file>", 'also write a Markdown summary to this file, e.g. --markdown "$GITHUB_STEP_SUMMARY"').action(async (pathArg, options) => {
|
|
7185
7840
|
if (options.fix && !options.explain) {
|
|
7186
7841
|
process.stderr.write("proctor: --fix only applies with --explain, e.g. proctor check --explain RH001 --fix\n");
|
|
7187
7842
|
process.exit(2);
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@kavishdua/proctor",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.1.0",
|
|
4
4
|
"description": "Catch AI agents gaming their test suites: deleted tests, skipped tests, weakened assertions, hardcoded outputs",
|
|
5
5
|
"homepage": "https://github.com/catfish-1234/proctor#readme",
|
|
6
6
|
"bugs": {
|