agent-roast 0.21.0 → 0.23.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +6 -6
- package/dist/{chunk-XVV5FNDV.js → chunk-L4XLCXC2.js} +64 -38
- package/dist/cli.js +12 -9
- package/dist/index.d.ts +1 -1
- package/dist/index.js +1 -1
- package/package.json +4 -2
package/README.md
CHANGED
|
@@ -37,7 +37,7 @@ npx agent-roast # review what you have not committed yet (on a clean tr
|
|
|
37
37
|
## Stop your coding agent from doing it
|
|
38
38
|
|
|
39
39
|
```bash
|
|
40
|
-
npx agent-roast hook install # Claude Code, Codex and
|
|
40
|
+
npx agent-roast hook install # Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot and Factory Droid
|
|
41
41
|
```
|
|
42
42
|
|
|
43
43
|
For Claude Code there is also a plugin: `/plugin install agent-roast --marketplace Nari-Ab/agent-roast`. Any agent that speaks
|
|
@@ -112,7 +112,7 @@ Measured, with the denominators and the caveats in [docs/benchmarks.md](docs/ben
|
|
|
112
112
|
deliberate, and not specific to AI. agent-roast is a tripwire that asks about them. On repositories it had not seen, between a quarter and
|
|
113
113
|
two thirds of its blocks were questions with a harmless answer, depending on the sample.**
|
|
114
114
|
|
|
115
|
-
- **
|
|
115
|
+
- **164 of 164** hand-labeled cheating edits are caught and **153 of 153** harmless look-alikes (reformats, moves, raised
|
|
116
116
|
thresholds, environment-gated skips) stay silent. The cases were written with the detector, so they guard against regressions;
|
|
117
117
|
they do not prove accuracy. One gap is documented and still open.
|
|
118
118
|
- **A catalog of cheats that someone else wrote.** Of the nine hacks in [reward-hack-zoo](https://github.com/MatthewWaller/reward-hack-zoo) (hand-written specimens; none from a model) four are reported (a skip, a patched target, a `conftest.py` that rewrites reports,
|
|
@@ -122,8 +122,8 @@ two thirds of its blocks were questions with a harmless answer, depending on the
|
|
|
122
122
|
(Claude, Copilot, Cursor, Codex, Devin, aider) and on 0.3% of the others. Inside the same repositories the odds for an
|
|
123
123
|
AI-signed commit were 0.81 times those of another commit (95% CI 0.40–1.67). There is no sign that AI-signed commits weaken
|
|
124
124
|
checks more often.
|
|
125
|
-
- **On repositories it was not written from, between a quarter and two thirds of the blocked commits were false alarms.**
|
|
126
|
-
or 40 public repositories (
|
|
125
|
+
- **On repositories it was not written from, between a quarter and two thirds of the blocked commits were false alarms.** Ten samples of 39
|
|
126
|
+
or 40 public repositories (396 in all, 113,401 commits) were replayed and every blocked commit was read. The share was 69% for 0.17.0 on the
|
|
127
127
|
first sample and on the fourth, 51% for 0.18.0 on the fourth and 27% on the fifth, which it had not seen either. This version, frozen before the
|
|
128
128
|
sixth sample was drawn, blocked 100 of its 11,335 commits: 34 changed a check a reviewer should see (an assertion that got weaker, a gate
|
|
129
129
|
that was relaxed, a test switched off or left focused) and 66 were false alarms, but 51 of the 66 were one defect in one repository, an array of
|
|
@@ -131,8 +131,8 @@ two thirds of its blocks were questions with a harmless answer, depending on the
|
|
|
131
131
|
test that asserts nothing counts as a block for a reviewer; counted as false alarms, 13 of 33 on the fifth sample and 72 of 100 on the sixth. Most of the rest, in the
|
|
132
132
|
earlier measurements too, were deliberate and explained. One reader (an AI assistant) sorted them, and the fixes were made from the same
|
|
133
133
|
sorting, so treat the figures as a first estimate. The published 0.19.0 blocked 31 of the 12,000 commits of the seventh sample: 16 for a reviewer and 15 false alarms (48%, 6 of them one
|
|
134
|
-
repository's lint configuration); the build with the fixes from reading it, tried on an eighth sample that it had not seen, blocked 26 of 11,081 commits: 16 for a reviewer and 10 false alarms (38%), and the fixes from reading that one are in this release too;
|
|
135
|
-
[section 10](docs/benchmarks.md#10-two-more-samples-and-what-the-fixes-from-sample-4-did), [section 11](docs/benchmarks.md#11-a-seventh-sample-and-two-things-that-were-missing), [section 12](docs/benchmarks.md#12-an-eighth-sample-the-first-measurement-of-the-fixes-from-sample-7), [section 14](docs/benchmarks.md#14-a-ninth-sample-the-shipped-build-on-repositories-it-had-not-seen)).
|
|
134
|
+
repository's lint configuration); the build with the fixes from reading it, tried on an eighth sample that it had not seen, blocked 26 of 11,081 commits: 16 for a reviewer and 10 false alarms (38%), and the fixes from reading that one are in this release too; 0.21.0 as it was frozen blocked 46 of 11,286 commits of a ninth sample that it had not seen: 28 for a reviewer and 18 false alarms (39%), and five of those are fixed in it; 0.21.0 as it shipped blocked 40 of the 11,191 commits of a tenth sample that it had not seen either: 27 for a reviewer and 13 false alarms (33%), and three of those 13 are fixed in this release ([section 9](docs/benchmarks.md#9-repositories-nobody-had-tuned-on),
|
|
135
|
+
[section 10](docs/benchmarks.md#10-two-more-samples-and-what-the-fixes-from-sample-4-did), [section 11](docs/benchmarks.md#11-a-seventh-sample-and-two-things-that-were-missing), [section 12](docs/benchmarks.md#12-an-eighth-sample-the-first-measurement-of-the-fixes-from-sample-7), [section 14](docs/benchmarks.md#14-a-ninth-sample-the-shipped-build-on-repositories-it-had-not-seen), [section 15](docs/benchmarks.md#15-a-tenth-sample-the-build-that-ships-on-repositories-it-had-not-seen), [section 16](docs/benchmarks.md#16-what-sample-10-led-to-three-fixes-fitted-and-a-guard-that-did-not-pay)).
|
|
136
136
|
- **9,732 commits compared in full** from 164 public repositories that use coding agents (5,454 of them AI-signed) were reviewed again for the
|
|
137
137
|
last two releases, and every new finding was read. 0.15.0's rules reported 30 new findings there and 7 were false alarms (fixed in 0.16.0);
|
|
138
138
|
0.17.0's rule for an exact check turned into a bound reported 18 in 14 commits, all of them loosened checks, after two false alarms were
|
|
@@ -1508,6 +1508,7 @@ function isCodeFile(filePath, customIgnorePaths) {
|
|
|
1508
1508
|
function isTestFile(filePath) {
|
|
1509
1509
|
return TEST_PATH_REGEX.test(filePath);
|
|
1510
1510
|
}
|
|
1511
|
+
var PYTHON_TEST_PATH = /(?:^|[/._-])(?:tests?|testing|specs?|conftest)(?:[/._-]|$)/i;
|
|
1511
1512
|
function auditFileDiffs(files, commitHash = "HEAD", commitDate = (/* @__PURE__ */ new Date()).toISOString(), options = {}) {
|
|
1512
1513
|
const infractions = [];
|
|
1513
1514
|
let hasSupportedChanges = false;
|
|
@@ -1538,7 +1539,7 @@ function auditFileDiffs(files, commitHash = "HEAD", commitDate = (/* @__PURE__ *
|
|
|
1538
1539
|
for (const start of masked.templateStarts) testCode[start] = "";
|
|
1539
1540
|
maskedLines.push({ code: cleanedLine, testCode: testCode.join(""), raw: rawLine, number: added.lineNumber });
|
|
1540
1541
|
const isComment = !cleanedLine.trim();
|
|
1541
|
-
if (python2 && !isComment) {
|
|
1542
|
+
if (python2 && !isComment && PYTHON_TEST_PATH.test(file.path)) {
|
|
1542
1543
|
for (const rule of PYTHON_TEST_SKIP_PATTERNS) {
|
|
1543
1544
|
if (rule.regex.test(cleanedLine)) {
|
|
1544
1545
|
if (rule.imperative && isGuardedSkip(maskedLines)) break;
|
|
@@ -1644,7 +1645,7 @@ function auditFileDiffs(files, commitHash = "HEAD", commitDate = (/* @__PURE__ *
|
|
|
1644
1645
|
}
|
|
1645
1646
|
if (python2) {
|
|
1646
1647
|
const reportedSkips = new Set(infractions.slice(firstFinding).filter((f) => f.type === "test-skip").map((f) => f.lineNumber));
|
|
1647
|
-
for (const match of source.matchAll(PYTHON_MARKER)) {
|
|
1648
|
+
for (const match of PYTHON_TEST_PATH.test(file.path) ? source.matchAll(PYTHON_MARKER) : []) {
|
|
1648
1649
|
const xfail = match[2] === "xfail";
|
|
1649
1650
|
let open2 = match.index + match[0].length;
|
|
1650
1651
|
while (source[open2] === " " || source[open2] === " ") open2++;
|
|
@@ -4407,11 +4408,13 @@ var THRESHOLD_PATTERNS = [
|
|
|
4407
4408
|
var SWITCHES = ["strict", "noImplicitAny", "strictNullChecks", "noImplicitReturns", "useUnknownInCatchVariables"];
|
|
4408
4409
|
var GATE_SCRIPT = /^(?:pre|post)?(?:test|lint|typecheck|type-check|check|ci|verify|validate)(?::[\w:.-]+)?$/;
|
|
4409
4410
|
var CI_RUNNER = new RegExp(
|
|
4410
|
-
String.raw`(?<![\w./-])(?:pytest|vitest|jest|mocha|tox|nox)(?![\w/-])` + String.raw`|(?<![\w./-])(?:npm|yarn|pnpm|bun|go|cargo|mvn|gradle|make|dotnet|flutter)\s+(?:run\s+)?(?:-\S+\s+)*test(?![\w/])` + // A package script that runs the tests with coverage: `yarn coverage`, `npm run coverage`.
|
|
4411
|
-
String.raw`|(?<![\w./-])(?:npm|yarn|pnpm|bun)\s+(?:run\s+)?
|
|
4411
|
+
String.raw`(?<![\w./-])(?:pytest|vitest|jest|mocha|tox|nox)(?![\w/-])` + String.raw`|(?<![\w./-])(?:npm|yarn|pnpm|bun|go|cargo|mvn|gradle|make|dotnet|flutter)\s+(?:run\s+)?(?:-\S+\s+)*test(?![\w/])` + // A package script that runs the tests with coverage: `yarn coverage`, `npm run coverage`, `npm run cover`.
|
|
4412
|
+
String.raw`|(?<![\w./-])(?:npm|yarn|pnpm|bun)\s+(?:run\s+)?cover(?:age)?(?![\w/-])` + // Monorepo runners: `nx affected --target=test`, `turbo run test`, `lerna run test`, `bazel test //...`.
|
|
4412
4413
|
String.raw`|(?<![\w./-])(?:nx|turbo|lerna|rush)\s+(?:run-many|affected|run|test)\b[^\n]*(?<![\w-])(?:test|tests)(?![\w-])` + String.raw`|(?<![\w./-])(?:bazel\s+test|ctest|phpunit|rspec|deno\s+test)(?![\w/-])` + // Python's own runners and the browser runners: `python3 -m unittest`, `nosetests`, `playwright test`, `cypress run`, `karma start`.
|
|
4413
4414
|
String.raw`|(?<![\w./-])python[\d.]*\s+-m\s+(?:unittest|pytest|nose2?)(?![\w/-])` + String.raw`|(?<![\w./-])(?:nosetests|nose2|py\.test|stestr|behave|jasmine|playwright\s+test|cypress\s+run|karma\s+start|hatch\s+(?:run\s+)?test|swift\s+test)(?![\w/-])` + // Task runners: `deno task test`, `just test`, `rake test`, `composer test`, `mix test`, `sbt test`, `./gradlew test`, and Node's own `node --test`.
|
|
4414
|
-
String.raw`|(?<![\w./-])(?:deno\s+(?:task|run)|just|task|poe|rake|composer(?:\s+run)?|mix|sbt|lein|stack|cabal|(?:\./)?(?:gradlew|mvnw)(?:\.bat|\.cmd)?)\s+(?:-\S+\s+)*test(?![\w/-])` + String.raw`|(?<![\w./-])(?:node|tsx)(?:\s+-\S+)*\s+--test(?![\w-])` + // A
|
|
4415
|
+
String.raw`|(?<![\w./-])(?:deno\s+(?:task|run)|just|task|poe|rake|composer(?:\s+run)?|mix|sbt|lein|stack|cabal|(?:\./)?(?:gradlew|mvnw)(?:\.bat|\.cmd)?)\s+(?:-\S+\s+)*test(?![\w/-])` + String.raw`|(?<![\w./-])(?:node|tsx)(?:\s+-\S+)*\s+--test(?![\w-])` + // A target of a task runner that has the word as one of its own words and not as the first: `make unit-test`, `just test-asyncio py314`, `pnpm run docs:test`. A target that sets up, installs, builds or type-checks
|
|
4416
|
+
// (`make setup-tests`, `pnpm run typecheck:tests`, `pnpm run build:native:test`) runs no tests.
|
|
4417
|
+
String.raw`|(?<![\w./-])(?:make|just|task|(?:npm|pnpm|bun)\s+run|yarn(?:\s+run)?|pnpm)\s+(?:-\S+\s+)*(?![\w.:-]*(?:setup|teardown|install|deps|typecheck|lint|build|docker|clean|prepare|publish|deploy|release|format|image|download|data))[\w.:-]*(?<![A-Za-z0-9])tests?(?![A-Za-z0-9])` + // A project's own test script, run through its interpreter or by path: `python scripts/run_tests.py`, `./scripts/test.sh`, `python tests/test_cli.py`.
|
|
4415
4418
|
String.raw`|(?<![\w./-])(?:(?:python[\d.]*|node|bash|sh|zsh|ruby|pwsh|tsx|ts-node)(?:\s+-\S+)*\s+|\./)(?:[\w.-]+/)*(?:(?:run[_-]?)?tests?(?:[_-](?:all|suite|runner|unit|ci))?|test_[\w-]+|[\w-]+_test)\.(?:py|sh|bash|js|mjs|cjs|ts|rb|ps1)(?![\w-])` + String.raw`|\brun:.*(?<=\s)(?:tests?|unit|e2e)(?![\w/.-])`
|
|
4416
4419
|
);
|
|
4417
4420
|
var SHELL_TEST = /(?:^|run:\s*|[;&|({]\s*|\b(?:if|then|do|else|elif|while|until)\s+|!\s+)test\s+!?\s*(?:-[bcdefghkLnOGprsStuwxzN](?=\s)|["$'])/g;
|
|
@@ -4531,12 +4534,16 @@ function ciTestSteps(file, oldText, newText) {
|
|
|
4531
4534
|
const after = commandsAfter.length;
|
|
4532
4535
|
return { file, before: commandsBefore.length, after, reusableAfter: matchingLines(now, CI_REUSABLE), ungatedAfter: after === 0 ? ciTestLines(now).length : 0, commandsBefore, commandsAfter };
|
|
4533
4536
|
}
|
|
4534
|
-
function
|
|
4537
|
+
function gatingCalls(file, text) {
|
|
4538
|
+
if (!(/\.gitlab-ci/.test(file) || CI_GATE.test(gatingTriggers(text) ?? text))) return [];
|
|
4539
|
+
return [...text.matchAll(/^\s*-?\s*uses:\s*["']?\.\/(\.github\/workflows\/[^\s"'#@]+\.ya?ml)/gm)].map((match) => match[1]);
|
|
4540
|
+
}
|
|
4541
|
+
function resolveCiTestSteps(counts, commit, elsewhere = /* @__PURE__ */ new Set(), called = /* @__PURE__ */ new Set()) {
|
|
4535
4542
|
const before = counts.reduce((sum, item) => sum + item.before, 0);
|
|
4536
4543
|
const after = counts.reduce((sum, item) => sum + item.after + item.reusableAfter, 0);
|
|
4537
4544
|
if (after >= before) return [];
|
|
4538
4545
|
const running = /* @__PURE__ */ new Set([...counts.flatMap((item) => item.commandsAfter ?? []), ...elsewhere]);
|
|
4539
|
-
return counts.filter((item) => item.before > 0 && item.after === 0 && item.reusableAfter === 0).filter((item) => item.commandsBefore === void 0 || item.commandsBefore.some((command) => !running.has(command))).map((item) => item.ungatedAfter > 0 ? finding("guard-weakened", item.file, 1, "tests no longer run on push or pull_request", "Workflow keeps its tests but no longer runs on pushes or pull requests", commit) : finding("guard-weakened", item.file, 1, "test command no longer runs in CI", "Test step removed from CI", commit));
|
|
4546
|
+
return counts.filter((item) => item.before > 0 && item.after === 0 && item.reusableAfter === 0).filter((item) => !(item.ungatedAfter > 0 && called.has(item.file))).filter((item) => item.commandsBefore === void 0 || item.commandsBefore.some((command) => !running.has(command))).map((item) => item.ungatedAfter > 0 ? finding("guard-weakened", item.file, 1, "tests no longer run on push or pull_request", "Workflow keeps its tests but no longer runs on pushes or pull requests", commit) : finding("guard-weakened", item.file, 1, "test command no longer runs in CI", "Test step removed from CI", commit));
|
|
4540
4547
|
}
|
|
4541
4548
|
var LENIENT = /continue-on-error:\s*true|allow_failure:\s*true|^\s*(?:-\s+)?if:\s*(?:\$\{\{\s*)?false\s*(?:\}\})?\s*(?:#.*)?$/;
|
|
4542
4549
|
var SHELL_SWALLOW = /\|\|\s*(?:true|exit\s+0|:|echo\b)|;\s*(?:true|exit\s+0)\b/;
|
|
@@ -4792,6 +4799,7 @@ async function compareSnapshot(options, cwd, snapshot) {
|
|
|
4792
4799
|
const tamperingFiles = [];
|
|
4793
4800
|
const guardFindings = [];
|
|
4794
4801
|
const ciCounts = [];
|
|
4802
|
+
const calledInChange = /* @__PURE__ */ new Set();
|
|
4795
4803
|
let productionChanged = false;
|
|
4796
4804
|
const decode = (content) => {
|
|
4797
4805
|
totalBytes += content.length;
|
|
@@ -4830,6 +4838,7 @@ async function compareSnapshot(options, cwd, snapshot) {
|
|
|
4830
4838
|
guardFindings.push(...analyzeGuardFile(change.newPath, oldText, newText, refs.headSha));
|
|
4831
4839
|
const ci = ciTestSteps(change.newPath, oldText, newText);
|
|
4832
4840
|
if (ci) ciCounts.push(ci);
|
|
4841
|
+
if (ci && newText !== null) for (const workflow of gatingCalls(change.newPath, newText)) calledInChange.add(workflow);
|
|
4833
4842
|
result.filesInspected++;
|
|
4834
4843
|
} catch (error) {
|
|
4835
4844
|
assertNotAborted(options.signal);
|
|
@@ -4887,14 +4896,16 @@ async function compareSnapshot(options, cwd, snapshot) {
|
|
|
4887
4896
|
await snapshot?.verify();
|
|
4888
4897
|
result.complete = result.skippedFiles.length === 0;
|
|
4889
4898
|
const rules = policy.config.rules ?? {};
|
|
4890
|
-
const elsewhere = /* @__PURE__ */ new Set();
|
|
4899
|
+
const elsewhere = /* @__PURE__ */ new Set(), called = new Set(calledInChange);
|
|
4891
4900
|
if (ciCounts.some((item) => item.before > 0 && item.after === 0 && item.reusableAfter === 0)) {
|
|
4892
4901
|
const touched = new Set(changes.flatMap((change) => [change.oldPath, change.newPath]));
|
|
4893
4902
|
const tree = /^[a-f0-9]{40,64}$/.test(refs.headSha) ? refs.headSha : refs.baseSha;
|
|
4894
4903
|
try {
|
|
4895
4904
|
for (const file of (await ciFilesAt(cwd, tree, options.signal)).filter((candidate) => !touched.has(candidate.path)).slice(0, 100)) {
|
|
4896
4905
|
try {
|
|
4897
|
-
|
|
4906
|
+
const text = new TextDecoder().decode(await readBlob(cwd, file.oid, options.signal, 512 * 1024));
|
|
4907
|
+
for (const command of enforcedTestCommands(file.path, text)) elsewhere.add(command);
|
|
4908
|
+
for (const workflow of gatingCalls(file.path, text)) called.add(workflow);
|
|
4898
4909
|
} catch (error) {
|
|
4899
4910
|
assertNotAborted(options.signal);
|
|
4900
4911
|
}
|
|
@@ -4903,7 +4914,7 @@ async function compareSnapshot(options, cwd, snapshot) {
|
|
|
4903
4914
|
assertNotAborted(options.signal);
|
|
4904
4915
|
}
|
|
4905
4916
|
}
|
|
4906
|
-
result.infractions.push(...[...resolveTampering(tamperingFiles, productionChanged), ...guardFindings, ...resolveCiTestSteps(ciCounts, refs.headSha, elsewhere)].filter((finding2) => rules[finding2.type] !== "off"));
|
|
4917
|
+
result.infractions.push(...[...resolveTampering(tamperingFiles, productionChanged), ...guardFindings, ...resolveCiTestSteps(ciCounts, refs.headSha, elsewhere, called)].filter((finding2) => rules[finding2.type] !== "off"));
|
|
4907
4918
|
const exceptions = applyExceptions(result.infractions, options.ignoreExceptions ? [] : policy.config.exceptions, /* @__PURE__ */ new Date(), existingCounts);
|
|
4908
4919
|
result.infractions = exceptions.findings;
|
|
4909
4920
|
result.suppressedFindings = exceptions.suppressedFindings;
|
|
@@ -5041,13 +5052,14 @@ function formatReplayTerminal(result, allRules = false) {
|
|
|
5041
5052
|
// src/hook.ts
|
|
5042
5053
|
import path8 from "path";
|
|
5043
5054
|
import * as fs7 from "fs/promises";
|
|
5044
|
-
var HOOK_AGENTS = ["claude", "codex", "cursor"];
|
|
5055
|
+
var HOOK_AGENTS = ["claude", "codex", "cursor", "gemini", "copilot", "droid"];
|
|
5045
5056
|
var DEFAULT_TIMEOUT_MS = 25e3;
|
|
5046
5057
|
var PAUSE_FILE = "agent-roast-hook-paused";
|
|
5047
5058
|
var LISTED = 6;
|
|
5048
5059
|
var LISTED_WARNINGS = 3;
|
|
5049
5060
|
function detectAgent(input) {
|
|
5050
5061
|
if ("workspace_roots" in input || "cursor_version" in input || "conversation_id" in input) return "cursor";
|
|
5062
|
+
if (input.hook_event_name === "AfterAgent" || "prompt_response" in input) return "gemini";
|
|
5051
5063
|
if ("turn_id" in input) return "codex";
|
|
5052
5064
|
return "claude";
|
|
5053
5065
|
}
|
|
@@ -5055,7 +5067,7 @@ function inputCwd(agent, input, env) {
|
|
|
5055
5067
|
const roots = input.workspace_roots;
|
|
5056
5068
|
if (agent === "cursor" && Array.isArray(roots) && typeof roots[0] === "string") return roots[0];
|
|
5057
5069
|
if (typeof input.cwd === "string" && input.cwd) return input.cwd;
|
|
5058
|
-
return env.CLAUDE_PROJECT_DIR || env.CURSOR_PROJECT_DIR || process.cwd();
|
|
5070
|
+
return env.CLAUDE_PROJECT_DIR || env.CURSOR_PROJECT_DIR || env.GEMINI_PROJECT_DIR || env.FACTORY_PROJECT_DIR || process.cwd();
|
|
5059
5071
|
}
|
|
5060
5072
|
function clip(text, limit) {
|
|
5061
5073
|
const flat = terminalText(text).replace(/\s+/g, " ").trim();
|
|
@@ -5133,8 +5145,8 @@ async function runHook(rawInput, options = {}) {
|
|
|
5133
5145
|
if (["off", "0", "false"].includes(mode)) return outcome("skip");
|
|
5134
5146
|
const advisory = options.advisory === true || mode === "advisory";
|
|
5135
5147
|
const agent = options.agent ?? detectAgent(input);
|
|
5136
|
-
const
|
|
5137
|
-
if (
|
|
5148
|
+
const finishing = agent === "gemini" ? "afteragent" : "stop";
|
|
5149
|
+
if (String(input.hook_event_name ?? finishing).toLowerCase() !== finishing) return outcome("skip");
|
|
5138
5150
|
if (agent === "cursor" && typeof input.status === "string" && input.status !== "completed") return outcome("skip");
|
|
5139
5151
|
const cwd = options.cwd ?? inputCwd(agent, input, env);
|
|
5140
5152
|
if (await hookPausedUntil(cwd, options.now)) return outcome("skip");
|
|
@@ -5150,15 +5162,17 @@ async function runHook(rawInput, options = {}) {
|
|
|
5150
5162
|
} finally {
|
|
5151
5163
|
clearTimeout(timer);
|
|
5152
5164
|
}
|
|
5153
|
-
if (!result.failedRules.length) return outcome("pass", agent === "cursor" ? "{}\n" : "", "", result);
|
|
5165
|
+
if (!result.failedRules.length) return outcome("pass", agent === "cursor" || agent === "gemini" ? "{}\n" : "", "", result);
|
|
5154
5166
|
const alreadyAsked = input.stop_hook_active === true || typeof input.loop_count === "number" && input.loop_count > 0;
|
|
5155
5167
|
if (alreadyAsked || advisory) {
|
|
5156
5168
|
const warning = formatUserWarning(result);
|
|
5157
|
-
|
|
5169
|
+
const notice = agent === "copilot" ? { type: "progress", message: warning } : { systemMessage: warning };
|
|
5170
|
+
return outcome("warn", agent === "cursor" ? "{}\n" : JSON.stringify(notice) + "\n", agent === "cursor" && advisory ? `${warning}
|
|
5158
5171
|
` : "", result);
|
|
5159
5172
|
}
|
|
5160
5173
|
const message = formatAgentFeedback(result);
|
|
5161
|
-
|
|
5174
|
+
const answer = agent === "cursor" ? { followup_message: message } : { decision: agent === "gemini" ? "deny" : "block", reason: message };
|
|
5175
|
+
return outcome("block", JSON.stringify(answer) + "\n", "", result);
|
|
5162
5176
|
}
|
|
5163
5177
|
|
|
5164
5178
|
// src/hook-install.ts
|
|
@@ -5168,8 +5182,9 @@ import path9 from "path";
|
|
|
5168
5182
|
import { randomUUID as randomUUID4 } from "crypto";
|
|
5169
5183
|
import { createRequire as createRequire2 } from "module";
|
|
5170
5184
|
import { spawnSync as spawnSync3 } from "child_process";
|
|
5171
|
-
var AGENT_LABELS = { claude: "Claude Code", codex: "Codex", cursor: "Cursor" };
|
|
5185
|
+
var AGENT_LABELS = { claude: "Claude Code", codex: "Codex", cursor: "Cursor", gemini: "Gemini CLI", copilot: "GitHub Copilot", droid: "Factory Droid" };
|
|
5172
5186
|
var TIMEOUT_SECONDS = 60;
|
|
5187
|
+
var FINISH_EVENT = { claude: "Stop", codex: "Stop", cursor: "stop", gemini: "AfterAgent", copilot: "Stop", droid: "Stop" };
|
|
5173
5188
|
function packageVersion() {
|
|
5174
5189
|
const require3 = createRequire2(import.meta.url);
|
|
5175
5190
|
return String(require3("../package.json").version);
|
|
@@ -5196,6 +5211,9 @@ function hookConfigPath(agent, scope, root, home, env = process.env) {
|
|
|
5196
5211
|
const claude = path9.join(scope === "global" ? home : root, ".claude");
|
|
5197
5212
|
if (agent === "claude") return path9.join(claude, scope === "local" ? "settings.local.json" : "settings.json");
|
|
5198
5213
|
if (agent === "cursor") return path9.join(scope === "global" ? home : root, ".cursor", "hooks.json");
|
|
5214
|
+
if (agent === "gemini") return path9.join(scope === "global" ? home : root, ".gemini", "settings.json");
|
|
5215
|
+
if (agent === "copilot") return path9.join(scope === "global" ? env.COPILOT_HOME || path9.join(home, ".copilot") : path9.join(root, ".github"), "hooks", "agent-roast.json");
|
|
5216
|
+
if (agent === "droid") return path9.join(scope === "global" ? home : root, ".factory", "hooks.json");
|
|
5199
5217
|
return path9.join(scope === "global" ? env.CODEX_HOME || path9.join(home, ".codex") : path9.join(root, ".codex"), "hooks.json");
|
|
5200
5218
|
}
|
|
5201
5219
|
function readJsonFile(file) {
|
|
@@ -5231,40 +5249,47 @@ function writeJsonFile(link, json) {
|
|
|
5231
5249
|
}
|
|
5232
5250
|
}
|
|
5233
5251
|
var asArray = (value) => Array.isArray(value) ? value : [];
|
|
5252
|
+
var isObject = (value) => Boolean(value) && typeof value === "object" && !Array.isArray(value);
|
|
5253
|
+
function eventMap(agent, data, create) {
|
|
5254
|
+
if (agent === "droid") return data;
|
|
5255
|
+
if (create && !isObject(data.hooks)) data.hooks = {};
|
|
5256
|
+
return isObject(data.hooks) ? data.hooks : void 0;
|
|
5257
|
+
}
|
|
5258
|
+
var isFlat = (agent) => agent === "cursor" || agent === "copilot";
|
|
5234
5259
|
function addStopHook(agent, data, command) {
|
|
5235
5260
|
const before = JSON.stringify(data);
|
|
5236
|
-
|
|
5237
|
-
if (agent
|
|
5261
|
+
const events = eventMap(agent, data, true), event = FINISH_EVENT[agent];
|
|
5262
|
+
if (isFlat(agent)) {
|
|
5238
5263
|
data.version ??= 1;
|
|
5239
|
-
const handlers = asArray(
|
|
5264
|
+
const handlers = asArray(events[event]);
|
|
5240
5265
|
const ours = handlers.find((handler) => isOurs(handler?.command));
|
|
5241
5266
|
if (ours) ours.command = command;
|
|
5242
|
-
else handlers.push({ command, timeout: TIMEOUT_SECONDS, loop_limit: 2 });
|
|
5243
|
-
|
|
5267
|
+
else handlers.push(agent === "cursor" ? { command, timeout: TIMEOUT_SECONDS, loop_limit: 2 } : { type: "command", command, timeoutSec: TIMEOUT_SECONDS });
|
|
5268
|
+
events[event] = handlers;
|
|
5244
5269
|
} else {
|
|
5245
|
-
const groups = asArray(
|
|
5270
|
+
const groups = asArray(events[event]);
|
|
5246
5271
|
const ours = groups.flatMap((group) => asArray(group?.hooks)).find((handler) => isOurs(handler?.command));
|
|
5247
5272
|
if (ours) ours.command = command;
|
|
5248
|
-
else groups.push({ hooks: [{ type: "command", command, timeout: TIMEOUT_SECONDS }] });
|
|
5249
|
-
|
|
5273
|
+
else groups.push(agent === "gemini" ? { matcher: "*", hooks: [{ name: "agent-roast", type: "command", command, timeout: TIMEOUT_SECONDS * 1e3 }] } : { hooks: [{ type: "command", command, timeout: TIMEOUT_SECONDS }] });
|
|
5274
|
+
events[event] = groups;
|
|
5250
5275
|
}
|
|
5251
5276
|
return JSON.stringify(data) !== before;
|
|
5252
5277
|
}
|
|
5253
5278
|
function removeStopHook(agent, data) {
|
|
5254
5279
|
const before = JSON.stringify(data);
|
|
5255
|
-
const
|
|
5256
|
-
if (!
|
|
5257
|
-
if (agent
|
|
5258
|
-
const kept = asArray(
|
|
5259
|
-
if (kept.length)
|
|
5260
|
-
else delete
|
|
5280
|
+
const events = eventMap(agent, data, false), event = FINISH_EVENT[agent];
|
|
5281
|
+
if (!events) return false;
|
|
5282
|
+
if (isFlat(agent)) {
|
|
5283
|
+
const kept = asArray(events[event]).filter((handler) => !isOurs(handler?.command));
|
|
5284
|
+
if (kept.length) events[event] = kept;
|
|
5285
|
+
else delete events[event];
|
|
5261
5286
|
} else {
|
|
5262
|
-
const groups = asArray(
|
|
5287
|
+
const groups = asArray(events[event]).map((group) => ({ ...group, hooks: asArray(group?.hooks).filter((handler) => !isOurs(handler?.command)) }));
|
|
5263
5288
|
const kept = groups.filter((group) => group.hooks.length);
|
|
5264
|
-
if (kept.length)
|
|
5265
|
-
else delete
|
|
5289
|
+
if (kept.length) events[event] = kept;
|
|
5290
|
+
else delete events[event];
|
|
5266
5291
|
}
|
|
5267
|
-
if (!Object.keys(
|
|
5292
|
+
if (events !== data && !Object.keys(events).length) delete data.hooks;
|
|
5268
5293
|
return JSON.stringify(data) !== before;
|
|
5269
5294
|
}
|
|
5270
5295
|
function context(options) {
|
|
@@ -5294,14 +5319,15 @@ function uninstallHooks(options) {
|
|
|
5294
5319
|
function hookStatus(options) {
|
|
5295
5320
|
const { root, home, env } = context({ ...options, scope: "project" });
|
|
5296
5321
|
const rows = [];
|
|
5297
|
-
for (const agent of ["claude", "codex", "cursor"]) {
|
|
5322
|
+
for (const agent of ["claude", "codex", "cursor", "gemini", "copilot", "droid"]) {
|
|
5298
5323
|
for (const scope of ["project", "local", "global"]) {
|
|
5299
5324
|
if (scope === "local" && agent !== "claude") continue;
|
|
5300
5325
|
const file = hookConfigPath(agent, scope, root, home, env);
|
|
5301
5326
|
let command;
|
|
5302
5327
|
try {
|
|
5303
5328
|
const data = readJsonFile(file).data;
|
|
5304
|
-
const
|
|
5329
|
+
const listed = asArray(eventMap(agent, data, false)?.[FINISH_EVENT[agent]]);
|
|
5330
|
+
const handlers = isFlat(agent) ? listed : listed.flatMap((group) => asArray(group?.hooks));
|
|
5305
5331
|
command = handlers.map((handler) => handler?.command).find(isOurs);
|
|
5306
5332
|
} catch {
|
|
5307
5333
|
}
|
package/dist/cli.js
CHANGED
|
@@ -38,7 +38,7 @@ import {
|
|
|
38
38
|
terminalText,
|
|
39
39
|
uninstallHooks,
|
|
40
40
|
writeBaseline
|
|
41
|
-
} from "./chunk-
|
|
41
|
+
} from "./chunk-L4XLCXC2.js";
|
|
42
42
|
|
|
43
43
|
// src/cli.ts
|
|
44
44
|
import { Command } from "commander";
|
|
@@ -100,7 +100,7 @@ function isDefaultInvocation(options) {
|
|
|
100
100
|
var NEXT_STEPS = [
|
|
101
101
|
"",
|
|
102
102
|
pc.bold("Next steps"),
|
|
103
|
-
` ${pc.cyan("npx agent-roast hook install")} stop your coding agent from weakening checks (Claude Code, Codex, Cursor)`,
|
|
103
|
+
` ${pc.cyan("npx agent-roast hook install")} stop your coding agent from weakening checks (Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, Factory Droid)`,
|
|
104
104
|
` ${pc.cyan("npx agent-roast init --ci")} add the pull-request check (GitHub Actions)`,
|
|
105
105
|
` ${pc.cyan("npx agent-roast demo")} see a scripted example`,
|
|
106
106
|
""
|
|
@@ -505,7 +505,7 @@ program.command("demo").description('See what agent-roast reports for a scripted
|
|
|
505
505
|
}
|
|
506
506
|
});
|
|
507
507
|
program.command("audit").description("Score the Git history by the shortcuts it adds (the original agent-roast report)").argument("[path]", "Path to git repository (default: current directory)", ".").action((repoPath) => rootAction(repoPath, { ...program.opts(), forceLegacy: true }));
|
|
508
|
-
var hook = program.command("hook").description("Guard a coding agent: send it back to fix the code when it weakens a check (Claude Code, Codex, Cursor)");
|
|
508
|
+
var hook = program.command("hook").description("Guard a coding agent: send it back to fix the code when it weakens a check (Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, Factory Droid)");
|
|
509
509
|
hook.command("run").description("Run by the agent when it is about to finish; reads the agent's JSON on stdin and answers in its dialect").option("--agent <agent>", `${HOOK_AGENTS.join(", ")} (detected from the input when omitted)`).option("--advisory", "only warn the person; never send the agent back", false).action(async (opts) => {
|
|
510
510
|
let agent;
|
|
511
511
|
try {
|
|
@@ -525,13 +525,13 @@ hook.command("run").description("Run by the agent when it is about to finish; re
|
|
|
525
525
|
process.exitCode = 0;
|
|
526
526
|
}
|
|
527
527
|
});
|
|
528
|
-
hook.command("install").description("Add the hook to Claude Code, Codex or
|
|
528
|
+
hook.command("install").description("Add the hook to Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot or Factory Droid in this repository").argument("[agents...]", `${HOOK_AGENTS.join(", ")} (default: the agents that already have a folder in this repository)`).option("--global", "install for every repository of this user instead", false).option("--local", "Claude Code only: use .claude/settings.local.json, which stays out of Git", false).option("--command <program>", "program to run instead of 'npx --yes agent-roast@<version>', for example 'agent-roast' when installed globally").option("--advisory", "try it without risk: the hook only warns you and never sends the agent back", false).option("--instructions", "also write a short 'keep the checks honest' paragraph into AGENTS.md / CLAUDE.md", false).option("--dry-run", "show what would change without writing anything", false).action((names, opts) => {
|
|
529
529
|
try {
|
|
530
530
|
if (opts.global && opts.local) throw new ConfigError("Choose --global or --local, not both.");
|
|
531
531
|
let agents = parseAgents(names);
|
|
532
532
|
const root = repositoryRoot(process.cwd());
|
|
533
|
-
if (!agents.length) agents = HOOK_AGENTS.filter((agent) => fs.existsSync(path.join(root, { claude: ".claude", codex: ".codex", cursor: ".cursor" }[agent])));
|
|
534
|
-
if (!agents.length) throw new ConfigError("Name the agent to guard: agent-roast hook install claude (or codex, cursor).");
|
|
533
|
+
if (!agents.length) agents = HOOK_AGENTS.filter((agent) => fs.existsSync(path.join(root, { claude: ".claude", codex: ".codex", cursor: ".cursor", gemini: ".gemini", copilot: ".github/hooks", droid: ".factory" }[agent])));
|
|
534
|
+
if (!agents.length) throw new ConfigError("Name the agent to guard: agent-roast hook install claude (or codex, cursor, gemini, copilot, droid).");
|
|
535
535
|
const scope = opts.global ? "global" : opts.local ? "local" : "project";
|
|
536
536
|
const advisory = opts.advisory || program.opts().advisory === true;
|
|
537
537
|
const changes = installHooks({ cwd: process.cwd(), agents, scope, command: opts.command, advisory, dryRun: opts.dryRun });
|
|
@@ -542,6 +542,9 @@ hook.command("install").description("Add the hook to Claude Code, Codex or Curso
|
|
|
542
542
|
for (const change of installInstructions({ cwd: process.cwd(), dryRun: opts.dryRun })) console.log(` + ${displayPath(change.file)} (${change.action}): keep-the-checks-honest paragraph`);
|
|
543
543
|
}
|
|
544
544
|
if (agents.includes("codex")) console.log("\nCodex runs a new hook only after you trust it: open Codex and review it with /hooks.");
|
|
545
|
+
if (agents.includes("gemini")) console.log("\nGemini CLI warns the first time it sees a project hook, and again when its command changes; /hooks panel shows it.");
|
|
546
|
+
if (agents.includes("copilot")) console.log("\nGitHub Copilot reads hook files when it starts: restart it. The cloud agent loads them from the default branch only.");
|
|
547
|
+
if (agents.includes("droid")) console.log("\nFactory Droid takes a snapshot of its hooks at startup: restart it, and review the hook with /hooks.");
|
|
545
548
|
if (!opts.dryRun && advisory) console.log("\nAdvisory mode: when the agent finishes with weakened checks you get a one-line warning and the agent is never sent back.\nStart sending it back: agent-roast hook install (without --advisory) \xB7 remove it: agent-roast hook uninstall");
|
|
546
549
|
else if (!opts.dryRun) console.log("\nWhen the agent is about to finish with weakened checks it is sent back once to fix the code.\nTurn it off for a while: agent-roast hook pause \xB7 only warn instead: agent-roast hook install --advisory \xB7 remove it: agent-roast hook uninstall");
|
|
547
550
|
} catch (error) {
|
|
@@ -549,7 +552,7 @@ hook.command("install").description("Add the hook to Claude Code, Codex or Curso
|
|
|
549
552
|
process.exitCode = 2;
|
|
550
553
|
}
|
|
551
554
|
});
|
|
552
|
-
hook.command("uninstall").description("Remove the hook from Claude Code, Codex or
|
|
555
|
+
hook.command("uninstall").description("Remove the hook from Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot or Factory Droid").argument("[agents...]", `${HOOK_AGENTS.join(", ")} (default: all)`).option("--global", "remove it from the user-level files", false).option("--local", "Claude Code only: remove it from .claude/settings.local.json", false).option("--dry-run", "show what would change without writing anything", false).action((names, opts) => {
|
|
553
556
|
try {
|
|
554
557
|
if (opts.global && opts.local) throw new ConfigError("Choose --global or --local, not both.");
|
|
555
558
|
const agents = names.length ? parseAgents(names) : [...HOOK_AGENTS].filter((agent) => !(opts.local && agent !== "claude"));
|
|
@@ -565,11 +568,11 @@ hook.command("uninstall").description("Remove the hook from Claude Code, Codex o
|
|
|
565
568
|
hook.command("status").description("Show where the hook is installed").action(async () => {
|
|
566
569
|
try {
|
|
567
570
|
const rows = hookStatus({ cwd: process.cwd() });
|
|
568
|
-
for (const row of rows) console.log(` ${row.installed ? pc.green("installed") : pc.dim("not set ")} ${AGENT_LABELS[row.agent].padEnd(
|
|
571
|
+
for (const row of rows) console.log(` ${row.installed ? pc.green("installed") : pc.dim("not set ")} ${AGENT_LABELS[row.agent].padEnd(16)} ${row.scope.padEnd(8)} ${displayPath(row.file)}${row.advisory ? pc.yellow(" (advisory: warns, never sends the agent back)") : ""}`);
|
|
569
572
|
const paused = await hookPausedUntil(process.cwd());
|
|
570
573
|
if (paused) console.log(`
|
|
571
574
|
Paused in this repository until ${paused.toLocaleString()} (agent-roast hook resume).`);
|
|
572
|
-
if (!rows.some((row) => row.installed)) console.log("\n Install it with: agent-roast hook install claude (or codex, cursor)");
|
|
575
|
+
if (!rows.some((row) => row.installed)) console.log("\n Install it with: agent-roast hook install claude (or codex, cursor, gemini, copilot, droid)");
|
|
573
576
|
} catch (error) {
|
|
574
577
|
console.error(`agent-roast hook status error: ${errorMessage(error)}`);
|
|
575
578
|
process.exitCode = 2;
|
package/dist/index.d.ts
CHANGED
|
@@ -475,7 +475,7 @@ declare function replayHistory(options?: ReplayOptions): Promise<ReplayResult>;
|
|
|
475
475
|
declare function formatReplayTerminal(result: ReplayResult, allRules?: boolean): string;
|
|
476
476
|
|
|
477
477
|
/** Coding agents that can run a command when they are about to finish and feed its answer back. */
|
|
478
|
-
type HookAgent = "claude" | "codex" | "cursor";
|
|
478
|
+
type HookAgent = "claude" | "codex" | "cursor" | "gemini" | "copilot" | "droid";
|
|
479
479
|
declare const HOOK_AGENTS: readonly HookAgent[];
|
|
480
480
|
interface HookOptions {
|
|
481
481
|
/** Which agent is calling. Detected from its input when omitted. */
|
package/dist/index.js
CHANGED
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-roast",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"description": "Stops AI
|
|
3
|
+
"version": "0.23.0",
|
|
4
|
+
"description": "Stops AI agents (and rushed humans) weakening tests, coverage gates or CI to get a green build. Hook for Claude Code, Codex, Cursor, Gemini CLI, Copilot, Droid; pre-commit; GitHub Action. Offline.",
|
|
5
5
|
"mcpName": "io.github.Nari-Ab/agent-roast",
|
|
6
6
|
"type": "module",
|
|
7
7
|
"main": "./dist/index.js",
|
|
@@ -34,6 +34,8 @@
|
|
|
34
34
|
"codex",
|
|
35
35
|
"cursor",
|
|
36
36
|
"copilot",
|
|
37
|
+
"gemini-cli",
|
|
38
|
+
"factory-droid",
|
|
37
39
|
"hooks",
|
|
38
40
|
"guardrails",
|
|
39
41
|
"code-review",
|