agent-roast 0.1.4 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +71 -14
- package/dist/chunk-FHRPVXCI.js +1298 -0
- package/dist/cli.js +70 -14
- package/dist/index.d.ts +114 -5
- package/dist/index.js +41 -3
- package/package.json +1 -1
- package/dist/chunk-EPPMO4JE.js +0 -560
package/README.md
CHANGED
|
@@ -2,23 +2,33 @@
|
|
|
2
2
|
|
|
3
3
|
<picture>
|
|
4
4
|
<source media="(prefers-color-scheme: dark)" srcset="docs/assets/logo-dark.svg">
|
|
5
|
-
<img src="docs/assets/logo.svg" alt="agent-roast" width="
|
|
5
|
+
<img src="docs/assets/logo.svg" alt="agent-roast" width="380">
|
|
6
6
|
</picture>
|
|
7
7
|
|
|
8
|
-
|
|
8
|
+
<h2>Audit your git history for AI coding agent infractions and panic loops</h2>
|
|
9
9
|
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
10
|
+
<p>
|
|
11
|
+
<a href="https://github.com/Nari-Ab/agent-roast/actions"><img src="https://img.shields.io/badge/CI-passing-2ea44f?logo=github&logoColor=white" alt="CI"></a>
|
|
12
|
+
<a href="https://github.com/Nari-Ab/agent-roast"><img src="https://img.shields.io/badge/roast%20audit-verified-2ea44f" alt="Roast Audit"></a>
|
|
13
|
+
<a href="https://www.npmjs.com/package/agent-roast"><img src="https://img.shields.io/npm/v/agent-roast?color=cb3837&logo=npm&logoColor=white" alt="npm"></a>
|
|
14
|
+
<a href="https://www.npmjs.com/package/agent-roast"><img src="https://img.shields.io/npm/dt/agent-roast?label=downloads&color=0969da" alt="downloads"></a>
|
|
15
|
+
<a href="https://nari-ab.github.io/agent-roast/"><img src="https://img.shields.io/badge/site-live-1f9d55" alt="site"></a>
|
|
16
|
+
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-0969da" alt="license"></a>
|
|
17
|
+
</p>
|
|
13
18
|
|
|
14
19
|
</div>
|
|
15
20
|
|
|
16
21
|
`agent-roast` scans recorded git diffs for shortcuts taken by AI coding assistants (Claude Code, Cursor, Aider). It checks for skipped tests, compiler type escapes (`as any`), swallowed errors, and panic revert loops, then outputs a discipline score and line proofs. 100% offline, zero API keys, zero telemetry.
|
|
17
22
|
|
|
18
23
|
```bash
|
|
24
|
+
# Audit repository history
|
|
19
25
|
npx agent-roast # scan AI-attributed commits (last 90 days)
|
|
20
26
|
npx agent-roast --all # scan entire repository history
|
|
21
27
|
npx agent-roast --verbose # show commit SHAs and line proofs
|
|
28
|
+
|
|
29
|
+
# Audit pull request diffs (CI gate)
|
|
30
|
+
npx agent-roast --base origin/main --head HEAD
|
|
31
|
+
npx agent-roast --base origin/main --head HEAD --fail-on test-skip,swallowed-error --format github
|
|
22
32
|
```
|
|
23
33
|
|
|
24
34
|
What a report looks like (a real repository audit, trimmed):
|
|
@@ -49,25 +59,72 @@ INFRACTIONS DETECTED (42)
|
|
|
49
59
|
|
|
50
60
|
---
|
|
51
61
|
|
|
62
|
+
### GitHub Actions Integration
|
|
63
|
+
|
|
64
|
+
Add `agent-roast` as a gate in your pull request workflow (`.github/workflows/audit.yml`):
|
|
65
|
+
|
|
66
|
+
```yaml
|
|
67
|
+
name: Agent Audit
|
|
68
|
+
on:
|
|
69
|
+
pull_request:
|
|
70
|
+
branches: [main]
|
|
71
|
+
|
|
72
|
+
permissions:
|
|
73
|
+
contents: read
|
|
74
|
+
|
|
75
|
+
jobs:
|
|
76
|
+
audit:
|
|
77
|
+
runs-on: ubuntu-latest
|
|
78
|
+
steps:
|
|
79
|
+
- uses: actions/checkout@v4
|
|
80
|
+
with:
|
|
81
|
+
fetch-depth: 0 # Required for complete merge-base history
|
|
82
|
+
|
|
83
|
+
- name: Run agent-roast audit
|
|
84
|
+
uses: Nari-Ab/agent-roast@v1
|
|
85
|
+
with:
|
|
86
|
+
fail-on: test-skip
|
|
87
|
+
format: github
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
> **Security Note:** `agent-roast` runs in an isolated runner directory (`$RUNNER_TEMP`) with `--ignore-scripts` to neutralize malicious `.npmrc` files in untrusted pull requests. `pull_request_target` is not supported.
|
|
91
|
+
|
|
92
|
+
---
|
|
93
|
+
|
|
52
94
|
### What it checks
|
|
53
95
|
|
|
54
|
-
Only **added lines** (`+`) in source files are scanned — file moves and
|
|
96
|
+
Only **added lines** (`+`) in source code files are scanned — deleted lines, file moves, and comments never trigger false alarms.
|
|
55
97
|
|
|
56
|
-
- **Skipped tests:** `it.skip`, `describe.skip`, `xit`, `pytest.skip`, `test.todo` added
|
|
57
|
-
- **Type escapes:** `as any`, `// @ts-ignore`, `// @ts-expect-error`, `# type: ignore`, `eslint-disable`.
|
|
58
|
-
- **Swallowed errors:** Empty `catch {}` or `except: pass` blocks hiding
|
|
59
|
-
- **Panic fix loops:**
|
|
98
|
+
- **Skipped tests (`test-skip`):** `it.skip`, `describe.skip`, `xit`, `pytest.skip`, `test.todo` added to silence failing tests. (Default blocking gate).
|
|
99
|
+
- **Type escapes (`type-escape`):** `as any`, `// @ts-ignore`, `// @ts-expect-error`, `# type: ignore`, `eslint-disable`.
|
|
100
|
+
- **Swallowed errors (`swallowed-error`):** Empty `catch {}` or `except: pass` blocks hiding runtime failures.
|
|
101
|
+
- **Panic fix loops (`panic-loop`):** Heuristic chain analysis of rapid-fire fix/revert commits by the same author within 15 minutes touching overlapping files. *(Observational metric for repository audits; not a PR blocker)*.
|
|
60
102
|
|
|
61
103
|
---
|
|
62
104
|
|
|
63
|
-
###
|
|
105
|
+
### Scoring Methodology
|
|
64
106
|
|
|
65
|
-
- **
|
|
66
|
-
|
|
67
|
-
|
|
107
|
+
- **Repository Audit Score:** Uses size-invariant defect density:
|
|
108
|
+
$$\text{Score} = 100 \cdot \exp\left(-\left(\frac{\text{codeDensity}}{R_0} + \frac{\text{panicDensity}}{R_1}\right)\right)$$
|
|
109
|
+
where defect points are normalized by effective sample volume ($k\text{Loc} + 0.5$). Repositories with fewer than 200 lines added return `null` (insufficient data) rather than an arbitrary 100.
|
|
110
|
+
- **Pull Request Mode:** Avoids unstable micro-sample scores. Reports verified infractions, pass/fail status, and reference defect density per 100 LOC. Author identities are hidden by default to prevent team blame games.
|
|
68
111
|
|
|
69
112
|
---
|
|
70
113
|
|
|
114
|
+
### Requirements & Privacy
|
|
115
|
+
|
|
116
|
+
- **Requirements:** Git 2.30+ (for `--end-of-options` argument isolation), Node.js 18+.
|
|
117
|
+
- **100% Local:** Instant streaming parser runs directly on your machine.
|
|
118
|
+
- **Zero Telemetry:** No cloud dependencies, no LLM API calls, no network traffic. Verified by static bundle inspection.
|
|
119
|
+
- **Hermetic:** Enforces `-c core.quotepath=false -c diff.noprefix=false` to guarantee identical behavior across all environments.
|
|
120
|
+
|
|
121
|
+
---
|
|
122
|
+
|
|
123
|
+
### Author & Support
|
|
124
|
+
|
|
125
|
+
Created by Nariman Abdukarimov ([abdukarimov.nariman07@gmail.com](mailto:abdukarimov.nariman07@gmail.com)).
|
|
126
|
+
|
|
71
127
|
### License
|
|
72
128
|
|
|
73
129
|
[MIT](LICENSE)
|
|
130
|
+
|