agent-roast 0.1.1 → 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -2,147 +2,70 @@
2
2
 
3
3
  <picture>
4
4
  <source media="(prefers-color-scheme: dark)" srcset="docs/assets/logo-dark.svg">
5
- <img src="docs/assets/logo.svg" alt="agent-roast" width="460">
5
+ <img src="docs/assets/logo.svg" alt="agent-roast" width="340">
6
6
  </picture>
7
7
 
8
- ### Audit your git history for AI coding agent shortcuts, infractions, and panic loops
8
+ ### Audit your git history for AI coding agent infractions and panic loops
9
9
 
10
- [![npm version](https://img.shields.io/npm/v/agent-roast?logo=npm&logoColor=white&color=cb3837&label=npm)](https://www.npmjs.com/package/agent-roast)
11
- [![telemetry: zero (offline)](https://img.shields.io/badge/telemetry-zero%20(offline)-1f9d55)](#privacy--zero-telemetry)
12
- [![speed: <200ms](https://img.shields.io/badge/speed-%3C200ms-blue)](#performance)
10
+ [![npm version](https://img.shields.io/npm/v/agent-roast?logo=npm&logoColor=white&color=cb3837)](https://www.npmjs.com/package/agent-roast)
11
+ [![telemetry: zero (offline)](https://img.shields.io/badge/telemetry-zero%20(offline)-1f9d55)](#privacy)
13
12
  [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
14
13
 
15
- <br/>
16
-
17
- ```bash
18
- npx agent-roast
19
- ```
20
-
21
14
  </div>
22
15
 
23
- ---
24
-
25
- Did your AI coding assistant write clean, maintainable software — or did it quietly delete failing tests, slap `as any` everywhere, and spam 6 panic commits in 8 minutes?
26
-
27
- **`agent-roast`** is an offline, zero-telemetry CLI scorecard that parses git diffs to detect AI shortcuts and calculate an objective **Discipline Score (0–100)** with a personality archetype.
28
-
29
- <div align="center">
30
- <img src="docs/assets/demo.svg" alt="agent-roast terminal audit demo" width="100%">
31
- </div>
32
-
33
- ---
34
-
35
- ## ⚡️ Quickstart
36
-
37
- Run directly in any repository (no install required):
16
+ `agent-roast` scans recorded git diffs for shortcuts taken by AI coding assistants (Claude Code, Cursor, Aider). It checks for skipped tests, compiler type escapes (`as any`), swallowed errors, and panic revert loops, then outputs a discipline score and line proofs. 100% offline, zero API keys, zero telemetry.
38
17
 
39
18
  ```bash
40
- # Audit AI-attributed commits (Claude Code, Cursor, Aider, Devin, etc.)
41
- npx agent-roast
42
-
43
- # Audit ALL commits across repo history (including unverified human commits)
44
- npx agent-roast --all
45
-
46
- # Print exact commit SHAs and line snippets for all infractions
47
- npx agent-roast --verbose
48
-
49
- # Audit a specific time window (default: last 90 days)
50
- npx agent-roast --since "30 days ago"
51
-
52
- # Output structured JSON for CI/CD pipelines
53
- npx agent-roast --json
19
+ npx agent-roast # scan AI-attributed commits (last 90 days)
20
+ npx agent-roast --all # scan entire repository history
21
+ npx agent-roast --verbose # show commit SHAs and line proofs
54
22
  ```
55
23
 
56
- ---
57
-
58
- ## 🔍 What It Detects
59
-
60
- `agent-roast` analyzes **only added lines** (`+`) in source code files, eliminating false alarms from refactoring or file renames:
61
-
62
- | Infraction | Patterns Detected | Penalty Weight | Why It Matters |
63
- | :--- | :--- | :---: | :--- |
64
- | **Skipped Tests** | `it.skip`, `describe.skip`, `xit`, `pytest.skip`, `test.todo` | **15 pts** | Disabling failing tests instead of fixing the root bug. |
65
- | **Swallowed Errors** | Empty `catch {}`, `except: pass` | **10 pts** | Silencing exceptions and sweeping runtime failures under the rug. |
66
- | **Panic Fix Loops** | Consecutive quickfix commits within 15m by same author | **8 pts** | Frantic thrashing without reading stack traces or logs. |
67
- | **Type Escapes** | `as any`, `// @ts-ignore`, `// @ts-expect-error`, `# type: ignore` | **3 pts** | Bypassing compiler guarantees to pretend code type-checks. |
24
+ What a report looks like (a real repository audit, trimmed):
68
25
 
69
- > [!NOTE]
70
- > String literals and regex patterns are automatically sanitized before analysis. Test fixtures containing pattern names as string data will never trigger false positives.
26
+ ```text
27
+ agent-roast — git discipline audit (304 AI commits in Soup)
71
28
 
72
- ---
29
+ SCORE: 55 / 100 · The Panic Looper
30
+ Prone to rapid-fire fix and revert thrashing cycles when wrestling stubborn bugs.
73
31
 
74
- ## 🎭 Agent Archetypes
32
+ INFRACTIONS DETECTED (42)
75
33
 
76
- Depending on the dominant infraction patterns, `agent-roast` assigns your agent an archetype:
34
+ ● panic-loop (15) — consecutive quick fixes < 15m apart
35
+ latest: 3df29a1 -> a910f21 (3m apart)
77
36
 
78
- | Archetype | Icon | Trigger Condition | Personality Dossier |
79
- | :--- | :---: | :--- | :--- |
80
- | **The Clean Coder** | 🧼 | Score $\ge 90$, 0 infractions | Spotless discipline. No skipped tests, zero panic loops, clean type safety. |
81
- | **The Pragmatic Builder** | 🛠️ | Score $\ge 90$, minor compromises | Overwhelmingly disciplined. Only rare, isolated shortcuts across thousands of lines. |
82
- | **The Panic Looper** | ↻ | Dominant panic loops ($\ge 2$) | Prone to rapid-fire fix and revert thrashing cycles when wrestling stubborn bugs. |
83
- | **The Silent Vandal** | ✖ | Dominant skipped tests ($\ge 3$) | When test suites push back, tends to disable or skip tests to keep pipelines green. |
84
- | **The Any Architect** | ▲ | Dominant type escapes ($\ge 5$) | Leans on compiler bypasses (`as any`, `@ts-ignore`) rather than strict type modeling. |
85
- | **The Secret Keeper** | 🤐 | Dominant empty catches ($\ge 2$) | Tends to silence errors with empty catch blocks, obscuring runtime failures. |
86
- | **The Chaos Gremlin** | 🔥 | Score $< 50$, mixed infractions | Mixes multiple shortcuts: skipped tests, type bypasses, and quick patches under pressure. |
37
+ ● type-escape (19) — compiler type bypasses in added lines
38
+ e.g. src/client.ts:42 — response.body as any
87
39
 
88
- ---
40
+ ● test-skip (6) — disabled or skipped test suites
41
+ e.g. tests/test_outbound.py:18 — pytest.skip(...)
89
42
 
90
- ## 📐 How Scoring Works
43
+ ● swallowed-error (2) — empty catch blocks or except: pass
44
+ ```
91
45
 
92
- 1. **Volume Dampening:** To prevent large repos from automatically scoring 99/100, penalties scale sub-linearly using a logarithmic volume curve:
93
- $$\text{Volume Factor} = 1 + 1.2 \cdot \ln\left(\max\left(1, \frac{\text{LOC Added}}{1000}\right)\right)$$
94
- 2. **Commit Caps:** Maximum 10 infractions counted per commit to prevent a single refactoring commit from dominating the entire audit.
95
- 3. **Small Sample Protection:** Repositories with fewer than 200 lines of added code display `INSUFFICIENT DATA` instead of a premature score.
46
+ <img src="docs/assets/case-study.gif" alt="agent-roast git audit session" width="100%">
96
47
 
97
48
  ---
98
49
 
99
- ## 🤖 AI Attribution
50
+ ### What it checks
100
51
 
101
- `agent-roast` recognizes commits authored or co-authored by AI tools through:
102
- - Git trailers (`Co-Authored-By: Claude <noreply@anthropic.com>`, Cursor, Aider, etc.)
103
- - Commit messages (`generated by aider`, `authored by cursor`)
104
- - Author emails and usernames containing recognized bot signatures
52
+ Only **added lines** (`+`) in source files are scanned — file moves and refactoring never trigger false alarms.
105
53
 
106
- To audit all commits regardless of attribution, pass the `--all` flag.
54
+ - **Skipped tests:** `it.skip`, `describe.skip`, `xit`, `pytest.skip`, `test.todo` added when tests fail.
55
+ - **Type escapes:** `as any`, `// @ts-ignore`, `// @ts-expect-error`, `# type: ignore`, `eslint-disable`.
56
+ - **Swallowed errors:** Empty `catch {}` or `except: pass` blocks hiding production errors.
57
+ - **Panic fix loops:** Rapid-fire fix and revert commits by the same author within 15 minutes.
107
58
 
108
59
  ---
109
60
 
110
- ## 🔒 Privacy & Zero Telemetry
61
+ ### Privacy
111
62
 
112
- - **100% Local Execution:** Runs streaming `git log` directly on your machine in under 200ms.
113
- - **Zero Network Calls:** No external API requests, no telemetry, no analytics.
63
+ - **100% Local:** Runs streaming `git log` on your machine in <200ms.
64
+ - **Zero Telemetry:** No cloud dependencies, no LLM API calls, no network traffic.
114
65
  - **Hermetic:** Source code never leaves your computer.
115
66
 
116
67
  ---
117
68
 
118
- ## 🚦 GitHub Actions CI Integration
119
-
120
- Fail a PR if an agent's discipline score drops below 60:
121
-
122
- ```yaml
123
- name: Agent Audit
124
- on: [pull_request]
125
-
126
- jobs:
127
- roast:
128
- runs-on: ubuntu-latest
129
- steps:
130
- - uses: actions/checkout@v4
131
- with:
132
- fetch-depth: 0
133
- - name: Run agent-roast
134
- run: |
135
- npx agent-roast --since "7 days ago" --json > roast.json
136
- SCORE=$(jq '.score' roast.json)
137
- echo "Discipline Score: $SCORE / 100"
138
- if [ "$SCORE" -lt 60 ]; then
139
- echo "❌ Agent discipline score below 60!"
140
- exit 1
141
- fi
142
- ```
143
-
144
- ---
145
-
146
- ## 📄 License
69
+ ### License
147
70
 
148
71
  [MIT](LICENSE)
@@ -425,71 +425,81 @@ var MetricsCollector = class {
425
425
  // src/renderer.ts
426
426
  import pc from "picocolors";
427
427
  function generateShareUrl(summary) {
428
- const text = `My AI coding agent scored ${summary.score}/100 on @agent_roast! \u{1F916}\u{1F525}
429
-
430
- Archetype: "${summary.archetype}"
428
+ const text = `My AI coding agent scored ${summary.score}/100 on agent-roast.
431
429
 
430
+ Archetype: ${summary.archetype}
432
431
  Infractions:
433
432
  \u2022 ${summary.testSkips.length} skipped tests
434
433
  \u2022 ${summary.typeEscapes.length} type escapes
435
434
  \u2022 ${summary.panicLoops.length} panic revert loops
436
435
 
437
- Audit your repo with: npx agent-roast`;
436
+ Audit your repo: npx agent-roast`;
438
437
  return `https://twitter.com/intent/tweet?text=${encodeURIComponent(text)}`;
439
438
  }
440
439
  function formatTerminalCard(summary, options) {
441
440
  const lines = [];
442
- const boxW = 60;
443
- const hLine = "\u2550".repeat(boxW - 2);
444
- lines.push(pc.cyan(`\u2554${hLine}\u2557`));
445
- lines.push(
446
- pc.cyan("\u2551") + pc.bold(pc.white(" \u{1F916} AGENT ROAST DOSSIER ")) + pc.cyan("\u2551")
447
- );
448
- lines.push(pc.cyan(`\u255A${hLine}\u255D`));
449
441
  lines.push("");
450
- lines.push(` ${pc.dim("Repository:")} ${pc.bold(pc.white(options.repoName))}`);
451
442
  lines.push(
452
- ` ${pc.dim("Commits:")} ${pc.yellow(summary.aiCommits)} AI-attributed / ${summary.totalCommits} total`
443
+ `${pc.bold(pc.cyan("agent-roast"))} ${pc.dim("\u2014 git discipline audit")} ${pc.white(options.repoName)}`
453
444
  );
454
445
  lines.push(
455
- ` ${pc.dim("Code Added:")} ${pc.green(summary.aiLinesAdded)} lines (AI) / ${summary.totalLinesAdded} lines (total)`
446
+ ` ${pc.dim("commits:")} ${pc.yellow(summary.aiCommits)} AI / ${summary.totalCommits} total ${pc.dim("lines added:")} ${pc.green(summary.aiLinesAdded)} AI / ${summary.totalLinesAdded} total`
456
447
  );
457
448
  lines.push("");
458
449
  if (!summary.isSufficientData) {
459
- lines.push(pc.yellow(` \u26A0 INSUFFICIENT DATA TO ROAST (< 200 lines added)`));
460
- lines.push(pc.dim(" Commit more AI code and run `npx agent-roast` again."));
450
+ lines.push(pc.yellow(` insufficient data (< 200 lines added by AI)`));
451
+ lines.push(pc.dim(" commit more code and run `npx agent-roast` again."));
461
452
  lines.push("");
462
453
  return lines.join("\n");
463
454
  }
464
455
  let scoreColor = pc.green;
465
456
  if (summary.score < 60) scoreColor = pc.red;
466
457
  else if (summary.score < 80) scoreColor = pc.yellow;
467
- lines.push(` ${pc.bold("AGENT ARCHETYPE:")} ${pc.magenta(pc.bold(summary.archetype))}`);
468
- lines.push(` ${pc.italic(summary.archetypeDescription)}`);
469
- lines.push("");
470
- lines.push(` ${pc.bold("DISCIPLINE SCORE:")} ${scoreColor(pc.bold(`${summary.score} / 100`))}`);
471
458
  lines.push(
472
- pc.dim(" (100 = spotless discipline, 0 = complete shortcut addiction)")
459
+ ` ${pc.bold("SCORE:")} ${scoreColor(pc.bold(`${summary.score} / 100`))} ${pc.dim("\xB7")} ${pc.magenta(pc.bold(summary.archetype))}`
473
460
  );
461
+ lines.push(` ${pc.dim(summary.archetypeDescription)}`);
474
462
  lines.push("");
475
- lines.push(pc.bold(" \u{1F4CA} INFRACTIONS BREAKDOWN:"));
476
- lines.push(pc.dim(" \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500"));
477
- const skipIcon = summary.testSkips.length > 0 ? pc.red("\u2716") : pc.green("\u2714");
478
- lines.push(
479
- ` ${skipIcon} ${pc.bold("Skipped tests:")} ${summary.testSkips.length} ${pc.dim("(it.skip, describe.skip, xit)")}`
480
- );
481
- const escapeIcon = summary.typeEscapes.length > 0 ? pc.yellow("\u25B2") : pc.green("\u2714");
482
- lines.push(
483
- ` ${escapeIcon} ${pc.bold("Type escapes:")} ${summary.typeEscapes.length} ${pc.dim("(as any, @ts-ignore, eslint-disable)")}`
484
- );
485
- const errorIcon = summary.swallowedErrors.length > 0 ? pc.red("\u2716") : pc.green("\u2714");
486
- lines.push(
487
- ` ${errorIcon} ${pc.bold("Swallowed errors:")} ${summary.swallowedErrors.length} ${pc.dim("(empty catch {}, except: pass)")}`
488
- );
489
- const panicIcon = summary.panicLoops.length > 0 ? pc.magenta("\u21BB") : pc.green("\u2714");
490
- lines.push(
491
- ` ${panicIcon} ${pc.bold("Panic fix loops:")} ${summary.panicLoops.length} ${pc.dim("(quick fixes < 15m apart)")}`
492
- );
463
+ const totalInfractions = summary.testSkips.length + summary.typeEscapes.length + summary.swallowedErrors.length + summary.panicLoops.length;
464
+ lines.push(pc.bold(` INFRACTIONS (${totalInfractions})`));
465
+ lines.push(pc.dim(" \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500"));
466
+ if (summary.panicLoops.length > 0) {
467
+ lines.push(
468
+ ` ${pc.magenta("\u25CF")} ${pc.bold("panic-loop")} (${summary.panicLoops.length}) ${pc.dim("consecutive quick fixes < 15m apart")}`
469
+ );
470
+ const sample = summary.panicLoops[0];
471
+ lines.push(` ${pc.dim(sample.reason)}`);
472
+ }
473
+ if (summary.typeEscapes.length > 0) {
474
+ lines.push(
475
+ ` ${pc.yellow("\u25CF")} ${pc.bold("type-escape")} (${summary.typeEscapes.length}) ${pc.dim("compiler bypasses (as any, @ts-ignore)")}`
476
+ );
477
+ const sample = summary.typeEscapes[0];
478
+ if (sample.file) {
479
+ lines.push(` ${pc.dim(`${sample.file}: ${sample.lineSnippet || ""}`)}`);
480
+ }
481
+ }
482
+ if (summary.testSkips.length > 0) {
483
+ lines.push(
484
+ ` ${pc.red("\u25CF")} ${pc.bold("test-skip")} (${summary.testSkips.length}) ${pc.dim("skipped or disabled tests")}`
485
+ );
486
+ const sample = summary.testSkips[0];
487
+ if (sample.file) {
488
+ lines.push(` ${pc.dim(`${sample.file}: ${sample.lineSnippet || ""}`)}`);
489
+ }
490
+ }
491
+ if (summary.swallowedErrors.length > 0) {
492
+ lines.push(
493
+ ` ${pc.red("\u25CF")} ${pc.bold("swallowed-error")} (${summary.swallowedErrors.length}) ${pc.dim("empty catch blocks or except: pass")}`
494
+ );
495
+ const sample = summary.swallowedErrors[0];
496
+ if (sample.file) {
497
+ lines.push(` ${pc.dim(`${sample.file}: ${sample.lineSnippet || ""}`)}`);
498
+ }
499
+ }
500
+ if (totalInfractions === 0) {
501
+ lines.push(` ${pc.green("\u25CF")} ${pc.dim("zero infractions detected across all inspected lines.")}`);
502
+ }
493
503
  lines.push("");
494
504
  if (options.verbose) {
495
505
  const allInfractions = [
@@ -499,27 +509,22 @@ function formatTerminalCard(summary, options) {
499
509
  ...summary.panicLoops
500
510
  ];
501
511
  if (allInfractions.length > 0) {
502
- lines.push(pc.bold(" \u{1F50D} VERBOSE PROOFS (First 10 items):"));
503
- lines.push(pc.dim(" \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500"));
512
+ lines.push(pc.bold(" EVIDENCE (First 10 occurrences):"));
513
+ lines.push(pc.dim(" \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500"));
504
514
  for (const inf of allInfractions.slice(0, 10)) {
505
- const fileLoc = inf.file ? `${inf.file}` : "git commit";
515
+ const fileLoc = inf.file ? `${inf.file}` : "commit history";
506
516
  lines.push(
507
- ` \u2022 ${pc.cyan(inf.commitHash.slice(0, 7))} [${inf.type}] ${pc.white(fileLoc)}`
517
+ ` \u25CF ${pc.cyan(inf.commitHash.slice(0, 7))} [${inf.type}] ${pc.white(fileLoc)}`
508
518
  );
509
519
  if (inf.lineSnippet) {
510
- lines.push(` ${pc.dim(inf.lineSnippet)}`);
520
+ lines.push(` ${pc.dim(`\u2514\u2500 ${inf.lineSnippet}`)}`);
511
521
  }
512
522
  }
513
523
  lines.push("");
514
524
  }
515
525
  }
516
- const shareUrl = generateShareUrl(summary);
517
- lines.push(pc.dim(" \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500"));
518
- lines.push(
519
- ` ${pc.cyan("\u{1F4E2} Share your score on X:")} ${pc.underline(pc.cyan(shareUrl))}`
520
- );
521
526
  lines.push(
522
- ` ${pc.dim("Tip: Run with --verbose to see exact commit SHAs and lines")}`
527
+ ` ${pc.dim("Share score:")} ${pc.cyan(generateShareUrl(summary))}`
523
528
  );
524
529
  lines.push("");
525
530
  return lines.join("\n");
package/dist/cli.js CHANGED
@@ -1,7 +1,7 @@
1
1
  #!/usr/bin/env node
2
2
  import {
3
3
  roastRepository
4
- } from "./chunk-RFCXDN22.js";
4
+ } from "./chunk-EPPMO4JE.js";
5
5
 
6
6
  // src/cli.ts
7
7
  import { Command } from "commander";
package/dist/index.js CHANGED
@@ -8,7 +8,7 @@ import {
8
8
  roastRepository,
9
9
  streamGitLog,
10
10
  stripStringAndRegexLiterals
11
- } from "./chunk-RFCXDN22.js";
11
+ } from "./chunk-EPPMO4JE.js";
12
12
  export {
13
13
  MetricsCollector,
14
14
  detectAiAttribution,
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "agent-roast",
3
- "version": "0.1.1",
3
+ "version": "0.1.2",
4
4
  "description": "Zero-telemetry CLI scorecard analyzing git history for AI coding agent infractions and panic loops",
5
5
  "type": "module",
6
6
  "main": "./dist/index.js",
@@ -44,6 +44,7 @@
44
44
  "picocolors": "^1.1.0"
45
45
  },
46
46
  "devDependencies": {
47
+ "@napi-rs/canvas": "^1.0.10",
47
48
  "@types/node": "^20.0.0",
48
49
  "gifenc": "^1.0.3",
49
50
  "tsup": "^8.0.0",