@chimix/agentscan 0.4.0 → 0.6.0

package/README.md

@@ -5,8 +5,8 @@
 </p>
 
 <p align="center">
-  <img src="https://img.shields.io/badge/checks-23-111111?style=flat-square" alt="23 checks">
-  <img src="https://img.shields.io/badge/tests-183%20passing-111111?style=flat-square" alt="183 tests">
+  <img src="https://img.shields.io/badge/checks-27-111111?style=flat-square" alt="27 checks">
+  <img src="https://img.shields.io/badge/tests-235%20passing-111111?style=flat-square" alt="235 tests">
   <img src="https://img.shields.io/badge/network-none-111111?style=flat-square" alt="No network">
   <img src="https://img.shields.io/badge/writes-none-111111?style=flat-square" alt="Writes nothing">
   <img src="https://img.shields.io/badge/runs%20on-node%20%C2%B7%20bun-111111?style=flat-square" alt="Node or Bun">
@@ -15,7 +15,7 @@
 </p>
 
 <p align="center">
-  <strong>23 checks &middot; 3.4k lines &middot; 0 network calls &middot; every check sourced to a published spec line</strong><br>
+  <strong>27 checks &middot; 4.2k lines &middot; 0 network calls &middot; every check sourced to a published spec line</strong><br>
   <sub>Alpha. An earlier build reported 37 findings across 17 real projects of which <strong>25 were false</strong> — two checks had been written from what real projects looked like instead of from the spec. Both were deleted, and every check that survived is recorded in <a href="docs/spec/">docs/spec/</a> with the URL it came from and the date it was read. That story is the reason this tool exists in its current shape.</sub>
 </p>
 
@@ -47,12 +47,11 @@ think you have has silently not existed for six weeks.
 ```console
 $ agentscan check
 
-WARN    hook:PreToolUse:.claude/hooks/guard-destructive-bash.js
-        rule:hook.missing-script
-        PreToolUse hook points at a script that does not exist
-        evidence: hook PreToolUse @ …/.claude/settings.json
+ERROR   rule:hook.missing-script
+        PreToolUse hook points at a script that does not exist: .claude/hooks/guard-destructive-bash.js
+          PreToolUse @ .claude/settings.json · .claude/hooks/guard-destructive-bash.js
 
-Summary: 6 warn · 4 info hidden (--verbose) · score 40/100
+Summary: 1 error · score 90/100
 ```
 
 That is the whole category: **a config file asserting something that is not true
@@ -65,7 +64,7 @@ No AI, no network, no heuristics. Read the config, read the disk, compare:
 ```
 1. Discover    .claude/ .agents/ .mcp.json AGENTS.md skills-lock.json
 2. Extract     immutable facts — never re-read during checking
-3. Check       23 checks, each against one published spec line
+3. Check       27 checks, each against one published spec line
 4. Report      text · --json · --output prompt (handoff for a fixing agent)
 ```
 
@@ -84,8 +83,16 @@ A check here has to point at a spec line. If it cannot, it does not ship.
 
 ## Try it in 30 seconds
 
-Run it against any project. It reads that project, writes nothing, and never
-leaves your machine:
+No project handy? Run the built-in demo — it builds a throwaway fixture with the
+killer case (a `PreToolUse` hook pointing at a missing script), prints the human
+report, and deletes the fixture:
+
+```bash
+npx @chimix/agentscan@latest demo
+```
+
+Against a real project, it reads that tree, writes nothing, and never leaves
+your machine:
 
 ```bash
 cd ~/your-project
@@ -104,8 +111,9 @@ bun add -d @chimix/agentscan     # then: bunx agentscan check
 ```
 
 Point it at a project that actually has agent config — a `.claude/` directory,
-an `.mcp.json`, an `AGENTS.md`. On a project with none of those it will
-correctly find nothing and say so.
+an `.mcp.json`, an `AGENTS.md`. A directory with agent config and no
+`package.json` is still scannable. On a directory with neither it exits with a
+clear error.
 
 ```bash
 # what it could possibly report, before you run it
@@ -148,10 +156,11 @@ bunx agentscan check                 # text report (cwd)
 bunx agentscan check ./my-app        # explicit root
 bunx agentscan check --json          # machine-readable
 bunx agentscan check --quiet         # summary only
-bunx agentscan check --verbose       # include KEEP and info-severity findings
+bunx agentscan check --verbose       # include KEEP, info findings, and each finding's id
 bunx agentscan check --fail-on warning
 bunx agentscan check --global        # also scan global skill dirs
 
+bunx agentscan demo                  # one-shot fixture (no project required)
 bunx agentscan explain <findingId>   # detail one finding
 bunx agentscan rules                 # every check + rule id that can fire
 bunx agentscan init                  # write .agentscanrc.json
@@ -166,7 +175,7 @@ Flags for `check`:
 | `--copy` | Also copy the report to the system clipboard |
 | `--no-color` | Never colour, even on a terminal (`NO_COLOR=1` does the same) |
 | `--quiet` | Summary line only |
-| `--verbose` | Show KEEP + info-severity findings |
+| `--verbose` | Show KEEP + info-severity findings, and print each finding's id |
 | `--fail-on <level>` | `never` (default) · `warning` · `error` |
 | `--fail-under <0-100>` | Fail when the score drops below this floor |
 | `--global` | Also scan `~/.claude/skills` and `~/.codex/skills` (see below) |
@@ -177,23 +186,28 @@ Flags for `check`:
 ## Sample output
 
 ```text
-  ┌─────┐   40/100  broken  ·  touchagency
-  │ ✕ ✕ │   ██████████░░░░░░░░░░░░░░░
-  │  ⌒  │   6 warn · 4 info hidden (--verbose)
+  ┌─────┐   34/100  broken  ·  touchagency
+  │ ✕ ✕ │   █████████░░░░░░░░░░░░░░░░
+  │  ⌒  │   6 error · 2 warning · 11 info hidden (--verbose)
   └─────┘
 
-agentscan v0.4.0 — touchagency
+agentscan v0.6.0 — touchagency
 
-Scanned: 46 deps · 54 skills · 1 mcp · 2 agents · packageManager=bun
+Scanned: 46 deps · 108 skills · 1 mcp · 2 agents · packageManager=bun
 
-WARN    rule:hook.missing-script  ×6
-        PreToolUse hook points at a script that does not exist
+ERROR   rule:hook.missing-script  ×6
+        Hook points at a script that does not exist
           PostToolUse @ .claude/settings.json · .claude/hooks/notify-related-tests.js
           PreToolUse @ .claude/settings.json · .claude/hooks/guard-destructive-bash.js
           PreToolUse @ .claude/settings.json · .claude/hooks/protect-artists-json.js
           PreToolUse @ .claude/settings.json · .claude/hooks/protect-env.js
 
-Summary: 6 warn · 4 info hidden (--verbose) · score 40/100 broken
+WARN    rule:agent.invalid-name  ×2
+        2 findings
+          .claude/agents/marketing-seo-specialist.md
+          .claude/agents/testing-accessibility-auditor.md
+
+Summary: 6 error · 2 warning · 11 info hidden (--verbose) · score 34/100 broken
 ```
 
 On a terminal the header is coloured green, yellow or red and the face tracks
@@ -201,14 +215,15 @@ the score. Redirect or pipe it and you get exactly the text above, with no
 escape sequences — the header box is dropped too, so `--json`, `--output
 prompt`, CI logs and `--copy` are unchanged by any of this.
 
-The `Stack:` line is orientation only — which project, how big. The summary lists
-only actions that actually occurred; a clean project prints `Summary: no findings`.
+The `Scanned:` line is orientation only — which project, how big. The summary
+lists severities (error / warning / info); a clean project prints
+`Summary: no findings`. Score and `--fail-on` still use severity under the hood.
 
 JSON shape (abridged):
 
 ```json
 {
-  "version": "0.1.0",
+  "version": "0.6.0",
   "root": "/path/to/project",
   "factsSummary": {
     "packageManager": "bun",
@@ -379,8 +394,9 @@ This is the only place agentscan starts a subprocess, and it runs only under
 
 `ignoreRules` disables a check project-wide, which is too blunt when a single
 hook uses an exotic launcher. `ignoreFindings` takes exact finding ids — the
-same strings `agentscan explain` accepts, so the value you already have from
-reading the report is the value you paste:
+same strings `agentscan explain` accepts. `check --verbose` prints the id under
+each finding, and `--json` carries it as `.findings[].id`, so the value you copy
+out of the report is the value you paste:
 
 ```json
 { "ignoreFindings": ["hook.missing-script:hook:PreToolUse:./bin/wrapper"] }
@@ -391,21 +407,26 @@ reading the report is the value you paste:
 Every check lives in `src/checks/` and runs on every `check`. `agentscan rules`
 lists all of them with their ids; `agentscan explain <id>` details any finding.
 
-Most validate one discovered item against its own file on disk. The five
-`budget.*` / `policy.*` entries at the bottom judge aggregate size instead —
-they are all **info**, they are sourced to `docs/spec/thresholds.md`, and they
-are retuned through `thresholds` in `.agentscanrc.json` rather than switched
-off.
+Most validate one discovered item against its own file on disk. The four
+`budget.*` entries at the bottom judge aggregate size instead, as does
+`skill.description-budget` — all five are **info**, all five are sourced to
+`docs/spec/thresholds.md`, and all five are retuned through `thresholds` in
+`.agentscanrc.json` rather than switched off.
 
 | id | Severity | Catches |
 |----|----------|---------|
 | `config.unreadable` | error | A config file that is not valid JSON, so whatever it declares is silently not in effect |
+| `scan.truncated` | info | A file past the scan cap, so the checks that read its body saw only a prefix — about this tool's reach, not about your project |
 | `hook.missing-script` | error | A registered hook whose script does not exist — it never runs |
 | `hook.unknown-event` | error | A hook registered under an event name that is never dispatched |
-| `agent.missing-frontmatter` | warning | An agent definition with no `---` block |
-| `agent.missing-description` | info | Agent frontmatter has no `description`, so it will not be dispatched on its own |
-| `mcp.no-launch` | error | An MCP server with neither `command` nor `url`; its tools are never available |
-| `mcp.url-without-type` | error | A remote MCP server with a `url` but no `type` — read as stdio, fails, and is skipped |
+| `agent.missing-frontmatter` | error | An agent definition with no `---` block |
+| `agent.missing-description` | error | Agent frontmatter has no `description` |
+| `agent.missing-name` | error | Agent frontmatter has no `name` |
+| `agent.invalid-name` | warning | Agent name is not lowercase letters, numbers, and hyphens |
+| `agent.duplicate-name` | error | Multiple agent files declare the same name |
+| `mcp.no-launch` | error | An MCP server with neither `command` nor `url` — unlaunchable by schema |
+| `mcp.command-missing` | error | An MCP `command` that is a path-like value whose file does not exist on disk |
+| `mcp.url-without-type` | error | A remote MCP server with a `url` but no `type` — read as stdio, misconfigured by schema, and skipped |
 | `mcp.hardcoded-secret` | error | A token-shaped literal in MCP config (the value is never echoed back) |
 | `mcp.literal-env` | warning | Long literal `env` values that should be `${VAR}` |
 | `skill.missing-skill-md` | warning | A directory under a skill path with no `SKILL.md` |
@@ -417,11 +438,14 @@ off.
 | `skill.not-in-lock` | info | A skill on disk that the lockfile does not track — local and unpinned |
 | `skill.description-budget` | info | Skill names + descriptions exceed the startup character budget |
 | `skill.no-lockfile` | info | Skills present with no lockfile at all (only with `requireLock`) |
+| `budget.agents-md` | info | `AGENTS.md` past the point where added lines stop helping (>150) |
+| `budget.claude-md` | info | `CLAUDE.md` past the instruction count a model reliably follows (>200) |
+| `budget.agents` | info | More agent definitions than a focused set (>8 in `.claude/agents`) |
+| `budget.mcp` | info | More MCP servers than the point where tool selection degrades (>5) |
 
 Agent definitions are checked for structure only. An agent's frontmatter `name`
-is a display name — 16 of 34 real files declare one that differs from the
-filename, and nothing keys on the filename — so it is deliberately not compared,
-and a test guards against re-adding that check. Model identifiers and tool names
+is not compared to the filename; it must simply be a valid lowercase identifier
+and unique. Model identifiers and tool names
 are likewise not validated: both would need a hardcoded list, which is the shape
 that already shipped a false error. See [plans/003](plans/003-validate-agent-definitions.md).
 
@@ -433,13 +457,14 @@ references measured across 17 projects, 1645 resolved skill-relative and 12 only
 at the root. Fenced code blocks are stripped first: a path in an example is
 illustration, not a pointer.
 
-`hook.missing-script` is deliberately conservative. A command is only resolved
-when it is a single invocation with a path-like argument; shell programs
-(`a && b`, `$(...)`, pipes) are skipped, because a hook written as
-`[ ! -f x ] || node x` already handles the missing file and flagging it would be
-a false positive. `node -e "<code>"` is never treated as a path. Only
-`$CLAUDE_PROJECT_DIR` is expanded — other variables are left alone rather than
-guessed at.
+`hook.missing-script` and `mcp.command-missing` are deliberately conservative.
+A value is only resolved when it is path-like and the answer is certain; shell
+programs (`a && b`, `$(...)`, pipes), bare PATH binaries (`npx`, `uvx`), and
+unresolved env vars are skipped. A false "broken" on a working `npx` server
+would be worse than silence. `node -e "<code>"` is never treated as a path.
+Only `$CLAUDE_PROJECT_DIR` is expanded — other variables are left alone rather
+than guessed at. Schema checks judge misconfiguration, not whether a correctly
+shaped entry will start at runtime.
 
 Checks are written against the published specs, not against what happens to
 appear in real projects. Two that were written the other way round shipped as
@@ -461,8 +486,15 @@ When adding a check, add its spec line there first.
   the `computedHash` in `skills-lock.json` is the obvious `refresh` source; the
   hash algorithm used by the installing tool is not documented here, and it is
   not reproducible from `SKILL.md` bytes alone, so it is not implemented.
-- **Bun only.** `import.meta.dir` / `import.meta.main`; it will not run on Node.
-- Policy files are read up to 100 KB, so `policyLines` undercounts past that.
+- **Runtime split.** Source development and the composite Action use Bun; the
+  published `dist/cli.js` runs on Node 20.11+ or Bun. The source entrypoint uses
+  Bun's `import.meta.dir` / `import.meta.main` and is not the Node entrypoint.
+- **Bounded reads.** A `SKILL.md` is read to 64 KB and a policy file to 100 KB,
+  so `skill.broken-reference` and `policyLines` see only that much of a larger
+  file. This is reported — `scan.truncated`, at `info` — rather than left
+  silent, and it says nothing about the file being valid: it is, and the tools
+  that read it see all of it. Reporting the cap as `config.unreadable` at
+  severity error is a defect this tool shipped and [plan 019](plans/019-findings-say-true-things.md) fixed.
 
 ## Development
 
@@ -480,14 +512,34 @@ bun run src/cli.ts check tests/fixtures/lock-drift
 | `test` | `bun test` |
 | `check` | `bun run src/cli.ts check` |
 
+### Release checklist
+
+Before publishing a release:
+
+1. Confirm `package.json` and `src/version.ts` contain the same version.
+2. Run `bun run build`, then verify `node dist/cli.js --version` prints it.
+3. Run `bun run typecheck`, `bun test`, and `bun run spec:check`.
+4. Refresh the test-count badge and version examples above from that release run.
+
 ## Design docs
 
 **This README is the source of truth for current behavior.** The documents under
 `docs/superpowers/specs/` are the original design and are **superseded** —
-several decisions changed while building: Bun-only runtime, the dep-to-skill map
-and its "orphan" heuristic dropped in favour of `skills-lock.json`, budget
-checks added, structural checks added, and the YAML rule engine deleted
-(`plans/010`). Read them as history, not as behavior.
+several decisions changed while building: the original Bun-only runtime claim was
+superseded by a Node-compatible published bundle, the dep-to-skill map and its
+"orphan" heuristic dropped in favour of `skills-lock.json`, budget checks added,
+structural checks added, and the YAML rule engine deleted (`plans/010`). Read
+them as history, not as behavior.
+
+## Skill
+
+A Cursor / Agent Skills file lives at [`skills/agentscan/SKILL.md`](skills/agentscan/SKILL.md). Copy the folder so the agent runs the audit before it edits hooks or claims a guard is in place:
+
+```bash
+cp -R skills/agentscan .cursor/skills/agentscan
+# or: .agents/skills/agentscan
+# or: .claude/skills/agentscan
+```
 
 ## License