agent-skill-manager 1.20.0 → 1.21.0

package/README.md

@@ -62,7 +62,7 @@ The more AI agents you use, the worse this gets. Every new tool adds another ski
 - **Install from GitHub in one command** — `asm install github:user/repo` handles cloning, validation, and placement. Supports single-skill repos, multi-skill collections, subfolder URLs, and private repos via SSH.
 - **Catch problems before they bite** — Built-in security scanning flags dangerous patterns (shell execution, network access, credential exposure, obfuscation) before you install. Duplicate audit finds and cleans redundant skills across providers.
 - **Create, test, and publish skills** — Scaffold new skills with `asm init`, symlink them for live development with `asm link`, audit for security issues, verify metadata, and publish to the [ASM Registry](https://github.com/luongnv89/asm-registry) with a single command. [See the full local dev workflow ↓](#build-test-and-ship-your-own-skills)
-- **Works with every major agent** — 17 providers built-in: Claude Code, Codex, OpenClaw, Cursor, Windsurf, Cline, Roo Code, Continue, GitHub Copilot, Aider, OpenCode, Zed, Augment, Amp, Gemini CLI, Google Antigravity, and a generic Agents provider. Add custom providers in seconds via config.
+- **Works with every major agent** — 18 providers built-in: Claude Code, Codex, OpenClaw, Cursor, Windsurf, Cline, Roo Code, Continue, GitHub Copilot, Aider, OpenCode, Zed, Augment, Amp, Gemini CLI, Google Antigravity, Hermes, and a generic Agents provider. Add custom providers in seconds via config.
 - **Two interfaces, one tool** — Full interactive TUI with keyboard navigation, search, and detail views. Or use the CLI with `--json` for scripting and automation.
 
 <p align="center">
@@ -275,9 +275,10 @@ asm publish --yes ./my-skill
 4. Test with your AI agent
 5. **Security audit** — `asm audit security awesome-skill`
 6. **Verify metadata** — `asm inspect awesome-skill`
-7. Push to GitHub
-8. **Verify install flow** — `asm install github:you/awesome-skill`
-9. **Publish to registry** — `asm publish ./awesome-skill`
+7. **Score quality** — `asm eval ./awesome-skill` (add `--runtime` for skillgrade runtime evals)
+8. Push to GitHub
+9. **Verify install flow** — `asm install github:you/awesome-skill`
+10. **Publish to registry** — `asm publish ./awesome-skill`
 
 Whether you're building skills for yourself or publishing them for the community, `asm` gives you the full create → develop → audit → ship pipeline in one tool.
 
@@ -317,6 +318,37 @@ asm index search "your-skill" --json
 
 Each indexed skill in the output JSON includes `"verified": true` or `"verified": false`. If verification fails, the ingestion debug log (set `ASM_DEBUG=1`) prints the specific reasons.
 
+### Runtime Evaluation (`asm eval`)
+
+Static verification tells you the SKILL.md is well-formed. `asm eval` goes further and answers two orthogonal questions about any skill on disk:
+
+1. **Is it well-written?** — `quality@1.0.0` ships by default and runs a scored rubric over structure, frontmatter, clarity, and safety.
+2. **Does it actually work?** — `asm eval --runtime` shells out to [skillgrade](https://github.com/mgechev/skillgrade) for deterministic + LLM-judge runtime evals in a Docker sandbox, with CI-ready exit codes.
+
+**Zero-setup install:** skillgrade ships as a direct dependency of `asm`. After `npm install -g agent-skill-manager`, `asm eval --runtime` just works — no `npm i -g skillgrade`, no PATH hijacking. Override with `ASM_SKILLGRADE_BIN=/path/to/skillgrade` if you want to point at a different binary.
+
+```bash
+# Static quality lint (default)
+asm eval ./my-skill
+
+# Scaffold eval.yaml for runtime tests
+asm eval ./my-skill --runtime init
+
+# Run the skillgrade runtime provider
+asm eval ./my-skill --runtime --preset smoke
+
+# CI-friendly JSON
+asm eval ./my-skill --runtime --machine --threshold 0.8
+
+# List registered eval providers
+asm eval-providers list
+
+# Diff two provider versions before promoting an upgrade
+asm eval ./my-skill --compare skillgrade@1.0.0,skillgrade@2.0.0-next
+```
+
+The eval surface is a pluggable provider framework: each provider implements a common `EvalProvider` contract and resolves via semver range, so you can pin a version in `~/.asm/config.yml`, diff two versions side-by-side with `--compare`, and add new providers without touching the CLI. See [`docs/eval-providers.md`](./docs/eval-providers.md) for the provider model and [`docs/skillgrade-integration.md`](./docs/skillgrade-integration.md) for skillgrade install, presets, and CI usage.
+
 ---
 
 ## ASM Registry — Install and Publish Skills by Name
@@ -443,7 +475,7 @@ asm install github:anthropics/skills --all
 
 ## Supported Agent Tools
 
-`asm` ships with **17 built-in providers**, all enabled by default. Disable any you don't need via `asm config edit`.
+`asm` ships with **18 built-in providers**, all enabled by default. Disable any you don't need via `asm config edit`.
 
 | Tool               | Global Path                       | Project Path            | Default |
 | ------------------ | --------------------------------- | ----------------------- | :-----: |
@@ -464,6 +496,7 @@ asm install github:anthropics/skills --all
 | Amp                | `~/.amp/skills/`                  | `.amp/skills/`          | enabled |
 | Gemini CLI         | `~/.gemini/skills/`               | `.gemini/skills/`       | enabled |
 | Google Antigravity | `~/.antigravity/skills/`          | `.antigravity/skills/`  | enabled |
+| Hermes             | `~/.hermes/skills/`               | `.hermes/skills/`       | enabled |
 
 Disable a provider — opens config in `$EDITOR`, set `"enabled": false` for any provider:
 
@@ -481,10 +514,10 @@ Need a tool not listed? Add a custom provider entry to the config.
 Yes. `asm` is MIT licensed and free forever. No accounts, no telemetry, no paywalls.
 
 **Is it actively maintained?**
-v1.20.0 shipped on April 12, 2026. The project has had 29 releases. Check the [changelog](docs/CHANGELOG.md) for the full history.
+v1.21.0 shipped on April 19, 2026. The project has had 30 releases. Check the [changelog](docs/CHANGELOG.md) for the full history.
 
 **Which AI agents does it support?**
-17 providers built-in: Claude Code, Codex, OpenClaw, Cursor, Windsurf, Cline, Roo Code, Continue, GitHub Copilot, Aider, OpenCode, Zed, Augment, Amp, Gemini CLI, Google Antigravity, and a generic Agents provider. All 17 are enabled by default; disable any you don't need via `asm config edit`. You can also add any custom agent that stores skills as directories with a `SKILL.md` file.
+18 providers built-in: Claude Code, Codex, OpenClaw, Cursor, Windsurf, Cline, Roo Code, Continue, GitHub Copilot, Aider, OpenCode, Zed, Augment, Amp, Gemini CLI, Google Antigravity, Hermes, and a generic Agents provider. All 18 are enabled by default; disable any you don't need via `asm config edit`. You can also add any custom agent that stores skills as directories with a `SKILL.md` file.
 
 **How does it compare to managing skills manually?**
 Manual management means remembering where each agent stores skills, cloning repos by hand, checking for duplicates yourself, and having no security scanning. `asm` automates all of that with one command.
@@ -545,6 +578,9 @@ asm
 | `asm link <path> [<path2> ...]` | Symlink one or more local skills for live development |
 | `asm audit`                     | Detect duplicate skills                               |
 | `asm audit security <name>`     | Run security audit on a skill                         |
+| `asm eval <skill>`              | Score a skill via the pluggable eval framework        |
+| `asm eval <skill> --runtime`    | Runtime evaluation via skillgrade (LLM-judge)         |
+| `asm eval-providers list`       | List registered eval providers and versions           |
 | `asm stats`                     | Show aggregate skill metrics dashboard                |
 | `asm export`                    | Export skill inventory as JSON manifest               |
 | `asm index ingest <repo>`       | Index a skill repo for searching                      |
@@ -606,6 +642,24 @@ Audit all installed skills:
 asm audit security --all
 ```
 
+Score a skill with the static quality provider:
+
+```bash
+asm eval ./my-skill
+```
+
+Run the skillgrade runtime evaluator (requires `skillgrade` on PATH):
+
+```bash
+asm eval ./my-skill --runtime --preset smoke
+```
+
+List registered eval providers:
+
+```bash
+asm eval-providers list
+```
+
 Scaffold a skill, link it for live testing, audit, and inspect:
 
 ```bash
@@ -769,7 +823,7 @@ The install command clones the repository, validates `SKILL.md` files, scans for
 <details>
 <summary><strong>Configuration</strong></summary>
 
-On first run, a config file is created at `~/.config/agent-skill-manager/config.json` with 17 default providers, all enabled:
+On first run, a config file is created at `~/.config/agent-skill-manager/config.json` with 18 default providers, all enabled:
 
 ```json
 {
@@ -1085,16 +1139,18 @@ agent-skill-manager/
 <details>
 <summary><strong>Documentation</strong></summary>
 
-| Document                              | Description                              |
-| ------------------------------------- | ---------------------------------------- |
-| [Architecture](docs/ARCHITECTURE.md)  | System design, components, and data flow |
-| [Development](docs/DEVELOPMENT.md)    | Local setup, testing, and debugging      |
-| [Deployment](docs/DEPLOYMENT.md)      | Publishing and CI pipeline               |
-| [Changelog](docs/CHANGELOG.md)        | Version history                          |
-| [Brand Kit](docs/brand_kit.md)        | Logo, colors, and typography             |
-| [Contributing](CONTRIBUTING.md)       | How to contribute                        |
-| [Security](SECURITY.md)               | Vulnerability reporting                  |
-| [Code of Conduct](CODE_OF_CONDUCT.md) | Community guidelines                     |
+| Document                                                 | Description                                              |
+| -------------------------------------------------------- | -------------------------------------------------------- |
+| [Architecture](docs/ARCHITECTURE.md)                     | System design, components, and data flow                 |
+| [Eval Providers](docs/eval-providers.md)                 | Pluggable eval framework, `--compare`, adding a provider |
+| [Skillgrade Integration](docs/skillgrade-integration.md) | Install, presets, CI usage, troubleshooting              |
+| [Development](docs/DEVELOPMENT.md)                       | Local setup, testing, and debugging                      |
+| [Deployment](docs/DEPLOYMENT.md)                         | Publishing and CI pipeline                               |
+| [Changelog](docs/CHANGELOG.md)                           | Version history                                          |
+| [Brand Kit](docs/brand_kit.md)                           | Logo, colors, and typography                             |
+| [Contributing](CONTRIBUTING.md)                          | How to contribute                                        |
+| [Security](SECURITY.md)                                  | Vulnerability reporting                                  |
+| [Code of Conduct](CODE_OF_CONDUCT.md)                    | Community guidelines                                     |
 
 </details>