agent-skill-eval 0.3.0__tar.gz → 0.4.1__tar.gz

{agent_skill_eval-0.3.0 → agent_skill_eval-0.4.1}/.github/ISSUE_TEMPLATE/bug_report.yml

@@ -15,7 +15,7 @@ body:
       label: Steps to reproduce
       description: Minimal steps to reproduce the behavior
       placeholder: |
-        1. skill-eval run --skill ... --evals ... --agent opencode
+        1. agent-skill-eval run --skill ... --evals ... --agent opencode
         2. ...
     validations:
       required: true
@@ -30,7 +30,7 @@ body:
     id: version
     attributes:
       label: Version
-      description: Output of `skill-eval --version`
+      description: Output of `agent-skill-eval --version`
   - type: input
     id: python
     attributes:

{agent_skill_eval-0.3.0 → agent_skill_eval-0.4.1}/.github/workflows/ci.yml

@@ -37,7 +37,7 @@ jobs:
       - uses: ./.github/actions/setup
         with:
           python-version: ${{ matrix.python-version }}
-      - run: pytest -v --junitxml=results.xml --cov=skill_eval --cov-report=xml:coverage.xml
+      - run: pytest -v --junitxml=results.xml --cov=agent_skill_eval --cov-report=xml:coverage.xml
       - name: Upload test results
         if: always()
         uses: actions/upload-artifact@v4

agent_skill_eval-0.4.1/.github/workflows/release.yml

@@ -0,0 +1,48 @@
+name: Release
+
+on:
+  push:
+    branches:
+      - main
+
+permissions:
+  contents: write
+  id-token: write
+
+jobs:
+  release:
+    runs-on: ubuntu-latest
+    concurrency: release
+    environment:
+      name: pypi
+      url: https://pypi.org/project/agent-skill-eval/
+
+    steps:
+    - uses: actions/checkout@v4
+      with:
+        fetch-depth: 0
+        token: ${{ secrets.GITHUB_TOKEN }}
+
+    - name: Install uv
+      uses: astral-sh/setup-uv@v4
+
+    - name: Set up Python
+      run: uv python install 3.12
+
+    - name: Configure Git
+      run: |
+        git config user.name "github-actions[bot]"
+        git config user.email "github-actions[bot]@users.noreply.github.com"
+
+    - name: Semantic Release
+      env:
+        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      run: |
+        uv run --with 'python-semantic-release>=9,<10' semantic-release version
+        uv run --with 'python-semantic-release>=9,<10' semantic-release publish
+
+    - name: Build package
+      run: uv build
+
+    - name: Publish to PyPI
+      uses: pypa/gh-action-pypi-publish@release/v1

{agent_skill_eval-0.3.0 → agent_skill_eval-0.4.1}/AGENTS.md

@@ -2,7 +2,7 @@
 
 ## Project
 
-Python CLI framework (`skill-eval`) for evaluating agent skills across OpenCode, Claude Code, and Codex. Uses Typer, Pydantic, OpenAI SDK, and Rich.
+Python CLI framework (`agent-skill-eval`) for evaluating agent skills across OpenCode, Claude Code, and Codex. Uses Typer, Pydantic, OpenAI SDK, and Rich.
 
 ## Quick commands
 
@@ -24,7 +24,7 @@ uv run --extra dev ruff check src/ tests/ && uv run --extra dev ruff format --ch
 ## Architecture
 
 ```
-src/skill_eval/
+src/agent_skill_eval/
 ├── cli.py            # Typer CLI app (run, report, grade, cleanup, init)
 ├── runner.py         # EvalRunner orchestrates eval execution
 ├── graders/__init__.py   # DeterministicGrader + LLMGrader (700+ lines, single file)
@@ -35,8 +35,8 @@ src/skill_eval/
 └── workspace.py      # Workspace creation (fresh git init or clone)
 ```
 
-- Entry point: `skill_eval.cli:app` (registered as `skill-eval` console script)
-- Package layout: `src/skill_eval/` (hatchling build, wheel packages `src/skill_eval`)
+- Entry point: `agent_skill_eval.cli:app` (registered as `agent-skill-eval` console script)
+- Package layout: `src/agent_skill_eval/` (hatchling build, wheel packages `src/agent_skill_eval`)
 - The `fake` harness (used in tests) is a no-op agent that produces deterministic output
 
 ## Testing

{agent_skill_eval-0.3.0 → agent_skill_eval-0.4.1}/CHANGELOG.md

@@ -1,28 +1,93 @@
 # Changelog
 
 All notable changes to this project will be documented in this file.
+Entries above the insertion flag are generated by python-semantic-release
+from conventional commits; entries below it are curated by hand.
 
-## [0.3.0] - 2026-06-11
+<!-- version list -->
+
+## v0.4.1 (2026-06-11)
+
+### Bug Fixes
+
+- Todo batch — grader honesty, cost reporting, curated changelog
+  ([#11](https://github.com/tardigrde/agent-skill-eval/pull/11),
+  [`33ab02c`](https://github.com/tardigrde/agent-skill-eval/commit/33ab02c6c5ac3459d1987ba46ae3cc85038f8685))
+
+* fix: grader honesty — skip-with-warning, workspace fallback warning, behavior-based negative
+  control
+
+- LLMGrader: grading errors and missing per-assertion results are now skipped (not failed) and
+  warned about on stderr, so grader failures never masquerade as agent failures. - Workspace
+  resolution for file-based checks is shared between the deterministic and LLM graders and warns
+  (once per run dir) when it falls back to guessing from the output directory layout. -
+  fix-failing-tests negative-control: reword the grader-run-dependent 'output contains the source
+  code' assertion to the behavior that matters (file read and displayed/described).
+
+Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
+
+* feat: cost column in report, validate flags LLM-rubric assertions, curated changelog
+
+- benchmark.json: per-config cost_usd mean/stddev and per-agent cost deltas; report gains a Cost
+  (USD) column (table and markdown). - validate: classify_assertion() reports which assertions match
+  no deterministic pattern and will be graded by the LLM rubric, instead of that happening silently
+  at run time. - CHANGELOG.md: restore curated pre-0.4 entries (PSR-style headings);
+  semantic-release now runs in insertion-only update mode with vendored templates that render commit
+  summary lines only — no more commit bodies or Co-Authored-By trailers leaking into the changelog.
+  - Version is single-sourced from pyproject.toml (PSR version_variables removed; __init__ falls
+  back to 0.0.0 when not installed). - TODO.md: tick everything done in this batch and items
+  verified already done on main.
+
+* fix: normalize assertion text when matching LLM grader responses
+
+LLMs echo assertions with minor drift (casing, trailing periods, whitespace); exact-match made those
+  falsely count as missing results. Addresses Gemini review on PR #11.
+
+* chore: remove vendored PSR templates, use defaults
+
+---------
+
+Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
+
+### Documentation
+
+- Update TODO — v0.4.0 state, subagent evals next, agent working notes
+  ([`4b5b76f`](https://github.com/tardigrde/agent-skill-eval/commit/4b5b76faed27678bf81a23cb9b2478e90c590c82))
+
+Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
+
+
+## v0.4.0 (2026-06-11)
+
+### Changed
+- Converged all names on `agent-skill-eval`: GitHub repo (was `skill-evals`), CLI entry point (was `skill-eval`), and Python module (was `skill_eval`) now all match the PyPI package name
+- Added short CLI alias `ase` (same app as `agent-skill-eval`)
+- Env var prefix renamed `SKILL_EVAL_*` -> `ASE_*` (`ASE_AGENT_TIMEOUT`, `ASE_AGENT_RETRIES`, `ASE_LIVE`)
+
+### Added
+- Codex harness fixes (`--sandbox workspace-write`, `--skip-git-repo-check`), `cost_usd` in `timing.json`, two-tier e2e tests, Makefile eval/test targets (merged via PR #9, previously unreleased)
+
+## v0.3.0 (2026-06-11)
 
 ### Added
 - `validate-config` example skill + eval suite: exercises bundled resources (agent runs skill's `scripts/` and reads its `references/`); graded via command-ran + file + content checks; fully offline
 - `review-diff` example skill + eval suite: read-only analysis (planted bugs found, documented decoy not flagged, nothing modified); graded via chat-output-only content + LLM rubric; fully offline
 
-## [0.2.1] - 2026-06-11
+## v0.2.1 (2026-06-11)
 
 ### Changed
 - Renamed package from `skill-eval` to `agent-skill-eval` (PyPI name collision)
 
-## [0.2.0] - 2026-06-10
+## v0.2.0 (2026-06-10)
 
 ### Added
 - `write-release-notes` example skill + eval suite: exercises LLM rubric grading (grouping, breaking-change prominence, anti-fabrication) instead of git-state checks
 - `--runs N` repeats every (eval, agent, config); `benchmark.json` now reports `full_pass_rate`, `pass_at_k`, and `k`
 - `--agent-model agent=model` (repeatable) and `--harness-base-url` to pin models and endpoints per agent CLI
 - `--timeout` / `--retries` flags; harnesses retry on timeout or non-zero exit with backoff and record `exit_code`, `timed_out`, `retries` in `timing.json`
-- `skill-eval validate`: schema + fixture-existence + duplicate-id checks for evals.json
-- `skill-eval list`: discover eval suites and skills under a directory
-- `skill-eval compare`: side-by-side pass rates of two iterations
+- `agent-skill-eval validate`: schema + fixture-existence + duplicate-id checks for evals.json
+- `agent-skill-eval list`: discover eval suites and skills under a directory
+- `agent-skill-eval compare`: side-by-side pass rates of two iterations
 - `report --format markdown` and `report --show-evidence` (failed/skipped assertion evidence)
 - JSON Schema for the eval suite format at `schemas/evals.schema.json` (with a sync test)
 - SKILL.md frontmatter validation warning on `run` (missing name/description, name mismatch)
@@ -39,7 +104,7 @@ All notable changes to this project will be documented in this file.
 - Multi-agent runs no longer report a misleading `Delta: 0.0%`: `benchmark.json` now has per-agent `deltas`; the legacy `delta` field is only set for single-agent runs (otherwise `null`)
 - Removed dead `cost_usd` handling in the Claude Code output parser
 
-## [0.1.0-alpha.3] - 2026-06-07
+## v0.1.0-alpha.3 (2026-06-07)
 
 ### Added
 - Shared test fixtures in `conftest.py`
@@ -57,7 +122,7 @@ All notable changes to this project will be documented in this file.
 - EvalCase contradiction validator (`force_skill_invocation` + `should_trigger`)
 - LLMGrader now fails on missing assertions instead of silently dropping them
 
-## [0.1.0-alpha.2] - 2026-06-02
+## v0.1.0-alpha.2 (2026-06-02)
 
 ### Added
 - CI workflow with Python 3.11/3.12/3.13 matrix, caching, coverage, and smoke test
@@ -71,7 +136,7 @@ All notable changes to this project will be documented in this file.
 - `github_repo_slug` function to parse GitHub URLs correctly
 - Handoff review fixes and PR grading tightening
 
-## [0.1.0-alpha.1] - 2026-05-27
+## v0.1.0-alpha.1 (2026-05-27)
 
 ### Added
 - Initial release: skill evaluation framework for OpenCode, Claude Code, and Codex
@@ -87,7 +152,7 @@ All notable changes to this project will be documented in this file.
 - OpenRouter grader support via `OPENROUTER_API_KEY`
 - Workspace isolation with fresh git repos per eval run
 - Token/timing extraction from each agent's native output format
-- `python -m skill_eval` entry point
+- `python -m agent_skill_eval` entry point
 
 ### Fixed
 - OpenRouter grader model resolution and opencode parser output handling

{agent_skill_eval-0.3.0 → agent_skill_eval-0.4.1}/CONTRIBUTING.md

@@ -3,8 +3,8 @@
 ## Setup
 
 ```bash
-git clone https://github.com/tardigrde/skill-evals.git
-cd skill-evals
+git clone https://github.com/tardigrde/agent-skill-eval.git
+cd agent-skill-eval
 uv venv
 uv pip install -e ".[dev]"
 ```

agent_skill_eval-0.4.1/Makefile

@@ -0,0 +1,107 @@
+# agent-skill-eval Makefile
+#
+# Usage:
+#   make cheap-eval SKILL=examples/fix-failing-tests EVALS=examples/fix-failing-tests/evals/evals.json
+#   make fake-eval  SKILL=examples/fix-failing-tests EVALS=examples/fix-failing-tests/evals/evals.json
+#   make test
+#   make lint
+#
+# OpenRouter setup (set in shell or .env):
+#   OPENROUTER_API_KEY=sk-or-...
+#   ANTHROPIC_BASE_URL=https://openrouter.ai/api      # claude-code reads this
+#   ANTHROPIC_AUTH_TOKEN=$OPENROUTER_API_KEY           # claude-code auth via OpenRouter
+#   ANTHROPIC_API_KEY=                                 # must be blank when using OpenRouter
+#   OPENAI_BASE_URL=https://openrouter.ai/api/v1       # opencode reads this
+
+# ── Defaults ────────────────────────────────────────────────────────────────
+
+# Read OPENROUTER_API_KEY from .env if not already in shell
+OPENROUTER_API_KEY ?= $(shell grep -s '^OPENROUTER_API_KEY' .env | cut -d= -f2- | tr -d '"' | tr -d "'")
+
+SKILL  ?= skills/fix-failing-tests
+EVALS  ?= examples/fix-failing-tests/evals/evals.json
+
+# Pinned free/cheap models (see config/baseline.env to override).
+# claude-code needs a real Anthropic model ID: the claude CLI validates
+# --model client-side and rejects OpenRouter-only slugs.
+CLAUDE_CODE_MODEL  ?= claude-haiku-4-5-20251001
+OPENCODE_MODEL     ?= deepseek/deepseek-v4-flash:free
+CODEX_MODEL        ?= gpt-5.4-mini
+GRADER_MODEL       ?= deepseek/deepseek-v4-flash:free
+
+# Agents to run (space-separated). Override: make cheap-eval AGENTS="fake codex"
+AGENTS ?= fake claude-code opencode
+
+WORKSPACE   ?= eval-workspace
+CONCURRENCY ?= 3
+
+-include config/baseline.env
+
+AGENT_FLAGS = $(foreach a,$(AGENTS),--agent $(a))
+
+# ── Targets ──────────────────────────────────────────────────────────────────
+
+.PHONY: cheap-eval full-eval fake-eval test test-e2e test-live lint help
+
+## cheap-eval: $(AGENTS) in parallel with cheap/free models
+## claude-code routes via OpenRouter (needs OPENROUTER_API_KEY in .env).
+## opencode routes via its own Zen provider (needs zen auth, no extra env).
+## codex uses ~/.codex auth + config.toml model (run via full-eval or AGENTS=).
+cheap-eval: export ANTHROPIC_BASE_URL    = https://openrouter.ai/api
+cheap-eval: export ANTHROPIC_AUTH_TOKEN  = $(OPENROUTER_API_KEY)
+cheap-eval: export ANTHROPIC_API_KEY     =
+cheap-eval:
+	uv run agent-skill-eval run \
+		--skill $(SKILL) \
+		--evals $(EVALS) \
+		$(AGENT_FLAGS) \
+		--agent-model "claude-code=$(CLAUDE_CODE_MODEL)" \
+		--agent-model "opencode=$(OPENCODE_MODEL)" \
+		--agent-model "codex=$(CODEX_MODEL)" \
+		--grader-model "$(GRADER_MODEL)" \
+		--workspace $(WORKSPACE) \
+		--concurrency $(CONCURRENCY) \
+		--no-baseline
+
+## full-eval: cheap-eval plus codex (needs ~/.codex/auth.json login)
+full-eval: AGENTS = fake claude-code opencode codex
+full-eval: cheap-eval
+
+## fake-eval: fake harness only — zero API cost, CI-safe, no env vars needed
+fake-eval:
+	uv run agent-skill-eval run \
+		--skill $(SKILL) \
+		--evals $(EVALS) \
+		--agent fake \
+		--grader-model "" \
+		--workspace $(WORKSPACE) \
+		--concurrency 1
+
+## test: run unit + smoke tests (includes free e2e pipeline test)
+test:
+	uv run pytest -v --cov=agent_skill_eval --cov-report=term-missing tests/
+
+## test-e2e: free end-to-end pipeline tests only (fake harness, no API calls)
+test-e2e:
+	uv run pytest -v tests/test_e2e.py -m "not live"
+
+## test-live: paid e2e tests against real agents + grader (claude-code, opencode, codex)
+test-live: export ASE_LIVE        = 1
+test-live: export ANTHROPIC_BASE_URL     = https://openrouter.ai/api
+test-live: export ANTHROPIC_AUTH_TOKEN   = $(OPENROUTER_API_KEY)
+test-live: export ANTHROPIC_API_KEY      =
+test-live: export CLAUDE_CODE_MODEL      := $(CLAUDE_CODE_MODEL)
+test-live: export OPENCODE_MODEL         := $(OPENCODE_MODEL)
+test-live: export CODEX_MODEL            := $(CODEX_MODEL)
+test-live: export GRADER_MODEL           := $(GRADER_MODEL)
+test-live:
+	uv run pytest -v tests/test_e2e.py -m live
+
+## lint: ruff check + format check
+lint:
+	uv run ruff check src/ tests/
+	uv run ruff format --check src/ tests/
+
+## help: list targets
+help:
+	@grep -E '^## ' Makefile | sed 's/## /  /'

{agent_skill_eval-0.3.0 → agent_skill_eval-0.4.1}/PKG-INFO

@@ -1,10 +1,10 @@
 Metadata-Version: 2.4
 Name: agent-skill-eval
-Version: 0.3.0
+Version: 0.4.1
 Summary: Framework for evaluating agent skills across OpenCode, Claude Code, and Codex
-Project-URL: Homepage, https://github.com/tardigrde/skill-evals
-Project-URL: Repository, https://github.com/tardigrde/skill-evals
-Author: skill-eval contributors
+Project-URL: Homepage, https://github.com/tardigrde/agent-skill-eval
+Project-URL: Repository, https://github.com/tardigrde/agent-skill-eval
+Author: agent-skill-eval contributors
 License-Expression: MIT
 License-File: LICENSE
 Keywords: agent,benchmark,evaluation,llm,skills
@@ -29,11 +29,11 @@ Requires-Dist: pytest>=8.0; extra == 'dev'
 Requires-Dist: ruff>=0.4; extra == 'dev'
 Description-Content-Type: text/markdown
 
-# skill-eval
+# agent-skill-eval
 
 **Evaluate agent skills through the real coding harnesses you use every day — Claude Code, Codex, and OpenCode — not the raw API.**
 
-You wrote a `SKILL.md`. Does it actually make your agent better? skill-eval answers that with data: it installs your skill into a fresh workspace, runs the *actual agent CLI* against your test prompts (with and without the skill), grades the results with deterministic state-diff checks plus an LLM rubric, and reports the measured impact — pass rates, pass@k across repeated runs, token costs, and wall-clock time.
+You wrote a `SKILL.md`. Does it actually make your agent better? agent-skill-eval answers that with data: it installs your skill into a fresh workspace, runs the *actual agent CLI* against your test prompts (with and without the skill), grades the results with deterministic state-diff checks plus an LLM rubric, and reports the measured impact — pass rates, pass@k across repeated runs, token costs, and wall-clock time.
 
 Because the eval goes through the full harness — system prompt, skill discovery, permissions, tool use — you get exactly the behavior you'll see in daily use, including the failure mode that matters most: *the agent never triggering your skill at all*.
 
@@ -47,20 +47,24 @@ Because the eval goes through the full harness — system prompt, skill discover
 - **Honest grading**: assertions the grader can't check are *skipped*, not failed; a missing API key warns upfront instead of silently zeroing your pass rate
 - **Pinned models**: `--agent-model claude-code=haiku --agent-model codex=gpt-5-mini` makes runs reproducible across machines
 - **Scoped cleanup**: only removes artifacts recorded in `cleanup.json`; never closes unrelated PRs or deletes unrelated branches
-- **Re-grading**: `skill-eval grade` re-grades saved outputs without re-running agents
-- **Markdown reports**: paste `skill-eval report --format markdown` straight into a PR or blog post
+- **Re-grading**: `agent-skill-eval grade` re-grades saved outputs without re-running agents
+- **Markdown reports**: paste `agent-skill-eval report --format markdown` straight into a PR or blog post
 
 ## Installation
 
 ```bash
-pip install skill-eval
+pip install agent-skill-eval
 ```
 
+This installs two identical commands: `agent-skill-eval` and the short alias `ase`.
+
+> **Coming soon:** subagent evals — evaluate custom subagent definitions the same way as skills, across the same harnesses.
+
 Or from source with `uv`:
 
 ```bash
-git clone https://github.com/tardigrde/skill-evals
-cd skill-evals
+git clone https://github.com/tardigrde/agent-skill-eval
+cd agent-skill-eval
 uv venv && uv pip install -e ".[dev]"
 ```
 
@@ -70,13 +74,13 @@ You also need the agent CLIs you want to evaluate (`claude`, `codex`, `opencode`
 
 ```bash
 # 1. See what's available
-skill-eval list
+agent-skill-eval list
 
 # 2. Validate an eval suite
-skill-eval validate examples/write-release-notes/evals/evals.json
+agent-skill-eval validate examples/write-release-notes/evals/evals.json
 
 # 3. Run it (pin cheap models while iterating)
-skill-eval run \
+agent-skill-eval run \
   --skill ./skills/write-release-notes \
   --evals ./examples/write-release-notes/evals/evals.json \
   --agent claude-code --agent-model claude-code=haiku \
@@ -84,13 +88,13 @@ skill-eval run \
   --runs 3
 
 # 4. Read the results
-skill-eval report --workspace ./eval-workspace/write-release-notes-workspace --show-evidence
+agent-skill-eval report --workspace ./eval-workspace/write-release-notes-workspace --show-evidence
 ```
 
 To start a suite for your own skill:
 
 ```bash
-skill-eval init my-skill
+agent-skill-eval init my-skill
 ```
 
 This scaffolds `my-skill/SKILL.md` (frontmatter template), `my-skill/evals/evals.json` (one positive case and one negative control), and `my-skill/evals/files/` for fixtures.
@@ -129,7 +133,7 @@ This scaffolds `my-skill/SKILL.md` (frontmatter template), `my-skill/evals/evals
 }
 ```
 
-A JSON Schema for this format ships at [`schemas/evals.schema.json`](schemas/evals.schema.json) — point your editor at it for autocompletion, and run `skill-eval validate <file>` in CI.
+A JSON Schema for this format ships at [`schemas/evals.schema.json`](schemas/evals.schema.json) — point your editor at it for autocompletion, and run `agent-skill-eval validate <file>` in CI.
 
 ### Eval fields
 
@@ -174,7 +178,7 @@ Anything else falls through to the LLM rubric. With `should_trigger: false`, the
 ### `run`
 
 ```bash
-skill-eval run \
+agent-skill-eval run \
   --skill <skill-dir> --evals <evals.json> \
   --agent opencode --agent claude-code --agent codex \
   [--agent-model claude-code=haiku] [--agent-model codex=gpt-5-mini] \
@@ -192,12 +196,12 @@ Key options:
 - `--agent-model, -m`: model per agent as `agent=model`; a bare value applies to all agents. Repeatable.
 - `--harness-base-url`: injected as `ANTHROPIC_BASE_URL` (claude-code) / `OPENAI_BASE_URL` (codex, opencode) into the agent process.
 - `--runs, -n`: repeat each (eval, agent, config) N times; enables pass@k stats. Results land in `run-1/`, `run-2/`, ... subdirectories.
-- `--timeout` / `--retries`: per-run agent timeout (default 600s) and retries on timeout or non-zero exit (default 1). Also settable via `SKILL_EVAL_AGENT_TIMEOUT` / `SKILL_EVAL_AGENT_RETRIES`.
+- `--timeout` / `--retries`: per-run agent timeout (default 600s) and retries on timeout or non-zero exit (default 1). Also settable via `ASE_AGENT_TIMEOUT` / `ASE_AGENT_RETRIES`.
 
 ### `report`
 
 ```bash
-skill-eval report --workspace <skill-workspace> [--iteration N] [--format table|markdown] [--show-evidence]
+agent-skill-eval report --workspace <skill-workspace> [--iteration N] [--format table|markdown] [--show-evidence]
 ```
 
 `--show-evidence` prints the evidence string for every failed or skipped assertion — the state diff for deterministic checks, the judge's reasoning for LLM checks. `--format markdown` emits a paste-ready table.
@@ -205,7 +209,7 @@ skill-eval report --workspace <skill-workspace> [--iteration N] [--format table|
 ### `compare`
 
 ```bash
-skill-eval compare --workspace <skill-workspace> 1 2
+agent-skill-eval compare --workspace <skill-workspace> 1 2
 ```
 
 Side-by-side pass rates of two iterations, per configuration, with the change — the feedback loop for iterating on a SKILL.md.
@@ -213,7 +217,7 @@ Side-by-side pass rates of two iterations, per configuration, with the change
 ### `validate`
 
 ```bash
-skill-eval validate path/to/evals.json
+agent-skill-eval validate path/to/evals.json
 ```
 
 Schema check plus referenced-fixture existence and duplicate-id detection. Exit code 1 on any problem (CI-friendly).
@@ -221,7 +225,7 @@ Schema check plus referenced-fixture existence and duplicate-id detection. Exit
 ### `list`
 
 ```bash
-skill-eval list [--root .]
+agent-skill-eval list [--root .]
 ```
 
 Discovers eval suites (`evals.json`) and skills (`SKILL.md`) under a directory.
@@ -229,7 +233,7 @@ Discovers eval suites (`evals.json`) and skills (`SKILL.md`) under a directory.
 ### `grade`
 
 ```bash
-skill-eval grade --workspace <iteration-dir> [--recompute-benchmark]
+agent-skill-eval grade --workspace <iteration-dir> [--recompute-benchmark]
 ```
 
 Re-grades existing outputs using saved `evals_meta.json` and state snapshots. Two caveats: LLM-graded assertions are re-evaluated from scratch and may flip verdicts, and because the original agent workspace is deleted after the run, the judge re-grades from the saved artifacts (agent output, logs) rather than the live workspace files.
@@ -237,7 +241,7 @@ Re-grades existing outputs using saved `evals_meta.json` and state snapshots. Tw
 ### `cleanup`
 
 ```bash
-skill-eval cleanup --workspace ./eval-workspace [--yes]
+agent-skill-eval cleanup --workspace ./eval-workspace [--yes]
 ```
 
 Only closes PRs and deletes remote branches recorded in `cleanup.json`. Never touches unrelated PRs, branches, or workspaces.
@@ -245,7 +249,7 @@ Only closes PRs and deletes remote branches recorded in `cleanup.json`. Never to
 ### `init`
 
 ```bash
-skill-eval init my-skill [--output ./examples]
+agent-skill-eval init my-skill [--output ./examples]
 ```
 
 ## Example skills
@@ -301,18 +305,18 @@ eval-workspace/
 }
 ```
 
-To debug a failure: find `"passed": false` entries, read `evidence`, compare `pre_state.json`/`post_state.json`, then check `outputs/output.txt` for the agent's full response. Or just run `skill-eval report --show-evidence`.
+To debug a failure: find `"passed": false` entries, read `evidence`, compare `pre_state.json`/`post_state.json`, then check `outputs/output.txt` for the agent's full response. Or just run `agent-skill-eval report --show-evidence`.
 
 ### Iterating on a skill
 
-1. `skill-eval run ...` → 2. `skill-eval report --show-evidence` → 3. edit SKILL.md → 4. `skill-eval run --iteration 2 ...` → 5. `skill-eval compare --workspace ... 1 2`
+1. `agent-skill-eval run ...` → 2. `agent-skill-eval report --show-evidence` → 3. edit SKILL.md → 4. `agent-skill-eval run --iteration 2 ...` → 5. `agent-skill-eval compare --workspace ... 1 2`
 
 ## Environment variables
 
 - `OPENROUTER_API_KEY` / `OPENAI_API_KEY`: API key for LLM rubric grading
 - `OPENAI_BASE_URL`: custom grader endpoint (defaults to OpenRouter)
-- `SKILL_EVAL_AGENT_TIMEOUT` / `SKILL_EVAL_AGENT_RETRIES`: harness timeout/retry defaults
-- `SKILL_EVAL_KEEP_WORKSPACE`: keep per-eval workspaces for debugging
+- `ASE_AGENT_TIMEOUT` / `ASE_AGENT_RETRIES`: harness timeout/retry defaults
+- `ASE_KEEP_WORKSPACE`: keep per-eval workspaces for debugging
 
 ## Agent-specific details