@fede0089/skill-eval 1.0.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Federico Mete
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,170 @@
+# skill-eval
+
+A CLI tool for evaluating Agent Skills locally. Tests whether your skill triggers reliably and produces the right output, using an LLM as the judge.
+
+## Why skill evals?
+
+Skills are instructions that change how the agent behaves. But a single successful run isn't enough to trust one — agents are non-deterministic, and a good isolated result can be exactly that: an isolated case.
+
+Skill evals let you turn that intuition into evidence: run several comparable tasks with and without the skill, measure them against the same criteria, and validate whether the agent improves consistently against a baseline.
+
+## How it works
+
+For each eval prompt, skill-eval spins up parallel agent processes — some with the skill installed, others without (the baseline). Each agent runs headlessly and produces a transcript. An LLM judge then grades each transcript against your expectations. Results are aggregated into **pass@k** metrics, giving you a clear measure of how much your skill actually improves the agent's behavior versus the unassisted baseline.
+
+```
+                  eval prompt
+                       │
+               ┌───────▼───────┐
+               │   skill-eval  │
+               └───────┬───────┘
+                       │
+           ┌───────────┴───────────┐
+      ─ with skill ─          ─ baseline ─
+      ┌──────┴──────┐         ┌─────┴──────┐
+    agent 1      agent 2   agent 3      agent 4
+      │              │         │             │
+    judge          judge     judge         judge
+      └──────┬──────┘         └──────┬──────┘
+             └──────────┬────────────┘
+                        │
+                     pass@k
+```
+
+> The `trigger` command only runs with-skill trials and checks whether the skill dispatch tool was actually invoked — no judge or baseline needed.
+
+## Quick Start
+
+```sh
+git clone <this-repo>
+cd skill-eval
+npm install
+npm run build
+npm link
+```
+
+This makes `skill-eval` available globally in your terminal.
+
+**Requirements:** Node.js, and the agent CLI you want to evaluate (e.g. `gemini`) installed and on `$PATH`.
+
+## Commands
+
+```sh
+# Checks that the skill is triggered (invoked) for each prompt
+skill-eval trigger --workspace <path> --skill <path> [options] [agent]
+
+# Checks that the skill produces correct output, measured against a baseline
+skill-eval functional --workspace <path> --skill <path> [options] [agent]
+```
+
+### Options
+
+| Flag | Required | Default | Description |
+|------|----------|---------|-------------|
+| `--workspace <path>` | yes | — | Path to the repo the agent will run in |
+| `--skill <path>` | yes | — | Path to the skill directory |
+| `--agents <number>` | no | `4` | Number of parallel agent processes |
+| `--trials <number>` | no | `3` | Trials per task (for pass@k) |
+| `--timeout <seconds>` | no | none | Kill the agent after this many seconds |
+| `--eval-id <id>` | no | all | Run only the eval with this numeric ID |
+| `-v, --debug` | no | `false` | Enable verbose debug logging |
+| `[agent]` | no | `gemini-cli` | Agent backend to use |
+
+### Skill directory structure
+
+```
+my-skill/
+├── SKILL.md                        # skill definition (required)
+└── evals/                          # evaluation suite (required)
+    ├── my-evals.json               # one or more eval files (*.json)
+    └── config/                     # runner configuration (optional but often needed)
+        └── gemini-cli/             # runner-specific config folder
+            └── settings.json       # copied to <worktree>/.gemini/ before each trial
+```
+
+All `.json` files in `evals/` are loaded and merged into a single suite — you can split them by feature or regression category.
+
+**Trigger eval** — `id` must be a unique integer across all eval files:
+```json
+{
+  "skill_name": "my-skill",
+  "evals": [
+    { "id": 1, "prompt": "Do the thing that my skill handles" }
+  ]
+}
+```
+
+**Functional eval** — add `expectations` for the LLM judge to evaluate:
+```json
+{
+  "skill_name": "my-skill",
+  "evals": [
+    {
+      "id": 1,
+      "prompt": "Create a file called hello.txt containing the word 'world'",
+      "expectations": [
+        "A file named hello.txt was created",
+        "The file contains the text 'world'"
+      ]
+    }
+  ]
+}
+```
+
+## Permissions
+
+**This is the most common cause of eval failures.**
+
+skill-eval runs the agent headlessly — stdin is closed, there is no terminal. If the agent encounters a tool that requires interactive approval, it will either fail immediately or hang until the trial timeout kills it.
+
+The runner already uses `--approval-mode auto_edit`, which auto-approves standard file operations (create, edit, delete). But if your skill needs to run shell commands, read environment variables, make network calls, or use any other tool category — those still require explicit permission.
+
+**Solution:** place a config file inside your skill at `evals/config/<runner>/`. Before every trial, skill-eval automatically copies that directory into the agent's config location inside the isolated worktree:
+
+```
+evals/config/gemini-cli/  →  <worktree>/.gemini/
+```
+
+Use this to ship both settings and policies alongside your evals. For Gemini CLI, for example, you can use `settings.json` to configure tool permissions and approval policies so that every tool your skill relies on runs without prompting:
+
+```json
+{
+  "telemetry": { "enabled": false }
+}
+```
+
+Refer to your runner's documentation for the full list of available settings and policy keys.
+
+> This config only applies inside the temporary worktree created for each trial. Your real workspace config is never touched.
+
+## Try it out
+
+This repo includes a `mock-skill/` directory — a complete, working example of a license-generator skill with trigger and functional evals. Run it directly with:
+
+```sh
+npm run test:trigger     # trigger evaluation against mock-skill
+npm run test:functional  # functional evaluation against mock-skill
+```
+
+Results are saved to `.project-skill-evals/runs/<timestamp>/` with logs, raw eval JSONs, and an HTML report.
+
+## Extending
+
+### Adding a new agent runner
+
+1. Create `src/runners/<your-agent>/runner.ts` implementing the `AgentRunner` interface (see `src/runners/runner.interface.ts`).
+2. Export it from `src/runners/<your-agent>/index.ts`.
+3. Register it in `src/runners/registry.ts`:
+   ```ts
+   '<your-agent>': { Runner: YourRunner, binary: '<cli-binary-name>' },
+   ```
+
+The factory, preflight check, and CLI all pick it up automatically.
+
+> Implement `applyRunnerConfig(evalConfigBaseDir, worktreePath)` to copy `evalConfigBaseDir/<your-agent>/` into the appropriate config directory in the worktree (e.g. `.claude/` for a Claude runner). No-op silently if the directory doesn't exist.
+
+### Adding a new report format
+
+1. Create `src/reporters/<format>-reporter.ts` implementing `Reporter`.
+2. Export it and add a case in `createReporter()` in `src/reporters/index.ts`.
+3. Add the format string to `ReportFormat` in `src/types/index.ts`.

package/dist/commands/functional.js

@@ -0,0 +1,225 @@
+import * as fs from 'fs';
+import * as path from 'path';
+import { EvalEnvironment } from '../core/environment.js';
+import { Logger } from '../utils/logger.js';
+import * as evalLoader from '../utils/eval-loader.js';
+import { ListrEvalUI } from '../utils/ui.js';
+import { EvalRunner } from '../core/eval-runner.js';
+import { AgentPool } from '../core/agent-pool.js';
+import { aggregatePassAtK, aggregateAssertionPassRate, aggregateTokenStats, aggregateDurationStats } from '../core/statistics.js';
+import { preflight } from '../core/preflight.js';
+import { withRetry } from '../core/trial-utils.js';
+import { renderFunctionalTable, renderRunHeader } from '../utils/table-renderer.js';
+import { JsonReporter } from '../reporters/index.js';
+export async function functionalCommand(agent, workspace, skillPath, maxAgents = 4, injectedSuite, numTrials = 3, reporter = new JsonReporter(), timeoutMs, evalId) {
+    if (!injectedSuite)
+        preflight(agent, workspace, skillPath);
+    const suite = injectedSuite || evalLoader.loadEvalSuite(skillPath);
+    if (evalId !== undefined) {
+        suite.tasks = suite.tasks.filter(t => t.id === evalId);
+        if (suite.tasks.length === 0) {
+            console.error(`No eval found with id ${evalId}`);
+            process.exit(1);
+        }
+    }
+    const { skill_name, tasks } = suite;
+    const env = new EvalEnvironment({ workspace });
+    await env.setup();
+    // Ensure worktrees are cleaned up even when the process is interrupted (Ctrl+C).
+    const cleanup = () => { env.teardown().finally(() => process.exit(1)); };
+    process.once('SIGINT', cleanup);
+    process.once('SIGTERM', cleanup);
+    // Setup Artifacts Directory (Always create, even if not in debug mode, for 'show' command)
+    const debug = !!process.env.DEBUG;
+    const startTime = new Date();
+    const timestamp = startTime.toISOString().replace(/[:.]/g, '-');
+    const runDir = path.resolve(workspace, '.project-skill-evals', 'runs', timestamp);
+    fs.mkdirSync(runDir, { recursive: true });
+    const taskResults = [];
+    let withSkillTasksAllPassedCount = 0;
+    let baselineTasksAllPassedCount = 0;
+    // Per-task trial storage, indexed by task.id
+    const withoutSkillTrialsByTask = new Map();
+    const withoutSkillRunner = new EvalRunner({
+        agent,
+        workspace,
+        skillPath,
+        skillName: skill_name,
+        runDir,
+        isBaseline: true,
+        debug,
+        timeoutMs
+    });
+    const withSkillRunner = new EvalRunner({
+        agent,
+        workspace,
+        skillPath,
+        skillName: skill_name,
+        runDir,
+        isBaseline: false,
+        debug,
+        timeoutMs
+    });
+    const pool = new AgentPool(maxAgents);
+    const ui = new ListrEvalUI();
+    // Barrier chain: each prompt waits until the previous prompt's trials have all acquired slots.
+    let barrier = Promise.resolve();
+    // Subtask labels: Without Skill 1..N then With Skill 1..N for each prompt
+    const subtaskLabels = [
+        ...Array.from({ length: numTrials }, (_, i) => `Without Skill ${i + 1}`),
+        ...Array.from({ length: numTrials }, (_, i) => `With Skill ${i + 1}`)
+    ];
+    try {
+        renderRunHeader({ command: 'functional', skillName: skill_name, agent, workspace, tasks: tasks.length, trials: numTrials, maxAgents, timeoutMs, runDir, evalId });
+        Logger.write(`──────────────────────────────────────────────────\n`);
+        for (let i = 0; i < tasks.length; i++) {
+            const task = tasks[i];
+            const promptSnippet = `${task.prompt.substring(0, 50)}${task.prompt.length > 50 ? '...' : ''}`;
+            const taskLabel = `${promptSnippet} (#${task.id})`;
+            const thisBarrier = barrier;
+            let resolveBarrier;
+            barrier = new Promise(r => { resolveBarrier = r; });
+            ui.addTask({
+                id: task.id,
+                title: taskLabel,
+                subtaskLabels,
+                task: async (uiCtx, multi) => {
+                    await thisBarrier;
+                    // ── Without Skill trials (subtask IDs 1..numTrials) ──────────────
+                    const baselineTrialPromises = [];
+                    for (let idx = 0; idx < numTrials; idx++) {
+                        const trialId = idx + 1;
+                        const trialCtx = multi?.getTrialCtx(trialId) ?? uiCtx;
+                        const release = await pool.acquire();
+                        const p = withRetry((attempt) => withoutSkillRunner.runFunctionalTask(task, i, trialId, trialCtx, attempt)
+                            .catch((error) => ({
+                            id: trialId,
+                            transcript: { error: error instanceof Error ? error.message : String(error) },
+                            assertionResults: [{ assertion: 'Without Skill Execution', passed: false, reason: String(error) }],
+                            trialPassed: false,
+                            isError: true
+                        })), 2, 1000, (nextAttempt, lastTrial) => {
+                            const reason = lastTrial.assertionResults[0]?.reason ?? 'infrastructure error';
+                            trialCtx.updateLog(`Retry ${nextAttempt}/2 — ${reason.substring(0, 50)}`);
+                        }).then(trial => {
+                            if (multi) {
+                                const reason = trial.assertionResults.find(r => !r.passed)?.reason;
+                                const passedCount = trial.assertionResults.filter(r => r.passed).length;
+                                const totalCount = trial.assertionResults.length;
+                                multi.markTrialComplete(trialId, trial.trialPassed, reason, trial.isError, passedCount, totalCount);
+                            }
+                            return trial;
+                        }).finally(release);
+                        baselineTrialPromises.push(p);
+                    }
+                    // ── With Skill trials (subtask IDs numTrials+1..2*numTrials) ─────
+                    const withSkillTrialPromises = [];
+                    for (let idx = 0; idx < numTrials; idx++) {
+                        const subtaskId = numTrials + idx + 1; // Listr subtask position
+                        const runnerTrialId = idx + 1; // 1-based trial number within WI pass
+                        const trialCtx = multi?.getTrialCtx(subtaskId) ?? uiCtx;
+                        const release = await pool.acquire();
+                        const p = withRetry((attempt) => withSkillRunner.runFunctionalTask(task, i, runnerTrialId, trialCtx, attempt)
+                            .catch((error) => ({
+                            id: runnerTrialId,
+                            transcript: { error: error instanceof Error ? error.message : String(error) },
+                            assertionResults: [{ assertion: 'With Skill Execution', passed: false, reason: String(error) }],
+                            trialPassed: false,
+                            isError: true
+                        })), 2, 1000, (nextAttempt, lastTrial) => {
+                            const reason = lastTrial.assertionResults[0]?.reason ?? 'infrastructure error';
+                            trialCtx.updateLog(`Retry ${nextAttempt}/2 — ${reason.substring(0, 50)}`);
+                        }).then(trial => {
+                            if (multi) {
+                                const reason = trial.assertionResults.find(r => !r.passed)?.reason;
+                                const passedCount = trial.assertionResults.filter(r => r.passed).length;
+                                const totalCount = trial.assertionResults.length;
+                                multi.markTrialComplete(subtaskId, trial.trialPassed, reason, trial.isError, passedCount, totalCount);
+                            }
+                            return trial;
+                        }).finally(release);
+                        withSkillTrialPromises.push(p);
+                    }
+                    // All WO + WI slots for this prompt have been acquired — signal the next prompt.
+                    resolveBarrier();
+                    const [woTrials, wiTrials] = await Promise.all([
+                        Promise.all(baselineTrialPromises),
+                        Promise.all(withSkillTrialPromises)
+                    ]);
+                    withoutSkillTrialsByTask.set(task.id, woTrials);
+                    const woPassedCount = woTrials.filter(t => t.trialPassed).length;
+                    if (woPassedCount === woTrials.length)
+                        baselineTasksAllPassedCount++;
+                    const wiPassedCount = wiTrials.filter(t => t.trialPassed).length;
+                    const score = wiTrials.length > 0 ? wiPassedCount / wiTrials.length : 0;
+                    const withoutSkillTrials = withoutSkillTrialsByTask.get(task.id) ?? [];
+                    const taskResult = {
+                        taskId: task.id,
+                        prompt: task.prompt,
+                        score,
+                        trials: wiTrials.map(t => ({ ...t, transcript: undefined })),
+                        withoutSkillTrials: withoutSkillTrials.map(t => ({ ...t, transcript: undefined }))
+                    };
+                    taskResults.push(taskResult);
+                    if (wiPassedCount === wiTrials.length) {
+                        withSkillTasksAllPassedCount++;
+                    }
+                    else if (!multi) {
+                        const failureReason = wiTrials.find(t => !t.trialPassed)?.assertionResults.find(r => !r.passed)?.reason || 'With Skill failed';
+                        throw new Error(failureReason);
+                    }
+                }
+            });
+        }
+        await ui.run(tasks.length);
+        // ==== REPORTING ====
+        const { passAtK } = aggregatePassAtK(taskResults, numTrials, r => r.trials);
+        const { passAtK: withoutSkillPassAtK } = aggregatePassAtK(taskResults, numTrials, r => r.withoutSkillTrials ?? []);
+        const assertionPassRate = aggregateAssertionPassRate(taskResults, r => r.trials);
+        const withoutSkillAssertionPassRate = aggregateAssertionPassRate(taskResults, r => r.withoutSkillTrials ?? []);
+        const withSkillPercentage = Math.round(assertionPassRate * 100);
+        const withoutSkillPercentage = Math.round(withoutSkillAssertionPassRate * 100);
+        const skillUplift = withSkillPercentage - withoutSkillPercentage;
+        const withSkillTokenStats = aggregateTokenStats(taskResults.flatMap(r => r.trials)) ?? undefined;
+        const withoutSkillTokenStats = aggregateTokenStats(taskResults.flatMap(r => r.withoutSkillTrials ?? [])) ?? undefined;
+        const withSkillDurationStats = aggregateDurationStats(taskResults.flatMap(r => r.trials)) ?? undefined;
+        const withoutSkillDurationStats = aggregateDurationStats(taskResults.flatMap(r => r.withoutSkillTrials ?? [])) ?? undefined;
+        const report = {
+            timestamp: startTime.toISOString(),
+            skill_name,
+            agent,
+            metrics: {
+                withSkillScore: `${withSkillPercentage}%`,
+                withoutSkillScore: `${withoutSkillPercentage}%`,
+                skillUplift: `${skillUplift > 0 ? '+' : ''}${skillUplift}%`,
+                passedCount: withSkillTasksAllPassedCount,
+                totalCount: tasks.length,
+                numTrials,
+                passAtK: Math.round(passAtK * 1000) / 1000,
+                withoutSkillPassAtK: Math.round(withoutSkillPassAtK * 1000) / 1000,
+                assertionPassRate: Math.round(assertionPassRate * 1000) / 1000,
+                withoutSkillAssertionPassRate: Math.round(withoutSkillAssertionPassRate * 1000) / 1000,
+                tokenStats: (withSkillTokenStats || withoutSkillTokenStats)
+                    ? { withSkill: withSkillTokenStats, withoutSkill: withoutSkillTokenStats }
+                    : undefined,
+                durationStats: (withSkillDurationStats || withoutSkillDurationStats)
+                    ? { withSkill: withSkillDurationStats, withoutSkill: withoutSkillDurationStats }
+                    : undefined
+            },
+            results: taskResults
+        };
+        Logger.write(`\nEVALUATION SUMMARY\n`);
+        Logger.write(`──────────────────────────────────────────────────\n`);
+        renderFunctionalTable(report);
+        const upliftSign = skillUplift > 0 ? '+' : '';
+        Logger.write(`\n   Skill Uplift:            ${upliftSign}${skillUplift}%\n`);
+        Logger.write('\n');
+        new JsonReporter().generate(report, runDir);
+        reporter.generate(report, runDir);
+    }
+    finally {
+        process.off('SIGINT', cleanup);
+        process.off('SIGTERM', cleanup);
+        await env.teardown();
+    }
+}

package/dist/commands/rate.js

@@ -0,0 +1,20 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.rateCommand = rateCommand;
+const logger_1 = require("../utils/logger");
+const errors_1 = require("../core/errors");
+async function rateCommand(score, options) {
+    const rating = parseInt(score, 10);
+    if (isNaN(rating) || rating < 1 || rating > 5) {
+        throw new errors_1.ValidationError('Rating must be a number between 1 and 5.');
+    }
+    logger_1.Logger.info('\n==========================================');
+    logger_1.Logger.info('       EXPERIENCE RATING VIEW           ');
+    logger_1.Logger.info('==========================================');
+    logger_1.Logger.info(`Rating: ${'★'.repeat(rating)}${'☆'.repeat(5 - rating)}`);
+    if (options.comment) {
+        logger_1.Logger.info(`Comment: ${options.comment}`);
+    }
+    logger_1.Logger.info('==========================================');
+    logger_1.Logger.info('Thank you for your feedback!\n');
+}

package/dist/commands/show.js

@@ -0,0 +1,43 @@
+import fs from 'fs';
+import path from 'path';
+import chalk from 'chalk';
+import { Logger } from '../utils/logger.js';
+import { AppError } from '../core/errors.js';
+import { renderTriggerTable, renderFunctionalTable } from '../utils/table-renderer.js';
+import { JsonReporter } from '../core/reporters/index.js';
+export async function showCommand(workspace, reporter = new JsonReporter()) {
+    const runsDir = path.resolve(workspace, '.project-skill-evals', 'runs');
+    if (!fs.existsSync(runsDir)) {
+        throw new AppError('No evaluation runs found. Run an evaluation first.');
+    }
+    const runs = fs.readdirSync(runsDir)
+        .filter(dir => fs.statSync(path.join(runsDir, dir)).isDirectory())
+        .sort((a, b) => b.localeCompare(a)); // Sort descending to get latest
+    if (runs.length === 0) {
+        throw new AppError('No evaluation runs found. Run an evaluation first.');
+    }
+    const latestRun = runs[0];
+    const summaryPath = path.join(runsDir, latestRun, 'summary.json');
+    if (!fs.existsSync(summaryPath)) {
+        throw new AppError(`Summary file not found for the latest run: ${latestRun}`);
+    }
+    const report = JSON.parse(fs.readFileSync(summaryPath, 'utf-8'));
+    const { skill_name, agent, metrics, results, timestamp } = report;
+    const numTrials = metrics.numTrials || 1;
+    const isFunctional = metrics.withoutSkillScore !== undefined;
+    Logger.write(`\n${chalk.bold('LATEST EVALUATION RESULTS')}\n`);
+    Logger.write(`Timestamp:  ${new Date(timestamp).toLocaleString()}\n`);
+    Logger.write(`Skill:      ${skill_name}\n`);
+    Logger.write(`Agent:      ${agent}\n`);
+    Logger.write(`Type:       ${isFunctional ? 'Functional' : 'Trigger'}\n`);
+    Logger.write(`──────────────────────────────────────────────────\n`);
+    if (isFunctional) {
+        renderFunctionalTable(report);
+    }
+    else {
+        renderTriggerTable(report);
+    }
+    Logger.write('\n');
+    const runDir = path.join(runsDir, latestRun);
+    reporter.generate(report, runDir);
+}

package/dist/commands/trigger.js

@@ -0,0 +1,154 @@
+import * as fs from 'fs';
+import * as path from 'path';
+import { EvalEnvironment } from '../core/environment.js';
+import { Logger } from '../utils/logger.js';
+import * as evalLoader from '../utils/eval-loader.js';
+import { ListrEvalUI } from '../utils/ui.js';
+import { EvalRunner } from '../core/eval-runner.js';
+import { AgentPool } from '../core/agent-pool.js';
+import { aggregatePassAtK, aggregateTokenStats, aggregateDurationStats } from '../core/statistics.js';
+import { preflight } from '../core/preflight.js';
+import { withRetry } from '../core/trial-utils.js';
+import { renderTriggerTable, renderRunHeader } from '../utils/table-renderer.js';
+import { JsonReporter } from '../reporters/index.js';
+export async function triggerCommand(agent, workspace, skillPath, maxAgents = 4, injectedSuite, numTrials = 3, reporter = new JsonReporter(), timeoutMs, evalId) {
+    if (!injectedSuite)
+        preflight(agent, workspace, skillPath);
+    const suite = injectedSuite || evalLoader.loadEvalSuite(skillPath);
+    if (evalId !== undefined) {
+        suite.tasks = suite.tasks.filter(t => t.id === evalId);
+        if (suite.tasks.length === 0) {
+            console.error(`No eval found with id ${evalId}`);
+            process.exit(1);
+        }
+    }
+    const { skill_name, tasks } = suite;
+    // Setup Environment (global setup)
+    const env = new EvalEnvironment({ workspace });
+    await env.setup();
+    // Ensure worktrees are cleaned up even when the process is interrupted (Ctrl+C).
+    const cleanup = () => { env.teardown().finally(() => process.exit(1)); };
+    process.once('SIGINT', cleanup);
+    process.once('SIGTERM', cleanup);
+    // Setup Artifacts Directory (Always create, even if not in debug mode, for 'show' command)
+    const debug = !!process.env.DEBUG;
+    const startTime = new Date();
+    const timestamp = startTime.toISOString().replace(/[:.]/g, '-');
+    const runDir = path.resolve(workspace, '.project-skill-evals', 'runs', timestamp);
+    fs.mkdirSync(runDir, { recursive: true });
+    const taskResults = [];
+    let tasksPassedCount = 0;
+    const runner = new EvalRunner({
+        agent,
+        workspace,
+        skillPath,
+        skillName: skill_name,
+        runDir,
+        debug,
+        timeoutMs
+    });
+    const pool = new AgentPool(maxAgents);
+    const ui = new ListrEvalUI();
+    // Barrier chain: each prompt waits until the previous prompt's trials have all acquired slots.
+    let barrier = Promise.resolve();
+    try {
+        renderRunHeader({ command: 'trigger', skillName: skill_name, agent, workspace, tasks: tasks.length, trials: numTrials, maxAgents, timeoutMs, runDir, evalId });
+        Logger.write(`--- Trigger Pass ---\n`);
+        Logger.write(`──────────────────────────────────────────────────\n`);
+        for (let i = 0; i < tasks.length; i++) {
+            const task = tasks[i];
+            const promptSnippet = `${task.prompt.substring(0, 50)}${task.prompt.length > 50 ? '...' : ''}`;
+            const taskLabel = `${promptSnippet} (#${task.id})`;
+            const thisBarrier = barrier;
+            let resolveBarrier;
+            barrier = new Promise(r => { resolveBarrier = r; });
+            ui.addTask({
+                id: task.id,
+                title: taskLabel,
+                numTrials,
+                task: async (uiCtx, multi) => {
+                    await thisBarrier;
+                    const trialPromises = [];
+                    for (let idx = 0; idx < numTrials; idx++) {
+                        const trialId = idx + 1;
+                        const trialCtx = multi?.getTrialCtx(trialId) ?? uiCtx;
+                        const release = await pool.acquire();
+                        const p = withRetry((attempt) => runner.runTriggerTask(task, i, trialId, trialCtx, attempt)
+                            .catch((error) => ({
+                            id: trialId,
+                            transcript: { error: error instanceof Error ? error.message : String(error) },
+                            assertionResults: [{
+                                    assertion: 'Runner Execution',
+                                    passed: false,
+                                    reason: error instanceof Error ? error.message : String(error)
+                                }],
+                            trialPassed: false,
+                            isError: true
+                        })), 2, 1000, (nextAttempt, lastTrial) => {
+                            const reason = lastTrial.assertionResults[0]?.reason ?? 'infrastructure error';
+                            trialCtx.updateLog(`Retry ${nextAttempt}/2 — ${reason.substring(0, 50)}`);
+                        }).then(trial => {
+                            if (multi) {
+                                const reason = trial.assertionResults.find(r => !r.passed)?.reason;
+                                multi.markTrialComplete(trialId, trial.trialPassed, reason, trial.isError);
+                            }
+                            return trial;
+                        }).finally(release);
+                        trialPromises.push(p);
+                    }
+                    // All trials for this prompt have acquired a slot — signal the next prompt.
+                    resolveBarrier();
+                    const trials = await Promise.all(trialPromises);
+                    const passedCount = trials.filter(t => t.trialPassed).length;
+                    const score = trials.length > 0 ? passedCount / trials.length : 0;
+                    const taskResult = {
+                        taskId: task.id,
+                        prompt: task.prompt,
+                        score,
+                        trials: trials.map(t => ({ ...t, transcript: undefined }))
+                    };
+                    taskResults.push(taskResult);
+                    if (passedCount === trials.length) {
+                        tasksPassedCount++;
+                    }
+                    else if (!multi) {
+                        const failureReason = trials.find(t => !t.trialPassed)?.assertionResults.find(r => !r.passed)?.reason || 'Task failed evaluation';
+                        throw new Error(failureReason);
+                    }
+                }
+            });
+        }
+        await ui.run(tasks.length);
+        // Compute aggregate metrics and build report
+        const { passAtK } = aggregatePassAtK(taskResults, numTrials, r => r.trials);
+        const percentage = Math.round(passAtK * 100);
+        const withSkillTokenStats = aggregateTokenStats(taskResults.flatMap(r => r.trials)) ?? undefined;
+        const withSkillDurationStats = aggregateDurationStats(taskResults.flatMap(r => r.trials)) ?? undefined;
+        const report = {
+            timestamp: startTime.toISOString(),
+            skill_name,
+            agent,
+            metrics: {
+                withSkillScore: `${percentage}%`,
+                passedCount: tasksPassedCount,
+                totalCount: tasks.length,
+                numTrials,
+                passAtK: Math.round(passAtK * 1000) / 1000,
+                tokenStats: withSkillTokenStats ? { withSkill: withSkillTokenStats } : undefined,
+                durationStats: withSkillDurationStats ? { withSkill: withSkillDurationStats } : undefined
+            },
+            results: taskResults
+        };
+        Logger.write(`\nEVALUATION SUMMARY\n`);
+        Logger.write(`──────────────────────────────────────────────────\n`);
+        renderTriggerTable(report);
+        Logger.write('\n\n');
+        new JsonReporter().generate(report, runDir);
+        reporter.generate(report, runDir);
+    }
+    finally {
+        process.off('SIGINT', cleanup);
+        process.off('SIGTERM', cleanup);
+        await env.teardown();
+    }
+}

package/dist/commands/view.js

@@ -0,0 +1,30 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.viewCommand = viewCommand;
+const logger_1 = require("../utils/logger");
+async function viewCommand(options) {
+    const ease = parseInt(options.ease || '0', 10);
+    const speed = parseInt(options.speed || '0', 10);
+    const accuracy = parseInt(options.accuracy || '0', 10);
+    const drawStars = (score) => {
+        const stars = '★'.repeat(score);
+        const empty = '☆'.repeat(5 - score);
+        return `[ ${stars}${empty} ]`;
+    };
+    logger_1.Logger.info('\n┌──────────────────────────────────────────┐');
+    logger_1.Logger.info('│          NUEVA VISTA DE CALIFICACIÓN     │');
+    logger_1.Logger.info('├──────────────────────────────────────────┤');
+    logger_1.Logger.info(`│ Facilidad de uso:   ${drawStars(ease)}     │`);
+    logger_1.Logger.info(`│ Velocidad:          ${drawStars(speed)}     │`);
+    logger_1.Logger.info(`│ Precisión:          ${drawStars(accuracy)}     │`);
+    logger_1.Logger.info('├──────────────────────────────────────────┤');
+    if (options.comment) {
+        logger_1.Logger.info(`│ Comentario:                              │`);
+        // Basic wrapping for the comment
+        const comment = options.comment.substring(0, 38);
+        logger_1.Logger.info(`│ ${comment.padEnd(40)} │`);
+    }
+    logger_1.Logger.info('├──────────────────────────────────────────┤');
+    logger_1.Logger.info('│ ¡Gracias por calificar tu experiencia!   │');
+    logger_1.Logger.info('└──────────────────────────────────────────┘\n');
+}