@fede0089/skill-eval 3.0.2 → 3.2.0

package/README.md

@@ -6,7 +6,7 @@ A CLI tool for evaluating Agent Skills locally. Tests whether your skill trigger
 
 skill-eval ships two commands, each targeting a different failure mode:
 
-- **Triggering** (`skill-eval trigger`) — checks whether the agent actually decides to invoke the skill in the right context. A skill that never gets triggered cannot help, no matter how good its instructions are.
+- **Triggering** (`skill-eval trigger`) — checks whether the agent actually decides to invoke the skill in the right context, and leaves it alone in the wrong one. A skill that never gets triggered cannot help, no matter how good its instructions are; one that triggers everywhere gets in the way.
 - **Functional correctness** (`skill-eval functional`) — checks whether the actions the agent takes while the skill is active match your expectations. An LLM judge grades each transcript against the expectation list you provide.
 
 ## Why run skill evals
@@ -38,7 +38,7 @@ For each eval prompt, skill-eval spins up parallel agent processes with the curr
                      pass@k
 ```
 
-> The `trigger` command only runs with-skill trials and checks whether the skill dispatch tool was actually invoked — no judge or baseline needed.
+> The `trigger` command only runs with-skill trials and checks whether the skill dispatch tool was actually invoked — no judge or baseline needed. Evals marked `should_trigger: false` assert the opposite: that it was *not* invoked.
 >
 > The baseline branch is opt-in: enable it with `--compare-baseline` (no-skill control) or `--compare-ref <ref>` (historical skill versions).
 
@@ -89,7 +89,8 @@ skill-eval functional --workspace <path> --skill <path> [options] [agent]
 | `--trials <number>` | no | `3` | Trials per task (for pass@k) |
 | `--timeout <seconds>` | no | none | Kill the agent after this many seconds |
 | `--eval-id <id>` | no | all | Run only the eval with this numeric ID |
-| `--compare-ref [refs...]` | no | — | Git references to compare against |
+| `--eval-file <name>` | no | all | Run only the evals from this file in `evals/` (e.g. `edge-cases.json`) |
+| `--compare-ref [refs...]` | no | — | Git references to compare against (variadic — put `[agent]` before it, not after) |
 | `--compare-baseline` | no | `false` | Also run the no-skill baseline alongside the skill |
 | `-v, --debug` | no | `false` | Enable verbose debug logging |
 | `[agent]` | no | `gemini-cli` | Agent backend to use |
@@ -118,6 +119,8 @@ my-skill/
 
 All `.json` files in `evals/` are loaded and merged into a single suite — you can split them by feature or regression category.
 
+Use `--eval-file <name>` to run just one of them while iterating (the `.json` extension is optional), and combine it with `--eval-id <id>` to narrow down to a single eval inside that file.
+
 **Trigger eval** — `id` must be a unique integer across all eval files:
 ```json
 {
@@ -128,6 +131,22 @@ All `.json` files in `evals/` are loaded and merged into a single suite — you
 }
 ```
 
+**Negative trigger eval** — add `should_trigger: false` to assert the skill must *not* fire:
+```json
+{
+  "skill_name": "my-skill",
+  "evals": [
+    { "id": 2, "prompt": "Something my skill has no business handling", "should_trigger": false }
+  ]
+}
+```
+
+Positive evals measure **under-triggering** (the skill never fires when it should). Negative evals measure the opposite failure, **over-triggering** — the one you introduce when you widen a skill's `description` to catch more cases and silently lose precision. Both kinds live in the same suite and feed a single success rate, so a change that improves one at the expense of the other shows up immediately.
+
+A negative eval fails if the agent *attempts* to activate the skill at all, even if the activation itself errors out. These evals are trigger-only: `skill-eval functional` skips them, since that command instructs the agent to use the skill.
+
+> Trigger detection for Codex is heuristic (it infers activations from the event stream), so negative evals are most reliable on `claude-code` and `gemini-cli`.
+
 **Functional eval** — add `expectations` for the LLM judge to evaluate:
 ```json
 {
@@ -224,7 +243,7 @@ Without `--debug` these files are not written, so reach for the flag when you ne
 
 ## Try it out
 
-This repo includes a `mock-skill/` directory — a complete, working example of a license-generator skill with trigger and functional evals. Run it directly with:
+This repo includes a `mock-skill/` directory — a complete, working example of a license-generator skill with positive trigger, negative trigger, and functional evals. Run it directly with:
 
 ```sh
 npm run test:unit        # run the unit test suite
@@ -234,6 +253,9 @@ npm run test:trigger -- codex            # run trigger evals with Codex
 npm run test:functional -- codex         # run functional evals with Codex
 npm run test:trigger -- claude-code      # run trigger evals with Claude Code
 npm run test:functional -- claude-code   # run functional evals with Claude Code
+
+npm run test:trigger -- --eval-file negative-triggers.json   # only the negative-trigger evals
+npm run test:trigger -- --eval-file edge-cases --eval-id 3   # a single eval inside one file
 ```
 
 ## Extending

package/dist/commands/functional.js

@@ -8,15 +8,29 @@ import { EvalRunner } from '../core/eval-runner.js';
 import { AgentPool } from '../core/agent-pool.js';
 import { aggregatePassAtK, aggregateAssertionPassRate, aggregateTokenStats, aggregateDurationStats } from '../core/statistics.js';
 import { preflight } from '../core/preflight.js';
+import { ConfigError } from '../core/errors.js';
 import { withRetry } from '../core/trial-utils.js';
 import { renderFunctionalTable, renderRunHeader } from '../utils/table-renderer.js';
 import { JsonReporter } from '../reporters/index.js';
 import chalk from 'chalk';
 import { git } from '../utils/git.js';
-export async function functionalCommand(agent, workspace, skillPath, maxAgents = 4, injectedSuite, numTrials = 3, reporter = new JsonReporter(), timeoutMs, evalId, compareRefs = [], compareBaseline = false) {
+export async function functionalCommand(agent, workspace, skillPath, maxAgents = 4, injectedSuite, numTrials = 3, reporter = new JsonReporter(), timeoutMs, evalId, compareRefs = [], compareBaseline = false, evalFile) {
     if (!injectedSuite)
         preflight(agent, workspace, skillPath);
-    const suite = injectedSuite || evalLoader.loadEvalSuite(skillPath);
+    const suite = injectedSuite || evalLoader.loadEvalSuite(skillPath, evalFile);
+    // Negative evals (should_trigger: false) only make sense for the trigger command:
+    // this pass instructs the agent that it MUST use the skill, which contradicts them.
+    const triggerOnly = suite.tasks.filter(t => t.should_trigger === false);
+    if (triggerOnly.length > 0) {
+        if (evalId !== undefined && triggerOnly.some(t => t.id === evalId)) {
+            throw new ConfigError(`Eval #${evalId} is trigger-only (should_trigger: false) and cannot run under 'functional'.`);
+        }
+        suite.tasks = suite.tasks.filter(t => t.should_trigger !== false);
+        if (suite.tasks.length === 0) {
+            throw new ConfigError(`No evals left to run: all ${triggerOnly.length} eval(s) in scope are trigger-only (should_trigger: false).`);
+        }
+        Logger.write(chalk.dim(`   Skipping ${triggerOnly.length} trigger-only eval(s)\n`));
+    }
     if (evalId !== undefined) {
         suite.tasks = suite.tasks.filter(t => t.id === evalId);
         if (suite.tasks.length === 0) {
@@ -83,7 +97,7 @@ export async function functionalCommand(agent, workspace, skillPath, maxAgents =
         ]
         : skillVersions.flatMap(v => Array.from({ length: numTrials }, (_, i) => `${v} ${i + 1}`));
     try {
-        renderRunHeader({ command: 'functional', skillName: skill_name, agent, workspace, tasks: tasks.length, trials: numTrials, maxAgents, timeoutMs, runDir, evalId });
+        renderRunHeader({ command: 'functional', skillName: skill_name, agent, workspace, tasks: tasks.length, trials: numTrials, maxAgents, timeoutMs, runDir, evalId, evalFile });
         Logger.write(`──────────────────────────────────────────────────\n`);
         for (let i = 0; i < tasks.length; i++) {
             const task = tasks[i];

package/dist/commands/trigger.js

@@ -13,10 +13,10 @@ import { renderTriggerTable, renderRunHeader } from '../utils/table-renderer.js'
 import { JsonReporter } from '../reporters/index.js';
 import chalk from 'chalk';
 import { git } from '../utils/git.js';
-export async function triggerCommand(agent, workspace, skillPath, maxAgents = 4, injectedSuite, numTrials = 3, reporter = new JsonReporter(), timeoutMs, evalId, compareRefs = []) {
+export async function triggerCommand(agent, workspace, skillPath, maxAgents = 4, injectedSuite, numTrials = 3, reporter = new JsonReporter(), timeoutMs, evalId, compareRefs = [], evalFile) {
     if (!injectedSuite)
         preflight(agent, workspace, skillPath);
-    const suite = injectedSuite || evalLoader.loadEvalSuite(skillPath);
+    const suite = injectedSuite || evalLoader.loadEvalSuite(skillPath, evalFile);
     if (evalId !== undefined) {
         suite.tasks = suite.tasks.filter(t => t.id === evalId);
         if (suite.tasks.length === 0) {
@@ -73,7 +73,7 @@ export async function triggerCommand(agent, workspace, skillPath, maxAgents = 4,
     const skillVersions = Array.from(variantRunners.keys());
     const subtaskLabels = skillVersions.flatMap(v => Array.from({ length: numTrials }, (_, i) => `${v} ${i + 1}`));
     try {
-        renderRunHeader({ command: 'trigger', skillName: skill_name, agent, workspace, tasks: tasks.length, trials: numTrials, maxAgents, timeoutMs, runDir, evalId });
+        renderRunHeader({ command: 'trigger', skillName: skill_name, agent, workspace, tasks: tasks.length, trials: numTrials, maxAgents, timeoutMs, runDir, evalId, evalFile });
         Logger.write(`--- Trigger Pass ---\n`);
         Logger.write(`──────────────────────────────────────────────────\n`);
         for (let i = 0; i < tasks.length; i++) {
@@ -149,7 +149,8 @@ export async function triggerCommand(agent, workspace, skillPath, maxAgents = 4,
                         taskId: task.id,
                         prompt: task.prompt,
                         baselineTrials: [],
-                        skillTrials: taskSkillTrials
+                        skillTrials: taskSkillTrials,
+                        shouldTrigger: task.should_trigger !== false
                     };
                     taskResults.push(taskResult);
                     if (!localAllPassed && !multi) {

package/dist/core/eval-runner.js

@@ -65,22 +65,59 @@ export class EvalRunner {
         const tokenStats = transcript
             ? parseTokenStats(transcript.response || '') ?? undefined
             : undefined;
-        let triggered = false;
+        // Negative evals (should_trigger: false) assert the opposite: the skill must NOT activate.
+        const shouldTrigger = task.should_trigger !== false;
+        const assertionLabel = shouldTrigger ? 'Skill was triggered' : 'Skill was not triggered';
+        let trialPassed = false;
         const assertionResults = [];
         if (transcript && !transcript.error) {
             uiCtx.updateLog('Grading…');
-            triggered = this.triggerGrader.gradeTrigger(transcript);
-            assertionResults.push({
-                assertion: 'Skill was triggered',
-                passed: triggered,
-                reason: triggered ? 'Detected skill activation in transcript' : 'No skill activation detected in transcript',
-                graderType: 'programmatic'
-            });
+            if (shouldTrigger) {
+                const triggered = this.triggerGrader.gradeTrigger(transcript);
+                trialPassed = triggered;
+                assertionResults.push({
+                    assertion: assertionLabel,
+                    passed: triggered,
+                    reason: triggered ? 'Detected skill activation in transcript' : 'No skill activation detected in transcript',
+                    graderType: 'programmatic'
+                });
+            }
+            else if (!this.triggerGrader.hasParsableEvents(transcript)) {
+                // Without any parsable events the absence of an activation proves nothing —
+                // report it as an infrastructure error instead of a vacuous pass.
+                const reason = 'No parsable agent events — cannot verify that the skill did not trigger';
+                assertionResults.push({
+                    assertion: assertionLabel,
+                    passed: false,
+                    reason,
+                    graderType: 'programmatic'
+                });
+                return {
+                    id: trialId,
+                    transcript,
+                    assertionResults,
+                    trialPassed: false,
+                    isError: true,
+                    tokenStats,
+                    durationMs
+                };
+            }
+            else {
+                // Any activation attempt counts, even one that failed: the skill still fired.
+                const triggered = this.triggerGrader.detectSkillAttempt(transcript);
+                trialPassed = !triggered;
+                assertionResults.push({
+                    assertion: assertionLabel,
+                    passed: !triggered,
+                    reason: triggered ? 'Skill activation detected in transcript' : 'No skill activation detected in transcript',
+                    graderType: 'programmatic'
+                });
+            }
         }
         else {
             const errorMsg = transcript?.error || 'Error: No transcript was produced';
             assertionResults.push({
-                assertion: 'Skill was triggered',
+                assertion: assertionLabel,
                 passed: false,
                 reason: errorMsg,
                 graderType: 'programmatic'
@@ -99,7 +136,7 @@ export class EvalRunner {
             id: trialId,
             transcript: transcript || { error: 'No transcript produced' },
             assertionResults: assertionResults,
-            trialPassed: triggered,
+            trialPassed,
             tokenStats,
             durationMs
         };

package/dist/core/evaluator.js

@@ -89,10 +89,19 @@ export class TriggerGrader {
         }
         return false;
     }
+    /**
+     * Whether the transcript produced any parsable NDJSON events.
+     * Negative trigger evals rely on the absence of an activation event, so a transcript
+     * with no events at all cannot prove anything and must not be graded as a pass.
+     */
+    hasParsableEvents(transcript) {
+        return parseNdjsonEvents(transcript.raw_output || '').length > 0;
+    }
     /**
      * Checks whether the skill was attempted (any tool_use event for activate_skill matching the skill name).
      * Unlike gradeTrigger, does NOT require a successful tool_result — any attempt counts.
-     * Used to detect invalid baseline runs where the agent tried to invoke the restricted skill.
+     * Used to detect invalid baseline runs where the agent tried to invoke the restricted skill,
+     * and to grade negative trigger evals (an attempted activation still counts as a trigger).
      */
     detectSkillAttempt(transcript) {
         const rawOutput = transcript.raw_output || '';

package/dist/index.js

@@ -3,7 +3,7 @@ import { Command } from 'commander';
 import { triggerCommand } from './commands/trigger.js';
 import { functionalCommand } from './commands/functional.js';
 import { Logger } from './utils/logger.js';
-import { AppError } from './core/errors.js';
+import { AppError, ConfigError } from './core/errors.js';
 import { HtmlReporter } from './reporters/index.js';
 import { DEFAULT_AGENT } from './runners/registry.js';
 import * as path from 'path';
@@ -25,6 +25,18 @@ const errorHandler = (err) => {
     }
     process.exit(1);
 };
+/**
+ * A variadic option used without values (`--compare-ref` on its own) is parsed as
+ * `true` by commander; reject it instead of iterating over a boolean.
+ */
+const parseCompareRefs = (value) => {
+    if (value === undefined)
+        return [];
+    if (Array.isArray(value))
+        return value;
+    errorHandler(new ConfigError('--compare-ref requires at least one git reference (e.g. --compare-ref HEAD~1).'));
+    return [];
+};
 program
     .name('skill-eval')
     .description('CLI to evaluate agent skills triggering and functionality')
@@ -42,6 +54,7 @@ program
     .option('--trials <number>', 'Number of trials per task for pass@k calculation', '3')
     .option('--timeout <seconds>', 'Agent timeout in seconds')
     .option('--eval-id <id>', 'Run only the eval with this ID (numeric)')
+    .option('--eval-file <name>', 'Run only the evals from this file in evals/ (e.g. edge-cases.json)')
     .option('--compare-ref [refs...]', 'Compare against historical git references')
     .action((agent, options) => {
     const workspace = path.resolve(options.workspace);
@@ -50,8 +63,8 @@ program
     const numTrials = parseInt(options.trials, 10);
     const timeoutMs = options.timeout ? parseInt(options.timeout, 10) * 1000 : undefined;
     const evalId = options.evalId !== undefined ? parseInt(options.evalId, 10) : undefined;
-    const compareRefs = options.compareRef || [];
-    triggerCommand(selectedAgent, workspace, options.skill, maxAgents, undefined, numTrials, new HtmlReporter(), timeoutMs, evalId, compareRefs).catch(errorHandler);
+    const compareRefs = parseCompareRefs(options.compareRef);
+    triggerCommand(selectedAgent, workspace, options.skill, maxAgents, undefined, numTrials, new HtmlReporter(), timeoutMs, evalId, compareRefs, options.evalFile).catch(errorHandler);
 });
 program
     .command('functional [agent]')
@@ -62,6 +75,7 @@ program
     .option('--trials <number>', 'Number of trials per task for pass@k calculation', '3')
     .option('--timeout <seconds>', 'Agent timeout in seconds')
     .option('--eval-id <id>', 'Run only the eval with this ID (numeric)')
+    .option('--eval-file <name>', 'Run only the evals from this file in evals/ (e.g. edge-cases.json)')
     .option('--compare-ref [refs...]', 'Compare against historical git references')
     .option('--compare-baseline', 'Also run the no-skill baseline alongside the skill')
     .action((agent, options) => {
@@ -71,9 +85,9 @@ program
     const numTrials = parseInt(options.trials, 10);
     const timeoutMs = options.timeout ? parseInt(options.timeout, 10) * 1000 : undefined;
     const evalId = options.evalId !== undefined ? parseInt(options.evalId, 10) : undefined;
-    const compareRefs = options.compareRef || [];
+    const compareRefs = parseCompareRefs(options.compareRef);
     const compareBaseline = !!options.compareBaseline;
-    functionalCommand(selectedAgent, workspace, options.skill, maxAgents, undefined, numTrials, new HtmlReporter(), timeoutMs, evalId, compareRefs, compareBaseline).catch(errorHandler);
+    functionalCommand(selectedAgent, workspace, options.skill, maxAgents, undefined, numTrials, new HtmlReporter(), timeoutMs, evalId, compareRefs, compareBaseline, options.evalFile).catch(errorHandler);
 });
 const isMain = process.argv[1] && (() => {
     try {

package/dist/reporters/html-reporter.js

@@ -215,9 +215,11 @@ function renderTaskTable(report) {
     const headerRow = `<tr><th>#</th><th>Prompt</th><th>Details</th></tr>`;
     const rows = results.map(result => {
         const prompt = escapeHtml(result.prompt);
+        // Negative trigger evals assert the opposite outcome — flag them so the row is not misread.
+        const badge = result.shouldTrigger === false ? '<span class="badge-neg">no-trigger</span>' : '';
         const detailsBtn = `<button class="details-btn" data-target="details-${result.taskId}">▶</button>`;
         const detailsRow = `<tr class="details-row"><td colspan="3">${renderTaskDetails(result, functional)}</td></tr>`;
-        return `<tr><td>${result.taskId}</td><td class="prompt-cell">${prompt}</td><td>${detailsBtn}</td></tr>${detailsRow}`;
+        return `<tr><td>${result.taskId}</td><td class="prompt-cell">${badge}${prompt}</td><td>${detailsBtn}</td></tr>${detailsRow}`;
     }).join('');
     return `<div class="table-wrap"><table><thead>${headerRow}</thead><tbody>${rows}</tbody></table></div>`;
 }
@@ -292,6 +294,7 @@ th { background: #f8fafc; font-size: 11px; font-weight: 600; text-transform: upp
 td { padding: 10px 12px; border-bottom: 1px solid #f1f5f9; vertical-align: top; }
 tr:last-child td { border-bottom: none; }
 .prompt-cell { max-width: 480px; word-break: break-word; color: #334155; }
+.badge-neg { display: inline-block; margin-right: 8px; padding: 1px 7px; border-radius: 10px; background: #e0f2fe; color: #0369a1; font-size: 11px; font-weight: 600; vertical-align: 1px; }
 
 /* Details */
 .details-btn { background: none; border: 1px solid #e2e8f0; border-radius: 4px; cursor: pointer; padding: 2px 8px; font-size: 11px; color: #64748b; transition: background 0.15s; }

package/dist/runners/codex/runner.js

@@ -57,6 +57,10 @@ function eventSignalsSkill(event, skillName) {
     const eventType = String(event.type ?? '').toLowerCase();
     const item = getEventItem(event);
     const itemType = String(item?.type ?? '').toLowerCase();
+    // Assistant prose is not a tool call: an agent merely naming the skill (often to say it is
+    // not needed) must not be reported as an activation. Real activations arrive as skill items.
+    if (['agent_message', 'assistant_message', 'message'].includes(itemType))
+        return false;
     return (serialized.includes(normalizedSkill) &&
         (serialized.includes('skill') ||
             eventType.includes('skill') ||

package/dist/utils/eval-loader.js

@@ -1,20 +1,36 @@
 import * as fs from 'fs';
 import * as path from 'path';
 import { ConfigError } from '../core/errors.js';
+/**
+ * Normalizes an --eval-file value to the bare file name used inside the evals directory.
+ * Accepts 'edge-cases', 'edge-cases.json' and 'path/to/evals/edge-cases.json' alike.
+ */
+function normalizeEvalFileName(evalFile) {
+    const base = path.basename(evalFile.trim());
+    return base.endsWith('.json') ? base : `${base}.json`;
+}
 /**
  * Loads and merges all JSON evaluation files from a skill's evals directory.
  * Aligned with Anthropic's recommendation to split evals by capability/regression.
  * Supports legacy 'tasks' and 'assertions' internally while maintaining 'evals' and 'expectations' in files.
+ * When 'evalFile' is given, only that file is loaded instead of the whole suite.
  */
-export function loadEvalSuite(skillPath) {
+export function loadEvalSuite(skillPath, evalFile) {
     const evalsDir = path.resolve(skillPath, 'evals');
     if (!fs.existsSync(evalsDir)) {
         throw new ConfigError(`Could not find evals directory at ${evalsDir}`);
     }
-    const files = fs.readdirSync(evalsDir).filter(file => file.endsWith('.json'));
+    let files = fs.readdirSync(evalsDir).filter(file => file.endsWith('.json'));
     if (files.length === 0) {
         throw new ConfigError(`No JSON evaluation files found in ${evalsDir}`);
     }
+    if (evalFile !== undefined) {
+        const wanted = normalizeEvalFileName(evalFile);
+        if (!files.includes(wanted)) {
+            throw new ConfigError(`Eval file '${wanted}' not found in ${evalsDir}. Available: ${[...files].sort().join(', ')}.`);
+        }
+        files = [wanted];
+    }
     let mergedSkillName = '';
     const mergedTasks = [];
     for (const file of files) {
@@ -44,18 +60,23 @@ export function loadEvalSuite(skillPath) {
             if (e.id === undefined || typeof e.id !== 'number') {
                 throw new ConfigError(`Invalid task ID in ${file}. ID must be a number.`);
             }
+            if (e.should_trigger !== undefined && typeof e.should_trigger !== 'boolean') {
+                throw new ConfigError(`Invalid 'should_trigger' for eval ${e.id} in ${file}. Must be a boolean.`);
+            }
             return {
                 id: e.id,
                 prompt: e.prompt,
                 expected_output: e.expected_output,
                 assertions: e.expectations || e.assertions,
-                files: e.files
+                files: e.files,
+                should_trigger: e.should_trigger
             };
         });
         mergedTasks.push(...mappedTasks);
     }
     if (mergedTasks.length === 0) {
-        throw new ConfigError(`No evaluations found in any of the JSON files in ${evalsDir}`);
+        const scope = evalFile !== undefined ? `${files[0]} in ${evalsDir}` : `any of the JSON files in ${evalsDir}`;
+        throw new ConfigError(`No evaluations found in ${scope}`);
     }
     return {
         skill_name: mergedSkillName,

package/dist/utils/table-renderer.js

@@ -98,7 +98,7 @@ function boxLabel(key, value) {
  * Always shown regardless of debug mode.
  */
 export function renderRunHeader(config) {
-    const { command, skillName, agent, workspace, tasks, trials, maxAgents, timeoutMs, runDir, evalId } = config;
+    const { command, skillName, agent, workspace, tasks, trials, maxAgents, timeoutMs, runDir, evalId, evalFile } = config;
     let timeoutStr = 'None';
     if (timeoutMs && timeoutMs > 0) {
         const timeoutSec = timeoutMs / 1000;
@@ -111,7 +111,12 @@ export function renderRunHeader(config) {
     const dashes = '─'.repeat(BOX_INNER - titleLabel.length);
     const top = chalk.gray('┌─ ') + chalk.bold(titleLabel) + ' ' + chalk.gray(dashes + '┐');
     const bottom = chalk.gray('└' + '─'.repeat(BOX_INNER + 2) + '┘');
-    const commandPart = evalId !== undefined ? `${command}  ·  eval #${evalId}` : command;
+    const filterParts = [];
+    if (evalFile !== undefined)
+        filterParts.push(evalFile);
+    if (evalId !== undefined)
+        filterParts.push(`eval #${evalId}`);
+    const commandPart = [command, ...filterParts].join('  ·  ');
     const title = chalk.bold.cyan(skillName) + chalk.gray(`  ·  ${commandPart}`);
     const runLine = `${tasks} task${tasks !== 1 ? 's' : ''}  ·  ${trials} trial${trials !== 1 ? 's' : ''}  ·  agents ${maxAgents}`;
     process.stdout.write('\n');
@@ -134,7 +139,11 @@ export function renderTriggerTable(report) {
     const numTrials = metrics.numTrials || 1;
     // Identify all skill versions present in the results
     const skillVersions = results.length > 0 ? Object.keys(results[0].skillTrials) : ['local'];
+    // Only surface the polarity column when the suite actually contains negative evals.
+    const hasNegativeEvals = results.some(r => r.shouldTrigger === false);
     const header = ['ID', 'Prompt'];
+    if (hasNegativeEvals)
+        header.push('Expect');
     if (numTrials > 1) {
         for (const version of skillVersions) {
             header.push(`${version} Trials`, `${version} Rate`);
@@ -150,6 +159,9 @@ export function renderTriggerTable(report) {
     for (const result of results) {
         const promptSnippet = result.prompt.substring(0, 40) + (result.prompt.length > 40 ? '...' : '');
         const row = [result.taskId.toString(), promptSnippet];
+        if (hasNegativeEvals) {
+            row.push(result.shouldTrigger === false ? chalk.cyan('no-trigger') : chalk.dim('trigger'));
+        }
         for (const version of skillVersions) {
             const trials = result.skillTrials[version] || [];
             const p1Cell = formatPassAt1(trials);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@fede0089/skill-eval",
-  "version": "3.0.2",
+  "version": "3.2.0",
   "description": "CLI to evaluate agent skills triggering and functionality",
   "main": "dist/index.js",
   "bin": {