selftune 0.1.4 → 0.2.1

package/skill/Workflows/Evolve.md

@@ -4,6 +4,14 @@ Improve a skill's description based on real usage signal. Analyzes failure
 patterns from eval sets and proposes description changes that catch more
 natural-language queries without breaking existing triggers.
 
+## When to Invoke
+
+Invoke this workflow when the user requests any of the following:
+- Improving or evolving a skill's trigger coverage
+- Fixing undertriggering or missed queries for a skill
+- Optimizing a skill description based on usage data
+- Any request containing "evolve", "improve triggers", or "fix skill matching"
+
 ## Default Command
 
 ```bash
@@ -19,8 +27,14 @@ selftune evolve --skill <name> --skill-path <path> [options]
 | `--eval-set <path>` | Pre-built eval set JSON | Auto-generated from logs |
 | `--agent <name>` | Agent CLI to use (claude, codex, opencode) | Auto-detected |
 | `--dry-run` | Propose and validate without deploying | Off |
-| `--confidence <n>` | Minimum confidence threshold (0-1) | 0.7 |
+| `--confidence <n>` | Minimum confidence threshold (0-1) | 0.6 |
 | `--max-iterations <n>` | Maximum retry iterations | 3 |
+| `--validation-model <model>` | Model for trigger-check validation LLM calls | `haiku` |
+| `--cheap-loop` | Use cheap models for loop, expensive for final gate | Off |
+| `--gate-model <model>` | Model for final gate validation | `sonnet` (when `--cheap-loop`) |
+| `--proposal-model <model>` | Model for proposal generation LLM calls | None |
+| `--sync-first` | Refresh source-truth telemetry before generating evals/failure patterns | Off |
+| `--sync-force` | Force a full source rescan during `--sync-first` | Off |
 
 ## Output Format
 
@@ -73,7 +87,96 @@ The evolution process writes multiple audit entries:
 
 ## Steps
 
-### 1. Load or Generate Eval Set
+### 0. Pre-Flight Configuration
+
+Before running the evolve command, present numbered configuration options to the user inline in your response, then wait for the user's answer before proceeding.
+
+If the user responds with "use defaults", "just run it", or similar shorthand, skip to step 1 using the recommended defaults marked below.
+
+Present the following options inline in your response:
+
+1. **Execution Mode**
+   - a) Dry run — preview proposal without deploying (recommended for first run)
+   - b) Live — validate and deploy if improved
+
+2. **Model Tier** (see SKILL.md Model Tier Reference)
+   - a) Fast (haiku) — cheapest, ~2s/call (recommended with cheap-loop)
+   - b) Balanced (sonnet) — good quality, ~5s/call
+   - c) Best (opus) — highest quality, ~10s/call
+
+3. **Cost Optimization**
+   - a) Cheap loop — haiku for iteration, sonnet for final gate (recommended)
+   - b) Single model — use one model throughout
+
+4. **Confidence Threshold:** 0.6 (default, higher = stricter)
+
+5. **Max Iterations:** 3 (default, more = longer but better results)
+
+6. **Multi-Candidate Selection**
+   - a) Single candidate — one proposal per iteration (recommended)
+   - b) Pareto mode — generate multiple candidates, pick best on frontier
+
+Ask: "Reply with your choices (e.g., '1a, 2a, 3a, defaults for rest') or 'use defaults' for recommended settings."
+
+After the user responds, parse their selections and map each choice to the corresponding CLI flags:
+
+| Selection | CLI Flag |
+|-----------|----------|
+| 1a (dry run) | `--dry-run` |
+| 1b (live) | _(no flag)_ |
+| 2a (haiku) | `--validation-model haiku` |
+| 2b (sonnet) | `--validation-model sonnet` |
+| 2c (opus) | `--validation-model opus` |
+| 3a (cheap loop) | `--cheap-loop` |
+| 3b (single model) | _(no flag)_ |
+| Custom confidence | `--confidence <value>` |
+| Custom iterations | `--max-iterations <value>` |
+| 6b (pareto) | `--pareto` |
+
+Show a confirmation summary to the user:
+
+```
+Configuration Summary:
+  Mode:        dry-run
+  Model:       haiku (cheap-loop: sonnet gate)
+  Confidence:  0.6
+  Iterations:  3
+  Pareto:      off
+
+Proceeding...
+```
+
+Build the CLI command string with all selected flags and continue to step 1.
+
+### 1. Read Evolution Context
+
+Before running, read `~/.selftune/memory/context.md` for session context:
+- Active evolutions and their current status
+- Known issues from previous runs
+- Last update timestamp
+
+This provides continuity across context resets. If the file doesn't exist,
+proceed normally -- it will be created after the first evolution.
+
+The evolution-guard hook (`hooks/evolution-guard.ts`) blocks direct SKILL.md
+edits on monitored skills during active evolution. This prevents conflicting
+changes while the evolve process is running. The guard is automatically
+engaged when evolve starts and released when it completes.
+
+### 2. Refresh Source Truth (Recommended)
+
+If the host has accumulated significant agent activity or is known to be
+polluted, prefer:
+
+```bash
+selftune evolve --skill <name> --skill-path <path> --sync-first
+```
+
+`--sync-first` runs the authoritative transcript/rollout sync before eval-set
+generation and failure-pattern extraction. Use `--sync-force` when you need
+to ignore markers and rescan everything.
+
+### 3. Load or Generate Eval Set
 
 If `--eval-set` is provided, use it directly. Otherwise, the command
 generates one from logs (equivalent to running `evals --skill <name>`).
@@ -81,7 +184,7 @@ generates one from logs (equivalent to running `evals --skill <name>`).
 An eval set is required for validation. Without enough telemetry data,
 evolution cannot reliably measure improvement.
 
-### 2. Extract Failure Patterns
+### 4. Extract Failure Patterns
 
 The command groups missed queries by invocation type:
 - Missed explicit: description is broken (rare, high priority)
@@ -90,7 +193,7 @@ The command groups missed queries by invocation type:
 
 See `references/invocation-taxonomy.md` for the taxonomy.
 
-### 3. Propose Description Changes
+### 5. Propose Description Changes
 
 An LLM generates a candidate description that would catch the missed
 queries. The candidate:
@@ -98,7 +201,7 @@ queries. The candidate:
 - Adds new phrases covering missed patterns
 - Maintains the description's structure and tone
 
-### 4. Validate Against Eval Set
+### 6. Validate Against Eval Set
 
 The candidate is tested against the full eval set:
 - Must improve overall pass rate
@@ -108,7 +211,7 @@ The candidate is tested against the full eval set:
 If validation fails, the command retries up to `--max-iterations` times
 with adjusted proposals.
 
-### 5. Deploy (or Preview)
+### 7. Deploy (or Preview)
 
 If `--dry-run`, the proposal is printed but not deployed. The audit log
 still records `created` and `validated` entries for review.
@@ -118,6 +221,15 @@ If deploying:
 2. The updated description is written to SKILL.md
 3. A `deployed` entry is logged to the evolution audit
 
+### 8. Update Memory
+
+After evolution completes (deploy or dry-run), the memory writer updates:
+- `~/.selftune/memory/context.md` -- records the evolution outcome and current state
+- `~/.selftune/memory/decisions.md` -- logs the decision rationale and proposal details
+
+This ensures the next evolve, watch, or rollback workflow has full context
+even after a context window reset.
+
 ### Stopping Criteria
 
 The evolution loop stops when any of these conditions is met (priority order):
@@ -130,24 +242,72 @@ The evolution loop stops when any of these conditions is met (priority order):
 | 4 | **Plateau** | Pass rate unchanged across 3 consecutive iterations |
 | 5 | **Continue** | None of the above -- keep iterating |
 
+## Cheap Loop Mode
+
+The `--cheap-loop` flag runs the entire evolution loop with cheap models (haiku)
+and only uses an expensive model (sonnet) for a final gate validation before
+deploying. This reduces cost while maintaining deployment quality.
+
+When `--cheap-loop` is set:
+- `--proposal-model` defaults to `haiku`
+- `--validation-model` defaults to `haiku`
+- `--gate-model` defaults to `sonnet`
+
+The gate validation is a new step between validation and deploy. It re-runs
+`validateProposal` using the gate model. If the gate fails, the proposal is
+not deployed.
+
+```bash
+# Cheap loop with default models
+selftune evolve --skill X --skill-path Y --cheap-loop
+
+# Cheap loop with opus gate
+selftune evolve --skill X --skill-path Y --cheap-loop --gate-model opus
+
+# Manual model control without cheap-loop
+selftune evolve --skill X --skill-path Y --proposal-model haiku --validation-model sonnet
+```
+
 ## Common Patterns
 
-**"Evolve the pptx skill"**
-> Need `--skill pptx` and `--skill-path /path/to/pptx/SKILL.md`.
-> If the user hasn't specified the path, search for the SKILL.md file
-> in the workspace or ask.
+**User asks to evolve a specific skill (e.g., "evolve the pptx skill"):**
+Requires `--skill pptx` and `--skill-path /path/to/pptx/SKILL.md`.
+If the user has not specified the path, search for the SKILL.md file
+in the workspace. If not found, ask the user for the path.
+
+**User wants a preview without deployment (e.g., "just show me what would change"):**
+Add `--dry-run` to preview proposals without deploying.
+
+**Evolution results are insufficient:**
+Check the eval set quality. Missing contextual examples limit
+what evolution can learn. Generate a richer eval set first using the Evals workflow.
+
+**Evolution keeps failing validation:**
+Lower `--confidence` slightly or increase `--max-iterations`.
+Also check if the eval set has contradictory expectations.
+
+**Agent CLI override needed:**
+The evolve command auto-detects the installed agent CLI.
+Use `--agent <name>` to override (claude, codex, opencode).
+
+## Subagent Escalation
+
+For high-stakes evolutions, consider spawning the `evolution-reviewer` agent
+as a subagent to review the proposal before deploying. This is especially
+valuable when the skill has a history of regressions, the evolution touches
+many trigger phrases, or the confidence score is near the threshold.
 
-**"Just show me what would change"**
-> Use `--dry-run` to preview proposals without deploying.
+## Autonomous Mode
 
-**"The evolution didn't help enough"**
-> Check the eval set quality. Missing contextual examples will limit
-> what evolution can learn. Generate a richer eval set first.
+When called by `selftune orchestrate` (via cron or --loop), evolution runs
+without user interaction:
 
-**"Evolution keeps failing validation"**
-> Lower `--confidence` slightly or increase `--max-iterations`.
-> Also check if the eval set has contradictory expectations.
+- Pre-flight is skipped entirely — defaults are used
+- The orchestrator selects candidate skills based on health scores
+- Evolution uses cheap-loop mode (Haiku) by default
+- Validation runs automatically against the eval set
+- Deploy happens if validation passes the regression threshold
+- Results are logged to orchestrate-runs.jsonl
 
-**"Which agent is being used?"**
-> The evolve command auto-detects your installed agent CLI.
-> Use `--agent <name>` to override (claude, codex, opencode).
+No user confirmation is needed. The safety controls (regression threshold,
+auto-rollback via watch, SKILL.md backup) provide the guardrails.

package/skill/Workflows/EvolveBody.md

@@ -0,0 +1,159 @@
+# selftune Evolve Body Workflow
+
+Evolve a skill's full body content or routing table, not just the description.
+Uses a teacher-student model: a stronger LLM generates proposals, a cheaper
+LLM validates them through a 3-gate pipeline.
+
+## Default Command
+
+```bash
+selftune evolve body --skill <name> --skill-path <path> --target <target> [options]
+```
+
+## Options
+
+| Flag | Description | Default |
+|------|-------------|---------|
+| `--skill <name>` | Skill name | Required |
+| `--skill-path <path>` | Path to the skill's SKILL.md | Required |
+| `--target <type>` | Evolution target: `routing_table` or `full_body` | Required |
+| `--teacher-agent <name>` | Agent CLI for proposal generation | Auto-detected |
+| `--student-agent <name>` | Agent CLI for validation | Same as teacher |
+| `--teacher-model <flag>` | Model flag for teacher (e.g. `opus`) | Agent default |
+| `--student-model <flag>` | Model flag for student (e.g. `haiku`) | Agent default |
+| `--eval-set <path>` | Pre-built eval set JSON | Auto-generated from logs |
+| `--dry-run` | Propose and validate without deploying | Off |
+| `--max-iterations <n>` | Maximum refinement iterations | 3 |
+| `--task-description <text>` | Context for the evolution goal | None |
+| `--validation-model <model>` | Model for trigger-check validation calls (overrides `--student-model` for validation) | None |
+| `--few-shot <paths>` | Comma-separated paths to example SKILL.md files | None |
+
+## Evolution Targets
+
+### `routing_table`
+
+Optimizes the `## Workflow Routing` markdown table in SKILL.md. The teacher
+LLM analyzes missed triggers and proposes new routing entries that map
+trigger keywords to the correct workflow files.
+
+### `full_body`
+
+Rewrites the entire SKILL.md body below the frontmatter. This includes
+the description, routing table, examples, and all other sections. The
+teacher generates a complete replacement, validated through 3 gates.
+
+## 3-Gate Validation Pipeline
+
+Every proposal passes through three sequential gates:
+
+| Gate | Type | What it checks | Cost |
+|------|------|---------------|------|
+| **Gate 1: Structural** | Pure code | YAML frontmatter present, `# Title` exists, `## Workflow Routing` preserved if original had one | Free |
+| **Gate 2: Trigger Accuracy** | Student LLM | YES/NO trigger check per eval entry on the extracted description | Cheap |
+| **Gate 3: Quality** | Student LLM | Body clarity and completeness score (0.0-1.0) | Cheap |
+
+If any gate fails, the teacher receives structured feedback and generates
+a refined proposal. This repeats up to `--max-iterations` times.
+
+## Steps
+
+### 0. Pre-Flight Configuration
+
+Before running evolve-body, present configuration options to the user.
+If the user says "use defaults" or similar, skip to step 1 with recommended defaults.
+
+Present these options:
+
+```
+selftune evolve body — Pre-Flight Configuration
+
+1. Evolution Target
+   a) Routing table — optimize the workflow routing table only
+   b) Full body — rewrite entire SKILL.md body (more aggressive)
+
+2. Execution Mode
+   a) Dry run — preview proposal without deploying (recommended)
+   b) Live — validate and deploy if improved
+
+3. Teacher Model (generates proposals)
+   a) Balanced (sonnet) — good quality proposals (recommended)
+   b) Best (opus) — highest quality, slower and more expensive
+
+4. Student Model (validates proposals)
+   a) Fast (haiku) — cheap validation (recommended)
+   b) Balanced (sonnet) — higher quality validation
+
+5. Max Iterations: [3] (default)
+
+6. Few-Shot Examples: [none] (paths to example SKILL.md files for guidance)
+
+→ Reply with your choices or "use defaults" for recommended settings.
+```
+
+After the user responds, show a confirmation summary:
+
+```
+Configuration Summary:
+  Target:        routing_table
+  Mode:          dry-run
+  Teacher model: sonnet
+  Student model: haiku
+  Iterations:    3
+  Few-shot:      none
+
+Proceeding...
+```
+
+### 1. Parse Current Skill
+
+The command reads SKILL.md and splits it into sections using `parseSkillSections()`:
+- Frontmatter (YAML between `---` markers)
+- Title (first `# Heading`)
+- Description (text between title and first `## Section`)
+- Workflow Routing (the `## Workflow Routing` section, if present)
+- Remaining Body (everything else)
+
+### 2. Build Eval Set
+
+If `--eval-set` is provided, use it directly. Otherwise, generate from logs
+(same as `selftune eval generate --skill <name>`).
+
+### 3. Extract Failure Patterns
+
+Groups missed queries by invocation type, same as the description evolution
+pipeline. See `references/invocation-taxonomy.md`.
+
+### 4. Generate Proposal (Teacher)
+
+The teacher LLM generates a proposal based on the target:
+- **routing_table**: Optimized `## Workflow Routing` markdown table
+- **full_body**: Complete SKILL.md body replacement
+
+Few-shot examples from `--few-shot` paths provide structural guidance.
+
+### 5. Validate (3 Gates)
+
+Each gate runs in sequence. If a gate fails, the teacher receives the
+failure details and generates a refined proposal.
+
+### 6. Deploy or Preview
+
+If `--dry-run`, prints the proposal without deploying. Otherwise:
+1. Creates a timestamped backup of the current SKILL.md
+2. Applies the change: `replaceSection()` for routing, `replaceBody()` for full_body
+3. Records audit entries
+4. Updates evolution memory
+
+## Common Patterns
+
+**"Evolve the routing table for the Research skill"**
+> `selftune evolve body --skill Research --skill-path ~/.claude/skills/Research/SKILL.md --target routing_table`
+
+**"Rewrite the entire skill body"**
+> `selftune evolve body --skill Research --skill-path ~/.claude/skills/Research/SKILL.md --target full_body --dry-run`
+
+**"Use a stronger model for generation"**
+> `selftune evolve body --skill pptx --skill-path /path/SKILL.md --target full_body --teacher-model opus --student-model haiku`
+
+**"Preview what would change"**
+> Always start with `--dry-run` to review the proposal before deploying.

package/skill/Workflows/Grade.md

@@ -74,15 +74,14 @@ for the full schema. Key fields:
 
 ### 1. Find the Session
 
-Read `~/.claude/session_telemetry_log.jsonl` and find the most recent entry
-where `skills_triggered` contains the target skill name.
-
-Note the `transcript_path`, `tool_calls`, `errors_encountered`, and
-`session_id` fields. See `references/logs.md` for the telemetry format.
+Read `~/.claude/session_telemetry_log.jsonl`. Find the most recent entry
+where `skills_triggered` contains the target skill name. Extract the
+`transcript_path`, `tool_calls`, `errors_encountered`, and `session_id`
+fields. See `references/logs.md` for the telemetry format.
 
 ### 2. Read the Transcript
 
-Parse the JSONL file at `transcript_path`. Identify:
+Parse the JSONL file at `transcript_path`. Extract:
 - User messages (what was asked)
 - Assistant tool calls (what the agent did)
 - Tool results (what happened)
@@ -92,16 +91,14 @@ See `references/logs.md` for transcript format variants.
 
 ### 3. Determine Expectations
 
-If the user provided `--expectations`, parse the semicolon-separated list.
-Otherwise, derive defaults. See `references/grading-methodology.md` for the
-full default expectations list.
-
-Always include at least one Process expectation and one Quality expectation.
+If `--expectations` was provided, parse the semicolon-separated list.
+Otherwise, derive defaults from `references/grading-methodology.md`.
+Ensure at least one Process expectation and one Quality expectation.
 
 ### 4. Grade Each Expectation
 
-For each expectation, search both the telemetry record and the transcript
-for evidence. Mark as:
+Search both the telemetry record and the transcript for evidence per
+expectation. Mark as:
 - **PASS** if evidence exists and supports the expectation
 - **FAIL** if evidence is absent or contradicts the expectation
 
@@ -109,22 +106,21 @@ Cite specific evidence: transcript line numbers, tool call names, bash output.
 
 ### 5. Extract Implicit Claims
 
-Pull 2-4 claims from the transcript that are not covered by the explicit
-expectations. Classify each as factual, process, or quality. Verify each
-against the transcript. See `references/grading-methodology.md` for claim
-types and examples.
+Pull 2-4 claims from the transcript not covered by explicit expectations.
+Classify each as factual, process, or quality. Verify each against the
+transcript. See `references/grading-methodology.md` for claim types.
 
 ### 6. Flag Eval Gaps
 
 Review each passed expectation. If it would also pass for wrong output,
-note it in `eval_feedback.suggestions`. See `references/grading-methodology.md`
-for gap flagging criteria.
+record it in `eval_feedback.suggestions`. See
+`references/grading-methodology.md` for gap flagging criteria.
 
 ### 7. Write grading.json
 
 Write the full grading result to `grading.json` in the current directory.
 
-### 8. Summarize
+### 8. Report Results
 
 Report to the user:
 - Pass rate (e.g., "2/3 passed, 67%")
@@ -136,17 +132,26 @@ Keep the summary concise. The full details are in `grading.json`.
 
 ## Common Patterns
 
-**"Grade my last pptx session"**
-> Find the most recent telemetry entry for `pptx`. Use default expectations.
-> Ask if the user wants custom expectations or proceed with defaults.
+**User asks to grade a skill session**
+> Run `selftune grade --skill <name>` with default expectations. Results are
+> written to `grading.json`. Read that file and report the pass rate and any
+> failures to the user.
+
+**User provides specific expectations**
+> Run `selftune grade --skill <name> --expectations "expect1;expect2;expect3"`.
+> Parse results and report.
+
+**User wants to grade from an eval set**
+> Run `selftune grade --skill <name> --evals-json path/to/evals.json`.
+> Optionally add `--eval-id N` for a specific scenario.
 
-**"Grade with these specific expectations"**
-> Pass `--expectations "expect1;expect2;expect3"` to override defaults.
+**Agent detection override needed**
+> The grader auto-detects the agent CLI. If detection fails or the user
+> specifies an agent, pass `--agent <name>` to override.
 
-**"Grade using an eval set"**
-> Pass `--evals-json path/to/evals.json` and optionally `--eval-id N`
-> to grade a specific eval scenario.
+## Autonomous Mode
 
-**"Which agent is being used?"**
-> The grader auto-detects your installed agent CLI (claude, codex, opencode).
-> Use `--agent <name>` to override the auto-detected agent.
+Grading runs implicitly during orchestrate as part of status computation.
+The orchestrator reads grading results to determine which skills are
+candidates for evolution. No explicit grade command is called — the
+grading results from previous sessions feed into candidate selection.

package/skill/Workflows/ImportSkillsBench.md

@@ -0,0 +1,117 @@
+# selftune Import SkillsBench Workflow
+
+## When to Use
+
+When the user wants to enrich eval sets with external benchmark tasks or import SkillsBench corpora.
+
+## Overview
+
+Import evaluation tasks from the SkillsBench corpus (87 real-world agent
+benchmarks) and convert them to selftune eval entries. This enriches
+your skill's eval set with externally validated test cases.
+
+## Default Command
+
+```bash
+selftune eval import --dir <path> --skill <name> --output <path> [options]
+```
+
+## Options
+
+| Flag | Description | Default |
+|------|-------------|---------|
+| `--dir <path>` | Path to SkillsBench tasks directory | Required |
+| `--skill <name>` | Target skill to match tasks against | Required |
+| `--output <path>` | Output eval set JSON file | Required |
+| `--match-strategy <type>` | Matching strategy: `exact` or `fuzzy` | `exact` |
+
+## Match Strategies
+
+### `exact`
+
+Matches tasks where `expected_skill` in `task.toml` exactly matches the
+target skill name. Precise but may miss relevant tasks.
+
+### `fuzzy`
+
+Uses keyword overlap between the task's category/tags and the skill name.
+Casts a wider net but may include marginally relevant tasks. Review the
+output and remove false matches.
+
+## SkillsBench Directory Structure
+
+The importer expects this layout:
+
+```
+tasks/
+├── task-001/
+│   ├── instruction.md     # Task description (used as query)
+│   └── task.toml          # Metadata (difficulty, category, tags, expected_skill)
+├── task-002/
+│   ├── instruction.md
+│   └── task.toml
+└── ...
+```
+
+### `task.toml` Format
+
+```toml
+difficulty = "medium"
+category = "research"
+tags = ["web-search", "analysis", "summarization"]
+expected_skill = "Research"
+expected_tools = ["WebSearch", "Read"]
+```
+
+All fields are optional. Tasks without `task.toml` use default values.
+
+## Output Format
+
+Standard selftune eval entries:
+
+```json
+[
+  {
+    "id": 1,
+    "query": "Find and summarize the latest papers on transformer architectures",
+    "expected": true,
+    "invocation_type": "implicit",
+    "skill_name": "Research",
+    "source_session": null,
+    "source": "skillsbench"
+  }
+]
+```
+
+## Steps
+
+### 1. Obtain SkillsBench Corpus
+
+Clone or download the SkillsBench repository containing the task directory.
+
+### 2. Import Tasks
+
+```bash
+selftune eval import --dir /path/to/skillsbench/tasks --skill Research --output evals-bench.json
+```
+
+### 3. Review Output
+
+Inspect the generated eval entries. Remove any that don't match your skill's
+intended scope. Adjust match strategy if needed.
+
+### 4. Merge with Existing Evals
+
+Combine imported entries with your existing eval set for a richer validation
+corpus. Use the merged set with `selftune evolve --eval-set merged-evals.json`.
+
+## Common Patterns
+
+**"Import SkillsBench tasks for Research"**
+> `selftune eval import --dir /path/tasks --skill Research --output bench-evals.json`
+
+**"Use fuzzy matching for broader coverage"**
+> `selftune eval import --dir /path/tasks --skill pptx --output bench-evals.json --match-strategy fuzzy`
+
+**"Enrich my eval set with external benchmarks"**
+> Import with `eval import`, then pass the output to `evolve --eval-set`.