harness-evolver 1.6.0 → 1.8.0

package/agents/harness-evolver-proposer.md

@@ -13,6 +13,16 @@ permissionMode: acceptEdits
 If your prompt contains a `<files_to_read>` block, you MUST use the Read tool to load
 every file listed there before performing any other actions. These files are your context.
 
+## Strategy Injection
+
+Your prompt may contain a `<strategy>` block defining your evolutionary role:
+- **exploitation**: Make targeted, conservative fixes to the current best
+- **exploration**: Try fundamentally different approaches, be bold
+- **crossover**: Combine strengths from two parent versions
+
+Follow the strategy. It determines your risk tolerance and parent selection.
+If no strategy block is present, default to exploitation (conservative improvement).
+
 ## Context7 — Enrich Your Knowledge
 
 You have access to Context7 MCP tools (`resolve-library-id` and `get-library-docs`) for looking up **current, version-specific documentation** of any library.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "harness-evolver",
-  "version": "1.6.0",
+  "version": "1.8.0",
   "description": "Meta-Harness-style autonomous harness optimization for Claude Code",
   "author": "Raphael Valdetaro",
   "license": "MIT",

package/skills/architect/SKILL.md

@@ -48,13 +48,21 @@ python3 $TOOLS/analyze_architecture.py \
     -o .harness-evolver/architecture_signals.json
 ```
 
-3. Dispatch subagent using the **Agent tool** with `subagent_type: "harness-evolver-architect"`:
+3. Read the architect agent definition:
+```bash
+cat ~/.claude/agents/harness-evolver-architect.md
+```
+
+4. Dispatch using the Agent tool — include the agent definition in the prompt:
 
 ```
 Agent(
-  subagent_type: "harness-evolver-architect",
   description: "Architect: topology analysis",
   prompt: |
+    <agent_instructions>
+    {paste the FULL content of harness-evolver-architect.md here}
+    </agent_instructions>
+
     <objective>
     Analyze the harness architecture and recommend the optimal multi-agent topology.
     {If called from evolve: "The evolution loop stagnated/regressed after N iterations."}

package/skills/critic/SKILL.md

@@ -22,13 +22,21 @@ TOOLS=$([ -d ".harness-evolver/tools" ] && echo ".harness-evolver/tools" || echo
 
 1. Read `summary.json` and identify the suspicious pattern (score jump, premature convergence).
 
-2. Dispatch subagent using the **Agent tool** with `subagent_type: "harness-evolver-critic"`:
+2. Read the critic agent definition:
+```bash
+cat ~/.claude/agents/harness-evolver-critic.md
+```
+
+3. Dispatch using the Agent tool — include the agent definition in the prompt:
 
 ```
 Agent(
-  subagent_type: "harness-evolver-critic",
   description: "Critic: analyze eval quality",
   prompt: |
+    <agent_instructions>
+    {paste the FULL content of harness-evolver-critic.md here}
+    </agent_instructions>
+
     <objective>
     Analyze eval quality for this harness evolution project.
     The best version is {version} with score {score} achieved in {iterations} iteration(s).

package/skills/evolve/SKILL.md

@@ -48,78 +48,184 @@ For the first iteration, use `baseline` as the version. For subsequent iteration
 
 These files are included in the proposer's `<files_to_read>` so it has real trace data for diagnosis.
 
-### 2. Propose
+### 2. Propose (3 parallel candidates)
 
-Dispatch a subagent using the **Agent tool** with `subagent_type: "harness-evolver-proposer"`.
+Spawn 3 proposer agents IN PARALLEL, each with a different evolutionary strategy.
+This follows the DGM/AlphaEvolve pattern: exploit + explore + crossover.
 
-The prompt MUST be the full XML block below. The `<files_to_read>` MUST include the LangSmith/Context7 files if they were gathered in step 1.5.
+First, read the proposer agent definition:
+```bash
+cat ~/.claude/agents/harness-evolver-proposer.md
+```
+
+Then determine parents for each strategy:
+- **Exploiter parent**: current best version (from summary.json `best.version`)
+- **Explorer parent**: a non-best version with low offspring count (read summary.json history, pick one that scored >0 but is NOT the best and has NOT been parent to many children)
+- **Crossover parents**: best version + a different high-scorer from a different lineage
 
+Spawn all 3 using the Agent tool. The first 2 use `run_in_background: true`, the 3rd blocks:
+
+**Candidate A (Exploiter)** — `run_in_background: true`:
 ```
 Agent(
-  subagent_type: "harness-evolver-proposer",
-  description: "Propose harness {version}",
+  description: "Proposer A (exploit): targeted fix for {version}",
+  run_in_background: true,
   prompt: |
+    <agent_instructions>
+    {FULL content of harness-evolver-proposer.md}
+    </agent_instructions>
+
+    <strategy>
+    APPROACH: exploitation
+    You are the EXPLOITER. Make the SMALLEST, most targeted change that fixes
+    the highest-impact failing tasks. Base your work on the current best version.
+    Do NOT restructure the code. Do NOT change the architecture.
+    Focus on: prompt tweaks, parameter tuning, fixing specific failure modes.
+    </strategy>
+
     <objective>
-    Propose harness version {version} that improves on the current best score of {best_score}.
+    Propose harness version {version}a that improves on {best_score}.
     </objective>
 
     <files_to_read>
     - .harness-evolver/summary.json
     - .harness-evolver/PROPOSER_HISTORY.md
     - .harness-evolver/config.json
-    - .harness-evolver/baseline/harness.py
     - .harness-evolver/harnesses/{best_version}/harness.py
     - .harness-evolver/harnesses/{best_version}/scores.json
     - .harness-evolver/harnesses/{best_version}/proposal.md
-    - .harness-evolver/langsmith_diagnosis.json (if exists — LangSmith failure analysis)
-    - .harness-evolver/langsmith_stats.json (if exists — LangSmith aggregate stats)
-    - .harness-evolver/context7_docs.md (if exists — current library documentation)
-    - .harness-evolver/architecture.json (if exists — architect topology recommendation)
+    - .harness-evolver/langsmith_diagnosis.json (if exists)
+    - .harness-evolver/langsmith_stats.json (if exists)
+    - .harness-evolver/architecture.json (if exists)
     </files_to_read>
 
     <output>
-    Create directory .harness-evolver/harnesses/{version}/ containing:
-    - harness.py (the improved harness)
-    - config.json (parameters, copy from parent if unchanged)
-    - proposal.md (reasoning, must start with "Based on v{PARENT}")
+    Create directory .harness-evolver/harnesses/{version}a/ containing:
+    - harness.py, config.json, proposal.md
     </output>
+)
+```
 
-    <success_criteria>
-    - harness.py maintains CLI interface (--input, --output, --traces-dir, --config)
-    - proposal.md documents evidence-based reasoning
-    - Changes are motivated by trace analysis (LangSmith data if available), not guesswork
-    - If context7_docs.md was provided, API usage must match current documentation
-    </success_criteria>
+**Candidate B (Explorer)** — `run_in_background: true`:
+```
+Agent(
+  description: "Proposer B (explore): bold change from {explorer_parent}",
+  run_in_background: true,
+  prompt: |
+    <agent_instructions>
+    {FULL content of harness-evolver-proposer.md}
+    </agent_instructions>
+
+    <strategy>
+    APPROACH: exploration
+    You are the EXPLORER. Try a FUNDAMENTALLY DIFFERENT approach.
+    Base your work on {explorer_parent} (NOT the current best — intentionally diverging).
+    Consider: different retrieval strategy, different prompt structure,
+    different output parsing, different error handling philosophy.
+    Be bold. A creative failure teaches more than a timid success.
+    </strategy>
+
+    <objective>
+    Propose harness version {version}b that takes a different approach.
+    </objective>
+
+    <files_to_read>
+    - .harness-evolver/summary.json
+    - .harness-evolver/PROPOSER_HISTORY.md
+    - .harness-evolver/config.json
+    - .harness-evolver/baseline/harness.py
+    - .harness-evolver/harnesses/{explorer_parent}/harness.py
+    - .harness-evolver/harnesses/{explorer_parent}/scores.json
+    - .harness-evolver/langsmith_diagnosis.json (if exists)
+    - .harness-evolver/architecture.json (if exists)
+    </files_to_read>
+
+    <output>
+    Create directory .harness-evolver/harnesses/{version}b/ containing:
+    - harness.py, config.json, proposal.md
+    </output>
+)
+```
+
+**Candidate C (Crossover)** — blocks (last one):
+```
+Agent(
+  description: "Proposer C (crossover): combine {parent_a} + {parent_b}",
+  prompt: |
+    <agent_instructions>
+    {FULL content of harness-evolver-proposer.md}
+    </agent_instructions>
+
+    <strategy>
+    APPROACH: crossover
+    You are the CROSSOVER agent. Combine the STRENGTHS of two different versions:
+    - {parent_a} (score: {score_a}): {summary of what it does well}
+    - {parent_b} (score: {score_b}): {summary of what it does well}
+    Take the best elements from each and merge them into a single harness.
+    </strategy>
+
+    <objective>
+    Propose harness version {version}c that combines the best of {parent_a} and {parent_b}.
+    </objective>
+
+    <files_to_read>
+    - .harness-evolver/summary.json
+    - .harness-evolver/PROPOSER_HISTORY.md
+    - .harness-evolver/config.json
+    - .harness-evolver/harnesses/{parent_a}/harness.py
+    - .harness-evolver/harnesses/{parent_a}/scores.json
+    - .harness-evolver/harnesses/{parent_b}/harness.py
+    - .harness-evolver/harnesses/{parent_b}/scores.json
+    - .harness-evolver/langsmith_diagnosis.json (if exists)
+    - .harness-evolver/architecture.json (if exists)
+    </files_to_read>
+
+    <output>
+    Create directory .harness-evolver/harnesses/{version}c/ containing:
+    - harness.py, config.json, proposal.md
+    </output>
 )
 ```
 
-Wait for `## PROPOSAL COMPLETE` in the response. The subagent gets a fresh context window and loads the agent definition from `~/.claude/agents/harness-evolver-proposer.md`.
+Wait for all 3 to complete. The background agents will notify when done.
+
+**Special case — iteration 1**: Only the exploiter and explorer can run (no second parent for crossover yet). Spawn 2 agents: exploiter (from baseline) and explorer (also from baseline but with bold strategy). Skip crossover.
 
-### 3. Validate
+**Special case — iteration 2+**: All 3 strategies. Explorer parent = fitness-weighted random from history excluding current best.
 
+### 3. Validate All Candidates
+
+For each candidate (a, b, c):
 ```bash
-python3 $TOOLS/evaluate.py validate \
-    --harness .harness-evolver/harnesses/{version}/harness.py \
-    --config .harness-evolver/harnesses/{version}/config.json
+python3 $TOOLS/evaluate.py validate --harness .harness-evolver/harnesses/{version}{suffix}/harness.py --config .harness-evolver/harnesses/{version}{suffix}/config.json
 ```
 
-If fails: one retry via proposer. If still fails: score 0.0, continue.
+Remove any that fail validation.
 
-### 4. Evaluate
+### 4. Evaluate All Candidates
 
+For each valid candidate:
 ```bash
 python3 $TOOLS/evaluate.py run \
-    --harness .harness-evolver/harnesses/{version}/harness.py \
-    --config .harness-evolver/harnesses/{version}/config.json \
+    --harness .harness-evolver/harnesses/{version}{suffix}/harness.py \
+    --config .harness-evolver/harnesses/{version}{suffix}/config.json \
     --tasks-dir .harness-evolver/eval/tasks/ \
     --eval .harness-evolver/eval/eval.py \
-    --traces-dir .harness-evolver/harnesses/{version}/traces/ \
-    --scores .harness-evolver/harnesses/{version}/scores.json \
+    --traces-dir .harness-evolver/harnesses/{version}{suffix}/traces/ \
+    --scores .harness-evolver/harnesses/{version}{suffix}/scores.json \
     --timeout 60
 ```
 
-### 5. Update State
+### 5. Select Winner + Update State
 
+Compare scores of all evaluated candidates. The winner is the one with highest combined_score.
+
+Rename the winner directory to the official version name:
+```bash
+mv .harness-evolver/harnesses/{version}{winning_suffix} .harness-evolver/harnesses/{version}
+```
+
+Update state with the winner:
 ```bash
 python3 $TOOLS/state.py update \
     --base-dir .harness-evolver \
@@ -128,6 +234,17 @@ python3 $TOOLS/state.py update \
     --proposal .harness-evolver/harnesses/{version}/proposal.md
 ```
 
+Report ALL candidates:
+```
+Iteration {i}/{N} — 3 candidates evaluated:
+  {version}a (exploit): {score_a} — {1-line summary from proposal.md}
+  {version}b (explore): {score_b} — {1-line summary}
+  {version}c (cross):   {score_c} — {1-line summary}
+  Winner: {version}{suffix} ({score}) ← promoted to {version}
+```
+
+Keep losing candidates in their directories (they're part of the archive — never discard, per DGM).
+
 ### 6. Report
 
 Read `summary.json`. Print: `Iteration {i}/{N}: {version} scored {score} (best: {best} at {best_score})`
@@ -150,13 +267,21 @@ python3 $TOOLS/evaluate.py run \
     --timeout 60
 ```
 
-Dispatch critic subagent using the **Agent tool** with `subagent_type: "harness-evolver-critic"`:
+First read the critic agent definition:
+```bash
+cat ~/.claude/agents/harness-evolver-critic.md
+```
+
+Then dispatch:
 
 ```
 Agent(
-  subagent_type: "harness-evolver-critic",
   description: "Critic: analyze eval quality",
   prompt: |
+    <agent_instructions>
+    {paste the FULL content of harness-evolver-critic.md here}
+    </agent_instructions>
+
     <objective>
     EVAL GAMING DETECTED: Score jumped from {parent_score} to {score} in one iteration.
     Analyze the eval quality and propose a stricter eval.
@@ -219,13 +344,21 @@ python3 $TOOLS/analyze_architecture.py \
     -o .harness-evolver/architecture_signals.json
 ```
 
-Dispatch architect subagent using the **Agent tool** with `subagent_type: "harness-evolver-architect"`:
+First read the architect agent definition:
+```bash
+cat ~/.claude/agents/harness-evolver-architect.md
+```
+
+Then dispatch:
 
 ```
 Agent(
-  subagent_type: "harness-evolver-architect",
   description: "Architect: analyze topology after {stagnation/regression}",
   prompt: |
+    <agent_instructions>
+    {paste the FULL content of harness-evolver-architect.md here}
+    </agent_instructions>
+
     <objective>
     The evolution loop has {stagnated/regressed} after {iterations} iterations (best: {best_score}).
     Analyze the harness architecture and recommend a topology change.