harness-evolver 1.5.0 → 1.6.0

package/agents/harness-evolver-architect.md

@@ -4,6 +4,7 @@ description: |
   Use this agent to analyze harness architecture and recommend optimal multi-agent topology.
   Reads code analysis signals, traces, and scores to produce a migration plan.
 tools: Read, Write, Bash, Grep, Glob
+color: blue
 ---
 
 ## Bootstrap

package/agents/harness-evolver-critic.md

@@ -4,6 +4,7 @@ description: |
   Use this agent to assess eval quality, detect eval gaming, and propose stricter evaluation.
   Triggered when scores converge suspiciously fast or on user request.
 tools: Read, Write, Bash, Grep, Glob
+color: red
 ---
 
 ## Bootstrap

package/agents/harness-evolver-proposer.md

@@ -4,6 +4,7 @@ description: |
   Use this agent when the evolve skill needs to propose a new harness candidate.
   Navigates the .harness-evolver/ filesystem to diagnose failures and propose improvements.
 tools: Read, Write, Edit, Bash, Glob, Grep
+color: green
 permissionMode: acceptEdits
 ---
 
@@ -12,6 +13,24 @@ permissionMode: acceptEdits
 If your prompt contains a `<files_to_read>` block, you MUST use the Read tool to load
 every file listed there before performing any other actions. These files are your context.
 
+## Context7 — Enrich Your Knowledge
+
+You have access to Context7 MCP tools (`resolve-library-id` and `get-library-docs`) for looking up **current, version-specific documentation** of any library.
+
+**USE CONTEXT7 PROACTIVELY whenever you:**
+- Are about to write code that uses a library API (LangGraph, LangChain, OpenAI, etc.)
+- Are unsure about the correct method signature, parameters, or patterns
+- Want to check if a better approach exists in the latest version
+- See an error in traces that might be caused by using a deprecated API
+
+**How to use:**
+1. `resolve-library-id` with the library name (e.g., "langchain", "langgraph")
+2. `get-library-docs` with a specific query (e.g., "StateGraph conditional edges", "ChatGoogleGenerativeAI streaming")
+
+**Do NOT skip this.** Your training data may be outdated. Context7 gives you the current docs. Even if you're confident about an API, a quick check takes seconds and prevents proposing deprecated patterns.
+
+If Context7 is not available, proceed with model knowledge but note in `proposal.md`: "API not verified against current docs."
+
 ## Return Protocol
 
 When done, end your response with:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "harness-evolver",
-  "version": "1.5.0",
+  "version": "1.6.0",
   "description": "Meta-Harness-style autonomous harness optimization for Claude Code",
   "author": "Raphael Valdetaro",
   "license": "MIT",

package/skills/evolve/SKILL.md

@@ -34,39 +34,19 @@ For each iteration:
 python3 -c "import json; s=json.load(open('.harness-evolver/summary.json')); print(f'v{s[\"iterations\"]+1:03d}')"
 ```
 
-### 1.5. Gather Diagnostic Context (LangSmith + Context7)
+### 1.5. Gather LangSmith Traces (MANDATORY after every evaluation)
 
-**This step is MANDATORY before every propose.** The orchestrator gathers data so the proposer receives it as files.
+**Run these commands unconditionally after EVERY evaluation** (including baseline). If langsmith-cli is not installed or there are no runs, the commands fail silently — that's fine. But you MUST attempt them.
 
-**LangSmith (if enabled):**
-
-Check if LangSmith is enabled and langsmith-cli is available:
 ```bash
-cat .harness-evolver/config.json | python3 -c "import sys,json; print(json.load(sys.stdin).get('eval',{}).get('langsmith',{}).get('enabled',False))"
-which langsmith-cli 2>/dev/null
-```
+langsmith-cli --json runs list --project harness-evolver-{last_evaluated_version} --failed --fields id,name,error,inputs --limit 10 > .harness-evolver/langsmith_diagnosis.json 2>/dev/null || echo "[]" > .harness-evolver/langsmith_diagnosis.json
 
-If BOTH are true AND at least one iteration has run, gather LangSmith data:
-```bash
-langsmith-cli --json runs list --project harness-evolver-{best_version} --failed --fields id,name,error,inputs --limit 10 > .harness-evolver/langsmith_diagnosis.json 2>/dev/null || echo "[]" > .harness-evolver/langsmith_diagnosis.json
-
-langsmith-cli --json runs stats --project harness-evolver-{best_version} > .harness-evolver/langsmith_stats.json 2>/dev/null || echo "{}" > .harness-evolver/langsmith_stats.json
-```
-
-**Context7 (if available):**
-
-Check `config.json` field `stack.detected`. For each detected library, use the Context7 MCP tools to fetch relevant documentation:
-
-```
-For each library in stack.detected:
-  1. resolve-library-id with the context7_id
-  2. get-library-docs with a query relevant to the current failure modes
-  3. Save output to .harness-evolver/context7_docs.md (append each library's docs)
+langsmith-cli --json runs stats --project harness-evolver-{last_evaluated_version} > .harness-evolver/langsmith_stats.json 2>/dev/null || echo "{}" > .harness-evolver/langsmith_stats.json
 ```
 
-This runs ONCE per iteration, not per library. Focus on the library most relevant to the current failures.
+For the first iteration, use `baseline` as the version. For subsequent iterations, use the latest evaluated version.
 
-If Context7 MCP is not available, skip silently.
+These files are included in the proposer's `<files_to_read>` so it has real trace data for diagnosis.
 
 ### 2. Propose
 

package/tools/init.py

@@ -298,10 +298,26 @@ def main():
             print("  Recommendation: install langsmith-cli for rich trace analysis:")
             print("    uv tool install langsmith-cli && langsmith-cli auth login")
 
-    # Detect stack
-    stack = _detect_stack(args.harness)
+    # Detect stack — try original harness first, then baseline copy, then scan entire source dir
+    stack = _detect_stack(os.path.abspath(args.harness))
+    if not stack:
+        stack = _detect_stack(os.path.join(base, "baseline", "harness.py"))
+    if not stack:
+        # Scan the original directory for any .py files with known imports
+        harness_dir = os.path.dirname(os.path.abspath(args.harness))
+        detect_stack_py = os.path.join(os.path.dirname(__file__), "detect_stack.py")
+        if os.path.exists(detect_stack_py):
+            try:
+                r = subprocess.run(
+                    ["python3", detect_stack_py, harness_dir],
+                    capture_output=True, text=True, timeout=30,
+                )
+                if r.returncode == 0 and r.stdout.strip():
+                    stack = json.loads(r.stdout)
+            except Exception:
+                pass
     config["stack"] = {
-        "detected": stack,
+        "detected": stack if stack else {},
         "documentation_hint": "use context7",
         "auto_detected": True,
     }