harness-evolver 2.9.0 → 3.0.0

package/skills/architect/SKILL.md

@@ -1,93 +0,0 @@
----
-name: harness-evolver:architect
-description: "Use when the user wants to analyze harness architecture, get a topology recommendation, understand if their agent pattern is optimal, or after stagnation in the evolution loop."
-argument-hint: "[--force]"
-allowed-tools: [Read, Write, Edit, Bash, Glob, Grep, Agent]
----
-
-# /harness-evolver:architect
-
-Analyze the current harness architecture and recommend the optimal multi-agent topology.
-
-## Prerequisites
-
-`.harness-evolver/` must exist. If not, tell user to run `harness-evolver:init` first.
-
-```bash
-if [ ! -d ".harness-evolver" ]; then
-  echo "ERROR: .harness-evolver/ not found. Run /harness-evolver:init first."
-  exit 1
-fi
-```
-
-## Resolve Tool Path
-
-```bash
-TOOLS=$([ -d ".harness-evolver/tools" ] && echo ".harness-evolver/tools" || echo "$HOME/.harness-evolver/tools")
-```
-
-Use `$TOOLS` prefix for all tool calls below.
-
-## What To Do
-
-1. Check `.harness-evolver/` exists.
-
-2. Run architecture analysis tool:
-```bash
-python3 $TOOLS/analyze_architecture.py \
-    --harness .harness-evolver/baseline/harness.py \
-    -o .harness-evolver/architecture_signals.json
-```
-
-If evolution has run, add trace and score data:
-```bash
-python3 $TOOLS/analyze_architecture.py \
-    --harness .harness-evolver/harnesses/{best}/harness.py \
-    --traces-dir .harness-evolver/harnesses/{best}/traces \
-    --summary .harness-evolver/summary.json \
-    -o .harness-evolver/architecture_signals.json
-```
-
-3. Dispatch using the Agent tool with `subagent_type: "harness-evolver-architect"`:
-
-```
-Agent(
-  subagent_type: "harness-evolver-architect",
-  description: "Architect: topology analysis",
-  prompt: |
-    <objective>
-    Analyze the harness architecture and recommend the optimal multi-agent topology.
-    {If called from evolve: "The evolution loop stagnated/regressed after N iterations."}
-    {If called by user: "The user requested an architecture analysis."}
-    </objective>
-
-    <files_to_read>
-    - .harness-evolver/architecture_signals.json
-    - .harness-evolver/config.json
-    - .harness-evolver/baseline/harness.py
-    - .harness-evolver/summary.json (if exists)
-    - .harness-evolver/PROPOSER_HISTORY.md (if exists)
-    </files_to_read>
-
-    <output>
-    Write:
-    - .harness-evolver/architecture.json
-    - .harness-evolver/architecture.md
-    </output>
-
-    <success_criteria>
-    - Classifies current topology correctly
-    - Recommendation includes migration path with concrete steps
-    - Considers detected stack and API key availability
-    - Confidence rating is honest (low/medium/high)
-    </success_criteria>
-)
-```
-
-4. Wait for `## ARCHITECTURE ANALYSIS COMPLETE`.
-
-5. Print summary: current -> recommended, confidence, migration steps.
-
-## Arguments
-
-- `--force` — re-run analysis even if `architecture.json` already exists. Without this flag, if `architecture.json` exists, just display the existing recommendation.

package/skills/compare/SKILL.md

@@ -1,73 +0,0 @@
----
-name: harness-evolver:compare
-description: "Use when the user wants to compare two harness versions, understand what changed between iterations, see why one version scored better than another, or debug a regression."
-argument-hint: "<vA> <vB>"
-allowed-tools: [Read, Bash, Glob, Grep]
----
-
-# /harness-evolver:compare
-
-Compare two harness versions side by side.
-
-## Arguments
-
-- `vA` — first version (e.g., `v001`, `baseline`)
-- `vB` — second version (e.g., `v003`)
-
-If only one version given, compare it against the current best.
-If no versions given, compare the two most recent.
-
-## What To Do
-
-### 1. Code Diff
-
-```bash
-diff .harness-evolver/harnesses/{vA}/harness.py .harness-evolver/harnesses/{vB}/harness.py
-```
-
-If config changed:
-```bash
-diff .harness-evolver/harnesses/{vA}/config.json .harness-evolver/harnesses/{vB}/config.json
-```
-
-### 2. Score Comparison
-
-```bash
-cat .harness-evolver/harnesses/{vA}/scores.json
-cat .harness-evolver/harnesses/{vB}/scores.json
-```
-
-Report: combined_score delta, per-task wins/losses.
-
-### 3. Per-Task Analysis
-
-For tasks where scores diverge, show what each version produced:
-
-```bash
-cat .harness-evolver/harnesses/{vA}/traces/task_{ID}/output.json
-cat .harness-evolver/harnesses/{vB}/traces/task_{ID}/output.json
-```
-
-### 4. Proposal Context
-
-```bash
-cat .harness-evolver/harnesses/{vB}/proposal.md
-```
-
-Show what the proposer intended and whether the result matched expectations.
-
-## Report Format
-
-```
-v001 (0.62) vs v003 (0.71) — +0.09 improvement
-
-Code changes:
-  + Added few-shot examples (3 examples)
-  ~ Changed prompt template
-  - Removed retry logic
-
-Per-task:
-  task_001: 1.0 → 1.0 (unchanged)
-  task_007: 0.0 → 1.0 (FIXED — was cardiac, now correctly classified)
-  task_008: 1.0 → 0.0 (REGRESSION — was neurological, now wrong)
-```

package/skills/critic/SKILL.md

@@ -1,67 +0,0 @@
----
-name: harness-evolver:critic
-description: "Use when scores converge suspiciously fast, eval quality is questionable, the harness reaches 1.0 in few iterations, or the user wants to validate that improvements are genuine. Also triggers automatically when score jumps >0.3 in one iteration."
-allowed-tools: [Read, Write, Edit, Bash, Glob, Grep, Agent]
----
-
-# /harness-evolver:critic
-
-Analyze eval quality and detect eval gaming.
-
-## Resolve Tool Path
-
-```bash
-TOOLS=$([ -d ".harness-evolver/tools" ] && echo ".harness-evolver/tools" || echo "$HOME/.harness-evolver/tools")
-```
-
-## Prerequisites
-
-`.harness-evolver/` must exist with at least one evaluated version (v001+).
-
-## What To Do
-
-1. Read `summary.json` and identify the suspicious pattern (score jump, premature convergence).
-
-2. Dispatch using the Agent tool with `subagent_type: "harness-evolver-critic"`:
-
-```
-Agent(
-  subagent_type: "harness-evolver-critic",
-  description: "Critic: analyze eval quality",
-  prompt: |
-    <objective>
-    Analyze eval quality for this harness evolution project.
-    The best version is {version} with score {score} achieved in {iterations} iteration(s).
-    {Specific concern: "Score jumped from X to Y in one iteration" or "Perfect score in N iterations"}
-    </objective>
-
-    <files_to_read>
-    - .harness-evolver/eval/eval.py
-    - .harness-evolver/summary.json
-    - .harness-evolver/harnesses/{best_version}/scores.json
-    - .harness-evolver/harnesses/{best_version}/harness.py
-    - .harness-evolver/harnesses/{best_version}/proposal.md
-    - .harness-evolver/config.json
-    - .harness-evolver/langsmith_stats.json (if exists)
-    </files_to_read>
-
-    <output>
-    Write:
-    - .harness-evolver/critic_report.md (human-readable analysis)
-    - .harness-evolver/eval/eval_improved.py (if weaknesses found)
-    </output>
-
-    <success_criteria>
-    - Identifies specific weaknesses in eval.py with examples
-    - If gaming detected, shows exact tasks/outputs that expose the weakness
-    - Improved eval preserves the --results-dir/--tasks-dir/--scores interface
-    - Re-scores the best version with improved eval to quantify the difference
-    </success_criteria>
-)
-```
-
-3. Wait for `## CRITIC REPORT COMPLETE`.
-
-4. Report findings to user. If `eval_improved.py` was written:
-   - Show score comparison (current eval vs improved eval)
-   - Ask: "Adopt the improved eval? This will affect future iterations."

package/skills/diagnose/SKILL.md

@@ -1,96 +0,0 @@
----
-name: harness-evolver:diagnose
-description: "Use when the user wants to understand why a specific harness version failed, investigate a regression, analyze trace data, or debug a low score. Also use when the user says 'why did v003 fail' or 'what went wrong'."
-argument-hint: "[version]"
-allowed-tools: [Read, Bash, Glob, Grep]
----
-
-# /harness-evolver:diagnose
-
-Deep analysis of a harness version's execution traces and scores.
-
-## Arguments
-
-- `version` — version to diagnose (e.g., `v003`). If not given, diagnose the worst or most recent regression.
-
-## Resolve Tool Path
-
-```bash
-TOOLS=$([ -d ".harness-evolver/tools" ] && echo ".harness-evolver/tools" || echo "$HOME/.harness-evolver/tools")
-```
-
-## What To Do
-
-### 1. Identify the Version
-
-If not specified, find the worst or most recent regression:
-
-```bash
-python3 $TOOLS/state.py show --base-dir .harness-evolver
-cat .harness-evolver/summary.json
-```
-
-### 2. Score Breakdown
-
-```bash
-cat .harness-evolver/harnesses/{version}/scores.json
-```
-
-Identify which tasks failed (`score: 0.0`) and which passed.
-
-### 3. Trace Analysis (failed tasks)
-
-For each failed task:
-
-```bash
-cat .harness-evolver/harnesses/{version}/traces/{task_id}/input.json
-cat .harness-evolver/harnesses/{version}/traces/{task_id}/output.json
-```
-
-Look for patterns: wrong format? wrong category? empty output? crash?
-
-### 4. Error Search
-
-```bash
-grep -r "error\|Error\|FAIL\|exception\|Traceback" .harness-evolver/harnesses/{version}/traces/
-cat .harness-evolver/harnesses/{version}/traces/stderr.log
-```
-
-### 5. Compare with Parent
-
-Read the proposal to find the parent version:
-
-```bash
-cat .harness-evolver/harnesses/{version}/proposal.md
-```
-
-Then diff:
-
-```bash
-diff .harness-evolver/harnesses/{parent}/harness.py .harness-evolver/harnesses/{version}/harness.py
-```
-
-### 6. LangSmith (if available)
-
-If `langsmith-cli` is installed and LangSmith is configured:
-
-```bash
-langsmith-cli --json runs list --project harness-evolver-{version} --failed --fields id,name,error,inputs
-langsmith-cli --json runs stats --project harness-evolver-{version}
-```
-
-### 7. Report
-
-```
-Diagnosis: v003 (score: 0.31) — REGRESSION from v001 (0.62)
-
-Root cause: Prompt template change broke JSON parsing
-  - 4/10 tasks returned malformed output
-  - stderr shows: json.JSONDecodeError on 4 tasks
-  - The change on line 42 removed the "Reply with ONLY..." instruction
-
-Affected tasks: task_002, task_005, task_007, task_010
-Unaffected tasks: task_001, task_003, task_004, task_006, task_008, task_009
-
-Recommendation: Revert the prompt change, keep the retry logic from v002
-```

package/skills/import-traces/SKILL.md

@@ -1,102 +0,0 @@
----
-name: harness-evolver:import-traces
-description: "Use when the user wants to import real production traces from LangSmith as test tasks, convert traces to eval tasks, enrich their eval set with real-world data, or pull production data into their harness evaluation."
-argument-hint: "[--project NAME] [--limit N]"
-allowed-tools: [Read, Write, Edit, Bash, Glob, Grep]
----
-
-# /harness-evolver:import-traces
-
-Import production traces from LangSmith and convert them into eval tasks. This enriches the test suite with real-world inputs, prioritizing traces with negative user feedback.
-
-## Prerequisites
-
-- `.harness-evolver/` must exist. If not, tell user to run `harness-evolver:init` first.
-- `langsmith-cli` must be available. Check:
-
-```bash
-which langsmith-cli 2>/dev/null
-```
-
-If not found: "Install langsmith-cli first: `uv tool install langsmith-cli && langsmith-cli auth login`"
-
-## Resolve Tool Path
-
-```bash
-TOOLS=$([ -d ".harness-evolver/tools" ] && echo ".harness-evolver/tools" || echo "$HOME/.harness-evolver/tools")
-```
-
-## Parse Arguments
-
-- `--project NAME` — LangSmith project name (if not provided, discover interactively)
-- `--limit N` — max traces to import (default: 20)
-
-## Phase 1: Discover Projects
-
-If `--project` not provided, list available projects:
-
-```bash
-langsmith-cli --json projects list --limit 20 2>/dev/null
-```
-
-Show the user a list of projects with run counts. Let them pick one, or use the most recent.
-
-If `--project` is provided, use it directly.
-
-## Phase 2: Fetch Traces
-
-```bash
-langsmith-cli --json runs list \
-    --project "{project_name}" \
-    --limit {limit} \
-    --fields id,name,inputs,outputs,error,feedback_stats,total_tokens \
-    > /tmp/harness_import_traces.json 2>/dev/null
-```
-
-Check the output has data:
-```bash
-python3 -c "import json; data=json.load(open('/tmp/harness_import_traces.json')); print(f'{len(data)} traces fetched')"
-```
-
-If no traces found, tell user the project may be empty or the name may be wrong.
-
-## Phase 3: Convert to Tasks
-
-```bash
-python3 $TOOLS/import_traces.py \
-    --traces-json /tmp/harness_import_traces.json \
-    --output-dir .harness-evolver/eval/tasks/ \
-    --prefix imported \
-    --max-tasks {limit}
-```
-
-## Phase 4: Report
-
-Read the tool output and report:
-- How many traces were imported
-- How many had negative feedback (high priority)
-- How many were skipped (no extractable input, duplicates)
-- Total tasks now in eval set
-
-```bash
-ls .harness-evolver/eval/tasks/*.json | wc -l
-```
-
-Print:
-```
-Imported {N} production traces as eval tasks.
-  {M} with negative user feedback (high priority)
-  {K} skipped (no input or duplicates)
-  Total eval tasks: {total}
-
-Next: run `harness-evolver:evolve` to optimize against real-world inputs.
-```
-
-## Gotchas
-
-- Traces with no extractable user input are skipped (e.g., system-only runs)
-- Duplicate traces (same run ID) are automatically skipped
-- Imported tasks are tagged with `metadata.source: "imported"` and `metadata.type: "production"`
-- Tasks with negative feedback get `metadata.user_feedback: "negative"` — the proposer should prioritize these
-- The `metadata.langsmith_run_id` field links back to the original trace for debugging
-- Cleanup: `rm /tmp/harness_import_traces.json` after import

package/skills/init/SKILL.md

@@ -1,253 +0,0 @@
----
-name: harness-evolver:init
-description: "Use when the user wants to set up harness optimization in their project, optimize an LLM agent, improve a harness, or mentions harness-evolver for the first time in a project without .harness-evolver/ directory."
-argument-hint: "[directory]"
-allowed-tools: [Read, Write, Edit, Bash, Glob, Grep, Agent, AskUserQuestion]
----
-
-# /harness-evolve-init
-
-Set up the Harness Evolver in a project. Scans the codebase, identifies the entry point, creates missing artifacts, runs baseline evaluation.
-
-## Resolve Tool Path
-
-```bash
-TOOLS=$([ -d ".harness-evolver/tools" ] && echo ".harness-evolver/tools" || echo "$HOME/.harness-evolver/tools")
-```
-
-Use `$TOOLS` prefix for all tool calls below.
-
-## Phase 1: Scan
-
-```bash
-find . -maxdepth 3 -type f -name "*.py" | head -30
-python3 $TOOLS/detect_stack.py .
-```
-
-Look for:
-- Entry points: files with `if __name__`, or named `main.py`, `app.py`, `agent.py`, `graph.py`, `pipeline.py`, `bot.py`
-- Existing eval: `eval.py`, `score.py`, `judge.py`
-- Existing tasks: directories with JSON files containing `id` + `input` fields
-- Config: `config.json`, `config.yaml`, `.env`
-
-## Phase 1.5: Confirm Detection (Interactive)
-
-After scanning, present what was found and ask the user to confirm before proceeding.
-
-Use AskUserQuestion:
-
-```
-Question: "Here's what I detected. Does this look right?"
-Header: "Confirm"
-Options:
-  - "Looks good, proceed" — Continue with detected paths
-  - "Let me adjust paths" — User will provide correct paths
-  - "Start over in different directory" — Abort and let user cd elsewhere
-
-Show in the question description:
-  - Harness: {path or "not found"}
-  - Eval: {path or "not found — will use LLM-as-judge"}
-  - Tasks: {path with N files, or "not found — will generate"}
-  - Stack: {detected frameworks or "none detected"}
-  - Architecture: {topology or "unknown"}
-```
-
-If user chose "Let me adjust paths", ask which paths to change and update accordingly.
-
-## Phase 1.8: Eval Mode (Interactive — only if NO eval found)
-
-If no eval.py was detected, ask the user which evaluation mode to use:
-
-```
-Question: "No eval script found. How should outputs be scored?"
-Header: "Eval mode"
-Options:
-  - "LLM-as-judge (zero-config)" — Claude Code scores outputs on accuracy, completeness, relevance, hallucination. No expected answers needed.
-  - "Keyword matching" — Simple string matching against expected answers. Requires 'expected' field in tasks.
-  - "I'll provide my own eval.py" — Pause and let user create their eval script.
-```
-
-If "LLM-as-judge": copy eval_passthrough.py as eval.py.
-If "Keyword matching": create a simple keyword eval (check if expected substrings appear in output).
-If "I'll provide my own": print instructions for the eval contract and wait.
-
-## Phase 1.9: LangSmith Project (Interactive — only if LANGSMITH_API_KEY detected)
-
-If a LangSmith API key is available, discover projects and ask which one has production traces:
-
-```bash
-langsmith-cli --json projects list --limit 10 2>/dev/null
-```
-
-Use AskUserQuestion:
-```
-Question: "LangSmith detected. Which project has your production traces?"
-Header: "LangSmith"
-Options: (build from discovered projects — pick top 3-4 by recent activity)
-  - "{project_name_1}" — {run_count} runs, last active {date}
-  - "{project_name_2}" — {run_count} runs, last active {date}
-  - "{project_name_3}" — {run_count} runs, last active {date}
-  - "Skip — don't use production traces" — Proceed without production data
-```
-
-If a project is selected, pass it as `--langsmith-project` to init.py.
-
-## Phase 2: Create What's Missing
-
-Three artifacts needed. For each — use existing if found, create if not.
-
-**Harness** (`harness.py`): If user's entry point doesn't match our CLI interface (`--input`, `--output`, `--traces-dir`, `--config`), create a thin wrapper that imports their code. Read their entry point first to understand the I/O format. Ask if unsure.
-
-**Eval** (`eval.py`): If an eval script exists, use it. If the user already chose an eval mode in Phase 1.8, follow that choice.
-
-If NO eval exists and no mode was chosen yet:
-- Copy `eval_passthrough.py` from `$TOOLS/eval_passthrough.py` as the project's eval.py:
-  ```bash
-  cp $TOOLS/eval_passthrough.py eval.py
-  ```
-- This passthrough eval collects outputs for the judge subagent to score during evolve.
-- Print: "No eval found. Using LLM-as-judge (Claude Code scores outputs directly)."
-
-**Tasks** (`tasks/`): If test tasks exist, use them.
-
-If NO tasks exist, generate them. First, identify all relevant source files:
-
-```bash
-find . -name "*.py" -not -path "./.venv/*" -not -path "./.harness-evolver/*" | head -10
-find . -name "*.json" -o -name "*.md" -o -name "*.txt" -o -name "*.yaml" -o -name "*.yml" | grep -v .venv | grep -v .harness-evolver | head -10
-```
-
-Then spawn testgen subagent with CONCRETE file paths (not placeholders):
-
-```
-Agent(
-  subagent_type: "harness-evolver-testgen",
-  description: "TestGen: generate 30 test cases",
-  prompt: |
-    <objective>
-    Generate 30 diverse test cases for this project. Write them to the tasks/ directory
-    in the current working directory.
-    </objective>
-
-    <project_context>
-    This project is at: {absolute path to project root}
-    Entry point: {the harness/agent file you identified, e.g., crew.py or pipeline/moderator.py}
-    Framework: {what you detected — CrewAI, LangGraph, etc.}
-    </project_context>
-
-    <files_to_read>
-    {LIST EVERY .py file and data file you found above — use ABSOLUTE PATHS}
-    Example:
-    - /home/rp/Desktop/test-crewai/crew.py
-    - /home/rp/Desktop/test-crewai/README.md
-    </files_to_read>
-
-    <production_traces>
-    {IF .harness-evolver/production_seed.md EXISTS, paste its full contents here.
-     This file contains real production inputs, traffic distribution, error patterns,
-     and user feedback from LangSmith. Use it to generate REALISTIC test cases that
-     match actual usage patterns instead of synthetic ones.
-
-     If the file does not exist, omit this entire block.}
-    </production_traces>
-
-    <output>
-    Create directory tasks/ (at project root) with 30 files: task_001.json through task_030.json.
-    Format: {"id": "task_001", "input": "...", "metadata": {"difficulty": "easy|medium|hard", "type": "standard|edge|cross_domain|adversarial"}}
-    No "expected" field needed — the judge subagent will score outputs.
-    Distribution: 40% standard, 20% edge, 20% cross-domain, 20% adversarial.
-    If production traces are available, match the real traffic distribution instead of uniform.
-    </output>
-)
-```
-
-Wait for `## TESTGEN COMPLETE`. If the subagent fails or returns with no tasks, generate them yourself inline (fallback).
-
-Print: "Generated {N} test cases from code analysis."
-
-If `.harness-evolver/production_seed.md` exists, also print:
-"Tasks enriched with production trace data from LangSmith."
-
-## Phase 3: Run Init
-
-First, check if the project has a LangSmith production project configured:
-
-```bash
-# Auto-detect from env vars or .env
-PROD_PROJECT=$(python3 -c "
-import os
-for v in ('LANGCHAIN_PROJECT', 'LANGSMITH_PROJECT'):
-    p = os.environ.get(v, '')
-    if p: print(p); exit()
-for f in ('.env', '.env.local'):
-    if os.path.exists(f):
-        for line in open(f):
-            line = line.strip()
-            if '=' in line and not line.startswith('#'):
-                k, _, val = line.partition('=')
-                if k.strip() in ('LANGCHAIN_PROJECT', 'LANGSMITH_PROJECT'):
-                    print(val.strip().strip('\"').strip(\"'\"))
-                    exit()
-" 2>/dev/null)
-```
-
-```bash
-python3 $TOOLS/init.py [directory] \
-    --harness harness.py --eval eval.py --tasks tasks/ \
-    --tools-dir $TOOLS \
-    ${PROD_PROJECT:+--langsmith-project "$PROD_PROJECT"}
-```
-
-Add `--harness-config config.json` if a config exists.
-
-For **LLM-powered agents** that make real API calls (LangGraph, CrewAI, etc.) and take
-more than 30 seconds per invocation, increase the validation timeout:
-
-```bash
-python3 $TOOLS/init.py [directory] \
-    --harness harness.py --eval eval.py --tasks tasks/ \
-    --tools-dir $TOOLS \
-    --validation-timeout 120
-```
-
-If validation keeps timing out but you've verified the harness works manually, skip it:
-
-```bash
-python3 $TOOLS/init.py [directory] \
-    --harness harness.py --eval eval.py --tasks tasks/ \
-    --tools-dir $TOOLS \
-    --skip-validation
-```
-
-## After Init — Report
-
-- What was detected vs created
-- Stack + integrations (LangSmith, Context7)
-- Baseline score
-- Next: `harness-evolver:evolve` to start
-
-## Architecture Hint
-
-After init completes, run a quick architecture analysis:
-
-```bash
-python3 $TOOLS/analyze_architecture.py --harness .harness-evolver/baseline/harness.py
-```
-
-If the analysis suggests the current topology may not be optimal for the task complexity, mention it:
-
-> Architecture note: Current topology is "{topology}". For tasks with {characteristics},
-> consider running `/harness-evolver:architect` for a detailed recommendation.
-
-This is advisory only — do not spawn the architect agent.
-
-## Gotchas
-
-- The harness must write valid JSON to `--output`. If the user's code returns non-JSON, the wrapper must serialize it.
-- Tasks must have unique `id` fields. Duplicate IDs cause silent eval errors.
-- The `expected` field is never shown to the harness — only the eval script sees it.
-- If `.harness-evolver/` already exists, warn before overwriting.
-- If no Python files exist in CWD, the user is probably in the wrong directory.
-- **Monorepo / venv mismatch**: In monorepos with dedicated venvs per app, the system `python3` may differ from the project's Python version. The harness wrapper should re-exec with the correct venv Python. The tools now use `sys.executable` instead of hardcoded `python3`.
-- **Stale site-packages**: If the project uses editable installs (`pip install -e .`), packages in `site-packages/` may have stale copies of data files (e.g. registry YAMLs). Run `uv pip install -e . --force-reinstall --no-deps` to sync.
-- **Validation timeout**: LLM agents making real API calls typically take 15-60s per invocation. Use `--validation-timeout 120` or `--skip-validation` to handle this.