claude-turing 1.1.0 → 1.3.0

package/.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "turing",
-  "version": "1.1.0",
-  "description": "Autonomous ML research harness — the autoresearch loop as a formal protocol. 16 commands, 2 specialized agents, cost-performance frontier analysis, model cards, model registry, hypothesis database with novelty guard, anti-cheating guardrails, and the taste-leverage loop. Inspired by Karpathy's autoresearch and the scientific method itself.",
+  "version": "1.3.0",
+  "description": "Autonomous ML research harness — the autoresearch loop as a formal protocol. 19 commands, 2 specialized agents, statistical rigor (multi-seed studies, reproducibility verification), tree-search hypothesis exploration (TreeQuest AB-MCTS), cost-performance frontier analysis, model cards, model registry, hypothesis database with novelty guard, anti-cheating guardrails, and the taste-leverage loop. Inspired by Karpathy's autoresearch and the scientific method itself.",
   "author": {
     "name": "pragnition"
   },

package/README.md

@@ -313,6 +313,8 @@ The index (`hypotheses.yaml`) is the lightweight queue. The detail files (`hypot
 | `/turing:try <hypothesis>` | Inject a hypothesis — free text or `archetype:model_comparison` |
 | `/turing:brief [--deep]` | Research briefing — campaign summary, failure patterns, literature-grounded suggestions |
 | `/turing:suggest` | Literature-grounded model architecture suggestions with citations |
+| `/turing:suggest --strategy treequest` | Tree-search hypothesis exploration (alias for `/turing:explore`) |
+| `/turing:explore` | AB-MCTS tree search over critique-scored hypothesis space |
 | `/turing:design <hyp-id>` | Generate structured experiment design from a hypothesis |
 | `/turing:mode <explore\|exploit\|replicate>` | Set research strategy — drives novelty guard policy |
 
@@ -321,6 +323,8 @@ The index (`hypotheses.yaml`) is the lightweight queue. The detail files (`hypot
 | Command | What it does |
 |---------|-------------|
 | `/turing:validate [--auto]` | Check metric stability — auto-configure multi-run if noisy |
+| `/turing:seed [N] [--quick]` | Multi-seed study — mean/std/CI, flag seed-sensitive results |
+| `/turing:reproduce <exp-id>` | Reproducibility verification — re-run and check tolerance |
 | `/turing:card` | Generate a model card — performance, limitations, intended use, artifact contract |
 | `/turing:logbook` | Generate HTML experiment logbook |
 | `/turing:report` | Generate research report |
@@ -390,6 +394,65 @@ After N experiments with no meaningful improvement, the agent stops and reports
 
 For noisy metrics, `/turing:validate` runs the pipeline multiple times and measures variance. If the coefficient of variation exceeds 5%, it auto-configures multi-run evaluation so the agent can't be rewarded for lucky single runs.
 
+## Statistical Rigor
+
+> *"Stop publishing lucky seeds. Start publishing distributions."*
+
+Before claiming a result, run a seed study:
+
+```
+/turing:seed              # 5 seeds on best experiment
+/turing:seed --quick      # 3 seeds for fast check
+/turing:seed 10           # 10 seeds for thorough study
+```
+
+This runs the same experiment across multiple random seeds and reports mean +/- std with 95% confidence intervals. If the coefficient of variation exceeds 5%, the result is flagged as **seed-sensitive** — meaning you should report the distribution, not a single number.
+
+To verify an experiment can be reproduced:
+
+```
+/turing:reproduce exp-042                  # Default: 3 runs, 2% tolerance
+/turing:reproduce exp-042 --strict         # Exact match required
+/turing:reproduce exp-042 --tolerance 0.05 # Custom tolerance
+```
+
+This re-runs the experiment from the logged config and checks that metrics fall within tolerance. It also detects environment drift — if library versions have changed since the original run, you'll know before a reviewer tells you.
+
+Seed study results automatically appear in `/turing:brief` and `/turing:card`.
+
+## Tree-Search Hypothesis Exploration
+
+> *"The learned coin-flipper weaves through the quadrillion-coin room with a preternatural air."*
+
+Sometimes the best experiment to try next isn't obvious from the literature or the agent's memory. `/turing:explore` uses [TreeQuest](https://github.com/SakanaAI/treequest)'s AB-MCTS (Adaptive Branching Monte Carlo Tree Search) to search the space of experiment *ideas* as a tree, scored by the critique engine (novelty x feasibility x impact).
+
+```
+/turing:explore                         # Run MCTS over hypothesis space
+/turing:explore --strategy greedy       # Greedy fallback (no TreeQuest needed)
+/turing:explore --iterations 50 --top 8 # Deeper search, more results
+/turing:suggest --strategy treequest    # Same thing via suggest
+```
+
+How it works:
+
+```
+         Seeds                    MCTS expands best-scoring branches
+           │
+    ┌──────┼──────┐               Each node is a hypothesis scored by:
+    ▼      ▼      ▼                 - Novelty (vs experiment history)
+  LightGBM Reg  Features            - Feasibility (hardware, deps)
+    │       │      │                - Expected impact (type success rate)
+    ▼       ▼      ▼
+  +dart   +L1   +poly             Top-K results queued as hypotheses
+    │              │              for the next /turing:train run
+    ▼              ▼
+  +subsamp      +target-enc
+```
+
+Unlike `/turing:suggest` (which searches the web for papers), `/turing:explore` searches the space of *refinement chains* — combinations and sequences of modifications that score well together. It discovers non-obvious experiment strategies that independent suggestions cannot find.
+
+Falls back to greedy best-first search when TreeQuest is not installed.
+
 ## Cost-Performance Frontier
 
 > *"This model is 2% better but takes 10x longer to train. Is that worth it?"*
@@ -451,11 +514,11 @@ Each project gets independent config, data, experiments, models, and agent memor
 
 ## Architecture of Turing Itself
 
-16 commands, 2 agents, 8 config files, 30 template scripts, model registry, artifact contract, cost-performance frontier, model cards, 345 tests, 16 ADRs. See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the full codemap.
+19 commands, 2 agents, 8 config files, 34 template scripts, model registry, artifact contract, cost-performance frontier, model cards, tree-search exploration, statistical rigor (seed studies + reproducibility), 407 tests, 16 ADRs. See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the full codemap.
 
 ```
 turing/
-├── commands/              15 skill files (core + taste-leverage + reporting)
+├── commands/              18 skill files (core + taste-leverage + reporting + exploration + statistical rigor)
 ├── agents/                2 agents (researcher: read/write, evaluator: read-only)
 ├── config/                8 files (lifecycle, taxonomy, archetypes, novelty aliases)
 ├── templates/             Scaffolded into user projects by /turing:init
@@ -464,7 +527,7 @@ turing/
 │   ├── train.py           Training code (AGENT-EDITABLE)
 │   ├── model_contract.md  Artifact schema for production consumers
 │   ├── model_registry.yaml  Available model architectures + hyperparams
-│   └── scripts/           25 Python scripts (core loop + analysis + infra)
+│   └── scripts/           26 Python scripts (core loop + analysis + infra + tree search)
 ├── tests/                 338 tests (unit + integration + anti-pattern + manifest)
 ├── src/                   5 JS installer files (npm deployment)
 ├── bin/                   CLI entry points
@@ -482,6 +545,7 @@ turing/
 - **[Principle of Least Privilege](https://en.wikipedia.org/wiki/Principle_of_least_privilege)** (Saltzer & Schroeder, 1975) — each agent has exactly the capabilities needed for its role
 - **[Early Stopping](https://en.wikipedia.org/wiki/Early_stopping)** (Prechelt, 1998) — convergence detection as discrete early stopping
 - **[Multi-Armed Bandits](https://en.wikipedia.org/wiki/Multi-armed_bandit)** — the explore-exploit tradeoff
+- **[TreeQuest](https://github.com/SakanaAI/treequest)** (Sakana AI, 2025) — AB-MCTS for inference-time scaling; repurposed here for hypothesis-space exploration
 - **[Version Control as Lab Notebook](https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1004668)** (Ram, 2013) — git as a scientific record-keeping system
 - **[Reproducibility Crisis](https://en.wikipedia.org/wiki/Replication_crisis)** — if the measurement can change between experiments, results are not reproducible
 

package/commands/explore.md

@@ -0,0 +1,107 @@
+---
+name: explore
+description: Tree-search-guided hypothesis exploration using AB-MCTS. Explores the space of experiment ideas as a search tree, scored by the critique engine. Discovers non-obvious refinement chains that linear suggestion cannot find.
+disable-model-invocation: true
+argument-hint: "[ml/project] [--iterations N] [--top N] [--strategy abmcts-a|abmcts-m|greedy]"
+allowed-tools: Read, Write, Bash(python scripts/*:*, source .venv/bin/activate:*), Grep, Glob
+---
+
+Explore the hypothesis space using tree search. Instead of suggesting independent ideas, this builds and searches a tree of refinement chains — each node is a hypothesis scored by novelty, feasibility, and expected impact.
+
+## Project Detection
+
+0. **Detect project directory:**
+   - If `$ARGUMENTS` contains a path (e.g., `ml/coding`), use that as the project directory
+   - Else if cwd contains `config.yaml` and `train.py`, use cwd
+   - Else search for `ml/*/` subdirectories containing `config.yaml`
+     - If exactly one found, use it
+     - If multiple found, list them and ask the user which to target
+   - All subsequent commands run from the detected project directory
+
+## Parse Options
+
+Extract from `$ARGUMENTS`:
+- `--iterations N` — search depth (default: 30)
+- `--top N` — number of results to return (default: 5)
+- `--strategy` — algorithm choice: `abmcts-a` (default), `abmcts-m` (Bayesian), or `greedy` (no TreeQuest needed)
+- `--seeds-only` — just show generated seeds without running search
+- `--json` — output as JSON for programmatic use
+
+## Steps
+
+### 1. Assess Current State
+
+```bash
+source .venv/bin/activate && python scripts/show_metrics.py --last 10 2>/dev/null || echo "No experiments yet"
+```
+
+Read `config.yaml` to understand the current model and metric.
+
+### 2. Run Tree Search
+
+```bash
+source .venv/bin/activate && python scripts/treequest_suggest.py \
+    --log experiments/log.jsonl \
+    --config config.yaml \
+    --top <N> \
+    --iterations <N> \
+    --strategy <strategy>
+```
+
+The script will:
+- Generate seed hypotheses from config and experiment history
+- Run AB-MCTS (or greedy fallback) over the hypothesis tree
+- Score each node using the critique engine
+- Return top-K ranked, deduplicated hypotheses
+
+### 3. Queue Best Hypotheses
+
+For each result, add to the hypothesis queue:
+
+```bash
+source .venv/bin/activate && python scripts/manage_hypotheses.py add "<description>" \
+    --priority medium --source treequest
+```
+
+### 4. Show Results
+
+Display the search output and confirm queuing:
+
+```
+TreeQuest Hypothesis Exploration (AB-MCTS-A)
+============================================
+Nodes explored: 35
+Top 5 hypotheses by critique score:
+
+  1. [PROCEED] (score: 7.8/10)
+     Switch to LightGBM with dart boosting; additionally add polynomial features
+     Novelty: 8  Feasibility: 9  Impact: 7
+     -> Queued as hyp-NNN
+
+  2. [PROCEED] (score: 7.2/10)
+     Use low learning rate (0.01) with 2000 estimators; additionally add L2 regularization
+     Novelty: 7  Feasibility: 8  Impact: 7
+     Depth: 1 (refined from parent)
+     -> Queued as hyp-NNN
+
+  ...
+
+Queued N hypotheses. Run /turing:train to test them.
+```
+
+## How It Differs From /turing:suggest
+
+| | `/turing:suggest` | `/turing:explore` |
+|---|---|---|
+| **Source** | Web literature search | Tree search over critique scores |
+| **Strategy** | Independent suggestions | Refinement chains (parent -> child) |
+| **Requires internet** | Yes | No |
+| **Discovers** | What papers recommend | What combinations score well |
+| **Best for** | Early-stage exploration | Mid-experiment optimization |
+
+## Integration
+
+- Results feed into `hypotheses.yaml` — the next `/turing:train` picks them up
+- `/turing:brief` shows queued treequest-sourced hypotheses
+- `/turing:suggest --strategy treequest` is an alias for this command
+- Human can override priority: `/turing:try` always takes precedence

package/commands/reproduce.md

@@ -0,0 +1,48 @@
+---
+name: reproduce
+description: Verify reproducibility of a specific experiment by re-running from logged config and checking metrics fall within tolerance.
+disable-model-invocation: true
+argument-hint: "<exp-id> [--tolerance 0.02] [--strict] [--runs 3]"
+allowed-tools: Read, Bash(*), Grep, Glob
+---
+
+Verify that a logged experiment can be reproduced with consistent results.
+
+## Steps
+
+1. **Activate environment:**
+   ```bash
+   source .venv/bin/activate
+   ```
+
+2. **Parse arguments from `$ARGUMENTS`:**
+   - First argument is the experiment ID (required), e.g. `exp-042`
+   - `--tolerance 0.02` sets the relative tolerance (default 2%)
+   - `--strict` requires exact float match (1e-6), overrides tolerance
+   - `--runs 3` sets number of reproduction runs (default 3, 1 for strict)
+
+3. **Run reproducibility verification:**
+   ```bash
+   python scripts/reproduce_experiment.py $ARGUMENTS
+   ```
+
+4. **Report results:**
+   - **reproducible:** metrics match exactly (deterministic algorithm)
+   - **approximately_reproducible:** metrics within tolerance or original falls in 95% CI
+   - **not_reproducible:** metrics outside tolerance and CI
+   - **environment_changed:** metrics diverge AND library versions differ
+   - Show environment diff if present (Python version, package versions)
+
+5. **Saved output:** report written to `experiments/reproductions/exp-NNN-repro.yaml`
+
+6. **If experiment ID not found:** list available experiment IDs from `experiments/log.jsonl`
+
+7. **If no training pipeline exists:** suggest `/turing:init` first.
+
+## Examples
+
+```
+/turing:reproduce exp-042                 # Default: 3 runs, 2% tolerance
+/turing:reproduce exp-042 --strict        # Exact match required
+/turing:reproduce exp-042 --tolerance 0.05 --runs 5   # Lenient, more runs
+```

package/commands/seed.md

@@ -0,0 +1,47 @@
+---
+name: seed
+description: Run multi-seed study on an experiment to compute mean/std/CI and flag seed-sensitive results. Prevents publishing lucky seeds.
+disable-model-invocation: true
+argument-hint: "[N] [--quick] [--exp-id <id>]"
+allowed-tools: Read, Bash(*), Grep, Glob
+---
+
+Run a multi-seed study to verify that experiment results are robust across random seeds.
+
+## Steps
+
+1. **Activate environment:**
+   ```bash
+   source .venv/bin/activate
+   ```
+
+2. **Parse arguments from `$ARGUMENTS`:**
+   - A bare number (e.g., `5`) sets the seed count
+   - `--quick` runs 3 seeds instead of 5
+   - `--exp-id exp-042` targets a specific experiment (defaults to best)
+   - `--seed-list 42,123,456` uses specific seed values
+
+3. **Run seed study:**
+   ```bash
+   python scripts/seed_runner.py $ARGUMENTS
+   ```
+
+4. **Report results:**
+   - Show the per-seed results table
+   - Show mean +/- std with 95% CI
+   - **STABLE (CV < 5%):** result is robust, safe to report
+   - **SEED-SENSITIVE (CV >= 5%):** result varies too much across seeds — do not report single-seed numbers
+   - If seed-sensitive, recommend reporting as mean +/- std over N seeds
+
+5. **Saved output:** results are written to `experiments/seed_studies/exp-NNN-seeds.yaml`
+
+6. **If no training pipeline exists:** suggest `/turing:init` first.
+
+## Examples
+
+```
+/turing:seed              # 5 seeds on best experiment
+/turing:seed --quick      # 3 seeds for fast check
+/turing:seed 10           # 10 seeds for thorough study
+/turing:seed --exp-id exp-042   # Specific experiment
+```

package/commands/suggest.md

@@ -6,9 +6,16 @@ argument-hint: "[task description override]"
 allowed-tools: Read, Write, Bash(python scripts/*:*, source .venv/bin/activate:*), Grep, Glob, WebSearch, WebFetch
 ---
 
-Suggest model architectures for the current ML task, grounded in recent literature. Hypotheses backed by papers, not vibes.
+Suggest model architectures for the current ML task. Supports two strategies:
 
-## Steps
+- **literature** (default): Web search for recent papers, synthesize grounded suggestions with citations.
+- **treequest**: Tree-search-guided hypothesis exploration using AB-MCTS over the critique scoring function. Explores refinement chains that literature search cannot find.
+
+## Strategy Detection
+
+If `$ARGUMENTS` contains `--strategy treequest` or `treequest`, use the TreeQuest strategy below. Otherwise use the default literature strategy.
+
+## Steps (Literature Strategy — default)
 
 ### 1. Understand the Task
 
@@ -84,12 +91,69 @@ Sources consulted: <N papers/articles>
 Queued N hypotheses. Run /turing:train to test them.
 ```
 
-## Fallback
+## Fallback (Literature Strategy)
 
 If web search returns insufficient results, suggest model families from `config/taxonomy.toml` based on what hasn't been tried yet. Note that suggestions are taxonomy-based, not literature-backed, and queue with `--source taxonomy`.
 
+## Steps (TreeQuest Strategy)
+
+When using `--strategy treequest`:
+
+### 1. Detect Project Directory
+
+Same detection logic as the literature strategy — find `config.yaml` + `train.py`.
+
+### 2. Run Tree Search
+
+```bash
+source .venv/bin/activate && python scripts/treequest_suggest.py \
+    --log experiments/log.jsonl \
+    --config config.yaml \
+    --top 5 \
+    --iterations 30 \
+    --strategy abmcts-a
+```
+
+If TreeQuest is not installed, the script automatically falls back to greedy best-first search.
+
+### 3. Queue Results
+
+For each result from the tree search, queue as a hypothesis:
+
+```bash
+source .venv/bin/activate && python scripts/manage_hypotheses.py add "<description>" --priority medium --source treequest
+```
+
+### 4. Show Results
+
+Display the tree search output and confirm hypotheses were queued:
+
+```
+TreeQuest Hypothesis Exploration (AB-MCTS-A)
+============================================
+Nodes explored: 35
+Top 5 hypotheses by critique score:
+
+  1. [PROCEED] (score: 7.8/10)
+     Switch to LightGBM with dart boosting; additionally add polynomial features
+     Novelty: 8  Feasibility: 9  Impact: 7
+
+  ...
+
+Queued N hypotheses. Run /turing:train to test them.
+```
+
+### TreeQuest Options
+
+Pass additional flags via `$ARGUMENTS`:
+- `--iterations N` — search depth (default: 30)
+- `--top N` — number of results (default: 5)
+- `--strategy abmcts-m` — use Bayesian mixed model variant (requires PyMC)
+- `--greedy` — force greedy fallback without TreeQuest
+
 ## Integration
 
 - Suggestions feed into `hypotheses.yaml` — the next `/turing:train` picks them up
-- `/turing:brief` shows queued literature-sourced hypotheses
+- `/turing:brief` shows queued literature-sourced and treequest-sourced hypotheses
+- `/turing:explore` runs the TreeQuest search as a standalone command
 - Human can override priority: `/turing:try` always takes precedence

package/commands/turing.md

@@ -20,7 +20,10 @@ You are the Turing ML research router. Detect the user's intent and route to the
 | "poster", "presentation", "one-pager", "visual summary" | `/turing:poster` | Document |
 | "report", "write-up", "findings", "document results" | `/turing:report` | Document |
 | "validate", "stability", "check variance", "noisy" | `/turing:validate` | Validate |
+| "seed", "seed study", "multi-seed", "lucky seed", "seed sensitivity" | `/turing:seed` | Validate |
+| "reproduce", "reproducibility", "verify results", "re-run experiment", "repro" | `/turing:reproduce` | Validate |
 | "suggest", "what model", "recommend", "which architecture", "literature" | `/turing:suggest` | Research |
+| "explore hypotheses", "tree search", "treequest", "search hypothesis space", "MCTS" | `/turing:explore` | Research |
 | "design", "plan experiment", "how should I test", "experiment design" | `/turing:design` | Design |
 | "mode", "explore", "exploit", "replicate", "strategy" | `/turing:mode` | Strategy |
 | "preflight", "resources", "VRAM", "memory", "can I run", "OOM", "GPU" | `/turing:preflight` | Check |
@@ -38,7 +41,10 @@ You are the Turing ML research router. Detect the user's intent and route to the
 | `/turing:brief` | Generate structured research intelligence report | @ml-evaluator |
 | `/turing:init` | Scaffold a new ML project | (inline) |
 | `/turing:validate` | Check metric stability, auto-fix if noisy | (inline) |
+| `/turing:seed [N] [--quick]` | Multi-seed study: mean/std/CI, flag seed-sensitive results | (inline) |
+| `/turing:reproduce <exp-id>` | Reproducibility verification with tolerance checking | (inline) |
 | `/turing:suggest` | Literature-grounded model architecture suggestions | (inline, uses WebSearch) |
+| `/turing:explore` | Tree-search hypothesis exploration via AB-MCTS | (inline) |
 | `/turing:design <hyp-id>` | Generate structured experiment design from hypothesis | (inline, uses WebSearch) |
 | `/turing:logbook` | HTML/markdown logbook with trajectory chart | (inline) |
 | `/turing:poster` | Single-page HTML research poster | (inline) |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "claude-turing",
-  "version": "1.1.0",
+  "version": "1.3.0",
   "type": "module",
   "description": "Autonomous ML research harness for Claude Code. The autoresearch loop as a formal protocol — iteratively trains, evaluates, and improves ML models with structured experiment tracking, convergence detection, immutable evaluation infrastructure, and safety guardrails.",
   "bin": {

package/src/claude-md.js

@@ -21,6 +21,7 @@ Autonomous ML research harness. The autoresearch loop as a formal protocol.
 | \`/turing:validate\` | Check metric stability, auto-fix if noisy |
 | \`/turing:try <hypothesis>\` | Inject a hypothesis into the experiment queue |
 | \`/turing:brief\` | Generate research intelligence report |
+| \`/turing:explore\` | Tree-search hypothesis exploration (AB-MCTS) |
 | \`/turing:preflight\` | Pre-flight resource check (VRAM/RAM/disk) |
 
 ### Agents

package/src/install.js

@@ -21,8 +21,8 @@ const PLUGIN_ROOT = join(__dirname, "..");
 // Single source of truth for sub-commands (DRY — used for dirs and file copy)
 const SUB_COMMANDS = [
   "init", "train", "status", "compare", "sweep", "validate",
-  "try", "brief", "suggest", "design", "logbook", "poster",
-  "report", "mode", "preflight", "card",
+  "try", "brief", "suggest", "explore", "design", "logbook", "poster",
+  "report", "mode", "preflight", "card", "seed", "reproduce",
 ];
 
 export async function install(opts = {}) {

package/src/verify.js

@@ -23,6 +23,7 @@ const EXPECTED_COMMANDS = [
   "try/SKILL.md",
   "brief/SKILL.md",
   "suggest/SKILL.md",
+  "explore/SKILL.md",
   "design/SKILL.md",
   "logbook/SKILL.md",
   "poster/SKILL.md",
@@ -30,6 +31,8 @@ const EXPECTED_COMMANDS = [
   "mode/SKILL.md",
   "preflight/SKILL.md",
   "card/SKILL.md",
+  "seed/SKILL.md",
+  "reproduce/SKILL.md",
 ];
 
 const EXPECTED_AGENTS = ["ml-researcher.md", "ml-evaluator.md"];

package/templates/config.yaml

@@ -19,6 +19,16 @@ evaluation:
   # Set to false for metrics where higher is better (accuracy, f1, auc)
   lower_is_better: false  # {{METRIC_DIRECTION}} -- change to true if lower is better
 
+  # Multi-seed configuration (Phase 10.1: /turing:seed)
+  # Seeds used for seed studies — diverse values for good coverage
+  seed_seeds: [42, 123, 456, 789, 1024, 1337, 2048, 3141, 4096, 7919]
+  seed_study_n_runs: 5        # Default number of seeds for /turing:seed
+  seed_sensitivity_threshold: 5.0  # CV% above this = seed-sensitive
+
+  # Reproducibility configuration (Phase 10.2: /turing:reproduce)
+  reproduce_tolerance: 0.02   # 2% relative tolerance for approximate match
+  reproduce_n_runs: 3         # Default reproduction runs for stochastic algorithms
+
 convergence:
   patience: 3                    # Consecutive non-improvements before stopping
   improvement_threshold: 0.005   # 0.5% relative improvement required

package/templates/program.md

@@ -170,6 +170,11 @@ The autoresearch experiment loop. Each iteration is one experiment — one hypot
    - N consecutive non-improvements (`config.yaml` → `convergence.patience`) = STOP
    - `max_iterations` reached = STOP
    - Report final best model and recommend next steps
+   - **Before declaring final results**, run a seed study to verify robustness:
+     ```bash
+     python scripts/seed_runner.py --quick
+     ```
+     If CV > 5%, the result is seed-sensitive — report mean ± std, not a single-seed number.
 
 10. **REPEAT** — return to step 1.
 

package/templates/requirements.txt

@@ -6,3 +6,7 @@ numpy>=2.0
 joblib>=1.4
 pyyaml>=6.0
 pytest>=8.0
+
+# Optional: tree-search-guided hypothesis exploration
+# Install with: pip install "treequest[all]"
+# treequest>=0.1