claude-turing 2.0.0 → 2.1.0

package/.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "turing",
-  "version": "2.0.0",
-  "description": "Autonomous ML research harness — the autoresearch loop as a formal protocol. 25 commands, 2 specialized agents, production model export (6 formats, equivalence verification, latency benchmarking), performance profiling, smart Pareto-based checkpoint management, experiment intelligence (error analysis, ablation studies, Pareto frontiers), statistical rigor (multi-seed studies, reproducibility verification), tree-search hypothesis exploration (TreeQuest AB-MCTS), cost-performance frontier analysis, model cards, model registry, hypothesis database with novelty guard, anti-cheating guardrails, and the taste-leverage loop. Inspired by Karpathy's autoresearch and the scientific method itself.",
+  "version": "2.1.0",
+  "description": "Autonomous ML research harness — the autoresearch loop as a formal protocol. 27 commands, 2 specialized agents, literature integration + paper section drafting, production model export (6 formats, equivalence, latency), performance profiling, smart Pareto-based checkpoint management, experiment intelligence (error analysis, ablation, Pareto frontiers), statistical rigor (seed studies, reproducibility), tree-search hypothesis exploration (TreeQuest AB-MCTS), cost-performance frontier, model cards, model registry, hypothesis database with novelty guard, anti-cheating guardrails, and the taste-leverage loop. Inspired by Karpathy's autoresearch and the scientific method itself.",
   "author": {
     "name": "pragnition"
   },

package/README.md

@@ -330,6 +330,8 @@ The index (`hypotheses.yaml`) is the lightweight queue. The detail files (`hypot
 | `/turing:frontier [--metrics]` | Pareto frontier — multi-objective tradeoff visualization |
 | `/turing:profile [exp-id]` | Computational profiling — timing, memory, throughput, bottleneck detection |
 | `/turing:checkpoint <action>` | Smart checkpoint management — list, prune (Pareto), average, resume, stats |
+| `/turing:lit <query>` | Literature search — papers, SOTA baselines, related work |
+| `/turing:paper [--sections] [--format]` | Draft paper sections from experiment logs (setup, results, ablation, hyperparams) |
 | `/turing:export [--format]` | Export model to production format with equivalence check + latency benchmark |
 | `/turing:card` | Generate a model card — performance, limitations, intended use, artifact contract |
 | `/turing:logbook` | Generate HTML experiment logbook |
@@ -520,11 +522,11 @@ Each project gets independent config, data, experiments, models, and agent memor
 
 ## Architecture of Turing Itself
 
-25 commands, 2 agents, 8 config files, 44 template scripts, model registry, artifact contract, cost-performance frontier, model cards, tree-search exploration, statistical rigor, experiment intelligence, performance profiling, smart checkpoints, production model export, 611 tests, 16 ADRs. See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the full codemap.
+27 commands, 2 agents, 8 config files, 46 template scripts, model registry, artifact contract, cost-performance frontier, model cards, tree-search exploration, statistical rigor, experiment intelligence, performance profiling, smart checkpoints, production model export, literature integration, paper section drafting, 664 tests, 16 ADRs. See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the full codemap.
 
 ```
 turing/
-├── commands/              24 skill files (core + taste-leverage + reporting + exploration + statistical rigor + experiment intelligence + performance + deployment)
+├── commands/              26 skill files (core + taste-leverage + reporting + exploration + statistical rigor + experiment intelligence + performance + deployment + research workflow)
 ├── agents/                2 agents (researcher: read/write, evaluator: read-only)
 ├── config/                8 files (lifecycle, taxonomy, archetypes, novelty aliases)
 ├── templates/             Scaffolded into user projects by /turing:init

package/commands/lit.md

@@ -0,0 +1,47 @@
+---
+name: lit
+description: Literature search scoped to the current experiment domain — find papers, SOTA baselines, and related work without leaving the terminal.
+disable-model-invocation: true
+argument-hint: "<query> | --baseline | --related <exp-id>"
+allowed-tools: Read, Bash(*), Grep, Glob, WebSearch
+---
+
+Search the literature for papers, baselines, and related work.
+
+## Steps
+
+1. **Activate environment:**
+   ```bash
+   source .venv/bin/activate
+   ```
+
+2. **Parse arguments from `$ARGUMENTS`:**
+   - **Free query:** `"gradient boosting for tabular data"` — searches Semantic Scholar
+   - **Baseline:** `--baseline` — finds SOTA results for the current task, compares against your best
+   - **Related:** `--related exp-042` — finds papers using similar methods to a specific experiment
+   - `--auto-queue` — auto-queues hypotheses from literature with `source: "literature"`
+   - `--limit 10` — max number of results
+
+3. **Run literature search:**
+   ```bash
+   python scripts/literature_search.py $ARGUMENTS
+   ```
+
+4. **Report results:**
+   - **Papers:** title, authors, year, venue, citations, abstract snippet, URL
+   - **Baseline mode:** SOTA comparison with gap analysis against current best
+   - **Related mode:** methodological differences worth investigating
+   - **Hypotheses:** if `--auto-queue`, shows queued experiments from findings
+
+5. **Saved output:** results written to `experiments/literature/query-YYYY-MM-DD-HHMMSS.md`
+
+6. **If API unavailable:** reports error and suggests manual search.
+
+## Examples
+
+```
+/turing:lit "gradient boosting missing values"    # Free query
+/turing:lit --baseline                             # SOTA comparison
+/turing:lit --related exp-042                      # Related work
+/turing:lit --auto-queue "ensemble methods"        # Queue hypotheses
+```

package/commands/paper.md

@@ -0,0 +1,44 @@
+---
+name: paper
+description: Draft mechanical paper sections (setup, results, ablation, hyperparameters) from experiment logs. LaTeX and markdown output.
+disable-model-invocation: true
+argument-hint: "[--sections setup,results,ablation] [--format latex|markdown]"
+allowed-tools: Read, Bash(*), Grep, Glob
+---
+
+Draft paper sections directly from experiment data.
+
+## Steps
+
+1. **Activate environment:**
+   ```bash
+   source .venv/bin/activate
+   ```
+
+2. **Parse arguments from `$ARGUMENTS`:**
+   - `--sections setup,results,ablation,hyperparameters` — which sections to draft (default: all)
+   - `--format latex|markdown` — output format (default: latex)
+
+3. **Run paper drafting:**
+   ```bash
+   python scripts/draft_paper_sections.py $ARGUMENTS
+   ```
+
+4. **Report results:**
+   - **setup:** Experimental setup prose (dataset, metrics, split, seed methodology)
+   - **results:** Comparison table with all model types, best bolded, seed study stats
+   - **ablation:** Ablation table from `/turing:ablate` results
+   - **hyperparameters:** Appendix-style parameter table per model
+
+5. **Output:** Each section saved to `paper/sections/` as `.tex` or `.md`
+
+6. **Numbers are pulled directly from experiment logs** — no manual transcription needed.
+
+## Examples
+
+```
+/turing:paper                                        # All sections, LaTeX
+/turing:paper --format markdown                      # All sections, markdown
+/turing:paper --sections setup,results               # Just setup + results
+/turing:paper --sections ablation --format latex      # Just ablation table
+```

package/commands/turing.md

@@ -31,6 +31,8 @@ You are the Turing ML research router. Detect the user's intent and route to the
 | "diagnose", "error analysis", "failure modes", "where does it fail", "confusion matrix" | `/turing:diagnose` | Analyze |
 | "ablate", "ablation", "remove component", "which features matter", "component impact" | `/turing:ablate` | Analyze |
 | "frontier", "pareto", "tradeoff", "tradeoffs", "multi-objective", "which model is best" | `/turing:frontier` | Analyze |
+| "lit", "literature", "papers", "SOTA", "baseline", "related work", "citations" | `/turing:lit` | Research |
+| "paper", "draft paper", "write paper", "results table", "latex", "experimental setup" | `/turing:paper` | Document |
 | "export", "deploy", "production", "onnx", "torchscript", "tflite", "ship model" | `/turing:export` | Deploy |
 | "profile", "profiling", "bottleneck", "slow training", "why is it slow", "timing" | `/turing:profile` | Check |
 | "checkpoint", "checkpoints", "prune checkpoints", "disk space", "resume training" | `/turing:checkpoint` | Check |
@@ -61,6 +63,8 @@ You are the Turing ML research router. Detect the user's intent and route to the
 | `/turing:diagnose [exp-id]` | Error analysis: failure modes, confused pairs, feature-range bias | (inline) |
 | `/turing:ablate [--components]` | Ablation study: remove components, measure impact, flag dead weight | (inline) |
 | `/turing:frontier [--metrics]` | Pareto frontier: multi-objective tradeoff visualization | (inline) |
+| `/turing:lit <query>` | Literature search: papers, SOTA baselines, related work | (inline, uses WebSearch) |
+| `/turing:paper [--sections] [--format]` | Draft paper sections from experiment logs (setup, results, ablation, hyperparams) | (inline) |
 | `/turing:export [exp-id] [--format]` | Export model to production format with equivalence check + latency benchmark | (inline) |
 | `/turing:profile [exp-id]` | Computational profiling: timing, memory, throughput, bottleneck detection | (inline) |
 | `/turing:checkpoint <action>` | Smart checkpoint management: list, prune (Pareto), average, resume, stats | (inline) |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "claude-turing",
-  "version": "2.0.0",
+  "version": "2.1.0",
   "type": "module",
   "description": "Autonomous ML research harness for Claude Code. The autoresearch loop as a formal protocol — iteratively trains, evaluates, and improves ML models with structured experiment tracking, convergence detection, immutable evaluation infrastructure, and safety guardrails.",
   "bin": {

package/src/install.js

@@ -24,6 +24,7 @@ const SUB_COMMANDS = [
   "try", "brief", "suggest", "explore", "design", "logbook", "poster",
   "report", "mode", "preflight", "card", "seed", "reproduce",
   "diagnose", "ablate", "frontier", "profile", "checkpoint", "export",
+  "lit", "paper",
 ];
 
 export async function install(opts = {}) {

package/src/verify.js

@@ -39,6 +39,8 @@ const EXPECTED_COMMANDS = [
   "profile/SKILL.md",
   "checkpoint/SKILL.md",
   "export/SKILL.md",
+  "lit/SKILL.md",
+  "paper/SKILL.md",
 ];
 
 const EXPECTED_AGENTS = ["ml-researcher.md", "ml-evaluator.md"];

package/templates/scripts/draft_paper_sections.py

@@ -0,0 +1,498 @@
+#!/usr/bin/env python3
+"""Paper section drafting from experiment logs.
+
+Drafts the mechanical sections of an ML paper directly from experiment
+data: experimental setup, results tables, ablation tables, and
+hyperparameter appendices. Eliminates transcription errors.
+
+Usage:
+    python scripts/draft_paper_sections.py                                    # All sections
+    python scripts/draft_paper_sections.py --sections setup,results           # Specific sections
+    python scripts/draft_paper_sections.py --format latex                     # LaTeX output
+    python scripts/draft_paper_sections.py --format markdown                  # Markdown output
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+from datetime import datetime, timezone
+from pathlib import Path
+
+import yaml
+
+from scripts.turing_io import load_config, load_experiments
+
+
+VALID_SECTIONS = ["setup", "results", "ablation", "hyperparameters"]
+DEFAULT_FORMAT = "latex"
+
+
+def load_seed_studies_for_paper(seed_dir: str = "experiments/seed_studies") -> dict[str, dict]:
+    """Load all seed studies indexed by experiment ID."""
+    path = Path(seed_dir)
+    studies = {}
+    if not path.exists():
+        return studies
+    for f in path.glob("*-seeds.yaml"):
+        try:
+            with open(f) as fh:
+                study = yaml.safe_load(fh)
+                if study and isinstance(study, dict) and "experiment_id" in study:
+                    studies[study["experiment_id"]] = study
+        except (yaml.YAMLError, OSError):
+            continue
+    return studies
+
+
+def load_ablation_studies(ablation_dir: str = "experiments/ablations") -> dict[str, dict]:
+    """Load all ablation studies indexed by experiment ID."""
+    path = Path(ablation_dir)
+    studies = {}
+    if not path.exists():
+        return studies
+    for f in path.glob("*-ablation.yaml"):
+        try:
+            with open(f) as fh:
+                study = yaml.safe_load(fh)
+                if study and isinstance(study, dict) and "experiment_id" in study:
+                    studies[study["experiment_id"]] = study
+        except (yaml.YAMLError, OSError):
+            continue
+    return studies
+
+
+def get_top_experiments(
+    experiments: list[dict],
+    metric: str,
+    lower_is_better: bool,
+    top_k: int = 10,
+) -> list[dict]:
+    """Get top-K kept experiments by primary metric."""
+    kept = [e for e in experiments if e.get("status") == "kept" and e.get("metrics", {}).get(metric) is not None]
+    kept.sort(key=lambda e: e["metrics"][metric], reverse=not lower_is_better)
+    return kept[:top_k]
+
+
+def group_by_model_type(experiments: list[dict]) -> dict[str, list[dict]]:
+    """Group experiments by model type, keeping best per type."""
+    groups: dict[str, list[dict]] = {}
+    for exp in experiments:
+        mt = exp.get("config", {}).get("model_type", "unknown")
+        groups.setdefault(mt, []).append(exp)
+    return groups
+
+
+def draft_setup_section(
+    config: dict,
+    experiments: list[dict],
+    seed_studies: dict[str, dict],
+    output_format: str = "latex",
+) -> str:
+    """Draft the experimental setup section."""
+    eval_cfg = config.get("evaluation", {})
+    data_cfg = config.get("data", {})
+    primary_metric = eval_cfg.get("primary_metric", "accuracy")
+    metrics = eval_cfg.get("metrics", [primary_metric])
+    lower_is_better = eval_cfg.get("lower_is_better", False)
+
+    task_desc = config.get("task_description", "the classification task")
+    data_source = data_cfg.get("source", "the provided dataset")
+    split_ratios = data_cfg.get("split_ratios", {})
+    random_state = data_cfg.get("random_state", 42)
+
+    # Determine seed study info
+    n_seeds = 0
+    for study in seed_studies.values():
+        n_seeds = max(n_seeds, len(study.get("seeds_run", [])))
+
+    # Build prose
+    split_text = ""
+    if split_ratios:
+        parts = [f"{int(v*100)}\\%" if output_format == "latex" else f"{int(v*100)}%" for v in split_ratios.values()]
+        split_names = list(split_ratios.keys())
+        split_text = "/".join(parts) + " " + "/".join(split_names) + " split"
+
+    direction = "lower" if lower_is_better else "higher"
+    metric_list = ", ".join(metrics)
+
+    if output_format == "latex":
+        lines = [
+            r"\subsection{Experimental Setup}",
+            "",
+            f"We evaluate on {data_source} using {metric_list} as evaluation metrics "
+            f"({direction} is better for {primary_metric}).",
+        ]
+        if split_text:
+            lines.append(f"Data is partitioned using a {split_text} with random state {random_state}.")
+        if n_seeds > 0:
+            lines.append(
+                f"Results are reported as mean $\\pm$ standard deviation over {n_seeds} random seeds "
+                f"to account for seed sensitivity."
+            )
+        lines.append(f"All experiments use {task_desc} as the target task.")
+    else:
+        lines = [
+            "## Experimental Setup",
+            "",
+            f"We evaluate on {data_source} using {metric_list} as evaluation metrics "
+            f"({direction} is better for {primary_metric}).",
+        ]
+        if split_text:
+            lines.append(f"Data is partitioned using a {split_text} with random state {random_state}.")
+        if n_seeds > 0:
+            lines.append(
+                f"Results are reported as mean +/- standard deviation over {n_seeds} random seeds "
+                f"to account for seed sensitivity."
+            )
+        lines.append(f"All experiments use {task_desc} as the target task.")
+
+    return "\n".join(lines)
+
+
+def draft_results_table(
+    experiments: list[dict],
+    metrics: list[str],
+    primary_metric: str,
+    lower_is_better: bool,
+    seed_studies: dict[str, dict],
+    output_format: str = "latex",
+    dataset_name: str = "the dataset",
+) -> str:
+    """Draft the results comparison table."""
+    # Group by model type, take best per type
+    groups = group_by_model_type(experiments)
+    rows = []
+    for mt, exps in groups.items():
+        # Best experiment per model type
+        best = exps[0]  # Already sorted by caller
+        row = {"model_type": mt, "experiment_id": best.get("experiment_id", "?")}
+        for m in metrics:
+            val = best.get("metrics", {}).get(m)
+            seed = seed_studies.get(best.get("experiment_id", ""))
+            if seed and seed.get("metric") == m:
+                row[m] = {"value": val, "mean": seed.get("mean"), "std": seed.get("std")}
+            else:
+                row[m] = {"value": val}
+        rows.append(row)
+
+    # Find best value per metric
+    best_per_metric = {}
+    for m in metrics:
+        values = [(r["model_type"], r[m].get("mean") or r[m].get("value")) for r in rows if r[m].get("value") is not None]
+        if values:
+            if lower_is_better:
+                best_per_metric[m] = min(values, key=lambda x: x[1] if x[1] is not None else float("inf"))[0]
+            else:
+                best_per_metric[m] = max(values, key=lambda x: x[1] if x[1] is not None else float("-inf"))[0]
+
+    if output_format == "latex":
+        return _format_results_latex(rows, metrics, best_per_metric, dataset_name)
+    else:
+        return _format_results_markdown(rows, metrics, best_per_metric, dataset_name)
+
+
+def _format_results_latex(rows: list[dict], metrics: list[str], best_per: dict, dataset: str) -> str:
+    """Format results as LaTeX table."""
+    n_cols = len(metrics)
+    col_spec = "l" + "c" * n_cols
+    metric_headers = " & ".join(m.replace("_", r"\_") for m in metrics)
+
+    lines = [
+        r"\begin{table}[h]",
+        r"\centering",
+        f"\\caption{{Comparison of model architectures on {dataset}.}}",
+        r"\label{tab:results}",
+        f"\\begin{{tabular}}{{{col_spec}}}",
+        r"\toprule",
+        f"Model & {metric_headers} \\\\",
+        r"\midrule",
+    ]
+
+    for row in rows:
+        mt = row["model_type"].replace("_", r"\_")
+        cells = [mt]
+        for m in metrics:
+            data = row[m]
+            val = data.get("mean") or data.get("value")
+            std = data.get("std")
+            if val is None:
+                cells.append("---")
+            elif std:
+                cell = f"{val:.3f} $\\pm$ {std:.3f}"
+                if best_per.get(m) == row["model_type"]:
+                    cell = f"\\textbf{{{cell}}}"
+                cells.append(cell)
+            else:
+                cell = f"{val:.4f}"
+                if best_per.get(m) == row["model_type"]:
+                    cell = f"\\textbf{{{cell}}}"
+                cells.append(cell)
+        lines.append(" & ".join(cells) + r" \\")
+
+    lines.extend([
+        r"\bottomrule",
+        r"\end{tabular}",
+        r"\end{table}",
+    ])
+    return "\n".join(lines)
+
+
+def _format_results_markdown(rows: list[dict], metrics: list[str], best_per: dict, dataset: str) -> str:
+    """Format results as markdown table."""
+    header = f"| Model |"
+    sep = "|-------|"
+    for m in metrics:
+        header += f" {m} |"
+        sep += f"{'---' * max(len(m) // 3, 1)}--|"
+
+    lines = [
+        f"## Results on {dataset}",
+        "",
+        header,
+        sep,
+    ]
+
+    for row in rows:
+        line = f"| {row['model_type']} |"
+        for m in metrics:
+            data = row[m]
+            val = data.get("mean") or data.get("value")
+            std = data.get("std")
+            if val is None:
+                line += " --- |"
+            elif std:
+                cell = f"{val:.3f} +/- {std:.3f}"
+                if best_per.get(m) == row["model_type"]:
+                    cell = f"**{cell}**"
+                line += f" {cell} |"
+            else:
+                cell = f"{val:.4f}"
+                if best_per.get(m) == row["model_type"]:
+                    cell = f"**{cell}**"
+                line += f" {cell} |"
+        lines.append(line)
+
+    return "\n".join(lines)
+
+
+def draft_ablation_table(
+    ablation_studies: dict[str, dict],
+    output_format: str = "latex",
+) -> str:
+    """Draft ablation table from ablation study results."""
+    if not ablation_studies:
+        return "No ablation studies available. Run `/turing:ablate` first."
+
+    # Use the most recent ablation study
+    study = list(ablation_studies.values())[-1]
+    metric = study.get("metric", "accuracy")
+    full_metric = study.get("full_model_metric", 0)
+    results = study.get("results", [])
+
+    if not results:
+        return "Ablation study has no results."
+
+    if output_format == "latex":
+        metric_escaped = metric.replace("_", r"\_")
+        lines = [
+            r"\begin{table}[h]",
+            r"\centering",
+            f"\\caption{{Ablation study results ({metric_escaped}).}}",
+            r"\label{tab:ablation}",
+            r"\begin{tabular}{lcc}",
+            r"\toprule",
+            f"Configuration & {metric_escaped} & $\\Delta$ from Full \\\\",
+            r"\midrule",
+            f"Full model & {full_metric:.4f} & --- \\\\",
+        ]
+        for r in results:
+            if r.get("status") == "failed":
+                continue
+            config = r.get("configuration", "?").replace("_", r"\_")
+            val = r.get("metric_value", 0)
+            delta = r.get("delta", 0)
+            delta_str = f"{delta:+.4f}" if delta is not None else "---"
+            lines.append(f"{config} & {val:.4f} & {delta_str} \\\\")
+        lines.extend([r"\bottomrule", r"\end{tabular}", r"\end{table}"])
+        return "\n".join(lines)
+    else:
+        lines = [
+            f"## Ablation Study ({metric})",
+            "",
+            f"| Configuration | {metric} | Delta from Full |",
+            f"|---------------|{'---' * max(len(metric) // 3, 1)}--|-----------------|",
+            f"| Full model | {full_metric:.4f} | --- |",
+        ]
+        for r in results:
+            if r.get("status") == "failed":
+                continue
+            config = r.get("configuration", "?")
+            val = r.get("metric_value", 0)
+            delta = r.get("delta", 0)
+            delta_str = f"{delta:+.4f}" if delta is not None else "---"
+            lines.append(f"| {config} | {val:.4f} | {delta_str} |")
+        return "\n".join(lines)
+
+
+def draft_hyperparameter_table(
+    experiments: list[dict],
+    output_format: str = "latex",
+) -> str:
+    """Draft hyperparameter appendix table."""
+    groups = group_by_model_type(experiments)
+
+    if output_format == "latex":
+        lines = [
+            r"\begin{table}[h]",
+            r"\centering",
+            r"\caption{Hyperparameters for reported models.}",
+            r"\label{tab:hyperparams}",
+            r"\begin{tabular}{llr}",
+            r"\toprule",
+            r"Model & Parameter & Value \\",
+            r"\midrule",
+        ]
+        for mt, exps in groups.items():
+            best = exps[0]
+            hyperparams = best.get("config", {}).get("hyperparams", {})
+            first = True
+            for param, value in sorted(hyperparams.items()):
+                model_col = mt.replace("_", r"\_") if first else ""
+                param_escaped = param.replace("_", r"\_")
+                lines.append(f"{model_col} & {param_escaped} & {value} \\\\")
+                first = False
+            lines.append(r"\midrule")
+        if lines[-1] == r"\midrule":
+            lines.pop()
+        lines.extend([r"\bottomrule", r"\end{tabular}", r"\end{table}"])
+        return "\n".join(lines)
+    else:
+        lines = [
+            "## Hyperparameters",
+            "",
+            "| Model | Parameter | Value |",
+            "|-------|-----------|-------|",
+        ]
+        for mt, exps in groups.items():
+            best = exps[0]
+            hyperparams = best.get("config", {}).get("hyperparams", {})
+            for param, value in sorted(hyperparams.items()):
+                lines.append(f"| {mt} | {param} | {value} |")
+        return "\n".join(lines)
+
+
+def save_paper_sections(sections: dict[str, str], output_dir: str = "paper/sections") -> list[Path]:
+    """Save each section to its own file."""
+    out_path = Path(output_dir)
+    out_path.mkdir(parents=True, exist_ok=True)
+
+    saved = []
+    for name, content in sections.items():
+        ext = ".tex" if r"\begin" in content else ".md"
+        filepath = out_path / f"{name}{ext}"
+        with open(filepath, "w") as f:
+            f.write(content)
+        saved.append(filepath)
+
+    return saved
+
+
+def draft_paper(
+    sections_str: str | None = None,
+    output_format: str = DEFAULT_FORMAT,
+    config_path: str = "config.yaml",
+    log_path: str = "experiments/log.jsonl",
+) -> dict:
+    """Draft all requested paper sections.
+
+    Args:
+        sections_str: Comma-separated section names (setup,results,ablation,hyperparameters).
+        output_format: "latex" or "markdown".
+        config_path: Path to config.yaml.
+        log_path: Path to experiment log.
+
+    Returns:
+        Dict with section name -> content mappings.
+    """
+    if sections_str:
+        sections = [s.strip() for s in sections_str.split(",")]
+    else:
+        sections = VALID_SECTIONS
+
+    config = load_config(config_path)
+    eval_cfg = config.get("evaluation", {})
+    primary_metric = eval_cfg.get("primary_metric", "accuracy")
+    all_metrics = eval_cfg.get("metrics", [primary_metric])
+    lower_is_better = eval_cfg.get("lower_is_better", False)
+
+    experiments = load_experiments(log_path)
+    top_exps = get_top_experiments(experiments, primary_metric, lower_is_better)
+
+    seed_studies = load_seed_studies_for_paper()
+    ablation_studies = load_ablation_studies()
+
+    dataset_name = config.get("data", {}).get("source", "the dataset")
+
+    result = {"format": output_format, "sections": {}}
+
+    for section in sections:
+        if section == "setup":
+            result["sections"]["setup"] = draft_setup_section(
+                config, experiments, seed_studies, output_format,
+            )
+        elif section == "results":
+            result["sections"]["results"] = draft_results_table(
+                top_exps, all_metrics, primary_metric, lower_is_better,
+                seed_studies, output_format, dataset_name,
+            )
+        elif section == "ablation":
+            result["sections"]["ablation"] = draft_ablation_table(
+                ablation_studies, output_format,
+            )
+        elif section == "hyperparameters":
+            result["sections"]["hyperparameters"] = draft_hyperparameter_table(
+                top_exps, output_format,
+            )
+
+    return result
+
+
+def main() -> None:
+    """CLI entry point."""
+    parser = argparse.ArgumentParser(description="Draft paper sections from experiment logs")
+    parser.add_argument("--sections", default=None, help="Comma-separated sections: setup,results,ablation,hyperparameters")
+    parser.add_argument("--format", default=DEFAULT_FORMAT, dest="output_format", choices=["latex", "markdown"])
+    parser.add_argument("--config", default="config.yaml", help="Path to config.yaml")
+    parser.add_argument("--log", default="experiments/log.jsonl", help="Path to experiment log")
+    parser.add_argument("--output", default="paper/sections", help="Output directory")
+    parser.add_argument("--json", action="store_true", help="Output raw JSON")
+    args = parser.parse_args()
+
+    result = draft_paper(
+        sections_str=args.sections,
+        output_format=args.output_format,
+        config_path=args.config,
+        log_path=args.log,
+    )
+
+    # Save sections
+    if result.get("sections"):
+        saved = save_paper_sections(result["sections"], args.output)
+        for f in saved:
+            print(f"Saved: {f}", file=sys.stderr)
+
+    if args.json:
+        print(json.dumps(result, indent=2, default=str))
+    else:
+        for name, content in result.get("sections", {}).items():
+            print(f"\n{'=' * 60}")
+            print(f"  {name.upper()}")
+            print(f"{'=' * 60}\n")
+            print(content)
+            print()
+
+
+if __name__ == "__main__":
+    main()

package/templates/scripts/literature_search.py

@@ -0,0 +1,421 @@
+#!/usr/bin/env python3
+"""Literature integration for ML experiments.
+
+Targeted literature search scoped to the current experiment's domain.
+Three modes: free query, baseline SOTA comparison, related papers.
+
+Uses Semantic Scholar API (free, no key required for basic search)
+with fallback to local-only mode when offline.
+
+Usage:
+    python scripts/literature_search.py "gradient boosting tabular"     # Free query
+    python scripts/literature_search.py --baseline                       # SOTA comparison
+    python scripts/literature_search.py --related exp-042                # Related papers
+    python scripts/literature_search.py --auto-queue "query"             # Queue hypotheses
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+import urllib.error
+import urllib.parse
+import urllib.request
+from datetime import datetime, timezone
+from pathlib import Path
+
+import yaml
+
+from scripts.turing_io import load_config, load_experiments
+
+
+SEMANTIC_SCHOLAR_API = "https://api.semanticscholar.org/graph/v1"
+DEFAULT_RESULT_COUNT = 5
+REQUEST_TIMEOUT = 15
+
+
+def search_semantic_scholar(
+    query: str,
+    limit: int = DEFAULT_RESULT_COUNT,
+    fields: str = "title,authors,year,venue,abstract,citationCount,externalIds",
+) -> list[dict]:
+    """Search Semantic Scholar for papers matching a query.
+
+    Returns list of paper dicts with title, authors, year, venue,
+    abstract, citation_count, and URLs.
+    """
+    params = urllib.parse.urlencode({
+        "query": query,
+        "limit": limit,
+        "fields": fields,
+    })
+    url = f"{SEMANTIC_SCHOLAR_API}/paper/search?{params}"
+
+    try:
+        req = urllib.request.Request(url, headers={"User-Agent": "turing-ml/2.0"})
+        with urllib.request.urlopen(req, timeout=REQUEST_TIMEOUT) as resp:
+            data = json.loads(resp.read().decode())
+    except (urllib.error.URLError, urllib.error.HTTPError, TimeoutError, json.JSONDecodeError) as e:
+        return [{"error": f"Semantic Scholar API failed: {e}"}]
+
+    papers = []
+    for item in data.get("data", []):
+        authors = [a.get("name", "?") for a in (item.get("authors") or [])]
+        ext_ids = item.get("externalIds") or {}
+
+        paper = {
+            "title": item.get("title", "Untitled"),
+            "authors": authors[:5],
+            "year": item.get("year"),
+            "venue": item.get("venue") or "N/A",
+            "abstract": (item.get("abstract") or "")[:300],
+            "citation_count": item.get("citationCount", 0),
+            "paper_id": item.get("paperId"),
+            "doi": ext_ids.get("DOI"),
+            "arxiv_id": ext_ids.get("ArXiv"),
+            "url": f"https://www.semanticscholar.org/paper/{item.get('paperId', '')}" if item.get("paperId") else None,
+        }
+        papers.append(paper)
+
+    return papers
+
+
+def build_query_from_config(config: dict) -> str:
+    """Build a search query from project config."""
+    parts = []
+
+    task_desc = config.get("task_description", "")
+    if task_desc:
+        parts.append(task_desc)
+
+    model_type = config.get("model", {}).get("type", "")
+    if model_type:
+        parts.append(model_type)
+
+    primary_metric = config.get("evaluation", {}).get("primary_metric", "")
+    if primary_metric:
+        parts.append(primary_metric)
+
+    data_source = config.get("data", {}).get("source", "")
+    if data_source and not data_source.startswith("{"):
+        parts.append(data_source)
+
+    return " ".join(parts) if parts else "machine learning"
+
+
+def build_query_from_experiment(experiment: dict) -> str:
+    """Build a search query from experiment metadata."""
+    parts = []
+
+    model_type = experiment.get("config", {}).get("model_type", "")
+    if model_type:
+        parts.append(model_type)
+
+    description = experiment.get("description", "")
+    if description:
+        parts.append(description[:100])
+
+    return " ".join(parts) if parts else "machine learning experiment"
+
+
+def search_baseline(
+    config: dict,
+    experiments: list[dict],
+    primary_metric: str,
+    lower_is_better: bool,
+) -> dict:
+    """Search for SOTA baselines and compare against current best.
+
+    Returns dict with SOTA results and gap analysis.
+    """
+    query = build_query_from_config(config)
+    query += " state of the art benchmark"
+
+    papers = search_semantic_scholar(query, limit=10)
+    if papers and "error" in papers[0]:
+        return {"error": papers[0]["error"], "query": query}
+
+    # Find current best
+    best = None
+    best_val = float("inf") if lower_is_better else float("-inf")
+    for exp in experiments:
+        if exp.get("status") != "kept":
+            continue
+        val = exp.get("metrics", {}).get(primary_metric)
+        if val is None:
+            continue
+        if (lower_is_better and val < best_val) or (not lower_is_better and val > best_val):
+            best_val = val
+            best = exp
+
+    result = {
+        "query": query,
+        "papers": papers,
+        "current_best": {
+            "experiment_id": best.get("experiment_id") if best else None,
+            "metric": primary_metric,
+            "value": round(best_val, 4) if best else None,
+        },
+    }
+
+    return result
+
+
+def search_related(
+    experiment: dict,
+    limit: int = DEFAULT_RESULT_COUNT,
+) -> dict:
+    """Find papers related to a specific experiment."""
+    query = build_query_from_experiment(experiment)
+    papers = search_semantic_scholar(query, limit=limit)
+
+    return {
+        "experiment_id": experiment.get("experiment_id", "?"),
+        "query": query,
+        "papers": papers,
+    }
+
+
+def generate_literature_hypotheses(papers: list[dict]) -> list[dict]:
+    """Generate hypotheses from literature findings.
+
+    Extracts methodological suggestions from paper titles/abstracts.
+    """
+    hypotheses = []
+    for i, paper in enumerate(papers):
+        if "error" in paper:
+            continue
+        title = paper.get("title", "")
+        if not title:
+            continue
+
+        hypotheses.append({
+            "id": f"hyp-lit-{i+1:03d}",
+            "description": f"Investigate approach from: {title}",
+            "source": "literature",
+            "status": "queued",
+            "priority": "normal",
+            "rationale": f"Paper: {title} ({paper.get('year', '?')}, {paper.get('citation_count', 0)} citations)",
+            "paper_url": paper.get("url"),
+            "created_at": datetime.now(timezone.utc).isoformat(),
+        })
+
+    return hypotheses[:5]
+
+
+def save_literature_results(results: dict, output_dir: str = "experiments/literature") -> Path:
+    """Save literature search results to markdown file."""
+    out_path = Path(output_dir)
+    out_path.mkdir(parents=True, exist_ok=True)
+
+    timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d-%H%M%S")
+    filepath = out_path / f"query-{timestamp}.md"
+
+    with open(filepath, "w") as f:
+        f.write(format_literature_report(results))
+
+    return filepath
+
+
+def format_literature_report(results: dict) -> str:
+    """Format literature search results as markdown."""
+    if "error" in results:
+        return f"ERROR: {results['error']}"
+
+    mode = results.get("mode", "query")
+    query = results.get("query", "")
+    papers = results.get("papers", [])
+
+    lines = [
+        f"# Literature Search",
+        "",
+        f"*Query: {query}*",
+        f"*Mode: {mode}*",
+        "",
+    ]
+
+    if not papers:
+        lines.append("No papers found.")
+        return "\n".join(lines)
+
+    if any("error" in p for p in papers):
+        error_paper = next(p for p in papers if "error" in p)
+        lines.append(f"**API Error:** {error_paper['error']}")
+        lines.append("")
+        lines.append("*Search may be offline. Try again later or use a manual search.*")
+        return "\n".join(lines)
+
+    # Papers table
+    lines.extend([
+        "## Results",
+        "",
+        "| # | Title | Year | Venue | Citations |",
+        "|---|-------|------|-------|-----------|",
+    ])
+
+    for i, paper in enumerate(papers, 1):
+        title = paper.get("title", "Untitled")
+        year = paper.get("year", "?")
+        venue = paper.get("venue", "N/A")
+        cites = paper.get("citation_count", 0)
+        lines.append(f"| {i} | {title} | {year} | {venue} | {cites} |")
+
+    # Paper details
+    lines.extend(["", "## Details", ""])
+    for i, paper in enumerate(papers, 1):
+        title = paper.get("title", "Untitled")
+        authors = ", ".join(paper.get("authors", [])[:3])
+        if len(paper.get("authors", [])) > 3:
+            authors += " et al."
+        abstract = paper.get("abstract", "")
+        url = paper.get("url", "")
+
+        lines.append(f"### {i}. {title}")
+        lines.append("")
+        lines.append(f"**Authors:** {authors}")
+        lines.append(f"**Year:** {paper.get('year', '?')} | **Venue:** {paper.get('venue', 'N/A')} | **Citations:** {paper.get('citation_count', 0)}")
+        if url:
+            lines.append(f"**URL:** {url}")
+        if abstract:
+            lines.append(f"**Abstract:** {abstract}...")
+        lines.append("")
+
+    # Baseline comparison
+    baseline = results.get("current_best")
+    if baseline and baseline.get("value") is not None:
+        lines.extend([
+            "## Current Performance",
+            "",
+            f"- **Best experiment:** {baseline.get('experiment_id', '?')}",
+            f"- **{baseline['metric']}:** {baseline['value']:.4f}",
+            "",
+            "*Compare against reported baselines in the papers above.*",
+        ])
+
+    return "\n".join(lines)
+
+
+def queue_literature_hypotheses(hypotheses: list[dict], queue_path: str = "hypotheses.yaml") -> int:
+    """Append literature hypotheses to the queue."""
+    path = Path(queue_path)
+    existing = []
+    if path.exists() and path.stat().st_size > 0:
+        with open(path) as f:
+            data = yaml.safe_load(f)
+            if isinstance(data, list):
+                existing = data
+
+    existing_ids = {h.get("id") for h in existing}
+    new = [h for h in hypotheses if h["id"] not in existing_ids]
+
+    if new:
+        existing.extend(new)
+        with open(path, "w") as f:
+            yaml.dump(existing, f, default_flow_style=False, sort_keys=False)
+
+    return len(new)
+
+
+def run_literature_search(
+    query: str | None = None,
+    baseline: bool = False,
+    related_exp_id: str | None = None,
+    auto_queue: bool = False,
+    config_path: str = "config.yaml",
+    log_path: str = "experiments/log.jsonl",
+    limit: int = DEFAULT_RESULT_COUNT,
+) -> dict:
+    """Run a literature search in the appropriate mode.
+
+    Args:
+        query: Free-text search query.
+        baseline: If True, search for SOTA baselines.
+        related_exp_id: If set, find papers related to this experiment.
+        auto_queue: Auto-queue hypotheses from findings.
+        config_path: Path to config.yaml.
+        log_path: Path to experiment log.
+        limit: Maximum number of results.
+
+    Returns:
+        Literature search result dict.
+    """
+    config = load_config(config_path)
+    eval_cfg = config.get("evaluation", {})
+    primary_metric = eval_cfg.get("primary_metric", "accuracy")
+    lower_is_better = eval_cfg.get("lower_is_better", False)
+    experiments = load_experiments(log_path)
+
+    if baseline:
+        result = search_baseline(config, experiments, primary_metric, lower_is_better)
+        result["mode"] = "baseline"
+    elif related_exp_id:
+        target = None
+        for exp in experiments:
+            if exp.get("experiment_id") == related_exp_id:
+                target = exp
+                break
+        if not target:
+            return {"error": f"Experiment {related_exp_id} not found", "mode": "related"}
+        result = search_related(target, limit=limit)
+        result["mode"] = "related"
+    elif query:
+        papers = search_semantic_scholar(query, limit=limit)
+        result = {"query": query, "papers": papers, "mode": "query"}
+    else:
+        # Default: search based on config
+        query = build_query_from_config(config)
+        papers = search_semantic_scholar(query, limit=limit)
+        result = {"query": query, "papers": papers, "mode": "query"}
+
+    result["timestamp"] = datetime.now(timezone.utc).isoformat()
+
+    # Generate and optionally queue hypotheses
+    papers = result.get("papers", [])
+    if papers and not any("error" in p for p in papers):
+        hypotheses = generate_literature_hypotheses(papers)
+        result["hypotheses"] = hypotheses
+
+        if auto_queue and hypotheses:
+            n_added = queue_literature_hypotheses(hypotheses)
+            result["hypotheses_queued"] = n_added
+            print(f"Queued {n_added} hypotheses from literature", file=sys.stderr)
+
+    return result
+
+
+def main() -> None:
+    """CLI entry point."""
+    parser = argparse.ArgumentParser(description="Literature search for ML experiments")
+    parser.add_argument("query", nargs="?", default=None, help="Free-text search query")
+    parser.add_argument("--baseline", action="store_true", help="Search for SOTA baselines")
+    parser.add_argument("--related", default=None, metavar="EXP_ID", help="Find related papers for experiment")
+    parser.add_argument("--auto-queue", action="store_true", help="Auto-queue hypotheses from findings")
+    parser.add_argument("--config", default="config.yaml", help="Path to config.yaml")
+    parser.add_argument("--log", default="experiments/log.jsonl", help="Path to experiment log")
+    parser.add_argument("--limit", type=int, default=DEFAULT_RESULT_COUNT, help="Max results")
+    parser.add_argument("--json", action="store_true", help="Output raw JSON")
+    args = parser.parse_args()
+
+    result = run_literature_search(
+        query=args.query,
+        baseline=args.baseline,
+        related_exp_id=args.related,
+        auto_queue=args.auto_queue,
+        config_path=args.config,
+        log_path=args.log,
+        limit=args.limit,
+    )
+
+    if "error" not in result:
+        filepath = save_literature_results(result)
+        print(f"Saved to {filepath}", file=sys.stderr)
+
+    if args.json:
+        print(json.dumps(result, indent=2, default=str))
+    else:
+        print(format_literature_report(result))
+
+
+if __name__ == "__main__":
+    main()

package/templates/scripts/scaffold.py

@@ -102,6 +102,8 @@ TEMPLATE_DIRS = {
         "equivalence_checker.py",
         "latency_benchmark.py",
         "export_card.py",
+        "literature_search.py",
+        "draft_paper_sections.py",
     ],
     "tests": ["__init__.py", "conftest.py"],
 }
@@ -118,6 +120,8 @@ DIRECTORIES_TO_CREATE = [
     "experiments/profiles",
     "experiments/checkpoints",
     "exports",
+    "experiments/literature",
+    "paper/sections",
     "models/best",
     "models/archive",
 ]