claude-turing 1.4.0 → 2.0.0

package/.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "turing",
-  "version": "1.4.0",
-  "description": "Autonomous ML research harness — the autoresearch loop as a formal protocol. 22 commands, 2 specialized agents, experiment intelligence (error analysis, ablation studies, Pareto frontiers), statistical rigor (multi-seed studies, reproducibility verification), tree-search hypothesis exploration (TreeQuest AB-MCTS), cost-performance frontier analysis, model cards, model registry, hypothesis database with novelty guard, anti-cheating guardrails, and the taste-leverage loop. Inspired by Karpathy's autoresearch and the scientific method itself.",
+  "version": "2.0.0",
+  "description": "Autonomous ML research harness — the autoresearch loop as a formal protocol. 25 commands, 2 specialized agents, production model export (6 formats, equivalence verification, latency benchmarking), performance profiling, smart Pareto-based checkpoint management, experiment intelligence (error analysis, ablation studies, Pareto frontiers), statistical rigor (multi-seed studies, reproducibility verification), tree-search hypothesis exploration (TreeQuest AB-MCTS), cost-performance frontier analysis, model cards, model registry, hypothesis database with novelty guard, anti-cheating guardrails, and the taste-leverage loop. Inspired by Karpathy's autoresearch and the scientific method itself.",
   "author": {
     "name": "pragnition"
   },

package/README.md

@@ -328,6 +328,9 @@ The index (`hypotheses.yaml`) is the lightweight queue. The detail files (`hypot
 | `/turing:diagnose [exp-id]` | Error analysis — failure modes, confused pairs, feature-range bias |
 | `/turing:ablate [--components]` | Ablation study — remove components, measure impact, flag dead weight |
 | `/turing:frontier [--metrics]` | Pareto frontier — multi-objective tradeoff visualization |
+| `/turing:profile [exp-id]` | Computational profiling — timing, memory, throughput, bottleneck detection |
+| `/turing:checkpoint <action>` | Smart checkpoint management — list, prune (Pareto), average, resume, stats |
+| `/turing:export [--format]` | Export model to production format with equivalence check + latency benchmark |
 | `/turing:card` | Generate a model card — performance, limitations, intended use, artifact contract |
 | `/turing:logbook` | Generate HTML experiment logbook |
 | `/turing:report` | Generate research report |
@@ -517,11 +520,11 @@ Each project gets independent config, data, experiments, models, and agent memor
 
 ## Architecture of Turing Itself
 
-22 commands, 2 agents, 8 config files, 37 template scripts, model registry, artifact contract, cost-performance frontier, model cards, tree-search exploration, statistical rigor, experiment intelligence (error analysis + ablation + Pareto frontier), 487 tests, 16 ADRs. See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the full codemap.
+25 commands, 2 agents, 8 config files, 44 template scripts, model registry, artifact contract, cost-performance frontier, model cards, tree-search exploration, statistical rigor, experiment intelligence, performance profiling, smart checkpoints, production model export, 611 tests, 16 ADRs. See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the full codemap.
 
 ```
 turing/
-├── commands/              21 skill files (core + taste-leverage + reporting + exploration + statistical rigor + experiment intelligence)
+├── commands/              24 skill files (core + taste-leverage + reporting + exploration + statistical rigor + experiment intelligence + performance + deployment)
 ├── agents/                2 agents (researcher: read/write, evaluator: read-only)
 ├── config/                8 files (lifecycle, taxonomy, archetypes, novelty aliases)
 ├── templates/             Scaffolded into user projects by /turing:init

package/commands/checkpoint.md

@@ -0,0 +1,47 @@
+---
+name: checkpoint
+description: Smart checkpoint management — list, prune (Pareto-based), average top-K, resume from any point, disk usage stats.
+disable-model-invocation: true
+argument-hint: "<list|prune|average|resume|stats> [exp-id] [--top 3] [--dry-run]"
+allowed-tools: Read, Bash(*), Grep, Glob
+---
+
+Manage model checkpoints intelligently using Pareto dominance.
+
+## Steps
+
+1. **Activate environment:**
+   ```bash
+   source .venv/bin/activate
+   ```
+
+2. **Parse arguments from `$ARGUMENTS`:**
+   - First word is the action: `list`, `prune`, `average`, `resume`, `stats`
+   - `resume` requires an experiment ID as second argument
+   - `--top 3` sets the number of checkpoints for averaging
+   - `--dry-run` previews pruning without deleting
+
+3. **Run checkpoint manager:**
+   ```bash
+   python scripts/checkpoint_manager.py $ARGUMENTS
+   ```
+
+4. **Report results by action:**
+   - **list:** Table of all checkpoints with metrics, size, and Pareto status
+   - **prune:** Removes dominated checkpoints, reports space saved
+   - **average:** Lists top-K checkpoints for weight averaging
+   - **resume:** Locates checkpoint for a specific experiment
+   - **stats:** Disk usage summary by total, average, and model type
+
+5. **Saved output:** report written to `experiments/checkpoints/checkpoint-report.yaml`
+
+## Examples
+
+```
+/turing:checkpoint list              # Show all checkpoints
+/turing:checkpoint stats             # Disk usage summary
+/turing:checkpoint prune --dry-run   # Preview what would be pruned
+/turing:checkpoint prune             # Remove dominated checkpoints
+/turing:checkpoint average --top 5   # Top 5 for averaging
+/turing:checkpoint resume exp-042    # Resume from checkpoint
+```

package/commands/export.md

@@ -0,0 +1,48 @@
+---
+name: export
+description: Export model to production format with equivalence verification, latency benchmarking, and deployment model card.
+disable-model-invocation: true
+argument-hint: "[exp-id] [--format joblib|xgboost_json|onnx|torchscript|tflite]"
+allowed-tools: Read, Bash(*), Grep, Glob
+---
+
+Export a trained model to a production-ready format.
+
+## Steps
+
+1. **Activate environment:**
+   ```bash
+   source .venv/bin/activate
+   ```
+
+2. **Parse arguments from `$ARGUMENTS`:**
+   - First argument can be an experiment ID (e.g., `exp-042`); defaults to best
+   - `--format joblib|xgboost_json|onnx|torchscript|tflite` specifies export format (auto-detected if omitted)
+   - `--skip-equivalence` skips inference equivalence check
+   - `--skip-latency` skips latency benchmark
+   - `--samples 100` sets test sample count
+
+3. **Run export pipeline:**
+   ```bash
+   python scripts/export_model.py $ARGUMENTS
+   ```
+
+4. **Report results:**
+   - **Export:** format, file size, output path, dependencies
+   - **Equivalence:** verdict (equivalent/approximately_equivalent/divergent), max delta
+   - **Latency:** p50/p95/p99 ms, speedup vs original
+   - **Model Card:** metrics, seed study, equivalence, latency, dependencies
+
+5. **Output:** exported model + model_card.yaml written to `exports/exp-NNN/`
+
+6. **If model file not found:** suggest checking models/best/ directory.
+
+## Examples
+
+```
+/turing:export                                     # Best experiment, default format
+/turing:export exp-042                             # Specific experiment
+/turing:export --format xgboost_json               # Native XGBoost JSON
+/turing:export --format onnx                       # ONNX format
+/turing:export --skip-equivalence --skip-latency   # Fast export
+```

package/commands/profile.md

@@ -0,0 +1,43 @@
+---
+name: profile
+description: Profile a training run — timing breakdown, memory usage, throughput, bottleneck detection with actionable recommendations.
+disable-model-invocation: true
+argument-hint: "[exp-id] [--seed 42]"
+allowed-tools: Read, Bash(*), Grep, Glob
+---
+
+Profile a training run to identify performance bottlenecks.
+
+## Steps
+
+1. **Activate environment:**
+   ```bash
+   source .venv/bin/activate
+   ```
+
+2. **Parse arguments from `$ARGUMENTS`:**
+   - First argument can be an experiment ID (e.g., `exp-042`); defaults to best
+   - `--seed 42` sets the random seed for the profiling run
+
+3. **Run profiling:**
+   ```bash
+   python scripts/profile_training.py $ARGUMENTS
+   ```
+
+4. **Report results:**
+   - **Timing:** total time, training time, overhead breakdown
+   - **Memory:** peak RSS, Python peak, GPU peak (if applicable)
+   - **Throughput:** samples/sec
+   - **Bottleneck:** identified bottleneck type and severity
+   - **Recommendations:** actionable fixes for the detected bottleneck
+
+5. **Saved output:** results written to `experiments/profiles/exp-NNN-profile.yaml`
+
+6. **If no training pipeline exists:** suggest `/turing:init` first.
+
+## Examples
+
+```
+/turing:profile              # Profile best experiment config
+/turing:profile exp-042      # Profile specific experiment
+```

package/commands/turing.md

@@ -31,6 +31,9 @@ You are the Turing ML research router. Detect the user's intent and route to the
 | "diagnose", "error analysis", "failure modes", "where does it fail", "confusion matrix" | `/turing:diagnose` | Analyze |
 | "ablate", "ablation", "remove component", "which features matter", "component impact" | `/turing:ablate` | Analyze |
 | "frontier", "pareto", "tradeoff", "tradeoffs", "multi-objective", "which model is best" | `/turing:frontier` | Analyze |
+| "export", "deploy", "production", "onnx", "torchscript", "tflite", "ship model" | `/turing:export` | Deploy |
+| "profile", "profiling", "bottleneck", "slow training", "why is it slow", "timing" | `/turing:profile` | Check |
+| "checkpoint", "checkpoints", "prune checkpoints", "disk space", "resume training" | `/turing:checkpoint` | Check |
 
 ## Sub-commands
 
@@ -58,6 +61,9 @@ You are the Turing ML research router. Detect the user's intent and route to the
 | `/turing:diagnose [exp-id]` | Error analysis: failure modes, confused pairs, feature-range bias | (inline) |
 | `/turing:ablate [--components]` | Ablation study: remove components, measure impact, flag dead weight | (inline) |
 | `/turing:frontier [--metrics]` | Pareto frontier: multi-objective tradeoff visualization | (inline) |
+| `/turing:export [exp-id] [--format]` | Export model to production format with equivalence check + latency benchmark | (inline) |
+| `/turing:profile [exp-id]` | Computational profiling: timing, memory, throughput, bottleneck detection | (inline) |
+| `/turing:checkpoint <action>` | Smart checkpoint management: list, prune (Pareto), average, resume, stats | (inline) |
 
 ## Proactive Detection
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "claude-turing",
-  "version": "1.4.0",
+  "version": "2.0.0",
   "type": "module",
   "description": "Autonomous ML research harness for Claude Code. The autoresearch loop as a formal protocol — iteratively trains, evaluates, and improves ML models with structured experiment tracking, convergence detection, immutable evaluation infrastructure, and safety guardrails.",
   "bin": {

package/src/install.js

@@ -23,7 +23,7 @@ const SUB_COMMANDS = [
   "init", "train", "status", "compare", "sweep", "validate",
   "try", "brief", "suggest", "explore", "design", "logbook", "poster",
   "report", "mode", "preflight", "card", "seed", "reproduce",
-  "diagnose", "ablate", "frontier",
+  "diagnose", "ablate", "frontier", "profile", "checkpoint", "export",
 ];
 
 export async function install(opts = {}) {

package/src/verify.js

@@ -36,6 +36,9 @@ const EXPECTED_COMMANDS = [
   "diagnose/SKILL.md",
   "ablate/SKILL.md",
   "frontier/SKILL.md",
+  "profile/SKILL.md",
+  "checkpoint/SKILL.md",
+  "export/SKILL.md",
 ];
 
 const EXPECTED_AGENTS = ["ml-researcher.md", "ml-evaluator.md"];