claude-turing 4.2.0 → 4.3.0

package/.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "turing",
-  "version": "4.2.0",
-  "description": "Autonomous ML research harness — the autoresearch loop as a formal protocol. 69 commands, 2 specialized agents, what-if analysis (whatif + counterfactual + simulate), collaboration (onboard + share + review), research communication (cite + present + changelog), experiment archaeology (trend + flashback + archive + annotate + search + template + replay), model surgery (prune + quantize + merge + surgery), feature & training intelligence, model debugging, pre-training intelligence, meta-intelligence, scaling & efficiency, model composition, deep analysis, experiment orchestration, literature + paper, model export, profiling, checkpoints, experiment intelligence, statistical rigor, tree-search, cost-performance, model cards, hypothesis database, novelty guard, anti-cheating, taste-leverage loop. Inspired by Karpathy's autoresearch and the scientific method itself.",
+  "version": "4.3.0",
+  "description": "Autonomous ML research harness — the autoresearch loop as a formal protocol. 71 commands, 2 specialized agents, model lifecycle (update + registry), what-if analysis (whatif + counterfactual + simulate), collaboration (onboard + share + review), research communication (cite + present + changelog), experiment archaeology (trend + flashback + archive + annotate + search + template + replay), model surgery (prune + quantize + merge + surgery), feature & training intelligence, model debugging, pre-training intelligence, meta-intelligence, scaling & efficiency, model composition, deep analysis, experiment orchestration, literature + paper, model export, profiling, checkpoints, experiment intelligence, statistical rigor, tree-search, cost-performance, model cards, hypothesis database, novelty guard, anti-cheating, taste-leverage loop. Inspired by Karpathy's autoresearch and the scientific method itself.",
   "author": {
     "name": "pragnition"
   },

package/README.md

@@ -380,6 +380,8 @@ The index (`hypotheses.yaml`) is the lightweight queue. The detail files (`hypot
 | `/turing:whatif "<question>"` | What-if analysis — answer hypotheticals from existing experiment data |
 | `/turing:counterfactual <exp-id>` | Counterfactual explanations — minimum input change to flip a prediction |
 | `/turing:simulate [--configs]` | Experiment outcome prediction — pre-filter configs, save budget |
+| `/turing:update <exp-id>` | Incremental model update — add new data without full retraining |
+| `/turing:registry [action]` | Model registry — track lifecycle from candidate to production with gates |
 
 And for fully hands-off operation:
 
@@ -564,11 +566,11 @@ Each project gets independent config, data, experiments, models, and agent memor
 
 ## Architecture of Turing Itself
 
-69 commands, 2 agents, 10 config files, 88 template scripts, model registry, artifact contract, cost-performance frontier, model cards, tree-search exploration, statistical rigor, experiment intelligence, performance profiling, smart checkpoints, production model export, literature integration, paper section drafting, experiment orchestration (queue + retry + fork), deep analysis (diff + watch + regress), model composition (ensemble + stitch + warm), scaling & efficiency (scale + budget + distill), meta-intelligence (transfer + audit), pre-training intelligence (sanity + baseline + leak), model debugging (xray + sensitivity + calibrate), feature & training intelligence (feature + curriculum), model surgery (prune + quantize + merge + surgery), experiment archaeology (trend + flashback + archive + annotate + search + template + replay), research communication (cite + present + changelog), collaboration (onboard + share + review), what-if analysis (whatif + counterfactual + simulate), 16 ADRs. See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the full codemap.
+71 commands, 2 agents, 10 config files, 90 template scripts, model registry, artifact contract, cost-performance frontier, model cards, tree-search exploration, statistical rigor, experiment intelligence, performance profiling, smart checkpoints, production model export, literature integration, paper section drafting, experiment orchestration (queue + retry + fork), deep analysis (diff + watch + regress), model composition (ensemble + stitch + warm), scaling & efficiency (scale + budget + distill), meta-intelligence (transfer + audit), pre-training intelligence (sanity + baseline + leak), model debugging (xray + sensitivity + calibrate), feature & training intelligence (feature + curriculum), model surgery (prune + quantize + merge + surgery), experiment archaeology (trend + flashback + archive + annotate + search + template + replay), research communication (cite + present + changelog), collaboration (onboard + share + review), what-if analysis (whatif + counterfactual + simulate), model lifecycle (update + registry), 16 ADRs. See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the full codemap.
 
 ```
 turing/
-├── commands/              65 skill files (core + taste-leverage + reporting + exploration + statistical rigor + experiment intelligence + performance + deployment + research workflow + orchestration + deep analysis + model composition + scaling & efficiency + meta-intelligence + pre-training intelligence + model debugging + feature & training intelligence + model surgery + experiment archaeology + research communication + what-if analysis)
+├── commands/              67 skill files (core + taste-leverage + reporting + exploration + statistical rigor + experiment intelligence + performance + deployment + research workflow + orchestration + deep analysis + model composition + scaling & efficiency + meta-intelligence + pre-training intelligence + model debugging + feature & training intelligence + model surgery + experiment archaeology + research communication + what-if analysis + model lifecycle)
 ├── agents/                2 agents (researcher: read/write, evaluator: read-only)
 ├── config/                8 files (lifecycle, taxonomy, archetypes, novelty aliases)
 ├── templates/             Scaffolded into user projects by /turing:init

package/commands/registry.md

@@ -0,0 +1,31 @@
+---
+name: registry
+description: Model registry — track, promote, and govern the model lifecycle from candidate to production.
+disable-model-invocation: true
+argument-hint: "[list|register|promote|demote|archive|history] [exp-id] [stage]"
+allowed-tools: Read, Bash(*), Grep, Glob
+---
+
+Track which model is production, staging, candidate, or archived. Promotion requires passing gates.
+
+## Steps
+1. `source .venv/bin/activate`
+2. `python scripts/model_lifecycle.py $ARGUMENTS`
+3. **Registry:** `experiments/registry.yaml`
+
+## Promotion gates
+- **candidate → staging:** regression check + seed study must PASS
+- **staging → production:** audit + calibration check must PASS
+- Use `--force` to skip gate checks
+
+## Examples
+```
+/turing:registry list
+/turing:registry register exp-095 --version v4.1
+/turing:registry promote exp-089 staging
+/turing:registry promote exp-089 production --force
+/turing:registry demote exp-078 staging --reason "latency regression"
+/turing:registry archive exp-042 --reason "superseded by v4"
+/turing:registry history
+/turing:registry history exp-089
+```

package/commands/turing.md

@@ -78,6 +78,8 @@ You are the Turing ML research router. Detect the user's intent and route to the
 | "what if", "what-if", "hypothetical", "estimate impact", "would it help" | `/turing:whatif` | Analyze |
 | "counterfactual", "flip prediction", "why this prediction", "minimum change", "explanation" | `/turing:counterfactual` | Explain |
 | "simulate", "predict outcome", "pre-filter", "which configs will work", "forecast" | `/turing:simulate` | Predict |
+| "update", "incremental", "new data", "add data", "fine-tune existing", "partial update" | `/turing:update` | Update |
+| "registry", "promote", "demote", "staging", "production", "which model is deployed", "model lifecycle" | `/turing:registry` | Govern |
 
 ## Sub-commands
 
@@ -152,6 +154,8 @@ You are the Turing ML research router. Detect the user's intent and route to the
 | `/turing:whatif "<question>"` | What-if analysis: route hypotheticals to existing estimators (scaling, ablation, sensitivity, ensemble, pruning) | (inline) |
 | `/turing:counterfactual <exp-id> --sample <index>` | Input-level counterfactual explanations: minimum input change to flip a prediction | (inline) |
 | `/turing:simulate [--configs] [--top-k]` | Experiment outcome prediction: pre-filter configs using surrogate model, save budget | (inline) |
+| `/turing:update <exp-id> --new-data <path>` | Incremental model update: add new data without full retraining, forgetting detection | (inline) |
+| `/turing:registry [list\|register\|promote\|demote\|history]` | Model registry: stage lifecycle (candidate → staging → production) with promotion gates | (inline) |
 
 ## Proactive Detection
 

package/commands/update.md

@@ -0,0 +1,27 @@
+---
+name: update
+description: Incremental model update — add new data without full retraining, with forgetting detection.
+disable-model-invocation: true
+argument-hint: "<exp-id> --new-data <path> [--replay-ratio 0.1] [--tolerance 0.005]"
+allowed-tools: Read, Bash(*), Grep, Glob
+---
+
+Add new data to an existing model without starting from scratch. Detects catastrophic forgetting.
+
+## Steps
+1. `source .venv/bin/activate`
+2. `python scripts/incremental_update.py $ARGUMENTS`
+3. **Saved:** `experiments/updates/`
+
+## Model-specific strategies
+- **XGBoost/LightGBM:** continued boosting with additional rounds
+- **Neural networks:** fine-tune with reduced LR + replay buffer from old data
+- **scikit-learn:** partial_fit() or warm_start=True
+
+## Examples
+```
+/turing:update exp-089 --new-data data/new_batch.csv
+/turing:update exp-089 --new-data data/new.csv --replay-ratio 0.2
+/turing:update exp-089 --new-data data/new.csv --tolerance 0.01
+/turing:update exp-089 --new-data data/new.csv --json
+```

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "claude-turing",
-  "version": "4.2.0",
+  "version": "4.3.0",
   "type": "module",
   "description": "Autonomous ML research harness for Claude Code. The autoresearch loop as a formal protocol — iteratively trains, evaluates, and improves ML models with structured experiment tracking, convergence detection, immutable evaluation infrastructure, and safety guardrails.",
   "bin": {

package/src/install.js

@@ -37,6 +37,7 @@ const SUB_COMMANDS = [
   "cite", "present", "changelog",
   "onboard", "share", "review",
   "whatif", "counterfactual", "simulate",
+  "update", "registry",
 ];
 
 export async function install(opts = {}) {

package/src/verify.js

@@ -83,6 +83,8 @@ const EXPECTED_COMMANDS = [
   "whatif/SKILL.md",
   "counterfactual/SKILL.md",
   "simulate/SKILL.md",
+  "update/SKILL.md",
+  "registry/SKILL.md",
 ];
 
 const EXPECTED_AGENTS = ["ml-researcher.md", "ml-evaluator.md"];

package/templates/scripts/generate_brief.py

@@ -404,6 +404,38 @@ def load_simulation_results(sim_dir: str = "experiments/simulations") -> dict |
         return None
 
 
+def load_registry_summary(registry_path: str = "experiments/registry.yaml") -> dict | None:
+    """Load model registry summary for briefing."""
+    path = Path(registry_path)
+    if not path.exists():
+        return None
+    try:
+        with open(path) as f:
+            data = yaml.safe_load(f)
+        if isinstance(data, dict) and data.get("models"):
+            return data
+    except (yaml.YAMLError, OSError):
+        pass
+    return None
+
+
+def load_update_history(update_dir: str = "experiments/updates") -> list[dict]:
+    """Load recent incremental update reports."""
+    path = Path(update_dir)
+    if not path.exists():
+        return []
+    results = []
+    for f in sorted(path.glob("*-update-*.yaml"))[-3:]:
+        try:
+            with open(f) as fh:
+                data = yaml.safe_load(fh)
+                if isinstance(data, dict):
+                    results.append(data)
+        except (yaml.YAMLError, OSError):
+            continue
+    return results
+
+
 def format_brief(
     campaign: dict,
     best: dict | None,
@@ -428,6 +460,8 @@ def format_brief(
     audit_report: dict | None = None,
     whatif_results: list[dict] | None = None,
     simulation_result: dict | None = None,
+    registry_summary: dict | None = None,
+    update_history: list[dict] | None = None,
 ) -> str:
     """Format the research briefing as markdown."""
     direction = "lower" if lower_is_better else "higher"
@@ -758,6 +792,29 @@ def format_brief(
             lines.append(f"**Last simulation:** {run} configs recommended, {skip} skipped ({savings}% budget savings)")
             lines.append("")
 
+    # Model Lifecycle section
+    if registry_summary or update_history:
+        lines.extend(["", "## Model Lifecycle", ""])
+
+        if registry_summary:
+            models = registry_summary.get("models", [])
+            for m in models:
+                if m.get("stage") != "archived":
+                    metric = f"{m['metric']:.4f}" if m.get("metric") is not None else "—"
+                    lines.append(f"- **{m['stage']}:** {m['exp_id']} ({m.get('version', '?')}, {m.get('metric_name', 'metric')}={metric})")
+            if not any(m.get("stage") != "archived" for m in models):
+                lines.append("- All models archived — register a new candidate with `/turing:registry register`")
+            lines.append("")
+
+        if update_history:
+            lines.append(f"**Recent updates:** {len(update_history)}")
+            for u in update_history[-2:]:
+                verdict = u.get("verdict", "?")
+                exp_id = u.get("experiment_id", "?")
+                strategy = u.get("plan", {}).get("strategy", "?")
+                lines.append(f"- {exp_id}: {strategy} — {verdict}")
+            lines.append("")
+
     lines.extend([
         "",
         "## Recommendations",
@@ -830,6 +887,8 @@ def generate_brief(
     audit_report = load_audit_report()
     whatif_results = load_whatif_results()
     simulation_result = load_simulation_results()
+    registry_summary = load_registry_summary()
+    update_history = load_update_history()
 
     return format_brief(
         campaign, best, trajectory, model_types, hypotheses,
@@ -848,6 +907,8 @@ def generate_brief(
         audit_report=audit_report,
         whatif_results=whatif_results if whatif_results else None,
         simulation_result=simulation_result,
+        registry_summary=registry_summary,
+        update_history=update_history if update_history else None,
     )
 
 

package/templates/scripts/generate_model_card.py

@@ -18,6 +18,8 @@ import sys
 from datetime import datetime, timezone
 from pathlib import Path
 
+import yaml
+
 from scripts.turing_io import load_config, load_experiments
 
 
@@ -93,22 +95,113 @@ def load_model_contract(contract_path: str) -> dict:
     return {"version": version, "bundle_format": bundle_format, "raw": text}
 
 
+def load_registry_status(registry_path: str = "experiments/registry.yaml") -> dict | None:
+    """Load registry status for the best model."""
+    path = Path(registry_path)
+    if not path.exists():
+        return None
+    try:
+        with open(path) as f:
+            data = yaml.safe_load(f)
+        if isinstance(data, dict) and data.get("models"):
+            return data
+    except (Exception,):
+        pass
+    return None
+
+
+def compute_fairness_metrics(
+    predictions: list | None = None,
+    labels: list | None = None,
+    protected_attribute: list | None = None,
+    group_names: list[str] | None = None,
+) -> dict | None:
+    """Compute demographic parity and equal opportunity metrics.
+
+    Args:
+        predictions: Model predictions.
+        labels: True labels.
+        protected_attribute: Group membership for each sample.
+        group_names: Names of groups.
+
+    Returns:
+        Fairness metrics dict or None if insufficient data.
+    """
+    if predictions is None or protected_attribute is None:
+        return None
+    if len(predictions) != len(protected_attribute):
+        return None
+    if len(predictions) == 0:
+        return None
+
+    import numpy as np
+
+    preds = np.array(predictions)
+    groups = np.array(protected_attribute)
+    unique_groups = sorted(set(groups))
+
+    if group_names is None:
+        group_names = [str(g) for g in unique_groups]
+
+    # Demographic parity: P(Y_hat=1 | G=g) for each group
+    group_positive_rates = {}
+    for g, name in zip(unique_groups, group_names):
+        mask = groups == g
+        if mask.sum() == 0:
+            continue
+        rate = float(preds[mask].mean()) if preds[mask].size > 0 else 0
+        group_positive_rates[name] = round(rate, 4)
+
+    # Demographic parity difference
+    rates = list(group_positive_rates.values())
+    dp_diff = round(max(rates) - min(rates), 4) if len(rates) >= 2 else 0
+
+    result = {
+        "group_positive_rates": group_positive_rates,
+        "demographic_parity_difference": dp_diff,
+        "n_groups": len(unique_groups),
+    }
+
+    # Equal opportunity (if labels available): P(Y_hat=1 | Y=1, G=g)
+    if labels is not None and len(labels) == len(predictions):
+        labs = np.array(labels)
+        group_tpr = {}
+        for g, name in zip(unique_groups, group_names):
+            mask = (groups == g) & (labs == 1)
+            if mask.sum() == 0:
+                continue
+            tpr = float(preds[mask].mean()) if preds[mask].size > 0 else 0
+            group_tpr[name] = round(tpr, 4)
+
+        result["group_true_positive_rates"] = group_tpr
+        tpr_vals = list(group_tpr.values())
+        result["equal_opportunity_difference"] = round(max(tpr_vals) - min(tpr_vals), 4) if len(tpr_vals) >= 2 else 0
+
+    return result
+
+
 def generate_card(
     config_path: str = "config.yaml",
     log_path: str = "experiments/log.jsonl",
     contract_path: str = "model_contract.md",
     output_path: str | None = None,
+    include_fairness: bool = False,
+    fairness_data: dict | None = None,
+    registry_path: str = "experiments/registry.yaml",
 ) -> str:
     """Produce a model card markdown document.
 
     Combines information from the project config, experiment log,
-    and model contract into a standardized model card.
+    model contract, registry, and optional fairness data.
 
     Args:
         config_path: Path to config.yaml.
         log_path: Path to experiments/log.jsonl.
         contract_path: Path to model_contract.md.
         output_path: If given, write the card to this file.
+        include_fairness: If True, add fairness section.
+        fairness_data: Pre-computed fairness data {predictions, labels, protected_attribute}.
+        registry_path: Path to registry YAML.
 
     Returns:
         The model card as a markdown string.
@@ -247,7 +340,6 @@ def generate_card(
     if best:
         seed_study_path = Path("experiments/seed_studies") / f"{best.get('experiment_id', 'unknown')}-seeds.yaml"
         if seed_study_path.exists():
-            import yaml
             with open(seed_study_path) as f:
                 seed_study = yaml.safe_load(f) or {}
             if seed_study and "mean" in seed_study:
@@ -306,6 +398,57 @@ def generate_card(
         "- Not intended for: <placeholder for user to fill>",
     ])
 
+    # --- Registry Status ---
+    registry_data = load_registry_status(registry_path)
+    if registry_data and best:
+        exp_id = best.get("experiment_id", "")
+        for model in registry_data.get("models", []):
+            if model.get("exp_id") == exp_id:
+                lines.extend([
+                    "",
+                    "## Registry Status",
+                    "",
+                    f"- **Stage:** {model.get('stage', 'unregistered')}",
+                    f"- **Version:** {model.get('version', 'N/A')}",
+                    f"- **Registered:** {model.get('registered_at', 'N/A')[:10]}",
+                    f"- **Gates passed:** {', '.join(model.get('gates_passed', [])) or 'none'}",
+                ])
+                break
+
+    # --- Fairness ---
+    if include_fairness:
+        lines.extend([
+            "",
+            "## Fairness Analysis",
+            "",
+        ])
+        if fairness_data:
+            fairness = compute_fairness_metrics(
+                predictions=fairness_data.get("predictions"),
+                labels=fairness_data.get("labels"),
+                protected_attribute=fairness_data.get("protected_attribute"),
+                group_names=fairness_data.get("group_names"),
+            )
+            if fairness:
+                lines.append("### Demographic Parity")
+                lines.append("")
+                for group, rate in fairness.get("group_positive_rates", {}).items():
+                    lines.append(f"- **{group}:** {rate:.4f}")
+                lines.append(f"- **Parity difference:** {fairness['demographic_parity_difference']:.4f}")
+
+                if "group_true_positive_rates" in fairness:
+                    lines.append("")
+                    lines.append("### Equal Opportunity")
+                    lines.append("")
+                    for group, tpr in fairness["group_true_positive_rates"].items():
+                        lines.append(f"- **{group}:** {tpr:.4f}")
+                    lines.append(f"- **Opportunity difference:** {fairness['equal_opportunity_difference']:.4f}")
+            else:
+                lines.append("- Fairness analysis requested but insufficient data provided")
+        else:
+            lines.append("- Fairness analysis requested but no protected attribute data available")
+            lines.append("- Provide `--fairness-data` with predictions, labels, and protected attributes")
+
     # --- Ethical Considerations ---
     lines.extend([
         "",
@@ -354,9 +497,17 @@ def main() -> None:
     parser.add_argument("--log", default="experiments/log.jsonl", help="Path to experiment log")
     parser.add_argument("--contract", default="model_contract.md", help="Path to model contract")
     parser.add_argument("--output", default=None, help="Output path (default: print to stdout)")
+    parser.add_argument("--include", default=None, help="Include extra sections (e.g., 'fairness')")
+    parser.add_argument("--registry", default="experiments/registry.yaml", help="Path to model registry")
     args = parser.parse_args()
 
-    card = generate_card(args.config, args.log, args.contract, args.output)
+    include_fairness = args.include and "fairness" in args.include
+
+    card = generate_card(
+        args.config, args.log, args.contract, args.output,
+        include_fairness=include_fairness,
+        registry_path=args.registry,
+    )
     if args.output:
         print(f"Model card written to {args.output}")
     else: