dsh-aris-panel 0.1.0

package/skills/result-to-claim/SKILL.md

@@ -0,0 +1,311 @@
+---
+name: result-to-claim
+description: Use when experiments complete to judge what claims the results support, what they don't, and what evidence is still missing. Codex MCP evaluates results against intended claims and routes to next action (pivot, supplement, or confirm). Use after experiments finish — before writing the paper or running ablations.
+argument-hint: "[experiment-description-or-wandb-run]"
+allowed-tools: Bash(*), Read, Grep, Glob, Write, Edit, mcp__codex__codex, mcp__codex__codex-reply
+---
+
+# Result-to-Claim Gate
+
+> 🔒 **Do not wrap this skill in `/loop`, `/schedule`, or `CronCreate`.** It is
+> verdict-bearing — it judges whether results support a claim. Re-running that
+> verdict on a wall-clock timer adds no new signal (the verdict changes only
+> when the *results* change, not when the clock ticks). What you actually want
+> to schedule is the *external wait that precedes it* — experiments done → then
+> run this gate **once**. See
+> [`shared-references/external-cadence.md`](../shared-references/external-cadence.md).
+
+Experiments produce numbers; this gate decides what those numbers *mean*. Collect results from available sources, get a Codex judgment, then auto-route based on the verdict.
+
+## Context: $ARGUMENTS
+
+## When to Use
+
+- After a set of experiments completes (main results, not just sanity checks)
+- Before committing to claims in a paper or review response
+- When results are ambiguous and you need an objective second opinion
+
+## Workflow
+
+### Step 1: Collect Results
+
+Gather experiment data from whatever sources are available in the project:
+
+1. **W&B** (preferred): `wandb.Api().run("<entity>/<project>/<run_id>").history()` — metrics, training curves, comparisons
+2. **EXPERIMENT_LOG.md**: full results table with baselines and verdicts
+3. **EXPERIMENT_TRACKER.md**: check which experiments are DONE vs still running
+4. **Log files**: `ssh server "tail -100 /path/to/training.log"` if no other source
+5. **`idea-stage/docs/research_contract.md`** (legacy fallback: `docs/research_contract.md`): intended claims and experiment design
+
+Assemble the key information:
+- What experiments were run (method, dataset, config)
+- Main metrics and baseline comparisons (deltas)
+- The intended claim these experiments were designed to test
+- Any known confounds or caveats
+
+### Step 1.5: Deterministic evidence pre-check (before spending a Codex call)
+
+For every claim that cites a specific number + a source file, verify the evidence
+*exists* mechanically — no model call — to catch **hallucinated evidence** before
+the jury runs (see [`shared-references/evidence-precheck.md`](../shared-references/evidence-precheck.md)).
+
+**1. Build the claims list.** From the cited numbers and their result files, write
+`[{"id", "value", "source"}, ...]` to `.aris/claims.json` (`source` is the result
+file/glob relative to the project root; `value` is the cited number or string).
+
+**2. Run the pre-check — this is a real step, not a suggestion.** Execute the block
+below (resolver per integration-contract §2, **Policy B**: warn-and-skip if the helper
+is unresolved — never block the audit):
+
+```bash
+# Policy B = warn-and-skip: nothing here may abort the audit. cd is non-fatal, the
+# helper run is explicitly non-blocking, no pipefail-fragile pipe.
+cd "$(git rev-parse --show-toplevel 2>/dev/null || pwd)" 2>/dev/null || true
+if [ -z "${ARIS_REPO:-}" ] && [ -f .aris/installed-skills.txt ]; then
+    ARIS_REPO=$(awk -F'\t' '$1=="repo_root"{print $2; exit}' .aris/installed-skills.txt 2>/dev/null) || true
+fi
+if [ -z "${ARIS_REPO:-}" ] && [ -f "$HOME/.aris/repo" ]; then
+    ARIS_REPO=$(cat "$HOME/.aris/repo" 2>/dev/null) || true
+fi
+EVIDENCE_CHECK=".aris/tools/evidence_check.py"
+[ -f "$EVIDENCE_CHECK" ] || EVIDENCE_CHECK="tools/evidence_check.py"
+[ -f "$EVIDENCE_CHECK" ] || { [ -n "${ARIS_REPO:-}" ] && EVIDENCE_CHECK="$ARIS_REPO/tools/evidence_check.py"; }
+[ -f "$EVIDENCE_CHECK" ] || EVIDENCE_CHECK=""
+
+mkdir -p .aris
+if [ -n "$EVIDENCE_CHECK" ]; then
+    # NB: evidence_check exits 1 when it FINDS hallucinated evidence (value_not_found /
+    # path_missing) — that is the useful signal, NOT a failure. So judge success by
+    # whether valid JSON was produced, never by exit code. `|| true` keeps set -e calm.
+    python3 "$EVIDENCE_CHECK" . --batch .aris/claims.json > .aris/evidence_precheck.json 2>.aris/evidence_precheck.err || true
+    if [ -s .aris/evidence_precheck.json ] && python3 -c "import json,sys;json.load(open('.aris/evidence_precheck.json'))" 2>/dev/null; then
+        cat .aris/evidence_precheck.json
+    else
+        echo "WARN: evidence_check produced no valid output (see .aris/evidence_precheck.err);" >&2
+        echo "      pre-check skipped (Policy B); the Codex jury still runs." >&2
+    fi
+else
+    echo "WARN: evidence_check.py not resolved at .aris/tools/, tools/, \$ARIS_REPO/tools/, or via ~/.aris/repo." >&2
+    echo "      Pre-check skipped (Policy B); the Codex jury still runs. Fix: rerun" >&2
+    echo "      bash tools/install_aris.sh, export ARIS_REPO, or copy the helper to tools/." >&2
+fi
+```
+
+The output is `{"results": [{id, value, source, status, ...}], "summary": {status: n}}`
+with `status ∈ {verified, value_not_found, path_missing, unparseable}`.
+
+**3. Act on the statuses.** Any claim returned `value_not_found` or `path_missing` is
+**hallucinated evidence** — mark it `claim_supported: no` with
+`integrity_status: evidence_not_found` immediately; do NOT spend a Codex call defending a
+number that isn't in the data. `unparseable` claims (no usable value/source) just go to
+the jury normally.
+
+**4. Carry the per-claim status into Step 2.** Feed a small
+`evidence pre-check: <id> → verified | value_not_found | path_missing | unparseable`
+table (from `.aris/evidence_precheck.json`) into the Step-2 Codex prompt so the jury knows
+which claims have real evidence to read. If the pre-check was skipped (helper unresolved),
+say so in that slot rather than omitting it.
+
+`verified` here means only that the cited evidence **exists** — whether it
+**supports** the claim is still the Codex jury's call in Step 2 (a deterministic
+gate DRIVES, it does not ACQUIT).
+
+### Step 2: Codex Judgment
+
+Send the collected results to Codex for objective evaluation. Include ONLY claims that passed the Step 1.5 pre-check — claims already terminally rejected (`evidence_not_found`) keep their deterministic verdict and are NOT re-litigated here:
+
+```
+mcp__codex__codex:
+  model: gpt-5.6-sol
+  config: {"model_reasoning_effort": "ultra"}
+  prompt: |
+    RESULT-TO-CLAIM EVALUATION
+
+    I need you to judge whether experimental results support the intended claim.
+
+    Intended claim: [the claim these experiments test]
+
+    Experiments run:
+    [list experiments with method, dataset, metrics]
+
+    Results:
+    [paste key numbers, comparison deltas, significance]
+
+    Evidence pre-check (deterministic, from Step 1.5):
+    [per-claim: <id> → verified | value_not_found | path_missing.
+     A value_not_found/path_missing means the cited number is NOT in its result
+     file — treat that claim as having no evidence; do not defend it. `verified`
+     means the number exists in the file — YOU still judge whether it supports
+     the claim.]
+
+    Baselines:
+    [baseline numbers and sources — reproduced or from paper]
+
+    Known caveats:
+    [any confounding factors, limited datasets, missing comparisons]
+
+    Please evaluate:
+    1. claim_supported: yes | partial | no
+    2. what_results_support: what the data actually shows
+    3. what_results_dont_support: where the data falls short of the claim
+    4. missing_evidence: specific evidence gaps
+    5. suggested_claim_revision: if the claim should be strengthened, weakened, or reframed
+    6. next_experiments_needed: specific experiments to fill gaps (if any)
+    7. confidence: high | medium | low
+
+    Be honest. Do not inflate claims beyond what the data supports.
+    A single positive result on one dataset does not support a general claim.
+```
+
+### Step 3: Parse and Normalize
+
+Extract structured fields from Codex response:
+
+```markdown
+- claim_supported: yes | partial | no
+- what_results_support: "..."
+- what_results_dont_support: "..."
+- missing_evidence: "..."
+- suggested_claim_revision: "..."
+- next_experiments_needed: "..."
+- confidence: high | medium | low
+```
+
+### Step 3.5: Check Experiment Integrity (if audit exists)
+
+**Skip this step if `EXPERIMENT_AUDIT.json` does not exist.**
+
+```
+if EXPERIMENT_AUDIT.json exists:
+    read integrity_status from file
+    attach to verdict output:
+        integrity_status: pass | warn | fail
+
+    if integrity_status == "fail":
+        append to verdict: "[INTEGRITY CONCERN] — audit found issues, see EXPERIMENT_AUDIT.md"
+        downgrade confidence to "low" regardless of Codex judgment
+
+    if integrity_status == "warn":
+        append to verdict: "[INTEGRITY: WARN] — audit flagged potential issues"
+else:
+    integrity_status = "unavailable"
+    verdict is labeled "provisional — no integrity audit run"
+    (this does NOT block anything — pipeline continues normally)
+```
+
+See `shared-references/experiment-integrity.md` for the full integrity protocol.
+
+### Step 4: Route Based on Verdict
+
+#### `no` — Claim not supported
+
+1. Record postmortem in findings.md (Research Findings section):
+   - What was tested, what failed, hypotheses for why
+   - Constraints for future attempts (what NOT to try again)
+2. Update CLAUDE.md Pipeline Status
+3. Decide whether to pivot to next idea from IDEA_CANDIDATES.md or try an alternative approach
+
+#### `partial` — Claim partially supported
+
+1. Update the working claim to reflect what IS supported
+2. Record the gap in findings.md
+3. Design and run supplementary experiments to fill evidence gaps
+4. Re-run result-to-claim after supplementary experiments complete
+5. **Multiple rounds of `partial` on the same claim** → record analysis in findings.md, consider whether to narrow the claim scope or switch ideas
+
+#### `yes` — Claim supported
+
+1. Record confirmed claim in project notes
+2. If ablation studies are incomplete → trigger `/ablation-planner`
+3. If all evidence is in → ready for paper writing
+
+### Step 5: Update Research Wiki (if active)
+
+**Skip this step entirely if `research-wiki/` does not exist.**
+
+If `research-wiki/` exists, resolve `$WIKI_SCRIPT` per the canonical
+chain documented in
+[`shared-references/wiki-helper-resolution.md`](../shared-references/wiki-helper-resolution.md)
+(Variant B — warn-and-skip for caller skills). The verdict / idea-outcome
+page edits below run on raw markdown and don't need the helper, but edges,
+query-pack rebuild, and the log line do. **This skill never edits a claim's
+`status` field and never creates a claim node** — claims are born (and their
+proof `status` set) by `/proof-checker`; here we only attach experiment edges.
+
+```bash
+cd "$(git rev-parse --show-toplevel 2>/dev/null || pwd)" || exit 1
+ARIS_REPO="${ARIS_REPO:-$(awk -F'\t' '$1=="repo_root"{print $2; exit}' .aris/installed-skills.txt 2>/dev/null)}"
+if [ -z "${ARIS_REPO:-}" ] && [ -f "$HOME/.aris/repo" ]; then
+  ARIS_REPO=$(cat "$HOME/.aris/repo" 2>/dev/null) || true
+fi
+WIKI_SCRIPT=".aris/tools/research_wiki.py"
+[ -f "$WIKI_SCRIPT" ] || WIKI_SCRIPT="tools/research_wiki.py"
+[ -f "$WIKI_SCRIPT" ] || { [ -n "${ARIS_REPO:-}" ] && WIKI_SCRIPT="$ARIS_REPO/tools/research_wiki.py"; }
+[ -f "$WIKI_SCRIPT" ] || {
+  echo "WARN: research_wiki.py not found; verdict will be reported but wiki edges/query-pack/log will be skipped. Fix: bash tools/install_aris.sh or smart_update.sh (refreshes ~/.aris/repo), export ARIS_REPO, or cp <ARIS-repo>/tools/research_wiki.py tools/." >&2
+  WIKI_SCRIPT=""
+}
+```
+
+```
+if research-wiki/ exists:
+    # 1. Create/refresh the experiment node FIRST (verdict OWNER → --update-on-exist so
+    #    a re-judge overwrites the stale verdict). The supports/invalidates edges in #2
+    #    point FROM exp:<id>, and add_edge does NOT verify node existence — so GATE those
+    #    edges on the experiment node having been born (EXP_NODE_OK), else they'd dangle
+    #    (the exact bug this closes). On failure: warn, skip the wiki edges, still report.
+    EXP_NODE_OK=0
+    if [ -n "$WIKI_SCRIPT" ]; then
+      if python3 "$WIKI_SCRIPT" add_experiment research-wiki/ \
+           --slug "<exp_id>" --idea "idea:<active_idea>" \
+           --verdict "<yes|partial|no>" --confidence "<high|medium|low>" \
+           --date "<date>" --hardware "<hw>" --duration "<dur>" \
+           --metrics "<key metrics>" --reasoning "<one-line why this verdict>" \
+           --provenance "<EXPERIMENT_AUDIT.md / run dir>" --update-on-exist; then
+        EXP_NODE_OK=1   # page written + idea--tested_by-->exp edge + index/query_pack rebuilt
+      else
+        echo "WARN: add_experiment failed for <exp_id>; skipping wiki edges (verdict still reported)." >&2
+      fi
+    fi
+
+    # 2. Record empirical support as EDGES ONLY — and ONLY when the exp node was born
+    #    ([ "$EXP_NODE_OK" = 1 ]), so no edge dangles off a missing node. Never edit the
+    #    claim page's `status`: that is the PROOF axis (verified / refuted / unproven /
+    #    sound-modulo-imports / drafted / retracted), owned by /proof-checker (the claim
+    #    birth point) — "supported"/"invalidated" are NOT valid claim statuses. The claim
+    #    target should ALREADY be born by /proof-checker; add_edge does not verify it.
+    for each claim resolved by this verdict (only if [ "$EXP_NODE_OK" = 1 ]):
+        if verdict == "yes":
+            python3 "$WIKI_SCRIPT" add_edge research-wiki/ --from "exp:<id>" --to "claim:<cid>" --type supports --evidence "<metric>"
+        elif verdict == "partial":
+            python3 "$WIKI_SCRIPT" add_edge research-wiki/ --from "exp:<id>" --to "claim:<cid>" --type supports --evidence "partial: <metric>"
+        else:
+            python3 "$WIKI_SCRIPT" add_edge research-wiki/ --from "exp:<id>" --to "claim:<cid>" --type invalidates --evidence "<why>"
+
+    # 3. Update idea outcome (raw markdown, helper-free)
+    Update research-wiki/ideas/<idea_id>.md:
+      - outcome: positive | mixed | negative
+      - If negative: fill "Failure / Risk Notes" and "Lessons Learned"
+      - If positive: fill "Actual Outcome" and "Reusable Components"
+
+    # 4. Rebuild + log (only if $WIKI_SCRIPT resolved)
+    [ -n "$WIKI_SCRIPT" ] && python3 "$WIKI_SCRIPT" rebuild_query_pack research-wiki/
+    [ -n "$WIKI_SCRIPT" ] && python3 "$WIKI_SCRIPT" log research-wiki/ "result-to-claim: exp:<id> verdict=<verdict> for idea:<idea_id>"
+
+    # 5. Re-ideation suggestion
+    Count failed/partial ideas since last /idea-creator run.
+    If >= 3: print "💡 3+ ideas tested since last ideation. Consider re-running /idea-creator — the wiki now knows what doesn't work."
+```
+
+## Rules
+
+- **Codex is the judge, not CC.** CC collects evidence and routes; Codex evaluates. This prevents post-hoc rationalization.
+- Do not inflate claims beyond what the data supports. If Codex says "partial", do not round up to "yes".
+- A single positive result on one dataset does not support a general claim. Be honest about scope.
+- If `confidence` is low, treat the judgment as inconclusive and add experiments rather than committing to a claim.
+- **Fail closed if the reviewer is unavailable.** If the Codex call fails, first walk the capability fallback chain in `shared-references/reviewer-routing.md` (`gpt-5.6-sol`+`ultra` → `gpt-5.6-sol`+`xhigh` → `gpt-5.5`+`xhigh`, capability errors only). If no allowed pair succeeds: write `CLAIMS_FROM_RESULTS.md` containing ONLY the first line `verdict: REVIEW_UNAVAILABLE` (a machine-checkable gate for pipeline callers), record the same in findings.md, and STOP — CC never substitutes its own claim judgment (a loop can drive, never acquit; `acceptance-gate.md`). Downstream steps (wiki `add_experiment` edges, ablation-planner, paper claims) must not consume a run without a Codex verdict. Exception: the deterministic evidence pre-check (Step 1.5) may still terminally mark a claim `claim_supported: no` for hallucinated evidence — a deterministic rejection needs no reviewer; only SUPPORTIVE or ambiguous outcomes require one.
+- Always record the verdict and reasoning in findings.md, regardless of outcome.
+
+## Review Tracing
+
+After each `mcp__codex__codex` or `mcp__codex__codex-reply` reviewer call, save the trace following `shared-references/review-tracing.md` (Policy C — forensic; never silently skip). Use `save_trace.sh` (resolved per the chain in `shared-references/integration-contract.md` §2) or write files directly to `.aris/traces/<skill>/<date>_run<NN>/`. Respect the `--- trace:` parameter (default: `full`).

package/skills/run-experiment/SKILL.md

@@ -0,0 +1,313 @@
+---
+name: run-experiment
+description: Deploy and run ML experiments on local, remote, Vast.ai, or Modal serverless GPU. Use when user says "run experiment", "deploy to server", "跑实验", or needs to launch training jobs.
+argument-hint: "[experiment-description]"
+allowed-tools: Bash(*), Read, Grep, Glob, Edit, Write, Skill(serverless-modal)
+---
+
+# Run Experiment
+
+Deploy and run ML experiment: $ARGUMENTS
+
+## Workflow
+
+### Step 1: Detect Environment
+
+Read the project's `CLAUDE.md` to determine the experiment environment:
+
+- **Local GPU** (`gpu: local`): Look for local CUDA/MPS setup info
+- **Remote server** (`gpu: remote`): Look for SSH alias, conda env, code directory
+- **Vast.ai** (`gpu: vast`): Check for `vast-instances.json` at project root — if a running instance exists, use it. Also check `CLAUDE.md` for a `## Vast.ai` section.
+- **Modal** (`gpu: modal`): Serverless GPU via Modal. No SSH, no Docker, auto scale-to-zero. Delegate to `/serverless-modal`.
+
+**Modal detection:** If `CLAUDE.md` has `gpu: modal` or a `## Modal` section, the entire deployment is handled by `/serverless-modal`. Jump to **Step 4: Deploy (Modal)** — Steps 2-3 are not needed (Modal handles code sync and GPU allocation automatically).
+
+**Environment contract** (`../shared-references/compute-env-contract.md`): before
+building or trusting any environment, read the provider's env ledger
+(`.aris/compute/<provider>.md`) — an unchanged spec hash means warm-reuse, a
+changed one means rebuild. New env → write the declarative spec first, render it
+for this provider's shape, and never declare it ready on import-success alone:
+run the seeded kernel witness, and after any rebuild/doc edit run the
+agent-follows-doc pass (a fresh subagent executes the documented invocation
+verbatim and reports doc-vs-reality divergence).
+
+**Vast.ai detection priority:**
+1. If `CLAUDE.md` has `gpu: vast` or a `## Vast.ai` section:
+   - If `vast-instances.json` exists and has a running instance → use that instance
+   - If no running instance → call `/vast-gpu provision` which analyzes the task, presents cost-optimized GPU options, and rents the user's choice
+2. If no server info is found in `CLAUDE.md`, ask the user.
+
+### Step 2: Pre-flight Check
+
+Check GPU availability on the target machine:
+
+**Remote (SSH):**
+```bash
+ssh <server> nvidia-smi --query-gpu=index,memory.used,memory.total --format=csv,noheader
+```
+
+**Remote (Vast.ai):**
+```bash
+ssh -p <PORT> root@<HOST> nvidia-smi --query-gpu=index,memory.used,memory.total --format=csv,noheader
+```
+(Read `ssh_host` and `ssh_port` from `vast-instances.json`, or run `vastai ssh-url <INSTANCE_ID>` which returns `ssh://root@HOST:PORT`)
+
+**Local:**
+```bash
+nvidia-smi --query-gpu=index,memory.used,memory.total --format=csv,noheader
+# or for Mac MPS:
+python -c "import torch; print('MPS available:', torch.backends.mps.is_available())"
+```
+
+Free GPU = memory.used < 500 MiB.
+
+### Step 3: Sync Code (Remote Only)
+
+Check the project's `CLAUDE.md` for a `code_sync` setting. If not specified, default to `rsync`.
+
+#### Option A: rsync (default)
+
+Only sync necessary files — NOT data, checkpoints, or large files:
+```bash
+rsync -avz --include='*.py' --exclude='*' <local_src>/ <server>:<remote_dst>/
+```
+
+#### Option B: git (when `code_sync: git` is set in CLAUDE.md)
+
+Push local changes to remote repo, then pull on the server:
+```bash
+# 1. Push from local
+git add -A && git commit -m "sync: experiment deployment" && git push
+
+# 2. Pull on server
+ssh <server> "cd <remote_dst> && git pull"
+```
+
+Benefits: version-tracked, multi-server sync with one push, no rsync include/exclude rules needed.
+
+#### Option C: Vast.ai instance
+
+Sync code to the vast.ai instance (always rsync, code dir is `/workspace/project/`):
+```bash
+rsync -avz -e "ssh -p <PORT>" \
+  --include='*.py' --include='*.yaml' --include='*.yml' --include='*.json' \
+  --include='*.txt' --include='*.sh' --include='*/' \
+  --exclude='*.pt' --exclude='*.pth' --exclude='*.ckpt' \
+  --exclude='__pycache__' --exclude='.git' --exclude='data/' \
+  --exclude='wandb/' --exclude='outputs/' \
+  ./ root@<HOST>:/workspace/project/
+```
+
+Install dependencies per the env contract (ordered phases — pins first, one
+`pip install` per phase; see `../shared-references/compute-env-contract.md`):
+```bash
+ssh -p <PORT> root@<HOST> "pip install -q torch==<pinned>"       # phase 1: pins
+ssh -p <PORT> root@<HOST> "pip install -q <remaining packages>"  # phase 2+
+```
+Legacy fallback — `requirements.txt` only, no env spec: install as one phase,
+and treat any version fight as the signal to convert to ordered phases:
+```bash
+scp -P <PORT> requirements.txt root@<HOST>:/workspace/
+ssh -p <PORT> root@<HOST> "pip install -q -r /workspace/requirements.txt"
+```
+
+### Step 3.5: W&B Integration (when `wandb: true` in CLAUDE.md)
+
+**Skip this step entirely if `wandb` is not set or is `false` in CLAUDE.md.**
+
+Before deploying, ensure the experiment scripts have W&B logging:
+
+1. **Check if wandb is already in the script** — look for `import wandb` or `wandb.init`. If present, skip to Step 4.
+
+2. **If not present, add W&B logging** to the training script:
+   ```python
+   import wandb
+   wandb.init(project=WANDB_PROJECT, name=EXP_NAME, config={...hyperparams...})
+
+   # Inside training loop:
+   wandb.log({"train/loss": loss, "train/lr": lr, "step": step})
+
+   # After eval:
+   wandb.log({"eval/loss": eval_loss, "eval/ppl": ppl, "eval/accuracy": acc})
+
+   # At end:
+   wandb.finish()
+   ```
+
+3. **Metrics to log** (add whichever apply to the experiment):
+   - `train/loss` — training loss per step
+   - `train/lr` — learning rate
+   - `eval/loss`, `eval/ppl`, `eval/accuracy` — eval metrics per epoch
+   - `gpu/memory_used` — GPU memory (via `torch.cuda.max_memory_allocated()`)
+   - `speed/samples_per_sec` — throughput
+   - Any custom metrics the experiment already computes
+
+4. **Verify wandb login on the target machine:**
+   ```bash
+   ssh <server> "wandb status"  # should show logged in
+   # If not logged in:
+   ssh <server> "wandb login <WANDB_API_KEY>"
+   ```
+
+> The W&B project name and API key come from `CLAUDE.md` (see example below). The experiment name is auto-generated from the script name + timestamp.
+
+### Step 4: Deploy
+
+#### Remote (via SSH + screen)
+
+For each experiment, create a dedicated screen session with GPU binding:
+```bash
+ssh <server> "screen -dmS <exp_name> bash -c '\
+  eval \"\$(<conda_path>/conda shell.bash hook)\" && \
+  conda activate <env> && \
+  CUDA_VISIBLE_DEVICES=<gpu_id> python <script> <args> 2>&1 | tee <log_file>'"
+```
+
+#### Vast.ai instance
+
+No conda needed — the Docker image has the environment. Use `/workspace/project/` as working dir:
+```bash
+ssh -p <PORT> root@<HOST> "screen -dmS <exp_name> bash -c '\
+  cd /workspace/project && \
+  CUDA_VISIBLE_DEVICES=<gpu_id> python <script> <args> 2>&1 | tee /workspace/<log_file>'"
+```
+
+After launching, update the `experiment` field in `vast-instances.json` for this instance.
+
+#### Modal (serverless)
+
+When `gpu: modal` is detected, delegate to `/serverless-modal`:
+
+1. **Analyze task** — determine VRAM needs, choose GPU, estimate cost
+2. **Generate launcher** — create a `modal_launcher.py` that wraps the training script using `modal.Mount.from_local_dir` for code and `modal.Volume` for results
+3. **Run** — `modal run modal_launcher.py` (runs locally, GPU executes remotely)
+4. **Collect results** — results return via Volume or stdout, no manual download needed
+
+Key Modal settings from `CLAUDE.md`:
+- `modal_gpu`: GPU override (default: auto-select based on VRAM analysis)
+- `modal_timeout`: Max seconds (default: 21600 = 6 hours)
+- `modal_volume`: Named volume for persistent results
+
+No SSH, no code sync, no screen sessions needed. Modal handles everything.
+
+#### Local
+
+```bash
+# Linux with CUDA
+CUDA_VISIBLE_DEVICES=<gpu_id> python <script> <args> 2>&1 | tee <log_file>
+
+# Mac with MPS (PyTorch uses MPS automatically)
+python <script> <args> 2>&1 | tee <log_file>
+```
+
+For local long-running jobs, use `run_in_background: true` to keep the conversation responsive.
+
+### Step 5: Verify Launch
+
+**Remote (SSH):**
+```bash
+ssh <server> "screen -ls"
+```
+
+**Remote (Vast.ai):**
+```bash
+ssh -p <PORT> root@<HOST> "screen -ls"
+```
+
+**Modal:**
+```bash
+modal app list         # Check app is running
+modal app logs <app>   # Stream logs
+```
+
+**Local:**
+Check process is running and GPU is allocated.
+
+### Step 6: Feishu Notification (if configured)
+
+After deployment is verified, check `~/.claude/feishu.json`:
+- Send `experiment_done` notification: which experiments launched, which GPUs, estimated time
+- If config absent or mode `"off"`: skip entirely (no-op)
+
+### Step 7: Auto-Destroy Vast.ai Instance (when `gpu: vast` and `auto_destroy: true`)
+
+**Skip this step if not using vast.ai or `auto_destroy` is `false`.**
+
+After the experiment completes (detected via `/monitor-experiment` or screen session ending):
+
+1. **Download results** from the instance:
+   ```bash
+   rsync -avz -e "ssh -p <PORT>" root@<HOST>:/workspace/project/results/ ./results/
+   ```
+
+2. **Download logs**:
+   ```bash
+   scp -P <PORT> root@<HOST>:/workspace/*.log ./logs/
+   ```
+
+3. **Destroy the instance** to stop billing:
+   ```bash
+   vastai destroy instance <INSTANCE_ID>
+   ```
+
+4. **Update `vast-instances.json`** — mark status as `destroyed`.
+
+5. **Report cost**:
+   ```
+   Vast.ai instance <ID> auto-destroyed.
+   - Duration: ~X.X hours
+   - Estimated cost: ~$X.XX
+   - Results saved to: ./results/
+   ```
+
+> This ensures users are never billed for idle instances. When `auto_destroy: true` (the default), the full lifecycle is automatic: rent → setup → run → collect → destroy.
+
+## Key Rules
+
+- ALWAYS check GPU availability first — never blindly assign GPUs (except Modal, which manages allocation automatically)
+- Each experiment gets its own screen session + GPU (remote) or background process (local)
+- Use `tee` to save logs for later inspection
+- Run deployment commands with `run_in_background: true` to keep conversation responsive
+- Report back: which GPU, which screen/process, what command, estimated time
+- If multiple experiments, launch them in parallel on different GPUs
+- **Vast.ai cost awareness**: When using `gpu: vast`, always report the running cost. If `auto_destroy: true`, destroy the instance as soon as all experiments on it complete
+- **Modal cost awareness**: Always estimate and display cost before running. Modal auto-scales to zero — no idle billing, no manual cleanup
+
+## CLAUDE.md Example
+
+Users should add their server info to their project's `CLAUDE.md`:
+
+```markdown
+## Remote Server
+- gpu: remote               # use pre-configured SSH server
+- SSH: `ssh my-gpu-server`
+- GPU: 4x A100 (80GB each)
+- Conda: `eval "$(/opt/conda/bin/conda shell.bash hook)" && conda activate research`
+- Code dir: `/home/user/experiments/`
+- code_sync: rsync          # default. Or set to "git" for git push/pull workflow
+- wandb: false              # set to "true" to auto-add W&B logging to experiment scripts
+- wandb_project: my-project # W&B project name (required if wandb: true)
+- wandb_entity: my-team     # W&B team/user (optional, uses default if omitted)
+
+## Vast.ai
+- gpu: vast                  # rent on-demand GPU from vast.ai
+- auto_destroy: true         # auto-destroy after experiment completes (default: true)
+- max_budget: 5.00           # optional: max total $ to spend per experiment
+
+## Modal
+- gpu: modal                 # serverless GPU via Modal (no SSH, auto scale-to-zero)
+- modal_gpu: A100-80GB       # optional: override GPU selection (default: auto-select)
+- modal_timeout: 21600       # optional: max seconds (default: 6 hours)
+- modal_volume: my-results   # optional: named volume for results persistence
+
+## Local Environment
+- gpu: local                 # use local GPU
+- Mac MPS / Linux CUDA
+- Conda env: `ml` (Python 3.10 + PyTorch)
+```
+
+> **Vast.ai setup**: Run `pip install vastai && vastai set api-key YOUR_KEY`. Upload your SSH public key at https://cloud.vast.ai/manage-keys/. Set `gpu: vast` in your `CLAUDE.md` — `/run-experiment` will automatically rent an instance, run the experiment, and destroy it when done.
+
+> **Modal setup**: Run `pip install modal && modal setup`. Bind a payment method at https://modal.com/settings (NEVER through CLI) to unlock the full $30/month free tier (without card: $5/month only). Set a workspace spending limit to prevent accidental charges. Set `gpu: modal` in your `CLAUDE.md` — ideal for users without a local GPU who need to debug code or run small-scale tests.
+
+> **W&B setup**: Run `wandb login` on your server once (or set `WANDB_API_KEY` env var). The skill reads project/entity from CLAUDE.md and adds `wandb.init()` + `wandb.log()` to your training scripts automatically. Dashboard: `https://wandb.ai/<entity>/<project>`.