@research-copilot/plugin 1.1.15

package/dist/skills/experiment-design/SKILL.md

@@ -0,0 +1,331 @@
+---
+name: experiment-design
+description: Designs and launches experiments. Handles long-running tasks via Monitor. Use for experiment tasks.
+triggers:
+  - "design experiment"
+  - "run training"
+  - "launch experiment"
+  - "run experiments"
+  - "execute experiments"
+---
+
+# Experiment Design
+
+Orchestrate experiment design and execution: create task → dispatch @rc-experiment → monitor long-running jobs → verify results.
+
+## When to Use
+
+Use this skill when:
+- User asks to "design experiment for X"
+- User wants to "run training" or "launch experiment"
+- User needs to "execute experiments" for research
+- Standalone experiment execution (NOT part of full pipeline)
+
+Do NOT use when:
+- Part of full-research-workflow (that handles experiments internally)
+- User only wants to analyze existing results (use data-analysis skill)
+
+## Task-First Protocol
+
+Before starting, check if task exists:
+
+```powershell
+# Check for existing experiment task
+$taskFile = "C:\PythonProject\research_copilot\.rc\tasks\experiment-design.json"
+if (Test-Path $taskFile) {
+    $task = Get-Content $taskFile | ConvertFrom-Json
+    Write-Host "Found existing task: $($task.experiment_name)"
+} else {
+    Write-Host "No existing task. Will create one."
+}
+```
+
+Create task if needed:
+
+```powershell
+# Create experiment design task
+rc task create --type experiment --name "user experiment name" --output .rc/tasks/experiment-design.json
+```
+
+## Auto-Context Loading
+
+Read task context and relevant specs before orchestration:
+
+```powershell
+# Load task context
+$taskFile = "C:\PythonProject\research_copilot\.rc\tasks\experiment-design.json"
+if (Test-Path $taskFile) {
+    $task = Get-Content $taskFile | ConvertFrom-Json
+    
+    Write-Host "Experiment Name: $($task.experiment_name)"
+    Write-Host "Target Metrics: $($task.target_metrics)"
+    Write-Host "Baseline Comparisons: $($task.baseline_count)"
+    
+    # Load PRD if referenced
+    if ($task.prd_file -and (Test-Path $task.prd_file)) {
+        $prd = Get-Content $task.prd_file -Raw
+        Write-Host "PRD loaded: $($task.prd_file)"
+        
+        # Extract metrics requirements from PRD
+        $metricsMatch = [regex]::Match($prd, '## Target Metrics\s+([\s\S]+?)(?=\n##|\z)')
+        if ($metricsMatch.Success) {
+            Write-Host "Target metrics from PRD:"
+            Write-Host $metricsMatch.Groups[1].Value
+        }
+    }
+    
+    # Load methodology specs if available
+    $methodFile = "C:\PythonProject\research_copilot\.rc\methodology.md"
+    if (Test-Path $methodFile) {
+        $method = Get-Content $methodFile -Raw
+        Write-Host "Methodology loaded: $methodFile"
+    }
+}
+```
+
+## Orchestration Logic
+
+Execute experiment via agent dispatch with long-task support:
+
+```powershell
+# Start task
+rc task start experiment-design
+
+# Dispatch to experiment agent
+Write-Host "Dispatching to @rc-experiment agent..."
+
+# Agent will:
+# 1. Parse experiment requirements from PRD
+# 2. Design experiment configuration (hyperparameters, seeds, splits)
+# 3. Set up experiment directory structure
+# 4. Launch training jobs (using Monitor for long-running tasks)
+# 5. Collect metrics and results
+# 6. Compare against baselines
+# 7. Generate result tables and figures
+# 8. Save all artifacts to .rc/experiments/
+
+# Check status (agent runs autonomously)
+$status = rc task status experiment-design
+Write-Host "Task status: $status"
+```
+
+## Long-Task Handling
+
+The experiment executor uses `Bash(run_in_background=true)` + `Monitor` for training jobs:
+
+```powershell
+# Example: Monitor long-running training job
+# The agent will set this up internally, shown here for reference
+
+# Start training in background
+$jobId = Start-Job -ScriptBlock {
+    python train.py --config config.yaml --output experiments/run_001/
+}
+
+# Monitor training progress via log file
+Monitor -description "Training experiment run_001" `
+        -persistent $true `
+        -command @"
+tail -f experiments/run_001/train.log | grep --line-buffered 'epoch=\|loss=\|accuracy=\|FINISHED\|ERROR\|FAILED'
+"@
+
+# Agent continues other work while training runs
+# Will be notified when training completes or errors occur
+```
+
+Key points for long-task handling:
+- Training jobs run in background using `run_in_background=true`
+- Monitor streams progress updates (metrics, errors) without blocking
+- Agent can launch multiple experiments in parallel
+- Each experiment logs to separate directory for isolation
+- Failed experiments recorded with error details for debugging
+
+## Verification and Quality Gates
+
+Verify results before marking complete:
+
+```powershell
+# Verify experiment outputs
+$expDir = "C:\PythonProject\research_copilot\.rc\experiments"
+
+# Check experiment config
+$configFile = Join-Path $expDir "config.yaml"
+if (Test-Path $configFile) {
+    $config = Get-Content $configFile -Raw | ConvertFrom-Yaml
+    Write-Host "Config loaded: seed=$($config.seed), epochs=$($config.epochs)"
+} else {
+    Write-Host "WARNING: No config file found"
+}
+
+# Check results file
+$resultsFile = Join-Path $expDir "results.json"
+if (Test-Path $resultsFile) {
+    $results = Get-Content $resultsFile | ConvertFrom-Json
+    Write-Host "Found results for $($results.runs.Count) runs"
+    
+    # Verify all required metrics present
+    $requiredMetrics = @('accuracy', 'f1_score', 'precision', 'recall')
+    foreach ($metric in $requiredMetrics) {
+        if (-not $results.metrics.$metric) {
+            Write-Host "WARNING: Missing metric: $metric"
+        }
+    }
+} else {
+    Write-Host "WARNING: No results file found"
+}
+
+# Check baseline comparisons
+$comparisonFile = Join-Path $expDir "baseline_comparison.json"
+if (Test-Path $comparisonFile) {
+    $comparison = Get-Content $comparisonFile | ConvertFrom-Json
+    Write-Host "Compared against $($comparison.baselines.Count) baselines"
+} else {
+    Write-Host "WARNING: No baseline comparison found"
+}
+
+# Check reproducibility artifacts
+$artifactsDir = Join-Path $expDir "artifacts"
+if (Test-Path $artifactsDir) {
+    $artifacts = Get-ChildItem $artifactsDir -Recurse
+    Write-Host "Saved $($artifacts.Count) reproducibility artifacts"
+} else {
+    Write-Host "WARNING: No artifacts directory"
+}
+```
+
+Complete task if quality gates pass:
+
+```powershell
+# Mark task complete
+rc task complete experiment-design
+```
+
+## Quality Gates
+
+Verify before marking complete:
+
+```powershell
+# Quality gate checks
+$passed = $true
+
+# Gate 1: All metrics from PRD achieved
+$resultsFile = "C:\PythonProject\research_copilot\.rc\experiments\results.json"
+if (Test-Path $resultsFile) {
+    $results = Get-Content $resultsFile | ConvertFrom-Json
+    $prdFile = "C:\PythonProject\research_copilot\.rc\prd.md"
+    
+    if (Test-Path $prdFile) {
+        $prd = Get-Content $prdFile -Raw
+        $metricsMatch = [regex]::Matches($prd, '(\w+):\s*≥\s*([\d.]+)')
+        
+        foreach ($match in $metricsMatch) {
+            $metricName = $match.Groups[1].Value
+            $targetValue = [double]$match.Groups[2].Value
+            $actualValue = $results.metrics.$metricName
+            
+            if ($actualValue -lt $targetValue) {
+                Write-Host "FAIL: $metricName = $actualValue < $targetValue (target)"
+                $passed = $false
+            }
+        }
+    }
+} else {
+    Write-Host "FAIL: No results file"
+    $passed = $false
+}
+
+# Gate 2: Results logged to artifacts/results/
+$artifactsDir = "C:\PythonProject\research_copilot\.rc\experiments\artifacts\results"
+if (Test-Path $artifactsDir) {
+    $resultFiles = Get-ChildItem $artifactsDir -Filter "*.json"
+    if ($resultFiles.Count -eq 0) {
+        Write-Host "FAIL: No result files in artifacts/results/"
+        $passed = $false
+    }
+} else {
+    Write-Host "FAIL: artifacts/results/ directory missing"
+    $passed = $false
+}
+
+# Gate 3: Config and seed recorded for reproducibility
+$configFile = "C:\PythonProject\research_copilot\.rc\experiments\config.yaml"
+if (Test-Path $configFile) {
+    $config = Get-Content $configFile -Raw
+    if ($config -notmatch 'seed:\s*\d+') {
+        Write-Host "FAIL: No seed recorded in config"
+        $passed = $false
+    }
+} else {
+    Write-Host "FAIL: No config file for reproducibility"
+    $passed = $false
+}
+
+if ($passed) {
+    Write-Host "All quality gates passed"
+} else {
+    Write-Host "Quality gates failed - review needed"
+}
+```
+
+## Report Format
+
+Generate final report:
+
+```markdown
+# Experiment Design Report
+
+**Experiment**: {experiment_name}
+**Date**: {date}
+**Runs Completed**: {run_count}
+**Status**: {status}
+
+## Configuration
+
+- **Seed**: {seed}
+- **Epochs**: {epochs}
+- **Batch Size**: {batch_size}
+- **Learning Rate**: {learning_rate}
+
+## Results
+
+| Metric | Value | Target | Status |
+|--------|-------|--------|--------|
+| Accuracy | {accuracy} | {target_accuracy} | {✓/✗} |
+| F1 Score | {f1_score} | {target_f1} | {✓/✗} |
+| Precision | {precision} | {target_precision} | {✓/✗} |
+| Recall | {recall} | {target_recall} | {✓/✗} |
+
+## Baseline Comparisons
+
+| Baseline | Metric | Their Score | Our Score | Improvement |
+|----------|--------|-------------|-----------|-------------|
+| {name} | {metric} | {baseline_score} | {our_score} | {delta}% |
+
+## Reproducibility
+
+All artifacts saved to: `.rc/experiments/artifacts/`
+
+- **Config**: `config.yaml` (seed={seed})
+- **Checkpoints**: `checkpoints/run_{id}/`
+- **Logs**: `logs/train.log`
+- **Results**: `artifacts/results/results_{timestamp}.json`
+
+## Next Steps
+
+- [ ] Verify results meet all PRD requirements
+- [ ] Generate figures for paper
+- [ ] Update paper draft with results
+- [ ] Archive experiment for future reference
+```
+
+## Example Usage
+
+```
+User: "Design and run experiments for the sentiment analysis model"
+
+Skill: [Reads PRD, creates task, dispatches @rc-experiment]
+Skill: [Agent launches training via Monitor, continues other work]
+Skill: [Receives notification when training completes]
+Skill: [Verifies metrics, generates report]
+Skill: "Experiment completed. Achieved accuracy=0.94 (target: 0.90). Results saved to .rc/experiments/."
+```

package/dist/skills/full-research-workflow/SKILL.md

@@ -0,0 +1,363 @@
+---
+name: full-research-workflow
+description: |
+  Orchestrate a complete research workflow from literature review to camera-ready paper.
+  Calls `rc` CLI commands to create tasks, dispatches agents for execution, verifies quality gates.
+  Follows Trellis philosophy: Task-first, Action-before-asking, Single source of truth.
+triggers:
+  - "run the full research workflow"
+  - "orchestrate research from start to finish"
+  - "execute the complete research pipeline"
+  - "run all research stages"
+---
+
+# Full Research Workflow
+
+Orchestrates a complete research workflow: literature → ideation → experiment → writing → polish → review → rebuttal.
+
+## When to Use
+
+Use this skill when:
+- User wants to run the entire research pipeline end-to-end
+- User says "run the full workflow", "execute all stages", "orchestrate research"
+- A `prd.md` exists with research goals and you need to execute all stages
+
+Do NOT use when:
+- User wants to run a single stage (use stage-specific skills instead)
+- No `prd.md` exists (guide user to create one first)
+
+## Task-First Principle
+
+If `prd.md` exists and has clear goals, proceed directly to Stage 1. Only ask questions if critical information is missing.
+
+## Auto-Context (Action-First)
+
+**Step 0**: Before asking clarifying questions, read the context files that might contain answers:
+
+```powershell
+# Check for PRD and existing state
+if (Test-Path prd.md) { Get-Content prd.md } else { "No PRD found" }
+rc task list --json 2>$null || "No tasks yet"
+if (Test-Path execute.jsonl) { "Execution log exists" } else { "No execution log" }
+if ((Test-Path baselines/) -and (Test-Path venue/)) { "Support dirs ready" } else { "No support dirs" }
+```
+
+**What to look for**:
+- `prd.md`: Research goals, target venue, constraints
+- `execute.jsonl`: Previous execution history, failed stages
+- Task list: Which stages are already complete/in-progress
+- Support dirs (`baselines/`, `venue/`): Whether infrastructure is ready
+
+**Decision tree**:
+1. If `prd.md` missing → Ask user to create one or provide research goals
+2. If `prd.md` exists but tasks already in progress → Resume from last incomplete stage
+3. If `prd.md` exists and no tasks → Start fresh from Stage 1
+
+Only ask clarifying questions if critical information is genuinely missing from these files.
+
+## Workflow Stages
+
+### Stage 1: Literature Review
+
+```powershell
+# Create literature review task and capture ID
+$output = rc task create `
+  --kind literature `
+  --title "Literature Review" `
+  --goal "Survey state-of-the-art in [research area from prd.md]"
+$LIT_TASK_ID = ($output | Select-String 'Task (\d+)').Matches.Groups[1].Value
+
+# Dispatch agent to execute
+rc agent dispatch --task-id $LIT_TASK_ID --agent literature-agent
+
+# Verify deliverables exist
+if (-not (Test-Path lit_review.md)) { "ERROR: lit_review.md missing" }
+if (-not (Test-Path references/)) { "ERROR: references/ missing" }
+
+# Mark complete
+rc task complete --task-id $LIT_TASK_ID
+```
+
+Quality gate: `lit_review.md` must exist with 15+ references, clear gaps identified.
+
+### Stage 2: Ideation
+
+```powershell
+# Verify literature task is complete
+$status = rc task status $LIT_TASK_ID
+if ($status -notmatch "completed") {
+  "ERROR: Literature task not completed yet"
+  exit 1
+}
+
+# Create ideation task (depends on literature)
+$output = rc task create `
+  --kind ideation `
+  --title "Ideation" `
+  --goal "Generate research ideas addressing gaps from literature" `
+  --depends-on $LIT_TASK_ID
+$IDEA_TASK_ID = ($output | Select-String 'Task (\d+)').Matches.Groups[1].Value
+
+# Dispatch agent
+rc agent dispatch --task-id $IDEA_TASK_ID --agent ideation-agent
+
+# Verify deliverables
+if (-not (Test-Path ideas.md)) { "ERROR: ideas.md missing" }
+if (-not (Select-String -Path ideas.md -Pattern "idea_" -Quiet)) { "ERROR: No structured ideas found" }
+
+# Mark complete
+rc task complete --task-id $IDEA_TASK_ID
+```
+
+Quality gate: `ideas.md` must contain 3+ structured ideas with novelty scores.
+
+### Stage 3: Experiment
+
+```powershell
+# Verify ideation task is complete
+$status = rc task status $IDEA_TASK_ID
+if ($status -notmatch "completed") {
+  "ERROR: Ideation task not completed yet"
+  exit 1
+}
+
+# Create experiment task (depends on ideation)
+$output = rc task create `
+  --kind experiment `
+  --title "Experiment Execution" `
+  --goal "Implement and run experiments for selected idea" `
+  --depends-on $IDEA_TASK_ID
+$EXP_TASK_ID = ($output | Select-String 'Task (\d+)').Matches.Groups[1].Value
+
+# Dispatch agent
+rc agent dispatch --task-id $EXP_TASK_ID --agent experiment-agent
+
+# Verify deliverables
+if (-not (Test-Path results.json)) { "ERROR: results.json missing" }
+if (-not (Test-Path figures/)) { "ERROR: figures/ missing" }
+if (-not (Test-Path experiment_log.md)) { "ERROR: experiment_log.md missing" }
+
+# Mark complete
+rc task complete --task-id $EXP_TASK_ID
+```
+
+Quality gate: `results.json` must have baseline comparisons, `figures/` must contain plots.
+
+### Stage 4: Writing
+
+```powershell
+# Verify experiment task is complete
+$status = rc task status $EXP_TASK_ID
+if ($status -notmatch "completed") {
+  "ERROR: Experiment task not completed yet"
+  exit 1
+}
+
+# Create writing task (depends on experiment)
+$output = rc task create `
+  --kind writing `
+  --title "Draft Paper" `
+  --goal "Write paper draft following venue template" `
+  --depends-on $EXP_TASK_ID
+$WRITE_TASK_ID = ($output | Select-String 'Task (\d+)').Matches.Groups[1].Value
+
+# Dispatch agent
+rc agent dispatch --task-id $WRITE_TASK_ID --agent writing-agent
+
+# Verify deliverables
+if (-not (Test-Path paper_draft.tex)) { "ERROR: paper_draft.tex missing" }
+pdflatex paper_draft.tex >$null 2>&1
+if ($LASTEXITCODE -ne 0) { "WARN: LaTeX compilation failed" }
+
+# Mark complete
+rc task complete --task-id $WRITE_TASK_ID
+```
+
+Quality gate: `paper_draft.tex` must compile, all sections present, references formatted.
+
+### Stage 5: Polish
+
+```powershell
+# Verify writing task is complete
+$status = rc task status $WRITE_TASK_ID
+if ($status -notmatch "completed") {
+  "ERROR: Writing task not completed yet"
+  exit 1
+}
+
+# Create polish task (depends on writing)
+$output = rc task create `
+  --kind polish `
+  --title "Polish Paper" `
+  --goal "Refine writing, check formatting, validate claims" `
+  --depends-on $WRITE_TASK_ID
+$POLISH_TASK_ID = ($output | Select-String 'Task (\d+)').Matches.Groups[1].Value
+
+# Dispatch agent
+rc agent dispatch --task-id $POLISH_TASK_ID --agent polish-agent
+
+# Verify deliverables
+if (-not (Test-Path paper_polished.tex)) { "ERROR: paper_polished.tex missing" }
+if (-not (Select-String -Path paper_polished.tex -Pattern "\\cite{" -Quiet)) { "WARN: No citations found" }
+
+# Mark complete
+rc task complete --task-id $POLISH_TASK_ID
+```
+
+Quality gate: Paper length within venue limits, all claims cited, figures captioned.
+
+### Stage 6: Review
+
+```powershell
+# Verify polish task is complete
+$status = rc task status $POLISH_TASK_ID
+if ($status -notmatch "completed") {
+  "ERROR: Polish task not completed yet"
+  exit 1
+}
+
+# Create review task (depends on polish)
+$output = rc task create `
+  --kind review `
+  --title "Internal Review" `
+  --goal "Simulate peer review, identify weaknesses" `
+  --depends-on $POLISH_TASK_ID
+$REVIEW_TASK_ID = ($output | Select-String 'Task (\d+)').Matches.Groups[1].Value
+
+# Dispatch agent
+rc agent dispatch --task-id $REVIEW_TASK_ID --agent review-agent
+
+# Verify deliverables
+if (-not (Test-Path review_report.md)) { "ERROR: review_report.md missing" }
+if (-not (Select-String -Path review_report.md -Pattern "weakness_" -Quiet)) { "ERROR: No structured weaknesses" }
+
+# Mark complete
+rc task complete --task-id $REVIEW_TASK_ID
+```
+
+Quality gate: `review_report.md` must identify 3+ weaknesses with severity scores.
+
+### Stage 7: Rebuttal (Optional)
+
+```powershell
+# Only run if review found critical issues
+if (Select-String -Path review_report.md -Pattern "severity: critical" -Quiet) {
+  # Verify review task is complete
+  $status = rc task status $REVIEW_TASK_ID
+  if ($status -notmatch "completed") {
+    "ERROR: Review task not completed yet"
+    exit 1
+  }
+  
+  $output = rc task create `
+    --kind rebuttal `
+    --title "Address Review Comments" `
+    --goal "Revise paper based on review feedback" `
+    --depends-on $REVIEW_TASK_ID
+  $REBUTTAL_TASK_ID = ($output | Select-String 'Task (\d+)').Matches.Groups[1].Value
+  
+  rc agent dispatch --task-id $REBUTTAL_TASK_ID --agent rebuttal-agent
+  
+  if (-not (Test-Path paper_revised.tex)) { "ERROR: paper_revised.tex missing" }
+  if (-not (Test-Path rebuttal_response.md)) { "ERROR: rebuttal_response.md missing" }
+  
+  rc task complete --task-id $REBUTTAL_TASK_ID
+}
+```
+
+Quality gate: All critical weaknesses addressed, rebuttal response provided.
+
+## Error Recovery
+
+### Executor Fails
+
+If `rc agent dispatch` fails or agent reports errors:
+
+1. Check `execute.jsonl` for the failure record
+2. Read the error message and context
+3. Record a gap in the task notes:
+   ```powershell
+   rc task update --task-id $TASK_ID `
+     --add-note "Agent failed: [error]. Manual intervention needed."
+   ```
+4. Ask user: "The [stage] agent failed with error: [error]. Would you like me to retry with different parameters, or handle this manually?"
+
+### Quality Gate Fails
+
+If deliverables are missing or malformed:
+
+1. Set task status back to `in_progress`:
+   ```powershell
+   rc task update --task-id $TASK_ID --status in_progress
+   ```
+2. Re-dispatch agent with explicit instructions:
+   ```powershell
+   rc agent dispatch --task-id $TASK_ID `
+     --agent <agent-name> `
+     --instruction "Previous run failed quality gate: [details]. Focus on [missing deliverable]."
+   ```
+3. If fails twice, ask user for guidance
+
+### MCP Unavailable
+
+If `rc` CLI commands fail because MCP server is not running:
+
+1. Record the gap: "MCP server unavailable, cannot create tasks programmatically"
+2. Provide manual fallback:
+   ```powershell
+   # Manual task creation
+   New-Item -ItemType Directory -Force -Path .research/tasks/
+   @"
+   {
+     "kind": "literature",
+     "title": "Literature Review",
+     "status": "pending"
+   }
+   "@ | Out-File -FilePath .research/tasks/literature.json -Encoding utf8
+   ```
+3. Ask user: "The `rc` CLI is unavailable. Should I proceed with manual task tracking?"
+
+## Report Format
+
+After all stages complete (or when blocked), provide a summary:
+
+```
+Research Workflow Summary
+========================
+
+Completed Stages:
+✓ Stage 1: Literature Review (lit_review.md, 18 references)
+✓ Stage 2: Ideation (ideas.md, 4 ideas generated)
+✓ Stage 3: Experiment (results.json, 3 baselines compared)
+✓ Stage 4: Writing (paper_draft.tex, 8 pages)
+✓ Stage 5: Polish (paper_polished.tex, formatting validated)
+✓ Stage 6: Review (review_report.md, 2 critical weaknesses found)
+✗ Stage 7: Rebuttal (skipped - no critical issues OR in progress)
+
+Deliverables:
+- Paper: paper_polished.tex (or paper_revised.tex if rebuttal ran)
+- Figures: figures/ (N plots)
+- References: references/ (M papers)
+- Execution log: execute.jsonl (full trace)
+
+Quality Gates Passed:
+- Literature: 15+ references ✓
+- Ideation: 3+ ideas ✓
+- Experiment: Baseline comparisons ✓
+- Writing: LaTeX compiles ✓
+- Polish: Within page limits ✓
+- Review: Weaknesses identified ✓
+
+Next Steps:
+- Review paper_polished.tex
+- Address any remaining review comments manually
+- Prepare submission package
+```
+
+If any stage failed, include:
+
+```
+Blockers:
+- Stage X failed: [error message]
+- Manual intervention needed: [specific action]
+```