@research-copilot/plugin 1.1.15 → 1.1.17

package/dist/skills/experiment-design/SKILL.md

@@ -1,331 +0,0 @@
----
-name: experiment-design
-description: Designs and launches experiments. Handles long-running tasks via Monitor. Use for experiment tasks.
-triggers:
-  - "design experiment"
-  - "run training"
-  - "launch experiment"
-  - "run experiments"
-  - "execute experiments"
----
-
-# Experiment Design
-
-Orchestrate experiment design and execution: create task → dispatch @rc-experiment → monitor long-running jobs → verify results.
-
-## When to Use
-
-Use this skill when:
-- User asks to "design experiment for X"
-- User wants to "run training" or "launch experiment"
-- User needs to "execute experiments" for research
-- Standalone experiment execution (NOT part of full pipeline)
-
-Do NOT use when:
-- Part of full-research-workflow (that handles experiments internally)
-- User only wants to analyze existing results (use data-analysis skill)
-
-## Task-First Protocol
-
-Before starting, check if task exists:
-
-```powershell
-# Check for existing experiment task
-$taskFile = "C:\PythonProject\research_copilot\.rc\tasks\experiment-design.json"
-if (Test-Path $taskFile) {
-    $task = Get-Content $taskFile | ConvertFrom-Json
-    Write-Host "Found existing task: $($task.experiment_name)"
-} else {
-    Write-Host "No existing task. Will create one."
-}
-```
-
-Create task if needed:
-
-```powershell
-# Create experiment design task
-rc task create --type experiment --name "user experiment name" --output .rc/tasks/experiment-design.json
-```
-
-## Auto-Context Loading
-
-Read task context and relevant specs before orchestration:
-
-```powershell
-# Load task context
-$taskFile = "C:\PythonProject\research_copilot\.rc\tasks\experiment-design.json"
-if (Test-Path $taskFile) {
-    $task = Get-Content $taskFile | ConvertFrom-Json
-    
-    Write-Host "Experiment Name: $($task.experiment_name)"
-    Write-Host "Target Metrics: $($task.target_metrics)"
-    Write-Host "Baseline Comparisons: $($task.baseline_count)"
-    
-    # Load PRD if referenced
-    if ($task.prd_file -and (Test-Path $task.prd_file)) {
-        $prd = Get-Content $task.prd_file -Raw
-        Write-Host "PRD loaded: $($task.prd_file)"
-        
-        # Extract metrics requirements from PRD
-        $metricsMatch = [regex]::Match($prd, '## Target Metrics\s+([\s\S]+?)(?=\n##|\z)')
-        if ($metricsMatch.Success) {
-            Write-Host "Target metrics from PRD:"
-            Write-Host $metricsMatch.Groups[1].Value
-        }
-    }
-    
-    # Load methodology specs if available
-    $methodFile = "C:\PythonProject\research_copilot\.rc\methodology.md"
-    if (Test-Path $methodFile) {
-        $method = Get-Content $methodFile -Raw
-        Write-Host "Methodology loaded: $methodFile"
-    }
-}
-```
-
-## Orchestration Logic
-
-Execute experiment via agent dispatch with long-task support:
-
-```powershell
-# Start task
-rc task start experiment-design
-
-# Dispatch to experiment agent
-Write-Host "Dispatching to @rc-experiment agent..."
-
-# Agent will:
-# 1. Parse experiment requirements from PRD
-# 2. Design experiment configuration (hyperparameters, seeds, splits)
-# 3. Set up experiment directory structure
-# 4. Launch training jobs (using Monitor for long-running tasks)
-# 5. Collect metrics and results
-# 6. Compare against baselines
-# 7. Generate result tables and figures
-# 8. Save all artifacts to .rc/experiments/
-
-# Check status (agent runs autonomously)
-$status = rc task status experiment-design
-Write-Host "Task status: $status"
-```
-
-## Long-Task Handling
-
-The experiment executor uses `Bash(run_in_background=true)` + `Monitor` for training jobs:
-
-```powershell
-# Example: Monitor long-running training job
-# The agent will set this up internally, shown here for reference
-
-# Start training in background
-$jobId = Start-Job -ScriptBlock {
-    python train.py --config config.yaml --output experiments/run_001/
-}
-
-# Monitor training progress via log file
-Monitor -description "Training experiment run_001" `
-        -persistent $true `
-        -command @"
-tail -f experiments/run_001/train.log | grep --line-buffered 'epoch=\|loss=\|accuracy=\|FINISHED\|ERROR\|FAILED'
-"@
-
-# Agent continues other work while training runs
-# Will be notified when training completes or errors occur
-```
-
-Key points for long-task handling:
-- Training jobs run in background using `run_in_background=true`
-- Monitor streams progress updates (metrics, errors) without blocking
-- Agent can launch multiple experiments in parallel
-- Each experiment logs to separate directory for isolation
-- Failed experiments recorded with error details for debugging
-
-## Verification and Quality Gates
-
-Verify results before marking complete:
-
-```powershell
-# Verify experiment outputs
-$expDir = "C:\PythonProject\research_copilot\.rc\experiments"
-
-# Check experiment config
-$configFile = Join-Path $expDir "config.yaml"
-if (Test-Path $configFile) {
-    $config = Get-Content $configFile -Raw | ConvertFrom-Yaml
-    Write-Host "Config loaded: seed=$($config.seed), epochs=$($config.epochs)"
-} else {
-    Write-Host "WARNING: No config file found"
-}
-
-# Check results file
-$resultsFile = Join-Path $expDir "results.json"
-if (Test-Path $resultsFile) {
-    $results = Get-Content $resultsFile | ConvertFrom-Json
-    Write-Host "Found results for $($results.runs.Count) runs"
-    
-    # Verify all required metrics present
-    $requiredMetrics = @('accuracy', 'f1_score', 'precision', 'recall')
-    foreach ($metric in $requiredMetrics) {
-        if (-not $results.metrics.$metric) {
-            Write-Host "WARNING: Missing metric: $metric"
-        }
-    }
-} else {
-    Write-Host "WARNING: No results file found"
-}
-
-# Check baseline comparisons
-$comparisonFile = Join-Path $expDir "baseline_comparison.json"
-if (Test-Path $comparisonFile) {
-    $comparison = Get-Content $comparisonFile | ConvertFrom-Json
-    Write-Host "Compared against $($comparison.baselines.Count) baselines"
-} else {
-    Write-Host "WARNING: No baseline comparison found"
-}
-
-# Check reproducibility artifacts
-$artifactsDir = Join-Path $expDir "artifacts"
-if (Test-Path $artifactsDir) {
-    $artifacts = Get-ChildItem $artifactsDir -Recurse
-    Write-Host "Saved $($artifacts.Count) reproducibility artifacts"
-} else {
-    Write-Host "WARNING: No artifacts directory"
-}
-```
-
-Complete task if quality gates pass:
-
-```powershell
-# Mark task complete
-rc task complete experiment-design
-```
-
-## Quality Gates
-
-Verify before marking complete:
-
-```powershell
-# Quality gate checks
-$passed = $true
-
-# Gate 1: All metrics from PRD achieved
-$resultsFile = "C:\PythonProject\research_copilot\.rc\experiments\results.json"
-if (Test-Path $resultsFile) {
-    $results = Get-Content $resultsFile | ConvertFrom-Json
-    $prdFile = "C:\PythonProject\research_copilot\.rc\prd.md"
-    
-    if (Test-Path $prdFile) {
-        $prd = Get-Content $prdFile -Raw
-        $metricsMatch = [regex]::Matches($prd, '(\w+):\s*≥\s*([\d.]+)')
-        
-        foreach ($match in $metricsMatch) {
-            $metricName = $match.Groups[1].Value
-            $targetValue = [double]$match.Groups[2].Value
-            $actualValue = $results.metrics.$metricName
-            
-            if ($actualValue -lt $targetValue) {
-                Write-Host "FAIL: $metricName = $actualValue < $targetValue (target)"
-                $passed = $false
-            }
-        }
-    }
-} else {
-    Write-Host "FAIL: No results file"
-    $passed = $false
-}
-
-# Gate 2: Results logged to artifacts/results/
-$artifactsDir = "C:\PythonProject\research_copilot\.rc\experiments\artifacts\results"
-if (Test-Path $artifactsDir) {
-    $resultFiles = Get-ChildItem $artifactsDir -Filter "*.json"
-    if ($resultFiles.Count -eq 0) {
-        Write-Host "FAIL: No result files in artifacts/results/"
-        $passed = $false
-    }
-} else {
-    Write-Host "FAIL: artifacts/results/ directory missing"
-    $passed = $false
-}
-
-# Gate 3: Config and seed recorded for reproducibility
-$configFile = "C:\PythonProject\research_copilot\.rc\experiments\config.yaml"
-if (Test-Path $configFile) {
-    $config = Get-Content $configFile -Raw
-    if ($config -notmatch 'seed:\s*\d+') {
-        Write-Host "FAIL: No seed recorded in config"
-        $passed = $false
-    }
-} else {
-    Write-Host "FAIL: No config file for reproducibility"
-    $passed = $false
-}
-
-if ($passed) {
-    Write-Host "All quality gates passed"
-} else {
-    Write-Host "Quality gates failed - review needed"
-}
-```
-
-## Report Format
-
-Generate final report:
-
-```markdown
-# Experiment Design Report
-
-**Experiment**: {experiment_name}
-**Date**: {date}
-**Runs Completed**: {run_count}
-**Status**: {status}
-
-## Configuration
-
-- **Seed**: {seed}
-- **Epochs**: {epochs}
-- **Batch Size**: {batch_size}
-- **Learning Rate**: {learning_rate}
-
-## Results
-
-| Metric | Value | Target | Status |
-|--------|-------|--------|--------|
-| Accuracy | {accuracy} | {target_accuracy} | {✓/✗} |
-| F1 Score | {f1_score} | {target_f1} | {✓/✗} |
-| Precision | {precision} | {target_precision} | {✓/✗} |
-| Recall | {recall} | {target_recall} | {✓/✗} |
-
-## Baseline Comparisons
-
-| Baseline | Metric | Their Score | Our Score | Improvement |
-|----------|--------|-------------|-----------|-------------|
-| {name} | {metric} | {baseline_score} | {our_score} | {delta}% |
-
-## Reproducibility
-
-All artifacts saved to: `.rc/experiments/artifacts/`
-
-- **Config**: `config.yaml` (seed={seed})
-- **Checkpoints**: `checkpoints/run_{id}/`
-- **Logs**: `logs/train.log`
-- **Results**: `artifacts/results/results_{timestamp}.json`
-
-## Next Steps
-
-- [ ] Verify results meet all PRD requirements
-- [ ] Generate figures for paper
-- [ ] Update paper draft with results
-- [ ] Archive experiment for future reference
-```
-
-## Example Usage
-
-```
-User: "Design and run experiments for the sentiment analysis model"
-
-Skill: [Reads PRD, creates task, dispatches @rc-experiment]
-Skill: [Agent launches training via Monitor, continues other work]
-Skill: [Receives notification when training completes]
-Skill: [Verifies metrics, generates report]
-Skill: "Experiment completed. Achieved accuracy=0.94 (target: 0.90). Results saved to .rc/experiments/."
-```

package/dist/skills/full-research-workflow/SKILL.md

@@ -1,363 +0,0 @@
----
-name: full-research-workflow
-description: |
-  Orchestrate a complete research workflow from literature review to camera-ready paper.
-  Calls `rc` CLI commands to create tasks, dispatches agents for execution, verifies quality gates.
-  Follows Trellis philosophy: Task-first, Action-before-asking, Single source of truth.
-triggers:
-  - "run the full research workflow"
-  - "orchestrate research from start to finish"
-  - "execute the complete research pipeline"
-  - "run all research stages"
----
-
-# Full Research Workflow
-
-Orchestrates a complete research workflow: literature → ideation → experiment → writing → polish → review → rebuttal.
-
-## When to Use
-
-Use this skill when:
-- User wants to run the entire research pipeline end-to-end
-- User says "run the full workflow", "execute all stages", "orchestrate research"
-- A `prd.md` exists with research goals and you need to execute all stages
-
-Do NOT use when:
-- User wants to run a single stage (use stage-specific skills instead)
-- No `prd.md` exists (guide user to create one first)
-
-## Task-First Principle
-
-If `prd.md` exists and has clear goals, proceed directly to Stage 1. Only ask questions if critical information is missing.
-
-## Auto-Context (Action-First)
-
-**Step 0**: Before asking clarifying questions, read the context files that might contain answers:
-
-```powershell
-# Check for PRD and existing state
-if (Test-Path prd.md) { Get-Content prd.md } else { "No PRD found" }
-rc task list --json 2>$null || "No tasks yet"
-if (Test-Path execute.jsonl) { "Execution log exists" } else { "No execution log" }
-if ((Test-Path baselines/) -and (Test-Path venue/)) { "Support dirs ready" } else { "No support dirs" }
-```
-
-**What to look for**:
-- `prd.md`: Research goals, target venue, constraints
-- `execute.jsonl`: Previous execution history, failed stages
-- Task list: Which stages are already complete/in-progress
-- Support dirs (`baselines/`, `venue/`): Whether infrastructure is ready
-
-**Decision tree**:
-1. If `prd.md` missing → Ask user to create one or provide research goals
-2. If `prd.md` exists but tasks already in progress → Resume from last incomplete stage
-3. If `prd.md` exists and no tasks → Start fresh from Stage 1
-
-Only ask clarifying questions if critical information is genuinely missing from these files.
-
-## Workflow Stages
-
-### Stage 1: Literature Review
-
-```powershell
-# Create literature review task and capture ID
-$output = rc task create `
-  --kind literature `
-  --title "Literature Review" `
-  --goal "Survey state-of-the-art in [research area from prd.md]"
-$LIT_TASK_ID = ($output | Select-String 'Task (\d+)').Matches.Groups[1].Value
-
-# Dispatch agent to execute
-rc agent dispatch --task-id $LIT_TASK_ID --agent literature-agent
-
-# Verify deliverables exist
-if (-not (Test-Path lit_review.md)) { "ERROR: lit_review.md missing" }
-if (-not (Test-Path references/)) { "ERROR: references/ missing" }
-
-# Mark complete
-rc task complete --task-id $LIT_TASK_ID
-```
-
-Quality gate: `lit_review.md` must exist with 15+ references, clear gaps identified.
-
-### Stage 2: Ideation
-
-```powershell
-# Verify literature task is complete
-$status = rc task status $LIT_TASK_ID
-if ($status -notmatch "completed") {
-  "ERROR: Literature task not completed yet"
-  exit 1
-}
-
-# Create ideation task (depends on literature)
-$output = rc task create `
-  --kind ideation `
-  --title "Ideation" `
-  --goal "Generate research ideas addressing gaps from literature" `
-  --depends-on $LIT_TASK_ID
-$IDEA_TASK_ID = ($output | Select-String 'Task (\d+)').Matches.Groups[1].Value
-
-# Dispatch agent
-rc agent dispatch --task-id $IDEA_TASK_ID --agent ideation-agent
-
-# Verify deliverables
-if (-not (Test-Path ideas.md)) { "ERROR: ideas.md missing" }
-if (-not (Select-String -Path ideas.md -Pattern "idea_" -Quiet)) { "ERROR: No structured ideas found" }
-
-# Mark complete
-rc task complete --task-id $IDEA_TASK_ID
-```
-
-Quality gate: `ideas.md` must contain 3+ structured ideas with novelty scores.
-
-### Stage 3: Experiment
-
-```powershell
-# Verify ideation task is complete
-$status = rc task status $IDEA_TASK_ID
-if ($status -notmatch "completed") {
-  "ERROR: Ideation task not completed yet"
-  exit 1
-}
-
-# Create experiment task (depends on ideation)
-$output = rc task create `
-  --kind experiment `
-  --title "Experiment Execution" `
-  --goal "Implement and run experiments for selected idea" `
-  --depends-on $IDEA_TASK_ID
-$EXP_TASK_ID = ($output | Select-String 'Task (\d+)').Matches.Groups[1].Value
-
-# Dispatch agent
-rc agent dispatch --task-id $EXP_TASK_ID --agent experiment-agent
-
-# Verify deliverables
-if (-not (Test-Path results.json)) { "ERROR: results.json missing" }
-if (-not (Test-Path figures/)) { "ERROR: figures/ missing" }
-if (-not (Test-Path experiment_log.md)) { "ERROR: experiment_log.md missing" }
-
-# Mark complete
-rc task complete --task-id $EXP_TASK_ID
-```
-
-Quality gate: `results.json` must have baseline comparisons, `figures/` must contain plots.
-
-### Stage 4: Writing
-
-```powershell
-# Verify experiment task is complete
-$status = rc task status $EXP_TASK_ID
-if ($status -notmatch "completed") {
-  "ERROR: Experiment task not completed yet"
-  exit 1
-}
-
-# Create writing task (depends on experiment)
-$output = rc task create `
-  --kind writing `
-  --title "Draft Paper" `
-  --goal "Write paper draft following venue template" `
-  --depends-on $EXP_TASK_ID
-$WRITE_TASK_ID = ($output | Select-String 'Task (\d+)').Matches.Groups[1].Value
-
-# Dispatch agent
-rc agent dispatch --task-id $WRITE_TASK_ID --agent writing-agent
-
-# Verify deliverables
-if (-not (Test-Path paper_draft.tex)) { "ERROR: paper_draft.tex missing" }
-pdflatex paper_draft.tex >$null 2>&1
-if ($LASTEXITCODE -ne 0) { "WARN: LaTeX compilation failed" }
-
-# Mark complete
-rc task complete --task-id $WRITE_TASK_ID
-```
-
-Quality gate: `paper_draft.tex` must compile, all sections present, references formatted.
-
-### Stage 5: Polish
-
-```powershell
-# Verify writing task is complete
-$status = rc task status $WRITE_TASK_ID
-if ($status -notmatch "completed") {
-  "ERROR: Writing task not completed yet"
-  exit 1
-}
-
-# Create polish task (depends on writing)
-$output = rc task create `
-  --kind polish `
-  --title "Polish Paper" `
-  --goal "Refine writing, check formatting, validate claims" `
-  --depends-on $WRITE_TASK_ID
-$POLISH_TASK_ID = ($output | Select-String 'Task (\d+)').Matches.Groups[1].Value
-
-# Dispatch agent
-rc agent dispatch --task-id $POLISH_TASK_ID --agent polish-agent
-
-# Verify deliverables
-if (-not (Test-Path paper_polished.tex)) { "ERROR: paper_polished.tex missing" }
-if (-not (Select-String -Path paper_polished.tex -Pattern "\\cite{" -Quiet)) { "WARN: No citations found" }
-
-# Mark complete
-rc task complete --task-id $POLISH_TASK_ID
-```
-
-Quality gate: Paper length within venue limits, all claims cited, figures captioned.
-
-### Stage 6: Review
-
-```powershell
-# Verify polish task is complete
-$status = rc task status $POLISH_TASK_ID
-if ($status -notmatch "completed") {
-  "ERROR: Polish task not completed yet"
-  exit 1
-}
-
-# Create review task (depends on polish)
-$output = rc task create `
-  --kind review `
-  --title "Internal Review" `
-  --goal "Simulate peer review, identify weaknesses" `
-  --depends-on $POLISH_TASK_ID
-$REVIEW_TASK_ID = ($output | Select-String 'Task (\d+)').Matches.Groups[1].Value
-
-# Dispatch agent
-rc agent dispatch --task-id $REVIEW_TASK_ID --agent review-agent
-
-# Verify deliverables
-if (-not (Test-Path review_report.md)) { "ERROR: review_report.md missing" }
-if (-not (Select-String -Path review_report.md -Pattern "weakness_" -Quiet)) { "ERROR: No structured weaknesses" }
-
-# Mark complete
-rc task complete --task-id $REVIEW_TASK_ID
-```
-
-Quality gate: `review_report.md` must identify 3+ weaknesses with severity scores.
-
-### Stage 7: Rebuttal (Optional)
-
-```powershell
-# Only run if review found critical issues
-if (Select-String -Path review_report.md -Pattern "severity: critical" -Quiet) {
-  # Verify review task is complete
-  $status = rc task status $REVIEW_TASK_ID
-  if ($status -notmatch "completed") {
-    "ERROR: Review task not completed yet"
-    exit 1
-  }
-  
-  $output = rc task create `
-    --kind rebuttal `
-    --title "Address Review Comments" `
-    --goal "Revise paper based on review feedback" `
-    --depends-on $REVIEW_TASK_ID
-  $REBUTTAL_TASK_ID = ($output | Select-String 'Task (\d+)').Matches.Groups[1].Value
-  
-  rc agent dispatch --task-id $REBUTTAL_TASK_ID --agent rebuttal-agent
-  
-  if (-not (Test-Path paper_revised.tex)) { "ERROR: paper_revised.tex missing" }
-  if (-not (Test-Path rebuttal_response.md)) { "ERROR: rebuttal_response.md missing" }
-  
-  rc task complete --task-id $REBUTTAL_TASK_ID
-}
-```
-
-Quality gate: All critical weaknesses addressed, rebuttal response provided.
-
-## Error Recovery
-
-### Executor Fails
-
-If `rc agent dispatch` fails or agent reports errors:
-
-1. Check `execute.jsonl` for the failure record
-2. Read the error message and context
-3. Record a gap in the task notes:
-   ```powershell
-   rc task update --task-id $TASK_ID `
-     --add-note "Agent failed: [error]. Manual intervention needed."
-   ```
-4. Ask user: "The [stage] agent failed with error: [error]. Would you like me to retry with different parameters, or handle this manually?"
-
-### Quality Gate Fails
-
-If deliverables are missing or malformed:
-
-1. Set task status back to `in_progress`:
-   ```powershell
-   rc task update --task-id $TASK_ID --status in_progress
-   ```
-2. Re-dispatch agent with explicit instructions:
-   ```powershell
-   rc agent dispatch --task-id $TASK_ID `
-     --agent <agent-name> `
-     --instruction "Previous run failed quality gate: [details]. Focus on [missing deliverable]."
-   ```
-3. If fails twice, ask user for guidance
-
-### MCP Unavailable
-
-If `rc` CLI commands fail because MCP server is not running:
-
-1. Record the gap: "MCP server unavailable, cannot create tasks programmatically"
-2. Provide manual fallback:
-   ```powershell
-   # Manual task creation
-   New-Item -ItemType Directory -Force -Path .research/tasks/
-   @"
-   {
-     "kind": "literature",
-     "title": "Literature Review",
-     "status": "pending"
-   }
-   "@ | Out-File -FilePath .research/tasks/literature.json -Encoding utf8
-   ```
-3. Ask user: "The `rc` CLI is unavailable. Should I proceed with manual task tracking?"
-
-## Report Format
-
-After all stages complete (or when blocked), provide a summary:
-
-```
-Research Workflow Summary
-========================
-
-Completed Stages:
-✓ Stage 1: Literature Review (lit_review.md, 18 references)
-✓ Stage 2: Ideation (ideas.md, 4 ideas generated)
-✓ Stage 3: Experiment (results.json, 3 baselines compared)
-✓ Stage 4: Writing (paper_draft.tex, 8 pages)
-✓ Stage 5: Polish (paper_polished.tex, formatting validated)
-✓ Stage 6: Review (review_report.md, 2 critical weaknesses found)
-✗ Stage 7: Rebuttal (skipped - no critical issues OR in progress)
-
-Deliverables:
-- Paper: paper_polished.tex (or paper_revised.tex if rebuttal ran)
-- Figures: figures/ (N plots)
-- References: references/ (M papers)
-- Execution log: execute.jsonl (full trace)
-
-Quality Gates Passed:
-- Literature: 15+ references ✓
-- Ideation: 3+ ideas ✓
-- Experiment: Baseline comparisons ✓
-- Writing: LaTeX compiles ✓
-- Polish: Within page limits ✓
-- Review: Weaknesses identified ✓
-
-Next Steps:
-- Review paper_polished.tex
-- Address any remaining review comments manually
-- Prepare submission package
-```
-
-If any stage failed, include:
-
-```
-Blockers:
-- Stage X failed: [error message]
-- Manual intervention needed: [specific action]
-```