get-research-done 1.1.0

package/get-research-done/templates/graduated-script.md

@@ -0,0 +1,180 @@
+# Graduated Script Template
+
+This template is used when graduating an exploration notebook to a validated Python script via `/grd:graduate`.
+
+## Template Variables
+
+| Variable | Source | Description |
+|----------|--------|-------------|
+| `{{experiment_name}}` | OBJECTIVE.md or notebook filename | Human-readable experiment name |
+| `{{source_notebook}}` | User input | Path to source notebook (e.g., `notebooks/exploration/001_initial.ipynb`) |
+| `{{source_run}}` | Run directory | Run that achieved PROCEED verdict (e.g., `experiments/run_003_baseline`) |
+| `{{critic_verdict}}` | CRITIC.md | Critic verdict (always PROCEED for graduation) |
+| `{{verdict_date}}` | CRITIC.md | ISO 8601 date of verdict |
+| `{{graduation_timestamp}}` | System | ISO 8601 timestamp when graduated |
+
+## Python Script Template
+
+```python
+"""
+Validated experiment: {{experiment_name}}
+
+Source notebook: {{source_notebook}}
+Source run: {{source_run}}
+Critic verdict: {{critic_verdict}} ({{verdict_date}})
+Graduated: {{graduation_timestamp}}
+
+MANUAL REFACTORING REQUIRED:
+- [ ] Remove/convert magic commands (grep "^%")
+- [ ] Extract code into functions
+- [ ] Replace parameter cell with argparse
+- [ ] Add docstrings and type hints
+- [ ] Set all random seeds explicitly
+- [ ] Write tests for core functions
+
+This script was auto-generated from a validated notebook.
+Review and complete the refactoring checklist above before production use.
+"""
+
+import argparse
+import random
+from typing import Any
+
+import numpy as np
+
+# Uncomment if using PyTorch:
+# import torch
+
+# Uncomment if using TensorFlow:
+# import tensorflow as tf
+
+
+def set_random_seeds(seed: int = 42) -> None:
+    """Set all random seeds for reproducibility.
+
+    Args:
+        seed: Random seed value (default: 42)
+    """
+    random.seed(seed)
+    np.random.seed(seed)
+
+    # Uncomment for PyTorch:
+    # torch.manual_seed(seed)
+    # torch.cuda.manual_seed_all(seed)
+    # torch.backends.cudnn.deterministic = True
+    # torch.backends.cudnn.benchmark = False
+
+    # Uncomment for TensorFlow:
+    # tf.random.set_seed(seed)
+
+
+def main(args: argparse.Namespace) -> dict[str, Any]:
+    """Main experiment entry point.
+
+    Args:
+        args: Parsed command line arguments
+
+    Returns:
+        Dictionary containing experiment results/metrics
+    """
+    set_random_seeds(args.random_seed)
+
+    # ---------------------------------------------------------------------
+    # TODO: Implement experiment logic here
+    # Extracted from: {{source_notebook}}
+    #
+    # Refactoring guidance:
+    # 1. Convert notebook cells to functions
+    # 2. Add type hints to all function signatures
+    # 3. Replace hardcoded values with args.* parameters
+    # 4. Return metrics as a dictionary for logging
+    # ---------------------------------------------------------------------
+
+    results = {
+        "status": "not_implemented",
+        "message": "Replace this with actual experiment implementation"
+    }
+
+    return results
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(
+        description="{{experiment_name}}",
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter
+    )
+
+    # Standard arguments
+    parser.add_argument(
+        "--random-seed",
+        type=int,
+        default=42,
+        help="Random seed for reproducibility"
+    )
+
+    # TODO: Add experiment-specific arguments
+    # Example:
+    # parser.add_argument("--learning-rate", type=float, default=0.001)
+    # parser.add_argument("--epochs", type=int, default=100)
+    # parser.add_argument("--batch-size", type=int, default=32)
+    # parser.add_argument("--data-path", type=str, required=True)
+
+    args = parser.parse_args()
+
+    results = main(args)
+    print(f"Experiment complete: {results}")
+```
+
+## Usage
+
+The graduation workflow:
+
+1. **Notebook achieves PROCEED verdict** via `/grd:research`
+2. **User initiates graduation** via `/grd:graduate`
+3. **System converts notebook to script** using nbconvert
+4. **System prepends this header template** with filled variables
+5. **Script lands in** `src/experiments/{{sanitized_experiment_name}}.py`
+6. **User completes refactoring checklist** before production use
+
+## Refactoring Checklist Details
+
+### Remove/convert magic commands
+```bash
+# Find magic commands in the generated script
+grep "^%" src/experiments/your_script.py
+```
+Common conversions:
+- `%matplotlib inline` - Remove (not needed in scripts)
+- `%load_ext autoreload` - Remove (not applicable)
+- `%%time` - Replace with `time.time()` or profiling
+- `!pip install` - Move to requirements.txt
+
+### Extract code into functions
+- Each logical block should be a function
+- Functions should have single responsibility
+- Return values instead of relying on globals
+
+### Replace parameter cell with argparse
+- Identify cells tagged 'parameters' or variable definitions
+- Convert each parameter to `parser.add_argument()`
+- Use type hints and defaults from original values
+
+### Add docstrings and type hints
+- Every function needs a docstring
+- Use Google or NumPy docstring style consistently
+- Add type hints to all arguments and return values
+
+### Set all random seeds explicitly
+- Call `set_random_seeds()` at start of `main()`
+- Pass seed through all library calls that accept it
+- Document any randomness that cannot be seeded
+
+### Write tests for core functions
+- Create `tests/test_{{sanitized_experiment_name}}.py`
+- Test pure functions with known inputs/outputs
+- Test edge cases and error conditions
+
+---
+
+*Template version: 1.0*
+*Phase: 06-notebook-support*

package/get-research-done/templates/iteration-summary.md

@@ -0,0 +1,234 @@
+# Iteration Summary Template
+
+Template for `experiments/archive/YYYY-MM-DD_hypothesis_name/ITERATION_SUMMARY.md`.
+
+Collapses all iteration attempts into a single summary document when archiving.
+
+---
+
+## File Template
+
+```markdown
+# Iteration Summary: {{hypothesis_name}}
+
+**Total Iterations:** {{N}}
+**Date Range:** {{first_run_date}} to {{last_run_date}}
+**Outcome:** Archived (hypothesis abandoned)
+
+## Iteration History
+
+| # | Run | Date | Verdict | Confidence | Key Metric | Notes |
+|---|-----|------|---------|------------|------------|-------|
+| 1 | run_001_baseline | YYYY-MM-DD | REVISE_METHOD | MEDIUM | F1=0.72 | Initial attempt |
+| 2 | run_002_tuned | YYYY-MM-DD | REVISE_METHOD | MEDIUM | F1=0.76 | Hyperparameter tuning |
+| 3 | run_003_final | YYYY-MM-DD | ESCALATE | LOW | F1=0.78 | Limit reached |
+
+## Metric Trend
+
+**Best achieved:** {{metric}}={{best_value}}
+**Target:** {{threshold}}
+**Gap:** {{best_value - threshold}}
+**Trend:** {{improving|stagnant|degrading}}
+
+## Verdict Distribution
+
+- PROCEED: {{count}}
+- REVISE_METHOD: {{count}}
+- REVISE_DATA: {{count}}
+- ESCALATE: {{count}}
+
+## Key Observations
+
+{{Summary of what was tried and why it didn't work}}
+
+## Preserved Artifacts
+
+- Final run: {{run_NNN_description}}/
+- All CRITIC_LOG.md files (merged or individual)
+- Final SCORECARD.json
+
+---
+
+*Summary generated on archive. See ARCHIVE_REASON.md for human rationale.*
+```
+
+---
+
+## Usage Notes
+
+**Field descriptions:**
+
+- **hypothesis_name:** Human-readable name from OBJECTIVE.md "what" section
+- **N:** Total iteration count (number of runs attempted)
+- **first_run_date:** Timestamp from earliest run directory
+- **last_run_date:** Timestamp from final run directory
+- **Outcome:** Always "Archived (hypothesis abandoned)" for this template
+
+**Iteration History table:**
+
+Populate from all run directories in experiments/:
+- **#:** Sequential iteration number (1, 2, 3...)
+- **Run:** Run directory name (run_001_baseline, run_002_tuned, etc.)
+- **Date:** Extraction date from CRITIC_LOG.md or directory timestamp
+- **Verdict:** Critic verdict (PROCEED, REVISE_METHOD, REVISE_DATA, ESCALATE)
+- **Confidence:** Critic confidence level (HIGH, MEDIUM, LOW)
+- **Key Metric:** Primary metric value from SCORECARD.json (e.g., "F1=0.72")
+- **Notes:** Brief description of what was tried (from run README.md or CRITIC_LOG.md)
+
+**Metric Trend section:**
+
+- **Best achieved:** Highest value attained across all iterations for primary metric
+- **Target:** Threshold from OBJECTIVE.md for that metric
+- **Gap:** Difference (best_value - threshold), show with sign
+- **Trend:** Classify based on metric progression:
+  - "improving" if values consistently increase across iterations
+  - "stagnant" if values plateau or fluctuate without progress
+  - "degrading" if values decrease (rare but possible with overfitting)
+
+**Verdict Distribution:**
+
+Count occurrences of each verdict type across all runs:
+```bash
+# Example calculation
+PROCEED=$(grep -h "^\*\*Verdict:\*\* PROCEED" experiments/run_*/CRITIC_LOG.md | wc -l)
+REVISE_METHOD=$(grep -h "^\*\*Verdict:\*\* REVISE_METHOD" experiments/run_*/CRITIC_LOG.md | wc -l)
+REVISE_DATA=$(grep -h "^\*\*Verdict:\*\* REVISE_DATA" experiments/run_*/CRITIC_LOG.md | wc -l)
+ESCALATE=$(grep -h "^\*\*Verdict:\*\* ESCALATE" experiments/run_*/CRITIC_LOG.md | wc -l)
+```
+
+**Key Observations section:**
+
+Synthesize learnings from all iterations. Extract from:
+- CRITIC_LOG.md strengths/weaknesses across runs
+- Patterns in metric progression
+- What approaches were attempted (architectures, hyperparameters, data transformations)
+- Why none succeeded (common failure mode)
+
+Examples:
+- "All iterations struggled with class imbalance (99.2% negative). Resampling techniques (SMOTE, undersampling) did not improve recall."
+- "Metric progression stagnated after run 2. Runs 3-5 showed no improvement despite hyperparameter changes, suggesting architectural limitation."
+- "Data leakage warnings in DATA_REPORT.md were confirmed. Removing leaked features in run 4-5 caused significant performance drop, revealing hypothesis depended on invalid signal."
+
+**Preserved Artifacts section:**
+
+List what's kept in the archive:
+- Final run directory (moved from experiments/)
+- CRITIC_LOG.md files (individual or merged)
+- Final SCORECARD.json
+- ARCHIVE_REASON.md (user rationale)
+- This ITERATION_SUMMARY.md
+
+**Example populated template:**
+
+```markdown
+# Iteration Summary: Ensemble Methods for Fraud Detection
+
+**Total Iterations:** 5
+**Date Range:** 2026-01-15 to 2026-01-30
+**Outcome:** Archived (hypothesis abandoned)
+
+## Iteration History
+
+| # | Run | Date | Verdict | Confidence | Key Metric | Notes |
+|---|-----|------|---------|------------|------------|-------|
+| 1 | run_001_baseline | 2026-01-15 | REVISE_METHOD | MEDIUM | F1=0.52 | Initial ensemble (RF+GBM) |
+| 2 | run_002_smote | 2026-01-18 | REVISE_METHOD | MEDIUM | F1=0.56 | Added SMOTE resampling |
+| 3 | run_003_weighted | 2026-01-22 | REVISE_METHOD | MEDIUM | F1=0.58 | Class weight optimization |
+| 4 | run_004_reduced | 2026-01-25 | REVISE_METHOD | LOW | F1=0.57 | Feature reduction (237→15) |
+| 5 | run_005_final | 2026-01-30 | ESCALATE | LOW | F1=0.58 | Limit reached |
+
+## Metric Trend
+
+**Best achieved:** F1=0.58
+**Target:** 0.85
+**Gap:** -0.27
+**Trend:** stagnant (plateaued at 0.56-0.58 after run 2)
+
+## Verdict Distribution
+
+- PROCEED: 0
+- REVISE_METHOD: 4
+- REVISE_DATA: 0
+- ESCALATE: 1
+
+## Key Observations
+
+All five iterations struggled with severe class imbalance (99.2% negative class, N=120 positive examples). Despite trying multiple approaches (SMOTE resampling, class weighting, feature reduction), F1 score plateaued at 0.56-0.58, far below the target of 0.85.
+
+**What was tried:**
+- Run 1: Baseline ensemble (Random Forest + Gradient Boosting)
+- Run 2: SMOTE oversampling to balance classes
+- Run 3: Class weight optimization (weighted loss functions)
+- Run 4: Feature reduction (237→15 features) to reduce overfitting
+- Run 5: Combined approach with focal loss
+
+**Common failure mode:**
+All iterations achieved high precision (>0.90) but poor recall (<0.42), indicating the model learned to be conservative due to extreme class imbalance. Resampling techniques introduced artificial patterns that didn't generalize. The fundamental issue is insufficient positive examples (N=120) for the hypothesis to be testable with ensemble methods.
+
+**Critic's repeated concerns:**
+- "Limited positive examples prevent ensemble diversity"
+- "High precision but poor recall suggests severe class imbalance"
+- "Feature space too large relative to positive sample size"
+
+## Preserved Artifacts
+
+- Final run: run_005_final/ (complete snapshot with DECISION.md)
+- All CRITIC_LOG.md files (archived individually in final_run/)
+- Final SCORECARD.json (F1=0.58, composite=0.64)
+- ARCHIVE_REASON.md (user rationale for abandonment)
+- This ITERATION_SUMMARY.md
+
+---
+
+*Summary generated on archive. See ARCHIVE_REASON.md for human rationale.*
+```
+
+---
+
+## Integration
+
+This template is used by `/grd:evaluate` command in Phase 5 (Archive Handling) when user selects "Archive" decision.
+
+**Inputs:**
+- All run directories in experiments/ (run_001, run_002, ...)
+- CRITIC_LOG.md from each run (verdict, confidence, recommendations)
+- SCORECARD.json from each run (metrics, composite score)
+- OBJECTIVE.md (hypothesis, target thresholds)
+
+**Generation logic:**
+
+```bash
+# Collect all runs
+RUNS=$(ls -1d experiments/run_* | sort)
+TOTAL=$(echo "$RUNS" | wc -l | tr -d ' ')
+
+# Extract date range
+FIRST_DATE=$(stat -f "%Sm" -t "%Y-%m-%d" $(echo "$RUNS" | head -1))
+LAST_DATE=$(stat -f "%Sm" -t "%Y-%m-%d" $(echo "$RUNS" | tail -1))
+
+# Build iteration history table
+for run in $RUNS; do
+  VERDICT=$(grep "^\*\*Verdict:\*\*" "$run/CRITIC_LOG.md" | head -1 | awk '{print $2}')
+  CONFIDENCE=$(grep "^\*\*Confidence:\*\*" "$run/CRITIC_LOG.md" | head -1 | awk '{print $2}')
+  METRIC=$(jq -r '.metrics[0] | "\(.name)=\(.value)"' "$run/metrics/SCORECARD.json")
+  NOTES=$(head -1 "$run/README.md" | sed 's/^# //')
+  echo "| $i | $(basename $run) | $DATE | $VERDICT | $CONFIDENCE | $METRIC | $NOTES |"
+done
+
+# Calculate metric trend
+BEST_METRIC=$(jq -s 'map(.metrics[0].value) | max' experiments/run_*/metrics/SCORECARD.json)
+TARGET=$(jq -r '.metrics[0].threshold' .planning/OBJECTIVE.md)
+GAP=$(echo "$BEST_METRIC - $TARGET" | bc)
+
+# Determine trend (simplified)
+if [ "$GAP" -lt -0.10 ]; then
+  TREND="stagnant (far from target)"
+else
+  TREND="improving (but insufficient)"
+fi
+```
+
+**Output:**
+- experiments/archive/YYYY-MM-DD_hypothesis_slug/ITERATION_SUMMARY.md
+- Referenced by ARCHIVE_REASON.md in same directory
+- Provides historical context for negative result

package/get-research-done/templates/milestone-archive.md

@@ -0,0 +1,123 @@
+# Milestone Archive Template
+
+This template is used by the complete-milestone workflow to create archive files in `.planning/milestones/`.
+
+---
+
+## File Template
+
+# Milestone v{{VERSION}}: {{MILESTONE_NAME}}
+
+**Status:** ✅ SHIPPED {{DATE}}
+**Phases:** {{PHASE_START}}-{{PHASE_END}}
+**Total Plans:** {{TOTAL_PLANS}}
+
+## Overview
+
+{{MILESTONE_DESCRIPTION}}
+
+## Phases
+
+{{PHASES_SECTION}}
+
+[For each phase in this milestone, include:]
+
+### Phase {{PHASE_NUM}}: {{PHASE_NAME}}
+
+**Goal**: {{PHASE_GOAL}}
+**Depends on**: {{DEPENDS_ON}}
+**Plans**: {{PLAN_COUNT}} plans
+
+Plans:
+
+- [x] {{PHASE}}-01: {{PLAN_DESCRIPTION}}
+- [x] {{PHASE}}-02: {{PLAN_DESCRIPTION}}
+      [... all plans ...]
+
+**Details:**
+{{PHASE_DETAILS_FROM_ROADMAP}}
+
+**For decimal phases, include (INSERTED) marker:**
+
+### Phase 2.1: Critical Security Patch (INSERTED)
+
+**Goal**: Fix authentication bypass vulnerability
+**Depends on**: Phase 2
+**Plans**: 1 plan
+
+Plans:
+
+- [x] 02.1-01: Patch auth vulnerability
+
+**Details:**
+{{PHASE_DETAILS_FROM_ROADMAP}}
+
+---
+
+## Milestone Summary
+
+**Decimal Phases:**
+
+- Phase 2.1: Critical Security Patch (inserted after Phase 2 for urgent fix)
+- Phase 5.1: Performance Hotfix (inserted after Phase 5 for production issue)
+
+**Key Decisions:**
+{{DECISIONS_FROM_PROJECT_STATE}}
+[Example:]
+
+- Decision: Use ROADMAP.md split (Rationale: Constant context cost)
+- Decision: Decimal phase numbering (Rationale: Clear insertion semantics)
+
+**Issues Resolved:**
+{{ISSUES_RESOLVED_DURING_MILESTONE}}
+[Example:]
+
+- Fixed context overflow at 100+ phases
+- Resolved phase insertion confusion
+
+**Issues Deferred:**
+{{ISSUES_DEFERRED_TO_LATER}}
+[Example:]
+
+- PROJECT-STATE.md tiering (deferred until decisions > 300)
+
+**Technical Debt Incurred:**
+{{SHORTCUTS_NEEDING_FUTURE_WORK}}
+[Example:]
+
+- Some workflows still have hardcoded paths (fix in Phase 5)
+
+---
+
+_For current project status, see .planning/ROADMAP.md_
+
+---
+
+## Usage Guidelines
+
+<guidelines>
+**When to create milestone archives:**
+- After completing all phases in a milestone (v1.0, v1.1, v2.0, etc.)
+- Triggered by complete-milestone workflow
+- Before planning next milestone work
+
+**How to fill template:**
+
+- Replace {{PLACEHOLDERS}} with actual values
+- Extract phase details from ROADMAP.md
+- Document decimal phases with (INSERTED) marker
+- Include key decisions from PROJECT-STATE.md or SUMMARY files
+- List issues resolved vs deferred
+- Capture technical debt for future reference
+
+**Archive location:**
+
+- Save to `.planning/milestones/v{VERSION}-{NAME}.md`
+- Example: `.planning/milestones/v1.0-mvp.md`
+
+**After archiving:**
+
+- Update ROADMAP.md to collapse completed milestone in `<details>` tag
+- Update PROJECT.md to brownfield format with Current State section
+- Continue phase numbering in next milestone (never restart at 01)
+  </guidelines>

package/get-research-done/templates/milestone.md

@@ -0,0 +1,115 @@
+# Milestone Entry Template
+
+Add this entry to `.planning/MILESTONES.md` when completing a milestone:
+
+```markdown
+## v[X.Y] [Name] (Shipped: YYYY-MM-DD)
+
+**Delivered:** [One sentence describing what shipped]
+
+**Phases completed:** [X-Y] ([Z] plans total)
+
+**Key accomplishments:**
+- [Major achievement 1]
+- [Major achievement 2]
+- [Major achievement 3]
+- [Major achievement 4]
+
+**Stats:**
+- [X] files created/modified
+- [Y] lines of code (primary language)
+- [Z] phases, [N] plans, [M] tasks
+- [D] days from start to ship (or milestone to milestone)
+
+**Git range:** `feat(XX-XX)` → `feat(YY-YY)`
+
+**What's next:** [Brief description of next milestone goals, or "Project complete"]
+
+---
+```
+
+<structure>
+If MILESTONES.md doesn't exist, create it with header:
+
+```markdown
+# Project Milestones: [Project Name]
+
+[Entries in reverse chronological order - newest first]
+```
+</structure>
+
+<guidelines>
+**When to create milestones:**
+- Initial v1.0 MVP shipped
+- Major version releases (v2.0, v3.0)
+- Significant feature milestones (v1.1, v1.2)
+- Before archiving planning (capture what was shipped)
+
+**Don't create milestones for:**
+- Individual phase completions (normal workflow)
+- Work in progress (wait until shipped)
+- Minor bug fixes that don't constitute a release
+
+**Stats to include:**
+- Count modified files: `git diff --stat feat(XX-XX)..feat(YY-YY) | tail -1`
+- Count LOC: `find . -name "*.swift" -o -name "*.ts" | xargs wc -l` (or relevant extension)
+- Phase/plan/task counts from ROADMAP
+- Timeline from first phase commit to last phase commit
+
+**Git range format:**
+- First commit of milestone → last commit of milestone
+- Example: `feat(01-01)` → `feat(04-01)` for phases 1-4
+</guidelines>
+
+<example>
+```markdown
+# Project Milestones: WeatherBar
+
+## v1.1 Security & Polish (Shipped: 2025-12-10)
+
+**Delivered:** Security hardening with Keychain integration and comprehensive error handling
+
+**Phases completed:** 5-6 (3 plans total)
+
+**Key accomplishments:**
+- Migrated API key storage from plaintext to macOS Keychain
+- Implemented comprehensive error handling for network failures
+- Added Sentry crash reporting integration
+- Fixed memory leak in auto-refresh timer
+
+**Stats:**
+- 23 files modified
+- 650 lines of Swift added
+- 2 phases, 3 plans, 12 tasks
+- 8 days from v1.0 to v1.1
+
+**Git range:** `feat(05-01)` → `feat(06-02)`
+
+**What's next:** v2.0 SwiftUI redesign with widget support
+
+---
+
+## v1.0 MVP (Shipped: 2025-11-25)
+
+**Delivered:** Menu bar weather app with current conditions and 3-day forecast
+
+**Phases completed:** 1-4 (7 plans total)
+
+**Key accomplishments:**
+- Menu bar app with popover UI (AppKit)
+- OpenWeather API integration with auto-refresh
+- Current weather display with conditions icon
+- 3-day forecast list with high/low temperatures
+- Code signed and notarized for distribution
+
+**Stats:**
+- 47 files created
+- 2,450 lines of Swift
+- 4 phases, 7 plans, 28 tasks
+- 12 days from start to ship
+
+**Git range:** `feat(01-01)` → `feat(04-01)`
+
+**What's next:** Security audit and hardening for v1.1
+```
+</example>