get-research-done 1.1.0

package/get-research-done/templates/roadmap.md

@@ -0,0 +1,202 @@
+# Roadmap Template
+
+Template for `.planning/ROADMAP.md`.
+
+## Initial Roadmap (v1.0 Greenfield)
+
+```markdown
+# Roadmap: [Project Name]
+
+## Overview
+
+[One paragraph describing the journey from start to finish]
+
+## Phases
+
+**Phase Numbering:**
+- Integer phases (1, 2, 3): Planned milestone work
+- Decimal phases (2.1, 2.2): Urgent insertions (marked with INSERTED)
+
+Decimal phases appear between their surrounding integers in numeric order.
+
+- [ ] **Phase 1: [Name]** - [One-line description]
+- [ ] **Phase 2: [Name]** - [One-line description]
+- [ ] **Phase 3: [Name]** - [One-line description]
+- [ ] **Phase 4: [Name]** - [One-line description]
+
+## Phase Details
+
+### Phase 1: [Name]
+**Goal**: [What this phase delivers]
+**Depends on**: Nothing (first phase)
+**Requirements**: [REQ-01, REQ-02, REQ-03]
+**Success Criteria** (what must be TRUE):
+  1. [Observable behavior from user perspective]
+  2. [Observable behavior from user perspective]
+  3. [Observable behavior from user perspective]
+**Plans**: [Number of plans, e.g., "3 plans" or "TBD"]
+
+Plans:
+- [ ] 01-01: [Brief description of first plan]
+- [ ] 01-02: [Brief description of second plan]
+- [ ] 01-03: [Brief description of third plan]
+
+### Phase 2: [Name]
+**Goal**: [What this phase delivers]
+**Depends on**: Phase 1
+**Requirements**: [REQ-04, REQ-05]
+**Success Criteria** (what must be TRUE):
+  1. [Observable behavior from user perspective]
+  2. [Observable behavior from user perspective]
+**Plans**: [Number of plans]
+
+Plans:
+- [ ] 02-01: [Brief description]
+- [ ] 02-02: [Brief description]
+
+### Phase 2.1: Critical Fix (INSERTED)
+**Goal**: [Urgent work inserted between phases]
+**Depends on**: Phase 2
+**Success Criteria** (what must be TRUE):
+  1. [What the fix achieves]
+**Plans**: 1 plan
+
+Plans:
+- [ ] 02.1-01: [Description]
+
+### Phase 3: [Name]
+**Goal**: [What this phase delivers]
+**Depends on**: Phase 2
+**Requirements**: [REQ-06, REQ-07, REQ-08]
+**Success Criteria** (what must be TRUE):
+  1. [Observable behavior from user perspective]
+  2. [Observable behavior from user perspective]
+  3. [Observable behavior from user perspective]
+**Plans**: [Number of plans]
+
+Plans:
+- [ ] 03-01: [Brief description]
+- [ ] 03-02: [Brief description]
+
+### Phase 4: [Name]
+**Goal**: [What this phase delivers]
+**Depends on**: Phase 3
+**Requirements**: [REQ-09, REQ-10]
+**Success Criteria** (what must be TRUE):
+  1. [Observable behavior from user perspective]
+  2. [Observable behavior from user perspective]
+**Plans**: [Number of plans]
+
+Plans:
+- [ ] 04-01: [Brief description]
+
+## Progress
+
+**Execution Order:**
+Phases execute in numeric order: 2 → 2.1 → 2.2 → 3 → 3.1 → 4
+
+| Phase | Plans Complete | Status | Completed |
+|-------|----------------|--------|-----------|
+| 1. [Name] | 0/3 | Not started | - |
+| 2. [Name] | 0/2 | Not started | - |
+| 3. [Name] | 0/2 | Not started | - |
+| 4. [Name] | 0/1 | Not started | - |
+```
+
+<guidelines>
+**Initial planning (v1.0):**
+- Phase count depends on depth setting (quick: 3-5, standard: 5-8, comprehensive: 8-12)
+- Each phase delivers something coherent
+- Phases can have 1+ plans (split if >3 tasks or multiple subsystems)
+- Plans use naming: {phase}-{plan}-PLAN.md (e.g., 01-02-PLAN.md)
+- No time estimates (this isn't enterprise PM)
+- Progress table updated by execute workflow
+- Plan count can be "TBD" initially, refined during planning
+
+**Success criteria:**
+- 2-5 observable behaviors per phase (from user's perspective)
+- Cross-checked against requirements during roadmap creation
+- Flow downstream to `must_haves` in plan-phase
+- Verified by verify-phase after execution
+- Format: "User can [action]" or "[Thing] works/exists"
+
+**After milestones ship:**
+- Collapse completed milestones in `<details>` tags
+- Add new milestone sections for upcoming work
+- Keep continuous phase numbering (never restart at 01)
+</guidelines>
+
+<status_values>
+- `Not started` - Haven't begun
+- `In progress` - Currently working
+- `Complete` - Done (add completion date)
+- `Deferred` - Pushed to later (with reason)
+</status_values>
+
+## Milestone-Grouped Roadmap (After v1.0 Ships)
+
+After completing first milestone, reorganize with milestone groupings:
+
+```markdown
+# Roadmap: [Project Name]
+
+## Milestones
+
+- ✅ **v1.0 MVP** - Phases 1-4 (shipped YYYY-MM-DD)
+- 🚧 **v1.1 [Name]** - Phases 5-6 (in progress)
+- 📋 **v2.0 [Name]** - Phases 7-10 (planned)
+
+## Phases
+
+<details>
+<summary>✅ v1.0 MVP (Phases 1-4) - SHIPPED YYYY-MM-DD</summary>
+
+### Phase 1: [Name]
+**Goal**: [What this phase delivers]
+**Plans**: 3 plans
+
+Plans:
+- [x] 01-01: [Brief description]
+- [x] 01-02: [Brief description]
+- [x] 01-03: [Brief description]
+
+[... remaining v1.0 phases ...]
+
+</details>
+
+### 🚧 v1.1 [Name] (In Progress)
+
+**Milestone Goal:** [What v1.1 delivers]
+
+#### Phase 5: [Name]
+**Goal**: [What this phase delivers]
+**Depends on**: Phase 4
+**Plans**: 2 plans
+
+Plans:
+- [ ] 05-01: [Brief description]
+- [ ] 05-02: [Brief description]
+
+[... remaining v1.1 phases ...]
+
+### 📋 v2.0 [Name] (Planned)
+
+**Milestone Goal:** [What v2.0 delivers]
+
+[... v2.0 phases ...]
+
+## Progress
+
+| Phase | Milestone | Plans Complete | Status | Completed |
+|-------|-----------|----------------|--------|-----------|
+| 1. Foundation | v1.0 | 3/3 | Complete | YYYY-MM-DD |
+| 2. Features | v1.0 | 2/2 | Complete | YYYY-MM-DD |
+| 5. Security | v1.1 | 0/2 | Not started | - |
+```
+
+**Notes:**
+- Milestone emoji: ✅ shipped, 🚧 in progress, 📋 planned
+- Completed milestones collapsed in `<details>` for readability
+- Current/future milestones expanded
+- Continuous phase numbering (01-99)
+- Progress table includes milestone column

package/get-research-done/templates/scorecard.json

@@ -0,0 +1,113 @@
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "$id": "grd-scorecard-v1",
+  "title": "GRD Experiment Scorecard",
+  "description": "Quantitative evaluation results for a validated experiment run",
+
+  "run_id": "{{run_NNN_description}}",
+  "timestamp": "{{ISO8601_timestamp}}",
+  "objective_ref": ".planning/OBJECTIVE.md",
+  "hypothesis": "{{brief_hypothesis_statement}}",
+  "iteration": "{{iteration_number}}",
+  "data_version": "{{sha256_hash_of_data}}",
+
+  "evaluation": {
+    "strategy": "{{k-fold|stratified-k-fold|time-series-split|holdout}}",
+    "k": "{{number_of_folds_or_null}}",
+    "test_size": "{{proportion_or_null}}",
+    "random_state": 42,
+    "folds_completed": "{{number_of_folds_executed}}"
+  },
+
+  "metrics": {
+    "{{metric_name}}": {
+      "mean": "{{float_mean_across_folds}}",
+      "std": "{{float_std_across_folds}}",
+      "per_fold": ["{{fold_1_value}}", "{{fold_2_value}}", "..."],
+      "threshold": "{{threshold_from_objective}}",
+      "comparison": "{{>=|<=|==}}",
+      "weight": "{{0.0-1.0_from_objective}}",
+      "result": "{{PASS|FAIL}}"
+    },
+    "{{additional_metrics}}": {
+      "...": "..."
+    }
+  },
+
+  "composite_score": "{{weighted_average_of_all_metrics}}",
+  "composite_threshold": "{{from_objective_or_default_0.5}}",
+  "overall_result": "{{PASS|FAIL}}",
+
+  "baseline_comparison": {
+    "experiment_score": "{{weighted_composite_score}}",
+    "baselines": [
+      {
+        "name": "{{primary_baseline_name}}",
+        "type": "primary",
+        "source": "{{own_implementation|literature_citation}}",
+        "score": "{{baseline_composite_score}}",
+        "experiment_score": "{{experiment_composite_score}}",
+        "improvement": "{{float_experiment_minus_baseline}}",
+        "improvement_pct": "{{percentage_string}}",
+        "significant": "{{true|false|not_tested}}",
+        "run_path": "{{experiments/run_NNN_baseline/}}",
+        "note": "{{optional_note_for_literature_baselines}}"
+      },
+      {
+        "name": "{{secondary_baseline_name}}",
+        "type": "secondary",
+        "source": "{{own_implementation|literature_citation}}",
+        "score": "{{baseline_composite_score}}",
+        "improvement": "{{float_improvement}}",
+        "improvement_pct": "{{percentage_string}}",
+        "significant": "{{true|false|not_tested}}",
+        "run_path": "{{experiments/run_NNN_secondary/|null_if_literature}}"
+      }
+    ],
+    "primary_baseline": "{{primary_baseline_name_or_null}}",
+    "secondary_baselines": ["{{secondary_name_1}}", "{{secondary_name_2}}"],
+    "warnings": ["{{warning_if_any_baseline_unavailable}}"]
+  },
+
+  "baseline_validation": {
+    "researcher_validated": "{{true|false}}",
+    "evaluator_validated": "{{true|false}}",
+    "validation_skipped": "{{true_if_skip_baseline_used|false}}",
+    "data_hash_match": "{{true_if_same_data|false_with_warning}}",
+    "notes": ["{{validation_notes_if_any}}"]
+  },
+
+  "confidence_interval": {
+    "composite_lower": "{{float_lower_bound}}",
+    "composite_upper": "{{float_upper_bound}}",
+    "confidence_level": 0.95,
+    "method": "{{bootstrap|t_distribution}}"
+  },
+
+  "provenance": {
+    "code_snapshot": "experiments/{{run_id}}/code/",
+    "config_file": "experiments/{{run_id}}/config.yaml",
+    "logs": "experiments/{{run_id}}/logs/",
+    "outputs": "experiments/{{run_id}}/outputs/"
+  },
+
+  "critic_summary": {
+    "verdict": "PROCEED",
+    "confidence": "{{HIGH|MEDIUM|LOW}}",
+    "log_path": "experiments/{{run_id}}/CRITIC_LOG.md"
+  },
+
+  "ready_for_human_review": true,
+  "next_phase": "Phase 5: Human Evaluation Gate",
+
+  "_notes": {
+    "description": "This template shows the structure and placeholder format for SCORECARD.json",
+    "usage": "grd-evaluator agent populates this template with actual evaluation results",
+    "weights_constraint": "All metric weights must sum to 1.0",
+    "baseline_structure": "baselines array supports multiple comparisons: first is primary (required), rest are secondary (optional)",
+    "baseline_types": "primary = required for experiment to proceed; secondary = optional additional comparisons",
+    "baseline_validation": "Tracks both Researcher (start) and Evaluator (end) validation states",
+    "mlflow_integration": "These fields are also logged to MLflow if available",
+    "phase_5_requirement": "ready_for_human_review: true signals Phase 5 can proceed"
+  }
+}

package/get-research-done/templates/state.md

@@ -0,0 +1,287 @@
+# State Template
+
+<!-- STATE.md template v2.0 - GRD research loop tracking -->
+
+Template for `.planning/STATE.md` — the project's living memory.
+
+---
+
+## File Template
+
+```markdown
+# Project State
+
+## Project Reference
+
+See: .planning/PROJECT.md (updated [date])
+
+**Core value:** [One-liner from PROJECT.md Core Value section]
+**Current focus:** [Current phase name]
+
+## Current Position
+
+Phase: [X] of [Y] ([Phase name])
+Plan: [A] of [B] in current phase
+Status: [Ready to plan / Planning / Ready to execute / In progress / Phase complete]
+Last activity: [YYYY-MM-DD] — [What happened]
+
+Progress: [░░░░░░░░░░] 0%
+
+## Research Loop State
+
+**Active Hypothesis:** {{hypothesis_id_or_none}}
+**Objective:** {{brief_hypothesis_statement}}
+**Status:** {{not_started|in_progress|pending_review|archived}}
+
+### Current Iteration
+
+- **Iteration:** {{N}} of {{limit}} (default limit: 5)
+- **Current Run:** experiments/{{run_NNN_description}}
+- **Phase:** {{researcher|critic|evaluator|human_review}}
+- **Data Revisions:** {{data_revision_count}} of {{data_revision_limit}} (default limit: 2)
+
+### Loop History
+
+| Iteration | Run | Verdict | Confidence | Metrics Summary |
+|-----------|-----|---------|------------|-----------------|
+| 1 | run_001_baseline | REVISE_METHOD | MEDIUM | acc=0.72 |
+| 2 | run_002_tuned | PROCEED | HIGH | acc=0.85 |
+
+### Verdict Trend
+
+- **Pattern:** {{improving|stagnant|degrading|mixed}}
+- **Consecutive same verdicts:** {{N}}
+- **Last 3 verdicts:** {{verdict1, verdict2, verdict3}}
+
+### Human Decisions
+
+| Timestamp | Decision | Rationale |
+|-----------|----------|-----------|
+| {{timestamp}} | {{Continue|Archive|Reset|Escalate}} | {{user_rationale}} |
+
+### Data Revisions
+
+Track REVISE_DATA cycles within current hypothesis:
+
+| Iteration | Concerns | Explorer Result | Action Taken |
+|-----------|----------|-----------------|--------------|
+| {{N}} | {{concern_list}} | {{result_summary}} | {{action}} |
+
+**Data Revision Limits:**
+- Current count: {{data_revision_count}} of {{data_revision_limit}}
+- If limit reached: Escalate to human (data quality may be insufficient for hypothesis)
+
+## Performance Metrics
+
+**Velocity:**
+- Total plans completed: [N]
+- Average duration: [X] min
+- Total execution time: [X.X] hours
+
+**By Phase:**
+
+| Phase | Plans | Total | Avg/Plan |
+|-------|-------|-------|----------|
+| - | - | - | - |
+
+**Recent Trend:**
+- Last 5 plans: [durations]
+- Trend: [Improving / Stable / Degrading]
+
+*Updated after each plan completion*
+
+## Accumulated Context
+
+### Decisions
+
+Decisions are logged in PROJECT.md Key Decisions table.
+Recent decisions affecting current work:
+
+- [Phase X]: [Decision summary]
+- [Phase Y]: [Decision summary]
+
+### Research Decisions
+
+| Decision | Iteration | Impact |
+|----------|-----------|--------|
+| {{decision_description}} | {{N}} | {{what_changed}} |
+
+### Pending Todos
+
+[From .planning/todos/pending/ — ideas captured during sessions]
+
+None yet.
+
+### Blockers/Concerns
+
+[Issues that affect future work]
+
+None yet.
+
+### Research Blockers
+
+- **Current:** {{blocker_or_none}}
+- **Requires:** {{human_action|data_fix|method_change}}
+
+### Research Blockers
+
+- **Current:** {{blocker_or_none}}
+- **Requires:** {{human_action|data_fix|method_change}}
+
+## Session Continuity
+
+Last session: [YYYY-MM-DD HH:MM]
+Stopped at: [Description of last completed action]
+Resume file: [Path to .continue-here*.md if exists, otherwise "None"]
+
+## Research Loop History
+
+**Active Loop:** [N/A - no active research loop]
+**Loop Status:** [idle/exploring/synthesizing/validating/complete]
+
+| Loop | Started | Focus Area | Status | Outcome |
+|------|---------|------------|--------|---------|
+| - | - | - | - | - |
+
+**Current Loop Progress:**
+- [ ] Data reconnaissance (Explorer)
+- [ ] Hypothesis synthesis (Architect)
+- [ ] Implementation (Researcher)
+- [ ] Validation (Critic)
+- [ ] Evaluation (Evaluator)
+
+**Loop Notes:**
+_Notes from current research iteration appear here_
+```
+
+<purpose>
+
+STATE.md is the project's short-term memory spanning all phases and sessions.
+
+**Problem it solves:** Information is captured in summaries, issues, and decisions but not systematically consumed. Sessions start without context.
+
+**Solution:** A single, small file that's:
+- Read first in every workflow
+- Updated after every significant action
+- Contains digest of accumulated context
+- Enables instant session restoration
+
+</purpose>
+
+<lifecycle>
+
+**Creation:** After ROADMAP.md is created (during init)
+- Reference PROJECT.md (read it for current context)
+- Initialize empty accumulated context sections
+- Set position to "Phase 1 ready to plan"
+
+**Reading:** First step of every workflow
+- progress: Present status to user
+- plan: Inform planning decisions
+- execute: Know current position
+- transition: Know what's complete
+
+**Writing:** After every significant action
+- execute: After SUMMARY.md created
+  - Update position (phase, plan, status)
+  - Note new decisions (detail in PROJECT.md)
+  - Add blockers/concerns
+- transition: After phase marked complete
+  - Update progress bar
+  - Clear resolved blockers
+  - Refresh Project Reference date
+
+</lifecycle>
+
+<sections>
+
+### Project Reference
+Points to PROJECT.md for full context. Includes:
+- Core value (the ONE thing that matters)
+- Current focus (which phase)
+- Last update date (triggers re-read if stale)
+
+Claude reads PROJECT.md directly for requirements, constraints, and decisions.
+
+### Current Position
+Where we are right now:
+- Phase X of Y — which phase
+- Plan A of B — which plan within phase
+- Status — current state
+- Last activity — what happened most recently
+- Progress bar — visual indicator of overall completion
+
+Progress calculation: (completed plans) / (total plans across all phases) × 100%
+
+### Performance Metrics
+Track velocity to understand execution patterns:
+- Total plans completed
+- Average duration per plan
+- Per-phase breakdown
+- Recent trend (improving/stable/degrading)
+
+Updated after each plan completion.
+
+### Accumulated Context
+
+**Decisions:** Reference to PROJECT.md Key Decisions table, plus recent decisions summary for quick access. Full decision log lives in PROJECT.md.
+
+**Pending Todos:** Ideas captured via /grd:add-todo
+- Count of pending todos
+- Reference to .planning/todos/pending/
+- Brief list if few, count if many (e.g., "5 pending todos — see /grd:check-todos")
+
+**Blockers/Concerns:** From "Next Phase Readiness" sections
+- Issues that affect future work
+- Prefix with originating phase
+- Cleared when addressed
+
+### Session Continuity
+Enables instant resumption:
+- When was last session
+- What was last completed
+- Is there a .continue-here file to resume from
+
+### Research Loop History
+Tracks recursive validation cycles (STATE-01 requirement):
+- **Active Loop**: Which research loop is currently running (or N/A)
+- **Loop Status**: Current stage (idle/exploring/synthesizing/validating/complete)
+- **Loop Table**: History of completed and ongoing loops with outcomes
+- **Current Loop Progress**: Checklist tracking which agents have contributed
+- **Loop Notes**: Insights, decisions, and findings from the current iteration
+
+When a research loop starts (future phases), this section tracks:
+- Explorer's data reconnaissance
+- Architect's hypothesis synthesis
+- Researcher's implementation
+- Critic's validation challenges
+- Evaluator's metric assessments
+
+This enables the recursive "hypothesis → experiment → validate → refine" cycle that distinguishes GRD from linear development workflows.
+
+### Data Revisions Table
+
+Tracks REVISE_DATA cycles within the current hypothesis:
+- **Iteration**: Which experiment iteration triggered data revision
+- **Concerns**: Summary of data concerns from Critic (truncated)
+- **Explorer Result**: Outcome of re-analysis (addressed, critical issue, etc.)
+- **Action Taken**: What happened next (loop continues, escalated, etc.)
+
+Data revisions are tracked separately from method revisions because:
+- Data issues are more fundamental than hyperparameter tuning
+- Lower limit (default 2) prevents infinite data loops
+- Multiple data revisions suggest hypothesis may not be viable with current data
+
+</sections>
+
+<size_constraint>
+
+Keep STATE.md under 100 lines.
+
+It's a DIGEST, not an archive. If accumulated context grows too large:
+- Keep only 3-5 recent decisions in summary (full log in PROJECT.md)
+- Keep only active blockers, remove resolved ones
+
+The goal is "read once, know where we are" — if it's too long, that fails.
+
+</size_constraint>