get-research-done 1.1.0

package/commands/grd/execute-phase.md

@@ -0,0 +1,339 @@
+---
+name: grd:execute-phase
+description: Execute all plans in a phase with wave-based parallelization
+argument-hint: "<phase-number> [--gaps-only]"
+allowed-tools:
+  - Read
+  - Write
+  - Edit
+  - Glob
+  - Grep
+  - Bash
+  - Task
+  - TodoWrite
+  - AskUserQuestion
+---
+
+<objective>
+Execute all plans in a phase using wave-based parallel execution.
+
+Orchestrator stays lean: discover plans, analyze dependencies, group into waves, spawn subagents, collect results. Each subagent loads the full execute-plan context and handles its own plan.
+
+Context budget: ~15% orchestrator, 100% fresh per subagent.
+</objective>
+
+<execution_context>
+@~/.claude/get-research-done/references/ui-brand.md
+@~/.claude/get-research-done/workflows/execute-phase.md
+</execution_context>
+
+<context>
+Phase: $ARGUMENTS
+
+**Flags:**
+- `--gaps-only` — Execute only gap closure plans (plans with `gap_closure: true` in frontmatter). Use after verify-work creates fix plans.
+
+@.planning/ROADMAP.md
+@.planning/STATE.md
+</context>
+
+<process>
+0. **Resolve Model Profile**
+
+   Read model profile for agent spawning:
+   ```bash
+   MODEL_PROFILE=$(cat .planning/config.json 2>/dev/null | grep -o '"model_profile"[[:space:]]*:[[:space:]]*"[^"]*"' | grep -o '"[^"]*"$' | tr -d '"' || echo "balanced")
+   ```
+
+   Default to "balanced" if not set.
+
+   **Model lookup table:**
+
+   | Agent | quality | balanced | budget |
+   |-------|---------|----------|--------|
+   | grd-executor | opus | sonnet | sonnet |
+   | grd-verifier | sonnet | sonnet | haiku |
+
+   Store resolved models for use in Task calls below.
+
+1. **Validate phase exists**
+   - Find phase directory matching argument
+   - Count PLAN.md files
+   - Error if no plans found
+
+2. **Discover plans**
+   - List all *-PLAN.md files in phase directory
+   - Check which have *-SUMMARY.md (already complete)
+   - If `--gaps-only`: filter to only plans with `gap_closure: true`
+   - Build list of incomplete plans
+
+3. **Group by wave**
+   - Read `wave` from each plan's frontmatter
+   - Group plans by wave number
+   - Report wave structure to user
+
+4. **Execute waves**
+   For each wave in order:
+   - Spawn `grd-executor` for each plan in wave (parallel Task calls)
+   - Wait for completion (Task blocks)
+   - Verify SUMMARYs created
+   - Proceed to next wave
+
+5. **Aggregate results**
+   - Collect summaries from all plans
+   - Report phase completion status
+
+6. **Commit any orchestrator corrections**
+   Check for uncommitted changes before verification:
+   ```bash
+   git status --porcelain
+   ```
+
+   **If changes exist:** Orchestrator made corrections between executor completions. Commit them:
+   ```bash
+   git add -u && git commit -m "fix({phase}): orchestrator corrections"
+   ```
+
+   **If clean:** Continue to verification.
+
+7. **Verify phase goal**
+   Check config: `WORKFLOW_VERIFIER=$(cat .planning/config.json 2>/dev/null | grep -o '"verifier"[[:space:]]*:[[:space:]]*[^,}]*' | grep -o 'true\|false' || echo "true")`
+
+   **If `workflow.verifier` is `false`:** Skip to step 8 (treat as passed).
+
+   **Otherwise:**
+   - Spawn `grd-verifier` subagent with phase directory and goal
+   - Verifier checks must_haves against actual codebase (not SUMMARY claims)
+   - Creates VERIFICATION.md with detailed report
+   - Route by status:
+     - `passed` → continue to step 8
+     - `human_needed` → present items, get approval or feedback
+     - `gaps_found` → present gaps, offer `/grd:plan-phase {X} --gaps`
+
+8. **Update roadmap and state**
+   - Update ROADMAP.md, STATE.md
+
+9. **Update requirements**
+   Mark phase requirements as Complete:
+   - Read ROADMAP.md, find this phase's `Requirements:` line (e.g., "AUTH-01, AUTH-02")
+   - Read REQUIREMENTS.md traceability table
+   - For each REQ-ID in this phase: change Status from "Pending" to "Complete"
+   - Write updated REQUIREMENTS.md
+   - Skip if: REQUIREMENTS.md doesn't exist, or phase has no Requirements line
+
+10. **Commit phase completion**
+    Check `COMMIT_PLANNING_DOCS` from config.json (default: true).
+    If false: Skip git operations for .planning/ files.
+    If true: Bundle all phase metadata updates in one commit:
+    - Stage: `git add .planning/ROADMAP.md .planning/STATE.md`
+    - Stage REQUIREMENTS.md if updated: `git add .planning/REQUIREMENTS.md`
+    - Commit: `docs({phase}): complete {phase-name} phase`
+
+11. **Offer next steps**
+    - Route to next action (see `<offer_next>`)
+</process>
+
+<offer_next>
+Output this markdown directly (not as a code block). Route based on status:
+
+| Status | Route |
+|--------|-------|
+| `gaps_found` | Route C (gap closure) |
+| `human_needed` | Present checklist, then re-route based on approval |
+| `passed` + more phases | Route A (next phase) |
+| `passed` + last phase | Route B (milestone complete) |
+
+---
+
+**Route A: Phase verified, more phases remain**
+
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+ GRD ► PHASE {Z} COMPLETE ✓
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+**Phase {Z}: {Name}**
+
+{Y} plans executed
+Goal verified ✓
+
+───────────────────────────────────────────────────────────────
+
+## ▶ Next Up
+
+**Phase {Z+1}: {Name}** — {Goal from ROADMAP.md}
+
+/grd:discuss-phase {Z+1} — gather context and clarify approach
+
+<sub>/clear first → fresh context window</sub>
+
+───────────────────────────────────────────────────────────────
+
+**Also available:**
+- /grd:plan-phase {Z+1} — skip discussion, plan directly
+- /grd:verify-work {Z} — manual acceptance testing before continuing
+
+───────────────────────────────────────────────────────────────
+
+---
+
+**Route B: Phase verified, milestone complete**
+
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+ GRD ► MILESTONE COMPLETE 🎉
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+**v1.0**
+
+{N} phases completed
+All phase goals verified ✓
+
+───────────────────────────────────────────────────────────────
+
+## ▶ Next Up
+
+**Audit milestone** — verify requirements, cross-phase integration, E2E flows
+
+/grd:audit-milestone
+
+<sub>/clear first → fresh context window</sub>
+
+───────────────────────────────────────────────────────────────
+
+**Also available:**
+- /grd:verify-work — manual acceptance testing
+- /grd:complete-milestone — skip audit, archive directly
+
+───────────────────────────────────────────────────────────────
+
+---
+
+**Route C: Gaps found — need additional planning**
+
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+ GRD ► PHASE {Z} GAPS FOUND ⚠
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+**Phase {Z}: {Name}**
+
+Score: {N}/{M} must-haves verified
+Report: .planning/phases/{phase_dir}/{phase}-VERIFICATION.md
+
+### What's Missing
+
+{Extract gap summaries from VERIFICATION.md}
+
+───────────────────────────────────────────────────────────────
+
+## ▶ Next Up
+
+**Plan gap closure** — create additional plans to complete the phase
+
+/grd:plan-phase {Z} --gaps
+
+<sub>/clear first → fresh context window</sub>
+
+───────────────────────────────────────────────────────────────
+
+**Also available:**
+- cat .planning/phases/{phase_dir}/{phase}-VERIFICATION.md — see full report
+- /grd:verify-work {Z} — manual testing before planning
+
+───────────────────────────────────────────────────────────────
+
+---
+
+After user runs /grd:plan-phase {Z} --gaps:
+1. Planner reads VERIFICATION.md gaps
+2. Creates plans 04, 05, etc. to close gaps
+3. User runs /grd:execute-phase {Z} again
+4. Execute-phase runs incomplete plans (04, 05...)
+5. Verifier runs again → loop until passed
+</offer_next>
+
+<wave_execution>
+**Parallel spawning:**
+
+Before spawning, read file contents. The `@` syntax does not work across Task() boundaries.
+
+```bash
+# Read each plan and STATE.md
+PLAN_01_CONTENT=$(cat "{plan_01_path}")
+PLAN_02_CONTENT=$(cat "{plan_02_path}")
+PLAN_03_CONTENT=$(cat "{plan_03_path}")
+STATE_CONTENT=$(cat .planning/STATE.md)
+```
+
+Spawn all plans in a wave with a single message containing multiple Task calls, with inlined content:
+
+```
+Task(prompt="Execute plan at {plan_01_path}\n\nPlan:\n{plan_01_content}\n\nProject state:\n{state_content}", subagent_type="grd-executor", model="{executor_model}")
+Task(prompt="Execute plan at {plan_02_path}\n\nPlan:\n{plan_02_content}\n\nProject state:\n{state_content}", subagent_type="grd-executor", model="{executor_model}")
+Task(prompt="Execute plan at {plan_03_path}\n\nPlan:\n{plan_03_content}\n\nProject state:\n{state_content}", subagent_type="grd-executor", model="{executor_model}")
+```
+
+All three run in parallel. Task tool blocks until all complete.
+
+**No polling.** No background agents. No TaskOutput loops.
+</wave_execution>
+
+<checkpoint_handling>
+Plans with `autonomous: false` have checkpoints. The execute-phase.md workflow handles the full checkpoint flow:
+- Subagent pauses at checkpoint, returns structured state
+- Orchestrator presents to user, collects response
+- Spawns fresh continuation agent (not resume)
+
+See `@~/.claude/get-research-done/workflows/execute-phase.md` step `checkpoint_handling` for complete details.
+</checkpoint_handling>
+
+<deviation_rules>
+During execution, handle discoveries automatically:
+
+1. **Auto-fix bugs** - Fix immediately, document in Summary
+2. **Auto-add critical** - Security/correctness gaps, add and document
+3. **Auto-fix blockers** - Can't proceed without fix, do it and document
+4. **Ask about architectural** - Major structural changes, stop and ask user
+
+Only rule 4 requires user intervention.
+</deviation_rules>
+
+<commit_rules>
+**Per-Task Commits:**
+
+After each task completes:
+1. Stage only files modified by that task
+2. Commit with format: `{type}({phase}-{plan}): {task-name}`
+3. Types: feat, fix, test, refactor, perf, chore
+4. Record commit hash for SUMMARY.md
+
+**Plan Metadata Commit:**
+
+After all tasks in a plan complete:
+1. Stage plan artifacts only: PLAN.md, SUMMARY.md
+2. Commit with format: `docs({phase}-{plan}): complete [plan-name] plan`
+3. NO code files (already committed per-task)
+
+**Phase Completion Commit:**
+
+After all plans in phase complete (step 7):
+1. Stage: ROADMAP.md, STATE.md, REQUIREMENTS.md (if updated), VERIFICATION.md
+2. Commit with format: `docs({phase}): complete {phase-name} phase`
+3. Bundles all phase-level state updates in one commit
+
+**NEVER use:**
+- `git add .`
+- `git add -A`
+- `git add src/` or any broad directory
+
+**Always stage files individually.**
+</commit_rules>
+
+<success_criteria>
+- [ ] All incomplete plans in phase executed
+- [ ] Each plan has SUMMARY.md
+- [ ] Phase goal verified (must_haves checked against codebase)
+- [ ] VERIFICATION.md created in phase directory
+- [ ] STATE.md reflects phase completion
+- [ ] ROADMAP.md updated
+- [ ] REQUIREMENTS.md updated (phase requirements marked Complete)
+- [ ] User informed of next steps
+</success_criteria>

package/commands/grd/explore.md

@@ -0,0 +1,258 @@
+---
+name: grd:explore
+description: Analyze raw data and generate DATA_REPORT.md with distributions, outliers, anomalies, and leakage detection
+allowed-tools:
+  - Read
+  - Bash
+  - Write
+  - Task
+  - AskUserQuestion
+---
+
+<objective>
+
+Explore and profile raw datasets to surface patterns, anomalies, and potential issues before hypothesis formation.
+
+This command performs data reconnaissance—generating a comprehensive DATA_REPORT.md that reveals distributions, outliers, missing data patterns, class imbalance, and potential data leakage risks.
+
+**Creates:**
+- `.planning/DATA_REPORT.md` — comprehensive data profile and leakage analysis
+
+**Use cases:**
+- New dataset: Understand data characteristics before modeling
+- Suspicious results: Check for leakage or data quality issues
+- Feature engineering: Identify patterns and anomalies to inform feature creation
+- Debugging: Profile data when model behavior is unexpected
+
+**After this command:** Review DATA_REPORT.md for issues, then proceed with hypothesis formation and experimentation.
+
+</objective>
+
+<execution_context>
+
+@~/.claude/get-research-done/templates/data-report.md
+
+</execution_context>
+
+<process>
+
+## Phase 1: Setup and Data Path Resolution
+
+**Check if project initialized:**
+
+```bash
+[ ! -f .planning/PROJECT.md ] && echo "ERROR: Project not initialized. Run /grd:new-project first." && exit 1
+```
+
+**Determine data path:**
+
+If [path] argument provided:
+- Use it directly
+- Validate path exists
+
+If no argument:
+- Check for common data directories (./data/, ./datasets/, ./raw/)
+- If found, list contents and ask user to select
+- If not found, ask user to provide path interactively
+
+**Validate data source:**
+
+```bash
+# Check if path exists
+if [ -f "$DATA_PATH" ]; then
+    echo "Single file: $DATA_PATH"
+elif [ -d "$DATA_PATH" ]; then
+    echo "Directory: $DATA_PATH"
+    # List data files
+    find "$DATA_PATH" -type f \( -name "*.csv" -o -name "*.parquet" -o -name "*.json" -o -name "*.jsonl" \) | head -20
+else
+    echo "ERROR: Path not found: $DATA_PATH"
+    exit 1
+fi
+```
+
+**Check for flags:**
+
+- `--detailed`: Enable full profiling mode (histograms, percentiles, skewness, correlation heatmaps)
+- Default: Quick profiling (basic stats, outliers, leakage checks)
+
+## Phase 2: Spawn Explorer Agent
+
+Display exploration banner:
+```
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+ GRD ► EXPLORING DATA
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+Analyzing: [data_path]
+Mode: [quick | detailed]
+```
+
+Spawn grd-explorer agent with context:
+
+```
+Task(prompt="
+<data_source>
+Path: [data_path]
+Type: [file | directory]
+</data_source>
+
+<profiling_mode>
+[quick | detailed]
+
+Quick: Basic stats, distributions, outliers, leakage checks (fast)
+Detailed: Full profiling with histograms, percentiles, skewness, correlation matrices (comprehensive)
+</profiling_mode>
+
+<project_context>
+@.planning/PROJECT.md
+
+Extract:
+- What this project is about
+- What the target variable might be (if ML project)
+- Any known data characteristics
+</project_context>
+
+<instructions>
+Execute data exploration workflow:
+
+1. Load data (handle CSV, Parquet, JSON, JSONL)
+2. Profile structure and distributions
+3. Detect missing data patterns (MCAR/MAR/MNAR)
+4. Find outliers and anomalies
+5. Check class balance (if target identified)
+6. Detect potential data leakage
+7. Generate recommendations
+
+Write: .planning/DATA_REPORT.md
+Use template: ~/.claude/get-research-done/templates/data-report.md
+</instructions>
+
+<output>
+Return DATA_REPORT.md path when complete.
+</output>
+", subagent_type="grd-explorer", model="sonnet", description="Explore and profile data")
+```
+
+## Phase 3: Present Results
+
+After agent completes, read DATA_REPORT.md and present key findings:
+
+```
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+ GRD ► EXPLORATION COMPLETE ✓
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+## Dataset Summary
+
+**Rows:** [count] | **Columns:** [count] | **Memory:** [size]
+
+## Key Findings
+
+### Must Address (Blocking)
+[List critical issues from recommendations]
+
+### Should Address (Non-blocking)
+[List recommended fixes]
+
+### Leakage Risks
+[High/medium confidence leakage indicators]
+
+### Data Quality
+[Missing data, outliers, imbalance summary]
+
+---
+
+**Full report:** `.planning/DATA_REPORT.md`
+
+**Next steps:**
+- Address blocking issues before modeling
+- Consider should-fix items for better results
+- Review leakage risks carefully—high confidence indicates problems
+```
+
+**If critical issues found:**
+
+Offer to create tasks/reminders for fixing them:
+
+Use AskUserQuestion:
+- header: "Data Issues"
+- question: "Critical data issues detected. Create follow-up tasks?"
+- options:
+  - "Yes" — Add to project backlog
+  - "No" — I'll handle manually
+
+If yes, create TODO entries or add to requirements tracking.
+
+</process>
+
+<arguments>
+
+**[path]** (optional)
+- Path to data file or directory
+- Examples: `./data/train.csv`, `s3://bucket/data.parquet`, `./datasets/`
+- If omitted, will prompt interactively
+
+**Flags:**
+
+`--detailed`
+- Enable comprehensive profiling
+- Includes: histograms, percentiles, skewness, kurtosis, correlation matrices
+- Use when: Deep analysis needed or investigating specific issues
+- Default: Quick mode (faster, covers essentials)
+
+</arguments>
+
+<examples>
+
+**Explore single file:**
+```
+/grd:explore ./data/train.csv
+```
+
+**Explore directory:**
+```
+/grd:explore ./datasets/
+```
+
+**Detailed profiling:**
+```
+/grd:explore ./data/train.csv --detailed
+```
+
+**Interactive mode (no arguments):**
+```
+/grd:explore
+# Will prompt for path
+```
+
+**Remote data:**
+```
+/grd:explore s3://my-bucket/data.parquet
+```
+
+</examples>
+
+<output>
+
+- `.planning/DATA_REPORT.md` — comprehensive data profile with:
+  - Data overview (shape, types, memory)
+  - Column summaries (types, nulls, unique values)
+  - Distributions and statistics
+  - Missing data analysis
+  - Outlier detection
+  - Class balance (if target specified)
+  - Data leakage analysis
+  - Recommendations
+
+</output>
+
+<success_criteria>
+
+- [ ] Data path resolved (from argument or user input)
+- [ ] grd-explorer agent spawned with context
+- [ ] DATA_REPORT.md generated in .planning/
+- [ ] Key findings presented to user
+- [ ] Critical issues surfaced with next steps
+
+</success_criteria>