deepflow 0.1.26 → 0.1.28

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "deepflow",
-  "version": "0.1.26",
+  "version": "0.1.28",
   "description": "Stay in flow state - lightweight spec-driven task orchestration for Claude Code",
   "keywords": [
     "claude",

package/src/commands/df/execute.md

@@ -29,6 +29,7 @@ Implement tasks from PLAN.md with parallel agents, atomic commits, and context-e
 | Agent | subagent_type | model | Purpose |
 |-------|---------------|-------|---------|
 | Implementation | `general-purpose` | `sonnet` | Task implementation |
+| Spike Verifier | `reasoner` | `opus` | Verify spike pass/fail is correct |
 | Debugger | `reasoner` | `opus` | Debugging failures |
 
 ## Context-Aware Execution
@@ -57,24 +58,90 @@ commit: abc1234
 summary: "one line"
 ```
 
+**Spike result file** `.deepflow/results/{task_id}.yaml` (additional fields):
+```yaml
+task: T1
+type: spike
+status: success|failed
+commit: abc1234
+summary: "one line"
+criteria:
+  - name: "throughput"
+    target: ">= 7000 g/s"
+    actual: "1500 g/s"
+    met: false
+  - name: "memory usage"
+    target: "< 500 MB"
+    actual: "320 MB"
+    met: true
+all_criteria_met: false  # ALL must be true for spike to pass
+experiment_file: ".deepflow/experiments/upload--streaming--failed.md"
+```
+
+**CRITICAL:** `status` MUST equal `success` only if `all_criteria_met: true`. The spike verifier will reject mismatches.
+
 ## Checkpoint & Resume
 
-**File:** `.deepflow/checkpoint.json` — stores completed tasks, current wave.
+**File:** `.deepflow/checkpoint.json` — stored in WORKTREE directory, not main.
 
-**On checkpoint:** Complete wave → update PLAN.md → save → exit.
-**Resume:** `--continue` loads checkpoint, skips completed tasks.
+**Schema:**
+```json
+{
+  "completed_tasks": ["T1", "T2"],
+  "current_wave": 2,
+  "worktree_path": ".deepflow/worktrees/df/doing-upload/20260202-1430",
+  "worktree_branch": "df/doing-upload/20260202-1430"
+}
+```
+
+**On checkpoint:** Complete wave → update PLAN.md → save to worktree → exit.
+**Resume:** `--continue` loads checkpoint, verifies worktree, skips completed tasks.
 
 ## Behavior
 
 ### 1. CHECK CHECKPOINT
 
 ```
---continue → Load and resume
+--continue → Load checkpoint
+  → If worktree_path exists:
+    → Verify worktree still exists on disk
+    → If missing: Error "Worktree deleted. Use --fresh"
+    → If exists: Use it, skip worktree creation
+  → Resume execution with completed tasks
 --fresh → Delete checkpoint, start fresh
 checkpoint exists → Prompt: "Resume? (y/n)"
 else → Start fresh
 ```
 
+### 1.5. CREATE WORKTREE
+
+Before spawning any agents, create an isolated worktree:
+
+```
+# Check main is clean (ignore untracked)
+git diff --quiet HEAD || Error: "Main has uncommitted changes. Commit or stash first."
+
+# Generate worktree path
+SPEC_NAME=$(basename spec/doing-*.md .md | sed 's/doing-//')
+TIMESTAMP=$(date +%Y%m%d-%H%M)
+BRANCH_NAME="df/${SPEC_NAME}/${TIMESTAMP}"
+WORKTREE_PATH=".deepflow/worktrees/${BRANCH_NAME}"
+
+# Create worktree
+git worktree add -b "${BRANCH_NAME}" "${WORKTREE_PATH}"
+
+# Store in checkpoint for resume
+checkpoint.worktree_path = WORKTREE_PATH
+checkpoint.worktree_branch = BRANCH_NAME
+```
+
+**Resume handling:**
+- If checkpoint has worktree_path → verify it exists, use it
+- If worktree missing → Error: "Worktree deleted. Use --fresh"
+
+**Existing worktree handling:**
+- If worktree exists for same spec → Prompt: "Resume existing worktree? (y/n/delete)"
+
 ### 2. LOAD PLAN
 
 ```
@@ -158,9 +225,57 @@ Same-file conflicts: spawn sequentially instead.
 **Spike Task Execution:**
 When spawning a spike task, the agent MUST:
 1. Execute the minimal validation method
-2. Record result in experiment file (update status: `--passed.md` or `--failed.md`)
-3. If passed: implementation tasks become unblocked
-4. If failed: record conclusion with "next hypothesis" for future planning
+2. Record structured criteria evaluation in result file (see spike result schema above)
+3. Write experiment file with `--active.md` status (verifier determines final status)
+4. Commit as `spike({spec}): validate {hypothesis}`
+
+**IMPORTANT:** Spike agent writes `--active.md`, NOT `--passed.md` or `--failed.md`. The verifier determines final status.
+
+### 6.5. VERIFY SPIKE RESULTS
+
+After spike completes, spawn verifier BEFORE unblocking implementation tasks.
+
+**Trigger:** Spike result file detected (`.deepflow/results/T{n}.yaml` with `type: spike`)
+
+**Spawn:**
+```
+Task(subagent_type="reasoner", model="opus", prompt=VERIFIER_PROMPT)
+```
+
+**Verifier Prompt:**
+```
+SPIKE VERIFICATION — Be skeptical. Catch false positives.
+
+Task: {task_id}
+Result: {worktree_path}/.deepflow/results/{task_id}.yaml
+Experiment: {worktree_path}/.deepflow/experiments/{topic}--{hypothesis}--active.md
+
+For each criterion in result file:
+1. Is `actual` a concrete number? (reject "good", "improved", "better")
+2. Does `actual` satisfy `target`? Do the math.
+3. Is `met` correct?
+
+Reject these patterns:
+- "Works but doesn't meet target" → FAILED
+- "Close enough" → FAILED
+- Actual 1500 vs Target >= 7000 → FAILED
+
+Output to {worktree_path}/.deepflow/results/{task_id}-verified.yaml:
+  verified_status: VERIFIED_PASS|VERIFIED_FAIL
+  override: true|false
+  reason: "one line"
+
+Then rename experiment:
+- VERIFIED_PASS → --passed.md
+- VERIFIED_FAIL → --failed.md (add "Next hypothesis:" to Conclusion)
+```
+
+**Gate:**
+```
+VERIFIED_PASS → Unblock, log "✓ Spike {task_id} verified"
+VERIFIED_FAIL → Block, log "✗ Spike {task_id} failed verification"
+  If override: log "⚠ Agent incorrectly marked as passed"
+```
 
 **On failure, use Task tool to spawn reasoner:**
 ```
@@ -178,42 +293,87 @@ Task tool parameters:
 Files: {target files}
 Spec: {spec_name}
 
+**IMPORTANT: Working Directory**
+All file operations MUST use this absolute path as base:
+{worktree_absolute_path}
+
+Example: To edit src/foo.ts, use:
+{worktree_absolute_path}/src/foo.ts
+
+Do NOT write files to the main project directory.
+
 Implement, test, commit as feat({spec}): {description}.
-Write result to .deepflow/results/{task_id}.yaml
+Write result to {worktree_absolute_path}/.deepflow/results/{task_id}.yaml
 ```
 
 **Spike Task:**
 ```
 {task_id} [SPIKE]: {hypothesis}
 Type: spike
-Method: {minimal steps to validate}
-Success criteria: {how to know it passed}
-Time-box: {duration}
-Experiment file: {.deepflow/experiments/{topic}--{hypothesis}--active.md}
-Spec: {spec_name}
+Method: {minimal steps}
+Success criteria: {measurable targets}
+Experiment file: {worktree_absolute_path}/.deepflow/experiments/{topic}--{hypothesis}--active.md
+
+Working directory: {worktree_absolute_path}
+
+Steps:
+1. Execute method
+2. For EACH criterion: record target, measure actual, compare (show math)
+3. Write experiment as --active.md (verifier determines final status)
+4. Commit: spike({spec}): validate {hypothesis}
+5. Write result to .deepflow/results/{task_id}.yaml (see spike result schema)
+
+Rules:
+- `met: true` ONLY if actual satisfies target
+- `status: success` ONLY if ALL criteria met
+- Worse than baseline = FAILED (baseline 7k, actual 1.5k → FAILED)
+- "Close enough" = FAILED
+- Verifier will check. False positives waste resources.
+```
+
+### 8. FAILURE HANDLING
 
-Execute the minimal validation:
-1. Follow the method steps exactly
-2. Measure against success criteria
-3. Update experiment file with result:
-   - If passed: rename to --passed.md, record findings
-   - If failed: rename to --failed.md, record conclusion with "next hypothesis"
-4. Commit as spike({spec}): validate {hypothesis}
-5. Write result to .deepflow/results/{task_id}.yaml
+When a task fails and cannot be auto-fixed:
 
-Result status:
-- success = hypothesis validated (passed)
-- failed = hypothesis invalidated (failed experiment, NOT agent error)
+**Behavior:**
+1. Leave worktree intact at `{worktree_path}`
+2. Keep checkpoint.json for potential resume
+3. Output debugging instructions
+
+**Output:**
 ```
+✗ Task T3 failed after retry
+
+Worktree preserved for debugging:
+  Path: .deepflow/worktrees/df/doing-upload/20260202-1430
+  Branch: df/doing-upload/20260202-1430
+
+To investigate:
+  cd .deepflow/worktrees/df/doing-upload/20260202-1430
+  # examine files, run tests, etc.
+
+To resume after fixing:
+  /df:execute --continue
+
+To discard and start fresh:
+  git worktree remove --force .deepflow/worktrees/df/doing-upload/20260202-1430
+  git branch -D df/doing-upload/20260202-1430
+  /df:execute --fresh
+```
+
+**Key points:**
+- Never auto-delete worktree on failure (cleanup_on_fail: false by default)
+- Always provide the exact cleanup commands
+- Checkpoint remains so --continue can work after manual fix
 
-### 8. COMPLETE SPECS
+### 9. COMPLETE SPECS
 
 When all tasks done for a `doing-*` spec:
 1. Embed history in spec: `## Completed` section
 2. Rename: `doing-upload.md` → `done-upload.md`
 3. Remove section from PLAN.md
 
-### 9. ITERATE
+### 10. ITERATE
 
 Repeat until: all done, all blocked, or checkpoint.
 
@@ -253,14 +413,14 @@ Checking experiment status...
   T2: Blocked by T1 (spike not validated)
   T3: Blocked by T1 (spike not validated)
 
-Wave 1: T1 [SPIKE] (context: 20%)
-  T1: success (abc1234) → upload--streaming--passed.md
+Wave 1: T1 [SPIKE] (context: 15%)
+  T1: complete, verifying...
 
-Checking experiment status...
-  T2: Experiment passed, unblocked
-  T3: Experiment passed, unblocked
+Verifying T1...
+  ✓ Spike T1 verified (throughput 8500 >= 7000)
+  → upload--streaming--passed.md
 
-Wave 2: T2, T3 parallel (context: 45%)
+Wave 2: T2, T3 parallel (context: 40%)
   T2: success (def5678)
   T3: success (ghi9012)
 
@@ -268,20 +428,38 @@ Wave 2: T2, T3 parallel (context: 45%)
 ✓ Complete: 3/3 tasks
 ```
 
-### Spike Failed
+### Spike Failed (Agent Correctly Reported)
 
 ```
 /df:execute (context: 10%)
 
-Wave 1: T1 [SPIKE] (context: 20%)
-  T1: failed → upload--streaming--failed.md
+Wave 1: T1 [SPIKE] (context: 15%)
+  T1: complete, verifying...
 
-Checking experiment status...
-  T2: ⚠ Blocked - Experiment failed
-  T3: ⚠ Blocked - Experiment failed
+Verifying T1...
+  ✗ Spike T1 failed verification (throughput 1500 < 7000)
+  → upload--streaming--failed.md
+
+⚠ Spike T1 invalidated hypothesis
+  → Run /df:plan to generate new hypothesis spike
+
+Complete: 1/3 tasks (2 blocked by failed experiment)
+```
+
+### Spike Failed (Verifier Override)
+
+```
+/df:execute (context: 10%)
+
+Wave 1: T1 [SPIKE] (context: 15%)
+  T1: complete (agent said: success), verifying...
+
+Verifying T1...
+  ✗ Spike T1 failed verification (throughput 1500 < 7000)
+  ⚠ Agent incorrectly marked as passed — overriding to FAILED
+  → upload--streaming--failed.md
 
 ⚠ Spike T1 invalidated hypothesis
-  Experiment: upload--streaming--failed.md
   → Run /df:plan to generate new hypothesis spike
 
 Complete: 1/3 tasks (2 blocked by failed experiment)

package/src/commands/df/verify.md

@@ -115,3 +115,46 @@ Learnings captured:
   → experiments/perf--streaming-upload--success.md
   → experiments/auth--jwt-refresh-rotation--success.md
 ```
+
+## Post-Verification: Worktree Merge & Cleanup
+
+After all verification passes:
+
+### 1. MERGE TO MAIN
+
+```bash
+# Get worktree info from checkpoint
+WORKTREE_BRANCH=$(cat .deepflow/checkpoint.json | jq -r '.worktree_branch')
+
+# Switch to main and merge
+git checkout main
+git merge "${WORKTREE_BRANCH}" --no-ff -m "feat({spec}): merge verified changes"
+```
+
+**On merge conflict:**
+- Keep worktree intact for manual resolution
+- Output: "Merge conflict detected. Resolve manually, then run /df:verify --merge-only"
+- Exit without cleanup
+
+### 2. CLEANUP WORKTREE
+
+After successful merge:
+
+```bash
+# Get worktree path from checkpoint
+WORKTREE_PATH=$(cat .deepflow/checkpoint.json | jq -r '.worktree_path')
+
+# Remove worktree and branch
+git worktree remove --force "${WORKTREE_PATH}"
+git branch -d "${WORKTREE_BRANCH}"
+
+# Remove checkpoint
+rm .deepflow/checkpoint.json
+```
+
+**Output on success:**
+```
+✓ Merged df/doing-upload/20260202-1430 to main
+✓ Cleaned up worktree and branch
+✓ Spec complete: doing-upload → done-upload
+```

package/templates/config-template.yaml

@@ -43,3 +43,21 @@ commits:
   format: "feat({spec}): {description}"
   atomic: true         # One task = one commit
   push_after: complete # Or "each" for every commit
+
+# Worktree isolation for /df:execute
+# Isolates all agent work in a git worktree, keeping main clean
+worktree:
+  # Enable worktree isolation (default: true)
+  enabled: true
+
+  # Base path for worktrees relative to project root
+  base_path: .deepflow/worktrees
+
+  # Branch name prefix for worktree branches
+  branch_prefix: df/
+
+  # Automatically cleanup worktree after successful verify
+  cleanup_on_success: true
+
+  # Keep worktree after failed execution for debugging
+  cleanup_on_fail: false