deepflow 0.1.27 → 0.1.29

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "deepflow",
-  "version": "0.1.27",
+  "version": "0.1.29",
   "description": "Stay in flow state - lightweight spec-driven task orchestration for Claude Code",
   "keywords": [
     "claude",

package/src/commands/df/execute.md

@@ -2,11 +2,11 @@
 
 ## Orchestrator Role
 
-You spawn agents and poll results. You never implement.
+You are a coordinator. Spawn agents, wait for results, update PLAN.md. Never implement code yourself.
 
-**NEVER:** Read source files, edit code, run tests, run git (except status), use `TaskOutput`
+**NEVER:** Read source files, edit code, run tests, run git commands (except status)
 
-**ONLY:** Read `PLAN.md` + `specs/doing-*.md`, spawn background agents, poll `.deepflow/results/`, update PLAN.md
+**ONLY:** Read PLAN.md, read specs/doing-*.md, spawn background agents, use TaskOutput to get results, update PLAN.md
 
 ---
 
@@ -29,6 +29,7 @@ Implement tasks from PLAN.md with parallel agents, atomic commits, and context-e
 | Agent | subagent_type | model | Purpose |
 |-------|---------------|-------|---------|
 | Implementation | `general-purpose` | `sonnet` | Task implementation |
+| Spike Verifier | `reasoner` | `opus` | Verify spike pass/fail is correct |
 | Debugger | `reasoner` | `opus` | Debugging failures |
 
 ## Context-Aware Execution
@@ -42,11 +43,16 @@ Statusline writes to `.deepflow/context.json`: `{"percentage": 45}`
 
 ## Agent Protocol
 
-Every task = one background agent. Poll result files, never `TaskOutput`.
+Each task = one background agent. Use TaskOutput to wait for results. Never poll files in a loop.
 
 ```python
-Task(subagent_type="general-purpose", run_in_background=True, prompt="T1: ...")
-# Poll: Glob(".deepflow/results/T*.yaml")
+# Spawn agents in parallel (single message, multiple Task calls)
+task_id_1 = Task(subagent_type="general-purpose", run_in_background=True, prompt="T1: ...")
+task_id_2 = Task(subagent_type="general-purpose", run_in_background=True, prompt="T2: ...")
+
+# Wait for all results (single message, multiple TaskOutput calls)
+TaskOutput(task_id=task_id_1)
+TaskOutput(task_id=task_id_2)
 ```
 
 Result file `.deepflow/results/{task_id}.yaml`:
@@ -57,6 +63,28 @@ commit: abc1234
 summary: "one line"
 ```
 
+**Spike result file** `.deepflow/results/{task_id}.yaml` (additional fields):
+```yaml
+task: T1
+type: spike
+status: success|failed
+commit: abc1234
+summary: "one line"
+criteria:
+  - name: "throughput"
+    target: ">= 7000 g/s"
+    actual: "1500 g/s"
+    met: false
+  - name: "memory usage"
+    target: "< 500 MB"
+    actual: "320 MB"
+    met: true
+all_criteria_met: false  # ALL must be true for spike to pass
+experiment_file: ".deepflow/experiments/upload--streaming--failed.md"
+```
+
+**CRITICAL:** `status` MUST equal `success` only if `all_criteria_met: true`. The spike verifier will reject mismatches.
+
 ## Checkpoint & Resume
 
 **File:** `.deepflow/checkpoint.json` — stored in WORKTREE directory, not main.
@@ -188,23 +216,78 @@ Ready = `[ ]` + all `blocked_by` complete + experiment validated (if applicable)
 
 Context ≥50%: checkpoint and exit.
 
-**Use Task tool to spawn all ready tasks in ONE message (parallel):**
+**CRITICAL: Spawn ALL ready tasks in a SINGLE response with MULTIPLE Task tool calls.**
+
+DO NOT spawn one task, wait, then spawn another. Instead, call Task tool multiple times in the SAME message block. This enables true parallelism.
+
+Example: If T1, T2, T3 are ready, send ONE message containing THREE Task tool invocations:
+
 ```
-Task tool parameters for each task:
-- subagent_type: "general-purpose"
-- model: "sonnet"
-- run_in_background: true
-- prompt: "{task details from PLAN.md}"
+// In a SINGLE assistant message, invoke Task THREE times:
+Task(subagent_type="general-purpose", model="sonnet", run_in_background=true, prompt="T1: ...")
+Task(subagent_type="general-purpose", model="sonnet", run_in_background=true, prompt="T2: ...")
+Task(subagent_type="general-purpose", model="sonnet", run_in_background=true, prompt="T3: ...")
 ```
 
+**WRONG (sequential):** Send message with Task for T1 → wait → send message with Task for T2 → wait → ...
+**RIGHT (parallel):** Send ONE message with Task for T1, T2, T3 all together
+
 Same-file conflicts: spawn sequentially instead.
 
 **Spike Task Execution:**
 When spawning a spike task, the agent MUST:
 1. Execute the minimal validation method
-2. Record result in experiment file (update status: `--passed.md` or `--failed.md`)
-3. If passed: implementation tasks become unblocked
-4. If failed: record conclusion with "next hypothesis" for future planning
+2. Record structured criteria evaluation in result file (see spike result schema above)
+3. Write experiment file with `--active.md` status (verifier determines final status)
+4. Commit as `spike({spec}): validate {hypothesis}`
+
+**IMPORTANT:** Spike agent writes `--active.md`, NOT `--passed.md` or `--failed.md`. The verifier determines final status.
+
+### 6.5. VERIFY SPIKE RESULTS
+
+After spike completes, spawn verifier BEFORE unblocking implementation tasks.
+
+**Trigger:** Spike result file detected (`.deepflow/results/T{n}.yaml` with `type: spike`)
+
+**Spawn:**
+```
+Task(subagent_type="reasoner", model="opus", prompt=VERIFIER_PROMPT)
+```
+
+**Verifier Prompt:**
+```
+SPIKE VERIFICATION — Be skeptical. Catch false positives.
+
+Task: {task_id}
+Result: {worktree_path}/.deepflow/results/{task_id}.yaml
+Experiment: {worktree_path}/.deepflow/experiments/{topic}--{hypothesis}--active.md
+
+For each criterion in result file:
+1. Is `actual` a concrete number? (reject "good", "improved", "better")
+2. Does `actual` satisfy `target`? Do the math.
+3. Is `met` correct?
+
+Reject these patterns:
+- "Works but doesn't meet target" → FAILED
+- "Close enough" → FAILED
+- Actual 1500 vs Target >= 7000 → FAILED
+
+Output to {worktree_path}/.deepflow/results/{task_id}-verified.yaml:
+  verified_status: VERIFIED_PASS|VERIFIED_FAIL
+  override: true|false
+  reason: "one line"
+
+Then rename experiment:
+- VERIFIED_PASS → --passed.md
+- VERIFIED_FAIL → --failed.md (add "Next hypothesis:" to Conclusion)
+```
+
+**Gate:**
+```
+VERIFIED_PASS → Unblock, log "✓ Spike {task_id} verified"
+VERIFIED_FAIL → Block, log "✗ Spike {task_id} failed verification"
+  If override: log "⚠ Agent incorrectly marked as passed"
+```
 
 **On failure, use Task tool to spawn reasoner:**
 ```
@@ -239,33 +322,25 @@ Write result to {worktree_absolute_path}/.deepflow/results/{task_id}.yaml
 ```
 {task_id} [SPIKE]: {hypothesis}
 Type: spike
-Method: {minimal steps to validate}
-Success criteria: {how to know it passed}
-Time-box: {duration}
+Method: {minimal steps}
+Success criteria: {measurable targets}
 Experiment file: {worktree_absolute_path}/.deepflow/experiments/{topic}--{hypothesis}--active.md
-Spec: {spec_name}
 
-**IMPORTANT: Working Directory**
-All file operations MUST use this absolute path as base:
-{worktree_absolute_path}
-
-Example: To edit src/foo.ts, use:
-{worktree_absolute_path}/src/foo.ts
-
-Do NOT write files to the main project directory.
+Working directory: {worktree_absolute_path}
 
-Execute the minimal validation:
-1. Follow the method steps exactly
-2. Measure against success criteria
-3. Update experiment file with result:
-   - If passed: rename to --passed.md, record findings
-   - If failed: rename to --failed.md, record conclusion with "next hypothesis"
-4. Commit as spike({spec}): validate {hypothesis}
-5. Write result to {worktree_absolute_path}/.deepflow/results/{task_id}.yaml
+Steps:
+1. Execute method
+2. For EACH criterion: record target, measure actual, compare (show math)
+3. Write experiment as --active.md (verifier determines final status)
+4. Commit: spike({spec}): validate {hypothesis}
+5. Write result to .deepflow/results/{task_id}.yaml (see spike result schema)
 
-Result status:
-- success = hypothesis validated (passed)
-- failed = hypothesis invalidated (failed experiment, NOT agent error)
+Rules:
+- `met: true` ONLY if actual satisfies target
+- `status: success` ONLY if ALL criteria met
+- Worse than baseline = FAILED (baseline 7k, actual 1.5k → FAILED)
+- "Close enough" = FAILED
+- Verifier will check. False positives waste resources.
 ```
 
 ### 8. FAILURE HANDLING
@@ -312,7 +387,18 @@ When all tasks done for a `doing-*` spec:
 
 ### 10. ITERATE
 
-Repeat until: all done, all blocked, or checkpoint.
+After spawning agents, wait for results using TaskOutput. Call TaskOutput for ALL running agents in a SINGLE message (parallel wait).
+
+```python
+# After spawning T1, T2, T3 in parallel, wait for all in parallel:
+TaskOutput(task_id=t1_id)  # These three calls go in ONE message
+TaskOutput(task_id=t2_id)
+TaskOutput(task_id=t3_id)
+```
+
+Then check which tasks completed, update PLAN.md, identify newly unblocked tasks, spawn next wave.
+
+Repeat until: all done, all blocked, or context ≥50% (checkpoint).
 
 ## Rules
 
@@ -338,6 +424,8 @@ Wave 2: T3 (context: 48%)
 
 ✓ doing-upload → done-upload
 ✓ Complete: 3/3 tasks
+
+Next: Run /df:verify to verify specs and merge to main
 ```
 
 ### Spike-First Execution
@@ -350,43 +438,65 @@ Checking experiment status...
   T2: Blocked by T1 (spike not validated)
   T3: Blocked by T1 (spike not validated)
 
-Wave 1: T1 [SPIKE] (context: 20%)
-  T1: success (abc1234) → upload--streaming--passed.md
+Wave 1: T1 [SPIKE] (context: 15%)
+  T1: complete, verifying...
 
-Checking experiment status...
-  T2: Experiment passed, unblocked
-  T3: Experiment passed, unblocked
+Verifying T1...
+  ✓ Spike T1 verified (throughput 8500 >= 7000)
+  → upload--streaming--passed.md
 
-Wave 2: T2, T3 parallel (context: 45%)
+Wave 2: T2, T3 parallel (context: 40%)
   T2: success (def5678)
   T3: success (ghi9012)
 
 ✓ doing-upload → done-upload
 ✓ Complete: 3/3 tasks
+
+Next: Run /df:verify to verify specs and merge to main
 ```
 
-### Spike Failed
+### Spike Failed (Agent Correctly Reported)
 
 ```
 /df:execute (context: 10%)
 
-Wave 1: T1 [SPIKE] (context: 20%)
-  T1: failed → upload--streaming--failed.md
+Wave 1: T1 [SPIKE] (context: 15%)
+  T1: complete, verifying...
 
-Checking experiment status...
-  T2: ⚠ Blocked - Experiment failed
-  T3: ⚠ Blocked - Experiment failed
+Verifying T1...
+  ✗ Spike T1 failed verification (throughput 1500 < 7000)
+  → upload--streaming--failed.md
 
 ⚠ Spike T1 invalidated hypothesis
-  Experiment: upload--streaming--failed.md
-  → Run /df:plan to generate new hypothesis spike
+Complete: 1/3 tasks (2 blocked by failed experiment)
 
+Next: Run /df:plan to generate new hypothesis spike
+```
+
+### Spike Failed (Verifier Override)
+
+```
+/df:execute (context: 10%)
+
+Wave 1: T1 [SPIKE] (context: 15%)
+  T1: complete (agent said: success), verifying...
+
+Verifying T1...
+  ✗ Spike T1 failed verification (throughput 1500 < 7000)
+  ⚠ Agent incorrectly marked as passed — overriding to FAILED
+  → upload--streaming--failed.md
+
+⚠ Spike T1 invalidated hypothesis
 Complete: 1/3 tasks (2 blocked by failed experiment)
+
+Next: Run /df:plan to generate new hypothesis spike
 ```
 
 ### With Checkpoint
 
 ```
 Wave 1 complete (context: 52%)
-Checkpoint saved. Run /df:execute --continue
+Checkpoint saved.
+
+Next: Run /df:execute --continue to resume execution
 ```

package/src/commands/df/plan.md

@@ -81,12 +81,15 @@ Include patterns in task descriptions for agents to follow.
 
 ### 4. ANALYZE CODEBASE
 
-**Use Task tool to spawn Explore agents in parallel:**
+**Spawn ALL Explore agents in ONE message, then wait for ALL with TaskOutput in ONE message:**
 ```
-Task tool parameters:
-- subagent_type: "Explore"
-- model: "haiku"
-- run_in_background: true (for parallel execution)
+// Spawn all in single message:
+t1 = Task(subagent_type="Explore", model="haiku", run_in_background=true, prompt="...")
+t2 = Task(subagent_type="Explore", model="haiku", run_in_background=true, prompt="...")
+
+// Wait all in single message:
+TaskOutput(task_id=t1)
+TaskOutput(task_id=t2)
 ```
 
 Scale agent count based on codebase size:

package/src/commands/df/spec.md

@@ -6,7 +6,7 @@ You coordinate agents and ask questions. You never search code directly.
 
 **NEVER:** Read source files, use Glob/Grep directly, run git
 
-**ONLY:** Spawn agents, poll results, ask user questions, write spec file
+**ONLY:** Spawn agents, use TaskOutput to get results, ask user questions, write spec file
 
 ---
 
@@ -31,12 +31,15 @@ Transform conversation context into a structured specification file.
 
 ### 1. GATHER CODEBASE CONTEXT
 
-**Use Task tool to spawn Explore agents in parallel:**
+**Spawn ALL Explore agents in ONE message, then wait for ALL with TaskOutput in ONE message:**
 ```
-Task tool parameters:
-- subagent_type: "Explore"
-- model: "haiku"
-- run_in_background: true
+// Spawn all in single message:
+t1 = Task(subagent_type="Explore", model="haiku", run_in_background=true, prompt="...")
+t2 = Task(subagent_type="Explore", model="haiku", run_in_background=true, prompt="...")
+
+// Wait all in single message:
+TaskOutput(task_id=t1)
+TaskOutput(task_id=t2)
 ```
 
 Find:

package/src/commands/df/verify.md

@@ -91,12 +91,15 @@ Default: L1-L3 (L4 optional, can be slow)
 
 ## Agent Usage
 
-**Use Task tool to spawn Explore agents:**
+**Spawn ALL Explore agents in ONE message, then wait for ALL with TaskOutput in ONE message:**
 ```
-Task tool parameters:
-- subagent_type: "Explore"
-- model: "haiku"
-- run_in_background: true (for parallel)
+// Spawn all in single message:
+t1 = Task(subagent_type="Explore", model="haiku", run_in_background=true, prompt="...")
+t2 = Task(subagent_type="Explore", model="haiku", run_in_background=true, prompt="...")
+
+// Wait all in single message:
+TaskOutput(task_id=t1)
+TaskOutput(task_id=t2)
 ```
 
 Scale: 1-2 agents per spec, cap 10.
@@ -157,4 +160,6 @@ rm .deepflow/checkpoint.json
 ✓ Merged df/doing-upload/20260202-1430 to main
 ✓ Cleaned up worktree and branch
 ✓ Spec complete: doing-upload → done-upload
+
+Workflow complete! Ready for next feature: /df:spec <name>
 ```