@nathapp/nax 0.18.1

package/docs/bugs/BUG-21-escalation-null-attempts.md

@@ -0,0 +1,48 @@
+# BUG-21: Escalation fails when PRD story has null/missing attempts field
+
+**Severity:** High
+**Component:** src/execution/runner.ts (escalation logic)
+**Found:** 2026-02-23
+**Status:** Open
+
+## Summary
+
+When a story attempts field is null or missing in the PRD, escalation breaks. The agent returns finalAction: escalate but the story is immediately marked as failed instead of being re-queued at a higher model tier.
+
+## Reproduction
+
+1. Create a PRD with stories that do NOT include an attempts field (or set it to null)
+2. Run nax run -f feature --headless
+3. When a story fails and returns finalAction: escalate, observe:
+   - Log shows Story failed - max attempts reached immediately
+   - No actual escalation to next tier occurs
+   - PRD shows attempts: null after the run
+
+## Root Cause
+
+The runner increments story.attempts via:
+    attempts: s.attempts + 1
+But if s.attempts is null or undefined, this produces NaN, breaking subsequent comparisons.
+
+Pre-iteration tier check (line ~338):
+    if (tierCfg && story.attempts >= tierCfg.attempts)
+null >= 5 evaluates to false in JS, so this check is silently skipped.
+
+Post-execution canEscalate check (line ~704):
+    const canEscalate = storiesToEscalate.every((s) => s.attempts < maxAttempts);
+null < 10 is true in JS, so canEscalate is true. But then attempts: s.attempts + 1 yields null + 1 = 1.
+The story still gets marked failed, suggesting the PRD save/reload cycle loses the updated value or the iteration loop exits before re-processing.
+
+## Suggested Fix
+
+1. Initialize attempts to 0 when loading PRD stories with null/undefined attempts
+2. Defensive coercion: attempts: (s.attempts ?? 0) + 1
+3. Add PRD validation on load to ensure all stories have attempts: number (default 0)
+
+## Observed Log
+
+[21:45:33] [execution] Agent session failed { rateLimited: false, storyId: US-002 }
+[21:45:33] [agent.complete] { storyId: US-002, success: false, finalAction: escalate, estimatedCost: 0.75 }
+[21:45:33] [execution] Story failed - max attempts reached { storyId: US-002 }
+
+No escalation log line between finalAction: escalate and Story failed, confirming escalation path was skipped.

package/docs/bugs-from-dogfood-run-c.md

@@ -0,0 +1,243 @@
+# Bugs Found: Dogfood Run C (2026-02-19, plan→analyze→run pipeline)
+
+## BUG-16: maxIterations is global, not per-story (CODE)
+
+**Severity:** High — causes infinite loops on stuck stories
+
+**Evidence:** Config had `maxIterations: 5` but nax ran **20 iterations**.
+The main loop at `runner.ts:140` checks `iterations < config.execution.maxIterations`,
+but the config value was overridden. Investigation shows the dogfood config had `maxIterations: 5`
+but per the summary the run did 20 iterations.
+
+**Root Cause:** `maxIterations` is a **global** cap across ALL stories, not per-story.
+But the real issue is that the per-story attempt counter (`story.attempts`) doesn't cap the
+story — only the escalation logic reads it. If escalation doesn't trigger (see BUG-17),
+the story retries indefinitely until the global iteration limit.
+
+**Expected Behavior:** Each story should respect the tier budget:
+- Per-story max attempts = sum of `tierOrder` attempts (default: 5+3+2=10)
+- After exhausting all tiers, mark story as FAILED and move to next story
+- `maxIterations` should be an override safety cap, not the primary limit
+
+**Fix Location:** `src/execution/runner.ts` — add per-story attempt check before retrying
+
+---
+
+## BUG-17: ASSET_CHECK_FAILED doesn't trigger escalation (CODE)
+
+**Severity:** High — story loops at same tier forever
+
+**Evidence:** US-004 failed ASSET_CHECK 16 times, always at `balanced` tier.
+Never escalated to `powerful` despite `countsTowardEscalation: true`.
+
+**Root Cause:** The escalation logic lives in the `case "escalate"` handler
+(`runner.ts:367`), but ASSET_CHECK failures flow through `post-verify.ts`
+which only increments `story.attempts` and reverts to `pending`. It never
+returns an `"escalate"` action to the runner — it just reverts the story.
+
+The escalation check happens in runner.ts case "escalate" but the pipeline
+never returns "escalate" for verification failures. The verify stage returns
+"continue" (tests passed), then post-verify reverts on ASSET_CHECK but the
+result is already "continue".
+
+**Flow:**
+```
+1. Pipeline runs → verify stage → tests pass → "continue"
+2. completion stage → marks story as passed
+3. post-verify → ASSET_CHECK fails → reverts to pending, increments attempts
+4. Runner sees "continue" from pipeline, never hits "escalate" case
+5. Next iteration picks up story at SAME tier (no escalation)
+```
+
+**Expected Behavior:** When `story.attempts` exceeds the current tier's budget,
+the runner should check tier escalation BEFORE starting the next iteration,
+not only in the `"escalate"` case handler.
+
+**Fix Location:** 
+- `src/execution/runner.ts` — add tier check at start of iteration (before agent spawn)
+- OR `src/execution/post-verify.ts` — escalate the story's `routing.modelTier` when attempts exceed tier budget
+
+---
+
+## BUG-18: ASSET_CHECK error not fed back to agent prompt (CODE)
+
+**Severity:** Medium — agent repeats same mistake endlessly
+
+**Evidence:** All 17 retries of US-004 show the exact same warnings:
+```
+⚠️  Relevant file not found: src/finder.ts (story: US-004)
+⚠️  Relevant file not found: test/finder.test.ts (story: US-004)
+```
+The agent kept writing to `src/discovery.ts` instead of `src/finder.ts`.
+The ASSET_CHECK error is stored in `story.priorErrors` (post-verify.ts line 102),
+but the "Prior Errors" section in the prompt only showed the initial ASSET_CHECK
+message, not a clear instruction like "You MUST create src/finder.ts".
+
+**Expected Behavior:** The ASSET_CHECK error should be prominent in the prompt,
+ideally as a mandatory instruction: "REQUIRED: Create these files: src/finder.ts, test/finder.test.ts"
+
+**Fix Location:** `src/pipeline/stages/prompt.ts` — format ASSET_CHECK errors as mandatory file creation instructions
+
+---
+
+## BUG-19: Simple complexity routes to balanced tier, not fast (CODE)
+
+**Severity:** Medium — wastes budget on wrong tier
+
+**Evidence:** US-001 (simple) and US-004 (simple) both show:
+```
+Complexity: simple | Model: balanced | TDD: test-after
+Routing: test-after: simple task (medium)
+```
+Should start at `fast` (Haiku) per `complexityRouting.simple: "fast"`.
+
+**Root Cause:** The routing display shows `(medium)` suggesting the actual
+routed tier is `medium`/`balanced`, not the expected `fast`. Likely the
+routing stage is using test strategy routing instead of complexity routing,
+or there's a fallback that overrides the tier.
+
+**Fix Location:** `src/pipeline/stages/routing.ts` or equivalent — check why
+simple stories get routed to balanced instead of fast.
+
+---
+
+## Test Coverage Gaps
+
+### Existing (35 tests in runner.test.ts)
+- ✅ Batch prompt building (3 tests)
+- ✅ Batch grouping (8 tests)
+- ✅ Batch precompute (5 tests)
+- ✅ Batch failure escalation (3 tests)
+- ✅ Queue commands (6 tests)
+- ✅ Escalation chain (7 tests)
+- ✅ Hook security/loading/env (19 tests in hooks.test.ts)
+
+### Missing (needed to prevent BUG-16–19)
+- ❌ **Per-story iteration capping** — story should fail after tier budget exhausted
+- ❌ **ASSET_CHECK → escalation trigger** — post-verify failure should escalate tier
+- ❌ **ASSET_CHECK error in prompt** — verify mandatory files appear in next prompt
+- ❌ **Complexity → tier routing accuracy** — simple=fast, medium=balanced, complex=powerful
+- ❌ **Post-verify revert + re-queue** — story reverted correctly after ASSET_CHECK
+- ❌ **End-to-end: story passes on retry after escalation** — integration test
+- ❌ **End-to-end: story fails permanently after all tiers exhausted** — integration test
+- ❌ **Verification unit tests** — no `test/verification.test.ts` exists
+- ❌ **Post-verify unit tests** — no `test/post-verify.test.ts` exists
+
+---
+
+*Filed 2026-02-19 from dogfood run C (plan→analyze→run pipeline test)*
+
+## BUG-21: No model name validation before run (CONFIG)
+
+**Severity:** Medium — causes silent failures, wasted retries
+
+**Evidence:** Dogfood Run D — `claude-opus-4` not recognized by Claude Code CLI.
+Agent exited with error message but exit code 0 on some attempts, exit code 1 on others.
+TDD test-writer session ran 3 times producing nothing. Wasted ~$0.13 and 3 minutes.
+
+**Root Cause:** No validation of model names in config against the agent's accepted models.
+`claude-opus-4` is not a valid Claude Code model name (`claude-opus-4-5` or `opus` alias is).
+
+**Expected Behavior:** Before starting a run, validate that all configured model names
+are accepted by the target agent. Fail fast with a clear error message.
+
+**Future Design:** When supporting multiple code agents (Claude, Cursor, Copilot, etc.),
+each agent adapter should expose a `validateModel(name: string)` method or provide
+a model registry. Worst case: maintain a `models.json` per provider.
+
+**Workaround:** Use CLI aliases (`haiku`, `sonnet`, `opus`) which always resolve to latest.
+
+**Fix Location:** `src/config/validate.ts` — add model validation step.
+Agent adapter interface: add optional `getSupportedModels()` or `validateModel()`.
+
+**Priority:** Low — workaround available (use aliases)
+
+---
+
+## BUG-21: Claude Code child processes orphaned after TDD session failure
+
+**Found:** Run D, US-007 TDD test-writer failure (2026-02-19 20:11)
+**Severity:** Medium (resource leak, CPU waste)
+**Component:** `src/tdd/orchestrator.ts` / `src/agents/claude-adapter.ts`
+
+### Symptoms
+- `bun test` (PID 76312) running at 99.9% CPU for 2+ hours after Run D completed
+- Process orphaned (PPID=1), original parent (PGID leader 76309) dead
+- Sibling `tail -5` (PID 76313) also orphaned, plus a zombie child (PID 76555)
+- Pipeline: `bun test 2>&1 | tail -5` — spawned by Claude Code internally during TDD test-writer session
+
+### Root Cause
+When Claude Code exits with code 1 (TDD session failure), it does NOT clean up shell commands it spawned internally. nax kills the Claude Code process via the agent adapter, but Claude Code's child processes (`bun test | tail -5`) are in a different process group (PGID 76309 vs Claude Code's own PID).
+
+nax's `executeWithTimeout()` in `verification.ts` properly kills process groups for commands IT spawns, but TDD session child processes are spawned by Claude Code, not by nax.
+
+### Process Tree at Failure
+```
+launchd (1)
+├── bun test (76312) ← orphaned, 99.9% CPU, PGID 76309
+├── tail -5 (76313) ← orphaned, sleeping, PGID 76309  
+└── <defunct> (76555) ← zombie child of 76312
+```
+Original PGID leader (76309) is dead — likely the shell Claude Code spawned.
+
+### Fix Options
+1. **nax-side (recommended):** After agent adapter returns failure, run `pkill -P <agent_pid>` recursively or `kill -- -<pgid>` to clean up the entire process tree. Add a `cleanupProcessTree(pid)` utility.
+2. **nax-side (belt+suspenders):** Track all child PIDs before/after TDD session via `pgrep -P`, kill any new orphans.
+3. **Upstream (Claude Code):** File issue — Claude Code should clean up child processes on abnormal exit.
+
+### Affected Code
+- `src/tdd/orchestrator.ts` — `runTddSession()` calls agent adapter but doesn't clean up process tree on failure
+- `src/agents/claude-adapter.ts` — `runSession()` kills Claude Code process but not its children
+
+### Workaround
+Manually kill orphaned processes: `kill -9 -76309` (kill entire PGID)
+
+---
+
+## BUG-22: TDD orchestrator treats verifier fix-and-commit as failure
+
+**Found:** Run D2, US-009 (2026-02-19 22:23)
+**Severity:** Medium (false positive pause, wastes human review time)
+**Component:** `src/tdd/orchestrator.ts`
+
+### Symptoms
+- US-009 verifier session fixed flaky watcher tests (sleep timing) and added README.md
+- All 355 tests pass, 98.7% coverage, clean commit `9f9b048`
+- nax paused with "Verifier session identified issues" requiring human review
+- No actual issues — the work is complete and correct
+
+### Root Cause
+`runThreeSessionTdd()` line 387:
+```typescript
+const allSuccessful = sessions.every((s) => s.success);
+```
+
+`session.success` is derived from the Claude Code agent's **exit code**, not the final test state. The verifier likely:
+1. Ran `bun test` → some tests failed (flaky watcher timing)
+2. Fixed the tests (increased sleep timers)
+3. Ran `bun test` again → 355 pass
+4. Committed the fix
+5. But Claude Code exited with code 1 (possibly from the initial failed test run, or from an internal error during the long session)
+
+The orchestrator checks `sessions.every(s => s.success)` which uses exit code, not actual test outcomes. A verifier that **finds and fixes issues is doing its job** — that's a success, not a failure.
+
+### Fix Options
+1. **Post-TDD verification (recommended):** After all 3 sessions complete, run `bun test` independently. If tests pass → mark success regardless of individual session exit codes.
+2. **Verifier exit code tolerance:** If verifier session has commits AND tests pass (checked via isolation), treat as success even with non-zero exit.
+3. **Two-phase verifier:** Split verifier into "check" (run tests, report) and "fix" (apply fixes). Only flag if "fix" also fails.
+
+### Evidence
+```
+git log -1: "fix: verify and adjust Comprehensive integration tests and documentation"
+  - 355 tests pass, 0 fail
+  - 98.70% function coverage, 95.52% line coverage
+  - Files changed: README.md (+261), test/integration.test.ts (+7/-7)
+
+nax output: "⏸ Human review needed: Verifier session identified issues"
+```
+
+### Impact
+- False pause blocks automated pipeline completion
+- Human must manually verify and resume — defeats automation purpose
+- Cost: $4.95 spent on US-009, then paused on a success
+- Combined with misrouting (US-009 shouldn't have been TDD), this story cost ~$5 for ~$0.15 of actual work