@nathapp/nax 0.28.0 → 0.29.0

package/docs/20260216-v02-review.md

@@ -1,685 +0,0 @@
-# v0.2 Code Review Report
-
-**Date:** 2026-02-16
-**Reviewer:** Claude Code
-**Scope:** v0.2 changes (commits baf529d, 1885a49, 1593093, 948f0a7)
-**Files Reviewed:** 8 core files
-**Test Status:** ✅ 132 tests passing
-**TypeScript:** ✅ No type errors
-
----
-
-## Executive Summary
-
-v0.2 introduces four major features: queue commands (PAUSE/ABORT/SKIP), story batching, context builder integration, and explicit 3-tier escalation. The implementation is **solid** with comprehensive test coverage and correct patterns. However, there are **6 findings** ranging from CRITICAL to MINOR that should be addressed.
-
-**Overall Grade: B+**
-
-### Key Strengths
-- ✅ Comprehensive test coverage (132 tests, 12 test files)
-- ✅ Type-safe implementation, no TypeScript errors
-- ✅ Clean separation of concerns (queue, context, execution)
-- ✅ Immutable patterns followed throughout
-- ✅ Good error handling and validation
-
-### Key Concerns
-- ⚠️ Queue command edge case: partial batch failure handling
-- ⚠️ Escalation chain edge case: batch execution with escalation
-- ⚠️ Context builder missing error boundary for malformed PRD data
-- ℹ️ Minor DRY violations in runner.ts
-
----
-
-## Findings by Category
-
-### 1️⃣ Queue Commands (PAUSE/ABORT/SKIP)
-
-**Grade: A-**
-
-**Location:** `src/queue/manager.ts`, `src/queue/types.ts`
-
-#### Strengths
-- ✅ Clean type-safe design with discriminated union for SKIP command
-- ✅ Case-insensitive parsing for user-friendliness
-- ✅ Clear separation of commands vs guidance text
-- ✅ Comprehensive test coverage (28 tests in `test/queue.test.ts`)
-- ✅ Proper whitespace handling and edge cases covered
-
-#### Issues Found
-
-**CRITICAL: Missing Edge Case — Batch SKIP Command** (Grade: C)
-
-**Issue:** When user issues `SKIP US-002` while executing a batch `[US-001, US-002, US-003]`, the current implementation doesn't gracefully handle mid-batch skips.
-
-**Location:** `src/execution/runner.ts:551-598`
-
-```typescript
-// Current code processes queue commands AFTER batch completion
-// If batch fails, only first story is marked for retry/escalation
-// Other stories in batch are left in limbo
-for (const cmd of queueCommands) {
-  // ...
-  } else if (typeof cmd === "object" && cmd.type === "SKIP") {
-    console.log(chalk.yellow(`   ⏭️  Skipping story ${cmd.storyId} by user request`));
-    markStorySkipped(prd, cmd.storyId);
-    await savePRD(prd, prdPath);
-  }
-}
-```
-
-**Problem:**
-1. Queue commands are only checked AFTER successful batch completion (line 551)
-2. If batch is in-progress, SKIP commands won't be processed until after completion
-3. User expects immediate effect when writing `SKIP US-002` to `.queue.txt`
-
-**Recommendation:**
-- Check queue file BEFORE starting batch execution
-- Filter out skipped stories from batch candidates
-- Re-validate batch after filtering
-
-**Impact:** Medium — User confusion, inefficient execution
-
----
-
-**MINOR: markSkipped Implementation** (Grade: B)
-
-**Issue:** `markSkipped` sets status to "failed" with error message "Skipped by user command"
-
-**Location:** `src/queue/manager.ts:182-192`
-
-```typescript
-markSkipped(storyId: string): void {
-  const item = this.items.find((i) => i.storyId === storyId);
-  if (!item) {
-    throw new Error(`Story ${storyId} not found in queue`);
-  }
-
-  // Mark as failed with skip reason (skipped is a type of failure)
-  item.status = "failed";
-  item.error = "Skipped by user command";
-  item.completedAt = new Date();
-}
-```
-
-**Problem:**
-- Comment says "skipped is a type of failure" but `QueueItemStatus` type only has `"pending" | "in-progress" | "completed" | "failed"`
-- Semantically, "skipped" is not the same as "failed" — causes confusion in stats/reporting
-- `markStorySkipped()` in PRD module correctly uses `status: "skipped"` (separate status)
-
-**Mismatch:** Queue module lacks `"skipped"` status that PRD module has
-
-**Recommendation:**
-- Add `"skipped"` to `QueueItemStatus` type
-- Update `markSkipped` to use correct status
-- Update `getStats()` to track skipped count separately
-
-**Impact:** Low — Stats are misleading but functional
-
----
-
-### 2️⃣ Story Batching
-
-**Grade: A**
-
-**Location:** `src/execution/runner.ts:131-186`, `test/runner.test.ts`
-
-#### Strengths
-- ✅ Clean, pure function `groupStoriesIntoBatches()` — easy to test
-- ✅ Correct max batch size enforcement (4 stories)
-- ✅ Proper handling of mixed complexity (stops at first non-simple)
-- ✅ Excellent test coverage (15 tests covering edge cases)
-- ✅ Clear batch vs individual execution logging
-
-#### Issues Found
-
-**MEDIUM: Batch Failure Escalation Logic** (Grade: B-)
-
-**Issue:** When batch execution fails, only the first story gets escalation treatment
-
-**Location:** `src/execution/runner.ts:599-651`
-
-```typescript
-} else {
-  // Handle failure — either escalate or mark failed
-  // For batched execution, only mark the first story for escalation/failure
-  // Others will be retried individually on next iteration
-  const failedStory = storiesToExecute[0];
-  const nextTier = escalateTier(routing.modelTier);
-
-  if (isBatchExecution) {
-    console.log(chalk.yellow(`   ⚠️  Batch execution failed — will retry stories individually`));
-  }
-  // ... escalation logic for failedStory only
-}
-```
-
-**Problem:**
-1. If batch `[US-001, US-002, US-003, US-004]` fails on `fast` tier
-2. Only `US-001` gets escalated to `balanced` tier
-3. Stories 2-4 remain at `fast` tier, will be retried at same tier
-4. Comment says "Others will be retried individually" but doesn't update their routing
-
-**Expected Behavior:** Debatable design choice, but should be explicit:
-- **Option A:** Escalate entire batch together (conservative)
-- **Option B:** Retry individually at same tier first, escalate only if individual retry fails (current, but undocumented)
-
-**Current Implementation:** Option B is implemented but not clearly documented
-
-**Recommendation:**
-- Add config option: `batch.escalateEntireBatchOnFailure: boolean`
-- Document chosen strategy in code comments
-- Add test case for batch failure escalation
-
-**Impact:** Medium — Unclear behavior, potential cost inefficiency
-
----
-
-**MINOR: Batch Routing Re-computation** (Grade: B)
-
-**Issue:** When batching stories, routing is re-computed for each candidate instead of using pre-computed routing from PRD
-
-**Location:** `src/execution/runner.ts:379-403`
-
-```typescript
-for (let i = currentIndex + 1; i < readyStories.length && batchCandidates.length < 4; i++) {
-  const candidate = readyStories[i];
-  const candidateRouting = routeTask(  // ← Re-computes routing
-    candidate.title,
-    candidate.description,
-    candidate.acceptanceCriteria,
-    candidate.tags,
-    config,
-  );
-
-  if (
-    candidateRouting.complexity === "simple" &&
-    candidateRouting.testStrategy === "test-after"
-  ) {
-    batchCandidates.push(candidate);
-  } else {
-    break;
-  }
-}
-```
-
-**Problem:**
-- Routing was already computed during `analyze` phase and stored in `story.routing`
-- Re-computing is wasteful and could produce different results if routing logic changes
-- Should trust the pre-computed routing
-
-**Recommendation:**
-```typescript
-// Use pre-computed routing
-if (
-  candidate.routing?.complexity === "simple" &&
-  candidate.routing?.testStrategy === "test-after"
-) {
-  batchCandidates.push(candidate);
-}
-```
-
-**Impact:** Low — Minor inefficiency, potential inconsistency
-
----
-
-### 3️⃣ Context Builder Integration
-
-**Grade: A**
-
-**Location:** `src/context/builder.ts`, `src/context/types.ts`, `test/context.test.ts`
-
-#### Strengths
-- ✅ Excellent design: clean interfaces, composable functions
-- ✅ Priority-based context selection with token budgeting
-- ✅ Proper truncation when budget exceeded
-- ✅ Comprehensive test coverage (45+ tests)
-- ✅ Token estimation conservative (1 token ≈ 3 chars is reasonable)
-- ✅ Immutable patterns (no array mutations)
-
-#### Issues Found
-
-**MINOR: Missing Error Boundary for Malformed Data** (Grade: B+)
-
-**Issue:** Context builder assumes well-formed PRD structure, no defensive checks
-
-**Location:** `src/context/builder.ts:125-188`
-
-```typescript
-export async function buildContext(
-  storyContext: StoryContext,
-  budget: ContextBudget,
-): Promise<BuiltContext> {
-  const { prd, currentStoryId } = storyContext;
-  const elements: ContextElement[] = [];
-
-  // Find current story
-  const currentStory = prd.userStories.find((s) => s.id === currentStoryId);
-  if (!currentStory) {
-    throw new Error(`Story ${currentStoryId} not found in PRD`);
-  }
-
-  // No validation of story structure
-  // Could crash if acceptanceCriteria is undefined/null
-  elements.push(createStoryContext(currentStory, 80));
-
-  // No validation of dependencies
-  if (currentStory.dependencies && currentStory.dependencies.length > 0) {
-    for (const depId of currentStory.dependencies) {
-      const depStory = prd.userStories.find((s) => s.id === depId);
-      if (depStory) {  // ← Silently skips missing dependencies
-        elements.push(createDependencyContext(depStory, 50));
-      }
-    }
-  }
-```
-
-**Problem:**
-1. If `acceptanceCriteria` is undefined/null, `formatStoryAsText()` will crash
-2. Missing dependencies are silently skipped (no warning logged)
-3. No validation that `priorErrors` is an array
-
-**Recommendation:**
-- Add defensive checks in `formatStoryAsText()`
-- Log warning when dependency story not found
-- Validate PRD structure or use Zod schema
-
-**Impact:** Low — PRD is generated by trusted code, but defensive programming is better
-
----
-
-**MINOR: Token Estimation Accuracy** (Grade: B)
-
-**Issue:** Token estimation is rough approximation (1 token ≈ 3 chars)
-
-**Location:** `src/context/builder.ts:14-16`
-
-```typescript
-export function estimateTokens(text: string): number {
-  return Math.ceil(text.length / 3);
-}
-```
-
-**Problem:**
-- Real tokenization varies: "hello" = 1 token, "anthropic" = 2 tokens
-- Character-based estimation can be off by 20-40% for code/technical text
-- Budget may be exceeded or underutilized
-
-**Reality Check:**
-- 1 token ≈ 4 chars for English prose (GPT tokenizer)
-- 1 token ≈ 2-3 chars for code (more special chars)
-- Current formula (divide by 3) is **reasonable middle ground**
-
-**Recommendation:**
-- Add comment documenting the approximation
-- Consider using `@anthropic-ai/tokenizer` for accurate counts (adds dependency)
-- Current approach is acceptable for MVP
-
-**Impact:** Very Low — Estimation is conservative, won't break
-
----
-
-### 4️⃣ Explicit 3-Tier Escalation Chain
-
-**Grade: A**
-
-**Location:** `src/execution/runner.ts:220-232`
-
-#### Strengths
-- ✅ Clean, explicit switch statement (no magic)
-- ✅ Returns `null` when max tier reached (type-safe)
-- ✅ Correct escalation path: fast → balanced → powerful → null
-- ✅ Well-tested in integration tests
-
-#### Issues Found
-
-**MINOR: Escalation Chain Documentation** (Grade: A-)
-
-**Issue:** Escalation chain is hardcoded, not configurable
-
-**Location:** `src/execution/runner.ts:220-232`
-
-```typescript
-export function escalateTier(current: ModelTier): ModelTier | null {
-  // Explicit escalation chain
-  switch (current) {
-    case "fast":
-      return "balanced";
-    case "balanced":
-      return "powerful";
-    case "powerful":
-      return null; // Max tier reached
-    default:
-      return null;
-  }
-}
-```
-
-**Problem:**
-- Escalation path is fixed: `fast → balanced → powerful`
-- Config has `escalation.maxAttempts` but no `escalation.tierOrder`
-- If user wants custom escalation (e.g., `fast → powerful`, skip balanced), no way to configure
-
-**Recommendation:**
-- Add config option: `escalation.tierOrder: ModelTier[]`
-- Default: `["fast", "balanced", "powerful"]`
-- Use config array to determine next tier
-
-**Impact:** Very Low — Current approach works for 99% of cases
-
----
-
-### 5️⃣ Configuration & Validation
-
-**Grade: A**
-
-**Location:** `src/config/schema.ts`, `src/config/validate.ts`
-
-#### Strengths
-- ✅ Type-safe config schema with good defaults
-- ✅ Comprehensive validation (version, limits, tier mappings)
-- ✅ Clear error messages
-- ✅ Supports both shorthand (string) and full ModelDef
-
-#### Issues Found
-
-**MINOR: Validation Error Messages** (Grade: B+)
-
-**Issue:** Validation errors in `validateConfig()` throw immediately, only one error shown at a time
-
-**Location:** `src/config/validate.ts:26-94`
-
-```typescript
-export function validateConfig(config: NgentConfig): ValidationResult {
-  const errors: string[] = [];
-
-  // Collects all errors
-  if (config.version !== 1) {
-    errors.push(`Invalid version: expected 1, got ${config.version}`);
-  }
-  // ... more checks
-
-  return {
-    valid: errors.length === 0,
-    errors,
-  };
-}
-```
-
-vs `src/config/schema.ts:166-205`
-
-```typescript
-export function validateConfig(config: NgentConfig): void {
-  // Throws on first error
-  if (config.version !== 1) {
-    throw new Error(`Invalid config version: ${config.version} (expected 1)`);
-  }
-  // ...
-}
-```
-
-**Problem:**
-- Two `validateConfig` functions with different signatures
-- `schema.ts` version throws immediately (stops at first error)
-- `validate.ts` version collects all errors (better UX)
-- Naming collision is confusing
-
-**Recommendation:**
-- Rename `schema.ts` version to `assertValidConfig()` or remove it
-- Use `validate.ts` version consistently (returns all errors)
-
-**Impact:** Low — Both work, but DRY violation
-
----
-
-### 6️⃣ CLI Integration
-
-**Grade: A**
-
-**Location:** `bin/ngent.ts`
-
-#### Strengths
-- ✅ Clean commander.js integration
-- ✅ Proper option parsing (--no-context, --no-batch)
-- ✅ Good help text and error messages
-
-#### Issues Found
-
-**MINOR: Default Flag Behavior** (Grade: B+)
-
-**Issue:** CLI uses `!options.noContext` instead of explicit defaults
-
-**Location:** `bin/ngent.ts:126-127`
-
-```typescript
-useContext: !options.noContext,
-useBatch: !options.noBatch,
-```
-
-**Problem:**
-- Double negatives are hard to reason about
-- If commander doesn't set `noContext` flag, undefined → falsy → true
-- Works correctly but confusing
-
-**Recommendation:**
-```typescript
-.option("--no-context", "Disable context builder", false)
-// Then access as options.context directly (commander handles --no- prefix)
-useContext: options.context ?? true,
-```
-
-**Impact:** Very Low — Works correctly, just style preference
-
----
-
-## Edge Cases Analysis
-
-### Queue Commands Edge Cases
-
-| Scenario | Behavior | Grade |
-|----------|----------|-------|
-| Empty `.queue.txt` | ✅ Returns empty commands array | A |
-| Commands before `--- PENDING ---` | ✅ Parsed as commands | A |
-| SKIP without story ID | ✅ Treated as guidance | A |
-| Multiple SKIP for same story | ⚠️ Processed multiple times (harmless) | B |
-| PAUSE mid-batch | ⚠️ Only checked after batch completion | C |
-| Invalid story ID in SKIP | ⚠️ No validation, will throw later | B- |
-
-### Batching Edge Cases
-
-| Scenario | Behavior | Grade |
-|----------|----------|-------|
-| All stories simple | ✅ Batched correctly | A |
-| Mixed complexity | ✅ Stops at first non-simple | A |
-| Single simple story | ✅ Executed as non-batch | A |
-| Batch size > 4 | ✅ Split into multiple batches | A |
-| Batch failure | ⚠️ Only first story escalated | B- |
-| Dependencies in batch | ✅ Filtered out by `getAllReadyStories()` | A |
-
-### Context Builder Edge Cases
-
-| Scenario | Behavior | Grade |
-|----------|----------|-------|
-| Story not found | ✅ Throws error | A |
-| Dependency not found | ⚠️ Silently skipped | B |
-| Budget exceeded | ✅ Truncates with [TRUNCATED] marker | A |
-| Empty PRD | ✅ Returns progress only | A |
-| Malformed story | ⚠️ May crash in `formatStoryAsText()` | B- |
-
-### Escalation Edge Cases
-
-| Scenario | Behavior | Grade |
-|----------|----------|-------|
-| Already at `powerful` | ✅ Returns null, marks failed | A |
-| Batch escalation | ⚠️ Only first story escalated | B- |
-| Max attempts reached | ✅ Marks failed, stops execution | A |
-| Invalid tier | ✅ Returns null (default case) | A |
-
----
-
-## DRY Violations
-
-### 1. Duplicate `validateConfig()` functions
-
-**Locations:**
-- `src/config/schema.ts:166-205` (throws immediately)
-- `src/config/validate.ts:26-94` (collects all errors)
-
-**Recommendation:** Remove or rename one
-
----
-
-### 2. Context building logic
-
-**Locations:**
-- `src/execution/runner.ts:189-217` (buildStoryContext wrapper)
-- `src/execution/runner.ts:247-263` (maybeGetContext wrapper)
-
-**DRY Opportunity:** Extract to shared function
-
----
-
-## Security Analysis
-
-**Grade: A**
-
-- ✅ No SQL injection risks (no raw SQL)
-- ✅ No command injection (queue file parsing is safe)
-- ✅ No XSS risks (CLI tool, no web output)
-- ✅ No hardcoded secrets
-- ✅ No unsafe file operations (uses Bun.file API safely)
-
-**Queue File Parsing Security:**
-- ✅ Case-insensitive matching is safe
-- ✅ No eval() or Function() constructor
-- ✅ Trim() prevents injection of whitespace-based attacks
-
----
-
-## Performance Analysis
-
-**Grade: A-**
-
-**Token Estimation:** O(n) where n = text length — acceptable
-
-**Story Batching:** O(n) where n = ready stories — acceptable
-
-**Context Building:** O(m) where m = elements — acceptable
-
-**Queue Parsing:** O(n) where n = lines — acceptable
-
-**Routing Re-computation:** ⚠️ Unnecessary work in batch candidate selection (see Finding 2.2)
-
----
-
-## Test Coverage Analysis
-
-**Grade: A**
-
-**Total Tests:** 132 passing
-**Test Files:** 12
-
-**Coverage by Module:**
-- Queue: 28 tests (`test/queue.test.ts`)
-- Context: 45+ tests (`test/context.test.ts`, `test/context-integration.test.ts`)
-- Runner/Batching: 15 tests (`test/runner.test.ts`)
-- Execution: 5+ tests (`test/execution.test.ts`)
-- Routing: Covered in execution tests
-- Escalation: Covered in execution tests
-
-**Missing Test Cases:**
-1. ⚠️ Batch failure escalation behavior
-2. ⚠️ Queue commands during batch execution
-3. ⚠️ Invalid SKIP story ID handling
-4. ⚠️ Context builder with malformed story data
-
----
-
-## Recommendations by Priority
-
-### CRITICAL (Fix Before Merging)
-
-None — all critical issues are design choices, not bugs
-
-### HIGH (Fix Soon)
-
-1. **Batch SKIP Command Handling** — Check queue file before batch execution
-2. **Batch Failure Escalation** — Document strategy and add config option
-3. **Add "skipped" status to QueueItemStatus** — Align with PRD module
-
-### MEDIUM (Next Sprint)
-
-4. **Use Pre-computed Routing** — Don't re-compute during batching
-5. **Context Builder Error Boundary** — Add defensive checks
-6. **Duplicate validateConfig()** — Remove or rename
-
-### LOW (Nice to Have)
-
-7. **Configurable Escalation Chain** — Add to config schema
-8. **CLI Flag Defaults** — Use explicit defaults instead of double negatives
-9. **Token Estimation Documentation** — Add comment explaining approximation
-
----
-
-## Overall Assessment
-
-**Final Grade: B+**
-
-**Strengths:**
-- Solid architecture with clean separation of concerns
-- Comprehensive test coverage (132 tests passing)
-- Type-safe implementation (no TypeScript errors)
-- Good error handling and validation
-- Immutable patterns followed throughout
-
-**Weaknesses:**
-- Queue command handling during batch execution (timing issue)
-- Batch failure escalation strategy not explicit
-- Minor DRY violations (duplicate validateConfig)
-- Missing edge case handling (malformed data, invalid story IDs)
-
-**Readiness:** **READY FOR MERGE** with follow-up tasks
-
-The v0.2 implementation is production-ready. The identified issues are primarily edge cases and design clarifications, not blocking bugs. I recommend:
-
-1. Merge v0.2 as-is
-2. Create GitHub issues for HIGH priority items
-3. Address in v0.2.1 or v0.3
-
-**Estimated Effort to Fix All Issues:** 4-6 hours
-
----
-
-## Appendix: Test Run Output
-
-```
-bun test v1.3.9 (cf6cdbbb)
-
-test/execution.test.ts: ✅
-test/context-integration.test.ts: ✅
-test/tdd-orchestrator.test.ts: ✅
-test/analyze.test.ts: ✅
-test/cost.test.ts: ✅
-test/isolation.test.ts: ✅
-test/progress.test.ts: ✅
-test/context.test.ts: ✅
-test/queue.test.ts: ✅
-test/routing.test.ts: ✅
-test/config.test.ts: ✅
-test/runner.test.ts: ✅
-
-132 pass
-0 fail
-352 expect() calls
-Ran 132 tests across 12 files. [122.00ms]
-```
-
-**TypeScript Check:**
-```
-$ bun x tsc --noEmit
-✅ No errors
-```
-
----
-
-**Review Completed:** 2026-02-16
-**Next Review:** v0.3 (agent execution implementation)