@nathapp/nax 0.18.1

package/docs/v0.7-spec.md

@@ -0,0 +1,177 @@
+# nax v0.7 Specification
+**Date:** 2026-02-17
+**Status:** Draft
+
+## Theme: Test Context Injection
+
+v0.7 addresses test redundancy caused by isolated story sessions. Each agent session currently writes tests without knowing what prior stories already covered, leading to duplicate coverage.
+
+## Problem
+
+During dogfooding (bun-kv-store, 8 stories), we observed:
+- 6 tests for "name is required" (missing, empty, whitespace, not string, null, undefined)
+- Each story independently writes "comprehensive tests" for its area
+- Validation stories re-test what CRUD stories already covered
+- **Root cause:** Each session's prompt has zero visibility into existing test files
+
+## Solution: Test Context Injection
+
+Inject a summary of existing test files into each story's prompt so the agent knows what's already covered and avoids duplication.
+
+### How It Works
+
+Before the **prompt stage**, scan the project's test directory and generate a concise summary:
+
+```
+## Existing Test Coverage
+
+### test/store.test.ts (45 tests)
+- CRUD operations: create, read, update, delete
+- Error handling: missing key, invalid value
+- Batch operations: getMany, setMany
+
+### test/validation.test.ts (12 tests)  
+- Input validation: name required, type checking
+- Size limits: max key length, max value size
+```
+
+This summary is injected into the prompt alongside the story context, constitution, and other context elements.
+
+### Context Element
+
+```typescript
+interface TestCoverageContext {
+  type: 'test-coverage';
+  priority: 85;           // Below constitution (95), above file context (50)
+  content: string;        // Formatted test summary
+  tokens: number;
+  source: string;         // e.g., "test/*.test.ts"
+}
+```
+
+### Summary Generation
+
+```typescript
+interface TestSummaryOptions {
+  /** Test directory to scan (default: auto-detect from config or common patterns) */
+  testDir?: string;
+  /** Glob pattern for test files (default: "**/*.test.{ts,js,tsx,jsx}") */
+  testPattern?: string;
+  /** Max tokens for the summary (default: 500) */
+  maxTokens?: number;
+  /** Summary detail level */
+  detail: 'names-only' | 'names-and-counts' | 'describe-blocks';
+}
+```
+
+**Detail levels:**
+- `names-only` — Just file names and test count: `test/store.test.ts (45 tests)`
+- `names-and-counts` — File names + top-level describe blocks with counts
+- `describe-blocks` — File names + describe blocks + test names (most expensive but most useful)
+
+**Default:** `names-and-counts` (good balance of info vs tokens)
+
+### Scanning Approach
+
+1. Find test files matching pattern
+2. For each file, extract `describe()` and `test()`/`it()` block names via regex (no AST parsing needed)
+3. Format as markdown summary
+4. Truncate to token budget
+
+### Prompt Injection
+
+Add to the story prompt:
+
+```
+## Existing Test Coverage
+
+The following tests already exist. DO NOT duplicate this coverage.
+Focus only on testing NEW behavior introduced by this story.
+
+### test/store.test.ts (45 tests)
+- CRUD operations (12 tests): create, read, update, delete, upsert
+- Validation (8 tests): required fields, type checks, size limits
+- Error handling (6 tests): not found, duplicate key, connection error
+...
+
+## Your Story
+
+US-007: Add input sanitization
+...
+```
+
+### Config
+
+```json
+{
+  "context": {
+    "testCoverage": {
+      "enabled": true,
+      "detail": "names-and-counts",
+      "maxTokens": 500,
+      "testDir": "test",
+      "testPattern": "**/*.test.{ts,js}"
+    }
+  }
+}
+```
+
+## Acceptance Criteria
+
+- [ ] Test files scanned and summarized before each story prompt
+- [ ] Summary injected into prompt at priority 85
+- [ ] `describe()` and `test()` names extracted via regex
+- [ ] Summary respects maxTokens budget
+- [ ] Config allows enabling/disabling and adjusting detail level
+- [ ] Summary updates between stories (reflects tests added by prior stories)
+- [ ] No performance regression (scanning should be <100ms for typical projects)
+
+## Implementation Plan
+
+### Phase 1: Test Scanner
+**Files:** `src/context/test-scanner.ts`
+- Scan test directory for test files
+- Extract describe/test block names via regex
+- Format as markdown summary
+- Respect token budget
+
+**Commit:** `feat(context): add test file scanner for coverage summary`
+
+### Phase 2: Context Integration
+**Files:** `src/context/builder.ts`, `src/pipeline/stages/context.ts`, `src/config/schema.ts`
+- Add TestCoverageContext element type
+- Wire scanner into context stage (runs before prompt assembly)
+- Add config schema for testCoverage settings
+- Summary refreshes between stories
+
+**Commit:** `feat(context): inject test coverage summary into story prompts`
+
+### Phase 3: Prompt Guidance
+**Files:** `src/execution/prompts.ts`
+- Add "DO NOT duplicate" instruction in prompt template
+- Reference existing coverage summary
+- Reinforce constitution test guidance
+
+**Commit:** `feat(prompts): add test dedup guidance referencing coverage summary`
+
+## Test Strategy
+- Mode: test-after
+- Unit tests: scanner regex extraction, summary formatting, token truncation
+- Integration: context builder includes test coverage element
+- Run `bun test && bun run typecheck` after each phase
+
+## Estimated Effort
+~300-400 LOC across 3 phases.
+
+## Measurement
+
+Compare v0.5.0 (no dedup) vs v0.7.0 (context injection) on the same dogfood project:
+
+| Metric | v0.5.0 | v0.7.0 |
+|:---|:---|:---|
+| Total tests generated | ? | ? |
+| Redundant tests | ? | ? |
+| Code quality grade | ? | ? |
+| Acceptance rate | ? | ? |
+| Total cost | ? | ? |
+| Total time | ? | ? |

package/docs/v0.8-llm-routing.md

@@ -0,0 +1,206 @@
+# v0.8 — LLM-Enhanced Routing
+
+> Priority: **HIGH** — keyword routing causes costly misroutes (e.g., US-008 simple barrel exports → powerful + TDD due to "public api" keyword match).
+
+## Problem
+
+Keyword-based routing is brittle and expensive:
+1. **False positives**: "public api" in title → three-session-tdd even for simple barrel exports ($1.25 wasted)
+2. **False negatives**: Complex integration work without magic keywords → test-after on fast tier
+3. **No semantic understanding**: Can't assess *actual* implementation effort from acceptance criteria
+4. **Keyword overlap**: Security keywords fire on simple "add auth header to request" stories
+
+Evidence from dogfood runs:
+- Run D2: US-008 ("Export public API and create barrel exports") — simple task, keyword matched "public api" → powerful + three-session-tdd. Cost: $1.25. Should have been: fast + test-after (~$0.10).
+
+## Design
+
+### Config
+
+```json
+{
+  "routing": {
+    "strategy": "llm",
+    "llm": {
+      "model": "fast",
+      "fallbackToKeywords": true,
+      "maxInputTokens": 2000,
+      "cacheDecisions": true
+    }
+  }
+}
+```
+
+- `model`: Tier used for the routing LLM call itself (default: `fast` — routing should be cheap)
+- `fallbackToKeywords`: If LLM call fails (timeout, parse error), fall back to keyword strategy (default: `true`)
+- `maxInputTokens`: Token budget for story context sent to LLM (default: `2000`)
+- `cacheDecisions`: Cache routing decisions per story ID within a run (default: `true`)
+
+### LLM Prompt
+
+The LLM receives a structured prompt with the story and must return a JSON decision:
+
+```
+You are a code task router. Given a user story, classify its complexity and select the appropriate execution strategy.
+
+## Story
+Title: {title}
+Description: {description}
+Acceptance Criteria:
+{acceptanceCriteria as numbered list}
+Tags: {tags}
+
+## Available Tiers
+- fast: Simple changes, typos, config updates, boilerplate. <30 min of coding.
+- balanced: Standard features, moderate logic, straightforward tests. 30-90 min.
+- powerful: Complex architecture, security-critical, multi-file refactors, novel algorithms. >90 min.
+
+## Available Test Strategies
+- test-after: Write implementation first, add tests after. For straightforward work.
+- three-session-tdd: Separate test-writer → implementer → verifier sessions. For complex/critical work where test design matters.
+
+## Rules
+- Default to the CHEAPEST option that will succeed.
+- three-session-tdd ONLY when: (a) security/auth logic, (b) complex algorithms, (c) public API contracts that consumers depend on.
+- Simple barrel exports, re-exports, or index files are ALWAYS test-after + fast, regardless of keywords.
+- A story touching many files doesn't automatically mean complex — copy-paste refactors are simple.
+
+Respond with ONLY this JSON (no markdown, no explanation):
+{"complexity":"simple|medium|complex|expert","modelTier":"fast|balanced|powerful","testStrategy":"test-after|three-session-tdd","reasoning":"<one line>"}
+```
+
+### Implementation
+
+Modify `src/routing/strategies/llm.ts`:
+
+```typescript
+export const llmStrategy: RoutingStrategy = {
+  name: "llm",
+
+  async route(story: UserStory, context: RoutingContext): Promise<RoutingDecision | null> {
+    const config = context.config;
+    const llmConfig = config.routing.llm;
+    if (!llmConfig) return null;
+
+    // Check cache
+    if (llmConfig.cacheDecisions && cachedDecisions.has(story.id)) {
+      return cachedDecisions.get(story.id)!;
+    }
+
+    try {
+      const prompt = buildRoutingPrompt(story, config);
+      const modelId = config.models[llmConfig.model ?? "fast"];
+
+      const result = await callLlm(modelId, prompt, {
+        maxTokens: 200,
+        timeout: 15_000, // 15s hard limit — routing shouldn't block
+      });
+
+      const decision = parseRoutingResponse(result, story, config);
+
+      if (llmConfig.cacheDecisions) {
+        cachedDecisions.set(story.id, decision);
+      }
+
+      return decision;
+    } catch (err) {
+      logger.warn(`LLM routing failed for ${story.id}: ${err.message}`);
+      return null; // Falls through to keyword strategy
+    }
+  },
+};
+```
+
+### LLM Call Mechanism
+
+nax already spawns Claude Code via `Bun.spawn`. For the routing LLM call, we need a **lightweight** approach:
+
+**Option A — Claude Code one-shot**: `claude -p "..." --model <model>` with 15s timeout.
+- Pro: Reuses existing infra, model aliases work.
+- Con: ~3-5s startup overhead per call. For 9 stories = 27-45s total.
+
+**Option B — Direct API call**: HTTP request to provider API (Anthropic/OpenAI/Google).
+- Pro: <1s per call, batch-friendly.
+- Con: Needs API key handling, provider-specific code.
+
+**Recommendation: Option A** for v0.8 (simplicity), with config option to batch all stories in one call:
+
+```
+// Single LLM call for all pending stories (batch mode)
+"Route these 9 stories:\n1. US-001: ...\n2. US-002: ...\n\nRespond with JSON array: [{id, complexity, modelTier, testStrategy, reasoning}]"
+```
+
+Batch mode cuts 9 calls → 1 call. ~5s total routing overhead.
+
+### Strategy Interface Change
+
+The current `RoutingStrategy.route()` is synchronous. LLM routing needs async:
+
+```typescript
+export interface RoutingStrategy {
+  readonly name: string;
+  route(story: UserStory, context: RoutingContext): RoutingDecision | null | Promise<RoutingDecision | null>;
+}
+```
+
+`StrategyChain.route()` becomes async (already called with `await` in `routeStory()`).
+
+### Error Handling
+
+| Failure | Behavior |
+|:---|:---|
+| LLM timeout (>15s) | Return null → keyword fallback |
+| JSON parse error | Return null → keyword fallback |
+| Invalid field values | Return null → keyword fallback |
+| LLM returns unknown complexity | Clamp to nearest valid value |
+| All failures logged | `logger.warn()` with story ID |
+
+### Logging
+
+```
+[routing] LLM classified US-008 as simple/fast/test-after: "Barrel export file, no logic to test"
+[routing] LLM routing failed for US-003: timeout after 15000ms, falling back to keyword
+```
+
+## Cost Analysis
+
+| Scenario | Keyword Cost | LLM Routing Cost | Savings |
+|:---|:---|:---|:---|
+| US-008 (barrel exports) | $1.25 (powerful+TDD) | $0.10 (fast+test-after) + $0.01 routing | **$1.14 saved** |
+| 9-story run (batch) | Variable | ~$0.02 routing overhead | Net positive if prevents 1+ misroute |
+
+LLM routing call: ~500 input tokens + 100 output tokens per story = ~$0.001 on Flash.
+Batch mode (9 stories): ~2000 input + 400 output = ~$0.003 total.
+
+**ROI: One prevented misroute pays for ~400 routing calls.**
+
+## Acceptance Criteria
+
+1. `config.routing.strategy = "llm"` activates LLM routing
+2. LLM strategy returns structured `RoutingDecision` with reasoning
+3. Falls back to keyword strategy on any LLM failure
+4. Batch mode: single LLM call routes all pending stories
+5. Routing decisions cached per story ID within a run
+6. Strategy chain async support (non-breaking — keyword still sync)
+7. Routing overhead < 10s for batch of 10 stories
+8. Logging: every LLM routing decision logged with reasoning
+
+## Files to Modify
+
+- `src/routing/strategies/llm.ts` — Main implementation
+- `src/routing/strategy.ts` — Make interface async-compatible
+- `src/routing/chain.ts` — `route()` becomes async
+- `src/config/schema.ts` — Add `LlmRoutingConfig` type
+- `src/config/defaults.ts` — Add LLM routing defaults
+- `test/routing/llm-strategy.test.ts` — Unit tests
+- `test/routing/chain.test.ts` — Update for async
+
+## Non-Goals (v0.8)
+
+- Direct API calls (Option B) — defer to v0.9+
+- Adaptive routing (learning from historical data) — existing stub, separate feature
+- Custom routing prompts — hardcoded prompt is fine for now
+
+---
+
+*Created 2026-02-19*

package/docs/v0.8-structured-logging.md

@@ -0,0 +1,132 @@
+# Feature: Structured Logging for nax v0.8
+
+## Problem
+
+nax currently uses raw `console.log` with chalk formatting throughout the codebase. Developers running `nax run` in headless mode have no way to:
+- Control verbosity (only see errors vs full debug output)
+- Get timing data per story/stage for performance analysis
+- Review token counts and API costs per story
+- Debug failures with full prompt/response dumps
+- Parse logs programmatically for CI/CD integration
+
+## Requirements
+
+**REQ-1: Log Levels**
+- Support 4 levels: `error`, `warn`, `info`, `debug`
+- Default level: `info`
+- CLI flags: `--verbose` (debug), `--quiet` (error+warn only), `--silent` (error only)
+- Environment variable override: `NAX_LOG_LEVEL=debug`
+
+**REQ-2: Structured Log Format**
+- Each log entry includes: `timestamp`, `level`, `stage`, `storyId`, `message`
+- Console output: human-readable with chalk (current style, but level-gated)
+- File output: JSON Lines (`.jsonl`) for machine parsing
+- File location: `nax/features/<name>/runs/<run-id>.jsonl`
+
+**REQ-3: Stage Lifecycle Events**
+- Emit structured events at each stage transition:
+  - `run.start` — feature name, story count, config
+  - `iteration.start` — iteration number, story id, complexity
+  - `context.built` — file count, token estimate
+  - `agent.start` — model, prompt size (chars/tokens), TDD strategy
+  - `agent.complete` — exit code, duration, stdout size, cost estimate
+  - `test.start` — test command
+  - `test.complete` — pass/fail, test count, duration
+  - `verification.start` — verification strategy
+  - `verification.complete` — pass/fail, issues found
+  - `story.complete` — story id, status, attempts, duration, cost
+  - `iteration.complete` — stories done this iteration
+  - `run.complete` — total stories, passed, failed, cost, duration
+
+**REQ-4: Per-Story Metrics**
+- Track and report per story: duration, API cost, token count (in/out), attempts, test count
+- Include in `prd.json` story metadata after run completes
+- Summary table at end of run (visible at `info` level)
+
+**REQ-5: Debug Mode**
+- `--debug` or `NAX_LOG_LEVEL=debug` enables:
+  - Full prompt text logged to file (not console)
+  - Full agent response logged to file
+  - Claude CLI command logged
+  - Environment variables passed to agent (sanitized — mask tokens)
+
+**REQ-6: Run History**
+- Each `nax run` creates a unique run ID (ISO timestamp or UUID)
+- Log file persisted at `nax/features/<name>/runs/<run-id>.jsonl`
+- Latest run symlinked as `nax/features/<name>/runs/latest.jsonl`
+- `nax runs list -f <feature>` lists past runs with summary
+- `nax runs show <run-id> -f <feature>` shows detailed run report
+
+**REQ-7: Logger API (Internal)**
+- Singleton logger instance, configured once at CLI entry
+- API: `logger.info(stage, storyId, message, data?)`, `logger.debug(...)`, etc.
+- Replace all `console.log` calls with logger calls
+- Logger writes to both console (filtered by level) and file (all levels)
+
+## Acceptance Criteria
+
+**AC-1:** `nax run -f foo --headless` shows same output as today at `info` level
+**AC-2:** `nax run -f foo --verbose` shows agent timing, token counts, and prompt sizes
+**AC-3:** `nax run -f foo --quiet` shows only warnings, errors, and final summary
+**AC-4:** After a run, `nax/features/foo/runs/latest.jsonl` contains structured events
+**AC-5:** Each JSONL line is valid JSON with `timestamp`, `level`, `stage`, `storyId` fields
+**AC-6:** `nax runs list -f foo` shows past runs with date, stories, cost, status
+**AC-7:** Per-story metrics (duration, cost, attempts) appear in the run summary table
+**AC-8:** Debug mode logs full prompts to file without printing to console
+**AC-9:** No `console.log` calls remain in src/ (all replaced with logger)
+
+## Technical Notes
+
+### Logger Implementation
+```typescript
+// src/logger.ts
+export type LogLevel = "error" | "warn" | "info" | "debug";
+
+export interface LogEntry {
+  timestamp: string;
+  level: LogLevel;
+  stage: string;
+  storyId?: string;
+  message: string;
+  data?: Record<string, unknown>;
+}
+
+export class Logger {
+  constructor(options: { level: LogLevel; filePath?: string });
+  error(stage: string, message: string, data?: Record<string, unknown>): void;
+  warn(stage: string, message: string, data?: Record<string, unknown>): void;
+  info(stage: string, message: string, data?: Record<string, unknown>): void;
+  debug(stage: string, message: string, data?: Record<string, unknown>): void;
+  withStory(storyId: string): StoryLogger; // scoped logger
+}
+```
+
+### Migration Strategy
+1. Create `src/logger.ts` with Logger class
+2. Add `--verbose`, `--quiet`, `--silent` flags to `bin/nax.ts`
+3. Replace `console.log` calls one module at a time
+4. Add stage events to orchestrator loop
+5. Add run history commands
+
+### Dependencies
+- None (use Bun built-in fs for file writing)
+- chalk remains for console formatting
+
+### File Structure
+```
+nax/features/<name>/runs/
+├── 2026-02-19T10-30-00Z.jsonl
+├── 2026-02-20T14-15-00Z.jsonl
+└── latest.jsonl -> 2026-02-20T14-15-00Z.jsonl
+```
+
+## Out of Scope
+- Remote log shipping (Datadog, Sentry, etc.)
+- Log rotation or cleanup policies
+- Real-time log streaming via WebSocket
+- Custom log formatters or plugins
+- Metrics dashboard or visualization
+
+---
+
+*Spec created 2026-02-19 for nax v0.8*

package/docs/v0.9.3-prompt-audit.md

@@ -0,0 +1,112 @@
+# v0.9.3 — Prompt Audit & Context Isolation
+
+**Status:** Draft
+**Author:** Subrina
+**Date:** 2026-02-23
+**Base:** v0.9.2
+
+## Overview
+
+Add tooling to inspect and verify story-scoped prompt isolation. Ensures each story's agent prompt contains only context relevant to that story — no cross-story leakage.
+
+## Motivation
+
+- No way to inspect what prompts agents actually receive without running a full `nax run`
+- `generateTestCoverageSummary` scans the entire repo's test files, leaking context from other stories into unrelated story prompts
+- No automated test verifying that `buildContext()` properly isolates per-story context
+- Prompt inspection is critical for debugging routing, context, and the upcoming v0.10 prompt optimizer
+
+## Deliverables
+
+### 1. `nax prompts` CLI Command
+
+New subcommand that assembles prompts for all stories in a feature without executing agents.
+
+```bash
+# Dump all story prompts to stdout
+nax prompts -f core
+
+# Write to directory
+nax prompts -f core --out ./prompt-dump/
+
+# Single story
+nax prompts -f core --story US-003
+```
+
+**Pipeline stages executed:** routing → context → constitution → prompt (stops before execution).
+
+**Output per story:**
+```
+prompt-dump/
+  US-001.prompt.md    # Final assembled prompt
+  US-001.context.md   # Context markdown only (for isolation audit)
+  US-002.prompt.md
+  US-002.context.md
+  ...
+```
+
+Each file includes a YAML frontmatter header:
+```yaml
+---
+storyId: US-003
+title: "Create health indicator interface"
+testStrategy: test-after
+modelTier: balanced
+contextTokens: 2450
+promptTokens: 3800
+dependencies: [US-001]
+contextElements:
+  - type: progress, tokens: 45
+  - type: story, storyId: US-003, tokens: 890
+  - type: dependency, storyId: US-001, tokens: 720
+  - type: test-coverage, tokens: 795
+---
+```
+
+**For three-session-tdd stories:** outputs `US-001.test-writer.md`, `US-001.implementer.md`, `US-001.verifier.md`.
+
+### 2. `buildContext` Isolation Unit Tests
+
+Test that `buildContext()` for a given story only includes:
+- The current story
+- Declared dependency stories
+- Progress summary (counts only, no story details)
+- Test coverage (to be scoped — see #3)
+- Error context from current story only
+- File context from current story's `contextFiles` only
+
+**Negative assertions:**
+- No story IDs from non-dependency stories appear in output
+- No acceptance criteria from unrelated stories leak through
+- Progress summary contains only aggregate counts, not story titles
+
+### 3. Scoped Test Coverage Scanner
+
+Fix `generateTestCoverageSummary` to scope results to story-relevant files:
+
+**Current behavior:** Scans all test files in `testDir` → agent sees coverage from every story.
+
+**New behavior:**
+1. If story has `contextFiles` → derive test file patterns from source paths (e.g., `src/health.service.ts` → `test/health.service.spec.ts`)
+2. If no `contextFiles` → fall back to full scan (current behavior) with a warning logged
+3. Add `context.testCoverage.scopeToStory` config option (default: `true`)
+
+## User Stories
+
+| # | Title | Complexity | Test Strategy | Dependencies |
+|:--|:------|:-----------|:--------------|:-------------|
+| US-001 | `nax prompts` CLI command with file output | medium | test-after | — |
+| US-002 | `buildContext` isolation unit tests | simple | test-after | — |
+| US-003 | Scope test coverage scanner to story-relevant files | medium | test-after | — |
+
+## Non-Goals
+
+- No changes to prompt assembly logic (that's v0.10 optimizer territory)
+- No `--optimized` flag yet (depends on v0.10)
+- No changes to TDD orchestrator prompt builders (just audit them)
+
+## Compatibility
+
+- `nax prompts` is additive — new CLI command, no existing behavior changed
+- Test coverage scoping is behind config flag with backward-compatible default
+- No breaking changes