@nathapp/nax 0.27.1 → 0.29.0

package/docs/v0.7-spec.md

@@ -1,177 +0,0 @@
-# nax v0.7 Specification
-**Date:** 2026-02-17
-**Status:** Draft
-
-## Theme: Test Context Injection
-
-v0.7 addresses test redundancy caused by isolated story sessions. Each agent session currently writes tests without knowing what prior stories already covered, leading to duplicate coverage.
-
-## Problem
-
-During dogfooding (bun-kv-store, 8 stories), we observed:
-- 6 tests for "name is required" (missing, empty, whitespace, not string, null, undefined)
-- Each story independently writes "comprehensive tests" for its area
-- Validation stories re-test what CRUD stories already covered
-- **Root cause:** Each session's prompt has zero visibility into existing test files
-
-## Solution: Test Context Injection
-
-Inject a summary of existing test files into each story's prompt so the agent knows what's already covered and avoids duplication.
-
-### How It Works
-
-Before the **prompt stage**, scan the project's test directory and generate a concise summary:
-
-```
-## Existing Test Coverage
-
-### test/store.test.ts (45 tests)
-- CRUD operations: create, read, update, delete
-- Error handling: missing key, invalid value
-- Batch operations: getMany, setMany
-
-### test/validation.test.ts (12 tests)  
-- Input validation: name required, type checking
-- Size limits: max key length, max value size
-```
-
-This summary is injected into the prompt alongside the story context, constitution, and other context elements.
-
-### Context Element
-
-```typescript
-interface TestCoverageContext {
-  type: 'test-coverage';
-  priority: 85;           // Below constitution (95), above file context (50)
-  content: string;        // Formatted test summary
-  tokens: number;
-  source: string;         // e.g., "test/*.test.ts"
-}
-```
-
-### Summary Generation
-
-```typescript
-interface TestSummaryOptions {
-  /** Test directory to scan (default: auto-detect from config or common patterns) */
-  testDir?: string;
-  /** Glob pattern for test files (default: "**/*.test.{ts,js,tsx,jsx}") */
-  testPattern?: string;
-  /** Max tokens for the summary (default: 500) */
-  maxTokens?: number;
-  /** Summary detail level */
-  detail: 'names-only' | 'names-and-counts' | 'describe-blocks';
-}
-```
-
-**Detail levels:**
-- `names-only` — Just file names and test count: `test/store.test.ts (45 tests)`
-- `names-and-counts` — File names + top-level describe blocks with counts
-- `describe-blocks` — File names + describe blocks + test names (most expensive but most useful)
-
-**Default:** `names-and-counts` (good balance of info vs tokens)
-
-### Scanning Approach
-
-1. Find test files matching pattern
-2. For each file, extract `describe()` and `test()`/`it()` block names via regex (no AST parsing needed)
-3. Format as markdown summary
-4. Truncate to token budget
-
-### Prompt Injection
-
-Add to the story prompt:
-
-```
-## Existing Test Coverage
-
-The following tests already exist. DO NOT duplicate this coverage.
-Focus only on testing NEW behavior introduced by this story.
-
-### test/store.test.ts (45 tests)
-- CRUD operations (12 tests): create, read, update, delete, upsert
-- Validation (8 tests): required fields, type checks, size limits
-- Error handling (6 tests): not found, duplicate key, connection error
-...
-
-## Your Story
-
-US-007: Add input sanitization
-...
-```
-
-### Config
-
-```json
-{
-  "context": {
-    "testCoverage": {
-      "enabled": true,
-      "detail": "names-and-counts",
-      "maxTokens": 500,
-      "testDir": "test",
-      "testPattern": "**/*.test.{ts,js}"
-    }
-  }
-}
-```
-
-## Acceptance Criteria
-
-- [ ] Test files scanned and summarized before each story prompt
-- [ ] Summary injected into prompt at priority 85
-- [ ] `describe()` and `test()` names extracted via regex
-- [ ] Summary respects maxTokens budget
-- [ ] Config allows enabling/disabling and adjusting detail level
-- [ ] Summary updates between stories (reflects tests added by prior stories)
-- [ ] No performance regression (scanning should be <100ms for typical projects)
-
-## Implementation Plan
-
-### Phase 1: Test Scanner
-**Files:** `src/context/test-scanner.ts`
-- Scan test directory for test files
-- Extract describe/test block names via regex
-- Format as markdown summary
-- Respect token budget
-
-**Commit:** `feat(context): add test file scanner for coverage summary`
-
-### Phase 2: Context Integration
-**Files:** `src/context/builder.ts`, `src/pipeline/stages/context.ts`, `src/config/schema.ts`
-- Add TestCoverageContext element type
-- Wire scanner into context stage (runs before prompt assembly)
-- Add config schema for testCoverage settings
-- Summary refreshes between stories
-
-**Commit:** `feat(context): inject test coverage summary into story prompts`
-
-### Phase 3: Prompt Guidance
-**Files:** `src/execution/prompts.ts`
-- Add "DO NOT duplicate" instruction in prompt template
-- Reference existing coverage summary
-- Reinforce constitution test guidance
-
-**Commit:** `feat(prompts): add test dedup guidance referencing coverage summary`
-
-## Test Strategy
-- Mode: test-after
-- Unit tests: scanner regex extraction, summary formatting, token truncation
-- Integration: context builder includes test coverage element
-- Run `bun test && bun run typecheck` after each phase
-
-## Estimated Effort
-~300-400 LOC across 3 phases.
-
-## Measurement
-
-Compare v0.5.0 (no dedup) vs v0.7.0 (context injection) on the same dogfood project:
-
-| Metric | v0.5.0 | v0.7.0 |
-|:---|:---|:---|
-| Total tests generated | ? | ? |
-| Redundant tests | ? | ? |
-| Code quality grade | ? | ? |
-| Acceptance rate | ? | ? |
-| Total cost | ? | ? |
-| Total time | ? | ? |

package/docs/v0.8-llm-routing.md

@@ -1,206 +0,0 @@
-# v0.8 — LLM-Enhanced Routing
-
-> Priority: **HIGH** — keyword routing causes costly misroutes (e.g., US-008 simple barrel exports → powerful + TDD due to "public api" keyword match).
-
-## Problem
-
-Keyword-based routing is brittle and expensive:
-1. **False positives**: "public api" in title → three-session-tdd even for simple barrel exports ($1.25 wasted)
-2. **False negatives**: Complex integration work without magic keywords → test-after on fast tier
-3. **No semantic understanding**: Can't assess *actual* implementation effort from acceptance criteria
-4. **Keyword overlap**: Security keywords fire on simple "add auth header to request" stories
-
-Evidence from dogfood runs:
-- Run D2: US-008 ("Export public API and create barrel exports") — simple task, keyword matched "public api" → powerful + three-session-tdd. Cost: $1.25. Should have been: fast + test-after (~$0.10).
-
-## Design
-
-### Config
-
-```json
-{
-  "routing": {
-    "strategy": "llm",
-    "llm": {
-      "model": "fast",
-      "fallbackToKeywords": true,
-      "maxInputTokens": 2000,
-      "cacheDecisions": true
-    }
-  }
-}
-```
-
-- `model`: Tier used for the routing LLM call itself (default: `fast` — routing should be cheap)
-- `fallbackToKeywords`: If LLM call fails (timeout, parse error), fall back to keyword strategy (default: `true`)
-- `maxInputTokens`: Token budget for story context sent to LLM (default: `2000`)
-- `cacheDecisions`: Cache routing decisions per story ID within a run (default: `true`)
-
-### LLM Prompt
-
-The LLM receives a structured prompt with the story and must return a JSON decision:
-
-```
-You are a code task router. Given a user story, classify its complexity and select the appropriate execution strategy.
-
-## Story
-Title: {title}
-Description: {description}
-Acceptance Criteria:
-{acceptanceCriteria as numbered list}
-Tags: {tags}
-
-## Available Tiers
-- fast: Simple changes, typos, config updates, boilerplate. <30 min of coding.
-- balanced: Standard features, moderate logic, straightforward tests. 30-90 min.
-- powerful: Complex architecture, security-critical, multi-file refactors, novel algorithms. >90 min.
-
-## Available Test Strategies
-- test-after: Write implementation first, add tests after. For straightforward work.
-- three-session-tdd: Separate test-writer → implementer → verifier sessions. For complex/critical work where test design matters.
-
-## Rules
-- Default to the CHEAPEST option that will succeed.
-- three-session-tdd ONLY when: (a) security/auth logic, (b) complex algorithms, (c) public API contracts that consumers depend on.
-- Simple barrel exports, re-exports, or index files are ALWAYS test-after + fast, regardless of keywords.
-- A story touching many files doesn't automatically mean complex — copy-paste refactors are simple.
-
-Respond with ONLY this JSON (no markdown, no explanation):
-{"complexity":"simple|medium|complex|expert","modelTier":"fast|balanced|powerful","testStrategy":"test-after|three-session-tdd","reasoning":"<one line>"}
-```
-
-### Implementation
-
-Modify `src/routing/strategies/llm.ts`:
-
-```typescript
-export const llmStrategy: RoutingStrategy = {
-  name: "llm",
-
-  async route(story: UserStory, context: RoutingContext): Promise<RoutingDecision | null> {
-    const config = context.config;
-    const llmConfig = config.routing.llm;
-    if (!llmConfig) return null;
-
-    // Check cache
-    if (llmConfig.cacheDecisions && cachedDecisions.has(story.id)) {
-      return cachedDecisions.get(story.id)!;
-    }
-
-    try {
-      const prompt = buildRoutingPrompt(story, config);
-      const modelId = config.models[llmConfig.model ?? "fast"];
-
-      const result = await callLlm(modelId, prompt, {
-        maxTokens: 200,
-        timeout: 15_000, // 15s hard limit — routing shouldn't block
-      });
-
-      const decision = parseRoutingResponse(result, story, config);
-
-      if (llmConfig.cacheDecisions) {
-        cachedDecisions.set(story.id, decision);
-      }
-
-      return decision;
-    } catch (err) {
-      logger.warn(`LLM routing failed for ${story.id}: ${err.message}`);
-      return null; // Falls through to keyword strategy
-    }
-  },
-};
-```
-
-### LLM Call Mechanism
-
-nax already spawns Claude Code via `Bun.spawn`. For the routing LLM call, we need a **lightweight** approach:
-
-**Option A — Claude Code one-shot**: `claude -p "..." --model <model>` with 15s timeout.
-- Pro: Reuses existing infra, model aliases work.
-- Con: ~3-5s startup overhead per call. For 9 stories = 27-45s total.
-
-**Option B — Direct API call**: HTTP request to provider API (Anthropic/OpenAI/Google).
-- Pro: <1s per call, batch-friendly.
-- Con: Needs API key handling, provider-specific code.
-
-**Recommendation: Option A** for v0.8 (simplicity), with config option to batch all stories in one call:
-
-```
-// Single LLM call for all pending stories (batch mode)
-"Route these 9 stories:\n1. US-001: ...\n2. US-002: ...\n\nRespond with JSON array: [{id, complexity, modelTier, testStrategy, reasoning}]"
-```
-
-Batch mode cuts 9 calls → 1 call. ~5s total routing overhead.
-
-### Strategy Interface Change
-
-The current `RoutingStrategy.route()` is synchronous. LLM routing needs async:
-
-```typescript
-export interface RoutingStrategy {
-  readonly name: string;
-  route(story: UserStory, context: RoutingContext): RoutingDecision | null | Promise<RoutingDecision | null>;
-}
-```
-
-`StrategyChain.route()` becomes async (already called with `await` in `routeStory()`).
-
-### Error Handling
-
-| Failure | Behavior |
-|:---|:---|
-| LLM timeout (>15s) | Return null → keyword fallback |
-| JSON parse error | Return null → keyword fallback |
-| Invalid field values | Return null → keyword fallback |
-| LLM returns unknown complexity | Clamp to nearest valid value |
-| All failures logged | `logger.warn()` with story ID |
-
-### Logging
-
-```
-[routing] LLM classified US-008 as simple/fast/test-after: "Barrel export file, no logic to test"
-[routing] LLM routing failed for US-003: timeout after 15000ms, falling back to keyword
-```
-
-## Cost Analysis
-
-| Scenario | Keyword Cost | LLM Routing Cost | Savings |
-|:---|:---|:---|:---|
-| US-008 (barrel exports) | $1.25 (powerful+TDD) | $0.10 (fast+test-after) + $0.01 routing | **$1.14 saved** |
-| 9-story run (batch) | Variable | ~$0.02 routing overhead | Net positive if prevents 1+ misroute |
-
-LLM routing call: ~500 input tokens + 100 output tokens per story = ~$0.001 on Flash.
-Batch mode (9 stories): ~2000 input + 400 output = ~$0.003 total.
-
-**ROI: One prevented misroute pays for ~400 routing calls.**
-
-## Acceptance Criteria
-
-1. `config.routing.strategy = "llm"` activates LLM routing
-2. LLM strategy returns structured `RoutingDecision` with reasoning
-3. Falls back to keyword strategy on any LLM failure
-4. Batch mode: single LLM call routes all pending stories
-5. Routing decisions cached per story ID within a run
-6. Strategy chain async support (non-breaking — keyword still sync)
-7. Routing overhead < 10s for batch of 10 stories
-8. Logging: every LLM routing decision logged with reasoning
-
-## Files to Modify
-
-- `src/routing/strategies/llm.ts` — Main implementation
-- `src/routing/strategy.ts` — Make interface async-compatible
-- `src/routing/chain.ts` — `route()` becomes async
-- `src/config/schema.ts` — Add `LlmRoutingConfig` type
-- `src/config/defaults.ts` — Add LLM routing defaults
-- `test/routing/llm-strategy.test.ts` — Unit tests
-- `test/routing/chain.test.ts` — Update for async
-
-## Non-Goals (v0.8)
-
-- Direct API calls (Option B) — defer to v0.9+
-- Adaptive routing (learning from historical data) — existing stub, separate feature
-- Custom routing prompts — hardcoded prompt is fine for now
-
----
-
-*Created 2026-02-19*

package/docs/v0.8-structured-logging.md

@@ -1,132 +0,0 @@
-# Feature: Structured Logging for nax v0.8
-
-## Problem
-
-nax currently uses raw `console.log` with chalk formatting throughout the codebase. Developers running `nax run` in headless mode have no way to:
-- Control verbosity (only see errors vs full debug output)
-- Get timing data per story/stage for performance analysis
-- Review token counts and API costs per story
-- Debug failures with full prompt/response dumps
-- Parse logs programmatically for CI/CD integration
-
-## Requirements
-
-**REQ-1: Log Levels**
-- Support 4 levels: `error`, `warn`, `info`, `debug`
-- Default level: `info`
-- CLI flags: `--verbose` (debug), `--quiet` (error+warn only), `--silent` (error only)
-- Environment variable override: `NAX_LOG_LEVEL=debug`
-
-**REQ-2: Structured Log Format**
-- Each log entry includes: `timestamp`, `level`, `stage`, `storyId`, `message`
-- Console output: human-readable with chalk (current style, but level-gated)
-- File output: JSON Lines (`.jsonl`) for machine parsing
-- File location: `nax/features/<name>/runs/<run-id>.jsonl`
-
-**REQ-3: Stage Lifecycle Events**
-- Emit structured events at each stage transition:
-  - `run.start` — feature name, story count, config
-  - `iteration.start` — iteration number, story id, complexity
-  - `context.built` — file count, token estimate
-  - `agent.start` — model, prompt size (chars/tokens), TDD strategy
-  - `agent.complete` — exit code, duration, stdout size, cost estimate
-  - `test.start` — test command
-  - `test.complete` — pass/fail, test count, duration
-  - `verification.start` — verification strategy
-  - `verification.complete` — pass/fail, issues found
-  - `story.complete` — story id, status, attempts, duration, cost
-  - `iteration.complete` — stories done this iteration
-  - `run.complete` — total stories, passed, failed, cost, duration
-
-**REQ-4: Per-Story Metrics**
-- Track and report per story: duration, API cost, token count (in/out), attempts, test count
-- Include in `prd.json` story metadata after run completes
-- Summary table at end of run (visible at `info` level)
-
-**REQ-5: Debug Mode**
-- `--debug` or `NAX_LOG_LEVEL=debug` enables:
-  - Full prompt text logged to file (not console)
-  - Full agent response logged to file
-  - Claude CLI command logged
-  - Environment variables passed to agent (sanitized — mask tokens)
-
-**REQ-6: Run History**
-- Each `nax run` creates a unique run ID (ISO timestamp or UUID)
-- Log file persisted at `nax/features/<name>/runs/<run-id>.jsonl`
-- Latest run symlinked as `nax/features/<name>/runs/latest.jsonl`
-- `nax runs list -f <feature>` lists past runs with summary
-- `nax runs show <run-id> -f <feature>` shows detailed run report
-
-**REQ-7: Logger API (Internal)**
-- Singleton logger instance, configured once at CLI entry
-- API: `logger.info(stage, storyId, message, data?)`, `logger.debug(...)`, etc.
-- Replace all `console.log` calls with logger calls
-- Logger writes to both console (filtered by level) and file (all levels)
-
-## Acceptance Criteria
-
-**AC-1:** `nax run -f foo --headless` shows same output as today at `info` level
-**AC-2:** `nax run -f foo --verbose` shows agent timing, token counts, and prompt sizes
-**AC-3:** `nax run -f foo --quiet` shows only warnings, errors, and final summary
-**AC-4:** After a run, `nax/features/foo/runs/latest.jsonl` contains structured events
-**AC-5:** Each JSONL line is valid JSON with `timestamp`, `level`, `stage`, `storyId` fields
-**AC-6:** `nax runs list -f foo` shows past runs with date, stories, cost, status
-**AC-7:** Per-story metrics (duration, cost, attempts) appear in the run summary table
-**AC-8:** Debug mode logs full prompts to file without printing to console
-**AC-9:** No `console.log` calls remain in src/ (all replaced with logger)
-
-## Technical Notes
-
-### Logger Implementation
-```typescript
-// src/logger.ts
-export type LogLevel = "error" | "warn" | "info" | "debug";
-
-export interface LogEntry {
-  timestamp: string;
-  level: LogLevel;
-  stage: string;
-  storyId?: string;
-  message: string;
-  data?: Record<string, unknown>;
-}
-
-export class Logger {
-  constructor(options: { level: LogLevel; filePath?: string });
-  error(stage: string, message: string, data?: Record<string, unknown>): void;
-  warn(stage: string, message: string, data?: Record<string, unknown>): void;
-  info(stage: string, message: string, data?: Record<string, unknown>): void;
-  debug(stage: string, message: string, data?: Record<string, unknown>): void;
-  withStory(storyId: string): StoryLogger; // scoped logger
-}
-```
-
-### Migration Strategy
-1. Create `src/logger.ts` with Logger class
-2. Add `--verbose`, `--quiet`, `--silent` flags to `bin/nax.ts`
-3. Replace `console.log` calls one module at a time
-4. Add stage events to orchestrator loop
-5. Add run history commands
-
-### Dependencies
-- None (use Bun built-in fs for file writing)
-- chalk remains for console formatting
-
-### File Structure
-```
-nax/features/<name>/runs/
-├── 2026-02-19T10-30-00Z.jsonl
-├── 2026-02-20T14-15-00Z.jsonl
-└── latest.jsonl -> 2026-02-20T14-15-00Z.jsonl
-```
-
-## Out of Scope
-- Remote log shipping (Datadog, Sentry, etc.)
-- Log rotation or cleanup policies
-- Real-time log streaming via WebSocket
-- Custom log formatters or plugins
-- Metrics dashboard or visualization
-
----
-
-*Spec created 2026-02-19 for nax v0.8*

package/docs/v0.9.3-prompt-audit.md

@@ -1,112 +0,0 @@
-# v0.9.3 — Prompt Audit & Context Isolation
-
-**Status:** Draft
-**Author:** Subrina
-**Date:** 2026-02-23
-**Base:** v0.9.2
-
-## Overview
-
-Add tooling to inspect and verify story-scoped prompt isolation. Ensures each story's agent prompt contains only context relevant to that story — no cross-story leakage.
-
-## Motivation
-
-- No way to inspect what prompts agents actually receive without running a full `nax run`
-- `generateTestCoverageSummary` scans the entire repo's test files, leaking context from other stories into unrelated story prompts
-- No automated test verifying that `buildContext()` properly isolates per-story context
-- Prompt inspection is critical for debugging routing, context, and the upcoming v0.10 prompt optimizer
-
-## Deliverables
-
-### 1. `nax prompts` CLI Command
-
-New subcommand that assembles prompts for all stories in a feature without executing agents.
-
-```bash
-# Dump all story prompts to stdout
-nax prompts -f core
-
-# Write to directory
-nax prompts -f core --out ./prompt-dump/
-
-# Single story
-nax prompts -f core --story US-003
-```
-
-**Pipeline stages executed:** routing → context → constitution → prompt (stops before execution).
-
-**Output per story:**
-```
-prompt-dump/
-  US-001.prompt.md    # Final assembled prompt
-  US-001.context.md   # Context markdown only (for isolation audit)
-  US-002.prompt.md
-  US-002.context.md
-  ...
-```
-
-Each file includes a YAML frontmatter header:
-```yaml
----
-storyId: US-003
-title: "Create health indicator interface"
-testStrategy: test-after
-modelTier: balanced
-contextTokens: 2450
-promptTokens: 3800
-dependencies: [US-001]
-contextElements:
-  - type: progress, tokens: 45
-  - type: story, storyId: US-003, tokens: 890
-  - type: dependency, storyId: US-001, tokens: 720
-  - type: test-coverage, tokens: 795
----
-```
-
-**For three-session-tdd stories:** outputs `US-001.test-writer.md`, `US-001.implementer.md`, `US-001.verifier.md`.
-
-### 2. `buildContext` Isolation Unit Tests
-
-Test that `buildContext()` for a given story only includes:
-- The current story
-- Declared dependency stories
-- Progress summary (counts only, no story details)
-- Test coverage (to be scoped — see #3)
-- Error context from current story only
-- File context from current story's `contextFiles` only
-
-**Negative assertions:**
-- No story IDs from non-dependency stories appear in output
-- No acceptance criteria from unrelated stories leak through
-- Progress summary contains only aggregate counts, not story titles
-
-### 3. Scoped Test Coverage Scanner
-
-Fix `generateTestCoverageSummary` to scope results to story-relevant files:
-
-**Current behavior:** Scans all test files in `testDir` → agent sees coverage from every story.
-
-**New behavior:**
-1. If story has `contextFiles` → derive test file patterns from source paths (e.g., `src/health.service.ts` → `test/health.service.spec.ts`)
-2. If no `contextFiles` → fall back to full scan (current behavior) with a warning logged
-3. Add `context.testCoverage.scopeToStory` config option (default: `true`)
-
-## User Stories
-
-| # | Title | Complexity | Test Strategy | Dependencies |
-|:--|:------|:-----------|:--------------|:-------------|
-| US-001 | `nax prompts` CLI command with file output | medium | test-after | — |
-| US-002 | `buildContext` isolation unit tests | simple | test-after | — |
-| US-003 | Scope test coverage scanner to story-relevant files | medium | test-after | — |
-
-## Non-Goals
-
-- No changes to prompt assembly logic (that's v0.10 optimizer territory)
-- No `--optimized` flag yet (depends on v0.10)
-- No changes to TDD orchestrator prompt builders (just audit them)
-
-## Compatibility
-
-- `nax prompts` is additive — new CLI command, no existing behavior changed
-- Test coverage scoping is behind config flag with backward-compatible default
-- No breaking changes