prjct-cli 1.7.5 → 1.9.0

package/CHANGELOG.md

@@ -1,12 +1,216 @@
 # Changelog
 
+## [1.9.0] - 2026-02-07
+
+### Features
+
+- add structured output schema to all LLM prompts (PRJ-264) (#150)
+- add mandatory model specification to AI provider (PRJ-265) (#149)
+
+### Bug Fixes
+
+- replace keyword domain detection with LLM semantic classification (PRJ-299) (#148)
+
+
+## [1.10.0] - 2026-02-07
+
+### Features
+- **Add structured output schema to all LLM prompts (PRJ-264)**: LLM prompts now include explicit JSON output schemas. Responses are validated with Zod before use. Invalid responses trigger re-prompt with structured error feedback.
+
+### Implementation Details
+- New `core/schemas/llm-output.ts`: Zod schemas for task classification, agent assignment, and subtask breakdown responses. Schema registry (`OUTPUT_SCHEMAS`) with examples that self-validate. `renderSchemaForPrompt()` serializes schemas as markdown format instructions for prompt injection.
+- New `core/agentic/response-validator.ts`: `validateLLMResponse()` handles JSON parsing (plain and markdown-wrapped `\`\`\`json` fences), Zod validation, and typed results. `buildReprompt()` generates retry messages with specific validation errors.
+- Replaced manual field-by-field validation in `domain-classifier.ts` with `TaskClassificationSchema.safeParse()` — the schema existed (PRJ-299) but was unused.
+- Added output schema injection to `prompt-builder.ts` `build()` method with `getSchemaTypeForCommand()` mapping commands to schemas.
+- 20 new unit tests in `core/__tests__/agentic/response-validator.test.ts`
+
+### Test Plan
+
+#### For QA
+1. Run `bun test core/__tests__/agentic/response-validator.test.ts` — all 20 tests pass
+2. Run `bun test` — full suite (677 tests) passes with no regressions
+3. Run `bun run build` — build succeeds cleanly
+4. Verify `renderSchemaForPrompt('classification')` returns markdown with OUTPUT FORMAT header
+5. Verify `validateLLMResponse()` handles plain JSON, markdown-wrapped JSON, and rejects non-JSON
+6. Verify OUTPUT_SCHEMAS registry examples validate against their own schemas
+
+#### For Users
+**What changed:** LLM prompts include explicit JSON output schemas. Domain classifier uses Zod validation. Response validator provides structured error handling with re-prompt.
+**How to use:** Automatic — schemas injected into prompts and validation runs transparently.
+**Breaking changes:** None — all changes are additive.
+
+## [1.9.0] - 2026-02-07
+
+### Features
+- **Add mandatory model specification to AI provider (PRJ-265)**: Provider configs now include `defaultModel`, `supportedModels`, and `minCliVersion` fields. Analysis and task metadata can record which model was used, enabling consistency tracking and mismatch warnings.
+
+### Implementation Details
+- New `core/schemas/model.ts`: Zod schemas defining supported models per provider (Claude: opus/sonnet/haiku, Gemini: 2.5-pro/2.5-flash/2.0-flash), default model resolution, semver comparison utilities, minimum CLI version validation, and model mismatch detection
+- Extended `AIProviderConfig` interface in `core/types/provider.ts` with `defaultModel`, `supportedModels`, `minCliVersion` fields
+- All 5 provider configs (Claude, Gemini, Cursor, Windsurf, Antigravity) updated with model specification fields
+- Added `modelMetadata` (optional) to `CurrentTaskSchema` in `core/schemas/state.ts` and `AnalysisSchema` in `core/schemas/analysis.ts`
+- Added `preferredModel` to `ProjectSettings` in `core/types/config.ts`
+- Added `validateCliVersion()` to `core/infrastructure/ai-provider.ts` with version warning integration into `detectProvider()`
+- Added `versionWarning` field to `ProviderDetectionResult`
+- 32 new unit tests in `core/__tests__/schemas/model.test.ts`
+
+### Test Plan
+
+#### For QA
+1. Verify `ClaudeProvider.defaultModel` is `'sonnet'` and `supportedModels` includes `['opus', 'sonnet', 'haiku']`
+2. Verify `GeminiProvider.defaultModel` is `'2.5-flash'` and `supportedModels` includes `['2.5-pro', '2.5-flash', '2.0-flash']`
+3. Verify multi-model IDEs (Cursor, Windsurf) have `null` defaultModel and empty supportedModels
+4. Run `bun test core/__tests__/schemas/model.test.ts` — all 32 tests pass
+5. Run `bun test` — full suite (657 tests) passes with no regressions
+6. Run `bun run build` — build succeeds cleanly
+
+#### For Users
+**What changed:** Provider configs now include model specification fields. Analysis and task metadata can record which model was used. Version validation warns if CLI is outdated.
+**How to use:** Existing configs work unchanged — model fields have sensible defaults. New `preferredModel` setting available in project settings.
+**Breaking changes:** None — all new fields are optional or have defaults.
+
+## [1.8.1] - 2026-02-07
+
+### Bug Fixes
+- **Replace keyword domain detection with LLM semantic classification (PRJ-299)**: Eliminated substring false positives in domain classification. "author" no longer matches "auth" → backend, "Build responsive dashboard" correctly routes to frontend.
+
+### Implementation Details
+- New `core/agentic/domain-classifier.ts`: LLM-based classifier with 4-level fallback chain (cache → confirmed history → Claude Haiku API → word-boundary heuristic)
+- New `core/schemas/classification.ts`: Zod schemas for TaskClassification, cache entries, and confirmed patterns
+- Replaced substring `includes()` matching in `smart-context.ts` and `orchestrator-executor.ts` with word-boundary regex (`\b`)
+- Removed ~230 lines of hardcoded keyword lists from both files
+- Classification results cached per (project + description hash) with 1-hour TTL
+- Successful classifications auto-persisted as confirmed patterns via `confirmClassification()`
+
+### Learnings
+- Word-boundary regex (`\b`) correctly rejects "author" matching "auth" because there's no boundary between "auth" and "or" in "author"
+- Using raw `fetch` to Claude API avoids adding `@anthropic-ai/sdk` dependency while keeping vendor-neutral design
+- Centralized classifier in `domain-classifier.ts` consumed by both `smart-context.ts` and `orchestrator-executor.ts` eliminates duplication
+
+### Test Plan
+
+#### For QA
+1. Run `bun test` — all 625 tests should pass
+2. Verify `detectDomain('Fix the author display on profile page')` returns `frontend` (not `backend`)
+3. Verify `detectDomain('Build responsive dashboard')` returns `frontend` (not `general`)
+4. Verify `detectDomain('Fix the auth middleware')` returns `backend` (standalone "auth" still works)
+5. Verify `classifyWithHeuristic` returns `general` with confidence 0.3 for unrecognizable tasks
+6. Run `bun run build` — build should succeed
+
+#### For Users
+**What changed:** Domain classification uses smarter word-boundary matching, eliminating false positives.
+**How to use:** No user-facing changes — classification happens automatically during `p. task`.
+**Breaking changes:** None for end users.
+
+## [1.8.0] - 2026-02-07
+
+### Features
+
+- add Fibonacci estimation with variance tracking (PRJ-295) (#145)
+- add PerformanceTracker for CLI metrics (PRJ-297) (#146)
+
+### Bug Fixes
+
+- replace hardcoded command lists with config-driven context (PRJ-298) (#147)
+
+
+## [1.8.0] - 2026-02-07
+
+### Features
+- **Fibonacci estimation with variance tracking (PRJ-295)**: Capture Fibonacci point estimates (1,2,3,5,8,13,21) on task start with automatic points-to-time conversion, record actual duration on done, and display estimation variance.
+
+### Implementation Details
+- New `core/domain/fibonacci.ts` module: `FIBONACCI_POINTS`, `pointsToMinutes()`, `pointsToTimeRange()`, `findClosestPoint()`, `suggestFromHistory()`
+- Added `estimatedPoints` and `estimatedMinutes` optional fields to `CurrentTaskSchema` and `SubtaskSchema`
+- Added `updateCurrentTask()` partial update method to `StateStorage`
+- `now()` handler returns `fibonacci` helper object with `storeEstimate(points)` for template use
+- `done()` handler records outcomes via `outcomeRecorder.record()` and displays variance: `est: 5pt (1h 30m) → +50%`
+
+### Test Plan
+
+#### For QA
+1. Start a task — verify `fibonacci` helper is returned with `storeEstimate()`, `pointsToMinutes()`, `pointsToTimeRange()`
+2. Call `storeEstimate(5)` — verify `estimatedPoints: 5` and `estimatedMinutes: 90` in state.json
+3. Complete task with `p. done` — verify outcome recorded to `outcomes/outcomes.jsonl`
+4. Verify variance display shows `est: 5pt (1h 30m) → +X%`
+5. Run `bun test` — 552 tests pass
+
+#### For Users
+**What changed:** Tasks now support Fibonacci point estimation with automatic time conversion and variance tracking on completion.
+**How to use:** Estimation is stored via `storeEstimate(points)` during task start; variance is auto-displayed on `p. done`.
+**Breaking changes:** None — estimation fields are optional.
+
+## [1.7.7] - 2026-02-07
+
+### Bug Fixes
+- **Config-driven command context (PRJ-298)**: Replaced 4 hardcoded command lists in `prompt-builder.ts` with a single `command-context.config.json` config file. New commands no longer silently get zero context — the wildcard `*` entry provides sensible defaults, and a heuristic classifier handles unknown commands.
+- **Quality checklists for ship/done**: `ship` and `done` commands now receive quality checklists (previously excluded from the hardcoded list).
+
+### Implementation Details
+- Created `core/config/command-context.config.json` mapping 25 commands + wildcard to context sections (agents, patterns, checklists, modules)
+- Zod schema in `core/schemas/command-context.ts` validates config at load time
+- `core/agentic/command-context.ts` provides `resolveCommandContextFull()` with fallback chain: config → cache → heuristic classify → wildcard
+- `core/agentic/command-classifier.ts` uses word-boundary keyword matching with score-based priority to classify unknown commands from template metadata
+- Auto-learn (Phase 3): after 3 identical heuristic classifications, persists to config file via fire-and-forget
+
+### Learnings
+- Keyword substring matching causes false positives (e.g., "check" matching inside "checks") — word boundaries via `\b` regex are essential
+- When quality and info keywords overlap, score-based priority (higher count wins) is more robust than boolean exclusion
+
+### Test Plan
+
+#### For QA
+1. Run `bun test ./core/__tests__/agentic/command-context.test.ts` — all 20 tests pass
+2. Run `bun test ./core/__tests__/agentic/prompt-builder.test.ts` — all 16 existing tests pass
+3. Run `bun run build` — compiles without errors
+4. Verify `ship` and `done` commands have `checklist: true` in config
+5. Verify unknown commands get wildcard defaults (agents: true, patterns: true)
+
+#### For Users
+**What changed:** Commands like `ship` and `done` now receive quality checklists. New commands automatically get sensible context instead of nothing.
+**How to use:** No user action needed — works automatically.
+**Breaking changes:** None
+
+## [1.7.6] - 2026-02-07
+
+### Features
+- **PerformanceTracker service (PRJ-297)**: New `core/infrastructure/performance-tracker.ts` singleton that automatically measures startup time, memory usage, and command durations on every CLI invocation. Data stored in append-only JSONL with 5MB rotation.
+- **`prjct perf` dashboard command**: Shows performance metrics vs targets for the last N days (default 7). Displays startup time, heap/RSS memory, context correctness rate, subtask handoff rate, and per-command duration breakdown.
+- **Zod schemas for performance metrics**: `core/schemas/performance.ts` with typed schemas for all metric types (timing, memory, context correctness, subtask handoff, analysis state).
+
+### Implementation Details
+- PerformanceTracker uses `process.hrtime.bigint()` for nanosecond-precision timing and `process.memoryUsage()` for memory snapshots
+- Startup time captured at top of `bin/prjct.ts` via `globalThis.__perfStartNs` and recorded in `core/index.ts` after command execution
+- All instrumentation wrapped in non-critical try/catch to prevent perf tracking from breaking CLI functionality
+- Uses existing `jsonl-helper.appendJsonLineWithRotation` for storage (5MB rotation limit)
+- 17 unit tests covering timing, memory, recording, context correctness, handoff, and report generation
+
+### Learnings
+- JSONL append-only pattern with rotation is ideal for time-series metrics (vs JSON write-through for stateful data)
+- `globalThis` works well for passing data between `bin/` entry point and `core/` modules without import coupling
+- `process.memoryUsage().heapUsed` can momentarily exceed `heapTotal` during GC — don't assert `<=`
+
+### Test Plan
+
+#### For QA
+1. Run `prjct status` then `prjct perf` — verify metrics appear (startup time, memory, command duration)
+2. Run multiple commands then `prjct perf 1` — verify all commands show in dashboard
+3. Check `~/.prjct-cli/projects/{id}/storage/performance.jsonl` exists with valid JSONL entries
+4. Verify `prjct perf` with no data shows "No performance data yet" message
+5. Verify target indicators: startup `<500ms` green, `>500ms` yellow warning
+
+#### For Users
+**What changed:** New `prjct perf` command shows a performance dashboard with startup time, memory usage, and command duration metrics.
+**How to use:** Run `prjct perf` (default 7 days) or `prjct perf 30` for 30-day view. Metrics are collected automatically.
+**Breaking changes:** None
+
+
 ## [1.7.5] - 2026-02-07
 
 ### Refactoring
 
 - remove unused deps and lazy-load @linear/sdk (PRJ-291) (#144)
 
-
 ## [1.7.5] - 2026-02-07
 
 ### Changed

package/bin/prjct.ts

@@ -8,6 +8,10 @@
  * auto-install on first CLI use. This is the reliable path.
  */
 
+// Performance: capture process start time (nanosecond precision)
+// Exposed via globalThis so core/index.ts can read it for startup time metrics
+;(globalThis as Record<string, unknown>).__perfStartNs = process.hrtime.bigint()
+
 import os from 'node:os'
 import path from 'node:path'
 import chalk from 'chalk'
@@ -85,6 +89,16 @@ async function trackSession(command: string): Promise<() => void> {
       return () => {
         const durationMs = Date.now() - start
         sessionTracker.trackCommand(projectId, command, durationMs).catch(() => {})
+
+        // Performance tracking (non-critical, lazy-loaded)
+        import('../core/infrastructure/performance-tracker')
+          .then(({ performanceTracker }) => {
+            performanceTracker
+              .recordTiming(projectId, 'command_duration', durationMs, { command })
+              .catch(() => {})
+            performanceTracker.recordMemory(projectId, { command }).catch(() => {})
+          })
+          .catch(() => {})
       }
     }
   } catch {

package/core/__tests__/agentic/command-context.test.ts

@@ -0,0 +1,281 @@
+/**
+ * Command Context Tests
+ *
+ * Tests for config-driven command context resolution,
+ * classification, caching, and auto-learn.
+ *
+ * @see PRJ-298
+ */
+
+import { describe, expect, it } from 'bun:test'
+import { classifyCommand } from '../../agentic/command-classifier'
+import {
+  cacheClassification,
+  getCachedClassification,
+  loadCommandContextConfig,
+  resolveCommandContext,
+  resolveCommandContextFull,
+  trackClassification,
+} from '../../agentic/command-context'
+import type { CommandContextEntry } from '../../schemas/command-context'
+import type { Template } from '../../types'
+
+// =============================================================================
+// Config Loading
+// =============================================================================
+
+describe('Command Context Config', () => {
+  it('should load and validate the config file', async () => {
+    const config = await loadCommandContextConfig()
+
+    expect(config.version).toBe('1.0.0')
+    expect(config.commands).toBeDefined()
+    expect(config.commands['*']).toBeDefined()
+  })
+
+  it('should have wildcard entry with sensible defaults', async () => {
+    const config = await loadCommandContextConfig()
+    const wildcard = config.commands['*']
+
+    expect(wildcard.agents).toBe(true)
+    expect(wildcard.patterns).toBe(true)
+    expect(wildcard.checklist).toBe(false)
+    expect(wildcard.modules).toEqual([])
+  })
+
+  it('should have explicit entries for known commands', async () => {
+    const config = await loadCommandContextConfig()
+
+    expect(config.commands.task).toBeDefined()
+    expect(config.commands.ship).toBeDefined()
+    expect(config.commands.bug).toBeDefined()
+    expect(config.commands.done).toBeDefined()
+    expect(config.commands.sync).toBeDefined()
+  })
+})
+
+// =============================================================================
+// Config Resolution
+// =============================================================================
+
+describe('resolveCommandContext', () => {
+  it('should return explicit config for known commands', async () => {
+    const config = await loadCommandContextConfig()
+    const entry = resolveCommandContext(config, 'task')
+
+    expect(entry.modules).toContain('CLAUDE-intelligence.md')
+    expect(entry.modules).toContain('CLAUDE-storage.md')
+  })
+
+  it('should return wildcard for unknown commands', async () => {
+    const config = await loadCommandContextConfig()
+    const entry = resolveCommandContext(config, 'nonexistent-command')
+    const wildcard = config.commands['*']
+
+    expect(entry).toEqual(wildcard)
+  })
+
+  it('should give ship command patterns and checklist', async () => {
+    const config = await loadCommandContextConfig()
+    const entry = resolveCommandContext(config, 'ship')
+
+    expect(entry.patterns).toBe(true)
+    expect(entry.checklist).toBe(true)
+  })
+
+  it('should give done command checklist', async () => {
+    const config = await loadCommandContextConfig()
+    const entry = resolveCommandContext(config, 'done')
+
+    expect(entry.checklist).toBe(true)
+  })
+
+  it('should give sync command no context sections', async () => {
+    const config = await loadCommandContextConfig()
+    const entry = resolveCommandContext(config, 'sync')
+
+    expect(entry.agents).toBe(false)
+    expect(entry.patterns).toBe(false)
+    expect(entry.checklist).toBe(false)
+    expect(entry.modules).toEqual([])
+  })
+})
+
+// =============================================================================
+// Full Resolution with Classification
+// =============================================================================
+
+describe('resolveCommandContextFull', () => {
+  it('should return source=config for known commands', async () => {
+    const config = await loadCommandContextConfig()
+    const result = resolveCommandContextFull(config, 'bug')
+
+    expect(result.source).toBe('config')
+    expect(result.entry.agents).toBe(true)
+  })
+
+  it('should classify unknown commands from template', async () => {
+    const config = await loadCommandContextConfig()
+    const template: Template = {
+      frontmatter: {
+        name: 'p:deploy',
+        description: 'Deploy the application to production',
+        'allowed-tools': ['Bash', 'Read'],
+      },
+      content: 'Build and deploy the project. Verify deployment status.',
+    }
+
+    const result = resolveCommandContextFull(config, 'deploy', template)
+    expect(result.source).toBe('classified')
+  })
+
+  it('should return source=cache for previously classified commands', async () => {
+    const config = await loadCommandContextConfig()
+    const entry: CommandContextEntry = {
+      agents: true,
+      patterns: false,
+      checklist: false,
+      modules: [],
+    }
+    cacheClassification('cached-cmd', entry)
+
+    const result = resolveCommandContextFull(config, 'cached-cmd')
+    expect(result.source).toBe('cache')
+    expect(result.entry).toEqual(entry)
+  })
+
+  it('should return source=wildcard when no template provided for unknown command', async () => {
+    const config = await loadCommandContextConfig()
+    const result = resolveCommandContextFull(config, 'truly-unknown-no-template')
+
+    expect(result.source).toBe('wildcard')
+  })
+})
+
+// =============================================================================
+// Command Classifier
+// =============================================================================
+
+describe('classifyCommand', () => {
+  it('should classify code-modifying commands with Write tool', () => {
+    const template: Template = {
+      frontmatter: {
+        name: 'p:scaffold',
+        description: 'Scaffold a new component',
+        'allowed-tools': ['Write', 'Read', 'Bash'],
+      },
+      content: 'Create the component files and implement the structure.',
+    }
+
+    const result = classifyCommand('scaffold', template)
+    expect(result.agents).toBe(true)
+    expect(result.patterns).toBe(true)
+  })
+
+  it('should classify info commands as needing no context', () => {
+    const template: Template = {
+      frontmatter: {
+        name: 'p:stats',
+        description: 'Show project statistics',
+        'allowed-tools': ['Read'],
+      },
+      content: 'Display a summary of the project status and metrics.',
+    }
+
+    const result = classifyCommand('stats', template)
+    expect(result.agents).toBe(false)
+    expect(result.patterns).toBe(false)
+  })
+
+  it('should classify quality commands with checklists', () => {
+    const template: Template = {
+      frontmatter: {
+        name: 'p:verify',
+        description: 'Verify project integrity',
+        'allowed-tools': ['Read', 'Bash'],
+      },
+      content: 'Validate all tests pass and lint checks succeed before release.',
+    }
+
+    const result = classifyCommand('verify', template)
+    expect(result.checklist).toBe(true)
+  })
+})
+
+// =============================================================================
+// Classification Cache
+// =============================================================================
+
+describe('Classification Cache', () => {
+  it('should cache and retrieve classifications', () => {
+    const entry: CommandContextEntry = {
+      agents: true,
+      patterns: true,
+      checklist: false,
+      modules: ['test.md'],
+    }
+    cacheClassification('test-cache', entry)
+
+    const cached = getCachedClassification('test-cache')
+    expect(cached).toEqual(entry)
+  })
+
+  it('should return undefined for uncached commands', () => {
+    const cached = getCachedClassification('never-cached')
+    expect(cached).toBeUndefined()
+  })
+})
+
+// =============================================================================
+// Auto-Learn Tracking
+// =============================================================================
+
+describe('Auto-Learn (trackClassification)', () => {
+  it('should not trigger persist on first classification', () => {
+    const entry: CommandContextEntry = {
+      agents: true,
+      patterns: true,
+      checklist: false,
+      modules: [],
+    }
+    const shouldPersist = trackClassification('learn-test-1', entry)
+
+    expect(shouldPersist).toBe(false)
+  })
+
+  it('should trigger persist after threshold reached', () => {
+    const entry: CommandContextEntry = {
+      agents: false,
+      patterns: true,
+      checklist: true,
+      modules: [],
+    }
+
+    trackClassification('learn-test-2', entry) // 1
+    trackClassification('learn-test-2', entry) // 2
+    const shouldPersist = trackClassification('learn-test-2', entry) // 3
+
+    expect(shouldPersist).toBe(true)
+  })
+
+  it('should reset count when classification changes', () => {
+    const entry1: CommandContextEntry = {
+      agents: true,
+      patterns: true,
+      checklist: false,
+      modules: [],
+    }
+    const entry2: CommandContextEntry = {
+      agents: false,
+      patterns: false,
+      checklist: true,
+      modules: [],
+    }
+
+    trackClassification('learn-test-3', entry1) // 1
+    trackClassification('learn-test-3', entry1) // 2
+    const shouldPersist = trackClassification('learn-test-3', entry2) // reset to 1
+
+    expect(shouldPersist).toBe(false)
+  })
+})