@boshu2/vibe-check 2.2.1 → 2.4.0

package/.agents/plans/2025-12-28-ai-safety-integration-plan.md

@@ -0,0 +1,326 @@
+---
+date: 2025-12-28
+type: Plan
+topic: "AI Safety Detection Module for vibe-check"
+research: ".agents/research/2025-12-28-ai-platform-security-integration.md"
+tags: [plan, ai-safety, llm-validation, security, vibe-check]
+status: READY_FOR_APPROVAL
+---
+
+# Plan: AI Safety Detection Module
+
+**Created:** 2025-12-28
+**Research:** `.agents/research/2025-12-28-ai-platform-security-integration.md`
+**Vibe Level:** L3 (Multiple files, following existing patterns)
+
+---
+
+## Overview
+
+Add a new `src/ai-safety/` module to vibe-check that detects LLM-specific antipatterns in commit history: **secret leakage**, **scope violations**, **contract drift**, and **token spirals**. This extends the existing inner-loop failure detection with patterns ported from ai-platform's production security stack.
+
+---
+
+## Approach
+
+Port security validation patterns from `ai-platform/tests/agents/test_agent_security.py` and adapt them for git-based analysis. Follow the existing `src/inner-loop/` module structure exactly.
+
+**Why this approach:**
+1. **Proven patterns:** ai-platform patterns are battle-tested for classified environments
+2. **Non-breaking:** Extends existing architecture without replacing anything
+3. **Familiar structure:** Same module organization as inner-loop
+4. **Zero new dependencies:** Pure TypeScript, uses existing git analysis
+
+---
+
+## Features
+
+### Feature 1: AI Safety Types and Orchestrator
+
+**Priority:** P0 (Foundation - must come first)
+**Type:** feature
+**Depends On:** None
+
+**Acceptance Criteria:**
+- [ ] `src/ai-safety/types.ts` defines all interfaces
+- [ ] `src/ai-safety/index.ts` exports orchestrator function
+- [ ] `AISafetyAnalysis` interface matches inner-loop pattern
+- [ ] Config supports enable/disable per detector
+- [ ] Compiles without errors
+
+**Files Affected:**
+- `src/ai-safety/types.ts` - New file: Type definitions
+- `src/ai-safety/index.ts` - New file: Orchestrator + exports
+
+**Test Strategy:** `npm run build` passes, types are importable
+
+---
+
+### Feature 2: Secret Leakage Detector
+
+**Priority:** P1 (Highest value, immediate utility)
+**Type:** feature
+**Depends On:** Feature 1
+
+**Acceptance Criteria:**
+- [ ] Detects OpenAI keys (`sk-[a-zA-Z0-9]{48}`)
+- [ ] Detects GitHub PATs (`ghp_[a-zA-Z0-9]{36}`)
+- [ ] Detects GitLab PATs (`glpat-[a-zA-Z0-9]{20}`)
+- [ ] Detects AWS keys (`AKIA[0-9A-Z]{16}`)
+- [ ] Detects Slack tokens (`xox[baprs]-*`)
+- [ ] Detects generic patterns (password=, secret=)
+- [ ] Reports file, line context, and pattern matched
+- [ ] Critical severity by default
+
+**Files Affected:**
+- `src/ai-safety/secret-leakage.ts` - New file: Secret detection logic
+- `src/ai-safety/index.ts` - Wire into orchestrator
+
+**Test Strategy:**
+1. Create test with fake secrets in commit diffs
+2. Verify detection of each pattern
+3. Verify no false positives on similar-looking strings
+
+---
+
+### Feature 3: Scope Violation Detector
+
+**Priority:** P2 (High value, needs config)
+**Type:** feature
+**Depends On:** Feature 1
+
+**Acceptance Criteria:**
+- [ ] Detects commits touching files outside declared scope
+- [ ] Scope defined via config file (`.vibe-check/scope.yaml`) or CLI
+- [ ] Supports glob patterns for scope definition
+- [ ] Reports which files violated scope
+- [ ] Warning severity by default
+
+**Files Affected:**
+- `src/ai-safety/scope-violation.ts` - New file: Scope check logic
+- `src/ai-safety/index.ts` - Wire into orchestrator
+- `src/types.ts` - Add AISafetyConfig to main types
+
+**Test Strategy:**
+1. Define scope as `src/feature/**`
+2. Create commit touching `src/other/file.ts`
+3. Verify scope violation detected
+
+---
+
+### Feature 4: Contract Drift Detector
+
+**Priority:** P2 (Medium value, pattern analysis)
+**Type:** feature
+**Depends On:** Feature 1
+
+**Acceptance Criteria:**
+- [ ] Tracks commit message format compliance over time
+- [ ] Detects degradation from conventional commits format
+- [ ] Tracks entropy (message quality declining)
+- [ ] Reports drift percentage and trend direction
+- [ ] Warning severity when >30% drift from baseline
+
+**Files Affected:**
+- `src/ai-safety/contract-drift.ts` - New file: Format drift logic
+- `src/ai-safety/index.ts` - Wire into orchestrator
+
+**Test Strategy:**
+1. Provide commits with good format then degrading format
+2. Verify drift percentage calculated
+3. Verify warning triggered at threshold
+
+---
+
+### Feature 5: Token Spiral Estimator
+
+**Priority:** P3 (Lower value, estimation-based)
+**Type:** feature
+**Depends On:** Feature 1
+
+**Acceptance Criteria:**
+- [ ] Estimates token usage from commit size
+- [ ] Uses heuristic: ~4 chars per token
+- [ ] Tracks cumulative session tokens
+- [ ] Detects explosion patterns (>2x baseline)
+- [ ] Info severity (advisory, not critical)
+
+**Files Affected:**
+- `src/ai-safety/token-spiral.ts` - New file: Token estimation logic
+- `src/ai-safety/index.ts` - Wire into orchestrator
+
+**Test Strategy:**
+1. Provide small commits then huge commit
+2. Verify explosion detected
+3. Verify reasonable token estimates
+
+---
+
+### Feature 6: Integration with Session End
+
+**Priority:** P1 (Makes feature visible to users)
+**Type:** feature
+**Depends On:** Features 1-2 (minimum)
+
+**Acceptance Criteria:**
+- [ ] `session end` JSON output includes `ai_safety` section
+- [ ] Terminal output shows AI safety warnings
+- [ ] Critical findings show in red
+- [ ] Recommendations generated for each finding
+
+**Files Affected:**
+- `src/commands/session.ts` - Add AI safety analysis call
+- `src/output/terminal.ts` - Format AI safety for terminal
+- `src/output/json.ts` - Format AI safety for JSON
+
+**Test Strategy:** `vibe-check session end --format json` includes ai_safety section
+
+---
+
+### Feature 7: Integration with Watch Command
+
+**Priority:** P2 (Real-time value)
+**Type:** feature
+**Depends On:** Feature 2 (secret detection)
+
+**Acceptance Criteria:**
+- [ ] Watch mode alerts on secret detection immediately
+- [ ] Uses red/bold for critical findings
+- [ ] Shows file and pattern matched
+- [ ] Rate-limited (don't spam alerts)
+
+**Files Affected:**
+- `src/commands/watch.ts` - Add secret detection to watch loop
+
+**Test Strategy:** Run watch, commit file with fake secret, see immediate alert
+
+---
+
+### Feature 8: Tests and Documentation
+
+**Priority:** P2 (Quality assurance)
+**Type:** task
+**Depends On:** Features 1-5
+
+**Acceptance Criteria:**
+- [ ] Unit tests for each detector
+- [ ] Integration test for orchestrator
+- [ ] CLAUDE.md updated with ai-safety commands
+- [ ] README.md mentions AI safety features
+
+**Files Affected:**
+- `tests/ai-safety.test.ts` - New file: Test suite
+- `CLAUDE.md` - Update documentation
+- `README.md` - Feature description
+
+**Test Strategy:** `npm test` passes with >80% coverage on new module
+
+---
+
+## Implementation Order
+
+| Step | Feature | Depends On | Validation |
+|------|---------|------------|------------|
+| 1 | Types and Orchestrator | - | `npm run build` passes |
+| 2 | Secret Leakage | Step 1 | Test with fake secrets |
+| 3 | Session Integration | Steps 1-2 | `session end` shows findings |
+| 4 | Scope Violation | Step 1 | Test with scope config |
+| 5 | Contract Drift | Step 1 | Test with degrading commits |
+| 6 | Token Spiral | Step 1 | Test with varying sizes |
+| 7 | Watch Integration | Step 2 | Real-time alert test |
+| 8 | Tests & Docs | Steps 1-6 | `npm test` passes |
+
+---
+
+## Beads Issues to Create
+
+After approval, these issues will be created:
+
+| ID | Title | Type | Priority | Depends On |
+|----|-------|------|----------|------------|
+| TBD | Epic: AI Safety Detection Module | epic | P1 | - |
+| TBD | AI Safety types and orchestrator | feature | P0 | Epic |
+| TBD | Secret leakage detector | feature | P1 | Types |
+| TBD | Session end AI safety integration | feature | P1 | Secret detector |
+| TBD | Scope violation detector | feature | P2 | Types |
+| TBD | Contract drift detector | feature | P2 | Types |
+| TBD | Token spiral estimator | feature | P3 | Types |
+| TBD | Watch command AI safety integration | feature | P2 | Secret detector |
+| TBD | Tests and documentation | task | P2 | All features |
+
+---
+
+## File Structure After Implementation
+
+```
+src/ai-safety/
+├── index.ts              # Orchestrator: analyzeAISafety()
+├── types.ts              # Interfaces: AISafetyAnalysis, configs
+├── secret-leakage.ts     # Regex-based secret detection
+├── scope-violation.ts    # Glob-based scope checking
+├── contract-drift.ts     # Commit format degradation
+└── token-spiral.ts       # Token usage estimation
+
+tests/ai-safety.test.ts   # Full test suite
+```
+
+---
+
+## Key Interfaces (Preview)
+
+```typescript
+// src/ai-safety/types.ts
+export interface AISafetyAnalysis {
+  secretLeakage: SecretLeakageResult;
+  scopeViolations: ScopeViolationResult;
+  contractDrift: ContractDriftResult;
+  tokenSpiral: TokenSpiralResult;
+  summary: {
+    totalIssues: number;
+    criticalIssues: number;
+    warningIssues: number;
+    overallHealth: 'healthy' | 'warning' | 'critical';
+  };
+  recommendations: string[];
+}
+
+export interface SecretLeakageResult {
+  detected: boolean;
+  findings: SecretFinding[];
+  message: string;
+}
+
+export interface SecretFinding {
+  commitHash: string;
+  file: string;
+  pattern: string;
+  lineContext: string;
+  severity: 'critical' | 'warning';
+}
+```
+
+---
+
+## Rollback Procedure
+
+If something goes wrong:
+
+1. **Module not loading:** Revert `src/ai-safety/` directory
+2. **Session command broken:** Revert `src/commands/session.ts` changes
+3. **Watch command broken:** Revert `src/commands/watch.ts` changes
+4. **Build fails:** Check `src/ai-safety/index.ts` exports
+
+All changes are additive - existing functionality is not modified, only extended.
+
+---
+
+## Next Steps
+
+1. **Review and approve this plan**
+2. **Create beads issues** (commands below)
+3. **`bd ready`** to see unblocked issues
+4. **`/implement`** to execute first issue
+
+---
+
+**Output:** .agents/plans/2025-12-28-ai-safety-integration-plan.md

package/.agents/plans/2025-12-29-complexity-driver-plan.md

@@ -0,0 +1,225 @@
+---
+date: 2025-12-29
+type: Plan
+topic: "Add Complexity Driver Architecture"
+research: ".agents/research/2025-12-29-complexity-driver-architecture.md"
+tags: [plan, architecture, complexity]
+status: READY_FOR_IMPLEMENTATION
+---
+
+# Plan: Complexity Driver Architecture
+
+**Created:** 2025-12-29
+**Research:** [Complexity Driver Architecture Research](.agents/research/2025-12-29-complexity-driver-architecture.md)
+
+---
+
+## Overview
+
+Add a "driver" system that allows language-specific complexity tools (radon, gocyclo, complexity-report) to feed metrics into vibe-check's modularity analyzer. vibe-check remains language-agnostic; drivers normalize tool output to a standard schema.
+
+---
+
+## Features
+
+### Feature 1: Define Complexity Schema and Loader
+
+**Priority:** P1
+**Type:** feature
+
+**Description:** Create TypeScript types for the standard complexity schema and a loader function.
+
+**Files to Create:**
+- `src/analyzers/complexity.ts`
+
+**Implementation:**
+```typescript
+// Types
+export interface ComplexityReport { ... }
+export interface FunctionComplexity { ... }
+
+// Loader
+export function loadComplexityData(rootDir: string): ComplexityReport | null
+
+// Helper
+export function getFileComplexity(data: ComplexityReport, file: string)
+```
+
+**Acceptance Criteria:**
+- [ ] Types exported and documented
+- [ ] Loader returns null if file doesn't exist
+- [ ] Loader handles malformed JSON gracefully
+- [ ] Helper returns null for unknown files
+
+---
+
+### Feature 2: Integrate Complexity into Modularity Scoring
+
+**Priority:** P1
+**Type:** feature
+**Depends On:** Feature 1
+
+**Description:** Modify `analyzeModularity()` to optionally accept complexity data and incorporate it into scoring.
+
+**Files to Modify:**
+- `src/analyzers/modularity.ts`
+
+**Changes:**
+1. Add optional `complexityData` parameter to `analyzeModularity()`
+2. Pass to `calculateScore()`
+3. Add scoring logic:
+   - Grade A/B: +2 bonus
+   - Grade C: -1, flag `moderate-complexity`
+   - Grade D/E: -2, flag `high-complexity`
+   - Grade F: -4, flag `extreme-complexity`
+4. Add new flags to `ModularityFlag` type
+
+**Acceptance Criteria:**
+- [ ] Backwards compatible (works without complexity data)
+- [ ] Flags appear in output when complexity issues detected
+- [ ] Score adjustments match spec
+
+---
+
+### Feature 3: Create Python Driver
+
+**Priority:** P1
+**Type:** feature
+
+**Description:** Shell script that wraps `radon` and outputs standard schema JSON.
+
+**Files to Create:**
+- `drivers/python.sh`
+
+**Implementation:**
+- Check radon installed
+- Run `radon cc <dir> -j`
+- Transform with `jq` to standard schema
+- Output to stdout
+
+**Acceptance Criteria:**
+- [ ] Exits 0 on success
+- [ ] Exits 1 with error JSON if radon not installed
+- [ ] Output validates against schema
+- [ ] Handles empty directories gracefully
+
+---
+
+### Feature 4: Create JavaScript/TypeScript Driver
+
+**Priority:** P2
+**Type:** feature
+
+**Description:** Shell script that wraps `complexity-report` for JS/TS analysis.
+
+**Files to Create:**
+- `drivers/javascript.sh`
+
+**Acceptance Criteria:**
+- [ ] Works with complexity-report npm package
+- [ ] Output validates against schema
+- [ ] Handles mixed .js/.ts codebases
+
+---
+
+### Feature 5: Add CLI Integration
+
+**Priority:** P2
+**Type:** feature
+**Depends On:** Features 1-3
+
+**Description:** Add CLI flags to run drivers and consume their output.
+
+**Files to Modify:**
+- `src/cli.ts`
+- `src/commands/analyze.ts`
+
+**New CLI Options:**
+```bash
+# Run driver before analysis
+vibe-check --with-complexity python
+
+# Use existing complexity file
+vibe-check --complexity-file .vibe-check/complexity.json
+
+# Driver subcommand
+vibe-check driver python ./src
+```
+
+**Acceptance Criteria:**
+- [ ] `--with-complexity` runs driver, saves to `.vibe-check/complexity.json`
+- [ ] `--complexity-file` uses existing file
+- [ ] `driver` subcommand outputs to stdout
+- [ ] Help text updated
+
+---
+
+### Feature 6: Documentation
+
+**Priority:** P2
+**Type:** docs
+**Depends On:** Features 1-5
+
+**Description:** Document the driver architecture in README and create driver authoring guide.
+
+**Files to Modify/Create:**
+- `README.md` - Add "Complexity Analysis" section
+- `docs/drivers.md` - Driver authoring guide
+
+**Acceptance Criteria:**
+- [ ] README shows basic usage
+- [ ] Driver guide explains schema and contract
+- [ ] Examples for Python and JS
+
+---
+
+## Implementation Order
+
+| Step | Feature | Validation |
+|------|---------|------------|
+| 1 | Schema + Loader | Unit tests pass |
+| 2 | Modularity Integration | `npm test` passes |
+| 3 | Python Driver | Manual test with radon |
+| 4 | CLI Integration | `vibe-check --with-complexity python` works |
+| 5 | JS Driver | Manual test with complexity-report |
+| 6 | Documentation | README updated |
+
+---
+
+## Beads Issues
+
+| ID | Title | Priority |
+|----|-------|----------|
+| TBD | Add complexity schema and loader | P1 |
+| TBD | Integrate complexity into modularity scoring | P1 |
+| TBD | Create Python driver (radon wrapper) | P1 |
+| TBD | Add CLI complexity flags | P2 |
+| TBD | Create JavaScript driver | P2 |
+| TBD | Document driver architecture | P2 |
+
+---
+
+## Test Strategy
+
+**Unit Tests:**
+- `tests/analyzers/complexity.test.ts` - Schema validation, loader edge cases
+- `tests/analyzers/modularity.test.ts` - Scoring with/without complexity data
+
+**Integration Tests:**
+- Run Python driver on known codebase, verify output
+- Run full vibe-check with complexity, verify report includes grades
+
+**Manual Testing:**
+- Test on ai-platform repo (known sync.py has F-grade functions)
+
+---
+
+## Next Steps
+
+1. `bd create` to create beads issues
+2. `/implement` to start with Feature 1
+3. Build incrementally, test each feature
+
+---
+
+**Output:** .agents/plans/2025-12-29-complexity-driver-plan.md