@leanspec/ui 0.2.7-dev.20251128020042 → 0.2.8

package/.next/standalone/packages/ui/dist/standalone/specs/018-spec-validation/README.md

@@ -0,0 +1,303 @@
+---
+status: complete
+created: '2025-11-02'
+tags:
+  - quality
+  - validation
+  - cli
+  - first-principles
+  - v0.2.0
+priority: critical
+related:
+  - 043-official-launch-02
+  - 012-sub-spec-files
+created_at: '2025-11-02T00:00:00Z'
+updated_at: '2025-11-06T07:08:45.538Z'
+transitions:
+  - status: in-progress
+    at: '2025-11-05T13:35:26.669Z'
+  - status: complete
+    at: '2025-11-06T07:00:00.000Z'
+completed: '2025-11-06'
+---
+
+# Comprehensive Spec Validation
+
+> **Status**: ✅ Complete · **Priority**: Critical · **Created**: 2025-11-02 · **Tags**: quality, validation, cli, first-principles, v0.2.0
+
+## Overview
+
+Provide comprehensive validation tooling that checks specs for quality issues including structure, frontmatter, content, sequence conflicts, and **file corruption**.
+
+**Current State:**
+- ✅ `lean-spec check` exists - checks sequence conflicts only
+- ✅ `lean-spec validate` exists - comprehensive validation framework
+- ✅ **Line count validation** - warns at 300 lines, errors at 400+ lines
+- ✅ **Frontmatter validation** - enforces required fields and valid values
+- ✅ **Structure validation** - duplicate headers, required sections
+- ✅ **Corruption detection** - unclosed code blocks, formatting issues, duplicates
+- ❌ No way to detect stale specs (optional, future phase)
+- ❌ No auto-fix capability (optional, future phase)
+
+**Implementation Approach:**
+Both `lean-spec check` and `lean-spec validate` exist as separate commands:
+
+```bash
+# Current commands
+lean-spec check                    # Check for sequence conflicts
+lean-spec validate [specs...]      # Validate specs for quality issues
+lean-spec validate --max-lines 500 # Custom line limit
+
+# Planned enhancements
+lean-spec validate --frontmatter   # Frontmatter validation
+lean-spec validate --structure     # Structure validation
+lean-spec validate --corruption    # File corruption detection
+lean-spec validate --sub-specs     # Sub-spec validation (spec 012)
+lean-spec validate --all           # All validation rules
+```
+
+**Use Cases:**
+1. CI/CD validation (block PRs with invalid specs)
+2. Pre-commit hooks (comprehensive quality checks)
+3. Local validation before creating PR
+4. Detecting stale/abandoned specs
+5. Enforcing team conventions (required fields, valid values)
+6. Quality gates for spec completion
+7. **Detecting corrupted specs from failed edits**
+8. **Validating sub-spec organization per spec 012**
+
+**What Success Looks Like:**
+```bash
+$ lean-spec validate --all
+Validating specs...
+
+Line Count:
+  ✓ 043-official-launch-02 (387 lines)
+  ⚠ 048-spec-complexity-analysis (356 lines - approaching limit)
+  ✗ 018-spec-validation (455 lines - exceeds limit!)
+     → Consider splitting into sub-specs (see spec 012)
+
+Frontmatter:
+  ✗ 1 spec has invalid frontmatter:
+    - specs/043-official-launch-02/README.md
+      • Invalid status: "wip" (expected: planned, in-progress, complete, archived)
+
+Structure:
+  ✓ All specs have valid structure
+
+Sub-Specs:
+  ⚠ 1 spec has warnings:
+    - specs/018-spec-validation/README.md
+      ⚠ Sub-spec TESTING.md (421 lines) exceeds 400 line limit
+      ⚠ Orphaned sub-spec: DEPRECATED.md (not linked from README.md)
+
+Corruption:
+  ✓ All 25 spec(s) passed
+
+Results: 25 specs validated, 5 error(s), 6 warning(s)
+```
+
+**Recent Improvements (2025-11-05):**
+- ✅ Fixed duplicate detection bug: sliding window was reporting adjacent lines as duplicates
+- ✅ Improved thresholds: 8 lines / 200 chars (was 5/100) for better signal-to-noise
+- ✅ Code blocks now included in duplicate detection (catches actual copy-paste errors)
+- ✅ Removed JSON/YAML validation (code examples show invalid syntax)
+- ✅ Smarter formatting checks: exclude inline code and list markers
+- 📉 Result: Zero false positives, catches only real corruption
+
+## Design
+
+This spec has been split into focused sub-documents for clarity and maintainability.
+
+### Core Documents
+
+📋 **[VALIDATION-RULES.md](./VALIDATION-RULES.md)** - What gets validated
+- Frontmatter validation rules
+- Structure validation rules
+- Content validation rules
+- Corruption detection rules
+- Staleness detection rules
+- Auto-fix capabilities
+
+🔧 **[CLI-DESIGN.md](./CLI-DESIGN.md)** - Command interface
+- Command syntax and flags
+- Output formats (console, JSON)
+- Backwards compatibility strategy
+- Exit codes
+- CI/CD integration examples
+
+⚙️ **[CONFIGURATION.md](./CONFIGURATION.md)** - Configuration schema
+- Complete config options
+- Rule customization
+- Template-specific rules
+- Default configuration
+- Configuration examples
+
+📝 **[CONFIGURATION-EXAMPLES.md](./CONFIGURATION-EXAMPLES.md)** - Real-world configuration examples and use cases
+
+🗺️ **[IMPLEMENTATION.md](./IMPLEMENTATION.md)** - Implementation plan
+- 8-phase implementation plan
+- Priority and scope decisions
+- Launch strategy (v0.2.0 vs v0.3.0)
+- Dependencies and risks
+- Migration path
+
+✅ **[TESTING.md](./TESTING.md)** - Test strategy
+- Test categories and coverage
+- Integration tests
+- Performance tests
+- Manual testing checklist
+
+## Design Decision
+
+**Implementation Note:** The original design proposed expanding `lean-spec check` into a unified validation command. However, the implementation created a separate `lean-spec validate` command instead, keeping both commands focused:
+
+- **`lean-spec check`** - Fast sequence conflict detection
+- **`lean-spec validate`** - Comprehensive quality validation
+
+**Rationale for Separate Commands:**
+
+1. **Clear separation of concerns:** Sequence checking is fast and targeted; validation is comprehensive
+2. **Performance:** Users can run quick checks without full validation overhead
+3. **Backwards compatible:** Existing `lean-spec check` behavior unchanged
+4. **Incremental adoption:** Can add validation rules without affecting check command
+5. **Clearer intent:** `validate` explicitly signals quality checking
+
+**Trade-offs:**
+- Two commands to remember (but both are intuitive)
+- More CLI surface area
+- Better performance and flexibility
+
+## Evolution
+
+| Version | Commands Available |
+|---------|--------------------|
+| v0.1.0 | `lean-spec check` (sequence conflicts only) |
+| v0.2.0+ | `lean-spec check` (sequences) + `lean-spec validate` (line counts) |
+| v0.3.0+ | Both commands with comprehensive validation rules |
+
+## Launch Strategy
+
+**v0.2.0 Scope (Current):**
+- ✅ `lean-spec check` for sequence conflicts
+- ✅ `lean-spec validate` with basic framework and line count validation
+- ⏳ Expand validation rules in upcoming phases
+
+**v0.3.0 Scope:**
+- **MUST HAVE:** Framework + frontmatter + structure validation
+- **HIGHLY RECOMMENDED:** Corruption detection (addresses real pain point)
+- **SHOULD HAVE:** Auto-fix capability
+- **NICE TO HAVE:** Content and staleness validation
+
+**Post-v0.3.0:**
+- Advanced features based on user feedback
+- Custom validation rules
+- Performance optimizations
+
+## Implementation Status
+
+See [IMPLEMENTATION.md](./IMPLEMENTATION.md) for detailed plan.
+
+**Status:** ✅ COMPLETE for v0.2.0 launch
+
+**✅ Completed (Phases 1a, 1b, 2, 3, 3.5):**
+- ✅ Validation framework architecture
+- ✅ **Phase 1a:** `LineCountValidator` with warning/error thresholds (300 line warning, 400 line error)
+- ✅ **Phase 1a:** `lean-spec validate` command with `--max-lines` flag
+- ✅ **Phase 1b:** `FrontmatterValidator` for comprehensive frontmatter validation
+  - Required fields (status, created)
+  - Valid status/priority values  
+  - Date format validation (ISO 8601)
+  - Tags format validation
+- ✅ **Phase 2:** `StructureValidator` for spec structure validation
+  - H1 title presence
+  - Required sections (Overview, Design)
+  - Empty section detection (with subsection handling)
+  - Duplicate section header detection
+- ✅ **Phase 3:** `CorruptionValidator` for file corruption detection
+  - Unclosed code block detection (visible syntax highlighting issues)
+  - Unclosed markdown formatting (bold, italic) in actual content
+  - Duplicate content block detection with improved tuning (8 lines, 200 chars)
+  - **Code blocks included** in duplicate detection (catches real copy-paste errors)
+  - **Bug fix:** Fixed sliding window algorithm reporting adjacent lines
+  - **Removed JSON/YAML validation** (examples show invalid syntax intentionally)
+  - Code block exclusion for all formatting checks
+  - Inline code and list marker exclusion
+  - Duplicate detection tuning: 3/50 → 5/100 (reduced false positives)
+  - Removed noisy checks (JSON/YAML, table/list validation)
+  - **Bug fix:** Fixed sliding window algorithm causing adjacent line false positives
+  - **Threshold increase:** 5/100 → 8/200 (block size/min chars) for less sensitivity
+  - **Reverted code block filtering:** Now includes code blocks in duplicate detection
+- ✅ **Phase 3.5:** `SubSpecValidator` for sub-spec file validation
+  - Sub-spec naming conventions (uppercase .md files)
+  - README.md references all sub-specs (orphan detection)
+  - Line count validation per sub-spec file (<400 lines)
+  - Cross-document reference validation
+  - Found real issues: 3 sub-specs exceeding limits in 2 specs
+- ✅ **370 total tests passing** (16 sub-spec + 354 existing)
+- ✅ Documentation and CLI integration
+- ✅ Tested with real repository specs
+
+**📊 Current Validation Results (2025-11-05):**
+```bash
+$ lean-spec validate
+Results: 25 specs validated, 5 error(s), 6 warning(s)
+
+Errors found:
+- Line count: 3 specs exceed 400 lines (048, 046, 045)
+- Sub-specs: 3 sub-spec files exceed 400 lines (049: 2 files, 018: 1 file)
+- Corruption: ✅ 0 errors (bug fixed, thresholds tuned!)
+
+Warnings:
+- Line count: 6 specs between 300-400 lines (approaching limit)
+- Corruption: ✅ 0 warnings (improved from 31 false positives!)
+
+All specs: ✅ Frontmatter passed, ✅ Structure passed, ✅ Sub-spec validation working
+```
+
+**✨ v0.2.0 Scope Complete:**
+Core validation is complete and working! Phases 1-3.5 deliver the essential quality checks including sub-spec validation. Additional phases (content validation, staleness detection, auto-fix) deferred to v0.3.0 based on user feedback.
+
+## Quick Links
+
+- **Validation Details:** [VALIDATION-RULES.md](./VALIDATION-RULES.md)
+- **CLI Reference:** [CLI-DESIGN.md](./CLI-DESIGN.md)
+- **Configuration:** [CONFIGURATION.md](./CONFIGURATION.md)
+- **Implementation:** [IMPLEMENTATION.md](./IMPLEMENTATION.md)
+- **Testing:** [TESTING.md](./TESTING.md)
+
+## Notes
+
+**Why This Matters:**
+
+This addresses real pain points we've experienced:
+- Spec corruption from failed AI edits
+- Invalid frontmatter causing issues
+- No way to enforce quality standards
+- Manual validation is time-consuming
+
+**Performance Goals:**
+- < 1s for 100 specs
+- Parallel checking
+- Incremental mode for auto-check
+- Caching for repeated checks
+
+**Integration:**
+```bash
+# CI/CD - Quick sequence check
+lean-spec check
+
+# CI/CD - Comprehensive validation
+lean-spec validate --all --format=json
+
+# Pre-commit hook - Fast validation
+lean-spec validate --max-lines 400
+
+# Manual comprehensive check
+lean-spec validate --all --fix
+```
+
+**References:**
+- Markdownlint: Markdown linting tool (inspiration)
+- JSON Schema: Validation schema standard
+- YAML Lint: YAML validation patterns

package/.next/standalone/packages/ui/dist/standalone/specs/018-spec-validation/TESTING.md

@@ -0,0 +1,368 @@
+# Testing Strategy
+
+Comprehensive test plan for the `lean-spec validate` command.
+
+**Note:** This spec originally proposed expanding `lean-spec check`, but the implementation created `lean-spec validate` as a separate command.
+
+## Test Categories
+
+### 1. Frontmatter Validation Tests ✅
+
+**Status: Phase 1b Complete - 27 tests passing**
+
+**Missing Required Fields:**
+- [x] Detects missing `status` field ✅
+- [x] Detects missing `created` field ✅
+- [x] Passes when all required fields present ✅
+- [x] Detects when no frontmatter exists ✅
+- [ ] Detects missing custom required fields (if configured) - future
+
+**Invalid Status Values:**
+- [x] Detects invalid status "wip" ✅
+- [x] Detects invalid status values (any non-standard) ✅
+- [x] Passes valid status "planned" ✅
+- [x] Passes valid status "in-progress" ✅
+- [x] Passes valid status "complete" ✅
+- [x] Passes valid status "archived" ✅
+- [x] Coerces non-string status to string and validates ✅
+
+**Invalid Priority Values:**
+- [x] Detects invalid priority "urgent" ✅
+- [x] Detects invalid priority values (any non-standard) ✅
+- [x] Passes valid priority "low" ✅
+- [x] Passes valid priority "medium" ✅
+- [x] Passes valid priority "high" ✅
+- [x] Passes valid priority "critical" ✅
+- [x] Passes when priority is optional and missing ✅
+- [x] Coerces non-string priority to string and validates ✅
+
+**Date Format Validation:**
+- [x] Detects invalid date formats (11/05/2025, etc.) ✅
+- [x] Detects invalid date values (2025-99-99) ✅
+- [x] Passes valid date "2025-11-04" ✅
+- [x] Passes valid ISO timestamps (with timezone, milliseconds) ✅
+- [x] Warns for invalid optional date fields (e.g., updated) ✅
+- [x] Fails for non-string/non-date created field ✅
+
+**Tags Validation:**
+- [x] Detects tags as string instead of array ✅
+- [x] Accepts tags with various types (YAML parsing) ✅
+- [x] Passes valid tags array ✅
+- [x] Passes empty tags array ✅
+
+**YAML Parsing:**
+- [x] Detects malformed YAML frontmatter ✅
+
+**Custom Configuration:**
+- [x] Respects custom valid statuses (configurable) ✅
+- [x] Validator metadata (name, description) ✅
+
+### 2. Structure Validation Tests
+
+**File Requirements:**
+- [ ] Detects missing README.md
+- [ ] Detects invalid YAML frontmatter syntax
+- [ ] Detects missing title (H1 heading)
+- [ ] Passes spec with valid structure
+
+**Section Requirements:**
+- [ ] Detects missing required section "Overview"
+- [ ] Detects missing required section "Design"
+- [ ] Detects missing required section "Plan"
+- [ ] Passes when all required sections present
+- [ ] Template-specific section validation
+
+**Empty Sections:**
+- [ ] Detects empty required sections (when configured)
+- [ ] Passes sections with only comments (if allowed)
+- [ ] Passes sections with content
+
+**Duplicate Sections:**
+- [ ] Detects duplicate "## Plan" sections
+- [ ] Detects duplicate "## Test" sections
+- [ ] Allows same section name at different levels
+- [ ] Passes unique sections
+
+### 3. Corruption Detection Tests
+
+**Duplicate Sections:**
+- [ ] Detects exact duplicate sections
+- [ ] Detects sections with same name at same level
+- [ ] Reports line numbers of duplicates
+- [ ] Passes specs with unique sections
+
+**Code Block Validation:**
+- [ ] Detects unclosed code block (missing closing ```)
+- [ ] Detects extra closing ``` without opening
+- [ ] Detects mismatched code fence lengths
+- [ ] Passes properly closed code blocks
+
+**JSON/YAML Validation:**
+- [ ] Detects incomplete JSON in code blocks
+- [ ] Detects invalid JSON syntax
+- [ ] Detects incomplete YAML in frontmatter
+- [ ] Detects invalid YAML syntax
+- [ ] Passes valid JSON blocks
+- [ ] Passes valid YAML blocks
+
+**Content Fragments:**
+- [ ] Detects duplicated content blocks
+- [ ] Detects partial duplicates (merge artifacts)
+- [ ] Detects remnants from failed edits
+- [ ] Passes unique content
+
+**Markdown Structure:**
+- [ ] Detects malformed lists
+- [ ] Detects broken tables
+- [ ] Detects unclosed formatting (bold, italic)
+- [ ] Passes valid markdown
+
+### 4. Content Validation Tests
+
+**Minimum Length:**
+- [ ] Detects specs below minimum length
+- [ ] Passes specs above minimum length
+- [ ] Respects configured minimum
+
+**TODO/FIXME Detection:**
+- [ ] Detects TODO in complete specs
+- [ ] Detects FIXME in complete specs
+- [ ] Allows TODO in non-complete specs
+- [ ] Passes complete specs without TODO
+
+**Link Validation:**
+- [ ] Detects broken internal links
+- [ ] Detects links to non-existent specs
+- [ ] Passes valid internal links
+- [ ] Passes external links (not validated)
+
+**Placeholder Text:**
+- [ ] Detects common placeholder patterns
+- [ ] Passes real content
+
+### 5. Staleness Tests
+
+**In-Progress Specs:**
+- [ ] Warns on in-progress spec > 30 days old
+- [ ] No warning for recent in-progress specs
+- [ ] Respects configured threshold
+
+**No Updates:**
+- [ ] Warns on spec with no updates > 90 days
+- [ ] No warning for recently updated specs
+- [ ] Uses git history when available
+- [ ] Falls back to file mtime
+
+**Planned Specs:**
+- [ ] Warns on planned spec > 60 days old
+- [ ] No warning for recent planned specs
+
+**Archived Specs:**
+- [ ] No staleness warnings for archived specs
+
+### 6. Auto-Fix Tests
+
+**Frontmatter Fixes:**
+- [ ] Adds missing `status` field with default
+- [ ] Adds missing `created` field with current date
+- [ ] Formats dates to ISO 8601
+- [ ] Sorts frontmatter fields alphabetically
+- [ ] Updates visual badges from frontmatter
+
+**Structure Fixes:**
+- [ ] Adds missing required sections as comments
+- [ ] Removes duplicate sections (keeps first)
+
+**Corruption Fixes:**
+- [ ] Closes unclosed code blocks
+- [ ] Removes exact duplicate sections
+
+**Non-Fixable Issues:**
+- [ ] Reports invalid status (requires manual fix)
+- [ ] Reports empty sections (requires content)
+- [ ] Reports broken links (requires investigation)
+
+**Fix Reporting:**
+- [ ] Reports what was fixed
+- [ ] Reports what couldn't be fixed
+- [ ] Doesn't break valid specs
+- [ ] Maintains spec readability
+
+### 7. Integration Tests
+
+**Multiple Specs:**
+- [ ] Validates all specs in project
+- [ ] Reports aggregate results
+- [ ] Handles large number of specs (100+)
+
+**Filtering:**
+- [ ] Filters by status work correctly
+- [ ] Filters by tag work correctly
+- [ ] Filters by priority work correctly
+- [ ] Path pattern filtering works
+
+**Output Formats:**
+- [ ] Console output is readable
+- [ ] JSON output is valid
+- [ ] Quiet mode produces minimal output
+- [ ] Verbose mode includes details
+
+**Exit Codes:**
+- [ ] Returns 0 when all checks pass
+- [ ] Returns 1 when errors found
+- [ ] Returns 2 when warnings in strict mode
+- [ ] Returns 3 on command error
+
+**Configuration:**
+- [ ] Respects custom config
+- [ ] Falls back to defaults
+- [ ] Command-line flags override config
+- [ ] Template-specific rules work
+
+**Backwards Compatibility:**
+- [ ] `lean-spec check` (sequence conflicts) behavior unchanged
+- [ ] `lean-spec validate` doesn't interfere with check command
+- [ ] Exit codes follow standard pattern
+- [ ] Output format is consistent
+
+### 8. Performance Tests
+
+**Speed:**
+- [ ] Validates 10 specs in < 100ms
+- [ ] Validates 100 specs in < 1s
+- [ ] Parallel validation improves performance
+
+**Memory:**
+- [ ] Memory usage stays reasonable
+- [ ] No memory leaks with many specs
+
+**Caching:**
+- [ ] Cached results speed up repeated checks
+- [ ] Cache invalidation works correctly
+
+### 9. Edge Cases
+
+**Empty Specs:**
+- [ ] Handles specs with only frontmatter
+- [ ] Handles completely empty files
+
+**Large Specs:**
+- [ ] Handles specs > 1000 lines
+- [ ] Handles specs with > 100 sections
+- [ ] Performance doesn't degrade
+
+**Special Characters:**
+- [ ] Handles Unicode in frontmatter
+- [ ] Handles special characters in content
+- [ ] Handles emoji in titles
+
+**Symlinks:**
+- [ ] Follows symlinks correctly
+- [ ] Detects circular symlinks
+
+**Sub-Spec Files:**
+- [ ] Validates specs with sub-files
+- [ ] Checks README.md in multi-file specs
+- [ ] Doesn't validate sub-files as specs
+
+## Test Organization
+
+```
+src/
+  commands/
+    check.test.ts              # Command-level tests
+    validators/
+      frontmatter.test.ts      # Frontmatter validator tests
+      structure.test.ts        # Structure validator tests
+      corruption.test.ts       # Corruption detector tests
+      content.test.ts          # Content validator tests
+      staleness.test.ts        # Staleness detector tests
+  integration/
+    check-integration.test.ts  # End-to-end tests
+```
+
+## Test Data
+
+```
+test-fixtures/
+  valid/
+    001-valid-spec/
+      README.md                # Completely valid spec
+  invalid-frontmatter/
+    002-missing-status/
+      README.md                # Missing status field
+    003-invalid-status/
+      README.md                # Invalid status value
+  invalid-structure/
+    004-missing-sections/
+      README.md                # Missing required sections
+    005-duplicate-sections/
+      README.md                # Duplicate section headers
+  corrupted/
+    006-unclosed-code-block/
+      README.md                # Unclosed code block
+    007-invalid-json/
+      README.md                # Malformed JSON in code block
+    008-duplicate-content/
+      README.md                # Duplicated content
+  stale/
+    009-old-in-progress/
+      README.md                # Old in-progress spec
+```
+
+## Continuous Testing
+
+**Pre-Commit:**
+```bash
+npm test  # Run all tests
+```
+
+**CI/CD:**
+```yaml
+- name: Run tests
+  run: npm test
+
+- name: Check coverage
+  run: npm run test:coverage
+```
+
+**Test Coverage Goals:**
+- Overall: >80%
+- Validators: >90%
+- Critical paths: 100%
+
+## Manual Testing Checklist
+
+**Before Release:**
+- [ ] Test on real project (this repo)
+- [ ] Test backwards compatibility
+- [ ] Test performance with 100+ specs
+- [ ] Test all output formats
+- [ ] Test all command-line flags
+- [ ] Test configuration options
+- [ ] Test auto-fix on corrupted specs
+- [ ] Test CI/CD integration
+- [ ] Test pre-commit hook
+
+**Dogfooding:**
+- [ ] Use comprehensive checking on lean-spec itself
+- [ ] Fix any issues found
+- [ ] Verify no false positives
+- [ ] Collect user feedback
+
+## Test Automation
+
+**Snapshot Testing:**
+- Output format snapshots
+- JSON structure snapshots
+- Error message snapshots
+
+**Property-Based Testing:**
+- Generate random specs
+- Verify validators don't crash
+- Verify auto-fix doesn't corrupt
+
+**Regression Testing:**
+- Save examples of past bugs
+- Ensure fixes remain effective
+- Prevent regressions