@leanspec/ui 0.2.7-dev.20251126100750 → 0.2.7

package/.next/standalone/packages/ui/dist/standalone/specs/018-spec-validation/TESTING.md

@@ -0,0 +1,368 @@
+# Testing Strategy
+
+Comprehensive test plan for the `lean-spec validate` command.
+
+**Note:** This spec originally proposed expanding `lean-spec check`, but the implementation created `lean-spec validate` as a separate command.
+
+## Test Categories
+
+### 1. Frontmatter Validation Tests ✅
+
+**Status: Phase 1b Complete - 27 tests passing**
+
+**Missing Required Fields:**
+- [x] Detects missing `status` field ✅
+- [x] Detects missing `created` field ✅
+- [x] Passes when all required fields present ✅
+- [x] Detects when no frontmatter exists ✅
+- [ ] Detects missing custom required fields (if configured) - future
+
+**Invalid Status Values:**
+- [x] Detects invalid status "wip" ✅
+- [x] Detects invalid status values (any non-standard) ✅
+- [x] Passes valid status "planned" ✅
+- [x] Passes valid status "in-progress" ✅
+- [x] Passes valid status "complete" ✅
+- [x] Passes valid status "archived" ✅
+- [x] Coerces non-string status to string and validates ✅
+
+**Invalid Priority Values:**
+- [x] Detects invalid priority "urgent" ✅
+- [x] Detects invalid priority values (any non-standard) ✅
+- [x] Passes valid priority "low" ✅
+- [x] Passes valid priority "medium" ✅
+- [x] Passes valid priority "high" ✅
+- [x] Passes valid priority "critical" ✅
+- [x] Passes when priority is optional and missing ✅
+- [x] Coerces non-string priority to string and validates ✅
+
+**Date Format Validation:**
+- [x] Detects invalid date formats (11/05/2025, etc.) ✅
+- [x] Detects invalid date values (2025-99-99) ✅
+- [x] Passes valid date "2025-11-04" ✅
+- [x] Passes valid ISO timestamps (with timezone, milliseconds) ✅
+- [x] Warns for invalid optional date fields (e.g., updated) ✅
+- [x] Fails for non-string/non-date created field ✅
+
+**Tags Validation:**
+- [x] Detects tags as string instead of array ✅
+- [x] Accepts tags with various types (YAML parsing) ✅
+- [x] Passes valid tags array ✅
+- [x] Passes empty tags array ✅
+
+**YAML Parsing:**
+- [x] Detects malformed YAML frontmatter ✅
+
+**Custom Configuration:**
+- [x] Respects custom valid statuses (configurable) ✅
+- [x] Validator metadata (name, description) ✅
+
+### 2. Structure Validation Tests
+
+**File Requirements:**
+- [ ] Detects missing README.md
+- [ ] Detects invalid YAML frontmatter syntax
+- [ ] Detects missing title (H1 heading)
+- [ ] Passes spec with valid structure
+
+**Section Requirements:**
+- [ ] Detects missing required section "Overview"
+- [ ] Detects missing required section "Design"
+- [ ] Detects missing required section "Plan"
+- [ ] Passes when all required sections present
+- [ ] Template-specific section validation
+
+**Empty Sections:**
+- [ ] Detects empty required sections (when configured)
+- [ ] Passes sections with only comments (if allowed)
+- [ ] Passes sections with content
+
+**Duplicate Sections:**
+- [ ] Detects duplicate "## Plan" sections
+- [ ] Detects duplicate "## Test" sections
+- [ ] Allows same section name at different levels
+- [ ] Passes unique sections
+
+### 3. Corruption Detection Tests
+
+**Duplicate Sections:**
+- [ ] Detects exact duplicate sections
+- [ ] Detects sections with same name at same level
+- [ ] Reports line numbers of duplicates
+- [ ] Passes specs with unique sections
+
+**Code Block Validation:**
+- [ ] Detects unclosed code block (missing closing ```)
+- [ ] Detects extra closing ``` without opening
+- [ ] Detects mismatched code fence lengths
+- [ ] Passes properly closed code blocks
+
+**JSON/YAML Validation:**
+- [ ] Detects incomplete JSON in code blocks
+- [ ] Detects invalid JSON syntax
+- [ ] Detects incomplete YAML in frontmatter
+- [ ] Detects invalid YAML syntax
+- [ ] Passes valid JSON blocks
+- [ ] Passes valid YAML blocks
+
+**Content Fragments:**
+- [ ] Detects duplicated content blocks
+- [ ] Detects partial duplicates (merge artifacts)
+- [ ] Detects remnants from failed edits
+- [ ] Passes unique content
+
+**Markdown Structure:**
+- [ ] Detects malformed lists
+- [ ] Detects broken tables
+- [ ] Detects unclosed formatting (bold, italic)
+- [ ] Passes valid markdown
+
+### 4. Content Validation Tests
+
+**Minimum Length:**
+- [ ] Detects specs below minimum length
+- [ ] Passes specs above minimum length
+- [ ] Respects configured minimum
+
+**TODO/FIXME Detection:**
+- [ ] Detects TODO in complete specs
+- [ ] Detects FIXME in complete specs
+- [ ] Allows TODO in non-complete specs
+- [ ] Passes complete specs without TODO
+
+**Link Validation:**
+- [ ] Detects broken internal links
+- [ ] Detects links to non-existent specs
+- [ ] Passes valid internal links
+- [ ] Passes external links (not validated)
+
+**Placeholder Text:**
+- [ ] Detects common placeholder patterns
+- [ ] Passes real content
+
+### 5. Staleness Tests
+
+**In-Progress Specs:**
+- [ ] Warns on in-progress spec > 30 days old
+- [ ] No warning for recent in-progress specs
+- [ ] Respects configured threshold
+
+**No Updates:**
+- [ ] Warns on spec with no updates > 90 days
+- [ ] No warning for recently updated specs
+- [ ] Uses git history when available
+- [ ] Falls back to file mtime
+
+**Planned Specs:**
+- [ ] Warns on planned spec > 60 days old
+- [ ] No warning for recent planned specs
+
+**Archived Specs:**
+- [ ] No staleness warnings for archived specs
+
+### 6. Auto-Fix Tests
+
+**Frontmatter Fixes:**
+- [ ] Adds missing `status` field with default
+- [ ] Adds missing `created` field with current date
+- [ ] Formats dates to ISO 8601
+- [ ] Sorts frontmatter fields alphabetically
+- [ ] Updates visual badges from frontmatter
+
+**Structure Fixes:**
+- [ ] Adds missing required sections as comments
+- [ ] Removes duplicate sections (keeps first)
+
+**Corruption Fixes:**
+- [ ] Closes unclosed code blocks
+- [ ] Removes exact duplicate sections
+
+**Non-Fixable Issues:**
+- [ ] Reports invalid status (requires manual fix)
+- [ ] Reports empty sections (requires content)
+- [ ] Reports broken links (requires investigation)
+
+**Fix Reporting:**
+- [ ] Reports what was fixed
+- [ ] Reports what couldn't be fixed
+- [ ] Doesn't break valid specs
+- [ ] Maintains spec readability
+
+### 7. Integration Tests
+
+**Multiple Specs:**
+- [ ] Validates all specs in project
+- [ ] Reports aggregate results
+- [ ] Handles large number of specs (100+)
+
+**Filtering:**
+- [ ] Filters by status work correctly
+- [ ] Filters by tag work correctly
+- [ ] Filters by priority work correctly
+- [ ] Path pattern filtering works
+
+**Output Formats:**
+- [ ] Console output is readable
+- [ ] JSON output is valid
+- [ ] Quiet mode produces minimal output
+- [ ] Verbose mode includes details
+
+**Exit Codes:**
+- [ ] Returns 0 when all checks pass
+- [ ] Returns 1 when errors found
+- [ ] Returns 2 when warnings in strict mode
+- [ ] Returns 3 on command error
+
+**Configuration:**
+- [ ] Respects custom config
+- [ ] Falls back to defaults
+- [ ] Command-line flags override config
+- [ ] Template-specific rules work
+
+**Backwards Compatibility:**
+- [ ] `lean-spec check` (sequence conflicts) behavior unchanged
+- [ ] `lean-spec validate` doesn't interfere with check command
+- [ ] Exit codes follow standard pattern
+- [ ] Output format is consistent
+
+### 8. Performance Tests
+
+**Speed:**
+- [ ] Validates 10 specs in < 100ms
+- [ ] Validates 100 specs in < 1s
+- [ ] Parallel validation improves performance
+
+**Memory:**
+- [ ] Memory usage stays reasonable
+- [ ] No memory leaks with many specs
+
+**Caching:**
+- [ ] Cached results speed up repeated checks
+- [ ] Cache invalidation works correctly
+
+### 9. Edge Cases
+
+**Empty Specs:**
+- [ ] Handles specs with only frontmatter
+- [ ] Handles completely empty files
+
+**Large Specs:**
+- [ ] Handles specs > 1000 lines
+- [ ] Handles specs with > 100 sections
+- [ ] Performance doesn't degrade
+
+**Special Characters:**
+- [ ] Handles Unicode in frontmatter
+- [ ] Handles special characters in content
+- [ ] Handles emoji in titles
+
+**Symlinks:**
+- [ ] Follows symlinks correctly
+- [ ] Detects circular symlinks
+
+**Sub-Spec Files:**
+- [ ] Validates specs with sub-files
+- [ ] Checks README.md in multi-file specs
+- [ ] Doesn't validate sub-files as specs
+
+## Test Organization
+
+```
+src/
+  commands/
+    check.test.ts              # Command-level tests
+    validators/
+      frontmatter.test.ts      # Frontmatter validator tests
+      structure.test.ts        # Structure validator tests
+      corruption.test.ts       # Corruption detector tests
+      content.test.ts          # Content validator tests
+      staleness.test.ts        # Staleness detector tests
+  integration/
+    check-integration.test.ts  # End-to-end tests
+```
+
+## Test Data
+
+```
+test-fixtures/
+  valid/
+    001-valid-spec/
+      README.md                # Completely valid spec
+  invalid-frontmatter/
+    002-missing-status/
+      README.md                # Missing status field
+    003-invalid-status/
+      README.md                # Invalid status value
+  invalid-structure/
+    004-missing-sections/
+      README.md                # Missing required sections
+    005-duplicate-sections/
+      README.md                # Duplicate section headers
+  corrupted/
+    006-unclosed-code-block/
+      README.md                # Unclosed code block
+    007-invalid-json/
+      README.md                # Malformed JSON in code block
+    008-duplicate-content/
+      README.md                # Duplicated content
+  stale/
+    009-old-in-progress/
+      README.md                # Old in-progress spec
+```
+
+## Continuous Testing
+
+**Pre-Commit:**
+```bash
+npm test  # Run all tests
+```
+
+**CI/CD:**
+```yaml
+- name: Run tests
+  run: npm test
+
+- name: Check coverage
+  run: npm run test:coverage
+```
+
+**Test Coverage Goals:**
+- Overall: >80%
+- Validators: >90%
+- Critical paths: 100%
+
+## Manual Testing Checklist
+
+**Before Release:**
+- [ ] Test on real project (this repo)
+- [ ] Test backwards compatibility
+- [ ] Test performance with 100+ specs
+- [ ] Test all output formats
+- [ ] Test all command-line flags
+- [ ] Test configuration options
+- [ ] Test auto-fix on corrupted specs
+- [ ] Test CI/CD integration
+- [ ] Test pre-commit hook
+
+**Dogfooding:**
+- [ ] Use comprehensive checking on lean-spec itself
+- [ ] Fix any issues found
+- [ ] Verify no false positives
+- [ ] Collect user feedback
+
+## Test Automation
+
+**Snapshot Testing:**
+- Output format snapshots
+- JSON structure snapshots
+- Error message snapshots
+
+**Property-Based Testing:**
+- Generate random specs
+- Verify validators don't crash
+- Verify auto-fix doesn't corrupt
+
+**Regression Testing:**
+- Save examples of past bugs
+- Ensure fixes remain effective
+- Prevent regressions

package/.next/standalone/packages/ui/dist/standalone/specs/018-spec-validation/VALIDATION-RULES.md

@@ -0,0 +1,327 @@
+# Validation Rules
+
+This document details what gets validated by `lean-spec validate` and the specific rules for each validation type.
+
+**Note:** This spec originally proposed expanding `lean-spec check`, but the implementation created `lean-spec validate` as a separate command.
+
+## Implementation Status
+
+- ✅ **Line Count Validation** - Phase 1a Complete
+- ✅ **Frontmatter Validation** - Phase 1b Complete
+- ⏳ **Structure Validation** - Phase 2 (Planned)
+- ⏳ **Corruption Detection** - Phase 3 (High Priority)
+- ⏳ **Sub-Spec Validation** - Phase 3.5 (High Priority)
+- ⏳ **Content Validation** - Phase 4 (Optional)
+- ⏳ **Staleness Detection** - Phase 5 (Optional)
+
+---
+
+## Line Count Validation ✅
+
+**Status:** Implemented in Phase 1a
+
+**Thresholds:**
+- **Warning**: 300-400 lines (approaching limit)
+- **Error**: >400 lines (exceeds limit)
+- **Configurable**: `--max-lines` flag to customize limit
+
+**Rationale:**
+Enforces **Context Economy** principle - specs must fit in working memory (human and AI). Files >400 lines become difficult to read, edit, and maintain.
+
+**Suggestions:**
+- Warns: "Consider simplification or splitting"
+- Errors: "Consider splitting into sub-specs using spec 012 pattern"
+
+---
+
+## Frontmatter Validation ✅
+
+**Status:** Implemented in Phase 1b (27 tests passing)
+
+**Required Fields:**
+- `status` - Must be present
+- `created` - Must be present
+
+**Status Values:**
+- Valid: `planned`, `in-progress`, `complete`, `archived`
+- Invalid: Any other value (e.g., `wip`, `draft`, `done`)
+
+**Priority Values:**
+- Valid: `low`, `medium`, `high`, `critical`
+- Optional field, but if present must be valid
+
+**Date Formats:**
+- ISO 8601 format (YYYY-MM-DD or full ISO timestamp)
+- `created` field must be valid date
+- Other date fields (if present) must be valid
+
+**Tags:**
+- Must be array of strings
+- Example: `["quality", "validation", "cli"]`
+
+**Custom Fields:**
+- If config schema defined, custom fields must match
+- Type validation based on config
+
+## Structure Validation
+
+**File Requirements:**
+- `README.md` must exist in spec directory
+- Frontmatter must be valid YAML
+- Must have title (H1 heading: `# Title`)
+
+**Section Requirements:**
+- Required sections based on template
+- Typically: Overview, Design, Plan, Test
+- No empty required sections (content beyond just comment)
+- No duplicate section headers at same level
+
+## Sub-Spec Validation
+
+**File Naming Conventions (from spec 012):**
+- Sub-spec files should follow recommended naming patterns
+- Standard names: `DESIGN.md`, `IMPLEMENTATION.md`, `TESTING.md`, `API.md`, `ARCHITECTURE.md`, `MIGRATION.md`, `CONFIGURATION.md`, etc.
+- Custom names allowed but should be descriptive (e.g., `PHASE1.md`, `RESEARCH.md`)
+- Assets should be in `assets/` subdirectory
+
+**README.md Integration:**
+- Main `README.md` should reference all sub-spec files
+- Links can be in any section (Overview, Design, etc.)
+- Missing links to sub-specs generate warnings
+- Example: `[Testing Strategy](TESTING.md)` or `See [DESIGN.md](DESIGN.md)`
+
+**Line Count Per Sub-Spec (Context Economy):**
+- Each sub-spec file should be < 400 lines (error threshold)
+- Warning at 300 lines (same as main spec)
+- Rationale: Context Economy applies to each file independently
+- Sub-specs exist to split complexity - they shouldn't become bloated
+
+**Orphan Detection:**
+- Detect sub-spec markdown files not referenced from README.md
+- Warn about orphaned sub-specs (may be deprecated)
+- Exception: README.md itself is never orphaned
+- Assets (.png, .svg, etc.) are checked separately
+
+**Cross-Document Validation:**
+- Validate internal links between sub-specs work correctly
+- Check relative paths are correct
+- Detect circular references (if spec A links to B, B to C, C to A)
+
+**Sub-Spec Structure:**
+- Each sub-spec should have clear heading structure
+- Should include backlink to main spec (recommended)
+- Example: `> Part of spec: [018-spec-validation](README.md)`
+
+## Content Validation
+
+**Minimum Requirements:**
+- Spec must have minimum content length (avoid stub specs)
+- Default: 100 characters (configurable)
+
+**Quality Checks:**
+- No TODO/FIXME in complete specs
+- Internal links should be valid (no broken references)
+- No placeholder text in complete specs
+
+## Corruption Detection
+
+**Duplicate Content:**
+- No duplicate section headers at same level
+- Detect sections appearing multiple times
+- Example: "## Plan" appearing twice in same spec
+
+**Code Block Validation:**
+- All code blocks properly closed (unclosed blocks break syntax highlighting)
+- Matching number of opening ``` and closing ```
+- This is the #1 indicator of corruption humans notice visually
+
+**Formatting Validation:**
+- Unclosed bold formatting (**) in actual content
+- Unclosed italic formatting (*) in actual content
+- Excludes code blocks, inline code, and list markers from checks
+- Only validates markdown content outside code fences
+
+**Duplicate Content Detection:**
+- Detects duplicated content blocks (5+ consecutive lines, 100+ chars)
+- Excludes code blocks to prevent false positives from code examples
+- Tuned to reduce noise: was 3 lines/50 chars (too sensitive)
+- Helps find merge artifacts and failed edits
+- Warnings only (not errors) as some duplication is intentional
+
+**Intentionally Excluded:**
+- ❌ JSON/YAML validation in code blocks (examples often show invalid syntax)
+- ❌ Table and list structure validation (too noisy, not actual corruption)
+
+**Sub-Spec Duplication:**
+- Detect if same content appears in multiple sub-specs
+- Warn about significant content overlap between files
+- Helps identify when content should be consolidated
+
+### Corruption Detection Philosophy
+
+**Focus on visually apparent corruption:**
+- What breaks rendering (unclosed code blocks, unclosed formatting)
+- What humans actually notice when reading specs
+- Not semantic issues or style preferences
+
+**Why we exclude certain checks:**
+- JSON/YAML in code blocks: Examples intentionally show invalid syntax
+- Duplicate content detection: Many false positives from boilerplate/templates
+- Table/list validation: Too noisy, rarely indicates actual corruption
+
+**Tuning for signal-to-noise:**
+- Originally: 3 lines, 50 chars → 13 duplicate warnings across all specs
+- Improved: 5 lines, 100 chars, exclude code blocks → More targeted detection
+- Goal: Only flag genuine corruption, not common patterns
+
+## Staleness Detection
+
+**In-Progress Specs:**
+- Warn if status is `in-progress` for > 30 days
+- Suggests spec might be stalled
+
+**No Updates:**
+- Warn if no updates in > 90 days
+- Based on git history or file mtime
+- Suggests spec might be abandoned
+
+**Planned Specs:**
+- Warn if status is `planned` for > 60 days
+- Suggests planning without execution
+
+**Archived Specs:**
+- No staleness warnings for archived specs
+- Archived specs are expected to be static
+
+## Auto-Fix Capabilities
+
+### Fixable Issues
+
+**Frontmatter:**
+- Add missing required fields (use defaults)
+- Format dates to ISO 8601 standard
+- Sort frontmatter fields alphabetically
+- Update visual badges to match frontmatter
+
+**Structure:**
+- Add missing required sections (as comments)
+- Remove duplicate sections (keep first occurrence)
+- Close unclosed code blocks (where unambiguous)
+
+**Sub-Specs:**
+- Add missing links to sub-specs in README.md
+- Generate suggested backlinks for sub-specs
+- Create "Documentation Structure" section if missing (when sub-specs exist)
+
+### Non-Fixable Issues
+
+These require manual intervention:
+- Invalid status/priority values (requires decision)
+- Empty required sections (requires content)
+- Broken links (requires investigation)
+- Stale specs (requires status update or work)
+- Complex corruption (requires human judgment)
+
+## Error Severity Levels
+
+**Error (blocks passing):**
+- Missing required fields
+- Invalid status/priority values
+- Duplicate sections
+- Corrupted code blocks
+- Invalid JSON/YAML syntax
+- Malformed markdown structure
+
+**Warning (passes with warning):**
+- Stale specs
+- Suggested improvements
+- Minor formatting issues
+- Empty optional sections
+
+**Info (informational):**
+- Best practice suggestions
+- Style recommendations
+- Optimization hints
+
+## Validation Rules Configuration
+
+Rules can be customized in `.lean-spec/config.json`:
+
+```json
+{
+  "check": {
+    "rules": {
+      "frontmatter": {
+        "required": ["status", "created"],
+        "allowedStatus": ["planned", "in-progress", "complete", "archived"],
+        "allowedPriority": ["low", "medium", "high", "critical"]
+      },
+      "structure": {
+        "requireReadme": true,
+        "requiredSections": ["Overview", "Design", "Plan"],
+        "forbidEmptySections": true
+      },
+      "content": {
+        "minLength": 100,
+        "forbidTodoInComplete": true,
+        "validateLinks": true
+      },
+      "corruption": {
+        "detectDuplicateSections": true,
+        "validateCodeBlocks": true,
+        "validateJsonYaml": true,
+        "detectFragments": true
+      },
+      "staleness": {
+        "inProgressMaxDays": 30,
+        "noUpdateMaxDays": 90,
+        "plannedMaxDays": 60
+      },
+      "subSpecs": {
+        "validateNaming": true,
+        "requireReferences": true,
+        "detectOrphans": true,
+        "maxLinesPerFile": 400,
+        "warnLinesPerFile": 300,
+        "validateCrossReferences": true,
+        "standardNames": [
+          "DESIGN.md",
+          "IMPLEMENTATION.md",
+          "TESTING.md",
+          "API.md",
+          "ARCHITECTURE.md",
+          "MIGRATION.md",
+          "CONFIGURATION.md"
+        ]
+      }
+    }
+  }
+}
+```
+
+## Why These Rules?
+
+**Frontmatter:**
+- Ensures specs are trackable and categorizable
+- Required fields enable automation and reporting
+- Valid values prevent typos and inconsistencies
+
+**Structure:**
+- Maintains consistency across team
+- Makes specs easier to navigate
+- Enables automated processing
+
+**Content:**
+- Prevents stub specs from cluttering project
+- Ensures specs are complete before marking done
+- Maintains quality standards
+
+**Corruption:**
+- Catches errors from failed AI edits
+- Prevents shipping broken specs
+- Early detection saves debugging time
+
+**Staleness:**
+- Identifies abandoned work
+- Prompts status updates
+- Keeps project health visible