@voodocs/cli 2.3.0 → 2.5.0

package/CHANGELOG.md

@@ -1,5 +1,76 @@
 # CHANGELOG
 
+## [2.5.0] - 2024-12-24
+
+### Added
+- **Priority Analyzer** - New `voodocs analyze` command to identify which files need annotations most urgently
+  - **Complexity Analysis**: LOC, cyclomatic complexity, function count
+  - **Security Keyword Detection**: auth, password, admin, crypto, SQL, payment, etc.
+  - **Dependency Analysis**: Import tracking and dependent file detection
+  - **Weighted Scoring**: Complexity (30%), Security (40%), Dependencies (20%), Annotations (10%)
+  - **Priority Levels**: CRITICAL (80-100), HIGH (60-79), MEDIUM (40-59), LOW (20-39), MINIMAL (0-19)
+  - **Multiple Output Formats**: text, JSON, CSV, Markdown
+  - **Filtering Options**: By priority level, minimum score, exclude patterns
+  - **Smart Suggestions**: Context-aware recommendations for each file
+- **Priority Analyzer Modules**:
+  - `lib/darkarts/priority_analyzer/complexity.py` - Complexity analysis engine
+  - `lib/darkarts/priority_analyzer/security.py` - Security keyword detector
+  - `lib/darkarts/priority_analyzer/dependencies.py` - Dependency tracker
+  - `lib/darkarts/priority_analyzer/analyzer.py` - Main priority analyzer
+  - `lib/cli/analyze.py` - CLI command implementation
+- **Comprehensive Test Suite** - 16 unit tests for priority analyzer functionality
+
+### Improved
+- Heuristics for identifying security-sensitive code
+- Dependency tracking for TypeScript, JavaScript, Python, and Solidity
+- Annotation coverage detection and penalty calculation
+- Priority scoring algorithm with empirical validation
+
+### Documentation
+- Added Priority Analyzer section to README.md
+- Added analyze command documentation with examples
+- Added priority level definitions and scoring factors
+- Added specification document for priority analyzer
+
+---
+
+## [2.4.0] - 2025-12-24
+
+### Added
+- **VooDocs Lite Format** - Ultra-compact symbolic notation with 27-30% token reduction
+  - New `voodocs convert` command to convert between Standard and Lite formats
+  - Single-character symbols (`>`, `@`, `!`, `<`, `=`, `~`, `#`) instead of Unicode
+  - Aggressive abbreviation dictionary (70+ common terms)
+  - Article removal and symbol replacements for maximum compression
+  - Bidirectional conversion (Standard ↔ Lite) with zero semantic loss
+  - Support for `@darkarts-lite` annotation format
+- **VooDocs Lite Parser** module (`lib/darkarts/voodocs_lite_parser.py`)
+  - `parse_lite()` - Parse Lite format annotations
+  - `parse_standard()` - Parse Standard format annotations
+  - `lite_to_standard()` - Convert Lite to Standard with expansion
+  - `standard_to_lite()` - Convert Standard to Lite with compression
+  - `detect_format()` - Auto-detect annotation format
+- **Ultra-Aggressive Compression** (`lib/darkarts/voodocs_lite_dict_v2.py`)
+  - `ultra_compress()` - Maximum compression with article removal
+  - `ultra_expand()` - Expand compressed text to full form
+  - 70+ abbreviations for common terms
+  - Symbol replacements (and→&, or→|, >=, <=, etc.)
+- **Comprehensive Test Suite** - 24 unit tests for VooDocs Lite functionality
+
+### Improved
+- Token efficiency for AI context (+27-30% reduction)
+- Scannability with single-character symbols
+- Format flexibility (choose Standard or Lite based on needs)
+- Backward compatibility (both formats supported)
+
+### Documentation
+- Added VooDocs Lite section to README.md
+- Added convert command documentation
+- Added format comparison examples
+- Added symbol mapping table
+
+---
+
 ## [2.3.0] - 2025-12-24
 
 ### Added

package/README.md

@@ -91,6 +91,80 @@ def authenticate_user(user_id: str, password: str) -> Optional[str]:
 
 ---
 
+## VooDocs Lite: Ultra-Compact Format
+
+**NEW in v2.4.0!** VooDocs Lite is an ultra-compact symbolic notation that reduces token count by **~30%** while maintaining all semantic information.
+
+### Why VooDocs Lite?
+
+- **Token Efficient**: 27-30% fewer tokens for AI context
+- **More Scannable**: Single-character symbols easier to read
+- **Same Information**: Zero semantic loss
+- **Bidirectional**: Convert between Standard and Lite freely
+
+### Format Comparison
+
+**Standard VooDocs:**
+```typescript
+/**@darkarts
+⊢{User authentication service with JWT token generation}
+∂{bcrypt, jsonwebtoken, database}
+⚠{Users must be stored in database}
+⊳{userId must be a valid UUID, password must be ≥8 characters}
+⊲{Returns JWT token ∨ null}
+⊨{Does ¬ modify database, Password is ¬ logged}
+⚡{O(1)}
+🔒{Password hashed with bcrypt, Token signed with secret}
+*/
+```
+**114 tokens**
+
+**VooDocs Lite:**
+```typescript
+/**@darkarts-lite
+>u auth svc w/ JWT tok gen
+@bcrypt,jsonwebtoken,database
+!us stored in db
+<id valid UUID, pw>=8 characters
+>ret JWT tok|N
+=!mod db, pw !logged
+~O(1)
+#pw hashed w/ bcrypt, tok signed w/ secret
+*/
+```
+**83 tokens** (27% reduction)
+
+### Symbol Mapping
+
+| Standard | Lite | Meaning |
+|----------|------|----------|
+| `⊢{}` | `>` | Purpose |
+| `∂{}` | `@` | Dependencies |
+| `⚠{}` | `!` | Assumptions |
+| `⊳{}` | `<` | Preconditions |
+| `⊲{}` | `>` | Postconditions |
+| `⊨{}` | `=` | Invariants |
+| `⚡{}` | `~` | Complexity |
+| `🔒{}` | `#` | Security |
+
+### Convert Between Formats
+
+```bash
+# Convert Standard to Lite
+voodocs convert src/ --to lite -r
+
+# Convert Lite to Standard
+voodocs convert src/ --to standard -r
+
+# Preview conversion
+voodocs convert src/ --to lite --dry-run
+
+# Modify files in-place
+voodocs convert src/ --to lite --in-place
+```
+
+---
+
 ## Companion Files for Compiled Languages
 
 **NEW in v2.3.0!** For compiled languages like Solidity, Rust, or C++ where inline annotations may interfere with compilation, VooDocs supports **companion documentation files**.
@@ -262,6 +336,44 @@ voodocs companion contracts/ --dry-run  # Preview changes
 voodocs companion contracts/ --force    # Overwrite existing
 ```
 
+### `voodocs convert`
+
+Convert between Standard and Lite VooDocs formats:
+
+```bash
+voodocs convert src/ --to lite      # Convert to Lite format
+voodocs convert src/ --to standard  # Convert to Standard format
+voodocs convert src/ --to lite -r   # Recursive conversion
+voodocs convert src/ --to lite --dry-run  # Preview changes
+voodocs convert src/ --to lite --in-place # Modify files in-place
+```
+
+### `voodocs analyze`
+
+**NEW in v2.5.0!** Analyze files and suggest annotation priorities based on complexity, security, and dependencies:
+
+```bash
+voodocs analyze src/                # Analyze directory
+voodocs analyze src/ --top 20       # Show top 20 files
+voodocs analyze src/ --priority critical  # Filter by priority
+voodocs analyze src/ --format json  # JSON output
+voodocs analyze src/ --min-score 60 # Only show files with score >= 60
+voodocs analyze src/ --exclude node_modules,dist  # Exclude patterns
+```
+
+**Priority Levels:**
+- 🔴 **CRITICAL** (80-100): Highly complex, security-sensitive, or widely-used files
+- 🟠 **HIGH** (60-79): Complex or security-sensitive files
+- 🟡 **MEDIUM** (40-59): Moderately complex files
+- 🟢 **LOW** (20-39): Simple files with some complexity
+- ⚪ **MINIMAL** (0-19): Very simple files
+
+**Scoring Factors:**
+- **Complexity** (30%): Lines of code, cyclomatic complexity, function count
+- **Security** (40%): Security-sensitive keywords (auth, password, admin, etc.)
+- **Dependencies** (20%): Import count and dependent files
+- **Annotations** (10%): Penalty for missing VooDocs annotations
+
 ### `voodocs generate`
 
 Generate documentation from annotations:

package/lib/cli/analyze.py

@@ -0,0 +1,277 @@
+"""
+CLI command for VooDocs Priority Analyzer
+
+Usage: voodocs analyze [path] [options]
+"""
+
+import os
+import sys
+from pathlib import Path
+
+# Add parent directory to path for imports
+sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '../..')))
+
+from lib.darkarts.priority_analyzer.analyzer import PriorityAnalyzer
+
+
+def cmd_analyze(args):
+    """Execute the analyze command."""
+    # Get path (default to current directory)
+    path = args.path if hasattr(args, 'path') and args.path else '.'
+    path = os.path.abspath(path)
+    
+    # Check if path exists
+    if not os.path.exists(path):
+        print(f"❌ Error: Path not found: {path}")
+        return 1
+    
+    # Determine if path is file or directory
+    is_file = os.path.isfile(path)
+    
+    # Initialize analyzer
+    project_root = path if not is_file else str(Path(path).parent)
+    analyzer = PriorityAnalyzer(project_root=project_root)
+    
+    # Analyze
+    if is_file:
+        scores = [analyzer.analyze_file(path)]
+    else:
+        recursive = getattr(args, 'recursive', True)
+        exclude = getattr(args, 'exclude', None)
+        exclude_patterns = exclude.split(',') if exclude else None
+        
+        print(f"🔍 Analyzing files in: {path}")
+        if recursive:
+            print("   (recursive mode)")
+        print()
+        
+        scores = analyzer.analyze_directory(path, recursive, exclude_patterns)
+    
+    # Filter by priority level if specified
+    if hasattr(args, 'priority') and args.priority:
+        priority_filter = args.priority.upper()
+        scores = [s for s in scores if s.priority_level == priority_filter]
+    
+    # Filter by minimum score
+    if hasattr(args, 'min_score') and args.min_score:
+        scores = [s for s in scores if s.priority_score >= args.min_score]
+    
+    # Filter out fully annotated files (unless --include-annotated)
+    if not getattr(args, 'include_annotated', False):
+        scores = [s for s in scores if s.annotation_coverage < 1.0]
+    
+    # Limit to top N
+    top_n = getattr(args, 'top', 10)
+    if top_n and top_n > 0:
+        scores = scores[:top_n]
+    
+    # Output format
+    output_format = getattr(args, 'format', 'text')
+    
+    if output_format == 'json':
+        _output_json(scores)
+    elif output_format == 'csv':
+        _output_csv(scores)
+    elif output_format == 'markdown':
+        _output_markdown(scores)
+    else:
+        _output_text(scores, path)
+    
+    return 0
+
+
+def _output_text(scores, path):
+    """Output results in human-readable text format."""
+    if not scores:
+        print("✅ No files need annotations!")
+        print("   All files are either fully annotated or below priority threshold.")
+        return
+    
+    # Calculate statistics
+    total_files = len(scores)
+    priority_dist = {
+        'CRITICAL': len([s for s in scores if s.priority_level == 'CRITICAL']),
+        'HIGH': len([s for s in scores if s.priority_level == 'HIGH']),
+        'MEDIUM': len([s for s in scores if s.priority_level == 'MEDIUM']),
+        'LOW': len([s for s in scores if s.priority_level == 'LOW']),
+        'MINIMAL': len([s for s in scores if s.priority_level == 'MINIMAL']),
+    }
+    
+    annotated_count = sum(1 for s in scores if s.annotation_coverage > 0)
+    coverage_pct = (annotated_count / total_files * 100) if total_files > 0 else 0
+    
+    # Header
+    print("━" * 70)
+    print("VooDocs Priority Analysis")
+    print("━" * 70)
+    print()
+    print(f"Project: {path}")
+    print(f"Files analyzed: {total_files}")
+    print(f"Annotations found: {annotated_count} ({coverage_pct:.0f}% coverage)")
+    print()
+    
+    # Priority distribution
+    print("Priority Distribution:")
+    if priority_dist['CRITICAL'] > 0:
+        print(f"🔴 CRITICAL (80-100): {priority_dist['CRITICAL']} files")
+    if priority_dist['HIGH'] > 0:
+        print(f"🟠 HIGH (60-79):     {priority_dist['HIGH']} files")
+    if priority_dist['MEDIUM'] > 0:
+        print(f"🟡 MEDIUM (40-59):   {priority_dist['MEDIUM']} files")
+    if priority_dist['LOW'] > 0:
+        print(f"🟢 LOW (20-39):      {priority_dist['LOW']} files")
+    if priority_dist['MINIMAL'] > 0:
+        print(f"⚪ MINIMAL (0-19):    {priority_dist['MINIMAL']} files")
+    print()
+    
+    # Top files
+    print("━" * 70)
+    print(f"Top {len(scores)} Files Needing Annotations")
+    print("━" * 70)
+    print()
+    
+    for i, score in enumerate(scores, 1):
+        # Priority emoji
+        emoji = {
+            'CRITICAL': '🔴',
+            'HIGH': '🟠',
+            'MEDIUM': '🟡',
+            'LOW': '🟢',
+            'MINIMAL': '⚪'
+        }.get(score.priority_level, '⚪')
+        
+        # Relative path
+        rel_path = os.path.relpath(score.filepath, path) if not os.path.isfile(path) else score.filepath
+        
+        print(f"{i}. {emoji} {rel_path} (Score: {score.priority_score:.0f})")
+        print(f"   Complexity: {score.complexity_score}  Security: {score.security_score}  Dependencies: {score.dependency_score}  Annotations: {score.annotation_penalty}")
+        print()
+        
+        # Reasons
+        if score.reasons:
+            print("   Reasons:")
+            for reason in score.reasons:
+                print(f"   • {reason}")
+            print()
+        
+        # Suggestions
+        if score.suggestions:
+            print("   Suggestions:")
+            for suggestion in score.suggestions:
+                print(f"   ✓ {suggestion}")
+            print()
+    
+    # Footer
+    print("━" * 70)
+    print()
+    print("💡 Tip: Start with CRITICAL files first. Use:")
+    if scores:
+        first_file = scores[0].filepath
+        print(f"   voodocs companion {first_file}")
+    print()
+    print("   Or convert to VooDocs Lite for faster annotation:")
+    print(f"   voodocs convert {path} --to lite --in-place")
+    print()
+
+
+def _output_json(scores):
+    """Output results in JSON format."""
+    import json
+    
+    data = []
+    for score in scores:
+        data.append({
+            'filepath': score.filepath,
+            'priority_score': score.priority_score,
+            'priority_level': score.priority_level,
+            'complexity_score': score.complexity_score,
+            'security_score': score.security_score,
+            'dependency_score': score.dependency_score,
+            'annotation_penalty': score.annotation_penalty,
+            'annotation_coverage': score.annotation_coverage,
+            'loc': score.loc,
+            'functions': score.functions,
+            'security_keywords': score.security_keywords,
+            'dependent_count': score.dependent_count,
+            'reasons': score.reasons,
+            'suggestions': score.suggestions
+        })
+    
+    print(json.dumps(data, indent=2))
+
+
+def _output_csv(scores):
+    """Output results in CSV format."""
+    import csv
+    import sys
+    
+    writer = csv.writer(sys.stdout)
+    
+    # Header
+    writer.writerow([
+        'filepath', 'priority_score', 'priority_level',
+        'complexity_score', 'security_score', 'dependency_score',
+        'annotation_penalty', 'annotation_coverage',
+        'loc', 'functions', 'dependent_count'
+    ])
+    
+    # Data
+    for score in scores:
+        writer.writerow([
+            score.filepath,
+            score.priority_score,
+            score.priority_level,
+            score.complexity_score,
+            score.security_score,
+            score.dependency_score,
+            score.annotation_penalty,
+            score.annotation_coverage,
+            score.loc,
+            score.functions,
+            score.dependent_count
+        ])
+
+
+def _output_markdown(scores):
+    """Output results in Markdown format."""
+    print("# VooDocs Priority Analysis")
+    print()
+    print("## Summary")
+    print()
+    print(f"- **Files analyzed:** {len(scores)}")
+    print()
+    print("## Top Files")
+    print()
+    
+    for i, score in enumerate(scores, 1):
+        emoji = {
+            'CRITICAL': '🔴',
+            'HIGH': '🟠',
+            'MEDIUM': '🟡',
+            'LOW': '🟢',
+            'MINIMAL': '⚪'
+        }.get(score.priority_level, '⚪')
+        
+        print(f"### {i}. {emoji} {score.filepath}")
+        print()
+        print(f"**Score:** {score.priority_score:.0f} ({score.priority_level})")
+        print()
+        print(f"- Complexity: {score.complexity_score}")
+        print(f"- Security: {score.security_score}")
+        print(f"- Dependencies: {score.dependency_score}")
+        print(f"- Annotation Coverage: {score.annotation_coverage * 100:.0f}%")
+        print()
+        
+        if score.reasons:
+            print("**Reasons:**")
+            print()
+            for reason in score.reasons:
+                print(f"- {reason}")
+            print()
+        
+        if score.suggestions:
+            print("**Suggestions:**")
+            print()
+            for suggestion in score.suggestions:
+                print(f"- {suggestion}")
+            print()

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@voodocs/cli",
-  "version": "2.3.0",
+  "version": "2.5.0",
   "description": "AI-Native Symbolic Documentation System - The world's first documentation tool using mathematical notation with semantic validation",
   "main": "voodocs_cli.py",
   "bin": {