@sun-asterisk/sunlint 1.3.1 → 1.3.3

package/docs/PERFORMANCE_OPTIMIZATION_PLAN.md

@@ -0,0 +1,255 @@
+# 🚀 SunLint Performance Optimization Plan
+
+## 📊 Current Performance Challenges
+
+### **Critical Issues Identified**
+1. **Timeout Issues**: `Engine heuristic failed: Engine undefined timed out after 30000ms`
+2. **Memory Exhaustion**: `Maximum call stack size exceeded` with large projects
+3. **Symbol Table Bottleneck**: Loading 73 rules × large file count = performance disaster
+4. **Batch Processing Gap**: No intelligent rule batching for large projects
+
+### **Performance Metrics (Current)**
+- **Total Rules**: 100 (Common: 34, Security: 49, Others: 17)
+- **SunLint Rules**: 73 rules with analyzers (3x growth from ~22)
+- **AST-Powered**: 65+ rules requiring semantic analysis
+- **Memory Impact**: Symbol table × file count × rule count = O(n³) complexity
+
+---
+
+## 🎯 Optimization Strategy
+
+### **Phase 1: Immediate Performance Fixes**
+
+#### **1.1 Timeout Management**
+```bash
+# Current: Fixed 30s timeout
+# Solution: Dynamic timeout based on project size
+--timeout=60000          # 60s for large projects
+--timeout=120000         # 120s for enterprise projects
+--adaptive-timeout       # Auto-scale based on file count
+```
+
+#### **1.2 File Filtering Enhancement**
+```bash
+# Current: Basic exclude patterns
+# Solution: Smart exclusion patterns
+--exclude-patterns="node_modules/**,.next/**,dist/**,build/**"
+--max-files=1000         # Hard limit for safety
+--smart-sampling         # Intelligent file sampling for large projects
+```
+
+#### **1.3 Rule Batching**
+```bash
+# Current: All rules processed together
+# Solution: Intelligent rule batching
+--batch-size=10          # Process 10 rules at a time
+--priority-rules=C019,S027,C041  # High-priority rules first
+--parallel-batches=2     # Parallel batch processing
+```
+
+### **Phase 2: Memory Optimization**
+
+#### **2.1 Symbol Table Streaming**
+```javascript
+// Current: Load all files into memory
+const symbolTable = await loadAllFiles(); // ❌ Memory explosion
+
+// Solution: Streaming symbol table
+const symbolTable = new StreamingSymbolTable({
+  maxMemoryFiles: 100,    // Keep max 100 files in memory
+  swapToTemp: true,       // Swap to temp files when needed
+  lazyLoading: true       // Load files on-demand
+});
+```
+
+#### **2.2 Rule-Specific Analysis**
+```javascript
+// Current: All rules analyze all files
+rules.forEach(rule => files.forEach(file => rule.analyze(file))); // ❌ O(n²)
+
+// Solution: Smart rule targeting
+const fileToRulesMap = buildFileRuleMapping(files, rules);
+fileToRulesMap.forEach((rules, file) => {
+  rules.forEach(rule => rule.analyze(file)); // ✅ Optimized
+});
+```
+
+### **Phase 3: Architecture Optimization**
+
+#### **3.1 Worker Process Architecture**
+```javascript
+// Solution: Multi-process rule execution
+const workers = {
+  syntaxRules: createWorker('./workers/syntax-rules.js'),
+  securityRules: createWorker('./workers/security-rules.js'),
+  semanticRules: createWorker('./workers/semantic-rules.js')
+};
+
+// Distribute rules across workers
+const results = await Promise.all([
+  workers.syntaxRules.analyze(files, syntaxRules),
+  workers.securityRules.analyze(files, securityRules),
+  workers.semanticRules.analyze(files, semanticRules)
+]);
+```
+
+#### **3.2 Progressive Analysis**
+```javascript
+// Solution: Progressive disclosure of violations
+const analyzer = new ProgressiveAnalyzer({
+  fastRules: ['C002', 'C003', 'C006'],      // Quick regex rules first
+  mediumRules: ['C019', 'C041', 'S027'],    // Moderate AST rules
+  slowRules: ['C033', 'C047', 'C076']       // Heavy semantic rules last
+});
+
+// Show results progressively
+analyzer.on('fastComplete', (violations) => {
+  console.log(`Quick scan: ${violations.length} issues found`);
+});
+```
+
+---
+
+## 🛠️ Implementation Roadmap
+
+### **Week 1: Critical Fixes**
+- [ ] **Dynamic timeout configuration**
+- [ ] **Enhanced file exclusion patterns** 
+- [ ] **Memory limit safeguards**
+- [ ] **Rule batching implementation**
+
+### **Week 2: Memory Optimization**
+- [ ] **Streaming symbol table**
+- [ ] **File-to-rule mapping optimization**
+- [ ] **Lazy loading for AST analysis**
+- [ ] **Memory monitoring & alerts**
+
+### **Week 3: Architecture Enhancement**
+- [ ] **Worker process architecture**
+- [ ] **Progressive analysis pipeline**
+- [ ] **Parallel rule execution**
+- [ ] **Result caching system**
+
+### **Week 4: Testing & Validation**
+- [ ] **Performance benchmarking**
+- [ ] **Large project testing**
+- [ ] **Memory leak detection**
+- [ ] **Timeout scenario testing**
+
+---
+
+## 📈 Expected Performance Improvements
+
+### **Target Metrics**
+| **Scenario** | **Current** | **Target** | **Improvement** |
+|--------------|-------------|------------|-----------------|
+| **Small Project** (< 100 files) | 30s | 10s | **3x faster** |
+| **Medium Project** (100-500 files) | Timeout | 45s | **Functional** |
+| **Large Project** (500+ files) | Memory crash | 120s | **Functional** |
+| **Enterprise Project** (1000+ files) | Impossible | 300s | **Breakthrough** |
+
+### **Memory Usage**
+| **Project Size** | **Current** | **Target** | **Reduction** |
+|------------------|-------------|------------|---------------|
+| **100 files** | 2GB | 500MB | **75% reduction** |
+| **500 files** | Crash | 1.5GB | **Functional** |
+| **1000+ files** | Crash | 3GB | **Controlled** |
+
+---
+
+## 🚦 Performance Monitoring
+
+### **Key Performance Indicators (KPIs)**
+```javascript
+const performanceMetrics = {
+  // Analysis Speed
+  rulesPerSecond: target >= 5,
+  filesPerSecond: target >= 20,
+  violationsPerSecond: target >= 100,
+  
+  // Memory Efficiency  
+  memoryUsage: target <= '2GB',
+  memoryGrowthRate: target <= '10MB/min',
+  garbageCollectionFreq: target <= '5/min',
+  
+  // Reliability
+  timeoutRate: target <= '1%',
+  crashRate: target <= '0.1%',
+  successRate: target >= '99%'
+};
+```
+
+### **Performance Alerts**
+```bash
+# Memory warnings
+if (memoryUsage > 1.5GB) warn("High memory usage detected");
+if (memoryUsage > 2.5GB) error("Memory limit approaching");
+
+# Timeout warnings  
+if (analysisTime > 60s) warn("Analysis taking longer than expected");
+if (analysisTime > 120s) error("Consider using --max-files or --batch-size");
+
+# Progress indicators
+echo "⚡ Processing batch 1/5 (20 rules)..."
+echo "📊 Analyzed 245/1000 files (24.5%)"
+echo "🎯 Found 23 violations so far..."
+```
+
+---
+
+## 🔧 CLI Enhancement
+
+### **New Performance Options**
+```bash
+# Timeout Management
+sunlint --timeout=120000                    # 2 minutes timeout
+sunlint --adaptive-timeout                  # Auto-scale timeout
+sunlint --no-timeout                        # Disable timeout (careful!)
+
+# Memory Management  
+sunlint --max-memory=2GB                    # Memory limit
+sunlint --max-files=1000                    # File count limit
+sunlint --streaming-analysis                # Use streaming mode
+
+# Batch Processing
+sunlint --batch-size=10                     # Rules per batch
+sunlint --parallel-batches=2                # Parallel processing
+sunlint --progressive-results               # Show results as available
+
+# Performance Profiles
+sunlint --performance-profile=fast          # Quick scan (30 rules)
+sunlint --performance-profile=balanced      # Standard scan (50 rules)  
+sunlint --performance-profile=comprehensive # Full scan (73 rules)
+```
+
+### **Smart Defaults Based on Project Size**
+```javascript
+const performanceProfiles = {
+  small: { files: '<100', timeout: 30000, batchSize: 20, rules: 'fast' },
+  medium: { files: '100-500', timeout: 60000, batchSize: 15, rules: 'balanced' },
+  large: { files: '500-1000', timeout: 120000, batchSize: 10, rules: 'essential' },
+  enterprise: { files: '>1000', timeout: 300000, batchSize: 5, rules: 'critical' }
+};
+```
+
+---
+
+## 🏆 Success Criteria
+
+### **Performance Goals**
+- ✅ **Zero timeouts** on projects < 1000 files
+- ✅ **Predictable memory usage** (linear growth, not exponential)
+- ✅ **Progressive results** (show violations as they're found)
+- ✅ **Graceful degradation** (intelligent fallbacks for large projects)
+
+### **User Experience Goals**
+- ✅ **Clear progress indicators** during long analyses
+- ✅ **Meaningful performance warnings** before problems occur  
+- ✅ **Smart defaults** based on project size
+- ✅ **Escape hatches** for power users (custom timeouts, batch sizes)
+
+---
+
+**Performance optimization is critical for SunLint adoption at scale. With 73 heuristic rules and growing, we must proactively address performance bottlenecks to maintain developer experience quality.**
+
+*Engineering Excellence • Performance • Scalability*

package/docs/QUICK_FILE_LIMITS.md

@@ -0,0 +1,64 @@
+# 🎯 Quick Reference: File Limits
+
+## **TL;DR: When to Use Each Option**
+
+### **🤖 Most Users (90%)**
+```bash
+# ✅ Just use auto-detection - no manual limits needed
+sunlint --all --input=src
+```
+
+### **⚡ Performance Tuning (10%)**
+```bash
+# ✅ Both limits for different purposes
+sunlint --all --input=src --max-files=1000 --max-semantic-files=300
+```
+
+---
+
+## **Quick Decision Matrix**
+
+| **Situation** | **Use `--max-files`** | **Use `--max-semantic-files`** |
+|---------------|----------------------|---------------------------------|
+| 🐌 **Slow analysis** | ✅ YES - Limit total work | ❌ Not needed |
+| 💾 **Memory issues** | ✅ YES - Reduce base memory | ✅ YES - Reduce symbol table |
+| 🔍 **TypeScript heavy** | ❌ Not main issue | ✅ YES - Limit ts-morph memory |
+| 🚀 **CI/CD timeout** | ✅ YES - Faster completion | ✅ YES - Less parsing |
+
+---
+
+## **📊 Memory Impact**
+
+### **Analysis Files (`--max-files`)**
+- **50MB** → 1000 files
+- **100MB** → 2000 files  
+- **Impact**: Base memory + file reading
+
+### **Symbol Table Files (`--max-semantic-files`)**  
+- **200MB** → 100 TypeScript files
+- **1GB** → 500 TypeScript files
+- **Impact**: AST parsing + type information
+
+---
+
+## **🎛️ Common Patterns**
+
+```bash
+# 🏢 Enterprise (safe defaults)
+--max-files=800 --max-semantic-files=200
+
+# 🚀 CI/CD (fast & reliable)  
+--max-files=500 --max-semantic-files=100
+
+# 🔍 TypeScript focus (more symbol table)
+--max-files=600 --max-semantic-files=400
+
+# 📦 JavaScript mainly (less symbol table)
+--max-files=1500 --max-semantic-files=50
+```
+
+---
+
+**💡 Key Insight: Symbol table limit should typically be smaller than analysis limit due to higher memory usage per file.**
+
+*See [FILE_LIMITS_EXPLANATION.md](./FILE_LIMITS_EXPLANATION.md) for detailed explanation.*

package/docs/SIMPLIFIED_USAGE_GUIDE.md

@@ -0,0 +1,208 @@
+# 🚀 SunLint Performance - Simplified Usage Guide
+
+## 🎯 **TÓM TẮT: 3 Commands Duy Nhất Bạn Cần Biết**
+
+### **1. 🏃‍♂️ Quick Start (90% use cases)**
+```bash
+sunlint --all --input=src
+```
+✅ **Auto-detects** project size và chọn settings tối ưu  
+✅ **Zero configuration** - chỉ cần chỉ định input folder  
+✅ **Works everywhere** - small projects đến enterprise  
+
+### **2. ⚡ Performance Modes (khi cần tùy chỉnh)**
+```bash
+# Fast scan (for testing/development)
+sunlint --all --input=src --performance=fast
+
+# Thorough analysis (for CI/CD)  
+sunlint --all --input=src --performance=careful
+```
+
+### **3. 🛠️ Custom Timeout (khi project rất lớn)**
+```bash
+sunlint --all --input=src --timeout=120000  # 2 minutes
+```
+
+---
+
+## 🤖 **Auto Performance Detection**
+
+SunLint **tự động phát hiện** project size và chọn settings tối ưu:
+
+| **Project Size** | **Files** | **Auto Settings** | **Timeout** |
+|------------------|-----------|-------------------|-------------|
+| **Small** | < 100 | Fast analysis | 30s |
+| **Medium** | 100-500 | Balanced | 60s |
+| **Large** | 500-1000 | Careful + progressive | 120s |
+| **Enterprise** | 1000+ | Conservative + streaming | 300s |
+
+### **Auto-Detection Logic**
+```bash
+# ✅ SunLint tự động:
+# - Đếm số files trong input folder
+# - Phát hiện TypeScript, Node.js project
+# - Chọn timeout và batch size phù hợp
+# - Bật progressive results cho large projects
+
+sunlint --all --input=src  # Làm tất cả tự động!
+```
+
+---
+
+## 📋 **Common Usage Patterns**
+
+### **Development (hàng ngày)**
+```bash
+# Quick feedback loop
+sunlint --rules=C019,C041,S027 --input=src
+
+# Check specific files
+sunlint --all --input=src/components --performance=fast
+```
+
+### **Code Review/PR**
+```bash
+# Check changed files only
+sunlint --all --changed-files
+
+# Quick but comprehensive
+sunlint --all --input=src --performance=fast --verbose
+```
+
+### **CI/CD Pipeline**
+```bash
+# Thorough analysis with auto-optimization
+sunlint --all --input=src --format=json --output=results.json
+
+# For large projects in CI
+sunlint --all --input=src --performance=careful --quiet
+```
+
+### **Weekly Code Quality Review**
+```bash
+# Full analysis with detailed reporting
+sunlint --all --input=src --verbose --format=table
+```
+
+---
+
+## 🚨 **Troubleshooting Simplified**
+
+### **❌ Getting Timeouts?**
+```bash
+# Try longer timeout
+sunlint --all --input=src --timeout=120000
+
+# Or limit files
+sunlint --all --input=src --max-files=500
+```
+
+### **❌ Taking Too Long?**
+```bash
+# Use fast mode
+sunlint --all --input=src --performance=fast
+
+# Or check specific rules
+sunlint --rules=C002,C019,S027 --input=src
+```
+
+### **❌ Memory Issues?**
+```bash
+# Automatic handling - just use auto mode
+sunlint --all --input=src --performance=auto
+```
+
+---
+
+## 🎛️ **Migration from Complex Commands**
+
+### **BEFORE (v3.x - Complex)**
+```bash
+# ❌ Too many options to remember
+sunlint --all --input=src \
+  --performance-profile=balanced \
+  --adaptive-timeout \
+  --max-memory=2GB \
+  --batch-size=10 \
+  --progressive-results \
+  --verbose
+```
+
+### **AFTER (v4.x - Simplified)**
+```bash
+# ✅ Simple and effective
+sunlint --all --input=src --verbose
+```
+
+### **Advanced Users Can Still Customize**
+```bash
+# For power users who need control
+sunlint --all --input=src --performance=careful --timeout=180000
+```
+
+---
+
+## 📊 **Performance Comparison**
+
+| **Command** | **Small Project** | **Large Project** | **Enterprise** |
+|-------------|-------------------|-------------------|----------------|
+| `--performance=auto` | ~10s | ~60s | ~120s |
+| `--performance=fast` | ~5s | ~30s | ~60s |
+| `--performance=careful` | ~15s | ~90s | ~180s |
+
+---
+
+## ✅ **Best Practices**
+
+### **🎯 DO (Recommended)**
+```bash
+✅ sunlint --all --input=src                    # Let auto-detection work
+✅ sunlint --all --input=src --verbose          # See what's happening  
+✅ sunlint --quality --input=src --performance=fast  # Quick quality check
+✅ sunlint --all --changed-files                # Only check changes
+```
+
+### **❌ DON'T (Avoid)**
+```bash
+❌ sunlint --all --input=src --performance-profile=conservative --batch-size=5 --streaming-analysis
+   # Too complex - just use --performance=careful
+
+❌ sunlint --all --input=src --timeout=5000     
+   # Too short - let auto-detection choose
+
+❌ sunlint --all --input=huge-project           
+   # Missing performance hint - add --performance=careful
+```
+
+---
+
+## 🏆 **Success Metrics**
+
+### **✅ Simplified CLI Achieved**
+- **3 main commands** cover 90% of use cases
+- **Auto-detection** eliminates guesswork  
+- **Zero configuration** for most projects
+- **Predictable performance** across project sizes
+
+### **✅ Backward Compatibility**
+- Old commands still work but show deprecation warnings
+- Gradual migration path for existing users
+- Advanced options available for power users
+
+---
+
+## 🚀 **Quick Start Checklist**
+
+- [ ] **Update to SunLint v4.x** with auto-performance
+- [ ] **Use basic command**: `sunlint --all --input=src`
+- [ ] **Add --verbose** if you want to see progress
+- [ ] **Use --performance=fast** for quick checks
+- [ ] **Use --performance=careful** for thorough analysis
+- [ ] **Test with your project** to validate performance
+
+---
+
+**🎯 Bottom Line: Chỉ cần nhớ `sunlint --all --input=src` - mọi thứ khác được tự động optimize!**
+
+*🚀 Simple • ⚡ Fast • 🎯 Effective*