@iservu-inc/adf-cli 0.9.1 → 0.11.0

package/.project/docs/designs/PATTERN-DECAY-ALGORITHM.md

@@ -0,0 +1,526 @@
+# Pattern Decay Algorithm Design
+
+**Phase:** 6 - Advanced Learning Features
+**Component:** Pattern Decay System
+**Version:** 1.0
+**Date:** 2025-10-26
+**Status:** Design Complete
+
+---
+
+## 🎯 Goal
+
+Reduce confidence scores over time for inactive patterns to keep learning data fresh and relevant, preventing stale patterns from affecting interview filtering.
+
+---
+
+## 📊 Problem Statement
+
+**Current Issue:**
+- Patterns remain at high confidence indefinitely
+- Old patterns may no longer reflect user preferences
+- User workflows change over time (e.g., prototypes → production apps)
+- No mechanism to "forget" outdated patterns
+- Stale patterns can cause incorrect filtering
+
+**Example Scenarios:**
+
+**Scenario 1: Workflow Change**
+- User builds prototypes for 3 months → skips deployment questions (pattern: 90% confidence)
+- User switches to production apps → needs deployment questions
+- Old pattern still active → incorrectly filters deployment questions
+- **Solution:** Pattern confidence decays, eventually removed
+
+**Scenario 2: Team Growth**
+- Solo developer → skips collaboration questions (pattern: 85% confidence)
+- Hires team → needs collaboration questions
+- Old pattern persists → filters important questions
+- **Solution:** Pattern confidence reduces over 2-3 months
+
+---
+
+## 🏗️ Algorithm Design
+
+### Decay Formula
+
+**Time-Based Exponential Decay:**
+
+```
+newConfidence = currentConfidence × (1 - decayRate)^monthsInactive
+
+Where:
+- currentConfidence: Current pattern confidence (0-100)
+- decayRate: Monthly decay rate (default 0.15 = 15% reduction per month)
+- monthsInactive: Months since lastSeen timestamp
+```
+
+**Alternative: Linear Decay (Simpler)**
+
+```
+newConfidence = currentConfidence - (decayRate × monthsInactive)
+
+Where:
+- decayRate: Points to reduce per month (default 10 points/month)
+```
+
+**Recommendation:** Use exponential decay for natural forgetting curve.
+
+### Decay Rates by Confidence Level
+
+Different decay rates based on initial confidence:
+
+```javascript
+function getDecayRate(confidence, baseRate = 0.15) {
+  if (confidence >= 90) return baseRate * 0.5;    // High confidence decays slower (7.5%/month)
+  if (confidence >= 75) return baseRate;          // Medium confidence decays normally (15%/month)
+  return baseRate * 1.5;                          // Low confidence decays faster (22.5%/month)
+}
+```
+
+**Rationale:**
+- High-confidence patterns (90%+) are strong signals → decay slower
+- Low-confidence patterns (<75%) are weak signals → decay faster
+- Natural forgetting: stronger memories fade slower
+
+---
+
+## 📐 Data Structure Changes
+
+### Pattern Data Structure (Enhanced)
+
+```json
+{
+  "id": "pattern_001",
+  "type": "consistent_skip",
+  "questionId": "q_deployment_host",
+  "confidence": 90,
+  "initialConfidence": 90,
+  "sessionsAnalyzed": 10,
+  "skipCount": 9,
+  "status": "active",
+  "userApproved": true,
+  "createdAt": "2025-09-01T10:00:00Z",
+  "lastSeen": "2025-10-26T15:30:00Z",
+  "lastDecayCalculation": "2025-10-26T15:30:00Z",
+  "decayRate": 0.15,
+  "timesRenewed": 0
+}
+```
+
+**New Fields:**
+- `createdAt`: When pattern was first detected
+- `lastSeen`: Last time pattern was observed (skip event matching this pattern)
+- `lastDecayCalculation`: Last time decay was applied
+- `initialConfidence`: Original confidence score (for tracking decay history)
+- `decayRate`: Custom decay rate for this pattern (overrides default)
+- `timesRenewed`: Count of how many times pattern was reconfirmed
+
+### Configuration Structure
+
+```json
+{
+  "decay": {
+    "enabled": true,
+    "baseDecayRate": 0.15,
+    "minConfidenceThreshold": 50,
+    "decayCheckInterval": "weekly",
+    "removeBelow": 40,
+    "highConfidenceThreshold": 90,
+    "lowConfidenceThreshold": 75,
+    "renewalBoost": 10
+  }
+}
+```
+
+**Configuration Options:**
+- `enabled`: Master switch for decay system
+- `baseDecayRate`: Default monthly decay rate (0-1, default 0.15)
+- `minConfidenceThreshold`: Patterns below this don't affect filtering
+- `decayCheckInterval`: How often to calculate decay (daily/weekly/monthly)
+- `removeBelow`: Auto-remove patterns below this confidence
+- `renewalBoost`: Confidence points added when pattern reconfirmed
+
+---
+
+## 🔄 Algorithm Pseudocode
+
+### Decay Calculation
+
+```javascript
+function calculateDecay(pattern, config) {
+  // Skip if decay disabled
+  if (!config.decay.enabled) return pattern;
+
+  // Skip if user manually approved (optional protection)
+  if (pattern.userApproved && config.decay.protectApproved) {
+    return pattern;
+  }
+
+  // Calculate time since last seen
+  const now = new Date();
+  const lastSeen = new Date(pattern.lastSeen);
+  const monthsInactive = (now - lastSeen) / (1000 * 60 * 60 * 24 * 30);
+
+  // Skip if recently seen (< 1 week)
+  if (monthsInactive < 0.25) return pattern;
+
+  // Get decay rate based on confidence
+  const decayRate = getDecayRate(pattern.confidence, config.decay.baseDecayRate);
+
+  // Apply exponential decay
+  const newConfidence = pattern.confidence * Math.pow(1 - decayRate, monthsInactive);
+
+  // Round to 2 decimal places
+  const roundedConfidence = Math.round(newConfidence * 100) / 100;
+
+  // Update pattern
+  return {
+    ...pattern,
+    confidence: Math.max(0, roundedConfidence),
+    lastDecayCalculation: now.toISOString()
+  };
+}
+```
+
+### Pattern Renewal (Reconfirmation)
+
+```javascript
+function renewPattern(pattern, config) {
+  // Pattern was observed again (skip event matched)
+  const now = new Date();
+
+  // Calculate time since creation (for natural growth)
+  const createdAt = new Date(pattern.createdAt);
+  const monthsSinceCreation = (now - createdAt) / (1000 * 60 * 60 * 24 * 30);
+
+  // Boost confidence by renewal amount
+  let newConfidence = pattern.confidence + config.decay.renewalBoost;
+
+  // Cap at initial confidence + some growth allowance
+  const maxConfidence = Math.min(pattern.initialConfidence + 5, 100);
+  newConfidence = Math.min(newConfidence, maxConfidence);
+
+  return {
+    ...pattern,
+    confidence: newConfidence,
+    lastSeen: now.toISOString(),
+    timesRenewed: pattern.timesRenewed + 1,
+    // Prevent immediate decay after renewal
+    lastDecayCalculation: now.toISOString()
+  };
+}
+```
+
+### Pattern Cleanup
+
+```javascript
+function cleanupPatterns(patterns, config) {
+  const now = new Date();
+  const activePatterns = [];
+  const removedPatterns = [];
+
+  for (const pattern of patterns) {
+    // Apply decay first
+    const decayedPattern = calculateDecay(pattern, config);
+
+    // Check if below removal threshold
+    if (decayedPattern.confidence < config.decay.removeBelow) {
+      removedPatterns.push({
+        ...decayedPattern,
+        status: 'removed',
+        removedAt: now.toISOString(),
+        removalReason: 'confidence_too_low'
+      });
+    }
+    // Check if inactive for too long (6+ months)
+    else if (isInactive(decayedPattern, 6)) {
+      removedPatterns.push({
+        ...decayedPattern,
+        status: 'removed',
+        removedAt: now.toISOString(),
+        removalReason: 'inactive_too_long'
+      });
+    }
+    else {
+      activePatterns.push(decayedPattern);
+    }
+  }
+
+  return { activePatterns, removedPatterns };
+}
+```
+
+---
+
+## 🎨 Edge Cases & Solutions
+
+### Edge Case 1: Pattern Reconfirmed
+**Scenario:** Pattern confidence decayed to 60%, then user skips question again
+**Solution:** Boost confidence by renewalBoost amount (default +10 points)
+**Result:** Confidence increases to 70%, lastSeen updated
+
+### Edge Case 2: User-Approved Pattern
+**Scenario:** User manually approved pattern, should it decay?
+**Options:**
+- **Option A:** Never decay user-approved patterns (protect forever)
+- **Option B:** Decay slower (half rate)
+- **Option C:** Decay normally (user can re-approve)
+**Recommendation:** Option B - decay at half rate, prompt user to re-approve if drops below 70%
+
+### Edge Case 3: Rapid Confidence Loss
+**Scenario:** High-confidence pattern (95%) decays quickly
+**Solution:** Apply decay rate cap: max 20 points/month
+**Result:** Gradual decline prevents sudden filtering changes
+
+### Edge Case 4: Pattern Oscillation
+**Scenario:** Pattern decays, gets renewed, decays again (oscillates)
+**Solution:** Track `timesRenewed` counter
+- If timesRenewed > 3 in 6 months → stable pattern, reduce decay rate
+- If timesRenewed = 0 in 6 months → unstable pattern, increase decay rate
+
+### Edge Case 5: Multiple Sessions Per Day
+**Scenario:** User runs 5 interviews in one day, pattern renewed 5x
+**Solution:** Limit renewals to 1 per day per pattern
+**Result:** Prevents artificial confidence inflation
+
+---
+
+## ⚙️ Configuration Options
+
+### User-Configurable Settings
+
+Accessible via `adf config` → Learning System → Pattern Decay Settings:
+
+```
+Pattern Decay Settings
+─────────────────────────────────────────
+
+[x] Enable Pattern Decay
+    Patterns lose confidence over time when inactive
+
+Base Decay Rate: [15%] per month
+    How quickly patterns fade (0-50%)
+
+Minimum Active Confidence: [50] points
+    Patterns below this won't affect filtering
+
+Auto-Remove Below: [40] points
+    Automatically remove patterns below this
+
+Protect User-Approved: [x]
+    User-approved patterns decay at half rate
+
+Renewal Boost: [+10] points
+    Confidence boost when pattern reconfirmed
+
+[Save Changes] [Reset to Defaults]
+```
+
+### Default Configuration
+
+```javascript
+const DEFAULT_DECAY_CONFIG = {
+  enabled: true,
+  baseDecayRate: 0.15,              // 15% per month
+  minConfidenceThreshold: 50,       // Don't filter if confidence < 50
+  removeBelow: 40,                  // Auto-remove if < 40
+  decayCheckInterval: 'weekly',     // Run decay calc weekly
+  renewalBoost: 10,                 // +10 points on renewal
+  protectApproved: true,            // Approved patterns decay at 0.5x rate
+  highConfidenceThreshold: 90,      // Decay slower if >= 90
+  lowConfidenceThreshold: 75,       // Decay faster if < 75
+  maxRenewalsPerDay: 1              // Limit renewal inflation
+};
+```
+
+---
+
+## 📊 Performance Considerations
+
+### Computational Complexity
+
+**Decay Calculation:**
+- Time: O(n) where n = number of patterns
+- Space: O(1) per pattern
+- Expected patterns: 10-50 per user
+- **Performance:** < 10ms for 50 patterns
+
+**Optimization:**
+- Only calculate decay weekly (not every session)
+- Cache decay calculations for current session
+- Lazy evaluation: calc on first use of pattern
+
+### Storage Impact
+
+**Additional Fields:**
+- 4 new timestamps per pattern (4 × 24 bytes = 96 bytes)
+- 2 new numbers (decayRate, timesRenewed = 16 bytes)
+- **Total:** ~112 bytes per pattern
+- **For 50 patterns:** ~5.5 KB (negligible)
+
+---
+
+## 🧪 Testing Strategy
+
+### Unit Tests
+
+```javascript
+describe('Pattern Decay Algorithm', () => {
+  test('decays confidence over time', () => {
+    const pattern = createPattern({ confidence: 90, lastSeen: '2025-08-01' });
+    const decayed = calculateDecay(pattern, config);
+    expect(decayed.confidence).toBeLessThan(90);
+  });
+
+  test('high confidence patterns decay slower', () => {
+    const high = createPattern({ confidence: 95, lastSeen: '2025-08-01' });
+    const low = createPattern({ confidence: 65, lastSeen: '2025-08-01' });
+
+    const highDecayed = calculateDecay(high, config);
+    const lowDecayed = calculateDecay(low, config);
+
+    // Calculate decay percentages
+    const highDecayPct = (high.confidence - highDecayed.confidence) / high.confidence;
+    const lowDecayPct = (low.confidence - lowDecayed.confidence) / low.confidence;
+
+    expect(highDecayPct).toBeLessThan(lowDecayPct);
+  });
+
+  test('pattern renewal increases confidence', () => {
+    const pattern = createPattern({ confidence: 60 });
+    const renewed = renewPattern(pattern, config);
+    expect(renewed.confidence).toBe(70);
+    expect(renewed.timesRenewed).toBe(1);
+  });
+
+  test('removes patterns below threshold', () => {
+    const patterns = [
+      createPattern({ confidence: 35 }),  // Below removeBelow (40)
+      createPattern({ confidence: 60 })   // Above threshold
+    ];
+
+    const { activePatterns, removedPatterns } = cleanupPatterns(patterns, config);
+    expect(activePatterns).toHaveLength(1);
+    expect(removedPatterns).toHaveLength(1);
+  });
+
+  test('user-approved patterns decay slower', () => {
+    const approved = createPattern({
+      confidence: 90,
+      lastSeen: '2025-08-01',
+      userApproved: true
+    });
+    const normal = createPattern({
+      confidence: 90,
+      lastSeen: '2025-08-01',
+      userApproved: false
+    });
+
+    const approvedDecayed = calculateDecay(approved, {
+      ...config,
+      decay: { ...config.decay, protectApproved: true }
+    });
+    const normalDecayed = calculateDecay(normal, config);
+
+    expect(approvedDecayed.confidence).toBeGreaterThan(normalDecayed.confidence);
+  });
+});
+```
+
+### Integration Tests
+
+- Test decay across multiple sessions
+- Test renewal after decay
+- Test auto-removal of stale patterns
+- Test configuration changes affect decay
+- Test migration from non-decay patterns
+
+---
+
+## 🚀 Implementation Plan
+
+### Phase 1: Data Structure (Week 1)
+1. Add new fields to pattern schema
+2. Create migration script for existing patterns
+3. Add decay configuration to learning config
+4. Update storage.js to handle new fields
+
+### Phase 2: Core Algorithm (Week 1-2)
+1. Implement calculateDecay()
+2. Implement renewPattern()
+3. Implement cleanupPatterns()
+4. Add decay rate calculation logic
+5. Write comprehensive unit tests
+
+### Phase 3: Integration (Week 2)
+1. Integrate decay into pattern-detector.js
+2. Add decay check to interview startup
+3. Update skip-tracker.js to handle renewals
+4. Add decay scheduling (weekly cron)
+
+### Phase 4: UI & Configuration (Week 2)
+1. Add decay settings to adf config
+2. Show decay status in learning dashboard
+3. Display pattern age and decay info
+4. Add user prompts for low-confidence patterns
+
+### Phase 5: Testing & Polish (Week 3)
+1. Integration testing with real sessions
+2. Performance testing with 100+ patterns
+3. User acceptance testing
+4. Documentation updates
+
+---
+
+## 📈 Success Metrics
+
+**Quantitative:**
+- ✅ 90%+ of stale patterns removed within 6 months
+- ✅ Decay calculation completes in < 10ms for 50 patterns
+- ✅ Zero data corruption from decay operations
+- ✅ Patterns correctly renewed when reconfirmed
+
+**Qualitative:**
+- ✅ Users report improved filtering accuracy over time
+- ✅ No user complaints about "forgotten" recent preferences
+- ✅ Clear communication of pattern status and decay
+- ✅ User control over decay parameters feels intuitive
+
+---
+
+## 🔮 Future Enhancements
+
+### Smart Decay Rates
+- Use machine learning to optimize decay rates per user
+- Adjust decay based on project type (prototypes vs production)
+- Seasonal patterns (higher decay for prototype patterns)
+
+### Pattern Resurrection
+- Allow manual reactivation of removed patterns
+- Suggest patterns for reactivation based on new behavior
+- "Undo removal" feature
+
+### Confidence Visualization
+- Graph showing pattern confidence over time
+- Visual decay trajectory prediction
+- Highlight patterns approaching removal
+
+---
+
+## ✅ Design Approval Checklist
+
+- [x] Algorithm mathematically sound
+- [x] Edge cases identified and handled
+- [x] Performance acceptable (< 10ms)
+- [x] Storage impact minimal (< 10 KB)
+- [x] User control provided
+- [x] Testing strategy comprehensive
+- [x] Implementation plan clear
+- [x] Success metrics defined
+- [x] Documentation complete
+
+---
+
+**Status:** ✅ Design Complete, Ready for Implementation
+**Next Step:** Review with stakeholder, then proceed to implementation phase
+**Estimated Implementation Time:** 2-3 weeks