bmad-elsabro 1.0.0

package/BUDGET_ALERT_TESTING.md

@@ -0,0 +1,325 @@
+# Budget Alert System - Testing Guide
+
+This guide provides instructions for manually testing the token budget alert system.
+
+## Overview
+
+The Budget Alert System monitors token budgets and triggers alerts at configurable thresholds:
+- **50%** - Info alert (informational update)
+- **75%** - Warning alert (consider conserving tokens)
+- **90%** - Critical alert (approaching limit, switch to Ultra-Save mode recommended)
+- **100%** - Budget exceeded alert
+
+## Manual Testing Instructions
+
+### Test 1: Create Budget and Trigger Alerts
+
+1. **Open the Electron app**
+   ```bash
+   npm run electron:dev
+   ```
+
+2. **Open Developer Tools** (View → Toggle Developer Tools or Ctrl+Shift+I)
+
+3. **Create a test budget in the console**:
+   ```javascript
+   // Create a budget with 1000 token limit
+   await window.electron.invoke('token:create-budget', {
+     name: 'Test Budget',
+     limit_tokens: 1000,
+     period_start: Date.now(),
+     period_end: Date.now() + (7 * 24 * 60 * 60 * 1000), // 7 days from now
+     alert_threshold: 0.75 // Alert at 75%
+   });
+   ```
+
+4. **Start a tracking session**:
+   ```javascript
+   const result = await window.electron.invoke('token:start-session', {
+     workflowId: 'test-workflow',
+     metadata: { test: true }
+   });
+   console.log('Session started:', result.sessionId);
+   ```
+
+5. **Track token usage to trigger alerts**:
+
+   **Test Info Alert (50%)**:
+   ```javascript
+   // Use 500 tokens (50% of 1000)
+   await window.electron.invoke('token:track-usage', {
+     tokensUsed: 500,
+     operationType: 'test',
+     model: 'claude-sonnet-4'
+   });
+   // You should see an INFO alert in the console
+   ```
+
+   **Test Warning Alert (75%)**:
+   ```javascript
+   // Use 250 more tokens (total: 750 = 75%)
+   await window.electron.invoke('token:track-usage', {
+     tokensUsed: 250,
+     operationType: 'test'
+   });
+   // You should see a WARNING alert
+   ```
+
+   **Test Critical Alert (90%)**:
+   ```javascript
+   // Use 150 more tokens (total: 900 = 90%)
+   await window.electron.invoke('token:track-usage', {
+     tokensUsed: 150,
+     operationType: 'test'
+   });
+   // You should see a CRITICAL alert
+   ```
+
+   **Test Exceeded Alert (100%)**:
+   ```javascript
+   // Use 100+ more tokens (total: 1000+ = 100%+)
+   await window.electron.invoke('token:track-usage', {
+     tokensUsed: 150,
+     operationType: 'test'
+   });
+   // You should see an EXCEEDED alert
+   ```
+
+6. **Listen for alert events**:
+   ```javascript
+   window.electron.receive('token:budget-alert', (data) => {
+     console.log('🟡 BUDGET ALERT:', data);
+   });
+
+   window.electron.receive('token:budget-exceeded', (data) => {
+     console.log('🔴 BUDGET EXCEEDED:', data);
+   });
+   ```
+
+### Test 2: View Alert History
+
+1. **Get alert history**:
+   ```javascript
+   const result = await window.electron.invoke('token:get-alert-history', {
+     budgetName: 'Test Budget',
+     limit: 10
+   });
+   console.log('Alert History:', result.alerts);
+   ```
+
+2. **Verify alert data includes**:
+   - Alert ID
+   - Timestamp
+   - Severity level (info, warning, critical, exceeded)
+   - Budget name
+   - Tokens used vs limit
+   - Percentage used
+   - Recommendation (if applicable)
+   - Recommended mode (if applicable)
+
+### Test 3: Alert Configuration
+
+1. **Get current alert configuration**:
+   ```javascript
+   const config = await window.electron.invoke('token:get-alert-config');
+   console.log('Alert Config:', config);
+   ```
+
+2. **Update alert configuration** (e.g., add custom threshold):
+   ```javascript
+   await window.electron.invoke('token:update-alert-config', {
+     thresholds: [
+       { percentage: 0.25, severity: 'info', message: 'Budget 25% used', suppressRepeats: true },
+       { percentage: 0.50, severity: 'info', message: 'Budget 50% used', suppressRepeats: true },
+       { percentage: 0.75, severity: 'warning', message: 'Budget 75% used', suppressRepeats: true },
+       { percentage: 0.85, severity: 'warning', message: 'Budget 85% used', suppressRepeats: false },
+       { percentage: 0.95, severity: 'critical', message: 'Budget 95% used!', suppressRepeats: false },
+       { percentage: 1.0, severity: 'exceeded', message: 'Budget exceeded', suppressRepeats: false }
+     ]
+   });
+   ```
+
+### Test 4: Alert Acknowledgement
+
+1. **Get unacknowledged alerts**:
+   ```javascript
+   const result = await window.electron.invoke('token:get-alert-history', {
+     budgetName: 'Test Budget'
+   });
+   const unacknowledged = result.alerts.filter(a => !a.acknowledged);
+   console.log('Unacknowledged alerts:', unacknowledged);
+   ```
+
+2. **Acknowledge an alert**:
+   ```javascript
+   const alertId = unacknowledged[0].id;
+   await window.electron.invoke('token:acknowledge-alert', alertId);
+   ```
+
+3. **Verify acknowledgement**:
+   ```javascript
+   const updated = await window.electron.invoke('token:get-alert-history', {
+     budgetName: 'Test Budget'
+   });
+   const alert = updated.alerts.find(a => a.id === alertId);
+   console.log('Alert acknowledged:', alert.acknowledged); // Should be true
+   ```
+
+### Test 5: Alert Suppression
+
+This test verifies that repeat alerts are suppressed within the suppression window (default: 5 minutes).
+
+1. **Track usage to trigger 75% alert**:
+   ```javascript
+   // Create fresh budget
+   await window.electron.invoke('token:create-budget', {
+     name: 'Suppression Test',
+     limit_tokens: 1000,
+     period_start: Date.now(),
+   });
+
+   // Trigger 75% alert
+   await window.electron.invoke('token:track-usage', {
+     tokensUsed: 750,
+     operationType: 'test'
+   });
+   // Should see WARNING alert
+   ```
+
+2. **Immediately track more usage at same threshold**:
+   ```javascript
+   await window.electron.invoke('token:track-usage', {
+     tokensUsed: 10, // Still at 76%, within same threshold
+     operationType: 'test'
+   });
+   // Should NOT see another alert (suppressed)
+   ```
+
+3. **Wait 5+ minutes and trigger again** (or update config to shorter window):
+   ```javascript
+   // Update suppression window to 10 seconds for testing
+   await window.electron.invoke('token:update-alert-config', {
+     suppressionWindow: 10000 // 10 seconds
+   });
+
+   // Wait 10+ seconds, then trigger
+   setTimeout(async () => {
+     await window.electron.invoke('token:track-usage', {
+       tokensUsed: 5,
+       operationType: 'test'
+     });
+     // Should see alert again (suppression window expired)
+   }, 11000);
+   ```
+
+### Test 6: Budget Reset
+
+1. **Reset budget alerts**:
+   ```javascript
+   await window.electron.invoke('token:reset-budget-alerts', 'Test Budget');
+   ```
+
+2. **Verify alerts can trigger again**:
+   ```javascript
+   // Track usage at 50% again
+   await window.electron.invoke('token:track-usage', {
+     tokensUsed: 500,
+     operationType: 'test'
+   });
+   // Should see INFO alert even if it was triggered before
+   ```
+
+## Expected Behavior
+
+### Alert Severity Colors
+- **INFO** (🔵): Informational updates (50% used)
+- **WARNING** (🟡): Warning to conserve tokens (75% used)
+- **CRITICAL** (🟠): Critical - approaching limit (90% used)
+- **EXCEEDED** (🔴): Budget limit exceeded (100%+ used)
+
+### Alert Content
+Each alert should include:
+1. **Message**: Short alert message (e.g., "Budget 75% used")
+2. **Description**: Detailed info with token counts and percentage
+3. **Recommendation**: Suggested action (if applicable)
+   - At 75%: "Consider switching to Balanced mode to conserve tokens"
+   - At 90%+: "Switch to Ultra-Save mode to maximize remaining budget"
+4. **Recommended Mode**: Suggested efficiency mode (ultra-save, balanced, power)
+
+### Event Forwarding
+All alerts should be forwarded to the renderer process via IPC:
+- `token:budget-alert` - For info, warning, and critical alerts
+- `token:budget-exceeded` - For exceeded alerts
+
+## Troubleshooting
+
+### No alerts appearing
+1. Check that budget is active and period is valid
+2. Verify token usage is being tracked correctly
+3. Check console for error messages
+4. Ensure IPC channels are whitelisted in preload.ts
+
+### Duplicate alerts
+1. Check alert suppression settings
+2. Verify suppressionWindow is set appropriately
+3. Check that alerts are being acknowledged
+
+### Alerts not in history
+1. Verify alert system is initialized
+2. Check that alerts are being created (console logs)
+3. Ensure alert history limit is not too small
+
+## Integration with UI
+
+To integrate budget alerts with the notification system:
+
+```javascript
+// In your React component
+import { useNotifications } from '../hooks/useNotifications';
+
+// Listen for alerts
+useEffect(() => {
+  window.electron.receive('token:budget-alert', (data) => {
+    const severity = data.percentageUsed >= 90 ? 'error' :
+                     data.percentageUsed >= 75 ? 'warning' : 'info';
+
+    notify(severity, `Budget Alert: ${data.budgetName}`, {
+      description: `${data.usedTokens.toLocaleString()} / ${data.limitTokens.toLocaleString()} tokens used (${data.percentageUsed.toFixed(1)}%)`,
+      duration: 10000 // 10 seconds
+    });
+  });
+
+  window.electron.receive('token:budget-exceeded', (data) => {
+    error(`Budget Exceeded: ${data.budgetName}`, {
+      description: `You've exceeded your budget by ${Math.abs(data.remainingTokens).toLocaleString()} tokens`,
+      duration: 0, // Don't auto-dismiss
+      action: {
+        label: 'Increase Budget',
+        onClick: () => {
+          // Navigate to budget settings
+        }
+      }
+    });
+  });
+}, []);
+```
+
+## Success Criteria
+
+✅ Budget alert system should:
+1. Trigger alerts at configured thresholds (50%, 75%, 90%, 100%)
+2. Include detailed information (tokens used, remaining, percentage, recommendations)
+3. Suppress duplicate alerts within suppression window
+4. Store alert history with timestamps and acknowledgement status
+5. Forward events to renderer process via IPC
+6. Allow configuration of thresholds and suppression settings
+7. Provide budget reset functionality
+8. Integrate with existing notification system
+
+## Notes
+
+- Alert thresholds are configurable and can be customized per installation
+- Alert suppression prevents spam but can be disabled per threshold
+- Budget recommendations use mode-based token estimation for accuracy
+- All alerts are stored in memory and can be retrieved via IPC
+- Alert system works independently of UI - can be used headless