@libredb/studio 0.9.7 → 0.9.12

package/docs/backlogs/003-QUERY_TIME_MACHINE.md

@@ -1,183 +0,0 @@
-# Query Time Machine
-
-## Overview
-Compare query results across different points in time. Answer questions like "What did this data look like yesterday vs today?"
-
-## Problem Statement
-Data analysts frequently need to understand how data has changed over time:
-- "Why did our sales drop? What changed since last week?"
-- "This customer had 5 orders yesterday, now shows 3. What happened?"
-- "Compare today's active users with last month"
-
-Currently, users must:
-- Manually save query results to files/spreadsheets
-- Re-run queries and manually compare
-- Rely on memory or external documentation
-
-## Proposed Solution
-Automatic query result snapshots with visual diff comparison.
-
-## Features
-
-### 1. Automatic Snapshots
-- Save query results with timestamp automatically
-- Configurable retention (7 days, 30 days, custom)
-- Storage optimization (compress, deduplicate)
-- Manual "Pin" option for important snapshots
-
-### 2. Time Selector UI
-```
-┌─────────────────────────────────────────────────────────────┐
-│ Query Results                    [Today ▼] vs [Yesterday ▼] │
-├─────────────────────────────────────────────────────────────┤
-│ Timeline: ───●────●────●────●────●───                       │
-│          Dec 20  21   22   23   24                          │
-└─────────────────────────────────────────────────────────────┘
-```
-- Dropdown to select comparison dates
-- Visual timeline with snapshot points
-- Quick presets: "vs Yesterday", "vs Last Week", "vs Last Month"
-
-### 3. Diff Visualization
-```
-┌─────────────────────────────────────────────────────────────┐
-│ Changes Summary: +12 new rows, -3 removed, ~5 modified      │
-├─────────────────────────────────────────────────────────────┤
-│   name          │ orders (Dec 23) │ orders (Dec 24) │ diff  │
-├─────────────────┼─────────────────┼─────────────────┼───────┤
-│ + Alice Smith   │ -               │ 15              │ NEW   │
-│ ~ John Doe      │ 42              │ 45              │ +3    │
-│ ~ Jane Wilson   │ 38              │ 35              │ -3    │
-│ - Bob Johnson   │ 12              │ -               │ DEL   │
-└─────────────────────────────────────────────────────────────┘
-
-Legend: + Added  ~ Modified  - Removed
-```
-- Side-by-side comparison
-- Inline diff with color coding
-- Summary statistics (added/removed/modified counts)
-- Filter: "Show only changes"
-
-### 4. Trend Analysis
-- Mini sparkline showing value changes over time
-- "This value increased 23% over the last 7 snapshots"
-- Anomaly detection: "Unusual drop detected on Dec 22"
-
-### 5. Query History Integration
-- Link snapshots to specific query executions
-- "Show me all snapshots for this query"
-- Compare results from different query versions
-
-## Technical Considerations
-
-### Storage Strategy
-```typescript
-interface QuerySnapshot {
-  id: string;
-  queryHash: string;           // Hash of the SQL query
-  connectionId: string;
-  timestamp: Date;
-  rowCount: number;
-  checksum: string;            // For quick equality check
-  data: CompressedData;        // Compressed row data
-  metadata: {
-    executionTime: number;
-    fields: string[];
-  };
-}
-```
-
-### Storage Options
-1. **LocalStorage/IndexedDB** - Client-side, limited capacity
-2. **Server-side SQLite** - Dedicated snapshot database
-3. **Cloud Storage** - S3/GCS for large datasets
-4. **Hybrid** - Recent in browser, older in cloud
-
-### Diff Algorithm
-- Use row primary key for matching
-- Hash-based comparison for quick equality check
-- Deep comparison for modified detection
-- Efficient diff for large datasets (streaming)
-
-### API Endpoints
-```
-GET /api/snapshots?queryHash=xxx
-POST /api/snapshots
-GET /api/snapshots/:id
-DELETE /api/snapshots/:id
-GET /api/snapshots/diff?from=id1&to=id2
-```
-
-## UI Components
-
-### New Components
-- `TimelineSelector.tsx` - Visual timeline with snapshot points
-- `SnapshotDiff.tsx` - Side-by-side comparison view
-- `SnapshotManager.tsx` - List and manage saved snapshots
-- `TrendSparkline.tsx` - Mini trend visualization
-
-### Integration Points
-- Results toolbar: "Compare with..." button
-- Query history: "View snapshots" action
-- New bottom panel tab: "Time Machine"
-
-## User Flow
-
-```
-1. User runs query
-   ↓
-2. System auto-saves snapshot (if enabled)
-   ↓
-3. User clicks "Compare with..." or "Time Machine" tab
-   ↓
-4. User selects comparison date/snapshot
-   ↓
-5. System shows diff view with changes highlighted
-   ↓
-6. User can drill down into specific changes
-```
-
-## Configuration Options
-- Auto-snapshot: On/Off
-- Retention period: 7/30/90 days
-- Max snapshots per query: 10/50/100
-- Storage limit: 50MB/200MB/1GB
-- Compression level: Low/Medium/High
-
-## Edge Cases
-- Query with no primary key (use row index)
-- Schema changes between snapshots
-- Very large result sets (pagination/sampling)
-- Identical results (show "No changes" state)
-
-## Empty States
-- "No snapshots yet. Run a query to create your first snapshot."
-- "This is the first snapshot. Run the query again later to compare."
-- "No changes between selected snapshots."
-
-## Acceptance Criteria
-- [ ] Query results are automatically saved as snapshots
-- [ ] User can select two snapshots to compare
-- [ ] Diff view shows added, removed, and modified rows
-- [ ] Changes are color-coded for easy identification
-- [ ] Summary shows counts of each change type
-- [ ] User can filter to show only changes
-- [ ] Snapshots can be manually pinned/deleted
-- [ ] Storage limits are enforced
-- [ ] Works with PostgreSQL, MySQL, SQLite
-
-## Dependencies
-- Query execution system
-- Storage system (IndexedDB or server-side)
-- Diff algorithm implementation
-
-## Estimated Effort
-High complexity
-
-## Priority
-P2 - Differentiating feature
-
-## Related Features
-- Query History (existing)
-- Data Visualization (existing)
-- AI Data Storyteller (planned)

package/docs/backlogs/004-AI_DATA_STORYTELLER.md

@@ -1,292 +0,0 @@
-# AI Data Storyteller
-
-## Overview
-Transform raw query results into meaningful narratives, insights, and executive summaries using AI. Move beyond "what" the data shows to "why it matters."
-
-## Problem Statement
-Data analysts spend significant time:
-- Interpreting query results manually
-- Writing reports explaining data findings
-- Identifying trends and anomalies by eye
-- Translating technical data into business language
-
-Non-technical stakeholders struggle to:
-- Understand raw data tables
-- Identify what's important in large datasets
-- Make decisions based on numbers alone
-
-## Proposed Solution
-AI-powered data interpretation that automatically generates:
-- Natural language summaries
-- Key insights and anomalies
-- Trend analysis
-- Executive-ready reports
-
-## Features
-
-### 1. One-Click Insights
-```
-┌─────────────────────────────────────────────────────────────┐
-│ Results (1,247 rows)                    [✨ Tell me a story]│
-├─────────────────────────────────────────────────────────────┤
-│                                                             │
-│  ┌─────────────────────────────────────────────────────┐   │
-│  │ 📊 Data Story                                        │   │
-│  │                                                      │   │
-│  │ Key Findings:                                        │   │
-│  │ • Sales increased 23% compared to last month        │   │
-│  │ • Top performer: Electronics category (+45%)        │   │
-│  │ • Warning: Returns in Clothing up 12%               │   │
-│  │                                                      │   │
-│  │ Anomalies Detected:                                  │   │
-│  │ • Dec 15: Unusual spike in orders (Black Friday?)   │   │
-│  │ • Region "West" underperforming by 2 std deviations │   │
-│  │                                                      │   │
-│  │ Recommendation:                                      │   │
-│  │ "Investigate Clothing returns - may indicate        │   │
-│  │  quality or sizing issues"                          │   │
-│  └─────────────────────────────────────────────────────┘   │
-└─────────────────────────────────────────────────────────────┘
-```
-
-### 2. Smart Question Interface
-Instead of writing SQL, ask questions in natural language:
-```
-User: "What's interesting about this data?"
-AI: "I notice three key patterns..."
-
-User: "Why did sales drop in March?"
-AI: "Looking at the data, the drop correlates with..."
-
-User: "Summarize this for my manager"
-AI: "Executive Summary: Q4 performance exceeded targets..."
-```
-
-### 3. Automatic Analysis Types
-
-#### Statistical Summary
-- Min, max, mean, median, std deviation
-- Distribution analysis
-- Correlation between columns
-
-#### Trend Detection
-- Time-series pattern recognition
-- Seasonality identification
-- Growth rate calculations
-
-#### Anomaly Detection
-- Outlier identification
-- Unexpected patterns
-- Missing data patterns
-
-#### Comparative Analysis
-- Group comparisons
-- Period-over-period changes
-- Benchmark comparisons
-
-### 4. Report Generation
-```
-┌─────────────────────────────────────────────────────────────┐
-│ Generate Report                                             │
-├─────────────────────────────────────────────────────────────┤
-│ Format:    [Executive Summary ▼]                            │
-│ Audience:  [Non-Technical ▼]                                │
-│ Include:   [✓] Charts  [✓] Key Metrics  [ ] Raw Data       │
-│ Tone:      [Professional ▼]                                 │
-│                                                             │
-│                              [Generate PDF] [Copy Markdown] │
-└─────────────────────────────────────────────────────────────┘
-```
-
-Report templates:
-- Executive Summary (1 page, high-level)
-- Detailed Analysis (comprehensive)
-- Technical Report (for data teams)
-- Presentation Slides (bullet points + charts)
-
-### 5. Interactive Exploration
-```
-"Tell me more about the Electronics category"
-     ↓
-AI drills down and shows subcategory breakdown
-
-"Compare this with last year"
-     ↓
-AI generates year-over-year comparison
-
-"What should we do about this?"
-     ↓
-AI provides actionable recommendations
-```
-
-## Technical Considerations
-
-### AI Integration
-```typescript
-interface StorytellerRequest {
-  result: QueryResult;
-  schemaContext: TableSchema[];
-  queryContext: string;           // The SQL that generated this
-  analysisType: 'summary' | 'trends' | 'anomalies' | 'full';
-  audience: 'technical' | 'business' | 'executive';
-  previousContext?: string;       // For follow-up questions
-}
-
-interface StorytellerResponse {
-  summary: string;
-  keyFindings: Finding[];
-  anomalies: Anomaly[];
-  recommendations: string[];
-  charts: ChartSuggestion[];
-  confidence: number;
-}
-```
-
-### LLM Prompting Strategy
-1. **Context Building**
-   - Schema information (table/column names, types)
-   - Query intent (what user was looking for)
-   - Result statistics (row count, value ranges)
-
-2. **Analysis Prompt**
-   ```
-   You are a data analyst. Given this query result:
-   - Schema: [tables and columns]
-   - Query: [SQL]
-   - Sample data: [first 100 rows]
-   - Statistics: [aggregates]
-
-   Provide insights in this format:
-   1. Key Findings (3-5 bullet points)
-   2. Anomalies (if any)
-   3. Trends (if time-series)
-   4. Recommendations
-   ```
-
-3. **Audience Adaptation**
-   - Technical: Include SQL, statistical terms
-   - Business: Focus on metrics, KPIs
-   - Executive: High-level, actionable
-
-### Data Sampling for Large Results
-- First 100 rows + random sample of 100
-- Statistical aggregates (min, max, avg, percentiles)
-- Column value distributions
-- Never send full dataset to LLM
-
-### Streaming Response
-- Stream AI response for better UX
-- Show "Analyzing..." with progress
-- Typewriter effect for narrative
-
-## UI Components
-
-### New Components
-- `DataStoryPanel.tsx` - Main storyteller interface
-- `InsightCard.tsx` - Individual insight display
-- `AnomalyBadge.tsx` - Anomaly highlight component
-- `ReportGenerator.tsx` - Report export modal
-- `QuestionInput.tsx` - Natural language question input
-
-### Integration Points
-- Results toolbar: "Tell me a story" button
-- New bottom panel mode or floating panel
-- Context menu on columns: "Analyze this column"
-- Query editor: Natural language input mode
-
-## User Flow
-
-```
-1. User runs query, sees results
-   ↓
-2. Clicks "✨ Tell me a story" button
-   ↓
-3. AI analyzes data (streaming response)
-   ↓
-4. Insights panel shows findings
-   ↓
-5. User can ask follow-up questions
-   ↓
-6. User exports as report (optional)
-```
-
-## Example Outputs
-
-### For Sales Data
-```
-📊 Sales Analysis - December 2024
-
-Key Findings:
-• Total revenue: $1.2M (+18% vs November)
-• Best day: December 15 ($89K) - likely Black Friday effect
-• Top category: Electronics (42% of revenue)
-
-⚠️ Anomalies:
-• Unusually high returns in Clothing (12% vs 5% average)
-• Western region 23% below other regions
-
-📈 Trends:
-• Steady growth since October (+8% MoM)
-• Weekend sales 34% higher than weekdays
-
-💡 Recommendations:
-1. Investigate Clothing returns - possible sizing issue
-2. Review Western region - may need marketing push
-```
-
-### For User Analytics
-```
-📊 User Activity Report
-
-Key Findings:
-• 15,234 active users this week (+5%)
-• Average session: 12 minutes
-• Most active: Tuesday 2-4 PM
-
-⚠️ Concerns:
-• 23% bounce rate on mobile (desktop: 8%)
-• New user retention dropped to 34%
-
-💡 Recommendations:
-1. Prioritize mobile experience optimization
-2. Review onboarding flow for new users
-```
-
-## Configuration Options
-- Default analysis depth: Quick / Standard / Deep
-- Preferred language style: Casual / Professional / Academic
-- Auto-analyze on query: On / Off
-- Include chart suggestions: On / Off
-
-## Privacy & Security
-- Data sampling (never full dataset to LLM)
-- Option to disable for sensitive queries
-- No PII in prompts (mask if detected)
-- Local-only mode (Ollama) for sensitive data
-
-## Acceptance Criteria
-- [ ] "Tell me a story" button appears on results
-- [ ] AI generates natural language summary
-- [ ] Key findings are extracted and highlighted
-- [ ] Anomalies are detected and flagged
-- [ ] User can ask follow-up questions
-- [ ] Reports can be exported as PDF/Markdown
-- [ ] Analysis adapts to audience type
-- [ ] Streaming response for better UX
-- [ ] Works with existing LLM providers
-
-## Dependencies
-- LLM integration (existing)
-- Data Visualization (for chart suggestions)
-- Export functionality
-
-## Estimated Effort
-High complexity
-
-## Priority
-P1 - Core differentiator
-
-## Related Features
-- AI Query Assistant (existing)
-- Data Visualization (existing)
-- Query Time Machine (planned)