@hailer/mcp 0.0.1

package/.claude/skills/preview-insight-skill/SKILL.md

@@ -0,0 +1,1290 @@
+---
+name: Preview Hailer Insights
+description: Complete guide for testing SQL queries before saving insights - use when debugging queries or designing reports
+---
+
+# Preview Hailer Insights - Complete Guide
+
+Complete reference for testing SQL queries on Hailer workflow data using the `preview_insight` MCP tool.
+
+## Table of Contents
+1. [Quick Reference](#quick-reference)
+2. [Overview](#overview)
+3. [Core Concepts](#core-concepts)
+4. [Basic Usage](#basic-usage)
+5. [Development Workflow](#development-workflow)
+6. [Testing Strategies](#testing-strategies)
+7. [Debugging Queries](#debugging-queries)
+8. [Common Scenarios](#common-scenarios)
+9. [Performance Considerations](#performance-considerations)
+10. [Best Practices](#best-practices)
+11. [Troubleshooting](#troubleshooting)
+
+## Quick Reference
+
+**Test Simple Query:**
+
+```javascript
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [
+      { name: 'title', meta: 'name' },
+      { name: 'dueDate', fieldId: 'due-date-field-id' }
+    ]
+  }],
+  query: 'SELECT title, dueDate FROM tasks WHERE dueDate > "2024-01-01"'
+})
+```
+
+**Test JOIN Query:**
+
+```javascript
+preview_insight({
+  sources: [
+    {
+      name: 'tasks',
+      workflowId: 'tasks-workflow-id',
+      fields: [
+        { name: 'taskName', meta: 'name' },
+        { name: 'projectLink', fieldId: 'project-link-field-id' }
+      ]
+    },
+    {
+      name: 'projects',
+      workflowId: 'projects-workflow-id',
+      fields: [
+        { name: 'id', meta: '_id' },
+        { name: 'projectName', meta: 'name' }
+      ]
+    }
+  ],
+  query: `
+    SELECT tasks.taskName, projects.projectName
+    FROM tasks
+    LEFT JOIN projects ON tasks.projectLink = projects.id
+  `
+})
+```
+
+**Preview Existing Insight Changes:**
+
+```javascript
+preview_insight({
+  insightId: 'existing-insight-id',  // Preview changes to existing insight
+  sources: [...],                     // New sources
+  query: 'SELECT ...'                 // New query
+})
+```
+
+**Key Features:**
+- Tests SQL queries without creating insights
+- Shows sample results (~20 activities)
+- Fast iteration for query development
+- Catches syntax errors immediately
+- Safe testing - doesn't save anything
+
+## Overview
+
+**What is Preview?**
+
+Preview lets you test SQL queries on a small sample of your workflow data before creating or updating an insight. It's the essential tool for insight development and debugging.
+
+**Think of it as:**
+- SQL query validation
+- Dry run before committing
+- Interactive query builder
+- Debug console for insights
+
+**Key Differences from create_insight:**
+
+| Feature | preview_insight | create_insight |
+|---------|----------------|----------------|
+| Saves insight | ❌ No | ✅ Yes |
+| Data returned | ~20 activities | Not directly |
+| Use case | Testing/debugging | Production use |
+| Speed | Very fast | Slower (saves) |
+| Iterations | Unlimited | Should minimize |
+
+**When to Use:**
+- Before creating new insights
+- When debugging SQL errors
+- Testing field mappings
+- Verifying JOIN conditions
+- Checking WHERE clause logic
+- Validating aggregations
+- Iterating on query design
+
+**When NOT to Use:**
+- Getting full dataset (use get_insight_data)
+- Running production reports (create insight first)
+- When query already works (just create it)
+
+## Core Concepts
+
+### Preview vs Create Workflow
+
+**Recommended Flow:**
+
+```
+1. Design query structure
+   ↓
+2. Preview with minimal data
+   ↓
+3. Fix errors, iterate
+   ↓
+4. Preview until successful
+   ↓
+5. Create insight
+   ↓
+6. Get full data with get_insight_data
+```
+
+**Anti-Pattern (Don't Do This):**
+
+```
+1. Create insight immediately
+   ↓
+2. Get error
+   ↓
+3. Debug by trial and error
+   ↓
+4. Remove and recreate multiple times
+```
+
+### Sample Data Limitation
+
+**Important:** Preview returns only ~20 activities per source.
+
+**What this means:**
+- Quick feedback on query structure
+- May not show all edge cases
+- Aggregations may not be representative
+- JOINs might miss some relationships
+
+**Good for:**
+- SQL syntax validation
+- Field mapping verification
+- JOIN logic testing
+- Column name checking
+
+**Not good for:**
+- Accurate COUNT results
+- Complete data analysis
+- Finding rare edge cases
+- Production reporting
+
+### Security Note
+
+**Regular Users:**
+- Can only preview with sources from existing insights they have access to
+- Cannot test arbitrary workflow combinations
+- Security restriction to prevent unauthorized data access
+
+**Workspace Admins:**
+- Can preview with any workflow sources
+- Full testing capabilities
+- No source restrictions
+
+## Basic Usage
+
+### Example 1: Test Simple Query
+
+```javascript
+// Step 1: Define sources and query
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: '68446dc05b30685f67c6fcd4',
+    fields: [
+      { name: 'title', meta: 'name' },
+      { name: 'priority', fieldId: 'priority-field-id' }
+    ]
+  }],
+  query: 'SELECT title, priority FROM tasks'
+})
+
+// Response:
+// ✅ Insight Preview Successful
+//
+// Query Results (showing up to 20 activities):
+// | title | priority |
+// |-------|----------|
+// | Task 1 | High |
+// | Task 2 | Medium |
+// ...
+```
+
+### Example 2: Test with Filter
+
+```javascript
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [
+      { name: 'title', meta: 'name' },
+      { name: 'priority', fieldId: 'priority-field-id' },
+      { name: 'status', meta: 'phase' }
+    ]
+  }],
+  query: `
+    SELECT title, priority, status
+    FROM tasks
+    WHERE priority = 'High' AND status != 'Done'
+  `
+})
+```
+
+### Example 3: Test JOIN
+
+```javascript
+preview_insight({
+  sources: [
+    {
+      name: 'tasks',
+      workflowId: 'tasks-workflow-id',
+      fields: [
+        { name: 'taskName', meta: 'name' },
+        { name: 'projectId', fieldId: 'project-field-id' }
+      ]
+    },
+    {
+      name: 'projects',
+      workflowId: 'projects-workflow-id',
+      fields: [
+        { name: 'id', meta: '_id' },
+        { name: 'projectName', meta: 'name' }
+      ]
+    }
+  ],
+  query: `
+    SELECT
+      tasks.taskName,
+      projects.projectName
+    FROM tasks
+    LEFT JOIN projects ON tasks.projectId = projects.id
+  `
+})
+```
+
+### Example 4: Test Aggregation
+
+```javascript
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [
+      { name: 'priority', fieldId: 'priority-field-id' }
+    ]
+  }],
+  query: `
+    SELECT priority, COUNT(*) AS count
+    FROM tasks
+    GROUP BY priority
+    ORDER BY count DESC
+  `
+})
+
+// Note: COUNT will only reflect ~20 sampled activities
+```
+
+## Development Workflow
+
+### Workflow 1: Build Query Iteratively
+
+**Start simple, add complexity gradually:**
+
+```javascript
+// Step 1: Start with SELECT *
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [
+      { name: 'title', meta: 'name' }
+    ]
+  }],
+  query: 'SELECT * FROM tasks'
+})
+// ✅ Works? Continue.
+
+// Step 2: Add specific columns
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [
+      { name: 'title', meta: 'name' },
+      { name: 'priority', fieldId: 'priority-field-id' }
+    ]
+  }],
+  query: 'SELECT title, priority FROM tasks'
+})
+// ✅ Works? Continue.
+
+// Step 3: Add WHERE clause
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [
+      { name: 'title', meta: 'name' },
+      { name: 'priority', fieldId: 'priority-field-id' }
+    ]
+  }],
+  query: 'SELECT title, priority FROM tasks WHERE priority = "High"'
+})
+// ✅ Works? Continue.
+
+// Step 4: Add ORDER BY
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [
+      { name: 'title', meta: 'name' },
+      { name: 'priority', fieldId: 'priority-field-id' },
+      { name: 'created', meta: 'created' }
+    ]
+  }],
+  query: `
+    SELECT title, priority, created
+    FROM tasks
+    WHERE priority = 'High'
+    ORDER BY created DESC
+  `
+})
+// ✅ Works? Create insight!
+```
+
+### Workflow 2: Test JOIN Relationships
+
+```javascript
+// Step 1: Verify both sources independently
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [
+      { name: 'id', meta: '_id' },
+      { name: 'title', meta: 'name' }
+    ]
+  }],
+  query: 'SELECT * FROM tasks LIMIT 5'
+})
+
+preview_insight({
+  sources: [{
+    name: 'projects',
+    workflowId: 'projects-workflow-id',
+    fields: [
+      { name: 'id', meta: '_id' },
+      { name: 'name', meta: 'name' }
+    ]
+  }],
+  query: 'SELECT * FROM projects LIMIT 5'
+})
+
+// Step 2: Test JOIN without filters
+preview_insight({
+  sources: [
+    {
+      name: 'tasks',
+      workflowId: 'tasks-workflow-id',
+      fields: [
+        { name: 'title', meta: 'name' },
+        { name: 'projectId', fieldId: 'project-field-id' }
+      ]
+    },
+    {
+      name: 'projects',
+      workflowId: 'projects-workflow-id',
+      fields: [
+        { name: 'id', meta: '_id' },
+        { name: 'projectName', meta: 'name' }
+      ]
+    }
+  ],
+  query: `
+    SELECT tasks.title, projects.projectName
+    FROM tasks
+    LEFT JOIN projects ON tasks.projectId = projects.id
+  `
+})
+
+// Step 3: Add filters and sorting
+preview_insight({
+  sources: [...],
+  query: `
+    SELECT tasks.title, projects.projectName
+    FROM tasks
+    LEFT JOIN projects ON tasks.projectId = projects.id
+    WHERE projects.projectName IS NOT NULL
+    ORDER BY projects.projectName, tasks.title
+  `
+})
+```
+
+### Workflow 3: Fix Field Mapping Errors
+
+```javascript
+// Step 1: Preview with suspected incorrect field
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [
+      { name: 'priority', fieldId: 'wrong-field-id' }  // Wrong ID
+    ]
+  }],
+  query: 'SELECT priority FROM tasks'
+})
+// ❌ Returns null or unexpected values
+
+// Step 2: Get correct field ID
+const schema = get_workflow_schema({
+  workflowId: 'tasks-workflow-id'
+})
+const priorityField = schema.fields.find(f => f.key === 'priority')
+console.log('Correct field ID:', priorityField._id)
+
+// Step 3: Retry with correct field ID
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [
+      { name: 'priority', fieldId: priorityField._id }  // Correct ID
+    ]
+  }],
+  query: 'SELECT priority FROM tasks'
+})
+// ✅ Works!
+```
+
+## Testing Strategies
+
+### Strategy 1: Column-by-Column Verification
+
+**Test each field individually:**
+
+```javascript
+// Test field 1
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [{ name: 'title', meta: 'name' }]
+  }],
+  query: 'SELECT title FROM tasks LIMIT 5'
+})
+
+// Test field 2
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [{ name: 'priority', fieldId: 'priority-field-id' }]
+  }],
+  query: 'SELECT priority FROM tasks LIMIT 5'
+})
+
+// Combine once both work
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [
+      { name: 'title', meta: 'name' },
+      { name: 'priority', fieldId: 'priority-field-id' }
+    ]
+  }],
+  query: 'SELECT title, priority FROM tasks LIMIT 5'
+})
+```
+
+### Strategy 2: WHERE Clause Testing
+
+**Test filters incrementally:**
+
+```javascript
+// No filter
+preview_insight({
+  sources: [...],
+  query: 'SELECT title, priority FROM tasks'
+})
+// Note: How many rows returned?
+
+// Add first filter
+preview_insight({
+  sources: [...],
+  query: 'SELECT title, priority FROM tasks WHERE priority = "High"'
+})
+// Note: Did rows decrease appropriately?
+
+// Add second filter
+preview_insight({
+  sources: [...],
+  query: `
+    SELECT title, priority, status
+    FROM tasks
+    WHERE priority = 'High' AND status != 'Done'
+  `
+})
+// Note: Further decrease makes sense?
+```
+
+### Strategy 3: JOIN Direction Testing
+
+**Test different JOIN types:**
+
+```javascript
+// Test INNER JOIN (only matching)
+preview_insight({
+  sources: [...],
+  query: `
+    SELECT tasks.title, projects.projectName
+    FROM tasks
+    INNER JOIN projects ON tasks.projectId = projects.id
+  `
+})
+// Note: Row count
+
+// Test LEFT JOIN (all tasks)
+preview_insight({
+  sources: [...],
+  query: `
+    SELECT tasks.title, projects.projectName
+    FROM tasks
+    LEFT JOIN projects ON tasks.projectId = projects.id
+  `
+})
+// Note: Should have more rows (tasks without projects included)
+
+// Test RIGHT JOIN (all projects)
+preview_insight({
+  sources: [...],
+  query: `
+    SELECT tasks.title, projects.projectName
+    FROM tasks
+    RIGHT JOIN projects ON tasks.projectId = projects.id
+  `
+})
+// Note: Different row count (projects without tasks included)
+```
+
+### Strategy 4: Aggregation Validation
+
+**Check aggregation logic:**
+
+```javascript
+// Simple count
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [{ name: 'id', meta: '_id' }]
+  }],
+  query: 'SELECT COUNT(*) AS total FROM tasks'
+})
+// Note: Total (but remember, only ~20 activities sampled)
+
+// Count by group
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [{ name: 'priority', fieldId: 'priority-field-id' }]
+  }],
+  query: `
+    SELECT priority, COUNT(*) AS count
+    FROM tasks
+    GROUP BY priority
+  `
+})
+// Note: Distributions (sample only)
+
+// Verify with detail query
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [{ name: 'priority', fieldId: 'priority-field-id' }]
+  }],
+  query: 'SELECT priority FROM tasks'
+})
+// Note: Manually count to verify GROUP BY logic
+```
+
+## Debugging Queries
+
+### Debug Pattern 1: SQL Syntax Errors
+
+**Error:** SQL syntax error
+
+**Debug steps:**
+
+```javascript
+// 1. Simplify to minimal query
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [{ name: 'title', meta: 'name' }]
+  }],
+  query: 'SELECT title FROM tasks'
+})
+// ✅ Works? Syntax is valid.
+
+// 2. Add back pieces one at a time
+preview_insight({
+  sources: [...],
+  query: 'SELECT title FROM tasks WHERE priority = "High"'
+})
+// ❌ Error? Issue is in WHERE clause.
+
+// 3. Fix WHERE clause (quotes)
+preview_insight({
+  sources: [...],
+  query: "SELECT title FROM tasks WHERE priority = 'High'"
+})
+// ✅ Works? Continue adding.
+```
+
+### Debug Pattern 2: Column Not Found
+
+**Error:** Column name doesn't exist
+
+**Debug steps:**
+
+```javascript
+// 1. List all available columns
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [
+      { name: 'field1', meta: 'name' },
+      { name: 'field2', fieldId: 'field-id' }
+    ]
+  }],
+  query: 'SELECT * FROM tasks LIMIT 1'
+})
+// Note: What columns appear in result?
+
+// 2. Use correct column name
+preview_insight({
+  sources: [...],
+  query: 'SELECT field1, field2 FROM tasks'
+})
+// Must match name in field mapping
+```
+
+### Debug Pattern 3: JOIN Returns No Rows
+
+**Problem:** JOIN returns empty result
+
+**Debug steps:**
+
+```javascript
+// 1. Check both tables have data
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [{ name: 'id', meta: '_id' }]
+  }],
+  query: 'SELECT COUNT(*) FROM tasks'
+})
+
+preview_insight({
+  sources: [{
+    name: 'projects',
+    workflowId: 'projects-workflow-id',
+    fields: [{ name: 'id', meta: '_id' }]
+  }],
+  query: 'SELECT COUNT(*) FROM projects'
+})
+
+// 2. Check JOIN keys have values
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [
+      { name: 'projectId', fieldId: 'project-field-id' }
+    ]
+  }],
+  query: 'SELECT projectId FROM tasks WHERE projectId IS NOT NULL'
+})
+
+// 3. Use LEFT JOIN to see all tasks
+preview_insight({
+  sources: [
+    {
+      name: 'tasks',
+      workflowId: 'tasks-workflow-id',
+      fields: [
+        { name: 'title', meta: 'name' },
+        { name: 'projectId', fieldId: 'project-field-id' }
+      ]
+    },
+    {
+      name: 'projects',
+      workflowId: 'projects-workflow-id',
+      fields: [
+        { name: 'id', meta: '_id' },
+        { name: 'projectName', meta: 'name' }
+      ]
+    }
+  ],
+  query: `
+    SELECT tasks.title, tasks.projectId, projects.projectName
+    FROM tasks
+    LEFT JOIN projects ON tasks.projectId = projects.id
+  `
+})
+// Shows which tasks don't match projects
+```
+
+### Debug Pattern 4: Unexpected NULL Values
+
+**Problem:** Fields show NULL when they shouldn't
+
+**Debug steps:**
+
+```javascript
+// 1. Check field mapping
+const schema = get_workflow_schema({
+  workflowId: 'workflow-id'
+})
+console.log('Available fields:', schema.fields.map(f => ({
+  id: f._id,
+  key: f.key,
+  label: f.label
+})))
+
+// 2. Verify field ID is correct
+const field = schema.fields.find(f => f.key === 'priority')
+console.log('Priority field ID:', field._id)
+
+// 3. Test with correct field ID
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'workflow-id',
+    fields: [
+      { name: 'priority', fieldId: field._id }  // Correct ID
+    ]
+  }],
+  query: 'SELECT priority FROM tasks'
+})
+
+// 4. Check if field values actually exist
+list_activities({
+  workflowId: 'workflow-id',
+  limit: 5,
+  returnFlat: true
+})
+// Verify activities have priority values
+```
+
+## Common Scenarios
+
+### Scenario 1: Design New Report
+
+```javascript
+// Goal: Create "High Priority Overdue Tasks" report
+
+// Step 1: Get field IDs
+const schema = get_workflow_schema({
+  workflowId: 'tasks-workflow-id'
+})
+
+// Step 2: Preview basic query
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [
+      { name: 'title', meta: 'name' },
+      { name: 'priority', fieldId: 'priority-field-id' },
+      { name: 'dueDate', fieldId: 'due-date-field-id' }
+    ]
+  }],
+  query: `
+    SELECT title, priority, dueDate
+    FROM tasks
+    WHERE priority = 'High'
+      AND dueDate < date('now')
+    ORDER BY dueDate ASC
+  `
+})
+// ✅ Works? Create insight.
+
+// Step 3: Create the insight
+create_insight({
+  name: 'High Priority Overdue Tasks',
+  sources: [...],  // Same as preview
+  query: '...'     // Same as preview
+})
+```
+
+### Scenario 2: Modify Existing Insight
+
+```javascript
+// Goal: Update insight to include status field
+
+// Step 1: Preview with new field
+preview_insight({
+  insightId: 'existing-insight-id',  // Optional: reference existing
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [
+      { name: 'title', meta: 'name' },
+      { name: 'priority', fieldId: 'priority-field-id' },
+      { name: 'status', meta: 'phase' }  // New field
+    ]
+  }],
+  query: `
+    SELECT title, priority, status
+    FROM tasks
+    WHERE priority = 'High'
+  `
+})
+// ✅ Works? Update insight.
+
+// Step 2: Update insight (use update_insight if available)
+// Or remove and recreate with new query
+```
+
+### Scenario 3: Debug Broken Insight
+
+```javascript
+// Goal: Fix insight that's returning errors
+
+// Step 1: Get insight definition
+const insights = list_insights()
+const broken = insights.find(i => i.name === 'Broken Report')
+
+// Step 2: Preview with same query
+preview_insight({
+  sources: broken.sources,
+  query: broken.query
+})
+// ❌ Error? Now you can debug interactively.
+
+// Step 3: Simplify query to find issue
+preview_insight({
+  sources: broken.sources,
+  query: 'SELECT * FROM tasks LIMIT 5'
+})
+// ✅ Works? Issue is in complex query logic.
+
+// Step 4: Add back complexity piece by piece
+// ... debug until you find the problem
+```
+
+### Scenario 4: Test Performance
+
+```javascript
+// Goal: Ensure query performs well on sample data
+
+// Test 1: Simple query (should be fast)
+preview_insight({
+  sources: [{
+    name: 'tasks',
+    workflowId: 'tasks-workflow-id',
+    fields: [{ name: 'title', meta: 'name' }]
+  }],
+  query: 'SELECT title FROM tasks'
+})
+// Note: Response time
+
+// Test 2: Complex query (might be slower)
+preview_insight({
+  sources: [
+    { name: 'tasks', ... },
+    { name: 'projects', ... },
+    { name: 'teams', ... }
+  ],
+  query: `
+    SELECT ...
+    FROM tasks t
+    JOIN projects p ON t.projectId = p.id
+    JOIN teams tm ON p.teamId = tm.id
+    WHERE ...
+    GROUP BY ...
+    ORDER BY ...
+  `
+})
+// Note: Response time (if slow on sample, will be very slow on full data)
+```
+
+## Performance Considerations
+
+### Preview Performance
+
+**Preview is fast because:**
+- Limited to ~20 activities per source
+- No data persistence
+- Simple validation
+- Quick response
+
+**Even preview can be slow if:**
+- Many sources (complex JOINs)
+- Complex aggregations
+- Large result set (many columns)
+- Server overloaded
+
+### Optimizing Queries
+
+**Fast queries:**
+```sql
+SELECT title, priority FROM tasks LIMIT 10
+SELECT COUNT(*) FROM tasks
+SELECT * FROM tasks WHERE status = 'Open'
+```
+
+**Slower queries:**
+```sql
+-- Many JOINs
+SELECT * FROM t1 JOIN t2 JOIN t3 JOIN t4 JOIN t5
+
+-- Complex aggregations
+SELECT a, b, c, COUNT(*), SUM(x), AVG(y), MIN(z), MAX(w)
+FROM tasks
+GROUP BY a, b, c
+HAVING COUNT(*) > 10
+
+-- String operations
+SELECT title || ' - ' || description || ' - ' || notes FROM tasks
+```
+
+## Best Practices
+
+### 1. Always Preview Before Creating
+
+**Bad:**
+```javascript
+// Create without testing
+create_insight({
+  name: 'Report',
+  sources: [...],
+  query: 'SELECT ...'  // Hope it works!
+})
+// ❌ Fails, have to debug and recreate
+```
+
+**Good:**
+```javascript
+// Preview first
+preview_insight({
+  sources: [...],
+  query: 'SELECT ...'
+})
+// ✅ Works!
+
+// Then create
+create_insight({
+  name: 'Report',
+  sources: [...],
+  query: 'SELECT ...'
+})
+```
+
+### 2. Start Simple, Build Complexity
+
+```javascript
+// Step 1: Basic SELECT
+preview_insight({ query: 'SELECT * FROM tasks' })
+
+// Step 2: Specific columns
+preview_insight({ query: 'SELECT title, priority FROM tasks' })
+
+// Step 3: Add filter
+preview_insight({ query: 'SELECT title, priority FROM tasks WHERE priority = "High"' })
+
+// Step 4: Add sorting
+preview_insight({ query: 'SELECT title, priority FROM tasks WHERE priority = "High" ORDER BY title' })
+```
+
+### 3. Use LIMIT During Testing
+
+```javascript
+// Good: Limited results for fast testing
+preview_insight({
+  sources: [...],
+  query: 'SELECT * FROM tasks LIMIT 5'
+})
+
+// Then remove LIMIT when creating
+create_insight({
+  sources: [...],
+  query: 'SELECT * FROM tasks'  // Full results
+})
+```
+
+### 4. Verify Field Mappings
+
+```javascript
+// Before preview, get field IDs
+const schema = get_workflow_schema({ workflowId: 'workflow-id' })
+schema.fields.forEach(f => {
+  if (f.key) {
+    console.log(`${f.key}: ${f._id}`)
+  }
+})
+
+// Use correct IDs in preview
+preview_insight({
+  sources: [{
+    fields: [
+      { name: 'priority', fieldId: 'verified-field-id' }
+    ]
+  }],
+  query: '...'
+})
+```
+
+### 5. Test JOINs Independently
+
+```javascript
+// Test each source independently first
+preview_insight({
+  sources: [{ name: 'tasks', ... }],
+  query: 'SELECT * FROM tasks LIMIT 5'
+})
+
+preview_insight({
+  sources: [{ name: 'projects', ... }],
+  query: 'SELECT * FROM projects LIMIT 5'
+})
+
+// Then test JOIN
+preview_insight({
+  sources: [
+    { name: 'tasks', ... },
+    { name: 'projects', ... }
+  ],
+  query: 'SELECT ... FROM tasks JOIN projects ...'
+})
+```
+
+### 6. Document Your Iterations
+
+```javascript
+// Keep track of what works
+console.log('Test 1: Basic query - ✅')
+preview_insight({ query: 'SELECT * FROM tasks' })
+
+console.log('Test 2: With filter - ✅')
+preview_insight({ query: 'SELECT * FROM tasks WHERE priority = "High"' })
+
+console.log('Test 3: With JOIN - ❌ Error: column not found')
+preview_insight({ query: 'SELECT ... JOIN ...' })
+
+console.log('Test 4: Fixed JOIN - ✅')
+preview_insight({ query: 'SELECT ... LEFT JOIN ...' })
+```
+
+### 7. Remember Sample Limitation
+
+```javascript
+// Preview only shows ~20 activities
+preview_insight({
+  sources: [...],
+  query: 'SELECT COUNT(*) FROM tasks'
+})
+// Returns: 20 (not actual total!)
+
+// To see actual count, create insight and use get_insight_data
+create_insight({ ... })
+get_insight_data({ insightId: '...' })
+// Returns: 1523 (actual total)
+```
+
+## Troubleshooting
+
+### Preview Returns No Data
+
+**Causes:**
+1. No activities in workflow
+2. WHERE clause too restrictive
+3. INNER JOIN eliminates all rows
+
+**Solutions:**
+
+```javascript
+// 1. Check activity count
+list_activities({
+  workflowId: 'workflow-id',
+  limit: 1
+})
+
+// 2. Remove WHERE clause
+preview_insight({
+  sources: [...],
+  query: 'SELECT * FROM tasks'  // No filter
+})
+
+// 3. Use LEFT JOIN
+preview_insight({
+  sources: [...],
+  query: 'SELECT ... FROM tasks LEFT JOIN projects ...'
+})
+```
+
+### Error: "Column not found"
+
+**Cause:** Field name in query doesn't match source definition.
+
+**Solution:**
+
+```javascript
+// Field names must match exactly
+sources: [{
+  fields: [
+    { name: 'taskTitle', meta: 'name' }  // Name is 'taskTitle'
+  ]
+}],
+query: 'SELECT taskTitle FROM tasks'  // Must use 'taskTitle'
+```
+
+### Error: "Table not found"
+
+**Cause:** Table name in query doesn't match source name.
+
+**Solution:**
+
+```javascript
+// Source names must match
+sources: [{
+  name: 'tasks',  // Name is 'tasks'
+  ...
+}],
+query: 'SELECT * FROM tasks'  // Must use 'tasks'
+```
+
+### Error: "Permission denied"
+
+**Cause:** Regular users can't test arbitrary workflow combinations.
+
+**Solutions:**
+1. Use sources from existing insights you have access to
+2. Request workspace admin to test
+3. Admins can test any sources
+
+### Preview Successful But create_insight Fails
+
+**Cause:** Preview uses sample data; full data may have issues.
+
+**Solutions:**
+1. Check for NULL values in full dataset
+2. Verify all activities have required fields
+3. Test with get_insight_data after creation
+
+## Integration with Other Tools
+
+### With create_insight
+
+Standard workflow:
+
+```javascript
+// 1. Preview
+preview_insight({
+  sources: [...],
+  query: '...'
+})
+
+// 2. Create (use exact same sources and query)
+create_insight({
+  name: 'My Report',
+  sources: [...],  // Same
+  query: '...'     // Same
+})
+```
+
+### With get_workflow_schema
+
+Get field IDs for preview:
+
+```javascript
+// 1. Get schema
+const schema = get_workflow_schema({
+  workflowId: 'workflow-id'
+})
+
+// 2. Build field mappings
+const fields = schema.fields
+  .filter(f => f.key)
+  .map(f => ({
+    name: f.key,
+    fieldId: f._id
+  }))
+
+// 3. Preview with fields
+preview_insight({
+  sources: [{
+    workflowId: 'workflow-id',
+    fields: fields
+  }],
+  query: '...'
+})
+```
+
+### With list_activities
+
+Compare results:
+
+```javascript
+// Preview result
+preview_insight({
+  sources: [...],
+  query: 'SELECT * FROM tasks WHERE priority = "High"'
+})
+// Shows ~20 rows
+
+// Compare with list_activities
+list_activities({
+  workflowId: 'workflow-id',
+  filters: [{ field: 'priority-field-id', operator: 'eq', value: 'High' }]
+})
+// Shows actual count
+```
+
+## Summary
+
+**Key Takeaways:**
+1. 🧪 **Test before create** - Always preview queries first
+2. 📊 **Sample data** - Preview shows ~20 activities only
+3. ⚡ **Fast iteration** - Quick feedback loop for development
+4. 🔍 **Debugging tool** - Catch errors before committing
+5. 🔒 **Security aware** - Regular users have restrictions
+6. 🏗️ **Iterative building** - Start simple, add complexity
+7. ✅ **Validation** - Verify syntax, fields, JOINs
+
+**Quick Decision Tree:**
+
+```
+Building an insight?
+│
+├─ Query untested? → Preview first
+├─ Getting SQL errors? → Preview to debug
+├─ Unsure about JOINs? → Preview each piece
+├─ Need full data? → Create insight, use get_insight_data
+└─ Query works in preview? → Create insight
+```
+
+## Additional Resources
+
+- See `create-insight-skill` for creating insights
+- See `get-insight-data-skill` for executing insights
+- See `remove-insight-skill` for deleting insights
+- See `insight-api` skill for comprehensive Insight API documentation
+- Use `get_workflow_schema` to find field IDs
+- Use `list_workflows` to find workflow IDs