aigroup-workflow 2.1.1 → 2.2.0

package/skills/database-optimizer/references/index-strategies.md

@@ -1,331 +0,0 @@
-# Index Strategies
-
-## Index Selection Methodology
-
-### Identify Index Candidates
-
-```sql
--- PostgreSQL: Find queries missing indexes
-SELECT query, calls, total_exec_time, mean_exec_time
-FROM pg_stat_statements
-WHERE mean_exec_time > 100
-ORDER BY total_exec_time DESC
-LIMIT 20;
-
--- PostgreSQL: Find sequential scans on large tables
-SELECT schemaname, tablename, seq_scan, seq_tup_read,
-       idx_scan, seq_tup_read / seq_scan as avg_seq_tup_read
-FROM pg_stat_user_tables
-WHERE seq_scan > 0
-  AND seq_tup_read / seq_scan > 10000
-ORDER BY seq_tup_read DESC;
-
--- MySQL: Check table scans
-SELECT * FROM sys.statements_with_full_table_scans
-WHERE db = 'your_database'
-ORDER BY exec_count DESC;
-```
-
-## B-Tree Indexes (Default)
-
-### Single Column Indexes
-
-```sql
--- Create index for WHERE clauses
-CREATE INDEX idx_users_email ON users(email);
-
--- Create index for JOIN conditions
-CREATE INDEX idx_orders_user_id ON orders(user_id);
-
--- Create index for ORDER BY
-CREATE INDEX idx_products_price ON products(price);
-
--- Unique constraint as index
-CREATE UNIQUE INDEX idx_users_username ON users(username);
-```
-
-### Multi-Column Indexes
-
-```sql
--- Order matters: most selective column first
-CREATE INDEX idx_orders_status_created
-ON orders(status, created_at);
-
--- Good for queries:
--- WHERE status = 'pending'
--- WHERE status = 'pending' AND created_at > '2024-01-01'
--- WHERE status = 'pending' ORDER BY created_at
-
--- NOT good for:
--- WHERE created_at > '2024-01-01' (status not specified)
-
--- Include commonly queried columns
-CREATE INDEX idx_users_active_email_name
-ON users(active, email) INCLUDE (name);
-```
-
-### Column Order Guidelines
-
-```sql
--- Rule 1: Equality before range
-CREATE INDEX idx_events_type_timestamp
-ON events(type, timestamp);  -- type = 'click' AND timestamp > ...
-
--- Rule 2: High selectivity first
-CREATE INDEX idx_orders_user_status
-ON orders(user_id, status);  -- user_id is more selective than status
-
--- Rule 3: Match query patterns
--- Query: WHERE country = 'US' AND city = 'NYC' AND zip = '10001'
-CREATE INDEX idx_locations_country_city_zip
-ON locations(country, city, zip);
-```
-
-## Covering Indexes
-
-### PostgreSQL INCLUDE Clause
-
-```sql
--- Include non-key columns for index-only scans
-CREATE INDEX idx_users_email_covering
-ON users(email) INCLUDE (name, created_at);
-
--- Query can be satisfied entirely from index
-EXPLAIN (ANALYZE, BUFFERS)
-SELECT name, created_at
-FROM users
-WHERE email = 'user@example.com';
--- Should show "Index Only Scan"
-```
-
-### MySQL Covering Indexes
-
-```sql
--- MySQL: Add columns to end of index
-CREATE INDEX idx_orders_user_covering
-ON orders(user_id, status, created_at, total);
-
--- Query uses covering index
-EXPLAIN
-SELECT status, created_at, total
-FROM orders
-WHERE user_id = 123;
--- Should show "Using index" in Extra column
-```
-
-## Partial Indexes
-
-### PostgreSQL Partial Indexes
-
-```sql
--- Index only active users
-CREATE INDEX idx_users_active_email
-ON users(email)
-WHERE active = true;
-
--- Index only recent orders
-CREATE INDEX idx_orders_recent
-ON orders(user_id, created_at)
-WHERE created_at > NOW() - INTERVAL '30 days';
-
--- Index only pending/processing orders (ignore completed)
-CREATE INDEX idx_orders_active
-ON orders(status, user_id)
-WHERE status IN ('pending', 'processing');
-
--- Smaller index = better performance + less storage
-```
-
-### MySQL Filtered Indexes (8.0+)
-
-```sql
--- MySQL 8.0+ supports functional indexes for similar effect
-CREATE INDEX idx_users_active
-ON users((CASE WHEN active = 1 THEN email END));
-```
-
-## Expression Indexes
-
-### PostgreSQL Function Indexes
-
-```sql
--- Index for case-insensitive search
-CREATE INDEX idx_users_email_lower
-ON users(LOWER(email));
-
--- Query must match expression
-SELECT * FROM users
-WHERE LOWER(email) = LOWER('User@Example.com');
-
--- Index for JSONB queries
-CREATE INDEX idx_users_settings_theme
-ON users((settings->>'theme'));
-
-SELECT * FROM users
-WHERE settings->>'theme' = 'dark';
-
--- Index for date truncation
-CREATE INDEX idx_orders_date
-ON orders(DATE(created_at));
-```
-
-### MySQL Generated Column Indexes
-
-```sql
--- Create generated column, then index it
-ALTER TABLE users
-ADD COLUMN email_lower VARCHAR(255)
-GENERATED ALWAYS AS (LOWER(email)) STORED;
-
-CREATE INDEX idx_users_email_lower
-ON users(email_lower);
-
--- Use in queries
-SELECT * FROM users
-WHERE email_lower = LOWER('User@Example.com');
-```
-
-## Specialized Index Types
-
-### PostgreSQL GIN Indexes (Full-Text, Arrays, JSONB)
-
-```sql
--- Full-text search
-CREATE INDEX idx_posts_search
-ON posts USING GIN(to_tsvector('english', title || ' ' || content));
-
-SELECT * FROM posts
-WHERE to_tsvector('english', title || ' ' || content)
-      @@ to_tsquery('english', 'database & optimization');
-
--- Array search
-CREATE INDEX idx_products_tags
-ON products USING GIN(tags);
-
-SELECT * FROM products
-WHERE tags @> ARRAY['electronics', 'sale'];
-
--- JSONB search
-CREATE INDEX idx_users_metadata
-ON users USING GIN(metadata);
-
-SELECT * FROM users
-WHERE metadata @> '{"plan": "premium"}';
-```
-
-### PostgreSQL GiST Indexes (Geometric, Range)
-
-```sql
--- Range types
-CREATE INDEX idx_events_time_range
-ON events USING GIST(time_range);
-
-SELECT * FROM events
-WHERE time_range && '[2024-01-01, 2024-01-31]'::tstzrange;
-
--- PostGIS geometric queries
-CREATE INDEX idx_locations_coords
-ON locations USING GIST(coordinates);
-```
-
-### MySQL Full-Text Indexes
-
-```sql
--- Full-text search
-CREATE FULLTEXT INDEX idx_posts_content
-ON posts(title, content);
-
-SELECT * FROM posts
-WHERE MATCH(title, content)
-      AGAINST('database optimization' IN NATURAL LANGUAGE MODE);
-
--- Boolean mode for complex searches
-SELECT * FROM posts
-WHERE MATCH(title, content)
-      AGAINST('+database -mysql' IN BOOLEAN MODE);
-```
-
-## Index Maintenance
-
-### PostgreSQL Maintenance
-
-```sql
--- Update statistics for query planner
-ANALYZE users;
-
--- Rebuild bloated index
-REINDEX INDEX CONCURRENTLY idx_users_email;
-
--- Check index bloat
-SELECT
-    schemaname, tablename, indexname,
-    pg_size_pretty(pg_relation_size(indexrelid)) as index_size,
-    idx_scan as scans,
-    idx_tup_read as tuples_read,
-    idx_tup_fetch as tuples_fetched
-FROM pg_stat_user_indexes
-ORDER BY pg_relation_size(indexrelid) DESC;
-
--- Find unused indexes
-SELECT
-    schemaname, tablename, indexname,
-    idx_scan,
-    pg_size_pretty(pg_relation_size(indexrelid)) as index_size
-FROM pg_stat_user_indexes
-WHERE idx_scan = 0
-  AND indexrelname NOT LIKE 'pg_toast%'
-ORDER BY pg_relation_size(indexrelid) DESC;
-```
-
-### MySQL Maintenance
-
-```sql
--- Update statistics
-ANALYZE TABLE users;
-
--- Rebuild index
-ALTER TABLE users DROP INDEX idx_users_email, ADD INDEX idx_users_email(email);
-
--- Check index usage
-SELECT
-    object_schema,
-    object_name,
-    index_name,
-    count_star,
-    count_read,
-    count_fetch
-FROM performance_schema.table_io_waits_summary_by_index_usage
-WHERE object_schema = 'your_database'
-ORDER BY count_star DESC;
-
--- Find unused indexes
-SELECT
-    object_schema,
-    object_name,
-    index_name
-FROM performance_schema.table_io_waits_summary_by_index_usage
-WHERE index_name IS NOT NULL
-  AND count_star = 0
-  AND object_schema = 'your_database';
-```
-
-## Index Anti-Patterns
-
-| Anti-Pattern | Issue | Solution |
-|-------------|-------|----------|
-| Index every column | Write overhead, storage waste | Index based on query patterns |
-| Redundant indexes | `(a)` + `(a,b)` | Keep only `(a,b)` |
-| Wrong column order | `(created_at, user_id)` for `WHERE user_id = ?` | Put filtered columns first |
-| Over-covering | Including rarely-used columns | Include only frequently accessed columns |
-| Ignoring WHERE clause | Full index for 5% of data | Use partial indexes |
-| Expression mismatch | Index `email`, query `LOWER(email)` | Create expression index |
-
-## Index Design Checklist
-
-1. **Analyze queries**: Use pg_stat_statements or slow query log
-2. **Check execution plans**: Look for Seq Scan on large tables
-3. **Design indexes**: Equality → Range → Include
-4. **Create concurrently**: Avoid locking (PostgreSQL)
-5. **Validate improvement**: Compare before/after EXPLAIN
-6. **Monitor usage**: Remove unused indexes after 30 days
-7. **Maintain regularly**: VACUUM, ANALYZE, REINDEX as needed