@pageai/ralph-loop 1.9.0 → 1.10.0

package/.agents/skills/mysql/references/index-maintenance.md

@@ -0,0 +1,110 @@
+---
+title: Index Maintenance and Cleanup
+description: Index maintenance
+tags: mysql, indexes, maintenance, unused-indexes, performance
+---
+
+# Index Maintenance
+
+## Find Unused Indexes
+
+```sql
+-- Requires performance_schema enabled (default in MySQL 5.7+)
+-- "Unused" here means no reads/writes since last restart.
+SELECT object_schema, object_name, index_name, COUNT_READ, COUNT_WRITE
+FROM performance_schema.table_io_waits_summary_by_index_usage
+WHERE object_schema = 'mydb'
+  AND index_name IS NOT NULL AND index_name != 'PRIMARY'
+  AND COUNT_READ = 0 AND COUNT_WRITE = 0
+ORDER BY COUNT_WRITE DESC;
+```
+
+Sometimes you'll also see indexes with **writes but no reads** (overhead without query benefit). Review these carefully: some are required for constraints (UNIQUE/PK) even if not used in query plans.
+
+```sql
+SELECT object_schema, object_name, index_name, COUNT_READ, COUNT_WRITE
+FROM performance_schema.table_io_waits_summary_by_index_usage
+WHERE object_schema = 'mydb'
+  AND index_name IS NOT NULL AND index_name != 'PRIMARY'
+  AND COUNT_READ = 0 AND COUNT_WRITE > 0
+ORDER BY COUNT_WRITE DESC;
+```
+
+Counters reset on restart — ensure 1+ full business cycle of uptime before dropping.
+
+## Find Redundant Indexes
+
+Index on `(a)` is redundant if `(a, b)` exists (leftmost prefix covers it). Pairs sharing only the first column (e.g. `(a,b)` vs `(a,c)`) need manual review — neither is redundant.
+
+```sql
+-- Prefer sys schema view (MySQL 5.7.7+)
+SELECT table_schema, table_name,
+  redundant_index_name, redundant_index_columns,
+  dominant_index_name, dominant_index_columns
+FROM sys.schema_redundant_indexes
+WHERE table_schema = 'mydb';
+```
+
+## Check Index Sizes
+
+```sql
+SELECT database_name, table_name, index_name,
+  ROUND(stat_value * @@innodb_page_size / 1024 / 1024, 2) AS size_mb
+FROM mysql.innodb_index_stats
+WHERE stat_name = 'size' AND database_name = 'mydb'
+ORDER BY stat_value DESC;
+-- stat_value is in pages; multiply by innodb_page_size for bytes
+```
+
+## Index Write Overhead
+Each index must be updated on INSERT, UPDATE, and DELETE operations. More indexes = slower writes.
+
+- **INSERT**: each secondary index adds a write
+- **UPDATE**: changing indexed columns updates all affected indexes
+- **DELETE**: removes entries from all indexes
+
+InnoDB can defer some secondary index updates via the change buffer, but excessive indexing still reduces write throughput.
+
+## Update Statistics (ANALYZE TABLE)
+The optimizer relies on index cardinality and distribution statistics. After large data changes, refresh statistics:
+
+```sql
+ANALYZE TABLE orders;
+```
+
+This updates statistics (does not rebuild the table).
+
+## Rebuild / Reclaim Space (OPTIMIZE TABLE)
+`OPTIMIZE TABLE` can reclaim space and rebuild indexes:
+
+```sql
+OPTIMIZE TABLE orders;
+```
+
+For InnoDB this effectively rebuilds the table and indexes and can be slow on large tables.
+
+## Invisible Indexes (MySQL 8.0+)
+Test removing an index without dropping it:
+
+```sql
+ALTER TABLE orders ALTER INDEX idx_status INVISIBLE;
+ALTER TABLE orders ALTER INDEX idx_status VISIBLE;
+```
+
+Invisible indexes are still maintained on writes (overhead remains), but the optimizer won't consider them.
+
+## Index Maintenance Tools
+
+### Online DDL (Built-in)
+Most add/drop index operations are online-ish but still take brief metadata locks:
+
+```sql
+ALTER TABLE orders ADD INDEX idx_status (status), ALGORITHM=INPLACE, LOCK=NONE;
+```
+
+### pt-online-schema-change / gh-ost
+For very large tables or high-write workloads, online schema change tools can reduce blocking by using a shadow table and a controlled cutover (tradeoffs: operational complexity, privileges, triggers/binlog requirements).
+
+## Guidelines
+- 1–5 indexes per table is normal. 6+: audit for redundancy.
+- Combine `performance_schema` data with `EXPLAIN` of frequent queries monthly.

package/.agents/skills/mysql/references/isolation-levels.md

@@ -0,0 +1,49 @@
+---
+title: InnoDB Transaction Isolation Levels
+description: Best practices for choosing and using isolation levels
+tags: mysql, transactions, isolation, innodb, locking, concurrency
+---
+
+# Isolation Levels (InnoDB Best Practices)
+
+**Default to REPEATABLE READ.** It is the InnoDB default, most tested, and prevents phantom reads. Only change per-session with a measured reason.
+
+```sql
+SELECT @@transaction_isolation;
+SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED;  -- per-session only
+```
+
+## Autocommit Interaction
+- Default: `autocommit=1` (each statement is its own transaction).
+- With `autocommit=0`, transactions span multiple statements until `COMMIT`/`ROLLBACK`.
+- Isolation level applies per transaction. SERIALIZABLE behavior differs based on autocommit setting (see SERIALIZABLE section).
+
+## Locking vs Non-Locking Reads
+- **Non-locking reads**: plain `SELECT` statements use consistent reads (MVCC snapshots). They don't acquire locks and don't block writers.
+- **Locking reads**: `SELECT ... FOR UPDATE` (exclusive) or `SELECT ... FOR SHARE` (shared) acquire locks and can block concurrent modifications.
+- `UPDATE` and `DELETE` statements are implicitly locking reads.
+
+## REPEATABLE READ (Default — Prefer This)
+- Consistent reads: snapshot established at first read; all plain SELECTs within the transaction read from that same snapshot (MVCC). Plain SELECTs are non-locking and don't block writers.
+- Locking reads/writes use **next-key locks** (row + gap) — prevents phantoms. Exception: a unique index with a unique search condition locks only the index record, not the gap.
+- **Use for**: OLTP, check-then-insert, financial logic, reports needing consistent snapshots.
+- **Avoid mixing** locking statements (`SELECT ... FOR UPDATE`, `UPDATE`, `DELETE`) with non-locking `SELECT` statements in the same transaction — they can observe different states (current vs snapshot) and lead to surprises.
+
+## READ COMMITTED (Per-Session Only, When Needed)
+- Fresh snapshot per SELECT; **record locks only** (gap locks disabled for searches/index scans, but still used for foreign-key and duplicate-key checks) — more concurrency, but phantoms possible.
+- **Switch only when**: gap-lock deadlocks confirmed via `SHOW ENGINE INNODB STATUS`, bulk imports with contention, or high-write concurrency on overlapping ranges.
+- **Never switch globally.** Check-then-insert patterns break — use `INSERT ... ON DUPLICATE KEY` or `FOR UPDATE` instead.
+
+## SERIALIZABLE — Avoid
+Converts all plain SELECTs to `SELECT ... FOR SHARE` **if autocommit is disabled**. If autocommit is enabled, SELECTs are consistent (non-locking) reads. SERIALIZABLE can cause massive contention when autocommit is disabled. Prefer explicit `SELECT ... FOR UPDATE` at REPEATABLE READ instead — same safety, far less lock scope.
+
+## READ UNCOMMITTED — Never Use
+Dirty reads with no valid production use case.
+
+## Decision Guide
+| Scenario | Recommendation |
+|---|---|
+| General OLTP / check-then-insert / reports | **REPEATABLE READ** (default) |
+| Bulk import or gap-lock deadlocks | **READ COMMITTED** (per-session), benchmark first |
+| Need serializability | Explicit `FOR UPDATE` at REPEATABLE READ; SERIALIZABLE only as last resort |
+

package/.agents/skills/mysql/references/json-column-patterns.md

@@ -0,0 +1,77 @@
+---
+title: JSON Column Best Practices
+description: When and how to use JSON columns safely
+tags: mysql, json, generated-columns, indexes, data-modeling
+---
+
+# JSON Column Patterns
+
+MySQL 5.7+ supports native JSON columns. Useful, but with important caveats.
+
+## When JSON Is Appropriate
+- Truly schema-less data (user preferences, metadata bags, webhook payloads).
+- Rarely filtered/joined — if you query a JSON path frequently, extract it to a real column.
+
+## Indexing JSON: Use Generated Columns
+You **cannot** index a JSON column directly. Create a virtual generated column and index that:
+```sql
+ALTER TABLE events
+  ADD COLUMN event_type VARCHAR(50) GENERATED ALWAYS AS (data->>'$.type') VIRTUAL,
+  ADD INDEX idx_event_type (event_type);
+```
+
+## Extraction Operators
+| Syntax | Returns | Use for |
+|---|---|---|
+| `JSON_EXTRACT(col, '$.key')` | JSON type value (e.g., `"foo"` for strings) | When you need JSON type semantics |
+| `col->'$.key'` | Same as `JSON_EXTRACT(col, '$.key')` | Shorthand |
+| `col->>'$.key'` | Unquoted scalar (equivalent to `JSON_UNQUOTE(JSON_EXTRACT(col, '$.key'))`) | WHERE comparisons, display |
+
+Always use `->>` (unquote) in WHERE clauses, otherwise you compare against `"foo"` (with quotes).
+
+Tip: the generated column example above can be written more concisely as:
+
+```sql
+ALTER TABLE events
+  ADD COLUMN event_type VARCHAR(50) GENERATED ALWAYS AS (data->>'$.type') VIRTUAL,
+  ADD INDEX idx_event_type (event_type);
+```
+
+## Multi-Valued Indexes (MySQL 8.0.17+)
+If you store arrays in JSON (e.g., `tags: ["electronics","sale"]`), MySQL 8.0.17+ supports multi-valued indexes to index array elements:
+
+```sql
+ALTER TABLE products
+  ADD INDEX idx_tags ((CAST(tags AS CHAR(50) ARRAY)));
+```
+
+This can accelerate membership queries such as:
+
+```sql
+SELECT * FROM products WHERE 'electronics' MEMBER OF (tags);
+```
+
+## Collation and Type Casting Pitfalls
+- **JSON type comparisons**: `JSON_EXTRACT` returns JSON type. Comparing directly to strings can be wrong for numbers/dates.
+
+```sql
+-- WRONG: lexicographic string comparison
+WHERE data->>'$.price' <= '1200'
+
+-- CORRECT: cast to numeric
+WHERE CAST(data->>'$.price' AS UNSIGNED) <= 1200
+```
+
+- **Collation**: values extracted with `->>` behave like strings and use a collation. Use `COLLATE` when you need a specific comparison behavior.
+
+```sql
+WHERE data->>'$.status' COLLATE utf8mb4_0900_as_cs = 'Active'
+```
+
+## Common Pitfalls
+- **Heavy update cost**: `JSON_SET`/`JSON_REPLACE` can touch large portions of a JSON document and generate significant redo/undo work on large blobs.
+- **No partial indexes**: You can only index extracted scalar paths via generated columns.
+- **Large documents hurt**: JSON stored inline in the row. Documents >8 KB spill to overflow pages, hurting read performance.
+- **Type mismatches**: `JSON_EXTRACT` returns a JSON type. Comparing with `= 'foo'` may not match — use `->>` or `JSON_UNQUOTE`.
+- **VIRTUAL vs STORED generated columns**: VIRTUAL columns compute on read (less storage, more CPU). STORED columns materialize on write (more storage, faster reads if selected often). Both can be indexed; for indexed paths, the index stores the computed value either way.
+

package/.agents/skills/mysql/references/n-plus-one.md

@@ -0,0 +1,77 @@
+---
+title: N+1 Query Detection and Fixes
+description: N+1 query solutions
+tags: mysql, n-plus-one, orm, query-optimization, performance
+---
+
+# N+1 Query Detection
+
+## What Is N+1?
+The N+1 pattern occurs when you fetch N parent records, then execute N additional queries (one per parent) to fetch related data.
+
+Example: 1 query for users + N queries for posts.
+
+## ORM Fixes (Quick Reference)
+
+- **SQLAlchemy 1.x**: `session.query(User).options(joinedload(User.posts))`
+- **SQLAlchemy 2.0**: `select(User).options(joinedload(User.posts))`
+- **Django**: `select_related('fk_field')` for FK/O2O, `prefetch_related('m2m_field')` for M2M/reverse FK
+- **ActiveRecord**: `User.includes(:orders)`
+- **Prisma**: `findMany({ include: { orders: true } })`
+- **Drizzle**: use `.leftJoin()` instead of loop queries
+
+```typescript
+// Drizzle example: avoid N+1 with a join
+const rows = await db
+  .select()
+  .from(users)
+  .leftJoin(posts, eq(users.id, posts.userId));
+```
+
+## Detecting in MySQL Production
+
+```sql
+-- High-frequency simple queries often indicate N+1
+-- Requires performance_schema enabled (default in MySQL 5.7+)
+SELECT digest_text, count_star, avg_timer_wait
+FROM performance_schema.events_statements_summary_by_digest
+ORDER BY count_star DESC LIMIT 20;
+```
+
+Also check the slow query log sorted by `count` for frequently repeated simple SELECTs.
+
+## Batch Consolidation
+Replace sequential queries with `WHERE id IN (...)`.
+
+Practical limits:
+- Total statement size is capped by `max_allowed_packet` (often 4MB by default).
+- Very large IN lists increase parsing/planning overhead and can hurt performance.
+
+Strategies:
+- Up to ~1000–5000 ids: `IN (...)` is usually fine.
+- Larger: chunk the list (e.g. batches of 500–1000) or use a temporary table and join.
+
+```sql
+-- Temporary table approach for large batches
+CREATE TEMPORARY TABLE temp_user_ids (id BIGINT PRIMARY KEY);
+INSERT INTO temp_user_ids VALUES (1), (2), (3);
+
+SELECT p.*
+FROM posts p
+JOIN temp_user_ids t ON p.user_id = t.id;
+```
+
+## Joins vs Separate Queries
+- Prefer **JOINs** when you need related data for most/all parent rows and the result set stays reasonable.
+- Prefer **separate queries** (batched) when JOINs would explode rows (one-to-many) or over-fetch too much data.
+
+## Eager Loading Caveats
+- **Over-fetching**: eager loading pulls *all* related rows unless you filter it.
+- **Memory**: loading large collections can blow up memory.
+- **Row multiplication**: JOIN-based eager loading can create huge result sets; in some ORMs, a "select-in" strategy is safer.
+
+## Prepared Statements
+Prepared statements reduce repeated parse/optimize overhead for repeated parameterized queries, but they do **not** eliminate N+1: you still execute N queries. Use batching/eager loading to reduce query count.
+
+## Pagination Pitfalls
+N+1 often reappears per page. Ensure eager loading or batching is applied to the paginated query, not inside the per-row loop.

package/.agents/skills/mysql/references/online-ddl.md

@@ -0,0 +1,53 @@
+---
+title: Online DDL and Schema Migrations
+description: Lock-safe ALTER TABLE guidance
+tags: mysql, ddl, schema-migration, alter-table, innodb
+---
+
+# Online DDL
+
+Not all `ALTER TABLE` is equal — some block writes for the entire duration.
+
+## Algorithm Spectrum
+
+| Algorithm | What Happens | DML During? |
+|---|---|---|
+| `INSTANT` | Metadata-only change | Yes |
+| `INPLACE` | Rebuilds in background | Usually yes |
+| `COPY` | Full table copy to tmp table | **Blocked** |
+
+MySQL picks the fastest available. Specify explicitly to fail-safe:
+```sql
+ALTER TABLE orders ADD COLUMN note VARCHAR(255) DEFAULT NULL, ALGORITHM=INSTANT;
+-- Fails loudly if INSTANT isn't possible, rather than silently falling back to COPY.
+```
+
+## What Supports INSTANT (MySQL 8.0+)
+- Adding a column (at any position as of 8.0.29; only at end before 8.0.29)
+- Dropping a column (8.0.29+)
+- Renaming a column (8.0.28+)
+
+**Not INSTANT**: adding indexes (uses INPLACE), dropping indexes (uses INPLACE; typically metadata-only), changing column type, extending VARCHAR (uses INPLACE), adding columns when INSTANT isn't supported for the table/operation.
+
+## Lock Levels
+`LOCK=NONE` (concurrent DML), `LOCK=SHARED` (reads only), `LOCK=EXCLUSIVE` (full block), `LOCK=DEFAULT` (server chooses maximum concurrency; default).
+
+Always request `LOCK=NONE` (and an explicit `ALGORITHM`) to surface conflicts early instead of silently falling back to a more blocking method.
+
+## Large Tables (millions+ rows)
+Even `INPLACE` operations typically hold brief metadata locks at start/end. The commit phase requires an exclusive metadata lock and will wait for concurrent transactions to finish; long-running transactions can block DDL from completing.
+
+On huge tables, consider external tools:
+- **pt-online-schema-change**: creates shadow table, syncs via triggers.
+- **gh-ost**: triggerless, uses binlog stream. Preferred for high-write tables.
+
+## Replication Considerations
+- DDL replicates to replicas and executes there, potentially causing lag (especially COPY-like rebuilds).
+- INSTANT operations minimize replication impact because they complete quickly.
+- INPLACE operations can still cause lag and metadata lock waits on replicas during apply.
+
+## PlanetScale Users
+On PlanetScale, use **deploy requests** instead of manual DDL tools. Vitess handles non-blocking migrations automatically. Use this whenever possible because it offers much safer schema migrations.
+
+## Key Rule
+Never run `ALTER TABLE` on production without checking the algorithm. A surprise `COPY` on a 100M-row table can lock writes for hours.

package/.agents/skills/mysql/references/partitioning.md

@@ -0,0 +1,92 @@
+---
+title: MySQL Partitioning
+description: Partition types and management operations
+tags: mysql, partitioning, range, list, hash, maintenance, data-retention
+---
+
+# Partitioning
+
+All columns used in the partitioning expression must be part of every UNIQUE/PRIMARY KEY.
+
+## Partition Pruning
+The optimizer can eliminate partitions that cannot contain matching rows based on the WHERE clause ("partition pruning"). Partitioning helps most when queries frequently filter by the partition key/expression:
+- Equality: `WHERE partition_key = ?` (HASH/KEY)
+- Ranges: `WHERE partition_key BETWEEN ? AND ?` (RANGE)
+- IN lists: `WHERE partition_key IN (...)` (LIST)
+
+## Types
+
+| Need | Type |
+|---|---|
+| Time-ordered / data retention | RANGE |
+| Discrete categories | LIST |
+| Even distribution | HASH / KEY |
+| Two access patterns | RANGE + HASH sub |
+
+```sql
+-- RANGE COLUMNS (direct date comparisons; avoids function wrapper)
+PARTITION BY RANGE COLUMNS (created_at) (
+  PARTITION p2025_q1 VALUES LESS THAN ('2025-04-01'),
+  PARTITION p_future VALUES LESS THAN (MAXVALUE)
+);
+
+-- RANGE with function (use when you must partition by an expression)
+PARTITION BY RANGE (TO_DAYS(created_at)) (
+  PARTITION p2025_q1 VALUES LESS THAN (TO_DAYS('2025-04-01')),
+  PARTITION p_future VALUES LESS THAN MAXVALUE
+);
+-- LIST (discrete categories — unlisted values cause errors, ensure full coverage)
+PARTITION BY LIST COLUMNS (region) (
+  PARTITION p_americas VALUES IN ('us', 'ca', 'br'),
+  PARTITION p_europe  VALUES IN ('uk', 'de', 'fr')
+);
+-- HASH/KEY (even distribution, equality pruning only)
+PARTITION BY HASH (user_id) PARTITIONS 8;
+```
+
+## Foreign Key Restrictions (InnoDB)
+Partitioned InnoDB tables do not support foreign keys:
+- A partitioned table cannot define foreign key constraints to other tables.
+- Other tables cannot reference a partitioned table with a foreign key.
+
+If you need foreign keys, partitioning may not be an option.
+
+## When Partitioning Helps vs Hurts
+**Helps:**
+- Very large tables (millions+ rows) with time-ordered access patterns
+- Data retention workflows (drop old partitions vs DELETE)
+- Queries that filter by the partition key/expression (enables pruning)
+- Maintenance on subsets of data (operate on partitions vs whole table)
+
+**Hurts:**
+- Small tables (overhead without benefit)
+- Queries that don't filter by the partition key (no pruning)
+- Workloads that require foreign keys
+- Complex UNIQUE key requirements (partition key columns must be included everywhere)
+
+## Management Operations
+
+```sql
+-- Add: split catch-all MAXVALUE partition
+ALTER TABLE events REORGANIZE PARTITION p_future INTO (
+  PARTITION p2026_01 VALUES LESS THAN (TO_DAYS('2026-02-01')),
+  PARTITION p_future VALUES LESS THAN MAXVALUE
+);
+-- Drop aged-out data (orders of magnitude faster than DELETE)
+ALTER TABLE events DROP PARTITION p2025_q1;
+-- Merge partitions
+ALTER TABLE events REORGANIZE PARTITION p2025_01, p2025_02, p2025_03 INTO (
+  PARTITION p2025_q1 VALUES LESS THAN (TO_DAYS('2025-04-01'))
+);
+-- Archive via exchange (LIKE creates non-partitioned copy; both must match structure)
+CREATE TABLE events_archive LIKE events;
+ALTER TABLE events_archive REMOVE PARTITIONING;
+ALTER TABLE events EXCHANGE PARTITION p2025_q1 WITH TABLE events_archive;
+```
+
+Notes:
+- `REORGANIZE PARTITION` rebuilds the affected partition(s).
+- `EXCHANGE PARTITION` requires an exact structure match (including indexes) and the target table must not be partitioned.
+- `DROP PARTITION` is DDL (fast) vs `DELETE` (DML; slow on large datasets).
+
+Always ask for human approval before dropping, deleting, or archiving data.

package/.agents/skills/mysql/references/primary-keys.md

@@ -0,0 +1,70 @@
+---
+title: Primary Key Design
+description: Primary key patterns
+tags: mysql, primary-keys, auto-increment, uuid, innodb
+---
+
+# Primary Keys
+
+InnoDB stores rows in primary key order (clustered index). This means:
+- **Sequential keys = optimal inserts**: new rows append, minimizing page splits and fragmentation.
+- **Random keys = fragmentation**: random inserts cause page splits to maintain PK order, wasting space and slowing inserts.
+- **Secondary index lookups**: secondary indexes store the PK value and use it to fetch the full row from the clustered index.
+
+## INT vs BIGINT for Primary Keys
+- **INT UNSIGNED**: 4 bytes, max ~4.3B rows.
+- **BIGINT UNSIGNED**: 8 bytes, max ~18.4 quintillion rows.
+
+Guideline: default to **BIGINT UNSIGNED** unless you're certain the table will never approach the INT limit. The extra 4 bytes is usually cheaper than the risk of exhausting INT.
+
+## Avoid Random UUID as Clustered PK
+- UUID PK stored as `BINARY(16)`: 16 bytes (vs 8 for BIGINT). Random inserts cause page splits, and every secondary index entry carries the PK.
+- UUID stored as `CHAR(36)`/`VARCHAR(36)`: 36 bytes (+ overhead) and is generally worse for storage and index size.
+- If external identifiers are required, store UUID as `BINARY(16)` in a secondary unique column:
+
+```sql
+CREATE TABLE users (
+  id BIGINT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
+  public_id BINARY(16) NOT NULL,
+  UNIQUE KEY idx_public_id (public_id)
+);
+-- UUID_TO_BIN(uuid, 1) reorders UUIDv1 bytes to be roughly time-sorted (reduces fragmentation)
+-- MySQL's UUID() returns UUIDv4 (random). For time-ordered IDs, use app-generated UUIDv7/ULID/Snowflake.
+INSERT INTO users (public_id) VALUES (UUID_TO_BIN(?, 1)); -- app provides UUID string
+```
+
+If UUIDs are required, prefer time-ordered variants such as UUIDv7 (app-generated) to reduce index fragmentation.
+
+## Secondary Indexes Include the Primary Key
+InnoDB secondary indexes store the primary key value with each index entry. Implications:
+- **Larger secondary indexes**: a secondary index entry includes (indexed columns + PK bytes).
+- **Covering reads**: `SELECT id FROM users WHERE email = ?` can often be satisfied from `INDEX(email)` because `id` (PK) is already present in the index entry.
+- **UUID penalty**: a `BINARY(16)` PK makes every secondary index entry 8 bytes larger than a BIGINT PK.
+
+## Auto-Increment Considerations
+- **Hot spot**: inserts target the end of the clustered index (usually fine; can bottleneck at extreme insert rates).
+- **Gaps are normal**: rollbacks or failed inserts can leave gaps.
+- **Locking**: auto-increment allocation can introduce contention under very high concurrency.
+
+## Alternative Ordered IDs (Snowflake / ULID / UUIDv7)
+If you need globally unique IDs generated outside the database:
+- **Snowflake-style**: 64-bit integers (fits in BIGINT), time-ordered, compact.
+- **ULID / UUIDv7**: 128-bit (store as `BINARY(16)`), time-ordered, better insert locality than random UUIDv4.
+
+Recommendation: prefer `BIGINT AUTO_INCREMENT` unless you need distributed ID generation or externally meaningful identifiers.
+
+## Replication Considerations
+- Random-key insert patterns (UUIDv4) can amplify page splits and I/O on replicas too, increasing lag.
+- Time-ordered IDs reduce fragmentation and tend to replicate more smoothly under heavy insert workloads.
+
+## Composite Primary Keys
+
+Use for join/many-to-many tables. Most-queried column first:
+
+```sql
+CREATE TABLE user_roles (
+  user_id BIGINT UNSIGNED NOT NULL,
+  role_id BIGINT UNSIGNED NOT NULL,
+  PRIMARY KEY (user_id, role_id)
+);
+```

package/.agents/skills/mysql/references/query-optimization-pitfalls.md

@@ -0,0 +1,117 @@
+---
+title: Query Optimization Pitfalls
+description: Common anti-patterns that silently kill performance
+tags: mysql, query-optimization, anti-patterns, performance, indexes
+---
+
+# Query Optimization Pitfalls
+
+These patterns look correct but bypass indexes or cause full scans.
+
+## Non-Sargable Predicates
+A **sargable** predicate can use an index. Common non-sargable patterns:
+- functions/arithmetic on indexed columns
+- implicit type conversions
+- leading wildcards (`LIKE '%x'`)
+- some negations (`!=`, `NOT IN`, `NOT LIKE`) depending on shape/data
+
+## Functions on Indexed Columns
+```sql
+-- BAD: function prevents index use on created_at
+WHERE YEAR(created_at) = 2024
+
+-- GOOD: sargable range
+WHERE created_at >= '2024-01-01' AND created_at < '2025-01-01'
+```
+
+MySQL 8.0+ can use expression (functional) indexes for some cases:
+
+```sql
+CREATE INDEX idx_users_upper_name ON users ((UPPER(name)));
+-- Now this can use idx_users_upper_name:
+WHERE UPPER(name) = 'SMITH'
+```
+
+## Implicit Type Conversions
+Implicit casts can make indexes unusable:
+
+```sql
+-- If phone is VARCHAR, this may force CAST(phone AS UNSIGNED) and scan
+WHERE phone = 1234567890
+
+-- Better: match the column type
+WHERE phone = '1234567890'
+```
+
+## LIKE Patterns
+```sql
+-- BAD: leading wildcard cannot use a B-Tree index
+WHERE name LIKE '%smith'
+WHERE name LIKE '%smith%'
+
+-- GOOD: prefix match can use an index
+WHERE name LIKE 'smith%'
+```
+
+For suffix search, consider storing a reversed generated column + prefix search:
+
+```sql
+ALTER TABLE users
+  ADD COLUMN name_reversed VARCHAR(255) AS (REVERSE(name)) STORED,
+  ADD INDEX idx_users_name_reversed (name_reversed);
+
+WHERE name_reversed LIKE CONCAT(REVERSE('smith'), '%');
+```
+
+For infix search at scale, use `FULLTEXT` (when appropriate) or a dedicated search engine.
+
+## `OR` Across Different Columns
+`OR` across different columns often prevents efficient index use.
+
+```sql
+-- Often suboptimal
+WHERE status = 'active' OR region = 'us-east'
+
+-- Often better: two indexed queries
+SELECT * FROM orders WHERE status = 'active'
+UNION ALL
+SELECT * FROM orders WHERE region = 'us-east';
+```
+
+MySQL can sometimes use `index_merge`, but it's frequently slower than a purpose-built composite index or a UNION rewrite.
+
+## ORDER BY + LIMIT Without an Index
+`LIMIT` does not automatically make sorting cheap. If no index supports the order, MySQL may sort many rows (`Using filesort`) and then apply LIMIT.
+
+```sql
+-- Needs an index on created_at (or it will filesort)
+SELECT * FROM orders ORDER BY created_at DESC LIMIT 10;
+
+-- For WHERE + ORDER BY, you usually need a composite index:
+-- (status, created_at DESC)
+SELECT * FROM orders
+WHERE status = 'pending'
+ORDER BY created_at DESC
+LIMIT 10;
+```
+
+## DISTINCT / GROUP BY
+`DISTINCT` and `GROUP BY` can trigger temp tables and sorts (`Using temporary`, `Using filesort`) when indexes don't match.
+
+```sql
+-- Often improved by an index on (status)
+SELECT DISTINCT status FROM orders;
+
+-- Often improved by an index on (status)
+SELECT status, COUNT(*) FROM orders GROUP BY status;
+```
+
+## Derived Tables / CTE Materialization
+Derived tables and CTEs may be materialized into temporary tables, which can be slower than a flattened query. If performance is surprising, check `EXPLAIN` and consider rewriting the query or adding supporting indexes.
+
+## Other Quick Rules
+- **`OFFSET` pagination**: `OFFSET N` scans and discards N rows. Use cursor-based pagination.
+- **`SELECT *`** defeats covering indexes. Select only needed columns.
+- **`NOT IN` with NULLs**: `NOT IN (subquery)` returns no rows if subquery contains any NULL. Use `NOT EXISTS`.
+- **`COUNT(*)` vs `COUNT(col)`**: `COUNT(*)` counts all rows; `COUNT(col)` skips NULLs.
+- **Arithmetic on indexed columns**: `WHERE price * 1.1 > 100` prevents index use. Rewrite to keep the column bare: `WHERE price > 100 / 1.1`.