clearctx 3.0.0 → 3.2.0

package/skills/postgresql/SKILL.md

@@ -0,0 +1,315 @@
+---
+name: postgresql
+description: Production-grade PostgreSQL schema design, query optimization, migrations, and operational patterns
+domain: database
+keywords: [postgresql, postgres, sql, schema, migrations, indexes, queries, transactions, pgbouncer]
+version: 1.0.0
+---
+
+# PostgreSQL — Expertise Guide
+
+## Worker Context
+
+You are a PostgreSQL specialist implementing production database code. Follow these rules exactly.
+
+### Naming Conventions
+
+- Tables: `snake_case`, singular (`user_account`, NOT `users` or `UserAccounts`)
+- Columns: `snake_case` (`created_at`, NOT `createdAt`)
+- Primary keys: `id` (always)
+- Foreign keys: `{referenced_table}_id` (`organization_id`, NOT `org` or `orgId`)
+- Indexes: `idx_{table}_{columns}` (`idx_order_customer_id`)
+- Constraints: `{type}_{table}_{columns}` — `pk_order`, `uq_user_account_email`, `fk_order_customer_id`, `ck_order_status`
+- Enums: lowercase strings matching application enums exactly (`'pending'`, `'in_progress'`, NOT `'Pending'`)
+
+### Data Type Selection
+
+| Use Case | Use This | NEVER This | Why |
+|----------|----------|------------|-----|
+| Primary keys | `uuid` via `gen_random_uuid()` | `serial`/`bigserial` | Non-guessable, safe for distributed systems, no sequence contention |
+| Timestamps | `timestamptz` | `timestamp` | `timestamp` silently drops timezone — data corruption across TZs |
+| Money/currency | `integer` (cents) or `numeric(19,4)` | `float`/`double precision`/`real` | Floating point loses precision: `0.1 + 0.2 != 0.3` |
+| Structured JSON | `jsonb` | `json` | `jsonb` is binary, indexable, deduplicates keys; `json` is raw text |
+| Boolean flags | `boolean` with `NOT NULL DEFAULT false` | `integer` 0/1 | Semantic clarity, type safety |
+| Variable text | `text` | `varchar(n)` | PostgreSQL stores both identically; `varchar(n)` adds check overhead with no storage benefit |
+| IP addresses | `inet` | `text` | Native validation, comparison operators, indexing |
+| Intervals/durations | `interval` | `integer` (seconds) | Native arithmetic: `now() + interval '30 days'` |
+
+CRITICAL: Every table MUST have `id uuid PRIMARY KEY DEFAULT gen_random_uuid()`, `created_at timestamptz NOT NULL DEFAULT now()`, and `updated_at timestamptz NOT NULL DEFAULT now()`.
+
+### Schema Design
+
+**Normalize by default.** Denormalize only when you can prove a read query requires 3+ JOINs on tables exceeding 1M rows AND the query runs >100ms after index optimization.
+
+When denormalizing:
+- Add a comment explaining WHY (`-- denormalized from order_item for dashboard query perf`)
+- Create a trigger or application hook to keep denormalized data in sync
+- NEVER denormalize without a sync mechanism — stale data is worse than slow reads
+
+**Foreign keys MUST have indexes.** Every `_id` column that references another table gets an index. No exceptions. Missing FK indexes cause full table scans on JOINs and CASCADE deletes.
+
+### Index Strategy
+
+```
+Default decision tree:
+  Equality lookup (WHERE x = ?)        → B-tree (default)
+  Range scan (WHERE x > ? AND x < ?)   → B-tree
+  Full-text search                      → GIN on tsvector column
+  JSONB containment (@>, ?)             → GIN
+  Array containment (@>, &&)            → GIN
+  Geometric/spatial                     → GiST
+  Pattern matching (LIKE 'prefix%')     → B-tree (with text_pattern_ops)
+  Pattern matching (LIKE '%infix%')     → GIN with pg_trgm
+```
+
+**Composite indexes:** Column order matters. Put equality columns first, range columns last. `(status, created_at)` for `WHERE status = 'active' AND created_at > ?` — NOT `(created_at, status)`.
+
+**Partial indexes:** Use when queries filter on a constant. `CREATE INDEX idx_order_active ON order (created_at) WHERE status = 'pending'` — indexes only pending orders, dramatically smaller.
+
+**When NOT to index:** Tables under 10K rows (seq scan is faster). Columns with <5% selectivity (e.g., boolean `is_active` where 95% are true). Write-heavy columns that rarely appear in WHERE clauses.
+
+### Migrations
+
+CRITICAL: Every migration MUST have a corresponding rollback (down migration). Test rollbacks before deploying.
+
+**Zero-downtime checklist:**
+1. NEVER `ALTER TABLE ... ALTER COLUMN ... TYPE` on large tables (rewrites entire table, locks exclusively)
+2. NEVER add `NOT NULL` column without a default (table rewrite on PG <11, lock on PG 11+)
+3. NEVER drop columns directly — first stop reading, deploy, then drop in a follow-up migration
+4. Add new columns as nullable first, backfill, then add NOT NULL constraint separately
+5. Rename columns using dual-write: add new column → copy data → update app → drop old column
+
+**Safe migration template:**
+```sql
+-- UP
+BEGIN;
+ALTER TABLE order ADD COLUMN IF NOT EXISTS discount_cents integer;
+COMMIT;
+
+-- DOWN
+BEGIN;
+ALTER TABLE order DROP COLUMN IF EXISTS discount_cents;
+COMMIT;
+```
+
+### Query Optimization
+
+**Read EXPLAIN ANALYZE output — focus on these red flags:**
+- `Seq Scan` on tables >10K rows → missing index
+- `Nested Loop` with high `rows` on inner side → consider `Hash Join` (check work_mem)
+- `Sort` with `external merge Disk` → increase `work_mem` for session
+- Estimated rows wildly off from actual rows → run `ANALYZE` on the table
+
+**N+1 prevention:** NEVER issue queries inside loops. Use JOINs, subqueries, or `WHERE id = ANY($1::uuid[])` with an array parameter.
+
+```sql
+-- GOOD: Single query with JOIN
+SELECT o.id, o.total_cents, c.name
+FROM order o
+JOIN customer c ON c.id = o.customer_id
+WHERE o.status = 'pending';
+
+-- BAD: N+1 — one query per order
+SELECT * FROM order WHERE status = 'pending';
+-- then for EACH order:
+SELECT * FROM customer WHERE id = $1;
+```
+
+**Pagination:** NEVER use `OFFSET` for deep pagination (scans and discards rows). Use keyset/cursor pagination:
+```sql
+-- GOOD: Keyset pagination (constant performance)
+SELECT * FROM order
+WHERE created_at < $cursor_timestamp
+ORDER BY created_at DESC
+LIMIT 20;
+
+-- BAD: Offset pagination (degrades linearly)
+SELECT * FROM order ORDER BY created_at DESC LIMIT 20 OFFSET 10000;
+```
+
+### Transactions
+
+| Isolation Level | Use When | Cost |
+|----------------|----------|------|
+| `READ COMMITTED` (default) | Standard CRUD operations. Fine for most workloads. | Lowest |
+| `REPEATABLE READ` | Reports, analytics that need consistent snapshot. Retrying on serialization failure is acceptable. | Medium |
+| `SERIALIZABLE` | Financial transactions, inventory decrements where race conditions cause data loss. MUST implement retry logic for serialization failures. | Highest |
+
+Keep transactions short. NEVER hold transactions open during external HTTP calls or user input waits.
+
+### Connection Pooling
+
+- Application → PgBouncer → PostgreSQL. NEVER connect directly from app to PG in production.
+- Use `transaction` pooling mode (default). Use `session` mode only if using prepared statements, LISTEN/NOTIFY, or advisory locks.
+- Set PG `max_connections` to cores * 2 + effective_spindle_count (typically 100-200). Let PgBouncer handle thousands of app connections.
+- Connection limit per pool: match your app's max concurrent DB operations, NOT total HTTP connections.
+
+---
+
+## Conventions
+
+### SQL Formatting
+```sql
+-- Keywords: UPPERCASE
+-- Identifiers: lowercase snake_case
+-- One clause per line
+-- Indent JOIN conditions and subqueries
+
+SELECT
+    o.id,
+    o.total_cents,
+    c.name AS customer_name
+FROM order o
+JOIN customer c ON c.id = o.customer_id
+WHERE o.status = 'pending'
+    AND o.created_at > now() - interval '30 days'
+ORDER BY o.created_at DESC
+LIMIT 50;
+```
+
+### Migration File Naming
+```
+{timestamp}_{description}.sql
+20260215143000_create_order_table.sql
+20260215144500_add_discount_to_order.sql
+```
+
+### Default Table Template
+```sql
+CREATE TABLE {table_name} (
+    id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
+    -- domain columns here
+    created_at timestamptz NOT NULL DEFAULT now(),
+    updated_at timestamptz NOT NULL DEFAULT now()
+);
+
+-- updated_at trigger
+CREATE TRIGGER set_{table_name}_updated_at
+    BEFORE UPDATE ON {table_name}
+    FOR EACH ROW
+    EXECUTE FUNCTION set_updated_at();
+
+-- Reusable trigger function (create once per database)
+CREATE OR REPLACE FUNCTION set_updated_at()
+RETURNS TRIGGER AS $$
+BEGIN
+    NEW.updated_at = now();
+    RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;
+```
+
+---
+
+## Common Patterns
+
+### 1. Soft Delete
+```sql
+ALTER TABLE user_account ADD COLUMN deleted_at timestamptz;
+CREATE INDEX idx_user_account_active ON user_account (id) WHERE deleted_at IS NULL;
+
+-- All queries filter: WHERE deleted_at IS NULL
+-- Partial index keeps it fast
+```
+
+### 2. Enum-as-Text with Check Constraint
+```sql
+-- Prefer check constraint over PostgreSQL ENUM type
+-- (ENUM types are painful to modify in production)
+ALTER TABLE order ADD COLUMN status text NOT NULL DEFAULT 'pending'
+    CONSTRAINT ck_order_status CHECK (status IN ('pending', 'confirmed', 'shipped', 'delivered', 'cancelled'));
+```
+
+### 3. JSONB for Flexible Metadata
+```sql
+ALTER TABLE order ADD COLUMN metadata jsonb NOT NULL DEFAULT '{}';
+CREATE INDEX idx_order_metadata ON order USING gin (metadata);
+
+-- Query: find orders with specific tag
+SELECT * FROM order WHERE metadata @> '{"priority": "high"}';
+```
+
+### 4. Advisory Locks for Job Processing
+```sql
+-- Claim a job without row-level lock contention
+SELECT * FROM job_queue
+WHERE status = 'pending'
+    AND pg_try_advisory_xact_lock(id::bigint)
+ORDER BY created_at
+LIMIT 1
+FOR UPDATE SKIP LOCKED;
+```
+
+### 5. Upsert (INSERT ON CONFLICT)
+```sql
+INSERT INTO user_preference (user_account_id, key, value)
+VALUES ($1, $2, $3)
+ON CONFLICT (user_account_id, key)
+DO UPDATE SET value = EXCLUDED.value, updated_at = now();
+```
+
+---
+
+## Anti-Patterns
+
+### 1. SELECT *
+NEVER use `SELECT *` in application code. It breaks when columns are added/removed, fetches unnecessary data, and prevents covering index optimization. Always list columns explicitly.
+
+### 2. Storing Money as Float
+`float` and `double precision` cause rounding errors. `19.99 + 0.01` might not equal `20.00`. Use `integer` (cents) or `numeric(19,4)`. See Data Type Selection above.
+
+### 3. Missing Indexes on Foreign Keys
+Every `_id` column referencing another table needs an index. Without it, JOINs and CASCADE DELETE perform full table scans. PostgreSQL does NOT auto-create indexes on foreign keys.
+
+### 4. Naive OFFSET Pagination
+`OFFSET 100000` scans and discards 100K rows on every query. Performance degrades linearly with page depth. Use keyset pagination — see Query Optimization section.
+
+### 5. Long-Running Transactions
+Transactions holding locks for >5 seconds block other writers. NEVER hold a transaction open during HTTP calls, queue operations, or user-facing waits. Fetch external data first, then open a short transaction to write.
+
+### 6. Using PostgreSQL ENUM Types
+```sql
+-- BAD: Modifying ENUM requires ALTER TYPE which takes ACCESS EXCLUSIVE lock
+CREATE TYPE order_status AS ENUM ('pending', 'shipped');
+-- Adding a value: ALTER TYPE order_status ADD VALUE 'cancelled'; -- Cannot run in transaction!
+
+-- GOOD: text + check constraint — easy to modify
+ALTER TABLE order ADD CONSTRAINT ck_order_status
+    CHECK (status IN ('pending', 'shipped', 'cancelled'));
+```
+
+---
+
+## Integration Notes
+
+### Multi-Session Orchestration
+When working as a database worker in a clearctx team:
+
+1. **Check inbox first** — `team_check_inbox` before any work
+2. **Read shared-conventions artifact** — Match response format, error format, enum values, and column names EXACTLY as defined
+3. **Publish schema artifact** — After creating tables, publish an artifact with table names, column names, and types so backend workers can generate correct queries:
+```json
+{
+  "artifactId": "db-schema",
+  "type": "schema-change",
+  "data": {
+    "tables": ["order", "customer"],
+    "filesCreated": ["migrations/20260215_create_order.sql"],
+    "columnMap": {
+      "order": ["id", "customer_id", "total_cents", "status", "created_at", "updated_at"]
+    }
+  }
+}
+```
+4. **Broadcast completion** — Let teammates know the schema is ready
+5. **Relative paths only** — `migrations/001_create_tables.sql`, NEVER absolute paths
+
+### Coordination with Backend Workers
+- Backend workers depend on your schema artifact for query generation
+- Agree on exact column names via shared-conventions BEFORE writing any SQL
+- If a backend worker asks about column names or types via `team_ask`, reference your published artifact
+- JSONB columns: document the expected shape in your artifact so backend workers serialize correctly
+
+### Coordination with Testing Workers
+- Provide seed data SQL or a migration that inserts test fixtures
+- Document any database-level constraints that affect test setup (unique constraints, check constraints, foreign keys)