session-orchestrator 3.16.0

package/rules/opt-in-domain/prompt-caching.md

@@ -0,0 +1,270 @@
+<!-- source: session-orchestrator plugin (canonical: rules/opt-in-domain/prompt-caching.md) -->
+---
+globs:
+  - "**/lib/ai/**"
+  - "**/lib/llm/**"
+  - "**/providers/**"
+tier: wave-only
+---
+# Prompt Caching Rules (Path-scoped — repos using `@anthropic-ai/sdk` or `@ai-sdk/anthropic`)
+
+> Path-scoped — applies anywhere a project imports `@anthropic-ai/sdk` or uses the `@ai-sdk/anthropic` provider. Out of scope: `session-orchestrator` itself (no SDK use; `backend.md` § "AI Provider Abstraction" already forbids direct SDK imports in business logic, and the orchestrator runs inside Claude Code's harness which manages caching at the platform layer).
+
+## Why
+
+Anthropic prompt caching pays back fast: cache **writes** cost 1.25× base input (5-min ephemeral) or 2.0× base input (1-hour extended), and cache **reads** cost 0.1× base input. A system prompt of ~600 tokens amortises the write penalty after **three** identical calls inside a 5-minute window. The most common adoption failure is silent: the breakpoint is placed on a per-request placeholder (user message, timestamp, request ID), the prefix hash differs every call, and the cache is never hit. This rule encodes the placement discipline and the pre-warming pattern once so consumer repos adopt in <30 LOC.
+
+The four PoC targets (PC-001..PC-007 are validated against these) are documented at the bottom under "Adoption candidates".
+
+## PC-001: System prompt as block array, not string
+
+The fundamental shape. The `system` field accepts a string OR an array of content blocks. Caching requires the array form because `cache_control` is a per-block marker.
+
+```typescript
+// BAD — string system prompt cannot be cached
+const response = await anthropic.messages.create({
+  model: 'claude-opus-4-7',
+  max_tokens: 1024,
+  system: longSystemPromptString, // 600+ tokens, paid in full on every call
+  messages: [{ role: 'user', content: userInput }],
+});
+
+// GOOD — block array with cache_control on the last shared block
+const response = await anthropic.messages.create({
+  model: 'claude-opus-4-7',
+  max_tokens: 1024,
+  system: [
+    {
+      type: 'text',
+      text: longSystemPromptString,
+      cache_control: { type: 'ephemeral' },
+    },
+  ],
+  messages: [{ role: 'user', content: userInput }],
+});
+```
+
+The block array form is mandatory for caching. There is no string-form opt-in.
+
+## PC-002: Breakpoint placement — last shared block, never on per-request placeholder
+
+The trap. Anthropic's docs are explicit: *"Place the `cache_control` breakpoint on the last block that is shared with the follow-up request… not on the placeholder user message."* A breakpoint on per-request content (the user's question, a timestamp, a request ID) hashes differently every call and never produces a cache hit.
+
+```typescript
+// BAD — breakpoint on the per-request user message
+await anthropic.messages.create({
+  model: 'claude-opus-4-7',
+  max_tokens: 1024,
+  system: [{ type: 'text', text: systemPrompt }],
+  messages: [
+    {
+      role: 'user',
+      content: [
+        { type: 'text', text: userQuestion, cache_control: { type: 'ephemeral' } }, // hashes differently every call
+      ],
+    },
+  ],
+});
+
+// BAD — system prompt includes a per-request timestamp before the breakpoint
+const systemWithTimestamp = `${systemPrompt}\n\nCurrent time: ${new Date().toISOString()}`;
+await anthropic.messages.create({
+  system: [{ type: 'text', text: systemWithTimestamp, cache_control: { type: 'ephemeral' } }],
+  // ...
+});
+
+// GOOD — breakpoint on the last STABLE block (system prompt), per-request content lives downstream
+await anthropic.messages.create({
+  model: 'claude-opus-4-7',
+  max_tokens: 1024,
+  system: [
+    { type: 'text', text: systemPrompt, cache_control: { type: 'ephemeral' } }, // stable across requests
+  ],
+  messages: [
+    { role: 'user', content: userQuestion }, // varies — must be AFTER the breakpoint
+  ],
+});
+
+// GOOD — multi-block: tool list cached, system prompt cached, per-session RAG context UNCACHED
+await anthropic.messages.create({
+  model: 'claude-opus-4-7',
+  max_tokens: 1024,
+  system: [
+    { type: 'text', text: toolListDescription, cache_control: { type: 'ephemeral' } },
+    { type: 'text', text: baseSystemPrompt, cache_control: { type: 'ephemeral' } },
+    { type: 'text', text: perSessionRagContext }, // NO cache_control — varies per session
+  ],
+  messages: [{ role: 'user', content: userQuestion }],
+});
+```
+
+Rule: a `cache_control` breakpoint must sit on the LAST block of a contiguous stable prefix. Everything before it (and including it) is cached. Everything after varies freely.
+
+## PC-003: TTL selection — 5min interactive, 1h batch with gaps
+
+Two TTLs are available, with different write-cost trade-offs.
+
+| Use case | TTL | Write cost | When |
+|---|---|---|---|
+| Interactive chat, tool-loop, sequential API calls | `{ type: "ephemeral" }` (default 5 min) | 1.25× base input | 95th-percentile inter-call gap ≤ 5 min |
+| Batch jobs spanning >5 min, scheduled enrichment | `{ type: "ephemeral", ttl: "1h" }` | 2.0× base input | 95th-percentile inter-call gap > 5 min, or single batch run >5 min wall-clock |
+
+Decision rule: **if the 95th-percentile gap between calls that share the prefix exceeds 5 minutes, choose 1h.** Otherwise the default 5-min TTL pays back faster (1.25× vs 2.0×).
+
+```typescript
+// 5-min — interactive chat (interactive assistant tool-loop)
+{ type: 'text', text: systemPrompt, cache_control: { type: 'ephemeral' } }
+
+// 1h — batch re-rank that processes 100 items over 8 min
+{ type: 'text', text: systemPrompt, cache_control: { type: 'ephemeral', ttl: '1h' } }
+```
+
+Instrument before flipping to 1h: log the time delta between successive cache-hitting calls and confirm a meaningful tail beyond 5 min. The 2.0× write cost only amortises if you actually use the longer window.
+
+## PC-004: Pre-warming with `max_tokens: 0`
+
+The mechanism. A `max_tokens: 0` request runs the **full prefill phase** and writes the cache at every `cache_control` breakpoint, then returns immediately with `content: []` and `stop_reason: "max_tokens"`. The first real user turn then pays a cache-read (0.1×) on TTFT instead of a cache-write (1.25×).
+
+```typescript
+// src/instrumentation.ts — Next.js boot hook
+import Anthropic from '@anthropic-ai/sdk';
+import { SYSTEM_PROMPT } from '@/lib/ai/prompts';
+
+export async function register() {
+  const anthropic = new Anthropic();
+  await anthropic.messages.create({
+    model: 'claude-opus-4-7',
+    max_tokens: 0, // returns immediately after prefill; cache is written
+    system: [
+      { type: 'text', text: SYSTEM_PROMPT, cache_control: { type: 'ephemeral' } },
+    ],
+    messages: [{ role: 'user', content: 'warmup' }],
+  });
+}
+```
+
+**When to pre-warm:**
+- **Boot-once** (short-lived edge runtimes, serverless cold starts): one `register()` call covers the runtime's window.
+- **Scheduled** (long-lived servers, persistent processes): cron at `TTL × 0.8` — every 4 minutes for the 5-min TTL, every 48 minutes for the 1h TTL. Skip pre-warming if traffic already keeps the cache warm (≥1 real call per `TTL × 0.8`).
+- **Skip entirely** for batch jobs where the first real call IS the cache write and subsequent items in the same batch read it (PC-003 batch row). The `max_tokens: 0` hop adds no value when the batch's own cadence keeps the cache warm.
+
+**Hard rejections** — Anthropic rejects `max_tokens: 0` combined with any of these. Use a regular `max_tokens: 1` call (and discard the single token) if you need any of:
+
+- `stream: true`
+- Extended thinking (`thinking: { type: 'enabled' }`)
+- Structured outputs (`response_format`)
+- `tool_choice: 'tool'` or `tool_choice: 'any'`
+- Message Batches API
+
+For the AI-SDK equivalent, see PC-006.
+
+## PC-005: Breakpoint budget + order
+
+| Limit | Value |
+|---|---|
+| Max breakpoints per request | **4** |
+| Breakpoint order (server processes in this order) | `tools → system → messages` |
+| Lookback window | **20 blocks** |
+| Concurrency | First request must complete before parallel requests can hit the same cache entry |
+
+With 4 breakpoints you typically allocate: 1 on `tools`, 1 on `system`, up to 2 on `messages` (e.g., long document blocks or RAG context that is stable across a multi-turn session). Spending all 4 on `system` blocks is rarely worthwhile — adjacent text blocks share the same prefix anyway; one breakpoint at the end of the system array caches the lot.
+
+**Verification signals in the response payload:**
+
+```typescript
+const response = await anthropic.messages.create({ /* ... */ });
+console.log(response.usage);
+// {
+//   input_tokens: 12,                  // tokens NOT served from cache
+//   cache_creation_input_tokens: 670,  // tokens written to cache (first call)
+//   cache_read_input_tokens: 0,        // tokens served from cache (subsequent calls)
+//   output_tokens: 248,
+// }
+```
+
+`cache_creation_input_tokens > 0` on the first call confirms a cache write happened. `cache_read_input_tokens > 0` on subsequent calls confirms the cache is being read at 0.1×. If both stay 0 across calls, the breakpoint is on a non-stable block — see PC-002.
+
+## PC-006: Vercel AI SDK adapter shape (`@ai-sdk/anthropic`)
+
+The AI-SDK exposes the same mechanism via `providerOptions`. The shape differs from the raw SDK but the placement discipline (PC-002) is identical.
+
+```typescript
+import { anthropic } from '@ai-sdk/anthropic';
+import { generateText } from 'ai';
+
+const result = await generateText({
+  model: anthropic('claude-opus-4-7'),
+  system: SYSTEM_PROMPT, // string is fine here; cache_control is set via providerOptions
+  messages: [{ role: 'user', content: userQuestion }],
+  providerOptions: {
+    anthropic: {
+      cacheControl: { type: 'ephemeral' }, // applied to the system message
+    },
+  },
+});
+```
+
+For multi-block control (e.g., caching tools + system separately), pass `system` as a structured message and set `providerOptions.anthropic.cacheControl` on each cacheable message via the `experimental_providerMetadata` field on each message. See the `@ai-sdk/anthropic` README for the current message-level shape.
+
+The canonical AI-SDK adoption point is **Candidate D** at `src/lib/llm/adapter.ts`: add an optional `cacheableSystem?: string` parameter; when present, emit the `providerOptions` block above on the system message. Tagging and enrichment call-sites flip it on.
+
+**Pre-warming via AI-SDK:** `max_tokens: 0` is not natively exposed by `generateText`. Use `maxTokens: 1` and discard the single-token output, or drop down to the raw SDK for the pre-warm call only.
+
+## PC-007: Verification — what to grep for
+
+After deployment, the cache must be observed working. Three signals, in order of authority.
+
+1. **Response payload** — log `response.usage.cache_creation_input_tokens` and `response.usage.cache_read_input_tokens` on every cached call. Surface both in your AI-observability pipeline (see `backend.md` § "AI Observability" for the `ai_usage_log` schema — extend it with two columns for these).
+
+2. **Smoke test** — three identical calls in <5 min against the deployed endpoint:
+
+```typescript
+// scripts/smoke-test-cache.ts
+const results = [];
+for (let i = 0; i < 3; i++) {
+  const r = await anthropic.messages.create({
+    model: 'claude-opus-4-7',
+    max_tokens: 16,
+    system: [{ type: 'text', text: SYSTEM_PROMPT, cache_control: { type: 'ephemeral' } }],
+    messages: [{ role: 'user', content: 'ping' }],
+  });
+  results.push({
+    call: i + 1,
+    cache_creation: r.usage.cache_creation_input_tokens ?? 0,
+    cache_read: r.usage.cache_read_input_tokens ?? 0,
+  });
+}
+console.table(results);
+// Expected:
+// call 1: cache_creation > 0, cache_read = 0   (write)
+// call 2: cache_creation = 0, cache_read > 0   (read)
+// call 3: cache_creation = 0, cache_read > 0   (read)
+```
+
+3. **Failure mode** — if both columns stay 0 across all three calls, the breakpoint is on a non-stable block. Most common causes: timestamp interpolated into the system prompt, user message marked with `cache_control`, request ID concatenated into a system block. Re-read PC-002 and grep the call-site for per-request data inside the cached prefix.
+
+For Vercel AI SDK consumers, the `cache_creation_input_tokens` / `cache_read_input_tokens` fields surface on `result.providerMetadata?.anthropic?.usage`. Same smoke-test pattern, different access path.
+
+## Adoption candidates (cross-reference)
+
+The four PoCs filed against this rule. Each adopts in <30 LOC and is independently verifiable via PC-007.
+
+- **Candidate A** (STRONG) — an opus tool-loop Next.js API; system prompt + tool-list = stable prefix across tool turns. Block-array + `cache_control` on last system block. TTL: 5min. Pre-warm: `register()` hook in Next.js `instrumentation.ts`.
+- **Candidate B** (STRONG) — a Next.js app with a ~668-line system prompt (textbook caching case); cache-read at 0.1× pays back after ~3 requests in a 5-min window even for Haiku-4.5 traffic. TTL: 5min for chat, 1h for the RAG-enhanced variant (breakpoint BEFORE the RAG block). Pre-warm: cron every 4 min during business hours.
+- **Candidate C** (MED) — an AI-SDK adapter service; add optional `cacheableSystem?: string` parameter, emit `providerOptions: { anthropic: { cacheControl: { type: 'ephemeral' } } }` on the system message when present. TTL: 5min. Nightly batch cadence keeps it warm — no pre-warm needed.
+- **Candidate D** (MED) — a batch re-rank + translate service with versioned prompt files. Add `cache_control` to the system block; **skip `max_tokens: 0` entirely** — let the first real call write the cache; subsequent items in the same batch read it. TTL: 5min default; switch to 1h if a single batch run takes >5 min (instrument first).
+
+## Anti-Patterns
+
+- **Breakpoint on the user/placeholder message** — hashes differently every call, never hits. PC-002 is the explicit guard. Re-read the Anthropic docs quote if tempted: *"not on the placeholder user message."*
+- **Forgetting to switch `system` from string to block array** — `cache_control` is silently dropped from a string-shaped system prompt. The request succeeds, the cache is never written, and there is no error.
+- **Cron cadence > TTL** — pre-warming every 6 min for the 5-min TTL means the cache expires between warms. Cron at `TTL × 0.8` (every 4 min for 5min TTL, every 48 min for 1h TTL).
+- **`stream: true` on the pre-warm call** — rejected by the API. Pre-warm calls must be non-streaming. The user-facing call after the warm can stream freely against the now-warm cache.
+- **Per-request content (timestamps, request IDs, user IDs) inside the cached prefix** — even a 5-character difference invalidates the cache. Keep all per-request data strictly AFTER the last `cache_control` breakpoint.
+- **Mixing `cache_control` with `tool_choice: 'tool' | 'any'`** — rejected by the API on the pre-warm call (`max_tokens: 0`). The regular call works, but the pre-warm has to drop the forced tool choice.
+- **Spending all 4 breakpoints on adjacent system text blocks** — wastes the budget. Adjacent blocks share a prefix; one breakpoint at the end caches the whole array. Reserve breakpoints for genuinely separate cacheable regions (tools, system, long-lived document context).
+- **Skipping verification** — every cache adoption ships with the PC-007 smoke test. A silent miss costs 12× more per call than a successful hit (1.25× write + 0.1× read amortised vs 1.0× × N forever).
+
+## See Also
+
+backend.md · testing.md · development.md

package/rules/opt-in-stack/backend-data.md

@@ -0,0 +1,188 @@
+<!-- source: session-orchestrator plugin (canonical: rules/opt-in-stack/backend-data.md) -->
+---
+globs:
+  - src/lib/db/**
+  - src/lib/cache/**
+  - src/services/db/**
+  - supabase/**
+  - migrations/**
+  - src/lib/redis/**
+  - src/lib/queue/**
+tier: wave-only
+---
+# Backend Data Rules (Path-scoped)
+
+## Database (Supabase)
+- RLS on every table. No exceptions.
+- Never use `service_role` key in client-side code.
+- Generate types after schema changes: `pnpm db:types` or `supabase gen types typescript`.
+- Naming: snake_case for tables and columns.
+- Always include `created_at`, `updated_at` timestamps.
+- Soft-delete pattern where appropriate (`deleted_at` column + RLS filter).
+- Connection pooling: use Supabase's built-in connection pooler (port 6543) for serverless environments. Direct connections (port 5432) for long-lived services.
+- Query optimization: use `.select()` to limit returned columns. Avoid `select('*')` in production code.
+
+## GDPR Cascade Deletion (DSGVO Art. 17)
+- On account deletion, ALL personal data must be cascade-deleted. No orphaned PII.
+- Use PostgreSQL `ON DELETE CASCADE` on foreign keys referencing `auth.users`. For complex cases, create a `delete_user_data(user_id uuid)` RPC function that handles all tables in the correct order.
+- Audit trail retention: anonymize PII fields (`SET name = NULL, email = NULL`) or replace with a one-way hash rather than hard-deleting rows required for compliance.
+- Define retention periods per table: invoices/receipts 7 years (AT BAO tax law), access logs 90 days, session data 30 days, marketing consent until withdrawal.
+- Immutable-retention regimes (AT BAO §132): corrective fixes MUST INSERT a new row with `effective_from = fix-date` — never UPDATE historical rows. Read semantics: highest `effective_from <= query_date` wins.
+- Soft-delete users first (`deleted_at` + anonymize PII), then hard-delete after retention period expires via a scheduled cleanup job.
+- Cache invalidation: on deletion, purge all cache keys for the user (`v1:user:{id}`, related entities).
+- Test cascade deletion in integration tests: insert a full user graph, call `delete_user_data()`, assert zero rows with that `user_id` across all PII-bearing tables.
+
+## Migration Patterns
+- Migrations: timestamped SQL files in `supabase/migrations/`. Test migrations before applying to production.
+- Always write reversible migrations: include both `up` and `down` logic.
+- Zero-downtime strategy: add new columns as nullable first, backfill, then add NOT NULL constraint in a follow-up migration.
+- Never rename columns directly in production — add new column, migrate data, drop old column across 3 separate migrations.
+- Lock-safe: avoid `ALTER TABLE ... ADD COLUMN ... DEFAULT` on large tables (takes ACCESS EXCLUSIVE lock in older PG). Use backfill instead.
+- Test migrations against a local Supabase instance (`supabase start`, `supabase db reset`) before pushing to staging/production.
+- One logical change per migration file. Never combine unrelated schema changes.
+- Migration ordering: production Supabase tracks creation order in `supabase_migrations.schema_migrations`; a naive `*.sql` glob lex-sorts by filename. A back-dated date prefix (seq `020` dated earlier than the dependency at seq `017`) runs out of dependency order and fails on the missing column. Sort by the 6-digit sequence, or drop the date prefix entirely (sequence-only naming).
+- Glob recursively: `for f in supabase/migrations/*.sql` is non-recursive — nested files (rsync without trailing slash, `migrations/migrations/`) are silently skipped and old ones re-run with no error and no log. Use `find supabase/migrations -name "*.sql" | sort`, plus a post-deploy `ls … | wc -l` local-vs-staging sanity check.
+- History drift: migrations applied via the SQL Editor do NOT update `supabase_migrations.schema_migrations`, so the next `supabase db push` re-applies them. Run `supabase migration repair --status applied <version> --linked` for each untracked migration first; always dry-run/compare the expected list (`supabase migration list --linked`) before pushing.
+- Seed files must be explicitly listed in `supabase/config.toml [db.seed] sql_paths` — a missing file is a silent skip, not a loud error. Verify each new seed actually ran by checking its `RAISE NOTICE` output in `db:reset`.
+- SQLite→Postgres trajectory (typical v1→v2): add a ~100-LOC lint banning SQLite-dialect tokens (`PRAGMA`, `AUTOINCREMENT`, `WITHOUT ROWID`, `ROWID`, `BLOB`, `DATETIME`) in migration SQL and wire it fail-fast into the lint gate from day 1 — a SQLite-only test suite will NOT catch dialect lock-in (e.g. a duplicate `CREATE TABLE schema_migrations`).
+- Single-row config/toggle tables: enforce row-count=1 at the DB layer with a boolean singleton PK + `ON CONFLICT DO NOTHING` seed — idempotent migration:
+```sql
+singleton_id boolean PRIMARY KEY DEFAULT true,
+CONSTRAINT one_row CHECK (singleton_id = true)
+-- seed: INSERT ... ON CONFLICT DO NOTHING
+```
+
+## RLS Performance (MUST)
+
+Supabase RLS policies can degrade query performance 10-100x without these optimizations. Full checklist with examples: see the baseline `security-compliance` rules § RLS Performance Checklist (not vendored into this plugin).
+
+- **Index policy columns:** Every column referenced in an RLS policy MUST have a btree index. Missing indexes are the #1 RLS performance killer.
+- **Wrap `auth.uid()` in `(SELECT ...)`:** Use `(SELECT auth.uid())` instead of bare `auth.uid()`. This enables PostgreSQL initPlan caching (9ms vs 179ms on 100K rows).
+- **Specify `TO authenticated`:** Always specify the target role to stop execution early for unauthenticated users.
+- **Avoid joins in policies:** Use `IN` or `ANY` subqueries instead of join chains.
+- **Never use `user_metadata`:** Users can modify `raw_user_meta_data` client-side. Use `raw_app_meta_data` (server-only) instead.
+- **`service_role` bypasses RLS — policies become dead code:** an admin/`service_role` client bypasses RLS entirely, so RLS INSERT/UPDATE policies on server-written tables never evaluate; superuser/`psql` probes prove nothing. Verify policies under `SET LOCAL ROLE authenticated` + `request.jwt.claims`, or route writes through the session client if RLS must enforce. When a migration `REVOKE`s table privileges, audit every `SECURITY INVOKER` function that writes there.
+
+## View Security (MUST)
+
+- All PostgreSQL views accessing RLS-protected tables MUST use `security_invoker = true` (PostgreSQL 15+). Without it, views execute as the view owner (typically superuser), bypassing RLS entirely.
+```sql
+-- UNSAFE: view bypasses RLS (executes as owner)
+CREATE VIEW my_view AS SELECT * FROM protected_table;
+
+-- SAFE: view respects caller's RLS policies
+CREATE VIEW my_view WITH (security_invoker = true) AS SELECT * FROM protected_table;
+```
+- This applies to ALL views exposed via Supabase client queries.
+- For materialized views, RLS does not apply — use explicit `WHERE` clauses and restrict access via `GRANT`.
+- **`SECURITY DEFINER` functions run as the OWNER, not the caller:** `current_user` is fixed to the function owner, so a `current_user`-based allowlist bypass is internally inconsistent (owner-in-list → always bypasses, the allowlist is useless; owner-not-in-list → never bypasses by role). For caller-aware logic use `SECURITY INVOKER`, `session_user`, or a hybrid (`current_user = session_user AND …`). A txn-local GUC gate is the safe escape hatch for immutability triggers while RLS stays the primary barrier.
+
+## PostgREST & Supabase Operational Gotchas
+
+- **Reload the PostgREST schema cache after DDL:** after `ALTER TABLE ADD COLUMN` or a new RPC, PostgREST keeps a STALE schema cache → silent `PGRST204` for the new column for hours. Reload by sending `SIGUSR1` to the rest container (`docker kill --signal=SIGUSR1 <supabase_rest_*>`); `NOTIFY pgrst` alone is NOT sufficient. Apply atomically with the migration and confirm the column landed in prod with `psql \d`.
+- **PostgREST silently caps at 1000 rows:** any Supabase query without `.range()` returns at most 1000 rows → skewed aggregations (the "everything is exactly 1000" tell). Always paginate full-dataset queries in scripts and tools.
+- **`pg` (node) returns `numeric`/`bigint`/`interval`/`time` as JS STRINGS** (precision preservation): at Zod boundaries use `z.coerce.number()`, never `z.number()` — the latter passes lint/typecheck but throws only in prod (e.g. when a SECURITY DEFINER RPC's numeric score hits the validator). Catch it with an integration smoke test, not unit mocks.
+- **Validate enum/CHECK-column INSERTs against REAL Postgres:** unit mocks accept any string, so they cannot prove a ternary mapping (e.g. `disposition → report|enforce`) is load-bearing. Run the INSERT through actual Postgres (`docker exec … psql`) — right-sized vs a full-app E2E when the changed flow is DB-level.
+
+## Caching
+- Upstash Redis for distributed caching and rate limiting.
+- Cache invalidation strategy: explicit invalidation on mutation, TTL as fallback.
+- Never cache user-specific sensitive data without encryption.
+- Cache key convention: `{service}:{entity}:{id}` (e.g., `bg:invoice:uuid-123`).
+- Default TTL: 5 minutes for API responses, 1 hour for config, 24 hours for static lookups.
+
+## Queue Processing
+- BullMQ + Redis for job queues.
+- Stalled job protection: lock duration, stalledInterval.
+- Idempotent job processing (handle duplicates gracefully).
+- Dead letter queue for persistent failures.
+
+## Cache Strategy
+
+### TTL-Based Caching
+- Use Upstash Redis for distributed caching. Key format: `v1:{entity}:{id}`.
+- Default TTLs: user profiles 5min, config/settings 15min, public listings 1min.
+- Probabilistic early expiration to prevent thundering herd: refresh at `TTL * 0.8 + random(0, TTL * 0.2)`.
+
+### Cache-Aside Pattern
+- Read: check cache → if miss, fetch from DB → write to cache → return.
+- Write: update DB → invalidate cache (never update cache directly on write).
+- Use `JSON.stringify`/`JSON.parse` for complex objects. Validate shape on read (cache corruption safety).
+
+```ts
+// Cache-aside with explicit invalidation
+async function getUser(id: string): Promise<User> {
+  const cached = await redis.get(`v1:user:${id}`);
+  if (cached) return JSON.parse(cached) as User;
+
+  const user = await supabase.from('users').select().eq('id', id).single();
+  await redis.set(`v1:user:${id}`, JSON.stringify(user.data), { ex: 300 });
+  return user.data as User;
+}
+
+// On mutation — invalidate, never update cache directly
+async function updateUser(id: string, data: Partial<User>): Promise<void> {
+  await supabase.from('users').update(data).eq('id', id);
+  await redis.del(`v1:user:${id}`);
+}
+```
+
+### Cache Key Conventions
+- Format: `v{version}:{entity}:{identifier}` (e.g., `v1:user:uuid`, `v1:invoice:uuid`).
+- Bump version prefix when schema changes (invalidates all old keys).
+- Namespace by environment: `{env}:v1:user:uuid` in shared Redis instances.
+
+### Cache Warming
+- Post-deployment: warm critical caches (frequently accessed configs, feature flags).
+- Use background jobs, not blocking startup.
+- Monitor cache hit rate — alert if < 80% sustained.
+
+## N+1 Query Prevention
+
+### Detection
+- Enable query logging in development: log all Supabase queries with timing.
+- Watch for patterns: N identical queries in a single request handler.
+- Use `performance.mark()` / `performance.measure()` to profile request handlers.
+
+### Prevention with Supabase
+- Use embedded selects: `.select('*, profiles(*)')` instead of separate queries.
+- Batch fetching: `.in('id', ids)` instead of looping `.eq('id', id)`.
+- For complex joins: use Supabase views or RPC functions.
+- Limit `.select()` to needed columns — avoid `select('*')` in production.
+
+### Patterns to Avoid
+- `for (const item of items) { await supabase.from('x').select().eq('id', item.id) }` — use `.in()` instead.
+- Fetching parent then children separately when a join would suffice.
+- Multiple sequential queries that could be parallelized with `Promise.all()`.
+
+## Realtime Subscriptions
+
+### Supabase Realtime Setup
+- Enable Realtime on tables via Supabase Dashboard → Database → Replication.
+- Subscribe pattern: `supabase.channel('name').on('postgres_changes', { event: '*', schema: 'public', table: 'messages' }, handler).subscribe()`.
+- RLS applies to Realtime — users only receive events for rows they can SELECT.
+
+### Client-Side Management
+- Always unsubscribe on component unmount: `supabase.removeChannel(channel)` in useEffect cleanup.
+- Handle reconnection: Realtime auto-reconnects, but re-subscribe if channel enters `CLOSED` state.
+- Debounce rapid updates: batch UI updates from high-frequency channels.
+
+### Best Practices
+- Limit subscriptions per client (max 10 concurrent channels recommended).
+- Use specific filters (`filter: 'user_id=eq.uuid'`) to reduce payload volume.
+- Never subscribe to entire tables without filters in production.
+- Monitor Realtime connections via Supabase Dashboard metrics.
+
+## Query Performance & Profiling
+- Run `EXPLAIN ANALYZE` before deploying complex queries (joins, subqueries, CTEs). Review actual vs estimated rows.
+- Index strategy: always index foreign keys. Use composite indexes for frequently combined WHERE clauses (leftmost prefix rule).
+- Use `pg_stat_statements` (enabled by default in Supabase) to identify slow queries. Review weekly.
+- Alert threshold: log and review any query exceeding 100ms. Investigate and optimize queries exceeding 500ms.
+- Connection pooling: use Supabase's pgbouncer (port 6543) for serverless/edge functions. Direct connections (port 5432) for long-lived backend services.
+- Avoid `SELECT *` in production queries. Always specify needed columns with `.select('col1, col2')`.
+- Pagination: use cursor-based pagination (keyset) for large datasets. Offset-based pagination degrades at high offsets.
+- Keyset over OFFSET when the dataset is MUTATED inside the paging loop (dedup, enrichment, cleanup): page on an immutable PK (`WHERE r.id > p_cursor_id`), never OFFSET — mutating WHERE-filter writes break the stable physical row order OFFSET depends on, silently skipping or re-visiting rows.
+- Batch operations: use `.upsert()` or `.insert()` with arrays instead of individual inserts in loops.
+
+## See Also
+development.md · security.md · security-web.md · testing.md · frontend.md · backend.md · swift.md · mvp-scope.md · cli-design.md · parallel-sessions.md