mustflow 2.38.0 → 2.58.0

package/templates/default/locales/en/.mustflow/skills/database-json-modeling-review/SKILL.md

@@ -0,0 +1,171 @@
+---
+mustflow_doc: skill.database-json-modeling-review
+locale: en
+canonical: true
+revision: 1
+lifecycle: mustflow-owned
+authority: procedure
+name: database-json-modeling-review
+description: Apply this skill when code, schema, migrations, repositories, ORM models, SQL, docs, tests, or reviews introduce, change, or assess database JSON, jsonb, metadata, extra, options, settings, attributes, properties, raw payload, event context, dynamic field, generated column, computed column, expression index, JSON search index, JSON path query, schemaVersion, JSON key registry, or JSON-to-column promotion behavior.
+metadata:
+  mustflow_schema: "1"
+  mustflow_kind: procedure
+  pack_id: mustflow.core
+  skill_id: mustflow.core.database-json-modeling-review
+  command_intents:
+    - changes_status
+    - changes_diff_summary
+    - test_related
+    - test
+    - lint
+    - build
+    - docs_validate_fast
+    - test_release
+    - mustflow_check
+---
+
+# Database JSON Modeling Review
+
+<!-- mustflow-section: purpose -->
+## Purpose
+
+Keep JSON columns from becoming a place where schema, ownership, validation, indexing, retention, and product decisions disappear.
+
+The review question is: "Is this value truly document-shaped context, or is it a real domain field pretending to be flexible?" JSON is acceptable for bounded optional context, raw external evidence, event-specific details, rare settings, or payloads that are inspected by humans. A key that drives filters, ordering, joins, permissions, uniqueness, status, dates, money, tenant scope, deletion, audit, or calculations is no longer harmless metadata.
+
+<!-- mustflow-section: use-when -->
+## Use When
+
+- A database column, ORM field, migration, fixture, repository, query, generated client, seed, or docs surface uses `JSON`, `jsonb`, `json`, `metadata`, `extra`, `options`, `settings`, `attributes`, `properties`, `payload`, `raw_payload`, `context`, `details`, or similar flexible fields.
+- Code filters, sorts, groups, joins, aggregates, validates, authorizes, displays, exports, deletes, audits, or backfills data inside a JSON column.
+- A JSON key might become a status, tenant key, permission bit, date, numeric value, money amount, uniqueness rule, foreign key, lifecycle state, search dimension, reporting dimension, or retention trigger.
+- A raw provider payload, webhook body, import row, API response, analytics event, audit detail, or event context is persisted and might be confused with operational truth.
+- A migration, review, or performance claim mentions generated columns, computed columns, expression indexes, GIN indexes, `jsonb_path_ops`, JSON search indexes, `JSON_VALUE`, `JSON_EXTRACT`, `ISJSON`, check constraints, schema versions, or key registries.
+- A JSON blob holds arrays of objects, mutable and immutable data together, multiple domains, unbounded maps, user-defined keys, or values owned by different teams or services.
+
+<!-- mustflow-section: do-not-use-when -->
+## Do Not Use When
+
+- The task only changes public API JSON, CLI JSON output, config files, serialization, parsing, or TypeScript object shapes with no database persistence; use the relevant API, CLI, config, or type skill.
+- The task is only database query latency, plan shape, N+1, pagination, or index-fit review; use `database-query-bottleneck-review` first and this skill only when JSON keys hide schema decisions.
+- The task is only migration rollout, online DDL, backfill safety, old/new code compatibility, or destructive schema change; use `database-migration-change` first and this skill as an adjunct for JSON field promotion or shape changes.
+- The task is generic persistence ownership with no flexible JSON column, JSON path, or raw payload surface; use `database-change-safety`.
+- The JSON is a small, bounded, never-queried fixture literal or test-only object whose shape is not persisted or exposed as product behavior.
+
+<!-- mustflow-section: required-inputs -->
+## Required Inputs
+
+- JSON column role: raw external payload, optional context, event details, rare settings, dynamic attributes, cache snapshot, API projection, audit diff, or operational state.
+- Database engine and feature set when known: PostgreSQL `jsonb` versus `json`, MySQL generated columns, SQL Server computed columns and `ISJSON`, Oracle JSON search indexes, SQLite JSON functions, ORM support, and migration tooling.
+- Key inventory: allowed keys, value types, null policy, default policy, units, precision, timezone or date-only policy, owner, reader, writer, removal policy, and schema version.
+- Query and behavior paths: JSON keys used in `WHERE`, `ORDER BY`, `GROUP BY`, joins, foreign keys, uniqueness, status transitions, permissions, tenant scope, retention, deletion, audit, reporting, search, or calculations.
+- Raw versus operational truth split: which fields are copied into typed columns or child tables, which payload remains immutable evidence, and how drift is reconciled.
+- Cardinality and shape: scalar values, bounded arrays, unbounded arrays, arrays of objects, free-form maps, nested objects, per-domain subdocuments, and expected growth.
+- Migration and compatibility expectations: existing rows, missing keys, old writers, new readers, backfill, dual-write, validation, rollback or roll-forward, generated/computed column deployment, index creation, and cleanup.
+- Tests, fixtures, docs, and configured command-intent contract entries relevant to schema, query, migration, release, and mustflow validation.
+
+<!-- mustflow-section: preconditions -->
+## Preconditions
+
+- The task matches the Use When conditions and does not match the Do Not Use When exclusions.
+- Required inputs are available, or missing JSON role, key ownership, engine, workload, or migration evidence can be reported without guessing.
+- Higher-priority instructions and `.mustflow/config/commands.toml` have been checked for the current scope.
+- If JSON keys affect authorization, personal data, secrets, retention, audit, tenant scope, or provider payloads, also use the relevant security, privacy, or adapter-boundary skill.
+- If JSON key promotion changes old/new schema compatibility, also use `database-migration-change`.
+- If JSON path predicates, indexes, generated columns, or planner claims are made, also use the matching database engine skill or `database-query-bottleneck-review` before claiming runtime behavior.
+
+<!-- mustflow-section: allowed-edits -->
+## Allowed Edits
+
+- Add or change schema, migrations, ORM models, generated/computed columns, check constraints, JSON key validation, child tables, indexes, repositories, tests, fixtures, docs, and directly synchronized template surfaces tied to the task.
+- Promote JSON keys into typed columns or child tables when they drive query, integrity, lifecycle, authorization, reporting, or high-volume behavior.
+- Preserve raw external payloads separately from operational columns when evidence or replay value is needed.
+- Add key registries, schema versions, owner documentation, and promotion criteria where flexible fields remain.
+- Do not treat "flexibility" as sufficient justification for storing business state in a JSON blob.
+- Do not add engine-specific JSON indexes, generated columns, computed columns, or search indexes without naming the query shape, write cost, migration path, and engine support boundary.
+
+<!-- mustflow-section: procedure -->
+## Procedure
+
+1. Classify the JSON column role.
+   - Raw evidence: provider payload, webhook body, import row, API response, or audit before/after detail. Keep immutable where possible and copy operational facts into typed fields.
+   - Optional context: bounded details that are displayed or logged but do not control product behavior.
+   - Rare settings: low-cardinality, low-query settings that have owners, defaults, validation, and a migration path.
+   - Dynamic attributes: customer-defined or integration-defined values. Keep query and authorization promises explicit before accepting the shape.
+   - Cache or projection: rebuildable data. State the source of truth and invalidation path.
+2. Run the promotion test for every JSON key mentioned by the diff.
+   Promote or duplicate into typed schema when a key is used for:
+   - `WHERE`, `ORDER BY`, `GROUP BY`, joins, foreign keys, uniqueness, search ranking, or report grouping.
+   - Status, workflow, permission, entitlement, tenant scope, ownership, visibility, delete lifecycle, retention, or audit policy.
+   - Numeric calculations, money, quota, inventory, dates, timestamps, timezones, durations, or service-level objectives.
+   - High-volume access, concurrency control, idempotency, retry state, reconciliation, or operator dashboards.
+3. Separate raw payload from operational truth.
+   - Store provider or event payloads as evidence, not as the only current state.
+   - Copy stable provider identifiers, event ids, status, amounts, timestamps, tenant ids, and reconciliation cursors into typed fields when code depends on them.
+   - Never require future maintainers to remember which nested path is the real customer status, permission bit, or deletion marker.
+4. Check shape and cardinality.
+   - Scalars may stay in JSON only when they are not behavior-driving and remain validated.
+   - Bounded arrays of simple values need a documented maximum and duplicate/null policy.
+   - Arrays of objects usually want child tables when they need filtering, joining, ordering, uniqueness, independent updates, audit, or growth.
+   - Unbounded maps need ownership, key namespace rules, quotas, validation, and export semantics.
+   - Split mutable data from static evidence so updates do not rewrite large raw payloads or blur audit history.
+   - Split unrelated domains. Payment details, UI preferences, AI policy state, security flags, and provider payloads do not belong in one shared `metadata` blob.
+5. Define the JSON contract if JSON remains.
+   - Document allowed keys, type, unit, default, nullability, owner, reader, writer, retention, deletion, and deprecation policy.
+   - Add `schemaVersion` or an equivalent version field when shape can evolve.
+   - Add validation at the database, model, or write-boundary level. Application-only comments are not a contract.
+   - Keep unknown-key behavior explicit: reject, ignore, preserve, namespace, or quarantine.
+6. Review engine-specific storage and indexing.
+   - PostgreSQL: prefer `jsonb` for queryable JSON, use ordinary columns for hot keys, use expression indexes for scalar paths, choose GIN only for containment/search shapes, and check whether `jsonb_path_ops` fits containment at the cost of operator coverage.
+   - MySQL: use generated columns for indexed JSON paths, keep path expressions stable, and verify type/collation casts before claiming index use.
+   - SQL Server: use `ISJSON` checks where appropriate, computed columns for indexed JSON values, and `JSON_VALUE`/`OPENJSON` paths with explicit type and length behavior.
+   - Oracle: distinguish function-based indexes for known scalar paths from JSON search indexes for broader document search, and document which query class each supports.
+   - SQLite: JSON functions are not a schema substitute; expression indexes and generated columns need version and runtime support checks.
+7. Check integrity, migrations, and compatibility.
+   - Backfill promoted columns from JSON with missing-key, malformed-value, duplicate, timezone, precision, and default handling.
+   - Keep old and new app versions compatible when rolling deploys can read or write both shapes.
+   - Decide whether JSON remains as raw evidence, becomes derived cache, or is removed after promotion.
+   - Add constraints and tests for typed promoted values, not only JSON parser success.
+8. Check observability and operations.
+   - Add visibility for malformed JSON, unknown keys, rejected keys, backfill counts, promoted-field drift, and JSON path query latency when the repository has such patterns.
+   - Make operator-facing docs and dashboards use typed fields for operational truth instead of nested-path folklore.
+
+<!-- mustflow-section: postconditions -->
+## Postconditions
+
+- JSON column role, key ownership, allowed shape, schema version, and raw-versus-operational split are explicit.
+- Behavior-driving JSON keys are promoted, duplicated into typed fields, or reported with a clear reason they remain JSON.
+- Engine-specific JSON indexing, generated/computed columns, validation, and migration claims are tied to evidence or marked unverified.
+- Existing rows, old/new writers, backfill, validation, rollback or roll-forward, and cleanup expectations are addressed.
+- Remaining JSON modeling risk is named instead of hidden under "flexibility".
+
+<!-- mustflow-section: verification -->
+## Verification
+
+- Run the narrowest configured tests covering schema, repository, migration, query, validation, fixtures, docs, and template packaging changed by the task.
+- Prefer assertions that prove promoted columns, child tables, key validation, schema versions, malformed-value handling, old-shape compatibility, and raw payload preservation.
+- Use configured docs and release checks when skill text, routes, templates, package metadata, or user-facing docs change.
+- If live database plans or provider payload samples are unavailable, report static-review evidence level rather than claiming production safety.
+
+<!-- mustflow-section: failure-handling -->
+## Failure Handling
+
+- If the engine or ORM feature support is unknown, stop at the modeling recommendation and report the engine-specific verification gap.
+- If promotion would require destructive or long-running migration work, report the expand/backfill/contract path instead of collapsing it into one step.
+- If a JSON blob is intentionally flexible for customer-defined fields, require namespace, quotas, validation, export, and query promises before accepting it.
+- If a configured test or build fails, preserve the failing intent and output tail, then fix only the JSON modeling contract or synchronized surface exercised by the failure.
+
+<!-- mustflow-section: output-format -->
+## Output Format
+
+Report:
+
+- JSON surface reviewed
+- JSON role and key inventory
+- Promotion decisions: typed column, child table, generated/computed column, expression index, JSON index, raw evidence, or remain JSON
+- Raw payload versus operational truth split
+- Engine-specific storage, validation, and index notes
+- Migration, backfill, compatibility, and cleanup status
+- Tests and command intents run
+- Evidence level: static diff risk, configured-test evidence, engine-feature evidence, plan evidence, production evidence, manual-only, missing, or not applicable
+- Remaining JSON modeling risk

package/templates/default/locales/en/.mustflow/skills/database-lock-contention-review/SKILL.md

@@ -0,0 +1,192 @@
+---
+mustflow_doc: skill.database-lock-contention-review
+locale: en
+canonical: true
+revision: 1
+lifecycle: mustflow-owned
+authority: procedure
+name: database-lock-contention-review
+description: Apply this skill when code is created, changed, reviewed, or reported and database lock contention, hot rows, deadlocks, lock waits, row locks, gap or next-key locks, metadata locks, DDL locks, `SELECT ... FOR UPDATE`, `SKIP LOCKED`, optimistic locking, conditional updates, queue table claiming, counter caches, stock or balance updates, long transactions, lock timeouts, idle-in-transaction sessions, or lock observability can make database-backed work block, serialize, deadlock, or overload under concurrent traffic.
+metadata:
+  mustflow_schema: "1"
+  mustflow_kind: procedure
+  pack_id: mustflow.core
+  skill_id: mustflow.core.database-lock-contention-review
+  command_intents:
+    - changes_status
+    - changes_diff_summary
+    - lint
+    - build
+    - test_related
+    - test
+    - docs_validate_fast
+    - test_release
+    - mustflow_check
+---
+
+# Database Lock Contention Review
+
+<!-- mustflow-section: purpose -->
+## Purpose
+
+Review database-backed write paths by asking whether traffic is forced through the same row, range, index prefix, metadata lock, queue head, or long transaction.
+
+The review question is not "is the database slow?" It is "which shared database resource are concurrent requests waiting on, and can the data shape, lock strength, lock order, transaction width, retry policy, and observability make that wait bounded and intentional?"
+
+<!-- mustflow-section: use-when -->
+## Use When
+
+- Code creates, changes, reviews, or reports a database write path, queue table, worker claim query, counter cache, balance or stock update, reservation flow, idempotency record, ledger projection, status transition, batch update, delete job, online DDL path, migration plan, or lock-sensitive read path.
+- The risk is hot-row contention, parent-row counters, shared aggregate rows, `SELECT ... FOR UPDATE`, `FOR NO KEY UPDATE`, `FOR SHARE`, `FOR KEY SHARE`, `NOWAIT`, `SKIP LOCKED`, optimistic version checks, conditional updates, advisory locks, distributed locks backed by database state, MySQL gap or next-key locks, SQL Server lock escalation, PostgreSQL DDL lock levels, MySQL metadata locks, deadlocks, lock waits, pool waits, or idle transactions.
+- Code or docs claim a path is safe under high write concurrency, deadlock-safe, lock-efficient, no-downtime, queue-consumer safe, stock-safe, balance-safe, order-preserving, or low-lock.
+- A database review needs a contention pass separate from query latency, migration compatibility, transaction correctness, generic race conditions, or backend observability.
+
+<!-- mustflow-section: do-not-use-when -->
+## Do Not Use When
+
+- The task is only query latency, cardinality, projection width, pagination, N+1 access, or plan shape without blocking or lock contention risk; use `database-query-bottleneck-review`.
+- The task is primarily old-code/new-schema compatibility, DDL rollout, backfill, generated clients, or destructive migration safety; use `database-migration-change` first and this skill only for lock-contention adjunct review.
+- The task is primarily read -> decision -> write correctness, durable constraints, rollback behavior, transaction propagation, or after-commit side effects; use `transaction-boundary-integrity-review` first and this skill only for blocking, deadlock, or hot-resource pressure.
+- The task is generic in-memory, filesystem, queue, socket, timer, or distributed shared-state interleaving with no database lock surface; use `race-condition-review` or `concurrency-invariant-review`.
+- The database work is a one-off, manually bounded maintenance action on tiny data with no concurrent application traffic and no production-safety claim.
+
+<!-- mustflow-section: required-inputs -->
+## Required Inputs
+
+- Contended resource: row, parent row, counter row, index prefix, range predicate, gap, queue head, table, partition, metadata lock, migration lock, advisory lock key, or connection pool that concurrent work can wait on.
+- Workload shape: write rate, same-tenant or same-resource concentration, queue consumer count, batch size, worker parallelism, retry behavior, traffic spikes, table size, and whether old transactions or long reports can overlap.
+- Database engine and version when known: PostgreSQL, MySQL/InnoDB, SQL Server, SQLite, managed database, ORM behavior, isolation level, lock wait timeout, statement timeout, transaction timeout, and pool limit.
+- Locking path: exact SQL or ORM calls, selected indexes, predicates, `FOR UPDATE` variant, isolation level, update order, lock order, transaction start and end, and external calls or calculations inside the transaction.
+- Data-shape alternatives: sharded counters, append-only ledgers, reservation tables, materialized summaries, per-tenant or per-shard queue lanes, partitioning, CQRS/read models, stage-specific tables, and conditional updates.
+- Failure and retry evidence: deadlock classification, serialization failures, lock timeout handling, optimistic-lock row count checks, full-command retry budget, idempotency keys, poison or starvation handling, and manual recovery.
+- Observability evidence: lock-wait logs, blocked-by and blocking query context, transaction age, pool wait metrics, deadlock logs, queue lag by shard, batch progress, migration lock waits, and configured command intents.
+
+<!-- mustflow-section: preconditions -->
+## Preconditions
+
+- The task matches the Use When conditions and does not match the Do Not Use When exclusions.
+- Higher-priority instructions and `.mustflow/config/commands.toml` have been checked for the current scope.
+- Required inputs are available, or missing engine, isolation, index, workload, lock-order, timeout, or observability evidence can be reported without guessing.
+- Existing local patterns for ledgers, reservations, counters, queue claiming, conditional updates, optimistic locks, outbox, retry classification, lock timeouts, and lock diagnostics have been searched before adding new shapes.
+- If the contention path affects money, credits, inventory, permissions, tenant data, or external side effects, also apply the relevant payment, credit-ledger, business-rule, security, transaction, idempotency, queue, or observability skill.
+
+<!-- mustflow-section: allowed-edits -->
+## Allowed Edits
+
+- Replace shared hot-row updates with append-only events, sharded counters, reservations, snapshots, materialized summaries, or stage-specific rows when local ownership and correctness allow it.
+- Replace select-then-update with conditional updates, atomic upserts, optimistic version checks, affected-row checks, or database constraints when they preserve the invariant and reduce waiting.
+- Narrow transaction scope by moving calculations, external API calls, object storage, email, webhooks, sleeps, slow serialization, and user-controlled work outside the lock-holding window.
+- Choose weaker database locks, stable lock order, `NOWAIT`, `SKIP LOCKED`, chunked batch writes, partitioned work, queue shard keys, and lock timeout policy only when the semantic tradeoff is explicit.
+- Add focused tests, fixtures, docs, or telemetry assertions for lock-timeout handling, deadlock retry, queue claiming, conditional-update failure, optimistic-lock conflicts, and synchronized template surfaces when local patterns support them.
+- Do not add speculative indexes, global serializable isolation, process-local mutexes, distributed locks, lock hints, or retries as magic fixes without naming the contended resource, correctness invariant, lock order, timeout behavior, and evidence gap.
+
+<!-- mustflow-section: procedure -->
+## Procedure
+
+1. Name the contended database resource.
+   - Identify the row, range, index prefix, parent row, counter, queue head, table, partition, metadata object, advisory lock key, or connection pool that concurrent executions can wait on.
+   - Treat `users.balance`, `products.stock`, `counters.value`, `posts.like_count`, queue `status = 'PENDING'`, and migration metadata as contention candidates even when each single statement looks cheap.
+2. Map the write concentration.
+   - Separate evenly distributed writes from same-account, same-product, same-tenant, same-campaign, same-post, same-day, or same-queue-head writes.
+   - If many requests must update one row to preserve a visible aggregate, ask whether the aggregate can be appended, sharded, reserved, summarized, or made eventually consistent.
+3. Prefer append or reservation models for durable histories.
+   - Money, credits, stock, coupons, usage, entitlement, and audit-sensitive paths usually need a ledger, event, reservation, capture, or release record before a mutable current-value row.
+   - Keep snapshot or cache updates separate from the durable fact when exact current display is less valuable than write throughput and auditability.
+4. Shorten the lock window.
+   - Finish price calculation, discount evaluation, permission checks, payload validation, and derived data construction before acquiring the lock when the decision can be revalidated cheaply.
+   - Keep only the final conditional write, version check, claim, or state transition inside the shortest transaction that preserves the invariant.
+   - Move HTTP calls, payment provider calls, email, webhooks, S3/object storage, search indexing, and queue publishing to after-commit or outbox paths.
+5. Challenge `SELECT ... FOR UPDATE`.
+   - Do not treat `FOR UPDATE` as a general safety blanket. Check whether the selected row is the real invariant owner and whether a weaker lock, conditional update, unique constraint, optimistic version, or parent guard is sufficient.
+   - In PostgreSQL, distinguish `FOR UPDATE`, `FOR NO KEY UPDATE`, `FOR SHARE`, and `FOR KEY SHARE` when key changes and foreign-key protection matter.
+   - Remember that row locks block competing writes and locks, not ordinary MVCC reads.
+6. Review MySQL/InnoDB index and range locking.
+   - Treat row locks as index-record locks. If the predicate lacks the right index, the engine may scan and lock more records than the business object implies.
+   - Check gap and next-key locks for range predicates under InnoDB defaults. A range can block inserts into values that do not yet exist.
+   - Consider `READ COMMITTED` only after deciding whether phantom rows are acceptable for the business rule.
+7. Match indexes to lock footprint, not only speed.
+   - For updates and deletes, ask how many index records the engine must inspect before finding the target set.
+   - Add or reshape indexes only when the predicate, write cost, uniqueness, migration lock behavior, and workload justify reducing the lock footprint.
+8. Make uniqueness and conditional writes do the guarding.
+   - Replace `SELECT missing -> INSERT` with unique constraints and insert-or-conflict handling.
+   - Replace stock, capacity, status, lease, and version updates with single conditional updates whose affected-row count is checked.
+   - For long user think-time, use optimistic version checks instead of holding database locks across the think time.
+9. Design queue tables around claiming, not polling.
+   - Use `SKIP LOCKED` or equivalent only for queue claiming where incomplete views are expected.
+   - Avoid one global `PENDING` lane that all workers scrape from the same index head. Add shard keys, `available_at`, priority, attempt count, `locked_at`, and deterministic ordering when the queue table is the contention point.
+   - Check starvation and fairness when high-priority or old locked rows can hide lower-priority work.
+10. Fix deadlock shape before blaming the engine.
+    - Establish a global order for touching accounts, products, rows, tables, partitions, or advisory lock keys.
+    - Do not trust `WHERE id IN (...)` or unordered ORM collections to acquire locks in a stable order when many rows are updated.
+    - Treat deadlock and serialization failures as normal retry branches only when the whole command is idempotent and retryable.
+11. Chunk large writes and deletes.
+    - Large `UPDATE` and `DELETE` operations can hold many locks, inflate undo or WAL, lag replicas, block vacuum or purge, and escalate locks on engines that support escalation.
+    - Prefer bounded key-range batches, checkpoints, pause or resume controls, and validation counts instead of one heroic statement on production-sized data.
+12. Use partitioning and table splits as contention boundaries.
+    - Partition by date, tenant, region, shard, or lifecycle stage only when it separates write conflicts and still fits the query model.
+    - Split state-machine rows into payments, fulfillments, shipments, settlements, or similar stage tables when independent workflows should not rewrite the same row.
+13. Review foreign-key and parent-row side effects.
+    - Parent deletes, key updates, child inserts, and child updates can create hidden locks around referential integrity.
+    - High-traffic child event tables that all reference a hot parent row may need a different ownership model, deferred aggregate, or explicit parent-row contention budget.
+14. Treat DDL and metadata locks as production traffic.
+    - Online index creation, concurrent index creation, constraint validation, metadata commits, and table rewrites can still wait on old transactions or briefly block application queries.
+    - Kill or drain `idle in transaction` and long-running transactions before migration windows when the repository owns that operational procedure; treat idle-in-transaction blockers as lock-contention inputs, not post-incident trivia.
+    - Prefer short lock timeouts for online DDL attempts that should back off rather than queue behind production traffic.
+15. Set lock timeout policy by operation.
+    - Interactive requests, checkout, admin jobs, backfills, migrations, and reconciliation should not share one infinite wait policy.
+    - Distinguish lock timeout, statement timeout, transaction timeout, pool wait timeout, and provider timeout in logs and retry handling.
+16. Separate lock waits from connection-pool waits.
+    - A request can look like a database lock incident while it is actually stuck waiting for a connection held by wide transactions, `REQUIRES_NEW`, per-item transactions, or slow external calls inside transactions.
+    - Review pool size, transaction width, nested transaction behavior, and per-request database checkout count before changing SQL locks.
+17. Add observability that names both waiter and blocker.
+    - Logs should identify operation, resource key, transaction age, lock wait duration, timeout class, retry attempt, idempotency key, queue shard, batch cursor, and blocking query or session when available.
+    - Metrics should avoid unbounded labels but preserve table or operation class, lock-wait bucket, pool-wait bucket, deadlock count, timeout count, and queue lag by bounded shard.
+    - For PostgreSQL, MySQL, SQL Server, or managed database diagnostics, use repository-approved observability paths and report manual-only diagnostics when no configured command exists.
+18. Label evidence honestly.
+    - Static review can identify likely lock contention but cannot prove p95 lock wait, deadlock rate, gap-lock behavior, or migration lock safety under production data.
+    - Treat engine docs, plan output, deadlock logs, `pg_locks`, InnoDB status, SQL Server DMVs, Query Store, APM traces, and production telemetry as stronger evidence only when they match the repository engine, version, schema, and workload.
+
+<!-- mustflow-section: postconditions -->
+## Postconditions
+
+- The contended resource, workload concentration, lock path, lock order, transaction width, timeout policy, retry behavior, queue claim model, DDL or metadata lock exposure, and observability evidence are explicit.
+- Hot rows, mutable counter caches, select-then-update races, over-strong locks, missing lock-footprint indexes, range or gap-lock surprises, unordered multi-row updates, large unchunked writes, queue head contention, hidden FK or parent-row locks, and idle-transaction DDL blockers are fixed or reported.
+- Correctness, auditability, idempotency, ordering, tenant isolation, and user-visible truth remain intact or are reported as tradeoffs.
+- Lock-safety claims are backed by configured tests, engine evidence, observability evidence, or labeled as static review risk.
+
+<!-- mustflow-section: verification -->
+## Verification
+
+Use configured oneshot command intents when available:
+
+- `changes_status`
+- `changes_diff_summary`
+- `lint`
+- `build`
+- `test_related`
+- `test`
+- `docs_validate_fast`
+- `test_release`
+- `mustflow_check`
+
+Use the narrowest configured test, build, docs, release, or mustflow intent that covers the changed lock-sensitive path and synchronized template surfaces. Do not infer raw database shells, live lock dashboards, production lock queries, migration execution, load tests, stress tests, server processes, watcher processes, or package-manager commands outside the command contract.
+
+<!-- mustflow-section: failure-handling -->
+## Failure Handling
+
+- If a configured command fails, preserve the failing intent, failing assertion or output tail, and the lock-sensitive invariant it exercised before editing again.
+- If the engine, isolation level, index shape, workload, table size, lock timeout, or observability surface is unknown, report static lock-contention risk instead of claiming the path is safe.
+- If a safe fix requires schema changes, online DDL, partitioning, queue redesign, outbox workers, materialized summaries, load tests, production diagnostics, or provider-console changes outside the current scope, report the missing boundary.
+- If no configured command can prove concurrent lock behavior, report the missing manual evidence and complete the configured checks that are available.
+
+<!-- mustflow-section: output-format -->
+## Output Format
+
+- Database lock-contention surface reviewed
+- Contended resource, workload concentration, lock path, engine and isolation evidence
+- Hot-row, event or reservation model, conditional write, lock strength, index lock footprint, gap or next-key lock, queue claim, lock order, batch chunking, partition, FK or parent-row, DDL or metadata lock, timeout, pool wait, retry, and observability findings
+- Lock-contention fix made or recommended
+- Evidence level: configured-test evidence, engine evidence, production telemetry, static review risk, manual-only, missing, or not applicable
+- Command intents run
+- Skipped lock diagnostics and reasons
+- Remaining database lock-contention risk

package/templates/default/locales/en/.mustflow/skills/database-migration-change/SKILL.md

@@ -2,11 +2,11 @@
 mustflow_doc: skill.database-migration-change
 locale: en
 canonical: true
-revision: 1
+revision: 2
 lifecycle: mustflow-owned
 authority: procedure
 name: database-migration-change
-description: Apply this skill when database migration files, schema migration history, ORM schema migrations, generated clients, schema dumps, SQL snapshots, backfills, rolling deploy compatibility, expand-and-contract changes, destructive database changes, migration rollback claims, or production database migration procedures are created, changed, reviewed, or reported.
+description: Apply this skill when database migration files, schema migration history, ORM schema migrations, generated clients, schema dumps, SQL snapshots, online DDL, large indexes, constraints, backfills, rolling deploy compatibility, expand-and-contract changes, destructive database changes, migration rollback or roll-forward claims, cut-over plans, lock or timeout policy, replication lag risk, migration observability, or production database migration procedures are created, changed, reviewed, or reported.
 metadata:
   mustflow_schema: "1"
   mustflow_kind: procedure
@@ -29,16 +29,17 @@ metadata:
 <!-- mustflow-section: purpose -->
 ## Purpose
 
-Keep database migrations safe for running systems by checking deploy compatibility, data preservation, backfill behavior, generated artifacts, ORM contracts, and rollback reality.
+Keep database migrations safe for running systems by checking deploy compatibility, data preservation, lock behavior, online DDL limits, backfill behavior, generated artifacts, ORM contracts, observability, and recovery reality.
 
 Do not treat migration authoring as "make a file that applies locally." Treat it as "old code and new code must survive the same database during rollout."
+Migration incidents usually happen in the interval where old code, new code, old data, and new data are all alive at once. Design that interval first.
 
 <!-- mustflow-section: use-when -->
 ## Use When
 
 - A database migration file, migration history entry, schema dump, ORM schema, SQL snapshot, generated client, seed, fixture, schema validator, or migration documentation is created or changed.
 - A change adds, removes, renames, splits, merges, backfills, rewrites, validates, constrains, indexes, foreign-keys, type-changes, defaults, nullable rules, enum values, tables, columns, generated columns, triggers, views, functions, row-level policies, or data migrations.
-- A task mentions rolling deploy, expand-and-contract, online migration, backfill, production schema change, rollback, down migration, migration lock, DDL transaction, generated ORM client, migration drift, schema drift, or database migration safety.
+- A task mentions rolling deploy, expand-and-contract, online migration, backfill, production schema change, rollback, roll-forward, down migration, migration lock, lock timeout, statement timeout, DDL transaction, `CREATE INDEX CONCURRENTLY`, MySQL `ALGORITHM=INSTANT`, MySQL `LOCK=NONE`, generated ORM client, migration drift, schema drift, or database migration safety.
 - Prisma, Drizzle, TypeORM, Rails Active Record, Django migrations, Alembic, Diesel, Ecto, Flyway, Liquibase, Knex, Sequelize, SQLx, or another migration tool changes schema, generated output, migration metadata, or deployment behavior.
 - A final report claims a database migration is safe, reversible, applied, validated, production-ready, no-downtime, rollback-safe, or tested from an old schema.
 
@@ -55,8 +56,8 @@ Do not treat migration authoring as "make a file that applies locally." Treat it
 - Source schema, target schema, migration files, migration history table or metadata, generated clients, schema dumps, SQL snapshots, seeds, fixtures, and affected queries.
 - Migration tool and ecosystem: Prisma, Drizzle, TypeORM, Rails, Django, Alembic, Diesel, Ecto, Flyway, Liquibase, Knex, Sequelize, SQLx, raw SQL, or another declared tool.
 - Deployment shape: single-step deploy, rolling deploy, blue-green, multiple app versions, background workers, read replicas, multiple services, serverless functions, mobile clients, or external integrations.
-- Database engine and operational surface: PostgreSQL, MySQL, SQLite, SQL Server, managed database, migration lock behavior, DDL transaction behavior, online DDL options, table size, expected lock time, statement timeout, lock timeout, and restore capability when known.
-- Data preservation needs, compatibility window, backfill size, batch strategy, restart marker, validation query, rollback type, and whether old code can run after the new schema lands.
+- Database engine and operational surface: PostgreSQL, MySQL, SQLite, SQL Server, managed database, migration lock behavior, DDL transaction behavior, online DDL options, table size, write load, long-running transactions, replication or CDC topology, expected lock time, statement timeout, lock timeout, and restore capability when known.
+- Data preservation needs, compatibility window, backfill size, batch strategy, cursor or checkpoint marker, validation query, observability query, rollback or roll-forward type, cut-over control, and whether old code can run after the new schema lands.
 - Relevant command-intent entries for build, generated-output checks, tests, docs, release, and mustflow validation.
 
 <!-- mustflow-section: preconditions -->
@@ -64,7 +65,7 @@ Do not treat migration authoring as "make a file that applies locally." Treat it
 
 - The task matches the Use When conditions and does not match the Do Not Use When exclusions.
 - Higher-priority instructions and `.mustflow/config/commands.toml` have been checked for the current scope.
-- The migration is classified before editing as expand, backfill, switch, contract, destructive, data-only, schema-only, generated-output-only, ORM metadata-only, or rollback documentation.
+- The migration is classified before editing as expand, backfill, switch, contract, destructive, data-only, schema-only, generated-output-only, ORM metadata-only, cut-over, or recovery documentation.
 - If personal data, tenant isolation, authorization, audit, retention, or secrets are involved, also use `security-privacy-review`.
 - If public API response shape changes because of the migration, also use `api-contract-change`.
 - If file paths or storage keys change with the database migration, also use `file-path-cross-platform-change`.
@@ -73,7 +74,7 @@ Do not treat migration authoring as "make a file that applies locally." Treat it
 ## Allowed Edits
 
 - Update migration files, ORM schema files, generated client expectations, schema dumps, SQL snapshots, seeds, fixtures, compatibility code, backfill code, validation checks, docs, and tests directly required by the migration.
-- Prefer expand-and-contract for live systems: add compatible shape, backfill safely, switch reads and writes, then contract only after compatibility is proven.
+- Prefer expand-and-contract for live systems: add compatible shape, dual-write or compatibility-read where needed, backfill safely, switch reads and writes, then contract only after compatibility is proven.
 - Keep destructive cleanup separate from expansion unless the repository explicitly proves a single-step deployment is safe.
 - Do not weaken tests, delete migration history, hand-edit generated client output, suppress migration drift, or claim rollback safety for lossy changes.
 
@@ -97,34 +98,70 @@ Do not treat migration authoring as "make a file that applies locally." Treat it
    - New code after backfill.
    - New code after contract.
    If any required state fails, the migration is not rolling-deploy safe.
-5. For column add, decide nullability, default behavior, backfill strategy, write path, read fallback, index need, and when a future `NOT NULL` or constraint can be enforced.
-6. For column drop, first prove all reads, writes, generated clients, old app versions, jobs, analytics, exports, seeds, fixtures, docs, and external consumers stopped using the column. Treat drop as a contract phase, not as the first migration.
-7. For rename, do not implement rename as drop plus add unless data loss is explicitly intended and approved. Prefer add new column, dual-write or copy, backfill, switch reads, validate, then drop old column later.
-8. For type changes, check whether the database rewrites the table, whether old values can fail conversion, whether indexes or constraints rebuild, whether application serializers change, and whether API or generated client types change.
-9. For nullable and default changes, distinguish schema default from application default. Avoid assuming existing rows are backfilled by a new default.
-10. For constraints and foreign keys, use staged validation when the database supports it. Add the constraint without validating existing rows when appropriate, backfill or repair data, then validate in a separate phase.
-11. For indexes, check table size, write load, lock behavior, concurrent or online index support, uniqueness validation, existing duplicates, and whether generated ORM queries will use the index.
-12. For enum changes, check old code behavior, generated client unions, database enum migration limits, default values, serialization, API compatibility, and whether removing or renaming values requires a data migration first.
-13. For table split, table merge, or relationship rewrite, preserve stable identifiers, foreign keys, audit references, external IDs, permissions, search documents, exports, and old-to-new mapping until all callers switch.
-14. For backfills, make them bounded, restartable, observable, and validated. Define batch size, ordering key, checkpoint, retry behavior, idempotency, timeout, lock expectation, dead-letter or manual review behavior, and validation queries.
-15. Do not run or recommend full-table updates on production-sized data without measured volume, lock expectation, batch plan, timeout policy, and recovery plan.
-16. Keep external side effects out of database migrations unless the repository has an explicit recovery model. Sending emails, calling payment APIs, deleting files, or mutating external providers from a migration usually breaks rollback.
-17. Check generated surfaces after schema changes: ORM clients, types, SQL snapshots, schema dumps, OpenAPI or GraphQL projections, API mocks, fixtures, seeds, admin screens, analytics, ETL, BI queries, and docs examples.
-18. Review ORM-specific traps.
+5. Split the deployment plan into expand, backfill, switch, and contract phases.
+   - Expansion adds shapes old code can ignore and new code can start writing.
+   - Backfill is bounded, restartable, idempotent, observable, and separately validated.
+   - Switch changes read paths through a feature flag, rollout gate, tenant gate, or compatible deploy step where possible.
+   - Contract removes old shapes only after at least one compatibility window proves no code, job, report, or manual SQL still depends on them.
+6. For column add, decide nullability, default behavior, backfill strategy, write path, read fallback, index need, and when a future `NOT NULL` or constraint can be enforced.
+   - Add nullable first unless a proven engine/version/table-size path makes the non-null default safe.
+   - Do not assume a database default backfills existing rows or matches ORM, API, batch, or application defaults.
+   - Enforce `NOT NULL` only after new writes populate the column, old rows are backfilled, and missing-value validation has passed.
+7. For column drop, first prove all reads, writes, generated clients, old app versions, jobs, analytics, exports, seeds, fixtures, docs, BI queries, support SQL, and external consumers stopped using the column. Treat `DROP COLUMN` as a contract phase, not as the first migration.
+8. For rename, do not implement rename as drop plus add unless data loss is explicitly intended and approved. Prefer add new column, dual-write or copy, backfill, switch reads, validate, then drop old column later.
+9. For type changes, check whether the database rewrites the table, whether old values can fail conversion, whether indexes or constraints rebuild, whether application serializers change, and whether API or generated client types change.
+10. For nullable and default changes, distinguish schema default from application default. Change application writes first when defaults must stay compatible across old and new versions.
+11. For constraints and foreign keys, use staged validation when the database supports it.
+    - PostgreSQL `NOT VALID` constraints and later `VALIDATE CONSTRAINT` can avoid validating old rows during the first lock-sensitive change while still protecting new writes.
+    - Foreign keys on large or hot tables need separate add, repair/backfill, validate, and fallback planning rather than one heroic migration.
+12. For indexes, check table size, write load, lock behavior, concurrent or online index support, uniqueness validation, existing duplicates, and whether generated ORM queries will use the index.
+    - PostgreSQL `CREATE INDEX CONCURRENTLY` avoids blocking ordinary writes but still scans more than once, waits for old transactions, consumes I/O, and cannot run inside a transaction block.
+    - MySQL online DDL and `ALGORITHM=INSTANT` have version, row-format, index, row-size, and operation limits; specify `ALGORITHM=INSTANT` or `LOCK=NONE` only when unsupported fallback should fail instead of silently rebuilding a table.
+    - Treat index deletion as a separate risk: usage counters can lie after restarts, stats resets, monthly jobs, or emergency queries.
+13. Review lock and timeout policy.
+    - PostgreSQL `ALTER TABLE` often takes strong locks unless a subcommand documents a lower level.
+    - Use short `lock_timeout` and an appropriate `statement_timeout` policy when the repository owns that migration surface.
+    - Do not wrap several DDL statements in one transaction just because the file looks cleaner; locks held until commit can block unrelated traffic.
+    - Very short metadata-only changes on the same table may be bundled only after engine/version/table-state evidence shows each subcommand stays metadata-only.
+14. For enum, partition, and table-ownership changes, check engine-specific limits.
+    - Enum value additions, removals, or renames can break old code, generated unions, rollback, defaults, and storage layout.
+    - Partition attach can scan existing rows unless a suitable `CHECK` constraint proves the range first.
+    - Table split, table merge, or relationship rewrite must preserve stable identifiers, foreign keys, audit references, external IDs, permissions, search documents, exports, and old-to-new mapping until all callers switch.
+15. For backfills, make them bounded, restartable, observable, and validated. Define batch size, cursor-based ordering key such as `id > last_id`, checkpoint, retry behavior, idempotency, timeout, lock expectation, throttle or pause/resume control, dead-letter or manual review behavior, and validation queries.
+16. Do not run or recommend full-table updates on production-sized data without measured volume, lock expectation, WAL or undo impact, replication lag risk, batch plan, timeout policy, and recovery plan.
+17. Review replication, CDC, and long-running transaction interactions.
+    - Online DDL can leave replicas, read traffic, backups, CDC connectors, or failover readiness behind even when the primary looks healthy.
+    - PostgreSQL concurrent index builds can wait on old transactions, idle-in-transaction sessions, backups, or long reports.
+    - MySQL online DDL can still need short exclusive locks at the beginning or final phase; plan cut-over timing rather than trusting ETA.
+18. Review cut-over control for ghost-table, copy-and-swap, or dual-write migrations.
+    - The dangerous moment is usually the final swap, name change, or read-path flip, not the long copy.
+    - Preserve a postpone, pause, resume, throttle, or manual approval point when the repository's migration tool supports it.
+    - Monitor dual-write mismatch and sample old/new values during the compatibility window; code intent is not proof that every path writes both sides.
+19. Prepare observability before apply.
+    - Pair the migration with read-only progress and safety queries for lock waits, index build progress, replication lag, backfill cursor, skipped rows, failed rows, duplicate rows, missing rows, dead tuples, or estimated remaining range when the engine supports them.
+    - Log or report dry-run selection counts, apply counts, skip reasons, batch durations, and recovery handles.
+    - A final `done` line is not enough evidence for a live migration.
+20. Decide rollback honestly and prefer roll-forward for partial live changes.
+    - Reversible: schema-only and data-preserving.
+    - App rollback: old and new code both tolerate the expanded shape, so the read path can move back without losing new writes.
+    - Forward-fix preferred: partial live migration can be corrected without restoring.
+    - Restore required: deletes, table merges, generated IDs, hashing, encryption, irreversible type conversions, external side effects, or lossy transforms.
+    Do not promise rollback for changes that cannot reconstruct old values.
+21. Keep external side effects out of database migrations unless the repository has an explicit recovery model. Sending emails, calling payment APIs, deleting files, or mutating external providers from a migration usually breaks rollback.
+22. Check generated surfaces after schema changes: ORM clients, types, SQL snapshots, schema dumps, OpenAPI or GraphQL projections, API mocks, fixtures, seeds, admin screens, analytics, ETL, BI queries, and docs examples.
+23. Review ORM-specific traps.
    - Prisma generated client and migration history must match the migration files that production applies.
    - Drizzle schema exports and generated SQL or snapshots must not drift.
    - TypeORM production `synchronize` must not be used as a migration strategy.
    - Django rename detection and autogeneration must be reviewed so rename does not become delete and create.
    - Alembic autogenerate output is a candidate, not a reviewed migration.
    - Rails migrations should not depend on current model callbacks or validations when historical data can outlive current model code.
-19. Decide rollback honestly.
-   - Reversible: schema-only and data-preserving.
-   - Forward-fix preferred: partial live migration can be corrected without restoring.
-   - Restore required: deletes, table merges, generated IDs, hashing, encryption, irreversible type conversions, external side effects, or lossy transforms.
-   Do not promise rollback for changes that cannot reconstruct old values.
-20. Rehearse from the previous production-like schema when possible. A migration that only applies to a freshly generated schema is not enough.
-21. Check dependent surfaces: application code, background jobs, cron tasks, workers, ETL, reports, admin tools, data exports, imports, mobile or older clients, webhooks, API clients, generated SDKs, test fixtures, seeds, docs, and monitoring.
-22. Select verification from the command contract. Major schema changes, generated client changes, rollback-sensitive changes, security-sensitive migrations, and production deployment changes need broader verification than local schema-only edits.
+24. Add or check automated guardrails when the repository owns migration linting.
+   - Flag raw `CREATE INDEX` on large PostgreSQL tables when `CONCURRENTLY` is expected.
+   - Flag first-phase `DROP COLUMN`, large-table `SET NOT NULL`, unvalidated foreign keys, transaction-wrapped concurrent index creation, unbounded `UPDATE`, offset-based backfills, and unchecked destructive operations.
+25. Rehearse from the previous production-like schema when possible. A migration that only applies to a freshly generated schema is not enough.
+26. Check dependent surfaces: application code, background jobs, cron tasks, workers, ETL, reports, admin tools, data exports, imports, mobile or older clients, webhooks, API clients, generated SDKs, test fixtures, seeds, docs, and monitoring.
+27. Select verification from the command contract. Major schema changes, generated client changes, rollback-sensitive changes, security-sensitive migrations, and production deployment changes need broader verification than local schema-only edits.
 
 <!-- mustflow-section: postconditions -->
 ## Postconditions
@@ -132,9 +169,10 @@ Do not treat migration authoring as "make a file that applies locally." Treat it
 - Source schema, target schema, migration files, generated artifacts, schema dumps, seeds, fixtures, and dependent code agree.
 - Expand, backfill, switch, and contract phases are separated or explicitly proven unnecessary.
 - Old-code/new-schema and new-code/expanded-schema compatibility is classified.
-- Backfill and validation behavior is bounded, restartable, and checkable where relevant.
-- Rollback claims distinguish schema rollback, data rollback, app rollback, forward-fix, and restore-required cases.
-- Destructive changes and production lock risks are either deferred, measured, or reported as remaining risk.
+- Backfill and validation behavior is cursor-based or otherwise bounded, restartable, idempotent, observable, and checkable where relevant.
+- Lock levels, online DDL support, long-running transaction waits, replication lag, cut-over control, timeout policy, and observability queries are explicit where production data may be affected.
+- Rollback claims distinguish schema rollback, data rollback, app rollback, roll-forward, forward-fix, and restore-required cases.
+- Destructive changes and production lock risks are either deferred, measured, guarded, or reported as remaining risk.
 
 <!-- mustflow-section: verification -->
 ## Verification
@@ -158,8 +196,10 @@ Prefer configured migration dry-run, generated-output, schema-diff, or database
 
 - If old/new application compatibility is unknown, do not call the migration rolling-deploy safe.
 - If production table size, lock behavior, or validation cost is unknown, report the operation as production-risky.
+- If online DDL support, long-running transaction behavior, replication lag, or cut-over control is unknown, report the migration as operationally unproven.
 - If an autogenerator proposes drop/create for a rename, stop and rewrite the migration plan.
 - If a migration is lossy, do not claim rollback beyond restore or forward corrective migration.
+- If a backfill is not idempotent, restartable, observable, and throttled or bounded, keep it out of a production migration claim.
 - If generated clients or schema dumps drift, fix the source of truth and regenerated surfaces together.
 - If configured verification is missing, report the missing command intent instead of inferring package-manager, ORM, or migration-tool commands.
 
@@ -170,7 +210,9 @@ Prefer configured migration dry-run, generated-output, schema-diff, or database
 - Tool and database engine surface inspected
 - Source schema, target schema, and migration phase
 - Old-code/new-schema and new-code/expanded-schema compatibility
-- Backfill, validation, lock, timeout, and rollback classification
+- Expand/backfill/switch/contract plan and destructive cleanup timing
+- Backfill cursor, idempotency, throttle, pause/resume, validation, lock, timeout, replication, cut-over, and observability classification
+- Rollback, app rollback, roll-forward, forward-fix, and restore-required classification
 - ORM/generated client/schema dump/snapshot surfaces synchronized
 - Dependent surfaces checked
 - Command intents run