strong_migrations 0.3.1 → 0.7.6

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: bbb45da961375e9a3499c6b651da754c164f05b0e33f9b5061a856ce508663b6
-  data.tar.gz: 1718eb87c7eb8b60553b1e800e8a4866cdbc134ae9c062e012caa6e8f3d4be6b
+  metadata.gz: f16481df30c6b961477b4a537034097aa1d41a262bd7f2741c40b7e6dae9cb9e
+  data.tar.gz: 816bc9a4f9810bce6ac6d5e0a52b185800515794ca5e3d27b5f4d512a9190115
 SHA512:
-  metadata.gz: 63dde3000e42dbe433914159a815a1dc4a1ebca89e78f3f342e9aa664bd1e921bbb3a1ee33140d6e56d50aef400b171e27a24509fd369da0350b0f548a108a25
-  data.tar.gz: d187c338c085b86ad7498ea09253a9d2015415740b4c09d78ec1f9f37abbc20261255b1590210d7bbf473fcb2ecfa1bad57a9e356a9e2480ec2f90d9fb355404
+  metadata.gz: 2ac9f7c38958c1cc2e80efa3f895730087ac92a3675be31c308f9f8df827ced3323ca8bc3e69c99e2a79b92ef0a5a8b595a76715bf55c45b5698fcdd43b5a586
+  data.tar.gz: 2cee8d3013ae8780f0bcbce33f22a219375c29f059335902cdfe2285ec557307229b7ec4544e6be59ea1ee7cd039f2b23e9b3693efc41f11f677cbbed1495896

data/CHANGELOG.md

@@ -1,78 +1,186 @@
-## 0.3.1
+## 0.7.6 (2021-01-17)
+
+- Fixed `NOT NULL` constraint check for quoted columns
+- Fixed deprecation warning with Active Record 6.1
+
+## 0.7.5 (2021-01-12)
+
+- Added checks for `add_check_constraint` and `validate_check_constraint`
+
+## 0.7.4 (2020-12-16)
+
+- Added `safe_by_default` option to install generator
+- Fixed warnings with Active Record 6.1
+
+## 0.7.3 (2020-11-24)
+
+- Added `safe_by_default` option
+
+## 0.7.2 (2020-10-25)
+
+- Added support for float timeouts
+
+## 0.7.1 (2020-07-27)
+
+- Added `target_version` option to replace database-specific options
+
+## 0.7.0 (2020-07-22)
+
+- Added `check_down` option
+- Added check for `change_column` with `null: false`
+- Added check for `validate_foreign_key`
+- Improved error messages
+- Made auto analyze less verbose in Postgres
+- Decreasing the length limit of a `varchar` column or adding a limit is not safe in Postgres
+- Removed safety checks for `db` rake tasks (Rails 5+ handles this)
+
+## 0.6.8 (2020-05-13)
+
+- `change_column_null` on a column with a `NOT NULL` constraint is safe in Postgres 12+
+
+## 0.6.7 (2020-05-13)
+
+- Improved comments in initializer
+- Fixed string timeouts for Postgres
+
+## 0.6.6 (2020-05-08)
+
+- Added warnings for missing and long lock timeouts
+- Added install generator
+
+## 0.6.5 (2020-05-06)
+
+- Fixed deprecation warnings with Ruby 2.7
+
+## 0.6.4 (2020-04-16)
+
+- Added check for `add_reference` with `foreign_key: true`
+
+## 0.6.3 (2020-04-04)
+
+- Increasing precision of `decimal` or `numeric` column is safe in Postgres 9.2+
+- Making `decimal` or `numeric` column unconstrained is safe in Postgres 9.2+
+- Changing between `timestamp` and `timestamptz` when session time zone is UTC in Postgres 12+
+- Increasing the length of a `varchar` column from under 255 up to 255 in MySQL and MariaDB
+- Increasing the length of a `varchar` column over 255 in MySQL and MariaDB
+
+## 0.6.2 (2020-02-03)
+
+- Fixed PostgreSQL version check
+
+## 0.6.1 (2020-01-28)
+
+- Fixed timeouts for PostgreSQL
+
+## 0.6.0 (2020-01-24)
+
+- Added `statement_timeout` and `lock_timeout`
+- Adding a column with a non-null default value is safe in MySQL 8.0.12+ and MariaDB 10.3.2+
+- Added `change_column_null` check for MySQL and MariaDB
+- Added `auto_analyze` for MySQL and MariaDB
+- Added `target_mysql_version` and `target_mariadb_version`
+- Switched to `up` for backfilling
+
+## 0.5.1 (2019-12-17)
+
+- Fixed migration name in error messages
+
+## 0.5.0 (2019-12-05)
+
+- Added ability to disable checks
+- Added Postgres-specific check for `change_column_null`
+- Added optional remove index check
+
+## 0.4.2 (2019-10-27)
+
+- Allow `add_reference` with concurrent indexes
+
+## 0.4.1 (2019-07-12)
+
+- Added `target_postgresql_version`
+- Added `unscoped` to backfill instructions
+
+## 0.4.0 (2019-05-27)
+
+- Added check for `add_foreign_key`
+- Fixed instructions for adding default value with NOT NULL constraint
+- Removed support for Rails 4.2
+
+## 0.3.1 (2018-10-18)
 
 - Fixed error with `remove_column` and `type` argument
 - Improved message customization
 
-## 0.3.0
+## 0.3.0 (2018-10-15)
 
 - Added support for custom checks
 - Adding a column with a non-null default value is safe in Postgres 11+
 - Added checks for `add_belongs_to`, `remove_belongs_to`, `remove_columns`, and `remove_reference`
 - Customized messages
 
-## 0.2.3
+## 0.2.3 (2018-07-22)
 
 - Added check for `change_column_null`
 - Added support for alphabetize columns with Makara
 - Fixed migration reversibility with `auto_analyze`
 
-## 0.2.2
+## 0.2.2 (2018-02-14)
 
 - Friendlier output
 - Better method of hooking into ActiveRecord
 
-## 0.2.1
+## 0.2.1 (2018-02-07)
 
 - Recommend `disable_ddl_transaction!` over `commit_db_transaction`
 - Suggest `jsonb` over `json` in Postgres 9.4+
 - Changing `varchar` to `text` is safe in Postgres 9.1+
 - Do not check number of columns for unique indexes
 
-## 0.2.0
+## 0.2.0 (2018-01-07)
 
 - Added customizable error messages
 - Updated instructions for adding a column with a default value
 
-## 0.1.9
+## 0.1.9 (2017-06-14)
 
 - Added `start_after` option
 
-## 0.1.8
+## 0.1.8 (2017-05-31)
 
 - Fixed error with `create_table`
 - Added check for executing arbitrary SQL
 
-## 0.1.7
+## 0.1.7 (2017-05-29)
 
 - Added check for `force` option with `create_table`
 - Added `auto_analyze` option
 
-## 0.1.6
+## 0.1.6 (2017-03-23)
 
 - Adding an index to a newly created table is now safe
 
-## 0.1.5
+## 0.1.5 (2016-07-23)
 
 - Fixed error with Ruby 2.3 frozen strings
 
-## 0.1.4
+## 0.1.4 (2016-03-22)
 
 - Added alphabetize columns
 
-## 0.1.3
+## 0.1.3 (2016-03-12)
 
 - Disabled dangerous rake tasks in production
 - Added ability to use `SAFETY_ASSURED` env var
 
-## 0.1.2
+## 0.1.2 (2016-02-24)
 
 - Skip checks on down migrations and rollbacks
 - Added check for indexes with more than 3 columns
 
-## 0.1.1
+## 0.1.1 (2015-11-29)
 
 - Fixed `add_index` check for MySQL
 
-## 0.1.0
+## 0.1.0 (2015-11-22)
 
 - First release

data/LICENSE.txt

@@ -1,4 +1,4 @@
-Copyright (c) 2013 Bob Remeika and David Waller, 2015-2018 Andrew Kane
+Copyright (c) 2013 Bob Remeika and David Waller, 2015-2021 Andrew Kane
 
 MIT License
 

data/README.md

@@ -1,10 +1,14 @@
 # Strong Migrations
 
-Catch unsafe migrations at dev time
+Catch unsafe migrations in development
+
+&nbsp;&nbsp;✓&nbsp;&nbsp;Detects potentially dangerous operations<br />&nbsp;&nbsp;✓&nbsp;&nbsp;Prevents them from running by default<br />&nbsp;&nbsp;✓&nbsp;&nbsp;Provides instructions on safer ways to do what you want
+
+Supports for PostgreSQL, MySQL, and MariaDB
 
 :tangerine: Battle-tested at [Instacart](https://www.instacart.com/opensource)
 
-[![Build Status](https://travis-ci.org/ankane/strong_migrations.svg?branch=master)](https://travis-ci.org/ankane/strong_migrations)
+[![Build Status](https://github.com/ankane/strong_migrations/workflows/build/badge.svg?branch=master)](https://github.com/ankane/strong_migrations/actions)
 
 ## Installation
 
@@ -14,57 +18,130 @@ Add this line to your application’s Gemfile:
 gem 'strong_migrations'
 ```
 
+And run:
+
+```sh
+bundle install
+rails generate strong_migrations:install
+```
+
+Strong Migrations sets a long statement timeout for migrations so you can set a [short statement timeout](#app-timeouts) for your application.
+
 ## How It Works
 
-Strong Migrations detects potentially dangerous operations in migrations, prevents them from running by default, and provides instructions on safer ways to do what you want. Here’s an example:
+When you run a migration that’s potentially dangerous, you’ll see an error message like:
 
-```
+```txt
 === Dangerous operation detected #strong_migrations ===
 
-ActiveRecord caches attributes which causes problems
+Active Record caches attributes, which causes problems
 when removing columns. Be sure to ignore the column:
 
 class User < ApplicationRecord
-  self.ignored_columns = ["some_column"]
+  self.ignored_columns = ["name"]
 end
 
 Deploy the code, then wrap this step in a safety_assured { ... } block.
 
-class RemoveColumn < ActiveRecord::Migration[5.2]
+class RemoveColumn < ActiveRecord::Migration[6.1]
   def change
-    safety_assured { remove_column :users, :some_column }
+    safety_assured { remove_column :users, :name }
+  end
+end
+```
+
+An operation is classified as dangerous if it either:
+
+- Blocks reads or writes for more than a few seconds (after a lock is acquired)
+- Has a good chance of causing application errors
+
+## Checks
+
+Potentially dangerous operations:
+
+- [removing a column](#removing-a-column)
+- [adding a column with a default value](#adding-a-column-with-a-default-value)
+- [backfilling data](#backfilling-data)
+- [changing the type of a column](#changing-the-type-of-a-column)
+- [renaming a column](#renaming-a-column)
+- [renaming a table](#renaming-a-table)
+- [creating a table with the force option](#creating-a-table-with-the-force-option)
+- [adding a check constraint](#adding-a-check-constraint)
+- [setting NOT NULL on an existing column](#setting-not-null-on-an-existing-column)
+- [executing SQL directly](#executing-SQL-directly)
+
+Postgres-specific checks:
+
+- [adding an index non-concurrently](#adding-an-index-non-concurrently)
+- [adding a reference](#adding-a-reference)
+- [adding a foreign key](#adding-a-foreign-key)
+- [adding a json column](#adding-a-json-column)
+
+Best practices:
+
+- [keeping non-unique indexes to three columns or less](#keeping-non-unique-indexes-to-three-columns-or-less)
+
+You can also add [custom checks](#custom-checks) or [disable specific checks](#disable-checks).
+
+### Removing a column
+
+#### Bad
+
+Active Record caches database columns at runtime, so if you drop a column, it can cause exceptions until your app reboots.
+
+```ruby
+class RemoveSomeColumnFromUsers < ActiveRecord::Migration[6.1]
+  def change
+    remove_column :users, :some_column
   end
 end
 ```
 
-## Dangerous Operations
+#### Good
 
-The following operations can cause downtime or errors:
+1. Tell Active Record to ignore the column from its cache
 
-- adding a column with a non-null default value to an existing table
-- removing a column
-- changing the type of a column
-- setting a `NOT NULL` constraint with a default value
-- renaming a column
-- renaming a table
-- creating a table with the `force` option
-- adding an index non-concurrently (Postgres only)
-- adding a `json` column to an existing table (Postgres only)
+  ```ruby
+  class User < ApplicationRecord
+    self.ignored_columns = ["some_column"]
+  end
+  ```
 
-Also checks for best practices:
+2. Deploy code
+3. Write a migration to remove the column (wrap in `safety_assured` block)
 
-- keeping non-unique indexes to three columns or less
+  ```ruby
+  class RemoveSomeColumnFromUsers < ActiveRecord::Migration[6.1]
+    def change
+      safety_assured { remove_column :users, :some_column }
+    end
+  end
+  ```
 
-## The Zero Downtime Way
+4. Deploy and run migration
 
 ### Adding a column with a default value
 
-Adding a column with a non-null default causes the entire table to be rewritten.
+#### Bad
+
+In earlier versions of Postgres, MySQL, and MariaDB, adding a column with a default value to an existing table causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
+
+```ruby
+class AddSomeColumnToUsers < ActiveRecord::Migration[6.1]
+  def change
+    add_column :users, :some_column, :text, default: "default_value"
+  end
+end
+```
+
+In Postgres 11+, MySQL 8.0.12+, and MariaDB 10.3.2+, this no longer requires a table rewrite and is safe.
+
+#### Good
 
 Instead, add the column without a default value, then change the default.
 
 ```ruby
-class AddSomeColumnToUsers < ActiveRecord::Migration[5.2]
+class AddSomeColumnToUsers < ActiveRecord::Migration[6.1]
   def up
     add_column :users, :some_column, :text
     change_column_default :users, :some_column, "default_value"
@@ -76,64 +153,71 @@ class AddSomeColumnToUsers < ActiveRecord::Migration[5.2]
 end
 ```
 
-Don’t backfill existing rows in this migration, as it can cause downtime. See the next section for how to do it safely.
-
-> With Postgres, this operation is safe as of Postgres 11
+See the next section for how to backfill.
 
 ### Backfilling data
 
-To backfill data, use the Rails console or a separate migration with `disable_ddl_transaction!`. Avoid backfilling in a transaction, especially one that alters a table. See [this great article](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/) on why.
+#### Bad
 
-```ruby
-class BackfillSomeColumn < ActiveRecord::Migration[5.2]
-  disable_ddl_transaction!
+Active Record creates a transaction around each migration, and backfilling in the same transaction that alters a table keeps the table locked for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
 
+```ruby
+class AddSomeColumnToUsers < ActiveRecord::Migration[6.1]
   def change
-    # Rails 5+
-    User.in_batches.update_all some_column: "default_value"
-
-    # Rails < 5
-    User.find_in_batches do |records|
-      User.where(id: records.map(&:id)).update_all some_column: "default_value"
-    end
+    add_column :users, :some_column, :text
+    User.update_all some_column: "default_value"
   end
 end
 ```
 
-### Removing a column
+Also, running a single query to update data can cause issues for large tables.
 
-ActiveRecord caches database columns at runtime, so if you drop a column, it can cause exceptions until your app reboots. To prevent this:
+#### Good
 
-1. Tell ActiveRecord to ignore the column from its cache
+There are three keys to backfilling safely: batching, throttling, and running it outside a transaction. Use the Rails console or a separate migration with `disable_ddl_transaction!`.
 
-  ```ruby
-  # For Rails 5+
-  class User < ApplicationRecord
-    self.ignored_columns = ["some_column"]
-  end
+```ruby
+class BackfillSomeColumn < ActiveRecord::Migration[6.1]
+  disable_ddl_transaction!
 
-  # For Rails < 5
-  class User < ActiveRecord::Base
-    def self.columns
-      super.reject { |c| c.name == "some_column" }
+  def up
+    User.unscoped.in_batches do |relation|
+      relation.update_all some_column: "default_value"
+      sleep(0.01) # throttle
     end
   end
-  ```
+end
+```
 
-2. Deploy code
-3. Write a migration to remove the column (wrap in `safety_assured` block)
+### Changing the type of a column
 
-  ```ruby
-  class RemoveSomeColumnFromUsers < ActiveRecord::Migration[5.2]
-    def change
-      safety_assured { remove_column :users, :some_column }
-    end
+#### Bad
+
+Changing the type of a column causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
+
+```ruby
+class ChangeSomeColumnType < ActiveRecord::Migration[6.1]
+  def change
+    change_column :users, :some_column, :new_type
   end
-  ```
+end
+```
 
-4. Deploy and run migration
+A few changes don’t require a table rewrite (and are safe) in Postgres:
+
+- Increasing the length limit of a `varchar` column (or removing the limit)
+- Changing a `varchar` column to a `text` column
+- Changing a `text` column to a `varchar` column with no length limit
+- Increasing the precision of a `decimal` or `numeric` column
+- Making a `decimal` or `numeric` column unconstrained
+- Changing between `timestamp` and `timestamptz` columns when session time zone is UTC in Postgres 12+
+
+And a few in MySQL and MariaDB:
 
-### Renaming or changing the type of a column
+- Increasing the length limit of a `varchar` column from under 255 up to 255
+- Increasing the length limit of a `varchar` column from over 255 to the max limit
+
+#### Good
 
 A safer approach is to:
 
@@ -144,10 +228,47 @@ A safer approach is to:
 5. Stop writing to the old column
 6. Drop the old column
 
-One exception is changing a `varchar` column to `text`, which is safe in Postgres 9.1+.
+### Renaming a column
+
+#### Bad
+
+Renaming a column that’s in use will cause errors in your application.
+
+```ruby
+class RenameSomeColumn < ActiveRecord::Migration[6.1]
+  def change
+    rename_column :users, :some_column, :new_name
+  end
+end
+```
+
+#### Good
+
+A safer approach is to:
+
+1. Create a new column
+2. Write to both columns
+3. Backfill data from the old column to the new column
+4. Move reads from the old column to the new column
+5. Stop writing to the old column
+6. Drop the old column
 
 ### Renaming a table
 
+#### Bad
+
+Renaming a table that’s in use will cause errors in your application.
+
+```ruby
+class RenameUsersToCustomers < ActiveRecord::Migration[6.1]
+  def change
+    rename_table :users, :customers
+  end
+end
+```
+
+#### Good
+
 A safer approach is to:
 
 1. Create a new table
@@ -157,12 +278,194 @@ A safer approach is to:
 5. Stop writing to the old table
 6. Drop the old table
 
-### Adding an index (Postgres)
+### Creating a table with the force option
+
+#### Bad
+
+The `force` option can drop an existing table.
+
+```ruby
+class CreateUsers < ActiveRecord::Migration[6.1]
+  def change
+    create_table :users, force: true do |t|
+      # ...
+    end
+  end
+end
+```
+
+#### Good
+
+Create tables without the `force` option.
+
+```ruby
+class CreateUsers < ActiveRecord::Migration[6.1]
+  def change
+    create_table :users do |t|
+      # ...
+    end
+  end
+end
+```
+
+If you intend to drop an existing table, run `drop_table` first.
+
+### Adding a check constraint
+
+:turtle: Safe by default available
+
+#### Bad
+
+Adding a check constraint blocks reads and writes in Postgres and blocks writes in MySQL and MariaDB while every row is checked.
+
+```ruby
+class AddCheckConstraint < ActiveRecord::Migration[6.1]
+  def change
+    add_check_constraint :users, "price > 0", name: "price_check"
+  end
+end
+```
+
+#### Good - Postgres
+
+Add the check constraint without validating existing rows:
+
+```ruby
+class AddCheckConstraint < ActiveRecord::Migration[6.1]
+  def change
+    add_check_constraint :users, "price > 0", name: "price_check", validate: false
+  end
+end
+```
+
+Then validate them in a separate migration.
+
+```ruby
+class ValidateCheckConstraint < ActiveRecord::Migration[6.1]
+  def change
+    validate_check_constraint :users, name: "price_check"
+  end
+end
+```
+
+#### Good - MySQL and MariaDB
+
+[Let us know](https://github.com/ankane/strong_migrations/issues/new) if you have a safe way to do this (check constraints can be added with `NOT ENFORCED`, but enforcing blocks writes).
+
+### Setting NOT NULL on an existing column
+
+:turtle: Safe by default available
+
+#### Bad
+
+Setting `NOT NULL` on an existing column blocks reads and writes while every row is checked.
+
+```ruby
+class SetSomeColumnNotNull < ActiveRecord::Migration[6.1]
+  def change
+    change_column_null :users, :some_column, false
+  end
+end
+```
+
+#### Good - Postgres
+
+Instead, add a check constraint.
+
+For Rails 6.1, use:
+
+```ruby
+class SetSomeColumnNotNull < ActiveRecord::Migration[6.1]
+  def change
+    add_check_constraint :users, "some_column IS NOT NULL", name: "users_some_column_null", validate: false
+  end
+end
+```
+
+For Rails < 6.1, use:
+
+```ruby
+class SetSomeColumnNotNull < ActiveRecord::Migration[6.0]
+  def change
+    safety_assured do
+      execute 'ALTER TABLE "users" ADD CONSTRAINT "users_some_column_null" CHECK ("some_column" IS NOT NULL) NOT VALID'
+    end
+  end
+end
+```
+
+Then validate it in a separate migration. A `NOT NULL` check constraint is [functionally equivalent](https://medium.com/doctolib/adding-a-not-null-constraint-on-pg-faster-with-minimal-locking-38b2c00c4d1c) to setting `NOT NULL` on the column (but it won’t show up in `schema.rb` in Rails < 6.1). In Postgres 12+, once the check constraint is validated, you can safely set `NOT NULL` on the column and drop the check constraint.
+
+For Rails 6.1, use:
+
+```ruby
+class ValidateSomeColumnNotNull < ActiveRecord::Migration[6.1]
+  def change
+    validate_check_constraint :users, name: "users_some_column_null"
+
+    # in Postgres 12+, you can then safely set NOT NULL on the column
+    change_column_null :users, :some_column, false
+    remove_check_constraint :users, name: "users_some_column_null"
+  end
+end
+```
+
+For Rails < 6.1, use:
+
+```ruby
+class ValidateSomeColumnNotNull < ActiveRecord::Migration[6.0]
+  def change
+    safety_assured do
+      execute 'ALTER TABLE "users" VALIDATE CONSTRAINT "users_some_column_null"'
+    end
+
+    # in Postgres 12+, you can then safely set NOT NULL on the column
+    change_column_null :users, :some_column, false
+    safety_assured do
+      execute 'ALTER TABLE "users" DROP CONSTRAINT "users_some_column_null"'
+    end
+  end
+end
+```
+
+#### Good - MySQL and MariaDB
+
+[Let us know](https://github.com/ankane/strong_migrations/issues/new) if you have a safe way to do this.
+
+### Executing SQL directly
+
+Strong Migrations can’t ensure safety for raw SQL statements. Make really sure that what you’re doing is safe, then use:
+
+```ruby
+class ExecuteSQL < ActiveRecord::Migration[6.1]
+  def change
+    safety_assured { execute "..." }
+  end
+end
+```
+
+### Adding an index non-concurrently
+
+:turtle: Safe by default available
+
+#### Bad
+
+In Postgres, adding an index non-concurrently blocks writes.
+
+```ruby
+class AddSomeIndexToUsers < ActiveRecord::Migration[6.1]
+  def change
+    add_index :users, :some_column
+  end
+end
+```
+
+#### Good
 
 Add indexes concurrently.
 
 ```ruby
-class AddSomeIndexToUsers < ActiveRecord::Migration[5.2]
+class AddSomeIndexToUsers < ActiveRecord::Migration[6.1]
   disable_ddl_transaction!
 
   def change
@@ -171,59 +474,201 @@ class AddSomeIndexToUsers < ActiveRecord::Migration[5.2]
 end
 ```
 
-If you forget `disable_ddl_transaction!`, the migration will fail. Also, note that indexes on new tables (those created in the same migration) don’t require this. Check out [gindex](https://github.com/ankane/gindex) to quickly generate index migrations without memorizing the syntax.
+If you forget `disable_ddl_transaction!`, the migration will fail. Also, note that indexes on new tables (those created in the same migration) don’t require this.
 
-Rails 5+ adds an index to references by default. To make sure this happens concurrently, use:
+With [gindex](https://github.com/ankane/gindex), you can generate an index migration instantly with:
+
+```sh
+rails g index table column
+```
+
+### Adding a reference
+
+:turtle: Safe by default available
+
+#### Bad
+
+Rails adds an index non-concurrently to references by default, which blocks writes in Postgres.
 
 ```ruby
-class AddSomeReferenceToUsers < ActiveRecord::Migration[5.2]
+class AddReferenceToUsers < ActiveRecord::Migration[6.1]
+  def change
+    add_reference :users, :city
+  end
+end
+```
+
+#### Good
+
+Make sure the index is added concurrently.
+
+```ruby
+class AddReferenceToUsers < ActiveRecord::Migration[6.1]
   disable_ddl_transaction!
 
   def change
-    add_reference :users, :reference, index: false
-    add_index :users, :reference_id, algorithm: :concurrently
+    add_reference :users, :city, index: {algorithm: :concurrently}
+  end
+end
+```
+
+### Adding a foreign key
+
+:turtle: Safe by default available
+
+#### Bad
+
+In Postgres, adding a foreign key blocks writes on both tables.
+
+```ruby
+class AddForeignKeyOnUsers < ActiveRecord::Migration[6.1]
+  def change
+    add_foreign_key :users, :orders
   end
 end
 ```
 
-For polymorphic references, add a compound index on type and id.
+or
 
-### Adding a json column (Postgres)
+```ruby
+class AddReferenceToUsers < ActiveRecord::Migration[6.1]
+  def change
+    add_reference :users, :order, foreign_key: true
+  end
+end
+```
 
-There’s no equality operator for the `json` column type, which causes issues for `SELECT DISTINCT` queries.
+#### Good
 
-If you’re on Postgres 9.4+, use `jsonb` instead.
+Add the foreign key without validating existing rows, then validate them in a separate migration.
 
-If you must use `json`, replace all calls to `uniq` with a custom scope.
+For Rails 5.2+, use:
 
 ```ruby
-class User < ApplicationRecord
-  scope :uniq_on_id, -> { select("DISTINCT ON (users.id) users.*") }
+class AddForeignKeyOnUsers < ActiveRecord::Migration[6.1]
+  def change
+    add_foreign_key :users, :orders, validate: false
+  end
+end
+```
+
+Then:
+
+```ruby
+class ValidateForeignKeyOnUsers < ActiveRecord::Migration[6.1]
+  def change
+    validate_foreign_key :users, :orders
+  end
+end
+```
+
+For Rails < 5.2, use:
+
+```ruby
+class AddForeignKeyOnUsers < ActiveRecord::Migration[5.1]
+  def change
+    safety_assured do
+      execute 'ALTER TABLE "users" ADD CONSTRAINT "fk_rails_c1e9b98e31" FOREIGN KEY ("order_id") REFERENCES "orders" ("id") NOT VALID'
+    end
+  end
 end
 ```
 
-Then add the column:
+Then:
 
 ```ruby
-class AddJsonColumnToUsers < ActiveRecord::Migration[5.2]
+class ValidateForeignKeyOnUsers < ActiveRecord::Migration[5.1]
   def change
-    safety_assured { add_column :users, :some_column, :json }
+    safety_assured do
+      execute 'ALTER TABLE "users" VALIDATE CONSTRAINT "fk_rails_c1e9b98e31"'
+    end
   end
 end
 ```
 
+### Adding a json column
+
+#### Bad
+
+In Postgres, there’s no equality operator for the `json` column type, which can cause errors for existing `SELECT DISTINCT` queries in your application.
+
+```ruby
+class AddPropertiesToUsers < ActiveRecord::Migration[6.1]
+  def change
+    add_column :users, :properties, :json
+  end
+end
+```
+
+#### Good
+
+Use `jsonb` instead.
+
+```ruby
+class AddPropertiesToUsers < ActiveRecord::Migration[6.1]
+  def change
+    add_column :users, :properties, :jsonb
+  end
+end
+```
+
+### Keeping non-unique indexes to three columns or less
+
+#### Bad
+
+Adding a non-unique index with more than three columns rarely improves performance.
+
+```ruby
+class AddSomeIndexToUsers < ActiveRecord::Migration[6.1]
+  def change
+    add_index :users, [:a, :b, :c, :d]
+  end
+end
+```
+
+#### Good
+
+Instead, start an index with columns that narrow down the results the most.
+
+```ruby
+class AddSomeIndexToUsers < ActiveRecord::Migration[6.1]
+  def change
+    add_index :users, [:b, :d]
+  end
+end
+```
+
+For Postgres, be sure to add them concurrently.
+
 ## Assuring Safety
 
-To mark a step in the migration as safe, despite using method that might otherwise be dangerous, wrap it in a `safety_assured` block.
+To mark a step in the migration as safe, despite using a method that might otherwise be dangerous, wrap it in a `safety_assured` block.
 
 ```ruby
-class MySafeMigration < ActiveRecord::Migration[5.2]
+class MySafeMigration < ActiveRecord::Migration[6.1]
   def change
     safety_assured { remove_column :users, :some_column }
   end
 end
 ```
 
+Certain methods like `execute` and `change_table` cannot be inspected and are prevented from running by default. Make sure what you’re doing is really safe and use this pattern.
+
+## Safe by Default
+
+Make operations safe by default.
+
+- adding and removing an index
+- adding a foreign key
+- adding a check constraint
+- setting NOT NULL on an existing column
+
+Add to `config/initializers/strong_migrations.rb`:
+
+```ruby
+StrongMigrations.safe_by_default = true
+```
+
 ## Custom Checks
 
 Add your own custom checks with:
@@ -238,39 +683,34 @@ end
 
 Use the `stop!` method to stop migrations.
 
-## Existing Migrations
-
-To mark migrations as safe that were created before installing this gem, create an initializer with:
-
-```ruby
-StrongMigrations.start_after = 20170101000000
-```
+Note: Since `remove_column` always requires a `safety_assured` block, it’s not possible to add a custom check for `remove_column` operations.
 
-Use the version from your latest migration.
+## Opt-in Checks
 
-## Dangerous Tasks
+### Removing an index non-concurrently
 
-For safety, dangerous rake tasks are disabled in production - `db:drop`, `db:reset`, `db:schema:load`, and `db:structure:load`. To get around this, use:
+Postgres supports removing indexes concurrently, but removing them non-concurrently shouldn’t be an issue for most applications. You can enable this check with:
 
-```sh
-SAFETY_ASSURED=1 rake db:drop
+```ruby
+StrongMigrations.enable_check(:remove_index)
 ```
 
-## Faster Migrations
+## Disable Checks
 
-Only dump the schema when adding a new migration. If you use Git, create an initializer with:
+Disable specific checks with:
 
 ```ruby
-ActiveRecord::Base.dump_schema_after_migration = Rails.env.development? &&
-  `git status db/migrate/ --porcelain`.present?
+StrongMigrations.disable_check(:add_index)
 ```
 
-## Schema Sanity
+Check the [source code](https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations.rb) for the list of keys.
+
+## Down Migrations / Rollbacks
 
-Columns can flip order in `db/schema.rb` when you have multiple developers. One way to prevent this is to [alphabetize them](https://www.pgrs.net/2008/03/13/alphabetize-schema-rb-columns/). Add to the end of your `Rakefile`:
+By default, checks are disabled when migrating down. Enable them with:
 
 ```ruby
-task "db:schema:dump": "strong_migrations:alphabetize_columns"
+StrongMigrations.check_down = true
 ```
 
 ## Custom Messages
@@ -283,7 +723,92 @@ StrongMigrations.error_messages[:add_column_default] = "Your custom instructions
 
 Check the [source code](https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations.rb) for the list of keys.
 
-## Analyze Tables (Postgres)
+## Migration Timeouts
+
+It’s extremely important to set a short lock timeout for migrations. This way, if a migration can’t acquire a lock in a timely manner, other statements won’t be stuck behind it. We also recommend setting a long statement timeout so migrations can run for a while.
+
+Create `config/initializers/strong_migrations.rb` with:
+
+```ruby
+StrongMigrations.lock_timeout = 10.seconds
+StrongMigrations.statement_timeout = 1.hour
+```
+
+Or set the timeouts directly on the database user that runs migrations. For Postgres, use:
+
+```sql
+ALTER ROLE myuser SET lock_timeout = '10s';
+ALTER ROLE myuser SET statement_timeout = '1h';
+```
+
+Note: If you use PgBouncer in transaction mode, you must set timeouts on the database user.
+
+## App Timeouts
+
+We recommend adding timeouts to `config/database.yml` to prevent connections from hanging and individual queries from taking up too many resources in controllers, jobs, the Rails console, and other places.
+
+For Postgres:
+
+```yml
+production:
+  connect_timeout: 5
+  variables:
+    statement_timeout: 15s
+    lock_timeout: 10s
+```
+
+Note: If you use PgBouncer in transaction mode, you must set the statement and lock timeouts on the database user as shown above.
+
+For MySQL:
+
+```yml
+production:
+  connect_timeout: 5
+  read_timeout: 5
+  write_timeout: 5
+  variables:
+    max_execution_time: 15000 # ms
+    lock_wait_timeout: 10 # sec
+
+```
+
+For MariaDB:
+
+```yml
+production:
+  connect_timeout: 5
+  read_timeout: 5
+  write_timeout: 5
+  variables:
+    max_statement_time: 15 # sec
+    lock_wait_timeout: 10 # sec
+```
+
+For HTTP connections, Redis, and other services, check out [this guide](https://github.com/ankane/the-ultimate-guide-to-ruby-timeouts).
+
+## Existing Migrations
+
+To mark migrations as safe that were created before installing this gem, create an initializer with:
+
+```ruby
+StrongMigrations.start_after = 20170101000000
+```
+
+Use the version from your latest migration.
+
+## Target Version
+
+If your development database version is different from production, you can specify the production version so the right checks run in development.
+
+```ruby
+StrongMigrations.target_version = 10 # or "8.0.12", "10.3.2", etc
+```
+
+The major version works well for Postgres, while the full version is recommended for MySQL and MariaDB.
+
+For safety, this option only affects development and test environments. In other environments, the actual server version is always used.
+
+## Analyze Tables
 
 Analyze tables automatically (to update planner statistics) after an index is added. Create an initializer with:
 
@@ -291,28 +816,41 @@ Analyze tables automatically (to update planner statistics) after an index is ad
 StrongMigrations.auto_analyze = true
 ```
 
-## Lock Timeout (Postgres)
+## Faster Migrations
 
-It’s a good idea to set a lock timeout for the database user that runs migrations. This way, if migrations can’t acquire a lock in a timely manner, other statements won’t be stuck behind it. Here’s a great explanation of [how lock queues work](https://www.citusdata.com/blog/2018/02/15/when-postgresql-blocks/).
+Only dump the schema when adding a new migration. If you use Git, create an initializer with:
 
-```sql
-ALTER ROLE myuser SET lock_timeout = '10s';
+```ruby
+ActiveRecord::Base.dump_schema_after_migration = Rails.env.development? &&
+  `git status db/migrate/ --porcelain`.present?
+```
+
+## Schema Sanity
+
+Columns can flip order in `db/schema.rb` when you have multiple developers. One way to prevent this is to [alphabetize them](https://www.pgrs.net/2008/03/12/alphabetize-schema-rb-columns/). Add to the end of your `Rakefile`:
+
+```ruby
+task "db:schema:dump": "strong_migrations:alphabetize_columns"
 ```
 
-There’s also [a gem](https://github.com/gocardless/activerecord-safer_migrations) you can use for this.
+## Permissions
 
-## Bigint Primary Keys (Postgres & MySQL)
+We recommend using a [separate database user](https://ankane.org/postgres-users) for migrations when possible so you don’t need to grant your app user permission to alter tables.
 
-Rails 5.1+ uses `bigint` for primary keys to keep you from running out of ids. To get this in earlier versions of Rails, check out [rails-bigint-primarykey](https://github.com/Shopify/rails-bigint-primarykey).
+## Smaller Projects
+
+You probably don’t need this gem for smaller projects, as operations that are unsafe at scale can be perfectly safe on smaller, low-traffic tables.
 
 ## Additional Reading
 
 - [Rails Migrations with No Downtime](https://pedro.herokuapp.com/past/2011/7/13/rails_migrations_with_no_downtime/)
-- [Safe Operations For High Volume PostgreSQL](https://www.braintreepayments.com/blog/safe-operations-for-high-volume-postgresql/)
+- [PostgreSQL at Scale: Database Schema Changes Without Downtime](https://medium.com/braintree-product-technology/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680)
+- [An Overview of DDL Algorithms in MySQL](https://mydbops.wordpress.com/2020/03/04/an-overview-of-ddl-algorithms-in-mysql-covers-mysql-8/)
+- [MariaDB InnoDB Online DDL Overview](https://mariadb.com/kb/en/innodb-online-ddl-overview/)
 
 ## Credits
 
-Thanks to Bob Remeika and David Waller for the [original code](https://github.com/foobarfighter/safe-migrations).
+Thanks to Bob Remeika and David Waller for the [original code](https://github.com/foobarfighter/safe-migrations) and [Sean Huber](https://github.com/LendingHome/zero_downtime_migrations) for the bad/good readme format.
 
 ## Contributing
 
@@ -323,11 +861,18 @@ Everyone is encouraged to help improve this project. Here are a few ways you can
 - Write, clarify, or fix documentation
 - Suggest or add new features
 
-To get started with development and testing:
+To get started with development:
 
 ```sh
 git clone https://github.com/ankane/strong_migrations.git
 cd strong_migrations
 bundle install
+
+# Postgres
+createdb strong_migrations_test
 bundle exec rake test
+
+# MySQL and MariaDB
+mysqladmin create strong_migrations_test
+ADAPTER=mysql2 bundle exec rake test
 ```