strong_migrations 0.6.8 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f64d60856479de1a9071b64be2186256e619c30a6caea46d2f8094fb1d3ded4b
-  data.tar.gz: 586c15c10b53ba6273685d3d4bbeaa84f0c23fc973643abac1a2687d3ad2e986
+  metadata.gz: 2ef314694846eec868e7e01b74fcfc9a29d361e01e3bc9102914a0bd3ba0facc
+  data.tar.gz: b27a43c73b0c394bd30579db15a412ade64d53fcc72a5d9a9b8fc35695d3aedd
 SHA512:
-  metadata.gz: d5aac390d0d1f965819311ec4ad0cc18b66eb310768f5ed95635670ca646f8804cf0fb672c94f4f1b59f6d812b027cc2d666607bf9b55d5a5f58f37726056d8c
-  data.tar.gz: f0ec6e96d589aa99df461bbcc29201142a25a74a951edbbdf72c49e8a1d81a58623ccb6f049d2630a5cbd517f9edf9119511087d51da90f95c9794a13f5c1fb1
+  metadata.gz: 793918328d63451ef6a6ec3b90b3d2a4082ef2c57e018fa0f84e223defe506c3722f7e2168ef97854fcade562dc115943d9655954fdd9a319a13dca6ec50648b
+  data.tar.gz: c353256d7854ea1240bfc1e3f6e0ce7b8c76b72404f728c37695f923c4189777463481d57abc55efdce7a0326621a6a61ca9fcb98638e2f2aa279496d126bd20

data/CHANGELOG.md

@@ -1,3 +1,13 @@
+## 0.7.0 (2020-07-22)
+
+- Added `check_down` option
+- Added check for `change_column` with `null: false`
+- Added check for `validate_foreign_key`
+- Improved error messages
+- Made auto analyze less verbose in Postgres
+- Decreasing the length limit of a `varchar` column or adding a limit is not safe in Postgres
+- Removed safety checks for `db` rake tasks (Rails 5+ handles this)
+
 ## 0.6.8 (2020-05-13)
 
 - `change_column_null` on a column with a `NOT NULL` constraint is safe in Postgres 12+

data/LICENSE.txt

@@ -1,4 +1,4 @@
-Copyright (c) 2013 Bob Remeika and David Waller, 2015-2019 Andrew Kane
+Copyright (c) 2013 Bob Remeika and David Waller, 2015-2020 Andrew Kane
 
 MIT License
 

data/README.md

@@ -25,6 +25,36 @@ bundle install
 rails generate strong_migrations:install
 ```
 
+Strong Migrations sets a long statement timeout for migrations so you can set a [short statement timeout](#app-timeouts) for your application.
+
+## How It Works
+
+When you run a migration that’s potentially dangerous, you’ll see an error message like:
+
+```txt
+=== Dangerous operation detected #strong_migrations ===
+
+Active Record caches attributes, which causes problems
+when removing columns. Be sure to ignore the column:
+
+class User < ApplicationRecord
+  self.ignored_columns = ["name"]
+end
+
+Deploy the code, then wrap this step in a safety_assured { ... } block.
+
+class RemoveColumn < ActiveRecord::Migration[6.0]
+  def change
+    safety_assured { remove_column :users, :name, :string }
+  end
+end
+```
+
+An operation is classified as dangerous if it either:
+
+- Blocks reads or writes for more than a few seconds (after a lock is acquired)
+- Has a good chance of causing application errors
+
 ## Checks
 
 Potentially dangerous operations:
@@ -32,11 +62,11 @@ Potentially dangerous operations:
 - [removing a column](#removing-a-column)
 - [adding a column with a default value](#adding-a-column-with-a-default-value)
 - [backfilling data](#backfilling-data)
-- [changing the type of a column](#renaming-or-changing-the-type-of-a-column)
-- [renaming a column](#renaming-or-changing-the-type-of-a-column)
+- [changing the type of a column](#changing-the-type-of-a-column)
+- [renaming a column](#renaming-a-column)
 - [renaming a table](#renaming-a-table)
 - [creating a table with the force option](#creating-a-table-with-the-force-option)
-- [using change_column_null with a default value](#using-change_column_null-with-a-default-value)
+- [setting NOT NULL on an existing column](#setting-not-null-on-an-existing-column)
 - [executing SQL directly](#executing-SQL-directly)
 
 Postgres-specific checks:
@@ -45,7 +75,6 @@ Postgres-specific checks:
 - [adding a reference](#adding-a-reference)
 - [adding a foreign key](#adding-a-foreign-key)
 - [adding a json column](#adding-a-json-column)
-- [setting NOT NULL on an existing column](#setting-not-null-on-an-existing-column)
 
 Best practices:
 
@@ -57,7 +86,7 @@ You can also add [custom checks](#custom-checks) or [disable specific checks](#d
 
 #### Bad
 
-ActiveRecord caches database columns at runtime, so if you drop a column, it can cause exceptions until your app reboots.
+Active Record caches database columns at runtime, so if you drop a column, it can cause exceptions until your app reboots.
 
 ```ruby
 class RemoveSomeColumnFromUsers < ActiveRecord::Migration[6.0]
@@ -69,7 +98,7 @@ end
 
 #### Good
 
-1. Tell ActiveRecord to ignore the column from its cache
+1. Tell Active Record to ignore the column from its cache
 
   ```ruby
   class User < ApplicationRecord
@@ -92,11 +121,9 @@ end
 
 ### Adding a column with a default value
 
-Note: This operation is safe in Postgres 11+, MySQL 8.0.12+, and MariaDB 10.3.2+.
-
 #### Bad
 
-Adding a column with a default value to an existing table causes the entire table to be rewritten.
+In earlier versions of Postgres, MySQL, and MariaDB, adding a column with a default value to an existing table causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
 
 ```ruby
 class AddSomeColumnToUsers < ActiveRecord::Migration[6.0]
@@ -106,6 +133,8 @@ class AddSomeColumnToUsers < ActiveRecord::Migration[6.0]
 end
 ```
 
+In Postgres 11+, MySQL 8.0.12+, and MariaDB 10.3.2+, this no longer requires a table rewrite and is safe.
+
 #### Good
 
 Instead, add the column without a default value, then change the default.
@@ -129,7 +158,7 @@ See the next section for how to backfill.
 
 #### Bad
 
-Backfilling in the same transaction that alters a table locks the table for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
+Active Record creates a transaction around each migration, and backfilling in the same transaction that alters a table keeps the table locked for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
 
 ```ruby
 class AddSomeColumnToUsers < ActiveRecord::Migration[6.0]
@@ -159,19 +188,11 @@ class BackfillSomeColumn < ActiveRecord::Migration[6.0]
 end
 ```
 
-### Renaming or changing the type of a column
+### Changing the type of a column
 
 #### Bad
 
-```ruby
-class RenameSomeColumn < ActiveRecord::Migration[6.0]
-  def change
-    rename_column :users, :some_column, :new_name
-  end
-end
-```
-
-or
+Changing the type of a column causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
 
 ```ruby
 class ChangeSomeColumnType < ActiveRecord::Migration[6.0]
@@ -181,17 +202,44 @@ class ChangeSomeColumnType < ActiveRecord::Migration[6.0]
 end
 ```
 
-A few changes are safe in Postgres:
+A few changes don’t require a table rewrite (and are safe) in Postgres:
 
-- Changing between `varchar` and `text` columns
+- Increasing the length limit of a `varchar` column (or removing the limit)
+- Changing a `varchar` column to a `text` column
+- Changing a `text` column to a `varchar` column with no length limit
 - Increasing the precision of a `decimal` or `numeric` column
 - Making a `decimal` or `numeric` column unconstrained
 - Changing between `timestamp` and `timestamptz` columns when session time zone is UTC in Postgres 12+
 
 And a few in MySQL and MariaDB:
 
-- Increasing the length of a `varchar` column from under 255 up to 255
-- Increasing the length of a `varchar` column over 255
+- Increasing the length limit of a `varchar` column from under 255 up to 255
+- Increasing the length limit of a `varchar` column from over 255 to the max limit
+
+#### Good
+
+A safer approach is to:
+
+1. Create a new column
+2. Write to both columns
+3. Backfill data from the old column to the new column
+4. Move reads from the old column to the new column
+5. Stop writing to the old column
+6. Drop the old column
+
+### Renaming a column
+
+#### Bad
+
+Renaming a column that’s in use will cause errors in your application.
+
+```ruby
+class RenameSomeColumn < ActiveRecord::Migration[6.0]
+  def change
+    rename_column :users, :some_column, :new_name
+  end
+end
+```
 
 #### Good
 
@@ -208,6 +256,8 @@ A safer approach is to:
 
 #### Bad
 
+Renaming a table that’s in use will cause errors in your application.
+
 ```ruby
 class RenameUsersToCustomers < ActiveRecord::Migration[6.0]
   def change
@@ -259,33 +309,55 @@ end
 
 If you intend to drop an existing table, run `drop_table` first.
 
-### Using change_column_null with a default value
+### Setting NOT NULL on an existing column
 
 #### Bad
 
-This generates a single `UPDATE` statement to set the default value.
+Setting `NOT NULL` on an existing column blocks reads and writes while the every row is checked.
 
 ```ruby
-class ChangeSomeColumnNull < ActiveRecord::Migration[6.0]
+class SetSomeColumnNotNull < ActiveRecord::Migration[6.0]
   def change
-    change_column_null :users, :some_column, false, "default_value"
+    change_column_null :users, :some_column, false
   end
 end
 ```
 
-#### Good
+#### Good - Postgres
+
+Instead, add a check constraint:
+
+```ruby
+class SetSomeColumnNotNull < ActiveRecord::Migration[6.0]
+  def change
+    safety_assured do
+      execute 'ALTER TABLE "users" ADD CONSTRAINT "users_some_column_null" CHECK ("some_column" IS NOT NULL) NOT VALID'
+    end
+  end
+end
+```
 
-Backfill the column [safely](#backfilling-data). Then use:
+Then validate it in a separate migration. A `NOT NULL` check constraint is [functionally equivalent](https://medium.com/doctolib/adding-a-not-null-constraint-on-pg-faster-with-minimal-locking-38b2c00c4d1c) to setting `NOT NULL` on the column, but it won’t show up in `schema.rb`. In Postgres 12+, once the check constraint is validated, you can safely set `NOT NULL` on the column and drop the check constraint.
 
 ```ruby
-class ChangeSomeColumnNull < ActiveRecord::Migration[6.0]
+class ValidateSomeColumnNotNull < ActiveRecord::Migration[6.0]
   def change
+    safety_assured do
+      execute 'ALTER TABLE "users" VALIDATE CONSTRAINT "users_some_column_null"'
+    end
+
+    # in Postgres 12+, you can then safely set NOT NULL on the column
     change_column_null :users, :some_column, false
+    safety_assured do
+      execute 'ALTER TABLE "users" DROP CONSTRAINT "users_some_column_null"'
+    end
   end
 end
 ```
 
-Note: In Postgres, `change_column_null` is still [not safe](#setting-not-null-on-an-existing-column) with this method.
+#### Good - MySQL and MariaDB
+
+[Let us know](https://github.com/ankane/strong_migrations/issues/new) if you have a safe way to do this.
 
 ### Executing SQL directly
 
@@ -303,7 +375,7 @@ end
 
 #### Bad
 
-In Postgres, adding an index non-concurrently locks the table.
+In Postgres, adding an index non-concurrently blocks writes.
 
 ```ruby
 class AddSomeIndexToUsers < ActiveRecord::Migration[6.0]
@@ -339,7 +411,7 @@ rails g index table column
 
 #### Bad
 
-Rails adds an index non-concurrently to references by default, which is problematic for Postgres.
+Rails adds an index non-concurrently to references by default, which blocks writes in Postgres.
 
 ```ruby
 class AddReferenceToUsers < ActiveRecord::Migration[6.0]
@@ -367,7 +439,7 @@ end
 
 #### Bad
 
-In Postgres, new foreign keys are validated by default, which acquires a `ShareRowExclusiveLock` that can be [expensive on large tables](https://travisofthenorth.com/blog/2017/2/2/postgres-adding-foreign-keys-with-zero-downtime).
+In Postgres, adding a foreign key blocks writes on both tables.
 
 ```ruby
 class AddForeignKeyOnUsers < ActiveRecord::Migration[6.0]
@@ -389,7 +461,7 @@ end
 
 #### Good
 
-Instead, validate it in a separate migration with a more agreeable `RowShareLock`. This approach is documented by Postgres to have “[the least impact on other work](https://www.postgresql.org/docs/current/sql-altertable.html).”
+Add the foreign key without validating existing rows, then validate them in a separate migration.
 
 For Rails 5.2+, use:
 
@@ -401,7 +473,7 @@ class AddForeignKeyOnUsers < ActiveRecord::Migration[6.0]
 end
 ```
 
-Then validate it in a separate migration.
+Then:
 
 ```ruby
 class ValidateForeignKeyOnUsers < ActiveRecord::Migration[6.0]
@@ -423,7 +495,7 @@ class AddForeignKeyOnUsers < ActiveRecord::Migration[5.1]
 end
 ```
 
-Then validate it in a separate migration.
+Then:
 
 ```ruby
 class ValidateForeignKeyOnUsers < ActiveRecord::Migration[5.1]
@@ -439,7 +511,7 @@ end
 
 #### Bad
 
-In Postgres, there’s no equality operator for the `json` column type, which can cause errors for existing `SELECT DISTINCT` queries.
+In Postgres, there’s no equality operator for the `json` column type, which can cause errors for existing `SELECT DISTINCT` queries in your application.
 
 ```ruby
 class AddPropertiesToUsers < ActiveRecord::Migration[6.0]
@@ -461,54 +533,6 @@ class AddPropertiesToUsers < ActiveRecord::Migration[6.0]
 end
 ```
 
-### Setting NOT NULL on an existing column
-
-#### Bad
-
-In Postgres, setting `NOT NULL` on an existing column requires an `AccessExclusiveLock`, which is expensive on large tables.
-
-```ruby
-class SetSomeColumnNotNull < ActiveRecord::Migration[6.0]
-  def change
-    change_column_null :users, :some_column, false
-  end
-end
-```
-
-#### Good
-
-Instead, add a constraint:
-
-```ruby
-class SetSomeColumnNotNull < ActiveRecord::Migration[6.0]
-  def change
-    safety_assured do
-      execute 'ALTER TABLE "users" ADD CONSTRAINT "users_some_column_null" CHECK ("some_column" IS NOT NULL) NOT VALID'
-    end
-  end
-end
-```
-
-Then validate it in a separate migration.
-
-```ruby
-class ValidateSomeColumnNotNull < ActiveRecord::Migration[6.0]
-  def change
-    safety_assured do
-      execute 'ALTER TABLE "users" VALIDATE CONSTRAINT "users_some_column_null"'
-    end
-
-    # in Postgres 12+, you can safely turn this into a traditional column constraint
-    change_column_null :users, :some_column, false
-    safety_assured do
-      execute 'ALTER TABLE "users" DROP CONSTRAINT "users_some_column_null"'
-    end
-  end
-end
-```
-
-Note: This is not 100% the same as `NOT NULL` column constraint before Postgres 12. Here’s a [good explanation](https://medium.com/doctolib/adding-a-not-null-constraint-on-pg-faster-with-minimal-locking-38b2c00c4d1c).
-
 ### Keeping non-unique indexes to three columns or less
 
 #### Bad
@@ -587,6 +611,14 @@ StrongMigrations.disable_check(:add_index)
 
 Check the [source code](https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations.rb) for the list of keys.
 
+## Down Migrations / Rollbacks [unreleased]
+
+By default, checks are disabled when migrating down. Enable them with:
+
+```ruby
+StrongMigrations.check_down = true
+```
+
 ## Custom Messages
 
 To customize specific messages, create an initializer with:
@@ -597,7 +629,7 @@ StrongMigrations.error_messages[:add_column_default] = "Your custom instructions
 
 Check the [source code](https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations.rb) for the list of keys.
 
-## Timeouts
+## Migration Timeouts
 
 It’s extremely important to set a short lock timeout for migrations. This way, if a migration can’t acquire a lock in a timely manner, other statements won’t be stuck behind it. We also recommend setting a long statement timeout so migrations can run for a while.
 
@@ -617,6 +649,49 @@ ALTER ROLE myuser SET statement_timeout = '1h';
 
 Note: If you use PgBouncer in transaction mode, you must set timeouts on the database user.
 
+## App Timeouts
+
+We recommend adding timeouts to `config/database.yml` to prevent connections from hanging and individual queries from taking up too many resources in controllers, jobs, the Rails console, and other places.
+
+For Postgres:
+
+```yml
+production:
+  connect_timeout: 5
+  variables:
+    statement_timeout: 15s
+    lock_timeout: 10s
+```
+
+Note: If you use PgBouncer in transaction mode, you must set the statement and lock timeouts on the database user as shown above.
+
+For MySQL:
+
+```yml
+production:
+  connect_timeout: 5
+  read_timeout: 5
+  write_timeout: 5
+  variables:
+    max_execution_time: 15000 # ms
+    lock_wait_timeout: 10 # sec
+
+```
+
+For MariaDB:
+
+```yml
+production:
+  connect_timeout: 5
+  read_timeout: 5
+  write_timeout: 5
+  variables:
+    max_statement_time: 15 # sec
+    lock_wait_timeout: 10 # sec
+```
+
+For HTTP connections, Redis, and other services, check out [this guide](https://github.com/ankane/the-ultimate-guide-to-ruby-timeouts).
+
 ## Existing Migrations
 
 To mark migrations as safe that were created before installing this gem, create an initializer with:
@@ -664,22 +739,20 @@ Columns can flip order in `db/schema.rb` when you have multiple developers. One
 task "db:schema:dump": "strong_migrations:alphabetize_columns"
 ```
 
-## Dangerous Tasks
-
-For safety, dangerous database tasks are disabled in production - `db:drop`, `db:reset`, `db:schema:load`, and `db:structure:load`. To get around this, use:
-
-```sh
-SAFETY_ASSURED=1 rails db:drop
-```
-
 ## Permissions
 
 We recommend using a [separate database user](https://ankane.org/postgres-users) for migrations when possible so you don’t need to grant your app user permission to alter tables.
 
+## Smaller Projects
+
+You probably don’t need this gem for smaller projects, as operations that are unsafe at scale can be perfectly safe on smaller, low-traffic tables.
+
 ## Additional Reading
 
 - [Rails Migrations with No Downtime](https://pedro.herokuapp.com/past/2011/7/13/rails_migrations_with_no_downtime/)
 - [PostgreSQL at Scale: Database Schema Changes Without Downtime](https://medium.com/braintree-product-technology/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680)
+- [An Overview of DDL Algorithms in MySQL](https://mydbops.wordpress.com/2020/03/04/an-overview-of-ddl-algorithms-in-mysql-covers-mysql-8/)
+- [MariaDB InnoDB Online DDL Overview](https://mariadb.com/kb/en/innodb-online-ddl-overview/)
 
 ## Credits
 

data/lib/strong_migrations.rb

@@ -17,7 +17,7 @@ module StrongMigrations
   class << self
     attr_accessor :auto_analyze, :start_after, :checks, :error_messages,
       :target_postgresql_version, :target_mysql_version, :target_mariadb_version,
-      :enabled_checks, :lock_timeout, :statement_timeout
+      :enabled_checks, :lock_timeout, :statement_timeout, :check_down
     attr_writer :lock_timeout_limit
   end
   self.auto_analyze = false
@@ -25,7 +25,7 @@ module StrongMigrations
   self.checks = []
   self.error_messages = {
     add_column_default:
-"Adding a column with a non-null default causes the entire table to be rewritten.
+"Adding a column with a non-null default blocks %{rewrite_blocks} while the entire table is rewritten.
 Instead, add the column without a default value, then change the default.
 
 class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
@@ -50,12 +50,18 @@ class Backfill%{migration_name} < ActiveRecord::Migration%{migration_suffix}
 end",
 
     add_column_json:
-"There's no equality operator for the json column type, which can
-cause errors for existing SELECT DISTINCT queries. Use jsonb instead.",
+"There's no equality operator for the json column type, which can cause errors for
+existing SELECT DISTINCT queries in your application. Use jsonb instead.
+
+class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
+  def change
+    %{command}
+  end
+end",
 
     change_column:
-"Changing the type of an existing column requires the entire
-table and indexes to be rewritten. A safer approach is to:
+"Changing the type of an existing column blocks %{rewrite_blocks}
+while the entire table is rewritten. A safer approach is to:
 
 1. Create a new column
 2. Write to both columns
@@ -64,7 +70,10 @@ table and indexes to be rewritten. A safer approach is to:
 5. Stop writing to the old column
 6. Drop the old column",
 
-    remove_column: "Active Record caches attributes which causes problems
+    change_column_with_not_null:
+"Changing the type is safe, but setting NOT NULL is not.",
+
+    remove_column: "Active Record caches attributes, which causes problems
 when removing columns. Be sure to ignore the column%{column_suffix}:
 
 class %{model} < %{base_model}
@@ -80,7 +89,8 @@ class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
 end",
 
     rename_column:
-"Renaming a column is dangerous. A safer approach is to:
+"Renaming a column that's in use will cause errors
+in your application. A safer approach is to:
 
 1. Create a new column
 2. Write to both columns
@@ -90,7 +100,8 @@ end",
 6. Drop the old column",
 
     rename_table:
-"Renaming a table is dangerous. A safer approach is to:
+"Renaming a table that's in use will cause errors
+in your application. A safer approach is to:
 
 1. Create a new table. Don't forget to recreate indexes from the old table
 2. Write to both tables
@@ -111,7 +122,7 @@ class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
 end",
 
     add_index:
-"Adding an index non-concurrently locks the table. Instead, use:
+"Adding an index non-concurrently blocks writes. Instead, use:
 
 class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
   disable_ddl_transaction!
@@ -122,7 +133,7 @@ class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
 end",
 
     remove_index:
-"Removing an index non-concurrently locks the table. Instead, use:
+"Removing an index non-concurrently blocks writes. Instead, use:
 
 class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
   disable_ddl_transaction!
@@ -165,9 +176,8 @@ class Backfill%{migration_name} < ActiveRecord::Migration%{migration_suffix}
 end",
 
     change_column_null_postgresql:
-"Setting NOT NULL on a column requires an AccessExclusiveLock,
-which is expensive on large tables. Instead, use a constraint and
-validate it in a separate migration with a more agreeable RowShareLock.
+"Setting NOT NULL on an existing column blocks reads and writes while every row is checked.
+Instead, add a check constraint and validate it in a separate migration.
 
 class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
   def change
@@ -185,9 +195,9 @@ end",
 "Setting NOT NULL on an existing column is not safe with your database engine.",
 
     add_foreign_key:
-"New foreign keys are validated by default. This acquires an AccessExclusiveLock,
-which is expensive on large tables. Instead, validate it in a separate migration
-with a more agreeable RowShareLock.
+"Adding a foreign key blocks writes on both tables. Instead,
+add the foreign key without validating existing rows,
+then validate them in a separate migration.
 
 class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
   def change
@@ -199,9 +209,14 @@ class Validate%{migration_name} < ActiveRecord::Migration%{migration_suffix}
   def change
     %{validate_foreign_key_code}
   end
-end"
+end",
+
+    validate_foreign_key:
+"Validating a foreign key while writes are blocked is dangerous.
+Use disable_ddl_transaction! or a separate migration."
   }
   self.enabled_checks = (error_messages.keys - [:remove_index]).map { |k| [k, {}] }.to_h
+  self.check_down = false
 
   # private
   def self.developer_env?

data/lib/strong_migrations/checker.rb

@@ -96,11 +96,13 @@ Then add the NOT NULL constraint in separate migrations."
               change_command: command_str("change_column_default", [table, column, default]),
               remove_command: command_str("remove_column", [table, column]),
               code: backfill_code(table, column, default),
-              append: append
+              append: append,
+              rewrite_blocks: rewrite_blocks
           end
 
           if type.to_s == "json" && postgresql?
-            raise_error :add_column_json
+            raise_error :add_column_json,
+              command: command_str("add_column", [table, column, :jsonb, options])
           end
         when :change_column
           table, column, type, options = args
@@ -109,15 +111,24 @@ Then add the NOT NULL constraint in separate migrations."
           safe = false
           existing_column = connection.columns(table).find { |c| c.name.to_s == column.to_s }
           if existing_column
-            sql_type = existing_column.sql_type.split("(").first
+            existing_type = existing_column.sql_type.split("(").first
             if postgresql?
               case type.to_s
-              when "string", "text"
-                # safe to change limit for varchar
-                safe = ["character varying", "text"].include?(sql_type)
+              when "string"
+                # safe to increase limit or remove it
+                # not safe to decrease limit or add a limit
+                case existing_type
+                when "character varying"
+                  safe = !options[:limit] || (existing_column.limit && options[:limit] >= existing_column.limit)
+                when "text"
+                  safe = !options[:limit]
+                end
+              when "text"
+                # safe to change varchar to text (and text to text)
+                safe = ["character varying", "text"].include?(existing_type)
               when "numeric", "decimal"
                 # numeric and decimal are equivalent and can be used interchangably
-                safe = ["numeric", "decimal"].include?(sql_type) &&
+                safe = ["numeric", "decimal"].include?(existing_type) &&
                   (
                     (
                       # unconstrained
@@ -130,7 +141,7 @@ Then add the NOT NULL constraint in separate migrations."
                     )
                   )
               when "datetime", "timestamp", "timestamptz"
-                safe = ["timestamp without time zone", "timestamp with time zone"].include?(sql_type) &&
+                safe = ["timestamp without time zone", "timestamp with time zone"].include?(existing_type) &&
                   postgresql_version >= Gem::Version.new("12") &&
                   connection.select_all("SHOW timezone").first["TimeZone"] == "UTC"
               end
@@ -142,13 +153,19 @@ Then add the NOT NULL constraint in separate migrations."
                 # increased limit, but doesn't change number of length bytes
                 # 1-255 = 1 byte, 256-65532 = 2 bytes, 65533+ = too big for varchar
                 limit = options[:limit] || 255
-                safe = ["varchar"].include?(sql_type) &&
+                safe = ["varchar"].include?(existing_type) &&
                   limit >= existing_column.limit &&
                   (limit <= 255 || existing_column.limit > 255)
               end
             end
           end
-          raise_error :change_column unless safe
+
+          # unsafe to set NOT NULL for safe types
+          if safe && existing_column.null && options[:null] == false
+            raise_error :change_column_with_not_null
+          end
+
+          raise_error :change_column, rewrite_blocks: rewrite_blocks unless safe
         when :create_table
           table, options = args
           options ||= {}
@@ -176,7 +193,7 @@ Then add the NOT NULL constraint in separate migrations."
               end
 
               if options.delete(:foreign_key)
-                headline = "Adding a validated foreign key locks the table."
+                headline = "Adding a foreign key blocks writes on both tables."
                 append = "
 
 Then add the foreign key in separate migrations."
@@ -247,6 +264,10 @@ Then add the foreign key in separate migrations."
                 validate_foreign_key_code: command_str("validate_foreign_key", [from_table, to_table])
             end
           end
+        when :validate_foreign_key
+          if postgresql? && writes_blocked?
+            raise_error :validate_foreign_key
+          end
         end
 
         StrongMigrations.checks.each do |check|
@@ -259,8 +280,7 @@ Then add the foreign key in separate migrations."
       # outdated statistics + a new index can hurt performance of existing queries
       if StrongMigrations.auto_analyze && direction == :up && method == :add_index
         if postgresql?
-          # TODO remove verbose in 0.7.0
-          connection.execute "ANALYZE VERBOSE #{connection.quote_table_name(args[0].to_s)}"
+          connection.execute "ANALYZE #{connection.quote_table_name(args[0].to_s)}"
         elsif mariadb? || mysql?
           connection.execute "ANALYZE TABLE #{connection.quote_table_name(args[0].to_s)}"
         end
@@ -269,6 +289,8 @@ Then add the foreign key in separate migrations."
       result
     end
 
+    private
+
     def set_timeouts
       if !@timeouts_set
         if StrongMigrations.statement_timeout
@@ -303,8 +325,6 @@ Then add the foreign key in separate migrations."
       end
     end
 
-    private
-
     def connection
       @migration.connection
     end
@@ -314,7 +334,8 @@ Then add the foreign key in separate migrations."
     end
 
     def safe?
-      @safe || ENV["SAFETY_ASSURED"] || @migration.is_a?(ActiveRecord::Schema) || direction == :down || version_safe?
+      @safe || ENV["SAFETY_ASSURED"] || @migration.is_a?(ActiveRecord::Schema) ||
+        (direction == :down && !StrongMigrations.check_down) || version_safe?
     end
 
     def version_safe?
@@ -419,11 +440,16 @@ Then add the foreign key in separate migrations."
     end
 
     def constraints(table_name)
-      query = <<-SQL
-        SELECT conname AS name, pg_get_constraintdef(oid) AS def FROM pg_constraint
-        WHERE contype = 'c'
-          AND convalidated
-          AND conrelid = #{connection.quote(connection.quote_table_name(table_name))}::regclass
+      query = <<~SQL
+        SELECT
+          conname AS name,
+          pg_get_constraintdef(oid) AS def
+        FROM
+          pg_constraint
+        WHERE
+          contype = 'c' AND
+          convalidated AND
+          conrelid = #{connection.quote(connection.quote_table_name(table_name))}::regclass
       SQL
       connection.select_all(query.squish).to_a
     end
@@ -472,6 +498,23 @@ Then add the foreign key in separate migrations."
       "#{command} #{str_args.join(", ")}"
     end
 
+    def writes_blocked?
+      query = <<~SQL
+        SELECT
+          relation::regclass::text
+        FROM
+          pg_locks
+        WHERE
+          mode IN ('ShareRowExclusiveLock', 'AccessExclusiveLock') AND
+          pid = pg_backend_pid()
+      SQL
+      connection.select_all(query.squish).any?
+    end
+
+    def rewrite_blocks
+      mysql? || mariadb? ? "writes" : "reads and writes"
+    end
+
     def backfill_code(table, column, default)
       model = table.to_s.classify
       "#{model}.unscoped.in_batches do |relation| \n      relation.update_all #{column}: #{default.inspect}\n      sleep(0.01)\n    end"

data/lib/strong_migrations/railtie.rb

@@ -5,10 +5,6 @@ module StrongMigrations
   class Railtie < Rails::Railtie
     rake_tasks do
       load "tasks/strong_migrations.rake"
-
-      ["db:drop", "db:reset", "db:schema:load", "db:structure:load"].each do |t|
-        Rake::Task[t].enhance ["strong_migrations:safety_assured"]
-      end
     end
   end
 end

data/lib/strong_migrations/version.rb

@@ -1,3 +1,3 @@
 module StrongMigrations
-  VERSION = "0.6.8"
+  VERSION = "0.7.0"
 end

data/lib/tasks/strong_migrations.rake

@@ -1,10 +1,4 @@
-# https://nithinbekal.com/posts/safe-rake-tasks
-
 namespace :strong_migrations do
-  task safety_assured: :environment do
-    raise "Set SAFETY_ASSURED=1 to run this task in production" if Rails.env.production? && !ENV["SAFETY_ASSURED"]
-  end
-
   # https://www.pgrs.net/2008/03/13/alphabetize-schema-rb-columns/
   task :alphabetize_columns do
     $stderr.puts "Dumping schema"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: strong_migrations
 version: !ruby/object:Gem::Version
-  version: 0.6.8
+  version: 0.7.0
 platform: ruby
 authors:
 - Andrew Kane
@@ -10,7 +10,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2020-05-14 00:00:00.000000000 Z
+date: 2020-07-23 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord