strong_migrations 1.6.4 → 1.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 5399a374f21a5e61350adbe125184d43664bc654cd1583804e7b01c9d6240aab
-  data.tar.gz: 71e638db667d72b1fdf433eb27cc4f382a6472bd0f75885f5ae244990c1b768a
+  metadata.gz: f6462f35144a092664bb7456db2a1ad402695c6b5b1ba7001c1f1f977ea88d3e
+  data.tar.gz: fc71c84ae237b8dbbd3413af7b47ab4fa9962dcbbdd6b3e99b9edd816d3ae009
 SHA512:
-  metadata.gz: 4c8dbc405cb94e4b8eb92a08161cb8689996e7a6ba216b326db27a83151f5dd796d53c42e5b4dd2f8e0cb224f3f6c8f94ff81edd23063438a8143f0d36c507b9
-  data.tar.gz: 532f43609d8006f71e325a708a74c45df8161d5957a5173701c77cc22e7dd3111562cbc70b715c6f94c955d08cda0fd43cfc577b9d902cda8a9e1fedcd4693cc
+  metadata.gz: 2c4e5bf21e4bb42046fc2d4a530e40d9fc5f843db7c630779789934ec1a47965398e6b637707dc6a7106eb339ef26231ecfea73c312fd9a04f66f96504fcdc6a
+  data.tar.gz: 542be70454fa38a8ad0c29e6bedcf670e54323ff8d9ae8f5eb59756476bbbaca6092a15f09c54163714e3db83261a8554e1fe51b00da8ad05ca7667a4612c3c8

data/CHANGELOG.md

@@ -1,3 +1,13 @@
+## 1.8.0 (2024-03-11)
+
+- Added check for `add_column` with auto-incrementing columns
+- Updated instructions for removing a column to append to `ignored_columns`
+- Fixed check for adding a column with a default value for MySQL and MariaDB
+
+## 1.7.0 (2024-01-05)
+
+- Added check for `add_unique_constraint`
+
 ## 1.6.4 (2023-10-17)
 
 - Fixed false positives with `revert`

data/LICENSE.txt

@@ -1,4 +1,4 @@
-Copyright (c) 2013 Bob Remeika and David Waller, 2015-2022 Andrew Kane
+Copyright (c) 2013 Bob Remeika and David Waller, 2015-2024 Andrew Kane
 
 MIT License
 

data/README.md

@@ -8,7 +8,7 @@ Supports PostgreSQL, MySQL, and MariaDB
 
 :tangerine: Battle-tested at [Instacart](https://www.instacart.com/opensource)
 
-[![Build Status](https://github.com/ankane/strong_migrations/workflows/build/badge.svg?branch=master)](https://github.com/ankane/strong_migrations/actions)
+[![Build Status](https://github.com/ankane/strong_migrations/actions/workflows/build.yml/badge.svg)](https://github.com/ankane/strong_migrations/actions)
 
 ## Installation
 
@@ -38,7 +38,7 @@ Active Record caches attributes, which causes problems
 when removing columns. Be sure to ignore the column:
 
 class User < ApplicationRecord
-  self.ignored_columns = ["name"]
+  self.ignored_columns += ["name"]
 end
 
 Deploy the code, then wrap this step in a safety_assured { ... } block.
@@ -60,24 +60,26 @@ An operation is classified as dangerous if it either:
 Potentially dangerous operations:
 
 - [removing a column](#removing-a-column)
-- [adding a column with a default value](#adding-a-column-with-a-default-value)
-- [backfilling data](#backfilling-data)
-- [adding a stored generated column](#adding-a-stored-generated-column)
 - [changing the type of a column](#changing-the-type-of-a-column)
 - [renaming a column](#renaming-a-column)
 - [renaming a table](#renaming-a-table)
 - [creating a table with the force option](#creating-a-table-with-the-force-option)
+- [adding an auto-incrementing column](#adding-an-auto-incrementing-column) [unreleased]
+- [adding a stored generated column](#adding-a-stored-generated-column)
 - [adding a check constraint](#adding-a-check-constraint)
 - [executing SQL directly](#executing-SQL-directly)
+- [backfilling data](#backfilling-data)
 
 Postgres-specific checks:
 
 - [adding an index non-concurrently](#adding-an-index-non-concurrently)
 - [adding a reference](#adding-a-reference)
 - [adding a foreign key](#adding-a-foreign-key)
+- [adding a unique constraint](#adding-a-unique-constraint)
 - [adding an exclusion constraint](#adding-an-exclusion-constraint)
 - [adding a json column](#adding-a-json-column)
 - [setting NOT NULL on an existing column](#setting-not-null-on-an-existing-column)
+- [adding a column with a default value](#adding-a-column-with-a-default-value)
 
 Config-specific checks:
 
@@ -109,7 +111,7 @@ end
 
   ```ruby
   class User < ApplicationRecord
-    self.ignored_columns = ["some_column"]
+    self.ignored_columns += ["some_column"]
   end
   ```
 
@@ -127,93 +129,6 @@ end
 4. Deploy and run the migration
 5. Remove the line added in step 1
 
-### Adding a column with a default value
-
-#### Bad
-
-In earlier versions of Postgres, MySQL, and MariaDB, adding a column with a default value to an existing table causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
-
-```ruby
-class AddSomeColumnToUsers < ActiveRecord::Migration[7.1]
-  def change
-    add_column :users, :some_column, :text, default: "default_value"
-  end
-end
-```
-
-In Postgres 11+, MySQL 8.0.12+, and MariaDB 10.3.2+, this no longer requires a table rewrite and is safe (except for volatile functions like `gen_random_uuid()`).
-
-#### Good
-
-Instead, add the column without a default value, then change the default.
-
-```ruby
-class AddSomeColumnToUsers < ActiveRecord::Migration[7.1]
-  def up
-    add_column :users, :some_column, :text
-    change_column_default :users, :some_column, "default_value"
-  end
-
-  def down
-    remove_column :users, :some_column
-  end
-end
-```
-
-See the next section for how to backfill.
-
-### Backfilling data
-
-#### Bad
-
-Active Record creates a transaction around each migration, and backfilling in the same transaction that alters a table keeps the table locked for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
-
-```ruby
-class AddSomeColumnToUsers < ActiveRecord::Migration[7.1]
-  def change
-    add_column :users, :some_column, :text
-    User.update_all some_column: "default_value"
-  end
-end
-```
-
-Also, running a single query to update data can cause issues for large tables.
-
-#### Good
-
-There are three keys to backfilling safely: batching, throttling, and running it outside a transaction. Use the Rails console or a separate migration with `disable_ddl_transaction!`.
-
-```ruby
-class BackfillSomeColumn < ActiveRecord::Migration[7.1]
-  disable_ddl_transaction!
-
-  def up
-    User.unscoped.in_batches do |relation|
-      relation.update_all some_column: "default_value"
-      sleep(0.01) # throttle
-    end
-  end
-end
-```
-
-### Adding a stored generated column
-
-#### Bad
-
-Adding a stored generated column causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
-
-```ruby
-class AddSomeColumnToUsers < ActiveRecord::Migration[7.1]
-  def change
-    add_column :users, :some_column, :virtual, type: :string, as: "...", stored: true
-  end
-end
-```
-
-#### Good
-
-Add a non-generated column and use callbacks or triggers instead (or a virtual generated column with MySQL and MariaDB).
-
 ### Changing the type of a column
 
 #### Bad
@@ -305,7 +220,7 @@ A safer approach is to:
 
 1. Create a new table
 2. Write to both tables
-3. Backfill data from the old table to new table
+3. Backfill data from the old table to the new table
 4. Move reads from the old table to the new table
 5. Stop writing to the old table
 6. Drop the old table
@@ -342,6 +257,44 @@ end
 
 If you intend to drop an existing table, run `drop_table` first.
 
+### Adding an auto-incrementing column
+
+#### Bad
+
+Adding an auto-incrementing column (`serial`/`bigserial` in Postgres and `AUTO_INCREMENT` in MySQL and MariaDB) causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
+
+```ruby
+class AddIdToCitiesUsers < ActiveRecord::Migration[7.1]
+  def change
+    add_column :cities_users, :id, :primary_key
+  end
+end
+```
+
+With MySQL and MariaDB, this can also [generate different values on replicas](https://dev.mysql.com/doc/mysql-replication-excerpt/8.0/en/replication-features-auto-increment.html) if using statement-based replication.
+
+#### Good
+
+Create a new table and migrate the data with the same steps as [renaming a table](#renaming-a-table).
+
+### Adding a stored generated column
+
+#### Bad
+
+Adding a stored generated column causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
+
+```ruby
+class AddSomeColumnToUsers < ActiveRecord::Migration[7.1]
+  def change
+    add_column :users, :some_column, :virtual, type: :string, as: "...", stored: true
+  end
+end
+```
+
+#### Good
+
+Add a non-generated column and use callbacks or triggers instead (or a virtual generated column with MySQL and MariaDB).
+
 ### Adding a check constraint
 
 :turtle: Safe by default available
@@ -396,6 +349,40 @@ class ExecuteSQL < ActiveRecord::Migration[7.1]
 end
 ```
 
+### Backfilling data
+
+#### Bad
+
+Active Record creates a transaction around each migration, and backfilling in the same transaction that alters a table keeps the table locked for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
+
+```ruby
+class AddSomeColumnToUsers < ActiveRecord::Migration[7.1]
+  def change
+    add_column :users, :some_column, :text
+    User.update_all some_column: "default_value"
+  end
+end
+```
+
+Also, running a single query to update data can cause issues for large tables.
+
+#### Good
+
+There are three keys to backfilling safely: batching, throttling, and running it outside a transaction. Use the Rails console or a separate migration with `disable_ddl_transaction!`.
+
+```ruby
+class BackfillSomeColumn < ActiveRecord::Migration[7.1]
+  disable_ddl_transaction!
+
+  def up
+    User.unscoped.in_batches do |relation|
+      relation.update_all some_column: "default_value"
+      sleep(0.01) # throttle
+    end
+  end
+end
+```
+
 ### Adding an index non-concurrently
 
 :turtle: Safe by default available
@@ -512,6 +499,39 @@ class ValidateForeignKeyOnUsers < ActiveRecord::Migration[7.1]
 end
 ```
 
+### Adding a unique constraint
+
+#### Bad
+
+In Postgres, adding a unique constraint creates a unique index, which blocks reads and writes.
+
+```ruby
+class AddUniqueContraint < ActiveRecord::Migration[7.1]
+  def change
+    add_unique_constraint :users, :some_column
+  end
+end
+```
+
+#### Good
+
+Create a unique index concurrently, then use it for the constraint.
+
+```ruby
+class AddUniqueContraint < ActiveRecord::Migration[7.1]
+  disable_ddl_transaction!
+
+  def up
+    add_index :users, :some_column, unique: true, algorithm: :concurrently
+    add_unique_constraint :users, using_index: "index_users_on_some_column"
+  end
+
+  def down
+    remove_unique_constraint :users, :some_column
+  end
+end
+```
+
 ### Adding an exclusion constraint
 
 #### Bad
@@ -632,6 +652,41 @@ class ValidateSomeColumnNotNull < ActiveRecord::Migration[6.0]
 end
 ```
 
+### Adding a column with a default value
+
+#### Bad
+
+In earlier versions of Postgres, adding a column with a default value to an existing table causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres.
+
+```ruby
+class AddSomeColumnToUsers < ActiveRecord::Migration[7.1]
+  def change
+    add_column :users, :some_column, :text, default: "default_value"
+  end
+end
+```
+
+In Postgres 11+, this no longer requires a table rewrite and is safe (except for volatile functions like `gen_random_uuid()`).
+
+#### Good
+
+Instead, add the column without a default value, then change the default.
+
+```ruby
+class AddSomeColumnToUsers < ActiveRecord::Migration[7.1]
+  def up
+    add_column :users, :some_column, :text
+    change_column_default :users, :some_column, "default_value"
+  end
+
+  def down
+    remove_column :users, :some_column
+  end
+end
+```
+
+Then [backfill the data](#backfilling-data).
+
 ### Changing the default value of a column
 
 #### Bad

data/lib/strong_migrations/adapters/abstract_adapter.rb

@@ -36,6 +36,10 @@ module StrongMigrations
         "reads and writes"
       end
 
+      def auto_incrementing_types
+        ["primary_key"]
+      end
+
       private
 
       def connection

data/lib/strong_migrations/adapters/mariadb_adapter.rb

@@ -25,7 +25,7 @@ module StrongMigrations
       end
 
       def add_column_default_safe?
-        server_version >= Gem::Version.new("10.3.2")
+        true
       end
     end
   end

data/lib/strong_migrations/adapters/mysql_adapter.rb

@@ -44,7 +44,7 @@ module StrongMigrations
       end
 
       def add_column_default_safe?
-        server_version >= Gem::Version.new("8.0.12")
+        true
       end
 
       def change_type_safe?(table, column, type, options, existing_column, existing_type)

data/lib/strong_migrations/adapters/postgresql_adapter.rb

@@ -169,6 +169,10 @@ module StrongMigrations
         rows.empty? || rows.any? { |r| r["provolatile"] == "v" }
       end
 
+      def auto_incrementing_types
+        ["primary_key", "serial", "bigserial"]
+      end
+
       private
 
       def set_timeout(setting, timeout)

data/lib/strong_migrations/checker.rb

@@ -48,6 +48,8 @@ module StrongMigrations
           check_add_index(*args)
         when :add_reference, :add_belongs_to
           check_add_reference(method, *args)
+        when :add_unique_constraint
+          check_add_unique_constraint(*args)
         when :change_column
           check_change_column(*args)
         when :change_column_default

data/lib/strong_migrations/checks.rb

@@ -42,9 +42,7 @@ module StrongMigrations
       if options.key?(:default) && (!adapter.add_column_default_safe? || (volatile = (postgresql? && type.to_s == "uuid" && default.to_s.include?("()") && adapter.default_volatile?(default))))
         if options[:null] == false
           options = options.except(:null)
-          append = "
-
-Then add the NOT NULL constraint in separate migrations."
+          append = "\n\nThen add the NOT NULL constraint in separate migrations."
         end
 
         if default.nil?
@@ -78,6 +76,13 @@ Then add the NOT NULL constraint in separate migrations."
       if type.to_s == "virtual" && options[:stored]
         raise_error :add_column_generated_stored, rewrite_blocks: adapter.rewrite_blocks
       end
+
+      if adapter.auto_incrementing_types.include?(type.to_s)
+        append = (mysql? || mariadb?) ? "\n\nIf using statement-based replication, this can also generate different values on replicas." : ""
+        raise_error :add_column_auto_incrementing,
+          rewrite_blocks: adapter.rewrite_blocks,
+          append: append
+      end
     end
 
     def check_add_exclusion_constraint(*args)
@@ -165,9 +170,7 @@ Then add the NOT NULL constraint in separate migrations."
 
           if options.delete(:foreign_key)
             headline = "Adding a foreign key blocks writes on both tables."
-            append = "
-
-Then add the foreign key in separate migrations."
+            append = "\n\nThen add the foreign key in separate migrations."
           else
             headline = "Adding an index non-concurrently locks the table."
           end
@@ -180,6 +183,21 @@ Then add the foreign key in separate migrations."
       end
     end
 
+    def check_add_unique_constraint(*args)
+      args.extract_options!
+      table, column = args
+
+      # column and using_index cannot be used together
+      # check for column to ensure error message can be generated
+      if column && !new_table?(table)
+        index_name = connection.index_name(table, {column: column})
+        raise_error :add_unique_constraint,
+          index_command: command_str(:add_index, [table, column, {unique: true, algorithm: :concurrently}]),
+          constraint_command: command_str(:add_unique_constraint, [table, {using_index: index_name}]),
+          remove_command: command_str(:remove_unique_constraint, [table, column])
+      end
+    end
+
     def check_change_column(*args)
       options = args.extract_options!
       table, column, type = args
@@ -338,7 +356,7 @@ Then add the foreign key in separate migrations."
           cols
         end
 
-      code = "self.ignored_columns = #{columns.inspect}"
+      code = "self.ignored_columns += #{columns.inspect}"
 
       raise_error :remove_column,
         model: args[0].to_s.classify,

data/lib/strong_migrations/error_messages.rb

@@ -53,6 +53,9 @@ end",
     add_column_generated_stored:
 "Adding a stored generated column blocks %{rewrite_blocks} while the entire table is rewritten.",
 
+    add_column_auto_incrementing:
+"Adding an auto-incrementing column blocks %{rewrite_blocks} while the entire table is rewritten.",
+
     change_column:
 "Changing the type of an existing column blocks %{rewrite_blocks}
 while the entire table is rewritten. A safer approach is to:
@@ -88,7 +91,7 @@ in your application. A safer approach is to:
 
 1. Create a new column
 2. Write to both columns
-3. Backfill data from the old column to new column
+3. Backfill data from the old column to the new column
 4. Move reads from the old column to the new column
 5. Stop writing to the old column
 6. Drop the old column",
@@ -99,7 +102,7 @@ in your application. A safer approach is to:
 
 1. Create a new table. Don't forget to recreate indexes from the old table
 2. Write to both tables
-3. Backfill data from the old table to new table
+3. Backfill data from the old table to the new table
 4. Move reads from the old table to the new table
 5. Stop writing to the old table
 6. Drop the old table",
@@ -244,7 +247,24 @@ end",
 Use disable_ddl_transaction! or a separate migration.",
 
     add_exclusion_constraint:
-"Adding an exclusion constraint blocks reads and writes while every row is checked."
+"Adding an exclusion constraint blocks reads and writes while every row is checked.",
+
+    add_unique_constraint:
+"Adding a unique constraint creates a unique index, which blocks reads and writes.
+Instead, create a unique index concurrently, then use it for the constraint.
+
+class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
+  disable_ddl_transaction!
+
+  def up
+    %{index_command}
+    %{constraint_command}
+  end
+
+  def down
+    %{remove_command}
+  end
+end"
   }
   self.enabled_checks = (error_messages.keys - [:remove_index]).map { |k| [k, {}] }.to_h
 end

data/lib/strong_migrations/version.rb

@@ -1,3 +1,3 @@
 module StrongMigrations
-  VERSION = "1.6.4"
+  VERSION = "1.8.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: strong_migrations
 version: !ruby/object:Gem::Version
-  version: 1.6.4
+  version: 1.8.0
 platform: ruby
 authors:
 - Andrew Kane
@@ -10,7 +10,7 @@ authors:
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2023-10-17 00:00:00.000000000 Z
+date: 2024-03-12 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord
@@ -75,7 +75,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.4.10
+rubygems_version: 3.5.3
 signing_key:
 specification_version: 4
 summary: Catch unsafe migrations in development