strong_migrations 1.7.0 → 1.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0b621e800497f9e9c77e0208a899f7e8d17a944caa61dffdea80911e8ddd5ead
-  data.tar.gz: 3f7d4275c416db6b954c1c2e1109ad5118a1c0a7cc6eb25a81203f82a4e47b35
+  metadata.gz: f6462f35144a092664bb7456db2a1ad402695c6b5b1ba7001c1f1f977ea88d3e
+  data.tar.gz: fc71c84ae237b8dbbd3413af7b47ab4fa9962dcbbdd6b3e99b9edd816d3ae009
 SHA512:
-  metadata.gz: '0243834c1f5b12bc637a00b49f8b7eb773dcdd5b8592a354b705d72b6e5b6a728babfb12fc3d058093086529ebfe7de34abd9f1c834bc95323b01f8a48417445'
-  data.tar.gz: a24deb56f0adbe0206791b9155707489d03c02f021842f9284eaadcf5a2b99c30fe6f3ae2c51fef15fc5fef9bf7066b1a72f84c446ed8c12919c43f29eee9f95
+  metadata.gz: 2c4e5bf21e4bb42046fc2d4a530e40d9fc5f843db7c630779789934ec1a47965398e6b637707dc6a7106eb339ef26231ecfea73c312fd9a04f66f96504fcdc6a
+  data.tar.gz: 542be70454fa38a8ad0c29e6bedcf670e54323ff8d9ae8f5eb59756476bbbaca6092a15f09c54163714e3db83261a8554e1fe51b00da8ad05ca7667a4612c3c8

data/CHANGELOG.md

@@ -1,3 +1,9 @@
+## 1.8.0 (2024-03-11)
+
+- Added check for `add_column` with auto-incrementing columns
+- Updated instructions for removing a column to append to `ignored_columns`
+- Fixed check for adding a column with a default value for MySQL and MariaDB
+
 ## 1.7.0 (2024-01-05)
 
 - Added check for `add_unique_constraint`

data/LICENSE.txt

@@ -1,4 +1,4 @@
-Copyright (c) 2013 Bob Remeika and David Waller, 2015-2022 Andrew Kane
+Copyright (c) 2013 Bob Remeika and David Waller, 2015-2024 Andrew Kane
 
 MIT License
 

data/README.md

@@ -38,7 +38,7 @@ Active Record caches attributes, which causes problems
 when removing columns. Be sure to ignore the column:
 
 class User < ApplicationRecord
-  self.ignored_columns = ["name"]
+  self.ignored_columns += ["name"]
 end
 
 Deploy the code, then wrap this step in a safety_assured { ... } block.
@@ -60,15 +60,15 @@ An operation is classified as dangerous if it either:
 Potentially dangerous operations:
 
 - [removing a column](#removing-a-column)
-- [adding a column with a default value](#adding-a-column-with-a-default-value)
-- [backfilling data](#backfilling-data)
-- [adding a stored generated column](#adding-a-stored-generated-column)
 - [changing the type of a column](#changing-the-type-of-a-column)
 - [renaming a column](#renaming-a-column)
 - [renaming a table](#renaming-a-table)
 - [creating a table with the force option](#creating-a-table-with-the-force-option)
+- [adding an auto-incrementing column](#adding-an-auto-incrementing-column) [unreleased]
+- [adding a stored generated column](#adding-a-stored-generated-column)
 - [adding a check constraint](#adding-a-check-constraint)
 - [executing SQL directly](#executing-SQL-directly)
+- [backfilling data](#backfilling-data)
 
 Postgres-specific checks:
 
@@ -79,6 +79,7 @@ Postgres-specific checks:
 - [adding an exclusion constraint](#adding-an-exclusion-constraint)
 - [adding a json column](#adding-a-json-column)
 - [setting NOT NULL on an existing column](#setting-not-null-on-an-existing-column)
+- [adding a column with a default value](#adding-a-column-with-a-default-value)
 
 Config-specific checks:
 
@@ -110,7 +111,7 @@ end
 
   ```ruby
   class User < ApplicationRecord
-    self.ignored_columns = ["some_column"]
+    self.ignored_columns += ["some_column"]
   end
   ```
 
@@ -128,93 +129,6 @@ end
 4. Deploy and run the migration
 5. Remove the line added in step 1
 
-### Adding a column with a default value
-
-#### Bad
-
-In earlier versions of Postgres, MySQL, and MariaDB, adding a column with a default value to an existing table causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
-
-```ruby
-class AddSomeColumnToUsers < ActiveRecord::Migration[7.1]
-  def change
-    add_column :users, :some_column, :text, default: "default_value"
-  end
-end
-```
-
-In Postgres 11+, MySQL 8.0.12+, and MariaDB 10.3.2+, this no longer requires a table rewrite and is safe (except for volatile functions like `gen_random_uuid()`).
-
-#### Good
-
-Instead, add the column without a default value, then change the default.
-
-```ruby
-class AddSomeColumnToUsers < ActiveRecord::Migration[7.1]
-  def up
-    add_column :users, :some_column, :text
-    change_column_default :users, :some_column, "default_value"
-  end
-
-  def down
-    remove_column :users, :some_column
-  end
-end
-```
-
-See the next section for how to backfill.
-
-### Backfilling data
-
-#### Bad
-
-Active Record creates a transaction around each migration, and backfilling in the same transaction that alters a table keeps the table locked for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
-
-```ruby
-class AddSomeColumnToUsers < ActiveRecord::Migration[7.1]
-  def change
-    add_column :users, :some_column, :text
-    User.update_all some_column: "default_value"
-  end
-end
-```
-
-Also, running a single query to update data can cause issues for large tables.
-
-#### Good
-
-There are three keys to backfilling safely: batching, throttling, and running it outside a transaction. Use the Rails console or a separate migration with `disable_ddl_transaction!`.
-
-```ruby
-class BackfillSomeColumn < ActiveRecord::Migration[7.1]
-  disable_ddl_transaction!
-
-  def up
-    User.unscoped.in_batches do |relation|
-      relation.update_all some_column: "default_value"
-      sleep(0.01) # throttle
-    end
-  end
-end
-```
-
-### Adding a stored generated column
-
-#### Bad
-
-Adding a stored generated column causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
-
-```ruby
-class AddSomeColumnToUsers < ActiveRecord::Migration[7.1]
-  def change
-    add_column :users, :some_column, :virtual, type: :string, as: "...", stored: true
-  end
-end
-```
-
-#### Good
-
-Add a non-generated column and use callbacks or triggers instead (or a virtual generated column with MySQL and MariaDB).
-
 ### Changing the type of a column
 
 #### Bad
@@ -343,6 +257,44 @@ end
 
 If you intend to drop an existing table, run `drop_table` first.
 
+### Adding an auto-incrementing column
+
+#### Bad
+
+Adding an auto-incrementing column (`serial`/`bigserial` in Postgres and `AUTO_INCREMENT` in MySQL and MariaDB) causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
+
+```ruby
+class AddIdToCitiesUsers < ActiveRecord::Migration[7.1]
+  def change
+    add_column :cities_users, :id, :primary_key
+  end
+end
+```
+
+With MySQL and MariaDB, this can also [generate different values on replicas](https://dev.mysql.com/doc/mysql-replication-excerpt/8.0/en/replication-features-auto-increment.html) if using statement-based replication.
+
+#### Good
+
+Create a new table and migrate the data with the same steps as [renaming a table](#renaming-a-table).
+
+### Adding a stored generated column
+
+#### Bad
+
+Adding a stored generated column causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
+
+```ruby
+class AddSomeColumnToUsers < ActiveRecord::Migration[7.1]
+  def change
+    add_column :users, :some_column, :virtual, type: :string, as: "...", stored: true
+  end
+end
+```
+
+#### Good
+
+Add a non-generated column and use callbacks or triggers instead (or a virtual generated column with MySQL and MariaDB).
+
 ### Adding a check constraint
 
 :turtle: Safe by default available
@@ -397,6 +349,40 @@ class ExecuteSQL < ActiveRecord::Migration[7.1]
 end
 ```
 
+### Backfilling data
+
+#### Bad
+
+Active Record creates a transaction around each migration, and backfilling in the same transaction that alters a table keeps the table locked for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
+
+```ruby
+class AddSomeColumnToUsers < ActiveRecord::Migration[7.1]
+  def change
+    add_column :users, :some_column, :text
+    User.update_all some_column: "default_value"
+  end
+end
+```
+
+Also, running a single query to update data can cause issues for large tables.
+
+#### Good
+
+There are three keys to backfilling safely: batching, throttling, and running it outside a transaction. Use the Rails console or a separate migration with `disable_ddl_transaction!`.
+
+```ruby
+class BackfillSomeColumn < ActiveRecord::Migration[7.1]
+  disable_ddl_transaction!
+
+  def up
+    User.unscoped.in_batches do |relation|
+      relation.update_all some_column: "default_value"
+      sleep(0.01) # throttle
+    end
+  end
+end
+```
+
 ### Adding an index non-concurrently
 
 :turtle: Safe by default available
@@ -666,6 +652,41 @@ class ValidateSomeColumnNotNull < ActiveRecord::Migration[6.0]
 end
 ```
 
+### Adding a column with a default value
+
+#### Bad
+
+In earlier versions of Postgres, adding a column with a default value to an existing table causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres.
+
+```ruby
+class AddSomeColumnToUsers < ActiveRecord::Migration[7.1]
+  def change
+    add_column :users, :some_column, :text, default: "default_value"
+  end
+end
+```
+
+In Postgres 11+, this no longer requires a table rewrite and is safe (except for volatile functions like `gen_random_uuid()`).
+
+#### Good
+
+Instead, add the column without a default value, then change the default.
+
+```ruby
+class AddSomeColumnToUsers < ActiveRecord::Migration[7.1]
+  def up
+    add_column :users, :some_column, :text
+    change_column_default :users, :some_column, "default_value"
+  end
+
+  def down
+    remove_column :users, :some_column
+  end
+end
+```
+
+Then [backfill the data](#backfilling-data).
+
 ### Changing the default value of a column
 
 #### Bad

data/lib/strong_migrations/adapters/abstract_adapter.rb

@@ -36,6 +36,10 @@ module StrongMigrations
         "reads and writes"
       end
 
+      def auto_incrementing_types
+        ["primary_key"]
+      end
+
       private
 
       def connection

data/lib/strong_migrations/adapters/mariadb_adapter.rb

@@ -25,7 +25,7 @@ module StrongMigrations
       end
 
       def add_column_default_safe?
-        server_version >= Gem::Version.new("10.3.2")
+        true
       end
     end
   end

data/lib/strong_migrations/adapters/mysql_adapter.rb

@@ -44,7 +44,7 @@ module StrongMigrations
       end
 
       def add_column_default_safe?
-        server_version >= Gem::Version.new("8.0.12")
+        true
       end
 
       def change_type_safe?(table, column, type, options, existing_column, existing_type)

data/lib/strong_migrations/adapters/postgresql_adapter.rb

@@ -169,6 +169,10 @@ module StrongMigrations
         rows.empty? || rows.any? { |r| r["provolatile"] == "v" }
       end
 
+      def auto_incrementing_types
+        ["primary_key", "serial", "bigserial"]
+      end
+
       private
 
       def set_timeout(setting, timeout)

data/lib/strong_migrations/checks.rb

@@ -42,9 +42,7 @@ module StrongMigrations
       if options.key?(:default) && (!adapter.add_column_default_safe? || (volatile = (postgresql? && type.to_s == "uuid" && default.to_s.include?("()") && adapter.default_volatile?(default))))
         if options[:null] == false
           options = options.except(:null)
-          append = "
-
-Then add the NOT NULL constraint in separate migrations."
+          append = "\n\nThen add the NOT NULL constraint in separate migrations."
         end
 
         if default.nil?
@@ -78,6 +76,13 @@ Then add the NOT NULL constraint in separate migrations."
       if type.to_s == "virtual" && options[:stored]
         raise_error :add_column_generated_stored, rewrite_blocks: adapter.rewrite_blocks
       end
+
+      if adapter.auto_incrementing_types.include?(type.to_s)
+        append = (mysql? || mariadb?) ? "\n\nIf using statement-based replication, this can also generate different values on replicas." : ""
+        raise_error :add_column_auto_incrementing,
+          rewrite_blocks: adapter.rewrite_blocks,
+          append: append
+      end
     end
 
     def check_add_exclusion_constraint(*args)
@@ -165,9 +170,7 @@ Then add the NOT NULL constraint in separate migrations."
 
           if options.delete(:foreign_key)
             headline = "Adding a foreign key blocks writes on both tables."
-            append = "
-
-Then add the foreign key in separate migrations."
+            append = "\n\nThen add the foreign key in separate migrations."
           else
             headline = "Adding an index non-concurrently locks the table."
           end
@@ -353,7 +356,7 @@ Then add the foreign key in separate migrations."
           cols
         end
 
-      code = "self.ignored_columns = #{columns.inspect}"
+      code = "self.ignored_columns += #{columns.inspect}"
 
       raise_error :remove_column,
         model: args[0].to_s.classify,

data/lib/strong_migrations/error_messages.rb

@@ -53,6 +53,9 @@ end",
     add_column_generated_stored:
 "Adding a stored generated column blocks %{rewrite_blocks} while the entire table is rewritten.",
 
+    add_column_auto_incrementing:
+"Adding an auto-incrementing column blocks %{rewrite_blocks} while the entire table is rewritten.",
+
     change_column:
 "Changing the type of an existing column blocks %{rewrite_blocks}
 while the entire table is rewritten. A safer approach is to:

data/lib/strong_migrations/version.rb

@@ -1,3 +1,3 @@
 module StrongMigrations
-  VERSION = "1.7.0"
+  VERSION = "1.8.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: strong_migrations
 version: !ruby/object:Gem::Version
-  version: 1.7.0
+  version: 1.8.0
 platform: ruby
 authors:
 - Andrew Kane
@@ -10,7 +10,7 @@ authors:
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2024-01-05 00:00:00.000000000 Z
+date: 2024-03-12 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord