strong_migrations 0.2.0 → 0.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 5b381f6d77227aa434bfae57e8bd63b5ee74181a
-  data.tar.gz: f1049edf7a7b01f3c075c0401003de048b7bde90
+  metadata.gz: 0dd3cbacfcf395cdb4ae90cb70d0da70d6411c18
+  data.tar.gz: 4e9d3a96ee6e57347b448b7f8bd3c7a4a412b0d5
 SHA512:
-  metadata.gz: 28d568d129cb527e9e5465f8c583ef7cab4972da9e9142a944a899753b22f8e90346f3c1a9d63c1c3ea32b939b058716aa6734030ebca352a1a8cc507e3bd565
-  data.tar.gz: 7c8bc5d03567a87fab67ec089c9a0ca71d2704079b5a2960530e7b97d3f76e78fd772a949d5e36b19582eea49024cdd2a7e7e6a8062344ae673a70a52be41d96
+  metadata.gz: fb8c4bc25cdea84f541199bf56cfdf27d64d8667e3c740729c06a1fdd3de2116c6b8d0ed794b8dc8a67486a7b84262c78f1a4dd428bf5a67518d00f2e9efadae
+  data.tar.gz: 5e3eef7979383a2dd249ddbfd15c0e7fca2caaad288b69acb10ed1f4cec3ad6d8155bb004dc1daba35b36dcc2a08abc99304dad0a7112327635b802b7c927775

data/CHANGELOG.md

@@ -1,3 +1,10 @@
+## 0.2.1
+
+- Recommend `disable_ddl_transaction!` over `commit_db_transaction`
+- Suggest `jsonb` over `json` in Postgres 9.4+
+- Changing `varchar` to `text` is safe in Postgres 9.1+
+- Do not check number of columns for unique indexes
+
 ## 0.2.0
 
 - Added customizable error messages

data/README.md

@@ -14,55 +14,59 @@ Add this line to your application’s Gemfile:
 gem 'strong_migrations'
 ```
 
+## How It Works
+
+Strong Migrations detects potentially dangerous operations in migrations, prevents them from running, and gives instructions on safer ways to do what you want.
+
+```
+ __          __     _____ _______ _
+ \ \        / /\   |_   _|__   __| |
+  \ \  /\  / /  \    | |    | |  | |
+   \ \/  \/ / /\ \   | |    | |  | |
+    \  /\  / ____ \ _| |_   | |  |_|
+     \/  \/_/    \_\_____|  |_|  (_)
+
+ActiveRecord caches attributes which causes problems
+when removing columns. Be sure to ignore the column:
+
+class User < ApplicationRecord
+  self.ignored_columns = %w(some_column)
+end
+
+Once that's deployed, wrap this step in a safety_assured { ... } block.
+
+More info: https://github.com/ankane/strong_migrations#removing-a-column
+```
+
 ## Dangerous Operations
 
 The following operations can cause downtime or errors:
 
 - adding a column with a non-null default value to an existing table
+- removing a column
 - changing the type of a column
-- renaming a table
 - renaming a column
-- removing a column
+- renaming a table
 - adding an index non-concurrently (Postgres only)
 - adding a `json` column to an existing table (Postgres only)
 
-For more info, check out:
-
-- [Rails Migrations with No Downtime](http://pedro.herokuapp.com/past/2011/7/13/rails_migrations_with_no_downtime/)
-- [Safe Operations For High Volume PostgreSQL](https://www.braintreepayments.com/blog/safe-operations-for-high-volume-postgresql/) (if it’s relevant)
-
 Also checks for best practices:
 
-- keeping indexes to three columns or less
+- keeping non-unique indexes to three columns or less
 
 ## The Zero Downtime Way
 
 ### Adding a column with a default value
 
-1. Add the column without a default value
-2. Add the default value
-3. Commit the transaction - **extremely important if you are backfilling in the migration**
-4. Backfill the column
+Adding a column with a non-null default causes the entire table to be rewritten.
+
+Instead, add the column without a default value, then change the default.
 
 ```ruby
-class AddSomeColumnToUsers < ActiveRecord::Migration
+class AddSomeColumnToUsers < ActiveRecord::Migration[5.1]
   def up
-    # 1
     add_column :users, :some_column, :text
-
-    # 2
     change_column_default :users, :some_column, "default_value"
-
-    # 3
-    commit_db_transaction
-
-    # 4.a (Rails 5+)
-    User.in_batches.update_all some_column: "default_value"
-
-    # 4.b (Rails < 5)
-    User.find_in_batches do |users|
-      User.where(id: users.map(&:id)).update_all some_column: "default_value"
-    end
   end
 
   def down
@@ -71,27 +75,25 @@ class AddSomeColumnToUsers < ActiveRecord::Migration
 end
 ```
 
-### Renaming or changing the type of a column
+### Backfilling data
 
-If you really have to:
+To backfill data, use the Rails console or a separate migration with `disable_ddl_transaction!`. Avoid backfilling in a transaction, especially one that alters a table. See [this great article](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/) on why.
 
-1. Create a new column
-2. Write to both columns
-3. Backfill data from the old column to the new column
-4. Move reads from the old column to the new column
-5. Stop writing to the old column
-6. Drop the old column
-
-### Renaming a table
+```ruby
+class BackfillSomeColumn < ActiveRecord::Migration[5.1]
+  disable_ddl_transaction!
 
-If you really have to:
+  def change
+    # Rails 5+
+    User.in_batches.update_all some_column: "default_value"
 
-1. Create a new table
-2. Write to both tables
-3. Backfill data from the old table to new table
-4. Move reads from the old table to the new table
-5. Stop writing to the old table
-6. Drop the old table
+    # Rails < 5
+    User.find_in_batches do |users|
+      User.where(id: users.map(&:id)).update_all some_column: "default_value"
+    end
+  end
+end
+```
 
 ### Removing a column
 
@@ -117,7 +119,7 @@ ActiveRecord caches database columns at runtime, so if you drop a column, it can
 3. Write a migration to remove the column (wrap in `safety_assured` block)
 
   ```ruby
-  class RemoveSomeColumnFromUsers < ActiveRecord::Migration
+  class RemoveSomeColumnFromUsers < ActiveRecord::Migration[5.1]
     def change
       safety_assured { remove_column :users, :some_column }
     end
@@ -126,25 +128,70 @@ ActiveRecord caches database columns at runtime, so if you drop a column, it can
 
 4. Deploy and run migration
 
+### Renaming or changing the type of a column
+
+If you really have to:
+
+1. Create a new column
+2. Write to both columns
+3. Backfill data from the old column to the new column
+4. Move reads from the old column to the new column
+5. Stop writing to the old column
+6. Drop the old column
+
+One exception is changing a `varchar` column to `text`, which is safe in Postgres 9.1+.
+
+### Renaming a table
+
+If you really have to:
+
+1. Create a new table
+2. Write to both tables
+3. Backfill data from the old table to new table
+4. Move reads from the old table to the new table
+5. Stop writing to the old table
+6. Drop the old table
+
 ### Adding an index (Postgres)
 
 Add indexes concurrently.
 
 ```ruby
-class AddSomeIndexToUsers < ActiveRecord::Migration
+class AddSomeIndexToUsers < ActiveRecord::Migration[5.1]
+  disable_ddl_transaction!
+
   def change
-    commit_db_transaction
     add_index :users, :some_index, algorithm: :concurrently
   end
 end
 ```
 
+If you forget `disable_ddl_transaction!`, the migration will fail.
+
+Also, note that indexes on new tables (those created in the same migration) don’t require this.
+
 ### Adding a json column (Postgres)
 
-There’s no equality operator for the `json` column type, which causes issues for `SELECT DISTINCT` queries. Replace all calls to `uniq` with a custom scope.
+There’s no equality operator for the `json` column type, which causes issues for `SELECT DISTINCT` queries.
+
+If you’re on Postgres 9.4+, use `jsonb` instead.
+
+If you must use `json`, replace all calls to `uniq` with a custom scope.
 
 ```ruby
-scope :uniq_on_id, -> { select("DISTINCT ON (your_table.id) your_table.*") }
+class User < ApplicationRecord
+  scope :uniq_on_id, -> { select("DISTINCT ON (users.id) users.*") }
+end
+```
+
+Then add the column:
+
+```ruby
+class AddJsonColumnToUsers < ActiveRecord::Migration[5.1]
+  def change
+    safety_assured { add_column :users, :some_column, :json }
+  end
+end
 ```
 
 ## Assuring Safety
@@ -152,7 +199,7 @@ scope :uniq_on_id, -> { select("DISTINCT ON (your_table.id) your_table.*") }
 To mark a step in the migration as safe, despite using method that might otherwise be dangerous, wrap it in a `safety_assured` block.
 
 ```ruby
-class MySafeMigration < ActiveRecord::Migration
+class MySafeMigration < ActiveRecord::Migration[5.1]
   def change
     safety_assured { remove_column :users, :some_column }
   end
@@ -194,15 +241,15 @@ Columns can flip order in `db/schema.rb` when you have multiple developers. One
 task "db:schema:dump": "strong_migrations:alphabetize_columns"
 ```
 
-## Custom Error Messages
+## Custom Messages
 
-To customize specific error messages, create an initializer with:
+To customize specific messages, create an initializer with:
 
 ```ruby
 StrongMigrations.error_messages[:add_column_default] = "Your custom instructions"
 ```
 
-Check the source code for the list of keys.
+Check the [source code](https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations.rb) for the list of keys.
 
 ## Analyze Tables (Postgres)
 
@@ -222,6 +269,11 @@ ALTER ROLE myuser SET lock_timeout = '10s';
 
 There’s also [a gem](https://github.com/gocardless/activerecord-safer_migrations) you can use for this.
 
+## Additional Reading
+
+- [Rails Migrations with No Downtime](http://pedro.herokuapp.com/past/2011/7/13/rails_migrations_with_no_downtime/)
+- [Safe Operations For High Volume PostgreSQL](https://www.braintreepayments.com/blog/safe-operations-for-high-volume-postgresql/)
+
 ## Credits
 
 Thanks to Bob Remeika and David Waller for the [original code](https://github.com/foobarfighter/safe-migrations).

data/lib/strong_migrations.rb

@@ -12,15 +12,27 @@ module StrongMigrations
   self.start_after = 0
   self.error_messages = {
     add_column_default:
-"Adding a column with a non-null default requires
-the entire table and indexes to be rewritten. Instead:
+"Adding a column with a non-null default causes
+the entire table to be rewritten.
 
-1. Add the column without a default value
-2. Add the default value
-3. Commit the transaction
-4. Backfill the column",
+Instead, add the column without a default value,
+then change the default.
+
+  def up
+    add_column :users, :some_column, :text
+    change_column_default :users, :some_column, \"default_value\"
+  end
+
+  def down
+    remove_column :users, :some_column
+  end
+
+More info: https://github.com/ankane/strong_migrations#adding-a-column-with-a-default-value",
 
     add_column_json:
+"Use jsonb instead.",
+
+    add_column_json_legacy:
 "There's no equality operator for the json column type.
 Replace all calls to uniq with a custom scope.
 
@@ -91,17 +103,19 @@ More info: https://github.com/ankane/strong_migrations#removing-a-column"
     add_reference:
 "Adding a non-concurrent index locks the table. Instead, use:
 
+  disable_ddl_transaction!
+
   def change
     add_reference :users, :reference, index: false
-    commit_db_transaction
     add_index :users, :reference_id, algorithm: :concurrently
   end",
 
     add_index:
 "Adding a non-concurrent index locks the table. Instead, use:
 
+  disable_ddl_transaction!
+
   def change
-    commit_db_transaction
     add_index :users, :some_column, algorithm: :concurrently
   end",
 

data/lib/strong_migrations/migration.rb

@@ -28,10 +28,10 @@ module StrongMigrations
           raise_error :rename_column
         when :add_index
           columns = args[1]
-          if columns.is_a?(Array) && columns.size > 3
+          options = args[2] || {}
+          if columns.is_a?(Array) && columns.size > 3 && !options[:unique]
             raise_error :add_index_columns
           end
-          options = args[2] || {}
           if postgresql? && options[:algorithm] != :concurrently && !@new_tables.to_a.include?(args[0].to_s)
             raise_error :add_index
           end
@@ -39,9 +39,21 @@ module StrongMigrations
           type = args[2]
           options = args[3] || {}
           raise_error :add_column_default unless options[:default].nil?
-          raise_error :add_column_json if type.to_s == "json"
+          if type.to_s == "json" && postgresql?
+            if postgresql_version >= 90400
+              raise_error :add_column_json
+            else
+              raise_error :add_column_json_legacy
+            end
+          end
         when :change_column
-          raise_error :change_column
+          safe = false
+          # assume Postgres 9.1+ since previous versions are EOL
+          if postgresql? && args[2].to_s == "text"
+            column = connection.columns(args[0]).find { |c| c.name.to_s == args[1].to_s }
+            safe = column && column.type == :string
+          end
+          raise_error :change_column unless safe
         when :create_table
           options = args[1] || {}
           raise_error :create_table if options[:force]
@@ -75,6 +87,10 @@ module StrongMigrations
       %w(PostgreSQL PostGIS).include?(connection.adapter_name)
     end
 
+    def postgresql_version
+      @postgresql_version ||= connection.execute("SHOW server_version_num").first["server_version_num"].to_i
+    end
+
     def version_safe?
       version && version <= StrongMigrations.start_after
     end

data/lib/strong_migrations/version.rb

@@ -1,3 +1,3 @@
 module StrongMigrations
-  VERSION = "0.2.0"
+  VERSION = "0.2.1"
 end

data/strong_migrations.gemspec

@@ -22,5 +22,5 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency "bundler"
   spec.add_development_dependency "rake"
   spec.add_development_dependency "minitest"
-  spec.add_development_dependency "pg"
+  spec.add_development_dependency "pg", "< 1"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: strong_migrations
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.2.1
 platform: ruby
 authors:
 - Bob Remeika
@@ -10,7 +10,7 @@ authors:
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2018-01-07 00:00:00.000000000 Z
+date: 2018-02-08 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord
@@ -72,16 +72,16 @@ dependencies:
   name: pg
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
+    - - "<"
       - !ruby/object:Gem::Version
-        version: '0'
+        version: '1'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
+    - - "<"
       - !ruby/object:Gem::Version
-        version: '0'
+        version: '1'
 description: 
 email:
 - bob.remeika@gmail.com