strong_migrations 1.6.0 → 2.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md CHANGED
@@ -8,7 +8,7 @@ Supports PostgreSQL, MySQL, and MariaDB
8
8
 
9
9
  :tangerine: Battle-tested at [Instacart](https://www.instacart.com/opensource)
10
10
 
11
- [![Build Status](https://github.com/ankane/strong_migrations/workflows/build/badge.svg?branch=master)](https://github.com/ankane/strong_migrations/actions)
11
+ [![Build Status](https://github.com/ankane/strong_migrations/actions/workflows/build.yml/badge.svg)](https://github.com/ankane/strong_migrations/actions)
12
12
 
13
13
  ## Installation
14
14
 
@@ -38,12 +38,12 @@ Active Record caches attributes, which causes problems
38
38
  when removing columns. Be sure to ignore the column:
39
39
 
40
40
  class User < ApplicationRecord
41
- self.ignored_columns = ["name"]
41
+ self.ignored_columns += ["name"]
42
42
  end
43
43
 
44
44
  Deploy the code, then wrap this step in a safety_assured { ... } block.
45
45
 
46
- class RemoveColumn < ActiveRecord::Migration[7.0]
46
+ class RemoveColumn < ActiveRecord::Migration[8.1]
47
47
  def change
48
48
  safety_assured { remove_column :users, :name }
49
49
  end
@@ -60,28 +60,34 @@ An operation is classified as dangerous if it either:
60
60
  Potentially dangerous operations:
61
61
 
62
62
  - [removing a column](#removing-a-column)
63
- - [adding a column with a default value](#adding-a-column-with-a-default-value)
64
- - [backfilling data](#backfilling-data)
65
- - [adding a stored generated column](#adding-a-stored-generated-column)
66
63
  - [changing the type of a column](#changing-the-type-of-a-column)
67
64
  - [renaming a column](#renaming-a-column)
68
65
  - [renaming a table](#renaming-a-table)
69
66
  - [creating a table with the force option](#creating-a-table-with-the-force-option)
67
+ - [adding an auto-incrementing column](#adding-an-auto-incrementing-column)
68
+ - [adding a stored generated column](#adding-a-stored-generated-column)
69
+ - [adding a foreign key](#adding-a-foreign-key)
70
70
  - [adding a check constraint](#adding-a-check-constraint)
71
71
  - [executing SQL directly](#executing-SQL-directly)
72
+ - [backfilling data](#backfilling-data)
72
73
 
73
74
  Postgres-specific checks:
74
75
 
75
76
  - [adding an index non-concurrently](#adding-an-index-non-concurrently)
76
77
  - [adding a reference](#adding-a-reference)
77
- - [adding a foreign key](#adding-a-foreign-key)
78
+ - [adding a unique constraint](#adding-a-unique-constraint)
78
79
  - [adding an exclusion constraint](#adding-an-exclusion-constraint)
79
80
  - [adding a json column](#adding-a-json-column)
81
+ - [adding a column with a volatile default value](#adding-a-column-with-a-volatile-default-value)
80
82
  - [setting NOT NULL on an existing column](#setting-not-null-on-an-existing-column)
83
+ - [renaming an enum value](#renaming-an-enum-value)
84
+ - [renaming a schema](#renaming-a-schema)
81
85
 
82
- Config-specific checks:
86
+ MySQL and MariaDB-specific checks:
83
87
 
84
- - [changing the default value of a column](#changing-the-default-value-of-a-column)
88
+ - [using the COPY algorithm](#using-the-copy-algorithm)
89
+ - [using shared or exclusive locking](#using-shared-or-exclusive-locking)
90
+ - [adding a column with an expression default value](#adding-a-column-with-an-expression-default-value)
85
91
 
86
92
  Best practices:
87
93
 
@@ -96,7 +102,7 @@ You can also add [custom checks](#custom-checks) or [disable specific checks](#d
96
102
  Active Record caches database columns at runtime, so if you drop a column, it can cause exceptions until your app reboots.
97
103
 
98
104
  ```ruby
99
- class RemoveSomeColumnFromUsers < ActiveRecord::Migration[7.0]
105
+ class RemoveSomeColumnFromUsers < ActiveRecord::Migration[8.1]
100
106
  def change
101
107
  remove_column :users, :some_column
102
108
  end
@@ -109,7 +115,7 @@ end
109
115
 
110
116
  ```ruby
111
117
  class User < ApplicationRecord
112
- self.ignored_columns = ["some_column"]
118
+ self.ignored_columns += ["some_column"]
113
119
  end
114
120
  ```
115
121
 
@@ -117,7 +123,7 @@ end
117
123
  3. Write a migration to remove the column (wrap in `safety_assured` block)
118
124
 
119
125
  ```ruby
120
- class RemoveSomeColumnFromUsers < ActiveRecord::Migration[7.0]
126
+ class RemoveSomeColumnFromUsers < ActiveRecord::Migration[8.1]
121
127
  def change
122
128
  safety_assured { remove_column :users, :some_column }
123
129
  end
@@ -127,93 +133,6 @@ end
127
133
  4. Deploy and run the migration
128
134
  5. Remove the line added in step 1
129
135
 
130
- ### Adding a column with a default value
131
-
132
- #### Bad
133
-
134
- In earlier versions of Postgres, MySQL, and MariaDB, adding a column with a default value to an existing table causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
135
-
136
- ```ruby
137
- class AddSomeColumnToUsers < ActiveRecord::Migration[7.0]
138
- def change
139
- add_column :users, :some_column, :text, default: "default_value"
140
- end
141
- end
142
- ```
143
-
144
- In Postgres 11+, MySQL 8.0.12+, and MariaDB 10.3.2+, this no longer requires a table rewrite and is safe (except for volatile functions like `gen_random_uuid()`).
145
-
146
- #### Good
147
-
148
- Instead, add the column without a default value, then change the default.
149
-
150
- ```ruby
151
- class AddSomeColumnToUsers < ActiveRecord::Migration[7.0]
152
- def up
153
- add_column :users, :some_column, :text
154
- change_column_default :users, :some_column, "default_value"
155
- end
156
-
157
- def down
158
- remove_column :users, :some_column
159
- end
160
- end
161
- ```
162
-
163
- See the next section for how to backfill.
164
-
165
- ### Backfilling data
166
-
167
- #### Bad
168
-
169
- Active Record creates a transaction around each migration, and backfilling in the same transaction that alters a table keeps the table locked for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
170
-
171
- ```ruby
172
- class AddSomeColumnToUsers < ActiveRecord::Migration[7.0]
173
- def change
174
- add_column :users, :some_column, :text
175
- User.update_all some_column: "default_value"
176
- end
177
- end
178
- ```
179
-
180
- Also, running a single query to update data can cause issues for large tables.
181
-
182
- #### Good
183
-
184
- There are three keys to backfilling safely: batching, throttling, and running it outside a transaction. Use the Rails console or a separate migration with `disable_ddl_transaction!`.
185
-
186
- ```ruby
187
- class BackfillSomeColumn < ActiveRecord::Migration[7.0]
188
- disable_ddl_transaction!
189
-
190
- def up
191
- User.unscoped.in_batches do |relation|
192
- relation.update_all some_column: "default_value"
193
- sleep(0.01) # throttle
194
- end
195
- end
196
- end
197
- ```
198
-
199
- ### Adding a stored generated column
200
-
201
- #### Bad
202
-
203
- Adding a stored generated column causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
204
-
205
- ```ruby
206
- class AddSomeColumnToUsers < ActiveRecord::Migration[7.0]
207
- def change
208
- add_column :users, :some_column, :virtual, type: :string, as: "...", stored: true
209
- end
210
- end
211
- ```
212
-
213
- #### Good
214
-
215
- Add a non-generated column and use callbacks or triggers instead (or a virtual generated column with MySQL and MariaDB).
216
-
217
136
  ### Changing the type of a column
218
137
 
219
138
  #### Bad
@@ -221,7 +140,7 @@ Add a non-generated column and use callbacks or triggers instead (or a virtual g
221
140
  Changing the type of a column causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
222
141
 
223
142
  ```ruby
224
- class ChangeSomeColumnType < ActiveRecord::Migration[7.0]
143
+ class ChangeSomeColumnType < ActiveRecord::Migration[8.1]
225
144
  def change
226
145
  change_column :users, :some_column, :new_type
227
146
  end
@@ -234,14 +153,14 @@ Type | Safe Changes
234
153
  --- | ---
235
154
  `cidr` | Changing to `inet`
236
155
  `citext` | Changing to `text` if not indexed, changing to `string` with no `:limit` if not indexed
237
- `datetime` | Increasing or removing `:precision`, changing to `timestamptz` when session time zone is UTC in Postgres 12+
156
+ `datetime` | Increasing or removing `:precision`, changing to `timestamptz` when session time zone is UTC
238
157
  `decimal` | Increasing `:precision` at same `:scale`, removing `:precision` and `:scale`
239
158
  `interval` | Increasing or removing `:precision`
240
159
  `numeric` | Increasing `:precision` at same `:scale`, removing `:precision` and `:scale`
241
160
  `string` | Increasing or removing `:limit`, changing to `text`, changing `citext` if not indexed
242
161
  `text` | Changing to `string` with no `:limit`, changing to `citext` if not indexed
243
162
  `time` | Increasing or removing `:precision`
244
- `timestamptz` | Increasing or removing `:limit`, changing to `datetime` when session time zone is UTC in Postgres 12+
163
+ `timestamptz` | Increasing or removing `:limit`, changing to `datetime` when session time zone is UTC
245
164
 
246
165
  And some in MySQL and MariaDB:
247
166
 
@@ -267,7 +186,7 @@ A safer approach is to:
267
186
  Renaming a column that’s in use will cause errors in your application.
268
187
 
269
188
  ```ruby
270
- class RenameSomeColumn < ActiveRecord::Migration[7.0]
189
+ class RenameSomeColumn < ActiveRecord::Migration[8.1]
271
190
  def change
272
191
  rename_column :users, :some_column, :new_name
273
192
  end
@@ -292,7 +211,7 @@ A safer approach is to:
292
211
  Renaming a table that’s in use will cause errors in your application.
293
212
 
294
213
  ```ruby
295
- class RenameUsersToCustomers < ActiveRecord::Migration[7.0]
214
+ class RenameUsersToCustomers < ActiveRecord::Migration[8.1]
296
215
  def change
297
216
  rename_table :users, :customers
298
217
  end
@@ -305,7 +224,7 @@ A safer approach is to:
305
224
 
306
225
  1. Create a new table
307
226
  2. Write to both tables
308
- 3. Backfill data from the old table to new table
227
+ 3. Backfill data from the old table to the new table
309
228
  4. Move reads from the old table to the new table
310
229
  5. Stop writing to the old table
311
230
  6. Drop the old table
@@ -317,7 +236,7 @@ A safer approach is to:
317
236
  The `force` option can drop an existing table.
318
237
 
319
238
  ```ruby
320
- class CreateUsers < ActiveRecord::Migration[7.0]
239
+ class CreateUsers < ActiveRecord::Migration[8.1]
321
240
  def change
322
241
  create_table :users, force: true do |t|
323
242
  # ...
@@ -331,7 +250,7 @@ end
331
250
  Create tables without the `force` option.
332
251
 
333
252
  ```ruby
334
- class CreateUsers < ActiveRecord::Migration[7.0]
253
+ class CreateUsers < ActiveRecord::Migration[8.1]
335
254
  def change
336
255
  create_table :users do |t|
337
256
  # ...
@@ -342,16 +261,125 @@ end
342
261
 
343
262
  If you intend to drop an existing table, run `drop_table` first.
344
263
 
264
+ ### Adding an auto-incrementing column
265
+
266
+ #### Bad
267
+
268
+ Adding an auto-incrementing column (`serial`/`bigserial` in Postgres and `AUTO_INCREMENT` in MySQL and MariaDB) causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
269
+
270
+ ```ruby
271
+ class AddIdToCitiesUsers < ActiveRecord::Migration[8.1]
272
+ def change
273
+ add_column :cities_users, :id, :primary_key
274
+ end
275
+ end
276
+ ```
277
+
278
+ With MySQL and MariaDB, this can also [generate different values on replicas](https://dev.mysql.com/doc/mysql-replication-excerpt/8.0/en/replication-features-auto-increment.html) if using statement-based replication.
279
+
280
+ #### Good
281
+
282
+ Create a new table and migrate the data with the same steps as [renaming a table](#renaming-a-table).
283
+
284
+ ### Adding a stored generated column
285
+
286
+ #### Bad
287
+
288
+ Adding a stored generated column causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
289
+
290
+ ```ruby
291
+ class AddSomeColumnToUsers < ActiveRecord::Migration[8.1]
292
+ def change
293
+ add_column :users, :some_column, :virtual, type: :string, as: "...", stored: true
294
+ end
295
+ end
296
+ ```
297
+
298
+ #### Good
299
+
300
+ Add a non-generated column and use callbacks or triggers instead (or a virtual generated column with MySQL and MariaDB).
301
+
302
+ ### Adding a foreign key
303
+
304
+ :turtle: Safe by default available for Postgres
305
+
306
+ #### Bad
307
+
308
+ Adding a foreign key blocks writes on both tables.
309
+
310
+ ```ruby
311
+ class AddForeignKeyOnUsers < ActiveRecord::Migration[8.1]
312
+ def change
313
+ add_foreign_key :users, :orders
314
+ end
315
+ end
316
+ ```
317
+
318
+ or
319
+
320
+ ```ruby
321
+ class AddReferenceToUsers < ActiveRecord::Migration[8.1]
322
+ def change
323
+ add_reference :users, :order, foreign_key: true
324
+ end
325
+ end
326
+ ```
327
+
328
+ #### Good - Postgres
329
+
330
+ Add the foreign key without validating existing rows:
331
+
332
+ ```ruby
333
+ class AddForeignKeyOnUsers < ActiveRecord::Migration[8.1]
334
+ def change
335
+ add_foreign_key :users, :orders, validate: false
336
+ end
337
+ end
338
+ ```
339
+
340
+ Then validate them in a separate migration.
341
+
342
+ ```ruby
343
+ class ValidateForeignKeyOnUsers < ActiveRecord::Migration[8.1]
344
+ def change
345
+ validate_foreign_key :users, :orders
346
+ end
347
+ end
348
+ ```
349
+
350
+ #### Good - MySQL and MariaDB
351
+
352
+ If you are 100% sure all rows are valid and migrations do not use a connection pooler, you can add the foreign key without validating existing rows:
353
+
354
+ ```ruby
355
+ class AddForeignKeyOnUsers < ActiveRecord::Migration[8.1]
356
+ def up
357
+ safety_assured do
358
+ begin
359
+ execute "SET SESSION foreign_key_checks = 0"
360
+ add_foreign_key :users, :orders
361
+ ensure
362
+ execute "SET SESSION foreign_key_checks = 1"
363
+ end
364
+ end
365
+ end
366
+
367
+ def down
368
+ remove_foreign_key :users, :orders
369
+ end
370
+ end
371
+ ```
372
+
345
373
  ### Adding a check constraint
346
374
 
347
- :turtle: Safe by default available
375
+ :turtle: Safe by default available for Postgres
348
376
 
349
377
  #### Bad
350
378
 
351
379
  Adding a check constraint blocks reads and writes in Postgres and blocks writes in MySQL and MariaDB while every row is checked.
352
380
 
353
381
  ```ruby
354
- class AddCheckConstraint < ActiveRecord::Migration[7.0]
382
+ class AddCheckConstraint < ActiveRecord::Migration[8.1]
355
383
  def change
356
384
  add_check_constraint :users, "price > 0", name: "price_check"
357
385
  end
@@ -363,7 +391,7 @@ end
363
391
  Add the check constraint without validating existing rows:
364
392
 
365
393
  ```ruby
366
- class AddCheckConstraint < ActiveRecord::Migration[7.0]
394
+ class AddCheckConstraint < ActiveRecord::Migration[8.1]
367
395
  def change
368
396
  add_check_constraint :users, "price > 0", name: "price_check", validate: false
369
397
  end
@@ -373,7 +401,7 @@ end
373
401
  Then validate them in a separate migration.
374
402
 
375
403
  ```ruby
376
- class ValidateCheckConstraint < ActiveRecord::Migration[7.0]
404
+ class ValidateCheckConstraint < ActiveRecord::Migration[8.1]
377
405
  def change
378
406
  validate_check_constraint :users, name: "price_check"
379
407
  end
@@ -389,13 +417,53 @@ end
389
417
  Strong Migrations can’t ensure safety for raw SQL statements. Make really sure that what you’re doing is safe, then use:
390
418
 
391
419
  ```ruby
392
- class ExecuteSQL < ActiveRecord::Migration[7.0]
420
+ class ExecuteSQL < ActiveRecord::Migration[8.1]
393
421
  def change
394
422
  safety_assured { execute "..." }
395
423
  end
396
424
  end
397
425
  ```
398
426
 
427
+ ### Backfilling data
428
+
429
+ Note: Strong Migrations does not detect dangerous backfills.
430
+
431
+ #### Bad
432
+
433
+ Active Record creates a transaction around each migration, and backfilling in the same transaction that alters a table keeps the table locked for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
434
+
435
+ ```ruby
436
+ class AddSomeColumnToUsers < ActiveRecord::Migration[8.1]
437
+ def change
438
+ add_column :users, :some_column, :text
439
+ User.update_all some_column: "default_value"
440
+ end
441
+ end
442
+ ```
443
+
444
+ Also, running a single query to update data can cause issues for large tables.
445
+
446
+ #### Good
447
+
448
+ There are three keys to backfilling safely: batching, throttling, and running it outside a transaction. Use the Rails console or a separate migration with `disable_ddl_transaction!`.
449
+
450
+ ```ruby
451
+ class BackfillSomeColumn < ActiveRecord::Migration[8.1]
452
+ disable_ddl_transaction!
453
+
454
+ def up
455
+ User.unscoped.in_batches(of: 10000) do |relation|
456
+ relation.where(some_column: nil).update_all some_column: "default_value"
457
+ sleep(0.01) # throttle
458
+ end
459
+ end
460
+ end
461
+ ```
462
+
463
+ Note: If backfilling with a method other than `update_all`, use `User.reset_column_information` to ensure the model has up-to-date column information.
464
+
465
+ ## Postgres Checks
466
+
399
467
  ### Adding an index non-concurrently
400
468
 
401
469
  :turtle: Safe by default available
@@ -405,7 +473,7 @@ end
405
473
  In Postgres, adding an index non-concurrently blocks writes.
406
474
 
407
475
  ```ruby
408
- class AddSomeIndexToUsers < ActiveRecord::Migration[7.0]
476
+ class AddSomeIndexToUsers < ActiveRecord::Migration[8.1]
409
477
  def change
410
478
  add_index :users, :some_column
411
479
  end
@@ -417,7 +485,7 @@ end
417
485
  Add indexes concurrently.
418
486
 
419
487
  ```ruby
420
- class AddSomeIndexToUsers < ActiveRecord::Migration[7.0]
488
+ class AddSomeIndexToUsers < ActiveRecord::Migration[8.1]
421
489
  disable_ddl_transaction!
422
490
 
423
491
  def change
@@ -443,7 +511,7 @@ rails g index table column
443
511
  Rails adds an index non-concurrently to references by default, which blocks writes in Postgres.
444
512
 
445
513
  ```ruby
446
- class AddReferenceToUsers < ActiveRecord::Migration[7.0]
514
+ class AddReferenceToUsers < ActiveRecord::Migration[8.1]
447
515
  def change
448
516
  add_reference :users, :city
449
517
  end
@@ -455,7 +523,7 @@ end
455
523
  Make sure the index is added concurrently.
456
524
 
457
525
  ```ruby
458
- class AddReferenceToUsers < ActiveRecord::Migration[7.0]
526
+ class AddReferenceToUsers < ActiveRecord::Migration[8.1]
459
527
  disable_ddl_transaction!
460
528
 
461
529
  def change
@@ -464,50 +532,35 @@ class AddReferenceToUsers < ActiveRecord::Migration[7.0]
464
532
  end
465
533
  ```
466
534
 
467
- ### Adding a foreign key
468
-
469
- :turtle: Safe by default available
535
+ ### Adding a unique constraint
470
536
 
471
537
  #### Bad
472
538
 
473
- In Postgres, adding a foreign key blocks writes on both tables.
539
+ In Postgres, adding a unique constraint creates a unique index, which blocks reads and writes.
474
540
 
475
541
  ```ruby
476
- class AddForeignKeyOnUsers < ActiveRecord::Migration[7.0]
542
+ class AddUniqueConstraint < ActiveRecord::Migration[8.1]
477
543
  def change
478
- add_foreign_key :users, :orders
479
- end
480
- end
481
- ```
482
-
483
- or
484
-
485
- ```ruby
486
- class AddReferenceToUsers < ActiveRecord::Migration[7.0]
487
- def change
488
- add_reference :users, :order, foreign_key: true
544
+ add_unique_constraint :users, :some_column
489
545
  end
490
546
  end
491
547
  ```
492
548
 
493
549
  #### Good
494
550
 
495
- Add the foreign key without validating existing rows:
551
+ Create a unique index concurrently, then use it for the constraint.
496
552
 
497
553
  ```ruby
498
- class AddForeignKeyOnUsers < ActiveRecord::Migration[7.0]
499
- def change
500
- add_foreign_key :users, :orders, validate: false
501
- end
502
- end
503
- ```
554
+ class AddUniqueConstraint < ActiveRecord::Migration[8.1]
555
+ disable_ddl_transaction!
504
556
 
505
- Then validate them in a separate migration.
557
+ def up
558
+ add_index :users, :some_column, unique: true, algorithm: :concurrently
559
+ add_unique_constraint :users, using_index: "index_users_on_some_column"
560
+ end
506
561
 
507
- ```ruby
508
- class ValidateForeignKeyOnUsers < ActiveRecord::Migration[7.0]
509
- def change
510
- validate_foreign_key :users, :orders
562
+ def down
563
+ remove_unique_constraint :users, :some_column
511
564
  end
512
565
  end
513
566
  ```
@@ -519,7 +572,7 @@ end
519
572
  In Postgres, adding an exclusion constraint blocks reads and writes while every row is checked.
520
573
 
521
574
  ```ruby
522
- class AddExclusionContraint < ActiveRecord::Migration[7.1]
575
+ class AddExclusionConstraint < ActiveRecord::Migration[8.1]
523
576
  def change
524
577
  add_exclusion_constraint :users, "number WITH =", using: :gist
525
578
  end
@@ -537,7 +590,7 @@ end
537
590
  In Postgres, there’s no equality operator for the `json` column type, which can cause errors for existing `SELECT DISTINCT` queries in your application.
538
591
 
539
592
  ```ruby
540
- class AddPropertiesToUsers < ActiveRecord::Migration[7.0]
593
+ class AddPropertiesToUsers < ActiveRecord::Migration[8.1]
541
594
  def change
542
595
  add_column :users, :properties, :json
543
596
  end
@@ -549,13 +602,46 @@ end
549
602
  Use `jsonb` instead.
550
603
 
551
604
  ```ruby
552
- class AddPropertiesToUsers < ActiveRecord::Migration[7.0]
605
+ class AddPropertiesToUsers < ActiveRecord::Migration[8.1]
553
606
  def change
554
607
  add_column :users, :properties, :jsonb
555
608
  end
556
609
  end
557
610
  ```
558
611
 
612
+ ### Adding a column with a volatile default value
613
+
614
+ #### Bad
615
+
616
+ Adding a column with a volatile default value to an existing table causes the entire table to be rewritten. During this time, reads and writes are blocked.
617
+
618
+ ```ruby
619
+ class AddSomeColumnToUsers < ActiveRecord::Migration[8.1]
620
+ def change
621
+ add_column :users, :some_column, :uuid, default: "gen_random_uuid()"
622
+ end
623
+ end
624
+ ```
625
+
626
+ #### Good
627
+
628
+ Instead, add the column without a default value, then change the default.
629
+
630
+ ```ruby
631
+ class AddSomeColumnToUsers < ActiveRecord::Migration[8.1]
632
+ def up
633
+ add_column :users, :some_column, :uuid
634
+ change_column_default :users, :some_column, "gen_random_uuid()"
635
+ end
636
+
637
+ def down
638
+ remove_column :users, :some_column
639
+ end
640
+ end
641
+ ```
642
+
643
+ Then [backfill the data](#backfilling-data).
644
+
559
645
  ### Setting NOT NULL on an existing column
560
646
 
561
647
  :turtle: Safe by default available
@@ -565,7 +651,7 @@ end
565
651
  In Postgres, setting `NOT NULL` on an existing column blocks reads and writes while every row is checked.
566
652
 
567
653
  ```ruby
568
- class SetSomeColumnNotNull < ActiveRecord::Migration[7.0]
654
+ class SetSomeColumnNotNull < ActiveRecord::Migration[8.1]
569
655
  def change
570
656
  change_column_null :users, :some_column, false
571
657
  end
@@ -576,92 +662,177 @@ end
576
662
 
577
663
  Instead, add a check constraint.
578
664
 
579
- For Rails 6.1, use:
580
-
581
665
  ```ruby
582
- class SetSomeColumnNotNull < ActiveRecord::Migration[7.0]
666
+ class SetSomeColumnNotNull < ActiveRecord::Migration[8.1]
583
667
  def change
584
668
  add_check_constraint :users, "some_column IS NOT NULL", name: "users_some_column_null", validate: false
585
669
  end
586
670
  end
587
671
  ```
588
672
 
589
- For Rails < 6.1, use:
673
+ Then validate it in a separate migration. Once the check constraint is validated, you can safely set `NOT NULL` on the column and drop the check constraint.
590
674
 
591
675
  ```ruby
592
- class SetSomeColumnNotNull < ActiveRecord::Migration[6.0]
593
- def change
594
- safety_assured do
595
- execute 'ALTER TABLE "users" ADD CONSTRAINT "users_some_column_null" CHECK ("some_column" IS NOT NULL) NOT VALID'
596
- end
676
+ class ValidateSomeColumnNotNull < ActiveRecord::Migration[8.1]
677
+ def up
678
+ validate_check_constraint :users, name: "users_some_column_null"
679
+ change_column_null :users, :some_column, false
680
+ remove_check_constraint :users, name: "users_some_column_null"
681
+ end
682
+
683
+ def down
684
+ add_check_constraint :users, "some_column IS NOT NULL", name: "users_some_column_null", validate: false
685
+ change_column_null :users, :some_column, true
597
686
  end
598
687
  end
599
688
  ```
600
689
 
601
- Then validate it in a separate migration. A `NOT NULL` check constraint is [functionally equivalent](https://medium.com/doctolib/adding-a-not-null-constraint-on-pg-faster-with-minimal-locking-38b2c00c4d1c) to setting `NOT NULL` on the column (but it won’t show up in `schema.rb` in Rails < 6.1). In Postgres 12+, once the check constraint is validated, you can safely set `NOT NULL` on the column and drop the check constraint.
690
+ ### Renaming an enum value
691
+
692
+ #### Bad
602
693
 
603
- For Rails 6.1, use:
694
+ Renaming an enum value that’s in use will cause errors in your application.
604
695
 
605
696
  ```ruby
606
- class ValidateSomeColumnNotNull < ActiveRecord::Migration[7.0]
697
+ class RenameDoneToCompleted < ActiveRecord::Migration[8.1]
607
698
  def change
608
- validate_check_constraint :users, name: "users_some_column_null"
699
+ rename_enum_value :status, from: "done", to: "completed"
700
+ end
701
+ end
702
+ ```
609
703
 
610
- # in Postgres 12+, you can then safely set NOT NULL on the column
611
- change_column_null :users, :some_column, false
612
- remove_check_constraint :users, name: "users_some_column_null"
704
+ #### Good
705
+
706
+ A safer approach is to:
707
+
708
+ 1. Add a new enum value before or after the old value
709
+ 2. Update application code to handle both values and write the new value
710
+ 3. Backfill data from the old value to the new value
711
+
712
+ ```ruby
713
+ class AddCompletedToStatus < ActiveRecord::Migration[8.1]
714
+ def up
715
+ add_enum_value :status, "completed", after: "done"
613
716
  end
614
717
  end
615
718
  ```
616
719
 
617
- For Rails < 6.1, use:
720
+ Removing enum values is not supported in Postgres (without creating a new enum).
721
+
722
+ ### Renaming a schema
723
+
724
+ #### Bad
725
+
726
+ Renaming a schema that’s in use will cause errors in your application.
618
727
 
619
728
  ```ruby
620
- class ValidateSomeColumnNotNull < ActiveRecord::Migration[6.0]
729
+ class RenameUsersToCustomers < ActiveRecord::Migration[8.1]
621
730
  def change
622
- safety_assured do
623
- execute 'ALTER TABLE "users" VALIDATE CONSTRAINT "users_some_column_null"'
624
- end
625
-
626
- # in Postgres 12+, you can then safely set NOT NULL on the column
627
- change_column_null :users, :some_column, false
628
- safety_assured do
629
- execute 'ALTER TABLE "users" DROP CONSTRAINT "users_some_column_null"'
630
- end
731
+ rename_schema :users, :customers
631
732
  end
632
733
  end
633
734
  ```
634
735
 
635
- ### Changing the default value of a column
736
+ #### Good
737
+
738
+ A safer approach is to:
739
+
740
+ 1. Create a new schema
741
+ 2. Write to both schemas
742
+ 3. Backfill data from the old schema to the new schema
743
+ 4. Move reads from the old schema to the new schema
744
+ 5. Stop writing to the old schema
745
+ 6. Drop the old schema
746
+
747
+ ## MySQL and MariaDB Checks
748
+
749
+ ### Using the COPY algorithm
636
750
 
637
751
  #### Bad
638
752
 
639
- Rails < 7 enables partial writes by default, which can cause incorrect values to be inserted when changing the default value of a column.
753
+ In MySQL and MariaDB, using the `COPY` algorithm blocks writes.
640
754
 
641
755
  ```ruby
642
- class ChangeSomeColumnDefault < ActiveRecord::Migration[6.1]
756
+ class AddSomeIndexToUsers < ActiveRecord::Migration[8.1]
643
757
  def change
644
- change_column_default :users, :some_column, from: "old", to: "new"
758
+ add_index :users, :some_column, algorithm: :copy
645
759
  end
646
760
  end
761
+ ```
647
762
 
648
- User.create!(some_column: "old") # can insert "new"
763
+ #### Good
764
+
765
+ Use the default algorithm.
766
+
767
+ ```ruby
768
+ class AddSomeIndexToUsers < ActiveRecord::Migration[8.1]
769
+ def change
770
+ add_index :users, :some_column
771
+ end
772
+ end
773
+ ```
774
+
775
+ ### Using shared or exclusive locking
776
+
777
+ #### Bad
778
+
779
+ In MySQL and MariaDB, using shared locking blocks writes, and using exclusive locking blocks reads and writes.
780
+
781
+ ```ruby
782
+ class AddSomeIndexToUsers < ActiveRecord::Migration[8.2]
783
+ def change
784
+ add_index :users, :some_column, lock: :shared
785
+ end
786
+ end
649
787
  ```
650
788
 
651
789
  #### Good
652
790
 
653
- Disable partial writes in `config/application.rb`. For Rails < 7, use:
791
+ Use the default locking or no locking.
654
792
 
655
793
  ```ruby
656
- config.active_record.partial_writes = false
794
+ class AddSomeIndexToUsers < ActiveRecord::Migration[8.2]
795
+ def change
796
+ add_index :users, :some_column
797
+ end
798
+ end
657
799
  ```
658
800
 
659
- For Rails 7, use:
801
+ ### Adding a column with an expression default value
802
+
803
+ #### Bad
804
+
805
+ In MySQL and MariaDB, adding a column with an expression default value to an existing table causes the entire table to be rewritten. During this time, writes are blocked.
806
+
807
+ ```ruby
808
+ class AddSomeColumnToUsers < ActiveRecord::Migration[8.1]
809
+ def change
810
+ add_column :users, :some_column, :datetime, default: -> { "(now())" }
811
+ end
812
+ end
813
+ ```
814
+
815
+ #### Good
816
+
817
+ Instead, add the column without a default value, then change the default.
660
818
 
661
819
  ```ruby
662
- config.active_record.partial_inserts = false
820
+ class AddSomeColumnToUsers < ActiveRecord::Migration[8.1]
821
+ def up
822
+ add_column :users, :some_column, :datetime
823
+ change_column_default :users, :some_column, -> { "(now())" }
824
+ end
825
+
826
+ def down
827
+ remove_column :users, :some_column
828
+ end
829
+ end
663
830
  ```
664
831
 
832
+ Then [backfill the data](#backfilling-data).
833
+
834
+ ## Best Practices
835
+
665
836
  ### Keeping non-unique indexes to three columns or less
666
837
 
667
838
  #### Bad
@@ -669,7 +840,7 @@ config.active_record.partial_inserts = false
669
840
  Adding a non-unique index with more than three columns rarely improves performance.
670
841
 
671
842
  ```ruby
672
- class AddSomeIndexToUsers < ActiveRecord::Migration[7.0]
843
+ class AddSomeIndexToUsers < ActiveRecord::Migration[8.1]
673
844
  def change
674
845
  add_index :users, [:a, :b, :c, :d]
675
846
  end
@@ -681,9 +852,9 @@ end
681
852
  Instead, start an index with columns that narrow down the results the most.
682
853
 
683
854
  ```ruby
684
- class AddSomeIndexToUsers < ActiveRecord::Migration[7.0]
855
+ class AddSomeIndexToUsers < ActiveRecord::Migration[8.1]
685
856
  def change
686
- add_index :users, [:b, :d]
857
+ add_index :users, [:d, :b]
687
858
  end
688
859
  end
689
860
  ```
@@ -695,7 +866,7 @@ For Postgres, be sure to add them concurrently.
695
866
  To mark a step in the migration as safe, despite using a method that might otherwise be dangerous, wrap it in a `safety_assured` block.
696
867
 
697
868
  ```ruby
698
- class MySafeMigration < ActiveRecord::Migration[7.0]
869
+ class MySafeMigration < ActiveRecord::Migration[8.1]
699
870
  def change
700
871
  safety_assured { remove_column :users, :some_column }
701
872
  end
@@ -706,7 +877,7 @@ Certain methods like `execute` and `change_table` cannot be inspected and are pr
706
877
 
707
878
  ## Safe by Default
708
879
 
709
- Make operations safe by default.
880
+ Make certain operations safe by default. This allows you to write the code under the "Bad" section, but the migration will be performed as if you had written the "Good" version.
710
881
 
711
882
  - adding and removing an index
712
883
  - adding a foreign key
@@ -755,6 +926,16 @@ StrongMigrations.disable_check(:add_index)
755
926
 
756
927
  Check the [source code](https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations/error_messages.rb) for the list of keys.
757
928
 
929
+ ## Skip Databases
930
+
931
+ Skip checks and other functionality for specific databases with:
932
+
933
+ ```ruby
934
+ StrongMigrations.skip_database(:catalog)
935
+ ```
936
+
937
+ Note: This does not affect `alphabetize_schema`.
938
+
758
939
  ## Down Migrations / Rollbacks
759
940
 
760
941
  By default, checks are disabled when migrating down. Enable them with:
@@ -791,26 +972,7 @@ ALTER ROLE myuser SET lock_timeout = '10s';
791
972
  ALTER ROLE myuser SET statement_timeout = '1h';
792
973
  ```
793
974
 
794
- Note: If you use PgBouncer in transaction mode, you must set timeouts on the database user.
795
-
796
- ## Lock Timeout Retries [experimental]
797
-
798
- There’s the option to automatically retry statements when the lock timeout is reached. Here’s how it works:
799
-
800
- - If a lock timeout happens outside a transaction, the statement is retried
801
- - If it happens inside the DDL transaction, the entire migration is retried (only applicable to Postgres)
802
-
803
- Add to `config/initializers/strong_migrations.rb`:
804
-
805
- ```ruby
806
- StrongMigrations.lock_timeout_retries = 3
807
- ```
808
-
809
- Set the delay between retries with:
810
-
811
- ```ruby
812
- StrongMigrations.lock_timeout_retry_delay = 10.seconds
813
- ```
975
+ Note: If you use a connection pooler like PgBouncer in transaction mode, you must set timeouts on the database user.
814
976
 
815
977
  ## App Timeouts
816
978
 
@@ -826,7 +988,7 @@ production:
826
988
  lock_timeout: 10s
827
989
  ```
828
990
 
829
- Note: If you use PgBouncer in transaction mode, you must set the statement and lock timeouts on the database user as shown above.
991
+ Note: If you use a connection pooler like PgBouncer in transaction mode, you must set the statement and lock timeouts on the database user as shown above.
830
992
 
831
993
  For MySQL:
832
994
 
@@ -855,12 +1017,43 @@ production:
855
1017
 
856
1018
  For HTTP connections, Redis, and other services, check out [this guide](https://github.com/ankane/the-ultimate-guide-to-ruby-timeouts).
857
1019
 
1020
+ ## Invalid Indexes
1021
+
1022
+ In Postgres, adding an index non-concurrently can leave behind an invalid index if the lock timeout is reached. Running the migration again can result in an error.
1023
+
1024
+ To automatically remove the invalid index when the migration runs again, use:
1025
+
1026
+ ```ruby
1027
+ StrongMigrations.remove_invalid_indexes = true
1028
+ ```
1029
+
1030
+ ## Lock Timeout Retries
1031
+
1032
+ Note: This feature is experimental.
1033
+
1034
+ There’s the option to automatically retry statements for migrations when the lock timeout is reached. Here’s how it works:
1035
+
1036
+ - If a lock timeout happens outside a transaction, the statement is retried
1037
+ - If it happens inside the DDL transaction, the entire migration is retried (only applicable to Postgres)
1038
+
1039
+ Add to `config/initializers/strong_migrations.rb`:
1040
+
1041
+ ```ruby
1042
+ StrongMigrations.lock_timeout_retries = 3
1043
+ ```
1044
+
1045
+ Set the delay between retries with:
1046
+
1047
+ ```ruby
1048
+ StrongMigrations.lock_timeout_retry_delay = 10.seconds
1049
+ ```
1050
+
858
1051
  ## Existing Migrations
859
1052
 
860
1053
  To mark migrations as safe that were created before installing this gem, create an initializer with:
861
1054
 
862
1055
  ```ruby
863
- StrongMigrations.start_after = 20170101000000
1056
+ StrongMigrations.start_after = 20250101000000
864
1057
  ```
865
1058
 
866
1059
  Use the version from your latest migration.
@@ -870,17 +1063,17 @@ Use the version from your latest migration.
870
1063
  If your development database version is different from production, you can specify the production version so the right checks run in development.
871
1064
 
872
1065
  ```ruby
873
- StrongMigrations.target_version = 10 # or "8.0.12", "10.3.2", etc
1066
+ StrongMigrations.target_version = 16
874
1067
  ```
875
1068
 
876
- The major version works well for Postgres, while the full version is recommended for MySQL and MariaDB.
1069
+ The major version works well for Postgres, while the major and minor version is recommended for MySQL and MariaDB.
877
1070
 
878
1071
  For safety, this option only affects development and test environments. In other environments, the actual server version is always used.
879
1072
 
880
- If your app has multiple databases with different versions, with Rails 6.1+, you can use:
1073
+ If your app has multiple databases with different versions, you can use:
881
1074
 
882
1075
  ```ruby
883
- StrongMigrations.target_version = {primary: 13, catalog: 15}
1076
+ StrongMigrations.target_version = {primary: 16, catalog: 18}
884
1077
  ```
885
1078
 
886
1079
  ## Analyze Tables
@@ -893,15 +1086,20 @@ StrongMigrations.auto_analyze = true
893
1086
 
894
1087
  ## Faster Migrations
895
1088
 
896
- Only dump the schema when adding a new migration. If you use Git, add to `config/environments/development.rb`:
1089
+ Only dump the schema when adding a new migration. If you use Git, add to the end of your `Rakefile`:
897
1090
 
898
1091
  ```rb
899
- config.active_record.dump_schema_after_migration = `git status db/migrate/ --porcelain`.present?
1092
+ task :faster_migrations do
1093
+ ActiveRecord.dump_schema_after_migration = Rails.env.development? &&
1094
+ `git status db/migrate/ --porcelain`.present?
1095
+ end
1096
+
1097
+ task "db:migrate" => "faster_migrations"
900
1098
  ```
901
1099
 
902
1100
  ## Schema Sanity
903
1101
 
904
- Columns can flip order in `db/schema.rb` when you have multiple developers. One way to prevent this is to [alphabetize them](https://www.pgrs.net/2008/03/12/alphabetize-schema-rb-columns/). Add to `config/initializers/strong_migrations.rb`:
1102
+ With Active Record < 8.1, columns can flip order in `db/schema.rb` when you have multiple developers. One way to prevent this is to [alphabetize them](https://www.pgrs.net/2008/03/12/alphabetize-schema-rb-columns/). Add to `config/initializers/strong_migrations.rb`:
905
1103
 
906
1104
  ```ruby
907
1105
  StrongMigrations.alphabetize_schema = true
@@ -917,15 +1115,18 @@ You probably don’t need this gem for smaller projects, as operations that are
917
1115
 
918
1116
  ## Additional Reading
919
1117
 
920
- - [Rails Migrations with No Downtime](https://pedro.herokuapp.com/past/2011/7/13/rails_migrations_with_no_downtime/)
921
1118
  - [PostgreSQL at Scale: Database Schema Changes Without Downtime](https://medium.com/braintree-product-technology/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680)
922
- - [An Overview of DDL Algorithms in MySQL](https://mydbops.wordpress.com/2020/03/04/an-overview-of-ddl-algorithms-in-mysql-covers-mysql-8/)
1119
+ - [MySQL InnoDB Online DDL Operations](https://dev.mysql.com/doc/refman/en/innodb-online-ddl-operations.html)
923
1120
  - [MariaDB InnoDB Online DDL Overview](https://mariadb.com/kb/en/innodb-online-ddl-overview/)
924
1121
 
925
1122
  ## Credits
926
1123
 
927
1124
  Thanks to Bob Remeika and David Waller for the [original code](https://github.com/foobarfighter/safe-migrations) and [Sean Huber](https://github.com/LendingHome/zero_downtime_migrations) for the bad/good readme format.
928
1125
 
1126
+ ## History
1127
+
1128
+ View the [changelog](https://github.com/ankane/strong_migrations/blob/master/CHANGELOG.md)
1129
+
929
1130
  ## Contributing
930
1131
 
931
1132
  Everyone is encouraged to help improve this project. Here are a few ways you can help: