strong_migrations 1.4.4 → 2.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +86 -1
- data/LICENSE.txt +1 -1
- data/README.md +285 -155
- data/lib/generators/strong_migrations/install_generator.rb +3 -7
- data/lib/generators/strong_migrations/templates/initializer.rb.tt +3 -0
- data/lib/strong_migrations/adapters/abstract_adapter.rb +13 -10
- data/lib/strong_migrations/adapters/mariadb_adapter.rb +2 -2
- data/lib/strong_migrations/adapters/mysql_adapter.rb +10 -5
- data/lib/strong_migrations/adapters/postgresql_adapter.rb +25 -28
- data/lib/strong_migrations/checker.rb +109 -19
- data/lib/strong_migrations/checks.rb +130 -85
- data/lib/strong_migrations/error_messages.rb +61 -14
- data/lib/strong_migrations/migration.rb +12 -6
- data/lib/strong_migrations/{database_tasks.rb → migration_context.rb} +20 -3
- data/lib/strong_migrations/migrator.rb +7 -5
- data/lib/strong_migrations/safe_methods.rb +60 -40
- data/lib/strong_migrations/schema_dumper.rb +15 -4
- data/lib/strong_migrations/version.rb +1 -1
- data/lib/strong_migrations.rb +9 -6
- metadata +7 -11
data/README.md
CHANGED
@@ -8,7 +8,7 @@ Supports PostgreSQL, MySQL, and MariaDB
|
|
8
8
|
|
9
9
|
:tangerine: Battle-tested at [Instacart](https://www.instacart.com/opensource)
|
10
10
|
|
11
|
-
[](https://github.com/ankane/strong_migrations/actions)
|
12
12
|
|
13
13
|
## Installation
|
14
14
|
|
@@ -38,12 +38,12 @@ Active Record caches attributes, which causes problems
|
|
38
38
|
when removing columns. Be sure to ignore the column:
|
39
39
|
|
40
40
|
class User < ApplicationRecord
|
41
|
-
self.ignored_columns
|
41
|
+
self.ignored_columns += ["name"]
|
42
42
|
end
|
43
43
|
|
44
44
|
Deploy the code, then wrap this step in a safety_assured { ... } block.
|
45
45
|
|
46
|
-
class RemoveColumn < ActiveRecord::Migration[
|
46
|
+
class RemoveColumn < ActiveRecord::Migration[8.0]
|
47
47
|
def change
|
48
48
|
safety_assured { remove_column :users, :name }
|
49
49
|
end
|
@@ -60,23 +60,31 @@ An operation is classified as dangerous if it either:
|
|
60
60
|
Potentially dangerous operations:
|
61
61
|
|
62
62
|
- [removing a column](#removing-a-column)
|
63
|
-
- [adding a column with a default value](#adding-a-column-with-a-default-value)
|
64
|
-
- [backfilling data](#backfilling-data)
|
65
63
|
- [changing the type of a column](#changing-the-type-of-a-column)
|
66
64
|
- [renaming a column](#renaming-a-column)
|
67
65
|
- [renaming a table](#renaming-a-table)
|
68
66
|
- [creating a table with the force option](#creating-a-table-with-the-force-option)
|
67
|
+
- [adding an auto-incrementing column](#adding-an-auto-incrementing-column)
|
68
|
+
- [adding a stored generated column](#adding-a-stored-generated-column)
|
69
69
|
- [adding a check constraint](#adding-a-check-constraint)
|
70
70
|
- [executing SQL directly](#executing-SQL-directly)
|
71
|
+
- [backfilling data](#backfilling-data)
|
71
72
|
|
72
73
|
Postgres-specific checks:
|
73
74
|
|
74
75
|
- [adding an index non-concurrently](#adding-an-index-non-concurrently)
|
75
76
|
- [adding a reference](#adding-a-reference)
|
76
77
|
- [adding a foreign key](#adding-a-foreign-key)
|
78
|
+
- [adding a unique constraint](#adding-a-unique-constraint)
|
77
79
|
- [adding an exclusion constraint](#adding-an-exclusion-constraint)
|
78
80
|
- [adding a json column](#adding-a-json-column)
|
79
81
|
- [setting NOT NULL on an existing column](#setting-not-null-on-an-existing-column)
|
82
|
+
- [adding a column with a volatile default value](#adding-a-column-with-a-volatile-default-value)
|
83
|
+
- [renaming a schema](#renaming-a-schema)
|
84
|
+
|
85
|
+
Config-specific checks:
|
86
|
+
|
87
|
+
- [changing the default value of a column](#changing-the-default-value-of-a-column)
|
80
88
|
|
81
89
|
Best practices:
|
82
90
|
|
@@ -91,7 +99,7 @@ You can also add [custom checks](#custom-checks) or [disable specific checks](#d
|
|
91
99
|
Active Record caches database columns at runtime, so if you drop a column, it can cause exceptions until your app reboots.
|
92
100
|
|
93
101
|
```ruby
|
94
|
-
class RemoveSomeColumnFromUsers < ActiveRecord::Migration[
|
102
|
+
class RemoveSomeColumnFromUsers < ActiveRecord::Migration[8.0]
|
95
103
|
def change
|
96
104
|
remove_column :users, :some_column
|
97
105
|
end
|
@@ -104,7 +112,7 @@ end
|
|
104
112
|
|
105
113
|
```ruby
|
106
114
|
class User < ApplicationRecord
|
107
|
-
self.ignored_columns
|
115
|
+
self.ignored_columns += ["some_column"]
|
108
116
|
end
|
109
117
|
```
|
110
118
|
|
@@ -112,7 +120,7 @@ end
|
|
112
120
|
3. Write a migration to remove the column (wrap in `safety_assured` block)
|
113
121
|
|
114
122
|
```ruby
|
115
|
-
class RemoveSomeColumnFromUsers < ActiveRecord::Migration[
|
123
|
+
class RemoveSomeColumnFromUsers < ActiveRecord::Migration[8.0]
|
116
124
|
def change
|
117
125
|
safety_assured { remove_column :users, :some_column }
|
118
126
|
end
|
@@ -122,75 +130,6 @@ end
|
|
122
130
|
4. Deploy and run the migration
|
123
131
|
5. Remove the line added in step 1
|
124
132
|
|
125
|
-
### Adding a column with a default value
|
126
|
-
|
127
|
-
#### Bad
|
128
|
-
|
129
|
-
In earlier versions of Postgres, MySQL, and MariaDB, adding a column with a default value to an existing table causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
|
130
|
-
|
131
|
-
```ruby
|
132
|
-
class AddSomeColumnToUsers < ActiveRecord::Migration[7.0]
|
133
|
-
def change
|
134
|
-
add_column :users, :some_column, :text, default: "default_value"
|
135
|
-
end
|
136
|
-
end
|
137
|
-
```
|
138
|
-
|
139
|
-
In Postgres 11+, MySQL 8.0.12+, and MariaDB 10.3.2+, this no longer requires a table rewrite and is safe (except for volatile functions like `gen_random_uuid()`).
|
140
|
-
|
141
|
-
#### Good
|
142
|
-
|
143
|
-
Instead, add the column without a default value, then change the default.
|
144
|
-
|
145
|
-
```ruby
|
146
|
-
class AddSomeColumnToUsers < ActiveRecord::Migration[7.0]
|
147
|
-
def up
|
148
|
-
add_column :users, :some_column, :text
|
149
|
-
change_column_default :users, :some_column, "default_value"
|
150
|
-
end
|
151
|
-
|
152
|
-
def down
|
153
|
-
remove_column :users, :some_column
|
154
|
-
end
|
155
|
-
end
|
156
|
-
```
|
157
|
-
|
158
|
-
See the next section for how to backfill.
|
159
|
-
|
160
|
-
### Backfilling data
|
161
|
-
|
162
|
-
#### Bad
|
163
|
-
|
164
|
-
Active Record creates a transaction around each migration, and backfilling in the same transaction that alters a table keeps the table locked for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
|
165
|
-
|
166
|
-
```ruby
|
167
|
-
class AddSomeColumnToUsers < ActiveRecord::Migration[7.0]
|
168
|
-
def change
|
169
|
-
add_column :users, :some_column, :text
|
170
|
-
User.update_all some_column: "default_value"
|
171
|
-
end
|
172
|
-
end
|
173
|
-
```
|
174
|
-
|
175
|
-
Also, running a single query to update data can cause issues for large tables.
|
176
|
-
|
177
|
-
#### Good
|
178
|
-
|
179
|
-
There are three keys to backfilling safely: batching, throttling, and running it outside a transaction. Use the Rails console or a separate migration with `disable_ddl_transaction!`.
|
180
|
-
|
181
|
-
```ruby
|
182
|
-
class BackfillSomeColumn < ActiveRecord::Migration[7.0]
|
183
|
-
disable_ddl_transaction!
|
184
|
-
|
185
|
-
def up
|
186
|
-
User.unscoped.in_batches do |relation|
|
187
|
-
relation.update_all some_column: "default_value"
|
188
|
-
sleep(0.01) # throttle
|
189
|
-
end
|
190
|
-
end
|
191
|
-
end
|
192
|
-
```
|
193
|
-
|
194
133
|
### Changing the type of a column
|
195
134
|
|
196
135
|
#### Bad
|
@@ -198,7 +137,7 @@ end
|
|
198
137
|
Changing the type of a column causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
|
199
138
|
|
200
139
|
```ruby
|
201
|
-
class ChangeSomeColumnType < ActiveRecord::Migration[
|
140
|
+
class ChangeSomeColumnType < ActiveRecord::Migration[8.0]
|
202
141
|
def change
|
203
142
|
change_column :users, :some_column, :new_type
|
204
143
|
end
|
@@ -244,7 +183,7 @@ A safer approach is to:
|
|
244
183
|
Renaming a column that’s in use will cause errors in your application.
|
245
184
|
|
246
185
|
```ruby
|
247
|
-
class RenameSomeColumn < ActiveRecord::Migration[
|
186
|
+
class RenameSomeColumn < ActiveRecord::Migration[8.0]
|
248
187
|
def change
|
249
188
|
rename_column :users, :some_column, :new_name
|
250
189
|
end
|
@@ -269,7 +208,7 @@ A safer approach is to:
|
|
269
208
|
Renaming a table that’s in use will cause errors in your application.
|
270
209
|
|
271
210
|
```ruby
|
272
|
-
class RenameUsersToCustomers < ActiveRecord::Migration[
|
211
|
+
class RenameUsersToCustomers < ActiveRecord::Migration[8.0]
|
273
212
|
def change
|
274
213
|
rename_table :users, :customers
|
275
214
|
end
|
@@ -282,7 +221,7 @@ A safer approach is to:
|
|
282
221
|
|
283
222
|
1. Create a new table
|
284
223
|
2. Write to both tables
|
285
|
-
3. Backfill data from the old table to new table
|
224
|
+
3. Backfill data from the old table to the new table
|
286
225
|
4. Move reads from the old table to the new table
|
287
226
|
5. Stop writing to the old table
|
288
227
|
6. Drop the old table
|
@@ -294,7 +233,7 @@ A safer approach is to:
|
|
294
233
|
The `force` option can drop an existing table.
|
295
234
|
|
296
235
|
```ruby
|
297
|
-
class CreateUsers < ActiveRecord::Migration[
|
236
|
+
class CreateUsers < ActiveRecord::Migration[8.0]
|
298
237
|
def change
|
299
238
|
create_table :users, force: true do |t|
|
300
239
|
# ...
|
@@ -308,7 +247,7 @@ end
|
|
308
247
|
Create tables without the `force` option.
|
309
248
|
|
310
249
|
```ruby
|
311
|
-
class CreateUsers < ActiveRecord::Migration[
|
250
|
+
class CreateUsers < ActiveRecord::Migration[8.0]
|
312
251
|
def change
|
313
252
|
create_table :users do |t|
|
314
253
|
# ...
|
@@ -319,6 +258,44 @@ end
|
|
319
258
|
|
320
259
|
If you intend to drop an existing table, run `drop_table` first.
|
321
260
|
|
261
|
+
### Adding an auto-incrementing column
|
262
|
+
|
263
|
+
#### Bad
|
264
|
+
|
265
|
+
Adding an auto-incrementing column (`serial`/`bigserial` in Postgres and `AUTO_INCREMENT` in MySQL and MariaDB) causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
|
266
|
+
|
267
|
+
```ruby
|
268
|
+
class AddIdToCitiesUsers < ActiveRecord::Migration[8.0]
|
269
|
+
def change
|
270
|
+
add_column :cities_users, :id, :primary_key
|
271
|
+
end
|
272
|
+
end
|
273
|
+
```
|
274
|
+
|
275
|
+
With MySQL and MariaDB, this can also [generate different values on replicas](https://dev.mysql.com/doc/mysql-replication-excerpt/8.0/en/replication-features-auto-increment.html) if using statement-based replication.
|
276
|
+
|
277
|
+
#### Good
|
278
|
+
|
279
|
+
Create a new table and migrate the data with the same steps as [renaming a table](#renaming-a-table).
|
280
|
+
|
281
|
+
### Adding a stored generated column
|
282
|
+
|
283
|
+
#### Bad
|
284
|
+
|
285
|
+
Adding a stored generated column causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
|
286
|
+
|
287
|
+
```ruby
|
288
|
+
class AddSomeColumnToUsers < ActiveRecord::Migration[8.0]
|
289
|
+
def change
|
290
|
+
add_column :users, :some_column, :virtual, type: :string, as: "...", stored: true
|
291
|
+
end
|
292
|
+
end
|
293
|
+
```
|
294
|
+
|
295
|
+
#### Good
|
296
|
+
|
297
|
+
Add a non-generated column and use callbacks or triggers instead (or a virtual generated column with MySQL and MariaDB).
|
298
|
+
|
322
299
|
### Adding a check constraint
|
323
300
|
|
324
301
|
:turtle: Safe by default available
|
@@ -328,7 +305,7 @@ If you intend to drop an existing table, run `drop_table` first.
|
|
328
305
|
Adding a check constraint blocks reads and writes in Postgres and blocks writes in MySQL and MariaDB while every row is checked.
|
329
306
|
|
330
307
|
```ruby
|
331
|
-
class AddCheckConstraint < ActiveRecord::Migration[
|
308
|
+
class AddCheckConstraint < ActiveRecord::Migration[8.0]
|
332
309
|
def change
|
333
310
|
add_check_constraint :users, "price > 0", name: "price_check"
|
334
311
|
end
|
@@ -340,7 +317,7 @@ end
|
|
340
317
|
Add the check constraint without validating existing rows:
|
341
318
|
|
342
319
|
```ruby
|
343
|
-
class AddCheckConstraint < ActiveRecord::Migration[
|
320
|
+
class AddCheckConstraint < ActiveRecord::Migration[8.0]
|
344
321
|
def change
|
345
322
|
add_check_constraint :users, "price > 0", name: "price_check", validate: false
|
346
323
|
end
|
@@ -350,7 +327,7 @@ end
|
|
350
327
|
Then validate them in a separate migration.
|
351
328
|
|
352
329
|
```ruby
|
353
|
-
class ValidateCheckConstraint < ActiveRecord::Migration[
|
330
|
+
class ValidateCheckConstraint < ActiveRecord::Migration[8.0]
|
354
331
|
def change
|
355
332
|
validate_check_constraint :users, name: "price_check"
|
356
333
|
end
|
@@ -366,13 +343,51 @@ end
|
|
366
343
|
Strong Migrations can’t ensure safety for raw SQL statements. Make really sure that what you’re doing is safe, then use:
|
367
344
|
|
368
345
|
```ruby
|
369
|
-
class ExecuteSQL < ActiveRecord::Migration[
|
346
|
+
class ExecuteSQL < ActiveRecord::Migration[8.0]
|
370
347
|
def change
|
371
348
|
safety_assured { execute "..." }
|
372
349
|
end
|
373
350
|
end
|
374
351
|
```
|
375
352
|
|
353
|
+
### Backfilling data
|
354
|
+
|
355
|
+
Note: Strong Migrations does not detect dangerous backfills.
|
356
|
+
|
357
|
+
#### Bad
|
358
|
+
|
359
|
+
Active Record creates a transaction around each migration, and backfilling in the same transaction that alters a table keeps the table locked for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
|
360
|
+
|
361
|
+
```ruby
|
362
|
+
class AddSomeColumnToUsers < ActiveRecord::Migration[8.0]
|
363
|
+
def change
|
364
|
+
add_column :users, :some_column, :text
|
365
|
+
User.update_all some_column: "default_value"
|
366
|
+
end
|
367
|
+
end
|
368
|
+
```
|
369
|
+
|
370
|
+
Also, running a single query to update data can cause issues for large tables.
|
371
|
+
|
372
|
+
#### Good
|
373
|
+
|
374
|
+
There are three keys to backfilling safely: batching, throttling, and running it outside a transaction. Use the Rails console or a separate migration with `disable_ddl_transaction!`.
|
375
|
+
|
376
|
+
```ruby
|
377
|
+
class BackfillSomeColumn < ActiveRecord::Migration[8.0]
|
378
|
+
disable_ddl_transaction!
|
379
|
+
|
380
|
+
def up
|
381
|
+
User.unscoped.in_batches(of: 10000) do |relation|
|
382
|
+
relation.where(some_column: nil).update_all some_column: "default_value"
|
383
|
+
sleep(0.01) # throttle
|
384
|
+
end
|
385
|
+
end
|
386
|
+
end
|
387
|
+
```
|
388
|
+
|
389
|
+
Note: If backfilling with a method other than `update_all`, use `User.reset_column_information` to ensure the model has up-to-date column information.
|
390
|
+
|
376
391
|
### Adding an index non-concurrently
|
377
392
|
|
378
393
|
:turtle: Safe by default available
|
@@ -382,7 +397,7 @@ end
|
|
382
397
|
In Postgres, adding an index non-concurrently blocks writes.
|
383
398
|
|
384
399
|
```ruby
|
385
|
-
class AddSomeIndexToUsers < ActiveRecord::Migration[
|
400
|
+
class AddSomeIndexToUsers < ActiveRecord::Migration[8.0]
|
386
401
|
def change
|
387
402
|
add_index :users, :some_column
|
388
403
|
end
|
@@ -394,7 +409,7 @@ end
|
|
394
409
|
Add indexes concurrently.
|
395
410
|
|
396
411
|
```ruby
|
397
|
-
class AddSomeIndexToUsers < ActiveRecord::Migration[
|
412
|
+
class AddSomeIndexToUsers < ActiveRecord::Migration[8.0]
|
398
413
|
disable_ddl_transaction!
|
399
414
|
|
400
415
|
def change
|
@@ -420,7 +435,7 @@ rails g index table column
|
|
420
435
|
Rails adds an index non-concurrently to references by default, which blocks writes in Postgres.
|
421
436
|
|
422
437
|
```ruby
|
423
|
-
class AddReferenceToUsers < ActiveRecord::Migration[
|
438
|
+
class AddReferenceToUsers < ActiveRecord::Migration[8.0]
|
424
439
|
def change
|
425
440
|
add_reference :users, :city
|
426
441
|
end
|
@@ -432,7 +447,7 @@ end
|
|
432
447
|
Make sure the index is added concurrently.
|
433
448
|
|
434
449
|
```ruby
|
435
|
-
class AddReferenceToUsers < ActiveRecord::Migration[
|
450
|
+
class AddReferenceToUsers < ActiveRecord::Migration[8.0]
|
436
451
|
disable_ddl_transaction!
|
437
452
|
|
438
453
|
def change
|
@@ -450,7 +465,7 @@ end
|
|
450
465
|
In Postgres, adding a foreign key blocks writes on both tables.
|
451
466
|
|
452
467
|
```ruby
|
453
|
-
class AddForeignKeyOnUsers < ActiveRecord::Migration[
|
468
|
+
class AddForeignKeyOnUsers < ActiveRecord::Migration[8.0]
|
454
469
|
def change
|
455
470
|
add_foreign_key :users, :orders
|
456
471
|
end
|
@@ -460,7 +475,7 @@ end
|
|
460
475
|
or
|
461
476
|
|
462
477
|
```ruby
|
463
|
-
class AddReferenceToUsers < ActiveRecord::Migration[
|
478
|
+
class AddReferenceToUsers < ActiveRecord::Migration[8.0]
|
464
479
|
def change
|
465
480
|
add_reference :users, :order, foreign_key: true
|
466
481
|
end
|
@@ -472,7 +487,7 @@ end
|
|
472
487
|
Add the foreign key without validating existing rows:
|
473
488
|
|
474
489
|
```ruby
|
475
|
-
class AddForeignKeyOnUsers < ActiveRecord::Migration[
|
490
|
+
class AddForeignKeyOnUsers < ActiveRecord::Migration[8.0]
|
476
491
|
def change
|
477
492
|
add_foreign_key :users, :orders, validate: false
|
478
493
|
end
|
@@ -482,13 +497,46 @@ end
|
|
482
497
|
Then validate them in a separate migration.
|
483
498
|
|
484
499
|
```ruby
|
485
|
-
class ValidateForeignKeyOnUsers < ActiveRecord::Migration[
|
500
|
+
class ValidateForeignKeyOnUsers < ActiveRecord::Migration[8.0]
|
486
501
|
def change
|
487
502
|
validate_foreign_key :users, :orders
|
488
503
|
end
|
489
504
|
end
|
490
505
|
```
|
491
506
|
|
507
|
+
### Adding a unique constraint
|
508
|
+
|
509
|
+
#### Bad
|
510
|
+
|
511
|
+
In Postgres, adding a unique constraint creates a unique index, which blocks reads and writes.
|
512
|
+
|
513
|
+
```ruby
|
514
|
+
class AddUniqueConstraint < ActiveRecord::Migration[8.0]
|
515
|
+
def change
|
516
|
+
add_unique_constraint :users, :some_column
|
517
|
+
end
|
518
|
+
end
|
519
|
+
```
|
520
|
+
|
521
|
+
#### Good
|
522
|
+
|
523
|
+
Create a unique index concurrently, then use it for the constraint.
|
524
|
+
|
525
|
+
```ruby
|
526
|
+
class AddUniqueConstraint < ActiveRecord::Migration[8.0]
|
527
|
+
disable_ddl_transaction!
|
528
|
+
|
529
|
+
def up
|
530
|
+
add_index :users, :some_column, unique: true, algorithm: :concurrently
|
531
|
+
add_unique_constraint :users, using_index: "index_users_on_some_column"
|
532
|
+
end
|
533
|
+
|
534
|
+
def down
|
535
|
+
remove_unique_constraint :users, :some_column
|
536
|
+
end
|
537
|
+
end
|
538
|
+
```
|
539
|
+
|
492
540
|
### Adding an exclusion constraint
|
493
541
|
|
494
542
|
#### Bad
|
@@ -496,7 +544,7 @@ end
|
|
496
544
|
In Postgres, adding an exclusion constraint blocks reads and writes while every row is checked.
|
497
545
|
|
498
546
|
```ruby
|
499
|
-
class
|
547
|
+
class AddExclusionConstraint < ActiveRecord::Migration[8.0]
|
500
548
|
def change
|
501
549
|
add_exclusion_constraint :users, "number WITH =", using: :gist
|
502
550
|
end
|
@@ -514,7 +562,7 @@ end
|
|
514
562
|
In Postgres, there’s no equality operator for the `json` column type, which can cause errors for existing `SELECT DISTINCT` queries in your application.
|
515
563
|
|
516
564
|
```ruby
|
517
|
-
class AddPropertiesToUsers < ActiveRecord::Migration[
|
565
|
+
class AddPropertiesToUsers < ActiveRecord::Migration[8.0]
|
518
566
|
def change
|
519
567
|
add_column :users, :properties, :json
|
520
568
|
end
|
@@ -526,7 +574,7 @@ end
|
|
526
574
|
Use `jsonb` instead.
|
527
575
|
|
528
576
|
```ruby
|
529
|
-
class AddPropertiesToUsers < ActiveRecord::Migration[
|
577
|
+
class AddPropertiesToUsers < ActiveRecord::Migration[8.0]
|
530
578
|
def change
|
531
579
|
add_column :users, :properties, :jsonb
|
532
580
|
end
|
@@ -542,7 +590,7 @@ end
|
|
542
590
|
In Postgres, setting `NOT NULL` on an existing column blocks reads and writes while every row is checked.
|
543
591
|
|
544
592
|
```ruby
|
545
|
-
class SetSomeColumnNotNull < ActiveRecord::Migration[
|
593
|
+
class SetSomeColumnNotNull < ActiveRecord::Migration[8.0]
|
546
594
|
def change
|
547
595
|
change_column_null :users, :some_column, false
|
548
596
|
end
|
@@ -553,60 +601,117 @@ end
|
|
553
601
|
|
554
602
|
Instead, add a check constraint.
|
555
603
|
|
556
|
-
For Rails 6.1, use:
|
557
|
-
|
558
604
|
```ruby
|
559
|
-
class SetSomeColumnNotNull < ActiveRecord::Migration[
|
605
|
+
class SetSomeColumnNotNull < ActiveRecord::Migration[8.0]
|
560
606
|
def change
|
561
607
|
add_check_constraint :users, "some_column IS NOT NULL", name: "users_some_column_null", validate: false
|
562
608
|
end
|
563
609
|
end
|
564
610
|
```
|
565
611
|
|
566
|
-
|
612
|
+
Then validate it in a separate migration. Once the check constraint is validated, you can safely set `NOT NULL` on the column and drop the check constraint.
|
567
613
|
|
568
614
|
```ruby
|
569
|
-
class
|
570
|
-
def
|
571
|
-
|
572
|
-
|
573
|
-
|
615
|
+
class ValidateSomeColumnNotNull < ActiveRecord::Migration[8.0]
|
616
|
+
def up
|
617
|
+
validate_check_constraint :users, name: "users_some_column_null"
|
618
|
+
change_column_null :users, :some_column, false
|
619
|
+
remove_check_constraint :users, name: "users_some_column_null"
|
620
|
+
end
|
621
|
+
|
622
|
+
def down
|
623
|
+
add_check_constraint :users, "some_column IS NOT NULL", name: "users_some_column_null", validate: false
|
624
|
+
change_column_null :users, :some_column, true
|
574
625
|
end
|
575
626
|
end
|
576
627
|
```
|
577
628
|
|
578
|
-
|
629
|
+
### Adding a column with a volatile default value
|
579
630
|
|
580
|
-
|
631
|
+
#### Bad
|
632
|
+
|
633
|
+
Adding a column with a volatile default value to an existing table causes the entire table to be rewritten. During this time, reads and writes are blocked.
|
581
634
|
|
582
635
|
```ruby
|
583
|
-
class
|
636
|
+
class AddSomeColumnToUsers < ActiveRecord::Migration[8.0]
|
584
637
|
def change
|
585
|
-
|
638
|
+
add_column :users, :some_column, :uuid, default: "gen_random_uuid()"
|
639
|
+
end
|
640
|
+
end
|
641
|
+
```
|
586
642
|
|
587
|
-
|
588
|
-
|
589
|
-
|
643
|
+
#### Good
|
644
|
+
|
645
|
+
Instead, add the column without a default value, then change the default.
|
646
|
+
|
647
|
+
```ruby
|
648
|
+
class AddSomeColumnToUsers < ActiveRecord::Migration[8.0]
|
649
|
+
def up
|
650
|
+
add_column :users, :some_column, :uuid
|
651
|
+
change_column_default :users, :some_column, from: nil, to: "gen_random_uuid()"
|
652
|
+
end
|
653
|
+
|
654
|
+
def down
|
655
|
+
remove_column :users, :some_column
|
590
656
|
end
|
591
657
|
end
|
592
658
|
```
|
593
659
|
|
594
|
-
|
660
|
+
Then [backfill the data](#backfilling-data).
|
661
|
+
|
662
|
+
### Renaming a schema
|
663
|
+
|
664
|
+
#### Bad
|
665
|
+
|
666
|
+
Renaming a schema that’s in use will cause errors in your application.
|
595
667
|
|
596
668
|
```ruby
|
597
|
-
class
|
669
|
+
class RenameUsersToCustomers < ActiveRecord::Migration[8.1]
|
598
670
|
def change
|
599
|
-
|
600
|
-
|
601
|
-
|
671
|
+
rename_schema :users, :customers
|
672
|
+
end
|
673
|
+
end
|
674
|
+
```
|
602
675
|
|
603
|
-
|
604
|
-
|
605
|
-
|
606
|
-
|
607
|
-
|
676
|
+
#### Good
|
677
|
+
|
678
|
+
A safer approach is to:
|
679
|
+
|
680
|
+
1. Create a new schema
|
681
|
+
2. Write to both schemas
|
682
|
+
3. Backfill data from the old schema to the new schema
|
683
|
+
4. Move reads from the old schema to the new schema
|
684
|
+
5. Stop writing to the old schema
|
685
|
+
6. Drop the old schema
|
686
|
+
|
687
|
+
### Changing the default value of a column
|
688
|
+
|
689
|
+
#### Bad
|
690
|
+
|
691
|
+
Rails < 7 enables partial writes by default, which can cause incorrect values to be inserted when changing the default value of a column.
|
692
|
+
|
693
|
+
```ruby
|
694
|
+
class ChangeSomeColumnDefault < ActiveRecord::Migration[6.1]
|
695
|
+
def change
|
696
|
+
change_column_default :users, :some_column, from: "old", to: "new"
|
608
697
|
end
|
609
698
|
end
|
699
|
+
|
700
|
+
User.create!(some_column: "old") # can insert "new"
|
701
|
+
```
|
702
|
+
|
703
|
+
#### Good
|
704
|
+
|
705
|
+
Disable partial writes in `config/application.rb`. For Rails < 7, use:
|
706
|
+
|
707
|
+
```ruby
|
708
|
+
config.active_record.partial_writes = false
|
709
|
+
```
|
710
|
+
|
711
|
+
For Rails 7+, use:
|
712
|
+
|
713
|
+
```ruby
|
714
|
+
config.active_record.partial_inserts = false
|
610
715
|
```
|
611
716
|
|
612
717
|
### Keeping non-unique indexes to three columns or less
|
@@ -616,7 +721,7 @@ end
|
|
616
721
|
Adding a non-unique index with more than three columns rarely improves performance.
|
617
722
|
|
618
723
|
```ruby
|
619
|
-
class AddSomeIndexToUsers < ActiveRecord::Migration[
|
724
|
+
class AddSomeIndexToUsers < ActiveRecord::Migration[8.0]
|
620
725
|
def change
|
621
726
|
add_index :users, [:a, :b, :c, :d]
|
622
727
|
end
|
@@ -628,9 +733,9 @@ end
|
|
628
733
|
Instead, start an index with columns that narrow down the results the most.
|
629
734
|
|
630
735
|
```ruby
|
631
|
-
class AddSomeIndexToUsers < ActiveRecord::Migration[
|
736
|
+
class AddSomeIndexToUsers < ActiveRecord::Migration[8.0]
|
632
737
|
def change
|
633
|
-
add_index :users, [:
|
738
|
+
add_index :users, [:d, :b]
|
634
739
|
end
|
635
740
|
end
|
636
741
|
```
|
@@ -642,7 +747,7 @@ For Postgres, be sure to add them concurrently.
|
|
642
747
|
To mark a step in the migration as safe, despite using a method that might otherwise be dangerous, wrap it in a `safety_assured` block.
|
643
748
|
|
644
749
|
```ruby
|
645
|
-
class MySafeMigration < ActiveRecord::Migration[
|
750
|
+
class MySafeMigration < ActiveRecord::Migration[8.0]
|
646
751
|
def change
|
647
752
|
safety_assured { remove_column :users, :some_column }
|
648
753
|
end
|
@@ -653,7 +758,7 @@ Certain methods like `execute` and `change_table` cannot be inspected and are pr
|
|
653
758
|
|
654
759
|
## Safe by Default
|
655
760
|
|
656
|
-
Make operations safe by default.
|
761
|
+
Make certain operations safe by default. This allows you to write the code under the "Bad" section, but the migration will be performed as if you had written the "Good" version.
|
657
762
|
|
658
763
|
- adding and removing an index
|
659
764
|
- adding a foreign key
|
@@ -702,6 +807,16 @@ StrongMigrations.disable_check(:add_index)
|
|
702
807
|
|
703
808
|
Check the [source code](https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations/error_messages.rb) for the list of keys.
|
704
809
|
|
810
|
+
## Skip Databases
|
811
|
+
|
812
|
+
Skip checks and other functionality for specific databases with:
|
813
|
+
|
814
|
+
```ruby
|
815
|
+
StrongMigrations.skip_database(:catalog)
|
816
|
+
```
|
817
|
+
|
818
|
+
Note: This does not affect `alphabetize_schema`.
|
819
|
+
|
705
820
|
## Down Migrations / Rollbacks
|
706
821
|
|
707
822
|
By default, checks are disabled when migrating down. Enable them with:
|
@@ -740,25 +855,6 @@ ALTER ROLE myuser SET statement_timeout = '1h';
|
|
740
855
|
|
741
856
|
Note: If you use PgBouncer in transaction mode, you must set timeouts on the database user.
|
742
857
|
|
743
|
-
## Lock Timeout Retries [experimental]
|
744
|
-
|
745
|
-
There’s the option to automatically retry statements when the lock timeout is reached. Here’s how it works:
|
746
|
-
|
747
|
-
- If a lock timeout happens outside a transaction, the statement is retried
|
748
|
-
- If it happens inside the DDL transaction, the entire migration is retried (only applicable to Postgres)
|
749
|
-
|
750
|
-
Add to `config/initializers/strong_migrations.rb`:
|
751
|
-
|
752
|
-
```ruby
|
753
|
-
StrongMigrations.lock_timeout_retries = 3
|
754
|
-
```
|
755
|
-
|
756
|
-
Set the delay between retries with:
|
757
|
-
|
758
|
-
```ruby
|
759
|
-
StrongMigrations.lock_timeout_retry_delay = 10.seconds
|
760
|
-
```
|
761
|
-
|
762
858
|
## App Timeouts
|
763
859
|
|
764
860
|
We recommend adding timeouts to `config/database.yml` to prevent connections from hanging and individual queries from taking up too many resources in controllers, jobs, the Rails console, and other places.
|
@@ -802,12 +898,43 @@ production:
|
|
802
898
|
|
803
899
|
For HTTP connections, Redis, and other services, check out [this guide](https://github.com/ankane/the-ultimate-guide-to-ruby-timeouts).
|
804
900
|
|
901
|
+
## Invalid Indexes
|
902
|
+
|
903
|
+
In Postgres, adding an index non-concurrently can leave behind an invalid index if the lock timeout is reached. Running the migration again can result in an error.
|
904
|
+
|
905
|
+
To automatically remove the invalid index when the migration runs again, use:
|
906
|
+
|
907
|
+
```ruby
|
908
|
+
StrongMigrations.remove_invalid_indexes = true
|
909
|
+
```
|
910
|
+
|
911
|
+
## Lock Timeout Retries
|
912
|
+
|
913
|
+
Note: This feature is experimental.
|
914
|
+
|
915
|
+
There’s the option to automatically retry statements for migrations when the lock timeout is reached. Here’s how it works:
|
916
|
+
|
917
|
+
- If a lock timeout happens outside a transaction, the statement is retried
|
918
|
+
- If it happens inside the DDL transaction, the entire migration is retried (only applicable to Postgres)
|
919
|
+
|
920
|
+
Add to `config/initializers/strong_migrations.rb`:
|
921
|
+
|
922
|
+
```ruby
|
923
|
+
StrongMigrations.lock_timeout_retries = 3
|
924
|
+
```
|
925
|
+
|
926
|
+
Set the delay between retries with:
|
927
|
+
|
928
|
+
```ruby
|
929
|
+
StrongMigrations.lock_timeout_retry_delay = 10.seconds
|
930
|
+
```
|
931
|
+
|
805
932
|
## Existing Migrations
|
806
933
|
|
807
934
|
To mark migrations as safe that were created before installing this gem, create an initializer with:
|
808
935
|
|
809
936
|
```ruby
|
810
|
-
StrongMigrations.start_after =
|
937
|
+
StrongMigrations.start_after = 20250101000000
|
811
938
|
```
|
812
939
|
|
813
940
|
Use the version from your latest migration.
|
@@ -817,14 +944,14 @@ Use the version from your latest migration.
|
|
817
944
|
If your development database version is different from production, you can specify the production version so the right checks run in development.
|
818
945
|
|
819
946
|
```ruby
|
820
|
-
StrongMigrations.target_version = 10 # or
|
947
|
+
StrongMigrations.target_version = 10 # or 8.0, 10.5, etc
|
821
948
|
```
|
822
949
|
|
823
|
-
The major version works well for Postgres, while the
|
950
|
+
The major version works well for Postgres, while the major and minor version is recommended for MySQL and MariaDB.
|
824
951
|
|
825
952
|
For safety, this option only affects development and test environments. In other environments, the actual server version is always used.
|
826
953
|
|
827
|
-
If your app has multiple databases with different versions,
|
954
|
+
If your app has multiple databases with different versions, you can use:
|
828
955
|
|
829
956
|
```ruby
|
830
957
|
StrongMigrations.target_version = {primary: 13, catalog: 15}
|
@@ -864,15 +991,18 @@ You probably don’t need this gem for smaller projects, as operations that are
|
|
864
991
|
|
865
992
|
## Additional Reading
|
866
993
|
|
867
|
-
- [Rails Migrations with No Downtime](https://pedro.herokuapp.com/past/2011/7/13/rails_migrations_with_no_downtime/)
|
868
994
|
- [PostgreSQL at Scale: Database Schema Changes Without Downtime](https://medium.com/braintree-product-technology/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680)
|
869
|
-
- [
|
995
|
+
- [MySQL InnoDB Online DDL Operations](https://dev.mysql.com/doc/refman/en/innodb-online-ddl-operations.html)
|
870
996
|
- [MariaDB InnoDB Online DDL Overview](https://mariadb.com/kb/en/innodb-online-ddl-overview/)
|
871
997
|
|
872
998
|
## Credits
|
873
999
|
|
874
1000
|
Thanks to Bob Remeika and David Waller for the [original code](https://github.com/foobarfighter/safe-migrations) and [Sean Huber](https://github.com/LendingHome/zero_downtime_migrations) for the bad/good readme format.
|
875
1001
|
|
1002
|
+
## History
|
1003
|
+
|
1004
|
+
View the [changelog](https://github.com/ankane/strong_migrations/blob/master/CHANGELOG.md)
|
1005
|
+
|
876
1006
|
## Contributing
|
877
1007
|
|
878
1008
|
Everyone is encouraged to help improve this project. Here are a few ways you can help:
|