strong_migrations 0.6.8 → 1.6.1

Sign up to get free protection for your applications and to get access to all the features.
data/README.md CHANGED
@@ -4,18 +4,18 @@ Catch unsafe migrations in development
4
4
 
5
5
  &nbsp;&nbsp;✓&nbsp;&nbsp;Detects potentially dangerous operations<br />&nbsp;&nbsp;✓&nbsp;&nbsp;Prevents them from running by default<br />&nbsp;&nbsp;✓&nbsp;&nbsp;Provides instructions on safer ways to do what you want
6
6
 
7
- Supports for PostgreSQL, MySQL, and MariaDB
7
+ Supports PostgreSQL, MySQL, and MariaDB
8
8
 
9
9
  :tangerine: Battle-tested at [Instacart](https://www.instacart.com/opensource)
10
10
 
11
- [![Build Status](https://travis-ci.org/ankane/strong_migrations.svg?branch=master)](https://travis-ci.org/ankane/strong_migrations)
11
+ [![Build Status](https://github.com/ankane/strong_migrations/workflows/build/badge.svg?branch=master)](https://github.com/ankane/strong_migrations/actions)
12
12
 
13
13
  ## Installation
14
14
 
15
15
  Add this line to your application’s Gemfile:
16
16
 
17
17
  ```ruby
18
- gem 'strong_migrations'
18
+ gem "strong_migrations"
19
19
  ```
20
20
 
21
21
  And run:
@@ -25,6 +25,36 @@ bundle install
25
25
  rails generate strong_migrations:install
26
26
  ```
27
27
 
28
+ Strong Migrations sets a long statement timeout for migrations so you can set a [short statement timeout](#app-timeouts) for your application.
29
+
30
+ ## How It Works
31
+
32
+ When you run a migration that’s potentially dangerous, you’ll see an error message like:
33
+
34
+ ```txt
35
+ === Dangerous operation detected #strong_migrations ===
36
+
37
+ Active Record caches attributes, which causes problems
38
+ when removing columns. Be sure to ignore the column:
39
+
40
+ class User < ApplicationRecord
41
+ self.ignored_columns = ["name"]
42
+ end
43
+
44
+ Deploy the code, then wrap this step in a safety_assured { ... } block.
45
+
46
+ class RemoveColumn < ActiveRecord::Migration[7.0]
47
+ def change
48
+ safety_assured { remove_column :users, :name }
49
+ end
50
+ end
51
+ ```
52
+
53
+ An operation is classified as dangerous if it either:
54
+
55
+ - Blocks reads or writes for more than a few seconds (after a lock is acquired)
56
+ - Has a good chance of causing application errors
57
+
28
58
  ## Checks
29
59
 
30
60
  Potentially dangerous operations:
@@ -32,11 +62,12 @@ Potentially dangerous operations:
32
62
  - [removing a column](#removing-a-column)
33
63
  - [adding a column with a default value](#adding-a-column-with-a-default-value)
34
64
  - [backfilling data](#backfilling-data)
35
- - [changing the type of a column](#renaming-or-changing-the-type-of-a-column)
36
- - [renaming a column](#renaming-or-changing-the-type-of-a-column)
65
+ - [adding a stored generated column](#adding-a-stored-generated-column)
66
+ - [changing the type of a column](#changing-the-type-of-a-column)
67
+ - [renaming a column](#renaming-a-column)
37
68
  - [renaming a table](#renaming-a-table)
38
69
  - [creating a table with the force option](#creating-a-table-with-the-force-option)
39
- - [using change_column_null with a default value](#using-change_column_null-with-a-default-value)
70
+ - [adding a check constraint](#adding-a-check-constraint)
40
71
  - [executing SQL directly](#executing-SQL-directly)
41
72
 
42
73
  Postgres-specific checks:
@@ -44,9 +75,14 @@ Postgres-specific checks:
44
75
  - [adding an index non-concurrently](#adding-an-index-non-concurrently)
45
76
  - [adding a reference](#adding-a-reference)
46
77
  - [adding a foreign key](#adding-a-foreign-key)
78
+ - [adding an exclusion constraint](#adding-an-exclusion-constraint)
47
79
  - [adding a json column](#adding-a-json-column)
48
80
  - [setting NOT NULL on an existing column](#setting-not-null-on-an-existing-column)
49
81
 
82
+ Config-specific checks:
83
+
84
+ - [changing the default value of a column](#changing-the-default-value-of-a-column)
85
+
50
86
  Best practices:
51
87
 
52
88
  - [keeping non-unique indexes to three columns or less](#keeping-non-unique-indexes-to-three-columns-or-less)
@@ -57,10 +93,10 @@ You can also add [custom checks](#custom-checks) or [disable specific checks](#d
57
93
 
58
94
  #### Bad
59
95
 
60
- ActiveRecord caches database columns at runtime, so if you drop a column, it can cause exceptions until your app reboots.
96
+ Active Record caches database columns at runtime, so if you drop a column, it can cause exceptions until your app reboots.
61
97
 
62
98
  ```ruby
63
- class RemoveSomeColumnFromUsers < ActiveRecord::Migration[6.0]
99
+ class RemoveSomeColumnFromUsers < ActiveRecord::Migration[7.0]
64
100
  def change
65
101
  remove_column :users, :some_column
66
102
  end
@@ -69,7 +105,7 @@ end
69
105
 
70
106
  #### Good
71
107
 
72
- 1. Tell ActiveRecord to ignore the column from its cache
108
+ 1. Tell Active Record to ignore the column from its cache
73
109
 
74
110
  ```ruby
75
111
  class User < ApplicationRecord
@@ -77,41 +113,42 @@ end
77
113
  end
78
114
  ```
79
115
 
80
- 2. Deploy code
116
+ 2. Deploy the code
81
117
  3. Write a migration to remove the column (wrap in `safety_assured` block)
82
118
 
83
119
  ```ruby
84
- class RemoveSomeColumnFromUsers < ActiveRecord::Migration[6.0]
120
+ class RemoveSomeColumnFromUsers < ActiveRecord::Migration[7.0]
85
121
  def change
86
122
  safety_assured { remove_column :users, :some_column }
87
123
  end
88
124
  end
89
125
  ```
90
126
 
91
- 4. Deploy and run migration
127
+ 4. Deploy and run the migration
128
+ 5. Remove the line added in step 1
92
129
 
93
130
  ### Adding a column with a default value
94
131
 
95
- Note: This operation is safe in Postgres 11+, MySQL 8.0.12+, and MariaDB 10.3.2+.
96
-
97
132
  #### Bad
98
133
 
99
- Adding a column with a default value to an existing table causes the entire table to be rewritten.
134
+ In earlier versions of Postgres, MySQL, and MariaDB, adding a column with a default value to an existing table causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
100
135
 
101
136
  ```ruby
102
- class AddSomeColumnToUsers < ActiveRecord::Migration[6.0]
137
+ class AddSomeColumnToUsers < ActiveRecord::Migration[7.0]
103
138
  def change
104
139
  add_column :users, :some_column, :text, default: "default_value"
105
140
  end
106
141
  end
107
142
  ```
108
143
 
144
+ In Postgres 11+, MySQL 8.0.12+, and MariaDB 10.3.2+, this no longer requires a table rewrite and is safe (except for volatile functions like `gen_random_uuid()`).
145
+
109
146
  #### Good
110
147
 
111
148
  Instead, add the column without a default value, then change the default.
112
149
 
113
150
  ```ruby
114
- class AddSomeColumnToUsers < ActiveRecord::Migration[6.0]
151
+ class AddSomeColumnToUsers < ActiveRecord::Migration[7.0]
115
152
  def up
116
153
  add_column :users, :some_column, :text
117
154
  change_column_default :users, :some_column, "default_value"
@@ -129,10 +166,10 @@ See the next section for how to backfill.
129
166
 
130
167
  #### Bad
131
168
 
132
- Backfilling in the same transaction that alters a table locks the table for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
169
+ Active Record creates a transaction around each migration, and backfilling in the same transaction that alters a table keeps the table locked for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
133
170
 
134
171
  ```ruby
135
- class AddSomeColumnToUsers < ActiveRecord::Migration[6.0]
172
+ class AddSomeColumnToUsers < ActiveRecord::Migration[7.0]
136
173
  def change
137
174
  add_column :users, :some_column, :text
138
175
  User.update_all some_column: "default_value"
@@ -147,7 +184,7 @@ Also, running a single query to update data can cause issues for large tables.
147
184
  There are three keys to backfilling safely: batching, throttling, and running it outside a transaction. Use the Rails console or a separate migration with `disable_ddl_transaction!`.
148
185
 
149
186
  ```ruby
150
- class BackfillSomeColumn < ActiveRecord::Migration[6.0]
187
+ class BackfillSomeColumn < ActiveRecord::Migration[7.0]
151
188
  disable_ddl_transaction!
152
189
 
153
190
  def up
@@ -159,39 +196,83 @@ class BackfillSomeColumn < ActiveRecord::Migration[6.0]
159
196
  end
160
197
  ```
161
198
 
162
- ### Renaming or changing the type of a column
199
+ ### Adding a stored generated column
163
200
 
164
201
  #### Bad
165
202
 
203
+ Adding a stored generated column causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
204
+
166
205
  ```ruby
167
- class RenameSomeColumn < ActiveRecord::Migration[6.0]
206
+ class AddSomeColumnToUsers < ActiveRecord::Migration[7.0]
168
207
  def change
169
- rename_column :users, :some_column, :new_name
208
+ add_column :users, :some_column, :virtual, type: :string, as: "...", stored: true
170
209
  end
171
210
  end
172
211
  ```
173
212
 
174
- or
213
+ #### Good
214
+
215
+ Add a non-generated column and use callbacks or triggers instead (or a virtual generated column with MySQL and MariaDB).
216
+
217
+ ### Changing the type of a column
218
+
219
+ #### Bad
220
+
221
+ Changing the type of a column causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
175
222
 
176
223
  ```ruby
177
- class ChangeSomeColumnType < ActiveRecord::Migration[6.0]
224
+ class ChangeSomeColumnType < ActiveRecord::Migration[7.0]
178
225
  def change
179
226
  change_column :users, :some_column, :new_type
180
227
  end
181
228
  end
182
229
  ```
183
230
 
184
- A few changes are safe in Postgres:
231
+ Some changes don’t require a table rewrite and are safe in Postgres:
185
232
 
186
- - Changing between `varchar` and `text` columns
187
- - Increasing the precision of a `decimal` or `numeric` column
188
- - Making a `decimal` or `numeric` column unconstrained
189
- - Changing between `timestamp` and `timestamptz` columns when session time zone is UTC in Postgres 12+
233
+ Type | Safe Changes
234
+ --- | ---
235
+ `cidr` | Changing to `inet`
236
+ `citext` | Changing to `text` if not indexed, changing to `string` with no `:limit` if not indexed
237
+ `datetime` | Increasing or removing `:precision`, changing to `timestamptz` when session time zone is UTC in Postgres 12+
238
+ `decimal` | Increasing `:precision` at same `:scale`, removing `:precision` and `:scale`
239
+ `interval` | Increasing or removing `:precision`
240
+ `numeric` | Increasing `:precision` at same `:scale`, removing `:precision` and `:scale`
241
+ `string` | Increasing or removing `:limit`, changing to `text`, changing `citext` if not indexed
242
+ `text` | Changing to `string` with no `:limit`, changing to `citext` if not indexed
243
+ `time` | Increasing or removing `:precision`
244
+ `timestamptz` | Increasing or removing `:limit`, changing to `datetime` when session time zone is UTC in Postgres 12+
190
245
 
191
- And a few in MySQL and MariaDB:
246
+ And some in MySQL and MariaDB:
192
247
 
193
- - Increasing the length of a `varchar` column from under 255 up to 255
194
- - Increasing the length of a `varchar` column over 255
248
+ Type | Safe Changes
249
+ --- | ---
250
+ `string` | Increasing `:limit` from under 63 up to 63, increasing `:limit` from over 63 to the max (the threshold can be different if using an encoding other than `utf8mb4` - for instance, it’s 85 for `utf8mb3` and 255 for `latin1`)
251
+
252
+ #### Good
253
+
254
+ A safer approach is to:
255
+
256
+ 1. Create a new column
257
+ 2. Write to both columns
258
+ 3. Backfill data from the old column to the new column
259
+ 4. Move reads from the old column to the new column
260
+ 5. Stop writing to the old column
261
+ 6. Drop the old column
262
+
263
+ ### Renaming a column
264
+
265
+ #### Bad
266
+
267
+ Renaming a column that’s in use will cause errors in your application.
268
+
269
+ ```ruby
270
+ class RenameSomeColumn < ActiveRecord::Migration[7.0]
271
+ def change
272
+ rename_column :users, :some_column, :new_name
273
+ end
274
+ end
275
+ ```
195
276
 
196
277
  #### Good
197
278
 
@@ -208,8 +289,10 @@ A safer approach is to:
208
289
 
209
290
  #### Bad
210
291
 
292
+ Renaming a table that’s in use will cause errors in your application.
293
+
211
294
  ```ruby
212
- class RenameUsersToCustomers < ActiveRecord::Migration[6.0]
295
+ class RenameUsersToCustomers < ActiveRecord::Migration[7.0]
213
296
  def change
214
297
  rename_table :users, :customers
215
298
  end
@@ -234,7 +317,7 @@ A safer approach is to:
234
317
  The `force` option can drop an existing table.
235
318
 
236
319
  ```ruby
237
- class CreateUsers < ActiveRecord::Migration[6.0]
320
+ class CreateUsers < ActiveRecord::Migration[7.0]
238
321
  def change
239
322
  create_table :users, force: true do |t|
240
323
  # ...
@@ -248,7 +331,7 @@ end
248
331
  Create tables without the `force` option.
249
332
 
250
333
  ```ruby
251
- class CreateUsers < ActiveRecord::Migration[6.0]
334
+ class CreateUsers < ActiveRecord::Migration[7.0]
252
335
  def change
253
336
  create_table :users do |t|
254
337
  # ...
@@ -259,40 +342,54 @@ end
259
342
 
260
343
  If you intend to drop an existing table, run `drop_table` first.
261
344
 
262
- ### Using change_column_null with a default value
345
+ ### Adding a check constraint
346
+
347
+ :turtle: Safe by default available
263
348
 
264
349
  #### Bad
265
350
 
266
- This generates a single `UPDATE` statement to set the default value.
351
+ Adding a check constraint blocks reads and writes in Postgres and blocks writes in MySQL and MariaDB while every row is checked.
267
352
 
268
353
  ```ruby
269
- class ChangeSomeColumnNull < ActiveRecord::Migration[6.0]
354
+ class AddCheckConstraint < ActiveRecord::Migration[7.0]
270
355
  def change
271
- change_column_null :users, :some_column, false, "default_value"
356
+ add_check_constraint :users, "price > 0", name: "price_check"
272
357
  end
273
358
  end
274
359
  ```
275
360
 
276
- #### Good
361
+ #### Good - Postgres
277
362
 
278
- Backfill the column [safely](#backfilling-data). Then use:
363
+ Add the check constraint without validating existing rows:
279
364
 
280
365
  ```ruby
281
- class ChangeSomeColumnNull < ActiveRecord::Migration[6.0]
366
+ class AddCheckConstraint < ActiveRecord::Migration[7.0]
282
367
  def change
283
- change_column_null :users, :some_column, false
368
+ add_check_constraint :users, "price > 0", name: "price_check", validate: false
284
369
  end
285
370
  end
286
371
  ```
287
372
 
288
- Note: In Postgres, `change_column_null` is still [not safe](#setting-not-null-on-an-existing-column) with this method.
373
+ Then validate them in a separate migration.
374
+
375
+ ```ruby
376
+ class ValidateCheckConstraint < ActiveRecord::Migration[7.0]
377
+ def change
378
+ validate_check_constraint :users, name: "price_check"
379
+ end
380
+ end
381
+ ```
382
+
383
+ #### Good - MySQL and MariaDB
384
+
385
+ [Let us know](https://github.com/ankane/strong_migrations/issues/new) if you have a safe way to do this (check constraints can be added with `NOT ENFORCED`, but enforcing blocks writes).
289
386
 
290
387
  ### Executing SQL directly
291
388
 
292
389
  Strong Migrations can’t ensure safety for raw SQL statements. Make really sure that what you’re doing is safe, then use:
293
390
 
294
391
  ```ruby
295
- class ExecuteSQL < ActiveRecord::Migration[6.0]
392
+ class ExecuteSQL < ActiveRecord::Migration[7.0]
296
393
  def change
297
394
  safety_assured { execute "..." }
298
395
  end
@@ -301,12 +398,14 @@ end
301
398
 
302
399
  ### Adding an index non-concurrently
303
400
 
401
+ :turtle: Safe by default available
402
+
304
403
  #### Bad
305
404
 
306
- In Postgres, adding an index non-concurrently locks the table.
405
+ In Postgres, adding an index non-concurrently blocks writes.
307
406
 
308
407
  ```ruby
309
- class AddSomeIndexToUsers < ActiveRecord::Migration[6.0]
408
+ class AddSomeIndexToUsers < ActiveRecord::Migration[7.0]
310
409
  def change
311
410
  add_index :users, :some_column
312
411
  end
@@ -318,7 +417,7 @@ end
318
417
  Add indexes concurrently.
319
418
 
320
419
  ```ruby
321
- class AddSomeIndexToUsers < ActiveRecord::Migration[6.0]
420
+ class AddSomeIndexToUsers < ActiveRecord::Migration[7.0]
322
421
  disable_ddl_transaction!
323
422
 
324
423
  def change
@@ -337,12 +436,14 @@ rails g index table column
337
436
 
338
437
  ### Adding a reference
339
438
 
439
+ :turtle: Safe by default available
440
+
340
441
  #### Bad
341
442
 
342
- Rails adds an index non-concurrently to references by default, which is problematic for Postgres.
443
+ Rails adds an index non-concurrently to references by default, which blocks writes in Postgres.
343
444
 
344
445
  ```ruby
345
- class AddReferenceToUsers < ActiveRecord::Migration[6.0]
446
+ class AddReferenceToUsers < ActiveRecord::Migration[7.0]
346
447
  def change
347
448
  add_reference :users, :city
348
449
  end
@@ -354,7 +455,7 @@ end
354
455
  Make sure the index is added concurrently.
355
456
 
356
457
  ```ruby
357
- class AddReferenceToUsers < ActiveRecord::Migration[6.0]
458
+ class AddReferenceToUsers < ActiveRecord::Migration[7.0]
358
459
  disable_ddl_transaction!
359
460
 
360
461
  def change
@@ -365,12 +466,14 @@ end
365
466
 
366
467
  ### Adding a foreign key
367
468
 
469
+ :turtle: Safe by default available
470
+
368
471
  #### Bad
369
472
 
370
- In Postgres, new foreign keys are validated by default, which acquires a `ShareRowExclusiveLock` that can be [expensive on large tables](https://travisofthenorth.com/blog/2017/2/2/postgres-adding-foreign-keys-with-zero-downtime).
473
+ In Postgres, adding a foreign key blocks writes on both tables.
371
474
 
372
475
  ```ruby
373
- class AddForeignKeyOnUsers < ActiveRecord::Migration[6.0]
476
+ class AddForeignKeyOnUsers < ActiveRecord::Migration[7.0]
374
477
  def change
375
478
  add_foreign_key :users, :orders
376
479
  end
@@ -380,7 +483,7 @@ end
380
483
  or
381
484
 
382
485
  ```ruby
383
- class AddReferenceToUsers < ActiveRecord::Migration[6.0]
486
+ class AddReferenceToUsers < ActiveRecord::Migration[7.0]
384
487
  def change
385
488
  add_reference :users, :order, foreign_key: true
386
489
  end
@@ -389,60 +492,52 @@ end
389
492
 
390
493
  #### Good
391
494
 
392
- Instead, validate it in a separate migration with a more agreeable `RowShareLock`. This approach is documented by Postgres to have “[the least impact on other work](https://www.postgresql.org/docs/current/sql-altertable.html).”
393
-
394
- For Rails 5.2+, use:
495
+ Add the foreign key without validating existing rows:
395
496
 
396
497
  ```ruby
397
- class AddForeignKeyOnUsers < ActiveRecord::Migration[6.0]
498
+ class AddForeignKeyOnUsers < ActiveRecord::Migration[7.0]
398
499
  def change
399
500
  add_foreign_key :users, :orders, validate: false
400
501
  end
401
502
  end
402
503
  ```
403
504
 
404
- Then validate it in a separate migration.
505
+ Then validate them in a separate migration.
405
506
 
406
507
  ```ruby
407
- class ValidateForeignKeyOnUsers < ActiveRecord::Migration[6.0]
508
+ class ValidateForeignKeyOnUsers < ActiveRecord::Migration[7.0]
408
509
  def change
409
510
  validate_foreign_key :users, :orders
410
511
  end
411
512
  end
412
513
  ```
413
514
 
414
- For Rails < 5.2, use:
515
+ ### Adding an exclusion constraint
415
516
 
416
- ```ruby
417
- class AddForeignKeyOnUsers < ActiveRecord::Migration[5.1]
418
- def change
419
- safety_assured do
420
- execute 'ALTER TABLE "users" ADD CONSTRAINT "fk_rails_c1e9b98e31" FOREIGN KEY ("order_id") REFERENCES "orders" ("id") NOT VALID'
421
- end
422
- end
423
- end
424
- ```
517
+ #### Bad
425
518
 
426
- Then validate it in a separate migration.
519
+ In Postgres, adding an exclusion constraint blocks reads and writes while every row is checked.
427
520
 
428
521
  ```ruby
429
- class ValidateForeignKeyOnUsers < ActiveRecord::Migration[5.1]
522
+ class AddExclusionContraint < ActiveRecord::Migration[7.1]
430
523
  def change
431
- safety_assured do
432
- execute 'ALTER TABLE "users" VALIDATE CONSTRAINT "fk_rails_c1e9b98e31"'
433
- end
524
+ add_exclusion_constraint :users, "number WITH =", using: :gist
434
525
  end
435
526
  end
436
527
  ```
437
528
 
529
+ #### Good
530
+
531
+ [Let us know](https://github.com/ankane/strong_migrations/issues/new) if you have a safe way to do this (exclusion constraints cannot be marked `NOT VALID`).
532
+
438
533
  ### Adding a json column
439
534
 
440
535
  #### Bad
441
536
 
442
- In Postgres, there’s no equality operator for the `json` column type, which can cause errors for existing `SELECT DISTINCT` queries.
537
+ In Postgres, there’s no equality operator for the `json` column type, which can cause errors for existing `SELECT DISTINCT` queries in your application.
443
538
 
444
539
  ```ruby
445
- class AddPropertiesToUsers < ActiveRecord::Migration[6.0]
540
+ class AddPropertiesToUsers < ActiveRecord::Migration[7.0]
446
541
  def change
447
542
  add_column :users, :properties, :json
448
543
  end
@@ -454,7 +549,7 @@ end
454
549
  Use `jsonb` instead.
455
550
 
456
551
  ```ruby
457
- class AddPropertiesToUsers < ActiveRecord::Migration[6.0]
552
+ class AddPropertiesToUsers < ActiveRecord::Migration[7.0]
458
553
  def change
459
554
  add_column :users, :properties, :jsonb
460
555
  end
@@ -463,12 +558,14 @@ end
463
558
 
464
559
  ### Setting NOT NULL on an existing column
465
560
 
561
+ :turtle: Safe by default available
562
+
466
563
  #### Bad
467
564
 
468
- In Postgres, setting `NOT NULL` on an existing column requires an `AccessExclusiveLock`, which is expensive on large tables.
565
+ In Postgres, setting `NOT NULL` on an existing column blocks reads and writes while every row is checked.
469
566
 
470
567
  ```ruby
471
- class SetSomeColumnNotNull < ActiveRecord::Migration[6.0]
568
+ class SetSomeColumnNotNull < ActiveRecord::Migration[7.0]
472
569
  def change
473
570
  change_column_null :users, :some_column, false
474
571
  end
@@ -477,7 +574,19 @@ end
477
574
 
478
575
  #### Good
479
576
 
480
- Instead, add a constraint:
577
+ Instead, add a check constraint.
578
+
579
+ For Rails 6.1, use:
580
+
581
+ ```ruby
582
+ class SetSomeColumnNotNull < ActiveRecord::Migration[7.0]
583
+ def change
584
+ add_check_constraint :users, "some_column IS NOT NULL", name: "users_some_column_null", validate: false
585
+ end
586
+ end
587
+ ```
588
+
589
+ For Rails < 6.1, use:
481
590
 
482
591
  ```ruby
483
592
  class SetSomeColumnNotNull < ActiveRecord::Migration[6.0]
@@ -489,7 +598,23 @@ class SetSomeColumnNotNull < ActiveRecord::Migration[6.0]
489
598
  end
490
599
  ```
491
600
 
492
- Then validate it in a separate migration.
601
+ Then validate it in a separate migration. A `NOT NULL` check constraint is [functionally equivalent](https://medium.com/doctolib/adding-a-not-null-constraint-on-pg-faster-with-minimal-locking-38b2c00c4d1c) to setting `NOT NULL` on the column (but it won’t show up in `schema.rb` in Rails < 6.1). In Postgres 12+, once the check constraint is validated, you can safely set `NOT NULL` on the column and drop the check constraint.
602
+
603
+ For Rails 6.1, use:
604
+
605
+ ```ruby
606
+ class ValidateSomeColumnNotNull < ActiveRecord::Migration[7.0]
607
+ def change
608
+ validate_check_constraint :users, name: "users_some_column_null"
609
+
610
+ # in Postgres 12+, you can then safely set NOT NULL on the column
611
+ change_column_null :users, :some_column, false
612
+ remove_check_constraint :users, name: "users_some_column_null"
613
+ end
614
+ end
615
+ ```
616
+
617
+ For Rails < 6.1, use:
493
618
 
494
619
  ```ruby
495
620
  class ValidateSomeColumnNotNull < ActiveRecord::Migration[6.0]
@@ -498,7 +623,7 @@ class ValidateSomeColumnNotNull < ActiveRecord::Migration[6.0]
498
623
  execute 'ALTER TABLE "users" VALIDATE CONSTRAINT "users_some_column_null"'
499
624
  end
500
625
 
501
- # in Postgres 12+, you can safely turn this into a traditional column constraint
626
+ # in Postgres 12+, you can then safely set NOT NULL on the column
502
627
  change_column_null :users, :some_column, false
503
628
  safety_assured do
504
629
  execute 'ALTER TABLE "users" DROP CONSTRAINT "users_some_column_null"'
@@ -507,7 +632,35 @@ class ValidateSomeColumnNotNull < ActiveRecord::Migration[6.0]
507
632
  end
508
633
  ```
509
634
 
510
- Note: This is not 100% the same as `NOT NULL` column constraint before Postgres 12. Here’s a [good explanation](https://medium.com/doctolib/adding-a-not-null-constraint-on-pg-faster-with-minimal-locking-38b2c00c4d1c).
635
+ ### Changing the default value of a column
636
+
637
+ #### Bad
638
+
639
+ Rails < 7 enables partial writes by default, which can cause incorrect values to be inserted when changing the default value of a column.
640
+
641
+ ```ruby
642
+ class ChangeSomeColumnDefault < ActiveRecord::Migration[6.1]
643
+ def change
644
+ change_column_default :users, :some_column, from: "old", to: "new"
645
+ end
646
+ end
647
+
648
+ User.create!(some_column: "old") # can insert "new"
649
+ ```
650
+
651
+ #### Good
652
+
653
+ Disable partial writes in `config/application.rb`. For Rails < 7, use:
654
+
655
+ ```ruby
656
+ config.active_record.partial_writes = false
657
+ ```
658
+
659
+ For Rails 7, use:
660
+
661
+ ```ruby
662
+ config.active_record.partial_inserts = false
663
+ ```
511
664
 
512
665
  ### Keeping non-unique indexes to three columns or less
513
666
 
@@ -516,7 +669,7 @@ Note: This is not 100% the same as `NOT NULL` column constraint before Postgres
516
669
  Adding a non-unique index with more than three columns rarely improves performance.
517
670
 
518
671
  ```ruby
519
- class AddSomeIndexToUsers < ActiveRecord::Migration[6.0]
672
+ class AddSomeIndexToUsers < ActiveRecord::Migration[7.0]
520
673
  def change
521
674
  add_index :users, [:a, :b, :c, :d]
522
675
  end
@@ -528,7 +681,7 @@ end
528
681
  Instead, start an index with columns that narrow down the results the most.
529
682
 
530
683
  ```ruby
531
- class AddSomeIndexToUsers < ActiveRecord::Migration[6.0]
684
+ class AddSomeIndexToUsers < ActiveRecord::Migration[7.0]
532
685
  def change
533
686
  add_index :users, [:b, :d]
534
687
  end
@@ -542,7 +695,7 @@ For Postgres, be sure to add them concurrently.
542
695
  To mark a step in the migration as safe, despite using a method that might otherwise be dangerous, wrap it in a `safety_assured` block.
543
696
 
544
697
  ```ruby
545
- class MySafeMigration < ActiveRecord::Migration[6.0]
698
+ class MySafeMigration < ActiveRecord::Migration[7.0]
546
699
  def change
547
700
  safety_assured { remove_column :users, :some_column }
548
701
  end
@@ -551,6 +704,21 @@ end
551
704
 
552
705
  Certain methods like `execute` and `change_table` cannot be inspected and are prevented from running by default. Make sure what you’re doing is really safe and use this pattern.
553
706
 
707
+ ## Safe by Default
708
+
709
+ Make operations safe by default.
710
+
711
+ - adding and removing an index
712
+ - adding a foreign key
713
+ - adding a check constraint
714
+ - setting NOT NULL on an existing column
715
+
716
+ Add to `config/initializers/strong_migrations.rb`:
717
+
718
+ ```ruby
719
+ StrongMigrations.safe_by_default = true
720
+ ```
721
+
554
722
  ## Custom Checks
555
723
 
556
724
  Add your own custom checks with:
@@ -585,7 +753,15 @@ Disable specific checks with:
585
753
  StrongMigrations.disable_check(:add_index)
586
754
  ```
587
755
 
588
- Check the [source code](https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations.rb) for the list of keys.
756
+ Check the [source code](https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations/error_messages.rb) for the list of keys.
757
+
758
+ ## Down Migrations / Rollbacks
759
+
760
+ By default, checks are disabled when migrating down. Enable them with:
761
+
762
+ ```ruby
763
+ StrongMigrations.check_down = true
764
+ ```
589
765
 
590
766
  ## Custom Messages
591
767
 
@@ -595,9 +771,9 @@ To customize specific messages, create an initializer with:
595
771
  StrongMigrations.error_messages[:add_column_default] = "Your custom instructions"
596
772
  ```
597
773
 
598
- Check the [source code](https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations.rb) for the list of keys.
774
+ Check the [source code](https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations/error_messages.rb) for the list of keys.
599
775
 
600
- ## Timeouts
776
+ ## Migration Timeouts
601
777
 
602
778
  It’s extremely important to set a short lock timeout for migrations. This way, if a migration can’t acquire a lock in a timely manner, other statements won’t be stuck behind it. We also recommend setting a long statement timeout so migrations can run for a while.
603
779
 
@@ -617,6 +793,68 @@ ALTER ROLE myuser SET statement_timeout = '1h';
617
793
 
618
794
  Note: If you use PgBouncer in transaction mode, you must set timeouts on the database user.
619
795
 
796
+ ## Lock Timeout Retries [experimental]
797
+
798
+ There’s the option to automatically retry statements when the lock timeout is reached. Here’s how it works:
799
+
800
+ - If a lock timeout happens outside a transaction, the statement is retried
801
+ - If it happens inside the DDL transaction, the entire migration is retried (only applicable to Postgres)
802
+
803
+ Add to `config/initializers/strong_migrations.rb`:
804
+
805
+ ```ruby
806
+ StrongMigrations.lock_timeout_retries = 3
807
+ ```
808
+
809
+ Set the delay between retries with:
810
+
811
+ ```ruby
812
+ StrongMigrations.lock_timeout_retry_delay = 10.seconds
813
+ ```
814
+
815
+ ## App Timeouts
816
+
817
+ We recommend adding timeouts to `config/database.yml` to prevent connections from hanging and individual queries from taking up too many resources in controllers, jobs, the Rails console, and other places.
818
+
819
+ For Postgres:
820
+
821
+ ```yml
822
+ production:
823
+ connect_timeout: 5
824
+ variables:
825
+ statement_timeout: 15s
826
+ lock_timeout: 10s
827
+ ```
828
+
829
+ Note: If you use PgBouncer in transaction mode, you must set the statement and lock timeouts on the database user as shown above.
830
+
831
+ For MySQL:
832
+
833
+ ```yml
834
+ production:
835
+ connect_timeout: 5
836
+ read_timeout: 5
837
+ write_timeout: 5
838
+ variables:
839
+ max_execution_time: 15000 # ms
840
+ lock_wait_timeout: 10 # sec
841
+
842
+ ```
843
+
844
+ For MariaDB:
845
+
846
+ ```yml
847
+ production:
848
+ connect_timeout: 5
849
+ read_timeout: 5
850
+ write_timeout: 5
851
+ variables:
852
+ max_statement_time: 15 # sec
853
+ lock_wait_timeout: 10 # sec
854
+ ```
855
+
856
+ For HTTP connections, Redis, and other services, check out [this guide](https://github.com/ankane/the-ultimate-guide-to-ruby-timeouts).
857
+
620
858
  ## Existing Migrations
621
859
 
622
860
  To mark migrations as safe that were created before installing this gem, create an initializer with:
@@ -632,13 +870,19 @@ Use the version from your latest migration.
632
870
  If your development database version is different from production, you can specify the production version so the right checks run in development.
633
871
 
634
872
  ```ruby
635
- StrongMigrations.target_postgresql_version = "10"
636
- StrongMigrations.target_mysql_version = "8.0.12"
637
- StrongMigrations.target_mariadb_version = "10.3.2"
873
+ StrongMigrations.target_version = 10 # or "8.0.12", "10.3.2", etc
638
874
  ```
639
875
 
876
+ The major version works well for Postgres, while the full version is recommended for MySQL and MariaDB.
877
+
640
878
  For safety, this option only affects development and test environments. In other environments, the actual server version is always used.
641
879
 
880
+ If your app has multiple databases with different versions, with Rails 6.1+, you can use:
881
+
882
+ ```ruby
883
+ StrongMigrations.target_version = {primary: 13, catalog: 15}
884
+ ```
885
+
642
886
  ## Analyze Tables
643
887
 
644
888
  Analyze tables automatically (to update planner statistics) after an index is added. Create an initializer with:
@@ -649,37 +893,34 @@ StrongMigrations.auto_analyze = true
649
893
 
650
894
  ## Faster Migrations
651
895
 
652
- Only dump the schema when adding a new migration. If you use Git, create an initializer with:
896
+ Only dump the schema when adding a new migration. If you use Git, add to `config/environments/development.rb`:
653
897
 
654
- ```ruby
655
- ActiveRecord::Base.dump_schema_after_migration = Rails.env.development? &&
656
- `git status db/migrate/ --porcelain`.present?
898
+ ```rb
899
+ config.active_record.dump_schema_after_migration = `git status db/migrate/ --porcelain`.present?
657
900
  ```
658
901
 
659
902
  ## Schema Sanity
660
903
 
661
- Columns can flip order in `db/schema.rb` when you have multiple developers. One way to prevent this is to [alphabetize them](https://www.pgrs.net/2008/03/12/alphabetize-schema-rb-columns/). Add to the end of your `Rakefile`:
904
+ Columns can flip order in `db/schema.rb` when you have multiple developers. One way to prevent this is to [alphabetize them](https://www.pgrs.net/2008/03/12/alphabetize-schema-rb-columns/). Add to `config/initializers/strong_migrations.rb`:
662
905
 
663
906
  ```ruby
664
- task "db:schema:dump": "strong_migrations:alphabetize_columns"
665
- ```
666
-
667
- ## Dangerous Tasks
668
-
669
- For safety, dangerous database tasks are disabled in production - `db:drop`, `db:reset`, `db:schema:load`, and `db:structure:load`. To get around this, use:
670
-
671
- ```sh
672
- SAFETY_ASSURED=1 rails db:drop
907
+ StrongMigrations.alphabetize_schema = true
673
908
  ```
674
909
 
675
910
  ## Permissions
676
911
 
677
912
  We recommend using a [separate database user](https://ankane.org/postgres-users) for migrations when possible so you don’t need to grant your app user permission to alter tables.
678
913
 
914
+ ## Smaller Projects
915
+
916
+ You probably don’t need this gem for smaller projects, as operations that are unsafe at scale can be perfectly safe on smaller, low-traffic tables.
917
+
679
918
  ## Additional Reading
680
919
 
681
920
  - [Rails Migrations with No Downtime](https://pedro.herokuapp.com/past/2011/7/13/rails_migrations_with_no_downtime/)
682
921
  - [PostgreSQL at Scale: Database Schema Changes Without Downtime](https://medium.com/braintree-product-technology/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680)
922
+ - [An Overview of DDL Algorithms in MySQL](https://mydbops.wordpress.com/2020/03/04/an-overview-of-ddl-algorithms-in-mysql-covers-mysql-8/)
923
+ - [MariaDB InnoDB Online DDL Overview](https://mariadb.com/kb/en/innodb-online-ddl-overview/)
683
924
 
684
925
  ## Credits
685
926