strong_migrations 0.6.8 → 1.2.0

Sign up to get free protection for your applications and to get access to all the features.
data/README.md CHANGED
@@ -4,18 +4,18 @@ Catch unsafe migrations in development
4
4
 
5
5
  &nbsp;&nbsp;✓&nbsp;&nbsp;Detects potentially dangerous operations<br />&nbsp;&nbsp;✓&nbsp;&nbsp;Prevents them from running by default<br />&nbsp;&nbsp;✓&nbsp;&nbsp;Provides instructions on safer ways to do what you want
6
6
 
7
- Supports for PostgreSQL, MySQL, and MariaDB
7
+ Supports PostgreSQL, MySQL, and MariaDB
8
8
 
9
9
  :tangerine: Battle-tested at [Instacart](https://www.instacart.com/opensource)
10
10
 
11
- [![Build Status](https://travis-ci.org/ankane/strong_migrations.svg?branch=master)](https://travis-ci.org/ankane/strong_migrations)
11
+ [![Build Status](https://github.com/ankane/strong_migrations/workflows/build/badge.svg?branch=master)](https://github.com/ankane/strong_migrations/actions)
12
12
 
13
13
  ## Installation
14
14
 
15
15
  Add this line to your application’s Gemfile:
16
16
 
17
17
  ```ruby
18
- gem 'strong_migrations'
18
+ gem "strong_migrations"
19
19
  ```
20
20
 
21
21
  And run:
@@ -25,6 +25,36 @@ bundle install
25
25
  rails generate strong_migrations:install
26
26
  ```
27
27
 
28
+ Strong Migrations sets a long statement timeout for migrations so you can set a [short statement timeout](#app-timeouts) for your application.
29
+
30
+ ## How It Works
31
+
32
+ When you run a migration that’s potentially dangerous, you’ll see an error message like:
33
+
34
+ ```txt
35
+ === Dangerous operation detected #strong_migrations ===
36
+
37
+ Active Record caches attributes, which causes problems
38
+ when removing columns. Be sure to ignore the column:
39
+
40
+ class User < ApplicationRecord
41
+ self.ignored_columns = ["name"]
42
+ end
43
+
44
+ Deploy the code, then wrap this step in a safety_assured { ... } block.
45
+
46
+ class RemoveColumn < ActiveRecord::Migration[7.0]
47
+ def change
48
+ safety_assured { remove_column :users, :name }
49
+ end
50
+ end
51
+ ```
52
+
53
+ An operation is classified as dangerous if it either:
54
+
55
+ - Blocks reads or writes for more than a few seconds (after a lock is acquired)
56
+ - Has a good chance of causing application errors
57
+
28
58
  ## Checks
29
59
 
30
60
  Potentially dangerous operations:
@@ -32,11 +62,11 @@ Potentially dangerous operations:
32
62
  - [removing a column](#removing-a-column)
33
63
  - [adding a column with a default value](#adding-a-column-with-a-default-value)
34
64
  - [backfilling data](#backfilling-data)
35
- - [changing the type of a column](#renaming-or-changing-the-type-of-a-column)
36
- - [renaming a column](#renaming-or-changing-the-type-of-a-column)
65
+ - [changing the type of a column](#changing-the-type-of-a-column)
66
+ - [renaming a column](#renaming-a-column)
37
67
  - [renaming a table](#renaming-a-table)
38
68
  - [creating a table with the force option](#creating-a-table-with-the-force-option)
39
- - [using change_column_null with a default value](#using-change_column_null-with-a-default-value)
69
+ - [adding a check constraint](#adding-a-check-constraint)
40
70
  - [executing SQL directly](#executing-SQL-directly)
41
71
 
42
72
  Postgres-specific checks:
@@ -57,10 +87,10 @@ You can also add [custom checks](#custom-checks) or [disable specific checks](#d
57
87
 
58
88
  #### Bad
59
89
 
60
- ActiveRecord caches database columns at runtime, so if you drop a column, it can cause exceptions until your app reboots.
90
+ Active Record caches database columns at runtime, so if you drop a column, it can cause exceptions until your app reboots.
61
91
 
62
92
  ```ruby
63
- class RemoveSomeColumnFromUsers < ActiveRecord::Migration[6.0]
93
+ class RemoveSomeColumnFromUsers < ActiveRecord::Migration[7.0]
64
94
  def change
65
95
  remove_column :users, :some_column
66
96
  end
@@ -69,7 +99,7 @@ end
69
99
 
70
100
  #### Good
71
101
 
72
- 1. Tell ActiveRecord to ignore the column from its cache
102
+ 1. Tell Active Record to ignore the column from its cache
73
103
 
74
104
  ```ruby
75
105
  class User < ApplicationRecord
@@ -81,7 +111,7 @@ end
81
111
  3. Write a migration to remove the column (wrap in `safety_assured` block)
82
112
 
83
113
  ```ruby
84
- class RemoveSomeColumnFromUsers < ActiveRecord::Migration[6.0]
114
+ class RemoveSomeColumnFromUsers < ActiveRecord::Migration[7.0]
85
115
  def change
86
116
  safety_assured { remove_column :users, :some_column }
87
117
  end
@@ -89,29 +119,30 @@ end
89
119
  ```
90
120
 
91
121
  4. Deploy and run migration
122
+ 5. Remove the line added in step 1
92
123
 
93
124
  ### Adding a column with a default value
94
125
 
95
- Note: This operation is safe in Postgres 11+, MySQL 8.0.12+, and MariaDB 10.3.2+.
96
-
97
126
  #### Bad
98
127
 
99
- Adding a column with a default value to an existing table causes the entire table to be rewritten.
128
+ In earlier versions of Postgres, MySQL, and MariaDB, adding a column with a default value to an existing table causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
100
129
 
101
130
  ```ruby
102
- class AddSomeColumnToUsers < ActiveRecord::Migration[6.0]
131
+ class AddSomeColumnToUsers < ActiveRecord::Migration[7.0]
103
132
  def change
104
133
  add_column :users, :some_column, :text, default: "default_value"
105
134
  end
106
135
  end
107
136
  ```
108
137
 
138
+ In Postgres 11+, MySQL 8.0.12+, and MariaDB 10.3.2+, this no longer requires a table rewrite and is safe.
139
+
109
140
  #### Good
110
141
 
111
142
  Instead, add the column without a default value, then change the default.
112
143
 
113
144
  ```ruby
114
- class AddSomeColumnToUsers < ActiveRecord::Migration[6.0]
145
+ class AddSomeColumnToUsers < ActiveRecord::Migration[7.0]
115
146
  def up
116
147
  add_column :users, :some_column, :text
117
148
  change_column_default :users, :some_column, "default_value"
@@ -129,10 +160,10 @@ See the next section for how to backfill.
129
160
 
130
161
  #### Bad
131
162
 
132
- Backfilling in the same transaction that alters a table locks the table for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
163
+ Active Record creates a transaction around each migration, and backfilling in the same transaction that alters a table keeps the table locked for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
133
164
 
134
165
  ```ruby
135
- class AddSomeColumnToUsers < ActiveRecord::Migration[6.0]
166
+ class AddSomeColumnToUsers < ActiveRecord::Migration[7.0]
136
167
  def change
137
168
  add_column :users, :some_column, :text
138
169
  User.update_all some_column: "default_value"
@@ -147,7 +178,7 @@ Also, running a single query to update data can cause issues for large tables.
147
178
  There are three keys to backfilling safely: batching, throttling, and running it outside a transaction. Use the Rails console or a separate migration with `disable_ddl_transaction!`.
148
179
 
149
180
  ```ruby
150
- class BackfillSomeColumn < ActiveRecord::Migration[6.0]
181
+ class BackfillSomeColumn < ActiveRecord::Migration[7.0]
151
182
  disable_ddl_transaction!
152
183
 
153
184
  def up
@@ -159,40 +190,66 @@ class BackfillSomeColumn < ActiveRecord::Migration[6.0]
159
190
  end
160
191
  ```
161
192
 
162
- ### Renaming or changing the type of a column
193
+ ### Changing the type of a column
163
194
 
164
195
  #### Bad
165
196
 
197
+ Changing the type of a column causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
198
+
166
199
  ```ruby
167
- class RenameSomeColumn < ActiveRecord::Migration[6.0]
200
+ class ChangeSomeColumnType < ActiveRecord::Migration[7.0]
168
201
  def change
169
- rename_column :users, :some_column, :new_name
202
+ change_column :users, :some_column, :new_type
170
203
  end
171
204
  end
172
205
  ```
173
206
 
174
- or
207
+ Some changes don’t require a table rewrite and are safe in Postgres:
208
+
209
+ Type | Safe Changes
210
+ --- | ---
211
+ `cidr` | Changing to `inet`
212
+ `citext` | Changing to `text` if not indexed, changing to `string` with no `:limit` if not indexed
213
+ `datetime` | Increasing or removing `:precision`, changing to `timestamptz` when session time zone is UTC in Postgres 12+
214
+ `decimal` | Increasing `:precision` at same `:scale`, removing `:precision` and `:scale`
215
+ `interval` | Increasing or removing `:precision`
216
+ `numeric` | Increasing `:precision` at same `:scale`, removing `:precision` and `:scale`
217
+ `string` | Increasing or removing `:limit`, changing to `text`, changing `citext` if not indexed
218
+ `text` | Changing to `string` with no `:limit`, changing to `citext` if not indexed
219
+ `time` | Increasing or removing `:precision`
220
+ `timestamptz` | Increasing or removing `:limit`, changing to `datetime` when session time zone is UTC in Postgres 12+
221
+
222
+ And some in MySQL and MariaDB:
223
+
224
+ Type | Safe Changes
225
+ --- | ---
226
+ `string` | Increasing `:limit` from under 255 up to 255, increasing `:limit` from over 255 to the max
227
+
228
+ #### Good
229
+
230
+ A safer approach is to:
231
+
232
+ 1. Create a new column
233
+ 2. Write to both columns
234
+ 3. Backfill data from the old column to the new column
235
+ 4. Move reads from the old column to the new column
236
+ 5. Stop writing to the old column
237
+ 6. Drop the old column
238
+
239
+ ### Renaming a column
240
+
241
+ #### Bad
242
+
243
+ Renaming a column that’s in use will cause errors in your application.
175
244
 
176
245
  ```ruby
177
- class ChangeSomeColumnType < ActiveRecord::Migration[6.0]
246
+ class RenameSomeColumn < ActiveRecord::Migration[7.0]
178
247
  def change
179
- change_column :users, :some_column, :new_type
248
+ rename_column :users, :some_column, :new_name
180
249
  end
181
250
  end
182
251
  ```
183
252
 
184
- A few changes are safe in Postgres:
185
-
186
- - Changing between `varchar` and `text` columns
187
- - Increasing the precision of a `decimal` or `numeric` column
188
- - Making a `decimal` or `numeric` column unconstrained
189
- - Changing between `timestamp` and `timestamptz` columns when session time zone is UTC in Postgres 12+
190
-
191
- And a few in MySQL and MariaDB:
192
-
193
- - Increasing the length of a `varchar` column from under 255 up to 255
194
- - Increasing the length of a `varchar` column over 255
195
-
196
253
  #### Good
197
254
 
198
255
  A safer approach is to:
@@ -208,8 +265,10 @@ A safer approach is to:
208
265
 
209
266
  #### Bad
210
267
 
268
+ Renaming a table that’s in use will cause errors in your application.
269
+
211
270
  ```ruby
212
- class RenameUsersToCustomers < ActiveRecord::Migration[6.0]
271
+ class RenameUsersToCustomers < ActiveRecord::Migration[7.0]
213
272
  def change
214
273
  rename_table :users, :customers
215
274
  end
@@ -234,7 +293,7 @@ A safer approach is to:
234
293
  The `force` option can drop an existing table.
235
294
 
236
295
  ```ruby
237
- class CreateUsers < ActiveRecord::Migration[6.0]
296
+ class CreateUsers < ActiveRecord::Migration[7.0]
238
297
  def change
239
298
  create_table :users, force: true do |t|
240
299
  # ...
@@ -248,7 +307,7 @@ end
248
307
  Create tables without the `force` option.
249
308
 
250
309
  ```ruby
251
- class CreateUsers < ActiveRecord::Migration[6.0]
310
+ class CreateUsers < ActiveRecord::Migration[7.0]
252
311
  def change
253
312
  create_table :users do |t|
254
313
  # ...
@@ -259,40 +318,54 @@ end
259
318
 
260
319
  If you intend to drop an existing table, run `drop_table` first.
261
320
 
262
- ### Using change_column_null with a default value
321
+ ### Adding a check constraint
322
+
323
+ :turtle: Safe by default available
263
324
 
264
325
  #### Bad
265
326
 
266
- This generates a single `UPDATE` statement to set the default value.
327
+ Adding a check constraint blocks reads and writes in Postgres and blocks writes in MySQL and MariaDB while every row is checked.
267
328
 
268
329
  ```ruby
269
- class ChangeSomeColumnNull < ActiveRecord::Migration[6.0]
330
+ class AddCheckConstraint < ActiveRecord::Migration[7.0]
270
331
  def change
271
- change_column_null :users, :some_column, false, "default_value"
332
+ add_check_constraint :users, "price > 0", name: "price_check"
272
333
  end
273
334
  end
274
335
  ```
275
336
 
276
- #### Good
337
+ #### Good - Postgres
277
338
 
278
- Backfill the column [safely](#backfilling-data). Then use:
339
+ Add the check constraint without validating existing rows:
279
340
 
280
341
  ```ruby
281
- class ChangeSomeColumnNull < ActiveRecord::Migration[6.0]
342
+ class AddCheckConstraint < ActiveRecord::Migration[7.0]
282
343
  def change
283
- change_column_null :users, :some_column, false
344
+ add_check_constraint :users, "price > 0", name: "price_check", validate: false
345
+ end
346
+ end
347
+ ```
348
+
349
+ Then validate them in a separate migration.
350
+
351
+ ```ruby
352
+ class ValidateCheckConstraint < ActiveRecord::Migration[7.0]
353
+ def change
354
+ validate_check_constraint :users, name: "price_check"
284
355
  end
285
356
  end
286
357
  ```
287
358
 
288
- Note: In Postgres, `change_column_null` is still [not safe](#setting-not-null-on-an-existing-column) with this method.
359
+ #### Good - MySQL and MariaDB
360
+
361
+ [Let us know](https://github.com/ankane/strong_migrations/issues/new) if you have a safe way to do this (check constraints can be added with `NOT ENFORCED`, but enforcing blocks writes).
289
362
 
290
363
  ### Executing SQL directly
291
364
 
292
365
  Strong Migrations can’t ensure safety for raw SQL statements. Make really sure that what you’re doing is safe, then use:
293
366
 
294
367
  ```ruby
295
- class ExecuteSQL < ActiveRecord::Migration[6.0]
368
+ class ExecuteSQL < ActiveRecord::Migration[7.0]
296
369
  def change
297
370
  safety_assured { execute "..." }
298
371
  end
@@ -301,12 +374,14 @@ end
301
374
 
302
375
  ### Adding an index non-concurrently
303
376
 
377
+ :turtle: Safe by default available
378
+
304
379
  #### Bad
305
380
 
306
- In Postgres, adding an index non-concurrently locks the table.
381
+ In Postgres, adding an index non-concurrently blocks writes.
307
382
 
308
383
  ```ruby
309
- class AddSomeIndexToUsers < ActiveRecord::Migration[6.0]
384
+ class AddSomeIndexToUsers < ActiveRecord::Migration[7.0]
310
385
  def change
311
386
  add_index :users, :some_column
312
387
  end
@@ -318,7 +393,7 @@ end
318
393
  Add indexes concurrently.
319
394
 
320
395
  ```ruby
321
- class AddSomeIndexToUsers < ActiveRecord::Migration[6.0]
396
+ class AddSomeIndexToUsers < ActiveRecord::Migration[7.0]
322
397
  disable_ddl_transaction!
323
398
 
324
399
  def change
@@ -337,12 +412,14 @@ rails g index table column
337
412
 
338
413
  ### Adding a reference
339
414
 
415
+ :turtle: Safe by default available
416
+
340
417
  #### Bad
341
418
 
342
- Rails adds an index non-concurrently to references by default, which is problematic for Postgres.
419
+ Rails adds an index non-concurrently to references by default, which blocks writes in Postgres.
343
420
 
344
421
  ```ruby
345
- class AddReferenceToUsers < ActiveRecord::Migration[6.0]
422
+ class AddReferenceToUsers < ActiveRecord::Migration[7.0]
346
423
  def change
347
424
  add_reference :users, :city
348
425
  end
@@ -354,7 +431,7 @@ end
354
431
  Make sure the index is added concurrently.
355
432
 
356
433
  ```ruby
357
- class AddReferenceToUsers < ActiveRecord::Migration[6.0]
434
+ class AddReferenceToUsers < ActiveRecord::Migration[7.0]
358
435
  disable_ddl_transaction!
359
436
 
360
437
  def change
@@ -365,12 +442,14 @@ end
365
442
 
366
443
  ### Adding a foreign key
367
444
 
445
+ :turtle: Safe by default available
446
+
368
447
  #### Bad
369
448
 
370
- In Postgres, new foreign keys are validated by default, which acquires a `ShareRowExclusiveLock` that can be [expensive on large tables](https://travisofthenorth.com/blog/2017/2/2/postgres-adding-foreign-keys-with-zero-downtime).
449
+ In Postgres, adding a foreign key blocks writes on both tables.
371
450
 
372
451
  ```ruby
373
- class AddForeignKeyOnUsers < ActiveRecord::Migration[6.0]
452
+ class AddForeignKeyOnUsers < ActiveRecord::Migration[7.0]
374
453
  def change
375
454
  add_foreign_key :users, :orders
376
455
  end
@@ -380,7 +459,7 @@ end
380
459
  or
381
460
 
382
461
  ```ruby
383
- class AddReferenceToUsers < ActiveRecord::Migration[6.0]
462
+ class AddReferenceToUsers < ActiveRecord::Migration[7.0]
384
463
  def change
385
464
  add_reference :users, :order, foreign_key: true
386
465
  end
@@ -389,107 +468,111 @@ end
389
468
 
390
469
  #### Good
391
470
 
392
- Instead, validate it in a separate migration with a more agreeable `RowShareLock`. This approach is documented by Postgres to have “[the least impact on other work](https://www.postgresql.org/docs/current/sql-altertable.html).”
393
-
394
- For Rails 5.2+, use:
471
+ Add the foreign key without validating existing rows:
395
472
 
396
473
  ```ruby
397
- class AddForeignKeyOnUsers < ActiveRecord::Migration[6.0]
474
+ class AddForeignKeyOnUsers < ActiveRecord::Migration[7.0]
398
475
  def change
399
476
  add_foreign_key :users, :orders, validate: false
400
477
  end
401
478
  end
402
479
  ```
403
480
 
404
- Then validate it in a separate migration.
481
+ Then validate them in a separate migration.
405
482
 
406
483
  ```ruby
407
- class ValidateForeignKeyOnUsers < ActiveRecord::Migration[6.0]
484
+ class ValidateForeignKeyOnUsers < ActiveRecord::Migration[7.0]
408
485
  def change
409
486
  validate_foreign_key :users, :orders
410
487
  end
411
488
  end
412
489
  ```
413
490
 
414
- For Rails < 5.2, use:
491
+ ### Adding a json column
492
+
493
+ #### Bad
494
+
495
+ In Postgres, there’s no equality operator for the `json` column type, which can cause errors for existing `SELECT DISTINCT` queries in your application.
415
496
 
416
497
  ```ruby
417
- class AddForeignKeyOnUsers < ActiveRecord::Migration[5.1]
498
+ class AddPropertiesToUsers < ActiveRecord::Migration[7.0]
418
499
  def change
419
- safety_assured do
420
- execute 'ALTER TABLE "users" ADD CONSTRAINT "fk_rails_c1e9b98e31" FOREIGN KEY ("order_id") REFERENCES "orders" ("id") NOT VALID'
421
- end
500
+ add_column :users, :properties, :json
422
501
  end
423
502
  end
424
503
  ```
425
504
 
426
- Then validate it in a separate migration.
505
+ #### Good
506
+
507
+ Use `jsonb` instead.
427
508
 
428
509
  ```ruby
429
- class ValidateForeignKeyOnUsers < ActiveRecord::Migration[5.1]
510
+ class AddPropertiesToUsers < ActiveRecord::Migration[7.0]
430
511
  def change
431
- safety_assured do
432
- execute 'ALTER TABLE "users" VALIDATE CONSTRAINT "fk_rails_c1e9b98e31"'
433
- end
512
+ add_column :users, :properties, :jsonb
434
513
  end
435
514
  end
436
515
  ```
437
516
 
438
- ### Adding a json column
517
+ ### Setting NOT NULL on an existing column
518
+
519
+ :turtle: Safe by default available
439
520
 
440
521
  #### Bad
441
522
 
442
- In Postgres, there’s no equality operator for the `json` column type, which can cause errors for existing `SELECT DISTINCT` queries.
523
+ In Postgres, setting `NOT NULL` on an existing column blocks reads and writes while every row is checked.
443
524
 
444
525
  ```ruby
445
- class AddPropertiesToUsers < ActiveRecord::Migration[6.0]
526
+ class SetSomeColumnNotNull < ActiveRecord::Migration[7.0]
446
527
  def change
447
- add_column :users, :properties, :json
528
+ change_column_null :users, :some_column, false
448
529
  end
449
530
  end
450
531
  ```
451
532
 
452
533
  #### Good
453
534
 
454
- Use `jsonb` instead.
535
+ Instead, add a check constraint.
536
+
537
+ For Rails 6.1, use:
455
538
 
456
539
  ```ruby
457
- class AddPropertiesToUsers < ActiveRecord::Migration[6.0]
540
+ class SetSomeColumnNotNull < ActiveRecord::Migration[7.0]
458
541
  def change
459
- add_column :users, :properties, :jsonb
542
+ add_check_constraint :users, "some_column IS NOT NULL", name: "users_some_column_null", validate: false
460
543
  end
461
544
  end
462
545
  ```
463
546
 
464
- ### Setting NOT NULL on an existing column
465
-
466
- #### Bad
467
-
468
- In Postgres, setting `NOT NULL` on an existing column requires an `AccessExclusiveLock`, which is expensive on large tables.
547
+ For Rails < 6.1, use:
469
548
 
470
549
  ```ruby
471
550
  class SetSomeColumnNotNull < ActiveRecord::Migration[6.0]
472
551
  def change
473
- change_column_null :users, :some_column, false
552
+ safety_assured do
553
+ execute 'ALTER TABLE "users" ADD CONSTRAINT "users_some_column_null" CHECK ("some_column" IS NOT NULL) NOT VALID'
554
+ end
474
555
  end
475
556
  end
476
557
  ```
477
558
 
478
- #### Good
559
+ Then validate it in a separate migration. A `NOT NULL` check constraint is [functionally equivalent](https://medium.com/doctolib/adding-a-not-null-constraint-on-pg-faster-with-minimal-locking-38b2c00c4d1c) to setting `NOT NULL` on the column (but it won’t show up in `schema.rb` in Rails < 6.1). In Postgres 12+, once the check constraint is validated, you can safely set `NOT NULL` on the column and drop the check constraint.
479
560
 
480
- Instead, add a constraint:
561
+ For Rails 6.1, use:
481
562
 
482
563
  ```ruby
483
- class SetSomeColumnNotNull < ActiveRecord::Migration[6.0]
564
+ class ValidateSomeColumnNotNull < ActiveRecord::Migration[7.0]
484
565
  def change
485
- safety_assured do
486
- execute 'ALTER TABLE "users" ADD CONSTRAINT "users_some_column_null" CHECK ("some_column" IS NOT NULL) NOT VALID'
487
- end
566
+ validate_check_constraint :users, name: "users_some_column_null"
567
+
568
+ # in Postgres 12+, you can then safely set NOT NULL on the column
569
+ change_column_null :users, :some_column, false
570
+ remove_check_constraint :users, name: "users_some_column_null"
488
571
  end
489
572
  end
490
573
  ```
491
574
 
492
- Then validate it in a separate migration.
575
+ For Rails < 6.1, use:
493
576
 
494
577
  ```ruby
495
578
  class ValidateSomeColumnNotNull < ActiveRecord::Migration[6.0]
@@ -498,7 +581,7 @@ class ValidateSomeColumnNotNull < ActiveRecord::Migration[6.0]
498
581
  execute 'ALTER TABLE "users" VALIDATE CONSTRAINT "users_some_column_null"'
499
582
  end
500
583
 
501
- # in Postgres 12+, you can safely turn this into a traditional column constraint
584
+ # in Postgres 12+, you can then safely set NOT NULL on the column
502
585
  change_column_null :users, :some_column, false
503
586
  safety_assured do
504
587
  execute 'ALTER TABLE "users" DROP CONSTRAINT "users_some_column_null"'
@@ -507,8 +590,6 @@ class ValidateSomeColumnNotNull < ActiveRecord::Migration[6.0]
507
590
  end
508
591
  ```
509
592
 
510
- Note: This is not 100% the same as `NOT NULL` column constraint before Postgres 12. Here’s a [good explanation](https://medium.com/doctolib/adding-a-not-null-constraint-on-pg-faster-with-minimal-locking-38b2c00c4d1c).
511
-
512
593
  ### Keeping non-unique indexes to three columns or less
513
594
 
514
595
  #### Bad
@@ -516,7 +597,7 @@ Note: This is not 100% the same as `NOT NULL` column constraint before Postgres
516
597
  Adding a non-unique index with more than three columns rarely improves performance.
517
598
 
518
599
  ```ruby
519
- class AddSomeIndexToUsers < ActiveRecord::Migration[6.0]
600
+ class AddSomeIndexToUsers < ActiveRecord::Migration[7.0]
520
601
  def change
521
602
  add_index :users, [:a, :b, :c, :d]
522
603
  end
@@ -528,7 +609,7 @@ end
528
609
  Instead, start an index with columns that narrow down the results the most.
529
610
 
530
611
  ```ruby
531
- class AddSomeIndexToUsers < ActiveRecord::Migration[6.0]
612
+ class AddSomeIndexToUsers < ActiveRecord::Migration[7.0]
532
613
  def change
533
614
  add_index :users, [:b, :d]
534
615
  end
@@ -542,7 +623,7 @@ For Postgres, be sure to add them concurrently.
542
623
  To mark a step in the migration as safe, despite using a method that might otherwise be dangerous, wrap it in a `safety_assured` block.
543
624
 
544
625
  ```ruby
545
- class MySafeMigration < ActiveRecord::Migration[6.0]
626
+ class MySafeMigration < ActiveRecord::Migration[7.0]
546
627
  def change
547
628
  safety_assured { remove_column :users, :some_column }
548
629
  end
@@ -551,6 +632,21 @@ end
551
632
 
552
633
  Certain methods like `execute` and `change_table` cannot be inspected and are prevented from running by default. Make sure what you’re doing is really safe and use this pattern.
553
634
 
635
+ ## Safe by Default
636
+
637
+ Make operations safe by default.
638
+
639
+ - adding and removing an index
640
+ - adding a foreign key
641
+ - adding a check constraint
642
+ - setting NOT NULL on an existing column
643
+
644
+ Add to `config/initializers/strong_migrations.rb`:
645
+
646
+ ```ruby
647
+ StrongMigrations.safe_by_default = true
648
+ ```
649
+
554
650
  ## Custom Checks
555
651
 
556
652
  Add your own custom checks with:
@@ -587,6 +683,14 @@ StrongMigrations.disable_check(:add_index)
587
683
 
588
684
  Check the [source code](https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations.rb) for the list of keys.
589
685
 
686
+ ## Down Migrations / Rollbacks
687
+
688
+ By default, checks are disabled when migrating down. Enable them with:
689
+
690
+ ```ruby
691
+ StrongMigrations.check_down = true
692
+ ```
693
+
590
694
  ## Custom Messages
591
695
 
592
696
  To customize specific messages, create an initializer with:
@@ -595,9 +699,9 @@ To customize specific messages, create an initializer with:
595
699
  StrongMigrations.error_messages[:add_column_default] = "Your custom instructions"
596
700
  ```
597
701
 
598
- Check the [source code](https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations.rb) for the list of keys.
702
+ Check the [source code](https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations/error_messages.rb) for the list of keys.
599
703
 
600
- ## Timeouts
704
+ ## Migration Timeouts
601
705
 
602
706
  It’s extremely important to set a short lock timeout for migrations. This way, if a migration can’t acquire a lock in a timely manner, other statements won’t be stuck behind it. We also recommend setting a long statement timeout so migrations can run for a while.
603
707
 
@@ -617,6 +721,68 @@ ALTER ROLE myuser SET statement_timeout = '1h';
617
721
 
618
722
  Note: If you use PgBouncer in transaction mode, you must set timeouts on the database user.
619
723
 
724
+ ## Lock Timeout Retries [experimental]
725
+
726
+ There’s the option to automatically retry statements when the lock timeout is reached. Here’s how it works:
727
+
728
+ - If a lock timeout happens outside a transaction, the statement is retried
729
+ - If it happens inside the DDL transaction, the entire migration is retried (only applicable to Postgres)
730
+
731
+ Add to `config/initializers/strong_migrations.rb`:
732
+
733
+ ```ruby
734
+ StrongMigrations.lock_timeout_retries = 3
735
+ ```
736
+
737
+ Set the delay between retries with:
738
+
739
+ ```ruby
740
+ StrongMigrations.lock_timeout_retry_delay = 10.seconds
741
+ ```
742
+
743
+ ## App Timeouts
744
+
745
+ We recommend adding timeouts to `config/database.yml` to prevent connections from hanging and individual queries from taking up too many resources in controllers, jobs, the Rails console, and other places.
746
+
747
+ For Postgres:
748
+
749
+ ```yml
750
+ production:
751
+ connect_timeout: 5
752
+ variables:
753
+ statement_timeout: 15s
754
+ lock_timeout: 10s
755
+ ```
756
+
757
+ Note: If you use PgBouncer in transaction mode, you must set the statement and lock timeouts on the database user as shown above.
758
+
759
+ For MySQL:
760
+
761
+ ```yml
762
+ production:
763
+ connect_timeout: 5
764
+ read_timeout: 5
765
+ write_timeout: 5
766
+ variables:
767
+ max_execution_time: 15000 # ms
768
+ lock_wait_timeout: 10 # sec
769
+
770
+ ```
771
+
772
+ For MariaDB:
773
+
774
+ ```yml
775
+ production:
776
+ connect_timeout: 5
777
+ read_timeout: 5
778
+ write_timeout: 5
779
+ variables:
780
+ max_statement_time: 15 # sec
781
+ lock_wait_timeout: 10 # sec
782
+ ```
783
+
784
+ For HTTP connections, Redis, and other services, check out [this guide](https://github.com/ankane/the-ultimate-guide-to-ruby-timeouts).
785
+
620
786
  ## Existing Migrations
621
787
 
622
788
  To mark migrations as safe that were created before installing this gem, create an initializer with:
@@ -632,11 +798,11 @@ Use the version from your latest migration.
632
798
  If your development database version is different from production, you can specify the production version so the right checks run in development.
633
799
 
634
800
  ```ruby
635
- StrongMigrations.target_postgresql_version = "10"
636
- StrongMigrations.target_mysql_version = "8.0.12"
637
- StrongMigrations.target_mariadb_version = "10.3.2"
801
+ StrongMigrations.target_version = 10 # or "8.0.12", "10.3.2", etc
638
802
  ```
639
803
 
804
+ The major version works well for Postgres, while the full version is recommended for MySQL and MariaDB.
805
+
640
806
  For safety, this option only affects development and test environments. In other environments, the actual server version is always used.
641
807
 
642
808
  ## Analyze Tables
@@ -649,11 +815,15 @@ StrongMigrations.auto_analyze = true
649
815
 
650
816
  ## Faster Migrations
651
817
 
652
- Only dump the schema when adding a new migration. If you use Git, create an initializer with:
818
+ Only dump the schema when adding a new migration. If you use Git, add to the end of your `Rakefile`:
653
819
 
654
- ```ruby
655
- ActiveRecord::Base.dump_schema_after_migration = Rails.env.development? &&
656
- `git status db/migrate/ --porcelain`.present?
820
+ ```rb
821
+ task :faster_migrations do
822
+ ActiveRecord::Base.dump_schema_after_migration = Rails.env.development? &&
823
+ `git status db/migrate/ --porcelain`.present?
824
+ end
825
+
826
+ task "db:migrate": "faster_migrations"
657
827
  ```
658
828
 
659
829
  ## Schema Sanity
@@ -664,22 +834,20 @@ Columns can flip order in `db/schema.rb` when you have multiple developers. One
664
834
  task "db:schema:dump": "strong_migrations:alphabetize_columns"
665
835
  ```
666
836
 
667
- ## Dangerous Tasks
668
-
669
- For safety, dangerous database tasks are disabled in production - `db:drop`, `db:reset`, `db:schema:load`, and `db:structure:load`. To get around this, use:
670
-
671
- ```sh
672
- SAFETY_ASSURED=1 rails db:drop
673
- ```
674
-
675
837
  ## Permissions
676
838
 
677
839
  We recommend using a [separate database user](https://ankane.org/postgres-users) for migrations when possible so you don’t need to grant your app user permission to alter tables.
678
840
 
841
+ ## Smaller Projects
842
+
843
+ You probably don’t need this gem for smaller projects, as operations that are unsafe at scale can be perfectly safe on smaller, low-traffic tables.
844
+
679
845
  ## Additional Reading
680
846
 
681
847
  - [Rails Migrations with No Downtime](https://pedro.herokuapp.com/past/2011/7/13/rails_migrations_with_no_downtime/)
682
848
  - [PostgreSQL at Scale: Database Schema Changes Without Downtime](https://medium.com/braintree-product-technology/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680)
849
+ - [An Overview of DDL Algorithms in MySQL](https://mydbops.wordpress.com/2020/03/04/an-overview-of-ddl-algorithms-in-mysql-covers-mysql-8/)
850
+ - [MariaDB InnoDB Online DDL Overview](https://mariadb.com/kb/en/innodb-online-ddl-overview/)
683
851
 
684
852
  ## Credits
685
853