strong_migrations 0.6.6 → 0.7.2

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 140b74a0133d3db034a54fc178e4f95e53fbdec0c569dc2ade222efc96b83a6b
4
- data.tar.gz: f4aadca5f15f85849b975630bdb937e3eef00498c0e806f6ca7cd8112d859fd4
3
+ metadata.gz: f3410e3379861436cbec2b46796069db388fffea222fa50ff46b054a153fbdaf
4
+ data.tar.gz: f328bee4412388114e888557d9b691ed3dad10bd0a5bf1ffb93c09f0b48d6696
5
5
  SHA512:
6
- metadata.gz: 2f79d4dbd4342e45af9799f4d303c8ad8af68e8aa24fc5e86667547ec2a4c8ce95e123a6b3f0eb1851573d7a1bebd89a6795f4afffd4a319a0a9f0d73293f3ee
7
- data.tar.gz: a784f11afffd45d827d7135d1f8fb822979ba851f6d8b54e09025ff9144f32f5091dce8e080901bac51237be74894571c6d007eddf373e3007de4886f5391b54
6
+ metadata.gz: 7e588f76685aa486f38949629b2f0aebe5f1acf92a5b53a29b4e4e1f5f334439eec3ef915240736caf55cff1f21197369305ec073c6adbbedaf6477aabb282cb
7
+ data.tar.gz: 3e350c8a68ec076295d36aac330ff9eae5ad75329b9775fb8839319f7fbc99c62b8ea58ca5fc57ec36c750a510f9ae743a76fd6fedc7fd755d1e07b7ab7230b5
@@ -1,3 +1,30 @@
1
+ ## 0.7.2 (2020-10-25)
2
+
3
+ - Added support for float timeouts
4
+
5
+ ## 0.7.1 (2020-07-27)
6
+
7
+ - Added `target_version` option to replace database-specific options
8
+
9
+ ## 0.7.0 (2020-07-22)
10
+
11
+ - Added `check_down` option
12
+ - Added check for `change_column` with `null: false`
13
+ - Added check for `validate_foreign_key`
14
+ - Improved error messages
15
+ - Made auto analyze less verbose in Postgres
16
+ - Decreasing the length limit of a `varchar` column or adding a limit is not safe in Postgres
17
+ - Removed safety checks for `db` rake tasks (Rails 5+ handles this)
18
+
19
+ ## 0.6.8 (2020-05-13)
20
+
21
+ - `change_column_null` on a column with a `NOT NULL` constraint is safe in Postgres 12+
22
+
23
+ ## 0.6.7 (2020-05-13)
24
+
25
+ - Improved comments in initializer
26
+ - Fixed string timeouts for Postgres
27
+
1
28
  ## 0.6.6 (2020-05-08)
2
29
 
3
30
  - Added warnings for missing and long lock timeouts
@@ -1,4 +1,4 @@
1
- Copyright (c) 2013 Bob Remeika and David Waller, 2015-2019 Andrew Kane
1
+ Copyright (c) 2013 Bob Remeika and David Waller, 2015-2020 Andrew Kane
2
2
 
3
3
  MIT License
4
4
 
data/README.md CHANGED
@@ -21,9 +21,40 @@ gem 'strong_migrations'
21
21
  And run:
22
22
 
23
23
  ```sh
24
+ bundle install
24
25
  rails generate strong_migrations:install
25
26
  ```
26
27
 
28
+ Strong Migrations sets a long statement timeout for migrations so you can set a [short statement timeout](#app-timeouts) for your application.
29
+
30
+ ## How It Works
31
+
32
+ When you run a migration that’s potentially dangerous, you’ll see an error message like:
33
+
34
+ ```txt
35
+ === Dangerous operation detected #strong_migrations ===
36
+
37
+ Active Record caches attributes, which causes problems
38
+ when removing columns. Be sure to ignore the column:
39
+
40
+ class User < ApplicationRecord
41
+ self.ignored_columns = ["name"]
42
+ end
43
+
44
+ Deploy the code, then wrap this step in a safety_assured { ... } block.
45
+
46
+ class RemoveColumn < ActiveRecord::Migration[6.0]
47
+ def change
48
+ safety_assured { remove_column :users, :name }
49
+ end
50
+ end
51
+ ```
52
+
53
+ An operation is classified as dangerous if it either:
54
+
55
+ - Blocks reads or writes for more than a few seconds (after a lock is acquired)
56
+ - Has a good chance of causing application errors
57
+
27
58
  ## Checks
28
59
 
29
60
  Potentially dangerous operations:
@@ -31,21 +62,19 @@ Potentially dangerous operations:
31
62
  - [removing a column](#removing-a-column)
32
63
  - [adding a column with a default value](#adding-a-column-with-a-default-value)
33
64
  - [backfilling data](#backfilling-data)
34
- - [changing the type of a column](#renaming-or-changing-the-type-of-a-column)
35
- - [renaming a column](#renaming-or-changing-the-type-of-a-column)
65
+ - [changing the type of a column](#changing-the-type-of-a-column)
66
+ - [renaming a column](#renaming-a-column)
36
67
  - [renaming a table](#renaming-a-table)
37
68
  - [creating a table with the force option](#creating-a-table-with-the-force-option)
38
- - [using change_column_null with a default value](#using-change_column_null-with-a-default-value)
69
+ - [setting NOT NULL on an existing column](#setting-not-null-on-an-existing-column)
39
70
  - [executing SQL directly](#executing-SQL-directly)
40
71
 
41
72
  Postgres-specific checks:
42
73
 
43
- - [adding an index non-concurrently](#adding-an-index)
44
- - [removing an index non-concurrently](#removing-an-index)
74
+ - [adding an index non-concurrently](#adding-an-index-non-concurrently)
45
75
  - [adding a reference](#adding-a-reference)
46
76
  - [adding a foreign key](#adding-a-foreign-key)
47
77
  - [adding a json column](#adding-a-json-column)
48
- - [setting NOT NULL on an existing column](#setting-not-null-on-an-existing-column)
49
78
 
50
79
  Best practices:
51
80
 
@@ -57,7 +86,7 @@ You can also add [custom checks](#custom-checks) or [disable specific checks](#d
57
86
 
58
87
  #### Bad
59
88
 
60
- ActiveRecord caches database columns at runtime, so if you drop a column, it can cause exceptions until your app reboots.
89
+ Active Record caches database columns at runtime, so if you drop a column, it can cause exceptions until your app reboots.
61
90
 
62
91
  ```ruby
63
92
  class RemoveSomeColumnFromUsers < ActiveRecord::Migration[6.0]
@@ -69,7 +98,7 @@ end
69
98
 
70
99
  #### Good
71
100
 
72
- 1. Tell ActiveRecord to ignore the column from its cache
101
+ 1. Tell Active Record to ignore the column from its cache
73
102
 
74
103
  ```ruby
75
104
  class User < ApplicationRecord
@@ -92,11 +121,9 @@ end
92
121
 
93
122
  ### Adding a column with a default value
94
123
 
95
- Note: This operation is safe in Postgres 11+, MySQL 8.0.12+, and MariaDB 10.3.2+.
96
-
97
124
  #### Bad
98
125
 
99
- Adding a column with a default value to an existing table causes the entire table to be rewritten.
126
+ In earlier versions of Postgres, MySQL, and MariaDB, adding a column with a default value to an existing table causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
100
127
 
101
128
  ```ruby
102
129
  class AddSomeColumnToUsers < ActiveRecord::Migration[6.0]
@@ -106,6 +133,8 @@ class AddSomeColumnToUsers < ActiveRecord::Migration[6.0]
106
133
  end
107
134
  ```
108
135
 
136
+ In Postgres 11+, MySQL 8.0.12+, and MariaDB 10.3.2+, this no longer requires a table rewrite and is safe.
137
+
109
138
  #### Good
110
139
 
111
140
  Instead, add the column without a default value, then change the default.
@@ -129,7 +158,7 @@ See the next section for how to backfill.
129
158
 
130
159
  #### Bad
131
160
 
132
- Backfilling in the same transaction that alters a table locks the table for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
161
+ Active Record creates a transaction around each migration, and backfilling in the same transaction that alters a table keeps the table locked for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
133
162
 
134
163
  ```ruby
135
164
  class AddSomeColumnToUsers < ActiveRecord::Migration[6.0]
@@ -159,19 +188,11 @@ class BackfillSomeColumn < ActiveRecord::Migration[6.0]
159
188
  end
160
189
  ```
161
190
 
162
- ### Renaming or changing the type of a column
191
+ ### Changing the type of a column
163
192
 
164
193
  #### Bad
165
194
 
166
- ```ruby
167
- class RenameSomeColumn < ActiveRecord::Migration[6.0]
168
- def change
169
- rename_column :users, :some_column, :new_name
170
- end
171
- end
172
- ```
173
-
174
- or
195
+ Changing the type of a column causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
175
196
 
176
197
  ```ruby
177
198
  class ChangeSomeColumnType < ActiveRecord::Migration[6.0]
@@ -181,17 +202,44 @@ class ChangeSomeColumnType < ActiveRecord::Migration[6.0]
181
202
  end
182
203
  ```
183
204
 
184
- A few changes are safe in Postgres:
205
+ A few changes don’t require a table rewrite (and are safe) in Postgres:
185
206
 
186
- - Changing between `varchar` and `text` columns
207
+ - Increasing the length limit of a `varchar` column (or removing the limit)
208
+ - Changing a `varchar` column to a `text` column
209
+ - Changing a `text` column to a `varchar` column with no length limit
187
210
  - Increasing the precision of a `decimal` or `numeric` column
188
211
  - Making a `decimal` or `numeric` column unconstrained
189
212
  - Changing between `timestamp` and `timestamptz` columns when session time zone is UTC in Postgres 12+
190
213
 
191
214
  And a few in MySQL and MariaDB:
192
215
 
193
- - Increasing the length of a `varchar` column from under 255 up to 255
194
- - Increasing the length of a `varchar` column over 255
216
+ - Increasing the length limit of a `varchar` column from under 255 up to 255
217
+ - Increasing the length limit of a `varchar` column from over 255 to the max limit
218
+
219
+ #### Good
220
+
221
+ A safer approach is to:
222
+
223
+ 1. Create a new column
224
+ 2. Write to both columns
225
+ 3. Backfill data from the old column to the new column
226
+ 4. Move reads from the old column to the new column
227
+ 5. Stop writing to the old column
228
+ 6. Drop the old column
229
+
230
+ ### Renaming a column
231
+
232
+ #### Bad
233
+
234
+ Renaming a column that’s in use will cause errors in your application.
235
+
236
+ ```ruby
237
+ class RenameSomeColumn < ActiveRecord::Migration[6.0]
238
+ def change
239
+ rename_column :users, :some_column, :new_name
240
+ end
241
+ end
242
+ ```
195
243
 
196
244
  #### Good
197
245
 
@@ -208,6 +256,8 @@ A safer approach is to:
208
256
 
209
257
  #### Bad
210
258
 
259
+ Renaming a table that’s in use will cause errors in your application.
260
+
211
261
  ```ruby
212
262
  class RenameUsersToCustomers < ActiveRecord::Migration[6.0]
213
263
  def change
@@ -257,33 +307,57 @@ class CreateUsers < ActiveRecord::Migration[6.0]
257
307
  end
258
308
  ```
259
309
 
260
- ### Using change_column_null with a default value
310
+ If you intend to drop an existing table, run `drop_table` first.
311
+
312
+ ### Setting NOT NULL on an existing column
261
313
 
262
314
  #### Bad
263
315
 
264
- This generates a single `UPDATE` statement to set the default value.
316
+ Setting `NOT NULL` on an existing column blocks reads and writes while every row is checked.
265
317
 
266
318
  ```ruby
267
- class ChangeSomeColumnNull < ActiveRecord::Migration[6.0]
319
+ class SetSomeColumnNotNull < ActiveRecord::Migration[6.0]
268
320
  def change
269
- change_column_null :users, :some_column, false, "default_value"
321
+ change_column_null :users, :some_column, false
270
322
  end
271
323
  end
272
324
  ```
273
325
 
274
- #### Good
326
+ #### Good - Postgres
327
+
328
+ Instead, add a check constraint:
329
+
330
+ ```ruby
331
+ class SetSomeColumnNotNull < ActiveRecord::Migration[6.0]
332
+ def change
333
+ safety_assured do
334
+ execute 'ALTER TABLE "users" ADD CONSTRAINT "users_some_column_null" CHECK ("some_column" IS NOT NULL) NOT VALID'
335
+ end
336
+ end
337
+ end
338
+ ```
275
339
 
276
- Backfill the column [safely](#backfilling-data). Then use:
340
+ Then validate it in a separate migration. A `NOT NULL` check constraint is [functionally equivalent](https://medium.com/doctolib/adding-a-not-null-constraint-on-pg-faster-with-minimal-locking-38b2c00c4d1c) to setting `NOT NULL` on the column, but it won’t show up in `schema.rb`. In Postgres 12+, once the check constraint is validated, you can safely set `NOT NULL` on the column and drop the check constraint.
277
341
 
278
342
  ```ruby
279
- class ChangeSomeColumnNull < ActiveRecord::Migration[6.0]
343
+ class ValidateSomeColumnNotNull < ActiveRecord::Migration[6.0]
280
344
  def change
345
+ safety_assured do
346
+ execute 'ALTER TABLE "users" VALIDATE CONSTRAINT "users_some_column_null"'
347
+ end
348
+
349
+ # in Postgres 12+, you can then safely set NOT NULL on the column
281
350
  change_column_null :users, :some_column, false
351
+ safety_assured do
352
+ execute 'ALTER TABLE "users" DROP CONSTRAINT "users_some_column_null"'
353
+ end
282
354
  end
283
355
  end
284
356
  ```
285
357
 
286
- Note: In Postgres, `change_column_null` is still [not safe](#setting-not-null-on-an-existing-column) with this method.
358
+ #### Good - MySQL and MariaDB
359
+
360
+ [Let us know](https://github.com/ankane/strong_migrations/issues/new) if you have a safe way to do this.
287
361
 
288
362
  ### Executing SQL directly
289
363
 
@@ -297,11 +371,11 @@ class ExecuteSQL < ActiveRecord::Migration[6.0]
297
371
  end
298
372
  ```
299
373
 
300
- ### Adding an index
374
+ ### Adding an index non-concurrently
301
375
 
302
376
  #### Bad
303
377
 
304
- In Postgres, adding an index non-concurrently locks the table.
378
+ In Postgres, adding an index non-concurrently blocks writes.
305
379
 
306
380
  ```ruby
307
381
  class AddSomeIndexToUsers < ActiveRecord::Migration[6.0]
@@ -333,41 +407,11 @@ With [gindex](https://github.com/ankane/gindex), you can generate an index migra
333
407
  rails g index table column
334
408
  ```
335
409
 
336
- ### Removing an index
337
-
338
- Note: This check is [opt-in](#opt-in-checks).
339
-
340
- #### Bad
341
-
342
- In Postgres, removing an index non-concurrently locks the table for a brief period.
343
-
344
- ```ruby
345
- class RemoveSomeIndexFromUsers < ActiveRecord::Migration[6.0]
346
- def change
347
- remove_index :users, :some_column
348
- end
349
- end
350
- ```
351
-
352
- #### Good
353
-
354
- Remove indexes concurrently.
355
-
356
- ```ruby
357
- class RemoveSomeIndexFromUsers < ActiveRecord::Migration[6.0]
358
- disable_ddl_transaction!
359
-
360
- def change
361
- remove_index :users, column: :some_column, algorithm: :concurrently
362
- end
363
- end
364
- ```
365
-
366
410
  ### Adding a reference
367
411
 
368
412
  #### Bad
369
413
 
370
- Rails adds an index non-concurrently to references by default, which is problematic for Postgres.
414
+ Rails adds an index non-concurrently to references by default, which blocks writes in Postgres.
371
415
 
372
416
  ```ruby
373
417
  class AddReferenceToUsers < ActiveRecord::Migration[6.0]
@@ -395,7 +439,7 @@ end
395
439
 
396
440
  #### Bad
397
441
 
398
- In Postgres, new foreign keys are validated by default, which acquires a `ShareRowExclusiveLock` that can be [expensive on large tables](https://travisofthenorth.com/blog/2017/2/2/postgres-adding-foreign-keys-with-zero-downtime).
442
+ In Postgres, adding a foreign key blocks writes on both tables.
399
443
 
400
444
  ```ruby
401
445
  class AddForeignKeyOnUsers < ActiveRecord::Migration[6.0]
@@ -417,7 +461,7 @@ end
417
461
 
418
462
  #### Good
419
463
 
420
- Instead, validate it in a separate migration with a more agreeable `RowShareLock`. This approach is documented by Postgres to have “[the least impact on other work](https://www.postgresql.org/docs/current/sql-altertable.html).”
464
+ Add the foreign key without validating existing rows, then validate them in a separate migration.
421
465
 
422
466
  For Rails 5.2+, use:
423
467
 
@@ -429,7 +473,7 @@ class AddForeignKeyOnUsers < ActiveRecord::Migration[6.0]
429
473
  end
430
474
  ```
431
475
 
432
- Then validate it in a separate migration.
476
+ Then:
433
477
 
434
478
  ```ruby
435
479
  class ValidateForeignKeyOnUsers < ActiveRecord::Migration[6.0]
@@ -451,7 +495,7 @@ class AddForeignKeyOnUsers < ActiveRecord::Migration[5.1]
451
495
  end
452
496
  ```
453
497
 
454
- Then validate it in a separate migration.
498
+ Then:
455
499
 
456
500
  ```ruby
457
501
  class ValidateForeignKeyOnUsers < ActiveRecord::Migration[5.1]
@@ -467,7 +511,7 @@ end
467
511
 
468
512
  #### Bad
469
513
 
470
- In Postgres, there’s no equality operator for the `json` column type, which can cause errors for existing `SELECT DISTINCT` queries.
514
+ In Postgres, there’s no equality operator for the `json` column type, which can cause errors for existing `SELECT DISTINCT` queries in your application.
471
515
 
472
516
  ```ruby
473
517
  class AddPropertiesToUsers < ActiveRecord::Migration[6.0]
@@ -489,48 +533,6 @@ class AddPropertiesToUsers < ActiveRecord::Migration[6.0]
489
533
  end
490
534
  ```
491
535
 
492
- ### Setting NOT NULL on an existing column
493
-
494
- #### Bad
495
-
496
- In Postgres, setting `NOT NULL` on an existing column requires an `AccessExclusiveLock`, which is expensive on large tables.
497
-
498
- ```ruby
499
- class SetSomeColumnNotNull < ActiveRecord::Migration[6.0]
500
- def change
501
- change_column_null :users, :some_column, false
502
- end
503
- end
504
- ```
505
-
506
- #### Good
507
-
508
- Instead, add a constraint:
509
-
510
- ```ruby
511
- class SetSomeColumnNotNull < ActiveRecord::Migration[6.0]
512
- def change
513
- safety_assured do
514
- execute 'ALTER TABLE "users" ADD CONSTRAINT "users_some_column_null" CHECK ("some_column" IS NOT NULL) NOT VALID'
515
- end
516
- end
517
- end
518
- ```
519
-
520
- Then validate it in a separate migration.
521
-
522
- ```ruby
523
- class ValidateSomeColumnNotNull < ActiveRecord::Migration[6.0]
524
- def change
525
- safety_assured do
526
- execute 'ALTER TABLE "users" VALIDATE CONSTRAINT "users_some_column_null"'
527
- end
528
- end
529
- end
530
- ```
531
-
532
- Note: This is not 100% the same as `NOT NULL` column constraint. Here’s a [good explanation](https://medium.com/doctolib/adding-a-not-null-constraint-on-pg-faster-with-minimal-locking-38b2c00c4d1c).
533
-
534
536
  ### Keeping non-unique indexes to three columns or less
535
537
 
536
538
  #### Bad
@@ -591,16 +593,12 @@ Note: Since `remove_column` always requires a `safety_assured` block, it’s not
591
593
 
592
594
  ## Opt-in Checks
593
595
 
594
- Some operations rarely cause issues in practice, but can be checked if desired. Enable checks with:
596
+ ### Removing an index non-concurrently
595
597
 
596
- ```ruby
597
- StrongMigrations.enable_check(:remove_index)
598
- ```
599
-
600
- To start a check only after a specific migration, use:
598
+ Postgres supports removing indexes concurrently, but removing them non-concurrently shouldn’t be an issue for most applications. You can enable this check with:
601
599
 
602
600
  ```ruby
603
- StrongMigrations.enable_check(:remove_index, start_after: 20170101000000)
601
+ StrongMigrations.enable_check(:remove_index)
604
602
  ```
605
603
 
606
604
  ## Disable Checks
@@ -613,6 +611,14 @@ StrongMigrations.disable_check(:add_index)
613
611
 
614
612
  Check the [source code](https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations.rb) for the list of keys.
615
613
 
614
+ ## Down Migrations / Rollbacks
615
+
616
+ By default, checks are disabled when migrating down. Enable them with:
617
+
618
+ ```ruby
619
+ StrongMigrations.check_down = true
620
+ ```
621
+
616
622
  ## Custom Messages
617
623
 
618
624
  To customize specific messages, create an initializer with:
@@ -623,7 +629,7 @@ StrongMigrations.error_messages[:add_column_default] = "Your custom instructions
623
629
 
624
630
  Check the [source code](https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations.rb) for the list of keys.
625
631
 
626
- ## Timeouts
632
+ ## Migration Timeouts
627
633
 
628
634
  It’s extremely important to set a short lock timeout for migrations. This way, if a migration can’t acquire a lock in a timely manner, other statements won’t be stuck behind it. We also recommend setting a long statement timeout so migrations can run for a while.
629
635
 
@@ -643,6 +649,49 @@ ALTER ROLE myuser SET statement_timeout = '1h';
643
649
 
644
650
  Note: If you use PgBouncer in transaction mode, you must set timeouts on the database user.
645
651
 
652
+ ## App Timeouts
653
+
654
+ We recommend adding timeouts to `config/database.yml` to prevent connections from hanging and individual queries from taking up too many resources in controllers, jobs, the Rails console, and other places.
655
+
656
+ For Postgres:
657
+
658
+ ```yml
659
+ production:
660
+ connect_timeout: 5
661
+ variables:
662
+ statement_timeout: 15s
663
+ lock_timeout: 10s
664
+ ```
665
+
666
+ Note: If you use PgBouncer in transaction mode, you must set the statement and lock timeouts on the database user as shown above.
667
+
668
+ For MySQL:
669
+
670
+ ```yml
671
+ production:
672
+ connect_timeout: 5
673
+ read_timeout: 5
674
+ write_timeout: 5
675
+ variables:
676
+ max_execution_time: 15000 # ms
677
+ lock_wait_timeout: 10 # sec
678
+
679
+ ```
680
+
681
+ For MariaDB:
682
+
683
+ ```yml
684
+ production:
685
+ connect_timeout: 5
686
+ read_timeout: 5
687
+ write_timeout: 5
688
+ variables:
689
+ max_statement_time: 15 # sec
690
+ lock_wait_timeout: 10 # sec
691
+ ```
692
+
693
+ For HTTP connections, Redis, and other services, check out [this guide](https://github.com/ankane/the-ultimate-guide-to-ruby-timeouts).
694
+
646
695
  ## Existing Migrations
647
696
 
648
697
  To mark migrations as safe that were created before installing this gem, create an initializer with:
@@ -658,11 +707,11 @@ Use the version from your latest migration.
658
707
  If your development database version is different from production, you can specify the production version so the right checks run in development.
659
708
 
660
709
  ```ruby
661
- StrongMigrations.target_postgresql_version = "10"
662
- StrongMigrations.target_mysql_version = "8.0.12"
663
- StrongMigrations.target_mariadb_version = "10.3.2"
710
+ StrongMigrations.target_version = 10 # or "8.0.12", "10.3.2", etc
664
711
  ```
665
712
 
713
+ The major version works well for Postgres, while the full version is recommended for MySQL and MariaDB.
714
+
666
715
  For safety, this option only affects development and test environments. In other environments, the actual server version is always used.
667
716
 
668
717
  ## Analyze Tables
@@ -690,22 +739,20 @@ Columns can flip order in `db/schema.rb` when you have multiple developers. One
690
739
  task "db:schema:dump": "strong_migrations:alphabetize_columns"
691
740
  ```
692
741
 
693
- ## Dangerous Tasks
694
-
695
- For safety, dangerous database tasks are disabled in production - `db:drop`, `db:reset`, `db:schema:load`, and `db:structure:load`. To get around this, use:
696
-
697
- ```sh
698
- SAFETY_ASSURED=1 rails db:drop
699
- ```
700
-
701
742
  ## Permissions
702
743
 
703
744
  We recommend using a [separate database user](https://ankane.org/postgres-users) for migrations when possible so you don’t need to grant your app user permission to alter tables.
704
745
 
746
+ ## Smaller Projects
747
+
748
+ You probably don’t need this gem for smaller projects, as operations that are unsafe at scale can be perfectly safe on smaller, low-traffic tables.
749
+
705
750
  ## Additional Reading
706
751
 
707
752
  - [Rails Migrations with No Downtime](https://pedro.herokuapp.com/past/2011/7/13/rails_migrations_with_no_downtime/)
708
753
  - [PostgreSQL at Scale: Database Schema Changes Without Downtime](https://medium.com/braintree-product-technology/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680)
754
+ - [An Overview of DDL Algorithms in MySQL](https://mydbops.wordpress.com/2020/03/04/an-overview-of-ddl-algorithms-in-mysql-covers-mysql-8/)
755
+ - [MariaDB InnoDB Online DDL Overview](https://mariadb.com/kb/en/innodb-online-ddl-overview/)
709
756
 
710
757
  ## Credits
711
758
 
@@ -12,6 +12,17 @@ module StrongMigrations
12
12
  def start_after
13
13
  Time.now.utc.strftime("%Y%m%d%H%M%S")
14
14
  end
15
+
16
+ def target_version
17
+ case ActiveRecord::Base.connection_config[:adapter].to_s
18
+ when /mysql/
19
+ # could try to connect to database and check for MariaDB
20
+ # but this should be fine
21
+ '"8.0.12"'
22
+ else
23
+ "10"
24
+ end
25
+ end
15
26
  end
16
27
  end
17
28
  end
@@ -1,14 +1,19 @@
1
1
  # Mark existing migrations as safe
2
2
  StrongMigrations.start_after = <%= start_after %>
3
3
 
4
- # Set timeouts
4
+ # Set timeouts for migrations
5
5
  # If you use PgBouncer in transaction mode, delete these lines and set timeouts on the database user
6
6
  StrongMigrations.lock_timeout = 10.seconds
7
7
  StrongMigrations.statement_timeout = 1.hour
8
8
 
9
- # Analyze tables automatically (to update planner statistics) after an index is added
9
+ # Analyze tables after indexes are added
10
+ # Outdated statistics can sometimes hurt performance
10
11
  StrongMigrations.auto_analyze = true
11
12
 
13
+ # Set the version of the production database
14
+ # so the right checks are run in development
15
+ # StrongMigrations.target_version = <%= target_version %>
16
+
12
17
  # Add custom checks
13
18
  # StrongMigrations.add_check do |method, args|
14
19
  # if method == :add_index && args[0].to_s == "users"
@@ -5,7 +5,6 @@ require "active_support"
5
5
  require "strong_migrations/checker"
6
6
  require "strong_migrations/database_tasks"
7
7
  require "strong_migrations/migration"
8
- require "strong_migrations/migration_helpers"
9
8
  require "strong_migrations/version"
10
9
 
11
10
  # integrations
@@ -18,7 +17,7 @@ module StrongMigrations
18
17
  class << self
19
18
  attr_accessor :auto_analyze, :start_after, :checks, :error_messages,
20
19
  :target_postgresql_version, :target_mysql_version, :target_mariadb_version,
21
- :enabled_checks, :lock_timeout, :statement_timeout, :helpers
20
+ :enabled_checks, :lock_timeout, :statement_timeout, :check_down, :target_version
22
21
  attr_writer :lock_timeout_limit
23
22
  end
24
23
  self.auto_analyze = false
@@ -26,7 +25,7 @@ module StrongMigrations
26
25
  self.checks = []
27
26
  self.error_messages = {
28
27
  add_column_default:
29
- "Adding a column with a non-null default causes the entire table to be rewritten.
28
+ "Adding a column with a non-null default blocks %{rewrite_blocks} while the entire table is rewritten.
30
29
  Instead, add the column without a default value, then change the default.
31
30
 
32
31
  class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
@@ -51,12 +50,18 @@ class Backfill%{migration_name} < ActiveRecord::Migration%{migration_suffix}
51
50
  end",
52
51
 
53
52
  add_column_json:
54
- "There's no equality operator for the json column type, which can
55
- cause errors for existing SELECT DISTINCT queries. Use jsonb instead.",
53
+ "There's no equality operator for the json column type, which can cause errors for
54
+ existing SELECT DISTINCT queries in your application. Use jsonb instead.
55
+
56
+ class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
57
+ def change
58
+ %{command}
59
+ end
60
+ end",
56
61
 
57
62
  change_column:
58
- "Changing the type of an existing column requires the entire
59
- table and indexes to be rewritten. A safer approach is to:
63
+ "Changing the type of an existing column blocks %{rewrite_blocks}
64
+ while the entire table is rewritten. A safer approach is to:
60
65
 
61
66
  1. Create a new column
62
67
  2. Write to both columns
@@ -65,7 +70,10 @@ table and indexes to be rewritten. A safer approach is to:
65
70
  5. Stop writing to the old column
66
71
  6. Drop the old column",
67
72
 
68
- remove_column: "Active Record caches attributes which causes problems
73
+ change_column_with_not_null:
74
+ "Changing the type is safe, but setting NOT NULL is not.",
75
+
76
+ remove_column: "Active Record caches attributes, which causes problems
69
77
  when removing columns. Be sure to ignore the column%{column_suffix}:
70
78
 
71
79
  class %{model} < %{base_model}
@@ -81,7 +89,8 @@ class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
81
89
  end",
82
90
 
83
91
  rename_column:
84
- "Renaming a column is dangerous. A safer approach is to:
92
+ "Renaming a column that's in use will cause errors
93
+ in your application. A safer approach is to:
85
94
 
86
95
  1. Create a new column
87
96
  2. Write to both columns
@@ -91,7 +100,8 @@ end",
91
100
  6. Drop the old column",
92
101
 
93
102
  rename_table:
94
- "Renaming a table is dangerous. A safer approach is to:
103
+ "Renaming a table that's in use will cause errors
104
+ in your application. A safer approach is to:
95
105
 
96
106
  1. Create a new table. Don't forget to recreate indexes from the old table
97
107
  2. Write to both tables
@@ -112,7 +122,7 @@ class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
112
122
  end",
113
123
 
114
124
  add_index:
115
- "Adding an index non-concurrently locks the table. Instead, use:
125
+ "Adding an index non-concurrently blocks writes. Instead, use:
116
126
 
117
127
  class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
118
128
  disable_ddl_transaction!
@@ -123,7 +133,7 @@ class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
123
133
  end",
124
134
 
125
135
  remove_index:
126
- "Removing an index non-concurrently locks the table. Instead, use:
136
+ "Removing an index non-concurrently blocks writes. Instead, use:
127
137
 
128
138
  class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
129
139
  disable_ddl_transaction!
@@ -166,9 +176,8 @@ class Backfill%{migration_name} < ActiveRecord::Migration%{migration_suffix}
166
176
  end",
167
177
 
168
178
  change_column_null_postgresql:
169
- "Setting NOT NULL on a column requires an AccessExclusiveLock,
170
- which is expensive on large tables. Instead, use a constraint and
171
- validate it in a separate migration with a more agreeable RowShareLock.
179
+ "Setting NOT NULL on an existing column blocks reads and writes while every row is checked.
180
+ Instead, add a check constraint and validate it in a separate migration.
172
181
 
173
182
  class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
174
183
  def change
@@ -182,26 +191,13 @@ class Validate%{migration_name} < ActiveRecord::Migration%{migration_suffix}
182
191
  end
183
192
  end",
184
193
 
185
- change_column_null_postgresql_helper:
186
- "Setting NOT NULL on a column requires an AccessExclusiveLock,
187
- which is expensive on large tables. Instead, we can use a constraint and
188
- validate it in a separate step with a more agreeable RowShareLock.
189
-
190
- class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
191
- disable_ddl_transaction!
192
-
193
- def change
194
- %{command}
195
- end
196
- end",
197
-
198
194
  change_column_null_mysql:
199
195
  "Setting NOT NULL on an existing column is not safe with your database engine.",
200
196
 
201
197
  add_foreign_key:
202
- "New foreign keys are validated by default. This acquires an AccessExclusiveLock,
203
- which is expensive on large tables. Instead, validate it in a separate migration
204
- with a more agreeable RowShareLock.
198
+ "Adding a foreign key blocks writes on both tables. Instead,
199
+ add the foreign key without validating existing rows,
200
+ then validate them in a separate migration.
205
201
 
206
202
  class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
207
203
  def change
@@ -215,21 +211,12 @@ class Validate%{migration_name} < ActiveRecord::Migration%{migration_suffix}
215
211
  end
216
212
  end",
217
213
 
218
- add_foreign_key_helper:
219
- "New foreign keys are validated by default. This acquires an AccessExclusiveLock,
220
- which is expensive on large tables. Instead, we can validate it in a separate step
221
- with a more agreeable RowShareLock.
222
-
223
- class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
224
- disable_ddl_transaction!
225
-
226
- def change
227
- %{command}
228
- end
229
- end",
214
+ validate_foreign_key:
215
+ "Validating a foreign key while writes are blocked is dangerous.
216
+ Use disable_ddl_transaction! or a separate migration."
230
217
  }
231
218
  self.enabled_checks = (error_messages.keys - [:remove_index]).map { |k| [k, {}] }.to_h
232
- self.helpers = false
219
+ self.check_down = false
233
220
 
234
221
  # private
235
222
  def self.developer_env?
@@ -263,13 +250,6 @@ end",
263
250
  false
264
251
  end
265
252
  end
266
-
267
- # def self.enable_helpers
268
- # unless helpers
269
- # ActiveRecord::Migration.include(StrongMigrations::MigrationHelpers)
270
- # self.helpers = true
271
- # end
272
- # end
273
253
  end
274
254
 
275
255
  ActiveSupport.on_load(:active_record) do
@@ -96,11 +96,13 @@ Then add the NOT NULL constraint in separate migrations."
96
96
  change_command: command_str("change_column_default", [table, column, default]),
97
97
  remove_command: command_str("remove_column", [table, column]),
98
98
  code: backfill_code(table, column, default),
99
- append: append
99
+ append: append,
100
+ rewrite_blocks: rewrite_blocks
100
101
  end
101
102
 
102
103
  if type.to_s == "json" && postgresql?
103
- raise_error :add_column_json
104
+ raise_error :add_column_json,
105
+ command: command_str("add_column", [table, column, :jsonb, options])
104
106
  end
105
107
  when :change_column
106
108
  table, column, type, options = args
@@ -109,15 +111,24 @@ Then add the NOT NULL constraint in separate migrations."
109
111
  safe = false
110
112
  existing_column = connection.columns(table).find { |c| c.name.to_s == column.to_s }
111
113
  if existing_column
112
- sql_type = existing_column.sql_type.split("(").first
114
+ existing_type = existing_column.sql_type.split("(").first
113
115
  if postgresql?
114
116
  case type.to_s
115
- when "string", "text"
116
- # safe to change limit for varchar
117
- safe = ["character varying", "text"].include?(sql_type)
117
+ when "string"
118
+ # safe to increase limit or remove it
119
+ # not safe to decrease limit or add a limit
120
+ case existing_type
121
+ when "character varying"
122
+ safe = !options[:limit] || (existing_column.limit && options[:limit] >= existing_column.limit)
123
+ when "text"
124
+ safe = !options[:limit]
125
+ end
126
+ when "text"
127
+ # safe to change varchar to text (and text to text)
128
+ safe = ["character varying", "text"].include?(existing_type)
118
129
  when "numeric", "decimal"
119
130
  # numeric and decimal are equivalent and can be used interchangably
120
- safe = ["numeric", "decimal"].include?(sql_type) &&
131
+ safe = ["numeric", "decimal"].include?(existing_type) &&
121
132
  (
122
133
  (
123
134
  # unconstrained
@@ -130,7 +141,7 @@ Then add the NOT NULL constraint in separate migrations."
130
141
  )
131
142
  )
132
143
  when "datetime", "timestamp", "timestamptz"
133
- safe = ["timestamp without time zone", "timestamp with time zone"].include?(sql_type) &&
144
+ safe = ["timestamp without time zone", "timestamp with time zone"].include?(existing_type) &&
134
145
  postgresql_version >= Gem::Version.new("12") &&
135
146
  connection.select_all("SHOW timezone").first["TimeZone"] == "UTC"
136
147
  end
@@ -142,13 +153,19 @@ Then add the NOT NULL constraint in separate migrations."
142
153
  # increased limit, but doesn't change number of length bytes
143
154
  # 1-255 = 1 byte, 256-65532 = 2 bytes, 65533+ = too big for varchar
144
155
  limit = options[:limit] || 255
145
- safe = ["varchar"].include?(sql_type) &&
156
+ safe = ["varchar"].include?(existing_type) &&
146
157
  limit >= existing_column.limit &&
147
158
  (limit <= 255 || existing_column.limit > 255)
148
159
  end
149
160
  end
150
161
  end
151
- raise_error :change_column unless safe
162
+
163
+ # unsafe to set NOT NULL for safe types
164
+ if safe && existing_column.null && options[:null] == false
165
+ raise_error :change_column_with_not_null
166
+ end
167
+
168
+ raise_error :change_column, rewrite_blocks: rewrite_blocks unless safe
152
169
  when :create_table
153
170
  table, options = args
154
171
  options ||= {}
@@ -176,7 +193,7 @@ Then add the NOT NULL constraint in separate migrations."
176
193
  end
177
194
 
178
195
  if options.delete(:foreign_key)
179
- headline = "Adding a validated foreign key locks the table."
196
+ headline = "Adding a foreign key blocks writes on both tables."
180
197
  append = "
181
198
 
182
199
  Then add the foreign key in separate migrations."
@@ -196,16 +213,25 @@ Then add the foreign key in separate migrations."
196
213
  table, column, null, default = args
197
214
  if !null
198
215
  if postgresql?
199
- if helpers?
200
- raise_error :change_column_null_postgresql_helper,
201
- command: command_str(:add_null_constraint_safely, [table, column])
202
- else
216
+ safe = false
217
+ if postgresql_version >= Gem::Version.new("12")
218
+ # TODO likely need to quote the column in some situations
219
+ safe = constraints(table).any? { |c| c["def"] == "CHECK ((#{column} IS NOT NULL))" }
220
+ end
221
+
222
+ unless safe
203
223
  # match https://github.com/nullobject/rein
204
224
  constraint_name = "#{table}_#{column}_null"
205
225
 
226
+ validate_constraint_code = String.new(constraint_str("ALTER TABLE %s VALIDATE CONSTRAINT %s", [table, constraint_name]))
227
+ if postgresql_version >= Gem::Version.new("12")
228
+ validate_constraint_code << "\n #{command_str(:change_column_null, [table, column, null])}"
229
+ validate_constraint_code << "\n #{constraint_str("ALTER TABLE %s DROP CONSTRAINT %s", [table, constraint_name])}"
230
+ end
231
+
206
232
  raise_error :change_column_null_postgresql,
207
233
  add_constraint_code: constraint_str("ALTER TABLE %s ADD CONSTRAINT %s CHECK (%s IS NOT NULL) NOT VALID", [table, constraint_name, column]),
208
- validate_constraint_code: constraint_str("ALTER TABLE %s VALIDATE CONSTRAINT %s", [table, constraint_name])
234
+ validate_constraint_code: validate_constraint_code
209
235
  end
210
236
  elsif mysql? || mariadb?
211
237
  raise_error :change_column_null_mysql
@@ -222,10 +248,7 @@ Then add the foreign key in separate migrations."
222
248
  validate = options.fetch(:validate, true) || ActiveRecord::VERSION::STRING < "5.2"
223
249
 
224
250
  if postgresql? && validate
225
- if helpers?
226
- raise_error :add_foreign_key_helper,
227
- command: command_str(:add_foreign_key_safely, [from_table, to_table, options])
228
- elsif ActiveRecord::VERSION::STRING < "5.2"
251
+ if ActiveRecord::VERSION::STRING < "5.2"
229
252
  # fk name logic from rails
230
253
  primary_key = options[:primary_key] || "id"
231
254
  column = options[:column] || "#{to_table.to_s.singularize}_id"
@@ -241,6 +264,10 @@ Then add the foreign key in separate migrations."
241
264
  validate_foreign_key_code: command_str("validate_foreign_key", [from_table, to_table])
242
265
  end
243
266
  end
267
+ when :validate_foreign_key
268
+ if postgresql? && writes_blocked?
269
+ raise_error :validate_foreign_key
270
+ end
244
271
  end
245
272
 
246
273
  StrongMigrations.checks.each do |check|
@@ -250,9 +277,10 @@ Then add the foreign key in separate migrations."
250
277
 
251
278
  result = yield
252
279
 
280
+ # outdated statistics + a new index can hurt performance of existing queries
253
281
  if StrongMigrations.auto_analyze && direction == :up && method == :add_index
254
282
  if postgresql?
255
- connection.execute "ANALYZE VERBOSE #{connection.quote_table_name(args[0].to_s)}"
283
+ connection.execute "ANALYZE #{connection.quote_table_name(args[0].to_s)}"
256
284
  elsif mariadb? || mysql?
257
285
  connection.execute "ANALYZE TABLE #{connection.quote_table_name(args[0].to_s)}"
258
286
  end
@@ -261,15 +289,17 @@ Then add the foreign key in separate migrations."
261
289
  result
262
290
  end
263
291
 
264
- # TODO allow string timeouts in 0.7.0
292
+ private
293
+
265
294
  def set_timeouts
266
295
  if !@timeouts_set
267
296
  if StrongMigrations.statement_timeout
268
297
  statement =
269
298
  if postgresql?
270
- "SET statement_timeout TO #{connection.quote(StrongMigrations.statement_timeout.to_i * 1000)}"
299
+ "SET statement_timeout TO #{connection.quote(postgresql_timeout(StrongMigrations.statement_timeout))}"
271
300
  elsif mysql?
272
- "SET max_execution_time = #{connection.quote(StrongMigrations.statement_timeout.to_i * 1000)}"
301
+ # use ceil to prevent no timeout for values under 1 ms
302
+ "SET max_execution_time = #{connection.quote((StrongMigrations.statement_timeout.to_f * 1000).ceil)}"
273
303
  elsif mariadb?
274
304
  "SET max_statement_time = #{connection.quote(StrongMigrations.statement_timeout)}"
275
305
  else
@@ -282,7 +312,7 @@ Then add the foreign key in separate migrations."
282
312
  if StrongMigrations.lock_timeout
283
313
  statement =
284
314
  if postgresql?
285
- "SET lock_timeout TO #{connection.quote(StrongMigrations.lock_timeout.to_i * 1000)}"
315
+ "SET lock_timeout TO #{connection.quote(postgresql_timeout(StrongMigrations.lock_timeout))}"
286
316
  elsif mysql? || mariadb?
287
317
  "SET lock_wait_timeout = #{connection.quote(StrongMigrations.lock_timeout)}"
288
318
  else
@@ -296,8 +326,6 @@ Then add the foreign key in separate migrations."
296
326
  end
297
327
  end
298
328
 
299
- private
300
-
301
329
  def connection
302
330
  @migration.connection
303
331
  end
@@ -307,7 +335,8 @@ Then add the foreign key in separate migrations."
307
335
  end
308
336
 
309
337
  def safe?
310
- @safe || ENV["SAFETY_ASSURED"] || @migration.is_a?(ActiveRecord::Schema) || direction == :down || version_safe?
338
+ @safe || ENV["SAFETY_ASSURED"] || @migration.is_a?(ActiveRecord::Schema) ||
339
+ (direction == :down && !StrongMigrations.check_down) || version_safe?
311
340
  end
312
341
 
313
342
  def version_safe?
@@ -352,6 +381,7 @@ Then add the foreign key in separate migrations."
352
381
  end
353
382
 
354
383
  def target_version(target_version)
384
+ target_version ||= StrongMigrations.target_version
355
385
  version =
356
386
  if target_version && StrongMigrations.developer_env?
357
387
  target_version.to_s
@@ -369,22 +399,25 @@ Then add the foreign key in separate migrations."
369
399
  lock_timeout = connection.select_all("SHOW lock_timeout").first["lock_timeout"]
370
400
  lock_timeout_sec = timeout_to_sec(lock_timeout)
371
401
  if lock_timeout_sec == 0
372
- warn "[strong_migrations] WARNING: No lock timeout set. This is dangerous."
402
+ warn "[strong_migrations] DANGER: No lock timeout set"
373
403
  elsif lock_timeout_sec > limit
374
- warn "[strong_migrations] WARNING: Lock timeout is longer than #{limit} seconds: #{lock_timeout}. This is dangerous."
404
+ warn "[strong_migrations] DANGER: Lock timeout is longer than #{limit} seconds: #{lock_timeout}"
375
405
  end
376
406
  elsif mysql? || mariadb?
377
407
  lock_timeout = connection.select_all("SHOW VARIABLES LIKE 'lock_wait_timeout'").first["Value"]
408
+ # lock timeout is an integer
378
409
  if lock_timeout.to_i > limit
379
- warn "[strong_migrations] WARNING: Lock timeout is longer than #{limit} seconds: #{lock_timeout}. This is dangerous."
410
+ warn "[strong_migrations] DANGER: Lock timeout is longer than #{limit} seconds: #{lock_timeout}"
380
411
  end
381
412
  end
382
413
  @lock_timeout_checked = true
383
414
  end
384
415
  end
385
416
 
417
+ # units: https://www.postgresql.org/docs/current/config-setting.html
386
418
  def timeout_to_sec(timeout)
387
- suffixes = {
419
+ units = {
420
+ "us" => 0.001,
388
421
  "ms" => 1,
389
422
  "s" => 1000,
390
423
  "min" => 1000 * 60,
@@ -392,7 +425,7 @@ Then add the foreign key in separate migrations."
392
425
  "d" => 1000 * 60 * 60 * 24
393
426
  }
394
427
  timeout_ms = timeout.to_i
395
- suffixes.each do |k, v|
428
+ units.each do |k, v|
396
429
  if timeout.end_with?(k)
397
430
  timeout_ms *= v
398
431
  break
@@ -401,8 +434,28 @@ Then add the foreign key in separate migrations."
401
434
  timeout_ms / 1000.0
402
435
  end
403
436
 
404
- def helpers?
405
- StrongMigrations.helpers
437
+ def postgresql_timeout(timeout)
438
+ if timeout.is_a?(String)
439
+ timeout
440
+ else
441
+ # use ceil to prevent no timeout for values under 1 ms
442
+ (timeout.to_f * 1000).ceil
443
+ end
444
+ end
445
+
446
+ def constraints(table_name)
447
+ query = <<~SQL
448
+ SELECT
449
+ conname AS name,
450
+ pg_get_constraintdef(oid) AS def
451
+ FROM
452
+ pg_constraint
453
+ WHERE
454
+ contype = 'c' AND
455
+ convalidated AND
456
+ conrelid = #{connection.quote(connection.quote_table_name(table_name))}::regclass
457
+ SQL
458
+ connection.select_all(query.squish).to_a
406
459
  end
407
460
 
408
461
  def raise_error(message_key, header: nil, append: nil, **vars)
@@ -449,6 +502,23 @@ Then add the foreign key in separate migrations."
449
502
  "#{command} #{str_args.join(", ")}"
450
503
  end
451
504
 
505
+ def writes_blocked?
506
+ query = <<~SQL
507
+ SELECT
508
+ relation::regclass::text
509
+ FROM
510
+ pg_locks
511
+ WHERE
512
+ mode IN ('ShareRowExclusiveLock', 'AccessExclusiveLock') AND
513
+ pid = pg_backend_pid()
514
+ SQL
515
+ connection.select_all(query.squish).any?
516
+ end
517
+
518
+ def rewrite_blocks
519
+ mysql? || mariadb? ? "writes" : "reads and writes"
520
+ end
521
+
452
522
  def backfill_code(table, column, default)
453
523
  model = table.to_s.classify
454
524
  "#{model}.unscoped.in_batches do |relation| \n relation.update_all #{column}: #{default.inspect}\n sleep(0.01)\n end"
@@ -5,10 +5,6 @@ module StrongMigrations
5
5
  class Railtie < Rails::Railtie
6
6
  rake_tasks do
7
7
  load "tasks/strong_migrations.rake"
8
-
9
- ["db:drop", "db:reset", "db:schema:load", "db:structure:load"].each do |t|
10
- Rake::Task[t].enhance ["strong_migrations:safety_assured"]
11
- end
12
8
  end
13
9
  end
14
10
  end
@@ -1,3 +1,3 @@
1
1
  module StrongMigrations
2
- VERSION = "0.6.6"
2
+ VERSION = "0.7.2"
3
3
  end
@@ -1,10 +1,4 @@
1
- # https://nithinbekal.com/posts/safe-rake-tasks
2
-
3
1
  namespace :strong_migrations do
4
- task safety_assured: :environment do
5
- raise "Set SAFETY_ASSURED=1 to run this task in production" if Rails.env.production? && !ENV["SAFETY_ASSURED"]
6
- end
7
-
8
2
  # https://www.pgrs.net/2008/03/13/alphabetize-schema-rb-columns/
9
3
  task :alphabetize_columns do
10
4
  $stderr.puts "Dumping schema"
metadata CHANGED
@@ -1,16 +1,16 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: strong_migrations
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.6.6
4
+ version: 0.7.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Andrew Kane
8
8
  - Bob Remeika
9
9
  - David Waller
10
- autorequire:
10
+ autorequire:
11
11
  bindir: bin
12
12
  cert_chain: []
13
- date: 2020-05-08 00:00:00.000000000 Z
13
+ date: 2020-10-25 00:00:00.000000000 Z
14
14
  dependencies:
15
15
  - !ruby/object:Gem::Dependency
16
16
  name: activerecord
@@ -96,7 +96,7 @@ dependencies:
96
96
  - - ">="
97
97
  - !ruby/object:Gem::Version
98
98
  version: '0'
99
- description:
99
+ description:
100
100
  email:
101
101
  - andrew@chartkick.com
102
102
  - bob.remeika@gmail.com
@@ -115,7 +115,6 @@ files:
115
115
  - lib/strong_migrations/checker.rb
116
116
  - lib/strong_migrations/database_tasks.rb
117
117
  - lib/strong_migrations/migration.rb
118
- - lib/strong_migrations/migration_helpers.rb
119
118
  - lib/strong_migrations/railtie.rb
120
119
  - lib/strong_migrations/version.rb
121
120
  - lib/tasks/strong_migrations.rake
@@ -123,7 +122,7 @@ homepage: https://github.com/ankane/strong_migrations
123
122
  licenses:
124
123
  - MIT
125
124
  metadata: {}
126
- post_install_message:
125
+ post_install_message:
127
126
  rdoc_options: []
128
127
  require_paths:
129
128
  - lib
@@ -138,8 +137,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
138
137
  - !ruby/object:Gem::Version
139
138
  version: '0'
140
139
  requirements: []
141
- rubygems_version: 3.1.2
142
- signing_key:
140
+ rubygems_version: 3.1.4
141
+ signing_key:
143
142
  specification_version: 4
144
143
  summary: Catch unsafe migrations in development
145
144
  test_files: []
@@ -1,117 +0,0 @@
1
- module StrongMigrations
2
- module MigrationHelpers
3
- def add_foreign_key_safely(from_table, to_table, **options)
4
- ensure_postgresql(__method__)
5
- ensure_not_in_transaction(__method__)
6
-
7
- reversible do |dir|
8
- dir.up do
9
- if ActiveRecord::VERSION::STRING >= "5.2"
10
- add_foreign_key(from_table, to_table, options.merge(validate: false))
11
- validate_foreign_key(from_table, to_table)
12
- else
13
- options = connection.foreign_key_options(from_table, to_table, options)
14
- fk_name, column, primary_key = options.values_at(:name, :column, :primary_key)
15
- primary_key ||= "id"
16
-
17
- statement = ["ALTER TABLE %s ADD CONSTRAINT %s FOREIGN KEY (%s) REFERENCES %s (%s)"]
18
- statement << on_delete_update_statement(:delete, options[:on_delete]) if options[:on_delete]
19
- statement << on_delete_update_statement(:update, options[:on_update]) if options[:on_update]
20
- statement << "NOT VALID"
21
-
22
- safety_assured do
23
- execute quote_identifiers(statement.join(" "), [from_table, fk_name, column, to_table, primary_key])
24
- execute quote_identifiers("ALTER TABLE %s VALIDATE CONSTRAINT %s", [from_table, fk_name])
25
- end
26
- end
27
- end
28
-
29
- dir.down do
30
- remove_foreign_key(from_table, to_table)
31
- end
32
- end
33
- end
34
-
35
- def add_null_constraint_safely(table_name, column_name, name: nil)
36
- ensure_postgresql(__method__)
37
- ensure_not_in_transaction(__method__)
38
-
39
- reversible do |dir|
40
- dir.up do
41
- name ||= null_constraint_name(table_name, column_name)
42
-
43
- safety_assured do
44
- execute quote_identifiers("ALTER TABLE %s ADD CONSTRAINT %s CHECK (%s IS NOT NULL) NOT VALID", [table_name, name, column_name])
45
- execute quote_identifiers("ALTER TABLE %s VALIDATE CONSTRAINT %s", [table_name, name])
46
- end
47
- end
48
-
49
- dir.down do
50
- remove_null_constraint_safely(table_name, column_name)
51
- end
52
- end
53
- end
54
-
55
- # removing constraints is safe, but this method is safe to reverse as well
56
- def remove_null_constraint_safely(table_name, column_name, name: nil)
57
- # could also ensure in transaction so it can be reversed
58
- # but that's more of a concern for a reversible migrations check
59
- ensure_postgresql(__method__)
60
-
61
- reversible do |dir|
62
- dir.up do
63
- name ||= null_constraint_name(table_name, column_name)
64
-
65
- safety_assured do
66
- execute quote_identifiers("ALTER TABLE %s DROP CONSTRAINT %s", [table_name, name])
67
- end
68
- end
69
-
70
- dir.down do
71
- add_null_constraint_safely(table_name, column_name)
72
- end
73
- end
74
- end
75
-
76
- private
77
-
78
- def ensure_postgresql(method_name)
79
- raise StrongMigrations::Error, "`#{method_name}` is intended for Postgres only" unless postgresql?
80
- end
81
-
82
- def postgresql?
83
- %w(PostgreSQL PostGIS).include?(connection.adapter_name)
84
- end
85
-
86
- def ensure_not_in_transaction(method_name)
87
- if connection.transaction_open?
88
- raise StrongMigrations::Error, "Cannot run `#{method_name}` inside a transaction. Use `disable_ddl_transaction` to disable the transaction."
89
- end
90
- end
91
-
92
- # match https://github.com/nullobject/rein
93
- def null_constraint_name(table_name, column_name)
94
- "#{table_name}_#{column_name}_null"
95
- end
96
-
97
- def on_delete_update_statement(delete_or_update, action)
98
- on = delete_or_update.to_s.upcase
99
-
100
- case action
101
- when :nullify
102
- "ON #{on} SET NULL"
103
- when :cascade
104
- "ON #{on} CASCADE"
105
- when :restrict
106
- "ON #{on} RESTRICT"
107
- else
108
- # same error message as Active Record
109
- raise "'#{action}' is not supported for :on_update or :on_delete.\nSupported values are: :nullify, :cascade, :restrict"
110
- end
111
- end
112
-
113
- def quote_identifiers(statement, identifiers)
114
- statement % identifiers.map { |v| connection.quote_table_name(v) }
115
- end
116
- end
117
- end