strong_migrations 0.3.1 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +6 -0
- data/LICENSE.txt +1 -1
- data/README.md +313 -81
- data/lib/strong_migrations/migration.rb +59 -20
- data/lib/strong_migrations/version.rb +1 -1
- data/lib/strong_migrations.rb +21 -15
- metadata +10 -11
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: d4a27bb71eb7436386540b0f86fb17d3bf82f2e100eff8e836acb842d22ffab6
|
4
|
+
data.tar.gz: 81b1f63c48a92599e7d421adbbd22411e26a467c052520dd1ee0982d2e150f6a
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 27f6188cf4fdb6391a3f5e7926e292029c9ae47260010349ee93fa908d7abd74a40b2ac6d9bc7970f1e9c0b35c85c60b0c8719b125ee32dac3903190823483cf
|
7
|
+
data.tar.gz: 5682113d7f4a56a11cf9c9dfe3bcefb373c101737bf8073970f1a56eb87efeb79e0a648c6908d7f8f77adf3ed68690a7b4523a43abef149ef09b8cff945ea8e2
|
data/CHANGELOG.md
CHANGED
data/LICENSE.txt
CHANGED
data/README.md
CHANGED
@@ -16,51 +16,88 @@ gem 'strong_migrations'
|
|
16
16
|
|
17
17
|
## How It Works
|
18
18
|
|
19
|
-
Strong Migrations detects potentially dangerous operations in migrations, prevents them from running by default, and provides instructions on safer ways to do what you want.
|
19
|
+
Strong Migrations detects potentially dangerous operations in migrations, prevents them from running by default, and provides instructions on safer ways to do what you want.
|
20
20
|
|
21
|
-
|
22
|
-
=== Dangerous operation detected #strong_migrations ===
|
21
|
+

|
23
22
|
|
24
|
-
|
25
|
-
when removing columns. Be sure to ignore the column:
|
23
|
+
## Dangerous Operations
|
26
24
|
|
27
|
-
|
28
|
-
|
29
|
-
|
25
|
+
The following operations can cause downtime or errors:
|
26
|
+
|
27
|
+
- [[+]](#removing-a-column) removing a column
|
28
|
+
- [[+]](#adding-a-column-with-a-default-value) adding a column with a non-null default value to an existing table
|
29
|
+
- [[+]](#backfilling-data) backfilling data
|
30
|
+
- [[+]](#adding-an-index) adding an index non-concurrently
|
31
|
+
- [[+]](#adding-a-reference) adding a reference
|
32
|
+
- [[+]](#adding-a-foreign-key) adding a foreign key
|
33
|
+
- [[+]](#renaming-or-changing-the-type-of-a-column) changing the type of a column
|
34
|
+
- [[+]](#renaming-or-changing-the-type-of-a-column) renaming a column
|
35
|
+
- [[+]](#renaming-a-table) renaming a table
|
36
|
+
- [[+]](#creating-a-table-with-the-force-option) creating a table with the `force` option
|
37
|
+
- [[+]](#using-change_column_null-with-a-default-value) using `change_column_null` with a default value
|
38
|
+
- [[+]](#adding-a-json-column) adding a `json` column
|
39
|
+
|
40
|
+
Also checks for best practices:
|
30
41
|
|
31
|
-
|
42
|
+
- [[+]](#) keeping non-unique indexes to three columns or less
|
32
43
|
|
33
|
-
|
44
|
+
## The Zero Downtime Way
|
45
|
+
|
46
|
+
### Removing a column
|
47
|
+
|
48
|
+
#### Bad
|
49
|
+
|
50
|
+
ActiveRecord caches database columns at runtime, so if you drop a column, it can cause exceptions until your app reboots.
|
51
|
+
|
52
|
+
```ruby
|
53
|
+
class RemoveSomeColumnFromUsers < ActiveRecord::Migration[5.2]
|
34
54
|
def change
|
35
|
-
|
55
|
+
remove_column :users, :some_column
|
36
56
|
end
|
37
57
|
end
|
38
58
|
```
|
39
59
|
|
40
|
-
|
60
|
+
#### Good
|
41
61
|
|
42
|
-
|
62
|
+
1. Tell ActiveRecord to ignore the column from its cache
|
43
63
|
|
44
|
-
|
45
|
-
|
46
|
-
|
47
|
-
|
48
|
-
|
49
|
-
- renaming a table
|
50
|
-
- creating a table with the `force` option
|
51
|
-
- adding an index non-concurrently (Postgres only)
|
52
|
-
- adding a `json` column to an existing table (Postgres only)
|
64
|
+
```ruby
|
65
|
+
class User < ApplicationRecord
|
66
|
+
self.ignored_columns = ["some_column"]
|
67
|
+
end
|
68
|
+
```
|
53
69
|
|
54
|
-
|
70
|
+
2. Deploy code
|
71
|
+
3. Write a migration to remove the column (wrap in `safety_assured` block)
|
55
72
|
|
56
|
-
|
73
|
+
```ruby
|
74
|
+
class RemoveSomeColumnFromUsers < ActiveRecord::Migration[5.2]
|
75
|
+
def change
|
76
|
+
safety_assured { remove_column :users, :some_column }
|
77
|
+
end
|
78
|
+
end
|
79
|
+
```
|
57
80
|
|
58
|
-
|
81
|
+
4. Deploy and run migration
|
59
82
|
|
60
83
|
### Adding a column with a default value
|
61
84
|
|
85
|
+
#### Bad
|
86
|
+
|
62
87
|
Adding a column with a non-null default causes the entire table to be rewritten.
|
63
88
|
|
89
|
+
```ruby
|
90
|
+
class AddSomeColumnToUsers < ActiveRecord::Migration[5.2]
|
91
|
+
def change
|
92
|
+
add_column :users, :some_column, :text, default: "default_value"
|
93
|
+
end
|
94
|
+
end
|
95
|
+
```
|
96
|
+
|
97
|
+
> This operation is safe in Postgres 11+
|
98
|
+
|
99
|
+
#### Good
|
100
|
+
|
64
101
|
Instead, add the column without a default value, then change the default.
|
65
102
|
|
66
103
|
```ruby
|
@@ -76,65 +113,191 @@ class AddSomeColumnToUsers < ActiveRecord::Migration[5.2]
|
|
76
113
|
end
|
77
114
|
```
|
78
115
|
|
79
|
-
|
80
|
-
|
81
|
-
> With Postgres, this operation is safe as of Postgres 11
|
116
|
+
See the next section for how to backfill.
|
82
117
|
|
83
118
|
### Backfilling data
|
84
119
|
|
85
|
-
|
120
|
+
#### Bad
|
121
|
+
|
122
|
+
Backfilling in the same transaction that alters a table locks the table for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
|
123
|
+
|
124
|
+
```ruby
|
125
|
+
class AddSomeColumnToUsers < ActiveRecord::Migration[5.2]
|
126
|
+
def change
|
127
|
+
add_column :users, :some_column, :text
|
128
|
+
User.update_all some_column: "default_value"
|
129
|
+
end
|
130
|
+
end
|
131
|
+
```
|
132
|
+
|
133
|
+
Also, running a single query to update data can cause issues for large tables.
|
134
|
+
|
135
|
+
#### Good
|
136
|
+
|
137
|
+
There are three keys: batching, throttling, and running it outside a transaction. Use the Rails console or a separate migration with `disable_ddl_transaction!`.
|
86
138
|
|
87
139
|
```ruby
|
88
140
|
class BackfillSomeColumn < ActiveRecord::Migration[5.2]
|
89
141
|
disable_ddl_transaction!
|
90
142
|
|
91
143
|
def change
|
92
|
-
|
93
|
-
|
94
|
-
|
95
|
-
# Rails < 5
|
96
|
-
User.find_in_batches do |records|
|
97
|
-
User.where(id: records.map(&:id)).update_all some_column: "default_value"
|
144
|
+
User.in_batches do |relation|
|
145
|
+
relation.update_all some_column: "default_value"
|
146
|
+
sleep(0.1) # throttle
|
98
147
|
end
|
99
148
|
end
|
100
149
|
end
|
101
150
|
```
|
102
151
|
|
103
|
-
###
|
152
|
+
### Adding an index
|
104
153
|
|
105
|
-
|
154
|
+
#### Bad
|
106
155
|
|
107
|
-
|
156
|
+
In Postgres, adding a non-concurrent indexes lock the table.
|
108
157
|
|
109
|
-
|
110
|
-
|
111
|
-
|
112
|
-
|
158
|
+
```ruby
|
159
|
+
class AddSomeIndexToUsers < ActiveRecord::Migration[5.2]
|
160
|
+
def change
|
161
|
+
add_index :users, :some_column
|
113
162
|
end
|
163
|
+
end
|
164
|
+
```
|
114
165
|
|
115
|
-
|
116
|
-
|
117
|
-
|
118
|
-
|
119
|
-
|
166
|
+
#### Good
|
167
|
+
|
168
|
+
Add indexes concurrently.
|
169
|
+
|
170
|
+
```ruby
|
171
|
+
class AddSomeIndexToUsers < ActiveRecord::Migration[5.2]
|
172
|
+
disable_ddl_transaction!
|
173
|
+
|
174
|
+
def change
|
175
|
+
add_index :users, :some_column, algorithm: :concurrently
|
120
176
|
end
|
121
|
-
|
177
|
+
end
|
178
|
+
```
|
122
179
|
|
123
|
-
|
124
|
-
3. Write a migration to remove the column (wrap in `safety_assured` block)
|
180
|
+
If you forget `disable_ddl_transaction!`, the migration will fail. Also, note that indexes on new tables (those created in the same migration) don’t require this. Check out [gindex](https://github.com/ankane/gindex) to quickly generate index migrations without memorizing the syntax.
|
125
181
|
|
126
|
-
|
127
|
-
|
128
|
-
|
129
|
-
|
182
|
+
### Adding a reference
|
183
|
+
|
184
|
+
#### Bad
|
185
|
+
|
186
|
+
Rails adds a non-concurrent index to references by default, which is problematic for Postgres.
|
187
|
+
|
188
|
+
```ruby
|
189
|
+
class AddReferenceToUsers < ActiveRecord::Migration[5.2]
|
190
|
+
def change
|
191
|
+
add_reference :users, :city
|
192
|
+
end
|
193
|
+
end
|
194
|
+
```
|
195
|
+
|
196
|
+
#### Good
|
197
|
+
|
198
|
+
Make sure the index is added concurrently.
|
199
|
+
|
200
|
+
```ruby
|
201
|
+
class AddReferenceToUsers < ActiveRecord::Migration[5.2]
|
202
|
+
disable_ddl_transaction!
|
203
|
+
|
204
|
+
def change
|
205
|
+
add_reference :users, :city, index: false
|
206
|
+
add_index :users, :city_id, algorithm: :concurrently
|
207
|
+
end
|
208
|
+
end
|
209
|
+
```
|
210
|
+
|
211
|
+
For polymorphic references, add a compound index on type and id.
|
212
|
+
|
213
|
+
### Adding a foreign key
|
214
|
+
|
215
|
+
#### Bad
|
216
|
+
|
217
|
+
In Postgres, new foreign keys are validated by default, which acquires an `AccessExclusiveLock` that can be [expensive on large tables](https://travisofthenorth.com/blog/2017/2/2/postgres-adding-foreign-keys-with-zero-downtime).
|
218
|
+
|
219
|
+
```ruby
|
220
|
+
class AddForeignKeyOnUsers < ActiveRecord::Migration[5.2]
|
221
|
+
def change
|
222
|
+
add_foreign_key :users, :orders
|
223
|
+
end
|
224
|
+
end
|
225
|
+
```
|
226
|
+
|
227
|
+
#### Good
|
228
|
+
|
229
|
+
Instead, validate it in a separate migration with a more agreeable `RowShareLock`. This approach is documented by Postgres to have “[the least impact on other work](https://www.postgresql.org/docs/current/sql-altertable.html).”
|
230
|
+
|
231
|
+
For Rails 5.2+, use:
|
232
|
+
|
233
|
+
```ruby
|
234
|
+
class AddForeignKeyOnUsers < ActiveRecord::Migration[5.2]
|
235
|
+
def change
|
236
|
+
add_foreign_key :users, :orders, validate: false
|
237
|
+
end
|
238
|
+
end
|
239
|
+
```
|
240
|
+
|
241
|
+
Then validate it in a separate migration.
|
242
|
+
|
243
|
+
```ruby
|
244
|
+
class ValidateForeignKeyOnUsers < ActiveRecord::Migration[5.2]
|
245
|
+
def change
|
246
|
+
validate_foreign_key :users, :orders
|
247
|
+
end
|
248
|
+
end
|
249
|
+
```
|
250
|
+
|
251
|
+
For Rails < 5.2, use:
|
252
|
+
|
253
|
+
```ruby
|
254
|
+
class AddForeignKeyOnUsers < ActiveRecord::Migration[5.1]
|
255
|
+
def change
|
256
|
+
safety_assured do
|
257
|
+
execute 'ALTER TABLE "users" ADD CONSTRAINT "fk_rails_c1e9b98e31" FOREIGN KEY ("order_id") REFERENCES "orders" ("id") NOT VALID'
|
130
258
|
end
|
131
259
|
end
|
132
|
-
|
260
|
+
end
|
261
|
+
```
|
133
262
|
|
134
|
-
|
263
|
+
Then validate it in a separate migration.
|
264
|
+
|
265
|
+
```ruby
|
266
|
+
class ValidateForeignKeyOnUsers < ActiveRecord::Migration[5.1]
|
267
|
+
def change
|
268
|
+
safety_assured do
|
269
|
+
execute 'ALTER TABLE "users" VALIDATE CONSTRAINT "fk_rails_c1e9b98e31"'
|
270
|
+
end
|
271
|
+
end
|
272
|
+
end
|
273
|
+
```
|
135
274
|
|
136
275
|
### Renaming or changing the type of a column
|
137
276
|
|
277
|
+
#### Bad
|
278
|
+
|
279
|
+
```ruby
|
280
|
+
class RenameSomeColumn < ActiveRecord::Migration[5.2]
|
281
|
+
def change
|
282
|
+
rename_column :users, :some_column, :new_name
|
283
|
+
end
|
284
|
+
end
|
285
|
+
```
|
286
|
+
|
287
|
+
or
|
288
|
+
|
289
|
+
```ruby
|
290
|
+
class ChangeSomeColumnType < ActiveRecord::Migration[5.2]
|
291
|
+
def change
|
292
|
+
change_column :users, :some_column, :new_type
|
293
|
+
end
|
294
|
+
end
|
295
|
+
```
|
296
|
+
|
297
|
+
One exception is changing a `varchar` column to `text`, which is safe in Postgres.
|
298
|
+
|
299
|
+
#### Good
|
300
|
+
|
138
301
|
A safer approach is to:
|
139
302
|
|
140
303
|
1. Create a new column
|
@@ -144,10 +307,20 @@ A safer approach is to:
|
|
144
307
|
5. Stop writing to the old column
|
145
308
|
6. Drop the old column
|
146
309
|
|
147
|
-
One exception is changing a `varchar` column to `text`, which is safe in Postgres 9.1+.
|
148
|
-
|
149
310
|
### Renaming a table
|
150
311
|
|
312
|
+
#### Bad
|
313
|
+
|
314
|
+
```ruby
|
315
|
+
class RenameUsersToCustomers < ActiveRecord::Migration[5.2]
|
316
|
+
def change
|
317
|
+
rename_table :users, :customers
|
318
|
+
end
|
319
|
+
end
|
320
|
+
```
|
321
|
+
|
322
|
+
#### Good
|
323
|
+
|
151
324
|
A safer approach is to:
|
152
325
|
|
153
326
|
1. Create a new table
|
@@ -157,64 +330,119 @@ A safer approach is to:
|
|
157
330
|
5. Stop writing to the old table
|
158
331
|
6. Drop the old table
|
159
332
|
|
160
|
-
###
|
333
|
+
### Creating a table with the `force` option
|
161
334
|
|
162
|
-
|
335
|
+
#### Bad
|
336
|
+
|
337
|
+
The `force` option can drop an existing table.
|
163
338
|
|
164
339
|
```ruby
|
165
|
-
class
|
166
|
-
|
340
|
+
class CreateUsers < ActiveRecord::Migration[5.2]
|
341
|
+
def change
|
342
|
+
create_table :users, force: true do |t|
|
343
|
+
# ...
|
344
|
+
end
|
345
|
+
end
|
346
|
+
end
|
347
|
+
```
|
348
|
+
|
349
|
+
#### Good
|
350
|
+
|
351
|
+
If you intend to drop a table, do it explicitly. Then create the new table without the `force` option:
|
167
352
|
|
353
|
+
```ruby
|
354
|
+
class CreateUsers < ActiveRecord::Migration[5.2]
|
168
355
|
def change
|
169
|
-
|
356
|
+
create_table :users do |t|
|
357
|
+
# ...
|
358
|
+
end
|
170
359
|
end
|
171
360
|
end
|
172
361
|
```
|
173
362
|
|
174
|
-
|
363
|
+
### Using `change_column_null` with a default value
|
364
|
+
|
365
|
+
#### Bad
|
175
366
|
|
176
|
-
|
367
|
+
This generates a single `UPDATE` statement to set the default value.
|
177
368
|
|
178
369
|
```ruby
|
179
|
-
class
|
180
|
-
|
370
|
+
class ChangeSomeColumnNull < ActiveRecord::Migration[5.2]
|
371
|
+
def change
|
372
|
+
change_column_null :users, :some_column, false, "default_value"
|
373
|
+
end
|
374
|
+
end
|
375
|
+
```
|
181
376
|
|
377
|
+
#### Good
|
378
|
+
|
379
|
+
Backfill the column [safely](#backfilling-data). Then use:
|
380
|
+
|
381
|
+
```ruby
|
382
|
+
class ChangeSomeColumnNull < ActiveRecord::Migration[5.2]
|
182
383
|
def change
|
183
|
-
|
184
|
-
add_index :users, :reference_id, algorithm: :concurrently
|
384
|
+
change_column_null :users, :some_column, false
|
185
385
|
end
|
186
386
|
end
|
187
387
|
```
|
188
388
|
|
189
|
-
|
389
|
+
### Adding a json column
|
190
390
|
|
191
|
-
|
391
|
+
#### Bad
|
192
392
|
|
193
|
-
|
393
|
+
In Postgres, there’s no equality operator for the `json` column type, which causes issues for `SELECT DISTINCT` queries.
|
394
|
+
|
395
|
+
```ruby
|
396
|
+
class AddPropertiesToUsers < ActiveRecord::Migration[5.2]
|
397
|
+
def change
|
398
|
+
add_column :users, :properties, :json
|
399
|
+
end
|
400
|
+
end
|
401
|
+
```
|
194
402
|
|
195
|
-
|
403
|
+
#### Good
|
196
404
|
|
197
|
-
|
405
|
+
Use `jsonb` instead.
|
198
406
|
|
199
407
|
```ruby
|
200
|
-
class
|
201
|
-
|
408
|
+
class AddPropertiesToUsers < ActiveRecord::Migration[5.2]
|
409
|
+
def change
|
410
|
+
add_column :users, :properties, :jsonb
|
411
|
+
end
|
202
412
|
end
|
203
413
|
```
|
204
414
|
|
205
|
-
|
415
|
+
## Best Practices
|
416
|
+
|
417
|
+
### Keeping non-unique indexes to three columns or less
|
418
|
+
|
419
|
+
#### Bad
|
420
|
+
|
421
|
+
Adding an index with more than three columns only helps on extremely large tables.
|
206
422
|
|
207
423
|
```ruby
|
208
|
-
class
|
424
|
+
class AddSomeIndexToUsers < ActiveRecord::Migration[5.2]
|
209
425
|
def change
|
210
|
-
|
426
|
+
add_index :users, [:a, :b, :c, :d]
|
211
427
|
end
|
212
428
|
end
|
213
429
|
```
|
214
430
|
|
431
|
+
#### Good
|
432
|
+
|
433
|
+
```ruby
|
434
|
+
class AddSomeIndexToUsers < ActiveRecord::Migration[5.2]
|
435
|
+
def change
|
436
|
+
add_index :users, [:a, :b, :c]
|
437
|
+
end
|
438
|
+
end
|
439
|
+
```
|
440
|
+
|
441
|
+
> For Postgres, be sure to add them concurrently
|
442
|
+
|
215
443
|
## Assuring Safety
|
216
444
|
|
217
|
-
To mark a step in the migration as safe, despite using method that might otherwise be dangerous, wrap it in a `safety_assured` block.
|
445
|
+
To mark a step in the migration as safe, despite using a method that might otherwise be dangerous, wrap it in a `safety_assured` block.
|
218
446
|
|
219
447
|
```ruby
|
220
448
|
class MySafeMigration < ActiveRecord::Migration[5.2]
|
@@ -224,6 +452,8 @@ class MySafeMigration < ActiveRecord::Migration[5.2]
|
|
224
452
|
end
|
225
453
|
```
|
226
454
|
|
455
|
+
Certain methods like `execute` and `change_table` cannot be inspected and are prevented from running by default. Make sure what you’re doing is really safe and use this pattern.
|
456
|
+
|
227
457
|
## Custom Checks
|
228
458
|
|
229
459
|
Add your own custom checks with:
|
@@ -238,6 +468,8 @@ end
|
|
238
468
|
|
239
469
|
Use the `stop!` method to stop migrations.
|
240
470
|
|
471
|
+
> Since `remove_column` always requires a `safety_assured` block, it’s not possible to add a custom check for `remove_column` operations
|
472
|
+
|
241
473
|
## Existing Migrations
|
242
474
|
|
243
475
|
To mark migrations as safe that were created before installing this gem, create an initializer with:
|
@@ -312,7 +544,7 @@ Rails 5.1+ uses `bigint` for primary keys to keep you from running out of ids. T
|
|
312
544
|
|
313
545
|
## Credits
|
314
546
|
|
315
|
-
Thanks to Bob Remeika and David Waller for the [original code](https://github.com/foobarfighter/safe-migrations).
|
547
|
+
Thanks to Bob Remeika and David Waller for the [original code](https://github.com/foobarfighter/safe-migrations) and [Sean Huber](https://github.com/LendingHome/zero_downtime_migrations) for the bad/good readme format.
|
316
548
|
|
317
549
|
## Contributing
|
318
550
|
|
@@ -15,8 +15,6 @@ module StrongMigrations
|
|
15
15
|
|
16
16
|
def method_missing(method, *args, &block)
|
17
17
|
unless @safe || ENV["SAFETY_ASSURED"] || is_a?(ActiveRecord::Schema) || @direction == :down || version_safe?
|
18
|
-
ar5 = ActiveRecord::VERSION::MAJOR >= 5
|
19
|
-
|
20
18
|
case method
|
21
19
|
when :remove_column, :remove_columns, :remove_timestamps, :remove_reference, :remove_belongs_to
|
22
20
|
columns =
|
@@ -36,7 +34,7 @@ module StrongMigrations
|
|
36
34
|
cols
|
37
35
|
end
|
38
36
|
|
39
|
-
code =
|
37
|
+
code = "self.ignored_columns = #{columns.inspect}"
|
40
38
|
|
41
39
|
raise_error :remove_column,
|
42
40
|
model: args[0].to_s.classify,
|
@@ -65,21 +63,30 @@ module StrongMigrations
|
|
65
63
|
default = options[:default]
|
66
64
|
|
67
65
|
if !default.nil? && !(postgresql? && postgresql_version >= 110000)
|
66
|
+
|
67
|
+
if options[:null] == false
|
68
|
+
options = options.except(:null)
|
69
|
+
append = "
|
70
|
+
|
71
|
+
Then add the NOT NULL constraint.
|
72
|
+
|
73
|
+
class %{migration_name}NotNull < ActiveRecord::Migration%{migration_suffix}
|
74
|
+
def change
|
75
|
+
#{command_str("change_column_null", [table, column, false])}
|
76
|
+
end
|
77
|
+
end"
|
78
|
+
end
|
79
|
+
|
68
80
|
raise_error :add_column_default,
|
69
81
|
add_command: command_str("add_column", [table, column, type, options.except(:default)]),
|
70
82
|
change_command: command_str("change_column_default", [table, column, default]),
|
71
83
|
remove_command: command_str("remove_column", [table, column]),
|
72
|
-
code: backfill_code(table, column, default)
|
84
|
+
code: backfill_code(table, column, default),
|
85
|
+
append: append
|
73
86
|
end
|
74
87
|
|
75
88
|
if type.to_s == "json" && postgresql?
|
76
|
-
|
77
|
-
raise_error :add_column_json
|
78
|
-
else
|
79
|
-
raise_error :add_column_json_legacy,
|
80
|
-
model: table.to_s.classify,
|
81
|
-
table: connection.quote_table_name(table.to_s)
|
82
|
-
end
|
89
|
+
raise_error :add_column_json
|
83
90
|
end
|
84
91
|
when :change_column
|
85
92
|
table, column, type = args
|
@@ -103,7 +110,7 @@ module StrongMigrations
|
|
103
110
|
table, reference, options = args
|
104
111
|
options ||= {}
|
105
112
|
|
106
|
-
index_value = options.fetch(:index,
|
113
|
+
index_value = options.fetch(:index, true)
|
107
114
|
if postgresql? && index_value
|
108
115
|
columns = options[:polymorphic] ? [:"#{reference}_type", :"#{reference}_id"] : :"#{reference}_id"
|
109
116
|
|
@@ -119,6 +126,32 @@ module StrongMigrations
|
|
119
126
|
raise_error :change_column_null,
|
120
127
|
code: backfill_code(table, column, default)
|
121
128
|
end
|
129
|
+
when :add_foreign_key
|
130
|
+
from_table, to_table, options = args
|
131
|
+
options ||= {}
|
132
|
+
validate = options.fetch(:validate, true)
|
133
|
+
|
134
|
+
if postgresql?
|
135
|
+
if ActiveRecord::VERSION::STRING >= "5.2"
|
136
|
+
if validate
|
137
|
+
raise_error :add_foreign_key,
|
138
|
+
add_foreign_key_code: command_str("add_foreign_key", [from_table, to_table, options.merge(validate: false)]),
|
139
|
+
validate_foreign_key_code: command_str("validate_foreign_key", [from_table, to_table])
|
140
|
+
end
|
141
|
+
else
|
142
|
+
# always validated before 5.2
|
143
|
+
|
144
|
+
# fk name logic from rails
|
145
|
+
primary_key = options[:primary_key] || "id"
|
146
|
+
column = options[:column] || "#{to_table.to_s.singularize}_id"
|
147
|
+
hashed_identifier = Digest::SHA256.hexdigest("#{from_table}_#{column}_fk").first(10)
|
148
|
+
fk_name = options[:name] || "fk_rails_#{hashed_identifier}"
|
149
|
+
|
150
|
+
raise_error :add_foreign_key,
|
151
|
+
add_foreign_key_code: foreign_key_str("ALTER TABLE %s ADD CONSTRAINT %s FOREIGN KEY (%s) REFERENCES %s (%s) NOT VALID", [from_table, fk_name, column, to_table, primary_key]),
|
152
|
+
validate_foreign_key_code: foreign_key_str("ALTER TABLE %s VALIDATE CONSTRAINT %s", [from_table, fk_name])
|
153
|
+
end
|
154
|
+
end
|
122
155
|
end
|
123
156
|
|
124
157
|
StrongMigrations.checks.each do |check|
|
@@ -152,15 +185,25 @@ module StrongMigrations
|
|
152
185
|
def raise_error(message_key, header: nil, **vars)
|
153
186
|
message = StrongMigrations.error_messages[message_key] || "Missing message"
|
154
187
|
|
155
|
-
ar5 = ActiveRecord::VERSION::MAJOR >= 5
|
156
188
|
vars[:migration_name] = self.class.name
|
157
|
-
vars[:migration_suffix] =
|
158
|
-
vars[:base_model] =
|
189
|
+
vars[:migration_suffix] = "[#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}]"
|
190
|
+
vars[:base_model] = "ApplicationRecord"
|
191
|
+
|
192
|
+
# interpolate variables in appended code
|
193
|
+
if vars[:append]
|
194
|
+
vars[:append] = vars[:append].gsub(/%(?!{)/, "%%") % vars
|
195
|
+
end
|
159
196
|
|
160
197
|
# escape % not followed by {
|
161
198
|
stop!(message.gsub(/%(?!{)/, "%%") % vars, header: header || "Dangerous operation detected")
|
162
199
|
end
|
163
200
|
|
201
|
+
def foreign_key_str(statement, identifiers)
|
202
|
+
# not all identifiers are tables, but this method of quoting should be fine
|
203
|
+
code = statement % identifiers.map { |v| connection.quote_table_name(v) }
|
204
|
+
"safety_assured do\n execute '#{code}' \n end"
|
205
|
+
end
|
206
|
+
|
164
207
|
def command_str(command, args)
|
165
208
|
str_args = args[0..-2].map { |a| a.inspect }
|
166
209
|
|
@@ -179,11 +222,7 @@ module StrongMigrations
|
|
179
222
|
|
180
223
|
def backfill_code(table, column, default)
|
181
224
|
model = table.to_s.classify
|
182
|
-
|
183
|
-
"#{model}.in_batches.update_all #{column}: #{default.inspect}"
|
184
|
-
else
|
185
|
-
"#{model}.find_in_batches do |records|\n #{model}.where(id: records.map(&:id)).update_all #{column}: #{default.inspect}\n end"
|
186
|
-
end
|
225
|
+
"#{model}.in_batches do |relation| \n relation.update_all #{column}: #{default.inspect}\n sleep(0.1)\n end"
|
187
226
|
end
|
188
227
|
|
189
228
|
def stop!(message, header: "Custom check")
|
data/lib/strong_migrations.rb
CHANGED
@@ -37,23 +37,12 @@ class Backfill%{migration_name} < ActiveRecord::Migration%{migration_suffix}
|
|
37
37
|
def change
|
38
38
|
%{code}
|
39
39
|
end
|
40
|
-
end",
|
40
|
+
end%{append}",
|
41
41
|
|
42
42
|
add_column_json:
|
43
43
|
"There's no equality operator for the json column type, which
|
44
44
|
causes issues for SELECT DISTINCT queries. Use jsonb instead.",
|
45
45
|
|
46
|
-
add_column_json_legacy:
|
47
|
-
"There's no equality operator for the json column type, which.
|
48
|
-
causes issues for SELECT DISTINCT queries.
|
49
|
-
Replace all calls to uniq with a custom scope.
|
50
|
-
|
51
|
-
class %{model} < %{base_model}
|
52
|
-
scope :uniq_on_id, -> { select('DISTINCT ON (%{table}.id) %{table}.*') }
|
53
|
-
end
|
54
|
-
|
55
|
-
Once it's deployed, wrap this step in a safety_assured { ... } block.",
|
56
|
-
|
57
46
|
change_column:
|
58
47
|
"Changing the type of an existing column requires the entire
|
59
48
|
table and indexes to be rewritten. A safer approach is to:
|
@@ -93,7 +82,7 @@ end",
|
|
93
82
|
rename_table:
|
94
83
|
"Renaming a table is dangerous. A safer approach is to:
|
95
84
|
|
96
|
-
1. Create a new table
|
85
|
+
1. Create a new table. Don't forget to recreate indexes from the old table
|
97
86
|
2. Write to both tables
|
98
87
|
3. Backfill data from the old table to new table
|
99
88
|
4. Move reads from the old table to the new table
|
@@ -143,7 +132,7 @@ Otherwise, remove the force option.",
|
|
143
132
|
execute call, so cannot help you here. Please make really sure that what
|
144
133
|
you're doing is safe before proceeding, then wrap it in a safety_assured { ... } block.",
|
145
134
|
|
146
|
-
|
135
|
+
change_column_null:
|
147
136
|
"Passing a default value to change_column_null runs a single UPDATE query,
|
148
137
|
which can cause downtime. Instead, backfill the existing rows in the
|
149
138
|
Rails console or a separate migration with disable_ddl_transaction!.
|
@@ -154,7 +143,24 @@ class Backfill%{migration_name} < ActiveRecord::Migration%{migration_suffix}
|
|
154
143
|
def change
|
155
144
|
%{code}
|
156
145
|
end
|
157
|
-
end"
|
146
|
+
end",
|
147
|
+
|
148
|
+
add_foreign_key:
|
149
|
+
"New foreign keys are validated by default. This acquires an AccessExclusiveLock,
|
150
|
+
which is expensive on large tables. Instead, validate it in a separate migration
|
151
|
+
with a more agreeable RowShareLock.
|
152
|
+
|
153
|
+
class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
|
154
|
+
def change
|
155
|
+
%{add_foreign_key_code}
|
156
|
+
end
|
157
|
+
end
|
158
|
+
|
159
|
+
class Validate%{migration_name} < ActiveRecord::Migration%{migration_suffix}
|
160
|
+
def change
|
161
|
+
%{validate_foreign_key_code}
|
162
|
+
end
|
163
|
+
end",
|
158
164
|
}
|
159
165
|
|
160
166
|
def self.add_check(&block)
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: strong_migrations
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.4.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Andrew Kane
|
@@ -10,7 +10,7 @@ authors:
|
|
10
10
|
autorequire:
|
11
11
|
bindir: bin
|
12
12
|
cert_chain: []
|
13
|
-
date:
|
13
|
+
date: 2019-05-27 00:00:00.000000000 Z
|
14
14
|
dependencies:
|
15
15
|
- !ruby/object:Gem::Dependency
|
16
16
|
name: activerecord
|
@@ -18,14 +18,14 @@ dependencies:
|
|
18
18
|
requirements:
|
19
19
|
- - ">="
|
20
20
|
- !ruby/object:Gem::Version
|
21
|
-
version:
|
21
|
+
version: '5'
|
22
22
|
type: :runtime
|
23
23
|
prerelease: false
|
24
24
|
version_requirements: !ruby/object:Gem::Requirement
|
25
25
|
requirements:
|
26
26
|
- - ">="
|
27
27
|
- !ruby/object:Gem::Version
|
28
|
-
version:
|
28
|
+
version: '5'
|
29
29
|
- !ruby/object:Gem::Dependency
|
30
30
|
name: bundler
|
31
31
|
requirement: !ruby/object:Gem::Requirement
|
@@ -72,16 +72,16 @@ dependencies:
|
|
72
72
|
name: pg
|
73
73
|
requirement: !ruby/object:Gem::Requirement
|
74
74
|
requirements:
|
75
|
-
- - "
|
75
|
+
- - ">="
|
76
76
|
- !ruby/object:Gem::Version
|
77
|
-
version: '
|
77
|
+
version: '0'
|
78
78
|
type: :development
|
79
79
|
prerelease: false
|
80
80
|
version_requirements: !ruby/object:Gem::Requirement
|
81
81
|
requirements:
|
82
|
-
- - "
|
82
|
+
- - ">="
|
83
83
|
- !ruby/object:Gem::Version
|
84
|
-
version: '
|
84
|
+
version: '0'
|
85
85
|
description:
|
86
86
|
email:
|
87
87
|
- andrew@chartkick.com
|
@@ -114,15 +114,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
114
114
|
requirements:
|
115
115
|
- - ">="
|
116
116
|
- !ruby/object:Gem::Version
|
117
|
-
version: '2.
|
117
|
+
version: '2.4'
|
118
118
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
119
119
|
requirements:
|
120
120
|
- - ">="
|
121
121
|
- !ruby/object:Gem::Version
|
122
122
|
version: '0'
|
123
123
|
requirements: []
|
124
|
-
|
125
|
-
rubygems_version: 2.7.7
|
124
|
+
rubygems_version: 3.0.3
|
126
125
|
signing_key:
|
127
126
|
specification_version: 4
|
128
127
|
summary: Catch unsafe migrations at dev time
|