strong_migrations 0.3.1 → 0.4.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +6 -0
- data/LICENSE.txt +1 -1
- data/README.md +313 -81
- data/lib/strong_migrations/migration.rb +59 -20
- data/lib/strong_migrations/version.rb +1 -1
- data/lib/strong_migrations.rb +21 -15
- metadata +10 -11
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: d4a27bb71eb7436386540b0f86fb17d3bf82f2e100eff8e836acb842d22ffab6
|
4
|
+
data.tar.gz: 81b1f63c48a92599e7d421adbbd22411e26a467c052520dd1ee0982d2e150f6a
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 27f6188cf4fdb6391a3f5e7926e292029c9ae47260010349ee93fa908d7abd74a40b2ac6d9bc7970f1e9c0b35c85c60b0c8719b125ee32dac3903190823483cf
|
7
|
+
data.tar.gz: 5682113d7f4a56a11cf9c9dfe3bcefb373c101737bf8073970f1a56eb87efeb79e0a648c6908d7f8f77adf3ed68690a7b4523a43abef149ef09b8cff945ea8e2
|
data/CHANGELOG.md
CHANGED
data/LICENSE.txt
CHANGED
data/README.md
CHANGED
@@ -16,51 +16,88 @@ gem 'strong_migrations'
|
|
16
16
|
|
17
17
|
## How It Works
|
18
18
|
|
19
|
-
Strong Migrations detects potentially dangerous operations in migrations, prevents them from running by default, and provides instructions on safer ways to do what you want.
|
19
|
+
Strong Migrations detects potentially dangerous operations in migrations, prevents them from running by default, and provides instructions on safer ways to do what you want.
|
20
20
|
|
21
|
-
|
22
|
-
=== Dangerous operation detected #strong_migrations ===
|
21
|
+
![Screenshot](https://ankane.org/images/strong-migrations.png)
|
23
22
|
|
24
|
-
|
25
|
-
when removing columns. Be sure to ignore the column:
|
23
|
+
## Dangerous Operations
|
26
24
|
|
27
|
-
|
28
|
-
|
29
|
-
|
25
|
+
The following operations can cause downtime or errors:
|
26
|
+
|
27
|
+
- [[+]](#removing-a-column) removing a column
|
28
|
+
- [[+]](#adding-a-column-with-a-default-value) adding a column with a non-null default value to an existing table
|
29
|
+
- [[+]](#backfilling-data) backfilling data
|
30
|
+
- [[+]](#adding-an-index) adding an index non-concurrently
|
31
|
+
- [[+]](#adding-a-reference) adding a reference
|
32
|
+
- [[+]](#adding-a-foreign-key) adding a foreign key
|
33
|
+
- [[+]](#renaming-or-changing-the-type-of-a-column) changing the type of a column
|
34
|
+
- [[+]](#renaming-or-changing-the-type-of-a-column) renaming a column
|
35
|
+
- [[+]](#renaming-a-table) renaming a table
|
36
|
+
- [[+]](#creating-a-table-with-the-force-option) creating a table with the `force` option
|
37
|
+
- [[+]](#using-change_column_null-with-a-default-value) using `change_column_null` with a default value
|
38
|
+
- [[+]](#adding-a-json-column) adding a `json` column
|
39
|
+
|
40
|
+
Also checks for best practices:
|
30
41
|
|
31
|
-
|
42
|
+
- [[+]](#) keeping non-unique indexes to three columns or less
|
32
43
|
|
33
|
-
|
44
|
+
## The Zero Downtime Way
|
45
|
+
|
46
|
+
### Removing a column
|
47
|
+
|
48
|
+
#### Bad
|
49
|
+
|
50
|
+
ActiveRecord caches database columns at runtime, so if you drop a column, it can cause exceptions until your app reboots.
|
51
|
+
|
52
|
+
```ruby
|
53
|
+
class RemoveSomeColumnFromUsers < ActiveRecord::Migration[5.2]
|
34
54
|
def change
|
35
|
-
|
55
|
+
remove_column :users, :some_column
|
36
56
|
end
|
37
57
|
end
|
38
58
|
```
|
39
59
|
|
40
|
-
|
60
|
+
#### Good
|
41
61
|
|
42
|
-
|
62
|
+
1. Tell ActiveRecord to ignore the column from its cache
|
43
63
|
|
44
|
-
|
45
|
-
|
46
|
-
|
47
|
-
|
48
|
-
|
49
|
-
- renaming a table
|
50
|
-
- creating a table with the `force` option
|
51
|
-
- adding an index non-concurrently (Postgres only)
|
52
|
-
- adding a `json` column to an existing table (Postgres only)
|
64
|
+
```ruby
|
65
|
+
class User < ApplicationRecord
|
66
|
+
self.ignored_columns = ["some_column"]
|
67
|
+
end
|
68
|
+
```
|
53
69
|
|
54
|
-
|
70
|
+
2. Deploy code
|
71
|
+
3. Write a migration to remove the column (wrap in `safety_assured` block)
|
55
72
|
|
56
|
-
|
73
|
+
```ruby
|
74
|
+
class RemoveSomeColumnFromUsers < ActiveRecord::Migration[5.2]
|
75
|
+
def change
|
76
|
+
safety_assured { remove_column :users, :some_column }
|
77
|
+
end
|
78
|
+
end
|
79
|
+
```
|
57
80
|
|
58
|
-
|
81
|
+
4. Deploy and run migration
|
59
82
|
|
60
83
|
### Adding a column with a default value
|
61
84
|
|
85
|
+
#### Bad
|
86
|
+
|
62
87
|
Adding a column with a non-null default causes the entire table to be rewritten.
|
63
88
|
|
89
|
+
```ruby
|
90
|
+
class AddSomeColumnToUsers < ActiveRecord::Migration[5.2]
|
91
|
+
def change
|
92
|
+
add_column :users, :some_column, :text, default: "default_value"
|
93
|
+
end
|
94
|
+
end
|
95
|
+
```
|
96
|
+
|
97
|
+
> This operation is safe in Postgres 11+
|
98
|
+
|
99
|
+
#### Good
|
100
|
+
|
64
101
|
Instead, add the column without a default value, then change the default.
|
65
102
|
|
66
103
|
```ruby
|
@@ -76,65 +113,191 @@ class AddSomeColumnToUsers < ActiveRecord::Migration[5.2]
|
|
76
113
|
end
|
77
114
|
```
|
78
115
|
|
79
|
-
|
80
|
-
|
81
|
-
> With Postgres, this operation is safe as of Postgres 11
|
116
|
+
See the next section for how to backfill.
|
82
117
|
|
83
118
|
### Backfilling data
|
84
119
|
|
85
|
-
|
120
|
+
#### Bad
|
121
|
+
|
122
|
+
Backfilling in the same transaction that alters a table locks the table for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
|
123
|
+
|
124
|
+
```ruby
|
125
|
+
class AddSomeColumnToUsers < ActiveRecord::Migration[5.2]
|
126
|
+
def change
|
127
|
+
add_column :users, :some_column, :text
|
128
|
+
User.update_all some_column: "default_value"
|
129
|
+
end
|
130
|
+
end
|
131
|
+
```
|
132
|
+
|
133
|
+
Also, running a single query to update data can cause issues for large tables.
|
134
|
+
|
135
|
+
#### Good
|
136
|
+
|
137
|
+
There are three keys: batching, throttling, and running it outside a transaction. Use the Rails console or a separate migration with `disable_ddl_transaction!`.
|
86
138
|
|
87
139
|
```ruby
|
88
140
|
class BackfillSomeColumn < ActiveRecord::Migration[5.2]
|
89
141
|
disable_ddl_transaction!
|
90
142
|
|
91
143
|
def change
|
92
|
-
|
93
|
-
|
94
|
-
|
95
|
-
# Rails < 5
|
96
|
-
User.find_in_batches do |records|
|
97
|
-
User.where(id: records.map(&:id)).update_all some_column: "default_value"
|
144
|
+
User.in_batches do |relation|
|
145
|
+
relation.update_all some_column: "default_value"
|
146
|
+
sleep(0.1) # throttle
|
98
147
|
end
|
99
148
|
end
|
100
149
|
end
|
101
150
|
```
|
102
151
|
|
103
|
-
###
|
152
|
+
### Adding an index
|
104
153
|
|
105
|
-
|
154
|
+
#### Bad
|
106
155
|
|
107
|
-
|
156
|
+
In Postgres, adding a non-concurrent indexes lock the table.
|
108
157
|
|
109
|
-
|
110
|
-
|
111
|
-
|
112
|
-
|
158
|
+
```ruby
|
159
|
+
class AddSomeIndexToUsers < ActiveRecord::Migration[5.2]
|
160
|
+
def change
|
161
|
+
add_index :users, :some_column
|
113
162
|
end
|
163
|
+
end
|
164
|
+
```
|
114
165
|
|
115
|
-
|
116
|
-
|
117
|
-
|
118
|
-
|
119
|
-
|
166
|
+
#### Good
|
167
|
+
|
168
|
+
Add indexes concurrently.
|
169
|
+
|
170
|
+
```ruby
|
171
|
+
class AddSomeIndexToUsers < ActiveRecord::Migration[5.2]
|
172
|
+
disable_ddl_transaction!
|
173
|
+
|
174
|
+
def change
|
175
|
+
add_index :users, :some_column, algorithm: :concurrently
|
120
176
|
end
|
121
|
-
|
177
|
+
end
|
178
|
+
```
|
122
179
|
|
123
|
-
|
124
|
-
3. Write a migration to remove the column (wrap in `safety_assured` block)
|
180
|
+
If you forget `disable_ddl_transaction!`, the migration will fail. Also, note that indexes on new tables (those created in the same migration) don’t require this. Check out [gindex](https://github.com/ankane/gindex) to quickly generate index migrations without memorizing the syntax.
|
125
181
|
|
126
|
-
|
127
|
-
|
128
|
-
|
129
|
-
|
182
|
+
### Adding a reference
|
183
|
+
|
184
|
+
#### Bad
|
185
|
+
|
186
|
+
Rails adds a non-concurrent index to references by default, which is problematic for Postgres.
|
187
|
+
|
188
|
+
```ruby
|
189
|
+
class AddReferenceToUsers < ActiveRecord::Migration[5.2]
|
190
|
+
def change
|
191
|
+
add_reference :users, :city
|
192
|
+
end
|
193
|
+
end
|
194
|
+
```
|
195
|
+
|
196
|
+
#### Good
|
197
|
+
|
198
|
+
Make sure the index is added concurrently.
|
199
|
+
|
200
|
+
```ruby
|
201
|
+
class AddReferenceToUsers < ActiveRecord::Migration[5.2]
|
202
|
+
disable_ddl_transaction!
|
203
|
+
|
204
|
+
def change
|
205
|
+
add_reference :users, :city, index: false
|
206
|
+
add_index :users, :city_id, algorithm: :concurrently
|
207
|
+
end
|
208
|
+
end
|
209
|
+
```
|
210
|
+
|
211
|
+
For polymorphic references, add a compound index on type and id.
|
212
|
+
|
213
|
+
### Adding a foreign key
|
214
|
+
|
215
|
+
#### Bad
|
216
|
+
|
217
|
+
In Postgres, new foreign keys are validated by default, which acquires an `AccessExclusiveLock` that can be [expensive on large tables](https://travisofthenorth.com/blog/2017/2/2/postgres-adding-foreign-keys-with-zero-downtime).
|
218
|
+
|
219
|
+
```ruby
|
220
|
+
class AddForeignKeyOnUsers < ActiveRecord::Migration[5.2]
|
221
|
+
def change
|
222
|
+
add_foreign_key :users, :orders
|
223
|
+
end
|
224
|
+
end
|
225
|
+
```
|
226
|
+
|
227
|
+
#### Good
|
228
|
+
|
229
|
+
Instead, validate it in a separate migration with a more agreeable `RowShareLock`. This approach is documented by Postgres to have “[the least impact on other work](https://www.postgresql.org/docs/current/sql-altertable.html).”
|
230
|
+
|
231
|
+
For Rails 5.2+, use:
|
232
|
+
|
233
|
+
```ruby
|
234
|
+
class AddForeignKeyOnUsers < ActiveRecord::Migration[5.2]
|
235
|
+
def change
|
236
|
+
add_foreign_key :users, :orders, validate: false
|
237
|
+
end
|
238
|
+
end
|
239
|
+
```
|
240
|
+
|
241
|
+
Then validate it in a separate migration.
|
242
|
+
|
243
|
+
```ruby
|
244
|
+
class ValidateForeignKeyOnUsers < ActiveRecord::Migration[5.2]
|
245
|
+
def change
|
246
|
+
validate_foreign_key :users, :orders
|
247
|
+
end
|
248
|
+
end
|
249
|
+
```
|
250
|
+
|
251
|
+
For Rails < 5.2, use:
|
252
|
+
|
253
|
+
```ruby
|
254
|
+
class AddForeignKeyOnUsers < ActiveRecord::Migration[5.1]
|
255
|
+
def change
|
256
|
+
safety_assured do
|
257
|
+
execute 'ALTER TABLE "users" ADD CONSTRAINT "fk_rails_c1e9b98e31" FOREIGN KEY ("order_id") REFERENCES "orders" ("id") NOT VALID'
|
130
258
|
end
|
131
259
|
end
|
132
|
-
|
260
|
+
end
|
261
|
+
```
|
133
262
|
|
134
|
-
|
263
|
+
Then validate it in a separate migration.
|
264
|
+
|
265
|
+
```ruby
|
266
|
+
class ValidateForeignKeyOnUsers < ActiveRecord::Migration[5.1]
|
267
|
+
def change
|
268
|
+
safety_assured do
|
269
|
+
execute 'ALTER TABLE "users" VALIDATE CONSTRAINT "fk_rails_c1e9b98e31"'
|
270
|
+
end
|
271
|
+
end
|
272
|
+
end
|
273
|
+
```
|
135
274
|
|
136
275
|
### Renaming or changing the type of a column
|
137
276
|
|
277
|
+
#### Bad
|
278
|
+
|
279
|
+
```ruby
|
280
|
+
class RenameSomeColumn < ActiveRecord::Migration[5.2]
|
281
|
+
def change
|
282
|
+
rename_column :users, :some_column, :new_name
|
283
|
+
end
|
284
|
+
end
|
285
|
+
```
|
286
|
+
|
287
|
+
or
|
288
|
+
|
289
|
+
```ruby
|
290
|
+
class ChangeSomeColumnType < ActiveRecord::Migration[5.2]
|
291
|
+
def change
|
292
|
+
change_column :users, :some_column, :new_type
|
293
|
+
end
|
294
|
+
end
|
295
|
+
```
|
296
|
+
|
297
|
+
One exception is changing a `varchar` column to `text`, which is safe in Postgres.
|
298
|
+
|
299
|
+
#### Good
|
300
|
+
|
138
301
|
A safer approach is to:
|
139
302
|
|
140
303
|
1. Create a new column
|
@@ -144,10 +307,20 @@ A safer approach is to:
|
|
144
307
|
5. Stop writing to the old column
|
145
308
|
6. Drop the old column
|
146
309
|
|
147
|
-
One exception is changing a `varchar` column to `text`, which is safe in Postgres 9.1+.
|
148
|
-
|
149
310
|
### Renaming a table
|
150
311
|
|
312
|
+
#### Bad
|
313
|
+
|
314
|
+
```ruby
|
315
|
+
class RenameUsersToCustomers < ActiveRecord::Migration[5.2]
|
316
|
+
def change
|
317
|
+
rename_table :users, :customers
|
318
|
+
end
|
319
|
+
end
|
320
|
+
```
|
321
|
+
|
322
|
+
#### Good
|
323
|
+
|
151
324
|
A safer approach is to:
|
152
325
|
|
153
326
|
1. Create a new table
|
@@ -157,64 +330,119 @@ A safer approach is to:
|
|
157
330
|
5. Stop writing to the old table
|
158
331
|
6. Drop the old table
|
159
332
|
|
160
|
-
###
|
333
|
+
### Creating a table with the `force` option
|
161
334
|
|
162
|
-
|
335
|
+
#### Bad
|
336
|
+
|
337
|
+
The `force` option can drop an existing table.
|
163
338
|
|
164
339
|
```ruby
|
165
|
-
class
|
166
|
-
|
340
|
+
class CreateUsers < ActiveRecord::Migration[5.2]
|
341
|
+
def change
|
342
|
+
create_table :users, force: true do |t|
|
343
|
+
# ...
|
344
|
+
end
|
345
|
+
end
|
346
|
+
end
|
347
|
+
```
|
348
|
+
|
349
|
+
#### Good
|
350
|
+
|
351
|
+
If you intend to drop a table, do it explicitly. Then create the new table without the `force` option:
|
167
352
|
|
353
|
+
```ruby
|
354
|
+
class CreateUsers < ActiveRecord::Migration[5.2]
|
168
355
|
def change
|
169
|
-
|
356
|
+
create_table :users do |t|
|
357
|
+
# ...
|
358
|
+
end
|
170
359
|
end
|
171
360
|
end
|
172
361
|
```
|
173
362
|
|
174
|
-
|
363
|
+
### Using `change_column_null` with a default value
|
364
|
+
|
365
|
+
#### Bad
|
175
366
|
|
176
|
-
|
367
|
+
This generates a single `UPDATE` statement to set the default value.
|
177
368
|
|
178
369
|
```ruby
|
179
|
-
class
|
180
|
-
|
370
|
+
class ChangeSomeColumnNull < ActiveRecord::Migration[5.2]
|
371
|
+
def change
|
372
|
+
change_column_null :users, :some_column, false, "default_value"
|
373
|
+
end
|
374
|
+
end
|
375
|
+
```
|
181
376
|
|
377
|
+
#### Good
|
378
|
+
|
379
|
+
Backfill the column [safely](#backfilling-data). Then use:
|
380
|
+
|
381
|
+
```ruby
|
382
|
+
class ChangeSomeColumnNull < ActiveRecord::Migration[5.2]
|
182
383
|
def change
|
183
|
-
|
184
|
-
add_index :users, :reference_id, algorithm: :concurrently
|
384
|
+
change_column_null :users, :some_column, false
|
185
385
|
end
|
186
386
|
end
|
187
387
|
```
|
188
388
|
|
189
|
-
|
389
|
+
### Adding a json column
|
190
390
|
|
191
|
-
|
391
|
+
#### Bad
|
192
392
|
|
193
|
-
|
393
|
+
In Postgres, there’s no equality operator for the `json` column type, which causes issues for `SELECT DISTINCT` queries.
|
394
|
+
|
395
|
+
```ruby
|
396
|
+
class AddPropertiesToUsers < ActiveRecord::Migration[5.2]
|
397
|
+
def change
|
398
|
+
add_column :users, :properties, :json
|
399
|
+
end
|
400
|
+
end
|
401
|
+
```
|
194
402
|
|
195
|
-
|
403
|
+
#### Good
|
196
404
|
|
197
|
-
|
405
|
+
Use `jsonb` instead.
|
198
406
|
|
199
407
|
```ruby
|
200
|
-
class
|
201
|
-
|
408
|
+
class AddPropertiesToUsers < ActiveRecord::Migration[5.2]
|
409
|
+
def change
|
410
|
+
add_column :users, :properties, :jsonb
|
411
|
+
end
|
202
412
|
end
|
203
413
|
```
|
204
414
|
|
205
|
-
|
415
|
+
## Best Practices
|
416
|
+
|
417
|
+
### Keeping non-unique indexes to three columns or less
|
418
|
+
|
419
|
+
#### Bad
|
420
|
+
|
421
|
+
Adding an index with more than three columns only helps on extremely large tables.
|
206
422
|
|
207
423
|
```ruby
|
208
|
-
class
|
424
|
+
class AddSomeIndexToUsers < ActiveRecord::Migration[5.2]
|
209
425
|
def change
|
210
|
-
|
426
|
+
add_index :users, [:a, :b, :c, :d]
|
211
427
|
end
|
212
428
|
end
|
213
429
|
```
|
214
430
|
|
431
|
+
#### Good
|
432
|
+
|
433
|
+
```ruby
|
434
|
+
class AddSomeIndexToUsers < ActiveRecord::Migration[5.2]
|
435
|
+
def change
|
436
|
+
add_index :users, [:a, :b, :c]
|
437
|
+
end
|
438
|
+
end
|
439
|
+
```
|
440
|
+
|
441
|
+
> For Postgres, be sure to add them concurrently
|
442
|
+
|
215
443
|
## Assuring Safety
|
216
444
|
|
217
|
-
To mark a step in the migration as safe, despite using method that might otherwise be dangerous, wrap it in a `safety_assured` block.
|
445
|
+
To mark a step in the migration as safe, despite using a method that might otherwise be dangerous, wrap it in a `safety_assured` block.
|
218
446
|
|
219
447
|
```ruby
|
220
448
|
class MySafeMigration < ActiveRecord::Migration[5.2]
|
@@ -224,6 +452,8 @@ class MySafeMigration < ActiveRecord::Migration[5.2]
|
|
224
452
|
end
|
225
453
|
```
|
226
454
|
|
455
|
+
Certain methods like `execute` and `change_table` cannot be inspected and are prevented from running by default. Make sure what you’re doing is really safe and use this pattern.
|
456
|
+
|
227
457
|
## Custom Checks
|
228
458
|
|
229
459
|
Add your own custom checks with:
|
@@ -238,6 +468,8 @@ end
|
|
238
468
|
|
239
469
|
Use the `stop!` method to stop migrations.
|
240
470
|
|
471
|
+
> Since `remove_column` always requires a `safety_assured` block, it’s not possible to add a custom check for `remove_column` operations
|
472
|
+
|
241
473
|
## Existing Migrations
|
242
474
|
|
243
475
|
To mark migrations as safe that were created before installing this gem, create an initializer with:
|
@@ -312,7 +544,7 @@ Rails 5.1+ uses `bigint` for primary keys to keep you from running out of ids. T
|
|
312
544
|
|
313
545
|
## Credits
|
314
546
|
|
315
|
-
Thanks to Bob Remeika and David Waller for the [original code](https://github.com/foobarfighter/safe-migrations).
|
547
|
+
Thanks to Bob Remeika and David Waller for the [original code](https://github.com/foobarfighter/safe-migrations) and [Sean Huber](https://github.com/LendingHome/zero_downtime_migrations) for the bad/good readme format.
|
316
548
|
|
317
549
|
## Contributing
|
318
550
|
|
@@ -15,8 +15,6 @@ module StrongMigrations
|
|
15
15
|
|
16
16
|
def method_missing(method, *args, &block)
|
17
17
|
unless @safe || ENV["SAFETY_ASSURED"] || is_a?(ActiveRecord::Schema) || @direction == :down || version_safe?
|
18
|
-
ar5 = ActiveRecord::VERSION::MAJOR >= 5
|
19
|
-
|
20
18
|
case method
|
21
19
|
when :remove_column, :remove_columns, :remove_timestamps, :remove_reference, :remove_belongs_to
|
22
20
|
columns =
|
@@ -36,7 +34,7 @@ module StrongMigrations
|
|
36
34
|
cols
|
37
35
|
end
|
38
36
|
|
39
|
-
code =
|
37
|
+
code = "self.ignored_columns = #{columns.inspect}"
|
40
38
|
|
41
39
|
raise_error :remove_column,
|
42
40
|
model: args[0].to_s.classify,
|
@@ -65,21 +63,30 @@ module StrongMigrations
|
|
65
63
|
default = options[:default]
|
66
64
|
|
67
65
|
if !default.nil? && !(postgresql? && postgresql_version >= 110000)
|
66
|
+
|
67
|
+
if options[:null] == false
|
68
|
+
options = options.except(:null)
|
69
|
+
append = "
|
70
|
+
|
71
|
+
Then add the NOT NULL constraint.
|
72
|
+
|
73
|
+
class %{migration_name}NotNull < ActiveRecord::Migration%{migration_suffix}
|
74
|
+
def change
|
75
|
+
#{command_str("change_column_null", [table, column, false])}
|
76
|
+
end
|
77
|
+
end"
|
78
|
+
end
|
79
|
+
|
68
80
|
raise_error :add_column_default,
|
69
81
|
add_command: command_str("add_column", [table, column, type, options.except(:default)]),
|
70
82
|
change_command: command_str("change_column_default", [table, column, default]),
|
71
83
|
remove_command: command_str("remove_column", [table, column]),
|
72
|
-
code: backfill_code(table, column, default)
|
84
|
+
code: backfill_code(table, column, default),
|
85
|
+
append: append
|
73
86
|
end
|
74
87
|
|
75
88
|
if type.to_s == "json" && postgresql?
|
76
|
-
|
77
|
-
raise_error :add_column_json
|
78
|
-
else
|
79
|
-
raise_error :add_column_json_legacy,
|
80
|
-
model: table.to_s.classify,
|
81
|
-
table: connection.quote_table_name(table.to_s)
|
82
|
-
end
|
89
|
+
raise_error :add_column_json
|
83
90
|
end
|
84
91
|
when :change_column
|
85
92
|
table, column, type = args
|
@@ -103,7 +110,7 @@ module StrongMigrations
|
|
103
110
|
table, reference, options = args
|
104
111
|
options ||= {}
|
105
112
|
|
106
|
-
index_value = options.fetch(:index,
|
113
|
+
index_value = options.fetch(:index, true)
|
107
114
|
if postgresql? && index_value
|
108
115
|
columns = options[:polymorphic] ? [:"#{reference}_type", :"#{reference}_id"] : :"#{reference}_id"
|
109
116
|
|
@@ -119,6 +126,32 @@ module StrongMigrations
|
|
119
126
|
raise_error :change_column_null,
|
120
127
|
code: backfill_code(table, column, default)
|
121
128
|
end
|
129
|
+
when :add_foreign_key
|
130
|
+
from_table, to_table, options = args
|
131
|
+
options ||= {}
|
132
|
+
validate = options.fetch(:validate, true)
|
133
|
+
|
134
|
+
if postgresql?
|
135
|
+
if ActiveRecord::VERSION::STRING >= "5.2"
|
136
|
+
if validate
|
137
|
+
raise_error :add_foreign_key,
|
138
|
+
add_foreign_key_code: command_str("add_foreign_key", [from_table, to_table, options.merge(validate: false)]),
|
139
|
+
validate_foreign_key_code: command_str("validate_foreign_key", [from_table, to_table])
|
140
|
+
end
|
141
|
+
else
|
142
|
+
# always validated before 5.2
|
143
|
+
|
144
|
+
# fk name logic from rails
|
145
|
+
primary_key = options[:primary_key] || "id"
|
146
|
+
column = options[:column] || "#{to_table.to_s.singularize}_id"
|
147
|
+
hashed_identifier = Digest::SHA256.hexdigest("#{from_table}_#{column}_fk").first(10)
|
148
|
+
fk_name = options[:name] || "fk_rails_#{hashed_identifier}"
|
149
|
+
|
150
|
+
raise_error :add_foreign_key,
|
151
|
+
add_foreign_key_code: foreign_key_str("ALTER TABLE %s ADD CONSTRAINT %s FOREIGN KEY (%s) REFERENCES %s (%s) NOT VALID", [from_table, fk_name, column, to_table, primary_key]),
|
152
|
+
validate_foreign_key_code: foreign_key_str("ALTER TABLE %s VALIDATE CONSTRAINT %s", [from_table, fk_name])
|
153
|
+
end
|
154
|
+
end
|
122
155
|
end
|
123
156
|
|
124
157
|
StrongMigrations.checks.each do |check|
|
@@ -152,15 +185,25 @@ module StrongMigrations
|
|
152
185
|
def raise_error(message_key, header: nil, **vars)
|
153
186
|
message = StrongMigrations.error_messages[message_key] || "Missing message"
|
154
187
|
|
155
|
-
ar5 = ActiveRecord::VERSION::MAJOR >= 5
|
156
188
|
vars[:migration_name] = self.class.name
|
157
|
-
vars[:migration_suffix] =
|
158
|
-
vars[:base_model] =
|
189
|
+
vars[:migration_suffix] = "[#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}]"
|
190
|
+
vars[:base_model] = "ApplicationRecord"
|
191
|
+
|
192
|
+
# interpolate variables in appended code
|
193
|
+
if vars[:append]
|
194
|
+
vars[:append] = vars[:append].gsub(/%(?!{)/, "%%") % vars
|
195
|
+
end
|
159
196
|
|
160
197
|
# escape % not followed by {
|
161
198
|
stop!(message.gsub(/%(?!{)/, "%%") % vars, header: header || "Dangerous operation detected")
|
162
199
|
end
|
163
200
|
|
201
|
+
def foreign_key_str(statement, identifiers)
|
202
|
+
# not all identifiers are tables, but this method of quoting should be fine
|
203
|
+
code = statement % identifiers.map { |v| connection.quote_table_name(v) }
|
204
|
+
"safety_assured do\n execute '#{code}' \n end"
|
205
|
+
end
|
206
|
+
|
164
207
|
def command_str(command, args)
|
165
208
|
str_args = args[0..-2].map { |a| a.inspect }
|
166
209
|
|
@@ -179,11 +222,7 @@ module StrongMigrations
|
|
179
222
|
|
180
223
|
def backfill_code(table, column, default)
|
181
224
|
model = table.to_s.classify
|
182
|
-
|
183
|
-
"#{model}.in_batches.update_all #{column}: #{default.inspect}"
|
184
|
-
else
|
185
|
-
"#{model}.find_in_batches do |records|\n #{model}.where(id: records.map(&:id)).update_all #{column}: #{default.inspect}\n end"
|
186
|
-
end
|
225
|
+
"#{model}.in_batches do |relation| \n relation.update_all #{column}: #{default.inspect}\n sleep(0.1)\n end"
|
187
226
|
end
|
188
227
|
|
189
228
|
def stop!(message, header: "Custom check")
|
data/lib/strong_migrations.rb
CHANGED
@@ -37,23 +37,12 @@ class Backfill%{migration_name} < ActiveRecord::Migration%{migration_suffix}
|
|
37
37
|
def change
|
38
38
|
%{code}
|
39
39
|
end
|
40
|
-
end",
|
40
|
+
end%{append}",
|
41
41
|
|
42
42
|
add_column_json:
|
43
43
|
"There's no equality operator for the json column type, which
|
44
44
|
causes issues for SELECT DISTINCT queries. Use jsonb instead.",
|
45
45
|
|
46
|
-
add_column_json_legacy:
|
47
|
-
"There's no equality operator for the json column type, which.
|
48
|
-
causes issues for SELECT DISTINCT queries.
|
49
|
-
Replace all calls to uniq with a custom scope.
|
50
|
-
|
51
|
-
class %{model} < %{base_model}
|
52
|
-
scope :uniq_on_id, -> { select('DISTINCT ON (%{table}.id) %{table}.*') }
|
53
|
-
end
|
54
|
-
|
55
|
-
Once it's deployed, wrap this step in a safety_assured { ... } block.",
|
56
|
-
|
57
46
|
change_column:
|
58
47
|
"Changing the type of an existing column requires the entire
|
59
48
|
table and indexes to be rewritten. A safer approach is to:
|
@@ -93,7 +82,7 @@ end",
|
|
93
82
|
rename_table:
|
94
83
|
"Renaming a table is dangerous. A safer approach is to:
|
95
84
|
|
96
|
-
1. Create a new table
|
85
|
+
1. Create a new table. Don't forget to recreate indexes from the old table
|
97
86
|
2. Write to both tables
|
98
87
|
3. Backfill data from the old table to new table
|
99
88
|
4. Move reads from the old table to the new table
|
@@ -143,7 +132,7 @@ Otherwise, remove the force option.",
|
|
143
132
|
execute call, so cannot help you here. Please make really sure that what
|
144
133
|
you're doing is safe before proceeding, then wrap it in a safety_assured { ... } block.",
|
145
134
|
|
146
|
-
|
135
|
+
change_column_null:
|
147
136
|
"Passing a default value to change_column_null runs a single UPDATE query,
|
148
137
|
which can cause downtime. Instead, backfill the existing rows in the
|
149
138
|
Rails console or a separate migration with disable_ddl_transaction!.
|
@@ -154,7 +143,24 @@ class Backfill%{migration_name} < ActiveRecord::Migration%{migration_suffix}
|
|
154
143
|
def change
|
155
144
|
%{code}
|
156
145
|
end
|
157
|
-
end"
|
146
|
+
end",
|
147
|
+
|
148
|
+
add_foreign_key:
|
149
|
+
"New foreign keys are validated by default. This acquires an AccessExclusiveLock,
|
150
|
+
which is expensive on large tables. Instead, validate it in a separate migration
|
151
|
+
with a more agreeable RowShareLock.
|
152
|
+
|
153
|
+
class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
|
154
|
+
def change
|
155
|
+
%{add_foreign_key_code}
|
156
|
+
end
|
157
|
+
end
|
158
|
+
|
159
|
+
class Validate%{migration_name} < ActiveRecord::Migration%{migration_suffix}
|
160
|
+
def change
|
161
|
+
%{validate_foreign_key_code}
|
162
|
+
end
|
163
|
+
end",
|
158
164
|
}
|
159
165
|
|
160
166
|
def self.add_check(&block)
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: strong_migrations
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.4.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Andrew Kane
|
@@ -10,7 +10,7 @@ authors:
|
|
10
10
|
autorequire:
|
11
11
|
bindir: bin
|
12
12
|
cert_chain: []
|
13
|
-
date:
|
13
|
+
date: 2019-05-27 00:00:00.000000000 Z
|
14
14
|
dependencies:
|
15
15
|
- !ruby/object:Gem::Dependency
|
16
16
|
name: activerecord
|
@@ -18,14 +18,14 @@ dependencies:
|
|
18
18
|
requirements:
|
19
19
|
- - ">="
|
20
20
|
- !ruby/object:Gem::Version
|
21
|
-
version:
|
21
|
+
version: '5'
|
22
22
|
type: :runtime
|
23
23
|
prerelease: false
|
24
24
|
version_requirements: !ruby/object:Gem::Requirement
|
25
25
|
requirements:
|
26
26
|
- - ">="
|
27
27
|
- !ruby/object:Gem::Version
|
28
|
-
version:
|
28
|
+
version: '5'
|
29
29
|
- !ruby/object:Gem::Dependency
|
30
30
|
name: bundler
|
31
31
|
requirement: !ruby/object:Gem::Requirement
|
@@ -72,16 +72,16 @@ dependencies:
|
|
72
72
|
name: pg
|
73
73
|
requirement: !ruby/object:Gem::Requirement
|
74
74
|
requirements:
|
75
|
-
- - "
|
75
|
+
- - ">="
|
76
76
|
- !ruby/object:Gem::Version
|
77
|
-
version: '
|
77
|
+
version: '0'
|
78
78
|
type: :development
|
79
79
|
prerelease: false
|
80
80
|
version_requirements: !ruby/object:Gem::Requirement
|
81
81
|
requirements:
|
82
|
-
- - "
|
82
|
+
- - ">="
|
83
83
|
- !ruby/object:Gem::Version
|
84
|
-
version: '
|
84
|
+
version: '0'
|
85
85
|
description:
|
86
86
|
email:
|
87
87
|
- andrew@chartkick.com
|
@@ -114,15 +114,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
114
114
|
requirements:
|
115
115
|
- - ">="
|
116
116
|
- !ruby/object:Gem::Version
|
117
|
-
version: '2.
|
117
|
+
version: '2.4'
|
118
118
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
119
119
|
requirements:
|
120
120
|
- - ">="
|
121
121
|
- !ruby/object:Gem::Version
|
122
122
|
version: '0'
|
123
123
|
requirements: []
|
124
|
-
|
125
|
-
rubygems_version: 2.7.7
|
124
|
+
rubygems_version: 3.0.3
|
126
125
|
signing_key:
|
127
126
|
specification_version: 4
|
128
127
|
summary: Catch unsafe migrations at dev time
|