strong_migrations 0.7.0 → 2.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +183 -1
- data/LICENSE.txt +1 -1
- data/README.md +385 -164
- data/lib/generators/strong_migrations/install_generator.rb +25 -0
- data/lib/generators/strong_migrations/templates/initializer.rb.tt +10 -3
- data/lib/strong_migrations/adapters/abstract_adapter.rb +76 -0
- data/lib/strong_migrations/adapters/mariadb_adapter.rb +32 -0
- data/lib/strong_migrations/adapters/mysql_adapter.rb +112 -0
- data/lib/strong_migrations/adapters/postgresql_adapter.rb +232 -0
- data/lib/strong_migrations/checker.rb +190 -440
- data/lib/strong_migrations/checks.rb +475 -0
- data/lib/strong_migrations/error_messages.rb +260 -0
- data/lib/strong_migrations/migration.rb +20 -3
- data/lib/strong_migrations/{database_tasks.rb → migration_context.rb} +20 -2
- data/lib/strong_migrations/migrator.rb +21 -0
- data/lib/strong_migrations/safe_methods.rb +123 -0
- data/lib/strong_migrations/schema_dumper.rb +32 -0
- data/lib/strong_migrations/version.rb +1 -1
- data/lib/strong_migrations.rb +46 -203
- data/lib/tasks/strong_migrations.rake +2 -7
- metadata +17 -83
- data/lib/strong_migrations/alphabetize_columns.rb +0 -11
data/README.md
CHANGED
|
@@ -4,18 +4,18 @@ Catch unsafe migrations in development
|
|
|
4
4
|
|
|
5
5
|
✓ Detects potentially dangerous operations<br /> ✓ Prevents them from running by default<br /> ✓ Provides instructions on safer ways to do what you want
|
|
6
6
|
|
|
7
|
-
Supports
|
|
7
|
+
Supports PostgreSQL, MySQL, and MariaDB
|
|
8
8
|
|
|
9
9
|
:tangerine: Battle-tested at [Instacart](https://www.instacart.com/opensource)
|
|
10
10
|
|
|
11
|
-
[](https://github.com/ankane/strong_migrations/actions)
|
|
12
12
|
|
|
13
13
|
## Installation
|
|
14
14
|
|
|
15
15
|
Add this line to your application’s Gemfile:
|
|
16
16
|
|
|
17
17
|
```ruby
|
|
18
|
-
gem
|
|
18
|
+
gem "strong_migrations"
|
|
19
19
|
```
|
|
20
20
|
|
|
21
21
|
And run:
|
|
@@ -38,14 +38,14 @@ Active Record caches attributes, which causes problems
|
|
|
38
38
|
when removing columns. Be sure to ignore the column:
|
|
39
39
|
|
|
40
40
|
class User < ApplicationRecord
|
|
41
|
-
self.ignored_columns
|
|
41
|
+
self.ignored_columns += ["name"]
|
|
42
42
|
end
|
|
43
43
|
|
|
44
44
|
Deploy the code, then wrap this step in a safety_assured { ... } block.
|
|
45
45
|
|
|
46
|
-
class RemoveColumn < ActiveRecord::Migration[
|
|
46
|
+
class RemoveColumn < ActiveRecord::Migration[8.0]
|
|
47
47
|
def change
|
|
48
|
-
safety_assured { remove_column :users, :name
|
|
48
|
+
safety_assured { remove_column :users, :name }
|
|
49
49
|
end
|
|
50
50
|
end
|
|
51
51
|
```
|
|
@@ -60,21 +60,30 @@ An operation is classified as dangerous if it either:
|
|
|
60
60
|
Potentially dangerous operations:
|
|
61
61
|
|
|
62
62
|
- [removing a column](#removing-a-column)
|
|
63
|
-
- [adding a column with a default value](#adding-a-column-with-a-default-value)
|
|
64
|
-
- [backfilling data](#backfilling-data)
|
|
65
63
|
- [changing the type of a column](#changing-the-type-of-a-column)
|
|
66
64
|
- [renaming a column](#renaming-a-column)
|
|
67
65
|
- [renaming a table](#renaming-a-table)
|
|
68
66
|
- [creating a table with the force option](#creating-a-table-with-the-force-option)
|
|
69
|
-
- [
|
|
67
|
+
- [adding an auto-incrementing column](#adding-an-auto-incrementing-column)
|
|
68
|
+
- [adding a stored generated column](#adding-a-stored-generated-column)
|
|
69
|
+
- [adding a check constraint](#adding-a-check-constraint)
|
|
70
70
|
- [executing SQL directly](#executing-SQL-directly)
|
|
71
|
+
- [backfilling data](#backfilling-data)
|
|
71
72
|
|
|
72
73
|
Postgres-specific checks:
|
|
73
74
|
|
|
74
75
|
- [adding an index non-concurrently](#adding-an-index-non-concurrently)
|
|
75
76
|
- [adding a reference](#adding-a-reference)
|
|
76
77
|
- [adding a foreign key](#adding-a-foreign-key)
|
|
78
|
+
- [adding a unique constraint](#adding-a-unique-constraint)
|
|
79
|
+
- [adding an exclusion constraint](#adding-an-exclusion-constraint)
|
|
77
80
|
- [adding a json column](#adding-a-json-column)
|
|
81
|
+
- [setting NOT NULL on an existing column](#setting-not-null-on-an-existing-column)
|
|
82
|
+
- [adding a column with a volatile default value](#adding-a-column-with-a-volatile-default-value)
|
|
83
|
+
|
|
84
|
+
Config-specific checks:
|
|
85
|
+
|
|
86
|
+
- [changing the default value of a column](#changing-the-default-value-of-a-column)
|
|
78
87
|
|
|
79
88
|
Best practices:
|
|
80
89
|
|
|
@@ -89,7 +98,7 @@ You can also add [custom checks](#custom-checks) or [disable specific checks](#d
|
|
|
89
98
|
Active Record caches database columns at runtime, so if you drop a column, it can cause exceptions until your app reboots.
|
|
90
99
|
|
|
91
100
|
```ruby
|
|
92
|
-
class RemoveSomeColumnFromUsers < ActiveRecord::Migration[
|
|
101
|
+
class RemoveSomeColumnFromUsers < ActiveRecord::Migration[8.0]
|
|
93
102
|
def change
|
|
94
103
|
remove_column :users, :some_column
|
|
95
104
|
end
|
|
@@ -102,91 +111,23 @@ end
|
|
|
102
111
|
|
|
103
112
|
```ruby
|
|
104
113
|
class User < ApplicationRecord
|
|
105
|
-
self.ignored_columns
|
|
114
|
+
self.ignored_columns += ["some_column"]
|
|
106
115
|
end
|
|
107
116
|
```
|
|
108
117
|
|
|
109
|
-
2. Deploy code
|
|
118
|
+
2. Deploy the code
|
|
110
119
|
3. Write a migration to remove the column (wrap in `safety_assured` block)
|
|
111
120
|
|
|
112
121
|
```ruby
|
|
113
|
-
class RemoveSomeColumnFromUsers < ActiveRecord::Migration[
|
|
122
|
+
class RemoveSomeColumnFromUsers < ActiveRecord::Migration[8.0]
|
|
114
123
|
def change
|
|
115
124
|
safety_assured { remove_column :users, :some_column }
|
|
116
125
|
end
|
|
117
126
|
end
|
|
118
127
|
```
|
|
119
128
|
|
|
120
|
-
4. Deploy and run migration
|
|
121
|
-
|
|
122
|
-
### Adding a column with a default value
|
|
123
|
-
|
|
124
|
-
#### Bad
|
|
125
|
-
|
|
126
|
-
In earlier versions of Postgres, MySQL, and MariaDB, adding a column with a default value to an existing table causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
|
|
127
|
-
|
|
128
|
-
```ruby
|
|
129
|
-
class AddSomeColumnToUsers < ActiveRecord::Migration[6.0]
|
|
130
|
-
def change
|
|
131
|
-
add_column :users, :some_column, :text, default: "default_value"
|
|
132
|
-
end
|
|
133
|
-
end
|
|
134
|
-
```
|
|
135
|
-
|
|
136
|
-
In Postgres 11+, MySQL 8.0.12+, and MariaDB 10.3.2+, this no longer requires a table rewrite and is safe.
|
|
137
|
-
|
|
138
|
-
#### Good
|
|
139
|
-
|
|
140
|
-
Instead, add the column without a default value, then change the default.
|
|
141
|
-
|
|
142
|
-
```ruby
|
|
143
|
-
class AddSomeColumnToUsers < ActiveRecord::Migration[6.0]
|
|
144
|
-
def up
|
|
145
|
-
add_column :users, :some_column, :text
|
|
146
|
-
change_column_default :users, :some_column, "default_value"
|
|
147
|
-
end
|
|
148
|
-
|
|
149
|
-
def down
|
|
150
|
-
remove_column :users, :some_column
|
|
151
|
-
end
|
|
152
|
-
end
|
|
153
|
-
```
|
|
154
|
-
|
|
155
|
-
See the next section for how to backfill.
|
|
156
|
-
|
|
157
|
-
### Backfilling data
|
|
158
|
-
|
|
159
|
-
#### Bad
|
|
160
|
-
|
|
161
|
-
Active Record creates a transaction around each migration, and backfilling in the same transaction that alters a table keeps the table locked for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
|
|
162
|
-
|
|
163
|
-
```ruby
|
|
164
|
-
class AddSomeColumnToUsers < ActiveRecord::Migration[6.0]
|
|
165
|
-
def change
|
|
166
|
-
add_column :users, :some_column, :text
|
|
167
|
-
User.update_all some_column: "default_value"
|
|
168
|
-
end
|
|
169
|
-
end
|
|
170
|
-
```
|
|
171
|
-
|
|
172
|
-
Also, running a single query to update data can cause issues for large tables.
|
|
173
|
-
|
|
174
|
-
#### Good
|
|
175
|
-
|
|
176
|
-
There are three keys to backfilling safely: batching, throttling, and running it outside a transaction. Use the Rails console or a separate migration with `disable_ddl_transaction!`.
|
|
177
|
-
|
|
178
|
-
```ruby
|
|
179
|
-
class BackfillSomeColumn < ActiveRecord::Migration[6.0]
|
|
180
|
-
disable_ddl_transaction!
|
|
181
|
-
|
|
182
|
-
def up
|
|
183
|
-
User.unscoped.in_batches do |relation|
|
|
184
|
-
relation.update_all some_column: "default_value"
|
|
185
|
-
sleep(0.01) # throttle
|
|
186
|
-
end
|
|
187
|
-
end
|
|
188
|
-
end
|
|
189
|
-
```
|
|
129
|
+
4. Deploy and run the migration
|
|
130
|
+
5. Remove the line added in step 1
|
|
190
131
|
|
|
191
132
|
### Changing the type of a column
|
|
192
133
|
|
|
@@ -195,26 +136,33 @@ end
|
|
|
195
136
|
Changing the type of a column causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
|
|
196
137
|
|
|
197
138
|
```ruby
|
|
198
|
-
class ChangeSomeColumnType < ActiveRecord::Migration[
|
|
139
|
+
class ChangeSomeColumnType < ActiveRecord::Migration[8.0]
|
|
199
140
|
def change
|
|
200
141
|
change_column :users, :some_column, :new_type
|
|
201
142
|
end
|
|
202
143
|
end
|
|
203
144
|
```
|
|
204
145
|
|
|
205
|
-
|
|
146
|
+
Some changes don’t require a table rewrite and are safe in Postgres:
|
|
206
147
|
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
148
|
+
Type | Safe Changes
|
|
149
|
+
--- | ---
|
|
150
|
+
`cidr` | Changing to `inet`
|
|
151
|
+
`citext` | Changing to `text` if not indexed, changing to `string` with no `:limit` if not indexed
|
|
152
|
+
`datetime` | Increasing or removing `:precision`, changing to `timestamptz` when session time zone is UTC in Postgres 12+
|
|
153
|
+
`decimal` | Increasing `:precision` at same `:scale`, removing `:precision` and `:scale`
|
|
154
|
+
`interval` | Increasing or removing `:precision`
|
|
155
|
+
`numeric` | Increasing `:precision` at same `:scale`, removing `:precision` and `:scale`
|
|
156
|
+
`string` | Increasing or removing `:limit`, changing to `text`, changing `citext` if not indexed
|
|
157
|
+
`text` | Changing to `string` with no `:limit`, changing to `citext` if not indexed
|
|
158
|
+
`time` | Increasing or removing `:precision`
|
|
159
|
+
`timestamptz` | Increasing or removing `:limit`, changing to `datetime` when session time zone is UTC in Postgres 12+
|
|
213
160
|
|
|
214
|
-
And
|
|
161
|
+
And some in MySQL and MariaDB:
|
|
215
162
|
|
|
216
|
-
|
|
217
|
-
|
|
163
|
+
Type | Safe Changes
|
|
164
|
+
--- | ---
|
|
165
|
+
`string` | Increasing `:limit` from under 63 up to 63, increasing `:limit` from over 63 to the max (the threshold can be different if using an encoding other than `utf8mb4` - for instance, it’s 85 for `utf8mb3` and 255 for `latin1`)
|
|
218
166
|
|
|
219
167
|
#### Good
|
|
220
168
|
|
|
@@ -234,7 +182,7 @@ A safer approach is to:
|
|
|
234
182
|
Renaming a column that’s in use will cause errors in your application.
|
|
235
183
|
|
|
236
184
|
```ruby
|
|
237
|
-
class RenameSomeColumn < ActiveRecord::Migration[
|
|
185
|
+
class RenameSomeColumn < ActiveRecord::Migration[8.0]
|
|
238
186
|
def change
|
|
239
187
|
rename_column :users, :some_column, :new_name
|
|
240
188
|
end
|
|
@@ -259,7 +207,7 @@ A safer approach is to:
|
|
|
259
207
|
Renaming a table that’s in use will cause errors in your application.
|
|
260
208
|
|
|
261
209
|
```ruby
|
|
262
|
-
class RenameUsersToCustomers < ActiveRecord::Migration[
|
|
210
|
+
class RenameUsersToCustomers < ActiveRecord::Migration[8.0]
|
|
263
211
|
def change
|
|
264
212
|
rename_table :users, :customers
|
|
265
213
|
end
|
|
@@ -272,7 +220,7 @@ A safer approach is to:
|
|
|
272
220
|
|
|
273
221
|
1. Create a new table
|
|
274
222
|
2. Write to both tables
|
|
275
|
-
3. Backfill data from the old table to new table
|
|
223
|
+
3. Backfill data from the old table to the new table
|
|
276
224
|
4. Move reads from the old table to the new table
|
|
277
225
|
5. Stop writing to the old table
|
|
278
226
|
6. Drop the old table
|
|
@@ -284,7 +232,7 @@ A safer approach is to:
|
|
|
284
232
|
The `force` option can drop an existing table.
|
|
285
233
|
|
|
286
234
|
```ruby
|
|
287
|
-
class CreateUsers < ActiveRecord::Migration[
|
|
235
|
+
class CreateUsers < ActiveRecord::Migration[8.0]
|
|
288
236
|
def change
|
|
289
237
|
create_table :users, force: true do |t|
|
|
290
238
|
# ...
|
|
@@ -298,7 +246,7 @@ end
|
|
|
298
246
|
Create tables without the `force` option.
|
|
299
247
|
|
|
300
248
|
```ruby
|
|
301
|
-
class CreateUsers < ActiveRecord::Migration[
|
|
249
|
+
class CreateUsers < ActiveRecord::Migration[8.0]
|
|
302
250
|
def change
|
|
303
251
|
create_table :users do |t|
|
|
304
252
|
# ...
|
|
@@ -309,76 +257,146 @@ end
|
|
|
309
257
|
|
|
310
258
|
If you intend to drop an existing table, run `drop_table` first.
|
|
311
259
|
|
|
312
|
-
###
|
|
260
|
+
### Adding an auto-incrementing column
|
|
313
261
|
|
|
314
262
|
#### Bad
|
|
315
263
|
|
|
316
|
-
|
|
264
|
+
Adding an auto-incrementing column (`serial`/`bigserial` in Postgres and `AUTO_INCREMENT` in MySQL and MariaDB) causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
|
|
317
265
|
|
|
318
266
|
```ruby
|
|
319
|
-
class
|
|
267
|
+
class AddIdToCitiesUsers < ActiveRecord::Migration[8.0]
|
|
320
268
|
def change
|
|
321
|
-
|
|
269
|
+
add_column :cities_users, :id, :primary_key
|
|
322
270
|
end
|
|
323
271
|
end
|
|
324
272
|
```
|
|
325
273
|
|
|
326
|
-
|
|
274
|
+
With MySQL and MariaDB, this can also [generate different values on replicas](https://dev.mysql.com/doc/mysql-replication-excerpt/8.0/en/replication-features-auto-increment.html) if using statement-based replication.
|
|
275
|
+
|
|
276
|
+
#### Good
|
|
277
|
+
|
|
278
|
+
Create a new table and migrate the data with the same steps as [renaming a table](#renaming-a-table).
|
|
327
279
|
|
|
328
|
-
|
|
280
|
+
### Adding a stored generated column
|
|
281
|
+
|
|
282
|
+
#### Bad
|
|
283
|
+
|
|
284
|
+
Adding a stored generated column causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
|
|
329
285
|
|
|
330
286
|
```ruby
|
|
331
|
-
class
|
|
287
|
+
class AddSomeColumnToUsers < ActiveRecord::Migration[8.0]
|
|
332
288
|
def change
|
|
333
|
-
|
|
334
|
-
execute 'ALTER TABLE "users" ADD CONSTRAINT "users_some_column_null" CHECK ("some_column" IS NOT NULL) NOT VALID'
|
|
335
|
-
end
|
|
289
|
+
add_column :users, :some_column, :virtual, type: :string, as: "...", stored: true
|
|
336
290
|
end
|
|
337
291
|
end
|
|
338
292
|
```
|
|
339
293
|
|
|
340
|
-
|
|
294
|
+
#### Good
|
|
295
|
+
|
|
296
|
+
Add a non-generated column and use callbacks or triggers instead (or a virtual generated column with MySQL and MariaDB).
|
|
297
|
+
|
|
298
|
+
### Adding a check constraint
|
|
299
|
+
|
|
300
|
+
:turtle: Safe by default available
|
|
301
|
+
|
|
302
|
+
#### Bad
|
|
303
|
+
|
|
304
|
+
Adding a check constraint blocks reads and writes in Postgres and blocks writes in MySQL and MariaDB while every row is checked.
|
|
341
305
|
|
|
342
306
|
```ruby
|
|
343
|
-
class
|
|
307
|
+
class AddCheckConstraint < ActiveRecord::Migration[8.0]
|
|
344
308
|
def change
|
|
345
|
-
|
|
346
|
-
|
|
347
|
-
|
|
309
|
+
add_check_constraint :users, "price > 0", name: "price_check"
|
|
310
|
+
end
|
|
311
|
+
end
|
|
312
|
+
```
|
|
348
313
|
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
|
|
352
|
-
|
|
353
|
-
|
|
314
|
+
#### Good - Postgres
|
|
315
|
+
|
|
316
|
+
Add the check constraint without validating existing rows:
|
|
317
|
+
|
|
318
|
+
```ruby
|
|
319
|
+
class AddCheckConstraint < ActiveRecord::Migration[8.0]
|
|
320
|
+
def change
|
|
321
|
+
add_check_constraint :users, "price > 0", name: "price_check", validate: false
|
|
322
|
+
end
|
|
323
|
+
end
|
|
324
|
+
```
|
|
325
|
+
|
|
326
|
+
Then validate them in a separate migration.
|
|
327
|
+
|
|
328
|
+
```ruby
|
|
329
|
+
class ValidateCheckConstraint < ActiveRecord::Migration[8.0]
|
|
330
|
+
def change
|
|
331
|
+
validate_check_constraint :users, name: "price_check"
|
|
354
332
|
end
|
|
355
333
|
end
|
|
356
334
|
```
|
|
357
335
|
|
|
358
336
|
#### Good - MySQL and MariaDB
|
|
359
337
|
|
|
360
|
-
[Let us know](https://github.com/ankane/strong_migrations/issues/new) if you have a safe way to do this.
|
|
338
|
+
[Let us know](https://github.com/ankane/strong_migrations/issues/new) if you have a safe way to do this (check constraints can be added with `NOT ENFORCED`, but enforcing blocks writes).
|
|
361
339
|
|
|
362
340
|
### Executing SQL directly
|
|
363
341
|
|
|
364
342
|
Strong Migrations can’t ensure safety for raw SQL statements. Make really sure that what you’re doing is safe, then use:
|
|
365
343
|
|
|
366
344
|
```ruby
|
|
367
|
-
class ExecuteSQL < ActiveRecord::Migration[
|
|
345
|
+
class ExecuteSQL < ActiveRecord::Migration[8.0]
|
|
368
346
|
def change
|
|
369
347
|
safety_assured { execute "..." }
|
|
370
348
|
end
|
|
371
349
|
end
|
|
372
350
|
```
|
|
373
351
|
|
|
352
|
+
### Backfilling data
|
|
353
|
+
|
|
354
|
+
Note: Strong Migrations does not detect dangerous backfills.
|
|
355
|
+
|
|
356
|
+
#### Bad
|
|
357
|
+
|
|
358
|
+
Active Record creates a transaction around each migration, and backfilling in the same transaction that alters a table keeps the table locked for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
|
|
359
|
+
|
|
360
|
+
```ruby
|
|
361
|
+
class AddSomeColumnToUsers < ActiveRecord::Migration[8.0]
|
|
362
|
+
def change
|
|
363
|
+
add_column :users, :some_column, :text
|
|
364
|
+
User.update_all some_column: "default_value"
|
|
365
|
+
end
|
|
366
|
+
end
|
|
367
|
+
```
|
|
368
|
+
|
|
369
|
+
Also, running a single query to update data can cause issues for large tables.
|
|
370
|
+
|
|
371
|
+
#### Good
|
|
372
|
+
|
|
373
|
+
There are three keys to backfilling safely: batching, throttling, and running it outside a transaction. Use the Rails console or a separate migration with `disable_ddl_transaction!`.
|
|
374
|
+
|
|
375
|
+
```ruby
|
|
376
|
+
class BackfillSomeColumn < ActiveRecord::Migration[8.0]
|
|
377
|
+
disable_ddl_transaction!
|
|
378
|
+
|
|
379
|
+
def up
|
|
380
|
+
User.unscoped.in_batches do |relation|
|
|
381
|
+
relation.update_all some_column: "default_value"
|
|
382
|
+
sleep(0.01) # throttle
|
|
383
|
+
end
|
|
384
|
+
end
|
|
385
|
+
end
|
|
386
|
+
```
|
|
387
|
+
|
|
388
|
+
Note: If backfilling with a method other than `update_all`, use `User.reset_column_information` to ensure the model has up-to-date column information.
|
|
389
|
+
|
|
374
390
|
### Adding an index non-concurrently
|
|
375
391
|
|
|
392
|
+
:turtle: Safe by default available
|
|
393
|
+
|
|
376
394
|
#### Bad
|
|
377
395
|
|
|
378
396
|
In Postgres, adding an index non-concurrently blocks writes.
|
|
379
397
|
|
|
380
398
|
```ruby
|
|
381
|
-
class AddSomeIndexToUsers < ActiveRecord::Migration[
|
|
399
|
+
class AddSomeIndexToUsers < ActiveRecord::Migration[8.0]
|
|
382
400
|
def change
|
|
383
401
|
add_index :users, :some_column
|
|
384
402
|
end
|
|
@@ -390,7 +408,7 @@ end
|
|
|
390
408
|
Add indexes concurrently.
|
|
391
409
|
|
|
392
410
|
```ruby
|
|
393
|
-
class AddSomeIndexToUsers < ActiveRecord::Migration[
|
|
411
|
+
class AddSomeIndexToUsers < ActiveRecord::Migration[8.0]
|
|
394
412
|
disable_ddl_transaction!
|
|
395
413
|
|
|
396
414
|
def change
|
|
@@ -409,12 +427,14 @@ rails g index table column
|
|
|
409
427
|
|
|
410
428
|
### Adding a reference
|
|
411
429
|
|
|
430
|
+
:turtle: Safe by default available
|
|
431
|
+
|
|
412
432
|
#### Bad
|
|
413
433
|
|
|
414
434
|
Rails adds an index non-concurrently to references by default, which blocks writes in Postgres.
|
|
415
435
|
|
|
416
436
|
```ruby
|
|
417
|
-
class AddReferenceToUsers < ActiveRecord::Migration[
|
|
437
|
+
class AddReferenceToUsers < ActiveRecord::Migration[8.0]
|
|
418
438
|
def change
|
|
419
439
|
add_reference :users, :city
|
|
420
440
|
end
|
|
@@ -426,7 +446,7 @@ end
|
|
|
426
446
|
Make sure the index is added concurrently.
|
|
427
447
|
|
|
428
448
|
```ruby
|
|
429
|
-
class AddReferenceToUsers < ActiveRecord::Migration[
|
|
449
|
+
class AddReferenceToUsers < ActiveRecord::Migration[8.0]
|
|
430
450
|
disable_ddl_transaction!
|
|
431
451
|
|
|
432
452
|
def change
|
|
@@ -437,12 +457,14 @@ end
|
|
|
437
457
|
|
|
438
458
|
### Adding a foreign key
|
|
439
459
|
|
|
460
|
+
:turtle: Safe by default available
|
|
461
|
+
|
|
440
462
|
#### Bad
|
|
441
463
|
|
|
442
464
|
In Postgres, adding a foreign key blocks writes on both tables.
|
|
443
465
|
|
|
444
466
|
```ruby
|
|
445
|
-
class AddForeignKeyOnUsers < ActiveRecord::Migration[
|
|
467
|
+
class AddForeignKeyOnUsers < ActiveRecord::Migration[8.0]
|
|
446
468
|
def change
|
|
447
469
|
add_foreign_key :users, :orders
|
|
448
470
|
end
|
|
@@ -452,7 +474,7 @@ end
|
|
|
452
474
|
or
|
|
453
475
|
|
|
454
476
|
```ruby
|
|
455
|
-
class AddReferenceToUsers < ActiveRecord::Migration[
|
|
477
|
+
class AddReferenceToUsers < ActiveRecord::Migration[8.0]
|
|
456
478
|
def change
|
|
457
479
|
add_reference :users, :order, foreign_key: true
|
|
458
480
|
end
|
|
@@ -461,52 +483,77 @@ end
|
|
|
461
483
|
|
|
462
484
|
#### Good
|
|
463
485
|
|
|
464
|
-
Add the foreign key without validating existing rows
|
|
465
|
-
|
|
466
|
-
For Rails 5.2+, use:
|
|
486
|
+
Add the foreign key without validating existing rows:
|
|
467
487
|
|
|
468
488
|
```ruby
|
|
469
|
-
class AddForeignKeyOnUsers < ActiveRecord::Migration[
|
|
489
|
+
class AddForeignKeyOnUsers < ActiveRecord::Migration[8.0]
|
|
470
490
|
def change
|
|
471
491
|
add_foreign_key :users, :orders, validate: false
|
|
472
492
|
end
|
|
473
493
|
end
|
|
474
494
|
```
|
|
475
495
|
|
|
476
|
-
Then
|
|
496
|
+
Then validate them in a separate migration.
|
|
477
497
|
|
|
478
498
|
```ruby
|
|
479
|
-
class ValidateForeignKeyOnUsers < ActiveRecord::Migration[
|
|
499
|
+
class ValidateForeignKeyOnUsers < ActiveRecord::Migration[8.0]
|
|
480
500
|
def change
|
|
481
501
|
validate_foreign_key :users, :orders
|
|
482
502
|
end
|
|
483
503
|
end
|
|
484
504
|
```
|
|
485
505
|
|
|
486
|
-
|
|
506
|
+
### Adding a unique constraint
|
|
507
|
+
|
|
508
|
+
#### Bad
|
|
509
|
+
|
|
510
|
+
In Postgres, adding a unique constraint creates a unique index, which blocks reads and writes.
|
|
487
511
|
|
|
488
512
|
```ruby
|
|
489
|
-
class
|
|
513
|
+
class AddUniqueConstraint < ActiveRecord::Migration[8.0]
|
|
490
514
|
def change
|
|
491
|
-
|
|
492
|
-
|
|
493
|
-
|
|
515
|
+
add_unique_constraint :users, :some_column
|
|
516
|
+
end
|
|
517
|
+
end
|
|
518
|
+
```
|
|
519
|
+
|
|
520
|
+
#### Good
|
|
521
|
+
|
|
522
|
+
Create a unique index concurrently, then use it for the constraint.
|
|
523
|
+
|
|
524
|
+
```ruby
|
|
525
|
+
class AddUniqueConstraint < ActiveRecord::Migration[8.0]
|
|
526
|
+
disable_ddl_transaction!
|
|
527
|
+
|
|
528
|
+
def up
|
|
529
|
+
add_index :users, :some_column, unique: true, algorithm: :concurrently
|
|
530
|
+
add_unique_constraint :users, using_index: "index_users_on_some_column"
|
|
531
|
+
end
|
|
532
|
+
|
|
533
|
+
def down
|
|
534
|
+
remove_unique_constraint :users, :some_column
|
|
494
535
|
end
|
|
495
536
|
end
|
|
496
537
|
```
|
|
497
538
|
|
|
498
|
-
|
|
539
|
+
### Adding an exclusion constraint
|
|
540
|
+
|
|
541
|
+
#### Bad
|
|
542
|
+
|
|
543
|
+
In Postgres, adding an exclusion constraint blocks reads and writes while every row is checked.
|
|
499
544
|
|
|
500
545
|
```ruby
|
|
501
|
-
class
|
|
546
|
+
class AddExclusionConstraint < ActiveRecord::Migration[8.0]
|
|
502
547
|
def change
|
|
503
|
-
|
|
504
|
-
execute 'ALTER TABLE "users" VALIDATE CONSTRAINT "fk_rails_c1e9b98e31"'
|
|
505
|
-
end
|
|
548
|
+
add_exclusion_constraint :users, "number WITH =", using: :gist
|
|
506
549
|
end
|
|
507
550
|
end
|
|
508
551
|
```
|
|
509
552
|
|
|
553
|
+
#### Good
|
|
554
|
+
|
|
555
|
+
[Let us know](https://github.com/ankane/strong_migrations/issues/new) if you have a safe way to do this (exclusion constraints cannot be marked `NOT VALID`).
|
|
556
|
+
|
|
510
557
|
### Adding a json column
|
|
511
558
|
|
|
512
559
|
#### Bad
|
|
@@ -514,7 +561,7 @@ end
|
|
|
514
561
|
In Postgres, there’s no equality operator for the `json` column type, which can cause errors for existing `SELECT DISTINCT` queries in your application.
|
|
515
562
|
|
|
516
563
|
```ruby
|
|
517
|
-
class AddPropertiesToUsers < ActiveRecord::Migration[
|
|
564
|
+
class AddPropertiesToUsers < ActiveRecord::Migration[8.0]
|
|
518
565
|
def change
|
|
519
566
|
add_column :users, :properties, :json
|
|
520
567
|
end
|
|
@@ -526,13 +573,121 @@ end
|
|
|
526
573
|
Use `jsonb` instead.
|
|
527
574
|
|
|
528
575
|
```ruby
|
|
529
|
-
class AddPropertiesToUsers < ActiveRecord::Migration[
|
|
576
|
+
class AddPropertiesToUsers < ActiveRecord::Migration[8.0]
|
|
530
577
|
def change
|
|
531
578
|
add_column :users, :properties, :jsonb
|
|
532
579
|
end
|
|
533
580
|
end
|
|
534
581
|
```
|
|
535
582
|
|
|
583
|
+
### Setting NOT NULL on an existing column
|
|
584
|
+
|
|
585
|
+
:turtle: Safe by default available
|
|
586
|
+
|
|
587
|
+
#### Bad
|
|
588
|
+
|
|
589
|
+
In Postgres, setting `NOT NULL` on an existing column blocks reads and writes while every row is checked.
|
|
590
|
+
|
|
591
|
+
```ruby
|
|
592
|
+
class SetSomeColumnNotNull < ActiveRecord::Migration[8.0]
|
|
593
|
+
def change
|
|
594
|
+
change_column_null :users, :some_column, false
|
|
595
|
+
end
|
|
596
|
+
end
|
|
597
|
+
```
|
|
598
|
+
|
|
599
|
+
#### Good
|
|
600
|
+
|
|
601
|
+
Instead, add a check constraint.
|
|
602
|
+
|
|
603
|
+
```ruby
|
|
604
|
+
class SetSomeColumnNotNull < ActiveRecord::Migration[8.0]
|
|
605
|
+
def change
|
|
606
|
+
add_check_constraint :users, "some_column IS NOT NULL", name: "users_some_column_null", validate: false
|
|
607
|
+
end
|
|
608
|
+
end
|
|
609
|
+
```
|
|
610
|
+
|
|
611
|
+
Then validate it in a separate migration. Once the check constraint is validated, you can safely set `NOT NULL` on the column and drop the check constraint.
|
|
612
|
+
|
|
613
|
+
```ruby
|
|
614
|
+
class ValidateSomeColumnNotNull < ActiveRecord::Migration[8.0]
|
|
615
|
+
def up
|
|
616
|
+
validate_check_constraint :users, name: "users_some_column_null"
|
|
617
|
+
change_column_null :users, :some_column, false
|
|
618
|
+
remove_check_constraint :users, name: "users_some_column_null"
|
|
619
|
+
end
|
|
620
|
+
|
|
621
|
+
def down
|
|
622
|
+
add_check_constraint :users, "some_column IS NOT NULL", name: "users_some_column_null", validate: false
|
|
623
|
+
change_column_null :users, :some_column, true
|
|
624
|
+
end
|
|
625
|
+
end
|
|
626
|
+
```
|
|
627
|
+
|
|
628
|
+
### Adding a column with a volatile default value
|
|
629
|
+
|
|
630
|
+
#### Bad
|
|
631
|
+
|
|
632
|
+
Adding a column with a volatile default value to an existing table causes the entire table to be rewritten. During this time, reads and writes are blocked.
|
|
633
|
+
|
|
634
|
+
```ruby
|
|
635
|
+
class AddSomeColumnToUsers < ActiveRecord::Migration[8.0]
|
|
636
|
+
def change
|
|
637
|
+
add_column :users, :some_column, :uuid, default: "gen_random_uuid()"
|
|
638
|
+
end
|
|
639
|
+
end
|
|
640
|
+
```
|
|
641
|
+
|
|
642
|
+
#### Good
|
|
643
|
+
|
|
644
|
+
Instead, add the column without a default value, then change the default.
|
|
645
|
+
|
|
646
|
+
```ruby
|
|
647
|
+
class AddSomeColumnToUsers < ActiveRecord::Migration[8.0]
|
|
648
|
+
def up
|
|
649
|
+
add_column :users, :some_column, :uuid
|
|
650
|
+
change_column_default :users, :some_column, from: nil, to: "gen_random_uuid()"
|
|
651
|
+
end
|
|
652
|
+
|
|
653
|
+
def down
|
|
654
|
+
remove_column :users, :some_column
|
|
655
|
+
end
|
|
656
|
+
end
|
|
657
|
+
```
|
|
658
|
+
|
|
659
|
+
Then [backfill the data](#backfilling-data).
|
|
660
|
+
|
|
661
|
+
### Changing the default value of a column
|
|
662
|
+
|
|
663
|
+
#### Bad
|
|
664
|
+
|
|
665
|
+
Rails < 7 enables partial writes by default, which can cause incorrect values to be inserted when changing the default value of a column.
|
|
666
|
+
|
|
667
|
+
```ruby
|
|
668
|
+
class ChangeSomeColumnDefault < ActiveRecord::Migration[6.1]
|
|
669
|
+
def change
|
|
670
|
+
change_column_default :users, :some_column, from: "old", to: "new"
|
|
671
|
+
end
|
|
672
|
+
end
|
|
673
|
+
|
|
674
|
+
User.create!(some_column: "old") # can insert "new"
|
|
675
|
+
```
|
|
676
|
+
|
|
677
|
+
#### Good
|
|
678
|
+
|
|
679
|
+
Disable partial writes in `config/application.rb`. For Rails < 7, use:
|
|
680
|
+
|
|
681
|
+
```ruby
|
|
682
|
+
config.active_record.partial_writes = false
|
|
683
|
+
```
|
|
684
|
+
|
|
685
|
+
For Rails 7+, use:
|
|
686
|
+
|
|
687
|
+
```ruby
|
|
688
|
+
config.active_record.partial_inserts = false
|
|
689
|
+
```
|
|
690
|
+
|
|
536
691
|
### Keeping non-unique indexes to three columns or less
|
|
537
692
|
|
|
538
693
|
#### Bad
|
|
@@ -540,7 +695,7 @@ end
|
|
|
540
695
|
Adding a non-unique index with more than three columns rarely improves performance.
|
|
541
696
|
|
|
542
697
|
```ruby
|
|
543
|
-
class AddSomeIndexToUsers < ActiveRecord::Migration[
|
|
698
|
+
class AddSomeIndexToUsers < ActiveRecord::Migration[8.0]
|
|
544
699
|
def change
|
|
545
700
|
add_index :users, [:a, :b, :c, :d]
|
|
546
701
|
end
|
|
@@ -552,9 +707,9 @@ end
|
|
|
552
707
|
Instead, start an index with columns that narrow down the results the most.
|
|
553
708
|
|
|
554
709
|
```ruby
|
|
555
|
-
class AddSomeIndexToUsers < ActiveRecord::Migration[
|
|
710
|
+
class AddSomeIndexToUsers < ActiveRecord::Migration[8.0]
|
|
556
711
|
def change
|
|
557
|
-
add_index :users, [:
|
|
712
|
+
add_index :users, [:d, :b]
|
|
558
713
|
end
|
|
559
714
|
end
|
|
560
715
|
```
|
|
@@ -566,7 +721,7 @@ For Postgres, be sure to add them concurrently.
|
|
|
566
721
|
To mark a step in the migration as safe, despite using a method that might otherwise be dangerous, wrap it in a `safety_assured` block.
|
|
567
722
|
|
|
568
723
|
```ruby
|
|
569
|
-
class MySafeMigration < ActiveRecord::Migration[
|
|
724
|
+
class MySafeMigration < ActiveRecord::Migration[8.0]
|
|
570
725
|
def change
|
|
571
726
|
safety_assured { remove_column :users, :some_column }
|
|
572
727
|
end
|
|
@@ -575,6 +730,21 @@ end
|
|
|
575
730
|
|
|
576
731
|
Certain methods like `execute` and `change_table` cannot be inspected and are prevented from running by default. Make sure what you’re doing is really safe and use this pattern.
|
|
577
732
|
|
|
733
|
+
## Safe by Default
|
|
734
|
+
|
|
735
|
+
Make certain operations safe by default. This allows you to write the code under the "Bad" section, but the migration will be performed as if you had written the "Good" version.
|
|
736
|
+
|
|
737
|
+
- adding and removing an index
|
|
738
|
+
- adding a foreign key
|
|
739
|
+
- adding a check constraint
|
|
740
|
+
- setting NOT NULL on an existing column
|
|
741
|
+
|
|
742
|
+
Add to `config/initializers/strong_migrations.rb`:
|
|
743
|
+
|
|
744
|
+
```ruby
|
|
745
|
+
StrongMigrations.safe_by_default = true
|
|
746
|
+
```
|
|
747
|
+
|
|
578
748
|
## Custom Checks
|
|
579
749
|
|
|
580
750
|
Add your own custom checks with:
|
|
@@ -609,9 +779,19 @@ Disable specific checks with:
|
|
|
609
779
|
StrongMigrations.disable_check(:add_index)
|
|
610
780
|
```
|
|
611
781
|
|
|
612
|
-
Check the [source code](https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations.rb) for the list of keys.
|
|
782
|
+
Check the [source code](https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations/error_messages.rb) for the list of keys.
|
|
783
|
+
|
|
784
|
+
## Skip Databases
|
|
613
785
|
|
|
614
|
-
|
|
786
|
+
Skip checks and other functionality for specific databases with:
|
|
787
|
+
|
|
788
|
+
```ruby
|
|
789
|
+
StrongMigrations.skip_database(:catalog)
|
|
790
|
+
```
|
|
791
|
+
|
|
792
|
+
Note: This does not affect `alphabetize_schema`.
|
|
793
|
+
|
|
794
|
+
## Down Migrations / Rollbacks
|
|
615
795
|
|
|
616
796
|
By default, checks are disabled when migrating down. Enable them with:
|
|
617
797
|
|
|
@@ -627,7 +807,7 @@ To customize specific messages, create an initializer with:
|
|
|
627
807
|
StrongMigrations.error_messages[:add_column_default] = "Your custom instructions"
|
|
628
808
|
```
|
|
629
809
|
|
|
630
|
-
Check the [source code](https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations.rb) for the list of keys.
|
|
810
|
+
Check the [source code](https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations/error_messages.rb) for the list of keys.
|
|
631
811
|
|
|
632
812
|
## Migration Timeouts
|
|
633
813
|
|
|
@@ -692,12 +872,45 @@ production:
|
|
|
692
872
|
|
|
693
873
|
For HTTP connections, Redis, and other services, check out [this guide](https://github.com/ankane/the-ultimate-guide-to-ruby-timeouts).
|
|
694
874
|
|
|
875
|
+
## Invalid Indexes
|
|
876
|
+
|
|
877
|
+
In Postgres, adding an index non-concurrently can leave behind an invalid index if the lock timeout is reached. Running the migration again can result in an error.
|
|
878
|
+
|
|
879
|
+
To automatically remove the invalid index when the migration runs again, use:
|
|
880
|
+
|
|
881
|
+
```ruby
|
|
882
|
+
StrongMigrations.remove_invalid_indexes = true
|
|
883
|
+
```
|
|
884
|
+
|
|
885
|
+
Note: This feature is experimental.
|
|
886
|
+
|
|
887
|
+
## Lock Timeout Retries
|
|
888
|
+
|
|
889
|
+
Note: This feature is experimental.
|
|
890
|
+
|
|
891
|
+
There’s the option to automatically retry statements for migrations when the lock timeout is reached. Here’s how it works:
|
|
892
|
+
|
|
893
|
+
- If a lock timeout happens outside a transaction, the statement is retried
|
|
894
|
+
- If it happens inside the DDL transaction, the entire migration is retried (only applicable to Postgres)
|
|
895
|
+
|
|
896
|
+
Add to `config/initializers/strong_migrations.rb`:
|
|
897
|
+
|
|
898
|
+
```ruby
|
|
899
|
+
StrongMigrations.lock_timeout_retries = 3
|
|
900
|
+
```
|
|
901
|
+
|
|
902
|
+
Set the delay between retries with:
|
|
903
|
+
|
|
904
|
+
```ruby
|
|
905
|
+
StrongMigrations.lock_timeout_retry_delay = 10.seconds
|
|
906
|
+
```
|
|
907
|
+
|
|
695
908
|
## Existing Migrations
|
|
696
909
|
|
|
697
910
|
To mark migrations as safe that were created before installing this gem, create an initializer with:
|
|
698
911
|
|
|
699
912
|
```ruby
|
|
700
|
-
StrongMigrations.start_after =
|
|
913
|
+
StrongMigrations.start_after = 20250101000000
|
|
701
914
|
```
|
|
702
915
|
|
|
703
916
|
Use the version from your latest migration.
|
|
@@ -707,13 +920,19 @@ Use the version from your latest migration.
|
|
|
707
920
|
If your development database version is different from production, you can specify the production version so the right checks run in development.
|
|
708
921
|
|
|
709
922
|
```ruby
|
|
710
|
-
StrongMigrations.
|
|
711
|
-
StrongMigrations.target_mysql_version = "8.0.12"
|
|
712
|
-
StrongMigrations.target_mariadb_version = "10.3.2"
|
|
923
|
+
StrongMigrations.target_version = 10 # or 8.0, 10.5, etc
|
|
713
924
|
```
|
|
714
925
|
|
|
926
|
+
The major version works well for Postgres, while the major and minor version is recommended for MySQL and MariaDB.
|
|
927
|
+
|
|
715
928
|
For safety, this option only affects development and test environments. In other environments, the actual server version is always used.
|
|
716
929
|
|
|
930
|
+
If your app has multiple databases with different versions, you can use:
|
|
931
|
+
|
|
932
|
+
```ruby
|
|
933
|
+
StrongMigrations.target_version = {primary: 13, catalog: 15}
|
|
934
|
+
```
|
|
935
|
+
|
|
717
936
|
## Analyze Tables
|
|
718
937
|
|
|
719
938
|
Analyze tables automatically (to update planner statistics) after an index is added. Create an initializer with:
|
|
@@ -724,19 +943,18 @@ StrongMigrations.auto_analyze = true
|
|
|
724
943
|
|
|
725
944
|
## Faster Migrations
|
|
726
945
|
|
|
727
|
-
Only dump the schema when adding a new migration. If you use Git,
|
|
946
|
+
Only dump the schema when adding a new migration. If you use Git, add to `config/environments/development.rb`:
|
|
728
947
|
|
|
729
|
-
```
|
|
730
|
-
|
|
731
|
-
`git status db/migrate/ --porcelain`.present?
|
|
948
|
+
```rb
|
|
949
|
+
config.active_record.dump_schema_after_migration = `git status db/migrate/ --porcelain`.present?
|
|
732
950
|
```
|
|
733
951
|
|
|
734
952
|
## Schema Sanity
|
|
735
953
|
|
|
736
|
-
Columns can flip order in `db/schema.rb` when you have multiple developers. One way to prevent this is to [alphabetize them](https://www.pgrs.net/2008/03/12/alphabetize-schema-rb-columns/). Add to
|
|
954
|
+
Columns can flip order in `db/schema.rb` when you have multiple developers. One way to prevent this is to [alphabetize them](https://www.pgrs.net/2008/03/12/alphabetize-schema-rb-columns/). Add to `config/initializers/strong_migrations.rb`:
|
|
737
955
|
|
|
738
956
|
```ruby
|
|
739
|
-
|
|
957
|
+
StrongMigrations.alphabetize_schema = true
|
|
740
958
|
```
|
|
741
959
|
|
|
742
960
|
## Permissions
|
|
@@ -749,15 +967,18 @@ You probably don’t need this gem for smaller projects, as operations that are
|
|
|
749
967
|
|
|
750
968
|
## Additional Reading
|
|
751
969
|
|
|
752
|
-
- [Rails Migrations with No Downtime](https://pedro.herokuapp.com/past/2011/7/13/rails_migrations_with_no_downtime/)
|
|
753
970
|
- [PostgreSQL at Scale: Database Schema Changes Without Downtime](https://medium.com/braintree-product-technology/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680)
|
|
754
|
-
- [
|
|
971
|
+
- [MySQL InnoDB Online DDL Operations](https://dev.mysql.com/doc/refman/en/innodb-online-ddl-operations.html)
|
|
755
972
|
- [MariaDB InnoDB Online DDL Overview](https://mariadb.com/kb/en/innodb-online-ddl-overview/)
|
|
756
973
|
|
|
757
974
|
## Credits
|
|
758
975
|
|
|
759
976
|
Thanks to Bob Remeika and David Waller for the [original code](https://github.com/foobarfighter/safe-migrations) and [Sean Huber](https://github.com/LendingHome/zero_downtime_migrations) for the bad/good readme format.
|
|
760
977
|
|
|
978
|
+
## History
|
|
979
|
+
|
|
980
|
+
View the [changelog](https://github.com/ankane/strong_migrations/blob/master/CHANGELOG.md)
|
|
981
|
+
|
|
761
982
|
## Contributing
|
|
762
983
|
|
|
763
984
|
Everyone is encouraged to help improve this project. Here are a few ways you can help:
|