online_migrations 0.20.2 → 0.21.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +12 -0
- data/README.md +64 -64
- data/docs/background_data_migrations.md +1 -1
- data/docs/background_schema_migrations.md +20 -14
- data/docs/configuring.md +1 -1
- data/lib/online_migrations/background_schema_migrations/migration_helpers.rb +20 -0
- data/lib/online_migrations/error_messages.rb +4 -0
- data/lib/online_migrations/lock_retrier.rb +1 -1
- data/lib/online_migrations/schema_statements.rb +47 -7
- data/lib/online_migrations/version.rb +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 3ecf0ad3b1c2bc1992e9ec205018212b4d3c663f2a3040271a0cce45d4b9f60c
|
4
|
+
data.tar.gz: fcc3a3ac5b96fc3bd893f251949969794ebde738e5f663eb367fd191fa50784d
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: c5a92bde9892d9f7bf545248d359527c9fac1a41b31e6eabf34f0da0a1342fdfc9e54ccc853aba6e35414d368328979d7b445246f71e72e0ac09cce80ad8a02f
|
7
|
+
data.tar.gz: d244cb531e3e2f58d140e75af1f8fc571a272d9d188d87f0e52d89b7efbc7233eb4c1df7bac63fe32d1e7b79643e1202872b951f1fcfc2b8ca5fa9c3abdc7746
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,17 @@
|
|
1
1
|
## master (unreleased)
|
2
2
|
|
3
|
+
## 0.21.0 (2024-12-09)
|
4
|
+
|
5
|
+
- Fix `add_foreign_key` when referencing same table via different columns
|
6
|
+
- Make `validate_not_null_constraint`, `remove_foreign_key` and `remove_check_constraint` idempotent
|
7
|
+
|
8
|
+
- Add helpers for validating constraints in background
|
9
|
+
|
10
|
+
```ruby
|
11
|
+
validate_foreign_key_in_background(:users, :companies)
|
12
|
+
validate_constraint_in_background(:users, "first_name_not_null")
|
13
|
+
```
|
14
|
+
|
3
15
|
## 0.20.2 (2024-11-11)
|
4
16
|
|
5
17
|
- Fix running background migrations over relations with duplicate records
|
data/README.md
CHANGED
@@ -67,7 +67,7 @@ An operation is classified as dangerous if it either:
|
|
67
67
|
Consider the following migration:
|
68
68
|
|
69
69
|
```ruby
|
70
|
-
class AddAdminToUsers < ActiveRecord::Migration[
|
70
|
+
class AddAdminToUsers < ActiveRecord::Migration[8.0]
|
71
71
|
def change
|
72
72
|
add_column :users, :admin, :boolean, default: false, null: false
|
73
73
|
end
|
@@ -79,7 +79,7 @@ If the `users` table is large, running this migration on a live PostgreSQL < 11
|
|
79
79
|
A safer approach would be to run something like the following:
|
80
80
|
|
81
81
|
```ruby
|
82
|
-
class AddAdminToUsers < ActiveRecord::Migration[
|
82
|
+
class AddAdminToUsers < ActiveRecord::Migration[8.0]
|
83
83
|
# Do not wrap the migration in a transaction so that locks are held for a shorter time.
|
84
84
|
disable_ddl_transaction!
|
85
85
|
|
@@ -124,7 +124,7 @@ A safer approach is to:
|
|
124
124
|
|
125
125
|
add_column_with_default takes care of all this steps:
|
126
126
|
|
127
|
-
class AddAdminToUsers < ActiveRecord::Migration[
|
127
|
+
class AddAdminToUsers < ActiveRecord::Migration[8.0]
|
128
128
|
disable_ddl_transaction!
|
129
129
|
|
130
130
|
def change
|
@@ -178,7 +178,7 @@ You can also add [custom checks](docs/configuring.md#custom-checks) or [disable
|
|
178
178
|
Active Record caches database columns at runtime, so if you drop a column, it can cause exceptions until your app reboots.
|
179
179
|
|
180
180
|
```ruby
|
181
|
-
class RemoveNameFromUsers < ActiveRecord::Migration[
|
181
|
+
class RemoveNameFromUsers < ActiveRecord::Migration[8.0]
|
182
182
|
def change
|
183
183
|
remove_column :users, :name
|
184
184
|
end
|
@@ -199,7 +199,7 @@ end
|
|
199
199
|
3. Wrap column removing in a `safety_assured` block:
|
200
200
|
|
201
201
|
```ruby
|
202
|
-
class RemoveNameFromUsers < ActiveRecord::Migration[
|
202
|
+
class RemoveNameFromUsers < ActiveRecord::Migration[8.0]
|
203
203
|
def change
|
204
204
|
safety_assured { remove_column :users, :name }
|
205
205
|
end
|
@@ -216,7 +216,7 @@ end
|
|
216
216
|
In earlier versions of PostgreSQL adding a column with a non-null default value to an existing table blocks reads and writes while the entire table is rewritten.
|
217
217
|
|
218
218
|
```ruby
|
219
|
-
class AddAdminToUsers < ActiveRecord::Migration[
|
219
|
+
class AddAdminToUsers < ActiveRecord::Migration[8.0]
|
220
220
|
def change
|
221
221
|
add_column :users, :admin, :boolean, default: false
|
222
222
|
end
|
@@ -236,7 +236,7 @@ A safer approach is to:
|
|
236
236
|
`add_column_with_default` helper takes care of all this steps:
|
237
237
|
|
238
238
|
```ruby
|
239
|
-
class AddAdminToUsers < ActiveRecord::Migration[
|
239
|
+
class AddAdminToUsers < ActiveRecord::Migration[8.0]
|
240
240
|
disable_ddl_transaction!
|
241
241
|
|
242
242
|
def change
|
@@ -254,7 +254,7 @@ end
|
|
254
254
|
Active Record wraps each migration in a transaction, and backfilling in the same transaction that alters a table keeps the table locked for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
|
255
255
|
|
256
256
|
```ruby
|
257
|
-
class AddAdminToUsers < ActiveRecord::Migration[
|
257
|
+
class AddAdminToUsers < ActiveRecord::Migration[8.0]
|
258
258
|
def change
|
259
259
|
add_column :users, :admin, :boolean
|
260
260
|
User.update_all(admin: false)
|
@@ -269,13 +269,13 @@ Also, running a single query to update data can cause issues for large tables.
|
|
269
269
|
There are three keys to backfilling safely: batching, throttling, and running it outside a transaction. Use a `update_column_in_batches` helper in a separate migration with `disable_ddl_transaction!`.
|
270
270
|
|
271
271
|
```ruby
|
272
|
-
class AddAdminToUsers < ActiveRecord::Migration[
|
272
|
+
class AddAdminToUsers < ActiveRecord::Migration[8.0]
|
273
273
|
def change
|
274
274
|
add_column :users, :admin, :boolean
|
275
275
|
end
|
276
276
|
end
|
277
277
|
|
278
|
-
class BackfillUsersAdminColumn < ActiveRecord::Migration[
|
278
|
+
class BackfillUsersAdminColumn < ActiveRecord::Migration[8.0]
|
279
279
|
disable_ddl_transaction!
|
280
280
|
|
281
281
|
def up
|
@@ -294,7 +294,7 @@ end
|
|
294
294
|
Changing the type of an existing column blocks reads and writes while the entire table is rewritten.
|
295
295
|
|
296
296
|
```ruby
|
297
|
-
class ChangeFilesSizeType < ActiveRecord::Migration[
|
297
|
+
class ChangeFilesSizeType < ActiveRecord::Migration[8.0]
|
298
298
|
def change
|
299
299
|
change_column :files, :size, :bigint
|
300
300
|
end
|
@@ -338,7 +338,7 @@ A safer approach can be accomplished in several steps:
|
|
338
338
|
1. Create a new column and keep column's data in sync:
|
339
339
|
|
340
340
|
```ruby
|
341
|
-
class InitializeChangeFilesSizeType < ActiveRecord::Migration[
|
341
|
+
class InitializeChangeFilesSizeType < ActiveRecord::Migration[8.0]
|
342
342
|
def change
|
343
343
|
initialize_column_type_change :files, :size, :bigint
|
344
344
|
end
|
@@ -351,7 +351,7 @@ A safer approach can be accomplished in several steps:
|
|
351
351
|
2. Backfill data from the old column to the new column:
|
352
352
|
|
353
353
|
```ruby
|
354
|
-
class BackfillChangeFilesSizeType < ActiveRecord::Migration[
|
354
|
+
class BackfillChangeFilesSizeType < ActiveRecord::Migration[8.0]
|
355
355
|
disable_ddl_transaction!
|
356
356
|
|
357
357
|
def up
|
@@ -372,7 +372,7 @@ during writes works automatically). For most column type changes, this does not
|
|
372
372
|
5. Copy indexes, foreign keys, check constraints, NOT NULL constraint, swap new column in place:
|
373
373
|
|
374
374
|
```ruby
|
375
|
-
class FinalizeChangeFilesSizeType < ActiveRecord::Migration[
|
375
|
+
class FinalizeChangeFilesSizeType < ActiveRecord::Migration[8.0]
|
376
376
|
disable_ddl_transaction!
|
377
377
|
|
378
378
|
def change
|
@@ -385,7 +385,7 @@ during writes works automatically). For most column type changes, this does not
|
|
385
385
|
7. Finally, if everything works as expected, remove copy trigger and old column:
|
386
386
|
|
387
387
|
```ruby
|
388
|
-
class CleanupChangeFilesSizeType < ActiveRecord::Migration[
|
388
|
+
class CleanupChangeFilesSizeType < ActiveRecord::Migration[8.0]
|
389
389
|
def up
|
390
390
|
cleanup_column_type_change :files, :size
|
391
391
|
end
|
@@ -406,7 +406,7 @@ during writes works automatically). For most column type changes, this does not
|
|
406
406
|
Renaming a column that's in use will cause errors in your application.
|
407
407
|
|
408
408
|
```ruby
|
409
|
-
class RenameUsersNameToFirstName < ActiveRecord::Migration[
|
409
|
+
class RenameUsersNameToFirstName < ActiveRecord::Migration[8.0]
|
410
410
|
def change
|
411
411
|
rename_column :users, :name, :first_name
|
412
412
|
end
|
@@ -477,7 +477,7 @@ nor any data/indexes/foreign keys copying will be made, so will be instantaneous
|
|
477
477
|
It will use a combination of a VIEW and column aliasing to work with both column names simultaneously
|
478
478
|
|
479
479
|
```ruby
|
480
|
-
class InitializeRenameUsersNameToFirstName < ActiveRecord::Migration[
|
480
|
+
class InitializeRenameUsersNameToFirstName < ActiveRecord::Migration[8.0]
|
481
481
|
def change
|
482
482
|
initialize_column_rename :users, :name, :first_name
|
483
483
|
end
|
@@ -500,7 +500,7 @@ It will use a combination of a VIEW and column aliasing to work with both column
|
|
500
500
|
9. Remove the VIEW created in step 3 and finally rename the column:
|
501
501
|
|
502
502
|
```ruby
|
503
|
-
class FinalizeRenameUsersNameToFirstName < ActiveRecord::Migration[
|
503
|
+
class FinalizeRenameUsersNameToFirstName < ActiveRecord::Migration[8.0]
|
504
504
|
def change
|
505
505
|
finalize_column_rename :users, :name, :first_name
|
506
506
|
end
|
@@ -516,7 +516,7 @@ It will use a combination of a VIEW and column aliasing to work with both column
|
|
516
516
|
Renaming a table that's in use will cause errors in your application.
|
517
517
|
|
518
518
|
```ruby
|
519
|
-
class RenameClientsToUsers < ActiveRecord::Migration[
|
519
|
+
class RenameClientsToUsers < ActiveRecord::Migration[8.0]
|
520
520
|
def change
|
521
521
|
rename_table :clients, :users
|
522
522
|
end
|
@@ -571,7 +571,7 @@ To work around this limitation, we need to tell Active Record to acquire this in
|
|
571
571
|
3. Create a VIEW:
|
572
572
|
|
573
573
|
```ruby
|
574
|
-
class InitializeRenameClientsToUsers < ActiveRecord::Migration[
|
574
|
+
class InitializeRenameClientsToUsers < ActiveRecord::Migration[8.0]
|
575
575
|
def change
|
576
576
|
initialize_table_rename :clients, :users
|
577
577
|
end
|
@@ -584,7 +584,7 @@ To work around this limitation, we need to tell Active Record to acquire this in
|
|
584
584
|
7. Remove the VIEW created in step 3:
|
585
585
|
|
586
586
|
```ruby
|
587
|
-
class FinalizeRenameClientsToUsers < ActiveRecord::Migration[
|
587
|
+
class FinalizeRenameClientsToUsers < ActiveRecord::Migration[8.0]
|
588
588
|
def change
|
589
589
|
finalize_table_rename :clients, :users
|
590
590
|
end
|
@@ -600,7 +600,7 @@ To work around this limitation, we need to tell Active Record to acquire this in
|
|
600
600
|
The `force` option can drop an existing table.
|
601
601
|
|
602
602
|
```ruby
|
603
|
-
class CreateUsers < ActiveRecord::Migration[
|
603
|
+
class CreateUsers < ActiveRecord::Migration[8.0]
|
604
604
|
def change
|
605
605
|
create_table :users, force: true do |t|
|
606
606
|
# ...
|
@@ -614,7 +614,7 @@ end
|
|
614
614
|
Create tables without the `force` option.
|
615
615
|
|
616
616
|
```ruby
|
617
|
-
class CreateUsers < ActiveRecord::Migration[
|
617
|
+
class CreateUsers < ActiveRecord::Migration[8.0]
|
618
618
|
def change
|
619
619
|
create_table :users do |t|
|
620
620
|
# ...
|
@@ -632,7 +632,7 @@ If you intend to drop an existing table, run `drop_table` first.
|
|
632
632
|
Adding a check constraint blocks reads and writes while every row is checked.
|
633
633
|
|
634
634
|
```ruby
|
635
|
-
class AddCheckConstraint < ActiveRecord::Migration[
|
635
|
+
class AddCheckConstraint < ActiveRecord::Migration[8.0]
|
636
636
|
def change
|
637
637
|
add_check_constraint :users, "char_length(name) >= 1", name: "name_check"
|
638
638
|
end
|
@@ -644,7 +644,7 @@ end
|
|
644
644
|
Add the check constraint without validating existing rows, and then validate them in a separate transaction:
|
645
645
|
|
646
646
|
```ruby
|
647
|
-
class AddCheckConstraint < ActiveRecord::Migration[
|
647
|
+
class AddCheckConstraint < ActiveRecord::Migration[8.0]
|
648
648
|
disable_ddl_transaction!
|
649
649
|
|
650
650
|
def change
|
@@ -663,7 +663,7 @@ end
|
|
663
663
|
Setting `NOT NULL` on an existing column blocks reads and writes while every row is checked.
|
664
664
|
|
665
665
|
```ruby
|
666
|
-
class ChangeUsersNameNull < ActiveRecord::Migration[
|
666
|
+
class ChangeUsersNameNull < ActiveRecord::Migration[8.0]
|
667
667
|
def change
|
668
668
|
change_column_null :users, :name, false
|
669
669
|
end
|
@@ -675,7 +675,7 @@ end
|
|
675
675
|
Instead, add a check constraint and validate it in a separate transaction:
|
676
676
|
|
677
677
|
```ruby
|
678
|
-
class ChangeUsersNameNull < ActiveRecord::Migration[
|
678
|
+
class ChangeUsersNameNull < ActiveRecord::Migration[8.0]
|
679
679
|
disable_ddl_transaction!
|
680
680
|
|
681
681
|
def change
|
@@ -690,7 +690,7 @@ end
|
|
690
690
|
A `NOT NULL` check constraint is functionally equivalent to setting `NOT NULL` on the column (but it won't show up in `schema.rb` in Rails < 6.1). In PostgreSQL 12+, once the check constraint is validated, you can safely set `NOT NULL` on the column and drop the check constraint.
|
691
691
|
|
692
692
|
```ruby
|
693
|
-
class ChangeUsersNameNullDropCheck < ActiveRecord::Migration[
|
693
|
+
class ChangeUsersNameNullDropCheck < ActiveRecord::Migration[8.0]
|
694
694
|
def change
|
695
695
|
# in PostgreSQL 12+, you can then safely set NOT NULL on the column
|
696
696
|
change_column_null :users, :name, false
|
@@ -704,7 +704,7 @@ end
|
|
704
704
|
Online Migrations does not support inspecting what happens inside an `execute` call, so cannot help you here. Make really sure that what you're doing is safe before proceeding, then wrap it in a `safety_assured { ... }` block:
|
705
705
|
|
706
706
|
```ruby
|
707
|
-
class ExecuteSQL < ActiveRecord::Migration[
|
707
|
+
class ExecuteSQL < ActiveRecord::Migration[8.0]
|
708
708
|
def change
|
709
709
|
safety_assured { execute "..." }
|
710
710
|
end
|
@@ -718,7 +718,7 @@ end
|
|
718
718
|
Adding an index non-concurrently blocks writes.
|
719
719
|
|
720
720
|
```ruby
|
721
|
-
class AddIndexOnUsersEmail < ActiveRecord::Migration[
|
721
|
+
class AddIndexOnUsersEmail < ActiveRecord::Migration[8.0]
|
722
722
|
def change
|
723
723
|
add_index :users, :email, unique: true
|
724
724
|
end
|
@@ -730,7 +730,7 @@ end
|
|
730
730
|
Add indexes concurrently.
|
731
731
|
|
732
732
|
```ruby
|
733
|
-
class AddIndexOnUsersEmail < ActiveRecord::Migration[
|
733
|
+
class AddIndexOnUsersEmail < ActiveRecord::Migration[8.0]
|
734
734
|
disable_ddl_transaction!
|
735
735
|
|
736
736
|
def change
|
@@ -748,7 +748,7 @@ end
|
|
748
748
|
While actual removing of an index is usually fast, removing it non-concurrently tries to obtain an `ACCESS EXCLUSIVE` lock on the table, waiting for all existing queries to complete and blocking all the subsequent queries (even `SELECT`s) on that table until the lock is obtained and index is removed.
|
749
749
|
|
750
750
|
```ruby
|
751
|
-
class RemoveIndexOnUsersEmail < ActiveRecord::Migration[
|
751
|
+
class RemoveIndexOnUsersEmail < ActiveRecord::Migration[8.0]
|
752
752
|
def change
|
753
753
|
remove_index :users, :email
|
754
754
|
end
|
@@ -760,7 +760,7 @@ end
|
|
760
760
|
Remove indexes concurrently.
|
761
761
|
|
762
762
|
```ruby
|
763
|
-
class RemoveIndexOnUsersEmail < ActiveRecord::Migration[
|
763
|
+
class RemoveIndexOnUsersEmail < ActiveRecord::Migration[8.0]
|
764
764
|
disable_ddl_transaction!
|
765
765
|
|
766
766
|
def change
|
@@ -778,7 +778,7 @@ end
|
|
778
778
|
Removing an old index before replacing it with the new one might result in slow queries while building the new index.
|
779
779
|
|
780
780
|
```ruby
|
781
|
-
class AddIndexOnCreationToProjects < ActiveRecord::Migration[
|
781
|
+
class AddIndexOnCreationToProjects < ActiveRecord::Migration[8.0]
|
782
782
|
disable_ddl_transaction!
|
783
783
|
|
784
784
|
def change
|
@@ -795,7 +795,7 @@ end
|
|
795
795
|
A safer approach is to create the new index and then delete the old one.
|
796
796
|
|
797
797
|
```ruby
|
798
|
-
class AddIndexOnCreationToProjects < ActiveRecord::Migration[
|
798
|
+
class AddIndexOnCreationToProjects < ActiveRecord::Migration[8.0]
|
799
799
|
disable_ddl_transaction!
|
800
800
|
|
801
801
|
def change
|
@@ -812,7 +812,7 @@ end
|
|
812
812
|
Rails adds an index non-concurrently to references by default, which blocks writes. Additionally, if `foreign_key` option (without `validate: false`) is provided, both tables are blocked while it is validated.
|
813
813
|
|
814
814
|
```ruby
|
815
|
-
class AddUserToProjects < ActiveRecord::Migration[
|
815
|
+
class AddUserToProjects < ActiveRecord::Migration[8.0]
|
816
816
|
def change
|
817
817
|
add_reference :projects, :user, foreign_key: true
|
818
818
|
end
|
@@ -825,7 +825,7 @@ Make sure the index is added concurrently and the foreign key is added in a sepa
|
|
825
825
|
Or you can use `add_reference_concurrently` helper. It will create a reference and take care of safely adding index and/or foreign key.
|
826
826
|
|
827
827
|
```ruby
|
828
|
-
class AddUserToProjects < ActiveRecord::Migration[
|
828
|
+
class AddUserToProjects < ActiveRecord::Migration[8.0]
|
829
829
|
disable_ddl_transaction!
|
830
830
|
|
831
831
|
def change
|
@@ -843,7 +843,7 @@ end
|
|
843
843
|
Adding a foreign key blocks writes on both tables.
|
844
844
|
|
845
845
|
```ruby
|
846
|
-
class AddForeignKeyToProjectsUser < ActiveRecord::Migration[
|
846
|
+
class AddForeignKeyToProjectsUser < ActiveRecord::Migration[8.0]
|
847
847
|
def change
|
848
848
|
add_foreign_key :projects, :users
|
849
849
|
end
|
@@ -853,7 +853,7 @@ end
|
|
853
853
|
or
|
854
854
|
|
855
855
|
```ruby
|
856
|
-
class AddReferenceToProjectsUser < ActiveRecord::Migration[
|
856
|
+
class AddReferenceToProjectsUser < ActiveRecord::Migration[8.0]
|
857
857
|
def change
|
858
858
|
add_reference :projects, :user, foreign_key: true
|
859
859
|
end
|
@@ -865,7 +865,7 @@ end
|
|
865
865
|
Add the foreign key without validating existing rows:
|
866
866
|
|
867
867
|
```ruby
|
868
|
-
class AddForeignKeyToProjectsUser < ActiveRecord::Migration[
|
868
|
+
class AddForeignKeyToProjectsUser < ActiveRecord::Migration[8.0]
|
869
869
|
def change
|
870
870
|
add_foreign_key :projects, :users, validate: false
|
871
871
|
end
|
@@ -875,7 +875,7 @@ end
|
|
875
875
|
Then validate them in a separate migration:
|
876
876
|
|
877
877
|
```ruby
|
878
|
-
class ValidateForeignKeyOnProjectsUser < ActiveRecord::Migration[
|
878
|
+
class ValidateForeignKeyOnProjectsUser < ActiveRecord::Migration[8.0]
|
879
879
|
def change
|
880
880
|
validate_foreign_key :projects, :users
|
881
881
|
end
|
@@ -889,7 +889,7 @@ end
|
|
889
889
|
Adding an exclusion constraint blocks reads and writes while every row is checked.
|
890
890
|
|
891
891
|
```ruby
|
892
|
-
class AddExclusionContraint < ActiveRecord::Migration[
|
892
|
+
class AddExclusionContraint < ActiveRecord::Migration[8.0]
|
893
893
|
def change
|
894
894
|
add_exclusion_constraint :users, "number WITH =", using: :gist
|
895
895
|
end
|
@@ -907,7 +907,7 @@ end
|
|
907
907
|
Adding a unique constraint blocks reads and writes while the underlying index is being built.
|
908
908
|
|
909
909
|
```ruby
|
910
|
-
class AddUniqueConstraint < ActiveRecord::Migration[
|
910
|
+
class AddUniqueConstraint < ActiveRecord::Migration[8.0]
|
911
911
|
def change
|
912
912
|
add_unique_constraint :sections, :position, deferrable: :deferred
|
913
913
|
end
|
@@ -919,7 +919,7 @@ end
|
|
919
919
|
A safer approach is to create a unique index first, and then create a unique key using that index.
|
920
920
|
|
921
921
|
```ruby
|
922
|
-
class AddUniqueConstraintAddIndex < ActiveRecord::Migration[
|
922
|
+
class AddUniqueConstraintAddIndex < ActiveRecord::Migration[8.0]
|
923
923
|
disable_ddl_transaction!
|
924
924
|
|
925
925
|
def change
|
@@ -929,7 +929,7 @@ end
|
|
929
929
|
```
|
930
930
|
|
931
931
|
```ruby
|
932
|
-
class AddUniqueConstraint < ActiveRecord::Migration[
|
932
|
+
class AddUniqueConstraint < ActiveRecord::Migration[8.0]
|
933
933
|
def up
|
934
934
|
add_unique_constraint :sections, :position, deferrable: :deferred, using_index: "index_sections_on_position"
|
935
935
|
end
|
@@ -947,7 +947,7 @@ end
|
|
947
947
|
There's no equality operator for the `json` column type, which can cause errors for existing `SELECT DISTINCT` queries in your application.
|
948
948
|
|
949
949
|
```ruby
|
950
|
-
class AddSettingsToProjects < ActiveRecord::Migration[
|
950
|
+
class AddSettingsToProjects < ActiveRecord::Migration[8.0]
|
951
951
|
def change
|
952
952
|
add_column :projects, :settings, :json
|
953
953
|
end
|
@@ -959,7 +959,7 @@ end
|
|
959
959
|
Use `jsonb` instead.
|
960
960
|
|
961
961
|
```ruby
|
962
|
-
class AddSettingsToProjects < ActiveRecord::Migration[
|
962
|
+
class AddSettingsToProjects < ActiveRecord::Migration[8.0]
|
963
963
|
def change
|
964
964
|
add_column :projects, :settings, :jsonb
|
965
965
|
end
|
@@ -973,7 +973,7 @@ end
|
|
973
973
|
Adding a stored generated column causes the entire table to be rewritten. During this time, reads and writes are blocked.
|
974
974
|
|
975
975
|
```ruby
|
976
|
-
class AddLowerEmailToUsers < ActiveRecord::Migration[
|
976
|
+
class AddLowerEmailToUsers < ActiveRecord::Migration[8.0]
|
977
977
|
def change
|
978
978
|
add_column :users, :lower_email, :virtual, type: :string, as: "LOWER(email)", stored: true
|
979
979
|
end
|
@@ -991,7 +991,7 @@ Add a non-generated column and use callbacks or triggers instead.
|
|
991
991
|
When using short integer types as primary key types, [there is a risk](https://m.signalvnoise.com/update-on-basecamp-3-being-stuck-in-read-only-as-of-nov-8-922am-cst/) of running out of IDs on inserts. The default type in Active Record < 5.1 for primary and foreign keys is `INTEGER`, which allows a little over of 2 billion records. Active Record 5.1 changed the default type to `BIGINT`.
|
992
992
|
|
993
993
|
```ruby
|
994
|
-
class CreateUsers < ActiveRecord::Migration[
|
994
|
+
class CreateUsers < ActiveRecord::Migration[8.0]
|
995
995
|
def change
|
996
996
|
create_table :users, id: :integer do |t|
|
997
997
|
# ...
|
@@ -1005,7 +1005,7 @@ end
|
|
1005
1005
|
Use one of `bigint`, `bigserial`, `uuid` instead.
|
1006
1006
|
|
1007
1007
|
```ruby
|
1008
|
-
class CreateUsers < ActiveRecord::Migration[
|
1008
|
+
class CreateUsers < ActiveRecord::Migration[8.0]
|
1009
1009
|
def change
|
1010
1010
|
create_table :users, id: :bigint do |t| # bigint is the default for Active Record >= 5.1
|
1011
1011
|
# ...
|
@@ -1021,7 +1021,7 @@ end
|
|
1021
1021
|
Hash index operations are not WAL-logged, so hash indexes might need to be rebuilt with `REINDEX` after a database crash if there were unwritten changes. Also, changes to hash indexes are not replicated over streaming or file-based replication after the initial base backup, so they give wrong answers to queries that subsequently use them. For these reasons, hash index use is discouraged.
|
1022
1022
|
|
1023
1023
|
```ruby
|
1024
|
-
class AddIndexToUsersOnEmail < ActiveRecord::Migration[
|
1024
|
+
class AddIndexToUsersOnEmail < ActiveRecord::Migration[8.0]
|
1025
1025
|
def change
|
1026
1026
|
add_index :users, :email, unique: true, using: :hash
|
1027
1027
|
end
|
@@ -1033,7 +1033,7 @@ end
|
|
1033
1033
|
Use B-tree indexes instead.
|
1034
1034
|
|
1035
1035
|
```ruby
|
1036
|
-
class AddIndexToUsersOnEmail < ActiveRecord::Migration[
|
1036
|
+
class AddIndexToUsersOnEmail < ActiveRecord::Migration[8.0]
|
1037
1037
|
def change
|
1038
1038
|
add_index :users, :email, unique: true # B-tree by default
|
1039
1039
|
end
|
@@ -1048,7 +1048,7 @@ Adding multiple foreign keys in a single migration blocks writes on all involved
|
|
1048
1048
|
Avoid adding foreign key more than once per migration file, unless the source and target tables are identical.
|
1049
1049
|
|
1050
1050
|
```ruby
|
1051
|
-
class CreateUserProjects < ActiveRecord::Migration[
|
1051
|
+
class CreateUserProjects < ActiveRecord::Migration[8.0]
|
1052
1052
|
def change
|
1053
1053
|
create_table :user_projects do |t|
|
1054
1054
|
t.belongs_to :user, foreign_key: true
|
@@ -1063,7 +1063,7 @@ end
|
|
1063
1063
|
Add additional foreign keys in separate migration files. See [adding a foreign key](#adding-a-foreign-key) for how to properly add foreign keys.
|
1064
1064
|
|
1065
1065
|
```ruby
|
1066
|
-
class CreateUserProjects < ActiveRecord::Migration[
|
1066
|
+
class CreateUserProjects < ActiveRecord::Migration[8.0]
|
1067
1067
|
def change
|
1068
1068
|
create_table :user_projects do |t|
|
1069
1069
|
t.belongs_to :user, foreign_key: true
|
@@ -1072,7 +1072,7 @@ class CreateUserProjects < ActiveRecord::Migration[7.2]
|
|
1072
1072
|
end
|
1073
1073
|
end
|
1074
1074
|
|
1075
|
-
class AddForeignKeyFromUserProjectsToProject < ActiveRecord::Migration[
|
1075
|
+
class AddForeignKeyFromUserProjectsToProject < ActiveRecord::Migration[8.0]
|
1076
1076
|
def change
|
1077
1077
|
add_foreign_key :user_projects, :projects
|
1078
1078
|
end
|
@@ -1089,7 +1089,7 @@ Remove all the foreign keys first.
|
|
1089
1089
|
Assuming, `projects` has foreign keys on `users.id` and `repositories.id`:
|
1090
1090
|
|
1091
1091
|
```ruby
|
1092
|
-
class DropProjects < ActiveRecord::Migration[
|
1092
|
+
class DropProjects < ActiveRecord::Migration[8.0]
|
1093
1093
|
def change
|
1094
1094
|
drop_table :projects
|
1095
1095
|
end
|
@@ -1101,13 +1101,13 @@ end
|
|
1101
1101
|
Remove all the foreign keys first:
|
1102
1102
|
|
1103
1103
|
```ruby
|
1104
|
-
class RemoveProjectsUserFk < ActiveRecord::Migration[
|
1104
|
+
class RemoveProjectsUserFk < ActiveRecord::Migration[8.0]
|
1105
1105
|
def change
|
1106
1106
|
remove_foreign_key :projects, :users
|
1107
1107
|
end
|
1108
1108
|
end
|
1109
1109
|
|
1110
|
-
class RemoveProjectsRepositoryFk < ActiveRecord::Migration[
|
1110
|
+
class RemoveProjectsRepositoryFk < ActiveRecord::Migration[8.0]
|
1111
1111
|
def change
|
1112
1112
|
remove_foreign_key :projects, :repositories
|
1113
1113
|
end
|
@@ -1117,7 +1117,7 @@ end
|
|
1117
1117
|
Then remove the table:
|
1118
1118
|
|
1119
1119
|
```ruby
|
1120
|
-
class DropProjects < ActiveRecord::Migration[
|
1120
|
+
class DropProjects < ActiveRecord::Migration[8.0]
|
1121
1121
|
def change
|
1122
1122
|
drop_table :projects
|
1123
1123
|
end
|
@@ -1134,7 +1134,7 @@ Otherwise, there's a risk of bugs caused by IDs representable by one type but no
|
|
1134
1134
|
Assuming, there is a `users` table with `bigint` primary key type:
|
1135
1135
|
|
1136
1136
|
```ruby
|
1137
|
-
class AddUserIdToProjects < ActiveRecord::Migration[
|
1137
|
+
class AddUserIdToProjects < ActiveRecord::Migration[8.0]
|
1138
1138
|
def change
|
1139
1139
|
add_column :projects, :user_id, :integer
|
1140
1140
|
end
|
@@ -1148,7 +1148,7 @@ Add a reference column of the same type as a referenced primary key.
|
|
1148
1148
|
Assuming, there is a `users` table with `bigint` primary key type:
|
1149
1149
|
|
1150
1150
|
```ruby
|
1151
|
-
class AddUserIdToProjects < ActiveRecord::Migration[
|
1151
|
+
class AddUserIdToProjects < ActiveRecord::Migration[8.0]
|
1152
1152
|
def change
|
1153
1153
|
add_column :projects, :user_id, :bigint
|
1154
1154
|
end
|
@@ -1162,7 +1162,7 @@ end
|
|
1162
1162
|
Adding a single table inheritance column might cause errors in old instances of your application.
|
1163
1163
|
|
1164
1164
|
```ruby
|
1165
|
-
class AddTypeToUsers < ActiveRecord::Migration[
|
1165
|
+
class AddTypeToUsers < ActiveRecord::Migration[8.0]
|
1166
1166
|
def change
|
1167
1167
|
add_column :users, :string, :type, default: "Member"
|
1168
1168
|
end
|
@@ -1194,7 +1194,7 @@ A safer approach is to:
|
|
1194
1194
|
Active Record < 7 enables partial writes by default, which can cause incorrect values to be inserted when changing the default value of a column.
|
1195
1195
|
|
1196
1196
|
```ruby
|
1197
|
-
class ChangeSomeColumnDefault < ActiveRecord::Migration[
|
1197
|
+
class ChangeSomeColumnDefault < ActiveRecord::Migration[8.0]
|
1198
1198
|
def change
|
1199
1199
|
change_column_default :users, :some_column, from: "old", to: "new"
|
1200
1200
|
end
|
@@ -1222,7 +1222,7 @@ config.active_record.partial_inserts = false
|
|
1222
1222
|
To mark a step in the migration as safe, despite using a method that might otherwise be dangerous, wrap it in a `safety_assured` block.
|
1223
1223
|
|
1224
1224
|
```ruby
|
1225
|
-
class MySafeMigration < ActiveRecord::Migration[
|
1225
|
+
class MySafeMigration < ActiveRecord::Migration[8.0]
|
1226
1226
|
def change
|
1227
1227
|
safety_assured { remove_column :users, :some_column }
|
1228
1228
|
end
|
@@ -78,7 +78,7 @@ You can enqueue your background migration to be run by the scheduler via:
|
|
78
78
|
|
79
79
|
```ruby
|
80
80
|
# db/migrate/xxxxxxxxxxxxxx_enqueue_backfill_project_issues_count.rb
|
81
|
-
class EnqueueBackfillProjectIssuesCount < ActiveRecord::Migration[
|
81
|
+
class EnqueueBackfillProjectIssuesCount < ActiveRecord::Migration[8.0]
|
82
82
|
def up
|
83
83
|
enqueue_background_data_migration("BackfillProjectIssuesCount")
|
84
84
|
end
|
@@ -34,33 +34,39 @@ or run it manually when the deployment is finished, from the rails console:
|
|
34
34
|
|
35
35
|
## Enqueueing a Background Schema Migration
|
36
36
|
|
37
|
-
Currently, only helpers for adding/removing indexes are provided.
|
37
|
+
Currently, only helpers for adding/removing indexes and validating constraints are provided.
|
38
38
|
|
39
|
-
Background schema migrations should be performed in 2 steps:
|
39
|
+
Background schema migrations should be performed in 2 steps (e.g. for index creation):
|
40
40
|
|
41
|
-
1. Create a PR that schedules the index to be created
|
42
|
-
2. Verify that the PR was deployed and that the index was actually created
|
43
|
-
Create a follow-up PR with a regular migration that creates
|
41
|
+
1. Create a PR that schedules the index to be created
|
42
|
+
2. Verify that the PR was deployed and that the index was actually created on production.
|
43
|
+
Create a follow-up PR with a regular migration that creates an index synchronously (will be a no op when run on production) and commit the schema changes for `schema.rb`/`structure.sql`
|
44
44
|
|
45
45
|
To schedule an index creation:
|
46
46
|
|
47
47
|
```ruby
|
48
|
-
|
49
|
-
def up
|
50
|
-
add_index_in_background(:users, :email, unique: true)
|
51
|
-
end
|
48
|
+
add_index_in_background(:users, :email, unique: true)
|
52
49
|
```
|
53
50
|
|
54
51
|
To schedule an index removal:
|
55
52
|
|
56
53
|
```ruby
|
57
|
-
|
58
|
-
|
59
|
-
|
60
|
-
|
54
|
+
remove_index_in_background(:users, name: "index_users_on_email")
|
55
|
+
```
|
56
|
+
|
57
|
+
To schedule a foreign key validation:
|
58
|
+
|
59
|
+
```ruby
|
60
|
+
validate_foreign_key_in_background(:users, :companies)
|
61
|
+
```
|
62
|
+
|
63
|
+
To schedule a constraint (`CHECK` constraint, `FOREIGN KEY` constraint) validation:
|
64
|
+
|
65
|
+
```ruby
|
66
|
+
validate_constraint_in_background(:users, "first_name_not_null")
|
61
67
|
```
|
62
68
|
|
63
|
-
|
69
|
+
All the helpers accept additional configuration options which controls how the background schema migration is run. Check the [source code](https://github.com/fatkodima/online_migrations/blob/master/lib/online_migrations/background_schema_migrations/migration_helpers.rb) for the list of all available configuration options.
|
64
70
|
|
65
71
|
## Depending on schema changes
|
66
72
|
|
data/docs/configuring.md
CHANGED
@@ -156,7 +156,7 @@ This is useful to demystify `online_migrations` inner workings, and to better in
|
|
156
156
|
Consider migration, running on PostgreSQL < 11:
|
157
157
|
|
158
158
|
```ruby
|
159
|
-
class AddAdminToUsers < ActiveRecord::Migration[
|
159
|
+
class AddAdminToUsers < ActiveRecord::Migration[8.0]
|
160
160
|
disable_ddl_transaction!
|
161
161
|
|
162
162
|
def change
|
@@ -45,6 +45,26 @@ module OnlineMigrations
|
|
45
45
|
enqueue_background_schema_migration(name, table_name, definition: definition, **migration_options)
|
46
46
|
end
|
47
47
|
|
48
|
+
def validate_foreign_key_in_background(from_table, to_table = nil, **options)
|
49
|
+
migration_options = options.extract!(:max_attempts, :statement_timeout, :connection_class_name)
|
50
|
+
|
51
|
+
if !foreign_key_exists?(from_table, to_table, **options)
|
52
|
+
Utils.raise_or_say("Foreign key validation was not enqueued because the foreign key does not exist.")
|
53
|
+
return
|
54
|
+
end
|
55
|
+
|
56
|
+
fk_name_to_validate = foreign_key_for!(from_table, to_table: to_table, **options).name
|
57
|
+
validate_constraint_in_background(from_table, fk_name_to_validate, **migration_options)
|
58
|
+
end
|
59
|
+
|
60
|
+
def validate_constraint_in_background(table_name, constraint_name, **options)
|
61
|
+
definition = <<~SQL.squish
|
62
|
+
ALTER TABLE #{quote_table_name(table_name)}
|
63
|
+
VALIDATE CONSTRAINT #{quote_table_name(constraint_name)}
|
64
|
+
SQL
|
65
|
+
enqueue_background_schema_migration(constraint_name, table_name, definition: definition, **options)
|
66
|
+
end
|
67
|
+
|
48
68
|
# Ensures that the background schema migration with the provided migration name succeeded.
|
49
69
|
#
|
50
70
|
# If the enqueued migration was not found in development (probably when resetting a dev environment
|
@@ -268,6 +268,8 @@ class <%= migration_name %> < <%= migration_parent %>
|
|
268
268
|
end
|
269
269
|
|
270
270
|
<% end %>
|
271
|
+
# You can use `validate_constraint_in_background` if you have a very large table
|
272
|
+
# and want to validate the constraint using background schema migrations.
|
271
273
|
<%= validate_constraint_code %>
|
272
274
|
<% if remove_constraint_code %>
|
273
275
|
|
@@ -404,6 +406,8 @@ end
|
|
404
406
|
|
405
407
|
class Validate<%= migration_name %> < <%= migration_parent %>
|
406
408
|
def change
|
409
|
+
# You can use `validate_foreign_key_in_background` if you have a very large table
|
410
|
+
# and want to validate the foreign key using background schema migrations.
|
407
411
|
<%= validate_code %>
|
408
412
|
end
|
409
413
|
end",
|
@@ -176,7 +176,7 @@ module OnlineMigrations
|
|
176
176
|
# # This will attempt 30 retries starting with delay of 10ms between each unsuccessful try, increasing exponentially
|
177
177
|
# # up to the maximum delay of 1 minute and 200ms set as lock timeout for each try:
|
178
178
|
#
|
179
|
-
# config.retrier = OnlineMigrations::
|
179
|
+
# config.retrier = OnlineMigrations::ExponentialLockRetrier.new(attempts: 30,
|
180
180
|
# base_delay: 0.01.seconds, max_delay: 1.minute, lock_timeout: 0.2.seconds)
|
181
181
|
#
|
182
182
|
class ExponentialLockRetrier < LockRetrier
|
@@ -520,7 +520,17 @@ module OnlineMigrations
|
|
520
520
|
#
|
521
521
|
def validate_not_null_constraint(table_name, column_name, name: nil)
|
522
522
|
name ||= __not_null_constraint_name(table_name, column_name)
|
523
|
-
|
523
|
+
column = column_for(table_name, column_name)
|
524
|
+
|
525
|
+
if column.null == false &&
|
526
|
+
!__not_null_constraint_exists?(table_name, column_name, name: name)
|
527
|
+
Utils.say(<<~MSG.squish)
|
528
|
+
NOT NULL constraint was not validated: it does not exist and
|
529
|
+
column #{table_name}.#{column_name} is already defined as `NOT NULL`
|
530
|
+
MSG
|
531
|
+
else
|
532
|
+
validate_check_constraint(table_name, name: name)
|
533
|
+
end
|
524
534
|
end
|
525
535
|
|
526
536
|
# Removes a NOT NULL constraint from the column
|
@@ -803,13 +813,13 @@ module OnlineMigrations
|
|
803
813
|
# @see https://edgeapi.rubyonrails.org/classes/ActiveRecord/ConnectionAdapters/SchemaStatements.html#method-i-add_foreign_key
|
804
814
|
#
|
805
815
|
def add_foreign_key(from_table, to_table, **options)
|
806
|
-
|
807
|
-
if foreign_key_exists?(from_table, to_table, **options.except(:validate))
|
808
|
-
message = +"Foreign key was not created because it already exists " \
|
809
|
-
"(this can be due to an aborted migration or similar): from_table: #{from_table}, to_table: #{to_table}"
|
810
|
-
message << ", #{options.inspect}" if options.any?
|
816
|
+
options = foreign_key_options(from_table, to_table, options)
|
811
817
|
|
812
|
-
|
818
|
+
if foreign_key_exists?(from_table, to_table, **options.slice(:column, :primary_key))
|
819
|
+
Utils.say(<<~MSG.squish)
|
820
|
+
Foreign key was not created because it already exists (this can be due to an aborted migration or similar).
|
821
|
+
from_table: #{from_table}, to_table: #{to_table}, options: #{options.inspect}
|
822
|
+
MSG
|
813
823
|
else
|
814
824
|
super
|
815
825
|
end
|
@@ -844,6 +854,21 @@ module OnlineMigrations
|
|
844
854
|
end
|
845
855
|
end
|
846
856
|
|
857
|
+
# Extends default method to be idempotent.
|
858
|
+
#
|
859
|
+
# @see https://edgeapi.rubyonrails.org/classes/ActiveRecord/ConnectionAdapters/SchemaStatements.html#method-i-remove_foreign_key
|
860
|
+
#
|
861
|
+
def remove_foreign_key(from_table, to_table = nil, **options)
|
862
|
+
if foreign_key_exists?(from_table, to_table, **options.slice(:name, :to_table, :column))
|
863
|
+
super
|
864
|
+
else
|
865
|
+
Utils.say(<<~MSG.squish)
|
866
|
+
Foreign key was not removed because it does not exist (this may be due to an aborted migration or similar).
|
867
|
+
from_table: #{from_table}, to_table: #{to_table}, options: #{options.inspect}
|
868
|
+
MSG
|
869
|
+
end
|
870
|
+
end
|
871
|
+
|
847
872
|
# Extends default method to be idempotent
|
848
873
|
#
|
849
874
|
# @see https://edgeapi.rubyonrails.org/classes/ActiveRecord/ConnectionAdapters/SchemaStatements.html#method-i-add_check_constraint
|
@@ -888,6 +913,21 @@ module OnlineMigrations
|
|
888
913
|
end
|
889
914
|
end
|
890
915
|
|
916
|
+
# Extends default method to be idempotent
|
917
|
+
#
|
918
|
+
# @see https://edgeapi.rubyonrails.org/classes/ActiveRecord/ConnectionAdapters/SchemaStatements.html#method-i-remove_check_constraint
|
919
|
+
#
|
920
|
+
def remove_check_constraint(table_name, expression = nil, **options)
|
921
|
+
if __check_constraint_exists?(table_name, expression: expression, **options)
|
922
|
+
super
|
923
|
+
else
|
924
|
+
Utils.say(<<~MSG.squish)
|
925
|
+
Check constraint was not removed because it does not exist (this may be due to an aborted migration or similar).
|
926
|
+
table_name: #{table_name}, expression: #{expression}, options: #{options.inspect}
|
927
|
+
MSG
|
928
|
+
end
|
929
|
+
end
|
930
|
+
|
891
931
|
if Utils.ar_version >= 7.1
|
892
932
|
def add_exclusion_constraint(table_name, expression, **options)
|
893
933
|
if __exclusion_constraint_exists?(table_name, expression: expression, **options)
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: online_migrations
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.21.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- fatkodima
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2024-
|
11
|
+
date: 2024-12-09 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activerecord
|