strong_migrations 1.2.0 → 1.6.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +48 -0
- data/README.md +87 -14
- data/lib/strong_migrations/adapters/abstract_adapter.rb +17 -1
- data/lib/strong_migrations/adapters/mariadb_adapter.rb +3 -0
- data/lib/strong_migrations/adapters/mysql_adapter.rb +27 -7
- data/lib/strong_migrations/adapters/postgresql_adapter.rb +9 -6
- data/lib/strong_migrations/checker.rb +10 -3
- data/lib/strong_migrations/checks.rb +78 -16
- data/lib/strong_migrations/error_messages.rb +25 -2
- data/lib/strong_migrations/safe_methods.rb +13 -3
- data/lib/strong_migrations/schema_dumper.rb +21 -0
- data/lib/strong_migrations/version.rb +1 -1
- data/lib/strong_migrations.rb +30 -15
- data/lib/tasks/strong_migrations.rake +2 -7
- metadata +4 -4
- data/lib/strong_migrations/alphabetize_columns.rb +0 -11
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: f84ebb54319b6d614f0908355239495e937676836e4dfaad26e0dc343f091343
|
4
|
+
data.tar.gz: 9b5dc06db468b4ce8f5d3460267b185e2e6c69df236c65bfb74ddcd5b04b3546
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: c3371f6b39b1c870efa5d2bc0737041ce3b1139c9f6a47e6018636007526ba437746917fee26721123ce17376a67829cd25c79fc9a8c29f2257f54189de314a9
|
7
|
+
data.tar.gz: 9ad328fd87f0d32d071235a5425443b378b35cfb4ab5546fb768f9a92f3bd808baf316764b03b9b7aa315cc1c0b41e60b8857bb08def411665ee7832e7f24068
|
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,51 @@
|
|
1
|
+
## 1.6.1 (2023-08-09)
|
2
|
+
|
3
|
+
- Fixed `safety_assured` for custom checks with `safe_by_default`
|
4
|
+
|
5
|
+
## 1.6.0 (2023-07-22)
|
6
|
+
|
7
|
+
- Added check for `change_column_default`
|
8
|
+
|
9
|
+
## 1.5.0 (2023-07-02)
|
10
|
+
|
11
|
+
- Added check for `add_column` with stored generated columns
|
12
|
+
- Fixed `add_reference` with `foreign_key` and `index: false`
|
13
|
+
|
14
|
+
## 1.4.4 (2023-03-08)
|
15
|
+
|
16
|
+
- Fixed `add_foreign_key` with `name` and `column` options with `safe_by_default`
|
17
|
+
|
18
|
+
## 1.4.3 (2023-02-19)
|
19
|
+
|
20
|
+
- Fixed check for `change_column` to account for charset with MySQL and MariaDB
|
21
|
+
|
22
|
+
## 1.4.2 (2023-01-29)
|
23
|
+
|
24
|
+
- Added `alphabetize_schema` option
|
25
|
+
|
26
|
+
## 1.4.1 (2023-01-05)
|
27
|
+
|
28
|
+
- Added support for multiple databases to `target_version`
|
29
|
+
|
30
|
+
## 1.4.0 (2022-10-31)
|
31
|
+
|
32
|
+
- Added check for `add_exclusion_constraint`
|
33
|
+
- Added support for `RACK_ENV`
|
34
|
+
- Fixed error when `Rails` defined without `Rails.env`
|
35
|
+
- Fixed error with `change_column_null` when table does not exist
|
36
|
+
|
37
|
+
## 1.3.2 (2022-10-09)
|
38
|
+
|
39
|
+
- Improved error message for `add_column` with `default: nil` with Postgres 10
|
40
|
+
|
41
|
+
## 1.3.1 (2022-09-21)
|
42
|
+
|
43
|
+
- Fixed check for `add_column` with `default: nil` with Postgres 10
|
44
|
+
|
45
|
+
## 1.3.0 (2022-08-30)
|
46
|
+
|
47
|
+
- Added check for `add_column` with `uuid` type and volatile default value
|
48
|
+
|
1
49
|
## 1.2.0 (2022-06-10)
|
2
50
|
|
3
51
|
- Added check for index corruption with Postgres 14.0 to 14.3
|
data/README.md
CHANGED
@@ -62,6 +62,7 @@ Potentially dangerous operations:
|
|
62
62
|
- [removing a column](#removing-a-column)
|
63
63
|
- [adding a column with a default value](#adding-a-column-with-a-default-value)
|
64
64
|
- [backfilling data](#backfilling-data)
|
65
|
+
- [adding a stored generated column](#adding-a-stored-generated-column)
|
65
66
|
- [changing the type of a column](#changing-the-type-of-a-column)
|
66
67
|
- [renaming a column](#renaming-a-column)
|
67
68
|
- [renaming a table](#renaming-a-table)
|
@@ -74,9 +75,14 @@ Postgres-specific checks:
|
|
74
75
|
- [adding an index non-concurrently](#adding-an-index-non-concurrently)
|
75
76
|
- [adding a reference](#adding-a-reference)
|
76
77
|
- [adding a foreign key](#adding-a-foreign-key)
|
78
|
+
- [adding an exclusion constraint](#adding-an-exclusion-constraint)
|
77
79
|
- [adding a json column](#adding-a-json-column)
|
78
80
|
- [setting NOT NULL on an existing column](#setting-not-null-on-an-existing-column)
|
79
81
|
|
82
|
+
Config-specific checks:
|
83
|
+
|
84
|
+
- [changing the default value of a column](#changing-the-default-value-of-a-column)
|
85
|
+
|
80
86
|
Best practices:
|
81
87
|
|
82
88
|
- [keeping non-unique indexes to three columns or less](#keeping-non-unique-indexes-to-three-columns-or-less)
|
@@ -107,7 +113,7 @@ end
|
|
107
113
|
end
|
108
114
|
```
|
109
115
|
|
110
|
-
2. Deploy code
|
116
|
+
2. Deploy the code
|
111
117
|
3. Write a migration to remove the column (wrap in `safety_assured` block)
|
112
118
|
|
113
119
|
```ruby
|
@@ -118,7 +124,7 @@ end
|
|
118
124
|
end
|
119
125
|
```
|
120
126
|
|
121
|
-
4. Deploy and run migration
|
127
|
+
4. Deploy and run the migration
|
122
128
|
5. Remove the line added in step 1
|
123
129
|
|
124
130
|
### Adding a column with a default value
|
@@ -135,7 +141,7 @@ class AddSomeColumnToUsers < ActiveRecord::Migration[7.0]
|
|
135
141
|
end
|
136
142
|
```
|
137
143
|
|
138
|
-
In Postgres 11+, MySQL 8.0.12+, and MariaDB 10.3.2+, this no longer requires a table rewrite and is safe.
|
144
|
+
In Postgres 11+, MySQL 8.0.12+, and MariaDB 10.3.2+, this no longer requires a table rewrite and is safe (except for volatile functions like `gen_random_uuid()`).
|
139
145
|
|
140
146
|
#### Good
|
141
147
|
|
@@ -190,6 +196,24 @@ class BackfillSomeColumn < ActiveRecord::Migration[7.0]
|
|
190
196
|
end
|
191
197
|
```
|
192
198
|
|
199
|
+
### Adding a stored generated column
|
200
|
+
|
201
|
+
#### Bad
|
202
|
+
|
203
|
+
Adding a stored generated column causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB.
|
204
|
+
|
205
|
+
```ruby
|
206
|
+
class AddSomeColumnToUsers < ActiveRecord::Migration[7.0]
|
207
|
+
def change
|
208
|
+
add_column :users, :some_column, :virtual, type: :string, as: "...", stored: true
|
209
|
+
end
|
210
|
+
end
|
211
|
+
```
|
212
|
+
|
213
|
+
#### Good
|
214
|
+
|
215
|
+
Add a non-generated column and use callbacks or triggers instead (or a virtual generated column with MySQL and MariaDB).
|
216
|
+
|
193
217
|
### Changing the type of a column
|
194
218
|
|
195
219
|
#### Bad
|
@@ -223,7 +247,7 @@ And some in MySQL and MariaDB:
|
|
223
247
|
|
224
248
|
Type | Safe Changes
|
225
249
|
--- | ---
|
226
|
-
`string` | Increasing `:limit` from under
|
250
|
+
`string` | Increasing `:limit` from under 63 up to 63, increasing `:limit` from over 63 to the max (the threshold can be different if using an encoding other than `utf8mb4` - for instance, it’s 85 for `utf8mb3` and 255 for `latin1`)
|
227
251
|
|
228
252
|
#### Good
|
229
253
|
|
@@ -488,6 +512,24 @@ class ValidateForeignKeyOnUsers < ActiveRecord::Migration[7.0]
|
|
488
512
|
end
|
489
513
|
```
|
490
514
|
|
515
|
+
### Adding an exclusion constraint
|
516
|
+
|
517
|
+
#### Bad
|
518
|
+
|
519
|
+
In Postgres, adding an exclusion constraint blocks reads and writes while every row is checked.
|
520
|
+
|
521
|
+
```ruby
|
522
|
+
class AddExclusionContraint < ActiveRecord::Migration[7.1]
|
523
|
+
def change
|
524
|
+
add_exclusion_constraint :users, "number WITH =", using: :gist
|
525
|
+
end
|
526
|
+
end
|
527
|
+
```
|
528
|
+
|
529
|
+
#### Good
|
530
|
+
|
531
|
+
[Let us know](https://github.com/ankane/strong_migrations/issues/new) if you have a safe way to do this (exclusion constraints cannot be marked `NOT VALID`).
|
532
|
+
|
491
533
|
### Adding a json column
|
492
534
|
|
493
535
|
#### Bad
|
@@ -590,6 +632,36 @@ class ValidateSomeColumnNotNull < ActiveRecord::Migration[6.0]
|
|
590
632
|
end
|
591
633
|
```
|
592
634
|
|
635
|
+
### Changing the default value of a column
|
636
|
+
|
637
|
+
#### Bad
|
638
|
+
|
639
|
+
Rails < 7 enables partial writes by default, which can cause incorrect values to be inserted when changing the default value of a column.
|
640
|
+
|
641
|
+
```ruby
|
642
|
+
class ChangeSomeColumnDefault < ActiveRecord::Migration[6.1]
|
643
|
+
def change
|
644
|
+
change_column_default :users, :some_column, from: "old", to: "new"
|
645
|
+
end
|
646
|
+
end
|
647
|
+
|
648
|
+
User.create!(some_column: "old") # can insert "new"
|
649
|
+
```
|
650
|
+
|
651
|
+
#### Good
|
652
|
+
|
653
|
+
Disable partial writes in `config/application.rb`. For Rails < 7, use:
|
654
|
+
|
655
|
+
```ruby
|
656
|
+
config.active_record.partial_writes = false
|
657
|
+
```
|
658
|
+
|
659
|
+
For Rails 7, use:
|
660
|
+
|
661
|
+
```ruby
|
662
|
+
config.active_record.partial_inserts = false
|
663
|
+
```
|
664
|
+
|
593
665
|
### Keeping non-unique indexes to three columns or less
|
594
666
|
|
595
667
|
#### Bad
|
@@ -681,7 +753,7 @@ Disable specific checks with:
|
|
681
753
|
StrongMigrations.disable_check(:add_index)
|
682
754
|
```
|
683
755
|
|
684
|
-
Check the [source code](https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations.rb) for the list of keys.
|
756
|
+
Check the [source code](https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations/error_messages.rb) for the list of keys.
|
685
757
|
|
686
758
|
## Down Migrations / Rollbacks
|
687
759
|
|
@@ -805,6 +877,12 @@ The major version works well for Postgres, while the full version is recommended
|
|
805
877
|
|
806
878
|
For safety, this option only affects development and test environments. In other environments, the actual server version is always used.
|
807
879
|
|
880
|
+
If your app has multiple databases with different versions, with Rails 6.1+, you can use:
|
881
|
+
|
882
|
+
```ruby
|
883
|
+
StrongMigrations.target_version = {primary: 13, catalog: 15}
|
884
|
+
```
|
885
|
+
|
808
886
|
## Analyze Tables
|
809
887
|
|
810
888
|
Analyze tables automatically (to update planner statistics) after an index is added. Create an initializer with:
|
@@ -815,23 +893,18 @@ StrongMigrations.auto_analyze = true
|
|
815
893
|
|
816
894
|
## Faster Migrations
|
817
895
|
|
818
|
-
Only dump the schema when adding a new migration. If you use Git, add to
|
896
|
+
Only dump the schema when adding a new migration. If you use Git, add to `config/environments/development.rb`:
|
819
897
|
|
820
898
|
```rb
|
821
|
-
|
822
|
-
ActiveRecord::Base.dump_schema_after_migration = Rails.env.development? &&
|
823
|
-
`git status db/migrate/ --porcelain`.present?
|
824
|
-
end
|
825
|
-
|
826
|
-
task "db:migrate": "faster_migrations"
|
899
|
+
config.active_record.dump_schema_after_migration = `git status db/migrate/ --porcelain`.present?
|
827
900
|
```
|
828
901
|
|
829
902
|
## Schema Sanity
|
830
903
|
|
831
|
-
Columns can flip order in `db/schema.rb` when you have multiple developers. One way to prevent this is to [alphabetize them](https://www.pgrs.net/2008/03/12/alphabetize-schema-rb-columns/). Add to
|
904
|
+
Columns can flip order in `db/schema.rb` when you have multiple developers. One way to prevent this is to [alphabetize them](https://www.pgrs.net/2008/03/12/alphabetize-schema-rb-columns/). Add to `config/initializers/strong_migrations.rb`:
|
832
905
|
|
833
906
|
```ruby
|
834
|
-
|
907
|
+
StrongMigrations.alphabetize_schema = true
|
835
908
|
```
|
836
909
|
|
837
910
|
## Permissions
|
@@ -50,7 +50,23 @@ module StrongMigrations
|
|
50
50
|
target_version ||= StrongMigrations.target_version
|
51
51
|
version =
|
52
52
|
if target_version && StrongMigrations.developer_env?
|
53
|
-
target_version.
|
53
|
+
if target_version.is_a?(Hash)
|
54
|
+
# Active Record 6.0 supports multiple databases
|
55
|
+
# but connection.pool.spec.name always returns "primary"
|
56
|
+
# in migrations with rails db:migrate
|
57
|
+
if ActiveRecord::VERSION::STRING.to_f < 6.1
|
58
|
+
# error class is not shown in db:migrate output so ensure message is descriptive
|
59
|
+
raise StrongMigrations::Error, "StrongMigrations.target_version does not support multiple databases for Active Record < 6.1"
|
60
|
+
end
|
61
|
+
|
62
|
+
db_config_name = connection.pool.db_config.name
|
63
|
+
target_version.stringify_keys.fetch(db_config_name) do
|
64
|
+
# error class is not shown in db:migrate output so ensure message is descriptive
|
65
|
+
raise StrongMigrations::Error, "StrongMigrations.target_version is not configured for :#{db_config_name} database"
|
66
|
+
end.to_s
|
67
|
+
else
|
68
|
+
target_version.to_s
|
69
|
+
end
|
54
70
|
else
|
55
71
|
yield
|
56
72
|
end
|
@@ -18,6 +18,9 @@ module StrongMigrations
|
|
18
18
|
end
|
19
19
|
|
20
20
|
def set_statement_timeout(timeout)
|
21
|
+
# fix deprecation warning with Active Record 7.1
|
22
|
+
timeout = timeout.value if timeout.is_a?(ActiveSupport::Duration)
|
23
|
+
|
21
24
|
select_all("SET max_statement_time = #{connection.quote(timeout)}")
|
22
25
|
end
|
23
26
|
|
@@ -25,6 +25,9 @@ module StrongMigrations
|
|
25
25
|
end
|
26
26
|
|
27
27
|
def set_lock_timeout(timeout)
|
28
|
+
# fix deprecation warning with Active Record 7.1
|
29
|
+
timeout = timeout.value if timeout.is_a?(ActiveSupport::Duration)
|
30
|
+
|
28
31
|
select_all("SET lock_wait_timeout = #{connection.quote(timeout)}")
|
29
32
|
end
|
30
33
|
|
@@ -49,14 +52,31 @@ module StrongMigrations
|
|
49
52
|
|
50
53
|
case type.to_s
|
51
54
|
when "string"
|
52
|
-
# https://dev.mysql.com/doc/refman/5.7/en/innodb-online-ddl-operations.html
|
53
|
-
# https://mariadb.com/kb/en/innodb-online-ddl-operations-with-the-instant-alter-algorithm/#changing-the-data-type-of-a-column
|
54
|
-
# increased limit, but doesn't change number of length bytes
|
55
|
-
# 1-255 = 1 byte, 256-65532 = 2 bytes, 65533+ = too big for varchar
|
56
55
|
limit = options[:limit] || 255
|
57
|
-
|
58
|
-
|
59
|
-
|
56
|
+
if ["varchar"].include?(existing_type) && limit >= existing_column.limit
|
57
|
+
# https://dev.mysql.com/doc/refman/5.7/en/innodb-online-ddl-operations.html
|
58
|
+
# https://mariadb.com/kb/en/innodb-online-ddl-operations-with-the-instant-alter-algorithm/#changing-the-data-type-of-a-column
|
59
|
+
# increased limit, but doesn't change number of length bytes
|
60
|
+
# 1-255 = 1 byte, 256-65532 = 2 bytes, 65533+ = too big for varchar
|
61
|
+
|
62
|
+
# account for charset
|
63
|
+
# https://dev.mysql.com/doc/refman/8.0/en/charset-mysql.html
|
64
|
+
# https://mariadb.com/kb/en/supported-character-sets-and-collations/
|
65
|
+
sql = <<~SQL
|
66
|
+
SELECT cs.MAXLEN
|
67
|
+
FROM INFORMATION_SCHEMA.CHARACTER_SETS cs
|
68
|
+
INNER JOIN INFORMATION_SCHEMA.COLLATIONS c ON c.CHARACTER_SET_NAME = cs.CHARACTER_SET_NAME
|
69
|
+
INNER JOIN INFORMATION_SCHEMA.TABLES t ON t.TABLE_COLLATION = c.COLLATION_NAME
|
70
|
+
WHERE t.TABLE_SCHEMA = database() AND t.TABLE_NAME = #{connection.quote(table)}
|
71
|
+
SQL
|
72
|
+
row = connection.select_all(sql).first
|
73
|
+
if row
|
74
|
+
threshold = 255 / row["MAXLEN"]
|
75
|
+
safe = limit <= threshold || existing_column.limit > threshold
|
76
|
+
else
|
77
|
+
warn "[strong_migrations] Could not determine charset"
|
78
|
+
end
|
79
|
+
end
|
60
80
|
end
|
61
81
|
|
62
82
|
safe
|
@@ -14,11 +14,7 @@ module StrongMigrations
|
|
14
14
|
target_version(StrongMigrations.target_postgresql_version) do
|
15
15
|
version = select_all("SHOW server_version_num").first["server_version_num"].to_i
|
16
16
|
# major and minor version
|
17
|
-
|
18
|
-
"#{version / 10000}.#{(version % 10000)}"
|
19
|
-
else
|
20
|
-
"#{version / 10000}.#{(version % 10000) / 100}"
|
21
|
-
end
|
17
|
+
"#{version / 10000}.#{(version % 10000)}"
|
22
18
|
end
|
23
19
|
end
|
24
20
|
end
|
@@ -76,7 +72,7 @@ module StrongMigrations
|
|
76
72
|
# but there doesn't seem to be a way to set/modify it
|
77
73
|
# https://wiki.postgresql.org/wiki/What%27s_new_in_PostgreSQL_9.2#Reduce_ALTER_TABLE_rewrites
|
78
74
|
when "numeric", "decimal"
|
79
|
-
# numeric and decimal are equivalent and can be used
|
75
|
+
# numeric and decimal are equivalent and can be used interchangeably
|
80
76
|
safe = ["numeric", "decimal"].include?(existing_type) &&
|
81
77
|
(
|
82
78
|
(
|
@@ -166,6 +162,13 @@ module StrongMigrations
|
|
166
162
|
!StrongMigrations.developer_env?
|
167
163
|
end
|
168
164
|
|
165
|
+
# default to true if unsure
|
166
|
+
def default_volatile?(default)
|
167
|
+
name = default.to_s.delete_suffix("()")
|
168
|
+
rows = select_all("SELECT provolatile FROM pg_proc WHERE proname = #{connection.quote(name)}").to_a
|
169
|
+
rows.empty? || rows.any? { |r| r["provolatile"] == "v" }
|
170
|
+
end
|
171
|
+
|
169
172
|
private
|
170
173
|
|
171
174
|
def set_timeout(setting, timeout)
|
@@ -8,6 +8,7 @@ module StrongMigrations
|
|
8
8
|
def initialize(migration)
|
9
9
|
@migration = migration
|
10
10
|
@new_tables = []
|
11
|
+
@new_columns = []
|
11
12
|
@safe = false
|
12
13
|
@timeouts_set = false
|
13
14
|
@committed = false
|
@@ -36,6 +37,8 @@ module StrongMigrations
|
|
36
37
|
check_add_check_constraint(*args)
|
37
38
|
when :add_column
|
38
39
|
check_add_column(*args)
|
40
|
+
when :add_exclusion_constraint
|
41
|
+
check_add_exclusion_constraint(*args)
|
39
42
|
when :add_foreign_key
|
40
43
|
check_add_foreign_key(*args)
|
41
44
|
when :add_index
|
@@ -44,6 +47,8 @@ module StrongMigrations
|
|
44
47
|
check_add_reference(method, *args)
|
45
48
|
when :change_column
|
46
49
|
check_change_column(*args)
|
50
|
+
when :change_column_default
|
51
|
+
check_change_column_default(*args)
|
47
52
|
when :change_column_null
|
48
53
|
check_change_column_null(*args)
|
49
54
|
when :change_table
|
@@ -73,9 +78,11 @@ module StrongMigrations
|
|
73
78
|
@committed = true
|
74
79
|
end
|
75
80
|
|
76
|
-
|
77
|
-
|
78
|
-
|
81
|
+
if !safe?
|
82
|
+
# custom checks
|
83
|
+
StrongMigrations.checks.each do |check|
|
84
|
+
@migration.instance_exec(method, args, &check)
|
85
|
+
end
|
79
86
|
end
|
80
87
|
end
|
81
88
|
|
@@ -32,7 +32,14 @@ module StrongMigrations
|
|
32
32
|
table, column, type = args
|
33
33
|
default = options[:default]
|
34
34
|
|
35
|
-
|
35
|
+
# keep track of new columns of change_column_default check
|
36
|
+
@new_columns << [table.to_s, column.to_s]
|
37
|
+
|
38
|
+
# Check key since DEFAULT NULL behaves differently from no default
|
39
|
+
#
|
40
|
+
# Also, Active Record has special case for uuid columns that allows function default values
|
41
|
+
# https://github.com/rails/rails/blob/v7.0.3.1/activerecord/lib/active_record/connection_adapters/postgresql/quoting.rb#L92-L93
|
42
|
+
if options.key?(:default) && (!adapter.add_column_default_safe? || (volatile = (postgresql? && type.to_s == "uuid" && default.to_s.include?("()") && adapter.default_volatile?(default))))
|
36
43
|
if options[:null] == false
|
37
44
|
options = options.except(:null)
|
38
45
|
append = "
|
@@ -40,13 +47,21 @@ module StrongMigrations
|
|
40
47
|
Then add the NOT NULL constraint in separate migrations."
|
41
48
|
end
|
42
49
|
|
43
|
-
|
44
|
-
|
45
|
-
|
46
|
-
|
47
|
-
|
48
|
-
|
49
|
-
|
50
|
+
if default.nil?
|
51
|
+
raise_error :add_column_default_null,
|
52
|
+
command: command_str("add_column", [table, column, type, options.except(:default)]),
|
53
|
+
append: append,
|
54
|
+
rewrite_blocks: adapter.rewrite_blocks
|
55
|
+
else
|
56
|
+
raise_error :add_column_default,
|
57
|
+
add_command: command_str("add_column", [table, column, type, options.except(:default)]),
|
58
|
+
change_command: command_str("change_column_default", [table, column, default]),
|
59
|
+
remove_command: command_str("remove_column", [table, column]),
|
60
|
+
code: backfill_code(table, column, default, volatile),
|
61
|
+
append: append,
|
62
|
+
rewrite_blocks: adapter.rewrite_blocks,
|
63
|
+
default_type: (volatile ? "volatile" : "non-null")
|
64
|
+
end
|
50
65
|
elsif default.is_a?(Proc) && postgresql?
|
51
66
|
# adding a column with a VOLATILE default is not safe
|
52
67
|
# https://www.postgresql.org/docs/current/sql-altertable.html#SQL-ALTERTABLE-NOTES
|
@@ -59,6 +74,18 @@ Then add the NOT NULL constraint in separate migrations."
|
|
59
74
|
raise_error :add_column_json,
|
60
75
|
command: command_str("add_column", [table, column, :jsonb, options])
|
61
76
|
end
|
77
|
+
|
78
|
+
if type.to_s == "virtual" && options[:stored]
|
79
|
+
raise_error :add_column_generated_stored, rewrite_blocks: adapter.rewrite_blocks
|
80
|
+
end
|
81
|
+
end
|
82
|
+
|
83
|
+
def check_add_exclusion_constraint(*args)
|
84
|
+
table = args[0]
|
85
|
+
|
86
|
+
unless new_table?(table)
|
87
|
+
raise_error :add_exclusion_constraint
|
88
|
+
end
|
62
89
|
end
|
63
90
|
|
64
91
|
# unlike add_index, we don't make an exception here for new tables
|
@@ -127,7 +154,7 @@ Then add the NOT NULL constraint in separate migrations."
|
|
127
154
|
if bad_index || options[:foreign_key]
|
128
155
|
if index_value.is_a?(Hash)
|
129
156
|
options[:index] = options[:index].merge(algorithm: :concurrently)
|
130
|
-
|
157
|
+
elsif index_value
|
131
158
|
options = options.merge(index: {algorithm: :concurrently})
|
132
159
|
end
|
133
160
|
|
@@ -158,7 +185,8 @@ Then add the foreign key in separate migrations."
|
|
158
185
|
table, column, type = args
|
159
186
|
|
160
187
|
safe = false
|
161
|
-
|
188
|
+
table_columns = connection.columns(table) rescue []
|
189
|
+
existing_column = table_columns.find { |c| c.name.to_s == column.to_s }
|
162
190
|
if existing_column
|
163
191
|
existing_type = existing_column.sql_type.sub(/\(\d+(,\d+)?\)/, "")
|
164
192
|
safe = adapter.change_type_safe?(table, column, type, options, existing_column, existing_type)
|
@@ -173,6 +201,18 @@ Then add the foreign key in separate migrations."
|
|
173
201
|
raise_error :change_column, rewrite_blocks: adapter.rewrite_blocks unless safe
|
174
202
|
end
|
175
203
|
|
204
|
+
def check_change_column_default(*args)
|
205
|
+
table, column, _default_or_changes = args
|
206
|
+
|
207
|
+
# just check ActiveRecord::Base, even though can override on model
|
208
|
+
partial_inserts = ar_version >= 7 ? ActiveRecord::Base.partial_inserts : ActiveRecord::Base.partial_writes
|
209
|
+
|
210
|
+
if partial_inserts && !new_column?(table, column)
|
211
|
+
raise_error :change_column_default,
|
212
|
+
config: ar_version >= 7 ? "partial_inserts" : "partial_writes"
|
213
|
+
end
|
214
|
+
end
|
215
|
+
|
176
216
|
def check_change_column_null(*args)
|
177
217
|
table, column, null, default = args
|
178
218
|
if !null
|
@@ -219,16 +259,22 @@ Then add the foreign key in separate migrations."
|
|
219
259
|
|
220
260
|
add_constraint_code =
|
221
261
|
if constraint_methods
|
222
|
-
#
|
223
|
-
expr_column = column.to_s =~ /\A[a-z0-9_]+\z/ ? column : connection.quote_column_name(column)
|
224
|
-
command_str(:add_check_constraint, [table, "#{expr_column} IS NOT NULL", {name: constraint_name, validate: false}])
|
262
|
+
command_str(:add_check_constraint, [table, "#{quote_column_if_needed(column)} IS NOT NULL", {name: constraint_name, validate: false}])
|
225
263
|
else
|
226
264
|
safety_assured_str(add_code)
|
227
265
|
end
|
228
266
|
|
267
|
+
validate_constraint_code =
|
268
|
+
if safe_with_check_constraint
|
269
|
+
down_code = "#{add_constraint_code}\n #{command_str(:change_column_null, [table, column, true])}"
|
270
|
+
"def up\n #{validate_constraint_code}\n end\n\n def down\n #{down_code}\n end"
|
271
|
+
else
|
272
|
+
"def change\n #{validate_constraint_code}\n end"
|
273
|
+
end
|
274
|
+
|
229
275
|
raise_error :change_column_null_postgresql,
|
230
276
|
add_constraint_code: add_constraint_code,
|
231
|
-
validate_constraint_code:
|
277
|
+
validate_constraint_code: validate_constraint_code
|
232
278
|
end
|
233
279
|
elsif mysql? || mariadb?
|
234
280
|
unless adapter.strict_mode?
|
@@ -409,13 +455,29 @@ Then add the foreign key in separate migrations."
|
|
409
455
|
"#{command} #{str_args.join(", ")}"
|
410
456
|
end
|
411
457
|
|
412
|
-
def backfill_code(table, column, default)
|
458
|
+
def backfill_code(table, column, default, function = false)
|
413
459
|
model = table.to_s.classify
|
414
|
-
|
460
|
+
if function
|
461
|
+
# update_all(column: Arel.sql(default)) also works in newer versions of Active Record
|
462
|
+
update_expr = "#{quote_column_if_needed(column)} = #{default}"
|
463
|
+
"#{model}.unscoped.in_batches do |relation| \n relation.where(#{column}: nil).update_all(#{update_expr.inspect})\n sleep(0.01)\n end"
|
464
|
+
else
|
465
|
+
"#{model}.unscoped.in_batches do |relation| \n relation.update_all #{column}: #{default.inspect}\n sleep(0.01)\n end"
|
466
|
+
end
|
467
|
+
end
|
468
|
+
|
469
|
+
# only quote when needed
|
470
|
+
# important! only use for display purposes
|
471
|
+
def quote_column_if_needed(column)
|
472
|
+
column.to_s =~ /\A[a-z0-9_]+\z/ ? column : connection.quote_column_name(column)
|
415
473
|
end
|
416
474
|
|
417
475
|
def new_table?(table)
|
418
476
|
@new_tables.include?(table.to_s)
|
419
477
|
end
|
478
|
+
|
479
|
+
def new_column?(table, column)
|
480
|
+
new_table?(table) || @new_columns.include?([table.to_s, column.to_s])
|
481
|
+
end
|
420
482
|
end
|
421
483
|
end
|
@@ -1,7 +1,7 @@
|
|
1
1
|
module StrongMigrations
|
2
2
|
self.error_messages = {
|
3
3
|
add_column_default:
|
4
|
-
"Adding a column with a
|
4
|
+
"Adding a column with a %{default_type} default blocks %{rewrite_blocks} while the entire table is rewritten.
|
5
5
|
Instead, add the column without a default value, then change the default.
|
6
6
|
|
7
7
|
class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
|
@@ -25,6 +25,16 @@ class Backfill%{migration_name} < ActiveRecord::Migration%{migration_suffix}
|
|
25
25
|
end
|
26
26
|
end",
|
27
27
|
|
28
|
+
add_column_default_null:
|
29
|
+
"Adding a column with a null default blocks %{rewrite_blocks} while the entire table is rewritten.
|
30
|
+
Instead, add the column without a default value.
|
31
|
+
|
32
|
+
class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
|
33
|
+
def change
|
34
|
+
%{command}
|
35
|
+
end
|
36
|
+
end",
|
37
|
+
|
28
38
|
add_column_default_callable:
|
29
39
|
"Strong Migrations does not support inspecting callable default values.
|
30
40
|
Please make really sure you're not calling a VOLATILE function,
|
@@ -40,6 +50,9 @@ class %{migration_name} < ActiveRecord::Migration%{migration_suffix}
|
|
40
50
|
end
|
41
51
|
end",
|
42
52
|
|
53
|
+
add_column_generated_stored:
|
54
|
+
"Adding a stored generated column blocks %{rewrite_blocks} while the entire table is rewritten.",
|
55
|
+
|
43
56
|
change_column:
|
44
57
|
"Changing the type of an existing column blocks %{rewrite_blocks}
|
45
58
|
while the entire table is rewritten. A safer approach is to:
|
@@ -148,6 +161,13 @@ Otherwise, remove the force option.",
|
|
148
161
|
execute call, so cannot help you here. Please make really sure that what
|
149
162
|
you're doing is safe before proceeding, then wrap it in a safety_assured { ... } block.",
|
150
163
|
|
164
|
+
change_column_default:
|
165
|
+
"Partial writes are enabled, which can cause incorrect values
|
166
|
+
to be inserted when changing the default value of a column.
|
167
|
+
Disable partial writes in config/application.rb:
|
168
|
+
|
169
|
+
config.active_record.%{config} = false",
|
170
|
+
|
151
171
|
change_column_null:
|
152
172
|
"Passing a default value to change_column_null runs a single UPDATE query,
|
153
173
|
which can cause downtime. Instead, backfill the existing rows in the
|
@@ -221,7 +241,10 @@ end",
|
|
221
241
|
|
222
242
|
validate_check_constraint:
|
223
243
|
"Validating a check constraint while writes are blocked is dangerous.
|
224
|
-
Use disable_ddl_transaction! or a separate migration."
|
244
|
+
Use disable_ddl_transaction! or a separate migration.",
|
245
|
+
|
246
|
+
add_exclusion_constraint:
|
247
|
+
"Adding an exclusion constraint blocks reads and writes while every row is checked."
|
225
248
|
}
|
226
249
|
self.enabled_checks = (error_messages.keys - [:remove_index]).map { |k| [k, {}] }.to_h
|
227
250
|
end
|
@@ -48,10 +48,20 @@ module StrongMigrations
|
|
48
48
|
dir.up do
|
49
49
|
@migration.add_foreign_key(from_table, to_table, *args, **options.merge(validate: false))
|
50
50
|
disable_transaction
|
51
|
-
|
51
|
+
validate_options = options.slice(:column, :name)
|
52
|
+
if ActiveRecord::VERSION::MAJOR >= 6
|
53
|
+
@migration.validate_foreign_key(from_table, to_table, **validate_options)
|
54
|
+
else
|
55
|
+
@migration.validate_foreign_key(from_table, validate_options.any? ? validate_options : to_table)
|
56
|
+
end
|
52
57
|
end
|
53
58
|
dir.down do
|
54
|
-
|
59
|
+
remove_options = options.slice(:column, :name)
|
60
|
+
if ActiveRecord::VERSION::MAJOR >= 6
|
61
|
+
@migration.remove_foreign_key(from_table, to_table, **remove_options)
|
62
|
+
else
|
63
|
+
@migration.remove_foreign_key(from_table, remove_options.any? ? remove_options : to_table)
|
64
|
+
end
|
55
65
|
end
|
56
66
|
end
|
57
67
|
end
|
@@ -64,7 +74,7 @@ module StrongMigrations
|
|
64
74
|
@migration.validate_check_constraint(table, **validate_options)
|
65
75
|
end
|
66
76
|
dir.down do
|
67
|
-
@migration.remove_check_constraint(table, expression, **add_options)
|
77
|
+
@migration.remove_check_constraint(table, expression, **add_options.except(:validate))
|
68
78
|
end
|
69
79
|
end
|
70
80
|
end
|
@@ -0,0 +1,21 @@
|
|
1
|
+
module StrongMigrations
|
2
|
+
module SchemaDumper
|
3
|
+
def initialize(connection, *args, **options)
|
4
|
+
return super unless StrongMigrations.alphabetize_schema
|
5
|
+
|
6
|
+
super(WrappedConnection.new(connection), *args, **options)
|
7
|
+
end
|
8
|
+
end
|
9
|
+
|
10
|
+
class WrappedConnection
|
11
|
+
delegate_missing_to :@connection
|
12
|
+
|
13
|
+
def initialize(connection)
|
14
|
+
@connection = connection
|
15
|
+
end
|
16
|
+
|
17
|
+
def columns(*args, **options)
|
18
|
+
@connection.columns(*args, **options).sort_by(&:name)
|
19
|
+
end
|
20
|
+
end
|
21
|
+
end
|
data/lib/strong_migrations.rb
CHANGED
@@ -2,22 +2,22 @@
|
|
2
2
|
require "active_support"
|
3
3
|
|
4
4
|
# adapters
|
5
|
-
|
6
|
-
|
7
|
-
|
8
|
-
|
5
|
+
require_relative "strong_migrations/adapters/abstract_adapter"
|
6
|
+
require_relative "strong_migrations/adapters/mysql_adapter"
|
7
|
+
require_relative "strong_migrations/adapters/mariadb_adapter"
|
8
|
+
require_relative "strong_migrations/adapters/postgresql_adapter"
|
9
9
|
|
10
10
|
# modules
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
11
|
+
require_relative "strong_migrations/checks"
|
12
|
+
require_relative "strong_migrations/safe_methods"
|
13
|
+
require_relative "strong_migrations/checker"
|
14
|
+
require_relative "strong_migrations/database_tasks"
|
15
|
+
require_relative "strong_migrations/migration"
|
16
|
+
require_relative "strong_migrations/migrator"
|
17
|
+
require_relative "strong_migrations/version"
|
18
18
|
|
19
19
|
# integrations
|
20
|
-
|
20
|
+
require_relative "strong_migrations/railtie" if defined?(Rails)
|
21
21
|
|
22
22
|
module StrongMigrations
|
23
23
|
class Error < StandardError; end
|
@@ -28,7 +28,8 @@ module StrongMigrations
|
|
28
28
|
attr_accessor :auto_analyze, :start_after, :checks, :error_messages,
|
29
29
|
:target_postgresql_version, :target_mysql_version, :target_mariadb_version,
|
30
30
|
:enabled_checks, :lock_timeout, :statement_timeout, :check_down, :target_version,
|
31
|
-
:safe_by_default, :target_sql_mode, :lock_timeout_retries, :lock_timeout_retry_delay
|
31
|
+
:safe_by_default, :target_sql_mode, :lock_timeout_retries, :lock_timeout_retry_delay,
|
32
|
+
:alphabetize_schema
|
32
33
|
attr_writer :lock_timeout_limit
|
33
34
|
end
|
34
35
|
self.auto_analyze = false
|
@@ -38,10 +39,21 @@ module StrongMigrations
|
|
38
39
|
self.checks = []
|
39
40
|
self.safe_by_default = false
|
40
41
|
self.check_down = false
|
42
|
+
self.alphabetize_schema = false
|
41
43
|
|
42
44
|
# private
|
43
45
|
def self.developer_env?
|
44
|
-
|
46
|
+
env == "development" || env == "test"
|
47
|
+
end
|
48
|
+
|
49
|
+
# private
|
50
|
+
def self.env
|
51
|
+
if defined?(Rails.env)
|
52
|
+
Rails.env
|
53
|
+
else
|
54
|
+
# default to production for safety
|
55
|
+
ENV["RACK_ENV"] || "production"
|
56
|
+
end
|
45
57
|
end
|
46
58
|
|
47
59
|
def self.lock_timeout_limit
|
@@ -74,7 +86,7 @@ module StrongMigrations
|
|
74
86
|
end
|
75
87
|
|
76
88
|
# load error messages
|
77
|
-
|
89
|
+
require_relative "strong_migrations/error_messages"
|
78
90
|
|
79
91
|
ActiveSupport.on_load(:active_record) do
|
80
92
|
ActiveRecord::Migration.prepend(StrongMigrations::Migration)
|
@@ -83,4 +95,7 @@ ActiveSupport.on_load(:active_record) do
|
|
83
95
|
if defined?(ActiveRecord::Tasks::DatabaseTasks)
|
84
96
|
ActiveRecord::Tasks::DatabaseTasks.singleton_class.prepend(StrongMigrations::DatabaseTasks)
|
85
97
|
end
|
98
|
+
|
99
|
+
require_relative "strong_migrations/schema_dumper"
|
100
|
+
ActiveRecord::SchemaDumper.prepend(StrongMigrations::SchemaDumper)
|
86
101
|
end
|
@@ -1,14 +1,9 @@
|
|
1
1
|
namespace :strong_migrations do
|
2
|
-
# https://www.pgrs.net/2008/03/
|
2
|
+
# https://www.pgrs.net/2008/03/12/alphabetize-schema-rb-columns/
|
3
3
|
task :alphabetize_columns do
|
4
4
|
$stderr.puts "Dumping schema"
|
5
5
|
ActiveRecord::Base.logger.level = Logger::INFO
|
6
6
|
|
7
|
-
|
8
|
-
ActiveRecord::Base.connection.class.prepend StrongMigrations::AlphabetizeColumns
|
9
|
-
if ActiveRecord::ConnectionAdapters.const_defined?('PostGISAdapter')
|
10
|
-
ActiveRecord::ConnectionAdapters::PostGISAdapter.prepend StrongMigrations::AlphabetizeColumns
|
11
|
-
end
|
12
|
-
ActiveRecord::ConnectionAdapters::AbstractAdapter.prepend StrongMigrations::AlphabetizeColumns
|
7
|
+
StrongMigrations.alphabetize_schema = true
|
13
8
|
end
|
14
9
|
end
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: strong_migrations
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.6.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Andrew Kane
|
@@ -10,7 +10,7 @@ authors:
|
|
10
10
|
autorequire:
|
11
11
|
bindir: bin
|
12
12
|
cert_chain: []
|
13
|
-
date:
|
13
|
+
date: 2023-08-09 00:00:00.000000000 Z
|
14
14
|
dependencies:
|
15
15
|
- !ruby/object:Gem::Dependency
|
16
16
|
name: activerecord
|
@@ -45,7 +45,6 @@ files:
|
|
45
45
|
- lib/strong_migrations/adapters/mariadb_adapter.rb
|
46
46
|
- lib/strong_migrations/adapters/mysql_adapter.rb
|
47
47
|
- lib/strong_migrations/adapters/postgresql_adapter.rb
|
48
|
-
- lib/strong_migrations/alphabetize_columns.rb
|
49
48
|
- lib/strong_migrations/checker.rb
|
50
49
|
- lib/strong_migrations/checks.rb
|
51
50
|
- lib/strong_migrations/database_tasks.rb
|
@@ -54,6 +53,7 @@ files:
|
|
54
53
|
- lib/strong_migrations/migrator.rb
|
55
54
|
- lib/strong_migrations/railtie.rb
|
56
55
|
- lib/strong_migrations/safe_methods.rb
|
56
|
+
- lib/strong_migrations/schema_dumper.rb
|
57
57
|
- lib/strong_migrations/version.rb
|
58
58
|
- lib/tasks/strong_migrations.rake
|
59
59
|
homepage: https://github.com/ankane/strong_migrations
|
@@ -75,7 +75,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
75
75
|
- !ruby/object:Gem::Version
|
76
76
|
version: '0'
|
77
77
|
requirements: []
|
78
|
-
rubygems_version: 3.
|
78
|
+
rubygems_version: 3.4.10
|
79
79
|
signing_key:
|
80
80
|
specification_version: 4
|
81
81
|
summary: Catch unsafe migrations in development
|