safe-pg-migrations 2.0.0 → 2.1.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 878eca03499c2fa44ef8e046f5828b2b0978e692ade4440f9489a29889a99b49
4
- data.tar.gz: 88a9aa0adf58a3a716565d8a3482839ffd546570ab7238f67daecb0c2396a152
3
+ metadata.gz: 2fafd98448f0ff697da386a337dc22753be2144feedba4793f87a351581a5607
4
+ data.tar.gz: 9653ae6395d4c9f27b69dba9dee9142a4535384c42e8d05fac9ca2e2ef1bdce6
5
5
  SHA512:
6
- metadata.gz: 15cd514d22cf2b5f182faf16a50a4775fa528754357eeeaa0c81f2f93bf25d12375ae2c7742e8e762addd01ae3b5aeeedb09c758e9e6da063f4b0beca8544c45
7
- data.tar.gz: d930ac0e741025ea05743a2ad8f6da979932f67cd9e93e13c64b22581bb073b164212144d152f9a0729a93e49098cb9279caba57a91b746502ccb259cb18bbcb
6
+ metadata.gz: 64297c9ba4f040e7afa21fefa2acb2a26e27fb0bfee3d9f9cbf902ee4bb0ac087f4e22591a7b5470cf73dc7bc73345e59904649c02d7540bd4693372e7069549
7
+ data.tar.gz: 7a2640fd2494480d560cfa6f19059550ecc316eaf729df6d8f97a434c189f4565c49d47acbb02f437d9b1285a466a5615a79a84d6ba1dfb3d94bb54f87b9e788
data/README.md CHANGED
@@ -109,7 +109,43 @@ Beware though, when adding a volatile default value:
109
109
  ```ruby
110
110
  add_column :users, :created_at, default: 'clock_timestamp()'
111
111
  ```
112
- PG will still needs to update every row of the table, and will most likely statement timeout for big table. In this case, your best bet is to add the column without default, set the default, and backfill existing rows.
112
+ PG will still needs to update every row of the table, and will most likely statement timeout for big table. In this case, **Safe PG Migrations** can automatically backfill data when the option `default_value_backfill:` is set to `:update_in_batches`.
113
+
114
+ </details>
115
+
116
+ <details>
117
+ <summary>Safe add_column - adding a volatile default value</summary>
118
+
119
+ **Safe PG Migrations** provides the extra option parameter `default_value_backfill:`. When your migration is adding a volatile default value, the option `:update_in_batches` can be set. It will automatically backfill the value in a safe manner.
120
+
121
+ ```ruby
122
+ add_column :users, :created_at, default: 'clock_timestamp()', default_value_backfill: :update_in_batches
123
+ ```
124
+
125
+ More specifically, it will:
126
+
127
+ 1. create the column without default value and without null constraint. This ensure the `ACCESS EXCLUSIVE` lock is acquired for the least amount of time;
128
+ 2. add the default value, without data backfill. An `ACCESS EXCLUSIVE` lock is acquired and released immediately;
129
+ 3. backfill data, in batch of `SafePgMigrations.config.backfill_batch_size` and with a pause of `SafePgMigrations.config.backfill_pause` between each batch;
130
+ 4. change the column to `null: false`, if defined in the parameters, following the algorithm we have defined below.
131
+
132
+ ---
133
+ **NOTE**
134
+
135
+ Data backfill take time. If your table is big, your migrations will (safely) hangs for a while. You might want to backfill data manually instead, to do so you will need two migrations
136
+
137
+ 1. First migration :
138
+
139
+ a. adds the column without default and without null constraint;
140
+
141
+ b. add the default value.
142
+
143
+ 2. manual data backfill (rake task, manual operation, ...)
144
+ 3. Second migration which change the column to null false (with **Safe PG Migrations**, `change_column_null` is safe and can be used; see section below)
145
+
146
+ ---
147
+
148
+ `default_value_backfill:` also accept the value `:auto` which is set by default. In this case, **Safe PG Migrations** will not backfill data and will let PostgreSQL handle it itself.
113
149
 
114
150
  </details>
115
151
 
@@ -130,13 +166,70 @@ If you still get lock timeout while adding / removing indexes, it might be for o
130
166
 
131
167
  Adding a foreign key requires a `SHARE ROW EXCLUSIVE` lock, which **prevent writing in the tables** while the migration is running.
132
168
 
133
- Adding the constraint itself is rather fast, the major part of the time is spent on validating this constraint. Thus safe-pg-migrations ensures that adding a foreign key holds blocking locks for the least amount of time by splitting the foreign key creation in two steps:
169
+ Adding the constraint itself is rather fast, the major part of the time is spent on validating this constraint. Thus **Safe PG Migrations** ensures that adding a foreign key holds blocking locks for the least amount of time by splitting the foreign key creation in two steps:
170
+
171
+ 1. adding the constraint *without validation*, will not validate existing rows;
172
+ 2. validating the constraint, will validate existing rows in the table, without blocking read or write on the table
173
+
174
+ </details>
175
+
176
+
177
+ <details><summary id="safe_add_check_constraint">Safe <code>add_check_constraint</code> (ActiveRecord > 6.1)</summary>
178
+
179
+ Adding a check constraint requires an `ACCESS EXCLUSIVE` lock, which **prevent writing and reading in the tables** [as soon as the lock is requested](https://medium.com/doctolib/stop-worrying-about-postgresql-locks-in-your-rails-migrations-3426027e9cc9).
180
+
181
+ Adding the constraint itself is rather fast, the major part of the time is spent on validating this constraint.
182
+ Thus **Safe PG Migrations** ensures that adding a constraints holds blocking locks for the least amount of time by
183
+ splitting the constraint addition in two steps:
134
184
 
135
185
  1. adding the constraint *without validation*, will not validate existing rows;
136
186
  2. validating the constraint, will validate existing rows in the table, without blocking read or write on the table
137
187
 
138
188
  </details>
139
189
 
190
+ <details><summary id="safe_change_column_null">Safe <code>change_column_null</code> (ActiveRecord and PG version dependant)</summary>
191
+
192
+ Changing the nullability of a column requires an `ACCESS EXCLUSIVE` lock, which **prevent writing and reading in the tables** [as soon as the lock is requested](https://medium.com/doctolib/stop-worrying-about-postgresql-locks-in-your-rails-migrations-3426027e9cc9).
193
+
194
+ Adding the constraint itself is rather fast, the major part of the time is spent on validating this constraint.
195
+
196
+ **Safe PG Migrations** acts differently depending on the version you are on.
197
+
198
+ ### Recent versions of PG and Active Record (> 12 and > 6.1)
199
+
200
+ Starting on PostgreSQL versions 12, adding the column NOT NULL constraint is safe if a check constraint validates the
201
+ nullability of the same column. **Safe PG Migrations** also relies on add_check_constraint, which was introduced in
202
+ ActiveRecord 6.1.
203
+
204
+ If these requirements are met, **Safe PG Migrations** ensures that adding a constraints holds blocking locks for the least
205
+ amount of time by splitting the constraint addition in several steps:
206
+
207
+ 1. adding a `IS NOT NULL` constraint *without validation*, will not validate existing rows but block read or write;
208
+ 2. validating the constraint, will validate existing rows in the table, without blocking read or write on the table;
209
+ 3. changing the not null status of the column, thanks to the NOT NULL constraint without having to scan the table sequentially;
210
+ 4. dropping the `IS NOT NULL` constraint.
211
+
212
+ ### Older versions of PG or ActiveRecord
213
+
214
+ If the version of PostgreSQL is below 12, or if the version of ActiveRecord is below 6.1, **Safe PG Migrations** will only
215
+ wrap ActiveRecord method into a statement timeout and lock timeout.
216
+
217
+ ### Call with a default parameter
218
+
219
+ Calling change_column_null with a default parameter [is dangerous](https://github.com/rails/rails/blob/716baea69f989b64f5bfeaff880c2512377bebab/activerecord/lib/active_record/connection_adapters/postgresql/schema_statements.rb#L446)
220
+ and is likely not to finish in the statement timeout defined by **Safe PG Migrations**. For this reason, when the default
221
+ parameter is given, **Safe PG Migrations** will simply forward it to activerecord methods without trying to improve it
222
+
223
+ ### Dropping a NULL constraint
224
+
225
+ Dropping a null constraint still requires an `ACCESS EXCLUSIVE` lock, but does not require extra operation to reduce the
226
+ amount of time during which the lock is held. **Safe PG Migrations** only wrap methods of activerecord in lock and statement
227
+ timeouts
228
+
229
+ </details>
230
+
231
+
232
+
140
233
  <details><summary>Retry after lock timeout</summary>
141
234
 
142
235
  When a statement fails with a lock timeout, **Safe PG Migrations** retries it (5 times max) [list of retriable statements](https://github.com/doctolib/safe-pg-migrations/blob/66933256252b6bbf12e404b829a361dbba30e684/lib/safe-pg-migrations/plugins/statement_retrier.rb#L5)
@@ -201,7 +294,9 @@ SafePgMigrations.config.blocking_activity_logger_verbose = true # Outputs the ra
201
294
 
202
295
  SafePgMigrations.config.blocking_activity_logger_margin = 1.second # Delay to output blocking queries before timeout. Must be shorter than safe_timeout
203
296
 
204
- SafePgMigrations.config.batch_size = 1000 # Size of the batches used for backfilling when adding a column with a default value pre-PG11
297
+ SafePgMigrations.config.backfill_batch_size = 100_000 # Size of the batches used for backfilling when adding a column with a default value
298
+
299
+ SafePgMigrations.config.backfill_pause = 0.5.second # Delay between each batch during a backfill. This ensure replication can happen safely.
205
300
 
206
301
  SafePgMigrations.config.retry_delay = 1.minute # Delay between retries for retryable statements
207
302
 
@@ -1,13 +1,16 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require 'safe-pg-migrations/configuration'
4
+ require 'safe-pg-migrations/helpers/satisfied_helper'
5
+ require 'safe-pg-migrations/helpers/index_helper'
6
+ require 'safe-pg-migrations/helpers/batch_over'
4
7
  require 'safe-pg-migrations/plugins/verbose_sql_logger'
5
8
  require 'safe-pg-migrations/plugins/blocking_activity_logger'
9
+ require 'safe-pg-migrations/plugins/statement_insurer/add_column'
6
10
  require 'safe-pg-migrations/plugins/statement_insurer'
7
11
  require 'safe-pg-migrations/plugins/statement_retrier'
8
12
  require 'safe-pg-migrations/plugins/idempotent_statements'
9
13
  require 'safe-pg-migrations/plugins/useless_statements_logger'
10
- require 'safe-pg-migrations/polyfills/satisfied_helper'
11
14
  require 'safe-pg-migrations/polyfills/index_definition_polyfill'
12
15
  require 'safe-pg-migrations/polyfills/verbose_query_logs_polyfill'
13
16
 
@@ -23,15 +26,16 @@ module SafePgMigrations
23
26
  ].freeze
24
27
 
25
28
  class << self
26
- attr_reader :current_migration
29
+ attr_reader :current_migration, :pg_version_num
27
30
 
28
31
  def setup_and_teardown(migration, connection, &block)
32
+ @pg_version_num = get_pg_version_num(connection)
29
33
  @alternate_connection = nil
30
34
  @current_migration = migration
31
35
  stdout_sql_logger = VerboseSqlLogger.new.setup if verbose?
32
36
  PLUGINS.each { |plugin| connection.extend(plugin) }
33
37
 
34
- connection.with_setting(:lock_timeout, SafePgMigrations.config.pg_safe_timeout, &block)
38
+ connection.with_setting :lock_timeout, SafePgMigrations.config.pg_lock_timeout, &block
35
39
  ensure
36
40
  close_alternate_connection
37
41
  @current_migration = nil
@@ -72,6 +76,10 @@ module SafePgMigrations
72
76
  def config
73
77
  @config ||= Configuration.new
74
78
  end
79
+
80
+ def get_pg_version_num(connection)
81
+ connection.query_value('SHOW server_version_num').to_i
82
+ end
75
83
  end
76
84
 
77
85
  module Migration
@@ -102,6 +110,7 @@ module SafePgMigrations
102
110
  add_belongs_to
103
111
  change_column_null
104
112
  add_foreign_key
113
+ add_check_constraint
105
114
  ].freeze
106
115
 
107
116
  SAFE_METHODS.each do |method|
@@ -4,22 +4,32 @@ require 'active_support/core_ext/numeric/time'
4
4
 
5
5
  module SafePgMigrations
6
6
  class Configuration
7
- attr_accessor :safe_timeout, :blocking_activity_logger_margin, :blocking_activity_logger_verbose, :batch_size,
8
- :retry_delay, :max_tries
7
+ attr_accessor :safe_timeout, :blocking_activity_logger_margin, :blocking_activity_logger_verbose,
8
+ :backfill_batch_size, :backfill_pause, :retry_delay, :max_tries
9
9
 
10
10
  def initialize
11
11
  self.safe_timeout = 5.seconds
12
12
  self.blocking_activity_logger_margin = 1.second
13
13
  self.blocking_activity_logger_verbose = true
14
- self.batch_size = 1000
14
+ self.backfill_batch_size = 100_000
15
+ self.backfill_pause = 0.5.second
15
16
  self.retry_delay = 1.minute
16
17
  self.max_tries = 5
17
18
  end
18
19
 
19
- def pg_safe_timeout
20
- pg_duration(safe_timeout)
20
+ def pg_statement_timeout
21
+ pg_duration safe_timeout
21
22
  end
22
23
 
24
+ def pg_lock_timeout
25
+ # if statement timeout and lock timeout have the same value, statement timeout will raise in priority. We actually
26
+ # need the opposite for BlockingActivityLogger to detect lock timeouts correctly.
27
+ # By reducing the lock timeout by a very small margin, we ensure that the lock timeout is raised in priority
28
+ pg_duration safe_timeout * 0.99
29
+ end
30
+
31
+ private
32
+
23
33
  def pg_duration(duration)
24
34
  value, unit = duration.integer? ? [duration, 's'] : [(duration * 1000).to_i, 'ms']
25
35
  "#{value}#{unit}"
@@ -0,0 +1,56 @@
1
+ # frozen_string_literal: true
2
+
3
+ module SafePgMigrations
4
+ module Helpers
5
+ class BatchOver
6
+ def initialize(model, of: SafePgMigrations.config.backfill_batch_size)
7
+ @model = model
8
+ @of = of
9
+
10
+ @current_range = nil
11
+ end
12
+
13
+ def each_batch
14
+ yield scope.where(primary_key => @current_range) while next_batch
15
+ end
16
+
17
+ private
18
+
19
+ def next_batch
20
+ return if endless?
21
+
22
+ first = next_scope.take
23
+
24
+ return unless first
25
+
26
+ last = next_scope.offset(@of).take
27
+
28
+ first_key = first[primary_key]
29
+ last_key = last.nil? ? nil : last[primary_key]
30
+
31
+ @current_range = first_key...last_key
32
+ end
33
+
34
+ def next_scope
35
+ return scope if @current_range.nil?
36
+ return scope.none if endless?
37
+
38
+ scope.where(primary_key => @current_range.end..)
39
+ end
40
+
41
+ def scope
42
+ @model.order(primary_key => :asc)
43
+ end
44
+
45
+ def endless?
46
+ return false if @current_range.nil?
47
+
48
+ @current_range.end.nil?
49
+ end
50
+
51
+ def primary_key
52
+ @model.primary_key
53
+ end
54
+ end
55
+ end
56
+ end
@@ -0,0 +1,24 @@
1
+ # frozen_string_literal: true
2
+
3
+ module SafePgMigrations
4
+ module Helpers
5
+ module IndexHelper
6
+ def index_definition(table_name, column_name, **options)
7
+ index_definition, = add_index_options(table_name, column_name, **options)
8
+ index_definition
9
+ end
10
+
11
+ private
12
+
13
+ def index_valid?(index_name)
14
+ query_value <<~SQL.squish
15
+ SELECT indisvalid
16
+ FROM pg_index i
17
+ JOIN pg_class c
18
+ ON i.indexrelid = c.oid
19
+ WHERE c.relname = '#{index_name}';
20
+ SQL
21
+ end
22
+ end
23
+ end
24
+ end
@@ -0,0 +1,31 @@
1
+ # frozen_string_literal: true
2
+
3
+ module SafePgMigrations
4
+ module Helpers
5
+ module SatisfiedHelper
6
+ class << self
7
+ def satisfies_change_column_null_requirements?
8
+ satisfies_add_check_constraints? && SafePgMigrations.pg_version_num >= 120_000
9
+ end
10
+
11
+ def satisfies_add_check_constraints!
12
+ return if satisfies_add_check_constraints?
13
+
14
+ raise NotImplementedError, 'add_check_constraint is not supported in your ActiveRecord version'
15
+ end
16
+
17
+ def satisfies_add_check_constraints?
18
+ satisfied? '>=6.1.0'
19
+ end
20
+
21
+ def satisfies_add_column_update_rows_backfill?
22
+ satisfies_change_column_null_requirements?
23
+ end
24
+
25
+ def satisfied?(version)
26
+ Gem::Requirement.new(version).satisfied_by? Gem::Version.new(::ActiveRecord::VERSION::STRING)
27
+ end
28
+ end
29
+ end
30
+ end
31
+ end
@@ -2,6 +2,8 @@
2
2
 
3
3
  module SafePgMigrations
4
4
  module IdempotentStatements
5
+ include SafePgMigrations::Helpers::IndexHelper
6
+
5
7
  ruby2_keywords def add_index(table_name, column_name, *args)
6
8
  options = args.last.is_a?(Hash) ? args.last : {}
7
9
 
@@ -80,23 +82,51 @@ module SafePgMigrations
80
82
  end
81
83
  end
82
84
 
83
- protected
85
+ def add_check_constraint(table_name, expression, **options)
86
+ constraint_definition = check_constraint_for table_name,
87
+ **check_constraint_options(table_name, expression, options)
88
+
89
+ return super if constraint_definition.nil?
90
+
91
+ SafePgMigrations.say "/!\\ Constraint '#{constraint_definition.name}' already exists. Skipping statement.", true
92
+ end
93
+
94
+ def change_column_null(table_name, column_name, null, *)
95
+ column = column_for(table_name, column_name)
84
96
 
85
- def index_definition(table_name, column_name, **options)
86
- index_definition, = add_index_options(table_name, column_name, **options)
87
- index_definition
97
+ return super if column.null != null
98
+
99
+ SafePgMigrations.say(
100
+ "/!\\ Column '#{table_name}.#{column.name}' is already set to 'null: #{null}'. Skipping statement.",
101
+ true
102
+ )
88
103
  end
89
104
 
90
- private
105
+ def validate_check_constraint(table_name, **options)
106
+ constraint_definition = check_constraint_for!(table_name, **options)
107
+
108
+ return super unless constraint_definition.validated?
91
109
 
92
- def index_valid?(index_name)
93
- query_value <<~SQL.squish
94
- SELECT indisvalid
95
- FROM pg_index i
96
- JOIN pg_class c
97
- ON i.indexrelid = c.oid
98
- WHERE c.relname = '#{index_name}';
99
- SQL
110
+ SafePgMigrations.say "/!\\ Constraint '#{constraint_definition.name}' already validated. Skipping statement.",
111
+ true
112
+ end
113
+
114
+ def change_column_default(table_name, column_name, default_or_changes)
115
+ column = column_for(table_name, column_name)
116
+
117
+ previous_alter_statement = change_column_default_for_alter(table_name, column_name, column.default)
118
+ new_alter_statement = change_column_default_for_alter(table_name, column_name, default_or_changes)
119
+
120
+ # NOTE: PG change_column_default is already idempotent.
121
+ # We try to detect it because it still takes an ACCESS EXCLUSIVE lock
122
+
123
+ return super if new_alter_statement != previous_alter_statement
124
+
125
+ SafePgMigrations.say(
126
+ "/!\\ Column '#{table_name}.#{column.name}' is already set to 'default: #{column.default}'. " \
127
+ 'Skipping statement.',
128
+ true
129
+ )
100
130
  end
101
131
  end
102
132
  end
@@ -0,0 +1,52 @@
1
+ # frozen_string_literal: true
2
+
3
+ module SafePgMigrations
4
+ module StatementInsurer
5
+ module AddColumn
6
+ ruby2_keywords def add_column(table_name, column_name, type, *args)
7
+ options = args.last.is_a?(Hash) && args.last
8
+ options ||= {}
9
+
10
+ if should_keep_default_implementation?(**options)
11
+ with_setting(:statement_timeout, SafePgMigrations.config.pg_statement_timeout) { return super }
12
+ end
13
+
14
+ default = options.delete(:default)
15
+ null = options.delete(:null)
16
+
17
+ with_setting(:statement_timeout, SafePgMigrations.config.pg_statement_timeout) do
18
+ SafePgMigrations.say_method_call(:add_column, table_name, column_name, type, options)
19
+ super table_name, column_name, type, **options
20
+ end
21
+
22
+ SafePgMigrations.say_method_call(:change_column_default, table_name, column_name, default)
23
+ change_column_default(table_name, column_name, default)
24
+
25
+ SafePgMigrations.say_method_call(:backfill_column_default, table_name, column_name)
26
+ without_statement_timeout do
27
+ backfill_column_default(table_name, column_name)
28
+ end
29
+
30
+ change_column_null(table_name, column_name, null) if null == false
31
+ end
32
+
33
+ private
34
+
35
+ def should_keep_default_implementation?(default: nil, default_value_backfill: :auto, **)
36
+ default_value_backfill != :update_in_batches || !default ||
37
+ !SafePgMigrations::Helpers::SatisfiedHelper.satisfies_add_column_update_rows_backfill?
38
+ end
39
+
40
+ def backfill_column_default(table_name, column_name)
41
+ model = Class.new(ActiveRecord::Base) { self.table_name = table_name }
42
+ quoted_column_name = quote_column_name(column_name)
43
+
44
+ SafePgMigrations::Helpers::BatchOver.new(model).each_batch do |batch|
45
+ batch
46
+ .update_all("#{quoted_column_name} = DEFAULT")
47
+ sleep SafePgMigrations.config.backfill_pause
48
+ end
49
+ end
50
+ end
51
+ end
52
+ end
@@ -2,18 +2,41 @@
2
2
 
3
3
  module SafePgMigrations
4
4
  module StatementInsurer
5
- %i[change_column_null change_column].each do |method|
5
+ include AddColumn
6
+
7
+ %i[change_column].each do |method|
6
8
  define_method method do |*args, &block|
7
- with_setting(:statement_timeout, SafePgMigrations.config.pg_safe_timeout) { super(*args, &block) }
9
+ with_setting(:statement_timeout, SafePgMigrations.config.pg_statement_timeout) { super(*args, &block) }
8
10
  end
9
11
  ruby2_keywords method
10
12
  end
11
13
 
14
+ def validate_check_constraint(table_name, **options)
15
+ without_statement_timeout do
16
+ super
17
+ end
18
+ end
19
+
20
+ def add_check_constraint(table_name, expression, **options)
21
+ Helpers::SatisfiedHelper.satisfies_add_check_constraints!
22
+ return unless supports_check_constraints?
23
+
24
+ options = check_constraint_options(table_name, expression, options)
25
+
26
+ SafePgMigrations.say_method_call :add_check_constraint, table_name, expression, **options, validate: false
27
+ super table_name, expression, **options, validate: false
28
+
29
+ return unless options.fetch(:validate, true)
30
+
31
+ SafePgMigrations.say_method_call :validate_check_constraint, table_name, name: options[:name]
32
+ validate_check_constraint table_name, name: options[:name]
33
+ end
34
+
12
35
  ruby2_keywords def add_foreign_key(from_table, to_table, *args)
13
36
  options = args.last.is_a?(Hash) ? args.last : {}
14
- validate_present = options.key? :validate
37
+ validate_present = options.key?(:validate)
15
38
  options[:validate] = false unless validate_present
16
- with_setting(:statement_timeout, SafePgMigrations.config.pg_safe_timeout) do
39
+ with_setting(:statement_timeout, SafePgMigrations.config.pg_statement_timeout) do
17
40
  super(from_table, to_table, **options)
18
41
  end
19
42
 
@@ -24,7 +47,7 @@ module SafePgMigrations
24
47
  end
25
48
 
26
49
  ruby2_keywords def create_table(*)
27
- with_setting(:statement_timeout, SafePgMigrations.config.pg_safe_timeout) do
50
+ with_setting(:statement_timeout, SafePgMigrations.config.pg_statement_timeout) do
28
51
  super do |td|
29
52
  yield td if block_given?
30
53
  td.indexes.map! do |key, index_options|
@@ -57,6 +80,22 @@ module SafePgMigrations
57
80
  without_timeout { super(table_name, **options) }
58
81
  end
59
82
 
83
+ def change_column_null(table_name, column_name, null, default = nil)
84
+ if default || null || !Helpers::SatisfiedHelper.satisfies_change_column_null_requirements?
85
+ with_setting(:statement_timeout, SafePgMigrations.config.pg_statement_timeout) { return super }
86
+ end
87
+
88
+ add_check_constraint table_name, "#{column_name} IS NOT NULL"
89
+
90
+ SafePgMigrations.say_method_call :change_column_null, table_name, column_name, false
91
+ with_setting(:statement_timeout, SafePgMigrations.config.pg_statement_timeout) do
92
+ super table_name, column_name, false
93
+ end
94
+
95
+ SafePgMigrations.say_method_call :remove_check_constraint, table_name, "#{column_name} IS NOT NULL"
96
+ remove_check_constraint table_name, "#{column_name} IS NOT NULL"
97
+ end
98
+
60
99
  def with_setting(key, value)
61
100
  old_value = query_value("SHOW #{key}")
62
101
  execute("SET #{key} TO #{quote(value)}")
@@ -3,7 +3,7 @@
3
3
  module SafePgMigrations
4
4
  module StatementRetrier
5
5
  RETRIABLE_SCHEMA_STATEMENTS = %i[
6
- add_column add_foreign_key remove_foreign_key change_column_default change_column_null remove_column
6
+ add_column add_foreign_key remove_foreign_key change_column_default change_column_null remove_column drop_table
7
7
  ].freeze
8
8
 
9
9
  RETRIABLE_SCHEMA_STATEMENTS.each do |method|
@@ -29,6 +29,14 @@ module SafePgMigrations
29
29
  super
30
30
  end
31
31
 
32
+ ruby2_keywords def add_check_constraint(*args)
33
+ options = args.last.is_a?(Hash) ? args.last : {}
34
+ if options[:validate] == false
35
+ UselessStatementsLogger.warn_useless '`validate: :false`', 'https://github.com/doctolib/safe-pg-migrations#safe_add_check_constraint'
36
+ end
37
+ super
38
+ end
39
+
32
40
  def warn_for_index(**options)
33
41
  return unless options[:algorithm] == :concurrently
34
42
 
@@ -3,14 +3,14 @@
3
3
  module SafePgMigrations
4
4
  module Polyfills
5
5
  module IndexDefinitionPolyfill
6
- include SatisfiedHelper
6
+ include Helpers::SatisfiedHelper
7
7
 
8
8
  protected
9
9
 
10
10
  IndexDefinition = Struct.new(:table, :name)
11
11
 
12
12
  def index_definition(table_name, column_name, **options)
13
- return super(table_name, column_name, **options) if satisfied? '>=6.1.0'
13
+ return super(table_name, column_name, **options) if Helpers::SatisfiedHelper.satisfied? '>=6.1.0'
14
14
 
15
15
  index_name = options.key?(:name) ? options[:name].to_s : index_name(table_name, index_column_names(column_name))
16
16
  validate_index_length!(table_name, index_name, options.fetch(:internal, false))
@@ -4,16 +4,14 @@ module SafePgMigrations
4
4
  module Polyfills
5
5
  module VerboseQueryLogsPolyfill
6
6
  class << self
7
- include SatisfiedHelper
8
-
9
7
  def verbose_query_logs
10
- return ActiveRecord.verbose_query_logs if satisfied? '>=7.0.0'
8
+ return ActiveRecord.verbose_query_logs if Helpers::SatisfiedHelper.satisfied? '>=7.0.0'
11
9
 
12
10
  ActiveRecord::Base.verbose_query_logs
13
11
  end
14
12
 
15
13
  def verbose_query_logs=(value)
16
- if satisfied? '>=7.0.0'
14
+ if Helpers::SatisfiedHelper.satisfied? '>=7.0.0'
17
15
  ActiveRecord.verbose_query_logs = value
18
16
  return
19
17
  end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module SafePgMigrations
4
- VERSION = '2.0.0'
4
+ VERSION = '2.1.0'
5
5
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: safe-pg-migrations
3
3
  version: !ruby/object:Gem::Version
4
- version: 2.0.0
4
+ version: 2.1.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Matthieu Prat
@@ -10,7 +10,7 @@ authors:
10
10
  autorequire:
11
11
  bindir: bin
12
12
  cert_chain: []
13
- date: 2023-03-17 00:00:00.000000000 Z
13
+ date: 2023-06-29 00:00:00.000000000 Z
14
14
  dependencies:
15
15
  - !ruby/object:Gem::Dependency
16
16
  name: activerecord
@@ -51,16 +51,19 @@ files:
51
51
  - lib/safe-pg-migrations.rb
52
52
  - lib/safe-pg-migrations/base.rb
53
53
  - lib/safe-pg-migrations/configuration.rb
54
+ - lib/safe-pg-migrations/helpers/batch_over.rb
54
55
  - lib/safe-pg-migrations/helpers/blocking_activity_formatter.rb
55
56
  - lib/safe-pg-migrations/helpers/blocking_activity_selector.rb
57
+ - lib/safe-pg-migrations/helpers/index_helper.rb
58
+ - lib/safe-pg-migrations/helpers/satisfied_helper.rb
56
59
  - lib/safe-pg-migrations/plugins/blocking_activity_logger.rb
57
60
  - lib/safe-pg-migrations/plugins/idempotent_statements.rb
58
61
  - lib/safe-pg-migrations/plugins/statement_insurer.rb
62
+ - lib/safe-pg-migrations/plugins/statement_insurer/add_column.rb
59
63
  - lib/safe-pg-migrations/plugins/statement_retrier.rb
60
64
  - lib/safe-pg-migrations/plugins/useless_statements_logger.rb
61
65
  - lib/safe-pg-migrations/plugins/verbose_sql_logger.rb
62
66
  - lib/safe-pg-migrations/polyfills/index_definition_polyfill.rb
63
- - lib/safe-pg-migrations/polyfills/satisfied_helper.rb
64
67
  - lib/safe-pg-migrations/polyfills/verbose_query_logs_polyfill.rb
65
68
  - lib/safe-pg-migrations/railtie.rb
66
69
  - lib/safe-pg-migrations/version.rb
@@ -1,13 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- module SafePgMigrations
4
- module Polyfills
5
- module SatisfiedHelper
6
- private
7
-
8
- def satisfied?(version)
9
- Gem::Requirement.new(version).satisfied_by? Gem::Version.new(::ActiveRecord::VERSION::STRING)
10
- end
11
- end
12
- end
13
- end