safe-pg-migrations 1.1.0 → 1.3.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +8 -5
- data/lib/safe-pg-migrations/base.rb +7 -2
- data/lib/safe-pg-migrations/configuration.rb +2 -6
- data/lib/safe-pg-migrations/plugins/blocking_activity_logger.rb +66 -36
- data/lib/safe-pg-migrations/plugins/idem_potent_statements.rb +24 -13
- data/lib/safe-pg-migrations/plugins/statement_insurer.rb +47 -24
- data/lib/safe-pg-migrations/plugins/statement_retrier.rb +2 -2
- data/lib/safe-pg-migrations/plugins/useless_statements_logger.rb +38 -0
- data/lib/safe-pg-migrations/version.rb +1 -1
- metadata +20 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: fdee6e752da3fbdec93e5fd5c72fb370497ddd66f1cc27f33609c2a31e364c53
|
4
|
+
data.tar.gz: 143a5dcbf614abed765fac78b16969d900ded5ec574f11d939d802b94d7e2586
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 41c264aa05c7b8ad5fe90f7ecdc88fa700504562e69695d5fd9f4dc9ac63315fb666818554e1387251b367aed241fccff31154589f70309905e1ed1cfae90c68
|
7
|
+
data.tar.gz: 9386e6a8b0366387543998ec5cec42e719d2a4138d5eb52f13f6580d2a3d31e48bd438760b358beb1ab5eb32bd8598927d69bafd03756108a514e88eeb431022
|
data/README.md
CHANGED
@@ -16,6 +16,8 @@ Just drop this line in your Gemfile:
|
|
16
16
|
gem 'safe-pg-migrations'
|
17
17
|
```
|
18
18
|
|
19
|
+
**Note: Do not run migrations via PgBouncer connection if it is configured to use transactional or statement pooling modes. You must run migrations via a direct Postgres connection, or configure PgBouncer to use session pooling mode.**
|
20
|
+
|
19
21
|
## Example
|
20
22
|
|
21
23
|
Consider the following migration:
|
@@ -61,7 +63,7 @@ Under the hood, **Safe PG Migrations** patches `ActiveRecord::Migration` and ext
|
|
61
63
|
|
62
64
|
## Motivation
|
63
65
|
|
64
|
-
Writing a safe migration can be daunting. Numerous articles have been written on the topic and a few gems are trying to address the problem. Even for someone who has a pretty good command of Postgres, remembering all the subtleties of explicit locking is not a piece of cake.
|
66
|
+
Writing a safe migration can be daunting. Numerous articles, [including ours](https://medium.com/doctolib/stop-worrying-about-postgresql-locks-in-your-rails-migrations-3426027e9cc9), have been written on the topic and a few gems are trying to address the problem. Even for someone who has a pretty good command of Postgres, remembering all the subtleties of explicit locking is not a piece of cake.
|
65
67
|
|
66
68
|
Active Record means developers don't have to be proficient in SQL to interact with a database. In the same way, **Safe PG Migrations** was created so that developers don't have to understand the ins and outs of Postgres to write a safe migration.
|
67
69
|
|
@@ -122,7 +124,7 @@ Note: the addition of the not null constraint may timeout. In that case, you may
|
|
122
124
|
|
123
125
|
</details>
|
124
126
|
|
125
|
-
<details><summary>Safe <code>add_index</code> and <code>remove_index</code></summary>
|
127
|
+
<details><summary id="safe_add_remove_index">Safe <code>add_index</code> and <code>remove_index</code></summary>
|
126
128
|
|
127
129
|
Creating an index requires a `SHARE` lock on the target table which blocks all write on the table while the index is created (which can take some time on a large table). This is usually not practical in a live environment. Thus, **Safe PG Migrations** ensures indexes are created concurrently.
|
128
130
|
|
@@ -135,7 +137,7 @@ If you still get lock timeout while adding / removing indexes, it might be for o
|
|
135
137
|
|
136
138
|
</details>
|
137
139
|
|
138
|
-
<details><summary>safe <code>add_foreign_key</code> (and <code>add_reference</code>)</summary>
|
140
|
+
<details><summary id="safe_add_foreign_key">safe <code>add_foreign_key</code> (and <code>add_reference</code>)</summary>
|
139
141
|
|
140
142
|
Adding a foreign key requires a `SHARE ROW EXCLUSIVE` lock, which **prevent writing in the tables** while the migration is running.
|
141
143
|
|
@@ -206,9 +208,9 @@ So you can actually check that the `CREATE INDEX` statement will be performed co
|
|
206
208
|
```ruby
|
207
209
|
SafePgMigrations.config.safe_timeout = 5.seconds # Lock and statement timeout used for all DDL operations except from CREATE / DROP INDEX
|
208
210
|
|
209
|
-
SafePgMigrations.config.
|
211
|
+
SafePgMigrations.config.blocking_activity_logger_verbose = true # Outputs the raw blocking queries on timeout. When false, outputs information about the lock instead
|
210
212
|
|
211
|
-
SafePgMigrations.config.blocking_activity_logger_margin = 1.second # Delay to output blocking queries before timeout. Must be
|
213
|
+
SafePgMigrations.config.blocking_activity_logger_margin = 1.second # Delay to output blocking queries before timeout. Must be shorter than safe_timeout
|
212
214
|
|
213
215
|
SafePgMigrations.config.batch_size = 1000 # Size of the batches used for backfilling when adding a column with a default value pre-PG11
|
214
216
|
|
@@ -254,3 +256,4 @@ Interesting reads:
|
|
254
256
|
- [Rails migrations with no downtime](https://pedro.herokuapp.com/past/2011/7/13/rails_migrations_with_no_downtime/)
|
255
257
|
- [Safe Operations For High Volume PostgreSQL](https://www.braintreepayments.com/blog/safe-operations-for-high-volume-postgresql/)
|
256
258
|
- [Rails Migrations with Zero Downtime](https://blog.codeship.com/rails-migrations-zero-downtime/)
|
259
|
+
- [Stop worrying about PostgreSQL locks in your Rails migrations](https://medium.com/doctolib/stop-worrying-about-postgresql-locks-in-your-rails-migrations-3426027e9cc9)
|
@@ -1,11 +1,13 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
|
+
require 'ruby2_keywords'
|
3
4
|
require 'safe-pg-migrations/configuration'
|
4
5
|
require 'safe-pg-migrations/plugins/verbose_sql_logger'
|
5
6
|
require 'safe-pg-migrations/plugins/blocking_activity_logger'
|
6
7
|
require 'safe-pg-migrations/plugins/statement_insurer'
|
7
8
|
require 'safe-pg-migrations/plugins/statement_retrier'
|
8
9
|
require 'safe-pg-migrations/plugins/idem_potent_statements'
|
10
|
+
require 'safe-pg-migrations/plugins/useless_statements_logger'
|
9
11
|
|
10
12
|
module SafePgMigrations
|
11
13
|
# Order matters: the bottom-most plugin will have precedence
|
@@ -14,6 +16,7 @@ module SafePgMigrations
|
|
14
16
|
IdemPotentStatements,
|
15
17
|
StatementRetrier,
|
16
18
|
StatementInsurer,
|
19
|
+
UselessStatementsLogger,
|
17
20
|
].freeze
|
18
21
|
|
19
22
|
class << self
|
@@ -48,13 +51,13 @@ module SafePgMigrations
|
|
48
51
|
@alternate_connection = nil
|
49
52
|
end
|
50
53
|
|
51
|
-
def say(*args)
|
54
|
+
ruby2_keywords def say(*args)
|
52
55
|
return unless current_migration
|
53
56
|
|
54
57
|
current_migration.say(*args)
|
55
58
|
end
|
56
59
|
|
57
|
-
def say_method_call(method, *args)
|
60
|
+
ruby2_keywords def say_method_call(method, *args)
|
58
61
|
say "#{method}(#{args.map(&:inspect) * ', '})", true
|
59
62
|
end
|
60
63
|
|
@@ -78,6 +81,7 @@ module SafePgMigrations
|
|
78
81
|
end
|
79
82
|
|
80
83
|
def disable_ddl_transaction
|
84
|
+
UselessStatementsLogger.warn_useless '`disable_ddl_transaction`' if super
|
81
85
|
true
|
82
86
|
end
|
83
87
|
|
@@ -88,6 +92,7 @@ module SafePgMigrations
|
|
88
92
|
|
89
93
|
safety_assured { super(*args) }
|
90
94
|
end
|
95
|
+
ruby2_keywords method
|
91
96
|
end
|
92
97
|
end
|
93
98
|
end
|
@@ -5,16 +5,16 @@ require 'active_support/core_ext/numeric/time'
|
|
5
5
|
module SafePgMigrations
|
6
6
|
class Configuration
|
7
7
|
attr_accessor :safe_timeout
|
8
|
-
attr_accessor :index_lock_timeout
|
9
8
|
attr_accessor :blocking_activity_logger_margin
|
9
|
+
attr_accessor :blocking_activity_logger_verbose
|
10
10
|
attr_accessor :batch_size
|
11
11
|
attr_accessor :retry_delay
|
12
12
|
attr_accessor :max_tries
|
13
13
|
|
14
14
|
def initialize
|
15
15
|
self.safe_timeout = 5.seconds
|
16
|
-
self.index_lock_timeout = 30.seconds
|
17
16
|
self.blocking_activity_logger_margin = 1.second
|
17
|
+
self.blocking_activity_logger_verbose = true
|
18
18
|
self.batch_size = 1000
|
19
19
|
self.retry_delay = 1.minute
|
20
20
|
self.max_tries = 5
|
@@ -24,10 +24,6 @@ module SafePgMigrations
|
|
24
24
|
pg_duration(safe_timeout)
|
25
25
|
end
|
26
26
|
|
27
|
-
def pg_index_lock_timeout
|
28
|
-
pg_duration(index_lock_timeout)
|
29
|
-
end
|
30
|
-
|
31
27
|
def pg_duration(duration)
|
32
28
|
value, unit = duration.integer? ? [duration, 's'] : [(duration * 1000).to_i, 'ms']
|
33
29
|
"#{value}#{unit}"
|
@@ -2,55 +2,64 @@
|
|
2
2
|
|
3
3
|
module SafePgMigrations
|
4
4
|
module BlockingActivityLogger
|
5
|
-
|
6
|
-
|
7
|
-
|
8
|
-
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
AND blocking_locks.transactionid IS NOT DISTINCT FROM blocked_locks.transactionid
|
18
|
-
AND blocking_locks.classid IS NOT DISTINCT FROM blocked_locks.classid
|
19
|
-
AND blocking_locks.objid IS NOT DISTINCT FROM blocked_locks.objid
|
20
|
-
AND blocking_locks.objsubid IS NOT DISTINCT FROM blocked_locks.objsubid
|
21
|
-
AND blocking_locks.pid != blocked_locks.pid
|
22
|
-
JOIN pg_catalog.pg_stat_activity blocking_activity
|
23
|
-
ON blocking_activity.pid = blocking_locks.pid
|
24
|
-
WHERE blocked_locks.pid = %d
|
25
|
-
SQL
|
5
|
+
FILTERED_COLUMNS = %w[
|
6
|
+
blocked_activity.xact_start
|
7
|
+
blocked_locks.locktype
|
8
|
+
blocked_locks.mode
|
9
|
+
blocking_activity.pid
|
10
|
+
blocked_locks.transactionid
|
11
|
+
].freeze
|
12
|
+
|
13
|
+
VERBOSE_COLUMNS = %w[
|
14
|
+
blocking_activity.query
|
15
|
+
blocked_activity.xact_start
|
16
|
+
].freeze
|
26
17
|
|
27
18
|
%i[
|
28
|
-
add_column remove_column add_foreign_key remove_foreign_key change_column_default
|
29
|
-
change_column_null create_table add_index remove_index
|
19
|
+
add_column remove_column add_foreign_key remove_foreign_key change_column_default change_column_null create_table
|
30
20
|
].each do |method|
|
31
21
|
define_method method do |*args, &block|
|
32
|
-
log_blocking_queries
|
22
|
+
log_blocking_queries { super(*args, &block) }
|
33
23
|
end
|
24
|
+
ruby2_keywords method
|
34
25
|
end
|
35
26
|
|
36
27
|
private
|
37
28
|
|
38
|
-
def
|
39
|
-
|
40
|
-
if %i[add_index remove_index].include?(method)
|
41
|
-
SafePgMigrations.config.index_lock_timeout
|
42
|
-
else
|
43
|
-
SafePgMigrations.config.safe_timeout
|
44
|
-
end
|
29
|
+
def select_blocking_queries_sql
|
30
|
+
columns = SafePgMigrations.config.blocking_activity_logger_verbose ? VERBOSE_COLUMNS : FILTERED_COLUMNS
|
45
31
|
|
46
|
-
|
32
|
+
<<~SQL.squish
|
33
|
+
SELECT #{columns.join(', ')}
|
34
|
+
FROM pg_catalog.pg_locks blocked_locks
|
35
|
+
JOIN pg_catalog.pg_stat_activity blocked_activity
|
36
|
+
ON blocked_activity.pid = blocked_locks.pid
|
37
|
+
JOIN pg_catalog.pg_locks blocking_locks
|
38
|
+
ON blocking_locks.locktype = blocked_locks.locktype
|
39
|
+
AND blocking_locks.DATABASE IS NOT DISTINCT FROM blocked_locks.DATABASE
|
40
|
+
AND blocking_locks.relation IS NOT DISTINCT FROM blocked_locks.relation
|
41
|
+
AND blocking_locks.page IS NOT DISTINCT FROM blocked_locks.page
|
42
|
+
AND blocking_locks.tuple IS NOT DISTINCT FROM blocked_locks.tuple
|
43
|
+
AND blocking_locks.virtualxid IS NOT DISTINCT FROM blocked_locks.virtualxid
|
44
|
+
AND blocking_locks.transactionid IS NOT DISTINCT FROM blocked_locks.transactionid
|
45
|
+
AND blocking_locks.classid IS NOT DISTINCT FROM blocked_locks.classid
|
46
|
+
AND blocking_locks.objid IS NOT DISTINCT FROM blocked_locks.objid
|
47
|
+
AND blocking_locks.objsubid IS NOT DISTINCT FROM blocked_locks.objsubid
|
48
|
+
AND blocking_locks.pid != blocked_locks.pid
|
49
|
+
JOIN pg_catalog.pg_stat_activity blocking_activity
|
50
|
+
ON blocking_activity.pid = blocking_locks.pid
|
51
|
+
WHERE blocked_locks.pid = %d
|
52
|
+
SQL
|
47
53
|
end
|
48
54
|
|
49
|
-
def log_blocking_queries
|
55
|
+
def log_blocking_queries
|
56
|
+
delay_before_logging =
|
57
|
+
SafePgMigrations.config.safe_timeout - SafePgMigrations.config.blocking_activity_logger_margin
|
58
|
+
|
50
59
|
blocking_queries_retriever_thread =
|
51
60
|
Thread.new do
|
52
|
-
sleep delay_before_logging
|
53
|
-
SafePgMigrations.alternate_connection.
|
61
|
+
sleep delay_before_logging
|
62
|
+
SafePgMigrations.alternate_connection.query(select_blocking_queries_sql % raw_connection.backend_pid)
|
54
63
|
end
|
55
64
|
|
56
65
|
yield
|
@@ -75,7 +84,7 @@ module SafePgMigrations
|
|
75
84
|
"Statement was being blocked by the following #{'query'.pluralize(queries.size)}:", true
|
76
85
|
)
|
77
86
|
SafePgMigrations.say '', true
|
78
|
-
queries
|
87
|
+
output_blocking_queries(queries)
|
79
88
|
SafePgMigrations.say(
|
80
89
|
'Beware, some of those queries might run in a transaction. In this case the locking query might be '\
|
81
90
|
'located elsewhere in the transaction',
|
@@ -86,5 +95,26 @@ module SafePgMigrations
|
|
86
95
|
|
87
96
|
raise
|
88
97
|
end
|
98
|
+
|
99
|
+
def output_blocking_queries(queries)
|
100
|
+
if SafePgMigrations.config.blocking_activity_logger_verbose
|
101
|
+
queries.each { |query, start_time| SafePgMigrations.say "#{format_start_time start_time}: #{query}", true }
|
102
|
+
else
|
103
|
+
queries.each do |start_time, locktype, mode, pid, transactionid|
|
104
|
+
SafePgMigrations.say(
|
105
|
+
"#{format_start_time(start_time)}: lock type: #{locktype || 'null'}, " \
|
106
|
+
"lock mode: #{mode || 'null'}, " \
|
107
|
+
"lock pid: #{pid || 'null'}, " \
|
108
|
+
"lock transactionid: #{transactionid || 'null'}",
|
109
|
+
true
|
110
|
+
)
|
111
|
+
end
|
112
|
+
end
|
113
|
+
end
|
114
|
+
|
115
|
+
def format_start_time(start_time, reference_time = Time.now)
|
116
|
+
duration = (reference_time - start_time).round
|
117
|
+
"transaction started #{duration} #{'second'.pluralize(duration)} ago"
|
118
|
+
end
|
89
119
|
end
|
90
120
|
end
|
@@ -2,7 +2,8 @@
|
|
2
2
|
|
3
3
|
module SafePgMigrations
|
4
4
|
module IdemPotentStatements
|
5
|
-
def add_index(table_name, column_name,
|
5
|
+
ruby2_keywords def add_index(table_name, column_name, *args)
|
6
|
+
options = args.last.is_a?(Hash) ? args.last : {}
|
6
7
|
index_name = options.key?(:name) ? options[:name].to_s : index_name(table_name, index_column_names(column_name))
|
7
8
|
return super unless index_name_exists?(table_name, index_name)
|
8
9
|
|
@@ -12,19 +13,20 @@ module SafePgMigrations
|
|
12
13
|
super
|
13
14
|
end
|
14
15
|
|
15
|
-
def add_column(table_name, column_name, type,
|
16
|
+
ruby2_keywords def add_column(table_name, column_name, type, *)
|
16
17
|
return super unless column_exists?(table_name, column_name)
|
17
18
|
|
18
19
|
SafePgMigrations.say("/!\\ Column '#{column_name}' already exists in '#{table_name}'. Skipping statement.", true)
|
19
20
|
end
|
20
21
|
|
21
|
-
def remove_column(table_name, column_name, type = nil,
|
22
|
+
ruby2_keywords def remove_column(table_name, column_name, type = nil, *)
|
22
23
|
return super if column_exists?(table_name, column_name)
|
23
24
|
|
24
25
|
SafePgMigrations.say("/!\\ Column '#{column_name}' not found on table '#{table_name}'. Skipping statement.", true)
|
25
26
|
end
|
26
27
|
|
27
|
-
def remove_index(table_name,
|
28
|
+
ruby2_keywords def remove_index(table_name, *args)
|
29
|
+
options = args.last.is_a?(Hash) ? args.last : {}
|
28
30
|
index_name = options.key?(:name) ? options[:name].to_s : index_name(table_name, options)
|
29
31
|
|
30
32
|
return super if index_name_exists?(table_name, index_name)
|
@@ -32,9 +34,10 @@ module SafePgMigrations
|
|
32
34
|
SafePgMigrations.say("/!\\ Index '#{index_name}' not found on table '#{table_name}'. Skipping statement.", true)
|
33
35
|
end
|
34
36
|
|
35
|
-
def add_foreign_key(from_table, to_table,
|
36
|
-
|
37
|
-
|
37
|
+
ruby2_keywords def add_foreign_key(from_table, to_table, *args)
|
38
|
+
options = args.last.is_a?(Hash) ? args.last : {}
|
39
|
+
suboptions = options.slice(:name, :column)
|
40
|
+
return super unless foreign_key_exists?(from_table, suboptions.present? ? nil : to_table, **suboptions)
|
38
41
|
|
39
42
|
SafePgMigrations.say(
|
40
43
|
"/!\\ Foreign key '#{from_table}' -> '#{to_table}' already exists. Skipping statement.",
|
@@ -42,13 +45,21 @@ module SafePgMigrations
|
|
42
45
|
)
|
43
46
|
end
|
44
47
|
|
45
|
-
def create_table(table_name,
|
46
|
-
|
48
|
+
ruby2_keywords def create_table(table_name, *args)
|
49
|
+
options = args.last.is_a?(Hash) ? args.last : {}
|
50
|
+
return super if options[:force] || !table_exists?(table_name)
|
47
51
|
|
48
|
-
SafePgMigrations.say
|
49
|
-
|
50
|
-
|
51
|
-
|
52
|
+
SafePgMigrations.say "/!\\ Table '#{table_name}' already exists.", true
|
53
|
+
|
54
|
+
td = create_table_definition(table_name, *args)
|
55
|
+
|
56
|
+
yield td if block_given?
|
57
|
+
|
58
|
+
SafePgMigrations.say(td.indexes.empty? ? '-- Skipping statement' : '-- Creating indexes', true)
|
59
|
+
|
60
|
+
td.indexes.each do |column_name, index_options|
|
61
|
+
add_index(table_name, column_name, **index_options)
|
62
|
+
end
|
52
63
|
end
|
53
64
|
|
54
65
|
private
|
@@ -1,19 +1,21 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
3
|
module SafePgMigrations
|
4
|
-
module StatementInsurer
|
4
|
+
module StatementInsurer # rubocop:disable Metrics/ModuleLength
|
5
5
|
PG_11_VERSION_NUM = 110_000
|
6
6
|
|
7
|
-
%i[change_column_null change_column
|
7
|
+
%i[change_column_null change_column].each do |method|
|
8
8
|
define_method method do |*args, &block|
|
9
9
|
with_setting(:statement_timeout, SafePgMigrations.config.pg_safe_timeout) { super(*args, &block) }
|
10
10
|
end
|
11
|
+
ruby2_keywords method
|
11
12
|
end
|
12
13
|
|
13
|
-
def add_column(table_name, column_name, type,
|
14
|
-
|
14
|
+
ruby2_keywords def add_column(table_name, column_name, type, *args) # rubocop:disable Metrics/CyclomaticComplexity
|
15
|
+
options = args.last.is_a?(Hash) ? args.last : {}
|
16
|
+
return super if SafePgMigrations.pg_version_num >= PG_11_VERSION_NUM
|
15
17
|
|
16
|
-
default = options.delete(:default)
|
18
|
+
default = options.delete(:default)
|
17
19
|
null = options.delete(:null)
|
18
20
|
|
19
21
|
if !default.nil? || null == false
|
@@ -22,7 +24,7 @@ module SafePgMigrations
|
|
22
24
|
|
23
25
|
super
|
24
26
|
|
25
|
-
|
27
|
+
unless default.nil?
|
26
28
|
SafePgMigrations.say_method_call(:change_column_default, table_name, column_name, default)
|
27
29
|
change_column_default(table_name, column_name, default)
|
28
30
|
|
@@ -36,29 +38,50 @@ module SafePgMigrations
|
|
36
38
|
end
|
37
39
|
end
|
38
40
|
|
39
|
-
def add_foreign_key(from_table, to_table,
|
41
|
+
ruby2_keywords def add_foreign_key(from_table, to_table, *args)
|
42
|
+
options = args.last.is_a?(Hash) ? args.last : {}
|
40
43
|
validate_present = options.key? :validate
|
41
44
|
options[:validate] = false unless validate_present
|
45
|
+
with_setting(:statement_timeout, SafePgMigrations.config.pg_safe_timeout) do
|
46
|
+
super(from_table, to_table, **options)
|
47
|
+
end
|
48
|
+
|
49
|
+
return if validate_present
|
42
50
|
|
43
|
-
|
51
|
+
suboptions = options.slice(:name, :column)
|
52
|
+
without_statement_timeout { validate_foreign_key from_table, suboptions.present? ? nil : to_table, **suboptions }
|
53
|
+
end
|
44
54
|
|
45
|
-
|
46
|
-
|
55
|
+
ruby2_keywords def create_table(*)
|
56
|
+
with_setting(:statement_timeout, SafePgMigrations.config.pg_safe_timeout) do
|
57
|
+
super do |td|
|
58
|
+
yield td if block_given?
|
59
|
+
td.indexes.map! do |key, index_options|
|
60
|
+
index_options[:algorithm] ||= :default
|
61
|
+
[key, index_options]
|
62
|
+
end
|
63
|
+
end
|
64
|
+
end
|
47
65
|
end
|
48
66
|
|
49
67
|
def add_index(table_name, column_name, **options)
|
50
|
-
options[:algorithm]
|
51
|
-
|
68
|
+
if options[:algorithm] == :default
|
69
|
+
options.delete :algorithm
|
70
|
+
else
|
71
|
+
options[:algorithm] = :concurrently
|
72
|
+
end
|
52
73
|
|
53
|
-
|
74
|
+
SafePgMigrations.say_method_call(:add_index, table_name, column_name, **options)
|
75
|
+
|
76
|
+
without_timeout { super(table_name, column_name, **options) }
|
54
77
|
end
|
55
78
|
|
56
|
-
def remove_index(table_name,
|
57
|
-
options = { column:
|
58
|
-
options[:algorithm] = :concurrently
|
59
|
-
SafePgMigrations.say_method_call(:remove_index, table_name, options)
|
79
|
+
ruby2_keywords def remove_index(table_name, *args)
|
80
|
+
options = args.last.is_a?(Hash) ? args.last : { column: args.last }
|
81
|
+
options[:algorithm] = :concurrently unless options.key?(:algorithm)
|
82
|
+
SafePgMigrations.say_method_call(:remove_index, table_name, **options)
|
60
83
|
|
61
|
-
|
84
|
+
without_timeout { super(table_name, **options) }
|
62
85
|
end
|
63
86
|
|
64
87
|
def backfill_column_default(table_name, column_name)
|
@@ -99,12 +122,12 @@ module SafePgMigrations
|
|
99
122
|
with_setting(:statement_timeout, 0) { yield }
|
100
123
|
end
|
101
124
|
|
102
|
-
def
|
103
|
-
|
104
|
-
|
105
|
-
|
106
|
-
|
107
|
-
|
125
|
+
def without_lock_timeout
|
126
|
+
with_setting(:lock_timeout, 0) { yield }
|
127
|
+
end
|
128
|
+
|
129
|
+
def without_timeout
|
130
|
+
without_statement_timeout { without_lock_timeout { yield } }
|
108
131
|
end
|
109
132
|
end
|
110
133
|
end
|
@@ -3,14 +3,14 @@
|
|
3
3
|
module SafePgMigrations
|
4
4
|
module StatementRetrier
|
5
5
|
RETRIABLE_SCHEMA_STATEMENTS = %i[
|
6
|
-
add_column add_foreign_key remove_foreign_key change_column_default
|
7
|
-
change_column_null add_index remove_index remove_column
|
6
|
+
add_column add_foreign_key remove_foreign_key change_column_default change_column_null remove_column
|
8
7
|
].freeze
|
9
8
|
|
10
9
|
RETRIABLE_SCHEMA_STATEMENTS.each do |method|
|
11
10
|
define_method method do |*args, &block|
|
12
11
|
retry_if_lock_timeout { super(*args, &block) }
|
13
12
|
end
|
13
|
+
ruby2_keywords method
|
14
14
|
end
|
15
15
|
|
16
16
|
private
|
@@ -0,0 +1,38 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module SafePgMigrations
|
4
|
+
module UselessStatementsLogger
|
5
|
+
class << self
|
6
|
+
ruby2_keywords def warn_useless(action, link = nil, *args)
|
7
|
+
SafePgMigrations.say "/!\\ No need to explicitly use #{action}, safe-pg-migrations does it for you", *args
|
8
|
+
SafePgMigrations.say "\t see #{link} for more details", *args if link
|
9
|
+
end
|
10
|
+
end
|
11
|
+
|
12
|
+
ruby2_keywords def add_index(*args)
|
13
|
+
options = args.last.is_a?(Hash) ? args.last : {}
|
14
|
+
warn_for_index(**options)
|
15
|
+
super
|
16
|
+
end
|
17
|
+
|
18
|
+
ruby2_keywords def remove_index(table_name, *args)
|
19
|
+
options = args.last.is_a?(Hash) ? args.last : {}
|
20
|
+
warn_for_index(**options) unless options.empty?
|
21
|
+
super
|
22
|
+
end
|
23
|
+
|
24
|
+
ruby2_keywords def add_foreign_key(*args)
|
25
|
+
options = args.last.is_a?(Hash) ? args.last : {}
|
26
|
+
if options[:validate] == false
|
27
|
+
UselessStatementsLogger.warn_useless '`validate: :false`', 'https://github.com/doctolib/safe-pg-migrations#safe_add_foreign_key'
|
28
|
+
end
|
29
|
+
super
|
30
|
+
end
|
31
|
+
|
32
|
+
def warn_for_index(**options)
|
33
|
+
return unless options[:algorithm] == :concurrently
|
34
|
+
|
35
|
+
UselessStatementsLogger.warn_useless '`algorithm: :concurrently`', 'https://github.com/doctolib/safe-pg-migrations#safe_add_remove_index'
|
36
|
+
end
|
37
|
+
end
|
38
|
+
end
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: safe-pg-migrations
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.3.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Matthieu Prat
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date:
|
12
|
+
date: 2021-06-28 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: activerecord
|
@@ -39,6 +39,20 @@ dependencies:
|
|
39
39
|
- - ">="
|
40
40
|
- !ruby/object:Gem::Version
|
41
41
|
version: '5.2'
|
42
|
+
- !ruby/object:Gem::Dependency
|
43
|
+
name: ruby2_keywords
|
44
|
+
requirement: !ruby/object:Gem::Requirement
|
45
|
+
requirements:
|
46
|
+
- - ">="
|
47
|
+
- !ruby/object:Gem::Version
|
48
|
+
version: 0.0.4
|
49
|
+
type: :runtime
|
50
|
+
prerelease: false
|
51
|
+
version_requirements: !ruby/object:Gem::Requirement
|
52
|
+
requirements:
|
53
|
+
- - ">="
|
54
|
+
- !ruby/object:Gem::Version
|
55
|
+
version: 0.0.4
|
42
56
|
- !ruby/object:Gem::Dependency
|
43
57
|
name: bundler
|
44
58
|
requirement: !ruby/object:Gem::Requirement
|
@@ -166,6 +180,7 @@ files:
|
|
166
180
|
- lib/safe-pg-migrations/plugins/idem_potent_statements.rb
|
167
181
|
- lib/safe-pg-migrations/plugins/statement_insurer.rb
|
168
182
|
- lib/safe-pg-migrations/plugins/statement_retrier.rb
|
183
|
+
- lib/safe-pg-migrations/plugins/useless_statements_logger.rb
|
169
184
|
- lib/safe-pg-migrations/plugins/verbose_sql_logger.rb
|
170
185
|
- lib/safe-pg-migrations/railtie.rb
|
171
186
|
- lib/safe-pg-migrations/version.rb
|
@@ -182,6 +197,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
182
197
|
- - ">="
|
183
198
|
- !ruby/object:Gem::Version
|
184
199
|
version: '2.5'
|
200
|
+
- - "<"
|
201
|
+
- !ruby/object:Gem::Version
|
202
|
+
version: '4'
|
185
203
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
186
204
|
requirements:
|
187
205
|
- - ">="
|