safe-pg-migrations 1.1.0 → 1.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +8 -5
- data/lib/safe-pg-migrations/base.rb +7 -2
- data/lib/safe-pg-migrations/configuration.rb +2 -6
- data/lib/safe-pg-migrations/plugins/blocking_activity_logger.rb +66 -36
- data/lib/safe-pg-migrations/plugins/idem_potent_statements.rb +24 -13
- data/lib/safe-pg-migrations/plugins/statement_insurer.rb +47 -24
- data/lib/safe-pg-migrations/plugins/statement_retrier.rb +2 -2
- data/lib/safe-pg-migrations/plugins/useless_statements_logger.rb +38 -0
- data/lib/safe-pg-migrations/version.rb +1 -1
- metadata +20 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: fdee6e752da3fbdec93e5fd5c72fb370497ddd66f1cc27f33609c2a31e364c53
|
4
|
+
data.tar.gz: 143a5dcbf614abed765fac78b16969d900ded5ec574f11d939d802b94d7e2586
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 41c264aa05c7b8ad5fe90f7ecdc88fa700504562e69695d5fd9f4dc9ac63315fb666818554e1387251b367aed241fccff31154589f70309905e1ed1cfae90c68
|
7
|
+
data.tar.gz: 9386e6a8b0366387543998ec5cec42e719d2a4138d5eb52f13f6580d2a3d31e48bd438760b358beb1ab5eb32bd8598927d69bafd03756108a514e88eeb431022
|
data/README.md
CHANGED
@@ -16,6 +16,8 @@ Just drop this line in your Gemfile:
|
|
16
16
|
gem 'safe-pg-migrations'
|
17
17
|
```
|
18
18
|
|
19
|
+
**Note: Do not run migrations via PgBouncer connection if it is configured to use transactional or statement pooling modes. You must run migrations via a direct Postgres connection, or configure PgBouncer to use session pooling mode.**
|
20
|
+
|
19
21
|
## Example
|
20
22
|
|
21
23
|
Consider the following migration:
|
@@ -61,7 +63,7 @@ Under the hood, **Safe PG Migrations** patches `ActiveRecord::Migration` and ext
|
|
61
63
|
|
62
64
|
## Motivation
|
63
65
|
|
64
|
-
Writing a safe migration can be daunting. Numerous articles have been written on the topic and a few gems are trying to address the problem. Even for someone who has a pretty good command of Postgres, remembering all the subtleties of explicit locking is not a piece of cake.
|
66
|
+
Writing a safe migration can be daunting. Numerous articles, [including ours](https://medium.com/doctolib/stop-worrying-about-postgresql-locks-in-your-rails-migrations-3426027e9cc9), have been written on the topic and a few gems are trying to address the problem. Even for someone who has a pretty good command of Postgres, remembering all the subtleties of explicit locking is not a piece of cake.
|
65
67
|
|
66
68
|
Active Record means developers don't have to be proficient in SQL to interact with a database. In the same way, **Safe PG Migrations** was created so that developers don't have to understand the ins and outs of Postgres to write a safe migration.
|
67
69
|
|
@@ -122,7 +124,7 @@ Note: the addition of the not null constraint may timeout. In that case, you may
|
|
122
124
|
|
123
125
|
</details>
|
124
126
|
|
125
|
-
<details><summary>Safe <code>add_index</code> and <code>remove_index</code></summary>
|
127
|
+
<details><summary id="safe_add_remove_index">Safe <code>add_index</code> and <code>remove_index</code></summary>
|
126
128
|
|
127
129
|
Creating an index requires a `SHARE` lock on the target table which blocks all write on the table while the index is created (which can take some time on a large table). This is usually not practical in a live environment. Thus, **Safe PG Migrations** ensures indexes are created concurrently.
|
128
130
|
|
@@ -135,7 +137,7 @@ If you still get lock timeout while adding / removing indexes, it might be for o
|
|
135
137
|
|
136
138
|
</details>
|
137
139
|
|
138
|
-
<details><summary>safe <code>add_foreign_key</code> (and <code>add_reference</code>)</summary>
|
140
|
+
<details><summary id="safe_add_foreign_key">safe <code>add_foreign_key</code> (and <code>add_reference</code>)</summary>
|
139
141
|
|
140
142
|
Adding a foreign key requires a `SHARE ROW EXCLUSIVE` lock, which **prevent writing in the tables** while the migration is running.
|
141
143
|
|
@@ -206,9 +208,9 @@ So you can actually check that the `CREATE INDEX` statement will be performed co
|
|
206
208
|
```ruby
|
207
209
|
SafePgMigrations.config.safe_timeout = 5.seconds # Lock and statement timeout used for all DDL operations except from CREATE / DROP INDEX
|
208
210
|
|
209
|
-
SafePgMigrations.config.
|
211
|
+
SafePgMigrations.config.blocking_activity_logger_verbose = true # Outputs the raw blocking queries on timeout. When false, outputs information about the lock instead
|
210
212
|
|
211
|
-
SafePgMigrations.config.blocking_activity_logger_margin = 1.second # Delay to output blocking queries before timeout. Must be
|
213
|
+
SafePgMigrations.config.blocking_activity_logger_margin = 1.second # Delay to output blocking queries before timeout. Must be shorter than safe_timeout
|
212
214
|
|
213
215
|
SafePgMigrations.config.batch_size = 1000 # Size of the batches used for backfilling when adding a column with a default value pre-PG11
|
214
216
|
|
@@ -254,3 +256,4 @@ Interesting reads:
|
|
254
256
|
- [Rails migrations with no downtime](https://pedro.herokuapp.com/past/2011/7/13/rails_migrations_with_no_downtime/)
|
255
257
|
- [Safe Operations For High Volume PostgreSQL](https://www.braintreepayments.com/blog/safe-operations-for-high-volume-postgresql/)
|
256
258
|
- [Rails Migrations with Zero Downtime](https://blog.codeship.com/rails-migrations-zero-downtime/)
|
259
|
+
- [Stop worrying about PostgreSQL locks in your Rails migrations](https://medium.com/doctolib/stop-worrying-about-postgresql-locks-in-your-rails-migrations-3426027e9cc9)
|
@@ -1,11 +1,13 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
|
+
require 'ruby2_keywords'
|
3
4
|
require 'safe-pg-migrations/configuration'
|
4
5
|
require 'safe-pg-migrations/plugins/verbose_sql_logger'
|
5
6
|
require 'safe-pg-migrations/plugins/blocking_activity_logger'
|
6
7
|
require 'safe-pg-migrations/plugins/statement_insurer'
|
7
8
|
require 'safe-pg-migrations/plugins/statement_retrier'
|
8
9
|
require 'safe-pg-migrations/plugins/idem_potent_statements'
|
10
|
+
require 'safe-pg-migrations/plugins/useless_statements_logger'
|
9
11
|
|
10
12
|
module SafePgMigrations
|
11
13
|
# Order matters: the bottom-most plugin will have precedence
|
@@ -14,6 +16,7 @@ module SafePgMigrations
|
|
14
16
|
IdemPotentStatements,
|
15
17
|
StatementRetrier,
|
16
18
|
StatementInsurer,
|
19
|
+
UselessStatementsLogger,
|
17
20
|
].freeze
|
18
21
|
|
19
22
|
class << self
|
@@ -48,13 +51,13 @@ module SafePgMigrations
|
|
48
51
|
@alternate_connection = nil
|
49
52
|
end
|
50
53
|
|
51
|
-
def say(*args)
|
54
|
+
ruby2_keywords def say(*args)
|
52
55
|
return unless current_migration
|
53
56
|
|
54
57
|
current_migration.say(*args)
|
55
58
|
end
|
56
59
|
|
57
|
-
def say_method_call(method, *args)
|
60
|
+
ruby2_keywords def say_method_call(method, *args)
|
58
61
|
say "#{method}(#{args.map(&:inspect) * ', '})", true
|
59
62
|
end
|
60
63
|
|
@@ -78,6 +81,7 @@ module SafePgMigrations
|
|
78
81
|
end
|
79
82
|
|
80
83
|
def disable_ddl_transaction
|
84
|
+
UselessStatementsLogger.warn_useless '`disable_ddl_transaction`' if super
|
81
85
|
true
|
82
86
|
end
|
83
87
|
|
@@ -88,6 +92,7 @@ module SafePgMigrations
|
|
88
92
|
|
89
93
|
safety_assured { super(*args) }
|
90
94
|
end
|
95
|
+
ruby2_keywords method
|
91
96
|
end
|
92
97
|
end
|
93
98
|
end
|
@@ -5,16 +5,16 @@ require 'active_support/core_ext/numeric/time'
|
|
5
5
|
module SafePgMigrations
|
6
6
|
class Configuration
|
7
7
|
attr_accessor :safe_timeout
|
8
|
-
attr_accessor :index_lock_timeout
|
9
8
|
attr_accessor :blocking_activity_logger_margin
|
9
|
+
attr_accessor :blocking_activity_logger_verbose
|
10
10
|
attr_accessor :batch_size
|
11
11
|
attr_accessor :retry_delay
|
12
12
|
attr_accessor :max_tries
|
13
13
|
|
14
14
|
def initialize
|
15
15
|
self.safe_timeout = 5.seconds
|
16
|
-
self.index_lock_timeout = 30.seconds
|
17
16
|
self.blocking_activity_logger_margin = 1.second
|
17
|
+
self.blocking_activity_logger_verbose = true
|
18
18
|
self.batch_size = 1000
|
19
19
|
self.retry_delay = 1.minute
|
20
20
|
self.max_tries = 5
|
@@ -24,10 +24,6 @@ module SafePgMigrations
|
|
24
24
|
pg_duration(safe_timeout)
|
25
25
|
end
|
26
26
|
|
27
|
-
def pg_index_lock_timeout
|
28
|
-
pg_duration(index_lock_timeout)
|
29
|
-
end
|
30
|
-
|
31
27
|
def pg_duration(duration)
|
32
28
|
value, unit = duration.integer? ? [duration, 's'] : [(duration * 1000).to_i, 'ms']
|
33
29
|
"#{value}#{unit}"
|
@@ -2,55 +2,64 @@
|
|
2
2
|
|
3
3
|
module SafePgMigrations
|
4
4
|
module BlockingActivityLogger
|
5
|
-
|
6
|
-
|
7
|
-
|
8
|
-
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
AND blocking_locks.transactionid IS NOT DISTINCT FROM blocked_locks.transactionid
|
18
|
-
AND blocking_locks.classid IS NOT DISTINCT FROM blocked_locks.classid
|
19
|
-
AND blocking_locks.objid IS NOT DISTINCT FROM blocked_locks.objid
|
20
|
-
AND blocking_locks.objsubid IS NOT DISTINCT FROM blocked_locks.objsubid
|
21
|
-
AND blocking_locks.pid != blocked_locks.pid
|
22
|
-
JOIN pg_catalog.pg_stat_activity blocking_activity
|
23
|
-
ON blocking_activity.pid = blocking_locks.pid
|
24
|
-
WHERE blocked_locks.pid = %d
|
25
|
-
SQL
|
5
|
+
FILTERED_COLUMNS = %w[
|
6
|
+
blocked_activity.xact_start
|
7
|
+
blocked_locks.locktype
|
8
|
+
blocked_locks.mode
|
9
|
+
blocking_activity.pid
|
10
|
+
blocked_locks.transactionid
|
11
|
+
].freeze
|
12
|
+
|
13
|
+
VERBOSE_COLUMNS = %w[
|
14
|
+
blocking_activity.query
|
15
|
+
blocked_activity.xact_start
|
16
|
+
].freeze
|
26
17
|
|
27
18
|
%i[
|
28
|
-
add_column remove_column add_foreign_key remove_foreign_key change_column_default
|
29
|
-
change_column_null create_table add_index remove_index
|
19
|
+
add_column remove_column add_foreign_key remove_foreign_key change_column_default change_column_null create_table
|
30
20
|
].each do |method|
|
31
21
|
define_method method do |*args, &block|
|
32
|
-
log_blocking_queries
|
22
|
+
log_blocking_queries { super(*args, &block) }
|
33
23
|
end
|
24
|
+
ruby2_keywords method
|
34
25
|
end
|
35
26
|
|
36
27
|
private
|
37
28
|
|
38
|
-
def
|
39
|
-
|
40
|
-
if %i[add_index remove_index].include?(method)
|
41
|
-
SafePgMigrations.config.index_lock_timeout
|
42
|
-
else
|
43
|
-
SafePgMigrations.config.safe_timeout
|
44
|
-
end
|
29
|
+
def select_blocking_queries_sql
|
30
|
+
columns = SafePgMigrations.config.blocking_activity_logger_verbose ? VERBOSE_COLUMNS : FILTERED_COLUMNS
|
45
31
|
|
46
|
-
|
32
|
+
<<~SQL.squish
|
33
|
+
SELECT #{columns.join(', ')}
|
34
|
+
FROM pg_catalog.pg_locks blocked_locks
|
35
|
+
JOIN pg_catalog.pg_stat_activity blocked_activity
|
36
|
+
ON blocked_activity.pid = blocked_locks.pid
|
37
|
+
JOIN pg_catalog.pg_locks blocking_locks
|
38
|
+
ON blocking_locks.locktype = blocked_locks.locktype
|
39
|
+
AND blocking_locks.DATABASE IS NOT DISTINCT FROM blocked_locks.DATABASE
|
40
|
+
AND blocking_locks.relation IS NOT DISTINCT FROM blocked_locks.relation
|
41
|
+
AND blocking_locks.page IS NOT DISTINCT FROM blocked_locks.page
|
42
|
+
AND blocking_locks.tuple IS NOT DISTINCT FROM blocked_locks.tuple
|
43
|
+
AND blocking_locks.virtualxid IS NOT DISTINCT FROM blocked_locks.virtualxid
|
44
|
+
AND blocking_locks.transactionid IS NOT DISTINCT FROM blocked_locks.transactionid
|
45
|
+
AND blocking_locks.classid IS NOT DISTINCT FROM blocked_locks.classid
|
46
|
+
AND blocking_locks.objid IS NOT DISTINCT FROM blocked_locks.objid
|
47
|
+
AND blocking_locks.objsubid IS NOT DISTINCT FROM blocked_locks.objsubid
|
48
|
+
AND blocking_locks.pid != blocked_locks.pid
|
49
|
+
JOIN pg_catalog.pg_stat_activity blocking_activity
|
50
|
+
ON blocking_activity.pid = blocking_locks.pid
|
51
|
+
WHERE blocked_locks.pid = %d
|
52
|
+
SQL
|
47
53
|
end
|
48
54
|
|
49
|
-
def log_blocking_queries
|
55
|
+
def log_blocking_queries
|
56
|
+
delay_before_logging =
|
57
|
+
SafePgMigrations.config.safe_timeout - SafePgMigrations.config.blocking_activity_logger_margin
|
58
|
+
|
50
59
|
blocking_queries_retriever_thread =
|
51
60
|
Thread.new do
|
52
|
-
sleep delay_before_logging
|
53
|
-
SafePgMigrations.alternate_connection.
|
61
|
+
sleep delay_before_logging
|
62
|
+
SafePgMigrations.alternate_connection.query(select_blocking_queries_sql % raw_connection.backend_pid)
|
54
63
|
end
|
55
64
|
|
56
65
|
yield
|
@@ -75,7 +84,7 @@ module SafePgMigrations
|
|
75
84
|
"Statement was being blocked by the following #{'query'.pluralize(queries.size)}:", true
|
76
85
|
)
|
77
86
|
SafePgMigrations.say '', true
|
78
|
-
queries
|
87
|
+
output_blocking_queries(queries)
|
79
88
|
SafePgMigrations.say(
|
80
89
|
'Beware, some of those queries might run in a transaction. In this case the locking query might be '\
|
81
90
|
'located elsewhere in the transaction',
|
@@ -86,5 +95,26 @@ module SafePgMigrations
|
|
86
95
|
|
87
96
|
raise
|
88
97
|
end
|
98
|
+
|
99
|
+
def output_blocking_queries(queries)
|
100
|
+
if SafePgMigrations.config.blocking_activity_logger_verbose
|
101
|
+
queries.each { |query, start_time| SafePgMigrations.say "#{format_start_time start_time}: #{query}", true }
|
102
|
+
else
|
103
|
+
queries.each do |start_time, locktype, mode, pid, transactionid|
|
104
|
+
SafePgMigrations.say(
|
105
|
+
"#{format_start_time(start_time)}: lock type: #{locktype || 'null'}, " \
|
106
|
+
"lock mode: #{mode || 'null'}, " \
|
107
|
+
"lock pid: #{pid || 'null'}, " \
|
108
|
+
"lock transactionid: #{transactionid || 'null'}",
|
109
|
+
true
|
110
|
+
)
|
111
|
+
end
|
112
|
+
end
|
113
|
+
end
|
114
|
+
|
115
|
+
def format_start_time(start_time, reference_time = Time.now)
|
116
|
+
duration = (reference_time - start_time).round
|
117
|
+
"transaction started #{duration} #{'second'.pluralize(duration)} ago"
|
118
|
+
end
|
89
119
|
end
|
90
120
|
end
|
@@ -2,7 +2,8 @@
|
|
2
2
|
|
3
3
|
module SafePgMigrations
|
4
4
|
module IdemPotentStatements
|
5
|
-
def add_index(table_name, column_name,
|
5
|
+
ruby2_keywords def add_index(table_name, column_name, *args)
|
6
|
+
options = args.last.is_a?(Hash) ? args.last : {}
|
6
7
|
index_name = options.key?(:name) ? options[:name].to_s : index_name(table_name, index_column_names(column_name))
|
7
8
|
return super unless index_name_exists?(table_name, index_name)
|
8
9
|
|
@@ -12,19 +13,20 @@ module SafePgMigrations
|
|
12
13
|
super
|
13
14
|
end
|
14
15
|
|
15
|
-
def add_column(table_name, column_name, type,
|
16
|
+
ruby2_keywords def add_column(table_name, column_name, type, *)
|
16
17
|
return super unless column_exists?(table_name, column_name)
|
17
18
|
|
18
19
|
SafePgMigrations.say("/!\\ Column '#{column_name}' already exists in '#{table_name}'. Skipping statement.", true)
|
19
20
|
end
|
20
21
|
|
21
|
-
def remove_column(table_name, column_name, type = nil,
|
22
|
+
ruby2_keywords def remove_column(table_name, column_name, type = nil, *)
|
22
23
|
return super if column_exists?(table_name, column_name)
|
23
24
|
|
24
25
|
SafePgMigrations.say("/!\\ Column '#{column_name}' not found on table '#{table_name}'. Skipping statement.", true)
|
25
26
|
end
|
26
27
|
|
27
|
-
def remove_index(table_name,
|
28
|
+
ruby2_keywords def remove_index(table_name, *args)
|
29
|
+
options = args.last.is_a?(Hash) ? args.last : {}
|
28
30
|
index_name = options.key?(:name) ? options[:name].to_s : index_name(table_name, options)
|
29
31
|
|
30
32
|
return super if index_name_exists?(table_name, index_name)
|
@@ -32,9 +34,10 @@ module SafePgMigrations
|
|
32
34
|
SafePgMigrations.say("/!\\ Index '#{index_name}' not found on table '#{table_name}'. Skipping statement.", true)
|
33
35
|
end
|
34
36
|
|
35
|
-
def add_foreign_key(from_table, to_table,
|
36
|
-
|
37
|
-
|
37
|
+
ruby2_keywords def add_foreign_key(from_table, to_table, *args)
|
38
|
+
options = args.last.is_a?(Hash) ? args.last : {}
|
39
|
+
suboptions = options.slice(:name, :column)
|
40
|
+
return super unless foreign_key_exists?(from_table, suboptions.present? ? nil : to_table, **suboptions)
|
38
41
|
|
39
42
|
SafePgMigrations.say(
|
40
43
|
"/!\\ Foreign key '#{from_table}' -> '#{to_table}' already exists. Skipping statement.",
|
@@ -42,13 +45,21 @@ module SafePgMigrations
|
|
42
45
|
)
|
43
46
|
end
|
44
47
|
|
45
|
-
def create_table(table_name,
|
46
|
-
|
48
|
+
ruby2_keywords def create_table(table_name, *args)
|
49
|
+
options = args.last.is_a?(Hash) ? args.last : {}
|
50
|
+
return super if options[:force] || !table_exists?(table_name)
|
47
51
|
|
48
|
-
SafePgMigrations.say
|
49
|
-
|
50
|
-
|
51
|
-
|
52
|
+
SafePgMigrations.say "/!\\ Table '#{table_name}' already exists.", true
|
53
|
+
|
54
|
+
td = create_table_definition(table_name, *args)
|
55
|
+
|
56
|
+
yield td if block_given?
|
57
|
+
|
58
|
+
SafePgMigrations.say(td.indexes.empty? ? '-- Skipping statement' : '-- Creating indexes', true)
|
59
|
+
|
60
|
+
td.indexes.each do |column_name, index_options|
|
61
|
+
add_index(table_name, column_name, **index_options)
|
62
|
+
end
|
52
63
|
end
|
53
64
|
|
54
65
|
private
|
@@ -1,19 +1,21 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
3
|
module SafePgMigrations
|
4
|
-
module StatementInsurer
|
4
|
+
module StatementInsurer # rubocop:disable Metrics/ModuleLength
|
5
5
|
PG_11_VERSION_NUM = 110_000
|
6
6
|
|
7
|
-
%i[change_column_null change_column
|
7
|
+
%i[change_column_null change_column].each do |method|
|
8
8
|
define_method method do |*args, &block|
|
9
9
|
with_setting(:statement_timeout, SafePgMigrations.config.pg_safe_timeout) { super(*args, &block) }
|
10
10
|
end
|
11
|
+
ruby2_keywords method
|
11
12
|
end
|
12
13
|
|
13
|
-
def add_column(table_name, column_name, type,
|
14
|
-
|
14
|
+
ruby2_keywords def add_column(table_name, column_name, type, *args) # rubocop:disable Metrics/CyclomaticComplexity
|
15
|
+
options = args.last.is_a?(Hash) ? args.last : {}
|
16
|
+
return super if SafePgMigrations.pg_version_num >= PG_11_VERSION_NUM
|
15
17
|
|
16
|
-
default = options.delete(:default)
|
18
|
+
default = options.delete(:default)
|
17
19
|
null = options.delete(:null)
|
18
20
|
|
19
21
|
if !default.nil? || null == false
|
@@ -22,7 +24,7 @@ module SafePgMigrations
|
|
22
24
|
|
23
25
|
super
|
24
26
|
|
25
|
-
|
27
|
+
unless default.nil?
|
26
28
|
SafePgMigrations.say_method_call(:change_column_default, table_name, column_name, default)
|
27
29
|
change_column_default(table_name, column_name, default)
|
28
30
|
|
@@ -36,29 +38,50 @@ module SafePgMigrations
|
|
36
38
|
end
|
37
39
|
end
|
38
40
|
|
39
|
-
def add_foreign_key(from_table, to_table,
|
41
|
+
ruby2_keywords def add_foreign_key(from_table, to_table, *args)
|
42
|
+
options = args.last.is_a?(Hash) ? args.last : {}
|
40
43
|
validate_present = options.key? :validate
|
41
44
|
options[:validate] = false unless validate_present
|
45
|
+
with_setting(:statement_timeout, SafePgMigrations.config.pg_safe_timeout) do
|
46
|
+
super(from_table, to_table, **options)
|
47
|
+
end
|
48
|
+
|
49
|
+
return if validate_present
|
42
50
|
|
43
|
-
|
51
|
+
suboptions = options.slice(:name, :column)
|
52
|
+
without_statement_timeout { validate_foreign_key from_table, suboptions.present? ? nil : to_table, **suboptions }
|
53
|
+
end
|
44
54
|
|
45
|
-
|
46
|
-
|
55
|
+
ruby2_keywords def create_table(*)
|
56
|
+
with_setting(:statement_timeout, SafePgMigrations.config.pg_safe_timeout) do
|
57
|
+
super do |td|
|
58
|
+
yield td if block_given?
|
59
|
+
td.indexes.map! do |key, index_options|
|
60
|
+
index_options[:algorithm] ||= :default
|
61
|
+
[key, index_options]
|
62
|
+
end
|
63
|
+
end
|
64
|
+
end
|
47
65
|
end
|
48
66
|
|
49
67
|
def add_index(table_name, column_name, **options)
|
50
|
-
options[:algorithm]
|
51
|
-
|
68
|
+
if options[:algorithm] == :default
|
69
|
+
options.delete :algorithm
|
70
|
+
else
|
71
|
+
options[:algorithm] = :concurrently
|
72
|
+
end
|
52
73
|
|
53
|
-
|
74
|
+
SafePgMigrations.say_method_call(:add_index, table_name, column_name, **options)
|
75
|
+
|
76
|
+
without_timeout { super(table_name, column_name, **options) }
|
54
77
|
end
|
55
78
|
|
56
|
-
def remove_index(table_name,
|
57
|
-
options = { column:
|
58
|
-
options[:algorithm] = :concurrently
|
59
|
-
SafePgMigrations.say_method_call(:remove_index, table_name, options)
|
79
|
+
ruby2_keywords def remove_index(table_name, *args)
|
80
|
+
options = args.last.is_a?(Hash) ? args.last : { column: args.last }
|
81
|
+
options[:algorithm] = :concurrently unless options.key?(:algorithm)
|
82
|
+
SafePgMigrations.say_method_call(:remove_index, table_name, **options)
|
60
83
|
|
61
|
-
|
84
|
+
without_timeout { super(table_name, **options) }
|
62
85
|
end
|
63
86
|
|
64
87
|
def backfill_column_default(table_name, column_name)
|
@@ -99,12 +122,12 @@ module SafePgMigrations
|
|
99
122
|
with_setting(:statement_timeout, 0) { yield }
|
100
123
|
end
|
101
124
|
|
102
|
-
def
|
103
|
-
|
104
|
-
|
105
|
-
|
106
|
-
|
107
|
-
|
125
|
+
def without_lock_timeout
|
126
|
+
with_setting(:lock_timeout, 0) { yield }
|
127
|
+
end
|
128
|
+
|
129
|
+
def without_timeout
|
130
|
+
without_statement_timeout { without_lock_timeout { yield } }
|
108
131
|
end
|
109
132
|
end
|
110
133
|
end
|
@@ -3,14 +3,14 @@
|
|
3
3
|
module SafePgMigrations
|
4
4
|
module StatementRetrier
|
5
5
|
RETRIABLE_SCHEMA_STATEMENTS = %i[
|
6
|
-
add_column add_foreign_key remove_foreign_key change_column_default
|
7
|
-
change_column_null add_index remove_index remove_column
|
6
|
+
add_column add_foreign_key remove_foreign_key change_column_default change_column_null remove_column
|
8
7
|
].freeze
|
9
8
|
|
10
9
|
RETRIABLE_SCHEMA_STATEMENTS.each do |method|
|
11
10
|
define_method method do |*args, &block|
|
12
11
|
retry_if_lock_timeout { super(*args, &block) }
|
13
12
|
end
|
13
|
+
ruby2_keywords method
|
14
14
|
end
|
15
15
|
|
16
16
|
private
|
@@ -0,0 +1,38 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module SafePgMigrations
|
4
|
+
module UselessStatementsLogger
|
5
|
+
class << self
|
6
|
+
ruby2_keywords def warn_useless(action, link = nil, *args)
|
7
|
+
SafePgMigrations.say "/!\\ No need to explicitly use #{action}, safe-pg-migrations does it for you", *args
|
8
|
+
SafePgMigrations.say "\t see #{link} for more details", *args if link
|
9
|
+
end
|
10
|
+
end
|
11
|
+
|
12
|
+
ruby2_keywords def add_index(*args)
|
13
|
+
options = args.last.is_a?(Hash) ? args.last : {}
|
14
|
+
warn_for_index(**options)
|
15
|
+
super
|
16
|
+
end
|
17
|
+
|
18
|
+
ruby2_keywords def remove_index(table_name, *args)
|
19
|
+
options = args.last.is_a?(Hash) ? args.last : {}
|
20
|
+
warn_for_index(**options) unless options.empty?
|
21
|
+
super
|
22
|
+
end
|
23
|
+
|
24
|
+
ruby2_keywords def add_foreign_key(*args)
|
25
|
+
options = args.last.is_a?(Hash) ? args.last : {}
|
26
|
+
if options[:validate] == false
|
27
|
+
UselessStatementsLogger.warn_useless '`validate: :false`', 'https://github.com/doctolib/safe-pg-migrations#safe_add_foreign_key'
|
28
|
+
end
|
29
|
+
super
|
30
|
+
end
|
31
|
+
|
32
|
+
def warn_for_index(**options)
|
33
|
+
return unless options[:algorithm] == :concurrently
|
34
|
+
|
35
|
+
UselessStatementsLogger.warn_useless '`algorithm: :concurrently`', 'https://github.com/doctolib/safe-pg-migrations#safe_add_remove_index'
|
36
|
+
end
|
37
|
+
end
|
38
|
+
end
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: safe-pg-migrations
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.3.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Matthieu Prat
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date:
|
12
|
+
date: 2021-06-28 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: activerecord
|
@@ -39,6 +39,20 @@ dependencies:
|
|
39
39
|
- - ">="
|
40
40
|
- !ruby/object:Gem::Version
|
41
41
|
version: '5.2'
|
42
|
+
- !ruby/object:Gem::Dependency
|
43
|
+
name: ruby2_keywords
|
44
|
+
requirement: !ruby/object:Gem::Requirement
|
45
|
+
requirements:
|
46
|
+
- - ">="
|
47
|
+
- !ruby/object:Gem::Version
|
48
|
+
version: 0.0.4
|
49
|
+
type: :runtime
|
50
|
+
prerelease: false
|
51
|
+
version_requirements: !ruby/object:Gem::Requirement
|
52
|
+
requirements:
|
53
|
+
- - ">="
|
54
|
+
- !ruby/object:Gem::Version
|
55
|
+
version: 0.0.4
|
42
56
|
- !ruby/object:Gem::Dependency
|
43
57
|
name: bundler
|
44
58
|
requirement: !ruby/object:Gem::Requirement
|
@@ -166,6 +180,7 @@ files:
|
|
166
180
|
- lib/safe-pg-migrations/plugins/idem_potent_statements.rb
|
167
181
|
- lib/safe-pg-migrations/plugins/statement_insurer.rb
|
168
182
|
- lib/safe-pg-migrations/plugins/statement_retrier.rb
|
183
|
+
- lib/safe-pg-migrations/plugins/useless_statements_logger.rb
|
169
184
|
- lib/safe-pg-migrations/plugins/verbose_sql_logger.rb
|
170
185
|
- lib/safe-pg-migrations/railtie.rb
|
171
186
|
- lib/safe-pg-migrations/version.rb
|
@@ -182,6 +197,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
182
197
|
- - ">="
|
183
198
|
- !ruby/object:Gem::Version
|
184
199
|
version: '2.5'
|
200
|
+
- - "<"
|
201
|
+
- !ruby/object:Gem::Version
|
202
|
+
version: '4'
|
185
203
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
186
204
|
requirements:
|
187
205
|
- - ">="
|