rails-pg-adapter 0.1.4 → 0.1.6
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +9 -1
- data/CODE_OF_CONDUCT.md +11 -11
- data/README.md +30 -12
- data/lib/rails_pg_adapter/configuration.rb +8 -1
- data/lib/rails_pg_adapter/patch.rb +66 -3
- data/lib/rails_pg_adapter/version.rb +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 8492318d406e356186bb5d0830087f1e3c21eeeefbc806b0881680a026c3e85b
|
4
|
+
data.tar.gz: 52d95814d9f0df186c464932e4a81f30ff6d2aabf8715740b64d4973877a7681
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: f51318474f857c6733e86f7cb6a19d7d2e7d84cd4de5125b780e1a44a7ff8bdb6325e9dd934c197b47566bf6d359454e0fc0761563b64e5629d752fa2886f10c
|
7
|
+
data.tar.gz: 7a98ab05cc5d99a1b26244ea000634fd4efdb8d855a1c81c101342e6aa1149ecac01fd5c8131837ff3ae0ae9d75e1c98bdbed2974be7c38e9810643b22580d8b
|
data/CHANGELOG.md
CHANGED
@@ -1,4 +1,12 @@
|
|
1
|
-
## [0.1.
|
1
|
+
## [0.1.6] - 2023-04-19
|
2
|
+
|
3
|
+
- Disconnect and remove connection when in read-only
|
4
|
+
|
5
|
+
## [0.1.5] - 2023-04-19
|
6
|
+
|
7
|
+
- Retry queries when not in transaction
|
8
|
+
|
9
|
+
## [0.1.4] - 2023-04-04
|
2
10
|
|
3
11
|
- Rescue and recover from `ActiveRecord::ConnectionNotEstablished`
|
4
12
|
|
data/CODE_OF_CONDUCT.md
CHANGED
@@ -10,21 +10,21 @@ We pledge to act and interact in ways that contribute to an open, welcoming, div
|
|
10
10
|
|
11
11
|
Examples of behavior that contributes to a positive environment for our community include:
|
12
12
|
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
13
|
+
- Demonstrating empathy and kindness toward other people
|
14
|
+
- Being respectful of differing opinions, viewpoints, and experiences
|
15
|
+
- Giving and gracefully accepting constructive feedback
|
16
|
+
- Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience
|
17
|
+
- Focusing on what is best not just for us as individuals, but for the overall community
|
18
18
|
|
19
19
|
Examples of unacceptable behavior include:
|
20
20
|
|
21
|
-
|
21
|
+
- The use of sexualized language or imagery, and sexual attention or
|
22
22
|
advances of any kind
|
23
|
-
|
24
|
-
|
25
|
-
|
23
|
+
- Trolling, insulting or derogatory comments, and personal or political attacks
|
24
|
+
- Public or private harassment
|
25
|
+
- Publishing others' private information, such as a physical or email
|
26
26
|
address, without their explicit permission
|
27
|
-
|
27
|
+
- Other conduct which could reasonably be considered inappropriate in a
|
28
28
|
professional setting
|
29
29
|
|
30
30
|
## Enforcement Responsibilities
|
@@ -67,7 +67,7 @@ Community leaders will follow these Community Impact Guidelines in determining t
|
|
67
67
|
|
68
68
|
### 4. Permanent Ban
|
69
69
|
|
70
|
-
**Community Impact**: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior,
|
70
|
+
**Community Impact**: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals.
|
71
71
|
|
72
72
|
**Consequence**: A permanent ban from any sort of public interaction within the community.
|
73
73
|
|
data/README.md
CHANGED
@@ -2,16 +2,6 @@
|
|
2
2
|
|
3
3
|
This project allows you to monkey patch `ActiveRecord` (PostgreSQL) and auto-heal applications in production when PostgreSQL database fails over or when a cached column (in `ActiveRecord` schema cache) is removed from the database from a migration in another process.
|
4
4
|
|
5
|
-
## How does it work
|
6
|
-
|
7
|
-
During a database failover in production, the `ActiveRecord` connection pool can become exhausted as queries are made against the database during the failover process. This can leave the `ActiveRecord` connection pools with stale or bad connections, even after the database has successfully recovered. Recovering from this issue usually requires a rolling restart of the application processes or containers.
|
8
|
-
|
9
|
-
`RailsPgAdapter` addresses this problem by resetting the connection pool and re-raises the original exception from an `ActiveRecord` monkey patch. This allows the application to auto-heal from stale connections on its own (after database recovery) when performing queries for a new request, without requiring manual intervention.
|
10
|
-
|
11
|
-
Another issue with `ActiveRecord` queries is `PG::UndefinedColumn`, which occurs when an `ActiveRecord` model includes a `SELECT` query with the name of a column that has been dropped from a Rails migration. This can happen even if the column isn't being referenced anywhere in the code. It occurs when a model is using `ignored_columns`, which prompts `ActiveRecord` to perform a dedicated lookup of the allowed columns in a select, such as `SELECT "users".name, "users".template_id...."`, instead of `SELECT "users".*`. When a column like `template_id` is dropped, PostgreSQL throws an undefined column error, which is bubbled up by `ActiveRecord` into `PG::UndefinedColumn`. Recovering from this issue also usually requires a rolling restart of the application processes or containers.
|
12
|
-
|
13
|
-
`RailsPgAdapter` solves this second issue by resetting the `ActiveRecord` schema cache and memoized model column information when it detects a `PG::UndefinedColumn` raised from a monkey patch. Resetting the column information forces `ActiveRecord` to refresh its schema cache by loading the table information from the database and no longer reference the dropped column for new queries, without requiring manual intervention.
|
14
|
-
|
15
5
|
## Installation
|
16
6
|
|
17
7
|
Install the gem and add to the application's Gemfile by executing:
|
@@ -27,7 +17,7 @@ If bundler is not being used to manage dependencies, install the gem by executin
|
|
27
17
|
### Auto healing connections when PostgreSQL database fails over
|
28
18
|
|
29
19
|
```ruby
|
30
|
-
# config/
|
20
|
+
# config/initializers/rails_pg_adapter.rb
|
31
21
|
|
32
22
|
RailsPgAdapter.configure do |c|
|
33
23
|
c.add_failover_patch = true
|
@@ -36,10 +26,28 @@ end
|
|
36
26
|
|
37
27
|
This will add the monkey patch which resets the `ActiveRecord` connections in the connection pool when the database fails over. The patch will reset the connection and re-raise the error each time it detects that an exception related to a database failover is detected.
|
38
28
|
|
29
|
+
### Retrying queries
|
30
|
+
|
31
|
+
When the database is failing you can retry queries that are not in a transaction. The gem will perform a back off retry in establishing the connection.
|
32
|
+
Once the back off is reached and no connection is found, it will bubble up the exception. Otherwise, the query will be retried with a new connection.
|
33
|
+
|
34
|
+
It is an opt-in functionality. You can supply your own back off figures for retries (in seconds) as following:
|
35
|
+
|
36
|
+
```ruby
|
37
|
+
# config/initializers/rails_pg_adapter.rb
|
38
|
+
|
39
|
+
RailsPgAdapter.configure do |c|
|
40
|
+
c.add_failover_patch = true
|
41
|
+
c.reconnect_with_backoff = [0.5, 1, 2, 4, 8, 16] # seconds
|
42
|
+
...
|
43
|
+
end
|
44
|
+
|
45
|
+
```
|
46
|
+
|
39
47
|
### Refresh model column information on the fly after an existing column is dropped
|
40
48
|
|
41
49
|
```ruby
|
42
|
-
# config/
|
50
|
+
# config/initializers/rails_pg_adapter.rb
|
43
51
|
|
44
52
|
RailsPgAdapter.configure do |c|
|
45
53
|
c.add_reset_column_information_patch = true
|
@@ -48,6 +56,16 @@ end
|
|
48
56
|
|
49
57
|
This will clear the `ActiveRecord` schema cache and reset the `ActiveRecord` column information memoized on the model. The patch will reset the relevant information and re-raise the error each time it detects that an exception related to a dropped column is raised.
|
50
58
|
|
59
|
+
## How does it work
|
60
|
+
|
61
|
+
During a database failover in production, the `ActiveRecord` connection pool can become exhausted as queries are made against the database during the failover process. This can leave the `ActiveRecord` connection pools with stale or bad connections, even after the database has successfully recovered. Recovering from this issue usually requires a rolling restart of the application processes or containers.
|
62
|
+
|
63
|
+
`RailsPgAdapter` addresses this problem by resetting the connection pool and re-raises the original exception from an `ActiveRecord` monkey patch. This allows the application to auto-heal from stale connections on its own (after database recovery) when performing queries for a new request, without requiring manual intervention.
|
64
|
+
|
65
|
+
Another issue with `ActiveRecord` queries is `PG::UndefinedColumn`, which occurs when an `ActiveRecord` model includes a `SELECT` query with the name of a column that has been dropped from a Rails migration. This can happen even if the column isn't being referenced anywhere in the code. It occurs when a model is using `ignored_columns`, which prompts `ActiveRecord` to perform a dedicated lookup of the allowed columns in a select, such as `SELECT "users".name, "users".template_id...."`, instead of `SELECT "users".*`. When a column like `template_id` is dropped, PostgreSQL throws an undefined column error, which is bubbled up by `ActiveRecord` into `PG::UndefinedColumn`. Recovering from this issue also usually requires a rolling restart of the application processes or containers.
|
66
|
+
|
67
|
+
`RailsPgAdapter` solves this second issue by resetting the `ActiveRecord` schema cache and memoized model column information when it detects a `PG::UndefinedColumn` raised from a monkey patch. Resetting the column information forces `ActiveRecord` to refresh its schema cache by loading the table information from the database and no longer reference the dropped column for new queries, without requiring manual intervention.
|
68
|
+
|
51
69
|
## Development
|
52
70
|
|
53
71
|
- Install ruby 3.0
|
@@ -2,11 +2,12 @@
|
|
2
2
|
|
3
3
|
module RailsPgAdapter
|
4
4
|
class Configuration
|
5
|
-
attr_accessor :add_failover_patch, :add_reset_column_information_patch
|
5
|
+
attr_accessor :add_failover_patch, :add_reset_column_information_patch, :reconnect_with_backoff
|
6
6
|
|
7
7
|
def initialize(attrs)
|
8
8
|
self.add_failover_patch = attrs[:add_failover_patch]
|
9
9
|
self.add_reset_column_information_patch = attrs[:add_reset_column_information_patch]
|
10
|
+
self.reconnect_with_backoff = attrs[:reconnect_with_backoff]
|
10
11
|
end
|
11
12
|
end
|
12
13
|
|
@@ -14,6 +15,7 @@ module RailsPgAdapter
|
|
14
15
|
@configuration ||= Configuration.new({
|
15
16
|
add_failover_patch: false,
|
16
17
|
add_reset_column_information_patch: false,
|
18
|
+
reconnect_with_backoff: [],
|
17
19
|
})
|
18
20
|
end
|
19
21
|
|
@@ -25,6 +27,10 @@ module RailsPgAdapter
|
|
25
27
|
RailsPgAdapter.configuration.add_failover_patch || false
|
26
28
|
end
|
27
29
|
|
30
|
+
def self.reconnect_with_backoff?
|
31
|
+
!RailsPgAdapter.configuration.reconnect_with_backoff.empty?
|
32
|
+
end
|
33
|
+
|
28
34
|
def self.reset_column_information_patch?
|
29
35
|
RailsPgAdapter.configuration.add_reset_column_information_patch || false
|
30
36
|
end
|
@@ -33,6 +39,7 @@ module RailsPgAdapter
|
|
33
39
|
@configuration = Configuration.new({
|
34
40
|
add_failover_patch: false,
|
35
41
|
add_reset_column_information_patch: false,
|
42
|
+
reconnect_with_backoff: [],
|
36
43
|
})
|
37
44
|
end
|
38
45
|
end
|
@@ -12,31 +12,56 @@ module RailsPgAdapter
|
|
12
12
|
"PG::ConnectionBad",
|
13
13
|
"the database system is starting up",
|
14
14
|
"connection is closed",
|
15
|
+
"could not connect",
|
15
16
|
].freeze
|
16
17
|
CONNECTION_ERROR_RE = /#{CONNECTION_ERROR.map { |w| Regexp.escape(w) }.join("|")}/.freeze
|
17
18
|
|
18
19
|
CONNECTION_SCHEMA_ERROR = ["PG::UndefinedColumn"].freeze
|
19
20
|
CONNECTION_SCHEMA_RE = /#{CONNECTION_SCHEMA_ERROR.map { |w| Regexp.escape(w) }.join("|")}/.freeze
|
20
21
|
|
22
|
+
CONNECTION_READ_ONLY_ERROR = [
|
23
|
+
"read-only",
|
24
|
+
"PG::ReadOnlySqlTransaction",
|
25
|
+
].freeze
|
26
|
+
CONNECTION_READ_ONLY_ERROR_RE = /#{CONNECTION_READ_ONLY_ERROR.map { |w| Regexp.escape(w) }.join("|")}/.freeze
|
27
|
+
|
21
28
|
private
|
22
29
|
|
23
30
|
def exec_cache(*args)
|
24
31
|
super(*args)
|
25
32
|
rescue ::ActiveRecord::StatementInvalid, ::ActiveRecord::ConnectionNotEstablished => e
|
26
|
-
|
33
|
+
raise unless supported_errors?(e)
|
34
|
+
|
35
|
+
try_reconnect?(e) ? retry : handle_error(e)
|
27
36
|
end
|
28
37
|
|
29
38
|
def exec_no_cache(*args)
|
30
39
|
super(*args)
|
31
40
|
rescue ::ActiveRecord::StatementInvalid, ::ActiveRecord::ConnectionNotEstablished => e
|
32
|
-
|
41
|
+
raise unless supported_errors?(e)
|
42
|
+
|
43
|
+
try_reconnect?(e) ? retry : handle_error(e)
|
44
|
+
end
|
45
|
+
|
46
|
+
def try_reconnect?(e)
|
47
|
+
return false if in_transaction?
|
48
|
+
return false unless failover_error?(e.message)
|
49
|
+
return false unless RailsPgAdapter.reconnect_with_backoff?
|
50
|
+
|
51
|
+
begin
|
52
|
+
disconnect_and_remove_conn! if read_only_error?(e.message)
|
53
|
+
reconnect!
|
54
|
+
true
|
55
|
+
rescue ::ActiveRecord::ConnectionNotEstablished
|
56
|
+
false
|
57
|
+
end
|
33
58
|
end
|
34
59
|
|
35
60
|
def handle_error(e)
|
36
61
|
if failover_error?(e.message) && RailsPgAdapter.failover_patch?
|
37
62
|
warn("clearing connections due to #{e} - #{e.message}")
|
38
63
|
disconnect_and_remove_conn!
|
39
|
-
raise
|
64
|
+
raise(e)
|
40
65
|
end
|
41
66
|
|
42
67
|
return unless missing_column_error?(e.message) && RailsPgAdapter.reset_column_information_patch?
|
@@ -70,7 +95,45 @@ module RailsPgAdapter
|
|
70
95
|
return if Rails.logger.nil?
|
71
96
|
::Rails.logger.warn("[RailsPgAdapter::Patch] #{msg}")
|
72
97
|
end
|
98
|
+
|
99
|
+
def supported_errors?(e)
|
100
|
+
return true if failover_error?(e.message) && RailsPgAdapter.failover_patch?
|
101
|
+
if missing_column_error?(e.message) && RailsPgAdapter.reset_column_information_patch?
|
102
|
+
return true
|
103
|
+
end
|
104
|
+
false
|
105
|
+
end
|
106
|
+
|
107
|
+
def read_only_error?(error_message)
|
108
|
+
CONNECTION_READ_ONLY_ERROR_RE.match?(error_message)
|
109
|
+
end
|
73
110
|
end
|
74
111
|
end
|
75
112
|
|
76
113
|
ActiveRecord::ConnectionAdapters::PostgreSQLAdapter.prepend(RailsPgAdapter::Patch)
|
114
|
+
|
115
|
+
# Override new client connection to bake in retries
|
116
|
+
module ActiveRecord
|
117
|
+
module ConnectionAdapters
|
118
|
+
class PostgreSQLAdapter
|
119
|
+
class << self
|
120
|
+
old_new_client_method = instance_method(:new_client)
|
121
|
+
|
122
|
+
define_method(:new_client) do |args|
|
123
|
+
sleep_times = RailsPgAdapter.configuration.reconnect_with_backoff.dup
|
124
|
+
begin
|
125
|
+
old_new_client_method.bind(self).call(args)
|
126
|
+
rescue ::ActiveRecord::ConnectionNotEstablished => e
|
127
|
+
raise(e) unless RailsPgAdapter.failover_patch? && RailsPgAdapter.reconnect_with_backoff?
|
128
|
+
|
129
|
+
sleep_time = sleep_times.shift
|
130
|
+
raise unless sleep_time
|
131
|
+
warn( "Could not establish a connection from new_client, retrying again in #{sleep_time} sec.")
|
132
|
+
sleep(sleep_time)
|
133
|
+
retry
|
134
|
+
end
|
135
|
+
end
|
136
|
+
end
|
137
|
+
end
|
138
|
+
end
|
139
|
+
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: rails-pg-adapter
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.1.
|
4
|
+
version: 0.1.6
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Tines Engineering
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2023-04-
|
11
|
+
date: 2023-04-19 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: rails
|