pgmq-ruby 0.6.0 → 0.6.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +15 -0
- data/README.md +35 -1
- data/lib/pgmq/connection.rb +151 -20
- data/lib/pgmq/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: bad592ec19aca0b0437ef08ef6cbcb8ecdb2f9155ddbc27196ef00e8b10b22ce
|
|
4
|
+
data.tar.gz: 8220e5b37d111f10afb169100c7a7d45231a19f7b6b31880b200a744abb427b5
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: c9236e3bba26ee065dcc21d87f945ff2a71439695202322fef2e1c87502b91e655e73d7cc498084109b75023073e56bcca060bf4f4da45fd403bfaa25c802049
|
|
7
|
+
data.tar.gz: f9c94727f9839d0cffad5928147015e90801ec41eeb3e32b4577fb8743da9891d5e2dc9cce622d3308853ac44abd15076d085165f56e6785c420752a2e8d56dd
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,20 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## Unreleased
|
|
4
|
+
|
|
5
|
+
## 0.6.2 (2026-05-08)
|
|
6
|
+
|
|
7
|
+
### Connection Management
|
|
8
|
+
- **[Feature]** Add class-level configuration for extending reconnectable error detection. `PGMQ::Connection.reconnectable_error_patterns` accepts additional `String` or `Regexp` patterns (strings matched as case-insensitive substrings; regexps against the original message). `PGMQ::Connection.reconnectable_error_classes` accepts additional `Exception` subclasses. This allows users to adapt to new pg gem / PostgreSQL / pooler disconnect signatures without waiting for a gem release.
|
|
9
|
+
- **[Fix]** `PGMQ::Connection#connection_lost_error?` now detects SSL-layer teardown errors (`"PQconsumeInput() SSL error: unexpected eof while reading"`, `"SSL SYSCALL error: EOF detected"`). Observed in production behind a managed Postgres pooler: an idle connection torn down at the SSL layer caused the next enqueue to raise `PGMQ::Errors::ConnectionError` without triggering `with_connection`'s single-retry path, because the message didn't match any of the existing substrings.
|
|
10
|
+
- **[Fix]** `PGMQ::Connection#connection_lost_error?` now also matches by class (`PG::ConnectionBad`, `PG::UnableToSend`) in addition to message substrings. These are dedicated connection-failure classes libpq raises when the socket is dead; class-matching catches future OS/pooler/TLS message variants without waiting for them to hit production.
|
|
11
|
+
|
|
12
|
+
## 0.6.1 (2026-04-16)
|
|
13
|
+
|
|
14
|
+
### Connection Management
|
|
15
|
+
- **[Fix]** `PGMQ::Connection#verify_connection!` now also resets connections whose status is `PG::CONNECTION_BAD`. Previously it only checked `conn.finished?`, which misses connections closed server-side by the database or an intermediate pooler (PgBouncer `server_idle_timeout` / `client_idle_timeout`, admin kill, TCP RST). A pooled connection with a dead socket would survive `verify_connection!` and fail on the next operation with `PQsocket() can't get socket descriptor`.
|
|
16
|
+
- **[Fix]** `PGMQ::Connection#connection_lost_error?` now recognises `"PQsocket() can't get socket descriptor"` (plus `"connection is closed"` / `"connection has been closed"`). This is the exact error the `pg` gem raises from its C extension when the cached libpq socket FD is gone. Without the match, the `auto_reconnect` retry path in `with_connection` skipped this error and producers failed on the first call following a server-side close.
|
|
17
|
+
|
|
3
18
|
## 0.6.0 (2026-04-02)
|
|
4
19
|
|
|
5
20
|
### Breaking Changes
|
data/README.md
CHANGED
|
@@ -290,10 +290,44 @@ client = PGMQ::Client.new(
|
|
|
290
290
|
)
|
|
291
291
|
```
|
|
292
292
|
|
|
293
|
+
#### Extending the lost-connection error matchers
|
|
294
|
+
|
|
295
|
+
PGMQ-Ruby ships with a curated list of `PG::Error` messages and classes
|
|
296
|
+
(`PG::ConnectionBad`, `PG::UnableToSend`) that trigger the auto-reconnect
|
|
297
|
+
retry. Different `pg` gem versions, PostgreSQL versions, and connection
|
|
298
|
+
poolers (PgBouncer, Supabase, RDS Proxy, etc.) occasionally surface new
|
|
299
|
+
disconnect signatures. Rather than wait for an upstream patch, you can
|
|
300
|
+
extend the matchers at boot time via class-level configuration:
|
|
301
|
+
|
|
302
|
+
```ruby
|
|
303
|
+
# In an initializer (e.g. config/initializers/pgmq.rb for Rails apps)
|
|
304
|
+
# Strings are matched as case-insensitive substrings against the error
|
|
305
|
+
# message; Regexps are matched against the original message.
|
|
306
|
+
PGMQ::Connection.reconnectable_error_patterns = [
|
|
307
|
+
"connection reset by peer",
|
|
308
|
+
/\Abroken pipe\b/i
|
|
309
|
+
]
|
|
310
|
+
|
|
311
|
+
# Any Exception subclass is accepted. Subclasses also match.
|
|
312
|
+
PGMQ::Connection.reconnectable_error_classes = [PG::ConnectionRefused]
|
|
313
|
+
```
|
|
314
|
+
|
|
315
|
+
The built-in defaults are always kept — your patterns and classes are
|
|
316
|
+
appended to them. Configuration errors (e.g. passing an Integer as a
|
|
317
|
+
pattern) raise `PGMQ::Errors::ConfigurationError` immediately so
|
|
318
|
+
misconfiguration can't silently disable retries.
|
|
319
|
+
|
|
320
|
+
> **Reserve these options for connection-level failures.** A "reconnectable"
|
|
321
|
+
> error means the socket is dead and a retry on a fresh connection is safe.
|
|
322
|
+
> Do **not** add patterns or classes for query-level errors (deadlocks,
|
|
323
|
+
> constraint violations, statement timeouts) — those will replay your
|
|
324
|
+
> operation against a healthy connection and may cause duplicate work or
|
|
325
|
+
> mask bugs.
|
|
326
|
+
|
|
293
327
|
**Connection Pool Benefits:**
|
|
294
328
|
- **Thread-safe** - Multiple threads can safely share a single client
|
|
295
329
|
- **Fiber-aware** - Works with Ruby 3.0+ Fiber Scheduler for non-blocking I/O (tested with the `async` gem)
|
|
296
|
-
- **Auto-reconnect** - Recovers from lost connections (configurable)
|
|
330
|
+
- **Auto-reconnect** - Recovers from lost connections (configurable, extendable)
|
|
297
331
|
- **Health checks** - Verifies connections before use to prevent stale connection errors
|
|
298
332
|
- **Monitoring** - Track pool utilization with `client.stats`
|
|
299
333
|
|
data/lib/pgmq/connection.rb
CHANGED
|
@@ -26,6 +26,97 @@ module PGMQ
|
|
|
26
26
|
# Default connection pool timeout in seconds
|
|
27
27
|
DEFAULT_POOL_TIMEOUT = 5
|
|
28
28
|
|
|
29
|
+
class << self
|
|
30
|
+
# Additional error message patterns (String or Regexp) that mean the
|
|
31
|
+
# connection is dead and a retry on a fresh socket is safe. Strings are
|
|
32
|
+
# matched as case-insensitive substrings; Regexps match the original
|
|
33
|
+
# message. The built-in LOST_CONNECTION_MESSAGES are always checked
|
|
34
|
+
# first — this list is appended to them.
|
|
35
|
+
#
|
|
36
|
+
# Thread-safe: reads are lock-free (frozen array swap); writes should
|
|
37
|
+
# be done at boot time before forking workers.
|
|
38
|
+
#
|
|
39
|
+
# @return [Array<String, Regexp>]
|
|
40
|
+
# @example
|
|
41
|
+
# PGMQ::Connection.reconnectable_error_patterns << "connection reset by peer"
|
|
42
|
+
# PGMQ::Connection.reconnectable_error_patterns << /\Abroken pipe\b/i
|
|
43
|
+
attr_reader :reconnectable_error_patterns
|
|
44
|
+
|
|
45
|
+
# Additional exception classes that mean the connection is dead.
|
|
46
|
+
# `PG::ConnectionBad` and `PG::UnableToSend` are always matched — this
|
|
47
|
+
# list is appended to them. Subclasses also match.
|
|
48
|
+
#
|
|
49
|
+
# Thread-safe: reads are lock-free; writes should be done at boot time.
|
|
50
|
+
#
|
|
51
|
+
# @return [Array<Class>]
|
|
52
|
+
# @example
|
|
53
|
+
# PGMQ::Connection.reconnectable_error_classes << PG::ConnectionRefused
|
|
54
|
+
attr_reader :reconnectable_error_classes
|
|
55
|
+
|
|
56
|
+
# Replaces the extra reconnectable error patterns.
|
|
57
|
+
#
|
|
58
|
+
# @param patterns [Array<String, Regexp>]
|
|
59
|
+
# @raise [PGMQ::Errors::ConfigurationError] if any element is invalid
|
|
60
|
+
def reconnectable_error_patterns=(patterns)
|
|
61
|
+
@reconnectable_error_patterns = normalize_patterns(patterns)
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
# Replaces the extra reconnectable error classes.
|
|
65
|
+
#
|
|
66
|
+
# @param classes [Array<Class>]
|
|
67
|
+
# @raise [PGMQ::Errors::ConfigurationError] if any element is invalid
|
|
68
|
+
def reconnectable_error_classes=(classes)
|
|
69
|
+
@reconnectable_error_classes = normalize_classes(classes)
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
private
|
|
73
|
+
|
|
74
|
+
# Normalizes user-supplied reconnectable error patterns.
|
|
75
|
+
#
|
|
76
|
+
# Strings are downcased once at configuration time so the hot path
|
|
77
|
+
# (`connection_lost_error?`) only does substring checks. Regexps are
|
|
78
|
+
# passed through unchanged.
|
|
79
|
+
#
|
|
80
|
+
# @param patterns [Array<String, Regexp>, String, Regexp, nil]
|
|
81
|
+
# @return [Array<String, Regexp>]
|
|
82
|
+
def normalize_patterns(patterns)
|
|
83
|
+
Array(patterns).map do |pattern|
|
|
84
|
+
case pattern
|
|
85
|
+
when Regexp
|
|
86
|
+
pattern
|
|
87
|
+
when String
|
|
88
|
+
pattern.downcase
|
|
89
|
+
else
|
|
90
|
+
raise(
|
|
91
|
+
PGMQ::Errors::ConfigurationError,
|
|
92
|
+
"reconnectable_error_patterns must contain Strings or Regexps, got #{pattern.class}"
|
|
93
|
+
)
|
|
94
|
+
end
|
|
95
|
+
end.freeze
|
|
96
|
+
end
|
|
97
|
+
|
|
98
|
+
# Normalizes user-supplied reconnectable error classes.
|
|
99
|
+
#
|
|
100
|
+
# @param classes [Array<Class>, Class, nil]
|
|
101
|
+
# @return [Array<Class>]
|
|
102
|
+
def normalize_classes(classes)
|
|
103
|
+
Array(classes).map do |klass|
|
|
104
|
+
unless klass.is_a?(Class) && klass <= Exception
|
|
105
|
+
raise(
|
|
106
|
+
PGMQ::Errors::ConfigurationError,
|
|
107
|
+
"reconnectable_error_classes must contain Exception subclasses, got #{klass.inspect}"
|
|
108
|
+
)
|
|
109
|
+
end
|
|
110
|
+
|
|
111
|
+
klass
|
|
112
|
+
end.freeze
|
|
113
|
+
end
|
|
114
|
+
end
|
|
115
|
+
|
|
116
|
+
# Initialize class-level defaults (empty arrays, users append at boot)
|
|
117
|
+
@reconnectable_error_patterns = [].freeze
|
|
118
|
+
@reconnectable_error_classes = [].freeze
|
|
119
|
+
|
|
29
120
|
# @return [ConnectionPool] the connection pool
|
|
30
121
|
attr_reader :pool
|
|
31
122
|
|
|
@@ -107,33 +198,73 @@ module PGMQ
|
|
|
107
198
|
|
|
108
199
|
private
|
|
109
200
|
|
|
110
|
-
#
|
|
201
|
+
# Messages libpq raises when the server/pooler has already torn down the
|
|
202
|
+
# socket. The list has grown organically with each pooler/TLS variant we
|
|
203
|
+
# see in the wild; the class check below catches future variants that
|
|
204
|
+
# libpq raises as `PG::ConnectionBad` or `PG::UnableToSend` without
|
|
205
|
+
# waiting for a new message to hit production.
|
|
206
|
+
LOST_CONNECTION_MESSAGES = [
|
|
207
|
+
"server closed the connection",
|
|
208
|
+
"connection not open",
|
|
209
|
+
"connection is closed",
|
|
210
|
+
"connection has been closed",
|
|
211
|
+
"no connection to the server",
|
|
212
|
+
"terminating connection",
|
|
213
|
+
"connection to server was lost",
|
|
214
|
+
"could not receive data from server",
|
|
215
|
+
"pqsocket() can't get socket descriptor",
|
|
216
|
+
"ssl error: unexpected eof",
|
|
217
|
+
"ssl syscall error"
|
|
218
|
+
].freeze
|
|
219
|
+
private_constant :LOST_CONNECTION_MESSAGES
|
|
220
|
+
|
|
221
|
+
# Checks if the error indicates a lost connection.
|
|
222
|
+
#
|
|
223
|
+
# Matches in three steps: first by class (`PG::ConnectionBad` /
|
|
224
|
+
# `PG::UnableToSend` are dedicated connection-failure classes libpq
|
|
225
|
+
# raises regardless of message, plus any user-supplied classes), then
|
|
226
|
+
# by built-in message substrings for the bare `PG::Error` cases where
|
|
227
|
+
# libpq doesn't reach for the specific subclass, and finally by
|
|
228
|
+
# user-supplied patterns (strings matched as case-insensitive
|
|
229
|
+
# substrings, Regexps matched against the original message).
|
|
230
|
+
#
|
|
111
231
|
# @param error [PG::Error] the error to check
|
|
112
|
-
# @return [Boolean] true if connection was lost
|
|
232
|
+
# @return [Boolean] true if the connection was lost and a retry on a
|
|
233
|
+
# fresh connection is appropriate
|
|
113
234
|
def connection_lost_error?(error)
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
235
|
+
return true if error.is_a?(PG::ConnectionBad) || error.is_a?(PG::UnableToSend)
|
|
236
|
+
|
|
237
|
+
extra_classes = self.class.reconnectable_error_classes
|
|
238
|
+
return true if extra_classes.any? { |klass| error.is_a?(klass) }
|
|
239
|
+
|
|
240
|
+
original_message = error.message.to_s
|
|
241
|
+
downcased = original_message.downcase
|
|
242
|
+
|
|
243
|
+
return true if LOST_CONNECTION_MESSAGES.any? { |pattern| downcased.include?(pattern) }
|
|
244
|
+
|
|
245
|
+
self.class.reconnectable_error_patterns.any? do |pattern|
|
|
246
|
+
case pattern
|
|
247
|
+
when Regexp then pattern.match?(original_message)
|
|
248
|
+
else downcased.include?(pattern)
|
|
249
|
+
end
|
|
250
|
+
end
|
|
126
251
|
end
|
|
127
252
|
|
|
128
|
-
# Verifies a connection is alive and working
|
|
253
|
+
# Verifies a connection is alive and working.
|
|
254
|
+
#
|
|
255
|
+
# Also resets when the connection reports `PG::CONNECTION_BAD`, which
|
|
256
|
+
# happens when the server (or an intermediate pooler such as PgBouncer)
|
|
257
|
+
# has closed the socket while the client-side `PG::Connection` object
|
|
258
|
+
# still exists. `#finished?` alone only catches connections closed
|
|
259
|
+
# explicitly from the client side.
|
|
260
|
+
#
|
|
129
261
|
# @param conn [PG::Connection] connection to verify
|
|
130
|
-
# @raise [PG::Error] if
|
|
262
|
+
# @raise [PG::Error] if the reset itself fails
|
|
131
263
|
def verify_connection!(conn)
|
|
132
|
-
|
|
133
|
-
return
|
|
264
|
+
return conn.reset if conn.finished?
|
|
265
|
+
return conn.reset if conn.status == PG::CONNECTION_BAD
|
|
134
266
|
|
|
135
|
-
|
|
136
|
-
conn.reset
|
|
267
|
+
nil
|
|
137
268
|
end
|
|
138
269
|
|
|
139
270
|
# Normalizes various connection parameter formats
|
data/lib/pgmq/version.rb
CHANGED