pgbus 0.25.1 → 0.26.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: d2e222e8f30d9e69de08103ecf574c4166a4804d90aff9db01df780451255b0f
4
- data.tar.gz: 8744453ae4652d2dc6e163b81c5104b5e25e62bf6c7dff57c5b4e5a5c5807208
3
+ metadata.gz: 9df17de1381f625e2b64dbab3e9e7beb5e1a9cd9638cf14e00f7ac164517bcc8
4
+ data.tar.gz: 69a9f6b8a1d51023bb390709c1d823932963634dff41228ec459e5a3a8f9fe14
5
5
  SHA512:
6
- metadata.gz: e0f35bcb78c6f707f6adb837f6845c21143543050891bc5dd677d935d571b2dfb93fe6e3d048ceecd2a4d75596c5f811cee2fa6076dba0c51c5b030a1ad527bb
7
- data.tar.gz: c4d933bc1d90cae70c82888f31a6bbc73d83cc1a8e1ca9beccd754c310d03289e4edce4170a11a85f05c6310ecb95ec6a005c50cf9791fc5ab0b594252f0c5a3
6
+ metadata.gz: 5be0b697d8fa0fc2434310b6c45801048dd8e317b12cfdc763c8f6c07378e93d3e9cf3631eeb2c1b4cc68a9536197dd3844cec6a0636b312f407bf0d3f2852a9
7
+ data.tar.gz: 4f725226ec945afdab5b82a64e895b40fd3178e01945fe01b3e63b6a1d34400aa422f5f7622f6872e6b7c24e6bf2338d26786b0e48a11573375e9f45086ee6a8
data/CHANGELOG.md CHANGED
@@ -1,17 +1,27 @@
1
1
  ## [Unreleased]
2
2
 
3
- ### Added
3
+ ### Changed
4
4
 
5
- - **Deploy tasks: `after:` makes a task wait until the tasks it names have completed, and every run records its outcome (issue #660).** `after:` takes a task key or a list of keys declared in the loaded configuration (any of `deploy_tasks_files`). The scheduler keeps such a task armed, and does not enqueue it, until each named task has a run with `completed_at` at its current version (any revision; an `every_deploy` dependency needs one on this revision). A rollover run and a Run now run both count, so a failed dependency that an operator fixes and runs again with Run now unblocks its dependents without a new deploy. A waiting task records no failure and no backoff, and fires on the first tick after its last dependency completes; the check is one query per tick, made only while an armed task has dependencies. At scheduler boot a task whose `after:` names an unknown key (`after: unknown task 'x'`), sits on a cycle (`after: cycle a → b → a`, a self-reference included), or names a task that is itself skipped (`after: 'x' is skipped`) is skipped with a warning. Run outcomes: `Deploy::Runner#enqueue` now creates the `pgbus_deploy_task_runs` row before the enqueue and passes its id in the job payload under `pgbus_deploy_task_run_id` (`Pgbus::ActiveJob::DeployTaskRunId`, included on `ActiveJob::Base` next to `BatchId`; an unmarked job's payload is unchanged), fills in `queue_name` and `msg_id` after it, and deletes the row when the enqueue fails. The executor stamps `completed_at` when `perform` returned having done its work without re-enqueuing the job (a `retry_on` attempt whose retry is queued, or a `discard_on` block that queues one, leaves it to the retry; a job that rescues inside `perform` and returns has completed) and `failed_at` when ActiveJob discarded the job without queuing a retry (`discard_on`, with or without a block, or a `retry_on` that gave up through its block, where one without a block re-raises and ends dead-lettered as before; `Pgbus::ActiveJob::DeployTaskRunId` marks the instance in ActiveJob's `run_after_discard_procs`, after the job's own `after_discard` procs, so a discarded dependency reads `failed` and its dependents keep waiting, issue #662), when the job is dead-lettered, discarded as a `:while_executing` duplicate, or its retry never left (dropped by its own concurrency limit, `on_conflict: :discard`, or aborted by a `before_enqueue` callback); a failed write is reported and never fails the job, and a job without the key pays one Hash lookup. An enqueue the job's own `on_conflict: :discard` / `:log` drops now counts as a failed enqueue (the run is deleted and the task re-armed with backoff). A run row without `queue_name` (its enqueue never finished) does not count as a run, so the version stays owed. The dispatcher's pruning keeps the newest completed run per task and version. Dashboard: a task's state says "Waiting for N dependencies to complete" and lists each one as `not run`, `running` or `failed`; the task page shows `after:` and an Outcome column in the run history; Run now on a waiting task is refused with a flash naming the dependencies (`DataSource#enqueue_deploy_task_now(id, ignore_dependencies: false)` returns `:blocked`) unless "Ignore dependencies" is ticked on the task page. `pgbus_deploy_tasks` (MCP) lists `dependencies` and `waiting_on`. New generator `pgbus:add_deploy_task_run_outcomes` adds `pgbus_deploy_task_runs.completed_at` / `failed_at`, `pgbus_deploy_tasks.dependencies` and an index on `(task_key, completed_at)`; `pgbus:update` queues it, and fresh `pgbus:install` / `pgbus:add_deploy_tasks` include the columns. Until it is migrated, a task with `after:` is skipped with a warning naming the generator and no outcome is written. **Upgrade note:** runs recorded before the migration have no outcome and satisfy nothing, so when an `after:` names a task whose only runs predate the upgrade, Run now that task once (or bump its version; an `every_deploy` dependency needs a run on the current revision, so only Run now helps). Delivery stays at-least-once: a scheduler that dies between claiming a task and enqueuing it leaves the task unarmed until the next scheduler boot re-arms it, and a run whose enqueue could not be recorded runs again after one, so deploy tasks must be idempotent. Refs #660.
5
+ - **A member-keyed presence `touch` no longer probes `information_schema` on a table without the seat columns, and a forced re-check no longer restarts the backoff (issue #666).** On a `pgbus_presence_members` table where `rails g pgbus:add_presence_seats` has not run, every member-keyed `touch`, `leave` and `sweep!` re-probed a cached "no seats" answer before it ran and restarted the #665 backoff at 30 s. The streamer touches once per connection per heartbeat, so a process with N member-keyed connections sent N catalog queries every `streams_heartbeat_interval` (15 s by default), and its backoff never took effect. A member-keyed `touch` now uses the cached answer and re-checks only on the backoff's schedule: three touches after a reset send one probe instead of four, and none while a "no" is cached. Assumed: on a table migrated under a running process, such a `touch` can also refresh that member's seat rows until the next re-check (at most 10 minutes, `SCHEMA_RECHECK_MAX_SECONDS`); it is an `UPDATE` of `last_seen_at`, so nothing is deleted. A member-keyed `leave` and `sweep!` still re-check a cached "no" before they delete, so they never delete seat rows on a migrated table (#515), now with one probe instead of up to two, answered from that probe. That re-check leaves the backoff alone: over a fresh "no" it changes neither the step nor the schedule however many disconnects force one, and over a "no" already due it is the timed re-check, one step. `reset_schema_cache!` still re-probes at once and starts again from 30 s. Refs #666.
6
+
7
+ ## [0.25.1] - 2026-11-15
6
8
 
7
9
  ### Changed
8
10
 
11
+ - **pgmq-ruby 0.9.0 (was 0.8.2); the gemspec now requires `~> 0.9.0` (issue #656).** What reaches pgbus users is the connection pool: pgmq-ruby's checkout health check now resets a pooled connection in any state other than `CONNECTION_OK`, so a connection a `Thread#raise` left half-way through a reset (`CONNECTION_STARTED`) is recovered on the next checkout. Before, the checkout handed it out again, libpq refused every send (`no connection to the server`), and pgbus's stale-connection retry only recovered it on its last attempt by dropping both pools (#645): now the call succeeds on the first attempt, with no retry, no pool reload and no warning. pgmq-ruby also resets a connection whose call raised `PG::ConnectionBad` or `PG::UnableToSend` before it goes back to the pool (not a connection pgbus hands it through a callable, which it neither resets nor closes: the shared ActiveRecord connection, and dedicated connections in `connection_guc_mode = :session` with database.yml `variables`); native statements re-prepare on the new backend as on any reconnect. pgbus keeps its reload before the last stale retry for a connection that still fails after the health check passed it. A checkout reset of a connection in `connection_guc_mode = :session` with `variables` does not re-run its `SET`s, as for a `CONNECTION_BAD` one before (#671). `produce_topic` and `produce_batch_topic` now raise `PGMQ::Errors::MessageInvalidError` for a message that is not a String or headers that are neither a String nor nil, as `produce` has since 0.8.0. `Client#publish_to_topic` (what `Pgbus.publish` and the outbox poller call; `Pgbus::Testing`'s fake and inline modes never reach PGMQ) serializes the payload and headers to JSON Strings first, so it is unaffected; `headers: false`, which pgmq-ruby dropped before with an integer delay (and stored as JSON `false` with a fractional one) and now rejects, is passed as nil, so it is dropped at any delay. `spec/integration/connection_failure_spec.rb` now asserts the mid-reset connection is recovered at checkout (it failed on 0.8.2), in place of the two examples that pinned the old last-retry recovery. Refs #656.
12
+
9
13
  - **Changed default: the streamer's LISTEN keepalive `streams_listen_health_check_ms` is 5 000 ms instead of 250, and the listener's ack and stop budgets no longer follow it (issue #664).** Every web process running the streamer sent `SELECT 1` on its LISTEN connection four times a second, idle or not: 345 600 queries a day per process, the most frequent statement on a small fleet's database. Since #630 a connect no longer waits on that cycle (the listener wakes through a pipe), so the interval only bounds how long a LISTEN the server dropped on a still-open socket goes unnoticed; now one `SELECT 1` per 5 s idle cycle, 20× fewer. The interval was also wired into three wait budgets, so raising it slowed the failure paths: they now come from a new setting, **`streams_listen_ack_timeout_ms`** (default 1 250, a positive Integer, today's 250 + 1 000). `Listener#ensure_listening` waits at most that for a stuck listener's ack, `Listener#stop` joins for that plus its 5 s grace, and with `streams_listen_scope :master` the worker's `HubClient` waits that plus 1 s before failing over, as before relative to the hub's own budget; the master hub's listener gets the same ack budget. The streamer's dedicated LISTEN connection (per worker, and the master hub's) now opens with libpq TCP keepalives, `keepalives=1 keepalives_idle=5 keepalives_interval=2 keepalives_count=3 tcp_user_timeout=11000`, so a dead socket errors within ~11 s by these settings (5 s idle + 3 probes 2 s apart, and the matching `tcp_user_timeout`; on Linux, where `tcp_user_timeout` applies; the bound follows the settings, it was not measured) even between keepalives, and the listener reconnects and re-LISTENs; a keyword the connection options already set wins, a connection that names a service (`service=` or `PGSERVICE`) gets none so its service file's values stay in force, and on libpq < 12 none is added (the same gate `Client` uses for its read bounds, now `Pgbus::DedicatedConnection.libpq_supports_socket_bounds?`). The worker `NotifyListener` and the doctor's connection are unchanged. The pool autoscaler's check runs on the keepalive tick, so a `streams_pool_autoscale_interval` under 5 s (the default is 300 s) now runs every 5 s. A dedicated connection URL ending in a bare `?` no longer gets an empty `?&` parameter that failed the connect. An explicit `streams_listen_health_check_ms` keeps exactly its value: **pin `config.streams_listen_health_check_ms = 250` for the old keepalive.** If you set it to 5 000 as the workaround, the ack budget drops from 6 s back to 1.25 s. Measured on the same machine (`rake bench:streams`, which pins 100 ms; `rake bench:streams_scale` at M = 100 with the default, 250 ms before and 5 000 ms after, two rounds each): section 4 concurrent connect 0.91–3.07 ms per connect before, 0.86–3.06 ms after; scale bench, all 100 clients registered in 0.11–0.12 s before, 0.09–0.11 s after (one 0.24 s round at load 32, 0.09 s on a re-run), 0 misses either way. Refs #664.
10
14
 
11
15
  ### Fixed
12
16
 
13
17
  - **A presence table without the seat columns is reported, and its schema probe backs off (issue #665).** `Pgbus::Streams::Presence.seats_supported?` kept a "no seats" answer for 30 s, so an app that upgraded without running `rails g pgbus:add_presence_seats` queried `information_schema.columns` twice a minute per process, forever, and nothing said why. A "no" is now kept for 30 s, then twice as long after each further "no", up to 10 minutes (`SCHEMA_RECHECK_MAX_SECONDS = 600`); a true answer is still kept for the life of the process, `reset_schema_cache!` still re-probes at once and starts the backoff again from 30 s, and two threads that probe at the same moment count as one step. The backoff holds in a process that makes no member-keyed `leave`, `touch` or `sweep!`: each of those still re-probes a cached "no" before it runs, because it would delete or refresh seat rows on a table migrated under the process, and restarts the backoff at 30 s. Connection-driven presence on such a table still touches once per connection per heartbeat, unchanged here (#666). A member-keyed `join` that hits the migrated index still re-probes at once. The first "no" logs one warning per process naming the fix (`rails generate pgbus:add_presence_seats && rails db:migrate`). `pgbus doctor` has a fifteenth check, `Presence seats`: a warning with the same fix when `pgbus_presence_members` exists without `connection_id`, ok when the column exists or presence is not installed; it runs in the boot preflight and never blocks a strict boot. New `Pgbus::Streams::Presence.seats_state` (`:absent`, `:missing`, `:present`), read on the connection presence uses. Refs #665.
14
18
 
19
+ ## [0.25.0] - 2026-11-14
20
+
21
+ ### Added
22
+
23
+ - **Deploy tasks: `after:` makes a task wait until the tasks it names have completed, and every run records its outcome (issue #660).** `after:` takes a task key or a list of keys declared in the loaded configuration (any of `deploy_tasks_files`). The scheduler keeps such a task armed, and does not enqueue it, until each named task has a run with `completed_at` at its current version (any revision; an `every_deploy` dependency needs one on this revision). A rollover run and a Run now run both count, so a failed dependency that an operator fixes and runs again with Run now unblocks its dependents without a new deploy. A waiting task records no failure and no backoff, and fires on the first tick after its last dependency completes; the check is one query per tick, made only while an armed task has dependencies. At scheduler boot a task whose `after:` names an unknown key (`after: unknown task 'x'`), sits on a cycle (`after: cycle a → b → a`, a self-reference included), or names a task that is itself skipped (`after: 'x' is skipped`) is skipped with a warning. Run outcomes: `Deploy::Runner#enqueue` now creates the `pgbus_deploy_task_runs` row before the enqueue and passes its id in the job payload under `pgbus_deploy_task_run_id` (`Pgbus::ActiveJob::DeployTaskRunId`, included on `ActiveJob::Base` next to `BatchId`; an unmarked job's payload is unchanged), fills in `queue_name` and `msg_id` after it, and deletes the row when the enqueue fails. The executor stamps `completed_at` when `perform` returned having done its work without re-enqueuing the job (a `retry_on` attempt whose retry is queued, or a `discard_on` block that queues one, leaves it to the retry; a job that rescues inside `perform` and returns has completed) and `failed_at` when ActiveJob discarded the job without queuing a retry (`discard_on`, with or without a block, or a `retry_on` that gave up through its block, where one without a block re-raises and ends dead-lettered as before; `Pgbus::ActiveJob::DeployTaskRunId` marks the instance in ActiveJob's `run_after_discard_procs`, after the job's own `after_discard` procs, so a discarded dependency reads `failed` and its dependents keep waiting, issue #662), when the job is dead-lettered, discarded as a `:while_executing` duplicate, or its retry never left (dropped by its own concurrency limit, `on_conflict: :discard`, or aborted by a `before_enqueue` callback); a failed write is reported and never fails the job, and a job without the key pays one Hash lookup. An enqueue the job's own `on_conflict: :discard` / `:log` drops now counts as a failed enqueue (the run is deleted and the task re-armed with backoff). A run row without `queue_name` (its enqueue never finished) does not count as a run, so the version stays owed. The dispatcher's pruning keeps the newest completed run per task and version. Dashboard: a task's state says "Waiting for N dependencies to complete" and lists each one as `not run`, `running` or `failed`; the task page shows `after:` and an Outcome column in the run history; Run now on a waiting task is refused with a flash naming the dependencies (`DataSource#enqueue_deploy_task_now(id, ignore_dependencies: false)` returns `:blocked`) unless "Ignore dependencies" is ticked on the task page. `pgbus_deploy_tasks` (MCP) lists `dependencies` and `waiting_on`. New generator `pgbus:add_deploy_task_run_outcomes` adds `pgbus_deploy_task_runs.completed_at` / `failed_at`, `pgbus_deploy_tasks.dependencies` and an index on `(task_key, completed_at)`; `pgbus:update` queues it, and fresh `pgbus:install` / `pgbus:add_deploy_tasks` include the columns. Until it is migrated, a task with `after:` is skipped with a warning naming the generator and no outcome is written. **Upgrade note:** runs recorded before the migration have no outcome and satisfy nothing, so when an `after:` names a task whose only runs predate the upgrade, Run now that task once (or bump its version; an `every_deploy` dependency needs a run on the current revision, so only Run now helps). Delivery stays at-least-once: a scheduler that dies between claiming a task and enqueuing it leaves the task unarmed until the next scheduler boot re-arms it, and a run whose enqueue could not be recorded runs again after one, so deploy tasks must be idempotent. Refs #660.
24
+
15
25
  ## [0.24.0] - 2026-11-11
16
26
 
17
27
  ### Added
data/README.md CHANGED
@@ -553,7 +553,7 @@ A custom RuboCop cop, `Pgbus/NoRubyTimeout`, bans `Timeout.timeout` project-wide
553
553
 
554
554
  ### Stale connection retries
555
555
 
556
- A `PGMQ::Errors::ConnectionError` whose message says the pooled connection was dead before it was used (`no connection to the server`, `connection is closed`, a PgBouncer-closed socket and the like) is retried twice, after about 0.1 s and 0.5 s. Errors that may have reached the server (`server closed the connection unexpectedly`) and pool timeouts are never retried. The first retry relies on pgmq-ruby's checkout health check, which replaces a closed connection. On a dedicated connection (`database_url` or `connection_params`), before the last retry, pgbus drops the idle connections of both pools (job and streams) and logs `Dropping pooled connections before the last stale-connection retry`. This recovers a connection the health check keeps handing out, such as one a `Thread#raise` left half-way through a reset (`CONNECTION_STARTED`). A read interrupted by the Ruby `Timeout` fallback above reloads the job pool the same way. `Pgbus.client.reload` drops the idle connections of both pools by hand. None of these reload anything on the shared ActiveRecord `Proc` connection, whose socket pgbus does not own.
556
+ A `PGMQ::Errors::ConnectionError` whose message says the pooled connection was dead before it was used (`no connection to the server`, `connection is closed`, a PgBouncer-closed socket and the like) is retried twice, after about 0.1 s and 0.5 s. Errors that may have reached the server (`server closed the connection unexpectedly`) and pool timeouts are never retried. The first retry relies on pgmq-ruby's checkout health check, which resets a pooled connection that is not healthy: one the server closed, or one a `Thread#raise` left half-way through a reset (`CONNECTION_STARTED`, recovered at checkout since pgmq-ruby 0.9.0). On a dedicated connection (`database_url` or `connection_params`), before the last retry, pgbus drops the idle connections of both pools (job and streams) and logs `Dropping pooled connections before the last stale-connection retry`. This recovers a connection that still fails after the health check passed it. A read interrupted by the Ruby `Timeout` fallback above reloads the job pool the same way. `Pgbus.client.reload` drops the idle connections of both pools by hand. None of these reload anything on the shared ActiveRecord `Proc` connection, whose socket pgbus does not own. With `connection_guc_mode: :session` and database.yml `variables`, a connection the health check resets comes back without its session `SET`s (#671).
557
557
 
558
558
  ### Prefetch flow control
559
559
 
@@ -1962,7 +1962,7 @@ Pgbus.configure do |c|
1962
1962
  end
1963
1963
  ```
1964
1964
 
1965
- Keep each ttl above `streams_heartbeat_interval` (pgbus warns at boot otherwise): a shorter one expires live seats between touches. On a seat table `sweep!` only expires member-keyed rows; seats expire through the heartbeat. A process that probed the table before the migration re-checks after 30 seconds, then after twice as long each time it still finds no seats, up to every 10 minutes. A member-keyed `leave`, `touch` or `sweep!` re-checks before it runs and starts that backoff again at 30 seconds, and a member-keyed `join` outside a transaction re-checks at once if it hits the new index. The first time it finds no seats it logs one warning naming the migration, and `pgbus doctor` warns under `Presence seats`; restart the processes after migrating to pick the columns up at once. A raising hook is logged, not propagated. Library authors can probe for seat support with `Pgbus::Streams::Presence.instance_method(:join).parameters.include?([:key, :connection_id])` (the gem) and `Pgbus::Streams::Presence.seats_supported?` (the table).
1965
+ Keep each ttl above `streams_heartbeat_interval` (pgbus warns at boot otherwise): a shorter one expires live seats between touches. On a seat table `sweep!` only expires member-keyed rows; seats expire through the heartbeat. A process that probed the table before the migration re-checks after 30 seconds, then after twice as long each time it still finds no seats, up to every 10 minutes. A member-keyed `leave` or `sweep!` does not trust a cached "no": it re-checks once before it runs. Over a "no" that is not yet due, that re-check leaves the backoff's schedule alone; over one that is due, it is the timed re-check, and the wait doubles as usual. A member-keyed `touch` (the streamer's heartbeat touch included) only re-checks on that schedule: on a table migrated under a running process it can keep that member's seat rows fresh until the next re-check, at most 10 minutes later, and deletes nothing. A member-keyed `join` outside a transaction re-checks at once if it hits the new index. The first time it finds no seats it logs one warning naming the migration, and `pgbus doctor` warns under `Presence seats`; restart the processes after migrating to pick the columns up at once. A raising hook is logged, not propagated. Library authors can probe for seat support with `Pgbus::Streams::Presence.instance_method(:join).parameters.include?([:key, :connection_id])` (the gem) and `Pgbus::Streams::Presence.seats_supported?` (the table).
1966
1966
 
1967
1967
  ### Stream stats (opt-in)
1968
1968
 
data/lib/pgbus/client.rb CHANGED
@@ -1123,7 +1123,9 @@ module Pgbus
1123
1123
  @pgmq.produce_topic(
1124
1124
  routing_key,
1125
1125
  serialize(payload),
1126
- headers: headers && serialize(headers),
1126
+ # pgmq-ruby 0.9.0 rejects headers that are neither a String nor
1127
+ # nil; `false` was dropped before and still is.
1128
+ headers: headers ? serialize(headers) : nil,
1127
1129
  delay: delay
1128
1130
  )
1129
1131
  end
@@ -1896,15 +1898,15 @@ module Pgbus
1896
1898
  # still gets a second, backed-off chance rather than failing an enqueue
1897
1899
  # the caller may never retry.
1898
1900
  #
1899
- # The last retry runs on freshly reloaded pools (issue #645). A reset that
1900
- # Thread#raise interrupted mid-poll leaves the pooled PG::Connection in
1901
- # CONNECTION_STARTED: neither finished? nor CONNECTION_BAD, so pgmq-ruby's
1902
- # checkout health check keeps handing it out and libpq refuses every send
1903
- # ("no connection to the server"). The first retry still goes through that
1904
- # health check alone, which replaces a socket the server closed without
1905
- # touching the rest of the pool; only when it fails too are both pools'
1906
- # idle connections dropped (reload_pools_before_last_stale_retry), as the
1907
- # #354 wedged-read path does.
1901
+ # The last retry runs on freshly reloaded pools (issue #645). The first
1902
+ # retry goes through pgmq-ruby's checkout health check alone, which resets
1903
+ # a socket the server closed without touching the rest of the pool, and
1904
+ # since pgmq-ruby 0.9.0 also a connection left in any other non-OK state,
1905
+ # such as the CONNECTION_STARTED a reset interrupted by Thread#raise leaves
1906
+ # (before 0.9.0 that one was handed out again and libpq refused every send,
1907
+ # "no connection to the server"). Only when the first retry fails too are
1908
+ # both pools' idle connections dropped
1909
+ # (reload_pools_before_last_stale_retry), as the #354 wedged-read path does.
1908
1910
  STALE_RETRY_ATTEMPTS = 2
1909
1911
  private_constant :STALE_RETRY_ATTEMPTS
1910
1912
 
@@ -1987,7 +1989,8 @@ module Pgbus
1987
1989
  # WEDGED-SOCKET RECOVERY (issue #354): when (3) fires on a genuinely
1988
1990
  # hung socket, libpq may leave the pooled PG::Connection reporting
1989
1991
  # CONNECTION_OK while it will in fact re-hang on reuse, and pgmq-ruby's
1990
- # checkout health check won't discard it (it isn't CONNECTION_BAD). So
1992
+ # checkout health check won't discard it (it still reports
1993
+ # CONNECTION_OK). So
1991
1994
  # the Timeout raises WedgedReadTimeout (a ReadTimeoutError subclass)
1992
1995
  # and the rescue below drops every pooled connection via @pgmq.reload
1993
1996
  # (pgmq-ruby >= 0.7.1) — the pool rebuilds lazily on next checkout.
@@ -114,26 +114,31 @@ module Pgbus
114
114
  end
115
115
  return cached unless cached.nil?
116
116
 
117
- rows = exec_params(<<~SQL, [])
118
- SELECT 1 AS present FROM information_schema.columns
119
- WHERE table_name = 'pgbus_presence_members' AND column_name = 'connection_id'
120
- AND table_schema = ANY (current_schemas(false))
121
- SQL
122
- supported = !rows.to_a.empty?
123
- answer, warn = store_schema_answer(supported, generation)
124
- warn_seats_missing if warn
125
- answer
117
+ probe_seats(generation)
126
118
  end
127
119
 
128
- # seats_supported?, but a cached "no seats" is probed again first.
129
- # For the destructive member-keyed statements: on a table migrated
130
- # under this process their SQL without `connection_id IS NULL`
131
- # succeeds silently and deletes seat rows.
120
+ # seats_supported?, but a cached "no seats" is not trusted: anything
121
+ # short of a cached true is probed, once, and this probe's own answer
122
+ # is returned. For the destructive member-keyed statements (leave,
123
+ # sweep!): on a table migrated under this process their SQL without
124
+ # `connection_id IS NULL` succeeds silently and deletes seat rows.
125
+ #
126
+ # Its "no" is stored like any probe's (issue #666): over a fresh "no"
127
+ # it changes nothing, so the backoff keeps its step and its schedule
128
+ # however many calls force a probe; over a stale or missing answer it
129
+ # is the timed re-check, one step. A reset_schema_cache! during the
130
+ # probe (a join that hit the migrated index) means its "no" may
131
+ # predate the migration, so it probes again.
132
132
  def seats_supported_now?
133
- return true if seats_supported?
133
+ loop do
134
+ cached, generation = @schema_mutex.synchronize do
135
+ [@seats_supported == true && schema_answer_fresh?(true, @probed_at, @recheck_seconds), @schema_generation]
136
+ end
137
+ return true if cached
134
138
 
135
- reset_schema_cache!
136
- seats_supported?
139
+ answer = probe_seats(generation)
140
+ return answer if answer || @schema_mutex.synchronize { generation == @schema_generation }
141
+ end
137
142
  end
138
143
 
139
144
  def reset_schema_cache!
@@ -247,6 +252,20 @@ module Pgbus
247
252
  nil
248
253
  end
249
254
 
255
+ # Probes the schema (outside @schema_mutex, issue #566), stores the
256
+ # answer and returns the answer to give.
257
+ def probe_seats(generation)
258
+ rows = exec_params(<<~SQL, [])
259
+ SELECT 1 AS present FROM information_schema.columns
260
+ WHERE table_name = 'pgbus_presence_members' AND column_name = 'connection_id'
261
+ AND table_schema = ANY (current_schemas(false))
262
+ SQL
263
+ supported = !rows.to_a.empty?
264
+ answer, warn = store_schema_answer(supported, generation)
265
+ warn_seats_missing if warn
266
+ answer
267
+ end
268
+
250
269
  def schema_answer_fresh?(supported, probed_at, recheck_seconds)
251
270
  return false if supported.nil? || probed_at.nil?
252
271
 
@@ -257,9 +276,10 @@ module Pgbus
257
276
  # Returns the answer to give and whether to log the missing-seats
258
277
  # warning (the first "no" of the process).
259
278
  #
260
- # A "no" over a fresh "no" (two threads probed at once) keeps the
261
- # first one's time and step, so concurrent probes are one step. A
262
- # true answer is never taken back.
279
+ # A "no" over a fresh "no" (two threads probed at once, or a forced
280
+ # probe from seats_supported_now?) keeps the first one's time and
281
+ # step, so concurrent and forced probes add no step. A true answer is
282
+ # never taken back.
263
283
  def store_schema_answer(supported, generation)
264
284
  @schema_mutex.synchronize do
265
285
  next [supported, false] unless generation == @schema_generation
@@ -416,6 +436,12 @@ module Pgbus
416
436
  # to stay in the member list, and a sweeper expires anyone who
417
437
  # hasn't pinged in N seconds.
418
438
  #
439
+ # Member-keyed, it trusts the cached schema answer (issue #666): the
440
+ # streamer touches once per connection per heartbeat. On a table
441
+ # migrated under this process it may also refresh that member's seat
442
+ # rows until the answer is re-checked (at most
443
+ # SCHEMA_RECHECK_MAX_SECONDS); it never deletes anything.
444
+ #
419
445
  # With `connection_id:` (a String or an Array of them) it refreshes
420
446
  # those connections' seat rows in one statement and returns the ids
421
447
  # it found — a missing id means the row was expired while the
@@ -429,7 +455,7 @@ module Pgbus
429
455
  exec_params(<<~SQL, [@stream.name, member_id])
430
456
  UPDATE pgbus_presence_members
431
457
  SET last_seen_at = NOW()
432
- WHERE stream_name = $1 AND member_id = $2#{" AND connection_id IS NULL" if seats_now?}
458
+ WHERE stream_name = $1 AND member_id = $2#{" AND connection_id IS NULL" if seats?}
433
459
  SQL
434
460
  end
435
461
 
data/lib/pgbus/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Pgbus
4
- VERSION = "0.25.1"
4
+ VERSION = "0.26.0"
5
5
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: pgbus
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.25.1
4
+ version: 0.26.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Mikael Henriksson
@@ -63,14 +63,14 @@ dependencies:
63
63
  requirements:
64
64
  - - "~>"
65
65
  - !ruby/object:Gem::Version
66
- version: 0.8.2
66
+ version: 0.9.0
67
67
  type: :runtime
68
68
  prerelease: false
69
69
  version_requirements: !ruby/object:Gem::Requirement
70
70
  requirements:
71
71
  - - "~>"
72
72
  - !ruby/object:Gem::Version
73
- version: 0.8.2
73
+ version: 0.9.0
74
74
  - !ruby/object:Gem::Dependency
75
75
  name: railties
76
76
  requirement: !ruby/object:Gem::Requirement