pgbus 0.15.1 → 0.15.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +8 -0
- data/README.md +10 -5
- data/Rakefile +6 -4
- data/lib/generators/pgbus/templates/initializer.rb.erb +1 -1
- data/lib/pgbus/client.rb +1 -2
- data/lib/pgbus/execution_pools/async_pool.rb +1 -2
- data/lib/pgbus/mcp/rack_app.rb +50 -5
- data/lib/pgbus/process/heartbeat.rb +29 -3
- data/lib/pgbus/process/notify_listener.rb +18 -9
- data/lib/pgbus/process/supervisor.rb +22 -3
- data/lib/pgbus/version.rb +1 -1
- data/lib/pgbus/web/authentication.rb +1 -1
- metadata +5 -5
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 9ccee33eff147c5f62d51634db10d7d2529806a9b84d3373c8e66c4b93dfe68d
|
|
4
|
+
data.tar.gz: b3ebfda680231e62bc65a92822af22bcbb33a6666bfa9997be15387dcf2100cf
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 76e267110b0064e3b257fa10d0dab5a05f9705b6dabc5b86884cf3655df5125498cc5957c589662d04603653d40b516c5f8d073db9d1937829bc012979c91f00
|
|
7
|
+
data.tar.gz: ddb7653b85459ff790f48386932f9096950386cb52643eb146c3f3f6c3dd53a8383935df80b69efa124874f7018884c9211264c35aa656494c0e8ff760bf7bd2
|
data/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,11 @@
|
|
|
2
2
|
|
|
3
3
|
### Fixed
|
|
4
4
|
|
|
5
|
+
- **`Pgbus::MCP.rack_app` works on a real hostname again with `mcp` 0.23+ / 1.x.** Since mcp 0.23 the `StreamableHTTPTransport` validates the `Host` header (DNS-rebinding protection, on by default, loopback hosts only) and pgbus had no way to pass the transport's options through — so a gated mount at `https://app.example.com/pgbus/mcp` answered every request `403 "Invalid Host header"`, and consumers pinned `mcp < 1.0` to dodge it (which only helps while the lock stays on 0.22). The rack app now exposes `allowed_hosts:`, `allowed_origins:` and `dns_rebinding_protection:`; the check **follows the gate by default** — off when `token:`/`auth:` is configured (a rebound browser page can never carry the bearer secret, so the check is redundant there), on for the warned-about unauthenticated mount — and `true`/`false` forces it. `mcp >= 0.23` is the floor for `rack_app` (older gems raise `Pgbus::Error` naming the fix); the gem's own bundle now tracks `mcp` 1.x, so lift that `< 1.0` pin. Stdio (`pgbus mcp`) is unaffected.
|
|
6
|
+
|
|
7
|
+
- **Worker forks no longer kill the supervisor's shared LISTEN connection (issue #437).** Under `worker_notify_scope = :supervisor` every fork produced one `[Pgbus::NotifyListener] connection error (PG::ConnectionBad: PQconsumeInput() server closed the connection unexpectedly …) — reconnecting` in the supervisor ~2 s after `Worker started`, and a LISTEN gap (polling fallback) until `reconnect!` completed. The child's fork hygiene closed the Ruby `IO` wrapper of the inherited LISTEN socket, but pg builds `socket_io` with `autoclose=false`, so the fd stayed open — and when the child's GC freed the inherited `PG::Connection`, its `PQfinish` sent a libpq Terminate down that fd, i.e. down the **parent's** connection. `NotifyListener#close_inherited_socket!` now repoints the fd at `/dev/null` (`socket_io.reopen(IO::NULL)`, the ActiveRecord `PostgreSQLAdapter#discard!` idiom) so the eventual `PQfinish` is harmless. Regression-covered by a real-fork integration spec that asserts the parent's `pgbus-listen` backend pid is unchanged and no reconnect is logged. Refs #437.
|
|
8
|
+
- **A process whose `pgbus_processes` row is deleted underneath it now re-registers instead of staying invisible forever (issue #438).** `Heartbeat#beat` updated its row by id; when the row was gone — the dispatcher's stale-process reaper after a ≥ 5-minute heartbeat gap, a manual cleanup, another host's clock skew — `update_all` matched 0 rows, raised nothing, and the process (healthy, logging, serving `/readyz`) was absent from `pgbus_processes` until it restarted, so the supervisor's DB loop-tick check silently fell back to the pipe and `ConsumerPriority` could not see it. The beat now treats an affected-row count of 0 as "row is gone": it logs one WARN naming the old id, kind and pid (so whatever deleted the row can be hunted), re-registers through the existing boot path, and lands that beat's `last_heartbeat_at` / metadata on the new row. A mutex plus a stopped flag keep a beat that races `stop` from resurrecting a row `deregister_process` just deleted. Zero extra queries on the happy path. Refs #438.
|
|
9
|
+
- **Supervisor logs a worker recycle as a clean exit, not a crash (issue #438).** A worker or consumer hitting `max_jobs` / `max_memory` / `max_lifetime` exits 0 by design, but the supervisor logged `Child worker pid=N exited unexpectedly (status=0)`, burying real crashes. A clean exit outside shutdown is now INFO `exited cleanly (status=0) — restarting (worker recycle)`; non-zero exits keep the WARN, and a signaled exit reports `signal=N` instead of an empty status so an OOM SIGKILL is distinguishable. Restart policy is unchanged. Refs #438.
|
|
5
10
|
- **`current_attributes` capture skips an unpersisted record instead of raising at enqueue (issue #435).** An attribute holding an Active Record instance with no id — a dev-mode fallback record assigned to `Current`, a form-built model captured before `save`, a destroyed record whose locate is guaranteed to fail — made every `perform_later` in that context raise `Pgbus::CurrentAttributesError`, even though such a record can never round-trip (no id → no GlobalID) and capture is ambient: the enqueuer never opted into persisting that attribute per-call, so its momentary state must not abort the enqueue. `capture` now skips any attribute value that answers `persisted?` falsey (so destroyed-but-id-bearing records are skipped too, not just `new_record?`) with a debug log naming the class, attribute and why; the rest of the class's attributes still persist. The `except:` guidance and the loud `CurrentAttributesError` remain for genuinely unserializable values — objects without the Active Record duck-type (`respond_to?(:persisted?)`) are untouched. Applies to jobs and event-bus publish alike (same capture path). Refs #435.
|
|
6
11
|
|
|
7
12
|
### Added
|
|
@@ -67,6 +72,9 @@
|
|
|
67
72
|
|
|
68
73
|
### Changed
|
|
69
74
|
|
|
75
|
+
- **The gem root's `Gemfile.lock` is now committed.** Every checkout and CI leg resolves the same gem set (the main-Gemfile legs and `release.yml` install it frozen, like the Rails 7.1 and docs locks already were), so a fresh checkout can no longer silently pick up a newer dev dependency than the last green run. The gemspec builds `spec.files` from a path allowlist, so the lock never ships in the gem (verified with `gem build`). `rake release` bumps the `pgbus (X.Y.Z)` pin in all three tracked lockfiles in the bump commit, and `spec/pgbus/frozen_lockfile_sync_spec.rb` guards the root lock too. Contributors: after pulling, `bundle install` is a no-op unless the lock moved; run `bundle install` (not `bundle lock`) when you change a dependency, and commit the lock with it.
|
|
76
|
+
- **Client build toolchain: bun 1.3.11 → 1.4.0.** `.bun-version`, the root `engines.bun` floor, and the docs-site CI job (now pinned to the same `.bun-version` file instead of `latest`) move together. Both `bun.lock` files install `--frozen-lockfile` unchanged; pgbus ships no bun-built artifacts, so nothing in the gem changes.
|
|
77
|
+
|
|
70
78
|
- **Shutdown budgets are now alignable end-to-end (issue #386).** New `config.shutdown_timeout` bounds how long the supervisor waits for children after forwarding TERM before escalating to SIGKILL — previously a hardcoded 30s, which silently SIGKILLed workers mid-drain the moment `drain_timeout` was raised past it. Default derives `drain_timeout + 5` so the deadline tracks the drain window automatically; an explicit value below `drain_timeout` logs a boot warning. `Consumer#shutdown`'s pool wait (its only drain bound) now follows `config.drain_timeout` instead of a hardcoded 30s, and `Worker#shutdown`'s post-drain residual wait drops from a second full 30s window to 5s — the drain loop already waited `drain_timeout`, and a job still running has proven it won't finish. Rule of thumb: orchestrator stop grace period > `shutdown_timeout` > `drain_timeout`. Refs #386.
|
|
71
79
|
|
|
72
80
|
- **Streams: one LISTEN connection per web host — `streams_listen_scope` (issue #382).** ⚠️ **Default behavior change.** Previously every Puma worker lazily opened its own dedicated streams LISTEN connection on first SSE use, so a web host pinned one direct connection per worker. Under the new default (`streams_listen_scope = :master`) the `pgbus_streams` Puma plugin runs a **MasterHub** in the preforking master: ONE `Web::Streamer::Listener` on the refcounted union of every worker's stream channels, fanning wakes — **including ephemeral payloads** — out to workers over a Unix domain socket with length-prefixed frames (`Streamer::HubProtocol`). Workers connect lazily (nothing is inherited across fork) and the synchronous `ensure_listening` ack contract is preserved cross-process: a sub is registered before LISTEN executes and acked only after, so the no-lost-broadcast guarantee holds. Backpressure follows the streams rules: durable wakes are droppable at a per-worker cap (they self-heal via `read_after`), **ephemeral wakes are never dropped** — a worker that stops draining is evicted, which triggers its own fallback. **Fallback is per-worker listeners, not loss**: whenever the hub is absent or dies (no `preload_app!`, single-mode Puma, crash, eviction) each worker's `FailoverListener` swaps in a real per-worker `Listener` and re-LISTENs its recorded subscriptions — connection footprint balloons back to pre-#382 levels (census-visible) but no broadcast semantics change; the worker stays local until it recycles. Measured (local PG, n=50): the master→worker hop is noise-level free — single-broadcast SSE roundtrip p50 16.00ms via the hub vs 16.93ms per-worker. **`:master` effectively requires `preload_app!`** (the hub waits for the app's pgbus initializer; without it the deadline expires quietly and workers stay per-worker). **Rollback:** `config.streams_listen_scope = :process`. Refs #382, builds on the #381 patterns.
|
data/README.md
CHANGED
|
@@ -6,7 +6,7 @@ PostgreSQL-native job processing and event bus for Rails, built on [PGMQ](https:
|
|
|
6
6
|
|
|
7
7
|
📖 **Documentation:** [pgbus.zoolutions.llc](https://pgbus.zoolutions.llc) — guides, flow diagrams, and a full configuration reference. (This README stays the canonical GitHub reference.)
|
|
8
8
|
|
|
9
|
-
[](https://github.com/zoolutions/pgbus/actions/workflows/main.yml)
|
|
10
10
|
|
|
11
11
|
## Table of contents
|
|
12
12
|
|
|
@@ -1035,7 +1035,7 @@ When `config.metrics_enabled = true` (default), the dashboard exposes Prometheus
|
|
|
1035
1035
|
|
|
1036
1036
|
Pgbus ships an optional, **read-only** [MCP](https://modelcontextprotocol.io) server so an AI agent (or any MCP client) can diagnose pgbus directly — "are queues backed up?", "is `read_ct` advancing?", "are workers heart-beating but not claiming?" — instead of hand-writing `pgmq` / `pg_stat_activity` SQL against production. It is a thin adapter over the same read layer the dashboard uses, so it adds no new database access path.
|
|
1037
1037
|
|
|
1038
|
-
Add the optional `mcp` gem to your `Gemfile` first (`gem "mcp"
|
|
1038
|
+
Add the optional `mcp` gem to your `Gemfile` first (`gem "mcp"`, 0.23 or newer — 1.x is fully supported); both entry points below tell you if it's missing.
|
|
1039
1039
|
|
|
1040
1040
|
#### Choosing a deployment
|
|
1041
1041
|
|
|
@@ -1075,9 +1075,14 @@ Options:
|
|
|
1075
1075
|
| `token:` | `nil` | Shared secret. When set, requests must send `Authorization: Bearer <token>` (constant-time compared). |
|
|
1076
1076
|
| `auth:` | `nil` | A callable `->(rack_request) { ... }` returning truthy to allow — mirrors `config.web_auth`. Wins over `token:`. |
|
|
1077
1077
|
| `allow_payloads:` | `false` | When true, tools honor a per-call `include_payloads` flag (see Security). |
|
|
1078
|
+
| `dns_rebinding_protection:` | `nil` | The `mcp` gem's Host/Origin validation (on by default since mcp 0.23, loopback hosts only). `nil` follows the gate: **off when `token:`/`auth:` is set, on when unauthenticated.** `true`/`false` forces it. |
|
|
1079
|
+
| `allowed_hosts:` | `nil` | Extra `Host` values accepted when the check is on (`"app.example.com"` matches any port, `"app.example.com:8443"` exactly). |
|
|
1080
|
+
| `allowed_origins:` | `nil` | Extra `Origin` values accepted beyond same-origin when the check is on. |
|
|
1078
1081
|
|
|
1079
1082
|
If you set neither `token:` nor `auth:`, pgbus logs a warning — an unauthenticated diagnostic endpoint exposes operational metadata to anyone who can reach it.
|
|
1080
1083
|
|
|
1084
|
+
> **Why the Host check follows the gate.** DNS-rebinding protection defends a server bound to `localhost` against a browser page whose DNS name was re-pointed at `127.0.0.1`. Such a page can never carry your bearer token (the secret doesn't exist at the attacker's origin), so on a gated mount the check is redundant — and left on, it rejects every request to a real hostname (`https://app.example.com/pgbus/mcp` → `403 Invalid Host header`). Pgbus therefore turns it off when a gate is configured and keeps it on for the (warned-about) unauthenticated mount. If your `auth:` callable trusts something a rebound page *would* have — a source-IP allowlist, say — pass `dns_rebinding_protection: true` plus `allowed_hosts:` for your hostname. Requires `mcp >= 0.23`; older gems raise `Pgbus::Error` naming the floor.
|
|
1085
|
+
|
|
1081
1086
|
> Clients must send `Accept: application/json` and `Content-Type: application/json` on every POST, or the transport replies `406 Not Acceptable`. MCP clients do this automatically.
|
|
1082
1087
|
|
|
1083
1088
|
Need a **standalone HTTP pod** instead of mounting in your main app? The same Rack app works under any Rack server, e.g. a one-line `config.ru`:
|
|
@@ -1220,9 +1225,9 @@ pgbus-health --port 9394 # or PGBUS_HEALTH_PORT=9394 pgbus-health
|
|
|
1220
1225
|
pgbus-health --port 9394 --path /livez --timeout 2
|
|
1221
1226
|
```
|
|
1222
1227
|
|
|
1223
|
-
### Rolling restarts (
|
|
1228
|
+
### Rolling restarts (dash, docker)
|
|
1224
1229
|
|
|
1225
|
-
|
|
1230
|
+
[dash](https://github.com/zoolutions/dash) (per-role health checks) can rolling-restart a non-proxied job role: start the new container, poll its docker `HEALTHCHECK` until healthy, and only then `docker stop` the old one. Wire the pgbus container into that gate:
|
|
1226
1231
|
|
|
1227
1232
|
```yaml
|
|
1228
1233
|
# config/deploy.yml
|
|
@@ -1253,7 +1258,7 @@ If the orchestrator's stop grace period is *shorter* than `shutdown_timeout`, do
|
|
|
1253
1258
|
|
|
1254
1259
|
**The overlap window is safe by construction.** Between "new container healthy" and "old container stopped", two supervisors run against the same database. Nothing double-fires: queue claims use `FOR UPDATE SKIP LOCKED`, `single_active_consumer` queues arbitrate via session-level advisory locks (released the instant a killed process's connection dies), two live recurring schedulers dedup on the `(task_key, run_at)` unique record, and dispatcher maintenance is idempotent. "One scheduler per deployment" is a steady-state rule; a deploy window may briefly violate it without consequence.
|
|
1255
1260
|
|
|
1256
|
-
**What a hard kill still costs.** Jobs killed past the drain window are redelivered after their visibility timeout (at-least-once holds) — but PGMQ's `read_ct` increments exactly like a logical failure, so a long-running job that straddles *repeated* deploy kills can be pushed to the DLQ without its code ever raising. `zombie_detection` logs exactly this pattern (`read_ct > 1` with no recorded failure). Keep jobs shorter than `drain_timeout`, or raise it (and `stop_timeout`) for queues that can't be. For `idempotent!` event handlers there is a separate crash-window caveat tracked in [#385](https://github.com/
|
|
1261
|
+
**What a hard kill still costs.** Jobs killed past the drain window are redelivered after their visibility timeout (at-least-once holds) — but PGMQ's `read_ct` increments exactly like a logical failure, so a long-running job that straddles *repeated* deploy kills can be pushed to the DLQ without its code ever raising. `zombie_detection` logs exactly this pattern (`read_ct > 1` with no recorded failure). Keep jobs shorter than `drain_timeout`, or raise it (and `stop_timeout`) for queues that can't be. For `idempotent!` event handlers there is a separate crash-window caveat tracked in [#385](https://github.com/zoolutions/pgbus/issues/385).
|
|
1257
1262
|
|
|
1258
1263
|
### Boot diagnostics banner
|
|
1259
1264
|
|
data/Rakefile
CHANGED
|
@@ -232,9 +232,11 @@ task :release, %i[version force] do |_t, args|
|
|
|
232
232
|
|
|
233
233
|
# Step 1b: Regenerate the frozen lockfiles that pin the pgbus path gem, so the
|
|
234
234
|
# bump ships with them in sync. These are installed with `--frozen`/deployment
|
|
235
|
-
# in CI, so if they still name the OLD version they instant-fail (the
|
|
236
|
-
#
|
|
237
|
-
#
|
|
235
|
+
# in CI, so if they still name the OLD version they instant-fail (the root
|
|
236
|
+
# Gemfile.lock on every main-Gemfile leg AND release.yml's own `bundle install`,
|
|
237
|
+
# the Rails 7.1 leg with exit 16, and docs-CI on any docs change). Regenerating
|
|
238
|
+
# here keeps the version-pin drift out of the release commit instead of
|
|
239
|
+
# surfacing on the next PR — or, worse, in the Release workflow itself.
|
|
238
240
|
header "Frozen lockfiles"
|
|
239
241
|
# The ONLY thing a version bump changes in these frozen lockfiles is the pgbus
|
|
240
242
|
# path-gem pin — so bump exactly that line, in place, with a string edit.
|
|
@@ -249,7 +251,7 @@ task :release, %i[version force] do |_t, args|
|
|
|
249
251
|
# fetch). A targeted pin edit sidesteps all of it, is deterministic on any
|
|
250
252
|
# machine, and produces the minimal 2-line diff (the PATH spec + the
|
|
251
253
|
# DEPENDENCIES pin). See #338/#341 and the surgical-bump fix.
|
|
252
|
-
frozen_lockfiles = %w[gemfiles/rails_7_1.gemfile.lock docs/Gemfile.lock]
|
|
254
|
+
frozen_lockfiles = %w[Gemfile.lock gemfiles/rails_7_1.gemfile.lock docs/Gemfile.lock]
|
|
253
255
|
regenerated_lockfiles = []
|
|
254
256
|
frozen_lockfiles.each do |lockfile|
|
|
255
257
|
unless File.exist?(lockfile)
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
-
# Pgbus configuration — https://github.com/
|
|
3
|
+
# Pgbus configuration — https://github.com/zoolutions/pgbus
|
|
4
4
|
#
|
|
5
5
|
# This is the real config surface. Every setting has a sensible default, so an
|
|
6
6
|
# empty block gives you a working install; uncomment and edit what you need.
|
data/lib/pgbus/client.rb
CHANGED
|
@@ -1753,11 +1753,10 @@ module Pgbus
|
|
|
1753
1753
|
timeout = config.read_timeout
|
|
1754
1754
|
return mapping_statement_timeout(&block) unless timeout&.positive?
|
|
1755
1755
|
|
|
1756
|
-
# rubocop:disable Pgbus/NoRubyTimeout -- deliberate last-resort bound; see above
|
|
1756
|
+
# rubocop:disable-next Pgbus/NoRubyTimeout -- deliberate last-resort bound; see above
|
|
1757
1757
|
Timeout.timeout(timeout + READ_TIMEOUT_SLACK, WedgedReadTimeout) do
|
|
1758
1758
|
mapping_statement_timeout(&block)
|
|
1759
1759
|
end
|
|
1760
|
-
# rubocop:enable Pgbus/NoRubyTimeout
|
|
1761
1760
|
rescue WedgedReadTimeout
|
|
1762
1761
|
reload_pool_after_wedged_timeout
|
|
1763
1762
|
raise
|
|
@@ -95,7 +95,7 @@ module Pgbus
|
|
|
95
95
|
"Add `gem \"async\"` to your Gemfile. Original error: #{e.message}"
|
|
96
96
|
end
|
|
97
97
|
|
|
98
|
-
# rubocop:disable Lint/RescueException
|
|
98
|
+
# rubocop:disable-next Lint/RescueException
|
|
99
99
|
def start_reactor
|
|
100
100
|
Thread.new do
|
|
101
101
|
Thread.current.name = "pgbus-async-reactor-#{object_id}"
|
|
@@ -122,7 +122,6 @@ module Pgbus
|
|
|
122
122
|
raise
|
|
123
123
|
end
|
|
124
124
|
end
|
|
125
|
-
# rubocop:enable Lint/RescueException
|
|
126
125
|
|
|
127
126
|
def wait_for_executions(semaphore)
|
|
128
127
|
loop do
|
data/lib/pgbus/mcp/rack_app.rb
CHANGED
|
@@ -20,8 +20,24 @@ module Pgbus
|
|
|
20
20
|
# Security: requests are rejected with 401 unless they carry the configured
|
|
21
21
|
# token (or pass the supplied auth callable). Run it on an internal network
|
|
22
22
|
# / behind your VPN, never internet-exposed.
|
|
23
|
+
#
|
|
24
|
+
# DNS-rebinding protection: since mcp 0.23 the transport validates the Host
|
|
25
|
+
# header (loopback only, by default) and the Origin header (same-origin
|
|
26
|
+
# only). That defends a server bound to localhost against a browser page
|
|
27
|
+
# whose DNS name was re-pointed at 127.0.0.1 — a page that can carry no
|
|
28
|
+
# bearer token, because the secret never reaches the attacker's origin. So
|
|
29
|
+
# when this app is gated (+token+ or +auth+) the check is redundant, and
|
|
30
|
+
# left on it rejects every request to a real hostname
|
|
31
|
+
# (https://app.example.com/pgbus/mcp → 403 "Invalid Host header"). The
|
|
32
|
+
# default therefore follows the gate: off when gated, on when unauthenticated.
|
|
33
|
+
# Override with +dns_rebinding_protection:+, and widen the accepted hosts /
|
|
34
|
+
# origins with +allowed_hosts:+ / +allowed_origins:+ when the check is on.
|
|
23
35
|
class RackApp
|
|
24
36
|
BEARER_PREFIX = "Bearer "
|
|
37
|
+
# First mcp release with the transport's allowed_hosts / allowed_origins /
|
|
38
|
+
# dns_rebinding_protection options. Older gems would raise ArgumentError
|
|
39
|
+
# on the pass-through; fail with the fix spelled out instead.
|
|
40
|
+
MIN_MCP_VERSION = Gem::Version.new("0.23.0")
|
|
25
41
|
# Only the JSON body string is frozen and reused. The outer response triple
|
|
26
42
|
# and its headers hash MUST be built fresh per call (#unauthorized) so
|
|
27
43
|
# downstream Rack middleware can mutate them — Rack::TempfileReaper assigns
|
|
@@ -40,14 +56,27 @@ module Pgbus
|
|
|
40
56
|
# @param auth [#call, nil] custom authenticator taking a Rack::Request and
|
|
41
57
|
# returning truthy to allow. Mirrors Pgbus.configuration.web_auth. Takes
|
|
42
58
|
# precedence over +token+ when both are given.
|
|
43
|
-
|
|
59
|
+
# @param allowed_hosts [Array<String>, nil] extra Host values the
|
|
60
|
+
# transport accepts beyond loopback when DNS-rebinding protection is on;
|
|
61
|
+
# a bare name matches any port, "host:port" matches exactly.
|
|
62
|
+
# @param allowed_origins [Array<String>, nil] extra Origin values accepted
|
|
63
|
+
# beyond same-origin when DNS-rebinding protection is on.
|
|
64
|
+
# @param dns_rebinding_protection [Boolean, nil] nil (default) = on only
|
|
65
|
+
# when the app is unauthenticated; true/false forces it. See the class
|
|
66
|
+
# docs for why the gate makes the check redundant.
|
|
67
|
+
def initialize(data_source: Pgbus::Web::DataSource.new, allow_payloads: false, token: nil, auth: nil,
|
|
68
|
+
allowed_hosts: nil, allowed_origins: nil, dns_rebinding_protection: nil)
|
|
69
|
+
check_mcp_version!
|
|
44
70
|
@token = token
|
|
45
71
|
@auth = auth
|
|
46
72
|
@server = Server.build(data_source: data_source, allow_payloads: allow_payloads)
|
|
47
73
|
@transport = ::MCP::Server::Transports::StreamableHTTPTransport.new(
|
|
48
|
-
@server,
|
|
74
|
+
@server,
|
|
75
|
+
stateless: true, enable_json_response: true,
|
|
76
|
+
allowed_hosts: allowed_hosts, allowed_origins: allowed_origins,
|
|
77
|
+
dns_rebinding_protection: dns_rebinding_protection.nil? ? unauthenticated? : dns_rebinding_protection
|
|
49
78
|
)
|
|
50
|
-
warn_unauthenticated! if
|
|
79
|
+
warn_unauthenticated! if unauthenticated?
|
|
51
80
|
end
|
|
52
81
|
|
|
53
82
|
# Mount THIS object, never the bare transport. The auth gate lives here
|
|
@@ -90,20 +119,36 @@ module Pgbus
|
|
|
90
119
|
Runner.secure_compare?(@token, header.delete_prefix(BEARER_PREFIX))
|
|
91
120
|
end
|
|
92
121
|
|
|
122
|
+
def unauthenticated?
|
|
123
|
+
@token.nil? && @auth.nil?
|
|
124
|
+
end
|
|
125
|
+
|
|
93
126
|
def warn_unauthenticated!
|
|
94
127
|
Pgbus.logger.warn do
|
|
95
128
|
"[Pgbus::MCP] HTTP diagnostic server mounted without authentication. " \
|
|
96
129
|
"Pass token: or auth: to Pgbus::MCP.rack_app, and keep it on an internal network."
|
|
97
130
|
end
|
|
98
131
|
end
|
|
132
|
+
|
|
133
|
+
def check_mcp_version!
|
|
134
|
+
installed = Gem::Version.new(::MCP::VERSION)
|
|
135
|
+
return if installed >= MIN_MCP_VERSION
|
|
136
|
+
|
|
137
|
+
raise Pgbus::Error,
|
|
138
|
+
"Pgbus::MCP.rack_app requires mcp >= #{MIN_MCP_VERSION} (the transport's DNS-rebinding " \
|
|
139
|
+
"options); mcp #{installed} is installed. Run `bundle update mcp`."
|
|
140
|
+
end
|
|
99
141
|
end
|
|
100
142
|
|
|
101
143
|
module_function
|
|
102
144
|
|
|
103
145
|
# Build a gated Rack app serving the read-only diagnostic tools over HTTP.
|
|
104
146
|
# See {RackApp} for the parameters and deployment guidance.
|
|
105
|
-
def rack_app(data_source: Pgbus::Web::DataSource.new, allow_payloads: false, token: nil, auth: nil
|
|
106
|
-
|
|
147
|
+
def rack_app(data_source: Pgbus::Web::DataSource.new, allow_payloads: false, token: nil, auth: nil,
|
|
148
|
+
allowed_hosts: nil, allowed_origins: nil, dns_rebinding_protection: nil)
|
|
149
|
+
RackApp.new(data_source: data_source, allow_payloads: allow_payloads, token: token, auth: auth,
|
|
150
|
+
allowed_hosts: allowed_hosts, allowed_origins: allowed_origins,
|
|
151
|
+
dns_rebinding_protection: dns_rebinding_protection)
|
|
107
152
|
end
|
|
108
153
|
end
|
|
109
154
|
end
|
|
@@ -18,6 +18,12 @@ module Pgbus
|
|
|
18
18
|
@loop_tick_supplier = loop_tick_supplier
|
|
19
19
|
@metadata_supplier = metadata_supplier
|
|
20
20
|
@timer = nil
|
|
21
|
+
@stopped = false
|
|
22
|
+
# Guards @process_id between the timer thread (beat) and the main
|
|
23
|
+
# thread (stop): TimerTask#shutdown does not wait for an in-flight
|
|
24
|
+
# beat, and a beat that re-registers after deregister_process ran
|
|
25
|
+
# would leave a zombie row (issue #438).
|
|
26
|
+
@mutex = Mutex.new
|
|
21
27
|
end
|
|
22
28
|
|
|
23
29
|
def start
|
|
@@ -27,18 +33,19 @@ module Pgbus
|
|
|
27
33
|
end
|
|
28
34
|
|
|
29
35
|
def stop
|
|
36
|
+
@stopped = true
|
|
30
37
|
@timer&.shutdown
|
|
31
|
-
deregister_process
|
|
38
|
+
@mutex.synchronize { deregister_process }
|
|
32
39
|
end
|
|
33
40
|
|
|
34
41
|
def beat
|
|
35
|
-
return unless @process_id
|
|
42
|
+
return unless @process_id && !@stopped
|
|
36
43
|
|
|
37
44
|
@on_beat&.call
|
|
38
45
|
updates = { last_heartbeat_at: Time.current }
|
|
39
46
|
metadata = beat_metadata
|
|
40
47
|
updates[:metadata] = metadata unless metadata.nil?
|
|
41
|
-
|
|
48
|
+
@mutex.synchronize { write_beat(updates) }
|
|
42
49
|
rescue StandardError => e
|
|
43
50
|
Pgbus.logger.warn { "[Pgbus] Heartbeat failed: #{e.message}" }
|
|
44
51
|
end
|
|
@@ -59,6 +66,25 @@ module Pgbus
|
|
|
59
66
|
metadata
|
|
60
67
|
end
|
|
61
68
|
|
|
69
|
+
# update_all by id matches 0 rows when the row was deleted underneath a
|
|
70
|
+
# live process (stale-process reaper after a heartbeat gap, manual
|
|
71
|
+
# cleanup, another host's clock skew). Nothing raises, so treat the
|
|
72
|
+
# count as the signal: re-register and log once so the cause of the
|
|
73
|
+
# deletion is discoverable, then land this beat's updates on the new
|
|
74
|
+
# row (issue #438). Skipped after stop so a beat racing deregistration
|
|
75
|
+
# cannot resurrect the row.
|
|
76
|
+
def write_beat(updates)
|
|
77
|
+
return if ProcessEntry.where(id: @process_id).update_all(updates).positive? || @stopped
|
|
78
|
+
|
|
79
|
+
old_id = @process_id
|
|
80
|
+
Pgbus.logger.warn do
|
|
81
|
+
"[Pgbus] Process row id=#{old_id} kind=#{@kind} pid=#{::Process.pid} is gone " \
|
|
82
|
+
"(stale-process reaper, manual cleanup, or clock skew?) — re-registering"
|
|
83
|
+
end
|
|
84
|
+
register_process
|
|
85
|
+
ProcessEntry.where(id: @process_id).update_all(updates) if @process_id != old_id
|
|
86
|
+
end
|
|
87
|
+
|
|
62
88
|
def register_process
|
|
63
89
|
record = ProcessEntry.create!(
|
|
64
90
|
kind: @kind,
|
|
@@ -143,13 +143,21 @@ module Pgbus
|
|
|
143
143
|
@state_mutex.synchronize { @running }
|
|
144
144
|
end
|
|
145
145
|
|
|
146
|
-
# Called ONLY inside a just-forked child (
|
|
147
|
-
# this process's copy of the LISTEN socket
|
|
146
|
+
# Called ONLY inside a just-forked child (issues #381 / #437): release
|
|
147
|
+
# this process's copy of the LISTEN socket WITHOUT PQfinish — #close
|
|
148
148
|
# would send a libpq Terminate over the socket shared with the parent,
|
|
149
|
-
# killing the parent's LISTEN session.
|
|
150
|
-
#
|
|
151
|
-
#
|
|
152
|
-
#
|
|
149
|
+
# killing the parent's LISTEN session.
|
|
150
|
+
#
|
|
151
|
+
# Closing the IO wrapper is not enough: pg builds socket_io with
|
|
152
|
+
# autoclose=false, so IO#close leaves the fd open, and when GC frees the
|
|
153
|
+
# inherited PG::Connection its PQfinish still writes Terminate on that
|
|
154
|
+
# fd — i.e. on the PARENT's connection (issue #437, one reconnect per
|
|
155
|
+
# fork). Repointing the fd at /dev/null (IO#reopen, the ActiveRecord
|
|
156
|
+
# PostgreSQLAdapter#discard! idiom) makes the eventual PQfinish harmless.
|
|
157
|
+
#
|
|
158
|
+
# The listener thread does not exist in the child (fork copies only the
|
|
159
|
+
# calling thread), so there is no concurrent owner and the single-owner
|
|
160
|
+
# rule (#375) does not apply.
|
|
153
161
|
def close_inherited_socket!
|
|
154
162
|
conn = @state_mutex.synchronize do
|
|
155
163
|
c = @conn
|
|
@@ -157,10 +165,11 @@ module Pgbus
|
|
|
157
165
|
@running = false
|
|
158
166
|
c
|
|
159
167
|
end
|
|
160
|
-
conn&.socket_io&.
|
|
168
|
+
conn&.socket_io&.reopen(IO::NULL)
|
|
161
169
|
rescue StandardError => e
|
|
162
|
-
# Best-effort
|
|
163
|
-
#
|
|
170
|
+
# Best-effort but never silent: the child keeps booting either way.
|
|
171
|
+
# If this fails the child's GC-time PQfinish will hit the parent's
|
|
172
|
+
# session, which the parent's reconnect! survives.
|
|
164
173
|
@logger.warn do
|
|
165
174
|
"[Pgbus::NotifyListener] inherited socket cleanup failed: #{e.class}: #{e.message}"
|
|
166
175
|
end
|
|
@@ -613,9 +613,7 @@ module Pgbus
|
|
|
613
613
|
if @shutting_down
|
|
614
614
|
Pgbus.logger.info { "[Pgbus] Child #{info[:type]} pid=#{pid} exited (status=#{status.exitstatus})" }
|
|
615
615
|
else
|
|
616
|
-
|
|
617
|
-
"[Pgbus] Child #{info[:type]} pid=#{pid} exited unexpectedly (status=#{status&.exitstatus})"
|
|
618
|
-
end
|
|
616
|
+
log_child_exit(info, pid, status)
|
|
619
617
|
schedule_restart(info, status)
|
|
620
618
|
end
|
|
621
619
|
rescue Errno::ECHILD
|
|
@@ -623,6 +621,27 @@ module Pgbus
|
|
|
623
621
|
end
|
|
624
622
|
end
|
|
625
623
|
|
|
624
|
+
# A clean exit outside shutdown is a worker/consumer recycle (max_jobs,
|
|
625
|
+
# max_memory, max_lifetime) — expected, so INFO. Anything else is a
|
|
626
|
+
# crash: WARN, naming the signal when there is one so an OOM SIGKILL
|
|
627
|
+
# reads differently from an exit code (issue #438).
|
|
628
|
+
def log_child_exit(info, pid, status)
|
|
629
|
+
if status&.success?
|
|
630
|
+
Pgbus.logger.info do
|
|
631
|
+
"[Pgbus] Child #{info[:type]} pid=#{pid} exited cleanly (status=0) — restarting (worker recycle)"
|
|
632
|
+
end
|
|
633
|
+
else
|
|
634
|
+
Pgbus.logger.warn do
|
|
635
|
+
detail = if status && status.exitstatus.nil? && status.signaled?
|
|
636
|
+
"signal=#{status.termsig}"
|
|
637
|
+
else
|
|
638
|
+
"status=#{status&.exitstatus}"
|
|
639
|
+
end
|
|
640
|
+
"[Pgbus] Child #{info[:type]} pid=#{pid} exited unexpectedly (#{detail})"
|
|
641
|
+
end
|
|
642
|
+
end
|
|
643
|
+
end
|
|
644
|
+
|
|
626
645
|
# Restart policy: a clean exit (worker recycling) or a crash after a
|
|
627
646
|
# stable run restarts immediately with a fresh crash streak. A crash
|
|
628
647
|
# within RESTART_STABLE_UPTIME of forking is a crash loop — the child
|
data/lib/pgbus/version.rb
CHANGED
|
@@ -44,7 +44,7 @@ module Pgbus
|
|
|
44
44
|
Pgbus.logger.warn do
|
|
45
45
|
"[Pgbus] Dashboard is accessible without authentication. " \
|
|
46
46
|
"Configure Pgbus.configuration.web_auth to restrict access. " \
|
|
47
|
-
"See: https://github.com/
|
|
47
|
+
"See: https://github.com/zoolutions/pgbus#dashboard-authentication"
|
|
48
48
|
end
|
|
49
49
|
Pgbus::Web::Authentication.auth_warned = true
|
|
50
50
|
end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: pgbus
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.15.
|
|
4
|
+
version: 0.15.3
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Mikael Henriksson
|
|
@@ -424,13 +424,13 @@ files:
|
|
|
424
424
|
- lib/tasks/pgbus_pgmq.rake
|
|
425
425
|
- lib/tasks/pgbus_queues.rake
|
|
426
426
|
- lib/tasks/pgbus_streams.rake
|
|
427
|
-
homepage: https://github.com/
|
|
427
|
+
homepage: https://github.com/zoolutions/pgbus
|
|
428
428
|
licenses:
|
|
429
429
|
- MIT
|
|
430
430
|
metadata:
|
|
431
|
-
homepage_uri: https://github.com/
|
|
432
|
-
source_code_uri: https://github.com/
|
|
433
|
-
changelog_uri: https://github.com/
|
|
431
|
+
homepage_uri: https://github.com/zoolutions/pgbus
|
|
432
|
+
source_code_uri: https://github.com/zoolutions/pgbus/tree/main
|
|
433
|
+
changelog_uri: https://github.com/zoolutions/pgbus/blob/main/CHANGELOG.md
|
|
434
434
|
documentation_uri: https://pgbus.zoolutions.llc
|
|
435
435
|
rubygems_mfa_required: 'true'
|
|
436
436
|
rdoc_options: []
|