pgbus 0.13.2 → 0.13.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 295404bb7b5b6d289a2bfd4f10d9e5bc6df442f8d949454cf1dd78adf620a25f
4
- data.tar.gz: 2ee9c8e2c40f47bb900e10d2ef4c55b5e21e3cd4d997c614c628b696ac759183
3
+ metadata.gz: 9f59e5211aa8a7dedbbca32416bf5f82943848229ea5cf0a967f3681f14c4613
4
+ data.tar.gz: 8db3311f9117fbb1980982bb4f0de9a25ac610ff59898e8877631fdbc5c92f5f
5
5
  SHA512:
6
- metadata.gz: 184484168fd458dbdb09b98d12ae5d906ebb510a05abc626fd3812ee4cfebb5cd575c090b8014b9524a9aa19f65e99d1737df50b191ce79367e77858f72c5024
7
- data.tar.gz: 92170d5077128d507968989f8d9d789bdc9afa7b4ccf92ecdc3b39b357a82608de39a83b4e9f2f899078466714bfb30f3f0c357a9dc977f35c9c21254c8b7dbd
6
+ metadata.gz: 478a033b2857060eec4240e8556e789af8055fdc34fc3c17639646159e168289e1f7e0ea80443137b9980294dde3d114488abbb76317e7a2f079d2df22e79fb5
7
+ data.tar.gz: 9e9ae02d538d96d30f141bed4a5da3e0420841da0364b452d0aa347b6ce242f9eae605514c148805542bdd24b44f949592e0d155f58b7a20f765e70231fd04b3
data/CHANGELOG.md CHANGED
@@ -1,5 +1,13 @@
1
1
  ## [Unreleased]
2
2
 
3
+ ### Fixed
4
+
5
+ - **SSE delivery no longer strips newlines from broadcast payloads — multiline payloads are framed as consecutive `data:` lines per the SSE spec (issue #392).** `Streams::Envelope.message` collapsed `\r`/`\n` in the payload to nothing before writing the single `data:` line, silently corrupting any whitespace-significant broadcast (pre-formatted `<pre>` content, textarea seeds, JSON-in-data frames) on **both** the ephemeral and durable delivery paths — HTML's whitespace tolerance is why it went unnoticed. A multiline payload is now split on `\r\n`/`\r`/`\n` into consecutive `data:` lines, which EventSource clients rejoin with `\n`, making delivery lossless (a trailing newline survives via an empty final `data:` line; `\r` variants normalize to `\n` — SSE line terminators cannot be carried raw). The original injection defense is preserved: every payload line carries the `data:` prefix followed by one space, so a crafted payload still cannot forge `id:`/`event:` fields, and single-line fields (event names, comments) still strip newlines. The `<pgbus-stream-source>` element's fetch-path parser had the matching client-side bug — it joined `data:` lines without `\n` *and* `trim()`ed payload whitespace — and now follows EventSource semantics (join with `\n`, strip only the single leading space). Refs #392.
6
+
7
+ - **Ephemeral broadcasts over the PG NOTIFY payload cap no longer fail — loudly on the sync path, silently in the coalescer — they auto-degrade to a durable publish (issue #391).** Ephemeral frames ride the NOTIFY payload itself, which PostgreSQL caps below 8000 bytes. Any rendered-component broadcast (a progress card with Tailwind classes easily exceeds it) previously raised `PGMQ::Errors::ConnectionError: … payload string too long` — an error class that sent diagnosis toward the connection, not the payload — and on the `coalesce:` path that raise happened inside the coalescer's flush thread, reaching no caller, no ErrorReporter, no log: small frames delivered, big frames vanished, and the operator saw "SSE works but updates don't arrive". Three changes: **(1)** `Stream#broadcast` now measures the wrapped JSON before the NOTIFY and publishes an over-budget frame durably instead (payload stored in PGMQ, the queue's insert trigger fires the NOTIFY as a bare wake on the same channel the subscriber already LISTENs on) — delivery semantics preserved on both the sync and coalesced paths, warn-logged and instrumented (`pgbus.stream.broadcast` with `ephemeral_fallback: true`). **(2)** Direct `Client#notify_stream` callers get publish-time validation: a typed `Pgbus::Streams::PayloadTooLarge` raised at the call site for payloads exceeding `Pgbus::Client::NotifyStream::NOTIFY_PAYLOAD_LIMIT_BYTES` (7999 bytes, the largest accepted payload), naming the stream, the byte count, and the durable-mode escape hatch. **(3)** The coalescer's flush thread routes every flush error through `ErrorReporter` (same report-don't-log reasoning as #352) — a background thread swallowing delivery failures is invisible to APM by construction. ⚠️ **Upgrade note for 0.13 installs:** `streams_default_broadcast_mode` defaults to `:ephemeral`, and that default is a **behavior change** for apps broadcasting rendered components (what `broadcast_render`-style usage produces) — before this fix, any frame over ~8KB was silently lost or misdiagnosed. Durable is the right mode for turbo-stream UI regardless (since-id replay needs the archive): pin `config.streams_default_broadcast_mode = :durable`, or use `streams_durable_patterns` for the streams that need it; the auto-fallback now covers whatever stays ephemeral. Refs #391.
8
+
9
+ - **Queue-age metrics no longer count vt-parked (scheduled/retrying) messages — one delayed job stops reading as a degraded queue (issue #389).** ⚠️ **Behavior change on the AppSignal `pgbus_queue_latency` gauge.** pgmq's `oldest_msg_age_sec` is computed from `enqueued_at` and ignores `vt`, but a job enqueued with `wait:` or parked on a long retry backoff lives in the queue table with a future `vt` — that *is* the delayed-delivery mechanism. So a single parked message made the age metric grow at wall-clock rate for hours on an otherwise drained queue, and any latency alert thresholding on it fired continuously ("oldest message is 17045s old" on a healthy queue with depth 1, `read_ct` 0). Every metrics surface now also exposes **`oldest_claimable_age_sec`** — `now() - min(vt)` over rows with `vt <= now()`, i.e. the age of the oldest message actually *eligible for pickup*: an immediately-enqueued message contributes from enqueue time (matching the old number on a plain backlog), a scheduled/backoff-parked message contributes nothing until due, an in-flight message (vt pushed forward) is excluded, and nil means "no claimable backlog" even when the table is non-empty. Surfaces: `Web::DataSource` (dashboard, JSON API, MCP `pgbus_queues` tool), a new Prometheus gauge `pgbus_queue_oldest_claimable_age_seconds`, a new AppSignal gauge `pgbus_queue_oldest_claimable_age_seconds`, `Pgbus::Client#oldest_claimable_ages` (raw-SQL reader, since pgmq's `metrics_result` type is frozen upstream), and a CLAIMABLE column in `pgbus queues`. The AppSignal **`pgbus_queue_latency` gauge now derives from the claimable age** and always emits — `(claimable_age || 0) * 1000`, 0 = no claimable backlog — so existing latency alerts stop false-firing with no dashboard changes; the raw `pgbus_queue_oldest_message_age_seconds` gauge keeps its enqueue-time semantics everywhere. The dashboard queue tables additionally split depth into **Parked** (`depth − visible`) and show the claimable age in place of the raw age, so a queue holding only backoff retries reads visibly healthy. Refs #389.
10
+
3
11
  ### Added
4
12
 
5
13
  - **Health-checked rolling restarts for the job container (issue #386).** ⚠️ **Behavior change on the standalone `/readyz`.** The supervisor's `health_port` server previously answered `/readyz` with the cluster-wide HealthAnalyzer verdict — so during a rolling deploy a freshly-booted container could pass an orchestrator's health gate on the strength of the *old* container's still-heartbeating workers, and the old container (with all its capacity) was stopped before the new one had forked a single child. The standalone `/readyz` is now **container-local**: 200 only when *this* supervisor verified its connection, bootstrapped queues, forked every configured child, and all of them are currently alive — with 503 bodies `BOOTING` (pre-boot), `DEGRADED` (a child died and is waiting out crash-restart backoff — precisely the state a deploy gate must fail on, keeping the old container running), and `DRAINING` (stop signal received). No database access on the probe path; the supervisor publishes an immutable snapshot per monitor pass and the accept thread reads it. The Rails-mounted `Pgbus::Web::HealthApp` keeps the cluster-wide verdict unchanged. Alongside it: **`pgbus-health`**, a shipped executable probe for docker `HEALTHCHECK` blocks (plain Ruby + stdlib sockets, loads neither Bundler nor the gem — cheap at 1–5s intervals, works in curl-less images; exit 0/1/2 = healthy/unhealthy/usage), and a README "Rolling restarts (Kamal, docker)" guide covering the healthcheck block, stop-timeout alignment, overlap-window duplicate-supervisor safety, and the `read_ct`-vs-deploy-kill DLQ caveat. Refs #386.
data/README.md CHANGED
@@ -991,6 +991,7 @@ When `config.metrics_enabled = true` (default), the dashboard exposes Prometheus
991
991
 
992
992
  | Metric | Description |
993
993
  |--------|-------------|
994
+ | `pgbus_queue_oldest_claimable_age_seconds` | Age of the oldest message eligible for pickup (visibility timeout elapsed) — safe to alert on: scheduled/backoff-parked messages don't count until due; the series is omitted entirely when no claimable backlog exists (while the raw `pgbus_queue_oldest_message_age_seconds` gauge may still report a parked message's age) |
994
995
  | `pgbus_table_dead_tuples` | Dead tuple count per PGMQ table |
995
996
  | `pgbus_table_live_tuples` | Live tuple count per PGMQ table |
996
997
  | `pgbus_table_bloat_ratio` | Dead / (dead + live) per table |
@@ -225,13 +225,17 @@ class PgbusStreamSourceElement extends HTMLElement {
225
225
 
226
226
  let id = null
227
227
  let event = "message"
228
- let data = ""
228
+ const dataLines = []
229
229
 
230
230
  for (const line of block.split("\n")) {
231
231
  if (line.startsWith("id:")) id = line.slice(3).trim()
232
232
  else if (line.startsWith("event:")) event = line.slice(6).trim()
233
- else if (line.startsWith("data:")) data += line.slice(5).trim()
233
+ // Per the SSE spec: strip only a single leading space after the colon
234
+ // (never trim — payload whitespace is significant, issue #392) and
235
+ // rejoin consecutive data: lines with \n, matching native EventSource.
236
+ else if (line.startsWith("data:")) dataLines.push(line.slice(5).replace(/^ /, ""))
234
237
  }
238
+ const data = dataLines.join("\n")
235
239
 
236
240
  if (id !== null) this.lastEventId = id
237
241
 
@@ -12,7 +12,8 @@
12
12
  <th class="px-4 py-3 text-left text-xs font-medium uppercase text-gray-500"><%= t("pgbus.dashboard.queues_table.headers.queue") %></th>
13
13
  <th class="px-4 py-3 text-right text-xs font-medium uppercase text-gray-500"><%= t("pgbus.dashboard.queues_table.headers.depth") %></th>
14
14
  <th class="px-4 py-3 text-right text-xs font-medium uppercase text-gray-500"><%= t("pgbus.dashboard.queues_table.headers.visible") %></th>
15
- <th class="px-4 py-3 text-right text-xs font-medium uppercase text-gray-500"><%= t("pgbus.dashboard.queues_table.headers.oldest") %></th>
15
+ <th class="px-4 py-3 text-right text-xs font-medium uppercase text-gray-500"><%= t("pgbus.dashboard.queues_table.headers.parked") %></th>
16
+ <th class="px-4 py-3 text-right text-xs font-medium uppercase text-gray-500"><%= t("pgbus.dashboard.queues_table.headers.oldest_claimable") %></th>
16
17
  <th class="px-4 py-3 text-right text-xs font-medium uppercase text-gray-500"><%= t("pgbus.dashboard.queues_table.headers.total") %></th>
17
18
  </tr>
18
19
  </thead>
@@ -25,12 +26,13 @@
25
26
  </td>
26
27
  <td data-label="Depth" class="px-4 py-3 text-sm text-right text-gray-700"><%= pgbus_number(q[:queue_length]) %></td>
27
28
  <td data-label="Visible" class="px-4 py-3 text-sm text-right text-gray-700"><%= pgbus_number(q[:queue_visible_length]) %></td>
28
- <td data-label="Oldest" class="px-4 py-3 text-sm text-right text-gray-500"><%= q[:oldest_msg_age_sec] || "—" %></td>
29
+ <td data-label="Parked" class="px-4 py-3 text-sm text-right text-gray-500"><%= pgbus_number(q[:parked_length]) %></td>
30
+ <td data-label="Oldest claimable" class="px-4 py-3 text-sm text-right text-gray-500"><%= q[:oldest_claimable_age_sec] || "—" %></td>
29
31
  <td data-label="Total" class="px-4 py-3 text-sm text-right text-gray-500"><%= pgbus_number(q[:total_messages]) %></td>
30
32
  </tr>
31
33
  <% end %>
32
34
  <% if @queues.empty? %>
33
- <tr><td colspan="5" class="px-4 py-8 text-center text-sm text-gray-400"><%= t("pgbus.dashboard.queues_table.empty") %></td></tr>
35
+ <tr><td colspan="6" class="px-4 py-8 text-center text-sm text-gray-400"><%= t("pgbus.dashboard.queues_table.empty") %></td></tr>
34
36
  <% end %>
35
37
  </tbody>
36
38
  </table>
@@ -6,7 +6,8 @@
6
6
  <th class="px-4 py-3 text-left text-xs font-medium uppercase text-gray-500"><%= t("pgbus.queues.queues_list.headers.queue") %></th>
7
7
  <th class="px-4 py-3 text-right text-xs font-medium uppercase text-gray-500"><%= t("pgbus.queues.queues_list.headers.depth") %></th>
8
8
  <th class="px-4 py-3 text-right text-xs font-medium uppercase text-gray-500"><%= t("pgbus.queues.queues_list.headers.visible") %></th>
9
- <th class="px-4 py-3 text-right text-xs font-medium uppercase text-gray-500"><%= t("pgbus.queues.queues_list.headers.oldest") %></th>
9
+ <th class="px-4 py-3 text-right text-xs font-medium uppercase text-gray-500"><%= t("pgbus.queues.queues_list.headers.parked") %></th>
10
+ <th class="px-4 py-3 text-right text-xs font-medium uppercase text-gray-500"><%= t("pgbus.queues.queues_list.headers.oldest_claimable") %></th>
10
11
  <th class="px-4 py-3 text-right text-xs font-medium uppercase text-gray-500"><%= t("pgbus.queues.queues_list.headers.newest") %></th>
11
12
  <th class="px-4 py-3 text-right text-xs font-medium uppercase text-gray-500"><%= t("pgbus.queues.queues_list.headers.total_ever") %></th>
12
13
  <th class="px-4 py-3 text-right text-xs font-medium uppercase text-gray-500"><%= t("pgbus.queues.queues_list.headers.actions") %></th>
@@ -24,7 +25,8 @@
24
25
  </td>
25
26
  <td data-label="Depth" class="px-4 py-3 text-sm text-right font-mono text-gray-700"><%= pgbus_number(q[:queue_length]) %></td>
26
27
  <td data-label="Visible" class="px-4 py-3 text-sm text-right font-mono text-gray-700"><%= pgbus_number(q[:queue_visible_length]) %></td>
27
- <td data-label="Oldest" class="px-4 py-3 text-sm text-right text-gray-500"><%= q[:oldest_msg_age_sec] || "—" %></td>
28
+ <td data-label="Parked" class="px-4 py-3 text-sm text-right font-mono text-gray-500"><%= pgbus_number(q[:parked_length]) %></td>
29
+ <td data-label="Oldest claimable" class="px-4 py-3 text-sm text-right text-gray-500"><%= q[:oldest_claimable_age_sec] || "—" %></td>
28
30
  <td data-label="Newest" class="px-4 py-3 text-sm text-right text-gray-500"><%= q[:newest_msg_age_sec] || "—" %></td>
29
31
  <td data-label="Total" class="px-4 py-3 text-sm text-right text-gray-500"><%= pgbus_number(q[:total_messages]) %></td>
30
32
  <td data-label="Actions" class="px-4 py-3 text-sm text-right space-x-2">
@@ -51,7 +53,7 @@
51
53
  </tr>
52
54
  <% end %>
53
55
  <% if @queues.empty? %>
54
- <tr><td colspan="7" class="px-4 py-8 text-center text-sm text-gray-400"><%= t("pgbus.queues.queues_list.empty") %></td></tr>
56
+ <tr><td colspan="8" class="px-4 py-8 text-center text-sm text-gray-400"><%= t("pgbus.queues.queues_list.empty") %></td></tr>
55
57
  <% end %>
56
58
  </tbody>
57
59
  </table>
@@ -13,6 +13,9 @@
13
13
  <p class="text-sm text-gray-500 mt-1">
14
14
  <%= t("pgbus.queues.show.depth") %> <span class="font-mono"><%= @queue[:queue_length] %></span> |
15
15
  <%= t("pgbus.queues.show.visible") %> <span class="font-mono"><%= @queue[:queue_visible_length] %></span> |
16
+ <%= t("pgbus.queues.show.parked") %> <span class="font-mono"><%= @queue[:parked_length] %></span> |
17
+ <%= t("pgbus.queues.show.oldest") %> <span class="font-mono"><%= @queue[:oldest_msg_age_sec] || "—" %></span> |
18
+ <%= t("pgbus.queues.show.oldest_claimable") %> <span class="font-mono"><%= @queue[:oldest_claimable_age_sec] || "—" %></span> |
16
19
  <%= t("pgbus.queues.show.total") %> <span class="font-mono"><%= pgbus_number(@queue[:total_messages]) %></span>
17
20
  </p>
18
21
  <% end %>
@@ -65,7 +65,8 @@ da:
65
65
  empty: Ingen køer fundet
66
66
  headers:
67
67
  depth: Dybde
68
- oldest: Ældste (s)
68
+ oldest_claimable: Ældste tilgængelig (s)
69
+ parked: Parkeret
69
70
  queue: Kø
70
71
  total: Total
71
72
  visible: Synlig
@@ -475,7 +476,8 @@ da:
475
476
  actions: Handlinger
476
477
  depth: Dybde
477
478
  newest: Nyeste (s)
478
- oldest: Ældste (s)
479
+ oldest_claimable: Ældste tilgængelig (s)
480
+ parked: Parkeret
479
481
  queue: Kø
480
482
  total_ever: Total nogensinde
481
483
  visible: Synlig
@@ -516,6 +518,9 @@ da:
516
518
  scheduled: 'Planlagt:'
517
519
  timezone: 'Tidszone:'
518
520
  visible_at: 'Synlig fra:'
521
+ oldest: 'Ældste:'
522
+ oldest_claimable: 'Ældste tilgængelig:'
523
+ parked: 'Parkeret:'
519
524
  pause: Pause
520
525
  pause_confirm: Pause behandling?
521
526
  purge_confirm: Rens alle beskeder?
@@ -65,7 +65,8 @@ de:
65
65
  empty: Keine Warteschlangen gefunden
66
66
  headers:
67
67
  depth: Tiefe
68
- oldest: Älteste (s)
68
+ oldest_claimable: Älteste verfügbar (s)
69
+ parked: Geparkt
69
70
  queue: Warteschlange
70
71
  total: Gesamt
71
72
  visible: Sichtbar
@@ -475,7 +476,8 @@ de:
475
476
  actions: Aktionen
476
477
  depth: Tiefe
477
478
  newest: Neueste (s)
478
- oldest: Älteste (s)
479
+ oldest_claimable: Älteste verfügbar (s)
480
+ parked: Geparkt
479
481
  queue: Warteschlange
480
482
  total_ever: Insgesamt jemals
481
483
  visible: Sichtbar
@@ -516,6 +518,9 @@ de:
516
518
  scheduled: 'Geplant:'
517
519
  timezone: 'Zeitzone:'
518
520
  visible_at: 'Sichtbar ab:'
521
+ oldest: 'Älteste:'
522
+ oldest_claimable: 'Älteste verfügbar:'
523
+ parked: 'Geparkt:'
519
524
  pause: Pause
520
525
  pause_confirm: Verarbeitung pausieren?
521
526
  purge_confirm: Alle Nachrichten löschen?
@@ -65,7 +65,8 @@ en:
65
65
  empty: No queues found
66
66
  headers:
67
67
  depth: Depth
68
- oldest: Oldest (s)
68
+ oldest_claimable: Oldest claimable (s)
69
+ parked: Parked
69
70
  queue: Queue
70
71
  total: Total
71
72
  visible: Visible
@@ -475,7 +476,8 @@ en:
475
476
  actions: Actions
476
477
  depth: Depth
477
478
  newest: Newest (s)
478
- oldest: Oldest (s)
479
+ oldest_claimable: Oldest claimable (s)
480
+ parked: Parked
479
481
  queue: Queue
480
482
  total_ever: Total Ever
481
483
  visible: Visible
@@ -516,6 +518,9 @@ en:
516
518
  scheduled: 'Scheduled at:'
517
519
  timezone: 'Timezone:'
518
520
  visible_at: 'Visible at:'
521
+ oldest: 'Oldest:'
522
+ oldest_claimable: 'Oldest claimable:'
523
+ parked: 'Parked:'
519
524
  pause: Pause
520
525
  pause_confirm: Pause processing?
521
526
  purge_confirm: Purge all messages?
@@ -65,7 +65,8 @@ es:
65
65
  empty: No se encontraron colas
66
66
  headers:
67
67
  depth: Profundidad
68
- oldest: El más antiguo (s)
68
+ oldest_claimable: Más antiguo disponible (s)
69
+ parked: Aparcados
69
70
  queue: Cola
70
71
  total: Total
71
72
  visible: Visible
@@ -475,7 +476,8 @@ es:
475
476
  actions: Acciones
476
477
  depth: Profundidad
477
478
  newest: Más nuevo (s)
478
- oldest: Más antiguo (s)
479
+ oldest_claimable: Más antiguo disponible (s)
480
+ parked: Aparcados
479
481
  queue: Cola
480
482
  total_ever: Total acumulado
481
483
  visible: Visible
@@ -516,6 +518,9 @@ es:
516
518
  scheduled: 'Programado:'
517
519
  timezone: 'Zona horaria:'
518
520
  visible_at: 'Visible en:'
521
+ oldest: 'Más antiguo:'
522
+ oldest_claimable: 'Más antiguo disponible:'
523
+ parked: 'Aparcados:'
519
524
  pause: Pausar
520
525
  pause_confirm: "¿Pausar el procesamiento?"
521
526
  purge_confirm: "¿Purgar todos los mensajes?"
@@ -65,7 +65,8 @@ fi:
65
65
  empty: Jonot eivät löytyneet
66
66
  headers:
67
67
  depth: Syvyys
68
- oldest: Vanhin (s)
68
+ oldest_claimable: Vanhin saatavilla (s)
69
+ parked: Pysäköidyt
69
70
  queue: Jono
70
71
  total: Yhteensä
71
72
  visible: Näkyvissä
@@ -475,7 +476,8 @@ fi:
475
476
  actions: Toiminnot
476
477
  depth: Syvyys
477
478
  newest: Uusimmat (s)
478
- oldest: Vanhimmat (s)
479
+ oldest_claimable: Vanhin saatavilla (s)
480
+ parked: Pysäköidyt
479
481
  queue: Jono
480
482
  total_ever: Yhteensä koskaan
481
483
  visible: Näkyvissä
@@ -516,6 +518,9 @@ fi:
516
518
  scheduled: 'Aikataulutettu:'
517
519
  timezone: 'Aikavyöhyke:'
518
520
  visible_at: 'Näkyvissä:'
521
+ oldest: 'Vanhin:'
522
+ oldest_claimable: 'Vanhin saatavilla:'
523
+ parked: 'Pysäköidyt:'
519
524
  pause: Tauko
520
525
  pause_confirm: Keskeytetäänkö käsittely?
521
526
  purge_confirm: Tyhjennetäänkö kaikki viestit?
@@ -65,7 +65,8 @@ fr:
65
65
  empty: Aucune file d'attente trouvée
66
66
  headers:
67
67
  depth: Profondeur
68
- oldest: Le plus ancien (s)
68
+ oldest_claimable: Plus ancien disponible (s)
69
+ parked: Différés
69
70
  queue: File d'attente
70
71
  total: Total
71
72
  visible: Visible
@@ -475,7 +476,8 @@ fr:
475
476
  actions: Actions
476
477
  depth: Profondeur
477
478
  newest: Le plus récent (s)
478
- oldest: Le plus ancien (s)
479
+ oldest_claimable: Plus ancien disponible (s)
480
+ parked: Différés
479
481
  queue: File d'attente
480
482
  total_ever: Total jamais
481
483
  visible: Visible
@@ -516,6 +518,9 @@ fr:
516
518
  scheduled: 'Planifié :'
517
519
  timezone: 'Fuseau horaire :'
518
520
  visible_at: 'Visible à :'
521
+ oldest: 'Plus ancien :'
522
+ oldest_claimable: 'Plus ancien disponible :'
523
+ parked: 'Différés :'
519
524
  pause: Pause
520
525
  pause_confirm: Mettre en pause le traitement ?
521
526
  purge_confirm: Purger tous les messages ?
@@ -65,7 +65,8 @@ it:
65
65
  empty: Nessuna coda trovata
66
66
  headers:
67
67
  depth: Profondità
68
- oldest: Più vecchio (s)
68
+ oldest_claimable: Più vecchio disponibile (s)
69
+ parked: Posticipati
69
70
  queue: Coda
70
71
  total: Totale
71
72
  visible: Visibile
@@ -475,7 +476,8 @@ it:
475
476
  actions: Azioni
476
477
  depth: Profondità
477
478
  newest: Più recente (s)
478
- oldest: Più vecchio (s)
479
+ oldest_claimable: Più vecchio disponibile (s)
480
+ parked: Posticipati
479
481
  queue: Coda
480
482
  total_ever: Totale mai
481
483
  visible: Visibile
@@ -516,6 +518,9 @@ it:
516
518
  scheduled: 'Programmato:'
517
519
  timezone: 'Fuso orario:'
518
520
  visible_at: 'Visibile alle:'
521
+ oldest: 'Più vecchio:'
522
+ oldest_claimable: 'Più vecchio disponibile:'
523
+ parked: 'Posticipati:'
519
524
  pause: Pausa
520
525
  pause_confirm: Mettere in pausa l'elaborazione?
521
526
  purge_confirm: Eliminare tutti i messaggi?
@@ -65,7 +65,8 @@ ja:
65
65
  empty: キューが見つかりません
66
66
  headers:
67
67
  depth: 深さ
68
- oldest: 最古 (秒)
68
+ oldest_claimable: 取得可能な最古 (秒)
69
+ parked: 待機中
69
70
  queue: キュー
70
71
  total: 合計
71
72
  visible: 表示中
@@ -475,7 +476,8 @@ ja:
475
476
  actions: アクション
476
477
  depth: 深さ
477
478
  newest: 最新 (秒)
478
- oldest: 最古 (秒)
479
+ oldest_claimable: 取得可能な最古 (秒)
480
+ parked: 待機中
479
481
  queue: キュー
480
482
  total_ever: 合計数
481
483
  visible: 表示中
@@ -516,6 +518,9 @@ ja:
516
518
  scheduled: スケジュール済み:
517
519
  timezone: タイムゾーン:
518
520
  visible_at: 表示可能日時:
521
+ oldest: '最古:'
522
+ oldest_claimable: '取得可能な最古:'
523
+ parked: '待機中:'
519
524
  pause: 一時停止
520
525
  pause_confirm: 処理を一時停止しますか?
521
526
  purge_confirm: すべてのメッセージを削除しますか?
@@ -65,7 +65,8 @@ nb:
65
65
  empty: Ingen køer funnet
66
66
  headers:
67
67
  depth: Dybde
68
- oldest: Eldste (s)
68
+ oldest_claimable: Eldste tilgjengelig (s)
69
+ parked: Parkert
69
70
  queue: Kø
70
71
  total: Totalt
71
72
  visible: Synlig
@@ -475,7 +476,8 @@ nb:
475
476
  actions: Handlinger
476
477
  depth: Dybde
477
478
  newest: Nyeste (s)
478
- oldest: Eldste (s)
479
+ oldest_claimable: Eldste tilgjengelig (s)
480
+ parked: Parkert
479
481
  queue: Kø
480
482
  total_ever: Totalt noensinne
481
483
  visible: Synlig
@@ -516,6 +518,9 @@ nb:
516
518
  scheduled: 'Planlagt:'
517
519
  timezone: 'Tidssone:'
518
520
  visible_at: 'Synlig fra:'
521
+ oldest: 'Eldste:'
522
+ oldest_claimable: 'Eldste tilgjengelig:'
523
+ parked: 'Parkert:'
519
524
  pause: Pause
520
525
  pause_confirm: Pause behandling?
521
526
  purge_confirm: Rens alle meldinger?
@@ -65,7 +65,8 @@ nl:
65
65
  empty: Geen wachtrijen gevonden
66
66
  headers:
67
67
  depth: Diepte
68
- oldest: Oudste (s)
68
+ oldest_claimable: Oudste beschikbaar (s)
69
+ parked: Geparkeerd
69
70
  queue: Wachtrij
70
71
  total: Totaal
71
72
  visible: Zichtbaar
@@ -475,7 +476,8 @@ nl:
475
476
  actions: Acties
476
477
  depth: Diepte
477
478
  newest: Nieuwste (s)
478
- oldest: Oudste (s)
479
+ oldest_claimable: Oudste beschikbaar (s)
480
+ parked: Geparkeerd
479
481
  queue: Wachtrij
480
482
  total_ever: Totaal ooit
481
483
  visible: Zichtbaar
@@ -516,6 +518,9 @@ nl:
516
518
  scheduled: 'Gepland:'
517
519
  timezone: 'Tijdzone:'
518
520
  visible_at: 'Zichtbaar op:'
521
+ oldest: 'Oudste:'
522
+ oldest_claimable: 'Oudste beschikbaar:'
523
+ parked: 'Geparkeerd:'
519
524
  pause: Pauzeren
520
525
  pause_confirm: Verwerking pauzeren?
521
526
  purge_confirm: Alle berichten verwijderen?
@@ -65,7 +65,8 @@ pt:
65
65
  empty: Nenhuma fila encontrada
66
66
  headers:
67
67
  depth: Profundidade
68
- oldest: Mais antigo (s)
68
+ oldest_claimable: Mais antigo disponível (s)
69
+ parked: Adiados
69
70
  queue: Fila
70
71
  total: Total
71
72
  visible: Visível
@@ -475,7 +476,8 @@ pt:
475
476
  actions: Ações
476
477
  depth: Profundidade
477
478
  newest: Mais novo (s)
478
- oldest: Mais antigo (s)
479
+ oldest_claimable: Mais antigo disponível (s)
480
+ parked: Adiados
479
481
  queue: Fila
480
482
  total_ever: Total de todos os tempos
481
483
  visible: Visível
@@ -516,6 +518,9 @@ pt:
516
518
  scheduled: 'Agendado:'
517
519
  timezone: 'Fuso horário:'
518
520
  visible_at: 'Visível em:'
521
+ oldest: 'Mais antigo:'
522
+ oldest_claimable: 'Mais antigo disponível:'
523
+ parked: 'Adiados:'
519
524
  pause: Pausar
520
525
  pause_confirm: Pausar processamento?
521
526
  purge_confirm: Limpar todas as mensagens?
@@ -65,7 +65,8 @@ sv:
65
65
  empty: Inga köer hittades
66
66
  headers:
67
67
  depth: Djup
68
- oldest: Äldst (s)
68
+ oldest_claimable: Äldsta tillgänglig (s)
69
+ parked: Parkerade
69
70
  queue: Kö
70
71
  total: Totalt
71
72
  visible: Synliga
@@ -475,7 +476,8 @@ sv:
475
476
  actions: Åtgärder
476
477
  depth: Djup
477
478
  newest: Nyaste (s)
478
- oldest: Äldsta (s)
479
+ oldest_claimable: Äldsta tillgänglig (s)
480
+ parked: Parkerade
479
481
  queue: Kö
480
482
  total_ever: Totalt någonsin
481
483
  visible: Synliga
@@ -516,6 +518,9 @@ sv:
516
518
  scheduled: 'Schemalagt:'
517
519
  timezone: 'Tidszon:'
518
520
  visible_at: 'Synlig vid:'
521
+ oldest: 'Äldsta:'
522
+ oldest_claimable: 'Äldsta tillgänglig:'
523
+ parked: 'Parkerade:'
519
524
  pause: Pausa
520
525
  pause_confirm: Pausa bearbetning?
521
526
  purge_confirm: Rensa alla meddelanden?
data/lib/pgbus/cli.rb CHANGED
@@ -177,14 +177,16 @@ module Pgbus
177
177
  def list_queues
178
178
  Pgbus.client.list_queues
179
179
  metrics = Pgbus.client.metrics
180
+ claimable_ages = Pgbus.client.oldest_claimable_ages
180
181
 
181
- puts "QUEUE DEPTH VISIBLE OLDEST (s) TOTAL "
182
- puts "-" * 95
182
+ puts "QUEUE DEPTH VISIBLE OLDEST (s) CLAIMABLE (s) TOTAL "
183
+ puts "-" * 111
183
184
 
184
185
  Array(metrics).each do |m|
185
- puts format("%-40s %-10s %-10s %-15s %-15s",
186
+ puts format("%-40s %-10s %-10s %-15s %-15s %-15s",
186
187
  m.queue_name, m.queue_length, m.queue_visible_length,
187
- m.oldest_msg_age_sec || "-", m.total_messages)
188
+ m.oldest_msg_age_sec || "-", claimable_ages[m.queue_name] || "-",
189
+ m.total_messages)
188
190
  end
189
191
  end
190
192
 
@@ -14,15 +14,24 @@ module Pgbus
14
14
  # no orphan tables.
15
15
  #
16
16
  # The payload is JSON-serialized into the NOTIFY's optional payload
17
- # parameter (max 8000 bytes in Postgres). Broadcasts exceeding this
18
- # limit will raise a PG::ProgramLimitExceeded error callers needing
19
- # large payloads should use durable mode (which inserts into PGMQ).
17
+ # parameter. Postgres caps NOTIFY payloads at < 8000 bytes; oversized
18
+ # payloads raise a typed Pgbus::Streams::PayloadTooLarge here, at the
19
+ # call site, instead of surfacing as a misleading
20
+ # PGMQ::Errors::ConnectionError ("payload string too long") from deep
21
+ # inside the driver (issue #391). Callers needing large payloads should
22
+ # use durable mode (which inserts into PGMQ).
20
23
  module NotifyStream
24
+ # PostgreSQL rejects NOTIFY payloads of 8000 bytes or more
25
+ # ("payload string too long"), so 7999 is the largest deliverable
26
+ # payload.
27
+ NOTIFY_PAYLOAD_LIMIT_BYTES = 7999
28
+
21
29
  def notify_stream(stream_name, payload)
22
30
  full_name = config.queue_name(stream_name)
23
31
  sanitized = QueueNameValidator.sanitize!(full_name)
24
32
  channel = "pgmq.q_#{sanitized}.INSERT"
25
33
  json = payload.is_a?(String) ? payload : JSON.generate(payload)
34
+ validate_notify_payload_size!(stream_name, json)
26
35
 
27
36
  Instrumentation.instrument("pgbus.stream.notify", stream: stream_name, bytes: json.bytesize) do
28
37
  with_stale_connection_retry do
@@ -34,6 +43,19 @@ module Pgbus
34
43
  end
35
44
  end
36
45
  end
46
+
47
+ private
48
+
49
+ def validate_notify_payload_size!(stream_name, json)
50
+ return if json.bytesize <= NOTIFY_PAYLOAD_LIMIT_BYTES
51
+
52
+ raise Pgbus::Streams::PayloadTooLarge,
53
+ "Ephemeral broadcast on stream #{stream_name.inspect} is #{json.bytesize} bytes; " \
54
+ "PostgreSQL caps NOTIFY payloads at #{NOTIFY_PAYLOAD_LIMIT_BYTES} bytes. " \
55
+ "Use durable mode for large payloads (payload stored in PGMQ, NOTIFY as wake) — " \
56
+ "e.g. broadcast(..., durable: true), a streams_durable_patterns match, or " \
57
+ "streams_default_broadcast_mode = :durable."
58
+ end
37
59
  end
38
60
  end
39
61
  end
data/lib/pgbus/client.rb CHANGED
@@ -543,6 +543,38 @@ module Pgbus
543
543
  end
544
544
  end
545
545
 
546
+ # Age (seconds) of the oldest message actually eligible for pickup, i.e.
547
+ # whose visibility timeout has elapsed. Unlike pgmq's oldest_msg_age_sec
548
+ # (computed from enqueued_at), a scheduled or backoff-parked message —
549
+ # future vt — contributes nothing until it comes due, so a queue holding
550
+ # only parked messages reads nil ("no claimable backlog") instead of an
551
+ # age growing at wall-clock rate (issue #389). pgmq's metrics_result type
552
+ # is frozen upstream, so this lives here rather than in the SQL function.
553
+ #
554
+ # With a queue name: the age for that (prefixed) queue, or nil.
555
+ # Without: a hash of every physical queue in pgmq.meta to its age.
556
+ #
557
+ # Routes through the pooled @pgmq.with_connection (health-checked, bounded
558
+ # by the statement/socket timeouts applied at Client#initialize) rather
559
+ # than a fresh unbounded PG.connect per call — same rationale as
560
+ # notify_trigger_current?. synchronized: on the shared-Proc path @pgmq
561
+ # rides the AR raw connection, so the query must serialize against
562
+ # concurrent PGMQ operations. One checkout spans all per-queue queries;
563
+ # nothing nests inside it, so the shared pool_size=1 path is safe.
564
+ def oldest_claimable_ages(queue_name = nil)
565
+ synchronized do
566
+ @pgmq.with_connection do |conn|
567
+ if queue_name
568
+ claimable_age_for(conn, config.queue_name(queue_name))
569
+ else
570
+ names = conn.exec("SELECT queue_name FROM pgmq.meta ORDER BY queue_name")
571
+ .map { |row| row["queue_name"] }
572
+ names.to_h { |name| [name, claimable_age_for(conn, name)] }
573
+ end
574
+ end
575
+ end
576
+ end
577
+
546
578
  # Snapshot of the PGMQ connection pool: {size:, available:, pool_timeout:}.
547
579
  #
548
580
  # Reads pgmq-ruby's own pool counters (@pgmq.stats -> {size:, available:})
@@ -948,6 +980,18 @@ module Pgbus
948
980
  end
949
981
  end
950
982
 
983
+ # queue_name is a physical (already prefixed) queue name; sanitized to a
984
+ # bare identifier before interpolation, same as the dashboard's DataSource.
985
+ def claimable_age_for(conn, queue_name)
986
+ qtable = "q_#{QueueNameValidator.sanitize!(queue_name)}"
987
+ row = conn.exec(<<~SQL).first
988
+ SELECT EXTRACT(epoch FROM (NOW() - min(vt)))::int AS age_sec
989
+ FROM pgmq.#{qtable}
990
+ WHERE vt <= NOW()
991
+ SQL
992
+ row && row["age_sec"]&.to_i
993
+ end
994
+
951
995
  def with_raw_connection
952
996
  opts = config.connection_options
953
997
  owned = false
@@ -85,10 +85,14 @@ module Pgbus
85
85
  gauge "queue_visible_depth", q[:queue_visible_length], tags
86
86
  gauge "queue_paused", q[:paused] ? 1 : 0, tags
87
87
  age = q[:oldest_msg_age_sec]
88
- if age
89
- gauge "queue_oldest_message_age_seconds", age, tags
90
- gauge "queue_latency", age * 1_000, tags
91
- end
88
+ gauge "queue_oldest_message_age_seconds", age, tags if age
89
+ claimable_age = q[:oldest_claimable_age_sec]
90
+ gauge "queue_oldest_claimable_age_seconds", claimable_age, tags if claimable_age
91
+ # Latency = time the oldest *claimable* message has waited for
92
+ # pickup; a queue holding only vt-parked (scheduled/backoff)
93
+ # messages is healthy, so 0 — not the raw enqueued_at age, which
94
+ # grows at wall-clock rate on a parked message (issue #389).
95
+ gauge "queue_latency", (claimable_age || 0) * 1_000, tags
92
96
  end
93
97
  rescue StandardError => e
94
98
  log_failure("queue metrics", e)
@@ -14,6 +14,12 @@ module Pgbus
14
14
  (messages whose visibility timeout has expired and are ready to be
15
15
  claimed), oldest/newest message age in seconds, lifetime total, and
16
16
  paused state. Use this to answer "are any queues backed up?".
17
+ oldest_claimable_age_sec is the age of the oldest message actually
18
+ eligible for pickup — nil means no message is currently claimable:
19
+ every remaining message is scheduled, backoff-parked, or in flight
20
+ with a future visibility timeout. A queue whose oldest_msg_age_sec
21
+ keeps growing while oldest_claimable_age_sec stays nil has no
22
+ starving backlog — nothing is waiting for a worker.
17
23
  DESC
18
24
 
19
25
  input_schema(properties: {}, required: [])
@@ -73,6 +73,12 @@ module Pgbus
73
73
  return unless entry
74
74
 
75
75
  @flush.call(stream_name: stream_name, target: target, payload: entry.payload, opts: entry.opts)
76
+ rescue StandardError => e
77
+ # The flush runs on the scheduler's thread — a raise here reaches no
78
+ # caller, so a swallowed error is invisible to APM by construction
79
+ # (issue #391: an oversized ephemeral frame died here without a
80
+ # trace). Route through ErrorReporter so configured reporters see it.
81
+ ErrorReporter.report(e, { component: "streams.coalescer", stream: stream_name, target: target })
76
82
  end
77
83
 
78
84
  # Default scheduler backed by Concurrent::ScheduledTask. Kept as a
@@ -12,13 +12,17 @@ module Pgbus
12
12
  # - `comment(text)` — a heartbeat or sentinel that the SSE parser ignores
13
13
  # - `retry_directive(ms)` — tells `EventSource` how long to wait before reconnecting
14
14
  #
15
- # All frames end with `\n\n` (the SSE event terminator). `data:` lines must not
16
- # contain newlines the SSE spec uses `\n` as the field terminator, so a multi-line
17
- # payload would arrive as multiple events. We strip `\r` and `\n` from data and
18
- # comment text rather than splitting into multiple `data:` lines, because Turbo
19
- # Stream HTML is already flat and the simpler encoding is easier to debug.
15
+ # All frames end with `\n\n` (the SSE event terminator). A multiline payload is
16
+ # framed as consecutive `data:` lines (issue #392) the client rejoins them with
17
+ # `\n`, so delivery is lossless. `\r\n` and lone `\r` are also SSE line terminators,
18
+ # so they become `data:` line breaks too (rejoined as `\n`; SSE cannot represent a
19
+ # raw `\r`). Every payload line carries the `data: ` prefix, so a crafted payload
20
+ # cannot inject forged id:/event: fields. Single-line fields (`event:`, comments)
21
+ # still strip newlines — there a `\r`/`\n` would terminate the field early and
22
+ # permit SSE field injection.
20
23
  module Envelope
21
24
  NEWLINES = /[\r\n]+/
25
+ DATA_LINE_BREAK = /\r\n|\r|\n/
22
26
 
23
27
  RESPONSE_HEADERS = "HTTP/1.1 200 OK\r\n" \
24
28
  "content-type: text/event-stream\r\n" \
@@ -31,11 +35,13 @@ module Pgbus
31
35
  raise ArgumentError, "id is required" if id.nil?
32
36
  raise ArgumentError, "event is required" if event.nil? || event.to_s.empty?
33
37
 
34
- # Strip newlines from BOTH event and data, not just data: each is
35
- # interpolated into its own SSE field line, so an unescaped \r/\n in
36
- # either would terminate the field early and let a crafted value
37
- # inject extra SSE fields (a forged id:/data:) into the frame.
38
- "id: #{id}\nevent: #{strip_newlines(event.to_s)}\ndata: #{strip_newlines(data.to_s)}\n\n"
38
+ # The event name is a single SSE field line, so newlines are stripped
39
+ # an unescaped \r/\n would terminate the field early and let a crafted
40
+ # value inject extra SSE fields (a forged id:/data:) into the frame.
41
+ # The payload is framed as one `data:` line per payload line instead:
42
+ # every line carries the `data: ` prefix, which is both spec-correct
43
+ # (the client rejoins with \n) and injection-safe.
44
+ "id: #{id}\nevent: #{strip_newlines(event.to_s)}\n#{data_lines(data.to_s)}\n"
39
45
  end
40
46
 
41
47
  def self.comment(text)
@@ -71,7 +77,16 @@ module Pgbus
71
77
  str.gsub(NEWLINES, "")
72
78
  end
73
79
 
74
- private_class_method :strip_newlines
80
+ # One `data: <line>\n` per payload line. The -1 limit keeps trailing
81
+ # empty strings, so a payload ending in \n round-trips as an empty
82
+ # final `data:` line (the client's rejoin restores the newline).
83
+ def self.data_lines(str)
84
+ lines = str.split(DATA_LINE_BREAK, -1)
85
+ lines = [""] if lines.empty? # "".split → [] — an empty payload still gets its data: line
86
+ lines.map { |line| "data: #{line}\n" }.join
87
+ end
88
+
89
+ private_class_method :strip_newlines, :data_lines
75
90
  end
76
91
  end
77
92
  end
data/lib/pgbus/streams.rb CHANGED
@@ -15,6 +15,18 @@ module Pgbus
15
15
  # this specifically can rescue Pgbus::Streams::StreamNameTooLong.
16
16
  class StreamNameTooLong < ArgumentError; end
17
17
 
18
+ # Raised when an ephemeral broadcast's JSON payload exceeds PostgreSQL's
19
+ # NOTIFY payload budget (< 8000 bytes). Ephemeral frames ride the NOTIFY
20
+ # itself, so the cap is a hard PostgreSQL limit — durable mode (payload
21
+ # stored in PGMQ, NOTIFY as a bare wake) has no such cap.
22
+ #
23
+ # Stream#broadcast never raises this: an oversized ephemeral frame
24
+ # auto-degrades to a durable publish (issue #391). The error exists for
25
+ # direct Client#notify_stream callers, where the previous failure mode
26
+ # was a misleading PGMQ::Errors::ConnectionError ("payload string too
27
+ # long") that pointed diagnosis at the connection instead of the payload.
28
+ class PayloadTooLarge < Pgbus::Error; end
29
+
18
30
  # The default SSE `event:` name for a broadcast frame. Turbo's
19
31
  # StreamObserver consumes frames the client re-dispatches as the
20
32
  # `message` DOM event; the client maps this SSE event name to
@@ -261,11 +273,47 @@ module Pgbus
261
273
 
262
274
  private
263
275
 
276
+ # Ephemeral frames ride the NOTIFY payload itself, which PostgreSQL
277
+ # caps below 8000 bytes. A frame over the cap auto-degrades to a
278
+ # durable publish (issue #391): payload stored in PGMQ, the queue's
279
+ # insert trigger fires the NOTIFY as a bare wake on the same channel
280
+ # the subscriber already LISTENs on — delivery semantics preserved,
281
+ # cap irrelevant. The JSON is generated once here and passed
282
+ # pre-serialized to notify_stream so the size check costs no extra
283
+ # allocation on the hot path.
264
284
  def broadcast_ephemeral(wrapped)
265
- @client.notify_stream(@name, wrapped)
285
+ json = JSON.generate(wrapped)
286
+ return durable_fallback(wrapped, json.bytesize) if json.bytesize > Client::NotifyStream::NOTIFY_PAYLOAD_LIMIT_BYTES
287
+
288
+ @client.notify_stream(@name, json)
266
289
  nil
267
290
  end
268
291
 
292
+ # The durable degrade path for an oversized ephemeral frame. Stays
293
+ # fire-and-forget like the ephemeral path it replaces: no after_commit
294
+ # deferral (pg_notify runs on the PGMQ pool connection, outside the
295
+ # request's AR transaction, so the ephemeral path never deferred
296
+ # either). Returns the msg_id like any durable publish.
297
+ def durable_fallback(wrapped, bytes)
298
+ Pgbus.logger.warn do
299
+ "[Pgbus::Streams] ephemeral broadcast on #{@name.inspect} is #{bytes} bytes " \
300
+ "(NOTIFY cap is #{Client::NotifyStream::NOTIFY_PAYLOAD_LIMIT_BYTES}); " \
301
+ "publishing durably instead. Consider durable mode for this stream " \
302
+ "(streams_durable_patterns or durable: true) to skip this check."
303
+ end
304
+ ensure_queue!
305
+ instrument_payload = {
306
+ stream: @name,
307
+ visible_to: wrapped["visible_to"],
308
+ deferred: false,
309
+ bytes: wrapped["html"].bytesize,
310
+ ephemeral_fallback: true
311
+ }
312
+ Instrumentation.instrument("pgbus.stream.broadcast", instrument_payload) do
313
+ @client.send_stream_message(@name, wrapped)
314
+ end
315
+ end
316
+
269
317
  # Submits a frame to the process-wide coalescer instead of
270
318
  # broadcasting now. Requires a target (the dedupe key — there's no
271
319
  # way to last-write-win without one). The window is `coalesce` in ms
data/lib/pgbus/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Pgbus
4
- VERSION = "0.13.2"
4
+ VERSION = "0.13.4"
5
5
  end
@@ -1177,7 +1177,8 @@ module Pgbus
1177
1177
  (SELECT EXTRACT(epoch FROM (NOW() - min(enqueued_at)))::int FROM pgmq.#{qtable}) AS oldest_msg_age_sec,
1178
1178
  (SELECT CASE WHEN is_called THEN last_value ELSE 0 END FROM pgmq.#{seq_name}) AS total_messages,
1179
1179
  (SELECT max(read_ct) FROM pgmq.#{qtable}) AS max_read_ct,
1180
- (SELECT count(*) FROM pgmq.#{qtable} WHERE vt <= NOW() AND read_ct = 0) AS visible_unread_length
1180
+ (SELECT count(*) FROM pgmq.#{qtable} WHERE vt <= NOW() AND read_ct = 0) AS visible_unread_length,
1181
+ (SELECT EXTRACT(epoch FROM (NOW() - min(vt)))::int FROM pgmq.#{qtable} WHERE vt <= NOW()) AS oldest_claimable_age_sec
1181
1182
  SQL
1182
1183
  rescue StandardError => e
1183
1184
  Pgbus.logger.debug { "[Pgbus::Web] Skipping queue metrics for #{name}: #{e.message}" }
@@ -1193,7 +1194,9 @@ module Pgbus
1193
1194
  name: row["queue_name"],
1194
1195
  queue_length: row["queue_length"].to_i,
1195
1196
  queue_visible_length: row["queue_visible_length"].to_i,
1197
+ parked_length: row["queue_length"].to_i - row["queue_visible_length"].to_i,
1196
1198
  oldest_msg_age_sec: row["oldest_msg_age_sec"]&.to_i,
1199
+ oldest_claimable_age_sec: row["oldest_claimable_age_sec"]&.to_i,
1197
1200
  newest_msg_age_sec: row["newest_msg_age_sec"]&.to_i,
1198
1201
  total_messages: row["total_messages"].to_i,
1199
1202
  max_read_ct: row["max_read_ct"]&.to_i,
@@ -1216,6 +1219,7 @@ module Pgbus
1216
1219
  count(CASE WHEN vt <= NOW() THEN 1 END) AS queue_visible_length,
1217
1220
  EXTRACT(epoch FROM (NOW() - max(enqueued_at)))::int AS newest_msg_age_sec,
1218
1221
  EXTRACT(epoch FROM (NOW() - min(enqueued_at)))::int AS oldest_msg_age_sec,
1222
+ EXTRACT(epoch FROM (NOW() - min(vt) FILTER (WHERE vt <= NOW())))::int AS oldest_claimable_age_sec,
1219
1223
  max(read_ct) AS max_read_ct,
1220
1224
  count(CASE WHEN vt <= NOW() AND read_ct = 0 THEN 1 END) AS visible_unread_length
1221
1225
  FROM pgmq.#{qtable}
@@ -1229,6 +1233,7 @@ module Pgbus
1229
1233
  q_summary.queue_visible_length,
1230
1234
  q_summary.newest_msg_age_sec,
1231
1235
  q_summary.oldest_msg_age_sec,
1236
+ q_summary.oldest_claimable_age_sec,
1232
1237
  q_summary.max_read_ct,
1233
1238
  q_summary.visible_unread_length,
1234
1239
  all_metrics.total_messages
@@ -1241,7 +1246,9 @@ module Pgbus
1241
1246
  name: queue_name,
1242
1247
  queue_length: row["queue_length"].to_i,
1243
1248
  queue_visible_length: row["queue_visible_length"].to_i,
1249
+ parked_length: row["queue_length"].to_i - row["queue_visible_length"].to_i,
1244
1250
  oldest_msg_age_sec: row["oldest_msg_age_sec"]&.to_i,
1251
+ oldest_claimable_age_sec: row["oldest_claimable_age_sec"]&.to_i,
1245
1252
  newest_msg_age_sec: row["newest_msg_age_sec"]&.to_i,
1246
1253
  total_messages: row["total_messages"].to_i,
1247
1254
  max_read_ct: row["max_read_ct"]&.to_i,
@@ -54,6 +54,15 @@ module Pgbus
54
54
  end
55
55
  end
56
56
 
57
+ gauge(lines, "pgbus_queue_oldest_claimable_age_seconds",
58
+ "Age of the oldest message eligible for pickup (visibility timeout elapsed)") do
59
+ queues.filter_map do |q|
60
+ next unless q[:oldest_claimable_age_sec]
61
+
62
+ [q[:oldest_claimable_age_sec], { queue: q[:name] }]
63
+ end
64
+ end
65
+
57
66
  gauge(lines, "pgbus_queue_paused", "Whether the queue is paused (1) or active (0)") do
58
67
  queues.map { |q| [q[:paused] ? 1 : 0, { queue: q[:name] }] }
59
68
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: pgbus
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.13.2
4
+ version: 0.13.4
5
5
  platform: ruby
6
6
  authors:
7
7
  - Mikael Henriksson