pgbus 0.13.3 → 0.13.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2e9af048ef1aa2d0fecab62f5471ca9ab6c3e2595353c7eb1a13cde57c37564a
-  data.tar.gz: 29a35e789733ac394136eaea8fc2b7f2e709617ceee818dd2b2752d9b017d45d
+  metadata.gz: 9f59e5211aa8a7dedbbca32416bf5f82943848229ea5cf0a967f3681f14c4613
+  data.tar.gz: 8db3311f9117fbb1980982bb4f0de9a25ac610ff59898e8877631fdbc5c92f5f
 SHA512:
-  metadata.gz: 392a67ed233feb5ffe0a98cf597efe3962b9e7c81d07a7e351f6c4e8aab8c03b3edc8e55204c3f0d0a4b82287b7f523cada5ca79e0684752f50c8f3272c9bdc0
-  data.tar.gz: 95b48a0ae66aaa239da08fa3c235a412241ddcfacea8384d84a66ca298308678062dc0192592b07484596fe12e27145a71474d5547b5046370bfca2785ae0f03
+  metadata.gz: 478a033b2857060eec4240e8556e789af8055fdc34fc3c17639646159e168289e1f7e0ea80443137b9980294dde3d114488abbb76317e7a2f079d2df22e79fb5
+  data.tar.gz: 9e9ae02d538d96d30f141bed4a5da3e0420841da0364b452d0aa347b6ce242f9eae605514c148805542bdd24b44f949592e0d155f58b7a20f765e70231fd04b3

data/CHANGELOG.md

@@ -2,6 +2,10 @@
 
 ### Fixed
 
+- **SSE delivery no longer strips newlines from broadcast payloads — multiline payloads are framed as consecutive `data:` lines per the SSE spec (issue #392).** `Streams::Envelope.message` collapsed `\r`/`\n` in the payload to nothing before writing the single `data:` line, silently corrupting any whitespace-significant broadcast (pre-formatted `<pre>` content, textarea seeds, JSON-in-data frames) on **both** the ephemeral and durable delivery paths — HTML's whitespace tolerance is why it went unnoticed. A multiline payload is now split on `\r\n`/`\r`/`\n` into consecutive `data:` lines, which EventSource clients rejoin with `\n`, making delivery lossless (a trailing newline survives via an empty final `data:` line; `\r` variants normalize to `\n` — SSE line terminators cannot be carried raw). The original injection defense is preserved: every payload line carries the `data:` prefix followed by one space, so a crafted payload still cannot forge `id:`/`event:` fields, and single-line fields (event names, comments) still strip newlines. The `<pgbus-stream-source>` element's fetch-path parser had the matching client-side bug — it joined `data:` lines without `\n` *and* `trim()`ed payload whitespace — and now follows EventSource semantics (join with `\n`, strip only the single leading space). Refs #392.
+
+- **Ephemeral broadcasts over the PG NOTIFY payload cap no longer fail — loudly on the sync path, silently in the coalescer — they auto-degrade to a durable publish (issue #391).** Ephemeral frames ride the NOTIFY payload itself, which PostgreSQL caps below 8000 bytes. Any rendered-component broadcast (a progress card with Tailwind classes easily exceeds it) previously raised `PGMQ::Errors::ConnectionError: … payload string too long` — an error class that sent diagnosis toward the connection, not the payload — and on the `coalesce:` path that raise happened inside the coalescer's flush thread, reaching no caller, no ErrorReporter, no log: small frames delivered, big frames vanished, and the operator saw "SSE works but updates don't arrive". Three changes: **(1)** `Stream#broadcast` now measures the wrapped JSON before the NOTIFY and publishes an over-budget frame durably instead (payload stored in PGMQ, the queue's insert trigger fires the NOTIFY as a bare wake on the same channel the subscriber already LISTENs on) — delivery semantics preserved on both the sync and coalesced paths, warn-logged and instrumented (`pgbus.stream.broadcast` with `ephemeral_fallback: true`). **(2)** Direct `Client#notify_stream` callers get publish-time validation: a typed `Pgbus::Streams::PayloadTooLarge` raised at the call site for payloads exceeding `Pgbus::Client::NotifyStream::NOTIFY_PAYLOAD_LIMIT_BYTES` (7999 bytes, the largest accepted payload), naming the stream, the byte count, and the durable-mode escape hatch. **(3)** The coalescer's flush thread routes every flush error through `ErrorReporter` (same report-don't-log reasoning as #352) — a background thread swallowing delivery failures is invisible to APM by construction. ⚠️ **Upgrade note for 0.13 installs:** `streams_default_broadcast_mode` defaults to `:ephemeral`, and that default is a **behavior change** for apps broadcasting rendered components (what `broadcast_render`-style usage produces) — before this fix, any frame over ~8KB was silently lost or misdiagnosed. Durable is the right mode for turbo-stream UI regardless (since-id replay needs the archive): pin `config.streams_default_broadcast_mode = :durable`, or use `streams_durable_patterns` for the streams that need it; the auto-fallback now covers whatever stays ephemeral. Refs #391.
+
 - **Queue-age metrics no longer count vt-parked (scheduled/retrying) messages — one delayed job stops reading as a degraded queue (issue #389).** ⚠️ **Behavior change on the AppSignal `pgbus_queue_latency` gauge.** pgmq's `oldest_msg_age_sec` is computed from `enqueued_at` and ignores `vt`, but a job enqueued with `wait:` or parked on a long retry backoff lives in the queue table with a future `vt` — that *is* the delayed-delivery mechanism. So a single parked message made the age metric grow at wall-clock rate for hours on an otherwise drained queue, and any latency alert thresholding on it fired continuously ("oldest message is 17045s old" on a healthy queue with depth 1, `read_ct` 0). Every metrics surface now also exposes **`oldest_claimable_age_sec`** — `now() - min(vt)` over rows with `vt <= now()`, i.e. the age of the oldest message actually *eligible for pickup*: an immediately-enqueued message contributes from enqueue time (matching the old number on a plain backlog), a scheduled/backoff-parked message contributes nothing until due, an in-flight message (vt pushed forward) is excluded, and nil means "no claimable backlog" even when the table is non-empty. Surfaces: `Web::DataSource` (dashboard, JSON API, MCP `pgbus_queues` tool), a new Prometheus gauge `pgbus_queue_oldest_claimable_age_seconds`, a new AppSignal gauge `pgbus_queue_oldest_claimable_age_seconds`, `Pgbus::Client#oldest_claimable_ages` (raw-SQL reader, since pgmq's `metrics_result` type is frozen upstream), and a CLAIMABLE column in `pgbus queues`. The AppSignal **`pgbus_queue_latency` gauge now derives from the claimable age** and always emits — `(claimable_age || 0) * 1000`, 0 = no claimable backlog — so existing latency alerts stop false-firing with no dashboard changes; the raw `pgbus_queue_oldest_message_age_seconds` gauge keeps its enqueue-time semantics everywhere. The dashboard queue tables additionally split depth into **Parked** (`depth − visible`) and show the claimable age in place of the raw age, so a queue holding only backoff retries reads visibly healthy. Refs #389.
 
 ### Added

data/app/assets/javascripts/pgbus/stream_source_element.js

@@ -225,13 +225,17 @@ class PgbusStreamSourceElement extends HTMLElement {
 
     let id = null
     let event = "message"
-    let data = ""
+    const dataLines = []
 
     for (const line of block.split("\n")) {
       if (line.startsWith("id:")) id = line.slice(3).trim()
       else if (line.startsWith("event:")) event = line.slice(6).trim()
-      else if (line.startsWith("data:")) data += line.slice(5).trim()
+      // Per the SSE spec: strip only a single leading space after the colon
+      // (never trim — payload whitespace is significant, issue #392) and
+      // rejoin consecutive data: lines with \n, matching native EventSource.
+      else if (line.startsWith("data:")) dataLines.push(line.slice(5).replace(/^ /, ""))
     }
+    const data = dataLines.join("\n")
 
     if (id !== null) this.lastEventId = id
 

data/lib/pgbus/client/notify_stream.rb

@@ -14,15 +14,24 @@ module Pgbus
     # no orphan tables.
     #
     # The payload is JSON-serialized into the NOTIFY's optional payload
-    # parameter (max 8000 bytes in Postgres). Broadcasts exceeding this
-    # limit will raise a PG::ProgramLimitExceeded error — callers needing
-    # large payloads should use durable mode (which inserts into PGMQ).
+    # parameter. Postgres caps NOTIFY payloads at < 8000 bytes; oversized
+    # payloads raise a typed Pgbus::Streams::PayloadTooLarge here, at the
+    # call site, instead of surfacing as a misleading
+    # PGMQ::Errors::ConnectionError ("payload string too long") from deep
+    # inside the driver (issue #391). Callers needing large payloads should
+    # use durable mode (which inserts into PGMQ).
     module NotifyStream
+      # PostgreSQL rejects NOTIFY payloads of 8000 bytes or more
+      # ("payload string too long"), so 7999 is the largest deliverable
+      # payload.
+      NOTIFY_PAYLOAD_LIMIT_BYTES = 7999
+
       def notify_stream(stream_name, payload)
         full_name = config.queue_name(stream_name)
         sanitized = QueueNameValidator.sanitize!(full_name)
         channel = "pgmq.q_#{sanitized}.INSERT"
         json = payload.is_a?(String) ? payload : JSON.generate(payload)
+        validate_notify_payload_size!(stream_name, json)
 
         Instrumentation.instrument("pgbus.stream.notify", stream: stream_name, bytes: json.bytesize) do
           with_stale_connection_retry do
@@ -34,6 +43,19 @@ module Pgbus
           end
         end
       end
+
+      private
+
+      def validate_notify_payload_size!(stream_name, json)
+        return if json.bytesize <= NOTIFY_PAYLOAD_LIMIT_BYTES
+
+        raise Pgbus::Streams::PayloadTooLarge,
+              "Ephemeral broadcast on stream #{stream_name.inspect} is #{json.bytesize} bytes; " \
+              "PostgreSQL caps NOTIFY payloads at #{NOTIFY_PAYLOAD_LIMIT_BYTES} bytes. " \
+              "Use durable mode for large payloads (payload stored in PGMQ, NOTIFY as wake) — " \
+              "e.g. broadcast(..., durable: true), a streams_durable_patterns match, or " \
+              "streams_default_broadcast_mode = :durable."
+      end
     end
   end
 end

data/lib/pgbus/streams/coalescer.rb

@@ -73,6 +73,12 @@ module Pgbus
         return unless entry
 
         @flush.call(stream_name: stream_name, target: target, payload: entry.payload, opts: entry.opts)
+      rescue StandardError => e
+        # The flush runs on the scheduler's thread — a raise here reaches no
+        # caller, so a swallowed error is invisible to APM by construction
+        # (issue #391: an oversized ephemeral frame died here without a
+        # trace). Route through ErrorReporter so configured reporters see it.
+        ErrorReporter.report(e, { component: "streams.coalescer", stream: stream_name, target: target })
       end
 
       # Default scheduler backed by Concurrent::ScheduledTask. Kept as a

data/lib/pgbus/streams/envelope.rb

@@ -12,13 +12,17 @@ module Pgbus
     #   - `comment(text)` — a heartbeat or sentinel that the SSE parser ignores
     #   - `retry_directive(ms)` — tells `EventSource` how long to wait before reconnecting
     #
-    # All frames end with `\n\n` (the SSE event terminator). `data:` lines must not
-    # contain newlines — the SSE spec uses `\n` as the field terminator, so a multi-line
-    # payload would arrive as multiple events. We strip `\r` and `\n` from data and
-    # comment text rather than splitting into multiple `data:` lines, because Turbo
-    # Stream HTML is already flat and the simpler encoding is easier to debug.
+    # All frames end with `\n\n` (the SSE event terminator). A multiline payload is
+    # framed as consecutive `data:` lines (issue #392) — the client rejoins them with
+    # `\n`, so delivery is lossless. `\r\n` and lone `\r` are also SSE line terminators,
+    # so they become `data:` line breaks too (rejoined as `\n`; SSE cannot represent a
+    # raw `\r`). Every payload line carries the `data: ` prefix, so a crafted payload
+    # cannot inject forged id:/event: fields. Single-line fields (`event:`, comments)
+    # still strip newlines — there a `\r`/`\n` would terminate the field early and
+    # permit SSE field injection.
     module Envelope
       NEWLINES = /[\r\n]+/
+      DATA_LINE_BREAK = /\r\n|\r|\n/
 
       RESPONSE_HEADERS = "HTTP/1.1 200 OK\r\n" \
                          "content-type: text/event-stream\r\n" \
@@ -31,11 +35,13 @@ module Pgbus
         raise ArgumentError, "id is required" if id.nil?
         raise ArgumentError, "event is required" if event.nil? || event.to_s.empty?
 
-        # Strip newlines from BOTH event and data, not just data: each is
-        # interpolated into its own SSE field line, so an unescaped \r/\n in
-        # either would terminate the field early and let a crafted value
-        # inject extra SSE fields (a forged id:/data:) into the frame.
-        "id: #{id}\nevent: #{strip_newlines(event.to_s)}\ndata: #{strip_newlines(data.to_s)}\n\n"
+        # The event name is a single SSE field line, so newlines are stripped —
+        # an unescaped \r/\n would terminate the field early and let a crafted
+        # value inject extra SSE fields (a forged id:/data:) into the frame.
+        # The payload is framed as one `data:` line per payload line instead:
+        # every line carries the `data: ` prefix, which is both spec-correct
+        # (the client rejoins with \n) and injection-safe.
+        "id: #{id}\nevent: #{strip_newlines(event.to_s)}\n#{data_lines(data.to_s)}\n"
       end
 
       def self.comment(text)
@@ -71,7 +77,16 @@ module Pgbus
         str.gsub(NEWLINES, "")
       end
 
-      private_class_method :strip_newlines
+      # One `data: <line>\n` per payload line. The -1 limit keeps trailing
+      # empty strings, so a payload ending in \n round-trips as an empty
+      # final `data:` line (the client's rejoin restores the newline).
+      def self.data_lines(str)
+        lines = str.split(DATA_LINE_BREAK, -1)
+        lines = [""] if lines.empty? # "".split → [] — an empty payload still gets its data: line
+        lines.map { |line| "data: #{line}\n" }.join
+      end
+
+      private_class_method :strip_newlines, :data_lines
     end
   end
 end

data/lib/pgbus/streams.rb

@@ -15,6 +15,18 @@ module Pgbus
     # this specifically can rescue Pgbus::Streams::StreamNameTooLong.
     class StreamNameTooLong < ArgumentError; end
 
+    # Raised when an ephemeral broadcast's JSON payload exceeds PostgreSQL's
+    # NOTIFY payload budget (< 8000 bytes). Ephemeral frames ride the NOTIFY
+    # itself, so the cap is a hard PostgreSQL limit — durable mode (payload
+    # stored in PGMQ, NOTIFY as a bare wake) has no such cap.
+    #
+    # Stream#broadcast never raises this: an oversized ephemeral frame
+    # auto-degrades to a durable publish (issue #391). The error exists for
+    # direct Client#notify_stream callers, where the previous failure mode
+    # was a misleading PGMQ::Errors::ConnectionError ("payload string too
+    # long") that pointed diagnosis at the connection instead of the payload.
+    class PayloadTooLarge < Pgbus::Error; end
+
     # The default SSE `event:` name for a broadcast frame. Turbo's
     # StreamObserver consumes frames the client re-dispatches as the
     # `message` DOM event; the client maps this SSE event name to
@@ -261,11 +273,47 @@ module Pgbus
 
       private
 
+      # Ephemeral frames ride the NOTIFY payload itself, which PostgreSQL
+      # caps below 8000 bytes. A frame over the cap auto-degrades to a
+      # durable publish (issue #391): payload stored in PGMQ, the queue's
+      # insert trigger fires the NOTIFY as a bare wake on the same channel
+      # the subscriber already LISTENs on — delivery semantics preserved,
+      # cap irrelevant. The JSON is generated once here and passed
+      # pre-serialized to notify_stream so the size check costs no extra
+      # allocation on the hot path.
       def broadcast_ephemeral(wrapped)
-        @client.notify_stream(@name, wrapped)
+        json = JSON.generate(wrapped)
+        return durable_fallback(wrapped, json.bytesize) if json.bytesize > Client::NotifyStream::NOTIFY_PAYLOAD_LIMIT_BYTES
+
+        @client.notify_stream(@name, json)
         nil
       end
 
+      # The durable degrade path for an oversized ephemeral frame. Stays
+      # fire-and-forget like the ephemeral path it replaces: no after_commit
+      # deferral (pg_notify runs on the PGMQ pool connection, outside the
+      # request's AR transaction, so the ephemeral path never deferred
+      # either). Returns the msg_id like any durable publish.
+      def durable_fallback(wrapped, bytes)
+        Pgbus.logger.warn do
+          "[Pgbus::Streams] ephemeral broadcast on #{@name.inspect} is #{bytes} bytes " \
+            "(NOTIFY cap is #{Client::NotifyStream::NOTIFY_PAYLOAD_LIMIT_BYTES}); " \
+            "publishing durably instead. Consider durable mode for this stream " \
+            "(streams_durable_patterns or durable: true) to skip this check."
+        end
+        ensure_queue!
+        instrument_payload = {
+          stream: @name,
+          visible_to: wrapped["visible_to"],
+          deferred: false,
+          bytes: wrapped["html"].bytesize,
+          ephemeral_fallback: true
+        }
+        Instrumentation.instrument("pgbus.stream.broadcast", instrument_payload) do
+          @client.send_stream_message(@name, wrapped)
+        end
+      end
+
       # Submits a frame to the process-wide coalescer instead of
       # broadcasting now. Requires a target (the dedupe key — there's no
       # way to last-write-win without one). The window is `coalesce` in ms

data/lib/pgbus/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Pgbus
-  VERSION = "0.13.3"
+  VERSION = "0.13.4"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: pgbus
 version: !ruby/object:Gem::Version
-  version: 0.13.3
+  version: 0.13.4
 platform: ruby
 authors:
 - Mikael Henriksson