pgbus 0.9.8 → 0.9.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +329 -0
- data/README.md +336 -25
- data/lib/generators/pgbus/{add_job_locks_generator.rb → add_uniqueness_keys_generator.rb} +5 -5
- data/lib/pgbus/active_job/adapter.rb +1 -1
- data/lib/pgbus/config_loader.rb +29 -2
- data/lib/pgbus/configuration.rb +202 -73
- data/lib/pgbus/doctor.rb +28 -3
- data/lib/pgbus/execution_pools/async_pool.rb +2 -2
- data/lib/pgbus/generators/config_converter.rb +7 -5
- data/lib/pgbus/generators/migration_detector.rb +20 -16
- data/lib/pgbus/instrumentation.rb +2 -1
- data/lib/pgbus/integrations/appsignal/subscriber.rb +17 -2
- data/lib/pgbus/metrics/subscriber.rb +26 -2
- data/lib/pgbus/metrics.rb +3 -2
- data/lib/pgbus/pgmq_schema/pgmq_v1.11.1.sql +2126 -0
- data/lib/pgbus/pgmq_schema.rb +7 -2
- data/lib/pgbus/process/consumer.rb +141 -12
- data/lib/pgbus/process/supervisor.rb +33 -10
- data/lib/pgbus/process/worker.rb +6 -16
- data/lib/pgbus/recurring/schedule.rb +1 -2
- data/lib/pgbus/serializer.rb +4 -4
- data/lib/pgbus/streams/broadcastable_override.rb +0 -8
- data/lib/pgbus/streams/signed_name.rb +2 -2
- data/lib/pgbus/uniqueness.rb +11 -12
- data/lib/pgbus/version.rb +1 -1
- data/lib/pgbus/web/data_source.rb +20 -4
- data/lib/pgbus/web/payload_filter.rb +3 -3
- data/lib/pgbus/web/streamer/instance.rb +6 -4
- data/lib/pgbus/web/streamer/listener.rb +21 -36
- data/lib/pgbus.rb +46 -6
- metadata +4 -6
- data/app/models/pgbus/job_lock.rb +0 -98
- data/lib/generators/pgbus/templates/add_job_locks.rb.erb +0 -21
- data/lib/rubocop/cop/pgbus/no_ruby_timeout.rb +0 -42
- data/lib/rubocop/pgbus.rb +0 -5
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 9848fb03a422ec1c28eda5f7c021ae41ecbdba5a9d4b8246635e14ea10d32213
|
|
4
|
+
data.tar.gz: 766159bb02748854335f90fac50b5fa4bbae82307bcbeacea366d3428a5b6c75
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 6da2678bd4e68b0d18fd264af33f9bb463b1944b33db8ec4ecfc0dfdf20167275b7b8067fa9c5b082e3ff9294185c6ea8a9c71538c63704b03165e6d85690e07
|
|
7
|
+
data.tar.gz: c39c733fe2c91107adedd58d1ae27b94123681e87cb7ec3ab15d60338f7b99b49fe05152f07ece13ab1a94efca4a68bdfce8a234bdbffe87ebd51fc153185135
|
data/CHANGELOG.md
CHANGED
|
@@ -1,7 +1,45 @@
|
|
|
1
1
|
## [Unreleased]
|
|
2
2
|
|
|
3
|
+
### Breaking Changes
|
|
4
|
+
|
|
5
|
+
- **The error hierarchy is unified under `Pgbus::Error` for the 1.0 API freeze.** Several call sites raised bare stdlib errors that no `rescue Pgbus::Error` could catch, freezing an inconsistent contract into 1.0. Now: **`Configuration#validate!` and every config setter raise `Pgbus::ConfigurationError`** (was `ArgumentError`) — this is the one that can break existing code: a `rescue ArgumentError` around `Pgbus.configure` or a boot-time config read will no longer catch config failures; switch to `rescue Pgbus::Error` (or `rescue Pgbus::ConfigurationError`). The ActiveJob adapter's `perform_all_later` msg_id-count mismatch now raises **`Pgbus::EnqueueError`** (new, was `RuntimeError`); `ExecutionPools::AsyncPool` shutdown/at-capacity raises now use **`Pgbus::ExecutionPoolError`** (new, was `RuntimeError`); `Serializer#locate_global_id` GlobalID rejections raise **`Pgbus::SerializationError`** (was `ArgumentError`). Four error classes that bypassed the base — `PgmqSchema::VersionNotFoundError`, `Streams::SignedName::InvalidSignedName`/`MissingSecret`, and `Generators::ConfigConverter::Error` — are re-parented under `Pgbus::Error`. Three classes that reject a malformed *argument shape* — `Configuration::CapsuleDSL::ParseError`, `Streams::Cursor::InvalidCursor`, `Streams::StreamNameTooLong` — **deliberately stay `ArgumentError` subclasses**; the policy ("argument-shape errors are `ArgumentError`, operational errors are `Pgbus::Error`") is documented in `lib/pgbus.rb` and pinned by `spec/pgbus/error_hierarchy_spec.rb`. Messages are unchanged. Refs #282.
|
|
6
|
+
|
|
7
|
+
### Added
|
|
8
|
+
|
|
9
|
+
- **`Pgbus.publish` / `Pgbus.publish_later` top-level shortcuts and a `config.drain_timeout` key (1.0 API freeze).** The event bus now has the same top-level ergonomics as streams: `Pgbus.publish(routing_key, payload, headers:, delay:)` and `Pgbus.publish_later(routing_key, payload, delay:, headers:)` delegate to `Pgbus::EventBus::Publisher` (long form still works), so tutorials no longer fossilize the fully-qualified call. `config.drain_timeout` (validated positive number, default 30) replaces the hardcoded `Worker::DRAIN_TIMEOUT = 30` constant — the graceful-shutdown drain deadline is now operator-tunable, so jobs that legitimately run longer than 30s aren't silently abandoned on every deploy/recycle. `pgbus doctor` gains a seventh check that warns (never fails) when `allowed_global_id_models` is `nil` (allow-all) in a Rails production env — the default is kept for upgrade continuity but is security-relevant. Refs #283.
|
|
10
|
+
|
|
11
|
+
### Changed
|
|
12
|
+
|
|
13
|
+
- **1.0 API freeze: config renames as deprecated aliases, and removal of provably-dead surface.** Renaming now is cheap; renaming after 1.0 is a major-version break, so the semver-audit renames land as **deprecated aliases** (old name works, warns once, removed in 2.0): `skip_recurring` → `recurring_enabled` (positive polarity, the only negative toggle among the `*_enabled` switches — the alias inverts the boolean); `dashboard_filter_parameters`/`dashboard_filter_sensitive` → `web_filter_parameters`/`web_filter_sensitive` (unify on the incumbent `web_` prefix); and `recurring_tasks_file` (singular), which now warns once when set alongside the plural `recurring_tasks_files` instead of being silently ignored. `ConfigLoader` accepts both spellings automatically (it dispatches on `respond_to?`). **Removed** (each verified zero-caller in the audit): the `lock_ttl:` keyword on `ensures_uniqueness` (validated and stored in metadata but read by nothing — passing it now raises `ArgumentError` naming the removal and the upgrade guide); the `pgbus:add_job_locks` generator and the `Pgbus::JobLock` model (new installs use `pgbus:add_uniqueness_keys`; `pgbus:migrate_job_locks` still retires the legacy `pgbus_job_locks` table); the internal `with_pgbus_durable` streams helper (use `with_pgbus_broadcast_opts(durable:)`); and the streamer's `reconnect_via_reset` fallback (`connection_factory` is now required and always injected, so reconnect always rebuilds a fresh connection). `log_format=` no longer clobbers a custom logger's formatter, and `streams_presence_patterns`/`streams_presence_member`, `group_mode`, and `streams_falcon_streaming_body` are annotated **experimental** (exempt from the 1.0 stability promise). The `workers=` Array form is documented as permanent (not "legacy") — it is the only way to express N identical anonymous capsules. Refs #283.
|
|
14
|
+
|
|
15
|
+
### Fixed
|
|
16
|
+
|
|
17
|
+
### Security
|
|
18
|
+
|
|
19
|
+
## [0.9.8.1] - 2026-07-04
|
|
20
|
+
|
|
21
|
+
### Changed
|
|
22
|
+
|
|
23
|
+
- **CI now proves the `railties >= 7.1, < 9.0` support matrix at both endpoints, not just the top.** Every CI leg previously resolved a single `Gemfile.lock` pinned to the latest 8.x, so Rails 7.1 — the lower bound the gemspec promises — was never exercised; shipping 1.0 with an untested support claim is a semver-integrity risk. Added `gemfiles/rails_7_1.gemfile` (+ a checked-in `gemfiles/rails_7_1.gemfile.lock`) that pins the Rails components to `~> 7.1.0` via a plain `BUNDLE_GEMFILE` override (no appraisal gem), and the `test` job now runs the unit suite (`spec/pgbus/ spec/generators/`) against both the main 8.x Gemfile (Ruby 3.3/3.4/4.0) and the 7.1 endpoint (Ruby 3.3 floor + 4.0 ceiling). The 7.1 legs pass green — the only fix needed was test-side: `spec/pgbus/web/data_source_spec.rb` mocked the ActiveRecord connection but let real model side-paths (`QueueState.paused`, `RecurringTask.count`) hit it, and AR 7.1's eager schema load raised RSpec's `MockExpectationError` (an `Exception`, not a `StandardError`, so the methods' rescue missed it); AR 8.1 resolves schema lazily and never surfaced the gap. The main Gemfile now reads an optional `RAILS_VERSION` env override (defaulting to the full supported range) so the endpoint gemfile can pin the lower bound without duplicating gem declarations. A `lint_floor` CI job also runs rubocop + `rake build` on Ruby 3.3 (the gemspec's `required_ruby_version` floor). Refs #284.
|
|
24
|
+
|
|
25
|
+
### Fixed
|
|
26
|
+
|
|
27
|
+
- **Metrics: `pgbus.consumer.recycle` now reaches both metrics subscribers, and `pgbus.client.pool` reaches `Metrics::Subscriber`.** Since v0.9.7 (#200, #205) two instrumentation events had no metrics consumer: `pgbus.client.pool` (emitted per worker heartbeat) was subscribed by neither `Metrics::Subscriber` nor the AppSignal subscriber — Prometheus/StatsD users got zero pool visibility, and even AppSignal only saw pool gauges via the separate minutely `Probe#track_pool` path; `pgbus.consumer.recycle` was subscribed by neither, so consumer recycling (#205) was invisible everywhere while `pgbus.worker.recycle` alone incremented `pgbus_worker_recycled`. `Metrics::Subscriber` now subscribes both: `pgbus.client.pool` forwards `size`/`available` as `pgbus_pool_size` / `pgbus_pool_available` gauges tagged `hostname` (matching the AppSignal probe's naming), and `pgbus.consumer.recycle` increments the *existing* `pgbus_worker_recycled` counter (no new metric) tagged `kind: "consumer"` alongside `reason`; `pgbus.worker.recycle` now also carries `kind: "worker"` so the tag is always present. The AppSignal subscriber gains the same `pgbus.consumer.recycle` handling but deliberately does **not** subscribe `pgbus.client.pool` — the minutely probe already covers AppSignal, and subscribing both would double-report and skew the gauges (documented as a comment at the subscription site). Also corrected the drifted `lib/pgbus/metrics.rb` docstring, which claimed a stale event count, to point at `Pgbus::Instrumentation` as the single source of truth for the emitted `pgbus.*` event catalog. Refs #275.
|
|
28
|
+
|
|
3
29
|
### Added
|
|
4
30
|
|
|
31
|
+
- **CI: nightly dependency-watch Action for the PGMQ SQL and pgmq-ruby upstreams.** Pgbus hand-tracks two upstreams with nothing watching them: the vendored PGMQ SQL (`lib/pgbus/pgmq_schema/pgmq_v*.sql`) and the `pgmq-ruby` gem (`~> 0.7.0` in `pgbus.gemspec`). New `.github/workflows/dependency-watch.yml` runs nightly (`0 5 * * *`, plus `workflow_dispatch`) and does two plain bash + `gh api`/`curl` + `jq` checks — no Ruby application boot: (1) derives the highest vendored PGMQ version from the `pgmq_v*.sql` filenames (never hardcoded) and compares it against `pgmq/pgmq`'s `releases/latest` tag; (2) resolves the locked `pgmq-ruby` version via `bundle lock` (Gemfile.lock is gitignored for this gem, so a fresh checkout has none) and compares it against the RubyGems API. Both comparisons use `sort -V` so a newer patch release (e.g. 1.11.0 → 1.11.1) is caught correctly. On drift it opens a GitHub issue titled `Upstream release: <dep> <version>`, deduplicated by an exact-title search so repeat runs are a no-op, and best-effort posts to Slack via the optional `SLACK_WEBHOOK_URL` repo secret (logged and skipped, never failing the job, when unset). Notification only — it never auto-vendors SQL or auto-bumps the gem. Refs #287.
|
|
32
|
+
|
|
33
|
+
- **Process model: Consumer brought to full operational parity with Worker — stall detection, circuit breaker, StatBuffer, and pool heartbeat.** #205 gave `Pgbus::Process::Consumer` recycling and NOTIFY wake-up, but four safety-net mechanisms Worker relies on were never carried over, leaving a consumer fork invisible to every operational net. All four now mirror Worker directly (no shared module — the third process type will trigger the extraction). **Stall detection:** the consumer stamps a wall-clock loop beacon (`@loop_tick_at`) at the top of every loop iteration and pokes an OS-level liveness pipe when the supervisor forked one — exactly like Worker — and its heartbeat now persists `loop_tick_at` via a `loop_tick_supplier`. `Supervisor#check_stalled_workers` and `#db_loop_tick_ages` were widened from worker-only to `%w[worker consumer]`, and `fork_consumer` now wires the same liveness pipe (`liveness_reader`/`last_pipe_tick_at`/`pipe_seen`) `fork_worker` does, so a wedged consumer is SIGKILLed and restarted through both the DB-row and the DB-independent pipe channel. `Web::DataSource#stall_threshold_for` gained a `"consumer"` branch — previously it returned `nil`, so `derive_process_status` reported a wedged consumer as `:healthy` forever. The consumer's empty-read wait can legitimately reach `NOTIFY_FALLBACK_POLL_SECONDS` (15s) when a live listener drives wake-up, so the threshold widens by `base + [polling_interval, NOTIFY_FALLBACK_POLL_SECONDS].max` when notify wake-up is enabled and `base + polling_interval` otherwise (mirroring how dispatcher/scheduler widen by their sleep interval). **Circuit breaker:** the consumer now builds a `Pgbus::CircuitBreaker` and consults it in the fetch path — a tripped queue is skipped instead of hammered forever — records success/failure per handled message, and rescues `Pgbus::ConnectionCircuitOpenError` before the generic rescue to idle the poll with no `ErrorReporter` call during a database outage (so a consumer pool no longer floods the error tracker). **StatBuffer:** when `config.stats_enabled`, the consumer builds a `Pgbus::StatBuffer` (using `stats_flush_size`/`stats_flush_interval`), records a `success`/`failed`/`dead_lettered` stat per handled message into the same `pgbus_job_stats` table workers use, and flushes at drain entry, on recycle, and on stop. **Pool heartbeat:** the consumer's heartbeat `on_beat` now emits `pgbus.client.pool` once per beat, so consumer processes surface connection-pool utilization like workers. Worker, Dispatcher, and Scheduler behavior is unchanged. Refs #274.
|
|
34
|
+
|
|
35
|
+
- **PGMQ: vendored schema v1.11.1, proving the multi-version upgrade path.** Pgbus vendored exactly one PGMQ schema (`lib/pgbus/pgmq_schema/pgmq_v1.11.0.sql`), so `PgmqSchema.latest_version`, the `pgbus_pgmq_schema_versions` tracking table, `rails g pgbus:upgrade_pgmq`, and `rake pgbus:pgmq:status`/`:versions` had never been exercised against a real delta. Vendoring upstream PGMQ v1.11.1 (2026-04-19) alongside v1.11.0 (keeping both) gives version discovery a genuine second target: dropping `pgmq_v1.11.1.sql` into the schema dir makes `available_versions` list both and `latest_version` return `"1.11.1"` with no code change (the loader globs `pgmq_v*.sql` and sorts by `Gem::Version`). The v1.11.1 delta is purely additive/cosmetic — the upstream release relocated the `pg_monitor` GRANT block to just after `pgmq.meta` and reworded one `pg_dump` comment; every function body is byte-identical to what pgbus already shipped (which was already a customized superset including `read_grouped_head` plus pgbus topic-routing and notify-insert extensions), so no function pgbus calls through `Pgbus::Client` (`create`, `send`, `read`, `archive`, `delete`, `purge_queue`, `metrics`, …) changed. No new `Client` API was added for `read_grouped_head` (the locked pgmq-ruby 0.7.0 client doesn't expose it and pgbus has no consumer). Proving the upgrade path end-to-end (install v1.11.0 → run the generated `upgrade_pgmq` migration's SQL → v1.11.1 with existing queues and messages surviving) surfaced and fixed a latent bug in `PgmqSchema.drop_pgmq_functions_sql`: its composite-type drop loop skipped every standalone `CREATE TYPE … AS (…)` (they get a `pg_class` shell with `relkind = 'c'`, which the old `pg_class.reltype` guard treated as a table-backed row-type), so `message_record`/`queue_record`/`metrics_result` were never dropped and the subsequent `install_sql`'s bare `CREATE TYPE` raised "type already exists" — meaning any upgrade migration failed. The guard now only skips true relation row-types (`relkind <> 'c'`), so the drop covers the composite type shells and the upgrade re-creates them cleanly. A DB-backed integration spec (guarded by `PGBUS_DATABASE_URL`, skipped cleanly without it) proves the full round-trip. Refs #286.
|
|
36
|
+
|
|
37
|
+
- **Docs: backfilled seven streams features shipped since v0.9.7 (#165–#171) that were name-dropped or missing from the docs site and README.** The [Real-time streams](https://pgbus.zoolutions.llc/docs/streams) page gains a "Broadcast options" table (`exclude:`, `event:`, `coalesce:`, `durable:`, `visible_to:`) plus focused subsections: `exclude:`/actor-echo suppression with the full `X-Pgbus-Connection` header round-trip and `pgbus:connected` frame (#165); `broadcast_render`'s three renderable resolutions — `String` → `#call` → `#render_in` → `#to_s` — and content-less actions (#166); typed SSE event names — server `event:` kwarg → SSE frame → `pgbus:<event>`/`pgbus:event` client dispatch, with the `listen-events` reconnect caveat (#170); `coalesce:` publish-side debounce semantics — trailing-edge-with-max-wait, safe only for idempotent replace/update, superseded frames never hit PGMQ (#171); connection-driven presence — auto-join/leave/touch lifecycle, the context-extractor contract, alongside the existing manual presence API (#169); `stream_key` single-String idempotency and `stream_key!` (#167); and the `msg_id`/revision client-side reconciliation recipe — track the highest applied msgId per target, skip stale morphs (#168). `ConfigReference::streams_presence_patterns`/`streams_presence_member` moved from `INTERNAL_ONLY` to the "Streams (SSE)" `GROUPS` entry in `docs/app/models/config_reference.rb` since they're user-facing opt-in configuration, unlike the genuinely internal streams tuning keys around them — the drift spec (`docs/spec/config_reference_spec.rb`) keeps the move honest. README's "Real-time broadcasts" section gets the same parity pass. No runtime code changed. Refs #280.
|
|
38
|
+
|
|
39
|
+
- **Docs: backfill operations & DX documentation shipped since 0.9.7.** `pgbus doctor`/`rake pgbus:doctor`, `pgbus dlq` (list/show/retry/retry-all/purge), the health endpoints (`Pgbus::Web::HealthApp`, `/livez`/`/readyz`, standalone `health_port`/`health_bind`), eager configuration validation (`eager_validation`), `Client#pool_stats`, the supervisor boot diagnostics banner, per-worker throughput rates on the dashboard's Processes panel, and libpq-native read timeouts (dedicated-connection `statement_timeout`/`tcp_user_timeout` vs. shared-AR `database.yml` guidance) were implemented in prior releases but undocumented on both the docs site and the README. Backfilled into `docs/app/views/docs/pages/cli_generators.rb`, `observability.rb`, `running_workers.rb`, `configuration.rb`, and `dashboard.rb`, plus matching README sections — the client-level `Client::ConnectionHealth` circuit breaker gets its own subsection distinguishing it from the per-queue `Pgbus::CircuitBreaker`, since conflating the two is the likeliest source of operator confusion. Refs #279.
|
|
40
|
+
|
|
41
|
+
- **Docs: "Upgrading pgbus" guide covering the 0.9.7 → 0.9.8 → 1.0.0 path.** No version-to-version upgrade narrative existed — the Migrate group covered switching from other gems only, and the `pgbus:update` story lived as a callout inside the installation page. New `docs/app/views/docs/pages/upgrading_pgbus.rb` (Migrate group) documents the standard upgrade procedure every hop follows (`bundle update pgbus` → `rails generate pgbus:update --dry-run`/apply → `rails db:migrate` → `rake pgbus:pgmq:status` + `rails generate pgbus:upgrade_pgmq` if the vendored PGMQ schema moved → deploy → `bundle exec pgbus doctor`), with a rolling-deploy note on heartbeat/metadata compatibility across mixed versions. A 0.9.x → 0.9.8 section documents the two behavior changes (queue-name validation now rejects dashes; eager configuration validation at boot, with the `eager_validation = false` escape hatch) plus the PGMQ 1.11.0 → 1.11.1 vendored schema bump as a worked example, and links the new opt-in features (`metrics_backend`, `health_port`, `pgbus dlq`, `pgbus doctor`). A forward-looking 0.9.8 → 1.0.0 section stubs the semver commitment and the planned renames/removals tracked by #282 (unified `Pgbus::Error` hierarchy) and #283 (config renames + dead-surface removal) — explicitly marked as a stub those issues must update as they land. The README's "Upgrading from an older pgbus?" callout now links the page. Refs #278.
|
|
42
|
+
|
|
5
43
|
- **Docs: a documentation site at [pgbus.zoolutions.llc](https://pgbus.zoolutions.llc).** A nested, stateless [docs-kit](https://github.com/mhenrixon/docs-kit) Rails app under `docs/` (the same monorepo pattern as docs-kit's own dogfood site and phlex-reactive) turns the single 1,786-line README into a browsable site: 25 pages across six groups (Getting started, Guide, Operations, Testing, Migrate, Reference), five hand-built theme-aware SVG flow diagrams (architecture, message lifecycle, event fanout, outbox, streams), full-text search, per-page Markdown twins, `/llms.txt`/`/llms-full.txt`, and a live read-only MCP endpoint (`POST /mcp`). Recurring tasks — previously implemented but undocumented — now have a dedicated page written from the source. The Configuration reference is generated from a data constant guarded by a drift spec that checks it against the real `Pgbus::Configuration` accessors in both directions, so a renamed or added option fails the build. The gem package is unchanged: `spec.files` globs only `app/ config/ exe/ lib/` + root files, so nothing under `docs/` ships in the gem. The three `docs/switch_from_*.md` migration guides now point to their site pages. Deployed via Kamal + the shared docs-kit reusable workflow; the docs site has its own CI job (`.github/workflows/docs-ci.yml`) scoped to `docs/**`. Refs #266.
|
|
6
44
|
|
|
7
45
|
- **Observability: per-worker throughput rates persisted in heartbeat metadata.** Each worker already tracked live throughput in a `RateCounter` (incremented on `:dequeued`/`:processed`/`:failed` and snapshotted every heartbeat via the worker's `on_beat` hook), but those rates never left the process: `Worker#stats` was in-process only and `Heartbeat#beat` persisted just `last_heartbeat_at` plus `loop_tick_at`. The dashboard's Processes page could show only static boot-time metadata (queues/threads/pid) — operators had no cluster-wide view of per-worker throughput. `Pgbus::Process::Heartbeat#initialize` now accepts a `metadata_supplier:` (a callable returning a Hash, default `nil`); `#beat` calls it **after** `@on_beat` (so the rate counter's `snapshot!` has already refreshed the rates) and merges the result into the persisted `metadata` alongside `loop_tick_at`, all inside the existing `rescue StandardError` so a raising supplier logs a warning and the timer survives. `Worker#start_heartbeat` wires a supplier returning `{ "rates" => {processed, failed, dequeued}, "jobs_processed", "jobs_failed", "in_flight" }`. Heartbeats with no supplier (supervisor, dispatcher, consumer, scheduler, outbox poller, streamer) are untouched — `beat_metadata` returns `nil` for them and leaves the column exactly as before. No SQL changes: `Web::DataSource#format_process` already passes `metadata` through, keeping the no-raw-SQL-outside-`Web::DataSource` rule intact. The Processes table renders the rates human-readable via a new `pgbus_worker_rates` helper (e.g. `12.4/s processed · 0.2/s failed`, zero rates omitted), with i18n keys added under `pgbus.processes.processes_table.rates` across all 12 locales; the raw `rates`/counter keys are hidden from the generic metadata dump. Refs #221.
|
|
@@ -63,6 +101,8 @@
|
|
|
63
101
|
|
|
64
102
|
### Fixed
|
|
65
103
|
|
|
104
|
+
- **The documented uniqueness onboarding path now creates the table the runtime actually reads.** The README's uniqueness Setup section told a new user to run `rails generate pgbus:add_job_locks`, which creates `pgbus_job_locks` — a table nothing in the runtime reads. `ensures_uniqueness` (`Pgbus::UniquenessKey` in `app/models/pgbus/uniqueness_key.rb`, `lib/pgbus/uniqueness.rb`) and the dispatcher's reaper (`cleanup_job_locks`/`reap_orphaned_uniqueness_keys`) work exclusively against `pgbus_uniqueness_keys`; the modern `add_uniqueness_keys.rb.erb` migration template existed but was reachable only through `pgbus:update`'s `MigrationDetector`, so a fresh install following the README silently never locked anything. New standalone generator `pgbus:add_uniqueness_keys` (mirrors `pgbus:add_queue_states`, including `--database=`) exposes that template directly; `MigrationDetector::GENERATOR_MAP[:uniqueness_keys]` now invokes it instead of `pgbus:migrate_job_locks`. `pgbus:add_job_locks` is not removed — installs mid-migration still legitimately need it alongside `pgbus:migrate_job_locks` — but it now prints a deprecation warning pointing new installs at `pgbus:add_uniqueness_keys`. README and the docs-site `concurrency-uniqueness`/`cli-generators` pages retargeted to the new generator. Refs #277.
|
|
105
|
+
|
|
66
106
|
- **The Turbo broadcast path now honors `streams_durable_patterns`.** `Pgbus::Streams::TurboBroadcastable#broadcast_stream_to` — the patch every `Turbo::Broadcastable` call (`broadcasts_to`, `broadcast_replace_to`, …) and phlex-reactive's `broadcast_*_to` funnel through — resolved durability from the thread-local override or `streams_default_broadcast_mode` only, then always passed an explicit `durable:` to `Pgbus.stream`. That short-circuited `Pgbus.stream`'s own resolver (`durable.nil? ? stream_durable?(name) : durable`), so `streams_durable_patterns` was dead configuration for the entire broadcast funnel: a matching pattern never promoted a broadcast to durable, and large component renders (> 8000 bytes) that were supposed to route through PGMQ instead went ephemeral through `ActionCable` and died with `PG::ProgramLimitExceeded: payload string too long`. `broadcast_stream_to` now resolves `override.nil? ? Pgbus.configuration.stream_durable?(name) : override`, letting the pattern resolver (which already falls back to `streams_default_broadcast_mode`) participate whenever no explicit thread-local override is present. Apps without `streams_durable_patterns` are unaffected; the thread-local override (set via `broadcastable_override`'s `durable:` kwarg) still wins over a matching pattern. Refs #267.
|
|
67
107
|
- **Reads are now bounded by libpq-native timeouts instead of Ruby `Timeout`.** `Pgbus::Client#with_read_timeout` wrapped every read in `Timeout.timeout(config.read_timeout, Pgbus::ReadTimeoutError)`, which interrupts via `Thread#raise`. That can fire at an arbitrary point — including mid-libpq call — and leave a pooled `PG::Connection` in a state that re-hangs or returns wrong results on reuse; on the shared-AR path it does so on a connection ActiveRecord also queries. On the dedicated-connection path (`database_url` / `connection_params`), `Client#initialize` now bakes two libpq-native bounds into the connection options: (1) a server-side `statement_timeout` (`options=-c statement_timeout=<read_timeout>ms`) so Postgres cleanly cancels an overrunning query — surfaced by pgmq-ruby as `PGMQ::Errors::ConnectionError` ("canceling statement due to statement timeout") and re-raised as `Pgbus::ReadTimeoutError`; and (2) client-side `tcp_user_timeout` + `keepalives` (sized at `read_timeout + 5s`, and only when the linked libpq is ≥ 12 — older libpq rejects the `tcp_user_timeout` conninfo keyword and would fail the connection) so a dead/hung socket makes libpq raise `PG::ConnectionBad` synchronously — no `Thread#raise`, no buffer corruption — which the circuit breaker and stale-connection retry already handle. A caller-supplied `:options` connection param (e.g. `-c search_path=…`) is preserved — the `statement_timeout` flag is appended, not overwritten. On Linux with `read_timeout` set and libpq ≥ 12, these fully cover both a slow query and a hung socket, so Ruby `Timeout` is never wired in (detected at init via `Socket.const_defined?(:TCP_USER_TIMEOUT)` and `PG.library_version`). Ruby `Timeout` remains only as a narrow last resort on a *dedicated* connection where libpq can't bound the socket — non-Linux hosts (`tcp_user_timeout`/`keepalives` are no-ops there) or libpq < 12. The shared-AR `Proc` path gets neither a baked-in bound nor the Ruby `Timeout` (pgbus doesn't own that socket); instead `Client#initialize` logs a one-time hint to configure the same libpq timeouts in `database.yml` (`variables: { statement_timeout: <ms> }` plus `tcp_user_timeout`/`keepalives`), which ActiveRecord passes straight through. A new custom RuboCop cop `Pgbus/NoRubyTimeout` bans `Timeout.timeout` project-wide (no autocorrect) to prevent regressions. Note: on the dedicated path writes gain the same connection-wide `statement_timeout` (acceptable — an enqueue that can't complete within `read_timeout` is already failing). No per-call hot-path change (`client_bench` unchanged; the added cost is a one-time transform in `initialize`). Refs #198.
|
|
68
108
|
- **Stream archive pruning no longer silently no-ops.** `Dispatcher#prune_stream_archives` read `config.archive_compaction_batch_size`, an accessor that was culled from `Configuration` into the `Pgbus::Process::Dispatcher::ARCHIVE_COMPACTION_BATCH_SIZE` constant in the 0.x config-cleanup refactor (#82). Because `prune_stream_archives` was introduced *after* that refactor and copied the pre-cull pattern, every invocation raised `NoMethodError`, which the method's outer `rescue StandardError` swallowed — so per-stream archive tables were never pruned and grew unbounded regardless of `streams_retention`/`streams_default_retention`. It now uses the constant, matching `#compact_archives`. Refs #204.
|
|
@@ -71,6 +111,295 @@
|
|
|
71
111
|
- **Defensive retry on stale pooled pgmq connections in the enqueue path.** `Pgbus::Client#send_message`, `#send_batch`, and `#publish_to_topic` now retry once when `@pgmq.produce*` raises `PGMQ::Errors::ConnectionError` with a message indicating the pooled `PG::Connection` was killed beneath pgmq-ruby — typically by PgBouncer hitting `server_idle_timeout` / `client_idle_timeout`, an admin disconnect, or a TCP RST. Observed in production as `PQsocket() can't get socket descriptor` on the first produce following an idle window. pgmq-ruby's `auto_reconnect` recovers on the *next* pool checkout, so a single retry is sufficient; non-stale errors (pool timeout, misconfiguration, unreachable database) still propagate unchanged. Upstream pgmq-ruby fix for the underlying misclassification is in-flight at mensfeld/pgmq-ruby#94.
|
|
72
112
|
- **Stale-connection retry now backs off and attempts twice.** The single immediate retry above recovers one dead pooled socket, but during a PgBouncer restart or a brief failover window the immediate second attempt lands in the same unavailable window and the error propagated — failing an enqueue the caller may never retry. `Pgbus::Client#with_stale_connection_retry` now retries a matched stale error up to `STALE_RETRY_ATTEMPTS` (2) times with a short backoff between attempts (`STALE_RETRY_DELAYS = [0.1, 0.5]`), giving a transient window time to clear. Safety is unchanged: the pattern list still matches only pre-flight/idle-socket errors (no SQL was sent), so retrying non-idempotent ops stays safe. The backoff runs error-path only — never on the success path — and the sleep is issued outside `synchronized`, so `@pgmq_mutex` is never held during a delay on the shared-connection path. No happy-path throughput or allocation change (`client_bench` identical before/after). Refs #196.
|
|
73
113
|
- **Stalled dispatcher and scheduler are no longer reported healthy on the processes page.** `Web::DataSource#derive_process_status` evaluated the loop beacon only for workers, returning `:healthy` for every other kind, and neither the dispatcher (`{ pid }`) nor the scheduler (`{ pid, tasks }`) heartbeat carried a beacon. Because heartbeats run on an independent `Concurrent::TimerTask` thread, a dispatcher wedged inside `run_maintenance` (e.g. a stuck vacuum in `run_table_maintenance`) or a scheduler stuck in `tick` kept heartbeating and showed healthy forever while doing no work. Both processes now stamp a wall-clock loop beacon (`@loop_tick_at`, a `Concurrent::AtomicReference`) at the top of each main-loop iteration and pass `loop_tick_supplier:` to their heartbeat, mirroring `Worker#stamp_loop_tick`; the heartbeat persists it as `metadata.loop_tick_at` on every beat. `derive_process_status` now evaluates the beacon for `"dispatcher"` and `"scheduler"` too, with kind-aware thresholds that account for each loop's sleep — workers keep `config.stall_threshold` (90s), dispatcher uses `stall_threshold + config.dispatch_interval`, scheduler uses `stall_threshold + config.recurring_schedule_interval` — so a beacon older than the threshold renders the `:stalled` badge. A missing beacon (an older process during a rolling deploy) stays `:healthy`, and `stall_threshold = nil` disables the check as before. Worker stall detection is unchanged. Refs #222.
|
|
114
|
+
- **Config hardening: three silent-failure holes closed.** (1) `Configuration#validate!` now validates `statsd_host`/`statsd_port`/`health_bind` unconditionally alongside the other keys added in the same window — a typo'd host or an out-of-range port previously passed boot and only failed later when the UDP socket opened or the health server bound. (2) `ConfigLoader.sectioned?` no longer misclassifies a legitimate flat `pgbus.yml` whose every top-level setting happens to be Hash-valued (`connection_params:`, `streams_retention:`, `recurring_tasks:`, …) as "sectioned" — it now also checks whether any top-level key is a real `Configuration` setter (env section names like `production` never are), and `ConfigLoader.load` logs a warning naming the current env and the available sections whenever a genuinely sectioned file has no matching section, instead of silently applying nothing. (3) `Pgbus::Generators::ConfigConverter::DEPRECATED_SETTINGS` no longer lists `pool_size` — it is a live `Configuration#pool_size` accessor operators set to override auto-tuning, not a removed setting, so `rails generate pgbus:update` no longer silently drops a deliberate override when converting legacy YAML. Refs #276.
|
|
115
|
+
|
|
116
|
+
## [0.9.7] - 2026-06-29
|
|
117
|
+
|
|
118
|
+
### Security
|
|
119
|
+
|
|
120
|
+
- **Dashboard: filter sensitive data in payload display.** New `Pgbus::Web::PayloadFilter` redacts sensitive keys (password, token, secret, api_key, etc.) as `[FILTERED]` in rendered job payloads; configurable via `dashboard_filter_parameters` and `dashboard_filter_sensitive`. Refs #186.
|
|
121
|
+
|
|
122
|
+
## [0.9.6] - 2026-06-28
|
|
123
|
+
|
|
124
|
+
### Added
|
|
125
|
+
|
|
126
|
+
- **AppSignal: magic dashboard definitions for auto-detected metrics.** Ships `dashboard.json` (main overview) plus health, throughput, and streams dashboards in `public_config` format so AppSignal auto-creates pgbus dashboards. Refs #185.
|
|
127
|
+
- **Metrics: `pgbus_queue_latency` gauge (oldest message age in ms) and `hostname` tag on all probe gauges.** Hostname tagging matches the Sidekiq/Puma convention. Refs #185.
|
|
128
|
+
|
|
129
|
+
### Changed
|
|
130
|
+
|
|
131
|
+
- **Dashboard: insights default time range is now 1h (with a new 4h option) instead of 30d,** better suited to post-deploy monitoring. Refs #184.
|
|
132
|
+
- **Dashboard: DLQ page gains pagination** with "Showing X–Y of N", prev/next controls, and a total-count badge. Refs #184.
|
|
133
|
+
- **Dashboard: UX polish** — Failed/DLQ stats card labels each number, truncated error messages show full text on hover, and queue detail pages get back-to-queues navigation. Refs #184.
|
|
134
|
+
|
|
135
|
+
## [0.9.5] - 2026-06-28
|
|
136
|
+
|
|
137
|
+
### Added
|
|
138
|
+
|
|
139
|
+
- **MCP: read-only MCP diagnostic server.** `pgbus mcp` (stdio) exposes 12 read-only tools over `Web::DataSource` (headline `pgbus_health` returns OK/DEGRADED/STALLED); payloads redacted by default, optional `PGBUS_MCP_TOKEN` gate. Refs #182.
|
|
140
|
+
- **Worker: liveness probe to detect and heal stalled claim loops.** Workers stamp a `loop_tick_at` beacon; a supervisor watchdog SIGKILLs and re-forks workers whose loop stalls past `stall_threshold` (default 90s), and client reads are wrapped in `read_timeout` (default 30s) raising `Pgbus::ReadTimeoutError`. Adds a three-state process status (healthy/stalled/stale) to the dashboard. Refs #181, #179.
|
|
141
|
+
|
|
142
|
+
## [0.9.4] - 2026-06-24
|
|
143
|
+
|
|
144
|
+
### Added
|
|
145
|
+
|
|
146
|
+
- **Streams: forward `exclude:`/`visible_to:`/`event:` through Turbo broadcast helpers.** Turbo `broadcast_*_to` helpers now thread all four pgbus opts (durable, exclude, visible_to, event) to `Pgbus.stream.broadcast`, completing the actor-echo suppression loop for gems that broadcast via standard Turbo helpers. Refs #178.
|
|
147
|
+
|
|
148
|
+
## [0.9.3] - 2026-06-22
|
|
149
|
+
|
|
150
|
+
### Added
|
|
151
|
+
|
|
152
|
+
- **Worker: NOTIFY-gated worker wakeups.** Each worker fork owns a dedicated PG connection (`NotifyListener`) that LISTENs on its queues' INSERT channels and wakes the worker immediately on insert instead of waiting for the next polling tick. On by default when `listen_notify` is enabled; configurable via `worker_notify_wakeup`, `worker_notify_host`, `worker_notify_port`, `worker_notify_database_url`. Refs #174.
|
|
153
|
+
|
|
154
|
+
### Fixed
|
|
155
|
+
|
|
156
|
+
- **Worker: prevent silent poll stall.** When `evict_missing_queues` removed all queues from a worker's list, it left an empty list that silently skipped every read forever; the list now recovers from the initial configured set. Added diagnostic logging when circuit-breaker filtering removes all active queues (previously silent), and mutex-guarded `NotifyListener` shared state with hardened reconnect. Refs #174.
|
|
157
|
+
|
|
158
|
+
## [0.9.2] - 2026-06-22
|
|
159
|
+
|
|
160
|
+
### Added
|
|
161
|
+
|
|
162
|
+
- **Streams: reactive primitives for phlex-reactive.** Idempotent `stream_key` for pre-built keys (+ `stream_key!`), actor-echo suppression via `exclude:` on broadcasts, `Stream#broadcast_render` (render + broadcast in one call), client-exposed `msg_id` revision for optimistic UI (`pgbus:message`), optional connection-driven presence (`streams_presence_patterns`), typed SSE `event:` names, and publish-side coalescing for high-frequency broadcasts (`coalesce:`). Refs #165, #166, #167, #168, #169, #170, #171, #173.
|
|
163
|
+
|
|
164
|
+
### Security
|
|
165
|
+
|
|
166
|
+
- **Streams: sanitize typed SSE event names.** `Envelope.message` strips CR/LF from `event` and the dispatcher drops non-string/blank/CRLF values, preventing forged `id:`/`data:` SSE field injection via a crafted event name. Refs #173.
|
|
167
|
+
|
|
168
|
+
### Fixed
|
|
169
|
+
|
|
170
|
+
- **Async: replace blocking sleep with IO pipe wake in reactor.** The reactor no longer polls with `Kernel#sleep`, which starved the Async scheduler and silently stalled all queue processing ~20-30s after boot under load; `post`/`shutdown`/`restore_capacity` now write a wake byte for sub-millisecond fiber-aware wake. Refs #174.
|
|
171
|
+
- **Dashboard: add `turbo_frame: _top` to batch links.** Batch links now break out of the enclosing Turbo frame. Refs #172.
|
|
172
|
+
|
|
173
|
+
## [0.9.0] - 2026-06-19
|
|
174
|
+
|
|
175
|
+
### Added
|
|
176
|
+
|
|
177
|
+
- **PGMQ v1.11 features via pgmq-ruby upgrade.** `Client` gains grouped reads (`read_grouped`, `read_grouped_rr`, `read_grouped_head`), FIFO index management (`create_fifo_index`, `create_fifo_indexes_all`), `wait_for_notify`, notify-insert throttle controls, and `convert_archive_partitioned` (pg_partman); new `pgbus:queues` / `pgbus:archives` rake tasks and group-mode config. Refs #161.
|
|
178
|
+
- **Dashboard Batches tab.** New `BatchesController` (index + show) with progress bars, status badges, job counts, and callback info; `Web::DataSource` batch methods, a `pgbus_batch_status_badge` helper, a nav link between Events and DLQ, and i18n for all 12 locales. Refs #162.
|
|
179
|
+
|
|
180
|
+
### Changed
|
|
181
|
+
|
|
182
|
+
- **Pin released pgmq-ruby v0.7.0** (`~> 0.7.0`), dropping the git-master pin. `Client#tune_autovacuum` now delegates to pgmq-ruby's `tune_autovacuum`, applying the same queue/archive vacuum and fillfactor settings in one pooled checkout. Refs #163.
|
|
183
|
+
|
|
184
|
+
## [0.8.4] - 2026-05-21
|
|
185
|
+
|
|
186
|
+
### Added
|
|
187
|
+
|
|
188
|
+
- **Streams: `streams_host` / `streams_port` / `streams_database_url` connection overrides.** Point only the Streamer's dedicated LISTEN/NOTIFY connection at a direct Postgres port while the rest of pgbus stays on a transaction-pool pooler (e.g. PgBouncer, PlanetScale Postgres), which otherwise silently drops live SSE delivery; `streams_database_url` wins over the host/port surgicals. Refs #160.
|
|
189
|
+
|
|
190
|
+
## [0.8.3] - 2026-05-20
|
|
191
|
+
|
|
192
|
+
### Fixed
|
|
193
|
+
|
|
194
|
+
- **Streams: release ActiveRecord connections after each dispatcher iteration.** The `StreamEventDispatcher` fiber kept its AR connection leased while parked on `@queue.pop`, so the next code reload's `clear_reloadable_connections!` blocked for the full rack-timeout (10s); it now calls `clear_active_connections!` in an ensure block per loop iteration. Refs #158.
|
|
195
|
+
|
|
196
|
+
## [0.8.2] - 2026-05-14
|
|
197
|
+
|
|
198
|
+
### Added
|
|
199
|
+
|
|
200
|
+
- **Streams: per-Turbo-helper durable opt-in.** All Turbo broadcast helpers now accept a `durable:` kwarg (e.g. `broadcasts_to :account, durable: true`, `broadcast_replace_to(record, durable: true)`) to control durable mode at the call site; omitting it falls back to `streams_durable_patterns` → `streams_default_broadcast_mode`. Refs #154, #153.
|
|
201
|
+
- **Streams: in-memory counters and auto-enabled stats for ephemeral broadcasts.** Adds always-on `StreamCounter` (broadcasts, active/total connections per stream, via `Concurrent::AtomicFixnum`) surfaced as "Live Stream Counters" in the Insights dashboard, and ephemeral broadcasts now always write `pgbus_stream_stats` regardless of `streams_stats_enabled`. Refs #156.
|
|
202
|
+
|
|
203
|
+
### Fixed
|
|
204
|
+
|
|
205
|
+
- **Streams: ephemeral broadcasts now reach browsers.** Ephemeral envelopes (negative `msg_id`) bypassed the durable-replay cursor check so every fresh SSE connection (since_id=0) rejected them; they now skip the cursor filter and do not advance `last_msg_id_sent`. Also fixes duplicate `DisconnectMessage`s over-decrementing `active_connections` by only decrementing when `Registry#unregister` actually removes a connection. Refs #157.
|
|
206
|
+
- **Client: wrap `notify_stream` in `with_stale_connection_retry`.** Stream wake-up notifications now survive a stale database connection. Refs #155.
|
|
207
|
+
|
|
208
|
+
## [0.8.1] - 2026-05-08
|
|
209
|
+
|
|
210
|
+
### Added
|
|
211
|
+
|
|
212
|
+
- **Streams: per-broadcast and per-stream durable opt-in.** `stream.broadcast(html, durable: true/false)` overrides the mode for a single call, and `config.streams_durable_patterns` (Array of exact strings or Regexps) resolves the mode for `Pgbus.stream(name)` before falling back to `streams_default_broadcast_mode`; `Pgbus.stream`'s `durable:` default changed from `true` to `nil` so the resolver runs. Refs #152.
|
|
213
|
+
|
|
214
|
+
## [0.8.0] - 2026-05-08
|
|
215
|
+
|
|
216
|
+
### Fixed
|
|
217
|
+
|
|
218
|
+
- **Streams: Turbo broadcasts default to ephemeral mode (PG NOTIFY only), preventing orphan PGMQ queues from accumulating unread messages.** Adds `durable: true/false` on `Stream#broadcast`, an hourly dispatcher sweep of empty stream queues, batched dashboard queue metrics, and `streams_default_broadcast_mode`/`streams_orphan_threshold`/`streams_orphan_sweep_interval` config. Refs #151, #149.
|
|
219
|
+
- **Streams: inline SSE `<script>` tag now carries a CSP nonce when available, fixing broken real-time updates under `script-src` nonce policies.** Falls back gracefully when the nonce is unavailable. Refs #150.
|
|
220
|
+
|
|
221
|
+
## [0.7.9] - 2026-05-06
|
|
222
|
+
|
|
223
|
+
### Added
|
|
224
|
+
|
|
225
|
+
- **Integrations: native AppSignal integration.** Passive `pgbus.*` subscriber (ActiveJob runs + event-bus handler invocations as background-job transactions, counters, distributions), a minutely probe reporting queue depth, oldest-message age, DLQ size, failed-events, dead tuples, MVCC horizon age, active processes, and stream stats, plus three importable dashboards (throughput/latency, health, streams). Gated on `defined?(::Appsignal)` via an Engine initializer; opt out with `config.appsignal_enabled` / `config.appsignal_probe_enabled`. Refs #148.
|
|
226
|
+
- **Instrumentation: new `AS::Notifications` events** — `pgbus.job_dead_lettered`, `pgbus.event_failed`, `pgbus.stream.broadcast`, `pgbus.outbox.publish`, `pgbus.recurring.enqueue`, `pgbus.worker.recycle`. Existing executor/handler instrument calls enriched with `job_id`, `provider_job_id`, `enqueued_at`, `arguments`, `routing_key`, `exception_object`; usable by Datadog/New Relic/OTel subscribers too. Refs #148.
|
|
227
|
+
|
|
228
|
+
## [0.7.8] - 2026-04-29
|
|
229
|
+
|
|
230
|
+
### Fixed
|
|
231
|
+
|
|
232
|
+
- **Event bus: `Handler#process` now runs inside the Rails executor** so ActiveRecord connections are checked back into the pool after each handler run, preventing connection leaks under sustained event processing. Refs #147.
|
|
233
|
+
|
|
234
|
+
## [0.7.7] - 2026-04-25
|
|
235
|
+
|
|
236
|
+
### Fixed
|
|
237
|
+
|
|
238
|
+
- **Executor: jobs now pick up code changes in development.** The executor wraps jobs in `Rails.application.reloader.wrap` in dev/test (and `executor.wrap` in production) instead of always using `executor.wrap`, so workers no longer run stale class definitions. Supports `enable_reloading` (Rails 7.1+) and `cache_classes` (older). Refs #146, closes #145.
|
|
239
|
+
|
|
240
|
+
## [0.7.6] - 2026-04-23
|
|
241
|
+
|
|
242
|
+
### Fixed
|
|
243
|
+
|
|
244
|
+
- **Client: wrap all PGMQ calls in stale-connection retry.** Added SSL-layer EOF variants ("PQconsumeInput() SSL error: unexpected eof while reading", "SSL SYSCALL error") to `STALE_CONNECTION_PATTERNS`, and extended `with_stale_connection_retry` to every `@pgmq.*` call site — reads (consumer path), ack/nack (delete/archive/set_vt), admin/dashboard ops (metrics/list/purge/drop), `ensure_queue`, and stream setup — so transient Postgres/SSL drops no longer crash worker loops or leak job redeliveries. Refs #143, #144.
|
|
245
|
+
|
|
246
|
+
## [0.7.5] - 2026-04-19
|
|
247
|
+
|
|
248
|
+
### Added
|
|
249
|
+
|
|
250
|
+
- **Maintenance: proactive table maintenance to reduce PGMQ bloat.** `fillfactor=70` on queue tables, dispatcher-driven targeted `VACUUM` via `pg_stat_user_tables` above a 10% dead-tuple threshold, and `REINDEX TABLE CONCURRENTLY`; existing installs upgrade via `rails generate pgbus:tune_fillfactor`. Refs #141.
|
|
251
|
+
- **Recurring: support multiple recurring task files.** Refs #142.
|
|
252
|
+
|
|
253
|
+
### Fixed
|
|
254
|
+
|
|
255
|
+
- **Client: retry enqueue once when a pooled pgmq connection was killed.** `send_message`, `send_batch`, and `publish_to_topic` now rescue `PGMQ::Errors::ConnectionError` and retry once when the pooled `PG::Connection` was closed server-side (e.g. behind PgBouncer); other variants propagate unchanged. Refs #140.
|
|
256
|
+
|
|
257
|
+
## [0.7.4] - 2026-04-14
|
|
258
|
+
|
|
259
|
+
### Added
|
|
260
|
+
|
|
261
|
+
- **Dashboard: pending event management.** Discard, edit-and-retry, reroute, mark-handled, and bulk-discard stuck/misrouted events in handler queues from the Events page. Refs #138.
|
|
262
|
+
|
|
263
|
+
### Fixed
|
|
264
|
+
|
|
265
|
+
- **Event bus: embed `routing_key` in the Publisher message body** so non-wildcard pattern handlers are actually invoked; previously they were silently skipped. Refs #137, closes #136.
|
|
266
|
+
|
|
267
|
+
## [0.7.3] - 2026-04-12
|
|
268
|
+
|
|
269
|
+
### Added
|
|
270
|
+
|
|
271
|
+
- **Testing: EventBus test primitives.** `Pgbus::Testing` with `:fake`/`:inline`/`:disabled` modes, a thread-safe `EventStore`, shared assertions (`assert_pgbus_published`, `perform_published_events`), RSpec `have_published_event` matchers (`.with_payload`/`.with_headers`/`.exactly`), and Minitest auto-setup/teardown; Zeitwerk-ignored so it never leaks into production. Refs #132, #131.
|
|
272
|
+
|
|
273
|
+
### Fixed
|
|
274
|
+
|
|
275
|
+
- **Streams: `streams_test_mode` prevents rack.hijack DB connection leaks in tests.** StreamApp returns a stub SSE response without hijacking or spawning polling threads; entering EventBus `fake!`/`inline!` toggles it automatically and restores it on scoped-block exit. Refs #134, #133.
|
|
276
|
+
|
|
277
|
+
## [0.7.2] - 2026-04-11
|
|
278
|
+
|
|
279
|
+
### Added
|
|
280
|
+
|
|
281
|
+
- **Observability: executor lifecycle logging and zombie message detection.** The ActiveJob executor now emits lifecycle log lines and the worker detects zombie messages, with a new configuration option and failed-event recording. Refs #130.
|
|
282
|
+
|
|
283
|
+
## [0.7.1] - 2026-04-11
|
|
284
|
+
|
|
285
|
+
### Fixed
|
|
286
|
+
|
|
287
|
+
- **Executor: fiber interrupts no longer silently leak uniqueness locks.** Under `execution_mode: :async` with `:fiber` isolation, `Async::Stop`/`Async::Cancel` (which inherit from `Exception`, not `StandardError`) propagated past the `rescue StandardError` in `Executor#execute` and `AsyncPool#perform`, skipping `release_lock` and stranding the key in `pgbus_uniqueness_keys` until VT expiry. Both rescues now catch `Exception` (still re-raising `SystemExit`/`Interrupt`/`SignalException`/`NoMemoryError`/`SystemStackError`), producing a visible `:failed` result, a `pgbus_failed_events` row, and a `pgbus.job_failed` notification; debug-level phase markers added for diagnosis. Refs #126, #129.
|
|
288
|
+
- **Bootstrap: `enable_notify_insert` is now idempotent to prevent fork deadlocks.** Multiple forked children racing `bootstrap_queues` triggered concurrent DROP+CREATE of the pgmq notify trigger under `AccessExclusiveLock`, deadlocking on the same queue table. A new `notify_trigger_current?` check skips the destructive path when the trigger already exists at the correct interval, and the supervisor now bootstraps queues in the parent before forking. Refs #125, #128.
|
|
289
|
+
|
|
290
|
+
## [0.7.0] - 2026-04-11
|
|
291
|
+
|
|
292
|
+
### Added
|
|
293
|
+
|
|
294
|
+
- **Streams: short-name helpers and tighter queue-name budget.** New `Pgbus::Streams::Key` (`stream_key`, `short_id`, `normalize`, hashing AR models to a 64-bit SHA-256 suffix), `Pgbus::Streams::Streamable` AR mixin, and top-level `Pgbus.stream_key(*parts)`; `MAX_QUEUE_NAME_LENGTH` dropped to 47 to match pgmq-ruby's runtime ceiling, and over-long names now raise `Pgbus::Streams::StreamNameTooLong` at stream construction before PGMQ is touched. Refs #120.
|
|
295
|
+
- **Error reporters: route caught exceptions to APM services.** New `Pgbus::ErrorReporter` and `config.error_reporters` array wired into all critical rescue sites (executor, worker, dispatcher, circuit breaker, supervisor, outbox, failed-event recording) so exceptions can reach Appsignal/Sentry instead of only being logged. Refs #121.
|
|
296
|
+
- **JSON log formatter and structured logging.** New `Pgbus::LogFormatter` with Text and JSON formatters; set `config.log_format = :json` for structured lines with separate timestamp/pid/tid/severity/component fields plus thread-local `ctx`. Refs #121.
|
|
297
|
+
|
|
298
|
+
### Fixed
|
|
299
|
+
|
|
300
|
+
- **Worker: cap `read_multi` total so multi-queue fetch cannot overflow the pool.** pgmq-ruby treats `qty:` as per-queue, so multi-queue capsules could fetch `queue_count * qty` messages and overflow the fixed-size execution pool, crash-looping the worker fork; `limit:` is now threaded through `Client#read_multi` and capped at pool capacity. Refs #124, #123.
|
|
301
|
+
|
|
302
|
+
## [0.6.9] - 2026-04-11
|
|
303
|
+
|
|
304
|
+
### Fixed
|
|
305
|
+
|
|
306
|
+
- **Generators: `--database` routing now honors `migrations_paths`.** All 14 migration generators previously hardcoded `db/pgbus_migrate/`, ignoring the path configured in `database.yml`; they now delegate to Rails' `db_migrate_path` via a shared `MigrationPath` module. Refs #119.
|
|
307
|
+
- **Schema: autovacuum tuning is reapplied after `db:schema:load`.** Ruby-format `schema.rb` drops `ALTER TABLE ... SET (reloptions)`, so fresh setups lost queue-table tuning; a new idempotent `pgbus:tune_autovacuum` task is hooked into `db:schema:load` (and `db:schema:load:pgbus`) to restore it. Refs #119.
|
|
308
|
+
|
|
309
|
+
## [0.6.8] - 2026-04-11
|
|
310
|
+
|
|
311
|
+
### Added
|
|
312
|
+
|
|
313
|
+
- **Dashboard: queue health observability and worker pool utilization.** Surfaces PostgreSQL vacuum stats, dead tuple counts, per-table bloat, MVCC horizon age, and worker pool utilization in the dashboard, queue detail views, and Prometheus endpoint (new `pgbus_table_dead_tuples`, `pgbus_table_bloat_ratio`, `pgbus_table_last_vacuum_age_seconds`, `pgbus_oldest_transaction_age_seconds`, `pgbus_worker_pool_capacity`, `pgbus_worker_pool_busy`, `pgbus_worker_pool_utilization` gauges). Refs #117.
|
|
314
|
+
- **Schema: autovacuum tuning for high-churn PGMQ and pgbus tables.** New `Pgbus::AutovacuumTuning` module and `pgbus:tune_autovacuum` generator apply aggressive vacuum settings to queue, archive, `pgbus_semaphores`, `pgbus_uniqueness_keys`, and `pgbus_processed_events` tables; applied on queue creation and auto-detected by `pgbus:update`. Refs #117.
|
|
315
|
+
|
|
316
|
+
### Fixed
|
|
317
|
+
|
|
318
|
+
- **i18n: fill in missing Swedish dashboard translations.** Refs #118.
|
|
319
|
+
|
|
320
|
+
## [0.6.7] - 2026-04-11
|
|
321
|
+
|
|
322
|
+
### Fixed
|
|
323
|
+
|
|
324
|
+
- **Streams: `pgbus_stream_from` now works from Rack middleware contexts.** `TurboStreamOverride` includes `Pgbus::StreamsHelper` so the helper is reachable when host tools (e.g. hotwire-livereload) call it via `ActionController::Base.helpers` outside engine views. Refs #116.
|
|
325
|
+
- **Streams: `stream_source_element.js` auto-loads for host apps.** The JS ships in `app/assets/javascripts/pgbus/` (Propshaft/Sprockets) with engine auto-precompile and importmap auto-pin, and `pgbus_stream_from` emits a module import tag so the `<pgbus-stream-source>` element activates and opens its SSE connection. Refs #116.
|
|
326
|
+
- **Streams: script + element output preserves `html_safe`.** Concatenation no longer collapses to a plain String, so the tags render instead of being HTML-escaped as visible text. Refs #116.
|
|
327
|
+
- **Streams: asset-pipeline JS imports from `@hotwired/turbo-rails`.** Matches the importmap pin name, fixing the browser `TypeError` from the unmapped `@hotwired/turbo` specifier. Refs #116.
|
|
328
|
+
- **Streams: SSE reconnect bugs fixed.** Reattached elements reconnect (`closed` reset in `connectedCallback`), a gracefully-ended fetch stream falls through to `EventSource`, and the initial `EventSource` URL carries a `?since=` watermark since the constructor cannot send `Last-Event-ID`. Refs #116.
|
|
329
|
+
|
|
330
|
+
## [0.6.6] - 2026-04-10
|
|
331
|
+
|
|
332
|
+
### Fixed
|
|
333
|
+
|
|
334
|
+
- **Streams: `turbo_stream_from` now uses the pgbus SSE transport.** `TurboStreamOverride` prepends onto `Turbo::StreamsHelper` and delegates `turbo_stream_from` to `pgbus_stream_from` when `streams_enabled`, so third-party gems (e.g. hotwire-livereload) subscribe over PGMQ/SSE instead of ActionCable — matching the publish side and delivering broadcasts. Falls through to turbo-rails when streams are disabled. Refs #115.
|
|
335
|
+
|
|
336
|
+
## [0.6.5] - 2026-04-10
|
|
337
|
+
|
|
338
|
+
### Added
|
|
339
|
+
|
|
340
|
+
- **Process: async/fiber execution mode for workers.** New `ExecutionPools` abstraction (`ThreadPool` + `AsyncPool`) runs jobs as bounded fibers on a single async reactor thread instead of a thread pool, cutting PostgreSQL connection usage for I/O-bound workloads; configurable via `config.execution_mode` (with per-worker override) and the CLI `--execution-mode` flag. Refs #112.
|
|
341
|
+
- **Streams: Falcon-native streaming body support.** New `streams_falcon_streaming_body` flag makes `StreamApp` return a `Protocol::HTTP::Body::Writable` instead of hijacking the socket, giving Falcon proper fiber-driven SSE streaming (takes priority over `rack.hijack?`). Refs #113.
|
|
342
|
+
- **Config: auto-tune pool size for async workers.** Pool-size auto-tuning now contributes 3 connections per async worker (reactor + polling + headroom) instead of one per fiber, and honors a global `config.execution_mode = :async` fallback. Refs #114.
|
|
343
|
+
|
|
344
|
+
## [0.6.4] - 2026-04-10
|
|
345
|
+
|
|
346
|
+
### Added
|
|
347
|
+
|
|
348
|
+
- **Streams: `config.streams_path` decouples the SSE endpoint URL.** Point `<pgbus-stream-source>` elements at a public route when the engine is mounted behind an auth constraint; resolution order is `config.streams_path` > engine route helper > fallback. Refs #111.
|
|
349
|
+
|
|
350
|
+
## [0.6.3] - 2026-04-10
|
|
351
|
+
|
|
352
|
+
### Added
|
|
353
|
+
|
|
354
|
+
- **Retry: exponential backoff for VT-based retries.** Failed jobs now back off exponentially via the visibility timeout instead of retrying at a fixed interval. Refs #110.
|
|
355
|
+
- **Web: Prometheus metrics export endpoint.** New endpoint exposes pgbus metrics in Prometheus format for scraping. Refs #109.
|
|
356
|
+
|
|
357
|
+
### Changed
|
|
358
|
+
|
|
359
|
+
- **Streamer: renamed `Web::Streamer::Dispatcher` to `StreamEventDispatcher`.** Disambiguates from `Process::Dispatcher`; log tags shift from `[Pgbus::Streamer::Dispatcher]` to `[Pgbus::Streamer::StreamEventDispatcher]` — update pinned log filters. Refs #98, #107.
|
|
360
|
+
|
|
361
|
+
### Fixed
|
|
362
|
+
|
|
363
|
+
- **Streams: normalize queue names with hyphens/dots instead of rejecting them.** Refs #108.
|
|
364
|
+
- **Streams: case-insensitive match for a missing PGMQ stream queue.** Refs #106.
|
|
365
|
+
- **Stats: JobStat memoization survives transient connection errors.** A single PG blip during boot no longer permanently disables job stat and latency recording for the process lifetime — only successful probes are memoized. Refs #98, #105.
|
|
366
|
+
|
|
367
|
+
## [0.6.2] - 2026-04-09
|
|
368
|
+
|
|
369
|
+
### Fixed
|
|
370
|
+
|
|
371
|
+
- **Streams: handle a missing PGMQ queue table on the first watermark read.** `Client::ReadAfter#stream_current_msg_id`, `#stream_oldest_msg_id`, and `#read_after` now rescue `PG::UndefinedTable` for the stream's own queue/archive table — treating "no queue" as "no messages" (watermark 0, oldest nil, empty replay) — so first-page renders of ephemeral per-subject streams no longer raise before the first broadcast lazily creates the table. Refs #101, #103.
|
|
372
|
+
|
|
373
|
+
## [0.6.1] - 2026-04-09
|
|
374
|
+
|
|
375
|
+
### Added
|
|
376
|
+
|
|
377
|
+
- **Generators: `pgbus:update` auto-detects and installs missing migrations.** Inspects the live DB schema against known pgbus tables/columns/indexes and invokes the matching sub-generators (e.g. `add_presence`, `add_stream_stats`) in one pass; auto-passes `--database=<name>` when a separate database is configured. Refs #100.
|
|
378
|
+
|
|
379
|
+
### Changed
|
|
380
|
+
|
|
381
|
+
- **Web: dashboard pagination and failed-archive pushed to SQL.** Moves paging and the failed-events archive off in-memory handling and into SQL queries. Refs #95.
|
|
382
|
+
|
|
383
|
+
### Fixed
|
|
384
|
+
|
|
385
|
+
- **Streams: rename `StreamStat` scopes to avoid turbo-rails collision.** `broadcasts`/`connects`/`disconnects` renamed to `broadcast_events`/`connect_events`/`disconnect_events`; the `broadcasts` scope collided with `Turbo::Broadcastable`'s class macro and crashed eager-load in any app combining streams with turbo-rails. `.summary` keys unchanged. Refs #94.
|
|
386
|
+
- **Generators: `pgbus:update` no longer leaks env-specific settings across all environments.** A setting present in only some environments (e.g. `polling_interval: 0.01` under `test:`) was emitted as an unconditional initializer line, applying test tuning to dev and prod; now classified constant only when present in every env and agreeing. Refs #96.
|
|
387
|
+
|
|
388
|
+
## [0.6.0] - 2026-04-09
|
|
389
|
+
|
|
390
|
+
### Added
|
|
391
|
+
|
|
392
|
+
- **Streams: SSE-based turbo-rails replacement.** New `Pgbus::Web::Streamer` subsystem — a threaded coordinator inside each Puma worker delivering Turbo Streams over SSE, fixing page-born-stale (rails/rails#52420) and lost-messages-on-reconnect (hotwired/turbo#1261). Adds 11 `streams_*` configuration options, `streams_helper`, `TurboBroadcastable`, watermark cache middleware, and a Puma plugin. Refs #73, #75, #76, #77.
|
|
393
|
+
- **Streams: transactional broadcasts.** Broadcasts enqueued inside a DB transaction only fire on commit. Refs #81.
|
|
394
|
+
- **Streams: Falcon server support.** `stream_app` runs under Falcon in addition to Puma. Refs #83.
|
|
395
|
+
- **Streams: `broadcasts_with_replay` via `replay:` helper option.** Late subscribers receive missed messages on connect. Refs #84.
|
|
396
|
+
- **Streams: server-side audience filtering.** New `Pgbus::Streams::Filters` scopes broadcasts per connection at the dispatcher. Refs #85.
|
|
397
|
+
- **Streams: presence tracking.** New `Pgbus::Streams::Presence` plus `add_presence` generator and migration. Refs #87.
|
|
398
|
+
- **Streams: opt-in stream stats + Insights integration.** New `StreamStat` model surfaced in the Insights dashboard. Refs #91.
|
|
399
|
+
|
|
400
|
+
### Changed
|
|
401
|
+
|
|
402
|
+
- **Streams: wake coalescing + Stream caching.** Dispatcher coalesces wake-ups and caches Stream lookups; adds a `streams_bench` benchmark harness. Refs #88.
|
|
74
403
|
|
|
75
404
|
## [0.5.1] - 2026-04-08
|
|
76
405
|
|