pgbus 0.10.0 → 0.11.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +11 -0
- data/Rakefile +49 -4
- data/lib/generators/pgbus/add_failed_events_index_generator.rb +1 -1
- data/lib/generators/pgbus/add_job_stats_generator.rb +1 -1
- data/lib/generators/pgbus/add_job_stats_latency_generator.rb +1 -1
- data/lib/generators/pgbus/add_job_stats_queue_index_generator.rb +1 -1
- data/lib/generators/pgbus/add_outbox_generator.rb +1 -1
- data/lib/generators/pgbus/add_presence_generator.rb +1 -1
- data/lib/generators/pgbus/add_queue_states_generator.rb +1 -1
- data/lib/generators/pgbus/add_recurring_generator.rb +1 -1
- data/lib/generators/pgbus/add_stream_queues_generator.rb +1 -1
- data/lib/generators/pgbus/add_stream_stats_generator.rb +1 -1
- data/lib/generators/pgbus/add_uniqueness_keys_generator.rb +1 -1
- data/lib/generators/pgbus/install_generator.rb +5 -2
- data/lib/generators/pgbus/migrate_job_locks_generator.rb +1 -1
- data/lib/generators/pgbus/migration_path.rb +73 -7
- data/lib/generators/pgbus/tune_autovacuum_generator.rb +1 -1
- data/lib/generators/pgbus/tune_fillfactor_generator.rb +1 -1
- data/lib/generators/pgbus/upgrade_pgmq_generator.rb +1 -1
- data/lib/pgbus/active_job/adapter.rb +11 -0
- data/lib/pgbus/configuration.rb +124 -52
- data/lib/pgbus/event_bus/registry.rb +61 -2
- data/lib/pgbus/process/worker.rb +9 -0
- data/lib/pgbus/streams/phlex_helpers.rb +32 -0
- data/lib/pgbus/uniqueness.rb +31 -0
- data/lib/pgbus/version.rb +1 -1
- data/lib/pgbus/web/authentication.rb +23 -0
- data/lib/pgbus.rb +18 -0
- metadata +2 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: a52dc949281b6f2f1e9c847d38709fcc2723badd12370486faeb68b40ed55c42
|
|
4
|
+
data.tar.gz: 70a75a14bfe2225f63846314cbb60c0942e86b966c14b20e43eec74afc63adef
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: dc59f389bf2b5e20b2e75a7a2c47301c242ea0b5857a240e5a59db3177effe34cbe694903d4396b07ef3ca67699f578a5d6b6d95f61d4318169ac477f237b526
|
|
7
|
+
data.tar.gz: a5a6a076c6cadb857c8690ef7d1363934c7f73bd76e8d91a87a936a91d05c4146c12bb67b5637204995d0ddef540833a8983ca4dd3d5ef1034a7b88473aa1e61
|
data/CHANGELOG.md
CHANGED
|
@@ -2,11 +2,16 @@
|
|
|
2
2
|
|
|
3
3
|
### Breaking Changes
|
|
4
4
|
|
|
5
|
+
- **`config.validate!` now rejects malformed values for 12 core config keys at boot instead of failing deep in a worker or silently misbehaving (issue #335).** Before the 1.0 surface freeze, these headline job-path keys had **zero** validation while streams keys had 14 checks — so a bad value surfaced as a crash inside a worker/dispatcher/poller/scheduler thread, per-enqueue, or by silently corrupting every derived queue name / leaving the dashboard open. `Pgbus.configure` (eager-validating by default) now raises `Pgbus::ConfigurationError` naming the key for: the worker-recycling trio `max_jobs_per_worker`/`max_memory_mb`/`max_worker_lifetime` (positive number or `nil` to disable); the intervals `dispatch_interval`/`outbox_poll_interval`/`recurring_schedule_interval` (positive number); `outbox_batch_size` (positive integer); `default_priority` (non-negative integer — `0` is a valid level); `queue_prefix`/`default_queue` (non-empty String — an empty prefix silently produced malformed queue names); `web_auth` (a callable or `nil` — a non-callable silently left the dashboard open); `error_reporters` (an Array); and `connects_to` (a Hash or `nil` — previously a malformed value raised a raw `TypeError` deep in engine boot). This only rejects configuration that was already broken; if a `Pgbus.configure` block set one of these to an invalid value it will now fail loudly at boot (the intended pre-freeze tightening). Set `config.eager_validation = false` to defer. Refs #335, #282.
|
|
6
|
+
- **`config.streams_queue_prefix` is removed (issue #335).** It had been an inert no-op since #308 — stream queues are named like job queues (`#{queue_prefix}_<name>`) and identified via the `pgbus_stream_queues` registry, so the prefix affected nothing — yet it was still a writable accessor documented as "Prefix for stream queues" (false). Rather than carry a dead, misleadingly-documented key into the 1.0 semver promise, the accessor is **removed**: `c.streams_queue_prefix = ...` now raises `NoMethodError`. **Action:** delete the line from your initializer — it never did anything. Refs #335, #308.
|
|
7
|
+
- **`ensures_uniqueness strategy: :until_executed` with no `key:` now raises when the job is enqueued *with arguments* — the class-name default silently collapsed per-record jobs (issue #333).** Without an explicit `key:`, the uniqueness key defaulted to the job class name. For a job that takes per-record arguments (`ImportOrderJob.perform_later(order_id)`), every distinct `order_id` resolved to the same class-name key and collapsed into ONE per-class singleton — order 1 and order 2 shared a lock, so the second was silently discarded/rejected — with no warning. The guard is at **resolve time, argument-aware**: a no-key `:until_executed` job enqueued **with arguments** now raises `ArgumentError` naming the collapse and pointing at the upgrade guide; **a no-argument job keeps the class-name default** (a legitimate "one logical instance at a time" — e.g. a recurring `CleanupJob` that must not overlap itself — is unaffected). `:while_executing` is unaffected (it acquires per-invocation at execution start, not by class-name identity at enqueue). Fix: pass `key: ->(*args) { "job-#{args.first}" }` on any argument-taking `:until_executed` job. Refs #333.
|
|
5
8
|
- **YAML config (`config/pgbus.yml`) is removed for 1.0 — the Ruby initializer is the only config surface.** Ruby was always the real config surface (every production setup is a `Pgbus.configure` initializer); `config/pgbus.yml` was only ever a legacy convenience, and pre-1.0 is the last low-risk moment to drop it before the config shape becomes a compatibility promise. Concretely: the engine no longer loads `config/pgbus.yml` at boot (the `pgbus.configure` initializer and `Pgbus::ConfigLoader` are deleted); `rails generate pgbus:install` generates `config/initializers/pgbus.rb` (done for 1.0, see below); and `rails generate pgbus:update` no longer converts YAML → initializer (`Pgbus::Generators::ConfigConverter`, its `--source`/`--destination`/`--skip-config` options, and the `pgbus.yml.erb` template are deleted) — `pgbus:update` now only detects and installs missing migrations. **If you still have a `config/pgbus.yml`, it is no longer loaded** — the engine warns once at boot (pointing at the docs) so the inert file doesn't quietly diverge from your real config. Port its settings into `config/initializers/pgbus.rb` as a `Pgbus.configure` block and delete the YAML; the last release that auto-converts it via `pgbus:update` is 0.9.x, so run that conversion (or port by hand) **before** upgrading to 1.0. The recurring-tasks YAML (`config/recurring.yml`, `Pgbus::Recurring::ConfigLoader`) is a separate surface and is unchanged. Refs #317.
|
|
6
9
|
- **The error hierarchy is unified under `Pgbus::Error` for the 1.0 API freeze.** Several call sites raised bare stdlib errors that no `rescue Pgbus::Error` could catch, freezing an inconsistent contract into 1.0. Now: **`Configuration#validate!` and every config setter raise `Pgbus::ConfigurationError`** (was `ArgumentError`) — this is the one that can break existing code: a `rescue ArgumentError` around `Pgbus.configure` or a boot-time config read will no longer catch config failures; switch to `rescue Pgbus::Error` (or `rescue Pgbus::ConfigurationError`). The ActiveJob adapter's `perform_all_later` msg_id-count mismatch now raises **`Pgbus::EnqueueError`** (new, was `RuntimeError`); `ExecutionPools::AsyncPool` shutdown/at-capacity raises now use **`Pgbus::ExecutionPoolError`** (new, was `RuntimeError`); `Serializer#locate_global_id` GlobalID rejections raise **`Pgbus::SerializationError`** (was `ArgumentError`). Three error classes that bypassed the base — `PgmqSchema::VersionNotFoundError` and `Streams::SignedName::InvalidSignedName`/`MissingSecret` — are re-parented under `Pgbus::Error`. Three classes that reject a malformed *argument shape* — `Configuration::CapsuleDSL::ParseError`, `Streams::Cursor::InvalidCursor`, `Streams::StreamNameTooLong` — **deliberately stay `ArgumentError` subclasses**; the policy ("argument-shape errors are `ArgumentError`, operational errors are `Pgbus::Error`") is documented in `lib/pgbus.rb` and pinned by `spec/pgbus/error_hierarchy_spec.rb`. Messages are unchanged. Refs #282.
|
|
7
10
|
|
|
8
11
|
### Added
|
|
9
12
|
|
|
13
|
+
- **`Pgbus::Streams::PhlexHelpers` — a Phlex-includable `pgbus_stream_from`, and `Pgbus.stream_name_budget` (issue #334).** Two small helpers that stop consuming apps reimplementing pgbus glue. (1) `pgbus_stream_from` is a Rails view helper, so a Phlex component couldn't call it without hand-registering an output-helper macro (phlex-rails ships no `Phlex::Rails::Helpers::PgbusStreamFrom`). `include Pgbus::Streams::PhlexHelpers` now bridges it exactly like phlex-rails' own `TurboStreamFrom`. phlex-rails stays an **optional** dependency — the module references its `HelperMacros` and is loaded on demand (`require "pgbus/streams/phlex_helpers"`), never eagerly. (2) `Pgbus.stream_name_budget` exposes the maximum stream-name length (the pgmq queue-name cap minus the queue prefix) so apps size/truncate stream identifiers up front instead of hand-computing `MAX_QUEUE_NAME_LENGTH - queue_prefix.length - 1` at every call site. Refs #334.
|
|
14
|
+
- **`EventBus::Registry#setup_all!(safe: true)` — a boot/rake-safe way to eagerly set up event subscribers (issue #334).** Calling `setup_all!` opens a PGMQ connection per subscriber (to create its queue + bind its topic), which is wrong during a `db:`/schema rake task (a live connection blocks `DROP DATABASE`) and crashes boot if the database isn't up yet — so apps wrapped it in a hand-rolled multi-class rescue plus a rake-task skip. `setup_all!(safe: true)` bakes that in: it skips entirely in a schema/asset rake context and swallows a connection error with a warning instead of raising. The default `setup_all!` is unchanged (raises). Refs #334.
|
|
10
15
|
- **Connection-pooler safety (`config.require_primary`, `config.connection_guc_mode`) — pgbus is now correct-and-loud behind a transaction/read-write-splitting pooler instead of silently stalling (issue #332).** Three linked fixes. (1) **Primary affinity:** a read/write-splitting pooler (pgdog/pgcat) can route pgmq's VOLATILE `read`/`archive`/`delete` to a read replica, where workers read nothing (`read_ct=0`) and jobs stop with a **healthy heartbeat** — the supervisor watchdog keys on loop advancement, not read progress, so nothing surfaced the stall. The job read path had zero primary-affinity defense (only the LISTEN/streamer paths validated the primary via `PrimaryValidator`). Setting `config.require_primary = true` (default **false** — a single-primary deployment is byte-identical) makes `verify_connection!` reject a connection that lands on a replica (`pg_is_in_recovery() => t`) at supervisor boot, raising a clear `ConfigurationError` instead of forking children that read nothing. (2) **GUC forwarding:** pgbus's AR-config extraction dropped database.yml's `variables:` block, so `client_min_messages` (and any GUC) never reached pgmq's dedicated raw connections (NOTICE flooding that three consuming apps hand-patched via `connection_params`). GUCs are now forwarded, on both the AR-extracted and explicit-`connection_params` paths, by `config.connection_guc_mode`: `:options` (default) bakes them into the libpq `options` STARTUP param (byte-compatible with today for the read-timeout `statement_timeout`, now also carrying `variables:`), while `:session` applies them via post-connect `SET` on a fresh connection instead — for a transaction-mode PgBouncer that **rejects** the `options` startup param with a FATAL. `pgbus doctor` gains a ninth check ("Primary affinity") that warns (never fails) when the job connection is currently on a replica, naming the direct-port remediation. DB-gated integration tests prove both GUC modes round-trip a forwarded GUC and that `require_primary` verifies on a real primary. Refs #332.
|
|
11
16
|
- **Job-burst tuning guidance + gate benchmark (`rake bench:job_burst`), and a decision NOT to autoscale the job pool (issue #323 phase 3).** Following the issue's measure-first mandate, `benchmarks/job_burst_bench.rb` sweeps worker thread count under a job burst in two connection regimes (pool matched to threads vs. pool fixed small). Measured finding: a job holds a pgmq connection only for the `read_batch` + `archive` round-trip (not the job body), so **the DB connection pool is the hard burst ceiling, not the thread count** — in the fixed-pool regime `peak_busy` never exceeds `pool_size` no matter how many threads run (extra threads just queue on checkout, latency climbs, throughput plateaus), while matching the pool to the thread count scales throughput ~linearly (≈8× from 2→16). The conclusion — issue #323's explicitly sanctioned outcome — is that **elastic job threads on a fixed connection pool cannot help**, so pgbus does *not* autoscale the job execution pool; the fix for job bursts is **static headroom**: raise `threads` (and let `pool_size` follow, or keep it ≥ `threads`). Documented in the Performance & tuning docs. The streams-pool autoscaling below stays the elastic story where it *does* pay off (the streams pool's connection cost genuinely scales with its size). Refs #323.
|
|
12
17
|
- **Fan-out writer throughput gate benchmark (`writer_burst_bench.rb`), and a decision NOT to make the writer pool elastic (issue #323 phase 1).** Following the issue's measure-first mandate, `benchmarks/writer_burst_bench.rb` drives a sustained fan-out burst (a fleet of connections × many wakes) through the `OutboundPump` while sweeping `streams_writer_threads`, across fast/typical socket-write profiles. Measured finding: throughput scales ~linearly with writer count (≈6× from 2→16) — but that is generic parallelism (N workers finish N independent blocking writes ~N× faster), which argues for the **existing static `streams_writer_threads` knob**, not for elastic writers: a fixed-N sweep can't demonstrate the *spikiness* that would make a static count wasteful enough to justify dynamic resize, and the genuinely harmful case (a slow/congested client) is already **shed** by the #320/#322 fan-out deadline + mark-dead path rather than absorbed by scaling writers. Given the elastic-writer rewrite is the issue's highest-risk phase (it would touch the ordering/cursor invariant #321's whole review exists to protect — mis-sizing risks out-of-order SSE frames), the conclusion — issue #323's sanctioned outcome — is that pgbus does **not** autoscale the writer pool; the fan-out-throughput knob is **static `streams_writer_threads`, sized for your peak fan-out fleet**. Documented in the Performance & tuning docs. Refs #323.
|
|
@@ -27,6 +32,12 @@
|
|
|
27
32
|
|
|
28
33
|
### Fixed
|
|
29
34
|
|
|
35
|
+
- **Migration generators now route to the pgbus database when `connects_to` is configured, even without an explicit `--database` (issue #344).** `rails g pgbus:add_stream_queues` (and every sibling generator via `Generators::MigrationPath`) keyed the separate-database decision purely on the `--database` flag. An app that had already set `config.connects_to = { database: { writing: :pgbus } }` (so its pgbus migrations live in `db/pgbus_migrate/` per database.yml's `migrations_paths:`) still got the migration written to the **primary** DB's `db/migrate/` on a bare invocation — where it then ran against the wrong database — and the "Next steps" output told the operator to run `rails db:migrate` instead of `rails db:migrate:pgbus`. `MigrationPath` now auto-detects the target database from `connects_to` (via the existing `DatabaseTargetDetector`: runtime config → initializer scan → application.rb scan) when `--database` is absent; an explicit `--database` still wins. The detected name drives both the migration path (resolving `migrations_paths` for that database, falling back to the `db/pgbus_migrate` convention) and the `db:migrate:<name>` suffix in every generator's post-install output. An app with no separate database is unaffected. Refs #344.
|
|
36
|
+
- **`extract_ar_connection_hash` no longer forces `host: "localhost"` / `port: 5432` on a socket-based (host-less) database.yml — pgmq's connections now match ActiveRecord's on Unix-socket dev setups (issue #343).** A local database.yml with no `host:` is a Unix-socket connection: ActiveRecord connects via libpq's default socket (`PGHOST` / the default socket dir), but pgbus's AR-config extraction defaulted the absent `host`/`port` to TCP `localhost:5432`, silently diverging from AR and pointing pgmq's dedicated raw connections at a *different* server (or nothing) on any machine where the socket dir isn't `localhost`. Apps on socket-based dev DBs therefore couldn't use the AR-extraction path at all and had to pin an explicit `connection_params` Hash whose only real job was *not* defaulting `host` (cosmos carried an `after_initialize` block for exactly this). The two fallbacks are dropped: `host`/`port` now pass through as-is and `.compact` removes them when absent, so libpq applies its own socket defaults and matches AR. A config that *does* set `host`/`port` is byte-identical to before. Refs #343.
|
|
37
|
+
- **`config.log_format` no longer breaks `ActiveSupport::TaggedLogging` — tags are preserved instead of dropped, and `logger.tagged` no longer raises (issue #334).** TaggedLogging works by extending the logger's formatter instance with a module that prepends the current tags. Installing a fresh pgbus formatter (`LogFormatter::JSON`/`Text`) stripped that extension, so `logger.tagged("req-42") { ... }` dropped the tag — and, worse, raised `NoMethodError: undefined method 'tagged'` on the pgbus formatter (a deploy-breaker several apps hit, then worked around with a dedicated `$stdout` logger). `log_format=` now re-extends the replacement formatter with `ActiveSupport::TaggedLogging::Formatter` when the previous one carried it, so tagging keeps working. Refs #334.
|
|
38
|
+
- **The "Dashboard is accessible without authentication" warning is no longer a false positive when a gating `base_controller_class` is set (issue #334).** The warning fired whenever `web_auth` was nil, ignoring that a non-default `base_controller_class` (e.g. an `AdminController` with its own `before_action`) already gates the dashboard — so apps set a redundant `web_auth` lambda purely to silence the log line. The warning now stays quiet when `base_controller_class` is anything other than the default `::ActionController::Base`. Refs #334.
|
|
39
|
+
- **`retry_on` on an `:until_executed` job no longer dead-letters the job instead of retrying (issue #333).** The uniqueness key is released only on success or DLQ, but ActiveJob's `retry_on` re-enqueues from *inside* `perform_now` (after incrementing `executions`), while the executor still holds the key. That retry re-enqueue hit the job's own still-held key, was rejected as a duplicate (`JobNotUnique` under `on_conflict: :reject`), and the original message dead-lettered — so a job with both `ensures_uniqueness :until_executed` and `retry_on` lost its retry and DLQ'd on the first transient failure. A retry re-enqueue (`executions > 0`) is now recognized as the same logical job re-acquiring its own key and is allowed through; the existing key row correctly stays held until the job finally succeeds or dead-letters. Cross-job uniqueness (a genuine fresh duplicate, `executions == 0`) is still rejected. Proven end-to-end with a DB-gated integration test (fail-once-then-succeed runs to success exactly once, no `JobNotUnique`, no DLQ, key cleaned up). Refs #333.
|
|
40
|
+
- **A wildcard (`queues: ['*']`) worker no longer adopts event-bus subscriber queues and dead-letters the events (issue #333).** Event-subscriber queues share the job namespace (`#{queue_prefix}_<handler>`) but carry event payloads, not ActiveJob jobs. `resolve_wildcard_queues` excluded only `_dlq` and stream queues, so a wildcard worker claimed event queues too, handed each event to the ActiveJob executor, failed to deserialize it, and DLQ-moved it out of the consumer's reach — forcing apps that run the event bus to hand-maintain an explicit queue list. Wildcard resolution now also excludes every registered event-subscriber queue (new `EventBus::Registry#event_queue_names`), mirroring the existing stream-queue exclusion. Refs #333.
|
|
30
41
|
- **A pooler-deaf NOTIFY listener no longer raises the worker poll floor to 15s — worse latency than a plain no-NOTIFY deployment (issue #332).** pgbus already detects a transaction-mode pooler / replica LISTEN connection with a self-NOTIFY probe (`NotifyProbe`) and logs a loud, actionable error, but the probe's boolean result was **discarded**: a live-but-deaf listener still reported `running? == true`, so `Worker#wake_timeout` treated it as a healthy NOTIFY source and raised the empty-fetch poll floor to `NOTIFY_FALLBACK_POLL_SECONDS` (15s) — behind a pooler the loop would then sleep up to 15s between polls while jobs sat ready. `NotifyListener` now records the probe result and exposes `delivering?`; `wake_timeout` treats a non-delivering listener as absent and keeps polling at the fast `polling_interval` until a direct-port connection is configured. The streamer path needed no change — its listener already waits on the short `streams_listen_health_check_ms` and has no equivalent 15s ceiling. Refs #332.
|
|
31
42
|
- **The dispatcher no longer logs a scary `Concurrency cleanup failed: ... terminating connection due to administrator command` WARN when an idle DB connection is killed between maintenance cycles.** The dispatcher is a long-lived loop that queries the DB on coarse intervals (concurrency cleanup every 300s, most tasks hourly) and, until now, kept its pooled ActiveRecord connection leased across the idle sleep. When the server terminated that idle backend — routine in **local development** (a Postgres restart, `rails db:migrate` calling `pg_terminate_backend` on idle backends, foreman/overmind restarting the DB) — the next cycle reused the dead socket and *every* maintenance task raised `PG::ConnectionBad` with PostgreSQL's `terminating connection due to administrator command` text, surfaced as an alarming WARN even though the process was healthy and self-heals on the next tick. Two changes: (1) `run_maintenance` now returns every leased AR connection to the pool after each cycle (`clear_active_connections!`, mirroring the Streamer's existing `release_ar_connections`), so an idle backend is never reused — the next cycle checks out afresh and AR's `verify!`-on-checkout transparently discards a dead socket and reconnects; (2) every maintenance task's failure log now routes through `log_maintenance_failure`, which recognizes the terminated-idle-connection signal and logs a calm INFO ("reconnecting on the next cycle") instead of a WARN. Non-connection failures still WARN unchanged, and maintenance backoff is unaffected (a self-healing cycle resets the failure streak on the next successful reconnect).
|
|
32
43
|
- **`priority_levels > 1` no longer silently breaks durable streams.** With job priority enabled, `QueueFactory.for` selects `PriorityStrategy`, and `Client#send_message` routes *every* send — including durable stream broadcasts (which pass no `priority:`) — to a `_p<default_priority>` sub-queue. But the streamer LISTENs and replays exclusively on the **bare** `pgmq.q_<stream>` queue (`read_after`, `notify_stream`), so the broadcast never reached the browser; on a fresh DB, `ensure_stream_queue` also raised `PG::UndefinedTable` because `ensure_queue` created only `_p0.._pN`, never the bare table the NOTIFY trigger and archive index attach to. Durable broadcasts now go through a dedicated `Client#send_stream_message` that always targets the bare queue, and `ensure_stream_queue` creates the bare queue directly via `ensure_single_queue` — both independent of the priority strategy, mirroring the bare-name read path. Streams are delivered by a non-consuming peek; priority sub-queues are meaningless for them. Ephemeral broadcasts (the default mode) were never affected. Refs #310.
|
data/Rakefile
CHANGED
|
@@ -209,17 +209,62 @@ task :release, %i[version force] do |_t, args|
|
|
|
209
209
|
success "Updated #{version_file}"
|
|
210
210
|
end
|
|
211
211
|
|
|
212
|
+
# Step 1b: Regenerate the frozen lockfiles that pin the pgbus path gem, so the
|
|
213
|
+
# bump ships with them in sync. These are installed with `--frozen`/deployment
|
|
214
|
+
# in CI, so if they still name the OLD version they instant-fail (the Rails 7.1
|
|
215
|
+
# leg with exit 16, and docs-CI on any docs change). Regenerating here keeps the
|
|
216
|
+
# version-pin drift out of the release commit instead of surfacing on the next PR.
|
|
217
|
+
header "Frozen lockfiles"
|
|
218
|
+
# The ONLY thing a version bump changes in these frozen lockfiles is the pgbus
|
|
219
|
+
# path-gem pin — so bump exactly that line, in place, with a string edit.
|
|
220
|
+
#
|
|
221
|
+
# We deliberately do NOT run `bundle lock` here: a full re-resolve trips over
|
|
222
|
+
# constraints that have nothing to do with pgbus. Concretely, docs/Gemfile.lock
|
|
223
|
+
# carries a broad PLATFORMS list (…-gnu / …-musl / arm-linux) for which a
|
|
224
|
+
# platform gem like `thruster` ships no variant, so `bundle lock` fails with
|
|
225
|
+
# "Could not find gems matching 'thruster' valid for all resolution platforms"
|
|
226
|
+
# on any machine whose cache doesn't already hold those exact gems — aborting
|
|
227
|
+
# the release. `bundle lock --local` was even worse (wrong-file write + no
|
|
228
|
+
# fetch). A targeted pin edit sidesteps all of it, is deterministic on any
|
|
229
|
+
# machine, and produces the minimal 2-line diff (the PATH spec + the
|
|
230
|
+
# DEPENDENCIES pin). See #338/#341 and the surgical-bump fix.
|
|
231
|
+
frozen_lockfiles = %w[gemfiles/rails_7_1.gemfile.lock docs/Gemfile.lock]
|
|
232
|
+
regenerated_lockfiles = []
|
|
233
|
+
frozen_lockfiles.each do |lockfile|
|
|
234
|
+
unless File.exist?(lockfile)
|
|
235
|
+
skip "#{lockfile} not present"
|
|
236
|
+
next
|
|
237
|
+
end
|
|
238
|
+
|
|
239
|
+
content = File.read(lockfile)
|
|
240
|
+
# Matches both the PATH-source spec (" pgbus (X.Y.Z)") and the
|
|
241
|
+
# DEPENDENCIES pin (" pgbus (X.Y.Z)"), leaving everything else untouched.
|
|
242
|
+
bumped = content.gsub(/^(\s+pgbus) \([^)]*\)$/, "\\1 (#{new_version})")
|
|
243
|
+
|
|
244
|
+
if bumped == content
|
|
245
|
+
skip "#{lockfile} — no pgbus pin to bump"
|
|
246
|
+
next
|
|
247
|
+
end
|
|
248
|
+
|
|
249
|
+
File.write(lockfile, bumped)
|
|
250
|
+
regenerated_lockfiles << lockfile
|
|
251
|
+
success "Bumped pgbus pin in #{lockfile}"
|
|
252
|
+
end
|
|
253
|
+
|
|
212
254
|
# Step 2: Verify gem builds cleanly
|
|
213
255
|
header "Build verification"
|
|
214
256
|
sh("gem build pgbus.gemspec --strict")
|
|
215
257
|
sh("rm -f pgbus-*.gem")
|
|
216
258
|
success "Gem builds cleanly"
|
|
217
259
|
|
|
218
|
-
# Step 3: Commit version bump
|
|
260
|
+
# Step 3: Commit version bump (+ any re-synced lockfiles)
|
|
219
261
|
header "Git commit"
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
262
|
+
paths_to_stage = [version_file, *regenerated_lockfiles]
|
|
263
|
+
staged_changes = paths_to_stage.any? do |path|
|
|
264
|
+
!`git diff #{path}`.strip.empty? || !`git diff --cached #{path}`.strip.empty?
|
|
265
|
+
end
|
|
266
|
+
if staged_changes
|
|
267
|
+
paths_to_stage.each { |path| sh("git add #{path}") }
|
|
223
268
|
sh("git commit -m 'chore: bump version to #{new_version}'")
|
|
224
269
|
success "Committed version bump"
|
|
225
270
|
else
|
|
@@ -29,7 +29,7 @@ module Pgbus
|
|
|
29
29
|
say "Pgbus failed events unique index added!", :green
|
|
30
30
|
say ""
|
|
31
31
|
say "Next steps:"
|
|
32
|
-
say " 1. Run: rails db:migrate#{
|
|
32
|
+
say " 1. Run: rails db:migrate#{migrate_command_suffix}"
|
|
33
33
|
say " 2. Failed jobs will now be tracked in the dashboard"
|
|
34
34
|
say ""
|
|
35
35
|
end
|
|
@@ -29,7 +29,7 @@ module Pgbus
|
|
|
29
29
|
say "Pgbus job stats table installed!", :green
|
|
30
30
|
say ""
|
|
31
31
|
say "Next steps:"
|
|
32
|
-
say " 1. Run: rails db:migrate#{
|
|
32
|
+
say " 1. Run: rails db:migrate#{migrate_command_suffix}"
|
|
33
33
|
say " 2. Stats collection is enabled by default"
|
|
34
34
|
say " 3. View insights at /pgbus/insights"
|
|
35
35
|
say ""
|
|
@@ -29,7 +29,7 @@ module Pgbus
|
|
|
29
29
|
say "Pgbus job stats latency columns installed!", :green
|
|
30
30
|
say ""
|
|
31
31
|
say "Next steps:"
|
|
32
|
-
say " 1. Run: rails db:migrate#{
|
|
32
|
+
say " 1. Run: rails db:migrate#{migrate_command_suffix}"
|
|
33
33
|
say " 2. Queue latency and retry metrics are now tracked automatically"
|
|
34
34
|
say " 3. View latency insights at /pgbus/insights"
|
|
35
35
|
say ""
|
|
@@ -29,7 +29,7 @@ module Pgbus
|
|
|
29
29
|
say "Pgbus job stats queue index added!", :green
|
|
30
30
|
say ""
|
|
31
31
|
say "Next steps:"
|
|
32
|
-
say " 1. Run: rails db:migrate#{
|
|
32
|
+
say " 1. Run: rails db:migrate#{migrate_command_suffix}"
|
|
33
33
|
say " 2. The Insights 'latency by queue' aggregation will now use the index"
|
|
34
34
|
say " instead of sequentially scanning pgbus_job_stats. Install this on"
|
|
35
35
|
say " heavy-traffic deployments with a large job stats retention window."
|
|
@@ -29,7 +29,7 @@ module Pgbus
|
|
|
29
29
|
say "Pgbus outbox installed!", :green
|
|
30
30
|
say ""
|
|
31
31
|
say "Next steps:"
|
|
32
|
-
say " 1. Run: rails db:migrate#{
|
|
32
|
+
say " 1. Run: rails db:migrate#{migrate_command_suffix}"
|
|
33
33
|
say " 2. Enable in config: config.outbox_enabled = true"
|
|
34
34
|
say " 3. Restart pgbus: bin/pgbus start"
|
|
35
35
|
say ""
|
|
@@ -29,7 +29,7 @@ module Pgbus
|
|
|
29
29
|
say "Pgbus presence installed!", :green
|
|
30
30
|
say ""
|
|
31
31
|
say "Next steps:"
|
|
32
|
-
say " 1. Run: rails db:migrate#{
|
|
32
|
+
say " 1. Run: rails db:migrate#{migrate_command_suffix}"
|
|
33
33
|
say " 2. Use in your code:"
|
|
34
34
|
say " Pgbus.stream(@room).presence.join(member_id: current_user.id.to_s)"
|
|
35
35
|
say " Pgbus.stream(@room).presence.members"
|
|
@@ -29,7 +29,7 @@ module Pgbus
|
|
|
29
29
|
say "Pgbus queue states table installed!", :green
|
|
30
30
|
say ""
|
|
31
31
|
say "Next steps:"
|
|
32
|
-
say " 1. Run: rails db:migrate#{
|
|
32
|
+
say " 1. Run: rails db:migrate#{migrate_command_suffix}"
|
|
33
33
|
say " 2. Restart pgbus: bin/pgbus start"
|
|
34
34
|
say ""
|
|
35
35
|
end
|
|
@@ -33,7 +33,7 @@ module Pgbus
|
|
|
33
33
|
say "Pgbus recurring jobs installed!", :green
|
|
34
34
|
say ""
|
|
35
35
|
say "Next steps:"
|
|
36
|
-
say " 1. Run: rails db:migrate#{
|
|
36
|
+
say " 1. Run: rails db:migrate#{migrate_command_suffix}"
|
|
37
37
|
say " 2. Edit config/recurring.yml to define your recurring tasks"
|
|
38
38
|
say " 3. Restart pgbus: bin/pgbus start"
|
|
39
39
|
say ""
|
|
@@ -35,7 +35,7 @@ module Pgbus
|
|
|
35
35
|
say "streams register themselves on their next broadcast after migrating."
|
|
36
36
|
say ""
|
|
37
37
|
say "Next steps:"
|
|
38
|
-
say " 1. Run: rails db:migrate#{
|
|
38
|
+
say " 1. Run: rails db:migrate#{migrate_command_suffix}"
|
|
39
39
|
say " 2. Restart pgbus: bin/pgbus start"
|
|
40
40
|
say ""
|
|
41
41
|
end
|
|
@@ -29,7 +29,7 @@ module Pgbus
|
|
|
29
29
|
say "Pgbus stream stats table installed!", :green
|
|
30
30
|
say ""
|
|
31
31
|
say "Next steps:"
|
|
32
|
-
say " 1. Run: rails db:migrate#{
|
|
32
|
+
say " 1. Run: rails db:migrate#{migrate_command_suffix}"
|
|
33
33
|
say " 2. Opt in by setting `config.streams_stats_enabled = true` in your"
|
|
34
34
|
say " pgbus initializer (disabled by default — stream event volume can"
|
|
35
35
|
say " be high, so stats recording is off unless you ask for it)."
|
|
@@ -29,7 +29,7 @@ module Pgbus
|
|
|
29
29
|
say "Pgbus uniqueness keys table installed!", :green
|
|
30
30
|
say ""
|
|
31
31
|
say "Next steps:"
|
|
32
|
-
say " 1. Run: rails db:migrate#{
|
|
32
|
+
say " 1. Run: rails db:migrate#{migrate_command_suffix}"
|
|
33
33
|
say " 2. Add `ensures_uniqueness` to your job classes (DSL is auto-included)"
|
|
34
34
|
say " 3. Restart pgbus: bin/pgbus start"
|
|
35
35
|
say ""
|
|
@@ -99,7 +99,7 @@ module Pgbus
|
|
|
99
99
|
|
|
100
100
|
say ""
|
|
101
101
|
say "Next steps:"
|
|
102
|
-
say " 1. Run: rails db:migrate#{
|
|
102
|
+
say " 1. Run: rails db:migrate#{migrate_command_suffix}"
|
|
103
103
|
say " 2. Edit config/initializers/pgbus.rb to configure workers"
|
|
104
104
|
say " 3. Start processing: bin/pgbus start"
|
|
105
105
|
say ""
|
|
@@ -115,8 +115,11 @@ module Pgbus
|
|
|
115
115
|
options[:pgmq_schema_mode]
|
|
116
116
|
end
|
|
117
117
|
|
|
118
|
+
# An explicit --database wins; otherwise fall back to a database already
|
|
119
|
+
# configured via connects_to so a bare re-install targets the right DB
|
|
120
|
+
# and its setup hints/output name it (issue #344).
|
|
118
121
|
def database_name
|
|
119
|
-
|
|
122
|
+
effective_database_name
|
|
120
123
|
end
|
|
121
124
|
end
|
|
122
125
|
end
|
|
@@ -34,7 +34,7 @@ module Pgbus
|
|
|
34
34
|
say " 3. Drop the old pgbus_job_locks table (8 columns, 3 indexes)"
|
|
35
35
|
say ""
|
|
36
36
|
say "Next steps:"
|
|
37
|
-
say " 1. Run: rails db:migrate#{
|
|
37
|
+
say " 1. Run: rails db:migrate#{migrate_command_suffix}"
|
|
38
38
|
say " 2. Restart pgbus: bin/pgbus start"
|
|
39
39
|
say ""
|
|
40
40
|
end
|
|
@@ -1,27 +1,93 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require "pgbus/generators/database_target_detector"
|
|
4
|
+
|
|
3
5
|
module Pgbus
|
|
4
6
|
module Generators
|
|
5
7
|
# Shared migration path logic for all pgbus generators.
|
|
6
8
|
#
|
|
7
|
-
#
|
|
8
|
-
#
|
|
9
|
-
#
|
|
9
|
+
# A separate pgbus database can be selected two ways:
|
|
10
|
+
#
|
|
11
|
+
# 1. --database=pgbus explicitly on the generator invocation, or
|
|
12
|
+
# 2. the host app has already configured Pgbus with a dedicated database
|
|
13
|
+
# (config.connects_to = { database: { writing: :pgbus } }).
|
|
10
14
|
#
|
|
11
|
-
#
|
|
15
|
+
# When either applies, migrations go to the separate-database path (e.g.
|
|
16
|
+
# db/pgbus_migrate) and the post-install output names the db:migrate:<name>
|
|
17
|
+
# task. When neither applies, everything falls back to db/migrate.
|
|
18
|
+
#
|
|
19
|
+
# Auto-detecting from connects_to (case 2) fixes issue #344: a bare
|
|
20
|
+
# `rails g pgbus:add_*` in an app configured for a separate database used to
|
|
21
|
+
# silently write to db/migrate and run against the WRONG database.
|
|
12
22
|
module MigrationPath
|
|
13
23
|
private
|
|
14
24
|
|
|
15
25
|
def pgbus_migrate_path
|
|
16
|
-
|
|
26
|
+
return "db/migrate" unless separate_database?
|
|
27
|
+
|
|
28
|
+
# --database was passed: Rails' db_migrate_path reads options[:database]
|
|
29
|
+
# and resolves migrations_paths from database.yml. When the database was
|
|
30
|
+
# instead auto-detected from connects_to, options[:database] is nil, so
|
|
31
|
+
# db_migrate_path can't see it — resolve the path for the detected
|
|
32
|
+
# database ourselves.
|
|
33
|
+
if options[:database].present?
|
|
17
34
|
db_migrate_path
|
|
18
35
|
else
|
|
19
|
-
|
|
36
|
+
resolve_detected_migrate_path
|
|
20
37
|
end
|
|
21
38
|
end
|
|
22
39
|
|
|
23
40
|
def separate_database?
|
|
24
|
-
|
|
41
|
+
effective_database_name.present?
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
# The database name that migrations should target: an explicit --database
|
|
45
|
+
# wins; otherwise auto-detect from the app's connects_to configuration.
|
|
46
|
+
def effective_database_name
|
|
47
|
+
return @effective_database_name if defined?(@effective_database_name)
|
|
48
|
+
|
|
49
|
+
@effective_database_name =
|
|
50
|
+
if options[:database].present?
|
|
51
|
+
options[:database]
|
|
52
|
+
else
|
|
53
|
+
DatabaseTargetDetector.new(destination_root: destination_root).detect
|
|
54
|
+
end
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
# The `:name` suffix appended to `rails db:migrate` in post-install
|
|
58
|
+
# output. Empty string for single-database mode so the line reads a plain
|
|
59
|
+
# `rails db:migrate`.
|
|
60
|
+
def migrate_command_suffix
|
|
61
|
+
separate_database? ? ":#{effective_database_name}" : ""
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
# Migrations path for an auto-detected separate database. Rails' own
|
|
65
|
+
# db_migrate_path is keyed on options[:database], which is nil here, so
|
|
66
|
+
# look up migrations_paths for the detected database directly and fall
|
|
67
|
+
# back to the db/pgbus_migrate convention if it isn't in database.yml.
|
|
68
|
+
def resolve_detected_migrate_path
|
|
69
|
+
migrations_path_for(effective_database_name) || "db/pgbus_migrate"
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
def migrations_path_for(database_name)
|
|
73
|
+
return nil unless defined?(::ActiveRecord::Base) && defined?(::Rails) && ::Rails.respond_to?(:env)
|
|
74
|
+
|
|
75
|
+
config = ::ActiveRecord::Base.configurations.configs_for(
|
|
76
|
+
env_name: ::Rails.env,
|
|
77
|
+
name: database_name
|
|
78
|
+
)
|
|
79
|
+
Array(config&.migrations_paths).first
|
|
80
|
+
rescue StandardError => e
|
|
81
|
+
# A missing config for `database_name` isn't an error — configs_for
|
|
82
|
+
# returns nil and we fall back to the db/pgbus_migrate convention. But a
|
|
83
|
+
# genuine failure here (malformed database.yml, an AR API change) would
|
|
84
|
+
# otherwise be invisible, so surface it via the generator's own output
|
|
85
|
+
# the way update_generator#resolve_connection does — not Pgbus.logger
|
|
86
|
+
# (unavailable/inappropriate at generate time; pgbus_failed_events is the
|
|
87
|
+
# runtime job-failure table, not a generator diagnostics sink).
|
|
88
|
+
say " ! could not resolve migrations_paths for #{database_name.inspect}: " \
|
|
89
|
+
"#{e.class}: #{e.message} (falling back to db/pgbus_migrate)", :yellow
|
|
90
|
+
nil
|
|
25
91
|
end
|
|
26
92
|
end
|
|
27
93
|
end
|
|
@@ -33,7 +33,7 @@ module Pgbus
|
|
|
33
33
|
say "automatically receive these settings."
|
|
34
34
|
say ""
|
|
35
35
|
say "Next steps:"
|
|
36
|
-
say " 1. Run: rails db:migrate#{
|
|
36
|
+
say " 1. Run: rails db:migrate#{migrate_command_suffix}"
|
|
37
37
|
say " 2. Restart pgbus: bin/pgbus start"
|
|
38
38
|
say ""
|
|
39
39
|
end
|
|
@@ -36,7 +36,7 @@ module Pgbus
|
|
|
36
36
|
say "this setting."
|
|
37
37
|
say ""
|
|
38
38
|
say "Next steps:"
|
|
39
|
-
say " 1. Run: rails db:migrate#{
|
|
39
|
+
say " 1. Run: rails db:migrate#{migrate_command_suffix}"
|
|
40
40
|
say " 2. Restart pgbus: bin/pgbus start"
|
|
41
41
|
say ""
|
|
42
42
|
end
|
|
@@ -31,7 +31,7 @@ module Pgbus
|
|
|
31
31
|
say ""
|
|
32
32
|
say "Next steps:"
|
|
33
33
|
say " 1. Review the migration in db/#{separate_database? ? "pgbus_migrate" : "migrate"}/"
|
|
34
|
-
say " 2. Run: rails db:migrate#{
|
|
34
|
+
say " 2. Run: rails db:migrate#{migrate_command_suffix}"
|
|
35
35
|
say ""
|
|
36
36
|
end
|
|
37
37
|
|
|
@@ -112,6 +112,17 @@ module Pgbus
|
|
|
112
112
|
uniqueness_key = Uniqueness.extract_key(payload_hash)
|
|
113
113
|
return false unless uniqueness_key
|
|
114
114
|
|
|
115
|
+
# A retry re-enqueue is the SAME logical job re-acquiring its OWN key.
|
|
116
|
+
# ActiveJob increments `executions` at the start of perform_now (before
|
|
117
|
+
# the body), then retry_on re-enqueues from inside perform_now — while
|
|
118
|
+
# the executor still holds the key (it releases only on success/DLQ). So
|
|
119
|
+
# a re-enqueue with executions > 0 would otherwise hit its own held key,
|
|
120
|
+
# be rejected as a duplicate (JobNotUnique), and dead-letter the original
|
|
121
|
+
# while losing the retry. Let the retry through; the existing key row
|
|
122
|
+
# correctly stays held until the job finally succeeds or dead-letters.
|
|
123
|
+
# See issue #333.
|
|
124
|
+
return false if active_job.executions.to_i.positive?
|
|
125
|
+
|
|
115
126
|
result = Uniqueness.acquire_enqueue_lock(uniqueness_key, active_job)
|
|
116
127
|
|
|
117
128
|
# :no_lock means no enqueue-time lock needed (e.g. :while_executing strategy)
|
data/lib/pgbus/configuration.rb
CHANGED
|
@@ -145,7 +145,7 @@ module Pgbus
|
|
|
145
145
|
attr_accessor :health_port, :health_bind
|
|
146
146
|
|
|
147
147
|
# Streams (turbo-rails replacement, SSE-based)
|
|
148
|
-
attr_accessor :streams_enabled, :streams_path, :
|
|
148
|
+
attr_accessor :streams_enabled, :streams_path, :streams_signed_name_secret,
|
|
149
149
|
:streams_default_retention, :streams_retention, :streams_heartbeat_interval,
|
|
150
150
|
:streams_max_connections, :streams_idle_timeout, :streams_listen_health_check_ms,
|
|
151
151
|
:streams_write_deadline_ms, :streams_falcon_streaming_body,
|
|
@@ -291,12 +291,11 @@ module Pgbus
|
|
|
291
291
|
|
|
292
292
|
@streams_enabled = true
|
|
293
293
|
@streams_path = nil
|
|
294
|
-
#
|
|
295
|
-
#
|
|
296
|
-
#
|
|
297
|
-
#
|
|
298
|
-
#
|
|
299
|
-
@streams_queue_prefix = "pgbus_stream"
|
|
294
|
+
# streams_queue_prefix was removed in 1.0 (issue #335): it had been an
|
|
295
|
+
# inert no-op since #308 (stream queues are named like job queues,
|
|
296
|
+
# `#{queue_prefix}_<name>`, and identified via the pgbus_stream_queues
|
|
297
|
+
# registry). Setting it now raises NoMethodError — remove it from your
|
|
298
|
+
# initializer.
|
|
300
299
|
# Streamer-only connection overrides. The Streamer's Listener owns a
|
|
301
300
|
# dedicated long-lived `wait_for_notify` PG connection that can't go
|
|
302
301
|
# through a PgBouncer in transaction mode (LISTEN/NOTIFY don't survive
|
|
@@ -495,12 +494,23 @@ module Pgbus
|
|
|
495
494
|
# pgbus-installed formatter. A user who set a custom logger with a custom
|
|
496
495
|
# formatter keeps it — log_format only records the intended format for
|
|
497
496
|
# pgbus's own defaults, it must not silently clobber their formatting.
|
|
498
|
-
|
|
499
|
-
|
|
500
|
-
|
|
501
|
-
|
|
502
|
-
|
|
503
|
-
|
|
497
|
+
old_formatter = @logger&.formatter
|
|
498
|
+
return unless pgbus_installable_formatter?(old_formatter)
|
|
499
|
+
|
|
500
|
+
new_formatter = case format
|
|
501
|
+
when :json then LogFormatter::JSON.new
|
|
502
|
+
when :text then LogFormatter::Text.new
|
|
503
|
+
end
|
|
504
|
+
|
|
505
|
+
# ActiveSupport::TaggedLogging works by extending the logger's formatter
|
|
506
|
+
# instance with a module that prepends the current tags (and adds a
|
|
507
|
+
# #tagged method the logger delegates to). Installing a fresh pgbus
|
|
508
|
+
# formatter would strip that, so `logger.tagged { ... }` would drop tags —
|
|
509
|
+
# and, worse, raise NoMethodError: undefined method `tagged` on our
|
|
510
|
+
# formatter. Re-extend the new formatter with the same module when the old
|
|
511
|
+
# one carried it, so tagging keeps working (issue #334).
|
|
512
|
+
preserve_tagged_logging(old_formatter, new_formatter)
|
|
513
|
+
@logger.formatter = new_formatter
|
|
504
514
|
end
|
|
505
515
|
|
|
506
516
|
VALID_BROADCAST_MODES = %i[ephemeral durable].freeze
|
|
@@ -631,12 +641,63 @@ module Pgbus
|
|
|
631
641
|
|
|
632
642
|
raise Pgbus::ConfigurationError, "require_primary must be true or false" unless [true, false].include?(require_primary)
|
|
633
643
|
|
|
644
|
+
validate_job_path_gaps!
|
|
634
645
|
validate_streams!
|
|
635
646
|
validate_metrics_backend!
|
|
636
647
|
|
|
637
648
|
self
|
|
638
649
|
end
|
|
639
650
|
|
|
651
|
+
# Pre-1.0 surface-freeze: reject malformed values for core job-path keys at
|
|
652
|
+
# boot rather than failing deep in a worker/dispatcher/poller/scheduler
|
|
653
|
+
# thread, per-enqueue, or by silently corrupting queue names / leaving the
|
|
654
|
+
# dashboard open. Mirrors the idioms already used above (issue #335).
|
|
655
|
+
def validate_job_path_gaps!
|
|
656
|
+
# Worker recycling trio: nil disables the limit; a set value must be a
|
|
657
|
+
# positive Numeric (mirror stall_threshold).
|
|
658
|
+
{ max_jobs_per_worker: max_jobs_per_worker, max_memory_mb: max_memory_mb,
|
|
659
|
+
max_worker_lifetime: max_worker_lifetime }.each do |name, value|
|
|
660
|
+
next if value.nil? || (value.is_a?(Numeric) && value.positive?)
|
|
661
|
+
|
|
662
|
+
raise Pgbus::ConfigurationError, "#{name} must be a positive number or nil to disable"
|
|
663
|
+
end
|
|
664
|
+
|
|
665
|
+
# Interval knobs: positive Numeric, never nil (mirror polling_interval).
|
|
666
|
+
{ dispatch_interval: dispatch_interval, outbox_poll_interval: outbox_poll_interval,
|
|
667
|
+
recurring_schedule_interval: recurring_schedule_interval }.each do |name, value|
|
|
668
|
+
raise Pgbus::ConfigurationError, "#{name} must be > 0" unless value.is_a?(Numeric) && value.positive?
|
|
669
|
+
end
|
|
670
|
+
|
|
671
|
+
unless outbox_batch_size.is_a?(Integer) && outbox_batch_size.positive?
|
|
672
|
+
raise Pgbus::ConfigurationError, "outbox_batch_size must be a positive integer"
|
|
673
|
+
end
|
|
674
|
+
|
|
675
|
+
# 0 is a valid priority level (queue_factory clamps to 0..priority_levels-1).
|
|
676
|
+
unless default_priority.is_a?(Integer) && default_priority >= 0
|
|
677
|
+
raise Pgbus::ConfigurationError, "default_priority must be a non-negative integer"
|
|
678
|
+
end
|
|
679
|
+
|
|
680
|
+
# Queue-name components: a nil/empty/non-String prefix silently corrupts
|
|
681
|
+
# every derived queue name (mirror health_bind).
|
|
682
|
+
{ queue_prefix: queue_prefix, default_queue: default_queue }.each do |name, value|
|
|
683
|
+
raise Pgbus::ConfigurationError, "#{name} must be a non-empty String" unless value.is_a?(String) && !value.empty?
|
|
684
|
+
end
|
|
685
|
+
|
|
686
|
+
# web_auth gates the dashboard; a non-callable value silently leaves it
|
|
687
|
+
# open (mirror streams_presence_member).
|
|
688
|
+
unless web_auth.nil? || web_auth.respond_to?(:call)
|
|
689
|
+
raise Pgbus::ConfigurationError, "web_auth must respond to #call (a Proc/lambda) or be nil"
|
|
690
|
+
end
|
|
691
|
+
|
|
692
|
+
raise Pgbus::ConfigurationError, "error_reporters must be an Array" unless error_reporters.is_a?(Array)
|
|
693
|
+
|
|
694
|
+
# connects_to is splatted into ActiveRecord's connects_to at engine boot;
|
|
695
|
+
# a non-Hash raises a raw TypeError there — surface a clean message here.
|
|
696
|
+
return if connects_to.nil? || connects_to.is_a?(Hash)
|
|
697
|
+
|
|
698
|
+
raise Pgbus::ConfigurationError, "connects_to must be a Hash or nil"
|
|
699
|
+
end
|
|
700
|
+
|
|
640
701
|
def validate_streams!
|
|
641
702
|
unless streams_default_retention.is_a?(Numeric) && streams_default_retention >= 0
|
|
642
703
|
raise Pgbus::ConfigurationError, "streams_default_retention must be a non-negative number"
|
|
@@ -1114,28 +1175,7 @@ module Pgbus
|
|
|
1114
1175
|
#
|
|
1115
1176
|
# Precedence: streams_database_url > streams_host/port override > base.
|
|
1116
1177
|
def streams_connection_options
|
|
1117
|
-
|
|
1118
|
-
|
|
1119
|
-
base = connection_options
|
|
1120
|
-
return base unless streams_host || streams_port
|
|
1121
|
-
|
|
1122
|
-
case base
|
|
1123
|
-
when Hash
|
|
1124
|
-
result = base.dup
|
|
1125
|
-
result[:host] = streams_host if streams_host
|
|
1126
|
-
result[:port] = streams_port if streams_port
|
|
1127
|
-
result
|
|
1128
|
-
when String
|
|
1129
|
-
# libpq's conninfo parser takes later key=value pairs as overrides
|
|
1130
|
-
# for earlier ones, so we just append. Handles both URI form
|
|
1131
|
-
# (postgres://...) and key=value form.
|
|
1132
|
-
parts = [base]
|
|
1133
|
-
parts << "host=#{streams_host}" if streams_host
|
|
1134
|
-
parts << "port=#{streams_port}" if streams_port
|
|
1135
|
-
parts.join(" ")
|
|
1136
|
-
else
|
|
1137
|
-
base
|
|
1138
|
-
end
|
|
1178
|
+
override_connection_options(url: streams_database_url, host: streams_host, port: streams_port)
|
|
1139
1179
|
end
|
|
1140
1180
|
|
|
1141
1181
|
# Connection options for the Worker's dedicated NotifyListener connection.
|
|
@@ -1143,38 +1183,64 @@ module Pgbus
|
|
|
1143
1183
|
# overridable via worker_notify_database_url / worker_notify_host /
|
|
1144
1184
|
# worker_notify_port so the LISTEN connection can bypass PgBouncer.
|
|
1145
1185
|
def worker_notify_connection_options
|
|
1146
|
-
|
|
1186
|
+
override_connection_options(url: worker_notify_database_url, host: worker_notify_host, port: worker_notify_port)
|
|
1187
|
+
end
|
|
1188
|
+
|
|
1189
|
+
# Resolved notify wakeup flag: defaults to listen_notify when nil.
|
|
1190
|
+
def worker_notify_wakeup?
|
|
1191
|
+
if @worker_notify_wakeup.nil?
|
|
1192
|
+
listen_notify
|
|
1193
|
+
else
|
|
1194
|
+
@worker_notify_wakeup
|
|
1195
|
+
end
|
|
1196
|
+
end
|
|
1197
|
+
|
|
1198
|
+
private
|
|
1199
|
+
|
|
1200
|
+
# Shared resolver for a dedicated connection (streamer LISTEN / worker
|
|
1201
|
+
# NotifyListener) that can override the base `connection_options` to bypass a
|
|
1202
|
+
# pooler. Precedence: a full `url` wins outright; otherwise `host`/`port`
|
|
1203
|
+
# surgically override the base (a Hash gets dup+merge, a String URL/conninfo
|
|
1204
|
+
# gets appended key=value pairs libpq treats as later-wins overrides); with
|
|
1205
|
+
# no override, or a base that is neither Hash nor String, the base passes
|
|
1206
|
+
# through unchanged. Extracted from the byte-identical
|
|
1207
|
+
# streams_connection_options / worker_notify_connection_options (issue #335).
|
|
1208
|
+
def override_connection_options(url:, host:, port:)
|
|
1209
|
+
return url if url
|
|
1147
1210
|
|
|
1148
1211
|
base = connection_options
|
|
1149
|
-
return base unless
|
|
1212
|
+
return base unless host || port
|
|
1150
1213
|
|
|
1151
1214
|
case base
|
|
1152
1215
|
when Hash
|
|
1153
1216
|
result = base.dup
|
|
1154
|
-
result[:host] =
|
|
1155
|
-
result[:port] =
|
|
1217
|
+
result[:host] = host if host
|
|
1218
|
+
result[:port] = port if port
|
|
1156
1219
|
result
|
|
1157
1220
|
when String
|
|
1158
1221
|
parts = [base]
|
|
1159
|
-
parts << "host=#{
|
|
1160
|
-
parts << "port=#{
|
|
1222
|
+
parts << "host=#{host}" if host
|
|
1223
|
+
parts << "port=#{port}" if port
|
|
1161
1224
|
parts.join(" ")
|
|
1162
1225
|
else
|
|
1163
1226
|
base
|
|
1164
1227
|
end
|
|
1165
1228
|
end
|
|
1166
1229
|
|
|
1167
|
-
#
|
|
1168
|
-
|
|
1169
|
-
|
|
1170
|
-
|
|
1171
|
-
|
|
1172
|
-
|
|
1173
|
-
|
|
1230
|
+
# When the logger's previous formatter was extended with
|
|
1231
|
+
# ActiveSupport::TaggedLogging::Formatter (the case for a Rails
|
|
1232
|
+
# Rails.logger), extend the replacement pgbus formatter with the same module
|
|
1233
|
+
# so `logger.tagged`/`push_tags` keep prepending tags instead of dropping
|
|
1234
|
+
# them (and calling #tagged on our formatter stops raising NoMethodError).
|
|
1235
|
+
# No-op when TaggedLogging isn't loaded or the old formatter wasn't tagged.
|
|
1236
|
+
def preserve_tagged_logging(old_formatter, new_formatter)
|
|
1237
|
+
return unless new_formatter
|
|
1238
|
+
return unless defined?(::ActiveSupport::TaggedLogging::Formatter)
|
|
1239
|
+
return unless old_formatter.is_a?(::ActiveSupport::TaggedLogging::Formatter)
|
|
1240
|
+
|
|
1241
|
+
new_formatter.extend(::ActiveSupport::TaggedLogging::Formatter)
|
|
1174
1242
|
end
|
|
1175
1243
|
|
|
1176
|
-
private
|
|
1177
|
-
|
|
1178
1244
|
# True when log_format= may install its own formatter: no formatter set yet
|
|
1179
1245
|
# (nil), a pgbus formatter we installed, or a framework DEFAULT formatter
|
|
1180
1246
|
# (Ruby's Logger::Formatter, Rails' SimpleFormatter) that the app didn't
|
|
@@ -1373,9 +1439,15 @@ module Pgbus
|
|
|
1373
1439
|
# Rails 7.1+ db_config.configuration_hash returns the full config
|
|
1374
1440
|
config_hash = db_config.configuration_hash
|
|
1375
1441
|
|
|
1442
|
+
# Do NOT default host/port here. An absent `host:` in database.yml is a
|
|
1443
|
+
# Unix-socket connection; AR connects via libpq's socket default, so pgmq's
|
|
1444
|
+
# raw connections must too. Forcing TCP localhost:5432 diverges from AR on
|
|
1445
|
+
# socket-based configs (issue #343). `.compact` drops the absent keys and
|
|
1446
|
+
# libpq applies its own defaults (PGHOST / default socket dir). A
|
|
1447
|
+
# present-but-nil port is preserved as nil (also dropped by compact).
|
|
1376
1448
|
base = {
|
|
1377
|
-
host: config_hash[:host]
|
|
1378
|
-
port:
|
|
1449
|
+
host: config_hash[:host],
|
|
1450
|
+
port: config_hash[:port]&.to_i,
|
|
1379
1451
|
dbname: config_hash[:database],
|
|
1380
1452
|
user: config_hash[:username],
|
|
1381
1453
|
password: config_hash[:password]
|
|
@@ -28,20 +28,79 @@ module Pgbus
|
|
|
28
28
|
subscriber
|
|
29
29
|
end
|
|
30
30
|
|
|
31
|
-
|
|
32
|
-
|
|
31
|
+
# Set up every registered subscriber (creates its queue + binds its topic
|
|
32
|
+
# via Pgbus.client, which opens a PGMQ connection).
|
|
33
|
+
#
|
|
34
|
+
# safe: false (default) — set up unconditionally; a connection error
|
|
35
|
+
# propagates. Use when you know the database is up.
|
|
36
|
+
# safe: true — boot/rake-safe: skip entirely in a schema/db: rake context
|
|
37
|
+
# (opening a PGMQ connection there would block DROP DATABASE and isn't
|
|
38
|
+
# wanted during schema load / asset precompile), and swallow a connection
|
|
39
|
+
# error with a warning instead of crashing boot when the DB isn't ready.
|
|
40
|
+
# This is the path host apps should call from an initializer so they stop
|
|
41
|
+
# hand-wrapping setup_all! in a multi-class rescue (issue #334).
|
|
42
|
+
def setup_all!(safe: false)
|
|
43
|
+
return if safe && schema_task_context?
|
|
44
|
+
|
|
45
|
+
# Snapshot under the mutex — subscribe/clear! mutate @subscribers under
|
|
46
|
+
# @mutex, and setup! does DB I/O we must NOT hold the lock across, so
|
|
47
|
+
# iterate a copy taken atomically.
|
|
48
|
+
subscribers = @mutex.synchronize { @subscribers.dup }
|
|
49
|
+
|
|
50
|
+
subscribers.each do |subscriber|
|
|
51
|
+
subscriber.setup!
|
|
52
|
+
rescue PGMQ::Errors::ConnectionError, PG::ConnectionBad => e
|
|
53
|
+
# Only a genuine CONNECTION failure ("database isn't up yet") is
|
|
54
|
+
# tolerable under safe:; a PG::Error subclass like a syntax/permission/
|
|
55
|
+
# missing-table error is a real setup bug and must still surface.
|
|
56
|
+
raise unless safe
|
|
57
|
+
|
|
58
|
+
Pgbus.logger.warn do
|
|
59
|
+
"[Pgbus] EventBus subscriber setup skipped (#{e.class}: #{e.message}) — " \
|
|
60
|
+
"the database isn't reachable yet; subscribers set up on the next attempt."
|
|
61
|
+
end
|
|
62
|
+
end
|
|
33
63
|
end
|
|
34
64
|
|
|
35
65
|
def handlers_for(routing_key)
|
|
36
66
|
@subscribers.select { |s| matches?(s.pattern, routing_key) }
|
|
37
67
|
end
|
|
38
68
|
|
|
69
|
+
# Physical PGMQ queue names for every registered event subscriber, so a
|
|
70
|
+
# wildcard (`queues: ['*']`) worker can exclude them — an event queue
|
|
71
|
+
# carries event payloads, not ActiveJob jobs, and a job worker that adopts
|
|
72
|
+
# one fails to deserialize and DLQ-moves the event (issue #333). Returns a
|
|
73
|
+
# Set of prefixed names (`#{queue_prefix}_<subscriber>`), matching the
|
|
74
|
+
# pgmq.meta rows the wildcard resolver diffs against.
|
|
75
|
+
def event_queue_names
|
|
76
|
+
@subscribers.to_set { |s| Pgbus.configuration.queue_name(s.queue_name) }
|
|
77
|
+
end
|
|
78
|
+
|
|
39
79
|
def clear!
|
|
40
80
|
@mutex.synchronize { @subscribers.clear }
|
|
41
81
|
end
|
|
42
82
|
|
|
43
83
|
private
|
|
44
84
|
|
|
85
|
+
# Rake task-name prefixes during which pgbus must NOT open a PGMQ
|
|
86
|
+
# connection: creating/dropping/migrating/loading the schema (a live
|
|
87
|
+
# connection blocks DROP DATABASE) and asset precompile (no DB expected).
|
|
88
|
+
SCHEMA_TASK_PREFIXES = %w[db: db_test: assets: webpacker: yarn:].freeze
|
|
89
|
+
private_constant :SCHEMA_TASK_PREFIXES
|
|
90
|
+
|
|
91
|
+
# True when running inside a rake schema/asset task, where setup_all!(safe:)
|
|
92
|
+
# should skip rather than open a connection. Detected from the invoked rake
|
|
93
|
+
# task names; false outside a Rake run.
|
|
94
|
+
def schema_task_context?
|
|
95
|
+
return false unless defined?(::Rake) && ::Rake.respond_to?(:application)
|
|
96
|
+
|
|
97
|
+
tasks = ::Rake.application.top_level_tasks
|
|
98
|
+
tasks.any? { |t| SCHEMA_TASK_PREFIXES.any? { |prefix| t.to_s.start_with?(prefix) } }
|
|
99
|
+
rescue StandardError => e
|
|
100
|
+
Pgbus.logger.debug { "[Pgbus] schema_task_context? detection failed, assuming non-schema: #{e.class}: #{e.message}" }
|
|
101
|
+
false
|
|
102
|
+
end
|
|
103
|
+
|
|
45
104
|
def matches?(pattern, routing_key)
|
|
46
105
|
regex = pattern
|
|
47
106
|
.gsub(".", "\\.")
|
data/lib/pgbus/process/worker.rb
CHANGED
|
@@ -415,11 +415,20 @@ module Pgbus
|
|
|
415
415
|
Pgbus::StreamQueue.reset_cache!
|
|
416
416
|
stream_names = Pgbus::StreamQueue.all_names
|
|
417
417
|
|
|
418
|
+
# Event-bus subscriber queues also share the job namespace (pgbus_<handler>)
|
|
419
|
+
# but carry event payloads, not ActiveJob jobs. A wildcard worker that
|
|
420
|
+
# adopts one hands the event to the executor, which fails to deserialize
|
|
421
|
+
# it and DLQ-moves it out of the consumer's reach — so an app running the
|
|
422
|
+
# event bus had to hand-maintain an explicit queue list. Exclude them,
|
|
423
|
+
# like stream queues (issue #333).
|
|
424
|
+
event_names = Pgbus::EventBus::Registry.instance.event_queue_names
|
|
425
|
+
|
|
418
426
|
conn = Pgbus.configuration.connects_to ? Pgbus::BusRecord.connection : ActiveRecord::Base.connection
|
|
419
427
|
all_queues = conn.select_values("SELECT queue_name FROM pgmq.meta ORDER BY queue_name")
|
|
420
428
|
resolved = all_queues
|
|
421
429
|
.reject { |q| q.end_with?(dlq_suffix) }
|
|
422
430
|
.reject { |q| stream_names.include?(q) }
|
|
431
|
+
.reject { |q| event_names.include?(q) }
|
|
423
432
|
.map { |q| q.delete_prefix(prefix) }
|
|
424
433
|
|
|
425
434
|
if resolved.empty?
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Pgbus
|
|
4
|
+
module Streams
|
|
5
|
+
# Phlex-includable version of the +pgbus_stream_from+ Rails view helper, so
|
|
6
|
+
# a Phlex component can subscribe a page to a pgbus stream the same way it
|
|
7
|
+
# would with turbo-rails' +turbo_stream_from+:
|
|
8
|
+
#
|
|
9
|
+
# class ChatRoom < Phlex::HTML
|
|
10
|
+
# include Pgbus::Streams::PhlexHelpers
|
|
11
|
+
#
|
|
12
|
+
# def view_template
|
|
13
|
+
# pgbus_stream_from(@room)
|
|
14
|
+
# end
|
|
15
|
+
# end
|
|
16
|
+
#
|
|
17
|
+
# phlex-rails doesn't ship a +Phlex::Rails::Helpers::PgbusStreamFrom+, so
|
|
18
|
+
# apps used to hand-register this module locally. It is defined exactly like
|
|
19
|
+
# phlex-rails' own +Phlex::Rails::Helpers::TurboStreamFrom+ (register the
|
|
20
|
+
# method as an output helper resolved against the Rails view context).
|
|
21
|
+
#
|
|
22
|
+
# This file is NOT autoloaded — phlex-rails is an OPTIONAL dependency, so the
|
|
23
|
+
# module references +Phlex::Rails::HelperMacros+, which only exists when the
|
|
24
|
+
# app has phlex-rails. Require it explicitly from a Phlex component or an
|
|
25
|
+
# initializer: require "pgbus/streams/phlex_helpers".
|
|
26
|
+
module PhlexHelpers
|
|
27
|
+
extend Phlex::Rails::HelperMacros
|
|
28
|
+
|
|
29
|
+
register_output_helper def pgbus_stream_from(...) = nil
|
|
30
|
+
end
|
|
31
|
+
end
|
|
32
|
+
end
|
data/lib/pgbus/uniqueness.rb
CHANGED
|
@@ -62,9 +62,18 @@ module Pgbus
|
|
|
62
62
|
raise ArgumentError, "on_conflict must be one of: #{VALID_CONFLICTS.join(", ")}" unless VALID_CONFLICTS.include?(on_conflict)
|
|
63
63
|
raise ArgumentError, "key must be callable (Proc or lambda)" if key && !key.respond_to?(:call)
|
|
64
64
|
|
|
65
|
+
# Record whether an explicit key was given. With NO explicit key the key
|
|
66
|
+
# defaults to the class name; that is safe for a no-argument job (one
|
|
67
|
+
# logical instance — e.g. a recurring CleanupJob that must not overlap
|
|
68
|
+
# itself) but a silent-correctness footgun for a job that takes per-record
|
|
69
|
+
# arguments: `ImportOrderJob.perform_later(order_id)` would collapse every
|
|
70
|
+
# order into ONE per-class singleton. resolve_key raises at resolve time
|
|
71
|
+
# when the class-name default meets non-empty arguments (see #333); the
|
|
72
|
+
# no-arg case keeps working. explicit_key marks which is which.
|
|
65
73
|
@pgbus_uniqueness = {
|
|
66
74
|
strategy: strategy,
|
|
67
75
|
key: key || ->(*) { name },
|
|
76
|
+
explicit_key: !key.nil?,
|
|
68
77
|
on_conflict: on_conflict
|
|
69
78
|
}.freeze
|
|
70
79
|
end
|
|
@@ -80,6 +89,7 @@ module Pgbus
|
|
|
80
89
|
return nil unless config
|
|
81
90
|
|
|
82
91
|
args = active_job.arguments
|
|
92
|
+
guard_class_name_default!(active_job, config, args)
|
|
83
93
|
last = args.last
|
|
84
94
|
key = if last.is_a?(Hash) && last.each_key.all?(Symbol)
|
|
85
95
|
config[:key].call(*args[...-1], **last)
|
|
@@ -93,6 +103,27 @@ module Pgbus
|
|
|
93
103
|
key
|
|
94
104
|
end
|
|
95
105
|
|
|
106
|
+
# Guards the class-name default key against the per-record collapse
|
|
107
|
+
# footgun (#333). When an :until_executed job was declared with NO explicit
|
|
108
|
+
# key (so the key is the class name) AND is enqueued WITH arguments, every
|
|
109
|
+
# distinct argument set would resolve to the same class-name key and
|
|
110
|
+
# collapse into one per-class singleton — almost never what the caller
|
|
111
|
+
# wants. Raise with an actionable message. A no-argument job keeps the
|
|
112
|
+
# class-name default (one logical instance, e.g. a recurring task that must
|
|
113
|
+
# not overlap itself), and :while_executing is unaffected (it acquires
|
|
114
|
+
# per-invocation at execution start, not by class-name identity at enqueue).
|
|
115
|
+
def guard_class_name_default!(active_job, config, args)
|
|
116
|
+
return if config[:explicit_key]
|
|
117
|
+
return unless config[:strategy] == :until_executed
|
|
118
|
+
return if args.nil? || args.empty?
|
|
119
|
+
|
|
120
|
+
raise ArgumentError,
|
|
121
|
+
"#{active_job.class.name} uses ensures_uniqueness strategy: :until_executed with no key: " \
|
|
122
|
+
"but is enqueued with arguments — the default key is the class name, which would collapse " \
|
|
123
|
+
"every distinct argument set into one per-class singleton. Pass an explicit " \
|
|
124
|
+
"key: ->(*args) { ... } that includes the arguments. See https://pgbus.dev/docs/upgrading-pgbus"
|
|
125
|
+
end
|
|
126
|
+
|
|
96
127
|
def inject_metadata(active_job, payload_hash)
|
|
97
128
|
config = uniqueness_config(active_job)
|
|
98
129
|
return payload_hash unless config
|
data/lib/pgbus/version.rb
CHANGED
|
@@ -13,6 +13,12 @@ module Pgbus
|
|
|
13
13
|
attr_accessor :auth_warned
|
|
14
14
|
end
|
|
15
15
|
|
|
16
|
+
# Default base controller — anything else means the app deliberately
|
|
17
|
+
# chose a controller (e.g. an AdminController) that already gates access,
|
|
18
|
+
# so the dashboard isn't actually open and the warning is a false positive.
|
|
19
|
+
DEFAULT_BASE_CONTROLLER = "::ActionController::Base"
|
|
20
|
+
private_constant :DEFAULT_BASE_CONTROLLER
|
|
21
|
+
|
|
16
22
|
private
|
|
17
23
|
|
|
18
24
|
def authenticate_pgbus!
|
|
@@ -30,6 +36,10 @@ module Pgbus
|
|
|
30
36
|
|
|
31
37
|
def warn_unauthenticated_dashboard
|
|
32
38
|
return if Pgbus::Web::Authentication.auth_warned
|
|
39
|
+
# A non-default base_controller_class signals host-level auth is in
|
|
40
|
+
# place; don't nag apps that gate the dashboard via a custom controller
|
|
41
|
+
# (they had to set a redundant web_auth lambda just to silence us — #334).
|
|
42
|
+
return unless default_base_controller?
|
|
33
43
|
|
|
34
44
|
Pgbus.logger.warn do
|
|
35
45
|
"[Pgbus] Dashboard is accessible without authentication. " \
|
|
@@ -38,6 +48,19 @@ module Pgbus
|
|
|
38
48
|
end
|
|
39
49
|
Pgbus::Web::Authentication.auth_warned = true
|
|
40
50
|
end
|
|
51
|
+
|
|
52
|
+
# True when base_controller_class is (still) the default ActionController::Base.
|
|
53
|
+
# Normalizes both sides so a Class value, the "::"-prefixed string, and the
|
|
54
|
+
# bare "ActionController::Base" string all compare equal — otherwise the
|
|
55
|
+
# warning would wrongly fire for an app that set the default in another form.
|
|
56
|
+
def default_base_controller?
|
|
57
|
+
normalize_controller_name(Pgbus.configuration.base_controller_class) ==
|
|
58
|
+
normalize_controller_name(DEFAULT_BASE_CONTROLLER)
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
def normalize_controller_name(value)
|
|
62
|
+
value.to_s.delete_prefix("::")
|
|
63
|
+
end
|
|
41
64
|
end
|
|
42
65
|
end
|
|
43
66
|
end
|
data/lib/pgbus.rb
CHANGED
|
@@ -103,6 +103,11 @@ module Pgbus
|
|
|
103
103
|
# the gem and loads the subsystem explicitly.
|
|
104
104
|
loader.ignore("#{__dir__}/pgbus/mcp")
|
|
105
105
|
loader.ignore("#{__dir__}/pgbus/mcp.rb")
|
|
106
|
+
# The Phlex stream helper references Phlex::Rails::HelperMacros from the
|
|
107
|
+
# optional `phlex-rails` gem. Keep it out of Zeitwerk so eager_load never
|
|
108
|
+
# touches those constants; a Phlex app requires it explicitly via
|
|
109
|
+
# `require "pgbus/streams/phlex_helpers"`.
|
|
110
|
+
loader.ignore("#{__dir__}/pgbus/streams/phlex_helpers.rb")
|
|
106
111
|
# Vendor integrations are loaded conditionally (when the vendor gem
|
|
107
112
|
# is present) by lib/pgbus/engine.rb. Keeping them out of Zeitwerk
|
|
108
113
|
# means we don't reference vendor constants at autoload time.
|
|
@@ -209,6 +214,19 @@ module Pgbus
|
|
|
209
214
|
Streams::Key.stream_key!(key)
|
|
210
215
|
end
|
|
211
216
|
|
|
217
|
+
# The maximum length (in characters) a stream name may be before it
|
|
218
|
+
# overflows the pgbus queue-name budget. A stream name becomes the physical
|
|
219
|
+
# queue `#{queue_prefix}_<name>`, and PGMQ caps queue names, so the usable
|
|
220
|
+
# budget is `QueueNameValidator::MAX_QUEUE_NAME_LENGTH - queue_prefix.length
|
|
221
|
+
# - 1`. Depends on `config.queue_prefix`. Use it to size or truncate stream
|
|
222
|
+
# identifiers up front instead of hand-computing the budget at every call
|
|
223
|
+
# site.
|
|
224
|
+
#
|
|
225
|
+
# name = candidate.length > Pgbus.stream_name_budget ? Pgbus.stream_key(record) : candidate
|
|
226
|
+
def stream_name_budget
|
|
227
|
+
Streams::Key.queue_name_budget
|
|
228
|
+
end
|
|
229
|
+
|
|
212
230
|
# Publish an event to the bus — the top-level shortcut for
|
|
213
231
|
# `Pgbus::EventBus::Publisher.publish`, symmetric with `Pgbus.stream`.
|
|
214
232
|
#
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: pgbus
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.11.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Mikael Henriksson
|
|
@@ -352,6 +352,7 @@ files:
|
|
|
352
352
|
- lib/pgbus/streams/envelope.rb
|
|
353
353
|
- lib/pgbus/streams/filters.rb
|
|
354
354
|
- lib/pgbus/streams/key.rb
|
|
355
|
+
- lib/pgbus/streams/phlex_helpers.rb
|
|
355
356
|
- lib/pgbus/streams/pool_autoscaler.rb
|
|
356
357
|
- lib/pgbus/streams/pool_trigger.rb
|
|
357
358
|
- lib/pgbus/streams/presence.rb
|