pgmq-ruby 0.7.2 → 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 570ab8b29056f8831e828d18b114f49cf708b9350a7a0b6fd20645d73d3cd6c2
4
- data.tar.gz: 246a625f56ca744cada699f99b631651d814232cafdf9fca6bf4b0c70d1e2b37
3
+ metadata.gz: 5e359ff6e278f15f414b12524d8922c6f7ae7cfb274ce770250ca347a88c160d
4
+ data.tar.gz: 821beca651d6115f7c1f9ef46efe63b5985eb07ac8f77ed3b31a26e51ff85b23
5
5
  SHA512:
6
- metadata.gz: f88edaa90fde3e5bfc585f3e22702c1a0545d0b807d7ae8807dc39bd4ea3dd1f0857a55885b6d3122332fe5a8a06a7e0d282a65524aee6e89e8f04882e0b0792
7
- data.tar.gz: 06a01af44cfa4dd807ec23eaff5985c4fd72a3c2363f9b6706c89a8b6bf2e6c348cb71a176e23f7010ba7643210f8c461dc541865ede37b82888fa67e057ff04
6
+ metadata.gz: 31215de55d5ae155d1e364263c36c1fce1bcbf1d07774c8afbd9bde05e8e607dbf0bdb1c4bd62db527431dc9332e3828f78109ad4361963319a61db3a5a23520
7
+ data.tar.gz: e52b05c020270c1d90f50c238d95e042af6e8d1b69849f483de1e62695cd1f085bac5b762de76c3d35466355d52fc3fa9bbca316a576bbbab577beaa45beca71
data/CHANGELOG.md CHANGED
@@ -1,5 +1,93 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.8.0 (2026-10-25)
4
+
5
+ ### Consumer
6
+ - **[Feature]** Add `PGMQ::Consumer`, a minimal single-threaded consumer loop modelled on the rdkafka-ruby simple
7
+ consumer. `PGMQ::Consumer.new(client, queue, vt:, batch_size:, max_poll_seconds:, poll_interval_ms:, on_success:,
8
+ max_attempts:)` wraps an existing client; `#each` long-polls with `read_with_poll`, yields one `PGMQ::Message` at a
9
+ time and deletes (or archives, `on_success: :archive`) it when the block returns. A block exception stops the loop
10
+ and is re-raised without acknowledging the message, so it reappears after `vt` (at-least-once delivery). Messages
11
+ whose `read_ct` exceeds `max_attempts` (requires `batch_size: 1`) are archived without being yielded. `#stop` only
12
+ sets a flag, so it is safe in a signal trap; the loop finishes the messages it already read, then returns.
13
+ - **[Feature]** `PGMQ::Consumer` consumes several queues when given an Array of names (`PGMQ::Consumer.new(client,
14
+ %w[orders emails])`), like an rdkafka-ruby consumer subscribed to several topics. It reads them with
15
+ `read_multi_with_poll`, rotating the read order each poll, sets `queue_name` on every yielded message and
16
+ acknowledges each message on its own queue. With `max_attempts`, it reads one message per poll across all queues.
17
+
18
+ ### Metrics
19
+ - **[Feature]** `PGMQ::Metrics` exposes `queue_visible_length` (messages whose visibility timeout has passed, PGMQ
20
+ v1.5.0+) and `default_partition_length` (estimated rows in the default partitions of a partitioned queue and its
21
+ archive, PGMQ v1.13.0+; `nil` for non-partitioned queues, non-zero means pg_partman maintenance is failing). Both
22
+ are raw values like the other fields, and `nil` on PGMQ versions that do not return them.
23
+
24
+ ### Queue Management
25
+ - **[Feature]** `create_partitioned` accepts `premake:`, the number of partitions pg_partman creates ahead of the
26
+ current one for the queue and archive tables (PGMQ's default is 4). Raise it when writes outrun the pre-made
27
+ partitions and land in the default partition. Requires PGMQ v1.13.0+ when given; without it the SQL call is
28
+ unchanged.
29
+
30
+ ### Documentation
31
+ - **[Fix]** YARD examples used a non-existent `msg.payload`; they now use `msg.message`.
32
+ - **[Fix]** The README long-polling example now treats the `read_with_poll` result as an Array of messages.
33
+ - **[Fix]** The `enable_notify_insert` and `wait_for_notify` docs and the README now say that on PGMQ older than
34
+ v1.13.1, insert notifications never fire for a queue name with uppercase letters (pgmq/pgmq#593). Create queues with
35
+ lower-case names, or use PGMQ v1.13.1+.
36
+
37
+ ### Producer
38
+ - **[Fix]** A Float `delay:` in `produce`, `produce_batch`, `produce_topic` and `produce_batch_topic` no longer fails
39
+ with `PGMQ::Errors::ConnectionError` (`invalid input syntax for type integer`). A fractional delay such as `1.5` now
40
+ makes the message visible 1.5 seconds later: it is sent to PGMQ's `timestamptz` overload as `clock_timestamp()`
41
+ plus the delay (PGMQ v1.10.0+), the same way PGMQ handles an integer delay. Integer and `Time` delays are unchanged.
42
+ - **[Breaking]** `produce` and `produce_batch` raise `PGMQ::Errors::MessageInvalidError` when a message is not a
43
+ String (including `nil`) or headers are neither a String nor `nil`. Previously a Hash or `nil` was bound as SQL
44
+ NULL, so the call returned a `msg_id` but the message or headers were stored as NULL without an error. Any other
45
+ non-String value now raises too, including Integer, Float, `true` and Array values that were stored as JSON before,
46
+ `false` headers that were dropped, and Hash values on a connection with a query type map
47
+ (`PG::BasicTypeMapForQueries`), which were stored as JSON objects. The error message names the invalid key
48
+ (`message` or `headers`; in a batch `messages.<index>.message` or `messages.<index>.headers`) and the reason, such
49
+ as `must be a JSON String`. Inside `transaction` the error is wrapped in `PGMQ::Errors::ConnectionError` like any
50
+ other error. Serialize payloads to JSON before producing. The check is the new `PGMQ::Contracts::Message`, built on
51
+ `PGMQ::Contractable`, the contract layer extracted from karafka-core (no new dependency).
52
+
53
+ ### Validation
54
+ - **[Breaking]** Every argument check now runs through a contract in `PGMQ::Contracts` (`Message`, `Batch`,
55
+ `MultiQueue`, `Consumer`, `Autovacuum`, `Connection`), with the error messages in `config/locales/errors.yml`.
56
+ Client calls raise new validation errors instead of `ArgumentError`, like WaterDrop:
57
+ `PGMQ::Errors::MessageInvalidError` (`produce`, `produce_batch`, `produce_batch_topic`),
58
+ `PGMQ::Errors::MultiQueueInvalidError` (`read_multi`, `read_multi_with_poll`, `pop_multi`, `delete_multi`,
59
+ `archive_multi`, `set_vt_multi`) and `PGMQ::Errors::AutovacuumSettingsInvalidError` (`tune_autovacuum`). They
60
+ inherit from `PGMQ::Errors::BaseError`, not from `ArgumentError`, so `rescue ArgumentError` no longer catches them.
61
+ `PGMQ::Consumer.new`, the connection parameters and the `PGMQ::Connection` reconnectable error settings still raise
62
+ `PGMQ::Errors::ConfigurationError`. The message is now a Hash of each invalid key and its reason, such as
63
+ `queue_names` and `cannot be empty`. The headers length error of `produce_batch` and `produce_batch_topic` no longer
64
+ includes the two lengths. New checks: `produce_batch` and `produce_batch_topic` raise
65
+ `PGMQ::Errors::MessageInvalidError` when messages is not an Array or headers is neither an Array nor `nil` (String
66
+ headers failed in SQL with `PGMQ::Errors::ConnectionError` before), and `tune_autovacuum` checks the queue and
67
+ archive settings before it changes any table (an unknown archive setting raised only after the queue table was
68
+ changed). A setting value that is not numeric raises `PGMQ::Errors::AutovacuumSettingsInvalidError` too, also before
69
+ any table changes (it raised `ArgumentError` from `Float()` or `Integer()` for a String, `TypeError` for other values
70
+ such as `nil` or `true`, so `rescue TypeError` no longer catches it). Two calls that returned without an error now
71
+ raise a validation error: `produce_batch` and `produce_batch_topic` with an empty messages Array and non-empty or
72
+ String headers (they returned `[]`), and `tune_autovacuum` (and `create` with `tune_autovacuum:`) with
73
+ `archive: false` and an unknown or non-numeric archive setting (it ignored the archive settings).
74
+
75
+ ### Transactions
76
+ - **[Fix]** The transactional client yielded by `PGMQ::Client#transaction` now raises the new
77
+ `PGMQ::Errors::TransactionError` when used from another thread or fiber, or after the block has returned.
78
+ Previously such calls checked out a different pool connection and committed outside the transaction, so a
79
+ rollback did not undo them (and with `pool_size: 1` they waited for the pool timeout). An unrescued
80
+ `TransactionError` propagates from `#transaction` as is, not wrapped in `PGMQ::Errors::ConnectionError`.
81
+
82
+ ### Testing
83
+ - **[Fix]** Transaction tests produce JSON strings instead of Hashes, which were stored as NULL.
84
+ - **[Change]** Add `spec/integration/concurrency_overlap_spec.rb`, which proves concurrency by asserting overlapping
85
+ execution intervals (not order or absolute timings): threads and fibers overlap on long polls and `LISTEN` waits,
86
+ the reactor keeps running, a pool of one serializes work, and the transactional client rejects other threads and
87
+ fibers. `ConcurrencyTracker` now uses the monotonic clock, and the fiber long-polling example asserts its result.
88
+ - **[Fix]** `FiberSchedulerHelper.with_scheduler` uses `Sync` instead of a top-level `Async`, so a failed assertion
89
+ inside a fiber scheduler integration spec now fails the spec instead of being only logged.
90
+
3
91
  ## 0.7.2 (2026-08-31)
4
92
 
5
93
  ### Dependencies
data/README.md CHANGED
@@ -15,7 +15,7 @@ PGMQ-Ruby is a Ruby client for PGMQ (PostgreSQL Message Queue). It provides dire
15
15
  - **Like Sidekiq/Resque** - but without Redis, using PostgreSQL for both data and queues
16
16
  - **Like rdkafka-ruby** - a thin, efficient wrapper around the underlying system (PGMQ SQL functions)
17
17
 
18
- > **Architecture Note**: This library follows the rdkafka-ruby/Karafka pattern - `pgmq-ruby` is the low-level foundation, while higher-level features (job processing, Rails integration, retry strategies) will live in `pgmq-framework` (similar to how Karafka builds on rdkafka-ruby).
18
+ > **Architecture Note**: This library follows the rdkafka-ruby/Karafka pattern - `pgmq-ruby` is the low-level foundation, while higher-level features (job processing, Rails integration, retry strategies, middleware) will live in `pgmq-framework` (similar to how Karafka builds on rdkafka-ruby). Like rdkafka-ruby's simple consumer, `pgmq-ruby` ships a minimal single-threaded [`PGMQ::Consumer`](#consumer) loop for one or more queues.
19
19
 
20
20
  ## Table of Contents
21
21
 
@@ -29,12 +29,14 @@ PGMQ-Ruby is a Ruby client for PGMQ (PostgreSQL Message Queue). It provides dire
29
29
  - [Sending Messages](#sending-messages)
30
30
  - [Reading Messages](#reading-messages)
31
31
  - [Grouped Reading](#grouped-reading)
32
+ - [Consumer](#consumer)
32
33
  - [Message Lifecycle](#message-lifecycle)
33
34
  - [Monitoring](#monitoring)
34
35
  - [Transaction Support](#transaction-support)
35
36
  - [Topic Routing](#topic-routing-amqp-like-patterns)
36
37
  - [Message Object](#message-object)
37
38
  - [Working with JSON](#working-with-json)
39
+ - [Error Handling](#error-handling)
38
40
  - [Development](#development)
39
41
  - [Author](#author)
40
42
 
@@ -78,7 +80,7 @@ This gem provides complete support for all core PGMQ SQL functions. Based on the
78
80
  | | `set_vt_batch` | Batch update visibility timeouts | ✅ |
79
81
  | | `set_vt_multi` | Update visibility timeouts across multiple queues | ✅ |
80
82
  | | `list_queues` | List all queues with metadata | ✅ |
81
- | | `metrics` | Get queue metrics (length, age, total messages) | ✅ |
83
+ | | `metrics` | Get queue metrics (length, visible length, age, total messages, default partition length) | ✅ |
82
84
  | | `metrics_all` | Get metrics for all queues | ✅ |
83
85
  | | `enable_notify_insert` | Enable PostgreSQL NOTIFY on insert | ✅ |
84
86
  | | `update_notify_insert` | Update throttle interval on an existing NOTIFY trigger | ✅ |
@@ -88,13 +90,14 @@ This gem provides complete support for all core PGMQ SQL functions. Based on the
88
90
  | **Ruby Enhancements** | Transaction Support | Atomic operations via `client.transaction do \|txn\|` | ✅ |
89
91
  | | Conditional Filtering | Server-side JSONB filtering with `conditional:` | ✅ |
90
92
  | | Multi-Queue Ops | Read/pop/delete/archive from multiple queues | ✅ |
93
+ | | Consumer | Minimal single-threaded `PGMQ::Consumer` loop over one or more queues with ack on success | ✅ |
91
94
  | | Queue Validation | 48-character limit and name validation | ✅ |
92
95
  | | Connection Pooling | Thread-safe connection pool for concurrency | ✅ |
93
96
  | | Pluggable Serializers | JSON (default) with custom serializer support | ✅ |
94
97
 
95
98
  ## Requirements
96
99
 
97
- - Ruby 3.2+
100
+ - Ruby 3.3+
98
101
  - PostgreSQL 14-18 with PGMQ extension installed
99
102
 
100
103
  ### Installing PGMQ Extension
@@ -371,7 +374,7 @@ misconfiguration can't silently disable retries.
371
374
 
372
375
  **Connection Pool Benefits:**
373
376
  - **Thread-safe** - Multiple threads can safely share a single client
374
- - **Fiber-aware** - Works with Ruby 3.0+ Fiber Scheduler for non-blocking I/O (tested with the `async` gem)
377
+ - **Fiber-aware** - Works with the Ruby 3.3+ Fiber Scheduler for non-blocking I/O (tested with the `async` gem)
375
378
  - **Auto-reconnect** - Recovers from lost connections (configurable, extendable)
376
379
  - **Health checks** - Verifies connections before use to prevent stale connection errors
377
380
  - **Monitoring** - Track pool utilization with `client.stats`
@@ -392,6 +395,13 @@ client.create_partitioned("queue_name",
392
395
  retention_interval: "7 days"
393
396
  ) # => true/false
394
397
 
398
+ # Pre-make more partitions ahead of the current one (default 4; PGMQ v1.13.0+)
399
+ client.create_partitioned("queue_name",
400
+ partition_interval: "10000",
401
+ retention_interval: "100000",
402
+ premake: 12
403
+ ) # => true/false
404
+
395
405
  # Create unlogged queue (faster, no crash recovery)
396
406
  client.create_unlogged("queue_name") # => true/false
397
407
 
@@ -576,8 +586,8 @@ msg = client.read("queue_name", vt: 30)
576
586
  # Read batch
577
587
  messages = client.read_batch("queue_name", vt: 30, qty: 10)
578
588
 
579
- # Read with long-polling
580
- msg = client.read_with_poll("queue_name",
589
+ # Read with long-polling (returns an Array of messages)
590
+ messages = client.read_with_poll("queue_name",
581
591
  vt: 30,
582
592
  qty: 1,
583
593
  max_poll_seconds: 5,
@@ -586,6 +596,8 @@ msg = client.read_with_poll("queue_name",
586
596
 
587
597
  # Event-driven consumption via LISTEN/NOTIFY (more efficient than long-polling at low message rates)
588
598
  # Step 1: enable server-side NOTIFY trigger once (idempotent)
599
+ # On PGMQ < v1.13.1, NOTIFY never fires for a queue name with uppercase letters
600
+ # (pgmq/pgmq#593): create queues with lower-case names, or use PGMQ v1.13.1+.
589
601
  client.enable_notify_insert("queue_name")
590
602
 
591
603
  # Step 2: consumer loop — wake up on notification, then drain all available messages.
@@ -752,6 +764,57 @@ CREATE INDEX idx_orders_status
752
764
  ON pgmq.q_orders USING gin ((message->'status'));
753
765
  ```
754
766
 
767
+ ### Consumer
768
+
769
+ `PGMQ::Consumer` is a minimal, single-threaded consumer loop for one or more queues, similar to the simple consumer of rdkafka-ruby. It wraps an existing client (it does not close it), long-polls with `read_with_poll` (`read_multi_with_poll` for several queues), yields one message at a time and acknowledges it on its own queue when the block returns.
770
+
771
+ ```ruby
772
+ client = PGMQ::Client.new(ENV.fetch("DATABASE_URL"))
773
+
774
+ consumer = PGMQ::Consumer.new(client, "orders",
775
+ vt: 30, # visibility timeout (seconds) of read messages
776
+ batch_size: 1, # messages read per queue per poll (still yielded one at a time)
777
+ max_poll_seconds: 5, # how long one poll waits for messages
778
+ poll_interval_ms: 100,
779
+ on_success: :delete, # or :archive
780
+ max_attempts: nil # nil = retry forever; N = archive once read_ct > N (needs batch_size: 1)
781
+ )
782
+
783
+ # stop only sets a flag, so it is safe in a signal trap
784
+ trap("TERM") { consumer.stop }
785
+ trap("INT") { consumer.stop }
786
+
787
+ consumer.each do |message|
788
+ process(JSON.parse(message.message))
789
+ end # returns after stop, once the messages already read are done
790
+
791
+ client.close
792
+ ```
793
+
794
+ To consume several queues (up to 50) with one consumer, pass an Array of names, like subscribing an rdkafka-ruby consumer to several topics. Every yielded message has `queue_name` set:
795
+
796
+ ```ruby
797
+ consumer = PGMQ::Consumer.new(client, %w[orders emails], vt: 30)
798
+
799
+ consumer.each do |message|
800
+ case message.queue_name
801
+ when "orders" then process_order(message)
802
+ when "emails" then send_email(message)
803
+ end
804
+ end
805
+ ```
806
+
807
+ Semantics:
808
+
809
+ - **Acknowledgement**: when the block returns, the message is deleted (or archived with `on_success: :archive`).
810
+ - **Errors**: when the block raises, the message is not acknowledged, the loop stops and the error is re-raised. The message becomes visible again after `vt`. Retry by calling `each` again (e.g. from a supervisor).
811
+ - **Retry limit**: with `max_attempts: N`, a message whose `read_ct` exceeds `N` is archived without being yielded. It requires `batch_size: 1`, and with several queues one message is then read per poll: every read raises `read_ct`, so messages released after another message failed would count a read they never got.
812
+ - **Stopping**: `stop` takes effect once the messages already read in the current poll are processed, or after the current poll (up to `max_poll_seconds`) when the queue is empty. A stopped consumer stays stopped.
813
+ - **Batches**: when the block raises, the messages of the batch that were not yielded yet are released (`vt` set to 0) right away.
814
+ - **Delivery**: at-least-once. If processing takes longer than `vt`, another reader may receive the same message; extend it from the block with `client.set_vt(message.queue_name, message.msg_id, vt: 30)` for long jobs.
815
+ - **Several queues**: one `read_multi_with_poll` query reads all queues. It polls from the client (sleeping `poll_interval_ms` between reads) instead of waiting on the server like `read_with_poll`. The read order rotates every poll, so a busy queue does not starve the others.
816
+ - **Scope**: one thread. Run one consumer per thread or process to scale; consumers can share a client, and each needs its own connection from the client's pool while polling.
817
+
755
818
  ### Message Lifecycle
756
819
 
757
820
  ```ruby
@@ -801,6 +864,8 @@ client.convert_archive_partitioned("queue_name",
801
864
  )
802
865
 
803
866
  # Enable PostgreSQL NOTIFY for a queue (for LISTEN-based consumers)
867
+ # On PGMQ < v1.13.1, NOTIFY never fires for a queue name with uppercase letters
868
+ # (pgmq/pgmq#593): create queues with lower-case names, or use PGMQ v1.13.1+.
804
869
  client.enable_notify_insert("queue_name", throttle_interval_ms: 250)
805
870
 
806
871
  # Update the throttle interval without disabling/re-enabling the trigger
@@ -834,6 +899,8 @@ puts metrics.queue_length # => 42
834
899
  puts metrics.oldest_msg_age_sec # => 120
835
900
  puts metrics.newest_msg_age_sec # => 5
836
901
  puts metrics.total_messages # => 1000
902
+ puts metrics.queue_visible_length # => 40 (PGMQ v1.5.0+)
903
+ puts metrics.default_partition_length # => nil for non-partitioned queues, 0 is healthy (PGMQ v1.13.0+)
837
904
 
838
905
  # Get all queue metrics
839
906
  all_metrics = client.metrics_all
@@ -902,6 +969,8 @@ end
902
969
  - All operations within the block execute in a single PostgreSQL transaction
903
970
  - If any operation fails, the entire transaction is rolled back automatically
904
971
  - The transactional client delegates all `PGMQ::Client` methods for convenience
972
+ - The transactional client is bound to the thread/fiber that opened the transaction and is valid only inside the
973
+ block: using it from another thread or fiber, or after the block returns, raises `PGMQ::Errors::TransactionError`
905
974
 
906
975
  **Use Cases:**
907
976
 
@@ -1143,6 +1212,40 @@ msg = helper.read("orders", vt: 30)
1143
1212
  puts msg.data["order_id"] # => 123
1144
1213
  ```
1145
1214
 
1215
+ ## Error Handling
1216
+
1217
+ All PGMQ-Ruby errors inherit from `PGMQ::Errors::BaseError`. When PostgreSQL rejects a call, the client raises
1218
+ `PGMQ::Errors::ConnectionError`, not only when the connection fails or is lost: a missing queue, invalid JSON or an
1219
+ invalid integer raise it too. Outside a transaction, the original `PG::Error` is in `#cause`.
1220
+
1221
+ ```ruby
1222
+ begin
1223
+ client.produce("orders", "not json")
1224
+ rescue PGMQ::Errors::ConnectionError => e
1225
+ e.message # => "Database connection error: ERROR: invalid input syntax for type json ..."
1226
+ e.cause.class # => PG::InvalidTextRepresentation
1227
+ end
1228
+ ```
1229
+
1230
+ Invalid arguments are rejected by a contract before they reach PostgreSQL, with a validation error:
1231
+
1232
+ - `PGMQ::Errors::MessageInvalidError` from `produce`, `produce_batch` and `produce_batch_topic`, for example when a
1233
+ message is not a String or its headers are neither a String nor `nil`
1234
+ - `PGMQ::Errors::MultiQueueInvalidError` from the multi-queue operations (`read_multi`, `read_multi_with_poll`,
1235
+ `pop_multi`, `delete_multi`, `archive_multi`, `set_vt_multi`)
1236
+ - `PGMQ::Errors::AutovacuumSettingsInvalidError` from `tune_autovacuum` for an unknown storage parameter or a
1237
+ non-numeric value
1238
+ - `PGMQ::Errors::ConfigurationError` from `PGMQ::Consumer.new` and invalid connection settings
1239
+
1240
+ The error message names each invalid key and the reason, for example `messages.1.message` for the second message of a
1241
+ batch.
1242
+ Inside a transaction it arrives as `PGMQ::Errors::ConnectionError`, as described below.
1243
+
1244
+ A failed `client.transaction` block, including an exception your own code raises in it, raises
1245
+ `PGMQ::Errors::ConnectionError` with the message `Transaction failed: ...`. Its `#cause` is the error raised in the
1246
+ block. For a database error that is the inner `PGMQ::Errors::ConnectionError`, so the `PG::Error` is in
1247
+ `e.cause.cause`. A `PGMQ::Errors::TransactionError` is raised unchanged.
1248
+
1146
1249
  ## Development
1147
1250
 
1148
1251
  ```bash
@@ -0,0 +1,46 @@
1
+ en:
2
+ validations:
3
+ message:
4
+ missing: needs to be present
5
+ message_format: must be a JSON String
6
+ headers_format: must be a JSON String or nil
7
+
8
+ batch:
9
+ missing: needs to be present
10
+ messages_format: must be an Array
11
+ headers_format: must be an Array or nil
12
+ headers_length: must have one element per message
13
+
14
+ multi_queue:
15
+ queue_names_format: must be an Array
16
+ queue_names_empty: cannot be empty
17
+ queue_names_too_many: cannot exceed 50 queues
18
+ deletions_format: must be a Hash
19
+ archives_format: must be a Hash
20
+ updates_format: must be a Hash
21
+
22
+ consumer:
23
+ missing: needs to be present
24
+ queue_names_empty: needs at least one queue name
25
+ queue_names_too_many: cannot exceed 50 queues
26
+ queue_names_unique: must be unique (case-insensitive)
27
+ on_success_format: must be :delete or :archive
28
+ vt_format: must be a positive Integer
29
+ batch_size_format: must be a positive Integer
30
+ max_poll_seconds_format: must be a positive Integer
31
+ poll_interval_ms_format: must be a positive Integer
32
+ max_attempts_format: must be a positive Integer or nil
33
+ max_attempts_batch_size: "requires batch_size: 1"
34
+
35
+ autovacuum:
36
+ missing: needs to be present
37
+ queue_settings_format: must contain only known storage parameters
38
+ archive_settings_format: must contain only known storage parameters
39
+ queue_settings_values: must contain only numeric values
40
+ archive_settings_values: must contain only numeric values
41
+
42
+ connection:
43
+ conn_params_empty: are required
44
+ conn_params_format: must be a connection String, a non-empty Hash or a callable
45
+ reconnectable_error_patterns_format: must contain Strings or Regexps
46
+ reconnectable_error_classes_format: must contain Exception subclasses
@@ -42,12 +42,10 @@ module PGMQ
42
42
  autovacuum_analyze_scale_factor: 0.05
43
43
  }.freeze
44
44
 
45
- # Storage parameters whose values are interpolated as Floats; everything else is interpolated as an Integer.
46
- FLOAT_SETTINGS = %i[
47
- autovacuum_vacuum_scale_factor
48
- autovacuum_analyze_scale_factor
49
- ].freeze
50
- private_constant :FLOAT_SETTINGS
45
+ # Validates the storage settings before they are interpolated into SQL
46
+ CONTRACT = Contracts::Autovacuum.new
47
+
48
+ private_constant :CONTRACT
51
49
 
52
50
  # Tunes autovacuum and storage parameters on a queue's underlying tables.
53
51
  #
@@ -64,6 +62,8 @@ module PGMQ
64
62
  # @param archive_settings [Hash{Symbol=>Numeric}] storage params for the archive table, merged onto
65
63
  # {DEFAULT_ARCHIVE_SETTINGS}
66
64
  # @return [void]
65
+ # @raise [PGMQ::Errors::AutovacuumSettingsInvalidError] if queue_settings or archive_settings has an unknown
66
+ # storage parameter or a value that is not numeric
67
67
  # @raise [PGMQ::Errors::InvalidQueueNameError] if the queue name is invalid
68
68
  # @raise [PGMQ::Errors::ConnectionError] if the database operation fails (e.g. the queue does not exist)
69
69
  #
@@ -109,11 +109,19 @@ module PGMQ
109
109
  # @param archive_settings [Hash] archive-table overrides, merged onto {DEFAULT_ARCHIVE_SETTINGS}
110
110
  # @return [void]
111
111
  def tune_autovacuum_on(conn, queue_name, queue_settings: {}, archive: true, archive_settings: {})
112
- alter_storage(conn, "q_#{queue_name}", DEFAULT_QUEUE_SETTINGS.merge(symbolize(queue_settings)))
112
+ queue_settings = symbolize(queue_settings)
113
+ archive_settings = symbolize(archive_settings)
114
+
115
+ CONTRACT.validate!(
116
+ { queue_settings: queue_settings, archive_settings: archive_settings },
117
+ Errors::AutovacuumSettingsInvalidError
118
+ )
119
+
120
+ alter_storage(conn, "q_#{queue_name}", DEFAULT_QUEUE_SETTINGS.merge(queue_settings))
113
121
 
114
122
  return unless archive
115
123
 
116
- alter_storage(conn, "a_#{queue_name}", DEFAULT_ARCHIVE_SETTINGS.merge(symbolize(archive_settings)))
124
+ alter_storage(conn, "a_#{queue_name}", DEFAULT_ARCHIVE_SETTINGS.merge(archive_settings))
117
125
  end
118
126
 
119
127
  # Issues a single ALTER TABLE setting the given storage parameters on one pgmq table.
@@ -125,8 +133,8 @@ module PGMQ
125
133
  #
126
134
  # Identifiers cannot be passed as bind parameters, so the table name is quoted with the connection's
127
135
  # +quote_ident+. The queue name has already passed {#validate_queue_name!} (strict identifier rules); the
128
- # parameter names are validated against the known default keys; and the values are coerced to Float/Integer, so
129
- # no untrusted text reaches the statement.
136
+ # parameter names and values are validated by {PGMQ::Contracts::Autovacuum}; and the values are coerced to
137
+ # Float/Integer, so no untrusted text reaches the statement.
130
138
  #
131
139
  # @param conn [PG::Connection] database connection
132
140
  # @param table [String] unqualified table name (e.g. "q_orders")
@@ -134,38 +142,19 @@ module PGMQ
134
142
  # @return [void]
135
143
  def alter_storage(conn, table, settings)
136
144
  qualified = "pgmq.#{conn.quote_ident(table.downcase)}"
137
- clause = settings.map { |key, value| "#{storage_param_name(key)} = #{coerce_setting(key, value)}" }.join(", ")
145
+ clause = settings.map { |key, value| "#{key} = #{coerce_setting(key, value)}" }.join(", ")
138
146
 
139
147
  conn.exec("ALTER TABLE #{qualified} SET (#{clause})")
140
148
  end
141
149
 
142
- # Validates a storage-parameter name against the known keys before it is interpolated into SQL.
143
- #
144
- # Parameter names cannot be bound; allow-listing them against the union of the queue and archive default keys
145
- # keeps an arbitrary Symbol from being injected as a raw identifier.
146
- #
147
- # @param key [Symbol] storage parameter name
148
- # @return [String] the validated parameter name
149
- # @raise [ArgumentError] if the key is not a recognised storage parameter
150
- def storage_param_name(key)
151
- return key.to_s if known_storage_params.include?(key)
152
-
153
- raise ArgumentError, "Unknown storage parameter #{key.inspect}; allowed: #{known_storage_params.to_a.sort}"
154
- end
155
-
156
- # @return [Set<Symbol>] the union of queue and archive default keys
157
- def known_storage_params
158
- @known_storage_params ||= (DEFAULT_QUEUE_SETTINGS.keys + DEFAULT_ARCHIVE_SETTINGS.keys).to_set
159
- end
160
-
161
- # Coerces a setting value to its SQL form: Float for scale factors, Integer for everything else. Raising on
162
- # non-numeric input is the injection guard for values.
150
+ # Coerces a setting value to its SQL form: Float for scale factors, Integer for everything else.
151
+ # {PGMQ::Contracts::Autovacuum} has already rejected values that do not coerce.
163
152
  #
164
153
  # @param key [Symbol] storage parameter name
165
154
  # @param value [Numeric, String] the value to coerce
166
155
  # @return [Float, Integer]
167
156
  def coerce_setting(key, value)
168
- FLOAT_SETTINGS.include?(key) ? Float(value) : Integer(value)
157
+ Contracts::Autovacuum::FLOAT_SETTINGS.include?(key) ? Float(value) : Integer(value)
169
158
  end
170
159
 
171
160
  # Symbolizes top-level keys of a settings Hash so callers may pass either Symbol or String keys.
@@ -17,7 +17,7 @@ module PGMQ
17
17
  # @example
18
18
  # msg = client.read("orders", vt: 30)
19
19
  # if msg
20
- # process(msg.payload)
20
+ # process(msg.message)
21
21
  # client.delete("orders", msg.msg_id)
22
22
  # end
23
23
  #
@@ -60,7 +60,7 @@ module PGMQ
60
60
  # @example
61
61
  # messages = client.read_batch("orders", vt: 30, qty: 10)
62
62
  # messages.each do |msg|
63
- # process(msg.payload)
63
+ # process(msg.message)
64
64
  # client.delete("orders", msg.msg_id)
65
65
  # end
66
66
  #
@@ -41,6 +41,9 @@ module PGMQ
41
41
  #
42
42
  # @example Disable throttling (notify on every insert)
43
43
  # client.enable_notify_insert("orders", throttle_interval_ms: 0)
44
+ # @note On PGMQ older than v1.13.1, notifications never fire for a queue name with uppercase letters
45
+ # (pgmq/pgmq#593). The call succeeds, but no NOTIFY is sent. Create the queue with a lower-case name, or use
46
+ # PGMQ v1.13.1+.
44
47
  def enable_notify_insert(queue_name, throttle_interval_ms: 250)
45
48
  validate_queue_name!(queue_name)
46
49
 
@@ -203,6 +206,9 @@ module PGMQ
203
206
  # end
204
207
  # @note Orchestration (retry loop, reconnect-on-drop, graceful shutdown) is the caller's responsibility.
205
208
  # This method is a thin primitive - it listens once, waits, and returns.
209
+ # @note On PGMQ older than v1.13.1, no notification arrives for a queue name with uppercase letters
210
+ # (pgmq/pgmq#593), so this method never returns a notification. Create the queue with a lower-case name,
211
+ # or use PGMQ v1.13.1+.
206
212
  def wait_for_notify(queue_name, timeout: nil)
207
213
  validate_queue_name!(queue_name)
208
214
  # PGMQ trigger fires pg_notify('pgmq.q_<queue>.INSERT', NULL)
@@ -14,7 +14,7 @@ module PGMQ
14
14
  #
15
15
  # @example
16
16
  # msg = client.pop("orders")
17
- # process(msg.payload) if msg
17
+ # process(msg.message) if msg
18
18
  def pop(queue_name)
19
19
  validate_queue_name!(queue_name)
20
20
 
@@ -35,7 +35,7 @@ module PGMQ
35
35
  #
36
36
  # @example Pop up to 10 messages
37
37
  # messages = client.pop_batch("orders", 10)
38
- # messages.each { |msg| process(msg.payload) }
38
+ # messages.each { |msg| process(msg.message) }
39
39
  def pop_batch(queue_name, qty)
40
40
  validate_queue_name!(queue_name)
41
41
  return [] if qty <= 0
@@ -109,6 +109,7 @@ module PGMQ
109
109
  #
110
110
  # @param deletions [Hash] hash of queue_name => array of msg_ids
111
111
  # @return [Hash] hash of queue_name => array of deleted msg_ids
112
+ # @raise [PGMQ::Errors::MultiQueueInvalidError] if deletions is not a Hash
112
113
  #
113
114
  # @example Delete messages from multiple queues
114
115
  # client.delete_multi({
@@ -123,7 +124,7 @@ module PGMQ
123
124
  # deletions = messages.group_by(&:queue_name).transform_values { |mss| mss.map(&:msg_id) }
124
125
  # client.delete_multi(deletions)
125
126
  def delete_multi(deletions)
126
- raise ArgumentError, "deletions must be a hash" unless deletions.is_a?(Hash)
127
+ MULTI_QUEUE_CONTRACT.validate!({ deletions: deletions }, Errors::MultiQueueInvalidError)
127
128
  return {} if deletions.empty?
128
129
 
129
130
  # Validate all queue names
@@ -202,6 +203,7 @@ module PGMQ
202
203
  #
203
204
  # @param archives [Hash] hash of queue_name => array of msg_ids
204
205
  # @return [Hash] hash of queue_name => array of archived msg_ids
206
+ # @raise [PGMQ::Errors::MultiQueueInvalidError] if archives is not a Hash
205
207
  #
206
208
  # @example Archive messages from multiple queues
207
209
  # client.archive_multi({
@@ -209,7 +211,7 @@ module PGMQ
209
211
  # 'notifications' => [5]
210
212
  # })
211
213
  def archive_multi(archives)
212
- raise ArgumentError, "archives must be a hash" unless archives.is_a?(Hash)
214
+ MULTI_QUEUE_CONTRACT.validate!({ archives: archives }, Errors::MultiQueueInvalidError)
213
215
  return {} if archives.empty?
214
216
 
215
217
  # Validate all queue names
@@ -317,6 +319,7 @@ module PGMQ
317
319
  # @param updates [Hash] hash of queue_name => array of msg_ids
318
320
  # @param vt [Integer, Time] visibility timeout as seconds offset or absolute timestamp
319
321
  # @return [Hash] hash of queue_name => array of updated PGMQ::Message objects
322
+ # @raise [PGMQ::Errors::MultiQueueInvalidError] if updates is not a Hash
320
323
  #
321
324
  # @example Extend visibility timeout for messages from multiple queues
322
325
  # client.set_vt_multi({
@@ -334,7 +337,7 @@ module PGMQ
334
337
  # updates = messages.group_by(&:queue_name).transform_values { |msgs| msgs.map(&:msg_id) }
335
338
  # client.set_vt_multi(updates, vt: 120)
336
339
  def set_vt_multi(updates, vt:)
337
- raise ArgumentError, "updates must be a hash" unless updates.is_a?(Hash)
340
+ MULTI_QUEUE_CONTRACT.validate!({ updates: updates }, Errors::MultiQueueInvalidError)
338
341
  return {} if updates.empty?
339
342
 
340
343
  # Validate all queue names