eventhub-processor2 1.28.1 → 1.28.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 30f1770bc8368dade5070d56b70f7d178c4f2b9b411b6c8cedc76d5cee74906a
-  data.tar.gz: 437a059f4496b259452423572dc03149dc35dd3992dadf1cf741cf153d76ac44
+  metadata.gz: dd2f7112cce3b298538f8fa33ad0218ced28e2c86a2064b8cd6054df7e5a5718
+  data.tar.gz: 759bcd489e715c26146e82982f1c894fa3a97615be8c30334159d719cee8f4a6
 SHA512:
-  metadata.gz: e2c36a143e30f630b64756c0a26e2f3d936ef37a34be14eefac20ed105535ec9a2d7c561ec5e482e16ca0df22e484f09d14d3f5ecf202c2d472fbcc5b4588603
-  data.tar.gz: 1e51eb0ae32377c32c2fc5372be3304a276fac5d5df9920658fb804bd8c8461b25a8a9470cccbbc3719963fdfc966e8b9b7f349d7cabb1c9943ddb60ae32cb3e
+  metadata.gz: aeacaf0d8a2878add3b3b7081f8a386cff4bc996fc66f0ca46832d898cd732e71f8f2efd0009510398fc7ef9ef7fa944ec5f66892e2a5304707c79d52f222921
+  data.tar.gz: b8b0d57be7bc929b3262b6092e8cac7989c1341071c43c69e7ab1556b78902088d6dc6cc6112eaea48896d053a7d528ed0f1a594a01df845c7a765bd767684f4

data/CHANGELOG.md

@@ -1,5 +1,11 @@
 # Changelog of EventHub::Processor2
 
+# 1.28.2 / 2026-05-20
+
+* Fix `CorrelationId.current` leaking across messages on the same Bunny consumer thread. `CorrelationId.with` now always saves and restores, even when called with nil/empty. Symptom in production: deadletter messages stamped with an earlier message's `execution_id`.
+* Listener now also resets `correlation_id` after each message (extra safety on top of the fix above, in case some future code writes it outside of `CorrelationId.with`).
+* README: clarify `correlation_id` is the AMQP **property** (envelope-level), not a custom field in the headers table. Producers must set the property; a header named `correlation_id` is invisible to the gem.
+
 # 1.28.1 / 2026-05-19
 
 * Fix `LoggerProxy` double-wrapping when callers pass a Hash to a logger method. Hash inputs are now merged (not nested under `:message`), so the resulting structured event has the caller's fields at the top level - restoring the behaviour that existed before `LoggerProxy` was introduced in 1.26.0, while keeping automatic thread-local injection of `correlation_id` and `execution_id`. Caller-provided values win over thread-locals via `||=`.

data/README.md

@@ -117,6 +117,31 @@ I, [2018-02-09T15:22:35.658522 #37966]  INFO -- : Listener is starting...
 I, [2018-02-09T15:22:35.699161 #37966]  INFO -- : Listening to queue [example]
 ```
 
+## Concurrency Model
+
+Processor2 runs **one consumer thread per queue** listed in `listener_queues`. Each consumer holds a Bunny worker thread with `prefetch(1)`, so it processes its messages sequentially. Multiple queues run their consumers in parallel.
+
+| `listener_queues`             | Concurrency                                    |
+|-------------------------------|------------------------------------------------|
+| `["q1"]`                      | 1 consumer thread, sequential                  |
+| `["q1", "q2"]`                | 2 consumer threads, parallel (one per queue)   |
+| `["q1", "q1", "q1"]`          | 3 consumer threads, all on the same queue      |
+
+To increase throughput for a single queue, either list it multiple times in `listener_queues` (each entry gets its own consumer thread) or run multiple processes. The single-process / single-consumer-per-queue model is what the gem is designed for.
+
+Correlation context (`correlation_id`, `execution_id`) is held in thread-local storage, so concurrent consumer threads cannot interfere with each other - each message's outbound `correlation_id` is the one it received (or its body's fallback), regardless of what other threads are doing.
+
+### A note on `prefetch`
+
+The AMQP `prefetch` value is **fixed at 1** and not currently exposed via configuration. Raising it would require:
+
+- bumping Bunny's `channel.work_pool.size` in lockstep (raising `prefetch` alone only buffers messages in Bunny - it does not add parallelism);
+- pooling or removing `ActorPublisher`'s single-mailbox bottleneck, otherwise N parallel consumers serialize through one publisher;
+- making `handle_message` a thread-safety contract for gem users (today it is implicitly single-threaded - any `@ivar` on the processor class is safe by accident);
+- making `Statistics` and any other shared state thread-safe.
+
+These trade-offs change the gem's call-surface contract, so a configurable `prefetch` would be a minor-version feature, not a patch. For now, the supported way to scale a single queue is to list it multiple times in `listener_queues` or run multiple processes.
+
 ## Logging
 
 By default, Processor2 logs to both stdout (standard format) and a logstash file. For containerized environments (Docker, Kubernetes), use the `--console-log-only` option to output structured JSON logs to stdout only:
@@ -135,10 +160,21 @@ This outputs logs in JSON format suitable for log aggregation systems:
 
 Processor2 supports automatic propagation of correlation IDs for distributed tracing. While EventHub messages already contain an `execution_id` in the message body for tracing, the AMQP `correlation_id` provides an additional benefit: it's part of the message metadata (envelope), not the payload. This means it's available even when the JSON body is invalid and cannot be parsed - useful for error tracking and debugging malformed messages.
 
-When an incoming AMQP message includes a `correlation_id` in its metadata:
+> **Important: `correlation_id` is an AMQP `property`, not a `header`.**
+>
+> In AMQP 0.9.1, `correlation_id` is one of the 14 standard message *properties* (top-level envelope fields, alongside `message_id`, `reply_to`, `content_type`, etc.). The *headers table* is a separate, optional dictionary inside the property block for application-defined fields. They are not interchangeable.
+>
+> Producers must set `correlation_id` as a property. A custom field called `correlation_id` placed inside the headers table is **invisible to this gem** and will silently fall back to the message body's `execution_id`.
+>
+> Concretely:
+> - In Bunny: `channel.default_exchange.publish(payload, correlation_id: "...")`  ✅ property
+> - In Bunny: `... publish(payload, headers: { "correlation_id" => "..." })`  ❌ custom header, not seen
+> - In the RabbitMQ Management UI's "Publish message" form: set the value in the **Properties** section, not in the **Headers** section. When inspecting a queued message, look at "Properties → correlation_id", not at "Headers".
+
+When an incoming AMQP message includes a `correlation_id` in its metadata (as a property):
 
 1. **Automatic logging**: All log messages during message processing will include `correlation_id` as a separate field in structured JSON output
-2. **Automatic publishing**: Any messages published during processing will automatically include the same correlation_id in their AMQP headers
+2. **Automatic publishing**: Any messages published during processing will automatically include the same correlation_id as an AMQP `correlation_id` property (envelope-level, not in the headers table)
 3. **Available in args**: The correlation_id is passed to `handle_message` via `args[:correlation_id]`
 4. **Consistent execution_id**: When creating new messages, `execution_id` is automatically set to match `correlation_id`, ensuring consistent tracing across both AMQP metadata and message body
 
@@ -174,7 +210,7 @@ If no `correlation_id` is present in the AMQP metadata, the message body's `exec
 3. **Stored**: The value is stored in thread-local storage (`Thread.current`) for the duration of message processing
 4. **Passed**: The `correlation_id` is passed to `handle_message` via `args[:correlation_id]`
 5. **Logging**: The logger automatically reads from thread-local storage and includes it in JSON output
-6. **Publishing**: The publisher automatically reads from thread-local storage and adds it to outgoing AMQP message headers (can be overwritten by passing `correlation_id:` explicitly)
+6. **Publishing**: The publisher automatically reads from thread-local storage and sets it as the outgoing AMQP `correlation_id` property (can be overwritten by passing `correlation_id:` explicitly)
 7. **New messages**: When creating a new `EventHub::Message`:
    - With `correlation_id` present → `execution_id` is set to match `correlation_id`
    - Without `correlation_id` → `execution_id` is set to a new UUID (default behavior)

data/lib/eventhub/actor_listener_amqp.rb

@@ -90,6 +90,12 @@ module EventHub
                                  " acknowledged")
           ensure
             ExecutionId.clear
+            # Belt-and-suspenders: CorrelationId.with's ensure already
+            # restores the prior value, but clearing here protects any
+            # future code path that writes CorrelationId.current outside
+            # of `.with` (e.g. handle_payload's fallback was the original
+            # leak source pre-1.28.2).
+            CorrelationId.clear
           end
         end
         queue.subscribe_with(consumer, block: false)

data/lib/eventhub/correlation_id.rb

@@ -16,18 +16,20 @@ module EventHub
         Thread.current[:eventhub_correlation_id] = nil
       end
 
-      # Execute block with correlation_id set, ensures cleanup
+      # Execute block with correlation_id set, ensures cleanup.
+      #
+      # Always saves the prior value and restores it on exit, even when
+      # called with nil/empty - otherwise any value written to `current`
+      # inside the block (e.g. handle_payload's fallback to the message
+      # body's execution_id) would leak onto the consumer thread and be
+      # picked up by the next message's processing.
       def with(correlation_id)
-        if correlation_id.nil? || correlation_id.to_s.empty?
+        old_value = current
+        begin
+          self.current = correlation_id unless correlation_id.nil? || correlation_id.to_s.empty?
           yield
-        else
-          old_value = current
-          begin
-            self.current = correlation_id
-            yield
-          ensure
-            self.current = old_value
-          end
+        ensure
+          self.current = old_value
         end
       end
     end

data/lib/eventhub/version.rb

@@ -1,3 +1,3 @@
 module EventHub
-  VERSION = "1.28.1".freeze
+  VERSION = "1.28.2".freeze
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: eventhub-processor2
 version: !ruby/object:Gem::Version
-  version: 1.28.1
+  version: 1.28.2
 platform: ruby
 authors:
 - Steiner, Thomas