waterdrop 2.10.5 → 2.10.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: d61c97700e2882107b4e3309176c22fc85d06df909e3f11f0779bb33cab0ec96
4
- data.tar.gz: 1b4c0e3c7ac54d6b86905544396865934ee783ba730dc0ccc6b3bd6b954a7ce8
3
+ metadata.gz: df6f8bc25170845840699e442886688586d193d69ddb89849bfbd0a6da2c7dbd
4
+ data.tar.gz: 15cd50dfc2d196580d5afb44aef4ecab2683b7ce11f966d06041c4f5f120cbfc
5
5
  SHA512:
6
- metadata.gz: 9d509ae8aafda829f217357e6d8447905910137a4af850cd8d4bac7a35bb3af703432339c05fa45117dd8acff2a481718f18a94dd7e7ea0ba0259a86b6ae8d42
7
- data.tar.gz: 4a8c01eb9aa11c49eb810888a65a520731298ea500bdcaa2c0b78bc163ffc346cc3975f8cad06ea7e50c9c9d21357457eaf745269b35e598a9eccebaf611e7a4
6
+ metadata.gz: 6a0dd37309d89f9ffa8d3932f338edf56f059bbd620befc1c008c6375c258f515e4aec054370815fc56411ba787f887ca9bf7bc5173ec7cf6117f6db44cec7ce
7
+ data.tar.gz: 87e75e5ec48f647bc070603d49848a9d5032ab89fb411ac4e33683311b70736e287de5d6cdfebb4c1a77c96d7f09438e9c85c597faea7673031e11ff64f25bfa
data/.rubocop.yml CHANGED
@@ -12,7 +12,7 @@ inherit_gem:
12
12
 
13
13
  AllCops:
14
14
  NewCops: enable
15
- TargetRubyVersion: 3.2
15
+ TargetRubyVersion: 3.3
16
16
  Include:
17
17
  - "**/*.rb"
18
18
  - "**/*.gemspec"
data/CHANGELOG.md CHANGED
@@ -1,9 +1,14 @@
1
1
  # WaterDrop changelog
2
2
 
3
+ ## 2.10.6 (2026-10-12)
4
+ - **[EOL]** Drop Ruby 3.2 support; require Ruby 3.3+.
5
+ - [Maintenance] Document that a middleware chain run is not atomic, so an in-place step followed by a raising step is applied again on the next `#flush`.
6
+ - [Maintenance] Stop the sync delivery error spec from flaking when librdkafka rejects the produce to the invalid topic inline instead of through the delivery report.
7
+
3
8
  ## 2.10.5 (2026-09-23)
4
- - [Fix] Stop `#flush` from running middleware twice on messages re-buffered after a failed flush (regression of #474 reintroduced by the failure/requeue path), which double-encrypted/compressed/serialized payloads and duplicated headers on retry.
5
- - [Fix] Re-check the producer liveness under `@buffer_mutex` in `#buffer`/`#buffer_many`, so a producer closed between the liveness check and the append raises `ProducerClosedError` instead of accepting the message into a closed producer's buffer and losing it silently.
6
- - [Fix] Restore both buffers when `#flush` fails before anything is dispatched (a middleware step raising, for example) instead of discarding the whole batch. Only `ProduceManyError` and `MessageInvalidError` were covered before.
9
+ - [Fix] Stop `#flush` from running middleware twice on messages re-buffered after a failed flush, which double-processed payloads and duplicated headers on retry (regression of #474).
10
+ - [Fix] Raise `ProducerClosedError` instead of silently losing a message buffered while the producer is closing.
11
+ - [Fix] Restore both buffers when `#flush` fails before anything is dispatched, instead of discarding the whole batch.
7
12
 
8
13
  ## 2.10.4 (2026-08-26)
9
14
  - [Fix] Avoid a `FrozenError` on `Producer#close` when the producer is configured with a frozen string id (for example a frozen string literal, `config.id = "rspec"`).
@@ -12,10 +17,10 @@
12
17
  - [Maintenance] Stop the `#partition_count when topic does not exist` spec from flaking on slow CI runners by waiting for authoritative broker metadata before asserting the count.
13
18
 
14
19
  ## 2.10.3 (2026-07-15)
15
- - [Feature] Add `wait_timeout_on_transaction_abort` (default `0`, disabled) - an opt-in mitigation for [librdkafka#4849](https://github.com/confluentinc/librdkafka/issues/4849) that waits (up to the given ms) for the first delivery to be acknowledged before aborting, avoiding a fatal `INVALID_TXN_STATE`. Off by default because it changes abort semantics: the awaited message is written to the log (aborted) instead of being purged.
20
+ - [Feature] Add `wait_timeout_on_transaction_abort` (disabled by default) to avoid a fatal `INVALID_TXN_STATE` on abort ([librdkafka#4849](https://github.com/confluentinc/librdkafka/issues/4849)). When enabled, the awaited message is delivered as aborted instead of purged.
16
21
 
17
22
  ## 2.10.2 (2026-06-15)
18
- - [Feature] Expose `Producer#current_variant` as a public method returning the variant active for the current dispatch on the current fiber, so middleware and instrumentation listeners can read the effective per-dispatch settings (`topic_config`, `max_wait_timeout`, `default?`).
23
+ - [Feature] Expose `Producer#current_variant`, returning the variant active for the current dispatch, for use in middleware and instrumentation listeners.
19
24
  - [Enhancement] Stop allocating one interpolated string per message in `LoggerListener` batch produce handlers.
20
25
  - [Enhancement] Use `Array#concat` in `Producer#buffer_many` instead of appending messages one by one.
21
26
  - [Enhancement] Skip building the `message.acknowledged` instrumentation payload in the delivery callback when nothing is subscribed to that event.
@@ -24,22 +29,22 @@
24
29
  - [Enhancement] Cache the variant validation contract in a constant instead of instantiating a new `Contracts::Variant` on every `Producer#with` / `Producer#variant` call.
25
30
  - [Enhancement] Cache the tombstone validation contract in a constant instead of instantiating a new `Contracts::Tombstone` per tombstone message.
26
31
  - [Enhancement] Replace explicit `Warning[:performance]` opt-in with a dynamic approach using `Warning.categories` to enable all stable opt-in warning categories in the test suite.
27
- - [Fix] Prevent a deadlock between a transactional single-message dispatch and `#close` caused by an inverted lock order; transactional dispatches now take `@transaction_mutex` before the operation is counted.
28
- - [Fix] Prevent a deadlock (`ThreadError: deadlock; recursive locking`) when closing an idempotent producer (with `reload_on_idempotent_fatal_error` enabled) whose final buffer flush surfaces a fatal librdkafka error; the idempotent reload is now skipped on the closing path.
32
+ - [Fix] Prevent a deadlock between a transactional single-message dispatch and `#close`.
33
+ - [Fix] Prevent a deadlock when closing an idempotent producer with `reload_on_idempotent_fatal_error` enabled whose final flush hits a fatal error.
29
34
  - [Fix] Make concurrent idempotent fatal-error reload thread-safe so a second thread's reload can no longer raise `NoMethodError` after the first reset `@client`.
30
35
  - [Fix] Stop `#flush_async` / `#flush_sync` from silently dropping valid buffered messages when the dispatch fails; unsent messages are now re-buffered so they can be retried instead of being lost.
31
36
  - [Fix] Make `Producer#close` fork-safe so the GC finalizer inherited by a forked child can no longer flush and close the parent's client.
32
- - [Fix] Guard the internal buffer appends in `Producer#buffer` and `Producer#buffer_many` with `@buffer_mutex` so a concurrent flush/purge/close swap can no longer drop a message into an orphaned array.
33
- - [Fix] Stop a nested same-producer variant call from clobbering the outer variant inside a variant `transaction` block; the wrapper now saves and restores the previous fiber-local entry instead of deleting it.
34
- - [Fix] Stop `ConnectionPool#shutdown` and `#reload` from silently dropping in-flight messages. They now close producers gracefully by default (`#reload` always; `#shutdown` unless called with the new `force: true`); pass `pool.shutdown(force: true)` to keep the old force-and-purge behavior.
35
- - [Fix] Close a race in the FD poller where a producer registered while the last one was being torn down could be left permanently unpolled; the poller now decides to stop and clears its thread reference in a single mutex section.
37
+ - [Fix] Prevent a concurrent flush, purge or close from dropping a message buffered with `#buffer` or `#buffer_many`.
38
+ - [Fix] Stop a nested variant call on the same producer from overriding the outer variant inside a variant `transaction` block.
39
+ - [Fix] Stop `ConnectionPool#shutdown` and `#reload` from dropping in-flight messages by closing producers gracefully. Use `pool.shutdown(force: true)` for the previous force-and-purge behavior.
40
+ - [Fix] Fix a race that could leave a newly registered producer permanently unpolled by the FD poller.
36
41
 
37
42
  ## 2.10.1 (2026-05-25)
38
- - [Fix] Prevent `Producer#close` from raising `ThreadError: can't be called from trap context` when invoked from a Ruby signal trap context (e.g. Puma's `after_stopped` DSL hook in single mode). `close` now detects this case and delegates to a background thread, joining it so the caller blocks until the producer is fully closed (#866).
43
+ - [Fix] Prevent `Producer#close` from raising `ThreadError: can't be called from trap context` when called from a signal trap (e.g. Puma's `after_stopped` hook in single mode). It now closes from a background thread and waits for it (#866).
39
44
 
40
45
  ## 2.10.0 (2026-05-07)
41
- - [Fix] Clean up native rdkafka client, global instrumentation callbacks, and poller registration when `init_transactions` fails during producer client construction, so failed attempts no longer leak native threads, file descriptors, and callback registry entries.
42
- - **[Breaking]** Skip emitting librdkafka statistics when nothing is subscribed to `statistics.emitted` at the time the underlying rdkafka client is constructed (`statistics.interval.ms` is forced to `0`), saving substantial allocations in the hot path. To use statistics, subscribe a listener to `statistics.emitted` BEFORE the first producer use.
46
+ - [Fix] Clean up the native client, instrumentation callbacks and poller registration when `init_transactions` fails during producer construction, so failed attempts no longer leak threads and file descriptors.
47
+ - **[Breaking]** Skip librdkafka statistics when nothing subscribes to `statistics.emitted` before the rdkafka client is built, to save allocations. To use statistics, subscribe a listener BEFORE the first producer use.
43
48
  - **[Breaking]** Raise `WaterDrop::Errors::StatisticsNotEnabledError` when attempting to subscribe to `statistics.emitted` on a monitor where librdkafka statistics have been disabled at client build time, replacing the previous "silent nothing" failure mode.
44
49
  - [Feature] Add tombstone API (`#tombstone_sync`, `#tombstone_async`, `#tombstone_many_sync`, `#tombstone_many_async`) for producing tombstone records (nil-payload messages) with required key and partition validation. Works with variants.
45
50
  - [Fix] Add `ensure_same_process!` to `Poller#unregister` for fork safety, so a child that inherited a pre-fork producer no longer deadlocks on `producer.close`.
@@ -48,14 +53,14 @@
48
53
  - [Fix] Use `delete` in the variant ensure block to avoid leaving stale nil entries in `Fiber.current.waterdrop_clients` and prevent memory leaks in long-running processes (#836).
49
54
  - [Fix] Exclude test files, `.github/`, and `log/` directories from gem releases to reduce package size.
50
55
  - **[Breaking]** Switch default polling mode from `:thread` to `:fd`. If you experience any issues, you can revert to the previous behavior by setting `config.polling.mode = :thread`. The `:thread` mode will be deprecated in 2.10 and removed in 2.11.
51
- - **[Breaking]** Statistics decorator now only decorates keys used by the built-in Datadog metrics listener (`tx`, `txretries`, `txerrs`, `rxerrs`) and skips unused subtrees, greatly reducing decoration cost on large clusters. Users who rely on other `_d` or `_fd` keys in custom instrumentation should provide a custom decorator via `config.statistics_decorator`.
56
+ - **[Breaking]** Decorate only the statistics keys used by the built-in Datadog listener (`tx`, `txretries`, `txerrs`, `rxerrs`). If you use other `_d` or `_fd` keys, set a custom `config.statistics_decorator`.
52
57
  - [Feature] Add `config.statistics_decorator` setting to allow full control over the `StatisticsDecorator` instance used for statistics decoration. Users can provide a custom decorator with different `only_keys` and `excluded_keys` to match their instrumentation needs.
53
58
  - [Change] Upscale default timeout values 3x closer to librdkafka defaults to prevent intermediate timeouts during node recovery (`message.timeout.ms`: 50s → 150s, `transaction.timeout.ms`: 55s → 165s, `max_wait_timeout`: 60s → 180s).
54
59
 
55
60
  **Upgrade Notes**: https://karafka.io/docs/Upgrades-WaterDrop-2.9/
56
61
 
57
62
  ## 2.8.16 (2026-02-25)
58
- - [Feature] Add FD-based polling mode (`config.polling.mode = :fd`) as an alternative to the default thread-based polling. FD mode uses a single Ruby thread with IO.select for efficient multiplexing, providing 39-54% higher throughput, lower memory usage, and fewer threads compared to the default thread mode.
63
+ - [Feature] Add FD-based polling (`config.polling.mode = :fd`): a single Ruby thread with `IO.select` instead of the default thread mode, with 39-54% higher throughput, lower memory usage and fewer threads.
59
64
  - [Feature] Add `config.polling.fd.max_time` setting (default: 100ms) to control maximum polling time per producer per cycle. This enables per-producer priority differentiation.
60
65
  - [Feature] Add `Producer#queue_size` (alias: `#queue_length`) to return the count of messages pending in the librdkafka queue. This counts messages that have been dispatched to librdkafka but not yet transmitted to the Kafka broker.
61
66
  - [Enhancement] Add `poll_nb` and `poll_drain` methods to rdkafka Producer for efficient non-blocking polling without GVL release overhead.
@@ -82,11 +87,7 @@
82
87
 
83
88
  ## 2.8.12 (2025-10-10)
84
89
  - [Enhancement] Introduce `reload_on_idempotent_fatal_error` to automatically reload librdkafka producer after fatal errors on idempotent (non-transactional) producers.
85
- - [Enhancement] Add configurable backoff and retry limits for fatal error recovery to prevent infinite reload loops:
86
- - `wait_backoff_on_idempotent_fatal_error` (default: 5000ms) - backoff before retrying after idempotent fatal error reload
87
- - `max_attempts_on_idempotent_fatal_error` (default: 5) - max reload attempts for idempotent fatal errors
88
- - `wait_backoff_on_transaction_fatal_error` (default: 1000ms) - backoff after transactional fatal error reload
89
- - `max_attempts_on_transaction_fatal_error` (default: 10) - max reload attempts for transactional fatal errors
90
+ - [Enhancement] Add backoff and retry limits for fatal error reloads to prevent infinite reload loops (defaults in brackets): `wait_backoff_on_idempotent_fatal_error` (5000ms), `max_attempts_on_idempotent_fatal_error` (5), `wait_backoff_on_transaction_fatal_error` (1000ms) and `max_attempts_on_transaction_fatal_error` (10).
90
91
  - [Enhancement] Ensure `error.occurred` is instrumented before idempotent fatal error reload for visibility.
91
92
  - [Enhancement] Automatically reset fatal error reload attempts counter on successful produce/transaction to allow recovery.
92
93
  - [Refactor] Extract idempotence-related logic into separate `WaterDrop::Producer::Idempotence` module.
@@ -112,7 +113,7 @@
112
113
 
113
114
  ## 2.8.8 (2025-09-23)
114
115
  - [Feature] Add `WaterDrop::ConnectionPool` for efficient connection pooling using the proven `connection_pool` gem.
115
- - [Feature] Add `WaterDrop.instrumentation` class-level instrumentation for producer lifecycle events. This allows external libraries to subscribe to `producer.created` and `producer.configured` events without needing producer instance references, enabling middleware injection and configuration by libraries like Datadog tracing.
116
+ - [Feature] Add class-level `WaterDrop.instrumentation` with `producer.created` and `producer.configured` events, so libraries (e.g. Datadog tracing) can configure producers without instance references.
116
117
  - **[EOL]** Remove Ruby `3.1` specs according to the EOL schedule.
117
118
 
118
119
  ## 2.8.7 (2025-09-02)
@@ -201,7 +202,7 @@ This release contains **BREAKING** changes. Make sure to read and apply upgrade
201
202
  - **[Breaking]** Remove no longer needed `wait_timeout` configuration option.
202
203
  - **[Breaking]** Do **not** validate or morph (via middleware) messages added to the buffer prior to `flush_sync` or `flush_async`.
203
204
  - [Enhancement] Provide `WaterDrop::Producer#transaction?` that returns only when producer has an active transaction running.
204
- - [Enhancement] Introduce `instrument_on_wait_queue_full` flag (defaults to `true`) to be able to configure whether non critical (retryable) queue full errors should be instrumented in the error pipeline. Useful when building high-performance pipes with WaterDrop queue retry backoff as a throttler.
205
+ - [Enhancement] Introduce `instrument_on_wait_queue_full` (defaults to `true`) to control whether retryable queue full errors are instrumented in the error pipeline.
205
206
  - [Enhancement] Protect critical `rdkafka` thread executable code sections.
206
207
  - [Enhancement] Treat the queue size as a gauge rather than a cumulative stat (isturdy).
207
208
  - [Fix] Fix a case where purge on non-initialized client would crash.
@@ -225,7 +226,7 @@ This release contains **BREAKING** changes. Make sure to read and apply upgrade
225
226
  - [Fix] Fix a case where buffered test client would not accumulate messages on failed transactions
226
227
 
227
228
  ## 2.6.10 (2023-10-24)
228
- - [Improvement] Introduce `message.purged` event to indicate that a message that was not delivered to Kafka was purged. This most of the time refers to messages that were part of a transaction and were not yet dispatched to Kafka. It always means, that given message was not delivered but in case of transactions it is expected. In case of non-transactional it usually means `#purge` usage or exceeding `message.timeout.ms` so `librdkafka` removes this message from its internal queue. Non-transactional producers do **not** use this and pipe purges to `error.occurred`.
229
+ - [Improvement] Introduce the `message.purged` event for a transactional message purged before delivery to Kafka. Non-transactional producers report purges via `error.occurred` instead.
229
230
  - [Fix] Fix a case where `message.acknowledged` would not have `caller` key.
230
231
  - [Fix] Fix a bug where critical errors (like `IRB::Abort`) would not abort the ongoing transaction.
231
232
 
@@ -279,7 +280,7 @@ This release contains **BREAKING** changes. Make sure to read and apply upgrade
279
280
  - [Improvement] Make `#produce` method private to avoid confusion and make sure it is not used directly (it is not part of the official API).
280
281
  - [Change] Change `wait_on_queue_full` from `false` to `true` as a default.
281
282
  - [Change] Rename `wait_on_queue_full_timeout` to `wait_backoff_on_queue_full` to match what it actually does.
282
- - [Enhancement] Introduce `wait_timeout_on_queue_full` with proper meaning. That is, this represents time after which despite backoff the error will be raised. This should allow to raise an error in case the backoff attempts were insufficient. This prevents from a case, where upon never deliverable messages we would end up with an infinite loop.
283
+ - [Enhancement] Introduce `wait_timeout_on_queue_full`: the time after which the error is raised despite backoff, so never-deliverable messages no longer cause an infinite loop.
283
284
  - [Fix] Provide `type` for queue full errors that references the appropriate public API method correctly.
284
285
 
285
286
  ## 2.5.3 (2023-05-26)
data/Gemfile CHANGED
@@ -4,7 +4,6 @@ source "https://rubygems.org"
4
4
 
5
5
  gemspec
6
6
 
7
- # Relaxed from 2.7 because we support Ruby 3.1
8
7
  gem "zeitwerk", "~> 2.8.0"
9
8
 
10
9
  group :development do
data/Gemfile.lock CHANGED
@@ -1,7 +1,7 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- waterdrop (2.10.5)
4
+ waterdrop (2.10.6)
5
5
  karafka-core (>= 2.5.12, < 3.0.0)
6
6
  karafka-rdkafka (>= 0.24.0)
7
7
  zeitwerk (~> 2.3)
@@ -14,12 +14,12 @@ GEM
14
14
  connection_pool (3.0.2)
15
15
  drb (2.2.3)
16
16
  ffi (1.17.4)
17
- io-console (0.9.2)
18
- json (2.21.2)
19
- karafka-core (2.6.3)
17
+ io-console (0.9.4)
18
+ json (3.0.2)
19
+ karafka-core (2.6.4)
20
20
  karafka-rdkafka (>= 0.20.0)
21
21
  logger (>= 1.6.0)
22
- karafka-rdkafka (0.28.0)
22
+ karafka-rdkafka (0.30.2)
23
23
  ffi (~> 1.17.1)
24
24
  json (> 2.0)
25
25
  logger
@@ -38,7 +38,7 @@ GEM
38
38
  reline (0.7.0)
39
39
  io-console (~> 0.5)
40
40
  ruby2_keywords (0.0.5)
41
- simplecov (1.3.0)
41
+ simplecov (1.3.2)
42
42
  warning (1.6.0)
43
43
  zeitwerk (2.8.3)
44
44
 
@@ -61,10 +61,10 @@ CHECKSUMS
61
61
  connection_pool (3.0.2) sha256=33fff5ba71a12d2aa26cb72b1db8bba2a1a01823559fb01d29eb74c286e62e0a
62
62
  drb (2.2.3) sha256=0b00d6fdb50995fe4a45dea13663493c841112e4068656854646f418fda13373
63
63
  ffi (1.17.4) sha256=bcd1642e06f0d16fc9e09ac6d49c3a7298b9789bcb58127302f934e437d60acf
64
- io-console (0.9.2) sha256=efa74f891dd03c0939a931dfc6e74c2813d904763d456ea9762b0525e748db08
65
- json (2.21.2) sha256=1f1d3b7cf2b3ba1a69beca0bb6db13d5438b80bff3cd54cdaaa620b9b07c1c6a
66
- karafka-core (2.6.3) sha256=83a783895d8f69e5fd4b7e77617f6a43c129bbb905c93e1babb9ff299470ba4a
67
- karafka-rdkafka (0.28.0) sha256=fd845140c42341c5892215cac1e8adacda53b0461688f2e16379d82cb33eb006
64
+ io-console (0.9.4) sha256=a403304f32928ee85e56d457b4612ca43f2703b561a3258844992bd24085fbb1
65
+ json (3.0.2) sha256=8e6d7e7b11384c21230430cef90b71f14849a34a1f4452796670f7c981bd19df
66
+ karafka-core (2.6.4) sha256=e08166ca7ef12fadf0500f2a3aba63b5ba918a98533c3feaac9f43c90f159363
67
+ karafka-rdkafka (0.30.2) sha256=11d565109ab9ebc6216020d6097bd9adbc39a9e599fb2a8cc5af93965b9008b7
68
68
  logger (1.7.0) sha256=196edec7cc44b66cfb40f9755ce11b392f21f7967696af15d274dde7edff0203
69
69
  mini_portile2 (2.8.9) sha256=0cd7c7f824e010c072e33f68bc02d85a00aeb6fce05bb4819c03dfd3c140c289
70
70
  minitest (6.0.6) sha256=153ea36d1d987a62942382b61075745042a2b3123b1cd48f4c3675af9cc7d6f1
@@ -74,9 +74,9 @@ CHECKSUMS
74
74
  rake (13.4.2) sha256=cb825b2bd5f1f8e91ca37bddb4b9aaf345551b4731da62949be002fa89283701
75
75
  reline (0.7.0) sha256=5b012d8e55dbf9d450f12bde2cf7d15ff546ae80b3f8f3b30e570d431815583d
76
76
  ruby2_keywords (0.0.5) sha256=ffd13740c573b7301cf7a2e61fc857b2a8e3d3aff32545d6f8300d8bae10e3ef
77
- simplecov (1.3.0) sha256=d9886307863c1ead47657dcbed869bfffd82318808b09014d8f0ff77dcfe7492
77
+ simplecov (1.3.2) sha256=2a22ec1759f1b8db60881a2e038758956fc69ef93ed669be0bde0678de6d0e85
78
78
  warning (1.6.0) sha256=a49cdfae19fb77d19afff2efbe45f8ab759e9cd25b4e4ce2c79dbaf46bdb6c9e
79
- waterdrop (2.10.5)
79
+ waterdrop (2.10.6)
80
80
  zeitwerk (2.8.3) sha256=2c85125a8467ce069e20123d1e709a08955c9d29c118c25b46b7b7fafdbb92e5
81
81
 
82
82
  BUNDLED WITH
data/README.md CHANGED
@@ -16,7 +16,7 @@ It:
16
16
  - Supports multiple delivery policies
17
17
  - Supports per-topic configuration alterations (variants)
18
18
  - Works with [async](https://github.com/socketry/async) gems ecosystem
19
- - Works with Kafka `1.0+` and Ruby `3.2+`
19
+ - Works with Kafka `1.0+` and Ruby `3.3+`
20
20
  - Works with and without Karafka
21
21
 
22
22
  ## Documentation
@@ -17,6 +17,14 @@ module WaterDrop
17
17
  # into a new object.
18
18
  # @note You need to decide yourself whether you don't use the message hash data anywhere else
19
19
  # and you want to save on memory by modifying it in place or do you want to do a deep copy
20
+ # @note Steps run sequentially and a chain run is not atomic. If an in-place step mutates the
21
+ # message and a later step raises, the message is left partially transformed. During a
22
+ # buffer flush that message is re-buffered as not yet processed, so the next flush runs the
23
+ # whole chain over it again and the earlier in-place steps are applied twice (for example
24
+ # double encryption or duplicated headers). Making the run atomic would mean copying every
25
+ # message before its chain runs, which defeats the memory saving in-place mode is for, so
26
+ # this is a deliberate trade-off. If an in-place step can be followed by a step that may
27
+ # raise, make the in-place step idempotent or have it return a copy instead.
20
28
  def run(message)
21
29
  return message if @count.zero?
22
30
 
@@ -115,6 +115,12 @@ module WaterDrop
115
115
  # ones whose chain completed are kept as returned and are not transformed again, while the
116
116
  # failing one and those after it stay on the middleware path. The failing one is not
117
117
  # requeued because that would dispatch it without the rest of its chain.
118
+ #
119
+ # Known edge case (#46840): the failing message is re-buffered in whatever state its chain
120
+ # left it. Copy-style steps leave it untouched, but in-place steps that ran before the
121
+ # raising one leave it partially transformed, and the next flush runs its full chain again.
122
+ # We accept this rather than snapshotting every message before its run, which would cost a
123
+ # copy per message on the success path as well. See Middleware#run.
118
124
  processed = []
119
125
 
120
126
  begin
@@ -2,5 +2,5 @@
2
2
 
3
3
  module WaterDrop
4
4
  # Current WaterDrop version
5
- VERSION = "2.10.5"
5
+ VERSION = "2.10.6"
6
6
  end
data/package-lock.json CHANGED
@@ -103,9 +103,9 @@
103
103
  }
104
104
  },
105
105
  "node_modules/fastq": {
106
- "version": "1.20.1",
107
- "resolved": "https://registry.npmjs.org/fastq/-/fastq-1.20.1.tgz",
108
- "integrity": "sha512-GGToxJ/w1x32s/D2EKND7kTil4n8OVk/9mycTc4VDza13lOvpUZTGX3mFSCtV9ksdGBVzvsyAVLM6mHFThxXxw==",
106
+ "version": "1.20.3",
107
+ "resolved": "https://registry.npmjs.org/fastq/-/fastq-1.20.3.tgz",
108
+ "integrity": "sha512-XKv5nnLs6nLF71NgiKJLIZFLkPyIEuOselLG7ujZnGrRfQK8HpvY+WqKhAJUAdLomwVHErVS4LfxFlPq0/FTAw==",
109
109
  "dev": true,
110
110
  "license": "ISC",
111
111
  "dependencies": {
@@ -286,9 +286,9 @@
286
286
  }
287
287
  },
288
288
  "node_modules/smol-toml": {
289
- "version": "1.8.0",
290
- "resolved": "https://registry.npmjs.org/smol-toml/-/smol-toml-1.8.0.tgz",
291
- "integrity": "sha512-kCZr2V3ch9i00x8zXRhjUNVcjG9ijES5dDudkXvUVCT5QlJNQWElSJdZqyPemffHoLNUYwOcou0Fy+ojN0uHSQ==",
289
+ "version": "1.9.0",
290
+ "resolved": "https://registry.npmjs.org/smol-toml/-/smol-toml-1.9.0.tgz",
291
+ "integrity": "sha512-hpd+HLON7HdZXqYchMM/+LaTTbdK0AU3NngIJ4KVyWbY9bfQqdL9cD+4yf6dUoU2Ap4VsU0JkQi6FxAI1B2mXQ==",
292
292
  "dev": true,
293
293
  "license": "BSD-3-Clause",
294
294
  "engines": {
@@ -312,9 +312,9 @@
312
312
  }
313
313
  },
314
314
  "node_modules/yaml": {
315
- "version": "2.9.0",
316
- "resolved": "https://registry.npmjs.org/yaml/-/yaml-2.9.0.tgz",
317
- "integrity": "sha512-2AvhNX3mb8zd6Zy7INTtSpl1F15HW6Wnqj0srWlkKLcpYl/gMIMJiyuGq2KeI2YFxUPjdlB+3Lc10seMLtL4cA==",
315
+ "version": "2.9.1",
316
+ "resolved": "https://registry.npmjs.org/yaml/-/yaml-2.9.1.tgz",
317
+ "integrity": "sha512-3NxN8+78OdzbT7C/WjGsyfPAtJaN3FNDsWxv7Y7mcDsT/oOmgW8BpyQQFFBnvZE3j9Y2Sdz1ULFLezL7Eb2yFw==",
318
318
  "dev": true,
319
319
  "license": "ISC",
320
320
  "bin": {
data/renovate.json CHANGED
@@ -1,4 +1,7 @@
1
1
  {
2
+ "bundler": {
3
+ "managerFilePatterns": ["/(^|/)Gemfile\\.lint$/"]
4
+ },
2
5
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
3
6
  "extends": [
4
7
  "config:recommended"
data/waterdrop.gemspec CHANGED
@@ -20,7 +20,7 @@ Gem::Specification.new do |spec|
20
20
  spec.add_dependency "karafka-rdkafka", ">= 0.24.0"
21
21
  spec.add_dependency "zeitwerk", "~> 2.3"
22
22
 
23
- spec.required_ruby_version = ">= 3.2.0"
23
+ spec.required_ruby_version = ">= 3.3.0"
24
24
 
25
25
  spec.files = `git ls-files -z`.split("\x0").reject { |f| f.match?(%r{^(spec|test|\.github|log)/}) }
26
26
  spec.executables = []
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: waterdrop
3
3
  version: !ruby/object:Gem::Version
4
- version: 2.10.5
4
+ version: 2.10.6
5
5
  platform: ruby
6
6
  authors:
7
7
  - Maciej Mensfeld
@@ -153,7 +153,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
153
153
  requirements:
154
154
  - - ">="
155
155
  - !ruby/object:Gem::Version
156
- version: 3.2.0
156
+ version: 3.3.0
157
157
  required_rubygems_version: !ruby/object:Gem::Requirement
158
158
  requirements:
159
159
  - - ">="