waterdrop 2.7.4 → 2.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 7829f5c177154b491525c3895e2f12c27c6d58b8e43d47130caf6ee44dc5de2f
4
- data.tar.gz: bb92b727e921030a5933735f8f1e0e01727fb8e91bc607b1c0adb74031b4881f
3
+ metadata.gz: 96435533f730edae9dcf7bed5426641a33715e87b55c804168fa6a02465f1431
4
+ data.tar.gz: 3737e88ec3ffaa407b1cdaa523cf1d24a25aa85c4d90989a249381dbfb7a80c2
5
5
  SHA512:
6
- metadata.gz: c7ed272bc46e9f7ea1e398c66588c730f1ca0f7427e7f215d4c89285498e6ccefc1c4af709a1c17f29382e7cc75ebdee403aef865661b8fefdacd164f95acf25
7
- data.tar.gz: 99d9dcd114f67d272e0d2650ee2bd9aa0b480e9d850fb9f0f32813e02a90603b435b021a27faab09a411f3dea438c9307b14bd12e349289bf977c1b984805604
6
+ metadata.gz: 1b83e8505ca1aa9be5c137c06af1b98e10bb8f39af93e77aefa42302bf8dc2bd95f1442eb2569d5f032e80ac0374b8013e774d1f36494d23e20af0766094e2b3
7
+ data.tar.gz: 198ea59a4f7ca6a63f9816e7026c97152aaed5e6154142c47e6e96c16696a2a3d4c60cd56bf72b407b092682e9d15c08f1adfa552eb5bdc17a308f77f7424dfc
checksums.yaml.gz.sig CHANGED
Binary file
@@ -22,7 +22,6 @@ jobs:
22
22
  - '3.3'
23
23
  - '3.2'
24
24
  - '3.1'
25
- - '3.0'
26
25
  include:
27
26
  - ruby: '3.3'
28
27
  coverage: 'true'
data/.ruby-version CHANGED
@@ -1 +1 @@
1
- 3.3.3
1
+ 3.3.5
data/CHANGELOG.md CHANGED
@@ -1,5 +1,102 @@
1
1
  # WaterDrop changelog
2
2
 
3
+ ## 2.8.0 (2024-09-16)
4
+
5
+ This release contains **BREAKING** changes. Make sure to read and apply upgrade notes.
6
+
7
+ - **[Breaking]** Require Ruby `3.1+`.
8
+ - **[Breaking]** Remove ability to abort transactions using `throw(:abort)`. Please use `raise WaterDrop::Errors::AbortTransaction`.
9
+ - **[Breaking]** Disallow (similar to ActiveRecord) exiting transactions with `return`, `break` or `throw`.
10
+ - **[Breaking]** License changed from MIT to LGPL with an additional commercial option. Note: there is no commercial code in this repository. The commercial license is available for companies unable to use LGPL-licensed software for legal reasons.
11
+ - [Enhancement] Make variants fiber safe.
12
+ - [Enhancement] In transactional mode do not return any `dispatched` messages as none will be dispatched due to rollback.
13
+ - [Enhancement] Align the `LoggerListener` async messages to reflect, that messages are delegated to the internal queue and not dispatched.
14
+ - [Fix] Ensure, that `:dispatched` key for `#produce_many_sync` always contains delivery handles (final) and not delivery reports.
15
+
16
+ ### Upgrade Notes
17
+
18
+ **PLEASE MAKE SURE TO READ AND APPLY THEM!**
19
+
20
+ #### `throw(:abort)` No Longer Allowed To Abort Transactions
21
+
22
+ Replace:
23
+
24
+ ```ruby
25
+ producer.transaction do
26
+ messages.each do |message|
27
+ # Pipe all events
28
+ producer.produce_async(topic: 'events', payload: message.raw_payload)
29
+ end
30
+
31
+ # And abort if more events are no longer needed
32
+ throw(:abort) if KnowledgeBase.more_events_needed?
33
+ end
34
+ ```
35
+
36
+ With:
37
+
38
+ ```ruby
39
+ producer.transaction do
40
+ messages.each do |message|
41
+ # Pipe all events
42
+ producer.produce_async(topic: 'events', payload: message.raw_payload)
43
+ end
44
+
45
+ # And abort if more events are no longer needed
46
+ raise(WaterDrop::AbortTransaction) if KnowledgeBase.more_events_needed?
47
+ end
48
+ ```
49
+
50
+ #### `return`, `break` and `throw` No Longer Allowed Inside Transaction Block
51
+
52
+ Previously, transactions would abort if you exited early using `return`, `break`, or `throw`. This could create unexpected behavior, where users might not notice the rollback or have different intentions. For example, the following would trigger a rollback:
53
+
54
+ ```ruby
55
+ MAX = 10
56
+
57
+ def process(messages)
58
+ count = 0
59
+
60
+ producer.transaction do
61
+ messages.each do |message|
62
+ count += 1
63
+
64
+ producer.produce_async(topic: 'events', payload: message.raw_payload)
65
+
66
+ # This would trigger a rollback.
67
+ return if count >= MAX
68
+ end
69
+ end
70
+ end
71
+ ```
72
+
73
+ This is a source of errors, hence such exits are no longer allowed. You can implement similar flow control inside of your methods that are wrapped in a WaterDrop transaction:
74
+
75
+ ```ruby
76
+ MAX = 10
77
+
78
+ def process(messages)
79
+ producer.transaction do
80
+ # Early return from this method will not affect the transaction.
81
+ # It will be committed
82
+ insert_with_limit(messages)
83
+ end
84
+ end
85
+
86
+ def insert_with_limit(messages)
87
+ count = 0
88
+
89
+ messages.each do |message|
90
+ count += 1
91
+
92
+ producer.produce_async(topic: 'events', payload: message.raw_payload)
93
+
94
+ # This would trigger a rollback.
95
+ return if count >= MAX
96
+ end
97
+ end
98
+ ```
99
+
3
100
  ## 2.7.4 (2024-07-04)
4
101
  - [Maintenance] Alias `WaterDrop::Errors::AbortTransaction` with `WaterDrop::AbortTransaction`.
5
102
  - [Maintenance] Lower the precision reporting to 100 microseconds in the logger listener.
data/Gemfile.lock CHANGED
@@ -1,70 +1,72 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- waterdrop (2.7.4)
4
+ waterdrop (2.8.0)
5
5
  karafka-core (>= 2.4.3, < 3.0.0)
6
- karafka-rdkafka (>= 0.15.1)
6
+ karafka-rdkafka (>= 0.17.5)
7
7
  zeitwerk (~> 2.3)
8
8
 
9
9
  GEM
10
10
  remote: https://rubygems.org/
11
11
  specs:
12
- activesupport (7.1.3.4)
12
+ activesupport (7.2.1)
13
13
  base64
14
14
  bigdecimal
15
- concurrent-ruby (~> 1.0, >= 1.0.2)
15
+ concurrent-ruby (~> 1.0, >= 1.3.1)
16
16
  connection_pool (>= 2.2.5)
17
17
  drb
18
18
  i18n (>= 1.6, < 2)
19
+ logger (>= 1.4.2)
19
20
  minitest (>= 5.1)
20
- mutex_m
21
- tzinfo (~> 2.0)
21
+ securerandom (>= 0.3)
22
+ tzinfo (~> 2.0, >= 2.0.5)
22
23
  base64 (0.2.0)
23
24
  bigdecimal (3.1.8)
24
25
  byebug (11.1.3)
25
- concurrent-ruby (1.3.3)
26
+ concurrent-ruby (1.3.4)
26
27
  connection_pool (2.4.1)
27
28
  diff-lcs (1.5.1)
28
- docile (1.4.0)
29
+ docile (1.4.1)
29
30
  drb (2.2.1)
30
- factory_bot (6.4.6)
31
+ factory_bot (6.5.0)
31
32
  activesupport (>= 5.0.0)
32
33
  ffi (1.17.0)
33
34
  i18n (1.14.5)
34
35
  concurrent-ruby (~> 1.0)
35
- karafka-core (2.4.3)
36
- karafka-rdkafka (>= 0.15.0, < 0.17.0)
37
- karafka-rdkafka (0.16.0)
36
+ karafka-core (2.4.4)
37
+ karafka-rdkafka (>= 0.15.0, < 0.18.0)
38
+ karafka-rdkafka (0.17.6)
38
39
  ffi (~> 1.15)
39
40
  mini_portile2 (~> 2.6)
40
41
  rake (> 12)
42
+ logger (1.6.1)
41
43
  mini_portile2 (2.8.7)
42
- minitest (5.24.0)
43
- mutex_m (0.2.0)
44
+ minitest (5.25.1)
44
45
  ostruct (0.6.0)
45
46
  rake (13.2.1)
46
47
  rspec (3.13.0)
47
48
  rspec-core (~> 3.13.0)
48
49
  rspec-expectations (~> 3.13.0)
49
50
  rspec-mocks (~> 3.13.0)
50
- rspec-core (3.13.0)
51
+ rspec-core (3.13.1)
51
52
  rspec-support (~> 3.13.0)
52
- rspec-expectations (3.13.1)
53
+ rspec-expectations (3.13.3)
53
54
  diff-lcs (>= 1.2.0, < 2.0)
54
55
  rspec-support (~> 3.13.0)
55
56
  rspec-mocks (3.13.1)
56
57
  diff-lcs (>= 1.2.0, < 2.0)
57
58
  rspec-support (~> 3.13.0)
58
59
  rspec-support (3.13.1)
60
+ securerandom (0.3.1)
59
61
  simplecov (0.22.0)
60
62
  docile (~> 1.1)
61
63
  simplecov-html (~> 0.11)
62
64
  simplecov_json_formatter (~> 0.1)
63
- simplecov-html (0.12.3)
65
+ simplecov-html (0.13.1)
64
66
  simplecov_json_formatter (0.1.4)
65
67
  tzinfo (2.0.6)
66
68
  concurrent-ruby (~> 1.0)
67
- zeitwerk (2.6.16)
69
+ zeitwerk (2.6.18)
68
70
 
69
71
  PLATFORMS
70
72
  ruby
@@ -79,4 +81,4 @@ DEPENDENCIES
79
81
  waterdrop!
80
82
 
81
83
  BUNDLED WITH
82
- 2.5.14
84
+ 2.5.16
data/LICENSE ADDED
@@ -0,0 +1,17 @@
1
+ Copyright (c) Maciej Mensfeld
2
+
3
+ WaterDrop is part of Karafka and it is an Open Source project licensed under the terms of
4
+ the LGPLv3 license. Please see <https://github.com/karafka/karafka/blob/master/LICENSE-LGPL>
5
+ for license text.
6
+
7
+ Karafka ecosystem can also be used under commercial-friendly license, with commercial support and commercial components.
8
+
9
+ All of the commercial components are present in the lib/karafka/pro and lib/karafka/web/pro
10
+ directory of this repository and their usage requires commercial license agreement.
11
+
12
+ By sending a pull request to the pro components, you are agreeing to transfer the copyright of your
13
+ code to Maciej Mensfeld.
14
+
15
+ You can find the commercial license in LICENSE-COMM <https://github.com/karafka/karafka/blob/master/LICENSE-COMM>.
16
+
17
+ Please see https://karafka.io for purchasing options.
data/README.md CHANGED
@@ -15,7 +15,7 @@ It:
15
15
  - Supports producing to multiple clusters
16
16
  - Supports multiple delivery policies
17
17
  - Supports per-topic configuration alterations (variants)
18
- - Works with Kafka `1.0+` and Ruby `3.0+`
18
+ - Works with Kafka `1.0+` and Ruby `3.1+`
19
19
  - Works with and without Karafka
20
20
 
21
21
  ## Documentation
data/certs/cert.pem ADDED
@@ -0,0 +1,26 @@
1
+ -----BEGIN CERTIFICATE-----
2
+ MIIEcDCCAtigAwIBAgIBATANBgkqhkiG9w0BAQsFADA/MRAwDgYDVQQDDAdjb250
3
+ YWN0MRcwFQYKCZImiZPyLGQBGRYHa2FyYWZrYTESMBAGCgmSJomT8ixkARkWAmlv
4
+ MB4XDTI0MDgyMzEwMTkyMFoXDTQ5MDgxNzEwMTkyMFowPzEQMA4GA1UEAwwHY29u
5
+ dGFjdDEXMBUGCgmSJomT8ixkARkWB2thcmFma2ExEjAQBgoJkiaJk/IsZAEZFgJp
6
+ bzCCAaIwDQYJKoZIhvcNAQEBBQADggGPADCCAYoCggGBAKjLhLjQqUlNayxkXnO+
7
+ PsmCDs/KFIzhrsYMfLZRZNaWmzV3ujljMOdDjd4snM2X06C41iVdQPWjpe3j8vVe
8
+ ZXEWR/twSbOP6Eeg8WVH2wCOo0x5i7yhVn4UBLH4JpfEMCbemVcWQ9ry9OMg4WpH
9
+ Uu4dRwxFV7hzCz3p0QfNLRI4miAxnGWcnlD98IJRjBAksTuR1Llj0vbOrDGsL9ZT
10
+ JeXP2gdRLd8SqzAFJEWrbeTBCBU7gfSh3oMg5SVDLjaqf7Kz5wC/8bDZydzanOxB
11
+ T6CDXPsCnllmvTNx2ei2T5rGYJOzJeNTmJLLK6hJWUlAvaQSvCwZRvFJ0tVGLEoS
12
+ flqSr6uGyyl1eMUsNmsH4BqPEYcAV6P2PKTv2vUR8AP0raDvZ3xL1TKvfRb8xRpo
13
+ vPopCGlY5XBWEc6QERHfVLTIVsjnls2/Ujj4h8/TSfqqYnaHKefIMLbuD/tquMjD
14
+ iWQsW2qStBV0T+U7FijKxVfrfqZP7GxQmDAc9o1iiyAa3QIDAQABo3cwdTAJBgNV
15
+ HRMEAjAAMAsGA1UdDwQEAwIEsDAdBgNVHQ4EFgQU3O4dTXmvE7YpAkszGzR9DdL9
16
+ sbEwHQYDVR0RBBYwFIESY29udGFjdEBrYXJhZmthLmlvMB0GA1UdEgQWMBSBEmNv
17
+ bnRhY3RAa2FyYWZrYS5pbzANBgkqhkiG9w0BAQsFAAOCAYEAVKTfoLXn7mqdSxIR
18
+ eqxcR6Huudg1jes81s1+X0uiRTR3hxxKZ3Y82cPsee9zYWyBrN8TA4KA0WILTru7
19
+ Ygxvzha0SRPsSiaKLmgOJ+61ebI4+bOORzIJLpD6GxCxu1r7MI4+0r1u1xe0EWi8
20
+ agkVo1k4Vi8cKMLm6Gl9b3wG9zQBw6fcgKwmpjKiNnOLP+OytzUANrIUJjoq6oal
21
+ TC+f/Uc0TLaRqUaW/bejxzDWWHoM3SU6aoLPuerglzp9zZVzihXwx3jPLUVKDFpF
22
+ Rl2lcBDxlpYGueGo0/oNzGJAAy6js8jhtHC9+19PD53vk7wHtFTZ/0ugDQYnwQ+x
23
+ oml2fAAuVWpTBCgOVFe6XCQpMKopzoxQ1PjKztW2KYxgJdIBX87SnL3aWuBQmhRd
24
+ i9zWxov0mr44TWegTVeypcWGd/0nxu1+QHVNHJrpqlPBRvwQsUm7fwmRInGpcaB8
25
+ ap8wNYvryYzrzvzUxIVFBVM5PacgkFqRmolCa8I7tdKQN+R1
26
+ -----END CERTIFICATE-----
data/docker-compose.yml CHANGED
@@ -3,7 +3,7 @@ version: '2'
3
3
  services:
4
4
  kafka:
5
5
  container_name: kafka
6
- image: confluentinc/cp-kafka:7.6.1
6
+ image: confluentinc/cp-kafka:7.7.1
7
7
 
8
8
  ports:
9
9
  - 9092:9092
@@ -47,6 +47,9 @@ module WaterDrop
47
47
  # graceful transaction aborting
48
48
  AbortTransaction = Class.new(BaseError)
49
49
 
50
+ # Do not use `break`, `return` or `throw` inside of the transaction blocks
51
+ EarlyTransactionExitNotAllowedError = Class.new(BaseError)
52
+
50
53
  # Raised when during messages producing something bad happened inline
51
54
  class ProduceManyError < ProduceError
52
55
  attr_reader :dispatched
@@ -26,7 +26,7 @@ module WaterDrop
26
26
  def on_message_produced_async(event)
27
27
  message = event[:message]
28
28
 
29
- info(event, "Async producing of a message to '#{message[:topic]}' topic")
29
+ info(event, "Message to '#{message[:topic]}' topic was delegated to a dispatch queue")
30
30
 
31
31
  return unless log_messages?
32
32
 
@@ -49,7 +49,10 @@ module WaterDrop
49
49
  messages = event[:messages]
50
50
  topics_count = messages.map { |message| "'#{message[:topic]}'" }.uniq.count
51
51
 
52
- info(event, "Async producing of #{messages.size} messages to #{topics_count} topics")
52
+ info(
53
+ event,
54
+ "#{messages.size} messages to #{topics_count} topics were delegated to a dispatch queue"
55
+ )
53
56
 
54
57
  return unless log_messages?
55
58
 
@@ -61,17 +61,37 @@ module WaterDrop
61
61
  messages.each { |message| validate_message!(message) }
62
62
 
63
63
  dispatched = []
64
+ inline_error = nil
64
65
 
65
66
  @monitor.instrument('messages.produced_sync', producer_id: id, messages: messages) do
66
- with_transaction_if_transactional do
67
- messages.each do |message|
68
- dispatched << produce(message)
67
+ # While most of the librdkafka errors are async and not inline, there are some like
68
+ # buffer overflow that can leak in during the `#produce` itself. When this happens, we
69
+ # still (since it's a sync mode) need to wait on deliveries of things that were
70
+ # successfully delegated to librdkafka. To do this, we catch the inline error and we
71
+ # wait on messages that were in the buffer to reach final state. Then if no error, we
72
+ # check each for error and if none all good. If there was an inline error, we re-raise
73
+ # it with the handlers in final states.
74
+ #
75
+ # Such flow ensures, that we never end up with handlers not being in the final states
76
+ # for the sync flow
77
+ begin
78
+ with_transaction_if_transactional do
79
+ messages.each do |message|
80
+ dispatched << produce(message)
81
+ end
69
82
  end
83
+ rescue *SUPPORTED_FLOW_ERRORS => e
84
+ inline_error = e
70
85
  end
71
86
 
72
- dispatched.map! do |handler|
73
- wait(handler)
74
- end
87
+ # This will ensure, that we have all verdicts before raising the failure, so we pass
88
+ # all delivery handles having a final verdict
89
+ dispatched.each { |handler| wait(handler, raise_response_error: false) }
90
+
91
+ raise(inline_error) if inline_error
92
+
93
+ # This will raise an error on the first error that have happened
94
+ dispatched.each { |handler| wait(handler) }
75
95
 
76
96
  dispatched
77
97
  end
@@ -82,7 +102,11 @@ module WaterDrop
82
102
  'error.occurred',
83
103
  producer_id: id,
84
104
  messages: messages,
85
- dispatched: dispatched,
105
+ # If it is a transactional producer nothing was successfully dispatched on error, thus
106
+ # we never return any dispatched handlers. While those messages might have reached
107
+ # Kafka, in transactional mode they will not be visible to consumers with correct
108
+ # isolation level.
109
+ dispatched: transactional? ? EMPTY_ARRAY : dispatched,
86
110
  error: re_raised,
87
111
  type: 'messages.produce_many_sync'
88
112
  )
@@ -35,7 +35,6 @@ module WaterDrop
35
35
  # with a transaction. One transaction per single dispatch and for `produce_many` it will be
36
36
  # a single transaction wrapping all messages dispatches (not one per message).
37
37
  #
38
- # @param block [Proc] block of code that should run
39
38
  # @return Block result or `nil` in case of early break/return
40
39
  #
41
40
  # @example Simple transaction
@@ -46,7 +45,7 @@ module WaterDrop
46
45
  # @example Aborted transaction - messages producer won't be visible by consumers
47
46
  # producer.transaction do
48
47
  # producer.produce_sync(topic: 'topic', payload: 'data')
49
- # throw(:abort)
48
+ # raise WaterDrop::AbortTransaction
50
49
  # end
51
50
  #
52
51
  # @example Use block result last handler to wait on all messages ack
@@ -55,7 +54,7 @@ module WaterDrop
55
54
  # end
56
55
  #
57
56
  # handler.wait
58
- def transaction(&block)
57
+ def transaction
59
58
  # This will safely allow us to support one operation transactions so a transactional
60
59
  # producer can work without the transactional block if needed
61
60
  return yield if @transaction_mutex.owned?
@@ -66,9 +65,26 @@ module WaterDrop
66
65
  transactional_instrument(:started) { client.begin_transaction }
67
66
  end
68
67
 
69
- result, commit = transactional_execute(&block)
68
+ result = nil
69
+ finished = false
70
70
 
71
- commit || raise(WaterDrop::Errors::AbortTransaction)
71
+ begin
72
+ result = yield
73
+ finished = true
74
+ rescue Exception => e
75
+ raise(e)
76
+ ensure
77
+ if !e && !finished
78
+ raise(
79
+ Errors::EarlyTransactionExitNotAllowedError,
80
+ <<~ERROR_MSG.tr("\n", ' ')
81
+ Using `return`, `break` or `throw` to exit a transaction block is not allowed.
82
+ If the `throw` came from `Timeout.timeout(duration)`, pass an exception class as
83
+ a second argument so it doesn't use `throw` to abort its block.
84
+ ERROR_MSG
85
+ )
86
+ end
87
+ end
72
88
 
73
89
  with_transactional_error_handling(:commit) do
74
90
  transactional_instrument(:committed) { client.commit_transaction }
@@ -168,27 +184,6 @@ module WaterDrop
168
184
  transactional? ? transaction(&block) : yield
169
185
  end
170
186
 
171
- # Executes the requested code in a transaction with error handling and ensures, that upon
172
- # early break we rollback the transaction instead of having it dangling and causing an issue
173
- # where transactional producer would end up in an error state.
174
- def transactional_execute
175
- result = nil
176
- commit = false
177
-
178
- catch(:abort) do
179
- result = yield
180
- commit = true
181
- end
182
-
183
- [result, commit]
184
- rescue Exception => e
185
- errored = true
186
-
187
- raise e
188
- ensure
189
- return [result, commit] unless errored
190
- end
191
-
192
187
  # Instruments the transactional operation with producer id
193
188
  #
194
189
  # @param key [Symbol] transaction operation key
@@ -76,11 +76,12 @@ module WaterDrop
76
76
  scope.instance_methods(false).each do |method_name|
77
77
  class_eval <<-RUBY, __FILE__, __LINE__ + 1
78
78
  def #{method_name}(*args, &block)
79
- Thread.current[@producer.id] = self
79
+ ref = Fiber.current.waterdrop_clients ||= {}
80
+ ref[@producer.id] = self
80
81
 
81
82
  @producer.#{method_name}(*args, &block)
82
83
  ensure
83
- Thread.current[@producer.id] = nil
84
+ ref[@producer.id] = nil
84
85
  end
85
86
  RUBY
86
87
  end
@@ -10,16 +10,22 @@ module WaterDrop
10
10
  include Transactions
11
11
  include ::Karafka::Core::Helpers::Time
12
12
 
13
+ # Local storage for given thread waterdrop client references for variants
14
+ ::Fiber.send(:attr_accessor, :waterdrop_clients)
15
+
13
16
  # Which of the inline flow errors do we want to intercept and re-bind
14
17
  SUPPORTED_FLOW_ERRORS = [
15
18
  Rdkafka::RdkafkaError,
16
19
  Rdkafka::Producer::DeliveryHandle::WaitTimeoutError
17
20
  ].freeze
18
21
 
19
- # Empty has to save on memory allocations
22
+ # Empty hash to save on memory allocations
20
23
  EMPTY_HASH = {}.freeze
21
24
 
22
- private_constant :SUPPORTED_FLOW_ERRORS, :EMPTY_HASH
25
+ # Empty array to save on memory allocations
26
+ EMPTY_ARRAY = [].freeze
27
+
28
+ private_constant :SUPPORTED_FLOW_ERRORS, :EMPTY_HASH, :EMPTY_ARRAY
23
29
 
24
30
  def_delegators :config
25
31
 
@@ -269,17 +275,21 @@ module WaterDrop
269
275
  # Waits on a given handler
270
276
  #
271
277
  # @param handler [Rdkafka::Producer::DeliveryHandle]
272
- def wait(handler)
278
+ # @param raise_response_error [Boolean] should we raise the response error after we receive the
279
+ # final result and it is an error.
280
+ def wait(handler, raise_response_error: true)
273
281
  handler.wait(
274
282
  # rdkafka max_wait_timeout is in seconds and we use ms
275
- max_wait_timeout: current_variant.max_wait_timeout / 1_000.0
283
+ max_wait_timeout: current_variant.max_wait_timeout / 1_000.0,
284
+ raise_response_error: raise_response_error
276
285
  )
277
286
  end
278
287
 
279
- # @return [Producer::Context] the variant config. Either custom if built using `#with` or
288
+ # @return [Producer::Variant] the variant config. Either custom if built using `#with` or
280
289
  # a default one.
281
290
  def current_variant
282
- Thread.current[id] || @default_variant
291
+ Fiber.current.waterdrop_clients ||= {}
292
+ Fiber.current.waterdrop_clients[id] || @default_variant
283
293
  end
284
294
 
285
295
  # Runs the client produce method with a given message
@@ -3,5 +3,5 @@
3
3
  # WaterDrop library
4
4
  module WaterDrop
5
5
  # Current WaterDrop version
6
- VERSION = '2.7.4'
6
+ VERSION = '2.8.0'
7
7
  end
data/waterdrop.gemspec CHANGED
@@ -14,19 +14,19 @@ Gem::Specification.new do |spec|
14
14
  spec.homepage = 'https://karafka.io'
15
15
  spec.summary = 'Kafka messaging made easy!'
16
16
  spec.description = spec.summary
17
- spec.license = 'MIT'
17
+ spec.licenses = %w[LGPL-3.0-only Commercial]
18
18
 
19
19
  spec.add_dependency 'karafka-core', '>= 2.4.3', '< 3.0.0'
20
- spec.add_dependency 'karafka-rdkafka', '>= 0.15.1'
20
+ spec.add_dependency 'karafka-rdkafka', '>= 0.17.5'
21
21
  spec.add_dependency 'zeitwerk', '~> 2.3'
22
22
 
23
- spec.required_ruby_version = '>= 3.0.0'
23
+ spec.required_ruby_version = '>= 3.1.0'
24
24
 
25
25
  if $PROGRAM_NAME.end_with?('gem')
26
26
  spec.signing_key = File.expand_path('~/.ssh/gem-private_key.pem')
27
27
  end
28
28
 
29
- spec.cert_chain = %w[certs/cert_chain.pem]
29
+ spec.cert_chain = %w[certs/cert.pem]
30
30
  spec.files = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(spec)/}) }
31
31
  spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
32
32
  spec.require_paths = %w[lib]
data.tar.gz.sig CHANGED
Binary file
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: waterdrop
3
3
  version: !ruby/object:Gem::Version
4
- version: 2.7.4
4
+ version: 2.8.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Maciej Mensfeld
@@ -12,30 +12,30 @@ cert_chain:
12
12
  -----BEGIN CERTIFICATE-----
13
13
  MIIEcDCCAtigAwIBAgIBATANBgkqhkiG9w0BAQsFADA/MRAwDgYDVQQDDAdjb250
14
14
  YWN0MRcwFQYKCZImiZPyLGQBGRYHa2FyYWZrYTESMBAGCgmSJomT8ixkARkWAmlv
15
- MB4XDTIzMDgyMTA3MjU1NFoXDTI0MDgyMDA3MjU1NFowPzEQMA4GA1UEAwwHY29u
15
+ MB4XDTI0MDgyMzEwMTkyMFoXDTQ5MDgxNzEwMTkyMFowPzEQMA4GA1UEAwwHY29u
16
16
  dGFjdDEXMBUGCgmSJomT8ixkARkWB2thcmFma2ExEjAQBgoJkiaJk/IsZAEZFgJp
17
- bzCCAaIwDQYJKoZIhvcNAQEBBQADggGPADCCAYoCggGBAOuZpyQKEwsTG9plLat7
18
- 8bUaNuNBEnouTsNMr6X+XTgvyrAxTuocdsyP1sNCjdS1B8RiiDH1/Nt9qpvlBWon
19
- sdJ1SYhaWNVfqiYStTDnCx3PRMmHRdD4KqUWKpN6VpZ1O/Zu+9Mw0COmvXgZuuO9
20
- wMSJkXRo6dTCfMedLAIxjMeBIxtoLR2e6Jm6MR8+8WYYVWrO9kSOOt5eKQLBY7aK
21
- b/Dc40EcJKPg3Z30Pia1M9ZyRlb6SOj6SKpHRqc7vbVQxjEw6Jjal1lZ49m3YZMd
22
- ArMAs9lQZNdSw5/UX6HWWURLowg6k10RnhTUtYyzO9BFev0JFJftHnmuk8vtb+SD
23
- 5VPmjFXg2VOcw0B7FtG75Vackk8QKfgVe3nSPhVpew2CSPlbJzH80wChbr19+e3+
24
- YGr1tOiaJrL6c+PNmb0F31NXMKpj/r+n15HwlTMRxQrzFcgjBlxf2XFGnPQXHhBm
25
- kp1OFnEq4GG9sON4glRldkwzi/f/fGcZmo5fm3d+0ZdNgwIDAQABo3cwdTAJBgNV
26
- HRMEAjAAMAsGA1UdDwQEAwIEsDAdBgNVHQ4EFgQUPVH5+dLA80A1kJ2Uz5iGwfOa
27
- 1+swHQYDVR0RBBYwFIESY29udGFjdEBrYXJhZmthLmlvMB0GA1UdEgQWMBSBEmNv
28
- bnRhY3RAa2FyYWZrYS5pbzANBgkqhkiG9w0BAQsFAAOCAYEAnpa0jcN7JzREHMTQ
29
- bfZ+xcvlrzuROMY6A3zIZmQgbnoZZNuX4cMRrT1p1HuwXpxdpHPw7dDjYqWw3+1h
30
- 3mXLeMuk7amjQpYoSWU/OIZMhIsARra22UN8qkkUlUj3AwTaChVKN/bPJOM2DzfU
31
- kz9vUgLeYYFfQbZqeI6SsM7ltilRV4W8D9yNUQQvOxCFxtLOetJ00fC/E7zMUzbK
32
- IBwYFQYsbI6XQzgAIPW6nGSYKgRhkfpmquXSNKZRIQ4V6bFrufa+DzD0bt2ZA3ah
33
- fMmJguyb5L2Gf1zpDXzFSPMG7YQFLzwYz1zZZvOU7/UCpQsHpID/YxqDp4+Dgb+Y
34
- qma0whX8UG/gXFV2pYWpYOfpatvahwi+A1TwPQsuZwkkhi1OyF1At3RY+hjSXyav
35
- AnG1dJU+yL2BK7vaVytLTstJME5mepSZ46qqIJXMuWob/YPDmVaBF39TDSG9e34s
36
- msG3BiCqgOgHAnL23+CN3Rt8MsuRfEtoTKpJVcCfoEoNHOkc
17
+ bzCCAaIwDQYJKoZIhvcNAQEBBQADggGPADCCAYoCggGBAKjLhLjQqUlNayxkXnO+
18
+ PsmCDs/KFIzhrsYMfLZRZNaWmzV3ujljMOdDjd4snM2X06C41iVdQPWjpe3j8vVe
19
+ ZXEWR/twSbOP6Eeg8WVH2wCOo0x5i7yhVn4UBLH4JpfEMCbemVcWQ9ry9OMg4WpH
20
+ Uu4dRwxFV7hzCz3p0QfNLRI4miAxnGWcnlD98IJRjBAksTuR1Llj0vbOrDGsL9ZT
21
+ JeXP2gdRLd8SqzAFJEWrbeTBCBU7gfSh3oMg5SVDLjaqf7Kz5wC/8bDZydzanOxB
22
+ T6CDXPsCnllmvTNx2ei2T5rGYJOzJeNTmJLLK6hJWUlAvaQSvCwZRvFJ0tVGLEoS
23
+ flqSr6uGyyl1eMUsNmsH4BqPEYcAV6P2PKTv2vUR8AP0raDvZ3xL1TKvfRb8xRpo
24
+ vPopCGlY5XBWEc6QERHfVLTIVsjnls2/Ujj4h8/TSfqqYnaHKefIMLbuD/tquMjD
25
+ iWQsW2qStBV0T+U7FijKxVfrfqZP7GxQmDAc9o1iiyAa3QIDAQABo3cwdTAJBgNV
26
+ HRMEAjAAMAsGA1UdDwQEAwIEsDAdBgNVHQ4EFgQU3O4dTXmvE7YpAkszGzR9DdL9
27
+ sbEwHQYDVR0RBBYwFIESY29udGFjdEBrYXJhZmthLmlvMB0GA1UdEgQWMBSBEmNv
28
+ bnRhY3RAa2FyYWZrYS5pbzANBgkqhkiG9w0BAQsFAAOCAYEAVKTfoLXn7mqdSxIR
29
+ eqxcR6Huudg1jes81s1+X0uiRTR3hxxKZ3Y82cPsee9zYWyBrN8TA4KA0WILTru7
30
+ Ygxvzha0SRPsSiaKLmgOJ+61ebI4+bOORzIJLpD6GxCxu1r7MI4+0r1u1xe0EWi8
31
+ agkVo1k4Vi8cKMLm6Gl9b3wG9zQBw6fcgKwmpjKiNnOLP+OytzUANrIUJjoq6oal
32
+ TC+f/Uc0TLaRqUaW/bejxzDWWHoM3SU6aoLPuerglzp9zZVzihXwx3jPLUVKDFpF
33
+ Rl2lcBDxlpYGueGo0/oNzGJAAy6js8jhtHC9+19PD53vk7wHtFTZ/0ugDQYnwQ+x
34
+ oml2fAAuVWpTBCgOVFe6XCQpMKopzoxQ1PjKztW2KYxgJdIBX87SnL3aWuBQmhRd
35
+ i9zWxov0mr44TWegTVeypcWGd/0nxu1+QHVNHJrpqlPBRvwQsUm7fwmRInGpcaB8
36
+ ap8wNYvryYzrzvzUxIVFBVM5PacgkFqRmolCa8I7tdKQN+R1
37
37
  -----END CERTIFICATE-----
38
- date: 2024-07-04 00:00:00.000000000 Z
38
+ date: 2024-09-16 00:00:00.000000000 Z
39
39
  dependencies:
40
40
  - !ruby/object:Gem::Dependency
41
41
  name: karafka-core
@@ -63,14 +63,14 @@ dependencies:
63
63
  requirements:
64
64
  - - ">="
65
65
  - !ruby/object:Gem::Version
66
- version: 0.15.1
66
+ version: 0.17.5
67
67
  type: :runtime
68
68
  prerelease: false
69
69
  version_requirements: !ruby/object:Gem::Requirement
70
70
  requirements:
71
71
  - - ">="
72
72
  - !ruby/object:Gem::Version
73
- version: 0.15.1
73
+ version: 0.17.5
74
74
  - !ruby/object:Gem::Dependency
75
75
  name: zeitwerk
76
76
  requirement: !ruby/object:Gem::Requirement
@@ -103,9 +103,9 @@ files:
103
103
  - CHANGELOG.md
104
104
  - Gemfile
105
105
  - Gemfile.lock
106
- - MIT-LICENSE
106
+ - LICENSE
107
107
  - README.md
108
- - certs/cert_chain.pem
108
+ - certs/cert.pem
109
109
  - config/locales/errors.yml
110
110
  - docker-compose.yml
111
111
  - lib/waterdrop.rb
@@ -144,7 +144,8 @@ files:
144
144
  - waterdrop.gemspec
145
145
  homepage: https://karafka.io
146
146
  licenses:
147
- - MIT
147
+ - LGPL-3.0-only
148
+ - Commercial
148
149
  metadata:
149
150
  funding_uri: https://karafka.io/#become-pro
150
151
  homepage_uri: https://karafka.io
@@ -161,14 +162,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
161
162
  requirements:
162
163
  - - ">="
163
164
  - !ruby/object:Gem::Version
164
- version: 3.0.0
165
+ version: 3.1.0
165
166
  required_rubygems_version: !ruby/object:Gem::Requirement
166
167
  requirements:
167
168
  - - ">="
168
169
  - !ruby/object:Gem::Version
169
170
  version: '0'
170
171
  requirements: []
171
- rubygems_version: 3.5.11
172
+ rubygems_version: 3.5.16
172
173
  signing_key:
173
174
  specification_version: 4
174
175
  summary: Kafka messaging made easy!
metadata.gz.sig CHANGED
Binary file
data/MIT-LICENSE DELETED
@@ -1,18 +0,0 @@
1
- Permission is hereby granted, free of charge, to any person obtaining
2
- a copy of this software and associated documentation files (the
3
- "Software"), to deal in the Software without restriction, including
4
- without limitation the rights to use, copy, modify, merge, publish,
5
- distribute, sublicense, and/or sell copies of the Software, and to
6
- permit persons to whom the Software is furnished to do so, subject to
7
- the following conditions:
8
-
9
- The above copyright notice and this permission notice shall be
10
- included in all copies or substantial portions of the Software.
11
-
12
- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
13
- EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
14
- MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
15
- NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
16
- LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
17
- OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
18
- WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/certs/cert_chain.pem DELETED
@@ -1,26 +0,0 @@
1
- -----BEGIN CERTIFICATE-----
2
- MIIEcDCCAtigAwIBAgIBATANBgkqhkiG9w0BAQsFADA/MRAwDgYDVQQDDAdjb250
3
- YWN0MRcwFQYKCZImiZPyLGQBGRYHa2FyYWZrYTESMBAGCgmSJomT8ixkARkWAmlv
4
- MB4XDTIzMDgyMTA3MjU1NFoXDTI0MDgyMDA3MjU1NFowPzEQMA4GA1UEAwwHY29u
5
- dGFjdDEXMBUGCgmSJomT8ixkARkWB2thcmFma2ExEjAQBgoJkiaJk/IsZAEZFgJp
6
- bzCCAaIwDQYJKoZIhvcNAQEBBQADggGPADCCAYoCggGBAOuZpyQKEwsTG9plLat7
7
- 8bUaNuNBEnouTsNMr6X+XTgvyrAxTuocdsyP1sNCjdS1B8RiiDH1/Nt9qpvlBWon
8
- sdJ1SYhaWNVfqiYStTDnCx3PRMmHRdD4KqUWKpN6VpZ1O/Zu+9Mw0COmvXgZuuO9
9
- wMSJkXRo6dTCfMedLAIxjMeBIxtoLR2e6Jm6MR8+8WYYVWrO9kSOOt5eKQLBY7aK
10
- b/Dc40EcJKPg3Z30Pia1M9ZyRlb6SOj6SKpHRqc7vbVQxjEw6Jjal1lZ49m3YZMd
11
- ArMAs9lQZNdSw5/UX6HWWURLowg6k10RnhTUtYyzO9BFev0JFJftHnmuk8vtb+SD
12
- 5VPmjFXg2VOcw0B7FtG75Vackk8QKfgVe3nSPhVpew2CSPlbJzH80wChbr19+e3+
13
- YGr1tOiaJrL6c+PNmb0F31NXMKpj/r+n15HwlTMRxQrzFcgjBlxf2XFGnPQXHhBm
14
- kp1OFnEq4GG9sON4glRldkwzi/f/fGcZmo5fm3d+0ZdNgwIDAQABo3cwdTAJBgNV
15
- HRMEAjAAMAsGA1UdDwQEAwIEsDAdBgNVHQ4EFgQUPVH5+dLA80A1kJ2Uz5iGwfOa
16
- 1+swHQYDVR0RBBYwFIESY29udGFjdEBrYXJhZmthLmlvMB0GA1UdEgQWMBSBEmNv
17
- bnRhY3RAa2FyYWZrYS5pbzANBgkqhkiG9w0BAQsFAAOCAYEAnpa0jcN7JzREHMTQ
18
- bfZ+xcvlrzuROMY6A3zIZmQgbnoZZNuX4cMRrT1p1HuwXpxdpHPw7dDjYqWw3+1h
19
- 3mXLeMuk7amjQpYoSWU/OIZMhIsARra22UN8qkkUlUj3AwTaChVKN/bPJOM2DzfU
20
- kz9vUgLeYYFfQbZqeI6SsM7ltilRV4W8D9yNUQQvOxCFxtLOetJ00fC/E7zMUzbK
21
- IBwYFQYsbI6XQzgAIPW6nGSYKgRhkfpmquXSNKZRIQ4V6bFrufa+DzD0bt2ZA3ah
22
- fMmJguyb5L2Gf1zpDXzFSPMG7YQFLzwYz1zZZvOU7/UCpQsHpID/YxqDp4+Dgb+Y
23
- qma0whX8UG/gXFV2pYWpYOfpatvahwi+A1TwPQsuZwkkhi1OyF1At3RY+hjSXyav
24
- AnG1dJU+yL2BK7vaVytLTstJME5mepSZ46qqIJXMuWob/YPDmVaBF39TDSG9e34s
25
- msG3BiCqgOgHAnL23+CN3Rt8MsuRfEtoTKpJVcCfoEoNHOkc
26
- -----END CERTIFICATE-----