waterdrop 2.7.4 → 2.8.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 7829f5c177154b491525c3895e2f12c27c6d58b8e43d47130caf6ee44dc5de2f
4
- data.tar.gz: bb92b727e921030a5933735f8f1e0e01727fb8e91bc607b1c0adb74031b4881f
3
+ metadata.gz: 96435533f730edae9dcf7bed5426641a33715e87b55c804168fa6a02465f1431
4
+ data.tar.gz: 3737e88ec3ffaa407b1cdaa523cf1d24a25aa85c4d90989a249381dbfb7a80c2
5
5
  SHA512:
6
- metadata.gz: c7ed272bc46e9f7ea1e398c66588c730f1ca0f7427e7f215d4c89285498e6ccefc1c4af709a1c17f29382e7cc75ebdee403aef865661b8fefdacd164f95acf25
7
- data.tar.gz: 99d9dcd114f67d272e0d2650ee2bd9aa0b480e9d850fb9f0f32813e02a90603b435b021a27faab09a411f3dea438c9307b14bd12e349289bf977c1b984805604
6
+ metadata.gz: 1b83e8505ca1aa9be5c137c06af1b98e10bb8f39af93e77aefa42302bf8dc2bd95f1442eb2569d5f032e80ac0374b8013e774d1f36494d23e20af0766094e2b3
7
+ data.tar.gz: 198ea59a4f7ca6a63f9816e7026c97152aaed5e6154142c47e6e96c16696a2a3d4c60cd56bf72b407b092682e9d15c08f1adfa552eb5bdc17a308f77f7424dfc
checksums.yaml.gz.sig CHANGED
Binary file
@@ -22,7 +22,6 @@ jobs:
22
22
  - '3.3'
23
23
  - '3.2'
24
24
  - '3.1'
25
- - '3.0'
26
25
  include:
27
26
  - ruby: '3.3'
28
27
  coverage: 'true'
data/.ruby-version CHANGED
@@ -1 +1 @@
1
- 3.3.3
1
+ 3.3.5
data/CHANGELOG.md CHANGED
@@ -1,5 +1,102 @@
1
1
  # WaterDrop changelog
2
2
 
3
+ ## 2.8.0 (2024-09-16)
4
+
5
+ This release contains **BREAKING** changes. Make sure to read and apply upgrade notes.
6
+
7
+ - **[Breaking]** Require Ruby `3.1+`.
8
+ - **[Breaking]** Remove ability to abort transactions using `throw(:abort)`. Please use `raise WaterDrop::Errors::AbortTransaction`.
9
+ - **[Breaking]** Disallow (similar to ActiveRecord) exiting transactions with `return`, `break` or `throw`.
10
+ - **[Breaking]** License changed from MIT to LGPL with an additional commercial option. Note: there is no commercial code in this repository. The commercial license is available for companies unable to use LGPL-licensed software for legal reasons.
11
+ - [Enhancement] Make variants fiber safe.
12
+ - [Enhancement] In transactional mode do not return any `dispatched` messages as none will be dispatched due to rollback.
13
+ - [Enhancement] Align the `LoggerListener` async messages to reflect, that messages are delegated to the internal queue and not dispatched.
14
+ - [Fix] Ensure, that `:dispatched` key for `#produce_many_sync` always contains delivery handles (final) and not delivery reports.
15
+
16
+ ### Upgrade Notes
17
+
18
+ **PLEASE MAKE SURE TO READ AND APPLY THEM!**
19
+
20
+ #### `throw(:abort)` No Longer Allowed To Abort Transactions
21
+
22
+ Replace:
23
+
24
+ ```ruby
25
+ producer.transaction do
26
+ messages.each do |message|
27
+ # Pipe all events
28
+ producer.produce_async(topic: 'events', payload: message.raw_payload)
29
+ end
30
+
31
+ # And abort if more events are no longer needed
32
+ throw(:abort) if KnowledgeBase.more_events_needed?
33
+ end
34
+ ```
35
+
36
+ With:
37
+
38
+ ```ruby
39
+ producer.transaction do
40
+ messages.each do |message|
41
+ # Pipe all events
42
+ producer.produce_async(topic: 'events', payload: message.raw_payload)
43
+ end
44
+
45
+ # And abort if more events are no longer needed
46
+ raise(WaterDrop::AbortTransaction) if KnowledgeBase.more_events_needed?
47
+ end
48
+ ```
49
+
50
+ #### `return`, `break` and `throw` No Longer Allowed Inside Transaction Block
51
+
52
+ Previously, transactions would abort if you exited early using `return`, `break`, or `throw`. This could create unexpected behavior, where users might not notice the rollback or have different intentions. For example, the following would trigger a rollback:
53
+
54
+ ```ruby
55
+ MAX = 10
56
+
57
+ def process(messages)
58
+ count = 0
59
+
60
+ producer.transaction do
61
+ messages.each do |message|
62
+ count += 1
63
+
64
+ producer.produce_async(topic: 'events', payload: message.raw_payload)
65
+
66
+ # This would trigger a rollback.
67
+ return if count >= MAX
68
+ end
69
+ end
70
+ end
71
+ ```
72
+
73
+ This is a source of errors, hence such exits are no longer allowed. You can implement similar flow control inside of your methods that are wrapped in a WaterDrop transaction:
74
+
75
+ ```ruby
76
+ MAX = 10
77
+
78
+ def process(messages)
79
+ producer.transaction do
80
+ # Early return from this method will not affect the transaction.
81
+ # It will be committed
82
+ insert_with_limit(messages)
83
+ end
84
+ end
85
+
86
+ def insert_with_limit(messages)
87
+ count = 0
88
+
89
+ messages.each do |message|
90
+ count += 1
91
+
92
+ producer.produce_async(topic: 'events', payload: message.raw_payload)
93
+
94
+ # This would trigger a rollback.
95
+ return if count >= MAX
96
+ end
97
+ end
98
+ ```
99
+
3
100
  ## 2.7.4 (2024-07-04)
4
101
  - [Maintenance] Alias `WaterDrop::Errors::AbortTransaction` with `WaterDrop::AbortTransaction`.
5
102
  - [Maintenance] Lower the precision reporting to 100 microseconds in the logger listener.
data/Gemfile.lock CHANGED
@@ -1,70 +1,72 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- waterdrop (2.7.4)
4
+ waterdrop (2.8.0)
5
5
  karafka-core (>= 2.4.3, < 3.0.0)
6
- karafka-rdkafka (>= 0.15.1)
6
+ karafka-rdkafka (>= 0.17.5)
7
7
  zeitwerk (~> 2.3)
8
8
 
9
9
  GEM
10
10
  remote: https://rubygems.org/
11
11
  specs:
12
- activesupport (7.1.3.4)
12
+ activesupport (7.2.1)
13
13
  base64
14
14
  bigdecimal
15
- concurrent-ruby (~> 1.0, >= 1.0.2)
15
+ concurrent-ruby (~> 1.0, >= 1.3.1)
16
16
  connection_pool (>= 2.2.5)
17
17
  drb
18
18
  i18n (>= 1.6, < 2)
19
+ logger (>= 1.4.2)
19
20
  minitest (>= 5.1)
20
- mutex_m
21
- tzinfo (~> 2.0)
21
+ securerandom (>= 0.3)
22
+ tzinfo (~> 2.0, >= 2.0.5)
22
23
  base64 (0.2.0)
23
24
  bigdecimal (3.1.8)
24
25
  byebug (11.1.3)
25
- concurrent-ruby (1.3.3)
26
+ concurrent-ruby (1.3.4)
26
27
  connection_pool (2.4.1)
27
28
  diff-lcs (1.5.1)
28
- docile (1.4.0)
29
+ docile (1.4.1)
29
30
  drb (2.2.1)
30
- factory_bot (6.4.6)
31
+ factory_bot (6.5.0)
31
32
  activesupport (>= 5.0.0)
32
33
  ffi (1.17.0)
33
34
  i18n (1.14.5)
34
35
  concurrent-ruby (~> 1.0)
35
- karafka-core (2.4.3)
36
- karafka-rdkafka (>= 0.15.0, < 0.17.0)
37
- karafka-rdkafka (0.16.0)
36
+ karafka-core (2.4.4)
37
+ karafka-rdkafka (>= 0.15.0, < 0.18.0)
38
+ karafka-rdkafka (0.17.6)
38
39
  ffi (~> 1.15)
39
40
  mini_portile2 (~> 2.6)
40
41
  rake (> 12)
42
+ logger (1.6.1)
41
43
  mini_portile2 (2.8.7)
42
- minitest (5.24.0)
43
- mutex_m (0.2.0)
44
+ minitest (5.25.1)
44
45
  ostruct (0.6.0)
45
46
  rake (13.2.1)
46
47
  rspec (3.13.0)
47
48
  rspec-core (~> 3.13.0)
48
49
  rspec-expectations (~> 3.13.0)
49
50
  rspec-mocks (~> 3.13.0)
50
- rspec-core (3.13.0)
51
+ rspec-core (3.13.1)
51
52
  rspec-support (~> 3.13.0)
52
- rspec-expectations (3.13.1)
53
+ rspec-expectations (3.13.3)
53
54
  diff-lcs (>= 1.2.0, < 2.0)
54
55
  rspec-support (~> 3.13.0)
55
56
  rspec-mocks (3.13.1)
56
57
  diff-lcs (>= 1.2.0, < 2.0)
57
58
  rspec-support (~> 3.13.0)
58
59
  rspec-support (3.13.1)
60
+ securerandom (0.3.1)
59
61
  simplecov (0.22.0)
60
62
  docile (~> 1.1)
61
63
  simplecov-html (~> 0.11)
62
64
  simplecov_json_formatter (~> 0.1)
63
- simplecov-html (0.12.3)
65
+ simplecov-html (0.13.1)
64
66
  simplecov_json_formatter (0.1.4)
65
67
  tzinfo (2.0.6)
66
68
  concurrent-ruby (~> 1.0)
67
- zeitwerk (2.6.16)
69
+ zeitwerk (2.6.18)
68
70
 
69
71
  PLATFORMS
70
72
  ruby
@@ -79,4 +81,4 @@ DEPENDENCIES
79
81
  waterdrop!
80
82
 
81
83
  BUNDLED WITH
82
- 2.5.14
84
+ 2.5.16
data/LICENSE ADDED
@@ -0,0 +1,17 @@
1
+ Copyright (c) Maciej Mensfeld
2
+
3
+ WaterDrop is part of Karafka and it is an Open Source project licensed under the terms of
4
+ the LGPLv3 license. Please see <https://github.com/karafka/karafka/blob/master/LICENSE-LGPL>
5
+ for license text.
6
+
7
+ Karafka ecosystem can also be used under commercial-friendly license, with commercial support and commercial components.
8
+
9
+ All of the commercial components are present in the lib/karafka/pro and lib/karafka/web/pro
10
+ directory of this repository and their usage requires commercial license agreement.
11
+
12
+ By sending a pull request to the pro components, you are agreeing to transfer the copyright of your
13
+ code to Maciej Mensfeld.
14
+
15
+ You can find the commercial license in LICENSE-COMM <https://github.com/karafka/karafka/blob/master/LICENSE-COMM>.
16
+
17
+ Please see https://karafka.io for purchasing options.
data/README.md CHANGED
@@ -15,7 +15,7 @@ It:
15
15
  - Supports producing to multiple clusters
16
16
  - Supports multiple delivery policies
17
17
  - Supports per-topic configuration alterations (variants)
18
- - Works with Kafka `1.0+` and Ruby `3.0+`
18
+ - Works with Kafka `1.0+` and Ruby `3.1+`
19
19
  - Works with and without Karafka
20
20
 
21
21
  ## Documentation
data/certs/cert.pem ADDED
@@ -0,0 +1,26 @@
1
+ -----BEGIN CERTIFICATE-----
2
+ MIIEcDCCAtigAwIBAgIBATANBgkqhkiG9w0BAQsFADA/MRAwDgYDVQQDDAdjb250
3
+ YWN0MRcwFQYKCZImiZPyLGQBGRYHa2FyYWZrYTESMBAGCgmSJomT8ixkARkWAmlv
4
+ MB4XDTI0MDgyMzEwMTkyMFoXDTQ5MDgxNzEwMTkyMFowPzEQMA4GA1UEAwwHY29u
5
+ dGFjdDEXMBUGCgmSJomT8ixkARkWB2thcmFma2ExEjAQBgoJkiaJk/IsZAEZFgJp
6
+ bzCCAaIwDQYJKoZIhvcNAQEBBQADggGPADCCAYoCggGBAKjLhLjQqUlNayxkXnO+
7
+ PsmCDs/KFIzhrsYMfLZRZNaWmzV3ujljMOdDjd4snM2X06C41iVdQPWjpe3j8vVe
8
+ ZXEWR/twSbOP6Eeg8WVH2wCOo0x5i7yhVn4UBLH4JpfEMCbemVcWQ9ry9OMg4WpH
9
+ Uu4dRwxFV7hzCz3p0QfNLRI4miAxnGWcnlD98IJRjBAksTuR1Llj0vbOrDGsL9ZT
10
+ JeXP2gdRLd8SqzAFJEWrbeTBCBU7gfSh3oMg5SVDLjaqf7Kz5wC/8bDZydzanOxB
11
+ T6CDXPsCnllmvTNx2ei2T5rGYJOzJeNTmJLLK6hJWUlAvaQSvCwZRvFJ0tVGLEoS
12
+ flqSr6uGyyl1eMUsNmsH4BqPEYcAV6P2PKTv2vUR8AP0raDvZ3xL1TKvfRb8xRpo
13
+ vPopCGlY5XBWEc6QERHfVLTIVsjnls2/Ujj4h8/TSfqqYnaHKefIMLbuD/tquMjD
14
+ iWQsW2qStBV0T+U7FijKxVfrfqZP7GxQmDAc9o1iiyAa3QIDAQABo3cwdTAJBgNV
15
+ HRMEAjAAMAsGA1UdDwQEAwIEsDAdBgNVHQ4EFgQU3O4dTXmvE7YpAkszGzR9DdL9
16
+ sbEwHQYDVR0RBBYwFIESY29udGFjdEBrYXJhZmthLmlvMB0GA1UdEgQWMBSBEmNv
17
+ bnRhY3RAa2FyYWZrYS5pbzANBgkqhkiG9w0BAQsFAAOCAYEAVKTfoLXn7mqdSxIR
18
+ eqxcR6Huudg1jes81s1+X0uiRTR3hxxKZ3Y82cPsee9zYWyBrN8TA4KA0WILTru7
19
+ Ygxvzha0SRPsSiaKLmgOJ+61ebI4+bOORzIJLpD6GxCxu1r7MI4+0r1u1xe0EWi8
20
+ agkVo1k4Vi8cKMLm6Gl9b3wG9zQBw6fcgKwmpjKiNnOLP+OytzUANrIUJjoq6oal
21
+ TC+f/Uc0TLaRqUaW/bejxzDWWHoM3SU6aoLPuerglzp9zZVzihXwx3jPLUVKDFpF
22
+ Rl2lcBDxlpYGueGo0/oNzGJAAy6js8jhtHC9+19PD53vk7wHtFTZ/0ugDQYnwQ+x
23
+ oml2fAAuVWpTBCgOVFe6XCQpMKopzoxQ1PjKztW2KYxgJdIBX87SnL3aWuBQmhRd
24
+ i9zWxov0mr44TWegTVeypcWGd/0nxu1+QHVNHJrpqlPBRvwQsUm7fwmRInGpcaB8
25
+ ap8wNYvryYzrzvzUxIVFBVM5PacgkFqRmolCa8I7tdKQN+R1
26
+ -----END CERTIFICATE-----
data/docker-compose.yml CHANGED
@@ -3,7 +3,7 @@ version: '2'
3
3
  services:
4
4
  kafka:
5
5
  container_name: kafka
6
- image: confluentinc/cp-kafka:7.6.1
6
+ image: confluentinc/cp-kafka:7.7.1
7
7
 
8
8
  ports:
9
9
  - 9092:9092
@@ -47,6 +47,9 @@ module WaterDrop
47
47
  # graceful transaction aborting
48
48
  AbortTransaction = Class.new(BaseError)
49
49
 
50
+ # Do not use `break`, `return` or `throw` inside of the transaction blocks
51
+ EarlyTransactionExitNotAllowedError = Class.new(BaseError)
52
+
50
53
  # Raised when during messages producing something bad happened inline
51
54
  class ProduceManyError < ProduceError
52
55
  attr_reader :dispatched
@@ -26,7 +26,7 @@ module WaterDrop
26
26
  def on_message_produced_async(event)
27
27
  message = event[:message]
28
28
 
29
- info(event, "Async producing of a message to '#{message[:topic]}' topic")
29
+ info(event, "Message to '#{message[:topic]}' topic was delegated to a dispatch queue")
30
30
 
31
31
  return unless log_messages?
32
32
 
@@ -49,7 +49,10 @@ module WaterDrop
49
49
  messages = event[:messages]
50
50
  topics_count = messages.map { |message| "'#{message[:topic]}'" }.uniq.count
51
51
 
52
- info(event, "Async producing of #{messages.size} messages to #{topics_count} topics")
52
+ info(
53
+ event,
54
+ "#{messages.size} messages to #{topics_count} topics were delegated to a dispatch queue"
55
+ )
53
56
 
54
57
  return unless log_messages?
55
58
 
@@ -61,17 +61,37 @@ module WaterDrop
61
61
  messages.each { |message| validate_message!(message) }
62
62
 
63
63
  dispatched = []
64
+ inline_error = nil
64
65
 
65
66
  @monitor.instrument('messages.produced_sync', producer_id: id, messages: messages) do
66
- with_transaction_if_transactional do
67
- messages.each do |message|
68
- dispatched << produce(message)
67
+ # While most of the librdkafka errors are async and not inline, there are some like
68
+ # buffer overflow that can leak in during the `#produce` itself. When this happens, we
69
+ # still (since it's a sync mode) need to wait on deliveries of things that were
70
+ # successfully delegated to librdkafka. To do this, we catch the inline error and we
71
+ # wait on messages that were in the buffer to reach final state. Then if no error, we
72
+ # check each for error and if none all good. If there was an inline error, we re-raise
73
+ # it with the handlers in final states.
74
+ #
75
+ # Such flow ensures, that we never end up with handlers not being in the final states
76
+ # for the sync flow
77
+ begin
78
+ with_transaction_if_transactional do
79
+ messages.each do |message|
80
+ dispatched << produce(message)
81
+ end
69
82
  end
83
+ rescue *SUPPORTED_FLOW_ERRORS => e
84
+ inline_error = e
70
85
  end
71
86
 
72
- dispatched.map! do |handler|
73
- wait(handler)
74
- end
87
+ # This will ensure, that we have all verdicts before raising the failure, so we pass
88
+ # all delivery handles having a final verdict
89
+ dispatched.each { |handler| wait(handler, raise_response_error: false) }
90
+
91
+ raise(inline_error) if inline_error
92
+
93
+ # This will raise an error on the first error that have happened
94
+ dispatched.each { |handler| wait(handler) }
75
95
 
76
96
  dispatched
77
97
  end
@@ -82,7 +102,11 @@ module WaterDrop
82
102
  'error.occurred',
83
103
  producer_id: id,
84
104
  messages: messages,
85
- dispatched: dispatched,
105
+ # If it is a transactional producer nothing was successfully dispatched on error, thus
106
+ # we never return any dispatched handlers. While those messages might have reached
107
+ # Kafka, in transactional mode they will not be visible to consumers with correct
108
+ # isolation level.
109
+ dispatched: transactional? ? EMPTY_ARRAY : dispatched,
86
110
  error: re_raised,
87
111
  type: 'messages.produce_many_sync'
88
112
  )
@@ -35,7 +35,6 @@ module WaterDrop
35
35
  # with a transaction. One transaction per single dispatch and for `produce_many` it will be
36
36
  # a single transaction wrapping all messages dispatches (not one per message).
37
37
  #
38
- # @param block [Proc] block of code that should run
39
38
  # @return Block result or `nil` in case of early break/return
40
39
  #
41
40
  # @example Simple transaction
@@ -46,7 +45,7 @@ module WaterDrop
46
45
  # @example Aborted transaction - messages producer won't be visible by consumers
47
46
  # producer.transaction do
48
47
  # producer.produce_sync(topic: 'topic', payload: 'data')
49
- # throw(:abort)
48
+ # raise WaterDrop::AbortTransaction
50
49
  # end
51
50
  #
52
51
  # @example Use block result last handler to wait on all messages ack
@@ -55,7 +54,7 @@ module WaterDrop
55
54
  # end
56
55
  #
57
56
  # handler.wait
58
- def transaction(&block)
57
+ def transaction
59
58
  # This will safely allow us to support one operation transactions so a transactional
60
59
  # producer can work without the transactional block if needed
61
60
  return yield if @transaction_mutex.owned?
@@ -66,9 +65,26 @@ module WaterDrop
66
65
  transactional_instrument(:started) { client.begin_transaction }
67
66
  end
68
67
 
69
- result, commit = transactional_execute(&block)
68
+ result = nil
69
+ finished = false
70
70
 
71
- commit || raise(WaterDrop::Errors::AbortTransaction)
71
+ begin
72
+ result = yield
73
+ finished = true
74
+ rescue Exception => e
75
+ raise(e)
76
+ ensure
77
+ if !e && !finished
78
+ raise(
79
+ Errors::EarlyTransactionExitNotAllowedError,
80
+ <<~ERROR_MSG.tr("\n", ' ')
81
+ Using `return`, `break` or `throw` to exit a transaction block is not allowed.
82
+ If the `throw` came from `Timeout.timeout(duration)`, pass an exception class as
83
+ a second argument so it doesn't use `throw` to abort its block.
84
+ ERROR_MSG
85
+ )
86
+ end
87
+ end
72
88
 
73
89
  with_transactional_error_handling(:commit) do
74
90
  transactional_instrument(:committed) { client.commit_transaction }
@@ -168,27 +184,6 @@ module WaterDrop
168
184
  transactional? ? transaction(&block) : yield
169
185
  end
170
186
 
171
- # Executes the requested code in a transaction with error handling and ensures, that upon
172
- # early break we rollback the transaction instead of having it dangling and causing an issue
173
- # where transactional producer would end up in an error state.
174
- def transactional_execute
175
- result = nil
176
- commit = false
177
-
178
- catch(:abort) do
179
- result = yield
180
- commit = true
181
- end
182
-
183
- [result, commit]
184
- rescue Exception => e
185
- errored = true
186
-
187
- raise e
188
- ensure
189
- return [result, commit] unless errored
190
- end
191
-
192
187
  # Instruments the transactional operation with producer id
193
188
  #
194
189
  # @param key [Symbol] transaction operation key
@@ -76,11 +76,12 @@ module WaterDrop
76
76
  scope.instance_methods(false).each do |method_name|
77
77
  class_eval <<-RUBY, __FILE__, __LINE__ + 1
78
78
  def #{method_name}(*args, &block)
79
- Thread.current[@producer.id] = self
79
+ ref = Fiber.current.waterdrop_clients ||= {}
80
+ ref[@producer.id] = self
80
81
 
81
82
  @producer.#{method_name}(*args, &block)
82
83
  ensure
83
- Thread.current[@producer.id] = nil
84
+ ref[@producer.id] = nil
84
85
  end
85
86
  RUBY
86
87
  end
@@ -10,16 +10,22 @@ module WaterDrop
10
10
  include Transactions
11
11
  include ::Karafka::Core::Helpers::Time
12
12
 
13
+ # Local storage for given thread waterdrop client references for variants
14
+ ::Fiber.send(:attr_accessor, :waterdrop_clients)
15
+
13
16
  # Which of the inline flow errors do we want to intercept and re-bind
14
17
  SUPPORTED_FLOW_ERRORS = [
15
18
  Rdkafka::RdkafkaError,
16
19
  Rdkafka::Producer::DeliveryHandle::WaitTimeoutError
17
20
  ].freeze
18
21
 
19
- # Empty has to save on memory allocations
22
+ # Empty hash to save on memory allocations
20
23
  EMPTY_HASH = {}.freeze
21
24
 
22
- private_constant :SUPPORTED_FLOW_ERRORS, :EMPTY_HASH
25
+ # Empty array to save on memory allocations
26
+ EMPTY_ARRAY = [].freeze
27
+
28
+ private_constant :SUPPORTED_FLOW_ERRORS, :EMPTY_HASH, :EMPTY_ARRAY
23
29
 
24
30
  def_delegators :config
25
31
 
@@ -269,17 +275,21 @@ module WaterDrop
269
275
  # Waits on a given handler
270
276
  #
271
277
  # @param handler [Rdkafka::Producer::DeliveryHandle]
272
- def wait(handler)
278
+ # @param raise_response_error [Boolean] should we raise the response error after we receive the
279
+ # final result and it is an error.
280
+ def wait(handler, raise_response_error: true)
273
281
  handler.wait(
274
282
  # rdkafka max_wait_timeout is in seconds and we use ms
275
- max_wait_timeout: current_variant.max_wait_timeout / 1_000.0
283
+ max_wait_timeout: current_variant.max_wait_timeout / 1_000.0,
284
+ raise_response_error: raise_response_error
276
285
  )
277
286
  end
278
287
 
279
- # @return [Producer::Context] the variant config. Either custom if built using `#with` or
288
+ # @return [Producer::Variant] the variant config. Either custom if built using `#with` or
280
289
  # a default one.
281
290
  def current_variant
282
- Thread.current[id] || @default_variant
291
+ Fiber.current.waterdrop_clients ||= {}
292
+ Fiber.current.waterdrop_clients[id] || @default_variant
283
293
  end
284
294
 
285
295
  # Runs the client produce method with a given message
@@ -3,5 +3,5 @@
3
3
  # WaterDrop library
4
4
  module WaterDrop
5
5
  # Current WaterDrop version
6
- VERSION = '2.7.4'
6
+ VERSION = '2.8.0'
7
7
  end
data/waterdrop.gemspec CHANGED
@@ -14,19 +14,19 @@ Gem::Specification.new do |spec|
14
14
  spec.homepage = 'https://karafka.io'
15
15
  spec.summary = 'Kafka messaging made easy!'
16
16
  spec.description = spec.summary
17
- spec.license = 'MIT'
17
+ spec.licenses = %w[LGPL-3.0-only Commercial]
18
18
 
19
19
  spec.add_dependency 'karafka-core', '>= 2.4.3', '< 3.0.0'
20
- spec.add_dependency 'karafka-rdkafka', '>= 0.15.1'
20
+ spec.add_dependency 'karafka-rdkafka', '>= 0.17.5'
21
21
  spec.add_dependency 'zeitwerk', '~> 2.3'
22
22
 
23
- spec.required_ruby_version = '>= 3.0.0'
23
+ spec.required_ruby_version = '>= 3.1.0'
24
24
 
25
25
  if $PROGRAM_NAME.end_with?('gem')
26
26
  spec.signing_key = File.expand_path('~/.ssh/gem-private_key.pem')
27
27
  end
28
28
 
29
- spec.cert_chain = %w[certs/cert_chain.pem]
29
+ spec.cert_chain = %w[certs/cert.pem]
30
30
  spec.files = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(spec)/}) }
31
31
  spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
32
32
  spec.require_paths = %w[lib]
data.tar.gz.sig CHANGED
Binary file
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: waterdrop
3
3
  version: !ruby/object:Gem::Version
4
- version: 2.7.4
4
+ version: 2.8.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Maciej Mensfeld
@@ -12,30 +12,30 @@ cert_chain:
12
12
  -----BEGIN CERTIFICATE-----
13
13
  MIIEcDCCAtigAwIBAgIBATANBgkqhkiG9w0BAQsFADA/MRAwDgYDVQQDDAdjb250
14
14
  YWN0MRcwFQYKCZImiZPyLGQBGRYHa2FyYWZrYTESMBAGCgmSJomT8ixkARkWAmlv
15
- MB4XDTIzMDgyMTA3MjU1NFoXDTI0MDgyMDA3MjU1NFowPzEQMA4GA1UEAwwHY29u
15
+ MB4XDTI0MDgyMzEwMTkyMFoXDTQ5MDgxNzEwMTkyMFowPzEQMA4GA1UEAwwHY29u
16
16
  dGFjdDEXMBUGCgmSJomT8ixkARkWB2thcmFma2ExEjAQBgoJkiaJk/IsZAEZFgJp
17
- bzCCAaIwDQYJKoZIhvcNAQEBBQADggGPADCCAYoCggGBAOuZpyQKEwsTG9plLat7
18
- 8bUaNuNBEnouTsNMr6X+XTgvyrAxTuocdsyP1sNCjdS1B8RiiDH1/Nt9qpvlBWon
19
- sdJ1SYhaWNVfqiYStTDnCx3PRMmHRdD4KqUWKpN6VpZ1O/Zu+9Mw0COmvXgZuuO9
20
- wMSJkXRo6dTCfMedLAIxjMeBIxtoLR2e6Jm6MR8+8WYYVWrO9kSOOt5eKQLBY7aK
21
- b/Dc40EcJKPg3Z30Pia1M9ZyRlb6SOj6SKpHRqc7vbVQxjEw6Jjal1lZ49m3YZMd
22
- ArMAs9lQZNdSw5/UX6HWWURLowg6k10RnhTUtYyzO9BFev0JFJftHnmuk8vtb+SD
23
- 5VPmjFXg2VOcw0B7FtG75Vackk8QKfgVe3nSPhVpew2CSPlbJzH80wChbr19+e3+
24
- YGr1tOiaJrL6c+PNmb0F31NXMKpj/r+n15HwlTMRxQrzFcgjBlxf2XFGnPQXHhBm
25
- kp1OFnEq4GG9sON4glRldkwzi/f/fGcZmo5fm3d+0ZdNgwIDAQABo3cwdTAJBgNV
26
- HRMEAjAAMAsGA1UdDwQEAwIEsDAdBgNVHQ4EFgQUPVH5+dLA80A1kJ2Uz5iGwfOa
27
- 1+swHQYDVR0RBBYwFIESY29udGFjdEBrYXJhZmthLmlvMB0GA1UdEgQWMBSBEmNv
28
- bnRhY3RAa2FyYWZrYS5pbzANBgkqhkiG9w0BAQsFAAOCAYEAnpa0jcN7JzREHMTQ
29
- bfZ+xcvlrzuROMY6A3zIZmQgbnoZZNuX4cMRrT1p1HuwXpxdpHPw7dDjYqWw3+1h
30
- 3mXLeMuk7amjQpYoSWU/OIZMhIsARra22UN8qkkUlUj3AwTaChVKN/bPJOM2DzfU
31
- kz9vUgLeYYFfQbZqeI6SsM7ltilRV4W8D9yNUQQvOxCFxtLOetJ00fC/E7zMUzbK
32
- IBwYFQYsbI6XQzgAIPW6nGSYKgRhkfpmquXSNKZRIQ4V6bFrufa+DzD0bt2ZA3ah
33
- fMmJguyb5L2Gf1zpDXzFSPMG7YQFLzwYz1zZZvOU7/UCpQsHpID/YxqDp4+Dgb+Y
34
- qma0whX8UG/gXFV2pYWpYOfpatvahwi+A1TwPQsuZwkkhi1OyF1At3RY+hjSXyav
35
- AnG1dJU+yL2BK7vaVytLTstJME5mepSZ46qqIJXMuWob/YPDmVaBF39TDSG9e34s
36
- msG3BiCqgOgHAnL23+CN3Rt8MsuRfEtoTKpJVcCfoEoNHOkc
17
+ bzCCAaIwDQYJKoZIhvcNAQEBBQADggGPADCCAYoCggGBAKjLhLjQqUlNayxkXnO+
18
+ PsmCDs/KFIzhrsYMfLZRZNaWmzV3ujljMOdDjd4snM2X06C41iVdQPWjpe3j8vVe
19
+ ZXEWR/twSbOP6Eeg8WVH2wCOo0x5i7yhVn4UBLH4JpfEMCbemVcWQ9ry9OMg4WpH
20
+ Uu4dRwxFV7hzCz3p0QfNLRI4miAxnGWcnlD98IJRjBAksTuR1Llj0vbOrDGsL9ZT
21
+ JeXP2gdRLd8SqzAFJEWrbeTBCBU7gfSh3oMg5SVDLjaqf7Kz5wC/8bDZydzanOxB
22
+ T6CDXPsCnllmvTNx2ei2T5rGYJOzJeNTmJLLK6hJWUlAvaQSvCwZRvFJ0tVGLEoS
23
+ flqSr6uGyyl1eMUsNmsH4BqPEYcAV6P2PKTv2vUR8AP0raDvZ3xL1TKvfRb8xRpo
24
+ vPopCGlY5XBWEc6QERHfVLTIVsjnls2/Ujj4h8/TSfqqYnaHKefIMLbuD/tquMjD
25
+ iWQsW2qStBV0T+U7FijKxVfrfqZP7GxQmDAc9o1iiyAa3QIDAQABo3cwdTAJBgNV
26
+ HRMEAjAAMAsGA1UdDwQEAwIEsDAdBgNVHQ4EFgQU3O4dTXmvE7YpAkszGzR9DdL9
27
+ sbEwHQYDVR0RBBYwFIESY29udGFjdEBrYXJhZmthLmlvMB0GA1UdEgQWMBSBEmNv
28
+ bnRhY3RAa2FyYWZrYS5pbzANBgkqhkiG9w0BAQsFAAOCAYEAVKTfoLXn7mqdSxIR
29
+ eqxcR6Huudg1jes81s1+X0uiRTR3hxxKZ3Y82cPsee9zYWyBrN8TA4KA0WILTru7
30
+ Ygxvzha0SRPsSiaKLmgOJ+61ebI4+bOORzIJLpD6GxCxu1r7MI4+0r1u1xe0EWi8
31
+ agkVo1k4Vi8cKMLm6Gl9b3wG9zQBw6fcgKwmpjKiNnOLP+OytzUANrIUJjoq6oal
32
+ TC+f/Uc0TLaRqUaW/bejxzDWWHoM3SU6aoLPuerglzp9zZVzihXwx3jPLUVKDFpF
33
+ Rl2lcBDxlpYGueGo0/oNzGJAAy6js8jhtHC9+19PD53vk7wHtFTZ/0ugDQYnwQ+x
34
+ oml2fAAuVWpTBCgOVFe6XCQpMKopzoxQ1PjKztW2KYxgJdIBX87SnL3aWuBQmhRd
35
+ i9zWxov0mr44TWegTVeypcWGd/0nxu1+QHVNHJrpqlPBRvwQsUm7fwmRInGpcaB8
36
+ ap8wNYvryYzrzvzUxIVFBVM5PacgkFqRmolCa8I7tdKQN+R1
37
37
  -----END CERTIFICATE-----
38
- date: 2024-07-04 00:00:00.000000000 Z
38
+ date: 2024-09-16 00:00:00.000000000 Z
39
39
  dependencies:
40
40
  - !ruby/object:Gem::Dependency
41
41
  name: karafka-core
@@ -63,14 +63,14 @@ dependencies:
63
63
  requirements:
64
64
  - - ">="
65
65
  - !ruby/object:Gem::Version
66
- version: 0.15.1
66
+ version: 0.17.5
67
67
  type: :runtime
68
68
  prerelease: false
69
69
  version_requirements: !ruby/object:Gem::Requirement
70
70
  requirements:
71
71
  - - ">="
72
72
  - !ruby/object:Gem::Version
73
- version: 0.15.1
73
+ version: 0.17.5
74
74
  - !ruby/object:Gem::Dependency
75
75
  name: zeitwerk
76
76
  requirement: !ruby/object:Gem::Requirement
@@ -103,9 +103,9 @@ files:
103
103
  - CHANGELOG.md
104
104
  - Gemfile
105
105
  - Gemfile.lock
106
- - MIT-LICENSE
106
+ - LICENSE
107
107
  - README.md
108
- - certs/cert_chain.pem
108
+ - certs/cert.pem
109
109
  - config/locales/errors.yml
110
110
  - docker-compose.yml
111
111
  - lib/waterdrop.rb
@@ -144,7 +144,8 @@ files:
144
144
  - waterdrop.gemspec
145
145
  homepage: https://karafka.io
146
146
  licenses:
147
- - MIT
147
+ - LGPL-3.0-only
148
+ - Commercial
148
149
  metadata:
149
150
  funding_uri: https://karafka.io/#become-pro
150
151
  homepage_uri: https://karafka.io
@@ -161,14 +162,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
161
162
  requirements:
162
163
  - - ">="
163
164
  - !ruby/object:Gem::Version
164
- version: 3.0.0
165
+ version: 3.1.0
165
166
  required_rubygems_version: !ruby/object:Gem::Requirement
166
167
  requirements:
167
168
  - - ">="
168
169
  - !ruby/object:Gem::Version
169
170
  version: '0'
170
171
  requirements: []
171
- rubygems_version: 3.5.11
172
+ rubygems_version: 3.5.16
172
173
  signing_key:
173
174
  specification_version: 4
174
175
  summary: Kafka messaging made easy!
metadata.gz.sig CHANGED
Binary file
data/MIT-LICENSE DELETED
@@ -1,18 +0,0 @@
1
- Permission is hereby granted, free of charge, to any person obtaining
2
- a copy of this software and associated documentation files (the
3
- "Software"), to deal in the Software without restriction, including
4
- without limitation the rights to use, copy, modify, merge, publish,
5
- distribute, sublicense, and/or sell copies of the Software, and to
6
- permit persons to whom the Software is furnished to do so, subject to
7
- the following conditions:
8
-
9
- The above copyright notice and this permission notice shall be
10
- included in all copies or substantial portions of the Software.
11
-
12
- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
13
- EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
14
- MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
15
- NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
16
- LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
17
- OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
18
- WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/certs/cert_chain.pem DELETED
@@ -1,26 +0,0 @@
1
- -----BEGIN CERTIFICATE-----
2
- MIIEcDCCAtigAwIBAgIBATANBgkqhkiG9w0BAQsFADA/MRAwDgYDVQQDDAdjb250
3
- YWN0MRcwFQYKCZImiZPyLGQBGRYHa2FyYWZrYTESMBAGCgmSJomT8ixkARkWAmlv
4
- MB4XDTIzMDgyMTA3MjU1NFoXDTI0MDgyMDA3MjU1NFowPzEQMA4GA1UEAwwHY29u
5
- dGFjdDEXMBUGCgmSJomT8ixkARkWB2thcmFma2ExEjAQBgoJkiaJk/IsZAEZFgJp
6
- bzCCAaIwDQYJKoZIhvcNAQEBBQADggGPADCCAYoCggGBAOuZpyQKEwsTG9plLat7
7
- 8bUaNuNBEnouTsNMr6X+XTgvyrAxTuocdsyP1sNCjdS1B8RiiDH1/Nt9qpvlBWon
8
- sdJ1SYhaWNVfqiYStTDnCx3PRMmHRdD4KqUWKpN6VpZ1O/Zu+9Mw0COmvXgZuuO9
9
- wMSJkXRo6dTCfMedLAIxjMeBIxtoLR2e6Jm6MR8+8WYYVWrO9kSOOt5eKQLBY7aK
10
- b/Dc40EcJKPg3Z30Pia1M9ZyRlb6SOj6SKpHRqc7vbVQxjEw6Jjal1lZ49m3YZMd
11
- ArMAs9lQZNdSw5/UX6HWWURLowg6k10RnhTUtYyzO9BFev0JFJftHnmuk8vtb+SD
12
- 5VPmjFXg2VOcw0B7FtG75Vackk8QKfgVe3nSPhVpew2CSPlbJzH80wChbr19+e3+
13
- YGr1tOiaJrL6c+PNmb0F31NXMKpj/r+n15HwlTMRxQrzFcgjBlxf2XFGnPQXHhBm
14
- kp1OFnEq4GG9sON4glRldkwzi/f/fGcZmo5fm3d+0ZdNgwIDAQABo3cwdTAJBgNV
15
- HRMEAjAAMAsGA1UdDwQEAwIEsDAdBgNVHQ4EFgQUPVH5+dLA80A1kJ2Uz5iGwfOa
16
- 1+swHQYDVR0RBBYwFIESY29udGFjdEBrYXJhZmthLmlvMB0GA1UdEgQWMBSBEmNv
17
- bnRhY3RAa2FyYWZrYS5pbzANBgkqhkiG9w0BAQsFAAOCAYEAnpa0jcN7JzREHMTQ
18
- bfZ+xcvlrzuROMY6A3zIZmQgbnoZZNuX4cMRrT1p1HuwXpxdpHPw7dDjYqWw3+1h
19
- 3mXLeMuk7amjQpYoSWU/OIZMhIsARra22UN8qkkUlUj3AwTaChVKN/bPJOM2DzfU
20
- kz9vUgLeYYFfQbZqeI6SsM7ltilRV4W8D9yNUQQvOxCFxtLOetJ00fC/E7zMUzbK
21
- IBwYFQYsbI6XQzgAIPW6nGSYKgRhkfpmquXSNKZRIQ4V6bFrufa+DzD0bt2ZA3ah
22
- fMmJguyb5L2Gf1zpDXzFSPMG7YQFLzwYz1zZZvOU7/UCpQsHpID/YxqDp4+Dgb+Y
23
- qma0whX8UG/gXFV2pYWpYOfpatvahwi+A1TwPQsuZwkkhi1OyF1At3RY+hjSXyav
24
- AnG1dJU+yL2BK7vaVytLTstJME5mepSZ46qqIJXMuWob/YPDmVaBF39TDSG9e34s
25
- msG3BiCqgOgHAnL23+CN3Rt8MsuRfEtoTKpJVcCfoEoNHOkc
26
- -----END CERTIFICATE-----