rdkafka 0.14.0.rc1 → 0.14.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/CHANGELOG.md +96 -92
- data/dist/librdkafka_2.2.0.tar.gz +0 -0
- data/ext/Rakefile +53 -26
- data/lib/rdkafka/abstract_handle.rb +3 -2
- data/lib/rdkafka/version.rb +1 -1
- data.tar.gz.sig +0 -0
- metadata +6 -5
- metadata.gz.sig +0 -0
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 4155932560c141d7110501ec00ac51d49311130335538deb684c670bcc0e4905
|
|
4
|
+
data.tar.gz: 9ab67e523adcbf68fd03b755a2384d790f46591c140fd1b545bf974be5cb5c3d
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 26a68921d4fc49dbfbcbc36b23d19c62f3de04f0ba5c8da488d9cc1bc1de2800ade788dcaa8bff381e83e7f9f5fbb3726aeb799f8a0ad703cec9fde3b0ed7e64
|
|
7
|
+
data.tar.gz: 266634c85162b6a0f46137e4f2080ac8eae470a46a10b13ee555c5184932ccb218e0dbbd91a250df2e0e7517f70723832002a4266bade55a75040c39fb3a971f
|
checksums.yaml.gz.sig
CHANGED
|
Binary file
|
data/CHANGELOG.md
CHANGED
|
@@ -1,140 +1,144 @@
|
|
|
1
1
|
# Rdkafka Changelog
|
|
2
2
|
|
|
3
|
-
## 0.14.
|
|
3
|
+
## 0.14.1 (2024-07-10)
|
|
4
|
+
- [Fix] Switch to local release of librdkafka to mitigate its unavailability.
|
|
5
|
+
|
|
6
|
+
## 0.14.0 (2023-11-21)
|
|
7
|
+
- [Enhancement] Add `raise_response_error` flag to the `Rdkafka::AbstractHandle`.
|
|
4
8
|
- [Enhancement] Allow for setting `statistics_callback` as nil to reset predefined settings configured by a different gem (mensfeld)
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
9
|
+
- [Enhancement] Get consumer position (thijsc & mensfeld)
|
|
10
|
+
- [Enhancement] Provide `#purge` to remove any outstanding requests from the producer (mensfeld)
|
|
11
|
+
- [Enhancement] Update `librdkafka` to `2.2.0` (mensfeld)
|
|
12
|
+
- [Enhancement] Introduce producer partitions count metadata cache (mensfeld)
|
|
13
|
+
- [Enhancement] Increase metadata timeout request from `250 ms` to `2000 ms` default to allow for remote cluster operations via `rdkafka-ruby` (mensfeld)
|
|
14
|
+
- [Enhancement] Introduce `#name` for producers and consumers (mensfeld)
|
|
15
|
+
- [Enhancement] Include backtrace in non-raised binded errors (mensfeld)
|
|
16
|
+
- [Fix] Reference to Opaque is not released when Admin, Consumer or Producer is closed (mensfeld)
|
|
17
|
+
- [Fix] Trigger `#poll` on native kafka creation to handle oauthbearer cb (mensfeld)
|
|
18
|
+
- [Fix] `#flush` does not handle the timeouts errors by making it return `true` if all flushed or `false` if failed. We do **not** raise an exception here to keep it backwards compatible (mensfeld)
|
|
19
|
+
- [Change] Remove support for Ruby 2.6 due to it being EOL and WeakMap incompatibilities (mensfeld)
|
|
20
|
+
- [Change] Update Kafka Docker with Confluent KRaft (mensfeld)
|
|
21
|
+
- [Change] Update librdkafka repo reference from edenhill to confluentinc (mensfeld)
|
|
18
22
|
|
|
19
23
|
## 0.13.0 (2023-07-24)
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
24
|
+
- Support cooperative sticky partition assignment in the rebalance callback (methodmissing)
|
|
25
|
+
- Support both string and symbol header keys (ColinDKelley)
|
|
26
|
+
- Handle tombstone messages properly (kgalieva)
|
|
27
|
+
- Add topic name to delivery report (maeve)
|
|
28
|
+
- Allow string partitioner config (mollyegibson)
|
|
29
|
+
- Fix documented type for DeliveryReport#error (jimmydo)
|
|
30
|
+
- Bump librdkafka to 2.0.2 (lmaia)
|
|
31
|
+
- Use finalizers to cleanly exit producer and admin (thijsc)
|
|
32
|
+
- Lock access to the native kafka client (thijsc)
|
|
33
|
+
- Fix potential race condition in multi-threaded producer (mensfeld)
|
|
34
|
+
- Fix leaking FFI resources in specs (mensfeld)
|
|
35
|
+
- Improve specs stability (mensfeld)
|
|
36
|
+
- Make metadata request timeout configurable (mensfeld)
|
|
37
|
+
- call_on_partitions_assigned and call_on_partitions_revoked only get a tpl passed in (thijsc)
|
|
34
38
|
|
|
35
39
|
## 0.12.0 (2022-06-17)
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
40
|
+
- Bumps librdkafka to 1.9.0
|
|
41
|
+
- Fix crash on empty partition key (mensfeld)
|
|
42
|
+
- Pass the delivery handle to the callback (gvisokinskas)
|
|
39
43
|
|
|
40
44
|
## 0.11.0 (2021-11-17)
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
45
|
+
- Upgrade librdkafka to 1.8.2
|
|
46
|
+
- Bump supported minimum Ruby version to 2.6
|
|
47
|
+
- Better homebrew path detection
|
|
44
48
|
|
|
45
49
|
## 0.10.0 (2021-09-07)
|
|
46
|
-
|
|
47
|
-
|
|
50
|
+
- Upgrade librdkafka to 1.5.0
|
|
51
|
+
- Add error callback config
|
|
48
52
|
|
|
49
53
|
## 0.9.0 (2021-06-23)
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
54
|
+
- Fixes for Ruby 3.0
|
|
55
|
+
- Allow any callable object for callbacks (gremerritt)
|
|
56
|
+
- Reduce memory allocations in Rdkafka::Producer#produce (jturkel)
|
|
57
|
+
- Use queue as log callback to avoid unsafe calls from trap context (breunigs)
|
|
58
|
+
- Allow passing in topic configuration on create_topic (dezka)
|
|
59
|
+
- Add each_batch method to consumer (mgrosso)
|
|
56
60
|
|
|
57
61
|
## 0.8.1 (2020-12-07)
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
+
- Fix topic_flag behaviour and add tests for Metadata (geoff2k)
|
|
63
|
+
- Add topic admin interface (geoff2k)
|
|
64
|
+
- Raise an exception if @native_kafka is nil (geoff2k)
|
|
65
|
+
- Option to use zstd compression (jasonmartens)
|
|
62
66
|
|
|
63
67
|
## 0.8.0 (2020-06-02)
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
68
|
+
- Upgrade librdkafka to 1.4.0
|
|
69
|
+
- Integrate librdkafka metadata API and add partition_key (by Adithya-copart)
|
|
70
|
+
- Ruby 2.7 compatibility fix (by Geoff Thé)A
|
|
71
|
+
- Add error to delivery report (by Alex Stanovsky)
|
|
72
|
+
- Don't override CPPFLAGS and LDFLAGS if already set on Mac (by Hiroshi Hatake)
|
|
73
|
+
- Allow use of Rake 13.x and up (by Tomasz Pajor)
|
|
70
74
|
|
|
71
75
|
## 0.7.0 (2019-09-21)
|
|
72
|
-
|
|
73
|
-
|
|
76
|
+
- Bump librdkafka to 1.2.0 (by rob-as)
|
|
77
|
+
- Allow customizing the wait time for delivery report availability (by mensfeld)
|
|
74
78
|
|
|
75
79
|
## 0.6.0 (2019-07-23)
|
|
76
|
-
|
|
77
|
-
|
|
80
|
+
- Bump librdkafka to 1.1.0 (by Chris Gaffney)
|
|
81
|
+
- Implement seek (by breunigs)
|
|
78
82
|
|
|
79
83
|
## 0.5.0 (2019-04-11)
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
84
|
+
- Bump librdkafka to 1.0.0 (by breunigs)
|
|
85
|
+
- Add cluster and member information (by dmexe)
|
|
86
|
+
- Support message headers for consumer & producer (by dmexe)
|
|
87
|
+
- Add consumer rebalance listener (by dmexe)
|
|
88
|
+
- Implement pause/resume partitions (by dmexe)
|
|
85
89
|
|
|
86
90
|
## 0.4.2 (2019-01-12)
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
91
|
+
- Delivery callback for producer
|
|
92
|
+
- Document list param of commit method
|
|
93
|
+
- Use default Homebrew openssl location if present
|
|
94
|
+
- Consumer lag handles empty topics
|
|
95
|
+
- End iteration in consumer when it is closed
|
|
96
|
+
- Add support for storing message offsets
|
|
97
|
+
- Add missing runtime dependency to rake
|
|
94
98
|
|
|
95
99
|
## 0.4.1 (2018-10-19)
|
|
96
|
-
|
|
100
|
+
- Bump librdkafka to 0.11.6
|
|
97
101
|
|
|
98
102
|
## 0.4.0 (2018-09-24)
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
103
|
+
- Improvements in librdkafka archive download
|
|
104
|
+
- Add global statistics callback
|
|
105
|
+
- Use Time for timestamps, potentially breaking change if you
|
|
102
106
|
rely on the previous behavior where it returns an integer with
|
|
103
107
|
the number of milliseconds.
|
|
104
|
-
|
|
105
|
-
|
|
108
|
+
- Bump librdkafka to 0.11.5
|
|
109
|
+
- Implement TopicPartitionList in Ruby so we don't have to keep
|
|
106
110
|
track of native objects.
|
|
107
|
-
|
|
108
|
-
|
|
111
|
+
- Support committing a topic partition list
|
|
112
|
+
- Add consumer assignment method
|
|
109
113
|
|
|
110
114
|
## 0.3.5 (2018-01-17)
|
|
111
|
-
|
|
112
|
-
|
|
115
|
+
- Fix crash when not waiting for delivery handles
|
|
116
|
+
- Run specs on Ruby 2.5
|
|
113
117
|
|
|
114
118
|
## 0.3.4 (2017-12-05)
|
|
115
|
-
|
|
119
|
+
- Bump librdkafka to 0.11.3
|
|
116
120
|
|
|
117
121
|
## 0.3.3 (2017-10-27)
|
|
118
|
-
|
|
122
|
+
- Fix bug that prevent display of `RdkafkaError` message
|
|
119
123
|
|
|
120
124
|
## 0.3.2 (2017-10-25)
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
+
- `add_topic` now supports using a partition count
|
|
126
|
+
- Add way to make errors clearer with an extra message
|
|
127
|
+
- Show topics in subscribe error message
|
|
128
|
+
- Show partition and topic in query watermark offsets error message
|
|
125
129
|
|
|
126
130
|
## 0.3.1 (2017-10-23)
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
131
|
+
- Bump librdkafka to 0.11.1
|
|
132
|
+
- Officially support ranges in `add_topic` for topic partition list.
|
|
133
|
+
- Add consumer lag calculator
|
|
130
134
|
|
|
131
135
|
## 0.3.0 (2017-10-17)
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
136
|
+
- Move both add topic methods to one `add_topic` in `TopicPartitionList`
|
|
137
|
+
- Add committed offsets to consumer
|
|
138
|
+
- Add query watermark offset to consumer
|
|
135
139
|
|
|
136
140
|
## 0.2.0 (2017-10-13)
|
|
137
|
-
|
|
141
|
+
- Some refactoring and add inline documentation
|
|
138
142
|
|
|
139
143
|
## 0.1.x (2017-09-10)
|
|
140
|
-
|
|
144
|
+
- Initial working version including producing and consuming
|
|
Binary file
|
data/ext/Rakefile
CHANGED
|
@@ -1,40 +1,67 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require File.expand_path('../../lib/rdkafka/version', __FILE__)
|
|
4
|
-
require "mini_portile2"
|
|
5
4
|
require "fileutils"
|
|
6
5
|
require "open-uri"
|
|
7
6
|
|
|
8
7
|
task :default => :clean do
|
|
9
|
-
#
|
|
10
|
-
|
|
8
|
+
# For nix users, nix can't locate the file paths because the packages it's requiring aren't managed by the system but are
|
|
9
|
+
# managed by nix itself, so using the normal file paths doesn't work for nix users.
|
|
10
|
+
#
|
|
11
|
+
# Mini_portile causes an issue because it's dependencies are downloaded on the fly and therefore don't exist/aren't
|
|
12
|
+
# accessible in the nix environment
|
|
13
|
+
if ENV.fetch('RDKAFKA_EXT_PATH', '').empty?
|
|
14
|
+
# Download and compile librdkafka if RDKAFKA_EXT_PATH is not set
|
|
15
|
+
require "mini_portile2"
|
|
16
|
+
recipe = MiniPortile.new("librdkafka", Rdkafka::LIBRDKAFKA_VERSION)
|
|
11
17
|
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
+
# Use default homebrew openssl if we're on mac and the directory exists
|
|
19
|
+
# and each of flags is not empty
|
|
20
|
+
if recipe.host&.include?("darwin") && system("which brew &> /dev/null") && Dir.exist?("#{homebrew_prefix = %x(brew --prefix openssl).strip}")
|
|
21
|
+
ENV["CPPFLAGS"] = "-I#{homebrew_prefix}/include" unless ENV["CPPFLAGS"]
|
|
22
|
+
ENV["LDFLAGS"] = "-L#{homebrew_prefix}/lib" unless ENV["LDFLAGS"]
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
releases = File.expand_path(File.join(File.dirname(__FILE__), '../dist'))
|
|
26
|
+
|
|
27
|
+
recipe.files << {
|
|
28
|
+
:url => "file://#{releases}/librdkafka_#{Rdkafka::LIBRDKAFKA_VERSION}.tar.gz",
|
|
29
|
+
:sha256 => Rdkafka::LIBRDKAFKA_SOURCE_SHA256
|
|
30
|
+
}
|
|
31
|
+
recipe.configure_options = ["--host=#{recipe.host}"]
|
|
32
|
+
|
|
33
|
+
# Disable using libc regex engine in favor of the embedded one
|
|
34
|
+
# The default regex engine of librdkafka does not always work exactly as most of the users
|
|
35
|
+
# would expect, hence this flag allows for changing it to the other one
|
|
36
|
+
if ENV.key?('RDKAFKA_DISABLE_REGEX_EXT')
|
|
37
|
+
recipe.configure_options << '--disable-regex-ext'
|
|
38
|
+
end
|
|
18
39
|
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
40
|
+
recipe.cook
|
|
41
|
+
# Move dynamic library we're interested in
|
|
42
|
+
if recipe.host.include?('darwin')
|
|
43
|
+
from_extension = '1.dylib'
|
|
44
|
+
to_extension = 'dylib'
|
|
45
|
+
else
|
|
46
|
+
from_extension = 'so.1'
|
|
47
|
+
to_extension = 'so'
|
|
48
|
+
end
|
|
49
|
+
lib_path = File.join(File.dirname(__FILE__), "ports/#{recipe.host}/librdkafka/#{Rdkafka::LIBRDKAFKA_VERSION}/lib/librdkafka.#{from_extension}")
|
|
50
|
+
FileUtils.mv(lib_path, File.join(File.dirname(__FILE__), "librdkafka.#{to_extension}"))
|
|
51
|
+
# Cleanup files created by miniportile we don't need in the gem
|
|
52
|
+
FileUtils.rm_rf File.join(File.dirname(__FILE__), "tmp")
|
|
53
|
+
FileUtils.rm_rf File.join(File.dirname(__FILE__), "ports")
|
|
29
54
|
else
|
|
30
|
-
|
|
31
|
-
|
|
55
|
+
# Otherwise, copy existing libraries to ./ext
|
|
56
|
+
if ENV['RDKAFKA_EXT_PATH'].nil? || ENV['RDKAFKA_EXT_PATH'].empty?
|
|
57
|
+
raise "RDKAFKA_EXT_PATH must be set in your nix config when running under nix"
|
|
58
|
+
end
|
|
59
|
+
files = [
|
|
60
|
+
File.join(ENV['RDKAFKA_EXT_PATH'], 'lib', 'librdkafka.dylib'),
|
|
61
|
+
File.join(ENV['RDKAFKA_EXT_PATH'], 'lib', 'librdkafka.so')
|
|
62
|
+
]
|
|
63
|
+
files.each { |ext| FileUtils.cp(ext, File.dirname(__FILE__)) if File.exist?(ext) }
|
|
32
64
|
end
|
|
33
|
-
lib_path = File.join(File.dirname(__FILE__), "ports/#{recipe.host}/librdkafka/#{Rdkafka::LIBRDKAFKA_VERSION}/lib/librdkafka.#{from_extension}")
|
|
34
|
-
FileUtils.mv(lib_path, File.join(File.dirname(__FILE__), "librdkafka.#{to_extension}"))
|
|
35
|
-
# Cleanup files created by miniportile we don't need in the gem
|
|
36
|
-
FileUtils.rm_rf File.join(File.dirname(__FILE__), "tmp")
|
|
37
|
-
FileUtils.rm_rf File.join(File.dirname(__FILE__), "ports")
|
|
38
65
|
end
|
|
39
66
|
|
|
40
67
|
task :clean do
|
|
@@ -48,12 +48,13 @@ module Rdkafka
|
|
|
48
48
|
# If this is nil it does not time out.
|
|
49
49
|
# @param wait_timeout [Numeric] Amount of time we should wait before we recheck if the
|
|
50
50
|
# operation has completed
|
|
51
|
+
# @param raise_response_error [Boolean] should we raise error when waiting finishes
|
|
51
52
|
#
|
|
52
53
|
# @return [Object] Operation-specific result
|
|
53
54
|
#
|
|
54
55
|
# @raise [RdkafkaError] When the operation failed
|
|
55
56
|
# @raise [WaitTimeoutError] When the timeout has been reached and the handle is still pending
|
|
56
|
-
def wait(max_wait_timeout: 60, wait_timeout: 0.1)
|
|
57
|
+
def wait(max_wait_timeout: 60, wait_timeout: 0.1, raise_response_error: true)
|
|
57
58
|
timeout = if max_wait_timeout
|
|
58
59
|
monotonic_now + max_wait_timeout
|
|
59
60
|
else
|
|
@@ -67,7 +68,7 @@ module Rdkafka
|
|
|
67
68
|
)
|
|
68
69
|
end
|
|
69
70
|
sleep wait_timeout
|
|
70
|
-
elsif self[:response] != 0
|
|
71
|
+
elsif self[:response] != 0 && raise_response_error
|
|
71
72
|
raise_error
|
|
72
73
|
else
|
|
73
74
|
return create_result
|
data/lib/rdkafka/version.rb
CHANGED
data.tar.gz.sig
CHANGED
|
Binary file
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: rdkafka
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.14.
|
|
4
|
+
version: 0.14.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Thijs Cadier
|
|
@@ -35,7 +35,7 @@ cert_chain:
|
|
|
35
35
|
AnG1dJU+yL2BK7vaVytLTstJME5mepSZ46qqIJXMuWob/YPDmVaBF39TDSG9e34s
|
|
36
36
|
msG3BiCqgOgHAnL23+CN3Rt8MsuRfEtoTKpJVcCfoEoNHOkc
|
|
37
37
|
-----END CERTIFICATE-----
|
|
38
|
-
date:
|
|
38
|
+
date: 2024-07-10 00:00:00.000000000 Z
|
|
39
39
|
dependencies:
|
|
40
40
|
- !ruby/object:Gem::Dependency
|
|
41
41
|
name: ffi
|
|
@@ -184,6 +184,7 @@ files:
|
|
|
184
184
|
- README.md
|
|
185
185
|
- Rakefile
|
|
186
186
|
- certs/cert_chain.pem
|
|
187
|
+
- dist/librdkafka_2.2.0.tar.gz
|
|
187
188
|
- docker-compose.yml
|
|
188
189
|
- ext/README.md
|
|
189
190
|
- ext/Rakefile
|
|
@@ -255,11 +256,11 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
|
255
256
|
version: '2.7'
|
|
256
257
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
257
258
|
requirements:
|
|
258
|
-
- - "
|
|
259
|
+
- - ">="
|
|
259
260
|
- !ruby/object:Gem::Version
|
|
260
|
-
version:
|
|
261
|
+
version: '0'
|
|
261
262
|
requirements: []
|
|
262
|
-
rubygems_version: 3.
|
|
263
|
+
rubygems_version: 3.5.14
|
|
263
264
|
signing_key:
|
|
264
265
|
specification_version: 4
|
|
265
266
|
summary: The rdkafka gem is a modern Kafka client library for Ruby based on librdkafka.
|
metadata.gz.sig
CHANGED
|
Binary file
|