waterdrop 2.6.7 → 2.6.9
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/.github/workflows/ci.yml +16 -9
- data/CHANGELOG.md +21 -8
- data/Gemfile.lock +6 -6
- data/README.md +19 -500
- data/docker-compose.yml +19 -12
- data/lib/waterdrop/clients/buffered.rb +59 -3
- data/lib/waterdrop/clients/dummy.rb +20 -0
- data/lib/waterdrop/clients/rdkafka.rb +6 -1
- data/lib/waterdrop/config.rb +6 -0
- data/lib/waterdrop/contracts/message.rb +4 -1
- data/lib/waterdrop/errors.rb +3 -0
- data/lib/waterdrop/instrumentation/callbacks/delivery.rb +6 -4
- data/lib/waterdrop/instrumentation/logger_listener.rb +26 -1
- data/lib/waterdrop/instrumentation/notifications.rb +6 -0
- data/lib/waterdrop/producer/async.rb +4 -2
- data/lib/waterdrop/producer/sync.rb +4 -2
- data/lib/waterdrop/producer/transactions.rb +170 -0
- data/lib/waterdrop/producer.rb +42 -3
- data/lib/waterdrop/version.rb +1 -1
- data/waterdrop.gemspec +3 -3
- data.tar.gz.sig +0 -0
- metadata +7 -6
- metadata.gz.sig +0 -0
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: a4811e6ec1f3cbea82e4604e95b9cd3deddce9337e9fd935f786bd97d008e914
|
4
|
+
data.tar.gz: baf54ded0ca2e0ec90e659c199138a36932cd5e2f8d6afffbfdb6b770ebbd29c
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 4b2b33cfd43d78ae534c14780accb6193a6ec6ff930168c7c7e2813eb9e2599702c2db923b05793a7366f236e9d51250b07894c7cb11b8f1eca81cb79bafa3b7
|
7
|
+
data.tar.gz: bde20c38e6ebbac05db6085381928516c050512adffd9a4a36a14c80edfed9788b8375edc1e058f38d809cf7862d742f4116fe94efca6fddc5bb0242d4ba216f
|
checksums.yaml.gz.sig
CHANGED
Binary file
|
data/.github/workflows/ci.yml
CHANGED
@@ -16,6 +16,7 @@ jobs:
|
|
16
16
|
fail-fast: false
|
17
17
|
matrix:
|
18
18
|
ruby:
|
19
|
+
- '3.3.0-preview2'
|
19
20
|
- '3.2'
|
20
21
|
- '3.1'
|
21
22
|
- '3.0'
|
@@ -24,29 +25,35 @@ jobs:
|
|
24
25
|
- ruby: '3.2'
|
25
26
|
coverage: 'true'
|
26
27
|
steps:
|
27
|
-
- uses: actions/checkout@
|
28
|
+
- uses: actions/checkout@v4
|
29
|
+
|
28
30
|
- name: Install package dependencies
|
29
31
|
run: "[ -e $APT_DEPS ] || sudo apt-get install -y --no-install-recommends $APT_DEPS"
|
32
|
+
|
30
33
|
- name: Set up Ruby
|
31
34
|
uses: ruby/setup-ruby@v1
|
32
35
|
with:
|
33
36
|
ruby-version: ${{matrix.ruby}}
|
34
|
-
|
35
|
-
|
37
|
+
bundler-cache: true
|
38
|
+
|
36
39
|
- name: Run Kafka with docker-compose
|
37
|
-
# We need to give Kafka enough time to start and create all the needed topics, etc
|
38
|
-
# If anyone has a better idea on how to do it smart and easily, please contact me
|
39
40
|
run: |
|
40
|
-
docker-compose up -d
|
41
|
-
|
41
|
+
docker-compose up -d || (sleep 5 && docker-compose up -d)
|
42
|
+
|
43
|
+
- name: Wait for Kafka
|
44
|
+
run: |
|
45
|
+
sleep 15
|
46
|
+
|
42
47
|
- name: Install latest bundler
|
43
48
|
run: |
|
44
49
|
gem install bundler --no-document
|
45
50
|
bundle config set without 'tools benchmarks docs'
|
51
|
+
|
46
52
|
- name: Bundle install
|
47
53
|
run: |
|
48
54
|
bundle config set without development
|
49
55
|
bundle install --jobs 4 --retry 3
|
56
|
+
|
50
57
|
- name: Run all tests
|
51
58
|
env:
|
52
59
|
GITHUB_COVERAGE: ${{matrix.coverage}}
|
@@ -57,7 +64,7 @@ jobs:
|
|
57
64
|
strategy:
|
58
65
|
fail-fast: false
|
59
66
|
steps:
|
60
|
-
- uses: actions/checkout@
|
67
|
+
- uses: actions/checkout@v4
|
61
68
|
with:
|
62
69
|
fetch-depth: 0
|
63
70
|
- name: Set up Ruby
|
@@ -76,7 +83,7 @@ jobs:
|
|
76
83
|
strategy:
|
77
84
|
fail-fast: false
|
78
85
|
steps:
|
79
|
-
- uses: actions/checkout@
|
86
|
+
- uses: actions/checkout@v4
|
80
87
|
with:
|
81
88
|
fetch-depth: 0
|
82
89
|
- name: Run Coditsu
|
data/CHANGELOG.md
CHANGED
@@ -1,37 +1,50 @@
|
|
1
1
|
# WaterDrop changelog
|
2
2
|
|
3
|
-
|
3
|
+
## 2.6.9 (2023-10-23)
|
4
|
+
- [Improvement] Introduce a `transaction.finished` event to indicate that transaction has finished whether it was aborted or committed.
|
5
|
+
- [Improvement] Use `transaction.committed` event to indicate that transaction has been committed.
|
6
|
+
|
7
|
+
## 2.6.8 (2023-10-20)
|
8
|
+
- **[Feature]** Introduce transactions support.
|
9
|
+
- [Improvement] Expand `LoggerListener` to inform about transactions (info level).
|
10
|
+
- [Improvement] Allow waterdrop to use topic as a symbol or a string.
|
11
|
+
- [Improvement] Enhance both `message.acknowledged` and `error.occurred` (for `librdkafka.dispatch_error`) with full delivery_report.
|
12
|
+
- [Improvement] Provide `#close!` that will force producer close even with outgoing data after the ma wait timeout.
|
13
|
+
- [Improvement] Provide `#purge` that will purge any outgoing data and data from the internal queues (both WaterDrop and librdkafka).
|
14
|
+
- [Fix] Fix the `librdkafka.dispatch_error` error dispatch for errors with negative code.
|
15
|
+
|
16
|
+
## 2.6.7 (2023-09-01)
|
4
17
|
- [Improvement] early flush data from `librdkafka` internal buffer before closing.
|
5
18
|
- [Maintenance] Update the signing cert as the old one expired.
|
6
19
|
|
7
|
-
|
20
|
+
## 2.6.6 (2023-08-03)
|
8
21
|
- [Improvement] Provide `log_messages` option to `LoggerListener` so the extensive messages data logging can disabled.
|
9
22
|
|
10
|
-
|
23
|
+
## 2.6.5 (2023-07-22)
|
11
24
|
- [Fix] Add cause to the errors that are passed into instrumentation (konalegi)
|
12
25
|
|
13
|
-
|
26
|
+
## 2.6.4 (2023-07-11)
|
14
27
|
- [Improvement] Use original error `#inspect` for `WaterDrop::Errors::ProduceError` and `WaterDrop::Errors::ProduceManyError` instead of the current empty string.
|
15
28
|
|
16
|
-
|
29
|
+
## 2.6.3 (2023-06-28)
|
17
30
|
- [Change] Use `Concurrent::AtomicFixnum` to track operations in progress to prevent potential race conditions on JRuby and TruffleRuby (not yet supported but this is for future usage).
|
18
31
|
- [Change] Require `karafka-rdkafka` `>= 0.13.2`.
|
19
32
|
- [Change] Require 'karafka-core' `>= 2.1.1`
|
20
33
|
|
21
|
-
|
34
|
+
## 2.6.2 (2023-06-21)
|
22
35
|
- [Refactor] Introduce a counter-based locking approach to make sure, that we close the producer safely but at the same time not to limit messages production with producing lock.
|
23
36
|
- [Refactor] Make private methods private.
|
24
37
|
- [Refactor] Validate that producer is not closed only when attempting to produce.
|
25
38
|
- [Refactor] Improve one 5 minute long spec to run in 10 seconds.
|
26
39
|
- [Refactor] clear client assignment after closing.
|
27
40
|
|
28
|
-
|
41
|
+
## 2.6.1 (2023-06-19)
|
29
42
|
- [Refactor] Remove no longer needed patches.
|
30
43
|
- [Fix] Fork detection on a short lived processes seems to fail. Clear the used parent process client reference not to close it in the finalizer (#356).
|
31
44
|
- [Change] Require `karafka-rdkafka` `>= 0.13.0`.
|
32
45
|
- [Change] Require 'karafka-core' `>= 2.1.0`
|
33
46
|
|
34
|
-
|
47
|
+
## 2.6.0 (2023-06-11)
|
35
48
|
- [Improvement] Introduce `client_class` setting for ability to replace underlying client with anything specific to a given env (dev, test, etc).
|
36
49
|
- [Improvement] Introduce `Clients::Buffered` useful for writing specs that do not have to talk with Kafka (id-ilych)
|
37
50
|
- [Improvement] Make `#produce` method private to avoid confusion and make sure it is not used directly (it is not part of the official API).
|
data/Gemfile.lock
CHANGED
@@ -1,8 +1,8 @@
|
|
1
1
|
PATH
|
2
2
|
remote: .
|
3
3
|
specs:
|
4
|
-
waterdrop (2.6.
|
5
|
-
karafka-core (>= 2.
|
4
|
+
waterdrop (2.6.9)
|
5
|
+
karafka-core (>= 2.2.3, < 3.0.0)
|
6
6
|
zeitwerk (~> 2.3)
|
7
7
|
|
8
8
|
GEM
|
@@ -17,15 +17,15 @@ GEM
|
|
17
17
|
concurrent-ruby (1.2.2)
|
18
18
|
diff-lcs (1.5.0)
|
19
19
|
docile (1.4.0)
|
20
|
-
factory_bot (6.
|
20
|
+
factory_bot (6.3.0)
|
21
21
|
activesupport (>= 5.0.0)
|
22
22
|
ffi (1.15.5)
|
23
23
|
i18n (1.14.1)
|
24
24
|
concurrent-ruby (~> 1.0)
|
25
|
-
karafka-core (2.
|
25
|
+
karafka-core (2.2.3)
|
26
26
|
concurrent-ruby (>= 1.1)
|
27
|
-
karafka-rdkafka (>= 0.13.
|
28
|
-
karafka-rdkafka (0.13.
|
27
|
+
karafka-rdkafka (>= 0.13.6, < 0.14.0)
|
28
|
+
karafka-rdkafka (0.13.6)
|
29
29
|
ffi (~> 1.15)
|
30
30
|
mini_portile2 (~> 2.6)
|
31
31
|
rake (> 12)
|