karafka-testing 2.0.0 → 2.0.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/CHANGELOG.md +9 -0
- data/Gemfile.lock +7 -7
- data/README.md +51 -17
- data/certs/cert_chain.pem +26 -0
- data/karafka-testing.gemspec +4 -4
- data/lib/karafka/testing/errors.rb +4 -0
- data/lib/karafka/testing/rspec/helpers.rb +96 -39
- data/lib/karafka/testing/rspec/proxy.rb +11 -5
- data/lib/karafka/testing/{dummy_client.rb → spec_consumer_client.rb} +2 -2
- data/lib/karafka/testing/spec_producer_client.rb +43 -0
- data/lib/karafka/testing/version.rb +1 -1
- data.tar.gz.sig +0 -0
- metadata +33 -31
- metadata.gz.sig +0 -0
- data/certs/mensfeld.pem +0 -25
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 9112bdf5ca189a7f448c2b5768aa2694a5958a96f591f8d8bad8cc204708f1a3
|
4
|
+
data.tar.gz: 61ae2749b2bac651143bca2b8cdb470900b87eccc058c13cbaa4644e1be1435e
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: adff2729f366403cc7e01a18135f972b05b9dd9faa8408a8cda7f75cf6425c9e676060815037fc78acaecfc4efa37be122e7b8d15623c6c31b7263fb2856bdc1
|
7
|
+
data.tar.gz: 29da133e08e85e5d1efa24156c1eab91aa858387bc341070cc4ae059d073e721e6955808f6b6923b39b102a3e713067e6cf856a889d0d9d5568674a27245926e
|
checksums.yaml.gz.sig
CHANGED
Binary file
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,14 @@
|
|
1
1
|
# Karafka Test gem changelog
|
2
2
|
|
3
|
+
## 2.0.2 (2022-09-29)
|
4
|
+
- Provide multi-consumer group testing support (#92)
|
5
|
+
- Fail fast if requested topic is present in multiple consumer groups but consumer group is not specified.
|
6
|
+
- Allow for usage of `Karafka.producer` directly and make it buffered.
|
7
|
+
- Rename `karafka.publish` to `karafka.produce` to align naming conventions.
|
8
|
+
|
9
|
+
## 2.0.1 (2022-08-05)
|
10
|
+
- Require non rc version of Karafka.
|
11
|
+
|
3
12
|
## 2.0.0 (2022-08-01)
|
4
13
|
- No changes. Just non-rc release.
|
5
14
|
|
data/Gemfile.lock
CHANGED
@@ -1,19 +1,19 @@
|
|
1
1
|
PATH
|
2
2
|
remote: .
|
3
3
|
specs:
|
4
|
-
karafka-testing (2.0.
|
5
|
-
karafka (>= 2.0
|
4
|
+
karafka-testing (2.0.2)
|
5
|
+
karafka (>= 2.0, < 3.0.0)
|
6
6
|
|
7
7
|
GEM
|
8
8
|
remote: https://rubygems.org/
|
9
9
|
specs:
|
10
10
|
concurrent-ruby (1.1.10)
|
11
11
|
ffi (1.15.5)
|
12
|
-
karafka (2.0.
|
13
|
-
karafka-core (>= 2.0.
|
14
|
-
rdkafka (>= 0.
|
12
|
+
karafka (2.0.10)
|
13
|
+
karafka-core (>= 2.0.2, < 3.0.0)
|
14
|
+
rdkafka (>= 0.12)
|
15
15
|
thor (>= 0.20)
|
16
|
-
waterdrop (>= 2.4.
|
16
|
+
waterdrop (>= 2.4.1, < 3.0.0)
|
17
17
|
zeitwerk (~> 2.3)
|
18
18
|
karafka-core (2.0.2)
|
19
19
|
concurrent-ruby (>= 1.1)
|
@@ -37,4 +37,4 @@ DEPENDENCIES
|
|
37
37
|
karafka-testing!
|
38
38
|
|
39
39
|
BUNDLED WITH
|
40
|
-
2.3.
|
40
|
+
2.3.22
|
data/README.md
CHANGED
@@ -1,14 +1,10 @@
|
|
1
1
|
# Karafka Testing library
|
2
2
|
|
3
|
-
**Note**: Documentation presented below works with Karafka `2.0.0.alpha3` and higher.
|
4
|
-
|
5
|
-
Please refer to [this](https://github.com/karafka/testing/tree/1.4) branch and its documentation for details about usage with Karafka `1.4`.
|
6
|
-
|
7
3
|
[![Build Status](https://github.com/karafka/karafka-testing/workflows/ci/badge.svg)](https://github.com/karafka/karafka-testing/actions?query=workflow%3Aci)
|
8
4
|
[![Gem Version](https://badge.fury.io/rb/karafka-testing.svg)](http://badge.fury.io/rb/karafka-testing)
|
9
5
|
[![Join the chat at https://slack.karafka.io](https://raw.githubusercontent.com/karafka/misc/master/slack.svg)](https://slack.karafka.io)
|
10
6
|
|
11
|
-
Karafka-Testing is a library that provides RSpec helpers, to make testing of Karafka consumers much easier.
|
7
|
+
Karafka-Testing is a library that provides RSpec helpers, to make testing of Karafka consumers and producer much easier.
|
12
8
|
|
13
9
|
## Installation
|
14
10
|
|
@@ -32,17 +28,22 @@ end
|
|
32
28
|
|
33
29
|
## Usage
|
34
30
|
|
35
|
-
Once included
|
31
|
+
Once included in your RSpec setup, this library will provide you with a special `#karafka` object that contains three methods that you can use within your specs:
|
36
32
|
|
37
33
|
- `#consumer_for` - creates a consumer instance for the desired topic. It **needs** to be set as the spec subject.
|
38
|
-
- `#
|
34
|
+
- `#produce` - "sends" message to the consumer instance.
|
35
|
+
- `#produced_messages` - contains all the messages "sent" to Kafka during spec execution.
|
36
|
+
|
37
|
+
**Note:** Messages sent using the `#produce` method and directly from `Karafka.producer` won't be sent to Kafka. They will be buffered and accessible in a per-spec buffer in case you want to test messages production.
|
39
38
|
|
40
|
-
|
39
|
+
Messages that target the topic built using the `karafka#consumer_for` method will additionally be delivered to the consumer you want to test.
|
40
|
+
|
41
|
+
### Testing messages consumption (consumers)
|
41
42
|
|
42
43
|
```ruby
|
43
44
|
RSpec.describe InlineBatchConsumer do
|
44
45
|
# This will create a consumer instance with all the settings defined for the given topic
|
45
|
-
subject(:consumer) { karafka.consumer_for(
|
46
|
+
subject(:consumer) { karafka.consumer_for('inline_batch_data') }
|
46
47
|
|
47
48
|
let(:nr1_value) { rand }
|
48
49
|
let(:nr2_value) { rand }
|
@@ -50,9 +51,11 @@ RSpec.describe InlineBatchConsumer do
|
|
50
51
|
|
51
52
|
before do
|
52
53
|
# Sends first message to Karafka consumer
|
53
|
-
karafka.
|
54
|
+
karafka.produce({ 'number' => nr1_value }.to_json)
|
55
|
+
|
54
56
|
# Sends second message to Karafka consumer
|
55
|
-
karafka.
|
57
|
+
karafka.produce({ 'number' => nr2_value }.to_json, partition: 2)
|
58
|
+
|
56
59
|
allow(Karafka.logger).to receive(:info)
|
57
60
|
end
|
58
61
|
|
@@ -63,22 +66,53 @@ RSpec.describe InlineBatchConsumer do
|
|
63
66
|
end
|
64
67
|
```
|
65
68
|
|
66
|
-
If your consumers use `producer` to dispatch messages, you can
|
69
|
+
If your consumers use `producer` to dispatch messages, you can check its operations as well:
|
67
70
|
|
68
71
|
```ruby
|
69
72
|
RSpec.describe InlineBatchConsumer do
|
70
73
|
subject(:consumer) { karafka.consumer_for(:inline_batch_data) }
|
71
74
|
|
72
|
-
before { karafka.
|
75
|
+
before { karafka.produce({ 'number' => 1 }.to_json) }
|
73
76
|
|
74
77
|
it 'expects to dispatch async message to messages topic with value bigger by 1' do
|
75
|
-
expect(consumer.producer)
|
76
|
-
.to receive(:produce_async)
|
77
|
-
.with(topic: 'messages', payload: { number: 2 }.to_json)
|
78
|
-
|
79
78
|
consumer.consume
|
79
|
+
|
80
|
+
expect(karafka.produced_messages.last.payload).to eq({ number: 2 }.to_json)
|
81
|
+
end
|
82
|
+
end
|
83
|
+
```
|
84
|
+
|
85
|
+
### Testing messages production (producer)
|
86
|
+
|
87
|
+
When running RSpec, Karafka will not dispatch messages to Kafka using `Karafka.producer` but will buffer them internally.
|
88
|
+
|
89
|
+
This means you can check your application flow, making sure your logic acts as expected:
|
90
|
+
|
91
|
+
```ruby
|
92
|
+
# Example class in which there is a message production
|
93
|
+
class UsersBuilder
|
94
|
+
def create(user_details)
|
95
|
+
user = ::User.create!(user_details)
|
96
|
+
|
97
|
+
Karafka.producer.produce_sync(
|
98
|
+
topic: 'users_changes',
|
99
|
+
payload: { user_id: user.id, type: 'user.created' },
|
100
|
+
key: user.id.to_s
|
101
|
+
)
|
102
|
+
|
103
|
+
user
|
80
104
|
end
|
81
105
|
end
|
106
|
+
|
107
|
+
RSpec.describe InlineBatchConsumer do
|
108
|
+
let(:created_user) { UsersBuilder.new.create(user_details) }
|
109
|
+
|
110
|
+
before { created_user }
|
111
|
+
|
112
|
+
it { expect(karafka.produced_messages.size).to eq(1) }
|
113
|
+
it { expect(karafka.produced_messages.first[:topic]).to eq('user.created') }
|
114
|
+
it { expect(karafka.produced_messages.first[:key]).to eq(created_user.id.to_s) }
|
115
|
+
end
|
82
116
|
```
|
83
117
|
|
84
118
|
## Note on contributions
|
@@ -0,0 +1,26 @@
|
|
1
|
+
-----BEGIN CERTIFICATE-----
|
2
|
+
MIIEcDCCAtigAwIBAgIBATANBgkqhkiG9w0BAQsFADA/MRAwDgYDVQQDDAdjb250
|
3
|
+
YWN0MRcwFQYKCZImiZPyLGQBGRYHa2FyYWZrYTESMBAGCgmSJomT8ixkARkWAmlv
|
4
|
+
MB4XDTIyMDgxOTE3MjEzN1oXDTIzMDgxOTE3MjEzN1owPzEQMA4GA1UEAwwHY29u
|
5
|
+
dGFjdDEXMBUGCgmSJomT8ixkARkWB2thcmFma2ExEjAQBgoJkiaJk/IsZAEZFgJp
|
6
|
+
bzCCAaIwDQYJKoZIhvcNAQEBBQADggGPADCCAYoCggGBAODzeO3L6lxdATzMHKNW
|
7
|
+
jFA/GGunoPuylO/BMzy8RiQHh7VIvysAKs0tHhTx3g2D0STDpF+hcQcPELFikiT2
|
8
|
+
F+1wOHj/SsrK7VKqfA8+gq04hKc5sQoX2Egf9k3V0YJ3eZ6R/koHkQ8A0TVt0w6F
|
9
|
+
ZQckoV4MqnEAx0g/FZN3mnHTlJ3VFLSBqJEIe+S6FZMl92mSv+hTrlUG8VaYxSfN
|
10
|
+
lTCvnKk284F6QZq5XIENLRmcDd/3aPBLnLwNnyMyhB+6gK8cUO+CFlDO5tjo/aBA
|
11
|
+
rUnl++wGG0JooF1ed0v+evOn9KoMBG6rHewcf79qJbVOscbD8qSAmo+sCXtcFryr
|
12
|
+
KRMTB8gNbowJkFRJDEe8tfRy11u1fYzFg/qNO82FJd62rKAw2wN0C29yCeQOPRb1
|
13
|
+
Cw9Y4ZwK9VFNEcV9L+3pHTHn2XfuZHtDaG198VweiF6raFO4yiEYccodH/USP0L5
|
14
|
+
cbcCFtmu/4HDSxL1ByQXO84A0ybJuk3/+aPUSXe9C9U8fwIDAQABo3cwdTAJBgNV
|
15
|
+
HRMEAjAAMAsGA1UdDwQEAwIEsDAdBgNVHQ4EFgQUSlcEakb7gfn/5E2WY6z73BF/
|
16
|
+
iZkwHQYDVR0RBBYwFIESY29udGFjdEBrYXJhZmthLmlvMB0GA1UdEgQWMBSBEmNv
|
17
|
+
bnRhY3RAa2FyYWZrYS5pbzANBgkqhkiG9w0BAQsFAAOCAYEA1aS+E7RXJ1w9g9mJ
|
18
|
+
G0NzFxe64OEuENosNlvYQCbRKGCXAU1qqelYkBQHseRgRKxLICrnypRo9IEobyHa
|
19
|
+
vDnJ4r7Tsb34dleqQW2zY/obG+cia3Ym2JsegXWF7dDOzCXJ4FN8MFoT2jHlqLLw
|
20
|
+
yrap0YO5zx0GSQ0Dwy8h2n2v2vanMEeCx7iNm3ERgR5WuN5sjzWoz2A/JLEEcK0C
|
21
|
+
EnAGKCWAd1fuG8IemDjT1edsd5FyYR4bIX0m+99oDuFZyPiiIbalmyYiSBBp59Yb
|
22
|
+
Q0P8zeBi4OfwCZNcxqz0KONmw9JLNv6DgyEAH5xe/4JzhMEgvIRiPj0pHfA7oqQF
|
23
|
+
KUNqvD1KlxbEC+bZfE5IZhnqYLdld/Ksqd22FI1RBhiS1Ejfsj99LVIm9cBuZEY2
|
24
|
+
Qf04B9ceLUaC4fPVEz10FyobjaFoY4i32xRto3XnrzeAgfEe4swLq8bQsR3w/EF3
|
25
|
+
MGU0FeSV2Yj7Xc2x/7BzLK8xQn5l7Yy75iPF+KP3vVmDHnNl
|
26
|
+
-----END CERTIFICATE-----
|
data/karafka-testing.gemspec
CHANGED
@@ -10,7 +10,7 @@ Gem::Specification.new do |spec|
|
|
10
10
|
spec.platform = Gem::Platform::RUBY
|
11
11
|
spec.version = Karafka::Testing::VERSION
|
12
12
|
spec.authors = ['Maciej Mensfeld']
|
13
|
-
spec.email = %w[
|
13
|
+
spec.email = %w[contact@karafka.io]
|
14
14
|
spec.summary = 'Library which provides helpers for easier Karafka consumers tests'
|
15
15
|
spec.description = 'Library which provides helpers for easier Karafka consumers tests'
|
16
16
|
spec.homepage = 'https://karafka.io'
|
@@ -18,7 +18,7 @@ Gem::Specification.new do |spec|
|
|
18
18
|
spec.files = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(spec)/}) }
|
19
19
|
spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
|
20
20
|
spec.require_paths = %w[lib]
|
21
|
-
spec.cert_chain = %w[certs/
|
21
|
+
spec.cert_chain = %w[certs/cert_chain.pem]
|
22
22
|
|
23
23
|
spec.required_ruby_version = '>= 2.7'
|
24
24
|
|
@@ -26,10 +26,10 @@ Gem::Specification.new do |spec|
|
|
26
26
|
spec.signing_key = File.expand_path('~/.ssh/gem-private_key.pem')
|
27
27
|
end
|
28
28
|
|
29
|
-
spec.add_dependency 'karafka', '>= 2.0
|
29
|
+
spec.add_dependency 'karafka', '>= 2.0', '< 3.0.0'
|
30
30
|
|
31
31
|
spec.metadata = {
|
32
|
-
'source_code_uri' => 'https://github.com/karafka/karafka',
|
32
|
+
'source_code_uri' => 'https://github.com/karafka/karafka-testing',
|
33
33
|
'rubygems_mfa_required' => 'true'
|
34
34
|
}
|
35
35
|
end
|
@@ -7,8 +7,12 @@ module Karafka
|
|
7
7
|
module Errors
|
8
8
|
# Base error for all the internal errors
|
9
9
|
BaseError = Class.new(StandardError)
|
10
|
+
|
10
11
|
# Raised when we want to build a consumer for a topic that does not exist
|
11
12
|
TopicNotFoundError = Class.new(BaseError)
|
13
|
+
|
14
|
+
# Raised when topic is in many consumer groups and not limited by consumer group expectation
|
15
|
+
TopicInManyConsumerGroupsError = Class.new(BaseError)
|
12
16
|
end
|
13
17
|
end
|
14
18
|
end
|
@@ -1,7 +1,9 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
|
+
require 'waterdrop'
|
3
4
|
require 'karafka/testing/errors'
|
4
|
-
require 'karafka/testing/
|
5
|
+
require 'karafka/testing/spec_consumer_client'
|
6
|
+
require 'karafka/testing/spec_producer_client'
|
5
7
|
require 'karafka/testing/rspec/proxy'
|
6
8
|
|
7
9
|
module Karafka
|
@@ -15,12 +17,24 @@ module Karafka
|
|
15
17
|
# Adds all the needed extra functionalities to the rspec group
|
16
18
|
# @param base [Class] RSpec example group we want to extend
|
17
19
|
def included(base)
|
18
|
-
#
|
19
|
-
# we run the consume
|
20
|
-
base.let(:_karafka_messages) { [] }
|
20
|
+
# RSpec local reference to Karafka proxy that allows us to build the consumer instance
|
21
21
|
base.let(:karafka) { Karafka::Testing::RSpec::Proxy.new(self) }
|
22
|
-
|
23
|
-
|
22
|
+
|
23
|
+
# Messages that are targeted to the consumer
|
24
|
+
# You can produce many messages from Karafka during specs and not all should go to the
|
25
|
+
# consumer for processing. This buffer holds only those that should go to consumer
|
26
|
+
base.let(:_karafka_consumer_messages) { [] }
|
27
|
+
# Consumer fake client to mock communication with Kafka
|
28
|
+
base.let(:_karafka_consumer_client) { Karafka::Testing::SpecConsumerClient.new }
|
29
|
+
# Producer fake client to mock communication with Kafka
|
30
|
+
base.let(:_karafka_producer_client) { Karafka::Testing::SpecProducerClient.new(self) }
|
31
|
+
|
32
|
+
base.before do
|
33
|
+
_karafka_consumer_messages.clear
|
34
|
+
_karafka_producer_client.reset
|
35
|
+
|
36
|
+
allow(Karafka.producer).to receive(:client).and_return(_karafka_producer_client)
|
37
|
+
end
|
24
38
|
end
|
25
39
|
end
|
26
40
|
|
@@ -28,6 +42,8 @@ module Karafka
|
|
28
42
|
#
|
29
43
|
# @param requested_topic [String, Symbol] name of the topic for which we want to
|
30
44
|
# create a consumer instance
|
45
|
+
# @param requested_consumer_group [String, Symbol, nil] optional name of the consumer group
|
46
|
+
# if we have multiple consumer groups listening on the same topic
|
31
47
|
# @return [Object] described_class instance
|
32
48
|
# @raise [Karafka::Testing::Errors::TopicNotFoundError] raised when we're unable to find
|
33
49
|
# topic that was requested
|
@@ -36,78 +52,119 @@ module Karafka
|
|
36
52
|
# RSpec.describe MyConsumer do
|
37
53
|
# subject(:consumer) { karafka.consumer_for(:my_requested_topic) }
|
38
54
|
# end
|
39
|
-
def
|
40
|
-
|
55
|
+
def _karafka_consumer_for(requested_topic, requested_consumer_group = nil)
|
56
|
+
all_topics = ::Karafka::App.consumer_groups.map(&:topics).flat_map(&:to_a)
|
41
57
|
|
42
|
-
#
|
43
|
-
|
44
|
-
|
45
|
-
selected_topic = topic if topic.name == requested_topic.to_s
|
46
|
-
end
|
58
|
+
# First select topics that match what we are looking for
|
59
|
+
selected_topics = all_topics.select do |topic|
|
60
|
+
topic.name == requested_topic.to_s
|
47
61
|
end
|
48
62
|
|
49
|
-
|
63
|
+
# And then narrow it down based on the consumer group criteria (if present)
|
64
|
+
selected_topics.delete_if do |topic|
|
65
|
+
requested_consumer_group && topic.consumer_group.name != requested_consumer_group.to_s
|
66
|
+
end
|
50
67
|
|
51
|
-
|
68
|
+
raise Errors::TopicInManyConsumerGroupsError, requested_topic if selected_topics.size > 1
|
69
|
+
raise Errors::TopicNotFoundError, requested_topic if selected_topics.empty?
|
52
70
|
|
53
|
-
|
54
|
-
consumer.topic = selected_topic
|
55
|
-
consumer.producer = Karafka::App.producer
|
56
|
-
consumer.client = Karafka::Testing::DummyClient.new
|
57
|
-
consumer.coordinator = coordinators.find_or_create(requested_topic, 0)
|
58
|
-
consumer
|
71
|
+
_karafka_build_consumer_for(selected_topics.first)
|
59
72
|
end
|
60
73
|
|
61
|
-
# Adds a new Karafka message instance with given payload and options into an
|
62
|
-
# buffer that will be used to simulate messages delivery to the consumer
|
63
|
-
#
|
64
|
-
# @param payload [String] anything you want to send
|
65
|
-
# @param opts [Hash] additional options with which you want to overwrite the
|
66
|
-
# message defaults (key, offset, etc)
|
74
|
+
# Adds a new Karafka message instance if needed with given payload and options into an
|
75
|
+
# internal consumer buffer that will be used to simulate messages delivery to the consumer
|
67
76
|
#
|
77
|
+
# @param message [Hash] message that was sent to Kafka
|
68
78
|
# @example Send a json message to consumer
|
69
79
|
# before do
|
70
|
-
# karafka.
|
80
|
+
# karafka.produce({ 'hello' => 'world' }.to_json)
|
71
81
|
# end
|
72
82
|
#
|
73
83
|
# @example Send a json message to consumer and simulate, that it is partition 6
|
74
84
|
# before do
|
75
|
-
# karafka.
|
85
|
+
# karafka.produce({ 'hello' => 'world' }.to_json, 'partition' => 6)
|
76
86
|
# end
|
77
|
-
def
|
78
|
-
|
79
|
-
|
80
|
-
|
87
|
+
def _karafka_add_message_to_consumer_if_needed(message)
|
88
|
+
# We target to the consumer only messages that were produced to it, since specs may also
|
89
|
+
# produce other messages targeting other topics
|
90
|
+
return unless message[:topic] == consumer.topic.name
|
91
|
+
|
92
|
+
# Build message metadata and copy any metadata that would come from the message
|
93
|
+
metadata = _karafka_message_metadata_defaults
|
94
|
+
|
95
|
+
metadata.keys.each do |key|
|
96
|
+
next unless message.key?(key)
|
97
|
+
|
98
|
+
metadata[key] = message.fetch(key)
|
99
|
+
end
|
81
100
|
|
82
|
-
# Add this message to previously
|
83
|
-
|
101
|
+
# Add this message to previously produced messages
|
102
|
+
_karafka_consumer_messages << Karafka::Messages::Message.new(
|
103
|
+
message[:payload],
|
104
|
+
Karafka::Messages::Metadata.new(metadata).freeze
|
105
|
+
)
|
84
106
|
|
85
107
|
# Update batch metadata
|
86
108
|
batch_metadata = Karafka::Messages::Builders::BatchMetadata.call(
|
87
|
-
|
109
|
+
_karafka_consumer_messages,
|
88
110
|
subject.topic,
|
89
111
|
Time.now
|
90
112
|
)
|
91
113
|
|
92
114
|
# Update consumer messages batch
|
93
|
-
subject.messages = Karafka::Messages::Messages.new(
|
115
|
+
subject.messages = Karafka::Messages::Messages.new(
|
116
|
+
_karafka_consumer_messages,
|
117
|
+
batch_metadata
|
118
|
+
)
|
119
|
+
end
|
120
|
+
|
121
|
+
# Produces message with a given payload to the consumer matching topic
|
122
|
+
# @param payload [String] payload we want to dispatch
|
123
|
+
# @param metadata [Hash] any metadata we want to dispatch alongside the payload
|
124
|
+
def _karafka_produce(payload, metadata = {})
|
125
|
+
Karafka.producer.produce_sync(
|
126
|
+
{
|
127
|
+
topic: subject.topic.name,
|
128
|
+
payload: payload
|
129
|
+
}.merge(metadata)
|
130
|
+
)
|
131
|
+
end
|
132
|
+
|
133
|
+
# @return [Array<Hash>] messages that were produced
|
134
|
+
def _karafka_produced_messages
|
135
|
+
_karafka_producer_client.messages
|
94
136
|
end
|
95
137
|
|
96
138
|
private
|
97
139
|
|
98
140
|
# @return [Hash] message default options
|
99
|
-
def
|
141
|
+
def _karafka_message_metadata_defaults
|
100
142
|
{
|
101
143
|
deserializer: subject.topic.deserializer,
|
102
144
|
timestamp: Time.now,
|
103
145
|
headers: {},
|
104
146
|
key: nil,
|
105
|
-
offset:
|
147
|
+
offset: _karafka_consumer_messages.size,
|
106
148
|
partition: 0,
|
107
149
|
received_at: Time.now,
|
108
150
|
topic: subject.topic.name
|
109
151
|
}
|
110
152
|
end
|
153
|
+
|
154
|
+
# Builds the consumer instance based on the provided topic
|
155
|
+
#
|
156
|
+
# @param topic [Karafka::Routing::Topic] topic for which we want to build the consumer
|
157
|
+
# @return [Object] karafka consumer
|
158
|
+
def _karafka_build_consumer_for(topic)
|
159
|
+
coordinators = Karafka::Processing::CoordinatorsBuffer.new
|
160
|
+
|
161
|
+
consumer = described_class.new
|
162
|
+
consumer.topic = topic
|
163
|
+
consumer.producer = Karafka::App.producer
|
164
|
+
consumer.client = _karafka_consumer_client
|
165
|
+
consumer.coordinator = coordinators.find_or_create(topic.name, 0)
|
166
|
+
consumer
|
167
|
+
end
|
111
168
|
end
|
112
169
|
end
|
113
170
|
end
|
@@ -4,20 +4,26 @@ module Karafka
|
|
4
4
|
module Testing
|
5
5
|
module RSpec
|
6
6
|
# Proxy object for a nicer `karafka.` API within RSpec
|
7
|
+
# None other should be used by the end users
|
7
8
|
class Proxy
|
8
9
|
# @param rspec_example [RSpec::ExampleGroups] rspec context
|
9
10
|
def initialize(rspec_example)
|
10
11
|
@rspec_example = rspec_example
|
11
12
|
end
|
12
13
|
|
13
|
-
# @param args Anything that the `#
|
14
|
+
# @param args Anything that the `#_karafka_consumer_for` accepts
|
14
15
|
def consumer_for(*args)
|
15
|
-
@rspec_example.
|
16
|
+
@rspec_example._karafka_consumer_for(*args)
|
16
17
|
end
|
17
18
|
|
18
|
-
# @param args Anything that
|
19
|
-
def
|
20
|
-
@rspec_example.
|
19
|
+
# @param args Anything that `#_karafka_produce` accepts
|
20
|
+
def produce(*args)
|
21
|
+
@rspec_example._karafka_produce(*args)
|
22
|
+
end
|
23
|
+
|
24
|
+
# @return [Array<Hash>] messages produced via `Karafka#producer`
|
25
|
+
def produced_messages
|
26
|
+
@rspec_example._karafka_produced_messages
|
21
27
|
end
|
22
28
|
end
|
23
29
|
end
|
@@ -2,10 +2,10 @@
|
|
2
2
|
|
3
3
|
module Karafka
|
4
4
|
module Testing
|
5
|
-
# A
|
5
|
+
# A spec client that takes over client delegated methods from the consumers
|
6
6
|
# For specs we do not mark anything as consumed, nor do we really send heartbeats.
|
7
7
|
# Those things are tested in the framework itself
|
8
|
-
class
|
8
|
+
class SpecConsumerClient
|
9
9
|
%i[
|
10
10
|
mark_as_consumed
|
11
11
|
mark_as_consumed!
|
@@ -0,0 +1,43 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Karafka
|
4
|
+
module Testing
|
5
|
+
# Spec producer client used to buffer messages that we send out in specs
|
6
|
+
class SpecProducerClient < ::WaterDrop::Producer::DummyClient
|
7
|
+
attr_accessor :messages
|
8
|
+
|
9
|
+
# Sync fake response for the message delivery to Kafka, since we do not dispatch anything
|
10
|
+
class SyncResponse
|
11
|
+
# @param _args Handler wait arguments (irrelevant as waiting is fake here)
|
12
|
+
def wait(*_args)
|
13
|
+
false
|
14
|
+
end
|
15
|
+
end
|
16
|
+
|
17
|
+
# @param rspec [RSpec::Core::ExampleGroup] rspec example we need to hold to trigger actions
|
18
|
+
# on it that are rspec context aware
|
19
|
+
def initialize(rspec)
|
20
|
+
super()
|
21
|
+
@rspec = rspec
|
22
|
+
self.messages = []
|
23
|
+
end
|
24
|
+
|
25
|
+
# "Produces" message to Kafka. That is, it acknowledges it locally, adds it to the internal
|
26
|
+
# buffer and adds it (if needed) into the current consumer messages buffer
|
27
|
+
# @param message [Hash] `Karafka.producer.produce_sync` message hash
|
28
|
+
def produce(message)
|
29
|
+
messages << message
|
30
|
+
|
31
|
+
@rspec._karafka_add_message_to_consumer_if_needed(message)
|
32
|
+
|
33
|
+
SyncResponse.new
|
34
|
+
end
|
35
|
+
|
36
|
+
# Clears internal buffer
|
37
|
+
# Used in between specs so messages do not leak out
|
38
|
+
def reset
|
39
|
+
messages.clear
|
40
|
+
end
|
41
|
+
end
|
42
|
+
end
|
43
|
+
end
|
data.tar.gz.sig
CHANGED
Binary file
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: karafka-testing
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 2.0.
|
4
|
+
version: 2.0.2
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Maciej Mensfeld
|
@@ -10,31 +10,32 @@ bindir: bin
|
|
10
10
|
cert_chain:
|
11
11
|
- |
|
12
12
|
-----BEGIN CERTIFICATE-----
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
|
24
|
-
|
25
|
-
|
26
|
-
|
27
|
-
|
28
|
-
|
29
|
-
|
30
|
-
|
31
|
-
/
|
32
|
-
|
33
|
-
|
34
|
-
|
35
|
-
|
13
|
+
MIIEcDCCAtigAwIBAgIBATANBgkqhkiG9w0BAQsFADA/MRAwDgYDVQQDDAdjb250
|
14
|
+
YWN0MRcwFQYKCZImiZPyLGQBGRYHa2FyYWZrYTESMBAGCgmSJomT8ixkARkWAmlv
|
15
|
+
MB4XDTIyMDgxOTE3MjEzN1oXDTIzMDgxOTE3MjEzN1owPzEQMA4GA1UEAwwHY29u
|
16
|
+
dGFjdDEXMBUGCgmSJomT8ixkARkWB2thcmFma2ExEjAQBgoJkiaJk/IsZAEZFgJp
|
17
|
+
bzCCAaIwDQYJKoZIhvcNAQEBBQADggGPADCCAYoCggGBAODzeO3L6lxdATzMHKNW
|
18
|
+
jFA/GGunoPuylO/BMzy8RiQHh7VIvysAKs0tHhTx3g2D0STDpF+hcQcPELFikiT2
|
19
|
+
F+1wOHj/SsrK7VKqfA8+gq04hKc5sQoX2Egf9k3V0YJ3eZ6R/koHkQ8A0TVt0w6F
|
20
|
+
ZQckoV4MqnEAx0g/FZN3mnHTlJ3VFLSBqJEIe+S6FZMl92mSv+hTrlUG8VaYxSfN
|
21
|
+
lTCvnKk284F6QZq5XIENLRmcDd/3aPBLnLwNnyMyhB+6gK8cUO+CFlDO5tjo/aBA
|
22
|
+
rUnl++wGG0JooF1ed0v+evOn9KoMBG6rHewcf79qJbVOscbD8qSAmo+sCXtcFryr
|
23
|
+
KRMTB8gNbowJkFRJDEe8tfRy11u1fYzFg/qNO82FJd62rKAw2wN0C29yCeQOPRb1
|
24
|
+
Cw9Y4ZwK9VFNEcV9L+3pHTHn2XfuZHtDaG198VweiF6raFO4yiEYccodH/USP0L5
|
25
|
+
cbcCFtmu/4HDSxL1ByQXO84A0ybJuk3/+aPUSXe9C9U8fwIDAQABo3cwdTAJBgNV
|
26
|
+
HRMEAjAAMAsGA1UdDwQEAwIEsDAdBgNVHQ4EFgQUSlcEakb7gfn/5E2WY6z73BF/
|
27
|
+
iZkwHQYDVR0RBBYwFIESY29udGFjdEBrYXJhZmthLmlvMB0GA1UdEgQWMBSBEmNv
|
28
|
+
bnRhY3RAa2FyYWZrYS5pbzANBgkqhkiG9w0BAQsFAAOCAYEA1aS+E7RXJ1w9g9mJ
|
29
|
+
G0NzFxe64OEuENosNlvYQCbRKGCXAU1qqelYkBQHseRgRKxLICrnypRo9IEobyHa
|
30
|
+
vDnJ4r7Tsb34dleqQW2zY/obG+cia3Ym2JsegXWF7dDOzCXJ4FN8MFoT2jHlqLLw
|
31
|
+
yrap0YO5zx0GSQ0Dwy8h2n2v2vanMEeCx7iNm3ERgR5WuN5sjzWoz2A/JLEEcK0C
|
32
|
+
EnAGKCWAd1fuG8IemDjT1edsd5FyYR4bIX0m+99oDuFZyPiiIbalmyYiSBBp59Yb
|
33
|
+
Q0P8zeBi4OfwCZNcxqz0KONmw9JLNv6DgyEAH5xe/4JzhMEgvIRiPj0pHfA7oqQF
|
34
|
+
KUNqvD1KlxbEC+bZfE5IZhnqYLdld/Ksqd22FI1RBhiS1Ejfsj99LVIm9cBuZEY2
|
35
|
+
Qf04B9ceLUaC4fPVEz10FyobjaFoY4i32xRto3XnrzeAgfEe4swLq8bQsR3w/EF3
|
36
|
+
MGU0FeSV2Yj7Xc2x/7BzLK8xQn5l7Yy75iPF+KP3vVmDHnNl
|
36
37
|
-----END CERTIFICATE-----
|
37
|
-
date: 2022-
|
38
|
+
date: 2022-09-29 00:00:00.000000000 Z
|
38
39
|
dependencies:
|
39
40
|
- !ruby/object:Gem::Dependency
|
40
41
|
name: karafka
|
@@ -42,7 +43,7 @@ dependencies:
|
|
42
43
|
requirements:
|
43
44
|
- - ">="
|
44
45
|
- !ruby/object:Gem::Version
|
45
|
-
version: 2.0
|
46
|
+
version: '2.0'
|
46
47
|
- - "<"
|
47
48
|
- !ruby/object:Gem::Version
|
48
49
|
version: 3.0.0
|
@@ -52,13 +53,13 @@ dependencies:
|
|
52
53
|
requirements:
|
53
54
|
- - ">="
|
54
55
|
- !ruby/object:Gem::Version
|
55
|
-
version: 2.0
|
56
|
+
version: '2.0'
|
56
57
|
- - "<"
|
57
58
|
- !ruby/object:Gem::Version
|
58
59
|
version: 3.0.0
|
59
60
|
description: Library which provides helpers for easier Karafka consumers tests
|
60
61
|
email:
|
61
|
-
-
|
62
|
+
- contact@karafka.io
|
62
63
|
executables: []
|
63
64
|
extensions: []
|
64
65
|
extra_rdoc_files: []
|
@@ -76,20 +77,21 @@ files:
|
|
76
77
|
- Gemfile.lock
|
77
78
|
- MIT-LICENSE
|
78
79
|
- README.md
|
79
|
-
- certs/
|
80
|
+
- certs/cert_chain.pem
|
80
81
|
- karafka-testing.gemspec
|
81
82
|
- lib/karafka-testing.rb
|
82
83
|
- lib/karafka/testing.rb
|
83
|
-
- lib/karafka/testing/dummy_client.rb
|
84
84
|
- lib/karafka/testing/errors.rb
|
85
85
|
- lib/karafka/testing/rspec/helpers.rb
|
86
86
|
- lib/karafka/testing/rspec/proxy.rb
|
87
|
+
- lib/karafka/testing/spec_consumer_client.rb
|
88
|
+
- lib/karafka/testing/spec_producer_client.rb
|
87
89
|
- lib/karafka/testing/version.rb
|
88
90
|
homepage: https://karafka.io
|
89
91
|
licenses:
|
90
92
|
- MIT
|
91
93
|
metadata:
|
92
|
-
source_code_uri: https://github.com/karafka/karafka
|
94
|
+
source_code_uri: https://github.com/karafka/karafka-testing
|
93
95
|
rubygems_mfa_required: 'true'
|
94
96
|
post_install_message:
|
95
97
|
rdoc_options: []
|
metadata.gz.sig
CHANGED
Binary file
|
data/certs/mensfeld.pem
DELETED
@@ -1,25 +0,0 @@
|
|
1
|
-
-----BEGIN CERTIFICATE-----
|
2
|
-
MIIEODCCAqCgAwIBAgIBATANBgkqhkiG9w0BAQsFADAjMSEwHwYDVQQDDBhtYWNp
|
3
|
-
ZWovREM9bWVuc2ZlbGQvREM9cGwwHhcNMjEwODExMTQxNTEzWhcNMjIwODExMTQx
|
4
|
-
NTEzWjAjMSEwHwYDVQQDDBhtYWNpZWovREM9bWVuc2ZlbGQvREM9cGwwggGiMA0G
|
5
|
-
CSqGSIb3DQEBAQUAA4IBjwAwggGKAoIBgQDV2jKH4Ti87GM6nyT6D+ESzTI0MZDj
|
6
|
-
ak2/TEwnxvijMJyCCPKT/qIkbW4/f0VHM4rhPr1nW73sb5SZBVFCLlJcOSKOBdUY
|
7
|
-
TMY+SIXN2EtUaZuhAOe8LxtxjHTgRHvHcqUQMBENXTISNzCo32LnUxweu66ia4Pd
|
8
|
-
1mNRhzOqNv9YiBZvtBf7IMQ+sYdOCjboq2dlsWmJiwiDpY9lQBTnWORnT3mQxU5x
|
9
|
-
vPSwnLB854cHdCS8fQo4DjeJBRZHhEbcE5sqhEMB3RZA3EtFVEXOxlNxVTS3tncI
|
10
|
-
qyNXiWDaxcipaens4ObSY1C2HTV7OWb7OMqSCIybeYTSfkaSdqmcl4S6zxXkjH1J
|
11
|
-
tnjayAVzD+QVXGijsPLE2PFnJAh9iDET2cMsjabO1f6l1OQNyAtqpcyQcgfnyW0z
|
12
|
-
g7tGxTYD+6wJHffM9d9txOUw6djkF6bDxyqB8lo4Z3IObCx18AZjI9XPS9QG7w6q
|
13
|
-
LCWuMG2lkCcRgASqaVk9fEf9yMc2xxz5o3kCAwEAAaN3MHUwCQYDVR0TBAIwADAL
|
14
|
-
BgNVHQ8EBAMCBLAwHQYDVR0OBBYEFBqUFCKCOe5IuueUVqOB991jyCLLMB0GA1Ud
|
15
|
-
EQQWMBSBEm1hY2llakBtZW5zZmVsZC5wbDAdBgNVHRIEFjAUgRJtYWNpZWpAbWVu
|
16
|
-
c2ZlbGQucGwwDQYJKoZIhvcNAQELBQADggGBADD0/UuTTFgW+CGk2U0RDw2RBOca
|
17
|
-
W2LTF/G7AOzuzD0Tc4voc7WXyrgKwJREv8rgBimLnNlgmFJLmtUCh2U/MgxvcilH
|
18
|
-
yshYcbseNvjkrtYnLRlWZR4SSB6Zei5AlyGVQLPkvdsBpNegcG6w075YEwzX/38a
|
19
|
-
8V9B/Yri2OGELBz8ykl7BsXUgNoUPA/4pHF6YRLz+VirOaUIQ4JfY7xGj6fSOWWz
|
20
|
-
/rQ/d77r6o1mfJYM/3BRVg73a3b7DmRnE5qjwmSaSQ7u802pJnLesmArch0xGCT/
|
21
|
-
fMmRli1Qb+6qOTl9mzD6UDMAyFR4t6MStLm0mIEqM0nBO5nUdUWbC7l9qXEf8XBE
|
22
|
-
2DP28p3EqSuS+lKbAWKcqv7t0iRhhmaod+Yn9mcrLN1sa3q3KSQ9BCyxezCD4Mk2
|
23
|
-
R2P11bWoCtr70BsccVrN8jEhzwXngMyI2gVt750Y+dbTu1KgRqZKp/ECe7ZzPzXj
|
24
|
-
pIy9vHxTANKYVyI4qj8OrFdEM5BQNu8oQpL0iQ==
|
25
|
-
-----END CERTIFICATE-----
|