karafka-testing 2.0.1 → 2.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/CHANGELOG.md +6 -0
- data/Gemfile.lock +3 -3
- data/README.md +51 -17
- data/certs/cert_chain.pem +26 -0
- data/karafka-testing.gemspec +3 -3
- data/lib/karafka/testing/errors.rb +4 -0
- data/lib/karafka/testing/rspec/helpers.rb +96 -39
- data/lib/karafka/testing/rspec/proxy.rb +11 -5
- data/lib/karafka/testing/{dummy_client.rb → spec_consumer_client.rb} +2 -2
- data/lib/karafka/testing/spec_producer_client.rb +43 -0
- data/lib/karafka/testing/version.rb +1 -1
- data.tar.gz.sig +0 -0
- metadata +31 -29
- metadata.gz.sig +0 -0
- data/certs/mensfeld.pem +0 -25
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 9112bdf5ca189a7f448c2b5768aa2694a5958a96f591f8d8bad8cc204708f1a3
|
4
|
+
data.tar.gz: 61ae2749b2bac651143bca2b8cdb470900b87eccc058c13cbaa4644e1be1435e
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: adff2729f366403cc7e01a18135f972b05b9dd9faa8408a8cda7f75cf6425c9e676060815037fc78acaecfc4efa37be122e7b8d15623c6c31b7263fb2856bdc1
|
7
|
+
data.tar.gz: 29da133e08e85e5d1efa24156c1eab91aa858387bc341070cc4ae059d073e721e6955808f6b6923b39b102a3e713067e6cf856a889d0d9d5568674a27245926e
|
checksums.yaml.gz.sig
CHANGED
Binary file
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,11 @@
|
|
1
1
|
# Karafka Test gem changelog
|
2
2
|
|
3
|
+
## 2.0.2 (2022-09-29)
|
4
|
+
- Provide multi-consumer group testing support (#92)
|
5
|
+
- Fail fast if requested topic is present in multiple consumer groups but consumer group is not specified.
|
6
|
+
- Allow for usage of `Karafka.producer` directly and make it buffered.
|
7
|
+
- Rename `karafka.publish` to `karafka.produce` to align naming conventions.
|
8
|
+
|
3
9
|
## 2.0.1 (2022-08-05)
|
4
10
|
- Require non rc version of Karafka.
|
5
11
|
|
data/Gemfile.lock
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
PATH
|
2
2
|
remote: .
|
3
3
|
specs:
|
4
|
-
karafka-testing (2.0.
|
4
|
+
karafka-testing (2.0.2)
|
5
5
|
karafka (>= 2.0, < 3.0.0)
|
6
6
|
|
7
7
|
GEM
|
@@ -9,7 +9,7 @@ GEM
|
|
9
9
|
specs:
|
10
10
|
concurrent-ruby (1.1.10)
|
11
11
|
ffi (1.15.5)
|
12
|
-
karafka (2.0.
|
12
|
+
karafka (2.0.10)
|
13
13
|
karafka-core (>= 2.0.2, < 3.0.0)
|
14
14
|
rdkafka (>= 0.12)
|
15
15
|
thor (>= 0.20)
|
@@ -37,4 +37,4 @@ DEPENDENCIES
|
|
37
37
|
karafka-testing!
|
38
38
|
|
39
39
|
BUNDLED WITH
|
40
|
-
2.3.
|
40
|
+
2.3.22
|
data/README.md
CHANGED
@@ -1,14 +1,10 @@
|
|
1
1
|
# Karafka Testing library
|
2
2
|
|
3
|
-
**Note**: Documentation presented below works with Karafka `2.0.0.alpha3` and higher.
|
4
|
-
|
5
|
-
Please refer to [this](https://github.com/karafka/testing/tree/1.4) branch and its documentation for details about usage with Karafka `1.4`.
|
6
|
-
|
7
3
|
[](https://github.com/karafka/karafka-testing/actions?query=workflow%3Aci)
|
8
4
|
[](http://badge.fury.io/rb/karafka-testing)
|
9
5
|
[](https://slack.karafka.io)
|
10
6
|
|
11
|
-
Karafka-Testing is a library that provides RSpec helpers, to make testing of Karafka consumers much easier.
|
7
|
+
Karafka-Testing is a library that provides RSpec helpers, to make testing of Karafka consumers and producer much easier.
|
12
8
|
|
13
9
|
## Installation
|
14
10
|
|
@@ -32,17 +28,22 @@ end
|
|
32
28
|
|
33
29
|
## Usage
|
34
30
|
|
35
|
-
Once included
|
31
|
+
Once included in your RSpec setup, this library will provide you with a special `#karafka` object that contains three methods that you can use within your specs:
|
36
32
|
|
37
33
|
- `#consumer_for` - creates a consumer instance for the desired topic. It **needs** to be set as the spec subject.
|
38
|
-
- `#
|
34
|
+
- `#produce` - "sends" message to the consumer instance.
|
35
|
+
- `#produced_messages` - contains all the messages "sent" to Kafka during spec execution.
|
36
|
+
|
37
|
+
**Note:** Messages sent using the `#produce` method and directly from `Karafka.producer` won't be sent to Kafka. They will be buffered and accessible in a per-spec buffer in case you want to test messages production.
|
39
38
|
|
40
|
-
|
39
|
+
Messages that target the topic built using the `karafka#consumer_for` method will additionally be delivered to the consumer you want to test.
|
40
|
+
|
41
|
+
### Testing messages consumption (consumers)
|
41
42
|
|
42
43
|
```ruby
|
43
44
|
RSpec.describe InlineBatchConsumer do
|
44
45
|
# This will create a consumer instance with all the settings defined for the given topic
|
45
|
-
subject(:consumer) { karafka.consumer_for(
|
46
|
+
subject(:consumer) { karafka.consumer_for('inline_batch_data') }
|
46
47
|
|
47
48
|
let(:nr1_value) { rand }
|
48
49
|
let(:nr2_value) { rand }
|
@@ -50,9 +51,11 @@ RSpec.describe InlineBatchConsumer do
|
|
50
51
|
|
51
52
|
before do
|
52
53
|
# Sends first message to Karafka consumer
|
53
|
-
karafka.
|
54
|
+
karafka.produce({ 'number' => nr1_value }.to_json)
|
55
|
+
|
54
56
|
# Sends second message to Karafka consumer
|
55
|
-
karafka.
|
57
|
+
karafka.produce({ 'number' => nr2_value }.to_json, partition: 2)
|
58
|
+
|
56
59
|
allow(Karafka.logger).to receive(:info)
|
57
60
|
end
|
58
61
|
|
@@ -63,22 +66,53 @@ RSpec.describe InlineBatchConsumer do
|
|
63
66
|
end
|
64
67
|
```
|
65
68
|
|
66
|
-
If your consumers use `producer` to dispatch messages, you can
|
69
|
+
If your consumers use `producer` to dispatch messages, you can check its operations as well:
|
67
70
|
|
68
71
|
```ruby
|
69
72
|
RSpec.describe InlineBatchConsumer do
|
70
73
|
subject(:consumer) { karafka.consumer_for(:inline_batch_data) }
|
71
74
|
|
72
|
-
before { karafka.
|
75
|
+
before { karafka.produce({ 'number' => 1 }.to_json) }
|
73
76
|
|
74
77
|
it 'expects to dispatch async message to messages topic with value bigger by 1' do
|
75
|
-
expect(consumer.producer)
|
76
|
-
.to receive(:produce_async)
|
77
|
-
.with(topic: 'messages', payload: { number: 2 }.to_json)
|
78
|
-
|
79
78
|
consumer.consume
|
79
|
+
|
80
|
+
expect(karafka.produced_messages.last.payload).to eq({ number: 2 }.to_json)
|
81
|
+
end
|
82
|
+
end
|
83
|
+
```
|
84
|
+
|
85
|
+
### Testing messages production (producer)
|
86
|
+
|
87
|
+
When running RSpec, Karafka will not dispatch messages to Kafka using `Karafka.producer` but will buffer them internally.
|
88
|
+
|
89
|
+
This means you can check your application flow, making sure your logic acts as expected:
|
90
|
+
|
91
|
+
```ruby
|
92
|
+
# Example class in which there is a message production
|
93
|
+
class UsersBuilder
|
94
|
+
def create(user_details)
|
95
|
+
user = ::User.create!(user_details)
|
96
|
+
|
97
|
+
Karafka.producer.produce_sync(
|
98
|
+
topic: 'users_changes',
|
99
|
+
payload: { user_id: user.id, type: 'user.created' },
|
100
|
+
key: user.id.to_s
|
101
|
+
)
|
102
|
+
|
103
|
+
user
|
80
104
|
end
|
81
105
|
end
|
106
|
+
|
107
|
+
RSpec.describe InlineBatchConsumer do
|
108
|
+
let(:created_user) { UsersBuilder.new.create(user_details) }
|
109
|
+
|
110
|
+
before { created_user }
|
111
|
+
|
112
|
+
it { expect(karafka.produced_messages.size).to eq(1) }
|
113
|
+
it { expect(karafka.produced_messages.first[:topic]).to eq('user.created') }
|
114
|
+
it { expect(karafka.produced_messages.first[:key]).to eq(created_user.id.to_s) }
|
115
|
+
end
|
82
116
|
```
|
83
117
|
|
84
118
|
## Note on contributions
|
@@ -0,0 +1,26 @@
|
|
1
|
+
-----BEGIN CERTIFICATE-----
|
2
|
+
MIIEcDCCAtigAwIBAgIBATANBgkqhkiG9w0BAQsFADA/MRAwDgYDVQQDDAdjb250
|
3
|
+
YWN0MRcwFQYKCZImiZPyLGQBGRYHa2FyYWZrYTESMBAGCgmSJomT8ixkARkWAmlv
|
4
|
+
MB4XDTIyMDgxOTE3MjEzN1oXDTIzMDgxOTE3MjEzN1owPzEQMA4GA1UEAwwHY29u
|
5
|
+
dGFjdDEXMBUGCgmSJomT8ixkARkWB2thcmFma2ExEjAQBgoJkiaJk/IsZAEZFgJp
|
6
|
+
bzCCAaIwDQYJKoZIhvcNAQEBBQADggGPADCCAYoCggGBAODzeO3L6lxdATzMHKNW
|
7
|
+
jFA/GGunoPuylO/BMzy8RiQHh7VIvysAKs0tHhTx3g2D0STDpF+hcQcPELFikiT2
|
8
|
+
F+1wOHj/SsrK7VKqfA8+gq04hKc5sQoX2Egf9k3V0YJ3eZ6R/koHkQ8A0TVt0w6F
|
9
|
+
ZQckoV4MqnEAx0g/FZN3mnHTlJ3VFLSBqJEIe+S6FZMl92mSv+hTrlUG8VaYxSfN
|
10
|
+
lTCvnKk284F6QZq5XIENLRmcDd/3aPBLnLwNnyMyhB+6gK8cUO+CFlDO5tjo/aBA
|
11
|
+
rUnl++wGG0JooF1ed0v+evOn9KoMBG6rHewcf79qJbVOscbD8qSAmo+sCXtcFryr
|
12
|
+
KRMTB8gNbowJkFRJDEe8tfRy11u1fYzFg/qNO82FJd62rKAw2wN0C29yCeQOPRb1
|
13
|
+
Cw9Y4ZwK9VFNEcV9L+3pHTHn2XfuZHtDaG198VweiF6raFO4yiEYccodH/USP0L5
|
14
|
+
cbcCFtmu/4HDSxL1ByQXO84A0ybJuk3/+aPUSXe9C9U8fwIDAQABo3cwdTAJBgNV
|
15
|
+
HRMEAjAAMAsGA1UdDwQEAwIEsDAdBgNVHQ4EFgQUSlcEakb7gfn/5E2WY6z73BF/
|
16
|
+
iZkwHQYDVR0RBBYwFIESY29udGFjdEBrYXJhZmthLmlvMB0GA1UdEgQWMBSBEmNv
|
17
|
+
bnRhY3RAa2FyYWZrYS5pbzANBgkqhkiG9w0BAQsFAAOCAYEA1aS+E7RXJ1w9g9mJ
|
18
|
+
G0NzFxe64OEuENosNlvYQCbRKGCXAU1qqelYkBQHseRgRKxLICrnypRo9IEobyHa
|
19
|
+
vDnJ4r7Tsb34dleqQW2zY/obG+cia3Ym2JsegXWF7dDOzCXJ4FN8MFoT2jHlqLLw
|
20
|
+
yrap0YO5zx0GSQ0Dwy8h2n2v2vanMEeCx7iNm3ERgR5WuN5sjzWoz2A/JLEEcK0C
|
21
|
+
EnAGKCWAd1fuG8IemDjT1edsd5FyYR4bIX0m+99oDuFZyPiiIbalmyYiSBBp59Yb
|
22
|
+
Q0P8zeBi4OfwCZNcxqz0KONmw9JLNv6DgyEAH5xe/4JzhMEgvIRiPj0pHfA7oqQF
|
23
|
+
KUNqvD1KlxbEC+bZfE5IZhnqYLdld/Ksqd22FI1RBhiS1Ejfsj99LVIm9cBuZEY2
|
24
|
+
Qf04B9ceLUaC4fPVEz10FyobjaFoY4i32xRto3XnrzeAgfEe4swLq8bQsR3w/EF3
|
25
|
+
MGU0FeSV2Yj7Xc2x/7BzLK8xQn5l7Yy75iPF+KP3vVmDHnNl
|
26
|
+
-----END CERTIFICATE-----
|
data/karafka-testing.gemspec
CHANGED
@@ -10,7 +10,7 @@ Gem::Specification.new do |spec|
|
|
10
10
|
spec.platform = Gem::Platform::RUBY
|
11
11
|
spec.version = Karafka::Testing::VERSION
|
12
12
|
spec.authors = ['Maciej Mensfeld']
|
13
|
-
spec.email = %w[
|
13
|
+
spec.email = %w[contact@karafka.io]
|
14
14
|
spec.summary = 'Library which provides helpers for easier Karafka consumers tests'
|
15
15
|
spec.description = 'Library which provides helpers for easier Karafka consumers tests'
|
16
16
|
spec.homepage = 'https://karafka.io'
|
@@ -18,7 +18,7 @@ Gem::Specification.new do |spec|
|
|
18
18
|
spec.files = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(spec)/}) }
|
19
19
|
spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
|
20
20
|
spec.require_paths = %w[lib]
|
21
|
-
spec.cert_chain = %w[certs/
|
21
|
+
spec.cert_chain = %w[certs/cert_chain.pem]
|
22
22
|
|
23
23
|
spec.required_ruby_version = '>= 2.7'
|
24
24
|
|
@@ -29,7 +29,7 @@ Gem::Specification.new do |spec|
|
|
29
29
|
spec.add_dependency 'karafka', '>= 2.0', '< 3.0.0'
|
30
30
|
|
31
31
|
spec.metadata = {
|
32
|
-
'source_code_uri' => 'https://github.com/karafka/karafka',
|
32
|
+
'source_code_uri' => 'https://github.com/karafka/karafka-testing',
|
33
33
|
'rubygems_mfa_required' => 'true'
|
34
34
|
}
|
35
35
|
end
|
@@ -7,8 +7,12 @@ module Karafka
|
|
7
7
|
module Errors
|
8
8
|
# Base error for all the internal errors
|
9
9
|
BaseError = Class.new(StandardError)
|
10
|
+
|
10
11
|
# Raised when we want to build a consumer for a topic that does not exist
|
11
12
|
TopicNotFoundError = Class.new(BaseError)
|
13
|
+
|
14
|
+
# Raised when topic is in many consumer groups and not limited by consumer group expectation
|
15
|
+
TopicInManyConsumerGroupsError = Class.new(BaseError)
|
12
16
|
end
|
13
17
|
end
|
14
18
|
end
|
@@ -1,7 +1,9 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
|
+
require 'waterdrop'
|
3
4
|
require 'karafka/testing/errors'
|
4
|
-
require 'karafka/testing/
|
5
|
+
require 'karafka/testing/spec_consumer_client'
|
6
|
+
require 'karafka/testing/spec_producer_client'
|
5
7
|
require 'karafka/testing/rspec/proxy'
|
6
8
|
|
7
9
|
module Karafka
|
@@ -15,12 +17,24 @@ module Karafka
|
|
15
17
|
# Adds all the needed extra functionalities to the rspec group
|
16
18
|
# @param base [Class] RSpec example group we want to extend
|
17
19
|
def included(base)
|
18
|
-
#
|
19
|
-
# we run the consume
|
20
|
-
base.let(:_karafka_messages) { [] }
|
20
|
+
# RSpec local reference to Karafka proxy that allows us to build the consumer instance
|
21
21
|
base.let(:karafka) { Karafka::Testing::RSpec::Proxy.new(self) }
|
22
|
-
|
23
|
-
|
22
|
+
|
23
|
+
# Messages that are targeted to the consumer
|
24
|
+
# You can produce many messages from Karafka during specs and not all should go to the
|
25
|
+
# consumer for processing. This buffer holds only those that should go to consumer
|
26
|
+
base.let(:_karafka_consumer_messages) { [] }
|
27
|
+
# Consumer fake client to mock communication with Kafka
|
28
|
+
base.let(:_karafka_consumer_client) { Karafka::Testing::SpecConsumerClient.new }
|
29
|
+
# Producer fake client to mock communication with Kafka
|
30
|
+
base.let(:_karafka_producer_client) { Karafka::Testing::SpecProducerClient.new(self) }
|
31
|
+
|
32
|
+
base.before do
|
33
|
+
_karafka_consumer_messages.clear
|
34
|
+
_karafka_producer_client.reset
|
35
|
+
|
36
|
+
allow(Karafka.producer).to receive(:client).and_return(_karafka_producer_client)
|
37
|
+
end
|
24
38
|
end
|
25
39
|
end
|
26
40
|
|
@@ -28,6 +42,8 @@ module Karafka
|
|
28
42
|
#
|
29
43
|
# @param requested_topic [String, Symbol] name of the topic for which we want to
|
30
44
|
# create a consumer instance
|
45
|
+
# @param requested_consumer_group [String, Symbol, nil] optional name of the consumer group
|
46
|
+
# if we have multiple consumer groups listening on the same topic
|
31
47
|
# @return [Object] described_class instance
|
32
48
|
# @raise [Karafka::Testing::Errors::TopicNotFoundError] raised when we're unable to find
|
33
49
|
# topic that was requested
|
@@ -36,78 +52,119 @@ module Karafka
|
|
36
52
|
# RSpec.describe MyConsumer do
|
37
53
|
# subject(:consumer) { karafka.consumer_for(:my_requested_topic) }
|
38
54
|
# end
|
39
|
-
def
|
40
|
-
|
55
|
+
def _karafka_consumer_for(requested_topic, requested_consumer_group = nil)
|
56
|
+
all_topics = ::Karafka::App.consumer_groups.map(&:topics).flat_map(&:to_a)
|
41
57
|
|
42
|
-
#
|
43
|
-
|
44
|
-
|
45
|
-
selected_topic = topic if topic.name == requested_topic.to_s
|
46
|
-
end
|
58
|
+
# First select topics that match what we are looking for
|
59
|
+
selected_topics = all_topics.select do |topic|
|
60
|
+
topic.name == requested_topic.to_s
|
47
61
|
end
|
48
62
|
|
49
|
-
|
63
|
+
# And then narrow it down based on the consumer group criteria (if present)
|
64
|
+
selected_topics.delete_if do |topic|
|
65
|
+
requested_consumer_group && topic.consumer_group.name != requested_consumer_group.to_s
|
66
|
+
end
|
50
67
|
|
51
|
-
|
68
|
+
raise Errors::TopicInManyConsumerGroupsError, requested_topic if selected_topics.size > 1
|
69
|
+
raise Errors::TopicNotFoundError, requested_topic if selected_topics.empty?
|
52
70
|
|
53
|
-
|
54
|
-
consumer.topic = selected_topic
|
55
|
-
consumer.producer = Karafka::App.producer
|
56
|
-
consumer.client = Karafka::Testing::DummyClient.new
|
57
|
-
consumer.coordinator = coordinators.find_or_create(requested_topic, 0)
|
58
|
-
consumer
|
71
|
+
_karafka_build_consumer_for(selected_topics.first)
|
59
72
|
end
|
60
73
|
|
61
|
-
# Adds a new Karafka message instance with given payload and options into an
|
62
|
-
# buffer that will be used to simulate messages delivery to the consumer
|
63
|
-
#
|
64
|
-
# @param payload [String] anything you want to send
|
65
|
-
# @param opts [Hash] additional options with which you want to overwrite the
|
66
|
-
# message defaults (key, offset, etc)
|
74
|
+
# Adds a new Karafka message instance if needed with given payload and options into an
|
75
|
+
# internal consumer buffer that will be used to simulate messages delivery to the consumer
|
67
76
|
#
|
77
|
+
# @param message [Hash] message that was sent to Kafka
|
68
78
|
# @example Send a json message to consumer
|
69
79
|
# before do
|
70
|
-
# karafka.
|
80
|
+
# karafka.produce({ 'hello' => 'world' }.to_json)
|
71
81
|
# end
|
72
82
|
#
|
73
83
|
# @example Send a json message to consumer and simulate, that it is partition 6
|
74
84
|
# before do
|
75
|
-
# karafka.
|
85
|
+
# karafka.produce({ 'hello' => 'world' }.to_json, 'partition' => 6)
|
76
86
|
# end
|
77
|
-
def
|
78
|
-
|
79
|
-
|
80
|
-
|
87
|
+
def _karafka_add_message_to_consumer_if_needed(message)
|
88
|
+
# We target to the consumer only messages that were produced to it, since specs may also
|
89
|
+
# produce other messages targeting other topics
|
90
|
+
return unless message[:topic] == consumer.topic.name
|
91
|
+
|
92
|
+
# Build message metadata and copy any metadata that would come from the message
|
93
|
+
metadata = _karafka_message_metadata_defaults
|
94
|
+
|
95
|
+
metadata.keys.each do |key|
|
96
|
+
next unless message.key?(key)
|
97
|
+
|
98
|
+
metadata[key] = message.fetch(key)
|
99
|
+
end
|
81
100
|
|
82
|
-
# Add this message to previously
|
83
|
-
|
101
|
+
# Add this message to previously produced messages
|
102
|
+
_karafka_consumer_messages << Karafka::Messages::Message.new(
|
103
|
+
message[:payload],
|
104
|
+
Karafka::Messages::Metadata.new(metadata).freeze
|
105
|
+
)
|
84
106
|
|
85
107
|
# Update batch metadata
|
86
108
|
batch_metadata = Karafka::Messages::Builders::BatchMetadata.call(
|
87
|
-
|
109
|
+
_karafka_consumer_messages,
|
88
110
|
subject.topic,
|
89
111
|
Time.now
|
90
112
|
)
|
91
113
|
|
92
114
|
# Update consumer messages batch
|
93
|
-
subject.messages = Karafka::Messages::Messages.new(
|
115
|
+
subject.messages = Karafka::Messages::Messages.new(
|
116
|
+
_karafka_consumer_messages,
|
117
|
+
batch_metadata
|
118
|
+
)
|
119
|
+
end
|
120
|
+
|
121
|
+
# Produces message with a given payload to the consumer matching topic
|
122
|
+
# @param payload [String] payload we want to dispatch
|
123
|
+
# @param metadata [Hash] any metadata we want to dispatch alongside the payload
|
124
|
+
def _karafka_produce(payload, metadata = {})
|
125
|
+
Karafka.producer.produce_sync(
|
126
|
+
{
|
127
|
+
topic: subject.topic.name,
|
128
|
+
payload: payload
|
129
|
+
}.merge(metadata)
|
130
|
+
)
|
131
|
+
end
|
132
|
+
|
133
|
+
# @return [Array<Hash>] messages that were produced
|
134
|
+
def _karafka_produced_messages
|
135
|
+
_karafka_producer_client.messages
|
94
136
|
end
|
95
137
|
|
96
138
|
private
|
97
139
|
|
98
140
|
# @return [Hash] message default options
|
99
|
-
def
|
141
|
+
def _karafka_message_metadata_defaults
|
100
142
|
{
|
101
143
|
deserializer: subject.topic.deserializer,
|
102
144
|
timestamp: Time.now,
|
103
145
|
headers: {},
|
104
146
|
key: nil,
|
105
|
-
offset:
|
147
|
+
offset: _karafka_consumer_messages.size,
|
106
148
|
partition: 0,
|
107
149
|
received_at: Time.now,
|
108
150
|
topic: subject.topic.name
|
109
151
|
}
|
110
152
|
end
|
153
|
+
|
154
|
+
# Builds the consumer instance based on the provided topic
|
155
|
+
#
|
156
|
+
# @param topic [Karafka::Routing::Topic] topic for which we want to build the consumer
|
157
|
+
# @return [Object] karafka consumer
|
158
|
+
def _karafka_build_consumer_for(topic)
|
159
|
+
coordinators = Karafka::Processing::CoordinatorsBuffer.new
|
160
|
+
|
161
|
+
consumer = described_class.new
|
162
|
+
consumer.topic = topic
|
163
|
+
consumer.producer = Karafka::App.producer
|
164
|
+
consumer.client = _karafka_consumer_client
|
165
|
+
consumer.coordinator = coordinators.find_or_create(topic.name, 0)
|
166
|
+
consumer
|
167
|
+
end
|
111
168
|
end
|
112
169
|
end
|
113
170
|
end
|
@@ -4,20 +4,26 @@ module Karafka
|
|
4
4
|
module Testing
|
5
5
|
module RSpec
|
6
6
|
# Proxy object for a nicer `karafka.` API within RSpec
|
7
|
+
# None other should be used by the end users
|
7
8
|
class Proxy
|
8
9
|
# @param rspec_example [RSpec::ExampleGroups] rspec context
|
9
10
|
def initialize(rspec_example)
|
10
11
|
@rspec_example = rspec_example
|
11
12
|
end
|
12
13
|
|
13
|
-
# @param args Anything that the `#
|
14
|
+
# @param args Anything that the `#_karafka_consumer_for` accepts
|
14
15
|
def consumer_for(*args)
|
15
|
-
@rspec_example.
|
16
|
+
@rspec_example._karafka_consumer_for(*args)
|
16
17
|
end
|
17
18
|
|
18
|
-
# @param args Anything that
|
19
|
-
def
|
20
|
-
@rspec_example.
|
19
|
+
# @param args Anything that `#_karafka_produce` accepts
|
20
|
+
def produce(*args)
|
21
|
+
@rspec_example._karafka_produce(*args)
|
22
|
+
end
|
23
|
+
|
24
|
+
# @return [Array<Hash>] messages produced via `Karafka#producer`
|
25
|
+
def produced_messages
|
26
|
+
@rspec_example._karafka_produced_messages
|
21
27
|
end
|
22
28
|
end
|
23
29
|
end
|
@@ -2,10 +2,10 @@
|
|
2
2
|
|
3
3
|
module Karafka
|
4
4
|
module Testing
|
5
|
-
# A
|
5
|
+
# A spec client that takes over client delegated methods from the consumers
|
6
6
|
# For specs we do not mark anything as consumed, nor do we really send heartbeats.
|
7
7
|
# Those things are tested in the framework itself
|
8
|
-
class
|
8
|
+
class SpecConsumerClient
|
9
9
|
%i[
|
10
10
|
mark_as_consumed
|
11
11
|
mark_as_consumed!
|
@@ -0,0 +1,43 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Karafka
|
4
|
+
module Testing
|
5
|
+
# Spec producer client used to buffer messages that we send out in specs
|
6
|
+
class SpecProducerClient < ::WaterDrop::Producer::DummyClient
|
7
|
+
attr_accessor :messages
|
8
|
+
|
9
|
+
# Sync fake response for the message delivery to Kafka, since we do not dispatch anything
|
10
|
+
class SyncResponse
|
11
|
+
# @param _args Handler wait arguments (irrelevant as waiting is fake here)
|
12
|
+
def wait(*_args)
|
13
|
+
false
|
14
|
+
end
|
15
|
+
end
|
16
|
+
|
17
|
+
# @param rspec [RSpec::Core::ExampleGroup] rspec example we need to hold to trigger actions
|
18
|
+
# on it that are rspec context aware
|
19
|
+
def initialize(rspec)
|
20
|
+
super()
|
21
|
+
@rspec = rspec
|
22
|
+
self.messages = []
|
23
|
+
end
|
24
|
+
|
25
|
+
# "Produces" message to Kafka. That is, it acknowledges it locally, adds it to the internal
|
26
|
+
# buffer and adds it (if needed) into the current consumer messages buffer
|
27
|
+
# @param message [Hash] `Karafka.producer.produce_sync` message hash
|
28
|
+
def produce(message)
|
29
|
+
messages << message
|
30
|
+
|
31
|
+
@rspec._karafka_add_message_to_consumer_if_needed(message)
|
32
|
+
|
33
|
+
SyncResponse.new
|
34
|
+
end
|
35
|
+
|
36
|
+
# Clears internal buffer
|
37
|
+
# Used in between specs so messages do not leak out
|
38
|
+
def reset
|
39
|
+
messages.clear
|
40
|
+
end
|
41
|
+
end
|
42
|
+
end
|
43
|
+
end
|
data.tar.gz.sig
CHANGED
Binary file
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: karafka-testing
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 2.0.
|
4
|
+
version: 2.0.2
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Maciej Mensfeld
|
@@ -10,31 +10,32 @@ bindir: bin
|
|
10
10
|
cert_chain:
|
11
11
|
- |
|
12
12
|
-----BEGIN CERTIFICATE-----
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
|
24
|
-
|
25
|
-
|
26
|
-
|
27
|
-
|
28
|
-
|
29
|
-
|
30
|
-
|
31
|
-
/
|
32
|
-
|
33
|
-
|
34
|
-
|
35
|
-
|
13
|
+
MIIEcDCCAtigAwIBAgIBATANBgkqhkiG9w0BAQsFADA/MRAwDgYDVQQDDAdjb250
|
14
|
+
YWN0MRcwFQYKCZImiZPyLGQBGRYHa2FyYWZrYTESMBAGCgmSJomT8ixkARkWAmlv
|
15
|
+
MB4XDTIyMDgxOTE3MjEzN1oXDTIzMDgxOTE3MjEzN1owPzEQMA4GA1UEAwwHY29u
|
16
|
+
dGFjdDEXMBUGCgmSJomT8ixkARkWB2thcmFma2ExEjAQBgoJkiaJk/IsZAEZFgJp
|
17
|
+
bzCCAaIwDQYJKoZIhvcNAQEBBQADggGPADCCAYoCggGBAODzeO3L6lxdATzMHKNW
|
18
|
+
jFA/GGunoPuylO/BMzy8RiQHh7VIvysAKs0tHhTx3g2D0STDpF+hcQcPELFikiT2
|
19
|
+
F+1wOHj/SsrK7VKqfA8+gq04hKc5sQoX2Egf9k3V0YJ3eZ6R/koHkQ8A0TVt0w6F
|
20
|
+
ZQckoV4MqnEAx0g/FZN3mnHTlJ3VFLSBqJEIe+S6FZMl92mSv+hTrlUG8VaYxSfN
|
21
|
+
lTCvnKk284F6QZq5XIENLRmcDd/3aPBLnLwNnyMyhB+6gK8cUO+CFlDO5tjo/aBA
|
22
|
+
rUnl++wGG0JooF1ed0v+evOn9KoMBG6rHewcf79qJbVOscbD8qSAmo+sCXtcFryr
|
23
|
+
KRMTB8gNbowJkFRJDEe8tfRy11u1fYzFg/qNO82FJd62rKAw2wN0C29yCeQOPRb1
|
24
|
+
Cw9Y4ZwK9VFNEcV9L+3pHTHn2XfuZHtDaG198VweiF6raFO4yiEYccodH/USP0L5
|
25
|
+
cbcCFtmu/4HDSxL1ByQXO84A0ybJuk3/+aPUSXe9C9U8fwIDAQABo3cwdTAJBgNV
|
26
|
+
HRMEAjAAMAsGA1UdDwQEAwIEsDAdBgNVHQ4EFgQUSlcEakb7gfn/5E2WY6z73BF/
|
27
|
+
iZkwHQYDVR0RBBYwFIESY29udGFjdEBrYXJhZmthLmlvMB0GA1UdEgQWMBSBEmNv
|
28
|
+
bnRhY3RAa2FyYWZrYS5pbzANBgkqhkiG9w0BAQsFAAOCAYEA1aS+E7RXJ1w9g9mJ
|
29
|
+
G0NzFxe64OEuENosNlvYQCbRKGCXAU1qqelYkBQHseRgRKxLICrnypRo9IEobyHa
|
30
|
+
vDnJ4r7Tsb34dleqQW2zY/obG+cia3Ym2JsegXWF7dDOzCXJ4FN8MFoT2jHlqLLw
|
31
|
+
yrap0YO5zx0GSQ0Dwy8h2n2v2vanMEeCx7iNm3ERgR5WuN5sjzWoz2A/JLEEcK0C
|
32
|
+
EnAGKCWAd1fuG8IemDjT1edsd5FyYR4bIX0m+99oDuFZyPiiIbalmyYiSBBp59Yb
|
33
|
+
Q0P8zeBi4OfwCZNcxqz0KONmw9JLNv6DgyEAH5xe/4JzhMEgvIRiPj0pHfA7oqQF
|
34
|
+
KUNqvD1KlxbEC+bZfE5IZhnqYLdld/Ksqd22FI1RBhiS1Ejfsj99LVIm9cBuZEY2
|
35
|
+
Qf04B9ceLUaC4fPVEz10FyobjaFoY4i32xRto3XnrzeAgfEe4swLq8bQsR3w/EF3
|
36
|
+
MGU0FeSV2Yj7Xc2x/7BzLK8xQn5l7Yy75iPF+KP3vVmDHnNl
|
36
37
|
-----END CERTIFICATE-----
|
37
|
-
date: 2022-
|
38
|
+
date: 2022-09-29 00:00:00.000000000 Z
|
38
39
|
dependencies:
|
39
40
|
- !ruby/object:Gem::Dependency
|
40
41
|
name: karafka
|
@@ -58,7 +59,7 @@ dependencies:
|
|
58
59
|
version: 3.0.0
|
59
60
|
description: Library which provides helpers for easier Karafka consumers tests
|
60
61
|
email:
|
61
|
-
-
|
62
|
+
- contact@karafka.io
|
62
63
|
executables: []
|
63
64
|
extensions: []
|
64
65
|
extra_rdoc_files: []
|
@@ -76,20 +77,21 @@ files:
|
|
76
77
|
- Gemfile.lock
|
77
78
|
- MIT-LICENSE
|
78
79
|
- README.md
|
79
|
-
- certs/
|
80
|
+
- certs/cert_chain.pem
|
80
81
|
- karafka-testing.gemspec
|
81
82
|
- lib/karafka-testing.rb
|
82
83
|
- lib/karafka/testing.rb
|
83
|
-
- lib/karafka/testing/dummy_client.rb
|
84
84
|
- lib/karafka/testing/errors.rb
|
85
85
|
- lib/karafka/testing/rspec/helpers.rb
|
86
86
|
- lib/karafka/testing/rspec/proxy.rb
|
87
|
+
- lib/karafka/testing/spec_consumer_client.rb
|
88
|
+
- lib/karafka/testing/spec_producer_client.rb
|
87
89
|
- lib/karafka/testing/version.rb
|
88
90
|
homepage: https://karafka.io
|
89
91
|
licenses:
|
90
92
|
- MIT
|
91
93
|
metadata:
|
92
|
-
source_code_uri: https://github.com/karafka/karafka
|
94
|
+
source_code_uri: https://github.com/karafka/karafka-testing
|
93
95
|
rubygems_mfa_required: 'true'
|
94
96
|
post_install_message:
|
95
97
|
rdoc_options: []
|
metadata.gz.sig
CHANGED
Binary file
|
data/certs/mensfeld.pem
DELETED
@@ -1,25 +0,0 @@
|
|
1
|
-
-----BEGIN CERTIFICATE-----
|
2
|
-
MIIEODCCAqCgAwIBAgIBATANBgkqhkiG9w0BAQsFADAjMSEwHwYDVQQDDBhtYWNp
|
3
|
-
ZWovREM9bWVuc2ZlbGQvREM9cGwwHhcNMjEwODExMTQxNTEzWhcNMjIwODExMTQx
|
4
|
-
NTEzWjAjMSEwHwYDVQQDDBhtYWNpZWovREM9bWVuc2ZlbGQvREM9cGwwggGiMA0G
|
5
|
-
CSqGSIb3DQEBAQUAA4IBjwAwggGKAoIBgQDV2jKH4Ti87GM6nyT6D+ESzTI0MZDj
|
6
|
-
ak2/TEwnxvijMJyCCPKT/qIkbW4/f0VHM4rhPr1nW73sb5SZBVFCLlJcOSKOBdUY
|
7
|
-
TMY+SIXN2EtUaZuhAOe8LxtxjHTgRHvHcqUQMBENXTISNzCo32LnUxweu66ia4Pd
|
8
|
-
1mNRhzOqNv9YiBZvtBf7IMQ+sYdOCjboq2dlsWmJiwiDpY9lQBTnWORnT3mQxU5x
|
9
|
-
vPSwnLB854cHdCS8fQo4DjeJBRZHhEbcE5sqhEMB3RZA3EtFVEXOxlNxVTS3tncI
|
10
|
-
qyNXiWDaxcipaens4ObSY1C2HTV7OWb7OMqSCIybeYTSfkaSdqmcl4S6zxXkjH1J
|
11
|
-
tnjayAVzD+QVXGijsPLE2PFnJAh9iDET2cMsjabO1f6l1OQNyAtqpcyQcgfnyW0z
|
12
|
-
g7tGxTYD+6wJHffM9d9txOUw6djkF6bDxyqB8lo4Z3IObCx18AZjI9XPS9QG7w6q
|
13
|
-
LCWuMG2lkCcRgASqaVk9fEf9yMc2xxz5o3kCAwEAAaN3MHUwCQYDVR0TBAIwADAL
|
14
|
-
BgNVHQ8EBAMCBLAwHQYDVR0OBBYEFBqUFCKCOe5IuueUVqOB991jyCLLMB0GA1Ud
|
15
|
-
EQQWMBSBEm1hY2llakBtZW5zZmVsZC5wbDAdBgNVHRIEFjAUgRJtYWNpZWpAbWVu
|
16
|
-
c2ZlbGQucGwwDQYJKoZIhvcNAQELBQADggGBADD0/UuTTFgW+CGk2U0RDw2RBOca
|
17
|
-
W2LTF/G7AOzuzD0Tc4voc7WXyrgKwJREv8rgBimLnNlgmFJLmtUCh2U/MgxvcilH
|
18
|
-
yshYcbseNvjkrtYnLRlWZR4SSB6Zei5AlyGVQLPkvdsBpNegcG6w075YEwzX/38a
|
19
|
-
8V9B/Yri2OGELBz8ykl7BsXUgNoUPA/4pHF6YRLz+VirOaUIQ4JfY7xGj6fSOWWz
|
20
|
-
/rQ/d77r6o1mfJYM/3BRVg73a3b7DmRnE5qjwmSaSQ7u802pJnLesmArch0xGCT/
|
21
|
-
fMmRli1Qb+6qOTl9mzD6UDMAyFR4t6MStLm0mIEqM0nBO5nUdUWbC7l9qXEf8XBE
|
22
|
-
2DP28p3EqSuS+lKbAWKcqv7t0iRhhmaod+Yn9mcrLN1sa3q3KSQ9BCyxezCD4Mk2
|
23
|
-
R2P11bWoCtr70BsccVrN8jEhzwXngMyI2gVt750Y+dbTu1KgRqZKp/ECe7ZzPzXj
|
24
|
-
pIy9vHxTANKYVyI4qj8OrFdEM5BQNu8oQpL0iQ==
|
25
|
-
-----END CERTIFICATE-----
|