event_store_subscriptions 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: f261c1fb47c48ba5354b9d02c1b1a066e64813e397933657bbb75c396ec06f85
4
+ data.tar.gz: ce40a5265ba3d2bc34f4392cae838df08a471c13c85bbecc292dd53628cf464f
5
+ SHA512:
6
+ metadata.gz: 97ce25b1bbcd50a345b2be4ed7f1b98048f540732a91eafff55d36c31d6d513ad759614938dba6a07139a448d0e2c3b35bdfc1bb1e96b77ee2de51643b07b84d
7
+ data.tar.gz: 995f4264d8a301b8dae0156ab0d08fbe3ca41744194b4357e6fddba9d29b73deec207d98d6c9859fad8f5cbc41e3204580c163d76b61e8f8ea4e92c814ab8ded
data/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2022 Yousty AG
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,284 @@
1
+ ![Run tests](https://github.com/yousty/event_store_subscriptions/workflows/Run%20tests/badge.svg?branch=main&event=push)
2
+ [![Gem Version](https://badge.fury.io/rb/event_store_subscriptions.svg)](https://badge.fury.io/rb/event_store_subscriptions)
3
+
4
+ # EventStoreSubscriptions
5
+
6
+ Extends the functionality of the [EventStoreDB ruby client](https://github.com/yousty/event_store_client) with a catch-up subscriptions manager.
7
+
8
+ By default `event_store_client` implements thread-blocking methods to subscribe to a stream. Those are `#subscribe_to_stream` and `#subscribe_to_all`. In order to subscribe to many streams/events, you need to implement asynchronous subscriptions on your own. This gem solves this task by putting each subscription into its own thread.
9
+
10
+ The thread-based implementation has a downside: any IO operation in your subscription's handlers will block all other threads. So it is important to consider how many subscriptions you put into a single process. There is a plan to integrate Ractors instead/alongside threads to provide the option to eliminate the IO-blocking issue.
11
+
12
+ ## Installation
13
+
14
+ Add this line to your application's Gemfile:
15
+
16
+ ```ruby
17
+ gem 'event_store_subscriptions'
18
+ ```
19
+
20
+ And then execute:
21
+
22
+ $ bundle install
23
+
24
+ Or install it yourself as:
25
+
26
+ $ gem install event_store_subscriptions
27
+
28
+ ## Usage
29
+
30
+ Use the `#create` and `#create_for_all` methods to subscribe to a stream. For the full list of available arguments see the documentation of the `EventStoreClient::GRPC::Client#subscribe_to_stream` method in the [event_store_client gem docs](https://rubydoc.info/gems/event_store_client). You may also want to check the [Catch-up subscriptions](https://github.com/yousty/event_store_client/blob/master/docs/catch_up_subscriptions.md) section as well.
31
+
32
+ ### Subscribing to a specific stream
33
+
34
+ Use the `#create` method in order to subscribe to specific stream:
35
+
36
+ ```ruby
37
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
38
+ handler = proc do |resp|
39
+ if resp.success?
40
+ do_something_with_resp(resp.success) # retrieve an event
41
+ else # resp.failure? => true
42
+ handle_failure(resp.failure)
43
+ end
44
+ end
45
+ subscriptions.create('some-stream', handler: handler)
46
+ subscriptions.listen_all
47
+ ```
48
+
49
+ You may provide any object which responds to `#call` as a handler:
50
+
51
+ ```ruby
52
+ class SomeStreamHandler
53
+ def call(resp)
54
+ if resp.success?
55
+ do_something_with_resp(resp.success) # retrieve an event
56
+ else # resp.failure? => true
57
+ handle_failure(resp.failure)
58
+ end
59
+ end
60
+ end
61
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
62
+ subscriptions.create('some-stream', handler: SomeStreamHandler.new)
63
+ subscriptions.listen_all
64
+ ```
65
+
66
+ ### Subscribing to the $all stream
67
+
68
+ Use the `#create_for_all` method to subscribe to the all stream:
69
+
70
+ ```ruby
71
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
72
+ handler = proc do |resp|
73
+ if resp.success?
74
+ do_something_with_resp(resp.success) # retrieve an event
75
+ else # resp.failure? => true
76
+ handle_failure(resp.failure)
77
+ end
78
+ end
79
+ subscriptions.create_for_all(handler: handler)
80
+ subscriptions.listen_all
81
+ ```
82
+
83
+ You may also explicitly pass `"$all"` stream name to the `#create` method:
84
+
85
+ ```ruby
86
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
87
+ handler = proc do |resp|
88
+ if resp.success?
89
+ do_something_with_resp(resp.success) # retrieve an event
90
+ else # resp.failure? => true
91
+ handle_failure(resp.failure)
92
+ end
93
+ end
94
+ subscriptions.create('$all', handler: handler)
95
+ subscriptions.listen_all
96
+ ```
97
+
98
+ ### Handling Subscription position updates
99
+
100
+ You may want to add a handler that will be executed each time a subscription gets position updates. Such updates happen when new events are added to the stream or when EventStore DB produces a checkpoint response.
101
+
102
+ #### Listening for position updates of a specific stream
103
+
104
+ A handler registered to receive position updates of a specific stream is called with the `EventStoreSubscriptions::SubscriptionRevision` class instance. It holds the current revision of the stream.
105
+
106
+ ```ruby
107
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
108
+ subscription = subscriptions.create('some-stream', handler: proc { |r| p r })
109
+ subscription.position.register_update_hook do |position|
110
+ puts "Current revision is #{position.revision}"
111
+ end
112
+ subscription.listen
113
+ ```
114
+
115
+ #### Listening for position updates of the $all stream
116
+
117
+ A handler registered to receive position updates of the `$all` stream is called with the `EventStoreSubscriptions::SubscriptionPosition` class instance. It holds the current `commit_position` and `prepare_position` of the `$all` stream.
118
+
119
+ ```ruby
120
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
121
+ subscription = subscriptions.create_for_all(handler: proc { |r| p r })
122
+ subscription.position.register_update_hook do |position|
123
+ puts "Current commit/prepare positions are #{position.commit_position}/#{position.prepare_position}"
124
+ end
125
+ subscription.listen
126
+ ```
127
+
128
+ ### Automatic restart of failed Subscriptions
129
+
130
+ This gem provides a possibility to watch over your subscription collections and restart a subscription in case it failed. Subscriptions may fail because an exception was raised in the handler or in the position update hook. A new subscription will be started, listening from the position the failed subscription has stopped.
131
+
132
+ Start watching over your subscriptions' collection:
133
+
134
+ ```ruby
135
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
136
+ subscriptions.create_for_all(handler: proc { |r| p r })
137
+ EventStoreSubscriptions::WatchDog.watch(subscriptions)
138
+ subscriptions.listen_all
139
+ ```
140
+
141
+ ### Async nature of this gem
142
+
143
+ `EventStoreSubscriptions::Subscriptions#listen_all`, `EventStoreSubscriptions::Subscriptions#stop_all`, `EventStoreSubscriptions::Subscription#listen`, `EventStoreSubscriptions::Subscription#stop_listening`, `EventStoreSubscriptions::WatchDog#watch`, `EventStoreSubscriptions::WatchDog#unwatch` methods are asynchronous. This means that they spawn thread that performs proper task in the background.
144
+
145
+ `EventStoreSubscriptions::Subscriptions#stop_all`, `EventStoreSubscriptions::Subscription#stop_listening` and `EventStoreSubscriptions::WatchDog#unwatch` methods has ending run time, meaning that they runners won't run forever.
146
+
147
+ `EventStoreSubscriptions::Subscriptions#listen_all`, `EventStoreSubscriptions::Subscription#listen` and `EventStoreSubscriptions::WatchDog#watch` methods will run forever.
148
+
149
+ In order to stop running `Subscription` or `WatchDog` you should initiate stop process and wait for finish.
150
+
151
+ #### Stopping Subscription
152
+
153
+ For single subscription:
154
+
155
+ ```ruby
156
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
157
+ subscription = subscriptions.create_for_all(handler: proc { |r| p r })
158
+ subscription.listen
159
+
160
+ # Initiate Subscription shutdown
161
+ subscription.stop_listening
162
+ # Wait for Subscription to finish. This will block current Thread.
163
+ subscription.wait_for_finish
164
+ ```
165
+
166
+ For the entire collection:
167
+
168
+ ```ruby
169
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
170
+ subscriptions.create_for_all(handler: proc { |r| p r })
171
+ subscriptions.listen_all
172
+
173
+ # Initiate shutdown for each Subscription in the collection
174
+ subscriptions.stop_all
175
+ # Wait for all Subscriptions to finish. This will block current Thread.
176
+ subscriptions.subscriptions.each(&:wait_for_finish)
177
+ ```
178
+
179
+ #### Stopping WatchDog
180
+
181
+ ```ruby
182
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
183
+ watcher = EventStoreSubscriptions::WatchDog.watch(subscriptions)
184
+
185
+ # Initiate WatchDog shutdown
186
+ watcher.unwatch
187
+ # Wait for WatchDog to finish. This will block current Thread.
188
+ watcher.wait_for_finish
189
+ ```
190
+
191
+ ### Graceful shutdown
192
+
193
+ You may want to gracefully shut down the process that handles the subscriptions. In order to do so, you should define a `Kernel.trap` handler to handle your kill signal:
194
+
195
+ ```ruby
196
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
197
+ subscriptions.create_for_all(handler: proc { |r| p r })
198
+ watcher = EventStoreSubscriptions::WatchDog.watch(subscriptions)
199
+ subscriptions.listen_all
200
+
201
+ Kernel.trap('TERM') do
202
+ # Because the implementation uses Mutex - wrap it into Thread to bypass the limitations of
203
+ # Kernel#trap
204
+ Thread.new do
205
+ # Initiate graceful shutdown. Need to shutdown watcher first, and then - subscriptions
206
+ watcher.unwatch.wait_for_finish
207
+ subscriptions.stop_all.each(&:wait_for_finish)
208
+ end.join
209
+ exit
210
+ end
211
+
212
+ # Wait while Subscriptions are working
213
+ subscriptions.each(&:wait_for_finish)
214
+ ```
215
+
216
+ Now just send the `TERM` signal if you want to gracefully shut down your process:
217
+
218
+ ```bash
219
+ kill -TERM <pid of your process>
220
+ ```
221
+
222
+ ### Monitoring Subscriptions
223
+
224
+ After you started listening your Subscriptions, you may want to monitor status of them. There is various built-in statistics which you can get.
225
+
226
+ ```ruby
227
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
228
+ subscriptions.create_for_all(handler: proc { |r| p r })
229
+ watcher = EventStoreSubscriptions::WatchDog.watch(subscriptions)
230
+ subscriptions.listen_all
231
+
232
+ loop do
233
+ sleep 1
234
+ subscriptions.subscriptions.each do |subscription|
235
+ puts "Current state is: #{subscription.state}"
236
+ puts "Current position: #{subscription.position.to_h}"
237
+ puts "Last error: #{subscription.statistic.last_error.inspect}"
238
+ puts "Last restart was at: #{subscription.statistic.last_restart_at || 'Never'}"
239
+ puts "Total errors/restarts: #{subscription.statistic.errors_count}"
240
+ puts "Events processed: #{subscription.statistic.events_processed}"
241
+ puts "Current watcher state is: #{watcher.state}"
242
+ end
243
+ end
244
+ ```
245
+
246
+ ### WatchDog and control of restart condition of Subscriptions
247
+
248
+ You may want to decide yourself whether `WhatchDog` should restart a `Subscription`. You can do so by providing a proc which, if thruthy result is returned, skips the restart of `Subscription`.
249
+
250
+ ```ruby
251
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
252
+ subscriptions.create_for_all(handler: proc { |r| p r })
253
+ # Do not restart Subscription if its id is even
254
+ restart_terminator = proc { |sub| sub.__id__ % 2 == 0 }
255
+ EventStoreSubscriptions::WatchDog.watch(subscriptions, restart_terminator: restart_terminator)
256
+ subscriptions.listen_all
257
+ ```
258
+
259
+ ## Development
260
+
261
+ You will have to install Docker first. It is needed to run EventStore DB. You can run EventStore DB with this command:
262
+
263
+ ```shell
264
+ docker-compose -f docker-compose-cluster.yml up
265
+ ```
266
+
267
+ Now you can enter a dev console by running `bin/console` or run tests by running the `rspec` command.
268
+
269
+ ## Contributing
270
+
271
+ Bug reports and pull requests are welcome on GitHub at https://github.com/yousty/event_store_subscriptions. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/yousty/event_store_subscriptions/blob/master/CODE_OF_CONDUCT.md).
272
+
273
+ ### Publishing new version
274
+
275
+ 1. Push commit with updated `version.rb` file to the `release` branch. The new version will be automatically pushed to [rubygems](https://rubygems.org).
276
+ 2. Create release on GitHub including change log.
277
+
278
+ ## License
279
+
280
+ The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
281
+
282
+ ## Code of Conduct
283
+
284
+ Everyone interacting in the EventStoreSubscriptions project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/yousty/event_store_subscriptions/blob/master/CODE_OF_CONDUCT.md).
@@ -0,0 +1,9 @@
1
+ # frozen_string_literal: true
2
+
3
+ module EventStoreSubscriptions
4
+ class Error < StandardError
5
+ end
6
+
7
+ class ThreadNotDeadError < Error
8
+ end
9
+ end
@@ -0,0 +1,36 @@
1
+ # frozen_string_literal: true
2
+
3
+ module EventStoreSubscriptions
4
+ # Defines various states. It is used to set and get current object's state.
5
+ class ObjectState
6
+ attr_accessor :state
7
+ attr_reader :semaphore
8
+ private :state, :state=, :semaphore
9
+
10
+ STATES = %i(initial running halting stopped dead).freeze
11
+
12
+ def initialize
13
+ @semaphore = Thread::Mutex.new
14
+ initial!
15
+ end
16
+
17
+ STATES.each do |state|
18
+ # Checks whether the object is in appropriate state
19
+ # @return [Boolean]
20
+ define_method "#{state}?" do
21
+ semaphore.synchronize { self.state == state }
22
+ end
23
+
24
+ # Sets the state.
25
+ # @return [Symbol]
26
+ define_method "#{state}!" do
27
+ semaphore.synchronize { self.state = state }
28
+ end
29
+ end
30
+
31
+ # @return [String] string representation of the #state
32
+ def to_s
33
+ state.to_s
34
+ end
35
+ end
36
+ end
@@ -0,0 +1,143 @@
1
+ # frozen_string_literal: true
2
+
3
+ module EventStoreSubscriptions
4
+ class Subscription
5
+ include WaitForFinish
6
+
7
+ FORCED_SHUTDOWN_DELAY = 60 # seconds
8
+
9
+ attr_accessor :runner
10
+ attr_reader :client, :setup, :state, :position, :statistic
11
+ private :runner, :runner=
12
+
13
+ # @param position [EventStoreSubscriptions::SubscriptionPosition, EventStoreSubscriptions::SubscriptionRevision]
14
+ # @param client [EventStoreClient::GRPC::Client]
15
+ # @param setup [EventStoreSubscriptions::SubscriptionSetup]
16
+ # @param statistic [EventStoreSubscriptions::SubscriptionStatistic]
17
+ def initialize(position:, client:, setup:, statistic: SubscriptionStatistic.new)
18
+ @position = position
19
+ @client = client
20
+ @setup = setup
21
+ @state = ObjectState.new
22
+ @statistic = statistic
23
+ @runner = nil
24
+ end
25
+
26
+ # Start listening for the events
27
+ # @return [EventStoreSubscriptions::Subscription] returns self
28
+ def listen
29
+ self.runner ||=
30
+ begin
31
+ state.running!
32
+ Thread.new do
33
+ Thread.current.abort_on_exception = false
34
+ Thread.current.report_on_exception = false
35
+ client.subscribe_to_stream(
36
+ *setup.args,
37
+ **adjusted_kwargs,
38
+ &setup.blk
39
+ )
40
+ rescue StandardError => e
41
+ statistic.last_error = e
42
+ statistic.errors_count += 1
43
+ state.dead!
44
+ raise
45
+ end
46
+ end
47
+ self
48
+ end
49
+
50
+ # Stops listening for events. This command is async - the result is not immediate. Use the #wait_for_finish
51
+ # method to wait until the runner has fully stopped.
52
+ # @return [EventStoreSubscriptions::Subscription] returns self
53
+ def stop_listening
54
+ return self unless runner&.alive?
55
+
56
+ state.halting!
57
+ Thread.new do
58
+ stopping_at = Time.now.utc
59
+ loop do
60
+ # Give Subscription up to FORCED_SHUTDOWN_DELAY seconds for graceful shutdown
61
+ runner&.exit if Time.now.utc - stopping_at > FORCED_SHUTDOWN_DELAY
62
+
63
+ unless runner&.alive?
64
+ state.stopped!
65
+ self.runner = nil
66
+ break
67
+ end
68
+ sleep 0.1
69
+ end
70
+ end
71
+ self
72
+ end
73
+
74
+ # Removes all properties of object and freezes it. You can't delete currently running
75
+ # Subscription though. You must stop it first.
76
+ # @return [EventStoreSubscriptions::Subscription] frozen object
77
+ # @raise [EventStoreSubscriptions::ThreadNotDeadError] raises this error in case runner Thread
78
+ # is still alive for some reason. Normally this should never happen.
79
+ def delete
80
+ if runner&.alive?
81
+ raise ThreadNotDeadError, "Can not delete alive Subscription #{self.inspect}"
82
+ end
83
+
84
+ instance_variables.each { |var| instance_variable_set(var, nil) }
85
+ freeze
86
+ end
87
+
88
+ private
89
+
90
+ # Wraps original handler into our own handler to provide extended functionality.
91
+ # @param original_handler [#call]
92
+ # @return [Proc]
93
+ def handler(original_handler)
94
+ proc do |result|
95
+ Thread.current.exit unless state.running?
96
+ original_result = result.success
97
+ result = EventStoreClient::GRPC::Shared::Streams::ProcessResponse.new.call(
98
+ original_result,
99
+ *process_response_args
100
+ )
101
+ original_handler.call(result) if result
102
+ statistic.events_processed += 1
103
+ position.update(original_result)
104
+ end
105
+ end
106
+
107
+ # Calculates "skip_deserialization" and "skip_decryption" arguments for the ProcessResponse
108
+ # class. Since we have overridden the original handler, we need to calculate the correct argument values
109
+ # to process the response on our own. This method implements the same behavior as
110
+ # the event_store_client gem implements (EventStoreClient::GRPC::Client#subscribe_to_stream
111
+ # method).
112
+ # @return [Array<Boolean>]
113
+ def process_response_args
114
+ skip_deserialization =
115
+ if setup.kwargs.key?(:skip_deserialization)
116
+ setup.kwargs[:skip_deserialization]
117
+ else
118
+ client.config.skip_deserialization
119
+ end
120
+ skip_decryption =
121
+ if setup.kwargs.key?(:skip_decryption)
122
+ setup.kwargs[:skip_decryption]
123
+ else
124
+ client.config.skip_decryption
125
+ end
126
+ [skip_deserialization, skip_decryption]
127
+ end
128
+
129
+ # Override keyword arguments, provided by dev in EventStoreSubscriptions::Subscriptions#create
130
+ # or EventStoreSubscriptions::Subscriptions#create_for_all methods. This is needed to provide
131
+ # our own handler and to override the starting position of the given stream.
132
+ # @return [Hash]
133
+ def adjusted_kwargs
134
+ kwargs = setup.dup.kwargs
135
+ kwargs.merge!(handler: handler(kwargs[:handler]), skip_deserialization: true)
136
+ return kwargs unless position.present?
137
+
138
+ kwargs[:options] ||= {}
139
+ kwargs[:options].merge!(position.to_option)
140
+ kwargs
141
+ end
142
+ end
143
+ end
@@ -0,0 +1,70 @@
1
+ # frozen_string_literal: true
2
+
3
+ module EventStoreSubscriptions
4
+ # This class is used to persist and update commit_position and prepare_position when subscribing
5
+ # to the "$all" stream.
6
+ class SubscriptionPosition < Struct.new(:commit_position, :prepare_position)
7
+ attr_reader :update_hooks
8
+
9
+ def initialize(*)
10
+ super
11
+ @update_hooks = []
12
+ end
13
+
14
+ # Updates the position from the GRPC response.
15
+ # @param response [EventStore::Client::Streams::ReadResp] GRPC EventStore object. See its
16
+ # structure in the lib/event_store_client/adapters/grpc/generated/streams_pb.rb file in
17
+ # `event_store_client` gem.
18
+ # @return [Boolean] whether the position was updated
19
+ def update(response)
20
+ source = response.checkpoint || response.event&.event
21
+ return false unless source
22
+
23
+ # Updating position values in memory first to prevent the situation when the update hook fails,
24
+ # and the position is not up to date.
25
+ self.commit_position, self.prepare_position =
26
+ source.commit_position, source.prepare_position
27
+
28
+ update_hooks.each do |handler|
29
+ handler.call(self)
30
+ end
31
+ true
32
+ end
33
+
34
+ # Adds a handler that will be executed when the position gets updates. You may add as many
35
+ # handlers as you want.
36
+ # Example:
37
+ # ```ruby
38
+ # subscription_position.register_update_hook do |position|
39
+ # # do something with the position. E.g. persist it somewhere
40
+ # end
41
+ # subscription_position.register_update_hook do |position|
42
+ # # do something else with the position
43
+ # end
44
+ # ```
45
+ # @return [void]
46
+ def register_update_hook(&blk)
47
+ update_hooks << blk
48
+ end
49
+
50
+ # Checks if position's properties are absent
51
+ # @return [Boolean]
52
+ def empty?
53
+ commit_position.nil? || prepare_position.nil?
54
+ end
55
+
56
+ # Checks if position's properties are present
57
+ # @return [Boolean]
58
+ def present?
59
+ !empty?
60
+ end
61
+
62
+ # Constructs a hash compatible for usage with EventStoreClient::GRPC::Client#subscribe_to_stream
63
+ # method. You can pass it into :options keyword argument of that method to set the starting
64
+ # position of the stream.
65
+ # @return [Hash]
66
+ def to_option
67
+ { from_position: { commit_position: commit_position, prepare_position: prepare_position } }
68
+ end
69
+ end
70
+ end
@@ -0,0 +1,67 @@
1
+ # frozen_string_literal: true
2
+
3
+ module EventStoreSubscriptions
4
+ # This class is used to persist and update the revision when subscribing to the specific stream.
5
+ # Specific streams are streams which names differ from "$all".
6
+ class SubscriptionRevision < Struct.new(:revision)
7
+ attr_reader :update_hooks
8
+
9
+ def initialize(*)
10
+ super
11
+ @update_hooks = []
12
+ end
13
+
14
+ # Updates the revision from GRPC response.
15
+ # @param response [EventStore::Client::Streams::ReadResp] GRPC EventStore object. See its
16
+ # structure in the lib/event_store_client/adapters/grpc/generated/streams_pb.rb file in the
17
+ # `event_store_client` gem.
18
+ # @return [Boolean] whether the revision was updated
19
+ def update(response)
20
+ return false unless response.event&.event
21
+
22
+ # Updating revision value in memory first to prevent the situation when update hook fails and,
23
+ # thus keeping the revision not up to date
24
+ self.revision = response.event.event.stream_revision
25
+ update_hooks.each do |handler|
26
+ handler.call(self)
27
+ end
28
+ true
29
+ end
30
+
31
+ # Adds a handler that will be executed when the revision gets updates. You may add as many
32
+ # handlers as you want.
33
+ # Example:
34
+ # ```ruby
35
+ # subscription_revision.register_update_hook do |position|
36
+ # # do something with the position. E.g. persist it somewhere
37
+ # end
38
+ # subscription_revision.register_update_hook do |position|
39
+ # # do something else with the position
40
+ # end
41
+ # ```
42
+ # @return [void]
43
+ def register_update_hook(&blk)
44
+ update_hooks << blk
45
+ end
46
+
47
+ # Checks if revision property is absent
48
+ # @return [Boolean]
49
+ def empty?
50
+ revision.nil?
51
+ end
52
+
53
+ # Checks if revision property is set
54
+ # @return [Boolean]
55
+ def present?
56
+ !empty?
57
+ end
58
+
59
+ # Constructs a hash compatible for usage with EventStoreClient::GRPC::Client#subscribe_to_stream
60
+ # method. You can pass it into :options keyword argument of that method to set the starting
61
+ # revision of the stream.
62
+ # @return [Hash]
63
+ def to_option
64
+ { from_revision: revision }
65
+ end
66
+ end
67
+ end
@@ -0,0 +1,32 @@
1
+ # frozen_string_literal: true
2
+
3
+ module EventStoreSubscriptions
4
+ # Handles arguments that were used to create a subscription. We need to persist them for
5
+ # later adjustment and delegation.
6
+ class SubscriptionSetup < Struct.new(:args, :kwargs, :blk)
7
+ # @return [EventStoreSubscriptions::SubscriptionSetup]
8
+ def dup
9
+ self.class.new(args.dup, deep_dup(kwargs), blk)
10
+ end
11
+
12
+ private
13
+
14
+ # @param hash [Hash]
15
+ # @return [Hash]
16
+ def deep_dup(hash)
17
+ result = {}
18
+ hash.each_pair do |k, v|
19
+ result[k] =
20
+ case v
21
+ when Hash
22
+ deep_dup(v)
23
+ when Array
24
+ v.dup
25
+ else
26
+ v
27
+ end
28
+ end
29
+ result
30
+ end
31
+ end
32
+ end
@@ -0,0 +1,15 @@
1
+ # frozen_string_literal: true
2
+
3
+ module EventStoreSubscriptions
4
+ # Holds Subscription statistic
5
+ class SubscriptionStatistic
6
+ attr_accessor :last_error, :errors_count, :events_processed, :last_restart_at
7
+
8
+ def initialize
9
+ @last_error = nil
10
+ @last_restart_at = nil
11
+ @errors_count = 0
12
+ @events_processed = 0
13
+ end
14
+ end
15
+ end
@@ -0,0 +1,78 @@
1
+ # frozen_string_literal: true
2
+
3
+ module EventStoreSubscriptions
4
+ # Implements Subscription-s collection
5
+ class Subscriptions
6
+ ALL_STREAM = '$all'
7
+
8
+ attr_reader :client
9
+ attr_reader :semaphore
10
+ private :semaphore
11
+
12
+ # @param client [EventStoreClient::GRPC::Client]
13
+ def initialize(client)
14
+ @client = client
15
+ @subscriptions = []
16
+ @semaphore = Mutex.new
17
+ end
18
+
19
+ # @see EventStoreClient::GRPC::Client#subscribe_to_stream documentation for available params
20
+ # @return [EventStoreSubscriptions::Subscription]
21
+ def create(*args, **kwargs, &blk)
22
+ setup = SubscriptionSetup.new(args, kwargs, blk)
23
+ subscription = Subscription.new(
24
+ position: position_class(args[0]).new, client: client, setup: setup
25
+ )
26
+ add(subscription)
27
+ subscription
28
+ end
29
+
30
+ # Shortcut to create a Subscription to subscribe to the '$all' stream
31
+ # @see EventStoreClient::GRPC::Client#subscribe_to_all documentation for available params
32
+ # @return [EventStoreSubscriptions::Subscription]
33
+ def create_for_all(**kwargs, &blk)
34
+ create(ALL_STREAM, **kwargs, &blk)
35
+ end
36
+
37
+ # Adds Subscription to the collection
38
+ # @param subscription [EventStoreSubscriptions::Subscription]
39
+ # @return [Array<EventStoreSubscriptions::Subscription>] current subscription's collection
40
+ def add(subscription)
41
+ semaphore.synchronize { @subscriptions << subscription }
42
+ end
43
+
44
+ # Removes subscription from the collection
45
+ # @param subscription [EventStoreSubscriptions::Subscription]
46
+ # @return [EventStoreSubscriptions::Subscription, nil] returns deleted subscription or nil if it
47
+ # wasn't present in the collection
48
+ def remove(subscription)
49
+ semaphore.synchronize { @subscriptions.delete(subscription) }
50
+ end
51
+
52
+ # Starts listening to all subscriptions in the collection
53
+ # @return [Array<EventStoreSubscriptions::Subscription>]
54
+ def listen_all
55
+ semaphore.synchronize { @subscriptions.each(&:listen) }
56
+ end
57
+
58
+ # Stops listening to all subscriptions in the collection
59
+ # @return [Array<EventStoreSubscriptions::Subscription>]
60
+ def stop_all
61
+ semaphore.synchronize { @subscriptions.each(&:stop_listening) }
62
+ end
63
+
64
+ # @return [Array<EventStoreSubscriptions::Subscription>]
65
+ def subscriptions
66
+ # Duping original collection to prevent potential mutable operations over it from user's side
67
+ semaphore.synchronize { @subscriptions.dup }
68
+ end
69
+
70
+ private
71
+
72
+ # @param stream_name [String]
73
+ # @return [Class<EventStoreSubscriptions::SubscriptionPosition>, Class<EventStoreSubscriptions::SubscriptionRevision>]
74
+ def position_class(stream_name)
75
+ stream_name == ALL_STREAM ? SubscriptionPosition : SubscriptionRevision
76
+ end
77
+ end
78
+ end
@@ -0,0 +1,5 @@
1
+ # frozen_string_literal: true
2
+
3
+ module EventStoreSubscriptions
4
+ VERSION = "1.0.0"
5
+ end
@@ -0,0 +1,15 @@
1
+ # frozen_string_literal: true
2
+
3
+ module EventStoreSubscriptions
4
+ module WaitForFinish
5
+ # Waits until state switches from :running to any other state.
6
+ # @return [void]
7
+ def wait_for_finish
8
+ loop do
9
+ break if state.stopped? || state.dead?
10
+
11
+ sleep 0.1
12
+ end
13
+ end
14
+ end
15
+ end
@@ -0,0 +1,110 @@
1
+ # frozen_string_literal: true
2
+
3
+ module EventStoreSubscriptions
4
+ # Watches over the given subscriptions collection and restarts dead subscriptions. It is useful
5
+ # in cases when your subscription's handler raises error. Its usage is optional.
6
+ class WatchDog
7
+ include WaitForFinish
8
+
9
+ CHECK_INTERVAL = 5 # seconds. How often to scan subscriptions
10
+
11
+ class << self
12
+ # @param collection [EventStoreSubscriptions::Subscriptions]
13
+ # @param restart_terminator [Proc, nil]
14
+ # @return [EventStoreSubscriptions::WatchDog]
15
+ def watch(collection, restart_terminator: nil)
16
+ new(collection).watch
17
+ end
18
+ end
19
+
20
+ attr_accessor :runner
21
+ attr_reader :collection, :state, :restart_terminator
22
+ private :runner, :runner=, :restart_terminator
23
+
24
+ # @param collection [EventStoreSubscriptions::Subscriptions]
25
+ # @param restart_terminator [Proc, nil] define a terminator that would halt Subscription restart
26
+ # process if the result of it execution is truthy. Subscription instance will be passed as a
27
+ # first argument into it, and, based on it, you should decide whether to process the restart
28
+ # or not.
29
+ def initialize(collection, restart_terminator: nil)
30
+ @collection = collection
31
+ @state = ObjectState.new
32
+ @runner = nil
33
+ @restart_terminator = restart_terminator
34
+ end
35
+
36
+ # Start watching over the given Subscriptions collection
37
+ # @return [EventStoreSubscriptions::WatchDog] returns self
38
+ def watch
39
+ self.runner ||=
40
+ begin
41
+ state.running!
42
+ Thread.new do
43
+ loop do
44
+ sleep CHECK_INTERVAL
45
+ break unless state.running?
46
+
47
+ collection.subscriptions.each do |sub|
48
+ break unless state.running?
49
+
50
+ restart_subscription(sub) if sub.state.dead?
51
+ end
52
+ end
53
+ rescue StandardError => e
54
+ state.dead!
55
+ raise
56
+ end
57
+ end
58
+ self
59
+ end
60
+
61
+ # Stop watching over the given subscriptions collection. This command is async - the result is
62
+ # not immediate. Use the #wait_for_finish method in order to wait until the runner has fully stopped .
63
+ # Example:
64
+ # ```ruby
65
+ # watch_dog.unwatch.wait_for_finish
66
+ # ```
67
+ # @return [EventStoreSubscriptions::WatchDog] returns self
68
+ def unwatch
69
+ return self unless runner&.alive?
70
+
71
+ state.halting!
72
+ Thread.new do
73
+ loop do
74
+ # If runner sleeps between runs we can safely shut it down. Even if the edge case happens,
75
+ # when a runner's status changes between its check and `runner.exit`, it is still ok, it
76
+ # would be shut down anyway because of the guard condition `break unless state.running?`
77
+ runner.exit if runner&.status == 'sleep'
78
+ unless runner&.alive?
79
+ state.stopped!
80
+ self.runner = nil
81
+ break
82
+ end
83
+ sleep 0.1
84
+ end
85
+ end
86
+ self
87
+ end
88
+
89
+ private
90
+
91
+ # @param failed_sub [EventStoreSubscriptions::Subscription]
92
+ # @return [EventStoreSubscriptions::Subscription] newly created Subscription
93
+ def restart_subscription(failed_sub)
94
+ return if restart_terminator&.call(failed_sub)
95
+ # Check if no one else did this job
96
+ return unless collection.remove(failed_sub)
97
+
98
+ new_sub = Subscription.new(
99
+ position: failed_sub.position,
100
+ client: failed_sub.client,
101
+ setup: failed_sub.setup,
102
+ statistic: failed_sub.statistic
103
+ )
104
+ new_sub.statistic.last_restart_at = Time.now.utc
105
+ collection.add(new_sub)
106
+ failed_sub.delete
107
+ new_sub.listen
108
+ end
109
+ end
110
+ end
@@ -0,0 +1,17 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'event_store_client'
4
+ require_relative 'event_store_subscriptions/version'
5
+ require_relative 'event_store_subscriptions/error'
6
+ require_relative 'event_store_subscriptions/wait_for_finish'
7
+ require_relative 'event_store_subscriptions/object_state'
8
+ require_relative 'event_store_subscriptions/subscription_statistic'
9
+ require_relative 'event_store_subscriptions/subscription'
10
+ require_relative 'event_store_subscriptions/subscription_position'
11
+ require_relative 'event_store_subscriptions/subscription_revision'
12
+ require_relative 'event_store_subscriptions/subscription_setup'
13
+ require_relative 'event_store_subscriptions/subscriptions'
14
+ require_relative 'event_store_subscriptions/watch_dog'
15
+
16
+ module EventStoreSubscriptions
17
+ end
metadata ADDED
@@ -0,0 +1,116 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: event_store_subscriptions
3
+ version: !ruby/object:Gem::Version
4
+ version: 1.0.0
5
+ platform: ruby
6
+ authors:
7
+ - Ivan Dzyzenko
8
+ autorequire:
9
+ bindir: exe
10
+ cert_chain: []
11
+ date: 2022-10-03 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: event_store_client
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - "~>"
18
+ - !ruby/object:Gem::Version
19
+ version: '2.0'
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - "~>"
25
+ - !ruby/object:Gem::Version
26
+ version: '2.0'
27
+ - !ruby/object:Gem::Dependency
28
+ name: pry
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - "~>"
32
+ - !ruby/object:Gem::Version
33
+ version: '0.14'
34
+ type: :development
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - "~>"
39
+ - !ruby/object:Gem::Version
40
+ version: '0.14'
41
+ - !ruby/object:Gem::Dependency
42
+ name: rspec
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - "~>"
46
+ - !ruby/object:Gem::Version
47
+ version: '3.11'
48
+ type: :development
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - "~>"
53
+ - !ruby/object:Gem::Version
54
+ version: '3.11'
55
+ - !ruby/object:Gem::Dependency
56
+ name: rake
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - "~>"
60
+ - !ruby/object:Gem::Version
61
+ version: '13.0'
62
+ type: :development
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - "~>"
67
+ - !ruby/object:Gem::Version
68
+ version: '13.0'
69
+ description: Implementation of subscription manager for `event_store_client` gem.
70
+ email:
71
+ - ivan.dzyzenko@gmail.com
72
+ executables: []
73
+ extensions: []
74
+ extra_rdoc_files: []
75
+ files:
76
+ - LICENSE.txt
77
+ - README.md
78
+ - lib/event_store_subscriptions.rb
79
+ - lib/event_store_subscriptions/error.rb
80
+ - lib/event_store_subscriptions/object_state.rb
81
+ - lib/event_store_subscriptions/subscription.rb
82
+ - lib/event_store_subscriptions/subscription_position.rb
83
+ - lib/event_store_subscriptions/subscription_revision.rb
84
+ - lib/event_store_subscriptions/subscription_setup.rb
85
+ - lib/event_store_subscriptions/subscription_statistic.rb
86
+ - lib/event_store_subscriptions/subscriptions.rb
87
+ - lib/event_store_subscriptions/version.rb
88
+ - lib/event_store_subscriptions/wait_for_finish.rb
89
+ - lib/event_store_subscriptions/watch_dog.rb
90
+ homepage: https://github.com/yousty/event_store_subscriptions
91
+ licenses:
92
+ - MIT
93
+ metadata:
94
+ allowed_push_host: https://rubygems.org
95
+ homepage_uri: https://github.com/yousty/event_store_subscriptions
96
+ source_code_uri: https://github.com/yousty/event_store_subscriptions
97
+ post_install_message:
98
+ rdoc_options: []
99
+ require_paths:
100
+ - lib
101
+ required_ruby_version: !ruby/object:Gem::Requirement
102
+ requirements:
103
+ - - ">="
104
+ - !ruby/object:Gem::Version
105
+ version: 2.7.0
106
+ required_rubygems_version: !ruby/object:Gem::Requirement
107
+ requirements:
108
+ - - ">="
109
+ - !ruby/object:Gem::Version
110
+ version: '0'
111
+ requirements: []
112
+ rubygems_version: 3.3.7
113
+ signing_key:
114
+ specification_version: 4
115
+ summary: Implementation of subscription manager for `event_store_client` gem.
116
+ test_files: []