event_store_subscriptions 1.0.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: f261c1fb47c48ba5354b9d02c1b1a066e64813e397933657bbb75c396ec06f85
4
+ data.tar.gz: ce40a5265ba3d2bc34f4392cae838df08a471c13c85bbecc292dd53628cf464f
5
+ SHA512:
6
+ metadata.gz: 97ce25b1bbcd50a345b2be4ed7f1b98048f540732a91eafff55d36c31d6d513ad759614938dba6a07139a448d0e2c3b35bdfc1bb1e96b77ee2de51643b07b84d
7
+ data.tar.gz: 995f4264d8a301b8dae0156ab0d08fbe3ca41744194b4357e6fddba9d29b73deec207d98d6c9859fad8f5cbc41e3204580c163d76b61e8f8ea4e92c814ab8ded
data/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2022 Yousty AG
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,284 @@
1
+ ![Run tests](https://github.com/yousty/event_store_subscriptions/workflows/Run%20tests/badge.svg?branch=main&event=push)
2
+ [![Gem Version](https://badge.fury.io/rb/event_store_subscriptions.svg)](https://badge.fury.io/rb/event_store_subscriptions)
3
+
4
+ # EventStoreSubscriptions
5
+
6
+ Extends the functionality of the [EventStoreDB ruby client](https://github.com/yousty/event_store_client) with a catch-up subscriptions manager.
7
+
8
+ By default `event_store_client` implements thread-blocking methods to subscribe to a stream. Those are `#subscribe_to_stream` and `#subscribe_to_all`. In order to subscribe to many streams/events, you need to implement asynchronous subscriptions on your own. This gem solves this task by putting each subscription into its own thread.
9
+
10
+ The thread-based implementation has a downside: any IO operation in your subscription's handlers will block all other threads. So it is important to consider how many subscriptions you put into a single process. There is a plan to integrate Ractors instead/alongside threads to provide the option to eliminate the IO-blocking issue.
11
+
12
+ ## Installation
13
+
14
+ Add this line to your application's Gemfile:
15
+
16
+ ```ruby
17
+ gem 'event_store_subscriptions'
18
+ ```
19
+
20
+ And then execute:
21
+
22
+ $ bundle install
23
+
24
+ Or install it yourself as:
25
+
26
+ $ gem install event_store_subscriptions
27
+
28
+ ## Usage
29
+
30
+ Use the `#create` and `#create_for_all` methods to subscribe to a stream. For the full list of available arguments see the documentation of the `EventStoreClient::GRPC::Client#subscribe_to_stream` method in the [event_store_client gem docs](https://rubydoc.info/gems/event_store_client). You may also want to check the [Catch-up subscriptions](https://github.com/yousty/event_store_client/blob/master/docs/catch_up_subscriptions.md) section as well.
31
+
32
+ ### Subscribing to a specific stream
33
+
34
+ Use the `#create` method in order to subscribe to specific stream:
35
+
36
+ ```ruby
37
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
38
+ handler = proc do |resp|
39
+ if resp.success?
40
+ do_something_with_resp(resp.success) # retrieve an event
41
+ else # resp.failure? => true
42
+ handle_failure(resp.failure)
43
+ end
44
+ end
45
+ subscriptions.create('some-stream', handler: handler)
46
+ subscriptions.listen_all
47
+ ```
48
+
49
+ You may provide any object which responds to `#call` as a handler:
50
+
51
+ ```ruby
52
+ class SomeStreamHandler
53
+ def call(resp)
54
+ if resp.success?
55
+ do_something_with_resp(resp.success) # retrieve an event
56
+ else # resp.failure? => true
57
+ handle_failure(resp.failure)
58
+ end
59
+ end
60
+ end
61
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
62
+ subscriptions.create('some-stream', handler: SomeStreamHandler.new)
63
+ subscriptions.listen_all
64
+ ```
65
+
66
+ ### Subscribing to the $all stream
67
+
68
+ Use the `#create_for_all` method to subscribe to the all stream:
69
+
70
+ ```ruby
71
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
72
+ handler = proc do |resp|
73
+ if resp.success?
74
+ do_something_with_resp(resp.success) # retrieve an event
75
+ else # resp.failure? => true
76
+ handle_failure(resp.failure)
77
+ end
78
+ end
79
+ subscriptions.create_for_all(handler: handler)
80
+ subscriptions.listen_all
81
+ ```
82
+
83
+ You may also explicitly pass `"$all"` stream name to the `#create` method:
84
+
85
+ ```ruby
86
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
87
+ handler = proc do |resp|
88
+ if resp.success?
89
+ do_something_with_resp(resp.success) # retrieve an event
90
+ else # resp.failure? => true
91
+ handle_failure(resp.failure)
92
+ end
93
+ end
94
+ subscriptions.create('$all', handler: handler)
95
+ subscriptions.listen_all
96
+ ```
97
+
98
+ ### Handling Subscription position updates
99
+
100
+ You may want to add a handler that will be executed each time a subscription gets position updates. Such updates happen when new events are added to the stream or when EventStore DB produces a checkpoint response.
101
+
102
+ #### Listening for position updates of a specific stream
103
+
104
+ A handler registered to receive position updates of a specific stream is called with the `EventStoreSubscriptions::SubscriptionRevision` class instance. It holds the current revision of the stream.
105
+
106
+ ```ruby
107
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
108
+ subscription = subscriptions.create('some-stream', handler: proc { |r| p r })
109
+ subscription.position.register_update_hook do |position|
110
+ puts "Current revision is #{position.revision}"
111
+ end
112
+ subscription.listen
113
+ ```
114
+
115
+ #### Listening for position updates of the $all stream
116
+
117
+ A handler registered to receive position updates of the `$all` stream is called with the `EventStoreSubscriptions::SubscriptionPosition` class instance. It holds the current `commit_position` and `prepare_position` of the `$all` stream.
118
+
119
+ ```ruby
120
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
121
+ subscription = subscriptions.create_for_all(handler: proc { |r| p r })
122
+ subscription.position.register_update_hook do |position|
123
+ puts "Current commit/prepare positions are #{position.commit_position}/#{position.prepare_position}"
124
+ end
125
+ subscription.listen
126
+ ```
127
+
128
+ ### Automatic restart of failed Subscriptions
129
+
130
+ This gem provides a possibility to watch over your subscription collections and restart a subscription in case it failed. Subscriptions may fail because an exception was raised in the handler or in the position update hook. A new subscription will be started, listening from the position the failed subscription has stopped.
131
+
132
+ Start watching over your subscriptions' collection:
133
+
134
+ ```ruby
135
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
136
+ subscriptions.create_for_all(handler: proc { |r| p r })
137
+ EventStoreSubscriptions::WatchDog.watch(subscriptions)
138
+ subscriptions.listen_all
139
+ ```
140
+
141
+ ### Async nature of this gem
142
+
143
+ `EventStoreSubscriptions::Subscriptions#listen_all`, `EventStoreSubscriptions::Subscriptions#stop_all`, `EventStoreSubscriptions::Subscription#listen`, `EventStoreSubscriptions::Subscription#stop_listening`, `EventStoreSubscriptions::WatchDog#watch`, `EventStoreSubscriptions::WatchDog#unwatch` methods are asynchronous. This means that they spawn thread that performs proper task in the background.
144
+
145
+ `EventStoreSubscriptions::Subscriptions#stop_all`, `EventStoreSubscriptions::Subscription#stop_listening` and `EventStoreSubscriptions::WatchDog#unwatch` methods has ending run time, meaning that they runners won't run forever.
146
+
147
+ `EventStoreSubscriptions::Subscriptions#listen_all`, `EventStoreSubscriptions::Subscription#listen` and `EventStoreSubscriptions::WatchDog#watch` methods will run forever.
148
+
149
+ In order to stop running `Subscription` or `WatchDog` you should initiate stop process and wait for finish.
150
+
151
+ #### Stopping Subscription
152
+
153
+ For single subscription:
154
+
155
+ ```ruby
156
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
157
+ subscription = subscriptions.create_for_all(handler: proc { |r| p r })
158
+ subscription.listen
159
+
160
+ # Initiate Subscription shutdown
161
+ subscription.stop_listening
162
+ # Wait for Subscription to finish. This will block current Thread.
163
+ subscription.wait_for_finish
164
+ ```
165
+
166
+ For the entire collection:
167
+
168
+ ```ruby
169
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
170
+ subscriptions.create_for_all(handler: proc { |r| p r })
171
+ subscriptions.listen_all
172
+
173
+ # Initiate shutdown for each Subscription in the collection
174
+ subscriptions.stop_all
175
+ # Wait for all Subscriptions to finish. This will block current Thread.
176
+ subscriptions.subscriptions.each(&:wait_for_finish)
177
+ ```
178
+
179
+ #### Stopping WatchDog
180
+
181
+ ```ruby
182
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
183
+ watcher = EventStoreSubscriptions::WatchDog.watch(subscriptions)
184
+
185
+ # Initiate WatchDog shutdown
186
+ watcher.unwatch
187
+ # Wait for WatchDog to finish. This will block current Thread.
188
+ watcher.wait_for_finish
189
+ ```
190
+
191
+ ### Graceful shutdown
192
+
193
+ You may want to gracefully shut down the process that handles the subscriptions. In order to do so, you should define a `Kernel.trap` handler to handle your kill signal:
194
+
195
+ ```ruby
196
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
197
+ subscriptions.create_for_all(handler: proc { |r| p r })
198
+ watcher = EventStoreSubscriptions::WatchDog.watch(subscriptions)
199
+ subscriptions.listen_all
200
+
201
+ Kernel.trap('TERM') do
202
+ # Because the implementation uses Mutex - wrap it into Thread to bypass the limitations of
203
+ # Kernel#trap
204
+ Thread.new do
205
+ # Initiate graceful shutdown. Need to shutdown watcher first, and then - subscriptions
206
+ watcher.unwatch.wait_for_finish
207
+ subscriptions.stop_all.each(&:wait_for_finish)
208
+ end.join
209
+ exit
210
+ end
211
+
212
+ # Wait while Subscriptions are working
213
+ subscriptions.each(&:wait_for_finish)
214
+ ```
215
+
216
+ Now just send the `TERM` signal if you want to gracefully shut down your process:
217
+
218
+ ```bash
219
+ kill -TERM <pid of your process>
220
+ ```
221
+
222
+ ### Monitoring Subscriptions
223
+
224
+ After you started listening your Subscriptions, you may want to monitor status of them. There is various built-in statistics which you can get.
225
+
226
+ ```ruby
227
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
228
+ subscriptions.create_for_all(handler: proc { |r| p r })
229
+ watcher = EventStoreSubscriptions::WatchDog.watch(subscriptions)
230
+ subscriptions.listen_all
231
+
232
+ loop do
233
+ sleep 1
234
+ subscriptions.subscriptions.each do |subscription|
235
+ puts "Current state is: #{subscription.state}"
236
+ puts "Current position: #{subscription.position.to_h}"
237
+ puts "Last error: #{subscription.statistic.last_error.inspect}"
238
+ puts "Last restart was at: #{subscription.statistic.last_restart_at || 'Never'}"
239
+ puts "Total errors/restarts: #{subscription.statistic.errors_count}"
240
+ puts "Events processed: #{subscription.statistic.events_processed}"
241
+ puts "Current watcher state is: #{watcher.state}"
242
+ end
243
+ end
244
+ ```
245
+
246
+ ### WatchDog and control of restart condition of Subscriptions
247
+
248
+ You may want to decide yourself whether `WhatchDog` should restart a `Subscription`. You can do so by providing a proc which, if thruthy result is returned, skips the restart of `Subscription`.
249
+
250
+ ```ruby
251
+ subscriptions = EventStoreSubscriptions::Subscriptions.new(EventStoreClient.client)
252
+ subscriptions.create_for_all(handler: proc { |r| p r })
253
+ # Do not restart Subscription if its id is even
254
+ restart_terminator = proc { |sub| sub.__id__ % 2 == 0 }
255
+ EventStoreSubscriptions::WatchDog.watch(subscriptions, restart_terminator: restart_terminator)
256
+ subscriptions.listen_all
257
+ ```
258
+
259
+ ## Development
260
+
261
+ You will have to install Docker first. It is needed to run EventStore DB. You can run EventStore DB with this command:
262
+
263
+ ```shell
264
+ docker-compose -f docker-compose-cluster.yml up
265
+ ```
266
+
267
+ Now you can enter a dev console by running `bin/console` or run tests by running the `rspec` command.
268
+
269
+ ## Contributing
270
+
271
+ Bug reports and pull requests are welcome on GitHub at https://github.com/yousty/event_store_subscriptions. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/yousty/event_store_subscriptions/blob/master/CODE_OF_CONDUCT.md).
272
+
273
+ ### Publishing new version
274
+
275
+ 1. Push commit with updated `version.rb` file to the `release` branch. The new version will be automatically pushed to [rubygems](https://rubygems.org).
276
+ 2. Create release on GitHub including change log.
277
+
278
+ ## License
279
+
280
+ The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
281
+
282
+ ## Code of Conduct
283
+
284
+ Everyone interacting in the EventStoreSubscriptions project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/yousty/event_store_subscriptions/blob/master/CODE_OF_CONDUCT.md).
@@ -0,0 +1,9 @@
1
+ # frozen_string_literal: true
2
+
3
+ module EventStoreSubscriptions
4
+ class Error < StandardError
5
+ end
6
+
7
+ class ThreadNotDeadError < Error
8
+ end
9
+ end
@@ -0,0 +1,36 @@
1
+ # frozen_string_literal: true
2
+
3
+ module EventStoreSubscriptions
4
+ # Defines various states. It is used to set and get current object's state.
5
+ class ObjectState
6
+ attr_accessor :state
7
+ attr_reader :semaphore
8
+ private :state, :state=, :semaphore
9
+
10
+ STATES = %i(initial running halting stopped dead).freeze
11
+
12
+ def initialize
13
+ @semaphore = Thread::Mutex.new
14
+ initial!
15
+ end
16
+
17
+ STATES.each do |state|
18
+ # Checks whether the object is in appropriate state
19
+ # @return [Boolean]
20
+ define_method "#{state}?" do
21
+ semaphore.synchronize { self.state == state }
22
+ end
23
+
24
+ # Sets the state.
25
+ # @return [Symbol]
26
+ define_method "#{state}!" do
27
+ semaphore.synchronize { self.state = state }
28
+ end
29
+ end
30
+
31
+ # @return [String] string representation of the #state
32
+ def to_s
33
+ state.to_s
34
+ end
35
+ end
36
+ end
@@ -0,0 +1,143 @@
1
+ # frozen_string_literal: true
2
+
3
+ module EventStoreSubscriptions
4
+ class Subscription
5
+ include WaitForFinish
6
+
7
+ FORCED_SHUTDOWN_DELAY = 60 # seconds
8
+
9
+ attr_accessor :runner
10
+ attr_reader :client, :setup, :state, :position, :statistic
11
+ private :runner, :runner=
12
+
13
+ # @param position [EventStoreSubscriptions::SubscriptionPosition, EventStoreSubscriptions::SubscriptionRevision]
14
+ # @param client [EventStoreClient::GRPC::Client]
15
+ # @param setup [EventStoreSubscriptions::SubscriptionSetup]
16
+ # @param statistic [EventStoreSubscriptions::SubscriptionStatistic]
17
+ def initialize(position:, client:, setup:, statistic: SubscriptionStatistic.new)
18
+ @position = position
19
+ @client = client
20
+ @setup = setup
21
+ @state = ObjectState.new
22
+ @statistic = statistic
23
+ @runner = nil
24
+ end
25
+
26
+ # Start listening for the events
27
+ # @return [EventStoreSubscriptions::Subscription] returns self
28
+ def listen
29
+ self.runner ||=
30
+ begin
31
+ state.running!
32
+ Thread.new do
33
+ Thread.current.abort_on_exception = false
34
+ Thread.current.report_on_exception = false
35
+ client.subscribe_to_stream(
36
+ *setup.args,
37
+ **adjusted_kwargs,
38
+ &setup.blk
39
+ )
40
+ rescue StandardError => e
41
+ statistic.last_error = e
42
+ statistic.errors_count += 1
43
+ state.dead!
44
+ raise
45
+ end
46
+ end
47
+ self
48
+ end
49
+
50
+ # Stops listening for events. This command is async - the result is not immediate. Use the #wait_for_finish
51
+ # method to wait until the runner has fully stopped.
52
+ # @return [EventStoreSubscriptions::Subscription] returns self
53
+ def stop_listening
54
+ return self unless runner&.alive?
55
+
56
+ state.halting!
57
+ Thread.new do
58
+ stopping_at = Time.now.utc
59
+ loop do
60
+ # Give Subscription up to FORCED_SHUTDOWN_DELAY seconds for graceful shutdown
61
+ runner&.exit if Time.now.utc - stopping_at > FORCED_SHUTDOWN_DELAY
62
+
63
+ unless runner&.alive?
64
+ state.stopped!
65
+ self.runner = nil
66
+ break
67
+ end
68
+ sleep 0.1
69
+ end
70
+ end
71
+ self
72
+ end
73
+
74
+ # Removes all properties of object and freezes it. You can't delete currently running
75
+ # Subscription though. You must stop it first.
76
+ # @return [EventStoreSubscriptions::Subscription] frozen object
77
+ # @raise [EventStoreSubscriptions::ThreadNotDeadError] raises this error in case runner Thread
78
+ # is still alive for some reason. Normally this should never happen.
79
+ def delete
80
+ if runner&.alive?
81
+ raise ThreadNotDeadError, "Can not delete alive Subscription #{self.inspect}"
82
+ end
83
+
84
+ instance_variables.each { |var| instance_variable_set(var, nil) }
85
+ freeze
86
+ end
87
+
88
+ private
89
+
90
+ # Wraps original handler into our own handler to provide extended functionality.
91
+ # @param original_handler [#call]
92
+ # @return [Proc]
93
+ def handler(original_handler)
94
+ proc do |result|
95
+ Thread.current.exit unless state.running?
96
+ original_result = result.success
97
+ result = EventStoreClient::GRPC::Shared::Streams::ProcessResponse.new.call(
98
+ original_result,
99
+ *process_response_args
100
+ )
101
+ original_handler.call(result) if result
102
+ statistic.events_processed += 1
103
+ position.update(original_result)
104
+ end
105
+ end
106
+
107
+ # Calculates "skip_deserialization" and "skip_decryption" arguments for the ProcessResponse
108
+ # class. Since we have overridden the original handler, we need to calculate the correct argument values
109
+ # to process the response on our own. This method implements the same behavior as
110
+ # the event_store_client gem implements (EventStoreClient::GRPC::Client#subscribe_to_stream
111
+ # method).
112
+ # @return [Array<Boolean>]
113
+ def process_response_args
114
+ skip_deserialization =
115
+ if setup.kwargs.key?(:skip_deserialization)
116
+ setup.kwargs[:skip_deserialization]
117
+ else
118
+ client.config.skip_deserialization
119
+ end
120
+ skip_decryption =
121
+ if setup.kwargs.key?(:skip_decryption)
122
+ setup.kwargs[:skip_decryption]
123
+ else
124
+ client.config.skip_decryption
125
+ end
126
+ [skip_deserialization, skip_decryption]
127
+ end
128
+
129
+ # Override keyword arguments, provided by dev in EventStoreSubscriptions::Subscriptions#create
130
+ # or EventStoreSubscriptions::Subscriptions#create_for_all methods. This is needed to provide
131
+ # our own handler and to override the starting position of the given stream.
132
+ # @return [Hash]
133
+ def adjusted_kwargs
134
+ kwargs = setup.dup.kwargs
135
+ kwargs.merge!(handler: handler(kwargs[:handler]), skip_deserialization: true)
136
+ return kwargs unless position.present?
137
+
138
+ kwargs[:options] ||= {}
139
+ kwargs[:options].merge!(position.to_option)
140
+ kwargs
141
+ end
142
+ end
143
+ end
@@ -0,0 +1,70 @@
1
+ # frozen_string_literal: true
2
+
3
+ module EventStoreSubscriptions
4
+ # This class is used to persist and update commit_position and prepare_position when subscribing
5
+ # to the "$all" stream.
6
+ class SubscriptionPosition < Struct.new(:commit_position, :prepare_position)
7
+ attr_reader :update_hooks
8
+
9
+ def initialize(*)
10
+ super
11
+ @update_hooks = []
12
+ end
13
+
14
+ # Updates the position from the GRPC response.
15
+ # @param response [EventStore::Client::Streams::ReadResp] GRPC EventStore object. See its
16
+ # structure in the lib/event_store_client/adapters/grpc/generated/streams_pb.rb file in
17
+ # `event_store_client` gem.
18
+ # @return [Boolean] whether the position was updated
19
+ def update(response)
20
+ source = response.checkpoint || response.event&.event
21
+ return false unless source
22
+
23
+ # Updating position values in memory first to prevent the situation when the update hook fails,
24
+ # and the position is not up to date.
25
+ self.commit_position, self.prepare_position =
26
+ source.commit_position, source.prepare_position
27
+
28
+ update_hooks.each do |handler|
29
+ handler.call(self)
30
+ end
31
+ true
32
+ end
33
+
34
+ # Adds a handler that will be executed when the position gets updates. You may add as many
35
+ # handlers as you want.
36
+ # Example:
37
+ # ```ruby
38
+ # subscription_position.register_update_hook do |position|
39
+ # # do something with the position. E.g. persist it somewhere
40
+ # end
41
+ # subscription_position.register_update_hook do |position|
42
+ # # do something else with the position
43
+ # end
44
+ # ```
45
+ # @return [void]
46
+ def register_update_hook(&blk)
47
+ update_hooks << blk
48
+ end
49
+
50
+ # Checks if position's properties are absent
51
+ # @return [Boolean]
52
+ def empty?
53
+ commit_position.nil? || prepare_position.nil?
54
+ end
55
+
56
+ # Checks if position's properties are present
57
+ # @return [Boolean]
58
+ def present?
59
+ !empty?
60
+ end
61
+
62
+ # Constructs a hash compatible for usage with EventStoreClient::GRPC::Client#subscribe_to_stream
63
+ # method. You can pass it into :options keyword argument of that method to set the starting
64
+ # position of the stream.
65
+ # @return [Hash]
66
+ def to_option
67
+ { from_position: { commit_position: commit_position, prepare_position: prepare_position } }
68
+ end
69
+ end
70
+ end
@@ -0,0 +1,67 @@
1
+ # frozen_string_literal: true
2
+
3
+ module EventStoreSubscriptions
4
+ # This class is used to persist and update the revision when subscribing to the specific stream.
5
+ # Specific streams are streams which names differ from "$all".
6
+ class SubscriptionRevision < Struct.new(:revision)
7
+ attr_reader :update_hooks
8
+
9
+ def initialize(*)
10
+ super
11
+ @update_hooks = []
12
+ end
13
+
14
+ # Updates the revision from GRPC response.
15
+ # @param response [EventStore::Client::Streams::ReadResp] GRPC EventStore object. See its
16
+ # structure in the lib/event_store_client/adapters/grpc/generated/streams_pb.rb file in the
17
+ # `event_store_client` gem.
18
+ # @return [Boolean] whether the revision was updated
19
+ def update(response)
20
+ return false unless response.event&.event
21
+
22
+ # Updating revision value in memory first to prevent the situation when update hook fails and,
23
+ # thus keeping the revision not up to date
24
+ self.revision = response.event.event.stream_revision
25
+ update_hooks.each do |handler|
26
+ handler.call(self)
27
+ end
28
+ true
29
+ end
30
+
31
+ # Adds a handler that will be executed when the revision gets updates. You may add as many
32
+ # handlers as you want.
33
+ # Example:
34
+ # ```ruby
35
+ # subscription_revision.register_update_hook do |position|
36
+ # # do something with the position. E.g. persist it somewhere
37
+ # end
38
+ # subscription_revision.register_update_hook do |position|
39
+ # # do something else with the position
40
+ # end
41
+ # ```
42
+ # @return [void]
43
+ def register_update_hook(&blk)
44
+ update_hooks << blk
45
+ end
46
+
47
+ # Checks if revision property is absent
48
+ # @return [Boolean]
49
+ def empty?
50
+ revision.nil?
51
+ end
52
+
53
+ # Checks if revision property is set
54
+ # @return [Boolean]
55
+ def present?
56
+ !empty?
57
+ end
58
+
59
+ # Constructs a hash compatible for usage with EventStoreClient::GRPC::Client#subscribe_to_stream
60
+ # method. You can pass it into :options keyword argument of that method to set the starting
61
+ # revision of the stream.
62
+ # @return [Hash]
63
+ def to_option
64
+ { from_revision: revision }
65
+ end
66
+ end
67
+ end
@@ -0,0 +1,32 @@
1
+ # frozen_string_literal: true
2
+
3
+ module EventStoreSubscriptions
4
+ # Handles arguments that were used to create a subscription. We need to persist them for
5
+ # later adjustment and delegation.
6
+ class SubscriptionSetup < Struct.new(:args, :kwargs, :blk)
7
+ # @return [EventStoreSubscriptions::SubscriptionSetup]
8
+ def dup
9
+ self.class.new(args.dup, deep_dup(kwargs), blk)
10
+ end
11
+
12
+ private
13
+
14
+ # @param hash [Hash]
15
+ # @return [Hash]
16
+ def deep_dup(hash)
17
+ result = {}
18
+ hash.each_pair do |k, v|
19
+ result[k] =
20
+ case v
21
+ when Hash
22
+ deep_dup(v)
23
+ when Array
24
+ v.dup
25
+ else
26
+ v
27
+ end
28
+ end
29
+ result
30
+ end
31
+ end
32
+ end
@@ -0,0 +1,15 @@
1
+ # frozen_string_literal: true
2
+
3
+ module EventStoreSubscriptions
4
+ # Holds Subscription statistic
5
+ class SubscriptionStatistic
6
+ attr_accessor :last_error, :errors_count, :events_processed, :last_restart_at
7
+
8
+ def initialize
9
+ @last_error = nil
10
+ @last_restart_at = nil
11
+ @errors_count = 0
12
+ @events_processed = 0
13
+ end
14
+ end
15
+ end
@@ -0,0 +1,78 @@
1
+ # frozen_string_literal: true
2
+
3
+ module EventStoreSubscriptions
4
+ # Implements Subscription-s collection
5
+ class Subscriptions
6
+ ALL_STREAM = '$all'
7
+
8
+ attr_reader :client
9
+ attr_reader :semaphore
10
+ private :semaphore
11
+
12
+ # @param client [EventStoreClient::GRPC::Client]
13
+ def initialize(client)
14
+ @client = client
15
+ @subscriptions = []
16
+ @semaphore = Mutex.new
17
+ end
18
+
19
+ # @see EventStoreClient::GRPC::Client#subscribe_to_stream documentation for available params
20
+ # @return [EventStoreSubscriptions::Subscription]
21
+ def create(*args, **kwargs, &blk)
22
+ setup = SubscriptionSetup.new(args, kwargs, blk)
23
+ subscription = Subscription.new(
24
+ position: position_class(args[0]).new, client: client, setup: setup
25
+ )
26
+ add(subscription)
27
+ subscription
28
+ end
29
+
30
+ # Shortcut to create a Subscription to subscribe to the '$all' stream
31
+ # @see EventStoreClient::GRPC::Client#subscribe_to_all documentation for available params
32
+ # @return [EventStoreSubscriptions::Subscription]
33
+ def create_for_all(**kwargs, &blk)
34
+ create(ALL_STREAM, **kwargs, &blk)
35
+ end
36
+
37
+ # Adds Subscription to the collection
38
+ # @param subscription [EventStoreSubscriptions::Subscription]
39
+ # @return [Array<EventStoreSubscriptions::Subscription>] current subscription's collection
40
+ def add(subscription)
41
+ semaphore.synchronize { @subscriptions << subscription }
42
+ end
43
+
44
+ # Removes subscription from the collection
45
+ # @param subscription [EventStoreSubscriptions::Subscription]
46
+ # @return [EventStoreSubscriptions::Subscription, nil] returns deleted subscription or nil if it
47
+ # wasn't present in the collection
48
+ def remove(subscription)
49
+ semaphore.synchronize { @subscriptions.delete(subscription) }
50
+ end
51
+
52
+ # Starts listening to all subscriptions in the collection
53
+ # @return [Array<EventStoreSubscriptions::Subscription>]
54
+ def listen_all
55
+ semaphore.synchronize { @subscriptions.each(&:listen) }
56
+ end
57
+
58
+ # Stops listening to all subscriptions in the collection
59
+ # @return [Array<EventStoreSubscriptions::Subscription>]
60
+ def stop_all
61
+ semaphore.synchronize { @subscriptions.each(&:stop_listening) }
62
+ end
63
+
64
+ # @return [Array<EventStoreSubscriptions::Subscription>]
65
+ def subscriptions
66
+ # Duping original collection to prevent potential mutable operations over it from user's side
67
+ semaphore.synchronize { @subscriptions.dup }
68
+ end
69
+
70
+ private
71
+
72
+ # @param stream_name [String]
73
+ # @return [Class<EventStoreSubscriptions::SubscriptionPosition>, Class<EventStoreSubscriptions::SubscriptionRevision>]
74
+ def position_class(stream_name)
75
+ stream_name == ALL_STREAM ? SubscriptionPosition : SubscriptionRevision
76
+ end
77
+ end
78
+ end
@@ -0,0 +1,5 @@
1
+ # frozen_string_literal: true
2
+
3
+ module EventStoreSubscriptions
4
+ VERSION = "1.0.0"
5
+ end
@@ -0,0 +1,15 @@
1
+ # frozen_string_literal: true
2
+
3
+ module EventStoreSubscriptions
4
+ module WaitForFinish
5
+ # Waits until state switches from :running to any other state.
6
+ # @return [void]
7
+ def wait_for_finish
8
+ loop do
9
+ break if state.stopped? || state.dead?
10
+
11
+ sleep 0.1
12
+ end
13
+ end
14
+ end
15
+ end
@@ -0,0 +1,110 @@
1
+ # frozen_string_literal: true
2
+
3
+ module EventStoreSubscriptions
4
+ # Watches over the given subscriptions collection and restarts dead subscriptions. It is useful
5
+ # in cases when your subscription's handler raises error. Its usage is optional.
6
+ class WatchDog
7
+ include WaitForFinish
8
+
9
+ CHECK_INTERVAL = 5 # seconds. How often to scan subscriptions
10
+
11
+ class << self
12
+ # @param collection [EventStoreSubscriptions::Subscriptions]
13
+ # @param restart_terminator [Proc, nil]
14
+ # @return [EventStoreSubscriptions::WatchDog]
15
+ def watch(collection, restart_terminator: nil)
16
+ new(collection).watch
17
+ end
18
+ end
19
+
20
+ attr_accessor :runner
21
+ attr_reader :collection, :state, :restart_terminator
22
+ private :runner, :runner=, :restart_terminator
23
+
24
+ # @param collection [EventStoreSubscriptions::Subscriptions]
25
+ # @param restart_terminator [Proc, nil] define a terminator that would halt Subscription restart
26
+ # process if the result of it execution is truthy. Subscription instance will be passed as a
27
+ # first argument into it, and, based on it, you should decide whether to process the restart
28
+ # or not.
29
+ def initialize(collection, restart_terminator: nil)
30
+ @collection = collection
31
+ @state = ObjectState.new
32
+ @runner = nil
33
+ @restart_terminator = restart_terminator
34
+ end
35
+
36
+ # Start watching over the given Subscriptions collection
37
+ # @return [EventStoreSubscriptions::WatchDog] returns self
38
+ def watch
39
+ self.runner ||=
40
+ begin
41
+ state.running!
42
+ Thread.new do
43
+ loop do
44
+ sleep CHECK_INTERVAL
45
+ break unless state.running?
46
+
47
+ collection.subscriptions.each do |sub|
48
+ break unless state.running?
49
+
50
+ restart_subscription(sub) if sub.state.dead?
51
+ end
52
+ end
53
+ rescue StandardError => e
54
+ state.dead!
55
+ raise
56
+ end
57
+ end
58
+ self
59
+ end
60
+
61
+ # Stop watching over the given subscriptions collection. This command is async - the result is
62
+ # not immediate. Use the #wait_for_finish method in order to wait until the runner has fully stopped .
63
+ # Example:
64
+ # ```ruby
65
+ # watch_dog.unwatch.wait_for_finish
66
+ # ```
67
+ # @return [EventStoreSubscriptions::WatchDog] returns self
68
+ def unwatch
69
+ return self unless runner&.alive?
70
+
71
+ state.halting!
72
+ Thread.new do
73
+ loop do
74
+ # If runner sleeps between runs we can safely shut it down. Even if the edge case happens,
75
+ # when a runner's status changes between its check and `runner.exit`, it is still ok, it
76
+ # would be shut down anyway because of the guard condition `break unless state.running?`
77
+ runner.exit if runner&.status == 'sleep'
78
+ unless runner&.alive?
79
+ state.stopped!
80
+ self.runner = nil
81
+ break
82
+ end
83
+ sleep 0.1
84
+ end
85
+ end
86
+ self
87
+ end
88
+
89
+ private
90
+
91
+ # @param failed_sub [EventStoreSubscriptions::Subscription]
92
+ # @return [EventStoreSubscriptions::Subscription] newly created Subscription
93
+ def restart_subscription(failed_sub)
94
+ return if restart_terminator&.call(failed_sub)
95
+ # Check if no one else did this job
96
+ return unless collection.remove(failed_sub)
97
+
98
+ new_sub = Subscription.new(
99
+ position: failed_sub.position,
100
+ client: failed_sub.client,
101
+ setup: failed_sub.setup,
102
+ statistic: failed_sub.statistic
103
+ )
104
+ new_sub.statistic.last_restart_at = Time.now.utc
105
+ collection.add(new_sub)
106
+ failed_sub.delete
107
+ new_sub.listen
108
+ end
109
+ end
110
+ end
@@ -0,0 +1,17 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'event_store_client'
4
+ require_relative 'event_store_subscriptions/version'
5
+ require_relative 'event_store_subscriptions/error'
6
+ require_relative 'event_store_subscriptions/wait_for_finish'
7
+ require_relative 'event_store_subscriptions/object_state'
8
+ require_relative 'event_store_subscriptions/subscription_statistic'
9
+ require_relative 'event_store_subscriptions/subscription'
10
+ require_relative 'event_store_subscriptions/subscription_position'
11
+ require_relative 'event_store_subscriptions/subscription_revision'
12
+ require_relative 'event_store_subscriptions/subscription_setup'
13
+ require_relative 'event_store_subscriptions/subscriptions'
14
+ require_relative 'event_store_subscriptions/watch_dog'
15
+
16
+ module EventStoreSubscriptions
17
+ end
metadata ADDED
@@ -0,0 +1,116 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: event_store_subscriptions
3
+ version: !ruby/object:Gem::Version
4
+ version: 1.0.0
5
+ platform: ruby
6
+ authors:
7
+ - Ivan Dzyzenko
8
+ autorequire:
9
+ bindir: exe
10
+ cert_chain: []
11
+ date: 2022-10-03 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: event_store_client
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - "~>"
18
+ - !ruby/object:Gem::Version
19
+ version: '2.0'
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - "~>"
25
+ - !ruby/object:Gem::Version
26
+ version: '2.0'
27
+ - !ruby/object:Gem::Dependency
28
+ name: pry
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - "~>"
32
+ - !ruby/object:Gem::Version
33
+ version: '0.14'
34
+ type: :development
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - "~>"
39
+ - !ruby/object:Gem::Version
40
+ version: '0.14'
41
+ - !ruby/object:Gem::Dependency
42
+ name: rspec
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - "~>"
46
+ - !ruby/object:Gem::Version
47
+ version: '3.11'
48
+ type: :development
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - "~>"
53
+ - !ruby/object:Gem::Version
54
+ version: '3.11'
55
+ - !ruby/object:Gem::Dependency
56
+ name: rake
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - "~>"
60
+ - !ruby/object:Gem::Version
61
+ version: '13.0'
62
+ type: :development
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - "~>"
67
+ - !ruby/object:Gem::Version
68
+ version: '13.0'
69
+ description: Implementation of subscription manager for `event_store_client` gem.
70
+ email:
71
+ - ivan.dzyzenko@gmail.com
72
+ executables: []
73
+ extensions: []
74
+ extra_rdoc_files: []
75
+ files:
76
+ - LICENSE.txt
77
+ - README.md
78
+ - lib/event_store_subscriptions.rb
79
+ - lib/event_store_subscriptions/error.rb
80
+ - lib/event_store_subscriptions/object_state.rb
81
+ - lib/event_store_subscriptions/subscription.rb
82
+ - lib/event_store_subscriptions/subscription_position.rb
83
+ - lib/event_store_subscriptions/subscription_revision.rb
84
+ - lib/event_store_subscriptions/subscription_setup.rb
85
+ - lib/event_store_subscriptions/subscription_statistic.rb
86
+ - lib/event_store_subscriptions/subscriptions.rb
87
+ - lib/event_store_subscriptions/version.rb
88
+ - lib/event_store_subscriptions/wait_for_finish.rb
89
+ - lib/event_store_subscriptions/watch_dog.rb
90
+ homepage: https://github.com/yousty/event_store_subscriptions
91
+ licenses:
92
+ - MIT
93
+ metadata:
94
+ allowed_push_host: https://rubygems.org
95
+ homepage_uri: https://github.com/yousty/event_store_subscriptions
96
+ source_code_uri: https://github.com/yousty/event_store_subscriptions
97
+ post_install_message:
98
+ rdoc_options: []
99
+ require_paths:
100
+ - lib
101
+ required_ruby_version: !ruby/object:Gem::Requirement
102
+ requirements:
103
+ - - ">="
104
+ - !ruby/object:Gem::Version
105
+ version: 2.7.0
106
+ required_rubygems_version: !ruby/object:Gem::Requirement
107
+ requirements:
108
+ - - ">="
109
+ - !ruby/object:Gem::Version
110
+ version: '0'
111
+ requirements: []
112
+ rubygems_version: 3.3.7
113
+ signing_key:
114
+ specification_version: 4
115
+ summary: Implementation of subscription manager for `event_store_client` gem.
116
+ test_files: []