event_sourcery 0.22.0 → 0.23.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +61 -2
- data/README.md +1 -1
- data/event_sourcery.gemspec +1 -1
- data/lib/event_sourcery/config.rb +2 -2
- data/lib/event_sourcery/event_processing/esp_runner.rb +46 -27
- data/lib/event_sourcery/version.rb +1 -1
- metadata +7 -8
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 99bf8987da0772344a41fc64908adbeaf61ec6586e7ca0598c0b64f46660c925
|
4
|
+
data.tar.gz: 0a19dcd0e8daf8522cbbbf713f712f688eceab25b18d8137cf3a0059cb030937
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 22cd971647408e0231de73e1a35d78bcd64f9ab396d147b6d7f1ab4e9757cdd5728705b6e10fe9c57547e2de6f5ee12662effc16570b5f62d5c10d08cbba913d
|
7
|
+
data.tar.gz: a13edee936cd4fa5dd203ee4758128d0c6342306c5d9028ee3868293e490d872f0fd27c92b087a24c65f7cab11b3d8ab949c29785873b0efff85c2ec3d448d9c
|
data/CHANGELOG.md
CHANGED
@@ -5,6 +5,64 @@ All notable changes to this project will be documented in this file.
|
|
5
5
|
The format is based on [Keep a Changelog](http://keepachangelog.com/)
|
6
6
|
and this project adheres to [Semantic Versioning](http://semver.org/).
|
7
7
|
|
8
|
+
## [Unreleased]
|
9
|
+
|
10
|
+
## [0.23.0] - 2019-07-11
|
11
|
+
### Added
|
12
|
+
- Add Ruby 2.6 to the CI test matrix.
|
13
|
+
- `ESPRunner` supports an `after_subprocess_termination` hook. This optional
|
14
|
+
initializer argument will will be executed when each child process
|
15
|
+
terminates. This allows for monitoring and alerts to be configured.
|
16
|
+
For example, Rollbar:
|
17
|
+
|
18
|
+
```ruby
|
19
|
+
EventSourcery::EventProcessing::ESPRunner.new(
|
20
|
+
event_processors: processors,
|
21
|
+
event_source: source,
|
22
|
+
after_subprocess_termination: proc do |processor:, runner:, exit_status:|
|
23
|
+
if exit_status != 0
|
24
|
+
Rollbar.error("Processor #{processor.processor_name} "\
|
25
|
+
"terminated with exit status #{exit_status}")
|
26
|
+
end
|
27
|
+
end
|
28
|
+
).start!
|
29
|
+
```
|
30
|
+
|
31
|
+
- `ESPRunner` exposes three new public methods `start_processor`, `shutdown`,
|
32
|
+
and `shutdown_requested?`. These provide options for handling subprocess
|
33
|
+
failure/termination. For example, shutting down the `ESPRunner`:
|
34
|
+
|
35
|
+
```ruby
|
36
|
+
EventSourcery::EventProcessing::ESPRunner.new(
|
37
|
+
event_processors: processors,
|
38
|
+
event_source: source,
|
39
|
+
after_subprocess_termination: proc do |processor:, runner:, exit_status:|
|
40
|
+
runner.shutdown
|
41
|
+
end
|
42
|
+
).start!
|
43
|
+
```
|
44
|
+
|
45
|
+
Or restarting the event processor:
|
46
|
+
|
47
|
+
```ruby
|
48
|
+
EventSourcery::EventProcessing::ESPRunner.new(
|
49
|
+
event_processors: processors,
|
50
|
+
event_source: source,
|
51
|
+
after_subprocess_termination: proc do |processor:, runner:, exit_status:|
|
52
|
+
runner.start_processor(processor) unless runner.shutdown_requested?
|
53
|
+
end
|
54
|
+
).start!
|
55
|
+
```
|
56
|
+
|
57
|
+
- `ESPRunner` checks for dead child processes every second. This means we
|
58
|
+
shouldn't see `[ruby] <defunct>` in the process list (ps) when a processor
|
59
|
+
fails.
|
60
|
+
- `ESPRunner` logs when child processes die.
|
61
|
+
- `ESPRunner` logs when sending signals to child processes.
|
62
|
+
|
63
|
+
### Removed
|
64
|
+
- Remove Ruby 2.2 from the CI test matrix.
|
65
|
+
|
8
66
|
## [0.22.0] - 2018-10-04
|
9
67
|
### Added
|
10
68
|
- Log critical exceptions to the application provided block via the new
|
@@ -29,7 +87,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
|
|
29
87
|
|
30
88
|
## [0.20.0] - 2018-06-21
|
31
89
|
### Changed
|
32
|
-
- Changed signature of `ESPProcess#initialize` to include a default value for `after_fork`. This prevents the
|
90
|
+
- Changed signature of `ESPProcess#initialize` to include a default value for `after_fork`. This prevents the
|
33
91
|
`after_fork` change from 0.19.0 from being a breaking change to external creators of ESPProcess.
|
34
92
|
- Added more logging when a fatal exception occurs in ESPProcess
|
35
93
|
|
@@ -163,7 +221,8 @@ moving all Postgres related code into a separate gem.
|
|
163
221
|
- EventSourcery no longer depends on Virtus.
|
164
222
|
- `Command` and `CommandHandler` have been removed.
|
165
223
|
|
166
|
-
[Unreleased]: https://github.com/envato/event_sourcery/compare/v0.
|
224
|
+
[Unreleased]: https://github.com/envato/event_sourcery/compare/v0.23.0...HEAD
|
225
|
+
[0.23.0]: https://github.com/envato/event_sourcery/compare/v0.22.0...v0.23.0
|
167
226
|
[0.22.0]: https://github.com/envato/event_sourcery/compare/v0.21.0...v0.22.0
|
168
227
|
[0.21.0]: https://github.com/envato/event_sourcery/compare/v0.20.0...v0.21.0
|
169
228
|
[0.20.0]: https://github.com/envato/event_sourcery/compare/v0.19.0...v0.20.0
|
data/README.md
CHANGED
@@ -218,7 +218,7 @@ The event store is a persistent store of events.
|
|
218
218
|
|
219
219
|
EventSourcery currently supports a Postgres-based event store via the [event_sourcery-postgres gem](https://github.com/envato/event_sourcery-postgres).
|
220
220
|
|
221
|
-
For more information about the `EventStore` API refer to [the postgres event store](https://github.com/envato/event_sourcery-postgres/blob/master/lib/event_sourcery/postgres/event_store.rb) or the [in memory event store in this repo](lib/event_sourcery/event_store
|
221
|
+
For more information about the `EventStore` API refer to [the postgres event store](https://github.com/envato/event_sourcery-postgres/blob/master/lib/event_sourcery/postgres/event_store.rb) or the [in memory event store in this repo](lib/event_sourcery/memory/event_store.rb)
|
222
222
|
|
223
223
|
#### Storing Events
|
224
224
|
|
data/event_sourcery.gemspec
CHANGED
@@ -25,7 +25,7 @@ Gem::Specification.new do |spec|
|
|
25
25
|
|
26
26
|
spec.required_ruby_version = '>= 2.2.0'
|
27
27
|
|
28
|
-
spec.add_development_dependency 'bundler'
|
28
|
+
spec.add_development_dependency 'bundler'
|
29
29
|
spec.add_development_dependency 'rake', '~> 10.0'
|
30
30
|
spec.add_development_dependency 'rspec'
|
31
31
|
spec.add_development_dependency 'pry'
|
@@ -58,10 +58,10 @@ module EventSourcery
|
|
58
58
|
end
|
59
59
|
|
60
60
|
# Logger instance used by EventSourcery.
|
61
|
-
# By default EventSourcery will log to STDOUT with a log level of Logger::
|
61
|
+
# By default EventSourcery will log to STDOUT with a log level of Logger::INFO
|
62
62
|
def logger
|
63
63
|
@logger ||= ::Logger.new(STDOUT).tap do |logger|
|
64
|
-
logger.level = Logger::
|
64
|
+
logger.level = Logger::INFO
|
65
65
|
end
|
66
66
|
end
|
67
67
|
|
@@ -7,14 +7,18 @@ module EventSourcery
|
|
7
7
|
event_source:,
|
8
8
|
max_seconds_for_processes_to_terminate: 30,
|
9
9
|
shutdown_requested: false,
|
10
|
-
after_fork: nil
|
10
|
+
after_fork: nil,
|
11
|
+
after_subprocess_termination: nil,
|
12
|
+
logger: EventSourcery.logger)
|
11
13
|
@event_processors = event_processors
|
12
14
|
@event_source = event_source
|
13
|
-
@pids =
|
15
|
+
@pids = {}
|
14
16
|
@max_seconds_for_processes_to_terminate = max_seconds_for_processes_to_terminate
|
15
17
|
@shutdown_requested = shutdown_requested
|
16
18
|
@exit_status = true
|
17
19
|
@after_fork = after_fork
|
20
|
+
@after_subprocess_termination = after_subprocess_termination
|
21
|
+
@logger = logger
|
18
22
|
end
|
19
23
|
|
20
24
|
# Start each Event Stream Processor in a new child process.
|
@@ -22,8 +26,9 @@ module EventSourcery
|
|
22
26
|
with_logging do
|
23
27
|
start_processes
|
24
28
|
listen_for_shutdown_signals
|
25
|
-
|
26
|
-
|
29
|
+
while_waiting_for_shutdown do
|
30
|
+
record_terminated_processes
|
31
|
+
end
|
27
32
|
terminate_remaining_processes
|
28
33
|
until all_processes_terminated? || waited_long_enough?
|
29
34
|
record_terminated_processes
|
@@ -34,25 +39,36 @@ module EventSourcery
|
|
34
39
|
exit_indicating_status_of_processes
|
35
40
|
end
|
36
41
|
|
42
|
+
def start_processor(event_processor)
|
43
|
+
process = ESPProcess.new(
|
44
|
+
event_processor: event_processor,
|
45
|
+
event_source: @event_source,
|
46
|
+
after_fork: @after_fork,
|
47
|
+
)
|
48
|
+
pid = Process.fork { process.start }
|
49
|
+
@pids[pid] = event_processor
|
50
|
+
end
|
51
|
+
|
52
|
+
def shutdown
|
53
|
+
@shutdown_requested = true
|
54
|
+
end
|
55
|
+
|
56
|
+
def shutdown_requested?
|
57
|
+
@shutdown_requested
|
58
|
+
end
|
59
|
+
|
37
60
|
private
|
38
61
|
|
62
|
+
attr_reader :logger
|
63
|
+
|
39
64
|
def with_logging
|
40
|
-
|
65
|
+
logger.info('ESPRunner: Forking processes')
|
41
66
|
yield
|
42
|
-
|
67
|
+
logger.info('ESPRunner: Processes shutdown')
|
43
68
|
end
|
44
69
|
|
45
70
|
def start_processes
|
46
|
-
@event_processors.each(&method(:
|
47
|
-
end
|
48
|
-
|
49
|
-
def start_process(event_processor)
|
50
|
-
process = ESPProcess.new(
|
51
|
-
event_processor: event_processor,
|
52
|
-
event_source: @event_source,
|
53
|
-
after_fork: @after_fork,
|
54
|
-
)
|
55
|
-
@pids << Process.fork { process.start }
|
71
|
+
@event_processors.each(&method(:start_processor))
|
56
72
|
end
|
57
73
|
|
58
74
|
def listen_for_shutdown_signals
|
@@ -61,12 +77,12 @@ module EventSourcery
|
|
61
77
|
end
|
62
78
|
end
|
63
79
|
|
64
|
-
def
|
65
|
-
|
66
|
-
|
67
|
-
|
68
|
-
|
69
|
-
|
80
|
+
def while_waiting_for_shutdown
|
81
|
+
loop do
|
82
|
+
yield
|
83
|
+
break if shutdown_requested?
|
84
|
+
sleep(1)
|
85
|
+
end
|
70
86
|
end
|
71
87
|
|
72
88
|
def terminate_remaining_processes
|
@@ -77,19 +93,22 @@ module EventSourcery
|
|
77
93
|
send_signal_to_remaining_processes(:KILL)
|
78
94
|
end
|
79
95
|
|
80
|
-
|
81
96
|
def send_signal_to_remaining_processes(signal)
|
82
|
-
|
97
|
+
return if all_processes_terminated?
|
98
|
+
|
99
|
+
logger.info("ESPRunner: Sending #{signal} to [#{@pids.values.map(&:processor_name).join(', ')}]")
|
100
|
+
Process.kill(signal, *@pids.keys)
|
83
101
|
rescue Errno::ESRCH
|
84
102
|
record_terminated_processes
|
85
103
|
retry
|
86
104
|
end
|
87
105
|
|
88
106
|
def record_terminated_processes
|
89
|
-
until all_processes_terminated? ||
|
90
|
-
|
91
|
-
@pids.delete(pid)
|
107
|
+
until all_processes_terminated? || (pid, status = Process.wait2(-1, Process::WNOHANG)).nil?
|
108
|
+
event_processor = @pids.delete(pid)
|
92
109
|
@exit_status &&= !!status.success?
|
110
|
+
logger.info("ESPRunner: Process #{event_processor.processor_name} terminated with exit status: #{status.exitstatus.inspect}")
|
111
|
+
@after_subprocess_termination&.call(processor: event_processor, runner: self, exit_status: status.exitstatus)
|
93
112
|
end
|
94
113
|
end
|
95
114
|
|
metadata
CHANGED
@@ -1,29 +1,29 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: event_sourcery
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.23.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Envato
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2019-07-11 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: bundler
|
15
15
|
requirement: !ruby/object:Gem::Requirement
|
16
16
|
requirements:
|
17
|
-
- - "
|
17
|
+
- - ">="
|
18
18
|
- !ruby/object:Gem::Version
|
19
|
-
version: '
|
19
|
+
version: '0'
|
20
20
|
type: :development
|
21
21
|
prerelease: false
|
22
22
|
version_requirements: !ruby/object:Gem::Requirement
|
23
23
|
requirements:
|
24
|
-
- - "
|
24
|
+
- - ">="
|
25
25
|
- !ruby/object:Gem::Version
|
26
|
-
version: '
|
26
|
+
version: '0'
|
27
27
|
- !ruby/object:Gem::Dependency
|
28
28
|
name: rake
|
29
29
|
requirement: !ruby/object:Gem::Requirement
|
@@ -144,8 +144,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
144
144
|
- !ruby/object:Gem::Version
|
145
145
|
version: '0'
|
146
146
|
requirements: []
|
147
|
-
|
148
|
-
rubygems_version: 2.7.3
|
147
|
+
rubygems_version: 3.0.4
|
149
148
|
signing_key:
|
150
149
|
specification_version: 4
|
151
150
|
summary: Event Sourcing Library
|