event_sourcery-postgres 0.4.0 → 0.8.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +5 -5
- data/CHANGELOG.md +63 -6
- data/README.md +2 -6
- data/event_sourcery-postgres.gemspec +8 -3
- data/lib/event_sourcery/postgres.rb +0 -1
- data/lib/event_sourcery/postgres/config.rb +31 -9
- data/lib/event_sourcery/postgres/event_store.rb +53 -12
- data/lib/event_sourcery/postgres/optimised_event_poll_waiter.rb +3 -3
- data/lib/event_sourcery/postgres/projector.rb +14 -7
- data/lib/event_sourcery/postgres/reactor.rb +11 -0
- data/lib/event_sourcery/postgres/schema.rb +30 -1
- data/lib/event_sourcery/postgres/table_owner.rb +11 -0
- data/lib/event_sourcery/postgres/tracker.rb +35 -7
- data/lib/event_sourcery/postgres/version.rb +1 -1
- metadata +19 -17
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
|
-
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: e0846cda7986faeccb438d159a1fe87e0388855b2a898a5703dbc95d8bbce425
|
|
4
|
+
data.tar.gz: c29112988c1049f8978e95c276f1bb96bfc1f5bdf36b491f68dad0dd9ac2b9ce
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 38a3097e2582d651b4af1d421a4f431f888f09b8cc369f44c059276449f17e9dbac29a2f08bfc1eea7dbf9eae39eebcb3788d5f1d9d72f8702e309a7b3cfba14
|
|
7
|
+
data.tar.gz: c384b9737f5888df616f1dc776db9411616434f0025d5ef888d6ad273d9757ebecfec56ff5ee4dc9eca1e05553f5c3cc6dc375aecbb8a979c62e1199b7dbacfb
|
data/CHANGELOG.md
CHANGED
|
@@ -1,4 +1,5 @@
|
|
|
1
1
|
# Change Log
|
|
2
|
+
|
|
2
3
|
All notable changes to this project will be documented in this file.
|
|
3
4
|
|
|
4
5
|
The format is based on [Keep a Changelog](http://keepachangelog.com/)
|
|
@@ -6,7 +7,53 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
|
|
|
6
7
|
|
|
7
8
|
## [Unreleased]
|
|
8
9
|
|
|
9
|
-
## [0.
|
|
10
|
+
## [0.8.1] - 2020-10-02
|
|
11
|
+
### Added
|
|
12
|
+
- Add Ruby 2.6 and 2.7 to the CI test matrix.
|
|
13
|
+
|
|
14
|
+
### Removed
|
|
15
|
+
- Remove Ruby 2.2 from the CI test matrix.
|
|
16
|
+
- Support for Boxen.
|
|
17
|
+
|
|
18
|
+
### Fixed
|
|
19
|
+
- Upgrade development dependency Rake to version 13. This resolves
|
|
20
|
+
[CVE-2020-8130](https://github.com/advisories/GHSA-jppv-gw3r-w3q8).
|
|
21
|
+
|
|
22
|
+
- Resolve warnings raised when running on Ruby 2.7.
|
|
23
|
+
|
|
24
|
+
## [0.8.0] - 2018-08-06
|
|
25
|
+
### Added
|
|
26
|
+
- Add a `on_events_recorded` config option, that defaults to a no-op proc, \
|
|
27
|
+
to handle any app specific logic after the events are recoded on `EventStore#sink`
|
|
28
|
+
|
|
29
|
+
## [0.7.0] - 2018-05-23
|
|
30
|
+
### Added
|
|
31
|
+
- Add a `projector_transaction_size` config option to control how many events
|
|
32
|
+
are processed before the transaction is commited. The default value is 1 to
|
|
33
|
+
match the existing behavour.
|
|
34
|
+
|
|
35
|
+
We suggest setting this to match the number of events returned from the event
|
|
36
|
+
store subscription. This is [now configurable](https://github.com/envato/event_sourcery/pull/197)
|
|
37
|
+
in event_sourcery by configuring `subscription_batch_size`.
|
|
38
|
+
|
|
39
|
+
### Removed
|
|
40
|
+
- Remove upper bound version restriction on `sequel` gem. Now accepts versions
|
|
41
|
+
5 and higher.
|
|
42
|
+
|
|
43
|
+
## [0.6.0] - 2018-01-02
|
|
44
|
+
### Changed
|
|
45
|
+
|
|
46
|
+
- Only send info log after processing a group of events
|
|
47
|
+
|
|
48
|
+
### Removed
|
|
49
|
+
- Remove `processes_events` and `projects_events` as these have been [removed
|
|
50
|
+
in event_sourcery](https://github.com/envato/event_sourcery/pull/161).
|
|
51
|
+
|
|
52
|
+
## [0.5.0] - 2017-07-27
|
|
53
|
+
- First Version of YARD documentation.
|
|
54
|
+
- Fix Sequel deprecation by globally loading pg extensions
|
|
55
|
+
|
|
56
|
+
## [0.4.0] - 2017-06-21
|
|
10
57
|
### Changed
|
|
11
58
|
- Reactors store the UUID of the event being processed in the `causation_id`
|
|
12
59
|
of any emitted events. This replaces the old behaviour of storing id of the
|
|
@@ -19,27 +66,37 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
|
|
|
19
66
|
- Added index on the `events` table for `correlation_id` and `causation_id`
|
|
20
67
|
columns.
|
|
21
68
|
|
|
22
|
-
## [0.3.0] - 2017-
|
|
69
|
+
## [0.3.0] - 2017-06-16
|
|
23
70
|
### Changed
|
|
24
71
|
- The event store persists the event `correlation_id` and `causation_id`.
|
|
25
72
|
To facilitate this `correlation_id` and `causation_id` columns have been
|
|
26
73
|
added to the `events` table and the `write_events` function has been
|
|
27
74
|
altered. Event Sourcery apps will need to ensure these DB changes have
|
|
28
75
|
been applied to use this version of Event Sourcery.
|
|
76
|
+
- The emit_events method now accepts typed events instead of symbols
|
|
77
|
+
- Remove dynamic emit events methods from reactors (e.g. emit_item_added)
|
|
29
78
|
|
|
30
|
-
## [0.2.0] - 2017-
|
|
79
|
+
## [0.2.0] - 2017-06-01
|
|
31
80
|
### Changed
|
|
32
81
|
- Make `EventSourcery::Postgres::OptimisedEventPollWaiter#shutdown` private
|
|
33
82
|
- Updated `EventSourcery::Postgres::OptimisedEventPollWaiter#poll` to ensure that `#shutdown!` is run when an error is raised
|
|
34
83
|
or when the loop stops
|
|
35
|
-
- Remove dynamic emit events methods from reactors (e.g. emit_item_added)
|
|
36
|
-
- The emit_events method now accepts typed events instead of symbols
|
|
37
84
|
|
|
38
85
|
### Added
|
|
39
86
|
- Configure projector tracker table name via `EventSourcery::Postgres.configure`
|
|
40
87
|
|
|
41
|
-
##
|
|
88
|
+
## 0.1.0 - 2017-05-26
|
|
42
89
|
### Changed
|
|
43
90
|
- Imported code from the [event_sourcery](https://github.com/envato/event_sourcery).
|
|
44
91
|
- Postgres related configuration is through `EventSourcery::Postgres.configure`
|
|
45
92
|
instead of `EventSourcery.configure`.
|
|
93
|
+
|
|
94
|
+
[Unreleased]: https://github.com/envato/event_sourcery-postgres/compare/v0.8.1...HEAD
|
|
95
|
+
[0.8.1]: https://github.com/envato/event_sourcery-postgres/compare/v0.8.0...v0.8.1
|
|
96
|
+
[0.8.0]: https://github.com/envato/event_sourcery-postgres/compare/v0.7.0...v0.8.0
|
|
97
|
+
[0.7.0]: https://github.com/envato/event_sourcery-postgres/compare/v0.6.0...v0.7.0
|
|
98
|
+
[0.6.0]: https://github.com/envato/event_sourcery-postgres/compare/v0.5.0...v0.6.0
|
|
99
|
+
[0.5.0]: https://github.com/envato/event_sourcery-postgres/compare/v0.4.0...v0.5.0
|
|
100
|
+
[0.4.0]: https://github.com/envato/event_sourcery-postgres/compare/v0.3.0...v0.4.0
|
|
101
|
+
[0.3.0]: https://github.com/envato/event_sourcery-postgres/compare/v0.2.0...v0.3.0
|
|
102
|
+
[0.2.0]: https://github.com/envato/event_sourcery-postgres/compare/v0.1.0...v0.2.0
|
data/README.md
CHANGED
|
@@ -22,7 +22,6 @@ gem 'event_sourcery-postgres'
|
|
|
22
22
|
EventSourcery::Postgres.configure do |config|
|
|
23
23
|
config.event_store_database = Sequel.connect(...)
|
|
24
24
|
config.projections_database = Sequel.connect(...)
|
|
25
|
-
config.use_optimistic_concurrency = true
|
|
26
25
|
config.write_events_function_name = 'writeEvents'
|
|
27
26
|
config.events_table_name = :events
|
|
28
27
|
config.aggregates_table_name = :aggregates
|
|
@@ -91,11 +90,8 @@ To release a new version:
|
|
|
91
90
|
|
|
92
91
|
1. Update the version number in `lib/event_sourcery/postgres/version.rb`
|
|
93
92
|
2. Get this change onto master via the normal PR process
|
|
94
|
-
3. Run `
|
|
95
|
-
|
|
96
|
-
push tags up to GitHub, and package the code in a `.gem` file.
|
|
97
|
-
4. Manually upload the generated gem file (`pkg/event_sourcery-postgres-#{version}.gem`) to
|
|
98
|
-
[rubygems.envato.com](https://rubygems.envato.com).
|
|
93
|
+
3. Run `bundle exec rake release`, this will create a git tag for the
|
|
94
|
+
version, push tags up to GitHub, and upload the gem to rubygems.org.
|
|
99
95
|
|
|
100
96
|
## Contributing
|
|
101
97
|
|
|
@@ -12,6 +12,11 @@ Gem::Specification.new do |spec|
|
|
|
12
12
|
|
|
13
13
|
spec.summary = 'Postgres event store for use with EventSourcery'
|
|
14
14
|
spec.homepage = 'https://github.com/envato/event_sourcery-postgres'
|
|
15
|
+
spec.metadata = {
|
|
16
|
+
'bug_tracker_uri' => 'https://github.com/envato/event_sourcery-postgres/issues',
|
|
17
|
+
'changelog_uri' => 'https://github.com/envato/event_sourcery-postgres/blob/master/CHANGELOG.md',
|
|
18
|
+
'source_code_uri' => 'https://github.com/envato/event_sourcery-postgres',
|
|
19
|
+
}
|
|
15
20
|
|
|
16
21
|
spec.files = `git ls-files -z`.split("\x0").reject do |f|
|
|
17
22
|
f.match(%r{^(\.|bin/|Gemfile|Rakefile|script/|spec/)})
|
|
@@ -22,11 +27,11 @@ Gem::Specification.new do |spec|
|
|
|
22
27
|
|
|
23
28
|
spec.required_ruby_version = '>= 2.2.0'
|
|
24
29
|
|
|
25
|
-
spec.add_dependency 'sequel', '
|
|
30
|
+
spec.add_dependency 'sequel', '>= 4.38'
|
|
26
31
|
spec.add_dependency 'pg'
|
|
27
32
|
spec.add_dependency 'event_sourcery', '>= 0.14.0'
|
|
28
|
-
spec.add_development_dependency 'bundler'
|
|
29
|
-
spec.add_development_dependency 'rake', '~>
|
|
33
|
+
spec.add_development_dependency 'bundler'
|
|
34
|
+
spec.add_development_dependency 'rake', '~> 13'
|
|
30
35
|
spec.add_development_dependency 'rspec', '~> 3.0'
|
|
31
36
|
spec.add_development_dependency 'pry'
|
|
32
37
|
spec.add_development_dependency 'benchmark-ips'
|
|
@@ -1,8 +1,7 @@
|
|
|
1
1
|
module EventSourcery
|
|
2
2
|
module Postgres
|
|
3
3
|
class Config
|
|
4
|
-
attr_accessor :
|
|
5
|
-
:lock_table_to_guarantee_linear_sequence_id_growth,
|
|
4
|
+
attr_accessor :lock_table_to_guarantee_linear_sequence_id_growth,
|
|
6
5
|
:write_events_function_name,
|
|
7
6
|
:events_table_name,
|
|
8
7
|
:aggregates_table_name,
|
|
@@ -10,10 +9,15 @@ module EventSourcery
|
|
|
10
9
|
:callback_interval_if_no_new_events,
|
|
11
10
|
:auto_create_projector_tracker,
|
|
12
11
|
:event_tracker,
|
|
13
|
-
:
|
|
14
|
-
:
|
|
15
|
-
|
|
16
|
-
|
|
12
|
+
:projector_transaction_size,
|
|
13
|
+
:on_events_recorded
|
|
14
|
+
|
|
15
|
+
attr_writer :event_store,
|
|
16
|
+
:event_source,
|
|
17
|
+
:event_sink
|
|
18
|
+
|
|
19
|
+
attr_reader :event_store_database,
|
|
20
|
+
:projections_database
|
|
17
21
|
|
|
18
22
|
def initialize
|
|
19
23
|
@lock_table_to_guarantee_linear_sequence_id_growth = true
|
|
@@ -24,6 +28,8 @@ module EventSourcery
|
|
|
24
28
|
@callback_interval_if_no_new_events = 10
|
|
25
29
|
@event_store_database = nil
|
|
26
30
|
@auto_create_projector_tracker = true
|
|
31
|
+
@projector_transaction_size = 1
|
|
32
|
+
@on_events_recorded = ->(events) {}
|
|
27
33
|
end
|
|
28
34
|
|
|
29
35
|
def event_store
|
|
@@ -38,9 +44,25 @@ module EventSourcery
|
|
|
38
44
|
@event_sink ||= ::EventSourcery::EventStore::EventSink.new(event_store)
|
|
39
45
|
end
|
|
40
46
|
|
|
41
|
-
def
|
|
42
|
-
|
|
43
|
-
|
|
47
|
+
def event_store_database=(db_connection)
|
|
48
|
+
setup_connection(db_connection)
|
|
49
|
+
|
|
50
|
+
@event_store_database = db_connection
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
def projections_database=(db_connection)
|
|
54
|
+
setup_connection(db_connection)
|
|
55
|
+
|
|
56
|
+
@projections_database = db_connection
|
|
57
|
+
@event_tracker = Postgres::Tracker.new(db_connection)
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
private
|
|
61
|
+
|
|
62
|
+
def setup_connection(db_connection)
|
|
63
|
+
return unless db_connection
|
|
64
|
+
|
|
65
|
+
db_connection.extension :pg_json
|
|
44
66
|
end
|
|
45
67
|
end
|
|
46
68
|
end
|
|
@@ -3,25 +3,39 @@ module EventSourcery
|
|
|
3
3
|
class EventStore
|
|
4
4
|
include EventSourcery::EventStore::EachByRange
|
|
5
5
|
|
|
6
|
-
def initialize(
|
|
6
|
+
def initialize(db_connection,
|
|
7
7
|
events_table_name: EventSourcery::Postgres.config.events_table_name,
|
|
8
8
|
lock_table: EventSourcery::Postgres.config.lock_table_to_guarantee_linear_sequence_id_growth,
|
|
9
9
|
write_events_function_name: EventSourcery::Postgres.config.write_events_function_name,
|
|
10
|
-
event_builder: EventSourcery.config.event_builder
|
|
11
|
-
|
|
10
|
+
event_builder: EventSourcery.config.event_builder,
|
|
11
|
+
on_events_recorded: EventSourcery::Postgres.config.on_events_recorded)
|
|
12
|
+
@db_connection = db_connection
|
|
12
13
|
@events_table_name = events_table_name
|
|
13
14
|
@write_events_function_name = write_events_function_name
|
|
14
15
|
@lock_table = lock_table
|
|
15
16
|
@event_builder = event_builder
|
|
17
|
+
@on_events_recorded = on_events_recorded
|
|
16
18
|
end
|
|
17
19
|
|
|
20
|
+
# Like water flowing into a sink eventually it will go down the drain
|
|
21
|
+
# into the goodness of the plumbing system.
|
|
22
|
+
# So to will the given events you put in this 'sink'. Except the plumbing
|
|
23
|
+
# system is the data base events table.
|
|
24
|
+
# This can raise db connection errors.
|
|
25
|
+
#
|
|
26
|
+
# @param event_or_events the event or events to save
|
|
27
|
+
# @param expected_version the version to save with the event, default nil
|
|
28
|
+
#
|
|
29
|
+
# @raise [DatabaseError] if something goes wrong with the database
|
|
30
|
+
# @raise [ConcurrencyError] if there was a concurrency conflict
|
|
18
31
|
def sink(event_or_events, expected_version: nil)
|
|
19
32
|
events = Array(event_or_events)
|
|
20
33
|
aggregate_ids = events.map(&:aggregate_id).uniq
|
|
21
34
|
raise AtomicWriteToMultipleAggregatesNotSupported unless aggregate_ids.count == 1
|
|
22
35
|
sql = write_events_sql(aggregate_ids.first, events, expected_version)
|
|
23
|
-
@
|
|
36
|
+
@db_connection.run(sql)
|
|
24
37
|
log_events_saved(events)
|
|
38
|
+
on_events_recorded.call(events)
|
|
25
39
|
true
|
|
26
40
|
rescue Sequel::DatabaseError => e
|
|
27
41
|
if e.message =~ /Concurrency conflict/
|
|
@@ -31,15 +45,29 @@ module EventSourcery
|
|
|
31
45
|
end
|
|
32
46
|
end
|
|
33
47
|
|
|
48
|
+
# Get the next set of events from the given event id. You can
|
|
49
|
+
# specify event types and a limit.
|
|
50
|
+
# Default limit is 1000 and the default event types will be all.
|
|
51
|
+
#
|
|
52
|
+
# @param id the event id to get next events from
|
|
53
|
+
# @param event_types the event types to filter, default nil = all
|
|
54
|
+
# @param limit the limit to the results, default 1000
|
|
55
|
+
#
|
|
56
|
+
# @return [Array] array of found events
|
|
34
57
|
def get_next_from(id, event_types: nil, limit: 1000)
|
|
35
58
|
query = events_table.
|
|
36
59
|
order(:id).
|
|
37
60
|
where(Sequel.lit('id >= ?', id)).
|
|
38
61
|
limit(limit)
|
|
39
62
|
query = query.where(type: event_types) if event_types
|
|
40
|
-
query.map { |event_row| build_event(event_row) }
|
|
63
|
+
query.map { |event_row| build_event(**event_row) }
|
|
41
64
|
end
|
|
42
65
|
|
|
66
|
+
# Get last event id for a given event types.
|
|
67
|
+
#
|
|
68
|
+
# @param event_types the type of event(s) to filter
|
|
69
|
+
#
|
|
70
|
+
# @return the latest event id
|
|
43
71
|
def latest_event_id(event_types: nil)
|
|
44
72
|
latest_event = events_table
|
|
45
73
|
latest_event = latest_event.where(type: event_types) if event_types
|
|
@@ -51,14 +79,25 @@ module EventSourcery
|
|
|
51
79
|
end
|
|
52
80
|
end
|
|
53
81
|
|
|
82
|
+
# Get the events for a given aggregate id.
|
|
83
|
+
#
|
|
84
|
+
# @param aggregate_id the aggregate id to filter for
|
|
85
|
+
#
|
|
86
|
+
# @return [Array] of found events
|
|
54
87
|
def get_events_for_aggregate_id(aggregate_id)
|
|
55
88
|
events_table.where(aggregate_id: aggregate_id.to_str).order(:version).map do |event_hash|
|
|
56
|
-
build_event(event_hash)
|
|
89
|
+
build_event(**event_hash)
|
|
57
90
|
end
|
|
58
91
|
end
|
|
59
92
|
|
|
93
|
+
# Subscribe to events.
|
|
94
|
+
#
|
|
95
|
+
# @param from_id subscribe from a starting event id. default will be from the start.
|
|
96
|
+
# @param event_types the event_types to subscribe to, default all.
|
|
97
|
+
# @param after_listen the after listen call back block. default nil.
|
|
98
|
+
# @param subscription_master the subscription master block
|
|
60
99
|
def subscribe(from_id:, event_types: nil, after_listen: nil, subscription_master:, &block)
|
|
61
|
-
poll_waiter = OptimisedEventPollWaiter.new(
|
|
100
|
+
poll_waiter = OptimisedEventPollWaiter.new(db_connection: @db_connection, after_listen: after_listen)
|
|
62
101
|
args = {
|
|
63
102
|
poll_waiter: poll_waiter,
|
|
64
103
|
event_store: self,
|
|
@@ -68,17 +107,19 @@ module EventSourcery
|
|
|
68
107
|
subscription_master: subscription_master,
|
|
69
108
|
on_new_events: block
|
|
70
109
|
}
|
|
71
|
-
EventSourcery::EventStore::Subscription.new(args).tap(&:start)
|
|
110
|
+
EventSourcery::EventStore::Subscription.new(**args).tap(&:start)
|
|
72
111
|
end
|
|
73
112
|
|
|
74
113
|
private
|
|
75
114
|
|
|
115
|
+
attr_reader :on_events_recorded
|
|
116
|
+
|
|
76
117
|
def events_table
|
|
77
|
-
@
|
|
118
|
+
@db_connection[@events_table_name]
|
|
78
119
|
end
|
|
79
120
|
|
|
80
|
-
def build_event(data)
|
|
81
|
-
@event_builder.build(data)
|
|
121
|
+
def build_event(**data)
|
|
122
|
+
@event_builder.build(**data)
|
|
82
123
|
end
|
|
83
124
|
|
|
84
125
|
def write_events_sql(aggregate_id, events, expected_version)
|
|
@@ -123,7 +164,7 @@ module EventSourcery
|
|
|
123
164
|
else
|
|
124
165
|
value
|
|
125
166
|
end
|
|
126
|
-
@
|
|
167
|
+
@db_connection.literal(wrapped_value)
|
|
127
168
|
end
|
|
128
169
|
|
|
129
170
|
def log_events_saved(events)
|
|
@@ -4,8 +4,8 @@ module EventSourcery
|
|
|
4
4
|
class OptimisedEventPollWaiter
|
|
5
5
|
ListenThreadDied = Class.new(StandardError)
|
|
6
6
|
|
|
7
|
-
def initialize(
|
|
8
|
-
@
|
|
7
|
+
def initialize(db_connection:, timeout: 30, after_listen: proc {})
|
|
8
|
+
@db_connection = db_connection
|
|
9
9
|
@timeout = timeout
|
|
10
10
|
@events_queue = QueueWithIntervalCallback.new
|
|
11
11
|
@after_listen = after_listen
|
|
@@ -65,7 +65,7 @@ module EventSourcery
|
|
|
65
65
|
end
|
|
66
66
|
|
|
67
67
|
def listen_for_new_events(loop: true, after_listen: nil, timeout: 30)
|
|
68
|
-
@
|
|
68
|
+
@db_connection.listen('new_event',
|
|
69
69
|
loop: loop,
|
|
70
70
|
after_listen: after_listen,
|
|
71
71
|
timeout: timeout) do |_channel, _pid, _payload|
|
|
@@ -10,7 +10,6 @@ module EventSourcery
|
|
|
10
10
|
|
|
11
11
|
class << self
|
|
12
12
|
alias_method :project, :process
|
|
13
|
-
alias_method :projects_events, :processes_events
|
|
14
13
|
alias_method :projector_name, :processor_name
|
|
15
14
|
end
|
|
16
15
|
end
|
|
@@ -18,23 +17,31 @@ module EventSourcery
|
|
|
18
17
|
|
|
19
18
|
module InstanceMethods
|
|
20
19
|
def initialize(tracker: EventSourcery::Postgres.config.event_tracker,
|
|
21
|
-
db_connection: EventSourcery::Postgres.config.projections_database
|
|
20
|
+
db_connection: EventSourcery::Postgres.config.projections_database,
|
|
21
|
+
transaction_size: EventSourcery::Postgres.config.projector_transaction_size)
|
|
22
22
|
@tracker = tracker
|
|
23
23
|
@db_connection = db_connection
|
|
24
|
+
@transaction_size = transaction_size
|
|
24
25
|
end
|
|
25
26
|
|
|
26
27
|
private
|
|
27
28
|
|
|
29
|
+
attr_reader :transaction_size
|
|
30
|
+
|
|
28
31
|
def process_events(events, subscription_master)
|
|
29
|
-
events.
|
|
32
|
+
events.each_slice(transaction_size) do |slice_of_events|
|
|
30
33
|
subscription_master.shutdown_if_requested
|
|
34
|
+
|
|
31
35
|
db_connection.transaction do
|
|
32
|
-
|
|
33
|
-
|
|
36
|
+
slice_of_events.each do |event|
|
|
37
|
+
process(event)
|
|
38
|
+
EventSourcery.logger.debug { "[#{processor_name}] Processed event: #{event.inspect}" }
|
|
39
|
+
end
|
|
40
|
+
tracker.processed_event(processor_name, slice_of_events.last.id)
|
|
34
41
|
end
|
|
35
|
-
EventSourcery.logger.debug { "[#{processor_name}] Processed event: #{event.inspect}" }
|
|
36
|
-
EventSourcery.logger.info { "[#{processor_name}] Processed up to event id: #{events.last.id}" }
|
|
37
42
|
end
|
|
43
|
+
|
|
44
|
+
EventSourcery.logger.info { "[#{processor_name}] Processed up to event id: #{events.last.id}" }
|
|
38
45
|
end
|
|
39
46
|
end
|
|
40
47
|
end
|
|
@@ -11,18 +11,29 @@ module EventSourcery
|
|
|
11
11
|
end
|
|
12
12
|
|
|
13
13
|
module ClassMethods
|
|
14
|
+
# Assign the types of events this reactor can emit.
|
|
15
|
+
#
|
|
16
|
+
# @param event_types the types of events this reactor can emit
|
|
14
17
|
def emits_events(*event_types)
|
|
15
18
|
@emits_event_types = event_types
|
|
16
19
|
end
|
|
17
20
|
|
|
21
|
+
# @return [Array] an array of the types of events this reactor can emit
|
|
18
22
|
def emit_events
|
|
19
23
|
@emits_event_types ||= []
|
|
20
24
|
end
|
|
21
25
|
|
|
26
|
+
# This will tell you if this reactor emits any type of event.
|
|
27
|
+
#
|
|
28
|
+
# @return [true, false] true if this emits events, false if not
|
|
22
29
|
def emits_events?
|
|
23
30
|
!emit_events.empty?
|
|
24
31
|
end
|
|
25
32
|
|
|
33
|
+
# Will check if this reactor emits the given type of event.
|
|
34
|
+
#
|
|
35
|
+
# @param event_type the event type to check
|
|
36
|
+
# @return [true, false] true if it does emit the given event false if not
|
|
26
37
|
def emits_event?(event_type)
|
|
27
38
|
emit_events.include?(event_type)
|
|
28
39
|
end
|
|
@@ -3,6 +3,12 @@ module EventSourcery
|
|
|
3
3
|
module Schema
|
|
4
4
|
module_function
|
|
5
5
|
|
|
6
|
+
# This will create the event store tables and functions
|
|
7
|
+
# (event, aggregates, tracker and create or update functions)
|
|
8
|
+
# for the given Postgres database.
|
|
9
|
+
# The default will be the one specified in the config.
|
|
10
|
+
#
|
|
11
|
+
# @param db the Postgres database to use
|
|
6
12
|
def create_event_store(db: EventSourcery::Postgres.config.event_store_database,
|
|
7
13
|
events_table_name: EventSourcery::Postgres.config.events_table_name,
|
|
8
14
|
aggregates_table_name: EventSourcery::Postgres.config.aggregates_table_name,
|
|
@@ -12,6 +18,11 @@ module EventSourcery
|
|
|
12
18
|
create_or_update_functions(db: db, events_table_name: events_table_name, function_name: write_events_function_name, aggregates_table_name: aggregates_table_name)
|
|
13
19
|
end
|
|
14
20
|
|
|
21
|
+
# Create the events table. Needs the database and the table name.
|
|
22
|
+
# The defaults will be whats specified in config.
|
|
23
|
+
#
|
|
24
|
+
# @param db the Postgres database to use
|
|
25
|
+
# @param table_name the name of the events table
|
|
15
26
|
def create_events(db: EventSourcery::Postgres.config.event_store_database,
|
|
16
27
|
table_name: EventSourcery::Postgres.config.events_table_name)
|
|
17
28
|
db.run 'CREATE EXTENSION IF NOT EXISTS "uuid-ossp"'
|
|
@@ -34,14 +45,27 @@ module EventSourcery
|
|
|
34
45
|
end
|
|
35
46
|
end
|
|
36
47
|
|
|
48
|
+
# Create the aggregates table. Needs the database and the table name.
|
|
49
|
+
# The defaults will be whats specified in config.
|
|
50
|
+
#
|
|
51
|
+
# @param db the Postgres database to use
|
|
52
|
+
# @param table_name the name of the aggregates table
|
|
37
53
|
def create_aggregates(db: EventSourcery::Postgres.config.event_store_database,
|
|
38
54
|
table_name: EventSourcery::Postgres.config.aggregates_table_name)
|
|
39
55
|
db.create_table(table_name) do
|
|
40
|
-
|
|
56
|
+
uuid :aggregate_id, primary_key: true
|
|
41
57
|
column :version, :bigint, default: 1
|
|
42
58
|
end
|
|
43
59
|
end
|
|
44
60
|
|
|
61
|
+
# Create the 'create or update' functions.
|
|
62
|
+
# Needs the database, table name, function name and aggregates table name.
|
|
63
|
+
# The defaults will be whats specified in config.
|
|
64
|
+
#
|
|
65
|
+
# @param db the Postgres database to use
|
|
66
|
+
# @param function_name the name of the write events function
|
|
67
|
+
# @param events_table_name the name of the events table
|
|
68
|
+
# @param aggregates_table_name the name of the aggregates table
|
|
45
69
|
def create_or_update_functions(db: EventSourcery::Postgres.config.event_store_database,
|
|
46
70
|
function_name: EventSourcery::Postgres.config.write_events_function_name,
|
|
47
71
|
events_table_name: EventSourcery::Postgres.config.events_table_name,
|
|
@@ -138,6 +162,11 @@ $$ language plpgsql;
|
|
|
138
162
|
SQL
|
|
139
163
|
end
|
|
140
164
|
|
|
165
|
+
# Create the projector tracker table. Needs the database and the table name.
|
|
166
|
+
# The defaults will be whats specified in config.
|
|
167
|
+
#
|
|
168
|
+
# @param db the Postgres database to use
|
|
169
|
+
# @param table_name the name of the aggregates table
|
|
141
170
|
def create_projector_tracker(db: EventSourcery::Postgres.config.projections_database,
|
|
142
171
|
table_name: EventSourcery::Postgres.config.tracker_table_name)
|
|
143
172
|
db.create_table(table_name) do
|
|
@@ -9,15 +9,23 @@ module EventSourcery
|
|
|
9
9
|
end
|
|
10
10
|
|
|
11
11
|
module ClassMethods
|
|
12
|
+
# Hash of the tables and their corresponding blocks.
|
|
13
|
+
#
|
|
14
|
+
# @return [Hash] hash keyed by table names and block values
|
|
12
15
|
def tables
|
|
13
16
|
@tables ||= {}
|
|
14
17
|
end
|
|
15
18
|
|
|
19
|
+
# For the given table name assign to give block as the value.
|
|
20
|
+
#
|
|
21
|
+
# @param name the name of the table
|
|
22
|
+
# @param block the block of code to assign for the table
|
|
16
23
|
def table(name, &block)
|
|
17
24
|
tables[name] = block
|
|
18
25
|
end
|
|
19
26
|
end
|
|
20
27
|
|
|
28
|
+
# Create each table.
|
|
21
29
|
def setup
|
|
22
30
|
self.class.tables.each do |table_name, schema_block|
|
|
23
31
|
prefixed_name = table_name_prefixed(table_name)
|
|
@@ -26,6 +34,7 @@ module EventSourcery
|
|
|
26
34
|
super if defined?(super)
|
|
27
35
|
end
|
|
28
36
|
|
|
37
|
+
# Reset by dropping each table.
|
|
29
38
|
def reset
|
|
30
39
|
self.class.tables.keys.each do |table_name|
|
|
31
40
|
prefixed_name = table_name_prefixed(table_name)
|
|
@@ -37,6 +46,8 @@ module EventSourcery
|
|
|
37
46
|
setup
|
|
38
47
|
end
|
|
39
48
|
|
|
49
|
+
# This will truncate all the tables and reset the tracker back to 0,
|
|
50
|
+
# done as a transaction.
|
|
40
51
|
def truncate
|
|
41
52
|
self.class.tables.each do |table_name, _|
|
|
42
53
|
@db_connection.transaction do
|
|
@@ -1,14 +1,21 @@
|
|
|
1
1
|
module EventSourcery
|
|
2
2
|
module Postgres
|
|
3
|
+
# This will set up a persisted event id tracker for processors.
|
|
3
4
|
class Tracker
|
|
4
|
-
|
|
5
|
+
|
|
6
|
+
def initialize(db_connection = EventSourcery::Postgres.config.projections_database,
|
|
5
7
|
table_name: EventSourcery::Postgres.config.tracker_table_name,
|
|
6
8
|
obtain_processor_lock: true)
|
|
7
|
-
@
|
|
9
|
+
@db_connection = db_connection
|
|
8
10
|
@table_name = table_name.to_sym
|
|
9
11
|
@obtain_processor_lock = obtain_processor_lock
|
|
10
12
|
end
|
|
11
13
|
|
|
14
|
+
# Set up the given processor.
|
|
15
|
+
# This will create the projector tracker table if it does not exist.
|
|
16
|
+
# If given a processor_name it will then attempt to get a lock on the db.
|
|
17
|
+
#
|
|
18
|
+
# @param processor_name the name of the processor
|
|
12
19
|
def setup(processor_name = nil)
|
|
13
20
|
create_table_if_not_exists if EventSourcery::Postgres.config.auto_create_projector_tracker
|
|
14
21
|
|
|
@@ -24,6 +31,11 @@ module EventSourcery
|
|
|
24
31
|
end
|
|
25
32
|
end
|
|
26
33
|
|
|
34
|
+
# This will updated the tracker table to the given event id value
|
|
35
|
+
# for the given processor name.
|
|
36
|
+
#
|
|
37
|
+
# @param processor_name the name of the processor to update
|
|
38
|
+
# @param event_id the event id number to update to
|
|
27
39
|
def processed_event(processor_name, event_id)
|
|
28
40
|
table.
|
|
29
41
|
where(name: processor_name.to_s).
|
|
@@ -31,22 +43,38 @@ module EventSourcery
|
|
|
31
43
|
true
|
|
32
44
|
end
|
|
33
45
|
|
|
46
|
+
# This allows you to process an event and update the tracker table in
|
|
47
|
+
# a single transaction. Will yield the given block first then update the
|
|
48
|
+
# the tracker table to the give event id for the given processor name.
|
|
49
|
+
#
|
|
50
|
+
# @param processor_name the name of the processor to update
|
|
51
|
+
# @param event_id the event id number to update to
|
|
34
52
|
def processing_event(processor_name, event_id)
|
|
35
|
-
@
|
|
53
|
+
@db_connection.transaction do
|
|
36
54
|
yield
|
|
37
55
|
processed_event(processor_name, event_id)
|
|
38
56
|
end
|
|
39
57
|
end
|
|
40
58
|
|
|
59
|
+
# This will reset the tracker to the start (0) for the given processor name.
|
|
60
|
+
#
|
|
61
|
+
# @param processor_name the name of the processor to reset to 0
|
|
41
62
|
def reset_last_processed_event_id(processor_name)
|
|
42
63
|
table.where(name: processor_name.to_s).update(last_processed_event_id: 0)
|
|
43
64
|
end
|
|
44
65
|
|
|
66
|
+
# This will return the last processed event id for the given processor name.
|
|
67
|
+
#
|
|
68
|
+
# @param processor_name the name of the processor you want to look up
|
|
69
|
+
# @return [Int, nil] the value of the last event_id processed
|
|
45
70
|
def last_processed_event_id(processor_name)
|
|
46
71
|
track_entry = table.where(name: processor_name.to_s).first
|
|
47
72
|
track_entry[:last_processed_event_id] if track_entry
|
|
48
73
|
end
|
|
49
74
|
|
|
75
|
+
# Will return an array of all known tracked processors.
|
|
76
|
+
#
|
|
77
|
+
# @return [Array] array of all known tracked processors
|
|
50
78
|
def tracked_processors
|
|
51
79
|
table.select_map(:name)
|
|
52
80
|
end
|
|
@@ -54,7 +82,7 @@ module EventSourcery
|
|
|
54
82
|
private
|
|
55
83
|
|
|
56
84
|
def obtain_global_lock_on_processor(processor_name)
|
|
57
|
-
lock_obtained = @
|
|
85
|
+
lock_obtained = @db_connection.fetch("select pg_try_advisory_lock(#{@track_entry_id})").to_a.first[:pg_try_advisory_lock]
|
|
58
86
|
if lock_obtained == false
|
|
59
87
|
raise UnableToLockProcessorError, "Unable to get a lock on #{processor_name} #{@track_entry_id}"
|
|
60
88
|
end
|
|
@@ -63,7 +91,7 @@ module EventSourcery
|
|
|
63
91
|
def create_table_if_not_exists
|
|
64
92
|
unless tracker_table_exists?
|
|
65
93
|
EventSourcery.logger.info { "Projector tracker missing - attempting to create 'projector_tracker' table" }
|
|
66
|
-
EventSourcery::Postgres::Schema.create_projector_tracker(db: @
|
|
94
|
+
EventSourcery::Postgres::Schema.create_projector_tracker(db: @db_connection, table_name: @table_name)
|
|
67
95
|
end
|
|
68
96
|
end
|
|
69
97
|
|
|
@@ -77,11 +105,11 @@ module EventSourcery
|
|
|
77
105
|
end
|
|
78
106
|
|
|
79
107
|
def table
|
|
80
|
-
@
|
|
108
|
+
@db_connection[@table_name]
|
|
81
109
|
end
|
|
82
110
|
|
|
83
111
|
def tracker_table_exists?
|
|
84
|
-
@
|
|
112
|
+
@db_connection.table_exists?(@table_name)
|
|
85
113
|
end
|
|
86
114
|
end
|
|
87
115
|
end
|
metadata
CHANGED
|
@@ -1,27 +1,27 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: event_sourcery-postgres
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.8.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Envato
|
|
8
|
-
autorequire:
|
|
8
|
+
autorequire:
|
|
9
9
|
bindir: exe
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date:
|
|
11
|
+
date: 2020-10-01 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: sequel
|
|
15
15
|
requirement: !ruby/object:Gem::Requirement
|
|
16
16
|
requirements:
|
|
17
|
-
- - "
|
|
17
|
+
- - ">="
|
|
18
18
|
- !ruby/object:Gem::Version
|
|
19
19
|
version: '4.38'
|
|
20
20
|
type: :runtime
|
|
21
21
|
prerelease: false
|
|
22
22
|
version_requirements: !ruby/object:Gem::Requirement
|
|
23
23
|
requirements:
|
|
24
|
-
- - "
|
|
24
|
+
- - ">="
|
|
25
25
|
- !ruby/object:Gem::Version
|
|
26
26
|
version: '4.38'
|
|
27
27
|
- !ruby/object:Gem::Dependency
|
|
@@ -56,30 +56,30 @@ dependencies:
|
|
|
56
56
|
name: bundler
|
|
57
57
|
requirement: !ruby/object:Gem::Requirement
|
|
58
58
|
requirements:
|
|
59
|
-
- - "
|
|
59
|
+
- - ">="
|
|
60
60
|
- !ruby/object:Gem::Version
|
|
61
|
-
version: '
|
|
61
|
+
version: '0'
|
|
62
62
|
type: :development
|
|
63
63
|
prerelease: false
|
|
64
64
|
version_requirements: !ruby/object:Gem::Requirement
|
|
65
65
|
requirements:
|
|
66
|
-
- - "
|
|
66
|
+
- - ">="
|
|
67
67
|
- !ruby/object:Gem::Version
|
|
68
|
-
version: '
|
|
68
|
+
version: '0'
|
|
69
69
|
- !ruby/object:Gem::Dependency
|
|
70
70
|
name: rake
|
|
71
71
|
requirement: !ruby/object:Gem::Requirement
|
|
72
72
|
requirements:
|
|
73
73
|
- - "~>"
|
|
74
74
|
- !ruby/object:Gem::Version
|
|
75
|
-
version: '
|
|
75
|
+
version: '13'
|
|
76
76
|
type: :development
|
|
77
77
|
prerelease: false
|
|
78
78
|
version_requirements: !ruby/object:Gem::Requirement
|
|
79
79
|
requirements:
|
|
80
80
|
- - "~>"
|
|
81
81
|
- !ruby/object:Gem::Version
|
|
82
|
-
version: '
|
|
82
|
+
version: '13'
|
|
83
83
|
- !ruby/object:Gem::Dependency
|
|
84
84
|
name: rspec
|
|
85
85
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -122,7 +122,7 @@ dependencies:
|
|
|
122
122
|
- - ">="
|
|
123
123
|
- !ruby/object:Gem::Version
|
|
124
124
|
version: '0'
|
|
125
|
-
description:
|
|
125
|
+
description:
|
|
126
126
|
email:
|
|
127
127
|
- rubygems@envato.com
|
|
128
128
|
executables: []
|
|
@@ -147,8 +147,11 @@ files:
|
|
|
147
147
|
- lib/event_sourcery/postgres/version.rb
|
|
148
148
|
homepage: https://github.com/envato/event_sourcery-postgres
|
|
149
149
|
licenses: []
|
|
150
|
-
metadata:
|
|
151
|
-
|
|
150
|
+
metadata:
|
|
151
|
+
bug_tracker_uri: https://github.com/envato/event_sourcery-postgres/issues
|
|
152
|
+
changelog_uri: https://github.com/envato/event_sourcery-postgres/blob/master/CHANGELOG.md
|
|
153
|
+
source_code_uri: https://github.com/envato/event_sourcery-postgres
|
|
154
|
+
post_install_message:
|
|
152
155
|
rdoc_options: []
|
|
153
156
|
require_paths:
|
|
154
157
|
- lib
|
|
@@ -163,9 +166,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
163
166
|
- !ruby/object:Gem::Version
|
|
164
167
|
version: '0'
|
|
165
168
|
requirements: []
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
signing_key:
|
|
169
|
+
rubygems_version: 3.1.4
|
|
170
|
+
signing_key:
|
|
169
171
|
specification_version: 4
|
|
170
172
|
summary: Postgres event store for use with EventSourcery
|
|
171
173
|
test_files: []
|