event_engine-store 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +32 -0
- data/MIT-LICENSE +20 -0
- data/README.md +416 -0
- data/Rakefile +9 -0
- data/app/assets/stylesheets/event_engine/store/application.css +15 -0
- data/app/models/event_engine/store/application_record.rb +7 -0
- data/app/models/event_engine/store/stored_event.rb +11 -0
- data/app/views/layouts/event_engine/store/application.html.erb +17 -0
- data/db/migrate/20260605000001_create_event_engine_store_stored_events.rb +22 -0
- data/lib/event_engine/store/engine.rb +14 -0
- data/lib/event_engine/store/projection_dispatcher.rb +9 -0
- data/lib/event_engine/store/recorder.rb +21 -0
- data/lib/event_engine/store/replay.rb +25 -0
- data/lib/event_engine/store/version.rb +5 -0
- data/lib/event_engine/store.rb +27 -0
- data/lib/event_engine-store.rb +1 -0
- data/lib/tasks/event_engine/store_tasks.rake +4 -0
- metadata +133 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: '0888fb8cb32a3a92676f641ce96b85d56753a890bdecf9027c9521d3b642ec9c'
|
|
4
|
+
data.tar.gz: b488a04eba91857bcc65ae37bdb4cf0608d94a8f8c4bc6d3e42798810fd7a264
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: fa6fd5beff74cfcea49f0917bbda4f82448592a2c1893d093d5074c97dc41f4cd506c4536398e4b6fe0061e5397414f44e48ca9cf6c586d14fb95e26a5624bb7
|
|
7
|
+
data.tar.gz: 964a48580c13ddb2f1be6d0803e8d0ce26b229fe62d6521c0fdd17f82e228ea3485898a2b9b48e2b23cb505e3009e91feac87873f05530e7e2a7e8e249378c3e
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project are documented here, following
|
|
4
|
+
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and
|
|
5
|
+
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
|
+
|
|
7
|
+
## [Unreleased]
|
|
8
|
+
|
|
9
|
+
## [0.1.0] - 2026-07-29
|
|
10
|
+
|
|
11
|
+
First published release of `event_engine-store`, the durable event record for the
|
|
12
|
+
EventEngine pipeline.
|
|
13
|
+
|
|
14
|
+
### Added
|
|
15
|
+
|
|
16
|
+
- `StoredEvent` — an immutable, append-only table the host owns, with a migration
|
|
17
|
+
installed via the engine.
|
|
18
|
+
- `Recorder` — a handler registered for every process type, so every dispatched event
|
|
19
|
+
is written to the record regardless of how it is otherwise processed.
|
|
20
|
+
- `ProjectionDispatcher` — a handler that feeds recorded events to projections.
|
|
21
|
+
- `Replay` — reconstructs `EventEngine::Event` objects from the stored record for
|
|
22
|
+
event sourcing.
|
|
23
|
+
|
|
24
|
+
### Notes
|
|
25
|
+
|
|
26
|
+
- Requires `event_engine >= 0.2.1`. `0.2.0` and earlier raise `UnroutableEventError`
|
|
27
|
+
on emit once a catalog has been built, which breaks apps like this one that process
|
|
28
|
+
events through handlers rather than a per-event processor.
|
|
29
|
+
- These are **handlers**, not processors. `event_engine` routes each event to at most
|
|
30
|
+
one processor, named in the host's rules file; a recorder needs to observe every
|
|
31
|
+
event, which is what a handler does. An app using only this gem leaves its rules
|
|
32
|
+
file undecided and nothing routes — the handlers still fire.
|
data/MIT-LICENSE
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
Copyright tylercschneider
|
|
2
|
+
|
|
3
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
|
4
|
+
a copy of this software and associated documentation files (the
|
|
5
|
+
"Software"), to deal in the Software without restriction, including
|
|
6
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
|
7
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
|
8
|
+
permit persons to whom the Software is furnished to do so, subject to
|
|
9
|
+
the following conditions:
|
|
10
|
+
|
|
11
|
+
The above copyright notice and this permission notice shall be
|
|
12
|
+
included in all copies or substantial portions of the Software.
|
|
13
|
+
|
|
14
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
|
15
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
|
16
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
|
17
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
|
18
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
|
19
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
|
20
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,416 @@
|
|
|
1
|
+
# EventEngine::Store
|
|
2
|
+
|
|
3
|
+
The permanent, queryable **event record** for
|
|
4
|
+
[EventEngine](https://github.com/DYB-Development/event_engine).
|
|
5
|
+
|
|
6
|
+
Where [`event_engine-delivery`](https://github.com/DYB-Development/event_engine-delivery)'s
|
|
7
|
+
outbox is a *transient delivery buffer* (rows are deleted once published and the
|
|
8
|
+
retention window passes), `event_engine-store` is the **durable source of truth**: an
|
|
9
|
+
immutable, append-only event log the host application owns. It registers a handler
|
|
10
|
+
with EventEngine that records **every dispatched event**, and provides
|
|
11
|
+
**event-sourcing replay** — rebuilding state by reading the log back in append order.
|
|
12
|
+
|
|
13
|
+
It deliberately does **not** aggregate or compute metrics — turning the log into
|
|
14
|
+
rollups is a separate concern.
|
|
15
|
+
|
|
16
|
+
| Gem | Keeps events…? |
|
|
17
|
+
|---|---|
|
|
18
|
+
| `event_engine-delivery` (outbox) | only until delivered + retention expires — a buffer |
|
|
19
|
+
| `event_engine-store` (this gem) | **forever, append-only** — the record |
|
|
20
|
+
|
|
21
|
+
The two are complementary; you can run both at once.
|
|
22
|
+
|
|
23
|
+
---
|
|
24
|
+
|
|
25
|
+
## Table of contents
|
|
26
|
+
|
|
27
|
+
- [Installation](#installation)
|
|
28
|
+
- [How it hooks into the core](#how-it-hooks-into-the-core)
|
|
29
|
+
- [The `StoredEvent` table](#the-storedevent-table)
|
|
30
|
+
- [Querying the log](#querying-the-log)
|
|
31
|
+
- [Customizing the store](#customizing-the-store) ← **start here for extensions**
|
|
32
|
+
- [Expanding the table with your own columns](#expanding-the-table-with-your-own-columns)
|
|
33
|
+
- [Adding query scopes to the model](#adding-query-scopes-to-the-model)
|
|
34
|
+
- [Recording only some events](#recording-only-some-events)
|
|
35
|
+
- [Replacing the recorder entirely](#replacing-the-recorder-entirely)
|
|
36
|
+
- [Disabling the default recorder](#disabling-the-default-recorder)
|
|
37
|
+
- [Making recording resilient / async](#making-recording-resilient--async)
|
|
38
|
+
- [Replay](#replay)
|
|
39
|
+
- [Projections](#projections)
|
|
40
|
+
- [License](#license)
|
|
41
|
+
|
|
42
|
+
---
|
|
43
|
+
|
|
44
|
+
## Installation
|
|
45
|
+
|
|
46
|
+
```ruby
|
|
47
|
+
# Gemfile
|
|
48
|
+
gem "event_engine"
|
|
49
|
+
gem "event_engine-store"
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
bundle install
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
Copy the migration into your app and run it:
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
bin/rails railties:install:migrations # copies the StoredEvent migration
|
|
60
|
+
bin/rails db:migrate
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
There is **no install generator, initializer, or configuration** — the store wires
|
|
64
|
+
itself up at boot. Once migrated, every dispatched event is recorded automatically.
|
|
65
|
+
|
|
66
|
+
---
|
|
67
|
+
|
|
68
|
+
## How it hooks into the core
|
|
69
|
+
|
|
70
|
+
At Rails boot the engine registers **two** handlers with the core, for all levels:
|
|
71
|
+
|
|
72
|
+
```ruby
|
|
73
|
+
# lib/event_engine/store/engine.rb
|
|
74
|
+
initializer "event_engine.store.register_recorder" do
|
|
75
|
+
config.after_initialize do
|
|
76
|
+
EventEngine.register_handler(Recorder.new, levels: :all)
|
|
77
|
+
EventEngine.register_handler(ProjectionDispatcher.new, levels: :all)
|
|
78
|
+
end
|
|
79
|
+
end
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
So on every `EventEngine.<event>` call:
|
|
83
|
+
|
|
84
|
+
1. `Recorder#call(event)` inserts a `StoredEvent` row.
|
|
85
|
+
2. `ProjectionDispatcher#call(event)` calls `apply(event)` on each registered
|
|
86
|
+
projection.
|
|
87
|
+
|
|
88
|
+
> Because both register at `levels: :all`, the store records **every** level —
|
|
89
|
+
> including non-durable levels 1 and 2. That's by design: the store is your record of
|
|
90
|
+
> what happened, independent of how it was delivered. If you only want to record some
|
|
91
|
+
> events, see [Recording only some events](#recording-only-some-events).
|
|
92
|
+
|
|
93
|
+
---
|
|
94
|
+
|
|
95
|
+
## The `StoredEvent` table
|
|
96
|
+
|
|
97
|
+
`EventEngine::Store::StoredEvent` (table `event_engine_store_stored_events`) is
|
|
98
|
+
**append-only**: once a row is persisted it is read-only (`readonly?` returns true on
|
|
99
|
+
persisted records), so an attempt to update raises `ActiveRecord::ReadOnlyRecord`.
|
|
100
|
+
|
|
101
|
+
| Column | Type | Indexed | Notes |
|
|
102
|
+
|---|---|---|---|
|
|
103
|
+
| `event_name` | string | ✓ | NOT NULL |
|
|
104
|
+
| `event_type` | string | | classification |
|
|
105
|
+
| `event_version` | integer | | schema version |
|
|
106
|
+
| `event_level` | integer | | dispatched level |
|
|
107
|
+
| `payload` | json | | event data |
|
|
108
|
+
| `metadata` | json | | context (actor, request id, …) |
|
|
109
|
+
| `occurred_at` | datetime | ✓ | logical event time |
|
|
110
|
+
| `idempotency_key` | string | ✓ | **not unique** — duplicates allowed |
|
|
111
|
+
| `aggregate_type` | string | | aggregate tracking |
|
|
112
|
+
| `aggregate_id` | string | | |
|
|
113
|
+
| `aggregate_version` | integer | | |
|
|
114
|
+
| `created_at` | datetime | | DB insert time (NOT NULL) |
|
|
115
|
+
|
|
116
|
+
The full event envelope is captured. The default migration:
|
|
117
|
+
|
|
118
|
+
```ruby
|
|
119
|
+
create_table :event_engine_store_stored_events do |t|
|
|
120
|
+
t.string :event_name, null: false
|
|
121
|
+
t.string :event_type
|
|
122
|
+
t.integer :event_version
|
|
123
|
+
t.integer :event_level
|
|
124
|
+
t.json :payload
|
|
125
|
+
t.json :metadata
|
|
126
|
+
t.datetime :occurred_at
|
|
127
|
+
t.string :idempotency_key
|
|
128
|
+
t.string :aggregate_type
|
|
129
|
+
t.string :aggregate_id
|
|
130
|
+
t.integer :aggregate_version
|
|
131
|
+
t.datetime :created_at, null: false
|
|
132
|
+
end
|
|
133
|
+
|
|
134
|
+
add_index :event_engine_store_stored_events, :event_name
|
|
135
|
+
add_index :event_engine_store_stored_events, :occurred_at
|
|
136
|
+
add_index :event_engine_store_stored_events, :idempotency_key
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
---
|
|
140
|
+
|
|
141
|
+
## Querying the log
|
|
142
|
+
|
|
143
|
+
The model ships with **no scopes** — query it with plain ActiveRecord:
|
|
144
|
+
|
|
145
|
+
```ruby
|
|
146
|
+
SE = EventEngine::Store::StoredEvent
|
|
147
|
+
|
|
148
|
+
SE.where(event_name: "order_placed")
|
|
149
|
+
SE.where("occurred_at > ?", 7.days.ago)
|
|
150
|
+
SE.where(aggregate_type: "Order", aggregate_id: order.id).order(:id) # one aggregate's history
|
|
151
|
+
SE.order(:id).find_each { |e| … } # full log, batched
|
|
152
|
+
SE.pluck(:event_name).tally # quick histogram
|
|
153
|
+
```
|
|
154
|
+
|
|
155
|
+
For richer queries, add your own scopes — see below.
|
|
156
|
+
|
|
157
|
+
---
|
|
158
|
+
|
|
159
|
+
## Customizing the store
|
|
160
|
+
|
|
161
|
+
This is the part most teams need. The store is deliberately minimal, so the
|
|
162
|
+
customization seams are explicit. The most common ask — **"I want more columns on the
|
|
163
|
+
event table"** — has an important catch, covered first.
|
|
164
|
+
|
|
165
|
+
### Expanding the table with your own columns
|
|
166
|
+
|
|
167
|
+
You'll often want first-class columns instead of digging into the `payload`/`metadata`
|
|
168
|
+
JSON on every query.
|
|
169
|
+
|
|
170
|
+
**Why expand the table:**
|
|
171
|
+
|
|
172
|
+
- **Indexable, fast filtering/joins** — e.g. `actor_id`, `tenant_id`, `correlation_id`
|
|
173
|
+
as real columns you can index and join on, instead of `metadata->>'actor_id'`.
|
|
174
|
+
- **Reporting & BI tools** that don't grok JSON columns well.
|
|
175
|
+
- **Foreign keys / constraints** to enforce integrity against other tables.
|
|
176
|
+
- **Partitioning / retention** by a real column (e.g. `tenant_id`, `created_at`).
|
|
177
|
+
|
|
178
|
+
Adding a column is **two** steps: a migration adds the column, and you extend the
|
|
179
|
+
`Recorder` to populate it (the recorder writes a fixed set of attributes, so a new
|
|
180
|
+
column stays `NULL` until the recorder fills it).
|
|
181
|
+
|
|
182
|
+
**Step 1 — migration** (use a timestamp *after* the gem's `20260605000001`):
|
|
183
|
+
|
|
184
|
+
```ruby
|
|
185
|
+
# db/migrate/20260701000000_extend_stored_events.rb
|
|
186
|
+
class ExtendStoredEvents < ActiveRecord::Migration[8.0]
|
|
187
|
+
def change
|
|
188
|
+
add_column :event_engine_store_stored_events, :actor_id, :integer
|
|
189
|
+
add_column :event_engine_store_stored_events, :tenant_id, :integer
|
|
190
|
+
add_column :event_engine_store_stored_events, :correlation_id, :string
|
|
191
|
+
|
|
192
|
+
add_index :event_engine_store_stored_events, :actor_id
|
|
193
|
+
add_index :event_engine_store_stored_events, [:tenant_id, :created_at]
|
|
194
|
+
end
|
|
195
|
+
end
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
**Step 2 — capture the values.** Wrap the recorder so it pulls your new fields out of
|
|
199
|
+
each event (typically from `metadata`) in addition to the defaults:
|
|
200
|
+
|
|
201
|
+
```ruby
|
|
202
|
+
# config/initializers/event_engine_store.rb
|
|
203
|
+
require "event_engine/store/recorder"
|
|
204
|
+
|
|
205
|
+
module EventEngine
|
|
206
|
+
module Store
|
|
207
|
+
class Recorder
|
|
208
|
+
# Re-open #call to add columns. Keep the default capture and extend it.
|
|
209
|
+
def call(event)
|
|
210
|
+
meta = event.metadata || {}
|
|
211
|
+
StoredEvent.create!(
|
|
212
|
+
event_name: event.event_name,
|
|
213
|
+
event_type: event.event_type,
|
|
214
|
+
event_version: event.event_version,
|
|
215
|
+
event_level: event.event_level,
|
|
216
|
+
payload: event.payload,
|
|
217
|
+
metadata: event.metadata,
|
|
218
|
+
occurred_at: event.occurred_at,
|
|
219
|
+
idempotency_key: event.idempotency_key,
|
|
220
|
+
aggregate_type: event.aggregate_type,
|
|
221
|
+
aggregate_id: event.aggregate_id,
|
|
222
|
+
aggregate_version: event.aggregate_version,
|
|
223
|
+
|
|
224
|
+
# your added columns:
|
|
225
|
+
actor_id: meta[:actor_id] || meta["actor_id"],
|
|
226
|
+
tenant_id: meta[:tenant_id] || meta["tenant_id"],
|
|
227
|
+
correlation_id: meta[:correlation_id] || meta["correlation_id"]
|
|
228
|
+
)
|
|
229
|
+
end
|
|
230
|
+
end
|
|
231
|
+
end
|
|
232
|
+
end
|
|
233
|
+
```
|
|
234
|
+
|
|
235
|
+
Now emit with the context in `metadata` and it lands in real columns:
|
|
236
|
+
|
|
237
|
+
```ruby
|
|
238
|
+
EventEngine.order_placed(order: order, metadata: { actor_id: current_user.id, tenant_id: tenant.id })
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
> **Tip — an upgrade-resilient alternative.** Re-opening `Recorder#call` replaces it,
|
|
242
|
+
> so it won't pick up changes to the default capture in a future gem version. If you'd
|
|
243
|
+
> rather not re-list the defaults, `prepend` a module that lets the gem do its insert
|
|
244
|
+
> and then patches your columns on the returned record:
|
|
245
|
+
>
|
|
246
|
+
> ```ruby
|
|
247
|
+
> module CaptureContext
|
|
248
|
+
> def call(event)
|
|
249
|
+
> record = super
|
|
250
|
+
> meta = event.metadata || {}
|
|
251
|
+
> record.update_columns( # update_columns bypasses the readonly guard
|
|
252
|
+
> actor_id: meta[:actor_id] || meta["actor_id"],
|
|
253
|
+
> tenant_id: meta[:tenant_id] || meta["tenant_id"]
|
|
254
|
+
> )
|
|
255
|
+
> record
|
|
256
|
+
> end
|
|
257
|
+
> end
|
|
258
|
+
> EventEngine::Store::Recorder.prepend(CaptureContext)
|
|
259
|
+
> ```
|
|
260
|
+
>
|
|
261
|
+
> This is resilient to default-capture changes, at the cost of a second write.
|
|
262
|
+
|
|
263
|
+
### Adding query scopes to the model
|
|
264
|
+
|
|
265
|
+
Re-open the model to add scopes/helpers (you can't add columns this way — that needs a
|
|
266
|
+
migration):
|
|
267
|
+
|
|
268
|
+
```ruby
|
|
269
|
+
# config/initializers/event_engine_store.rb (or app/models/…)
|
|
270
|
+
EventEngine::Store::StoredEvent.class_eval do
|
|
271
|
+
scope :named, ->(name) { where(event_name: name) }
|
|
272
|
+
scope :for_aggregate, ->(type, id) { where(aggregate_type: type, aggregate_id: id).order(:id) }
|
|
273
|
+
scope :since, ->(time) { where("occurred_at >= ?", time) }
|
|
274
|
+
end
|
|
275
|
+
|
|
276
|
+
EventEngine::Store::StoredEvent.named("order_placed").since(1.day.ago)
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
### Recording only some events
|
|
280
|
+
|
|
281
|
+
The default records everything. To record a subset, prepend a filter that skips the
|
|
282
|
+
rest:
|
|
283
|
+
|
|
284
|
+
```ruby
|
|
285
|
+
module RecordDomainOnly
|
|
286
|
+
RECORDED = %w[order_placed payment_captured].freeze
|
|
287
|
+
def call(event)
|
|
288
|
+
return unless RECORDED.include?(event.event_name.to_s)
|
|
289
|
+
super
|
|
290
|
+
end
|
|
291
|
+
end
|
|
292
|
+
EventEngine::Store::Recorder.prepend(RecordDomainOnly)
|
|
293
|
+
```
|
|
294
|
+
|
|
295
|
+
**Why:** keep the log focused on domain/audit-worthy events and avoid recording noisy
|
|
296
|
+
level-1 system pings.
|
|
297
|
+
|
|
298
|
+
### Replacing the recorder entirely
|
|
299
|
+
|
|
300
|
+
For a fundamentally different write path (a different table, sharding, an external
|
|
301
|
+
audit service), register your own handler and turn off the default one:
|
|
302
|
+
|
|
303
|
+
```ruby
|
|
304
|
+
class MyAuditRecorder
|
|
305
|
+
def call(event)
|
|
306
|
+
AuditLog.create!(kind: event.event_name, data: event.payload, at: event.occurred_at)
|
|
307
|
+
end
|
|
308
|
+
end
|
|
309
|
+
|
|
310
|
+
Rails.application.config.after_initialize do
|
|
311
|
+
EventEngine.register_handler(MyAuditRecorder.new, levels: :all)
|
|
312
|
+
end
|
|
313
|
+
```
|
|
314
|
+
|
|
315
|
+
(See [Disabling the default recorder](#disabling-the-default-recorder) to stop the
|
|
316
|
+
built-in one from also writing.)
|
|
317
|
+
|
|
318
|
+
### Disabling the default recorder
|
|
319
|
+
|
|
320
|
+
There's no config flag to skip it, so neutralize it in an initializer. Cleanest is to
|
|
321
|
+
make `Recorder#call` a no-op:
|
|
322
|
+
|
|
323
|
+
```ruby
|
|
324
|
+
module DisableDefaultRecorder
|
|
325
|
+
def call(_event); end
|
|
326
|
+
end
|
|
327
|
+
EventEngine::Store::Recorder.prepend(DisableDefaultRecorder)
|
|
328
|
+
```
|
|
329
|
+
|
|
330
|
+
> Avoid `EventEngine.reset_handlers!` for this — it clears **all** handlers, including
|
|
331
|
+
> `event_engine-delivery`'s and the projection dispatcher. Only reach for it if you're
|
|
332
|
+
> fully taking over routing.
|
|
333
|
+
|
|
334
|
+
### Making recording resilient / async
|
|
335
|
+
|
|
336
|
+
The default `Recorder` calls `StoredEvent.create!` **synchronously** inside dispatch.
|
|
337
|
+
A DB hiccup will therefore raise straight back into your emitting code. If recording
|
|
338
|
+
must never break emission, wrap it to rescue (and optionally enqueue a retry):
|
|
339
|
+
|
|
340
|
+
```ruby
|
|
341
|
+
module ResilientRecord
|
|
342
|
+
def call(event)
|
|
343
|
+
super
|
|
344
|
+
rescue => e
|
|
345
|
+
Rails.logger.error("[store] failed to record #{event.event_name}: #{e.class} #{e.message}")
|
|
346
|
+
# optional: RecordEventLaterJob.perform_later(event.to_h)
|
|
347
|
+
nil
|
|
348
|
+
end
|
|
349
|
+
end
|
|
350
|
+
EventEngine::Store::Recorder.prepend(ResilientRecord)
|
|
351
|
+
```
|
|
352
|
+
|
|
353
|
+
**Trade-off:** rescuing protects emission but means a recording failure no longer
|
|
354
|
+
surfaces loudly — make sure you alert on the log line.
|
|
355
|
+
|
|
356
|
+
---
|
|
357
|
+
|
|
358
|
+
## Replay
|
|
359
|
+
|
|
360
|
+
`EventEngine::Store::Replay.each` reconstructs `EventEngine::Event` objects from the
|
|
361
|
+
log in append order (ordered by `id`, batched via `find_each`):
|
|
362
|
+
|
|
363
|
+
```ruby
|
|
364
|
+
EventEngine::Store::Replay.each do |event|
|
|
365
|
+
# event is a fully-rehydrated EventEngine::Event (symbol-keyed payload)
|
|
366
|
+
puts "#{event.occurred_at} #{event.event_name}"
|
|
367
|
+
end
|
|
368
|
+
|
|
369
|
+
# Without a block you get an Enumerator (a *live* query, not a snapshot):
|
|
370
|
+
enum = EventEngine::Store::Replay.each
|
|
371
|
+
enum.count
|
|
372
|
+
```
|
|
373
|
+
|
|
374
|
+
Use replay to rebuild read models, backfill a new projection, or audit history.
|
|
375
|
+
|
|
376
|
+
---
|
|
377
|
+
|
|
378
|
+
## Projections
|
|
379
|
+
|
|
380
|
+
A projection is any object with `apply(event)`. Register it and it's updated **live**
|
|
381
|
+
as events are dispatched, and can be **rebuilt** from the full log on demand.
|
|
382
|
+
|
|
383
|
+
```ruby
|
|
384
|
+
class OrdersByDay
|
|
385
|
+
def initialize; @counts = Hash.new(0); end
|
|
386
|
+
attr_reader :counts
|
|
387
|
+
|
|
388
|
+
def apply(event)
|
|
389
|
+
@counts[event.occurred_at.to_date] += 1 if event.event_name == :order_placed
|
|
390
|
+
end
|
|
391
|
+
end
|
|
392
|
+
|
|
393
|
+
projection = OrdersByDay.new
|
|
394
|
+
EventEngine::Store.register_projection(projection) # live updates from now on
|
|
395
|
+
|
|
396
|
+
EventEngine::Store.rebuild(projection) # replay the whole log into it
|
|
397
|
+
EventEngine::Store.reset_projections! # clear all (e.g. in tests)
|
|
398
|
+
```
|
|
399
|
+
|
|
400
|
+
API: `register_projection(p)`, `projections`, `reset_projections!`, `rebuild(p)`.
|
|
401
|
+
|
|
402
|
+
**Why projections:** maintain a denormalized read model (counters, dashboards,
|
|
403
|
+
search indexes) that you can always recompute from the authoritative log — the core
|
|
404
|
+
event-sourcing payoff.
|
|
405
|
+
|
|
406
|
+
> **Note:** projections run **synchronously inside dispatch**. If a projection's
|
|
407
|
+
> `apply` raises, it propagates and can break event emission (and stops later
|
|
408
|
+
> handlers). Keep `apply` fast and defensive; for heavy work, have `apply` enqueue a
|
|
409
|
+
> job instead of doing the work inline.
|
|
410
|
+
|
|
411
|
+
---
|
|
412
|
+
|
|
413
|
+
## License
|
|
414
|
+
|
|
415
|
+
Available as open source under the terms of the
|
|
416
|
+
[MIT License](https://opensource.org/licenses/MIT).
|
data/Rakefile
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
/*
|
|
2
|
+
* This is a manifest file that'll be compiled into application.css, which will include all the files
|
|
3
|
+
* listed below.
|
|
4
|
+
*
|
|
5
|
+
* Any CSS and SCSS file within this directory, lib/assets/stylesheets, vendor/assets/stylesheets,
|
|
6
|
+
* or any plugin's vendor/assets/stylesheets directory can be referenced here using a relative path.
|
|
7
|
+
*
|
|
8
|
+
* You're free to add application-wide styles to this file and they'll appear at the bottom of the
|
|
9
|
+
* compiled file so the styles you add here take precedence over styles defined in any other CSS/SCSS
|
|
10
|
+
* files in this directory. Styles in this file should be added after the last require_* statement.
|
|
11
|
+
* It is generally better to create a new file per style scope.
|
|
12
|
+
*
|
|
13
|
+
*= require_tree .
|
|
14
|
+
*= require_self
|
|
15
|
+
*/
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
<!DOCTYPE html>
|
|
2
|
+
<html>
|
|
3
|
+
<head>
|
|
4
|
+
<title>Event engine store</title>
|
|
5
|
+
<%= csrf_meta_tags %>
|
|
6
|
+
<%= csp_meta_tag %>
|
|
7
|
+
|
|
8
|
+
<%= yield :head %>
|
|
9
|
+
|
|
10
|
+
<%= stylesheet_link_tag "event_engine/store/application", media: "all" %>
|
|
11
|
+
</head>
|
|
12
|
+
<body>
|
|
13
|
+
|
|
14
|
+
<%= yield %>
|
|
15
|
+
|
|
16
|
+
</body>
|
|
17
|
+
</html>
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
class CreateEventEngineStoreStoredEvents < ActiveRecord::Migration[8.1]
|
|
2
|
+
def change
|
|
3
|
+
create_table :event_engine_store_stored_events do |t|
|
|
4
|
+
t.string :event_name, null: false
|
|
5
|
+
t.string :event_type
|
|
6
|
+
t.integer :event_version
|
|
7
|
+
t.string :process_type
|
|
8
|
+
t.json :payload
|
|
9
|
+
t.json :metadata
|
|
10
|
+
t.datetime :occurred_at
|
|
11
|
+
t.string :idempotency_key
|
|
12
|
+
t.string :aggregate_type
|
|
13
|
+
t.string :aggregate_id
|
|
14
|
+
t.integer :aggregate_version
|
|
15
|
+
t.datetime :created_at, null: false
|
|
16
|
+
end
|
|
17
|
+
|
|
18
|
+
add_index :event_engine_store_stored_events, :event_name
|
|
19
|
+
add_index :event_engine_store_stored_events, :occurred_at
|
|
20
|
+
add_index :event_engine_store_stored_events, :idempotency_key
|
|
21
|
+
end
|
|
22
|
+
end
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
module EventEngine
|
|
2
|
+
module Store
|
|
3
|
+
class Engine < ::Rails::Engine
|
|
4
|
+
isolate_namespace EventEngine::Store
|
|
5
|
+
|
|
6
|
+
initializer "event_engine.store.register_recorder" do
|
|
7
|
+
config.after_initialize do
|
|
8
|
+
EventEngine.register_handler(Recorder.new, process_types: :all)
|
|
9
|
+
EventEngine.register_handler(ProjectionDispatcher.new, process_types: :all)
|
|
10
|
+
end
|
|
11
|
+
end
|
|
12
|
+
end
|
|
13
|
+
end
|
|
14
|
+
end
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
module EventEngine
|
|
2
|
+
module Store
|
|
3
|
+
class Recorder
|
|
4
|
+
def call(event)
|
|
5
|
+
StoredEvent.create!(
|
|
6
|
+
event_name: event.event_name,
|
|
7
|
+
event_type: event.event_type,
|
|
8
|
+
event_version: event.event_version,
|
|
9
|
+
process_type: event.process_type,
|
|
10
|
+
payload: event.payload,
|
|
11
|
+
metadata: event.metadata,
|
|
12
|
+
occurred_at: event.occurred_at,
|
|
13
|
+
idempotency_key: event.idempotency_key,
|
|
14
|
+
aggregate_type: event.aggregate_type,
|
|
15
|
+
aggregate_id: event.aggregate_id,
|
|
16
|
+
aggregate_version: event.aggregate_version
|
|
17
|
+
)
|
|
18
|
+
end
|
|
19
|
+
end
|
|
20
|
+
end
|
|
21
|
+
end
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
module EventEngine
|
|
2
|
+
module Store
|
|
3
|
+
module Replay
|
|
4
|
+
def self.each
|
|
5
|
+
return to_enum(:each) unless block_given?
|
|
6
|
+
|
|
7
|
+
StoredEvent.order(:id).find_each do |stored|
|
|
8
|
+
yield EventEngine::Event.new(
|
|
9
|
+
event_name: stored.event_name,
|
|
10
|
+
event_type: stored.event_type,
|
|
11
|
+
event_version: stored.event_version,
|
|
12
|
+
process_type: stored.process_type,
|
|
13
|
+
payload: stored.payload,
|
|
14
|
+
metadata: stored.metadata,
|
|
15
|
+
occurred_at: stored.occurred_at,
|
|
16
|
+
idempotency_key: stored.idempotency_key,
|
|
17
|
+
aggregate_type: stored.aggregate_type,
|
|
18
|
+
aggregate_id: stored.aggregate_id,
|
|
19
|
+
aggregate_version: stored.aggregate_version
|
|
20
|
+
)
|
|
21
|
+
end
|
|
22
|
+
end
|
|
23
|
+
end
|
|
24
|
+
end
|
|
25
|
+
end
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
require "event_engine/store/version"
|
|
2
|
+
require "event_engine/store/engine"
|
|
3
|
+
require "event_engine/store/recorder"
|
|
4
|
+
require "event_engine/store/replay"
|
|
5
|
+
require "event_engine/store/projection_dispatcher"
|
|
6
|
+
|
|
7
|
+
module EventEngine
|
|
8
|
+
module Store
|
|
9
|
+
class << self
|
|
10
|
+
def projections
|
|
11
|
+
@projections ||= []
|
|
12
|
+
end
|
|
13
|
+
|
|
14
|
+
def register_projection(projection)
|
|
15
|
+
projections << projection
|
|
16
|
+
end
|
|
17
|
+
|
|
18
|
+
def reset_projections!
|
|
19
|
+
projections.clear
|
|
20
|
+
end
|
|
21
|
+
|
|
22
|
+
def rebuild(projection)
|
|
23
|
+
Replay.each { |event| projection.apply(event) }
|
|
24
|
+
end
|
|
25
|
+
end
|
|
26
|
+
end
|
|
27
|
+
end
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
require "event_engine/store"
|
metadata
ADDED
|
@@ -0,0 +1,133 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: event_engine-store
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 0.1.0
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- tylercschneider
|
|
8
|
+
bindir: bin
|
|
9
|
+
cert_chain: []
|
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
|
+
dependencies:
|
|
12
|
+
- !ruby/object:Gem::Dependency
|
|
13
|
+
name: railties
|
|
14
|
+
requirement: !ruby/object:Gem::Requirement
|
|
15
|
+
requirements:
|
|
16
|
+
- - ">="
|
|
17
|
+
- !ruby/object:Gem::Version
|
|
18
|
+
version: 7.1.6
|
|
19
|
+
- - "<"
|
|
20
|
+
- !ruby/object:Gem::Version
|
|
21
|
+
version: '9'
|
|
22
|
+
type: :runtime
|
|
23
|
+
prerelease: false
|
|
24
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
25
|
+
requirements:
|
|
26
|
+
- - ">="
|
|
27
|
+
- !ruby/object:Gem::Version
|
|
28
|
+
version: 7.1.6
|
|
29
|
+
- - "<"
|
|
30
|
+
- !ruby/object:Gem::Version
|
|
31
|
+
version: '9'
|
|
32
|
+
- !ruby/object:Gem::Dependency
|
|
33
|
+
name: activerecord
|
|
34
|
+
requirement: !ruby/object:Gem::Requirement
|
|
35
|
+
requirements:
|
|
36
|
+
- - ">="
|
|
37
|
+
- !ruby/object:Gem::Version
|
|
38
|
+
version: 7.1.6
|
|
39
|
+
- - "<"
|
|
40
|
+
- !ruby/object:Gem::Version
|
|
41
|
+
version: '9'
|
|
42
|
+
type: :runtime
|
|
43
|
+
prerelease: false
|
|
44
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
45
|
+
requirements:
|
|
46
|
+
- - ">="
|
|
47
|
+
- !ruby/object:Gem::Version
|
|
48
|
+
version: 7.1.6
|
|
49
|
+
- - "<"
|
|
50
|
+
- !ruby/object:Gem::Version
|
|
51
|
+
version: '9'
|
|
52
|
+
- !ruby/object:Gem::Dependency
|
|
53
|
+
name: event_engine
|
|
54
|
+
requirement: !ruby/object:Gem::Requirement
|
|
55
|
+
requirements:
|
|
56
|
+
- - ">="
|
|
57
|
+
- !ruby/object:Gem::Version
|
|
58
|
+
version: 0.2.1
|
|
59
|
+
type: :runtime
|
|
60
|
+
prerelease: false
|
|
61
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
62
|
+
requirements:
|
|
63
|
+
- - ">="
|
|
64
|
+
- !ruby/object:Gem::Version
|
|
65
|
+
version: 0.2.1
|
|
66
|
+
- !ruby/object:Gem::Dependency
|
|
67
|
+
name: json
|
|
68
|
+
requirement: !ruby/object:Gem::Requirement
|
|
69
|
+
requirements:
|
|
70
|
+
- - "<"
|
|
71
|
+
- !ruby/object:Gem::Version
|
|
72
|
+
version: '3'
|
|
73
|
+
type: :runtime
|
|
74
|
+
prerelease: false
|
|
75
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
76
|
+
requirements:
|
|
77
|
+
- - "<"
|
|
78
|
+
- !ruby/object:Gem::Version
|
|
79
|
+
version: '3'
|
|
80
|
+
description: 'The durable record layer for EventEngine: an immutable, append-only
|
|
81
|
+
event table the host owns, a handler that records every dispatched event, and event-sourcing
|
|
82
|
+
replay. Depends on event_engine for event definitions.'
|
|
83
|
+
email:
|
|
84
|
+
- tylercschneider@gmail.com
|
|
85
|
+
executables: []
|
|
86
|
+
extensions: []
|
|
87
|
+
extra_rdoc_files: []
|
|
88
|
+
files:
|
|
89
|
+
- CHANGELOG.md
|
|
90
|
+
- MIT-LICENSE
|
|
91
|
+
- README.md
|
|
92
|
+
- Rakefile
|
|
93
|
+
- app/assets/stylesheets/event_engine/store/application.css
|
|
94
|
+
- app/models/event_engine/store/application_record.rb
|
|
95
|
+
- app/models/event_engine/store/stored_event.rb
|
|
96
|
+
- app/views/layouts/event_engine/store/application.html.erb
|
|
97
|
+
- db/migrate/20260605000001_create_event_engine_store_stored_events.rb
|
|
98
|
+
- lib/event_engine-store.rb
|
|
99
|
+
- lib/event_engine/store.rb
|
|
100
|
+
- lib/event_engine/store/engine.rb
|
|
101
|
+
- lib/event_engine/store/projection_dispatcher.rb
|
|
102
|
+
- lib/event_engine/store/recorder.rb
|
|
103
|
+
- lib/event_engine/store/replay.rb
|
|
104
|
+
- lib/event_engine/store/version.rb
|
|
105
|
+
- lib/tasks/event_engine/store_tasks.rake
|
|
106
|
+
homepage: https://github.com/DYB-Development/event_engine-store
|
|
107
|
+
licenses:
|
|
108
|
+
- MIT
|
|
109
|
+
metadata:
|
|
110
|
+
allowed_push_host: https://rubygems.org
|
|
111
|
+
homepage_uri: https://github.com/DYB-Development/event_engine-store
|
|
112
|
+
source_code_uri: https://github.com/DYB-Development/event_engine-store
|
|
113
|
+
changelog_uri: https://github.com/tylercschneider/event_engine-store/blob/main/CHANGELOG.md
|
|
114
|
+
bug_tracker_uri: https://github.com/tylercschneider/event_engine-store/issues
|
|
115
|
+
documentation_uri: https://github.com/tylercschneider/event_engine-store#readme
|
|
116
|
+
rdoc_options: []
|
|
117
|
+
require_paths:
|
|
118
|
+
- lib
|
|
119
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
120
|
+
requirements:
|
|
121
|
+
- - ">="
|
|
122
|
+
- !ruby/object:Gem::Version
|
|
123
|
+
version: 3.2.0
|
|
124
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
125
|
+
requirements:
|
|
126
|
+
- - ">="
|
|
127
|
+
- !ruby/object:Gem::Version
|
|
128
|
+
version: '0'
|
|
129
|
+
requirements: []
|
|
130
|
+
rubygems_version: 4.0.20
|
|
131
|
+
specification_version: 4
|
|
132
|
+
summary: Permanent, queryable event record for EventEngine
|
|
133
|
+
test_files: []
|