event_rail 0.4.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +32 -1
- data/README.md +162 -4
- data/lib/event_rail/errors.rb +7 -4
- data/lib/event_rail/internal/execution.rb +9 -0
- data/lib/event_rail/internal/notifications.rb +1 -1
- data/lib/event_rail/internal/registry.rb +1 -0
- data/lib/event_rail/internal/stager.rb +99 -0
- data/lib/event_rail/internal/stamping.rb +17 -3
- data/lib/event_rail/publish.rb +10 -6
- data/lib/event_rail/railtie.rb +6 -0
- data/lib/event_rail/stage.rb +79 -0
- data/lib/event_rail/staged_publication.rb +34 -0
- data/lib/event_rail/version.rb +1 -1
- data/lib/event_rail.rb +3 -0
- metadata +4 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 1306e3a4d6e6b69ce720544a4368fa1f7201e29c55651ec6a2e5d8a4d35efcd2
|
|
4
|
+
data.tar.gz: 58597eac171bf85d946e4b063edafe88b71869c0daba79061384ea649d1a27e1
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: fd0e456db39d2fce3e1c14fd5e302b011e5b67f4c9d04ecbed62ff19e545c0ab7f2964bce49945bed15f578a82d03b51b6cf307b30cbe50226c74e6b3424c391
|
|
7
|
+
data.tar.gz: 8f460442f9509d71b6132236252399a109456e22e0d72cda816ce618948215bc5a90de844067b31cb0cb7efbaf512fb75aae9acae1172ab4a3860668b6c2ef03
|
data/CHANGELOG.md
CHANGED
|
@@ -13,6 +13,36 @@ here unless a release changes how they behave.
|
|
|
13
13
|
|
|
14
14
|
## [Unreleased]
|
|
15
15
|
|
|
16
|
+
## [0.5.0] - 2026-09-29
|
|
17
|
+
|
|
18
|
+
### Added
|
|
19
|
+
|
|
20
|
+
- **`EventRail.stage(event, identity:, source:)` records an event with the transaction that
|
|
21
|
+
changes the state it reports.** It stamps the event exactly as `publish` does, builds one
|
|
22
|
+
job per subscriber, and hands them all to your stager in one call instead of enqueuing
|
|
23
|
+
them. The stager persists them with the caller's transaction, all or nothing, and hands
|
|
24
|
+
them to the queue after the commit, so the intent to deliver commits or rolls back with the
|
|
25
|
+
data. This closes the gap at a boundary, such as a controller action, that has no job retry
|
|
26
|
+
to republish after a crash. EventRail ships no table or relay; the README gives two recipes,
|
|
27
|
+
an outbox table and a queue that shares the application's database.
|
|
28
|
+
- **`config.event_rail.stager`**: the name of a constant that responds to `stage(jobs)`,
|
|
29
|
+
resolved on every staging so a reloaded class is always the current one, or an object used
|
|
30
|
+
as given. Preparation fails for a name that resolves to nothing or a stager without `stage`.
|
|
31
|
+
An application that never stages configures nothing.
|
|
32
|
+
- **`EventRail::StagedPublication`**, what `stage` returns: the stamped event, the staged
|
|
33
|
+
jobs, and their subscriber classes. It has no accepted or skipped outcomes, because a
|
|
34
|
+
subscriber's enqueue callbacks run when its job is enqueued, not when it is staged.
|
|
35
|
+
- **`stage.event_rail`**, a fifth notification, carrying `subscriber_count` beside the usual
|
|
36
|
+
keys.
|
|
37
|
+
|
|
38
|
+
### Changed
|
|
39
|
+
|
|
40
|
+
- `EventRail::TransactionalPublicationError` now names staging through a configured stager
|
|
41
|
+
as the alternative to publishing after the commit. `publish` inside a transaction still
|
|
42
|
+
raises.
|
|
43
|
+
- Staging shares publication's duplicate check within one job attempt: staging or publishing
|
|
44
|
+
the same fact twice raises `EventRail::DuplicatePublicationError`.
|
|
45
|
+
|
|
16
46
|
## [0.4.0] - 2026-09-26
|
|
17
47
|
|
|
18
48
|
### Changed
|
|
@@ -158,7 +188,8 @@ here unless a release changes how they behave.
|
|
|
158
188
|
- A typed error hierarchy rooted at `EventRail::Error`, distinguishing declaration,
|
|
159
189
|
casting, context, serialization, and publication failures.
|
|
160
190
|
|
|
161
|
-
[Unreleased]: https://github.com/alexdmtv/event-rail/compare/v0.
|
|
191
|
+
[Unreleased]: https://github.com/alexdmtv/event-rail/compare/v0.5.0...HEAD
|
|
192
|
+
[0.5.0]: https://github.com/alexdmtv/event-rail/compare/v0.4.0...v0.5.0
|
|
162
193
|
[0.4.0]: https://github.com/alexdmtv/event-rail/compare/v0.3.0...v0.4.0
|
|
163
194
|
[0.3.0]: https://github.com/alexdmtv/event-rail/compare/v0.2.0...v0.3.0
|
|
164
195
|
[0.2.0]: https://github.com/alexdmtv/event-rail/compare/v0.1.0...v0.2.0
|
data/README.md
CHANGED
|
@@ -37,7 +37,7 @@ EventRail is narrow on purpose: durable fanout across a boundary inside one appl
|
|
|
37
37
|
### What it does not do
|
|
38
38
|
|
|
39
39
|
- **No event log and no synchronous handlers.** No history, replay, read-model rebuild, or browser UI, and every subscriber crosses the queue. A subscriber can append every event to a store of your own, keyed on `(source, id)`, but that store is a downstream log, not event sourcing: publication follows the commit, with no ordering or expected-version guarantees, so the store is a copy of what happened rather than its source of truth.
|
|
40
|
-
- **No outbox
|
|
40
|
+
- **No outbox of its own.** Publication is refused inside a transaction, so a publisher that publishes commits first, and a process that dies between the two loses the event. Publishing from a job makes that recoverable, because the retry republishes under the same identity (see [Replay-safe publishers](#replay-safe-publishers)). A controller action has no such retry, so at a boundary [stage the event](#staging-inside-a-database-transaction) instead: its subscriber jobs are recorded in the same transaction as the data, through a stager your application supplies, and handed to the queue after the commit. EventRail ships no table and no relay for it; the README gives two recipes.
|
|
41
41
|
- **Discovery is enforced, not advisory.** A subscriber outside a [discovery root](#where-discovery-looks) fails the boot rather than being quietly ignored: a silent delivery bug traded for a loud startup error, at the price of a layout rule.
|
|
42
42
|
|
|
43
43
|
## Requirements
|
|
@@ -240,10 +240,167 @@ Retries, backoff, discarding, and dead-letter handling stay where they already a
|
|
|
240
240
|
|
|
241
241
|
### Publishing inside a database transaction
|
|
242
242
|
|
|
243
|
-
Don't. Publication raises `EventRail::TransactionalPublicationError` when a transaction is open, in every environment
|
|
243
|
+
Don't. Publication raises `EventRail::TransactionalPublicationError` when a transaction is open, in every environment. The fix is to publish after the transaction commits, or to [stage the event](#staging-inside-a-database-transaction) within it.
|
|
244
244
|
|
|
245
245
|
Both queue deferral settings are wrong inside a transaction, in opposite directions. With `enqueue_after_transaction_commit` on, the enqueue is deferred past the point where its failure can be reported, so a publication that silently enqueued nothing looks successful. With it off, the enqueue announces a fact that a rollback then contradicts. The check is on the open transaction itself, so it does not depend on the setting, or on Active Record being present at all.
|
|
246
246
|
|
|
247
|
+
### Staging inside a database transaction
|
|
248
|
+
|
|
249
|
+
`EventRail.stage` records an event with the transaction that changes the state it reports. It stamps the event exactly as `publish` does, builds one job per subscriber, and hands them all to your **stager** in one call instead of enqueuing them:
|
|
250
|
+
|
|
251
|
+
```ruby
|
|
252
|
+
staged = EventRail.stage(
|
|
253
|
+
Docs::OrderPlaced.new(order_id: "A-1003", total: "10.00", placed_at: Time.now.utc.iso8601)
|
|
254
|
+
)
|
|
255
|
+
|
|
256
|
+
staged # an EventRail::StagedPublication
|
|
257
|
+
staged.event # the stamped, immutable fact, as publish would stamp it
|
|
258
|
+
staged.staged_jobs # one unenqueued job per subscriber, as the stager received them
|
|
259
|
+
staged.staged_subscribers # their classes
|
|
260
|
+
```
|
|
261
|
+
|
|
262
|
+
The stager persists the jobs with the caller's transaction and hands them to the queue once it commits, so the intent to deliver commits or rolls back with the state change. Configure it once, by name:
|
|
263
|
+
|
|
264
|
+
```ruby
|
|
265
|
+
# doc:illustrative
|
|
266
|
+
# config/application.rb
|
|
267
|
+
config.event_rail.stager = "StagedJob" # resolved on every staging, so a reload is followed
|
|
268
|
+
|
|
269
|
+
# app/models/order.rb
|
|
270
|
+
def ship!
|
|
271
|
+
transaction do
|
|
272
|
+
update!(state: "shipped")
|
|
273
|
+
EventRail.stage(Orders::OrderShipped.new(order_id: id))
|
|
274
|
+
end
|
|
275
|
+
end
|
|
276
|
+
```
|
|
277
|
+
|
|
278
|
+
`config.event_rail.stager` takes the name of a constant, as a String or a Symbol, resolved each time an event is staged, or an object used as given. Use the name for anything in your application's reloadable code: a class held by reference fails at boot or goes stale after a reload. An object suits a stager in a gem or under `autoload_once_paths`; `lib/` is reloadable once `config.autoload_lib` covers it. Preparation checks that the name resolves and that the stager responds to `stage`, so a typo fails the boot. `EventRail.stage` with no stager configured raises `EventRail::ConfigurationError`; an application that never stages configures nothing.
|
|
279
|
+
|
|
280
|
+
Staging shares publication's duplicate check. Staging or publishing the same fact twice in one job attempt raises `EventRail::DuplicatePublicationError`, and a stager that raises leaves the fact free to be staged again in that attempt, under the same ID. One consequence: a job that stages, has its transaction roll back, and stages again in the same attempt fails as a duplicate, and only its own `retry_on` runs it again. Outside a job there is no duplicate check, and an event without declared identity gets a new ID each time, so **an event staged at a boundary, where a request may be repeated, should declare its identity** or be staged with `identity:`.
|
|
281
|
+
|
|
282
|
+
#### The stager contract
|
|
283
|
+
|
|
284
|
+
A stager is any object with one method, `stage(jobs)`, which receives an array of unenqueued Active Job instances, possibly empty, and its own to change. EventRail calls it even for an event with no subscribers, so the stager's own checks hold either way. Everything else is the stager's:
|
|
285
|
+
|
|
286
|
+
- **All or nothing.** It persists the whole list with the caller's open transaction, or raises. A failure part-way through must not leave earlier jobs behind, even if the caller rescues the error and commits. Atomicity with the business state comes from letting the error propagate: rescuing a staging error and committing drops the event, as rescuing any failed write drops that write.
|
|
287
|
+
- **The transaction check.** Only the stager knows which database it writes to, so it decides whether a suitable transaction is open, and it should raise when none is. Staging outside one reopens the gap staging exists to close: an `update!` that autocommits, then a staged row written separately.
|
|
288
|
+
- **Persistence, not delivery.** What commits with the transaction is the intent to deliver. Delivery then depends on the hand-over, the relay, the queue, and each subscriber's own enqueue callbacks, which run when a job is enqueued rather than when it is staged. A callback that declines the job is the subscriber's decision; the stager decides what it means.
|
|
289
|
+
- **Deferral.** A stager that deletes a staged job must do so only after the queue has actually accepted it. Active Job reports a deferred enqueue as successful before it happens, so hand over only once every open transaction has committed (`ActiveRecord.after_all_transactions_commit`), when an enqueue is immediate whatever the setting. Rails 7.2 applications on `load_defaults "7.2"` defer by default. Whether deferral is in effect differs by version, and Active Job reads the setting when a job is enqueued: on 7.2, `:never` is off, `:always` is on, and any other value, `false` included, asks the queue adapter; on 8.0, `:never` and `:default` are off and `:always` is on, with a deprecation warning, and otherwise a truthy value is on; from 8.1 any truthy value is on, the old symbols included.
|
|
290
|
+
- **Hand-over errors.** A hand-over runs after the caller's transaction has committed, so an error there has nobody to go to: rescue it and leave the job for the relay.
|
|
291
|
+
- **Duplicates.** A job handed over twice is delivered twice under the same event ID. A retried staging builds new jobs, with new job IDs, under the same event ID. Subscribers are idempotent on `(source, id)` either way.
|
|
292
|
+
- **What a staged job carries.** It is built at staging, with the staging context's origin time already in place. Its queue and priority are resolved when the stager serializes or enqueues it; options passed through `set`, and anything a custom `perform_later` does, are not applied. It names its job class and holds EventRail's serialized event, so a renamed subscriber or a changed format follows the same rules as a job already in the queue.
|
|
293
|
+
- **Limits.** Staging the same fact again from inside `stage` is not supported. EventRail provides no storage, relay, ordering or retention for staged jobs.
|
|
294
|
+
|
|
295
|
+
For a database per module, one stager can route: stage into whichever store has an open application transaction, and raise when none or several do. A transaction should never span modules, so several open is a bug, and call sites stay `EventRail.stage(event)` when a module moves to its own database.
|
|
296
|
+
|
|
297
|
+
#### Recipe: an outbox table
|
|
298
|
+
|
|
299
|
+
A table in the application's database, written in the caller's transaction and drained after the commit, with a relay for anything the immediate hand-over missed:
|
|
300
|
+
|
|
301
|
+
```ruby
|
|
302
|
+
# doc:illustrative
|
|
303
|
+
# create_table :staged_jobs do |t|
|
|
304
|
+
# t.string :job_id, null: false, index: { unique: true }
|
|
305
|
+
# t.json :payload, null: false
|
|
306
|
+
# t.datetime :created_at, null: false
|
|
307
|
+
# end
|
|
308
|
+
class StagedJob < ApplicationRecord
|
|
309
|
+
# One insert_all! statement writes every row or none. Not insert_all, which skips a
|
|
310
|
+
# conflicting row without saying so.
|
|
311
|
+
def self.stage(jobs)
|
|
312
|
+
raise ArgumentError, "#{name}.stage needs an open transaction to commit with" unless current_transaction.open?
|
|
313
|
+
return if jobs.empty?
|
|
314
|
+
|
|
315
|
+
now = Time.current
|
|
316
|
+
insert_all!(jobs.map { |job| { job_id: job.job_id, payload: job.serialize, created_at: now } })
|
|
317
|
+
|
|
318
|
+
# After every open transaction commits, so the enqueue is immediate whatever the
|
|
319
|
+
# deferral setting, and a row is never deleted for an enqueue that was only deferred.
|
|
320
|
+
ActiveRecord.after_all_transactions_commit { jobs.each { |job| hand_over(job) } }
|
|
321
|
+
end
|
|
322
|
+
|
|
323
|
+
# Deletes the row once the queue took the job, or once a subscriber's own enqueue callback
|
|
324
|
+
# declined it; anything else leaves it for the relay.
|
|
325
|
+
def self.hand_over(job)
|
|
326
|
+
job.enqueue
|
|
327
|
+
if job.enqueue_error
|
|
328
|
+
Rails.logger.warn("Staged job #{job.job_id} left for the relay: #{job.enqueue_error.message}")
|
|
329
|
+
else
|
|
330
|
+
Rails.logger.warn("Staged job #{job.job_id} declined by its enqueue callback") unless job.successfully_enqueued?
|
|
331
|
+
where(job_id: job.job_id).delete_all
|
|
332
|
+
end
|
|
333
|
+
rescue => error
|
|
334
|
+
Rails.logger.warn("Staged job #{job.job_id} left for the relay: #{error.class}: #{error.message}")
|
|
335
|
+
end
|
|
336
|
+
|
|
337
|
+
# Run on a schedule, every second or so. A row that cannot even be loaded -- its job class
|
|
338
|
+
# renamed while it waited -- is left and logged, and the rows after it are still handed over.
|
|
339
|
+
def self.relay(older_than: 5.seconds)
|
|
340
|
+
where(created_at: ...older_than.ago).find_each do |row|
|
|
341
|
+
hand_over(ActiveJob::Base.deserialize(row.payload))
|
|
342
|
+
rescue => error
|
|
343
|
+
Rails.logger.warn("Staged job #{row.job_id} left for the relay: #{error.class}: #{error.message}")
|
|
344
|
+
end
|
|
345
|
+
end
|
|
346
|
+
end
|
|
347
|
+
```
|
|
348
|
+
|
|
349
|
+
#### Recipe: a queue in your database
|
|
350
|
+
|
|
351
|
+
A queue whose tables share the application's database and connection, such as GoodJob or Solid Queue configured without a separate database, needs no outbox: enqueuing in the transaction is itself the staging. Two conditions EventRail cannot check for you: the queue must use the same connection as the transaction, and deferral must actually be off for every staged job, or the enqueue becomes an after-commit hook that is not durable. Subscribers' enqueue callbacks run inside the transaction here.
|
|
352
|
+
|
|
353
|
+
```ruby
|
|
354
|
+
# doc:illustrative
|
|
355
|
+
module QueueStager
|
|
356
|
+
module_function
|
|
357
|
+
|
|
358
|
+
def stage(jobs)
|
|
359
|
+
raise ArgumentError, "QueueStager.stage needs an open transaction" unless ApplicationRecord.current_transaction.open?
|
|
360
|
+
|
|
361
|
+
deferred = jobs.map(&:class).uniq.select { |job_class| deferred?(job_class) }
|
|
362
|
+
raise ArgumentError, "#{deferred.map(&:name).join(", ")} defer enqueuing until after commit" if deferred.any?
|
|
363
|
+
|
|
364
|
+
# A savepoint, so a later job's failure takes the earlier ones with it even if the
|
|
365
|
+
# caller rescues the error and commits.
|
|
366
|
+
ApplicationRecord.transaction(requires_new: true) do
|
|
367
|
+
jobs.each do |job|
|
|
368
|
+
job.enqueue
|
|
369
|
+
raise job.enqueue_error if job.enqueue_error
|
|
370
|
+
end
|
|
371
|
+
end
|
|
372
|
+
end
|
|
373
|
+
|
|
374
|
+
# Whether Active Job would defer this class's enqueue to after the commit. The setting's
|
|
375
|
+
# meaning moved between versions, and it is interpreted when a job is enqueued, not when it
|
|
376
|
+
# is set.
|
|
377
|
+
def deferred?(job_class)
|
|
378
|
+
setting = job_class.enqueue_after_transaction_commit
|
|
379
|
+
|
|
380
|
+
if ActiveJob.version < Gem::Version.new("8.0")
|
|
381
|
+
# 7.2: the adapter decides unless the class said :never or :always.
|
|
382
|
+
case setting
|
|
383
|
+
when :never then false
|
|
384
|
+
when :always then true
|
|
385
|
+
else job_class.queue_adapter.enqueue_after_transaction_commit?
|
|
386
|
+
end
|
|
387
|
+
elsif ActiveJob.version < Gem::Version.new("8.1")
|
|
388
|
+
# 8.0: the old symbols still mean what they meant, with a deprecation warning.
|
|
389
|
+
case setting
|
|
390
|
+
when :never, :default then false
|
|
391
|
+
when :always then true
|
|
392
|
+
else setting ? true : false
|
|
393
|
+
end
|
|
394
|
+
else
|
|
395
|
+
# 8.1 on: any truthy value defers, the old symbols included.
|
|
396
|
+
setting ? true : false
|
|
397
|
+
end
|
|
398
|
+
end
|
|
399
|
+
end
|
|
400
|
+
```
|
|
401
|
+
|
|
402
|
+
Both recipes are exercised against real transactions in EventRail's own test suite, adapted to its test tables, including a check of the deferral rule against what each supported Rails version actually does.
|
|
403
|
+
|
|
247
404
|
### Replay-safe publishers
|
|
248
405
|
|
|
249
406
|
A publisher that applies a state transition and then publishes should make the transition idempotent and attempt publication unconditionally:
|
|
@@ -462,7 +619,7 @@ assert_enqueued_with(job: Docs::Billing::ChargeCardJob, args: [ publication.even
|
|
|
462
619
|
perform_enqueued_jobs
|
|
463
620
|
```
|
|
464
621
|
|
|
465
|
-
`assert_enqueued_with(args:)` works because events are value objects. For observability assertions, subscribe to the notifications below. EventRail ships no assertion library, observer, or contract-test helper; Active Job's helpers are the whole assertion surface.
|
|
622
|
+
`assert_enqueued_with(args:)` works because events are value objects. A staged event reaches the same assertions once its transaction commits and your stager hands its jobs over, so configure your real stager in the test environment: Rails' transactional tests hide their own transaction, so a transaction your code opens runs its commit callbacks, and the hand-over, when its block completes. For observability assertions, subscribe to the notifications below. EventRail ships no assertion library, observer, or contract-test helper; Active Job's helpers are the whole assertion surface.
|
|
466
623
|
|
|
467
624
|
### Tests that need their own fixtures
|
|
468
625
|
|
|
@@ -522,11 +679,12 @@ Activation replaces a process-wide registry, so it is not safe under `paralleliz
|
|
|
522
679
|
|
|
523
680
|
## Notifications
|
|
524
681
|
|
|
525
|
-
|
|
682
|
+
Five `ActiveSupport::Notifications` events, all in block form so Rails' own exception keys report failures:
|
|
526
683
|
|
|
527
684
|
| Name | Additional payload |
|
|
528
685
|
| --- | --- |
|
|
529
686
|
| `publish.event_rail` | `subscriber_count`, `accepted`, `skipped` |
|
|
687
|
+
| `stage.event_rail` | `subscriber_count` |
|
|
530
688
|
| `enqueue_subscriber.event_rail` | `job_class`, `outcome` (`accepted`, `skipped`, `failed`) |
|
|
531
689
|
| `deserialize.event_rail` | `format_version` |
|
|
532
690
|
| `perform_subscriber.event_rail` | `job_class` |
|
data/lib/event_rail/errors.rb
CHANGED
|
@@ -14,8 +14,10 @@ module EventRail
|
|
|
14
14
|
end
|
|
15
15
|
|
|
16
16
|
# An application's own configuration is wrong: a discovery root that is not an autoload
|
|
17
|
-
# root of the application or any engine
|
|
18
|
-
#
|
|
17
|
+
# root of the application or any engine, or a stager that cannot be resolved or cannot
|
|
18
|
+
# stage. Distinct from a declaration fault, which is about a class body. Raised at
|
|
19
|
+
# preparation where it can be, so it fails the boot that introduced it, and by
|
|
20
|
+
# `EventRail.stage` when no stager is configured at all.
|
|
19
21
|
class ConfigurationError < Error
|
|
20
22
|
end
|
|
21
23
|
|
|
@@ -171,12 +173,13 @@ module EventRail
|
|
|
171
173
|
# Publication happened inside an open application database transaction. Both queue
|
|
172
174
|
# deferral settings are wrong there, in opposite directions: a deferred enqueue
|
|
173
175
|
# cannot report its own failure, and an immediate one announces a fact a rollback
|
|
174
|
-
# then contradicts.
|
|
176
|
+
# then contradicts. Staging is the way to record an event with the transaction.
|
|
175
177
|
class TransactionalPublicationError < PublicationError
|
|
176
178
|
def initialize(message = nil)
|
|
177
179
|
super(
|
|
178
180
|
message ||
|
|
179
|
-
"cannot publish inside an open database transaction; publish after the transaction commits"
|
|
181
|
+
"cannot publish inside an open database transaction; publish after the transaction commits, " \
|
|
182
|
+
"or stage the event within it through a configured stager (EventRail.stage)"
|
|
180
183
|
)
|
|
181
184
|
end
|
|
182
185
|
end
|
|
@@ -73,6 +73,15 @@ module EventRail
|
|
|
73
73
|
@publications[key] = Record.new(event: event, succeeded: succeeded)
|
|
74
74
|
end
|
|
75
75
|
|
|
76
|
+
# Puts back what `record` returned earlier: nil forgets the key entirely.
|
|
77
|
+
def restore!(key, record)
|
|
78
|
+
if record.nil?
|
|
79
|
+
@publications.delete(key)
|
|
80
|
+
else
|
|
81
|
+
@publications[key] = record
|
|
82
|
+
end
|
|
83
|
+
end
|
|
84
|
+
|
|
76
85
|
# What one logical publication looked like the last time this attempt tried it.
|
|
77
86
|
# Retaining the failed case is what lets the same attempt retry complete fanout
|
|
78
87
|
# under the identity it already stamped, instead of manufacturing a second one.
|
|
@@ -2,7 +2,7 @@ require "active_support/notifications"
|
|
|
2
2
|
|
|
3
3
|
module EventRail
|
|
4
4
|
module Internal
|
|
5
|
-
# Payloads for the
|
|
5
|
+
# Payloads for the five public notifications.
|
|
6
6
|
#
|
|
7
7
|
# Contract, identity, and lineage only. Domain data and extensions are excluded by
|
|
8
8
|
# construction rather than by filtering, because a notification payload reaches logs
|
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
module EventRail
|
|
2
|
+
module Internal
|
|
3
|
+
# Finds the application's stager: what `EventRail.stage` hands an event's subscriber
|
|
4
|
+
# jobs to, in place of enqueuing them.
|
|
5
|
+
#
|
|
6
|
+
# Configured by name as a rule. A stager is usually an application model, which cannot be
|
|
7
|
+
# loaded while `config/application.rb` or an initializer runs and is replaced on every
|
|
8
|
+
# reload in development, so a class held by reference would either fail the boot or go
|
|
9
|
+
# stale. The name is resolved again on every staging, the way `belongs_to` resolves
|
|
10
|
+
# `class_name:`. An object is used as given, for a stager that is never reloaded.
|
|
11
|
+
module Stager
|
|
12
|
+
SETTING = "config.event_rail.stager".freeze
|
|
13
|
+
# Ruby's own rule for a constant name, Unicode included: an uppercase letter, then word
|
|
14
|
+
# characters.
|
|
15
|
+
CONSTANT_PATH = /\A(?:::)?[[:upper:]][[:word:]]*(?:::[[:upper:]][[:word:]]*)*\z/
|
|
16
|
+
|
|
17
|
+
module_function
|
|
18
|
+
|
|
19
|
+
# The configured value, or a configuration error when there is no Rails application
|
|
20
|
+
# to read it from: staging has nowhere else to find a stager.
|
|
21
|
+
def configured
|
|
22
|
+
unless defined?(::Rails) && ::Rails.respond_to?(:application) && ::Rails.application
|
|
23
|
+
raise ConfigurationError,
|
|
24
|
+
"EventRail.stage reads #{SETTING} from the Rails application, and there is none"
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
::Rails.application.config.event_rail&.stager
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
def resolve(configured)
|
|
31
|
+
if configured.nil?
|
|
32
|
+
raise ConfigurationError,
|
|
33
|
+
"EventRail.stage needs a stager; set #{SETTING} to the name of a class that responds to stage(jobs)"
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
stager = configured.is_a?(String) || configured.is_a?(Symbol) ? constantize(configured.to_s) : configured
|
|
37
|
+
return stager if stager.respond_to?(:stage)
|
|
38
|
+
|
|
39
|
+
raise ConfigurationError, "#{SETTING} is #{configured.inspect}, which does not respond to stage(jobs)"
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
# Called from preparation, which runs after the autoloaders are set up and again on
|
|
43
|
+
# every reload, so a misspelled name or a missing method fails the boot that
|
|
44
|
+
# introduced it rather than the first staging. Nothing to check without a Rails
|
|
45
|
+
# application, or with no stager configured: an application that never stages needs
|
|
46
|
+
# none.
|
|
47
|
+
def validate!
|
|
48
|
+
return unless defined?(::Rails) && ::Rails.respond_to?(:application) && ::Rails.application
|
|
49
|
+
|
|
50
|
+
value = configured
|
|
51
|
+
resolve(value) unless value.nil?
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
# Only a constant the name itself denotes being missing is a configuration error: the
|
|
55
|
+
# name, or one of the namespaces it passes through. That is decided by where the lookup
|
|
56
|
+
# failed -- which constant was missing, and in which namespace -- rather than by comparing
|
|
57
|
+
# strings, so a name that passes through an alias of a namespace is judged by the
|
|
58
|
+
# namespace it reaches. Anything else raised while loading the stager is a bug in its
|
|
59
|
+
# file, left alone so its backtrace points there: a NameError naming no constant, one
|
|
60
|
+
# missing from some other namespace, or Zeitwerk's, which means the file exists and
|
|
61
|
+
# defines the wrong thing.
|
|
62
|
+
def constantize(name)
|
|
63
|
+
unless name.match?(CONSTANT_PATH)
|
|
64
|
+
raise ConfigurationError, "#{SETTING} is #{name.inspect}, which is not a constant name"
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
ActiveSupport::Inflector.constantize(name)
|
|
68
|
+
rescue NameError => error
|
|
69
|
+
raise if defined?(::Zeitwerk::NameError) && error.is_a?(::Zeitwerk::NameError)
|
|
70
|
+
raise unless missing_from(name, error)
|
|
71
|
+
|
|
72
|
+
raise ConfigurationError, "#{SETTING} names #{name.inspect}, which is not a constant"
|
|
73
|
+
end
|
|
74
|
+
|
|
75
|
+
def missing_from(name, error)
|
|
76
|
+
return false if error.name.nil?
|
|
77
|
+
|
|
78
|
+
receiver = begin
|
|
79
|
+
error.receiver
|
|
80
|
+
rescue ArgumentError
|
|
81
|
+
return false # raised by hand, with no namespace it was looked up in
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
segments = name.delete_prefix("::").split("::")
|
|
85
|
+
segments.each_index.any? do |index|
|
|
86
|
+
segments[index] == error.name.to_s && namespace(segments.first(index)).equal?(receiver)
|
|
87
|
+
end
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
def namespace(segments)
|
|
91
|
+
return Object if segments.empty?
|
|
92
|
+
|
|
93
|
+
ActiveSupport::Inflector.constantize(segments.join("::"))
|
|
94
|
+
rescue NameError
|
|
95
|
+
nil
|
|
96
|
+
end
|
|
97
|
+
end
|
|
98
|
+
end
|
|
99
|
+
end
|
|
@@ -8,7 +8,9 @@ module EventRail
|
|
|
8
8
|
module Stamping
|
|
9
9
|
module_function
|
|
10
10
|
|
|
11
|
-
|
|
11
|
+
# `previous` is the record this publication found for its fact, nil for a first attempt, so
|
|
12
|
+
# a caller that fails before delivering anything can put it back (`abandon!`).
|
|
13
|
+
Prepared = Struct.new(:event, :execution, :logical_key, :relayed, :previous, keyword_init: true) do
|
|
12
14
|
def relayed?
|
|
13
15
|
relayed
|
|
14
16
|
end
|
|
@@ -55,7 +57,7 @@ module EventRail
|
|
|
55
57
|
|
|
56
58
|
execution.record!(logical_key, event: stamped, succeeded: false) if execution&.derives_identity?
|
|
57
59
|
|
|
58
|
-
Prepared.new(event: stamped, execution: execution, logical_key: logical_key, relayed: false)
|
|
60
|
+
Prepared.new(event: stamped, execution: execution, logical_key: logical_key, relayed: false, previous: recorded)
|
|
59
61
|
end
|
|
60
62
|
|
|
61
63
|
def succeeded!(prepared)
|
|
@@ -65,6 +67,17 @@ module EventRail
|
|
|
65
67
|
execution.record!(prepared.logical_key, event: prepared.event, succeeded: true)
|
|
66
68
|
end
|
|
67
69
|
|
|
70
|
+
# Undoes `prepare` for a publication that failed before handing anything over, restoring
|
|
71
|
+
# what the execution recorded before it: nothing for a first attempt, the earlier failed
|
|
72
|
+
# attempt otherwise. Without this a failure that delivered nothing would still turn the
|
|
73
|
+
# next publication of the fact into a retry, bound to a payload nobody received.
|
|
74
|
+
def abandon!(prepared)
|
|
75
|
+
execution = prepared.execution
|
|
76
|
+
return unless execution&.derives_identity?
|
|
77
|
+
|
|
78
|
+
execution.restore!(prepared.logical_key, prepared.previous)
|
|
79
|
+
end
|
|
80
|
+
|
|
68
81
|
# A relayed fact belongs to its origin. Its ID, source, occurrence time, lineage
|
|
69
82
|
# -- an absent causation included -- and extensions are preserved, and the relaying
|
|
70
83
|
# application's own context is deliberately not merged in: local baggage is not
|
|
@@ -85,6 +98,7 @@ module EventRail
|
|
|
85
98
|
|
|
86
99
|
execution = Execution.current
|
|
87
100
|
logical_key = record_key(event.source, event, event.id)
|
|
101
|
+
recorded = nil
|
|
88
102
|
|
|
89
103
|
if execution&.derives_identity?
|
|
90
104
|
recorded = execution.record(logical_key)
|
|
@@ -97,7 +111,7 @@ module EventRail
|
|
|
97
111
|
execution.record!(logical_key, event: event, succeeded: false)
|
|
98
112
|
end
|
|
99
113
|
|
|
100
|
-
Prepared.new(event: event, execution: execution, logical_key: logical_key, relayed: true)
|
|
114
|
+
Prepared.new(event: event, execution: execution, logical_key: logical_key, relayed: true, previous: recorded)
|
|
101
115
|
end
|
|
102
116
|
private_class_method :relay
|
|
103
117
|
|
data/lib/event_rail/publish.rb
CHANGED
|
@@ -19,12 +19,7 @@ module EventRail
|
|
|
19
19
|
# subscriber's enqueue fails, and the retry repeats complete fanout under the same
|
|
20
20
|
# event ID, so a subscriber may see the same event more than once.
|
|
21
21
|
def publish(event, identity: nil, source: nil, **unknown)
|
|
22
|
-
|
|
23
|
-
raise ArgumentError,
|
|
24
|
-
"publish's key: is now identity:, and it names a fact across every job rather than within one; " \
|
|
25
|
-
"check the value names one occurrence per source and event type before renaming it"
|
|
26
|
-
end
|
|
27
|
-
raise ArgumentError, "unknown keyword#{"s" if unknown.size > 1}: #{unknown.keys.map(&:inspect).join(", ")}" if unknown.any?
|
|
22
|
+
reject_unknown_keywords!(:publish, unknown)
|
|
28
23
|
|
|
29
24
|
Internal::Transaction.check!
|
|
30
25
|
|
|
@@ -55,6 +50,15 @@ module EventRail
|
|
|
55
50
|
end
|
|
56
51
|
|
|
57
52
|
private
|
|
53
|
+
def reject_unknown_keywords!(verb, unknown)
|
|
54
|
+
if unknown.key?(:key)
|
|
55
|
+
raise ArgumentError,
|
|
56
|
+
"#{verb}'s key: is now identity:, and it names a fact across every job rather than within one; " \
|
|
57
|
+
"check the value names one occurrence per source and event type before renaming it"
|
|
58
|
+
end
|
|
59
|
+
raise ArgumentError, "unknown keyword#{"s" if unknown.size > 1}: #{unknown.keys.map(&:inspect).join(", ")}" if unknown.any?
|
|
60
|
+
end
|
|
61
|
+
|
|
58
62
|
# Three outcomes, not two. Active Job returns false both when an adapter reports
|
|
59
63
|
# failure and when an enqueue callback aborts, so the return value alone cannot
|
|
60
64
|
# tell a fault from a decision. The job instance can, through `enqueue_error`, and
|
data/lib/event_rail/railtie.rb
CHANGED
|
@@ -12,6 +12,12 @@ module EventRail
|
|
|
12
12
|
config.event_rail = ActiveSupport::OrderedOptions.new
|
|
13
13
|
config.event_rail.roots = Internal::Registry::DEFAULT_ROOTS.dup
|
|
14
14
|
|
|
15
|
+
# What `EventRail.stage` hands an event's subscriber jobs to: the name of a constant,
|
|
16
|
+
# resolved on every staging so a reloaded class is always the current one, or an object
|
|
17
|
+
# for a stager that is never reloaded. Either responds to `stage(jobs)`. Assigned
|
|
18
|
+
# explicitly, so the option exists before an application sets it.
|
|
19
|
+
config.event_rail.stager = nil
|
|
20
|
+
|
|
15
21
|
config.to_prepare do
|
|
16
22
|
# Unqualified, so lexical lookup reaches the private Internal namespace that a
|
|
17
23
|
# qualified EventRail::Internal reference would be refused.
|
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
require "active_support/notifications"
|
|
2
|
+
|
|
3
|
+
module EventRail
|
|
4
|
+
class << self
|
|
5
|
+
# Stages an event: stamps it exactly as `publish` does, builds one job per declared
|
|
6
|
+
# subscriber without enqueuing any, and hands them all to the application's stager in one
|
|
7
|
+
# call, inside the caller's database transaction.
|
|
8
|
+
#
|
|
9
|
+
# Order.transaction do
|
|
10
|
+
# order.update!(state: "shipped")
|
|
11
|
+
# EventRail.stage(Orders::OrderShipped.new(order_id: order.id))
|
|
12
|
+
# end
|
|
13
|
+
#
|
|
14
|
+
# The stager (`config.event_rail.stager`) persists the jobs with that transaction, all or
|
|
15
|
+
# nothing, or raises, and hands them to the queue once it commits. So the intent to
|
|
16
|
+
# deliver commits or rolls back with the state change it reports. Delivery itself is the
|
|
17
|
+
# stager's: its hand-over, its relay, the queue, and each subscriber's own enqueue
|
|
18
|
+
# callbacks, which run when a job is enqueued rather than here.
|
|
19
|
+
#
|
|
20
|
+
# Unlike `publish`, staging inside an open transaction is the point, so there is no
|
|
21
|
+
# transaction check. Whether the right transaction is open is the stager's to judge: only
|
|
22
|
+
# it knows which database it writes to.
|
|
23
|
+
def stage(event, identity: nil, source: nil, **unknown)
|
|
24
|
+
reject_unknown_keywords!(:stage, unknown)
|
|
25
|
+
|
|
26
|
+
# Every check that can fail on configuration or readiness runs before stamping, which
|
|
27
|
+
# records the fact in the current execution: a failure here must not leave a record that
|
|
28
|
+
# makes the next staging in the same attempt look like a retry. Readiness before the
|
|
29
|
+
# stager, so staging from an initializer says "not ready" rather than failing to find a
|
|
30
|
+
# class the autoloader cannot load yet.
|
|
31
|
+
configured = Internal::Stager.configured
|
|
32
|
+
snapshot = Internal::Registry.snapshot
|
|
33
|
+
stager = Internal::Stager.resolve(configured)
|
|
34
|
+
|
|
35
|
+
prepared = Internal::Stamping.prepare(event, identity: identity, source: source)
|
|
36
|
+
stamped = prepared.event
|
|
37
|
+
handed_over = false
|
|
38
|
+
|
|
39
|
+
begin
|
|
40
|
+
jobs = snapshot.subscribers_for(stamped.class).map { |job_class| staged_job(job_class, stamped) }.freeze
|
|
41
|
+
|
|
42
|
+
ActiveSupport::Notifications.instrument("stage.event_rail", Internal::Notifications.payload_for(stamped)) do |payload|
|
|
43
|
+
payload[:subscriber_count] = jobs.length
|
|
44
|
+
handed_over = true
|
|
45
|
+
|
|
46
|
+
# Called even with no jobs, so the stager's own transaction check applies whether or
|
|
47
|
+
# not the event has subscribers yet: otherwise adding the first one would make a
|
|
48
|
+
# previously silent call site start raising. A copy, so the stager may sort or trim
|
|
49
|
+
# its argument without touching what the result reports.
|
|
50
|
+
stager.stage(jobs.dup)
|
|
51
|
+
end
|
|
52
|
+
ensure
|
|
53
|
+
# A failure before the stager was called -- a subscriber that cannot be built, a
|
|
54
|
+
# notification handler failing as the block starts -- handed nothing over, so it must
|
|
55
|
+
# not leave the fact recorded: the next staging in this attempt is a first staging, not
|
|
56
|
+
# a retry bound to a payload no stager saw. From the stager on, a failure leaves the
|
|
57
|
+
# fact unfinished, and staging it again is a retry.
|
|
58
|
+
Internal::Stamping.abandon!(prepared) unless handed_over
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
# After the notification block, as for `publish`.
|
|
62
|
+
Internal::Stamping.succeeded!(prepared)
|
|
63
|
+
|
|
64
|
+
StagedPublication.new(event: stamped, staged_jobs: jobs)
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
private
|
|
68
|
+
# The job `perform_later` would have built, without the enqueue. Its context entry is
|
|
69
|
+
# built now, while the staging context is current, so the stager may serialize the job
|
|
70
|
+
# whenever and wherever it likes and still write what enqueuing it here would have
|
|
71
|
+
# written. Through the entry alone, not `serialize`: serializing would resolve and
|
|
72
|
+
# memoize the job's queue and priority blocks inside the caller's context and run its
|
|
73
|
+
# argument serializers for nothing. Every subscriber includes JobContext; preparation
|
|
74
|
+
# refuses one that does not.
|
|
75
|
+
def staged_job(job_class, event)
|
|
76
|
+
job_class.new(event).tap { |job| job.send(:__event_rail_entry__) }
|
|
77
|
+
end
|
|
78
|
+
end
|
|
79
|
+
end
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
module EventRail
|
|
2
|
+
# What `EventRail.stage` returns: the stamped fact, and the subscriber jobs handed to the
|
|
3
|
+
# stager in place of being enqueued.
|
|
4
|
+
#
|
|
5
|
+
# Not a `Publication`. Accepted and skipped are outcomes of an enqueue, and staging performs
|
|
6
|
+
# none: whether each job reaches the queue, and whether a subscriber's own enqueue callback
|
|
7
|
+
# declines it, is decided later, when the stager hands it over. The jobs are here so a
|
|
8
|
+
# caller can match their IDs with what its stager wrote.
|
|
9
|
+
class StagedPublication
|
|
10
|
+
attr_reader :event, :staged_jobs
|
|
11
|
+
|
|
12
|
+
def initialize(event:, staged_jobs:)
|
|
13
|
+
@event = event
|
|
14
|
+
@staged_jobs = staged_jobs.dup.freeze
|
|
15
|
+
freeze
|
|
16
|
+
end
|
|
17
|
+
|
|
18
|
+
def staged_subscribers
|
|
19
|
+
staged_jobs.map(&:class)
|
|
20
|
+
end
|
|
21
|
+
|
|
22
|
+
def subscriber_count
|
|
23
|
+
staged_jobs.length
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
def id
|
|
27
|
+
event.id
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
def inspect
|
|
31
|
+
"#<EventRail::StagedPublication event=#{event.inspect} staged=#{subscriber_count}>"
|
|
32
|
+
end
|
|
33
|
+
end
|
|
34
|
+
end
|
data/lib/event_rail/version.rb
CHANGED
data/lib/event_rail.rb
CHANGED
|
@@ -30,8 +30,11 @@ require "event_rail/contract"
|
|
|
30
30
|
require "event_rail/envelope"
|
|
31
31
|
require "event_rail/internal/event_serializer"
|
|
32
32
|
require "event_rail/internal/transaction"
|
|
33
|
+
require "event_rail/internal/stager"
|
|
33
34
|
require "event_rail/publication"
|
|
35
|
+
require "event_rail/staged_publication"
|
|
34
36
|
require "event_rail/publish"
|
|
37
|
+
require "event_rail/stage"
|
|
35
38
|
require "event_rail/railtie"
|
|
36
39
|
|
|
37
40
|
# The subscription macro is class-level only: it changes nothing about serialization
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: event_rail
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.5.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Alex Dmitriev
|
|
@@ -140,6 +140,7 @@ files:
|
|
|
140
140
|
- lib/event_rail/internal/notifications.rb
|
|
141
141
|
- lib/event_rail/internal/portable_value.rb
|
|
142
142
|
- lib/event_rail/internal/registry.rb
|
|
143
|
+
- lib/event_rail/internal/stager.rb
|
|
143
144
|
- lib/event_rail/internal/stamping.rb
|
|
144
145
|
- lib/event_rail/internal/subscriber_execution.rb
|
|
145
146
|
- lib/event_rail/internal/timestamp.rb
|
|
@@ -152,6 +153,8 @@ files:
|
|
|
152
153
|
- lib/event_rail/publication.rb
|
|
153
154
|
- lib/event_rail/publish.rb
|
|
154
155
|
- lib/event_rail/railtie.rb
|
|
156
|
+
- lib/event_rail/stage.rb
|
|
157
|
+
- lib/event_rail/staged_publication.rb
|
|
155
158
|
- lib/event_rail/subscriptions.rb
|
|
156
159
|
- lib/event_rail/test_helper.rb
|
|
157
160
|
- lib/event_rail/version.rb
|