event_rail 0.3.0 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1a5dd4da961fafe8b7a42cadb79cc9fcad3f9da337181a6e53f93ac2e2495f80
-  data.tar.gz: adea011ca9a862030527c8178d67e7165e2d9e17b8902328ed96f698910c9835
+  metadata.gz: 1306e3a4d6e6b69ce720544a4368fa1f7201e29c55651ec6a2e5d8a4d35efcd2
+  data.tar.gz: 58597eac171bf85d946e4b063edafe88b71869c0daba79061384ea649d1a27e1
 SHA512:
-  metadata.gz: 382236795f226f5dad4979cb2fbb29cac28fd31a4bd0ca7bc77bd8b7b0f7f7279275a4e97f3df1550d3f7dd5fd826e2ed96a682ccdd8b792d0c6d6bd7af73e3a
-  data.tar.gz: b84bba4b3e33f2fd809c7f833cd8913655f555f298828f7080bc8c27e8925cbd02125e3503d2658444d27f16061bc555749235229935dfa1e9cf4342e2be470b
+  metadata.gz: fd0e456db39d2fce3e1c14fd5e302b011e5b67f4c9d04ecbed62ff19e545c0ab7f2964bce49945bed15f578a82d03b51b6cf307b30cbe50226c74e6b3424c391
+  data.tar.gz: 8f460442f9509d71b6132236252399a109456e22e0d72cda816ce618948215bc5a90de844067b31cb0cb7efbaf512fb75aae9acae1172ab4a3860668b6c2ef03

data/CHANGELOG.md

@@ -13,6 +13,94 @@ here unless a release changes how they behave.
 
 ## [Unreleased]
 
+## [0.5.0] - 2026-09-29
+
+### Added
+
+- **`EventRail.stage(event, identity:, source:)` records an event with the transaction that
+  changes the state it reports.** It stamps the event exactly as `publish` does, builds one
+  job per subscriber, and hands them all to your stager in one call instead of enqueuing
+  them. The stager persists them with the caller's transaction, all or nothing, and hands
+  them to the queue after the commit, so the intent to deliver commits or rolls back with the
+  data. This closes the gap at a boundary, such as a controller action, that has no job retry
+  to republish after a crash. EventRail ships no table or relay; the README gives two recipes,
+  an outbox table and a queue that shares the application's database.
+- **`config.event_rail.stager`**: the name of a constant that responds to `stage(jobs)`,
+  resolved on every staging so a reloaded class is always the current one, or an object used
+  as given. Preparation fails for a name that resolves to nothing or a stager without `stage`.
+  An application that never stages configures nothing.
+- **`EventRail::StagedPublication`**, what `stage` returns: the stamped event, the staged
+  jobs, and their subscriber classes. It has no accepted or skipped outcomes, because a
+  subscriber's enqueue callbacks run when its job is enqueued, not when it is staged.
+- **`stage.event_rail`**, a fifth notification, carrying `subscriber_count` beside the usual
+  keys.
+
+### Changed
+
+- `EventRail::TransactionalPublicationError` now names staging through a configured stager
+  as the alternative to publishing after the commit. `publish` inside a transaction still
+  raises.
+- Staging shares publication's duplicate check within one job attempt: staging or publishing
+  the same fact twice raises `EventRail::DuplicatePublicationError`.
+
+## [0.4.0] - 2026-09-26
+
+### Changed
+
+- **Breaking: an event with declared identity derives its ID from the fact, not from the
+  job that publishes it.** With `identity_by` or `identity:`, the ID now comes from the
+  event's source, event type and identity only: not the job class, the job's ID, the
+  execution or the schema version. The same fact has one ID wherever and however often it
+  is published, in a controller or in any job, and in every version of its event type.
+  Previously a caller that ran twice enqueued a new job with a new ID and published the same
+  fact under a new event ID.
+- **Breaking: `publish(event, key:)` is now `publish(event, identity:)`.** A key used to be
+  scoped to one job; an identity names a fact across every job. `key:` raises with
+  directions, and `identity:` is refused for a class that declares `identity_by`. An explicit
+  identity derives the same ID as a single string `identity_by` attribute with the same value.
+- **Breaking: every derived ID changes.** The derivation has a new namespace and separate,
+  documented rules for declared facts and for executions, pinned by vectors published in the
+  README so other implementations can reproduce them. Strings are hashed as UTF-8 and invalid
+  UTF-8 is refused.
+- **Breaking: random IDs are UUIDv7**, time-ordered, for events without declared identity
+  published outside a job. Derived IDs remain UUIDv5.
+- An event without declared identity published by a subscriber derives its ID from the
+  handled event's source as well as its ID, so causes from two producers that share an ID are
+  scoped apart.
+- The duplicate check within one execution includes the source, so one job can publish the
+  same fact for two producers, and relay two events that share an ID from different sources.
+- A relayed event keeps its lineage as it arrived: an absent causation is no longer filled
+  from the relaying context. A relay retried with a different payload, occurrence time or
+  extensions is refused, as a local retry is.
+- One publication record per `(source, type, version, id)` within an execution: publishing
+  the stamped event of a failed local publication, for instance to recover from an
+  `EnqueueError`, is its retry, and publishing it after success is a duplicate. A relayed ID
+  is compared as opaque bytes, so an ID that is not valid UTF-8 relays as before.
+- **Breaking: a declared identity attribute cannot be an empty string**, as it cannot be nil:
+  every blank event would otherwise be one fact.
+- `EventRail.with_context` opened inside a running job keeps the job's execution, so an
+  undeclared event published in the block keeps its retry-stable ID and its duplicate check;
+  it used to get a random ID.
+- Versioning: every version of an event type keeps its declared identity values; changing
+  what an event identifies needs a new event type, not a new version.
+
+### Added
+
+- `publish(event, source:)` publishes a locally built event on behalf of another producer,
+  such as an inbound webhook; relaying an event under a different source than it carries is
+  refused.
+
+### Upgrading
+
+- Audit every `identity_by` and every former `key:` before upgrading. A value that was safe
+  within one job, such as a line number or a product ID for a fact that recurs, now names one
+  fact across every job and would merge distinct facts silently. Identity must name the
+  occurrence, not the thing it is about.
+- IDs derived before and after the upgrade differ. A publication retried across the deploy, a
+  delayed job, a backfill or a dead-letter redelivery can deliver a fact again under a new ID;
+  subscribers that also deduplicate on a business key are unaffected. Draining publishing jobs
+  before deploying narrows the window.
+
 ## [0.3.0] - 2026-09-21
 
 ### Changed
@@ -100,7 +188,9 @@ here unless a release changes how they behave.
 - A typed error hierarchy rooted at `EventRail::Error`, distinguishing declaration,
   casting, context, serialization, and publication failures.
 
-[Unreleased]: https://github.com/alexdmtv/event-rail/compare/v0.3.0...HEAD
+[Unreleased]: https://github.com/alexdmtv/event-rail/compare/v0.5.0...HEAD
+[0.5.0]: https://github.com/alexdmtv/event-rail/compare/v0.4.0...v0.5.0
+[0.4.0]: https://github.com/alexdmtv/event-rail/compare/v0.3.0...v0.4.0
 [0.3.0]: https://github.com/alexdmtv/event-rail/compare/v0.2.0...v0.3.0
 [0.2.0]: https://github.com/alexdmtv/event-rail/compare/v0.1.0...v0.2.0
 [0.1.0]: https://github.com/alexdmtv/event-rail/releases/tag/v0.1.0

data/README.md

@@ -27,7 +27,7 @@ EventRail is narrow on purpose: durable fanout across a boundary inside one appl
 
 ### What it gives you
 
-- **Retry-stable identity**, derived from the publishing execution rather than generated, so a redelivered cause produces the same event rather than a second copy of it. Idempotency holds across hops, not only across one subscriber's retries.
+- **Stable identity.** An event that declares what fact it is has one ID wherever and however often it is published, so a consumer recognises it arriving again; any other event keeps its ID across retries and redeliveries. Idempotency holds across hops, not only across one subscriber's retries.
 - **Subscribers that are the jobs**, each keeping its own queue, retry policy, and concurrency limits rather than sharing the single wrapper job other gems route subscribers through.
 - **Boot-time validation**: a wrong `perform` arity, a missing `EventRail::JobContext`, or a late declaration fails the boot that introduced it rather than the first publication in production.
 - **A typed boundary**: a `Date`, a `BigDecimal`, a record, or a GlobalID is refused rather than serialized into something a worker cannot restore.
@@ -36,8 +36,8 @@ EventRail is narrow on purpose: durable fanout across a boundary inside one appl
 
 ### What it does not do
 
-- **No event log and no synchronous handlers.** No history, replay, read-model rebuild, or browser UI, and every subscriber crosses the queue.
-- **No outbox, and so a dual-write gap.** Publication is refused inside a transaction, so a publisher commits and then publishes, and a process that dies between the two loses the event. Publishing from a job makes that recoverable, because the retry republishes under the same identity (see [Replay-safe publishers](#replay-safe-publishers)); from a controller action there is no such guarantee. An application that wants recovery beyond that can record an intent and republish it from a sweeper, but a republication from a different job derives a different event ID, so the intent's own key belongs in the payload for consumers to deduplicate on.
+- **No event log and no synchronous handlers.** No history, replay, read-model rebuild, or browser UI, and every subscriber crosses the queue. A subscriber can append every event to a store of your own, keyed on `(source, id)`, but that store is a downstream log, not event sourcing: publication follows the commit, with no ordering or expected-version guarantees, so the store is a copy of what happened rather than its source of truth.
+- **No outbox of its own.** Publication is refused inside a transaction, so a publisher that publishes commits first, and a process that dies between the two loses the event. Publishing from a job makes that recoverable, because the retry republishes under the same identity (see [Replay-safe publishers](#replay-safe-publishers)). A controller action has no such retry, so at a boundary [stage the event](#staging-inside-a-database-transaction) instead: its subscriber jobs are recorded in the same transaction as the data, through a stager your application supplies, and handed to the queue after the commit. EventRail ships no table and no relay for it; the README gives two recipes.
 - **Discovery is enforced, not advisory.** A subscriber outside a [discovery root](#where-discovery-looks) fails the boot rather than being quietly ignored: a silent delivery bug traded for a loud startup error, at the price of a layout rule.
 
 ## Requirements
@@ -230,7 +230,9 @@ Two exceptions. Code that is not reloadable at all, such as a gem or a file plai
 
 ### At-least-once delivery, and what that means for subscribers
 
-Fanout is individual `perform_later` calls, one per subscriber. Jobs already accepted are never rolled back when a later subscriber's enqueue fails, and the retry repeats complete fanout under the same event ID. A subscriber must therefore be idempotent, and `event.id` is the key to be idempotent on: it is stable across retries of the publishing execution and across redeliveries of a subscriber's own cause.
+Fanout is individual `perform_later` calls, one per subscriber. Jobs already accepted are never rolled back when a later subscriber's enqueue fails, and the retry repeats complete fanout under the same event ID. A subscriber must therefore be idempotent. An event's full identity is `(source, id)`, so a subscriber records `(subscriber, event.source, event.id)` in the same transaction as its effect and does nothing when that record already exists; a business-key constraint (one refund per order) remains worth having beside it. The ID is stable across retries of the publishing execution and redeliveries of a subscriber's own cause, and, for an event with [declared identity](#identity-occurrence-time-and-source), across every publication of the fact.
+
+Two copies of one fact, published by different executions, can differ: a later `occurred_at`, another causation, or a payload read from state that has since changed. EventRail stores nothing and orders nothing, so it cannot say which copy is authoritative; each subscriber decides, and publishers keep copies equal by building events from persisted data.
 
 No ordering is promised, between subscribers or between events.
 
@@ -238,10 +240,167 @@ Retries, backoff, discarding, and dead-letter handling stay where they already a
 
 ### Publishing inside a database transaction
 
-Don't. Publication raises `EventRail::TransactionalPublicationError` when a transaction is open, in every environment, and the fix is to publish after the transaction commits.
+Don't. Publication raises `EventRail::TransactionalPublicationError` when a transaction is open, in every environment. The fix is to publish after the transaction commits, or to [stage the event](#staging-inside-a-database-transaction) within it.
 
 Both queue deferral settings are wrong inside a transaction, in opposite directions. With `enqueue_after_transaction_commit` on, the enqueue is deferred past the point where its failure can be reported, so a publication that silently enqueued nothing looks successful. With it off, the enqueue announces a fact that a rollback then contradicts. The check is on the open transaction itself, so it does not depend on the setting, or on Active Record being present at all.
 
+### Staging inside a database transaction
+
+`EventRail.stage` records an event with the transaction that changes the state it reports. It stamps the event exactly as `publish` does, builds one job per subscriber, and hands them all to your **stager** in one call instead of enqueuing them:
+
+```ruby
+staged = EventRail.stage(
+  Docs::OrderPlaced.new(order_id: "A-1003", total: "10.00", placed_at: Time.now.utc.iso8601)
+)
+
+staged                     # an EventRail::StagedPublication
+staged.event               # the stamped, immutable fact, as publish would stamp it
+staged.staged_jobs         # one unenqueued job per subscriber, as the stager received them
+staged.staged_subscribers  # their classes
+```
+
+The stager persists the jobs with the caller's transaction and hands them to the queue once it commits, so the intent to deliver commits or rolls back with the state change. Configure it once, by name:
+
+```ruby
+# doc:illustrative
+# config/application.rb
+config.event_rail.stager = "StagedJob"   # resolved on every staging, so a reload is followed
+
+# app/models/order.rb
+def ship!
+  transaction do
+    update!(state: "shipped")
+    EventRail.stage(Orders::OrderShipped.new(order_id: id))
+  end
+end
+```
+
+`config.event_rail.stager` takes the name of a constant, as a String or a Symbol, resolved each time an event is staged, or an object used as given. Use the name for anything in your application's reloadable code: a class held by reference fails at boot or goes stale after a reload. An object suits a stager in a gem or under `autoload_once_paths`; `lib/` is reloadable once `config.autoload_lib` covers it. Preparation checks that the name resolves and that the stager responds to `stage`, so a typo fails the boot. `EventRail.stage` with no stager configured raises `EventRail::ConfigurationError`; an application that never stages configures nothing.
+
+Staging shares publication's duplicate check. Staging or publishing the same fact twice in one job attempt raises `EventRail::DuplicatePublicationError`, and a stager that raises leaves the fact free to be staged again in that attempt, under the same ID. One consequence: a job that stages, has its transaction roll back, and stages again in the same attempt fails as a duplicate, and only its own `retry_on` runs it again. Outside a job there is no duplicate check, and an event without declared identity gets a new ID each time, so **an event staged at a boundary, where a request may be repeated, should declare its identity** or be staged with `identity:`.
+
+#### The stager contract
+
+A stager is any object with one method, `stage(jobs)`, which receives an array of unenqueued Active Job instances, possibly empty, and its own to change. EventRail calls it even for an event with no subscribers, so the stager's own checks hold either way. Everything else is the stager's:
+
+- **All or nothing.** It persists the whole list with the caller's open transaction, or raises. A failure part-way through must not leave earlier jobs behind, even if the caller rescues the error and commits. Atomicity with the business state comes from letting the error propagate: rescuing a staging error and committing drops the event, as rescuing any failed write drops that write.
+- **The transaction check.** Only the stager knows which database it writes to, so it decides whether a suitable transaction is open, and it should raise when none is. Staging outside one reopens the gap staging exists to close: an `update!` that autocommits, then a staged row written separately.
+- **Persistence, not delivery.** What commits with the transaction is the intent to deliver. Delivery then depends on the hand-over, the relay, the queue, and each subscriber's own enqueue callbacks, which run when a job is enqueued rather than when it is staged. A callback that declines the job is the subscriber's decision; the stager decides what it means.
+- **Deferral.** A stager that deletes a staged job must do so only after the queue has actually accepted it. Active Job reports a deferred enqueue as successful before it happens, so hand over only once every open transaction has committed (`ActiveRecord.after_all_transactions_commit`), when an enqueue is immediate whatever the setting. Rails 7.2 applications on `load_defaults "7.2"` defer by default. Whether deferral is in effect differs by version, and Active Job reads the setting when a job is enqueued: on 7.2, `:never` is off, `:always` is on, and any other value, `false` included, asks the queue adapter; on 8.0, `:never` and `:default` are off and `:always` is on, with a deprecation warning, and otherwise a truthy value is on; from 8.1 any truthy value is on, the old symbols included.
+- **Hand-over errors.** A hand-over runs after the caller's transaction has committed, so an error there has nobody to go to: rescue it and leave the job for the relay.
+- **Duplicates.** A job handed over twice is delivered twice under the same event ID. A retried staging builds new jobs, with new job IDs, under the same event ID. Subscribers are idempotent on `(source, id)` either way.
+- **What a staged job carries.** It is built at staging, with the staging context's origin time already in place. Its queue and priority are resolved when the stager serializes or enqueues it; options passed through `set`, and anything a custom `perform_later` does, are not applied. It names its job class and holds EventRail's serialized event, so a renamed subscriber or a changed format follows the same rules as a job already in the queue.
+- **Limits.** Staging the same fact again from inside `stage` is not supported. EventRail provides no storage, relay, ordering or retention for staged jobs.
+
+For a database per module, one stager can route: stage into whichever store has an open application transaction, and raise when none or several do. A transaction should never span modules, so several open is a bug, and call sites stay `EventRail.stage(event)` when a module moves to its own database.
+
+#### Recipe: an outbox table
+
+A table in the application's database, written in the caller's transaction and drained after the commit, with a relay for anything the immediate hand-over missed:
+
+```ruby
+# doc:illustrative
+# create_table :staged_jobs do |t|
+#   t.string :job_id, null: false, index: { unique: true }
+#   t.json :payload, null: false
+#   t.datetime :created_at, null: false
+# end
+class StagedJob < ApplicationRecord
+  # One insert_all! statement writes every row or none. Not insert_all, which skips a
+  # conflicting row without saying so.
+  def self.stage(jobs)
+    raise ArgumentError, "#{name}.stage needs an open transaction to commit with" unless current_transaction.open?
+    return if jobs.empty?
+
+    now = Time.current
+    insert_all!(jobs.map { |job| { job_id: job.job_id, payload: job.serialize, created_at: now } })
+
+    # After every open transaction commits, so the enqueue is immediate whatever the
+    # deferral setting, and a row is never deleted for an enqueue that was only deferred.
+    ActiveRecord.after_all_transactions_commit { jobs.each { |job| hand_over(job) } }
+  end
+
+  # Deletes the row once the queue took the job, or once a subscriber's own enqueue callback
+  # declined it; anything else leaves it for the relay.
+  def self.hand_over(job)
+    job.enqueue
+    if job.enqueue_error
+      Rails.logger.warn("Staged job #{job.job_id} left for the relay: #{job.enqueue_error.message}")
+    else
+      Rails.logger.warn("Staged job #{job.job_id} declined by its enqueue callback") unless job.successfully_enqueued?
+      where(job_id: job.job_id).delete_all
+    end
+  rescue => error
+    Rails.logger.warn("Staged job #{job.job_id} left for the relay: #{error.class}: #{error.message}")
+  end
+
+  # Run on a schedule, every second or so. A row that cannot even be loaded -- its job class
+  # renamed while it waited -- is left and logged, and the rows after it are still handed over.
+  def self.relay(older_than: 5.seconds)
+    where(created_at: ...older_than.ago).find_each do |row|
+      hand_over(ActiveJob::Base.deserialize(row.payload))
+    rescue => error
+      Rails.logger.warn("Staged job #{row.job_id} left for the relay: #{error.class}: #{error.message}")
+    end
+  end
+end
+```
+
+#### Recipe: a queue in your database
+
+A queue whose tables share the application's database and connection, such as GoodJob or Solid Queue configured without a separate database, needs no outbox: enqueuing in the transaction is itself the staging. Two conditions EventRail cannot check for you: the queue must use the same connection as the transaction, and deferral must actually be off for every staged job, or the enqueue becomes an after-commit hook that is not durable. Subscribers' enqueue callbacks run inside the transaction here.
+
+```ruby
+# doc:illustrative
+module QueueStager
+  module_function
+
+  def stage(jobs)
+    raise ArgumentError, "QueueStager.stage needs an open transaction" unless ApplicationRecord.current_transaction.open?
+
+    deferred = jobs.map(&:class).uniq.select { |job_class| deferred?(job_class) }
+    raise ArgumentError, "#{deferred.map(&:name).join(", ")} defer enqueuing until after commit" if deferred.any?
+
+    # A savepoint, so a later job's failure takes the earlier ones with it even if the
+    # caller rescues the error and commits.
+    ApplicationRecord.transaction(requires_new: true) do
+      jobs.each do |job|
+        job.enqueue
+        raise job.enqueue_error if job.enqueue_error
+      end
+    end
+  end
+
+  # Whether Active Job would defer this class's enqueue to after the commit. The setting's
+  # meaning moved between versions, and it is interpreted when a job is enqueued, not when it
+  # is set.
+  def deferred?(job_class)
+    setting = job_class.enqueue_after_transaction_commit
+
+    if ActiveJob.version < Gem::Version.new("8.0")
+      # 7.2: the adapter decides unless the class said :never or :always.
+      case setting
+      when :never then false
+      when :always then true
+      else job_class.queue_adapter.enqueue_after_transaction_commit?
+      end
+    elsif ActiveJob.version < Gem::Version.new("8.1")
+      # 8.0: the old symbols still mean what they meant, with a deprecation warning.
+      case setting
+      when :never, :default then false
+      when :always then true
+      else setting ? true : false
+      end
+    else
+      # 8.1 on: any truthy value defers, the old symbols included.
+      setting ? true : false
+    end
+  end
+end
+```
+
+Both recipes are exercised against real transactions in EventRail's own test suite, adapted to its test tables, including a check of the deferral rule against what each supported Rails version actually does.
+
 ### Replay-safe publishers
 
 A publisher that applies a state transition and then publishes should make the transition idempotent and attempt publication unconditionally:
@@ -261,22 +420,113 @@ Guarding publication behind "did I already transition?" is the failure mode to a
 
 ## Identity, occurrence time, and source
 
-Inside an opted-in job, an event's ID is derived from a permanent EventRail namespace and the resolved source, executing job class, execution scope, event type, version, and logical identity. The execution scope is the same value the event records as its causation: a regular job's own ID, or, for a subscriber, the ID of the event it is handling. That is what makes identity survive more than one hop: a follow-up event published while handling a redelivered cause derives the ID it derived the first time.
+An event's ID says which fact it is, so a consumer can recognise the same fact arriving again. It is chosen by one of three rules, in order:
+
+1. **An event that already carries an ID** — one reconstructed from an envelope — keeps it, with its source and its lineage.
+2. **An event with declared identity** — its class's `identity_by` attributes, or an `identity:` passed to `publish` — derives its ID from its source, its event type and that identity, and from nothing else: not the job that published it, not the execution, not the schema version. The same fact has the same ID wherever and however often it is published, inside a job or in a controller, and in version 1 or version 2.
+3. **Any other event** derives its ID from the execution publishing it: stable across that job's retries, and, for an event a subscriber publishes, across redeliveries of the event it handles. A separate job publishing it again gets a new ID. Outside a job it gets a random, time-ordered ID.
 
-Logical identity is chosen in this order: an explicit event ID (an inbound external event), an explicit `key:` passed to `publish`, the class's `identity_by` attributes, or a singleton marker for the first publication of that type in the execution. Provide an explicit key when neither declared identity nor the singleton default can tell two legitimate publications apart:
+**Identity names the occurrence, not the thing it is about.** This is the rule to get right, because getting it wrong fails silently. `PaymentCaptured identity_by :reference` is right when a payment is captured once. `PriceChanged identity_by :product_id` is wrong: every price change of that product would carry one ID, and consumers would drop all but the first, with no error anywhere. A fact that can happen again for the same thing keys on what distinguishes each occurrence — a persisted change ID, a revision, an import row — never on a hash of the payload or a loop index:
 
 ```ruby
-EventRail.publish(
-  Docs::OrderPlaced.new(order_id: "A-1003", total: "5.00", placed_at: Time.now.utc.iso8601),
-  key: "adjustment-7"
-)
+module Docs
+  module Inventory
+    # One base class per producer, so its source is declared once.
+    class Event < EventRail::Event
+      default_source "acme.inventory"
+    end
+
+    # Stock is adjusted many times; each adjustment is one fact.
+    class StockAdjusted < Event
+      event_type "docs.stock_adjusted"
+      version 1
+      identity_by :adjustment_id
+
+      attribute :adjustment_id, :string
+      attribute :sku, :string
+      attribute :quantity, :integer
+    end
+
+    class StockCounted < Event
+      event_type "docs.stock_counted"
+      version 1
+
+      attribute :sku, :string
+      attribute :counted, :integer
+    end
+  end
+end
+
+first = EventRail.publish(Docs::Inventory::StockAdjusted.new(adjustment_id: "adj-7", sku: "MUG", quantity: -2)).event
+again = EventRail.publish(Docs::Inventory::StockAdjusted.new(adjustment_id: "adj-7", sku: "MUG", quantity: -2)).event
+raise "one adjustment, one ID" unless first.id == again.id
+
+# A class without identity_by can name the fact at the call site instead.
+EventRail.publish(Docs::Inventory::StockCounted.new(sku: "MUG", counted: 38), identity: "count-2026-09-01-MUG")
 ```
 
-Changing `identity_by`, changing `default_source`, or renaming a subscriber class all change the identities that derive from them, so each is a breaking change. Outside a job execution, events receive random IDs.
+Beside that rule:
+
+- **`identity:` names a fact across every job**, so it must be unique per source and event type. It is refused for a class that declares `identity_by`, so the two cannot disagree, and it derives the same ID as a single string `identity_by` attribute with the same value, so a class can move from one to the other without changing IDs; an integer `42` and the string `"42"` are different identities.
+- **A tenant belongs in the identity** (or the source) when tenants number their facts independently; an extension is not part of an event's identity.
+- **Never use personal data as identity.** The derivation is a hash, and a hash over an email address or a name is reversible by guessing.
+- **Events that cross an application boundary should declare identity.** An execution-derived ID changes when the publishing code moves, a declared one does not.
+- **Changing `source`, `event_type` or declared identity changes IDs**, so each is a breaking change. Renaming a subscriber class changes the IDs of undeclared events it publishes.
+
+Upgrading from a release where `publish` took `key:`: that key was scoped to one job and is now `identity:`, scoped to every job. Before renaming it, and for every `identity_by`, check that the value names one occurrence per source and event type; a value that used to be safe inside one job, such as a line number, now merges facts.
+
+`occurred_at` is the logical publication time: an explicit timezone-aware value is preserved as the same instant, and otherwise it is the start of the current execution, stable across that execution's retries and never inherited from a cause. Use a persisted domain timestamp when business occurrence time matters, and build an event from persisted data, so that two publications of one fact carry the same payload. Stored times are UTC at microsecond precision.
+
+`source` names the logical producer of an event — `acme.orders` — not an environment, queue, topic, cluster, or deployment. An event ID is unique within its source, so `(source, id)` is an event's full identity, as in CloudEvents, and the source is half of every declared ID. Choose it for the producer rather than the deployment, and a module can move to another application without its events changing IDs. Declare it once, with `default_source` on a module's base event class; an event built on behalf of another producer can be published with `source:` (see [Crossing a network boundary](#crossing-a-network-boundary)). Any bounded non-empty string is accepted, and EventRail does no URI parsing. A follow-up event records its cause's ID as its causation, not its cause's source.
+
+### How an ID is derived
+
+Other implementations can derive the same IDs. An ID is a UUIDv5 ([RFC 9562](https://www.rfc-editor.org/rfc/rfc9562.html)) under the namespace `21fedac0-42c6-443f-b01c-6980aab52f32`, over a name that concatenates encoded components:
 
-`occurred_at` is the logical publication time: an explicit timezone-aware value is preserved as the same instant, and otherwise it is the start of the current execution, stable across that execution's retries and never inherited from a cause. Use a persisted domain timestamp when business occurrence time matters. Stored times are UTC at microsecond precision.
+- **Declared identity:** `"fact"`, source, event type, and the identity as a list — the `identity_by` values in declaration order, or the one `identity:` value.
+- **Execution:** `"execution"`, source, job class name, scope, event type, version, and a singleton marker. The scope is a regular job's Active Job ID, or, for a subscriber, the list `[source, id]` of the event it handles.
 
-`source` identifies a logical producer, not an environment, queue, topic, cluster, or deployment. Any bounded non-empty string is accepted; a stable namespaced value such as `acme.orders` is recommended, and EventRail does no URI parsing.
+Each component is encoded as a tag, its byte length in decimal, a colon, and its bytes:
+
+| Value | Tag | Bytes |
+| --- | --- | --- |
+| String | `s` | its UTF-8 bytes, without Unicode normalization; invalid UTF-8 is refused |
+| Integer | `i` | decimal, with a leading `-` when negative |
+| Decimal | `d` | plain notation: an optional `-`, the integer digits, `.`, and the fractional digits with trailing zeros removed but at least one kept (`12.5`, `100.0`, `0.0`, `-0.0`) |
+| Float | `f` | 17 significant digits with trailing zeros removed, as C's `printf("%.17g")` writes them: positional notation, or exponent notation (`1e+22`, `9.9999999999999995e-08`) when the decimal exponent is below -4 or at least 17, the exponent with a sign and at least two digits; `-0` stays distinct from `0`; non-finite is refused |
+| Boolean | `b` | `true` or `false` |
+| Date | `D` | ISO 8601 (`2026-09-01`) |
+| Timestamp | `T` | converted to UTC, truncated to microseconds, and written as ISO 8601 with six fractional digits (`2026-09-01T10:30:00.123456Z`) |
+| List | `L` | the item count in decimal, a colon, then each item encoded |
+| Singleton marker | `*` | empty |
+
+These vectors are part of the contract, and the test suite asserts them. For declared identity the source is `acme.orders` and the event type `orders.order_placed`; for execution, the job class is `Orders::PlaceOrderJob` and the version `1`:
+
+| Vector | Encoded name | ID |
+| --- | --- | --- |
+| a string | `s4:facts11:acme.orderss19:orders.order_placedL8:1:s3:o-1` | `13b2cda9-e1f7-57b6-ba1d-bb12a7553655` |
+| a non-ASCII string | `s4:facts11:acme.orderss19:orders.order_placedL12:1:s7:Zürich` | `8ad8bbf8-3e10-5162-97da-53cb6752973f` |
+| an integer | `s4:facts11:acme.orderss19:orders.order_placedL7:1:i2:42` | `43230f76-4e80-55d0-bfaf-beab63ac76b9` |
+| a negative integer | `s4:facts11:acme.orderss19:orders.order_placedL7:1:i2:-7` | `4feffb91-0dec-5a7b-a5a0-67968abbe971` |
+| true | `s4:facts11:acme.orderss19:orders.order_placedL9:1:b4:true` | `b1a75e27-6aa2-5f7e-9159-fb429bddc17f` |
+| false | `s4:facts11:acme.orderss19:orders.order_placedL10:1:b5:false` | `631f1712-795b-59fa-b497-103b5f951306` |
+| a decimal | `s4:facts11:acme.orderss19:orders.order_placedL9:1:d4:12.5` | `fc8e4be7-4912-5702-b22b-92d63d710e01` |
+| an integral decimal | `s4:facts11:acme.orderss19:orders.order_placedL10:1:d5:100.0` | `4ab51db7-7864-591d-8e1a-b1f5d9567169` |
+| a zero decimal | `s4:facts11:acme.orderss19:orders.order_placedL8:1:d3:0.0` | `0a066901-77e5-55e3-99c4-a973b5018500` |
+| a negative zero decimal | `s4:facts11:acme.orderss19:orders.order_placedL9:1:d4:-0.0` | `784aef76-53c9-517c-ba3f-732053cd537d` |
+| a float | `s4:facts11:acme.orderss19:orders.order_placedL25:1:f19:0.10000000000000001` | `cc223477-2cb1-5a6d-8d0f-6b25aaf989b7` |
+| an integral float | `s4:facts11:acme.orderss19:orders.order_placedL6:1:f1:1` | `901836d7-5477-5780-8e6b-dd39ed49d272` |
+| a large float | `s4:facts11:acme.orderss19:orders.order_placedL10:1:f5:1e+22` | `d4e88ebb-fdaa-5cc4-9dbe-c576abae6f36` |
+| a small float | `s4:facts11:acme.orderss19:orders.order_placedL28:1:f22:9.9999999999999995e-08` | `83b232ee-36bc-53dc-a455-5a9e31a003a2` |
+| negative zero | `s4:facts11:acme.orderss19:orders.order_placedL7:1:f2:-0` | `7cab07fc-874c-59ff-a946-76d00585d229` |
+| a date | `s4:facts11:acme.orderss19:orders.order_placedL16:1:D10:2026-09-01` | `84aaf8b3-7dc1-5446-880d-a002bfe8cab6` |
+| a timestamp | `s4:facts11:acme.orderss19:orders.order_placedL33:1:T27:2026-09-01T10:30:00.123456Z` | `874e797f-7b0c-5742-b8e2-6b05ae3f738a` |
+| a timestamp with an offset and sub-microsecond digits | `s4:facts11:acme.orderss19:orders.order_placedL33:1:T27:2026-09-01T10:30:01.234567Z` | `7fcd39f5-1a94-5ad2-b899-22ac849d510d` |
+| several values | `s4:facts11:acme.orderss19:orders.order_placedL12:2:s3:o-1i1:2` | `fe61e6d0-cbb6-5db0-96a4-1fd78d86dab9` |
+| a regular job's scope | `s9:executions11:acme.orderss21:Orders::PlaceOrderJobs5:job-1s19:orders.order_placedi1:1*0:` | `e3c5921d-a9bf-5816-b9a7-f36f426cb998` |
+| a subscriber's scope | `s9:executions11:acme.orderss21:Orders::PlaceOrderJobL59:2:s13:acme.paymentss36:0b1e2c3d-4e5f-4a6b-8c7d-9e0f1a2b3c4ds19:orders.order_placedi1:1*0:` | `a3a31754-e302-5c23-a4a7-ea353accfeb6` |
+
+Random IDs, for events without declared identity published outside a job, are UUIDv7, so a log indexes them in time order, to the millisecond, and the version digit tells a random ID (7) from a derived one (5).
 
 ## Logical context
 
@@ -336,7 +586,9 @@ OpenTelemetry is not required. When standard Active Job instrumentation is insta
 
 ## Versioning events
 
-`event_type` and `version` form the durable, language-neutral contract, independent of Ruby class names. A compatible optional addition may keep the same version: an older worker preserves the unknown field as opaque data with no reader and includes it again on export. Removing, renaming, requiring, or retyping a field, changing its meaning, or changing identity requires a new version and a distinct class, and the old class stays registered while messages that reference it can still arrive.
+`event_type` and `version` form the durable, language-neutral contract, independent of Ruby class names. A compatible optional addition may keep the same version: an older worker preserves the unknown field as opaque data with no reader and includes it again on export. Removing, renaming, requiring, or retyping a field, or changing its meaning requires a new version and a distinct class, and the old class stays registered while messages that reference it can still arrive.
+
+A new version is a new representation of the same fact, so every version of a type keeps the same declared identity values, in the same order (an attribute may be renamed), and a fact with declared identity carries one ID in every version; an ID derived from the execution includes the version. A consumer subscribed to both versions while old messages drain deduplicates them on that ID. Keeping both representations is a different need: an archive stores `(source, id, version)`, and it must receive both, because a broker or inbox that deduplicates on `(source, id)` before routing by version passes only one of them on. Changing what an event identifies is not a new version but a new event type.
 
 EventRail provides no upcasting and no schema registry.
 
@@ -344,7 +596,7 @@ EventRail provides no upcasting and no schema registry.
 
 | Value | Limit |
 | --- | ---: |
-| Event, correlation, causation, or boundary identifier | 512 bytes |
+| Event, correlation, causation, or boundary identifier, or `identity:` | 512 bytes |
 | Source | 255 bytes |
 | Event type | 255 bytes |
 | Extension entries | 32 |
@@ -367,7 +619,7 @@ assert_enqueued_with(job: Docs::Billing::ChargeCardJob, args: [ publication.even
 perform_enqueued_jobs
 ```
 
-`assert_enqueued_with(args:)` works because events are value objects. For observability assertions, subscribe to the notifications below. EventRail ships no assertion library, observer, or contract-test helper; Active Job's helpers are the whole assertion surface.
+`assert_enqueued_with(args:)` works because events are value objects. A staged event reaches the same assertions once its transaction commits and your stager hands its jobs over, so configure your real stager in the test environment: Rails' transactional tests hide their own transaction, so a transaction your code opens runs its commit callbacks, and the hand-over, when its block completes. For observability assertions, subscribe to the notifications below. EventRail ships no assertion library, observer, or contract-test helper; Active Job's helpers are the whole assertion surface.
 
 ### Tests that need their own fixtures
 
@@ -427,11 +679,12 @@ Activation replaces a process-wide registry, so it is not safe under `paralleliz
 
 ## Notifications
 
-Four `ActiveSupport::Notifications` events, all in block form so Rails' own exception keys report failures:
+Five `ActiveSupport::Notifications` events, all in block form so Rails' own exception keys report failures:
 
 | Name | Additional payload |
 | --- | --- |
 | `publish.event_rail` | `subscriber_count`, `accepted`, `skipped` |
+| `stage.event_rail` | `subscriber_count` |
 | `enqueue_subscriber.event_rail` | `job_class`, `outcome` (`accepted`, `skipped`, `failed`) |
 | `deserialize.event_rail` | `format_version` |
 | `perform_subscriber.event_rail` | `job_class` |
@@ -458,7 +711,18 @@ event_class = ACCEPTED.fetch([ envelope.event_type, envelope.version ])
 EventRail.publish(envelope.to_event(event_class))
 ```
 
-A relayed event keeps its origin's ID, source, occurrence time, correlation, and extensions; the relaying application's own context extensions are not merged into it, and only a missing causation is filled.
+A relayed event keeps its origin's ID, source, occurrence time, correlation, causation (an absent one stays absent), and extensions; the relaying application's own context extensions are not merged into it. Relaying it with a different `source:` is refused rather than re-attributing somebody else's fact.
+
+A producer that sends no event ID, a supplier's webhook say, is mapped to a local event built on the producer's behalf. Its declared identity and the producer's source make a retried delivery the same fact:
+
+```ruby
+EventRail.publish(
+  Docs::Inventory::StockAdjusted.new(adjustment_id: "delivery-4411", sku: "MUG", quantity: 120),
+  source: "supplier.warehouse"
+)
+```
+
+Source is a claim, not a credential. Before relaying, an inbound adapter checks `envelope.source` against the sources that peer may speak for: a declared fact's ID is predictable, so an unchecked envelope claiming a local source could arrive first under a local fact's ID and make consumers drop the genuine one.
 
 Loop prevention is application policy: an export policy that refuses to export an event whose source is not the local application stops a relayed event from bouncing back to the system it came from. EventRail has no concept of internal versus external, direction, topic, or transport marker.
 

data/lib/event_rail/current.rb

@@ -49,7 +49,15 @@ module EventRail
         inherited[:extensions], extensions, error: InvalidContext
       )
 
-      execution = Internal::Execution.new(scope: resolved_message_id, started_at: resolved_originated_at)
+      # A block opened inside a running job stays part of that job's execution: its
+      # publications keep the job's identity rules and its duplicate record. Only a
+      # block with no job around it is a boundary of its own.
+      running = Internal::Execution.current
+      execution = if running&.derives_identity?
+        running
+      else
+        Internal::Execution.new(scope: resolved_message_id, started_at: resolved_originated_at)
+      end
 
       # A nested scope keeps its parent's causation: the message that caused this flow
       # does not change because the application opened an inner block.

data/lib/event_rail/errors.rb

@@ -14,8 +14,10 @@ module EventRail
   end
 
   # An application's own configuration is wrong: a discovery root that is not an autoload
-  # root of the application or any engine. Distinct from a declaration fault, which is about
-  # a class body, and raised at preparation so it fails the boot that introduced it.
+  # root of the application or any engine, or a stager that cannot be resolved or cannot
+  # stage. Distinct from a declaration fault, which is about a class body. Raised at
+  # preparation where it can be, so it fails the boot that introduced it, and by
+  # `EventRail.stage` when no stager is configured at all.
   class ConfigurationError < Error
   end
 
@@ -171,12 +173,13 @@ module EventRail
   # Publication happened inside an open application database transaction. Both queue
   # deferral settings are wrong there, in opposite directions: a deferred enqueue
   # cannot report its own failure, and an immediate one announces a fact a rollback
-  # then contradicts.
+  # then contradicts. Staging is the way to record an event with the transaction.
   class TransactionalPublicationError < PublicationError
     def initialize(message = nil)
       super(
         message ||
-          "cannot publish inside an open database transaction; publish after the transaction commits"
+          "cannot publish inside an open database transaction; publish after the transaction commits, " \
+          "or stage the event within it through a configured stager (EventRail.stage)"
       )
     end
   end

data/lib/event_rail/event.rb

@@ -232,17 +232,19 @@ module EventRail
         self.class.send(:__event_rail_copy__, self, state: { :@metadata => stamped_metadata })
       end
 
-      # A declared identity attribute that is nil cannot produce a canonical value, so
-      # the event can never be published. The local constructor is the strict door and
+      # A declared identity attribute that is nil cannot produce a canonical value, and
+      # one that is empty would make every blank event the same fact, so neither can be
+      # published. The local constructor is the strict door and
       # the only place whose backtrace points at the code that left the field empty.
       # Trusted reconstruction stays permissive: an external event of this contract may
       # legitimately omit a field, and it arrives with an identity already assigned.
       def validate_local_identity!
-        missing = self.class.identity_by.select { |name| public_send(name).nil? }
+        missing = self.class.identity_by.select { |name| public_send(name).then { |value| value.nil? || value == "" } }
         return if missing.empty?
 
         raise InvalidEvent,
-          "#{self.class} declares #{missing.sort.join(", ")} as logical identity, so it cannot be nil"
+          "#{self.class} declares #{missing.sort.join(", ")} as logical identity, so it cannot be nil or empty: " \
+          "every event with a blank identity would be one fact"
       end
 
       def unknown_attributes_message(unknown)

data/lib/event_rail/internal/execution.rb

@@ -73,6 +73,15 @@ module EventRail
         @publications[key] = Record.new(event: event, succeeded: succeeded)
       end
 
+      # Puts back what `record` returned earlier: nil forgets the key entirely.
+      def restore!(key, record)
+        if record.nil?
+          @publications.delete(key)
+        else
+          @publications[key] = record
+        end
+      end
+
       # What one logical publication looked like the last time this attempt tried it.
       # Retaining the failed case is what lets the same attempt retry complete fanout
       # under the identity it already stamped, instead of manufacturing a second one.