acta 0.2.0 → 0.4.0.alpha.1

data/docs/event_driven_pub_sub.md

@@ -0,0 +1,258 @@
+# Event-driven pub/sub
+
+The simplest useful shape for an Acta app: one domain event, multiple
+independent subscribers, no event sourcing. The event is the
+publication; reactors are the subscribers; the events table is your
+audit log for free.
+
+## The scenario
+
+A user signs up. As a result, several independent things should
+happen:
+
+- A welcome email is sent.
+- An analytics service is pinged.
+- The signup is recorded in an audit log.
+
+These concerns have different owners, change at different rates, and
+fail in different ways. Coupling them in the controller (or worse, in
+an `after_create_commit` callback on the `User` model) means every
+new concern requires editing the same file, and a flaky third-party
+analytics call can roll back the user creation.
+
+With Acta:
+
+```ruby
+# app/events/user_signed_up.rb
+class UserSignedUp < Acta::Event
+  stream :user, key: :user_id
+
+  attribute :user_id, :string
+  attribute :email, :string
+  attribute :referral_code, :string
+
+  validates :user_id, :email, presence: true
+end
+```
+
+The signup path creates the AR record and emits the event in the
+same transaction:
+
+```ruby
+# app/controllers/registrations_controller.rb
+class RegistrationsController < ApplicationController
+  def create
+    ApplicationRecord.transaction do
+      user = User.create!(user_params)
+
+      Acta.emit(UserSignedUp.new(
+        user_id:       user.id,
+        email:         user.email,
+        referral_code: params[:referral_code]
+      ))
+    end
+
+    redirect_to dashboard_path
+  end
+end
+```
+
+The explicit `transaction` block is the load-bearing detail. `Acta.emit`
+opens its own inner transaction (with `requires_new: true`), which
+becomes a savepoint inside the outer one — so either the user row
+*and* the event row commit together, or neither does. Without the
+outer transaction these would be two independent commits, and a
+process crash or event validation error between them would leave
+you with a user who has no audit trail, no welcome email, and no
+analytics ping.
+
+That's it for the publisher. Each subscriber lives in its own file,
+declares what it cares about, and ignores everything else.
+
+## Subscribers
+
+```ruby
+# app/reactors/welcome_email_reactor.rb
+class WelcomeEmailReactor < Acta::Reactor
+  on UserSignedUp do |event|
+    UserMailer.welcome(event.user_id).deliver_later
+  end
+end
+```
+
+```ruby
+# app/reactors/analytics_reactor.rb
+class AnalyticsReactor < Acta::Reactor
+  on UserSignedUp do |event|
+    AnalyticsClient.track(
+      user_id: event.user_id,
+      event:   "signup",
+      props:   { referral_code: event.referral_code }
+    )
+  end
+end
+```
+
+The audit log subscriber doesn't exist as code — Acta writes every
+emitted event to the `events` table by default. Browse it at `/acta`
+(see the [Acta::Web engine][acta-web]) or query directly via
+`Acta.events`.
+
+[acta-web]: ../README.md#acta-web
+
+## What just happened
+
+Each reactor runs **after** the database commit that wrote the event,
+**asynchronously** by default (via ActiveJob). So:
+
+- Because the controller wraps both writes in
+  `ApplicationRecord.transaction`, the user row and the event row
+  commit together. If either raises, neither is persisted — no
+  welcome email to a user that doesn't exist.
+- Each reactor enqueues its own job. The welcome email and the
+  analytics ping run in parallel, isolated from each other.
+- A failing analytics call doesn't roll back the signup, doesn't
+  block the email, doesn't surface to the user. ActiveJob's retry
+  semantics apply per-reactor.
+- New subscribers are additive. To send a referral credit when a
+  signup uses a code, write a third reactor — no change to the
+  controller, the event, or the existing reactors.
+
+### A subtle caveat about reactor enqueue timing
+
+Reactors are dispatched after Acta's inner savepoint releases but
+*before* the outer transaction commits. Whether that opens a
+"reactor fired but the user write rolled back" window depends on
+your ActiveJob queue adapter:
+
+- **DB-backed queues** (Solid Queue, GoodJob, Que) — the enqueue is
+  a row insert that participates in the outer transaction. A
+  rollback un-enqueues the job. Atomic.
+- **Redis-backed queues** (Sidekiq) — the enqueue hits Redis
+  immediately and survives a rollback. Small window where the email
+  goes out but the user doesn't exist. Rails 7.2+ exposes
+  `enqueue_after_transaction_commit` to opt into deferred enqueue,
+  which closes the window.
+- **Sync reactors** (`sync!`) — run inline during dispatch. Side
+  effects (email sent, third-party API called) happen before the
+  outer commits and can't be undone by a rollback. Reach for
+  `sync!` only when the side effect is itself a DB write inside the
+  same transaction, or when "fired but rolled back" is acceptable.
+
+On the Rails 8.x + Solid Queue default stack, the right behaviour
+falls out without extra configuration.
+
+## Synchronous when you need it
+
+For tests and the rare side effect that must happen inside the same
+request, opt a reactor into sync mode:
+
+```ruby
+class CreateBillingAccountReactor < Acta::Reactor
+  sync!
+
+  on UserSignedUp do |event|
+    BillingAccount.create!(user_id: event.user_id, plan: "free")
+  end
+end
+```
+
+Sync reactors run **after-commit but in the caller's thread**. They
+still don't block the DB transaction (so they can't roll the signup
+back), but they do block the response. Reach for this when the
+follow-up state must exist before the next user action — and only
+then.
+
+## Testing
+
+Reactor tests usually just want to assert that a side effect was
+triggered. Use the matchers:
+
+```ruby
+require "acta/testing"
+require "acta/testing/matchers"
+
+RSpec.describe RegistrationsController do
+  it "publishes UserSignedUp on successful signup" do
+    expect {
+      post :create, params: { user: { email: "alice@example.com" } }
+    }.to emit(UserSignedUp).with(email: "alice@example.com")
+  end
+end
+```
+
+For the reactor itself, run it inline so the side effect actually
+fires:
+
+```ruby
+RSpec.describe WelcomeEmailReactor do
+  it "sends the welcome email" do
+    Acta::Testing.test_mode do
+      Acta.emit(UserSignedUp.new(user_id: "u_1", email: "alice@example.com"))
+    end
+
+    expect(UserMailer.deliveries.last.to).to eq([ "alice@example.com" ])
+  end
+end
+```
+
+`Acta::Testing.test_mode` runs reactors inline for the duration of
+the block, regardless of the `sync!` declaration on the class. It
+keeps reactor tests synchronous without committing the whole reactor
+to sync mode in production.
+
+## When this isn't the right shape
+
+This pattern works when the AR records (`User`, `BillingAccount`) are
+the source of truth and the events are notifications about state
+changes happening elsewhere. It does **not** make the event log the
+authoritative source of state — `User.create!` happens before any
+event is emitted, and dropping the events table doesn't recreate
+users on the next `Acta.rebuild!`.
+
+When you want the events to *be* the source of truth — when
+`Acta.rebuild!` should reproduce the projected state from the log
+alone — reach for projections instead. See the [event sourcing][es]
+pattern.
+
+[es]: ../README.md#4-project-state-event-sourced
+
+## Compared to the alternatives
+
+| | AR callbacks | `ActiveSupport::Notifications` | [Wisper][wisper] | Acta event-driven |
+|---|---|---|---|---|
+| Persistence | None | None | None | Yes — full payload, actor, timestamps |
+| Async by default | No (in tx) | No (in caller) | No (in caller); async via wisper-sidekiq | Yes (ActiveJob) |
+| Failure isolation | No (rolls back tx) | Sometimes | Subscriber errors propagate to publisher | Yes (per-reactor jobs) |
+| Replay-able | No | No | No | Yes (the events are still there) |
+| Payload typing | AR attributes | Untyped hash | Untyped args | ActiveModel-typed attributes with validations |
+| Subscriber discovery | Reading the model file | Grep the codebase for `subscribe` | Subscriber registration code | `app/reactors/` directory |
+| Test ergonomics | Stubs all the way down | Subscribe a block in spec | wisper-rspec matchers | Built-in matchers + `test_mode` |
+
+[wisper]: https://github.com/krisleech/wisper
+
+`ActiveSupport::Notifications` is the in-process, ephemeral cousin —
+fire-and-forget, no persistence, ideal for instrumentation (metrics,
+traces, logs) but a poor fit for domain events that other parts of
+the system need to react to.
+
+**Wisper** is the long-standing prior art for Rails domain pub/sub —
+publish a symbol-named event, subscribers register interest, the
+gem dispatches. It's at v3.0.0 (May 2024) with light ongoing
+maintenance; not abandoned, not actively developed either. Reach for
+Wisper when you want to decouple callbacks without buying into
+event sourcing or a persistent log: subscriptions are dynamic,
+events are untyped (any args you want), and the runtime is
+process-local. Acta differs in three load-bearing ways: events are
+**typed classes** with validated payload schemas (so a typo in a
+field name is a class-load error, not a runtime nil); subscribers
+are **after-commit + async** by default (so a flaky external API
+call doesn't roll back the publisher's transaction); and every
+publication is **persisted** in the events table (so you have an
+audit log, can replay history, and can survive a process restart
+mid-flight on a notification).
+
+The honest summary: AS::Notifications for instrumentation, Wisper
+for lightweight in-process pub/sub without persistence, Acta when
+the publication itself needs to be durable and the subscribers are
+fan-out side effects you want isolated from the request path.

data/docs/upcasters.md

@@ -0,0 +1,303 @@
+# Schema evolution with upcasters
+
+Acta records are immutable: once an event lands in the events table,
+nothing edits it. That's the property the audit log relies on. But
+app schemas evolve — a new attribute appears, a semantic shifts, an
+event type gets renamed. The straightforward options are unappealing:
+
+- **Mutate history** — rewrite the events table. Breaks immutability
+  and any external consumer of the log.
+- **Snapshot the boundary** — preserve projections at the cut and
+  declare replay-from-zero unsupported. Loses event sourcing's core
+  promise.
+- **Accept that replay-from-zero is broken** — keep emitting new
+  shapes but admit `Acta.rebuild!` can't produce the current
+  projection from scratch. Corrosive over time.
+
+**Upcasters** are the standard event-sourcing answer: at replay
+time, transform old-shape records into new-shape records in memory,
+before projections see them. The stored rows are never touched. The
+transformation logic lives in code, where it's tested and audited.
+
+## When to reach for an upcaster
+
+- You renamed an event type (`ItemCreated` → `WorkspaceCreated`) and
+  want old records to apply to the new projection.
+- You added a required field to an event class and old records lack
+  it; you can derive a default at replay time.
+- You're dropping an obsolete event type and want pre-deprecation
+  records to be skipped on replay.
+- You're splitting one logical event into several finer-grained ones
+  (a 1-to-many fan-out at replay).
+
+If your schema change is purely additive *and* every site that reads
+the field tolerates `nil`, you can probably skip upcasters: bump the
+event class to add the attribute, leave `event_version` alone, and
+projections cope with missing-field cases inline. Reach for an
+upcaster when "tolerate missing field" turns into more conditional
+logic than the transform would be.
+
+## The shape of an upcaster
+
+```ruby
+# app/upcasters/workspace_migration_upcasters.rb
+module WorkspaceMigrationUpcasters
+  include Acta::Upcaster
+
+  upcasts "ItemCreated", from: 1, to: 2 do |event, context|
+    payload = event.payload
+
+    if payload["item_type"] == "goal"
+      # A v1 goal becomes a v2 workspace. Record the mapping so
+      # descendant items can resolve their workspace_id below.
+      context[:goal_to_workspace][payload["item_id"]] = payload["item_id"]
+
+      event.upcast_to(
+        type: "WorkspaceCreated",
+        payload: {
+          "workspace_id" => payload["item_id"],
+          "title"        => payload["title"]
+        },
+        schema_version: 2
+      )
+    else
+      workspace_id =
+        context[:goal_to_workspace][payload["parent_id"]] ||
+        context[:item_to_workspace][payload["parent_id"]]
+
+      if workspace_id.nil?
+        context.fail_replay!(
+          "Unmappable item #{payload['item_id']}: no goal ancestor"
+        )
+      end
+
+      context[:item_to_workspace][payload["item_id"]] = workspace_id
+
+      event.upcast_to(
+        payload: payload.merge("workspace_id" => workspace_id),
+        schema_version: 2
+      )
+    end
+  end
+end
+```
+
+Register it once at boot:
+
+```ruby
+# config/initializers/acta_upcasters.rb
+Acta.register_upcaster(WorkspaceMigrationUpcasters)
+```
+
+Then bump the new emit path so freshly emitted events carry
+`event_version: 2`:
+
+```ruby
+class ItemCreated < Acta::Event
+  def self.event_version = 2
+  attribute :item_id,      :string
+  attribute :workspace_id, :string
+  # ...
+end
+```
+
+That's the whole feature surface. New writes are at v2; old reads
+get upcasted to v2 before they hit projections.
+
+## What blocks can return
+
+Inside an `upcasts` block, the return value controls what the
+pipeline does next:
+
+| Return value                          | Effect                                           |
+| ------------------------------------- | ------------------------------------------------ |
+| `event.upcast_to(...)`                | Continue chaining at the new (type, version)     |
+| Array of `event.upcast_to(...)`       | Fan-out: each branch chains independently        |
+| `nil` or `[]`                         | Drop the record from this replay                 |
+| `context.fail_replay!("reason")`      | Halt with `Acta::ReplayHaltedByUpcaster`         |
+
+If you need to leave a record alone at the current version — e.g. a
+boundary-marker event that's already in its final shape — use the
+NO_OP sentinel:
+
+```ruby
+upcasts "GoalPromotedToWorkspace", from: 2, to: 2, &Acta::Upcaster::NO_OP
+```
+
+## Stateless vs stateful upcasters
+
+Upcasters come in two flavors and the distinction matters for which
+read surfaces will produce correct output.
+
+**Stateless** — the transform depends only on the record itself.
+Adding a default for a new field, renaming a key in the payload,
+or unconditionally bumping the event type all qualify. The
+`context` argument is ignored.
+
+```ruby
+upcasts "ItemCreated", from: 1, to: 2 do |event, _ctx|
+  event.upcast_to(
+    payload: event.payload.merge("workspace_id" => "default"),
+    schema_version: 2
+  )
+end
+```
+
+**Stateful** — the transform depends on context populated by an
+earlier event in the same replay. Resolving a descendant's
+`workspace_id` from a goal seen earlier in the stream is the
+canonical example.
+
+```ruby
+upcasts "ItemCreated", from: 1, to: 2 do |event, ctx|
+  payload = event.payload
+  if payload["item_type"] == "goal"
+    ctx[:goal_to_workspace][payload["item_id"]] = payload["item_id"]
+    event.upcast_to(type: "WorkspaceCreated", ...)
+  else
+    workspace_id = ctx[:goal_to_workspace][payload["parent_id"]]
+    event.upcast_to(payload: payload.merge("workspace_id" => workspace_id), schema_version: 2)
+  end
+end
+```
+
+Stateful upcasters require **global insertion order**, which is
+exactly what `Acta.rebuild!` (and `Acta.events.all` / `#each`)
+provides. They will silently produce incomplete output on read
+surfaces that can't supply that order — see the next section.
+
+## Context semantics across read surfaces
+
+Different read paths give upcasters different views of the world.
+Stateless upcasters are unaffected by any of this; stateful
+upcasters need to know.
+
+| Read surface                          | Context lifetime                 | Safe for stateful upcasters? |
+| ------------------------------------- | -------------------------------- | ---------------------------- |
+| `Acta.rebuild!`                       | One shared, full insertion order | Yes                          |
+| `Acta.events.all` / `#each`           | One shared, full insertion order | Yes                          |
+| `Acta.events.find_by_uuid(uuid)`      | Fresh per call                   | No — incomplete resolution   |
+| `Acta.events.first` / `.last`         | Fresh per call                   | No — incomplete resolution   |
+| `Acta.events.for_stream(...)#all`     | Shared, but stream-ordered       | Usually no — wrong order     |
+| `Acta::ReactorJob#perform`            | Fresh, single record             | No — incomplete resolution   |
+| Web admin (`Acta::Web::EventsController`) | N/A — shows raw stored rows  | N/A                          |
+
+The pattern: any time you hand the pipeline a full ordered stream,
+stateful upcasters work. Any time you hand it one record (or a
+stream-reordered subset), they can't reconstruct the state they
+need.
+
+### Implication for stateful migrations
+
+A stateful migration is fundamentally a `rebuild!`-shaped operation.
+The cutover playbook is:
+
+1. Deploy code that emits at the new `event_version` and includes
+   the upcasters.
+2. Drain the reactor queue. Jobs enqueued before the deploy will
+   re-hydrate their events through the upcaster pipeline with a
+   fresh context — fine for stateless upcasters, possibly
+   incomplete for stateful ones.
+3. Run `Acta.rebuild!` to regenerate projections from the full
+   ordered log under the new schema.
+4. Flip reads.
+
+Apps that need stateful read-time resolution outside `rebuild!`
+should consider whether the resolved field belongs in the
+projection rather than in the upcaster — projections are the
+durable, queryable view, and once `rebuild!` has run, projections
+hold the post-upcast state without needing the pipeline at read
+time.
+
+## Chaining across N versions
+
+Upcasters can be declared on a single event type across many
+versions; the pipeline walks them in order:
+
+```ruby
+module SuccessiveBumps
+  include Acta::Upcaster
+
+  upcasts "ItemCreated", from: 1, to: 2 do |e, _|
+    e.upcast_to(payload: e.payload.merge("workspace_id" => "?"), schema_version: 2)
+  end
+
+  upcasts "ItemCreated", from: 2, to: 3 do |e, _|
+    e.upcast_to(payload: e.payload.except("legacy_kind"), schema_version: 3)
+  end
+end
+```
+
+A v1 record passes through both transforms in sequence. A v2 record
+(emitted between the two migrations) picks up the second transform
+only. Events emitted by current v3 code pass through identity.
+
+## Testing upcasters
+
+Two helpers live in `Acta::Testing::DSL`:
+
+- `acta_seed_event(type:, payload:, event_version: 1, ...)` —
+  inserts an event row directly, bypassing `Acta.emit` (which always
+  stamps the *current* code's `event_version`).
+- `acta_replay(events:, upcasters: [])` — registers the supplied
+  upcasters, seeds events, and runs `Acta.rebuild!`.
+
+```ruby
+RSpec.describe "Workspaces migration" do
+  include Acta::Testing::DSL
+
+  it "promotes goals to workspaces and rewires descendants" do
+    acta_replay(
+      upcasters: [ WorkspaceMigrationUpcasters ],
+      events: [
+        { type: "ItemCreated", event_version: 1,
+          payload: { "item_id" => "g_1", "item_type" => "goal", "title" => "Q3" } },
+        { type: "ItemCreated", event_version: 1,
+          payload: { "item_id" => "i_2", "parent_id" => "g_1", "title" => "Plan" } }
+      ]
+    )
+
+    expect(Workspace.pluck(:id)).to eq([ "g_1" ])
+    expect(Item.find("i_2").workspace_id).to eq("g_1")
+  end
+end
+```
+
+The existing `ensure_replay_deterministic` matcher implicitly
+exercises the upcaster pipeline twice — impure upcasters (state
+leaking outside the per-replay context) surface as a snapshot diff
+on the second pass.
+
+## What upcasters intentionally do *not* do
+
+- **They don't rewrite the event store.** Stored rows are immutable;
+  transforms exist only in memory during a replay pass.
+- **They don't run on the live emit path.** `Acta.emit` stamps the
+  current code's `event_version` and dispatches the in-memory event
+  directly — no read round-trip, no upcaster pass. Live writes are
+  always at the latest version.
+- **They're not a migration framework.** The decision to bump
+  `event_version` and write an upcaster is the schema migration. The
+  upcaster's job is just replay correctness.
+- **They're not cross-tenant.** Each tenant's events table replays
+  independently; upcaster context is per-replay-pass, per-tenant.
+
+## Edge cases worth knowing about
+
+- **Future-version records.** If a replay sees an event whose stored
+  `event_version` exceeds the highest `to` any registered upcaster
+  knows how to reach for that type, the pipeline raises
+  `Acta::FutureSchemaVersion`. Typically: an older deployment is
+  replaying events emitted by a newer one. Halting is the safe call.
+- **Type renames remove the need to keep old classes around.**
+  Upcasters operate on raw records pre-hydration, so the original
+  `ItemCreated` constant can be deleted from the codebase the moment
+  its upcaster renames the type. Hydration only ever happens for
+  classes the upcaster pipeline produces.
+- **1-to-many composes with chaining.** Each record an upcaster
+  fans out into walks the version ladder independently — including
+  through more 1-to-many transforms if you have them.
+- **Conflicting registrations raise at boot.** Two upcaster classes
+  that claim the same `(event_type, from)` pair surface as
+  `Acta::UpcasterRegistryError` the moment the second one registers.
+  Pick an owner.

data/gemfiles/rails_7_2.gemfile

@@ -0,0 +1,20 @@
+# frozen_string_literal: true
+
+source "https://rubygems.org"
+
+gem "activejob",     "~> 7.2.0"
+gem "activemodel",   "~> 7.2.0"
+gem "activerecord",  "~> 7.2.0"
+gem "activesupport", "~> 7.2.0"
+gem "railties",      "~> 7.2.0"
+
+group :development, :test do
+  gem "irb"
+  gem "pg", "~> 1.5"
+  gem "rake", "~> 13.0"
+  gem "rspec", "~> 3.13"
+  gem "rubocop-rails-omakase", require: false
+  gem "sqlite3", "~> 2.0"
+end
+
+gemspec path: ".."

data/gemfiles/rails_8_0.gemfile

@@ -0,0 +1,20 @@
+# frozen_string_literal: true
+
+source "https://rubygems.org"
+
+gem "activejob",     "~> 8.0.0"
+gem "activemodel",   "~> 8.0.0"
+gem "activerecord",  "~> 8.0.0"
+gem "activesupport", "~> 8.0.0"
+gem "railties",      "~> 8.0.0"
+
+group :development, :test do
+  gem "irb"
+  gem "pg", "~> 1.5"
+  gem "rake", "~> 13.0"
+  gem "rspec", "~> 3.13"
+  gem "rubocop-rails-omakase", require: false
+  gem "sqlite3", "~> 2.0"
+end
+
+gemspec path: ".."

data/gemfiles/rails_8_1.gemfile

@@ -0,0 +1,20 @@
+# frozen_string_literal: true
+
+source "https://rubygems.org"
+
+gem "activejob",     "~> 8.1.0"
+gem "activemodel",   "~> 8.1.0"
+gem "activerecord",  "~> 8.1.0"
+gem "activesupport", "~> 8.1.0"
+gem "railties",      "~> 8.1.0"
+
+group :development, :test do
+  gem "irb"
+  gem "pg", "~> 1.5"
+  gem "rake", "~> 13.0"
+  gem "rspec", "~> 3.13"
+  gem "rubocop-rails-omakase", require: false
+  gem "sqlite3", "~> 2.0"
+end
+
+gemspec path: ".."

data/lib/acta/errors.rb

@@ -99,4 +99,42 @@ module Acta
       )
     end
   end
+
+  # Raised by `context.fail_replay!(reason)` inside an upcaster block. Halts
+  # replay so the operator can investigate rather than land a partial,
+  # possibly-corrupt projection.
+  class ReplayHaltedByUpcaster < Error
+    attr_reader :record, :reason
+
+    def initialize(record:, reason:)
+      @record = record
+      @reason = reason
+      super(
+        "Upcaster halted replay on event id=#{record.id} uuid=#{record.uuid} " \
+        "(#{record.event_type} v#{record.event_version}): #{reason}"
+      )
+    end
+  end
+
+  # Raised at registration time when an upcaster set is malformed — e.g.
+  # `from` >= `to`, or two upcasters claim the same (event_type, from).
+  class UpcasterRegistryError < Error; end
+
+  # Raised when a record's stored event_version exceeds anything the
+  # currently-loaded upcaster registry knows how to reach. Typically means
+  # an older deployment is replaying events emitted by a newer one.
+  class FutureSchemaVersion < Error
+    attr_reader :record, :latest_known_version
+
+    def initialize(record:, latest_known_version:)
+      @record = record
+      @latest_known_version = latest_known_version
+      super(
+        "Event id=#{record.id} uuid=#{record.uuid} (#{record.event_type}) is at " \
+        "event_version #{record.event_version}, but the loaded upcaster registry " \
+        "only knows up to v#{latest_known_version}. Likely an older deployment " \
+        "replaying events emitted by a newer one."
+      )
+    end
+  end
 end