active_durable 0.5.0 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 593d546eb6e8e46a70c03be3945934ed0c8a189937a5a27256478783cd345df0
-  data.tar.gz: 3226b77ada034c6bf49830dfdb30457904e14104b430b4cfb0a9def3d3de4124
+  metadata.gz: ca565450baafb042b5ec363d2d7c101f24edd3281b180d43c173eaeff8dd8df8
+  data.tar.gz: 96ceecb07cf0d85b0d8aaf604dececee56301983d16055de408e533c53059da6
 SHA512:
-  metadata.gz: f0f0adcff14966f0b8cf7710218a6a6df9b27152593ea4d0d29c90f1968967817b467c33b9803dac0b7aaec6ff105c291a50ecf1335f15882b1969c4d7f7d2bd
-  data.tar.gz: 1ecba4e399e4f9a247ea9717e9727a50c86d7389aa41a4e53fe3aa9817a100e04a91ecfdc7a2dc51333b24ae9b94a666f1a3c2d5e1987132a48949970a210592
+  metadata.gz: a6dafa621dba36aac0429fc264922d876d378390aa7fccebf05a085bbcb1f2e001d7bf57d202075299e47cbbb20d29af86df62000adca22367bec76ab18dd16b
+  data.tar.gz: 7b4c90ddb5104811ab17be9532fe2c4ca029bae139eeeaafdef1bd04a222fa4c2896c34b2ce44376d3ff13222473e76276a991c125fcdbe057937d7e6a649896

data/.yardopts

@@ -0,0 +1,8 @@
+--markup markdown
+--readme README.md
+--title "ActiveDurable"
+--no-private
+--hide-api private
+lib/**/*.rb
+-
+CHANGELOG.md

data/CHANGELOG.md

@@ -2,13 +2,110 @@
 
 All notable changes to this project are documented here. The format follows
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project uses
-[Semantic Versioning](https://semver.org/). Before 1.0 the database schema may change between minor versions:
-regenerate the migration when upgrading.
+[Semantic Versioning](https://semver.org/). Schema changes ship as new migrations: after updating the gem, run
+`bin/rails generate active_durable:upgrade` and `bin/rails db:migrate`.
 
-## [Unreleased]
+## [0.7.0] - 2026-10-07
+
+Fixes from an architecture and security review. Most of them stop a saga from paying twice or undoing the wrong
+thing.
+
+Upgrading from 0.6: `bin/rails generate active_durable:upgrade && bin/rails db:migrate`. The new migration only
+changes MySQL databases: it rebuilds the three tables and blocks writes to them while it runs, so run it at a quiet
+time, or prune first. Note the behavior changes below: more errors block instead of undoing, `retry` no longer
+resets every failed step, rerun ids are random, and `Durable.start` refuses some ids.
+
+### Changed
+
+- More errors count as bugs and block the saga instead of being retried and undone: `ArgumentError`, `TypeError`,
+  `IndexError` and `KeyError`, `FrozenError`, `ZeroDivisionError`, `RangeError`, `NoMatchingPatternError`,
+  `LocalJumpError`, `RegexpError` and `EncodingError`, besides `NameError` and `NoMethodError`. The list is the new
+  `config.code_errors`, which takes classes or names, so an app can add its own errors or drop one.
+- A step that blocks is written in the notebook as `blocked`, and runs again after `ActiveDurable.retry`. Rescuing
+  its error in the recipe no longer lets the saga carry on.
+- `ActiveDurable.retry` gives a fresh set of attempts only to the step that blocked the saga (with its whole
+  `flow.parallel`) and to steps that hit a bug. Compensating, it still resets every failed undo.
+- Rerun ids are `<id>~rerun-<random>` instead of `<id>~rerun-<count>`.
+- `ActiveDurable.compensate` refuses `running` executions, even after their lease ran out.
+- `Durable.start` refuses ids that are blank, contain `/`, a NUL character or bytes that are not UTF-8.
+- The serializer converts binary strings that are valid UTF-8 (such as `Net::HTTP` bodies) and strings in other
+  encodings to UTF-8, and rejects NUL characters and invalid bytes, in values and keys.
+
+### Fixed
+
+- `retry` ran again a failed step the recipe had already handled: with a fallback (Stripe fails, the recipe pays
+  with PayPal) and a later bug, retrying charged twice. `rerun` did the same; it now copies the failed steps before
+  the chosen one too.
+- A step whose result the database refused (a NUL on PostgreSQL, bytes that are not UTF-8 on MySQL and SQLite, any
+  failed write) counted as a failed attempt: it ran again, then the saga was undone without undoing it. It now
+  blocks.
+- Undoing a saga by hand skipped a step declared `undo_on_failure` that was between attempts or blocked, and the
+  finished branches of a `flow.parallel` stopped halfway.
+- An operator could undo a saga while its worker was still inside a step longer than `lease_duration`.
+- A pending signal nobody was waiting for (a duplicate webhook) made the saga enqueue itself again forever.
+- Signals sent to a blocked saga were refused and lost.
+- `rerun` lost the undos of the `flow.parallel` branches before the chosen step, and from a branch it ran nothing;
+  the dashboard no longer offers branches.
+- On MySQL, ids and step names that differ only in case or accents were treated as the same. The new migration
+  makes those columns `utf8mb4_bin`.
+- `raise ActiveRecord::Rollback` inside `flow.transaction`, an undo or a hook counted as success.
+- On Rails 6.1 and 7.0 with MySQL, two concurrent `start` calls with the same id inside the app's transaction made
+  the second raise `RecordNotFound`.
+- Rerun ids collided after pruning, and could give a new rerun the tickets of a pruned one.
+- An id with `/` broke the whole dashboard list.
+- `LoadError`, `NotImplementedError` and `SystemStackError` left the saga running, retried forever by the sweeper.
+- A bug in an undo used up all its attempts before blocking; it now blocks at once.
+- Error messages are cleaned up (invalid bytes, NUL) before they are stored.
+- README: how to run Solid Queue in development, and the sweeper and cleanup entries go under the `production:` key
+  Rails already wrote in `config/recurring.yml`. Pasting a second `production:` key silently dropped Rails' own
+  `clear_solid_queue_finished_jobs` task.
+
+### Added
+
+- Benchmarks in `benchmarks/`: the cost of a step and of resuming a long saga (`throughput.rb`), and many worker
+  processes on the same sagas, optionally killing one with SIGKILL every second (`load.rb`, `CHAOS=1`). The README
+  has a "Performance" section with the results on PostgreSQL, MySQL and SQLite.
+- README: what counts as a bug, `config.code_errors`, and new limits: ids are forgotten after pruning, the lease and
+  slow steps, clocks, and secrets in error messages.
+
+## [0.6.0] - 2026-10-06
+
+Usable in a real app: found by a first-time user's walkthrough, plus hooks, cleanup, upgrades and an API reference.
+
+Upgrading from 0.5: `bin/rails generate active_durable:upgrade && bin/rails db:migrate`. Note the behavior change
+below: a plain exception raised by a recipe now blocks the saga instead of undoing it; use `flow.abort!` for business
+rejections.
+
+### Added
+
+- `flow.on(:completed) { ... }` and `flow.on(:compensated) { ... }`: hooks to update your own records when a saga
+  ends. Declared before the first step, so a saga undone early still knows them. Each runs once, in a transaction
+  with the notebook entry that records it (exactly once for database changes, covered by the crash tester); a failing
+  hook blocks the execution and `ActiveDurable.retry` runs only the hook. The dashboard lists them in the notebook,
+  and OpenTelemetry traces them.
+- Cleanup: `ActiveDurable.prune(older_than:)`, `ActiveDurable::PruneJob` and `bin/rails active_durable:prune` delete
+  finished executions (completed, compensated, superseded) older than `config.keep_finished_for` (30 days by
+  default), with their notebook and signals, in batches. Active and blocked executions are never deleted.
+- An API reference: the public API has YARD docs (parameters, returns, errors, examples) and the internals are
+  marked `@api private`, so rubydoc.info shows only what an app calls. `.yardopts` configures it.
+- `bin/rails generate active_durable:upgrade` adds the migrations a newer version needs, skipping the ones the app
+  already has. The first one is an index on `durable_executions (status, updated_at)` for the cleanup and the
+  sweeper; new installs get it from `active_durable:install`.
 
 ### Changed
 
+- **A bug no longer undoes a saga.** Only a step that fails for good, or `flow.abort!`, undoes the finished steps.
+  Any other error raised by the recipe, and a `NameError` or `NoMethodError` inside a step (now
+  `ActiveDurable::CodeError`, not retried), blocks the execution instead: a typo in a deploy used to refund every
+  saga that woke up with it. Fix the code and call `ActiveDurable.retry`. A business rejection raised as a plain
+  exception in the recipe body now blocks too: use `flow.abort!` (or raise `ActiveDurable::Abort`).
+- The repository moved to [webresstudio/active_durable](https://github.com/webresstudio/active_durable). Links to the
+  old address redirect.
+- A website, in English and Spanish, with an interactive simulator of a checkout: pick what goes wrong (a crash, a
+  refused parcel, a declined card…) and watch the steps, the notebook and the outside world. Source in `site/`, built
+  by `bin/site` and published to GitHub Pages by `.github/workflows/pages.yml`. The gem's homepage and both READMEs
+  link to it. Its simulator also shows a bug in a deploy (the saga is blocked, nothing is undone, a retry carries it
+  on) and the order's own status, set by `flow.on(:compensated)`.
 - README: the quick start recipe reads top to bottom, with one-line undos and the Stripe calls in a `Payments`
   module where charging and refunding sit side by side. A new section, "In a Rails app", sets up a Rails app step by
   step: install, job backend, sweeper, the initializer with every setting, routes, where each piece of code goes
@@ -16,6 +113,20 @@ regenerate the migration when upgrading.
   before going to production. The settings moved there from "Observability".
 - Specs cover undos without arguments (`-> { ... }`), a `Method` as an undo, and `flow.abort!` inside a step, which
   skips the step's remaining retries.
+- gemspec: Active Job, Active Record and Active Support are required `>= 6.1, < 9`, the versions CI tests, instead
+  of any version from 6.1 on. The duplicate homepage link is gone, so `gem build` no longer warns.
+
+### Fixed
+
+- The rake tasks were loaded twice (Rails already loads an engine's `lib/tasks`), so each one ran twice.
+- `bin/rails active_durable:sweep` with the `:async` adapter (the Rails default in development) enqueued jobs that died
+  with the rake process. It now runs the due executions itself.
+- Dashboard: the execution id no longer widens its column (long ids wrap), and long problems take two lines, with the
+  full message on hover.
+- README: `Payments` turns Stripe's errors into its own, so the recipe does not depend on Stripe; development with
+  `:async` is explained (sagas started from the console are lost until the sweeper runs); a Minitest example. The
+  order page shows the order's own status, written by the saga (`mark_shipped`, `flow.on(:compensated)`), instead of
+  the saga's status, which said "Processing…" for days after the parcel left.
 
 ## [0.5.0] - 2026-10-06
 

data/README.md

@@ -5,22 +5,25 @@
 </p>
 
 <p align="center">
-  <a href="https://github.com/williamromero/active_durable/actions/workflows/main.yml"><img src="https://github.com/williamromero/active_durable/actions/workflows/main.yml/badge.svg" alt="CI"></a>
+  <a href="https://github.com/webresstudio/active_durable/actions/workflows/main.yml"><img src="https://github.com/webresstudio/active_durable/actions/workflows/main.yml/badge.svg" alt="CI"></a>
   <img src="https://img.shields.io/badge/ruby-3.1%2B-CC342D?logo=ruby&logoColor=white" alt="Ruby 3.1 and newer">
   <img src="https://img.shields.io/badge/rails-6.1%2B-D30001?logo=rubyonrails&logoColor=white" alt="Rails 6.1 and newer">
   <img src="https://img.shields.io/badge/PostgreSQL%20%C2%B7%20MySQL%20%C2%B7%20SQLite-tested-3DD6A0" alt="PostgreSQL, MySQL and SQLite">
   <img src="https://img.shields.io/badge/no%20Redis-no%20extra%20servers-7EA6FF" alt="No Redis, no extra servers">
   <a href="LICENSE.txt"><img src="https://img.shields.io/badge/license-MIT-B08CFF" alt="MIT license"></a>
+  <a href="https://webresstudio.github.io/active_durable/"><img src="https://img.shields.io/badge/website-try%20the%20simulator-F2B641" alt="Website: try the interactive simulator"></a>
 </p>
 
 <p align="center">
+  <a href="https://webresstudio.github.io/active_durable/"><b>Website</b></a> ·
   <a href="#quick-start">Quick start</a> ·
   <a href="#in-a-rails-app">In a Rails app</a> ·
   <a href="#how-it-works">How it works</a> ·
   <a href="#the-building-blocks">Building blocks</a> ·
   <a href="#dashboard">Dashboard</a> ·
   <a href="#testing-the-crash-tester">Crash tester</a> ·
-  <a href="#compatibility">Compatibility</a>
+  <a href="#compatibility">Compatibility</a> ·
+  <a href="https://rubydoc.info/gems/active_durable">API reference</a>
 </p>
 
 ---
@@ -61,6 +64,7 @@ Write the recipe once, in `app/sagas/`:
 # app/sagas/checkout_saga.rb
 CheckoutSaga = Durable.define(:checkout) do |flow, order_id:|
   order = Order.find(order_id)
+  flow.on(:compensated) { order.update!(status: "cancelled") } # once every finished step was undone
 
   # Touches only your database: committed together with its checkpoint, so it runs exactly once.
   flow.transaction :reserve_stock, undo: -> { order.release_stock! } do
@@ -71,7 +75,7 @@ CheckoutSaga = Durable.define(:checkout) do |flow, order_id:|
   # Talks to the outside world: the ticket is an idempotency key that never changes for this step.
   payment = flow.step :charge, undo: ->(charge, ticket) { Payments.refund(charge, ticket) } do |ticket|
     Payments.charge(order, ticket)
-  rescue Stripe::CardError => e
+  rescue Payments::CardDeclined => e
     flow.abort!(e.message) # a declined card is not retried: the stock is released right away
   end
 
@@ -79,6 +83,7 @@ CheckoutSaga = Durable.define(:checkout) do |flow, order_id:|
   flow.pivot :dispatch do |ticket|
     { "tracking" => Carrier.ship(order.id, reference: ticket).tracking_number, "payment" => payment["id"] }
   end
+  flow.transaction(:mark_shipped) { order.update!(status: "shipped") } # your own status, for your pages
 
   flow.step(:confirmation_email) { OrderMailer.shipped(order.id).deliver_now && true }
   flow.sleep(:wait_for_delivery, 3.days) # no worker is held while it sleeps
@@ -91,12 +96,16 @@ The calls to Stripe live in a plain module, doing and undoing side by side. The
 ```ruby
 # app/services/payments.rb
 module Payments
+  class CardDeclined < StandardError; end
+
   def self.charge(order, ticket)
     intent = Stripe::PaymentIntent.create(
       { amount: order.total_cents, currency: "usd", customer: order.user.stripe_id, confirm: true },
       { idempotency_key: ticket }
     )
     { "id" => intent.id } # written in the notebook, so it must fit in JSON
+  rescue Stripe::CardError => e
+    raise CardDeclined, e.message # the recipe speaks your language, not Stripe's
   end
 
   def self.refund(charge, ticket)
@@ -131,6 +140,9 @@ Run the three commands from the [quick start](#quick-start). The migration creat
 exactly once only because the notebook and your data commit together. A queue in its own database, like Solid
 Queue's in Rails 8, is fine.
 
+When you update the gem, run `bin/rails generate active_durable:upgrade` and then `bin/rails db:migrate`: it adds only
+the migrations your app is missing, and running it twice changes nothing.
+
 ### 2. A job backend
 
 ActiveDurable runs on Active Job, so it uses the backend you already have. New Rails 8 apps come with Solid Queue;
@@ -145,13 +157,26 @@ config.solid_queue.connects_to = { database: { writing: :queue } }
 
 - **Workers must listen to the saga queue.** It is `:default` unless you change `config.queue_name`; if you do, add
   it to your workers (Solid Queue's `config/queue.yml`, Sidekiq's `-q`).
-- **In development** Rails' default `:async` adapter runs jobs inside the server, sleeps and retries included. With
-  Solid Queue, run `bin/jobs` next to the server.
+- **In development** Rails' default `:async` adapter keeps each job in the memory of the process that enqueued it.
+  The server runs its own, sleeps and retries included, but a saga started from the console, `bin/rails runner`, a
+  rake task or `db/seeds.rb` is lost when that process exits, and scheduled wake-ups are lost when the server
+  restarts. A minute later, `bin/rails active_durable:sweep` picks them up: with `:async` it runs them right there.
+  Or run Solid Queue in development too, so jobs live in the database: give `development` a `queue` database in
+  `config/database.yml` (like the one `production` has, with `migrations_paths: db/queue_migrate`), add the two
+  lines below, run `bin/rails db:prepare`, and start `bin/jobs` next to the server.
+
+```ruby
+# config/environments/development.rb
+config.active_job.queue_adapter = :solid_queue
+config.solid_queue.connects_to = { database: { writing: :queue } }
+```
 
 ### 3. The sweeper
 
 The safety net: every minute it enqueues executions that lost their job, because the process died between the
-commit and the enqueue or a worker died holding a lease. With Solid Queue:
+commit and the enqueue or a worker died holding a lease. With Solid Queue, add these entries under the
+`production:` key Rails already wrote in `config/recurring.yml` (a second `production:` key would silently replace
+it), and under a `development:` key too if you run Solid Queue in development:
 
 ```yaml
 # config/recurring.yml
@@ -159,11 +184,18 @@ production:
   active_durable_sweep:
     class: ActiveDurable::SweepJob
     schedule: every minute
+  active_durable_prune:
+    class: ActiveDurable::PruneJob
+    schedule: every day at 4am
 ```
 
 With another backend, schedule `ActiveDurable::SweepJob` in its own scheduler (GoodJob cron, sidekiq-cron), or run
 `bin/rails active_durable:sweep` from cron.
 
+The second entry is the cleanup: finished executions (completed, undone or superseded) are kept for
+`config.keep_finished_for`, 30 days by default, and then `ActiveDurable::PruneJob` deletes them with their notebook.
+Active and blocked executions are never deleted. Without it, every saga stays in the database forever.
+
 ### 4. The initializer
 
 The settings, with their defaults:
@@ -179,6 +211,8 @@ ActiveDurable.configure do |config|
   config.backoff = ->(attempt) { [2**attempt, 3600].min } # or [5, 30, 300], or a number
   config.parallel_concurrency = 4       # threads per flow.parallel
   config.sweep_grace = 1.minute         # the sweeper leaves executions this young alone
+  config.keep_finished_for = 30.days    # then ActiveDurable::PruneJob deletes finished ones
+  config.code_errors += ["Payments::Misconfigured"] # your own errors that mean a bug (see "A bug is not a failure")
 
   # Who may open the dashboard outside development and test. With Devise (HTTP basic auth: see Dashboard):
   config.dashboard_authorize = ->(controller) { controller.request.env["warden"]&.user&.admin? }
@@ -237,7 +271,7 @@ class OrdersController < ApplicationController
   end
 end
 
-# app/models/order.rb
+# app/models/order.rb: orders has a status column ("placed" by default) that the saga updates
 class Order < ApplicationRecord
   def checkout
     Durable.find("checkout-#{id}")
@@ -245,15 +279,16 @@ class Order < ApplicationRecord
 end
 ```
 
-The `id:` ties the saga to its order, so the order page can show where the saga is:
+The `id:` ties the saga to its order: `order.checkout` is for your team and the dashboard. The page shows the order's
+own status, which the saga writes as it goes (`mark_shipped`, `flow.on(:compensated)`), not the saga's: a shipped
+order still has a saga sleeping three days before it asks for a review.
 
 ```erb
 <%# app/views/orders/show.html.erb %>
-<% case @order.checkout.status %>
-<% when "completed" %>   Your order is confirmed.
-<% when "compensated" %> We could not complete it and refunded you.
-<% when "blocked" %>     We are looking into it.
-<% else %>               Processing…
+<% case @order.status %>
+<% when "shipped" %>   Your order is on its way.
+<% when "cancelled" %> We could not complete it and refunded you.
+<% else %>             Processing…
 <% end %>
 ```
 
@@ -275,6 +310,7 @@ the notebook, not to the job: if your backend retries the job too, the copy find
 | stop retrying a business failure | `flow.abort!`, like the declined card above |
 | see what is running | the [dashboard](#dashboard), or `ActiveDurable::RunJob` in your backend's UI |
 | hear about a stuck saga | the `blocked.active_durable` [event](#observability) |
+| delete finished sagas | schedule `ActiveDurable::PruneJob` once a day (step 3) |
 | run a saga inline in tests | `ActiveDurable::Testing.drain(id)` |
 | start or wake sagas from your own jobs | call `Durable.start` or `Durable.signal` there |
 
@@ -301,6 +337,26 @@ it "charges once and confirms the order" do
 end
 ```
 
+With Minitest, the Rails default:
+
+```ruby
+# test/test_helper.rb
+require "active_durable/testing"
+
+class ActiveSupport::TestCase
+  setup { ActiveDurable::Testing.reset! }
+end
+
+# test/services/place_order_test.rb
+class PlaceOrderTest < ActiveSupport::TestCase
+  test "charges once and confirms the order" do
+    order = PlaceOrder.call(email: "ana@example.com", total_cents: 4200)
+
+    assert_equal "completed", ActiveDurable::Testing.drain(order.checkout.id).status
+  end
+end
+```
+
 `drain` runs the saga right there, without a worker. Stub Stripe as you already do, then let the
 [crash tester](#testing-the-crash-tester) kill the saga at every point.
 
@@ -308,6 +364,7 @@ end
 
 - [ ] Workers are running and listen to `config.queue_name`.
 - [ ] The sweeper runs every minute.
+- [ ] The cleanup runs every day, or you keep every execution on purpose.
 - [ ] `dashboard_authorize` is set; without it the dashboard answers 403.
 - [ ] `lease_duration` is longer than your slowest step.
 - [ ] Every step that calls an outside service passes the ticket as its idempotency key.
@@ -334,8 +391,8 @@ stateDiagram-v2
   sleeping --> running: wake-up time
   waiting --> running: Durable.signal
   running --> completed: every step done
-  running --> compensated: failure before the pivot, undos ran
-  running --> blocked: needs a person
+  running --> compensated: a step failed for good, or flow.abort!, before the pivot
+  running --> blocked: a bug, a failure after the pivot or a failing hook
   blocked --> pending: ActiveDurable.retry
 ```
 
@@ -357,6 +414,7 @@ Three rules follow from replaying the recipe:
 | `flow.parallel(name) { \|branches\| ... }` | several steps at the same time | each branch like a step |
 | `flow.sleep(name, 3.days)` | waiting without holding a worker | — |
 | `flow.wait_for(name, timeout:)` | waiting for `Durable.signal` | a timeout fails the saga |
+| `flow.on(:completed) { ... }` | updating your own records when the saga ends (also `:compensated`) | blocks; a retry runs the hook again |
 
 Steps take `undo:`, `retry:` (`3`, `false` or `{ attempts:, backoff: }`) and `undo_on_failure:`.
 
@@ -377,11 +435,38 @@ idempotency key, Stripe answers with the first result instead of charging twice.
 
 <br>
 
-When a step runs out of attempts, raises `ActiveDurable::Abort`, or the recipe raises, every finished step is undone,
-last one first. Each undo is written in the notebook too, so a crash in the middle of undoing resumes where it
+When a step runs out of attempts or calls `flow.abort!`, every finished step is undone, last one first. Each undo is written in the notebook too, so a crash in the middle of undoing resumes where it
 stopped. An undo receives `(result, undo_ticket, step_ticket)` and takes as many as it declares: `-> { ... }` takes
 none. Any object that responds to `call` works too, such as `Payments.method(:refund)`.
 
+**A bug is not a failure.** Undoing a saga refunds money and releases stock, so only a step that failed for good or
+`flow.abort!` triggers it. These block the execution instead, without retrying, because a typo in a deploy must never
+refund your customers:
+
+- anything the recipe raises outside a step;
+- inside a step, an error that means the code is wrong: `NameError` (and `NoMethodError`), `ArgumentError`,
+  `TypeError`, `IndexError` (and `KeyError`), `FrozenError`, `ZeroDivisionError`, `RangeError`,
+  `NoMatchingPatternError`, `LocalJumpError`, `RegexpError`, `EncodingError`, and the errors outside
+  `StandardError` such as `LoadError`, `NotImplementedError` and `SystemStackError`;
+- a step whose result cannot be stored (a NUL character, bytes that are not UTF-8): it already acted.
+
+Fix the code and call `ActiveDurable.retry(id)`, or press Retry in the dashboard, and the saga carries on from where it
+stopped. Rescuing the error in the recipe does not help: the saga stops at the next step and blocks anyway. To reject
+the work for a business reason, call `flow.abort!(reason)`.
+
+The list is `config.code_errors`. Add your own errors, as a class or as a name (a name also matches subclasses, and
+works before the gem that defines it is loaded), or drop one:
+
+```ruby
+ActiveDurable.configure do |config|
+  config.code_errors << "Payments::Misconfigured"
+  config.code_errors -= ["ArgumentError"] # if a gem you call raises it for bad input
+end
+```
+
+An `ArgumentError` from bad data, like an invalid date, blocks too: a person decides. To treat one of these errors as
+a failure in a single step, rescue it inside the step and raise another error, or call `flow.abort!`.
+
 A step that failed is not undone, because it did not happen. The exception is a step whose failure may hide a
 success, like a charge whose answer timed out: declare `undo_on_failure: true` and its undo runs with `nil` as the
 result, so it can look the outcome up with the step's ticket.
@@ -409,13 +494,39 @@ everything. After it, steps cannot declare `undo:` and are retried with backoff
 
 </details>
 
+<details>
+<summary><b>When a saga ends: hooks</b></summary>
+
+<br>
+
+`flow.on(:completed)` and `flow.on(:compensated)` run once the saga ends that way, to update your own records:
+
+```ruby
+CheckoutSaga = Durable.define(:checkout) do |flow, order_id:|
+  order = Order.find(order_id)
+  flow.on(:completed) { order.update!(status: "delivered") }
+  flow.on(:compensated) { order.update!(status: "cancelled") }
+
+  flow.transaction(:reserve_stock, undo: -> { order.release_stock! }) { ... }
+end
+```
+
+Declare them before the first step: a saga undone at its first step never reaches the lines after it. `completed`
+runs after the last step and `compensated` after the last undo, each in a transaction together with the notebook
+entry that records it, so a hook that only touches your database runs exactly once, even if the process dies. If a
+hook raises, the execution is blocked, and `ActiveDurable.retry` runs the hook again, not the steps.
+
+For progress before the end (paid, shipped), write a step: `flow.transaction(:mark_shipped) { ... }`.
+
+</details>
+
 <details>
 <summary><b>Sleeping and waiting for signals</b></summary>
 
 <br>
 
 `flow.sleep` writes the wake-up time down and releases the worker. `flow.wait_for` does the same until a signal
-arrives. Signals can arrive before the saga starts waiting.
+arrives. Signals can arrive before the saga starts waiting, or while it is blocked: they are kept until it does.
 
 ```ruby
 kyc = flow.wait_for(:kyc_done, timeout: 2.hours)
@@ -508,13 +619,21 @@ end
 ## Operating sagas
 
 ```ruby
-ActiveDurable.retry("checkout-7")                                   # blocked: try again where it stopped
+ActiveDurable.retry("checkout-7")                                   # blocked: try again the step that blocked it
 ActiveDurable.compensate("checkout-7", reason: "customer cancelled") # undo everything (only before the pivot)
 ActiveDurable.rerun("checkout-7", from: :ship)                      # a new execution reusing steps before :ship
+ActiveDurable.prune(older_than: 30.days)                            # delete finished executions and their notebook
 ```
 
-All three refuse an execution a worker is running right now. A rerun runs the chosen step and the following ones
-again with new tickets, so they have effects again; a blocked original becomes `superseded`.
+`retry` gives the step that blocked the saga a fresh set of attempts. Failed steps the recipe already handled stay
+failed: if Stripe failed and the recipe paid with PayPal instead, a retry never runs Stripe again.
+
+A rerun (`<id>~rerun-<random>`) keeps the steps before the chosen one, failed ones and parallel branches included,
+and runs the chosen step and the following ones again with new tickets, so they have effects again. A blocked
+original becomes `superseded`.
+
+None of them touches an execution a worker is running. `compensate` refuses a `running` execution even after its
+lease ran out: the worker may still be inside a slow step, and if it died, the sweeper resumes it.
 
 ## Testing: the crash tester
 
@@ -554,7 +673,7 @@ end
 
 | Event | Payload |
 | --- | --- |
-| `execution` / `step` / `compensation` / `undo` | `execution_id` (and `recipe`, `step`, `kind`) |
+| `execution` / `step` / `compensation` / `undo` / `hook` | `execution_id` (and `recipe`, `step`, `kind`) |
 | `completed` / `compensated` | `execution_id`, `recipe` |
 | `blocked` | `execution_id`, `recipe`, `error` |
 | `retried` / `compensation_requested` / `rerun` | operator actions |
@@ -601,12 +720,45 @@ Every feature works on every version. Things your app may need on older Rails, u
 | Temporal | the Temporal server | written by hand | a Temporal cluster |
 | **ActiveDurable** | **your database** | **yes, in reverse, with a point of no return** | **nothing extra** |
 
+## Performance
+
+Measured on an Apple M1 Ultra with Ruby 4.0.7 and Rails 8.1, each database on the same machine. The scripts are in
+[`benchmarks/`](benchmarks/README.md), so you can run them on yours.
+
+| | PostgreSQL 16 | MySQL 9.6 | SQLite 3 |
+| --- | --- | --- | --- |
+| A step (`flow.step`), one process | 1.8 ms | 3.0 ms | 0.45 ms |
+| A step (`flow.transaction`), one process | 2.0 ms | 2.8 ms | 0.45 ms |
+| Resuming a saga with 1,000 finished steps | 17 ms | 21 ms | 9 ms |
+| 2,000 sagas of 5 steps, 8 worker processes | 6.9 s (290 sagas/s) | 8.1 s (246 sagas/s) | 1,000 sagas, 4 workers: 6.5 s |
+| The same, killing a worker with SIGKILL every second | 9.2 s, 9 workers killed | 13.2 s, 13 killed | — |
+| Duplicate charges | 0 | 0 | 0 |
+
+A step costs about four queries: the lease check that fences the write, the notebook row and the commit that makes
+it durable. In the load tests each call to the outside world takes 5 ms, so a saga spends most of its time waiting,
+as in a real app. When a worker dies mid-step, another one runs that step again with the same ticket: on PostgreSQL 2
+of the 2,000 charges were sent twice, and the idempotency key made them one.
+
+The same test in a Rails 8 app with Solid Queue: 300 checkouts, every Solid Queue process killed with SIGKILL twice
+while sagas were halfway through. All of them settled, the refunds and hooks ran once, and no order was charged twice.
+
 ## Guarantees and limits
 
 - A step runs **at least once**; with an idempotency key it has its effect once. `flow.transaction` runs exactly
   once, because its change and its checkpoint commit together.
 - One worker at a time per execution: taking an execution and every write are fenced by a lease token.
+- A hook (`flow.on`) runs once; exactly once if it only touches your database.
+- A bug never undoes a saga: an error in the code blocks it until you fix it and call `ActiveDurable.retry`.
 - Sagas do not isolate each other: two sagas can see each other's intermediate states.
+- `Durable.start(id:)` is idempotent while the execution exists. Once it is pruned (`keep_finished_for`, 30 days),
+  the same id starts a new saga with the same tickets: keep finished executions longer than a webhook can be
+  delivered again.
+- The lease is renewed on each notebook write. A step longer than `lease_duration` can run twice at the same time,
+  with the same ticket. Keep the workers' clocks in sync (NTP): the lease compares times written by different
+  machines.
+- Error messages are stored in the notebook and reach the dashboard, the logs, the `blocked` event and
+  OpenTelemetry: keep secrets and personal data out of them.
+- Ids cannot be blank or contain `/`. Ids and step names compare exactly, also on MySQL.
 - `flow.transaction` is atomic only when the notebook lives in the same database as your data.
 
 ## Development

data/app/controllers/active_durable/executions_controller.rb

@@ -23,15 +23,17 @@ module ActiveDurable
       @execution = Execution.find(params[:id])
       steps = @execution.steps.to_a
       @steps = steps
-      @forward_steps = steps.reject(&:undo?).sort_by { |step| step.position.to_i }
+      @forward_steps = steps.select(&:forward?).sort_by { |step| step.position.to_i }
+      @rerun_steps = @forward_steps.select(&:position) # a parallel branch cannot be a starting point
       @undo_steps = steps.select(&:undo?)
+      @hook_steps = steps.select(&:hook?)
       @signals = @execution.signals.to_a
       @reruns = Execution.where(forked_from: @execution.id).order(:created_at).to_a
     end
 
     def retry_now
       ActiveDurable.retry(params[:id])
-      redirect_to execution_path(params[:id]), notice: "Retrying. Failed steps got a fresh set of attempts."
+      redirect_to execution_path(params[:id]), notice: "Retrying. The step that blocked it got a fresh set of attempts."
     end
 
     def compensate

data/app/helpers/active_durable/dashboard_helper.rb

@@ -57,6 +57,13 @@ module ActiveDurable
       safe_join(name.to_s.split(/(?<=_)/), tag.wbr)
     end
 
+    # Ids with a slash cannot be routed; versions before 0.7 accepted them, so they are listed without a link.
+    def execution_link(id, **options)
+      return tag.span(id, title: "Ids with a slash have no page; use the console", **options) if id.include?("/")
+
+      link_to(id, execution_path(id), **options)
+    end
+
     def ticket_for(execution, step)
       "#{execution.id}:#{step.name}"
     end
@@ -75,7 +82,7 @@ module ActiveDurable
     end
 
     def can_compensate?(execution, steps)
-      %w[blocked pending running sleeping waiting].include?(execution.status) && !execution.compensating &&
+      %w[blocked pending sleeping waiting].include?(execution.status) && !execution.compensating &&
         steps.none? { |step| step.kind == "pivot" && step.completed? }
     end
 
@@ -87,7 +94,7 @@ module ActiveDurable
     # their parallel step, undone steps are marked, and an active execution gets a "next" ghost block.
     def track_items(execution, steps)
       undone = steps.select { |step| step.undo? && step.completed? }.to_set { |step| step.name.delete_suffix(":undo") }
-      branches, main = steps.reject(&:undo?).partition { |step| step.position.nil? }
+      branches, main = steps.select(&:forward?).partition { |step| step.position.nil? }
 
       items = main.sort_by(&:position).map do |step|
         build_item(step, undone, branches.select { |branch| branch.name.start_with?("#{step.name}/") })
@@ -109,7 +116,8 @@ module ActiveDurable
 
     def state_word(state)
       {
-        "completed" => "done", "failed" => "failed", "retrying" => "retrying", "waiting" => "waiting for a signal",
+        "completed" => "done", "failed" => "failed", "blocked" => "blocked", "retrying" => "retrying",
+        "waiting" => "waiting for a signal",
         "sleeping" => "sleeping", "undone" => "undone", "next" => "up next", "running" => "running"
       }.fetch(state.to_s, state.to_s)
     end

data/app/views/active_durable/executions/index.html.erb

@@ -42,8 +42,8 @@
         <% @executions.each do |execution| %>
           <% items = track_items(execution, @steps_by_execution.fetch(execution.id, [])) %>
           <tr data-id="<%= execution.id %>" data-status="<%= execution.status %>">
-            <td>
-              <%= link_to execution.id, execution_path(execution), class: "exec-link" %>
+            <td class="exec">
+              <%= execution_link(execution.id, class: "exec-link") %>
               <span class="recipe"><%= execution.recipe %>, version <%= execution.recipe_version %></span>
             </td>
             <td>
@@ -69,7 +69,8 @@
               <% if compensating_now?(execution) %><%= status_pill("compensating") %><% end %>
             </td>
             <td class="when"><%= execution.wake_at ? when_text(execution.wake_at) : tag.span("—", class: "muted") %></td>
-            <td class="problem"><%= execution.error&.dig("message").to_s.truncate(110) %></td>
+            <% problem = execution.error&.dig("message").to_s %>
+            <td class="problem"><span title="<%= problem %>"><%= problem.truncate(110) %></span></td>
             <td class="when"><%= when_text(execution.updated_at) %></td>
           </tr>
         <% end %>