active_durable 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (41) hide show
  1. checksums.yaml +7 -0
  2. data/CHANGELOG.md +131 -0
  3. data/LICENSE.txt +21 -0
  4. data/README.md +630 -0
  5. data/Rakefile +12 -0
  6. data/app/controllers/active_durable/application_controller.rb +25 -0
  7. data/app/controllers/active_durable/executions_controller.rb +59 -0
  8. data/app/helpers/active_durable/dashboard_helper.rb +163 -0
  9. data/app/views/active_durable/executions/index.html.erb +90 -0
  10. data/app/views/active_durable/executions/show.html.erb +197 -0
  11. data/app/views/layouts/active_durable/application.html.erb +422 -0
  12. data/config/routes.rb +13 -0
  13. data/lib/active_durable/configuration.rb +59 -0
  14. data/lib/active_durable/engine.rb +24 -0
  15. data/lib/active_durable/errors.rb +61 -0
  16. data/lib/active_durable/execution.rb +43 -0
  17. data/lib/active_durable/flow.rb +277 -0
  18. data/lib/active_durable/flow_parallel.rb +200 -0
  19. data/lib/active_durable/lease.rb +50 -0
  20. data/lib/active_durable/notebook.rb +100 -0
  21. data/lib/active_durable/open_telemetry.rb +94 -0
  22. data/lib/active_durable/operations.rb +111 -0
  23. data/lib/active_durable/parallel.rb +54 -0
  24. data/lib/active_durable/record.rb +13 -0
  25. data/lib/active_durable/registry.rb +89 -0
  26. data/lib/active_durable/retry_policy.rb +42 -0
  27. data/lib/active_durable/run_job.rb +12 -0
  28. data/lib/active_durable/runner.rb +157 -0
  29. data/lib/active_durable/serializer.rb +40 -0
  30. data/lib/active_durable/signal_record.rb +20 -0
  31. data/lib/active_durable/step.rb +34 -0
  32. data/lib/active_durable/sweep_job.rb +12 -0
  33. data/lib/active_durable/sweeper.rb +22 -0
  34. data/lib/active_durable/testing.rb +118 -0
  35. data/lib/active_durable/version.rb +5 -0
  36. data/lib/active_durable.rb +145 -0
  37. data/lib/generators/active_durable/install/install_generator.rb +27 -0
  38. data/lib/generators/active_durable/install/templates/create_active_durable_tables.rb.tt +60 -0
  39. data/lib/tasks/active_durable.rake +24 -0
  40. data/sig/active_durable.rbs +4 -0
  41. metadata +134 -0
data/README.md ADDED
@@ -0,0 +1,630 @@
1
+ <p align="center"><b>English</b> · <a href="README.es.md">Español</a></p>
2
+
3
+ <p align="center">
4
+ <img src="docs/assets/hero.svg" alt="ActiveDurable: durable sagas for Rails. Finish the work or undo it in order, even if the server dies halfway." width="100%">
5
+ </p>
6
+
7
+ <p align="center">
8
+ <a href="https://github.com/williamromero/active_durable/actions/workflows/main.yml"><img src="https://github.com/williamromero/active_durable/actions/workflows/main.yml/badge.svg" alt="CI"></a>
9
+ <img src="https://img.shields.io/badge/ruby-3.1%2B-CC342D?logo=ruby&logoColor=white" alt="Ruby 3.1 and newer">
10
+ <img src="https://img.shields.io/badge/rails-6.1%2B-D30001?logo=rubyonrails&logoColor=white" alt="Rails 6.1 and newer">
11
+ <img src="https://img.shields.io/badge/PostgreSQL%20%C2%B7%20MySQL%20%C2%B7%20SQLite-tested-3DD6A0" alt="PostgreSQL, MySQL and SQLite">
12
+ <img src="https://img.shields.io/badge/no%20Redis-no%20extra%20servers-7EA6FF" alt="No Redis, no extra servers">
13
+ <a href="LICENSE.txt"><img src="https://img.shields.io/badge/license-MIT-B08CFF" alt="MIT license"></a>
14
+ </p>
15
+
16
+ <p align="center">
17
+ <a href="#quick-start">Quick start</a> ·
18
+ <a href="#in-a-rails-app">In a Rails app</a> ·
19
+ <a href="#how-it-works">How it works</a> ·
20
+ <a href="#the-building-blocks">Building blocks</a> ·
21
+ <a href="#dashboard">Dashboard</a> ·
22
+ <a href="#testing-the-crash-tester">Crash tester</a> ·
23
+ <a href="#compatibility">Compatibility</a>
24
+ </p>
25
+
26
+ ---
27
+
28
+ A checkout reserves stock, charges a card, ships a parcel and sends an email. `ActiveRecord::Base.transaction`
29
+ can roll back your tables, but **a ROLLBACK cannot reach Stripe**. If the server dies after the charge, or the
30
+ carrier refuses the parcel, you end up with money taken and no order.
31
+
32
+ **ActiveDurable** turns that flow into a durable saga stored in your own database:
33
+
34
+ - **Nothing is done twice.** Every finished step is written down. After a crash, another worker continues where
35
+ the first one stopped.
36
+ - **Nothing is left half done.** If a step fails for good, the steps that finished are undone, last one first.
37
+ - **Some things cannot be undone.** Mark the point of no return; after it, steps are retried instead.
38
+ - **Nothing extra to run.** Active Record and Active Job: no Redis, no workflow server.
39
+
40
+ ## See it happen
41
+
42
+ <p align="center">
43
+ <img src="docs/assets/crash.svg" alt="Animation: a checkout runs two steps, the server dies, a new worker reads the notebook, skips the finished steps and Stripe charges only once" width="100%">
44
+ </p>
45
+
46
+ <p align="center">
47
+ <img src="docs/assets/undo.svg" alt="Animation: the dispatch step fails before the point of no return, so the charge is refunded and then the stock is released" width="100%">
48
+ </p>
49
+
50
+ ## Quick start
51
+
52
+ ```bash
53
+ bundle add active_durable
54
+ bin/rails generate active_durable:install
55
+ bin/rails db:migrate
56
+ ```
57
+
58
+ Write the recipe once, in `app/sagas/`:
59
+
60
+ ```ruby
61
+ # app/sagas/checkout_saga.rb
62
+ CheckoutSaga = Durable.define(:checkout) do |flow, order_id:|
63
+ order = Order.find(order_id)
64
+
65
+ # Touches only your database: committed together with its checkpoint, so it runs exactly once.
66
+ flow.transaction :reserve_stock, undo: -> { order.release_stock! } do
67
+ order.reserve_stock!
68
+ { "reserved" => true }
69
+ end
70
+
71
+ # Talks to the outside world: the ticket is an idempotency key that never changes for this step.
72
+ payment = flow.step :charge, undo: ->(charge, ticket) { Payments.refund(charge, ticket) } do |ticket|
73
+ Payments.charge(order, ticket)
74
+ rescue Stripe::CardError => e
75
+ flow.abort!(e.message) # a declined card is not retried: the stock is released right away
76
+ end
77
+
78
+ # The point of no return: before it failures are undone, after it steps are retried.
79
+ flow.pivot :dispatch do |ticket|
80
+ { "tracking" => Carrier.ship(order.id, reference: ticket).tracking_number, "payment" => payment["id"] }
81
+ end
82
+
83
+ flow.step(:confirmation_email) { OrderMailer.shipped(order.id).deliver_now && true }
84
+ flow.sleep(:wait_for_delivery, 3.days) # no worker is held while it sleeps
85
+ flow.step(:ask_for_review) { ReviewMailer.ask(order.id).deliver_now && true }
86
+ end
87
+ ```
88
+
89
+ The calls to Stripe live in a plain module, doing and undoing side by side. The recipe hands them the ticket:
90
+
91
+ ```ruby
92
+ # app/services/payments.rb
93
+ module Payments
94
+ def self.charge(order, ticket)
95
+ intent = Stripe::PaymentIntent.create(
96
+ { amount: order.total_cents, currency: "usd", customer: order.user.stripe_id, confirm: true },
97
+ { idempotency_key: ticket }
98
+ )
99
+ { "id" => intent.id } # written in the notebook, so it must fit in JSON
100
+ end
101
+
102
+ def self.refund(charge, ticket)
103
+ Stripe::Refund.create({ payment_intent: charge["id"] }, { idempotency_key: ticket })
104
+ end
105
+ end
106
+ ```
107
+
108
+ Start it in the same transaction that creates the order. The saga is saved with the order and the job is enqueued
109
+ only after the commit, so a crash in between cannot lose it:
110
+
111
+ ```ruby
112
+ Order.transaction do
113
+ order = Order.create!(order_params)
114
+ Durable.start(:checkout, order_id: order.id)
115
+ end
116
+ ```
117
+
118
+ To run it for real, with a job backend, the sweeper, the initializer, the dashboard and tests, follow
119
+ [In a Rails app](#in-a-rails-app).
120
+
121
+ > `Durable` is a short alias for `ActiveDurable`. It is skipped if your app already defines a `Durable` constant.
122
+
123
+ ## In a Rails app
124
+
125
+ Everything a Rails app needs, in order. Steps 1 to 5 are done once; step 6 is the code you write for each saga.
126
+
127
+ ### 1. Install
128
+
129
+ Run the three commands from the [quick start](#quick-start). The migration creates `durable_executions`,
130
+ `durable_steps` and `durable_signals` in your **primary database**, next to your models: `flow.transaction` runs
131
+ exactly once only because the notebook and your data commit together. A queue in its own database, like Solid
132
+ Queue's in Rails 8, is fine.
133
+
134
+ ### 2. A job backend
135
+
136
+ ActiveDurable runs on Active Job, so it uses the backend you already have. New Rails 8 apps come with Solid Queue;
137
+ on older apps, `bundle add solid_queue` and `bin/rails solid_queue:install` write these lines. With Sidekiq or
138
+ GoodJob, set their adapter instead.
139
+
140
+ ```ruby
141
+ # config/environments/production.rb
142
+ config.active_job.queue_adapter = :solid_queue
143
+ config.solid_queue.connects_to = { database: { writing: :queue } }
144
+ ```
145
+
146
+ - **Workers must listen to the saga queue.** It is `:default` unless you change `config.queue_name`; if you do, add
147
+ it to your workers (Solid Queue's `config/queue.yml`, Sidekiq's `-q`).
148
+ - **In development** Rails' default `:async` adapter runs jobs inside the server, sleeps and retries included. With
149
+ Solid Queue, run `bin/jobs` next to the server.
150
+
151
+ ### 3. The sweeper
152
+
153
+ The safety net: every minute it enqueues executions that lost their job, because the process died between the
154
+ commit and the enqueue or a worker died holding a lease. With Solid Queue:
155
+
156
+ ```yaml
157
+ # config/recurring.yml
158
+ production:
159
+ active_durable_sweep:
160
+ class: ActiveDurable::SweepJob
161
+ schedule: every minute
162
+ ```
163
+
164
+ With another backend, schedule `ActiveDurable::SweepJob` in its own scheduler (GoodJob cron, sidekiq-cron), or run
165
+ `bin/rails active_durable:sweep` from cron.
166
+
167
+ ### 4. The initializer
168
+
169
+ The settings, with their defaults:
170
+
171
+ ```ruby
172
+ # config/initializers/active_durable.rb
173
+ ActiveDurable.configure do |config|
174
+ config.queue_name = :default # the queue of ActiveDurable::RunJob and SweepJob
175
+ config.lease_duration = 5.minutes # longer than your slowest step
176
+ config.step_attempts = 3 # before the pivot, then undo
177
+ config.after_pivot_attempts = 25 # after the pivot, then block
178
+ config.undo_attempts = 10 # then block
179
+ config.backoff = ->(attempt) { [2**attempt, 3600].min } # or [5, 30, 300], or a number
180
+ config.parallel_concurrency = 4 # threads per flow.parallel
181
+ config.sweep_grace = 1.minute # the sweeper leaves executions this young alone
182
+
183
+ # Who may open the dashboard outside development and test. With Devise (HTTP basic auth: see Dashboard):
184
+ config.dashboard_authorize = ->(controller) { controller.request.env["warden"]&.user&.admin? }
185
+ end
186
+
187
+ # Tell someone when a saga needs a person.
188
+ ActiveSupport::Notifications.subscribe("blocked.active_durable") do |event|
189
+ Sentry.capture_message("Saga blocked", extra: event.payload)
190
+ end
191
+ ```
192
+
193
+ For traces, see [OpenTelemetry](#observability).
194
+
195
+ ### 5. Routes
196
+
197
+ ```ruby
198
+ # config/routes.rb
199
+ mount ActiveDurable::Engine => "/durable"
200
+ ```
201
+
202
+ ### 6. Your code
203
+
204
+ Each recipe lives in `app/sagas/<name>_saga.rb` and is assigned to `<Name>Saga`, so a worker can load
205
+ `:checkout` from `CheckoutSaga` by its name.
206
+
207
+ ```text
208
+ app/
209
+ sagas/checkout_saga.rb the recipe
210
+ services/payments.rb charge and refund, side by side
211
+ services/place_order.rb creates the order and starts the saga
212
+ controllers/orders_controller.rb calls PlaceOrder and answers right away
213
+ ```
214
+
215
+ The controller never calls Stripe. It starts the saga and answers at once; a job runs the steps.
216
+
217
+ ```ruby
218
+ # app/services/place_order.rb
219
+ class PlaceOrder
220
+ def self.call(params)
221
+ Order.transaction do
222
+ order = Order.create!(params)
223
+ Durable.start(:checkout, id: "checkout-#{order.id}", order_id: order.id)
224
+ order
225
+ end
226
+ end
227
+ end
228
+
229
+ # app/controllers/orders_controller.rb
230
+ class OrdersController < ApplicationController
231
+ def create
232
+ redirect_to PlaceOrder.call(order_params)
233
+ end
234
+
235
+ def show
236
+ @order = Order.find(params[:id])
237
+ end
238
+ end
239
+
240
+ # app/models/order.rb
241
+ class Order < ApplicationRecord
242
+ def checkout
243
+ Durable.find("checkout-#{id}")
244
+ end
245
+ end
246
+ ```
247
+
248
+ The `id:` ties the saga to its order, so the order page can show where the saga is:
249
+
250
+ ```erb
251
+ <%# app/views/orders/show.html.erb %>
252
+ <% case @order.checkout.status %>
253
+ <% when "completed" %> Your order is confirmed.
254
+ <% when "compensated" %> We could not complete it and refunded you.
255
+ <% when "blocked" %> We are looking into it.
256
+ <% else %> Processing…
257
+ <% end %>
258
+ ```
259
+
260
+ The customer does not see "card declined" in the same response: the page says "Processing…" and updates itself
261
+ with polling or Turbo Streams. In exchange, nobody is ever charged for half an order.
262
+
263
+ ### 7. The job
264
+
265
+ You do not write one. Once the transaction commits, ActiveDurable enqueues its own `ActiveDurable::RunJob` with the
266
+ execution id, on the Active Job backend you already use (Solid Queue, Sidekiq, GoodJob…). Every time the saga wakes
267
+ up, after a sleep, a retry or a signal, it enqueues that job again. Retries belong to each step and are written in
268
+ the notebook, not to the job: if your backend retries the job too, the copy finds the lease taken and returns.
269
+
270
+ | You want to | Do this |
271
+ | --- | --- |
272
+ | choose the queue | `config.queue_name = :sagas` |
273
+ | set a priority or other job options | the same as for any job, in an initializer: `ActiveDurable::RunJob.queue_with_priority 10` |
274
+ | retry a step more or fewer times | `retry:` on the step, or `config.step_attempts` |
275
+ | stop retrying a business failure | `flow.abort!`, like the declined card above |
276
+ | see what is running | the [dashboard](#dashboard), or `ActiveDurable::RunJob` in your backend's UI |
277
+ | hear about a stuck saga | the `blocked.active_durable` [event](#observability) |
278
+ | run a saga inline in tests | `ActiveDurable::Testing.drain(id)` |
279
+ | start or wake sagas from your own jobs | call `Durable.start` or `Durable.signal` there |
280
+
281
+ > Do not wrap `Durable.start` in a job of your own. The order and its saga would no longer be saved together, and a
282
+ > crash between the two would leave an order without a saga.
283
+
284
+ ### 8. Tests
285
+
286
+ ```ruby
287
+ # spec/rails_helper.rb
288
+ require "active_durable/testing"
289
+
290
+ RSpec.configure do |config|
291
+ config.before { ActiveDurable::Testing.reset! } # forgets simulated time and crash hooks
292
+ end
293
+ ```
294
+
295
+ ```ruby
296
+ # spec/services/place_order_spec.rb
297
+ it "charges once and confirms the order" do
298
+ order = PlaceOrder.call(order_params)
299
+
300
+ expect(ActiveDurable::Testing.drain(order.checkout.id).status).to eq("completed")
301
+ end
302
+ ```
303
+
304
+ `drain` runs the saga right there, without a worker. Stub Stripe as you already do, then let the
305
+ [crash tester](#testing-the-crash-tester) kill the saga at every point.
306
+
307
+ ### 9. Before going to production
308
+
309
+ - [ ] Workers are running and listen to `config.queue_name`.
310
+ - [ ] The sweeper runs every minute.
311
+ - [ ] `dashboard_authorize` is set; without it the dashboard answers 403.
312
+ - [ ] `lease_duration` is longer than your slowest step.
313
+ - [ ] Every step that calls an outside service passes the ticket as its idempotency key.
314
+ - [ ] The crash tester passes for every recipe.
315
+
316
+ ## How it works
317
+
318
+ Every execution has a **notebook**: one row per step, with its status and its result. A worker takes the
319
+ execution with a lease, then runs the recipe from the top. For each step it looks at the notebook first:
320
+
321
+ | The notebook says | The worker |
322
+ | --- | --- |
323
+ | ✔ done | does not run the step and returns the recorded result |
324
+ | nothing yet | runs the step and writes the result down |
325
+ | retrying, failed or waiting | waits, retries, compensates or blocks, as described below |
326
+
327
+ ```mermaid
328
+ stateDiagram-v2
329
+ direction LR
330
+ [*] --> pending: Durable.start
331
+ pending --> running: a worker takes the lease
332
+ running --> sleeping: flow.sleep or a retry
333
+ running --> waiting: flow.wait_for
334
+ sleeping --> running: wake-up time
335
+ waiting --> running: Durable.signal
336
+ running --> completed: every step done
337
+ running --> compensated: failure before the pivot, undos ran
338
+ running --> blocked: needs a person
339
+ blocked --> pending: ActiveDurable.retry
340
+ ```
341
+
342
+ Three rules follow from replaying the recipe:
343
+
344
+ 1. **Everything that changes the world goes inside a step.** Code outside steps runs again on every replay:
345
+ reading is fine; writing, charging or sending is not.
346
+ 2. **Step results are JSON.** They come back from the notebook with string keys, and the first run returns the same
347
+ JSON so it behaves exactly like a replay: `payment["id"]`, never `payment[:id]`.
348
+ 3. **Step names are keys.** Each step needs a unique name within its recipe.
349
+
350
+ ## The building blocks
351
+
352
+ | Call | Use it for | When it fails |
353
+ | --- | --- | --- |
354
+ | `flow.step(name) { \|ticket\| ... }` | anything that talks to the outside world | retried, then the saga is undone |
355
+ | `flow.transaction(name) { ... }` | changes to your own database only (exactly once) | rolled back with its checkpoint, retried, then undone |
356
+ | `flow.pivot(name) { \|ticket\| ... }` | the step after which there is no going back | retried, then the saga is undone |
357
+ | `flow.parallel(name) { \|branches\| ... }` | several steps at the same time | each branch like a step |
358
+ | `flow.sleep(name, 3.days)` | waiting without holding a worker | — |
359
+ | `flow.wait_for(name, timeout:)` | waiting for `Durable.signal` | a timeout fails the saga |
360
+
361
+ Steps take `undo:`, `retry:` (`3`, `false` or `{ attempts:, backoff: }`) and `undo_on_failure:`.
362
+
363
+ <details>
364
+ <summary><b>Tickets: a step that runs twice still has an effect once</b></summary>
365
+
366
+ <br>
367
+
368
+ Each step receives a ticket, `"<execution id>:<step name>"`, that is the same every time the step runs. If a worker
369
+ dies after calling Stripe but before writing the result down, the step runs again; with the ticket as the
370
+ idempotency key, Stripe answers with the first result instead of charging twice. Undos get their own ticket,
371
+ `"...:<step name>:undo"`.
372
+
373
+ </details>
374
+
375
+ <details>
376
+ <summary><b>Undo in reverse</b></summary>
377
+
378
+ <br>
379
+
380
+ When a step runs out of attempts, raises `ActiveDurable::Abort`, or the recipe raises, every finished step is undone,
381
+ last one first. Each undo is written in the notebook too, so a crash in the middle of undoing resumes where it
382
+ stopped. An undo receives `(result, undo_ticket, step_ticket)` and takes as many as it declares: `-> { ... }` takes
383
+ none. Any object that responds to `call` works too, such as `Payments.method(:refund)`.
384
+
385
+ A step that failed is not undone, because it did not happen. The exception is a step whose failure may hide a
386
+ success, like a charge whose answer timed out: declare `undo_on_failure: true` and its undo runs with `nil` as the
387
+ result, so it can look the outcome up with the step's ticket.
388
+
389
+ To take another path instead, rescue the failure in the recipe:
390
+
391
+ ```ruby
392
+ begin
393
+ flow.step(:charge_with_stripe, retry: 2) { |ticket| ... }
394
+ rescue ActiveDurable::StepFailed
395
+ flow.step(:charge_with_paypal) { |ticket| ... }
396
+ end
397
+ ```
398
+
399
+ </details>
400
+
401
+ <details>
402
+ <summary><b>The point of no return</b></summary>
403
+
404
+ <br>
405
+
406
+ A shipped parcel or a wire transfer cannot be undone. Mark that step with `flow.pivot`. Before it, a failure undoes
407
+ everything. After it, steps cannot declare `undo:` and are retried with backoff (`config.after_pivot_attempts`,
408
+ 25 by default); if they still fail, the execution is **blocked** for a person.
409
+
410
+ </details>
411
+
412
+ <details>
413
+ <summary><b>Sleeping and waiting for signals</b></summary>
414
+
415
+ <br>
416
+
417
+ `flow.sleep` writes the wake-up time down and releases the worker. `flow.wait_for` does the same until a signal
418
+ arrives. Signals can arrive before the saga starts waiting.
419
+
420
+ ```ruby
421
+ kyc = flow.wait_for(:kyc_done, timeout: 2.hours)
422
+
423
+ # in the webhook controller
424
+ Durable.signal("loan-42", :kyc_done, verified: true)
425
+ ```
426
+
427
+ Pass `id:` to `Durable.start` to choose the execution id (the call becomes idempotent).
428
+
429
+ </details>
430
+
431
+ <details>
432
+ <summary><b>Parallel branches</b></summary>
433
+
434
+ <br>
435
+
436
+ ```ruby
437
+ reservations = flow.parallel :reserve_stock do |branches|
438
+ order.warehouses.each do |warehouse|
439
+ branches.step warehouse.code, undo: ->(r, ticket) { warehouse.release(r["id"], key: ticket) } do |ticket|
440
+ { "id" => warehouse.reserve(order.items_for(warehouse), key: ticket) }
441
+ end
442
+ end
443
+ end
444
+ reservations # => { "MEX" => { "id" => ... }, "GDL" => { "id" => ... } }
445
+ ```
446
+
447
+ Each branch runs in its own thread (`config.parallel_concurrency`, 4 by default), has its own notebook row
448
+ (`reserve_stock/MEX`), ticket and retries. After a crash only unfinished branches run again; if one fails for good,
449
+ the finished branches are undone in the order they finished. Give your connection pool at least
450
+ `parallel_concurrency + 1` connections.
451
+
452
+ </details>
453
+
454
+ <details>
455
+ <summary><b>Changing a recipe while sagas are in flight</b></summary>
456
+
457
+ <br>
458
+
459
+ If a replay reaches a step the notebook did not record, ActiveDurable blocks that execution with
460
+ `ActiveDurable::RecipeChanged` and names both steps, instead of guessing. To change a recipe safely, keep the old
461
+ one and add a version:
462
+
463
+ ```ruby
464
+ CheckoutSaga = Durable.define(:checkout, version: 2) { |flow, order_id:| ... }
465
+ Durable.define(:checkout, version: 1) { |flow, order_id:| ... } # keep until nothing uses it
466
+ ```
467
+
468
+ New executions use the highest version; each execution keeps the one it started with.
469
+ `bin/rails active_durable:versions` lists the versions unfinished executions still use.
470
+
471
+ </details>
472
+
473
+ ## Dashboard
474
+
475
+ ```ruby
476
+ # config/routes.rb
477
+ mount ActiveDurable::Engine => "/durable"
478
+ ```
479
+
480
+ <table>
481
+ <tr>
482
+ <td width="50%"><img src="docs/assets/dashboard-list.png" alt="Dashboard: executions by status, each drawn as a row of blocks"></td>
483
+ <td width="50%"><img src="docs/assets/dashboard-saga.png" alt="Dashboard: one saga step by step, with the point of no return and a sleeping step"></td>
484
+ </tr>
485
+ <tr>
486
+ <td colspan="2"><img src="docs/assets/dashboard-undone.png" alt="Dashboard: a saga undone in reverse after the dispatch step failed"></td>
487
+ </tr>
488
+ </table>
489
+
490
+ Every saga is drawn as a row of blocks, and every animation means something: a running step beats, a waiting one
491
+ pings like a radar, a sleeping one fills a ring until it wakes, a failed one shakes, undone ones are striped and
492
+ wired backwards. Countdowns are live, and live mode refreshes the list and flashes the rows that changed. It has
493
+ buttons to retry, undo everything or run a saga again from a chosen step.
494
+
495
+ It needs no asset pipeline and works in `rails new --api` apps, with its own session for CSRF protection. Outside
496
+ development and test it stays **closed** until you decide who can open it:
497
+
498
+ ```ruby
499
+ # config/initializers/active_durable.rb
500
+ ActiveDurable.config.dashboard_authorize = lambda do |controller|
501
+ controller.authenticate_or_request_with_http_basic do |user, password|
502
+ ActiveSupport::SecurityUtils.secure_compare(user, ENV.fetch("DURABLE_USER")) &
503
+ ActiveSupport::SecurityUtils.secure_compare(password, ENV.fetch("DURABLE_PASSWORD"))
504
+ end
505
+ end
506
+ ```
507
+
508
+ ## Operating sagas
509
+
510
+ ```ruby
511
+ ActiveDurable.retry("checkout-7") # blocked: try again where it stopped
512
+ ActiveDurable.compensate("checkout-7", reason: "customer cancelled") # undo everything (only before the pivot)
513
+ ActiveDurable.rerun("checkout-7", from: :ship) # a new execution reusing steps before :ship
514
+ ```
515
+
516
+ All three refuse an execution a worker is running right now. A rerun runs the chosen step and the following ones
517
+ again with new tickets, so they have effects again; a blocked original becomes `superseded`.
518
+
519
+ ## Testing: the crash tester
520
+
521
+ ```ruby
522
+ require "active_durable/testing"
523
+
524
+ it "survives a crash at any point" do
525
+ ActiveDurable::Testing.crash_everywhere(:checkout, order_id: order.id) do |execution, point|
526
+ expect(execution.status).to eq("completed")
527
+ expect(FakeStripe.charges.size).to eq(1)
528
+ end
529
+ end
530
+ ```
531
+
532
+ `crash_everywhere` runs the saga once to find every point where a process could die (before each step, after the
533
+ call but before the checkpoint, after the checkpoint, and the same for undos). Then it runs a fresh execution per
534
+ point, kills it right there, and finishes it with a new worker. It is the fastest way to find a step that is not
535
+ idempotent.
536
+
537
+ `ActiveDurable::Testing.drain(id, signals: { name => payload })` runs an execution synchronously, fast-forwarding
538
+ sleeps, retries and expired leases.
539
+
540
+ ## Observability
541
+
542
+ <details>
543
+ <summary><b>Events and OpenTelemetry</b></summary>
544
+
545
+ <br>
546
+
547
+ Subscribe to `blocked.active_durable` to page someone:
548
+
549
+ ```ruby
550
+ ActiveSupport::Notifications.subscribe("blocked.active_durable") do |event|
551
+ Sentry.capture_message("Saga blocked", extra: event.payload)
552
+ end
553
+ ```
554
+
555
+ | Event | Payload |
556
+ | --- | --- |
557
+ | `execution` / `step` / `compensation` / `undo` | `execution_id` (and `recipe`, `step`, `kind`) |
558
+ | `completed` / `compensated` | `execution_id`, `recipe` |
559
+ | `blocked` | `execution_id`, `recipe`, `error` |
560
+ | `retried` / `compensation_requested` / `rerun` | operator actions |
561
+
562
+ With `opentelemetry-sdk` configured, every worker run becomes a span with its steps, undos and compensation nested
563
+ inside, parallel branches included:
564
+
565
+ ```ruby
566
+ require "active_durable/open_telemetry"
567
+ ActiveDurable::OpenTelemetry.install!
568
+ ```
569
+
570
+ </details>
571
+
572
+ The settings are in [the initializer](#4-the-initializer).
573
+
574
+ ## Compatibility
575
+
576
+ Every combination below runs the full test suite in CI, against **PostgreSQL**, **MySQL 8+** and **SQLite 3**.
577
+
578
+ | | Rails 6.1 | Rails 7.0 | Rails 7.1 | Rails 7.2 | Rails 8.0 | Rails 8.1 |
579
+ | --- | :---: | :---: | :---: | :---: | :---: | :---: |
580
+ | **Ruby 3.1** | ✔ | ✔ | ✔ | ✔ | needs Ruby 3.2 | needs Ruby 3.2 |
581
+ | **Ruby 3.2** | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
582
+ | **Ruby 3.3** | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
583
+ | **Ruby 3.4** | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
584
+ | **Ruby 4.0** | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
585
+
586
+ Every feature works on every version. Things your app may need on older Rails, unrelated to ActiveDurable:
587
+
588
+ - **MySQL on Rails 6.1 and 7.0** uses the `mysql2` adapter (`trilogy` ships with Active Record 7.1+).
589
+ - **Rails 6.1 on Ruby 3.4+** needs `base64`, `benchmark`, `bigdecimal`, `drb`, `logger`, `mutex_m`, `observer`
590
+ and `ostruct` in the Gemfile: Rails 6.1 uses them, and Ruby no longer ships them by default.
591
+ - **`unknown keyword: quirks_mode`** comes from some Active Support versions (seen with 7.1 and 8.0) and json 3:
592
+ add `gem "json", "< 3"`.
593
+
594
+ ## How it compares
595
+
596
+ | | Keeps progress in | Undoes steps | Needs |
597
+ | --- | --- | --- | --- |
598
+ | Active Job Continuations (Rails 8.1) | the job (a cursor) | no | nothing extra |
599
+ | ChronoForge | your database | not in its docs | nothing extra |
600
+ | ruby_reactor | Redis | yes | Redis and Sidekiq |
601
+ | Temporal | the Temporal server | written by hand | a Temporal cluster |
602
+ | **ActiveDurable** | **your database** | **yes, in reverse, with a point of no return** | **nothing extra** |
603
+
604
+ ## Guarantees and limits
605
+
606
+ - A step runs **at least once**; with an idempotency key it has its effect once. `flow.transaction` runs exactly
607
+ once, because its change and its checkpoint commit together.
608
+ - One worker at a time per execution: taking an execution and every write are fenced by a lease token.
609
+ - Sagas do not isolate each other: two sagas can see each other's intermediate states.
610
+ - `flow.transaction` is atomic only when the notebook lives in the same database as your data.
611
+
612
+ ## Development
613
+
614
+ ```bash
615
+ bundle install
616
+ bundle exec rspec # PostgreSQL, Rails 8.1
617
+ DB=mysql bundle exec rspec # MySQL
618
+ DB=sqlite3 bundle exec rspec # SQLite
619
+ BUNDLE_GEMFILE=gemfiles/rails-7.1.gemfile bundle exec rspec # any Rails in gemfiles/
620
+ bundle exec rubocop
621
+ bin/demo # the dashboard with sample sagas
622
+ ruby docs/assets/generate.rb # rebuild the animated SVGs of this README
623
+ ```
624
+
625
+ This README exists in two languages: every change goes to `README.md` and `README.es.md` (see `CONTRIBUTING.md`).
626
+ The design notes, in Spanish, are in `docs/`.
627
+
628
+ ## License
629
+
630
+ MIT. See [LICENSE.txt](LICENSE.txt).
data/Rakefile ADDED
@@ -0,0 +1,12 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "bundler/gem_tasks"
4
+ require "rspec/core/rake_task"
5
+
6
+ RSpec::Core::RakeTask.new(:spec)
7
+
8
+ require "rubocop/rake_task"
9
+
10
+ RuboCop::RakeTask.new
11
+
12
+ task default: %i[spec rubocop]
@@ -0,0 +1,25 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ActiveDurable
4
+ # Base controller for the dashboard. It does not inherit from your ApplicationController on purpose:
5
+ # the dashboard must work the same in full and API-only apps.
6
+ class ApplicationController < ActionController::Base
7
+ protect_from_forgery with: :exception
8
+ layout "active_durable/application"
9
+ helper ActiveDurable::DashboardHelper
10
+
11
+ before_action :authorize_dashboard!
12
+
13
+ private
14
+
15
+ def authorize_dashboard!
16
+ rule = ActiveDurable.config.dashboard_authorize
17
+ # Rails.env.local? only exists since Rails 7.1; before that it silently answers false.
18
+ allowed = rule ? rule.call(self) : Rails.env.development? || Rails.env.test?
19
+ return if performed? || allowed
20
+
21
+ render plain: "The ActiveDurable dashboard is closed. Set ActiveDurable.config.dashboard_authorize " \
22
+ "to decide who can open it.", status: :forbidden
23
+ end
24
+ end
25
+ end