ruby_reactor 0.5.2 → 0.5.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 66c0a0c5591cd862dc61063d752f4d92111370808256bfdae749825fec68429b
4
- data.tar.gz: 212ac4ce7ef87e5d28606cab0aff8358afde437389fbb5b2d717b1b5874daa77
3
+ metadata.gz: b41cd34a4e2437fdeead7ad0b9ddeebc493f711e0cc7db8a31ab7c61f3e2be8d
4
+ data.tar.gz: 4179974d1156150a84ca60ef75e04b619dc9a7a2b26fefa5ef7e40beba5c8e14
5
5
  SHA512:
6
- metadata.gz: 75e3bd7ead2281ef7bd1a74fe42a1aaaa1ff5ac92db2b72172e779b0fa9591878268b9829c939def027a6d368ad0ad7b00eddaa05bbb5497f0678229b68b17c0
7
- data.tar.gz: 46c44e73bef1e7f2a11d7a5de51de83a2a9a2505aa8c5b06cf64215bc97c4c272f67f23a220187431a188a6dcca216ee13741566fc61e1ead6a8d7dc7b392d74
6
+ metadata.gz: bc39fd70ce8a3318dbd41cd49deb4e7ebd66c084e7fb007ff5651d26cf1853c97cf32577ffda863c754328a49518b629aa41f1341398c49e43cd4e5df5070439
7
+ data.tar.gz: 83eddedaebfe39c7b3dad5b88eeaab15c08e11d78a8c894473ab82044047d8c392a4387674199fbb80d4542cf1698f64ce95b7448b8a8cf421ef7ca703b1cbb6
@@ -1,3 +1,3 @@
1
1
  {
2
- ".": "0.5.2"
2
+ ".": "0.5.4"
3
3
  }
data/CHANGELOG.md CHANGED
@@ -1,5 +1,19 @@
1
1
  # Changelog
2
2
 
3
+ ## [0.5.4](https://github.com/arturictus/ruby_reactor/compare/v0.5.3...v0.5.4) (2026-06-18)
4
+
5
+
6
+ ### documentation
7
+
8
+ * emphasize class-based steps as preferred way ([#38](https://github.com/arturictus/ruby_reactor/issues/38)) ([0ee6234](https://github.com/arturictus/ruby_reactor/commit/0ee62346fd0c49d97c57cef780a6a7135d4253cd))
9
+
10
+ ## [0.5.3](https://github.com/arturictus/ruby_reactor/compare/v0.5.2...v0.5.3) (2026-06-17)
11
+
12
+
13
+ ### Features
14
+
15
+ * Durability & Recovery ([#39](https://github.com/arturictus/ruby_reactor/issues/39)) ([103e583](https://github.com/arturictus/ruby_reactor/commit/103e5835b413eec2302fa63f3e998d487cfd9eaf))
16
+
3
17
  ## [0.5.2](https://github.com/arturictus/ruby_reactor/compare/v0.5.1...v0.5.2) (2026-06-14)
4
18
 
5
19
 
data/README.md CHANGED
@@ -36,6 +36,7 @@ The key value is **Reliability**: if any part of your workflow fails, Ruby React
36
36
  | Locks / sem / rate / per | Yes | No | No | Manual |
37
37
  | Built-in web dashboard | Yes | No | No | No |
38
38
  | Async with Sidekiq | Yes | No | Limited | Yes |
39
+ | Durable crash recovery | Yes | No | No | Manual |
39
40
 
40
41
  ## Real-World Use Cases
41
42
 
@@ -44,12 +45,14 @@ The key value is **Reliability**: if any part of your workflow fails, Ruby React
44
45
  - **Subscription Billing**: Coordinate Stripe charges, invoice email generation, and internal entitlement updates. Use interrupts to pause the workflow when 3rd-party APIs are required to continue the workflow or when specific customer approval is needed.
45
46
 
46
47
  ## Table of Contents
48
+
47
49
  - [Features](#features)
48
50
  - [Comparison](#comparison)
49
51
  - [Real-World Use Cases](#real-world-use-cases)
50
52
  - [Installation](#installation)
51
53
  - [Configuration](#configuration)
52
54
  - [Quick Start](#quick-start)
55
+ - [Defining Steps](#defining-steps)
53
56
  - [Web Dashboard](#web-dashboard)
54
57
  - [Rails Installation](#rails-installation)
55
58
  - [Usage](#usage)
@@ -57,6 +60,7 @@ The key value is **Reliability**: if any part of your workflow fails, Ruby React
57
60
  - [Async Execution](#async-execution)
58
61
  - [Full Reactor Async](#full-reactor-async)
59
62
  - [Step-Level Async](#step-level-async)
63
+ - [Durability & Recovery](#durability--recovery)
60
64
  - [Interrupts (Pause & Resume)](#interrupts-pause--resume)
61
65
  - [Locks, Semaphores & Ordered Locks](#locks-semaphores--ordered-locks)
62
66
  - [Map & Parallel Execution](#map--parallel-execution)
@@ -90,53 +94,96 @@ Or install it yourself as:
90
94
 
91
95
  ## Configuration
92
96
 
93
- Configure RubyReactor with your Sidekiq and Redis settings:
97
+ Every setting is **optional** — RubyReactor ships with the defaults shown. Drop
98
+ this into an initializer (e.g. `config/initializers/ruby_reactor.rb`); pasted as-is
99
+ it changes nothing, so it doubles as a reference of every knob.
94
100
 
95
- Every setting below is **optional** RubyReactor ships with the defaults shown. Override only what you need.
101
+ > **Reading the block:** lines starting with `##` are documentation. Lines starting
102
+ > with a single `#` (a `config.…` call) are real settings commented at their
103
+ > default — uncomment one to enable it.
96
104
 
97
105
  ```ruby
98
106
  RubyReactor.configure do |config|
99
- # Storage adapter. Default: :redis (the only adapter shipped today).
100
- config.storage.adapter = :redis
101
- # Redis URL. Default: "redis://localhost:6379/0".
107
+ ## === Storage (Redis) ===
108
+
109
+ ## Storage adapter. Default: :redis (the only adapter shipped today).
110
+ # config.storage.adapter = :redis
111
+
112
+ ## Redis URL. Default: "redis://localhost:6379/0".
102
113
  config.storage.redis_url = ENV.fetch("REDIS_URL", "redis://localhost:6379/0")
103
- # Extra options passed to Redis.new. Default: {}.
104
- config.storage.redis_options = { timeout: 1 }
105
-
106
- # Sidekiq queue used by RubyReactor's async worker. Default: :default.
107
- config.sidekiq_queue = :default
108
- # Sidekiq retry count for infrastructure failures only (deserialization,
109
- # Redis, network). Step retries are managed separately. Default: 3.
110
- config.sidekiq_retry_count = 3
111
-
112
- # Lock/semaphore/rate-limit/ordered-lock contention snooze behavior for
113
- # async reactors. When a Sidekiq worker cannot acquire a primitive it
114
- # re-enqueues itself with `lock_snooze_base_delay + rand(0..lock_snooze_jitter)`
115
- # seconds (rate-limit uses a precise `retry_after_seconds` hint from the error;
116
- # ordered-lock waits re-poll at the base delay so a successor catches its
117
- # blocker finishing fast), up to `lock_snooze_max_attempts` times before
118
- # marking the context :failed. Defaults: 5 / 5 / 20. Set max_attempts to
119
- # :infinity to never give up.
120
- config.lock_snooze_base_delay = 5
121
- config.lock_snooze_jitter = 5
122
- config.lock_snooze_max_attempts = 20
123
-
124
- # Named rate limits shared across reactors. Reference them with
125
- # `with_rate_limit(:stripe)`. See Locks, Semaphores, Rate Limits & Periods.
126
- config.rate_limits.register(:stripe, limits: { second: 3, minute: 100 })
127
114
 
128
- # Logger. Default: Logger.new($stderr).
129
- config.logger = Logger.new($stdout)
115
+ ## Extra options passed to Redis.new. Default: {}.
116
+ # config.storage.redis_options = { timeout: 1 }
117
+
118
+ ## === Sidekiq ===
119
+
120
+ ## Sidekiq queue used by RubyReactor's async worker. Default: :default.
121
+ # config.sidekiq_queue = :default
122
+
123
+ ## Sidekiq retry count for infrastructure failures only (deserialization,
124
+ ## Redis, network). Step retries are managed separately. Default: 3.
125
+ # config.sidekiq_retry_count = 3
126
+
127
+ ## === Contention snooze (locks / semaphores / rate limits / ordered locks) ===
128
+
129
+ ## When a Sidekiq worker cannot acquire a primitive it re-enqueues itself with
130
+ ## `lock_snooze_base_delay + rand(0..lock_snooze_jitter)` seconds (rate-limit
131
+ ## uses a precise `retry_after_seconds` hint from the error; ordered-lock waits
132
+ ## re-poll at the base delay so a successor catches its blocker finishing fast),
133
+ ## up to `lock_snooze_max_attempts` times before marking the context :failed.
134
+ ## Set max_attempts to :infinity to never give up.
135
+ # config.lock_snooze_base_delay = 5
136
+ # config.lock_snooze_jitter = 5
137
+ # config.lock_snooze_max_attempts = 20
138
+
139
+ ## === Durability & crash recovery (see "Durability & Recovery" below) ===
140
+
141
+ ## Retention TTL (seconds) for stored reactor/map state. Must exceed your
142
+ ## worst-case snooze/retry window; re-stamped on every write. Default: 86_400.
143
+ # config.context_ttl = 86_400
144
+
145
+ ## TTL (seconds) for the per-context liveness lock. A live worker auto-extends
146
+ ## it; its absence is the sweeper's "worker died" signal. Must exceed the
147
+ ## longest a single step can run without yielding the GIL. Default: 60.
148
+ # config.context_lock_ttl = 60
149
+
150
+ ## Minimum seconds between per-step checkpoints within one run. 0 = checkpoint
151
+ ## after every step (strongest guarantee). Raise to coalesce mid-run writes for
152
+ ## long reactors — only safe when steps are idempotent. Default: 0.
153
+ # config.checkpoint_min_interval = 0
130
154
 
131
- # Async router. Default: RubyReactor::SidekiqAdapter. Swap for a custom
132
- # adapter if you don't use Sidekiq — the adapter only needs to respond to
133
- # `perform_async(serialized_context, reactor_class_name, **)`.
155
+ ## Recovery sweeper (the chain is kicked once by `RubyReactor.start_sweeper!`).
156
+ # config.sweeper_enabled = true # run recovery by default
157
+ # config.sweeper_interval = 30 # seconds between sweeps = recovery-latency bound
158
+ # config.sweeper_limit = 1000 # max contexts/maps inspected per sweep
159
+
160
+ ## === Misc ===
161
+
162
+ ## Logger. Default: Logger.new($stdout).
163
+ # config.logger = Logger.new($stdout)
164
+
165
+ ## Async router. Default: RubyReactor::SidekiqAdapter. Swap for a custom adapter
166
+ ## if you don't use Sidekiq — it only needs to respond to
167
+ ## `perform_async(context_id, reactor_class_name, **)`.
134
168
  # config.async_router = MyCustomAdapter
169
+
170
+ ## === Examples (no default — set these to use the feature) ===
171
+
172
+ ## Named rate limits shared across reactors. Reference with `with_rate_limit(:stripe)`.
173
+ # config.rate_limits.register(:stripe, limits: { second: 3, minute: 100 })
174
+
175
+ ## OpenTelemetry / custom middlewares. Default: [].
176
+ # config.middlewares = [RubyReactor::OpenTelemetry]
135
177
  end
136
178
  ```
137
179
 
138
180
  You can also leave out the `configure` block entirely — defaults work for local development against a Redis on `localhost:6379`.
139
181
 
182
+ > **Crash recovery needs a kick.** The `sweeper_*` settings above only configure
183
+ > the recovery sweeper — they do not start it. Call `RubyReactor.start_sweeper!`
184
+ > once at boot (ideally from a Sidekiq `on(:startup)` hook) or no crashed reactor
185
+ > will ever resume. See [Durability & Recovery](#durability--recovery).
186
+
140
187
 
141
188
  ## Quick Start
142
189
 
@@ -152,6 +199,60 @@ result = HelloReactor.run
152
199
  puts result.value # => "Hello from Ruby Reactor!"
153
200
  ```
154
201
 
202
+ > **Note:** Examples in this README use inline `step` blocks where a step is trivial. For production workflows, prefer [class-based steps](#defining-steps).
203
+
204
+ ## Defining Steps
205
+
206
+ RubyReactor supports two ways to define step logic:
207
+
208
+ | Style | Best for |
209
+ |-------|----------|
210
+ | **Class steps** (preferred) | Real business logic, compensation/undo, shared steps, testability |
211
+ | **Inline blocks** | Quick prototypes, trivial one-liners, documentation examples |
212
+
213
+ Whichever style you use, a step's `run` returns one of three signals — all exposed as bare helpers in both class steps and inline blocks:
214
+
215
+ - **`Success(value)`** — step succeeded; `value` flows to dependent steps.
216
+ - **`Failure(error)`** — step failed; the reactor rolls back completed steps (compensate/undo).
217
+ - **`Skipped(reason:)`** — clean halt: stop the reactor, keep partial progress, **no rollback**. See [Skipping a reactor cleanly](documentation/core_concepts.md#skipping-a-reactor-cleanly).
218
+
219
+ **Class steps** are plain Ruby classes that include `RubyReactor::Step` and implement `run`, and optionally `compensate` and `undo`:
220
+
221
+ ```ruby
222
+ class ReserveInventoryStep
223
+ include RubyReactor::Step
224
+
225
+ def self.run(arguments, context)
226
+ reservation_id = InventoryService.reserve(arguments[:order][:items])
227
+ Success(reservation_id: reservation_id)
228
+ end
229
+
230
+ def self.compensate(error, arguments, context)
231
+ InventoryService.release_partial(arguments[:order][:items])
232
+ Success()
233
+ end
234
+
235
+ def self.undo(result, arguments, context)
236
+ InventoryService.release(result[:reservation_id])
237
+ Success()
238
+ end
239
+ end
240
+
241
+ class OrderProcessingReactor < RubyReactor::Reactor
242
+ step :reserve_inventory, ReserveInventoryStep do
243
+ argument :order, result(:validate_order)
244
+ end
245
+ end
246
+ ```
247
+
248
+ **Why prefer class steps?**
249
+
250
+ - **Testability** — unit-test `run`, `compensate`, and `undo` in isolation without booting the whole reactor
251
+ - **Composability** — share the same step class across multiple reactors and compose larger workflows from small, focused units
252
+ - **Readability** — reactor files stay orchestration-only; business logic lives in named classes instead of growing inline blocks
253
+
254
+ See [Core Concepts — Step Classes](documentation/core_concepts.md#step-classes-preferred) for the full reference. Usage examples below mix class and inline steps — inline where the logic is trivial.
255
+
155
256
  ## Web Dashboard
156
257
 
157
258
  RubyReactor ships with a built-in web dashboard to inspect reactor executions, view logs, and retry failed steps. The dashboard is a Rack app (a [Roda](https://roda.jeremyevans.net/) application) bundled inside the gem with its pre-compiled JS/CSS assets — no extra install or asset build step is required.
@@ -191,60 +292,59 @@ You can secure the dashboard using standard Rails authentication methods (e.g.,
191
292
 
192
293
  ## Usage
193
294
 
194
- RubyReactor allows you to define complex workflows as "reactors" with steps that can depend on each other, handle failures with compensations, and validate inputs.
295
+ RubyReactor allows you to define complex workflows as "reactors" with steps that can depend on each other, handle failures with compensations, and validate inputs. Examples in this section mix class steps with inline blocks; see [Defining Steps](#defining-steps) for guidance.
195
296
 
196
297
  ### Basic Example: User Registration
197
298
 
198
299
  ```ruby
199
300
  require 'ruby_reactor'
200
301
 
302
+ class ValidateEmailStep
303
+ include RubyReactor::Step
304
+
305
+ def self.run(arguments, _context)
306
+ email = arguments[:email]
307
+ email&.include?('@') ? Success(email.strip) : Failure("Email must contain @")
308
+ end
309
+ end
310
+
311
+ class CreateUserStep
312
+ include RubyReactor::Step
313
+
314
+ def self.run(arguments, _context)
315
+ Success(
316
+ id: rand(10000),
317
+ email: arguments[:email],
318
+ password_hash: arguments[:password_hash],
319
+ created_at: Time.now
320
+ )
321
+ end
322
+
323
+ def self.compensate(_error, arguments, _context)
324
+ Notify.to(arguments[:email])
325
+ Success()
326
+ end
327
+ end
328
+
201
329
  class UserRegistrationReactor < RubyReactor::Reactor
202
- # Define inputs with optional validation
203
330
  input :email
204
331
  input :password
205
332
 
206
- # Define steps with their dependencies
207
- step :validate_email do
333
+ step :validate_email, ValidateEmailStep do
208
334
  argument :email, input(:email)
209
-
210
- run do |args, context|
211
- if args[:email] && args[:email].include?('@')
212
- Success(args[:email].strip)
213
- else
214
- Failure("Email must contain @")
215
- end
216
- end
217
335
  end
218
336
 
219
337
  step :hash_password do
220
338
  argument :password, input(:password)
221
-
222
- run do |args, context|
339
+ run do |args, _context|
223
340
  require 'digest'
224
- hashed = Digest::SHA256.hexdigest(args[:password])
225
- Success(hashed)
341
+ Success(Digest::SHA256.hexdigest(args[:password]))
226
342
  end
227
343
  end
228
344
 
229
- step :create_user do
230
- # Arguments can reference results from other steps
345
+ step :create_user, CreateUserStep do
231
346
  argument :email, result(:validate_email)
232
347
  argument :password_hash, result(:hash_password)
233
-
234
- run do |args, context|
235
- user = {
236
- id: rand(10000),
237
- email: args[:email],
238
- password_hash: args[:password_hash],
239
- created_at: Time.now
240
- }
241
- Success(user)
242
- end
243
-
244
- compensate do |error, args, context|
245
- Notify.to(args[:email])
246
- Success()
247
- end
248
348
  end
249
349
 
250
350
  step :notify_user do
@@ -256,12 +356,12 @@ class UserRegistrationReactor < RubyReactor::Reactor
256
356
  Success()
257
357
  end
258
358
 
259
- compensate do |error, args, context|
359
+ compensate do |_error, args, _context|
260
360
  Email.send("support@acme.com", "Email verification for #{args[:email]} couldn't be sent")
261
361
  Success()
262
362
  end
263
363
  end
264
- # Specify which step's result to return
364
+
265
365
  returns :create_user
266
366
  end
267
367
 
@@ -341,6 +441,73 @@ def create(params)
341
441
  end
342
442
  ```
343
443
 
444
+ ### Durability & Recovery
445
+
446
+ Async reactors are durable: state lives in Redis, not in the job payload. Before
447
+ any background job is enqueued the root context is persisted, and after every
448
+ completed step a checkpoint advances the stored blob — so a crash re-runs at most
449
+ one step, never the whole reactor. Each running reactor also holds a short
450
+ **liveness lock** that a live worker auto-extends; its absence is how a dead
451
+ worker is detected.
452
+
453
+ **Recovery is not automatic until you start the sweeper.** A crashed worker's
454
+ reactor only resumes when the recovery sweeper notices the lapsed liveness lock
455
+ and re-enqueues it. The sweeper is a self-rescheduling chain — **kick it once per
456
+ process boot:**
457
+
458
+ The recommended spot is a Sidekiq server startup hook, so only the worker
459
+ process runs recovery (not your web/console/client processes):
460
+
461
+ ```ruby
462
+ # config/initializers/sidekiq.rb
463
+ Sidekiq.configure_server do |config|
464
+ config.on(:startup) { RubyReactor.start_sweeper! }
465
+ end
466
+ ```
467
+
468
+ Anywhere that runs once at boot works too — e.g. a Rails initializer:
469
+
470
+ ```ruby
471
+ # config/initializers/ruby_reactor.rb
472
+ RubyReactor.start_sweeper!
473
+ ```
474
+
475
+ That's all that's required: `start_sweeper!` is idempotent (safe to call on every
476
+ boot — duplicate kicks collapse to one chain), runs both the top-level reactor
477
+ sweeper and the map sweeper every `config.sweeper_interval` seconds, and stops if
478
+ you set `config.sweeper_enabled = false`. The interval is your recovery-latency
479
+ bound.
480
+
481
+ > **Sidekiq Enterprise `super_fetch` compatibility:** the chain is safe under
482
+ > reliable fetch. `super_fetch` re-runs a job whose worker died mid-execution, so
483
+ > a tick that crashes *after* enqueuing its successor but *before* acking would,
484
+ > with naive single-flight, be recovered alongside that successor and fork the
485
+ > chain (doubling every interval). RubyReactor avoids this: it never relies on
486
+ > "one job in the chain" — each next tick is claimed by a per-time-window lock, so
487
+ > a `super_fetch`-recovered tick computes the same window, loses the claim, and
488
+ > collapses back to a single successor. The startup hook above is likewise
489
+ > idempotent across multiple `super_fetch` server processes.
490
+
491
+ **Prefer your own scheduler?** Set `config.sweeper_enabled = false` (which makes
492
+ `start_sweeper!` a no-op) and drive recovery from cron, a Kubernetes `CronJob`,
493
+ `sidekiq-cron`, `sidekiq-scheduler`, or Rails recurring tasks. Each tick is one
494
+ call:
495
+
496
+ ```ruby
497
+ RubyReactor.sweep_once # => { reactors: <n re-enqueued>, maps: <n recovered> }
498
+ ```
499
+
500
+ For example, a rake task a system cron / CronJob can invoke:
501
+
502
+ ```ruby
503
+ # lib/tasks/ruby_reactor.rake
504
+ namespace :ruby_reactor do
505
+ task sweep: :environment do
506
+ RubyReactor.sweep_once
507
+ end
508
+ end
509
+ ```
510
+
344
511
  ### Interrupts (Pause & Resume)
345
512
 
346
513
  Pause execution to wait for external events like webhooks or user approvals.
@@ -500,14 +667,14 @@ result.success? # true (Skipped is a Success subclass)
500
667
  result.skipped? # true on dedup hit, false otherwise
501
668
  ```
502
669
 
503
- A step's `run` block can also return `RubyReactor.Skipped(reason: "...")` to halt the reactor cleanly — remaining steps don't execute, **and already-completed steps are NOT compensated**. Use it when the rest of the workflow is unnecessary and partial progress should be kept (`Failure` is for "stop and roll back").
670
+ A step's `run` block can also return `Skipped(reason: "...")` to halt the reactor cleanly — remaining steps don't execute, **and already-completed steps are NOT compensated**. Use it when the rest of the workflow is unnecessary and partial progress should be kept (`Failure` is for "stop and roll back"). `Skipped` is a bare helper just like `Success`/`Failure` (or use the fully-qualified `RubyReactor.Skipped(...)`).
504
671
 
505
672
  ```ruby
506
673
  step :ensure_active do
507
674
  argument :user, result(:fetch_user)
508
- run do |args|
509
- next RubyReactor.Skipped(reason: "user_opted_out") if args[:user].opted_out?
510
- RubyReactor.Success(args[:user])
675
+ run do |args, _ctx|
676
+ next Skipped(reason: "user_opted_out") if args[:user].opted_out?
677
+ Success(args[:user])
511
678
  end
512
679
  end
513
680
  ```
@@ -976,7 +1143,7 @@ end
976
1143
 
977
1144
  ### Testing
978
1145
 
979
- RubyReactor provides testing utilities for RSpec. See the [Testing with RSpec](documentation/testing.md) guide for comprehensive documentation.
1146
+ RubyReactor provides testing utilities for RSpec. See the [Testing with RSpec](documentation/testing.md) guide for comprehensive documentation — including [unit-testing class-based steps](documentation/testing.md#testing-step-classes) directly.
980
1147
 
981
1148
  ```ruby
982
1149
  RSpec.describe PaymentReactor do
@@ -1003,7 +1170,7 @@ end
1003
1170
  For detailed documentation, see the following guides:
1004
1171
 
1005
1172
  ### [Core Concepts](documentation/core_concepts.md)
1006
- Learn about the fundamental building blocks of RubyReactor: Reactors, Steps, Context, and Results. Understand how steps are defined, how data flows between them, and how the context maintains state throughout execution.
1173
+ Learn about the fundamental building blocks of RubyReactor: Reactors, Steps, Context, and Results. Covers class-based steps (the preferred approach) and inline blocks, how data flows between steps, and how the context maintains state throughout execution.
1007
1174
 
1008
1175
  ### [DAG (Directed Acyclic Graph)](documentation/DAG.md)
1009
1176
  Deep dive into how RubyReactor manages dependencies. This guide explains how the Directed Acyclic Graph is constructed to ensure steps execute in the correct topological order, enabling automatic parallelization of independent steps.
@@ -9,12 +9,76 @@ module RubyReactor
9
9
 
10
10
  attr_writer :sidekiq_queue, :sidekiq_retry_count, :logger, :async_router,
11
11
  :lock_snooze_base_delay, :lock_snooze_jitter, :lock_snooze_max_attempts,
12
- :middlewares
12
+ :middlewares, :context_ttl, :context_lock_ttl, :checkpoint_min_interval,
13
+ :sweeper_enabled, :sweeper_interval, :sweeper_limit
13
14
 
14
15
  def sidekiq_queue
15
16
  @sidekiq_queue ||= :default
16
17
  end
17
18
 
19
+ # Retention TTL (seconds) for a stored reactor context. Storage is
20
+ # load-bearing for resume, so this must comfortably exceed the worst-case
21
+ # snooze/retry window. Refreshed on every checkpoint write.
22
+ def context_ttl
23
+ @context_ttl ||= 86_400
24
+ end
25
+
26
+ # Minimum wall-clock seconds between two PER-STEP durable checkpoints within a
27
+ # single worker run. The save-per-step checkpoint (`on_step_complete`) bounds
28
+ # crash re-execution to one step, but re-serializes and re-writes the WHOLE
29
+ # root blob after every Success — O(steps × context_size) writes for a long,
30
+ # large reactor. This throttle coalesces the mid-run intermediate checkpoints:
31
+ # a checkpoint is written only if at least this many seconds have elapsed since
32
+ # the last one. The final terminal/handoff state is ALWAYS persisted (by the
33
+ # run's ensure-save and the pre-enqueue checkpoint), so throttling only affects
34
+ # mid-run granularity. Tradeoff: with interval > 0, a crash may re-run every
35
+ # step completed inside the last interval — safe only when those steps are
36
+ # idempotent or side-effect-free.
37
+ #
38
+ # Default 0 -> checkpoint after EVERY step (strongest guarantee, no coalescing).
39
+ def checkpoint_min_interval
40
+ @checkpoint_min_interval ||= 0
41
+ end
42
+
43
+ # Whether the recovery sweepers run. The host kicks the self-rescheduling
44
+ # chain once (`RubyReactor.start_sweeper!`, e.g. from an initializer); each
45
+ # tick re-checks this flag, so flipping it to false stops the chain at the
46
+ # next tick. Default on: durability is inert without a running sweeper, so
47
+ # recovery must work out of the box.
48
+ def sweeper_enabled
49
+ @sweeper_enabled = true if @sweeper_enabled.nil?
50
+ @sweeper_enabled
51
+ end
52
+
53
+ # Seconds between sweeps. This is the upper bound on recovery latency for a
54
+ # dead worker — lower it for faster recovery, raise it to cut scan load.
55
+ def sweeper_interval
56
+ @sweeper_interval ||= 30
57
+ end
58
+
59
+ # Max contexts/maps inspected per sweep (passed to each sweeper's run_once).
60
+ def sweeper_limit
61
+ @sweeper_limit ||= 1000
62
+ end
63
+
64
+ # TTL (seconds) for the per-context liveness lock (`async:<id>`). Short by
65
+ # design — it is a liveness signal, not retention. A live worker auto-extends
66
+ # it (every ttl/3 s, from a background thread); its absence is the sweeper's
67
+ # "worker died" signal.
68
+ #
69
+ # SAFETY CONSTRAINT: this MUST exceed the longest a single step can run
70
+ # WITHOUT letting the auto-extend thread make progress. Under MRI the
71
+ # extender shares the GIL, so a step that holds the GIL continuously for
72
+ # longer than this TTL (a long CPU-bound pure-Ruby loop, a C extension that
73
+ # never releases the GIL, or a stop-the-world GC pause) lets the lock lapse.
74
+ # A lapsed lock looks "dead" to the sweeper, which may re-enqueue a duplicate
75
+ # that runs CONCURRENTLY with the still-live original — a double-run. I/O-bound
76
+ # steps release the GIL and keep the lock fresh, so the default 60s suits
77
+ # typical workloads; raise it if you run long synchronous CPU-bound steps.
78
+ def context_lock_ttl
79
+ @context_lock_ttl ||= 60
80
+ end
81
+
18
82
  def sidekiq_retry_count
19
83
  @sidekiq_retry_count ||= 3
20
84
  end
@@ -36,7 +100,7 @@ module RubyReactor
36
100
  end
37
101
 
38
102
  def logger
39
- @logger ||= Logger.new($stderr)
103
+ @logger ||= Logger.new($stdout)
40
104
  end
41
105
 
42
106
  def async_router
@@ -19,13 +19,18 @@ module RubyReactor
19
19
 
20
20
  def deserialize(serialized_data)
21
21
  decompressed = decompress_if_needed(serialized_data)
22
- data = JSON.parse(decompressed, symbolize_names: false)
22
+ deserialize_hash(JSON.parse(decompressed, symbolize_names: false))
23
+ rescue JSON::ParserError => e
24
+ raise RubyReactor::Error::DeserializationError, "Failed to parse serialized context: #{e.message}"
25
+ end
23
26
 
27
+ # Deserialize from an already-parsed Hash (e.g. what the storage adapter's
28
+ # `retrieve_context` returns). Lets the rehydrate-by-id worker path avoid a
29
+ # second JSON parse while still schema-validating. Schema validation lives
30
+ # here so both the string and Hash entry points enforce it.
31
+ def deserialize_hash(data)
24
32
  validate_schema_version(data)
25
-
26
33
  Context.deserialize_from_retry(data)
27
- rescue JSON::ParserError => e
28
- raise RubyReactor::Error::DeserializationError, "Failed to parse serialized context: #{e.message}"
29
34
  end
30
35
 
31
36
  # rubocop:disable Metrics/CyclomaticComplexity, Metrics/MethodLength
@@ -19,7 +19,7 @@ module RubyReactor
19
19
  RubyReactor::Template::Element.new(map_name, path)
20
20
  end
21
21
 
22
- # Make Success and Failure available in DSL contexts
22
+ # Make Success, Failure, and Skipped available in DSL contexts
23
23
  # rubocop:disable Naming/MethodName
24
24
  def Success(value = nil)
25
25
  # rubocop:enable Naming/MethodName
@@ -31,6 +31,12 @@ module RubyReactor
31
31
  # rubocop:enable Naming/MethodName
32
32
  RubyReactor.Failure(error)
33
33
  end
34
+
35
+ # rubocop:disable Naming/MethodName
36
+ def Skipped(reason: nil, **kwargs)
37
+ # rubocop:enable Naming/MethodName
38
+ RubyReactor.Skipped(reason: reason, **kwargs)
39
+ end
34
40
  end
35
41
  end
36
42
  end
@@ -180,7 +180,7 @@ module RubyReactor
180
180
  end
181
181
 
182
182
  def stored_context_status
183
- reactor_class_name = @reactor_class.name || "AnonymousReactor-#{@reactor_class.object_id}"
183
+ reactor_class_name = RubyReactor.reactor_storage_name(@reactor_class)
184
184
  data = RubyReactor.configuration.storage_adapter.retrieve_context(@context.context_id, reactor_class_name)
185
185
  return nil unless data
186
186
 
@@ -48,7 +48,7 @@ module RubyReactor
48
48
  @context.root_context || @context
49
49
  end
50
50
 
51
- reactor_class_name = context_to_serialize.reactor_class.name
51
+ reactor_class_name = RubyReactor.reactor_storage_name(context_to_serialize.reactor_class)
52
52
 
53
53
  @middlewares.on(:before_async_enqueue, context_to_serialize)
54
54
 
@@ -72,7 +72,12 @@ module RubyReactor
72
72
  fail_fast: map_args[:fail_fast]
73
73
  )
74
74
  else
75
- configuration.async_router.perform_in(delay, serialized_context, reactor_class_name)
75
+ # Persist BEFORE enqueue — the job payload is identity-only (F2). The
76
+ # rescheduled job rehydrates the root by id from storage.
77
+ configuration.storage_adapter.store_context(
78
+ context_to_serialize.context_id, serialized_context, reactor_class_name
79
+ )
80
+ configuration.async_router.perform_in(delay, context_to_serialize.context_id, reactor_class_name)
76
81
  end
77
82
  end
78
83