ruby_reactor 0.8.6 → 0.8.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (47) hide show
  1. checksums.yaml +4 -4
  2. data/.release-please-manifest.json +1 -1
  3. data/.specify/feature.json +1 -1
  4. data/CHANGELOG.md +81 -0
  5. data/CLAUDE.md +1 -1
  6. data/README.md +42 -3
  7. data/lib/ruby_reactor/dsl/map_builder.rb +20 -1
  8. data/lib/ruby_reactor/error/recorded_failure.rb +18 -0
  9. data/lib/ruby_reactor/executor/compensation_manager.rb +27 -1
  10. data/lib/ruby_reactor/executor/result_handler.rb +16 -0
  11. data/lib/ruby_reactor/executor/retry_manager.rb +2 -2
  12. data/lib/ruby_reactor/executor/step_executor.rb +20 -0
  13. data/lib/ruby_reactor/executor.rb +62 -10
  14. data/lib/ruby_reactor/interrupt_claims.rb +81 -0
  15. data/lib/ruby_reactor/map/step_rollback.rb +121 -0
  16. data/lib/ruby_reactor/reactor.rb +270 -105
  17. data/lib/ruby_reactor/rspec/matchers.rb +64 -16
  18. data/lib/ruby_reactor/rspec/test_subject.rb +25 -24
  19. data/lib/ruby_reactor/step/compose_step.rb +10 -0
  20. data/lib/ruby_reactor/step/map_step.rb +12 -0
  21. data/lib/ruby_reactor/storage/adapter.rb +17 -0
  22. data/lib/ruby_reactor/storage/redis_adapter.rb +42 -0
  23. data/lib/ruby_reactor/version.rb +1 -1
  24. data/lib/ruby_reactor/web/api.rb +17 -0
  25. data/lib/ruby_reactor/web/public/assets/{index-C6ejszCQ.js → index-DxjQlW7Q.js} +8 -8
  26. data/lib/ruby_reactor/web/public/index.html +1 -1
  27. data/lib/ruby_reactor/worker.rb +73 -6
  28. data/specs/010-interrupt-in-composed-child/checklists/requirements.md +43 -0
  29. data/specs/010-interrupt-in-composed-child/contracts/api-surface.md +75 -0
  30. data/specs/010-interrupt-in-composed-child/data-model.md +40 -0
  31. data/specs/010-interrupt-in-composed-child/plan.md +149 -0
  32. data/specs/010-interrupt-in-composed-child/quickstart.md +64 -0
  33. data/specs/010-interrupt-in-composed-child/research.md +207 -0
  34. data/specs/010-interrupt-in-composed-child/spec.md +267 -0
  35. data/specs/010-interrupt-in-composed-child/tasks.md +315 -0
  36. data/specs/010-rollback-follow-ups/checklists/requirements.md +47 -0
  37. data/specs/010-rollback-follow-ups/contracts/api-surface.md +123 -0
  38. data/specs/010-rollback-follow-ups/contracts/resume-protocol.md +132 -0
  39. data/specs/010-rollback-follow-ups/data-model.md +139 -0
  40. data/specs/010-rollback-follow-ups/plan.md +273 -0
  41. data/specs/010-rollback-follow-ups/pr-description.md +110 -0
  42. data/specs/010-rollback-follow-ups/quickstart.md +157 -0
  43. data/specs/010-rollback-follow-ups/research.md +440 -0
  44. data/specs/010-rollback-follow-ups/spec.md +489 -0
  45. data/specs/010-rollback-follow-ups/tasks.md +1022 -0
  46. data/specs/future_improvements.md +17 -14
  47. metadata +22 -2
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: c59b0af2a9d34b620e3400cff78e0df3b2e580d1138b37ed5aee1b9e5a78f548
4
- data.tar.gz: 4e3c2b92b6ae58797beae470b60c69015e15b2b8d6855ce710e71a999d0ae62f
3
+ metadata.gz: 1d262a9eed4af540e9573901be72e882f2c5e013550d51881dccef9205c6e873
4
+ data.tar.gz: 7a499dd2203c8059bccf9638b621974784593cabe54fddbb88e16036b7c634c5
5
5
  SHA512:
6
- metadata.gz: a3ac367241909b09a5f0451b9278489776e7bedc56b506043c786892b86247a646b297a7c6192be6e5935b5e9a861f36e66e2f370a7c3e0741820cbcaa1ee409
7
- data.tar.gz: 1fd3ff1f692008747021d11d5356613163a8c7777e33ead22260da4b3959bc5f6ec7fff3cb5928444815298011ff4b0e1e2e10e34c55a9781b98d9ab80893e40
6
+ metadata.gz: 578c3ceef421e1718c8ec48e746feae08e226e085c235b6594ac881ce0aedcf0fea8e684508c966584a6bc803bfa11a4fbf24a0666b065285f1161b8b12505dd
7
+ data.tar.gz: 6ee564153b8a3ce213fddf09e91df37be6b967b5a284c5cd0cdbbc4bca8b68ae4a5b3b064c314492f6f53b1bfde4ae29173cef2439886064d10c742c677cc128
@@ -1,3 +1,3 @@
1
1
  {
2
- ".": "0.8.6"
2
+ ".": "0.8.8"
3
3
  }
@@ -1,3 +1,3 @@
1
1
  {
2
- "feature_directory": "specs/009-distributed-map-undo"
2
+ "feature_directory": "specs/010-rollback-follow-ups"
3
3
  }
data/CHANGELOG.md CHANGED
@@ -239,8 +239,57 @@ Every breaking or shape-changing item of the rollback work, with what to change.
239
239
  9. **`rollback_failures` entries may carry `map_step:` / `element_index:`** and the reasons
240
240
  `:context_unavailable` / `:element_in_flight` (additive).
241
241
 
242
+ ### Migration notes: rollback and resume follow-ups
243
+
244
+ 1. **A resume that meets the reactor's held `with_lock` or `with_semaphore` no longer raises.**
245
+ `continue` used to raise `Lock::AcquisitionError` / `Semaphore::AcquisitionError` and leave
246
+ the run paused. It now validates the payload, accepts the resume, and returns a
247
+ `RubyReactor::DispatchResult`; a worker waits for the lock and finishes the run. A `rescue` of
248
+ those errors around `continue` simply stops firing. An invalid payload is still answered at
249
+ once, as before.
250
+ 2. **A second resume of the same interrupt raises `ValidationError`** ("Cannot resume: interrupt
251
+ :x was already resumed"), in every execution mode, including `Sidekiq::Testing.inline!`.
252
+ 3. **A resume for another ready interrupt of a `running` run is accepted** (it used to raise "the
253
+ reactor is running") and returns a `DispatchResult`.
254
+ 4. **Interrupt attempt counts restart** for runs paused across the upgrade: they move out of the
255
+ run's context into their own record, so such a run may take up to `max_attempts - 1` extra
256
+ invalid payloads.
257
+ 5. **A synchronous `Reactor.run` holds the run's liveness lock** while it executes. A
258
+ `Reactor.undo(id)` of a run still executing in its caller's process now waits for it, then
259
+ raises `Lock::AcquisitionError`, as it does for a run executing in a worker.
260
+
242
261
  ### Features
243
262
 
263
+ * **An `interrupt` inside a composed child pauses the top-level run.** Before, the compose step
264
+ failed with `NoMethodError` and the run rolled back. Now the top-level run is stored `paused`, at
265
+ any compose depth and after a `fan_out` map in the child, and the paused result carries its id.
266
+ Resume it on the top-level class by the interrupt's step path, an Array of the compose step names
267
+ then the interrupt (`continue(id:, payload:, step_name: [:approval, :wait_for_manager])`), by id
268
+ or by the child interrupt's correlation id. Payload validation, `max_attempts`,
269
+ `resume: :background` and the resume guards apply as for a top-level interrupt. A resume
270
+ contended on the child's own `with_lock`/`with_semaphore` fails and rolls back the run. Contention
271
+ on the top-level reactor's lock hands the resume to a worker. `Reactor.undo`
272
+ of the paused run rolls back the child's completed steps, then the parent's. A composed child
273
+ refuses a `continue` of its own. New `Reactor#ready_interrupt_steps` lists the pending
274
+ interrupts (Symbols, and paths for nested ones). `be_paused_at`, `have_ready_interrupts` and
275
+ `TestSubject#resume(step:)` accept paths. A `continue` must now name a pending interrupt: naming
276
+ another ready step, which used to store the payload as that step's result, raises
277
+ `ValidationError`. An `interrupt` inside a `map` element now fails that element with "not
278
+ supported inside a map element" instead of `NoMethodError`.
279
+ See [Interrupts inside composed reactors](documentation/interrupts.md#interrupts-inside-composed-reactors).
280
+ * **`undo_all` on a map: one call rolls back every completed element.** A map declaring
281
+ `undo_all { |completed_results| ... }` is rolled back (an atomic map's failure, a later step's
282
+ failure, or `Reactor.undo`) by calling the block once with a lazy, index-ordered Enumerable of
283
+ the completed elements' results, instead of replaying each element's own undos. A fan-out map
284
+ reads its stored results and dispatches no element rollback jobs; nothing is held in memory all
285
+ at once. Failed elements still roll themselves back; a block that raises or returns a
286
+ `Failure` is reported as one rollback failure (`kind: :undo_all`) and the steps before the map
287
+ are still undone. New matcher: `have_run_undo_all(:map).with_elements(n)`.
288
+ * **A resume contended on the reactor's lock or semaphore is accepted, not lost** (see migration
289
+ note 1). New test helpers: `resume(..., process_jobs: false)` and the `be_resume_deferred`
290
+ matcher.
291
+ * **Several interrupts can be resumed at once** (see migration note 3): each accepted resume is
292
+ applied once.
244
293
  * **`fan_out` without `batch_size` is back-pressured: at most 50 element jobs per throw**
245
294
  (`RubyReactor::Map::DEFAULT_BATCH_SIZE`), forward and rollback. Before, every element was
246
295
  enqueued at once. A map of more than 50 elements with no declared `batch_size` now runs in
@@ -342,6 +391,24 @@ Every breaking or shape-changing item of the rollback work, with what to change.
342
391
 
343
392
  ### Bug Fixes
344
393
 
394
+ * **The recovery sweep no longer re-runs a run still executing in its caller's process.** A
395
+ synchronous `Reactor.run` (or an inline `continue`) now holds the run's liveness lock, renewed
396
+ while it executes, so a sweep during a long step leaves it alone; a killed process is still
397
+ recovered once the lock lapses (`context_lock_ttl`).
398
+ * **A caller's final save no longer overwrites a worker's progress.** A synchronous run that hands
399
+ off at a fan-out map (or a `background` step) keeps its lock until its final save, and a Worker
400
+ now takes the run's lock (waiting up to 2s) before it reads the run, so it always resumes from
401
+ the caller's final state.
402
+ * **Two resumes of one interrupt at the same instant: exactly one is accepted**, even in inline
403
+ job-testing mode, where both used to run.
404
+ * **A resumed run waiting on a reactor lock or semaphore is no longer marked `failed` without
405
+ rolling back.** The snooze limit (`lock_snooze_max_attempts`) now applies only before a run is
406
+ admitted; an admitted run (a deferred or `resume: :background` resume) keeps waiting and logs
407
+ `ruby_reactor.resume.waiting` once.
408
+ * **Manual undo finishes a `compensate` an interruption cut off.** A run aborted while its failing
409
+ step's `compensate` ran now records that step; `Reactor.undo(id)` runs the `compensate` again,
410
+ with the same arguments and reason, before undoing the completed steps (also inside composed
411
+ children and inline map elements). The dashboard flags such a run.
345
412
  * **A fan-out map inside a composed reactor no longer leaves the root running forever.** The map's
346
413
  completion resumed the child as a run of its own, and nothing resumed the root. Now the root
347
414
  resumes and finishes, and a failure of the map or of any later step rolls back through the root.
@@ -444,6 +511,20 @@ Every breaking or shape-changing item of the rollback work, with what to change.
444
511
  * Docs: a park keeps each level's lock without a second `:lock_acquired` only while the gap stays
445
512
  within the lock's `ttl`; a lapsed lock is acquired again.
446
513
 
514
+ ## [0.8.8](https://github.com/arturictus/ruby_reactor/compare/v0.8.7...v0.8.8) (2026-10-10)
515
+
516
+
517
+ ### Bug Fixes
518
+
519
+ * Rollback and Resume Follow-ups ([#72](https://github.com/arturictus/ruby_reactor/issues/72)) ([251fe9f](https://github.com/arturictus/ruby_reactor/commit/251fe9f6a9499738b6abcd6befd3a99e7cebec9f))
520
+
521
+ ## [0.8.7](https://github.com/arturictus/ruby_reactor/compare/v0.8.6...v0.8.7) (2026-10-08)
522
+
523
+
524
+ ### Features
525
+
526
+ * add support for interrupts inside composed children ([#71](https://github.com/arturictus/ruby_reactor/issues/71)) ([7be968e](https://github.com/arturictus/ruby_reactor/commit/7be968eb5dbfc6458e9419d8c6fddbf9d89cc80e))
527
+
447
528
  ## [0.8.6](https://github.com/arturictus/ruby_reactor/compare/v0.8.5...v0.8.6) (2026-10-08)
448
529
 
449
530
 
data/CLAUDE.md CHANGED
@@ -1,5 +1,5 @@
1
1
  <!-- SPECKIT START -->
2
2
  For additional context about technologies to be used, project structure,
3
3
  shell commands, and other important information, read the current plan
4
- at specs/009-distributed-map-undo/plan.md
4
+ at specs/010-rollback-follow-ups/plan.md
5
5
  <!-- SPECKIT END -->
data/README.md CHANGED
@@ -595,7 +595,12 @@ any background job is enqueued the root context is persisted, and after every
595
595
  completed step a checkpoint advances the stored blob — so a crash re-runs at most
596
596
  one step, never the whole reactor. Each running reactor also holds a short
597
597
  **liveness lock** that a live worker auto-extends; its absence is how a dead
598
- worker is detected.
598
+ worker is detected. A run executing in the caller's process (a synchronous
599
+ `Reactor.run`, an inline `continue`) holds the same lock while it executes, so a
600
+ sweep never runs it a second time, however long a step takes; if that process is
601
+ killed, the lock lapses within `context_lock_ttl` and the run is recovered like a
602
+ worker's. A worker takes the run's lock before it reads the run, so it always
603
+ resumes from the last save of whoever held it.
599
604
 
600
605
  **Recovery is not automatic until you start the sweeper.** A crashed worker's
601
606
  reactor only resumes when the recovery sweeper notices the lapsed liveness lock
@@ -704,6 +709,16 @@ ApprovalReactor.continue_by_correlation_id(
704
709
  )
705
710
  ```
706
711
 
712
+ An interrupt inside a `compose`d child pauses the top-level run; resume it there by the step path,
713
+ e.g. `step_name: [:approval, :wait_for_manager]`. See
714
+ [Interrupts inside composed reactors](documentation/interrupts.md#interrupts-inside-composed-reactors).
715
+ `continue` validates the payload in the calling process first: an invalid one is answered with its
716
+ validation failure and nothing is stored. A valid one claims its interrupt, so of two resumes of the
717
+ same interrupt exactly one is accepted (the other raises "already resumed"). When the run is busy,
718
+ another interrupt's resume is still running, or the reactor's `with_lock`/`with_semaphore` is held,
719
+ the accepted resume is handed to a worker and `continue` returns a `DispatchResult` instead of
720
+ raising, so a webhook's resume is never lost. See [Resuming Execution](documentation/interrupts.md#resuming-execution).
721
+
707
722
  ### Locks, Semaphores & Ordered Locks
708
723
 
709
724
  Coordinate across processes with Redis-backed primitives:
@@ -714,6 +729,8 @@ Coordinate across processes with Redis-backed primitives:
714
729
  - **`with_period`** — run at most once per calendar bucket (dedup / once-per-day, once-per-month, etc).
715
730
  - **`with_ordered_lock`** — strict transaction ordering via a monotonically increasing nonce assigned at enqueue. Workers can only proceed when their nonce equals `last_completed + 1`.
716
731
 
732
+ An interrupt's resume that meets the reactor's held lock or semaphore is accepted and deferred to a worker, which waits for it (`continue` returns a `DispatchResult`); a run already admitted is never failed by the snooze limit for waiting.
733
+
717
734
  ```ruby
718
735
  class RefundOrderReactor < RubyReactor::Reactor
719
736
  input :order_id
@@ -899,8 +916,8 @@ A `fan_out` map is a **hand-off point**: the reactor stops at the map (the calle
899
916
 
900
917
  By using `fan_out` with `batch_size`, the system applies **Back Pressure** to efficiently manage resources. [Read more about Back Pressure & Resource Management](documentation/data_pipelines.md#back-pressure--resource-management).
901
918
 
902
- **Rollback.** A map rolls back like a composed reactor, with no map-level rollback DSL: the `undo`s
903
- already declared on the element reactor's steps are each element's rollback. When the map fails
919
+ **Rollback.** A map rolls back like a composed reactor: the `undo`s already declared on the element
920
+ reactor's steps are each element's rollback, unless the map declares `undo_all` (below). When the map fails
904
921
  (an element fails in an `atomic` map, the default, or `collect` raises), and when a later step fails or the run is
905
922
  undone manually, every element that completed is rolled back, newest-started first, and the run ends
906
923
  with the same `Failure` in inline and fan-out mode alike. A map rolls back the way it ran: an inline
@@ -911,6 +928,28 @@ failing atomic fan-out map lets elements already in flight finish before it repo
911
928
  element `undo`s idempotent: one cut off by a killed worker runs again. See
912
929
  [Rollback](documentation/data_pipelines.md#rollback).
913
930
 
931
+ **One bulk undo.** When an API can roll back many elements in one call (a bulk refund), declare
932
+ `undo_all` on the map. Every rollback of the map then calls it **once** with the completed elements'
933
+ results, instead of replaying each element's undos:
934
+
935
+ ```ruby
936
+ map :charges, ChargeReactor do
937
+ source input(:payments)
938
+ argument :payment, element(:charges)
939
+ fan_out batch_size: 100
940
+
941
+ # A lazy Enumerable of the completed elements' results, in index order.
942
+ undo_all do |charges|
943
+ Payments.bulk_refund(charges.map { |charge| charge[:id] }.to_a)
944
+ end
945
+ end
946
+ ```
947
+
948
+ A fan-out map dispatches no element rollback jobs for it, and nothing holds every result at once.
949
+ Failed elements still roll themselves back; a block that raises or returns a `Failure` is reported as
950
+ one rollback failure (`kind: :undo_all`), and the steps before the map are still undone. See
951
+ [`undo_all`](documentation/data_pipelines.md#undo_all-one-bulk-rollback).
952
+
914
953
  `batch_size` is optional: with `fan_out` alone, no throw enqueues more than 50 element jobs (`RubyReactor::Map::DEFAULT_BATCH_SIZE`), forward and rollback, and the outcomes are aggregated into a `ResultEnumerator`. Set `batch_size` to change the throw size. See [`fan_out` Without `batch_size`](documentation/data_pipelines.md#fan_out-without-batch_size).
915
954
 
916
955
  > **Breaking change:** `async true` inside a `map` block has been **removed** — it read like `async_step`/`async_reactor`, which dispatch independent units the reactor does not stop for. It now raises at class-definition time. The exact replacement is `fan_out` (`fan_out batch_size: N`).
@@ -84,6 +84,19 @@ module RubyReactor
84
84
  @collect_block = block
85
85
  end
86
86
 
87
+ # One call that rolls back every completed element (010 US7), instead of
88
+ # replaying each element's own undos: a bulk refund, a batch delete. The
89
+ # block gets a lazy Enumerable of the completed elements' results, in
90
+ # index order, on every rollback of the map.
91
+ def undo_all(&block)
92
+ raise RubyReactor::Error::ValidationError.new("map :#{@name} undo_all needs a block", step: @name) unless block
93
+ if @undo_all_block
94
+ raise RubyReactor::Error::ValidationError.new("map :#{@name} declares undo_all twice", step: @name)
95
+ end
96
+
97
+ @undo_all_block = block
98
+ end
99
+
87
100
  # On (the default), the map succeeds only if every element does: the
88
101
  # first element failure fails it, no new element starts, and every
89
102
  # completed element is rolled back. Off, it completes with every
@@ -144,7 +157,8 @@ module RubyReactor
144
157
  batch_size: { source: RubyReactor::Template::Value.new(@batch_size) },
145
158
  collect_block: { source: RubyReactor::Template::Value.new(@collect_block) },
146
159
  atomic: { source: RubyReactor::Template::Value.new(@atomic) },
147
- fan_out: { source: RubyReactor::Template::Value.new(@fan_out) }
160
+ fan_out: { source: RubyReactor::Template::Value.new(@fan_out) },
161
+ **undo_all_argument
148
162
  },
149
163
  run_block: nil,
150
164
  compensate_block: nil,
@@ -155,6 +169,11 @@ module RubyReactor
155
169
  }
156
170
  end
157
171
 
172
+ # Read at rollback from this declaration (`MapStep#undo_all_block`).
173
+ def undo_all_argument
174
+ @undo_all_block ? { undo_all_block: { source: RubyReactor::Template::Value.new(@undo_all_block) } } : {}
175
+ end
176
+
158
177
  def extract_dependencies_from_mappings
159
178
  dependencies = []
160
179
  @argument_mappings.each_value do |source|
@@ -0,0 +1,18 @@
1
+ # frozen_string_literal: true
2
+
3
+ module RubyReactor
4
+ module Error
5
+ # The reason a cut-off `compensate` receives when a manual undo runs it
6
+ # again (010 R-10): the run was aborted while it ran, and the original
7
+ # exception cannot be rebuilt from storage. It carries that exception's
8
+ # message, and its class name as `original_class`.
9
+ class RecordedFailure < Base
10
+ attr_reader :original_class
11
+
12
+ def initialize(message, original_class:)
13
+ super(message)
14
+ @original_class = original_class
15
+ end
16
+ end
17
+ end
18
+ end
@@ -41,12 +41,32 @@ module RubyReactor
41
41
  @rollback_failures.replace(Array(ContextSerializer.deserialize_value(list)))
42
42
  end
43
43
 
44
+ # A step that paused (a compose whose child hit an interrupt) is already
45
+ # on top as a partial run; its re-run replaces that entry, so the step is
46
+ # undone once (010 R-04).
47
+ # ponytail: assumes the paused step's entry is on top when it re-runs,
48
+ # which holds because a run stops at a pause and resumes at that step.
44
49
  def add_to_undo_stack(step_info)
50
+ undo_stack.pop if undo_stack.last && undo_stack.last[:step].name == step_info[:step].name
45
51
  @context.undo_stack << step_info
46
52
  end
47
53
 
54
+ # A failing step whose `compensate` was cut off by an interruption, as
55
+ # an aborted run keeps it for manual undo (010 R-10, DM §4); nil once that
56
+ # `compensate` returned. The `arguments` key marks the record: 009's
57
+ # hand-off states carry none.
58
+ def pending_record
59
+ return unless @pending && !@pending[:compensated]
60
+
61
+ { "trigger" => "failure", "step" => @pending[:step].to_s, "compensated" => false,
62
+ "arguments" => ContextSerializer.serialize_value(@pending[:arguments] || {}),
63
+ "error" => recorded_error(@pending[:error]),
64
+ "failures" => ContextSerializer.serialize_value(@rollback_failures) }
65
+ end
66
+
48
67
  def handle_step_failure(step_config, error, arguments)
49
- @pending = { step: step_config.name, error: error, compensated: false }
68
+ @pending = { step: step_config.name, error: error, compensated: false,
69
+ arguments: step_config.rollback_arguments(arguments) }
50
70
  # A step whose OWN coordination acquisition failed (contention, or a
51
71
  # bad key proc) never ran its body — "no step compensates, the
52
72
  # contended step's work has not been attempted" (US3-1/T018), which
@@ -115,6 +135,12 @@ module RubyReactor
115
135
 
116
136
  private
117
137
 
138
+ def recorded_error(error)
139
+ return error if error.is_a?(String)
140
+
141
+ { "class" => error.class.name, "message" => error.respond_to?(:message) ? error.message : error.to_s }
142
+ end
143
+
118
144
  def middlewares
119
145
  @context.middlewares || RubyReactor::MiddlewareRunner.new([])
120
146
  end
@@ -27,6 +27,8 @@ module RubyReactor
27
27
  handle_retries_exhausted(step_config, result, resolved_arguments)
28
28
  when RubyReactor::Failure
29
29
  handle_failure(step_config, result, resolved_arguments)
30
+ when RubyReactor::InterruptResult
31
+ handle_interrupted(step_config, resolved_arguments)
30
32
  else
31
33
  handle_unknown_result(step_config, result, resolved_arguments)
32
34
  end
@@ -148,6 +150,20 @@ module RubyReactor
148
150
  result
149
151
  end
150
152
 
153
+ # A composed child paused at an interrupt (010): the step stays
154
+ # incomplete, and this run pauses on it. `with_step` cleared
155
+ # `current_step` on the way out; it is the run's resume cursor. The
156
+ # child's completed steps are only undone through this step, so it joins
157
+ # the undo stack as a partial run, as an interrupted compose does (R-04).
158
+ def handle_interrupted(step_config, resolved_arguments)
159
+ @context.current_step = step_config.name
160
+ return unless step_config.undoes_partial_run?
161
+
162
+ @compensation_manager.add_to_undo_stack({ step: step_config,
163
+ arguments: step_config.rollback_arguments(resolved_arguments),
164
+ result: RubyReactor.Success(nil) })
165
+ end
166
+
151
167
  # A step returned `RubyReactor.Skipped(...)`: the run continues exactly as
152
168
  # for a Success (value, hand-off, period mark), but the step is NOT pushed
153
169
  # for undo — it had nothing to do, so there is nothing to revert (008 R-19).
@@ -108,8 +108,8 @@ module RubyReactor
108
108
  result
109
109
  when RubyReactor::Failure
110
110
  handle_failure_result(step_config, reactor_class, result)
111
- when RetryQueuedResult, RubyReactor::DispatchResult
112
- # Pass through async results
111
+ when RetryQueuedResult, RubyReactor::DispatchResult, RubyReactor::InterruptResult
112
+ # Pass through async results, and a composed child's pause
113
113
  result
114
114
  else
115
115
  clear_retry_state
@@ -431,6 +431,15 @@ module RubyReactor
431
431
  return RubyReactor.Success(result)
432
432
  end
433
433
 
434
+ # A map element cannot pause: nothing would resume it. The step fails
435
+ # like any other, rolling back what the element completed (010 R-08).
436
+ if inside_map_element?
437
+ failure = RubyReactor::Failure("interrupt :#{step_config.name} is not supported inside a map element",
438
+ step_name: step_config.name, reactor_name: @reactor_class.name,
439
+ retryable: false)
440
+ return @result_handler.handle_step_result(step_config, failure, {})
441
+ end
442
+
434
443
  # We are pausing
435
444
  correlation_id = nil
436
445
  correlation_id = step_config.correlation_id_block.call(@context) if step_config.correlation_id_block
@@ -445,6 +454,17 @@ module RubyReactor
445
454
  )
446
455
  end
447
456
 
457
+ # This run, or one it is composed into, is a map element.
458
+ def inside_map_element?
459
+ context = @context
460
+ while context
461
+ return true if context.map_metadata
462
+
463
+ context = context.parent_context
464
+ end
465
+ false
466
+ end
467
+
448
468
  def configuration
449
469
  RubyReactor::Configuration.instance
450
470
  end
@@ -21,6 +21,12 @@ module RubyReactor
21
21
  attr_reader :reactor_class, :context, :dependency_graph, :compensation_manager, :retry_manager, :result_handler,
22
22
  :step_executor, :result, :middlewares
23
23
 
24
+ # The owner of a run lock (`async:<id>`) the caller already holds: a Worker
25
+ # or `continue` that locked the run before loading it (010 R-02, R-03).
26
+ # `acquire_context_lock` then re-enters it (the lock counts by owner), and
27
+ # the outer holder releases last.
28
+ attr_writer :context_lock_owner
29
+
24
30
  def initialize(reactor_class, inputs = {}, context = nil)
25
31
  # Resume, map, compose and background workers build an Executor without
26
32
  # going through Reactor#run; the inferred wiring must exist there too.
@@ -109,6 +115,12 @@ module RubyReactor
109
115
  input_validator = InputValidator.new(@reactor_class, @context)
110
116
  input_validator.validate!
111
117
 
118
+ # A run in the caller's process holds its liveness lock too (010 R-01),
119
+ # so the sweeper never mistakes it for a dead worker's run and a Worker
120
+ # resumed by its hand-off waits for its final save. In a worker the
121
+ # executor runs `resume_execution`, which takes it there.
122
+ acquire_context_lock unless @context.inline_async_execution
123
+
112
124
  reset_held_lock_keys!
113
125
  acquire_locks_with_telemetry
114
126
 
@@ -168,6 +180,10 @@ module RubyReactor
168
180
  release_locks unless @parked
169
181
  leave_ordered_lock_scope
170
182
  save_context if persist_context? && !skip_context_persist?
183
+ # Released only after that save (J-3, 009 R-13): a Worker waiting on the
184
+ # lock then reads this run's final state.
185
+ @acquired_context_lock&.release
186
+ @acquired_context_lock = nil
171
187
 
172
188
  emit_lifecycle_completion(completed)
173
189
  end
@@ -221,6 +237,10 @@ module RubyReactor
221
237
  # holds it — composed/nested children resume inline under the root worker
222
238
  # and must not contend on the root's own key.
223
239
  acquire_context_lock
240
+ # The only place a claimed resume payload enters a context (010 R-04,
241
+ # J-5): under the run's lock, before the reactor-level lock or
242
+ # semaphore, so a contended resume still saves the payload it applied.
243
+ InterruptClaims.apply!(@context) if (@context.root_context || @context).equal?(@context)
224
244
 
225
245
  reset_held_lock_keys!
226
246
 
@@ -249,7 +269,6 @@ module RubyReactor
249
269
  @context.admit!
250
270
  prepare_for_resume
251
271
  save_context
252
- @past_gates = true
253
272
 
254
273
  @result = if @context.current_step
255
274
  execute_current_step_and_continue
@@ -313,6 +332,7 @@ module RubyReactor
313
332
  def undo_all(&after_pop)
314
333
  saved = @context.rollback && @context.rollback["failures"]
315
334
  @compensation_manager.restore_rollback_failures(saved) if saved
335
+ compensate_pending!
316
336
  @compensation_manager.rollback_completed_steps(&after_pop)
317
337
  clear_saved_rollback_failures
318
338
  rescue Error::RollbackHandedOff
@@ -390,12 +410,6 @@ module RubyReactor
390
410
  RubyReactor.reactor_storage_name(@reactor_class))
391
411
  end
392
412
 
393
- # True once `resume_execution` is past its reactor-level gates (context
394
- # lock, lock, semaphore, period): from there on it may have run steps.
395
- def past_gates?
396
- @past_gates == true
397
- end
398
-
399
413
  # Reached only by an interruption — a signal, an exit, out of memory, an
400
414
  # enclosing timeout (008 R-08, R-16); every other exception was rescued as
401
415
  # `Error::Rescuable` and rolled back. Running user rollback code now is
@@ -404,7 +418,13 @@ module RubyReactor
404
418
  # `Reactor#undo`; the `ensure` persists it, best effort. A worker run stays
405
419
  # `running` and its job is redelivered.
406
420
  def mark_aborted
407
- @context.status = :aborted unless @context.inline_async_execution
421
+ return if @context.inline_async_execution
422
+
423
+ @context.status = :aborted
424
+ # The failing step's `compensate`, if the interruption cut it off: a
425
+ # manual undo runs it again before the undo stack (010 R-10, J-9).
426
+ record = @compensation_manager.pending_record
427
+ @context.rollback = record if record
408
428
  end
409
429
 
410
430
  # A rollback run from a `rescue Error::Rescuable` body is outside the
@@ -663,6 +683,8 @@ module RubyReactor
663
683
  # defeat the guard. Only the root executor acquires — a composed/nested child
664
684
  # resumes inline under the root worker and shares the root's lock, so it must
665
685
  # not try to re-acquire the same key with a different owner (self-deadlock).
686
+ # Held by a worker's resume and by a run in the caller's process alike (010
687
+ # R-01); an outer holder's owner, when set, is re-entered (R-03).
666
688
  def acquire_context_lock
667
689
  root = @context.root_context || @context
668
690
  return unless root.equal?(@context) # only the root executor holds it
@@ -980,7 +1002,7 @@ module RubyReactor
980
1002
  # gives (I-7).
981
1003
  def finish_rollback(state)
982
1004
  @compensation_manager.restore_rollback_failures(state["failures"])
983
- return finish_undo if state["trigger"] == "undo"
1005
+ return finish_undo(state) if state["trigger"] == "undo"
984
1006
 
985
1007
  failure = ContextSerializer.deserialize_value(state["failure"])
986
1008
  step_config = state["step"] && @reactor_class.steps[state["step"].to_sym]
@@ -998,8 +1020,38 @@ module RubyReactor
998
1020
  @result_handler.handle_execution_error(e)
999
1021
  end
1000
1022
 
1001
- def finish_undo
1023
+ # A manual undo ends `cancelled`, or `failed` when it was the failure of
1024
+ # an interrupt whose payload attempts ran out (010 R-08).
1025
+ # An aborted run's failing step whose `compensate` an interruption cut off
1026
+ # (010 R-10, J-9): run it again, with the recorded arguments and reason,
1027
+ # before the undo stack. Only that record carries `arguments`; 009's
1028
+ # hand-off states, which name the same keys, are finished elsewhere.
1029
+ def compensate_pending!
1030
+ state = @context.rollback
1031
+ return unless state.is_a?(Hash) && state.key?("arguments") && state["compensated"] == false
1032
+
1033
+ step_config = state["step"] && @reactor_class.steps[state["step"].to_sym]
1034
+ return unless step_config
1035
+
1036
+ @compensation_manager.compensate(step_config, recorded_reason(state["error"]),
1037
+ ContextSerializer.deserialize_value(state["arguments"]))
1038
+ state["compensated"] = true
1039
+ end
1040
+
1041
+ def recorded_reason(error)
1042
+ return error unless error.is_a?(Hash)
1043
+
1044
+ Error::RecordedFailure.new(error["message"], original_class: error["class"])
1045
+ end
1046
+
1047
+ def finish_undo(state)
1002
1048
  @compensation_manager.rollback_completed_steps
1049
+ if state["failure_reason"]
1050
+ @context.status = "failed"
1051
+ @context.failure_reason = state["failure_reason"]
1052
+ return nil
1053
+ end
1054
+
1003
1055
  @context.cancelled = true
1004
1056
  @context.cancellation_reason = "Undo triggered"
1005
1057
  @context.status = "cancelled"
@@ -0,0 +1,81 @@
1
+ # frozen_string_literal: true
2
+
3
+ module RubyReactor
4
+ # A resume claims its interrupt before anything runs (010 R-04): the first
5
+ # `SET NX` wins, in every execution mode, and the claim holds the validated
6
+ # payload. Only the execution that owns the run's lock copies a claim into
7
+ # the context (J-5), at the start of `Executor#resume_execution`, so a
8
+ # resume never writes the run from outside it.
9
+ module InterruptClaims
10
+ module_function
11
+
12
+ # true when this call won the interrupt; false when a resume already did.
13
+ # `path` is the interrupt's step path (010 R-05); a nested one is keyed `a.b`.
14
+ def claim!(context, path, payload)
15
+ storage.claim_interrupt_resume(context.context_id, storage_name(context), key(Array(path)),
16
+ JSON.generate(ContextSerializer.serialize_value(payload)))
17
+ end
18
+
19
+ # true when a resume already claimed the interrupt, applied or not.
20
+ def claimed?(context, path)
21
+ storage.retrieve_interrupt_resumes(context.context_id, storage_name(context), [key(Array(path))]).any?
22
+ end
23
+
24
+ # `{ step => payload }` (a nested interrupt: `{ [child, step] => payload }`)
25
+ # for the claimed interrupts that have no result yet.
26
+ def unapplied(context)
27
+ paths = pending_interrupts(context)
28
+ return {} if paths.empty?
29
+
30
+ raw = storage.retrieve_interrupt_resumes(context.context_id, storage_name(context), paths.map { |p| key(p) })
31
+ paths.filter_map do |path|
32
+ if raw.key?(key(path))
33
+ [path.one? ? path.first : path,
34
+ ContextSerializer.deserialize_value(JSON.parse(raw[key(path)]))]
35
+ end
36
+ end.to_h
37
+ end
38
+
39
+ # Copies every unapplied claim into the context that paused at it (010 R-06).
40
+ # Returns their steps.
41
+ def apply!(context)
42
+ claims = unapplied(context)
43
+ claims.each do |step, payload|
44
+ path = Array(step)
45
+ path[0...-1].reduce(context) { |ctx, name| child(ctx, name) }.set_result(path.last, payload)
46
+ end
47
+ claims.keys
48
+ end
49
+
50
+ # Step paths of the interrupts without a result, here and in paused composed children.
51
+ def pending_interrupts(context, prefix = [])
52
+ steps = context.reactor_class.respond_to?(:steps) ? context.reactor_class.steps : {}
53
+ interrupts = steps.select do |name, config|
54
+ config.respond_to?(:interrupt?) && config.interrupt? && !context.has_result?(name)
55
+ end
56
+ own = interrupts.keys.map { |name| prefix + [name] }
57
+ nested = steps.keys.flat_map do |name|
58
+ (c = child(context, name)) ? pending_interrupts(c, prefix + [name]) : []
59
+ end
60
+ own + nested
61
+ end
62
+
63
+ def child(context, name)
64
+ entry = context.composed_contexts[name] || context.composed_contexts[name.to_s]
65
+ c = entry.is_a?(Hash) && (entry[:context] || entry["context"])
66
+ c if c.is_a?(Context) && c.status.to_s == "paused"
67
+ end
68
+
69
+ def key(path)
70
+ path.join(".")
71
+ end
72
+
73
+ def storage_name(context)
74
+ RubyReactor.reactor_storage_name(context.reactor_class)
75
+ end
76
+
77
+ def storage
78
+ RubyReactor.configuration.storage_adapter
79
+ end
80
+ end
81
+ end