ruby_reactor 0.8.7 → 0.8.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (35) hide show
  1. checksums.yaml +4 -4
  2. data/.release-please-manifest.json +1 -1
  3. data/.specify/feature.json +1 -1
  4. data/CHANGELOG.md +59 -2
  5. data/CLAUDE.md +1 -1
  6. data/README.md +38 -3
  7. data/lib/ruby_reactor/dsl/map_builder.rb +20 -1
  8. data/lib/ruby_reactor/error/recorded_failure.rb +18 -0
  9. data/lib/ruby_reactor/executor/compensation_manager.rb +21 -1
  10. data/lib/ruby_reactor/executor.rb +62 -10
  11. data/lib/ruby_reactor/interrupt_claims.rb +81 -0
  12. data/lib/ruby_reactor/map/step_rollback.rb +121 -0
  13. data/lib/ruby_reactor/reactor.rb +210 -95
  14. data/lib/ruby_reactor/rspec/matchers.rb +49 -0
  15. data/lib/ruby_reactor/rspec/test_subject.rb +17 -6
  16. data/lib/ruby_reactor/step/map_step.rb +12 -0
  17. data/lib/ruby_reactor/storage/adapter.rb +17 -0
  18. data/lib/ruby_reactor/storage/redis_adapter.rb +42 -0
  19. data/lib/ruby_reactor/version.rb +1 -1
  20. data/lib/ruby_reactor/web/api.rb +17 -0
  21. data/lib/ruby_reactor/web/public/assets/{index-C6ejszCQ.js → index-DxjQlW7Q.js} +8 -8
  22. data/lib/ruby_reactor/web/public/index.html +1 -1
  23. data/lib/ruby_reactor/worker.rb +73 -6
  24. data/specs/010-rollback-follow-ups/checklists/requirements.md +47 -0
  25. data/specs/010-rollback-follow-ups/contracts/api-surface.md +123 -0
  26. data/specs/010-rollback-follow-ups/contracts/resume-protocol.md +132 -0
  27. data/specs/010-rollback-follow-ups/data-model.md +139 -0
  28. data/specs/010-rollback-follow-ups/plan.md +273 -0
  29. data/specs/010-rollback-follow-ups/pr-description.md +110 -0
  30. data/specs/010-rollback-follow-ups/quickstart.md +157 -0
  31. data/specs/010-rollback-follow-ups/research.md +440 -0
  32. data/specs/010-rollback-follow-ups/spec.md +489 -0
  33. data/specs/010-rollback-follow-ups/tasks.md +1022 -0
  34. data/specs/future_improvements.md +17 -2
  35. metadata +14 -2
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: a0d1aa7711547d66d5a7f19acf5e0b13c7e208a9660d3f1714eb8df2e7eff004
4
- data.tar.gz: a0b52b6139f80b5133f9585f39ff5fe227e46b34dfb9d1b8767bf4f068e5678f
3
+ metadata.gz: 1d262a9eed4af540e9573901be72e882f2c5e013550d51881dccef9205c6e873
4
+ data.tar.gz: 7a499dd2203c8059bccf9638b621974784593cabe54fddbb88e16036b7c634c5
5
5
  SHA512:
6
- metadata.gz: 5457e6cf85bb33a4bd12f00161d810a6dc491a590c3cefaaf2c5c967987d78e783e199c77bdff77d2b2745f569098c9492d07f4aebb0a7ad74f023bc5274f525
7
- data.tar.gz: 58ad78fe9e8e1c1e6b8b7106c12c384534aae6db6747860202cfb38dd75e3922f35c88b65f91320cfaef9fb121b6b0cae63dce4f42c677a5ae7ee4c6b0ff91b6
6
+ metadata.gz: 578c3ceef421e1718c8ec48e746feae08e226e085c235b6594ac881ce0aedcf0fea8e684508c966584a6bc803bfa11a4fbf24a0666b065285f1161b8b12505dd
7
+ data.tar.gz: 6ee564153b8a3ce213fddf09e91df37be6b967b5a284c5cd0cdbbc4bca8b68ae4a5b3b064c314492f6f53b1bfde4ae29173cef2439886064d10c742c677cc128
@@ -1,3 +1,3 @@
1
1
  {
2
- ".": "0.8.7"
2
+ ".": "0.8.8"
3
3
  }
@@ -1,3 +1,3 @@
1
1
  {
2
- "feature_directory": "specs/010-interrupt-in-composed-child"
2
+ "feature_directory": "specs/010-rollback-follow-ups"
3
3
  }
data/CHANGELOG.md CHANGED
@@ -239,6 +239,25 @@ Every breaking or shape-changing item of the rollback work, with what to change.
239
239
  9. **`rollback_failures` entries may carry `map_step:` / `element_index:`** and the reasons
240
240
  `:context_unavailable` / `:element_in_flight` (additive).
241
241
 
242
+ ### Migration notes: rollback and resume follow-ups
243
+
244
+ 1. **A resume that meets the reactor's held `with_lock` or `with_semaphore` no longer raises.**
245
+ `continue` used to raise `Lock::AcquisitionError` / `Semaphore::AcquisitionError` and leave
246
+ the run paused. It now validates the payload, accepts the resume, and returns a
247
+ `RubyReactor::DispatchResult`; a worker waits for the lock and finishes the run. A `rescue` of
248
+ those errors around `continue` simply stops firing. An invalid payload is still answered at
249
+ once, as before.
250
+ 2. **A second resume of the same interrupt raises `ValidationError`** ("Cannot resume: interrupt
251
+ :x was already resumed"), in every execution mode, including `Sidekiq::Testing.inline!`.
252
+ 3. **A resume for another ready interrupt of a `running` run is accepted** (it used to raise "the
253
+ reactor is running") and returns a `DispatchResult`.
254
+ 4. **Interrupt attempt counts restart** for runs paused across the upgrade: they move out of the
255
+ run's context into their own record, so such a run may take up to `max_attempts - 1` extra
256
+ invalid payloads.
257
+ 5. **A synchronous `Reactor.run` holds the run's liveness lock** while it executes. A
258
+ `Reactor.undo(id)` of a run still executing in its caller's process now waits for it, then
259
+ raises `Lock::AcquisitionError`, as it does for a run executing in a worker.
260
+
242
261
  ### Features
243
262
 
244
263
  * **An `interrupt` inside a composed child pauses the top-level run.** Before, the compose step
@@ -248,8 +267,8 @@ Every breaking or shape-changing item of the rollback work, with what to change.
248
267
  then the interrupt (`continue(id:, payload:, step_name: [:approval, :wait_for_manager])`), by id
249
268
  or by the child interrupt's correlation id. Payload validation, `max_attempts`,
250
269
  `resume: :background` and the resume guards apply as for a top-level interrupt. A resume
251
- contended on the child's own `with_lock`/`with_semaphore` fails and rolls back the run. Only
252
- contention on the top-level reactor's lock leaves it paused. `Reactor.undo`
270
+ contended on the child's own `with_lock`/`with_semaphore` fails and rolls back the run. Contention
271
+ on the top-level reactor's lock hands the resume to a worker. `Reactor.undo`
253
272
  of the paused run rolls back the child's completed steps, then the parent's. A composed child
254
273
  refuses a `continue` of its own. New `Reactor#ready_interrupt_steps` lists the pending
255
274
  interrupts (Symbols, and paths for nested ones). `be_paused_at`, `have_ready_interrupts` and
@@ -258,6 +277,19 @@ Every breaking or shape-changing item of the rollback work, with what to change.
258
277
  `ValidationError`. An `interrupt` inside a `map` element now fails that element with "not
259
278
  supported inside a map element" instead of `NoMethodError`.
260
279
  See [Interrupts inside composed reactors](documentation/interrupts.md#interrupts-inside-composed-reactors).
280
+ * **`undo_all` on a map: one call rolls back every completed element.** A map declaring
281
+ `undo_all { |completed_results| ... }` is rolled back (an atomic map's failure, a later step's
282
+ failure, or `Reactor.undo`) by calling the block once with a lazy, index-ordered Enumerable of
283
+ the completed elements' results, instead of replaying each element's own undos. A fan-out map
284
+ reads its stored results and dispatches no element rollback jobs; nothing is held in memory all
285
+ at once. Failed elements still roll themselves back; a block that raises or returns a
286
+ `Failure` is reported as one rollback failure (`kind: :undo_all`) and the steps before the map
287
+ are still undone. New matcher: `have_run_undo_all(:map).with_elements(n)`.
288
+ * **A resume contended on the reactor's lock or semaphore is accepted, not lost** (see migration
289
+ note 1). New test helpers: `resume(..., process_jobs: false)` and the `be_resume_deferred`
290
+ matcher.
291
+ * **Several interrupts can be resumed at once** (see migration note 3): each accepted resume is
292
+ applied once.
261
293
  * **`fan_out` without `batch_size` is back-pressured: at most 50 element jobs per throw**
262
294
  (`RubyReactor::Map::DEFAULT_BATCH_SIZE`), forward and rollback. Before, every element was
263
295
  enqueued at once. A map of more than 50 elements with no declared `batch_size` now runs in
@@ -359,6 +391,24 @@ Every breaking or shape-changing item of the rollback work, with what to change.
359
391
 
360
392
  ### Bug Fixes
361
393
 
394
+ * **The recovery sweep no longer re-runs a run still executing in its caller's process.** A
395
+ synchronous `Reactor.run` (or an inline `continue`) now holds the run's liveness lock, renewed
396
+ while it executes, so a sweep during a long step leaves it alone; a killed process is still
397
+ recovered once the lock lapses (`context_lock_ttl`).
398
+ * **A caller's final save no longer overwrites a worker's progress.** A synchronous run that hands
399
+ off at a fan-out map (or a `background` step) keeps its lock until its final save, and a Worker
400
+ now takes the run's lock (waiting up to 2s) before it reads the run, so it always resumes from
401
+ the caller's final state.
402
+ * **Two resumes of one interrupt at the same instant: exactly one is accepted**, even in inline
403
+ job-testing mode, where both used to run.
404
+ * **A resumed run waiting on a reactor lock or semaphore is no longer marked `failed` without
405
+ rolling back.** The snooze limit (`lock_snooze_max_attempts`) now applies only before a run is
406
+ admitted; an admitted run (a deferred or `resume: :background` resume) keeps waiting and logs
407
+ `ruby_reactor.resume.waiting` once.
408
+ * **Manual undo finishes a `compensate` an interruption cut off.** A run aborted while its failing
409
+ step's `compensate` ran now records that step; `Reactor.undo(id)` runs the `compensate` again,
410
+ with the same arguments and reason, before undoing the completed steps (also inside composed
411
+ children and inline map elements). The dashboard flags such a run.
362
412
  * **A fan-out map inside a composed reactor no longer leaves the root running forever.** The map's
363
413
  completion resumed the child as a run of its own, and nothing resumed the root. Now the root
364
414
  resumes and finishes, and a failure of the map or of any later step rolls back through the root.
@@ -461,6 +511,13 @@ Every breaking or shape-changing item of the rollback work, with what to change.
461
511
  * Docs: a park keeps each level's lock without a second `:lock_acquired` only while the gap stays
462
512
  within the lock's `ttl`; a lapsed lock is acquired again.
463
513
 
514
+ ## [0.8.8](https://github.com/arturictus/ruby_reactor/compare/v0.8.7...v0.8.8) (2026-10-10)
515
+
516
+
517
+ ### Bug Fixes
518
+
519
+ * Rollback and Resume Follow-ups ([#72](https://github.com/arturictus/ruby_reactor/issues/72)) ([251fe9f](https://github.com/arturictus/ruby_reactor/commit/251fe9f6a9499738b6abcd6befd3a99e7cebec9f))
520
+
464
521
  ## [0.8.7](https://github.com/arturictus/ruby_reactor/compare/v0.8.6...v0.8.7) (2026-10-08)
465
522
 
466
523
 
data/CLAUDE.md CHANGED
@@ -1,5 +1,5 @@
1
1
  <!-- SPECKIT START -->
2
2
  For additional context about technologies to be used, project structure,
3
3
  shell commands, and other important information, read the current plan
4
- at specs/010-interrupt-in-composed-child/plan.md
4
+ at specs/010-rollback-follow-ups/plan.md
5
5
  <!-- SPECKIT END -->
data/README.md CHANGED
@@ -595,7 +595,12 @@ any background job is enqueued the root context is persisted, and after every
595
595
  completed step a checkpoint advances the stored blob — so a crash re-runs at most
596
596
  one step, never the whole reactor. Each running reactor also holds a short
597
597
  **liveness lock** that a live worker auto-extends; its absence is how a dead
598
- worker is detected.
598
+ worker is detected. A run executing in the caller's process (a synchronous
599
+ `Reactor.run`, an inline `continue`) holds the same lock while it executes, so a
600
+ sweep never runs it a second time, however long a step takes; if that process is
601
+ killed, the lock lapses within `context_lock_ttl` and the run is recovered like a
602
+ worker's. A worker takes the run's lock before it reads the run, so it always
603
+ resumes from the last save of whoever held it.
599
604
 
600
605
  **Recovery is not automatic until you start the sweeper.** A crashed worker's
601
606
  reactor only resumes when the recovery sweeper notices the lapsed liveness lock
@@ -707,6 +712,12 @@ ApprovalReactor.continue_by_correlation_id(
707
712
  An interrupt inside a `compose`d child pauses the top-level run; resume it there by the step path,
708
713
  e.g. `step_name: [:approval, :wait_for_manager]`. See
709
714
  [Interrupts inside composed reactors](documentation/interrupts.md#interrupts-inside-composed-reactors).
715
+ `continue` validates the payload in the calling process first: an invalid one is answered with its
716
+ validation failure and nothing is stored. A valid one claims its interrupt, so of two resumes of the
717
+ same interrupt exactly one is accepted (the other raises "already resumed"). When the run is busy,
718
+ another interrupt's resume is still running, or the reactor's `with_lock`/`with_semaphore` is held,
719
+ the accepted resume is handed to a worker and `continue` returns a `DispatchResult` instead of
720
+ raising, so a webhook's resume is never lost. See [Resuming Execution](documentation/interrupts.md#resuming-execution).
710
721
 
711
722
  ### Locks, Semaphores & Ordered Locks
712
723
 
@@ -718,6 +729,8 @@ Coordinate across processes with Redis-backed primitives:
718
729
  - **`with_period`** — run at most once per calendar bucket (dedup / once-per-day, once-per-month, etc).
719
730
  - **`with_ordered_lock`** — strict transaction ordering via a monotonically increasing nonce assigned at enqueue. Workers can only proceed when their nonce equals `last_completed + 1`.
720
731
 
732
+ An interrupt's resume that meets the reactor's held lock or semaphore is accepted and deferred to a worker, which waits for it (`continue` returns a `DispatchResult`); a run already admitted is never failed by the snooze limit for waiting.
733
+
721
734
  ```ruby
722
735
  class RefundOrderReactor < RubyReactor::Reactor
723
736
  input :order_id
@@ -903,8 +916,8 @@ A `fan_out` map is a **hand-off point**: the reactor stops at the map (the calle
903
916
 
904
917
  By using `fan_out` with `batch_size`, the system applies **Back Pressure** to efficiently manage resources. [Read more about Back Pressure & Resource Management](documentation/data_pipelines.md#back-pressure--resource-management).
905
918
 
906
- **Rollback.** A map rolls back like a composed reactor, with no map-level rollback DSL: the `undo`s
907
- already declared on the element reactor's steps are each element's rollback. When the map fails
919
+ **Rollback.** A map rolls back like a composed reactor: the `undo`s already declared on the element
920
+ reactor's steps are each element's rollback, unless the map declares `undo_all` (below). When the map fails
908
921
  (an element fails in an `atomic` map, the default, or `collect` raises), and when a later step fails or the run is
909
922
  undone manually, every element that completed is rolled back, newest-started first, and the run ends
910
923
  with the same `Failure` in inline and fan-out mode alike. A map rolls back the way it ran: an inline
@@ -915,6 +928,28 @@ failing atomic fan-out map lets elements already in flight finish before it repo
915
928
  element `undo`s idempotent: one cut off by a killed worker runs again. See
916
929
  [Rollback](documentation/data_pipelines.md#rollback).
917
930
 
931
+ **One bulk undo.** When an API can roll back many elements in one call (a bulk refund), declare
932
+ `undo_all` on the map. Every rollback of the map then calls it **once** with the completed elements'
933
+ results, instead of replaying each element's undos:
934
+
935
+ ```ruby
936
+ map :charges, ChargeReactor do
937
+ source input(:payments)
938
+ argument :payment, element(:charges)
939
+ fan_out batch_size: 100
940
+
941
+ # A lazy Enumerable of the completed elements' results, in index order.
942
+ undo_all do |charges|
943
+ Payments.bulk_refund(charges.map { |charge| charge[:id] }.to_a)
944
+ end
945
+ end
946
+ ```
947
+
948
+ A fan-out map dispatches no element rollback jobs for it, and nothing holds every result at once.
949
+ Failed elements still roll themselves back; a block that raises or returns a `Failure` is reported as
950
+ one rollback failure (`kind: :undo_all`), and the steps before the map are still undone. See
951
+ [`undo_all`](documentation/data_pipelines.md#undo_all-one-bulk-rollback).
952
+
918
953
  `batch_size` is optional: with `fan_out` alone, no throw enqueues more than 50 element jobs (`RubyReactor::Map::DEFAULT_BATCH_SIZE`), forward and rollback, and the outcomes are aggregated into a `ResultEnumerator`. Set `batch_size` to change the throw size. See [`fan_out` Without `batch_size`](documentation/data_pipelines.md#fan_out-without-batch_size).
919
954
 
920
955
  > **Breaking change:** `async true` inside a `map` block has been **removed** — it read like `async_step`/`async_reactor`, which dispatch independent units the reactor does not stop for. It now raises at class-definition time. The exact replacement is `fan_out` (`fan_out batch_size: N`).
@@ -84,6 +84,19 @@ module RubyReactor
84
84
  @collect_block = block
85
85
  end
86
86
 
87
+ # One call that rolls back every completed element (010 US7), instead of
88
+ # replaying each element's own undos: a bulk refund, a batch delete. The
89
+ # block gets a lazy Enumerable of the completed elements' results, in
90
+ # index order, on every rollback of the map.
91
+ def undo_all(&block)
92
+ raise RubyReactor::Error::ValidationError.new("map :#{@name} undo_all needs a block", step: @name) unless block
93
+ if @undo_all_block
94
+ raise RubyReactor::Error::ValidationError.new("map :#{@name} declares undo_all twice", step: @name)
95
+ end
96
+
97
+ @undo_all_block = block
98
+ end
99
+
87
100
  # On (the default), the map succeeds only if every element does: the
88
101
  # first element failure fails it, no new element starts, and every
89
102
  # completed element is rolled back. Off, it completes with every
@@ -144,7 +157,8 @@ module RubyReactor
144
157
  batch_size: { source: RubyReactor::Template::Value.new(@batch_size) },
145
158
  collect_block: { source: RubyReactor::Template::Value.new(@collect_block) },
146
159
  atomic: { source: RubyReactor::Template::Value.new(@atomic) },
147
- fan_out: { source: RubyReactor::Template::Value.new(@fan_out) }
160
+ fan_out: { source: RubyReactor::Template::Value.new(@fan_out) },
161
+ **undo_all_argument
148
162
  },
149
163
  run_block: nil,
150
164
  compensate_block: nil,
@@ -155,6 +169,11 @@ module RubyReactor
155
169
  }
156
170
  end
157
171
 
172
+ # Read at rollback from this declaration (`MapStep#undo_all_block`).
173
+ def undo_all_argument
174
+ @undo_all_block ? { undo_all_block: { source: RubyReactor::Template::Value.new(@undo_all_block) } } : {}
175
+ end
176
+
158
177
  def extract_dependencies_from_mappings
159
178
  dependencies = []
160
179
  @argument_mappings.each_value do |source|
@@ -0,0 +1,18 @@
1
+ # frozen_string_literal: true
2
+
3
+ module RubyReactor
4
+ module Error
5
+ # The reason a cut-off `compensate` receives when a manual undo runs it
6
+ # again (010 R-10): the run was aborted while it ran, and the original
7
+ # exception cannot be rebuilt from storage. It carries that exception's
8
+ # message, and its class name as `original_class`.
9
+ class RecordedFailure < Base
10
+ attr_reader :original_class
11
+
12
+ def initialize(message, original_class:)
13
+ super(message)
14
+ @original_class = original_class
15
+ end
16
+ end
17
+ end
18
+ end
@@ -51,8 +51,22 @@ module RubyReactor
51
51
  @context.undo_stack << step_info
52
52
  end
53
53
 
54
+ # A failing step whose `compensate` was cut off by an interruption, as
55
+ # an aborted run keeps it for manual undo (010 R-10, DM §4); nil once that
56
+ # `compensate` returned. The `arguments` key marks the record: 009's
57
+ # hand-off states carry none.
58
+ def pending_record
59
+ return unless @pending && !@pending[:compensated]
60
+
61
+ { "trigger" => "failure", "step" => @pending[:step].to_s, "compensated" => false,
62
+ "arguments" => ContextSerializer.serialize_value(@pending[:arguments] || {}),
63
+ "error" => recorded_error(@pending[:error]),
64
+ "failures" => ContextSerializer.serialize_value(@rollback_failures) }
65
+ end
66
+
54
67
  def handle_step_failure(step_config, error, arguments)
55
- @pending = { step: step_config.name, error: error, compensated: false }
68
+ @pending = { step: step_config.name, error: error, compensated: false,
69
+ arguments: step_config.rollback_arguments(arguments) }
56
70
  # A step whose OWN coordination acquisition failed (contention, or a
57
71
  # bad key proc) never ran its body — "no step compensates, the
58
72
  # contended step's work has not been attempted" (US3-1/T018), which
@@ -121,6 +135,12 @@ module RubyReactor
121
135
 
122
136
  private
123
137
 
138
+ def recorded_error(error)
139
+ return error if error.is_a?(String)
140
+
141
+ { "class" => error.class.name, "message" => error.respond_to?(:message) ? error.message : error.to_s }
142
+ end
143
+
124
144
  def middlewares
125
145
  @context.middlewares || RubyReactor::MiddlewareRunner.new([])
126
146
  end
@@ -21,6 +21,12 @@ module RubyReactor
21
21
  attr_reader :reactor_class, :context, :dependency_graph, :compensation_manager, :retry_manager, :result_handler,
22
22
  :step_executor, :result, :middlewares
23
23
 
24
+ # The owner of a run lock (`async:<id>`) the caller already holds: a Worker
25
+ # or `continue` that locked the run before loading it (010 R-02, R-03).
26
+ # `acquire_context_lock` then re-enters it (the lock counts by owner), and
27
+ # the outer holder releases last.
28
+ attr_writer :context_lock_owner
29
+
24
30
  def initialize(reactor_class, inputs = {}, context = nil)
25
31
  # Resume, map, compose and background workers build an Executor without
26
32
  # going through Reactor#run; the inferred wiring must exist there too.
@@ -109,6 +115,12 @@ module RubyReactor
109
115
  input_validator = InputValidator.new(@reactor_class, @context)
110
116
  input_validator.validate!
111
117
 
118
+ # A run in the caller's process holds its liveness lock too (010 R-01),
119
+ # so the sweeper never mistakes it for a dead worker's run and a Worker
120
+ # resumed by its hand-off waits for its final save. In a worker the
121
+ # executor runs `resume_execution`, which takes it there.
122
+ acquire_context_lock unless @context.inline_async_execution
123
+
112
124
  reset_held_lock_keys!
113
125
  acquire_locks_with_telemetry
114
126
 
@@ -168,6 +180,10 @@ module RubyReactor
168
180
  release_locks unless @parked
169
181
  leave_ordered_lock_scope
170
182
  save_context if persist_context? && !skip_context_persist?
183
+ # Released only after that save (J-3, 009 R-13): a Worker waiting on the
184
+ # lock then reads this run's final state.
185
+ @acquired_context_lock&.release
186
+ @acquired_context_lock = nil
171
187
 
172
188
  emit_lifecycle_completion(completed)
173
189
  end
@@ -221,6 +237,10 @@ module RubyReactor
221
237
  # holds it — composed/nested children resume inline under the root worker
222
238
  # and must not contend on the root's own key.
223
239
  acquire_context_lock
240
+ # The only place a claimed resume payload enters a context (010 R-04,
241
+ # J-5): under the run's lock, before the reactor-level lock or
242
+ # semaphore, so a contended resume still saves the payload it applied.
243
+ InterruptClaims.apply!(@context) if (@context.root_context || @context).equal?(@context)
224
244
 
225
245
  reset_held_lock_keys!
226
246
 
@@ -249,7 +269,6 @@ module RubyReactor
249
269
  @context.admit!
250
270
  prepare_for_resume
251
271
  save_context
252
- @past_gates = true
253
272
 
254
273
  @result = if @context.current_step
255
274
  execute_current_step_and_continue
@@ -313,6 +332,7 @@ module RubyReactor
313
332
  def undo_all(&after_pop)
314
333
  saved = @context.rollback && @context.rollback["failures"]
315
334
  @compensation_manager.restore_rollback_failures(saved) if saved
335
+ compensate_pending!
316
336
  @compensation_manager.rollback_completed_steps(&after_pop)
317
337
  clear_saved_rollback_failures
318
338
  rescue Error::RollbackHandedOff
@@ -390,12 +410,6 @@ module RubyReactor
390
410
  RubyReactor.reactor_storage_name(@reactor_class))
391
411
  end
392
412
 
393
- # True once `resume_execution` is past its reactor-level gates (context
394
- # lock, lock, semaphore, period): from there on it may have run steps.
395
- def past_gates?
396
- @past_gates == true
397
- end
398
-
399
413
  # Reached only by an interruption — a signal, an exit, out of memory, an
400
414
  # enclosing timeout (008 R-08, R-16); every other exception was rescued as
401
415
  # `Error::Rescuable` and rolled back. Running user rollback code now is
@@ -404,7 +418,13 @@ module RubyReactor
404
418
  # `Reactor#undo`; the `ensure` persists it, best effort. A worker run stays
405
419
  # `running` and its job is redelivered.
406
420
  def mark_aborted
407
- @context.status = :aborted unless @context.inline_async_execution
421
+ return if @context.inline_async_execution
422
+
423
+ @context.status = :aborted
424
+ # The failing step's `compensate`, if the interruption cut it off: a
425
+ # manual undo runs it again before the undo stack (010 R-10, J-9).
426
+ record = @compensation_manager.pending_record
427
+ @context.rollback = record if record
408
428
  end
409
429
 
410
430
  # A rollback run from a `rescue Error::Rescuable` body is outside the
@@ -663,6 +683,8 @@ module RubyReactor
663
683
  # defeat the guard. Only the root executor acquires — a composed/nested child
664
684
  # resumes inline under the root worker and shares the root's lock, so it must
665
685
  # not try to re-acquire the same key with a different owner (self-deadlock).
686
+ # Held by a worker's resume and by a run in the caller's process alike (010
687
+ # R-01); an outer holder's owner, when set, is re-entered (R-03).
666
688
  def acquire_context_lock
667
689
  root = @context.root_context || @context
668
690
  return unless root.equal?(@context) # only the root executor holds it
@@ -980,7 +1002,7 @@ module RubyReactor
980
1002
  # gives (I-7).
981
1003
  def finish_rollback(state)
982
1004
  @compensation_manager.restore_rollback_failures(state["failures"])
983
- return finish_undo if state["trigger"] == "undo"
1005
+ return finish_undo(state) if state["trigger"] == "undo"
984
1006
 
985
1007
  failure = ContextSerializer.deserialize_value(state["failure"])
986
1008
  step_config = state["step"] && @reactor_class.steps[state["step"].to_sym]
@@ -998,8 +1020,38 @@ module RubyReactor
998
1020
  @result_handler.handle_execution_error(e)
999
1021
  end
1000
1022
 
1001
- def finish_undo
1023
+ # A manual undo ends `cancelled`, or `failed` when it was the failure of
1024
+ # an interrupt whose payload attempts ran out (010 R-08).
1025
+ # An aborted run's failing step whose `compensate` an interruption cut off
1026
+ # (010 R-10, J-9): run it again, with the recorded arguments and reason,
1027
+ # before the undo stack. Only that record carries `arguments`; 009's
1028
+ # hand-off states, which name the same keys, are finished elsewhere.
1029
+ def compensate_pending!
1030
+ state = @context.rollback
1031
+ return unless state.is_a?(Hash) && state.key?("arguments") && state["compensated"] == false
1032
+
1033
+ step_config = state["step"] && @reactor_class.steps[state["step"].to_sym]
1034
+ return unless step_config
1035
+
1036
+ @compensation_manager.compensate(step_config, recorded_reason(state["error"]),
1037
+ ContextSerializer.deserialize_value(state["arguments"]))
1038
+ state["compensated"] = true
1039
+ end
1040
+
1041
+ def recorded_reason(error)
1042
+ return error unless error.is_a?(Hash)
1043
+
1044
+ Error::RecordedFailure.new(error["message"], original_class: error["class"])
1045
+ end
1046
+
1047
+ def finish_undo(state)
1002
1048
  @compensation_manager.rollback_completed_steps
1049
+ if state["failure_reason"]
1050
+ @context.status = "failed"
1051
+ @context.failure_reason = state["failure_reason"]
1052
+ return nil
1053
+ end
1054
+
1003
1055
  @context.cancelled = true
1004
1056
  @context.cancellation_reason = "Undo triggered"
1005
1057
  @context.status = "cancelled"
@@ -0,0 +1,81 @@
1
+ # frozen_string_literal: true
2
+
3
+ module RubyReactor
4
+ # A resume claims its interrupt before anything runs (010 R-04): the first
5
+ # `SET NX` wins, in every execution mode, and the claim holds the validated
6
+ # payload. Only the execution that owns the run's lock copies a claim into
7
+ # the context (J-5), at the start of `Executor#resume_execution`, so a
8
+ # resume never writes the run from outside it.
9
+ module InterruptClaims
10
+ module_function
11
+
12
+ # true when this call won the interrupt; false when a resume already did.
13
+ # `path` is the interrupt's step path (010 R-05); a nested one is keyed `a.b`.
14
+ def claim!(context, path, payload)
15
+ storage.claim_interrupt_resume(context.context_id, storage_name(context), key(Array(path)),
16
+ JSON.generate(ContextSerializer.serialize_value(payload)))
17
+ end
18
+
19
+ # true when a resume already claimed the interrupt, applied or not.
20
+ def claimed?(context, path)
21
+ storage.retrieve_interrupt_resumes(context.context_id, storage_name(context), [key(Array(path))]).any?
22
+ end
23
+
24
+ # `{ step => payload }` (a nested interrupt: `{ [child, step] => payload }`)
25
+ # for the claimed interrupts that have no result yet.
26
+ def unapplied(context)
27
+ paths = pending_interrupts(context)
28
+ return {} if paths.empty?
29
+
30
+ raw = storage.retrieve_interrupt_resumes(context.context_id, storage_name(context), paths.map { |p| key(p) })
31
+ paths.filter_map do |path|
32
+ if raw.key?(key(path))
33
+ [path.one? ? path.first : path,
34
+ ContextSerializer.deserialize_value(JSON.parse(raw[key(path)]))]
35
+ end
36
+ end.to_h
37
+ end
38
+
39
+ # Copies every unapplied claim into the context that paused at it (010 R-06).
40
+ # Returns their steps.
41
+ def apply!(context)
42
+ claims = unapplied(context)
43
+ claims.each do |step, payload|
44
+ path = Array(step)
45
+ path[0...-1].reduce(context) { |ctx, name| child(ctx, name) }.set_result(path.last, payload)
46
+ end
47
+ claims.keys
48
+ end
49
+
50
+ # Step paths of the interrupts without a result, here and in paused composed children.
51
+ def pending_interrupts(context, prefix = [])
52
+ steps = context.reactor_class.respond_to?(:steps) ? context.reactor_class.steps : {}
53
+ interrupts = steps.select do |name, config|
54
+ config.respond_to?(:interrupt?) && config.interrupt? && !context.has_result?(name)
55
+ end
56
+ own = interrupts.keys.map { |name| prefix + [name] }
57
+ nested = steps.keys.flat_map do |name|
58
+ (c = child(context, name)) ? pending_interrupts(c, prefix + [name]) : []
59
+ end
60
+ own + nested
61
+ end
62
+
63
+ def child(context, name)
64
+ entry = context.composed_contexts[name] || context.composed_contexts[name.to_s]
65
+ c = entry.is_a?(Hash) && (entry[:context] || entry["context"])
66
+ c if c.is_a?(Context) && c.status.to_s == "paused"
67
+ end
68
+
69
+ def key(path)
70
+ path.join(".")
71
+ end
72
+
73
+ def storage_name(context)
74
+ RubyReactor.reactor_storage_name(context.reactor_class)
75
+ end
76
+
77
+ def storage
78
+ RubyReactor.configuration.storage_adapter
79
+ end
80
+ end
81
+ end
@@ -115,6 +115,127 @@ module RubyReactor
115
115
  missing.map { |index| rollback_entry(index, :context_unavailable, "context expired") }
116
116
  end
117
117
 
118
+ # 010 R-13 (P-6): one call to the map's `undo_all` block, with the
119
+ # completed elements' results (index order, read lazily, never all at
120
+ # once), instead of replaying each element. A fan-out map reads its
121
+ # result slots and dispatches no rollback jobs; an inline map reads its
122
+ # element contexts one at a time, and an element an interruption left
123
+ # `aborted` still replays its own completed steps (it has no result).
124
+ def bulk_rollback(map_id, step_name, block)
125
+ failures, count, results =
126
+ dispatched_map_id(step_name) ? fan_out_completed(map_id) : inline_completed(map_id, step_name)
127
+ failures << call_undo_all(block, results, step_name, count) if count.positive?
128
+ finish_rollback(step_name, count, failures.compact)
129
+ end
130
+
131
+ def fan_out_completed(map_id)
132
+ meta = rollback_storage.retrieve_map_metadata(map_id, context.reactor_class.name)
133
+ return [[rollback_entry(nil, :context_unavailable, "map records expired")], 0, []] unless meta
134
+
135
+ total = meta["count"].to_i
136
+ failures = []
137
+ count = 0
138
+ each_result_slot(map_id, total) do |index, raw|
139
+ next failures << rollback_entry(index, :context_unavailable, "context expired") if raw.nil?
140
+
141
+ count += 1 if completed_slot?(raw)
142
+ end
143
+ results = Enumerator.new do |yielder|
144
+ each_result_slot(map_id, total) do |_index, raw|
145
+ yielder << ContextSerializer.deserialize_value(raw) if raw && completed_slot?(raw)
146
+ end
147
+ end
148
+ [failures, count, results.lazy]
149
+ end
150
+
151
+ # ponytail: 1,000 slots per HMGET bounds memory; raise it if round trips dominate.
152
+ def each_result_slot(map_id, total, &block)
153
+ (0...total).each_slice(RubyReactor::Map::ResultEnumerator::DEFAULT_BATCH_SIZE) do |indexes|
154
+ slots = rollback_storage.retrieve_map_result_slots(map_id, context.reactor_class.name, indexes)
155
+ indexes.zip(slots).each(&block)
156
+ end
157
+ end
158
+
159
+ def completed_slot?(raw)
160
+ !(raw.is_a?(Hash) && (raw.key?("_error") || raw.key?("_halt") || raw.key?("_skipped")))
161
+ end
162
+
163
+ # Pass 1 replays aborted elements and counts completed ones; the lazy
164
+ # pass 2 is what the block reads.
165
+ def inline_completed(map_id, step_name)
166
+ total = rollback_storage.count_map_element_context_ids(map_id, context.reactor_class.name)
167
+ failures = []
168
+ seen = Set.new
169
+ unnamed = 0
170
+ count = 0
171
+ each_element(map_id, total) do |id, element|
172
+ next unnamed += 1 unless element
173
+
174
+ seen << Utils::FetchIndifferent.call(element.map_metadata || {}, :index).to_i
175
+ case element.status.to_s
176
+ when "completed" then count += 1
177
+ when "aborted"
178
+ outcome = inline_element_rollback(map_id, id, step_name)
179
+ failures.concat(ContextSerializer.deserialize_value(outcome["failures"]))
180
+ end
181
+ end
182
+ unseen = ->(indexes) { indexes.reject { |index| seen.include?(index) } }
183
+ results = Enumerator.new do |yielder|
184
+ each_element(map_id, total) do |_id, element|
185
+ yielder << element_result(element) if element && element.status.to_s == "completed"
186
+ end
187
+ end
188
+ [unavailable_entries(step_name, unnamed, unseen) + failures, count, results.lazy]
189
+ end
190
+
191
+ # Element contexts in index order (head of the index first), one loaded
192
+ # at a time, `Map::ROLLBACK_CHUNK` ids per read.
193
+ def each_element(map_id, total)
194
+ storage_name = RubyReactor.reactor_storage_name(element_class)
195
+ (0...total).each_slice(RubyReactor::Map::ROLLBACK_CHUNK) do |heads|
196
+ ids = rollback_storage.retrieve_map_element_context_ids_from_tail(
197
+ map_id, context.reactor_class.name, total - 1 - heads.last, heads.size, total: total
198
+ ).reverse
199
+ ids.each do |id|
200
+ data = rollback_storage.retrieve_context(id, storage_name)
201
+ yield id, data && Context.deserialize_from_retry(data)
202
+ end
203
+ end
204
+ end
205
+
206
+ # What the element returned, as the map collected it
207
+ # (`ResultHandler#final_result`): its `returns` step, else every result.
208
+ def element_result(element)
209
+ klass = element.reactor_class || element_class
210
+ klass.return_step ? element.get_result(klass.return_step) : element.intermediate_results
211
+ end
212
+
213
+ def call_undo_all(block, results, step_name, count)
214
+ log_rollback("undo_all.started", step_name, count: count)
215
+ outcome = begin
216
+ block.call(results)
217
+ rescue RubyReactor::Error::Rescuable => e
218
+ e
219
+ end
220
+ failure = undo_all_failure(step_name, outcome)
221
+ context.append_execution_trace({ type: :undo_all, step: step_name, count: count, timestamp: Time.now,
222
+ result: failure ? failure[:message] : :ok })
223
+ log_rollback("undo_all.completed", step_name, count: count, failed: failure ? 1 : 0)
224
+ failure
225
+ end
226
+
227
+ def undo_all_failure(step_name, outcome)
228
+ return unless outcome.is_a?(Exception) || outcome.is_a?(RubyReactor::Failure)
229
+
230
+ error = outcome.is_a?(RubyReactor::Failure) ? outcome.error : outcome
231
+ { step: step_name.to_sym, kind: :undo_all, reason: outcome.is_a?(Exception) ? :raised : :returned_failure,
232
+ message: error.respond_to?(:message) ? error.message : error.to_s }
233
+ end
234
+
235
+ def rollback_storage
236
+ RubyReactor.configuration.storage_adapter
237
+ end
238
+
118
239
  def finish_rollback(step_name, total, failures)
119
240
  log_rollback("completed", step_name, total: total, failed: failures.size)
120
241
  return RubyReactor.Success() if failures.empty?