ruby_reactor 0.8.4 → 0.8.5

data/lib/ruby_reactor/web/public/index.html

@@ -5,7 +5,7 @@
     <link rel="icon" type="image/svg+xml" href="./vite.svg" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
     <title>ui</title>
-    <script type="module" crossorigin src="./assets/index-CeZU-ESu.js"></script>
+    <script type="module" crossorigin src="./assets/index-CQbgHtd0.js"></script>
     <link rel="stylesheet" crossorigin href="./assets/index-BQvIWPdx.css">
   </head>
   <body>

data/lib/ruby_reactor/worker.rb

@@ -8,7 +8,7 @@ module RubyReactor
   # `Adapters::ActiveJob::Compat` on ActiveJob::Base) — nothing here references
   # a specific backend.
   module Worker
-    TERMINAL_STATUSES = %w[completed failed cancelled skipped].freeze
+    TERMINAL_STATUSES = %w[completed failed cancelled skipped aborted].freeze
 
     # Use the error's `retry_after_seconds` hint when available
     # (RateLimit::ExceededError carries the time until the bucket rolls);
@@ -89,6 +89,8 @@ module RubyReactor
       reactor_class_name ||= RubyReactor.reactor_storage_name(nil)
       data = RubyReactor.configuration.storage_adapter.retrieve_context(context_id, reactor_class_name)
       return if data.nil?
+      # An aborted run is never resumed forward: only a manual undo applies (008 R-08).
+      return if (data["status"] || data[:status]).to_s == "aborted"
 
       begin
         context = ContextSerializer.deserialize_hash(data)

data/lib/ruby_reactor.rb

@@ -103,10 +103,11 @@ module RubyReactor
     undef_method :skipped?
   end
 
-  # Marks a single step as skipped while the reactor continues. Behaves
-  # exactly like Success — the value flows to dependants via `result(:step)` —
-  # except `skipped?` is true and the step is not enrolled for rollback
-  # (nothing happened, so there is nothing to undo).
+  # Marks a single step as skipped: an instrumentation mark (008 R-19). The
+  # run continues exactly as for a Success — the value flows to dependants via
+  # `result(:step)`, hand-offs and period marks happen — except `skipped?` is
+  # true, the trace records it, and the step is not enrolled for undo (it had
+  # nothing to do, so there is nothing to revert).
   class Skipped < Success
     # Sentinel distinguishing "no value argument given" (the old Halt call
     # shape reused this class's name) from an explicit `Skipped(nil)`.
@@ -133,6 +134,10 @@ module RubyReactor
   end
 
   class Failure
+    # A stack overflow's backtrace runs to thousands of frames; the Failure is
+    # stored with the context, so keep the frames that locate the cause.
+    MAX_BACKTRACE_FRAMES = 100
+
     attr_reader :error, :retryable, :step_name, :inputs, :backtrace, :reactor_name, :step_arguments, :exception_class,
                 :file_path, :line_number, :code_snippet, :validation_errors, :rollback_failures
 
@@ -173,7 +178,7 @@ module RubyReactor
       @inputs = inputs
       @step_arguments = step_arguments
       raw_backtrace ||= backtrace || (@error.respond_to?(:backtrace) ? @error.backtrace : caller)
-      @backtrace = filter_backtrace(raw_backtrace)
+      @backtrace = cap_backtrace(filter_backtrace(raw_backtrace))
       @redact_inputs = redact_inputs
       @exception_class = exception_class || (@error.is_a?(Exception) ? @error.class.name : nil)
       @file_path = file_path
@@ -294,6 +299,12 @@ module RubyReactor
       msg << backtrace.take(10).map { |line| "  #{line}" }.join("\n")
     end
 
+    def cap_backtrace(backtrace)
+      return backtrace if backtrace.nil? || backtrace.size <= MAX_BACKTRACE_FRAMES
+
+      backtrace.first(MAX_BACKTRACE_FRAMES) << "... #{backtrace.size - MAX_BACKTRACE_FRAMES} more frames"
+    end
+
     def filter_backtrace(backtrace)
       return backtrace if ENV["RUBY_REACTOR_DEBUG"] == "true"
       return backtrace if backtrace.nil? || backtrace.empty?
@@ -345,7 +356,7 @@ module RubyReactor
       }
     end
 
-    ROLLBACK_FAILURE_SYMBOLS = %i[step kind reason].freeze
+    ROLLBACK_FAILURE_SYMBOLS = %i[step kind reason map_step].freeze
     private_constant :ROLLBACK_FAILURE_SYMBOLS
 
     # A stored failure comes back with string keys (and, through plain JSON,

data/specs/007-execution-flow-analysis/analysis/README.md

@@ -0,0 +1,147 @@
+# Execution Flow & Compensation Analysis
+
+Research into how RubyReactor orders forward execution and rollback across every construct, under
+locks, retries and failures. Its purpose is to judge whether compensation is **predictable** and
+whether the DSL makes it **visible**. Documentation only: no library, test-suite or demo-app change.
+
+## Scope & baseline
+
+- **Code**: commit `faf90e8d` (ruby_reactor 0.8.3, with step-scoped retries #61 and inputs
+  protection #63).
+- **Constructs**: plain steps, `compose`, `map` (inline and `fan_out`, `fail_fast` on/off),
+  `async_step`, `async_reactor`, `background` reactors, interrupts, manual `cancel`/`undo`.
+- **Conditions**: reactor- and step-level locks/semaphores (ordered lock, rate limit and period by
+  reading), retries (inline, worker, element, unit), and every failure kind in
+  [execution-order.md §1](execution-order.md#failure-kinds--path).
+- **Evidence**: 63 probe scenarios run against real Redis through the real worker bodies
+  (Sidekiq fake mode + drain; `inline!` only where labelled). **64/64 match** the sequences quoted
+  in this report. Re-run: [../quickstart.md](../quickstart.md).
+- **Out of scope**: the ActiveJob backend is not probed separately (its adapters delegate to the
+  same shared bodies). Neither are rate limits and periods beyond their "never re-taken for rollback"
+  rule, the dashboard, or OpenTelemetry spans.
+- **Documentation**: README.md and `./documentation` are **audited, not edited**
+  ([findings-and-options.md §2](findings-and-options.md#2-documentation-audit)). Writing current
+  behavior in as contract before the follow-up decision would lock in behavior that may be
+  unintended. Each remedy updates the docs it affects.
+
+## How to read
+
+- *compensate* = the failing step's own cleanup; *undo* = rollback of a completed step;
+  *rollback* = both; *left in place* = completed work nothing rolls back; *unit* = an
+  `async_step`/`async_reactor` dispatch.
+- Evidence labels: `[R: file:line]` read in source · `[O: S-…]` observed, see
+  [../evidence/output.txt](../evidence/output.txt) · `[T: spec:line]` covered by an existing spec.
+- Invariant status: HOLDS · VIOLATED · CONDITIONAL · UNDETERMINED.
+  Finding severity: High · Medium · Low
+  ([findings-and-options.md](findings-and-options.md) header).
+
+---
+
+## Answers
+
+### Q1 · Are already-executed map elements compensated individually?
+
+**No.** Only the element that **fails** is rolled back, individually, by its own reactor.
+Elements that already **succeeded** are never compensated or undone. That holds whether the map
+itself fails or a later step fails.
+
+| Situation | Failed element | Elements that succeeded | Evidence |
+|---|---|---|---|
+| Inline, `fail_fast` (default), element k fails | compensated + its steps undone | **left in place**: elements before k. Elements after k never run | [O: S-map-01] |
+| Fan-out, `fail_fast`, element k fails | same, in its own job | **left in place**: whichever elements started before the failure. The set depends on **job scheduling** | [O: S-map-04] [O: S-map-04b] |
+| `fail_fast false` (inline or fan-out) | rolled back individually | kept. The map **succeeds** and the consumer inspects the results | [O: S-map-02] [O: S-map-05] |
+| Map completed, a **later step** fails | — | **all left in place** | [O: S-map-03] [O: S-map-06] |
+| Map inside a compose child / compose inside an element | same rules, nested | same | [O: S-map-07] [O: S-map-08] |
+
+Why: `MapStep#compensate` is a stub (`# TODO: Implement compensation for map steps` → `Success()`)
+[R: lib/ruby_reactor/step/map_step.rb:29-31], the map has no `undo`
+[R: lib/ruby_reactor/step.rb:57], and the DSL offers no hook
+[R: lib/ruby_reactor/dsl/map_builder.rb:130-131]. `rollback_failures` stays empty, because nothing
+is attempted. → [F-01](findings-and-options.md#f-01--high--map-elements-that-already-succeeded-are-never-rolled-back),
+[F-05](findings-and-options.md#f-05--medium--fan-out-fail-fast-leaves-a-scheduling-dependent-set-of-elements-in-place),
+INV-19/20/22 in [invariants.md](invariants.md#d-map). No existing spec covers it.
+
+### Q2 · When a composed reactor fails, are previously completed composed reactors compensated?
+
+**Yes.** Composition is fully wired into rollback, at any depth and in worker mode:
+
+1. The failing child rolls **itself** back first (its compensate, then its own undos).
+2. The parent compensates the compose step (a no-op by then), then undoes its own completed steps
+   newest-first. **Each earlier compose is undone by replaying its child's undo stack in reverse.**
+
+```text
+compose x(x1 → x2) → compose y(y1 → y2 ✗)
+run:x.x1 run:x.x2 run:y.y1 run:y.y2 compensate:y.y2 undo:y.y1 undo:x.x2 undo:x.x1 ⇒ failure(y)
+```
+
+[O: S-compose-03] · also [O: S-compose-01] [O: S-compose-02] [O: S-compose-04] [O: S-compose-06] ·
+[T: spec/compose_spec.rb:192, :197] · INV-15–18.
+
+**Except** in three cases:
+
+- `compose` with `retries`: a retried child **resumes** with its already-undone steps counted as
+  done, so they are not re-run and their stale results flow on
+  ([F-02](findings-and-options.md#f-02--high--compose-with-retries-resumes-a-child-whose-earlier-steps-were-already-undone),
+  [O: S-compose-05]).
+- A child's `Halt` does **not** stop the parent. The parent continues with `nil`
+  ([F-07](findings-and-options.md#f-07--medium--halt-means-different-things-at-different-nesting-levels),
+  [O: S-compose-08]).
+- A `map` or async unit **inside** a child keeps its own rules (Q1, and
+  [F-04](findings-and-options.md#f-04--high--an-async_steps-own-compensate--undo-never-run-the-docs-say-they-do)/[F-09](findings-and-options.md#f-09--medium--dispatched-units-run-after-their-dispatcher-rolled-back-including-units-of-a-failed-map-element)).
+
+### Q3 · Would `compensate_all` / `compensate_each` on `map` close a real gap?
+
+**Yes, the gap is real**: INV-19 and INV-20 are VIOLATED and have no spec coverage. Both proposed
+shapes would close it, and they are complementary rather than alternatives: per-element vs bulk is
+a matter of how the author wants to clean up. Before either works, three things must be settled:
+
+1. **Two moments, not one.** Cleanup is needed when the map **fails** (the library's
+   *compensate*) and when a **later step** fails after the map completed (the library's *undo*).
+   `compensate_*` as named covers only the first. The hooks should be named for both moments, or one
+   block should be documented to cover both.
+2. **Plumbing.** In fan-out mode the map step must be on the parent's undo stack (today it never is,
+   [R: lib/ruby_reactor/map/helpers.rb:97]). Raw per-element results must also stay available even
+   when a `collect` block transformed them.
+3. **In-flight fan-out elements.** Elements still running when the hook fires finish afterwards and
+   escape it, unless the collector waits for them
+   ([O-05-a](findings-and-options.md#options-for-f-05-scheduling-dependent-fan-out-leftovers)).
+
+There is also a third shape: implicit **element-undo replay**, which makes `map` behave like
+`compose` and needs no new DSL. It is the most consistent option, but the only breaking one.
+Full comparison on fail_fast, inline/fan-out, retries, later failures and data availability:
+[O-01 evaluation](findings-and-options.md#o-01-evaluation-compensate_all-vs-compensate_each).
+
+---
+
+## Top findings
+
+**High**
+
+- [F-01](findings-and-options.md#f-01--high--map-elements-that-already-succeeded-are-never-rolled-back): map elements that succeeded are never rolled back (map failure or later failure).
+- [F-02](findings-and-options.md#f-02--high--compose-with-retries-resumes-a-child-whose-earlier-steps-were-already-undone): `compose` + `retries` resumes a child whose earlier steps were already undone.
+- [F-03](findings-and-options.md#f-03--high--some-failures-skip-rollback-entirely): argument-resolution errors and non-`StandardError` exceptions skip rollback entirely.
+- [F-04](findings-and-options.md#f-04--high--an-async_steps-own-compensate--undo-never-run-the-docs-say-they-do): an `async_step`'s own `compensate`/`undo` never run. The docs say they do.
+
+**Medium**: F-05 scheduling-dependent fan-out leftovers · F-06 raising `where`/`guard`
+compensates a never-run step · F-07 nested `Halt` inconsistency · F-08 `Reactor.undo` outside the
+reactor lock · F-09 units run after their dispatcher's rollback (also escaping a failed map
+element) · F-10 composite rollback coverage is asymmetric and invisible in the DSL.
+
+**Low**: F-11 to F-16 (lock gap, async retry telemetry, missing failure attribution, dead collect
+default, lock event key asymmetry, interrupt status docs).
+
+What reliably **holds**: compensate-then-reverse-undo ordering (DAG included). Rollback failures
+never stop the rollback and are always reported. Retries always finish before compensation, which
+runs exactly once. Never-started steps are not compensated (except F-06). Composition rollback is
+full, at any depth. Worker runs match inline order. Reactor locks are held through rollback.
+Step locks are re-taken for it. A crash re-drives from the last checkpoint.
+29 of 41 invariants hold ([invariants.md](invariants.md#coverage-summary)).
+
+## Files
+
+| File | Answers |
+|---|---|
+| [execution-order.md](execution-order.md) | Rollback algorithm, failure-kind table, construct lifecycles, full order matrix, lock and retry cross-sections |
+| [invariants.md](invariants.md) | 41 invariants with status, evidence and existing-spec coverage |
+| [findings-and-options.md](findings-and-options.md) | 16 ranked findings, documentation audit, improvement options (proposals) |
+| [../evidence/](../evidence/) | Probe harness, 7 probe files, `output.txt` transcript |

data/specs/007-execution-flow-analysis/analysis/execution-order.md

@@ -0,0 +1,359 @@
+# Execution Order & Rollback
+
+What runs, in what order, and what is **left in place** when something fails, for every
+construct RubyReactor offers. Baseline commit `faf90e8d` (ruby_reactor 0.8.3 + #61, #63).
+
+**Updated for 008 (Reliable Rollback Across Constructs).** Rows and rules marked *changed in 008*
+describe the behavior after [specs/008-rollback-reliability](../../008-rollback-reliability/spec.md);
+the text they replace is kept in git history. The transcript was re-run: 64 scenarios, 64 match (S-edge-03b added by the 2026-09-27 revision).
+Line references `[R: …]` in unchanged rows still point at the baseline.
+
+Labels: `[R: file:line]` = read in source · `[O: S-…]` = observed, see
+[`../evidence/output.txt`](../evidence/output.txt) · `[T: spec:line]` = existing spec.
+*Compensate* = the failing step's own cleanup; *undo* = rollback of a completed step;
+*rollback* = both. *Unit* = an `async_step` or `async_reactor` dispatch.
+
+Contents: [1. Rollback algorithm](#1-rollback-algorithm) ·
+[2. Construct lifecycles](#2-construct-lifecycles) · [3. Order matrix](#3-order-matrix) ·
+[4. Cross-cutting: locks](#4-cross-cutting-locks) · [5. Cross-cutting: retries](#5-cross-cutting-retries)
+
+---
+
+## 1. Rollback algorithm
+
+One mechanism does all rollback: the **undo stack** of the reactor execution that owns the step
+(`Context#undo_stack`, serialized with the context `[R: lib/ruby_reactor/context.rb:173]`).
+
+```mermaid
+flowchart TD
+  S[Step reaches a result] --> K{Kind}
+  K -->|Success| P[Push step, args, result on undo stack<br/>unless async unit / Skipped]
+  P --> N[Next ready step]
+  K -->|Skipped| N
+  K -->|Halt| H[Stop. No rollback]
+  K -->|Failure after retries| NS{Body never started?<br/>own contention / key error /<br/>dispatch refused / argument error}
+  NS -->|yes| U
+  NS -->|no| C[compensate failing step]
+  C --> U[Undo stack, newest first:<br/>undo each completed step]
+  U --> F[Failure: rollback_failures lists<br/>every compensate/undo that did not complete]
+```
+
+Rules, each confirmed on plain steps:
+
+| # | Rule | Evidence |
+|---|---|---|
+| R1 | Steps run in dependency order. Among ready steps, definition order. Each `Success` pushes `{step, arguments, result}` on the undo stack. | `[R: lib/ruby_reactor/executor/result_handler.rb:137]` `[O: S-plain-09]` |
+| R2 | On failure the failing step's **compensate runs first**, then **every completed step's undo, newest first** (reverse completion order, also across DAG branches). No later step runs. | `[R: lib/ruby_reactor/executor/compensation_manager.rb:35]` `[R: …/compensation_manager.rb:69]` `[O: S-plain-01, S-plain-02, S-plain-09]` |
+| R3 | A step whose own coordination was never acquired (contention, key error, refused dispatch) is **not** compensated. Earlier steps are still undone. | `[R: …/compensation_manager.rb:11, :44]` `[O: S-lock-03, S-edge-01]` |
+| R4 | A compensate or undo that returns `Failure` or raises does **not** stop the rollback. It is listed on `Failure#rollback_failures`. A compensate failure also turns the final error into `CompensationError` ("Execution error: …", no `step_name`). | `[R: …/compensation_manager.rb:57, :202]` `[O: S-plain-03, S-plain-04, S-compose-07]` |
+| R5 | `Halt` stops without rollback. `Skipped` steps are never pushed, so they are never undone. *Since 008 (R-19)* `Skipped` changes nothing else: an `after:` hand-off and a period mark treat it as a completed step. | `[R: …/result_handler.rb handle_skipped]` `[O: S-plain-05, S-plain-06]` |
+| R6 | Async units (`async_step`, `async_reactor`) are **never pushed**: the parent never undoes them. *Changed in 008:* the step says so itself (`StepConfig#rollback_tracked?` is false for async units); the coordinator has no async special case. | `[R: lib/ruby_reactor/dsl/step_builder.rb rollback_tracked?]` `[O: S-async-03, S-async-06]` |
+| R7 | *Changed in 008.* Every exception after completed work rolls back, `StandardError` or not (R-16). An argument source/transform/result path that raises (`ArgumentResolutionError`) is a never-started failure: the step is not compensated, completed steps are undone. Any other exception outside a step body rolls back too, attributed to the executing step. Only an interruption (signal, exit, out of memory, enclosing timeout) runs no rollback: a caller-process run is stored `aborted` for a manual `Reactor.undo(id)`; a worker run stays `running` and is redelivered. (`where`/`guard` were removed, R-15.) | `[R: lib/ruby_reactor/error/rescuable.rb]` `[R: lib/ruby_reactor/executor.rb mark_aborted]` `[O: S-plain-07, S-edge-03, S-edge-03b]` |
+| R8 | Rollback runs in whichever process detects the failure: the caller (inline), the reactor worker, the map collector, or (for units) nobody. | §2 |
+
+### Failure kinds → path
+
+| Failure kind | Where it happens | Failing step compensated? | Completed steps undone? | Evidence |
+|---|---|---|---|---|
+| Body returns `Failure` / raises `StandardError` | body | yes | yes | `[O: S-plain-01, S-plain-02]` |
+| Retries exhausted | body, last attempt | yes, **once** | yes | `[O: S-retry-01, S-retry-04, S-bg-03]` |
+| `validate_output` fails | after body | yes (side effect exists) | yes | `[R: …/result_handler.rb:277]` `[O: S-plain-08]` |
+| Step input/argument validation fails | before body | no | yes | `[O: S-edge-05]` |
+| Argument source/transform/result path raises `StandardError` (*changed in 008*) | before body | no (never started, `ArgumentResolutionError`) | yes; failure names the step | `[O: S-plain-07]` |
+| Any other `StandardError` outside a step body (*changed in 008*) | executor | — | yes; failure names the executing step | `spec/ruby_reactor/rollback/failure_rollback_spec.rb` |
+| Non-`StandardError` exception in body, not an interruption (`NotImplementedError`, custom `Exception`; *changed in 008*) | body | yes | yes; failure names the step and the original class | `[O: S-edge-03]` `spec/ruby_reactor/rollback/failure_rollback_spec.rb` |
+| Interruption in body (signal incl. `Interrupt`, `SystemExit`, `NoMemoryError`, enclosing timeout; *changed in 008*) | body | no | no rollback code runs; the exception propagates unchanged; a caller-process run is stored **`aborted`** with the entries not yet undone, and `Reactor.undo(id)` rolls it back | `[O: S-edge-03b]` `spec/ruby_reactor/rollback/aborted_execution_spec.rb` |
+| Own lock/semaphore contended (inline) | before body | no | yes | `[O: S-lock-03]` |
+| Own contention past `lock_snooze_max_attempts` (worker) | before body | no | yes | `[O: S-edge-01]` |
+| Reader's `result(:unit)` wait times out | argument resolution | no (never started; *changed in 008:* wrapped as the reader's `ArgumentResolutionError`, so the failure names the reader) | yes | `[O: S-async-08]` |
+| Compensate fails | rollback | — | yes, continues; *changed in 008:* the `CompensationError` failure names the step | `[O: S-plain-03]` |
+| Undo fails / raises | rollback | — | yes, continues | `[O: S-plain-04]` |
+| Undo cannot re-take its step lock within `rollback_wait` | rollback | — | that undo **skipped**, reported | `[O: S-lock-05]` |
+| Reactor input validation | before any step | — | nothing ran | `[O: S-edge-06]` |
+| `Halt` returned | body | no | **no** (by design) | `[O: S-plain-05]` |
+| Interrupt payload invalid past `max_attempts` | `continue` | — | yes | `[O: S-intr-02]` |
+| Worker crash (process dies) | anywhere | no | no; the job is redelivered and resumes from the last checkpoint | `[O: S-edge-02]` |
+
+---
+
+## 2. Construct lifecycles
+
+### 2.1 Step (inline)
+
+1. Resolve arguments (`result(...)`, `input(...)`, transforms). This is **outside** the step's
+   rescue `[R: lib/ruby_reactor/executor/step_executor.rb:83]`.
+2. Retry loop `[R: lib/ruby_reactor/executor/retry_manager.rb:11]`. Each attempt runs
+   `argument validation → step coordination (lock, semaphore, rate limit…) → body` (`where`/`guard`
+   were removed in 008).
+3. Result handling: Success pushes onto the undo stack. Failure (after the last attempt) runs rollback (§1).
+4. **Rollback hooks**: `compensate` (this step failing), `undo` (a later step failing).
+5. **Locks**: a step-level lock is held only around the body. Rollback re-takes it (§4).
+
+### 2.2 `compose`
+
+1. `ComposeStep#run` builds (or, on resume, **reuses**) the child context and runs the child
+   reactor inline, in the same process, with its own undo stack
+   `[R: lib/ruby_reactor/step/compose_step.rb:10, :97]`. *Changed in 008:* a stored child whose
+   status is `failed` is a previous **retry attempt** that already rolled itself back, so the
+   compose starts a **fresh** child and records a `compose_attempt_discarded` trace entry (with
+   that attempt's `rollback_failures`). A running/paused/parked child is still resumed.
+2. Child fails → the child rolls itself back (child compensate, child undos) **before** returning
+   `Failure` to the parent. The parent then compensates the compose step and undoes its own
+   completed steps.
+3. Child succeeds → the compose step is pushed on the parent's undo stack. Its `undo` (= its
+   `compensate`, the same method) replays the **child's** undo stack newest-first
+   `[R: …/compose_step.rb:31, :47]`.
+4. **Rollback hooks**: none declarable. `ComposeBuilder` has no `compensate`/`undo`
+   `[R: lib/ruby_reactor/dsl/compose_builder.rb:60]`. *Since 008*, `retries` on a compose is rejected
+   at definition time: the child's own steps retry `[O: S-compose-05, S-compose-05b]`.
+5. **Locks**: the child re-enters the parent's reactor lock (same root owner, counted) and hands it
+   back without releasing the parent's hold `[O: S-lock-06]`.
+
+**Resume entry points (008 R-05 audit).** Every path that resumes a stored context, and whether
+it can resume one after that context rolled back:
+
+| Entry point | Can resume after a rollback? |
+| --- | --- |
+| `Worker` (root / `async_reactor` child) | no: `failed`/`cancelled` runs have nothing left to run, and an `aborted` run is skipped |
+| `Map::Collector` → parent `resume_execution` | no: it resumes the parent before any rollback, and skips a finished parent |
+| `Map::ElementExecutor` requeue (retry / park) | no: it requeues before the element rolls back |
+| `Reactor#continue` (interrupt) | no: after an undo the context is `cancelled` |
+| `ComposeStep#run` on a retry | was **yes** at the baseline; removed in 008: a compose cannot be retried (R-14) |
+
+### 2.3 `map`, inline (default)
+
+1. `MapStep#run_inline` runs one child reactor execution **per element, sequentially**, each with
+   its own context and undo stack `[R: lib/ruby_reactor/step/map_step.rb:100]`.
+2. An element fails → that element's executor rolls **that element** back (its compensate + its
+   undos). With `fail_fast` (default) the map stops at once and returns the element's `Failure`
+   `[R: …/map_step.rb:112]`. Otherwise the `Failure` is collected and the map continues.
+3. The parent then treats the map step as failed and compensates it. *Changed in 008:*
+   `MapStep#compensate` replays the undo stack of every element whose stored context is
+   `completed`, **highest index first**, found through the map's element-context index; failed
+   elements already rolled themselves back. Then the parent undoes its earlier steps
+   `[O: S-map-01]`. A raising `collect` counts as a map failure.
+4. Map succeeds → the map step is pushed on the parent undo stack. *Changed in 008:* its `undo`
+   is the same element replay, so a later failure or a manual undo rolls back every completed
+   element `[O: S-map-03]`. Element rollback failures carry `map_step`/`element_index`; an
+   expired element context is reported `context_unavailable`.
+5. **Rollback hooks**: none declarable. `MapBuilder` builds `compensate_block: nil, undo_block: nil`
+   and has no `retries` `[R: lib/ruby_reactor/dsl/map_builder.rb:130-131]`.
+
+### 2.4 `map` with `fan_out`
+
+1. The parent persists its context, sets a counter, dispatches one `MapElementWorker` job per
+   element (in `batch_size` batches), enqueues an eager collector, and **stops**
+   (`DispatchResult`) `[R: lib/ruby_reactor/step/map_step.rb:172-195]`.
+2. Each element job runs its element reactor (`Map::ElementExecutor`). A failed element is rolled
+   back inside its own job. With `fail_fast` it records the failure and triggers the collector
+   `[R: lib/ruby_reactor/map/element_executor.rb:176-195]`.
+3. An element job that **starts after** a fail-fast failure is recorded skips itself
+   `[R: …/element_executor.rb:61, :145]`. *Changed in 008:* it stores a `_skipped` result slot,
+   and the dispatcher settles every index it never dispatched the same way. The collector applies
+   the failure only once **every index has settled**, so elements in flight finish first. Which
+   elements run still depends on scheduling; what is left in place does not: nothing
+   `[O: S-map-04, S-map-04b]`.
+4. The collector resumes the parent in a worker. On failure it compensates the map step (the
+   element replay above) and runs the parent's rollback there. On success, *since 008*, it
+   pushes the map step on the parent's undo stack with an empty record, so a later failure
+   undoes every element `[O: S-map-06]`.
+5. Element retries re-enqueue the element job, so other elements run in between
+   `[O: S-map-09]`. Inline map retries sleep in place `[O: S-map-10]`.
+
+### 2.5 `async_step`
+
+1. Dispatch: durable record + enqueue, then the step is marked complete **for scheduling only**
+   (no result) and the parent keeps going `[R: lib/ruby_reactor/executor/async_step_dispatch.rb:25-47]`.
+   Nothing is pushed on the parent's undo stack.
+2. `StepWorker` runs the body in its own job, retries in a **loop inside that job**, and writes a
+   terminal record. *Changed in 008:* after the **final** attempt fails (body started), it calls
+   the step's `compensate` **once**, in that job, and records `compensation` on the unit's record
+   before completing it. An inline `undo` on `async_step` is rejected at class definition (a step
+   class's `undo` is warned about) `[O: S-async-01, S-async-07]`.
+3. A reader (`result(:u)`) blocks until the record is terminal and receives the value, or the
+   `Failure` **object** as an argument `[R: lib/ruby_reactor/template/result.rb:68-88]`. The parent
+   is affected only if the reader itself returns `Failure`. Then the **reader** is compensated and
+   the parent's completed steps are undone. The unit is not compensated a second time
+   `[O: S-async-02]`.
+4. A parent that fails and rolls back does not stop or undo the unit. In fake-queue ordering the
+   unit body runs **after** the parent's rollback `[O: S-async-03]`.
+
+### 2.6 `async_reactor`
+
+1. Dispatch validates the child inputs, persists the child as its own execution (linked only by
+   `parent_context_id`), and enqueues it. The step returns `Success(nil)` and is not pushed
+   `[R: lib/ruby_reactor/step/async_reactor_step.rb:5-8, :146]`.
+2. The child is an ordinary reactor. On its own failure it rolls back its own steps in its worker
+   `[O: S-async-04]`.
+3. A reader gets the child's real `Success`/`Failure` `[R: lib/ruby_reactor/template/result.rb:170]`.
+   Opting in (reader fails) rolls back the parent **after** the child has already rolled itself
+   back `[O: S-async-05]`.
+4. A parent failure never undoes a completed child. A child still queued runs after the parent's
+   rollback `[O: S-async-06]`.
+
+### 2.7 `background` reactor (`all:` / `before:` / `after:`)
+
+1. The caller runs steps up to the hand-off point, checkpoints the context (**including the undo
+   stack**), and enqueues the worker `[R: lib/ruby_reactor/executor/step_executor.rb:364]`.
+2. The worker resumes (`Executor#resume_execution`) with the persisted undo stack. So a worker-side
+   failure also undoes steps that ran **in the caller** `[O: S-bg-02]`.
+3. Retries inside the worker re-enqueue the job with backoff. Compensation happens once, in the
+   delivery that exhausts the attempts `[R: lib/ruby_reactor/executor/retry_manager.rb:143]`
+   `[O: S-bg-03]`.
+4. Order is identical to inline for the same shape `[O: S-bg-01, S-compose-06]`.
+
+### 2.8 Interrupt (pause / continue / cancel / undo)
+
+1. Reaching an `interrupt` step returns `InterruptResult`. The context is saved `paused`, and a
+   reactor-level lock is **released** (a pause is not a park) `[R: lib/ruby_reactor/executor.rb:161]`
+   `[O: S-intr-01]`.
+2. `continue` re-acquires the reactor lock and resumes. A later failure undoes the steps completed
+   **before** the pause (persisted undo stack). The interrupt step itself is never on the stack
+   `[O: S-intr-01]`.
+3. Invalid payload past `max_attempts` → `undo` + `failed` `[R: lib/ruby_reactor/reactor.rb:411]`
+   `[O: S-intr-02]`.
+4. `Reactor.cancel` never rolls back `[R: …/reactor.rb:194]` `[O: S-intr-03]`. `Reactor.undo(id)`
+   undoes the stack and cancels, **without taking the reactor-level lock**
+   `[R: …/reactor.rb:63, :188]` `[O: S-intr-04]`.
+
+---
+
+## 3. Order matrix
+
+Rows are probes, and every row is `MATCH` in the transcript. Events are exactly what the probe
+recorded. `⇒` is the final outcome of the top-level execution. In *Left in place*, `—` means
+nothing is left in place. Rows without a scenario id are combinations that are not reachable or
+not probed, with the reason given.
+
+### 3.1 Plain steps
+
+| Scenario | Shape | Failure at | Mode | Ordered events | Left in place | Evidence |
+|---|---|---|---|---|---|---|
+| S-plain-01 | a → b → c | b returns Failure | inline | run:a run:b compensate:b undo:a ⇒ failure(b) | — | [O: S-plain-01] |
+| S-plain-02 | a → b → c | b raises | inline | run:a run:b compensate:b undo:a ⇒ failure(b) | — | [O: S-plain-02] |
+| S-plain-03 | a → b | b fails, its compensate fails | inline | run:a run:b compensate:b undo:a ⇒ failure(b) (CompensationError) *(changed in 008)* | — (reported: b/compensate) | [O: S-plain-03] |
+| S-plain-04 | a → b → c | c fails, b's undo raises | inline | run:a run:b run:c compensate:c undo:b undo:a ⇒ failure(c) | b's effect, if its undo failed (reported) | [O: S-plain-04] |
+| S-plain-05 | a → b → c | b returns Halt | inline | run:a run:b ⇒ halt | a, b (by design) | [O: S-plain-05] |
+| S-plain-06 | a → b(Skipped) → c | c fails | inline | run:a run:b run:c compensate:c undo:a ⇒ failure(c) | — | [O: S-plain-06] |
+| S-plain-07 | a → b | b's argument transform raises | inline | run:a undo:a ⇒ failure(b) *(changed in 008)* | — | [O: S-plain-07] |
+| S-plain-08 | a → b | b's output fails `validate_output` | inline | run:a run:b compensate:b undo:a ⇒ failure(b) | — | [O: S-plain-08] |
+| S-plain-09 | a, b → c → d | d fails | inline | run:a run:b run:c run:d compensate:d undo:c undo:b undo:a ⇒ failure(d) | — | [O: S-plain-09] |
+
+### 3.2 `compose`
+
+| Scenario | Shape | Failure at | Mode | Ordered events | Left in place | Evidence |
+|---|---|---|---|---|---|---|
+| S-compose-01 | a → compose(c1 → c2) → b | c2 (in child) | inline | run:a run:child.c1 run:child.c2 compensate:child.c2 undo:child.c1 undo:a ⇒ failure(child) | — | [O: S-compose-01] |
+| S-compose-02 | a → compose(c1 → c2) → b | b (after child) | inline | run:a run:child.c1 run:child.c2 run:b compensate:b undo:child.c2 undo:child.c1 undo:a ⇒ failure(b) | — | [O: S-compose-02] |
+| S-compose-03 | compose x(x1 → x2) → compose y(y1 → y2) | y2 | inline | run:x.x1 run:x.x2 run:y.y1 run:y.y2 compensate:y.y2 undo:y.y1 undo:x.x2 undo:x.x1 ⇒ failure(y) | — | [O: S-compose-03] |
+| S-compose-04 | a → compose outer(o1 → compose inner(i1 → i2)) | i2 (depth 2) | inline | run:a run:outer.o1 run:inner.i1 run:inner.i2 compensate:inner.i2 undo:inner.i1 undo:outer.o1 undo:a ⇒ failure(outer) | — | [O: S-compose-04] |
+| S-compose-05 | compose(c1 → c2) declaring `retries max_attempts: 2` | class definition | inline | ⇒ `DeprecatedDslError` *(changed in 008, R-14)* | — | [O: S-compose-05] |
+| S-compose-05b | compose(c1 → c2 with its own `retries`) → b | c2 fails once, then b fails | inline | run:child.c1 run:child.c2 retry:c2#1 run:child.c2 run:b compensate:b undo:child.c2 undo:child.c1 ⇒ failure(b) *(changed in 008)* | — | [O: S-compose-05b] |
+| S-compose-06 | `background all:` a → compose(c1 → c2) → b | b | worker | same as S-compose-02 | — | [O: S-compose-06] |
+| S-compose-07 | a → compose(c1(undo raises) → c2) | c2 | inline | run:a run:child.c1 run:child.c2 compensate:child.c2 undo:child.c1 undo:a ⇒ failure(child) | c1's effect (reported: c1/undo, flattened into parent) | [O: S-compose-07] |
+| S-compose-08 | a → compose(c1 → c2) → b | c2 returns **Halt** | inline | run:a run:child.c1 run:child.c2 **run:b** ⇒ success | — (the child's halt does not stop the parent) | [O: S-compose-08] |
+| — | compose with `fan_out` map inside | — | worker | **not probed: known bug.** The root never resumes after the map (`specs/future_improvements.md:225`) | — | [R: lib/ruby_reactor/map/helpers.rb:128-138] |
+| — | compose-level `compensate`/`undo` hook | — | — | **not reachable**: the DSL offers none | — | [R: lib/ruby_reactor/dsl/compose_builder.rb:60] |
+
+### 3.3 `map`
+
+Element reactor: `e1 → e2`, both with compensate/undo. Element `i == 2` fails at `e2`. Items `[0,1,2,3]`.
+
+| Scenario | Shape | Failure at | Mode | Ordered events | Left in place | Evidence |
+|---|---|---|---|---|---|---|
+| S-map-01 | a → map → b | element 2 | inline, fail_fast | run:a, e1/e2 [0], e1/e2 [1], run:e1[2] run:e2[2] compensate:e2[2] undo:e1[2], undo:e2[1] undo:e1[1] undo:e2[0] undo:e1[0], undo:a ⇒ failure(m) *(changed in 008)* | — (element 3 never runs) | [O: S-map-01] |
+| S-map-02 | a → map → b | element 2 | inline, `fail_fast false` | run:a, [0], [1], [2] + compensate:e2[2] undo:e1[2], [3], run:b ⇒ **success** | elements 0, 1, 3 (by design, the map succeeds) | [O: S-map-02] |
+| S-map-03 | a → map → b | b (after map) | inline | run:a, [0..3], run:b compensate:b, undo [3], [2], [1], [0], undo:a ⇒ failure(b) *(changed in 008)* | — | [O: S-map-03] |
+| S-map-04 | a → map → b | element 2 | fan_out, fail_fast (jobs in order 0..3) | same as S-map-01 *(changed in 008)* | — (element 3 skipped) | [O: S-map-04] |
+| S-map-04b | a → map → b | element 2 | fan_out, fail_fast, jobs performed 3,2,1,0 | run:a, [3], run:e1[2] run:e2[2] compensate:e2[2] undo:e1[2], undo:e2[3] undo:e1[3], undo:a ⇒ failure(m) *(changed in 008)* | — (0, 1 skipped) | [O: S-map-04b] |
+| S-map-05 | a → map → b | element 2 | fan_out, `fail_fast false` | same as S-map-02 | elements 0, 1, 3 | [O: S-map-05] |
+| S-map-06 | a → map → b | b (after map) | fan_out | same as S-map-03 *(changed in 008)* | — | [O: S-map-06] |
+| S-map-07 | a → compose(c0 → map) | element 2 | inline | run:a run:child.c0, [0], [1], [2]+rollback, undo [1], [0], undo:child.c0 undo:a ⇒ failure(child) *(changed in 008)* | — | [O: S-map-07] |
+| S-map-08 | map(element: e1 → compose(k1) → e2) | element 2's e2 | inline | …, run:e1[2] run:k.k1 run:e2[2] compensate:e2[2] undo:k.k1 undo:e1[2], then elements 1, 0 each undo:e2 undo:k.k1 undo:e1 ⇒ failure(m) *(changed in 008)* | — | [O: S-map-08] |
+| S-map-09 | a → map → b | every e2 fails once, `retries 2` | fan_out | run:a run:e1[0] run:e2[0] retry:e2#1 run:e1[1] run:e2[1] retry:e2#1 run:e2[0] run:e2[1] run:b ⇒ success | — | [O: S-map-09] |
+| S-map-10 | a → map → b | same | inline | run:a run:e1[0] run:e2[0] retry:e2#1 run:e2[0] run:e1[1] run:e2[1] retry:e2#1 run:e2[1] run:b ⇒ success | — | [O: S-map-10] |
+| S-map-11 | a → map → b | element 1 returns **Halt** | inline | run:a, [0], run:e1[1] run:e2[1] ⇒ **halt** | a, elements 0, 1 (by design) | [O: S-map-11] |
+| — | map-level `retries` / `compensate` / `undo` | — | — | **not reachable**: `MapBuilder` has none (the element steps' `undo`s are the map's rollback, 008) | — | [R: lib/ruby_reactor/dsl/map_builder.rb:130-131] |
+| S-map-12 | a → map(element: e1 → async_step u, e2) → b | element 1's e2 | fan_out, fail_fast | run:a, [0] e1 e2, run:e1[1] run:e2[1] compensate:e2[1] undo:e1[1], undo:e2[0] undo:e1[0], undo:a, **run:u[0] run:u[1]** ⇒ failure(m) *(changed in 008)* | **both units, incl. failed element 1's** (async units are independent, INV-25; F-09) | [O: S-map-12] [R: lib/ruby_reactor/executor/async_step_dispatch.rb:20-24] vs [R: lib/ruby_reactor/map/element_executor.rb:51-55] |
+
+### 3.4 `async_step` / `async_reactor`
+
+| Scenario | Shape | Failure at | Mode | Ordered events | Left in place | Evidence |
+|---|---|---|---|---|---|---|
+| S-async-01 | a → async_step u; b | u, no reader | inline + StepWorker | run:a run:b run:u compensate:u ⇒ success *(changed in 008)* | a, b (by design: no reader) | [O: S-async-01] |
+| S-async-02 | a → async_step u → r reads u | u, r fails on it | inline + Sidekiq inline! | run:a run:u compensate:u run:r compensate:r undo:a ⇒ failure(r) *(changed in 008)* | — | [O: S-async-02] |
+| S-async-03 | a → async_step u; b | b, u succeeds | inline + StepWorker | run:a run:b compensate:b undo:a **run:u** ⇒ failure(b) | **u (runs after the rollback)** | [O: S-async-03] |
+| S-async-04 | a → async_reactor child(c1 → c2); b | c2 in child, no reader | inline + Worker | run:a run:b run:child.c1 run:child.c2 compensate:child.c2 undo:child.c1 ⇒ success | a, b | [O: S-async-04] |
+| S-async-05 | a → async_reactor child → r reads child | c2 in child, r fails | inline + Sidekiq inline! | run:a [child rolls back itself] run:r compensate:r undo:a ⇒ failure(r) | — | [O: S-async-05] |
+| S-async-06 | a → async_reactor child; b | b, child succeeds | inline + Worker | run:a run:b compensate:b undo:a **run:child.c1 run:child.c2** ⇒ failure(b) | **child's c1, c2** | [O: S-async-06] |
+| S-async-07 | a → async_step u(`retries 3`); b | u always fails | inline + StepWorker | run:a run:b run:u run:u run:u compensate:u ⇒ success *(changed in 008)* | a, b (no reader); no retry events | [O: S-async-07] |
+| S-async-08 | a → async_step u → r reads u | reader wait times out | inline (fake queue) | run:a undo:a **run:u** compensate:u ⇒ failure(r) *(changed in 008)* | **u, if it succeeds (runs after the rollback, F-09)** | [O: S-async-08] |
+
+### 3.5 `background`
+
+| Scenario | Shape | Failure at | Mode | Ordered events | Left in place | Evidence |
+|---|---|---|---|---|---|---|
+| S-bg-01 | `all:` a → b → c | c | worker | run:a run:b run:c compensate:c undo:b undo:a ⇒ failure(c) | — | [O: S-bg-01] |
+| S-bg-02 | a \| `after: :a` \| b → c | c | caller + worker | run:a (caller) run:b run:c compensate:c undo:b undo:a (worker) ⇒ failure(c) | — | [O: S-bg-02] |
+| S-bg-03 | `all:` a → b(`retries 3`) | b always | worker | run:a run:b retry:b#1 run:b retry:b#2 run:b compensate:b undo:a ⇒ failure(b) | — | [O: S-bg-03] |
+| S-bg-04 | `all:` a → b(`retries 2`) → c | b once | worker | run:a run:b retry:b#1 run:b run:c ⇒ success | — | [O: S-bg-04] |
+
+### 3.6 Coordination, retries, interrupts, edge cases
+
+| Scenario | Shape | Failure at | Mode | Ordered events | Left in place | Evidence |
+|---|---|---|---|---|---|---|
+| S-lock-01 | reactor `with_lock rk`: a → b | b | inline | lock_acquired:rk run:a run:b compensate:b undo:a lock_released:lock:rk ⇒ failure(b) | — | [O: S-lock-01] |
+| S-lock-02 | a → b(step lock sk) → c | c | inline | run:a lock_acquired:sk run:b lock_released:sk run:c compensate:c lock_acquired:sk undo:b lock_released:sk undo:a ⇒ failure(c) | — | [O: S-lock-02] |
+| S-lock-03 | a → b(step lock sk, held elsewhere) | b contended | inline | run:a undo:a ⇒ failure(b) | — (b never ran, not compensated) | [O: S-lock-03] |
+| S-lock-04 | a → b(step semaphore 1) → c | c | inline | as S-lock-02 with semaphore events | — | [O: S-lock-04] |
+| S-lock-05 | a → b(step lock, `rollback_wait 0.3`) → c (grabs sk) | c | inline | run:a lock_acquired:sk run:b lock_released:sk run:c compensate:c undo:a ⇒ failure(c) | **b** (undo skipped, reported `coordination_unavailable`) | [O: S-lock-05] |
+| S-lock-06 | reactor lock rk: a → compose(child lock rk) → b | b | inline | lock_acquired:rk run:a lock_acquired:rk run:child.c1 lock_released:lock:rk run:b(rk count=1) compensate:b undo:child.c1 undo:a lock_released:lock:rk ⇒ failure(b) | — | [O: S-lock-06] |
+| S-retry-01 | a → b(`retries 3`) | b always | inline | run:a run:b retry:b#1 run:b retry:b#2 run:b compensate:b undo:a ⇒ failure(b) | — | [O: S-retry-01] |
+| S-retry-02 | a → b(`fail!(retry: false)`) | b | inline | run:a run:b compensate:b undo:a ⇒ failure(b) | — | [O: S-retry-02] |
+| S-retry-03 | a → b(`retries 2`) → c | b once | inline | run:a run:b retry:b#1 run:b run:c ⇒ success | — | [O: S-retry-03] |
+| S-retry-04 | a → b raises (`retries 2`) | b always | inline | run:a run:b retry:b#1 run:b compensate:b undo:a ⇒ failure(b) | — | [O: S-retry-04] |
+| S-intr-01 | reactor lock: a → interrupt → c | c after continue | inline | lock_acquired:rk run:a lock_released:lock:rk ⏸ lock_acquired:rk run:c compensate:c undo:a lock_released:lock:rk ⇒ failure(c) | — | [O: S-intr-01] |
+| S-intr-02 | a → interrupt(`max_attempts 1`) | invalid payload | inline | run:a undo:a ⇒ failure(approval) | — | [O: S-intr-02] |
+| S-intr-03 | a → interrupt | `Reactor.cancel` | inline | run:a ⇒ cancelled | **a** (by design) | [O: S-intr-03] |
+| S-intr-04 | completed a → b, reactor lock | `Reactor.undo(id)` while rk held by another owner | inline | … undo:b undo:a ⇒ cancelled | — (undo ran **outside** the reactor lock) | [O: S-intr-04] |
+| S-edge-01 | `all:` a → b(step lock held elsewhere) | contention ceiling | worker | run:a undo:a ⇒ failure(b) | — | [O: S-edge-01] |
+| S-edge-02 | `all:` a → b → c | worker crash in b | worker, redelivered | run:a run:b ✖ run:b run:c ⇒ success | b's first partial run (at-least-once) | [O: S-edge-02] |
+| S-edge-03 | a → b | b raises a custom `Exception` | inline | run:a run:b compensate:b undo:a ⇒ failure(b) *(changed in 008, R-16)* | — | [O: S-edge-03] |
+| S-edge-03b | a → b | b raises `Interrupt` | inline | run:a run:b ⇒ exception raised to caller; stored status `aborted` *(new in 008)* | **a**, until a manual `Reactor.undo(id)` | [O: S-edge-03b] |
+| S-edge-04 | a → b | b declares `where` | inline | ⇒ `DeprecatedDslError` at class definition *(changed in 008, R-15)* | — | [O: S-edge-04] |
+| S-edge-05 | a → b | b input type invalid | inline | run:a undo:a ⇒ failure(b) | — | [O: S-edge-05] |
+| S-edge-06 | reactor input invalid | before start | inline | ⇒ failure | — | [O: S-edge-06] |
+
+---
+
+## 4. Cross-cutting: locks
+
+| Primitive | Held from → to | During rollback | Evidence |
+|---|---|---|---|
+| Reactor `with_lock` / `with_semaphore` | admission → executor `ensure` (after the last undo) | **held** through rollback | `[R: lib/ruby_reactor/executor.rb:161]` `[O: S-lock-01]` |
+| … across a park (worker contention / pending async read) | kept (detached, re-adopted on redelivery) | — | `[R: lib/ruby_reactor/executor.rb:639-668]` |
+| … across an interrupt pause | **released**, re-acquired on `continue` | — | `[O: S-intr-01]` |
+| … for `Reactor.undo(id)` | **not taken** | undo runs without it | `[R: lib/ruby_reactor/reactor.rb:188]` `[O: S-intr-04]` |
+| … composed child with the same key | re-entrant (same root owner, counter) | parent hold intact | `[O: S-lock-06]` |
+| … `async_reactor` child with the same key | **refused at dispatch** (deadlock guard): that step fails, normal rollback | — | `[R: lib/ruby_reactor/step/async_reactor_step.rb:104]` |
+| Step `with_lock` / `with_semaphore` | body only | **re-taken** around that step's compensate/undo, waiting `rollback_wait` (default lock `ttl` / 60 s) | `[R: lib/ruby_reactor/executor/step_coordination.rb:207-251]` `[O: S-lock-02, S-lock-04]` |
+| … when it cannot be re-taken | — | that rollback **skipped** and reported, the rest continues | `[O: S-lock-05]` `[T: spec/ruby_reactor/step_coordination/rollback_under_contention_spec.rb:32]` |
+| Step rate limit / period / ordered lock | body only | **not** re-taken (quota never blocks cleanup) | `[R: …/step_coordination.rb:196-206]` `[T: spec/ruby_reactor/step_coordination/rollback_spec.rb:101]` |
+| Contention on a step's own lock | — | step never started → not compensated. Inline: fails at once. Worker: parks, then fails at the ceiling | `[O: S-lock-03, S-edge-01]` |
+
+Between a step's forward release and its rollback re-acquire there is a window in which
+another execution can take the key and change the protected resource. The re-acquire serializes
+the undo with that execution. It does not guarantee the resource is still as the step left it.
+
+## 5. Cross-cutting: retries
+
+| Where the step runs | How a retry happens | Compensation | Evidence |
+|---|---|---|---|
+| Inline reactor | `sleep(backoff)` in the caller's thread | once, after the last attempt | `[R: lib/ruby_reactor/executor/retry_manager.rb:171-175]` `[O: S-retry-01]` |
+| `background` worker | job re-enqueued with backoff (`RetryQueuedResult`) | once, in the delivery that exhausts | `[O: S-bg-03]` |
+| Map element, inline | `sleep` in place, element after element | per element, once | `[O: S-map-10]` |
+| Map element, fan_out | element job re-enqueued, other elements interleave | per element, once | `[O: S-map-09]` |
+| `async_step` | loop **inside** the StepWorker job, no retry middleware event | **never** (unit rollback hooks do not run) | `[R: lib/ruby_reactor/step_worker.rb:275-289]` `[O: S-async-07]` |
+| `compose` | *since 008*, `retries` on the compose is rejected at definition time; the child's own steps retry inside the child (baseline: the child was resumed with its undone steps counted as completed) | as for the child's steps | `[O: S-compose-05, S-compose-05b]` |
+| `fail!(…, retry: false)` | not retried | once | `[O: S-retry-02]` |
+| Successful retry | — | none (no compensate for the failed attempts) | `[O: S-retry-03, S-bg-04]` |