ruby_reactor 0.8.4 → 0.8.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (72) hide show
  1. checksums.yaml +4 -4
  2. data/.release-please-manifest.json +1 -1
  3. data/.specify/feature.json +1 -1
  4. data/CHANGELOG.md +196 -0
  5. data/CLAUDE.md +1 -1
  6. data/README.md +47 -11
  7. data/lib/ruby_reactor/dsl/async_macros.rb +30 -1
  8. data/lib/ruby_reactor/dsl/async_reactor_builder.rb +12 -6
  9. data/lib/ruby_reactor/dsl/compose_builder.rb +12 -6
  10. data/lib/ruby_reactor/dsl/interrupt_builder.rb +1 -3
  11. data/lib/ruby_reactor/dsl/map_builder.rb +0 -2
  12. data/lib/ruby_reactor/dsl/step_builder.rb +91 -19
  13. data/lib/ruby_reactor/error/argument_resolution_error.rb +19 -0
  14. data/lib/ruby_reactor/error/rescuable.rb +28 -0
  15. data/lib/ruby_reactor/executor/compensation_manager.rb +30 -26
  16. data/lib/ruby_reactor/executor/result_handler.rb +18 -16
  17. data/lib/ruby_reactor/executor/step_coordination.rb +11 -8
  18. data/lib/ruby_reactor/executor/step_executor.rb +59 -49
  19. data/lib/ruby_reactor/executor.rb +38 -4
  20. data/lib/ruby_reactor/map/collector.rb +21 -11
  21. data/lib/ruby_reactor/map/dispatcher.rb +29 -3
  22. data/lib/ruby_reactor/map/element_executor.rb +9 -3
  23. data/lib/ruby_reactor/map/helpers.rb +32 -2
  24. data/lib/ruby_reactor/map/result_enumerator.rb +18 -12
  25. data/lib/ruby_reactor/reactor.rb +24 -0
  26. data/lib/ruby_reactor/rspec/matchers.rb +19 -3
  27. data/lib/ruby_reactor/step/compose_step.rb +7 -1
  28. data/lib/ruby_reactor/step/map_step.rb +109 -4
  29. data/lib/ruby_reactor/step.rb +7 -0
  30. data/lib/ruby_reactor/step_worker.rb +46 -22
  31. data/lib/ruby_reactor/storage/adapter.rb +4 -0
  32. data/lib/ruby_reactor/storage/redis_adapter.rb +9 -0
  33. data/lib/ruby_reactor/storage/redis_reactor_scan.rb +1 -1
  34. data/lib/ruby_reactor/version.rb +1 -1
  35. data/lib/ruby_reactor/web/api.rb +1 -1
  36. data/lib/ruby_reactor/web/public/assets/{index-CeZU-ESu.js → index-CQbgHtd0.js} +10 -10
  37. data/lib/ruby_reactor/web/public/index.html +1 -1
  38. data/lib/ruby_reactor/worker.rb +3 -1
  39. data/lib/ruby_reactor.rb +17 -6
  40. data/specs/007-execution-flow-analysis/analysis/README.md +147 -0
  41. data/specs/007-execution-flow-analysis/analysis/execution-order.md +359 -0
  42. data/specs/007-execution-flow-analysis/analysis/findings-and-options.md +502 -0
  43. data/specs/007-execution-flow-analysis/analysis/invariants.md +109 -0
  44. data/specs/007-execution-flow-analysis/checklists/requirements.md +39 -0
  45. data/specs/007-execution-flow-analysis/contracts/report-structure.md +71 -0
  46. data/specs/007-execution-flow-analysis/data-model.md +83 -0
  47. data/specs/007-execution-flow-analysis/evidence/harness.rb +229 -0
  48. data/specs/007-execution-flow-analysis/evidence/output.txt +333 -0
  49. data/specs/007-execution-flow-analysis/evidence/probes/01_plain.rb +122 -0
  50. data/specs/007-execution-flow-analysis/evidence/probes/02_compose.rb +182 -0
  51. data/specs/007-execution-flow-analysis/evidence/probes/03_map.rb +232 -0
  52. data/specs/007-execution-flow-analysis/evidence/probes/04_async.rb +132 -0
  53. data/specs/007-execution-flow-analysis/evidence/probes/05_background.rb +58 -0
  54. data/specs/007-execution-flow-analysis/evidence/probes/06_coordination.rb +158 -0
  55. data/specs/007-execution-flow-analysis/evidence/probes/07_interrupts_manual.rb +185 -0
  56. data/specs/007-execution-flow-analysis/evidence/run.rb +15 -0
  57. data/specs/007-execution-flow-analysis/plan.md +127 -0
  58. data/specs/007-execution-flow-analysis/quickstart.md +51 -0
  59. data/specs/007-execution-flow-analysis/research.md +202 -0
  60. data/specs/007-execution-flow-analysis/spec.md +270 -0
  61. data/specs/007-execution-flow-analysis/tasks.md +257 -0
  62. data/specs/008-rollback-reliability/checklists/requirements.md +43 -0
  63. data/specs/008-rollback-reliability/contracts/api-surface.md +126 -0
  64. data/specs/008-rollback-reliability/contracts/rollback-semantics.md +76 -0
  65. data/specs/008-rollback-reliability/data-model.md +139 -0
  66. data/specs/008-rollback-reliability/plan.md +233 -0
  67. data/specs/008-rollback-reliability/quickstart.md +105 -0
  68. data/specs/008-rollback-reliability/research.md +653 -0
  69. data/specs/008-rollback-reliability/spec.md +561 -0
  70. data/specs/008-rollback-reliability/tasks.md +1110 -0
  71. data/specs/future_improvements.md +48 -0
  72. metadata +35 -2
@@ -0,0 +1,502 @@
1
+ # Findings, Documentation Audit & Options
2
+
3
+ Severity (research D7): **High** = completed side effects silently left in place, work that can
4
+ run twice, or rollback of work that never ran. **Medium** = defensible but differs by
5
+ mode/nesting, or contradicts documentation. **Low** = clarity/observability; predictable, but the
6
+ DSL or telemetry gives no hint.
7
+
8
+ All options are **proposals pending later analysis**, not decisions.
9
+
10
+ - [1. Findings](#1-findings)
11
+ - [2. Documentation audit](#2-documentation-audit)
12
+ - [3. Options](#3-options) (includes [the map `compensate_all` / `compensate_each` evaluation](#o-01-evaluation-compensate_all-vs-compensate_each))
13
+
14
+ ---
15
+
16
+ ## 1. Findings
17
+
18
+ ### F-01 · High · Map elements that already succeeded are never rolled back
19
+
20
+ - **Resolved by 008**: map rollback replays every completed element's undo stack, highest index first, in inline and fan-out mode (008 R-02, R-03). See `spec/ruby_reactor/rollback/map_rollback_spec.rb`.
21
+
22
+ - **Scenarios**: [O: S-map-01] [O: S-map-03] [O: S-map-04] [O: S-map-04b] [O: S-map-06] [O: S-map-07] [O: S-map-08]
23
+ - **Reader expects**: a map is "a step that happens N times". When the map fails, or a later step
24
+ fails, the elements that completed are rolled back the way a `compose` child is (README: "automatic
25
+ rollback of completed steps").
26
+ - **Actual**: only the **failed** element rolls back its own steps. Succeeded elements keep
27
+ their effects in both cases:
28
+ 1. **The map fails** (fail_fast): elements before the failure are left in place, including their
29
+ nested composes (S-map-08). `MapStep#compensate` is a stub
30
+ (`# TODO: Implement compensation for map steps` → `Success()`)
31
+ [R: lib/ruby_reactor/step/map_step.rb:29-31].
32
+ 2. **A later step fails**: every element is left in place. Inline, the map step's `undo` is the
33
+ base `Skipped` [R: lib/ruby_reactor/step.rb:57], and the element contexts are discarded. Fan-out,
34
+ the collector never even pushes the map step on the undo stack
35
+ [R: lib/ruby_reactor/map/helpers.rb:97].
36
+ 3. There is no DSL surface to fix it locally: `MapBuilder` builds `compensate_block: nil,
37
+ undo_block: nil` [R: lib/ruby_reactor/dsl/map_builder.rb:130-131]. `undo` blocks written on the
38
+ element reactor's steps run only when **that element** fails.
39
+ - **Rollback reporting**: `rollback_failures` is empty. Nothing was attempted, so nothing is reported.
40
+ - **Doc conflict**: README.md:16, :25, :1309, :1398; documentation/data_pipelines.md:167;
41
+ documentation/DAG.md:228-240 (see §2).
42
+ - **Related**: INV-19, INV-20 (both VIOLATED, no spec coverage).
43
+
44
+ ### F-02 · High · `compose` with `retries` resumes a child whose earlier steps were already undone
45
+
46
+ - **Resolved by 008**: a compose retry after a failed attempt starts a fresh child (008 R-05). See `spec/ruby_reactor/rollback/compose_retry_spec.rb`.
47
+
48
+ - **Scenarios**: [O: S-compose-05] [O: S-compose-05b]
49
+ - **Reader expects**: `retries` on a compose retries the sub-saga. After a failed attempt that
50
+ rolled the child back, the next attempt starts the child again.
51
+ - **Actual**: the failed attempt undoes `c1` (its effect is gone). The retry reuses the same child
52
+ context and `resume_execution`s it. `c1`'s result is still in `intermediate_results`, so `c1` is
53
+ treated as completed and **not re-run**. `c2` then runs against the result of an undone step, and
54
+ the reactor returns `c1`'s stale value. If a later parent step fails, only `c2` is undone
55
+ (S-compose-05b). The child ends "successful" with its first step's side effect missing.
56
+ [R: lib/ruby_reactor/step/compose_step.rb:97] [R: lib/ruby_reactor/executor/graph_manager.rb (mark_completed_steps_from_context)]
57
+ - **Doc conflict**: documentation/composition.md:184 ("can be configured with different retry
58
+ strategies") is silent on this.
59
+ - **Related**: INV-13 (VIOLATED, no coverage).
60
+
61
+ ### F-03 · High · Some failures skip rollback entirely
62
+
63
+ - **Resolved by 008**: argument, condition and unknown `StandardError`s roll back and carry `step_name`; a non-`StandardError` marks a caller-process run `aborted` for a manual undo (008 R-06–R-08). See `spec/ruby_reactor/rollback/failure_rollback_spec.rb`, `aborted_execution_spec.rb`.
64
+
65
+ - **Scenarios**: [O: S-plain-07] [O: S-edge-03]
66
+ - **Reader expects**: "if any part of your workflow fails … automatically triggers compensation"
67
+ (README.md:16).
68
+ - **Actual**: two failure classes roll back nothing:
69
+ 1. A `StandardError` raised while **resolving a step's arguments** (a `transform:` lambda, a
70
+ dynamic source, a `result(:x, path)` that raises). Resolution runs outside the step's rescue
71
+ [R: lib/ruby_reactor/executor/step_executor.rb:83]. The error reaches
72
+ `ResultHandler#build_execution_failure`, whose non-`Error::Base` branch is
73
+ "Unknown errors - don't rollback" [R: lib/ruby_reactor/executor/result_handler.rb:68-70].
74
+ The result is `Failure("Execution failed: …")` with no step name and an empty `rollback_failures`.
75
+ 2. A non-`StandardError` exception in a body (e.g. `Timeout::ExitException`-like, custom
76
+ `Exception` subclasses) propagates to the caller with no rollback. A worker would redeliver
77
+ (INV-32); an inline caller gets partial effects.
78
+ - **Related**: INV-06 (VIOLATED, no coverage).
79
+
80
+ ### F-04 · High · An `async_step`'s own `compensate` / `undo` never run; the docs say they do
81
+
82
+ - **Resolved by 008**: an `async_step` unit compensates itself once, in its own job, after its final attempt; an inline `undo` is rejected, a class `undo` warned (008 R-09). See `spec/ruby_reactor/rollback/async_step_compensate_spec.rb`.
83
+
84
+ - **Scenarios**: [O: S-async-02] [O: S-async-07]
85
+ - **Reader expects** (documentation/background_and_async.md:291-292): "`compensate` / `undo` blocks
86
+ declared on an `async_step` still register; they run only if the failure is surfaced into the
87
+ parent's compensation path this way".
88
+ - **Actual**: when a reader surfaces the unit's failure, the **reader** is compensated and the
89
+ parent's steps are undone, but `compensate:u` never runs. `StepWorker` never invokes rollback
90
+ hooks [R: lib/ruby_reactor/step_worker.rb:240-393], and the parent never pushed the unit
91
+ [R: lib/ruby_reactor/executor/result_handler.rb:137]. The blocks are accepted by the DSL and are
92
+ dead code. The same holds when the unit exhausts its retries (S-async-07).
93
+ - **Coverage gap**: [T: spec/ruby_reactor/dsl/async_step_spec.rb:111] asserts only that `:setup`
94
+ is compensated, so it passes either way.
95
+ - **Related**: INV-24, INV-29.
96
+
97
+ ### F-05 · Medium · Fan-out fail-fast leaves a scheduling-dependent set of elements in place
98
+
99
+ - **Resolved by 008**: a fail-fast fan-out map settles every index before applying its failure, and every completed element is rolled back, so nothing is left in place whatever the job order (008 R-04). See `spec/ruby_reactor/rollback/map_fan_out_settle_spec.rb`.
100
+
101
+ - **Scenarios**: [O: S-map-04] vs [O: S-map-04b]
102
+ - **Actual**: the fail-fast marker is checked only when an element job **starts**
103
+ [R: lib/ruby_reactor/map/element_executor.rb:61, :145]. Elements that started, or finished,
104
+ before the failing element are kept. Later ones skip. Same input, different job order →
105
+ different leftovers (elements 0, 1 vs element 3). Combined with F-01, the leftover set is
106
+ neither rolled back nor predictable.
107
+ - **Doc conflict**: documentation/data_pipelines.md:167 ("fails immediately").
108
+ - **Related**: INV-22 (CONDITIONAL).
109
+
110
+ ### F-06 · Medium · A raising `where`/`guard` compensates a step whose body never ran
111
+
112
+ - **Resolved by 008**: a raising `where`/`guard` is a never-started `ConditionError`: not compensated, not retried (008 R-06).
113
+
114
+ - **Scenario**: [O: S-edge-04]
115
+ - **Actual**: conditions are evaluated inside the step's rescue
116
+ [R: lib/ruby_reactor/executor/step_executor.rb:342]. The resulting `Failure` goes through
117
+ normal failure handling, which compensates the step. Contention, key errors and argument
118
+ validation are all correctly treated as never-started (S-lock-03, S-edge-05). This path isn't.
119
+ - **Doc conflict**: documentation/locks_and_semaphores.md:778 states the never-started
120
+ principle (for contention only).
121
+ - **Related**: INV-07 (CONDITIONAL).
122
+
123
+ ### F-07 · Medium · `Halt` means different things at different nesting levels
124
+
125
+ - **Scenarios**: [O: S-compose-08] vs [O: S-map-11]
126
+ - **Actual**: a map element's `Halt` halts the parent. A composed child's `Halt` is converted to
127
+ `Success(nil)` [R: lib/ruby_reactor/step/compose_step.rb:106-114], and the parent **continues**
128
+ with `nil` as the compose result.
129
+ - **Doc conflict**: documentation/core_concepts.md:277 and documentation/composition.md:195
130
+ describe Halt and compose without this case.
131
+ - **Related**: INV-09 (VIOLATED).
132
+
133
+ ### F-08 · Medium · `Reactor.undo(id)` runs outside the reactor-level lock
134
+
135
+ - **Scenario**: [O: S-intr-04]
136
+ - **Actual**: `Reactor#undo` builds an executor and calls `undo_all` directly
137
+ [R: lib/ruby_reactor/reactor.rb:188-192]. It never acquires the reactor's `with_lock` /
138
+ `with_semaphore`. The undo succeeded while another owner held `rk`. Step-level locks *are*
139
+ re-taken, so only reactor-level exclusion is lost.
140
+ - **Doc conflict**: documentation/interrupts.md:150-157 (Cancellation & Undo) is silent on
141
+ locking.
142
+ - **Related**: INV-35 (CONDITIONAL).
143
+
144
+ ### F-09 · Medium · Dispatched units run after their dispatcher rolled back, including units of a failed map element
145
+
146
+ - **Scenarios**: [O: S-async-03] [O: S-async-06] [O: S-async-08] [O: S-map-12]
147
+ - **Actual**: parent rollback neither cancels nor undoes an `async_step`/`async_reactor`
148
+ (documented independence). A queued unit then performs its side effect **after** the rollback.
149
+ For an `async_step` inside a fan-out map element the unit escapes even the element: the element
150
+ rolled back, the unit still ran (S-map-12). This contradicts `ElementExecutor`'s stated intent
151
+ that async work "must execute inline here" [R: lib/ruby_reactor/map/element_executor.rb:51-55],
152
+ because `async_step` dispatch is deliberately not gated on that flag
153
+ [R: lib/ruby_reactor/executor/async_step_dispatch.rb:20-24].
154
+ - **Doc conflict**: README.md:545-550 and documentation/background_and_async.md:285-289 explain
155
+ independence, but not that a unit can run *after* its dispatcher's rollback.
156
+ - **Related**: INV-27 (VIOLATED).
157
+
158
+ ### F-10 · Medium · Rollback coverage of composite constructs is asymmetric and invisible in the DSL
159
+
160
+ - **Scenarios**: [O: S-compose-02] vs [O: S-map-03] vs [O: S-async-06]
161
+ - **Actual**: three constructs that all "run more steps" differ completely, and nothing in the
162
+ reactor definition shows it:
163
+
164
+ | Construct | Child/element/unit completed, later parent failure | Child/element/unit fails |
165
+ |---|---|---|
166
+ | `compose` | undone (full child replay) | child self-rolls back, parent rolls back |
167
+ | `map` | **left in place** | failed element self-rolls back; siblings left |
168
+ | `async_step` | left in place | own hooks never run; parent unaffected unless a reader opts in |
169
+ | `async_reactor` | left in place | child self-rolls back; parent unaffected unless a reader opts in |
170
+
171
+ - **Resolved by 008** (the table rebuilt from 008's rollback contract, RS §2): one rule for every
172
+ construct, and each construct's own `compensate`/`undo` decides what rollback means for it. The
173
+ coordinator asks the step whether a success is tracked for undo (`rollback_tracked?`).
174
+
175
+ | Construct | Child/element/unit completed, later parent failure | Child/element/unit fails |
176
+ |---|---|---|
177
+ | `compose` | undone (full child replay) | child self-rolls back, parent rolls back; a retry starts a fresh child |
178
+ | `map` | **undone** (every completed element, highest index first) | failed element self-rolls back; completed siblings undone, then the parent rolls back |
179
+ | `async_step` | left in place (independent by design, documented) | the unit compensates itself once, in its own job; parent unaffected unless a reader opts in |
180
+ | `async_reactor` | left in place (independent by design, documented) | child self-rolls back; parent unaffected unless a reader opts in |
181
+
182
+ `compose`, `map` and the async macros accept no `compensate`/`undo` declaration of their own, so
183
+ a reader cannot tell from the class body what will be rolled back. This is the user's
184
+ "does the DSL help clearly understand" question.
185
+ - **Related**: F-01, F-04, INV-16, INV-19, INV-20, INV-25.
186
+
187
+ ### F-11 · Low · Step-lock gap between the forward release and the rollback re-acquire
188
+
189
+ - **Scenarios**: [O: S-lock-02] [O: S-lock-05]
190
+ - **Actual**: a step lock is held for the body only. Rollback re-takes it (documented) but another
191
+ execution can hold it in between and change the protected resource. The undo is serialized with
192
+ that execution, but it runs against a possibly changed resource. It can also be skipped (reported)
193
+ when the key stays busy past `rollback_wait`.
194
+ - **Doc**: documentation/locks_and_semaphores.md:852-866 documents the re-take, not the gap.
195
+ - **Related**: INV-35, INV-36.
196
+
197
+ ### F-12 · Low · `async_step` retries are invisible to middleware
198
+
199
+ - **Scenario**: [O: S-async-07]. There is no `retry_attempt` event, unlike every other retry path
200
+ [R: lib/ruby_reactor/step_worker.rb:275-289]. **Related**: INV-29.
201
+
202
+ ### F-13 · Low · Failures without step attribution after rollback
203
+
204
+ - **Resolved by 008**: argument, condition, unknown and compensation failures all carry `step_name` and `reactor_name` (008 R-06, R-07).
205
+
206
+ - **Scenarios**: [O: S-plain-03] (compensate fails → `CompensationError` → "Execution error: …",
207
+ no `step_name`), [O: S-plain-07] (argument resolution → no `step_name`).
208
+ [R: lib/ruby_reactor/executor/result_handler.rb:64-70]. Violates Constitution IV ("every failure
209
+ MUST carry … step name").
210
+
211
+ ### F-14 · Low · Dead, contradictory map collection default
212
+
213
+ - `Map::Helpers#apply_collect_block` ("Default behavior: fail if any failure")
214
+ [R: lib/ruby_reactor/map/helpers.rb:36-52] is shadowed by `Collector.apply_collect_block`
215
+ ("Default behavior: Return Success(Enumerator)") [R: lib/ruby_reactor/map/collector.rb:108-126].
216
+ Only the latter is called. A reader of `helpers.rb` gets the wrong semantics.
217
+
218
+ ### F-15 · Low · Reactor-level `lock_released` names a different key than `lock_acquired`
219
+
220
+ - **Scenario**: [O: S-lock-01] (`lock_acquired:rk`, `lock_released:lock:rk`). Known and pinned by
221
+ [T: spec/ruby_reactor/step_coordination/attribution_spec.rb:41]. **Related**: INV-38.
222
+
223
+ ### F-16 · Low · Interrupt docs say "cancelled"; exhausted payload validation marks the run `failed`
224
+
225
+ - **Scenario**: [O: S-intr-02] ends `failure(approval)`, status `failed`
226
+ [R: lib/ruby_reactor/reactor.rb:411-415]. documentation/interrupts.md:40, :62, :141 say the
227
+ reactor is "cancelled and compensated". The rollback itself is as documented.
228
+
229
+ ---
230
+
231
+ ## 2. Documentation audit
232
+
233
+ Claims about ordering or rollback in README.md and `./documentation`. **CONFIRMED** rows are
234
+ listed too, so the audit is complete. Per plan.md Complexity Tracking, none of these files is edited
235
+ by this initiative. Each fix ships with the remedy chosen for its finding.
236
+
237
+ | File:line | Claim (quoted) | Actual | Finding |
238
+ |---|---|---|---|
239
+ | README.md:16 | "if any part of your workflow fails, Ruby Reactor automatically triggers compensation logic to undo previous steps, ensuring your system never ends up in a corrupted half-state" | Not for map elements, async units, argument-resolution errors or non-`StandardError` exceptions | F-01, F-03, F-04, F-09 |
240
+ | README.md:25 | "**Compensation**: Automatic rollback of completed steps when a failure occurs." | Same exceptions | F-01, F-03 |
241
+ | README.md:35 | "Auto compensation/undo \| Yes" | Partial (see F-10 table) | F-10 |
242
+ | README.md:545-550 | "A reader that returns `Failure` triggers compensation normally" | Triggers the reader's compensation and the parent's undos only. The unit's hooks never run, and the unit may run after the rollback | F-04, F-09 |
243
+ | README.md:841 | Halt: "already-completed steps are NOT compensated" | CONFIRMED [O: S-plain-05]. Nested behavior unstated | F-07 |
244
+ | README.md:1309 | "When a step fails, RubyReactor automatically undoes completed steps in reverse order, compensate only runs in the failing step…" | CONFIRMED for steps and composes [O: S-plain-09, S-compose-03]. Not for map elements | F-01 |
245
+ | README.md:1398 | "A rollback that did not complete is never silent." | True for **attempted** rollbacks [O: S-plain-04, S-lock-05]. Rollbacks never attempted (map elements, F-03) leave `rollback_failures` empty | F-01, F-03 |
246
+ | documentation/background_and_async.md:165-167 | "Compensation is unchanged. A worker-side failure compensates exactly as a same-process failure does" | CONFIRMED [O: S-bg-01, S-bg-02] | — |
247
+ | documentation/background_and_async.md:279-283 | "A later step that reads the result and returns `Failure` triggers compensation normally — so no failure is ever unrecoverable" | Reader + parent only; the unit's effect stays | F-04 |
248
+ | documentation/background_and_async.md:285-289 | "the dispatch itself never enters the parent's undo stack … independent compensation flows" | CONFIRMED [O: S-async-03, S-async-06]. Unstated: the unit can run *after* the rollback, and an `async_step` has **no** compensation flow at all | F-04, F-09 |
249
+ | documentation/background_and_async.md:291-292 | "`compensate` / `undo` blocks declared on an `async_step` still register; they run only if the failure is surfaced into the parent's compensation path this way." | **Contradicted**: they never run [O: S-async-02] | F-04 |
250
+ | documentation/composition.md:184 | "can be configured with different retry strategies" | A compose-level retry resumes an already-rolled-back child | F-02 |
251
+ | documentation/composition.md:195 | "Compensation \| fully linked — a child failure rolls the parent back" | CONFIRMED, including earlier sibling composes [O: S-compose-01, S-compose-03]. Missing row: a child `Halt` does not stop the parent | F-07 |
252
+ | documentation/data_pipelines.md:167 | "the entire map operation fails immediately if any single element fails" | Completed elements are kept, not rolled back. In fan-out, "immediately" means not-yet-started elements skip themselves | F-01, F-05 |
253
+ | documentation/data_pipelines.md:179 | `fail_fast false` collects successes and failures | CONFIRMED [O: S-map-02, S-map-05]. Unstated: failed elements roll back individually | — |
254
+ | documentation/data_pipelines.md:233-235 | per-element `retries` | CONFIRMED. Unstated: inline sleeps, fan-out re-enqueues [O: S-map-09, S-map-10] | — |
255
+ | documentation/core_concepts.md:321-331 | "Compensation runs in reverse order of successful steps" | CONFIRMED for steps [O: S-plain-09]. Map elements and async units are excluded without saying so | F-01, F-10 |
256
+ | documentation/core_concepts.md:277 | Halt: "completed steps are **not** compensated" | CONFIRMED | F-07 (nesting) |
257
+ | documentation/DAG.md:228-240 | "Cascading Compensation … continue to Step 1 → Consistent State Achieved" | Not achieved for maps, async units or F-03 failures | F-01, F-03 |
258
+ | documentation/locks_and_semaphores.md:777-778 | "the contended step itself does not compensate — … its own work was never attempted" | CONFIRMED [O: S-lock-03, S-edge-01]. The same principle is broken for a raising `where`/`guard` | F-06 |
259
+ | documentation/locks_and_semaphores.md:852-866 | Step rollback re-takes lock then semaphore, waiting `rollback_wait` | CONFIRMED [O: S-lock-02, S-lock-04, S-lock-05] | F-11 (gap unstated) |
260
+ | documentation/getting_started.md:231-232 | compensate the failing step, then undo in reverse | CONFIRMED | — |
261
+ | documentation/interrupts.md:40, :62, :141 | exhausted payload validation → "cancelled and compensated" | Rolled back as stated, but the status is **failed** [O: S-intr-02] | F-16 |
262
+ | documentation/interrupts.md:155-157 | `undo` runs undo blocks in reverse, then cancels | CONFIRMED [O: S-intr-04]. Unstated: no reactor lock | F-08 |
263
+
264
+ ---
265
+
266
+ ## 3. Options
267
+
268
+ Every option addresses finding(s), lists at least one con, and is a **proposal** only.
269
+ Compatibility: SemVer impact per Constitution V.
270
+
271
+ ### Options for F-01 (map rollback) — and the user's Q3
272
+
273
+ The user's question: should `map` get `compensate_all` / `compensate_each`? The first design point
274
+ is that **two different moments** need covering, and the library already has a word for each:
275
+
276
+ - **The map itself fails** (fail-fast, partial success): the elements that succeeded so far must be
277
+ cleaned up. In library terms that is the map's **compensate**.
278
+ - **The map completed, and a later step fails**: all elements must be cleaned up. That is the map's
279
+ **undo**.
280
+
281
+ `compensate_all` / `compensate_each` as named would cover only the first, unless their semantics
282
+ also cover undo. That naming mismatch is itself a DSL-clarity issue (F-10).
283
+
284
+ #### O-01-a · Implicit element-undo replay (make `map` behave like `compose`)
285
+
286
+ - **Sketch**: `MapStep#compensate` and a new `MapStep#undo` replay each **succeeded element's own
287
+ undo stack**, newest element first, exactly as `ComposeStep#undo` replays a child
288
+ [R: lib/ruby_reactor/step/compose_step.rb:31-47]. No new DSL: the `undo` blocks authors already
289
+ write on element steps start running.
290
+ - **Pros**: consistent with compose (closes the F-10 asymmetry). Rollback granularity is per
291
+ element step, so partial failures inside an undo are reported per step on `rollback_failures`.
292
+ - **Cons**: the element contexts must be **kept**. Inline maps discard them today, and keeping N
293
+ contexts inside the parent blob risks `ContextTooLargeError`. Fan-out needs a rollback fan-out
294
+ (N jobs, or a serial loop in the collector). It is a **behavior change** for every existing map
295
+ whose element steps declare `undo` (MAJOR). "Newest element first" is ill-defined under fan-out
296
+ concurrency.
297
+ - **Open questions**: keep element contexts by id (fan-out already does:
298
+ `store_map_element_context_id`) or embed? Serial or parallel element undo? Honour `fail_fast false`?
299
+
300
+ #### O-01-b · `compensate_each` / `undo_each` block on the map (user proposal, per element)
301
+
302
+ - **Sketch**:
303
+
304
+ ```ruby
305
+ map :charges, ChargeElement do
306
+ source input(:orders)
307
+ argument :order, element(:charges)
308
+ undo_each { |element_result, element| Refund.call(element_result[:charge_id]) }
309
+ end
310
+ ```
311
+
312
+ Called once per **succeeded** element, on map failure (compensate) and on a later failure (undo).
313
+ - **Pros**: visible in the DSL. Needs only element **results**, which fan-out already stores
314
+ (`store_map_result`) and inline has in memory, so no context retention. Per-element error
315
+ isolation: one failed call becomes one `rollback_failures` entry and the rest continue.
316
+ - **Cons**: duplicates the element reactor's own step `undo`s (two places for the same cleanup)
317
+ and is coarser than them. For the undo moment the map step must be pushed on the undo stack in
318
+ fan-out mode (it isn't today, [R: lib/ruby_reactor/map/helpers.rb:97]). A `collect` block that
319
+ transformed the results means the raw per-element results must be kept separately. Under fan-out
320
+ fail-fast, elements still in flight when it runs finish later and escape it (see F-05 options).
321
+ - **Open questions**: argument order/shape (`element` = source item, `result` = element output)?
322
+ Sequential or parallel calls in fan-out?
323
+
324
+ #### O-01-c · `compensate_all` / `undo_all` block on the map (user proposal, bulk)
325
+
326
+ - **Sketch**: `undo_all { |results, elements| Charge.where(id: results.map { _1[:charge_id] }).refund_all }`,
327
+ called once with every succeeded element's result.
328
+ - **Pros**: matches bulk cleanup (one `DELETE … WHERE id IN`), which is the user's
329
+ `all_elements.destroy` example. It is a single rollback call to reason about and report. It works
330
+ identically inline and fan-out (the collector already holds a `ResultEnumerator`).
331
+ - **Cons**: all-or-nothing error handling. A failure halfway through a bulk undo is one opaque
332
+ `rollback_failures` entry. The full result set must stay available until the parent finishes
333
+ (fan-out results live under the context TTL). The in-flight-element problem from O-01-b applies
334
+ unchanged.
335
+ - **Open questions**: pass an enumerator (lazy, fan-out friendly) or an array? Should
336
+ `fail_fast false` successes be included when a later step fails? (Probably yes.)
337
+
338
+ #### O-01-d · Keep behavior, make it explicit
339
+
340
+ - **Sketch**: document "map elements are never rolled back", recommend `fail_fast false` plus a
341
+ following step whose `undo` cleans up from the collected results, and remove the `# TODO` stub.
342
+ - **Pros**: no behavior change (PATCH). Honest.
343
+ - **Cons**: the gap stays. Every author re-implements the same pattern. It contradicts the README
344
+ reliability promise, which would also have to change.
345
+
346
+ #### O-01 evaluation: `compensate_all` vs `compensate_each`
347
+
348
+ | Dimension | `compensate_each` / `undo_each` (O-01-b) | `compensate_all` / `undo_all` (O-01-c) | Element-undo replay (O-01-a) |
349
+ |---|---|---|---|
350
+ | Map fails, `fail_fast` on | called for each element that succeeded before the failure. Inline = deterministic set; fan-out = scheduling-dependent set (F-05) | one call with that same set | replays those elements' step undos |
351
+ | Map succeeds with `fail_fast false`, later step fails | called for each **successful** element (failed ones already self-rolled back) | one call with the successful subset | replay successful elements |
352
+ | Map fails with `fail_fast false` | n/a (map does not fail, unless a `collect` block raises, which then needs the compensate moment) | same | same |
353
+ | Inline | results in memory: easy | easy | needs element contexts kept (currently discarded) |
354
+ | Fan-out | results stored per index: available. In-flight elements finish after the call and escape it | same | contexts stored by id: available. Needs a rollback fan-out |
355
+ | Element retries | an element that succeeded after retries is included. One that exhausted is excluded (already self-rolled back) | same | same |
356
+ | Later-step failure (undo moment) | requires pushing the map step on the undo stack in fan-out mode | same | same |
357
+ | Data the block needs | per element: source item + result | all results (+ items) | none (element steps' own undo args/results) |
358
+ | Failure isolation | per element | whole batch | per element step |
359
+ | DSL visibility | explicit on the map | explicit on the map | implicit (like compose) |
360
+ | Duplication with element step `undo`s | yes | yes | none |
361
+ | Compatibility | additive (MINOR) if opt-in | additive (MINOR) if opt-in | behavior change (MAJOR) |
362
+
363
+ **Reading of the evidence (not a decision)**: the gap is real (INV-19 and INV-20 VIOLATED, zero
364
+ coverage). O-01-b and O-01-c are complementary rather than alternatives: per-element vs bulk is
365
+ the author's cleanup granularity, and both need the same two plumbing pieces:
366
+
367
+ 1. The map step on the undo stack in every mode.
368
+ 2. Retained element results.
369
+
370
+ They also share one unsolved problem: fan-out elements still in flight at compensate time (F-05).
371
+ O-01-a is the most consistent with `compose`, but the most expensive and the only breaking one.
372
+ Whatever is chosen, name the hooks after the two moments (`compensate…` for map failure,
373
+ `undo…` for a later failure) or document that one block covers both.
374
+
375
+ ### Options for F-02 (compose retry resumes an undone child)
376
+
377
+ - **O-02-a · Fresh child per attempt**: when the compose step is retried after a failed attempt,
378
+ discard the stored child context and start a new one.
379
+ *Pros*: "retry the sub-saga" semantics, a one-line change at
380
+ [R: lib/ruby_reactor/step/compose_step.rb:97]. *Cons*: child steps without `undo` (left in
381
+ place) run again, so their side effect happens twice. Park/resume of a child mid-attempt must
382
+ still resume, so the fresh-child rule must distinguish "retry after failure" from "redelivery
383
+ after park". *Compat*: behavior fix (MINOR/PATCH, arguably a bug fix).
384
+ - **O-02-b · Rollback clears completion marks**: `rollback_completed_steps` removes each undone
385
+ step's result from `intermediate_results`, so any later resume re-runs it.
386
+ *Pros*: fixes the class of bug (any resume after a rollback), not just compose.
387
+ *Cons*: changes context state that dashboards and `execution_trace` readers see. Undone results
388
+ disappear, so post-mortem inspection loses data unless moved elsewhere.
389
+ *Compat*: MINOR, observable.
390
+ - **O-02-c · Disallow `retries` on `compose`** (definition-time error, point to inner-step
391
+ retries). *Pros*: removes the ambiguity. *Cons*: breaking (MAJOR). Loses a legitimate use (retry
392
+ a whole sub-saga after a transient failure deep inside it).
393
+
394
+ ### Options for F-03 (failures that skip rollback)
395
+
396
+ - **O-03-a · Resolve arguments inside the step's rescue, as a never-started failure**: an
397
+ argument/transform error becomes the step's `Failure` (with `step_name`), classified like
398
+ contention: no compensate, completed steps undone.
399
+ *Pros*: consistent with INV-07 and fixes F-13's attribution too. *Cons*: the "deferred
400
+ resolution" paths (`async_step`, `background before:`) need the same treatment in their own
401
+ process. Needs a new never-started error class.
402
+ - **O-03-b · Roll back on every `StandardError`**: remove the "Unknown errors - don't rollback"
403
+ branch [R: lib/ruby_reactor/executor/result_handler.rb:68-70].
404
+ *Pros*: tiny change, covers unknown future paths. *Cons*: the branch exists deliberately.
405
+ Rolling back after an internal executor bug runs user undo code on possibly inconsistent state.
406
+ Error attribution is still missing (F-13).
407
+ - **O-03-c · Document the non-`StandardError` boundary** (for the `Exception` half): running user
408
+ undo code on `SignalException`/`NoMemoryError` is unsafe. Document that an inline run offers no
409
+ rollback here, and that a worker run is redelivered (INV-32).
410
+ *Cons*: inline callers keep partial effects. It is only a documentation fix.
411
+
412
+ ### Options for F-04 (`async_step` rollback hooks never run)
413
+
414
+ - **O-04-a · Make the docs and DSL honest**: state that unit hooks never run, and reject (or warn
415
+ on) `compensate`/`undo` inside `async_step` at definition time.
416
+ *Pros*: no runtime change, removes the dead-code trap. *Cons*: loses a natural place for cleanup.
417
+ Authors must put it in the reader's `compensate`, far from the step. Rejecting would be MAJOR.
418
+ Warning is MINOR.
419
+ - **O-04-b · Unit-local saga**: `StepWorker` calls the unit's own `compensate` when its body finally
420
+ fails (after retries), inside the unit's job. This mirrors how an `async_reactor` child rolls
421
+ itself back (INV-28).
422
+ *Pros*: symmetric with `async_reactor`, the author's block becomes live, no cross-process state.
423
+ *Cons*: runs whether or not anyone reads the result (a behavior change for existing units that
424
+ declare `compensate`). Still no `undo` moment, since nobody tells a *successful* unit to roll back.
425
+ - **O-04-c · Parent-driven unit compensation on surfaced failure** (what the docs promise): when a
426
+ reader returns `Failure`, the parent's rollback also calls the unit's `compensate` with the
427
+ unit's recorded arguments/error, re-taking the unit step's locks.
428
+ *Pros*: matches documented intent. *Cons*: the parent's executor must run a step config against
429
+ another execution's recorded arguments and coordination owner. It only covers **read** units, and
430
+ it happens in the reader's process, after an arbitrary delay.
431
+
432
+ ### Options for F-05 (scheduling-dependent fan-out leftovers)
433
+
434
+ - **O-05-a · Collector waits for in-flight elements before resolving a fail-fast failure**, then
435
+ runs the map compensation (O-01-*) over every success.
436
+ *Pros*: deterministic leftover set (all successes), required anyway for O-01-b/c to be complete.
437
+ *Cons*: failure latency grows to the slowest in-flight element. Needs "dispatched" vs "not
438
+ started" accounting per index.
439
+ - **O-05-b · Late elements self-compensate**: an element that finishes after the fail-fast marker
440
+ is set rolls itself back instead of storing its success.
441
+ *Pros*: no waiting. *Cons*: the element must run its whole saga before undoing it (wasted side
442
+ effects), and it races with the collector.
443
+
444
+ ### Options for F-06 (raising `where`/`guard` compensates)
445
+
446
+ - **O-06-a** Classify condition errors as never-started (wrap them in a never-started error class
447
+ before result handling). *Cons*: slightly more error-class surface.
448
+ - **O-06-b** Document that conditions must not raise. *Cons*: leaves an easy trap.
449
+
450
+ ### Options for F-07 (nested `Halt`)
451
+
452
+ - **O-07-a** Propagate a composed child's `Halt` as the parent's `Halt` (map-consistent).
453
+ *Cons*: behavior change for anyone relying on a child halting "locally".
454
+ - **O-07-b** Make an element's `Halt` stop only that element (compose-consistent).
455
+ *Cons*: changes map semantics. The collector must represent "halted element" in results
456
+ (it already can: `_halt` [R: lib/ruby_reactor/map/element_executor.rb:166-171]).
457
+ - **O-07-c** Keep both, document them side by side. *Cons*: the asymmetry remains.
458
+
459
+ ### Options for F-08 (`Reactor.undo` outside the reactor lock)
460
+
461
+ - **O-08-a** `Reactor#undo` acquires the reactor-level lock/semaphore (with the configured `wait`)
462
+ before `undo_all`. *Cons*: a manual undo can now fail on contention. Needs an error shape.
463
+ - **O-08-b** Document it. *Cons*: concurrent forward run + manual undo stays possible.
464
+
465
+ ### Options for F-09 (units run after rollback)
466
+
467
+ - **O-09-a · Cancellation marker**: a failing dispatcher marks its not-yet-started units cancelled.
468
+ `StepWorker`/`Worker` checks the marker before running the body. *Cons*: units already running
469
+ are unaffected (still a race). Adds a storage write per rollback.
470
+ - **O-09-b · Gate `async_step` dispatch inside map elements** (run inline there, as
471
+ `ElementExecutor` intends [R: lib/ruby_reactor/map/element_executor.rb:51-55]). *Cons*:
472
+ reverses a deliberate decision [R: lib/ruby_reactor/executor/async_step_dispatch.rb:20-24] and
473
+ serializes the unit into the element.
474
+ - **O-09-c** Document the ordering explicitly. *Cons*: behavior unchanged.
475
+
476
+ ### Options for F-10 (DSL visibility)
477
+
478
+ - **O-10-a · Uniform rollback declarations on composites**: every construct that runs "more
479
+ steps" states its rollback policy in the DSL (`map … undo_each/undo_all`, `compose … rollback:
480
+ :replay` default, `async_step … compensate` either live or rejected).
481
+ *Cons*: more DSL surface. Composes would gain a knob they may not need.
482
+ - **O-10-b · Rollback plan introspection**: `Reactor.rollback_plan` (and a dashboard view) lists,
483
+ per step, what a failure after it would roll back and what it would leave in place, derived from
484
+ the rules in [execution-order.md](execution-order.md#1-rollback-algorithm).
485
+ *Pros*: no semantic change, helps readers and reviewers. *Cons*: the plan must be kept in sync
486
+ with the executor. It is descriptive only.
487
+
488
+ ### Options for Low findings
489
+
490
+ | Finding | Option | Con |
491
+ |---|---|---|
492
+ | F-11 lock gap | Document the gap next to `rollback_wait` and recommend idempotent, state-checking undos | documentation only |
493
+ | F-12 async retries invisible | Emit `retry_attempt` from `StepWorker`'s loop (middlewares are already built there) | more events on an existing hook (observability change) |
494
+ | F-13 missing attribution | Carry `step_name` onto `CompensationError`-derived and resolution failures (overlaps O-03-a) | failure shape grows |
495
+ | F-14 dead collect default | Delete `Map::Helpers#apply_collect_block` | none known. Confirm no external caller |
496
+ | F-15 lock key asymmetry | Emit the unprefixed key on release (the spec pins the current shape, so update it) | event payload change (observability MINOR) |
497
+ | F-16 interrupt status docs | Fix interrupts.md to say `failed` | documentation only |
498
+
499
+ ---
500
+
501
+ *All options above are proposals pending later analysis. Nothing in this initiative changes
502
+ library behavior, tests or the demo application.*
@@ -0,0 +1,109 @@
1
+ # Invariants
2
+
3
+ Testable propositions about ordering and rollback. Each has a status, evidence and existing-spec
4
+ coverage. Status scale: **HOLDS** (evidence agrees, no counter-example found) · **VIOLATED**
5
+ (a reproducible counter-example exists, cited as `[O]`) · **CONDITIONAL** (holds only under the
6
+ listed conditions) · **UNDETERMINED** (evidence insufficient).
7
+
8
+ Every `[O: S-…]` is a block in [`../evidence/output.txt`](../evidence/output.txt).
9
+ Details and sequences are in [execution-order.md](execution-order.md).
10
+
11
+ **Updated for 008.** INV-06, 07, 13, 19, 20, 22 and 24 now **HOLD**, each covered by a spec under
12
+ `spec/ruby_reactor/rollback/` that fails on the baseline. Their original counter-examples are in
13
+ git history.
14
+
15
+ ## A. Ordering & the rollback algorithm
16
+
17
+ | Id | Statement | Scope | Status | Evidence | Coverage |
18
+ |---|---|---|---|---|---|
19
+ | INV-01 | The failing step's compensate runs **before** any undo. | all reactor executions | HOLDS | [R: lib/ruby_reactor/executor/compensation_manager.rb:35-65] [O: S-plain-01, S-compose-01, S-map-01] | [T: spec/ruby_reactor/order_processing_reactor_spec.rb:91] |
20
+ | INV-02 | Completed steps are undone in reverse completion order, including across independent DAG branches. | one execution's undo stack | HOLDS | [R: …/compensation_manager.rb:69] [O: S-plain-09] | [T: spec/ruby_reactor/order_processing_reactor_spec.rb:91] (linear only; DAG branches: none) |
21
+ | INV-03 | After a step fails, no further step of the same execution runs. | inline and worker | HOLDS | [R: lib/ruby_reactor/executor/step_executor.rb:47] [O: S-plain-01, S-bg-01] | implicit in most failure specs; no dedicated example |
22
+ | INV-04 | A compensate or undo that fails (returns `Failure` or raises) does not stop the remaining undos, and it is listed on `Failure#rollback_failures`. | all | HOLDS | [O: S-plain-03, S-plain-04, S-compose-07] | [T: spec/ruby_reactor/compensation_failure_spec.rb:30] [T: spec/ruby_reactor/step_coordination/rollback_under_contention_spec.rb:51, :59, :67, :78] |
23
+ | INV-05 | `Halt` never rolls back. `Skipped` steps are never undone. | inline | HOLDS | [O: S-plain-05, S-plain-06] | [T: spec/ruby_reactor/halt_status_spec.rb:30] [T: spec/ruby_reactor/skipped_rollback_spec.rb:6] |
24
+ | INV-06 | **Every** failure that occurs after at least one step has completed rolls back the completed steps. | all | HOLDS (008) for every exception, `StandardError` or not (R-16). Only an interruption (signal, exit, out of memory, enclosing timeout) runs no rollback, by design; the caller-process run is stored `aborted` and `Reactor.undo(id)` rolls it back | [O: S-plain-07, S-edge-03, S-edge-03b] | [T: spec/ruby_reactor/rollback/failure_rollback_spec.rb] [T: spec/ruby_reactor/rollback/aborted_execution_spec.rb] |
25
+ | INV-07 | A step whose body never started is never compensated. | all | HOLDS (008) | Own-coordination contention [O: S-lock-03, S-edge-01], refused async dispatch, argument/type validation [O: S-edge-05], and (since 008) argument resolution errors [O: S-plain-07]. `where`/`guard` were removed (008 R-15) [O: S-edge-04] | [T: spec/ruby_reactor/step_coordination/contention_spec.rb:173] [T: spec/ruby_reactor/rollback/failure_rollback_spec.rb] |
26
+ | INV-08 | A step whose body ran and then failed (including an invalid output) is compensated. | same-process steps | HOLDS (exception: async units, INV-24) | [O: S-plain-01, S-plain-08] | [T: spec/ruby_reactor/validations_spec.rb:1139] |
27
+ | INV-09 | Halt semantics are the same at every nesting level: a `Halt` anywhere stops the top-level execution without rollback. | compose, map | **VIOLATED** | A map element's `Halt` halts the parent [O: S-map-11]. A composed child's `Halt` becomes a plain `Success(nil)` and the parent **continues** [O: S-compose-08]. [R: lib/ruby_reactor/step/compose_step.rb:106-114] | none |
28
+
29
+ ## B. Retries
30
+
31
+ | Id | Statement | Scope | Status | Evidence | Coverage |
32
+ |---|---|---|---|---|---|
33
+ | INV-10 | Retries are exhausted before compensation. Compensation runs exactly once, after the last attempt. | same-process steps, inline and worker | HOLDS | [O: S-retry-01, S-retry-04, S-bg-03] | [T: spec/ruby_reactor/step_retries/execution_paths_spec.rb:81] [T: spec/map/map_retry_spec.rb:49] |
34
+ | INV-11 | A failed attempt followed by a successful one is never compensated. | same-process steps | HOLDS | [O: S-retry-03, S-bg-04, S-map-09, S-map-10] | [T: spec/ruby_reactor/retry_reexecution_spec.rb:13] [T: spec/ruby_reactor/retry_signals_spec.rb:14] |
35
+ | INV-12 | A `retry: false` failure is not retried. | all | HOLDS | [O: S-retry-02] | [T: spec/ruby_reactor/retry_signals_spec.rb:32] |
36
+ | INV-13 | A retried unit of work re-runs from a state in which its already-rolled-back work is **not** treated as done. | step `retries` (incl. a composed child's steps) | HOLDS (008) | Only steps retry: `retries` on a `compose`/`async_reactor` is rejected at definition time (008 R-14) [O: S-compose-05]. A child's own step retries inside the child and a later failure undoes each child step once [O: S-compose-05b]. A park/resume still resumes | [T: spec/ruby_reactor/rollback/compose_retry_spec.rb] |
37
+ | INV-14 | Retry delivery depends only on where the step runs. Inline: in-process `sleep`. Worker/background: re-enqueue. Fan-out element: element re-enqueue. `async_step`: loop inside its job. | all | HOLDS (descriptive) | [O: S-retry-01, S-bg-03, S-map-09, S-map-10, S-async-07] | [T: spec/async_retry_integration_spec.rb:75] |
38
+
39
+ ## C. Composition
40
+
41
+ | Id | Statement | Scope | Status | Evidence | Coverage |
42
+ |---|---|---|---|---|---|
43
+ | INV-15 | A failing composed child rolls back its own completed steps **before** the parent starts its rollback. | compose, any depth | HOLDS | [O: S-compose-01, S-compose-04] | [T: spec/compose_spec.rb:197] |
44
+ | INV-16 | A completed composed child is fully undone (all its completed steps, reverse order) when a later parent step fails. This includes **earlier sibling composes** when a later compose fails. | compose | HOLDS | [O: S-compose-02, S-compose-03, S-compose-06] | [T: spec/compose_spec.rb:192] (single child; sibling case: none) |
45
+ | INV-17 | Nesting unwinds innermost-first. Nesting depth never changes the rule. | compose ⊂ compose, map ⊂ compose, compose ⊂ map element | HOLDS | [O: S-compose-04, S-map-07, S-map-08] | none |
46
+ | INV-18 | A composed child's rollback failures reach the parent's `rollback_failures`. | compose | HOLDS | [O: S-compose-07] | [T: spec/ruby_reactor/step_coordination/rollback_under_contention_spec.rb:78] |
47
+
48
+ ## D. Map
49
+
50
+ | Id | Statement | Scope | Status | Evidence | Coverage |
51
+ |---|---|---|---|---|---|
52
+ | INV-19 | When a map fails, its elements that already **succeeded** are rolled back. | map, fail_fast | HOLDS (008) | `MapStep#compensate` replays every completed element's undo stack, highest index first [O: S-map-01, S-map-04, S-map-04b, S-map-07, S-map-08] | [T: spec/ruby_reactor/rollback/map_rollback_spec.rb] [T: spec/ruby_reactor/rollback/map_fan_out_settle_spec.rb] |
53
+ | INV-20 | When a step **after** a completed map fails, the map's elements are rolled back. | map | HOLDS (008) | The map step's `undo` is the same element replay; a fan-out map is pushed on the undo stack by the collector [O: S-map-03, S-map-06] | [T: spec/ruby_reactor/rollback/map_rollback_spec.rb] [T: spec/ruby_reactor/rollback/map_fan_out_settle_spec.rb] |
54
+ | INV-21 | A failed element rolls back its own completed steps (compensate failing step, undo the rest) regardless of mode. | map element | HOLDS | [O: S-map-01, S-map-02, S-map-04, S-map-05, S-map-08] | none dedicated |
55
+ | INV-22 | *Restated in 008.* After a fail-fast failure, the set of elements **left in place** is deterministic: it is empty. (Which elements run in fan-out mode still depends on job scheduling, inherently.) | map, fail_fast | HOLDS (008) | The collector applies the failure only once every index has settled, then rolls back every completed element [O: S-map-04, S-map-04b] | [T: spec/ruby_reactor/rollback/map_fan_out_settle_spec.rb] (100 shuffled job orders, SC-003) |
56
+ | INV-23 | `fail_fast false`: the map step succeeds even when elements fail. Failed elements are rolled back individually, successes kept, and the consumer must inspect the results. | map | HOLDS (by design) | [O: S-map-02, S-map-05] | [T: spec/map/map_fail_fast_spec.rb:77] [T: spec/map/fail_fast_spec.rb:172, :196] |
57
+
58
+ ## E. Async units (`async_step`, `async_reactor`)
59
+
60
+ | Id | Statement | Scope | Status | Evidence | Coverage |
61
+ |---|---|---|---|---|---|
62
+ | INV-24 | *Restated in 008.* A unit's own `compensate` runs once, in its own job, after its final attempt fails, whether or not a reader surfaces it; an `undo` on `async_step` is rejected (inline) or warned (class). | async_step | HOLDS (008) | [O: S-async-01, S-async-02, S-async-07] | [T: spec/ruby_reactor/rollback/async_step_compensate_spec.rb] [T: spec/ruby_reactor/dsl/async_step_spec.rb:111] (tightened) |
63
+ | INV-25 | A unit never enters the parent's undo stack, so parent rollback never touches it. | async_step, async_reactor | HOLDS (by design, documented) | [R: lib/ruby_reactor/executor/result_handler.rb:137] [O: S-async-03, S-async-06] | [T: spec/ruby_reactor/dsl/async_reactor_spec.rb:26, :123] |
64
+ | INV-26 | A unit's failure fails the parent only if a reader returns `Failure`. | async_step, async_reactor | HOLDS (by design) | [O: S-async-01, S-async-02, S-async-04, S-async-05] | [T: spec/ruby_reactor/dsl/async_step_spec.rb:98] [T: spec/ruby_reactor/dsl/async_reactor_spec.rb:16, :77] |
65
+ | INV-27 | A unit dispatched by an execution that then fails and rolls back does not perform its side effect **after** that rollback. | async_step, async_reactor, async_step inside a map element | **VIOLATED** | Unit bodies run after the parent's undo [O: S-async-03, S-async-06, S-async-08, S-map-12]. In production the relative order is a race | none |
66
+ | INV-28 | An `async_reactor` child rolls back its own steps on its own failure, in its worker. | async_reactor | HOLDS | [O: S-async-04, S-async-05] | [T: spec/ruby_reactor/dsl/async_reactor_spec.rb:16] |
67
+ | INV-29 | An async unit's retries behave like a same-process step's (one compensation at exhaustion, retry middleware events). | async_step | **VIOLATED** (compensation half fixed in 008) | One compensation at exhaustion since 008 [O: S-async-07]. Retries still loop inside the job with no `retry_attempt` event [R: lib/ruby_reactor/step_worker.rb:275-289] | [T: spec/ruby_reactor/dsl/async_step_spec.rb:123] (attempt count only) [T: spec/ruby_reactor/rollback/async_step_compensate_spec.rb] |
68
+ | INV-30 | A reader that cannot get the unit's result in time fails and rolls back the parent. | async_step reader | HOLDS | [O: S-async-08] | [T: spec/ruby_reactor/async_waiter_spec.rb] (waiter timeout only) |
69
+
70
+ ## F. background / worker parity
71
+
72
+ | Id | Statement | Scope | Status | Evidence | Coverage |
73
+ |---|---|---|---|---|---|
74
+ | INV-31 | A worker-side run produces the same order and rollback coverage as an inline run of the same shape, including steps that ran in the caller before a `background after:`/`before:` hand-off. | background, fan-out map, compose in worker | HOLDS | [O: S-bg-01, S-bg-02, S-compose-06, S-map-04] (vs S-map-01) | [T: spec/ruby_reactor/step_retries/execution_paths_spec.rb:81] |
75
+ | INV-32 | A worker crash re-drives from the last checkpoint. Completed steps are not re-run, and the in-flight step may run twice (at-least-once). | background | HOLDS | [O: S-edge-02] [R: lib/ruby_reactor/executor.rb:55] | [T: spec/ruby_reactor/checkpoint_spec.rb:85] |
76
+
77
+ ## G. Locks & coordination
78
+
79
+ | Id | Statement | Scope | Status | Evidence | Coverage |
80
+ |---|---|---|---|---|---|
81
+ | INV-33 | A reactor-level lock/semaphore is held from before the first step until after the last undo. | reactor `with_lock`/`with_semaphore` | HOLDS | [O: S-lock-01] [R: lib/ruby_reactor/executor.rb:161] | [T: spec/ruby_reactor/telemetry_spec.rb:353] (events only) |
82
+ | INV-34 | A reactor-level lock is released while the execution is paused at an interrupt and re-acquired on `continue`. | interrupts | HOLDS | [O: S-intr-01] | none |
83
+ | INV-35 | Every rollback of a step is serialized with forward runs by the same locks the forward run held. | step locks, `Reactor.undo` | **CONDITIONAL** | Step-level lock/semaphore is re-taken around compensate/undo [O: S-lock-02, S-lock-04]. The **reactor-level** lock is **not** taken by `Reactor.undo(id)` [O: S-intr-04] [R: lib/ruby_reactor/reactor.rb:188] | [T: spec/ruby_reactor/step_coordination/rollback_spec.rb:31, :52]; Reactor.undo: none |
84
+ | INV-36 | A rollback that cannot re-take its step lock within `rollback_wait` is skipped **and reported**, never silently dropped. | step locks | HOLDS | [O: S-lock-05] | [T: spec/ruby_reactor/step_coordination/rollback_under_contention_spec.rb:32] |
85
+ | INV-37 | A composed child re-enters the parent's reactor lock without releasing the parent's hold. | compose + reactor lock | HOLDS | [O: S-lock-06] (count back to 1 in `b`) | [T: spec/ruby_reactor/step_coordination/reentrancy_spec.rb:53] |
86
+ | INV-38 | Lock middleware events name the same key on acquire and release. | reactor-level lock | **VIOLATED** (known, pinned) | `lock_acquired:rk` vs `lock_released:lock:rk` [O: S-lock-01] | [T: spec/ruby_reactor/step_coordination/attribution_spec.rb:41] (documents the asymmetry) |
87
+
88
+ ## H. Interrupts & manual control
89
+
90
+ | Id | Statement | Scope | Status | Evidence | Coverage |
91
+ |---|---|---|---|---|---|
92
+ | INV-39 | Steps completed before a pause are undone when the run fails after `continue`. | interrupts | HOLDS | [O: S-intr-01] | [T: spec/ruby_reactor/interrupt_undo_spec.rb:77] (explicit undo) |
93
+ | INV-40 | Invalid interrupt payload past `max_attempts` rolls back completed steps. | interrupts | HOLDS | [O: S-intr-02] | [T: spec/integration/interrupt_max_attempts_spec.rb:9] |
94
+ | INV-41 | `Reactor.cancel` never rolls back. `Reactor.undo(id)` rolls back and then cancels. | manual | HOLDS | [O: S-intr-03, S-intr-04] | [T: spec/ruby_reactor/interrupt_undo_spec.rb:77, :94] |
95
+
96
+ ## Coverage summary
97
+
98
+ | Status | Count | Ids |
99
+ |---|---|---|
100
+ | HOLDS | 36 | INV-01–08, 10–26, 28, 30–34, 36, 37, 39–41 |
101
+ | VIOLATED | 4 | INV-09, 27, 29, 38 |
102
+ | CONDITIONAL | 1 | INV-35 |
103
+ | UNDETERMINED | 0 | — |
104
+
105
+ (41 invariants in total. INV-38's violation is known and pinned by a spec.)
106
+
107
+ **No existing spec covers** (after 008): INV-09, INV-27, INV-34, the `Reactor.undo` half of
108
+ INV-35, the DAG half of INV-02, and the sibling-compose half of INV-16. INV-17 is now covered by
109
+ the nested map/compose examples in `spec/ruby_reactor/rollback/map_rollback_spec.rb`.