ruby_reactor 0.7.0 → 0.7.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (51) hide show
  1. checksums.yaml +4 -4
  2. data/.claude/skills/demo-app-e2e-verify/SKILL.md +226 -0
  3. data/.claude/skills/speckit-demo-tests/SKILL.md +144 -0
  4. data/.release-please-manifest.json +1 -1
  5. data/.specify/feature.json +1 -1
  6. data/.specify/memory/constitution.md +79 -12
  7. data/.specify/templates/tasks-template.md +7 -0
  8. data/CHANGELOG.md +11 -0
  9. data/CLAUDE.md +2 -2
  10. data/README.md +24 -9
  11. data/lib/ruby_reactor/context.rb +7 -0
  12. data/lib/ruby_reactor/context_serializer.rb +13 -0
  13. data/lib/ruby_reactor/dsl/lockable.rb +2 -2
  14. data/lib/ruby_reactor/dsl/template_helpers.rb +11 -3
  15. data/lib/ruby_reactor/executor/compensation_manager.rb +58 -45
  16. data/lib/ruby_reactor/executor/ordered_lock_support.rb +9 -9
  17. data/lib/ruby_reactor/executor/result_handler.rb +35 -10
  18. data/lib/ruby_reactor/executor/retry_manager.rb +4 -1
  19. data/lib/ruby_reactor/executor/step_executor.rb +19 -13
  20. data/lib/ruby_reactor/executor.rb +17 -15
  21. data/lib/ruby_reactor/map/element_executor.rb +7 -1
  22. data/lib/ruby_reactor/map/helpers.rb +9 -7
  23. data/lib/ruby_reactor/map/result_enumerator.rb +2 -0
  24. data/lib/ruby_reactor/map/sweeper.rb +1 -1
  25. data/lib/ruby_reactor/open_telemetry.rb +7 -4
  26. data/lib/ruby_reactor/ordered_lock.rb +3 -3
  27. data/lib/ruby_reactor/rspec/matchers.rb +61 -11
  28. data/lib/ruby_reactor/rspec/test_subject.rb +8 -8
  29. data/lib/ruby_reactor/step/map_step.rb +5 -1
  30. data/lib/ruby_reactor/step.rb +10 -4
  31. data/lib/ruby_reactor/step_signals.rb +33 -0
  32. data/lib/ruby_reactor/storage/adapter.rb +4 -0
  33. data/lib/ruby_reactor/storage/redis_adapter.rb +1 -72
  34. data/lib/ruby_reactor/storage/redis_reactor_scan.rb +116 -0
  35. data/lib/ruby_reactor/version.rb +1 -1
  36. data/lib/ruby_reactor/web/api.rb +23 -6
  37. data/lib/ruby_reactor/web/public/assets/index-BQvIWPdx.css +1 -0
  38. data/lib/ruby_reactor/web/public/assets/index-Dw4KV4QY.js +22 -0
  39. data/lib/ruby_reactor/web/public/index.html +2 -2
  40. data/lib/ruby_reactor.rb +56 -7
  41. metadata +7 -11
  42. data/lib/ruby_reactor/web/public/assets/index-B46p-M6K.css +0 -1
  43. data/lib/ruby_reactor/web/public/assets/index-DPmP4yXT.js +0 -22
  44. data/specs/001-background-async-steps/checklists/requirements.md +0 -39
  45. data/specs/001-background-async-steps/contracts/public-dsl.md +0 -154
  46. data/specs/001-background-async-steps/data-model.md +0 -117
  47. data/specs/001-background-async-steps/plan.md +0 -168
  48. data/specs/001-background-async-steps/quickstart.md +0 -102
  49. data/specs/001-background-async-steps/research.md +0 -150
  50. data/specs/001-background-async-steps/spec.md +0 -146
  51. data/specs/001-background-async-steps/tasks.md +0 -271
@@ -1,39 +0,0 @@
1
- # Specification Quality Checklist: Background Execution & Real Async Steps
2
-
3
- **Purpose**: Validate specification completeness and quality before proceeding to planning
4
- **Created**: 2026-08-16
5
- **Feature**: [spec.md](../spec.md)
6
-
7
- ## Content Quality
8
-
9
- - [x] No implementation details (languages, frameworks, APIs)
10
- - [x] Focused on user value and business needs
11
- - [x] Written for non-technical stakeholders
12
- - [x] All mandatory sections completed
13
-
14
- ## Requirement Completeness
15
-
16
- - [x] No [NEEDS CLARIFICATION] markers remain
17
- - [x] Requirements are testable and unambiguous
18
- - [x] Success criteria are measurable
19
- - [x] Success criteria are technology-agnostic (no implementation details)
20
- - [x] All acceptance scenarios are defined
21
- - [x] Edge cases are identified
22
- - [x] Scope is clearly bounded
23
- - [x] Dependencies and assumptions identified
24
-
25
- ## Feature Readiness
26
-
27
- - [x] All functional requirements have clear acceptance criteria
28
- - [x] User scenarios cover primary flows
29
- - [x] Feature meets measurable outcomes defined in Success Criteria
30
- - [x] No implementation details leak into specification
31
-
32
- ## Notes
33
-
34
- - Items marked incomplete require spec updates before `/speckit-clarify` or `/speckit-plan`.
35
- - Note: the spec names concrete terms (`Sidekiq`, `ActiveJob`, `Redis`) only inside the **Assumptions** section, where they describe existing constraints already codified in the project constitution, not new implementation choices being introduced by this feature. All requirement- and success-criteria-level language remains technology-agnostic.
36
- - 2026-08-16 (post-plan review): added FR-014/SC-006 covering dashboard visibility for `async_step`/`async_reactor`, a gap found while reviewing plan.md against the constitution's Observability principle. Re-validated against this checklist — all items still pass.
37
- - 2026-08-21 (final consistency pass): fixed stale "blocking-poll" wording left from the superseded wait design (plan Summary/tree, contract, data-model); fixed the async_step dispatch ordering in data-model (record + ref written BEFORE enqueue, matching the F2 rule its own Step Result Record row states); fixed "record absent = pending" to "record still `dispatched` = pending"; reconciled plan Scale/Scope with the tree (dependency_graph.rb unchanged; executor.rb added for the completion-signal publish); aligned FR-015 with contract/research on single-slot semaphores. All items re-verified passing.
38
- - 2026-08-20 (wait-mechanism & locks review): FR-005 revised from tight-poll to notified wait (record-first publish, subscribe-first check, fallback re-check — durable record stays the source of truth, timeout bound unchanged); added FR-015 (no lock-owner sharing across the async boundary + dispatch-time same-key deadlock guard; reentrancy stays compose-only) and FR-016 (async_reactor dispatch reuses the full pre-enqueue sequence: child input validation + ordered-lock nonce, a gap found during the locks review). Clarifications Session 2026-08-20 records both decisions. All checklist items re-verified passing.
39
- - 2026-08-16 (full-artifact review): fixed FR-006 (claimed a nonexistent "existing step-result storage"; now: new per-step record for `async_step`, own execution record for `async_reactor`); clarified FR-003 scope (compose's `async` flag is removed too — it sets the same flag; map's `async` is a different mechanism and stays); added edge cases for `returns` × async unit, `background` × whole-reactor `async true`, and a paused awaited `async_reactor` child. Contract additionally pins failure-read semantics for `result()` on async units and lock-window behavior. All checklist items re-verified passing.
@@ -1,154 +0,0 @@
1
- # Contract: Public DSL Surface
2
-
3
- This gem's "interface" is its Ruby DSL, consumed by host applications that subclass `RubyReactor::Reactor`. This contract documents the exact public surface this feature adds, changes, or removes, so implementation and tests can be checked against it directly.
4
-
5
- ## Removed
6
-
7
- ```ruby
8
- class MyReactor < RubyReactor::Reactor
9
- step :do_thing do
10
- async true # REMOVED — raises a definition-time error (FR-003)
11
- end
12
- end
13
- ```
14
-
15
- ```ruby
16
- class MyReactor < RubyReactor::Reactor
17
- compose :sub_flow, SubReactor do
18
- async true # ALSO REMOVED — same StepConfig hand-off flag, same definition-time error
19
- end
20
- end
21
- ```
22
-
23
- **Error contract**: attempting to call `async` inside a `step` **or `compose`** block MUST raise a `RubyReactor::Error::ValidationError` (or a new dedicated error subclass — implementation's choice, but it MUST be raised at reactor **class-definition** time, not at `run` time) whose message names both the removed syntax and its replacement(s): `background after:`/`background before:`, `async_step`, `async_reactor`. (`ComposeBuilder#async` sets the very `StepConfig` flag this feature removes — `dsl/compose_builder.rb:62` — so it cannot survive; the exact migration for a compose that handed off is `background before: :<that compose step>`, which reproduces the old semantics precisely — the flagged step and everything after it moved to the worker — without the author having to identify a predecessor.)
24
-
25
- **Not removed**: the `async` option inside a `map` block (`MapBuilder#async`, `dsl/map_builder.rb:43`) is a map-internal element-dispatch mode passed as a step *argument* — the map's own `StepConfig` is hardcoded `async: false` (`map_builder.rb:111`), so it does not touch the removed flag and keeps working unchanged.
26
-
27
- ## Added: `background after:` / `background before:`
28
-
29
- ```ruby
30
- class MyReactor < RubyReactor::Reactor
31
- step :first
32
- step :second
33
- background after: :second # :second is the LAST step in the calling process
34
- step :third
35
- end
36
-
37
- class EquivalentInLinearFlow < RubyReactor::Reactor
38
- step :first
39
- step :second
40
- background before: :third # :third is the FIRST step in the worker
41
- step :third
42
- end
43
- ```
44
-
45
- - **Signature**: `self.background(after: nil, before: nil)` — reactor class macro, exactly one keyword supplied.
46
- - **The two forms** name one cut point from opposite sides, and each carries a *guarantee about the step it names*:
47
- - `after: :x` — `:x` runs in the calling process, and is the last step to do so.
48
- - `before: :x` — `:x` runs in the worker, and is the first step to do so; it never executes in the calling process.
49
-
50
- In a linear reactor where `:third` immediately follows `:second`, `after: :second` and `before: :third` are equivalent. In a branching workflow they are not, and the author picks whichever step they actually need pinned.
51
- - **Constraints** (all raise at class-definition time, FR-002):
52
- - at most one `background` declaration per reactor class;
53
- - the named step must be defined in the class (either keyword);
54
- - the named step must not be an `interrupt` (either keyword) — an interrupt re-enters the reactor from a foreground process, so `after:` would silently never fire on resume and `before:` would enqueue a worker that instantly pauses and swallows the `InterruptResult`; the error points to `interrupt :name, resume: :background` as the correct spelling;
55
- - exactly one of `after:`/`before:` — supplying both, or neither, raises;
56
- - combining `background` with whole-reactor `async true` raises (the hand-off point would be silently meaningless inside a reactor that already runs entirely in a worker — see spec Edge Cases).
57
- - **Runtime contract**: hand-off is triggered by *reaching the named step*, not by the declaration's lexical position — the declaration may sit anywhere in the class body. For `after: :x`, the trigger fires when `:x` completes; for `before: :x`, it fires when `:x` is selected to run, and `:x` is left unexecuted for the worker to run. Either way: checkpoint, enqueue the remainder via `configuration.async_router`, return an `DispatchResult` to the caller. In a DAG with parallel branches, any independent step that became ready and executed before the trigger fired has already run in the calling process; everything not yet executed at the trigger moment runs in the worker — this caveat is identical for both forms. Compensation for worker-side step failures works exactly as it does for any same-process step failure — `background` only changes *where* code runs, not the saga/compensation contract (US1 acceptance scenarios 1-2). Inside the worker the hand-off never re-triggers (the existing `inline_async_execution` guard).
58
- - **Never-reached trigger**: if the named step is skipped by a `where`/guard condition, or the reactor fails before reaching it, the hand-off simply never fires and the run completes in the calling process. No step is stranded — the hand-off only ever relocates *remaining* work.
59
- - **`before:` naming the first step** is legal, and is not the same as whole-reactor `async true`: every step body runs in the worker, but input validation still happens in the calling process, so invalid inputs fail the caller synchronously instead of failing inside a worker.
60
-
61
- ## Added: `async_step`
62
-
63
- ```ruby
64
- class MyReactor < RubyReactor::Reactor
65
- async_step :send_email do
66
- argument :to, input(:email)
67
- run { |args| Mailer.send(args[:to]) }
68
- end
69
-
70
- step :do_something_same_thread do
71
- run { do_work }
72
- end
73
-
74
- step :check_email do
75
- argument :email, result(:send_email) # blocks (notified wait, bounded by timeout) until :send_email is done
76
- run { |args| ... }
77
- end
78
- end
79
- ```
80
-
81
- - **Signature**: `self.async_step(name, impl = nil, &block)` — same call shape as `step`, builds the same `StepConfig` fields (`argument`, `run`, `compensate`, `undo`, `validate_args`, `validate_output`, `retries`, etc. all still work identically inside the block).
82
- - **Runtime contract**:
83
- - Dispatches the step's work to an independent worker job; does **not** halt the calling reactor's execution of other ready steps that don't depend on it (US2 acceptance scenario 1).
84
- - Any step that references `result(:async_step_name)` blocks — notified wait: woken by the completion signal the finishing worker publishes after its durable write, with a coarse fallback re-check of the durable record, bounded overall by `Configuration#async_wait_timeout` (spec Clarifications, Session 2026-08-20) — until the async step's terminal result is available. **Read semantics**: on `Success`, the reader receives the same deserialized raw value a same-process step's result would produce (US2 acceptance scenario 2, FR-006); on `Failure`, the reader receives the `Failure` object itself as the argument value — a same-process step's failure would have halted the reactor before any reader ran, so there is no sync-behavior to mirror here, and injecting the `Failure` is what lets the reader "see the failure and decide" per the spec's clarified compensation model (US2 acceptance scenario 3). This matches `async_reactor`'s wrapped-result-on-inspection pattern.
85
- - `returns :async_step_name` raises at class-definition time — the reactor's return value must come from a same-process step (spec Edge Cases).
86
- - Dispatch is **not** suppressed inside a worker: an `async_step` declared after a `background` hand-off point (or reached during a worker resume) still dispatches to its own independent job — the existing `inline_async_execution` guard suppresses only the *hand-off* re-trigger, never `async_step`/`async_reactor` dispatch (spec Edge Cases).
87
- - Reactor-level `lock`/`semaphore`/`rate_limit` windows are held by the process executing the reactor's own steps — the async step's work runs *outside* those windows (in its own job, which acquires nothing). A step body that needs mutual exclusion must arrange it itself.
88
- - If the async step fails and no later step reads its result, the parent reactor's compensation is **not** automatically triggered (US2 acceptance scenario 3, FR-011). A later step that does read the result and observes failure may itself return `Failure` to trigger compensation.
89
- - `compensate`/`undo` blocks declared on an `async_step` still register normally — they only run if the step's own failure is surfaced into the parent's compensation path via the opt-in mechanism above, never automatically.
90
- - A reference to the dispatched unit is recorded on the parent's own context (`composed_contexts[:send_email] = { type: :async_step_ref, ... }`) at dispatch time, and the web dashboard renders `send_email` as an `async_step`-typed node (FR-008, FR-014).
91
-
92
- ## Added: `async_reactor`
93
-
94
- ```ruby
95
- class MyReactor < RubyReactor::Reactor
96
- async_reactor :create_profile, CreateProfileReactor # fire-and-forget, no downstream reference
97
-
98
- async_reactor :create_account, CreateAccountReactor do
99
- argument :user_id, input(:user_id)
100
- end
101
-
102
- step :verify_all do
103
- argument :account, result(:create_account) # blocks until create_account finishes
104
- run do |args, context|
105
- if args[:account].success?
106
- Success(args[:account].value)
107
- else
108
- Failure(args[:account].error) # opt-in compensation trigger
109
- end
110
- end
111
- end
112
- end
113
- ```
114
-
115
- - **Signature**: `self.async_reactor(name, child_reactor_class, &block)` — `argument` inside the block maps parent-visible sources to the child reactor's inputs, same shape as `compose`.
116
- - **Dispatch contract** (the part that runs in the parent's process, FR-015/FR-016):
117
- - Dispatch applies the same pre-enqueue safeguards as a top-level async run: the child's inputs are validated and, if the child declares `with_ordered_lock`, its ordering nonce is assigned at enqueue. A child-input validation failure fails **the dispatching step** (normal saga handling in the parent) — this is a dispatch failure, not a child-execution failure, and is deliberately outside FR-009's no-auto-compensation rule.
118
- - Deadlock guard: if the child declares an exclusive `lock` (or a `semaphore` with `limit: 1`) whose resolved key equals one the dispatching execution currently holds, the dispatch step fails immediately with an error naming the lock key and both reactor classes. The error message MUST enumerate the three remediations, ranked: (1) use `compose` if the child belongs in the parent's critical section and its result is needed — the wait means the work is sequential anyway; (2) narrow the lock keys if parent and child actually protect different resources; (3) restructure so the locked reactor never reads the child's result — fire-and-forget with verification in the child itself or in a successor reactor outside the lock window. Lock ownership is never shared across the async boundary (parent and child run concurrently — shared ownership would break mutual exclusion); owner-based reentrancy remains for `compose` only. Transitive cross-execution cycles are out of the guard's reach (undetectable at dispatch) — documentation advises consistent key-acquisition order, with the FR-005 timeout as backstop.
119
- - **Runtime contract**:
120
- - Dispatches `child_reactor_class` to run independently via `configuration.async_router`, linked to the parent by the child's `execution_id` for traceability/logging (FR-008, US3 acceptance scenario 4) — never added to the parent's compensation graph (FR-009).
121
- - If nothing in the parent reads `result(:name)`, the child's eventual failure never affects the parent (US3 acceptance scenario 1).
122
- - If a later step reads `result(:name)`, it blocks (same notified-wait policy as `async_step` — the child publishes its completion signal after its terminal save) until the child reactor reaches a terminal state, then receives the child's actual `Success`/`Failure` result object (not the enqueue-time `DispatchResult`), and may inspect `.success?`/`.value`/`.error` to decide whether to itself return `Failure` (US3 acceptance scenarios 2-3, FR-010).
123
- - A reference is recorded on the parent's own context (`composed_contexts[:create_profile] = { type: :async_reactor_ref, execution_id:, reactor_class_name:, ... }`) at dispatch time — the web dashboard renders `create_profile`/`create_account` as `async_reactor`-typed nodes and lets an operator open the linked child execution, the same drill-down `compose`/`map` already offer (FR-008, FR-014, US3 acceptance scenario 4).
124
- - `returns :async_reactor_name` raises at class-definition time, same as for `async_step`.
125
- - A child that *pauses* at an interrupt step is not terminal: a reader keeps polling and hits the FR-005 timeout unless the child is resumed within the bound (spec Edge Cases). The child is an ordinary independently-recoverable execution — the existing sweeper/durability machinery covers its crash recovery with no new mechanism.
126
-
127
- ## Added: `interrupt ..., resume: :background`
128
-
129
- ```ruby
130
- class MyReactor < RubyReactor::Reactor
131
- interrupt :webhook, resume: :background do
132
- validate_payload { required(:status).filled(:string) }
133
- end
134
-
135
- step :heavy_work do
136
- argument :status, result(:webhook)
137
- run { |args| ... } # runs in a worker, never in the process that delivered the payload
138
- end
139
- end
140
- ```
141
-
142
- - **Signature**: `self.interrupt(name, resume: :inline, &block)` — `:inline` (today's behavior, the default) or `:background`. Any other value raises `RubyReactor::Error::ValidationError` at class-definition time.
143
- - **Why it exists**: `background after:/before:` is edge-triggered on reaching one named step, but an interrupt re-enters the reactor from a foreground process (`continue` runs in whatever process delivered the payload). `resume: :background` makes that re-entry itself the hand-off: the delivering process validates and stores the payload, then enqueues the remainder.
144
- - **Runtime contract**:
145
- - Payload validation (`validate_payload`), attempt counting, and `max_attempts` exhaustion (undo + failed) all run **synchronously in the calling process**, before anything is enqueued — validate in the caller, execute in the worker, the same split as `background before: <first step>` and FR-016. An invalid payload enqueues nothing and leaves the reactor paused.
146
- - On a valid payload: result stored, context persisted (job payload is identity-only), `before_async_enqueue` middleware fires, remainder enqueued via `configuration.async_router`, and `continue` returns an `DispatchResult` (with `execution_id`) instead of the final result.
147
- - The worker resumes through the existing path: rehydrate by id, `resume_execution`, interrupt's stored result found, remaining steps run under `inline_async_execution` (so a `background` hand-off point never re-triggers there).
148
-
149
- ## Unchanged (explicitly out of scope, called out to prevent accidental regression)
150
-
151
- - Reactor-level `async true` ("Full Reactor Async") — `self.class.async?`, `lib/ruby_reactor/dsl/reactor.rb:44-50`. (Its only new interaction: combining it with `background after:` is a definition-time error, see above.)
152
- - `compose` itself — synchronous, fully compensation-linked nested execution, untouched. (Its `async` flag is removed — see the Removed section — but everything else about `compose` is unchanged.)
153
- - `map`'s dispatch/collection machinery and its full DSL including the map-internal `async` element-dispatch option — reused as an architectural pattern (see research.md) but untouched.
154
- - `result(:name)` for a **synchronous** step's result — resolves exactly as it does today (`Template::Result#resolve`), with zero added latency; the new notified-wait path only activates for `async_step`/`async_reactor` references.
@@ -1,117 +0,0 @@
1
- # Data Model: Background Execution & Real Async Steps
2
-
3
- This is a library feature — "entities" are DSL/runtime constructs and the storage records backing them, not application data.
4
-
5
- ## Background Hand-off Point
6
-
7
- Reactor-class-level declaration, one per reactor.
8
-
9
- | Field | Type | Notes |
10
- |---|---|---|
11
- | `mode` | Symbol — `:after` or `:before` | Which side of the cut point the declaration named. Derived from which keyword the author supplied. |
12
- | `step` | Symbol | The named step. For `:after`, the last step to run in the calling process; for `:before`, the first step to run in the worker. Must reference a step defined in the same reactor (validated at class-definition time, FR-002). |
13
-
14
- Exposed to the runtime, `TestSubject`, and the dashboard as a single normalized reader — `background_handoff → { mode:, step: }` — never as a one-sided `background_after`. One concept with two trigger positions, not two parallel features: every consumer branches on `mode`, so no consumer can be accidentally implemented for `after:` only.
15
-
16
- **Storage**: not persisted as its own record — it compiles into which step reaching which position triggers the `StepExecutor#handle_async_step`-style enqueue. Enforced-single via a class-level guard.
17
-
18
- **Validation rules** (all definition-time errors):
19
- - Exactly zero or one `background` declaration per reactor class (FR-002).
20
- - Exactly one of `after:`/`before:` per declaration — both or neither raises (FR-002).
21
- - The named step must exist in `steps` at the time `background` is evaluated, or at class-definition-close time if steps can be declared afterward (implementation detail for tasks phase).
22
- - `background` combined with whole-reactor `async true` is rejected — the hand-off point would be silently meaningless inside a reactor that already runs entirely in a worker (spec Edge Cases).
23
- - `returns` naming an `async_step` or `async_reactor` is rejected — the return value must come from a same-process step (spec Edge Cases).
24
-
25
- ## Async Step
26
-
27
- A step declared with `async_step :name` (or `async_step :name, ImplClass`) instead of `step`.
28
-
29
- | Field | Type | Notes |
30
- |---|---|---|
31
- | `name` | Symbol | Step name, same namespace as regular steps. |
32
- | `arguments` / `run_block` / `impl` | (existing `StepConfig` fields) | Same shape as a regular step — `async_step` is a `StepConfig` with a dispatch-mode marker, not a new config type. |
33
- | dispatch-mode marker | Boolean/Symbol | Distinguishes "run inline" vs "dispatch as an independent unit" at `StepExecutor#execute_step` time. |
34
-
35
- **Lifecycle** (state machine, tracked via the new Step Result Record below, keyed by `(context_id, step_name)`):
36
-
37
- ```
38
- dispatched -> running -> completed(Success)
39
- -> completed(Failure)
40
- ```
41
-
42
- - `dispatched`: parent process has, synchronously and in this order (durable-write-before-enqueue, F2): (1) written the Step Result Record with status `dispatched` and the `composed_contexts[step_name] = { type: :async_step_ref, name:, dispatched_at: }` reference onto its own context (see Async Step ↔ context linkage below), (2) enqueued the `StepWorker` job, (3) marked the step graph-complete for scheduling purposes (siblings may now proceed). No result exists yet.
43
- - `running`/`completed`: opaque to the parent process except through the Step Result Record; the parent only observes "record carries a terminal value" or "record still `dispatched`" (still-pending — keep waiting, subject to FR-005's timeout). A record *absent* entirely means the step was never dispatched — `result()` does not wait in that case. The parent may reach its own terminal state while a record is still `dispatched`; that is the fire-and-forget contract (FR-018), and the parent's status makes no claim about the unit's outcome.
44
-
45
- **Relationships**: An `async_step` is a dependency-graph node like any other step — other steps that declare `argument :x, result(:async_step_name)` get an automatic DAG edge (existing `DependencyGraph#add_step` behavior, unchanged) and, per FR-005, enter the notified wait for the terminal record when they resolve that argument.
46
-
47
- **Async Step ↔ context linkage (FR-008, FR-014)**: the *reference* (not the result) lives in `context.composed_contexts[step_name]`, the same field `compose`/`map` already populate for their own children — see research.md decision 8. The dashboard's existing `hydrate_composed_contexts` pipeline (`lib/ruby_reactor/web/api.rb`) is extended with a branch for `type: :async_step_ref` that resolves the Step Result Record to show current status/result, mirroring how it already resolves `:map_ref`.
48
-
49
- ## Async Reactor
50
-
51
- A step declared with `async_reactor :name, ChildReactorClass`.
52
-
53
- | Field | Type | Notes |
54
- |---|---|---|
55
- | `name` | Symbol | Step name in the parent. |
56
- | `child_reactor_class` | Class | Must be a `RubyReactor::Reactor` subclass. |
57
- | `argument_mappings` | Hash | Same shape as `compose`'s `argument_mappings` — maps parent-visible sources to the child's inputs. |
58
-
59
- **Dispatch-time behavior** (FR-015, FR-016 — 2026-08-20 session):
60
- - Dispatch reuses the full pre-enqueue sequence of a top-level async run (child input validation → ordered-lock nonce assignment where the child declares one → persist child context → enqueue), never raw `perform_async` — see research.md decision 3. A validation failure fails the dispatching step itself (normal parent saga handling), distinct from child-execution failure (FR-009).
61
- - Deadlock guard: the child's `lock_config[:key_proc]` (and `semaphore_config` with `limit: 1`) is resolved against the mapped child inputs; a key matching one the dispatching execution currently holds fails the dispatch step immediately with an error naming the key and both reactor classes (research.md decision 9). No lock-owner sharing across the async boundary — reentrancy stays `compose`-only.
62
-
63
- **Relationships**:
64
- - Linked to the parent via `context.composed_contexts[step_name] = { type: :async_reactor_ref, name:, execution_id:, reactor_class_name:, dispatched_at: }` — written synchronously by the dispatching step, same field and pattern `compose`/`map` already use (research.md decision 8). Unlike `async_step`, no separate Step Result Record is needed for the *outcome*: the child is a normal, independently addressable `Reactor` with its own context row, so its terminal result is reached via the existing `storage.retrieve_context(execution_id, reactor_class_name)` / `ChildReactorClass.find(execution_id)` — the same lookup any other reactor execution uses.
65
- - **Not** added to the parent's compensation graph — no `compensate`/`undo` block is registered for this step (see spec Clarifications: compensation is opt-in via a later step reading the result, never automatic).
66
- - FR-008/FR-014: this `composed_contexts` entry is what the web dashboard's `hydrate_composed_contexts` reads to render the reference and drill into the child's own step structure (`build_structure` recursion, same as `compose`/`map`'s `nested_structure`).
67
-
68
- ## Step Result Record (new storage-level entity)
69
-
70
- The durable record backing `async_step` completion. (`async_reactor` needs no equivalent record — per the relationship above, its outcome is simply its own context row, reached by execution id through the existing `retrieve_context`/`find` path.) This avoids what would otherwise be a race-prone write into the parent's context blob from a worker running concurrently with the still-executing parent process.
71
-
72
- | Field | Type | Notes |
73
- |---|---|---|
74
- | `context_id` | String (UUID) | The **parent** reactor's context id — the bucket is scoped per parent execution. |
75
- | `step_name` | Symbol/String | The `async_step`'s name within that parent. |
76
- | `status` | Enum: `dispatched`, `completed` | `dispatched` written synchronously **before** the job is enqueued (same checkpoint-before-enqueue ordering the existing hand-off uses, F2) — so a crash after enqueue can never find a job with no record, and this record doubles as the **re-attach marker** (FR-017): on recovery/resume the dispatch path finds a record in any status and skips enqueue entirely, marking the node graph-complete as the original dispatch did, rather than duplicating the side effect. The `async_reactor` equivalent is the `:async_reactor_ref` entry in `composed_contexts` (research.md decision 10). `completed` written by the step's own worker. |
77
- | `serialized_result` | String (via `ContextSerializer.serialize_value`) | The step's `Success`/`Failure` value, same serialization the existing map-result bucket uses. |
78
- | `reactor_class_name` | String | Needed for storage-key namespacing, mirrors every other storage primitive's `reactor_class_name` parameter. |
79
-
80
- **Storage interface additions** (`RubyReactor::Storage::Adapter`, implemented by `RedisAdapter`):
81
-
82
- ```ruby
83
- store_step_result(context_id, step_name, serialized_result, reactor_class_name)
84
- retrieve_step_result(context_id, step_name, reactor_class_name)
85
- ```
86
-
87
- Modeled directly on the existing `store_map_result(map_id, index, serialized_result, reactor_class_name, strict_ordering:)` / `retrieve_map_results(...)` pair (`lib/ruby_reactor/storage/adapter.rb:14-20`) — same TTL policy as `context_ttl` (records must not outlive the parent context's own retention window).
88
-
89
- **Retention across a fire-and-forget parent (FR-018)**: the worker loads the *parent* context by id, so the parent must outlive the dispatched unit — including the common case where the parent completes immediately and nothing ever waits on the unit. Dispatch therefore refreshes the parent context's TTL, and the record is stamped with the same window. A worker that still finds no parent context (swept, or beyond the window) writes a `completed`/`Failure` record for its unit and logs it per FR-012 rather than raising — an unhandled raise would only hand the job to the backend's retry machinery to fail identically N more times. See research.md decision 10.
90
-
91
- ## Completion Signal (new, ephemeral — not a stored entity)
92
-
93
- The wake-up channel for FR-005's notified wait (research.md decision 4). Pure latency optimization: at-most-once, unpersisted, never load-bearing — every path falls back to the durable record above (or the child's context row).
94
-
95
- | Channel | Published by | When |
96
- |---|---|---|
97
- | `rr:done:<parent_context_id>:<step_name>` | the `async_step`'s StepWorker | after `store_step_result` write |
98
- | `rr:done:<child_execution_id>` | the `async_reactor` child's executor | after its terminal context save (unconditional — no-subscriber publish is near-free) |
99
-
100
- Uses the existing, currently-unused `Storage::Adapter#publish`/`#subscribe` primitives (`adapter.rb:38-44`, implemented at `redis_adapter.rb:177-183`). Ordering contract: durable write **before** publish; waiter subscribes **before** its first record check; waiter re-checks the record on a coarse fallback interval. The subscriber MUST use a dedicated Redis connection (`SUBSCRIBE` puts a connection into subscriber mode — blocking the shared client would poison all other storage calls).
101
-
102
- ## Configuration additions
103
-
104
- | Knob | Default | Notes |
105
- |---|---|---|
106
- | `Configuration#async_wait_timeout` | `30` (seconds) | Seconds a `result()` notified wait will block before failing the referencing step with a timeout. Single global value — no per-reactor/per-reference override (Clarifications, Question 3). Rationale for 30s in research.md decision 5. |
107
-
108
- **Derived (not configurable)**: the notified wait's fallback re-check interval is `async_wait_timeout / 10`, clamped to `1..5` seconds (3s at the default). It is a latency backstop for a lost signal, not a tuning surface — the clamp guarantees ≥10 re-checks inside any bound, so a dropped notification costs at most ~10% of the timeout. See research.md decision 4.
109
-
110
- ## State/behavior changes to existing entities
111
-
112
- - **`StepConfig`** (`lib/ruby_reactor/dsl/step_builder.rb`): the `async`/`async?` accessor is removed; using `async true` inside a `step` block raises a definition-time error naming the replacement DSL (FR-003). `ComposeBuilder#async` (`dsl/compose_builder.rb:31-33`) is **removed too** — it sets the very `StepConfig` `async:` flag being deleted (`compose_builder.rb:62`), so it cannot survive the removal; it raises the same definition-time error (migration: `background before: :<that compose step>`, which reproduces the old flag's semantics exactly). `MapBuilder#async` (`dsl/map_builder.rb:43`) is genuinely unaffected: it is a map-internal element-dispatch mode passed as a step *argument*, and the map's own `StepConfig` is hardcoded `async: false` (`map_builder.rb:111`) — it never touched the removed flag. (An earlier pass of this document had the compose/map carve-outs backwards; corrected after verifying both builders.)
113
- - **`DependencyGraph`**: no schema change; `complete_step` is now called for an `async_step` at dispatch time rather than at true completion — a deliberate, documented divergence from every other step type, captured here so it isn't mistaken for a bug during implementation review.
114
- - **Reactor-class DSL** (`lib/ruby_reactor/dsl/reactor.rb`): three new class macros — `background(after: nil, before: nil)`, `async_step(name, impl = nil, &block)`, `async_reactor(name, child_reactor_class, &block)` — alongside the existing `step`, `compose`, `map`, `interrupt`. The existing reactor-level `async`/`async?` (whole-reactor async) is unchanged.
115
- - **`Context#composed_contexts`**: gains two new `type:` tags in its value union — `:async_step_ref` and `:async_reactor_ref` — alongside the existing `:composed` and `:map_ref`. No schema/serialization change (it's already a generic `Hash`, already serialized/deserialized as-is); this is purely a new convention for the `type:` field's allowed values, consumed by `Web::API.hydrate_composed_contexts` and by the new `TestSubject#async_step`/`#async_reactor` traversal helpers (mirroring `#composed`/`#map`).
116
- - **`Web::API`** (`lib/ruby_reactor/web/api.rb`): `determine_step_type` gains `async_step`/`async_reactor` branches (replacing the removed `config.async?` branch); `build_structure` drops the per-step `async:` field and instead exposes the reactor's single hand-off point once, as the normalized `{ mode:, step: }` pair (so the dashboard can mark the cut regardless of which side declared it); `hydrate_composed_contexts` gains resolution branches for the two new ref types, mirroring `hydrate_map_ref`. See research.md decision 8.
117
- - **GUI** (`gui/src/components/DagVisualizer.tsx`, `StepInspector.tsx`): need new rendering cases for the `'async_step'`/`'async_reactor'` step types surfaced by the API above — a task-phase implementation item, called out here so it isn't missed (constitution Principle IV: dashboard must stay current with the reactor state model).
@@ -1,168 +0,0 @@
1
- # Implementation Plan: Background Execution & Real Async Steps
2
-
3
- **Branch**: `001-background-async-steps` | **Date**: 2026-08-16 | **Spec**: [spec.md](./spec.md)
4
-
5
- **Input**: Feature specification from `/specs/001-background-async-steps/spec.md`
6
-
7
- ## Summary
8
-
9
- Replace the confusing per-step `async: true` flag (only the first flagged step in a reactor actually takes effect — the rest are silently ignored) with a single, unambiguous reactor-level `background` declaration that hands off all remaining steps to an independent worker job. The cut point is nameable from either side — `background after: :second` (that step is the last to run in the calling process) or `background before: :third` (that step is the first to run in the worker) — which coincide in a linear chain but pin different steps in a DAG. Add two genuinely new capabilities on top of that: `async_step`, whose unit of work is dispatched to run in its own independent worker while the rest of the reactor keeps executing in the calling process; and `async_reactor`, which dispatches a whole nested reactor to run independently, linked to the parent by execution id but excluded from the parent's compensation graph. Both `async_step` and `async_reactor` results are consumed via the existing `result(:name)` template helper, which gains a notified wait (completion signal via the storage adapter's pub/sub, durable-record-first, bounded fallback re-check — see research.md decision 4) when the referenced work hasn't finished yet. Per the clarified spec, neither `async_step` nor `async_reactor` failures auto-compensate the parent — compensation only happens if a later step explicitly reads the result and decides to fail. Lock ownership is never shared across the async boundary; a same-key collision between parent and child fails at dispatch (FR-015), and dispatch reuses the full pre-enqueue sequence including child input validation (FR-016). All dispatch continues to go through the existing pluggable `configuration.async_router` (Sidekiq or ActiveJob), unchanged.
10
-
11
- ## Technical Context
12
-
13
- **Language/Version**: Ruby >= 3.0.0 (per constitution's Technical Constraints)
14
-
15
- **Primary Dependencies**: `sidekiq` and/or `activejob` (pluggable via `RubyReactor::Configuration#async_router`, already introduced by the ActiveJob Support feature on `main`), `redis` (storage), `dry-validation` (input validation DSL, unaffected by this feature)
16
-
17
- **Storage**: Redis via `RubyReactor::Storage::RedisAdapter` (implements `RubyReactor::Storage::Adapter`). This feature adds one new storage primitive pair for per-step async results (see data-model.md) modeled directly on the existing `store_map_result` / `retrieve_map_results` pair used by `map`.
18
-
19
- **Testing**: RSpec (mandatory per constitution), in two lanes (research.md decision 6):
20
-
21
- - *Unit lane* — the existing `RubyReactor::RSpec::TestSubject` DSL (`test_reactor`, `have_run_step`, `drain_async_jobs`/`process_pending_jobs` via `AsyncTestHelpers`) over the in-memory queue fakes, for everything that does not involve a caller blocked on `result()`: definition-time guards, dispatch shape, `composed_contexts` refs, read semantics against an already-terminal record.
22
- - *Orchestration lane* — a **real worker consuming a real queue**, required by constitution Principle III for every async orchestration path (a caller blocked in the notified wait while an independent worker completes the work; the interleaving is not reproducible under an in-process fake, which only runs jobs when the spec itself calls `drain_async_jobs` — and the spec is blocked). Sidekiq backend: a live `sidekiq` process booted against the test Redis. ActiveJob backend: the `:async` adapter (a real queue runner, not the `:test` fake).
23
-
24
- No parallel test *framework* — per explicit instruction, only the gem's built-in spec helpers are used; the orchestration lane is one new shared context (`spec/support/real_async_backend.rb`) plus new helper methods on the existing `AsyncTestHelpers`/`TestSubject` modules, and the `TestSubject` interceptor logic is updated where it hardcodes the old per-step `async?` flag (`prepare_execution_class`/`apply_mock_interceptor`).
25
-
26
- **Target Platform**: Server-side Ruby (gem consumed by Rails/Sinatra-style host apps); demo validated against `demo_app/` (Rails app already in the repo)
27
-
28
- **Project Type**: Library (Ruby gem) with a bundled `demo_app/` integration example — matches the constitution's Gem-First Design principle
29
-
30
- **Performance Goals**: Not a throughput-sensitive change — dispatch overhead should stay within the same order of magnitude as the existing `map` per-element dispatch path it reuses patterns from. No new SLO introduced.
31
-
32
- **Constraints**: Blocking waits on `result()` (FR-005) are notified waits — completion signal via the storage adapter's existing (currently unused) `publish`/`subscribe` primitives, durable-record-first ordering, subscribe-before-check on the waiter, coarse fallback re-check, all bounded by a single library-wide `Configuration#async_wait_timeout` (new knob; see research.md decision 4 and data-model.md "Completion Signal"); the subscriber uses a dedicated Redis connection. Never an unbounded wait. Lock ownership is never shared across the async boundary (FR-015 dispatch-time deadlock guard; reentrancy stays `compose`-only). Must not change behavior of the untouched reactor-level `async` flag (`self.class.async?`, "Full Reactor Async"), of `compose`'s execution/compensation semantics (its `async` flag is removed with the shared `StepConfig` flag, per FR-003 — see contracts/public-dsl.md), or of `map` (including its map-internal `async` element-dispatch option, which is a different mechanism and stays).
33
-
34
- **Scale/Scope**: Single-gem change; touches DSL (`dsl/reactor.rb`, `dsl/step_builder.rb`, `dsl/compose_builder.rb`), executor (`executor/step_executor.rb` for dispatch, `executor.rb` for the completion-signal publish on terminal save; `dependency_graph.rb` itself unchanged), a new `async_reactor` dispatch step (`step/async_reactor_step.rb`), a framework-agnostic async-step worker body (`step_worker.rb`) plus its two adapter bindings and a `perform_step_async` entry point on both routers (mirroring `Adapters::{Sidekiq,ActiveJob}::MapElementWorker`), the new `async_waiter.rb` shared wait core, two new error classes, `context.rb` (documented `composed_contexts` ref types), `template/result.rb`, `storage/adapter.rb` + `storage/redis_adapter.rb`, `configuration.rb`, RSpec helpers plus the new real-worker shared context in `spec/support/`, the bundled web dashboard (`lib/ruby_reactor/web/api.rb` + `gui/src/components/{DagVisualizer,StepInspector}.tsx`, per FR-014), `demo_app/`, `documentation/`, `README.md`.
35
-
36
- ## Constitution Check
37
-
38
- *GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
39
-
40
- - **I. Gem-First Design**: PASS. New DSL (`background`, `async_step`, `async_reactor`) ships in `lib/ruby_reactor/dsl/`; the new worker class ships in both `lib/ruby_reactor/adapters/sidekiq/` and `lib/ruby_reactor/adapters/active_job/`, mirroring the existing adapter-isolation pattern — no host-application coupling, Sidekiq/ActiveJob stay optional per the existing pluggable-router mechanism.
41
- - **II. Saga Pattern Integrity (NON-NEGOTIABLE)**: PASS, with a deliberate, spec-clarified narrowing. `background` hand-off (either form) preserves full compensation — it is the same underlying mechanism as today's step-level hand-off, just triggered once, unambiguously, and nameable from either side. `async_step`/`async_reactor` do NOT auto-compensate on failure — this was explicitly clarified with the user (see spec Clarifications) as an intentional escape hatch mirroring the existing `compose`/whole-reactor-async model, not a silent gap: a later step that reads the result via `result(:name)` can always trigger compensation itself, so no failure is ever unrecoverable, it is opt-in rather than automatic. This mirrors the already-shipped behavior of `async_reactor`-adjacent whole-reactor `async true` composition today, so it is not a new category of principle exception.
42
- - **III. Test-First with Real Infrastructure**: PASS, after correction. Tests are written first and run against real Redis. An earlier pass of this plan claimed PASS while resting the *entire* async-orchestration test strategy on `Sidekiq::Testing.fake!` + in-process `drain_async_jobs` and the ActiveJob `:test` adapter — those are mocked Sidekiq/queue state, which Principle III forbids on async orchestration paths, and they also cannot reproduce the behavior under test (a caller blocked in the notified wait cannot call `drain_async_jobs`, so the job would never run and every awaited-result spec would time out rather than pass). Corrected: every spec exercising a blocked caller plus a concurrently-completing worker runs in the orchestration lane — a live `sidekiq` process against the test Redis, and the ActiveJob `:async` adapter — via the shared context in `spec/support/real_async_backend.rb`. The in-memory fakes are retained strictly for unit-level assertions with no blocked caller (definition-time guards, dispatch shape, read semantics against an already-terminal record), which is exactly the carve-out Principle III allows.
43
- - **IV. Observability by Default**: PASS, and specifically checked against "The web dashboard MUST remain current with the reactor state model": FR-012 requires machine-parseable (key=value/JSON) structured log entries for hand-off/dispatch/completion carrying reactor name, step name, and execution id — plus, on any failure entry, the failure reason and redacted inputs, which Principle IV mandates and an earlier pass of FR-012 omitted; this matters more here than elsewhere because the fire-and-forget model (FR-009/FR-011) can leave a worker-side failure with no other surface. Implemented following the existing `middlewares.on(:before_async_enqueue, ...)` pattern; FR-008/FR-014 additionally require the async link to be recorded on the parent's own context (reusing `composed_contexts`, the same field `compose`/`map` already use — research.md decision 8) and rendered/drillable in `Web::API` + the `gui/` dashboard, not just logged. An earlier pass of this plan only logged the link and missed the dashboard requirement; corrected after review.
44
- - **V. Simplicity and Semantic Versioning**: PASS. This is a MAJOR (breaking) change to the public step DSL (removal of per-step `async`), documented per FR-013. The design deliberately reuses three existing mechanisms (step-level hand-off in `StepExecutor#handle_async_step`, `map`'s per-unit dispatch-and-collect pattern, and the existing `DispatchResult`/`result()` template mechanism) rather than inventing a fourth. No speculative per-reactor/per-reference timeout override is introduced (resolved via clarification) — a single global config value only, added when a second real use case exists.
45
-
46
- No violations requiring the Complexity Tracking table.
47
-
48
- ## Project Structure
49
-
50
- ### Documentation (this feature)
51
-
52
- ```text
53
- specs/001-background-async-steps/
54
- ├── plan.md # This file (/speckit-plan command output)
55
- ├── research.md # Phase 0 output (/speckit-plan command)
56
- ├── data-model.md # Phase 1 output (/speckit-plan command)
57
- ├── quickstart.md # Phase 1 output (/speckit-plan command)
58
- ├── contracts/ # Phase 1 output (/speckit-plan command)
59
- │ └── public-dsl.md
60
- └── tasks.md # Phase 2 output (/speckit-tasks command - NOT created by /speckit-plan)
61
- ```
62
-
63
- ### Source Code (repository root)
64
-
65
- ```text
66
- lib/ruby_reactor/
67
- ├── dsl/
68
- │ ├── reactor.rb # ADD: `background(after:/before:)`, `async_step`, `async_reactor`
69
- │ │ # class macros + definition-time guards (dup background,
70
- │ │ # unknown step, both-or-neither key, background×async-true,
71
- │ │ # returns×async-unit); whole-reactor `async` stays
72
- │ ├── step_builder.rb # REMOVE: `async` step-level flag (raise on use, FR-003)
73
- │ └── compose_builder.rb # REMOVE: `async` flag too — it sets the same StepConfig flag
74
- │ # (raise on use, FR-003); map_builder.rb untouched
75
- ├── executor/
76
- │ └── step_executor.rb # REPLACE: `handle_async_step` keyed off the reactor's single
77
- │ # hand-off point (post-execution check for `after:`,
78
- │ # pre-execution check for `before:`), not per-step `async?`;
79
- │ # ADD: async_step dispatch (fire-and-continue, marks the
80
- │ # node complete in the DependencyGraph at dispatch time so
81
- │ # siblings aren't blocked) and async_reactor dispatch;
82
- │ # both re-attach instead of re-dispatching when a durable
83
- │ # dispatch marker already exists (FR-017), and extend the
84
- │ # parent context's retention window at dispatch (FR-018)
85
- ├── executor.rb # ADD: publish completion signal after terminal context save
86
- │ # (async_reactor completer side, research.md decision 4)
87
- ├── async_waiter.rb # NEW: shared FR-005 wait core — subscribe-first, durable check,
88
- │ # signal wake + coarse fallback re-check, timeout bound
89
- ├── step_worker.rb # NEW: framework-agnostic async_step worker body (load parent
90
- │ # context, resolve args, run, store record, publish signal;
91
- │ # missing-parent-context handling per FR-018)
92
- ├── context.rb # UPDATE: document the two new `composed_contexts` ref types
93
- │ # (`:async_step_ref`, `:async_reactor_ref`) — no
94
- │ # serialization change
95
- ├── dependency_graph.rb # UNCHANGED (no schema change; dispatch-time complete_step is
96
- │ # called by the executor, see data-model.md)
97
- ├── error/
98
- │ ├── async_wait_timeout_error.rb # NEW: raised when an FR-005 wait exceeds its bound
99
- │ └── deprecated_dsl_error.rb # NEW: definition-time error for the removed `async` flag (FR-003)
100
- ├── step/
101
- │ └── async_reactor_step.rb # NEW: async_reactor dispatch step — full pre-enqueue sequence
102
- │ # (FR-016) + FR-015 deadlock guard, no compensate/undo
103
- ├── adapters/
104
- │ ├── sidekiq/step_worker.rb # NEW: independent one-shot worker for a single async_step
105
- │ ├── sidekiq/router.rb # ADD: `perform_step_async` dispatch entry point
106
- │ ├── active_job/step_worker.rb # NEW: ActiveJob counterpart
107
- │ └── active_job/router.rb # ADD: `perform_step_async` dispatch entry point
108
- ├── template/
109
- │ └── result.rb # UPDATE: notified wait (subscribe-first, record check, fallback
110
- │ # re-check, timeout) when the referenced async result
111
- │ # is not yet available (FR-005, research.md decision 4)
112
- ├── storage/
113
- │ ├── adapter.rb # ADD: `store_step_result` / `retrieve_step_result` interface methods
114
- │ │ # (async_step outcome only — async_reactor reuses the existing
115
- │ │ # retrieve_context/find path, no new primitive needed there)
116
- │ └── redis_adapter.rb # implement them (mirrors store_map_result/retrieve_map_results);
117
- │ # ADD dedicated-connection subscribe (SUBSCRIBE must not block
118
- │ # the shared client)
119
- ├── configuration.rb # ADD: `async_wait_timeout` config knob (default 30s)
120
- ├── web/
121
- │ └── api.rb # UPDATE: `determine_step_type`/`build_structure` (drop removed
122
- │ # `config.async?` field, expose the normalized
123
- │ # `background_handoff { mode:, step: }` pair once per
124
- │ # reactor, add async_step/async_reactor branches) and
125
- │ # `hydrate_composed_contexts` (resolve the two new
126
- │ # composed_contexts ref types) — FR-008, FR-014
127
- └── rspec/
128
- └── test_subject.rb # UPDATE: `prepare_execution_class`/interceptors to understand the
129
- # normalized `background_handoff`/`async_step`/
130
- # `async_reactor` instead of the removed per-step `async?`;
131
- # ADD `#async_step`/`#async_reactor` traversal helpers
132
- # mirroring `#composed`/`#map`
133
-
134
- gui/src/components/
135
- ├── DagVisualizer.tsx # UPDATE: render 'async_step'/'async_reactor' step types (FR-014)
136
- └── StepInspector.tsx # UPDATE: same
137
-
138
- spec/ruby_reactor/ # gem's own unit/integration specs (mirrors lib/ layout above)
139
- spec/support/reactors/ # new fixture reactors for background/async_step/async_reactor
140
- spec/support/async_backends.rb # NEW: shared context parameterizing a group over both backends
141
- spec/support/real_async_backend.rb # NEW: orchestration lane — live `sidekiq` process against the
142
- # test Redis / ActiveJob `:async` adapter (Principle III)
143
-
144
- demo_app/app/reactors/
145
- ├── full_async_reactor.rb # UNCHANGED (whole-reactor async, out of scope)
146
- ├── partial_async_reactor.rb # REPLACED by `background_demo_reactor.rb`, a `background after:`
147
- │ # example (old per-step `async true` syntax is removed, FR-003);
148
- │ # migrated in Phase 2 with the rest of the call sites, since the
149
- │ # old syntax stops parsing the moment the flag is removed
150
- ├── async_step_demo_reactor.rb # NEW: demonstrates async_step + result() wait (send_email example)
151
- └── async_reactor_demo_reactor.rb # NEW: demonstrates async_reactor fire-and-forget vs. awaited
152
- demo_app/spec/reactors/ # matching specs, using the same built-in TestSubject helpers
153
-
154
- documentation/
155
- ├── async_reactors.md # REWRITE "Step-Level Async" to `background after:`/`before:`;
156
- │ # ADD sections for `async_step` and `async_reactor`
157
- └── composition.md # cross-reference `async_reactor` vs. `compose`
158
- demo_app/documentation/ # kept in sync with the same edits (duplicate copy, see research.md)
159
-
160
- README.md # rewrite "Step-Level Async" subsection, add async_step/async_reactor
161
- CHANGELOG.md # breaking-change entry (FR-013)
162
- ```
163
-
164
- **Structure Decision**: Single-project Ruby gem layout (existing `lib/ruby_reactor/**`, `spec/**`, plus the bundled `demo_app/` Rails integration example and `documentation/**`). No new top-level directories — every change lands inside the existing module boundaries (`dsl/`, `executor/`, `adapters/`, `template/`, `storage/`, `rspec/`), consistent with Principle I (Gem-First Design).
165
-
166
- ## Complexity Tracking
167
-
168
- *No Constitution Check violations — table not needed.*
@@ -1,102 +0,0 @@
1
- # Quickstart: Validating Background Execution & Real Async Steps
2
-
3
- Prerequisites: repo checked out on this branch, `bundle install` run (repo root and `demo_app/`), Redis running (`docker-compose up -d` at repo root; the gem's suite reads `RUBY_REACTOR_TEST_REDIS_URL`, defaulting to `redis://localhost:6780` — see `spec/spec_helper.rb:13`). Real Redis, not mocked, per the constitution's storage requirement.
4
-
5
- ## 1. Gem-level unit/integration specs
6
-
7
- ```bash
8
- bundle exec rspec spec/ruby_reactor/dsl/reactor_background_spec.rb # background after:/before:
9
- bundle exec rspec spec/ruby_reactor/dsl/async_step_spec.rb # async_step
10
- bundle exec rspec spec/ruby_reactor/dsl/async_reactor_spec.rb # async_reactor
11
- bundle exec rspec spec/ruby_reactor/rspec/test_subject_async_spec.rb # TestSubject support for the new DSL
12
- ```
13
-
14
- (Exact spec file names are placeholders for the tasks phase — see contracts/public-dsl.md for the acceptance scenarios each must cover, taken directly from spec.md's User Stories 1–3.)
15
-
16
- Expected outcomes, mapped to spec.md acceptance scenarios:
17
-
18
- - A reactor with `background after: :second` runs `:first`/`:second` inline and `:third` via a dispatched job; `MyReactor.run` returns an `DispatchResult`; `TestSubject`'s `drain_async_jobs` completes it (US1 scenario 1, SC-001).
19
- - The same reactor declared `background before: :third` behaves identically in this linear case, and `:third` never executes in the calling process (US1 scenario 2). In a branching fixture the two forms pin different steps — assert the specific step each one guarantees.
20
- - A `background` declaration that is duplicated, names an unknown step, carries both `after:` and `before:`, carries neither, or sits on a whole-reactor-`async` reactor each raise at class-definition time (US1 scenarios 4-5, FR-002).
21
- - A reactor still using `step { async true }` raises a `ValidationError` at class-definition time, not at run time (US1 scenario 3, SC-004).
22
- - An `async_step` reactor: a sibling step with no dependency on it completes without waiting; a step declaring `argument :x, result(:async_step_name)` receives the correct value once available (US2, SC-002).
23
- - An `async_step` failure with no downstream reader does NOT flip the parent to `failed`/trigger compensation; a downstream reader that inspects the failure and returns `Failure` DOES trigger compensation (US2 acceptance scenario 3, Clarifications).
24
- - An `async_reactor` with no reader: forcing the child to fail does not compensate the parent. An `async_reactor` with a reader: the reader's `run` block sees the child's real `Success`/`Failure` and can choose to propagate (US3, SC-003).
25
- - A `result()` reference to a never-completing async unit fails with a timeout error, not an indefinite hang (SC-005) — verify by pointing `Configuration#async_wait_timeout` at a short value in the spec and never draining the corresponding job.
26
- - The notified wait is race-free (FR-005, Session 2026-08-20 clarification): a completion that lands *before* the waiter subscribes is still found (subscribe-then-check), and a dropped signal is caught by the fallback re-check — verify with a spec that completes the async unit before the reader step runs, and one that publishes nothing and relies on the record alone.
27
- - An `async_reactor` whose child declares the same `lock` key the parent holds fails at dispatch with an error naming the key and both reactors (FR-015) — not a wait-then-timeout.
28
- - Dispatching an `async_reactor` with invalid child inputs fails the dispatching step in the parent (FR-016), while a child that fails *during execution* still never auto-compensates the parent (FR-009).
29
-
30
- ## 2. Sidekiq AND ActiveJob backends both pass
31
-
32
- Per Assumptions (spec.md) and Technical Context (plan.md), this feature must not hardcode a backend. Run the relevant spec files twice — once with the default (`Sidekiq::Testing.fake!`) and once with `config.async_router = RubyReactor::Adapters::ActiveJob::Router` + `ActiveJob::Base.queue_adapter = :test` — however the existing test suite's backend-parameterization convention already does this (check `spec/support/` for a shared-example/shared-context wrapping both backends before inventing a new one).
33
-
34
- ## 3. `demo_app` end-to-end
35
-
36
- ```bash
37
- cd demo_app
38
- bin/rails db:test:prepare # if needed
39
- bundle exec rspec spec/reactors/background_demo_reactor_spec.rb
40
- bundle exec rspec spec/reactors/async_step_demo_reactor_spec.rb
41
- bundle exec rspec spec/reactors/async_reactor_demo_reactor_spec.rb
42
- ```
43
-
44
- These exercise the new example reactors (`app/reactors/*.rb`, replacing `partial_async_reactor.rb`'s old syntax) through the full Rails/Sidekiq(or ActiveJob) stack the demo app wires up, giving a real (not just unit-tested) confirmation that the feature works end-to-end — the closest thing this library has to a "run it in a browser" check for a non-UI gem.
45
-
46
- ## 4. Manual smoke check (optional, for a human reviewer)
47
-
48
- ```ruby
49
- # bin/console or demo_app's bin/console
50
- result = MyReactor.run(...)
51
- result.class # => RubyReactor::DispatchResult (background) or RubyReactor::Success (no background)
52
- RubyReactor::RSpec::AsyncTestHelpers.drain_async_jobs if defined?(RubyReactor::RSpec) # in a test/console context
53
- MyReactor.find(result.execution_id).result
54
- ```
55
-
56
- ## 5. Web dashboard visibility (FR-008, FR-014, SC-006)
57
-
58
- ```bash
59
- bundle exec rspec spec/ruby_reactor/web/api_spec.rb # determine_step_type / hydrate_composed_contexts additions
60
- cd gui && npm run dev # or npm test if component tests exist
61
- ```
62
-
63
- Run a reactor with an `async_step` and one with an `async_reactor` (e.g. the new `demo_app` example reactors from step 3), open the dashboard, and confirm: the step renders with a distinct `async_step`/`async_reactor` badge (not falling back to generic `step`), and for `async_reactor`, clicking through opens the linked child execution — the same drill-down `compose`/`map` already provide. This is a UI-affecting change (constitution: "For UI or frontend changes, start the dev server and use the feature in a browser before reporting the task as complete") — do not report FR-014 done from passing specs alone.
64
-
65
- ## 6. Documentation review
66
-
67
- - `documentation/async_reactors.md` and its duplicate `demo_app/documentation/async_reactors.md` render correctly and no longer show `step { async true }` as the recommended step-level pattern.
68
- - `README.md`'s "Step-Level Async" subsection reflects `background after:`/`before:`.
69
- - `CHANGELOG.md` has a breaking-change entry under the correct semantic heading (constitution Development Workflow requirement).
70
-
71
- ## Done criteria
72
-
73
- All specs above pass; `bundle exec rubocop` is clean (constitution requirement, no `--disable-pending-cops`); `demo_app`'s specs pass against both configured backends if the demo app is wired to test both (verify via its `config/` — otherwise document which single backend the demo exercises).
74
-
75
- ---
76
-
77
- ## Validation run — 2026-08-22
78
-
79
- | Step | Result |
80
- |---|---|
81
- | 1. Gem specs | **733 examples, 0 failures**, 1 pre-existing pending |
82
- | 2. Both backends | Every async spec group runs twice via `for_each_async_backend` (Sidekiq fake mode + ActiveJob `:test`); the blocked-caller groups run twice via `for_each_real_async_backend` (a **live sidekiq process** against the test Redis + the ActiveJob `:async` adapter) |
83
- | 3. `demo_app` | **94 examples, 0 failures** (13 of them the new async examples) |
84
- | 5. Dashboard — API | Driven through the real `Web::Application` Rack app for a reactor carrying all three new constructs: `async_step`/`async_reactor` step types render as themselves (not generic `step`), the dead per-step `async` field is gone, `background_handoff` is exposed once per reactor, the async_reactor node carries `nested_structure` plus a child `execution_id` that resolves to an addressable execution, and the `async_step` ref hydrates to its Step Result Record |
85
- | 5. Dashboard — GUI | `tsc -b` clean, `npm run build` succeeds, 36 component tests pass, `eslint` at its pre-existing baseline |
86
- | 5. Dashboard — browser | **Not performed.** See below. |
87
- | 6. Documentation | Both trees rewritten and kept in sync; no remaining example anywhere shows the removed `step { async true }` |
88
- | Rubocop | **216 files, no offenses** (no `--disable-pending-cops`) |
89
-
90
- ### Open item: visual browser confirmation
91
-
92
- Step 5 asks for a human to open the dashboard and confirm the new nodes render
93
- and the async_reactor drill-down opens the child execution. That was **not**
94
- done: the implementing session was non-interactive with no browser available.
95
-
96
- Everything the browser check would depend on is verified programmatically above
97
- — the payload shape the components read, the components' own tests, the
98
- typecheck, and a clean production build — but the rendered result itself has not
99
- been looked at. **A reviewer should still open the dashboard once** (`cd gui &&
100
- npm run dev` against a reactor with an `async_step` and an `async_reactor`, e.g.
101
- the new `demo_app` examples) and confirm the node styling and the child link,
102
- before considering FR-014 closed.