ruby_reactor 0.7.0 → 0.8.0

data/specs/001-background-async-steps/research.md

@@ -1,150 +0,0 @@
-# Phase 0 Research: Background Execution & Real Async Steps
-
-No `[NEEDS CLARIFICATION]` markers remained in the spec after `/speckit-clarify`. This research phase instead grounds the plan's technical decisions in the current codebase (which now includes the ActiveJob Support feature merged to `main` after the spec was first drafted) — the "unknowns" here are architectural design decisions, not missing product requirements.
-
-## 1. `background after:` / `background before:` — reuse the existing step-level hand-off mechanism
-
-**Decision**: Implement `background` by moving the trigger condition in `StepExecutor#execute_step` from "does *this* step have `async?` set" to "have we reached the reactor's single declared hand-off point." The actual hand-off body (`StepExecutor#handle_async_step`, `lib/ruby_reactor/executor/step_executor.rb:200-224`) is reused almost unchanged for both forms: persist the root context, call `configuration.async_router.perform_async`, return an `DispatchResult` that halts `execute_all_steps`.
-
-The two forms differ only in *where in `execute_step` the check sits* (2026-08-22 session):
-
-- `after: :x` — a **post-execution** check: `:x` runs normally, and the hand-off fires once its result is recorded. Existing behavior, just re-keyed off the reactor-level declaration.
-- `before: :x` — a **pre-execution** check at the top of `execute_step`: when `:x` is selected to run, hand off *instead of* running it, leaving `:x` unexecuted and its dependency-graph node incomplete so the worker runs it. This is strictly simpler than the `after:` path (nothing to record first), and it reuses the identical checkpoint-and-enqueue body.
-
-Both must sit inside the existing `inline_async_execution` guard so the hand-off never re-triggers once the worker picks the reactor up.
-
-**Rationale**: `handle_async_step` already does exactly what `background after:` needs — checkpoint, enqueue via the pluggable router, halt. The bug the user is renaming away from (`documentation/async_reactors.md:87`, `demo_app/app/reactors/partial_async_reactor.rb`) is purely in *how the trigger is decided*: today it's "the first step encountered with `async?` true," which silently produces the same one-hand-off-point behavior as `background after:` would, but expressed as a per-step flag that reads as if each flagged step independently hands off (it doesn't — `lib/ruby_reactor/executor.rb:530-561`, `execute_current_step_and_continue`, shows any step reached with `inline_async_execution == true` — i.e. every step from the second async-flagged one onward — just runs inline in the worker). Moving the flag to the reactor class level and enforcing "at most one" (FR-002) makes the existing correct mechanism impossible to misread.
-
-**Why support both sides rather than just `after:`** (2026-08-22): they are not redundant in a DAG. Each form guarantees something about the step it names — `after: :x` guarantees `:x` ran in the calling process; `before: :x` guarantees `:x` runs in the worker. In a linear chain those coincide, so `after: :second` ≡ `before: :third`; in a branching workflow the author frequently cares about exactly one of the two (e.g. "the payment charge must happen synchronously so the caller sees it fail" → `after: :charge`; "the report render must never block the request" → `before: :render_report`). Naming the cut from the wrong side in a DAG silently pins the wrong step, so offering both is what makes the intent expressible rather than approximated. Cost is one extra check position in `execute_step` and one extra definition-time guard.
-
-**Removal blast radius** (verified against both builders): `ComposeBuilder#async` (`dsl/compose_builder.rb:31-33`) sets the same `StepConfig` `async:` flag being removed (`compose_builder.rb:62`), so it is removed with it and raises the same definition-time error (migration: `background before: :<that compose step>`, which reproduces the old flag semantics exactly). `MapBuilder#async` (`dsl/map_builder.rb:43`) is untouched: it is a map-internal element-dispatch mode passed as a step *argument*, and the map's `StepConfig` is hardcoded `async: false` (`map_builder.rb:111`) — different mechanism, out of scope. Three definition-time guards accompany the new macro (spec Edge Cases): `background` + whole-reactor `async true` is rejected (the hand-off would be a silent no-op inside a reactor already running in a worker), a `background` carrying both or neither of `after:`/`before:` is rejected, and `returns` naming an async unit is rejected. Inside the worker, the hand-off never re-triggers — the existing `inline_async_execution` guard keeps that job (and that job only; see decision 2 on why async_step dispatch ignores it).
-
-**Alternatives considered**:
-- *Support only `after:`, treat `before:` as sugar the author can hand-translate* — rejected: the translation is only valid in a linear chain. In a DAG "hand off before `:x`" has no reliable `after:` spelling (there may be several predecessors, or the relevant one may vary by branch), so the author would have to over-approximate the cut.
-- *Allow both keys in one declaration as a range (`after: :a, before: :b`)* — rejected: implies two cut points and a "region", which the single-hand-off model does not have; the reactor either has handed off or has not. Rejected at definition time instead.
-- *Leave the mechanism as-is, only rename the DSL keyword* — rejected: doesn't fix the "second flag silently ignored" confusion, since the underlying per-step check would remain.
-- *Build a new hand-off primitive from scratch* — rejected: `handle_async_step` is already correct, tested, and durability-safe (checkpoint-before-enqueue, F2); duplicating it risks a durability regression the constitution's Principle III explicitly warns about.
-
-## 2. `async_step` — dispatch a single step's work as an independent unit, mirroring `map`'s per-element dispatch
-
-**Decision**: `async_step :name` dispatches ONE lightweight job (new `Adapters::{Sidekiq,ActiveJob}::StepWorker`) that: loads the parent context by id, resolves just that step's arguments against it, runs the step body, and writes the outcome to a **new, separate storage bucket** — `store_step_result(context_id, step_name, serialized_result, reactor_class_name)` — never back into the parent's serialized context blob directly. Meanwhile, in the calling process, `StepExecutor#execute_step` marks the async_step's node `dependency_graph.complete_step(name)` immediately after dispatch (not after the real result exists) so sibling steps with no dependency on it become ready and continue executing in the same loop iteration, instead of the loop returning early the way `DispatchResult` does today.
-
-Dispatch is deliberately **not** gated on `context.inline_async_execution`. That flag exists so a worker resuming a handed-off reactor doesn't re-hand-off (old per-step flag, new `background`) — but an `async_step` reached during a worker resume (e.g. declared after the `background` point, per the spec's edge case) must still dispatch to its own independent job, or the feature silently degrades to inline execution exactly where the spec says it must not. The old flag's inline-forcing behavior in `Worker#perform` and `Map::ElementExecutor` (`element_executor.rb:50-55`) loses its async-step meaning with the flag's removal and remains only as the background-hand-off re-trigger guard.
-
-**Rationale**: This is structurally the same problem `map` already solved for N elements — dispatch independent work to a worker, and let the parent collect results later without blocking on each one individually (`lib/ruby_reactor/map/dispatcher.rb`, `lib/ruby_reactor/map/element_executor.rb`). The critical detail `map` gets right that a naive implementation would miss: it does **not** read-modify-write the whole parent context blob from the worker (`ElementExecutor#handle_result` calls `storage.store_map_result`, a dedicated per-index bucket) — because the parent context may be concurrently mutated by the still-running parent process. `async_step` has exactly the same race (the calling process keeps running other steps, possibly writing its own checkpoints, while the async_step's worker is independently about to write its result) and needs the same fix: a dedicated per-step result bucket, not a context-blob merge.
-
-**Alternatives considered**:
-- *Reuse `handle_async_step`'s whole-context hand-off for a single step* — rejected: that halts the ENTIRE reactor (returns `DispatchResult`, stops the loop); `async_step` must let the calling process keep going, which is the entire point distinguishing it from `background after:`.
-- *Write the async_step's result directly into the parent's context blob (read-modify-write on completion)* — rejected: lost-update race against the concurrently-running parent process, exactly the hazard `map` already engineered around.
-- *Use `compose`'s nested-executor model* — rejected: `compose` is synchronous-by-default and its `async` flag is really the per-step hand-off (option above) applied to a compose step, not an independent fire-and-continue unit.
-
-## 3. `async_reactor` — dispatch a nested reactor run, linked but uncompensated
-
-**Decision**: `async_reactor :name, ChildReactorClass` builds a step whose implementation constructs a fresh child `Context`, persists it, and dispatches it via the *same* `configuration.async_router.perform_async` used by whole-reactor `async true` today (`lib/ruby_reactor/reactor.rb:110-134`) — i.e. the child reactor runs exactly like any other independently-dispatched top-level reactor run, requiring no new dispatch primitive. The parent step records the child's `execution_id` (and reactor class name) in `context.composed_contexts[step_name]` as an `:async_reactor_ref` (decision 8) — written synchronously by the dispatching process itself, so no cross-process write race — and returns `Success` immediately (fire-and-forget) without registering a `compensate`/`undo` block, so the compensation manager never touches it, matching the spec's clarified compensation model (see spec.md Clarifications). No `store_step_result` record is needed for the outcome: the child's terminal state lives in its own context row, reached via the existing `retrieve_context`/`find` path.
-
-**Rationale**: The whole-reactor `async true` path (`Reactor#run`, `perform_async_run`) already implements "persist context, enqueue via pluggable router, return an `DispatchResult` immediately" for a *top-level* run. `async_reactor` needs the identical mechanics, just invoked from inside a parent step instead of from `Reactor.run`. No new enqueue primitive is needed on `Adapters::*::Router` — only the identity-only-payload dispatch that already exists (`Router.perform_async(context_id, reactor_class_name, ...)`).
-
-**Dispatch must reuse the FULL pre-enqueue sequence, not raw `perform_async`** (FR-016, found during the 2026-08-20 locks review): `Reactor#run` does three load-bearing things before enqueueing an async reactor that a naive `Context.new` + `perform_async` would silently skip — (1) child input validation (`validate_inputs`, `reactor.rb:93`; the worker's `resume_execution` path never validates, so skipping here means a child starts with garbage inputs), (2) enqueue-time ordered-lock nonce assignment (`assign_ordered_lock_nonce!`, `reactor.rb:108` — deliberately assign-at-enqueue so ordering matches caller order; a child with `with_ordered_lock` dispatched without a nonce runs with ordering silently unenforced), and (3) persist-before-enqueue (F2). The dispatch step therefore extracts/reuses this exact sequence (validate → nonce → persist → enqueue). A validation failure at dispatch fails the *dispatching step* (normal parent saga handling) — a deliberate contrast with failures during the child's independent *execution*, which never auto-compensate the parent (FR-009).
-
-**Alternatives considered**:
-- *Build `async_reactor` on top of `compose`* — rejected: `compose`'s compensate/undo blocks (`Step::ComposeStep.compensate`/`.undo`, `lib/ruby_reactor/step/compose_step.rb:32-56`) actively wire the child into the parent's rollback path; disabling that per-instance would be more invasive than not registering it in the first place.
-- *Require the child reactor class to be marked `async true` itself* — rejected: conflates "this reactor happens to always run async" (a property of the child class) with "this particular invocation, from this parent, should run independently" (a property of the call site); the spec's `async_reactor :create_profile` syntax says the latter is what's wanted, and forcing the former would make an otherwise-normal reactor class unusable synchronously elsewhere.
-
-## 4. `result(:name)` notified wait — record-first publish, subscribe-first check, fallback re-check
-
-**Decision** (revised 2026-08-20, superseding the tight-poll-only mechanism; the block-on-calling-thread contract and timeout bound are unchanged): `Template::Result#resolve` (`lib/ruby_reactor/template/result.rb:14-23`) gains a check: if `context.get_result(@step_name)` is absent AND `context.composed_contexts[@step_name]` carries one of the new async ref types (decision 8), it enters a notified wait bounded by `Configuration#async_wait_timeout`:
-
-- **Completer side** (StepWorker for `async_step`; the child's executor terminal-save for `async_reactor`): write the durable outcome **first** (the `store_step_result` record / the child's own terminal context save), **then** `storage_adapter.publish` a completion signal — `rr:done:<parent_context_id>:<step_name>` for an async_step, `rr:done:<child_execution_id>` for an async_reactor (the child publishes unconditionally on reaching a terminal state; publishing to a channel with no subscribers is near-free, so no "am I awaited?" marker is needed).
-- **Waiter side**: `storage_adapter.subscribe` to the channel **first**, *then* check the durable target once (an `:async_step_ref` checks `retrieve_step_result`; an `:async_reactor_ref` checks the linked execution's context row via `retrieve_context(execution_id, reactor_class_name)` for a terminal status) — subscribing before checking closes the race where completion lands between check and subscribe. Then block on the subscription, waking on signal OR on a coarse fallback interval (re-check the durable target each wake; Redis pub/sub is at-most-once and unpersisted, so a signal missed during a reconnect must only cost fallback latency, never correctness). **Fallback interval**: `[async_wait_timeout / 10, clamped to 1..5 seconds]` — i.e. 3s at the 30s default, at least one re-check per second for very short timeouts, and never hotter than 1s no matter how short. Derived rather than configured: it is a pure latency backstop on a path whose common case is signal-driven, so a second knob would be YAGNI (constitution Principle V); the clamp guarantees at least ~10 re-checks inside any bound, so a lost signal costs at most ~10% of the timeout. On timeout, raise/return the FR-005 timeout Failure.
-- The subscribe MUST use a **dedicated Redis connection** — `SUBSCRIBE` puts a connection into subscriber mode, so blocking the shared `@redis` client would poison every other storage call in the process. The `subscribe`/`publish` primitives already exist on the adapter interface and `RedisAdapter` (`storage/adapter.rb:38-44`, `redis_adapter.rb:177-183`, currently unused); the connection-isolation detail lands with the implementation.
-
-A *paused* async_reactor child is not terminal — the wait continues to timeout unless the child is resumed (spec Edge Cases). Read semantics on arrival are unchanged from the earlier pass: `async_step` Success injects the raw value (same shape as a sync step); `async_step` Failure and any `async_reactor` outcome inject the result object itself, so the reader can inspect success/failure and decide (contract, spec Clarifications). Synchronous (non-async) `result()` references are completely unaffected — the check is a fast no-op when the referenced step has no async ref.
-
-**Rationale**: The calling-thread blocking contract came from the 2026-08-16 clarification; the 2026-08-20 session revised the *mechanism* because a tight poll couples wake-up latency to poll frequency and Redis load. Record-then-publish / subscribe-then-check is the standard race-free pairing for Redis's at-most-once pub/sub: every code path ends at the durable record, so the signal is pure latency optimization — lost signal ⇒ fallback re-check finds the record; early signal ⇒ post-subscribe check finds the record; duplicate signal ⇒ idempotent re-read. SC-005 (bounded, never hangs) holds through all of it.
-
-**Alternatives considered**:
-- *Tight fixed-interval poll only (the original design)* — rejected as primary: latency floor = poll interval, and every concurrent waiter hammers Redis for the whole wait; kept as the fallback re-check where it is load-harmless.
-- *Pub/sub only, no durable-record confirmation* — rejected outright: Redis pub/sub is fire-and-forget; a waiter that subscribes late or reconnects mid-wait would hang to timeout on work that already finished — precisely the wrong-answer class the durable record exists to prevent.
-- *`BLPOP` on a per-result list token* — workable and even simpler for a single waiter (token persists if pushed before the waiter arrives), but `BLPOP` *consumes* the token: with two readers of the same result (legal — any number of steps may reference `result(:name)`), the second waiter starves. Broadcast pub/sub + shared durable record handles N waiters naturally.
-- *Poll the whole context blob instead of a dedicated bucket* — rejected: same lost-update/staleness concern as decision 2 — the dedicated record/child-row is written exactly once by exactly the finishing worker, so there is no ambiguity about freshness.
-
-## 5. Global timeout configuration
-
-**Decision**: Add `Configuration#async_wait_timeout` (default **30 seconds**, documented explicitly per FR-005) alongside the existing knobs in `lib/ruby_reactor/configuration.rb` (`context_ttl`, `context_lock_ttl`, `lock_snooze_*`), following the same `attr_writer` + memoized-reader pattern already used there. No per-reactor or per-`result()` override (resolved via clarification, Question 3 of 3) — YAGNI per constitution Principle V until a second real use case for overriding it emerges.
-
-**Rationale**: Matches the exact idiom already established by every other timing knob in `Configuration`.
-
-**Why 30s**: it must exceed normal end-to-end dispatch→worker-pickup→completion latency for a small unit of work under a healthy queue (sub-second when a worker is free; seconds when the queue is backed up) by a comfortable margin, while staying well under the request/job timeouts of typical hosts (Sidekiq's 25s default shutdown grace, Puma's 60s, Rails' 15s DB checkout) so a wait fails *loudly on our terms* rather than being killed from outside. It also bounds the damage of a genuinely lost unit: a reader step fails after 30s instead of pinning a worker slot. Callers whose async work is legitimately slower raise the single global knob.
-
-## 6. Testing approach — extend `TestSubject`, not a parallel harness
-
-**Decision**: `RubyReactor::RSpec::TestSubject` (`lib/ruby_reactor/rspec/test_subject.rb`) already drives the full async lifecycle for tests: `AsyncTestHelpers.drain_async_jobs` transparently drains whichever backend (Sidekiq fake mode or ActiveJob `:test` adapter) is active, and `ensure_executed!`/`result` already re-poll status after draining. The only place today's per-step `async?` flag is baked into test scaffolding is `prepare_execution_class`'s `force_sync` branch (test_subject.rb:467-477, mutates `config.instance_variable_set(:@async, false)` per step) and `apply_mock_interceptor` (test_subject.rb:609, same). These two spots are updated to understand the normalized `background_handoff` (`{ mode: :after|:before, step: }` — one concept, two trigger positions; never a one-sided `background_after` reader) plus `async_step`/`async_reactor`, instead of the removed per-step `@async`; everything else (drain helpers, `have_run_step` matcher, `map`/`composed` traversal) needs no change because it already operates at the job-queue/context level, not the DSL-flag level. The `async: false` / `run_async(false)` option's meaning is redefined accordingly: it suppresses the `background` hand-off AND runs `async_step`/`async_reactor` units inline in the calling process (today it clears per-step flags and mocks reactor-level `async?`) — so a spec can exercise a reactor's full logic synchronously without draining any queue, preserving the option's existing purpose under the new DSL.
-
-**Two lanes, because the in-memory queue fakes cannot express this feature's core behavior** (added after a constitution review — see plan.md's corrected Principle III check):
-
-- **Unit lane** — `Sidekiq::Testing.fake!` / ActiveJob `:test` + `drain_async_jobs`, as above. Correct and sufficient for everything with **no blocked caller**: definition-time guards, the deprecated-flag errors, dispatch ordering and the `composed_contexts` refs it writes, `Web::API` shapes, and read semantics resolved against a record that is *already* terminal when the reader runs.
-- **Orchestration lane** — a **real worker against a real queue**, mandatory for anything where a caller blocks in the FR-005 wait while an independent worker completes the work. Two independent reasons, either one sufficient: (a) constitution Principle III forbids mocked Sidekiq/queue state on async orchestration paths, and `fake!` is exactly that; (b) it *cannot work* — under a fake queue jobs run only when the spec calls `drain_async_jobs`, and the spec is the thing that's blocked inside `.run`, so the awaited job would never run and every such spec would time out rather than pass. Implementation: a shared context `spec/support/real_async_backend.rb` that, for the Sidekiq backend, boots one live `sidekiq -r <spec bootstrap>` process against the test Redis for the group (started once, health-checked, torn down in an `after(:context)`), and for the ActiveJob backend switches `queue_adapter` to `:async` — a real queue runner rather than the `:test` fake. Specs in this lane assert on real elapsed interleaving (sibling completes while the unit is still in flight; reader unblocks on real completion; timeout fires when no worker ever picks the job up).
-
-The lane split is the *smallest* change that satisfies Principle III: everything cheap stays cheap and fast on the fakes, and only the genuinely concurrent scenarios pay for a real worker.
-
-**Rationale**: Directly satisfies the instruction to test "only with the built-in spec helpers even if we have to add new test_helpers" — the gap is narrow (two call sites hardcoding the old flag name, plus one shared context for the real-worker lane) and the fix is additive to the existing modules (`TestSubject`, `AsyncTestHelpers`), not a new framework.
-
-**Alternatives considered**:
-- *Fakes only, everywhere* — rejected: violates Principle III and, more concretely, cannot produce a passing awaited-result spec at all (reason (b) above).
-- *Keep the fake queue but drain it from a spawned thread while the caller blocks* — rejected: it does produce the interleaving, but the "worker" is a thread in the caller's own process sharing its memory and Redis client — precisely the mock/real divergence in async and locking paths that Principle III's rationale cites as having burned this project before. It would also silently pass if the worker accidentally depended on in-process state (e.g. a class-level fixture), which is the main class of bug the real lane exists to catch.
-- *Run the whole suite against a real worker* — rejected: a live process per example group is slow, and unnecessary for the majority of specs that never block a caller.
-
-## 7. Documentation scope
-
-**Decision**: `documentation/async_reactors.md`'s "Step-Level Async" section (lines ~85-190) is rewritten to describe both `background after:` and `background before:`, including which step each form pins; new sections are added for `async_step` and `async_reactor`. `documentation/composition.md` gets a short cross-reference distinguishing `async_reactor` (fire-and-forget, no compensation link) from `compose` (synchronous, fully compensation-linked). `README.md`'s "Step-Level Async" subsection (`README.md:386` onward) is rewritten the same way. `demo_app/documentation/` is a **duplicate copy** of the top-level `documentation/` folder (confirmed: the ActiveJob Support commit touched both `documentation/async_reactors.md` and `demo_app/documentation/async_reactors.md` in the same PR) and must be kept in sync — every documentation edit in this feature is applied to both copies.
-
-**Rationale**: Grounded directly in `git show --stat 0fb6dc4` showing both paths edited together for the prior feature; treating them as one logical file (edited twice) avoids drift.
-
-## 8. Linking async units into the context (and therefore the dashboard) — reuse `composed_contexts`, don't invent a parallel channel
-
-**Decision**: The *reference* to an `async_step`/`async_reactor` (as opposed to its eventual *result*, decision 2/3 above) is written into `context.composed_contexts[step_name]` — the exact same field `compose` and `map` already use (`Step::ComposeStep#store_child_context`, `lib/ruby_reactor/step/compose_step.rb:90-96`; `map`'s `:map_ref` entries) — tagged with a new `type:` (`:async_step_ref` or `:async_reactor_ref`) instead of `:composed`/`:map_ref`. This write happens synchronously in the *dispatching* step, before it returns — the same process that's about to keep running other steps, so there's no race (unlike the *result*, which a separate worker writes later — that still goes through the race-free channel from decisions 2/3).
-
-**Rationale**: An earlier pass of this plan put the async link in a brand-new, disconnected field, missing that `composed_contexts` is precisely "children of this context, referenced for later drill-down" and is *already* wired end-to-end: `Context` already serializes/deserializes it, `RubyReactor::Web::API.hydrate_composed_contexts` (`lib/ruby_reactor/web/api.rb:223-234`) already switches on `type:` to decide how to hydrate a reference for the dashboard (today: pass `:composed` through as-is, resolve `:map_ref` into a representative element context), and `RubyReactor::RSpec::TestSubject#composed`/`#map`/`#traverse_composed` (`lib/ruby_reactor/rspec/test_subject.rb:54-160`) already know how to traverse it in tests. Reusing this one field means: (a) FR-008's "linked... for traceability" and FR-014's "dashboard renders and drills into it" are satisfied by two small additions to code that already exists for exactly this purpose, rather than a new subsystem; (b) `async_reactor`'s link needs *no new storage primitive at all* — the child is a normal, fully addressable `Reactor` with its own context row, so hydrating its reference just calls the existing `storage.retrieve_context(execution_id, reactor_class_name)` / `ChildReactorClass.find(execution_id)`, the same as any other reactor lookup; only `async_step` (which has no reactor class/context of its own — it's one step's worth of work) still needs the dedicated `store_step_result`/`retrieve_step_result` bucket from decision 2 to hold its actual outcome, reached via the `composed_contexts` reference the same way `map_ref` reaches into the map-result bucket.
-
-**Consequence for data-model.md**: `composed_contexts[step_name]` entry shape:
-- `async_step_ref`: `{ type: :async_step_ref, name:, dispatched_at: }` — the actual result lives in the Step Result Record bucket (decision 2), looked up by `(context_id, step_name)`.
-- `async_reactor_ref`: `{ type: :async_reactor_ref, name:, execution_id:, reactor_class_name:, dispatched_at: }` — the actual result lives in that execution's own context row, looked up by `(execution_id, reactor_class_name)` via the existing `retrieve_context`/`Reactor.find`.
-
-**Web/GUI follow-on** (in scope for this feature, not a future nice-to-have — constitution Principle IV: "The web dashboard MUST remain current with the reactor state model"):
-- `Web::API.determine_step_type` (`api.rb:202-214`) currently ends with `elsif config.async? → "async"`, which breaks the moment `StepConfig#async?` is removed (decision "Removed" in contracts/public-dsl.md). It gains two new branches keyed off new argument keys the DSL macros will set (mirroring how `compose`/`map` are detected via `arguments.key?(:composed_reactor_class)` / `:mapped_reactor_class`): `async_step` (detected via a marker key on the step's arguments, e.g. `:async_step_marker`) and `async_reactor` (detected via `arguments.key?(:async_reactor_class)`, with `extract_inner_class`/`nested_structure` reused exactly as `compose`/`map` already do, so the dashboard can show the child reactor's own step graph).
-- `Web::API.build_structure`'s per-step `async: config.async?` field is removed (source gone) and not replaced — the reactor-level `background after:` point is exposed once, alongside `steps`, not per-step (implementation detail for tasks phase: e.g. `{ steps: {...}, background_after: reactor_class.background_after }`).
-- `Web::API.hydrate_composed_contexts` gains two branches paralleling `hydrate_map_ref`: one resolving `:async_step_ref` via `retrieve_step_result`, one resolving `:async_reactor_ref` via `retrieve_context` on the linked execution id.
-- `gui/src/components/DagVisualizer.tsx` and `StepInspector.tsx` read `struct[key].type` as a plain string (confirmed: `gui/src/components/DagVisualizer.tsx:341` branches on `struct[key].type === 'map'`) — they need matching cases added for `'async_step'`/`'async_reactor'` (icon/color/drill-down link), a frontend task-phase item, not a backend one.
-
-**Alternatives considered**:
-- *Log-only linkage (FR-012's structured logs alone)* — rejected: satisfies "traceability" loosely but not FR-008's stronger "linked... on the context" requirement or FR-014's dashboard-drill-down requirement; logs aren't queryable from the context object itself and depend on log retention.
-- *A wholly new `context.async_links` field, separate from `composed_contexts`* — rejected (this was the original, corrected design): duplicates a mechanism that already exists for exactly this purpose, and would require teaching `Web::API`/`TestSubject` a second traversal pattern instead of extending the one they already have.
-
-## 9. Exclusive locks across the async boundary — no shared ownership, dispatch-time deadlock guard (FR-015)
-
-**Decision** (2026-08-20 session): lock ownership is NOT extended across the `async_reactor` boundary. The existing owner-based reentrancy (`Lock` owner = root context id, `executor.rb:429-430` — "allow re-entrancy across nested reactors") continues to cover `compose`, and only `compose`. An `async_reactor` child keeps its own fresh context id as owner. The deadlock this leaves open — parent holds `lock(key)` for its entire execution (released in `execute`'s `ensure`, i.e. *after* any reader step's FR-005 wait), child declares `lock` resolving to the same key, child snoozes forever while parent waits on the child — is closed at **dispatch time**: the dispatching step resolves the child's `lock_config[:key_proc]` against the mapped child inputs and, if the resulting key matches a lock the dispatching execution currently holds (the executor's `@acquired_lock`, plus any held by its root chain), fails the dispatch step immediately with an error naming the key and both reactor classes.
-
-**Rationale**: Reentrancy is a property of a single logical thread of control. `compose` qualifies — the child executes *inline, sequentially* inside the parent's frame, so "the same execution re-entering its own lock" is sound. An `async_reactor` child is definitionally concurrent with its parent; giving it the parent's owner would let both sides sit inside the critical section simultaneously — mutual exclusion silently broken, data corruption instead of a visible stall. Between "loud, immediate, explained failure" and "silent guaranteed timeout" and "silently broken lock", the first is the only acceptable one. The codebase already has this exact judgment call in precedent: nested same-key *ordered*-lock invocation is detected at nonce-assignment time via `OrderedLockSupport.active_keys` and surfaced rather than left to deadlock (`reactor.rb:441-456`). The guard is best-effort by design: it catches the same-execution circular wait (the only *guaranteed* deadlock); ordinary cross-execution contention on the same key remains normal snooze-and-retry behavior, unchanged.
-
-**Scope notes**:
-- `async_step` needs no guard: its worker acquires no reactor-level locks at all (contract — async step work runs outside the parent's lock window).
-- Semaphores with `limit: 1` have the same circular-wait shape; the guard checks the held semaphore key too when its limit is 1. Higher limits are contention, not guaranteed deadlock — left to snooze.
-- Users who need the child inside the parent's critical section have an existing, correct tool: `compose`. The error message says so.
-
-**Alternatives considered**:
-- *Reentrant across the async boundary (share the parent's owner)* — rejected: breaks mutual exclusion under true concurrency (both executions inside the critical section at once); also wrecks release/auto-extend accounting — the parent's `ensure` release would decrement/free the lock while the child still believes it holds it.
-- *Do nothing, rely on the FR-005 timeout* — rejected: turns a statically-detectable programming error into a runtime stall of `async_wait_timeout` seconds followed by an opaque timeout failure, burning a worker slot on the snoozing child the whole time.
-- *Queue the dispatch until the parent releases* — rejected: hidden sequencing that contradicts `async_reactor`'s "runs concurrently, fire-and-forget" contract, and unbounded dispatch deferral is its own liveness hazard.
-
-## 10. Recovery re-attach and parent-context lifetime (FR-017, FR-018)
-
-**Decision**: The durable dispatch marker written *before* enqueue (decision 2's Step Result Record at status `dispatched`; decision 3/8's `composed_contexts[:name] = { type: :async_reactor_ref, execution_id:, ... }`) is also the re-attach marker. On recovery/resume, `StepExecutor`'s async dispatch path checks for that marker first and, if present, skips enqueue entirely and marks the node graph-complete exactly as the original dispatch did — the pending work is already out there with its own durable identity, so re-dispatching would duplicate a side effect (a second email sent, a second child reactor execution) for no benefit. Readers are unaffected: they wait on the same record/child row either way.
-
-**Rationale**: This falls out of the checkpoint-before-enqueue ordering already required by F2 — the marker exists precisely so a crash can never leave a job with no record, and "record exists" is therefore exactly the question "was this already dispatched?". No new state and no new storage primitive; a presence check in the dispatch path. The alternative (re-dispatch on recovery) is the classic at-least-once duplicate side effect, and unlike an ordinary retried step an `async_step` has no compensation wired into the parent to clean it up.
-
-**Parent-context lifetime**: the async worker loads the parent context by id, so the parent's `context_ttl` must outlive the dispatched unit — including the fire-and-forget case where the parent completes immediately and nothing ever waits. Dispatch therefore refreshes the parent context's TTL (and stamps the Step Result Record with the same window, per data-model.md). A worker that still finds no parent context — swept, or dispatched longer ago than the retention window — writes a `completed`/`Failure` record for its unit and logs it per FR-012 instead of raising: the unit's work cannot be resolved without its parent's inputs, and an unhandled raise would just hand it to the backend's retry machinery to fail identically N more times. A reader that later asks for that result gets a `Failure` explaining the expiry, which is strictly better than a 30s timeout.
-
-**Alternatives considered**:
-- *A separate "in-flight async units" registry consulted on recovery* — rejected: duplicates what the dispatch marker already records, and adds a second thing that can disagree with the first.
-- *Give async result records their own longer TTL than the parent context* — rejected: the worker needs the parent context, not just the record, so a longer record TTL alone fixes nothing; refreshing the parent's TTL at dispatch is the single lever that actually covers the window.
-- *Block parent completion until dispatched units finish* — rejected outright: that is exactly the fire-and-forget contract (FR-018, spec Edge Cases) inverted, and would make `async_step` behave like a synchronous step.

data/specs/001-background-async-steps/spec.md

@@ -1,146 +0,0 @@
-# Feature Specification: Background Execution & Real Async Steps
-
-**Feature Branch**: `001-background-async-steps`
-
-**Created**: 2026-08-16
-
-**Status**: Draft
-
-**Input**: User description: "I want to rename to fix some confusing naming and create a feature for proper async steps. The current async naming doesn't makes much sense, it should be renamed to background... The other change is to for real do async steps and reactors. Understanding an async step running in an independent process, sidekiq worker in this case..."
-
-## Clarifications
-
-### Session 2026-08-16
-
-- Q: When an `async_step`'s independent worker fails and nothing in the parent reactor is waiting on its result, should the parent's saga compensation trigger automatically? → A: No. `async_step` and `async_reactor` share the same compensation model — a failure only surfaces, and compensation only happens, if a later step explicitly reads that step's/reactor's result via `result(:name)` and itself decides to return `Failure`. An async unit's own failure never automatically compensates its parent.
-- Q: When a step still running in the calling process reaches a `result(:name)` reference that isn't ready yet, how should the wait be implemented? → A: The calling thread blocks in a bounded poll loop against the durable store until the result appears or the timeout elapses. The reactor's existing synchronous `.run` call contract is unchanged — waiting never implicitly hands remaining execution off to a worker.
-- Q: How should the FR-005 wait timeout be configured? → A: A single library-wide default set via the existing global `Configuration` is sufficient for v1; no per-reactor or per-`result()` override syntax is introduced by this feature.
-
-### Session 2026-08-20
-
-- Q: Is a tight poll loop good enough for the FR-005 wait, or should completion be pushed to the waiter? → A: Revised (supersedes the "bounded poll loop" mechanism from the 2026-08-16 session; the blocking-on-the-calling-thread contract and the timeout bound are unchanged): the primary wake-up is a completion notification — the finishing worker durably writes the result record **first**, then publishes a completion signal; the waiter subscribes **before** checking the record (closing the completed-before-subscribe race), then blocks on the notification with a coarse periodic fallback re-check of the record (notifications are at-most-once and may be missed on reconnect — the durable record remains the sole source of truth, so a missed signal degrades to fallback-poll latency, never to a wrong answer). The FR-005 timeout still bounds the total wait.
-- Q: Should locks be reentrant across the `async_reactor` boundary, to avoid deadlocks when parent and child declare the same lock key? → A: No. Owner-based reentrancy stays as-is for `compose` (sequential, same logical thread of control), but an `async_reactor` child runs concurrently with its parent — sharing the lock owner would put both inside the critical section at once, silently breaking mutual exclusion, which is worse than the deadlock. Instead the deadlock is made impossible to hit silently: at dispatch time, if the child's lock key collides with a lock the parent execution currently holds, the dispatch step fails immediately with a clear error (see FR-015) rather than guaranteeing a wait-timeout later.
-
-### Session 2026-08-22
-
-- Q: Should the hand-off point be expressible relative to the *following* step as well, not just the preceding one? → A: Yes — `background before: :third` is supported alongside `background after: :second`. The two are complementary ways of naming a single cut point: `after:` pins the named step as the **last** step to run in the calling process; `before:` pins it as the **first** step to run in the worker. In a linear reactor the two coincide (`after: :second` ≡ `before: :third` when `:third` immediately follows `:second`); in a branching workflow they pin different steps, which is precisely when the author needs to choose which guarantee they actually want. Still exactly one hand-off point per reactor: one `background` declaration carrying exactly one of the two keys (FR-002). `background before: <the reactor's first step>` is legal and meaningful — it moves every step body to the worker while still validating inputs in the calling process, so an invalid-input failure surfaces synchronously to the caller instead of inside a worker (which is what distinguishes it from whole-reactor async).
-
-## User Scenarios & Testing *(mandatory)*
-
-### User Story 1 - Unambiguous background hand-off (Priority: P1)
-
-A reactor author currently marks a step `async: true` to hand execution off to a background worker, but if several steps carry that flag only the first one actually takes effect — the rest are silently ignored. The author wants one clear, reactor-level declaration that names the hand-off point, with no ambiguity about which point is the real one — and wants to express it from whichever side reads more naturally for the workflow: "everything after this step runs in the background" (`after:`) or "this step and everything past it runs in the background" (`before:`).
-
-**Why this priority**: This is a correctness and clarity fix on an existing, already-shipped capability. It removes a footgun that silently produces wrong behavior today, and it is the foundation the other stories build on.
-
-**Independent Test**: Define a reactor with `step :first`, `step :second`, `background after: :second`, `step :third`. Confirm `:first` and `:second` run in the calling process, and `:third` runs after hand-off to a worker process. Repeat with the same reactor declared as `background before: :third` and confirm identical behavior in this linear case. Confirm the reactor definition has no remaining per-step async flag.
-
-**Acceptance Scenarios**:
-
-1. **Given** a reactor with `background after: :second` declared once, **When** the reactor executes, **Then** steps before and including `:second` run in the current process and steps after `:second` run via an independent worker.
-2. **Given** a reactor with `background before: :third` declared once, **When** the reactor executes, **Then** `:third` and every step not yet executed at that moment run via an independent worker, and `:third` itself never runs in the calling process.
-3. **Given** a reactor definition that still uses the old per-step `async: true` flag, **When** the reactor class is loaded/defined, **Then** the system raises a clear definition-time error identifying the deprecated syntax instead of silently accepting it.
-4. **Given** a reactor with a `background` declaration naming a step that is not defined in that reactor (via either `after:` or `before:`), **When** the reactor class is defined, **Then** the system raises a clear definition-time error.
-5. **Given** a `background` declaration carrying both `after:` and `before:`, or carrying neither, **When** the reactor class is defined, **Then** the system raises a clear definition-time error — a hand-off point must be named exactly once, from exactly one side.
-
----
-
-### User Story 2 - Async steps that truly run independently (Priority: P2)
-
-A reactor author wants a single step (e.g. `send_email`) to execute in an independent worker process while the rest of the reactor keeps running in the current process. A later step in the same reactor needs to consume that async step's output once it is ready.
-
-**Why this priority**: This is the first genuinely new capability — running one unit of work off the critical path while the reactor continues — and it is the dependency other in-reactor steps rely on via `result(...)`. Compensation for an `async_step`'s failure is opt-in via that same `result(...)` read, not automatic (see Clarifications).
-
-**Independent Test**: Define a reactor with `async_step :send_email`, followed by `step :do_something_same_thread`, followed by `step :check_email` that declares `argument :email, result(:send_email)`. Confirm `:do_something_same_thread` can complete without waiting on `:send_email`, and `:check_email` receives the correct, fully-formed result of `:send_email` once it becomes available.
-
-**Acceptance Scenarios**:
-
-1. **Given** a reactor with `async_step :send_email` followed by a same-process step with no dependency on it, **When** the reactor executes, **Then** the same-process step is not blocked waiting for `:send_email` to finish.
-2. **Given** a later step that declares `argument :email, result(:send_email)`, **When** that step is reached before `:send_email` has completed, **Then** the reactor waits for `:send_email`'s durable result before running the step, and injects the deserialized result as the argument.
-3. **Given** an `async_step` that raises/fails in its independent worker, **When** the failure occurs and no later step reads its result, **Then** the owning reactor's saga compensation is NOT automatically triggered; the failure is recorded/logged only. Compensation only happens if a later step reads `result(:send_email)` and, on seeing a failure, itself returns `Failure`.
-
----
-
-### User Story 3 - Fire-and-forget async reactors (Priority: P3)
-
-A reactor author wants to kick off an entire nested reactor (e.g. `create_profile`) to run independently, tracked for observability but explicitly outside the parent reactor's compensation graph — its failure should never automatically roll back the parent. For cases where the parent does care about the outcome (e.g. `create_account`), a later step should be able to read the async reactor's result and decide for itself whether to trigger compensation.
-
-**Why this priority**: This extends story 2's independent-execution model from a single step to a whole nested reactor, and is the most involved because it touches the saga/compensation boundary between parent and child.
-
-**Independent Test**: Define a reactor with `async_reactor :create_profile` and no step referencing its result — confirm a forced failure of `create_profile` does not compensate the parent. Separately, define `async_reactor :create_account` followed by `step :verify_all` with `argument :account, result(:create_account)` and a `run` block that inspects success/failure — confirm the block receives the account reactor's outcome and can choose to return `Failure` (triggering parent compensation) or `Success`.
-
-**Acceptance Scenarios**:
-
-1. **Given** an `async_reactor :create_profile` with no downstream step reading its result, **When** `create_profile`'s execution fails, **Then** the parent reactor's already-completed steps are not compensated as a result of that failure.
-2. **Given** an `async_reactor :create_account` and a later step that declares `argument :account, result(:create_account)`, **When** the parent reaches that step before `create_account` has finished, **Then** the parent waits for `create_account`'s durable result before running the step.
-3. **Given** the `run` block of `:verify_all` inspecting `args[:account]` and explicitly returning `Failure`, **When** that block executes, **Then** the parent reactor's compensation is triggered as it would be for any other step returning `Failure`.
-4. **Given** an `async_reactor` execution, **When** it starts, **Then** its execution id is linked to the parent reactor's execution for traceability (e.g. in logs/dashboard), without adding it to the parent's compensation graph.
-
----
-
-### Edge Cases
-
-- What happens when a reactor declares `background` more than once (in any combination of `after:`/`before:`)? → System MUST reject the reactor definition at definition time with a clear error (see FR-002); only a single hand-off point is permitted.
-- What happens when `background before:` names the reactor's very first step? → Legal and meaningful: every step body runs in the worker, but inputs are still validated in the calling process, so an invalid-input failure surfaces synchronously to the caller. This is the deliberate difference from whole-reactor async (which validates inside the worker), so it is not rejected as redundant.
-- What happens when the step named by `background` never executes (skipped by a `where`/guard condition, or the reactor fails earlier)? → The hand-off never triggers and the reactor completes or fails entirely in the calling process. Identical for both `after:` and `before:`; no step is ever stranded, because the hand-off only ever moves *remaining* work.
-- What happens when a step waiting on `result(:async_step_name)` never receives a result because the independent worker crashed or never ran? → Governed by the wait policy in FR-005.
-- What happens when `async_step`/`async_reactor` is declared but nothing ever references its result? → It still executes to completion in its independent worker; no waiting occurs anywhere in the parent. The parent may therefore reach a terminal state (`completed`/`failed`) with that work still pending — this is the declared fire-and-forget model, not an inconsistency, and the parent's terminal status makes no claim about the async unit's outcome.
-- What happens when the parent context is TTL-expired or swept before an unread async unit's worker runs? → Dispatch extends the parent context's retention window so it outlives the dispatched unit (FR-018); if the context is nevertheless gone, the worker records a failure outcome for that unit and logs it (FR-012) instead of raising or retrying forever. Nothing waits on it by definition — an unread unit has no reader — so the only requirement is that the failure is observable.
-- What happens when an `async_step` is declared after a `background after:` hand-off point? → The async step's work is already off the calling process by virtue of the hand-off; it still runs in its own independent worker and follows the same result/wait/failure semantics.
-- What happens if a reactor is recovered/resumed (crash recovery) while one of its `async_step`/`async_reactor` results is still pending? → Recovery MUST re-attach to the still-pending async work rather than re-dispatching it, consistent with existing durability/recovery guarantees.
-- What happens when `returns` names an `async_step` or `async_reactor`? → Rejected at definition time with a clear error: the reactor's return value must come from a same-process step. A reactor that needs an async unit's outcome as its return value must read it through an ordinary step (`argument :x, result(:name)`) and return that step instead. (Allowing it would make reactor completion itself block or return an absent value — out of scope for v1.)
-- What happens when a reactor marked with the whole-reactor async flag also declares `background` (either form)? → Rejected at definition time: the entire reactor already runs in a worker, so a hand-off point inside it is meaningless and would otherwise be silently ignored — the exact silent-no-op failure mode this feature exists to eliminate.
-- What happens when an awaited `async_reactor`'s child pauses at an interrupt step instead of finishing? → Paused is not terminal, so the reader's wait continues and, if the child is not resumed within the bound, ends in the FR-005 timeout failure. Documented behavior, not an error: an operator resumes the child (existing interrupt mechanism) and a retry of the reading step then finds the result.
-- What happens when an `async_reactor`'s child declares the same exclusive-lock key its parent currently holds? → The dispatch step fails immediately with a clear error (FR-015). Without this guard the pattern is a guaranteed deadlock-until-timeout: the parent holds the lock for its whole execution (including any wait on the child's result) while the child snoozes waiting for that same lock. A child that genuinely needs to share the parent's critical section belongs in `compose`, not `async_reactor`.
-- What happens when a reactor that already passed its `background` hand-off pauses at an interrupt and is later resumed? → `continue` runs the remaining steps inline in the delivering process — the hand-off is edge-triggered on reaching its named step and never re-fires. A reactor that needs the resume itself to go to a worker declares `interrupt :name, resume: :background`, which validates the payload synchronously in the caller and enqueues the remainder (see contracts/public-dsl.md).
-- What happens when the completion notification is lost (waiter reconnecting, signal published before subscribe)? → Nothing is lost but latency: the durable record is written before the signal is published, the waiter checks the record after subscribing, and a coarse fallback re-check catches any missed signal within the FR-005 bound.
-
-## Requirements *(mandatory)*
-
-### Functional Requirements
-
-- **FR-001**: System MUST provide a reactor-level `background` declaration that marks a single hand-off point, after which all remaining steps execute via an independent worker process. The point MUST be expressible from either side: `background after: <step_name>` (the named step is the last to run in the calling process) or `background before: <step_name>` (the named step is the first to run in the worker, and never runs in the calling process).
-- **FR-002**: System MUST reject, at reactor-definition time, any reactor that declares `background` more than once, that names a step not defined in that reactor, or whose `background` declaration carries both `after:` and `before:` or neither — a hand-off point must be named exactly once, from exactly one side.
-- **FR-003**: System MUST remove the per-step hand-off `async` flag/DSL method entirely — both on regular steps and on `compose` steps (which carry the same flag) — and reactors still using it MUST fail at definition time with an error that names the deprecated syntax and points to its replacement (`background`, `async_step`, `async_reactor`). The `async` option inside a `map` block is NOT covered by this removal: it is a map-internal element-dispatch mode, a different mechanism with different semantics, and stays as-is.
-- **FR-004**: System MUST provide an `async_step` declaration whose work executes in an independent worker process, without blocking other same-process steps that do not depend on it.
-- **FR-005**: When any step declares `argument :x, result(:name)` referencing an `async_step` or `async_reactor` named `:name` that has not yet completed, System MUST block the executing process/thread — without implicitly changing the reactor's synchronous execution mode — until the durable result is available or a single library-wide configurable timeout (set via the existing global `Configuration`, no per-reactor or per-reference override in this feature) is exceeded, then fail that step with a clear timeout error. The wait MUST be notification-driven (completion signal published by the finishing worker after the durable record is written; waiter subscribes before checking the record) with a coarse periodic fallback re-check of the durable record, so that the common case completes with near-zero added latency and a missed notification degrades only to fallback-poll latency, never to a wrong or lost result (see Clarifications, Session 2026-08-20).
-- **FR-006**: System MUST durably persist `async_step` and `async_reactor` outcomes using the existing serialization mechanism — an `async_reactor`'s outcome through the nested execution's own persisted state, an `async_step`'s outcome in a durable per-step record — so that a dependent step deserializes the same result shape a same-process step would produce.
-- **FR-007**: System MUST provide an `async_reactor` declaration that dispatches a nested reactor to run in an independent worker process.
-- **FR-008**: System MUST link an `async_reactor`'s execution id to the parent reactor's execution/context for traceability (logs, dashboard) without adding that nested execution to the parent's compensation graph. This link MUST be recorded on the parent's own context (in the same structural location the system already uses to reference other child executions, e.g. `compose`/`map`), not only in an external log line, so it survives independently of logging configuration and can be reloaded/queried later.
-- **FR-009**: System MUST NOT automatically trigger parent-reactor compensation when an `async_reactor`'s execution fails; the parent's own steps MUST remain the only place compensation for that parent is decided.
-- **FR-010**: A step in the parent reactor MUST be able to reference an `async_reactor`'s result via `result(:name)` the same way it references an `async_step`'s result, subject to the wait/timeout policy in FR-005, and inspect that result's success/failure to decide whether to return `Success` or `Failure` itself.
-- **FR-011**: If an `async_step` fails in its independent worker, System MUST NOT automatically trigger the owning reactor's saga compensation — the same fire-and-forget compensation model as `async_reactor` (see FR-009). Compensation for an `async_step` failure MUST only happen when a later step explicitly reads its result via `result(:name)` and itself decides to return `Failure`.
-- **FR-012**: System MUST emit structured, machine-parseable log entries (key=value or JSON) for: background hand-off, `async_step` dispatch and completion, and `async_reactor` dispatch and completion. Every entry MUST carry reactor name, step name, and execution id. Entries recording a **failure** — an `async_step` failing in its worker, an `async_reactor` child reaching a failed terminal state, or a dispatch-time failure — MUST additionally carry the failure reason and the unit's redacted inputs, because under the fire-and-forget compensation model (FR-009, FR-011) such a failure may never surface anywhere else.
-- **FR-017**: When a reactor execution is recovered or resumed while an `async_step` or `async_reactor` it dispatched is still pending, System MUST re-attach to that pending work rather than re-dispatching it — the dispatch path MUST consult the durable dispatch marker (the Step Result Record's `dispatched` status for an `async_step`; the recorded `:async_reactor_ref` execution id for an `async_reactor`) before enqueueing, and skip enqueue when one is already present. Re-attaching means the step is treated as already dispatched: readers wait on the existing record exactly as they would have before the crash.
-- **FR-018**: A parent reactor MUST be permitted to reach a terminal state while `async_step`/`async_reactor` work it dispatched is still pending — that is the declared fire-and-forget model, not an error — and the pending work MUST still be able to complete. Dispatch MUST therefore extend the parent context's retention window so it outlives the dispatched unit, and a worker that nonetheless finds its parent context missing (expired or swept) MUST record a failure outcome for that unit and log it per FR-012, rather than raising an unhandled error or retrying indefinitely.
-- **FR-013**: System MUST document the breaking change (removal of the per-step `async` flag) with a migration note, per the project's semantic-versioning policy for public API changes.
-- **FR-014**: The existing web dashboard MUST render `async_step` and `async_reactor` as distinct, recognizable step types (not fall back to a generic/unknown type), and MUST let an operator drill into an `async_reactor`'s linked execution the same way it already lets them drill into a `compose`d or `map`ped child, using the FR-008 link recorded on the parent's context.
-- **FR-015**: Exclusive-lock ownership MUST NOT be shared across the `async_reactor` boundary (parent and child run concurrently — shared ownership would break mutual exclusion). Instead, at dispatch time, if the child reactor declares an exclusive lock (or a single-slot semaphore, which has the same circular-wait shape) whose resolved key equals one currently held by the dispatching execution, the dispatch step MUST fail immediately with an error naming both the lock key and the parent/child reactors — never proceed into a wait that can only end in timeout. Lock reentrancy for `compose` (same logical thread of control) is unchanged.
-- **FR-016**: Dispatching an `async_reactor` MUST apply the same pre-enqueue safeguards as a top-level asynchronous reactor run — child input validation and, where the child declares ordered locking, enqueue-time ordering assignment — so a child never starts with invalid inputs or silently loses ordering guarantees. A dispatch-time validation failure fails the dispatching step itself (normal saga handling in the parent), distinct from a failure during the child's independent execution (which follows FR-009).
-
-### Key Entities
-
-- **Background Hand-off Point**: The single `background` declaration in a reactor that marks where remaining execution moves from the calling process to an independent worker, named from exactly one side — `after: <step_name>` (that step is the last to run in the calling process) or `before: <step_name>` (that step is the first to run in the worker).
-- **Async Step**: A step whose unit of work executes in an independent worker process, producing a durably stored result that other steps in the same reactor may depend on and wait for.
-- **Async Reactor**: A nested reactor execution dispatched to an independent worker, linked to its parent's context by execution id (in the same structural location `compose`/`map` already use, so the dashboard can render and drill into it) for traceability, but excluded from the parent's automatic compensation graph.
-- **Step Result Record**: The durable, serialized record of a completed async step's output, keyed for lookup by any step that references it via `result(:name)`. (An async reactor's output is reached through that nested execution's own durable record rather than a separate copy.)
-
-## Success Criteria *(mandatory)*
-
-### Measurable Outcomes
-
-- **SC-001**: Reactor authors can express a background hand-off with exactly one declarative line per reactor, and zero reactor definitions in the codebase reference the old per-step async flag after migration.
-- **SC-002**: In a reactor with an `async_step`, same-process steps that do not depend on it complete without waiting on it, while a step that explicitly depends on its result always receives the correct, fully-deserialized result once available.
-- **SC-003**: An `async_step`'s or `async_reactor`'s failure never causes the parent reactor's completed steps to be rolled back unless a later step explicitly reads its result and triggers compensation itself — verified by failure-injection specs covering both paths (failure with no reader, failure with a reader that propagates) with zero unintended rollbacks.
-- **SC-004**: 100% of reactor definitions still written against the old per-step async flag fail fast at definition time with an actionable error, rather than silently running with incorrect behavior.
-- **SC-005**: A step waiting on an async result is never left hanging indefinitely — it either receives the result or a clear timeout failure within the configured bound, in all tested scenarios.
-- **SC-006**: An operator viewing a running reactor in the dashboard can identify every `async_step` and `async_reactor` it launched and, for `async_reactor`, open the linked execution — with zero manual log-correlation required.
-
-## Assumptions
-
-- The project's existing pluggable job backend (Sidekiq or ActiveJob, selected via `configuration.async_router`) remains the independent worker mechanism for background hand-off, `async_step`, and `async_reactor` dispatch — this feature does not hardcode Sidekiq and must work identically on either configured backend.
-- Redis remains the durable store for step/reactor results, consistent with the project's existing durability model.
-- "Independent process" means a separate worker job (a new Sidekiq or ActiveJob job, per the configured backend), not a thread or fiber within the originating process.
-- This is a breaking (MAJOR, per SemVer) change to the public step DSL; updating `README.md`, `CHANGELOG.md`, and `demo_app/` to the new syntax is required delivery work but not itself a testable acceptance criterion of this spec.
-- The wait timeout for `result()` references to async work (FR-005) is a single library-wide value exposed via the existing global `Configuration`; per-reactor or per-reference overrides are out of scope for this feature. Its default is 30 seconds.
-- Verification of async orchestration (a caller blocked on `result()` while an independent worker completes the work) runs against a **real** worker consuming a **real** queue — a live `sidekiq` process against the test Redis for the Sidekiq backend, and the ActiveJob `:async` adapter for the ActiveJob backend — per constitution Principle III. In-memory queue fakes (`Sidekiq::Testing.fake!`, ActiveJob `:test`) remain in use only for unit-level assertions that do not involve a blocked caller (definition-time guards, dispatch shape, read semantics on an already-completed record).