ruby_reactor 0.6.0 → 0.7.0

data/specs/001-background-async-steps/plan.md

@@ -0,0 +1,168 @@
+# Implementation Plan: Background Execution & Real Async Steps
+
+**Branch**: `001-background-async-steps` | **Date**: 2026-08-16 | **Spec**: [spec.md](./spec.md)
+
+**Input**: Feature specification from `/specs/001-background-async-steps/spec.md`
+
+## Summary
+
+Replace the confusing per-step `async: true` flag (only the first flagged step in a reactor actually takes effect — the rest are silently ignored) with a single, unambiguous reactor-level `background` declaration that hands off all remaining steps to an independent worker job. The cut point is nameable from either side — `background after: :second` (that step is the last to run in the calling process) or `background before: :third` (that step is the first to run in the worker) — which coincide in a linear chain but pin different steps in a DAG. Add two genuinely new capabilities on top of that: `async_step`, whose unit of work is dispatched to run in its own independent worker while the rest of the reactor keeps executing in the calling process; and `async_reactor`, which dispatches a whole nested reactor to run independently, linked to the parent by execution id but excluded from the parent's compensation graph. Both `async_step` and `async_reactor` results are consumed via the existing `result(:name)` template helper, which gains a notified wait (completion signal via the storage adapter's pub/sub, durable-record-first, bounded fallback re-check — see research.md decision 4) when the referenced work hasn't finished yet. Per the clarified spec, neither `async_step` nor `async_reactor` failures auto-compensate the parent — compensation only happens if a later step explicitly reads the result and decides to fail. Lock ownership is never shared across the async boundary; a same-key collision between parent and child fails at dispatch (FR-015), and dispatch reuses the full pre-enqueue sequence including child input validation (FR-016). All dispatch continues to go through the existing pluggable `configuration.async_router` (Sidekiq or ActiveJob), unchanged.
+
+## Technical Context
+
+**Language/Version**: Ruby >= 3.0.0 (per constitution's Technical Constraints)
+
+**Primary Dependencies**: `sidekiq` and/or `activejob` (pluggable via `RubyReactor::Configuration#async_router`, already introduced by the ActiveJob Support feature on `main`), `redis` (storage), `dry-validation` (input validation DSL, unaffected by this feature)
+
+**Storage**: Redis via `RubyReactor::Storage::RedisAdapter` (implements `RubyReactor::Storage::Adapter`). This feature adds one new storage primitive pair for per-step async results (see data-model.md) modeled directly on the existing `store_map_result` / `retrieve_map_results` pair used by `map`.
+
+**Testing**: RSpec (mandatory per constitution), in two lanes (research.md decision 6):
+
+- *Unit lane* — the existing `RubyReactor::RSpec::TestSubject` DSL (`test_reactor`, `have_run_step`, `drain_async_jobs`/`process_pending_jobs` via `AsyncTestHelpers`) over the in-memory queue fakes, for everything that does not involve a caller blocked on `result()`: definition-time guards, dispatch shape, `composed_contexts` refs, read semantics against an already-terminal record.
+- *Orchestration lane* — a **real worker consuming a real queue**, required by constitution Principle III for every async orchestration path (a caller blocked in the notified wait while an independent worker completes the work; the interleaving is not reproducible under an in-process fake, which only runs jobs when the spec itself calls `drain_async_jobs` — and the spec is blocked). Sidekiq backend: a live `sidekiq` process booted against the test Redis. ActiveJob backend: the `:async` adapter (a real queue runner, not the `:test` fake).
+
+No parallel test *framework* — per explicit instruction, only the gem's built-in spec helpers are used; the orchestration lane is one new shared context (`spec/support/real_async_backend.rb`) plus new helper methods on the existing `AsyncTestHelpers`/`TestSubject` modules, and the `TestSubject` interceptor logic is updated where it hardcodes the old per-step `async?` flag (`prepare_execution_class`/`apply_mock_interceptor`).
+
+**Target Platform**: Server-side Ruby (gem consumed by Rails/Sinatra-style host apps); demo validated against `demo_app/` (Rails app already in the repo)
+
+**Project Type**: Library (Ruby gem) with a bundled `demo_app/` integration example — matches the constitution's Gem-First Design principle
+
+**Performance Goals**: Not a throughput-sensitive change — dispatch overhead should stay within the same order of magnitude as the existing `map` per-element dispatch path it reuses patterns from. No new SLO introduced.
+
+**Constraints**: Blocking waits on `result()` (FR-005) are notified waits — completion signal via the storage adapter's existing (currently unused) `publish`/`subscribe` primitives, durable-record-first ordering, subscribe-before-check on the waiter, coarse fallback re-check, all bounded by a single library-wide `Configuration#async_wait_timeout` (new knob; see research.md decision 4 and data-model.md "Completion Signal"); the subscriber uses a dedicated Redis connection. Never an unbounded wait. Lock ownership is never shared across the async boundary (FR-015 dispatch-time deadlock guard; reentrancy stays `compose`-only). Must not change behavior of the untouched reactor-level `async` flag (`self.class.async?`, "Full Reactor Async"), of `compose`'s execution/compensation semantics (its `async` flag is removed with the shared `StepConfig` flag, per FR-003 — see contracts/public-dsl.md), or of `map` (including its map-internal `async` element-dispatch option, which is a different mechanism and stays).
+
+**Scale/Scope**: Single-gem change; touches DSL (`dsl/reactor.rb`, `dsl/step_builder.rb`, `dsl/compose_builder.rb`), executor (`executor/step_executor.rb` for dispatch, `executor.rb` for the completion-signal publish on terminal save; `dependency_graph.rb` itself unchanged), a new `async_reactor` dispatch step (`step/async_reactor_step.rb`), a framework-agnostic async-step worker body (`step_worker.rb`) plus its two adapter bindings and a `perform_step_async` entry point on both routers (mirroring `Adapters::{Sidekiq,ActiveJob}::MapElementWorker`), the new `async_waiter.rb` shared wait core, two new error classes, `context.rb` (documented `composed_contexts` ref types), `template/result.rb`, `storage/adapter.rb` + `storage/redis_adapter.rb`, `configuration.rb`, RSpec helpers plus the new real-worker shared context in `spec/support/`, the bundled web dashboard (`lib/ruby_reactor/web/api.rb` + `gui/src/components/{DagVisualizer,StepInspector}.tsx`, per FR-014), `demo_app/`, `documentation/`, `README.md`.
+
+## Constitution Check
+
+*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
+
+- **I. Gem-First Design**: PASS. New DSL (`background`, `async_step`, `async_reactor`) ships in `lib/ruby_reactor/dsl/`; the new worker class ships in both `lib/ruby_reactor/adapters/sidekiq/` and `lib/ruby_reactor/adapters/active_job/`, mirroring the existing adapter-isolation pattern — no host-application coupling, Sidekiq/ActiveJob stay optional per the existing pluggable-router mechanism.
+- **II. Saga Pattern Integrity (NON-NEGOTIABLE)**: PASS, with a deliberate, spec-clarified narrowing. `background` hand-off (either form) preserves full compensation — it is the same underlying mechanism as today's step-level hand-off, just triggered once, unambiguously, and nameable from either side. `async_step`/`async_reactor` do NOT auto-compensate on failure — this was explicitly clarified with the user (see spec Clarifications) as an intentional escape hatch mirroring the existing `compose`/whole-reactor-async model, not a silent gap: a later step that reads the result via `result(:name)` can always trigger compensation itself, so no failure is ever unrecoverable, it is opt-in rather than automatic. This mirrors the already-shipped behavior of `async_reactor`-adjacent whole-reactor `async true` composition today, so it is not a new category of principle exception.
+- **III. Test-First with Real Infrastructure**: PASS, after correction. Tests are written first and run against real Redis. An earlier pass of this plan claimed PASS while resting the *entire* async-orchestration test strategy on `Sidekiq::Testing.fake!` + in-process `drain_async_jobs` and the ActiveJob `:test` adapter — those are mocked Sidekiq/queue state, which Principle III forbids on async orchestration paths, and they also cannot reproduce the behavior under test (a caller blocked in the notified wait cannot call `drain_async_jobs`, so the job would never run and every awaited-result spec would time out rather than pass). Corrected: every spec exercising a blocked caller plus a concurrently-completing worker runs in the orchestration lane — a live `sidekiq` process against the test Redis, and the ActiveJob `:async` adapter — via the shared context in `spec/support/real_async_backend.rb`. The in-memory fakes are retained strictly for unit-level assertions with no blocked caller (definition-time guards, dispatch shape, read semantics against an already-terminal record), which is exactly the carve-out Principle III allows.
+- **IV. Observability by Default**: PASS, and specifically checked against "The web dashboard MUST remain current with the reactor state model": FR-012 requires machine-parseable (key=value/JSON) structured log entries for hand-off/dispatch/completion carrying reactor name, step name, and execution id — plus, on any failure entry, the failure reason and redacted inputs, which Principle IV mandates and an earlier pass of FR-012 omitted; this matters more here than elsewhere because the fire-and-forget model (FR-009/FR-011) can leave a worker-side failure with no other surface. Implemented following the existing `middlewares.on(:before_async_enqueue, ...)` pattern; FR-008/FR-014 additionally require the async link to be recorded on the parent's own context (reusing `composed_contexts`, the same field `compose`/`map` already use — research.md decision 8) and rendered/drillable in `Web::API` + the `gui/` dashboard, not just logged. An earlier pass of this plan only logged the link and missed the dashboard requirement; corrected after review.
+- **V. Simplicity and Semantic Versioning**: PASS. This is a MAJOR (breaking) change to the public step DSL (removal of per-step `async`), documented per FR-013. The design deliberately reuses three existing mechanisms (step-level hand-off in `StepExecutor#handle_async_step`, `map`'s per-unit dispatch-and-collect pattern, and the existing `DispatchResult`/`result()` template mechanism) rather than inventing a fourth. No speculative per-reactor/per-reference timeout override is introduced (resolved via clarification) — a single global config value only, added when a second real use case exists.
+
+No violations requiring the Complexity Tracking table.
+
+## Project Structure
+
+### Documentation (this feature)
+
+```text
+specs/001-background-async-steps/
+├── plan.md              # This file (/speckit-plan command output)
+├── research.md          # Phase 0 output (/speckit-plan command)
+├── data-model.md         # Phase 1 output (/speckit-plan command)
+├── quickstart.md        # Phase 1 output (/speckit-plan command)
+├── contracts/            # Phase 1 output (/speckit-plan command)
+│   └── public-dsl.md
+└── tasks.md              # Phase 2 output (/speckit-tasks command - NOT created by /speckit-plan)
+```
+
+### Source Code (repository root)
+
+```text
+lib/ruby_reactor/
+├── dsl/
+│   ├── reactor.rb                  # ADD: `background(after:/before:)`, `async_step`, `async_reactor`
+│   │                                #      class macros + definition-time guards (dup background,
+│   │                                #      unknown step, both-or-neither key, background×async-true,
+│   │                                #      returns×async-unit); whole-reactor `async` stays
+│   ├── step_builder.rb             # REMOVE: `async` step-level flag (raise on use, FR-003)
+│   └── compose_builder.rb          # REMOVE: `async` flag too — it sets the same StepConfig flag
+│                                    #         (raise on use, FR-003); map_builder.rb untouched
+├── executor/
+│   └── step_executor.rb            # REPLACE: `handle_async_step` keyed off the reactor's single
+│                                    #          hand-off point (post-execution check for `after:`,
+│                                    #          pre-execution check for `before:`), not per-step `async?`;
+│                                    #          ADD: async_step dispatch (fire-and-continue, marks the
+│                                    #          node complete in the DependencyGraph at dispatch time so
+│                                    #          siblings aren't blocked) and async_reactor dispatch;
+│                                    #          both re-attach instead of re-dispatching when a durable
+│                                    #          dispatch marker already exists (FR-017), and extend the
+│                                    #          parent context's retention window at dispatch (FR-018)
+├── executor.rb                     # ADD: publish completion signal after terminal context save
+│                                    #      (async_reactor completer side, research.md decision 4)
+├── async_waiter.rb                 # NEW: shared FR-005 wait core — subscribe-first, durable check,
+│                                    #      signal wake + coarse fallback re-check, timeout bound
+├── step_worker.rb                  # NEW: framework-agnostic async_step worker body (load parent
+│                                    #      context, resolve args, run, store record, publish signal;
+│                                    #      missing-parent-context handling per FR-018)
+├── context.rb                      # UPDATE: document the two new `composed_contexts` ref types
+│                                    #         (`:async_step_ref`, `:async_reactor_ref`) — no
+│                                    #         serialization change
+├── dependency_graph.rb             # UNCHANGED (no schema change; dispatch-time complete_step is
+│                                    # called by the executor, see data-model.md)
+├── error/
+│   ├── async_wait_timeout_error.rb # NEW: raised when an FR-005 wait exceeds its bound
+│   └── deprecated_dsl_error.rb     # NEW: definition-time error for the removed `async` flag (FR-003)
+├── step/
+│   └── async_reactor_step.rb       # NEW: async_reactor dispatch step — full pre-enqueue sequence
+│                                    #      (FR-016) + FR-015 deadlock guard, no compensate/undo
+├── adapters/
+│   ├── sidekiq/step_worker.rb      # NEW: independent one-shot worker for a single async_step
+│   ├── sidekiq/router.rb           # ADD: `perform_step_async` dispatch entry point
+│   ├── active_job/step_worker.rb   # NEW: ActiveJob counterpart
+│   └── active_job/router.rb        # ADD: `perform_step_async` dispatch entry point
+├── template/
+│   └── result.rb                   # UPDATE: notified wait (subscribe-first, record check, fallback
+│                                    #         re-check, timeout) when the referenced async result
+│                                    #         is not yet available (FR-005, research.md decision 4)
+├── storage/
+│   ├── adapter.rb                  # ADD: `store_step_result` / `retrieve_step_result` interface methods
+│   │                                #      (async_step outcome only — async_reactor reuses the existing
+│   │                                #      retrieve_context/find path, no new primitive needed there)
+│   └── redis_adapter.rb            # implement them (mirrors store_map_result/retrieve_map_results);
+│                                    # ADD dedicated-connection subscribe (SUBSCRIBE must not block
+│                                    # the shared client)
+├── configuration.rb                # ADD: `async_wait_timeout` config knob (default 30s)
+├── web/
+│   └── api.rb                      # UPDATE: `determine_step_type`/`build_structure` (drop removed
+│                                    #         `config.async?` field, expose the normalized
+│                                    #         `background_handoff { mode:, step: }` pair once per
+│                                    #         reactor, add async_step/async_reactor branches) and
+│                                    #         `hydrate_composed_contexts` (resolve the two new
+│                                    #         composed_contexts ref types) — FR-008, FR-014
+└── rspec/
+    └── test_subject.rb             # UPDATE: `prepare_execution_class`/interceptors to understand the
+                                     #         normalized `background_handoff`/`async_step`/
+                                     #         `async_reactor` instead of the removed per-step `async?`;
+                                     #         ADD `#async_step`/`#async_reactor` traversal helpers
+                                     #         mirroring `#composed`/`#map`
+
+gui/src/components/
+├── DagVisualizer.tsx               # UPDATE: render 'async_step'/'async_reactor' step types (FR-014)
+└── StepInspector.tsx               # UPDATE: same
+
+spec/ruby_reactor/                  # gem's own unit/integration specs (mirrors lib/ layout above)
+spec/support/reactors/              # new fixture reactors for background/async_step/async_reactor
+spec/support/async_backends.rb      # NEW: shared context parameterizing a group over both backends
+spec/support/real_async_backend.rb  # NEW: orchestration lane — live `sidekiq` process against the
+                                     #      test Redis / ActiveJob `:async` adapter (Principle III)
+
+demo_app/app/reactors/
+├── full_async_reactor.rb           # UNCHANGED (whole-reactor async, out of scope)
+├── partial_async_reactor.rb        # REPLACED by `background_demo_reactor.rb`, a `background after:`
+│                                    # example (old per-step `async true` syntax is removed, FR-003);
+│                                    # migrated in Phase 2 with the rest of the call sites, since the
+│                                    # old syntax stops parsing the moment the flag is removed
+├── async_step_demo_reactor.rb      # NEW: demonstrates async_step + result() wait (send_email example)
+└── async_reactor_demo_reactor.rb   # NEW: demonstrates async_reactor fire-and-forget vs. awaited
+demo_app/spec/reactors/             # matching specs, using the same built-in TestSubject helpers
+
+documentation/
+├── async_reactors.md               # REWRITE "Step-Level Async" to `background after:`/`before:`;
+│                                    # ADD sections for `async_step` and `async_reactor`
+└── composition.md                  # cross-reference `async_reactor` vs. `compose`
+demo_app/documentation/             # kept in sync with the same edits (duplicate copy, see research.md)
+
+README.md                           # rewrite "Step-Level Async" subsection, add async_step/async_reactor
+CHANGELOG.md                        # breaking-change entry (FR-013)
+```
+
+**Structure Decision**: Single-project Ruby gem layout (existing `lib/ruby_reactor/**`, `spec/**`, plus the bundled `demo_app/` Rails integration example and `documentation/**`). No new top-level directories — every change lands inside the existing module boundaries (`dsl/`, `executor/`, `adapters/`, `template/`, `storage/`, `rspec/`), consistent with Principle I (Gem-First Design).
+
+## Complexity Tracking
+
+*No Constitution Check violations — table not needed.*

data/specs/001-background-async-steps/quickstart.md

@@ -0,0 +1,102 @@
+# Quickstart: Validating Background Execution & Real Async Steps
+
+Prerequisites: repo checked out on this branch, `bundle install` run (repo root and `demo_app/`), Redis running (`docker-compose up -d` at repo root; the gem's suite reads `RUBY_REACTOR_TEST_REDIS_URL`, defaulting to `redis://localhost:6780` — see `spec/spec_helper.rb:13`). Real Redis, not mocked, per the constitution's storage requirement.
+
+## 1. Gem-level unit/integration specs
+
+```bash
+bundle exec rspec spec/ruby_reactor/dsl/reactor_background_spec.rb        # background after:/before:
+bundle exec rspec spec/ruby_reactor/dsl/async_step_spec.rb                # async_step
+bundle exec rspec spec/ruby_reactor/dsl/async_reactor_spec.rb             # async_reactor
+bundle exec rspec spec/ruby_reactor/rspec/test_subject_async_spec.rb      # TestSubject support for the new DSL
+```
+
+(Exact spec file names are placeholders for the tasks phase — see contracts/public-dsl.md for the acceptance scenarios each must cover, taken directly from spec.md's User Stories 1–3.)
+
+Expected outcomes, mapped to spec.md acceptance scenarios:
+
+- A reactor with `background after: :second` runs `:first`/`:second` inline and `:third` via a dispatched job; `MyReactor.run` returns an `DispatchResult`; `TestSubject`'s `drain_async_jobs` completes it (US1 scenario 1, SC-001).
+- The same reactor declared `background before: :third` behaves identically in this linear case, and `:third` never executes in the calling process (US1 scenario 2). In a branching fixture the two forms pin different steps — assert the specific step each one guarantees.
+- A `background` declaration that is duplicated, names an unknown step, carries both `after:` and `before:`, carries neither, or sits on a whole-reactor-`async` reactor each raise at class-definition time (US1 scenarios 4-5, FR-002).
+- A reactor still using `step { async true }` raises a `ValidationError` at class-definition time, not at run time (US1 scenario 3, SC-004).
+- An `async_step` reactor: a sibling step with no dependency on it completes without waiting; a step declaring `argument :x, result(:async_step_name)` receives the correct value once available (US2, SC-002).
+- An `async_step` failure with no downstream reader does NOT flip the parent to `failed`/trigger compensation; a downstream reader that inspects the failure and returns `Failure` DOES trigger compensation (US2 acceptance scenario 3, Clarifications).
+- An `async_reactor` with no reader: forcing the child to fail does not compensate the parent. An `async_reactor` with a reader: the reader's `run` block sees the child's real `Success`/`Failure` and can choose to propagate (US3, SC-003).
+- A `result()` reference to a never-completing async unit fails with a timeout error, not an indefinite hang (SC-005) — verify by pointing `Configuration#async_wait_timeout` at a short value in the spec and never draining the corresponding job.
+- The notified wait is race-free (FR-005, Session 2026-08-20 clarification): a completion that lands *before* the waiter subscribes is still found (subscribe-then-check), and a dropped signal is caught by the fallback re-check — verify with a spec that completes the async unit before the reader step runs, and one that publishes nothing and relies on the record alone.
+- An `async_reactor` whose child declares the same `lock` key the parent holds fails at dispatch with an error naming the key and both reactors (FR-015) — not a wait-then-timeout.
+- Dispatching an `async_reactor` with invalid child inputs fails the dispatching step in the parent (FR-016), while a child that fails *during execution* still never auto-compensates the parent (FR-009).
+
+## 2. Sidekiq AND ActiveJob backends both pass
+
+Per Assumptions (spec.md) and Technical Context (plan.md), this feature must not hardcode a backend. Run the relevant spec files twice — once with the default (`Sidekiq::Testing.fake!`) and once with `config.async_router = RubyReactor::Adapters::ActiveJob::Router` + `ActiveJob::Base.queue_adapter = :test` — however the existing test suite's backend-parameterization convention already does this (check `spec/support/` for a shared-example/shared-context wrapping both backends before inventing a new one).
+
+## 3. `demo_app` end-to-end
+
+```bash
+cd demo_app
+bin/rails db:test:prepare   # if needed
+bundle exec rspec spec/reactors/background_demo_reactor_spec.rb
+bundle exec rspec spec/reactors/async_step_demo_reactor_spec.rb
+bundle exec rspec spec/reactors/async_reactor_demo_reactor_spec.rb
+```
+
+These exercise the new example reactors (`app/reactors/*.rb`, replacing `partial_async_reactor.rb`'s old syntax) through the full Rails/Sidekiq(or ActiveJob) stack the demo app wires up, giving a real (not just unit-tested) confirmation that the feature works end-to-end — the closest thing this library has to a "run it in a browser" check for a non-UI gem.
+
+## 4. Manual smoke check (optional, for a human reviewer)
+
+```ruby
+# bin/console or demo_app's bin/console
+result = MyReactor.run(...)
+result.class            # => RubyReactor::DispatchResult (background) or RubyReactor::Success (no background)
+RubyReactor::RSpec::AsyncTestHelpers.drain_async_jobs if defined?(RubyReactor::RSpec) # in a test/console context
+MyReactor.find(result.execution_id).result
+```
+
+## 5. Web dashboard visibility (FR-008, FR-014, SC-006)
+
+```bash
+bundle exec rspec spec/ruby_reactor/web/api_spec.rb   # determine_step_type / hydrate_composed_contexts additions
+cd gui && npm run dev                                  # or npm test if component tests exist
+```
+
+Run a reactor with an `async_step` and one with an `async_reactor` (e.g. the new `demo_app` example reactors from step 3), open the dashboard, and confirm: the step renders with a distinct `async_step`/`async_reactor` badge (not falling back to generic `step`), and for `async_reactor`, clicking through opens the linked child execution — the same drill-down `compose`/`map` already provide. This is a UI-affecting change (constitution: "For UI or frontend changes, start the dev server and use the feature in a browser before reporting the task as complete") — do not report FR-014 done from passing specs alone.
+
+## 6. Documentation review
+
+- `documentation/async_reactors.md` and its duplicate `demo_app/documentation/async_reactors.md` render correctly and no longer show `step { async true }` as the recommended step-level pattern.
+- `README.md`'s "Step-Level Async" subsection reflects `background after:`/`before:`.
+- `CHANGELOG.md` has a breaking-change entry under the correct semantic heading (constitution Development Workflow requirement).
+
+## Done criteria
+
+All specs above pass; `bundle exec rubocop` is clean (constitution requirement, no `--disable-pending-cops`); `demo_app`'s specs pass against both configured backends if the demo app is wired to test both (verify via its `config/` — otherwise document which single backend the demo exercises).
+
+---
+
+## Validation run — 2026-08-22
+
+| Step | Result |
+|---|---|
+| 1. Gem specs | **733 examples, 0 failures**, 1 pre-existing pending |
+| 2. Both backends | Every async spec group runs twice via `for_each_async_backend` (Sidekiq fake mode + ActiveJob `:test`); the blocked-caller groups run twice via `for_each_real_async_backend` (a **live sidekiq process** against the test Redis + the ActiveJob `:async` adapter) |
+| 3. `demo_app` | **94 examples, 0 failures** (13 of them the new async examples) |
+| 5. Dashboard — API | Driven through the real `Web::Application` Rack app for a reactor carrying all three new constructs: `async_step`/`async_reactor` step types render as themselves (not generic `step`), the dead per-step `async` field is gone, `background_handoff` is exposed once per reactor, the async_reactor node carries `nested_structure` plus a child `execution_id` that resolves to an addressable execution, and the `async_step` ref hydrates to its Step Result Record |
+| 5. Dashboard — GUI | `tsc -b` clean, `npm run build` succeeds, 36 component tests pass, `eslint` at its pre-existing baseline |
+| 5. Dashboard — browser | **Not performed.** See below. |
+| 6. Documentation | Both trees rewritten and kept in sync; no remaining example anywhere shows the removed `step { async true }` |
+| Rubocop | **216 files, no offenses** (no `--disable-pending-cops`) |
+
+### Open item: visual browser confirmation
+
+Step 5 asks for a human to open the dashboard and confirm the new nodes render
+and the async_reactor drill-down opens the child execution. That was **not**
+done: the implementing session was non-interactive with no browser available.
+
+Everything the browser check would depend on is verified programmatically above
+— the payload shape the components read, the components' own tests, the
+typecheck, and a clean production build — but the rendered result itself has not
+been looked at. **A reviewer should still open the dashboard once** (`cd gui &&
+npm run dev` against a reactor with an `async_step` and an `async_reactor`, e.g.
+the new `demo_app` examples) and confirm the node styling and the child link,
+before considering FR-014 closed.

data/specs/001-background-async-steps/research.md

@@ -0,0 +1,150 @@
+# Phase 0 Research: Background Execution & Real Async Steps
+
+No `[NEEDS CLARIFICATION]` markers remained in the spec after `/speckit-clarify`. This research phase instead grounds the plan's technical decisions in the current codebase (which now includes the ActiveJob Support feature merged to `main` after the spec was first drafted) — the "unknowns" here are architectural design decisions, not missing product requirements.
+
+## 1. `background after:` / `background before:` — reuse the existing step-level hand-off mechanism
+
+**Decision**: Implement `background` by moving the trigger condition in `StepExecutor#execute_step` from "does *this* step have `async?` set" to "have we reached the reactor's single declared hand-off point." The actual hand-off body (`StepExecutor#handle_async_step`, `lib/ruby_reactor/executor/step_executor.rb:200-224`) is reused almost unchanged for both forms: persist the root context, call `configuration.async_router.perform_async`, return an `DispatchResult` that halts `execute_all_steps`.
+
+The two forms differ only in *where in `execute_step` the check sits* (2026-08-22 session):
+
+- `after: :x` — a **post-execution** check: `:x` runs normally, and the hand-off fires once its result is recorded. Existing behavior, just re-keyed off the reactor-level declaration.
+- `before: :x` — a **pre-execution** check at the top of `execute_step`: when `:x` is selected to run, hand off *instead of* running it, leaving `:x` unexecuted and its dependency-graph node incomplete so the worker runs it. This is strictly simpler than the `after:` path (nothing to record first), and it reuses the identical checkpoint-and-enqueue body.
+
+Both must sit inside the existing `inline_async_execution` guard so the hand-off never re-triggers once the worker picks the reactor up.
+
+**Rationale**: `handle_async_step` already does exactly what `background after:` needs — checkpoint, enqueue via the pluggable router, halt. The bug the user is renaming away from (`documentation/async_reactors.md:87`, `demo_app/app/reactors/partial_async_reactor.rb`) is purely in *how the trigger is decided*: today it's "the first step encountered with `async?` true," which silently produces the same one-hand-off-point behavior as `background after:` would, but expressed as a per-step flag that reads as if each flagged step independently hands off (it doesn't — `lib/ruby_reactor/executor.rb:530-561`, `execute_current_step_and_continue`, shows any step reached with `inline_async_execution == true` — i.e. every step from the second async-flagged one onward — just runs inline in the worker). Moving the flag to the reactor class level and enforcing "at most one" (FR-002) makes the existing correct mechanism impossible to misread.
+
+**Why support both sides rather than just `after:`** (2026-08-22): they are not redundant in a DAG. Each form guarantees something about the step it names — `after: :x` guarantees `:x` ran in the calling process; `before: :x` guarantees `:x` runs in the worker. In a linear chain those coincide, so `after: :second` ≡ `before: :third`; in a branching workflow the author frequently cares about exactly one of the two (e.g. "the payment charge must happen synchronously so the caller sees it fail" → `after: :charge`; "the report render must never block the request" → `before: :render_report`). Naming the cut from the wrong side in a DAG silently pins the wrong step, so offering both is what makes the intent expressible rather than approximated. Cost is one extra check position in `execute_step` and one extra definition-time guard.
+
+**Removal blast radius** (verified against both builders): `ComposeBuilder#async` (`dsl/compose_builder.rb:31-33`) sets the same `StepConfig` `async:` flag being removed (`compose_builder.rb:62`), so it is removed with it and raises the same definition-time error (migration: `background before: :<that compose step>`, which reproduces the old flag semantics exactly). `MapBuilder#async` (`dsl/map_builder.rb:43`) is untouched: it is a map-internal element-dispatch mode passed as a step *argument*, and the map's `StepConfig` is hardcoded `async: false` (`map_builder.rb:111`) — different mechanism, out of scope. Three definition-time guards accompany the new macro (spec Edge Cases): `background` + whole-reactor `async true` is rejected (the hand-off would be a silent no-op inside a reactor already running in a worker), a `background` carrying both or neither of `after:`/`before:` is rejected, and `returns` naming an async unit is rejected. Inside the worker, the hand-off never re-triggers — the existing `inline_async_execution` guard keeps that job (and that job only; see decision 2 on why async_step dispatch ignores it).
+
+**Alternatives considered**:
+- *Support only `after:`, treat `before:` as sugar the author can hand-translate* — rejected: the translation is only valid in a linear chain. In a DAG "hand off before `:x`" has no reliable `after:` spelling (there may be several predecessors, or the relevant one may vary by branch), so the author would have to over-approximate the cut.
+- *Allow both keys in one declaration as a range (`after: :a, before: :b`)* — rejected: implies two cut points and a "region", which the single-hand-off model does not have; the reactor either has handed off or has not. Rejected at definition time instead.
+- *Leave the mechanism as-is, only rename the DSL keyword* — rejected: doesn't fix the "second flag silently ignored" confusion, since the underlying per-step check would remain.
+- *Build a new hand-off primitive from scratch* — rejected: `handle_async_step` is already correct, tested, and durability-safe (checkpoint-before-enqueue, F2); duplicating it risks a durability regression the constitution's Principle III explicitly warns about.
+
+## 2. `async_step` — dispatch a single step's work as an independent unit, mirroring `map`'s per-element dispatch
+
+**Decision**: `async_step :name` dispatches ONE lightweight job (new `Adapters::{Sidekiq,ActiveJob}::StepWorker`) that: loads the parent context by id, resolves just that step's arguments against it, runs the step body, and writes the outcome to a **new, separate storage bucket** — `store_step_result(context_id, step_name, serialized_result, reactor_class_name)` — never back into the parent's serialized context blob directly. Meanwhile, in the calling process, `StepExecutor#execute_step` marks the async_step's node `dependency_graph.complete_step(name)` immediately after dispatch (not after the real result exists) so sibling steps with no dependency on it become ready and continue executing in the same loop iteration, instead of the loop returning early the way `DispatchResult` does today.
+
+Dispatch is deliberately **not** gated on `context.inline_async_execution`. That flag exists so a worker resuming a handed-off reactor doesn't re-hand-off (old per-step flag, new `background`) — but an `async_step` reached during a worker resume (e.g. declared after the `background` point, per the spec's edge case) must still dispatch to its own independent job, or the feature silently degrades to inline execution exactly where the spec says it must not. The old flag's inline-forcing behavior in `Worker#perform` and `Map::ElementExecutor` (`element_executor.rb:50-55`) loses its async-step meaning with the flag's removal and remains only as the background-hand-off re-trigger guard.
+
+**Rationale**: This is structurally the same problem `map` already solved for N elements — dispatch independent work to a worker, and let the parent collect results later without blocking on each one individually (`lib/ruby_reactor/map/dispatcher.rb`, `lib/ruby_reactor/map/element_executor.rb`). The critical detail `map` gets right that a naive implementation would miss: it does **not** read-modify-write the whole parent context blob from the worker (`ElementExecutor#handle_result` calls `storage.store_map_result`, a dedicated per-index bucket) — because the parent context may be concurrently mutated by the still-running parent process. `async_step` has exactly the same race (the calling process keeps running other steps, possibly writing its own checkpoints, while the async_step's worker is independently about to write its result) and needs the same fix: a dedicated per-step result bucket, not a context-blob merge.
+
+**Alternatives considered**:
+- *Reuse `handle_async_step`'s whole-context hand-off for a single step* — rejected: that halts the ENTIRE reactor (returns `DispatchResult`, stops the loop); `async_step` must let the calling process keep going, which is the entire point distinguishing it from `background after:`.
+- *Write the async_step's result directly into the parent's context blob (read-modify-write on completion)* — rejected: lost-update race against the concurrently-running parent process, exactly the hazard `map` already engineered around.
+- *Use `compose`'s nested-executor model* — rejected: `compose` is synchronous-by-default and its `async` flag is really the per-step hand-off (option above) applied to a compose step, not an independent fire-and-continue unit.
+
+## 3. `async_reactor` — dispatch a nested reactor run, linked but uncompensated
+
+**Decision**: `async_reactor :name, ChildReactorClass` builds a step whose implementation constructs a fresh child `Context`, persists it, and dispatches it via the *same* `configuration.async_router.perform_async` used by whole-reactor `async true` today (`lib/ruby_reactor/reactor.rb:110-134`) — i.e. the child reactor runs exactly like any other independently-dispatched top-level reactor run, requiring no new dispatch primitive. The parent step records the child's `execution_id` (and reactor class name) in `context.composed_contexts[step_name]` as an `:async_reactor_ref` (decision 8) — written synchronously by the dispatching process itself, so no cross-process write race — and returns `Success` immediately (fire-and-forget) without registering a `compensate`/`undo` block, so the compensation manager never touches it, matching the spec's clarified compensation model (see spec.md Clarifications). No `store_step_result` record is needed for the outcome: the child's terminal state lives in its own context row, reached via the existing `retrieve_context`/`find` path.
+
+**Rationale**: The whole-reactor `async true` path (`Reactor#run`, `perform_async_run`) already implements "persist context, enqueue via pluggable router, return an `DispatchResult` immediately" for a *top-level* run. `async_reactor` needs the identical mechanics, just invoked from inside a parent step instead of from `Reactor.run`. No new enqueue primitive is needed on `Adapters::*::Router` — only the identity-only-payload dispatch that already exists (`Router.perform_async(context_id, reactor_class_name, ...)`).
+
+**Dispatch must reuse the FULL pre-enqueue sequence, not raw `perform_async`** (FR-016, found during the 2026-08-20 locks review): `Reactor#run` does three load-bearing things before enqueueing an async reactor that a naive `Context.new` + `perform_async` would silently skip — (1) child input validation (`validate_inputs`, `reactor.rb:93`; the worker's `resume_execution` path never validates, so skipping here means a child starts with garbage inputs), (2) enqueue-time ordered-lock nonce assignment (`assign_ordered_lock_nonce!`, `reactor.rb:108` — deliberately assign-at-enqueue so ordering matches caller order; a child with `with_ordered_lock` dispatched without a nonce runs with ordering silently unenforced), and (3) persist-before-enqueue (F2). The dispatch step therefore extracts/reuses this exact sequence (validate → nonce → persist → enqueue). A validation failure at dispatch fails the *dispatching step* (normal parent saga handling) — a deliberate contrast with failures during the child's independent *execution*, which never auto-compensate the parent (FR-009).
+
+**Alternatives considered**:
+- *Build `async_reactor` on top of `compose`* — rejected: `compose`'s compensate/undo blocks (`Step::ComposeStep.compensate`/`.undo`, `lib/ruby_reactor/step/compose_step.rb:32-56`) actively wire the child into the parent's rollback path; disabling that per-instance would be more invasive than not registering it in the first place.
+- *Require the child reactor class to be marked `async true` itself* — rejected: conflates "this reactor happens to always run async" (a property of the child class) with "this particular invocation, from this parent, should run independently" (a property of the call site); the spec's `async_reactor :create_profile` syntax says the latter is what's wanted, and forcing the former would make an otherwise-normal reactor class unusable synchronously elsewhere.
+
+## 4. `result(:name)` notified wait — record-first publish, subscribe-first check, fallback re-check
+
+**Decision** (revised 2026-08-20, superseding the tight-poll-only mechanism; the block-on-calling-thread contract and timeout bound are unchanged): `Template::Result#resolve` (`lib/ruby_reactor/template/result.rb:14-23`) gains a check: if `context.get_result(@step_name)` is absent AND `context.composed_contexts[@step_name]` carries one of the new async ref types (decision 8), it enters a notified wait bounded by `Configuration#async_wait_timeout`:
+
+- **Completer side** (StepWorker for `async_step`; the child's executor terminal-save for `async_reactor`): write the durable outcome **first** (the `store_step_result` record / the child's own terminal context save), **then** `storage_adapter.publish` a completion signal — `rr:done:<parent_context_id>:<step_name>` for an async_step, `rr:done:<child_execution_id>` for an async_reactor (the child publishes unconditionally on reaching a terminal state; publishing to a channel with no subscribers is near-free, so no "am I awaited?" marker is needed).
+- **Waiter side**: `storage_adapter.subscribe` to the channel **first**, *then* check the durable target once (an `:async_step_ref` checks `retrieve_step_result`; an `:async_reactor_ref` checks the linked execution's context row via `retrieve_context(execution_id, reactor_class_name)` for a terminal status) — subscribing before checking closes the race where completion lands between check and subscribe. Then block on the subscription, waking on signal OR on a coarse fallback interval (re-check the durable target each wake; Redis pub/sub is at-most-once and unpersisted, so a signal missed during a reconnect must only cost fallback latency, never correctness). **Fallback interval**: `[async_wait_timeout / 10, clamped to 1..5 seconds]` — i.e. 3s at the 30s default, at least one re-check per second for very short timeouts, and never hotter than 1s no matter how short. Derived rather than configured: it is a pure latency backstop on a path whose common case is signal-driven, so a second knob would be YAGNI (constitution Principle V); the clamp guarantees at least ~10 re-checks inside any bound, so a lost signal costs at most ~10% of the timeout. On timeout, raise/return the FR-005 timeout Failure.
+- The subscribe MUST use a **dedicated Redis connection** — `SUBSCRIBE` puts a connection into subscriber mode, so blocking the shared `@redis` client would poison every other storage call in the process. The `subscribe`/`publish` primitives already exist on the adapter interface and `RedisAdapter` (`storage/adapter.rb:38-44`, `redis_adapter.rb:177-183`, currently unused); the connection-isolation detail lands with the implementation.
+
+A *paused* async_reactor child is not terminal — the wait continues to timeout unless the child is resumed (spec Edge Cases). Read semantics on arrival are unchanged from the earlier pass: `async_step` Success injects the raw value (same shape as a sync step); `async_step` Failure and any `async_reactor` outcome inject the result object itself, so the reader can inspect success/failure and decide (contract, spec Clarifications). Synchronous (non-async) `result()` references are completely unaffected — the check is a fast no-op when the referenced step has no async ref.
+
+**Rationale**: The calling-thread blocking contract came from the 2026-08-16 clarification; the 2026-08-20 session revised the *mechanism* because a tight poll couples wake-up latency to poll frequency and Redis load. Record-then-publish / subscribe-then-check is the standard race-free pairing for Redis's at-most-once pub/sub: every code path ends at the durable record, so the signal is pure latency optimization — lost signal ⇒ fallback re-check finds the record; early signal ⇒ post-subscribe check finds the record; duplicate signal ⇒ idempotent re-read. SC-005 (bounded, never hangs) holds through all of it.
+
+**Alternatives considered**:
+- *Tight fixed-interval poll only (the original design)* — rejected as primary: latency floor = poll interval, and every concurrent waiter hammers Redis for the whole wait; kept as the fallback re-check where it is load-harmless.
+- *Pub/sub only, no durable-record confirmation* — rejected outright: Redis pub/sub is fire-and-forget; a waiter that subscribes late or reconnects mid-wait would hang to timeout on work that already finished — precisely the wrong-answer class the durable record exists to prevent.
+- *`BLPOP` on a per-result list token* — workable and even simpler for a single waiter (token persists if pushed before the waiter arrives), but `BLPOP` *consumes* the token: with two readers of the same result (legal — any number of steps may reference `result(:name)`), the second waiter starves. Broadcast pub/sub + shared durable record handles N waiters naturally.
+- *Poll the whole context blob instead of a dedicated bucket* — rejected: same lost-update/staleness concern as decision 2 — the dedicated record/child-row is written exactly once by exactly the finishing worker, so there is no ambiguity about freshness.
+
+## 5. Global timeout configuration
+
+**Decision**: Add `Configuration#async_wait_timeout` (default **30 seconds**, documented explicitly per FR-005) alongside the existing knobs in `lib/ruby_reactor/configuration.rb` (`context_ttl`, `context_lock_ttl`, `lock_snooze_*`), following the same `attr_writer` + memoized-reader pattern already used there. No per-reactor or per-`result()` override (resolved via clarification, Question 3 of 3) — YAGNI per constitution Principle V until a second real use case for overriding it emerges.
+
+**Rationale**: Matches the exact idiom already established by every other timing knob in `Configuration`.
+
+**Why 30s**: it must exceed normal end-to-end dispatch→worker-pickup→completion latency for a small unit of work under a healthy queue (sub-second when a worker is free; seconds when the queue is backed up) by a comfortable margin, while staying well under the request/job timeouts of typical hosts (Sidekiq's 25s default shutdown grace, Puma's 60s, Rails' 15s DB checkout) so a wait fails *loudly on our terms* rather than being killed from outside. It also bounds the damage of a genuinely lost unit: a reader step fails after 30s instead of pinning a worker slot. Callers whose async work is legitimately slower raise the single global knob.
+
+## 6. Testing approach — extend `TestSubject`, not a parallel harness
+
+**Decision**: `RubyReactor::RSpec::TestSubject` (`lib/ruby_reactor/rspec/test_subject.rb`) already drives the full async lifecycle for tests: `AsyncTestHelpers.drain_async_jobs` transparently drains whichever backend (Sidekiq fake mode or ActiveJob `:test` adapter) is active, and `ensure_executed!`/`result` already re-poll status after draining. The only place today's per-step `async?` flag is baked into test scaffolding is `prepare_execution_class`'s `force_sync` branch (test_subject.rb:467-477, mutates `config.instance_variable_set(:@async, false)` per step) and `apply_mock_interceptor` (test_subject.rb:609, same). These two spots are updated to understand the normalized `background_handoff` (`{ mode: :after|:before, step: }` — one concept, two trigger positions; never a one-sided `background_after` reader) plus `async_step`/`async_reactor`, instead of the removed per-step `@async`; everything else (drain helpers, `have_run_step` matcher, `map`/`composed` traversal) needs no change because it already operates at the job-queue/context level, not the DSL-flag level. The `async: false` / `run_async(false)` option's meaning is redefined accordingly: it suppresses the `background` hand-off AND runs `async_step`/`async_reactor` units inline in the calling process (today it clears per-step flags and mocks reactor-level `async?`) — so a spec can exercise a reactor's full logic synchronously without draining any queue, preserving the option's existing purpose under the new DSL.
+
+**Two lanes, because the in-memory queue fakes cannot express this feature's core behavior** (added after a constitution review — see plan.md's corrected Principle III check):
+
+- **Unit lane** — `Sidekiq::Testing.fake!` / ActiveJob `:test` + `drain_async_jobs`, as above. Correct and sufficient for everything with **no blocked caller**: definition-time guards, the deprecated-flag errors, dispatch ordering and the `composed_contexts` refs it writes, `Web::API` shapes, and read semantics resolved against a record that is *already* terminal when the reader runs.
+- **Orchestration lane** — a **real worker against a real queue**, mandatory for anything where a caller blocks in the FR-005 wait while an independent worker completes the work. Two independent reasons, either one sufficient: (a) constitution Principle III forbids mocked Sidekiq/queue state on async orchestration paths, and `fake!` is exactly that; (b) it *cannot work* — under a fake queue jobs run only when the spec calls `drain_async_jobs`, and the spec is the thing that's blocked inside `.run`, so the awaited job would never run and every such spec would time out rather than pass. Implementation: a shared context `spec/support/real_async_backend.rb` that, for the Sidekiq backend, boots one live `sidekiq -r <spec bootstrap>` process against the test Redis for the group (started once, health-checked, torn down in an `after(:context)`), and for the ActiveJob backend switches `queue_adapter` to `:async` — a real queue runner rather than the `:test` fake. Specs in this lane assert on real elapsed interleaving (sibling completes while the unit is still in flight; reader unblocks on real completion; timeout fires when no worker ever picks the job up).
+
+The lane split is the *smallest* change that satisfies Principle III: everything cheap stays cheap and fast on the fakes, and only the genuinely concurrent scenarios pay for a real worker.
+
+**Rationale**: Directly satisfies the instruction to test "only with the built-in spec helpers even if we have to add new test_helpers" — the gap is narrow (two call sites hardcoding the old flag name, plus one shared context for the real-worker lane) and the fix is additive to the existing modules (`TestSubject`, `AsyncTestHelpers`), not a new framework.
+
+**Alternatives considered**:
+- *Fakes only, everywhere* — rejected: violates Principle III and, more concretely, cannot produce a passing awaited-result spec at all (reason (b) above).
+- *Keep the fake queue but drain it from a spawned thread while the caller blocks* — rejected: it does produce the interleaving, but the "worker" is a thread in the caller's own process sharing its memory and Redis client — precisely the mock/real divergence in async and locking paths that Principle III's rationale cites as having burned this project before. It would also silently pass if the worker accidentally depended on in-process state (e.g. a class-level fixture), which is the main class of bug the real lane exists to catch.
+- *Run the whole suite against a real worker* — rejected: a live process per example group is slow, and unnecessary for the majority of specs that never block a caller.
+
+## 7. Documentation scope
+
+**Decision**: `documentation/async_reactors.md`'s "Step-Level Async" section (lines ~85-190) is rewritten to describe both `background after:` and `background before:`, including which step each form pins; new sections are added for `async_step` and `async_reactor`. `documentation/composition.md` gets a short cross-reference distinguishing `async_reactor` (fire-and-forget, no compensation link) from `compose` (synchronous, fully compensation-linked). `README.md`'s "Step-Level Async" subsection (`README.md:386` onward) is rewritten the same way. `demo_app/documentation/` is a **duplicate copy** of the top-level `documentation/` folder (confirmed: the ActiveJob Support commit touched both `documentation/async_reactors.md` and `demo_app/documentation/async_reactors.md` in the same PR) and must be kept in sync — every documentation edit in this feature is applied to both copies.
+
+**Rationale**: Grounded directly in `git show --stat 0fb6dc4` showing both paths edited together for the prior feature; treating them as one logical file (edited twice) avoids drift.
+
+## 8. Linking async units into the context (and therefore the dashboard) — reuse `composed_contexts`, don't invent a parallel channel
+
+**Decision**: The *reference* to an `async_step`/`async_reactor` (as opposed to its eventual *result*, decision 2/3 above) is written into `context.composed_contexts[step_name]` — the exact same field `compose` and `map` already use (`Step::ComposeStep#store_child_context`, `lib/ruby_reactor/step/compose_step.rb:90-96`; `map`'s `:map_ref` entries) — tagged with a new `type:` (`:async_step_ref` or `:async_reactor_ref`) instead of `:composed`/`:map_ref`. This write happens synchronously in the *dispatching* step, before it returns — the same process that's about to keep running other steps, so there's no race (unlike the *result*, which a separate worker writes later — that still goes through the race-free channel from decisions 2/3).
+
+**Rationale**: An earlier pass of this plan put the async link in a brand-new, disconnected field, missing that `composed_contexts` is precisely "children of this context, referenced for later drill-down" and is *already* wired end-to-end: `Context` already serializes/deserializes it, `RubyReactor::Web::API.hydrate_composed_contexts` (`lib/ruby_reactor/web/api.rb:223-234`) already switches on `type:` to decide how to hydrate a reference for the dashboard (today: pass `:composed` through as-is, resolve `:map_ref` into a representative element context), and `RubyReactor::RSpec::TestSubject#composed`/`#map`/`#traverse_composed` (`lib/ruby_reactor/rspec/test_subject.rb:54-160`) already know how to traverse it in tests. Reusing this one field means: (a) FR-008's "linked... for traceability" and FR-014's "dashboard renders and drills into it" are satisfied by two small additions to code that already exists for exactly this purpose, rather than a new subsystem; (b) `async_reactor`'s link needs *no new storage primitive at all* — the child is a normal, fully addressable `Reactor` with its own context row, so hydrating its reference just calls the existing `storage.retrieve_context(execution_id, reactor_class_name)` / `ChildReactorClass.find(execution_id)`, the same as any other reactor lookup; only `async_step` (which has no reactor class/context of its own — it's one step's worth of work) still needs the dedicated `store_step_result`/`retrieve_step_result` bucket from decision 2 to hold its actual outcome, reached via the `composed_contexts` reference the same way `map_ref` reaches into the map-result bucket.
+
+**Consequence for data-model.md**: `composed_contexts[step_name]` entry shape:
+- `async_step_ref`: `{ type: :async_step_ref, name:, dispatched_at: }` — the actual result lives in the Step Result Record bucket (decision 2), looked up by `(context_id, step_name)`.
+- `async_reactor_ref`: `{ type: :async_reactor_ref, name:, execution_id:, reactor_class_name:, dispatched_at: }` — the actual result lives in that execution's own context row, looked up by `(execution_id, reactor_class_name)` via the existing `retrieve_context`/`Reactor.find`.
+
+**Web/GUI follow-on** (in scope for this feature, not a future nice-to-have — constitution Principle IV: "The web dashboard MUST remain current with the reactor state model"):
+- `Web::API.determine_step_type` (`api.rb:202-214`) currently ends with `elsif config.async? → "async"`, which breaks the moment `StepConfig#async?` is removed (decision "Removed" in contracts/public-dsl.md). It gains two new branches keyed off new argument keys the DSL macros will set (mirroring how `compose`/`map` are detected via `arguments.key?(:composed_reactor_class)` / `:mapped_reactor_class`): `async_step` (detected via a marker key on the step's arguments, e.g. `:async_step_marker`) and `async_reactor` (detected via `arguments.key?(:async_reactor_class)`, with `extract_inner_class`/`nested_structure` reused exactly as `compose`/`map` already do, so the dashboard can show the child reactor's own step graph).
+- `Web::API.build_structure`'s per-step `async: config.async?` field is removed (source gone) and not replaced — the reactor-level `background after:` point is exposed once, alongside `steps`, not per-step (implementation detail for tasks phase: e.g. `{ steps: {...}, background_after: reactor_class.background_after }`).
+- `Web::API.hydrate_composed_contexts` gains two branches paralleling `hydrate_map_ref`: one resolving `:async_step_ref` via `retrieve_step_result`, one resolving `:async_reactor_ref` via `retrieve_context` on the linked execution id.
+- `gui/src/components/DagVisualizer.tsx` and `StepInspector.tsx` read `struct[key].type` as a plain string (confirmed: `gui/src/components/DagVisualizer.tsx:341` branches on `struct[key].type === 'map'`) — they need matching cases added for `'async_step'`/`'async_reactor'` (icon/color/drill-down link), a frontend task-phase item, not a backend one.
+
+**Alternatives considered**:
+- *Log-only linkage (FR-012's structured logs alone)* — rejected: satisfies "traceability" loosely but not FR-008's stronger "linked... on the context" requirement or FR-014's dashboard-drill-down requirement; logs aren't queryable from the context object itself and depend on log retention.
+- *A wholly new `context.async_links` field, separate from `composed_contexts`* — rejected (this was the original, corrected design): duplicates a mechanism that already exists for exactly this purpose, and would require teaching `Web::API`/`TestSubject` a second traversal pattern instead of extending the one they already have.
+
+## 9. Exclusive locks across the async boundary — no shared ownership, dispatch-time deadlock guard (FR-015)
+
+**Decision** (2026-08-20 session): lock ownership is NOT extended across the `async_reactor` boundary. The existing owner-based reentrancy (`Lock` owner = root context id, `executor.rb:429-430` — "allow re-entrancy across nested reactors") continues to cover `compose`, and only `compose`. An `async_reactor` child keeps its own fresh context id as owner. The deadlock this leaves open — parent holds `lock(key)` for its entire execution (released in `execute`'s `ensure`, i.e. *after* any reader step's FR-005 wait), child declares `lock` resolving to the same key, child snoozes forever while parent waits on the child — is closed at **dispatch time**: the dispatching step resolves the child's `lock_config[:key_proc]` against the mapped child inputs and, if the resulting key matches a lock the dispatching execution currently holds (the executor's `@acquired_lock`, plus any held by its root chain), fails the dispatch step immediately with an error naming the key and both reactor classes.
+
+**Rationale**: Reentrancy is a property of a single logical thread of control. `compose` qualifies — the child executes *inline, sequentially* inside the parent's frame, so "the same execution re-entering its own lock" is sound. An `async_reactor` child is definitionally concurrent with its parent; giving it the parent's owner would let both sides sit inside the critical section simultaneously — mutual exclusion silently broken, data corruption instead of a visible stall. Between "loud, immediate, explained failure" and "silent guaranteed timeout" and "silently broken lock", the first is the only acceptable one. The codebase already has this exact judgment call in precedent: nested same-key *ordered*-lock invocation is detected at nonce-assignment time via `OrderedLockSupport.active_keys` and surfaced rather than left to deadlock (`reactor.rb:441-456`). The guard is best-effort by design: it catches the same-execution circular wait (the only *guaranteed* deadlock); ordinary cross-execution contention on the same key remains normal snooze-and-retry behavior, unchanged.
+
+**Scope notes**:
+- `async_step` needs no guard: its worker acquires no reactor-level locks at all (contract — async step work runs outside the parent's lock window).
+- Semaphores with `limit: 1` have the same circular-wait shape; the guard checks the held semaphore key too when its limit is 1. Higher limits are contention, not guaranteed deadlock — left to snooze.
+- Users who need the child inside the parent's critical section have an existing, correct tool: `compose`. The error message says so.
+
+**Alternatives considered**:
+- *Reentrant across the async boundary (share the parent's owner)* — rejected: breaks mutual exclusion under true concurrency (both executions inside the critical section at once); also wrecks release/auto-extend accounting — the parent's `ensure` release would decrement/free the lock while the child still believes it holds it.
+- *Do nothing, rely on the FR-005 timeout* — rejected: turns a statically-detectable programming error into a runtime stall of `async_wait_timeout` seconds followed by an opaque timeout failure, burning a worker slot on the snoozing child the whole time.
+- *Queue the dispatch until the parent releases* — rejected: hidden sequencing that contradicts `async_reactor`'s "runs concurrently, fire-and-forget" contract, and unbounded dispatch deferral is its own liveness hazard.
+
+## 10. Recovery re-attach and parent-context lifetime (FR-017, FR-018)
+
+**Decision**: The durable dispatch marker written *before* enqueue (decision 2's Step Result Record at status `dispatched`; decision 3/8's `composed_contexts[:name] = { type: :async_reactor_ref, execution_id:, ... }`) is also the re-attach marker. On recovery/resume, `StepExecutor`'s async dispatch path checks for that marker first and, if present, skips enqueue entirely and marks the node graph-complete exactly as the original dispatch did — the pending work is already out there with its own durable identity, so re-dispatching would duplicate a side effect (a second email sent, a second child reactor execution) for no benefit. Readers are unaffected: they wait on the same record/child row either way.
+
+**Rationale**: This falls out of the checkpoint-before-enqueue ordering already required by F2 — the marker exists precisely so a crash can never leave a job with no record, and "record exists" is therefore exactly the question "was this already dispatched?". No new state and no new storage primitive; a presence check in the dispatch path. The alternative (re-dispatch on recovery) is the classic at-least-once duplicate side effect, and unlike an ordinary retried step an `async_step` has no compensation wired into the parent to clean it up.
+
+**Parent-context lifetime**: the async worker loads the parent context by id, so the parent's `context_ttl` must outlive the dispatched unit — including the fire-and-forget case where the parent completes immediately and nothing ever waits. Dispatch therefore refreshes the parent context's TTL (and stamps the Step Result Record with the same window, per data-model.md). A worker that still finds no parent context — swept, or dispatched longer ago than the retention window — writes a `completed`/`Failure` record for its unit and logs it per FR-012 instead of raising: the unit's work cannot be resolved without its parent's inputs, and an unhandled raise would just hand it to the backend's retry machinery to fail identically N more times. A reader that later asks for that result gets a `Failure` explaining the expiry, which is strictly better than a 30s timeout.
+
+**Alternatives considered**:
+- *A separate "in-flight async units" registry consulted on recovery* — rejected: duplicates what the dispatch marker already records, and adds a second thing that can disagree with the first.
+- *Give async result records their own longer TTL than the parent context* — rejected: the worker needs the parent context, not just the record, so a longer record TTL alone fixes nothing; refreshing the parent's TTL at dispatch is the single lever that actually covers the window.
+- *Block parent completion until dispatched units finish* — rejected outright: that is exactly the fire-and-forget contract (FR-018, spec Edge Cases) inverted, and would make `async_step` behave like a synchronous step.