RubyGems - ruby_reactor - Versions diffs - 0.5.4 → 0.7.0 - Mend

ruby_reactor 0.5.4 → 0.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (128) hide show

checksums.yaml +4 -4
data/.claude/skills/speckit-agent-context-update/SKILL.md +32 -0
data/.claude/skills/speckit-analyze/SKILL.md +262 -0
data/.claude/skills/speckit-checklist/SKILL.md +374 -0
data/.claude/skills/speckit-clarify/SKILL.md +286 -0
data/.claude/skills/speckit-constitution/SKILL.md +157 -0
data/.claude/skills/speckit-converge/SKILL.md +277 -0
data/.claude/skills/speckit-implement/SKILL.md +224 -0
data/.claude/skills/speckit-plan/SKILL.md +171 -0
data/.claude/skills/speckit-specify/SKILL.md +346 -0
data/.claude/skills/speckit-tasks/SKILL.md +215 -0
data/.claude/skills/speckit-taskstoissues/SKILL.md +110 -0
data/.release-please-manifest.json +1 -1
data/.specify/extensions/.registry +19 -0
data/.specify/extensions/agent-context/README.md +66 -0
data/.specify/extensions/agent-context/agent-context-config.yml +5 -0
data/.specify/extensions/agent-context/commands/speckit.agent-context.update.md +27 -0
data/.specify/extensions/agent-context/extension.yml +34 -0
data/.specify/extensions/agent-context/scripts/bash/update-agent-context.sh +282 -0
data/.specify/extensions/agent-context/scripts/powershell/update-agent-context.ps1 +353 -0
data/.specify/extensions.yml +23 -0
data/.specify/feature.json +3 -0
data/.specify/init-options.json +9 -0
data/.specify/integration.json +15 -0
data/.specify/integrations/claude.manifest.json +17 -0
data/.specify/integrations/speckit.manifest.json +17 -0
data/.specify/memory/constitution.md +134 -0
data/.specify/scripts/bash/check-prerequisites.sh +189 -0
data/.specify/scripts/bash/common.sh +619 -0
data/.specify/scripts/bash/create-new-feature.sh +299 -0
data/.specify/scripts/bash/setup-plan.sh +84 -0
data/.specify/scripts/bash/setup-tasks.sh +91 -0
data/.specify/templates/checklist-template.md +40 -0
data/.specify/templates/constitution-template.md +50 -0
data/.specify/templates/plan-template.md +113 -0
data/.specify/templates/spec-template.md +131 -0
data/.specify/templates/tasks-template.md +252 -0
data/.specify/workflows/speckit/workflow.yml +77 -0
data/.specify/workflows/workflow-registry.json +13 -0
data/CHANGELOG.md +121 -1
data/CLAUDE.md +5 -0
data/README.md +181 -39
data/lib/ruby_reactor/adapters/active_job/compat.rb +24 -0
data/lib/ruby_reactor/adapters/active_job/map_collector_worker.rb +19 -0
data/lib/ruby_reactor/adapters/active_job/map_element_worker.rb +19 -0
data/lib/ruby_reactor/adapters/active_job/router.rb +106 -0
data/lib/ruby_reactor/adapters/active_job/step_worker.rb +20 -0
data/lib/ruby_reactor/adapters/active_job/sweeper_worker.rb +16 -0
data/lib/ruby_reactor/adapters/active_job/worker.rb +29 -0
data/lib/ruby_reactor/adapters/sidekiq/map_collector_worker.rb +15 -0
data/lib/ruby_reactor/adapters/sidekiq/map_element_worker.rb +15 -0
data/lib/ruby_reactor/adapters/sidekiq/router.rb +106 -0
data/lib/ruby_reactor/adapters/sidekiq/step_worker.rb +17 -0
data/lib/ruby_reactor/adapters/sidekiq/sweeper_worker.rb +19 -0
data/lib/ruby_reactor/adapters/sidekiq/worker.rb +28 -0
data/lib/ruby_reactor/async_waiter.rb +113 -0
data/lib/ruby_reactor/configuration.rb +54 -5
data/lib/ruby_reactor/context.rb +21 -0
data/lib/ruby_reactor/context_serializer.rb +2 -0
data/lib/ruby_reactor/dsl/async_macros.rb +176 -0
data/lib/ruby_reactor/dsl/async_reactor_builder.rb +66 -0
data/lib/ruby_reactor/dsl/compose_builder.rb +23 -4
data/lib/ruby_reactor/dsl/interrupt_builder.rb +3 -2
data/lib/ruby_reactor/dsl/interrupt_step_config.rb +7 -1
data/lib/ruby_reactor/dsl/lockable.rb +1 -1
data/lib/ruby_reactor/dsl/map_builder.rb +1 -2
data/lib/ruby_reactor/dsl/reactor.rb +29 -10
data/lib/ruby_reactor/dsl/step_builder.rb +30 -11
data/lib/ruby_reactor/error/async_result_pending.rb +21 -0
data/lib/ruby_reactor/error/async_wait_timeout_error.rb +10 -0
data/lib/ruby_reactor/error/deprecated_dsl_error.rb +11 -0
data/lib/ruby_reactor/executor/async_step_dispatch.rb +110 -0
data/lib/ruby_reactor/executor/ordered_lock_support.rb +2 -2
data/lib/ruby_reactor/executor/result_handler.rb +12 -1
data/lib/ruby_reactor/executor/retry_manager.rb +9 -5
data/lib/ruby_reactor/executor/step_executor.rb +58 -11
data/lib/ruby_reactor/executor.rb +162 -8
data/lib/ruby_reactor/lock.rb +19 -0
data/lib/ruby_reactor/map/element_executor.rb +7 -2
data/lib/ruby_reactor/map/helpers.rb +1 -1
data/lib/ruby_reactor/map/result_enumerator.rb +5 -1
data/lib/ruby_reactor/map/result_summary.rb +63 -0
data/lib/ruby_reactor/open_telemetry.rb +1 -1
data/lib/ruby_reactor/reactor.rb +25 -2
data/lib/ruby_reactor/rspec/active_job_helpers.rb +52 -0
data/lib/ruby_reactor/rspec/async_test_helpers.rb +41 -0
data/lib/ruby_reactor/rspec/sidekiq_helpers.rb +4 -3
data/lib/ruby_reactor/rspec/step_executor_patch.rb +2 -2
data/lib/ruby_reactor/rspec/test_subject.rb +77 -18
data/lib/ruby_reactor/rspec.rb +4 -0
data/lib/ruby_reactor/semaphore.rb +10 -0
data/lib/ruby_reactor/step/async_reactor_step.rb +207 -0
data/lib/ruby_reactor/step/compose_step.rb +1 -1
data/lib/ruby_reactor/step/map_step.rb +1 -1
data/lib/ruby_reactor/step_sweeper.rb +72 -0
data/lib/ruby_reactor/step_worker.rb +260 -0
data/lib/ruby_reactor/storage/adapter.rb +22 -1
data/lib/ruby_reactor/storage/redis_adapter.rb +17 -13
data/lib/ruby_reactor/storage/redis_locking.rb +7 -0
data/lib/ruby_reactor/storage/redis_pub_sub.rb +31 -0
data/lib/ruby_reactor/storage/redis_step_results.rb +49 -0
data/lib/ruby_reactor/sweeper.rb +7 -1
data/lib/ruby_reactor/sweeper_job.rb +71 -0
data/lib/ruby_reactor/template/result.rb +151 -5
data/lib/ruby_reactor/version.rb +1 -1
data/lib/ruby_reactor/web/api.rb +94 -16
data/lib/ruby_reactor/web/public/assets/index-B46p-M6K.css +1 -0
data/lib/ruby_reactor/web/public/assets/index-DPmP4yXT.js +22 -0
data/lib/ruby_reactor/web/public/index.html +2 -2
data/lib/ruby_reactor/worker.rb +276 -0
data/lib/ruby_reactor.rb +64 -4
data/specs/001-background-async-steps/checklists/requirements.md +39 -0
data/specs/001-background-async-steps/contracts/public-dsl.md +154 -0
data/specs/001-background-async-steps/data-model.md +117 -0
data/specs/001-background-async-steps/plan.md +168 -0
data/specs/001-background-async-steps/quickstart.md +102 -0
data/specs/001-background-async-steps/research.md +150 -0
data/specs/001-background-async-steps/spec.md +146 -0
data/specs/001-background-async-steps/tasks.md +271 -0
data/specs/active_job.md +259 -0
metadata +80 -8
data/lib/ruby_reactor/sidekiq_adapter.rb +0 -87
data/lib/ruby_reactor/sidekiq_workers/map_collector_worker.rb +0 -13
data/lib/ruby_reactor/sidekiq_workers/map_element_worker.rb +0 -13
data/lib/ruby_reactor/sidekiq_workers/sweeper_worker.rb +0 -73
data/lib/ruby_reactor/sidekiq_workers/worker.rb +0 -222
data/lib/ruby_reactor/web/public/assets/index-CCnNVQy5.css +0 -1
data/lib/ruby_reactor/web/public/assets/index-D7IBZvos.js +0 -21

data/lib/ruby_reactor/web/public/index.html CHANGED Viewed

@@ -5,8 +5,8 @@
     <link rel="icon" type="image/svg+xml" href="./vite.svg" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
     <title>ui</title>
-    <script type="module" crossorigin src="./assets/index-D7IBZvos.js"></script>
-    <link rel="stylesheet" crossorigin href="./assets/index-CCnNVQy5.css">
+    <script type="module" crossorigin src="./assets/index-DPmP4yXT.js"></script>
+    <link rel="stylesheet" crossorigin href="./assets/index-B46p-M6K.css">
   </head>
   <body>
     <div id="root"></div>

data/lib/ruby_reactor/worker.rb ADDED Viewed

@@ -0,0 +1,276 @@
+# frozen_string_literal: true
+module RubyReactor
+  # Framework-agnostic resume/snooze/escalate logic shared by every queueing
+  # backend's worker class. Each backend (`Adapters::Sidekiq::Worker`,
+  # `Adapters::ActiveJob::Worker`, ...) includes this and supplies its own
+  # `self.class.perform_in` (native on Sidekiq::Worker, via
+  # `Adapters::ActiveJob::Compat` on ActiveJob::Base) — nothing here references
+  # a specific backend.
+  module Worker
+    TERMINAL_STATUSES = %w[completed failed cancelled skipped].freeze
+    # Last line of observability when a job burns its whole retry budget on an
+    # infrastructure failure and the backend then discards it (Sidekiq runs
+    # with `dead: false`): without this, the context would stay "running"
+    # forever with zero surface anywhere, and every reader would wait out its
+    # full timeout. Called from the backends' retries-exhausted hooks with the
+    # job's own args. Best-effort — never raises back into the backend.
+    def self.record_retries_exhausted(args, exception)
+      context_id, reactor_class_name = args
+      return unless context_id
+      reactor_class_name ||= RubyReactor.reactor_storage_name(nil)
+      storage = RubyReactor.configuration.storage_adapter
+      data = storage.retrieve_context(context_id, reactor_class_name)
+      return if data.nil? || TERMINAL_STATUSES.include?((data["status"] || data[:status]).to_s)
+      data["status"] = "failed"
+      data["failure_reason"] = {
+        "message" => "job retries exhausted: #{exception.class.name}: #{exception.message}",
+        "exception_class" => exception.class.name
+      }
+      storage.store_context(context_id, JSON.generate(data), reactor_class_name)
+      storage.publish(RubyReactor.async_reactor_channel(context_id), "failed")
+    rescue StandardError => e
+      RubyReactor.configuration.logger.error(
+        "RubyReactor: could not record retries-exhausted failure for #{context_id}: #{e.class.name}: #{e.message}"
+      )
+    end
+    # Identity-only payload: storage is the source of truth. Rehydrate the live
+    # context from storage by id, then resume. A nil read means the context was
+    # swept, expired, or already terminal-and-collected — nothing to resume.
+    def perform(context_id, reactor_class_name = nil, snooze_count = 0)
+      # Normalize so a nil/omitted name resolves to the same storage key the
+      # enqueue path wrote (always via reactor_storage_name). Without this a
+      # nil here builds "reactor::context:<id>" and misses the stored
+      # "reactor:AnonymousReactor:context:<id>", silently no-op'ing.
+      reactor_class_name ||= RubyReactor.reactor_storage_name(nil)
+      data = RubyReactor.configuration.storage_adapter.retrieve_context(context_id, reactor_class_name)
+      return if data.nil?
+      begin
+        context = ContextSerializer.deserialize_hash(data)
+      rescue RubyReactor::Error::DeserializationError,
+             RubyReactor::Error::SchemaVersionError => e
+        # Permanent failures — re-reading the same stored blob will keep
+        # failing. Mark the context as failed (best-effort) and return so
+        # the job does not burn its retry budget.
+        handle_deserialization_failure(context_id, reactor_class_name, e)
+        return
+      end
+      resolve_reactor_class!(context, reactor_class_name)
+      unless context.reactor_class
+        # Still unresolved (class not loaded, or an anonymous reactor with no
+        # storage name to look up) — Executor.new below would blow up on a nil
+        # class and burn the job's retry budget forever. Fail the context now.
+        error = RubyReactor::Error::DeserializationError.new(
+          "reactor class '#{reactor_class_name}' could not be resolved"
+        )
+        handle_deserialization_failure(context_id, reactor_class_name, error)
+        return
+      end
+      # Mark that we're executing inline to prevent nested async calls
+      context.inline_async_execution = true
+      begin
+        # Resume execution from the failed step
+        executor = Executor.new(context.reactor_class, {}, context)
+        executor.resume_execution
+        # No explicit save here: resume_execution's ensure block already persists
+        # the final root state (`save_context unless skip_context_persist?`), and
+        # in the worker the executor's context IS the root, so an extra checkpoint!
+        # would just re-write the identical blob to the identical key. The
+        # skip_context_persist? guard (stale-batch redelivery of an already-terminal
+        # context) is likewise honored there.
+        # Return the executor (which now has the result stored in it)
+        executor
+      rescue RubyReactor::Lock::AcquisitionError,
+             RubyReactor::Semaphore::AcquisitionError,
+             RubyReactor::RateLimit::ExceededError,
+             RubyReactor::OrderedLock::WaitError,
+             RubyReactor::Error::AsyncResultPending => e
+        # Snooze on expected concurrency, rate, or ordering contention.
+        # OrderedLock::WaitError carries a poison-pill-derived retry hint,
+        # consumed by compute_snooze_delay below. We avoid the framework's native
+        # retry path so this doesn't burn the job's retry budget or appear
+        # as an error in dashboards. After the configured cap is reached we
+        # escalate by marking the reactor as failed.
+        handle_snooze(context_id, reactor_class_name, context, snooze_count, e)
+      rescue RubyReactor::RateLimitRegistry::UnknownLimitError => e
+        # Permanent configuration error — snoozing or retrying the same job
+        # will keep failing. Mark the context failed immediately.
+        escalate_snooze(context, snooze_count, e)
+      end
+    end
+    private
+    # If reactor_class_name is provided, use it to get the reactor class.
+    # This handles cases where the class can't be found via const_get.
+    def resolve_reactor_class!(context, reactor_class_name)
+      return unless reactor_class_name && context.reactor_class.nil?
+      begin
+        context.reactor_class = Object.const_get(reactor_class_name)
+      rescue NameError
+        # If not found, try to find it in the current namespace
+        # This is a fallback for test environments
+        begin
+          context.reactor_class = reactor_class_name.constantize if reactor_class_name.respond_to?(:constantize)
+        rescue NameError
+          # Leave reactor_class nil: the caller's guard fails the context with
+          # a durable record. Letting this second NameError escape would burn
+          # the job's whole retry budget on an error retries can never fix,
+          # then vanish — leaving the context "running" forever and any reader
+          # waiting out its full timeout.
+          nil
+        end
+      end
+    end
+    def handle_snooze(context_id, reactor_class_name, context, snooze_count, error)
+      config = RubyReactor.configuration
+      max = config.lock_snooze_max_attempts
+      # OrderedLock::WaitError bypasses the snooze cap. The gate's
+      # poison_pill_timeout is the only meaningful upper bound on how long a
+      # nonce can legitimately wait; capping snoozes would either fail jobs
+      # prematurely or strand the nonce in `assigned_at` until poison_pill
+      # eventually advances past it. Snooze until the gate passes (or poison
+      # auto-advance moves the cursor past us).
+      # The per-context liveness lock (`async:<id>`) is also uncapped: a
+      # duplicate of the *same* execution may wait arbitrarily long for the
+      # live original to finish (e.g. a sweeper re-enqueue racing a slow but
+      # alive worker). Capping it would fail a legitimately-waiting duplicate.
+      # A parked async wait is likewise uncapped HERE: its bound is
+      # `async_park_timeout`, enforced against `dispatched_at` at the wait
+      # site — counting snoozes would double-bound it with the wrong unit.
+      capped = !(error.is_a?(RubyReactor::OrderedLock::WaitError) ||
+                 error.is_a?(RubyReactor::Lock::ContextLockContention) ||
+                 error.is_a?(RubyReactor::Error::AsyncResultPending))
+      if capped && max != :infinity && snooze_count >= max
+        escalate_snooze(context, snooze_count, error)
+        return
+      end
+      delay = compute_snooze_delay(config, error)
+      # Re-enqueue by id: the context is already persisted in storage, so the
+      # rescheduled job rehydrates fresh state (no stale blob).
+      self.class.perform_in(delay, context_id, reactor_class_name, snooze_count + 1)
+    end
+    # Use the error's `retry_after_seconds` hint when available
+    # (RateLimit::ExceededError carries the time until the bucket rolls);
+    # otherwise fall back to the configured base + jitter for lock/semaphore
+    # contention which has no precise hint.
+    #
+    # OrderedLock::WaitError is deliberately excluded from the hint path: its
+    # `retry_after_seconds` is the poison-pill window (the upper bound before
+    # a *dead* blocker is force-advanced), NOT how long the *live* blocker
+    # will take — which is usually milliseconds. Snoozing for the full window
+    # would make every out-of-order nonce sleep up to poison_pill_timeout even
+    # though its blocker finishes immediately, collapsing throughput. Re-poll
+    # at the base delay instead; poison auto-advance still clears a genuinely
+    # dead blocker on a later gate.
+    def compute_snooze_delay(config, error)
+      jitter = config.lock_snooze_jitter.to_f
+      jitter_amount = jitter.positive? ? rand(0.0..jitter) : 0.0
+      if hinted_retry?(error)
+        [error.retry_after_seconds.to_f, 0.1].max + jitter_amount
+      else
+        config.lock_snooze_base_delay.to_f + jitter_amount
+      end
+    end
+    def hinted_retry?(error)
+      return false if error.is_a?(RubyReactor::OrderedLock::WaitError)
+      error.respond_to?(:retry_after_seconds) && error.retry_after_seconds
+    end
+    def escalate_snooze(context, snooze_count, error)
+      RubyReactor.configuration.logger.warn(
+        "RubyReactor snooze limit reached after #{snooze_count} attempts " \
+        "for context #{context.context_id}: #{error.message}"
+      )
+      context.status = :failed
+      context.failure_reason = {
+        message: error.message,
+        exception_class: error.class.name,
+        snooze_attempts: snooze_count
+      }
+      serialized = ContextSerializer.serialize(context)
+      reactor_class_name = RubyReactor.reactor_storage_name(context.reactor_class)
+      RubyReactor.configuration.storage_adapter.store_context(
+        context.context_id,
+        serialized,
+        reactor_class_name
+      )
+      # Escalation is a terminal Failure that never reaches the Executor's
+      # ensure path, so advance the ordered-lock cursor here. Without this
+      # the nonce stays stranded in assigned_at (successors stall for the
+      # full poison_pill_timeout) and, worse, the strict-mode chain marker
+      # is never recorded — successors would RUN instead of being skipped.
+      info = Executor::OrderedLockSupport.info_from(context)
+      Executor::OrderedLockSupport.advance_with_retry(info, failed: true) if info
+    end
+    def log_infrastructure_failure(msg, exception)
+      RubyReactor.configuration.logger.error("RubyReactor infrastructure failure: #{exception.message}")
+      RubyReactor.configuration.logger.error("Job details: #{msg.inspect}")
+    end
+    # The id-only payload already carries context_id and reactor_class_name, so
+    # there is no blob to parse for metadata — just mark the stored context
+    # failed (best-effort) so the job stops retrying a permanently-broken blob.
+    def handle_deserialization_failure(context_id, reactor_class_name, error)
+      RubyReactor.configuration.logger.error(
+        "RubyReactor deserialization failure for context " \
+        "#{context_id || "unknown"}: #{error.class.name}: #{error.message}"
+      )
+      return unless context_id && reactor_class_name
+      payload = build_failed_context_payload(context_id, reactor_class_name, error)
+      RubyReactor.configuration.storage_adapter.store_context(
+        context_id,
+        payload,
+        reactor_class_name
+      )
+      # Written first, signalled second — wake any reader blocked (or parked)
+      # on this execution so it fails fast with the real cause instead of
+      # waiting out its timeout.
+      RubyReactor.configuration.storage_adapter.publish(
+        RubyReactor.async_reactor_channel(context_id), "failed"
+      )
+    rescue StandardError => e
+      # Don't let a persistence failure mask the original deserialization error.
+      RubyReactor.configuration.logger.error(
+        "RubyReactor failed to persist deserialization failure: #{e.class.name}: #{e.message}"
+      )
+    end
+    def build_failed_context_payload(context_id, reactor_class_name, error)
+      JSON.generate(
+        "schema_version" => ContextSerializer::SCHEMA_VERSION,
+        "context_id" => context_id,
+        "reactor_class" => reactor_class_name,
+        "status" => "failed",
+        "failure_reason" => {
+          "message" => error.message,
+          "exception_class" => error.class.name
+        }
+      )
+    end
+  end
+end

data/lib/ruby_reactor.rb CHANGED Viewed

@@ -3,6 +3,7 @@
 require "zeitwerk"
 require "pathname"
 require "securerandom"
+require "time"
 require_relative "ruby_reactor/registry"
 require_relative "ruby_reactor/utils/code_extractor"
 require_relative "ruby_reactor/dsl/lockable" # Add this
@@ -26,6 +27,13 @@ rescue LoadError
   # sidekiq is optional, async features won't be available
 end
+# Load active_job if available (for the ActiveJob async adapter)
+begin
+  require "active_job"
+rescue LoadError
+  # active_job is optional, only needed when using the ActiveJob adapter
+end
 loader = Zeitwerk::Loader.for_gem
 loader.inflector.inflect("api" => "API", "rspec" => "RSpec")
 loader.setup
@@ -285,8 +293,10 @@ module RubyReactor
     end
   end
-  # Async result for background job execution
-  class AsyncResult
+  # Sentinel returned when a step's work is handed off to a worker job and is
+  # not yet resolved — produced by `background`, `async_step`, `async_reactor`,
+  # and map's async element dispatch alike.
+  class DispatchResult
     attr_reader :job_id, :intermediate_results, :execution_id
     def initialize(job_id:, intermediate_results: {}, execution_id: nil)
@@ -341,6 +351,23 @@ module RubyReactor
   # (A truly anonymous class can't be reconstituted by name in another process,
   # so cross-process resume of one is inherently unsupported; this only keeps
   # the keys self-consistent within a process — e.g. inline tests.)
+  # Completion-signal channels for the notified wait. One place, because
+  # the publishing side and the waiting side must agree exactly and they live in
+  # different files (StepWorker / Executor vs Template::Result).
+  def self.async_step_channel(context_id, step_name)
+    "rr:done:#{context_id}:#{step_name}"
+  end
+  def self.async_reactor_channel(execution_id)
+    "rr:done:#{execution_id}"
+  end
+  # Liveness lock for one dispatched `async_step`. Held by StepWorker for the
+  # life of the unit, so StepSweeper can tell a slow unit from a lost job.
+  def self.async_step_lock_key(context_id, step_name)
+    "async_step:#{context_id}:#{step_name}"
+  end
   def self.reactor_storage_name(reactor_class)
     return "AnonymousReactor" if reactor_class.nil?
@@ -356,7 +383,39 @@ module RubyReactor
   def self.start_sweeper!
     return unless configuration.sweeper_enabled
-    SidekiqWorkers::SweeperWorker.schedule_next
+    sweeper_job_class.schedule_next
+  end
+  # The sweeper job class living alongside the configured `async_router`
+  # (e.g. `Adapters::Sidekiq::Router` -> `Adapters::Sidekiq::SweeperWorker`),
+  # so the chain is kicked through whichever backend is configured instead of
+  # a hardcoded Sidekiq class. Known built-in routers are mapped explicitly;
+  # a custom `configuration.async_router` falls back to a sibling-namespace
+  # lookup, with a clear error (instead of a bare NameError) when that
+  # convention doesn't hold.
+  def self.sweeper_job_class
+    router = configuration.async_router
+    case router.name
+    when "RubyReactor::Adapters::Sidekiq::Router" then Adapters::Sidekiq::SweeperWorker
+    when "RubyReactor::Adapters::ActiveJob::Router" then Adapters::ActiveJob::SweeperWorker
+    else
+      inferred_sweeper_job_class(router)
+    end
+  end
+  def self.inferred_sweeper_job_class(router)
+    namespace_name = router.name.to_s.rpartition("::").first
+    raise sweeper_job_class_error(router) if namespace_name.empty?
+    Object.const_get(namespace_name).const_get(:SweeperWorker)
+  rescue NameError
+    raise sweeper_job_class_error(router)
+  end
+  def self.sweeper_job_class_error(router)
+    "RubyReactor: cannot infer a sweeper job class for custom async_router " \
+      "#{router.inspect}. Define a `SweeperWorker` class alongside it " \
+      "(same namespace), or override `RubyReactor.sweeper_job_class`."
   end
   # Run both recovery sweepers exactly once and return their counts. The
@@ -368,7 +427,8 @@ module RubyReactor
     limit ||= configuration.sweeper_limit
     {
       reactors: Sweeper.run_once(limit: limit),
-      maps: Map::Sweeper.run_once(limit: limit)
+      maps: Map::Sweeper.run_once(limit: limit),
+      async_steps: StepSweeper.run_once(limit: limit)
     }
   end

data/specs/001-background-async-steps/checklists/requirements.md ADDED Viewed

@@ -0,0 +1,39 @@
+# Specification Quality Checklist: Background Execution & Real Async Steps
+**Purpose**: Validate specification completeness and quality before proceeding to planning
+**Created**: 2026-08-16
+**Feature**: [spec.md](../spec.md)
+## Content Quality
+- [x] No implementation details (languages, frameworks, APIs)
+- [x] Focused on user value and business needs
+- [x] Written for non-technical stakeholders
+- [x] All mandatory sections completed
+## Requirement Completeness
+- [x] No [NEEDS CLARIFICATION] markers remain
+- [x] Requirements are testable and unambiguous
+- [x] Success criteria are measurable
+- [x] Success criteria are technology-agnostic (no implementation details)
+- [x] All acceptance scenarios are defined
+- [x] Edge cases are identified
+- [x] Scope is clearly bounded
+- [x] Dependencies and assumptions identified
+## Feature Readiness
+- [x] All functional requirements have clear acceptance criteria
+- [x] User scenarios cover primary flows
+- [x] Feature meets measurable outcomes defined in Success Criteria
+- [x] No implementation details leak into specification
+## Notes
+- Items marked incomplete require spec updates before `/speckit-clarify` or `/speckit-plan`.
+- Note: the spec names concrete terms (`Sidekiq`, `ActiveJob`, `Redis`) only inside the **Assumptions** section, where they describe existing constraints already codified in the project constitution, not new implementation choices being introduced by this feature. All requirement- and success-criteria-level language remains technology-agnostic.
+- 2026-08-16 (post-plan review): added FR-014/SC-006 covering dashboard visibility for `async_step`/`async_reactor`, a gap found while reviewing plan.md against the constitution's Observability principle. Re-validated against this checklist — all items still pass.
+- 2026-08-21 (final consistency pass): fixed stale "blocking-poll" wording left from the superseded wait design (plan Summary/tree, contract, data-model); fixed the async_step dispatch ordering in data-model (record + ref written BEFORE enqueue, matching the F2 rule its own Step Result Record row states); fixed "record absent = pending" to "record still `dispatched` = pending"; reconciled plan Scale/Scope with the tree (dependency_graph.rb unchanged; executor.rb added for the completion-signal publish); aligned FR-015 with contract/research on single-slot semaphores. All items re-verified passing.
+- 2026-08-20 (wait-mechanism & locks review): FR-005 revised from tight-poll to notified wait (record-first publish, subscribe-first check, fallback re-check — durable record stays the source of truth, timeout bound unchanged); added FR-015 (no lock-owner sharing across the async boundary + dispatch-time same-key deadlock guard; reentrancy stays compose-only) and FR-016 (async_reactor dispatch reuses the full pre-enqueue sequence: child input validation + ordered-lock nonce, a gap found during the locks review). Clarifications Session 2026-08-20 records both decisions. All checklist items re-verified passing.
+- 2026-08-16 (full-artifact review): fixed FR-006 (claimed a nonexistent "existing step-result storage"; now: new per-step record for `async_step`, own execution record for `async_reactor`); clarified FR-003 scope (compose's `async` flag is removed too — it sets the same flag; map's `async` is a different mechanism and stays); added edge cases for `returns` × async unit, `background` × whole-reactor `async true`, and a paused awaited `async_reactor` child. Contract additionally pins failure-read semantics for `result()` on async units and lock-window behavior. All checklist items re-verified passing.

data/specs/001-background-async-steps/contracts/public-dsl.md ADDED Viewed

@@ -0,0 +1,154 @@
+# Contract: Public DSL Surface
+This gem's "interface" is its Ruby DSL, consumed by host applications that subclass `RubyReactor::Reactor`. This contract documents the exact public surface this feature adds, changes, or removes, so implementation and tests can be checked against it directly.
+## Removed
+```ruby
+class MyReactor < RubyReactor::Reactor
+  step :do_thing do
+    async true   # REMOVED — raises a definition-time error (FR-003)
+  end
+end
+```
+```ruby
+class MyReactor < RubyReactor::Reactor
+  compose :sub_flow, SubReactor do
+    async true   # ALSO REMOVED — same StepConfig hand-off flag, same definition-time error
+  end
+end
+```
+**Error contract**: attempting to call `async` inside a `step` **or `compose`** block MUST raise a `RubyReactor::Error::ValidationError` (or a new dedicated error subclass — implementation's choice, but it MUST be raised at reactor **class-definition** time, not at `run` time) whose message names both the removed syntax and its replacement(s): `background after:`/`background before:`, `async_step`, `async_reactor`. (`ComposeBuilder#async` sets the very `StepConfig` flag this feature removes — `dsl/compose_builder.rb:62` — so it cannot survive; the exact migration for a compose that handed off is `background before: :<that compose step>`, which reproduces the old semantics precisely — the flagged step and everything after it moved to the worker — without the author having to identify a predecessor.)
+**Not removed**: the `async` option inside a `map` block (`MapBuilder#async`, `dsl/map_builder.rb:43`) is a map-internal element-dispatch mode passed as a step *argument* — the map's own `StepConfig` is hardcoded `async: false` (`map_builder.rb:111`), so it does not touch the removed flag and keeps working unchanged.
+## Added: `background after:` / `background before:`
+```ruby
+class MyReactor < RubyReactor::Reactor
+  step :first
+  step :second
+  background after: :second     # :second is the LAST step in the calling process
+  step :third
+end
+class EquivalentInLinearFlow < RubyReactor::Reactor
+  step :first
+  step :second
+  background before: :third     # :third is the FIRST step in the worker
+  step :third
+end
+```
+- **Signature**: `self.background(after: nil, before: nil)` — reactor class macro, exactly one keyword supplied.
+- **The two forms** name one cut point from opposite sides, and each carries a *guarantee about the step it names*:
+  - `after: :x` — `:x` runs in the calling process, and is the last step to do so.
+  - `before: :x` — `:x` runs in the worker, and is the first step to do so; it never executes in the calling process.
+  In a linear reactor where `:third` immediately follows `:second`, `after: :second` and `before: :third` are equivalent. In a branching workflow they are not, and the author picks whichever step they actually need pinned.
+- **Constraints** (all raise at class-definition time, FR-002):
+  - at most one `background` declaration per reactor class;
+  - the named step must be defined in the class (either keyword);
+  - the named step must not be an `interrupt` (either keyword) — an interrupt re-enters the reactor from a foreground process, so `after:` would silently never fire on resume and `before:` would enqueue a worker that instantly pauses and swallows the `InterruptResult`; the error points to `interrupt :name, resume: :background` as the correct spelling;
+  - exactly one of `after:`/`before:` — supplying both, or neither, raises;
+  - combining `background` with whole-reactor `async true` raises (the hand-off point would be silently meaningless inside a reactor that already runs entirely in a worker — see spec Edge Cases).
+- **Runtime contract**: hand-off is triggered by *reaching the named step*, not by the declaration's lexical position — the declaration may sit anywhere in the class body. For `after: :x`, the trigger fires when `:x` completes; for `before: :x`, it fires when `:x` is selected to run, and `:x` is left unexecuted for the worker to run. Either way: checkpoint, enqueue the remainder via `configuration.async_router`, return an `DispatchResult` to the caller. In a DAG with parallel branches, any independent step that became ready and executed before the trigger fired has already run in the calling process; everything not yet executed at the trigger moment runs in the worker — this caveat is identical for both forms. Compensation for worker-side step failures works exactly as it does for any same-process step failure — `background` only changes *where* code runs, not the saga/compensation contract (US1 acceptance scenarios 1-2). Inside the worker the hand-off never re-triggers (the existing `inline_async_execution` guard).
+- **Never-reached trigger**: if the named step is skipped by a `where`/guard condition, or the reactor fails before reaching it, the hand-off simply never fires and the run completes in the calling process. No step is stranded — the hand-off only ever relocates *remaining* work.
+- **`before:` naming the first step** is legal, and is not the same as whole-reactor `async true`: every step body runs in the worker, but input validation still happens in the calling process, so invalid inputs fail the caller synchronously instead of failing inside a worker.
+## Added: `async_step`
+```ruby
+class MyReactor < RubyReactor::Reactor
+  async_step :send_email do
+    argument :to, input(:email)
+    run { |args| Mailer.send(args[:to]) }
+  end
+  step :do_something_same_thread do
+    run { do_work }
+  end
+  step :check_email do
+    argument :email, result(:send_email)   # blocks (notified wait, bounded by timeout) until :send_email is done
+    run { |args| ... }
+  end
+end
+```
+- **Signature**: `self.async_step(name, impl = nil, &block)` — same call shape as `step`, builds the same `StepConfig` fields (`argument`, `run`, `compensate`, `undo`, `validate_args`, `validate_output`, `retries`, etc. all still work identically inside the block).
+- **Runtime contract**:
+  - Dispatches the step's work to an independent worker job; does **not** halt the calling reactor's execution of other ready steps that don't depend on it (US2 acceptance scenario 1).
+  - Any step that references `result(:async_step_name)` blocks — notified wait: woken by the completion signal the finishing worker publishes after its durable write, with a coarse fallback re-check of the durable record, bounded overall by `Configuration#async_wait_timeout` (spec Clarifications, Session 2026-08-20) — until the async step's terminal result is available. **Read semantics**: on `Success`, the reader receives the same deserialized raw value a same-process step's result would produce (US2 acceptance scenario 2, FR-006); on `Failure`, the reader receives the `Failure` object itself as the argument value — a same-process step's failure would have halted the reactor before any reader ran, so there is no sync-behavior to mirror here, and injecting the `Failure` is what lets the reader "see the failure and decide" per the spec's clarified compensation model (US2 acceptance scenario 3). This matches `async_reactor`'s wrapped-result-on-inspection pattern.
+  - `returns :async_step_name` raises at class-definition time — the reactor's return value must come from a same-process step (spec Edge Cases).
+  - Dispatch is **not** suppressed inside a worker: an `async_step` declared after a `background` hand-off point (or reached during a worker resume) still dispatches to its own independent job — the existing `inline_async_execution` guard suppresses only the *hand-off* re-trigger, never `async_step`/`async_reactor` dispatch (spec Edge Cases).
+  - Reactor-level `lock`/`semaphore`/`rate_limit` windows are held by the process executing the reactor's own steps — the async step's work runs *outside* those windows (in its own job, which acquires nothing). A step body that needs mutual exclusion must arrange it itself.
+  - If the async step fails and no later step reads its result, the parent reactor's compensation is **not** automatically triggered (US2 acceptance scenario 3, FR-011). A later step that does read the result and observes failure may itself return `Failure` to trigger compensation.
+  - `compensate`/`undo` blocks declared on an `async_step` still register normally — they only run if the step's own failure is surfaced into the parent's compensation path via the opt-in mechanism above, never automatically.
+  - A reference to the dispatched unit is recorded on the parent's own context (`composed_contexts[:send_email] = { type: :async_step_ref, ... }`) at dispatch time, and the web dashboard renders `send_email` as an `async_step`-typed node (FR-008, FR-014).
+## Added: `async_reactor`
+```ruby
+class MyReactor < RubyReactor::Reactor
+  async_reactor :create_profile, CreateProfileReactor  # fire-and-forget, no downstream reference
+  async_reactor :create_account, CreateAccountReactor do
+    argument :user_id, input(:user_id)
+  end
+  step :verify_all do
+    argument :account, result(:create_account)   # blocks until create_account finishes
+    run do |args, context|
+      if args[:account].success?
+        Success(args[:account].value)
+      else
+        Failure(args[:account].error)   # opt-in compensation trigger
+      end
+    end
+  end
+end
+```
+- **Signature**: `self.async_reactor(name, child_reactor_class, &block)` — `argument` inside the block maps parent-visible sources to the child reactor's inputs, same shape as `compose`.
+- **Dispatch contract** (the part that runs in the parent's process, FR-015/FR-016):
+  - Dispatch applies the same pre-enqueue safeguards as a top-level async run: the child's inputs are validated and, if the child declares `with_ordered_lock`, its ordering nonce is assigned at enqueue. A child-input validation failure fails **the dispatching step** (normal saga handling in the parent) — this is a dispatch failure, not a child-execution failure, and is deliberately outside FR-009's no-auto-compensation rule.
+  - Deadlock guard: if the child declares an exclusive `lock` (or a `semaphore` with `limit: 1`) whose resolved key equals one the dispatching execution currently holds, the dispatch step fails immediately with an error naming the lock key and both reactor classes. The error message MUST enumerate the three remediations, ranked: (1) use `compose` if the child belongs in the parent's critical section and its result is needed — the wait means the work is sequential anyway; (2) narrow the lock keys if parent and child actually protect different resources; (3) restructure so the locked reactor never reads the child's result — fire-and-forget with verification in the child itself or in a successor reactor outside the lock window. Lock ownership is never shared across the async boundary (parent and child run concurrently — shared ownership would break mutual exclusion); owner-based reentrancy remains for `compose` only. Transitive cross-execution cycles are out of the guard's reach (undetectable at dispatch) — documentation advises consistent key-acquisition order, with the FR-005 timeout as backstop.
+- **Runtime contract**:
+  - Dispatches `child_reactor_class` to run independently via `configuration.async_router`, linked to the parent by the child's `execution_id` for traceability/logging (FR-008, US3 acceptance scenario 4) — never added to the parent's compensation graph (FR-009).
+  - If nothing in the parent reads `result(:name)`, the child's eventual failure never affects the parent (US3 acceptance scenario 1).
+  - If a later step reads `result(:name)`, it blocks (same notified-wait policy as `async_step` — the child publishes its completion signal after its terminal save) until the child reactor reaches a terminal state, then receives the child's actual `Success`/`Failure` result object (not the enqueue-time `DispatchResult`), and may inspect `.success?`/`.value`/`.error` to decide whether to itself return `Failure` (US3 acceptance scenarios 2-3, FR-010).
+  - A reference is recorded on the parent's own context (`composed_contexts[:create_profile] = { type: :async_reactor_ref, execution_id:, reactor_class_name:, ... }`) at dispatch time — the web dashboard renders `create_profile`/`create_account` as `async_reactor`-typed nodes and lets an operator open the linked child execution, the same drill-down `compose`/`map` already offer (FR-008, FR-014, US3 acceptance scenario 4).
+  - `returns :async_reactor_name` raises at class-definition time, same as for `async_step`.
+  - A child that *pauses* at an interrupt step is not terminal: a reader keeps polling and hits the FR-005 timeout unless the child is resumed within the bound (spec Edge Cases). The child is an ordinary independently-recoverable execution — the existing sweeper/durability machinery covers its crash recovery with no new mechanism.
+## Added: `interrupt ..., resume: :background`
+```ruby
+class MyReactor < RubyReactor::Reactor
+  interrupt :webhook, resume: :background do
+    validate_payload { required(:status).filled(:string) }
+  end
+  step :heavy_work do
+    argument :status, result(:webhook)
+    run { |args| ... }   # runs in a worker, never in the process that delivered the payload
+  end
+end
+```
+- **Signature**: `self.interrupt(name, resume: :inline, &block)` — `:inline` (today's behavior, the default) or `:background`. Any other value raises `RubyReactor::Error::ValidationError` at class-definition time.
+- **Why it exists**: `background after:/before:` is edge-triggered on reaching one named step, but an interrupt re-enters the reactor from a foreground process (`continue` runs in whatever process delivered the payload). `resume: :background` makes that re-entry itself the hand-off: the delivering process validates and stores the payload, then enqueues the remainder.
+- **Runtime contract**:
+  - Payload validation (`validate_payload`), attempt counting, and `max_attempts` exhaustion (undo + failed) all run **synchronously in the calling process**, before anything is enqueued — validate in the caller, execute in the worker, the same split as `background before: <first step>` and FR-016. An invalid payload enqueues nothing and leaves the reactor paused.
+  - On a valid payload: result stored, context persisted (job payload is identity-only), `before_async_enqueue` middleware fires, remainder enqueued via `configuration.async_router`, and `continue` returns an `DispatchResult` (with `execution_id`) instead of the final result.
+  - The worker resumes through the existing path: rehydrate by id, `resume_execution`, interrupt's stored result found, remaining steps run under `inline_async_execution` (so a `background` hand-off point never re-triggers there).
+## Unchanged (explicitly out of scope, called out to prevent accidental regression)
+- Reactor-level `async true` ("Full Reactor Async") — `self.class.async?`, `lib/ruby_reactor/dsl/reactor.rb:44-50`. (Its only new interaction: combining it with `background after:` is a definition-time error, see above.)
+- `compose` itself — synchronous, fully compensation-linked nested execution, untouched. (Its `async` flag is removed — see the Removed section — but everything else about `compose` is unchanged.)
+- `map`'s dispatch/collection machinery and its full DSL including the map-internal `async` element-dispatch option — reused as an architectural pattern (see research.md) but untouched.
+- `result(:name)` for a **synchronous** step's result — resolves exactly as it does today (`Template::Result#resolve`), with zero added latency; the new notified-wait path only activates for `async_step`/`async_reactor` references.