ruby_reactor 0.8.1 → 0.8.3

data/lib/ruby_reactor/web/public/index.html

@@ -5,7 +5,7 @@
     <link rel="icon" type="image/svg+xml" href="./vite.svg" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
     <title>ui</title>
-    <script type="module" crossorigin src="./assets/index-Dw4KV4QY.js"></script>
+    <script type="module" crossorigin src="./assets/index-CeZU-ESu.js"></script>
     <link rel="stylesheet" crossorigin href="./assets/index-BQvIWPdx.css">
   </head>
   <body>

data/lib/ruby_reactor/worker.rb

@@ -10,6 +10,46 @@ module RubyReactor
   module Worker
     TERMINAL_STATUSES = %w[completed failed cancelled skipped].freeze
 
+    # Use the error's `retry_after_seconds` hint when available
+    # (RateLimit::ExceededError carries the time until the bucket rolls);
+    # otherwise fall back to the configured base + jitter for lock/semaphore
+    # contention which has no precise hint. Module-level (not just an
+    # instance method) so `StepCoordination::Contended` — which wraps a
+    # contention error but is not itself a snooze-worthy reactor-level
+    # error — can reuse the identical hint logic from the async_step worker
+    # (T036) without including this whole module.
+    #
+    # OrderedLock::WaitError is deliberately excluded from the hint path: its
+    # `retry_after_seconds` is the poison-pill window (the upper bound before
+    # a *dead* blocker is force-advanced), NOT how long the *live* blocker
+    # will take — which is usually milliseconds. Snoozing for the full window
+    # would make every out-of-order nonce sleep up to poison_pill_timeout even
+    # though its blocker finishes immediately, collapsing throughput. Re-poll
+    # at the base delay instead; poison auto-advance still clears a genuinely
+    # dead blocker on a later gate.
+    def self.snooze_delay(config, error)
+      jitter = config.lock_snooze_jitter.to_f
+      jitter_amount = jitter.positive? ? rand(0.0..jitter) : 0.0
+
+      if hinted_retry?(error)
+        [error.retry_after_seconds.to_f, 0.1].max + jitter_amount
+      else
+        config.lock_snooze_base_delay.to_f + jitter_amount
+      end
+    end
+
+    def self.hinted_retry?(error)
+      # `StepCoordination::Contended` wraps the WaitError as `.original` —
+      # unwrap so the same exclusion applies whether the caller is the
+      # reactor-level ordered lock (raises WaitError directly) or a step's
+      # (raises Contended, whose OWN `retry_after_seconds` just forwards the
+      # wrapped error's hint unchanged).
+      original = error.respond_to?(:original) ? error.original : error
+      return false if original.is_a?(RubyReactor::OrderedLock::WaitError)
+
+      error.respond_to?(:retry_after_seconds) && error.retry_after_seconds
+    end
+
     # Last line of observability when a job burns its whole retry budget on an
     # infrastructure failure and the backend then discards it (Sidekiq runs
     # with `dead: false`): without this, the context would stay "running"
@@ -93,8 +133,11 @@ module RubyReactor
              RubyReactor::Semaphore::AcquisitionError,
              RubyReactor::RateLimit::ExceededError,
              RubyReactor::OrderedLock::WaitError,
-             RubyReactor::Error::AsyncResultPending => e
-        # Snooze on expected concurrency, rate, or ordering contention.
+             RubyReactor::Error::ExecutionParked => e
+        # Snooze on expected concurrency, rate, or ordering contention, or on
+        # a park signal (a step's contention, or an awaited background result)
+        # — raised at any nesting depth, after every executor on the stack has
+        # parked its own holds and saved.
         # OrderedLock::WaitError carries a poison-pill-derived retry hint,
         # consumed by compute_snooze_delay below. We avoid the framework's native
         # retry path so this doesn't burn the job's retry budget or appear
@@ -147,12 +190,14 @@ module RubyReactor
       # duplicate of the *same* execution may wait arbitrarily long for the
       # live original to finish (e.g. a sweeper re-enqueue racing a slow but
       # alive worker). Capping it would fail a legitimately-waiting duplicate.
-      # A parked async wait is likewise uncapped HERE: its bound is
-      # `async_park_timeout`, enforced against `dispatched_at` at the wait
-      # site — counting snoozes would double-bound it with the wrong unit.
+      # A park signal is likewise uncapped HERE, bounded where it is raised:
+      # an async wait by `async_park_timeout` against `dispatched_at`, a
+      # step's contention by `lock_snooze_max_attempts` on that step's own
+      # contention counter (`StepExecutor#handle_contention`) — counting
+      # snoozes too would double-bound either with the wrong unit.
       capped = !(error.is_a?(RubyReactor::OrderedLock::WaitError) ||
                  error.is_a?(RubyReactor::Lock::ContextLockContention) ||
-                 error.is_a?(RubyReactor::Error::AsyncResultPending))
+                 error.is_a?(RubyReactor::Error::ExecutionParked))
 
       if capped && max != :infinity && snooze_count >= max
         escalate_snooze(context, snooze_count, error)
@@ -165,34 +210,15 @@ module RubyReactor
       self.class.perform_in(delay, context_id, reactor_class_name, snooze_count + 1)
     end
 
-    # Use the error's `retry_after_seconds` hint when available
-    # (RateLimit::ExceededError carries the time until the bucket rolls);
-    # otherwise fall back to the configured base + jitter for lock/semaphore
-    # contention which has no precise hint.
-    #
-    # OrderedLock::WaitError is deliberately excluded from the hint path: its
-    # `retry_after_seconds` is the poison-pill window (the upper bound before
-    # a *dead* blocker is force-advanced), NOT how long the *live* blocker
-    # will take — which is usually milliseconds. Snoozing for the full window
-    # would make every out-of-order nonce sleep up to poison_pill_timeout even
-    # though its blocker finishes immediately, collapsing throughput. Re-poll
-    # at the base delay instead; poison auto-advance still clears a genuinely
-    # dead blocker on a later gate.
+    # Instance methods delegate to the module functions above — worker
+    # behavior is unchanged, just relocated so other callers (StepWorker,
+    # T036) can reuse the same logic without a Worker instance.
     def compute_snooze_delay(config, error)
-      jitter = config.lock_snooze_jitter.to_f
-      jitter_amount = jitter.positive? ? rand(0.0..jitter) : 0.0
-
-      if hinted_retry?(error)
-        [error.retry_after_seconds.to_f, 0.1].max + jitter_amount
-      else
-        config.lock_snooze_base_delay.to_f + jitter_amount
-      end
+      Worker.snooze_delay(config, error)
     end
 
     def hinted_retry?(error)
-      return false if error.is_a?(RubyReactor::OrderedLock::WaitError)
-
-      error.respond_to?(:retry_after_seconds) && error.retry_after_seconds
+      Worker.hinted_retry?(error)
     end
 
     def escalate_snooze(context, snooze_count, error)

data/lib/ruby_reactor.rb

@@ -133,14 +133,14 @@ module RubyReactor
 
   class Failure
     attr_reader :error, :retryable, :step_name, :inputs, :backtrace, :reactor_name, :step_arguments, :exception_class,
-                :file_path, :line_number, :code_snippet, :validation_errors
+                :file_path, :line_number, :code_snippet, :validation_errors, :rollback_failures
 
-    # rubocop:disable Metrics/ParameterLists, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity
+    # rubocop:disable Metrics/ParameterLists, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity, Metrics/MethodLength
     def initialize(error, retryable: nil, step_name: nil, inputs: {}, backtrace: nil, redact_inputs: [],
                    reactor_name: nil, step_arguments: {}, exception_class: nil,
                    file_path: nil, line_number: nil, code_snippet: nil, invalid_payload: false, validation_errors: nil,
-                   **opts)
-      # rubocop:enable Metrics/ParameterLists, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity
+                   rollback_failures: nil, **opts)
+      # rubocop:enable Metrics/ParameterLists, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity, Metrics/MethodLength
       retryable = opts[:retry] if opts.key?(:retry) # `retry:` wins over `retryable:` when both are given
       @error = error
 
@@ -159,6 +159,7 @@ module RubyReactor
         line_number ||= attributes[:line_number]
         code_snippet ||= attributes[:code_snippet]
         validation_errors ||= attributes[:validation_errors]
+        rollback_failures ||= attributes[:rollback_failures]
       end
 
       @retryable = if retryable.nil?
@@ -179,6 +180,9 @@ module RubyReactor
       @code_snippet = code_snippet
       @invalid_payload = invalid_payload
       @validation_errors = validation_errors
+      # Every undo/compensation that did not complete during this failure's
+      # rollback (005 FR-004): `{ step:, kind:, key:, reason:, message: }`.
+      @rollback_failures = normalize_rollback_failures(rollback_failures)
     end
 
     def success?
@@ -236,6 +240,7 @@ module RubyReactor
         line_number: @line_number,
         code_snippet: @code_snippet,
         validation_errors: @validation_errors,
+        rollback_failures: @rollback_failures,
         backtrace: @backtrace
       }
     end
@@ -334,9 +339,26 @@ module RubyReactor
         file_path: err[:file_path],
         line_number: err[:line_number],
         code_snippet: err[:code_snippet],
-        validation_errors: err[:validation_errors]
+        validation_errors: err[:validation_errors],
+        rollback_failures: err[:rollback_failures]
       }
     end
+
+    ROLLBACK_FAILURE_SYMBOLS = %i[step kind reason].freeze
+    private_constant :ROLLBACK_FAILURE_SYMBOLS
+
+    # A stored failure comes back with string keys (and, through plain JSON,
+    # string values); `step`, `kind` and `reason` are Symbols on a live one.
+    def normalize_rollback_failures(entries)
+      return [] unless entries.is_a?(Array)
+
+      entries.map do |entry|
+        entry.to_h do |k, v|
+          key = k.to_sym
+          [key, ROLLBACK_FAILURE_SYMBOLS.include?(key) && v ? v.to_sym : v]
+        end
+      end
+    end
   end
 
   # Sentinel returned when a step's work is handed off to a worker job and is

data/specs/future_improvements.md

@@ -0,0 +1,250 @@
+# Future improvements for review
+
+## Guard
+
+I want to add another hook feature to the step: `guard`
+A guard concept is to execute after validations and before run.
+example:
+
+```ruby
+class SendEmail < RubyReactor::Step
+  input :email, :string, format?: /\A[^@\s]+@[^@\s]+\z/
+
+  def guard
+    fail!("Prevent spamming") if EmailService.sent_today?(inputs[:email])
+    success! # optional
+  end
+
+  def run
+    # do the work
+  end
+end
+```    
+## Fenced context writes
+
+**Status:** proposal, not scheduled. Follows from 005 (step coordination remediation), R-18 and
+R-19 in `specs/005-step-coordination-remediation/research.md`.
+
+### The rule
+
+A context is written by one process only: the controlled execution that owns it.
+
+- Async children (`async_step`, `async_reactor`, map elements) never write their parent. The
+  parent holds only the link written at dispatch; each child keeps its own record, and the
+  dashboard rebuilds its view from the links. 005 made this true for `async_step` (R-18).
+- Today the rule is a convention: any code with a storage adapter can `SET` any context. This
+  proposal makes it a guarantee that the storage enforces.
+
+### Why a convention is not enough
+
+Holding the lock at some point is not the same as owning the data when the write lands:
+
+| Failure | What happens today |
+|---|---|
+| **Paused holder**: a worker holding the `async:<root id>` lock stalls (GC, network, a long rollback wait) past `context_lock_ttl`. The lock expires, a redelivery takes it and makes progress, then the first worker wakes. | The first worker's next `store_context` is a plain `SET`. It overwrites the newer state. Its auto-extender fails silently. Nothing stops the write. |
+| **Stale read**: `Worker#perform` loads the context *before* `resume_execution` takes the context lock. | A previous holder can write between that read and the lock. The new holder then runs on the older snapshot and saves over the newer one. |
+| **Writer outside any execution**: a path writes with no lock at all. | It overwrites whatever the live execution saved. 005 fixed two of these (`StepWorker#save_root`, the map collector's post-resume save); others remain (below). |
+
+The first two are the textbook reasons distributed locks need fencing. A lock that only *excludes*
+is not enough. The storage must also *reject writes from anyone who no longer holds the lock*.
+
+### Remaining writers outside a controlled execution
+
+Found while auditing for 005 R-19:
+
+| Writer | Where | Today |
+|---|---|---|
+| `Worker.record_retries_exhausted` | backend's retries-exhausted hook | Plain read-modify-write, no lock. Marks the context failed even if a redelivery is live. |
+| `Worker#escalate_snooze` | after the executor returned | Runs after the executor released the context lock. |
+| `Worker#handle_deserialization_failure` | before any executor | Writes a failed payload with no lock. |
+| Map collector, failure branch | `Map::Helpers#resume_parent_execution` | Holds `map_collect:<map_id>`, never the parent's `async:` lock. |
+| `Reactor.cancel`, `Reactor.undo` | any process (app code, console) | `find` → mutate → `save_context`: a blind write racing a live worker. |
+| `Reactor#continue` (interrupt resume) | web request / app code | Writes the payload before `resume_execution` takes the lock. |
+| Synchronous `Reactor.run` / `Executor#execute` | caller's process | Never takes the context lock, so its saves are unfenced. |
+
+Writers that already belong to the owning execution:
+- `Executor#save_context` and `#checkpoint!` on the worker path;
+- `RetryManager#requeue_job`;
+- `StepExecutor#checkpoint_root!`;
+- `MapStep#prepare_async_execution`.
+
+Two writers only *create* a row that nothing has read yet:
+- `Reactor#save_context` at enqueue;
+- `AsyncReactorStep#save`, the child's first row.
+
+### Proposal
+
+Four parts. Parts 1 and 2 are the mechanism; parts 3 and 4 route the remaining writers through it.
+
+#### 1. Ownership-checked, versioned writes (the fence)
+
+- Every context row gets a version: a small key next to the blob,
+  `reactor:<Class>:context:<id>:v`, with the same TTL as the blob.
+- Every write goes through one Lua script. The script takes:
+  - the lock key it claims to hold (`lock:async:<root id>`, or the map element's
+    `lock:map_element:<map>:<index>`);
+  - the owner token of that lock: the per-execution UUID `Executor#acquire_context_lock`
+    already generates;
+  - the version the writer loaded.
+- The script writes only if **the lock is held by that token** *and* **the stored version equals
+  the loaded one**. It then bumps the version.
+- It returns `ok`, `lost_lock` or `stale`:
+
+```lua
+-- KEYS: context_key, version_key, lock_key   ARGV: blob, ttl, owner, expected_version
+if redis.call('hget', KEYS[3], 'owner') ~= ARGV[3] then return 'lost_lock' end
+local current = tonumber(redis.call('get', KEYS[2]) or '0')
+if current ~= tonumber(ARGV[4]) then return 'stale' end
+redis.call('set', KEYS[1], ARGV[1], 'EX', ARGV[2])
+redis.call('set', KEYS[2], current + 1, 'EX', ARGV[2])
+return 'ok'
+```
+
+Why both checks:
+- The **owner check** stops a paused holder from writing at all, even before anyone else has
+  written.
+- The **version check** catches a stale read by the current holder, and any path this proposal
+  missed.
+
+The lock and the data live in the same Redis, and the check runs atomically inside it, so the
+owner comparison does the job of a fencing token. There are no clock or ordering assumptions.
+If locks and contexts ever move to different stores, switch to classic monotonic fencing tokens
+(`INCR` on acquire, highest-token-wins at the store).
+
+**Creating a row** uses the same script with `expected_version = 0` and no lock check: a
+create-only `SET NX` equivalent. It covers `Reactor#save_context` at enqueue and an
+`async_reactor` child's first row.
+
+#### 2. Lock, then load
+
+- A worker takes the context lock **before** it reads the context. The restructure:
+  `Worker#perform` acquires `async:<root id>`, then retrieves and deserializes, then hands
+  the token and the loaded version to `Executor`. Today `resume_execution` acquires the lock
+  mid-way.
+- A synchronous `Reactor.run` takes the same lock for its whole run. It is one `SET NX` and a
+  release; today it takes none.
+- Composed children already run inside the root's lock and write through the root. They reuse
+  the root's token.
+
+#### 3. External actors send requests, not writes
+
+Operator actions must not write a live context. Make them requests the owning execution
+consumes, the same pattern as async children writing their own records:
+
+- **`Reactor.cancel` / `.undo`**:
+  - write a `cancel_requested` key, `reactor:<Class>:context:<id>:cancel`;
+  - the owning execution checks it at each step boundary and cancels or undoes itself, under its
+    own lock;
+  - if no execution is live (the context lock is free), `cancel` takes the lock itself and
+    applies the request directly: lock, load, fenced write.
+- **`Reactor#continue`**: take the lock first (bounded wait, because it is a user action), load,
+  store the payload, resume. If the lock is held, return "busy — retry" instead of racing.
+- **Worker bookkeeping** (`record_retries_exhausted`, `escalate_snooze`, deserialization
+  failure):
+  - take the context lock with `wait: 0`;
+  - if another process holds it, that process owns the outcome, so log and skip;
+  - move `escalate_snooze` inside the executor's lock scope instead of after it.
+- **Map collector**: take the parent's `async:` lock for both branches. The success branch passes
+  the token into `resume_execution`, which must accept an already-held lock (re-entrant by owner).
+
+#### 4. What a rejected write does
+
+A write that comes back `lost_lock` or `stale` raises `Error::ContextOwnershipLost`:
+- The executor stops at once. It saves nothing else, runs no further steps, and releases only
+  holds it still owns (every release is already owner-checked).
+- It logs one structured line: `event="ruby_reactor.context.ownership_lost"`, with the context
+  id, the lock key and the expected and actual version.
+- The worker does **not** snooze or retry the job. The current owner is responsible for the run.
+- A rejection is a correctness signal, not an error to hide. Expose it in the log and in the
+  `:failed_reactor` middleware event, but never mark the context failed: its owner decides the
+  outcome.
+
+### Constraints and open questions
+
+- **Redis Cluster**: the script touches three keys, which must share a hash slot. Tag them with
+  the root id: `lock:async:{<root>}`, `reactor:<Class>:context:{<root>}`, and the version key. The
+  ordered lock already does this. Changing the key format needs a migration or a dual-read window.
+- **Composed children's own rows**: `Executor#save_context` of a composed child also stores the
+  child under its own id. That is a second key in a different slot. Either tag it with the root
+  id too, or drop these rows. They are an observability path: the root blob already embeds the
+  child.
+- **Inline test mode**: `Sidekiq::Testing.inline!` re-enters the worker inside a frame that holds
+  the lock, which is why `acquire_context_lock` is skipped there today. The fence needs the same
+  exemption, or re-entrancy by owner token.
+- **Redis failover**: replication is asynchronous, so a failover can lose an acknowledged lock
+  or write. This proposal gives single-primary guarantees, the same as today's locks. Stronger
+  guarantees mean `WAIT`/`WAITAOF` on the lock write, or a different store. Document; don't solve
+  here.
+- **Other records**: Step Result Records and map element results have the same shape:
+  - the dispatcher creates the record;
+  - the unit's job updates it under its liveness lock;
+  - it could be fenced the same way.
+
+  Nothing races there today, because the liveness lock drops concurrent duplicates. Decide
+  whether to fence them now or later.
+
+### Rollout
+
+- **SemVer**: MINOR, behind `config.fenced_context_writes` (default `false`) for one release.
+  Existing contexts have no version key, which is read as `0`.
+- A rolling deploy mixes old workers (plain `SET`, no version bump) with new ones, so the new
+  workers would see spurious `stale` results. Enable the flag only after every worker runs the new
+  version. Flip the default in the next MAJOR.
+- **Cost**: one `EVAL` in place of one `SET` per save; checkpoints already save once per step.
+  One extra `SET NX` and release per synchronous run.
+
+### Test plan
+
+All against real Redis, per the constitution.
+
+1. **Paused holder**:
+   - take the lock with a short TTL and let it expire;
+   - a second owner takes it and writes;
+   - the first owner's write returns `lost_lock`, and the stored blob is the second owner's.
+2. **Stale read**:
+   - load at version N;
+   - another writer (holding the lock through a handover) bumps the version to N+1;
+   - the write returns `stale`.
+3. **Lock, then load**: a redelivery that races a finishing holder always runs on the holder's
+   final state. This is the same repro shape as 005 P4.
+4. **One spec per writer** in the tables above, showing it goes through the fence, or takes the
+   lock and skips when it is held.
+5. **Cancel of a live run**: the request is consumed at the next step boundary. The running
+   execution's saves are never overwritten by the canceller.
+6. **Cluster slot co-location**, if Redis Cluster is supported: every key a script touches
+   hashes to one slot.
+
+### Size
+
+- Storage adapter: one new script and a versioned `retrieve_context`.
+- Executor and `Worker`: lock, then load, and pass the token.
+- Six writer paths rerouted, plus the cancel/undo request key.
+- Docs: `locks_and_semaphores.md`, `background_and_async.md`, and the README's "Durability &
+  Recovery" section.
+
+## Fan-out map inside a composed child
+
+**Status:** bug, already present on main (0.8.1). Raised by the PR #56 review.
+
+A root that composes a child, where the child runs a `fan_out` map, never finishes. The root
+stays `running`.
+
+- `MapStep#prepare_async_execution` stores the child under its own id.
+- The map collector loads that child blob. Its `root_context` is nil, so the collector resumes
+  the child as a standalone execution. On a park, it requeues the child the same way.
+- The child completes in its own blob. Nothing resumes the root, and the root's embedded copy of
+  the child stays at the map step.
+
+To reproduce: `Root` composes `Child`, and `Child` runs a `map ... fan_out batch_size: 1`. Drain
+the element, collector and `Worker` jobs. The root is still `running`.
+
+A literal patch would have the collector write the root's blob. That breaks the single-writer
+rule, because the collector holds only `map_collect:`, never the root's `async:` lock.
+
+Direction:
+- The collector records the map's completion on the map's own records, which it already does.
+- The collector requeues the ROOT worker and does not resume the child itself.
+- On re-entry, `MapStep` adopts the finished map: collect the results, don't dispatch again.
+
+Spec: compose → fan-out map, with and without a park after the map. The root completes either
+way.

data/specs/scoped_rspec_matchers.md

@@ -0,0 +1,158 @@
+# Scoping the RSpec matchers to `type: :reactor`
+
+## Goal
+
+`RubyReactor::RSpec.configure(config)` should add **nothing** to a host app's
+example groups unless they opt in with `type: :reactor`. Helpers already work
+that way. The 19 matchers do not — they are global the moment
+`ruby_reactor/rspec` is required, and moving them behind the tag needs a
+one-line-per-matcher change plus a migration.
+
+## Current state
+
+[lib/ruby_reactor/rspec.rb:20-42](../lib/ruby_reactor/rspec.rb#L20-L42):
+
+| Piece | Scoped to `type: :reactor`? |
+| --- | --- |
+| `Helpers` (`test_reactor`) | yes — `config.include …, REACTOR_METADATA` |
+| `SidekiqHelpers` (`drain_async_jobs`, `pending_async_jobs`) | yes |
+| `before(:each)` setup (fake mode, storage wipe, snooze reset) | yes |
+| `Matchers` (19 matchers) | **no** |
+| `StorageReset.install!`, `StepExecutor.prepend` | no — load-time monkeypatches, out of scope for this doc |
+
+## Why the `config.include` line does nothing today
+
+[matchers.rb](../lib/ruby_reactor/rspec/matchers.rb) declares every matcher as:
+
+```ruby
+module Matchers
+  ::RSpec::Matchers.define :be_success do
+    ...
+  end
+end
+```
+
+`define` is [`RSpec::Matchers::DSL#define`](https://github.com/rspec/rspec-expectations/blob/main/lib/rspec/matchers/dsl.rb),
+and its entire body is a `define_method` **on the receiver**:
+
+```ruby
+def define(name, &declarations)
+  warn_about_block_args(name, declarations)
+  define_method name do |*expected, &block_arg|
+    RSpec::Matchers::DSL::Matcher.new(name, declarations, self, *expected, &block_arg)
+  end
+end
+alias_method :matcher, :define
+```
+
+The receiver is `::RSpec::Matchers`, which RSpec includes into *every* example
+group. So the matcher methods land in the global spec namespace at require
+time, and `RubyReactor::RSpec::Matchers` ends up an empty module — the
+`config.include RubyReactor::RSpec::Matchers` line in
+[rspec.rb:32](../lib/ruby_reactor/rspec.rb#L32) includes nothing. Adding
+`REACTOR_METADATA` to that line scopes nothing either.
+
+## The change
+
+Extend the DSL into **our** module, so `define_method` targets
+`RubyReactor::RSpec::Matchers`:
+
+```ruby
+module RubyReactor
+  module RSpec
+    module Matchers
+      extend ::RSpec::Matchers::DSL   # <- add
+
+      matcher :be_success do          # <- was ::RSpec::Matchers.define
+        ...
+      end
+    end
+  end
+end
+```
+
+Then gate the include:
+
+```ruby
+config.include RubyReactor::RSpec::Matchers, REACTOR_METADATA
+```
+
+Mechanically: one `extend` line, 19 `::RSpec::Matchers.define` →
+`matcher` rewrites (`sed` handles it), one changed line in `rspec.rb`, and the
+comment at [rspec.rb:29-30](../lib/ruby_reactor/rspec.rb#L29-L30) — which
+currently explains why matchers *can't* be scoped — deleted.
+
+Nothing inside the matcher bodies changes. `Matchers.coordination_adapter`
+([matchers.rb:260](../lib/ruby_reactor/rspec/matchers.rb#L260)) is called with
+an explicit receiver and resolves lexically at match time, so it is unaffected.
+
+### Verified
+
+A probe against this repo's `spec_helper`, with one matcher declared each way:
+
+```
+UNTAGGED scoped: expected :nope to respond to `probe_scoped?`   # not included -> falls through
+UNTAGGED global: GLOBAL-MATCHER-RAN: expected :ok, got :nope    # still global
+TAGGED   scoped: SCOPED-MATCHER-RAN: expected :ok, got :nope    # real matcher, chains and all
+```
+
+Note the first line: scoping a matcher out does **not** produce a
+`NoMethodError`. See below.
+
+## The trap: `be_*` / `have_*` fall through to a predicate matcher
+
+All 19 matcher names start with `be_` or `have_`, which is exactly the prefix
+RSpec's dynamic predicate matchers claim. Once a matcher is no longer in scope,
+`expect(x).to be_success` stops meaning "RubyReactor's `be_success`" and starts
+meaning "call `x.success?`". Two consequences:
+
+1. **`respond_to?(:be_success)` is not a scoping test.** It returns `true` in
+   any example group for any `be_*`/`have_*` name, because
+   `RSpec::Matchers#respond_to_missing?` claims the whole prefix. Test scoping
+   by running an expectation and reading the failure message (as the probe
+   above does), not by probing `respond_to?`.
+2. **Some call sites degrade silently rather than breaking.** `TestSubject` and
+   the result objects define `success?`, `failure?`, `paused?`, `halted?` and
+   `skipped?` ([test_subject.rb:388-405](../lib/ruby_reactor/rspec/test_subject.rb#L388-L405),
+   [ruby_reactor.rb:50-62](../lib/ruby_reactor.rb#L50-L62)), so an untagged
+   `expect(result).to be_success` keeps passing via the predicate — with a
+   generic failure message instead of the rich one in
+   [matchers.rb:14-23](../lib/ruby_reactor/rspec/matchers.rb#L14-L23), and
+   without the `ensure_executed!` nudge the real matcher does. Anything with a
+   chain (`have_run_step(:x).returning(y)`, `be_halted.because(:period)`,
+   `be_locked.by(owner)`) or a non-predicate name fails loudly instead.
+
+So the migration cost is not "find the red specs" — a suite can go green while
+quietly asserting something weaker. Grep for the matcher names rather than
+trusting a passing run.
+
+## Migration
+
+In this repo, 34 spec files use these matchers; 24 of them are not tagged
+`type: :reactor`. Most only need the tag added to the `RSpec.describe` line, the
+same edit made for `test_reactor` in `0a3ebdab`. Order of work:
+
+1. Rewrite `matchers.rb` to the DSL-extend form, leave the include ungated —
+   suite must stay green (proves the rewrite alone changes nothing).
+2. Gate the include, run the suite, tag the files that fail.
+3. Grep the 24 untagged files for matcher names and tag the rest by hand —
+   step 2 will not catch the silent-degradation cases above.
+4. Update [documentation/testing.md](../documentation/testing.md#L17-L35),
+   whose Setup section currently says matchers are available everywhere, and
+   the comment at [rspec.rb:29-30](../lib/ruby_reactor/rspec.rb#L29-L30).
+
+## Breaking change
+
+Downstream suites calling these matchers in untagged groups are affected, with
+the same silent-degradation caveat: a user's suite may stay green while
+asserting less. This warrants a `BREAKING CHANGE:` footer and a release note
+that names the fallback behavior explicitly, not just "matchers are now
+scoped".
+
+## Alternative considered
+
+Keep matchers global, on the grounds that `be_success` / `have_run_step` are
+already ambiguous with RSpec's predicate matchers and that a host app gains
+little from gating them. This costs nothing and breaks nobody — worth taking if
+the namespace complaint is theoretical rather than something a host app has
+actually hit.