ruby_reactor 0.8.4 → 0.8.6

data/specs/007-execution-flow-analysis/checklists/requirements.md

@@ -0,0 +1,39 @@
+# Specification Quality Checklist: Execution Flow & Compensation Analysis
+
+**Purpose**: Validate specification completeness and quality before proceeding to planning
+**Created**: 2026-09-26
+**Feature**: [spec.md](../spec.md)
+
+## Content Quality
+
+- [x] No implementation details (languages, frameworks, APIs)
+- [x] Focused on user value and business needs
+- [x] Written for non-technical stakeholders
+- [x] All mandatory sections completed
+
+## Requirement Completeness
+
+- [x] No [NEEDS CLARIFICATION] markers remain
+- [x] Requirements are testable and unambiguous
+- [x] Success criteria are measurable
+- [x] Success criteria are technology-agnostic (no implementation details)
+- [x] All acceptance scenarios are defined
+- [x] Edge cases are identified
+- [x] Scope is clearly bounded
+- [x] Dependencies and assumptions identified
+
+## Feature Readiness
+
+- [x] All functional requirements have clear acceptance criteria
+- [x] User scenarios cover primary flows
+- [x] Feature meets measurable outcomes defined in Success Criteria
+- [x] No implementation details leak into specification
+
+## Notes
+
+- Stakeholders for this research are library maintainers and reactor authors, so library
+  vocabulary (step, compose, map, compensate, undo, inline/background) is the domain language, not
+  implementation detail. The `compensate_all` / `compensate_each` names appear only because the
+  user proposed them as candidates to evaluate.
+- Validation passed on iteration 1. No clarifications needed: scope ambiguities (interrupts,
+  doc edits, evidence method) were resolved as documented assumptions.

data/specs/007-execution-flow-analysis/contracts/report-structure.md

@@ -0,0 +1,71 @@
+# Contract: Report Structure
+
+The deliverable's "interface" is the set of documents a maintainer reads and the probe output a
+reviewer re-runs. This contract fixes what each must contain so the report can be checked
+mechanically (see quickstart.md).
+
+## `analysis/README.md`
+
+1. **Scope & baseline**: commit, constructs covered, what is out of scope.
+2. **How to read**: vocabulary (research D4), evidence labels (D5), status/severity scales (D6/D7).
+3. **Answers**: one subsection per user question, in this order:
+   - Q1 *Are already-executed map elements compensated individually?*
+   - Q2 *When a composed reactor fails, are previously completed composed reactors compensated?*
+   - Q3 *Would `compensate_all` / `compensate_each` on `map` close a real gap?*
+
+   Each answer MUST open with a one-line verdict (`Yes` / `No` / `Depends: …`), then list the
+   conditions, then evidence labels, then links to the matrix rows, invariants and findings.
+4. **Top findings**: the High-severity findings, one line each with links.
+5. **File index**.
+
+## `analysis/execution-order.md`
+
+1. **Construct lifecycles**: one subsection per construct (step, compose, map inline, map fan-out,
+   async_step, async_reactor, background reactor, interrupt). Each has a numbered lifecycle from
+   scheduling to terminal state, saying where rollback hooks attach and where locks are held.
+2. **Rollback algorithm**: the generic compensate → reverse-undo sequence with a Mermaid diagram.
+3. **Order matrix**: one table per construct. Columns: `Scenario id | shape | failure at | mode |
+   ordered events | left in place | evidence`. No blank cells. Unreachable cells say
+   `not reachable: <reason>`.
+4. **Cross-cutting sections**: locks (lifetime vs rollback), retries (attempts vs rollback),
+   failure kinds (which path each takes: rollback / no rollback / never-started), each with its
+   own table.
+
+## `analysis/invariants.md`
+
+A table or list of `INV-nn` entries with every Invariant field from data-model.md, grouped by
+area. It ends with a **coverage summary**: counts by status, and the list of invariants with
+`coverage: none`.
+
+## `analysis/findings-and-options.md`
+
+1. **Findings**, ranked High → Low, with every Finding field.
+2. **Documentation audit**: a table `file:line | quoted claim | actual | finding id`.
+3. **Options**, grouped under the finding they address, with every Option field. The map
+   `compensate_all` / `compensate_each` evaluation MUST appear here and compare both shapes on:
+   failure semantics (fail_fast on/off), inline vs fan-out, interaction with element retries,
+   interaction with a later step failing, and data availability (what `all_elements` / `element`
+   would contain).
+4. A closing note: every option is a proposal pending later analysis.
+
+## `evidence/output.txt`
+
+Produced by `evidence/run.rb`. It has one block per scenario:
+
+```text
+== S-plain-01  a → b(fails) → c   [inline]
+expected: run:a run:b compensate:b undo:a => failure(b)
+observed: run:a run:b compensate:b undo:a => failure(b)
+MATCH
+```
+
+`MISMATCH` blocks are kept, not hidden. A `MISMATCH` means the report's expectation was wrong and
+the report MUST be corrected to the observed sequence (the observation wins). Scenarios whose
+purpose is to show a counter-example set `expected` to the *observed* behavior and state the
+violation in the report.
+
+## Cross-reference rules
+
+- Every `[O: S-…]` label in `analysis/*.md` resolves to a block in `output.txt`.
+- Every finding lists at least one scenario or `[R]` citation.
+- Every High finding has at least two options, each with a stated con (SC-004).

data/specs/007-execution-flow-analysis/data-model.md

@@ -0,0 +1,83 @@
+# Data Model: Execution Flow & Compensation Analysis
+
+The "data" here is the report's own structure. Every entity below has a stable ID so report files
+can cross-reference each other and cite probes.
+
+## Scenario
+
+A reactor shape + failure location + conditions. It is one matrix cell, and usually one probe.
+
+| Field | Description |
+|---|---|
+| `id` | `S-<area>-<nn>`; area ∈ `plain`, `retry`, `compose`, `map`, `async`, `bg`, `lock`, `intr`, `edge` |
+| `shape` | Constructs and nesting, e.g. `a → compose(c1 → c2) → b` |
+| `failure_at` | Which step fails and how (`returns Failure`, `raises`, `contended`, `validation`, `compensate fails`, …) |
+| `mode` | `inline` or `worker` (+ which worker path) |
+| `conditions` | Locks / retries / fail_fast / fan_out flags in effect |
+| `expected` | Ordered event list the report claims (see Trace event) |
+| `left_in_place` | Completed work that nothing rolls back |
+| `evidence` | `[R]`, `[O]`, `[T]` labels (research D5) |
+
+Rule: a matrix cell is **filled** when `expected` and `left_in_place` are both stated, or when it is
+marked `not reachable` with a one-line reason (SC-001).
+
+## Trace event
+
+One entry in an observed or expected sequence.
+
+| Kind | Format | Source |
+|---|---|---|
+| Body | `run:<step>`, `compensate:<step>`, `undo:<step>` | probe step bodies |
+| Element body | `run:<step>[<i>]`, `undo:<step>[<i>]` | probe map element steps |
+| Middleware | `<event>:<subject>` e.g. `lock_acquired:acct:1`, `retry_attempt:charge#2` | recorder middleware |
+| Outcome | `=> success`, `=> failure(<step>)`, `=> halt`, `=> paused` | probe footer |
+
+Nested reactors prefix the step with the child name when they would otherwise be ambiguous,
+e.g. `run:child.c1`.
+
+## Invariant
+
+| Field | Description |
+|---|---|
+| `id` | `INV-<nn>` |
+| `statement` | A testable proposition ("Compensation never runs for a step whose body never started") |
+| `scope` | Constructs/modes it applies to |
+| `status` | `HOLDS` · `VIOLATED` · `CONDITIONAL` · `UNDETERMINED` (research D6) |
+| `conditions` | Required when `CONDITIONAL` |
+| `evidence` | `[R]`/`[O]`/`[T]` labels; a `VIOLATED` status MUST cite a reproducible counter-example (`[O]`) |
+| `coverage` | Existing spec(s) exercising it, or `none` |
+
+## Finding
+
+| Field | Description |
+|---|---|
+| `id` | `F-<nn>` |
+| `title` | One line |
+| `severity` | `High` · `Medium` · `Low` (research D7) |
+| `scenario` | Scenario id(s) that show it |
+| `reader_expects` | What someone reading the DSL/docs would expect |
+| `actual` | What happens, with evidence |
+| `doc_conflict` | Quoted README/documentation text + `file:line`, or `none` |
+| `related` | Invariant ids |
+
+## Improvement option
+
+| Field | Description |
+|---|---|
+| `id` | `O-<finding>-<letter>` e.g. `O-03-a` |
+| `addresses` | Finding id(s) |
+| `sketch` | DSL/behavior shape (pseudo-code allowed) |
+| `pros` / `cons` | At least one con is required (SC-004) |
+| `compatibility` | Breaking? Migration impact? |
+| `open_questions` | What must be decided before choosing |
+| `status` | Always `proposal`, never a decision |
+
+## Relationships
+
+```text
+Scenario 1─* Trace event
+Scenario *─* Invariant   (a scenario is evidence for invariants)
+Finding  *─* Scenario    (shown by)
+Finding  *─* Invariant   (violates / conditions)
+Option   *─1..* Finding  (addresses)
+```

data/specs/007-execution-flow-analysis/evidence/harness.rb

@@ -0,0 +1,231 @@
+# frozen_string_literal: true
+
+# Evidence harness for specs/007-execution-flow-analysis. Research artifact,
+# NOT library or test-suite code. It runs throwaway reactors against the real
+# test Redis through the real worker bodies (Sidekiq fake mode + drain) and
+# prints the observed forward/rollback event order next to the order the
+# report claims.
+
+require "bundler/setup"
+require "logger"
+require "ruby_reactor"
+require "sidekiq/testing"
+
+Sidekiq::Testing.fake!
+Sidekiq.configure_client do |config|
+  config.redis = { url: ENV.fetch("RUBY_REACTOR_TEST_REDIS_URL", "redis://localhost:6780") }
+  config.logger = Logger.new(IO::NULL)
+end
+
+module Probe
+  REDIS_URL = ENV.fetch("RUBY_REACTOR_TEST_REDIS_URL", "redis://localhost:6780")
+
+  # Drained in this order until quiet. Delayed jobs (perform_in) run at once.
+  WORKERS = [
+    RubyReactor::Adapters::Sidekiq::Worker,
+    RubyReactor::Adapters::Sidekiq::MapElementWorker,
+    RubyReactor::Adapters::Sidekiq::MapCollectorWorker,
+    # 009 R-04: a fan-out map rolls back one element per job.
+    RubyReactor::Adapters::Sidekiq::MapElementRollbackWorker,
+    RubyReactor::Adapters::Sidekiq::StepWorker
+  ].freeze
+
+  @events = []
+  @notes = []
+  @counters = Hash.new(0)
+  @tally = { match: 0, mismatch: 0 }
+
+  class << self
+    attr_reader :events, :notes, :counters, :tally
+
+    def rec(event) = @events << event
+    def note(text) = @notes << text
+
+    # Uniform step labels: "run:a", "undo:child.c1", "run:e1[2]".
+    def label(prefix, name, index = nil)
+      base = [prefix, name].compact.join(".")
+      index.nil? ? base : "#{base}[#{index}]"
+    end
+
+    # The value a probe `compensate` / `undo` body returns.
+    def rollback_result(mode, what)
+      case mode
+      when :fail then RubyReactor.Failure("#{what} failed")
+      when :raise then raise "#{what} raised"
+      else RubyReactor.Success()
+      end
+    end
+
+    # Same loop as RubyReactor::RSpec::SidekiqHelpers.drain_async_jobs, inlined
+    # because requiring that file pulls in the RSpec matchers.
+    def drain(max_iterations: 100)
+      max_iterations.times do
+        busy = false
+        WORKERS.each do |worker|
+          while (job = worker.jobs.shift)
+            worker.new.perform(*job["args"])
+            busy = true
+          end
+        end
+        return unless busy
+      end
+    end
+
+    def pending_jobs = WORKERS.sum { |w| w.jobs.size }
+
+    # Jobs run synchronously at enqueue (Sidekiq::Testing.inline!). Used only
+    # where a same-process reader must see a unit finish; labelled in the mode.
+    def inline_jobs(&block) = Sidekiq::Testing.inline!(&block)
+
+    # Run, drain every worker job, then reload the execution's terminal state.
+    def run_async(reactor_class, inputs = {})
+      dispatched = reactor_class.run(inputs)
+      drain
+      reactor_class.find(dispatched.execution_id)
+    end
+
+    # Prints Failure#rollback_failures as step/kind/reason; returns the result.
+    def rollback_note(result)
+      list = result.respond_to?(:rollback_failures) ? Array(result.rollback_failures) : []
+      note("rollback_failures=#{list.map { |f| "#{f[:step]}/#{f[:kind]}/#{f[:reason]}" }}")
+      result
+    end
+
+    def outcome(result)
+      result = result.result if result.is_a?(RubyReactor::Reactor)
+      case result
+      when String then result
+      when RubyReactor::Halt then "halt"
+      when RubyReactor::Success then "success"
+      when RubyReactor::Failure then "failure(#{result.step_name || "?"})"
+      when RubyReactor::InterruptResult then "paused"
+      when RubyReactor::DispatchResult then "dispatched"
+      when :unexecuted then "running"
+      else result.inspect
+      end
+    end
+
+    def reset!
+      RubyReactor.configuration.storage_adapter.instance_variable_get(:@redis).flushdb
+      WORKERS.each { |w| w.jobs.clear }
+      [@events, @notes].each(&:clear)
+      @counters.clear
+    end
+
+    # Runs one scenario and prints expected vs observed. The block returns the
+    # final result (a Result, a reloaded Reactor, or a String).
+    def scenario(id, title, mode:, expected:, &block)
+      return if ENV["PROBE"] && !id.include?(ENV["PROBE"])
+
+      reset!
+      observed = @events + ["=>", outcome(run_block(&block))]
+      report(id, title, mode, expected, observed)
+    end
+
+    private
+
+    def run_block(&block)
+      block.call
+    rescue StandardError => e
+      "raised(#{e.class}: #{e.message.lines.first&.strip})"
+    end
+
+    def report(id, title, mode, expected, observed)
+      ok = observed == expected
+      @tally[ok ? :match : :mismatch] += 1
+      puts "== #{id}  #{title}   [#{mode}]"
+      puts "expected: #{expected.join(" ")}"
+      puts "observed: #{observed.join(" ")}"
+      @notes.each { |n| puts "note: #{n}" }
+      puts ok ? "MATCH" : "MISMATCH"
+      puts
+    end
+  end
+
+  # Recording middleware: the shipped hook surface, no patching. Only events
+  # that say something about ordering relative to rollback are kept.
+  class Recorder < RubyReactor::Middleware
+    def on(event, *args)
+      case event
+      when :lock_acquired, :lock_released, :semaphore_acquired, :semaphore_released
+        Probe.rec("#{event}:#{args[0]}")
+      when :retry_attempt
+        Probe.rec("retry:#{args[0]}##{args[1]}")
+      end
+    end
+  end
+
+  # Reactor-class DSL sugar so a probe step is one line:
+  #
+  #   pstep :b, after: :a, fail: true
+  #   pstep :e1, idx: true, fail: ->(inputs) { inputs.i == 2 }
+  #   pstep :c, fail: :raise, compensate: :fail, undo: :raise
+  #   pstep :d, fail_times: 1, retries: { max_attempts: 2, base_delay: 0 }
+  module Steps
+    def tag(value = nil)
+      value ? @probe_tag = value : @probe_tag
+    end
+
+    # opts: fail:, fail_times:, retries: {…}, compensate: :ok|:fail|:raise, undo: (same),
+    #       kind: :step (default) | :async_step
+    def pstep(name, after: nil, idx: false, **opts, &extra)
+      body = Steps.body(tag, name, idx, opts)
+      on_compensate = Steps.rollback(tag, name, idx, "compensate", opts[:compensate])
+      on_undo = Steps.rollback(tag, name, idx, "undo", opts[:undo])
+      public_send(opts.fetch(:kind, :step), name) do
+        wait_for(*Array(after)) if after
+        argument :i, input(:i) if idx
+        retries(**opts[:retries]) if opts[:retries]
+        run(&body)
+        compensate(&on_compensate)
+        # 008 R-09: an async_step is never undone, so `undo` on one is rejected.
+        undo(&on_undo) unless opts[:kind] == :async_step
+        instance_eval(&extra) if extra
+      end
+    end
+
+    # `failure`: `fail: true | :raise | ->(inputs) {}` or `fail_times: n`.
+    def self.body(prefix, name, idx, failure)
+      lambda do |inputs, _ctx|
+        lbl = Probe.label(prefix, name, idx ? inputs.i : nil)
+        Probe.rec("run:#{lbl}")
+        failing = failing?(lbl, inputs, failure)
+        raise "boom #{lbl}" if failing && failure[:fail] == :raise
+
+        failing ? RubyReactor.Failure("boom #{lbl}") : RubyReactor.Success("#{lbl}-value")
+      end
+    end
+
+    def self.failing?(lbl, inputs, failure)
+      return (Probe.counters[lbl] += 1) <= failure[:fail_times] if failure[:fail_times]
+
+      fail = failure[:fail]
+      fail.respond_to?(:call) ? fail.call(inputs) : fail
+    end
+
+    def self.rollback(prefix, name, idx, kind, mode)
+      lambda do |_error_or_result, inputs, _ctx|
+        lbl = Probe.label(prefix, name, idx ? inputs.i : nil)
+        Probe.rec("#{kind}:#{lbl}")
+        Probe.rollback_result(mode, "#{kind} #{lbl}")
+      end
+    end
+  end
+end
+
+RubyReactor.configure do |config|
+  config.storage.adapter = :redis
+  config.storage.redis_url = Probe::REDIS_URL
+  config.async_router = RubyReactor::Adapters::Sidekiq::Router
+  config.logger = Logger.new(IO::NULL)
+  config.middlewares = [Probe::Recorder]
+  config.lock_snooze_jitter = 0
+  config.async_wait_timeout = 2
+end
+
+# Probe reactors live under P so workers can resolve them by constant name.
+module P
+  class Base < RubyReactor::Reactor
+    extend Probe::Steps
+  end
+end