ruby_reactor 0.3.2 → 0.4.0

data/documentation/async_reactors.md

@@ -1,381 +0,0 @@
-# Async Reactors
-
-RubyReactor supports two asynchronous execution models: **Full Reactor Async** and **Step-Level Async**. Both models use Sidekiq for background processing with non-blocking retry mechanisms.
-
-## Overview
-
-Async execution provides several benefits:
-
-- **Non-blocking**: Workers are freed during retry delays
-- **Scalable**: Better resource utilization with large worker pools
-- **Reliable**: Automatic retry with configurable backoff strategies
-- **Observable**: Full visibility into execution state and retry attempts
-
-## Full Reactor Async
-
-When a reactor is marked as `async true`, the entire execution happens in a Sidekiq worker, including input validation.
-
-### Configuration
-
-```ruby
-class OrderProcessingReactor < RubyReactor::Reactor
-  async true  # Enable full reactor async
-
-  step :validate_order do
-    run { validate_order_logic }
-  end
-
-  step :process_payment do
-    run { process_payment_logic }
-  end
-
-  step :send_confirmation do
-    run { send_confirmation_logic }
-  end
-end
-```
-
-### Execution Flow
-
-```ruby
-# Synchronous call returns immediately
-async_result = OrderProcessingReactor.run(order_id: 123)
-
-async_result.execution_id       # UUID for reloading state
-async_result.intermediate_results # Whatever was computed before handoff
-
-# Inspect status later by reloading from storage
-reactor = OrderProcessingReactor.find(async_result.execution_id)
-case reactor.context.status.to_s
-when "running"   then puts "Execution is in progress"
-when "completed" then puts "Done: #{reactor.result.value}"
-when "failed"    then puts "Failed: #{reactor.result.error}"
-when "paused"    then puts "Waiting on interrupt"
-end
-```
-
-> **Note:** `AsyncResult` itself does not poll. It only carries the job handle and any results computed before handoff (`job_id`, `execution_id`, `intermediate_results`). To check progress, reload via `Reactor.find(execution_id)` and inspect `context.status`, or use the web dashboard.
-
-### Architecture
-
-```mermaid
-graph LR
-    A[Client] --> B[Reactor.run<br/>async: true]
-    B --> C[Validate Inputs<br/>Synchronously]
-    C --> D[Queue Sidekiq Job<br/>with Context]
-    D --> E[Sidekiq Worker]
-    E --> F[Deserialize Context]
-    F --> G[Execute All Steps<br/>Sequentially]
-    G --> H{Result?}
-    H -->|Success| I[Return Success]
-    H -->|Failure| J[Run Compensation<br/>in Worker]
-    J --> K[Return Failure]
-```
-
-```
-Client → Reactor.run() → Queue Job → Sidekiq Worker → Execute All Steps
-```
-
-## Step-Level Async
-
-Individual steps can be marked as `async: true`. Execution runs synchronously until the first async step, then hands off to a Sidekiq worker for all remaining execution.
-
-### Configuration
-
-```ruby
-class OrderProcessingReactor < RubyReactor::Reactor
-  step :validate_order do
-    # Runs synchronously
-    run { |args, _ctx| validate_order_logic(args) }
-  end
-
-  step :process_payment do
-    async true # First async step — triggers handoff to worker
-    run { |args, _ctx| process_payment_logic(args) }
-  end
-
-  step :update_inventory do
-    # Runs in worker after handoff
-    run { |args, _ctx| update_inventory_logic(args) }
-  end
-
-  step :send_confirmation do
-    # Runs in same worker
-    run { |args, _ctx| send_confirmation_logic(args) }
-  end
-end
-```
-
-> **DSL note:** `async` is declared inside the step block (`async true`), not as a keyword on `step :name, async: true`. The `step` method only accepts a step name and an optional implementation class as positional arguments.
-
-### Execution Flow
-
-```ruby
-# Runs validate_order synchronously, then hands off
-async_result = OrderProcessingReactor.run(order_id: 123)
-
-# All remaining steps execute in a single worker
-# Compensation and rollback work within the worker context
-```
-
-### Architecture
-
-```mermaid
-graph LR
-    A[Client] --> B[Reactor.run]
-    B --> C[Execute Sync Steps<br/>Until First Async]
-    C --> D{First Async<br/>Step Found?}
-    D -->|No| E[Execute All Steps<br/>Synchronously]
-    D -->|Yes| F[Queue Sidekiq Job<br/>with Context]
-    F --> G[Sidekiq Worker]
-    G --> H[Execute Remaining Steps<br/>Sequentially]
-    H --> I{Result?}
-    I -->|Success| J[Return Success]
-    I -->|Failure| K[Run Compensation<br/>in Worker]
-    K --> L[Return Failure]
-```
-
-```
-Client → Reactor.run() → Sync Steps → Queue Job → Worker → Remaining Steps
-```
-
-## Async Steps
-
-Individual steps can be configured with `async true`, which changes the execution behavior at the point where the first async step is encountered.
-
-### Key Behavior
-
-When an async step is encountered during synchronous execution:
-
-1. **All previous steps have already executed synchronously** in the main thread
-2. **The Sidekiq job is queued** at the moment the async step would be executed
-3. **The async step itself and all subsequent steps execute** in the Sidekiq worker
-4. **The main thread returns immediately** with an async result handle
-
-### Important Distinction
-
-- **Before async step**: All execution is synchronous
-- **At async step**: Job is queued instead of executing the step
-- **After async step**: All remaining execution happens in the worker
-
-### Example
-
-```ruby
-class OrderProcessingReactor < RubyReactor::Reactor
-  step :validate_input do
-    # Executes synchronously in main thread
-    run { validate_order_input }
-  end
-
-  step :check_inventory do
-    # Executes synchronously in main thread
-    run { check_inventory_levels }
-  end
-
-  step :process_payment do
-    async true
-    # Job is queued here - this step executes in worker
-    run { process_payment_logic }
-  end
-
-  step :update_inventory do
-    # Executes in worker
-    run { update_inventory_records }
-  end
-
-  step :send_notification do
-    # Executes in worker
-    run { send_order_confirmation }
-  end
-end
-```
-
-### Execution Flow with Mermaid
-
-```mermaid
-sequenceDiagram
-    participant Client
-    participant Reactor
-    participant Sidekiq
-    participant Worker
-
-    Client->>Reactor: run(order_data)
-    Reactor->>Reactor: Execute validate_input (sync)
-    Reactor->>Reactor: Execute check_inventory (sync)
-    Reactor->>Reactor: Reach process_payment (async: true)
-    Reactor->>Sidekiq: Queue job with context
-    Reactor->>Client: Return AsyncResult (pending)
-    Sidekiq->>Worker: Process job
-    Worker->>Worker: Execute process_payment
-    Worker->>Worker: Execute update_inventory
-    Worker->>Worker: Execute send_notification
-    Worker->>Sidekiq: Mark job complete
-```
-
-```mermaid
-graph TD
-    A[Client Calls Reactor.run] --> B[Execute Previous Steps Synchronously]
-    B --> C[Encounter First Async Step]
-    C --> D[Queue Sidekiq Job<br/>with Current Context]
-    D --> E[Return AsyncResult to Client<br/>Status: :pending]
-    D --> F[Sidekiq Worker Receives Job]
-    F --> G[Deserialize Context]
-    G --> H[Execute Async Step<br/>and All Subsequent Steps]
-    H --> I{Execution<br/>Successful?}
-    I -->|Yes| J[Mark AsyncResult<br/>Status: :success]
-    I -->|No| K[Run Compensation<br/>in Worker Context]
-    K --> L[Mark AsyncResult<br/>Status: :failed]
-```
-
-### Critical Points
-
-- **Synchronous Prefix Guarantee**: Steps before the first async step always complete synchronously
-- **Single Handoff Point**: Only one job is queued per reactor execution
-- **Worker Execution**: The async step and all following steps run in the same worker
-- **Context Preservation**: Execution state is serialized and passed to the worker
-- **Compensation Scope**: All compensation for failed async execution happens in the worker
-
-## Retry Configuration
-
-Both async models support sophisticated retry mechanisms with non-blocking job requeuing.
-
-### Step-Level Retry
-
-```ruby
-class PaymentProcessingReactor < RubyReactor::Reactor
-  async true
-
-  step :validate_payment do
-    retries max_attempts: 3, backoff: :exponential, base_delay: 1.second
-    run { |args, _ctx| validate_payment_logic(args) }
-  end
-
-  step :charge_card do
-    async true
-    retries max_attempts: 5, backoff: :linear, base_delay: 5.seconds
-    run { |args, _ctx| charge_card_logic(args) }
-  end
-
-  step :update_records do
-    # No retry - critical step
-    run { |args, _ctx| update_records_logic(args) }
-  end
-end
-```
-
-### Reactor-Level Defaults
-
-```ruby
-class PaymentProcessingReactor < RubyReactor::Reactor
-  async true
-
-  # Set defaults for all steps
-  retry_defaults max_attempts: 3, backoff: :exponential, base_delay: 2.seconds
-
-  step :validate_payment do
-    # Inherits reactor defaults (3 attempts, exponential backoff)
-    run { validate_payment_logic }
-  end
-
-  step :charge_card do
-    # Override defaults for this step
-    retries max_attempts: 5, backoff: :linear, base_delay: 10.seconds
-    run { charge_card_logic }
-  end
-end
-```
-
-## Retry Strategies
-
-### Backoff Algorithms
-
-- **`:exponential`**: Delay doubles with each attempt (1s, 2s, 4s, 8s...)
-- **`:linear`**: Delay increases linearly (5s, 10s, 15s, 20s...)
-- **`:fixed`**: Same delay for each attempt (5s, 5s, 5s, 5s...)
-
-## Error Handling and Compensation
-
-Async reactors support full compensation and rollback in the worker context:
-
-```ruby
-class OrderProcessingReactor < RubyReactor::Reactor
-  async true
-
-  step :process_payment do
-    argument :order, result(:validate_order)
-    run { |args, _ctx| process_payment_logic(args[:order]) }
-
-    undo do |result, _args, _ctx|
-      # Runs in worker when a LATER step fails
-      PaymentService.refund(result[:payment_id])
-      Success()
-    end
-
-    compensate do |error, args, _ctx|
-      # Runs in worker if THIS step fails
-      AuditService.log_payment_failure(args[:order].id, error.message)
-      Success()
-    end
-  end
-
-  step :update_inventory do
-    argument :order, result(:validate_order)
-    run { |args, _ctx| update_inventory_logic(args[:order]) }
-
-    compensate do |_error, args, _ctx|
-      # Runs in worker on failure
-      InventoryService.restore(args[:order])
-      Success()
-    end
-  end
-end
-```
-
-## Monitoring and Observability
-
-### Job Visibility
-
-Retries are visible in Sidekiq web UI with:
-- Step name and attempt number
-- Retry delay and timing
-- Success/failure status
-- Execution context
-
-## Configuration
-
-### Sidekiq Worker Setup
-
-```ruby
-# config/initializers/ruby_reactor.rb (Rails) or load before booting Sidekiq
-RubyReactor.configure do |config|
-  config.storage.adapter = :redis
-  config.storage.redis_url = ENV.fetch("REDIS_URL", "redis://localhost:6379/0")
-
-  config.sidekiq_queue = :default
-  config.sidekiq_retry_count = 3
-  config.logger = Logger.new('log/ruby_reactor.log')
-end
-```
-
-Sidekiq workers are loaded automatically via Zeitwerk when `ruby_reactor` is required — no extra `require` is needed.
-
-## Performance Considerations
-
-### Worker Pool Sizing
-
-- **Full Reactor Async**: Size pool based on total reactor throughput
-- **Step-Level Async**: Size pool based on async step frequency
-
-### Context Size Limits
-
-- Redis has job size limits (~512MB)
-- TODO: Large contexts are automatically compressed
-- Consider external storage for very large execution states
-
-### Monitoring Metrics
-
-Track these key metrics:
-- Retry attempt counts per step
-- Average retry delays
-- Success rates after retries
-- Worker utilization during peak loads

data/documentation/composition.md

@@ -1,199 +0,0 @@
-# Composition
-
-RubyReactor allows you to compose reactors within other reactors using the `compose` DSL. This enables you to build complex workflows by reusing existing reactors or defining sub-workflows inline.
-
-## Inline Composition
-
-You can define a composed reactor inline using a block. This is useful for grouping related steps or defining a sub-workflow that doesn't need to be reused elsewhere.
-
-```ruby
-class UpdateUserReactor < RubyReactor::Reactor
-  input :user_id
-  input :profile_data
-
-  step :validate_user do
-    argument :user_id, input(:user_id)
-    run { |args| ... }
-  end
-
-  # Define a sub-workflow inline
-  compose :update_profile do
-    # You can define inputs for the inline reactor
-    argument :user_id, input(:user_id)
-    argument :data, input(:profile_data)
-
-    # Configure async execution for the sub-workflow
-    async true
-    
-    # Configure retries for steps within the sub-workflow
-    retries max_attempts: 3
-
-    step :update_bio do
-      argument :user_id, input(:user_id)
-      argument :bio, input(:data, :bio)
-      run { |args| ... }
-    end
-
-    step :update_avatar do
-      argument :user_id, input(:user_id)
-      argument :avatar, input(:data, :avatar)
-      run { |args| ... }
-    end
-  end
-
-  step :notify_user do
-    wait_for :update_profile
-    run { |args| ... }
-  end
-end
-```
-
-## Class-based Composition
-
-You can also compose an existing reactor class. This is ideal for reusable workflows.
-
-```ruby
-class ProfileUpdateReactor < RubyReactor::Reactor
-  input :user_id
-  input :data
-  
-  step :update_bio do ... end
-  step :update_avatar do ... end
-end
-
-class MainReactor < RubyReactor::Reactor
-  input :user_id
-  input :profile_data
-
-  # Compose the existing reactor
-  compose :update_profile, ProfileUpdateReactor do
-    argument :user_id, input(:user_id)
-    argument :data, input(:profile_data)
-  end
-end
-```
-
-## Multiple Compose Declarations
-
-A single reactor can include multiple `compose` declarations, allowing you to orchestrate several sub-workflows. You can mix both class-based and inline compositions, and combine them with regular steps.
-
-```ruby
-class OrderProcessingReactor < RubyReactor::Reactor
-  input :order_id
-  input :customer_data
-  input :payment_info
-
-  step :validate_order do
-    argument :order_id, input(:order_id)
-    run { |args| ... }
-  end
-
-  # First compose: Class-based reactor
-  compose :update_customer_profile, CustomerProfileReactor do
-    argument :customer_data, input(:customer_data)
-  end
-
-  # Second compose: Inline reactor
-  compose :process_payment do
-    argument :order_id, input(:order_id)
-    argument :payment_info, input(:payment_info)
-    
-    async true  # This sub-workflow can run async
-    
-    step :authorize_payment do
-      run { |args| ... }
-    end
-    
-    step :capture_payment do
-      run { |args| ... }
-    end
-  end
-
-  # Third compose: Another inline reactor
-  compose :allocate_inventory do
-    argument :order_id, input(:order_id)
-    argument :order, input(:validate_order)
-    
-    step :check_availability do
-      run { |args| ... }
-    end
-    
-    step :reserve_items do
-      run { |args| ... }
-    end
-  end
-
-  step :send_confirmation do
-    # Wait for all compose steps to complete
-    wait_for :update_customer_profile, :process_payment, :allocate_inventory
-    
-    argument :customer_email, input(:customer_data)
-    argument :order_id, input(:order_id)
-    run { |args| ... }
-  end
-end
-```
-
-### Execution Flow
-
-When you have multiple `compose` declarations:
-
-1. **Execution Order**: Composed reactors execute in topological order based on their dependencies. Dependencies are determined automatically when you reference results from other steps (using `result(:step_name)`), or explicitly using `wait_for`. If no dependencies exist.
-
-2. **Access Compose Results**: A compose step returns the final result of the composed reactor. By default, this is a hash containing all of its step results, unless the composed reactor uses the `returns` DSL to specify a custom return value:
-
-```ruby
-step :final_step do
-  # Get the complete result hash from the composed reactor
-  argument :payment_result, result(:process_payment)
-  
-  run { |args| 
-    # args[:payment_result] contains the full hash: 
-    # { authorize_payment: ..., capture_payment: ... }
-    # 
-    # args[:payment_status] contains just the capture_payment result
-  }
-end
-```
-
-3. **Async Execution**: If a composed reactor is marked with `async true`, execution will pause at that compose step, serialize the entire reactor context, and queue a background job. The worker will resume execution from that compose step and continue sequentially through remaining steps. Only one worker executes the main reactor at a time.
-
-4. **Shared Context**: All composed reactors share access to the parent reactor's inputs and results of previous steps and can be configured with different retry strategies.
-
-### Async Compose Execution Flow
-
-When you mark a compose as async:
-
-```ruby
-compose :process_payment do
-  async true
-  # ... steps
-end
-```
-
-The execution flow is:
-
-1. Parent reactor executes steps up to `process_payment`
-2. Serializes entire context and queues a background job
-3. Returns `AsyncResult` to caller
-4. Worker picks up job and resumes from `process_payment`
-5. After `process_payment` completes, continues to next step sequentially
-6. If another async step is encountered, the process repeats
-
-This ensures proper ordering and state consistency across async boundaries.
-
-## Nested Async Retries
-
-One of the powerful features of composition in RubyReactor is the handling of asynchronous retries within nested reactors.
-
-When a step inside a composed reactor fails and is configured to retry asynchronously (e.g., via Sidekiq), RubyReactor ensures that the entire execution context is preserved.
-
-1.  **Context Serialization**: The entire reactor tree, including the state of the parent reactor and the composed reactor, is serialized.
-2.  **Resume on Retry**: When the retry job executes, it resumes execution exactly from the failed step within the composed reactor.
-3.  **State Preservation**: All intermediate results and inputs from the parent reactor are available, ensuring that the composed reactor has everything it needs to complete.
-
-This behavior works seamlessly whether you are using inline composition or class-based composition.
-
-## Inspection
-
-The execution state of composed reactors is stored in the parent context under `composed_contexts`. This allows for inspection of the full execution tree, which is useful for debugging and building monitoring tools.