fractor 0.1.0

data/README.adoc

@@ -0,0 +1,755 @@
+= Fractor: Function-driven Ractors framework
+
+Fractor is a lightweight Ruby framework designed to simplify the process of
+distributing computational work across multiple Ractors.
+
+== Introduction
+
+Fractor stands for *Function-driven Ractors framework*. It is a lightweight Ruby
+framework designed to simplify the process of distributing computational work
+across multiple Ractors (Ruby's actor-like concurrency model).
+
+The primary goal of Fractor is to provide a structured way to define work,
+process it in parallel using Ractors, and aggregate the results, while
+abstracting away much of the boilerplate code involved in Ractor management and
+communication.
+
+== Installation
+
+=== Using RubyGems
+
+[source,sh]
+----
+gem install fractor
+----
+
+=== Using Bundler
+
+Add this line to your application's Gemfile:
+
+[source,ruby]
+----
+gem 'fractor'
+----
+
+And then execute:
+
+[source,sh]
+----
+bundle install
+----
+
+
+=== Key concepts
+
+* *Function-driven:* You define the core processing logic by subclassing
+  `Fractor::Worker` and implementing the `process` method.
+
+* *Parallel execution:* Work items are automatically distributed to available
+  worker Ractors for concurrent processing.
+
+* *Result aggregation:* The framework collects both successful results and
+  errors from the workers.
+
+* *Separation of concerns:* Keeps the framework logic (`fractor.rb`) separate
+  from the client's specific implementation (`sample.rb`).
+
+== Scope
+
+This document describes the design, implementation, and usage of the Fractor
+framework. It provides detailed information about the framework's components,
+their interactions, and how to use them to implement parallel processing in Ruby
+applications.
+
+[bibliography]
+== Normative references
+
+* [[[ruby-ractor,Ruby Ractor Documentation]]], https://docs.ruby-lang.org/en/master/Ractor.html
+
+== Terms and definitions
+
+=== ractor
+
+concurrent programming abstraction in Ruby that enables parallel execution
+with thread safety
+
+[.source]
+<<ruby>>
+
+=== worker
+
+component that processes work items to produce work results
+
+=== work item
+
+unit of computation to be processed by a ractor
+
+=== work result
+
+result of processing a work item, either successful or an error
+
+=== work item class
+
+class that represents a work item, typically subclassing `Fractor::Work`
+
+=== worker class
+
+class that represents a worker, typically subclassing `Fractor::Worker`
+
+=== wrapped ractor
+
+component that manages a single ractor and its associated worker
+
+=== supervisor
+
+component that manages the pool of workers and distributes work items
+
+=== result aggregator
+
+component that collects and organizes work results from workers
+
+
+
+
+== Core components
+
+=== General
+
+The Fractor framework consists of the following main classes, all residing
+within the `Fractor` module.
+
+
+=== Fractor::Worker
+
+The abstract base class for defining how work should be processed.
+
+Client code must subclass this and implement the `process(work)` method.
+
+The `process` method receives a `Fractor::Work` object (or a subclass) and
+   should return a `Fractor::WorkResult` object.
+
+=== Fractor::Work
+
+The abstract base class for representing a unit of work.
+
+Typically holds the input data needed by the `Worker`.
+
+Client code should subclass this to define specific types of work items.
+
+=== Fractor::WorkResult
+
+A container object returned by the `Worker#process` method.
+
+Holds either the successful `:result` of the computation or an `:error`
+message if processing failed.
+
+Includes a reference back to the original `:work` item.
+
+Provides a `success?` method.
+
+=== Fractor::ResultAggregator
+
+Collects and stores all `WorkResult` objects generated by the workers.
+
+Separates results into `results` (successful) and `errors` arrays.
+
+=== Fractor::WrappedRactor
+
+Manages an individual Ruby `Ractor`.
+
+Instantiates the client-provided `Worker` subclass within the Ractor.
+
+Handles receiving `Work` items, calling the `Worker#process` method, and
+yielding `WorkResult` objects (or errors) back to the `Supervisor`.
+
+=== Fractor::Supervisor
+
+The main orchestrator of the framework.
+
+Initializes and manages a pool of `WrappedRactor` instances.
+
+Manages a `work_queue` of input data.
+
+Distributes work items (wrapped in the client's `Work` subclass) to available
+Ractors.
+
+Listens for results and errors from Ractors using `Ractor.select`.
+
+Uses `ResultAggregator` to store outcomes.
+
+Handles graceful shutdown on `SIGINT` (Ctrl+C).
+
+
+
+== Quick start guide
+
+=== General
+
+This quick start guide shows the minimum steps needed to get a simple parallel
+execution working with Fractor.
+
+=== Step 1: Create a minimal Work class
+
+The Work class represents a unit of work to be processed by a Worker. It
+encapsulates the input data needed for processing.
+
+[source,ruby]
+----
+require 'fractor'
+
+class MyWork < Fractor::Work
+  # The base class already provides input storage and basic functionality
+  # You can optionally override to_s for better debugging
+
+  def initialize(input)
+    super # This stores input in @input
+    # Add any additional initialization or replace @input with your own logic
+  end
+
+  def to_s
+    "MyWork: #{@input}"
+  end
+end
+----
+
+A Work is instantiated with the input data it will process this way:
+
+[source,ruby]
+----
+work_item = MyWork.new(42)
+puts work_item.to_s  # Output: MyWork: 42
+----
+
+
+=== Step 2: Create a minimal Worker class
+
+The Worker class defines the processing logic for work items. Each Worker
+instance runs within its own Ractor and processes Work objects sent to it.
+
+It must implement the `process(work)` method, which takes a Work object as
+input and returns a `Fractor::WorkResult` object.
+
+The `process` method should handle both successful processing and error
+conditions.
+
+[source,ruby]
+----
+class MyWorker < Fractor::Worker
+  def process(work)
+    # Your processing logic here
+    result = work.input * 2
+
+    # Return a success result
+    Fractor::WorkResult.new(result: result, work: work)
+  rescue => e
+    # Return an error result if something goes wrong
+    Fractor::WorkResult.new(error: e.message, work: work)
+  end
+end
+----
+
+The `process` method can perform any computation you need. In this example, it
+multiplies the input by 2. If an error occurs, it catches the exception and
+returns an error result.
+
+=== Step 3: Set up and run the Supervisor
+
+The Supervisor class orchestrates the entire framework, managing worker Ractors,
+distributing work, and collecting results.
+
+It initializes a pool of Ractors, each running an instance of the Worker
+class. The Supervisor handles the communication between the main thread and
+the Ractors, including sending work items and receiving results.
+
+The Supervisor also manages the work queue and the ResultAggregator, which
+collects and organizes all results from the workers.
+
+To set up the Supervisor, you need to specify the Worker and Work classes you
+created earlier. You can also specify the number of parallel Ractors to use.
+The default is 2, but you can increase this for more parallelism.
+
+[source,ruby]
+----
+# Create the supervisor
+supervisor = Fractor::Supervisor.new(
+  worker_class: MyWorker,
+  work_class: MyWork,
+  num_workers: 4  # Number of parallel Ractors
+)
+
+# Add work items (raw data)
+supervisor.add_work([1, 2, 3, 4, 5].map { |i| MyWork.new(i) })
+
+# Run the processing
+supervisor.run
+
+# Access results
+puts "Results: #{supervisor.results.results.map(&:result)}"
+puts "Errors: #{supervisor.results.errors.size}"
+----
+
+That's it! With these three simple steps, you have a working parallel processing
+system using Fractor.
+
+
+== Detailed guides
+
+=== Work class
+
+==== Purpose and responsibilities
+
+The `Fractor::Work` class represents a unit of work to be processed by a Worker.
+Its primary responsibility is to encapsulate the input data needed for
+processing.
+
+==== Implementation requirements
+
+At minimum, your Work subclass should:
+
+. Inherit from `Fractor::Work`
+. Pass the input data to the superclass constructor
+
+[source,ruby]
+----
+class MyWork < Fractor::Work
+  def initialize(input)
+    super(input) # This stores input in @input
+    # Add any additional initialization if needed
+  end
+end
+----
+
+==== Advanced usage
+
+You can extend your Work class to include additional data or methods:
+
+[source,ruby]
+----
+class ComplexWork < Fractor::Work
+  attr_reader :options
+
+  def initialize(input, options = {})
+    super(input)
+    @options = options
+  end
+
+  def high_priority?
+    @options[:priority] == :high
+  end
+
+  def to_s
+    "ComplexWork: #{@input} (#{@options[:priority]} priority)"
+  end
+end
+----
+
+[TIP]
+====
+* Keep Work objects lightweight and serializable since they will be passed
+  between Ractors
+* Implement a meaningful `to_s` method for better debugging
+* Consider adding validation in the initializer to catch issues early
+====
+
+=== Worker class
+
+==== Purpose and responsibilities
+
+The `Fractor::Worker` class defines the processing logic for work items. Each
+Worker instance runs within its own Ractor and processes Work objects sent to
+it.
+
+==== Implementation requirements
+
+Your Worker subclass must:
+
+. Inherit from `Fractor::Worker`
+. Implement the `process(work)` method
+. Return a `Fractor::WorkResult` object from the `process` method
+. Handle both successful processing and error conditions
+
+[source,ruby]
+----
+class MyWorker < Fractor::Worker
+  def process(work)
+    # Process the work
+
+    if work.input < 0
+      return Fractor::WorkResult.new(
+        error: "Cannot process negative numbers",
+        work: work
+      )
+    end
+
+    # Normal processing...
+    result = work.input * 2
+
+    # Return a WorkResult
+    Fractor::WorkResult.new(result: result, work: work)
+  end
+end
+----
+
+
+==== Error handling
+
+The Worker class should handle two types of errors.
+
+
+===== Handled errors
+
+These are expected error conditions that your code explicitly checks for.
+
+[source,ruby]
+----
+def process(work)
+  if work.input < 0
+    return Fractor::WorkResult.new(
+      error: "Cannot process negative numbers",
+      work: work
+    )
+  end
+
+  # Normal processing...
+  Fractor::WorkResult.new(result: calculated_value, work: work)
+end
+----
+
+=== Unexpected errors caught by rescue
+
+These are unexpected exceptions that may occur during processing. You should
+catch these and convert them into error results.
+
+[source,ruby]
+----
+def process(work)
+  # Processing that might raise exceptions
+  result = complex_calculation(work.input)
+
+  Fractor::WorkResult.new(result: result, work: work)
+rescue StandardError => e
+  # Catch and convert any unexpected exceptions to error results
+  Fractor::WorkResult.new(error: "An unexpected error occurred: #{e.message}", work: work)
+end
+----
+
+[TIP]
+* Keep the `process` method focused on a single responsibility
+* Use meaningful error messages that help diagnose issues
+* Consider adding logging within the `process` method for debugging
+* Ensure all paths return a valid `WorkResult` object
+
+=== WorkResult class
+
+==== Purpose and responsibilities
+
+The `Fractor::WorkResult` class is a container that holds either the successful
+result of processing or an error message, along with a reference to the original
+work item.
+
+==== Creating results
+
+To create a successful result:
+
+[source,ruby]
+----
+# For successful processing
+Fractor::WorkResult.new(result: calculated_value, work: work_object)
+----
+
+To create an error result:
+
+[source,ruby]
+----
+# For error conditions
+Fractor::WorkResult.new(error: "Error message", work: work_object)
+----
+
+==== Checking result status
+
+You can check if a result was successful:
+
+[source,ruby]
+----
+if work_result.success?
+  # Handle successful result
+  processed_value = work_result.result
+else
+  # Handle error
+  error_message = work_result.error
+end
+----
+
+==== Accessing original work
+
+The original work item is always available:
+
+[source,ruby]
+----
+original_work = work_result.work
+input_value = original_work.input
+----
+
+=== ResultAggregator class
+
+==== Purpose and responsibilities
+
+The `Fractor::ResultAggregator` collects and organizes all results from the
+workers, separating successful results from errors.
+
+Completed work results may be order independent or order dependent.
+
+* For order independent results, the results may be utilized (popped) as they
+are received.
+
+* For order dependent results, the results are aggregated in the order they
+are received. The order of results is important for re-assembly or
+further processing.
+
+* For results that require aggregation, the `ResultsAggregator` is used to determine
+whether the results are completed, which signify that all work items have
+been processed and ready for further processing.
+
+
+==== Accessing results
+
+To access successful results:
+
+[source,ruby]
+----
+# Get all successful results
+successful_results = supervisor.results.results
+
+# Extract just the result values
+result_values = successful_results.map(&:result)
+----
+
+To access errors:
+
+[source,ruby]
+----
+# Get all error results
+error_results = supervisor.results.errors
+
+# Extract error messages
+error_messages = error_results.map(&:error)
+
+# Get the work items that failed
+failed_work_items = error_results.map(&:work)
+----
+
+
+[TIP]
+====
+* Check both successful results and errors after processing completes
+* Consider implementing custom reporting based on the aggregated results
+====
+
+
+=== WrappedRactor class
+
+==== Purpose and responsibilities
+
+The `Fractor::WrappedRactor` class manages an individual Ruby Ractor, handling
+the communication between the Supervisor and the Worker instance running inside
+the Ractor.
+
+==== Usage notes
+
+This class is primarily used internally by the Supervisor, but understanding its
+role helps with debugging:
+
+* Each WrappedRactor creates and manages one Ractor
+* The Worker instance lives inside the Ractor
+* Work items are sent to the Ractor via the WrappedRactor's `send` method
+* Results are yielded back to the Supervisor
+
+==== Error propagation
+
+The WrappedRactor handles error propagation in two ways:
+
+. Errors from the Worker's `process` method are wrapped in a WorkResult and
+  yielded back
+. Unexpected errors in the Ractor itself are caught and logged
+
+=== Supervisor class
+
+==== Purpose and responsibilities
+
+The `Fractor::Supervisor` class orchestrates the entire framework, managing
+worker Ractors, distributing work, and collecting results.
+
+==== Configuration options
+
+When creating a Supervisor, you can configure:
+
+[source,ruby]
+----
+supervisor = Fractor::Supervisor.new(
+  worker_class: MyWorker,     # Required: Your Worker subclass
+  work_class: MyWork,         # Required: Your Work subclass
+  num_workers: 4              # Optional: Number of Ractors (default: 2)
+)
+----
+
+==== Adding work
+
+You can add work items individually or in batches:
+
+[source,ruby]
+----
+# Add a single item
+supervisor.add_work([42])
+
+# Add multiple items
+supervisor.add_work([1, 2, 3, 4, 5])
+
+# Add complex items
+supervisor.add_work([
+  {id: 1, data: "foo"},
+  {id: 2, data: "bar"}
+])
+----
+
+==== Running and monitoring
+
+To start processing:
+
+[source,ruby]
+----
+# Start processing and block until complete
+supervisor.run
+----
+
+The Supervisor automatically handles:
+
+* Starting the worker Ractors
+* Distributing work items to available workers
+* Collecting results and errors
+* Graceful shutdown on completion or interruption (Ctrl+C)
+
+==== Accessing results
+
+After processing completes:
+
+[source,ruby]
+----
+# Get the ResultAggregator
+aggregator = supervisor.results
+
+# Check counts
+puts "Processed #{aggregator.results.size} items successfully"
+puts "Encountered #{aggregator.errors.size} errors"
+
+# Access successful results
+aggregator.results.each do |result|
+  puts "Work item #{result.work.input} produced #{result.result}"
+end
+
+# Access errors
+aggregator.errors.each do |error_result|
+  puts "Work item #{error_result.work.input} failed: #{error_result.error}"
+end
+----
+
+==== Advanced usage patterns
+
+===== Custom work distribution
+
+For more complex scenarios, you might want to prioritize certain work items:
+
+[source,ruby]
+----
+# Add high-priority items first
+supervisor.add_work(high_priority_items)
+
+# Run with just enough workers for high-priority items
+supervisor.run
+
+# Add and process lower-priority items
+supervisor.add_work(low_priority_items)
+supervisor.run
+----
+
+===== Handling large datasets
+
+For very large datasets, consider processing in batches:
+
+[source,ruby]
+----
+large_dataset.each_slice(1000) do |batch|
+  supervisor.add_work(batch)
+  supervisor.run
+
+  # Process this batch's results before continuing
+  process_batch_results(supervisor.results)
+end
+----
+
+== Running the example
+
+. Install the gem as described in the Installation section.
+
+. Create a new Ruby file (e.g., `my_fractor_example.rb`) with your implementation:
+
+[source,ruby]
+----
+require 'fractor'
+
+# Define your Work class
+class MyWork < Fractor::Work
+  def to_s
+    "MyWork: #{@input}"
+  end
+end
+
+# Define your Worker class
+class MyWorker < Fractor::Worker
+  def process(work)
+    if work.input == 5
+      # Return a Fractor::WorkResult for errors
+      return Fractor::WorkResult.new(error: "Error processing work #{work.input}", work: work)
+    end
+
+    calculated = work.input * 2
+    # Return a Fractor::WorkResult for success
+    Fractor::WorkResult.new(result: calculated, work: work)
+  end
+end
+
+# Create supervisor
+supervisor = Fractor::Supervisor.new(
+  worker_class: MyWorker,
+  work_class: MyWork,
+  num_workers: 2
+)
+
+# Add work items (1..10)
+supervisor.add_work((1..10).to_a)
+
+# Run processing
+supervisor.run
+
+# Display results
+puts "Results: #{supervisor.results.results.map(&:result).join(', ')}"
+puts "Errors: #{supervisor.results.errors.map { |e| e.work.input }.join(', ')}"
+----
+
+. Run the example from your terminal:
+
+[source,sh]
+----
+ruby my_fractor_example.rb
+----
+
+You will see output showing Ractors starting, receiving work, processing it, and
+the final aggregated results, including any errors encountered. Press `Ctrl+C`
+during execution to test the graceful shutdown.
+
+
+
+== Copyright and license
+
+Copyright Ribose.
+
+Licensed under the MIT License.