fractor 0.1.6 → 0.1.7

data/examples/auto_detection/README.adoc

@@ -1,31 +1,158 @@
 = Auto-Detection Example
 
-This example demonstrates Fractor's automatic worker detection feature.
-
 == Purpose
 
-Shows how Fractor can automatically detect the number of available processors on your system and create the optimal number of workers without manual configuration.
+Demonstrates Fractor's automatic worker detection feature, showing how the framework adapts to different hardware environments by automatically detecting and utilizing available CPU cores.
+
+== Focus
 
-== What This Demonstrates
+This example focuses on demonstrating:
 
-* How Fractor automatically detects the number of available processors
+* Automatic detection of available processors using `Etc.nprocessors`
 * Comparison between auto-detection and explicit worker configuration
-* Mixed configuration (some pools with auto-detection, some explicit)
-* How to verify the number of workers being used
+* Mixed configuration strategies (combining both approaches)
+* Resource utilization optimization without manual tuning
+* Portable code that adapts to different environments
+
+== Architecture
+
+.Auto-Detection vs Explicit Configuration
+[source]
+----
+System Hardware
+    │
+    ├─→ Etc.nprocessors
+    │       │
+    │       └─→ Returns: 8 (example)
+    │
+    ▼
+┌─────────────────────────────────────────────┐
+│         Fractor Supervisor                  │
+└─────────────────────────────────────────────┘
+    │
+    ├─→ Auto-Detection Mode
+    │       │
+    │       └─→ Worker Pool { worker_class: ComputeWorker }
+    │               │
+    │               └─→ Creates 8 Ractors (auto-detected)
+    │                       │
+    │                       ├─→ Ractor 1 (ComputeWorker)
+    │                       ├─→ Ractor 2 (ComputeWorker)
+    │                       ├─→ Ractor 3 (ComputeWorker)
+    │                       ├─→ ...
+    │                       └─→ Ractor 8 (ComputeWorker)
+    │
+    └─→ Explicit Configuration Mode
+            │
+            └─→ Worker Pool { worker_class: ComputeWorker, num_workers: 4 }
+                    │
+                    └─→ Creates 4 Ractors (explicit)
+                            │
+                            ├─→ Ractor 1 (ComputeWorker)
+                            ├─→ Ractor 2 (ComputeWorker)
+                            ├─→ Ractor 3 (ComputeWorker)
+                            └─→ Ractor 4 (ComputeWorker)
+----
+
+.Resource Utilization Comparison
+[source]
+----
+Machine A (4 cores)          Machine B (16 cores)
+────────────────────────────────────────────────────────
 
-== When to Use Auto-Detection
+Auto-Detection:
+  4 workers created            16 workers created
+  100% CPU utilization         100% CPU utilization
+  ✓ Optimal                    ✓ Optimal
 
-* For portable code that adapts to different environments
-* When you want optimal resource utilization without manual tuning
-* For development where the number of cores varies across machines
+Explicit (num_workers: 4):
+  4 workers created            4 workers created
+  100% CPU utilization         25% CPU utilization
+  ✓ Optimal                    ✗ Underutilized
 
-== When to Set Explicit Values
+Explicit (num_workers: 8):
+  8 workers created            8 workers created
+  200% oversubscribed          50% CPU utilization
+  ✗ Oversubscribed             ✗ Underutilized
+----
+
+== Key Components
+
+=== Auto-Detection Configuration
+
+Create supervisor without specifying `num_workers`:
+
+[source,ruby]
+----
+supervisor = Fractor::Supervisor.new(
+  worker_pools: [
+    { worker_class: ComputeWorker }  # <1>
+  ]
+)
+----
+<1> Omitting `num_workers` triggers auto-detection
 
-* When you need precise control over resource usage
-* For production environments with specific requirements
-* When limiting workers due to memory or other constraints
+Fractor internally uses:
+
+[source,ruby]
+----
+require 'etc'
+
+num_workers = Etc.nprocessors  # <1>
+# Creates num_workers Ractor instances
+----
+<1> Returns the number of available processors
+
+=== Explicit Configuration
+
+Specify exact number of workers:
+
+[source,ruby]
+----
+supervisor = Fractor::Supervisor.new(
+  worker_pools: [
+    { worker_class: ComputeWorker, num_workers: 4 }  # <1>
+  ]
+)
+----
+<1> Always creates exactly 4 workers regardless of hardware
+
+=== Mixed Configuration
+
+Combine both approaches in different worker pools:
+
+[source,ruby]
+----
+supervisor = Fractor::Supervisor.new(
+  worker_pools: [
+    { worker_class: FastWorker },              # <1>
+    { worker_class: SlowWorker, num_workers: 2 }  # <2>
+  ]
+)
+----
+<1> Auto-detects optimal workers for FastWorker
+<2> Limits SlowWorker to 2 workers (e.g., memory intensive)
 
-== Running the Example
+=== Worker Implementation
+
+Simple worker for demonstration:
+
+[source,ruby]
+----
+class ComputeWorker < Fractor::Worker
+  def process(work)
+    result = work.value * work.value  # <1>
+    Fractor::WorkResult.new(result: result, work: work)
+  rescue StandardError => e
+    Fractor::WorkResult.new(error: e, work: work)
+  end
+end
+----
+<1> Simple computation (squares the input value)
+
+== Usage
+
+Run the example from the project root:
 
 [source,shell]
 ----
@@ -34,19 +161,270 @@ ruby examples/auto_detection/auto_detection.rb
 
 == Expected Output
 
-The script will:
+[example]
+====
+[source]
+----
+================================================================================
+Fractor Auto-Detection Example
+================================================================================
+
+System Information:
+  Available processors: 8
+
+--------------------------------------------------------------------------------
+Example 1: Auto-Detection
+--------------------------------------------------------------------------------
+Creating supervisor WITHOUT specifying num_workers...
+Fractor will automatically detect and use 8 workers
+
+Processing 10 work items with auto-detected workers...
+Results: 1, 4, 9, 16, 25, 36, 49, 64, 81, 100
+✓ Auto-detection successful!
+
+--------------------------------------------------------------------------------
+Example 2: Explicit Configuration
+--------------------------------------------------------------------------------
+Creating supervisor WITH explicit num_workers=4...
+
+Processing 10 work items with 4 explicitly configured workers...
+Results: 121, 144, 169, 196, 225, 256, 289, 324, 361, 400
+✓ Explicit configuration successful!
+
+--------------------------------------------------------------------------------
+Example 3: Mixed Auto-Detection and Explicit Configuration
+--------------------------------------------------------------------------------
+Creating supervisor with multiple worker pools:
+  - Pool 1: Auto-detected workers
+  - Pool 2: 2 explicitly configured workers
+
+Processing 10 work items with mixed configuration...
+Results: 441, 484, 529, 576, 625, 676, 729, 784, 841, 900
+✓ Mixed configuration successful!
+
+================================================================================
+Summary
+================================================================================
+
+Auto-detection provides:
+  ✓ Automatic adaptation to different environments
+  ✓ Optimal resource utilization by default
+  ✓ Less configuration needed
+  ✓ Portability across machines with different CPU counts
+
+Explicit configuration provides:
+  ✓ Precise control over worker count
+  ✓ Ability to limit resource usage
+  ✓ Predictable behavior in production
+
+Best practice: Use auto-detection for development and testing,
+               then tune explicitly for production if needed.
+
+================================================================================
+----
+====
+
+== Learning Points
+
+=== When to Use Auto-Detection
+
+Auto-detection is ideal for:
+
+* **Development environments**: Different developers may have different hardware
+* **Portable code**: Applications that run on various machines
+* **Default optimization**: When you want optimal performance without tuning
+* **CI/CD pipelines**: Runners may have varying CPU counts
+* **Container deployments**: CPU allocation may vary by environment
+
+Example use cases:
+[source,ruby]
+----
+# Development script - works well on any machine
+supervisor = Fractor::Supervisor.new(
+  worker_pools: [{ worker_class: DataProcessor }]
+)
+
+# CI pipeline - adapts to runner hardware
+supervisor = Fractor::Supervisor.new(
+  worker_pools: [{ worker_class: TestRunner }]
+)
+----
+
+=== When to Use Explicit Configuration
+
+Explicit configuration is better for:
+
+* **Production systems**: Predictable resource usage for capacity planning
+* **Resource-constrained environments**: Limit workers to avoid overwhelming system
+* **Memory-intensive tasks**: Fewer workers to prevent memory exhaustion
+* **I/O-bound tasks**: More workers than CPUs for better throughput
+* **Mixed workloads**: Different worker counts for different task types
+
+Example use cases:
+[source,ruby]
+----
+# Memory-intensive processing - limit workers
+supervisor = Fractor::Supervisor.new(
+  worker_pools: [
+    { worker_class: MemoryHeavyWorker, num_workers: 2 }
+  ]
+)
+
+# I/O-bound tasks - more workers than CPUs
+num_io_workers = Etc.nprocessors * 2  # <1>
+supervisor = Fractor::Supervisor.new(
+  worker_pools: [
+    { worker_class: ApiWorker, num_workers: num_io_workers }
+  ]
+)
+----
+<1> I/O-bound tasks benefit from more workers
+
+=== Auto-Detection Mechanism
+
+Fractor uses Ruby's `Etc.nprocessors`:
+
+[source,ruby]
+----
+require 'etc'
+
+# Returns number of available processors
+# - Physical cores on bare metal
+# - Virtual CPUs in VMs
+# - CPU quota in containers (if set)
+num_processors = Etc.nprocessors
+----
+
+Behavior in different environments:
+
+* **Bare metal**: Returns physical CPU cores
+* **Virtual machines**: Returns allocated vCPUs
+* **Docker containers**: Returns container CPU quota (if set) or host CPUs
+* **Kubernetes pods**: Returns CPU limit (if set) or node CPUs
+
+=== Performance Considerations
+
+==== CPU-Bound Tasks
+
+For CPU-intensive work, auto-detection provides optimal parallelism:
+
+[source,ruby]
+----
+# Good: One worker per CPU core
+supervisor = Fractor::Supervisor.new(
+  worker_pools: [{ worker_class: CpuIntensiveWorker }]
+)
+----
+
+==== I/O-Bound Tasks
+
+For I/O-heavy work, consider explicit over-subscription:
+
+[source,ruby]
+----
+# Better: More workers than CPUs for I/O tasks
+io_workers = Etc.nprocessors * 2
+supervisor = Fractor::Supervisor.new(
+  worker_pools: [
+    { worker_class: ApiClient, num_workers: io_workers }
+  ]
+)
+----
+
+==== Memory-Intensive Tasks
+
+For memory-heavy work, limit workers to prevent exhaustion:
+
+[source,ruby]
+----
+# Better: Fewer workers for memory-intensive tasks
+max_workers = [Etc.nprocessors / 2, 2].max  # <1>
+supervisor = Fractor::Supervisor.new(
+  worker_pools: [
+    { worker_class: LargeDataProcessor, num_workers: max_workers }
+  ]
+)
+----
+<1> Use half the CPUs, minimum of 2
+
+=== Mixed Configuration Strategy
+
+Combine auto-detection with explicit limits:
+
+[source,ruby]
+----
+supervisor = Fractor::Supervisor.new(
+  worker_pools: [
+    # Light CPU work - auto-detect
+    { worker_class: FastWorker },
+
+    # Heavy memory work - limit to 2
+    { worker_class: MemoryHeavyWorker, num_workers: 2 },
+
+    # I/O work - 2x CPUs
+    { worker_class: IoWorker, num_workers: Etc.nprocessors * 2 }
+  ]
+)
+----
+
+== Best Practices
+
+=== Development
+
+Use auto-detection for development:
+
+[source,ruby]
+----
+# Good for development - adapts to developer's machine
+supervisor = Fractor::Supervisor.new(
+  worker_pools: [{ worker_class: MyWorker }]
+)
+----
+
+=== Production
+
+Start with auto-detection, then tune based on monitoring:
+
+[source,ruby]
+----
+# Production approach
+num_workers = ENV.fetch('NUM_WORKERS', Etc.nprocessors).to_i
+
+supervisor = Fractor::Supervisor.new(
+  worker_pools: [
+    { worker_class: MyWorker, num_workers: num_workers }
+  ]
+)
+----
+
+This allows:
+- Auto-detection by default
+- Environment-based override for production tuning
+- Easy testing of different worker counts
+
+=== Container Environments
+
+In containers, consider CPU quotas:
+
+[source,ruby]
+----
+# Respect container CPU limits
+def optimal_workers
+  # Auto-detection respects container quotas
+  Etc.nprocessors
+end
+
+supervisor = Fractor::Supervisor.new(
+  worker_pools: [
+    { worker_class: MyWorker, num_workers: optimal_workers }
+  ]
+)
+----
 
-1. Display the number of processors detected on your system
-2. Run three examples:
-   * Example 1: Auto-detection (uses all available processors)
-   * Example 2: Explicit configuration (uses exactly 4 workers)
-   * Example 3: Mixed configuration (combines both approaches)
-3. Show results from processing work items in parallel
-4. Provide a summary of benefits for each approach
+== Next Steps
 
-== Key Takeaways
+After understanding auto-detection, explore:
 
-* Auto-detection provides automatic adaptation to different environments
-* Explicit configuration provides precise control when needed
-* You can mix both approaches in the same supervisor
-* Best practice: use auto-detection for development, tune for production if needed
+* link:../simple/README.adoc[Simple Example] - Basic Fractor concepts
+* link:../specialized_workers/README.adoc[Specialized Workers] - Multiple worker pools with different configurations
+* link:../multi_work_type/README.adoc[Multi Work Type] - Handling different work types efficiently

data/examples/continuous_chat_common/message_protocol.rb

@@ -41,7 +41,7 @@ module ContinuousChat
     def self.parse_packet(json_string)
       data = JSON.parse(json_string)
       type = data["type"]&.to_sym
-      content = data["data"]
+      content = data["data"] || {}
       timestamp = data["timestamp"] || Time.now.to_i
 
       MessagePacket.new(type, content, timestamp)

data/examples/error_reporting.rb

@@ -0,0 +1,207 @@
+# frozen_string_literal: true
+
+require_relative "../lib/fractor"
+require_relative "../lib/fractor/error_reporter"
+
+# Example demonstrating comprehensive error reporting and analytics
+#
+# This example shows how to:
+# 1. Set up an ErrorReporter to track errors
+# 2. Record successes and failures
+# 3. Generate comprehensive error reports
+# 4. Export metrics to Prometheus format
+# 5. Set up error handlers for real-time notifications
+
+# Simulate various types of workers
+class NetworkWorker < Fractor::Worker
+  def process(work)
+    # Simulate network errors
+    if work.input[:fail]
+      Fractor::WorkResult.new(
+        error: SocketError.new("Connection refused"),
+        error_code: :connection_refused,
+        error_context: { endpoint: work.input[:endpoint], attempt: 1 },
+        work: work,
+      )
+    else
+      Fractor::WorkResult.new(result: "Data fetched", work: work)
+    end
+  end
+end
+
+class ValidationWorker < Fractor::Worker
+  def process(work)
+    # Simulate validation errors
+    if work.input[:invalid]
+      Fractor::WorkResult.new(
+        error: ArgumentError.new("Invalid input"),
+        error_code: :validation_failed,
+        error_context: { field: "email", value: work.input[:value] },
+        work: work,
+      )
+    else
+      Fractor::WorkResult.new(result: "Valid", work: work)
+    end
+  end
+end
+
+class CriticalWorker < Fractor::Worker
+  def process(work)
+    # Simulate critical errors
+    if work.input[:critical]
+      Fractor::WorkResult.new(
+        error: SystemStackError.new("Stack overflow"),
+        error_severity: :critical,
+        error_context: { stack_depth: 10000 },
+        work: work,
+      )
+    else
+      Fractor::WorkResult.new(result: "Success", work: work)
+    end
+  end
+end
+
+puts "=" * 80
+puts "Error Reporting Example"
+puts "=" * 80
+puts ""
+
+# Initialize error reporter
+reporter = Fractor::ErrorReporter.new
+
+# Set up error handler for real-time notifications
+reporter.on_error do |work_result, job_name|
+  if work_result.critical?
+    puts "🚨 CRITICAL ERROR DETECTED!"
+    puts "   Job: #{job_name || 'unknown'}"
+    puts "   Error: #{work_result.error.message}"
+    puts ""
+  end
+end
+
+puts "1. Recording various work results..."
+puts "-" * 80
+
+# Simulate successful work
+5.times do |i|
+  work = Fractor::Work.new({ endpoint: "api.example.com", id: i })
+  result = NetworkWorker.new.process(work)
+  reporter.record(result, job_name: "network_job")
+end
+puts "✓ Recorded 5 successful network operations"
+
+# Simulate network errors
+3.times do |i|
+  work = Fractor::Work.new({ fail: true, endpoint: "api.example.com", id: i })
+  result = NetworkWorker.new.process(work)
+  reporter.record(result, job_name: "network_job")
+end
+puts "✗ Recorded 3 network errors"
+
+# Simulate validation errors
+4.times do |i|
+  work = Fractor::Work.new({ invalid: true, value: "bad-email-#{i}" })
+  result = ValidationWorker.new.process(work)
+  reporter.record(result, job_name: "validation_job")
+end
+puts "✗ Recorded 4 validation errors"
+
+# Simulate critical error
+work = Fractor::Work.new({ critical: true })
+result = CriticalWorker.new.process(work)
+reporter.record(result, job_name: "critical_job")
+puts "🚨 Recorded 1 critical error"
+
+# Add some successful validations
+3.times do |i|
+  work = Fractor::Work.new({ valid: true, value: "good-email-#{i}@example.com" })
+  result = ValidationWorker.new.process(work)
+  reporter.record(result, job_name: "validation_job")
+end
+puts "✓ Recorded 3 successful validations"
+
+puts ""
+puts "2. Overall Statistics"
+puts "-" * 80
+puts "Total Successes: #{reporter.total_successes}"
+puts "Total Errors:    #{reporter.total_errors}"
+puts "Error Rate:      #{reporter.overall_error_rate}%"
+puts ""
+
+puts "3. Top Error Categories"
+puts "-" * 80
+reporter.top_categories.each do |category, count|
+  puts "#{category.to_s.ljust(15)}: #{count} errors"
+end
+puts ""
+
+puts "4. Top Error Jobs"
+puts "-" * 80
+reporter.top_jobs.each do |job, count|
+  puts "#{job.to_s.ljust(20)}: #{count} errors"
+end
+puts ""
+
+puts "5. Critical Errors"
+puts "-" * 80
+critical = reporter.critical_errors
+if critical.empty?
+  puts "No critical errors"
+else
+  critical.each do |error_info|
+    puts "Category: #{error_info[:category]}"
+    puts "Count:    #{error_info[:count]}"
+    puts "Recent:"
+    error_info[:recent].each do |err|
+      puts "  - #{err[:error_class]}: #{err[:error_message]}"
+    end
+  end
+end
+puts ""
+
+puts "6. Errors by Severity"
+puts "-" * 80
+reporter.errors_by_severity.each do |severity, count|
+  icon = case severity
+         when :critical then "🚨"
+         when :error then "❌"
+         when :warning then "⚠️"
+         else "ℹ️"
+         end
+  puts "#{icon} #{severity.to_s.ljust(10)}: #{count}"
+end
+puts ""
+
+puts "7. Job-Specific Statistics"
+puts "-" * 80
+reporter.top_jobs.each_key do |job_name|
+  stats = reporter.job_stats(job_name)
+  puts "Job: #{job_name}"
+  puts "  Total Errors:     #{stats[:total_count]}"
+  puts "  Error Rate:       #{stats[:error_rate]}/s"
+  puts "  Most Common Code: #{stats[:most_common_code]}"
+  puts "  Highest Severity: #{stats[:highest_severity]}"
+  puts "  Trend:            #{stats[:trending]}"
+  puts ""
+end
+
+puts "8. Formatted Text Report"
+puts "-" * 80
+puts ""
+puts reporter.formatted_report
+puts ""
+
+puts "9. Prometheus Metrics Export"
+puts "-" * 80
+puts reporter.to_prometheus
+puts ""
+
+puts "10. JSON Export"
+puts "-" * 80
+require "json"
+puts JSON.pretty_generate(reporter.report)
+puts ""
+
+puts "=" * 80
+puts "Example completed successfully!"
+puts "=" * 80