breaker_machines 0.2.1 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 137bce590971c9f523f97f4078b32c0ce1f6ac6aa4e524e7801a752a260fd37c
-  data.tar.gz: 4a3ed75814ff59705befee7493669e47c894be761d9e37c3fe05e1d3bf9457bf
+  metadata.gz: dc152fbf822d0eae49f25c2f8acea2c3268c141d8a62a7aa313168007499e8a5
+  data.tar.gz: 5e7d529cde4b41cbb32fca7502249c15465413c4fad31e9a29056fbc99196e09
 SHA512:
-  metadata.gz: 82752b18f244e21bb2c9f682bc93cf85e17e42eb488b02059302aa6990cf78573316e82dfab025efa4523ff397205f72cf8510426ab60d4f4fe4b419a5e50b7d
-  data.tar.gz: 712adb17cc2e47a920a952537b4bbb7af0649b604de996eeeb50d73e40a51c7b2620d4e53232aff6e1ed01acda2fef7d4ea894017ff632ac6f29eeaea29a43c1
+  metadata.gz: fa71a3f703c2d9814a836d2eb160eaf89f1ea991cf80d96a237aa9b59654d13d7b0459c41c4b8a5018fc27b83fd2a6729b7740238372a8691e0aff944f5965a3
+  data.tar.gz: 59c7fd005d672ad412f9824c4880f2fc9f0141fa56f0bc405d63f9f3a95f05278b54fdceb5e97e93a18116c4e4101dc80db730eecfcd854656e0324fb1074d3d

data/README.md

@@ -1,5 +1,7 @@
 # BreakerMachines
 
+> The circuit breaker that went where no Ruby has gone before! ⭐
+
 A battle-tested Ruby implementation of the Circuit Breaker pattern, built on `state_machines` for reliable distributed systems protection.
 
 ## Quick Start
@@ -26,6 +28,18 @@ class PaymentService
 end
 ```
 
+## A Message to the Resistance
+
+So AI took your job while you were waiting for Fireship to drop the next JavaScript framework?
+
+Welcome to April 2005—when Git was born, branches were just `master`, and nobody cared about your pronouns. This is the pattern your company's distributed systems desperately need, explained in a way that won't make you fall asleep and impulse-buy developer swag just to feel something.
+
+Still reading? Good. Because in space, nobody can hear you scream about microservices. It's all just patterns and pain.
+
+### The Pattern They Don't Want You to Know
+
+Built on the battle-tested `state_machines` gem, because I don't reinvent wheels here—I stop them from catching fire and burning down your entire infrastructure.
+
 ## Features
 
 - **Thread-safe** circuit breaker implementation
@@ -60,86 +74,89 @@ Built on the battle-tested `state_machines` gem, BreakerMachines provides produc
 
 See [Why I Open Sourced This](docs/WHY_OPEN_SOURCE.md) for the full story.
 
-## Production-Ready Features
+## Chapter 1: The Year is 2025 (Stardate 2025.186)
 
-### Hedged Requests
-Reduce latency by sending duplicate requests and using the first successful response:
+The Resistance huddles in the server rooms, the last bastion against the cascade failures. Outside, the microservices burn. Redis Ship Com is down. PostgreSQL Life Support is flatlining.
+
+And somewhere in the darkness, a junior developer is about to write:
 
 ```ruby
-circuit :api do
-  hedged do
-    delay 100           # Start second request after 100ms
-    max_requests 3      # Maximum parallel requests
+def fetch_user_data
+  retry_count = 0
+  begin
+    @redis.get(user_id)
+  rescue => e
+    retry_count += 1
+    retry if retry_count < Float::INFINITY  # "It'll work eventually"
   end
 end
 ```
 
-### Multiple Backends
-Configure automatic failover across multiple service endpoints:
+"This," whispers the grizzled ops engineer, "is how civilizations fall."
 
-```ruby
-circuit :multi_region do
-  backends [
-    -> { fetch_from_primary },
-    -> { fetch_from_secondary },
-    -> { fetch_from_tertiary }
-  ]
-end
-```
+## The Hidden State Machine
 
-### Percentage-Based Thresholds
-Open circuits based on error rates instead of absolute counts:
+They built this on `state_machines` because sometimes, Resistance, you need a tank, not another JavaScript framework.
 
-```ruby
-circuit :high_traffic do
-  threshold failure_rate: 0.5, minimum_calls: 10, within: 60
-end
-```
+See the [Circuit Breaker State Machine diagram](docs/DIAGRAMS.md#the-circuit-breaker-state-machine) for a visual representation of hope, despair, and the eternal cycle of production failures.
 
-### Dynamic Circuit Breakers
-Create circuit breakers at runtime for webhook delivery, API proxies, or per-tenant isolation:
+## What You Think You're Doing vs Reality
 
-```ruby
-class WebhookService
-  include BreakerMachines::DSL
+### You Think: "I'm implementing retry logic for resilience!"
+### Reality: You're DDOSing your own infrastructure
 
-  circuit_template :webhook_default do
-    threshold failures: 3, within: 1.minute
-    fallback { |error| { delivered: false, error: error.message } }
-  end
+See [The Retry Death Spiral diagram](docs/DIAGRAMS.md#the-retry-death-spiral) to understand how your well-intentioned retries become a self-inflicted distributed denial of service attack.
 
-  def deliver_webhook(url, payload)
-    domain = URI.parse(url).host
-    circuit_name = "webhook_#{domain}".to_sym
-    
-    dynamic_circuit(circuit_name, template: :webhook_default) do
-      # Custom per-domain configuration
-      if domain.include?('reliable-service.com')
-        threshold failures: 5, within: 2.minutes
-      end
-    end.wrap do
-      send_webhook(url, payload)
-    end
-  end
-end
-```
+## Advanced Features
+
+- **Hedged Requests** - Reduce latency with duplicate requests
+- **Multiple Backends** - Automatic failover across endpoints
+- **Percentage-Based Thresholds** - Open on error rates, not just counts
+- **Dynamic Circuit Breakers** - Runtime creation with templates
+- **Apocalypse-Resistant Storage** - Cascading fallbacks when Redis dies
+- **Custom Storage Backends** - SysV semaphores, distributed locks, etc.
+
+See [Advanced Patterns](docs/ADVANCED_PATTERNS.md) for detailed examples and implementation guides.
+
+## A Word from the RMNS Atlas Monkey
+
+*The Universal Commentary Engine crackles to life:*
+
+"In space, nobody can hear your pronouns. But they can hear your services failing.
 
-## Contributing
+The universe doesn't care about your bootcamp certificate or your Medium articles about 'Why I Switched to Rust.' It cares about one thing:
 
-1. Fork it
-2. Create your feature branch (`git checkout -b my-new-feature`)
-3. Commit your changes (`git commit -am 'Add some feature'`)
-4. Push to the branch (`git push origin my-new-feature`)
-5. Create new Pull Request
+Does your system stay up when Redis has a bad day?
+
+If not, welcome to the Resistance. We have circuit breakers.
+
+Remember: The pattern isn't about preventing failures—it's about failing fast, failing smart, and living to deploy another day.
+
+As I always say when contemplating the void: 'It's better to break a circuit than to break production.'"
+
+*— Universal Commentary Engine, Log Entry 42*
+
+## Contributing to the Resistance
+
+1. Fork it (like it's 2005)
+2. Create your feature branch (`git checkout -b feature/save-the-fleet`)
+3. Commit your changes (`git commit -am 'Add quantum circuit breaker'`)
+4. Push to the branch (`git push origin feature/save-the-fleet`)
+5. Create a new Pull Request (and wait for the Council of Elders to review)
 
 ## License
 
 MIT License. See [LICENSE](LICENSE) file for details.
 
+## Acknowledgments
+
+- The `state_machines` gem - The reliable engine under our hood
+- Every service that ever timed out - You taught me well
+- The RMNS Atlas Monkey - For philosophical guidance
+- The Resistance - For never giving up
+
 ## Author
 
 Built with ❤️ and ☕ by the Resistance against cascading failures.
 
----
-
-*Remember: Without circuit breakers, even AI can enter infinite loops of existential confusion. Don't let your services have an existential crisis.*
+**Remember: In space, nobody can hear your Redis timeout. But they can feel your circuit breaker failing over to localhost.**

data/lib/breaker_machines/async_support.rb

@@ -18,7 +18,7 @@ module BreakerMachines
 
     # Execute a call with async support (fiber-safe mode)
     def execute_call_async(&)
-      start_time = monotonic_time
+      start_time = BreakerMachines.monotonic_time
 
       begin
         # Execute with hedged requests if enabled
@@ -28,14 +28,14 @@ module BreakerMachines
                    execute_with_async_timeout(@config[:timeout], &)
                  end
 
-        record_success(monotonic_time - start_time)
+        record_success(BreakerMachines.monotonic_time - start_time)
         handle_success
         result
       rescue StandardError => e
         # Re-raise if it's not an async timeout or configured exception
         raise unless e.is_a?(async_timeout_error_class) || @config[:exceptions].any? { |klass| e.is_a?(klass) }
 
-        record_failure(monotonic_time - start_time, e)
+        record_failure(BreakerMachines.monotonic_time - start_time, e)
         handle_failure
         raise unless @config[:fallback]
 

data/lib/breaker_machines/cascading_circuit.rb

@@ -0,0 +1,175 @@
+# frozen_string_literal: true
+
+module BreakerMachines
+  # CascadingCircuit extends the base Circuit class with the ability to automatically
+  # trip dependent circuits when this circuit opens. This enables sophisticated
+  # failure cascade modeling, similar to how critical system failures in a starship
+  # would cascade to dependent subsystems.
+  #
+  # @example Starship network dependency
+  #   # Network circuit that cascades to dependent systems
+  #   network_circuit = BreakerMachines::CascadingCircuit.new('subspace_network', {
+  #     failure_threshold: 1,
+  #     cascades_to: ['weapons_targeting', 'navigation_sensors', 'communications'],
+  #     emergency_protocol: :red_alert
+  #   })
+  #
+  #   # When network fails, all dependent systems are automatically tripped
+  #   network_circuit.call { raise 'Subspace relay offline!' }
+  #   # => All dependent circuits are now open
+  #
+  class CascadingCircuit < Circuit
+    attr_reader :dependent_circuits, :emergency_protocol
+
+    def initialize(name, config = {})
+      @dependent_circuits = Array(config.delete(:cascades_to))
+      @emergency_protocol = config.delete(:emergency_protocol)
+
+      super
+    end
+
+    # Override the trip method to include cascading behavior
+    def trip!
+      result = super
+      perform_cascade if result && @dependent_circuits.any?
+      result
+    end
+
+    # Force cascade failure to all dependent circuits
+    def cascade_failure!
+      perform_cascade
+    end
+
+    # Get the current status of all dependent circuits
+    def dependent_status
+      return {} if @dependent_circuits.empty?
+
+      @dependent_circuits.each_with_object({}) do |circuit_name, status|
+        circuit = BreakerMachines.registry.find(circuit_name)
+        status[circuit_name] = circuit ? circuit.status_name : :not_found
+      end
+    end
+
+    # Check if any dependent circuits are open
+    def dependents_compromised?
+      @dependent_circuits.any? do |circuit_name|
+        circuit = BreakerMachines.registry.find(circuit_name)
+        circuit&.open?
+      end
+    end
+
+    # Summary that includes cascade information
+    def summary
+      base_summary = super
+      return base_summary if @dependent_circuits.empty?
+
+      if @cascade_triggered_at&.value
+        compromised_count = dependent_status.values.count(:open)
+        " CASCADE TRIGGERED: #{compromised_count}/#{@dependent_circuits.length} dependent systems compromised."
+      else
+        " Monitoring #{@dependent_circuits.length} dependent systems."
+      end
+
+      base_summary + cascade_info_text
+    end
+
+    # Provide cascade info for introspection
+    def cascade_info
+      BreakerMachines::CascadeInfo.new(
+        dependent_circuits: @dependent_circuits,
+        emergency_protocol: @emergency_protocol,
+        cascade_triggered_at: @cascade_triggered_at&.value,
+        dependent_status: dependent_status
+      )
+    end
+
+    private
+
+    def perform_cascade
+      return if @dependent_circuits.empty?
+
+      cascade_results = []
+      @cascade_triggered_at ||= Concurrent::AtomicReference.new
+      @cascade_triggered_at.value = BreakerMachines.monotonic_time
+
+      @dependent_circuits.each do |circuit_name|
+        # First try to find circuit in registry
+        circuit = BreakerMachines.registry.find(circuit_name)
+
+        # If not found and we have an owner, try to get it from the owner
+        if !circuit && @config[:owner]
+          owner = @config[:owner]
+          # Handle WeakRef if present
+          owner = owner.__getobj__ if owner.is_a?(WeakRef)
+
+          circuit = owner.circuit(circuit_name) if owner.respond_to?(:circuit)
+        end
+
+        next unless circuit
+        next unless circuit.closed? || circuit.half_open?
+
+        # Force the dependent circuit to open
+        circuit.force_open!
+        cascade_results << circuit_name
+
+        BreakerMachines.instrument('cascade_failure', {
+                                     source_circuit: @name,
+                                     target_circuit: circuit_name,
+                                     emergency_protocol: @emergency_protocol
+                                   })
+      end
+
+      # Trigger emergency protocol if configured
+      trigger_emergency_protocol(cascade_results) if @emergency_protocol && cascade_results.any?
+
+      # Invoke cascade callback if configured
+      if @config[:on_cascade]
+        begin
+          @config[:on_cascade].call(cascade_results) if @config[:on_cascade].respond_to?(:call)
+        rescue StandardError => e
+          # Log callback error but don't fail the cascade
+          BreakerMachines.logger&.error "Cascade callback error: #{e.message}"
+        end
+      end
+
+      cascade_results
+    end
+
+    def trigger_emergency_protocol(affected_circuits)
+      BreakerMachines.instrument('emergency_protocol_triggered', {
+                                   protocol: @emergency_protocol,
+                                   source_circuit: @name,
+                                   affected_circuits: affected_circuits
+                                 })
+
+      # Allow custom emergency protocol handling
+      owner = @config[:owner]
+      if owner&.respond_to?(@emergency_protocol, true)
+        begin
+          owner.send(@emergency_protocol, affected_circuits)
+        rescue StandardError => e
+          BreakerMachines.logger&.error "Emergency protocol error: #{e.message}"
+        end
+      elsif respond_to?(@emergency_protocol, true)
+        begin
+          send(@emergency_protocol, affected_circuits)
+        rescue StandardError => e
+          BreakerMachines.logger&.error "Emergency protocol error: #{e.message}"
+        end
+      end
+    end
+
+    # Override the on_circuit_open callback to include cascading
+    def on_circuit_open
+      super # Call the original implementation
+      perform_cascade if @dependent_circuits.any?
+    end
+
+    # Force open should also cascade
+    def force_open!
+      result = super
+      perform_cascade if result && @dependent_circuits.any?
+      result
+    end
+  end
+end

data/lib/breaker_machines/circuit/execution.rb

@@ -89,7 +89,7 @@ module BreakerMachines
           return execute_call_async(&block)
         end
 
-        start_time = monotonic_time
+        start_time = BreakerMachines.monotonic_time
 
         begin
           # IMPORTANT: We do NOT implement forceful timeouts as they are inherently unsafe
@@ -113,11 +113,11 @@ module BreakerMachines
                      block.call
                    end
 
-          record_success(monotonic_time - start_time)
+          record_success(BreakerMachines.monotonic_time - start_time)
           handle_success
           result
         rescue *@config[:exceptions] => e
-          record_failure(monotonic_time - start_time, e)
+          record_failure(BreakerMachines.monotonic_time - start_time, e)
           handle_failure
           raise unless @config[:fallback]
 
@@ -217,7 +217,7 @@ module BreakerMachines
       end
 
       def record_failure(duration, error = nil)
-        @last_failure_at.value = monotonic_time
+        @last_failure_at.value = BreakerMachines.monotonic_time
         @last_error.value = error if error
         @metrics&.record_failure(@name, duration)
         @storage&.record_failure(@name, duration)
@@ -226,10 +226,6 @@ module BreakerMachines
         @storage.record_event_with_details(@name, :failure, duration,
                                            error: error)
       end
-
-      def monotonic_time
-        Process.clock_gettime(Process::CLOCK_MONOTONIC)
-      end
     end
   end
 end

data/lib/breaker_machines/circuit/introspection.rb

@@ -12,7 +12,7 @@ module BreakerMachines
       # - half_open? (returns true when status == :half_open)
 
       def stats
-        {
+        BreakerMachines::Stats.new(
           state: status_name,
           failure_count: @storage.failure_count(@name, @config[:failure_window]),
           success_count: @storage.success_count(@name, @config[:failure_window]),
@@ -20,7 +20,7 @@ module BreakerMachines
           opened_at: @opened_at.value,
           half_open_attempts: @half_open_attempts.value,
           half_open_successes: @half_open_successes.value
-        }
+        )
       end
 
       def configuration
@@ -36,41 +36,56 @@ module BreakerMachines
       end
 
       def to_h
-        {
+        hash = {
           name: @name,
           state: status_name,
-          stats: stats,
+          stats: stats.to_h,
           config: configuration.except(:owner, :storage, :metrics),
           event_log: event_log || [],
-          last_error: last_error_info
+          last_error: last_error_info&.to_h
         }
+
+        # Add cascade-specific information if this is a cascading circuit
+        hash[:cascade_info] = cascade_info.to_h if is_a?(CascadingCircuit)
+
+        hash
       end
 
       def summary
-        case status_name
-        when :closed
-          "Circuit '#{@name}' is CLOSED. #{stats[:failure_count]} failures recorded."
-        when :open
-          reset_time = Time.at(@opened_at.value + @config[:reset_timeout])
-          opened_time = Time.at(@opened_at.value)
-          error_info = @last_error.value ? " The last error was #{@last_error.value.class}." : ''
-          "Circuit '#{@name}' is OPEN until #{reset_time}. " \
-            "It opened at #{opened_time} after #{@config[:failure_threshold]} failures.#{error_info}"
-        when :half_open
-          "Circuit '#{@name}' is HALF-OPEN. Testing with limited requests " \
-          "(#{@half_open_attempts.value}/#{@config[:half_open_calls]} attempts)."
+        base_summary = case status_name
+                       when :closed
+                         "Circuit '#{@name}' is CLOSED. #{stats.failure_count} failures recorded."
+                       when :open
+                         # Calculate time remaining until reset
+                         time_since_open = BreakerMachines.monotonic_time - @opened_at.value
+                         time_until_reset = @config[:reset_timeout] - time_since_open
+                         reset_time = time_until_reset.seconds.from_now
+                         opened_time = time_since_open.seconds.ago
+                         error_info = @last_error.value ? " The last error was #{@last_error.value.class}." : ''
+                         "Circuit '#{@name}' is OPEN until #{reset_time}. " \
+                           "It opened at #{opened_time} after #{@config[:failure_threshold]} failures.#{error_info}"
+                       when :half_open
+                         "Circuit '#{@name}' is HALF-OPEN. Testing with limited requests " \
+                         "(#{@half_open_attempts.value}/#{@config[:half_open_calls]} attempts)."
+                       end
+
+        # Add cascade information if this is a cascading circuit
+        if is_a?(CascadingCircuit) && @dependent_circuits.any?
+          base_summary += " [Cascades to: #{@dependent_circuits.join(', ')}]"
         end
+
+        base_summary
       end
 
       def last_error_info
         error = @last_error.value
         return nil unless error
 
-        {
-          class: error.class.name,
+        BreakerMachines::ErrorInfo.new(
+          error_class: error.class.name,
           message: error.message,
           occurred_at: @last_failure_at.value
-        }
+        )
       end
     end
   end

data/lib/breaker_machines/circuit/state_management.rb

@@ -19,6 +19,7 @@ module BreakerMachines
 
           event :reset do
             transition %i[open half_open] => :closed
+            transition closed: :closed
           end
 
           event :force_open do
@@ -46,7 +47,7 @@ module BreakerMachines
       private
 
       def on_circuit_open
-        @opened_at.value = monotonic_time
+        @opened_at.value = BreakerMachines.monotonic_time
         @storage&.set_status(@name, :open, @opened_at.value)
         if @storage.respond_to?(:record_event_with_details)
           @storage.record_event_with_details(@name, :state_change, 0,
@@ -85,8 +86,8 @@ module BreakerMachines
         stored_status = @storage.get_status(@name)
         return unless stored_status
 
-        self.status = stored_status[:status].to_s
-        @opened_at.value = stored_status[:opened_at] if stored_status[:opened_at]
+        self.status = stored_status.status.to_s
+        @opened_at.value = stored_status.opened_at if stored_status.opened_at
       end
 
       def reset_timeout_elapsed?
@@ -98,7 +99,7 @@ module BreakerMachines
         jitter_multiplier = 1.0 + (((rand * 2) - 1) * jitter_factor)
         timeout_with_jitter = @config[:reset_timeout] * jitter_multiplier
 
-        monotonic_time - @opened_at.value >= timeout_with_jitter
+        BreakerMachines.monotonic_time - @opened_at.value >= timeout_with_jitter
       end
     end
   end

data/lib/breaker_machines/circuit.rb

@@ -1,6 +1,5 @@
 # frozen_string_literal: true
 
-require 'state_machines'
 require 'concurrent-ruby'
 
 module BreakerMachines

data/lib/breaker_machines/console.rb

@@ -114,9 +114,9 @@ module BreakerMachines
 
         printf "%-20s %-12s %-10d %-10d %s\n",
                circuit.name,
-               colorize_state(stats[:state]),
-               stats[:failure_count],
-               stats[:success_count],
+               colorize_state(stats.state),
+               stats.failure_count,
+               stats.success_count,
                error_info
       end
 
@@ -167,22 +167,22 @@ module BreakerMachines
         stats = circuit.stats
         config = circuit.configuration
 
-        puts "Current State: #{colorize_state(stats[:state])}"
-        puts "Failure Count: #{stats[:failure_count]} / #{config[:failure_threshold]}"
-        puts "Success Count: #{stats[:success_count]}"
+        puts "Current State: #{colorize_state(stats.state)}"
+        puts "Failure Count: #{stats.failure_count} / #{config[:failure_threshold]}"
+        puts "Success Count: #{stats.success_count}"
 
-        if stats[:opened_at]
-          puts "Opened At: #{Time.at(stats[:opened_at])}"
-          reset_time = Time.at(stats[:opened_at] + config[:reset_timeout])
+        if stats.opened_at
+          puts "Opened At: #{Time.at(stats.opened_at)}"
+          reset_time = Time.at(stats.opened_at + config[:reset_timeout])
           puts "Reset At: #{reset_time} (in #{(reset_time - Time.now).to_i}s)"
         end
 
         if circuit.last_error
           error_info = circuit.last_error_info
           puts "\nLast Error:"
-          puts "  Class: #{error_info[:class]}"
-          puts "  Message: #{error_info[:message]}"
-          puts "  Time: #{Time.at(error_info[:occurred_at])}"
+          puts "  Class: #{error_info.error_class}"
+          puts "  Message: #{error_info.message}"
+          puts "  Time: #{Time.at(error_info.occurred_at)}"
         end
 
         puts "\nConfiguration:"

data/lib/breaker_machines/dsl/cascading_circuit_builder.rb

@@ -0,0 +1,20 @@
+# frozen_string_literal: true
+
+module BreakerMachines
+  module DSL
+    # Builder for cascading circuit breaker configuration
+    class CascadingCircuitBuilder < CircuitBuilder
+      def cascades_to(*circuit_names)
+        @config[:cascades_to] = circuit_names.flatten
+      end
+
+      def emergency_protocol(protocol_name)
+        @config[:emergency_protocol] = protocol_name
+      end
+
+      def on_cascade(&block)
+        @config[:on_cascade] = block
+      end
+    end
+  end
+end