chaotic_job 0.4.0 → 0.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 98b01482666de5069e40dbe4eeb828cb527e4dd31004b3900a134ceaec5cb42a
-  data.tar.gz: f3df1be0fa3192aa842b74fa56754e56ed071f48d3c2fd027ca88d05da3f5e90
+  metadata.gz: 10db3cce1a0eedffe044477a348122d781e19cfbb6f561c9e389926d7de21153
+  data.tar.gz: a48f0d63526613593a25cc92d1c3eca794adb99466ef697a3c932e581a08434a
 SHA512:
-  metadata.gz: aff436f38402fddae8e62a7dbb441dc8949169abd1aa992fdf5e59f33e007f8dc3aba29787a3cea4bf3d421698ed1ac76c171388c26a3d2ab884e7880296ba84
-  data.tar.gz: 72986fa7274aade0e1fe7f9a5b71e4911467f18bf4929b90960ba62b03721611e46da8958c0536a92f92e98f4b4a2c7315e674db73bbcc6866cc177070a6f92f
+  metadata.gz: a1ea02d93adf652616f25478c8d7f628f30d56137485b10988bd2f08b9a62153327cd60c787555d7ba4286a7c26f94616ee45c92a58778e0689011461a39bb3b
+  data.tar.gz: 6d94662caef5559bca3de08c69674ad10dd4fa349fbd8434a42dc65c8baa8ba6da13f6cc8d13b9af815f4610148a874ba3f9fa0b9e5c8aa97e7b131afc5c3689

data/CHANGELOG.md

@@ -1,5 +1,13 @@
 ## [Unreleased]
 
+## [0.6.0] - 2025-06-08
+
+- `run_scenario` requires a Glitch instance [#5](https://github.com/fractaledmind/chaotic_job/pull/5)
+
+## [0.5.0] - 2025-06-04
+
+- Add a Tracer class [#3](https://github.com/fractaledmind/chaotic_job/pull/3)
+
 ## [0.4.0] - 2025-05-27
 
 - Allow a Glitch to be defined for a method call or method return [#4](https://github.com/fractaledmind/chaotic_job/pull/4)

data/README.md

@@ -107,7 +107,7 @@ test "scenario of a simple job" do
     def step_3; ChaoticJob::Journal.log; end
   end
 
-  run_scenario(Job.new, glitch: [:before_call, "Job#step_3"])
+  run_scenario(Job.new, glitch: ChaoticJob::Glitch.before_call("Job#step_3"))
 
   assert_equal 5, ChaoticJob::Journal.total
 end
@@ -124,14 +124,16 @@ end
 > | `Journal.entries` | get all of the logged values under the default scope |
 > | `Journal.entries(scope: :special)` | get all of the logged values under a particular scope |
 
-In this example, the job being tested is defined within the test case. You can, of course, also test jobs defined in your application. The key detail is the `glitch` keyword argument. A "glitch" is simply a tuple that describes precisely where you would like the failure to occur. The first element of the tuple is the _kind_ of glitch, which can be either `:before_line`, `:before_call`, or `:before_return`. These refer to the three kinds of `TracePoint` events that the gem hooks into. The second element is the _key_ for the code that will be affected by the glitch. This _key_ is a specially formatted string that defines the specific bit of code that the glitch should be inserted before. The different kinds of glitches are identified by different kinds of keys:
+In this example, the job being tested is defined within the test case. You can, of course, also test jobs defined in your application. The key detail is the `glitch` keyword argument.
+
+A "glitch" is describes precisely where you would like the failure to occur. The description is composed first of the _kind_ of glitch, which can be either `before_line`, `before_call`, or `before_return`. These refer to the three kinds of `TracePoint` events that the gem hooks into. The second element is the _key_ for the code that will be affected by the glitch. This _key_ is a specially formatted string that defines the specific bit of code that the glitch should be inserted before. The different kinds of glitches are identified by different kinds of keys:
 |kind|key format|key example|
 |---|---|---|
-|`:before_line`|`"#{file_path}:#{line_number}"`|`"/Users/you/path/to/file.rb:123"`|
-|`:before_call`|`"#{YourClass.name}(.|#)#{method_name}"`|`"YourClass.some_class_method"`|
-|`:before_return`|`"#{YourClass.name}(.|#)#{method_name}"`|`"YourClass#some_instance_method"`|
+|`before_line`|`"#{file_path}:#{line_number}"`|`"/Users/you/path/to/file.rb:123"`|
+|`before_call`|`"#{YourClass.name}(.|#)#{method_name}"`|`"YourClass.some_class_method"`|
+|`before_return`|`"#{YourClass.name}(.|#)#{method_name}"`|`"YourClass#some_instance_method"`|
 
-As you can see, the `:before_call` and `:before_return` keys are formatted the same, and can identify any instance (`#`) or class (`.`) method.
+As you can see, the `before_call` and `before_return` keys are formatted the same, and can identify any instance (`#`) or class (`.`) method.
 
 What the example scenario above does is inject a glitch before the `step_3` method is called, here:
 
@@ -144,10 +146,10 @@ def perform
 end
 ```
 
-If we wanted to inject a glitch right before the `step_3` method finishes, we could define the glitch as a `:before_return`, like this:
+If we wanted to inject a glitch right before the `step_3` method finishes, we could define the glitch as a `before_return`, like this:
 
 ```ruby
-run_scenario(Job.new, glitch: [:before_return, "Job#step_3"])
+run_scenario(Job.new, glitch: ChaoticJob::Glitch.before_return("Job#step_3"))
 ```
 
 and it would inject the transient error right here:
@@ -159,19 +161,21 @@ def step_3
 end
 ```
 
-Finally, if you need to inject a glitch right before a particular line of code is executed that is neither a method call nor a method return, you can use the `:before_line` key, like this:
+Finally, if you need to inject a glitch right before a particular line of code is executed that is neither a method call nor a method return, you can use the `before_line` key, like this:
 
 ```ruby
-run_scenario(Job.new, glitch: [:before_line, "#{__FILE__}:6"])
+run_scenario(Job.new, glitch: ChaoticJob::Glitch.before_line("#{__FILE__}:6"))
 ```
 
-If you want to simulate multiple glitches affecting a job run, you can use the plural `glitches` keyword argument instead and pass an array of tuples:
+If you want to simulate multiple glitches affecting a job run, you simply define additional failure points using the fluid interface of the `ChaoticJob::Glitch` class:
 
 ```ruby
-run_scenario(Job.new, glitches: [
-  [:before_call, "Job#step_1"],
-  [:before_return, "Job#step_1"]
-])
+run_scenario(
+  Job.new,
+  glitch: ChaoticJob::Glitch
+    .before_call("Job#step_1")
+    .before_return("Job#step_3")
+)
 ```
 
 Scenario testing is useful to test the behavior of a job under a specific set of conditions. But, if you want to test the behavior of a job under a variety of conditions, you can use the `run_simulation` method. Instead of running a single scenario, a simulation will run the full set of possible error scenarios for your job.
@@ -217,7 +221,24 @@ More specifically, it will create a scenario injecting a glitch before every lin
 
 It generates all possible glitch scenarios by performing your job once with a [`TracePoint`](https://docs.ruby-lang.org/en/master/TracePoint.html) that captures each line executed in your job. It then computes all possible glitch locations to produce a set of scenarios that will be run. The block that you pass to `run_simulation` will be called for each scenario, allowing you to make assertions about the behavior of your job under all scenarios.
 
-In your application tests, you will want to make assertions about the side-effects that your job performs, asserting that they are correctly idempotent (only occur once) and result in the correct state.
+If you want to have the simulation run against a larger collection of scenarios, you can capture a custom callstack using the `ChaoticJob::Tracer` class and pass it to the `run_simulation` method as the `callstack` parameter. A `Tracer` is initialized with a block that determines which `TracePoint` events to collect. You then call `capture` with a block that defines the code to be traced. The default `Simulation` tracer collects all events for the passed job and then traces the job execution, essentially like this:
+
+```ruby
+job_file_path = YourJob.instance_method(:perform).source_location&.first
+tracer = Tracer.new { |tp| tp.path == job_file_path || tp.defined_class == YourJob }
+tracer.capture { YourJob.perform_now }
+```
+
+To capture, for example, a custom callstack that includes all events within your application, you can use the `ChaoticJob::Tracer` class as follows:
+
+```ruby
+tracer = ChaoticJob::Tracer.new { |tp| tp.path.start_with?(Rails.root.to_s) }
+tracer.capture { YourJob.perform_now }
+```
+
+If you passed this callstack to your simulation, it would test what happens to your job whenever a transient glitch is injected anywhere in your application code called as a part of executing the job under test.
+
+Remember, in your application tests, you will want to make assertions about the side-effects that your job performs, asserting that they are correctly idempotent (only occur once) and result in the correct state.
 
 ## Development
 

data/lib/chaotic_job/glitch.rb

@@ -1,12 +1,24 @@
 # frozen_string_literal: true
 
-# Glitch.new.before_line("job_crucible.rb:10") { do_anything }
-# Glitch.new.before_call("Model#method", String, name: "Joel") { do_anything }
-# Glitch.new.before_return("Model#method", String, name: "Joel") { do_anything }
-# Glitch.new.inject! { execute code to glitch }
+# Glitch.before_line("job_crucible.rb:10") { do_anything }
+# Glitch.before_call("Model#method", String, name: "Joel") { do_anything }
+# Glitch.before_return("Model#method", String, name: "Joel") { do_anything }
+# Glitch.inject! { execute code to glitch }
 
 module ChaoticJob
   class Glitch
+    def self.before_line(key, &block)
+      new.before_line(key, &block)
+    end
+
+    def self.before_call(key, ...)
+      new.before_call(key, ...)
+    end
+
+    def self.before_return(key, return_type = nil, &block)
+      new.before_return(key, return_type, &block)
+    end
+
     def initialize
       @breakpoints = {}
     end
@@ -26,6 +38,15 @@ module ChaoticJob
       self
     end
 
+    def set_action(force: false, &block)
+      @breakpoints.each do |_key, handlers|
+        handlers.each do |_event, handler|
+          handler[:block] = block if handler[:block].nil? || force
+        end
+      end
+      self
+    end
+
     def inject!(&block)
       breakpoints = @breakpoints
 
@@ -45,17 +66,11 @@ module ChaoticJob
     end
 
     def all_executed?
-      @breakpoints.all? do |_location, handlers|
-        handlers.all? { |_position, handler| handler[:executed] }
+      @breakpoints.all? do |_key, handlers|
+        handlers.all? { |_event, handler| handler[:executed] }
       end
     end
 
-    # def inspect
-    #   @breakpoints.flat_map do |location, configs|
-    #     configs.keys.map { |position| "#{position}-#{location}" }
-    #   end.join("|>")
-    # end
-
     private
 
     def set_breakpoint(key, event, *args, retval: nil, **kwargs, &block)

data/lib/chaotic_job/scenario.rb

@@ -7,20 +7,20 @@ module ChaoticJob
   class Scenario
     attr_reader :events
 
-    def initialize(job, glitches:, raise: RetryableError, capture: /active_job/)
+    def initialize(job, glitch:, raise: RetryableError, capture: /active_job/)
       @job = job
-      @glitches = glitches
+      @glitch = (Glitch === glitch) ? glitch : (raise Error.new("glitch: must be a Glitch instance, but got #{glitch.inspect}"))
       @raise = binding.local_variable_get(:raise)
       @capture = capture
-      @glitch = nil
       @events = []
     end
 
     def run(&block)
       @job.class.retry_on RetryableError, attempts: 10, wait: 1, jitter: 0
+      @glitch.set_action { raise @raise }
 
       ActiveSupport::Notifications.subscribed(->(event) { @events << event.dup }, @capture) do
-        glitch.inject! do
+        @glitch.inject! do
           @job.enqueue
           if block
             block.call
@@ -30,25 +30,11 @@ module ChaoticJob
         end
       end
 
-      # TODO: assert that all glitches ran
-    end
-
-    def to_s
-      @glitches.map { |position, location| "#{position}-#{location}" }.join("|>")
+      # TODO: assert that all glitch ran
     end
 
     def all_glitched?
       @glitch.all_executed?
     end
-
-    private
-
-    def glitch
-      @glitch ||= Glitch.new.tap do |glitch|
-        @glitches.each do |kind, location, _description|
-          glitch.public_send(kind, location) { raise @raise }
-        end
-      end
-    end
   end
 end

data/lib/chaotic_job/simulation.rb

@@ -6,13 +6,16 @@
 # Simulation.new(job).scenarios
 module ChaoticJob
   class Simulation
-    def initialize(job, depth: 1, variations: 100, test: nil, seed: nil)
+    def initialize(job, callstack: nil, depth: 1, variations: 100, test: nil, seed: nil)
       @template = job
+      @callstack = callstack || capture_callstack
       @depth = depth
       @variations = variations
       @test = test
       @seed = seed || Random.new_seed
       @random = Random.new(@seed)
+
+      raise Error.new("callstack must be a generated via the ChaoticJob::Tracer") unless @callstack.is_a?(Stack)
     end
 
     def run(&callback)
@@ -27,9 +30,8 @@ module ChaoticJob
     end
 
     def permutations
-      callstack = capture_callstack.map { |path, line| "#{path}:#{line}" }
-      error_locations = callstack.map do |path, lineno|
-        [:before_line, "#{path}:#{lineno}"]
+      error_locations = @callstack.map do |event, key|
+        ["before_#{event}", key]
       end
       error_locations.permutation(@depth)
     end
@@ -43,7 +45,8 @@ module ChaoticJob
     def scenarios
       variants.map do |glitches|
         job = clone_job_template
-        scenario = Scenario.new(job, glitches: glitches)
+        glitch = Glitch.new.tap { |g| glitches.each { |event, key| g.public_send(event, key) } }
+        scenario = Scenario.new(job, glitch: glitch)
         job.job_id = scenario.to_s
         scenario
       end
@@ -52,23 +55,13 @@ module ChaoticJob
     private
 
     def capture_callstack
-      return @callstack if defined?(@callstack)
-
-      @callstack = Set.new
       job_class = @template.class
       job_file_path = job_class.instance_method(:perform).source_location&.first
+      tracer = Tracer.new { |tp| tp.path == job_file_path || tp.defined_class == job_class }
+      callstack = tracer.capture { @template.dup.perform_now }
 
-      trace = TracePoint.new(:line) do |tp|
-        next if tp.defined_class == self.class
-        next unless tp.path == job_file_path ||
-          tp.defined_class == job_class
-
-        @callstack << [tp.path, tp.lineno]
-      end
-
-      trace.enable { @template.dup.perform_now }
       @template.class.queue_adapter.enqueued_jobs = []
-      @callstack
+      callstack
     end
 
     def run_scenario(scenario, &callback)

data/lib/chaotic_job/tracer.rb

@@ -0,0 +1,50 @@
+# frozen_string_literal: true
+
+# Tracer.new { |tp| tp.path.start_with? "foo" }
+# Tracer.new.capture { code_execution_to_trace_callstack }
+
+module ChaoticJob
+  class Tracer
+    def initialize(&constraint)
+      @constraint = constraint
+      @callstack = Stack.new
+    end
+
+    def capture(&block)
+      trace = TracePoint.new(:line, :call, :return) do |tp|
+        # :nocov: SimpleCov cannot track code executed _within_ a TracePoint
+        next if tp.defined_class == self.class
+        next unless @constraint.call(tp)
+
+        case tp.event
+        when :line
+          key = line_key(tp)
+        when :call, :return
+          key = call_key(tp)
+        end
+
+        @callstack << [tp.event, key]
+        # :nocov:
+      end
+
+      trace.enable(&block)
+      @callstack
+    end
+
+    private
+
+    # :nocov: SimpleCov cannot track code executed _within_ a TracePoint
+    def line_key(event)
+      "#{event.path}:#{event.lineno}"
+    end
+
+    def call_key(event)
+      if Module === event.self
+        "#{event.self}.#{event.method_id}"
+      else
+        "#{event.defined_class}##{event.method_id}"
+      end
+    end
+    # :nocov:
+  end
+end

data/lib/chaotic_job/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module ChaoticJob
-  VERSION = "0.4.0"
+  VERSION = "0.6.0"
 end

data/lib/chaotic_job.rb

@@ -3,13 +3,16 @@
 require_relative "chaotic_job/version"
 require_relative "chaotic_job/journal"
 require_relative "chaotic_job/performer"
+require_relative "chaotic_job/tracer"
 require_relative "chaotic_job/glitch"
 require_relative "chaotic_job/scenario"
 require_relative "chaotic_job/simulation"
+require "set"
 
 module ChaoticJob
-  class RetryableError < StandardError
-  end
+  Error = Class.new(StandardError)
+  RetryableError = Class.new(Error)
+  Stack = Set
 
   def self.log_to_journal!(item = nil, scope: nil)
     if item && scope
@@ -63,19 +66,23 @@ module ChaoticJob
       Performer.perform_all_after(time)
     end
 
-    def run_simulation(job, depth: nil, variations: nil, &block)
+    def run_simulation(job, depth: nil, variations: nil, callstack: nil, &block)
       seed = defined?(RSpec) ? RSpec.configuration.seed : Minitest.seed
       kwargs = {test: self, seed: seed}
       kwargs[:depth] = depth if depth
       kwargs[:variations] = variations if variations
+      kwargs[:callstack] = callstack if callstack
       self.simulation_scenario = nil
       Simulation.new(job, **kwargs).run(&block)
     end
 
-    def run_scenario(job, glitch: nil, glitches: nil, raise: nil, capture: nil, &block)
-      kwargs = {glitches: glitches || [glitch]}
+    def run_scenario(job, glitch:, raise: nil, capture: nil, &block)
+      kwargs = {}
+
+      kwargs[:glitch] = glitch
       kwargs[:raise] = binding.local_variable_get(:raise) if binding.local_variable_get(:raise)
       kwargs[:capture] = capture if capture
+
       if block
         Scenario.new(job, **kwargs).run(&block)
       else
@@ -83,6 +90,8 @@ module ChaoticJob
       end
     end
 
+    private
+
     def assert(test, msg = nil)
       return super unless @simulation_scenario
 

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: chaotic_job
 version: !ruby/object:Gem::Version
-  version: 0.4.0
+  version: 0.6.0
 platform: ruby
 authors:
 - Stephen Margheim
 bindir: exe
 cert_chain: []
-date: 2025-05-27 00:00:00.000000000 Z
+date: 2025-06-08 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activejob
@@ -41,6 +41,7 @@ files:
 - lib/chaotic_job/performer.rb
 - lib/chaotic_job/scenario.rb
 - lib/chaotic_job/simulation.rb
+- lib/chaotic_job/tracer.rb
 - lib/chaotic_job/version.rb
 - sig/chaotic_job.rbs
 homepage: https://github.com/fractaledmind/chaotic_job