chaotic_job 0.5.0 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0c4da24d7fc4aa39030976d4f7249a14c87995ef5d5b05c523ad5e1e83724f37
-  data.tar.gz: 0a7be90ecb5381a5c9e67c6673bb41bc3ad2997c904ca42bfedbb4c701ef6bf0
+  metadata.gz: 94de9ce766a9042a925882e70f29a8022fbd3b22fbd175ed3254260e04962cba
+  data.tar.gz: e05d2546038d72cd9181fef3eff223d7adec84225b0fa1267d89d0b3880351ef
 SHA512:
-  metadata.gz: 11a054f050a3e95aa40c392e281b69bf2c43dbba3b91bce3d17473d589f4e2057d26a18af5071d7a5f4cb2b055775bbdd293af6c712792fbe1d042c4c99cca85
-  data.tar.gz: d6750f62adecc71334dd42cfb15e35e33fb36680be62a6c9d0d5f4e2ed29eb598f9f92fac6a09ec1a4636816789f13996bc0241faa5a1ef9eb4acac0577fc06f
+  metadata.gz: 29fedcda5ca8ec4c98fa68d4f1e0661194c558a72a7a70b6b19a445de0e4bb0741e873c1365911315a8be73461248ccbca3005dd604e0a1f8229cf75aa4f7f69
+  data.tar.gz: 93423ffd252f3faf4b3c0e0c0e7097fc9e14468d884935fed12421ca653cc99a47819ab80833d4068e0c8ebf23752e45385923384c56e4072e08554d7fe872f2

data/CHANGELOG.md

@@ -1,5 +1,16 @@
 ## [Unreleased]
 
+## [0.7.0] - 2025-06-09
+
+- Glitch only works with singular event + key definition [#6](https://github.com/fractaledmind/chaotic_job/pull/6)
+- Scenarios assert the glitch was executed [#7](https://github.com/fractaledmind/chaotic_job/pull/7)
+- Add helper methods to create a Glitch of the various kinds [#8](https://github.com/fractaledmind/chaotic_job/pull/8)
+- Improve test coverage [#9](https://github.com/fractaledmind/chaotic_job/pull/9)
+
+## [0.6.0] - 2025-06-08
+
+- `run_scenario` requires a Glitch instance [#5](https://github.com/fractaledmind/chaotic_job/pull/5)
+
 ## [0.5.0] - 2025-06-04
 
 - Add a Tracer class [#3](https://github.com/fractaledmind/chaotic_job/pull/3)

data/README.md

@@ -3,7 +3,7 @@
 [![Gem Version](https://badge.fury.io/rb/chaotic_job.svg)](https://rubygems.org/gems/chaotic_job)
 [![Gem Downloads](https://img.shields.io/gem/dt/chaotic_job)](https://rubygems.org/gems/chaotic_job)
 ![Tests](https://github.com/fractaledmind/chaotic_job/actions/workflows/main.yml/badge.svg)
-![Coverage](https://img.shields.io/badge/code%20coverage-92%25-success)
+![Coverage](https://img.shields.io/badge/code%20coverage-98%25-success)
 [![Sponsors](https://img.shields.io/github/sponsors/fractaledmind?color=eb4aaa&logo=GitHub%20Sponsors)](https://github.com/sponsors/fractaledmind)
 [![Twitter Follow](https://img.shields.io/twitter/url?label=%40fractaledmind&style=social&url=https%3A%2F%2Ftwitter.com%2Ffractaledmind)](https://twitter.com/fractaledmind)
 
@@ -107,7 +107,7 @@ test "scenario of a simple job" do
     def step_3; ChaoticJob::Journal.log; end
   end
 
-  run_scenario(Job.new, glitch: [:before_call, "Job#step_3"])
+  run_scenario(Job.new, glitch: ChaoticJob::Glitch.before_call("Job#step_3"))
 
   assert_equal 5, ChaoticJob::Journal.total
 end
@@ -124,14 +124,16 @@ end
 > | `Journal.entries` | get all of the logged values under the default scope |
 > | `Journal.entries(scope: :special)` | get all of the logged values under a particular scope |
 
-In this example, the job being tested is defined within the test case. You can, of course, also test jobs defined in your application. The key detail is the `glitch` keyword argument. A "glitch" is simply a tuple that describes precisely where you would like the failure to occur. The first element of the tuple is the _kind_ of glitch, which can be either `:before_line`, `:before_call`, or `:before_return`. These refer to the three kinds of `TracePoint` events that the gem hooks into. The second element is the _key_ for the code that will be affected by the glitch. This _key_ is a specially formatted string that defines the specific bit of code that the glitch should be inserted before. The different kinds of glitches are identified by different kinds of keys:
+In this example, the job being tested is defined within the test case. You can, of course, also test jobs defined in your application. The key detail is the `glitch` keyword argument.
+
+A "glitch" is describes precisely where you would like the failure to occur. The description is composed first of the _kind_ of glitch, which can be either `before_line`, `before_call`, or `before_return`. These refer to the three kinds of `TracePoint` events that the gem hooks into. The second element is the _key_ for the code that will be affected by the glitch. This _key_ is a specially formatted string that defines the specific bit of code that the glitch should be inserted before. The different kinds of glitches are identified by different kinds of keys:
 |kind|key format|key example|
 |---|---|---|
-|`:before_line`|`"#{file_path}:#{line_number}"`|`"/Users/you/path/to/file.rb:123"`|
-|`:before_call`|`"#{YourClass.name}(.|#)#{method_name}"`|`"YourClass.some_class_method"`|
-|`:before_return`|`"#{YourClass.name}(.|#)#{method_name}"`|`"YourClass#some_instance_method"`|
+|`before_line`|`"#{file_path}:#{line_number}"`|`"/Users/you/path/to/file.rb:123"`|
+|`before_call`|`"#{YourClass.name}(.|#)#{method_name}"`|`"YourClass.some_class_method"`|
+|`before_return`|`"#{YourClass.name}(.|#)#{method_name}"`|`"YourClass#some_instance_method"`|
 
-As you can see, the `:before_call` and `:before_return` keys are formatted the same, and can identify any instance (`#`) or class (`.`) method.
+As you can see, the `before_call` and `before_return` keys are formatted the same, and can identify any instance (`#`) or class (`.`) method.
 
 What the example scenario above does is inject a glitch before the `step_3` method is called, here:
 
@@ -144,10 +146,10 @@ def perform
 end
 ```
 
-If we wanted to inject a glitch right before the `step_3` method finishes, we could define the glitch as a `:before_return`, like this:
+If we wanted to inject a glitch right before the `step_3` method finishes, we could define the glitch as a `before_return`, like this:
 
 ```ruby
-run_scenario(Job.new, glitch: [:before_return, "Job#step_3"])
+run_scenario(Job.new, glitch: ChaoticJob::Glitch.before_return("Job#step_3"))
 ```
 
 and it would inject the transient error right here:
@@ -159,19 +161,10 @@ def step_3
 end
 ```
 
-Finally, if you need to inject a glitch right before a particular line of code is executed that is neither a method call nor a method return, you can use the `:before_line` key, like this:
+Finally, if you need to inject a glitch right before a particular line of code is executed that is neither a method call nor a method return, you can use the `before_line` key, like this:
 
 ```ruby
-run_scenario(Job.new, glitch: [:before_line, "#{__FILE__}:6"])
-```
-
-If you want to simulate multiple glitches affecting a job run, you can use the plural `glitches` keyword argument instead and pass an array of tuples:
-
-```ruby
-run_scenario(Job.new, glitches: [
-  [:before_call, "Job#step_1"],
-  [:before_return, "Job#step_1"]
-])
+run_scenario(Job.new, glitch: ChaoticJob::Glitch.before_line("#{__FILE__}:6"))
 ```
 
 Scenario testing is useful to test the behavior of a job under a specific set of conditions. But, if you want to test the behavior of a job under a variety of conditions, you can use the `run_simulation` method. Instead of running a single scenario, a simulation will run the full set of possible error scenarios for your job.
@@ -199,23 +192,21 @@ end
 More specifically, it will create a scenario injecting a glitch before every line of code executed in your job. So, in this example, the simulation will run 12 scenarios:
 
 ```ruby
-[
-  [[:before_line, "test_chaotic_job.rb:69"]],
-  [[:before_line, "test_chaotic_job.rb:75"]],
-  [[:before_line, "test_chaotic_job.rb:74"]],
-  [[:before_line, "test_chaotic_job.rb:74"]],
-  [[:before_line, "test_chaotic_job.rb:68"]],
-  [[:before_line, "test_chaotic_job.rb:70"]],
-  [[:before_line, "test_chaotic_job.rb:68"]],
-  [[:before_line, "test_chaotic_job.rb:73"]],
-  [[:before_line, "test_chaotic_job.rb:75"]],
-  [[:before_line, "test_chaotic_job.rb:69"]],
-  [[:before_line, "test_chaotic_job.rb:70"]],
-  [[:before_line, "test_chaotic_job.rb:73"]]
-]
-```
-
-It generates all possible glitch scenarios by performing your job once with a [`TracePoint`](https://docs.ruby-lang.org/en/master/TracePoint.html) that captures each line executed in your job. It then computes all possible glitch locations to produce a set of scenarios that will be run. The block that you pass to `run_simulation` will be called for each scenario, allowing you to make assertions about the behavior of your job under all scenarios.
+#<Set:
+ {[:call, "Job#perform"],
+  [:line, "file.rb:3"],
+  [:call, "Job#step_1"],
+  [:return, "Job#step_1"],
+  [:line, "file.rb:4"],
+  [:call, "Job#step_2"],
+  [:return, "Job#step_2"],
+  [:line, "file.rb:5"],
+  [:call, "Job#step_3"],
+  [:return, "Job#step_3"],
+  [:return, "Job#perform"]}>
+```
+
+It generates all possible glitch scenarios by performing your job once with a [`TracePoint`](https://docs.ruby-lang.org/en/master/TracePoint.html) that captures every event executed as a part of your job running. The block that you pass to `run_simulation` will be called for each scenario, allowing you to make assertions about the behavior of your job under all scenarios.
 
 If you want to have the simulation run against a larger collection of scenarios, you can capture a custom callstack using the `ChaoticJob::Tracer` class and pass it to the `run_simulation` method as the `callstack` parameter. A `Tracer` is initialized with a block that determines which `TracePoint` events to collect. You then call `capture` with a block that defines the code to be traced. The default `Simulation` tracer collects all events for the passed job and then traces the job execution, essentially like this:
 

data/lib/chaotic_job/glitch.rb

@@ -1,73 +1,64 @@
 # frozen_string_literal: true
 
-# Glitch.new.before_line("job_crucible.rb:10") { do_anything }
-# Glitch.new.before_call("Model#method", String, name: "Joel") { do_anything }
-# Glitch.new.before_return("Model#method", String, name: "Joel") { do_anything }
-# Glitch.new.inject! { execute code to glitch }
+# Glitch.before_line("job_crucible.rb:10") { do_anything }
+# Glitch.before_call("Model#method", String, name: "Joel") { do_anything }
+# Glitch.before_return("Model#method", String, name: "Joel") { do_anything }
+# Glitch.inject! { execute code to glitch }
 
 module ChaoticJob
   class Glitch
-    def initialize
-      @breakpoints = {}
+    def self.before_line(key, &block)
+      new(key, :line, &block)
     end
 
-    def before_line(key, &block)
-      set_breakpoint(key, :line, &block)
-      self
+    def self.before_call(key, ...)
+      new(key, :call, ...)
     end
 
-    def before_call(key, ...)
-      set_breakpoint(key, :call, ...)
-      self
+    def self.before_return(key, return_type = nil, &block)
+      new(key, :return, retval: return_type, &block)
     end
 
-    def before_return(key, return_type = nil, &block)
-      set_breakpoint(key, :return, retval: return_type, &block)
-      self
+    def initialize(key, event, *args, retval: nil, **kwargs, &block)
+      @event = event
+      @key = key
+      @args = args
+      @retval = retval
+      @kwargs = kwargs
+      @block = block
+      @executed = false
     end
 
-    def inject!(&block)
-      breakpoints = @breakpoints
+    def set_action(force: false, &block)
+      @block = block if @block.nil? || force
+    end
 
-      trace = TracePoint.new(:line, :call, :return) do |tp|
+    def inject!(&block)
+      trace = TracePoint.new(@event) do |tp|
         # :nocov: SimpleCov cannot track code executed _within_ a TracePoint
         key = derive_key(tp)
-        matchers = derive_matchers(tp)
+        next unless @key == key
 
-        next unless (defn = breakpoints.dig(key, tp.event))
-        next unless matches?(defn, matchers)
+        matchers = derive_matchers(tp)
+        next unless matches?(matchers)
 
-        execute_block(defn)
+        execute_block
         # :nocov:
       end
 
       trace.enable(&block)
     end
 
-    def all_executed?
-      @breakpoints.all? do |_location, handlers|
-        handlers.all? { |_position, handler| handler[:executed] }
-      end
+    def executed?
+      @executed
     end
 
-    # def inspect
-    #   @breakpoints.flat_map do |location, configs|
-    #     configs.keys.map { |position| "#{position}-#{location}" }
-    #   end.join("|>")
-    # end
-
     private
 
-    def set_breakpoint(key, event, *args, retval: nil, **kwargs, &block)
-      @breakpoints[key] ||= {}
-      @breakpoints[key][event] = {args: args, kwargs: kwargs, retval: retval, block: block, executed: false}
-    end
-
     # :nocov: SimpleCov cannot track code executed _within_ a TracePoint
-    def matches?(defn, matchers)
-      return true if defn.nil?
+    def matches?(matchers)
       return true if matchers.nil?
-      return true if defn[:args].empty? && defn[:kwargs].empty? && defn[:retval].nil?
+      return true if @args.empty? && @kwargs.empty? && @retval.nil?
 
       args = []
       kwargs = {}
@@ -92,25 +83,24 @@ module ChaoticJob
         end
       end
 
-      defn[:args].each_with_index do |type, index|
+      @args.each_with_index do |type, index|
         return false unless type === args[index]
       end
 
-      defn[:kwargs].each do |key, type|
+      @kwargs.each do |key, type|
         return false unless type === kwargs[key]
       end
 
-      return false unless defn[:retval] === retval
+      return false unless @retval === retval
 
       true
     end
 
-    def execute_block(handler)
-      return unless handler
-      return if handler[:executed]
+    def execute_block
+      return if @executed
 
-      handler[:executed] = true
-      handler[:block].call
+      @executed = true
+      @block.call
     end
 
     def derive_key(trace)

data/lib/chaotic_job/performer.rb

@@ -76,8 +76,11 @@ module ChaoticJob
         cutoff.from_now
       in Time
         cutoff
+      else
+        raise Error.new("cutoff must be Time or ActiveSupport::Duration, but got #{cutoff.inspect}")
       end
       delta = (Time.now - time).abs.floor
+
       changeset = case delta
       when 0..59                    # seconds
         {usec: 0}
@@ -85,7 +88,7 @@ module ChaoticJob
         {sec: 0, usec: 0}
       when 3600..86_399             # hours
         {min: 0, sec: 0, usec: 0}
-      when 86_400..Float::INFINITY  # days+
+      else  # days+
         {hour: 0, min: 0, sec: 0, usec: 0}
       end
       time.change(**changeset)

data/lib/chaotic_job/scenario.rb

@@ -5,22 +5,22 @@
 
 module ChaoticJob
   class Scenario
-    attr_reader :events
+    attr_reader :events, :glitch, :job
 
-    def initialize(job, glitches:, raise: RetryableError, capture: /active_job/)
+    def initialize(job, glitch:, raise: RetryableError, capture: /active_job/)
       @job = job
-      @glitches = glitches
+      @glitch = (Glitch === glitch) ? glitch : (raise Error.new("glitch: must be a Glitch instance, but got #{glitch.inspect}"))
       @raise = binding.local_variable_get(:raise)
       @capture = capture
-      @glitch = nil
       @events = []
     end
 
     def run(&block)
       @job.class.retry_on RetryableError, attempts: 10, wait: 1, jitter: 0
+      @glitch.set_action { raise @raise }
 
       ActiveSupport::Notifications.subscribed(->(event) { @events << event.dup }, @capture) do
-        glitch.inject! do
+        @glitch.inject! do
           @job.enqueue
           if block
             block.call
@@ -30,25 +30,11 @@ module ChaoticJob
         end
       end
 
-      # TODO: assert that all glitches ran
+      self
     end
 
-    def to_s
-      @glitches.map { |position, location| "#{position}-#{location}" }.join("|>")
-    end
-
-    def all_glitched?
-      @glitch.all_executed?
-    end
-
-    private
-
-    def glitch
-      @glitch ||= Glitch.new.tap do |glitch|
-        @glitches.each do |kind, location, _description|
-          glitch.public_send(kind, location) { raise @raise }
-        end
-      end
+    def glitched?
+      @glitch.executed?
     end
   end
 end

data/lib/chaotic_job/simulation.rb

@@ -1,53 +1,47 @@
 # frozen_string_literal: true
 
 # Simulation.new(job).run { |scenario| ... }
-# Simulation.new(job).permutations
 # Simulation.new(job).variants
 # Simulation.new(job).scenarios
 module ChaoticJob
   class Simulation
-    def initialize(job, callstack: nil, depth: 1, variations: 100, test: nil, seed: nil)
+    def initialize(job, callstack: nil, variations: nil, test: nil, seed: nil)
       @template = job
       @callstack = callstack || capture_callstack
-      @depth = depth
       @variations = variations
       @test = test
       @seed = seed || Random.new_seed
       @random = Random.new(@seed)
 
-      raise Error.new("callstack must be a generated via the ChaoticJob::Tracer") unless @callstack.is_a?(Stack)
+      raise Error.new("callstack must be a generated via ChaoticJob::Tracer") unless @callstack.is_a?(Stack)
     end
 
-    def run(&callback)
-      @template.class.retry_on RetryableError, attempts: @depth + 2, wait: 1, jitter: 0
+    def run(&assertions)
+      @template.class.retry_on RetryableError, attempts: 3, wait: 1, jitter: 0
 
-      debug "👾 Running #{variants.size} simulations of the total #{permutations.size} possibilities..."
+      debug "👾 Running #{@variations || "all"} simulations of the total #{variants.size} possibilities..."
 
       scenarios.map do |scenario|
-        run_scenario(scenario, &callback)
+        run_scenario(scenario, &assertions)
         print "·"
       end
     end
 
-    def permutations
+    def variants
       error_locations = @callstack.map do |event, key|
         ["before_#{event}", key]
       end
-      error_locations.permutation(@depth)
-    end
 
-    def variants
-      return permutations if @variations.nil?
+      return error_locations if @variations.nil?
 
-      permutations.to_a.sample(@variations, random: @random)
+      error_locations.sample(@variations, random: @random)
     end
 
     def scenarios
-      variants.map do |glitches|
+      variants.map do |(event, key)|
         job = clone_job_template
-        scenario = Scenario.new(job, glitches: glitches)
-        job.job_id = scenario.to_s
-        scenario
+        glitch = Glitch.public_send(event, key)
+        Scenario.new(job, glitch: glitch)
       end
     end
 
@@ -63,13 +57,14 @@ module ChaoticJob
       callstack
     end
 
-    def run_scenario(scenario, &callback)
+    def run_scenario(scenario, &assertions)
       debug "👾 Running simulation with scenario: #{scenario}"
       @test.before_setup
-      @test.simulation_scenario = scenario.to_s
+      @test.simulation_scenario = scenario
       scenario.run
       @test.after_teardown
-      callback.call(scenario)
+      @test.assert scenario.glitched?, "Scenario did not execute glitch: #{scenario.glitch}"
+      assertions.call(scenario)
     ensure
       @test.simulation_scenario = nil
     end

data/lib/chaotic_job/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module ChaoticJob
-  VERSION = "0.5.0"
+  VERSION = "0.7.0"
 end

data/lib/chaotic_job.rb

@@ -66,20 +66,22 @@ module ChaoticJob
       Performer.perform_all_after(time)
     end
 
-    def run_simulation(job, depth: nil, variations: nil, callstack: nil, &block)
+    def run_simulation(job, variations: nil, callstack: nil, &block)
       seed = defined?(RSpec) ? RSpec.configuration.seed : Minitest.seed
       kwargs = {test: self, seed: seed}
-      kwargs[:depth] = depth if depth
       kwargs[:variations] = variations if variations
       kwargs[:callstack] = callstack if callstack
       self.simulation_scenario = nil
       Simulation.new(job, **kwargs).run(&block)
     end
 
-    def run_scenario(job, glitch: nil, glitches: nil, raise: nil, capture: nil, &block)
-      kwargs = {glitches: glitches || [glitch]}
+    def run_scenario(job, glitch:, raise: nil, capture: nil, &block)
+      kwargs = {}
+
+      kwargs[:glitch] = glitch
       kwargs[:raise] = binding.local_variable_get(:raise) if binding.local_variable_get(:raise)
       kwargs[:capture] = capture if capture
+
       if block
         Scenario.new(job, **kwargs).run(&block)
       else
@@ -87,6 +89,18 @@ module ChaoticJob
       end
     end
 
+    def glitch_before_line(key, &block)
+      Glitch.before_line(key, &block)
+    end
+
+    def glitch_before_call(key, ...)
+      Glitch.before_call(key, ...)
+    end
+
+    def glitch_before_return(key, return_type = nil, &block)
+      Glitch.before_return(key, return_type, &block)
+    end
+
     def assert(test, msg = nil)
       return super unless @simulation_scenario
 
@@ -95,7 +109,8 @@ module ChaoticJob
         default_msg = "Expected #{mu_pp test} to be truthy."
         custom_msg = msg.is_a?(Proc) ? msg.call : msg
         full_msg = custom_msg || default_msg
-        "  #{@simulation_scenario}\n#{full_msg}"
+        indented_scenario = @simulation_scenario.to_s.split("\n").join("\n  ")
+        "  #{indented_scenario}\n#{full_msg}"
       end
 
       super(test, contextual_msg)

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: chaotic_job
 version: !ruby/object:Gem::Version
-  version: 0.5.0
+  version: 0.7.0
 platform: ruby
 authors:
 - Stephen Margheim
 bindir: exe
 cert_chain: []
-date: 2025-06-03 00:00:00.000000000 Z
+date: 2025-06-09 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activejob