sidekiq-rescue 0.2.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d8bcb824ca2945f9cf090e5cb6adfd0a4bc77366461c068915008e5ea953db31
-  data.tar.gz: 97011a0a37cefea2f6500e7d96147b50059aa46289112de6ce09a5aa9773f963
+  metadata.gz: 01671c36411fb052587442c08db979d740c563e6d74311ed1c4d17293b266745
+  data.tar.gz: 90de1649e943e5a422dc6d601d7b7425ee74958418ed9f89ef635c27aac1b0b7
 SHA512:
-  metadata.gz: 1f192b50d1b21265ef6902ce095129c0287df13c6684fc4eeaa125a70bdfe722aec40a2d7cc74dd1bbbf937bdeca436e3870ebe00ef1d3b7d818433df6efda08
-  data.tar.gz: 79a2a91abcdb591ed762084676ec58732dfb4ce797fbd7335b42b8497933b1e383ec126ed64b16e5dabd5b7925d8c1b7bf0e931cd7cc71b90051bb6a23a0d045
+  metadata.gz: c60d5673e1c25f7634820a760e4e4f0463ab618187147f96a412527db866944f7a37b44327d50053066c83b2e2694c09570befba78902053969b27a116968df7
+  data.tar.gz: 291b24d3f090b425702bd9ec2f644f8bb92e374deb7cb20d9553574b30b3502d7091544de81a2769238fbb3f4ea3851b9939299a2630dd54d2b416285db3c587

data/CHANGELOG.md

@@ -1,5 +1,16 @@
 ## [Unreleased]
 
+## [0.3.0] - 2024-05-30
+
+- Fix issue with RSpec matcher when job is not rescueable
+- Add support for multiple invocations of the DSL #3
+- Update documentation with new features
+
+## [0.2.1] - 2024-02-27
+
+- Fix readme with correct middleware name
+- Add RSpec matchers
+
 ## [0.2.0] - 2024-02-03
 
 - Rename `Sidekiq::Rescue::DSL` to `Sidekiq::Rescue::Dsl`
@@ -15,6 +26,7 @@
 - Add documentation
 - Add CI
 
-[Unreleased]: https://github.com/moofkit/sidekiq-rescue/compare/v0.2.0...HEAD
+[Unreleased]: https://github.com/moofkit/sidekiq-rescue/compare/v0.2.1...HEAD
+[0.2.1]: https://github.com/moofkit/sidekiq-rescue/releases/tag/v0.2.1
 [0.2.0]: https://github.com/moofkit/sidekiq-rescue/releases/tag/v0.2.0
 [0.1.0]: https://github.com/moofkit/sidekiq-rescue/releases/tag/v0.1.0

data/README.md

@@ -3,6 +3,31 @@
 [![Build Status](https://github.com/moofkit/sidekiq-rescue/actions/workflows/main.yml/badge.svg?branch=main)](https://github.com/moofkit/sidekiq-rescue/actions/workflows/main.yml)
 
 [Sidekiq](https://github.com/sidekiq/sidekiq) plugin to rescue jobs from expected errors and retry them later.
+Catch expected errors and retry the job with a delay and a limit. It's useful when you want to retry jobs that failed due to expected errors and not spam your exception tracker with these errors. If the exception will getting raised beyond the limit, it will be re-raised and will be handled by Sidekiq standard retry mechanism.
+
+Handlers are searched from bottom to top, and up the inheritance chain. The first handler that `exception.is_a?(klass)` holds true will be used.
+
+## Example
+
+```ruby
+class MyJob
+  include Sidekiq::Job
+  include Sidekiq::Rescue::Dsl
+
+  sidekiq_rescue CustomAppException # defaults to 60 seconds delay and 10 retries
+  sidekiq_rescue AnotherCustomAppException, delay: ->(counter) { counter * 2 }
+  sidekiq_rescue CustomInfrastructureException, delay: 5.minutes
+  sidekiq_rescue ActiveRecord::Deadlocked, delay: 5.seconds, limit: 3
+  sidekiq_rescue Net::OpenTimeout, Timeout::Error, limit: 10 # retries at most 10 times for Net::OpenTimeout and Timeout::Error combined
+
+  def perform(*args)
+    # Might raise CustomAppException, AnotherCustomAppException, or YetAnotherCustomAppException for something domain specific
+    # Might raise ActiveRecord::Deadlocked when a local db deadlock is detected
+    # Might raise Net::OpenTimeout or Timeout::Error when the remote service is down
+  end
+end
+```
+
 
 ## Installation
 
@@ -27,7 +52,7 @@ Or install it yourself as:
 ```ruby
 Sidekiq.configure_server do |config|
   config.server_middleware do |chain|
-    chain.add Sidekiq::Rescue::Middleware
+    chain.add Sidekiq::Rescue::ServerMiddleware
   end
 end
 ```
@@ -93,6 +118,68 @@ Sidekiq::Rescue.configure do |config|
 end
 ```
 
+### Testing
+
+1. Unit tests (recommended)
+
+In case you want to test the rescue configuration, this gem provides RSpec matchers:
+
+```ruby
+RSpec.cofigure do |config|
+  config.include Sidekiq::Rescue::RSpec::Matchers, type: :job
+end
+
+RSpec.describe MyJob do
+  it "rescues from expected errors" do
+    expect(MyJob).to have_sidekiq_rescue(ExpectedError)
+  end
+end
+```
+
+It also provides a way to test the delay and limit:
+
+```ruby
+RSpec.describe MyJob do
+  it "rescues from expected errors with custom delay and limit" do
+    expect(MyJob).to have_sidekiq_rescue(ExpectedError).with_delay(60).with_limit(5)
+  end
+end
+```
+
+2. Integration tests with `Sidekiq::Testing`
+Firstly, you need to configure `Sidekiq::Testing` to use `Sidekiq::Rescue::ServerMiddleware` middleware:
+
+```ruby
+# spec/spec_helper.rb or spec/rails_helper.rb
+require "sidekiq/testing"
+
+RSpec.configure do |config|
+  config.before(:all) do
+    Sidekiq::Testing.fake!
+
+    Sidekiq::Testing.server_middleware do |chain|
+      chain.add Sidekiq::Rescue::ServerMiddleware
+    end
+  end
+end
+```
+
+And test the job with the next snippet
+
+```ruby
+# spec/jobs/my_job_spec.rb
+RSpec.describe MyJob do
+  before do
+    allow(ApiClient).to receive(:new).and_raise(ApiClient::SomethingWentWrongError)
+  end
+
+  it "retries job if it fails with ExpectedError" do
+    MyJob.perform_async('test')
+    expect { MyJob.perform_one }.not_to raise_error # pefrom_one is a method from Sidekiq::Testing that runs the job once
+  end
+end
+```
+
 ## Use cases
 
 Sidekiq::Rescue is useful when you want to retry jobs that failed due to expected errors and not spam your exception tracker with these errors. For example, you may want to retry a job that failed due to a network error or a temporary outage of a third party service, rather than a bug in your code.
@@ -162,7 +249,7 @@ end
 ## Motivation
 
 Sidekiq provides a retry mechanism for jobs that failed due to unexpected errors. However, it does not provide a way to retry jobs that failed due to expected errors. This gem aims to fill this gap.
-In addition, it provides a way to configure the number of retries and the delay between retries independently from the Sidekiq standard retry mechanism.
+In addition, it provides a way to configure the number of retries and the delay between retries independently from the Sidekiq standard retry mechanism. Mostly inspired by [ActiveJob](https://edgeapi.rubyonrails.org/classes/ActiveJob/Exceptions/ClassMethods.html#method-i-retry_on)
 
 ## Supported Ruby versions
 

data/lib/sidekiq/rescue/dsl.rb

@@ -24,16 +24,15 @@ module Sidekiq
         # @raise [ArgumentError] if limit is not an Integer
         # @example
         #  sidekiq_rescue NetworkError, delay: 60, limit: 10
-        def sidekiq_rescue(*error, delay: nil, limit: nil)
-          error = validate_and_unpack_error_argument(error)
+        def sidekiq_rescue(*errors, delay: Sidekiq::Rescue.config.delay, limit: Sidekiq::Rescue.config.limit)
+          unpacked_errors = validate_and_unpack_error_argument(errors)
           validate_delay_argument(delay)
           validate_limit_argument(limit)
+          assign_sidekiq_rescue_options(unpacked_errors, delay, limit)
+        end
 
-          self.sidekiq_rescue_options = {
-            error: error,
-            delay: delay || Sidekiq::Rescue.config.delay,
-            limit: limit || Sidekiq::Rescue.config.limit
-          }
+        def sidekiq_rescue_options_for(error)
+          sidekiq_rescue_options&.find { |k, _v| k.include?(error) }&.last
         end
 
         private
@@ -63,6 +62,11 @@ module Sidekiq
         def validate_limit_argument(limit)
           raise ArgumentError, "limit must be integer" if limit && !limit.is_a?(Integer)
         end
+
+        def assign_sidekiq_rescue_options(errors, delay, limit)
+          self.sidekiq_rescue_options ||= {}
+          self.sidekiq_rescue_options.merge!(errors => { delay: delay, limit: limit })
+        end
       end
     end
     # Alias for Dsl; TODO: remove in 1.0.0

data/lib/sidekiq/rescue/rspec/matchers.rb

@@ -0,0 +1,57 @@
+# frozen_string_literal: true
+
+return unless defined?(RSpec)
+
+require "rspec/matchers"
+
+module Sidekiq
+  module Rescue
+    module RSpec
+      # RSpec matchers for Sidekiq::Rescue
+      module Matchers
+        ::RSpec::Matchers.define :have_sidekiq_rescue do |expected| # rubocop:disable Metrics/BlockLength
+          description { "be rescueable with #{expected}" }
+          failure_message do |actual|
+            str = "expected #{actual} to be rescueable with #{expected}"
+            str += " and delay #{@delay}" if @delay
+            str += " and limit #{@limit}" if @limit
+            str
+          end
+          failure_message_when_negated { |actual| "expected #{actual} not to be rescueable with #{expected}" }
+
+          chain :with_delay do |delay|
+            @delay = delay
+          end
+
+          chain :with_limit do |limit|
+            @limit = limit
+          end
+
+          match do |actual|
+            matched = actual.is_a?(Class) &&
+                      actual.include?(Sidekiq::Rescue::Dsl) &&
+                      actual.respond_to?(:sidekiq_rescue_options) &&
+                      actual&.sidekiq_rescue_options&.keys&.flatten&.include?(expected)
+
+            return false unless matched
+
+            options = actual.sidekiq_rescue_options_for(expected)
+
+            (@delay.nil? || options.fetch(:delay) == @delay) &&
+              (@limit.nil? || options.fetch(:limit) == @limit)
+          end
+
+          match_when_negated do |actual|
+            raise NotImplementedError, "it's confusing to use `not_to be_rescueable` with `with_delay`" if @delay
+            raise NotImplementedError, "it's confusing to use `not_to be_rescueable` with `with_limit`" if @limit
+
+            actual.is_a?(Class) &&
+              actual.include?(Sidekiq::Rescue::Dsl) &&
+              actual.respond_to?(:sidekiq_rescue_options) &&
+              !Array(actual&.sidekiq_rescue_options&.[](:error)).include?(expected)
+          end
+        end
+      end
+    end
+  end
+end

data/lib/sidekiq/rescue/server_middleware.rb

@@ -11,9 +11,8 @@ module Sidekiq
 
       def call(job_instance, job_payload, _queue, &block)
         job_class = job_instance.class
-        options = job_class.sidekiq_rescue_options if job_class.respond_to?(:sidekiq_rescue_options)
-        if options
-          sidekiq_rescue(job_payload, **options, &block)
+        if job_class.respond_to?(:sidekiq_rescue_options) && !job_class.sidekiq_rescue_options.nil?
+          sidekiq_rescue(job_payload, job_class, &block)
         else
           yield
         end
@@ -21,19 +20,30 @@ module Sidekiq
 
       private
 
-      def sidekiq_rescue(job_payload, delay:, limit:, error:, **)
+      def sidekiq_rescue(job_payload, job_class)
         yield
-      rescue *error => e
-        rescue_counter = increment_rescue_counter(job_payload)
-        raise e if rescue_counter > limit
+      rescue StandardError => e
+        error_group, options = job_class.sidekiq_rescue_options.reverse_each.find do |error_group, _options|
+          Array(error_group).any? { |error| e.is_a?(error) }
+        end
+        raise e unless error_group
+
+        rescue_error(e, error_group, options, job_payload)
+      end
+
+      def rescue_error(error, error_group, options, job_payload)
+        delay, limit = options.fetch_values(:delay, :limit)
+        rescue_counter = increment_rescue_counter_for(error_group, job_payload)
+        raise error if rescue_counter > limit
 
         reschedule_at = calculate_reschedule_time(delay, rescue_counter)
-        log_reschedule_info(rescue_counter, e, reschedule_at)
-        reschedule_job(job_payload, reschedule_at, rescue_counter)
+        log_reschedule_info(rescue_counter, error, reschedule_at)
+        reschedule_job(job_payload: job_payload, reschedule_at: reschedule_at, rescue_counter: rescue_counter,
+                       error_group: error_group)
       end
 
-      def increment_rescue_counter(job_payload)
-        rescue_counter = job_payload["sidekiq_rescue_counter"].to_i
+      def increment_rescue_counter_for(error_group, job_payload)
+        rescue_counter = job_payload.dig("sidekiq_rescue_exceptions_counter", error_group.to_s) || 0
         rescue_counter += 1
         rescue_counter
       end
@@ -52,8 +62,10 @@ module Sidekiq
                                     "#{error.message}; rescheduling at #{reschedule_at}")
       end
 
-      def reschedule_job(job_payload, reschedule_at, rescue_counter)
-        Sidekiq::Client.push(job_payload.merge("at" => reschedule_at, "sidekiq_rescue_counter" => rescue_counter))
+      def reschedule_job(job_payload:, reschedule_at:, rescue_counter:, error_group:)
+        payload = job_payload.merge("at" => reschedule_at,
+                                    "sidekiq_rescue_exceptions_counter" => { error_group.to_s => rescue_counter })
+        Sidekiq::Client.push(payload)
       end
     end
   end

data/lib/sidekiq/rescue/version.rb

@@ -2,6 +2,6 @@
 
 module Sidekiq
   module Rescue
-    VERSION = "0.2.0"
+    VERSION = "0.3.0"
   end
 end

data/lib/sidekiq_rescue.rb

@@ -7,3 +7,4 @@ require_relative "sidekiq/rescue"
 require_relative "sidekiq/rescue/version"
 require_relative "sidekiq/rescue/dsl"
 require_relative "sidekiq/rescue/server_middleware"
+require_relative "sidekiq/rescue/rspec/matchers"

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: sidekiq-rescue
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.3.0
 platform: ruby
 authors:
 - Dmitrii Ivliev
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2024-02-03 00:00:00.000000000 Z
+date: 2024-05-30 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: sidekiq
@@ -39,6 +39,7 @@ files:
 - lib/sidekiq/rescue.rb
 - lib/sidekiq/rescue/config.rb
 - lib/sidekiq/rescue/dsl.rb
+- lib/sidekiq/rescue/rspec/matchers.rb
 - lib/sidekiq/rescue/server_middleware.rb
 - lib/sidekiq/rescue/version.rb
 - lib/sidekiq_rescue.rb
@@ -49,7 +50,7 @@ metadata:
   homepage_uri: https://github.com/moofkit/sidekiq-rescue
   source_code_uri: https://github.com/moofkit/sidekiq-rescue
   changelog_uri: https://github.com/moofkit/sidekiq-rescue/blob/master/CHANGELOG.md
-  documentation_uri: https://rubydoc.info/gems/sidekiq-rescue/0.2.0
+  documentation_uri: https://rubydoc.info/gems/sidekiq-rescue/0.3.0
   rubygems_mfa_required: 'true'
 post_install_message:
 rdoc_options: []
@@ -66,7 +67,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.4.10
+rubygems_version: 3.5.9
 signing_key:
 specification_version: 4
 summary: Rescue Sidekiq jobs on expected error and reschedule them