sidekiq-disposal 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8ecd41e2b17b1168b119bf60002c1680c960561f0f4c1d972648a08386ed1423
-  data.tar.gz: 14df2e7593177e8c2fb90d96d1acbe3ed57206987cb59763e06fdf9db4a7ba63
+  metadata.gz: 8d1b1751539032559a1b02629ef6e6c71df4a906ade1d44d8951ac18e190d44a
+  data.tar.gz: 3faafb76a82332c5ca86fbfbd3ea3f8363ebfc52c22537cdfc5014a5e0881680
 SHA512:
-  metadata.gz: 9e5e2a2084643fe72cafc7cbe7eb44dfacf500a63744d75585a6a38f63f8bf853090589183de163e487fbbeb975cd3e8e3ee8f11459c5b3e03a0ef890da2770e
-  data.tar.gz: 7bc4001093d88ef742f20102b5b11d98087726e9b076a7e733c8f695c200799a9c11282ef7c43d6b5d7de05fbd6a879ff2c83638e79ab86eaecc90b3e1aec24f
+  metadata.gz: 5194a09ea875c4a876b92d9e08cf1566365ed5ffeedc6aaee3217182d323614d07b3ca893291038f50fa41960ed445894e97e71ce1754d614cb90ee220847093
+  data.tar.gz: 9d4b2232fce24cf733eb540bbb64280c9aedd4b341eafdd075107e15ba42228a382f3d10ea80ca3ff240ba79eb7f77f1da1e33f7cecc60845ed9d8fa49bcb422

data/CHANGELOG.md

@@ -7,4 +7,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
-- A new thing!
+## [0.2.0] - 2024-12-19
+
+### Changed
+
+- **Breaking**: Replace all usages of "drop" with "discard" to align with Sidekiq terminology ([#4](https://github.com/hibachrach/sidekiq-disposal/pull/4)).
+  - To upgrade:
+    1. Replace all usages of `:drop` with `:discard`
+    1. Replace all usages of `Sidekiq::Disposal::Client::REDIS_DROP_TARGET_SET` with `Sidekiq::Disposal::Client::REDIS_DISCARD_TARGET_SET`
+    1. Replace all usages of `Sidekiq::Disposal::JobDropped` with `Sidekiq::Disposal::JobDiscarded`
+    1. Replace all usages of `Sidekiq::Disposal::Client#drop_target?` with `Sidekiq::Disposal::Client#discard_target?`
+    1. In Redis, execute `COPY sidekiq-disposal:drop_targets sidekiq-disposal:discard_targets`
+- Eliminate round trip time when calling Redis in Sidekiq middleware ([#3](https://github.com/hibachrach/sidekiq-disposal/pull/3))
+
+## [0.1.0] - 2024-12-13
+
+- A new thing! The initial release of `sidekiq-disposal`. 🎉

data/README.md

@@ -1,6 +1,8 @@
 # Sidekiq::Disposal
 
-A [Sidekiq][sidekiq] extension mark Sidekiq Jobs to be disposed of based on the Job ID, Batch ID, or Job Class.
+[![Gem Version](https://badge.fury.io/rb/sidekiq-disposal.svg?icon=si%3Arubygems&icon_color=%23ff2600)](https://badge.fury.io/rb/sidekiq-disposal)
+
+A [Sidekiq][sidekiq] extension to mark Sidekiq Jobs to be disposed of based on the Job ID, Batch ID, or Job Class.
 Disposal here means to either `:kill` the Job (send to the Dead queue) or `:discard` it (throw it away), at the time the job is picked up and processed by Sidekiq.
 A disposed Job's `#perform` method will _not_ be called.
 
@@ -16,13 +18,13 @@ Or… any number of other ways that some Job, Batch, or Job Class has been enque
 Install the gem and add to the application's Gemfile by executing:
 
 ```console
-bundle add UPDATE_WITH_YOUR_GEM_NAME_IMMEDIATELY_AFTER_RELEASE_TO_RUBYGEMS_ORG
+bundle add sidekiq-disposal
 ```
 
 If bundler is not being used to manage dependencies, install the gem by executing:
 
 ```console
-gem install UPDATE_WITH_YOUR_GEM_NAME_IMMEDIATELY_AFTER_RELEASE_TO_RUBYGEMS_ORG
+gem install sidekiq-disposal
 ```
 
 ## Usage
@@ -37,8 +39,8 @@ client = Sidekiq::Disposal::Client.new
 
 A Job marked to be killed means it will be moved to the Dead queue.
 
-```
-# Mark a specific Job, by its ID to be killed
+```ruby
+# Mark a specific Job to be killed by specifying its Job ID
 client.mark(:kill, :jid, some_job_id)
 
 # Mark a Batch of Jobs to be killed, by Batch ID
@@ -50,7 +52,7 @@ client.mark(:kill, :bid, "SomeJobClass")
 
 A Job, Batch, or Job Class can also be `#unmarked` for disposal via a corresponding API.
 
-```
+```ruby
 # Un-mark a specific Job from being killed, by Job ID
 client.unmark(:kill, :jid, some_job_id)
 
@@ -61,50 +63,50 @@ client.unmark(:kill, :bid, some_batch_id)
 client.unmark(:kill, :bid, "SomeJobClass")
 ```
 
-### Marking to Drop
+### Marking to Discard
 
-Similarly, a Job, Batch, or Job Class can be marked to be dropped.
-Dropped jobs are discarded by Sidekiq - think of them as simply being deleted from the queue, without ever being run.
+Similarly, a Job, Batch, or Job Class can be marked to be discarded.
+Discarded jobs are thrown away by Sidekiq - think of them as simply being deleted from the queue, without ever being run.
 
-```
-# Mark a specific Job, by its ID to be dropped
-client.mark(:drop, :jid, some_job_id)
+```ruby
+# Mark a specific Job to be discarded by specifying its Job ID 
+client.mark(:discard, :jid, some_job_id)
 
-# Mark a Batch of Jobs to be dropped, by Batch ID
-client.mark(:drop, :bid, some_batch_id)
+# Mark a Batch of Jobs to be discarded, by Batch ID
+client.mark(:discard, :bid, some_batch_id)
 
-# Mark an entire Job Class to be dropped
-client.mark(:drop, :bid, "SomeJobClass")
+# Mark an entire Job Class to be discarded
+client.mark(:discard, :bid, "SomeJobClass")
 ```
 
-And again, there is a corresponding API for un-marking a Job, Batch, or Job Class from being dropped.
+And again, there is a corresponding API for un-marking a Job, Batch, or Job Class from being discarded.
 
-```
-# Un-mark a specific Job from being dropped, by Job ID
-client.unmark(:drop, :jid, some_job_id)
+```ruby
+# Un-mark a specific Job from being discarded, by Job ID
+client.unmark(:discard, :jid, some_job_id)
 
-# Un-mark a Batch of Jobs from being dropped, by Batch ID
-client.unmark(:drop, :bid, some_batch_id)
+# Un-mark a Batch of Jobs from being discarded, by Batch ID
+client.unmark(:discard, :bid, some_batch_id)
 
-# Un-mark an entire Job Class from being dropped
-client.unmark(:drop, :bid, "SomeJobClass")
+# Un-mark an entire Job Class from being discarded
+client.unmark(:discard, :bid, "SomeJobClass")
 ```
 
 ### Un-marking All
 
-Clearing all `:kill` or `:drop` marks can be done in one fell swoop as well.
+Clearing all `:kill` or `:discard` marks can be done in one fell swoop as well.
 
 ```ruby
 client.unmark_all(:kill)
 
 # or …
 
-client.unmark_all(:drop)
+client.unmark_all(:discard)
 ```
 
 ## Configuration
 
-With `sidekiq-dispose` installed, [register its Sidekiq server middleware][sidekiq-register-middleware].
+With `sidekiq-disposal` installed, [register its Sidekiq server middleware][sidekiq-register-middleware].
 Typically this is done via `config/initializers/sidekiq.rb` in a Rails app.
 
 ```ruby
@@ -115,24 +117,24 @@ Sidekiq.configure_server do |config|
 end
 ```
 
-This piece of middleware checks each job, after it's been dequeued, but before it's `#perform` has been called, to see if it should be disposed of.
-If the job is marked for disposal (by Job ID, Batch ID, or Job Class) a corresponding error is raised by the middleware.
+This piece of middleware checks each job, after it's been dequeued, but before its `#perform` has been called, to see if it should be disposed of.
+If the job is marked for disposal (by Job ID, Batch ID, or Job Class), a corresponding error is raised by the middleware.
 
-A Job marked `:kill` will raise a `Sidekiq::Disposal::JobKilled` error, while one marked `:drop` will raise `Sidekiq::Disposal::JobDropped`.
-Out of the box these errors will cause [Sidekiq's error handling and retry mechanism][sidekiq-retries] to kick in, re-enqueueing the Job.
+A Job marked `:kill` will raise a `Sidekiq::Disposal::JobKilled` error, while one marked `:discard` will raise `Sidekiq::Disposal::JobDiscarded`.
+Out of the box, these errors will cause [Sidekiq's error handling and retry mechanism][sidekiq-retries] to kick in, re-enqueueing the Job.
 And round-and-round it will go until the default error/death handling kicks in.
 
 To avoid this, you need to handle those specific `Sidekiq::Disposal` errors accordingly.
 
-Adjust the base Sidekiq Job class, often called `ApplicationJob` or similar, to overwrite or tweak the `#sidekiq_retry_in` method:
+Adjust the base Sidekiq Job class, often called `ApplicationJob` or similar, so the `sidekiq_retry_in` uses a block similar to this:
 
 ```ruby
 sidekiq_retry_in do |_count, exception, jobhash|
   case exception
-  when Infrastructure::Sidekiq::Disposal::JobKilled
+  when Sidekiq::Disposal::JobKilled
     # Optionally log/collect telemetry here too…
     :kill
-  when Infrastructure::Sidekiq::Disposal::JobDropped
+  when Sidekiq::Disposal::JobDiscarded
     # Optionally log/collect telemetry here too…
     :discard
   end
@@ -142,12 +144,12 @@ end
 _NOTE_: If is not a base job, consider adding one, or you'll need to add this to every job you want to be disposable.
 
 Returning `:kill` from this method will cause Sidekiq to immediately move the Job to the Dead Queue.
-Similarly, returning `:discard` will cause Sidekiq to drop the job on the floor.
+Similarly, returning `:discard` will cause Sidekiq to discard the job on the floor.
 Either way, the Job's `#perform` is never called.
 
 ### Non-Disposable Jobs
 
-By default all Jobs are disposable, meaning they _can_ be marked to be `:kill`-ed or `:drop`-ed.
+By default all Jobs are disposable, meaning they _can_ be marked to be `:kill`-ed or `:discard`-ed.
 However, checking if a specific Job should be disposed of is not free; it requires round trip(s) to Redis.
 Therefore, you might want to make some Jobs non-disposable to avoid these extra round trips.
 Or because there are some Jobs that simply should never be disposed of for… _reasons_.

data/lib/sidekiq/disposal/client.rb

@@ -5,7 +5,7 @@ require "sidekiq"
 module Sidekiq
   module Disposal
     # A client for marking enqueued jobs for disposal. Disposal can be a job
-    # getting "killed" (sent straight to the dead queue/morgue) or "dropped"
+    # getting "killed" (sent straight to the dead queue/morgue) or "discarded"
     # (hard deleted entirely).
     #
     # This task is accomplished with "markers": A job can be "marked" for a
@@ -18,7 +18,7 @@ module Sidekiq
     # information).
     class Client
       REDIS_KILL_TARGET_SET = "sidekiq-disposal:kill_targets"
-      REDIS_DROP_TARGET_SET = "sidekiq-disposal:drop_targets"
+      REDIS_DISCARD_TARGET_SET = "sidekiq-disposal:discard_targets"
 
       ALLOWED_MARKER_TYPES = [
         :jid,
@@ -30,7 +30,7 @@ module Sidekiq
         @sidekiq_api = sidekiq_api
       end
 
-      # @param disposal_method [:kill, :drop] How to handle job
+      # @param disposal_method [:kill, :discard] How to handle job
       # @param marker_type [:jid, :bid, :class_name]
       # @param marker [String]
       def mark(disposal_method, marker_type, marker)
@@ -41,7 +41,7 @@ module Sidekiq
         end
       end
 
-      # @param disposal_method [:kill, :drop] How to handle job
+      # @param disposal_method [:kill, :discard] How to handle job
       # @param marker_type [:jid, :bid, :class_name]
       # @param marker [String]
       def unmark(disposal_method, marker_type, marker)
@@ -68,8 +68,32 @@ module Sidekiq
         job_in_target_set?(job, disposal_target_set(:kill))
       end
 
-      def drop_target?(job)
-        job_in_target_set?(job, disposal_target_set(:drop))
+      def discard_target?(job)
+        job_in_target_set?(job, disposal_target_set(:discard))
+      end
+
+      # @return [:kill, :discard, nil] Which disposal method the job
+      # is targeted for. If job is not targeted for disposal, `nil`
+      # is returned.
+      def target_disposal_method(job)
+        # We're using `pipelined` as an optimization here to avoid
+        # two separate trips to Redis
+        kill_matches, discard_matches = redis do |conn|
+          conn.pipelined do |pipeline|
+            # `SMISEMBERS setname element1 [element2 ...]` asks whether each
+            # element given is in `setname`; redis-client (the low-level redis
+            # api used by Sidekiq) returns an array of integer answers for
+            # each element: 1 if it's a member, and 0 otherwise.
+            pipeline.smismember(disposal_target_set(:kill), formatted_markers(job))
+            pipeline.smismember(disposal_target_set(:discard), formatted_markers(job))
+          end
+        end
+
+        if kill_matches.any? { |match| match == 1 }
+          :kill
+        elsif discard_matches.any? { |match| match == 1 }
+          :discard
+        end
       end
 
       private
@@ -82,10 +106,10 @@ module Sidekiq
         case disposal_method
         when :kill
           REDIS_KILL_TARGET_SET
-        when :drop
-          REDIS_DROP_TARGET_SET
+        when :discard
+          REDIS_DISCARD_TARGET_SET
         else
-          raise ArgumentError, "disposal_method must be either :kill or :drop, instead got: #{disposal_method}"
+          raise ArgumentError, "disposal_method must be either :kill or :discard, instead got: #{disposal_method}"
         end
       end
 

data/lib/sidekiq/disposal/server_middleware.rb

@@ -15,10 +15,13 @@ module Sidekiq
       def call(job_instance, job, _queue)
         if job_instance && !job_instance.class.get_sidekiq_options.fetch("disposable", true)
           yield
-        elsif client.kill_target?(job)
-          raise JobKilled
-        elsif client.drop_target?(job)
-          raise JobDropped
+        elsif (disposal_method = client.target_disposal_method(job))
+          case disposal_method
+          when :kill
+            raise JobKilled
+          when :discard
+            raise JobDiscarded
+          end
         else
           yield
         end

data/lib/sidekiq/disposal/version.rb

@@ -2,6 +2,6 @@
 
 module Sidekiq
   module Disposal
-    VERSION = "0.1.0"
+    VERSION = "0.2.0"
   end
 end

data/lib/sidekiq/disposal.rb

@@ -7,11 +7,11 @@ require_relative "disposal/server_middleware"
 module Sidekiq
   # Namespace for everything related to job disposal: the process of putting
   # jobs' markers (i.e. identifying features) on a list so they can be "killed"
-  # (sent immediately to dead set/morgue) or "dropped" (completely discarded
+  # (sent immediately to dead set/morgue) or "discarded" (completely discarded
   # from Sidekiq) when picked up from the queue.
   module Disposal
     Error = Class.new(StandardError)
     JobKilled = Class.new(Error)
-    JobDropped = Class.new(Error)
+    JobDiscarded = Class.new(Error)
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: sidekiq-disposal
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors:
 - Hazel Bachrach
@@ -9,7 +9,7 @@ authors:
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2024-12-13 00:00:00.000000000 Z
+date: 2024-12-19 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: sidekiq