acidic_job 1.0.0.pre10 → 1.0.0.pre13

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 95b8fbdebe38a639f9b905df6ac1f5b853b09383d0810696c89c6c9e001efffe
-  data.tar.gz: b7e92f9b98461e2f39122405b6e3278fb7ed8a0edec65963026f7e2665bb5042
+  metadata.gz: ff4783bfe9e3ff6a03683ed14a9f21a5227f6114bc5824cdee71a66f252d7fa1
+  data.tar.gz: d905d8ad1848d77ae9fade6b75527ae113675ba3c39b66149c750f9adb64d83b
 SHA512:
-  metadata.gz: 13b5ea59163e833f5b698625d841fdcdc0a630b46880526df3274926e696b227b41946fd6622e121b9afd90ad3cc94589f6e864188e871d70f03c77168e3ad5e
-  data.tar.gz: 27e5b74147706f227875f4f4a61f084298c42e615149cb0a74670af841df8fcce9eb4f95d11935561912af61d6d8759c21c9ca9862e897f765b49c78b1e42ebd
+  metadata.gz: 1af39ccff6bb5d414ab32344887d60f4cf721d9272ec0c98c9686053d3426fd2da98d82b9fe35fd45ebc6b719b8e1df6ce922abbec86653a91bea736f9522696
+  data.tar.gz: f0f8db378646cd3e948a2406c2237b44c9e6a551dafcbe1a687cbfadc5a15713e3b6a177a18ccf81146b024cb64b44085a14c8250fd8b364f16c911d3a202f5b

data/.tool-versions

@@ -0,0 +1 @@
+ruby 3.0.2

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    acidic_job (1.0.0.pre10)
+    acidic_job (1.0.0.pre13)
       activerecord (>= 6.1.0)
       activesupport
 

data/README.md

@@ -1,5 +1,8 @@
 # AcidicJob
 
+[![Gem Version](https://badge.fury.io/rb/acidic_job.svg)](https://badge.fury.io/rb/acidic_job)
+![main workflow](https://github.com/fractaledmind/acidic_job/actions/workflows/main.yml/badge.svg)
+
 ### Idempotent operations for Rails apps (for ActiveJob or Sidekiq)
 
 At the conceptual heart of basically any software are "operations"—the discrete actions the software performs. Rails provides a powerful abstraction layer for building operations in the form of `ActiveJob`, or we Rubyists can use the tried and true power of pure `Sidekiq`. With either we can easily trigger from other Ruby code throughout our Rails application (controller actions, model methods, model callbacks, etc.); we can run operations both synchronously (blocking execution and then returning its response to the caller) and asychronously (non-blocking and the caller doesn't know its response); and we can also retry a specific operation if needed seamlessly.
@@ -64,6 +67,7 @@ It provides a suite of functionality that empowers you to create complex, robust
 * Transactional Steps — break your job into a series of steps, each of which will be run within an acidic database transaction, allowing retries to jump back to the last "recovery point".
 * Persisted Attributes — when retrying jobs at later steps, we need to ensure that data created in previous steps is still available to later steps on retry.
 * Transactionally Staged Jobs — enqueue additional jobs within the acidic transaction safely
+* Custom Idempotency Keys — use something other than the job ID for the idempotency key of the job run
 * Sidekiq Callbacks — bring ActiveJob-like callbacks into your pure Sidekiq Workers
 * Sidekiq Batches — leverage the power of Sidekiq Pro's `batch` functionality without the hassle
 
@@ -163,6 +167,51 @@ class RideCreateJob < ActiveJob::Base
 end
 ```
 
+### Custom Idempotency Keys
+
+By default, `AcidicJob` uses the job identifier provided by the queueing system (ActiveJob or Sidekiq) as the idempotency key for the job run. The idempotency key is what is used to guarantee that no two runs of the same job occur. However, sometimes we need particular jobs to be idempotent based on some other criteria. In these cases, `AcidicJob` provides a collection of tools to allow you to ensure the idempotency of your jobs.
+
+Firstly, you can configure your job class to explicitly use either the job identifier or the job arguments as the foundation for the idempotency key. A job class that calls the `acidic_by_job_id` class method (which is the default behavior) will simply make the job run's idempotency key the job's identifier:
+
+```ruby
+class ExampleJob < ActiveJob::Base
+  include AcidicJob
+  acidic_by_job_id
+
+  def perform
+  end
+end
+```
+
+Conversely, a job class can use the `acidic_by_job_args` method to configure that job class to use the arguments passed to the job as the foundation for the job run's idempotency key:
+
+```ruby
+class ExampleJob < ActiveJob::Base
+  include AcidicJob
+  acidic_by_job_args
+
+  def perform(arg_1, arg_2)
+    # the idempotency key will be based on whatever the values of `arg_1` and `arg_2` are
+  end
+end
+```
+
+These options cover the two common situations, but sometimes our systems need finer-grained control. For example, our job might take some record as the job argument, but we need to use a combination of the record identifier and record status as the foundation for the idempotency key. In these cases you can pass a `Proc` to an `acidic_by` class method:
+
+```ruby
+class ExampleJob < ActiveJob::Base
+  include AcidicJob
+  acidic_by ->(record:) { [record.id, record.status] }
+
+  def perform(record:)
+    # the idempotency key will be based on whatever the values of `record.id` and `record.status` are
+  end
+end
+```
+
+> **Note:** The signature of the `acidic_by` proc _needs to match the signature_ of the job's `perform` method.
+
+
 ### Sidekiq Callbacks
 
 In order to ensure that `AcidicJob::Staged` records are only destroyed once the related job has been successfully performed, whether it is an ActiveJob or a Sidekiq Worker, `acidic_job` also extends Sidekiq to support the [ActiveJob callback interface](https://edgeguides.rubyonrails.org/active_job_basics.html#callbacks).

data/lib/acidic_job/extensions/action_mailer.rb

@@ -7,21 +7,14 @@ module AcidicJob
     module ActionMailer
       extend ActiveSupport::Concern
 
-      def deliver_acidicly(_options = {}, idempotency_key: nil, unique_by: nil)
+      def deliver_acidicly(_options = {})
         job_class = ::ActionMailer::MailDeliveryJob
 
         job_args = [@mailer_class.name, @action.to_s, "deliver_now", @params, *@args]
         # for Sidekiq, this depends on the Sidekiq::Serialization extension
         serialized_job = job_class.new(job_args).serialize
         acidic_identifier = job_class.respond_to?(:acidic_identifier) ? job_class.acidic_identifier : :job_id
-        # use either [1] provided key, [2] provided uniqueness constraint, or [3] computed key
-        key = if idempotency_key
-                idempotency_key
-              elsif unique_by
-                IdempotencyKey.generate(unique_by: unique_by, job_class: job_class.name)
-              else
-                IdempotencyKey.new(acidic_identifier).value_for(serialized_job)
-              end
+        key = IdempotencyKey.new(acidic_identifier).value_for(serialized_job)
 
         AcidicJob::Run.create!(
           staged: true,

data/lib/acidic_job/extensions/active_job.rb

@@ -24,15 +24,7 @@ module AcidicJob
           raise UnsupportedExtension unless defined?(::ActiveJob) && self < ::ActiveJob::Base
 
           serialized_job = serialize_with_arguments(*args, **kwargs)
-          # use either [1] provided key, [2] provided uniqueness constraint, or [3] computed key
-          key = if kwargs.key?(:idempotency_key) || kwargs.key?("idempotency_key")
-                  kwargs[:idempotency_key] || kwargs["idempotency_key"]
-                elsif kwargs.key?(:unique_by) || kwargs.key?("unique_by")
-                  unique_by = [kwargs[:unique_by], kwargs["unique_by"]].compact.first
-                  IdempotencyKey.generate(unique_by: unique_by, job_class: name)
-                else
-                  IdempotencyKey.new(acidic_identifier).value_for(serialized_job)
-                end
+          key = IdempotencyKey.new(acidic_identifier).value_for(serialized_job)
 
           AcidicJob::Run.create!(
             staged: true,

data/lib/acidic_job/extensions/noticed.rb

@@ -6,12 +6,12 @@ module AcidicJob
       extend ActiveSupport::Concern
 
       class_methods do
-        def deliver_acidicly(recipients, idempotency_key: nil, unique_by: nil)
-          new.deliver_acidicly(recipients, idempotency_key: idempotency_key, unique_by: unique_by)
+        def deliver_acidicly(recipients)
+          new.deliver_acidicly(recipients)
         end
       end
 
-      def deliver_acidicly(recipients, idempotency_key: nil, unique_by: nil)
+      def deliver_acidicly(recipients)
         # THIS IS A HACK THAT COPIES AND PASTES KEY PARTS OF THE `Noticed::Base` CODE
         # IN ORDER TO ALLOW US TO TRANSACTIONALLY DELIVER NOTIFICATIONS
         # THIS IS THUS LIABLE TO BREAK WHENEVER THAT GEM IS UPDATED
@@ -37,14 +37,7 @@ module AcidicJob
             }
             serialized_job = job_class.send(:job_or_instantiate, args).serialize
             acidic_identifier = job_class.respond_to?(:acidic_identifier) ? job_class.acidic_identifier : :job_id
-            # use either [1] provided key, [2] provided uniqueness constraint, or [3] computed key
-            key = if idempotency_key
-                    idempotency_key
-                  elsif unique_by
-                    IdempotencyKey.generate(unique_by: unique_by, job_class: job_class.name)
-                  else
-                    IdempotencyKey.new(acidic_identifier).value_for(serialized_job)
-                  end
+            key = IdempotencyKey.new(acidic_identifier).value_for(serialized_job)
 
             AcidicJob::Run.create!(
               staged: true,

data/lib/acidic_job/extensions/sidekiq.rb

@@ -34,9 +34,10 @@ module AcidicJob
           end
         end
 
-        def serialize_job(args = [], _kwargs = nil)
+        def serialize_job(*args, **kwargs)
           # `@args` is only set via `deserialize`; it is not a standard Sidekiq thing
           arguments = args || @args
+          arguments += [kwargs] unless kwargs.empty?
           normalized_args = ::Sidekiq.load_json(::Sidekiq.dump_json(arguments))
           item = { "class" => self.class, "args" => normalized_args, "jid" => jid }
           sidekiq_options = sidekiq_options_hash || {}
@@ -58,15 +59,7 @@ module AcidicJob
         class_methods do
           def perform_acidicly(*args, **kwargs)
             serialized_job = serialize_with_arguments(*args, **kwargs)
-            # use either [1] provided key, [2] provided uniqueness constraint, or [3] computed key
-            key = if kwargs.key?(:idempotency_key) || kwargs.key?("idempotency_key")
-                    kwargs[:idempotency_key] || kwargs["idempotency_key"]
-                  elsif kwargs.key?(:unique_by) || kwargs.key?("unique_by")
-                    unique_by = [kwargs[:unique_by], kwargs["unique_by"]].compact.first
-                    IdempotencyKey.generate(unique_by: unique_by, job_class: name)
-                  else
-                    IdempotencyKey.new(acidic_identifier).value_for(serialized_job)
-                  end
+            key = IdempotencyKey.new(acidic_identifier).value_for(serialized_job)
 
             AcidicJob::Run.create!(
               staged: true,

data/lib/acidic_job/idempotency_key.rb

@@ -2,21 +2,22 @@
 
 module AcidicJob
   class IdempotencyKey
-    def self.generate(unique_by:, job_class:)
-      new(:job_args).value_for({ "job_class" => job_class }, Marshal.dump(unique_by))
-    end
-
     def initialize(identifier = :job_id)
       @identifier = identifier
     end
 
     def value_for(hash_or_job, *args, **kwargs)
-      return value_from_job_args(hash_or_job, *args, **kwargs) if @identifier == :job_args
-
-      value = if hash_or_job.is_a?(Hash)
-                value_from_job_id_for_hash(hash_or_job)
+      value = case @identifier
+              when Proc
+                value_from_proc(hash_or_job, *args, **kwargs)
+              when :job_args
+                value_from_job_args(hash_or_job, *args, **kwargs)
               else
-                value_from_job_id_for_obj(hash_or_job)
+                if hash_or_job.is_a?(Hash)
+                  value_from_job_id_for_hash(hash_or_job)
+                else
+                  value_from_job_id_for_obj(hash_or_job)
+                end
               end
 
       value || value_from_job_args(hash_or_job, *args, **kwargs)
@@ -62,5 +63,12 @@ module AcidicJob
 
       Digest::SHA1.hexdigest [worker_class, args, kwargs].flatten.join
     end
+
+    def value_from_proc(_hash_or_job, *args, **kwargs)
+      return if args.empty? && kwargs.empty?
+
+      idempotency_args = Array(@identifier.call(*args, **kwargs))
+      Digest::SHA1.hexdigest idempotency_args.flatten.join
+    end
   end
 end

data/lib/acidic_job/perform_wrapper.rb

@@ -4,46 +4,19 @@ module AcidicJob
   # NOTE: it is essential that this be a bare module and not an ActiveSupport::Concern
   module PerformWrapper
     def perform(*args, **kwargs)
-      super_method = method(:perform).super_method
+      @__acidic_job_args = args
+      @__acidic_job_kwargs = kwargs
 
       # we don't want to run the `perform` callbacks twice, since ActiveJob already handles that for us
-      if aj_job?
-        __acidic_job_perform_for_aj(super_method, *args, **kwargs)
-      elsif sk_job?
-        __acidic_job_perform_for_sk(super_method, *args, **kwargs)
+      if defined?(ActiveJob) && self.class < ActiveJob::Base
+        super(*args, **kwargs)
+      elsif defined?(Sidekiq) && self.class.include?(Sidekiq::Worker)
+        run_callbacks :perform do
+          super(*args, **kwargs)
+        end
       else
         raise UnknownJobAdapter
       end
     end
-
-    def sk_job?
-      defined?(Sidekiq) && self.class.include?(Sidekiq::Worker)
-    end
-
-    def aj_job?
-      defined?(ActiveJob) && self.class < ActiveJob::Base
-    end
-
-    private
-
-    # don't run `perform` callbacks, as ActiveJob already does this
-    def __acidic_job_perform_for_aj(super_method, *args, **kwargs)
-      __acidic_job_perform_base(super_method, *args, **kwargs)
-    end
-
-    # ensure to run `perform` callbacks
-    def __acidic_job_perform_for_sk(super_method, *args, **kwargs)
-      run_callbacks :perform do
-        __acidic_job_perform_base(super_method, *args, **kwargs)
-      end
-    end
-
-    # capture arguments passed to `perform` to be used by AcidicJob later
-    def __acidic_job_perform_base(super_method, *args, **kwargs)
-      @__acidic_job_args = args
-      @__acidic_job_kwargs = kwargs
-
-      super_method.call(*args, **kwargs)
-    end
   end
 end

data/lib/acidic_job/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module AcidicJob
-  VERSION = "1.0.0.pre10"
+  VERSION = "1.0.0.pre13"
 end

data/lib/acidic_job.rb

@@ -50,6 +50,7 @@ module AcidicJob
     klass.instance_variable_set(:@acidic_identifier, :job_id)
     klass.define_singleton_method(:acidic_by_job_id) { @acidic_identifier = :job_id }
     klass.define_singleton_method(:acidic_by_job_args) { @acidic_identifier = :job_args }
+    klass.define_singleton_method(:acidic_by) { |proc| @acidic_identifier = proc }
   end
 
   included do
@@ -67,6 +68,19 @@ module AcidicJob
     end
   end
 
+  def initialize(*args, **kwargs)
+    # ensure this instance variable is always defined
+    @__acidic_job_steps = []
+    @__acidic_job_args = args
+    @__acidic_job_kwargs = kwargs
+
+    if method(__method__).super_method.arity.zero?
+      super()
+    else
+      super(*args, **kwargs)
+    end
+  end
+
   def with_acidity(providing: {})
     # ensure this instance variable is always defined
     @__acidic_job_steps = []
@@ -107,7 +121,7 @@ module AcidicJob
 
     acidic_identifier = self.class.acidic_identifier
     @__acidic_job_idempotency_key ||= IdempotencyKey.new(acidic_identifier)
-                                                    .value_for(self, @__acidic_job_args, @__acidic_job_kwargs)
+                                                    .value_for(self, *@__acidic_job_args, **@__acidic_job_kwargs)
   end
   # rubocop:enable Naming/MemoizedInstanceVariableName
 
@@ -191,7 +205,7 @@ module AcidicJob
 
     ActiveRecord::Base.transaction(isolation: isolation_level) do
       run = Run.find_by(idempotency_key: idempotency_key)
-      serialized_job = serialize_job(@__acidic_job_args, @__acidic_job_kwargs)
+      serialized_job = serialize_job(*@__acidic_job_args, **@__acidic_job_kwargs)
 
       if run.present?
         # Programs enqueuing multiple jobs with different parameters but the

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: acidic_job
 version: !ruby/object:Gem::Version
-  version: 1.0.0.pre10
+  version: 1.0.0.pre13
 platform: ruby
 authors:
 - fractaledmind
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2022-03-13 00:00:00.000000000 Z
+date: 2022-03-16 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord
@@ -63,6 +63,7 @@ files:
 - ".gitignore"
 - ".rubocop.yml"
 - ".ruby_version"
+- ".tool-versions"
 - Gemfile
 - Gemfile.lock
 - LICENSE