job-iteration 1.14.0 → 1.15.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a6cd8bb708ce42d3333f9631f344bb33b506cd2e3d0fc8a5d73882f2f52a811a
-  data.tar.gz: 30b9ece97b5e981a61fbef2110e60fdffcbd95dfdeec14def3652f9384f7e4d5
+  metadata.gz: dbb861f47602c89b8163b0f36f662c8b1f10c34528c0d06b0ce74918973e89db
+  data.tar.gz: b00adea778b0a722dc727fdd4a9954c60442df995ee273ec82ff9d85b9fa222f
 SHA512:
-  metadata.gz: cc9bd924dfbce3e582356a9b29edec62ca597c6b50cad39afb29970e9d1b1ce799678797b7441b6aa6732d25e8b5e8fd896f88b7dd118cae552f5acc50a31143
-  data.tar.gz: 532e91769910a4172d8f7773bfd59b3fa54372fed4b9fd22d6309afa0861a626a878dedb50972127658a76e90b651cf1837a7d87fc61deebc81655ee418698d9
+  metadata.gz: 1d085184d274fcc7315277c0e11e45093674e96a152d6bf49eff6644be23974df17f6d778e0abf0d1289ac2203af8ea59a9c7a7d9aeedd46de26b6ed16649d86
+  data.tar.gz: 50fea852189c3f13f4b0c84f465f532e4b27804d5d71238f044fa0f6beec67c8372c83561b0690e1605eb09c47a3f07cfad24a417a3dd612a4b57bfda840696d

data/CHANGELOG.md

@@ -16,6 +16,12 @@ nil
 
 nil
 
+## v1.15.0 (Jun 4, 2026)
+
+### Features
+
+- [715](https://github.com/Shopify/job-iteration/pull/715) - Add support for keyword arguments in `build_enumerator` and `each_iteration` while preserving positional params Hash compatibility for existing jobs. This compatibility path is transitional; migrate jobs away from positional params Hash signatures paired with `perform_later(keyword: value)`.
+
 ## v1.14.0 (May 14, 2026)
 
 ### Breaking Changes

data/README.md

@@ -101,6 +101,21 @@ class BatchesAsRelationJob < ApplicationJob
 end
 ```
 
+```ruby
+class ParallelIterationJob < ApplicationJob
+  include JobIteration::Iteration
+  
+  def build_enumerator(cursor:)
+    enumerator_builder.parallel_active_record_on_records(User.all, instances: 5, cursor: cursor)
+  end
+
+  # Runs in 5 separate jobs, with each job processing a subset of the users
+  def each_iteration(user)
+    user.notify_about_something
+  end
+end
+```
+
 ```ruby
 class ArrayJob < ApplicationJob
   include JobIteration::Iteration
@@ -178,20 +193,21 @@ Support for older platforms that have reached end of life may occasionally be dr
 * [Best practices](guides/best-practices.md)
 * [Writing custom enumerator](guides/custom-enumerator.md)
 * [Throttling](guides/throttling.md)
+* [Parallel iteration](guides/parallel-iteration.md)
 
 For more detailed documentation, see [rubydoc](https://www.rubydoc.info/gems/job-iteration).
 
 ## Requirements
 
-ActiveJob is the primary requirement for Iteration. While there's nothing that prevents it, Iteration is not yet compatible with [vanilla](https://github.com/mperham/sidekiq/wiki/Active-Job) Sidekiq API.
+Active Job is the primary requirement for Iteration. For iteration without Active Job in Sidekiq, see [Sidekiq Iteration](https://github.com/sidekiq/sidekiq/wiki/Iteration).
 
 ### API
 
-Iteration job must respond to `build_enumerator` and `each_iteration` methods. `build_enumerator` must return [Enumerator](http://ruby-doc.org/core-2.5.1/Enumerator.html) object that respects the `cursor` value.
+Iteration job must respond to `build_enumerator` and `each_iteration` methods. `build_enumerator` must return an [Enumerator](http://ruby-doc.org/core-2.5.1/Enumerator.html) object that respects the `cursor` value.
 
 ### Sidekiq adapter
 
-Unless you are running on Heroku, we recommend you to tune Sidekiq's [timeout](https://github.com/mperham/sidekiq/wiki/Deployment#overview) option from the default 8 seconds to 25-30 seconds, to allow the last `each_iteration` to complete and gracefully shutdown.
+Running iterating jobs on Sidekiq should work with the default configuration. The most important setting is Sidekiq's [timeout](https://github.com/mperham/sidekiq/wiki/Deployment#overview) option, which defaults to 25 seconds. That allows the last `each_iteration` to complete and gracefully shutdown.
 
 ### Resque adapter
 
@@ -203,11 +219,11 @@ There a few configuration assumptions that are required for Iteration to work wi
 
 **What happens when my job is interrupted?** A checkpoint will be persisted to Redis after the current `each_iteration`, and the job will be re-enqueued. Once it's popped off the queue, the worker will work off from the next iteration.
 
-**What happens with retries?** An interruption of a job does not count as a retry. If an exception occurs, the job will retry or be discarded as normal using Active Job configuration for the job. If the job retries, it processes the iteration that originally failed and progress will continue from there on if successful.
+**What happens with retries?** An interruption of a job does not count as a retry. If an exception occurs, the job will retry or be discarded as normal using Active Job configuration for the job. If the job retries, it re-processes the iteration that originally failed and progress will continue from there on if successful.
 
 **What happens if my iteration takes a long time?** We recommend that a single `each_iteration` should take no longer than 30 seconds. In the future, this may raise an exception.
 
-**Why is it important that `each_iteration` takes less than 30 seconds?** When the job worker is scheduled for restart or shutdown, it gets a notice to finish remaining unit of work. To guarantee that no progress is lost we need to make sure that `each_iteration` completes within a reasonable amount of time.
+**Why is it important that `each_iteration` runs quickly?** When the job worker is scheduled for restart or shutdown, it gets a notice to finish remaining unit of work. To guarantee that no progress is lost we need to make sure that `each_iteration` completes within a reasonable amount of time. The exact timeout depends on your queue adapter configuration.
 
 **Why do I use have to use this ugly helper in `build_enumerator`? Why can't you automatically infer it?** This is how the first version of the API worked. We checked the type of object returned by `build_enumerable`, and whether it was ActiveRecord Relation or an Array, we used the matching adapter. This caused opaque type branching in Iteration internals and it didn’t allow developers to craft their own Enumerators and control the cursor value. We made a decision to _always_ return Enumerator instance from `build_enumerator`. Now we provide explicit helpers to convert ActiveRecord Relation or an Array to Enumerator, and for more complex iteration flows developers can build their own `Enumerator` objects.
 

data/lib/job-iteration/iteration.rb

@@ -107,8 +107,8 @@ module JobIteration
       self.total_time = Float(job_data["total_time"] || 0.0)
     end
 
-    def perform(*params) # @private
-      interruptible_perform(*params)
+    def perform(...) # @private
+      interruptible_perform(...)
 
       nil
     end
@@ -128,12 +128,12 @@ module JobIteration
       JobIteration.enumerator_builder.new(self)
     end
 
-    def interruptible_perform(*arguments)
+    def interruptible_perform(*args, **kwargs)
       self.start_time = Time.now.utc
 
       enumerator = nil
       ActiveSupport::Notifications.instrument("build_enumerator.iteration", instrumentation_tags) do
-        enumerator = build_enumerator(*arguments, cursor: cursor_position)
+        enumerator = call_job_iteration_build_enumerator(args, kwargs)
       end
 
       unless enumerator
@@ -161,7 +161,7 @@ module JobIteration
       end
 
       completed = catch(:abort) do
-        iterate_with_enumerator(enumerator, arguments)
+        iterate_with_enumerator(enumerator, args, kwargs)
       end
 
       run_callbacks(:shutdown)
@@ -178,8 +178,7 @@ module JobIteration
       end
     end
 
-    def iterate_with_enumerator(enumerator, arguments)
-      arguments = arguments.dup.freeze
+    def iterate_with_enumerator(enumerator, args, kwargs)
       found_record = false
       @needs_reenqueue = false
 
@@ -191,7 +190,7 @@ module JobIteration
         ActiveSupport::Notifications.instrument("each_iteration.iteration", tags) do
           found_record = true
           run_callbacks(:iterate) do
-            each_iteration(object_from_enumerator, *arguments)
+            call_job_iteration_each_iteration(object_from_enumerator, args, kwargs)
           end
           self.cursor_position = cursor_from_enumerator
         end
@@ -215,6 +214,64 @@ module JobIteration
       adjust_total_time
     end
 
+    def call_job_iteration_build_enumerator(args, kwargs)
+      positional_args, keyword_args = normalize_job_iteration_arguments(args, kwargs, :build_enumerator)
+
+      if keyword_args&.key?(:cursor)
+        raise ArgumentError, "The keyword argument `cursor` is reserved for the job iteration framework. " \
+          "Please remove `cursor` from the arguments passed to the job or rename it"
+      end
+
+      # `keyword_args || {}` because splatting `nil` raises on Ruby < 3.4; it only
+      # became a no-op in 3.4 (https://bugs.ruby-lang.org/issues/20064).
+      build_enumerator(*positional_args, **(keyword_args || {}), cursor: cursor_position)
+    end
+
+    def call_job_iteration_each_iteration(object_from_enumerator, args, kwargs)
+      positional_args, keyword_args = normalize_job_iteration_arguments(args, kwargs, :each_iteration)
+      # `keyword_args || {}` because splatting `nil` raises on Ruby < 3.4; it only
+      # became a no-op in 3.4 (https://bugs.ruby-lang.org/issues/20064).
+      each_iteration(object_from_enumerator, *positional_args, **(keyword_args || {}))
+    end
+
+    # Normalize Active Job kwargs for job-iteration dispatch. If the target method
+    # accepts job keyword parameters other than build_enumerator's reserved cursor:,
+    # keep kwargs separate so they can be splatted. Otherwise, append kwargs as a
+    # positional params Hash for transitional compatibility with existing jobs
+    # enqueued with keyword syntax.
+    #: (Array[top], Hash[Symbol, top], Symbol) -> [Array[top], Hash[Symbol, top]?]
+    def normalize_job_iteration_arguments(args, kwargs, method_name)
+      if kwargs.empty?
+        [args.dup.freeze, nil]
+      elsif job_has_keyword_parameters?(method_name)
+        [args.dup.freeze, kwargs.dup.freeze]
+      else
+        params_hash_args = args + [kwargs]
+        [params_hash_args.dup.freeze, nil]
+      end
+    end
+
+    #: (Symbol) -> bool
+    def job_has_keyword_parameters?(method_name)
+      @job_has_keyword_parameters ||= {}
+      return @job_has_keyword_parameters[method_name] if @job_has_keyword_parameters.key?(method_name)
+
+      @job_has_keyword_parameters[method_name] = method_parameters(method_name).any? do |type, name|
+        job_keyword_argument?(method_name, type, name)
+      end
+    end
+
+    def job_keyword_argument?(method_name, type, name)
+      # Match keyword parameters: double-splat (**kwargs), required (key:) or optional (key: default).
+      return false unless type == :keyrest || type == :keyreq || type == :key
+
+      # Ignore the `cursor:` argument, which is part of the `job-iteration`
+      # framework, and not a argument in the job's serialized argument list.
+      return false if method_name == :build_enumerator && name == :cursor
+
+      true
+    end
+
     def reenqueue_iteration_job
       ActiveSupport::Notifications.instrument(
         "interrupted.iteration",

data/lib/job-iteration/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module JobIteration
-  VERSION = "1.14.0"
+  VERSION = "1.15.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: job-iteration
 version: !ruby/object:Gem::Version
-  version: 1.14.0
+  version: 1.15.0
 platform: ruby
 authors:
 - Shopify
@@ -78,7 +78,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 4.0.11
+rubygems_version: 4.0.12
 specification_version: 4
 summary: Makes your background jobs interruptible and resumable.
 test_files: []