sidekiq 6.2.2 → 6.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1b0da541124d097e05936cc860a93009ba170d714cbd8dbc760c1245818e8ed7
-  data.tar.gz: 63995ccbb57fa16e4954cda4ab9610506b0a8ff35084492b520fc18608face35
+  metadata.gz: 9622b2851203b0c5a80695ab7801ca77e15dc63a641ae79132cda9a2fcbe0cc6
+  data.tar.gz: dd943a02d2cf910f51866d02254f3a7845cb159735bd54d223dd7127a1fbd2fa
 SHA512:
-  metadata.gz: f3b864db74530840437c0af43475ed0e12a876d6d2cbbd808ecc72722adddbd8d5548c9b777d4e34af5deddf94871567717518f70d787ea7f6a089f2f5b2f4ed
-  data.tar.gz: 4569eed5baa10134e99a0b1404c213e3b35c08ec158e2fa8b006918298750919e8d34074aa10d3b4dfb621bb18cbe68be467d1e7822fb855012cff0f64dcdc7b
+  metadata.gz: 65bcd542866d8699ecf5958d81a15fd322cd1c51dfc1dbc6a8b15402b70862510c134e2b0ed9f9dbcdbb4dea3a59c1d12878ff926145e503079a59896f279ff3
+  data.tar.gz: 36143e85dc7fd4611f8a43fe39788dc590725ac63a827fbc174916da2d59fdadb3fb64fa0d819c4ef90694f6f292ea15f7fb58fb29f368c0a06c5b90f1b4bca7

data/Changes.md

@@ -1,6 +1,74 @@
 # Sidekiq Changes
 
-[Sidekiq Changes](https://github.com/mperham/sidekiq/blob/master/Changes.md) | [Sidekiq Pro Changes](https://github.com/mperham/sidekiq/blob/master/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/mperham/sidekiq/blob/master/Ent-Changes.md)
+[Sidekiq Changes](https://github.com/mperham/sidekiq/blob/main/Changes.md) | [Sidekiq Pro Changes](https://github.com/mperham/sidekiq/blob/main/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/mperham/sidekiq/blob/main/Ent-Changes.md)
+
+6.4.0
+---------
+
+- **SECURITY**: Validate input to avoid possible DoS in Web UI.
+- Add **strict argument checking** [#5071]
+  Sidekiq will now log a warning if JSON-unsafe arguments are passed to `perform_async`.
+  Add `Sidekiq.strict_args!(false)` to your initializer to disable this warning.
+  This warning will switch to an exception in Sidekiq 7.0.
+- Note that Delayed Extensions will be removed in Sidekiq 7.0 [#5076]
+- Add `perform_{inline,sync}` in Sidekiq::Job to run a job synchronously [#5061, hasan-ally]
+```ruby
+SomeJob.perform_async(args...)
+SomeJob.perform_sync(args...)
+SomeJob.perform_inline(args...)
+```
+  You can also dynamically redirect a job to run synchronously:
+```ruby
+SomeJob.set("sync": true).perform_async(args...) # will run via perform_inline
+```
+- Replace Sidekiq::Worker `app/workers` generator with Sidekiq::Job `app/sidekiq` generator [#5055]
+```
+bin/rails generate sidekiq:job ProcessOrderJob
+```
+- Fix job retries losing CurrentAttributes [#5090]
+- Tweak shutdown to give long-running threads time to cleanup [#5095]
+- Add keyword arguments support in extensions
+
+6.3.1
+---------
+
+- Fix keyword arguments error with CurrentAttributes on Ruby 3.0 [#5048]
+
+6.3.0
+---------
+
+- **BREAK**: The Web UI has been refactored to remove jQuery. Any UI extensions
+  which use jQuery will break.
+- **FEATURE**: Sidekiq.logger has been enhanced so any `Rails.logger`
+  output in jobs now shows up in the Sidekiq console. Remove any logger
+  hacks in your initializer and see if it Just Works™ now. [#5021]
+- **FEATURE**: Add `Sidekiq::Job` alias for `Sidekiq::Worker`, to better
+  reflect industry standard terminology. You can now do this:
+```ruby
+class MyJob
+  include Sidekiq::Job
+  sidekiq_options ...
+  def perform(args)
+  end
+end
+```
+- **FEATURE**: Support for serializing ActiveSupport::CurrentAttributes into each job. [#4982]
+```ruby
+# config/initializers/sidekiq.rb
+require "sidekiq/middleware/current_attributes"
+Sidekiq::CurrentAttributes.persist(Myapp::Current) # Your AS::CurrentAttributes singleton
+```
+- **FEATURE**: Add `Sidekiq::Worker.perform_bulk` for enqueuing jobs in bulk,
+  similar to `Sidekiq::Client.push_bulk` [#5042]
+```ruby
+MyJob.perform_bulk([[1], [2], [3]])
+```
+- Implement `queue_as`, `wait` and `wait_until` for ActiveJob compatibility [#5003]
+- Scheduler now uses Lua to reduce Redis load and network roundtrips [#5044]
+- Retry Redis operation if we get an `UNBLOCKED` Redis error [#4985]
+- Run existing signal traps, if any, before running Sidekiq's trap [#4991]
+- Fix fetch bug when using weighted queues which caused Sidekiq to stop
+  processing queues randomly [#5031]
 
 6.2.2
 ---------

data/LICENSE

@@ -4,6 +4,6 @@ Sidekiq is an Open Source project licensed under the terms of
 the LGPLv3 license.  Please see <http://www.gnu.org/licenses/lgpl-3.0.html>
 for license text.
 
-Sidekiq Pro has a commercial-friendly license allowing private forks
-and modifications of Sidekiq.  Please see https://sidekiq.org/products/pro.html for
-more detail.  You can find the commercial license terms in COMM-LICENSE.txt.
+Sidekiq Pro and Sidekiq Enterprise have a commercial-friendly license.
+You can find the commercial license in COMM-LICENSE.txt.
+Please see https://sidekiq.org for purchasing options.

data/README.md

@@ -43,10 +43,10 @@ Getting Started
 -----------------
 
 See the [Getting Started wiki page](https://github.com/mperham/sidekiq/wiki/Getting-Started) and follow the simple setup process.
-You can watch [this Youtube playlist](https://www.youtube.com/playlist?list=PLjeHh2LSCFrWGT5uVjUuFKAcrcj5kSai1) to learn all about
+You can watch [this YouTube playlist](https://www.youtube.com/playlist?list=PLjeHh2LSCFrWGT5uVjUuFKAcrcj5kSai1) to learn all about
 Sidekiq and see its features in action.  Here's the Web UI:
 
-![Web UI](https://github.com/mperham/sidekiq/raw/master/examples/web-ui.png)
+![Web UI](https://github.com/mperham/sidekiq/raw/main/examples/web-ui.png)
 
 
 Want to Upgrade?
@@ -84,7 +84,7 @@ See the [Sidekiq support page](https://sidekiq.org/support.html) for details.
 License
 -----------------
 
-Please see [LICENSE](https://github.com/mperham/sidekiq/blob/master/LICENSE) for licensing details.
+Please see [LICENSE](https://github.com/mperham/sidekiq/blob/main/LICENSE) for licensing details.
 
 
 Author

data/lib/generators/sidekiq/job_generator.rb

@@ -0,0 +1,57 @@
+require "rails/generators/named_base"
+
+module Sidekiq
+  module Generators # :nodoc:
+    class JobGenerator < ::Rails::Generators::NamedBase # :nodoc:
+      desc "This generator creates a Sidekiq Job in app/sidekiq and a corresponding test"
+
+      check_class_collision suffix: "Job"
+
+      def self.default_generator_root
+        File.dirname(__FILE__)
+      end
+
+      def create_job_file
+        template "job.rb.erb", File.join("app/sidekiq", class_path, "#{file_name}_job.rb")
+      end
+
+      def create_test_file
+        return unless test_framework
+
+        if test_framework == :rspec
+          create_job_spec
+        else
+          create_job_test
+        end
+      end
+
+      private
+
+      def create_job_spec
+        template_file = File.join(
+          "spec/sidekiq",
+          class_path,
+          "#{file_name}_job_spec.rb"
+        )
+        template "job_spec.rb.erb", template_file
+      end
+
+      def create_job_test
+        template_file = File.join(
+          "test/sidekiq",
+          class_path,
+          "#{file_name}_job_test.rb"
+        )
+        template "job_test.rb.erb", template_file
+      end
+
+      def file_name
+        @_file_name ||= super.sub(/_?job\z/i, "")
+      end
+
+      def test_framework
+        ::Rails.application.config.generators.options[:rails][:test_framework]
+      end
+    end
+  end
+end

data/lib/generators/sidekiq/templates/{worker.rb.erb → job.rb.erb}

@@ -1,6 +1,6 @@
 <% module_namespacing do -%>
-class <%= class_name %>Worker
-  include Sidekiq::Worker
+class <%= class_name %>Job
+  include Sidekiq::Job
 
   def perform(*args)
     # Do something

data/lib/generators/sidekiq/templates/{worker_spec.rb.erb → job_spec.rb.erb}

@@ -1,6 +1,6 @@
 require 'rails_helper'
 <% module_namespacing do -%>
-RSpec.describe <%= class_name %>Worker, type: :worker do
+RSpec.describe <%= class_name %>Job, type: :job do
   pending "add some examples to (or delete) #{__FILE__}"
 end
 <% end -%>

data/lib/generators/sidekiq/templates/{worker_test.rb.erb → job_test.rb.erb}

@@ -1,6 +1,6 @@
 require 'test_helper'
 <% module_namespacing do -%>
-class <%= class_name %>WorkerTest < Minitest::Test
+class <%= class_name %>JobTest < Minitest::Test
   def test_example
     skip "add some examples to (or delete) #{__FILE__}"
   end

data/lib/sidekiq/api.rb

@@ -113,6 +113,7 @@ module Sidekiq
 
       @stats[:workers_size] = workers_size
       @stats[:enqueued] = enqueued
+      @stats
     end
 
     def fetch_stats!
@@ -160,6 +161,8 @@ module Sidekiq
 
     class History
       def initialize(days_previous, start_date = nil)
+        # we only store five years of data in Redis
+        raise ArgumentError if days_previous < 1 || days_previous > (5 * 365)
         @days_previous = days_previous
         @start_date = start_date || Time.now.utc.to_date
       end
@@ -818,10 +821,10 @@ module Sidekiq
 
         hash = Sidekiq.load_json(info)
         yield Process.new(hash.merge("busy" => busy.to_i,
-                                     "beat" => at_s.to_f,
-                                     "quiet" => quiet,
-                                     "rss" => rss.to_i,
-                                     "rtt_us" => rtt.to_i))
+          "beat" => at_s.to_f,
+          "quiet" => quiet,
+          "rss" => rss.to_i,
+          "rtt_us" => rtt.to_i))
       end
     end
 

data/lib/sidekiq/cli.rb

@@ -46,7 +46,15 @@ module Sidekiq
       # USR1 and USR2 don't work on the JVM
       sigs << "USR2" if Sidekiq.pro? && !jruby?
       sigs.each do |sig|
-        trap sig do
+        old_handler = Signal.trap(sig) do
+          if old_handler.respond_to?(:call)
+            begin
+              old_handler.call
+            rescue Exception => exc
+              # signal handlers can't use Logger so puts only
+              puts ["Error in #{sig} handler", exc].inspect
+            end
+          end
           self_write.puts(sig)
         end
       rescue ArgumentError
@@ -372,7 +380,9 @@ module Sidekiq
     end
 
     def parse_config(path)
-      opts = YAML.load(ERB.new(File.read(path)).result) || {}
+      erb = ERB.new(File.read(path))
+      erb.filename = File.expand_path(path)
+      opts = YAML.load(erb.result) || {}
 
       if opts.respond_to? :deep_symbolize_keys!
         opts.deep_symbolize_keys!
@@ -396,7 +406,7 @@ module Sidekiq
       opts[:queues] ||= []
       opts[:strict] = true if opts[:strict].nil?
       raise ArgumentError, "queues: #{queue} cannot be defined twice" if opts[:queues].include?(queue)
-      [weight.to_i, 1].max.times { opts[:queues] << queue }
+      [weight.to_i, 1].max.times { opts[:queues] << queue.to_s }
       opts[:strict] = false if weight.to_i > 0
     end
 

data/lib/sidekiq/client.rb

@@ -2,9 +2,12 @@
 
 require "securerandom"
 require "sidekiq/middleware/chain"
+require "sidekiq/job_util"
 
 module Sidekiq
   class Client
+    include Sidekiq::JobUtil
+
     ##
     # Define client-side middleware:
     #
@@ -95,7 +98,7 @@ module Sidekiq
       return [] if args.empty? # no jobs to push
 
       at = items.delete("at")
-      raise ArgumentError, "Job 'at' must be a Numeric or an Array of Numeric timestamps" if at && (Array(at).empty? || !Array(at).all?(Numeric))
+      raise ArgumentError, "Job 'at' must be a Numeric or an Array of Numeric timestamps" if at && (Array(at).empty? || !Array(at).all? { |entry| entry.is_a?(Numeric) })
       raise ArgumentError, "Job 'at' Array must have same size as 'args' Array" if at.is_a?(Array) && at.size != args.size
 
       normed = normalize_item(items)
@@ -186,7 +189,7 @@ module Sidekiq
 
     def raw_push(payloads)
       @redis_pool.with do |conn|
-        conn.multi do
+        conn.pipelined do
           atomic_push(conn, payloads)
         end
       end
@@ -218,42 +221,5 @@ module Sidekiq
         item
       end
     end
-
-    def validate(item)
-      raise(ArgumentError, "Job must be a Hash with 'class' and 'args' keys: `#{item}`") unless item.is_a?(Hash) && item.key?("class") && item.key?("args")
-      raise(ArgumentError, "Job args must be an Array: `#{item}`") unless item["args"].is_a?(Array)
-      raise(ArgumentError, "Job class must be either a Class or String representation of the class name: `#{item}`") unless item["class"].is_a?(Class) || item["class"].is_a?(String)
-      raise(ArgumentError, "Job 'at' must be a Numeric timestamp: `#{item}`") if item.key?("at") && !item["at"].is_a?(Numeric)
-      raise(ArgumentError, "Job tags must be an Array: `#{item}`") if item["tags"] && !item["tags"].is_a?(Array)
-    end
-
-    def normalize_item(item)
-      validate(item)
-      # raise(ArgumentError, "Arguments must be native JSON types, see https://github.com/mperham/sidekiq/wiki/Best-Practices") unless JSON.load(JSON.dump(item['args'])) == item['args']
-
-      # merge in the default sidekiq_options for the item's class and/or wrapped element
-      # this allows ActiveJobs to control sidekiq_options too.
-      defaults = normalized_hash(item["class"])
-      defaults = defaults.merge(item["wrapped"].get_sidekiq_options) if item["wrapped"].respond_to?("get_sidekiq_options")
-      item = defaults.merge(item)
-
-      raise(ArgumentError, "Job must include a valid queue name") if item["queue"].nil? || item["queue"] == ""
-
-      item["class"] = item["class"].to_s
-      item["queue"] = item["queue"].to_s
-      item["jid"] ||= SecureRandom.hex(12)
-      item["created_at"] ||= Time.now.to_f
-
-      item
-    end
-
-    def normalized_hash(item_class)
-      if item_class.is_a?(Class)
-        raise(ArgumentError, "Message must include a Sidekiq::Worker class, not class name: #{item_class.ancestors.inspect}") unless item_class.respond_to?("get_sidekiq_options")
-        item_class.get_sidekiq_options
-      else
-        Sidekiq.default_worker_options
-      end
-    end
   end
 end

data/lib/sidekiq/delay.rb

@@ -3,6 +3,8 @@
 module Sidekiq
   module Extensions
     def self.enable_delay!
+      Sidekiq.logger.error "Sidekiq's Delayed Extensions will be removed in Sidekiq 7.0. #{caller(1..1).first}"
+
       if defined?(::ActiveSupport)
         require "sidekiq/extensions/active_record"
         require "sidekiq/extensions/action_mailer"

data/lib/sidekiq/extensions/action_mailer.rb

@@ -16,8 +16,8 @@ module Sidekiq
       include Sidekiq::Worker
 
       def perform(yml)
-        (target, method_name, args) = YAML.load(yml)
-        msg = target.public_send(method_name, *args)
+        (target, method_name, args, kwargs) = YAML.load(yml)
+        msg = kwargs.empty? ? target.public_send(method_name, *args) : target.public_send(method_name, *args, **kwargs)
         # The email method can return nil, which causes ActionMailer to return
         # an undeliverable empty message.
         if msg

data/lib/sidekiq/extensions/active_record.rb

@@ -18,8 +18,8 @@ module Sidekiq
       include Sidekiq::Worker
 
       def perform(yml)
-        (target, method_name, args) = YAML.load(yml)
-        target.__send__(method_name, *args)
+        (target, method_name, args, kwargs) = YAML.load(yml)
+        kwargs.empty? ? target.__send__(method_name, *args) : target.__send__(method_name, *args, **kwargs)
       end
     end
 

data/lib/sidekiq/extensions/class_methods.rb

@@ -16,8 +16,8 @@ module Sidekiq
       include Sidekiq::Worker
 
       def perform(yml)
-        (target, method_name, args) = YAML.load(yml)
-        target.__send__(method_name, *args)
+        (target, method_name, args, kwargs) = YAML.load(yml)
+        kwargs.empty? ? target.__send__(method_name, *args) : target.__send__(method_name, *args, **kwargs)
       end
     end
 

data/lib/sidekiq/extensions/generic_proxy.rb

@@ -13,13 +13,13 @@ module Sidekiq
         @opts = options
       end
 
-      def method_missing(name, *args)
+      def method_missing(name, *args, **kwargs)
         # Sidekiq has a limitation in that its message must be JSON.
         # JSON can't round trip real Ruby objects so we use YAML to
         # serialize the objects to a String.  The YAML will be converted
         # to JSON and then deserialized on the other side back into a
         # Ruby object.
-        obj = [@target, name, args]
+        obj = [@target, name, args, kwargs]
         marshalled = ::YAML.dump(obj)
         if marshalled.size > SIZE_LIMIT
           ::Sidekiq.logger.warn { "#{@target}.#{name} job argument is #{marshalled.bytesize} bytes, you should refactor it to reduce the size" }

data/lib/sidekiq/fetch.rb

@@ -79,9 +79,10 @@ module Sidekiq
       if @strictly_ordered_queues
         @queues
       else
-        queues = @queues.shuffle!.uniq
-        queues << TIMEOUT
-        queues
+        permute = @queues.shuffle
+        permute.uniq!
+        permute << TIMEOUT
+        permute
       end
     end
   end

data/lib/sidekiq/job.rb

@@ -1,8 +1,13 @@
 require "sidekiq/worker"
 
 module Sidekiq
-  # Sidekiq::Job is a new alias for Sidekiq::Worker, coming in 6.3.0.
-  # You can opt into this by requiring 'sidekiq/job' in your initializer
-  # and then using `include Sidekiq::Job` rather than `Sidekiq::Worker`.
+  # Sidekiq::Job is a new alias for Sidekiq::Worker as of Sidekiq 6.3.0.
+  # Use `include Sidekiq::Job` rather than `include Sidekiq::Worker`.
+  #
+  # The term "worker" is too generic and overly confusing, used in several
+  # different contexts meaning different things. Many people call a Sidekiq
+  # process a "worker". Some people call the thread that executes jobs a
+  # "worker". This change brings Sidekiq closer to ActiveJob where your job
+  # classes extend ApplicationJob.
   Job = Worker
 end

data/lib/sidekiq/job_retry.rb

@@ -34,9 +34,10 @@ module Sidekiq
   # The job will be retried this number of times before giving up. (If simply
   # 'true', Sidekiq retries 25 times)
   #
-  # We'll add a bit more data to the job to support retries:
+  # Relevant options for job retries:
   #
-  #  * 'queue' - the queue to use
+  #  * 'queue' - the queue for the initial job
+  #  * 'retry_queue' - if job retries should be pushed to a different (e.g. lower priority) queue
   #  * 'retry_count' - number of times we've retried so far.
   #  * 'error_message' - the message from the exception
   #  * 'error_class' - the exception class
@@ -52,11 +53,12 @@ module Sidekiq
   #
   #   Sidekiq.options[:max_retries] = 7
   #
-  # or limit the number of retries for a particular worker with:
+  # or limit the number of retries for a particular worker and send retries to
+  # a low priority queue with:
   #
   #    class MyWorker
   #      include Sidekiq::Worker
-  #      sidekiq_options :retry => 10
+  #      sidekiq_options retry: 10, retry_queue: 'low'
   #    end
   #
   class JobRetry

data/lib/sidekiq/job_util.rb

@@ -0,0 +1,65 @@
+require "securerandom"
+require "time"
+
+module Sidekiq
+  module JobUtil
+    # These functions encapsulate various job utilities.
+    # They must be simple and free from side effects.
+
+    def validate(item)
+      raise(ArgumentError, "Job must be a Hash with 'class' and 'args' keys: `#{item}`") unless item.is_a?(Hash) && item.key?("class") && item.key?("args")
+      raise(ArgumentError, "Job args must be an Array: `#{item}`") unless item["args"].is_a?(Array)
+      raise(ArgumentError, "Job class must be either a Class or String representation of the class name: `#{item}`") unless item["class"].is_a?(Class) || item["class"].is_a?(String)
+      raise(ArgumentError, "Job 'at' must be a Numeric timestamp: `#{item}`") if item.key?("at") && !item["at"].is_a?(Numeric)
+      raise(ArgumentError, "Job tags must be an Array: `#{item}`") if item["tags"] && !item["tags"].is_a?(Array)
+
+      if Sidekiq.options[:on_complex_arguments] == :raise
+        msg = <<~EOM
+          Job arguments to #{item["class"]} must be native JSON types, see https://github.com/mperham/sidekiq/wiki/Best-Practices.
+          To disable this error, remove `Sidekiq.strict_args!` from your initializer.
+        EOM
+        raise(ArgumentError, msg) unless json_safe?(item)
+      elsif Sidekiq.options[:on_complex_arguments] == :warn
+        Sidekiq.logger.warn <<~EOM unless json_safe?(item)
+          Job arguments to #{item["class"]} do not serialize to JSON safely. This will raise an error in
+          Sidekiq 7.0. See https://github.com/mperham/sidekiq/wiki/Best-Practices or raise an error today
+          by calling `Sidekiq.strict_args!` during Sidekiq initialization.
+        EOM
+      end
+    end
+
+    def normalize_item(item)
+      validate(item)
+
+      # merge in the default sidekiq_options for the item's class and/or wrapped element
+      # this allows ActiveJobs to control sidekiq_options too.
+      defaults = normalized_hash(item["class"])
+      defaults = defaults.merge(item["wrapped"].get_sidekiq_options) if item["wrapped"].respond_to?(:get_sidekiq_options)
+      item = defaults.merge(item)
+
+      raise(ArgumentError, "Job must include a valid queue name") if item["queue"].nil? || item["queue"] == ""
+
+      item["class"] = item["class"].to_s
+      item["queue"] = item["queue"].to_s
+      item["jid"] ||= SecureRandom.hex(12)
+      item["created_at"] ||= Time.now.to_f
+
+      item
+    end
+
+    def normalized_hash(item_class)
+      if item_class.is_a?(Class)
+        raise(ArgumentError, "Message must include a Sidekiq::Worker class, not class name: #{item_class.ancestors.inspect}") unless item_class.respond_to?(:get_sidekiq_options)
+        item_class.get_sidekiq_options
+      else
+        Sidekiq.default_worker_options
+      end
+    end
+
+    private
+
+    def json_safe?(item)
+      JSON.parse(JSON.dump(item["args"])) == item["args"]
+    end
+  end
+end

data/lib/sidekiq/launcher.rb

@@ -69,10 +69,12 @@ module Sidekiq
 
     private unless $TESTING
 
+    BEAT_PAUSE = 5
+
     def start_heartbeat
       loop do
         heartbeat
-        sleep 5
+        sleep BEAT_PAUSE
       end
       Sidekiq.logger.info("Heartbeat stopping...")
     end
@@ -211,6 +213,8 @@ module Sidekiq
           Your Redis network connection is performing extremely poorly.
           Last RTT readings were #{RTT_READINGS.buffer.inspect}, ideally these should be < 1000.
           Ensure Redis is running in the same AZ or datacenter as Sidekiq.
+          If these values are close to 100,000, that means your Sidekiq process may be
+          CPU overloaded; see https://github.com/mperham/sidekiq/discussions/5039
         EOM
         RTT_READINGS.reset
       end

data/lib/sidekiq/manager.rb

@@ -55,9 +55,6 @@ module Sidekiq
       fire_event(:quiet, reverse: true)
     end
 
-    # hack for quicker development / testing environment #2774
-    PAUSE_TIME = $stdout.tty? ? 0.1 : 0.5
-
     def stop(deadline)
       quiet
       fire_event(:shutdown, reverse: true)
@@ -69,12 +66,7 @@ module Sidekiq
       return if @workers.empty?
 
       logger.info { "Pausing to allow workers to finish..." }
-      remaining = deadline - ::Process.clock_gettime(::Process::CLOCK_MONOTONIC)
-      while remaining > PAUSE_TIME
-        return if @workers.empty?
-        sleep PAUSE_TIME
-        remaining = deadline - ::Process.clock_gettime(::Process::CLOCK_MONOTONIC)
-      end
+      wait_for(deadline) { @workers.empty? }
       return if @workers.empty?
 
       hard_shutdown
@@ -130,6 +122,12 @@ module Sidekiq
       cleanup.each do |processor|
         processor.kill
       end
+
+      # when this method returns, we immediately call `exit` which may not give
+      # the remaining threads time to run `ensure` blocks, etc. We pause here up
+      # to 3 seconds to give threads a minimal amount of time to run `ensure` blocks.
+      deadline = ::Process.clock_gettime(::Process::CLOCK_MONOTONIC) + 3
+      wait_for(deadline) { @workers.empty? }
     end
   end
 end

data/lib/sidekiq/middleware/current_attributes.rb

@@ -0,0 +1,57 @@
+require "active_support/current_attributes"
+
+module Sidekiq
+  ##
+  # Automatically save and load any current attributes in the execution context
+  # so context attributes "flow" from Rails actions into any associated jobs.
+  # This can be useful for multi-tenancy, i18n locale, timezone, any implicit
+  # per-request attribute. See +ActiveSupport::CurrentAttributes+.
+  #
+  # @example
+  #
+  #   # in your initializer
+  #   require "sidekiq/middleware/current_attributes"
+  #   Sidekiq::CurrentAttributes.persist(Myapp::Current)
+  #
+  module CurrentAttributes
+    class Save
+      def initialize(cattr)
+        @klass = cattr
+      end
+
+      def call(_, job, _, _)
+        attrs = @klass.attributes
+        if job.has_key?("cattr")
+          job["cattr"].merge!(attrs)
+        else
+          job["cattr"] = attrs
+        end
+        yield
+      end
+    end
+
+    class Load
+      def initialize(cattr)
+        @klass = cattr
+      end
+
+      def call(_, job, _, &block)
+        if job.has_key?("cattr")
+          @klass.set(job["cattr"], &block)
+        else
+          yield
+        end
+      end
+    end
+
+    def self.persist(klass)
+      Sidekiq.configure_client do |config|
+        config.client_middleware.add Save, klass
+      end
+      Sidekiq.configure_server do |config|
+        config.client_middleware.add Save, klass
+        config.server_middleware.add Load, klass
+      end
+    end
+  end
+end