sidekiq 5.0.5 → 5.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 96202101069aaee71e462e2f3b91f5d4c58e952f
-  data.tar.gz: ad75e0b5dc019bb8d1058b124535190f00b06f60
+  metadata.gz: 4ec2845461ba55bb7f8424359c49ee8e339ae4be
+  data.tar.gz: b578722c371f8dc196294da1058279a8794e960e
 SHA512:
-  metadata.gz: 4de121e1a00611b97f5c8838c5beb1d5a4fa5e6fb37e7cd7b2ac2074239d8a48d1d9526d4a8c0a94fcfcebe7ee3507209735569c78309f18d29363fbab24b992
-  data.tar.gz: 239a247dd69bab58148229f64d7b8ff12a658bb696c08c9f00b9ee9170c5d40ea052891eef5d89742633282d99095b4eb6004df298aa2f06b5f114fbbcfb10de
+  metadata.gz: 8c900629c592c5eb88b63863e4202667575501cf472257bbf58c211ef743ee887a288bc14744d5da50ea0ff9163285427d93a8946bfb48a26896cd55b5c7b4b3
+  data.tar.gz: 639a844489ccf182d8a3444594c1e5528978f536b3a7c0954d5a8d47d203002285e28ee84e0f47829d1ed7545475400ff3928b6304b65aee7e7595ccc3dd0765

data/.github/issue_template.md

@@ -4,6 +4,8 @@ Sidekiq / Pro / Enterprise version(s):
 Please include your initializer and any error message with the full backtrace.
 
 Are you using an old version?
-Have you checked the changelog to see if your issue has been fixed in a later version?
+Have you checked the changelogs to see if your issue has been fixed in a later version?
 
 https://github.com/mperham/sidekiq/blob/master/Changes.md
+https://github.com/mperham/sidekiq/blob/master/Pro-Changes.md
+https://github.com/mperham/sidekiq/blob/master/Ent-Changes.md

data/.travis.yml

@@ -7,7 +7,8 @@ before_install:
   - gem install bundler
   - gem update bundler
 rvm:
-  - 2.2.4
-  - 2.3.0
-  - 2.4.0
-  - jruby-9.1.6.0
+  - 2.2.2
+  - 2.3.5
+  - 2.4.2
+  - 2.5.0
+  - jruby-9.1.14.0

data/Changes.md

@@ -2,12 +2,27 @@
 
 [Sidekiq Changes](https://github.com/mperham/sidekiq/blob/master/Changes.md) | [Sidekiq Pro Changes](https://github.com/mperham/sidekiq/blob/master/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/mperham/sidekiq/blob/master/Ent-Changes.md)
 
-HEAD
+5.1.0
+-----------
+
+- **NEW** Global death handlers - called when your job exhausts all
+  retries and dies.  Now you can take action when a job fails permanently. [#3721]
+- **NEW** Enable ActiveRecord query cache within jobs by default [#3718, sobrinho]
+  This will prevent duplicate SELECTS; cache is cleared upon any UPDATE/INSERT/DELETE.
+  See the issue for how to bypass the cache or disable it completely.
+- Scheduler timing is now more accurate, 15 -> 5 seconds [#3734]
+- Exceptions during the :startup event will now kill the process [#3717]
+- Make `Sidekiq::Client.via` reentrant [#3715]
+- Fix use of Sidekiq logger outside of the server process [#3714]
+- Tweak `constantize` to better match Rails class lookup. [#3701, caffeinated-tech]
+
+5.0.5
 -----------
 
 - Update gemspec to allow newer versions of the Redis gem [#3617]
 - Refactor Worker.set so it can be memoized [#3602]
 - Fix display of Redis URL in web footer, broken in 5.0.3 [#3560]
+- Update `Sidekiq::Job#display_args` to avoid mutation [#3621]
 
 5.0.4
 -----------

data/Ent-Changes.md

@@ -4,6 +4,13 @@
 
 Please see [http://sidekiq.org/](http://sidekiq.org/) for more details and how to buy.
 
+HEAD
+-------------
+
+- **NEW FEATURE** [Rolling restarts](https://github.com/mperham/sidekiq/wiki/Ent-Rolling-Restarts) - great for long running jobs!
+- Adjust middleware so failed unique jobs don't affect batches [#3662]
+- Add new unlimited rate limiter, useful for testing [#3743]
+
 1.6.1
 -------------
 

data/Gemfile

@@ -2,31 +2,7 @@ source 'https://rubygems.org'
 gemspec
 
 gem 'rails', '>= 5.0.1'
-gem "hiredis"
+#gem "hiredis"
 gem 'simplecov'
 gem 'minitest'
-#gem 'minitest-utils'
-gem 'toxiproxy'
-
-# For Redis 4.0 support. Unreleased 9cb81bf.
-gem 'redis-namespace', git: 'https://github.com/resque/redis-namespace'
-
-platforms :rbx do
-  gem 'rubysl', '~> 2.0'         # if using anything in the ruby standard library
-  gem 'psych'                    # if using yaml
-  gem 'rubinius-developer_tools' # if using any of coverage, debugger, profiler
-end
-
-platforms :ruby do
-  gem 'sqlite3'
-end
-
-platforms :mri do
-  gem 'pry-byebug'
-  gem 'ruby-prof'
-end
-
-#platforms :jruby do
-  #gem 'jruby-openssl'
-  #gem 'activerecord-jdbcsqlite3-adapter'
-#end
+#gem 'toxiproxy'

data/Pro-4.0-Upgrade.md

@@ -0,0 +1,32 @@
+# Welcome to Sidekiq Pro 4.0!
+
+Sidekiq Pro 4.0 is designed to work with Sidekiq 5.0.
+
+## What's New
+
+* Batches now "die" if any of their jobs die.  You can enumerate the set
+  of dead batches and their associated dead jobs.  The success callback
+  for a dead batch will never fire unless these jobs are fixed.
+```ruby
+Sidekiq::Batch::DeadSet.new.each do |status|
+  status.dead? # => true
+  status.dead_jobs # => [...]
+end
+```
+* The older `reliable_fetch` and `timed_fetch` algorithms have been
+  removed.  Only super\_fetch is available in 4.0.
+* The statsd middleware has been tweaked to remove support for legacy,
+  pre-3.6.0 configuration.
+
+## Upgrade
+
+* Upgrade to the latest Sidekiq Pro 3.x.
+```ruby
+gem 'sidekiq-pro', '< 4'
+```
+* Fix any deprecation warnings you see.
+* Upgrade to 4.x.
+```ruby
+gem 'sidekiq-pro', '< 5'
+```
+

data/Pro-Changes.md

@@ -4,6 +4,69 @@
 
 Please see [http://sidekiq.org/](http://sidekiq.org/) for more details and how to buy.
 
+HEAD
+---------
+
+- See the [Sidekiq Pro 4.0](Pro-4.0-Upgrade.md) release notes.
+
+
+3.7.1
+---------
+
+- Deprecate timed\_fetch.  Switch to super\_fetch:
+```ruby
+config.super_fetch!
+```
+
+
+3.7.0
+---------
+
+- Refactor batch job success/failure to gracefully handle several edge
+  cases with regard to Sidekiq::Shutdown.  This should greatly reduce
+  the chances of seeing the long-standing "negative pending count" problem. [#3710]
+
+
+3.6.1
+---------
+
+- Add support for Datadog::Statsd, it is the recommended Statsd client.  [#3699]
+```ruby
+Sidekiq::Pro.dogstatsd = ->{ Datadog::Statsd.new("metrics.example.com", 8125) }
+```
+- Size the statsd connection pool based on Sidekiq's concurrency [#3700]
+
+
+3.6.0
+---------
+
+This release overhauls the Statsd metrics support and adds more
+metrics for tracking Pro feature usage.  In your initializer:
+```ruby
+Sidekiq::Pro.statsd = ->{ ::Statsd.new("127.0.0.1", 8125) }
+```
+Sidekiq Pro will emit more metrics to Statsd:
+```
+jobs.expired - when a job is expired
+jobs.recovered.push - when a job is recovered by reliable_push after network outage
+jobs.recovered.fetch - when a job is recovered by super_fetch after process crash
+batch.created - when a batch is created
+batch.complete - when a batch is completed
+batch.success - when a batch is successful
+```
+Sidekiq Pro's existing Statsd middleware has been rewritten to leverage the new API.
+Everything should be backwards compatible with one deprecation notice.
+
+
+3.5.4
+---------
+
+- Fix case in SuperFetch where Redis downtime can lead to processor thread death [#3684]
+- Fix case where TimedFetch might not recover some pending jobs
+- Fix edge case in Batch::Status#poll leading to premature completion [#3640]
+- Adjust scan API to check 100 elements at a time, to minimize network round trips
+  when scanning large sets.
+
 3.5.3
 ---------
 

data/README.md

@@ -77,16 +77,18 @@ Problems?
 If you have a problem, please review the [FAQ](https://github.com/mperham/sidekiq/wiki/FAQ) and [Troubleshooting](https://github.com/mperham/sidekiq/wiki/Problems-and-Troubleshooting) wiki pages.
 Searching the [issues](https://github.com/mperham/sidekiq/issues) for your problem is also a good idea.
 
+Sidekiq Pro and Sidekiq Enterprise customers get private email support.  You can purchase at http://sidekiq.org; email support@contribsys.com for help.
+
 Useful resources:
 
 * Product documentation is in the [wiki](https://github.com/mperham/sidekiq/wiki).
 * Release announcements are made to the [@sidekiq](https://twitter.com/sidekiq) Twitter account.
 * The [Sidekiq tag](https://stackoverflow.com/questions/tagged/sidekiq) on Stack Overflow has lots of useful Q & A.
 
-**No support via Twitter, 140 characters is not enough.**
+**No support via Twitter**
 
 Every Friday morning is Sidekiq happy hour: I video chat and answer questions.
-See the [Sidekiq support page](http://sidekiq.org/support) for details.
+See the [Sidekiq support page](http://sidekiq.org/support.html) for details.
 
 Thanks
 -----------------

data/Rakefile

@@ -6,7 +6,3 @@ Rake::TestTask.new(:test) do |test|
 end
 
 task :default => :test
-
-task :appraise do
-  exec("cd myapp && rake appraise")
-end

data/bin/sidekiqload

@@ -72,7 +72,7 @@ def handle_signal(launcher, sig)
     launcher.quiet
   when 'TTIN'
     Thread.list.each do |thread|
-      Sidekiq.logger.warn "Thread TID-#{thread.object_id.to_s(36)} #{thread['label']}"
+      Sidekiq.logger.warn "Thread TID-#{(thread.object_id ^ ::Process.pid).to_s(36)} #{thread['label']}"
       if thread.backtrace
         Sidekiq.logger.warn thread.backtrace.join("\n")
       else

data/lib/sidekiq.rb

@@ -23,8 +23,9 @@ module Sidekiq
     environment: nil,
     timeout: 8,
     poll_interval_average: nil,
-    average_scheduled_poll_interval: 15,
+    average_scheduled_poll_interval: 5,
     error_handlers: [],
+    death_handlers: [],
     lifecycle_events: {
       startup: [],
       quiet: [],
@@ -156,16 +157,23 @@ module Sidekiq
     defined?(@default_worker_options) ? @default_worker_options : DEFAULT_WORKER_OPTIONS
   end
 
+  def self.default_retries_exhausted=(prok)
+    logger.info { "default_retries_exhausted is deprecated, please use `config.death_handlers << -> {|job, ex| }`" }
+    return nil unless prok
+    death_handlers << prok
+  end
+
+  ##
+  # Death handlers are called when all retries for a job have been exhausted and
+  # the job dies.  It's the notification to your application
+  # that this job will not succeed without manual intervention.
+  #
   # Sidekiq.configure_server do |config|
-  #   config.default_retries_exhausted = -> (job, ex) do
+  #   config.death_handlers << ->(job, ex) do
   #   end
   # end
-  def self.default_retries_exhausted=(prok)
-    @default_retries_exhausted = prok
-  end
-  @default_retries_exhausted = ->(job, ex) { }
-  def self.default_retries_exhausted
-    @default_retries_exhausted
+  def self.death_handlers
+    options[:death_handlers]
   end
 
   def self.load_json(string)

data/lib/sidekiq/api.rb

@@ -548,7 +548,7 @@ module Sidekiq
         end
         break if elements.empty?
         page -= 1
-        elements.each do |element, score|
+        elements.reverse.each do |element, score|
           yield SortedEntry.new(self, score, element)
         end
         offset_size = initial_size - @_size
@@ -656,7 +656,7 @@ module Sidekiq
       super 'dead'
     end
 
-    def kill(message)
+    def kill(message, opts={})
       now = Time.now.to_f
       Sidekiq.redis do |conn|
         conn.multi do
@@ -665,6 +665,16 @@ module Sidekiq
           conn.zremrangebyrank(name, 0, - self.class.max_jobs)
         end
       end
+
+      if opts[:notify_failure] != false
+        job = Sidekiq.load_json(message)
+        r = RuntimeError.new("Job killed by API")
+        r.set_backtrace(caller)
+        Sidekiq.death_handlers.each do |handle|
+          handle.call(job, r)
+        end
+      end
+      true
     end
 
     def retry_all

data/lib/sidekiq/cli.rb

@@ -89,7 +89,7 @@ module Sidekiq
 
       # Before this point, the process is initializing with just the main thread.
       # Starting here the process will now have multiple threads running.
-      fire_event(:startup)
+      fire_event(:startup, reverse: false, reraise: true)
 
       logger.debug { "Client Middleware: #{Sidekiq.client_middleware.map(&:klass).join(', ')}" }
       logger.debug { "Server Middleware: #{Sidekiq.server_middleware.map(&:klass).join(', ')}" }
@@ -135,37 +135,45 @@ module Sidekiq
 }
     end
 
-    def handle_signal(sig)
-      Sidekiq.logger.debug "Got #{sig} signal"
-      case sig
-      when 'INT'
-        # Handle Ctrl-C in JRuby like MRI
-        # http://jira.codehaus.org/browse/JRUBY-4637
-        raise Interrupt
-      when 'TERM'
-        # Heroku sends TERM and then waits 10 seconds for process to exit.
-        raise Interrupt
-      when 'USR1'
+    SIGNAL_HANDLERS = {
+      # Ctrl-C in terminal
+      'INT' => ->(cli) { raise Interrupt },
+      # TERM is the signal that Sidekiq must exit.
+      # Heroku sends TERM and then waits 30 seconds for process to exit.
+      'TERM' => ->(cli) { raise Interrupt },
+      'USR1' => ->(cli) {
         Sidekiq.logger.info "Received USR1, no longer accepting new work"
-        launcher.quiet
-      when 'TSTP'
-        # USR1 is not available on JVM, allow TSTP as an alternate signal
+        cli.launcher.quiet
+      },
+      'TSTP' => ->(cli) {
         Sidekiq.logger.info "Received TSTP, no longer accepting new work"
-        launcher.quiet
-      when 'USR2'
+        cli.launcher.quiet
+      },
+      'USR2' => ->(cli) {
         if Sidekiq.options[:logfile]
           Sidekiq.logger.info "Received USR2, reopening log file"
           Sidekiq::Logging.reopen_logs
         end
-      when 'TTIN'
+      },
+      'TTIN' => ->(cli) {
         Thread.list.each do |thread|
-          Sidekiq.logger.warn "Thread TID-#{thread.object_id.to_s(36)} #{thread['sidekiq_label']}"
+          Sidekiq.logger.warn "Thread TID-#{(thread.object_id ^ ::Process.pid).to_s(36)} #{thread['sidekiq_label']}"
           if thread.backtrace
             Sidekiq.logger.warn thread.backtrace.join("\n")
           else
             Sidekiq.logger.warn "<no backtrace available>"
           end
         end
+      },
+    }
+
+    def handle_signal(sig)
+      Sidekiq.logger.debug "Got #{sig} signal"
+      handy = SIGNAL_HANDLERS[sig]
+      if handy
+        handy.call(self)
+      else
+        Sidekiq.logger.info { "No signal handler for #{sig}" }
       end
     end
 
@@ -214,6 +222,14 @@ module Sidekiq
       @environment = cli_env || ENV['RAILS_ENV'] || ENV['RACK_ENV'] || 'development'
     end
 
+    def symbolize_keys_deep!(hash)
+      hash.keys.each do |k|
+        symkey = k.respond_to?(:to_sym) ? k.to_sym : k
+        hash[symkey] = hash.delete k
+        symbolize_keys_deep! hash[symkey] if hash[symkey].kind_of? Hash
+      end
+    end
+
     alias_method :die, :exit
     alias_method :☠, :exit
 
@@ -282,7 +298,7 @@ module Sidekiq
       if !File.exist?(options[:require]) ||
          (File.directory?(options[:require]) && !File.exist?("#{options[:require]}/config/application.rb"))
         logger.info "=================================================================="
-        logger.info "  Please point sidekiq to a Rails 3/4 application or a Ruby file  "
+        logger.info "  Please point sidekiq to a Rails 4/5 application or a Ruby file  "
         logger.info "  to load your worker classes with -r [DIR|FILE]."
         logger.info "=================================================================="
         logger.info @parser
@@ -386,7 +402,14 @@ module Sidekiq
       opts = {}
       if File.exist?(cfile)
         opts = YAML.load(ERB.new(IO.read(cfile)).result) || opts
-        opts = opts.merge(opts.delete(environment) || {})
+
+        if opts.respond_to? :deep_symbolize_keys!
+          opts.deep_symbolize_keys!
+        else
+          symbolize_keys_deep!(opts)
+        end
+
+        opts = opts.merge(opts.delete(environment.to_sym) || {})
         parse_queues(opts, opts.delete(:queues) || [])
       else
         # allow a non-existent config file so Sidekiq

data/lib/sidekiq/client.rb

@@ -119,11 +119,10 @@ module Sidekiq
     def self.via(pool)
       raise ArgumentError, "No pool given" if pool.nil?
       current_sidekiq_pool = Thread.current[:sidekiq_via_pool]
-      raise RuntimeError, "Sidekiq::Client.via is not re-entrant" if current_sidekiq_pool && current_sidekiq_pool != pool
       Thread.current[:sidekiq_via_pool] = pool
       yield
     ensure
-      Thread.current[:sidekiq_via_pool] = nil
+      Thread.current[:sidekiq_via_pool] = current_sidekiq_pool
     end
 
     class << self

data/lib/sidekiq/exception_handler.rb

@@ -7,11 +7,10 @@ module Sidekiq
     class Logger
       def call(ex, ctxHash)
         Sidekiq.logger.warn(Sidekiq.dump_json(ctxHash)) if !ctxHash.empty?
-        Sidekiq.logger.warn "#{ex.class.name}: #{ex.message}"
-        Sidekiq.logger.warn ex.backtrace.join("\n") unless ex.backtrace.nil?
+        Sidekiq.logger.warn("#{ex.class.name}: #{ex.message}")
+        Sidekiq.logger.warn(ex.backtrace.join("\n")) unless ex.backtrace.nil?
       end
 
-      # Set up default handler which just logs the error
       Sidekiq.error_handlers << Sidekiq::ExceptionHandler::Logger.new
     end
 
@@ -26,6 +25,5 @@ module Sidekiq
         end
       end
     end
-
   end
 end

data/lib/sidekiq/job_retry.rb

@@ -172,10 +172,18 @@ module Sidekiq
     def retries_exhausted(worker, msg, exception)
       logger.debug { "Retries exhausted for job" }
       begin
-        block = worker && worker.sidekiq_retries_exhausted_block || Sidekiq.default_retries_exhausted
+        block = worker && worker.sidekiq_retries_exhausted_block
         block.call(msg, exception) if block
       rescue => e
-        handle_exception(e, { context: "Error calling retries_exhausted for #{msg['class']}", job: msg })
+        handle_exception(e, { context: "Error calling retries_exhausted", job: msg })
+      end
+
+      Sidekiq.death_handlers.each do |handler|
+        begin
+          handler.call(msg, exception)
+        rescue => e
+          handle_exception(e, { context: "Error calling death handler", job: msg })
+        end
       end
 
       send_to_morgue(msg) unless msg['dead'] == false
@@ -184,7 +192,7 @@ module Sidekiq
     def send_to_morgue(msg)
       Sidekiq.logger.info { "Adding dead #{msg['class']} job #{msg['jid']}" }
       payload = Sidekiq.dump_json(msg)
-      DeadSet.new.kill(payload)
+      DeadSet.new.kill(payload, notify_failure: false)
     end
 
     def retry_attempts_from(msg_retry, default)

data/lib/sidekiq/logging.rb

@@ -11,7 +11,7 @@ module Sidekiq
 
       # Provide a call() method that returns the formatted message.
       def call(severity, time, program_name, message)
-        "#{time.utc.iso8601(3)} #{::Process.pid} TID-#{Thread.current.object_id.to_s(36)}#{context} #{severity}: #{message}\n"
+        "#{time.utc.iso8601(3)} #{::Process.pid} TID-#{Sidekiq::Logging.tid}#{context} #{severity}: #{message}\n"
       end
 
       def context
@@ -22,10 +22,14 @@ module Sidekiq
 
     class WithoutTimestamp < Pretty
       def call(severity, time, program_name, message)
-        "#{::Process.pid} TID-#{Thread.current.object_id.to_s(36)}#{context} #{severity}: #{message}\n"
+        "#{::Process.pid} TID-#{Sidekiq::Logging.tid}#{context} #{severity}: #{message}\n"
       end
     end
 
+    def self.tid
+      Thread.current['sidekiq_tid'] ||= (Thread.current.object_id ^ ::Process.pid).to_s(36)
+    end
+
     def self.job_hash_context(job_hash)
       # If we're using a wrapper class, like ActiveJob, use the "wrapped"
       # attribute to expose the underlying thing.

data/lib/sidekiq/manager.rb

@@ -54,7 +54,7 @@ module Sidekiq
 
       logger.info { "Terminating quiet workers" }
       @workers.each { |x| x.terminate }
-      fire_event(:quiet, true)
+      fire_event(:quiet, reverse: true)
     end
 
     # hack for quicker development / testing environment #2774
@@ -62,7 +62,7 @@ module Sidekiq
 
     def stop(deadline)
       quiet
-      fire_event(:shutdown, true)
+      fire_event(:shutdown, reverse: true)
 
       # some of the shutdown events can be async,
       # we don't have any way to know when they're done but

data/lib/sidekiq/middleware/server/active_record_cache.rb

@@ -0,0 +1,11 @@
+module Sidekiq
+  module Middleware
+    module Server
+      class ActiveRecordCache
+        def call(*args, &block)
+          ::ActiveRecord::Base.cache(&block)
+        end
+      end
+    end
+  end
+end

data/lib/sidekiq/processor.rb

@@ -111,9 +111,7 @@ module Sidekiq
       if !@down
         @down = Time.now
         logger.error("Error fetching job: #{ex}")
-        ex.backtrace.each do |bt|
-          logger.error(bt)
-        end
+        handle_exception(ex)
       end
       sleep(1)
       nil
@@ -159,7 +157,8 @@ module Sidekiq
           job_hash = Sidekiq.load_json(jobstr)
         rescue => ex
           handle_exception(ex, { :context => "Invalid JSON for job", :jobstr => jobstr })
-          DeadSet.new.kill(jobstr)
+          # we can't notify because the job isn't a valid hash payload.
+          DeadSet.new.kill(jobstr, notify_failure: false)
           ack = true
           raise
         end
@@ -188,16 +187,12 @@ module Sidekiq
       worker.perform(*cloned_args)
     end
 
-    def thread_identity
-      @str ||= Thread.current.object_id.to_s(36)
-    end
-
     WORKER_STATE = Concurrent::Map.new
     PROCESSED = Concurrent::AtomicFixnum.new
     FAILURE = Concurrent::AtomicFixnum.new
 
     def stats(job_hash, queue)
-      tid = thread_identity
+      tid = Sidekiq::Logging.tid
       WORKER_STATE[tid] = {:queue => queue, :payload => job_hash, :run_at => Time.now.to_i }
 
       begin
@@ -223,7 +218,9 @@ module Sidekiq
       names.shift if names.empty? || names.first.empty?
 
       names.inject(Object) do |constant, name|
-        constant.const_defined?(name) ? constant.const_get(name) : constant.const_missing(name)
+        # the false flag limits search for name to under the constant namespace
+        #   which mimics Rails' behaviour
+        constant.const_defined?(name, false) ? constant.const_get(name, false) : constant.const_missing(name)
       end
     end
 

data/lib/sidekiq/rails.rb

@@ -9,10 +9,15 @@ module Sidekiq
     # class block. Definitely before config/environments/*.rb and
     # config/initializers/*.rb.
     config.before_configuration do
-      if ::Rails::VERSION::MAJOR < 5 && defined?(::ActiveRecord)
+      if defined?(::ActiveRecord)
         Sidekiq.server_middleware do |chain|
-          require 'sidekiq/middleware/server/active_record'
-          chain.add Sidekiq::Middleware::Server::ActiveRecord
+          if ::Rails::VERSION::MAJOR < 5
+            require 'sidekiq/middleware/server/active_record'
+            chain.add Sidekiq::Middleware::Server::ActiveRecord
+          end
+
+          require 'sidekiq/middleware/server/active_record_cache'
+          chain.add Sidekiq::Middleware::Server::ActiveRecordCache
         end
       end
     end
@@ -49,3 +54,9 @@ module Sidekiq
     end
   end if defined?(::Rails)
 end
+
+if defined?(::Rails) && ::Rails::VERSION::MAJOR < 4
+  $stderr.puts("**************************************************")
+  $stderr.puts("⛔️ WARNING: Sidekiq server is no longer supported by Rails 3.2 - please ensure your server/workers are updated")
+  $stderr.puts("**************************************************")
+end

data/lib/sidekiq/redis_connection.rb

@@ -38,7 +38,7 @@ module Sidekiq
       #   - enterprise's leader election
       #   - enterprise's cron support
       def verify_sizing(size, concurrency)
-        raise ArgumentError, "Your Redis connection pool is too small for Sidekiq to work. Your pool has #{size} connections but really needs to have at least #{concurrency + 2}" if size <= concurrency
+        raise ArgumentError, "Your Redis connection pool is too small for Sidekiq to work. Your pool has #{size} connections but must have at least #{concurrency + 2}" if size <= concurrency
       end
 
       def build_client(options)

data/lib/sidekiq/scheduled.rb

@@ -79,9 +79,7 @@ module Sidekiq
           # Most likely a problem with redis networking.
           # Punt and try again at the next interval
           logger.error ex.message
-          ex.backtrace.each do |bt|
-            logger.error(bt)
-          end
+          handle_exception(ex)
         end
       end
 
@@ -95,7 +93,7 @@ module Sidekiq
         # if poll_interval_average hasn't been calculated yet, we can
         # raise an error trying to reach Redis.
         logger.error ex.message
-        logger.error ex.backtrace.first
+        handle_exception(ex)
         sleep 5
       end
 

data/lib/sidekiq/util.rb

@@ -46,7 +46,10 @@ module Sidekiq
       @@identity ||= "#{hostname}:#{$$}:#{process_nonce}"
     end
 
-    def fire_event(event, reverse=false)
+    def fire_event(event, options={})
+      reverse = options[:reverse]
+      reraise = options[:reraise]
+
       arr = Sidekiq.options[:lifecycle_events][event]
       arr.reverse! if reverse
       arr.each do |block|
@@ -54,6 +57,7 @@ module Sidekiq
           block.call
         rescue => ex
           handle_exception(ex, { context: "Exception during Sidekiq lifecycle event.", event: event })
+          raise ex if reraise
         end
       end
       arr.clear

data/lib/sidekiq/version.rb

@@ -1,4 +1,4 @@
 # frozen_string_literal: true
 module Sidekiq
-  VERSION = "5.0.5"
+  VERSION = "5.1.0"
 end

data/lib/sidekiq/web/action.rb

@@ -77,7 +77,7 @@ module Sidekiq
     private
 
     def _erb(file, locals)
-      locals.each {|k, v| define_singleton_method(k){ v } } if locals
+      locals.each {|k, v| define_singleton_method(k){ v } unless (singleton_methods.include? k)} if locals
 
       if file.kind_of?(String)
         ERB.new(file).result(binding)

data/lib/sidekiq/web/helpers.rb

@@ -148,8 +148,10 @@ module Sidekiq
     end
 
     def redis_connection
-      attrs = Sidekiq.redis { |conn| conn.connection }
-      "redis://#{attrs[:location]}/#{attrs[:db]}"
+      Sidekiq.redis do |conn|
+        c = conn._client
+        "redis://#{c.location}/#{c.db}"
+      end
     end
 
     def namespace

data/web/locales/en.yml

@@ -77,3 +77,4 @@ en: # <---- change this to your locale code
   NotYetEnqueued: Not yet enqueued
   CreatedAt: Created At
   BackToApp: Back to App
+  Latency: Latency

data/web/locales/es.yml

@@ -67,3 +67,4 @@ es:
   Thread: Hilo
   Threads: Hilos
   Jobs: Trabajos
+  Latency: Latencia

data/web/locales/ja.yml

@@ -60,9 +60,9 @@ ja:
   SixMonths: 6 ヶ月
   Batches: バッチ
   Failures: 失敗
-  DeadJobs: 死亡したジョブ
-  NoDeadJobsFound: 死亡したジョブはありません
-  Dead: 死亡
+  DeadJobs: デッドジョブ
+  NoDeadJobsFound: デッドジョブはありません
+  Dead: デッド
   Processes: プロセス
   Thread: スレッド
   Threads: スレッド
@@ -76,3 +76,5 @@ ja:
   Plugins: プラグイン
   NotYetEnqueued: キューに入っていません
   CreatedAt: 作成日時
+  BackToApp: アプリに戻る
+  Latency: レイテンシ

data/web/views/queues.erb

@@ -5,6 +5,7 @@
     <thead>
       <th><%= t('Queue') %></th>
       <th><%= t('Size') %></th>
+      <th><%= t('Latency') %></th>
       <th><%= t('Actions') %></th>
     </thead>
     <% @queues.each do |queue| %>
@@ -16,6 +17,7 @@
           <% end %>
         </td>
         <td><%= number_with_delimiter(queue.size) %> </td>
+        <td><%= number_with_delimiter(queue.latency.round(2)) %> </td>
         <td class="delete-confirm">
           <form action="<%=root_path %>queues/<%= CGI.escape(queue.name) %>" method="post">
             <%= csrf_tag %>

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: sidekiq
 version: !ruby/object:Gem::Version
-  version: 5.0.5
+  version: 5.1.0
 platform: ruby
 authors:
 - Mike Perham
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2017-10-02 00:00:00.000000000 Z
+date: 2018-02-02 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: redis
@@ -169,6 +169,7 @@ files:
 - LICENSE
 - Pro-2.0-Upgrade.md
 - Pro-3.0-Upgrade.md
+- Pro-4.0-Upgrade.md
 - Pro-Changes.md
 - README.md
 - Rakefile
@@ -200,6 +201,7 @@ files:
 - lib/sidekiq/middleware/chain.rb
 - lib/sidekiq/middleware/i18n.rb
 - lib/sidekiq/middleware/server/active_record.rb
+- lib/sidekiq/middleware/server/active_record_cache.rb
 - lib/sidekiq/paginator.rb
 - lib/sidekiq/processor.rb
 - lib/sidekiq/rails.rb
@@ -289,7 +291,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.5.2
+rubygems_version: 2.6.13
 signing_key: 
 specification_version: 4
 summary: Simple, efficient background processing for Ruby