sidekiq 6.0.7 → 6.2.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (65) hide show
  1. checksums.yaml +4 -4
  2. data/Changes.md +70 -1
  3. data/README.md +2 -6
  4. data/bin/sidekiq +7 -2
  5. data/lib/sidekiq.rb +5 -3
  6. data/lib/sidekiq/api.rb +28 -10
  7. data/lib/sidekiq/cli.rb +23 -7
  8. data/lib/sidekiq/client.rb +16 -11
  9. data/lib/sidekiq/extensions/action_mailer.rb +3 -2
  10. data/lib/sidekiq/extensions/active_record.rb +4 -3
  11. data/lib/sidekiq/extensions/class_methods.rb +5 -4
  12. data/lib/sidekiq/fetch.rb +29 -21
  13. data/lib/sidekiq/job_retry.rb +1 -0
  14. data/lib/sidekiq/launcher.rb +55 -3
  15. data/lib/sidekiq/logger.rb +3 -2
  16. data/lib/sidekiq/manager.rb +4 -4
  17. data/lib/sidekiq/middleware/chain.rb +1 -1
  18. data/lib/sidekiq/processor.rb +4 -4
  19. data/lib/sidekiq/rails.rb +16 -18
  20. data/lib/sidekiq/redis_connection.rb +15 -12
  21. data/lib/sidekiq/sd_notify.rb +1 -1
  22. data/lib/sidekiq/testing.rb +1 -1
  23. data/lib/sidekiq/version.rb +1 -1
  24. data/lib/sidekiq/web.rb +35 -72
  25. data/lib/sidekiq/web/application.rb +9 -5
  26. data/lib/sidekiq/web/csrf_protection.rb +177 -0
  27. data/lib/sidekiq/web/helpers.rb +26 -8
  28. data/lib/sidekiq/web/router.rb +5 -2
  29. data/lib/sidekiq/worker.rb +2 -5
  30. data/sidekiq.gemspec +11 -4
  31. data/web/assets/images/apple-touch-icon.png +0 -0
  32. data/web/assets/javascripts/application.js +3 -8
  33. data/web/assets/stylesheets/application-dark.css +71 -33
  34. data/web/assets/stylesheets/application.css +24 -8
  35. data/web/locales/fr.yml +1 -1
  36. data/web/locales/pl.yml +4 -4
  37. data/web/locales/ru.yml +4 -0
  38. data/web/views/busy.erb +44 -14
  39. data/web/views/layout.erb +1 -0
  40. data/web/views/morgue.erb +1 -1
  41. data/web/views/queues.erb +1 -1
  42. data/web/views/retries.erb +1 -1
  43. data/web/views/scheduled.erb +1 -1
  44. metadata +17 -45
  45. data/.circleci/config.yml +0 -60
  46. data/.github/contributing.md +0 -32
  47. data/.github/issue_template.md +0 -11
  48. data/.gitignore +0 -13
  49. data/.standard.yml +0 -20
  50. data/3.0-Upgrade.md +0 -70
  51. data/4.0-Upgrade.md +0 -53
  52. data/5.0-Upgrade.md +0 -56
  53. data/6.0-Upgrade.md +0 -72
  54. data/COMM-LICENSE +0 -97
  55. data/Ent-2.0-Upgrade.md +0 -37
  56. data/Ent-Changes.md +0 -256
  57. data/Gemfile +0 -24
  58. data/Gemfile.lock +0 -208
  59. data/Pro-2.0-Upgrade.md +0 -138
  60. data/Pro-3.0-Upgrade.md +0 -44
  61. data/Pro-4.0-Upgrade.md +0 -35
  62. data/Pro-5.0-Upgrade.md +0 -25
  63. data/Pro-Changes.md +0 -782
  64. data/Rakefile +0 -10
  65. data/code_of_conduct.md +0 -50
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 3dd5d7a0dd8a7d877484df4ae3b9988f15e9cc4bc0bb4da23224be0049fa733e
4
- data.tar.gz: e2b7ddb4c045817837e996356851648bf32108b37fa1abcfb72795a185b4a238
3
+ metadata.gz: 990d3e553aed906265ffa0cafb74fa4c79e0ccde957f21ccc3d09531d01e75bf
4
+ data.tar.gz: 92a68ab1ea824dc78b91610e98c3d10ce4a5794e7aebd66803dc6f29db419420
5
5
  SHA512:
6
- metadata.gz: 5ce3f911b835b38ce212309bcdf81b4789e24a12898a0f2071b9adb9cbcdfa23a33bb7216c2f7c6dcab26d484f79a1271594d3218f63ed041f0027deab5a26f3
7
- data.tar.gz: 685a6890a43c990b2fd9709c49f9c82c96a487cdcf6fc0607ab15b9dbb47d52c9f797ec5094d3f92ec35c5737e661f966a6b81006ee4b7d99a7f31285180b3db
6
+ metadata.gz: 37664695dd79557af0395fe20855db1d250753dd8c874023a866c27d69e78bfa36b3f70501dad18a6ac64673e62fa8fdf413c77d0052a9ca0d59f25a17df41f9
7
+ data.tar.gz: 4d6ef75d6eb4be49e8b8d21d56bd0e300a284a4a6db9a0f0357724206b5e1b81b8bf75b53d8442319fd5fe5a9f08a467d75c59289d82d6aa70a00aa0d5102445
data/Changes.md CHANGED
@@ -2,6 +2,69 @@
2
2
 
3
3
  [Sidekiq Changes](https://github.com/mperham/sidekiq/blob/master/Changes.md) | [Sidekiq Pro Changes](https://github.com/mperham/sidekiq/blob/master/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/mperham/sidekiq/blob/master/Ent-Changes.md)
4
4
 
5
+ 6.2.0
6
+ ---------
7
+
8
+ - Store Redis RTT and log if poor [#4824]
9
+ - Add process/thread stats to Busy page [#4806]
10
+ - Improve Web UI on mobile devices [#4840]
11
+ - **Refactor Web UI session usage** [#4804]
12
+ Numerous people have hit "Forbidden" errors and struggled with Sidekiq's
13
+ Web UI session requirement. If you have code in your initializer for
14
+ Web sessions, it's quite possible it will need to be removed. Here's
15
+ an overview:
16
+ ```
17
+ Sidekiq::Web needs a valid Rack session for CSRF protection. If this is a Rails app,
18
+ make sure you mount Sidekiq::Web *inside* your routes in `config/routes.rb` so
19
+ Sidekiq can reuse the Rails session:
20
+
21
+ Rails.application.routes.draw do
22
+ mount Sidekiq::Web => "/sidekiq"
23
+ ....
24
+ end
25
+
26
+ If this is a bare Rack app, use a session middleware before Sidekiq::Web:
27
+
28
+ # first, use IRB to create a shared secret key for sessions and commit it
29
+ require 'securerandom'; File.open(".session.key", "w") {|f| f.write(SecureRandom.hex(32)) }
30
+
31
+ # now, update your Rack app to include the secret with a session cookie middleware
32
+ use Rack::Session::Cookie, secret: File.read(".session.key"), same_site: true, max_age: 86400
33
+ run Sidekiq::Web
34
+ ```
35
+
36
+ 6.1.3
37
+ ---------
38
+
39
+ - Warn if Redis is configured to evict data under memory pressure [#4752]
40
+ - Add process RSS on the Busy page [#4717]
41
+
42
+ 6.1.2
43
+ ---------
44
+
45
+ - Improve readability in dark mode Web UI [#4674]
46
+ - Fix Web UI crash with corrupt session [#4672]
47
+ - Allow middleware to yield arguments [#4673, @eugeneius]
48
+ - Migrate CI from CircleCI to GitHub Actions [#4677]
49
+
50
+ 6.1.1
51
+ ---------
52
+
53
+ - Jobs are now sorted by age in the Busy Workers table. [#4641]
54
+ - Fix "check all" JS logic in Web UI [#4619]
55
+
56
+ 6.1.0
57
+ ---------
58
+
59
+ - Web UI - Dark Mode fixes [#4543, natematykiewicz]
60
+ - Ensure `Rack::ContentLength` is loaded as middleware for correct Web UI responses [#4541]
61
+ - Avoid exception dumping SSL store in Redis connection logging [#4532]
62
+ - Better error messages in Sidekiq::Client [#4549]
63
+ - Remove rack-protection, reimplement CSRF protection [#4588]
64
+ - Require redis-rb 4.2 [#4591]
65
+ - Update to jquery 1.12.4 [#4593]
66
+ - Refactor internal fetch logic and API [#4602]
67
+
5
68
  6.0.7
6
69
  ---------
7
70
 
@@ -124,7 +187,7 @@ assert_equal 1, Sidekiq::Extensions::DelayedMailer.jobs_for(FooMailer).size
124
187
 
125
188
  This release has major breaking changes. Read and test carefully in production.
126
189
 
127
- - With Rails 6.0.1+, ActiveJobs can now use `sidekiq_options` directly to configure Sidekiq
190
+ - With Rails 6.0.2+, ActiveJobs can now use `sidekiq_options` directly to configure Sidekiq
128
191
  features/internals like the retry subsystem. [#4213, pirj]
129
192
  ```ruby
130
193
  class MyJob < ActiveJob::Base
@@ -160,6 +223,12 @@ See the [Logging wiki page](https://github.com/mperham/sidekiq/wiki/Logging) for
160
223
  - Integrate the StandardRB code formatter to ensure consistent code
161
224
  styling. [#4114, gearnode]
162
225
 
226
+ 5.2.9
227
+ ---------
228
+
229
+ - Release Rack lock due to a cascade of CVEs. [#4566]
230
+ Pro-tip: don't lock Rack.
231
+
163
232
  5.2.8
164
233
  ---------
165
234
 
data/README.md CHANGED
@@ -2,11 +2,7 @@ Sidekiq
2
2
  ==============
3
3
 
4
4
  [![Gem Version](https://badge.fury.io/rb/sidekiq.svg)](https://rubygems.org/gems/sidekiq)
5
- [![Code Climate](https://codeclimate.com/github/mperham/sidekiq.svg)](https://codeclimate.com/github/mperham/sidekiq)
6
- [![Test Coverage](https://codeclimate.com/github/mperham/sidekiq/badges/coverage.svg)](https://codeclimate.com/github/mperham/sidekiq/coverage)
7
- [![Build Status](https://circleci.com/gh/mperham/sidekiq/tree/master.svg?style=svg)](https://circleci.com/gh/mperham/sidekiq/tree/master)
8
- [![Gitter Chat](https://badges.gitter.im/mperham/sidekiq.svg)](https://gitter.im/mperham/sidekiq)
9
-
5
+ ![Build](https://github.com/mperham/sidekiq/workflows/CI/badge.svg)
10
6
 
11
7
  Simple, efficient background processing for Ruby.
12
8
 
@@ -94,4 +90,4 @@ Please see [LICENSE](https://github.com/mperham/sidekiq/blob/master/LICENSE) for
94
90
  Author
95
91
  -----------------
96
92
 
97
- Mike Perham, [@mperham@mastodon.xyz](https://mastodon.xyz/@mperham) / [@sidekiq](https://twitter.com/sidekiq), [https://www.mikeperham.com](https://www.mikeperham.com) / [https://www.contribsys.com](https://www.contribsys.com)
93
+ Mike Perham, [@getajobmike](https://twitter.com/getajobmike) / [@sidekiq](https://twitter.com/sidekiq), [https://www.mikeperham.com](https://www.mikeperham.com) / [https://www.contribsys.com](https://www.contribsys.com)
data/bin/sidekiq CHANGED
@@ -31,7 +31,12 @@ begin
31
31
  cli.run
32
32
  rescue => e
33
33
  raise e if $DEBUG
34
- STDERR.puts e.message
35
- STDERR.puts e.backtrace.join("\n")
34
+ if Sidekiq.error_handlers.length == 0
35
+ STDERR.puts e.message
36
+ STDERR.puts e.backtrace.join("\n")
37
+ else
38
+ cli.handle_exception e
39
+ end
40
+
36
41
  exit 1
37
42
  end
data/lib/sidekiq.rb CHANGED
@@ -20,6 +20,7 @@ module Sidekiq
20
20
  labels: [],
21
21
  concurrency: 10,
22
22
  require: ".",
23
+ strict: true,
23
24
  environment: nil,
24
25
  timeout: 25,
25
26
  poll_interval_average: nil,
@@ -95,10 +96,11 @@ module Sidekiq
95
96
  retryable = true
96
97
  begin
97
98
  yield conn
98
- rescue Redis::CommandError => ex
99
+ rescue Redis::BaseError => ex
99
100
  # 2550 Failover can cause the server to become a replica, need
100
101
  # to disconnect and reopen the socket to get back to the primary.
101
- if retryable && ex.message =~ /READONLY/
102
+ # 4495 Use the same logic if we have a "Not enough replicas" error from the primary
103
+ if retryable && ex.message =~ /READONLY|NOREPLICAS/
102
104
  conn.disconnect!
103
105
  retryable = false
104
106
  retry
@@ -196,7 +198,7 @@ module Sidekiq
196
198
  end
197
199
 
198
200
  def self.logger
199
- @logger ||= Sidekiq::Logger.new(STDOUT, level: Logger::INFO)
201
+ @logger ||= Sidekiq::Logger.new($stdout, level: Logger::INFO)
200
202
  end
201
203
 
202
204
  def self.logger=(logger)
data/lib/sidekiq/api.rb CHANGED
@@ -85,10 +85,10 @@ module Sidekiq
85
85
 
86
86
  default_queue_latency = if (entry = pipe1_res[6].first)
87
87
  job = begin
88
- Sidekiq.load_json(entry)
89
- rescue
90
- {}
91
- end
88
+ Sidekiq.load_json(entry)
89
+ rescue
90
+ {}
91
+ end
92
92
  now = Time.now.to_f
93
93
  thence = job["enqueued_at"] || now
94
94
  now - thence
@@ -791,19 +791,23 @@ module Sidekiq
791
791
  # you'll be happier this way
792
792
  conn.pipelined do
793
793
  procs.each do |key|
794
- conn.hmget(key, "info", "busy", "beat", "quiet")
794
+ conn.hmget(key, "info", "busy", "beat", "quiet", "rss", "rtt_us")
795
795
  end
796
796
  end
797
797
  }
798
798
 
799
- result.each do |info, busy, at_s, quiet|
799
+ result.each do |info, busy, at_s, quiet, rss, rtt|
800
800
  # If a process is stopped between when we query Redis for `procs` and
801
801
  # when we query for `result`, we will have an item in `result` that is
802
802
  # composed of `nil` values.
803
803
  next if info.nil?
804
804
 
805
805
  hash = Sidekiq.load_json(info)
806
- yield Process.new(hash.merge("busy" => busy.to_i, "beat" => at_s.to_f, "quiet" => quiet))
806
+ yield Process.new(hash.merge("busy" => busy.to_i,
807
+ "beat" => at_s.to_f,
808
+ "quiet" => quiet,
809
+ "rss" => rss.to_i,
810
+ "rtt_us" => rtt.to_i))
807
811
  end
808
812
  end
809
813
 
@@ -815,6 +819,17 @@ module Sidekiq
815
819
  Sidekiq.redis { |conn| conn.scard("processes") }
816
820
  end
817
821
 
822
+ # Total number of threads available to execute jobs.
823
+ # For Sidekiq Enterprise customers this number (in production) must be
824
+ # less than or equal to your licensed concurrency.
825
+ def total_concurrency
826
+ sum { |x| x["concurrency"] }
827
+ end
828
+
829
+ def total_rss
830
+ sum { |x| x["rss"] || 0 }
831
+ end
832
+
818
833
  # Returns the identity of the current cluster leader or "" if no leader.
819
834
  # This is a Sidekiq Enterprise feature, will always return "" in Sidekiq
820
835
  # or Sidekiq Pro.
@@ -916,12 +931,13 @@ module Sidekiq
916
931
  class Workers
917
932
  include Enumerable
918
933
 
919
- def each
934
+ def each(&block)
935
+ results = []
920
936
  Sidekiq.redis do |conn|
921
937
  procs = conn.sscan_each("processes").to_a
922
938
  procs.sort.each do |key|
923
939
  valid, workers = conn.pipelined {
924
- conn.exists(key)
940
+ conn.exists?(key)
925
941
  conn.hgetall("#{key}:workers")
926
942
  }
927
943
  next unless valid
@@ -930,10 +946,12 @@ module Sidekiq
930
946
  p = hsh["payload"]
931
947
  # avoid breaking API, this is a side effect of the JSON optimization in #4316
932
948
  hsh["payload"] = Sidekiq.load_json(p) if p.is_a?(String)
933
- yield key, tid, hsh
949
+ results << [key, tid, hsh]
934
950
  end
935
951
  end
936
952
  end
953
+
954
+ results.sort_by { |(_, _, hsh)| hsh["run_at"] }.each(&block)
937
955
  end
938
956
 
939
957
  # Note that #size is only as accurate as Sidekiq's heartbeat,
data/lib/sidekiq/cli.rb CHANGED
@@ -33,8 +33,9 @@ module Sidekiq
33
33
  # Code within this method is not tested because it alters
34
34
  # global process state irreversibly. PRs which improve the
35
35
  # test coverage of Sidekiq::CLI are welcomed.
36
- def run
37
- boot_system
36
+ def run(boot_app: true)
37
+ boot_application if boot_app
38
+
38
39
  if environment == "development" && $stdout.tty? && Sidekiq.log_formatter.is_a?(Sidekiq::Logger::Formatters::Pretty)
39
40
  print_banner
40
41
  end
@@ -43,7 +44,7 @@ module Sidekiq
43
44
  self_read, self_write = IO.pipe
44
45
  sigs = %w[INT TERM TTIN TSTP]
45
46
  # USR1 and USR2 don't work on the JVM
46
- sigs << "USR2" unless jruby?
47
+ sigs << "USR2" if Sidekiq.pro? && !jruby?
47
48
  sigs.each do |sig|
48
49
  trap sig do
49
50
  self_write.puts(sig)
@@ -58,9 +59,22 @@ module Sidekiq
58
59
 
59
60
  # touch the connection pool so it is created before we
60
61
  # fire startup and start multithreading.
61
- ver = Sidekiq.redis_info["redis_version"]
62
+ info = Sidekiq.redis_info
63
+ ver = info["redis_version"]
62
64
  raise "You are connecting to Redis v#{ver}, Sidekiq requires Redis v4.0.0 or greater" if ver < "4"
63
65
 
66
+ maxmemory_policy = info["maxmemory_policy"]
67
+ if maxmemory_policy != "noeviction"
68
+ logger.warn <<~EOM
69
+
70
+
71
+ WARNING: Your Redis instance will evict Sidekiq data under heavy load.
72
+ The 'noeviction' maxmemory policy is recommended (current policy: '#{maxmemory_policy}').
73
+ See: https://github.com/mperham/sidekiq/wiki/Using-Redis#memory
74
+
75
+ EOM
76
+ end
77
+
64
78
  # Since the user can pass us a connection pool explicitly in the initializer, we
65
79
  # need to verify the size is large enough or else Sidekiq's performance is dramatically slowed.
66
80
  cursize = Sidekiq.redis_pool.size
@@ -228,8 +242,7 @@ module Sidekiq
228
242
  opts = parse_config(opts[:config_file]).merge(opts) if opts[:config_file]
229
243
 
230
244
  # set defaults
231
- opts[:queues] = ["default"] if opts[:queues].nil? || opts[:queues].empty?
232
- opts[:strict] = true if opts[:strict].nil?
245
+ opts[:queues] = ["default"] if opts[:queues].nil?
233
246
  opts[:concurrency] = Integer(ENV["RAILS_MAX_THREADS"]) if opts[:concurrency].nil? && ENV["RAILS_MAX_THREADS"]
234
247
 
235
248
  # merge with defaults
@@ -240,7 +253,7 @@ module Sidekiq
240
253
  Sidekiq.options
241
254
  end
242
255
 
243
- def boot_system
256
+ def boot_application
244
257
  ENV["RACK_ENV"] = ENV["RAILS_ENV"] = environment
245
258
 
246
259
  if File.directory?(options[:require])
@@ -368,6 +381,8 @@ module Sidekiq
368
381
  end
369
382
 
370
383
  opts = opts.merge(opts.delete(environment.to_sym) || {})
384
+ opts.delete(:strict)
385
+
371
386
  parse_queues(opts, opts.delete(:queues) || [])
372
387
 
373
388
  opts
@@ -379,6 +394,7 @@ module Sidekiq
379
394
 
380
395
  def parse_queue(opts, queue, weight = nil)
381
396
  opts[:queues] ||= []
397
+ opts[:strict] = true if opts[:strict].nil?
382
398
  raise ArgumentError, "queues: #{queue} cannot be defined twice" if opts[:queues].include?(queue)
383
399
  [weight.to_i, 1].max.times { opts[:queues] << queue }
384
400
  opts[:strict] = false if weight.to_i > 0
@@ -19,7 +19,7 @@ module Sidekiq
19
19
  #
20
20
  def middleware(&block)
21
21
  @chain ||= Sidekiq.client_middleware
22
- if block_given?
22
+ if block
23
23
  @chain = @chain.dup
24
24
  yield @chain
25
25
  end
@@ -90,16 +90,17 @@ module Sidekiq
90
90
  # Returns an array of the of pushed jobs' jids. The number of jobs pushed can be less
91
91
  # than the number given if the middleware stopped processing for one or more jobs.
92
92
  def push_bulk(items)
93
- arg = items["args"].first
94
- return [] unless arg # no jobs to push
95
- raise ArgumentError, "Bulk arguments must be an Array of Arrays: [[1], [2]]" unless arg.is_a?(Array)
93
+ args = items["args"]
94
+ raise ArgumentError, "Bulk arguments must be an Array of Arrays: [[1], [2]]" unless args.is_a?(Array) && args.all?(Array)
95
+ return [] if args.empty? # no jobs to push
96
96
 
97
97
  at = items.delete("at")
98
98
  raise ArgumentError, "Job 'at' must be a Numeric or an Array of Numeric timestamps" if at && (Array(at).empty? || !Array(at).all?(Numeric))
99
+ raise ArgumentError, "Job 'at' Array must have same size as 'args' Array" if at.is_a?(Array) && at.size != args.size
99
100
 
100
101
  normed = normalize_item(items)
101
- payloads = items["args"].map.with_index { |args, index|
102
- copy = normed.merge("args" => args, "jid" => SecureRandom.hex(12), "enqueued_at" => Time.now.to_f)
102
+ payloads = args.map.with_index { |job_args, index|
103
+ copy = normed.merge("args" => job_args, "jid" => SecureRandom.hex(12), "enqueued_at" => Time.now.to_f)
103
104
  copy["at"] = (at.is_a?(Array) ? at[index] : at) if at
104
105
 
105
106
  result = process_single(items["class"], copy)
@@ -218,16 +219,20 @@ module Sidekiq
218
219
  end
219
220
  end
220
221
 
222
+ def validate(item)
223
+ raise(ArgumentError, "Job must be a Hash with 'class' and 'args' keys: `#{item}`") unless item.is_a?(Hash) && item.key?("class") && item.key?("args")
224
+ raise(ArgumentError, "Job args must be an Array: `#{item}`") unless item["args"].is_a?(Array)
225
+ raise(ArgumentError, "Job class must be either a Class or String representation of the class name: `#{item}`") unless item["class"].is_a?(Class) || item["class"].is_a?(String)
226
+ raise(ArgumentError, "Job 'at' must be a Numeric timestamp: `#{item}`") if item.key?("at") && !item["at"].is_a?(Numeric)
227
+ raise(ArgumentError, "Job tags must be an Array: `#{item}`") if item["tags"] && !item["tags"].is_a?(Array)
228
+ end
229
+
221
230
  def normalize_item(item)
222
231
  # 6.0.0 push_bulk bug, #4321
223
232
  # TODO Remove after a while...
224
233
  item.delete("at") if item.key?("at") && item["at"].nil?
225
234
 
226
- raise(ArgumentError, "Job must be a Hash with 'class' and 'args' keys: { 'class' => SomeWorker, 'args' => ['bob', 1, :foo => 'bar'] }") unless item.is_a?(Hash) && item.key?("class") && item.key?("args")
227
- raise(ArgumentError, "Job args must be an Array") unless item["args"].is_a?(Array)
228
- raise(ArgumentError, "Job class must be either a Class or String representation of the class name") unless item["class"].is_a?(Class) || item["class"].is_a?(String)
229
- raise(ArgumentError, "Job 'at' must be a Numeric timestamp") if item.key?("at") && !item["at"].is_a?(Numeric)
230
- raise(ArgumentError, "Job tags must be an Array") if item["tags"] && !item["tags"].is_a?(Array)
235
+ validate(item)
231
236
  # raise(ArgumentError, "Arguments must be native JSON types, see https://github.com/mperham/sidekiq/wiki/Best-Practices") unless JSON.load(JSON.dump(item['args'])) == item['args']
232
237
 
233
238
  # merge in the default sidekiq_options for the item's class and/or wrapped element
@@ -5,9 +5,10 @@ require "sidekiq/extensions/generic_proxy"
5
5
  module Sidekiq
6
6
  module Extensions
7
7
  ##
8
- # Adds 'delay', 'delay_for' and `delay_until` methods to ActionMailer to offload arbitrary email
9
- # delivery to Sidekiq. Example:
8
+ # Adds +delay+, +delay_for+ and +delay_until+ methods to ActionMailer to offload arbitrary email
9
+ # delivery to Sidekiq.
10
10
  #
11
+ # @example
11
12
  # UserMailer.delay.send_welcome_email(new_user)
12
13
  # UserMailer.delay_for(5.days).send_welcome_email(new_user)
13
14
  # UserMailer.delay_until(5.days.from_now).send_welcome_email(new_user)
@@ -5,10 +5,11 @@ require "sidekiq/extensions/generic_proxy"
5
5
  module Sidekiq
6
6
  module Extensions
7
7
  ##
8
- # Adds 'delay', 'delay_for' and `delay_until` methods to ActiveRecord to offload instance method
9
- # execution to Sidekiq. Examples:
8
+ # Adds +delay+, +delay_for+ and +delay_until+ methods to ActiveRecord to offload instance method
9
+ # execution to Sidekiq.
10
10
  #
11
- # User.recent_signups.each { |user| user.delay.mark_as_awesome }
11
+ # @example
12
+ # User.recent_signups.each { |user| user.delay.mark_as_awesome }
12
13
  #
13
14
  # Please note, this is not recommended as this will serialize the entire
14
15
  # object to Redis. Your Sidekiq jobs should pass IDs, not entire instances.
@@ -5,11 +5,12 @@ require "sidekiq/extensions/generic_proxy"
5
5
  module Sidekiq
6
6
  module Extensions
7
7
  ##
8
- # Adds 'delay', 'delay_for' and `delay_until` methods to all Classes to offload class method
9
- # execution to Sidekiq. Examples:
8
+ # Adds `delay`, `delay_for` and `delay_until` methods to all Classes to offload class method
9
+ # execution to Sidekiq.
10
10
  #
11
- # User.delay.delete_inactive
12
- # Wikipedia.delay.download_changes_for(Date.today)
11
+ # @example
12
+ # User.delay.delete_inactive
13
+ # Wikipedia.delay.download_changes_for(Date.today)
13
14
  #
14
15
  class DelayedClass
15
16
  include Sidekiq::Worker
data/lib/sidekiq/fetch.rb CHANGED
@@ -25,8 +25,10 @@ module Sidekiq
25
25
  }
26
26
 
27
27
  def initialize(options)
28
- @strictly_ordered_queues = !!options[:strict]
29
- @queues = options[:queues].map { |q| "queue:#{q}" }
28
+ raise ArgumentError, "missing queue list" unless options[:queues]
29
+ @options = options
30
+ @strictly_ordered_queues = !!@options[:strict]
31
+ @queues = @options[:queues].map { |q| "queue:#{q}" }
30
32
  if @strictly_ordered_queues
31
33
  @queues.uniq!
32
34
  @queues << TIMEOUT
@@ -34,28 +36,19 @@ module Sidekiq
34
36
  end
35
37
 
36
38
  def retrieve_work
37
- work = Sidekiq.redis { |conn| conn.brpop(*queues_cmd) }
38
- UnitOfWork.new(*work) if work
39
- end
40
-
41
- # Creating the Redis#brpop command takes into account any
42
- # configured queue weights. By default Redis#brpop returns
43
- # data from the first queue that has pending elements. We
44
- # recreate the queue command each time we invoke Redis#brpop
45
- # to honor weights and avoid queue starvation.
46
- def queues_cmd
47
- if @strictly_ordered_queues
48
- @queues
49
- else
50
- queues = @queues.shuffle!.uniq
51
- queues << TIMEOUT
52
- queues
39
+ qs = queues_cmd
40
+ # 4825 Sidekiq Pro with all queues paused will return an
41
+ # empty set of queues with a trailing TIMEOUT value.
42
+ if qs.size <= 1
43
+ sleep(2)
44
+ return nil
53
45
  end
46
+
47
+ work = Sidekiq.redis { |conn| conn.brpop(*qs) }
48
+ UnitOfWork.new(*work) if work
54
49
  end
55
50
 
56
- # By leaving this as a class method, it can be pluggable and used by the Manager actor. Making it
57
- # an instance method will make it async to the Fetcher actor
58
- def self.bulk_requeue(inprogress, options)
51
+ def bulk_requeue(inprogress, options)
59
52
  return if inprogress.empty?
60
53
 
61
54
  Sidekiq.logger.debug { "Re-queueing terminated jobs" }
@@ -76,5 +69,20 @@ module Sidekiq
76
69
  rescue => ex
77
70
  Sidekiq.logger.warn("Failed to requeue #{inprogress.size} jobs: #{ex.message}")
78
71
  end
72
+
73
+ # Creating the Redis#brpop command takes into account any
74
+ # configured queue weights. By default Redis#brpop returns
75
+ # data from the first queue that has pending elements. We
76
+ # recreate the queue command each time we invoke Redis#brpop
77
+ # to honor weights and avoid queue starvation.
78
+ def queues_cmd
79
+ if @strictly_ordered_queues
80
+ @queues
81
+ else
82
+ queues = @queues.shuffle!.uniq
83
+ queues << TIMEOUT
84
+ queues
85
+ end
86
+ end
79
87
  end
80
88
  end