sidekiq 6.3.1 → 6.4.2

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of sidekiq might be problematic. Click here for more details.

Files changed (49) hide show
  1. checksums.yaml +4 -4
  2. data/Changes.md +56 -0
  3. data/LICENSE +3 -3
  4. data/README.md +7 -2
  5. data/bin/sidekiq +3 -3
  6. data/bin/sidekiqload +57 -65
  7. data/bin/sidekiqmon +1 -1
  8. data/lib/generators/sidekiq/job_generator.rb +57 -0
  9. data/lib/generators/sidekiq/templates/{worker.rb.erb → job.rb.erb} +2 -2
  10. data/lib/generators/sidekiq/templates/{worker_spec.rb.erb → job_spec.rb.erb} +1 -1
  11. data/lib/generators/sidekiq/templates/{worker_test.rb.erb → job_test.rb.erb} +1 -1
  12. data/lib/sidekiq/api.rb +68 -62
  13. data/lib/sidekiq/cli.rb +16 -6
  14. data/lib/sidekiq/client.rb +44 -64
  15. data/lib/sidekiq/delay.rb +2 -0
  16. data/lib/sidekiq/extensions/generic_proxy.rb +1 -1
  17. data/lib/sidekiq/fetch.rb +2 -2
  18. data/lib/sidekiq/job_logger.rb +15 -27
  19. data/lib/sidekiq/job_retry.rb +28 -26
  20. data/lib/sidekiq/job_util.rb +67 -0
  21. data/lib/sidekiq/launcher.rb +37 -36
  22. data/lib/sidekiq/logger.rb +8 -18
  23. data/lib/sidekiq/manager.rb +13 -15
  24. data/lib/sidekiq/middleware/chain.rb +4 -4
  25. data/lib/sidekiq/middleware/current_attributes.rb +6 -1
  26. data/lib/sidekiq/middleware/i18n.rb +4 -4
  27. data/lib/sidekiq/monitor.rb +1 -1
  28. data/lib/sidekiq/paginator.rb +8 -8
  29. data/lib/sidekiq/processor.rb +27 -27
  30. data/lib/sidekiq/rails.rb +11 -4
  31. data/lib/sidekiq/redis_connection.rb +2 -2
  32. data/lib/sidekiq/scheduled.rb +13 -3
  33. data/lib/sidekiq/testing/inline.rb +4 -4
  34. data/lib/sidekiq/testing.rb +36 -35
  35. data/lib/sidekiq/util.rb +13 -0
  36. data/lib/sidekiq/version.rb +1 -1
  37. data/lib/sidekiq/web/application.rb +5 -2
  38. data/lib/sidekiq/web/csrf_protection.rb +2 -2
  39. data/lib/sidekiq/web/helpers.rb +4 -4
  40. data/lib/sidekiq/web.rb +3 -3
  41. data/lib/sidekiq/worker.rb +71 -16
  42. data/lib/sidekiq.rb +27 -15
  43. data/web/assets/javascripts/application.js +58 -26
  44. data/web/assets/stylesheets/application-dark.css +13 -17
  45. data/web/assets/stylesheets/application.css +4 -5
  46. data/web/views/_summary.erb +1 -1
  47. data/web/views/busy.erb +3 -3
  48. metadata +8 -7
  49. data/lib/generators/sidekiq/worker_generator.rb +0 -57
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 33cb573c76378ff933909bdc009358cc94b81e9e50c17b6dd8160170fe744f25
4
- data.tar.gz: f13c16da01f5cc0ef9514396aed899c8e1f69049d2d425892d3f5e774d68c6a7
3
+ metadata.gz: 70d899b76bdd764a2ec7e5f23b6d056494b58897daaf49db672daad3e0f60237
4
+ data.tar.gz: daae0430ecaf8eb7f172c70e628581c20a17e1a7cb1a58170b9082fc3dde1ea2
5
5
  SHA512:
6
- metadata.gz: 9829304cbe0810266d2a320b14386561a4f22965aa3f9fab311253c1edd9f3075298dadba28afc0f69f898ed69d858833e744e52db1012a0d23037ecbdf33ce3
7
- data.tar.gz: b38f253759ab7e3479829faad0e60e42ad2a6281b4dc022c66b0af652c53e85e5d7362bc07ecf02ae3ae3c530e3619572ad070267d2690aa5b4fafc08762d73e
6
+ metadata.gz: 930a0feb7ff47473eb995f8ba695468498afecb311c25296c93042ecfa00cd2bf48e19efd5ba775b4895c065705a344043e0c0e5d7b4115c2dceb1ec7cf341f9
7
+ data.tar.gz: 14ab645609e7c9f3fa954432097cd46d6e9e74e5bddc9ed1541697d13f3399add407ffb5af23c89c82651793c97e2c54b8fd4f845ca20287ce6bf033fd9caed2
data/Changes.md CHANGED
@@ -2,6 +2,55 @@
2
2
 
3
3
  [Sidekiq Changes](https://github.com/mperham/sidekiq/blob/main/Changes.md) | [Sidekiq Pro Changes](https://github.com/mperham/sidekiq/blob/main/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/mperham/sidekiq/blob/main/Ent-Changes.md)
4
4
 
5
+ 6.4.2
6
+ ---------
7
+
8
+ - Strict argument checking now runs after client-side middleware [#5246]
9
+ - Fix page events with live polling [#5184]
10
+ - Many under-the-hood changes to remove all usage of the term "worker"
11
+ from the Sidekiq codebase and APIs. This mostly involved RDoc and local
12
+ variable names but a few constants and public APIs were changed. The old
13
+ APIs will be removed in Sidekiq 7.0.
14
+ ```
15
+ Sidekiq::DEFAULT_WORKER_OPTIONS -> Sidekiq.default_job_options
16
+ Sidekiq.default_worker_options -> Sidekiq.default_job_options
17
+ Sidekiq::Queues["default"].jobs_by_worker(HardJob) -> Sidekiq::Queues["default"].jobs_by_class(HardJob)
18
+ ```
19
+
20
+ 6.4.1
21
+ ---------
22
+
23
+ - Fix pipeline/multi deprecations in redis-rb 4.6
24
+ - Fix sidekiq.yml YAML load errors on Ruby 3.1 [#5141]
25
+ - Sharding support for `perform_bulk` [#5129]
26
+ - Refactor job logger for SPEEEEEEED
27
+
28
+ 6.4.0
29
+ ---------
30
+
31
+ - **SECURITY**: Validate input to avoid possible DoS in Web UI.
32
+ - Add **strict argument checking** [#5071]
33
+ Sidekiq will now log a warning if JSON-unsafe arguments are passed to `perform_async`.
34
+ Add `Sidekiq.strict_args!(false)` to your initializer to disable this warning.
35
+ This warning will switch to an exception in Sidekiq 7.0.
36
+ - Note that Delayed Extensions will be removed in Sidekiq 7.0 [#5076]
37
+ - Add `perform_{inline,sync}` in Sidekiq::Job to run a job synchronously [#5061, hasan-ally]
38
+ ```ruby
39
+ SomeJob.perform_async(args...)
40
+ SomeJob.perform_sync(args...)
41
+ SomeJob.perform_inline(args...)
42
+ ```
43
+ You can also dynamically redirect a job to run synchronously:
44
+ ```ruby
45
+ SomeJob.set("sync": true).perform_async(args...) # will run via perform_inline
46
+ ```
47
+ - Replace Sidekiq::Worker `app/workers` generator with Sidekiq::Job `app/sidekiq` generator [#5055]
48
+ ```
49
+ bin/rails generate sidekiq:job ProcessOrderJob
50
+ ```
51
+ - Fix job retries losing CurrentAttributes [#5090]
52
+ - Tweak shutdown to give long-running threads time to cleanup [#5095]
53
+
5
54
  6.3.1
6
55
  ---------
7
56
 
@@ -285,6 +334,13 @@ See the [Logging wiki page](https://github.com/mperham/sidekiq/wiki/Logging) for
285
334
  - Integrate the StandardRB code formatter to ensure consistent code
286
335
  styling. [#4114, gearnode]
287
336
 
337
+ 5.2.10
338
+ ---------
339
+
340
+ - Backport fix for CVE-2022-23837.
341
+ - Migrate to `exists?` for redis-rb.
342
+ - Lock redis-rb to <4.6 to avoid deprecations.
343
+
288
344
  5.2.9
289
345
  ---------
290
346
 
data/LICENSE CHANGED
@@ -4,6 +4,6 @@ Sidekiq is an Open Source project licensed under the terms of
4
4
  the LGPLv3 license. Please see <http://www.gnu.org/licenses/lgpl-3.0.html>
5
5
  for license text.
6
6
 
7
- Sidekiq Pro has a commercial-friendly license allowing private forks
8
- and modifications of Sidekiq. Please see https://sidekiq.org/products/pro.html for
9
- more detail. You can find the commercial license terms in COMM-LICENSE.txt.
7
+ Sidekiq Pro and Sidekiq Enterprise have a commercial-friendly license.
8
+ You can find the commercial license in COMM-LICENSE.txt.
9
+ Please see https://sidekiq.org for purchasing options.
data/README.md CHANGED
@@ -36,14 +36,14 @@ Sidekiq 6.0 supports Rails 5.0+ but does not require it.
36
36
  Installation
37
37
  -----------------
38
38
 
39
- gem install sidekiq
39
+ bundle add sidekiq
40
40
 
41
41
 
42
42
  Getting Started
43
43
  -----------------
44
44
 
45
45
  See the [Getting Started wiki page](https://github.com/mperham/sidekiq/wiki/Getting-Started) and follow the simple setup process.
46
- You can watch [this Youtube playlist](https://www.youtube.com/playlist?list=PLjeHh2LSCFrWGT5uVjUuFKAcrcj5kSai1) to learn all about
46
+ You can watch [this YouTube playlist](https://www.youtube.com/playlist?list=PLjeHh2LSCFrWGT5uVjUuFKAcrcj5kSai1) to learn all about
47
47
  Sidekiq and see its features in action. Here's the Web UI:
48
48
 
49
49
  ![Web UI](https://github.com/mperham/sidekiq/raw/main/examples/web-ui.png)
@@ -80,6 +80,11 @@ Useful resources:
80
80
  Every Friday morning is Sidekiq happy hour: I video chat and answer questions.
81
81
  See the [Sidekiq support page](https://sidekiq.org/support.html) for details.
82
82
 
83
+ Contributing
84
+ -----------------
85
+
86
+ Please see [the contributing guidelines](https://github.com/mperham/sidekiq/blob/main/.github/contributing.md).
87
+
83
88
 
84
89
  License
85
90
  -----------------
data/bin/sidekiq CHANGED
@@ -4,7 +4,7 @@
4
4
  # RUBYOPT=-w bundle exec sidekiq
5
5
  $TESTING = false
6
6
 
7
- require_relative '../lib/sidekiq/cli'
7
+ require_relative "../lib/sidekiq/cli"
8
8
 
9
9
  def integrate_with_systemd
10
10
  return unless ENV["NOTIFY_SOCKET"]
@@ -32,8 +32,8 @@ begin
32
32
  rescue => e
33
33
  raise e if $DEBUG
34
34
  if Sidekiq.error_handlers.length == 0
35
- STDERR.puts e.message
36
- STDERR.puts e.backtrace.join("\n")
35
+ warn e.message
36
+ warn e.backtrace.join("\n")
37
37
  else
38
38
  cli.handle_exception e
39
39
  end
data/bin/sidekiqload CHANGED
@@ -4,19 +4,18 @@
4
4
  # RUBYOPT=-w bundle exec sidekiq
5
5
  $TESTING = false
6
6
 
7
- #require 'ruby-prof'
8
- require 'bundler/setup'
7
+ # require "ruby-prof"
8
+ require "bundler/setup"
9
9
  Bundler.require(:default, :load_test)
10
10
 
11
- require_relative '../lib/sidekiq/cli'
12
- require_relative '../lib/sidekiq/launcher'
13
-
14
- include Sidekiq::Util
11
+ require_relative "../lib/sidekiq/cli"
12
+ require_relative "../lib/sidekiq/launcher"
15
13
 
16
14
  Sidekiq.configure_server do |config|
17
15
  config.options[:concurrency] = 10
18
- config.redis = { db: 13, port: 6380, driver: :hiredis }
19
- config.options[:queues] << 'default'
16
+ config.redis = {db: 13, port: 6380}
17
+ # config.redis = { db: 13, port: 6380, driver: :hiredis}
18
+ config.options[:queues] << "default"
20
19
  config.logger.level = Logger::ERROR
21
20
  config.average_scheduled_poll_interval = 2
22
21
  config.reliable! if defined?(Sidekiq::Pro)
@@ -29,53 +28,50 @@ class LoadWorker
29
28
  1
30
29
  end
31
30
 
32
- def perform(idx, ts=nil)
33
- puts(Time.now.to_f - ts) if ts != nil
34
- #raise idx.to_s if idx % 100 == 1
31
+ def perform(idx, ts = nil)
32
+ puts(Time.now.to_f - ts) if !ts.nil?
33
+ # raise idx.to_s if idx % 100 == 1
35
34
  end
36
35
  end
37
36
 
38
37
  # brew tap shopify/shopify
39
38
  # brew install toxiproxy
40
- # gem install toxiproxy
41
39
  # run `toxiproxy-server` in a separate terminal window.
42
- require 'toxiproxy'
40
+ require "toxiproxy"
43
41
  # simulate a non-localhost network for realer-world conditions.
44
42
  # adding 1ms of network latency has an ENORMOUS impact on benchmarks
45
43
  Toxiproxy.populate([{
46
- "name": "redis",
47
- "listen": "127.0.0.1:6380",
48
- "upstream": "127.0.0.1:6379"
44
+ name: "redis",
45
+ listen: "127.0.0.1:6380",
46
+ upstream: "127.0.0.1:6379"
49
47
  }])
50
48
 
51
49
  self_read, self_write = IO.pipe
52
- %w(INT TERM TSTP TTIN).each do |sig|
53
- begin
54
- trap sig do
55
- self_write.puts(sig)
56
- end
57
- rescue ArgumentError
58
- puts "Signal #{sig} not supported"
50
+ %w[INT TERM TSTP TTIN].each do |sig|
51
+ trap sig do
52
+ self_write.puts(sig)
59
53
  end
54
+ rescue ArgumentError
55
+ puts "Signal #{sig} not supported"
60
56
  end
61
57
 
62
- Sidekiq.redis {|c| c.flushdb}
58
+ Sidekiq.redis { |c| c.flushdb }
63
59
  def handle_signal(launcher, sig)
64
60
  Sidekiq.logger.debug "Got #{sig} signal"
65
61
  case sig
66
- when 'INT'
62
+ when "INT"
67
63
  # Handle Ctrl-C in JRuby like MRI
68
64
  # http://jira.codehaus.org/browse/JRUBY-4637
69
65
  raise Interrupt
70
- when 'TERM'
66
+ when "TERM"
71
67
  # Heroku sends TERM and then waits 30 seconds for process to exit.
72
68
  raise Interrupt
73
- when 'TSTP'
69
+ when "TSTP"
74
70
  Sidekiq.logger.info "Received TSTP, no longer accepting new work"
75
71
  launcher.quiet
76
- when 'TTIN'
72
+ when "TTIN"
77
73
  Thread.list.each do |thread|
78
- Sidekiq.logger.warn "Thread TID-#{(thread.object_id ^ ::Process.pid).to_s(36)} #{thread['label']}"
74
+ Sidekiq.logger.warn "Thread TID-#{(thread.object_id ^ ::Process.pid).to_s(36)} #{thread["label"]}"
79
75
  if thread.backtrace
80
76
  Sidekiq.logger.warn thread.backtrace.join("\n")
81
77
  else
@@ -89,50 +85,46 @@ def Process.rss
89
85
  `ps -o rss= -p #{Process.pid}`.chomp.to_i
90
86
  end
91
87
 
92
- iter = 10
88
+ iter = 50
93
89
  count = 10_000
94
90
 
95
91
  iter.times do
96
- arr = Array.new(count) do
97
- []
98
- end
99
- count.times do |idx|
100
- arr[idx][0] = idx
101
- end
102
- Sidekiq::Client.push_bulk('class' => LoadWorker, 'args' => arr)
92
+ arr = Array.new(count) { |idx| [idx] }
93
+ Sidekiq::Client.push_bulk("class" => LoadWorker, "args" => arr)
103
94
  end
104
- Sidekiq.logger.error "Created #{count*iter} jobs"
95
+ Sidekiq.logger.error "Created #{count * iter} jobs"
105
96
 
106
97
  start = Time.now
107
98
 
108
99
  Monitoring = Thread.new do
109
- watchdog("monitor thread") do
110
- while true
100
+ while true
101
+ sleep 0.2
102
+ qsize = Sidekiq.redis do |conn|
103
+ conn.llen "queue:default"
104
+ end
105
+ total = qsize
106
+ # Sidekiq.logger.error("RSS: #{Process.rss} Pending: #{total}")
107
+ if total == 0
108
+ Sidekiq.logger.error("Done, #{iter * count} jobs in #{Time.now - start} sec")
109
+ Sidekiq.logger.error("Now here's the latency for three jobs")
110
+
111
+ LoadWorker.perform_async(1, Time.now.to_f)
112
+ LoadWorker.perform_async(2, Time.now.to_f)
113
+ LoadWorker.perform_async(3, Time.now.to_f)
114
+
111
115
  sleep 0.2
112
- qsize = Sidekiq.redis do |conn|
113
- conn.llen "queue:default"
114
- end
115
- total = qsize
116
- #Sidekiq.logger.error("RSS: #{Process.rss} Pending: #{total}")
117
- if total == 0
118
- Sidekiq.logger.error("Done, #{iter * count} jobs in #{Time.now - start} sec")
119
- Sidekiq.logger.error("Now here's the latency for three jobs")
120
-
121
- LoadWorker.perform_async(1, Time.now.to_f)
122
- LoadWorker.perform_async(2, Time.now.to_f)
123
- LoadWorker.perform_async(3, Time.now.to_f)
124
-
125
- sleep 0.2
126
- exit(0)
127
- end
116
+ exit(0)
128
117
  end
129
118
  end
130
119
  end
131
120
 
132
121
  begin
133
- #RubyProf::exclude_threads = [ Monitoring ]
134
- #RubyProf.start
135
- fire_event(:startup)
122
+ # RubyProf::exclude_threads = [ Monitoring ]
123
+ # RubyProf.start
124
+ events = Sidekiq.options[:lifecycle_events][:startup]
125
+ events.each(&:call)
126
+ events.clear
127
+
136
128
  Sidekiq.logger.error "Simulating 1ms of latency between Sidekiq and redis"
137
129
  Toxiproxy[:redis].downstream(:latency, latency: 1).apply do
138
130
  launcher = Sidekiq::Launcher.new(Sidekiq.options)
@@ -144,14 +136,14 @@ begin
144
136
  end
145
137
  end
146
138
  rescue SystemExit => e
147
- #Sidekiq.logger.error("Profiling...")
148
- #result = RubyProf.stop
149
- #printer = RubyProf::GraphHtmlPrinter.new(result)
150
- #printer.print(File.new("output.html", "w"), :min_percent => 1)
139
+ # Sidekiq.logger.error("Profiling...")
140
+ # result = RubyProf.stop
141
+ # printer = RubyProf::GraphHtmlPrinter.new(result)
142
+ # printer.print(File.new("output.html", "w"), :min_percent => 1)
151
143
  # normal
152
144
  rescue => e
153
145
  raise e if $DEBUG
154
- STDERR.puts e.message
155
- STDERR.puts e.backtrace.join("\n")
146
+ warn e.message
147
+ warn e.backtrace.join("\n")
156
148
  exit 1
157
149
  end
data/bin/sidekiqmon CHANGED
@@ -1,6 +1,6 @@
1
1
  #!/usr/bin/env ruby
2
2
 
3
- require 'sidekiq/monitor'
3
+ require "sidekiq/monitor"
4
4
 
5
5
  section = "all"
6
6
  section = ARGV[0] if ARGV.size == 1
@@ -0,0 +1,57 @@
1
+ require "rails/generators/named_base"
2
+
3
+ module Sidekiq
4
+ module Generators # :nodoc:
5
+ class JobGenerator < ::Rails::Generators::NamedBase # :nodoc:
6
+ desc "This generator creates a Sidekiq Job in app/sidekiq and a corresponding test"
7
+
8
+ check_class_collision suffix: "Job"
9
+
10
+ def self.default_generator_root
11
+ File.dirname(__FILE__)
12
+ end
13
+
14
+ def create_job_file
15
+ template "job.rb.erb", File.join("app/sidekiq", class_path, "#{file_name}_job.rb")
16
+ end
17
+
18
+ def create_test_file
19
+ return unless test_framework
20
+
21
+ if test_framework == :rspec
22
+ create_job_spec
23
+ else
24
+ create_job_test
25
+ end
26
+ end
27
+
28
+ private
29
+
30
+ def create_job_spec
31
+ template_file = File.join(
32
+ "spec/sidekiq",
33
+ class_path,
34
+ "#{file_name}_job_spec.rb"
35
+ )
36
+ template "job_spec.rb.erb", template_file
37
+ end
38
+
39
+ def create_job_test
40
+ template_file = File.join(
41
+ "test/sidekiq",
42
+ class_path,
43
+ "#{file_name}_job_test.rb"
44
+ )
45
+ template "job_test.rb.erb", template_file
46
+ end
47
+
48
+ def file_name
49
+ @_file_name ||= super.sub(/_?job\z/i, "")
50
+ end
51
+
52
+ def test_framework
53
+ ::Rails.application.config.generators.options[:rails][:test_framework]
54
+ end
55
+ end
56
+ end
57
+ end
@@ -1,6 +1,6 @@
1
1
  <% module_namespacing do -%>
2
- class <%= class_name %>Worker
3
- include Sidekiq::Worker
2
+ class <%= class_name %>Job
3
+ include Sidekiq::Job
4
4
 
5
5
  def perform(*args)
6
6
  # Do something
@@ -1,6 +1,6 @@
1
1
  require 'rails_helper'
2
2
  <% module_namespacing do -%>
3
- RSpec.describe <%= class_name %>Worker, type: :worker do
3
+ RSpec.describe <%= class_name %>Job, type: :job do
4
4
  pending "add some examples to (or delete) #{__FILE__}"
5
5
  end
6
6
  <% end -%>
@@ -1,6 +1,6 @@
1
1
  require 'test_helper'
2
2
  <% module_namespacing do -%>
3
- class <%= class_name %>WorkerTest < Minitest::Test
3
+ class <%= class_name %>JobTest < Minitest::Test
4
4
  def test_example
5
5
  skip "add some examples to (or delete) #{__FILE__}"
6
6
  end
data/lib/sidekiq/api.rb CHANGED
@@ -54,14 +54,14 @@ module Sidekiq
54
54
  # O(1) redis calls
55
55
  def fetch_stats_fast!
56
56
  pipe1_res = Sidekiq.redis { |conn|
57
- conn.pipelined do
58
- conn.get("stat:processed")
59
- conn.get("stat:failed")
60
- conn.zcard("schedule")
61
- conn.zcard("retry")
62
- conn.zcard("dead")
63
- conn.scard("processes")
64
- conn.lrange("queue:default", -1, -1)
57
+ conn.pipelined do |pipeline|
58
+ pipeline.get("stat:processed")
59
+ pipeline.get("stat:failed")
60
+ pipeline.zcard("schedule")
61
+ pipeline.zcard("retry")
62
+ pipeline.zcard("dead")
63
+ pipeline.scard("processes")
64
+ pipeline.lrange("queue:default", -1, -1)
65
65
  end
66
66
  }
67
67
 
@@ -101,9 +101,9 @@ module Sidekiq
101
101
  }
102
102
 
103
103
  pipe2_res = Sidekiq.redis { |conn|
104
- conn.pipelined do
105
- processes.each { |key| conn.hget(key, "busy") }
106
- queues.each { |queue| conn.llen("queue:#{queue}") }
104
+ conn.pipelined do |pipeline|
105
+ processes.each { |key| pipeline.hget(key, "busy") }
106
+ queues.each { |queue| pipeline.llen("queue:#{queue}") }
107
107
  end
108
108
  }
109
109
 
@@ -147,9 +147,9 @@ module Sidekiq
147
147
  Sidekiq.redis do |conn|
148
148
  queues = conn.sscan_each("queues").to_a
149
149
 
150
- lengths = conn.pipelined {
150
+ lengths = conn.pipelined { |pipeline|
151
151
  queues.each do |queue|
152
- conn.llen("queue:#{queue}")
152
+ pipeline.llen("queue:#{queue}")
153
153
  end
154
154
  }
155
155
 
@@ -161,6 +161,8 @@ module Sidekiq
161
161
 
162
162
  class History
163
163
  def initialize(days_previous, start_date = nil)
164
+ # we only store five years of data in Redis
165
+ raise ArgumentError if days_previous < 1 || days_previous > (5 * 365)
164
166
  @days_previous = days_previous
165
167
  @start_date = start_date || Time.now.utc.to_date
166
168
  end
@@ -285,9 +287,9 @@ module Sidekiq
285
287
 
286
288
  def clear
287
289
  Sidekiq.redis do |conn|
288
- conn.multi do
289
- conn.unlink(@rname)
290
- conn.srem("queues", name)
290
+ conn.multi do |transaction|
291
+ transaction.unlink(@rname)
292
+ transaction.srem("queues", name)
291
293
  end
292
294
  end
293
295
  end
@@ -352,27 +354,31 @@ module Sidekiq
352
354
  def display_args
353
355
  # Unwrap known wrappers so they show up in a human-friendly manner in the Web UI
354
356
  @display_args ||= case klass
355
- when /\ASidekiq::Extensions::Delayed/
356
- safe_load(args[0], args) do |_, _, arg|
357
- arg
358
- end
359
- when "ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper"
360
- job_args = self["wrapped"] ? args[0]["arguments"] : []
361
- if (self["wrapped"] || args[0]) == "ActionMailer::DeliveryJob"
362
- # remove MailerClass, mailer_method and 'deliver_now'
363
- job_args.drop(3)
364
- elsif (self["wrapped"] || args[0]) == "ActionMailer::MailDeliveryJob"
365
- # remove MailerClass, mailer_method and 'deliver_now'
366
- job_args.drop(3).first["args"]
367
- else
368
- job_args
369
- end
370
- else
371
- if self["encrypt"]
372
- # no point in showing 150+ bytes of random garbage
373
- args[-1] = "[encrypted data]"
374
- end
375
- args
357
+ when /\ASidekiq::Extensions::Delayed/
358
+ safe_load(args[0], args) do |_, _, arg, kwarg|
359
+ if !kwarg || kwarg.empty?
360
+ arg
361
+ else
362
+ [arg, kwarg]
363
+ end
364
+ end
365
+ when "ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper"
366
+ job_args = self["wrapped"] ? args[0]["arguments"] : []
367
+ if (self["wrapped"] || args[0]) == "ActionMailer::DeliveryJob"
368
+ # remove MailerClass, mailer_method and 'deliver_now'
369
+ job_args.drop(3)
370
+ elsif (self["wrapped"] || args[0]) == "ActionMailer::MailDeliveryJob"
371
+ # remove MailerClass, mailer_method and 'deliver_now'
372
+ job_args.drop(3).first["args"]
373
+ else
374
+ job_args
375
+ end
376
+ else
377
+ if self["encrypt"]
378
+ # no point in showing 150+ bytes of random garbage
379
+ args[-1] = "[encrypted data]"
380
+ end
381
+ args
376
382
  end
377
383
  end
378
384
 
@@ -517,9 +523,9 @@ module Sidekiq
517
523
 
518
524
  def remove_job
519
525
  Sidekiq.redis do |conn|
520
- results = conn.multi {
521
- conn.zrangebyscore(parent.name, score, score)
522
- conn.zremrangebyscore(parent.name, score, score)
526
+ results = conn.multi { |transaction|
527
+ transaction.zrangebyscore(parent.name, score, score)
528
+ transaction.zremrangebyscore(parent.name, score, score)
523
529
  }.first
524
530
 
525
531
  if results.size == 1
@@ -540,9 +546,9 @@ module Sidekiq
540
546
  yield msg if msg
541
547
 
542
548
  # push the rest back onto the sorted set
543
- conn.multi do
549
+ conn.multi do |transaction|
544
550
  nonmatched.each do |message|
545
- conn.zadd(parent.name, score.to_f.to_s, message)
551
+ transaction.zadd(parent.name, score.to_f.to_s, message)
546
552
  end
547
553
  end
548
554
  end
@@ -729,10 +735,10 @@ module Sidekiq
729
735
  def kill(message, opts = {})
730
736
  now = Time.now.to_f
731
737
  Sidekiq.redis do |conn|
732
- conn.multi do
733
- conn.zadd(name, now.to_s, message)
734
- conn.zremrangebyscore(name, "-inf", now - self.class.timeout)
735
- conn.zremrangebyrank(name, 0, - self.class.max_jobs)
738
+ conn.multi do |transaction|
739
+ transaction.zadd(name, now.to_s, message)
740
+ transaction.zremrangebyscore(name, "-inf", now - self.class.timeout)
741
+ transaction.zremrangebyrank(name, 0, - self.class.max_jobs)
736
742
  end
737
743
  end
738
744
 
@@ -780,9 +786,9 @@ module Sidekiq
780
786
  count = 0
781
787
  Sidekiq.redis do |conn|
782
788
  procs = conn.sscan_each("processes").to_a.sort
783
- heartbeats = conn.pipelined {
789
+ heartbeats = conn.pipelined { |pipeline|
784
790
  procs.each do |key|
785
- conn.hget(key, "info")
791
+ pipeline.hget(key, "info")
786
792
  end
787
793
  }
788
794
 
@@ -804,9 +810,9 @@ module Sidekiq
804
810
  # We're making a tradeoff here between consuming more memory instead of
805
811
  # making more roundtrips to Redis, but if you have hundreds or thousands of workers,
806
812
  # you'll be happier this way
807
- conn.pipelined do
813
+ conn.pipelined do |pipeline|
808
814
  procs.each do |key|
809
- conn.hmget(key, "info", "busy", "beat", "quiet", "rss", "rtt_us")
815
+ pipeline.hmget(key, "info", "busy", "beat", "quiet", "rss", "rtt_us")
810
816
  end
811
817
  end
812
818
  }
@@ -819,10 +825,10 @@ module Sidekiq
819
825
 
820
826
  hash = Sidekiq.load_json(info)
821
827
  yield Process.new(hash.merge("busy" => busy.to_i,
822
- "beat" => at_s.to_f,
823
- "quiet" => quiet,
824
- "rss" => rss.to_i,
825
- "rtt_us" => rtt.to_i))
828
+ "beat" => at_s.to_f,
829
+ "quiet" => quiet,
830
+ "rss" => rss.to_i,
831
+ "rtt_us" => rtt.to_i))
826
832
  end
827
833
  end
828
834
 
@@ -920,9 +926,9 @@ module Sidekiq
920
926
  def signal(sig)
921
927
  key = "#{identity}-signals"
922
928
  Sidekiq.redis do |c|
923
- c.multi do
924
- c.lpush(key, sig)
925
- c.expire(key, 60)
929
+ c.multi do |transaction|
930
+ transaction.lpush(key, sig)
931
+ transaction.expire(key, 60)
926
932
  end
927
933
  end
928
934
  end
@@ -956,9 +962,9 @@ module Sidekiq
956
962
  Sidekiq.redis do |conn|
957
963
  procs = conn.sscan_each("processes").to_a
958
964
  procs.sort.each do |key|
959
- valid, workers = conn.pipelined {
960
- conn.exists?(key)
961
- conn.hgetall("#{key}:workers")
965
+ valid, workers = conn.pipelined { |pipeline|
966
+ pipeline.exists?(key)
967
+ pipeline.hgetall("#{key}:work")
962
968
  }
963
969
  next unless valid
964
970
  workers.each_pair do |tid, json|
@@ -986,9 +992,9 @@ module Sidekiq
986
992
  if procs.empty?
987
993
  0
988
994
  else
989
- conn.pipelined {
995
+ conn.pipelined { |pipeline|
990
996
  procs.each do |key|
991
- conn.hget(key, "busy")
997
+ pipeline.hget(key, "busy")
992
998
  end
993
999
  }.sum(&:to_i)
994
1000
  end