sidekiq 5.2.6

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of sidekiq might be problematic. Click here for more details.

Files changed (118) hide show
  1. checksums.yaml +7 -0
  2. data/.github/contributing.md +32 -0
  3. data/.github/issue_template.md +11 -0
  4. data/.gitignore +15 -0
  5. data/.travis.yml +11 -0
  6. data/3.0-Upgrade.md +70 -0
  7. data/4.0-Upgrade.md +53 -0
  8. data/5.0-Upgrade.md +56 -0
  9. data/COMM-LICENSE +97 -0
  10. data/Changes.md +1529 -0
  11. data/Ent-Changes.md +238 -0
  12. data/Gemfile +23 -0
  13. data/LICENSE +9 -0
  14. data/Pro-2.0-Upgrade.md +138 -0
  15. data/Pro-3.0-Upgrade.md +44 -0
  16. data/Pro-4.0-Upgrade.md +35 -0
  17. data/Pro-Changes.md +759 -0
  18. data/README.md +109 -0
  19. data/Rakefile +9 -0
  20. data/bin/sidekiq +18 -0
  21. data/bin/sidekiqctl +20 -0
  22. data/bin/sidekiqload +149 -0
  23. data/code_of_conduct.md +50 -0
  24. data/lib/generators/sidekiq/templates/worker.rb.erb +9 -0
  25. data/lib/generators/sidekiq/templates/worker_spec.rb.erb +6 -0
  26. data/lib/generators/sidekiq/templates/worker_test.rb.erb +8 -0
  27. data/lib/generators/sidekiq/worker_generator.rb +49 -0
  28. data/lib/sidekiq.rb +237 -0
  29. data/lib/sidekiq/api.rb +940 -0
  30. data/lib/sidekiq/cli.rb +445 -0
  31. data/lib/sidekiq/client.rb +243 -0
  32. data/lib/sidekiq/core_ext.rb +1 -0
  33. data/lib/sidekiq/ctl.rb +221 -0
  34. data/lib/sidekiq/delay.rb +42 -0
  35. data/lib/sidekiq/exception_handler.rb +29 -0
  36. data/lib/sidekiq/extensions/action_mailer.rb +57 -0
  37. data/lib/sidekiq/extensions/active_record.rb +40 -0
  38. data/lib/sidekiq/extensions/class_methods.rb +40 -0
  39. data/lib/sidekiq/extensions/generic_proxy.rb +31 -0
  40. data/lib/sidekiq/fetch.rb +81 -0
  41. data/lib/sidekiq/job_logger.rb +25 -0
  42. data/lib/sidekiq/job_retry.rb +262 -0
  43. data/lib/sidekiq/launcher.rb +173 -0
  44. data/lib/sidekiq/logging.rb +122 -0
  45. data/lib/sidekiq/manager.rb +137 -0
  46. data/lib/sidekiq/middleware/chain.rb +150 -0
  47. data/lib/sidekiq/middleware/i18n.rb +42 -0
  48. data/lib/sidekiq/middleware/server/active_record.rb +23 -0
  49. data/lib/sidekiq/paginator.rb +43 -0
  50. data/lib/sidekiq/processor.rb +279 -0
  51. data/lib/sidekiq/rails.rb +58 -0
  52. data/lib/sidekiq/redis_connection.rb +144 -0
  53. data/lib/sidekiq/scheduled.rb +174 -0
  54. data/lib/sidekiq/testing.rb +333 -0
  55. data/lib/sidekiq/testing/inline.rb +29 -0
  56. data/lib/sidekiq/util.rb +66 -0
  57. data/lib/sidekiq/version.rb +4 -0
  58. data/lib/sidekiq/web.rb +213 -0
  59. data/lib/sidekiq/web/action.rb +89 -0
  60. data/lib/sidekiq/web/application.rb +353 -0
  61. data/lib/sidekiq/web/helpers.rb +325 -0
  62. data/lib/sidekiq/web/router.rb +100 -0
  63. data/lib/sidekiq/worker.rb +220 -0
  64. data/sidekiq.gemspec +21 -0
  65. data/web/assets/images/favicon.ico +0 -0
  66. data/web/assets/images/logo.png +0 -0
  67. data/web/assets/images/status.png +0 -0
  68. data/web/assets/javascripts/application.js +92 -0
  69. data/web/assets/javascripts/dashboard.js +315 -0
  70. data/web/assets/stylesheets/application-rtl.css +246 -0
  71. data/web/assets/stylesheets/application.css +1144 -0
  72. data/web/assets/stylesheets/bootstrap-rtl.min.css +9 -0
  73. data/web/assets/stylesheets/bootstrap.css +5 -0
  74. data/web/locales/ar.yml +81 -0
  75. data/web/locales/cs.yml +78 -0
  76. data/web/locales/da.yml +68 -0
  77. data/web/locales/de.yml +69 -0
  78. data/web/locales/el.yml +68 -0
  79. data/web/locales/en.yml +81 -0
  80. data/web/locales/es.yml +70 -0
  81. data/web/locales/fa.yml +80 -0
  82. data/web/locales/fr.yml +78 -0
  83. data/web/locales/he.yml +79 -0
  84. data/web/locales/hi.yml +75 -0
  85. data/web/locales/it.yml +69 -0
  86. data/web/locales/ja.yml +80 -0
  87. data/web/locales/ko.yml +68 -0
  88. data/web/locales/nb.yml +77 -0
  89. data/web/locales/nl.yml +68 -0
  90. data/web/locales/pl.yml +59 -0
  91. data/web/locales/pt-br.yml +68 -0
  92. data/web/locales/pt.yml +67 -0
  93. data/web/locales/ru.yml +78 -0
  94. data/web/locales/sv.yml +68 -0
  95. data/web/locales/ta.yml +75 -0
  96. data/web/locales/uk.yml +76 -0
  97. data/web/locales/ur.yml +80 -0
  98. data/web/locales/zh-cn.yml +68 -0
  99. data/web/locales/zh-tw.yml +68 -0
  100. data/web/views/_footer.erb +20 -0
  101. data/web/views/_job_info.erb +88 -0
  102. data/web/views/_nav.erb +52 -0
  103. data/web/views/_paging.erb +23 -0
  104. data/web/views/_poll_link.erb +7 -0
  105. data/web/views/_status.erb +4 -0
  106. data/web/views/_summary.erb +40 -0
  107. data/web/views/busy.erb +98 -0
  108. data/web/views/dashboard.erb +75 -0
  109. data/web/views/dead.erb +34 -0
  110. data/web/views/layout.erb +40 -0
  111. data/web/views/morgue.erb +75 -0
  112. data/web/views/queue.erb +46 -0
  113. data/web/views/queues.erb +30 -0
  114. data/web/views/retries.erb +80 -0
  115. data/web/views/retry.erb +34 -0
  116. data/web/views/scheduled.erb +54 -0
  117. data/web/views/scheduled_job_info.erb +8 -0
  118. metadata +229 -0
@@ -0,0 +1,109 @@
1
+ Sidekiq
2
+ ==============
3
+
4
+ [![Gem Version](https://badge.fury.io/rb/sidekiq.svg)](https://rubygems.org/gems/sidekiq)
5
+ [![Code Climate](https://codeclimate.com/github/mperham/sidekiq.svg)](https://codeclimate.com/github/mperham/sidekiq)
6
+ [![Build Status](https://travis-ci.org/mperham/sidekiq.svg)](https://travis-ci.org/mperham/sidekiq)
7
+ [![Gitter Chat](https://badges.gitter.im/mperham/sidekiq.svg)](https://gitter.im/mperham/sidekiq)
8
+
9
+
10
+ Simple, efficient background processing for Ruby.
11
+
12
+ Sidekiq uses threads to handle many jobs at the same time in the
13
+ same process. It does not require Rails but will integrate tightly with
14
+ Rails to make background processing dead simple.
15
+
16
+ Sidekiq is compatible with Resque. It uses the exact same
17
+ message format as Resque so it can integrate into an existing Resque processing farm.
18
+ You can have Sidekiq and Resque run side-by-side at the same time and
19
+ use the Resque client to enqueue jobs in Redis to be processed by Sidekiq.
20
+
21
+ Performance
22
+ ---------------
23
+
24
+ Version | Latency | Garbage created for 10,000 jobs | Time to process 100,000 jobs | Throughput
25
+ -----------------|------|---------|---------|------------------------
26
+ Sidekiq 4.0.0 | 10ms | 151 MB | 22 sec | **4500 jobs/sec**
27
+ Sidekiq 3.5.1 | 22ms | 1257 MB | 125 sec | 800 jobs/sec
28
+ Resque 1.25.2 | - | - | 420 sec | 240 jobs/sec
29
+ DelayedJob 4.1.1 | - | - | 465 sec | 215 jobs/sec
30
+
31
+ <small>This benchmark can be found in `bin/sidekiqload`.</small>
32
+
33
+ Requirements
34
+ -----------------
35
+
36
+ Sidekiq supports CRuby 2.2.2+ and JRuby 9k.
37
+
38
+ All Rails releases >= 4.0 are officially supported.
39
+
40
+ Redis 2.8 or greater is required. 3.0.3+ is recommended for large
41
+ installations with thousands of worker threads.
42
+
43
+
44
+ Installation
45
+ -----------------
46
+
47
+ gem install sidekiq
48
+
49
+
50
+ Getting Started
51
+ -----------------
52
+
53
+ See the [Getting Started wiki page](https://github.com/mperham/sidekiq/wiki/Getting-Started) and follow the simple setup process.
54
+ You can watch [this Youtube playlist](https://www.youtube.com/playlist?list=PLjeHh2LSCFrWGT5uVjUuFKAcrcj5kSai1) to learn all about
55
+ Sidekiq and see its features in action. Here's the Web UI:
56
+
57
+ ![Web UI](https://github.com/mperham/sidekiq/raw/master/examples/web-ui.png)
58
+
59
+
60
+ Want to Upgrade?
61
+ -------------------
62
+
63
+ I also sell Sidekiq Pro and Sidekiq Enterprise, extensions to Sidekiq which provide more
64
+ features, a commercial-friendly license and allow you to support high
65
+ quality open source development all at the same time. Please see the
66
+ [Sidekiq](http://sidekiq.org/) homepage for more detail.
67
+
68
+ Subscribe to the **[quarterly newsletter](https://tinyletter.com/sidekiq)** to stay informed about the latest
69
+ features and changes to Sidekiq and its bigger siblings.
70
+
71
+
72
+ Problems?
73
+ -----------------
74
+
75
+ **Please do not directly email any Sidekiq committers with questions or problems.** A community is best served when discussions are held in public.
76
+
77
+ If you have a problem, please review the [FAQ](https://github.com/mperham/sidekiq/wiki/FAQ) and [Troubleshooting](https://github.com/mperham/sidekiq/wiki/Problems-and-Troubleshooting) wiki pages.
78
+ Searching the [issues](https://github.com/mperham/sidekiq/issues) for your problem is also a good idea.
79
+
80
+ Sidekiq Pro and Sidekiq Enterprise customers get private email support. You can purchase at http://sidekiq.org; email support@contribsys.com for help.
81
+
82
+ Useful resources:
83
+
84
+ * Product documentation is in the [wiki](https://github.com/mperham/sidekiq/wiki).
85
+ * Release announcements are made to the [@sidekiq](https://twitter.com/sidekiq) Twitter account.
86
+ * The [Sidekiq tag](https://stackoverflow.com/questions/tagged/sidekiq) on Stack Overflow has lots of useful Q &amp; A.
87
+
88
+ **No support via Twitter**
89
+
90
+ Every Friday morning is Sidekiq happy hour: I video chat and answer questions.
91
+ See the [Sidekiq support page](http://sidekiq.org/support.html) for details.
92
+
93
+ Thanks
94
+ -----------------
95
+
96
+ Sidekiq stays fast by using the [JProfiler java profiler](http://www.ej-technologies.com/products/jprofiler/overview.html) to find and fix
97
+ performance problems on JRuby. Unfortunately MRI does not have good multithreaded profiling tools.
98
+
99
+
100
+ License
101
+ -----------------
102
+
103
+ Please see [LICENSE](https://github.com/mperham/sidekiq/blob/master/LICENSE) for licensing details.
104
+
105
+
106
+ Author
107
+ -----------------
108
+
109
+ Mike Perham, [@mperham](https://twitter.com/mperham) / [@sidekiq](https://twitter.com/sidekiq), [http://www.mikeperham.com](http://www.mikeperham.com) / [http://www.contribsys.com](http://www.contribsys.com)
@@ -0,0 +1,9 @@
1
+ require 'bundler/gem_tasks'
2
+ require 'rake/testtask'
3
+
4
+ Rake::TestTask.new(:test) do |test|
5
+ test.warning = true
6
+ test.pattern = 'test/**/test_*.rb'
7
+ end
8
+
9
+ task default: :test
@@ -0,0 +1,18 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ # Quiet some warnings we see when running in warning mode:
4
+ # RUBYOPT=-w bundle exec sidekiq
5
+ $TESTING = false
6
+
7
+ require_relative '../lib/sidekiq/cli'
8
+
9
+ begin
10
+ cli = Sidekiq::CLI.instance
11
+ cli.parse
12
+ cli.run
13
+ rescue => e
14
+ raise e if $DEBUG
15
+ STDERR.puts e.message
16
+ STDERR.puts e.backtrace.join("\n")
17
+ exit 1
18
+ end
@@ -0,0 +1,20 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require 'fileutils'
4
+ require 'sidekiq/api'
5
+ require 'sidekiq/ctl'
6
+
7
+ if ARGV[0] == 'status'
8
+ Sidekiq::Ctl::Status.new.display(ARGV[1])
9
+ else
10
+ if ARGV.length < 2
11
+ Sidekiq::Ctl.print_usage
12
+ else
13
+ stage = ARGV[0]
14
+ pidfile = ARGV[1]
15
+ timeout = ARGV[2].to_i
16
+ timeout = Sidekiq::Ctl::DEFAULT_KILL_TIMEOUT if timeout == 0
17
+
18
+ Sidekiq::Ctl.new(stage, pidfile, timeout)
19
+ end
20
+ end
@@ -0,0 +1,149 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ # Quiet some warnings we see when running in warning mode:
4
+ # RUBYOPT=-w bundle exec sidekiq
5
+ $TESTING = false
6
+
7
+ #require 'ruby-prof'
8
+ Bundler.require(:default)
9
+
10
+ require_relative '../lib/sidekiq/cli'
11
+ require_relative '../lib/sidekiq/launcher'
12
+
13
+ include Sidekiq::Util
14
+
15
+ Sidekiq.configure_server do |config|
16
+ #config.options[:concurrency] = 1
17
+ config.redis = { db: 13 }
18
+ config.options[:queues] << 'default'
19
+ config.logger.level = Logger::ERROR
20
+ config.average_scheduled_poll_interval = 2
21
+ config.reliable! if defined?(Sidekiq::Pro)
22
+ end
23
+
24
+ class LoadWorker
25
+ include Sidekiq::Worker
26
+ sidekiq_options retry: 1
27
+ sidekiq_retry_in do |x|
28
+ 1
29
+ end
30
+
31
+ def perform(idx)
32
+ #raise idx.to_s if idx % 100 == 1
33
+ end
34
+ end
35
+
36
+ # brew tap shopify/shopify
37
+ # brew install toxiproxy
38
+ # gem install toxiproxy
39
+ #require 'toxiproxy'
40
+ # simulate a non-localhost network for realer-world conditions.
41
+ # adding 1ms of network latency has an ENORMOUS impact on benchmarks
42
+ #Toxiproxy.populate([{
43
+ #"name": "redis",
44
+ #"listen": "127.0.0.1:6380",
45
+ #"upstream": "127.0.0.1:6379"
46
+ #}])
47
+
48
+ self_read, self_write = IO.pipe
49
+ %w(INT TERM TSTP TTIN).each do |sig|
50
+ begin
51
+ trap sig do
52
+ self_write.puts(sig)
53
+ end
54
+ rescue ArgumentError
55
+ puts "Signal #{sig} not supported"
56
+ end
57
+ end
58
+
59
+ Sidekiq.redis {|c| c.flushdb}
60
+ def handle_signal(launcher, sig)
61
+ Sidekiq.logger.debug "Got #{sig} signal"
62
+ case sig
63
+ when 'INT'
64
+ # Handle Ctrl-C in JRuby like MRI
65
+ # http://jira.codehaus.org/browse/JRUBY-4637
66
+ raise Interrupt
67
+ when 'TERM'
68
+ # Heroku sends TERM and then waits 30 seconds for process to exit.
69
+ raise Interrupt
70
+ when 'TSTP'
71
+ Sidekiq.logger.info "Received TSTP, no longer accepting new work"
72
+ launcher.quiet
73
+ when 'TTIN'
74
+ Thread.list.each do |thread|
75
+ Sidekiq.logger.warn "Thread TID-#{(thread.object_id ^ ::Process.pid).to_s(36)} #{thread['label']}"
76
+ if thread.backtrace
77
+ Sidekiq.logger.warn thread.backtrace.join("\n")
78
+ else
79
+ Sidekiq.logger.warn "<no backtrace available>"
80
+ end
81
+ end
82
+ end
83
+ end
84
+
85
+ def Process.rss
86
+ `ps -o rss= -p #{Process.pid}`.chomp.to_i
87
+ end
88
+
89
+ iter = 10
90
+ count = 10_000
91
+
92
+ iter.times do
93
+ arr = Array.new(count) do
94
+ []
95
+ end
96
+ count.times do |idx|
97
+ arr[idx][0] = idx
98
+ end
99
+ Sidekiq::Client.push_bulk('class' => LoadWorker, 'args' => arr)
100
+ end
101
+ Sidekiq.logger.error "Created #{count*iter} jobs"
102
+
103
+ Monitoring = Thread.new do
104
+ watchdog("monitor thread") do
105
+ while true
106
+ sleep 1
107
+ qsize, retries = Sidekiq.redis do |conn|
108
+ conn.pipelined do
109
+ conn.llen "queue:default"
110
+ conn.zcard "retry"
111
+ end
112
+ end.map(&:to_i)
113
+ total = qsize + retries
114
+ #GC.start
115
+ Sidekiq.logger.error("RSS: #{Process.rss} Pending: #{total}")
116
+ if total == 0
117
+ Sidekiq.logger.error("Done")
118
+ exit(0)
119
+ end
120
+ end
121
+ end
122
+ end
123
+
124
+ begin
125
+ #RubyProf::exclude_threads = [ Monitoring ]
126
+ #RubyProf.start
127
+ fire_event(:startup)
128
+ #Sidekiq.logger.error "Simulating 1ms of latency between Sidekiq and redis"
129
+ #Toxiproxy[:redis].downstream(:latency, latency: 1).apply do
130
+ launcher = Sidekiq::Launcher.new(Sidekiq.options)
131
+ launcher.run
132
+
133
+ while readable_io = IO.select([self_read])
134
+ signal = readable_io.first[0].gets.strip
135
+ handle_signal(launcher, signal)
136
+ end
137
+ #end
138
+ rescue SystemExit => e
139
+ #Sidekiq.logger.error("Profiling...")
140
+ #result = RubyProf.stop
141
+ #printer = RubyProf::GraphHtmlPrinter.new(result)
142
+ #printer.print(File.new("output.html", "w"), :min_percent => 1)
143
+ # normal
144
+ rescue => e
145
+ raise e if $DEBUG
146
+ STDERR.puts e.message
147
+ STDERR.puts e.backtrace.join("\n")
148
+ exit 1
149
+ end
@@ -0,0 +1,50 @@
1
+ # Contributor Code of Conduct
2
+
3
+ As contributors and maintainers of this project, and in the interest of
4
+ fostering an open and welcoming community, we pledge to respect all people who
5
+ contribute through reporting issues, posting feature requests, updating
6
+ documentation, submitting pull requests or patches, and other activities.
7
+
8
+ We are committed to making participation in this project a harassment-free
9
+ experience for everyone, regardless of level of experience, gender, gender
10
+ identity and expression, sexual orientation, disability, personal appearance,
11
+ body size, race, ethnicity, age, religion, or nationality.
12
+
13
+ Examples of unacceptable behavior by participants include:
14
+
15
+ * The use of sexualized language or imagery
16
+ * Personal attacks
17
+ * Trolling or insulting/derogatory comments
18
+ * Public or private harassment
19
+ * Publishing other's private information, such as physical or electronic
20
+ addresses, without explicit permission
21
+ * Other unethical or unprofessional conduct
22
+
23
+ Project maintainers have the right and responsibility to remove, edit, or
24
+ reject comments, commits, code, wiki edits, issues, and other contributions
25
+ that are not aligned to this Code of Conduct, or to ban temporarily or
26
+ permanently any contributor for other behaviors that they deem inappropriate,
27
+ threatening, offensive, or harmful.
28
+
29
+ By adopting this Code of Conduct, project maintainers commit themselves to
30
+ fairly and consistently applying these principles to every aspect of managing
31
+ this project. Project maintainers who do not follow or enforce the Code of
32
+ Conduct may be permanently removed from the project team.
33
+
34
+ This Code of Conduct applies both within project spaces and in public spaces
35
+ when an individual is representing the project or its community.
36
+
37
+ Instances of abusive, harassing, or otherwise unacceptable behavior may be
38
+ reported by contacting the project maintainer at mperham AT gmail.com. All
39
+ complaints will be reviewed and investigated and will result in a response that
40
+ is deemed necessary and appropriate to the circumstances. Maintainers are
41
+ obligated to maintain confidentiality with regard to the reporter of an
42
+ incident.
43
+
44
+
45
+ This Code of Conduct is adapted from the [Contributor Covenant][homepage],
46
+ version 1.3.0, available at
47
+ [http://contributor-covenant.org/version/1/3/0/][version]
48
+
49
+ [homepage]: http://contributor-covenant.org
50
+ [version]: http://contributor-covenant.org/version/1/3/0/
@@ -0,0 +1,9 @@
1
+ <% module_namespacing do -%>
2
+ class <%= class_name %>Worker
3
+ include Sidekiq::Worker
4
+
5
+ def perform(*args)
6
+ # Do something
7
+ end
8
+ end
9
+ <% end -%>
@@ -0,0 +1,6 @@
1
+ require 'rails_helper'
2
+ <% module_namespacing do -%>
3
+ RSpec.describe <%= class_name %>Worker, type: :worker do
4
+ pending "add some examples to (or delete) #{__FILE__}"
5
+ end
6
+ <% end -%>
@@ -0,0 +1,8 @@
1
+ require 'test_helper'
2
+ <% module_namespacing do -%>
3
+ class <%= class_name %>WorkerTest < <% if defined? Minitest::Test %>Minitest::Test<% else %>MiniTest::Unit::TestCase<% end %>
4
+ def test_example
5
+ skip "add some examples to (or delete) #{__FILE__}"
6
+ end
7
+ end
8
+ <% end -%>
@@ -0,0 +1,49 @@
1
+ require 'rails/generators/named_base'
2
+
3
+ module Sidekiq
4
+ module Generators # :nodoc:
5
+ class WorkerGenerator < ::Rails::Generators::NamedBase # :nodoc:
6
+ desc 'This generator creates a Sidekiq Worker in app/workers and a corresponding test'
7
+
8
+ check_class_collision suffix: 'Worker'
9
+
10
+ def self.default_generator_root
11
+ File.dirname(__FILE__)
12
+ end
13
+
14
+ def create_worker_file
15
+ template 'worker.rb.erb', File.join('app/workers', class_path, "#{file_name}_worker.rb")
16
+ end
17
+
18
+ def create_test_file
19
+ if defined?(RSpec)
20
+ create_worker_spec
21
+ else
22
+ create_worker_test
23
+ end
24
+ end
25
+
26
+ private
27
+
28
+ def create_worker_spec
29
+ template_file = File.join(
30
+ 'spec/workers',
31
+ class_path,
32
+ "#{file_name}_worker_spec.rb"
33
+ )
34
+ template 'worker_spec.rb.erb', template_file
35
+ end
36
+
37
+ def create_worker_test
38
+ template_file = File.join(
39
+ 'test/workers',
40
+ class_path,
41
+ "#{file_name}_worker_test.rb"
42
+ )
43
+ template 'worker_test.rb.erb', template_file
44
+ end
45
+
46
+
47
+ end
48
+ end
49
+ end
@@ -0,0 +1,237 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'sidekiq/version'
4
+ fail "Sidekiq #{Sidekiq::VERSION} does not support Ruby versions below 2.2.2." if RUBY_PLATFORM != 'java' && Gem::Version.new(RUBY_VERSION) < Gem::Version.new('2.2.2')
5
+
6
+ require 'sidekiq/logging'
7
+ require 'sidekiq/client'
8
+ require 'sidekiq/worker'
9
+ require 'sidekiq/redis_connection'
10
+ require 'sidekiq/delay'
11
+
12
+ require 'json'
13
+
14
+ module Sidekiq
15
+ NAME = 'Sidekiq'
16
+ LICENSE = 'See LICENSE and the LGPL-3.0 for licensing details.'
17
+
18
+ DEFAULTS = {
19
+ queues: [],
20
+ labels: [],
21
+ concurrency: 10,
22
+ require: '.',
23
+ environment: nil,
24
+ timeout: 8,
25
+ poll_interval_average: nil,
26
+ average_scheduled_poll_interval: 5,
27
+ error_handlers: [],
28
+ death_handlers: [],
29
+ lifecycle_events: {
30
+ startup: [],
31
+ quiet: [],
32
+ shutdown: [],
33
+ heartbeat: [],
34
+ },
35
+ dead_max_jobs: 10_000,
36
+ dead_timeout_in_seconds: 180 * 24 * 60 * 60, # 6 months
37
+ reloader: proc { |&block| block.call },
38
+ }
39
+
40
+ DEFAULT_WORKER_OPTIONS = {
41
+ 'retry' => true,
42
+ 'queue' => 'default'
43
+ }
44
+
45
+ FAKE_INFO = {
46
+ "redis_version" => "9.9.9",
47
+ "uptime_in_days" => "9999",
48
+ "connected_clients" => "9999",
49
+ "used_memory_human" => "9P",
50
+ "used_memory_peak_human" => "9P"
51
+ }
52
+
53
+ def self.❨╯°□°❩╯︵┻━┻
54
+ puts "Calm down, yo."
55
+ end
56
+
57
+ def self.options
58
+ @options ||= DEFAULTS.dup
59
+ end
60
+
61
+ def self.options=(opts)
62
+ @options = opts
63
+ end
64
+
65
+ ##
66
+ # Configuration for Sidekiq server, use like:
67
+ #
68
+ # Sidekiq.configure_server do |config|
69
+ # config.redis = { :namespace => 'myapp', :size => 25, :url => 'redis://myhost:8877/0' }
70
+ # config.server_middleware do |chain|
71
+ # chain.add MyServerHook
72
+ # end
73
+ # end
74
+ def self.configure_server
75
+ yield self if server?
76
+ end
77
+
78
+ ##
79
+ # Configuration for Sidekiq client, use like:
80
+ #
81
+ # Sidekiq.configure_client do |config|
82
+ # config.redis = { :namespace => 'myapp', :size => 1, :url => 'redis://myhost:8877/0' }
83
+ # end
84
+ def self.configure_client
85
+ yield self unless server?
86
+ end
87
+
88
+ def self.server?
89
+ defined?(Sidekiq::CLI)
90
+ end
91
+
92
+ def self.redis
93
+ raise ArgumentError, "requires a block" unless block_given?
94
+ redis_pool.with do |conn|
95
+ retryable = true
96
+ begin
97
+ yield conn
98
+ rescue Redis::CommandError => ex
99
+ #2550 Failover can cause the server to become a replica, need
100
+ # to disconnect and reopen the socket to get back to the primary.
101
+ (conn.disconnect!; retryable = false; retry) if retryable && ex.message =~ /READONLY/
102
+ raise
103
+ end
104
+ end
105
+ end
106
+
107
+ def self.redis_info
108
+ redis do |conn|
109
+ begin
110
+ # admin commands can't go through redis-namespace starting
111
+ # in redis-namespace 2.0
112
+ if conn.respond_to?(:namespace)
113
+ conn.redis.info
114
+ else
115
+ conn.info
116
+ end
117
+ rescue Redis::CommandError => ex
118
+ #2850 return fake version when INFO command has (probably) been renamed
119
+ raise unless ex.message =~ /unknown command/
120
+ FAKE_INFO
121
+ end
122
+ end
123
+ end
124
+
125
+ def self.redis_pool
126
+ @redis ||= Sidekiq::RedisConnection.create
127
+ end
128
+
129
+ def self.redis=(hash)
130
+ @redis = if hash.is_a?(ConnectionPool)
131
+ hash
132
+ else
133
+ Sidekiq::RedisConnection.create(hash)
134
+ end
135
+ end
136
+
137
+ def self.client_middleware
138
+ @client_chain ||= Middleware::Chain.new
139
+ yield @client_chain if block_given?
140
+ @client_chain
141
+ end
142
+
143
+ def self.server_middleware
144
+ @server_chain ||= default_server_middleware
145
+ yield @server_chain if block_given?
146
+ @server_chain
147
+ end
148
+
149
+ def self.default_server_middleware
150
+ Middleware::Chain.new
151
+ end
152
+
153
+ def self.default_worker_options=(hash)
154
+ # stringify
155
+ @default_worker_options = default_worker_options.merge(Hash[hash.map{|k, v| [k.to_s, v]}])
156
+ end
157
+ def self.default_worker_options
158
+ defined?(@default_worker_options) ? @default_worker_options : DEFAULT_WORKER_OPTIONS
159
+ end
160
+
161
+ def self.default_retries_exhausted=(prok)
162
+ logger.info { "default_retries_exhausted is deprecated, please use `config.death_handlers << -> {|job, ex| }`" }
163
+ return nil unless prok
164
+ death_handlers << prok
165
+ end
166
+
167
+ ##
168
+ # Death handlers are called when all retries for a job have been exhausted and
169
+ # the job dies. It's the notification to your application
170
+ # that this job will not succeed without manual intervention.
171
+ #
172
+ # Sidekiq.configure_server do |config|
173
+ # config.death_handlers << ->(job, ex) do
174
+ # end
175
+ # end
176
+ def self.death_handlers
177
+ options[:death_handlers]
178
+ end
179
+
180
+ def self.load_json(string)
181
+ JSON.parse(string)
182
+ end
183
+ def self.dump_json(object)
184
+ JSON.generate(object)
185
+ end
186
+
187
+ def self.logger
188
+ Sidekiq::Logging.logger
189
+ end
190
+ def self.logger=(log)
191
+ Sidekiq::Logging.logger = log
192
+ end
193
+
194
+ # How frequently Redis should be checked by a random Sidekiq process for
195
+ # scheduled and retriable jobs. Each individual process will take turns by
196
+ # waiting some multiple of this value.
197
+ #
198
+ # See sidekiq/scheduled.rb for an in-depth explanation of this value
199
+ def self.average_scheduled_poll_interval=(interval)
200
+ self.options[:average_scheduled_poll_interval] = interval
201
+ end
202
+
203
+ # Register a proc to handle any error which occurs within the Sidekiq process.
204
+ #
205
+ # Sidekiq.configure_server do |config|
206
+ # config.error_handlers << proc {|ex,ctx_hash| MyErrorService.notify(ex, ctx_hash) }
207
+ # end
208
+ #
209
+ # The default error handler logs errors to Sidekiq.logger.
210
+ def self.error_handlers
211
+ self.options[:error_handlers]
212
+ end
213
+
214
+ # Register a block to run at a point in the Sidekiq lifecycle.
215
+ # :startup, :quiet or :shutdown are valid events.
216
+ #
217
+ # Sidekiq.configure_server do |config|
218
+ # config.on(:shutdown) do
219
+ # puts "Goodbye cruel world!"
220
+ # end
221
+ # end
222
+ def self.on(event, &block)
223
+ raise ArgumentError, "Symbols only please: #{event}" unless event.is_a?(Symbol)
224
+ raise ArgumentError, "Invalid event name: #{event}" unless options[:lifecycle_events].key?(event)
225
+ options[:lifecycle_events][event] << block
226
+ end
227
+
228
+ # We are shutting down Sidekiq but what about workers that
229
+ # are working on some long job? This error is
230
+ # raised in workers that have not finished within the hard
231
+ # timeout limit. This is needed to rollback db transactions,
232
+ # otherwise Ruby's Thread#kill will commit. See #377.
233
+ # DO NOT RESCUE THIS ERROR IN YOUR WORKERS
234
+ class Shutdown < Interrupt; end
235
+ end
236
+
237
+ require 'sidekiq/rails' if defined?(::Rails::Engine)