sidekiq 6.0.1 → 6.2.2

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of sidekiq might be problematic. Click here for more details.

Files changed (87) hide show
  1. checksums.yaml +4 -4
  2. data/Changes.md +147 -2
  3. data/LICENSE +1 -1
  4. data/README.md +4 -7
  5. data/bin/sidekiq +26 -2
  6. data/lib/generators/sidekiq/worker_generator.rb +1 -1
  7. data/lib/sidekiq/api.rb +151 -111
  8. data/lib/sidekiq/cli.rb +39 -10
  9. data/lib/sidekiq/client.rb +26 -15
  10. data/lib/sidekiq/extensions/action_mailer.rb +3 -2
  11. data/lib/sidekiq/extensions/active_record.rb +4 -3
  12. data/lib/sidekiq/extensions/class_methods.rb +5 -4
  13. data/lib/sidekiq/extensions/generic_proxy.rb +3 -1
  14. data/lib/sidekiq/fetch.rb +29 -21
  15. data/lib/sidekiq/job.rb +8 -0
  16. data/lib/sidekiq/job_logger.rb +2 -2
  17. data/lib/sidekiq/job_retry.rb +11 -12
  18. data/lib/sidekiq/launcher.rb +104 -24
  19. data/lib/sidekiq/logger.rb +12 -11
  20. data/lib/sidekiq/manager.rb +4 -4
  21. data/lib/sidekiq/middleware/chain.rb +6 -4
  22. data/lib/sidekiq/monitor.rb +2 -17
  23. data/lib/sidekiq/processor.rb +17 -39
  24. data/lib/sidekiq/rails.rb +16 -18
  25. data/lib/sidekiq/redis_connection.rb +21 -13
  26. data/lib/sidekiq/scheduled.rb +7 -1
  27. data/lib/sidekiq/sd_notify.rb +149 -0
  28. data/lib/sidekiq/systemd.rb +24 -0
  29. data/lib/sidekiq/testing.rb +2 -4
  30. data/lib/sidekiq/util.rb +28 -2
  31. data/lib/sidekiq/version.rb +1 -1
  32. data/lib/sidekiq/web/action.rb +2 -2
  33. data/lib/sidekiq/web/application.rb +30 -19
  34. data/lib/sidekiq/web/csrf_protection.rb +180 -0
  35. data/lib/sidekiq/web/helpers.rb +35 -24
  36. data/lib/sidekiq/web/router.rb +6 -5
  37. data/lib/sidekiq/web.rb +37 -73
  38. data/lib/sidekiq/worker.rb +4 -7
  39. data/lib/sidekiq.rb +14 -8
  40. data/sidekiq.gemspec +12 -5
  41. data/web/assets/images/apple-touch-icon.png +0 -0
  42. data/web/assets/javascripts/application.js +25 -27
  43. data/web/assets/stylesheets/application-dark.css +146 -124
  44. data/web/assets/stylesheets/application.css +35 -135
  45. data/web/locales/ar.yml +8 -2
  46. data/web/locales/de.yml +14 -2
  47. data/web/locales/en.yml +5 -0
  48. data/web/locales/es.yml +18 -2
  49. data/web/locales/fr.yml +10 -3
  50. data/web/locales/ja.yml +5 -0
  51. data/web/locales/lt.yml +83 -0
  52. data/web/locales/pl.yml +4 -4
  53. data/web/locales/ru.yml +4 -0
  54. data/web/locales/vi.yml +83 -0
  55. data/web/views/_job_info.erb +1 -1
  56. data/web/views/busy.erb +50 -19
  57. data/web/views/dashboard.erb +14 -6
  58. data/web/views/dead.erb +1 -1
  59. data/web/views/layout.erb +2 -1
  60. data/web/views/morgue.erb +6 -6
  61. data/web/views/queue.erb +1 -1
  62. data/web/views/queues.erb +10 -2
  63. data/web/views/retries.erb +7 -7
  64. data/web/views/retry.erb +1 -1
  65. data/web/views/scheduled.erb +1 -1
  66. metadata +26 -50
  67. data/.circleci/config.yml +0 -82
  68. data/.github/contributing.md +0 -32
  69. data/.github/issue_template.md +0 -11
  70. data/.gitignore +0 -13
  71. data/.standard.yml +0 -20
  72. data/3.0-Upgrade.md +0 -70
  73. data/4.0-Upgrade.md +0 -53
  74. data/5.0-Upgrade.md +0 -56
  75. data/6.0-Upgrade.md +0 -72
  76. data/COMM-LICENSE +0 -97
  77. data/Ent-2.0-Upgrade.md +0 -37
  78. data/Ent-Changes.md +0 -256
  79. data/Gemfile +0 -24
  80. data/Gemfile.lock +0 -196
  81. data/Pro-2.0-Upgrade.md +0 -138
  82. data/Pro-3.0-Upgrade.md +0 -44
  83. data/Pro-4.0-Upgrade.md +0 -35
  84. data/Pro-5.0-Upgrade.md +0 -25
  85. data/Pro-Changes.md +0 -776
  86. data/Rakefile +0 -10
  87. data/code_of_conduct.md +0 -50
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 792692164ce00b070d352bda8869518e3d20d92cfdb70cda88456b53d5a4a627
4
- data.tar.gz: b489c2ecc7b3708dd578a0184a9da58ac32d36e737a3b704d094e1ef375e93e1
3
+ metadata.gz: 1b0da541124d097e05936cc860a93009ba170d714cbd8dbc760c1245818e8ed7
4
+ data.tar.gz: 63995ccbb57fa16e4954cda4ab9610506b0a8ff35084492b520fc18608face35
5
5
  SHA512:
6
- metadata.gz: 80d7bd6a6fc9ef0a6026c9c5bb8aaf727db7323e1f1d9e057550681329991ddf82de030df70dc80e8f3a3608fc1f999ecb5d251765a5bcf09f23f764d86365a7
7
- data.tar.gz: 3148011168f88fbc4fa67f244a56be0b0e7da36d5009f4d465e0df2a7df93ddf8a99fc1c57c43fa0365a0cdb1de9fb770306daf070bcea5f330c1f3522210bf9
6
+ metadata.gz: f3b864db74530840437c0af43475ed0e12a876d6d2cbbd808ecc72722adddbd8d5548c9b777d4e34af5deddf94871567717518f70d787ea7f6a089f2f5b2f4ed
7
+ data.tar.gz: 4569eed5baa10134e99a0b1404c213e3b35c08ec158e2fa8b006918298750919e8d34074aa10d3b4dfb621bb18cbe68be467d1e7822fb855012cff0f64dcdc7b
data/Changes.md CHANGED
@@ -2,7 +2,140 @@
2
2
 
3
3
  [Sidekiq Changes](https://github.com/mperham/sidekiq/blob/master/Changes.md) | [Sidekiq Pro Changes](https://github.com/mperham/sidekiq/blob/master/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/mperham/sidekiq/blob/master/Ent-Changes.md)
4
4
 
5
- HEAD
5
+ 6.2.2
6
+ ---------
7
+
8
+ - Reduce retry jitter, add jitter to `sidekiq_retry_in` values [#4957]
9
+ - Minimize scheduler load on Redis at scale [#4882]
10
+ - Improve logging of delay jobs [#4904, BuonOno]
11
+ - Minor CSS improvements for buttons and tables, design PRs always welcome!
12
+ - Tweak Web UI `Cache-Control` header [#4966]
13
+ - Rename internal API class `Sidekiq::Job` to `Sidekiq::JobRecord` [#4955]
14
+
15
+ 6.2.1
16
+ ---------
17
+
18
+ - Update RTT warning logic to handle transient RTT spikes [#4851]
19
+ - Fix very low priority CVE on unescaped queue name [#4852]
20
+ - Add note about sessions and Rails apps in API mode
21
+
22
+ 6.2.0
23
+ ---------
24
+
25
+ - Store Redis RTT and log if poor [#4824]
26
+ - Add process/thread stats to Busy page [#4806]
27
+ - Improve Web UI on mobile devices [#4840]
28
+ - **Refactor Web UI session usage** [#4804]
29
+ Numerous people have hit "Forbidden" errors and struggled with Sidekiq's
30
+ Web UI session requirement. If you have code in your initializer for
31
+ Web sessions, it's quite possible it will need to be removed. Here's
32
+ an overview:
33
+ ```
34
+ Sidekiq::Web needs a valid Rack session for CSRF protection. If this is a Rails app,
35
+ make sure you mount Sidekiq::Web *inside* your routes in `config/routes.rb` so
36
+ Sidekiq can reuse the Rails session:
37
+
38
+ Rails.application.routes.draw do
39
+ mount Sidekiq::Web => "/sidekiq"
40
+ ....
41
+ end
42
+
43
+ If this is a bare Rack app, use a session middleware before Sidekiq::Web:
44
+
45
+ # first, use IRB to create a shared secret key for sessions and commit it
46
+ require 'securerandom'; File.open(".session.key", "w") {|f| f.write(SecureRandom.hex(32)) }
47
+
48
+ # now, update your Rack app to include the secret with a session cookie middleware
49
+ use Rack::Session::Cookie, secret: File.read(".session.key"), same_site: true, max_age: 86400
50
+ run Sidekiq::Web
51
+
52
+ If this is a Rails app in API mode, you need to enable sessions.
53
+
54
+ https://guides.rubyonrails.org/api_app.html#using-session-middlewares
55
+ ```
56
+
57
+ 6.1.3
58
+ ---------
59
+
60
+ - Warn if Redis is configured to evict data under memory pressure [#4752]
61
+ - Add process RSS on the Busy page [#4717]
62
+
63
+ 6.1.2
64
+ ---------
65
+
66
+ - Improve readability in dark mode Web UI [#4674]
67
+ - Fix Web UI crash with corrupt session [#4672]
68
+ - Allow middleware to yield arguments [#4673, @eugeneius]
69
+ - Migrate CI from CircleCI to GitHub Actions [#4677]
70
+
71
+ 6.1.1
72
+ ---------
73
+
74
+ - Jobs are now sorted by age in the Busy Workers table. [#4641]
75
+ - Fix "check all" JS logic in Web UI [#4619]
76
+
77
+ 6.1.0
78
+ ---------
79
+
80
+ - Web UI - Dark Mode fixes [#4543, natematykiewicz]
81
+ - Ensure `Rack::ContentLength` is loaded as middleware for correct Web UI responses [#4541]
82
+ - Avoid exception dumping SSL store in Redis connection logging [#4532]
83
+ - Better error messages in Sidekiq::Client [#4549]
84
+ - Remove rack-protection, reimplement CSRF protection [#4588]
85
+ - Require redis-rb 4.2 [#4591]
86
+ - Update to jquery 1.12.4 [#4593]
87
+ - Refactor internal fetch logic and API [#4602]
88
+
89
+ 6.0.7
90
+ ---------
91
+
92
+ - Refactor systemd integration to work better with custom binaries [#4511]
93
+ - Don't connect to Redis at process exit if not needed [#4502]
94
+ - Remove Redis connection naming [#4479]
95
+ - Fix Redis Sentinel password redaction [#4499]
96
+ - Add Vietnamese locale (vi) [#4528]
97
+
98
+ 6.0.6
99
+ ---------
100
+
101
+ - **Integrate with systemd's watchdog and notification features** [#4488]
102
+ Set `Type=notify` in [sidekiq.service](https://github.com/mperham/sidekiq/blob/4b8a8bd3ae42f6e48ae1fdaf95ed7d7af18ed8bb/examples/systemd/sidekiq.service#L30-L39). The integration works automatically.
103
+ - Use `setTimeout` rather than `setInterval` to avoid thundering herd [#4480]
104
+ - Fix edge case where a job can be pushed without a queue.
105
+ - Flush job stats at exit [#4498]
106
+ - Check RAILS_ENV before RACK_ENV [#4493]
107
+ - Add Lithuanian locale [#4476]
108
+
109
+ 6.0.5
110
+ ---------
111
+
112
+ - Fix broken Web UI response when using NewRelic and Rack 2.1.2+. [#4440]
113
+ - Update APIs to use `UNLINK`, not `DEL`. [#4449]
114
+ - Fix Ruby 2.7 warnings [#4412]
115
+ - Add support for `APP_ENV` [[95fa5d9]](https://github.com/mperham/sidekiq/commit/95fa5d90192148026e52ca2902f1b83c70858ce8)
116
+
117
+ 6.0.4
118
+ ---------
119
+
120
+ - Fix ActiveJob's `sidekiq_options` integration [#4404]
121
+ - Sidekiq Pro users will now see a Pause button next to each queue in
122
+ the Web UI, allowing them to pause queues manually [#4374, shayonj]
123
+ - Fix Sidekiq::Workers API unintentional change in 6.0.2 [#4387]
124
+
125
+
126
+ 6.0.3
127
+ ---------
128
+
129
+ - Fix `Sidekiq::Client.push_bulk` API which was erroneously putting
130
+ invalid `at` values in the job payloads [#4321]
131
+
132
+ 6.0.2
133
+ ---------
134
+
135
+ - Fix Sidekiq Enterprise's rolling restart functionality, broken by refactoring in 6.0.0. [#4334]
136
+ - More internal refactoring and performance tuning [fatkodima]
137
+
138
+ 6.0.1
6
139
  ---------
7
140
 
8
141
  - **Performance tuning**, Sidekiq should be 10-15% faster now [#4303, 4299,
@@ -75,7 +208,7 @@ assert_equal 1, Sidekiq::Extensions::DelayedMailer.jobs_for(FooMailer).size
75
208
 
76
209
  This release has major breaking changes. Read and test carefully in production.
77
210
 
78
- - With Rails 6.0.1+, ActiveJobs can now use `sidekiq_options` directly to configure Sidekiq
211
+ - With Rails 6.0.2+, ActiveJobs can now use `sidekiq_options` directly to configure Sidekiq
79
212
  features/internals like the retry subsystem. [#4213, pirj]
80
213
  ```ruby
81
214
  class MyJob < ActiveJob::Base
@@ -111,6 +244,18 @@ See the [Logging wiki page](https://github.com/mperham/sidekiq/wiki/Logging) for
111
244
  - Integrate the StandardRB code formatter to ensure consistent code
112
245
  styling. [#4114, gearnode]
113
246
 
247
+ 5.2.9
248
+ ---------
249
+
250
+ - Release Rack lock due to a cascade of CVEs. [#4566]
251
+ Pro-tip: don't lock Rack.
252
+
253
+ 5.2.8
254
+ ---------
255
+
256
+ - Lock to Rack 2.0.x to prevent future incompatibilities
257
+ - Fix invalid reference in `sidekiqctl`
258
+
114
259
  5.2.7
115
260
  ---------
116
261
 
data/LICENSE CHANGED
@@ -6,4 +6,4 @@ for license text.
6
6
 
7
7
  Sidekiq Pro has a commercial-friendly license allowing private forks
8
8
  and modifications of Sidekiq. Please see https://sidekiq.org/products/pro.html for
9
- more detail. You can find the commercial license terms in COMM-LICENSE.
9
+ more detail. You can find the commercial license terms in COMM-LICENSE.txt.
data/README.md CHANGED
@@ -2,11 +2,7 @@ Sidekiq
2
2
  ==============
3
3
 
4
4
  [![Gem Version](https://badge.fury.io/rb/sidekiq.svg)](https://rubygems.org/gems/sidekiq)
5
- [![Code Climate](https://codeclimate.com/github/mperham/sidekiq.svg)](https://codeclimate.com/github/mperham/sidekiq)
6
- [![Test Coverage](https://codeclimate.com/github/mperham/sidekiq/badges/coverage.svg)](https://codeclimate.com/github/mperham/sidekiq/coverage)
7
- [![Build Status](https://circleci.com/gh/mperham/sidekiq/tree/master.svg?style=svg)](https://circleci.com/gh/mperham/sidekiq/tree/master)
8
- [![Gitter Chat](https://badges.gitter.im/mperham/sidekiq.svg)](https://gitter.im/mperham/sidekiq)
9
-
5
+ ![Build](https://github.com/mperham/sidekiq/workflows/CI/badge.svg)
10
6
 
11
7
  Simple, efficient background processing for Ruby.
12
8
 
@@ -19,7 +15,8 @@ Performance
19
15
 
20
16
  Version | Latency | Garbage created for 10k jobs | Time to process 100k jobs | Throughput | Ruby
21
17
  -----------------|------|---------|---------|------------------------|-----
22
- Sidekiq 6.0.0 | 3 ms | 156 MB | 19 sec | **5200 jobs/sec** | MRI 2.6.3
18
+ Sidekiq 6.0.2 | 3 ms | 156 MB | 14.0 sec| **7100 jobs/sec** | MRI 2.6.3
19
+ Sidekiq 6.0.0 | 3 ms | 156 MB | 19 sec | 5200 jobs/sec | MRI 2.6.3
23
20
  Sidekiq 4.0.0 | 10 ms | 151 MB | 22 sec | 4500 jobs/sec |
24
21
  Sidekiq 3.5.1 | 22 ms | 1257 MB | 125 sec | 800 jobs/sec |
25
22
  Resque 1.25.2 | - | - | 420 sec | 240 jobs/sec |
@@ -93,4 +90,4 @@ Please see [LICENSE](https://github.com/mperham/sidekiq/blob/master/LICENSE) for
93
90
  Author
94
91
  -----------------
95
92
 
96
- Mike Perham, [@mperham@mastodon.xyz](https://mastodon.xyz/@mperham) / [@sidekiq](https://twitter.com/sidekiq), [https://www.mikeperham.com](https://www.mikeperham.com) / [https://www.contribsys.com](https://www.contribsys.com)
93
+ Mike Perham, [@getajobmike](https://twitter.com/getajobmike) / [@sidekiq](https://twitter.com/sidekiq), [https://www.mikeperham.com](https://www.mikeperham.com) / [https://www.contribsys.com](https://www.contribsys.com)
data/bin/sidekiq CHANGED
@@ -6,13 +6,37 @@ $TESTING = false
6
6
 
7
7
  require_relative '../lib/sidekiq/cli'
8
8
 
9
+ def integrate_with_systemd
10
+ return unless ENV["NOTIFY_SOCKET"]
11
+
12
+ Sidekiq.configure_server do |config|
13
+ Sidekiq.logger.info "Enabling systemd notification integration"
14
+ require "sidekiq/sd_notify"
15
+ config.on(:startup) do
16
+ Sidekiq::SdNotify.ready
17
+ end
18
+ config.on(:shutdown) do
19
+ Sidekiq::SdNotify.stopping
20
+ end
21
+ Sidekiq.start_watchdog if Sidekiq::SdNotify.watchdog?
22
+ end
23
+ end
24
+
9
25
  begin
10
26
  cli = Sidekiq::CLI.instance
11
27
  cli.parse
28
+
29
+ integrate_with_systemd
30
+
12
31
  cli.run
13
32
  rescue => e
14
33
  raise e if $DEBUG
15
- STDERR.puts e.message
16
- STDERR.puts e.backtrace.join("\n")
34
+ if Sidekiq.error_handlers.length == 0
35
+ STDERR.puts e.message
36
+ STDERR.puts e.backtrace.join("\n")
37
+ else
38
+ cli.handle_exception e
39
+ end
40
+
17
41
  exit 1
18
42
  end
@@ -18,7 +18,7 @@ module Sidekiq
18
18
  def create_test_file
19
19
  return unless test_framework
20
20
 
21
- if defined?(RSpec)
21
+ if test_framework == :rspec
22
22
  create_worker_spec
23
23
  else
24
24
  create_worker_test