sidekiq 6.0.0 → 6.4.0

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of sidekiq might be problematic. Click here for more details.

Files changed (102) hide show
  1. checksums.yaml +4 -4
  2. data/Changes.md +290 -2
  3. data/LICENSE +3 -3
  4. data/README.md +7 -9
  5. data/bin/sidekiq +26 -2
  6. data/bin/sidekiqload +8 -4
  7. data/bin/sidekiqmon +4 -5
  8. data/lib/generators/sidekiq/job_generator.rb +57 -0
  9. data/lib/generators/sidekiq/templates/{worker.rb.erb → job.rb.erb} +2 -2
  10. data/lib/generators/sidekiq/templates/{worker_spec.rb.erb → job_spec.rb.erb} +1 -1
  11. data/lib/generators/sidekiq/templates/{worker_test.rb.erb → job_test.rb.erb} +1 -1
  12. data/lib/sidekiq/api.rb +222 -145
  13. data/lib/sidekiq/cli.rb +67 -28
  14. data/lib/sidekiq/client.rb +17 -34
  15. data/lib/sidekiq/delay.rb +2 -0
  16. data/lib/sidekiq/extensions/action_mailer.rb +5 -4
  17. data/lib/sidekiq/extensions/active_record.rb +6 -5
  18. data/lib/sidekiq/extensions/class_methods.rb +7 -6
  19. data/lib/sidekiq/extensions/generic_proxy.rb +5 -3
  20. data/lib/sidekiq/fetch.rb +36 -27
  21. data/lib/sidekiq/job.rb +13 -0
  22. data/lib/sidekiq/job_logger.rb +13 -5
  23. data/lib/sidekiq/job_retry.rb +33 -21
  24. data/lib/sidekiq/job_util.rb +65 -0
  25. data/lib/sidekiq/launcher.rb +110 -28
  26. data/lib/sidekiq/logger.rb +109 -12
  27. data/lib/sidekiq/manager.rb +10 -12
  28. data/lib/sidekiq/middleware/chain.rb +17 -6
  29. data/lib/sidekiq/middleware/current_attributes.rb +57 -0
  30. data/lib/sidekiq/monitor.rb +3 -18
  31. data/lib/sidekiq/paginator.rb +7 -2
  32. data/lib/sidekiq/processor.rb +22 -24
  33. data/lib/sidekiq/rails.rb +27 -18
  34. data/lib/sidekiq/redis_connection.rb +19 -13
  35. data/lib/sidekiq/scheduled.rb +48 -12
  36. data/lib/sidekiq/sd_notify.rb +149 -0
  37. data/lib/sidekiq/systemd.rb +24 -0
  38. data/lib/sidekiq/testing.rb +14 -4
  39. data/lib/sidekiq/util.rb +40 -1
  40. data/lib/sidekiq/version.rb +1 -1
  41. data/lib/sidekiq/web/action.rb +2 -2
  42. data/lib/sidekiq/web/application.rb +41 -31
  43. data/lib/sidekiq/web/csrf_protection.rb +180 -0
  44. data/lib/sidekiq/web/helpers.rb +51 -33
  45. data/lib/sidekiq/web/router.rb +6 -5
  46. data/lib/sidekiq/web.rb +37 -73
  47. data/lib/sidekiq/worker.rb +133 -16
  48. data/lib/sidekiq.rb +29 -8
  49. data/sidekiq.gemspec +13 -6
  50. data/web/assets/images/apple-touch-icon.png +0 -0
  51. data/web/assets/javascripts/application.js +83 -64
  52. data/web/assets/javascripts/dashboard.js +53 -53
  53. data/web/assets/stylesheets/application-dark.css +143 -0
  54. data/web/assets/stylesheets/application-rtl.css +0 -4
  55. data/web/assets/stylesheets/application.css +43 -232
  56. data/web/locales/ar.yml +8 -2
  57. data/web/locales/de.yml +14 -2
  58. data/web/locales/en.yml +6 -1
  59. data/web/locales/es.yml +18 -2
  60. data/web/locales/fr.yml +10 -3
  61. data/web/locales/ja.yml +5 -0
  62. data/web/locales/lt.yml +83 -0
  63. data/web/locales/pl.yml +4 -4
  64. data/web/locales/ru.yml +4 -0
  65. data/web/locales/vi.yml +83 -0
  66. data/web/views/_footer.erb +1 -1
  67. data/web/views/_job_info.erb +3 -2
  68. data/web/views/_poll_link.erb +2 -5
  69. data/web/views/_summary.erb +7 -7
  70. data/web/views/busy.erb +54 -20
  71. data/web/views/dashboard.erb +22 -14
  72. data/web/views/dead.erb +3 -3
  73. data/web/views/layout.erb +3 -1
  74. data/web/views/morgue.erb +9 -6
  75. data/web/views/queue.erb +19 -10
  76. data/web/views/queues.erb +10 -2
  77. data/web/views/retries.erb +11 -8
  78. data/web/views/retry.erb +3 -3
  79. data/web/views/scheduled.erb +5 -2
  80. metadata +34 -54
  81. data/.circleci/config.yml +0 -61
  82. data/.github/contributing.md +0 -32
  83. data/.github/issue_template.md +0 -11
  84. data/.gitignore +0 -13
  85. data/.standard.yml +0 -20
  86. data/3.0-Upgrade.md +0 -70
  87. data/4.0-Upgrade.md +0 -53
  88. data/5.0-Upgrade.md +0 -56
  89. data/6.0-Upgrade.md +0 -70
  90. data/COMM-LICENSE +0 -97
  91. data/Ent-2.0-Upgrade.md +0 -37
  92. data/Ent-Changes.md +0 -250
  93. data/Gemfile +0 -24
  94. data/Gemfile.lock +0 -196
  95. data/Pro-2.0-Upgrade.md +0 -138
  96. data/Pro-3.0-Upgrade.md +0 -44
  97. data/Pro-4.0-Upgrade.md +0 -35
  98. data/Pro-5.0-Upgrade.md +0 -25
  99. data/Pro-Changes.md +0 -768
  100. data/Rakefile +0 -10
  101. data/code_of_conduct.md +0 -50
  102. data/lib/generators/sidekiq/worker_generator.rb +0 -47
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: fda3ed37e1d981e1b32eea70669c678d400c071a9146a919848bcc598612de25
4
- data.tar.gz: 6580ab6c188b2514efe807479ca25c06ea584d1def175d84d94639bc20f80b1e
3
+ metadata.gz: 9622b2851203b0c5a80695ab7801ca77e15dc63a641ae79132cda9a2fcbe0cc6
4
+ data.tar.gz: dd943a02d2cf910f51866d02254f3a7845cb159735bd54d223dd7127a1fbd2fa
5
5
  SHA512:
6
- metadata.gz: 25682d3dbf99d29fa3f56b7c74419863d3549b6e0c182e69c83fbc02a2955d053bca85eec53b474d4d22c7526e44e865f378aa4111152a91377b373be27c4d47
7
- data.tar.gz: e55d07434b1e2ab26df6748101bf10640fc08d4d93f00b75ed04d82ed309ee8529d652982430cd00671c98b5e53cc733bef4b686df58a6b0e116ba04c12139aa
6
+ metadata.gz: 65bcd542866d8699ecf5958d81a15fd322cd1c51dfc1dbc6a8b15402b70862510c134e2b0ed9f9dbcdbb4dea3a59c1d12878ff926145e503079a59896f279ff3
7
+ data.tar.gz: 36143e85dc7fd4611f8a43fe39788dc590725ac63a827fbc174916da2d59fdadb3fb64fa0d819c4ef90694f6f292ea15f7fb58fb29f368c0a06c5b90f1b4bca7
data/Changes.md CHANGED
@@ -1,13 +1,282 @@
1
1
  # Sidekiq Changes
2
2
 
3
- [Sidekiq Changes](https://github.com/mperham/sidekiq/blob/master/Changes.md) | [Sidekiq Pro Changes](https://github.com/mperham/sidekiq/blob/master/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/mperham/sidekiq/blob/master/Ent-Changes.md)
3
+ [Sidekiq Changes](https://github.com/mperham/sidekiq/blob/main/Changes.md) | [Sidekiq Pro Changes](https://github.com/mperham/sidekiq/blob/main/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/mperham/sidekiq/blob/main/Ent-Changes.md)
4
+
5
+ 6.4.0
6
+ ---------
7
+
8
+ - **SECURITY**: Validate input to avoid possible DoS in Web UI.
9
+ - Add **strict argument checking** [#5071]
10
+ Sidekiq will now log a warning if JSON-unsafe arguments are passed to `perform_async`.
11
+ Add `Sidekiq.strict_args!(false)` to your initializer to disable this warning.
12
+ This warning will switch to an exception in Sidekiq 7.0.
13
+ - Note that Delayed Extensions will be removed in Sidekiq 7.0 [#5076]
14
+ - Add `perform_{inline,sync}` in Sidekiq::Job to run a job synchronously [#5061, hasan-ally]
15
+ ```ruby
16
+ SomeJob.perform_async(args...)
17
+ SomeJob.perform_sync(args...)
18
+ SomeJob.perform_inline(args...)
19
+ ```
20
+ You can also dynamically redirect a job to run synchronously:
21
+ ```ruby
22
+ SomeJob.set("sync": true).perform_async(args...) # will run via perform_inline
23
+ ```
24
+ - Replace Sidekiq::Worker `app/workers` generator with Sidekiq::Job `app/sidekiq` generator [#5055]
25
+ ```
26
+ bin/rails generate sidekiq:job ProcessOrderJob
27
+ ```
28
+ - Fix job retries losing CurrentAttributes [#5090]
29
+ - Tweak shutdown to give long-running threads time to cleanup [#5095]
30
+ - Add keyword arguments support in extensions
31
+
32
+ 6.3.1
33
+ ---------
34
+
35
+ - Fix keyword arguments error with CurrentAttributes on Ruby 3.0 [#5048]
36
+
37
+ 6.3.0
38
+ ---------
39
+
40
+ - **BREAK**: The Web UI has been refactored to remove jQuery. Any UI extensions
41
+ which use jQuery will break.
42
+ - **FEATURE**: Sidekiq.logger has been enhanced so any `Rails.logger`
43
+ output in jobs now shows up in the Sidekiq console. Remove any logger
44
+ hacks in your initializer and see if it Just Works™ now. [#5021]
45
+ - **FEATURE**: Add `Sidekiq::Job` alias for `Sidekiq::Worker`, to better
46
+ reflect industry standard terminology. You can now do this:
47
+ ```ruby
48
+ class MyJob
49
+ include Sidekiq::Job
50
+ sidekiq_options ...
51
+ def perform(args)
52
+ end
53
+ end
54
+ ```
55
+ - **FEATURE**: Support for serializing ActiveSupport::CurrentAttributes into each job. [#4982]
56
+ ```ruby
57
+ # config/initializers/sidekiq.rb
58
+ require "sidekiq/middleware/current_attributes"
59
+ Sidekiq::CurrentAttributes.persist(Myapp::Current) # Your AS::CurrentAttributes singleton
60
+ ```
61
+ - **FEATURE**: Add `Sidekiq::Worker.perform_bulk` for enqueuing jobs in bulk,
62
+ similar to `Sidekiq::Client.push_bulk` [#5042]
63
+ ```ruby
64
+ MyJob.perform_bulk([[1], [2], [3]])
65
+ ```
66
+ - Implement `queue_as`, `wait` and `wait_until` for ActiveJob compatibility [#5003]
67
+ - Scheduler now uses Lua to reduce Redis load and network roundtrips [#5044]
68
+ - Retry Redis operation if we get an `UNBLOCKED` Redis error [#4985]
69
+ - Run existing signal traps, if any, before running Sidekiq's trap [#4991]
70
+ - Fix fetch bug when using weighted queues which caused Sidekiq to stop
71
+ processing queues randomly [#5031]
72
+
73
+ 6.2.2
74
+ ---------
75
+
76
+ - Reduce retry jitter, add jitter to `sidekiq_retry_in` values [#4957]
77
+ - Minimize scheduler load on Redis at scale [#4882]
78
+ - Improve logging of delay jobs [#4904, BuonOno]
79
+ - Minor CSS improvements for buttons and tables, design PRs always welcome!
80
+ - Tweak Web UI `Cache-Control` header [#4966]
81
+ - Rename internal API class `Sidekiq::Job` to `Sidekiq::JobRecord` [#4955]
82
+
83
+ 6.2.1
84
+ ---------
85
+
86
+ - Update RTT warning logic to handle transient RTT spikes [#4851]
87
+ - Fix very low priority CVE on unescaped queue name [#4852]
88
+ - Add note about sessions and Rails apps in API mode
89
+
90
+ 6.2.0
91
+ ---------
92
+
93
+ - Store Redis RTT and log if poor [#4824]
94
+ - Add process/thread stats to Busy page [#4806]
95
+ - Improve Web UI on mobile devices [#4840]
96
+ - **Refactor Web UI session usage** [#4804]
97
+ Numerous people have hit "Forbidden" errors and struggled with Sidekiq's
98
+ Web UI session requirement. If you have code in your initializer for
99
+ Web sessions, it's quite possible it will need to be removed. Here's
100
+ an overview:
101
+ ```
102
+ Sidekiq::Web needs a valid Rack session for CSRF protection. If this is a Rails app,
103
+ make sure you mount Sidekiq::Web *inside* your routes in `config/routes.rb` so
104
+ Sidekiq can reuse the Rails session:
105
+
106
+ Rails.application.routes.draw do
107
+ mount Sidekiq::Web => "/sidekiq"
108
+ ....
109
+ end
110
+
111
+ If this is a bare Rack app, use a session middleware before Sidekiq::Web:
112
+
113
+ # first, use IRB to create a shared secret key for sessions and commit it
114
+ require 'securerandom'; File.open(".session.key", "w") {|f| f.write(SecureRandom.hex(32)) }
115
+
116
+ # now, update your Rack app to include the secret with a session cookie middleware
117
+ use Rack::Session::Cookie, secret: File.read(".session.key"), same_site: true, max_age: 86400
118
+ run Sidekiq::Web
119
+
120
+ If this is a Rails app in API mode, you need to enable sessions.
121
+
122
+ https://guides.rubyonrails.org/api_app.html#using-session-middlewares
123
+ ```
124
+
125
+ 6.1.3
126
+ ---------
127
+
128
+ - Warn if Redis is configured to evict data under memory pressure [#4752]
129
+ - Add process RSS on the Busy page [#4717]
130
+
131
+ 6.1.2
132
+ ---------
133
+
134
+ - Improve readability in dark mode Web UI [#4674]
135
+ - Fix Web UI crash with corrupt session [#4672]
136
+ - Allow middleware to yield arguments [#4673, @eugeneius]
137
+ - Migrate CI from CircleCI to GitHub Actions [#4677]
138
+
139
+ 6.1.1
140
+ ---------
141
+
142
+ - Jobs are now sorted by age in the Busy Workers table. [#4641]
143
+ - Fix "check all" JS logic in Web UI [#4619]
144
+
145
+ 6.1.0
146
+ ---------
147
+
148
+ - Web UI - Dark Mode fixes [#4543, natematykiewicz]
149
+ - Ensure `Rack::ContentLength` is loaded as middleware for correct Web UI responses [#4541]
150
+ - Avoid exception dumping SSL store in Redis connection logging [#4532]
151
+ - Better error messages in Sidekiq::Client [#4549]
152
+ - Remove rack-protection, reimplement CSRF protection [#4588]
153
+ - Require redis-rb 4.2 [#4591]
154
+ - Update to jquery 1.12.4 [#4593]
155
+ - Refactor internal fetch logic and API [#4602]
156
+
157
+ 6.0.7
158
+ ---------
159
+
160
+ - Refactor systemd integration to work better with custom binaries [#4511]
161
+ - Don't connect to Redis at process exit if not needed [#4502]
162
+ - Remove Redis connection naming [#4479]
163
+ - Fix Redis Sentinel password redaction [#4499]
164
+ - Add Vietnamese locale (vi) [#4528]
165
+
166
+ 6.0.6
167
+ ---------
168
+
169
+ - **Integrate with systemd's watchdog and notification features** [#4488]
170
+ Set `Type=notify` in [sidekiq.service](https://github.com/mperham/sidekiq/blob/4b8a8bd3ae42f6e48ae1fdaf95ed7d7af18ed8bb/examples/systemd/sidekiq.service#L30-L39). The integration works automatically.
171
+ - Use `setTimeout` rather than `setInterval` to avoid thundering herd [#4480]
172
+ - Fix edge case where a job can be pushed without a queue.
173
+ - Flush job stats at exit [#4498]
174
+ - Check RAILS_ENV before RACK_ENV [#4493]
175
+ - Add Lithuanian locale [#4476]
176
+
177
+ 6.0.5
178
+ ---------
179
+
180
+ - Fix broken Web UI response when using NewRelic and Rack 2.1.2+. [#4440]
181
+ - Update APIs to use `UNLINK`, not `DEL`. [#4449]
182
+ - Fix Ruby 2.7 warnings [#4412]
183
+ - Add support for `APP_ENV` [[95fa5d9]](https://github.com/mperham/sidekiq/commit/95fa5d90192148026e52ca2902f1b83c70858ce8)
184
+
185
+ 6.0.4
186
+ ---------
187
+
188
+ - Fix ActiveJob's `sidekiq_options` integration [#4404]
189
+ - Sidekiq Pro users will now see a Pause button next to each queue in
190
+ the Web UI, allowing them to pause queues manually [#4374, shayonj]
191
+ - Fix Sidekiq::Workers API unintentional change in 6.0.2 [#4387]
192
+
193
+
194
+ 6.0.3
195
+ ---------
196
+
197
+ - Fix `Sidekiq::Client.push_bulk` API which was erroneously putting
198
+ invalid `at` values in the job payloads [#4321]
199
+
200
+ 6.0.2
201
+ ---------
202
+
203
+ - Fix Sidekiq Enterprise's rolling restart functionality, broken by refactoring in 6.0.0. [#4334]
204
+ - More internal refactoring and performance tuning [fatkodima]
205
+
206
+ 6.0.1
207
+ ---------
208
+
209
+ - **Performance tuning**, Sidekiq should be 10-15% faster now [#4303, 4299,
210
+ 4269, fatkodima]
211
+ - **Dark Mode support in Web UI** (further design polish welcome!) [#4227, mperham,
212
+ fatkodima, silent-e]
213
+ - **Job-specific log levels**, allowing you to turn on debugging for
214
+ problematic workers. [fatkodima, #4287]
215
+ ```ruby
216
+ MyWorker.set(log_level: :debug).perform_async(...)
217
+ ```
218
+ - **Ad-hoc job tags**. You can tag your jobs with, e.g, subdomain, tenant, country,
219
+ locale, application, version, user/client, "alpha/beta/pro/ent", types of jobs,
220
+ teams/people responsible for jobs, additional metadata, etc.
221
+ Tags are shown on different pages with job listings. Sidekiq Pro users
222
+ can filter based on them [fatkodima, #4280]
223
+ ```ruby
224
+ class MyWorker
225
+ include Sidekiq::Worker
226
+ sidekiq_options tags: ['bank-ops', 'alpha']
227
+ ...
228
+ end
229
+ ```
230
+ - Fetch scheduled jobs in batches before pushing into specific queues.
231
+ This will decrease enqueueing time of scheduled jobs by a third. [fatkodima, #4273]
232
+ ```
233
+ ScheduledSet with 10,000 jobs
234
+ Before: 56.6 seconds
235
+ After: 39.2 seconds
236
+ ```
237
+ - Compress error backtraces before pushing into Redis, if you are
238
+ storing error backtraces, this will halve the size of your RetrySet
239
+ in Redis [fatkodima, #4272]
240
+ ```
241
+ RetrySet with 100,000 jobs
242
+ Before: 261 MB
243
+ After: 129 MB
244
+ ```
245
+ - Support display of ActiveJob 6.0 payloads in the Web UI [#4263]
246
+ - Add `SortedSet#scan` for pattern based scanning. For large sets this API will be **MUCH** faster
247
+ than standard iteration using each. [fatkodima, #4262]
248
+ ```ruby
249
+ Sidekiq::DeadSet.new.scan("UnreliableApi") do |job|
250
+ job.retry
251
+ end
252
+ ```
253
+ - Dramatically speed up SortedSet#find\_job(jid) by using Redis's ZSCAN
254
+ support, approx 10x faster. [fatkodima, #4259]
255
+ ```
256
+ zscan 0.179366 0.047727 0.227093 ( 1.161376)
257
+ enum 8.522311 0.419826 8.942137 ( 9.785079)
258
+ ```
259
+ - Respect rails' generators `test_framework` option and gracefully handle extra `worker` suffix on generator [fatkodima, #4256]
260
+ - Add ability to sort 'Enqueued' page on Web UI by position in the queue [fatkodima, #4248]
261
+ - Support `Client.push_bulk` with different delays [fatkodima, #4243]
262
+ ```ruby
263
+ Sidekiq::Client.push_bulk("class" => FooJob, "args" => [[1], [2]], "at" => [1.minute.from_now.to_f, 5.minutes.from_now.to_f])
264
+ ```
265
+ - Easier way to test enqueuing specific ActionMailer and ActiveRecord delayed jobs. Instead of manually
266
+ parsing embedded class, you can now test by fetching jobs for specific classes. [fatkodima, #4292]
267
+ ```ruby
268
+ assert_equal 1, Sidekiq::Extensions::DelayedMailer.jobs_for(FooMailer).size
269
+ ```
270
+ - Add `sidekiqmon` to gemspec executables [#4242]
271
+ - Gracefully handle `Sidekiq.logger = nil` [#4240]
272
+ - Inject Sidekiq::LogContext module if user-supplied logger does not include it [#4239]
4
273
 
5
274
  6.0
6
275
  ---------
7
276
 
8
277
  This release has major breaking changes. Read and test carefully in production.
9
278
 
10
- - ActiveJobs can now use `sidekiq_options` directly to configure Sidekiq
279
+ - With Rails 6.0.2+, ActiveJobs can now use `sidekiq_options` directly to configure Sidekiq
11
280
  features/internals like the retry subsystem. [#4213, pirj]
12
281
  ```ruby
13
282
  class MyJob < ActiveJob::Base
@@ -17,6 +286,13 @@ class MyJob < ActiveJob::Base
17
286
  end
18
287
  end
19
288
  ```
289
+ - Logging has been redesigned to allow for pluggable log formatters:
290
+ ```ruby
291
+ Sidekiq.configure_server do |config|
292
+ config.log_formatter = Sidekiq::Logger::Formatters::JSON.new
293
+ end
294
+ ```
295
+ See the [Logging wiki page](https://github.com/mperham/sidekiq/wiki/Logging) for more details.
20
296
  - **BREAKING CHANGE** Validate proper usage of the `REDIS_PROVIDER`
21
297
  variable. This variable is meant to hold the name of the environment
22
298
  variable which contains your Redis URL, so that you can switch Redis
@@ -36,6 +312,18 @@ end
36
312
  - Integrate the StandardRB code formatter to ensure consistent code
37
313
  styling. [#4114, gearnode]
38
314
 
315
+ 5.2.9
316
+ ---------
317
+
318
+ - Release Rack lock due to a cascade of CVEs. [#4566]
319
+ Pro-tip: don't lock Rack.
320
+
321
+ 5.2.8
322
+ ---------
323
+
324
+ - Lock to Rack 2.0.x to prevent future incompatibilities
325
+ - Fix invalid reference in `sidekiqctl`
326
+
39
327
  5.2.7
40
328
  ---------
41
329
 
data/LICENSE CHANGED
@@ -4,6 +4,6 @@ Sidekiq is an Open Source project licensed under the terms of
4
4
  the LGPLv3 license. Please see <http://www.gnu.org/licenses/lgpl-3.0.html>
5
5
  for license text.
6
6
 
7
- Sidekiq Pro has a commercial-friendly license allowing private forks
8
- and modifications of Sidekiq. Please see https://sidekiq.org/products/pro.html for
9
- more detail. You can find the commercial license terms in COMM-LICENSE.
7
+ Sidekiq Pro and Sidekiq Enterprise have a commercial-friendly license.
8
+ You can find the commercial license in COMM-LICENSE.txt.
9
+ Please see https://sidekiq.org for purchasing options.
data/README.md CHANGED
@@ -2,10 +2,7 @@ Sidekiq
2
2
  ==============
3
3
 
4
4
  [![Gem Version](https://badge.fury.io/rb/sidekiq.svg)](https://rubygems.org/gems/sidekiq)
5
- [![Code Climate](https://codeclimate.com/github/mperham/sidekiq.svg)](https://codeclimate.com/github/mperham/sidekiq)
6
- [![Build Status](https://circleci.com/gh/mperham/sidekiq/tree/master.svg?style=svg)](https://circleci.com/gh/mperham/sidekiq/tree/master)
7
- [![Gitter Chat](https://badges.gitter.im/mperham/sidekiq.svg)](https://gitter.im/mperham/sidekiq)
8
-
5
+ ![Build](https://github.com/mperham/sidekiq/workflows/CI/badge.svg)
9
6
 
10
7
  Simple, efficient background processing for Ruby.
11
8
 
@@ -18,7 +15,8 @@ Performance
18
15
 
19
16
  Version | Latency | Garbage created for 10k jobs | Time to process 100k jobs | Throughput | Ruby
20
17
  -----------------|------|---------|---------|------------------------|-----
21
- Sidekiq 6.0.0 | 3 ms | 156 MB | 19 sec | **5200 jobs/sec** | MRI 2.6.3
18
+ Sidekiq 6.0.2 | 3 ms | 156 MB | 14.0 sec| **7100 jobs/sec** | MRI 2.6.3
19
+ Sidekiq 6.0.0 | 3 ms | 156 MB | 19 sec | 5200 jobs/sec | MRI 2.6.3
22
20
  Sidekiq 4.0.0 | 10 ms | 151 MB | 22 sec | 4500 jobs/sec |
23
21
  Sidekiq 3.5.1 | 22 ms | 1257 MB | 125 sec | 800 jobs/sec |
24
22
  Resque 1.25.2 | - | - | 420 sec | 240 jobs/sec |
@@ -45,10 +43,10 @@ Getting Started
45
43
  -----------------
46
44
 
47
45
  See the [Getting Started wiki page](https://github.com/mperham/sidekiq/wiki/Getting-Started) and follow the simple setup process.
48
- You can watch [this Youtube playlist](https://www.youtube.com/playlist?list=PLjeHh2LSCFrWGT5uVjUuFKAcrcj5kSai1) to learn all about
46
+ You can watch [this YouTube playlist](https://www.youtube.com/playlist?list=PLjeHh2LSCFrWGT5uVjUuFKAcrcj5kSai1) to learn all about
49
47
  Sidekiq and see its features in action. Here's the Web UI:
50
48
 
51
- ![Web UI](https://github.com/mperham/sidekiq/raw/master/examples/web-ui.png)
49
+ ![Web UI](https://github.com/mperham/sidekiq/raw/main/examples/web-ui.png)
52
50
 
53
51
 
54
52
  Want to Upgrade?
@@ -86,10 +84,10 @@ See the [Sidekiq support page](https://sidekiq.org/support.html) for details.
86
84
  License
87
85
  -----------------
88
86
 
89
- Please see [LICENSE](https://github.com/mperham/sidekiq/blob/master/LICENSE) for licensing details.
87
+ Please see [LICENSE](https://github.com/mperham/sidekiq/blob/main/LICENSE) for licensing details.
90
88
 
91
89
 
92
90
  Author
93
91
  -----------------
94
92
 
95
- Mike Perham, [@mperham@mastodon.xyz](https://mastodon.xyz/@mperham) / [@sidekiq](https://twitter.com/sidekiq), [https://www.mikeperham.com](https://www.mikeperham.com) / [https://www.contribsys.com](https://www.contribsys.com)
93
+ Mike Perham, [@getajobmike](https://twitter.com/getajobmike) / [@sidekiq](https://twitter.com/sidekiq), [https://www.mikeperham.com](https://www.mikeperham.com) / [https://www.contribsys.com](https://www.contribsys.com)
data/bin/sidekiq CHANGED
@@ -6,13 +6,37 @@ $TESTING = false
6
6
 
7
7
  require_relative '../lib/sidekiq/cli'
8
8
 
9
+ def integrate_with_systemd
10
+ return unless ENV["NOTIFY_SOCKET"]
11
+
12
+ Sidekiq.configure_server do |config|
13
+ Sidekiq.logger.info "Enabling systemd notification integration"
14
+ require "sidekiq/sd_notify"
15
+ config.on(:startup) do
16
+ Sidekiq::SdNotify.ready
17
+ end
18
+ config.on(:shutdown) do
19
+ Sidekiq::SdNotify.stopping
20
+ end
21
+ Sidekiq.start_watchdog if Sidekiq::SdNotify.watchdog?
22
+ end
23
+ end
24
+
9
25
  begin
10
26
  cli = Sidekiq::CLI.instance
11
27
  cli.parse
28
+
29
+ integrate_with_systemd
30
+
12
31
  cli.run
13
32
  rescue => e
14
33
  raise e if $DEBUG
15
- STDERR.puts e.message
16
- STDERR.puts e.backtrace.join("\n")
34
+ if Sidekiq.error_handlers.length == 0
35
+ STDERR.puts e.message
36
+ STDERR.puts e.backtrace.join("\n")
37
+ else
38
+ cli.handle_exception e
39
+ end
40
+
17
41
  exit 1
18
42
  end
data/bin/sidekiqload CHANGED
@@ -5,7 +5,8 @@
5
5
  $TESTING = false
6
6
 
7
7
  #require 'ruby-prof'
8
- Bundler.require(:default)
8
+ require 'bundler/setup'
9
+ Bundler.require(:default, :load_test)
9
10
 
10
11
  require_relative '../lib/sidekiq/cli'
11
12
  require_relative '../lib/sidekiq/launcher'
@@ -102,17 +103,20 @@ iter.times do
102
103
  end
103
104
  Sidekiq.logger.error "Created #{count*iter} jobs"
104
105
 
106
+ start = Time.now
107
+
105
108
  Monitoring = Thread.new do
106
109
  watchdog("monitor thread") do
107
110
  while true
108
- sleep 0.5
111
+ sleep 0.2
109
112
  qsize = Sidekiq.redis do |conn|
110
113
  conn.llen "queue:default"
111
114
  end
112
115
  total = qsize
113
- Sidekiq.logger.error("RSS: #{Process.rss} Pending: #{total}")
116
+ #Sidekiq.logger.error("RSS: #{Process.rss} Pending: #{total}")
114
117
  if total == 0
115
- Sidekiq.logger.error("Done, now here's the latency for three jobs")
118
+ Sidekiq.logger.error("Done, #{iter * count} jobs in #{Time.now - start} sec")
119
+ Sidekiq.logger.error("Now here's the latency for three jobs")
116
120
 
117
121
  LoadWorker.perform_async(1, Time.now.to_f)
118
122
  LoadWorker.perform_async(2, Time.now.to_f)
data/bin/sidekiqmon CHANGED
@@ -2,8 +2,7 @@
2
2
 
3
3
  require 'sidekiq/monitor'
4
4
 
5
- if ARGV[0] == 'status'
6
- Sidekiq::Monitor::Status.new.display(ARGV[1])
7
- else
8
- Sidekiq::Monitor.print_usage
9
- end
5
+ section = "all"
6
+ section = ARGV[0] if ARGV.size == 1
7
+
8
+ Sidekiq::Monitor::Status.new.display(section)
@@ -0,0 +1,57 @@
1
+ require "rails/generators/named_base"
2
+
3
+ module Sidekiq
4
+ module Generators # :nodoc:
5
+ class JobGenerator < ::Rails::Generators::NamedBase # :nodoc:
6
+ desc "This generator creates a Sidekiq Job in app/sidekiq and a corresponding test"
7
+
8
+ check_class_collision suffix: "Job"
9
+
10
+ def self.default_generator_root
11
+ File.dirname(__FILE__)
12
+ end
13
+
14
+ def create_job_file
15
+ template "job.rb.erb", File.join("app/sidekiq", class_path, "#{file_name}_job.rb")
16
+ end
17
+
18
+ def create_test_file
19
+ return unless test_framework
20
+
21
+ if test_framework == :rspec
22
+ create_job_spec
23
+ else
24
+ create_job_test
25
+ end
26
+ end
27
+
28
+ private
29
+
30
+ def create_job_spec
31
+ template_file = File.join(
32
+ "spec/sidekiq",
33
+ class_path,
34
+ "#{file_name}_job_spec.rb"
35
+ )
36
+ template "job_spec.rb.erb", template_file
37
+ end
38
+
39
+ def create_job_test
40
+ template_file = File.join(
41
+ "test/sidekiq",
42
+ class_path,
43
+ "#{file_name}_job_test.rb"
44
+ )
45
+ template "job_test.rb.erb", template_file
46
+ end
47
+
48
+ def file_name
49
+ @_file_name ||= super.sub(/_?job\z/i, "")
50
+ end
51
+
52
+ def test_framework
53
+ ::Rails.application.config.generators.options[:rails][:test_framework]
54
+ end
55
+ end
56
+ end
57
+ end
@@ -1,6 +1,6 @@
1
1
  <% module_namespacing do -%>
2
- class <%= class_name %>Worker
3
- include Sidekiq::Worker
2
+ class <%= class_name %>Job
3
+ include Sidekiq::Job
4
4
 
5
5
  def perform(*args)
6
6
  # Do something
@@ -1,6 +1,6 @@
1
1
  require 'rails_helper'
2
2
  <% module_namespacing do -%>
3
- RSpec.describe <%= class_name %>Worker, type: :worker do
3
+ RSpec.describe <%= class_name %>Job, type: :job do
4
4
  pending "add some examples to (or delete) #{__FILE__}"
5
5
  end
6
6
  <% end -%>
@@ -1,6 +1,6 @@
1
1
  require 'test_helper'
2
2
  <% module_namespacing do -%>
3
- class <%= class_name %>WorkerTest < Minitest::Test
3
+ class <%= class_name %>JobTest < Minitest::Test
4
4
  def test_example
5
5
  skip "add some examples to (or delete) #{__FILE__}"
6
6
  end