sidekiq 5.2.4 → 7.2.4

Sign up to get free protection for your applications and to get access to all the features.
Files changed (153) hide show
  1. checksums.yaml +4 -4
  2. data/Changes.md +672 -8
  3. data/LICENSE.txt +9 -0
  4. data/README.md +48 -51
  5. data/bin/multi_queue_bench +271 -0
  6. data/bin/sidekiq +22 -3
  7. data/bin/sidekiqload +213 -115
  8. data/bin/sidekiqmon +11 -0
  9. data/lib/generators/sidekiq/job_generator.rb +57 -0
  10. data/lib/generators/sidekiq/templates/{worker.rb.erb → job.rb.erb} +2 -2
  11. data/lib/generators/sidekiq/templates/{worker_spec.rb.erb → job_spec.rb.erb} +1 -1
  12. data/lib/generators/sidekiq/templates/{worker_test.rb.erb → job_test.rb.erb} +1 -1
  13. data/lib/sidekiq/api.rb +623 -352
  14. data/lib/sidekiq/capsule.rb +127 -0
  15. data/lib/sidekiq/cli.rb +214 -229
  16. data/lib/sidekiq/client.rb +127 -102
  17. data/lib/sidekiq/component.rb +68 -0
  18. data/lib/sidekiq/config.rb +287 -0
  19. data/lib/sidekiq/deploy.rb +62 -0
  20. data/lib/sidekiq/embedded.rb +61 -0
  21. data/lib/sidekiq/fetch.rb +49 -42
  22. data/lib/sidekiq/job.rb +374 -0
  23. data/lib/sidekiq/job_logger.rb +33 -7
  24. data/lib/sidekiq/job_retry.rb +157 -108
  25. data/lib/sidekiq/job_util.rb +107 -0
  26. data/lib/sidekiq/launcher.rb +206 -106
  27. data/lib/sidekiq/logger.rb +131 -0
  28. data/lib/sidekiq/manager.rb +43 -46
  29. data/lib/sidekiq/metrics/query.rb +156 -0
  30. data/lib/sidekiq/metrics/shared.rb +95 -0
  31. data/lib/sidekiq/metrics/tracking.rb +140 -0
  32. data/lib/sidekiq/middleware/chain.rb +113 -56
  33. data/lib/sidekiq/middleware/current_attributes.rb +95 -0
  34. data/lib/sidekiq/middleware/i18n.rb +7 -7
  35. data/lib/sidekiq/middleware/modules.rb +21 -0
  36. data/lib/sidekiq/monitor.rb +146 -0
  37. data/lib/sidekiq/paginator.rb +28 -16
  38. data/lib/sidekiq/processor.rb +126 -117
  39. data/lib/sidekiq/rails.rb +52 -38
  40. data/lib/sidekiq/redis_client_adapter.rb +111 -0
  41. data/lib/sidekiq/redis_connection.rb +41 -112
  42. data/lib/sidekiq/ring_buffer.rb +29 -0
  43. data/lib/sidekiq/scheduled.rb +112 -50
  44. data/lib/sidekiq/sd_notify.rb +149 -0
  45. data/lib/sidekiq/systemd.rb +24 -0
  46. data/lib/sidekiq/testing/inline.rb +6 -5
  47. data/lib/sidekiq/testing.rb +91 -90
  48. data/lib/sidekiq/transaction_aware_client.rb +51 -0
  49. data/lib/sidekiq/version.rb +3 -1
  50. data/lib/sidekiq/web/action.rb +20 -11
  51. data/lib/sidekiq/web/application.rb +202 -80
  52. data/lib/sidekiq/web/csrf_protection.rb +183 -0
  53. data/lib/sidekiq/web/helpers.rb +165 -114
  54. data/lib/sidekiq/web/router.rb +23 -19
  55. data/lib/sidekiq/web.rb +68 -107
  56. data/lib/sidekiq/worker_compatibility_alias.rb +13 -0
  57. data/lib/sidekiq.rb +92 -182
  58. data/sidekiq.gemspec +25 -16
  59. data/web/assets/images/apple-touch-icon.png +0 -0
  60. data/web/assets/javascripts/application.js +152 -61
  61. data/web/assets/javascripts/base-charts.js +106 -0
  62. data/web/assets/javascripts/chart.min.js +13 -0
  63. data/web/assets/javascripts/chartjs-plugin-annotation.min.js +7 -0
  64. data/web/assets/javascripts/dashboard-charts.js +182 -0
  65. data/web/assets/javascripts/dashboard.js +35 -293
  66. data/web/assets/javascripts/metrics.js +298 -0
  67. data/web/assets/stylesheets/application-dark.css +147 -0
  68. data/web/assets/stylesheets/application-rtl.css +10 -93
  69. data/web/assets/stylesheets/application.css +124 -522
  70. data/web/assets/stylesheets/bootstrap.css +1 -1
  71. data/web/locales/ar.yml +71 -65
  72. data/web/locales/cs.yml +62 -62
  73. data/web/locales/da.yml +60 -53
  74. data/web/locales/de.yml +65 -53
  75. data/web/locales/el.yml +43 -24
  76. data/web/locales/en.yml +86 -66
  77. data/web/locales/es.yml +70 -54
  78. data/web/locales/fa.yml +65 -65
  79. data/web/locales/fr.yml +83 -62
  80. data/web/locales/gd.yml +99 -0
  81. data/web/locales/he.yml +65 -64
  82. data/web/locales/hi.yml +59 -59
  83. data/web/locales/it.yml +53 -53
  84. data/web/locales/ja.yml +75 -64
  85. data/web/locales/ko.yml +52 -52
  86. data/web/locales/lt.yml +83 -0
  87. data/web/locales/nb.yml +61 -61
  88. data/web/locales/nl.yml +52 -52
  89. data/web/locales/pl.yml +45 -45
  90. data/web/locales/pt-br.yml +83 -55
  91. data/web/locales/pt.yml +51 -51
  92. data/web/locales/ru.yml +68 -63
  93. data/web/locales/sv.yml +53 -53
  94. data/web/locales/ta.yml +60 -60
  95. data/web/locales/uk.yml +62 -61
  96. data/web/locales/ur.yml +64 -64
  97. data/web/locales/vi.yml +83 -0
  98. data/web/locales/zh-cn.yml +43 -16
  99. data/web/locales/zh-tw.yml +42 -8
  100. data/web/views/_footer.erb +18 -3
  101. data/web/views/_job_info.erb +21 -4
  102. data/web/views/_metrics_period_select.erb +12 -0
  103. data/web/views/_nav.erb +1 -1
  104. data/web/views/_paging.erb +2 -0
  105. data/web/views/_poll_link.erb +3 -6
  106. data/web/views/_summary.erb +7 -7
  107. data/web/views/busy.erb +79 -29
  108. data/web/views/dashboard.erb +48 -18
  109. data/web/views/dead.erb +3 -3
  110. data/web/views/filtering.erb +7 -0
  111. data/web/views/layout.erb +3 -1
  112. data/web/views/metrics.erb +91 -0
  113. data/web/views/metrics_for_job.erb +59 -0
  114. data/web/views/morgue.erb +14 -15
  115. data/web/views/queue.erb +33 -24
  116. data/web/views/queues.erb +19 -5
  117. data/web/views/retries.erb +16 -17
  118. data/web/views/retry.erb +3 -3
  119. data/web/views/scheduled.erb +17 -15
  120. metadata +71 -72
  121. data/.github/contributing.md +0 -32
  122. data/.github/issue_template.md +0 -11
  123. data/.gitignore +0 -15
  124. data/.travis.yml +0 -17
  125. data/3.0-Upgrade.md +0 -70
  126. data/4.0-Upgrade.md +0 -53
  127. data/5.0-Upgrade.md +0 -56
  128. data/Appraisals +0 -9
  129. data/COMM-LICENSE +0 -95
  130. data/Ent-Changes.md +0 -225
  131. data/Gemfile +0 -29
  132. data/LICENSE +0 -9
  133. data/Pro-2.0-Upgrade.md +0 -138
  134. data/Pro-3.0-Upgrade.md +0 -44
  135. data/Pro-4.0-Upgrade.md +0 -35
  136. data/Pro-Changes.md +0 -752
  137. data/Rakefile +0 -9
  138. data/bin/sidekiqctl +0 -237
  139. data/code_of_conduct.md +0 -50
  140. data/gemfiles/rails_4.gemfile +0 -31
  141. data/gemfiles/rails_5.gemfile +0 -31
  142. data/lib/generators/sidekiq/worker_generator.rb +0 -49
  143. data/lib/sidekiq/core_ext.rb +0 -1
  144. data/lib/sidekiq/delay.rb +0 -42
  145. data/lib/sidekiq/exception_handler.rb +0 -29
  146. data/lib/sidekiq/extensions/action_mailer.rb +0 -57
  147. data/lib/sidekiq/extensions/active_record.rb +0 -40
  148. data/lib/sidekiq/extensions/class_methods.rb +0 -40
  149. data/lib/sidekiq/extensions/generic_proxy.rb +0 -31
  150. data/lib/sidekiq/logging.rb +0 -122
  151. data/lib/sidekiq/middleware/server/active_record.rb +0 -23
  152. data/lib/sidekiq/util.rb +0 -66
  153. data/lib/sidekiq/worker.rb +0 -215
data/Changes.md CHANGED
@@ -1,6 +1,670 @@
1
1
  # Sidekiq Changes
2
2
 
3
- [Sidekiq Changes](https://github.com/mperham/sidekiq/blob/master/Changes.md) | [Sidekiq Pro Changes](https://github.com/mperham/sidekiq/blob/master/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/mperham/sidekiq/blob/master/Ent-Changes.md)
3
+ [Sidekiq Changes](https://github.com/sidekiq/sidekiq/blob/main/Changes.md) | [Sidekiq Pro Changes](https://github.com/sidekiq/sidekiq/blob/main/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/sidekiq/sidekiq/blob/main/Ent-Changes.md)
4
+
5
+ 7.2.4
6
+ ----------
7
+
8
+ - Fix XSS in metrics filtering introduced in 7.2.0, CVE-2024-32887
9
+ Thanks to @UmerAdeemCheema for the security report.
10
+
11
+ 7.2.3
12
+ ----------
13
+
14
+ - [Support Dragonfly.io](https://www.mikeperham.com/2024/02/01/supporting-dragonfly/) as an alternative Redis implementation
15
+ - Fix error unpacking some compressed error backtraces [#6241]
16
+ - Fix potential heartbeat data leak [#6227]
17
+ - Add ability to find a currently running work by jid [#6212, fatkodima]
18
+
19
+ 7.2.2
20
+ ----------
21
+
22
+ - Add `Process.warmup` call in Ruby 3.3+
23
+ - Batch jobs now skip transactional push [#6160]
24
+
25
+ 7.2.1
26
+ ----------
27
+
28
+ - Add `Sidekiq::Work` type which replaces the raw Hash as the third parameter in
29
+ `Sidekiq::WorkSet#each { |pid, tid, hash| ... }` [#6145]
30
+ - **DEPRECATED**: direct access to the attributes within the `hash` block parameter above.
31
+ The `Sidekiq::Work` instance contains accessor methods to get at the same data, e.g.
32
+ ```ruby
33
+ work["queue"] # Old
34
+ work.queue # New
35
+ ```
36
+ - Fix Ruby 3.3 warnings around `base64` gem [#6151, earlopain]
37
+
38
+ 7.2.0
39
+ ----------
40
+
41
+ - `sidekiq_retries_exhausted` can return `:discard` to avoid the deadset
42
+ and all death handlers [#6091]
43
+ - Metrics filtering by job class in Web UI [#5974]
44
+ - Better readability and formatting for numbers within the Web UI [#6080]
45
+ - Add explicit error if user code tries to nest test modes [#6078]
46
+ ```ruby
47
+ Sidekiq::Testing.inline! # global setting
48
+ Sidekiq::Testing.fake! do # override within block
49
+ # ok
50
+ Sidekiq::Testing.inline! do # can't override the override
51
+ # not ok, nested
52
+ end
53
+ end
54
+ ```
55
+ - **SECURITY** Forbid inline JavaScript execution in Web UI [#6074]
56
+ - Adjust redis-client adapter to avoid `method_missing` [#6083]
57
+ This can result in app code breaking if your app's Redis API usage was
58
+ depending on Sidekiq's adapter to correct invalid redis-client API usage.
59
+ One example:
60
+ ```ruby
61
+ # bad, not redis-client native
62
+ # Unsupported command argument type: TrueClass (TypeError)
63
+ Sidekiq.redis { |c| c.set("key", "value", nx: true, ex: 15) }
64
+ # good
65
+ Sidekiq.redis { |c| c.set("key", "value", "nx", "ex", 15) }
66
+ ```
67
+
68
+ 7.1.6
69
+ ----------
70
+
71
+ - The block forms of testing modes (inline, fake) are now thread-safe so you can have
72
+ a multithreaded test suite which uses different modes for different tests. [#6069]
73
+ - Fix breakage with non-Proc error handlers [#6065]
74
+
75
+ 7.1.5
76
+ ----------
77
+
78
+ - **FEATURE**: Job filtering within the Web UI. This feature has been open
79
+ sourced from Sidekiq Pro. [#6052]
80
+ - **API CHANGE** Error handlers now take three arguments `->(ex, context, config)`.
81
+ The previous calling convention will work until Sidekiq 8.0 but will print
82
+ out a deprecation warning. [#6051]
83
+ - Fix issue with the `batch_size` and `at` options in `S::Client.push_bulk` [#6040]
84
+ - Fix inline testing firing batch callbacks early [#6057]
85
+ - Use new log broadcast API in Rails 7.1 [#6054]
86
+ - Crash if user tries to use RESP2 `protocol: 2` [#6061]
87
+
88
+ 7.1.4
89
+ ----------
90
+
91
+ - Fix empty `retry_for` logic [#6035]
92
+
93
+ 7.1.3
94
+ ----------
95
+
96
+ - Add `sidekiq_options retry_for: 48.hours` to allow time-based retry windows [#6029]
97
+ - Support sidekiq_retry_in and sidekiq_retries_exhausted_block in ActiveJobs (#5994)
98
+ - Lowercase all Rack headers for Rack 3.0 [#5951]
99
+ - Validate Sidekiq::Web page refresh delay to avoid potential DoS,
100
+ CVE-2023-26141, thanks for reporting Keegan!
101
+
102
+ 7.1.2
103
+ ----------
104
+
105
+ - Mark Web UI assets as private so CDNs won't cache them [#5936]
106
+ - Fix stackoverflow when using Oj and the JSON log formatter [#5920]
107
+ - Remove spurious `enqueued_at` from scheduled ActiveJobs [#5937]
108
+
109
+ 7.1.1
110
+ ----------
111
+
112
+ - Support multiple CurrentAttributes [#5904]
113
+ - Speed up latency fetch with large queues on Redis <7 [#5910]
114
+ - Allow a larger default client pool [#5886]
115
+ - Ensure Sidekiq.options[:environment] == RAILS_ENV [#5932]
116
+
117
+ 7.1.0
118
+ ----------
119
+
120
+ - Improve display of ActiveJob arguments in Web UI [#5825, cover]
121
+ - Update `push_bulk` to push `batch_size` jobs at a time and allow laziness [#5827, fatkodima]
122
+ This allows Sidekiq::Client to push unlimited jobs as long as it has enough memory for the batch_size.
123
+ - Update `perform_bulk` to use `push_bulk` internally.
124
+ - Change return value of `push_bulk` to map 1-to-1 with arguments.
125
+ If you call `push_bulk(args: [[1], [2], [3]])`, you will now always get
126
+ an array of 3 values as the result: `["jid1", nil, "jid3"]` where nil means
127
+ that particular job did not push successfully (possibly due to middleware
128
+ stopping it). Previously nil values were removed so it was impossible to tell
129
+ which jobs pushed successfully and which did not.
130
+ - Migrate away from all deprecated Redis commands [#5788]
131
+ Sidekiq will now print a warning if you use one of those deprecated commands.
132
+ - Prefix all Sidekiq thread names [#5872]
133
+
134
+ 7.0.9
135
+ ----------
136
+
137
+ - Restore confirmation dialogs in Web UI [#5881, shevaun]
138
+ - Increase fetch timeout to minimize ReadTimeoutError [#5874]
139
+ - Reverse histogram tooltip ordering [#5868]
140
+ - Add Scottish Gaelic (gd) locale [#5867, GunChleoc]
141
+
142
+ 7.0.8
143
+ ----------
144
+
145
+ - **SECURITY** Sanitize `period` input parameter on Metrics pages.
146
+ Specially crafted values can lead to XSS. This functionality
147
+ was introduced in 7.0.4. Thank you to spercex @ huntr.dev [#5694]
148
+ - Add job hash as 3rd parameter to the `sidekiq_retry_in` block.
149
+
150
+ 7.0.7
151
+ ----------
152
+
153
+ - Fix redis-client API usage which could result in stuck Redis
154
+ connections [#5823]
155
+ - Fix AS::Duration with `sidekiq_retry_in` [#5806]
156
+ - Restore dumping config options on startup with `-v` [#5822]
157
+
158
+ 7.0.5,7.0.6
159
+ ----------
160
+
161
+ - More context for debugging json unsafe errors [#5787]
162
+
163
+ 7.0.4
164
+ ----------
165
+
166
+ - Performance and memory optimizations [#5768, fatkodima]
167
+ - Add 1-8 hour period selector to Metrics pages [#5694]
168
+ - Fix process display with `sidekiqmon` [#5733]
169
+
170
+ 7.0.3
171
+ ----------
172
+
173
+ - Don't warn about memory policy on Redis Enterprise [#5712]
174
+ - Don't allow Quiet/Stop on embedded Sidekiq instances [#5716]
175
+ - Fix `size: X` for configuring the default Redis pool size [#5702]
176
+ - Improve the display of queue weights on Busy page [#5642]
177
+ - Freeze CurrentAttributes on a job once initially set [#5692]
178
+
179
+ 7.0.2
180
+ ----------
181
+
182
+ - Improve compatibility with custom loggers [#5673]
183
+ - Add queue weights on Busy page [#5640]
184
+ - Add BID link on job_info page if job is part of a Batch [#5623]
185
+ - Allow custom extensions to add rows/links within Job detail pages [#5624]
186
+ ```ruby
187
+ Sidekiq::Web.custom_job_info_rows << AddAccountLink.new
188
+
189
+ class AddAccountLink
190
+ include CGI::Util
191
+ def add_pair(job)
192
+ # yield a (name, value) pair
193
+ # You can include HTML tags and CSS, Sidekiq does not do any
194
+ # escaping so beware user data injection! Note how we use CGI's
195
+ # `h` escape helper.
196
+ aid = job["account_id"]
197
+ yield "Account", "<a href='/accounts/#{h aid}'>#{h aid}</a>" if aid
198
+ end
199
+ end
200
+ ```
201
+
202
+ 7.0.1
203
+ ----------
204
+
205
+ - Allow an embedding process to reuse its own heartbeat thread
206
+ - Update zh-cn localization
207
+
208
+ 7.0.0
209
+ ----------
210
+
211
+ - Embedded mode!
212
+ - Capsules!!
213
+ - Job Execution metrics!!!
214
+ - See `docs/7.0-Upgrade.md` for release notes
215
+
216
+ 6.5.{10,11,12}
217
+ ----------
218
+
219
+ - Fixes for Rails 7.1 [#6067, #6070]
220
+
221
+ 6.5.9
222
+ ----------
223
+
224
+ - Ensure Sidekiq.options[:environment] == RAILS_ENV [#5932]
225
+
226
+ 6.5.8
227
+ ----------
228
+
229
+ - Fail if using a bad version of scout_apm [#5616]
230
+ - Add pagination to Busy page [#5556]
231
+ - Speed up WorkSet#each [#5559]
232
+ - Adjust CurrentAttributes to work with the String class name so we aren't referencing the Class within a Rails initializer [#5536]
233
+
234
+ 6.5.7
235
+ ----------
236
+
237
+ - Updates for JA and ZH locales
238
+ - Further optimizations for scheduled polling [#5513]
239
+
240
+ 6.5.6
241
+ ----------
242
+
243
+ - Fix deprecation warnings with redis-rb 4.8.0 [#5484]
244
+ - Lock redis-rb to < 5.0 as we are moving to redis-client in Sidekiq 7.0
245
+
246
+ 6.5.5
247
+ ----------
248
+
249
+ - Fix require issue with job_retry.rb [#5462]
250
+ - Improve Sidekiq::Web compatibility with Rack 3.x
251
+
252
+ 6.5.4
253
+ ----------
254
+
255
+ - Fix invalid code on Ruby 2.5 [#5460]
256
+ - Fix further metrics dependency issues [#5457]
257
+
258
+ 6.5.3
259
+ ----------
260
+
261
+ - Don't require metrics code without explicit opt-in [#5456]
262
+
263
+ 6.5.2
264
+ ----------
265
+
266
+ - [Job Metrics are under active development, help wanted!](https://github.com/sidekiq/sidekiq/wiki/Metrics#contributing) **BETA**
267
+ - Add `Context` column on queue page which shows any CurrentAttributes [#5450]
268
+ - `sidekiq_retry_in` may now return `:discard` or `:kill` to dynamically stop job retries [#5406]
269
+ - Smarter sorting of processes in /busy Web UI [#5398]
270
+ - Fix broken hamburger menu in mobile UI [#5428]
271
+ - Require redis-rb 4.5.0. Note that Sidekiq will break if you use the
272
+ [`Redis.exists_returns_integer = false`](https://github.com/redis/redis-rb/blob/master/CHANGELOG.md#450) flag. [#5394]
273
+
274
+ 6.5.1
275
+ ----------
276
+
277
+ - Fix `push_bulk` breakage [#5387]
278
+
279
+ 6.5.0
280
+ ---------
281
+
282
+ - Substantial refactoring of Sidekiq server internals, part of a larger effort
283
+ to reduce Sidekiq's internal usage of global methods and data, see [docs/global_to_local.md](docs/global_to_local.md) and [docs/middleware.md](docs/middleware.md).
284
+ - **Add beta support for the `redis-client` gem**. This will become the default Redis driver in Sidekiq 7.0. [#5298]
285
+ Read more: https://github.com/sidekiq/sidekiq/wiki/Using-redis-client
286
+ - **Add beta support for DB transaction-aware client** [#5291]
287
+ Add this line to your initializer and any jobs created during a transaction
288
+ will only be pushed to Redis **after the transaction commits**. You will need to add the
289
+ `after_commit_everywhere` gem to your Gemfile.
290
+ ```ruby
291
+ Sidekiq.transactional_push!
292
+ ```
293
+ This feature does not have a lot of production usage yet; please try it out and let us
294
+ know if you have any issues. It will be fully supported in Sidekiq 7.0 or removed if it
295
+ proves problematic.
296
+ - Fix regression with middleware arguments [#5312]
297
+
298
+ 6.4.2
299
+ ---------
300
+
301
+ - Strict argument checking now runs after client-side middleware [#5246]
302
+ - Fix page events with live polling [#5184]
303
+ - Many under-the-hood changes to remove all usage of the term "worker"
304
+ from the Sidekiq codebase and APIs. This mostly involved RDoc and local
305
+ variable names but a few constants and public APIs were changed. The old
306
+ APIs will be removed in Sidekiq 7.0.
307
+ ```
308
+ Sidekiq::DEFAULT_WORKER_OPTIONS -> Sidekiq.default_job_options
309
+ Sidekiq.default_worker_options -> Sidekiq.default_job_options
310
+ Sidekiq::Queues["default"].jobs_by_worker(HardJob) -> Sidekiq::Queues["default"].jobs_by_class(HardJob)
311
+ ```
312
+
313
+ 6.4.1
314
+ ---------
315
+
316
+ - Fix pipeline/multi deprecations in redis-rb 4.6
317
+ - Fix sidekiq.yml YAML load errors on Ruby 3.1 [#5141]
318
+ - Sharding support for `perform_bulk` [#5129]
319
+ - Refactor job logger for SPEEEEEEED
320
+
321
+ 6.4.0
322
+ ---------
323
+
324
+ - **SECURITY**: Validate input to avoid possible DoS in Web UI.
325
+ - Add **strict argument checking** [#5071]
326
+ Sidekiq will now log a warning if JSON-unsafe arguments are passed to `perform_async`.
327
+ Add `Sidekiq.strict_args!(false)` to your initializer to disable this warning.
328
+ This warning will switch to an exception in Sidekiq 7.0.
329
+ - Note that Delayed Extensions will be removed in Sidekiq 7.0 [#5076]
330
+ - Add `perform_{inline,sync}` in Sidekiq::Job to run a job synchronously [#5061, hasan-ally]
331
+ ```ruby
332
+ SomeJob.perform_async(args...)
333
+ SomeJob.perform_sync(args...)
334
+ SomeJob.perform_inline(args...)
335
+ ```
336
+ You can also dynamically redirect a job to run synchronously:
337
+ ```ruby
338
+ SomeJob.set("sync": true).perform_async(args...) # will run via perform_inline
339
+ ```
340
+ - Replace Sidekiq::Worker `app/workers` generator with Sidekiq::Job `app/sidekiq` generator [#5055]
341
+ ```
342
+ bin/rails generate sidekiq:job ProcessOrderJob
343
+ ```
344
+ - Fix job retries losing CurrentAttributes [#5090]
345
+ - Tweak shutdown to give long-running threads time to cleanup [#5095]
346
+
347
+ 6.3.1
348
+ ---------
349
+
350
+ - Fix keyword arguments error with CurrentAttributes on Ruby 3.0 [#5048]
351
+
352
+ 6.3.0
353
+ ---------
354
+
355
+ - **BREAK**: The Web UI has been refactored to remove jQuery. Any UI extensions
356
+ which use jQuery will break.
357
+ - **FEATURE**: Sidekiq.logger has been enhanced so any `Rails.logger`
358
+ output in jobs now shows up in the Sidekiq console. Remove any logger
359
+ hacks in your initializer and see if it Just Works™ now. [#5021]
360
+ - **FEATURE**: Add `Sidekiq::Job` alias for `Sidekiq::Worker`, to better
361
+ reflect industry standard terminology. You can now do this:
362
+ ```ruby
363
+ class MyJob
364
+ include Sidekiq::Job
365
+ sidekiq_options ...
366
+ def perform(args)
367
+ end
368
+ end
369
+ ```
370
+ - **FEATURE**: Support for serializing ActiveSupport::CurrentAttributes into each job. [#4982]
371
+ ```ruby
372
+ # config/initializers/sidekiq.rb
373
+ require "sidekiq/middleware/current_attributes"
374
+ Sidekiq::CurrentAttributes.persist(Myapp::Current) # Your AS::CurrentAttributes singleton
375
+ ```
376
+ - **FEATURE**: Add `Sidekiq::Worker.perform_bulk` for enqueuing jobs in bulk,
377
+ similar to `Sidekiq::Client.push_bulk` [#5042]
378
+ ```ruby
379
+ MyJob.perform_bulk([[1], [2], [3]])
380
+ ```
381
+ - Implement `queue_as`, `wait` and `wait_until` for ActiveJob compatibility [#5003]
382
+ - Scheduler now uses Lua to reduce Redis load and network roundtrips [#5044]
383
+ - Retry Redis operation if we get an `UNBLOCKED` Redis error [#4985]
384
+ - Run existing signal traps, if any, before running Sidekiq's trap [#4991]
385
+ - Fix fetch bug when using weighted queues which caused Sidekiq to stop
386
+ processing queues randomly [#5031]
387
+
388
+ 6.2.2
389
+ ---------
390
+
391
+ - Reduce retry jitter, add jitter to `sidekiq_retry_in` values [#4957]
392
+ - Minimize scheduler load on Redis at scale [#4882]
393
+ - Improve logging of delay jobs [#4904, BuonOno]
394
+ - Minor CSS improvements for buttons and tables, design PRs always welcome!
395
+ - Tweak Web UI `Cache-Control` header [#4966]
396
+ - Rename internal API class `Sidekiq::Job` to `Sidekiq::JobRecord` [#4955]
397
+
398
+ 6.2.1
399
+ ---------
400
+
401
+ - Update RTT warning logic to handle transient RTT spikes [#4851]
402
+ - Fix very low priority CVE on unescaped queue name [#4852]
403
+ - Add note about sessions and Rails apps in API mode
404
+
405
+ 6.2.0
406
+ ---------
407
+
408
+ - Store Redis RTT and log if poor [#4824]
409
+ - Add process/thread stats to Busy page [#4806]
410
+ - Improve Web UI on mobile devices [#4840]
411
+ - **Refactor Web UI session usage** [#4804]
412
+ Numerous people have hit "Forbidden" errors and struggled with Sidekiq's
413
+ Web UI session requirement. If you have code in your initializer for
414
+ Web sessions, it's quite possible it will need to be removed. Here's
415
+ an overview:
416
+ ```
417
+ Sidekiq::Web needs a valid Rack session for CSRF protection. If this is a Rails app,
418
+ make sure you mount Sidekiq::Web *inside* your routes in `config/routes.rb` so
419
+ Sidekiq can reuse the Rails session:
420
+
421
+ Rails.application.routes.draw do
422
+ mount Sidekiq::Web => "/sidekiq"
423
+ ....
424
+ end
425
+
426
+ If this is a bare Rack app, use a session middleware before Sidekiq::Web:
427
+
428
+ # first, use IRB to create a shared secret key for sessions and commit it
429
+ require 'securerandom'; File.open(".session.key", "w") {|f| f.write(SecureRandom.hex(32)) }
430
+
431
+ # now, update your Rack app to include the secret with a session cookie middleware
432
+ use Rack::Session::Cookie, secret: File.read(".session.key"), same_site: true, max_age: 86400
433
+ run Sidekiq::Web
434
+
435
+ If this is a Rails app in API mode, you need to enable sessions.
436
+
437
+ https://guides.rubyonrails.org/api_app.html#using-session-middlewares
438
+ ```
439
+
440
+ 6.1.3
441
+ ---------
442
+
443
+ - Warn if Redis is configured to evict data under memory pressure [#4752]
444
+ - Add process RSS on the Busy page [#4717]
445
+
446
+ 6.1.2
447
+ ---------
448
+
449
+ - Improve readability in dark mode Web UI [#4674]
450
+ - Fix Web UI crash with corrupt session [#4672]
451
+ - Allow middleware to yield arguments [#4673, @eugeneius]
452
+ - Migrate CI from CircleCI to GitHub Actions [#4677]
453
+
454
+ 6.1.1
455
+ ---------
456
+
457
+ - Jobs are now sorted by age in the Busy Workers table. [#4641]
458
+ - Fix "check all" JS logic in Web UI [#4619]
459
+
460
+ 6.1.0
461
+ ---------
462
+
463
+ - Web UI - Dark Mode fixes [#4543, natematykiewicz]
464
+ - Ensure `Rack::ContentLength` is loaded as middleware for correct Web UI responses [#4541]
465
+ - Avoid exception dumping SSL store in Redis connection logging [#4532]
466
+ - Better error messages in Sidekiq::Client [#4549]
467
+ - Remove rack-protection, reimplement CSRF protection [#4588]
468
+ - Require redis-rb 4.2 [#4591]
469
+ - Update to jquery 1.12.4 [#4593]
470
+ - Refactor internal fetch logic and API [#4602]
471
+
472
+ 6.0.7
473
+ ---------
474
+
475
+ - Refactor systemd integration to work better with custom binaries [#4511]
476
+ - Don't connect to Redis at process exit if not needed [#4502]
477
+ - Remove Redis connection naming [#4479]
478
+ - Fix Redis Sentinel password redaction [#4499]
479
+ - Add Vietnamese locale (vi) [#4528]
480
+
481
+ 6.0.6
482
+ ---------
483
+
484
+ - **Integrate with systemd's watchdog and notification features** [#4488]
485
+ Set `Type=notify` in [sidekiq.service](https://github.com/sidekiq/sidekiq/blob/4b8a8bd3ae42f6e48ae1fdaf95ed7d7af18ed8bb/examples/systemd/sidekiq.service#L30-L39). The integration works automatically.
486
+ - Use `setTimeout` rather than `setInterval` to avoid thundering herd [#4480]
487
+ - Fix edge case where a job can be pushed without a queue.
488
+ - Flush job stats at exit [#4498]
489
+ - Check RAILS_ENV before RACK_ENV [#4493]
490
+ - Add Lithuanian locale [#4476]
491
+
492
+ 6.0.5
493
+ ---------
494
+
495
+ - Fix broken Web UI response when using NewRelic and Rack 2.1.2+. [#4440]
496
+ - Update APIs to use `UNLINK`, not `DEL`. [#4449]
497
+ - Fix Ruby 2.7 warnings [#4412]
498
+ - Add support for `APP_ENV` [[95fa5d9]](https://github.com/sidekiq/sidekiq/commit/95fa5d90192148026e52ca2902f1b83c70858ce8)
499
+
500
+ 6.0.4
501
+ ---------
502
+
503
+ - Fix ActiveJob's `sidekiq_options` integration [#4404]
504
+ - Sidekiq Pro users will now see a Pause button next to each queue in
505
+ the Web UI, allowing them to pause queues manually [#4374, shayonj]
506
+ - Fix Sidekiq::Workers API unintentional change in 6.0.2 [#4387]
507
+
508
+
509
+ 6.0.3
510
+ ---------
511
+
512
+ - Fix `Sidekiq::Client.push_bulk` API which was erroneously putting
513
+ invalid `at` values in the job payloads [#4321]
514
+
515
+ 6.0.2
516
+ ---------
517
+
518
+ - Fix Sidekiq Enterprise's rolling restart functionality, broken by refactoring in 6.0.0. [#4334]
519
+ - More internal refactoring and performance tuning [fatkodima]
520
+
521
+ 6.0.1
522
+ ---------
523
+
524
+ - **Performance tuning**, Sidekiq should be 10-15% faster now [#4303, 4299,
525
+ 4269, fatkodima]
526
+ - **Dark Mode support in Web UI** (further design polish welcome!) [#4227, mperham,
527
+ fatkodima, silent-e]
528
+ - **Job-specific log levels**, allowing you to turn on debugging for
529
+ problematic workers. [fatkodima, #4287]
530
+ ```ruby
531
+ MyWorker.set(log_level: :debug).perform_async(...)
532
+ ```
533
+ - **Ad-hoc job tags**. You can tag your jobs with, e.g, subdomain, tenant, country,
534
+ locale, application, version, user/client, "alpha/beta/pro/ent", types of jobs,
535
+ teams/people responsible for jobs, additional metadata, etc.
536
+ Tags are shown on different pages with job listings. Sidekiq Pro users
537
+ can filter based on them [fatkodima, #4280]
538
+ ```ruby
539
+ class MyWorker
540
+ include Sidekiq::Worker
541
+ sidekiq_options tags: ['bank-ops', 'alpha']
542
+ ...
543
+ end
544
+ ```
545
+ - Fetch scheduled jobs in batches before pushing into specific queues.
546
+ This will decrease enqueueing time of scheduled jobs by a third. [fatkodima, #4273]
547
+ ```
548
+ ScheduledSet with 10,000 jobs
549
+ Before: 56.6 seconds
550
+ After: 39.2 seconds
551
+ ```
552
+ - Compress error backtraces before pushing into Redis, if you are
553
+ storing error backtraces, this will halve the size of your RetrySet
554
+ in Redis [fatkodima, #4272]
555
+ ```
556
+ RetrySet with 100,000 jobs
557
+ Before: 261 MB
558
+ After: 129 MB
559
+ ```
560
+ - Support display of ActiveJob 6.0 payloads in the Web UI [#4263]
561
+ - Add `SortedSet#scan` for pattern based scanning. For large sets this API will be **MUCH** faster
562
+ than standard iteration using each. [fatkodima, #4262]
563
+ ```ruby
564
+ Sidekiq::DeadSet.new.scan("UnreliableApi") do |job|
565
+ job.retry
566
+ end
567
+ ```
568
+ - Dramatically speed up SortedSet#find\_job(jid) by using Redis's ZSCAN
569
+ support, approx 10x faster. [fatkodima, #4259]
570
+ ```
571
+ zscan 0.179366 0.047727 0.227093 ( 1.161376)
572
+ enum 8.522311 0.419826 8.942137 ( 9.785079)
573
+ ```
574
+ - Respect rails' generators `test_framework` option and gracefully handle extra `worker` suffix on generator [fatkodima, #4256]
575
+ - Add ability to sort 'Enqueued' page on Web UI by position in the queue [fatkodima, #4248]
576
+ - Support `Client.push_bulk` with different delays [fatkodima, #4243]
577
+ ```ruby
578
+ Sidekiq::Client.push_bulk("class" => FooJob, "args" => [[1], [2]], "at" => [1.minute.from_now.to_f, 5.minutes.from_now.to_f])
579
+ ```
580
+ - Easier way to test enqueuing specific ActionMailer and ActiveRecord delayed jobs. Instead of manually
581
+ parsing embedded class, you can now test by fetching jobs for specific classes. [fatkodima, #4292]
582
+ ```ruby
583
+ assert_equal 1, Sidekiq::Extensions::DelayedMailer.jobs_for(FooMailer).size
584
+ ```
585
+ - Add `sidekiqmon` to gemspec executables [#4242]
586
+ - Gracefully handle `Sidekiq.logger = nil` [#4240]
587
+ - Inject Sidekiq::LogContext module if user-supplied logger does not include it [#4239]
588
+
589
+ 6.0
590
+ ---------
591
+
592
+ This release has major breaking changes. Read and test carefully in production.
593
+
594
+ - With Rails 6.0.2+, ActiveJobs can now use `sidekiq_options` directly to configure Sidekiq
595
+ features/internals like the retry subsystem. [#4213, pirj]
596
+ ```ruby
597
+ class MyJob < ActiveJob::Base
598
+ queue_as :myqueue
599
+ sidekiq_options retry: 10, backtrace: 20
600
+ def perform(...)
601
+ end
602
+ end
603
+ ```
604
+ - Logging has been redesigned to allow for pluggable log formatters:
605
+ ```ruby
606
+ Sidekiq.configure_server do |config|
607
+ config.log_formatter = Sidekiq::Logger::Formatters::JSON.new
608
+ end
609
+ ```
610
+ See the [Logging wiki page](https://github.com/sidekiq/sidekiq/wiki/Logging) for more details.
611
+ - **BREAKING CHANGE** Validate proper usage of the `REDIS_PROVIDER`
612
+ variable. This variable is meant to hold the name of the environment
613
+ variable which contains your Redis URL, so that you can switch Redis
614
+ providers quickly and easily with a single variable change. It is not
615
+ meant to hold the actual Redis URL itself. If you want to manually set
616
+ the Redis URL (not recommended as it implies you have no failover),
617
+ then you may set `REDIS_URL` directly. [#3969]
618
+ - **BREAKING CHANGE** Increase default shutdown timeout from 8 seconds
619
+ to 25 seconds. Both Heroku and ECS now use 30 second shutdown timeout
620
+ by default and we want Sidekiq to take advantage of this time. If you
621
+ have deployment scripts which depend on the old default timeout, use `-t 8` to
622
+ get the old behavior. [#3968]
623
+ - **BREAKING CHANGE** Remove the daemonization, logfile and pidfile
624
+ arguments to Sidekiq. Use a proper process supervisor (e.g. systemd or
625
+ foreman) to manage Sidekiq. See the Deployment wiki page for links to
626
+ more resources.
627
+ - Integrate the StandardRB code formatter to ensure consistent code
628
+ styling. [#4114, gearnode]
629
+
630
+ 5.2.10
631
+ ---------
632
+
633
+ - Backport fix for CVE-2022-23837.
634
+ - Migrate to `exists?` for redis-rb.
635
+ - Lock redis-rb to <4.6 to avoid deprecations.
636
+
637
+ 5.2.9
638
+ ---------
639
+
640
+ - Release Rack lock due to a cascade of CVEs. [#4566]
641
+ Pro-tip: don't lock Rack.
642
+
643
+ 5.2.8
644
+ ---------
645
+
646
+ - Lock to Rack 2.0.x to prevent future incompatibilities
647
+ - Fix invalid reference in `sidekiqctl`
648
+
649
+ 5.2.7
650
+ ---------
651
+
652
+ - Fix stale `enqueued_at` when retrying [#4149]
653
+ - Move build to [Circle CI](https://circleci.com/gh/mperham/sidekiq) [#4120]
654
+
655
+ 5.2.6
656
+ ---------
657
+
658
+ - Fix edge case where a job failure during Redis outage could result in a lost job [#4141]
659
+ - Better handling of malformed job arguments in payload [#4095]
660
+ - Restore bootstap's dropdown css component [#4099, urkle]
661
+ - Display human-friendly time diff for longer queue latencies [#4111, interlinked]
662
+ - Allow `Sidekiq::Worker#set` to be chained
663
+
664
+ 5.2.5
665
+ ---------
666
+
667
+ - Fix default usage of `config/sidekiq.yml` [#4077, Tensho]
4
668
 
5
669
  5.2.4
6
670
  ---------
@@ -135,7 +799,7 @@ Sidekiq::Middleware::Server::Logging -> Sidekiq::JobLogger
135
799
  - The `SomeWorker.set(options)` API was re-written to avoid thread-local state. [#2152]
136
800
  - Sidekiq Enterprise's encrypted jobs now display "[encrypted data]" in the Web UI instead
137
801
  of random hex bytes.
138
- - Please see the [5.0 Upgrade notes](5.0-Upgrade.md) for more detail.
802
+ - Please see the [5.0 Upgrade notes](docs/5.0-Upgrade.md) for more detail.
139
803
 
140
804
  4.2.10
141
805
  -----------
@@ -353,7 +1017,7 @@ Sidekiq::Queues.clear_all
353
1017
  - Sidekiq's internals have been completely overhauled for performance
354
1018
  and to remove dependencies. This has resulted in major speedups, as
355
1019
  [detailed on my blog](http://www.mikeperham.com/2015/10/14/optimizing-sidekiq/).
356
- - See the [4.0 upgrade notes](4.0-Upgrade.md) for more detail.
1020
+ - See the [4.0 upgrade notes](docs/4.0-Upgrade.md) for more detail.
357
1021
 
358
1022
  3.5.4
359
1023
  -----------
@@ -390,7 +1054,7 @@ Sidekiq::Queues.clear_all
390
1054
  - **FIX MEMORY LEAK** Under rare conditions, threads may leak [#2598, gazay]
391
1055
  - Add Ukrainian locale [#2561, elrakita]
392
1056
  - Disconnect and retry Redis operations if we see a READONLY error [#2550]
393
- - Add server middleware testing harness; see [wiki](https://github.com/mperham/sidekiq/wiki/Testing#testing-server-middleware) [#2534, ryansch]
1057
+ - Add server middleware testing harness; see [wiki](https://github.com/sidekiq/sidekiq/wiki/Testing#testing-server-middleware) [#2534, ryansch]
394
1058
 
395
1059
  3.5.0
396
1060
  -----------
@@ -408,7 +1072,7 @@ Sidekiq::Queues.clear_all
408
1072
  - Fix CSRF vulnerability in Web UI, thanks to Egor Homakov for
409
1073
  reporting. [#2422] If you are running the Web UI as a standalone Rack app,
410
1074
  ensure you have a [session middleware
411
- configured](https://github.com/mperham/sidekiq/wiki/Monitoring#standalone):
1075
+ configured](https://github.com/sidekiq/sidekiq/wiki/Monitoring#standalone):
412
1076
  ```ruby
413
1077
  use Rack::Session::Cookie, :secret => "some unique secret string here"
414
1078
  ```
@@ -620,7 +1284,7 @@ sidekiq_options :dead => false, :retry => 5
620
1284
  3.0.0
621
1285
  -----------
622
1286
 
623
- Please see [3.0-Upgrade.md](3.0-Upgrade.md) for more comprehensive upgrade notes.
1287
+ Please see [3.0-Upgrade.md](docs/3.0-Upgrade.md) for more comprehensive upgrade notes.
624
1288
 
625
1289
  - **Dead Job Queue** - jobs which run out of retries are now moved to a dead
626
1290
  job queue. These jobs must be retried manually or they will expire
@@ -664,7 +1328,7 @@ Sidekiq::Client.via(ConnectionPool.new { Redis.new }) do
664
1328
  end
665
1329
  ```
666
1330
  **Sharding support does require a breaking change to client-side
667
- middleware, see 3.0-Upgrade.md.**
1331
+ middleware, see docs/3.0-Upgrade.md.**
668
1332
  - New Chinese, Greek, Swedish and Czech translations for the Web UI.
669
1333
  - Updated most languages translations for the new UI features.
670
1334
  - **Remove official Capistrano integration** - this integration has been
@@ -792,7 +1456,7 @@ middleware, see 3.0-Upgrade.md.**
792
1456
  appear to be doing any work. [#1194]
793
1457
  - Sidekiq's testing behavior is now dynamic. You can choose between
794
1458
  `inline` and `fake` behavior in your tests. See
795
- [Testing](https://github.com/mperham/sidekiq/wiki/Testing) for detail. [#1193]
1459
+ [Testing](https://github.com/sidekiq/sidekiq/wiki/Testing) for detail. [#1193]
796
1460
  - The Retries table has a new column for the error message.
797
1461
  - The Web UI topbar now contains the status and live poll button.
798
1462
  - Orphaned worker records are now auto-vacuumed when you visit the