sidekiq 6.5.12 → 7.3.4
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/Changes.md +303 -20
- data/README.md +43 -35
- data/bin/multi_queue_bench +271 -0
- data/bin/sidekiq +3 -8
- data/bin/sidekiqload +204 -118
- data/bin/sidekiqmon +3 -0
- data/lib/active_job/queue_adapters/sidekiq_adapter.rb +88 -0
- data/lib/generators/sidekiq/job_generator.rb +2 -0
- data/lib/sidekiq/api.rb +196 -138
- data/lib/sidekiq/capsule.rb +132 -0
- data/lib/sidekiq/cli.rb +60 -75
- data/lib/sidekiq/client.rb +87 -38
- data/lib/sidekiq/component.rb +4 -1
- data/lib/sidekiq/config.rb +305 -0
- data/lib/sidekiq/deploy.rb +64 -0
- data/lib/sidekiq/embedded.rb +63 -0
- data/lib/sidekiq/fetch.rb +11 -14
- data/lib/sidekiq/iterable_job.rb +55 -0
- data/lib/sidekiq/job/interrupt_handler.rb +24 -0
- data/lib/sidekiq/job/iterable/active_record_enumerator.rb +53 -0
- data/lib/sidekiq/job/iterable/csv_enumerator.rb +47 -0
- data/lib/sidekiq/job/iterable/enumerators.rb +135 -0
- data/lib/sidekiq/job/iterable.rb +294 -0
- data/lib/sidekiq/job.rb +382 -10
- data/lib/sidekiq/job_logger.rb +23 -12
- data/lib/sidekiq/job_retry.rb +42 -19
- data/lib/sidekiq/job_util.rb +53 -15
- data/lib/sidekiq/launcher.rb +71 -65
- data/lib/sidekiq/logger.rb +2 -27
- data/lib/sidekiq/manager.rb +9 -11
- data/lib/sidekiq/metrics/query.rb +9 -4
- data/lib/sidekiq/metrics/shared.rb +21 -9
- data/lib/sidekiq/metrics/tracking.rb +40 -26
- data/lib/sidekiq/middleware/chain.rb +19 -18
- data/lib/sidekiq/middleware/current_attributes.rb +70 -20
- data/lib/sidekiq/middleware/modules.rb +2 -0
- data/lib/sidekiq/monitor.rb +18 -4
- data/lib/sidekiq/paginator.rb +2 -2
- data/lib/sidekiq/processor.rb +62 -57
- data/lib/sidekiq/rails.rb +21 -10
- data/lib/sidekiq/redis_client_adapter.rb +31 -71
- data/lib/sidekiq/redis_connection.rb +44 -115
- data/lib/sidekiq/ring_buffer.rb +2 -0
- data/lib/sidekiq/scheduled.rb +22 -23
- data/lib/sidekiq/systemd.rb +2 -0
- data/lib/sidekiq/testing.rb +37 -46
- data/lib/sidekiq/transaction_aware_client.rb +11 -5
- data/lib/sidekiq/version.rb +6 -1
- data/lib/sidekiq/web/action.rb +15 -5
- data/lib/sidekiq/web/application.rb +89 -17
- data/lib/sidekiq/web/csrf_protection.rb +10 -7
- data/lib/sidekiq/web/helpers.rb +102 -42
- data/lib/sidekiq/web/router.rb +5 -2
- data/lib/sidekiq/web.rb +65 -17
- data/lib/sidekiq/worker_compatibility_alias.rb +13 -0
- data/lib/sidekiq.rb +78 -274
- data/sidekiq.gemspec +12 -10
- data/web/assets/javascripts/application.js +44 -0
- data/web/assets/javascripts/base-charts.js +106 -0
- data/web/assets/javascripts/dashboard-charts.js +192 -0
- data/web/assets/javascripts/dashboard.js +11 -233
- data/web/assets/javascripts/metrics.js +151 -115
- data/web/assets/stylesheets/application-dark.css +4 -0
- data/web/assets/stylesheets/application-rtl.css +10 -89
- data/web/assets/stylesheets/application.css +53 -298
- data/web/locales/ar.yml +70 -70
- data/web/locales/cs.yml +62 -62
- data/web/locales/da.yml +60 -53
- data/web/locales/de.yml +65 -65
- data/web/locales/el.yml +2 -7
- data/web/locales/en.yml +78 -71
- data/web/locales/es.yml +68 -68
- data/web/locales/fa.yml +65 -65
- data/web/locales/fr.yml +80 -67
- data/web/locales/gd.yml +98 -0
- data/web/locales/he.yml +65 -64
- data/web/locales/hi.yml +59 -59
- data/web/locales/it.yml +53 -53
- data/web/locales/ja.yml +67 -70
- data/web/locales/ko.yml +52 -52
- data/web/locales/lt.yml +66 -66
- data/web/locales/nb.yml +61 -61
- data/web/locales/nl.yml +52 -52
- data/web/locales/pl.yml +45 -45
- data/web/locales/pt-br.yml +78 -69
- data/web/locales/pt.yml +51 -51
- data/web/locales/ru.yml +67 -66
- data/web/locales/sv.yml +53 -53
- data/web/locales/ta.yml +60 -60
- data/web/locales/tr.yml +100 -0
- data/web/locales/uk.yml +85 -61
- data/web/locales/ur.yml +64 -64
- data/web/locales/vi.yml +67 -67
- data/web/locales/zh-cn.yml +20 -19
- data/web/locales/zh-tw.yml +10 -2
- data/web/views/_footer.erb +17 -2
- data/web/views/_job_info.erb +18 -2
- data/web/views/_metrics_period_select.erb +12 -0
- data/web/views/_paging.erb +2 -0
- data/web/views/_poll_link.erb +1 -1
- data/web/views/_summary.erb +7 -7
- data/web/views/busy.erb +46 -35
- data/web/views/dashboard.erb +28 -7
- data/web/views/filtering.erb +7 -0
- data/web/views/layout.erb +6 -6
- data/web/views/metrics.erb +48 -26
- data/web/views/metrics_for_job.erb +43 -71
- data/web/views/morgue.erb +5 -9
- data/web/views/queue.erb +10 -14
- data/web/views/queues.erb +9 -3
- data/web/views/retries.erb +5 -9
- data/web/views/scheduled.erb +12 -13
- metadata +53 -39
- data/lib/sidekiq/delay.rb +0 -43
- data/lib/sidekiq/extensions/action_mailer.rb +0 -48
- data/lib/sidekiq/extensions/active_record.rb +0 -43
- data/lib/sidekiq/extensions/class_methods.rb +0 -43
- data/lib/sidekiq/extensions/generic_proxy.rb +0 -33
- data/lib/sidekiq/metrics/deploy.rb +0 -47
- data/lib/sidekiq/worker.rb +0 -370
- data/web/assets/javascripts/graph.js +0 -16
- /data/{LICENSE → LICENSE.txt} +0 -0
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 4e334af4f9c2d4bbf79ec6b72e2fe344d07f4660534ca4869ccdfa95274a0d58
|
4
|
+
data.tar.gz: 1cbea7241abbe7a85800e5e9df75491695d1c10a8af66f4dd7b84e174c23b7ea
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 8dd6fc27c641bf6ded12b52784dfdc99691d810d40a160f2a5dd19e0a49bf1530a6314b48bc248b6ec66127fa108c2186b00d4affe69505040306e0aca5a08eb
|
7
|
+
data.tar.gz: 7a108824b94fd1e80c62e510cc5efc9aba2b62b8af83c6d4fe4c7cca19eb5a4a17a46abe32b74f0ab2a1cb3187f5688a0058f8f63ef0b1121cb44faae50e1546
|
data/Changes.md
CHANGED
@@ -1,17 +1,301 @@
|
|
1
1
|
# Sidekiq Changes
|
2
2
|
|
3
|
-
[Sidekiq Changes](https://github.com/
|
3
|
+
[Sidekiq Changes](https://github.com/sidekiq/sidekiq/blob/main/Changes.md) | [Sidekiq Pro Changes](https://github.com/sidekiq/sidekiq/blob/main/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/sidekiq/sidekiq/blob/main/Ent-Changes.md)
|
4
4
|
|
5
|
-
|
5
|
+
7.3.4
|
6
6
|
----------
|
7
7
|
|
8
|
-
- Fix
|
8
|
+
- Fix FrozenError when starting Sidekiq [#6470]
|
9
9
|
|
10
|
-
|
10
|
+
7.3.3
|
11
11
|
----------
|
12
12
|
|
13
|
-
-
|
14
|
-
-
|
13
|
+
- Freeze global configuration once boot is complete, to avoid configuration race conditions [#6466, #6465]
|
14
|
+
- Sidekiq now warns if a job iteration takes longer than the `-t` timeout setting (defaults to 25 seconds)
|
15
|
+
- Iteration callbacks now have easy access to job arguments via the `arguments` method:
|
16
|
+
```ruby
|
17
|
+
def on_stop
|
18
|
+
p arguments # => `[123, "string", {"key" => "value"}]`
|
19
|
+
id, str, hash = arguments
|
20
|
+
end
|
21
|
+
```
|
22
|
+
- Iterable jobs can be cancelled via `Sidekiq::Client#cancel!`:
|
23
|
+
```ruby
|
24
|
+
c = Sidekiq::Client.new
|
25
|
+
jid = c.push("class" => SomeJob, "args" => [123])
|
26
|
+
c.cancel!(jid) # => true
|
27
|
+
```
|
28
|
+
- Take over support for ActiveJob's `:sidekiq` adapter [#6430, fatkodima]
|
29
|
+
- Ensure CurrentAttributes are in scope when creating batch callbacks [#6455]
|
30
|
+
- Add `Sidekiq.gem_version` API.
|
31
|
+
- Update Ukranian translations
|
32
|
+
|
33
|
+
7.3.2
|
34
|
+
----------
|
35
|
+
|
36
|
+
- Adjust ActiveRecord batch iteration to restart an interrupted batch from the beginning.
|
37
|
+
Each batch should be processed as a single transaction in order to be idempotent. [#6405]
|
38
|
+
- Fix typo in Sidekiq::DeadSet#kill [#6397]
|
39
|
+
- Fix CSS issue with bottom bar in Web UI [#6414]
|
40
|
+
|
41
|
+
7.3.1
|
42
|
+
----------
|
43
|
+
|
44
|
+
- Don't count job interruptions as failures in metrics [#6386]
|
45
|
+
- Add frozen string literal to a number of .rb files.
|
46
|
+
- Fix frozen string error with style_tag and script_tag [#6371]
|
47
|
+
- Fix an error on Ruby 2.7 because of usage of `Hash#except` [#6376]
|
48
|
+
|
49
|
+
7.3.0
|
50
|
+
----------
|
51
|
+
|
52
|
+
- **NEW FEATURE** Add `Sidekiq::IterableJob`, iteration support for long-running jobs. [#6286, fatkodima]
|
53
|
+
Iterable jobs are interruptible and can restart quickly if
|
54
|
+
running during a deploy. You must ensure that `each_iteration`
|
55
|
+
doesn't take more than Sidekiq's `-t` timeout (default: 25 seconds). Iterable jobs must not implement `perform`.
|
56
|
+
```ruby
|
57
|
+
class ProcessArrayJob
|
58
|
+
include Sidekiq::IterableJob
|
59
|
+
def build_enumerator(*args, **kwargs)
|
60
|
+
array_enumerator(args, **kwargs)
|
61
|
+
end
|
62
|
+
def each_iteration(arg)
|
63
|
+
puts arg
|
64
|
+
end
|
65
|
+
end
|
66
|
+
ProcessArrayJob.perform_async(1, 2, 3)
|
67
|
+
```
|
68
|
+
See the [Iteration](//github.com/sidekiq/sidekiq/wiki/Iteration) wiki page and the RDoc in `Sidekiq::IterableJob`.
|
69
|
+
This feature should be considered BETA until the next minor release.
|
70
|
+
- **SECURITY** The Web UI no longer allows extensions to use `<script>`.
|
71
|
+
Adjust CSP to disallow inline scripts within the Web UI. Please see
|
72
|
+
`examples/webui-ext` for how to register Web UI extensions and use
|
73
|
+
dynamic CSS and JS. This will make Sidekiq immune to XSS attacks. [#6270]
|
74
|
+
- Add config option, `:skip_default_job_logging` to disable Sidekiq's default
|
75
|
+
start/finish job logging. [#6200]
|
76
|
+
- Allow `Sidekiq::Limiter.redis` to use Redis Cluster [#6288]
|
77
|
+
- Retain CurrentAttributeѕ after inline execution [#6307]
|
78
|
+
- Ignore non-existent CurrentAttributes attributes when restoring [#6341]
|
79
|
+
- Raise default Redis {read,write,connect} timeouts from 1 to 3 seconds
|
80
|
+
to minimize ReadTimeoutErrors [#6162]
|
81
|
+
- Add `logger` as a dependency since it will become bundled in Ruby 3.5 [#6320]
|
82
|
+
- Ignore unsupported locales in the Web UI [#6313]
|
83
|
+
|
84
|
+
7.2.4
|
85
|
+
----------
|
86
|
+
|
87
|
+
- Fix XSS in metrics filtering introduced in 7.2.0, CVE-2024-32887
|
88
|
+
Thanks to @UmerAdeemCheema for the security report.
|
89
|
+
|
90
|
+
7.2.3
|
91
|
+
----------
|
92
|
+
|
93
|
+
- [Support Dragonfly.io](https://www.mikeperham.com/2024/02/01/supporting-dragonfly/) as an alternative Redis implementation
|
94
|
+
- Fix error unpacking some compressed error backtraces [#6241]
|
95
|
+
- Fix potential heartbeat data leak [#6227]
|
96
|
+
- Add ability to find a currently running work by jid [#6212, fatkodima]
|
97
|
+
|
98
|
+
7.2.2
|
99
|
+
----------
|
100
|
+
|
101
|
+
- Add `Process.warmup` call in Ruby 3.3+
|
102
|
+
- Batch jobs now skip transactional push [#6160]
|
103
|
+
|
104
|
+
7.2.1
|
105
|
+
----------
|
106
|
+
|
107
|
+
- Add `Sidekiq::Work` type which replaces the raw Hash as the third parameter in
|
108
|
+
`Sidekiq::WorkSet#each { |pid, tid, hash| ... }` [#6145]
|
109
|
+
- **DEPRECATED**: direct access to the attributes within the `hash` block parameter above.
|
110
|
+
The `Sidekiq::Work` instance contains accessor methods to get at the same data, e.g.
|
111
|
+
```ruby
|
112
|
+
work["queue"] # Old
|
113
|
+
work.queue # New
|
114
|
+
```
|
115
|
+
- Fix Ruby 3.3 warnings around `base64` gem [#6151, earlopain]
|
116
|
+
|
117
|
+
7.2.0
|
118
|
+
----------
|
119
|
+
|
120
|
+
- `sidekiq_retries_exhausted` can return `:discard` to avoid the deadset
|
121
|
+
and all death handlers [#6091]
|
122
|
+
- Metrics filtering by job class in Web UI [#5974]
|
123
|
+
- Better readability and formatting for numbers within the Web UI [#6080]
|
124
|
+
- Add explicit error if user code tries to nest test modes [#6078]
|
125
|
+
```ruby
|
126
|
+
Sidekiq::Testing.inline! # global setting
|
127
|
+
Sidekiq::Testing.fake! do # override within block
|
128
|
+
# ok
|
129
|
+
Sidekiq::Testing.inline! do # can't override the override
|
130
|
+
# not ok, nested
|
131
|
+
end
|
132
|
+
end
|
133
|
+
```
|
134
|
+
- **SECURITY** Forbid inline JavaScript execution in Web UI [#6074]
|
135
|
+
- Adjust redis-client adapter to avoid `method_missing` [#6083]
|
136
|
+
This can result in app code breaking if your app's Redis API usage was
|
137
|
+
depending on Sidekiq's adapter to correct invalid redis-client API usage.
|
138
|
+
One example:
|
139
|
+
```ruby
|
140
|
+
# bad, not redis-client native
|
141
|
+
# Unsupported command argument type: TrueClass (TypeError)
|
142
|
+
Sidekiq.redis { |c| c.set("key", "value", nx: true, ex: 15) }
|
143
|
+
# good
|
144
|
+
Sidekiq.redis { |c| c.set("key", "value", "nx", "ex", 15) }
|
145
|
+
```
|
146
|
+
|
147
|
+
7.1.6
|
148
|
+
----------
|
149
|
+
|
150
|
+
- The block forms of testing modes (inline, fake) are now thread-safe so you can have
|
151
|
+
a multithreaded test suite which uses different modes for different tests. [#6069]
|
152
|
+
- Fix breakage with non-Proc error handlers [#6065]
|
153
|
+
|
154
|
+
7.1.5
|
155
|
+
----------
|
156
|
+
|
157
|
+
- **FEATURE**: Job filtering within the Web UI. This feature has been open
|
158
|
+
sourced from Sidekiq Pro. [#6052]
|
159
|
+
- **API CHANGE** Error handlers now take three arguments `->(ex, context, config)`.
|
160
|
+
The previous calling convention will work until Sidekiq 8.0 but will print
|
161
|
+
out a deprecation warning. [#6051]
|
162
|
+
- Fix issue with the `batch_size` and `at` options in `S::Client.push_bulk` [#6040]
|
163
|
+
- Fix inline testing firing batch callbacks early [#6057]
|
164
|
+
- Use new log broadcast API in Rails 7.1 [#6054]
|
165
|
+
- Crash if user tries to use RESP2 `protocol: 2` [#6061]
|
166
|
+
|
167
|
+
7.1.4
|
168
|
+
----------
|
169
|
+
|
170
|
+
- Fix empty `retry_for` logic [#6035]
|
171
|
+
|
172
|
+
7.1.3
|
173
|
+
----------
|
174
|
+
|
175
|
+
- Add `sidekiq_options retry_for: 48.hours` to allow time-based retry windows [#6029]
|
176
|
+
- Support sidekiq_retry_in and sidekiq_retries_exhausted_block in ActiveJobs (#5994)
|
177
|
+
- Lowercase all Rack headers for Rack 3.0 [#5951]
|
178
|
+
- Validate Sidekiq::Web page refresh delay to avoid potential DoS,
|
179
|
+
CVE-2023-26141, thanks for reporting Keegan!
|
180
|
+
|
181
|
+
7.1.2
|
182
|
+
----------
|
183
|
+
|
184
|
+
- Mark Web UI assets as private so CDNs won't cache them [#5936]
|
185
|
+
- Fix stackoverflow when using Oj and the JSON log formatter [#5920]
|
186
|
+
- Remove spurious `enqueued_at` from scheduled ActiveJobs [#5937]
|
187
|
+
|
188
|
+
7.1.1
|
189
|
+
----------
|
190
|
+
|
191
|
+
- Support multiple CurrentAttributes [#5904]
|
192
|
+
- Speed up latency fetch with large queues on Redis <7 [#5910]
|
193
|
+
- Allow a larger default client pool [#5886]
|
194
|
+
- Ensure Sidekiq.options[:environment] == RAILS_ENV [#5932]
|
195
|
+
|
196
|
+
7.1.0
|
197
|
+
----------
|
198
|
+
|
199
|
+
- Improve display of ActiveJob arguments in Web UI [#5825, cover]
|
200
|
+
- Update `push_bulk` to push `batch_size` jobs at a time and allow laziness [#5827, fatkodima]
|
201
|
+
This allows Sidekiq::Client to push unlimited jobs as long as it has enough memory for the batch_size.
|
202
|
+
- Update `perform_bulk` to use `push_bulk` internally.
|
203
|
+
- Change return value of `push_bulk` to map 1-to-1 with arguments.
|
204
|
+
If you call `push_bulk(args: [[1], [2], [3]])`, you will now always get
|
205
|
+
an array of 3 values as the result: `["jid1", nil, "jid3"]` where nil means
|
206
|
+
that particular job did not push successfully (possibly due to middleware
|
207
|
+
stopping it). Previously nil values were removed so it was impossible to tell
|
208
|
+
which jobs pushed successfully and which did not.
|
209
|
+
- Migrate away from all deprecated Redis commands [#5788]
|
210
|
+
Sidekiq will now print a warning if you use one of those deprecated commands.
|
211
|
+
- Prefix all Sidekiq thread names [#5872]
|
212
|
+
|
213
|
+
7.0.9
|
214
|
+
----------
|
215
|
+
|
216
|
+
- Restore confirmation dialogs in Web UI [#5881, shevaun]
|
217
|
+
- Increase fetch timeout to minimize ReadTimeoutError [#5874]
|
218
|
+
- Reverse histogram tooltip ordering [#5868]
|
219
|
+
- Add Scottish Gaelic (gd) locale [#5867, GunChleoc]
|
220
|
+
|
221
|
+
7.0.8
|
222
|
+
----------
|
223
|
+
|
224
|
+
- **SECURITY** Sanitize `period` input parameter on Metrics pages.
|
225
|
+
Specially crafted values can lead to XSS. This functionality
|
226
|
+
was introduced in 7.0.4. Thank you to spercex @ huntr.dev [#5694]
|
227
|
+
- Add job hash as 3rd parameter to the `sidekiq_retry_in` block.
|
228
|
+
|
229
|
+
7.0.7
|
230
|
+
----------
|
231
|
+
|
232
|
+
- Fix redis-client API usage which could result in stuck Redis
|
233
|
+
connections [#5823]
|
234
|
+
- Fix AS::Duration with `sidekiq_retry_in` [#5806]
|
235
|
+
- Restore dumping config options on startup with `-v` [#5822]
|
236
|
+
|
237
|
+
7.0.5,7.0.6
|
238
|
+
----------
|
239
|
+
|
240
|
+
- More context for debugging json unsafe errors [#5787]
|
241
|
+
|
242
|
+
7.0.4
|
243
|
+
----------
|
244
|
+
|
245
|
+
- Performance and memory optimizations [#5768, fatkodima]
|
246
|
+
- Add 1-8 hour period selector to Metrics pages [#5694]
|
247
|
+
- Fix process display with `sidekiqmon` [#5733]
|
248
|
+
|
249
|
+
7.0.3
|
250
|
+
----------
|
251
|
+
|
252
|
+
- Don't warn about memory policy on Redis Enterprise [#5712]
|
253
|
+
- Don't allow Quiet/Stop on embedded Sidekiq instances [#5716]
|
254
|
+
- Fix `size: X` for configuring the default Redis pool size [#5702]
|
255
|
+
- Improve the display of queue weights on Busy page [#5642]
|
256
|
+
- Freeze CurrentAttributes on a job once initially set [#5692]
|
257
|
+
|
258
|
+
7.0.2
|
259
|
+
----------
|
260
|
+
|
261
|
+
- Improve compatibility with custom loggers [#5673]
|
262
|
+
- Add queue weights on Busy page [#5640]
|
263
|
+
- Add BID link on job_info page if job is part of a Batch [#5623]
|
264
|
+
- Allow custom extensions to add rows/links within Job detail pages [#5624]
|
265
|
+
```ruby
|
266
|
+
Sidekiq::Web.custom_job_info_rows << AddAccountLink.new
|
267
|
+
|
268
|
+
class AddAccountLink
|
269
|
+
include CGI::Util
|
270
|
+
def add_pair(job)
|
271
|
+
# yield a (name, value) pair
|
272
|
+
# You can include HTML tags and CSS, Sidekiq does not do any
|
273
|
+
# escaping so beware user data injection! Note how we use CGI's
|
274
|
+
# `h` escape helper.
|
275
|
+
aid = job["account_id"]
|
276
|
+
yield "Account", "<a href='/accounts/#{h aid}'>#{h aid}</a>" if aid
|
277
|
+
end
|
278
|
+
end
|
279
|
+
```
|
280
|
+
|
281
|
+
7.0.1
|
282
|
+
----------
|
283
|
+
|
284
|
+
- Allow an embedding process to reuse its own heartbeat thread
|
285
|
+
- Update zh-cn localization
|
286
|
+
|
287
|
+
7.0.0
|
288
|
+
----------
|
289
|
+
|
290
|
+
- Embedded mode!
|
291
|
+
- Capsules!!
|
292
|
+
- Job Execution metrics!!!
|
293
|
+
- See `docs/7.0-Upgrade.md` for release notes
|
294
|
+
|
295
|
+
6.5.{10,11,12}
|
296
|
+
----------
|
297
|
+
|
298
|
+
- Fixes for Rails 7.1 [#6067, #6070]
|
15
299
|
|
16
300
|
6.5.9
|
17
301
|
----------
|
@@ -24,8 +308,7 @@
|
|
24
308
|
- Fail if using a bad version of scout_apm [#5616]
|
25
309
|
- Add pagination to Busy page [#5556]
|
26
310
|
- Speed up WorkSet#each [#5559]
|
27
|
-
- Adjust CurrentAttributes to work with the String class name so we aren't referencing
|
28
|
-
the Class within a Rails initializer [#5536]
|
311
|
+
- Adjust CurrentAttributes to work with the String class name so we aren't referencing the Class within a Rails initializer [#5536]
|
29
312
|
|
30
313
|
6.5.7
|
31
314
|
----------
|
@@ -59,7 +342,7 @@ the Class within a Rails initializer [#5536]
|
|
59
342
|
6.5.2
|
60
343
|
----------
|
61
344
|
|
62
|
-
- [Job Metrics are under active development, help wanted!](https://github.com/
|
345
|
+
- [Job Metrics are under active development, help wanted!](https://github.com/sidekiq/sidekiq/wiki/Metrics#contributing) **BETA**
|
63
346
|
- Add `Context` column on queue page which shows any CurrentAttributes [#5450]
|
64
347
|
- `sidekiq_retry_in` may now return `:discard` or `:kill` to dynamically stop job retries [#5406]
|
65
348
|
- Smarter sorting of processes in /busy Web UI [#5398]
|
@@ -78,7 +361,7 @@ the Class within a Rails initializer [#5536]
|
|
78
361
|
- Substantial refactoring of Sidekiq server internals, part of a larger effort
|
79
362
|
to reduce Sidekiq's internal usage of global methods and data, see [docs/global_to_local.md](docs/global_to_local.md) and [docs/middleware.md](docs/middleware.md).
|
80
363
|
- **Add beta support for the `redis-client` gem**. This will become the default Redis driver in Sidekiq 7.0. [#5298]
|
81
|
-
Read more: https://github.com/
|
364
|
+
Read more: https://github.com/sidekiq/sidekiq/wiki/Using-redis-client
|
82
365
|
- **Add beta support for DB transaction-aware client** [#5291]
|
83
366
|
Add this line to your initializer and any jobs created during a transaction
|
84
367
|
will only be pushed to Redis **after the transaction commits**. You will need to add the
|
@@ -278,7 +561,7 @@ If this is a Rails app in API mode, you need to enable sessions.
|
|
278
561
|
---------
|
279
562
|
|
280
563
|
- **Integrate with systemd's watchdog and notification features** [#4488]
|
281
|
-
Set `Type=notify` in [sidekiq.service](https://github.com/
|
564
|
+
Set `Type=notify` in [sidekiq.service](https://github.com/sidekiq/sidekiq/blob/4b8a8bd3ae42f6e48ae1fdaf95ed7d7af18ed8bb/examples/systemd/sidekiq.service#L30-L39). The integration works automatically.
|
282
565
|
- Use `setTimeout` rather than `setInterval` to avoid thundering herd [#4480]
|
283
566
|
- Fix edge case where a job can be pushed without a queue.
|
284
567
|
- Flush job stats at exit [#4498]
|
@@ -291,7 +574,7 @@ If this is a Rails app in API mode, you need to enable sessions.
|
|
291
574
|
- Fix broken Web UI response when using NewRelic and Rack 2.1.2+. [#4440]
|
292
575
|
- Update APIs to use `UNLINK`, not `DEL`. [#4449]
|
293
576
|
- Fix Ruby 2.7 warnings [#4412]
|
294
|
-
- Add support for `APP_ENV` [[95fa5d9]](https://github.com/
|
577
|
+
- Add support for `APP_ENV` [[95fa5d9]](https://github.com/sidekiq/sidekiq/commit/95fa5d90192148026e52ca2902f1b83c70858ce8)
|
295
578
|
|
296
579
|
6.0.4
|
297
580
|
---------
|
@@ -403,7 +686,7 @@ Sidekiq.configure_server do |config|
|
|
403
686
|
config.log_formatter = Sidekiq::Logger::Formatters::JSON.new
|
404
687
|
end
|
405
688
|
```
|
406
|
-
See the [Logging wiki page](https://github.com/
|
689
|
+
See the [Logging wiki page](https://github.com/sidekiq/sidekiq/wiki/Logging) for more details.
|
407
690
|
- **BREAKING CHANGE** Validate proper usage of the `REDIS_PROVIDER`
|
408
691
|
variable. This variable is meant to hold the name of the environment
|
409
692
|
variable which contains your Redis URL, so that you can switch Redis
|
@@ -595,7 +878,7 @@ Sidekiq::Middleware::Server::Logging -> Sidekiq::JobLogger
|
|
595
878
|
- The `SomeWorker.set(options)` API was re-written to avoid thread-local state. [#2152]
|
596
879
|
- Sidekiq Enterprise's encrypted jobs now display "[encrypted data]" in the Web UI instead
|
597
880
|
of random hex bytes.
|
598
|
-
- Please see the [5.0 Upgrade notes](5.0-Upgrade.md) for more detail.
|
881
|
+
- Please see the [5.0 Upgrade notes](docs/5.0-Upgrade.md) for more detail.
|
599
882
|
|
600
883
|
4.2.10
|
601
884
|
-----------
|
@@ -813,7 +1096,7 @@ Sidekiq::Queues.clear_all
|
|
813
1096
|
- Sidekiq's internals have been completely overhauled for performance
|
814
1097
|
and to remove dependencies. This has resulted in major speedups, as
|
815
1098
|
[detailed on my blog](http://www.mikeperham.com/2015/10/14/optimizing-sidekiq/).
|
816
|
-
- See the [4.0 upgrade notes](4.0-Upgrade.md) for more detail.
|
1099
|
+
- See the [4.0 upgrade notes](docs/4.0-Upgrade.md) for more detail.
|
817
1100
|
|
818
1101
|
3.5.4
|
819
1102
|
-----------
|
@@ -850,7 +1133,7 @@ Sidekiq::Queues.clear_all
|
|
850
1133
|
- **FIX MEMORY LEAK** Under rare conditions, threads may leak [#2598, gazay]
|
851
1134
|
- Add Ukrainian locale [#2561, elrakita]
|
852
1135
|
- Disconnect and retry Redis operations if we see a READONLY error [#2550]
|
853
|
-
- Add server middleware testing harness; see [wiki](https://github.com/
|
1136
|
+
- Add server middleware testing harness; see [wiki](https://github.com/sidekiq/sidekiq/wiki/Testing#testing-server-middleware) [#2534, ryansch]
|
854
1137
|
|
855
1138
|
3.5.0
|
856
1139
|
-----------
|
@@ -868,7 +1151,7 @@ Sidekiq::Queues.clear_all
|
|
868
1151
|
- Fix CSRF vulnerability in Web UI, thanks to Egor Homakov for
|
869
1152
|
reporting. [#2422] If you are running the Web UI as a standalone Rack app,
|
870
1153
|
ensure you have a [session middleware
|
871
|
-
configured](https://github.com/
|
1154
|
+
configured](https://github.com/sidekiq/sidekiq/wiki/Monitoring#standalone):
|
872
1155
|
```ruby
|
873
1156
|
use Rack::Session::Cookie, :secret => "some unique secret string here"
|
874
1157
|
```
|
@@ -1080,7 +1363,7 @@ sidekiq_options :dead => false, :retry => 5
|
|
1080
1363
|
3.0.0
|
1081
1364
|
-----------
|
1082
1365
|
|
1083
|
-
Please see [3.0-Upgrade.md](3.0-Upgrade.md) for more comprehensive upgrade notes.
|
1366
|
+
Please see [3.0-Upgrade.md](docs/3.0-Upgrade.md) for more comprehensive upgrade notes.
|
1084
1367
|
|
1085
1368
|
- **Dead Job Queue** - jobs which run out of retries are now moved to a dead
|
1086
1369
|
job queue. These jobs must be retried manually or they will expire
|
@@ -1124,7 +1407,7 @@ Sidekiq::Client.via(ConnectionPool.new { Redis.new }) do
|
|
1124
1407
|
end
|
1125
1408
|
```
|
1126
1409
|
**Sharding support does require a breaking change to client-side
|
1127
|
-
middleware, see 3.0-Upgrade.md.**
|
1410
|
+
middleware, see docs/3.0-Upgrade.md.**
|
1128
1411
|
- New Chinese, Greek, Swedish and Czech translations for the Web UI.
|
1129
1412
|
- Updated most languages translations for the new UI features.
|
1130
1413
|
- **Remove official Capistrano integration** - this integration has been
|
@@ -1252,7 +1535,7 @@ middleware, see 3.0-Upgrade.md.**
|
|
1252
1535
|
appear to be doing any work. [#1194]
|
1253
1536
|
- Sidekiq's testing behavior is now dynamic. You can choose between
|
1254
1537
|
`inline` and `fake` behavior in your tests. See
|
1255
|
-
[Testing](https://github.com/
|
1538
|
+
[Testing](https://github.com/sidekiq/sidekiq/wiki/Testing) for detail. [#1193]
|
1256
1539
|
- The Retries table has a new column for the error message.
|
1257
1540
|
- The Web UI topbar now contains the status and live poll button.
|
1258
1541
|
- Orphaned worker records are now auto-vacuumed when you visit the
|
data/README.md
CHANGED
@@ -2,7 +2,7 @@ Sidekiq
|
|
2
2
|
==============
|
3
3
|
|
4
4
|
[![Gem Version](https://badge.fury.io/rb/sidekiq.svg)](https://rubygems.org/gems/sidekiq)
|
5
|
-
![Build](https://github.com/
|
5
|
+
![Build](https://github.com/sidekiq/sidekiq/workflows/CI/badge.svg)
|
6
6
|
|
7
7
|
Simple, efficient background processing for Ruby.
|
8
8
|
|
@@ -10,28 +10,15 @@ Sidekiq uses threads to handle many jobs at the same time in the
|
|
10
10
|
same process. It does not require Rails but will integrate tightly with
|
11
11
|
Rails to make background processing dead simple.
|
12
12
|
|
13
|
-
Performance
|
14
|
-
---------------
|
15
|
-
|
16
|
-
Version | Latency | Garbage created for 10k jobs | Time to process 100k jobs | Throughput | Ruby
|
17
|
-
-----------------|------|---------|---------|------------------------|-----
|
18
|
-
Sidekiq 6.0.2 | 3 ms | 156 MB | 14.0 sec| **7100 jobs/sec** | MRI 2.6.3
|
19
|
-
Sidekiq 6.0.0 | 3 ms | 156 MB | 19 sec | 5200 jobs/sec | MRI 2.6.3
|
20
|
-
Sidekiq 4.0.0 | 10 ms | 151 MB | 22 sec | 4500 jobs/sec |
|
21
|
-
Sidekiq 3.5.1 | 22 ms | 1257 MB | 125 sec | 800 jobs/sec |
|
22
|
-
Resque 1.25.2 | - | - | 420 sec | 240 jobs/sec |
|
23
|
-
DelayedJob 4.1.1 | - | - | 465 sec | 215 jobs/sec |
|
24
|
-
|
25
|
-
This benchmark can be found in `bin/sidekiqload` and assumes a Redis network latency of 1ms.
|
26
13
|
|
27
14
|
Requirements
|
28
15
|
-----------------
|
29
16
|
|
30
|
-
- Redis:
|
31
|
-
- Ruby: MRI 2.
|
32
|
-
|
33
|
-
Sidekiq 6.0 supports Rails 5.0+ but does not require it.
|
17
|
+
- Redis: Redis 6.2+ or Dragonfly 1.13+
|
18
|
+
- Ruby: MRI 2.7+ or JRuby 9.3+.
|
34
19
|
|
20
|
+
Sidekiq 7.0 supports Rails 6.0+ but does not require it.
|
21
|
+
As of 7.2, Sidekiq supports Dragonfly as an alternative to Redis for data storage.
|
35
22
|
|
36
23
|
Installation
|
37
24
|
-----------------
|
@@ -42,57 +29,78 @@ Installation
|
|
42
29
|
Getting Started
|
43
30
|
-----------------
|
44
31
|
|
45
|
-
See the [Getting Started wiki page](https://github.com/
|
32
|
+
See the [Getting Started wiki page](https://github.com/sidekiq/sidekiq/wiki/Getting-Started) and follow the simple setup process.
|
46
33
|
You can watch [this YouTube playlist](https://www.youtube.com/playlist?list=PLjeHh2LSCFrWGT5uVjUuFKAcrcj5kSai1) to learn all about
|
47
34
|
Sidekiq and see its features in action. Here's the Web UI:
|
48
35
|
|
49
|
-
![Web UI](https://github.com/
|
36
|
+
![Web UI](https://github.com/sidekiq/sidekiq/raw/main/examples/web-ui.png)
|
37
|
+
|
38
|
+
Performance
|
39
|
+
---------------
|
40
|
+
|
41
|
+
The benchmark in `bin/sidekiqload` creates 500,000 no-op jobs and drains them as fast as possible, assuming a fixed Redis network latency of 1ms.
|
42
|
+
This requires a lot of Redis network I/O and JSON parsing.
|
43
|
+
This benchmark is IO-bound so we increase the concurrency to 25.
|
44
|
+
If your application is sending lots of emails or performing other network-intensive work, you could see a similar benefit but be careful not to saturate the CPU.
|
50
45
|
|
46
|
+
Version | Time to process 500k jobs | Throughput (jobs/sec) | Ruby | Concurrency | Job Type
|
47
|
+
-----------------|------|---------|---------|------------------------|---
|
48
|
+
Sidekiq 7.0.3 | 21.3 sec| 23,500 | 3.2.0+yjit | 30 | Sidekiq::Job
|
49
|
+
Sidekiq 7.0.3 | 33.8 sec| 14,700 | 3.2.0+yjit | 30 | ActiveJob 7.0.4
|
50
|
+
Sidekiq 7.0.3 | 23.5 sec| 21,300 | 3.2.0 | 30 | Sidekiq::Job
|
51
|
+
Sidekiq 7.0.3 | 46.5 sec| 10,700 | 3.2.0 | 30 | ActiveJob 7.0.4
|
52
|
+
Sidekiq 7.0.3 | 23.0 sec| 21,700 | 2.7.5 | 30 | Sidekiq::Job
|
53
|
+
Sidekiq 7.0.3 | 46.5 sec| 10,850 | 2.7.5 | 30 | ActiveJob 7.0.4
|
54
|
+
|
55
|
+
Most of Sidekiq's overhead is Redis network I/O.
|
56
|
+
ActiveJob adds a notable amount of CPU overhead due to argument deserialization and callbacks.
|
57
|
+
Concurrency of 30 was determined experimentally to maximize one CPU without saturating it.
|
51
58
|
|
52
59
|
Want to Upgrade?
|
53
60
|
-------------------
|
54
61
|
|
62
|
+
Use `bundle up sidekiq` to upgrade Sidekiq and all its dependencies.
|
63
|
+
Upgrade notes between each major version can be found in the `docs/` directory.
|
64
|
+
|
55
65
|
I also sell Sidekiq Pro and Sidekiq Enterprise, extensions to Sidekiq which provide more
|
56
66
|
features, a commercial-friendly license and allow you to support high
|
57
67
|
quality open source development all at the same time. Please see the
|
58
68
|
[Sidekiq](https://sidekiq.org/) homepage for more detail.
|
59
69
|
|
60
|
-
Subscribe to the **[quarterly newsletter](https://tinyletter.com/sidekiq)** to stay informed about the latest
|
61
|
-
features and changes to Sidekiq and its bigger siblings.
|
62
|
-
|
63
70
|
|
64
71
|
Problems?
|
65
72
|
-----------------
|
66
73
|
|
67
|
-
**Please do not directly email any Sidekiq committers with questions or problems.**
|
74
|
+
**Please do not directly email any Sidekiq committers with questions or problems.**
|
75
|
+
A community is best served when discussions are held in public.
|
68
76
|
|
69
|
-
If you have a problem, please review the [FAQ](https://github.com/
|
70
|
-
Searching the [issues](https://github.com/
|
77
|
+
If you have a problem, please review the [FAQ](https://github.com/sidekiq/sidekiq/wiki/FAQ) and [Troubleshooting](https://github.com/sidekiq/sidekiq/wiki/Problems-and-Troubleshooting) wiki pages.
|
78
|
+
Searching the [issues](https://github.com/sidekiq/sidekiq/issues) for your problem is also a good idea.
|
71
79
|
|
72
|
-
Sidekiq Pro and Sidekiq Enterprise customers get private email support.
|
80
|
+
Sidekiq Pro and Sidekiq Enterprise customers get private email support.
|
81
|
+
You can purchase at https://sidekiq.org; email support@contribsys.com for help.
|
73
82
|
|
74
83
|
Useful resources:
|
75
84
|
|
76
|
-
* Product documentation is in the [wiki](https://github.com/
|
77
|
-
* Occasional announcements are made to the [@sidekiq](https://
|
85
|
+
* Product documentation is in the [wiki](https://github.com/sidekiq/sidekiq/wiki).
|
86
|
+
* Occasional announcements are made to the [@sidekiq](https://ruby.social/@sidekiq) Mastodon account.
|
78
87
|
* The [Sidekiq tag](https://stackoverflow.com/questions/tagged/sidekiq) on Stack Overflow has lots of useful Q & A.
|
79
88
|
|
80
|
-
Every
|
89
|
+
Every Thursday morning is Sidekiq office hour: I video chat and answer questions.
|
81
90
|
See the [Sidekiq support page](https://sidekiq.org/support.html) for details.
|
82
91
|
|
83
92
|
Contributing
|
84
93
|
-----------------
|
85
94
|
|
86
|
-
Please see [the contributing guidelines](https://github.com/
|
87
|
-
|
95
|
+
Please see [the contributing guidelines](https://github.com/sidekiq/sidekiq/blob/main/.github/contributing.md).
|
88
96
|
|
89
97
|
License
|
90
98
|
-----------------
|
91
99
|
|
92
|
-
Please see [LICENSE](https://github.com/
|
93
|
-
|
100
|
+
Please see [LICENSE.txt](https://github.com/sidekiq/sidekiq/blob/main/LICENSE.txt) for licensing details.
|
101
|
+
The license for Sidekiq Pro and Sidekiq Enterprise can be found in [COMM-LICENSE.txt](https://github.com/sidekiq/sidekiq/blob/main/COMM-LICENSE.txt).
|
94
102
|
|
95
103
|
Author
|
96
104
|
-----------------
|
97
105
|
|
98
|
-
Mike Perham, [@getajobmike](https://
|
106
|
+
Mike Perham, [@getajobmike](https://ruby.social/@getajobmike) / [@sidekiq](https://ruby.social/@sidekiq), [https://www.mikeperham.com](https://www.mikeperham.com) / [https://www.contribsys.com](https://www.contribsys.com)
|