sidekiq 6.5.1 → 7.3.6
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/Changes.md +376 -12
- data/README.md +43 -35
- data/bin/multi_queue_bench +271 -0
- data/bin/sidekiq +3 -8
- data/bin/sidekiqload +213 -118
- data/bin/sidekiqmon +3 -0
- data/lib/active_job/queue_adapters/sidekiq_adapter.rb +88 -0
- data/lib/generators/sidekiq/job_generator.rb +2 -0
- data/lib/sidekiq/api.rb +378 -173
- data/lib/sidekiq/capsule.rb +132 -0
- data/lib/sidekiq/cli.rb +61 -63
- data/lib/sidekiq/client.rb +89 -40
- data/lib/sidekiq/component.rb +6 -2
- data/lib/sidekiq/config.rb +305 -0
- data/lib/sidekiq/deploy.rb +64 -0
- data/lib/sidekiq/embedded.rb +63 -0
- data/lib/sidekiq/fetch.rb +11 -14
- data/lib/sidekiq/iterable_job.rb +55 -0
- data/lib/sidekiq/job/interrupt_handler.rb +24 -0
- data/lib/sidekiq/job/iterable/active_record_enumerator.rb +53 -0
- data/lib/sidekiq/job/iterable/csv_enumerator.rb +47 -0
- data/lib/sidekiq/job/iterable/enumerators.rb +135 -0
- data/lib/sidekiq/job/iterable.rb +294 -0
- data/lib/sidekiq/job.rb +382 -10
- data/lib/sidekiq/job_logger.rb +8 -7
- data/lib/sidekiq/job_retry.rb +89 -46
- data/lib/sidekiq/job_util.rb +53 -15
- data/lib/sidekiq/launcher.rb +77 -69
- data/lib/sidekiq/logger.rb +2 -27
- data/lib/sidekiq/manager.rb +9 -11
- data/lib/sidekiq/metrics/query.rb +158 -0
- data/lib/sidekiq/metrics/shared.rb +106 -0
- data/lib/sidekiq/metrics/tracking.rb +148 -0
- data/lib/sidekiq/middleware/chain.rb +84 -48
- data/lib/sidekiq/middleware/current_attributes.rb +87 -20
- data/lib/sidekiq/middleware/modules.rb +2 -0
- data/lib/sidekiq/monitor.rb +19 -5
- data/lib/sidekiq/paginator.rb +11 -3
- data/lib/sidekiq/processor.rb +67 -56
- data/lib/sidekiq/rails.rb +22 -16
- data/lib/sidekiq/redis_client_adapter.rb +31 -71
- data/lib/sidekiq/redis_connection.rb +44 -117
- data/lib/sidekiq/ring_buffer.rb +2 -0
- data/lib/sidekiq/scheduled.rb +62 -35
- data/lib/sidekiq/systemd.rb +2 -0
- data/lib/sidekiq/testing.rb +37 -46
- data/lib/sidekiq/transaction_aware_client.rb +11 -5
- data/lib/sidekiq/version.rb +6 -1
- data/lib/sidekiq/web/action.rb +15 -5
- data/lib/sidekiq/web/application.rb +94 -24
- data/lib/sidekiq/web/csrf_protection.rb +10 -7
- data/lib/sidekiq/web/helpers.rb +118 -45
- data/lib/sidekiq/web/router.rb +5 -2
- data/lib/sidekiq/web.rb +67 -15
- data/lib/sidekiq/worker_compatibility_alias.rb +13 -0
- data/lib/sidekiq.rb +78 -266
- data/sidekiq.gemspec +12 -10
- data/web/assets/javascripts/application.js +46 -1
- data/web/assets/javascripts/base-charts.js +106 -0
- data/web/assets/javascripts/chart.min.js +13 -0
- data/web/assets/javascripts/chartjs-plugin-annotation.min.js +7 -0
- data/web/assets/javascripts/dashboard-charts.js +192 -0
- data/web/assets/javascripts/dashboard.js +11 -250
- data/web/assets/javascripts/metrics.js +298 -0
- data/web/assets/stylesheets/application-dark.css +4 -0
- data/web/assets/stylesheets/application-rtl.css +10 -89
- data/web/assets/stylesheets/application.css +98 -295
- data/web/locales/ar.yml +70 -70
- data/web/locales/cs.yml +62 -62
- data/web/locales/da.yml +60 -53
- data/web/locales/de.yml +65 -65
- data/web/locales/el.yml +43 -24
- data/web/locales/en.yml +83 -69
- data/web/locales/es.yml +68 -68
- data/web/locales/fa.yml +65 -65
- data/web/locales/fr.yml +80 -67
- data/web/locales/gd.yml +98 -0
- data/web/locales/he.yml +65 -64
- data/web/locales/hi.yml +59 -59
- data/web/locales/it.yml +85 -54
- data/web/locales/ja.yml +72 -68
- data/web/locales/ko.yml +52 -52
- data/web/locales/lt.yml +66 -66
- data/web/locales/nb.yml +61 -61
- data/web/locales/nl.yml +52 -52
- data/web/locales/pl.yml +45 -45
- data/web/locales/pt-br.yml +78 -69
- data/web/locales/pt.yml +51 -51
- data/web/locales/ru.yml +67 -66
- data/web/locales/sv.yml +53 -53
- data/web/locales/ta.yml +60 -60
- data/web/locales/tr.yml +100 -0
- data/web/locales/uk.yml +85 -61
- data/web/locales/ur.yml +64 -64
- data/web/locales/vi.yml +67 -67
- data/web/locales/zh-cn.yml +42 -16
- data/web/locales/zh-tw.yml +41 -8
- data/web/views/_footer.erb +17 -2
- data/web/views/_job_info.erb +18 -2
- data/web/views/_metrics_period_select.erb +12 -0
- data/web/views/_nav.erb +1 -1
- data/web/views/_paging.erb +2 -0
- data/web/views/_poll_link.erb +1 -1
- data/web/views/_summary.erb +7 -7
- data/web/views/busy.erb +49 -33
- data/web/views/dashboard.erb +28 -6
- data/web/views/filtering.erb +6 -0
- data/web/views/layout.erb +6 -6
- data/web/views/metrics.erb +90 -0
- data/web/views/metrics_for_job.erb +59 -0
- data/web/views/morgue.erb +5 -9
- data/web/views/queue.erb +15 -15
- data/web/views/queues.erb +9 -3
- data/web/views/retries.erb +5 -9
- data/web/views/scheduled.erb +12 -13
- metadata +61 -26
- data/lib/sidekiq/.DS_Store +0 -0
- data/lib/sidekiq/delay.rb +0 -43
- data/lib/sidekiq/extensions/action_mailer.rb +0 -48
- data/lib/sidekiq/extensions/active_record.rb +0 -43
- data/lib/sidekiq/extensions/class_methods.rb +0 -43
- data/lib/sidekiq/extensions/generic_proxy.rb +0 -33
- data/lib/sidekiq/worker.rb +0 -367
- /data/{LICENSE → LICENSE.txt} +0 -0
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 6c9bd0a03b27535a34a0525397e84c429a9332e6754dba43c088558ab8f362e9
|
4
|
+
data.tar.gz: '08fee2db2d83996d1df6f78e7eb9a226cb98d951cba336dd422771f9127cf95a'
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: f4876eb7594d0c87e4dba08a752dbcb65bbf45ae8213fb9b9ee6a1e12d7a48750c161ffe632f3b13eacfed73de3c3c73a9dc06a2548ddf7e5f6dde0f7de161bd
|
7
|
+
data.tar.gz: a0f485adef4ba6751114daf0d2c1d509966d530bc620e00ebf17a9c244f1eb0821ae5e8214d88d3652106d932cae7acfd2b66874647acc8c33fe6ec2e3b9d691
|
data/Changes.md
CHANGED
@@ -1,6 +1,370 @@
|
|
1
1
|
# Sidekiq Changes
|
2
2
|
|
3
|
-
[Sidekiq Changes](https://github.com/
|
3
|
+
[Sidekiq Changes](https://github.com/sidekiq/sidekiq/blob/main/Changes.md) | [Sidekiq Pro Changes](https://github.com/sidekiq/sidekiq/blob/main/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/sidekiq/sidekiq/blob/main/Ent-Changes.md)
|
4
|
+
|
5
|
+
7.3.6
|
6
|
+
----------
|
7
|
+
|
8
|
+
- Forward compatibility fixes for Ruby 3.4
|
9
|
+
- Filtering in the Web UI now works via GET so you can bookmark a filtered view. [#6497]
|
10
|
+
|
11
|
+
7.3.5
|
12
|
+
----------
|
13
|
+
|
14
|
+
- Reimplement `retry_all` and `kill_all` API methods to use ZPOPMIN,
|
15
|
+
approximately 30-60% faster. [#6481]
|
16
|
+
- Add preload testing binary at `examples/testing/sidekiq_boot` to verify your Rails app boots correctly with Sidekiq Enterprise's app preloading.
|
17
|
+
- Fix circular require with ActiveJob adapter [#6477]
|
18
|
+
- Fix potential race condition leading to incorrect serialized values for CurrentAttributes [#6475]
|
19
|
+
- Restore missing elapsed time when default job logging is disabled
|
20
|
+
|
21
|
+
7.3.4
|
22
|
+
----------
|
23
|
+
|
24
|
+
- Fix FrozenError when starting Sidekiq [#6470]
|
25
|
+
|
26
|
+
7.3.3
|
27
|
+
----------
|
28
|
+
|
29
|
+
- Freeze global configuration once boot is complete, to avoid configuration race conditions [#6466, #6465]
|
30
|
+
- Sidekiq now warns if a job iteration takes longer than the `-t` timeout setting (defaults to 25 seconds)
|
31
|
+
- Iteration callbacks now have easy access to job arguments via the `arguments` method:
|
32
|
+
```ruby
|
33
|
+
def on_stop
|
34
|
+
p arguments # => `[123, "string", {"key" => "value"}]`
|
35
|
+
id, str, hash = arguments
|
36
|
+
end
|
37
|
+
```
|
38
|
+
- Iterable jobs can be cancelled via `Sidekiq::Client#cancel!`:
|
39
|
+
```ruby
|
40
|
+
c = Sidekiq::Client.new
|
41
|
+
jid = c.push("class" => SomeJob, "args" => [123])
|
42
|
+
c.cancel!(jid) # => true
|
43
|
+
```
|
44
|
+
- Take over support for ActiveJob's `:sidekiq` adapter [#6430, fatkodima]
|
45
|
+
- Ensure CurrentAttributes are in scope when creating batch callbacks [#6455]
|
46
|
+
- Add `Sidekiq.gem_version` API.
|
47
|
+
- Update Ukranian translations
|
48
|
+
|
49
|
+
7.3.2
|
50
|
+
----------
|
51
|
+
|
52
|
+
- Adjust ActiveRecord batch iteration to restart an interrupted batch from the beginning.
|
53
|
+
Each batch should be processed as a single transaction in order to be idempotent. [#6405]
|
54
|
+
- Fix typo in Sidekiq::DeadSet#kill [#6397]
|
55
|
+
- Fix CSS issue with bottom bar in Web UI [#6414]
|
56
|
+
|
57
|
+
7.3.1
|
58
|
+
----------
|
59
|
+
|
60
|
+
- Don't count job interruptions as failures in metrics [#6386]
|
61
|
+
- Add frozen string literal to a number of .rb files.
|
62
|
+
- Fix frozen string error with style_tag and script_tag [#6371]
|
63
|
+
- Fix an error on Ruby 2.7 because of usage of `Hash#except` [#6376]
|
64
|
+
|
65
|
+
7.3.0
|
66
|
+
----------
|
67
|
+
|
68
|
+
- **NEW FEATURE** Add `Sidekiq::IterableJob`, iteration support for long-running jobs. [#6286, fatkodima]
|
69
|
+
Iterable jobs are interruptible and can restart quickly if
|
70
|
+
running during a deploy. You must ensure that `each_iteration`
|
71
|
+
doesn't take more than Sidekiq's `-t` timeout (default: 25 seconds). Iterable jobs must not implement `perform`.
|
72
|
+
```ruby
|
73
|
+
class ProcessArrayJob
|
74
|
+
include Sidekiq::IterableJob
|
75
|
+
def build_enumerator(*args, **kwargs)
|
76
|
+
array_enumerator(args, **kwargs)
|
77
|
+
end
|
78
|
+
def each_iteration(arg)
|
79
|
+
puts arg
|
80
|
+
end
|
81
|
+
end
|
82
|
+
ProcessArrayJob.perform_async(1, 2, 3)
|
83
|
+
```
|
84
|
+
See the [Iteration](//github.com/sidekiq/sidekiq/wiki/Iteration) wiki page and the RDoc in `Sidekiq::IterableJob`.
|
85
|
+
This feature should be considered BETA until the next minor release.
|
86
|
+
- **SECURITY** The Web UI no longer allows extensions to use `<script>`.
|
87
|
+
Adjust CSP to disallow inline scripts within the Web UI. Please see
|
88
|
+
`examples/webui-ext` for how to register Web UI extensions and use
|
89
|
+
dynamic CSS and JS. This will make Sidekiq immune to XSS attacks. [#6270]
|
90
|
+
- Add config option, `:skip_default_job_logging` to disable Sidekiq's default
|
91
|
+
start/finish job logging. [#6200]
|
92
|
+
- Allow `Sidekiq::Limiter.redis` to use Redis Cluster [#6288]
|
93
|
+
- Retain CurrentAttributeѕ after inline execution [#6307]
|
94
|
+
- Ignore non-existent CurrentAttributes attributes when restoring [#6341]
|
95
|
+
- Raise default Redis {read,write,connect} timeouts from 1 to 3 seconds
|
96
|
+
to minimize ReadTimeoutErrors [#6162]
|
97
|
+
- Add `logger` as a dependency since it will become bundled in Ruby 3.5 [#6320]
|
98
|
+
- Ignore unsupported locales in the Web UI [#6313]
|
99
|
+
|
100
|
+
7.2.4
|
101
|
+
----------
|
102
|
+
|
103
|
+
- Fix XSS in metrics filtering introduced in 7.2.0, CVE-2024-32887
|
104
|
+
Thanks to @UmerAdeemCheema for the security report.
|
105
|
+
|
106
|
+
7.2.3
|
107
|
+
----------
|
108
|
+
|
109
|
+
- [Support Dragonfly.io](https://www.mikeperham.com/2024/02/01/supporting-dragonfly/) as an alternative Redis implementation
|
110
|
+
- Fix error unpacking some compressed error backtraces [#6241]
|
111
|
+
- Fix potential heartbeat data leak [#6227]
|
112
|
+
- Add ability to find a currently running work by jid [#6212, fatkodima]
|
113
|
+
|
114
|
+
7.2.2
|
115
|
+
----------
|
116
|
+
|
117
|
+
- Add `Process.warmup` call in Ruby 3.3+
|
118
|
+
- Batch jobs now skip transactional push [#6160]
|
119
|
+
|
120
|
+
7.2.1
|
121
|
+
----------
|
122
|
+
|
123
|
+
- Add `Sidekiq::Work` type which replaces the raw Hash as the third parameter in
|
124
|
+
`Sidekiq::WorkSet#each { |pid, tid, hash| ... }` [#6145]
|
125
|
+
- **DEPRECATED**: direct access to the attributes within the `hash` block parameter above.
|
126
|
+
The `Sidekiq::Work` instance contains accessor methods to get at the same data, e.g.
|
127
|
+
```ruby
|
128
|
+
work["queue"] # Old
|
129
|
+
work.queue # New
|
130
|
+
```
|
131
|
+
- Fix Ruby 3.3 warnings around `base64` gem [#6151, earlopain]
|
132
|
+
|
133
|
+
7.2.0
|
134
|
+
----------
|
135
|
+
|
136
|
+
- `sidekiq_retries_exhausted` can return `:discard` to avoid the deadset
|
137
|
+
and all death handlers [#6091]
|
138
|
+
- Metrics filtering by job class in Web UI [#5974]
|
139
|
+
- Better readability and formatting for numbers within the Web UI [#6080]
|
140
|
+
- Add explicit error if user code tries to nest test modes [#6078]
|
141
|
+
```ruby
|
142
|
+
Sidekiq::Testing.inline! # global setting
|
143
|
+
Sidekiq::Testing.fake! do # override within block
|
144
|
+
# ok
|
145
|
+
Sidekiq::Testing.inline! do # can't override the override
|
146
|
+
# not ok, nested
|
147
|
+
end
|
148
|
+
end
|
149
|
+
```
|
150
|
+
- **SECURITY** Forbid inline JavaScript execution in Web UI [#6074]
|
151
|
+
- Adjust redis-client adapter to avoid `method_missing` [#6083]
|
152
|
+
This can result in app code breaking if your app's Redis API usage was
|
153
|
+
depending on Sidekiq's adapter to correct invalid redis-client API usage.
|
154
|
+
One example:
|
155
|
+
```ruby
|
156
|
+
# bad, not redis-client native
|
157
|
+
# Unsupported command argument type: TrueClass (TypeError)
|
158
|
+
Sidekiq.redis { |c| c.set("key", "value", nx: true, ex: 15) }
|
159
|
+
# good
|
160
|
+
Sidekiq.redis { |c| c.set("key", "value", "nx", "ex", 15) }
|
161
|
+
```
|
162
|
+
|
163
|
+
7.1.6
|
164
|
+
----------
|
165
|
+
|
166
|
+
- The block forms of testing modes (inline, fake) are now thread-safe so you can have
|
167
|
+
a multithreaded test suite which uses different modes for different tests. [#6069]
|
168
|
+
- Fix breakage with non-Proc error handlers [#6065]
|
169
|
+
|
170
|
+
7.1.5
|
171
|
+
----------
|
172
|
+
|
173
|
+
- **FEATURE**: Job filtering within the Web UI. This feature has been open
|
174
|
+
sourced from Sidekiq Pro. [#6052]
|
175
|
+
- **API CHANGE** Error handlers now take three arguments `->(ex, context, config)`.
|
176
|
+
The previous calling convention will work until Sidekiq 8.0 but will print
|
177
|
+
out a deprecation warning. [#6051]
|
178
|
+
- Fix issue with the `batch_size` and `at` options in `S::Client.push_bulk` [#6040]
|
179
|
+
- Fix inline testing firing batch callbacks early [#6057]
|
180
|
+
- Use new log broadcast API in Rails 7.1 [#6054]
|
181
|
+
- Crash if user tries to use RESP2 `protocol: 2` [#6061]
|
182
|
+
|
183
|
+
7.1.4
|
184
|
+
----------
|
185
|
+
|
186
|
+
- Fix empty `retry_for` logic [#6035]
|
187
|
+
|
188
|
+
7.1.3
|
189
|
+
----------
|
190
|
+
|
191
|
+
- Add `sidekiq_options retry_for: 48.hours` to allow time-based retry windows [#6029]
|
192
|
+
- Support sidekiq_retry_in and sidekiq_retries_exhausted_block in ActiveJobs (#5994)
|
193
|
+
- Lowercase all Rack headers for Rack 3.0 [#5951]
|
194
|
+
- Validate Sidekiq::Web page refresh delay to avoid potential DoS,
|
195
|
+
CVE-2023-26141, thanks for reporting Keegan!
|
196
|
+
|
197
|
+
7.1.2
|
198
|
+
----------
|
199
|
+
|
200
|
+
- Mark Web UI assets as private so CDNs won't cache them [#5936]
|
201
|
+
- Fix stackoverflow when using Oj and the JSON log formatter [#5920]
|
202
|
+
- Remove spurious `enqueued_at` from scheduled ActiveJobs [#5937]
|
203
|
+
|
204
|
+
7.1.1
|
205
|
+
----------
|
206
|
+
|
207
|
+
- Support multiple CurrentAttributes [#5904]
|
208
|
+
- Speed up latency fetch with large queues on Redis <7 [#5910]
|
209
|
+
- Allow a larger default client pool [#5886]
|
210
|
+
- Ensure Sidekiq.options[:environment] == RAILS_ENV [#5932]
|
211
|
+
|
212
|
+
7.1.0
|
213
|
+
----------
|
214
|
+
|
215
|
+
- Improve display of ActiveJob arguments in Web UI [#5825, cover]
|
216
|
+
- Update `push_bulk` to push `batch_size` jobs at a time and allow laziness [#5827, fatkodima]
|
217
|
+
This allows Sidekiq::Client to push unlimited jobs as long as it has enough memory for the batch_size.
|
218
|
+
- Update `perform_bulk` to use `push_bulk` internally.
|
219
|
+
- Change return value of `push_bulk` to map 1-to-1 with arguments.
|
220
|
+
If you call `push_bulk(args: [[1], [2], [3]])`, you will now always get
|
221
|
+
an array of 3 values as the result: `["jid1", nil, "jid3"]` where nil means
|
222
|
+
that particular job did not push successfully (possibly due to middleware
|
223
|
+
stopping it). Previously nil values were removed so it was impossible to tell
|
224
|
+
which jobs pushed successfully and which did not.
|
225
|
+
- Migrate away from all deprecated Redis commands [#5788]
|
226
|
+
Sidekiq will now print a warning if you use one of those deprecated commands.
|
227
|
+
- Prefix all Sidekiq thread names [#5872]
|
228
|
+
|
229
|
+
7.0.9
|
230
|
+
----------
|
231
|
+
|
232
|
+
- Restore confirmation dialogs in Web UI [#5881, shevaun]
|
233
|
+
- Increase fetch timeout to minimize ReadTimeoutError [#5874]
|
234
|
+
- Reverse histogram tooltip ordering [#5868]
|
235
|
+
- Add Scottish Gaelic (gd) locale [#5867, GunChleoc]
|
236
|
+
|
237
|
+
7.0.8
|
238
|
+
----------
|
239
|
+
|
240
|
+
- **SECURITY** Sanitize `period` input parameter on Metrics pages.
|
241
|
+
Specially crafted values can lead to XSS. This functionality
|
242
|
+
was introduced in 7.0.4. Thank you to spercex @ huntr.dev [#5694]
|
243
|
+
- Add job hash as 3rd parameter to the `sidekiq_retry_in` block.
|
244
|
+
|
245
|
+
7.0.7
|
246
|
+
----------
|
247
|
+
|
248
|
+
- Fix redis-client API usage which could result in stuck Redis
|
249
|
+
connections [#5823]
|
250
|
+
- Fix AS::Duration with `sidekiq_retry_in` [#5806]
|
251
|
+
- Restore dumping config options on startup with `-v` [#5822]
|
252
|
+
|
253
|
+
7.0.5,7.0.6
|
254
|
+
----------
|
255
|
+
|
256
|
+
- More context for debugging json unsafe errors [#5787]
|
257
|
+
|
258
|
+
7.0.4
|
259
|
+
----------
|
260
|
+
|
261
|
+
- Performance and memory optimizations [#5768, fatkodima]
|
262
|
+
- Add 1-8 hour period selector to Metrics pages [#5694]
|
263
|
+
- Fix process display with `sidekiqmon` [#5733]
|
264
|
+
|
265
|
+
7.0.3
|
266
|
+
----------
|
267
|
+
|
268
|
+
- Don't warn about memory policy on Redis Enterprise [#5712]
|
269
|
+
- Don't allow Quiet/Stop on embedded Sidekiq instances [#5716]
|
270
|
+
- Fix `size: X` for configuring the default Redis pool size [#5702]
|
271
|
+
- Improve the display of queue weights on Busy page [#5642]
|
272
|
+
- Freeze CurrentAttributes on a job once initially set [#5692]
|
273
|
+
|
274
|
+
7.0.2
|
275
|
+
----------
|
276
|
+
|
277
|
+
- Improve compatibility with custom loggers [#5673]
|
278
|
+
- Add queue weights on Busy page [#5640]
|
279
|
+
- Add BID link on job_info page if job is part of a Batch [#5623]
|
280
|
+
- Allow custom extensions to add rows/links within Job detail pages [#5624]
|
281
|
+
```ruby
|
282
|
+
Sidekiq::Web.custom_job_info_rows << AddAccountLink.new
|
283
|
+
|
284
|
+
class AddAccountLink
|
285
|
+
include CGI::Util
|
286
|
+
def add_pair(job)
|
287
|
+
# yield a (name, value) pair
|
288
|
+
# You can include HTML tags and CSS, Sidekiq does not do any
|
289
|
+
# escaping so beware user data injection! Note how we use CGI's
|
290
|
+
# `h` escape helper.
|
291
|
+
aid = job["account_id"]
|
292
|
+
yield "Account", "<a href='/accounts/#{h aid}'>#{h aid}</a>" if aid
|
293
|
+
end
|
294
|
+
end
|
295
|
+
```
|
296
|
+
|
297
|
+
7.0.1
|
298
|
+
----------
|
299
|
+
|
300
|
+
- Allow an embedding process to reuse its own heartbeat thread
|
301
|
+
- Update zh-cn localization
|
302
|
+
|
303
|
+
7.0.0
|
304
|
+
----------
|
305
|
+
|
306
|
+
- Embedded mode!
|
307
|
+
- Capsules!!
|
308
|
+
- Job Execution metrics!!!
|
309
|
+
- See `docs/7.0-Upgrade.md` for release notes
|
310
|
+
|
311
|
+
6.5.{10,11,12}
|
312
|
+
----------
|
313
|
+
|
314
|
+
- Fixes for Rails 7.1 [#6067, #6070]
|
315
|
+
|
316
|
+
6.5.9
|
317
|
+
----------
|
318
|
+
|
319
|
+
- Ensure Sidekiq.options[:environment] == RAILS_ENV [#5932]
|
320
|
+
|
321
|
+
6.5.8
|
322
|
+
----------
|
323
|
+
|
324
|
+
- Fail if using a bad version of scout_apm [#5616]
|
325
|
+
- Add pagination to Busy page [#5556]
|
326
|
+
- Speed up WorkSet#each [#5559]
|
327
|
+
- Adjust CurrentAttributes to work with the String class name so we aren't referencing the Class within a Rails initializer [#5536]
|
328
|
+
|
329
|
+
6.5.7
|
330
|
+
----------
|
331
|
+
|
332
|
+
- Updates for JA and ZH locales
|
333
|
+
- Further optimizations for scheduled polling [#5513]
|
334
|
+
|
335
|
+
6.5.6
|
336
|
+
----------
|
337
|
+
|
338
|
+
- Fix deprecation warnings with redis-rb 4.8.0 [#5484]
|
339
|
+
- Lock redis-rb to < 5.0 as we are moving to redis-client in Sidekiq 7.0
|
340
|
+
|
341
|
+
6.5.5
|
342
|
+
----------
|
343
|
+
|
344
|
+
- Fix require issue with job_retry.rb [#5462]
|
345
|
+
- Improve Sidekiq::Web compatibility with Rack 3.x
|
346
|
+
|
347
|
+
6.5.4
|
348
|
+
----------
|
349
|
+
|
350
|
+
- Fix invalid code on Ruby 2.5 [#5460]
|
351
|
+
- Fix further metrics dependency issues [#5457]
|
352
|
+
|
353
|
+
6.5.3
|
354
|
+
----------
|
355
|
+
|
356
|
+
- Don't require metrics code without explicit opt-in [#5456]
|
357
|
+
|
358
|
+
6.5.2
|
359
|
+
----------
|
360
|
+
|
361
|
+
- [Job Metrics are under active development, help wanted!](https://github.com/sidekiq/sidekiq/wiki/Metrics#contributing) **BETA**
|
362
|
+
- Add `Context` column on queue page which shows any CurrentAttributes [#5450]
|
363
|
+
- `sidekiq_retry_in` may now return `:discard` or `:kill` to dynamically stop job retries [#5406]
|
364
|
+
- Smarter sorting of processes in /busy Web UI [#5398]
|
365
|
+
- Fix broken hamburger menu in mobile UI [#5428]
|
366
|
+
- Require redis-rb 4.5.0. Note that Sidekiq will break if you use the
|
367
|
+
[`Redis.exists_returns_integer = false`](https://github.com/redis/redis-rb/blob/master/CHANGELOG.md#450) flag. [#5394]
|
4
368
|
|
5
369
|
6.5.1
|
6
370
|
----------
|
@@ -13,7 +377,7 @@
|
|
13
377
|
- Substantial refactoring of Sidekiq server internals, part of a larger effort
|
14
378
|
to reduce Sidekiq's internal usage of global methods and data, see [docs/global_to_local.md](docs/global_to_local.md) and [docs/middleware.md](docs/middleware.md).
|
15
379
|
- **Add beta support for the `redis-client` gem**. This will become the default Redis driver in Sidekiq 7.0. [#5298]
|
16
|
-
Read more: https://github.com/
|
380
|
+
Read more: https://github.com/sidekiq/sidekiq/wiki/Using-redis-client
|
17
381
|
- **Add beta support for DB transaction-aware client** [#5291]
|
18
382
|
Add this line to your initializer and any jobs created during a transaction
|
19
383
|
will only be pushed to Redis **after the transaction commits**. You will need to add the
|
@@ -213,7 +577,7 @@ If this is a Rails app in API mode, you need to enable sessions.
|
|
213
577
|
---------
|
214
578
|
|
215
579
|
- **Integrate with systemd's watchdog and notification features** [#4488]
|
216
|
-
Set `Type=notify` in [sidekiq.service](https://github.com/
|
580
|
+
Set `Type=notify` in [sidekiq.service](https://github.com/sidekiq/sidekiq/blob/4b8a8bd3ae42f6e48ae1fdaf95ed7d7af18ed8bb/examples/systemd/sidekiq.service#L30-L39). The integration works automatically.
|
217
581
|
- Use `setTimeout` rather than `setInterval` to avoid thundering herd [#4480]
|
218
582
|
- Fix edge case where a job can be pushed without a queue.
|
219
583
|
- Flush job stats at exit [#4498]
|
@@ -226,7 +590,7 @@ If this is a Rails app in API mode, you need to enable sessions.
|
|
226
590
|
- Fix broken Web UI response when using NewRelic and Rack 2.1.2+. [#4440]
|
227
591
|
- Update APIs to use `UNLINK`, not `DEL`. [#4449]
|
228
592
|
- Fix Ruby 2.7 warnings [#4412]
|
229
|
-
- Add support for `APP_ENV` [[95fa5d9]](https://github.com/
|
593
|
+
- Add support for `APP_ENV` [[95fa5d9]](https://github.com/sidekiq/sidekiq/commit/95fa5d90192148026e52ca2902f1b83c70858ce8)
|
230
594
|
|
231
595
|
6.0.4
|
232
596
|
---------
|
@@ -338,7 +702,7 @@ Sidekiq.configure_server do |config|
|
|
338
702
|
config.log_formatter = Sidekiq::Logger::Formatters::JSON.new
|
339
703
|
end
|
340
704
|
```
|
341
|
-
See the [Logging wiki page](https://github.com/
|
705
|
+
See the [Logging wiki page](https://github.com/sidekiq/sidekiq/wiki/Logging) for more details.
|
342
706
|
- **BREAKING CHANGE** Validate proper usage of the `REDIS_PROVIDER`
|
343
707
|
variable. This variable is meant to hold the name of the environment
|
344
708
|
variable which contains your Redis URL, so that you can switch Redis
|
@@ -530,7 +894,7 @@ Sidekiq::Middleware::Server::Logging -> Sidekiq::JobLogger
|
|
530
894
|
- The `SomeWorker.set(options)` API was re-written to avoid thread-local state. [#2152]
|
531
895
|
- Sidekiq Enterprise's encrypted jobs now display "[encrypted data]" in the Web UI instead
|
532
896
|
of random hex bytes.
|
533
|
-
- Please see the [5.0 Upgrade notes](5.0-Upgrade.md) for more detail.
|
897
|
+
- Please see the [5.0 Upgrade notes](docs/5.0-Upgrade.md) for more detail.
|
534
898
|
|
535
899
|
4.2.10
|
536
900
|
-----------
|
@@ -748,7 +1112,7 @@ Sidekiq::Queues.clear_all
|
|
748
1112
|
- Sidekiq's internals have been completely overhauled for performance
|
749
1113
|
and to remove dependencies. This has resulted in major speedups, as
|
750
1114
|
[detailed on my blog](http://www.mikeperham.com/2015/10/14/optimizing-sidekiq/).
|
751
|
-
- See the [4.0 upgrade notes](4.0-Upgrade.md) for more detail.
|
1115
|
+
- See the [4.0 upgrade notes](docs/4.0-Upgrade.md) for more detail.
|
752
1116
|
|
753
1117
|
3.5.4
|
754
1118
|
-----------
|
@@ -785,7 +1149,7 @@ Sidekiq::Queues.clear_all
|
|
785
1149
|
- **FIX MEMORY LEAK** Under rare conditions, threads may leak [#2598, gazay]
|
786
1150
|
- Add Ukrainian locale [#2561, elrakita]
|
787
1151
|
- Disconnect and retry Redis operations if we see a READONLY error [#2550]
|
788
|
-
- Add server middleware testing harness; see [wiki](https://github.com/
|
1152
|
+
- Add server middleware testing harness; see [wiki](https://github.com/sidekiq/sidekiq/wiki/Testing#testing-server-middleware) [#2534, ryansch]
|
789
1153
|
|
790
1154
|
3.5.0
|
791
1155
|
-----------
|
@@ -803,7 +1167,7 @@ Sidekiq::Queues.clear_all
|
|
803
1167
|
- Fix CSRF vulnerability in Web UI, thanks to Egor Homakov for
|
804
1168
|
reporting. [#2422] If you are running the Web UI as a standalone Rack app,
|
805
1169
|
ensure you have a [session middleware
|
806
|
-
configured](https://github.com/
|
1170
|
+
configured](https://github.com/sidekiq/sidekiq/wiki/Monitoring#standalone):
|
807
1171
|
```ruby
|
808
1172
|
use Rack::Session::Cookie, :secret => "some unique secret string here"
|
809
1173
|
```
|
@@ -1015,7 +1379,7 @@ sidekiq_options :dead => false, :retry => 5
|
|
1015
1379
|
3.0.0
|
1016
1380
|
-----------
|
1017
1381
|
|
1018
|
-
Please see [3.0-Upgrade.md](3.0-Upgrade.md) for more comprehensive upgrade notes.
|
1382
|
+
Please see [3.0-Upgrade.md](docs/3.0-Upgrade.md) for more comprehensive upgrade notes.
|
1019
1383
|
|
1020
1384
|
- **Dead Job Queue** - jobs which run out of retries are now moved to a dead
|
1021
1385
|
job queue. These jobs must be retried manually or they will expire
|
@@ -1059,7 +1423,7 @@ Sidekiq::Client.via(ConnectionPool.new { Redis.new }) do
|
|
1059
1423
|
end
|
1060
1424
|
```
|
1061
1425
|
**Sharding support does require a breaking change to client-side
|
1062
|
-
middleware, see 3.0-Upgrade.md.**
|
1426
|
+
middleware, see docs/3.0-Upgrade.md.**
|
1063
1427
|
- New Chinese, Greek, Swedish and Czech translations for the Web UI.
|
1064
1428
|
- Updated most languages translations for the new UI features.
|
1065
1429
|
- **Remove official Capistrano integration** - this integration has been
|
@@ -1187,7 +1551,7 @@ middleware, see 3.0-Upgrade.md.**
|
|
1187
1551
|
appear to be doing any work. [#1194]
|
1188
1552
|
- Sidekiq's testing behavior is now dynamic. You can choose between
|
1189
1553
|
`inline` and `fake` behavior in your tests. See
|
1190
|
-
[Testing](https://github.com/
|
1554
|
+
[Testing](https://github.com/sidekiq/sidekiq/wiki/Testing) for detail. [#1193]
|
1191
1555
|
- The Retries table has a new column for the error message.
|
1192
1556
|
- The Web UI topbar now contains the status and live poll button.
|
1193
1557
|
- Orphaned worker records are now auto-vacuumed when you visit the
|
data/README.md
CHANGED
@@ -2,7 +2,7 @@ Sidekiq
|
|
2
2
|
==============
|
3
3
|
|
4
4
|
[![Gem Version](https://badge.fury.io/rb/sidekiq.svg)](https://rubygems.org/gems/sidekiq)
|
5
|
-
![Build](https://github.com/
|
5
|
+
![Build](https://github.com/sidekiq/sidekiq/workflows/CI/badge.svg)
|
6
6
|
|
7
7
|
Simple, efficient background processing for Ruby.
|
8
8
|
|
@@ -10,28 +10,15 @@ Sidekiq uses threads to handle many jobs at the same time in the
|
|
10
10
|
same process. It does not require Rails but will integrate tightly with
|
11
11
|
Rails to make background processing dead simple.
|
12
12
|
|
13
|
-
Performance
|
14
|
-
---------------
|
15
|
-
|
16
|
-
Version | Latency | Garbage created for 10k jobs | Time to process 100k jobs | Throughput | Ruby
|
17
|
-
-----------------|------|---------|---------|------------------------|-----
|
18
|
-
Sidekiq 6.0.2 | 3 ms | 156 MB | 14.0 sec| **7100 jobs/sec** | MRI 2.6.3
|
19
|
-
Sidekiq 6.0.0 | 3 ms | 156 MB | 19 sec | 5200 jobs/sec | MRI 2.6.3
|
20
|
-
Sidekiq 4.0.0 | 10 ms | 151 MB | 22 sec | 4500 jobs/sec |
|
21
|
-
Sidekiq 3.5.1 | 22 ms | 1257 MB | 125 sec | 800 jobs/sec |
|
22
|
-
Resque 1.25.2 | - | - | 420 sec | 240 jobs/sec |
|
23
|
-
DelayedJob 4.1.1 | - | - | 465 sec | 215 jobs/sec |
|
24
|
-
|
25
|
-
This benchmark can be found in `bin/sidekiqload` and assumes a Redis network latency of 1ms.
|
26
13
|
|
27
14
|
Requirements
|
28
15
|
-----------------
|
29
16
|
|
30
|
-
- Redis:
|
31
|
-
- Ruby: MRI 2.
|
32
|
-
|
33
|
-
Sidekiq 6.0 supports Rails 5.0+ but does not require it.
|
17
|
+
- Redis: Redis 6.2+ or Dragonfly 1.13+
|
18
|
+
- Ruby: MRI 2.7+ or JRuby 9.3+.
|
34
19
|
|
20
|
+
Sidekiq 7.0 supports Rails 6.0+ but does not require it.
|
21
|
+
As of 7.2, Sidekiq supports Dragonfly as an alternative to Redis for data storage.
|
35
22
|
|
36
23
|
Installation
|
37
24
|
-----------------
|
@@ -42,57 +29,78 @@ Installation
|
|
42
29
|
Getting Started
|
43
30
|
-----------------
|
44
31
|
|
45
|
-
See the [Getting Started wiki page](https://github.com/
|
32
|
+
See the [Getting Started wiki page](https://github.com/sidekiq/sidekiq/wiki/Getting-Started) and follow the simple setup process.
|
46
33
|
You can watch [this YouTube playlist](https://www.youtube.com/playlist?list=PLjeHh2LSCFrWGT5uVjUuFKAcrcj5kSai1) to learn all about
|
47
34
|
Sidekiq and see its features in action. Here's the Web UI:
|
48
35
|
|
49
|
-
![Web UI](https://github.com/
|
36
|
+
![Web UI](https://github.com/sidekiq/sidekiq/raw/main/examples/web-ui.png)
|
37
|
+
|
38
|
+
Performance
|
39
|
+
---------------
|
40
|
+
|
41
|
+
The benchmark in `bin/sidekiqload` creates 500,000 no-op jobs and drains them as fast as possible, assuming a fixed Redis network latency of 1ms.
|
42
|
+
This requires a lot of Redis network I/O and JSON parsing.
|
43
|
+
This benchmark is IO-bound so we increase the concurrency to 25.
|
44
|
+
If your application is sending lots of emails or performing other network-intensive work, you could see a similar benefit but be careful not to saturate the CPU.
|
50
45
|
|
46
|
+
Version | Time to process 500k jobs | Throughput (jobs/sec) | Ruby | Concurrency | Job Type
|
47
|
+
-----------------|------|---------|---------|------------------------|---
|
48
|
+
Sidekiq 7.0.3 | 21.3 sec| 23,500 | 3.2.0+yjit | 30 | Sidekiq::Job
|
49
|
+
Sidekiq 7.0.3 | 33.8 sec| 14,700 | 3.2.0+yjit | 30 | ActiveJob 7.0.4
|
50
|
+
Sidekiq 7.0.3 | 23.5 sec| 21,300 | 3.2.0 | 30 | Sidekiq::Job
|
51
|
+
Sidekiq 7.0.3 | 46.5 sec| 10,700 | 3.2.0 | 30 | ActiveJob 7.0.4
|
52
|
+
Sidekiq 7.0.3 | 23.0 sec| 21,700 | 2.7.5 | 30 | Sidekiq::Job
|
53
|
+
Sidekiq 7.0.3 | 46.5 sec| 10,850 | 2.7.5 | 30 | ActiveJob 7.0.4
|
54
|
+
|
55
|
+
Most of Sidekiq's overhead is Redis network I/O.
|
56
|
+
ActiveJob adds a notable amount of CPU overhead due to argument deserialization and callbacks.
|
57
|
+
Concurrency of 30 was determined experimentally to maximize one CPU without saturating it.
|
51
58
|
|
52
59
|
Want to Upgrade?
|
53
60
|
-------------------
|
54
61
|
|
62
|
+
Use `bundle up sidekiq` to upgrade Sidekiq and all its dependencies.
|
63
|
+
Upgrade notes between each major version can be found in the `docs/` directory.
|
64
|
+
|
55
65
|
I also sell Sidekiq Pro and Sidekiq Enterprise, extensions to Sidekiq which provide more
|
56
66
|
features, a commercial-friendly license and allow you to support high
|
57
67
|
quality open source development all at the same time. Please see the
|
58
68
|
[Sidekiq](https://sidekiq.org/) homepage for more detail.
|
59
69
|
|
60
|
-
Subscribe to the **[quarterly newsletter](https://tinyletter.com/sidekiq)** to stay informed about the latest
|
61
|
-
features and changes to Sidekiq and its bigger siblings.
|
62
|
-
|
63
70
|
|
64
71
|
Problems?
|
65
72
|
-----------------
|
66
73
|
|
67
|
-
**Please do not directly email any Sidekiq committers with questions or problems.**
|
74
|
+
**Please do not directly email any Sidekiq committers with questions or problems.**
|
75
|
+
A community is best served when discussions are held in public.
|
68
76
|
|
69
|
-
If you have a problem, please review the [FAQ](https://github.com/
|
70
|
-
Searching the [issues](https://github.com/
|
77
|
+
If you have a problem, please review the [FAQ](https://github.com/sidekiq/sidekiq/wiki/FAQ) and [Troubleshooting](https://github.com/sidekiq/sidekiq/wiki/Problems-and-Troubleshooting) wiki pages.
|
78
|
+
Searching the [issues](https://github.com/sidekiq/sidekiq/issues) for your problem is also a good idea.
|
71
79
|
|
72
|
-
Sidekiq Pro and Sidekiq Enterprise customers get private email support.
|
80
|
+
Sidekiq Pro and Sidekiq Enterprise customers get private email support.
|
81
|
+
You can purchase at https://sidekiq.org; email support@contribsys.com for help.
|
73
82
|
|
74
83
|
Useful resources:
|
75
84
|
|
76
|
-
* Product documentation is in the [wiki](https://github.com/
|
77
|
-
* Occasional announcements are made to the [@sidekiq](https://
|
85
|
+
* Product documentation is in the [wiki](https://github.com/sidekiq/sidekiq/wiki).
|
86
|
+
* Occasional announcements are made to the [@sidekiq](https://ruby.social/@sidekiq) Mastodon account.
|
78
87
|
* The [Sidekiq tag](https://stackoverflow.com/questions/tagged/sidekiq) on Stack Overflow has lots of useful Q & A.
|
79
88
|
|
80
|
-
Every
|
89
|
+
Every Thursday morning is Sidekiq office hour: I video chat and answer questions.
|
81
90
|
See the [Sidekiq support page](https://sidekiq.org/support.html) for details.
|
82
91
|
|
83
92
|
Contributing
|
84
93
|
-----------------
|
85
94
|
|
86
|
-
Please see [the contributing guidelines](https://github.com/
|
87
|
-
|
95
|
+
Please see [the contributing guidelines](https://github.com/sidekiq/sidekiq/blob/main/.github/contributing.md).
|
88
96
|
|
89
97
|
License
|
90
98
|
-----------------
|
91
99
|
|
92
|
-
Please see [LICENSE](https://github.com/
|
93
|
-
|
100
|
+
Please see [LICENSE.txt](https://github.com/sidekiq/sidekiq/blob/main/LICENSE.txt) for licensing details.
|
101
|
+
The license for Sidekiq Pro and Sidekiq Enterprise can be found in [COMM-LICENSE.txt](https://github.com/sidekiq/sidekiq/blob/main/COMM-LICENSE.txt).
|
94
102
|
|
95
103
|
Author
|
96
104
|
-----------------
|
97
105
|
|
98
|
-
Mike Perham, [@getajobmike](https://
|
106
|
+
Mike Perham, [@getajobmike](https://ruby.social/@getajobmike) / [@sidekiq](https://ruby.social/@sidekiq), [https://www.mikeperham.com](https://www.mikeperham.com) / [https://www.contribsys.com](https://www.contribsys.com)
|