sidekiq 5.2.10 → 6.5.12

Sign up to get free protection for your applications and to get access to all the features.
Files changed (126) hide show
  1. checksums.yaml +4 -4
  2. data/Changes.md +422 -1
  3. data/LICENSE +3 -3
  4. data/README.md +24 -35
  5. data/bin/sidekiq +27 -3
  6. data/bin/sidekiqload +79 -67
  7. data/bin/sidekiqmon +8 -0
  8. data/lib/generators/sidekiq/job_generator.rb +57 -0
  9. data/lib/generators/sidekiq/templates/{worker.rb.erb → job.rb.erb} +2 -2
  10. data/lib/generators/sidekiq/templates/{worker_spec.rb.erb → job_spec.rb.erb} +1 -1
  11. data/lib/generators/sidekiq/templates/{worker_test.rb.erb → job_test.rb.erb} +1 -1
  12. data/lib/sidekiq/api.rb +527 -310
  13. data/lib/sidekiq/cli.rb +204 -208
  14. data/lib/sidekiq/client.rb +78 -82
  15. data/lib/sidekiq/component.rb +65 -0
  16. data/lib/sidekiq/delay.rb +8 -7
  17. data/lib/sidekiq/extensions/action_mailer.rb +13 -22
  18. data/lib/sidekiq/extensions/active_record.rb +13 -10
  19. data/lib/sidekiq/extensions/class_methods.rb +14 -11
  20. data/lib/sidekiq/extensions/generic_proxy.rb +7 -5
  21. data/lib/sidekiq/fetch.rb +50 -40
  22. data/lib/sidekiq/job.rb +13 -0
  23. data/lib/sidekiq/job_logger.rb +33 -7
  24. data/lib/sidekiq/job_retry.rb +126 -106
  25. data/lib/sidekiq/job_util.rb +71 -0
  26. data/lib/sidekiq/launcher.rb +177 -83
  27. data/lib/sidekiq/logger.rb +156 -0
  28. data/lib/sidekiq/manager.rb +40 -41
  29. data/lib/sidekiq/metrics/deploy.rb +47 -0
  30. data/lib/sidekiq/metrics/query.rb +153 -0
  31. data/lib/sidekiq/metrics/shared.rb +94 -0
  32. data/lib/sidekiq/metrics/tracking.rb +134 -0
  33. data/lib/sidekiq/middleware/chain.rb +102 -46
  34. data/lib/sidekiq/middleware/current_attributes.rb +63 -0
  35. data/lib/sidekiq/middleware/i18n.rb +7 -7
  36. data/lib/sidekiq/middleware/modules.rb +21 -0
  37. data/lib/sidekiq/monitor.rb +133 -0
  38. data/lib/sidekiq/paginator.rb +28 -16
  39. data/lib/sidekiq/processor.rb +104 -97
  40. data/lib/sidekiq/rails.rb +46 -37
  41. data/lib/sidekiq/redis_client_adapter.rb +154 -0
  42. data/lib/sidekiq/redis_connection.rb +108 -77
  43. data/lib/sidekiq/ring_buffer.rb +29 -0
  44. data/lib/sidekiq/scheduled.rb +105 -42
  45. data/lib/sidekiq/sd_notify.rb +149 -0
  46. data/lib/sidekiq/systemd.rb +24 -0
  47. data/lib/sidekiq/testing/inline.rb +6 -5
  48. data/lib/sidekiq/testing.rb +68 -58
  49. data/lib/sidekiq/transaction_aware_client.rb +45 -0
  50. data/lib/sidekiq/version.rb +2 -1
  51. data/lib/sidekiq/web/action.rb +15 -11
  52. data/lib/sidekiq/web/application.rb +103 -77
  53. data/lib/sidekiq/web/csrf_protection.rb +180 -0
  54. data/lib/sidekiq/web/helpers.rb +125 -95
  55. data/lib/sidekiq/web/router.rb +23 -19
  56. data/lib/sidekiq/web.rb +65 -105
  57. data/lib/sidekiq/worker.rb +259 -109
  58. data/lib/sidekiq.rb +170 -62
  59. data/sidekiq.gemspec +23 -16
  60. data/web/assets/images/apple-touch-icon.png +0 -0
  61. data/web/assets/javascripts/application.js +113 -61
  62. data/web/assets/javascripts/chart.min.js +13 -0
  63. data/web/assets/javascripts/chartjs-plugin-annotation.min.js +7 -0
  64. data/web/assets/javascripts/dashboard.js +53 -89
  65. data/web/assets/javascripts/graph.js +16 -0
  66. data/web/assets/javascripts/metrics.js +262 -0
  67. data/web/assets/stylesheets/application-dark.css +143 -0
  68. data/web/assets/stylesheets/application-rtl.css +0 -4
  69. data/web/assets/stylesheets/application.css +88 -233
  70. data/web/locales/ar.yml +8 -2
  71. data/web/locales/de.yml +14 -2
  72. data/web/locales/el.yml +43 -19
  73. data/web/locales/en.yml +13 -1
  74. data/web/locales/es.yml +18 -2
  75. data/web/locales/fr.yml +10 -3
  76. data/web/locales/ja.yml +14 -1
  77. data/web/locales/lt.yml +83 -0
  78. data/web/locales/pl.yml +4 -4
  79. data/web/locales/pt-br.yml +27 -9
  80. data/web/locales/ru.yml +4 -0
  81. data/web/locales/vi.yml +83 -0
  82. data/web/locales/zh-cn.yml +36 -11
  83. data/web/locales/zh-tw.yml +32 -7
  84. data/web/views/_footer.erb +1 -1
  85. data/web/views/_job_info.erb +3 -2
  86. data/web/views/_nav.erb +1 -1
  87. data/web/views/_poll_link.erb +2 -5
  88. data/web/views/_summary.erb +7 -7
  89. data/web/views/busy.erb +61 -22
  90. data/web/views/dashboard.erb +23 -14
  91. data/web/views/dead.erb +3 -3
  92. data/web/views/layout.erb +3 -1
  93. data/web/views/metrics.erb +69 -0
  94. data/web/views/metrics_for_job.erb +87 -0
  95. data/web/views/morgue.erb +9 -6
  96. data/web/views/queue.erb +23 -10
  97. data/web/views/queues.erb +10 -2
  98. data/web/views/retries.erb +11 -8
  99. data/web/views/retry.erb +3 -3
  100. data/web/views/scheduled.erb +5 -2
  101. metadata +58 -63
  102. data/.circleci/config.yml +0 -61
  103. data/.github/contributing.md +0 -32
  104. data/.github/issue_template.md +0 -11
  105. data/.gitignore +0 -15
  106. data/.travis.yml +0 -11
  107. data/3.0-Upgrade.md +0 -70
  108. data/4.0-Upgrade.md +0 -53
  109. data/5.0-Upgrade.md +0 -56
  110. data/COMM-LICENSE +0 -97
  111. data/Ent-Changes.md +0 -238
  112. data/Gemfile +0 -19
  113. data/Pro-2.0-Upgrade.md +0 -138
  114. data/Pro-3.0-Upgrade.md +0 -44
  115. data/Pro-4.0-Upgrade.md +0 -35
  116. data/Pro-Changes.md +0 -759
  117. data/Rakefile +0 -9
  118. data/bin/sidekiqctl +0 -20
  119. data/code_of_conduct.md +0 -50
  120. data/lib/generators/sidekiq/worker_generator.rb +0 -49
  121. data/lib/sidekiq/core_ext.rb +0 -1
  122. data/lib/sidekiq/ctl.rb +0 -221
  123. data/lib/sidekiq/exception_handler.rb +0 -29
  124. data/lib/sidekiq/logging.rb +0 -122
  125. data/lib/sidekiq/middleware/server/active_record.rb +0 -23
  126. data/lib/sidekiq/util.rb +0 -66
data/Pro-Changes.md DELETED
@@ -1,759 +0,0 @@
1
- # Sidekiq Pro Changelog
2
-
3
- [Sidekiq Changes](https://github.com/mperham/sidekiq/blob/master/Changes.md) | [Sidekiq Pro Changes](https://github.com/mperham/sidekiq/blob/master/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/mperham/sidekiq/blob/master/Ent-Changes.md)
4
-
5
- Please see [http://sidekiq.org/](http://sidekiq.org/) for more details and how to buy.
6
-
7
- HEAD
8
- ---------
9
-
10
- - Add ES translations, see issues [#3949](https://github.com/mperham/sidekiq/issues/3949) and [#3951](https://github.com/mperham/sidekiq/issues/3951) to add your own language.
11
-
12
- 4.0.5
13
- ---------
14
-
15
- - Increase super\_fetch retriever thread count from 1 to 2 to make it
16
- less sensitive to Redis latency.
17
- - Better handling of invalid job JSON by reliable scheduler [#4053]
18
- - Added ZH, PT, JA and RU translations.
19
-
20
- 4.0.4
21
- ---------
22
-
23
- - Update Sidekiq::Client patches to work with new Module#prepend
24
- mechanism in Sidekiq 5.2.0. [#3930]
25
-
26
- 4.0.3
27
- ---------
28
-
29
- - Add at\_exit handler to push any saved jobs in `reliable_push` when exiting. [#3823]
30
- - Implement batch death callback. This is fired the first time a job within a batch dies. [#3841]
31
- ```ruby
32
- batch = Sidekiq::Batch.new
33
- batch.on(:death, ...)
34
- ```
35
-
36
- 4.0.2
37
- ---------
38
-
39
- - Remove super\_fetch edge case leading to an unnecessary `sleep(1)`
40
- call and resulting latency [#3790]
41
- - Fix possible bad statsd metric call on super\_fetch startup
42
- - Remove superfluous `freeze` calls on Strings [#3759]
43
-
44
- 4.0.1
45
- ---------
46
-
47
- - Fix incompatibility with the statsd-ruby gem [#3740]
48
- - Add tags to Statsd metrics when using Datadog [#3744]
49
-
50
- 4.0.0
51
- ---------
52
-
53
- - See the [Sidekiq Pro 4.0](Pro-4.0-Upgrade.md) release notes.
54
-
55
-
56
- 3.7.1
57
- ---------
58
-
59
- - Deprecate timed\_fetch. Switch to super\_fetch:
60
- ```ruby
61
- config.super_fetch!
62
- ```
63
-
64
-
65
- 3.7.0
66
- ---------
67
-
68
- - Refactor batch job success/failure to gracefully handle several edge
69
- cases with regard to Sidekiq::Shutdown. This should greatly reduce
70
- the chances of seeing the long-standing "negative pending count" problem. [#3710]
71
-
72
-
73
- 3.6.1
74
- ---------
75
-
76
- - Add support for Datadog::Statsd, it is the recommended Statsd client. [#3699]
77
- ```ruby
78
- Sidekiq::Pro.dogstatsd = ->{ Datadog::Statsd.new("metrics.example.com", 8125) }
79
- ```
80
- - Size the statsd connection pool based on Sidekiq's concurrency [#3700]
81
-
82
-
83
- 3.6.0
84
- ---------
85
-
86
- This release overhauls the Statsd metrics support and adds more
87
- metrics for tracking Pro feature usage. In your initializer:
88
- ```ruby
89
- Sidekiq::Pro.statsd = ->{ ::Statsd.new("127.0.0.1", 8125) }
90
- ```
91
- Sidekiq Pro will emit more metrics to Statsd:
92
- ```
93
- jobs.expired - when a job is expired
94
- jobs.recovered.push - when a job is recovered by reliable_push after network outage
95
- jobs.recovered.fetch - when a job is recovered by super_fetch after process crash
96
- batch.created - when a batch is created
97
- batch.complete - when a batch is completed
98
- batch.success - when a batch is successful
99
- ```
100
- Sidekiq Pro's existing Statsd middleware has been rewritten to leverage the new API.
101
- Everything should be backwards compatible with one deprecation notice.
102
-
103
-
104
- 3.5.4
105
- ---------
106
-
107
- - Fix case in SuperFetch where Redis downtime can lead to processor thread death [#3684]
108
- - Fix case where TimedFetch might not recover some pending jobs
109
- - Fix edge case in Batch::Status#poll leading to premature completion [#3640]
110
- - Adjust scan API to check 100 elements at a time, to minimize network round trips
111
- when scanning large sets.
112
-
113
- 3.5.3
114
- ---------
115
-
116
- - Restore error check for super\_fetch's job ack [#3601]
117
- - Trim error messages saved in Batch's failure hash, preventing huge
118
- messages from bloating Redis. [#3570]
119
-
120
- 3.5.2
121
- ---------
122
-
123
- - Fix `Status#completed?` when run against a Batch that had succeeded
124
- and was deleted. [#3519]
125
-
126
- 3.5.1
127
- ---------
128
-
129
- - Work with Sidekiq 5.0.2+
130
- - Improve performance of super\_fetch with weighted queues [#3489]
131
-
132
- 3.5.0
133
- ---------
134
-
135
- - Add queue pause/unpause endpoints for scripting via curl [#3445]
136
- - Change how super\_fetch names private queues to avoid hostname/queue clashes. [#3443]
137
- - Re-implement `Sidekiq::Queue#delete_job` to avoid O(n) runtime [#3408]
138
- - Batch page displays Pending JIDs if less than 10 [#3130]
139
- - Batch page has a Search button to find associated Retries [#3130]
140
- - Make Batch UI progress bar more friendly to the colorblind [#3387]
141
-
142
- 3.4.5
143
- ---------
144
-
145
- - Fix potential job loss with reliable scheduler when lots of jobs are scheduled
146
- at precisely the same time. Thanks to raivil for his hard work in
147
- reproducing the bug. [#3371]
148
-
149
- 3.4.4
150
- ---------
151
-
152
- - Optimize super\_fetch shutdown to restart jobs quicker [#3249]
153
-
154
- 3.4.3
155
- ---------
156
-
157
- - Limit reliable scheduler to enqueue up to 100 jobs per call, minimizing Redis latency [#3332]
158
- - Fix bug in super\_fetch logic for queues with `_` in the name [#3339]
159
-
160
- 3.4.2
161
- ---------
162
-
163
- - Add `Batch::Status#invalidated?` API which returns true if any/all
164
- JIDs were invalidated within the batch. [#3326]
165
-
166
- 3.4.1
167
- ---------
168
-
169
- - Allow super\_fetch's orphan job check to happen as often as every hour [#3273]
170
- - Officially deprecate reliable\_fetch algorithm, I now recommend you use `super_fetch` instead:
171
- ```ruby
172
- Sidekiq.configure_server do |config|
173
- config.super_fetch!
174
- end
175
- ```
176
- Also note that Sidekiq's `-i/--index` option is no longer used/relevant with super\_fetch.
177
- - Don't display "Delete/Retry All" buttons when filtering in Web UI [#3243]
178
- - Reimplement Sidekiq::JobSet#find\_job with ZSCAN [#3197]
179
-
180
- 3.4.0
181
- ---------
182
-
183
- - Introducing the newest reliable fetching algorithm: `super_fetch`! This
184
- algorithm will replace reliable\_fetch in Pro 4.0. super\_fetch is
185
- bullet-proof across all environments, no longer requiring stable
186
- hostnames or an index to be set per-process. [#3077]
187
- ```ruby
188
- Sidekiq.configure_server do |config|
189
- config.super_fetch!
190
- end
191
- ```
192
- Thank you to @jonhyman for code review and the Sidekiq Pro customers that
193
- beta tested super\_fetch.
194
-
195
- 3.3.3
196
- ---------
197
-
198
- - Update Web UI extension to work with Sidekiq 4.2.0's new Web UI. [#3075]
199
-
200
- 3.3.2
201
- ---------
202
-
203
- - Minimize batch memory usage after success [#3083]
204
- - Extract batch's 24 hr linger expiry to a LINGER constant so it can be tuned. [#3011]
205
-
206
- 3.3.1
207
- ---------
208
-
209
- - If environment is unset, treat it as development so reliable\_fetch works as before 3.2.2.
210
-
211
- 3.3.0
212
- ---------
213
-
214
- - Don't delete batches immediately upon success but set a 24 hr expiry, this allows
215
- Sidekiq::Batch::Status#poll to work, even after batch success. [#3011]
216
- - New `Sidekiq::PendingSet#destroy(jid)` API to remove poison pill jobs [#3015]
217
-
218
- 3.2.2
219
- ---------
220
-
221
- - A default value for -i is only set in development now, staging or
222
- other environments must set an index if you wish to use reliable\_fetch. [#2971]
223
- - Fix nil dereference when checking for jobs over timeout in timed\_fetch
224
-
225
-
226
- 3.2.1
227
- ---------
228
-
229
- - timed\_fetch now works with namespaces. [ryansch]
230
-
231
-
232
- 3.2.0
233
- ---------
234
-
235
- - Fixed detection of missing batches, `NoSuchBatch` should be raised
236
- properly now if `Sidekiq::Batch.new(bid)` is called on a batch no
237
- longer in Redis.
238
- - Remove support for Pro 1.x format batches. This version will no
239
- longer seamlessly process batches created with Sidekiq Pro 1.x.
240
- As always, upgrade one major version at a time to ensure a smooth
241
- transition.
242
- - Fix edge case where a parent batch could expire before a child batch
243
- was finished processing, leading to missing batches [#2889]
244
-
245
- 2.1.5
246
- ---------
247
-
248
- - Fix edge case where a parent batch could expire before a child batch
249
- was finished processing, leading to missing batches [#2889]
250
-
251
- 3.1.0
252
- ---------
253
-
254
- - New container-friendly fetch algorithm: `timed_fetch`. See the
255
- [wiki documentation](https://github.com/mperham/sidekiq/wiki/Pro-Reliability-Server)
256
- for trade offs between the two reliability options. You should
257
- use this if you are on Heroku, Docker, Amazon ECS or EBS or
258
- another container-based system.
259
-
260
-
261
- 3.0.6
262
- ---------
263
-
264
- - Fix race condition on reliable fetch shutdown
265
-
266
- 3.0.5
267
- ---------
268
-
269
- - Statsd metrics now account for ActiveJob class names
270
- - Allow reliable fetch internals to be overridden [jonhyman]
271
-
272
- 3.0.4
273
- ---------
274
-
275
- - Queue pausing no longer requires reliable fetch. [#2786]
276
-
277
- 3.0.3, 2.1.4
278
- ------------
279
-
280
- - Convert Lua-based `Sidekiq::Queue#delete_by_class` to Ruby-based, to
281
- avoid O(N^2) performance and possible Redis failure. [#2806]
282
-
283
- 3.0.2
284
- -----------
285
-
286
- - Make job registration with batch part of the atomic push so batch
287
- metadata can't get out of sync with the job data. [#2714]
288
-
289
- 3.0.1
290
- -----------
291
-
292
- - Remove a number of Redis version checks since we can assume 2.8+ now.
293
- - Fix expiring jobs client middleware not loaded on server
294
-
295
- 3.0.0
296
- -----------
297
-
298
- - See the [Pro 3.0 release notes](Pro-3.0-Upgrade.md).
299
-
300
- 2.1.3
301
- -----------
302
-
303
- - Don't enable strict priority if using weighted queueing like `-q a,1 -q b,1`
304
- - Safer JSON mangling in Lua [#2639]
305
-
306
- 2.1.2
307
- -----------
308
-
309
- - Lock Sidekiq Pro 2.x to Sidekiq 3.x.
310
-
311
- 2.1.1
312
- -----------
313
-
314
- - Make ShardSet lazier so Redis can first be initialized at startup. [#2603]
315
-
316
-
317
- 2.1.0
318
- -----------
319
-
320
- - Explicit support for sharding batches. You list your Redis shards and
321
- Sidekiq Pro will randomly spread batches across the shards. The BID
322
- will indicate which shard contains the batch data. Jobs within a
323
- batch may be spread across all shards too. [#2548, jonhyman]
324
- - Officially deprecate Sidekiq::Notifications code. Notifications have
325
- been undocumented for months now. [#2575]
326
-
327
-
328
- 2.0.8
329
- -----------
330
-
331
- - Fix reliable scheduler mangling large numeric arguments. Lua's CJSON
332
- library cannot accurately encode numbers larger than 14 digits! [#2478]
333
-
334
- 2.0.7
335
- -----------
336
-
337
- - Optimize delete of enormous batches (100,000s of jobs) [#2458]
338
-
339
- 2.0.6, 1.9.3
340
- --------------
341
-
342
- - CSRF protection in Sidekiq 3.4.2 broke job filtering in the Web UI [#2442]
343
- - Sidekiq Pro 1.x is now limited to Sidekiq < 3.5.0.
344
-
345
- 2.0.5
346
- -----------
347
-
348
- - Atomic scheduler now sets `enqueued_at` [#2414]
349
- - Batches now account for jobs which are stopped by client middleware [#2406]
350
- - Ignore redundant calls to `Sidekiq::Client.reliable_push!` [#2408]
351
-
352
- 2.0.4
353
- -----------
354
-
355
- - Reliable push now supports sharding [#2409]
356
- - Reliable push now only catches Redis exceptions [#2307]
357
-
358
- 2.0.3
359
- -----------
360
-
361
- - Display Batch callback data on the Batch details page. [#2347]
362
- - Fix incompatibility with Pro Web and Rack middleware. [#2344] Thank
363
- you to Jason Clark for the tip on how to fix it.
364
-
365
- 2.0.2
366
- -----------
367
-
368
- - Multiple Web UIs can now run in the same process. [#2267] If you have
369
- multiple Redis shards, you can mount UIs for all in the same process:
370
- ```ruby
371
- POOL1 = ConnectionPool.new { Redis.new(:url => "redis://localhost:6379/0") }
372
- POOL2 = ConnectionPool.new { Redis.new(:url => "redis://localhost:6378/0") }
373
-
374
- mount Sidekiq::Pro::Web => '/sidekiq' # default
375
- mount Sidekiq::Pro::Web.with(redis_pool: POOL1), at: '/sidekiq1', as: 'sidekiq1' # shard1
376
- mount Sidekiq::Pro::Web.with(redis_pool: POOL2), at: '/sidekiq2', as: 'sidekiq2' # shard2
377
- ```
378
- - **SECURITY** Fix batch XSS in error data. Thanks to moneybird.com for
379
- reporting the issue.
380
-
381
- 2.0.1
382
- -----------
383
-
384
- - Add `batch.callback_queue` so batch callbacks can use a higher
385
- priority queue than jobs. [#2200]
386
- - Gracefully recover if someone runs `SCRIPT FLUSH` on Redis. [#2240]
387
- - Ignore errors when attempting `bulk_requeue`, allowing clean shutdown
388
-
389
- 2.0.0
390
- -----------
391
-
392
- - See [the Upgrade Notes](Pro-2.0-Upgrade.md) for detailed notes.
393
-
394
- 1.9.2
395
- -----------
396
-
397
- - As of 1/1/2015, Sidekiq Pro is hosted on a new dedicated server.
398
- Happy new year and let's hope for 100% uptime!
399
- - Fix bug in reliable\_fetch where jobs could be duplicated if a Sidekiq
400
- process crashed and you were using weighted queues. [#2120]
401
-
402
- 1.9.1
403
- -----------
404
-
405
- - **SECURITY** Fix XSS in batch description, thanks to intercom.io for reporting the
406
- issue. If you don't use batch descriptions, you don't need the fix.
407
-
408
- 1.9.0
409
- -----------
410
-
411
- - Add new expiring jobs feature [#1982]
412
- - Show batch expiration on Batch details page [#1981]
413
- - Add '$' batch success token to the pubsub support. [#1953]
414
-
415
-
416
- 1.8.0
417
- -----------
418
-
419
- - Fix race condition where Batches can complete
420
- before they have been fully defined or only half-defined. Requires
421
- Sidekiq 3.2.3. [#1919]
422
-
423
-
424
- 1.7.6
425
- -----------
426
-
427
- - Quick release to verify #1919
428
-
429
-
430
- 1.7.5
431
- -----------
432
-
433
- - Fix job filtering within the Dead tab.
434
- - Add APIs and wiki documentation for invalidating jobs within a batch.
435
-
436
-
437
- 1.7.4
438
- -----------
439
-
440
- - Awesome ANSI art startup banner!
441
-
442
-
443
- 1.7.3
444
- -----------
445
-
446
- - Batch callbacks should use the same queue as the associated jobs.
447
-
448
- 1.7.2
449
- -----------
450
-
451
- - **DEPRECATION** Use `Batch#on(:complete)` instead of `Batch#notify`.
452
- The specific Campfire, HipChat, email and other notification schemes
453
- will be removed in 2.0.0.
454
- - Remove batch from UI when successful. [#1745]
455
- - Convert batch callbacks to be asynchronous jobs for error handling [#1744]
456
-
457
- 1.7.1
458
- -----------
459
-
460
- - Fix for paused queues being processed for a few seconds when starting
461
- a new Sidekiq process.
462
- - Add a 5 sec delay when starting reliable fetch on Heroku to minimize
463
- any duplicate job processing with another process shutting down.
464
-
465
- 1.7.0
466
- -----------
467
-
468
- - Add ability to pause reliable queues via API.
469
- ```ruby
470
- q = Sidekiq::Queue.new("critical")
471
- q.pause!
472
- q.paused? # => true
473
- q.unpause!
474
- ```
475
-
476
- Sidekiq polls Redis every 10 seconds for paused queues so pausing will take
477
- a few seconds to take effect.
478
-
479
- 1.6.0
480
- -----------
481
-
482
- - Compatible with Sidekiq 3.
483
-
484
- 1.5.1
485
- -----------
486
-
487
- - Due to a breaking API change in Sidekiq 3.0, this version is limited
488
- to Sidekiq 2.x.
489
-
490
- 1.5.0
491
- -----------
492
-
493
- - Fix issue on Heroku where reliable fetch could orphan jobs [#1573]
494
-
495
-
496
- 1.4.3
497
- -----------
498
-
499
- - Reverse sorting of Batches in Web UI [#1098]
500
- - Refactoring for Sidekiq 3.0, Pro now requires Sidekiq 2.17.5
501
-
502
- 1.4.2
503
- -----------
504
-
505
- - Tolerate expired Batches in the web UI.
506
- - Fix 100% CPU usage when using weighted queues and reliable fetch.
507
-
508
- 1.4.1
509
- -----------
510
-
511
- - Add batch progress bar to batch detail page. [#1398]
512
- - Fix race condition in initializing Lua scripts
513
-
514
-
515
- 1.4.0
516
- -----------
517
-
518
- - Default batch expiration has been extended to 3 days, from 1 day previously.
519
- - Batches now sort in the Web UI according to expiry time, not creation time.
520
- - Add user-configurable batch expiry. If your batches might take longer
521
- than 72 hours to process, you can extend the expiration date.
522
-
523
- ```ruby
524
- b = Sidekiq::Batch.new
525
- b.expires_in 5.days
526
- ...
527
- ```
528
-
529
- 1.3.2
530
- -----------
531
-
532
- - Lazy load Lua scripts so a Redis connection is not required on bootup.
533
-
534
- 1.3.1
535
- -----------
536
-
537
- - Fix a gemspec packaging issue which broke the Batch UI.
538
-
539
- 1.3.0
540
- -----------
541
-
542
- Thanks to @jonhyman for his contributions to this Sidekiq Pro release.
543
-
544
- This release includes new functionality based on the SCAN command newly
545
- added to Redis 2.8. Pro still works with Redis 2.4 but some
546
- functionality will be unavailable.
547
-
548
- - Job Filtering in the Web UI!
549
- You can now filter retries and scheduled jobs in the Web UI so you
550
- only see the jobs relevant to your needs. Queues cannot be filtered;
551
- Redis does not provide the same SCAN operation on the LIST type.
552
- **Redis 2.8**
553
- ![Filtering](https://f.cloud.github.com/assets/2911/1619465/f47529f2-5657-11e3-8cd1-33899eb72aad.png)
554
- - SCAN support in the Sidekiq::SortedSet API. Here's an example that
555
- finds all jobs which contain the substring "Warehouse::OrderShip"
556
- and deletes all matching retries. If the set is large, this API
557
- will be **MUCH** faster than standard iteration using each.
558
- **Redis 2.8**
559
- ```ruby
560
- Sidekiq::RetrySet.new.scan("Warehouse::OrderShip") do |job|
561
- job.delete
562
- end
563
- ```
564
-
565
- - Sidekiq::Batch#jobs now returns the set of JIDs added to the batch.
566
- - Sidekiq::Batch#jids returns the complete set of JIDs associated with the batch.
567
- - Sidekiq::Batch#remove\_jobs(jid, jid, ...) removes JIDs from the set, allowing early termination of jobs if they become irrelevant according to application logic.
568
- - Sidekiq::Batch#include?(jid) allows jobs to check if they are still
569
- relevant to a Batch and exit early if not.
570
- - Sidekiq::SortedSet#find\_job(jid) now uses server-side Lua if possible **Redis 2.6** [jonhyman]
571
- - The statsd integration now sets global job counts:
572
- ```ruby
573
- jobs.count
574
- jobs.success
575
- jobs.failure
576
- ```
577
-
578
- - Change shutdown logic to push leftover jobs in the private queue back
579
- into the public queue when shutting down with Reliable Fetch. This
580
- allows the safe decommission of a Sidekiq Pro process when autoscaling. [jonhyman]
581
- - Add support for weighted random fetching with Reliable Fetch [jonhyman]
582
- - Pro now requires Sidekiq 2.17.0
583
-
584
- 1.2.5
585
- -----------
586
-
587
- - Convert Batch UI to use Sidekiq 2.16's support for extension localization.
588
- - Update reliable\_push to work with Sidekiq::Client refactoring in 2.16
589
- - Pro now requires Sidekiq 2.16.0
590
-
591
- 1.2.4
592
- -----------
593
-
594
- - Convert Batch UI to Bootstrap 3
595
- - Pro now requires Sidekiq 2.15.0
596
- - Add Sidekiq::Batch::Status#delete [#1205]
597
-
598
- 1.2.3
599
- -----------
600
-
601
- - Pro now requires Sidekiq 2.14.0
602
- - Fix bad exception handling in batch callbacks [#1134]
603
- - Convert Batch UI to ERB
604
-
605
- 1.2.2
606
- -----------
607
-
608
- - Problem with reliable fetch which could lead to lost jobs when Sidekiq
609
- is shut down normally. Thanks to MikaelAmborn for the report. [#1109]
610
-
611
- 1.2.1
612
- -----------
613
-
614
- - Forgot to push paging code necessary for `delete_job` performance.
615
-
616
- 1.2.0
617
- -----------
618
-
619
- - **LEAK** Fix batch key which didn't expire in Redis. Keys match
620
- /b-[a-f0-9]{16}-pending/, e.g. "b-4f55163ddba10aa0-pending" [#1057]
621
- - **Reliable fetch now supports multiple queues**, using the algorithm spec'd
622
- by @jackrg [#1102]
623
- - Fix issue with reliable\_push where it didn't return the JID for a pushed
624
- job when sending previously cached jobs to Redis.
625
- - Add fast Sidekiq::Queue#delete\_job(jid) API which leverages Lua so job lookup is
626
- 100% server-side. Benchmark vs Sidekiq's Job#delete API. **Redis 2.6**
627
-
628
- ```
629
- Sidekiq Pro API
630
- 0.030000 0.020000 0.050000 ( 1.640659)
631
- Sidekiq API
632
- 17.250000 2.220000 19.470000 ( 22.193300)
633
- ```
634
-
635
- - Add fast Sidekiq::Queue#delete\_by\_class(klass) API to remove all
636
- jobs of a given type. Uses server-side Lua for performance. **Redis 2.6**
637
-
638
- 1.1.0
639
- -----------
640
-
641
- - New `sidekiq/pro/reliable_push` which makes Sidekiq::Client resiliant
642
- to Redis network failures. [#793]
643
- - Move `sidekiq/reliable_fetch` to `sidekiq/pro/reliable_fetch`
644
-
645
-
646
- 1.0.0
647
- -----------
648
-
649
- - Sidekiq Pro changelog moved to mperham/sidekiq for public visibility.
650
- - Add new Rack endpoint for easy polling of batch status via JavaScript. See `sidekiq/rack/batch_status`
651
-
652
- 0.9.3
653
- -----------
654
-
655
- - Fix bad /batches path in Web UI
656
- - Fix Sinatra conflict with sidekiq-failures
657
-
658
- 0.9.2
659
- -----------
660
-
661
- - Fix issue with lifecycle notifications not firing.
662
-
663
- 0.9.1
664
- -----------
665
-
666
- - Update due to Sidekiq API changes.
667
-
668
- 0.9.0
669
- -----------
670
-
671
- - Rearchitect Sidekiq's Fetch code to support different fetch
672
- strategies. Add a ReliableFetch strategy which works with Redis'
673
- RPOPLPUSH to ensure we don't lose messages, even when the Sidekiq
674
- process crashes unexpectedly. [mperham/sidekiq#607]
675
-
676
- 0.8.2
677
- -----------
678
-
679
- - Reimplement existing notifications using batch on_complete events.
680
-
681
- 0.8.1
682
- -----------
683
-
684
- - Rejigger batch callback notifications.
685
-
686
-
687
- 0.8.0
688
- -----------
689
-
690
- - Add new Batch 'callback' notification support, for in-process
691
- notification.
692
- - Symbolize option keys passed to Pony [mperham/sidekiq#603]
693
- - Batch no longer requires the Web UI since Web UI usage is optional.
694
- You must require is manually in your Web process:
695
-
696
- ```ruby
697
- require 'sidekiq/web'
698
- require 'sidekiq/batch/web'
699
- mount Sidekiq::Web => '/sidekiq'
700
- ```
701
-
702
-
703
- 0.7.1
704
- -----------
705
-
706
- - Worker instances can access the associated jid and bid via simple
707
- accessors.
708
- - Batches can now be modified while being processed so, e.g. a batch
709
- job can add additional jobs to its own batch.
710
-
711
- ```ruby
712
- def perform(...)
713
- batch = Sidekiq::Batch.new(bid) # instantiate batch associated with this job
714
- batch.jobs do
715
- SomeWorker.perform_async # add another job
716
- end
717
- end
718
- ```
719
-
720
- - Save error backtraces in batch's failure info for display in Web UI.
721
- - Clean up email notification a bit.
722
-
723
-
724
- 0.7.0
725
- -----------
726
-
727
- - Add optional batch description
728
- - Mutable batches. Batches can now be modified to add additional jobs
729
- at runtime. Example would be a batch job which needs to create more
730
- jobs based on the data it is processing.
731
-
732
- ```ruby
733
- batch = Sidekiq::Batch.new(bid)
734
- batch.jobs do
735
- # define more jobs here
736
- end
737
- ```
738
- - Fix issues with symbols vs strings in option hashes
739
-
740
-
741
- 0.6.1
742
- -----------
743
-
744
- - Webhook notification support
745
-
746
-
747
- 0.6
748
- -----------
749
-
750
- - Redis pubsub
751
- - Email polish
752
-
753
-
754
- 0.5
755
- -----------
756
-
757
- - Batches
758
- - Notifications
759
- - Statsd middleware