sidekiq 6.1.1 → 6.5.10

Sign up to get free protection for your applications and to get access to all the features.
Files changed (118) hide show
  1. checksums.yaml +4 -4
  2. data/Changes.md +250 -3
  3. data/LICENSE +3 -3
  4. data/README.md +10 -6
  5. data/bin/sidekiq +3 -3
  6. data/bin/sidekiqload +70 -66
  7. data/bin/sidekiqmon +1 -1
  8. data/lib/generators/sidekiq/job_generator.rb +57 -0
  9. data/lib/generators/sidekiq/templates/{worker.rb.erb → job.rb.erb} +2 -2
  10. data/lib/generators/sidekiq/templates/{worker_spec.rb.erb → job_spec.rb.erb} +1 -1
  11. data/lib/generators/sidekiq/templates/{worker_test.rb.erb → job_test.rb.erb} +1 -1
  12. data/lib/sidekiq/api.rb +352 -156
  13. data/lib/sidekiq/cli.rb +86 -41
  14. data/lib/sidekiq/client.rb +49 -73
  15. data/lib/sidekiq/{util.rb → component.rb} +12 -14
  16. data/lib/sidekiq/delay.rb +3 -1
  17. data/lib/sidekiq/extensions/action_mailer.rb +3 -2
  18. data/lib/sidekiq/extensions/active_record.rb +1 -1
  19. data/lib/sidekiq/extensions/generic_proxy.rb +4 -2
  20. data/lib/sidekiq/fetch.rb +31 -20
  21. data/lib/sidekiq/job.rb +13 -0
  22. data/lib/sidekiq/job_logger.rb +16 -28
  23. data/lib/sidekiq/job_retry.rb +79 -59
  24. data/lib/sidekiq/job_util.rb +71 -0
  25. data/lib/sidekiq/launcher.rb +126 -65
  26. data/lib/sidekiq/logger.rb +11 -20
  27. data/lib/sidekiq/manager.rb +35 -34
  28. data/lib/sidekiq/metrics/deploy.rb +47 -0
  29. data/lib/sidekiq/metrics/query.rb +153 -0
  30. data/lib/sidekiq/metrics/shared.rb +94 -0
  31. data/lib/sidekiq/metrics/tracking.rb +134 -0
  32. data/lib/sidekiq/middleware/chain.rb +88 -42
  33. data/lib/sidekiq/middleware/current_attributes.rb +63 -0
  34. data/lib/sidekiq/middleware/i18n.rb +6 -4
  35. data/lib/sidekiq/middleware/modules.rb +21 -0
  36. data/lib/sidekiq/monitor.rb +2 -2
  37. data/lib/sidekiq/paginator.rb +17 -9
  38. data/lib/sidekiq/processor.rb +47 -41
  39. data/lib/sidekiq/rails.rb +32 -4
  40. data/lib/sidekiq/redis_client_adapter.rb +154 -0
  41. data/lib/sidekiq/redis_connection.rb +84 -55
  42. data/lib/sidekiq/ring_buffer.rb +29 -0
  43. data/lib/sidekiq/scheduled.rb +96 -32
  44. data/lib/sidekiq/testing/inline.rb +4 -4
  45. data/lib/sidekiq/testing.rb +38 -39
  46. data/lib/sidekiq/transaction_aware_client.rb +45 -0
  47. data/lib/sidekiq/version.rb +1 -1
  48. data/lib/sidekiq/web/action.rb +3 -3
  49. data/lib/sidekiq/web/application.rb +41 -16
  50. data/lib/sidekiq/web/csrf_protection.rb +32 -5
  51. data/lib/sidekiq/web/helpers.rb +52 -30
  52. data/lib/sidekiq/web/router.rb +4 -1
  53. data/lib/sidekiq/web.rb +38 -78
  54. data/lib/sidekiq/worker.rb +142 -16
  55. data/lib/sidekiq.rb +114 -31
  56. data/sidekiq.gemspec +12 -4
  57. data/web/assets/images/apple-touch-icon.png +0 -0
  58. data/web/assets/javascripts/application.js +114 -60
  59. data/web/assets/javascripts/chart.min.js +13 -0
  60. data/web/assets/javascripts/chartjs-plugin-annotation.min.js +7 -0
  61. data/web/assets/javascripts/dashboard.js +50 -67
  62. data/web/assets/javascripts/graph.js +16 -0
  63. data/web/assets/javascripts/metrics.js +262 -0
  64. data/web/assets/stylesheets/application-dark.css +61 -51
  65. data/web/assets/stylesheets/application-rtl.css +0 -4
  66. data/web/assets/stylesheets/application.css +84 -243
  67. data/web/locales/ar.yml +8 -2
  68. data/web/locales/el.yml +43 -19
  69. data/web/locales/en.yml +11 -1
  70. data/web/locales/es.yml +18 -2
  71. data/web/locales/fr.yml +8 -1
  72. data/web/locales/ja.yml +10 -0
  73. data/web/locales/lt.yml +1 -1
  74. data/web/locales/pt-br.yml +27 -9
  75. data/web/locales/ru.yml +4 -0
  76. data/web/locales/zh-cn.yml +36 -11
  77. data/web/locales/zh-tw.yml +32 -7
  78. data/web/views/_footer.erb +1 -1
  79. data/web/views/_job_info.erb +1 -1
  80. data/web/views/_nav.erb +1 -1
  81. data/web/views/_poll_link.erb +2 -5
  82. data/web/views/_summary.erb +7 -7
  83. data/web/views/busy.erb +57 -21
  84. data/web/views/dashboard.erb +23 -14
  85. data/web/views/dead.erb +1 -1
  86. data/web/views/layout.erb +2 -1
  87. data/web/views/metrics.erb +69 -0
  88. data/web/views/metrics_for_job.erb +87 -0
  89. data/web/views/morgue.erb +6 -6
  90. data/web/views/queue.erb +15 -11
  91. data/web/views/queues.erb +4 -4
  92. data/web/views/retries.erb +7 -7
  93. data/web/views/retry.erb +1 -1
  94. data/web/views/scheduled.erb +1 -1
  95. metadata +52 -39
  96. data/.circleci/config.yml +0 -71
  97. data/.github/contributing.md +0 -32
  98. data/.github/issue_template.md +0 -11
  99. data/.gitignore +0 -13
  100. data/.standard.yml +0 -20
  101. data/3.0-Upgrade.md +0 -70
  102. data/4.0-Upgrade.md +0 -53
  103. data/5.0-Upgrade.md +0 -56
  104. data/6.0-Upgrade.md +0 -72
  105. data/COMM-LICENSE +0 -97
  106. data/Ent-2.0-Upgrade.md +0 -37
  107. data/Ent-Changes.md +0 -275
  108. data/Gemfile +0 -24
  109. data/Gemfile.lock +0 -208
  110. data/Pro-2.0-Upgrade.md +0 -138
  111. data/Pro-3.0-Upgrade.md +0 -44
  112. data/Pro-4.0-Upgrade.md +0 -35
  113. data/Pro-5.0-Upgrade.md +0 -25
  114. data/Pro-Changes.md +0 -795
  115. data/Rakefile +0 -10
  116. data/code_of_conduct.md +0 -50
  117. data/lib/generators/sidekiq/worker_generator.rb +0 -57
  118. data/lib/sidekiq/exception_handler.rb +0 -27
data/Pro-Changes.md DELETED
@@ -1,795 +0,0 @@
1
- # Sidekiq Pro Changelog
2
-
3
- [Sidekiq Changes](https://github.com/mperham/sidekiq/blob/master/Changes.md) | [Sidekiq Pro Changes](https://github.com/mperham/sidekiq/blob/master/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/mperham/sidekiq/blob/master/Ent-Changes.md)
4
-
5
- Please see [sidekiq.org](https://sidekiq.org/) for more details and how to buy.
6
-
7
- 5.1.1
8
- ---------
9
-
10
- - Fix broken basic fetcher [#4616]
11
-
12
- 5.1.0
13
- ---------
14
-
15
- - Remove old Statsd metrics with `WorkerName` in the name [#4377]
16
- ```
17
- job.WorkerName.count -> job.count with tag worker:WorkerName
18
- job.WorkerName.perform -> job.perform with tag worker:WorkerName
19
- job.WorkerName.failure -> job.failure with tag worker:WorkerName
20
- ```
21
- - Remove `concurrent-ruby` gem dependency [#4586]
22
- - Update `constantize` for batch callbacks. [#4469]
23
- - Add queue tag to `jobs.recovered.fetch` metric [#4594]
24
- - Refactor Pro's fetch infrastructure [#4602]
25
-
26
- 5.0.1
27
- ---------
28
-
29
- - Rejigger batch failures UI to add direct links to retries and scheduled jobs [#4209]
30
- - Delete batch data with `UNLINK` [#4155]
31
- - Fix bug where a scheduled job can lose its scheduled time when using reliable push [#4267]
32
- - Sidekiq::JobSet#scan and #find_job APIs have been promoted to Sidekiq OSS. [#4259]
33
-
34
- 5.0.0
35
- ---------
36
-
37
- - There is no significant migration from Sidekiq Pro 4.0 to 5.0
38
- but make sure you read the [update notes for Sidekiq
39
- 6.0](https://github.com/mperham/sidekiq/blob/master/6.0-Upgrade.md).
40
- - Removed various deprecated APIs and associated warnings.
41
- - **BREAKING CHANGE** Remove the `Sidekiq::Batch::Status#dead_jobs` API in favor of
42
- `Sidekiq::Batch::Status#dead_jids`. [#4217]
43
- - Update Sidekiq Pro codebase to use StandardRB formatting
44
- - Fix lingering "b-XXX-died" elements in Redis which could cause
45
- excessive memory usage. [#4217]
46
- - Add ES translations, see issues [#3949](https://github.com/mperham/sidekiq/issues/3949) and [#3951](https://github.com/mperham/sidekiq/issues/3951) to add your own language.
47
-
48
- 4.0.5
49
- ---------
50
-
51
- - Increase super\_fetch retriever thread count from 1 to 2 to make it
52
- less sensitive to Redis latency.
53
- - Better handling of invalid job JSON by reliable scheduler [#4053]
54
- - Added ZH, PT, JA and RU translations.
55
-
56
- 4.0.4
57
- ---------
58
-
59
- - Update Sidekiq::Client patches to work with new Module#prepend
60
- mechanism in Sidekiq 5.2.0. [#3930]
61
-
62
- 4.0.3
63
- ---------
64
-
65
- - Add at\_exit handler to push any saved jobs in `reliable_push` when exiting. [#3823]
66
- - Implement batch death callback. This is fired the first time a job within a batch dies. [#3841]
67
- ```ruby
68
- batch = Sidekiq::Batch.new
69
- batch.on(:death, ...)
70
- ```
71
-
72
- 4.0.2
73
- ---------
74
-
75
- - Remove super\_fetch edge case leading to an unnecessary `sleep(1)`
76
- call and resulting latency [#3790]
77
- - Fix possible bad statsd metric call on super\_fetch startup
78
- - Remove superfluous `freeze` calls on Strings [#3759]
79
-
80
- 4.0.1
81
- ---------
82
-
83
- - Fix incompatibility with the statsd-ruby gem [#3740]
84
- - Add tags to Statsd metrics when using Datadog [#3744]
85
-
86
- 4.0.0
87
- ---------
88
-
89
- - See the [Sidekiq Pro 4.0](Pro-4.0-Upgrade.md) release notes.
90
-
91
-
92
- 3.7.1
93
- ---------
94
-
95
- - Deprecate timed\_fetch. Switch to super\_fetch:
96
- ```ruby
97
- config.super_fetch!
98
- ```
99
-
100
-
101
- 3.7.0
102
- ---------
103
-
104
- - Refactor batch job success/failure to gracefully handle several edge
105
- cases with regard to Sidekiq::Shutdown. This should greatly reduce
106
- the chances of seeing the long-standing "negative pending count" problem. [#3710]
107
-
108
-
109
- 3.6.1
110
- ---------
111
-
112
- - Add support for Datadog::Statsd, it is the recommended Statsd client. [#3699]
113
- ```ruby
114
- Sidekiq::Pro.dogstatsd = ->{ Datadog::Statsd.new("metrics.example.com", 8125) }
115
- ```
116
- - Size the statsd connection pool based on Sidekiq's concurrency [#3700]
117
-
118
-
119
- 3.6.0
120
- ---------
121
-
122
- This release overhauls the Statsd metrics support and adds more
123
- metrics for tracking Pro feature usage. In your initializer:
124
- ```ruby
125
- Sidekiq::Pro.statsd = ->{ ::Statsd.new("127.0.0.1", 8125) }
126
- ```
127
- Sidekiq Pro will emit more metrics to Statsd:
128
- ```
129
- jobs.expired - when a job is expired
130
- jobs.recovered.push - when a job is recovered by reliable_push after network outage
131
- jobs.recovered.fetch - when a job is recovered by super_fetch after process crash
132
- batch.created - when a batch is created
133
- batch.complete - when a batch is completed
134
- batch.success - when a batch is successful
135
- ```
136
- Sidekiq Pro's existing Statsd middleware has been rewritten to leverage the new API.
137
- Everything should be backwards compatible with one deprecation notice.
138
-
139
-
140
- 3.5.4
141
- ---------
142
-
143
- - Fix case in SuperFetch where Redis downtime can lead to processor thread death [#3684]
144
- - Fix case where TimedFetch might not recover some pending jobs
145
- - Fix edge case in Batch::Status#poll leading to premature completion [#3640]
146
- - Adjust scan API to check 100 elements at a time, to minimize network round trips
147
- when scanning large sets.
148
-
149
- 3.5.3
150
- ---------
151
-
152
- - Restore error check for super\_fetch's job ack [#3601]
153
- - Trim error messages saved in Batch's failure hash, preventing huge
154
- messages from bloating Redis. [#3570]
155
-
156
- 3.5.2
157
- ---------
158
-
159
- - Fix `Status#completed?` when run against a Batch that had succeeded
160
- and was deleted. [#3519]
161
-
162
- 3.5.1
163
- ---------
164
-
165
- - Work with Sidekiq 5.0.2+
166
- - Improve performance of super\_fetch with weighted queues [#3489]
167
-
168
- 3.5.0
169
- ---------
170
-
171
- - Add queue pause/unpause endpoints for scripting via curl [#3445]
172
- - Change how super\_fetch names private queues to avoid hostname/queue clashes. [#3443]
173
- - Re-implement `Sidekiq::Queue#delete_job` to avoid O(n) runtime [#3408]
174
- - Batch page displays Pending JIDs if less than 10 [#3130]
175
- - Batch page has a Search button to find associated Retries [#3130]
176
- - Make Batch UI progress bar more friendly to the colorblind [#3387]
177
-
178
- 3.4.5
179
- ---------
180
-
181
- - Fix potential job loss with reliable scheduler when lots of jobs are scheduled
182
- at precisely the same time. Thanks to raivil for his hard work in
183
- reproducing the bug. [#3371]
184
-
185
- 3.4.4
186
- ---------
187
-
188
- - Optimize super\_fetch shutdown to restart jobs quicker [#3249]
189
-
190
- 3.4.3
191
- ---------
192
-
193
- - Limit reliable scheduler to enqueue up to 100 jobs per call, minimizing Redis latency [#3332]
194
- - Fix bug in super\_fetch logic for queues with `_` in the name [#3339]
195
-
196
- 3.4.2
197
- ---------
198
-
199
- - Add `Batch::Status#invalidated?` API which returns true if any/all
200
- JIDs were invalidated within the batch. [#3326]
201
-
202
- 3.4.1
203
- ---------
204
-
205
- - Allow super\_fetch's orphan job check to happen as often as every hour [#3273]
206
- - Officially deprecate reliable\_fetch algorithm, I now recommend you use `super_fetch` instead:
207
- ```ruby
208
- Sidekiq.configure_server do |config|
209
- config.super_fetch!
210
- end
211
- ```
212
- Also note that Sidekiq's `-i/--index` option is no longer used/relevant with super\_fetch.
213
- - Don't display "Delete/Retry All" buttons when filtering in Web UI [#3243]
214
- - Reimplement Sidekiq::JobSet#find\_job with ZSCAN [#3197]
215
-
216
- 3.4.0
217
- ---------
218
-
219
- - Introducing the newest reliable fetching algorithm: `super_fetch`! This
220
- algorithm will replace reliable\_fetch in Pro 4.0. super\_fetch is
221
- bullet-proof across all environments, no longer requiring stable
222
- hostnames or an index to be set per-process. [#3077]
223
- ```ruby
224
- Sidekiq.configure_server do |config|
225
- config.super_fetch!
226
- end
227
- ```
228
- Thank you to @jonhyman for code review and the Sidekiq Pro customers that
229
- beta tested super\_fetch.
230
-
231
- 3.3.3
232
- ---------
233
-
234
- - Update Web UI extension to work with Sidekiq 4.2.0's new Web UI. [#3075]
235
-
236
- 3.3.2
237
- ---------
238
-
239
- - Minimize batch memory usage after success [#3083]
240
- - Extract batch's 24 hr linger expiry to a LINGER constant so it can be tuned. [#3011]
241
-
242
- 3.3.1
243
- ---------
244
-
245
- - If environment is unset, treat it as development so reliable\_fetch works as before 3.2.2.
246
-
247
- 3.3.0
248
- ---------
249
-
250
- - Don't delete batches immediately upon success but set a 24 hr expiry, this allows
251
- Sidekiq::Batch::Status#poll to work, even after batch success. [#3011]
252
- - New `Sidekiq::PendingSet#destroy(jid)` API to remove poison pill jobs [#3015]
253
-
254
- 3.2.2
255
- ---------
256
-
257
- - A default value for -i is only set in development now, staging or
258
- other environments must set an index if you wish to use reliable\_fetch. [#2971]
259
- - Fix nil dereference when checking for jobs over timeout in timed\_fetch
260
-
261
-
262
- 3.2.1
263
- ---------
264
-
265
- - timed\_fetch now works with namespaces. [ryansch]
266
-
267
-
268
- 3.2.0
269
- ---------
270
-
271
- - Fixed detection of missing batches, `NoSuchBatch` should be raised
272
- properly now if `Sidekiq::Batch.new(bid)` is called on a batch no
273
- longer in Redis.
274
- - Remove support for Pro 1.x format batches. This version will no
275
- longer seamlessly process batches created with Sidekiq Pro 1.x.
276
- As always, upgrade one major version at a time to ensure a smooth
277
- transition.
278
- - Fix edge case where a parent batch could expire before a child batch
279
- was finished processing, leading to missing batches [#2889]
280
-
281
- 2.1.5
282
- ---------
283
-
284
- - Fix edge case where a parent batch could expire before a child batch
285
- was finished processing, leading to missing batches [#2889]
286
-
287
- 3.1.0
288
- ---------
289
-
290
- - New container-friendly fetch algorithm: `timed_fetch`. See the
291
- [wiki documentation](https://github.com/mperham/sidekiq/wiki/Pro-Reliability-Server)
292
- for trade offs between the two reliability options. You should
293
- use this if you are on Heroku, Docker, Amazon ECS or EBS or
294
- another container-based system.
295
-
296
-
297
- 3.0.6
298
- ---------
299
-
300
- - Fix race condition on reliable fetch shutdown
301
-
302
- 3.0.5
303
- ---------
304
-
305
- - Statsd metrics now account for ActiveJob class names
306
- - Allow reliable fetch internals to be overridden [jonhyman]
307
-
308
- 3.0.4
309
- ---------
310
-
311
- - Queue pausing no longer requires reliable fetch. [#2786]
312
-
313
- 3.0.3, 2.1.4
314
- ------------
315
-
316
- - Convert Lua-based `Sidekiq::Queue#delete_by_class` to Ruby-based, to
317
- avoid O(N^2) performance and possible Redis failure. [#2806]
318
-
319
- 3.0.2
320
- -----------
321
-
322
- - Make job registration with batch part of the atomic push so batch
323
- metadata can't get out of sync with the job data. [#2714]
324
-
325
- 3.0.1
326
- -----------
327
-
328
- - Remove a number of Redis version checks since we can assume 2.8+ now.
329
- - Fix expiring jobs client middleware not loaded on server
330
-
331
- 3.0.0
332
- -----------
333
-
334
- - See the [Pro 3.0 release notes](Pro-3.0-Upgrade.md).
335
-
336
- 2.1.3
337
- -----------
338
-
339
- - Don't enable strict priority if using weighted queueing like `-q a,1 -q b,1`
340
- - Safer JSON mangling in Lua [#2639]
341
-
342
- 2.1.2
343
- -----------
344
-
345
- - Lock Sidekiq Pro 2.x to Sidekiq 3.x.
346
-
347
- 2.1.1
348
- -----------
349
-
350
- - Make ShardSet lazier so Redis can first be initialized at startup. [#2603]
351
-
352
-
353
- 2.1.0
354
- -----------
355
-
356
- - Explicit support for sharding batches. You list your Redis shards and
357
- Sidekiq Pro will randomly spread batches across the shards. The BID
358
- will indicate which shard contains the batch data. Jobs within a
359
- batch may be spread across all shards too. [#2548, jonhyman]
360
- - Officially deprecate Sidekiq::Notifications code. Notifications have
361
- been undocumented for months now. [#2575]
362
-
363
-
364
- 2.0.8
365
- -----------
366
-
367
- - Fix reliable scheduler mangling large numeric arguments. Lua's CJSON
368
- library cannot accurately encode numbers larger than 14 digits! [#2478]
369
-
370
- 2.0.7
371
- -----------
372
-
373
- - Optimize delete of enormous batches (100,000s of jobs) [#2458]
374
-
375
- 2.0.6, 1.9.3
376
- --------------
377
-
378
- - CSRF protection in Sidekiq 3.4.2 broke job filtering in the Web UI [#2442]
379
- - Sidekiq Pro 1.x is now limited to Sidekiq < 3.5.0.
380
-
381
- 2.0.5
382
- -----------
383
-
384
- - Atomic scheduler now sets `enqueued_at` [#2414]
385
- - Batches now account for jobs which are stopped by client middleware [#2406]
386
- - Ignore redundant calls to `Sidekiq::Client.reliable_push!` [#2408]
387
-
388
- 2.0.4
389
- -----------
390
-
391
- - Reliable push now supports sharding [#2409]
392
- - Reliable push now only catches Redis exceptions [#2307]
393
-
394
- 2.0.3
395
- -----------
396
-
397
- - Display Batch callback data on the Batch details page. [#2347]
398
- - Fix incompatibility with Pro Web and Rack middleware. [#2344] Thank
399
- you to Jason Clark for the tip on how to fix it.
400
-
401
- 2.0.2
402
- -----------
403
-
404
- - Multiple Web UIs can now run in the same process. [#2267] If you have
405
- multiple Redis shards, you can mount UIs for all in the same process:
406
- ```ruby
407
- POOL1 = ConnectionPool.new { Redis.new(:url => "redis://localhost:6379/0") }
408
- POOL2 = ConnectionPool.new { Redis.new(:url => "redis://localhost:6378/0") }
409
-
410
- mount Sidekiq::Pro::Web => '/sidekiq' # default
411
- mount Sidekiq::Pro::Web.with(redis_pool: POOL1), at: '/sidekiq1', as: 'sidekiq1' # shard1
412
- mount Sidekiq::Pro::Web.with(redis_pool: POOL2), at: '/sidekiq2', as: 'sidekiq2' # shard2
413
- ```
414
- - **SECURITY** Fix batch XSS in error data. Thanks to moneybird.com for
415
- reporting the issue.
416
-
417
- 2.0.1
418
- -----------
419
-
420
- - Add `batch.callback_queue` so batch callbacks can use a higher
421
- priority queue than jobs. [#2200]
422
- - Gracefully recover if someone runs `SCRIPT FLUSH` on Redis. [#2240]
423
- - Ignore errors when attempting `bulk_requeue`, allowing clean shutdown
424
-
425
- 2.0.0
426
- -----------
427
-
428
- - See [the Upgrade Notes](Pro-2.0-Upgrade.md) for detailed notes.
429
-
430
- 1.9.2
431
- -----------
432
-
433
- - As of 1/1/2015, Sidekiq Pro is hosted on a new dedicated server.
434
- Happy new year and let's hope for 100% uptime!
435
- - Fix bug in reliable\_fetch where jobs could be duplicated if a Sidekiq
436
- process crashed and you were using weighted queues. [#2120]
437
-
438
- 1.9.1
439
- -----------
440
-
441
- - **SECURITY** Fix XSS in batch description, thanks to intercom.io for reporting the
442
- issue. If you don't use batch descriptions, you don't need the fix.
443
-
444
- 1.9.0
445
- -----------
446
-
447
- - Add new expiring jobs feature [#1982]
448
- - Show batch expiration on Batch details page [#1981]
449
- - Add '$' batch success token to the pubsub support. [#1953]
450
-
451
-
452
- 1.8.0
453
- -----------
454
-
455
- - Fix race condition where Batches can complete
456
- before they have been fully defined or only half-defined. Requires
457
- Sidekiq 3.2.3. [#1919]
458
-
459
-
460
- 1.7.6
461
- -----------
462
-
463
- - Quick release to verify #1919
464
-
465
-
466
- 1.7.5
467
- -----------
468
-
469
- - Fix job filtering within the Dead tab.
470
- - Add APIs and wiki documentation for invalidating jobs within a batch.
471
-
472
-
473
- 1.7.4
474
- -----------
475
-
476
- - Awesome ANSI art startup banner!
477
-
478
-
479
- 1.7.3
480
- -----------
481
-
482
- - Batch callbacks should use the same queue as the associated jobs.
483
-
484
- 1.7.2
485
- -----------
486
-
487
- - **DEPRECATION** Use `Batch#on(:complete)` instead of `Batch#notify`.
488
- The specific Campfire, HipChat, email and other notification schemes
489
- will be removed in 2.0.0.
490
- - Remove batch from UI when successful. [#1745]
491
- - Convert batch callbacks to be asynchronous jobs for error handling [#1744]
492
-
493
- 1.7.1
494
- -----------
495
-
496
- - Fix for paused queues being processed for a few seconds when starting
497
- a new Sidekiq process.
498
- - Add a 5 sec delay when starting reliable fetch on Heroku to minimize
499
- any duplicate job processing with another process shutting down.
500
-
501
- 1.7.0
502
- -----------
503
-
504
- - Add ability to pause reliable queues via API.
505
- ```ruby
506
- q = Sidekiq::Queue.new("critical")
507
- q.pause!
508
- q.paused? # => true
509
- q.unpause!
510
- ```
511
-
512
- Sidekiq polls Redis every 10 seconds for paused queues so pausing will take
513
- a few seconds to take effect.
514
-
515
- 1.6.0
516
- -----------
517
-
518
- - Compatible with Sidekiq 3.
519
-
520
- 1.5.1
521
- -----------
522
-
523
- - Due to a breaking API change in Sidekiq 3.0, this version is limited
524
- to Sidekiq 2.x.
525
-
526
- 1.5.0
527
- -----------
528
-
529
- - Fix issue on Heroku where reliable fetch could orphan jobs [#1573]
530
-
531
-
532
- 1.4.3
533
- -----------
534
-
535
- - Reverse sorting of Batches in Web UI [#1098]
536
- - Refactoring for Sidekiq 3.0, Pro now requires Sidekiq 2.17.5
537
-
538
- 1.4.2
539
- -----------
540
-
541
- - Tolerate expired Batches in the web UI.
542
- - Fix 100% CPU usage when using weighted queues and reliable fetch.
543
-
544
- 1.4.1
545
- -----------
546
-
547
- - Add batch progress bar to batch detail page. [#1398]
548
- - Fix race condition in initializing Lua scripts
549
-
550
-
551
- 1.4.0
552
- -----------
553
-
554
- - Default batch expiration has been extended to 3 days, from 1 day previously.
555
- - Batches now sort in the Web UI according to expiry time, not creation time.
556
- - Add user-configurable batch expiry. If your batches might take longer
557
- than 72 hours to process, you can extend the expiration date.
558
-
559
- ```ruby
560
- b = Sidekiq::Batch.new
561
- b.expires_in 5.days
562
- ...
563
- ```
564
-
565
- 1.3.2
566
- -----------
567
-
568
- - Lazy load Lua scripts so a Redis connection is not required on bootup.
569
-
570
- 1.3.1
571
- -----------
572
-
573
- - Fix a gemspec packaging issue which broke the Batch UI.
574
-
575
- 1.3.0
576
- -----------
577
-
578
- Thanks to @jonhyman for his contributions to this Sidekiq Pro release.
579
-
580
- This release includes new functionality based on the SCAN command newly
581
- added to Redis 2.8. Pro still works with Redis 2.4 but some
582
- functionality will be unavailable.
583
-
584
- - Job Filtering in the Web UI!
585
- You can now filter retries and scheduled jobs in the Web UI so you
586
- only see the jobs relevant to your needs. Queues cannot be filtered;
587
- Redis does not provide the same SCAN operation on the LIST type.
588
- **Redis 2.8**
589
- ![Filtering](https://f.cloud.github.com/assets/2911/1619465/f47529f2-5657-11e3-8cd1-33899eb72aad.png)
590
- - SCAN support in the Sidekiq::SortedSet API. Here's an example that
591
- finds all jobs which contain the substring "Warehouse::OrderShip"
592
- and deletes all matching retries. If the set is large, this API
593
- will be **MUCH** faster than standard iteration using each.
594
- **Redis 2.8**
595
- ```ruby
596
- Sidekiq::RetrySet.new.scan("Warehouse::OrderShip") do |job|
597
- job.delete
598
- end
599
- ```
600
-
601
- - Sidekiq::Batch#jobs now returns the set of JIDs added to the batch.
602
- - Sidekiq::Batch#jids returns the complete set of JIDs associated with the batch.
603
- - Sidekiq::Batch#remove\_jobs(jid, jid, ...) removes JIDs from the set, allowing early termination of jobs if they become irrelevant according to application logic.
604
- - Sidekiq::Batch#include?(jid) allows jobs to check if they are still
605
- relevant to a Batch and exit early if not.
606
- - Sidekiq::SortedSet#find\_job(jid) now uses server-side Lua if possible **Redis 2.6** [jonhyman]
607
- - The statsd integration now sets global job counts:
608
- ```ruby
609
- jobs.count
610
- jobs.success
611
- jobs.failure
612
- ```
613
-
614
- - Change shutdown logic to push leftover jobs in the private queue back
615
- into the public queue when shutting down with Reliable Fetch. This
616
- allows the safe decommission of a Sidekiq Pro process when autoscaling. [jonhyman]
617
- - Add support for weighted random fetching with Reliable Fetch [jonhyman]
618
- - Pro now requires Sidekiq 2.17.0
619
-
620
- 1.2.5
621
- -----------
622
-
623
- - Convert Batch UI to use Sidekiq 2.16's support for extension localization.
624
- - Update reliable\_push to work with Sidekiq::Client refactoring in 2.16
625
- - Pro now requires Sidekiq 2.16.0
626
-
627
- 1.2.4
628
- -----------
629
-
630
- - Convert Batch UI to Bootstrap 3
631
- - Pro now requires Sidekiq 2.15.0
632
- - Add Sidekiq::Batch::Status#delete [#1205]
633
-
634
- 1.2.3
635
- -----------
636
-
637
- - Pro now requires Sidekiq 2.14.0
638
- - Fix bad exception handling in batch callbacks [#1134]
639
- - Convert Batch UI to ERB
640
-
641
- 1.2.2
642
- -----------
643
-
644
- - Problem with reliable fetch which could lead to lost jobs when Sidekiq
645
- is shut down normally. Thanks to MikaelAmborn for the report. [#1109]
646
-
647
- 1.2.1
648
- -----------
649
-
650
- - Forgot to push paging code necessary for `delete_job` performance.
651
-
652
- 1.2.0
653
- -----------
654
-
655
- - **LEAK** Fix batch key which didn't expire in Redis. Keys match
656
- /b-[a-f0-9]{16}-pending/, e.g. "b-4f55163ddba10aa0-pending" [#1057]
657
- - **Reliable fetch now supports multiple queues**, using the algorithm spec'd
658
- by @jackrg [#1102]
659
- - Fix issue with reliable\_push where it didn't return the JID for a pushed
660
- job when sending previously cached jobs to Redis.
661
- - Add fast Sidekiq::Queue#delete\_job(jid) API which leverages Lua so job lookup is
662
- 100% server-side. Benchmark vs Sidekiq's Job#delete API. **Redis 2.6**
663
-
664
- ```
665
- Sidekiq Pro API
666
- 0.030000 0.020000 0.050000 ( 1.640659)
667
- Sidekiq API
668
- 17.250000 2.220000 19.470000 ( 22.193300)
669
- ```
670
-
671
- - Add fast Sidekiq::Queue#delete\_by\_class(klass) API to remove all
672
- jobs of a given type. Uses server-side Lua for performance. **Redis 2.6**
673
-
674
- 1.1.0
675
- -----------
676
-
677
- - New `sidekiq/pro/reliable_push` which makes Sidekiq::Client resiliant
678
- to Redis network failures. [#793]
679
- - Move `sidekiq/reliable_fetch` to `sidekiq/pro/reliable_fetch`
680
-
681
-
682
- 1.0.0
683
- -----------
684
-
685
- - Sidekiq Pro changelog moved to mperham/sidekiq for public visibility.
686
- - Add new Rack endpoint for easy polling of batch status via JavaScript. See `sidekiq/rack/batch_status`
687
-
688
- 0.9.3
689
- -----------
690
-
691
- - Fix bad /batches path in Web UI
692
- - Fix Sinatra conflict with sidekiq-failures
693
-
694
- 0.9.2
695
- -----------
696
-
697
- - Fix issue with lifecycle notifications not firing.
698
-
699
- 0.9.1
700
- -----------
701
-
702
- - Update due to Sidekiq API changes.
703
-
704
- 0.9.0
705
- -----------
706
-
707
- - Rearchitect Sidekiq's Fetch code to support different fetch
708
- strategies. Add a ReliableFetch strategy which works with Redis'
709
- RPOPLPUSH to ensure we don't lose messages, even when the Sidekiq
710
- process crashes unexpectedly. [mperham/sidekiq#607]
711
-
712
- 0.8.2
713
- -----------
714
-
715
- - Reimplement existing notifications using batch on_complete events.
716
-
717
- 0.8.1
718
- -----------
719
-
720
- - Rejigger batch callback notifications.
721
-
722
-
723
- 0.8.0
724
- -----------
725
-
726
- - Add new Batch 'callback' notification support, for in-process
727
- notification.
728
- - Symbolize option keys passed to Pony [mperham/sidekiq#603]
729
- - Batch no longer requires the Web UI since Web UI usage is optional.
730
- You must require is manually in your Web process:
731
-
732
- ```ruby
733
- require 'sidekiq/web'
734
- require 'sidekiq/batch/web'
735
- mount Sidekiq::Web => '/sidekiq'
736
- ```
737
-
738
-
739
- 0.7.1
740
- -----------
741
-
742
- - Worker instances can access the associated jid and bid via simple
743
- accessors.
744
- - Batches can now be modified while being processed so, e.g. a batch
745
- job can add additional jobs to its own batch.
746
-
747
- ```ruby
748
- def perform(...)
749
- batch = Sidekiq::Batch.new(bid) # instantiate batch associated with this job
750
- batch.jobs do
751
- SomeWorker.perform_async # add another job
752
- end
753
- end
754
- ```
755
-
756
- - Save error backtraces in batch's failure info for display in Web UI.
757
- - Clean up email notification a bit.
758
-
759
-
760
- 0.7.0
761
- -----------
762
-
763
- - Add optional batch description
764
- - Mutable batches. Batches can now be modified to add additional jobs
765
- at runtime. Example would be a batch job which needs to create more
766
- jobs based on the data it is processing.
767
-
768
- ```ruby
769
- batch = Sidekiq::Batch.new(bid)
770
- batch.jobs do
771
- # define more jobs here
772
- end
773
- ```
774
- - Fix issues with symbols vs strings in option hashes
775
-
776
-
777
- 0.6.1
778
- -----------
779
-
780
- - Webhook notification support
781
-
782
-
783
- 0.6
784
- -----------
785
-
786
- - Redis pubsub
787
- - Email polish
788
-
789
-
790
- 0.5
791
- -----------
792
-
793
- - Batches
794
- - Notifications
795
- - Statsd middleware