sidekiq 6.1.2 → 6.5.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.


This version of sidekiq might be problematic. Click here for more details.

Files changed (113) hide show
  1. checksums.yaml +4 -4
  2. data/Changes.md +215 -2
  3. data/LICENSE +3 -3
  4. data/README.md +9 -4
  5. data/bin/sidekiq +3 -3
  6. data/bin/sidekiqload +70 -66
  7. data/bin/sidekiqmon +1 -1
  8. data/lib/generators/sidekiq/job_generator.rb +57 -0
  9. data/lib/generators/sidekiq/templates/{worker.rb.erb → job.rb.erb} +2 -2
  10. data/lib/generators/sidekiq/templates/{worker_spec.rb.erb → job_spec.rb.erb} +1 -1
  11. data/lib/generators/sidekiq/templates/{worker_test.rb.erb → job_test.rb.erb} +1 -1
  12. data/lib/sidekiq/api.rb +321 -145
  13. data/lib/sidekiq/cli.rb +73 -40
  14. data/lib/sidekiq/client.rb +48 -72
  15. data/lib/sidekiq/{util.rb → component.rb} +12 -14
  16. data/lib/sidekiq/delay.rb +3 -1
  17. data/lib/sidekiq/extensions/generic_proxy.rb +4 -2
  18. data/lib/sidekiq/fetch.rb +31 -20
  19. data/lib/sidekiq/job.rb +13 -0
  20. data/lib/sidekiq/job_logger.rb +16 -28
  21. data/lib/sidekiq/job_retry.rb +79 -59
  22. data/lib/sidekiq/job_util.rb +71 -0
  23. data/lib/sidekiq/launcher.rb +126 -65
  24. data/lib/sidekiq/logger.rb +11 -20
  25. data/lib/sidekiq/manager.rb +35 -34
  26. data/lib/sidekiq/metrics/deploy.rb +47 -0
  27. data/lib/sidekiq/metrics/query.rb +153 -0
  28. data/lib/sidekiq/metrics/shared.rb +94 -0
  29. data/lib/sidekiq/metrics/tracking.rb +134 -0
  30. data/lib/sidekiq/middleware/chain.rb +87 -41
  31. data/lib/sidekiq/middleware/current_attributes.rb +63 -0
  32. data/lib/sidekiq/middleware/i18n.rb +6 -4
  33. data/lib/sidekiq/middleware/modules.rb +21 -0
  34. data/lib/sidekiq/monitor.rb +1 -1
  35. data/lib/sidekiq/paginator.rb +8 -8
  36. data/lib/sidekiq/processor.rb +47 -41
  37. data/lib/sidekiq/rails.rb +22 -4
  38. data/lib/sidekiq/redis_client_adapter.rb +154 -0
  39. data/lib/sidekiq/redis_connection.rb +84 -55
  40. data/lib/sidekiq/ring_buffer.rb +29 -0
  41. data/lib/sidekiq/scheduled.rb +55 -25
  42. data/lib/sidekiq/testing/inline.rb +4 -4
  43. data/lib/sidekiq/testing.rb +38 -39
  44. data/lib/sidekiq/transaction_aware_client.rb +45 -0
  45. data/lib/sidekiq/version.rb +1 -1
  46. data/lib/sidekiq/web/action.rb +3 -3
  47. data/lib/sidekiq/web/application.rb +37 -13
  48. data/lib/sidekiq/web/csrf_protection.rb +30 -8
  49. data/lib/sidekiq/web/helpers.rb +60 -28
  50. data/lib/sidekiq/web/router.rb +4 -1
  51. data/lib/sidekiq/web.rb +38 -78
  52. data/lib/sidekiq/worker.rb +136 -13
  53. data/lib/sidekiq.rb +114 -31
  54. data/sidekiq.gemspec +12 -4
  55. data/web/assets/images/apple-touch-icon.png +0 -0
  56. data/web/assets/javascripts/application.js +113 -60
  57. data/web/assets/javascripts/chart.min.js +13 -0
  58. data/web/assets/javascripts/chartjs-plugin-annotation.min.js +7 -0
  59. data/web/assets/javascripts/dashboard.js +50 -67
  60. data/web/assets/javascripts/graph.js +16 -0
  61. data/web/assets/javascripts/metrics.js +262 -0
  62. data/web/assets/stylesheets/application-dark.css +36 -36
  63. data/web/assets/stylesheets/application-rtl.css +0 -4
  64. data/web/assets/stylesheets/application.css +82 -237
  65. data/web/locales/ar.yml +8 -2
  66. data/web/locales/el.yml +43 -19
  67. data/web/locales/en.yml +11 -1
  68. data/web/locales/es.yml +18 -2
  69. data/web/locales/fr.yml +8 -1
  70. data/web/locales/ja.yml +3 -0
  71. data/web/locales/lt.yml +1 -1
  72. data/web/locales/pt-br.yml +27 -9
  73. data/web/views/_footer.erb +1 -1
  74. data/web/views/_job_info.erb +1 -1
  75. data/web/views/_nav.erb +1 -1
  76. data/web/views/_poll_link.erb +2 -5
  77. data/web/views/_summary.erb +7 -7
  78. data/web/views/busy.erb +50 -19
  79. data/web/views/dashboard.erb +23 -14
  80. data/web/views/dead.erb +1 -1
  81. data/web/views/layout.erb +2 -1
  82. data/web/views/metrics.erb +69 -0
  83. data/web/views/metrics_for_job.erb +87 -0
  84. data/web/views/morgue.erb +6 -6
  85. data/web/views/queue.erb +15 -11
  86. data/web/views/queues.erb +3 -3
  87. data/web/views/retries.erb +7 -7
  88. data/web/views/retry.erb +1 -1
  89. data/web/views/scheduled.erb +1 -1
  90. metadata +43 -36
  91. data/.github/ISSUE_TEMPLATE/bug_report.md +0 -20
  92. data/.github/contributing.md +0 -32
  93. data/.github/workflows/ci.yml +0 -41
  94. data/.gitignore +0 -13
  95. data/.standard.yml +0 -20
  96. data/3.0-Upgrade.md +0 -70
  97. data/4.0-Upgrade.md +0 -53
  98. data/5.0-Upgrade.md +0 -56
  99. data/6.0-Upgrade.md +0 -72
  100. data/COMM-LICENSE +0 -97
  101. data/Ent-2.0-Upgrade.md +0 -37
  102. data/Ent-Changes.md +0 -281
  103. data/Gemfile +0 -24
  104. data/Gemfile.lock +0 -192
  105. data/Pro-2.0-Upgrade.md +0 -138
  106. data/Pro-3.0-Upgrade.md +0 -44
  107. data/Pro-4.0-Upgrade.md +0 -35
  108. data/Pro-5.0-Upgrade.md +0 -25
  109. data/Pro-Changes.md +0 -805
  110. data/Rakefile +0 -10
  111. data/code_of_conduct.md +0 -50
  112. data/lib/generators/sidekiq/worker_generator.rb +0 -57
  113. data/lib/sidekiq/exception_handler.rb +0 -27
data/Pro-Changes.md DELETED
@@ -1,805 +0,0 @@
1
- # Sidekiq Pro Changelog
2
-
3
- [Sidekiq Changes](https://github.com/mperham/sidekiq/blob/master/Changes.md) | [Sidekiq Pro Changes](https://github.com/mperham/sidekiq/blob/master/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/mperham/sidekiq/blob/master/Ent-Changes.md)
4
-
5
- Please see [sidekiq.org](https://sidekiq.org/) for more details and how to buy.
6
-
7
- 5.2.0
8
- ---------
9
-
10
- - The Sidekiq Pro and Enterprise gem servers now `bundle install` much faster with **Bundler 2.2+** [#4158]
11
- - Fix issue with reliable push and multiple shards [#4669]
12
- - Fix Pro memory leak due to fetch refactoring in Sidekiq 6.1 [#4652]
13
- - Gracefully handle poison pill jobs [#4633]
14
- - Remove support for multi-shard batches [#4642]
15
- - Rename `Sidekiq::Rack::BatchStatus` to `Sidekiq::Pro::BatchStatus` [#4655]
16
-
17
- 5.1.1
18
- ---------
19
-
20
- - Fix broken basic fetcher [#4616]
21
-
22
- 5.1.0
23
- ---------
24
-
25
- - Remove old Statsd metrics with `WorkerName` in the name [#4377]
26
- ```
27
- job.WorkerName.count -> job.count with tag worker:WorkerName
28
- job.WorkerName.perform -> job.perform with tag worker:WorkerName
29
- job.WorkerName.failure -> job.failure with tag worker:WorkerName
30
- ```
31
- - Remove `concurrent-ruby` gem dependency [#4586]
32
- - Update `constantize` for batch callbacks. [#4469]
33
- - Add queue tag to `jobs.recovered.fetch` metric [#4594]
34
- - Refactor Pro's fetch infrastructure [#4602]
35
-
36
- 5.0.1
37
- ---------
38
-
39
- - Rejigger batch failures UI to add direct links to retries and scheduled jobs [#4209]
40
- - Delete batch data with `UNLINK` [#4155]
41
- - Fix bug where a scheduled job can lose its scheduled time when using reliable push [#4267]
42
- - Sidekiq::JobSet#scan and #find_job APIs have been promoted to Sidekiq OSS. [#4259]
43
-
44
- 5.0.0
45
- ---------
46
-
47
- - There is no significant migration from Sidekiq Pro 4.0 to 5.0
48
- but make sure you read the [update notes for Sidekiq
49
- 6.0](https://github.com/mperham/sidekiq/blob/master/6.0-Upgrade.md).
50
- - Removed various deprecated APIs and associated warnings.
51
- - **BREAKING CHANGE** Remove the `Sidekiq::Batch::Status#dead_jobs` API in favor of
52
- `Sidekiq::Batch::Status#dead_jids`. [#4217]
53
- - Update Sidekiq Pro codebase to use StandardRB formatting
54
- - Fix lingering "b-XXX-died" elements in Redis which could cause
55
- excessive memory usage. [#4217]
56
- - Add ES translations, see issues [#3949](https://github.com/mperham/sidekiq/issues/3949) and [#3951](https://github.com/mperham/sidekiq/issues/3951) to add your own language.
57
-
58
- 4.0.5
59
- ---------
60
-
61
- - Increase super\_fetch retriever thread count from 1 to 2 to make it
62
- less sensitive to Redis latency.
63
- - Better handling of invalid job JSON by reliable scheduler [#4053]
64
- - Added ZH, PT, JA and RU translations.
65
-
66
- 4.0.4
67
- ---------
68
-
69
- - Update Sidekiq::Client patches to work with new Module#prepend
70
- mechanism in Sidekiq 5.2.0. [#3930]
71
-
72
- 4.0.3
73
- ---------
74
-
75
- - Add at\_exit handler to push any saved jobs in `reliable_push` when exiting. [#3823]
76
- - Implement batch death callback. This is fired the first time a job within a batch dies. [#3841]
77
- ```ruby
78
- batch = Sidekiq::Batch.new
79
- batch.on(:death, ...)
80
- ```
81
-
82
- 4.0.2
83
- ---------
84
-
85
- - Remove super\_fetch edge case leading to an unnecessary `sleep(1)`
86
- call and resulting latency [#3790]
87
- - Fix possible bad statsd metric call on super\_fetch startup
88
- - Remove superfluous `freeze` calls on Strings [#3759]
89
-
90
- 4.0.1
91
- ---------
92
-
93
- - Fix incompatibility with the statsd-ruby gem [#3740]
94
- - Add tags to Statsd metrics when using Datadog [#3744]
95
-
96
- 4.0.0
97
- ---------
98
-
99
- - See the [Sidekiq Pro 4.0](Pro-4.0-Upgrade.md) release notes.
100
-
101
-
102
- 3.7.1
103
- ---------
104
-
105
- - Deprecate timed\_fetch. Switch to super\_fetch:
106
- ```ruby
107
- config.super_fetch!
108
- ```
109
-
110
-
111
- 3.7.0
112
- ---------
113
-
114
- - Refactor batch job success/failure to gracefully handle several edge
115
- cases with regard to Sidekiq::Shutdown. This should greatly reduce
116
- the chances of seeing the long-standing "negative pending count" problem. [#3710]
117
-
118
-
119
- 3.6.1
120
- ---------
121
-
122
- - Add support for Datadog::Statsd, it is the recommended Statsd client. [#3699]
123
- ```ruby
124
- Sidekiq::Pro.dogstatsd = ->{ Datadog::Statsd.new("metrics.example.com", 8125) }
125
- ```
126
- - Size the statsd connection pool based on Sidekiq's concurrency [#3700]
127
-
128
-
129
- 3.6.0
130
- ---------
131
-
132
- This release overhauls the Statsd metrics support and adds more
133
- metrics for tracking Pro feature usage. In your initializer:
134
- ```ruby
135
- Sidekiq::Pro.statsd = ->{ ::Statsd.new("127.0.0.1", 8125) }
136
- ```
137
- Sidekiq Pro will emit more metrics to Statsd:
138
- ```
139
- jobs.expired - when a job is expired
140
- jobs.recovered.push - when a job is recovered by reliable_push after network outage
141
- jobs.recovered.fetch - when a job is recovered by super_fetch after process crash
142
- batch.created - when a batch is created
143
- batch.complete - when a batch is completed
144
- batch.success - when a batch is successful
145
- ```
146
- Sidekiq Pro's existing Statsd middleware has been rewritten to leverage the new API.
147
- Everything should be backwards compatible with one deprecation notice.
148
-
149
-
150
- 3.5.4
151
- ---------
152
-
153
- - Fix case in SuperFetch where Redis downtime can lead to processor thread death [#3684]
154
- - Fix case where TimedFetch might not recover some pending jobs
155
- - Fix edge case in Batch::Status#poll leading to premature completion [#3640]
156
- - Adjust scan API to check 100 elements at a time, to minimize network round trips
157
- when scanning large sets.
158
-
159
- 3.5.3
160
- ---------
161
-
162
- - Restore error check for super\_fetch's job ack [#3601]
163
- - Trim error messages saved in Batch's failure hash, preventing huge
164
- messages from bloating Redis. [#3570]
165
-
166
- 3.5.2
167
- ---------
168
-
169
- - Fix `Status#completed?` when run against a Batch that had succeeded
170
- and was deleted. [#3519]
171
-
172
- 3.5.1
173
- ---------
174
-
175
- - Work with Sidekiq 5.0.2+
176
- - Improve performance of super\_fetch with weighted queues [#3489]
177
-
178
- 3.5.0
179
- ---------
180
-
181
- - Add queue pause/unpause endpoints for scripting via curl [#3445]
182
- - Change how super\_fetch names private queues to avoid hostname/queue clashes. [#3443]
183
- - Re-implement `Sidekiq::Queue#delete_job` to avoid O(n) runtime [#3408]
184
- - Batch page displays Pending JIDs if less than 10 [#3130]
185
- - Batch page has a Search button to find associated Retries [#3130]
186
- - Make Batch UI progress bar more friendly to the colorblind [#3387]
187
-
188
- 3.4.5
189
- ---------
190
-
191
- - Fix potential job loss with reliable scheduler when lots of jobs are scheduled
192
- at precisely the same time. Thanks to raivil for his hard work in
193
- reproducing the bug. [#3371]
194
-
195
- 3.4.4
196
- ---------
197
-
198
- - Optimize super\_fetch shutdown to restart jobs quicker [#3249]
199
-
200
- 3.4.3
201
- ---------
202
-
203
- - Limit reliable scheduler to enqueue up to 100 jobs per call, minimizing Redis latency [#3332]
204
- - Fix bug in super\_fetch logic for queues with `_` in the name [#3339]
205
-
206
- 3.4.2
207
- ---------
208
-
209
- - Add `Batch::Status#invalidated?` API which returns true if any/all
210
- JIDs were invalidated within the batch. [#3326]
211
-
212
- 3.4.1
213
- ---------
214
-
215
- - Allow super\_fetch's orphan job check to happen as often as every hour [#3273]
216
- - Officially deprecate reliable\_fetch algorithm, I now recommend you use `super_fetch` instead:
217
- ```ruby
218
- Sidekiq.configure_server do |config|
219
- config.super_fetch!
220
- end
221
- ```
222
- Also note that Sidekiq's `-i/--index` option is no longer used/relevant with super\_fetch.
223
- - Don't display "Delete/Retry All" buttons when filtering in Web UI [#3243]
224
- - Reimplement Sidekiq::JobSet#find\_job with ZSCAN [#3197]
225
-
226
- 3.4.0
227
- ---------
228
-
229
- - Introducing the newest reliable fetching algorithm: `super_fetch`! This
230
- algorithm will replace reliable\_fetch in Pro 4.0. super\_fetch is
231
- bullet-proof across all environments, no longer requiring stable
232
- hostnames or an index to be set per-process. [#3077]
233
- ```ruby
234
- Sidekiq.configure_server do |config|
235
- config.super_fetch!
236
- end
237
- ```
238
- Thank you to @jonhyman for code review and the Sidekiq Pro customers that
239
- beta tested super\_fetch.
240
-
241
- 3.3.3
242
- ---------
243
-
244
- - Update Web UI extension to work with Sidekiq 4.2.0's new Web UI. [#3075]
245
-
246
- 3.3.2
247
- ---------
248
-
249
- - Minimize batch memory usage after success [#3083]
250
- - Extract batch's 24 hr linger expiry to a LINGER constant so it can be tuned. [#3011]
251
-
252
- 3.3.1
253
- ---------
254
-
255
- - If environment is unset, treat it as development so reliable\_fetch works as before 3.2.2.
256
-
257
- 3.3.0
258
- ---------
259
-
260
- - Don't delete batches immediately upon success but set a 24 hr expiry, this allows
261
- Sidekiq::Batch::Status#poll to work, even after batch success. [#3011]
262
- - New `Sidekiq::PendingSet#destroy(jid)` API to remove poison pill jobs [#3015]
263
-
264
- 3.2.2
265
- ---------
266
-
267
- - A default value for -i is only set in development now, staging or
268
- other environments must set an index if you wish to use reliable\_fetch. [#2971]
269
- - Fix nil dereference when checking for jobs over timeout in timed\_fetch
270
-
271
-
272
- 3.2.1
273
- ---------
274
-
275
- - timed\_fetch now works with namespaces. [ryansch]
276
-
277
-
278
- 3.2.0
279
- ---------
280
-
281
- - Fixed detection of missing batches, `NoSuchBatch` should be raised
282
- properly now if `Sidekiq::Batch.new(bid)` is called on a batch no
283
- longer in Redis.
284
- - Remove support for Pro 1.x format batches. This version will no
285
- longer seamlessly process batches created with Sidekiq Pro 1.x.
286
- As always, upgrade one major version at a time to ensure a smooth
287
- transition.
288
- - Fix edge case where a parent batch could expire before a child batch
289
- was finished processing, leading to missing batches [#2889]
290
-
291
- 2.1.5
292
- ---------
293
-
294
- - Fix edge case where a parent batch could expire before a child batch
295
- was finished processing, leading to missing batches [#2889]
296
-
297
- 3.1.0
298
- ---------
299
-
300
- - New container-friendly fetch algorithm: `timed_fetch`. See the
301
- [wiki documentation](https://github.com/mperham/sidekiq/wiki/Pro-Reliability-Server)
302
- for trade offs between the two reliability options. You should
303
- use this if you are on Heroku, Docker, Amazon ECS or EBS or
304
- another container-based system.
305
-
306
-
307
- 3.0.6
308
- ---------
309
-
310
- - Fix race condition on reliable fetch shutdown
311
-
312
- 3.0.5
313
- ---------
314
-
315
- - Statsd metrics now account for ActiveJob class names
316
- - Allow reliable fetch internals to be overridden [jonhyman]
317
-
318
- 3.0.4
319
- ---------
320
-
321
- - Queue pausing no longer requires reliable fetch. [#2786]
322
-
323
- 3.0.3, 2.1.4
324
- ------------
325
-
326
- - Convert Lua-based `Sidekiq::Queue#delete_by_class` to Ruby-based, to
327
- avoid O(N^2) performance and possible Redis failure. [#2806]
328
-
329
- 3.0.2
330
- -----------
331
-
332
- - Make job registration with batch part of the atomic push so batch
333
- metadata can't get out of sync with the job data. [#2714]
334
-
335
- 3.0.1
336
- -----------
337
-
338
- - Remove a number of Redis version checks since we can assume 2.8+ now.
339
- - Fix expiring jobs client middleware not loaded on server
340
-
341
- 3.0.0
342
- -----------
343
-
344
- - See the [Pro 3.0 release notes](Pro-3.0-Upgrade.md).
345
-
346
- 2.1.3
347
- -----------
348
-
349
- - Don't enable strict priority if using weighted queueing like `-q a,1 -q b,1`
350
- - Safer JSON mangling in Lua [#2639]
351
-
352
- 2.1.2
353
- -----------
354
-
355
- - Lock Sidekiq Pro 2.x to Sidekiq 3.x.
356
-
357
- 2.1.1
358
- -----------
359
-
360
- - Make ShardSet lazier so Redis can first be initialized at startup. [#2603]
361
-
362
-
363
- 2.1.0
364
- -----------
365
-
366
- - Explicit support for sharding batches. You list your Redis shards and
367
- Sidekiq Pro will randomly spread batches across the shards. The BID
368
- will indicate which shard contains the batch data. Jobs within a
369
- batch may be spread across all shards too. [#2548, jonhyman]
370
- - Officially deprecate Sidekiq::Notifications code. Notifications have
371
- been undocumented for months now. [#2575]
372
-
373
-
374
- 2.0.8
375
- -----------
376
-
377
- - Fix reliable scheduler mangling large numeric arguments. Lua's CJSON
378
- library cannot accurately encode numbers larger than 14 digits! [#2478]
379
-
380
- 2.0.7
381
- -----------
382
-
383
- - Optimize delete of enormous batches (100,000s of jobs) [#2458]
384
-
385
- 2.0.6, 1.9.3
386
- --------------
387
-
388
- - CSRF protection in Sidekiq 3.4.2 broke job filtering in the Web UI [#2442]
389
- - Sidekiq Pro 1.x is now limited to Sidekiq < 3.5.0.
390
-
391
- 2.0.5
392
- -----------
393
-
394
- - Atomic scheduler now sets `enqueued_at` [#2414]
395
- - Batches now account for jobs which are stopped by client middleware [#2406]
396
- - Ignore redundant calls to `Sidekiq::Client.reliable_push!` [#2408]
397
-
398
- 2.0.4
399
- -----------
400
-
401
- - Reliable push now supports sharding [#2409]
402
- - Reliable push now only catches Redis exceptions [#2307]
403
-
404
- 2.0.3
405
- -----------
406
-
407
- - Display Batch callback data on the Batch details page. [#2347]
408
- - Fix incompatibility with Pro Web and Rack middleware. [#2344] Thank
409
- you to Jason Clark for the tip on how to fix it.
410
-
411
- 2.0.2
412
- -----------
413
-
414
- - Multiple Web UIs can now run in the same process. [#2267] If you have
415
- multiple Redis shards, you can mount UIs for all in the same process:
416
- ```ruby
417
- POOL1 = ConnectionPool.new { Redis.new(:url => "redis://localhost:6379/0") }
418
- POOL2 = ConnectionPool.new { Redis.new(:url => "redis://localhost:6378/0") }
419
-
420
- mount Sidekiq::Pro::Web => '/sidekiq' # default
421
- mount Sidekiq::Pro::Web.with(redis_pool: POOL1), at: '/sidekiq1', as: 'sidekiq1' # shard1
422
- mount Sidekiq::Pro::Web.with(redis_pool: POOL2), at: '/sidekiq2', as: 'sidekiq2' # shard2
423
- ```
424
- - **SECURITY** Fix batch XSS in error data. Thanks to moneybird.com for
425
- reporting the issue.
426
-
427
- 2.0.1
428
- -----------
429
-
430
- - Add `batch.callback_queue` so batch callbacks can use a higher
431
- priority queue than jobs. [#2200]
432
- - Gracefully recover if someone runs `SCRIPT FLUSH` on Redis. [#2240]
433
- - Ignore errors when attempting `bulk_requeue`, allowing clean shutdown
434
-
435
- 2.0.0
436
- -----------
437
-
438
- - See [the Upgrade Notes](Pro-2.0-Upgrade.md) for detailed notes.
439
-
440
- 1.9.2
441
- -----------
442
-
443
- - As of 1/1/2015, Sidekiq Pro is hosted on a new dedicated server.
444
- Happy new year and let's hope for 100% uptime!
445
- - Fix bug in reliable\_fetch where jobs could be duplicated if a Sidekiq
446
- process crashed and you were using weighted queues. [#2120]
447
-
448
- 1.9.1
449
- -----------
450
-
451
- - **SECURITY** Fix XSS in batch description, thanks to intercom.io for reporting the
452
- issue. If you don't use batch descriptions, you don't need the fix.
453
-
454
- 1.9.0
455
- -----------
456
-
457
- - Add new expiring jobs feature [#1982]
458
- - Show batch expiration on Batch details page [#1981]
459
- - Add '$' batch success token to the pubsub support. [#1953]
460
-
461
-
462
- 1.8.0
463
- -----------
464
-
465
- - Fix race condition where Batches can complete
466
- before they have been fully defined or only half-defined. Requires
467
- Sidekiq 3.2.3. [#1919]
468
-
469
-
470
- 1.7.6
471
- -----------
472
-
473
- - Quick release to verify #1919
474
-
475
-
476
- 1.7.5
477
- -----------
478
-
479
- - Fix job filtering within the Dead tab.
480
- - Add APIs and wiki documentation for invalidating jobs within a batch.
481
-
482
-
483
- 1.7.4
484
- -----------
485
-
486
- - Awesome ANSI art startup banner!
487
-
488
-
489
- 1.7.3
490
- -----------
491
-
492
- - Batch callbacks should use the same queue as the associated jobs.
493
-
494
- 1.7.2
495
- -----------
496
-
497
- - **DEPRECATION** Use `Batch#on(:complete)` instead of `Batch#notify`.
498
- The specific Campfire, HipChat, email and other notification schemes
499
- will be removed in 2.0.0.
500
- - Remove batch from UI when successful. [#1745]
501
- - Convert batch callbacks to be asynchronous jobs for error handling [#1744]
502
-
503
- 1.7.1
504
- -----------
505
-
506
- - Fix for paused queues being processed for a few seconds when starting
507
- a new Sidekiq process.
508
- - Add a 5 sec delay when starting reliable fetch on Heroku to minimize
509
- any duplicate job processing with another process shutting down.
510
-
511
- 1.7.0
512
- -----------
513
-
514
- - Add ability to pause reliable queues via API.
515
- ```ruby
516
- q = Sidekiq::Queue.new("critical")
517
- q.pause!
518
- q.paused? # => true
519
- q.unpause!
520
- ```
521
-
522
- Sidekiq polls Redis every 10 seconds for paused queues so pausing will take
523
- a few seconds to take effect.
524
-
525
- 1.6.0
526
- -----------
527
-
528
- - Compatible with Sidekiq 3.
529
-
530
- 1.5.1
531
- -----------
532
-
533
- - Due to a breaking API change in Sidekiq 3.0, this version is limited
534
- to Sidekiq 2.x.
535
-
536
- 1.5.0
537
- -----------
538
-
539
- - Fix issue on Heroku where reliable fetch could orphan jobs [#1573]
540
-
541
-
542
- 1.4.3
543
- -----------
544
-
545
- - Reverse sorting of Batches in Web UI [#1098]
546
- - Refactoring for Sidekiq 3.0, Pro now requires Sidekiq 2.17.5
547
-
548
- 1.4.2
549
- -----------
550
-
551
- - Tolerate expired Batches in the web UI.
552
- - Fix 100% CPU usage when using weighted queues and reliable fetch.
553
-
554
- 1.4.1
555
- -----------
556
-
557
- - Add batch progress bar to batch detail page. [#1398]
558
- - Fix race condition in initializing Lua scripts
559
-
560
-
561
- 1.4.0
562
- -----------
563
-
564
- - Default batch expiration has been extended to 3 days, from 1 day previously.
565
- - Batches now sort in the Web UI according to expiry time, not creation time.
566
- - Add user-configurable batch expiry. If your batches might take longer
567
- than 72 hours to process, you can extend the expiration date.
568
-
569
- ```ruby
570
- b = Sidekiq::Batch.new
571
- b.expires_in 5.days
572
- ...
573
- ```
574
-
575
- 1.3.2
576
- -----------
577
-
578
- - Lazy load Lua scripts so a Redis connection is not required on bootup.
579
-
580
- 1.3.1
581
- -----------
582
-
583
- - Fix a gemspec packaging issue which broke the Batch UI.
584
-
585
- 1.3.0
586
- -----------
587
-
588
- Thanks to @jonhyman for his contributions to this Sidekiq Pro release.
589
-
590
- This release includes new functionality based on the SCAN command newly
591
- added to Redis 2.8. Pro still works with Redis 2.4 but some
592
- functionality will be unavailable.
593
-
594
- - Job Filtering in the Web UI!
595
- You can now filter retries and scheduled jobs in the Web UI so you
596
- only see the jobs relevant to your needs. Queues cannot be filtered;
597
- Redis does not provide the same SCAN operation on the LIST type.
598
- **Redis 2.8**
599
- ![Filtering](https://f.cloud.github.com/assets/2911/1619465/f47529f2-5657-11e3-8cd1-33899eb72aad.png)
600
- - SCAN support in the Sidekiq::SortedSet API. Here's an example that
601
- finds all jobs which contain the substring "Warehouse::OrderShip"
602
- and deletes all matching retries. If the set is large, this API
603
- will be **MUCH** faster than standard iteration using each.
604
- **Redis 2.8**
605
- ```ruby
606
- Sidekiq::RetrySet.new.scan("Warehouse::OrderShip") do |job|
607
- job.delete
608
- end
609
- ```
610
-
611
- - Sidekiq::Batch#jobs now returns the set of JIDs added to the batch.
612
- - Sidekiq::Batch#jids returns the complete set of JIDs associated with the batch.
613
- - Sidekiq::Batch#remove\_jobs(jid, jid, ...) removes JIDs from the set, allowing early termination of jobs if they become irrelevant according to application logic.
614
- - Sidekiq::Batch#include?(jid) allows jobs to check if they are still
615
- relevant to a Batch and exit early if not.
616
- - Sidekiq::SortedSet#find\_job(jid) now uses server-side Lua if possible **Redis 2.6** [jonhyman]
617
- - The statsd integration now sets global job counts:
618
- ```ruby
619
- jobs.count
620
- jobs.success
621
- jobs.failure
622
- ```
623
-
624
- - Change shutdown logic to push leftover jobs in the private queue back
625
- into the public queue when shutting down with Reliable Fetch. This
626
- allows the safe decommission of a Sidekiq Pro process when autoscaling. [jonhyman]
627
- - Add support for weighted random fetching with Reliable Fetch [jonhyman]
628
- - Pro now requires Sidekiq 2.17.0
629
-
630
- 1.2.5
631
- -----------
632
-
633
- - Convert Batch UI to use Sidekiq 2.16's support for extension localization.
634
- - Update reliable\_push to work with Sidekiq::Client refactoring in 2.16
635
- - Pro now requires Sidekiq 2.16.0
636
-
637
- 1.2.4
638
- -----------
639
-
640
- - Convert Batch UI to Bootstrap 3
641
- - Pro now requires Sidekiq 2.15.0
642
- - Add Sidekiq::Batch::Status#delete [#1205]
643
-
644
- 1.2.3
645
- -----------
646
-
647
- - Pro now requires Sidekiq 2.14.0
648
- - Fix bad exception handling in batch callbacks [#1134]
649
- - Convert Batch UI to ERB
650
-
651
- 1.2.2
652
- -----------
653
-
654
- - Problem with reliable fetch which could lead to lost jobs when Sidekiq
655
- is shut down normally. Thanks to MikaelAmborn for the report. [#1109]
656
-
657
- 1.2.1
658
- -----------
659
-
660
- - Forgot to push paging code necessary for `delete_job` performance.
661
-
662
- 1.2.0
663
- -----------
664
-
665
- - **LEAK** Fix batch key which didn't expire in Redis. Keys match
666
- /b-[a-f0-9]{16}-pending/, e.g. "b-4f55163ddba10aa0-pending" [#1057]
667
- - **Reliable fetch now supports multiple queues**, using the algorithm spec'd
668
- by @jackrg [#1102]
669
- - Fix issue with reliable\_push where it didn't return the JID for a pushed
670
- job when sending previously cached jobs to Redis.
671
- - Add fast Sidekiq::Queue#delete\_job(jid) API which leverages Lua so job lookup is
672
- 100% server-side. Benchmark vs Sidekiq's Job#delete API. **Redis 2.6**
673
-
674
- ```
675
- Sidekiq Pro API
676
- 0.030000 0.020000 0.050000 ( 1.640659)
677
- Sidekiq API
678
- 17.250000 2.220000 19.470000 ( 22.193300)
679
- ```
680
-
681
- - Add fast Sidekiq::Queue#delete\_by\_class(klass) API to remove all
682
- jobs of a given type. Uses server-side Lua for performance. **Redis 2.6**
683
-
684
- 1.1.0
685
- -----------
686
-
687
- - New `sidekiq/pro/reliable_push` which makes Sidekiq::Client resiliant
688
- to Redis network failures. [#793]
689
- - Move `sidekiq/reliable_fetch` to `sidekiq/pro/reliable_fetch`
690
-
691
-
692
- 1.0.0
693
- -----------
694
-
695
- - Sidekiq Pro changelog moved to mperham/sidekiq for public visibility.
696
- - Add new Rack endpoint for easy polling of batch status via JavaScript. See `sidekiq/rack/batch_status`
697
-
698
- 0.9.3
699
- -----------
700
-
701
- - Fix bad /batches path in Web UI
702
- - Fix Sinatra conflict with sidekiq-failures
703
-
704
- 0.9.2
705
- -----------
706
-
707
- - Fix issue with lifecycle notifications not firing.
708
-
709
- 0.9.1
710
- -----------
711
-
712
- - Update due to Sidekiq API changes.
713
-
714
- 0.9.0
715
- -----------
716
-
717
- - Rearchitect Sidekiq's Fetch code to support different fetch
718
- strategies. Add a ReliableFetch strategy which works with Redis'
719
- RPOPLPUSH to ensure we don't lose messages, even when the Sidekiq
720
- process crashes unexpectedly. [mperham/sidekiq#607]
721
-
722
- 0.8.2
723
- -----------
724
-
725
- - Reimplement existing notifications using batch on_complete events.
726
-
727
- 0.8.1
728
- -----------
729
-
730
- - Rejigger batch callback notifications.
731
-
732
-
733
- 0.8.0
734
- -----------
735
-
736
- - Add new Batch 'callback' notification support, for in-process
737
- notification.
738
- - Symbolize option keys passed to Pony [mperham/sidekiq#603]
739
- - Batch no longer requires the Web UI since Web UI usage is optional.
740
- You must require is manually in your Web process:
741
-
742
- ```ruby
743
- require 'sidekiq/web'
744
- require 'sidekiq/batch/web'
745
- mount Sidekiq::Web => '/sidekiq'
746
- ```
747
-
748
-
749
- 0.7.1
750
- -----------
751
-
752
- - Worker instances can access the associated jid and bid via simple
753
- accessors.
754
- - Batches can now be modified while being processed so, e.g. a batch
755
- job can add additional jobs to its own batch.
756
-
757
- ```ruby
758
- def perform(...)
759
- batch = Sidekiq::Batch.new(bid) # instantiate batch associated with this job
760
- batch.jobs do
761
- SomeWorker.perform_async # add another job
762
- end
763
- end
764
- ```
765
-
766
- - Save error backtraces in batch's failure info for display in Web UI.
767
- - Clean up email notification a bit.
768
-
769
-
770
- 0.7.0
771
- -----------
772
-
773
- - Add optional batch description
774
- - Mutable batches. Batches can now be modified to add additional jobs
775
- at runtime. Example would be a batch job which needs to create more
776
- jobs based on the data it is processing.
777
-
778
- ```ruby
779
- batch = Sidekiq::Batch.new(bid)
780
- batch.jobs do
781
- # define more jobs here
782
- end
783
- ```
784
- - Fix issues with symbols vs strings in option hashes
785
-
786
-
787
- 0.6.1
788
- -----------
789
-
790
- - Webhook notification support
791
-
792
-
793
- 0.6
794
- -----------
795
-
796
- - Redis pubsub
797
- - Email polish
798
-
799
-
800
- 0.5
801
- -----------
802
-
803
- - Batches
804
- - Notifications
805
- - Statsd middleware