sidekiq 4.2.4 → 5.2.0

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of sidekiq might be problematic. Click here for more details.

Files changed (106) hide show
  1. checksums.yaml +4 -4
  2. data/.github/issue_template.md +8 -1
  3. data/.gitignore +1 -0
  4. data/.travis.yml +5 -3
  5. data/5.0-Upgrade.md +56 -0
  6. data/COMM-LICENSE +1 -1
  7. data/Changes.md +151 -0
  8. data/Ent-Changes.md +77 -2
  9. data/Gemfile +10 -25
  10. data/LICENSE +1 -1
  11. data/Pro-4.0-Upgrade.md +35 -0
  12. data/Pro-Changes.md +156 -2
  13. data/README.md +9 -6
  14. data/Rakefile +1 -2
  15. data/bin/sidekiqctl +1 -1
  16. data/bin/sidekiqload +15 -33
  17. data/lib/generators/sidekiq/templates/worker_spec.rb.erb +1 -1
  18. data/lib/generators/sidekiq/templates/worker_test.rb.erb +1 -1
  19. data/lib/sidekiq/api.rb +157 -67
  20. data/lib/sidekiq/cli.rb +71 -26
  21. data/lib/sidekiq/client.rb +25 -18
  22. data/lib/sidekiq/core_ext.rb +1 -106
  23. data/lib/sidekiq/delay.rb +42 -0
  24. data/lib/sidekiq/exception_handler.rb +2 -4
  25. data/lib/sidekiq/extensions/generic_proxy.rb +7 -1
  26. data/lib/sidekiq/fetch.rb +1 -1
  27. data/lib/sidekiq/job_logger.rb +25 -0
  28. data/lib/sidekiq/job_retry.rb +241 -0
  29. data/lib/sidekiq/launcher.rb +45 -37
  30. data/lib/sidekiq/logging.rb +18 -2
  31. data/lib/sidekiq/manager.rb +3 -4
  32. data/lib/sidekiq/middleware/server/active_record.rb +10 -0
  33. data/lib/sidekiq/processor.rb +91 -34
  34. data/lib/sidekiq/rails.rb +15 -51
  35. data/lib/sidekiq/redis_connection.rb +31 -5
  36. data/lib/sidekiq/scheduled.rb +35 -8
  37. data/lib/sidekiq/testing.rb +24 -7
  38. data/lib/sidekiq/util.rb +6 -2
  39. data/lib/sidekiq/version.rb +1 -1
  40. data/lib/sidekiq/web/action.rb +2 -6
  41. data/lib/sidekiq/web/application.rb +28 -21
  42. data/lib/sidekiq/web/helpers.rb +67 -23
  43. data/lib/sidekiq/web/router.rb +14 -10
  44. data/lib/sidekiq/web.rb +4 -4
  45. data/lib/sidekiq/worker.rb +97 -14
  46. data/lib/sidekiq.rb +23 -24
  47. data/sidekiq.gemspec +7 -10
  48. data/web/assets/javascripts/application.js +0 -0
  49. data/web/assets/javascripts/dashboard.js +18 -13
  50. data/web/assets/stylesheets/application-rtl.css +246 -0
  51. data/web/assets/stylesheets/application.css +336 -4
  52. data/web/assets/stylesheets/bootstrap-rtl.min.css +9 -0
  53. data/web/assets/stylesheets/bootstrap.css +2 -2
  54. data/web/locales/ar.yml +80 -0
  55. data/web/locales/en.yml +1 -0
  56. data/web/locales/es.yml +4 -3
  57. data/web/locales/fa.yml +80 -0
  58. data/web/locales/he.yml +79 -0
  59. data/web/locales/ja.yml +5 -3
  60. data/web/locales/ur.yml +80 -0
  61. data/web/views/_footer.erb +5 -2
  62. data/web/views/_job_info.erb +1 -1
  63. data/web/views/_nav.erb +1 -1
  64. data/web/views/_paging.erb +1 -1
  65. data/web/views/busy.erb +9 -5
  66. data/web/views/dashboard.erb +3 -3
  67. data/web/views/layout.erb +11 -2
  68. data/web/views/morgue.erb +14 -10
  69. data/web/views/queue.erb +10 -10
  70. data/web/views/queues.erb +4 -2
  71. data/web/views/retries.erb +13 -11
  72. data/web/views/retry.erb +1 -1
  73. data/web/views/scheduled.erb +2 -2
  74. metadata +26 -160
  75. data/lib/sidekiq/middleware/server/logging.rb +0 -40
  76. data/lib/sidekiq/middleware/server/retry_jobs.rb +0 -205
  77. data/test/config.yml +0 -9
  78. data/test/env_based_config.yml +0 -11
  79. data/test/fake_env.rb +0 -1
  80. data/test/fixtures/en.yml +0 -2
  81. data/test/helper.rb +0 -75
  82. data/test/test_actors.rb +0 -138
  83. data/test/test_api.rb +0 -528
  84. data/test/test_cli.rb +0 -418
  85. data/test/test_client.rb +0 -266
  86. data/test/test_exception_handler.rb +0 -56
  87. data/test/test_extensions.rb +0 -127
  88. data/test/test_fetch.rb +0 -50
  89. data/test/test_launcher.rb +0 -95
  90. data/test/test_logging.rb +0 -35
  91. data/test/test_manager.rb +0 -50
  92. data/test/test_middleware.rb +0 -158
  93. data/test/test_processor.rb +0 -235
  94. data/test/test_rails.rb +0 -22
  95. data/test/test_redis_connection.rb +0 -132
  96. data/test/test_retry.rb +0 -326
  97. data/test/test_retry_exhausted.rb +0 -149
  98. data/test/test_scheduled.rb +0 -115
  99. data/test/test_scheduling.rb +0 -58
  100. data/test/test_sidekiq.rb +0 -107
  101. data/test/test_testing.rb +0 -143
  102. data/test/test_testing_fake.rb +0 -357
  103. data/test/test_testing_inline.rb +0 -94
  104. data/test/test_util.rb +0 -13
  105. data/test/test_web.rb +0 -726
  106. data/test/test_web_helpers.rb +0 -54
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 581035f739e6ecf4fdd07d42b7ff78b1012f3ad2
4
- data.tar.gz: 26c02597dab733a1fdf7d948a7a274029033ae21
3
+ metadata.gz: 7d22391d1c7f5f4a237554a77f1648b42849a472
4
+ data.tar.gz: 4d1c7e75fec7f25a571868172aade268ab95a0b3
5
5
  SHA512:
6
- metadata.gz: ace3433e3bc81e986d56177f392a54f24e9a6f5cd1a050107d5273271cc61df5b022dfae32f99fb0cd74b71d5c9470daebca8073a5106bbe605f71d788196500
7
- data.tar.gz: abca98c2d32c438dfc4f60e8dabcea79cc0f28c616d62942a02f768bb0353199abd139aacdc12db546ee19e66b95fb0a36644104dd47c663f034f02882bffd24
6
+ metadata.gz: e70c87a39e46a8250ffde65b66f550e1849df7353940ed916765eb843a99c01fece8ce720d7eb08aa9ec5d8084363413e433628d852f42116670be423daaa138
7
+ data.tar.gz: bee6d32074c1a94dd5d22d87567a1499a1a655c371b011b2ea0c8bac3943eff3f6b3c82b91507bceb92b4e4e3265a9ef05560eabd95649361e60124b693fcdfa
@@ -1,4 +1,11 @@
1
1
  Ruby version:
2
2
  Sidekiq / Pro / Enterprise version(s):
3
3
 
4
- If relevant, please include your initializer and any error message with the full backtrace.
4
+ Please include your initializer and any error message with the full backtrace.
5
+
6
+ Are you using an old version?
7
+ Have you checked the changelogs to see if your issue has been fixed in a later version?
8
+
9
+ https://github.com/mperham/sidekiq/blob/master/Changes.md
10
+ https://github.com/mperham/sidekiq/blob/master/Pro-Changes.md
11
+ https://github.com/mperham/sidekiq/blob/master/Ent-Changes.md
data/.gitignore CHANGED
@@ -10,3 +10,4 @@ vendor/
10
10
  .bundle/
11
11
  .sass-cache/
12
12
  tmp/
13
+ pkg/*.gem
data/.travis.yml CHANGED
@@ -7,6 +7,8 @@ before_install:
7
7
  - gem install bundler
8
8
  - gem update bundler
9
9
  rvm:
10
- - 2.2.4
11
- - 2.3.0
12
- - jruby-head
10
+ - 2.2.10
11
+ - 2.3.7
12
+ - 2.4.4
13
+ - 2.5.1
14
+ - jruby-9.1.17.0
data/5.0-Upgrade.md ADDED
@@ -0,0 +1,56 @@
1
+ # Welcome to Sidekiq 5.0!
2
+
3
+ Sidekiq 5.0 contains a reworked job dispatch and execution core to integrate
4
+ better with the new Rails 5.0 Executor. It also drops support for older
5
+ versions of Ruby and Rails and adds support for RTL languages in the Web UI.
6
+
7
+ ## What's New
8
+
9
+ * Integrate job logging and retry logic directly in with the job
10
+ execution logic in Sidekiq::Processor. Previously this logic was
11
+ defined as middleware. In Rails 5.0, ActiveSupport::Executor handles ActiveRecord
12
+ connection management, job callbacks, development mode class loading,
13
+ etc. Because of its extensive responsibilities, the Executor can't be
14
+ integrated as Sidekiq middleware; the logging/retry logic had to be pulled out
15
+ too. Sidekiq 4.2 had a hack to make it work but this redesign provides
16
+ a cleaner integration. [#3235]
17
+ * The Delayed Extensions `delay`, `delay_in` and `delay_until` APIs are
18
+ no longer available by default. The extensions allow you to marshal
19
+ job arguments as YAML, leading to cases where job payloads could be many
20
+ 100s of KB or larger if not careful, leading to Redis networking
21
+ timeouts or other problems. As noted in the Best Practices wiki page,
22
+ Sidekiq is designed for jobs with small, simple arguments.
23
+
24
+ Add this line to your initializer to re-enable them and get the old behavior:
25
+ ```ruby
26
+ Sidekiq::Extensions.enable_delay!
27
+ ```
28
+ The old `Sidekiq.remove_delay!` API has been removed as it is now the default. [#3299]
29
+ * Sidekiq's quiet signal is now `TSTP` (think of it as **T**hread
30
+ **ST**o**P**) instead of USR1 as USR1 is not available on JRuby.
31
+ USR1 will continue to be supported in Sidekiq 5.x for backwards
32
+ compatibility and will be removed in Sidekiq 6.x. [#3302]
33
+ * The Web UI is now bi-directional - it can render either LTR
34
+ (left-to-right) or RTL languages. With this change, **Farsi, Arabic,
35
+ Hebrew and Urdu** are officially supported. [#3381]
36
+ * Jobs which can't be parsed due to invalid JSON are now pushed
37
+ immediately to the Dead set since they require manual intervention and
38
+ will never execute successfully as is. The Web UI has been updated to
39
+ more gracefully display these jobs. [#3296]
40
+ * **Rails 3.2** is no longer supported.
41
+ * **Ruby 2.0 and Ruby 2.1** are no longer supported. Ruby 2.2.2+ is required.
42
+
43
+ ## Upgrade
44
+
45
+ As always, please upgrade Sidekiq **one major version at a time**.
46
+ If you are already running Sidekiq 4.x, then:
47
+
48
+ * Upgrade to the latest Sidekiq 4.x.
49
+ ```ruby
50
+ gem 'sidekiq', '< 5'
51
+ ```
52
+ * Fix any deprecation warnings you see.
53
+ * Upgrade to 5.x.
54
+ ```ruby
55
+ gem 'sidekiq', '< 6'
56
+ ```
data/COMM-LICENSE CHANGED
@@ -38,7 +38,7 @@ under the terms of the GNU Lesser General Public License versions 3.0
38
38
 
39
39
  5. Fees and Payment. The Software license fees will be due and payable in full as set forth in the applicable invoice or at the time of purchase. If the Software does not function properly within two weeks of purchase, please contact us within those two weeks for a refund. You shall be responsible for all taxes, withholdings, duties and levies arising from the order (excluding taxes based on the net income of Contributed Systems).
40
40
 
41
- 6. Support, Maintenance and Services. Subject to the terms and conditions of this Agreement, as set forth in your invoice, and as set forth on the Sidekiq Pro support page (https://github.com/mperham/sidekiq/wiki/Pro-Support), support and maintenance services may be included with the purchase of your license subscription.
41
+ 6. Support, Maintenance and Services. Subject to the terms and conditions of this Agreement, as set forth in your invoice, and as set forth on the Sidekiq Pro support page (https://github.com/mperham/sidekiq/wiki/Commercial-Support), support and maintenance services may be included with the purchase of your license subscription.
42
42
 
43
43
  7. Term of Agreement.
44
44
 
data/Changes.md CHANGED
@@ -1,5 +1,155 @@
1
1
  # Sidekiq Changes
2
2
 
3
+ [Sidekiq Changes](https://github.com/mperham/sidekiq/blob/master/Changes.md) | [Sidekiq Pro Changes](https://github.com/mperham/sidekiq/blob/master/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/mperham/sidekiq/blob/master/Ent-Changes.md)
4
+
5
+ HEAD
6
+ -----------
7
+
8
+ - **Decrease default concurrency from 25 to 10** [#3892]
9
+ - Verify connection pool sizing upon startup [#3917]
10
+ - Smoother scheduling for large Sidekiq clusters [#3889]
11
+ - Switch Sidekiq::Testing impl from alias\_method to Module#prepend, for resiliency [#3852]
12
+ - Update Sidekiq APIs to use SCAN for scalability [#3848, ffiller]
13
+ - Remove concurrent-ruby gem dependency [#3830]
14
+ - Optimize Web UI's bootstrap.css [#3914]
15
+
16
+ 5.1.3
17
+ -----------
18
+
19
+ - Fix version comparison so Ruby 2.2.10 works. [#3808, nateberkopec]
20
+
21
+ 5.1.2
22
+ -----------
23
+
24
+ - Add link to docs in Web UI footer
25
+ - Fix crash on Ctrl-C in Windows [#3775, Bernica]
26
+ - Remove `freeze` calls on String constants. This is superfluous with Ruby
27
+ 2.3+ and `frozen_string_literal: true`. [#3759]
28
+ - Fix use of AR middleware outside of Rails [#3787]
29
+ - Sidekiq::Worker `sidekiq_retry_in` block can now return nil or 0 to use
30
+ the default backoff delay [#3796, dsalahutdinov]
31
+
32
+ 5.1.1
33
+ -----------
34
+
35
+ - Fix Web UI incompatibility with Redis 3.x gem [#3749]
36
+
37
+ 5.1.0
38
+ -----------
39
+
40
+ - **NEW** Global death handlers - called when your job exhausts all
41
+ retries and dies. Now you can take action when a job fails permanently. [#3721]
42
+ - **NEW** Enable ActiveRecord query cache within jobs by default [#3718, sobrinho]
43
+ This will prevent duplicate SELECTS; cache is cleared upon any UPDATE/INSERT/DELETE.
44
+ See the issue for how to bypass the cache or disable it completely.
45
+ - Scheduler timing is now more accurate, 15 -> 5 seconds [#3734]
46
+ - Exceptions during the :startup event will now kill the process [#3717]
47
+ - Make `Sidekiq::Client.via` reentrant [#3715]
48
+ - Fix use of Sidekiq logger outside of the server process [#3714]
49
+ - Tweak `constantize` to better match Rails class lookup. [#3701, caffeinated-tech]
50
+
51
+ 5.0.5
52
+ -----------
53
+
54
+ - Update gemspec to allow newer versions of the Redis gem [#3617]
55
+ - Refactor Worker.set so it can be memoized [#3602]
56
+ - Fix display of Redis URL in web footer, broken in 5.0.3 [#3560]
57
+ - Update `Sidekiq::Job#display_args` to avoid mutation [#3621]
58
+
59
+ 5.0.4
60
+ -----------
61
+
62
+ - Fix "slow startup" performance regression from 5.0.2. [#3525]
63
+ - Allow users to disable ID generation since some redis providers disable the CLIENT command. [#3521]
64
+
65
+ 5.0.3
66
+ -----------
67
+
68
+ - Fix overriding `class_attribute` core extension from ActiveSupport with Sidekiq one [PikachuEXE, #3499]
69
+ - Allow job logger to be overridden [AlfonsoUceda, #3502]
70
+ - Set a default Redis client identifier for debugging [#3516]
71
+ - Fix "Uninitialized constant" errors on startup with the delayed extensions [#3509]
72
+
73
+ 5.0.2
74
+ -----------
75
+
76
+ - fix broken release, thanks @nateberkopec
77
+
78
+ 5.0.1
79
+ -----------
80
+
81
+ - Fix incorrect server identity when daemonizing [jwilm, #3496]
82
+ - Work around error running Web UI against Redis Cluster [#3492]
83
+ - Remove core extensions, Sidekiq is now monkeypatch-free! [#3474]
84
+ - Reimplement Web UI's HTTP\_ACCEPT\_LANGUAGE parsing because the spec is utterly
85
+ incomprehensible for various edge cases. [johanlunds, natematykiewicz, #3449]
86
+ - Update `class_attribute` core extension to avoid warnings
87
+ - Expose `job_hash_context` from `Sidekiq::Logging` to support log customization
88
+
89
+ 5.0.0
90
+ -----------
91
+
92
+ - **BREAKING CHANGE** Job dispatch was refactored for safer integration with
93
+ Rails 5. The **Logging** and **RetryJobs** server middleware were removed and
94
+ functionality integrated directly into Sidekiq::Processor. These aren't
95
+ commonly used public APIs so this shouldn't impact most users.
96
+ ```
97
+ Sidekiq::Middleware::Server::RetryJobs -> Sidekiq::JobRetry
98
+ Sidekiq::Middleware::Server::Logging -> Sidekiq::JobLogger
99
+ ```
100
+ - Quieting Sidekiq is now done via the TSTP signal, the USR1 signal is deprecated.
101
+ - The `delay` extension APIs are no longer available by default, you
102
+ must opt into them.
103
+ - The Web UI is now BiDi and can render RTL languages like Arabic, Farsi and Hebrew.
104
+ - Rails 3.2 and Ruby 2.0 and 2.1 are no longer supported.
105
+ - The `SomeWorker.set(options)` API was re-written to avoid thread-local state. [#2152]
106
+ - Sidekiq Enterprise's encrypted jobs now display "[encrypted data]" in the Web UI instead
107
+ of random hex bytes.
108
+ - Please see the [5.0 Upgrade notes](5.0-Upgrade.md) for more detail.
109
+
110
+ 4.2.10
111
+ -----------
112
+
113
+ - Scheduled jobs can now be moved directly to the Dead queue via API [#3390]
114
+ - Fix edge case leading to job duplication when using Sidekiq Pro's
115
+ reliability feature [#3388]
116
+ - Fix error class name display on retry page [#3348]
117
+ - More robust latency calculation [#3340]
118
+
119
+ 4.2.9
120
+ -----------
121
+
122
+ - Rollback [#3303] which broke Heroku Redis users [#3311]
123
+ - Add support for TSTP signal, for Sidekiq 5.0 forward compatibility. [#3302]
124
+
125
+ 4.2.8
126
+ -----------
127
+
128
+ - Fix rare edge case with Redis driver that can create duplicate jobs [#3303]
129
+ - Fix Rails 5 loading issue [#3275]
130
+ - Restore missing tooltips to timestamps in Web UI [#3310]
131
+ - Work on **Sidekiq 5.0** is now active! [#3301]
132
+
133
+ 4.2.7
134
+ -----------
135
+
136
+ - Add new integration testing to verify code loading and job execution
137
+ in development and production modes with Rails 4 and 5 [#3241]
138
+ - Fix delayed extensions in development mode [#3227, DarthSim]
139
+ - Use Worker's `retry` default if job payload does not have a retry
140
+ attribute [#3234, mlarraz]
141
+
142
+ 4.2.6
143
+ -----------
144
+
145
+ - Run Rails Executor when in production [#3221, eugeneius]
146
+
147
+ 4.2.5
148
+ -----------
149
+
150
+ - Re-enable eager loading of all code when running non-development Rails 5. [#3203]
151
+ - Better root URL handling for zany web servers [#3207]
152
+
3
153
  4.2.4
4
154
  -----------
5
155
 
@@ -19,6 +169,7 @@ Sidekiq::Web.set :sessions, false
19
169
  - Fix Web UI sharding support broken in 4.2.2. [#3169]
20
170
  - Fix timestamps not updating during UI polling [#3193, shaneog]
21
171
  - Relax rack-protection version to >= 1.5.0
172
+ - Provide consistent interface to exception handlers, changing the structure of the context hash. [#3161]
22
173
 
23
174
  4.2.2
24
175
  -----------
data/Ent-Changes.md CHANGED
@@ -1,8 +1,83 @@
1
- Sidekiq Enterprise Changelog
2
- =======================
1
+ # Sidekiq Enterprise Changelog
2
+
3
+ [Sidekiq Changes](https://github.com/mperham/sidekiq/blob/master/Changes.md) | [Sidekiq Pro Changes](https://github.com/mperham/sidekiq/blob/master/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/mperham/sidekiq/blob/master/Ent-Changes.md)
3
4
 
4
5
  Please see [http://sidekiq.org/](http://sidekiq.org/) for more details and how to buy.
5
6
 
7
+ HEAD
8
+ -------------
9
+
10
+ - Add support for sidekiqswarm memory monitoring on FreeBSD [#3884]
11
+
12
+ 1.7.1
13
+ -------------
14
+
15
+ - Fix Lua error in concurrent rate limiter under heavy contention
16
+ - Remove superfluous `freeze` calls on Strings [#3759]
17
+
18
+ 1.7.0
19
+ -------------
20
+
21
+ - **NEW FEATURE** [Rolling restarts](https://github.com/mperham/sidekiq/wiki/Ent-Rolling-Restarts) - great for long running jobs!
22
+ - Adjust middleware so unique jobs that don't push aren't registered in a Batch [#3662]
23
+ - Add new unlimited rate limiter, useful for testing [#3743]
24
+ ```ruby
25
+ limiter = Sidekiq::Limiter.unlimited(...any args...)
26
+ ```
27
+
28
+ 1.6.1
29
+ -------------
30
+
31
+ - Fix crash in rate limiter middleware when used with custom exceptions [#3604]
32
+
33
+ 1.6.0
34
+ -------------
35
+
36
+ - Show process "leader" tag on Busy page, requires Sidekiq 5.0.2 [#2867]
37
+ - Capture custom metrics with the `save_history` API. [#2815]
38
+ - Implement new `unique_until: 'start'` policy option. [#3471]
39
+
40
+ 1.5.4
41
+ -------------
42
+
43
+ - Fix broken Cron page in Web UI [#3458]
44
+
45
+ 1.5.3
46
+ -------------
47
+
48
+ - Remove dependency on the algorithms gem [#3446]
49
+ - Allow user to specify max memory in megabytes with SIDEKIQ\_MAXMEM\_MB [#3451]
50
+ - Implement logic to detect app startup failure, sidekiqswarm will exit
51
+ rather than try to restart the app forever [#3450]
52
+ - Another fix for doubly-encrypted arguments [#3368]
53
+
54
+ 1.5.2
55
+ -------------
56
+
57
+ - Fix encrypted arguments double-encrypted by retry or rate limiting [#3368]
58
+ - Fix leak in concurrent rate limiter, run this in Rails console to clean up existing data [#3323]
59
+ ```ruby
60
+ expiry = 1.month.to_i; Sidekiq::Limiter.redis { |c| c.scan_each(match: "lmtr-cfree-*") { |key| c.expire(key, expiry) } }
61
+ ```
62
+
63
+ 1.5.1
64
+ -------------
65
+
66
+ - Fix issue with census startup when not using Bundler configuration for
67
+ source credentials.
68
+
69
+ 1.5.0
70
+ -------------
71
+
72
+ - Add new web authorization API [#3251]
73
+ - Update all sidekiqswarm env vars to use SIDEKIQ\_ prefix [#3218]
74
+ - Add census reporting, the leader will ping contribsys nightly with aggregate usage metrics
75
+
76
+ 1.4.0
77
+ -------------
78
+
79
+ - No functional changes, require latest Sidekiq and Sidekiq Pro versions
80
+
6
81
  1.3.2
7
82
  -------------
8
83
 
data/Gemfile CHANGED
@@ -1,29 +1,14 @@
1
1
  source 'https://rubygems.org'
2
2
  gemspec
3
3
 
4
- gem 'rails', '5.0.0'
5
- gem "hiredis"
6
- gem 'simplecov'
7
- gem 'minitest'
8
- gem 'minitest-utils'
9
- gem 'toxiproxy'
10
-
11
- platforms :rbx do
12
- gem 'rubysl', '~> 2.0' # if using anything in the ruby standard library
13
- gem 'psych' # if using yaml
14
- gem 'rubinius-developer_tools' # if using any of coverage, debugger, profiler
15
- end
16
-
17
- platforms :ruby do
18
- gem 'sqlite3'
4
+ # load testing
5
+ #gem "hiredis"
6
+ #gem 'toxiproxy'
7
+
8
+ group :test do
9
+ gem 'rails', '>= 5.0.1'
10
+ gem 'minitest'
11
+ gem 'rake'
12
+ gem 'redis-namespace'
13
+ gem 'simplecov'
19
14
  end
20
-
21
- platforms :mri do
22
- gem 'pry-byebug'
23
- gem 'ruby-prof'
24
- end
25
-
26
- #platforms :jruby do
27
- #gem 'jruby-openssl'
28
- #gem 'activerecord-jdbcsqlite3-adapter'
29
- #end
data/LICENSE CHANGED
@@ -5,5 +5,5 @@ the LGPLv3 license. Please see <http://www.gnu.org/licenses/lgpl-3.0.html>
5
5
  for license text.
6
6
 
7
7
  Sidekiq Pro has a commercial-friendly license allowing private forks
8
- and modifications of Sidekiq. Please see http://sidekiq.org/pro/ for
8
+ and modifications of Sidekiq. Please see https://sidekiq.org/products/pro.html for
9
9
  more detail. You can find the commercial license terms in COMM-LICENSE.
@@ -0,0 +1,35 @@
1
+ # Welcome to Sidekiq Pro 4.0!
2
+
3
+ Sidekiq Pro 4.0 is designed to work with Sidekiq 5.0.
4
+
5
+ ## What's New
6
+
7
+ * Batches now "die" if any of their jobs die. You can enumerate the set
8
+ of dead batches and their associated dead jobs. The success callback
9
+ for a dead batch will never fire unless these jobs are fixed.
10
+ ```ruby
11
+ Sidekiq::Batch::DeadSet.new.each do |status|
12
+ status.dead? # => true
13
+ status.dead_jobs # => [...]
14
+ end
15
+ ```
16
+ This API allows you to enumerate the batches which need help.
17
+ If you fix the issue and the dead jobs succeed, the batch will succeed.
18
+ * The older `reliable_fetch` and `timed_fetch` algorithms have been
19
+ removed. Only super\_fetch is available in 4.0.
20
+ * The statsd middleware has been tweaked to remove support for legacy,
21
+ pre-3.6.0 configuration and add relevant tags.
22
+ * Requires Sidekiq 5.0.5+.
23
+
24
+ ## Upgrade
25
+
26
+ * Upgrade to the latest Sidekiq Pro 3.x.
27
+ ```ruby
28
+ gem 'sidekiq-pro', '< 4'
29
+ ```
30
+ * Fix any deprecation warnings you see.
31
+ * Upgrade to 4.x.
32
+ ```ruby
33
+ gem 'sidekiq-pro', '< 5'
34
+ ```
35
+
data/Pro-Changes.md CHANGED
@@ -1,8 +1,162 @@
1
- Sidekiq Pro Changelog
2
- =======================
1
+ # Sidekiq Pro Changelog
2
+
3
+ [Sidekiq Changes](https://github.com/mperham/sidekiq/blob/master/Changes.md) | [Sidekiq Pro Changes](https://github.com/mperham/sidekiq/blob/master/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/mperham/sidekiq/blob/master/Ent-Changes.md)
3
4
 
4
5
  Please see [http://sidekiq.org/](http://sidekiq.org/) for more details and how to buy.
5
6
 
7
+ 4.0.3
8
+ ---------
9
+
10
+ - Add at\_exit handler to push any saved jobs in `reliable_push` when exiting. [#3823]
11
+ - Implement batch death callback. This is fired the first time a job within a batch dies. [#3841]
12
+ ```ruby
13
+ batch = Sidekiq::Batch.new
14
+ batch.on(:death, ...)
15
+ ```
16
+
17
+ 4.0.2
18
+ ---------
19
+
20
+ - Remove super\_fetch edge case leading to an unnecessary `sleep(1)`
21
+ call and resulting latency [#3790]
22
+ - Fix possible bad statsd metric call on super\_fetch startup
23
+ - Remove superfluous `freeze` calls on Strings [#3759]
24
+
25
+ 4.0.1
26
+ ---------
27
+
28
+ - Fix incompatibility with the statsd-ruby gem [#3740]
29
+
30
+ 4.0.0
31
+ ---------
32
+
33
+ - See the [Sidekiq Pro 4.0](Pro-4.0-Upgrade.md) release notes.
34
+
35
+
36
+ 3.7.1
37
+ ---------
38
+
39
+ - Deprecate timed\_fetch. Switch to super\_fetch:
40
+ ```ruby
41
+ config.super_fetch!
42
+ ```
43
+
44
+
45
+ 3.7.0
46
+ ---------
47
+
48
+ - Refactor batch job success/failure to gracefully handle several edge
49
+ cases with regard to Sidekiq::Shutdown. This should greatly reduce
50
+ the chances of seeing the long-standing "negative pending count" problem. [#3710]
51
+
52
+
53
+ 3.6.1
54
+ ---------
55
+
56
+ - Add support for Datadog::Statsd, it is the recommended Statsd client. [#3699]
57
+ ```ruby
58
+ Sidekiq::Pro.dogstatsd = ->{ Datadog::Statsd.new("metrics.example.com", 8125) }
59
+ ```
60
+ - Size the statsd connection pool based on Sidekiq's concurrency [#3700]
61
+
62
+
63
+ 3.6.0
64
+ ---------
65
+
66
+ This release overhauls the Statsd metrics support and adds more
67
+ metrics for tracking Pro feature usage. In your initializer:
68
+ ```ruby
69
+ Sidekiq::Pro.statsd = ->{ ::Statsd.new("127.0.0.1", 8125) }
70
+ ```
71
+ Sidekiq Pro will emit more metrics to Statsd:
72
+ ```
73
+ jobs.expired - when a job is expired
74
+ jobs.recovered.push - when a job is recovered by reliable_push after network outage
75
+ jobs.recovered.fetch - when a job is recovered by super_fetch after process crash
76
+ batch.created - when a batch is created
77
+ batch.complete - when a batch is completed
78
+ batch.success - when a batch is successful
79
+ ```
80
+ Sidekiq Pro's existing Statsd middleware has been rewritten to leverage the new API.
81
+ Everything should be backwards compatible with one deprecation notice.
82
+
83
+
84
+ 3.5.4
85
+ ---------
86
+
87
+ - Fix case in SuperFetch where Redis downtime can lead to processor thread death [#3684]
88
+ - Fix case where TimedFetch might not recover some pending jobs
89
+ - Fix edge case in Batch::Status#poll leading to premature completion [#3640]
90
+ - Adjust scan API to check 100 elements at a time, to minimize network round trips
91
+ when scanning large sets.
92
+
93
+ 3.5.3
94
+ ---------
95
+
96
+ - Restore error check for super\_fetch's job ack [#3601]
97
+ - Trim error messages saved in Batch's failure hash, preventing huge
98
+ messages from bloating Redis. [#3570]
99
+
100
+ 3.5.2
101
+ ---------
102
+
103
+ - Fix `Status#completed?` when run against a Batch that had succeeded
104
+ and was deleted. [#3519]
105
+
106
+ 3.5.1
107
+ ---------
108
+
109
+ - Work with Sidekiq 5.0.2+
110
+ - Improve performance of super\_fetch with weighted queues [#3489]
111
+
112
+ 3.5.0
113
+ ---------
114
+
115
+ - Add queue pause/unpause endpoints for scripting via curl [#3445]
116
+ - Change how super\_fetch names private queues to avoid hostname/queue clashes. [#3443]
117
+ - Re-implement `Sidekiq::Queue#delete_job` to avoid O(n) runtime [#3408]
118
+ - Batch page displays Pending JIDs if less than 10 [#3130]
119
+ - Batch page has a Search button to find associated Retries [#3130]
120
+ - Make Batch UI progress bar more friendly to the colorblind [#3387]
121
+
122
+ 3.4.5
123
+ ---------
124
+
125
+ - Fix potential job loss with reliable scheduler when lots of jobs are scheduled
126
+ at precisely the same time. Thanks to raivil for his hard work in
127
+ reproducing the bug. [#3371]
128
+
129
+ 3.4.4
130
+ ---------
131
+
132
+ - Optimize super\_fetch shutdown to restart jobs quicker [#3249]
133
+
134
+ 3.4.3
135
+ ---------
136
+
137
+ - Limit reliable scheduler to enqueue up to 100 jobs per call, minimizing Redis latency [#3332]
138
+ - Fix bug in super\_fetch logic for queues with `_` in the name [#3339]
139
+
140
+ 3.4.2
141
+ ---------
142
+
143
+ - Add `Batch::Status#invalidated?` API which returns true if any/all
144
+ JIDs were invalidated within the batch. [#3326]
145
+
146
+ 3.4.1
147
+ ---------
148
+
149
+ - Allow super\_fetch's orphan job check to happen as often as every hour [#3273]
150
+ - Officially deprecate reliable\_fetch algorithm, I now recommend you use `super_fetch` instead:
151
+ ```ruby
152
+ Sidekiq.configure_server do |config|
153
+ config.super_fetch!
154
+ end
155
+ ```
156
+ Also note that Sidekiq's `-i/--index` option is no longer used/relevant with super\_fetch.
157
+ - Don't display "Delete/Retry All" buttons when filtering in Web UI [#3243]
158
+ - Reimplement Sidekiq::JobSet#find\_job with ZSCAN [#3197]
159
+
6
160
  3.4.0
7
161
  ---------
8
162
 
data/README.md CHANGED
@@ -28,13 +28,14 @@ Sidekiq 3.5.1 | 22ms | 1257 MB | 125 sec | 800 jobs/sec
28
28
  Resque 1.25.2 | - | - | 420 sec | 240 jobs/sec
29
29
  DelayedJob 4.1.1 | - | - | 465 sec | 215 jobs/sec
30
30
 
31
+ <small>This benchmark can be found in `bin/sidekiqload`.</small>
31
32
 
32
33
  Requirements
33
34
  -----------------
34
35
 
35
- Sidekiq supports CRuby 2.0+ and JRuby 9k.
36
+ Sidekiq supports CRuby 2.2.2+ and JRuby 9k.
36
37
 
37
- All Rails releases >= 3.2 are officially supported.
38
+ All Rails releases >= 4.0 are officially supported.
38
39
 
39
40
  Redis 2.8 or greater is required. 3.0.3+ is recommended for large
40
41
  installations with thousands of worker threads.
@@ -50,7 +51,8 @@ Getting Started
50
51
  -----------------
51
52
 
52
53
  See the [Getting Started wiki page](https://github.com/mperham/sidekiq/wiki/Getting-Started) and follow the simple setup process.
53
- You can watch [Railscast #366](http://railscasts.com/episodes/366-sidekiq) to see Sidekiq in action. If you do everything right, you should see this:
54
+ You can watch [this Youtube playlist](https://www.youtube.com/playlist?list=PLjeHh2LSCFrWGT5uVjUuFKAcrcj5kSai1) to learn all about
55
+ Sidekiq and see its features in action. Here's the Web UI:
54
56
 
55
57
  ![Web UI](https://github.com/mperham/sidekiq/raw/master/examples/web-ui.png)
56
58
 
@@ -75,17 +77,18 @@ Problems?
75
77
  If you have a problem, please review the [FAQ](https://github.com/mperham/sidekiq/wiki/FAQ) and [Troubleshooting](https://github.com/mperham/sidekiq/wiki/Problems-and-Troubleshooting) wiki pages.
76
78
  Searching the [issues](https://github.com/mperham/sidekiq/issues) for your problem is also a good idea.
77
79
 
80
+ Sidekiq Pro and Sidekiq Enterprise customers get private email support. You can purchase at http://sidekiq.org; email support@contribsys.com for help.
81
+
78
82
  Useful resources:
79
83
 
80
84
  * Product documentation is in the [wiki](https://github.com/mperham/sidekiq/wiki).
81
85
  * Release announcements are made to the [@sidekiq](https://twitter.com/sidekiq) Twitter account.
82
- * Here's a [Reddit forum](https://reddit.com/r/sidekiq) dedicated to Sidekiq discussion
83
86
  * The [Sidekiq tag](https://stackoverflow.com/questions/tagged/sidekiq) on Stack Overflow has lots of useful Q &amp; A.
84
87
 
85
- **No support via Twitter, 140 characters is not enough.**
88
+ **No support via Twitter**
86
89
 
87
90
  Every Friday morning is Sidekiq happy hour: I video chat and answer questions.
88
- See the [Sidekiq support page](http://sidekiq.org/support) for details.
91
+ See the [Sidekiq support page](http://sidekiq.org/support.html) for details.
89
92
 
90
93
  Thanks
91
94
  -----------------
data/Rakefile CHANGED
@@ -1,8 +1,7 @@
1
1
  require 'bundler/gem_tasks'
2
2
  require 'rake/testtask'
3
3
  Rake::TestTask.new(:test) do |test|
4
- #SO MUCH NOISE
5
- #test.warning = true
4
+ test.warning = true
6
5
  test.pattern = 'test/**/test_*.rb'
7
6
  end
8
7
 
data/bin/sidekiqctl CHANGED
@@ -64,7 +64,7 @@ class Sidekiqctl
64
64
  end
65
65
 
66
66
  def quiet
67
- `kill -USR1 #{pid}`
67
+ `kill -TSTP #{pid}`
68
68
  end
69
69
 
70
70
  def stop