sidekiq 6.1.2 → 6.2.1

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of sidekiq might be problematic. Click here for more details.

Files changed (47) hide show
  1. checksums.yaml +4 -4
  2. data/Changes.md +49 -1
  3. data/lib/sidekiq/api.rb +22 -7
  4. data/lib/sidekiq/cli.rb +14 -1
  5. data/lib/sidekiq/client.rb +1 -5
  6. data/lib/sidekiq/fetch.rb +9 -1
  7. data/lib/sidekiq/job_retry.rb +1 -0
  8. data/lib/sidekiq/launcher.rb +58 -1
  9. data/lib/sidekiq/logger.rb +3 -2
  10. data/lib/sidekiq/util.rb +28 -0
  11. data/lib/sidekiq/version.rb +1 -1
  12. data/lib/sidekiq/web.rb +33 -78
  13. data/lib/sidekiq/web/action.rb +1 -1
  14. data/lib/sidekiq/web/application.rb +11 -3
  15. data/lib/sidekiq/web/csrf_protection.rb +28 -6
  16. data/lib/sidekiq/web/helpers.rb +23 -2
  17. data/lib/sidekiq/web/router.rb +4 -1
  18. data/sidekiq.gemspec +10 -2
  19. data/web/assets/images/apple-touch-icon.png +0 -0
  20. data/web/assets/stylesheets/application-dark.css +17 -0
  21. data/web/assets/stylesheets/application.css +17 -2
  22. data/web/locales/fr.yml +1 -1
  23. data/web/views/busy.erb +45 -14
  24. data/web/views/layout.erb +1 -0
  25. data/web/views/queue.erb +1 -1
  26. metadata +11 -26
  27. data/.github/ISSUE_TEMPLATE/bug_report.md +0 -20
  28. data/.github/contributing.md +0 -32
  29. data/.github/workflows/ci.yml +0 -41
  30. data/.gitignore +0 -13
  31. data/.standard.yml +0 -20
  32. data/3.0-Upgrade.md +0 -70
  33. data/4.0-Upgrade.md +0 -53
  34. data/5.0-Upgrade.md +0 -56
  35. data/6.0-Upgrade.md +0 -72
  36. data/COMM-LICENSE +0 -97
  37. data/Ent-2.0-Upgrade.md +0 -37
  38. data/Ent-Changes.md +0 -281
  39. data/Gemfile +0 -24
  40. data/Gemfile.lock +0 -192
  41. data/Pro-2.0-Upgrade.md +0 -138
  42. data/Pro-3.0-Upgrade.md +0 -44
  43. data/Pro-4.0-Upgrade.md +0 -35
  44. data/Pro-5.0-Upgrade.md +0 -25
  45. data/Pro-Changes.md +0 -805
  46. data/Rakefile +0 -10
  47. data/code_of_conduct.md +0 -50
data/Ent-2.0-Upgrade.md DELETED
@@ -1,37 +0,0 @@
1
- # Welcome to Sidekiq Enterprise 2.0!
2
-
3
- Sidekiq Enterprise 2.0 adds a few new features and adds the requirement that license
4
- credentials be available at runtime. Note that Sidekiq 6.0 does have major breaking changes.
5
-
6
- ## What's New
7
-
8
- * Sidekiq Enterprise now requires license credentials at runtime. If you
9
- configured Bundler as described in the access email you need do
10
- nothing, everything should just work. If you are vendoring Sidekiq
11
- Enterprise you will need to configure Bundler also or set
12
- `SIDEKIQ_ENT_USERNAME=abcdef12 bundle exec sidekiq...` when starting the
13
- process. [#4232]
14
- * Dead jobs now release any unique locks they were holding when they died [#4162]
15
- * Backoff can now be customized per rate limiter by passing in a Proc [#4219]
16
- ```ruby
17
- limiter = Sidekiq::Limiter.bucket(:stripe, 10, :second, backoff: ->(limiter, job) {
18
- return job['overrated'] || 5 # wait for N seconds, where N is the number of
19
- # times we've failed the rate limit
20
- })
21
- ```
22
- * Removed deprecated APIs and warnings.
23
- * Various changes for Sidekiq 6.0
24
- * Requires Ruby 2.5+ and Redis 4.0+
25
- * Requires Sidekiq 6.0+ and Sidekiq Pro 5.0+
26
-
27
- ## Upgrade
28
-
29
- * Upgrade to the latest Sidekiq Enterprise 1.x.
30
- ```ruby
31
- gem 'sidekiq-ent', '< 2'
32
- ```
33
- * Fix any deprecation warnings you see.
34
- * Upgrade to 2.x.
35
- ```ruby
36
- gem 'sidekiq-ent', '< 3'
37
- ```
data/Ent-Changes.md DELETED
@@ -1,281 +0,0 @@
1
- # Sidekiq Enterprise Changelog
2
-
3
- [Sidekiq Changes](https://github.com/mperham/sidekiq/blob/master/Changes.md) | [Sidekiq Pro Changes](https://github.com/mperham/sidekiq/blob/master/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/mperham/sidekiq/blob/master/Ent-Changes.md)
4
-
5
- Please see [sidekiq.org](https://sidekiq.org) for more details and how to buy.
6
-
7
- 2.1.2
8
- -------------
9
-
10
- - The Sidekiq Pro and Enterprise gem servers now `bundle install` much faster with **Bundler 2.2+** [#4158]
11
- - Now that ActiveJobs support `sidekiq_options`, add support for uniqueness in AJs [#4667]
12
-
13
- 2.1.1
14
- -------------
15
-
16
- - Add optional **app preload** in swarm, saves even more memory [#4646]
17
- - Fix incorrect queue tags in historical metrics [#4377]
18
-
19
- 2.1.0
20
- -------------
21
-
22
- - Move historical metrics to use tags rather than interpolating name [#4377]
23
- ```
24
- sidekiq.enqueued.#{name} -> sidekiq.queue.size with tag queue:#{name}
25
- sidekiq.latency.#{name} -> sidekiq.queue.latency with tag queue:#{name}
26
- ```
27
- - Remove `concurrent-ruby` gem dependency [#4586]
28
- - Add systemd `Type=notify` support for swarm [#4511]
29
- - Length swarm's boot timeout to 60 sec [#4544]
30
- - Add NL locale
31
-
32
- 2.0.1
33
- -------------
34
-
35
- - Periodic job registration API adjusted to avoid loading classes in initializer [#4271]
36
- - Remove support for deprecated ENV variables (COUNT, MAXMEM\_MB, INDEX) in swarm code
37
-
38
- 2.0.0
39
- -------------
40
-
41
- - Except for the [newly required credentials](https://github.com/mperham/sidekiq/issues/4232), Sidekiq Enterprise 2.0 does
42
- not have any significant migration steps.
43
- - Sidekiq Enterprise must now be started with valid license credentials. [#4232]
44
- - Call `GC.compact` if possible in sidekiqswarm before forking [#4181]
45
- - Changes for forward-compatibility with Sidekiq 6.0.
46
- - Add death handler to remove any lingering unique locks [#4162]
47
- - Backoff can now be customized per rate limiter [#4219]
48
- - Code formatting changes for StandardRB
49
-
50
- 1.8.1
51
- -------------
52
-
53
- - Fix excessive lock reclaims with concurrent limiter [#4105]
54
- - Add ES translations, see issues [#3949](https://github.com/mperham/sidekiq/issues/3949) and [#3951](https://github.com/mperham/sidekiq/issues/3951) to add your own language.
55
-
56
- 1.8.0
57
- -------------
58
-
59
- - Require Sidekiq Pro 4.0 and Sidekiq 5.2.
60
- - Refactor historical metrics API to use revamped Statsd support in Sidekiq Pro
61
- - Add a gauge to historical metrics for `default` queue latency [#4079]
62
-
63
- 1.7.2
64
- -------------
65
-
66
- - Add PT and JA translations
67
- - Fix elapsed time calculations to use monotonic clock [#4000, sj26]
68
- - Fix edge case where flapping leadership would cause old periodic
69
- jobs to be fired once [#3974]
70
- - Add support for sidekiqswarm memory monitoring on FreeBSD [#3884]
71
-
72
- 1.7.1
73
- -------------
74
-
75
- - Fix Lua error in concurrent rate limiter under heavy contention
76
- - Remove superfluous `freeze` calls on Strings [#3759]
77
-
78
- 1.7.0
79
- -------------
80
-
81
- - **NEW FEATURE** [Rolling restarts](https://github.com/mperham/sidekiq/wiki/Ent-Rolling-Restarts) - great for long running jobs!
82
- - Adjust middleware so unique jobs that don't push aren't registered in a Batch [#3662]
83
- - Add new unlimited rate limiter, useful for testing [#3743]
84
- ```ruby
85
- limiter = Sidekiq::Limiter.unlimited(...any args...)
86
- ```
87
-
88
- 1.6.1
89
- -------------
90
-
91
- - Fix crash in rate limiter middleware when used with custom exceptions [#3604]
92
-
93
- 1.6.0
94
- -------------
95
-
96
- - Show process "leader" tag on Busy page, requires Sidekiq 5.0.2 [#2867]
97
- - Capture custom metrics with the `save_history` API. [#2815]
98
- - Implement new `unique_until: 'start'` policy option. [#3471]
99
-
100
- 1.5.4
101
- -------------
102
-
103
- - Fix broken Cron page in Web UI [#3458]
104
-
105
- 1.5.3
106
- -------------
107
-
108
- - Remove dependency on the algorithms gem [#3446]
109
- - Allow user to specify max memory in megabytes with SIDEKIQ\_MAXMEM\_MB [#3451]
110
- - Implement logic to detect app startup failure, sidekiqswarm will exit
111
- rather than try to restart the app forever [#3450]
112
- - Another fix for doubly-encrypted arguments [#3368]
113
-
114
- 1.5.2
115
- -------------
116
-
117
- - Fix encrypted arguments double-encrypted by retry or rate limiting [#3368]
118
- - Fix leak in concurrent rate limiter, run this in Rails console to clean up existing data [#3323]
119
- ```ruby
120
- expiry = 1.month.to_i; Sidekiq::Limiter.redis { |c| c.scan_each(match: "lmtr-cfree-*") { |key| c.expire(key, expiry) } }
121
- ```
122
-
123
- 1.5.1
124
- -------------
125
-
126
- - Fix issue with census startup when not using Bundler configuration for
127
- source credentials.
128
-
129
- 1.5.0
130
- -------------
131
-
132
- - Add new web authorization API [#3251]
133
- - Update all sidekiqswarm env vars to use SIDEKIQ\_ prefix [#3218]
134
- - Add census reporting, the leader will ping contribsys nightly with aggregate usage metrics
135
-
136
- 1.4.0
137
- -------------
138
-
139
- - No functional changes, require latest Sidekiq and Sidekiq Pro versions
140
-
141
- 1.3.2
142
- -------------
143
-
144
- - Upgrade encryption to use OpenSSL's more secure GCM mode. [#3060]
145
-
146
- 1.3.1
147
- -------------
148
-
149
- - Fix multi-process memory monitoring on CentOS 6.x [#3063]
150
- - Polish the new encryption feature a bit.
151
-
152
- 1.3.0
153
- -------------
154
-
155
- - **BETA** [New encryption feature](https://github.com/mperham/sidekiq/wiki/Ent-Encryption)
156
- which automatically encrypts the last argument of a Worker, aka the secret bag.
157
-
158
- 1.2.4
159
- -------------
160
-
161
- - Fix issue causing some minutely jobs to execute every other minute.
162
- - Log a warning if slow periodic processing causes us to miss a clock tick.
163
-
164
- 1.2.3
165
- -------------
166
-
167
- - Periodic jobs could stop executing until process restart if Redis goes down [#3047]
168
-
169
- 1.2.2
170
- -------------
171
-
172
- - Add API to check if a unique lock is present. See [#2932] for details.
173
- - Tune concurrent limiters to minimize thread thrashing under heavy contention. [#2944]
174
- - Add option for tuning which Bundler groups get preloaded with `sidekiqswarm` [#3025]
175
- ```
176
- SIDEKIQ_PRELOAD=default,production bin/sidekiqswarm ...
177
- # Use an empty value for maximum application compatibility
178
- SIDEKIQ_PRELOAD= bin/sidekiqswarm ...
179
- ```
180
-
181
- 1.2.1
182
- -------------
183
-
184
- - Multi-Process mode can now monitor the RSS memory of children and
185
- restart any that grow too large. To limit children to 1GB each:
186
- ```
187
- MAXMEM_KB=1048576 COUNT=2 bundle exec sidekiqswarm ...
188
- ```
189
-
190
- 1.2.0
191
- -------------
192
-
193
- - **NEW FEATURE** Multi-process mode! Sidekiq Enterprise can now fork multiple worker
194
- processes, enabling significant memory savings. See the [wiki
195
- documentation](https://github.com/mperham/sidekiq/wiki/Ent-Multi-Process) for details.
196
-
197
-
198
- 0.7.10
199
- -------------
200
-
201
- - More precise gemspec dependency versioning
202
-
203
- 1.1.0
204
- -------------
205
-
206
- - **NEW FEATURE** Historical queue metrics, [documented in the wiki](https://github.com/mperham/sidekiq/wiki/Ent-Historical-Metrics) [#2719]
207
-
208
- 0.7.9, 1.0.2
209
- -------------
210
-
211
- - Window limiters can now accept arbitrary window sizes [#2686]
212
- - Fix race condition in window limiters leading to non-stop OverLimit [#2704]
213
- - Fix invalid overage counts when nesting concurrent limiters
214
-
215
- 1.0.1
216
- ----------
217
-
218
- - Fix crash in periodic subsystem when a follower shuts down, thanks
219
- to @justinko for reporting.
220
-
221
- 1.0.0
222
- ----------
223
-
224
- - Enterprise 1.x targets Sidekiq 4.x.
225
- - Rewrite several features to remove Celluloid dependency. No
226
- functional changes.
227
-
228
- 0.7.8
229
- ----------
230
-
231
- - Fix `unique_for: false` [#2658]
232
-
233
-
234
- 0.7.7
235
- ----------
236
-
237
- - Enterprise 0.x targets Sidekiq 3.x.
238
- - Fix racy shutdown event which could lead to disappearing periodic
239
- jobs, requires Sidekiq >= 3.5.3.
240
- - Add new :leader event which is fired when a process gains leadership.
241
-
242
- 0.7.6
243
- ----------
244
-
245
- - Redesign how overrated jobs are rescheduled to avoid creating new
246
- jobs. [#2619]
247
-
248
- 0.7.5
249
- ----------
250
-
251
- - Fix dynamic creation of concurrent limiters [#2617]
252
-
253
- 0.7.4
254
- ----------
255
- - Add additional check to prevent duplicate periodic job creation
256
- - Allow user-specified TTLs for rate limiters [#2607]
257
- - Paginate rate limiter index page [#2606]
258
-
259
- 0.7.3
260
- ----------
261
-
262
- - Rework `Sidekiq::Limiter` redis handling to match global redis handling.
263
- - Allow user to customize rate limit backoff logic and handle custom
264
- rate limit errors.
265
- - Fix scalability issue with Limiter index page.
266
-
267
- 0.7.2
268
- ----------
269
-
270
- - Fix typo which prevented limiters with '0' in their names.
271
-
272
- 0.7.1
273
- ----------
274
-
275
- - Fix issue where unique scheduled jobs can't be enqueued upon schedule
276
- due to the existing unique lock. [#2499]
277
-
278
- 0.7.0
279
- ----------
280
-
281
- Initial release.
data/Gemfile DELETED
@@ -1,24 +0,0 @@
1
- source "https://rubygems.org"
2
-
3
- gemspec
4
-
5
- gem "rake"
6
- gem "redis-namespace"
7
- gem "rails", ">= 6.0.2"
8
- gem "sqlite3", platforms: :ruby
9
- gem "activerecord-jdbcsqlite3-adapter", platforms: :jruby
10
-
11
- group :test do
12
- gem "minitest"
13
- gem "simplecov"
14
- gem "codecov", require: false
15
- end
16
-
17
- group :development, :test do
18
- gem "standard"
19
- end
20
-
21
- group :load_test do
22
- gem "hiredis"
23
- gem "toxiproxy"
24
- end
data/Gemfile.lock DELETED
@@ -1,192 +0,0 @@
1
- PATH
2
- remote: .
3
- specs:
4
- sidekiq (6.1.2)
5
- connection_pool (>= 2.2.2)
6
- rack (~> 2.0)
7
- redis (>= 4.2.0)
8
-
9
- GEM
10
- remote: https://rubygems.org/
11
- specs:
12
- actioncable (6.0.3.2)
13
- actionpack (= 6.0.3.2)
14
- nio4r (~> 2.0)
15
- websocket-driver (>= 0.6.1)
16
- actionmailbox (6.0.3.2)
17
- actionpack (= 6.0.3.2)
18
- activejob (= 6.0.3.2)
19
- activerecord (= 6.0.3.2)
20
- activestorage (= 6.0.3.2)
21
- activesupport (= 6.0.3.2)
22
- mail (>= 2.7.1)
23
- actionmailer (6.0.3.2)
24
- actionpack (= 6.0.3.2)
25
- actionview (= 6.0.3.2)
26
- activejob (= 6.0.3.2)
27
- mail (~> 2.5, >= 2.5.4)
28
- rails-dom-testing (~> 2.0)
29
- actionpack (6.0.3.2)
30
- actionview (= 6.0.3.2)
31
- activesupport (= 6.0.3.2)
32
- rack (~> 2.0, >= 2.0.8)
33
- rack-test (>= 0.6.3)
34
- rails-dom-testing (~> 2.0)
35
- rails-html-sanitizer (~> 1.0, >= 1.2.0)
36
- actiontext (6.0.3.2)
37
- actionpack (= 6.0.3.2)
38
- activerecord (= 6.0.3.2)
39
- activestorage (= 6.0.3.2)
40
- activesupport (= 6.0.3.2)
41
- nokogiri (>= 1.8.5)
42
- actionview (6.0.3.2)
43
- activesupport (= 6.0.3.2)
44
- builder (~> 3.1)
45
- erubi (~> 1.4)
46
- rails-dom-testing (~> 2.0)
47
- rails-html-sanitizer (~> 1.1, >= 1.2.0)
48
- activejob (6.0.3.2)
49
- activesupport (= 6.0.3.2)
50
- globalid (>= 0.3.6)
51
- activemodel (6.0.3.2)
52
- activesupport (= 6.0.3.2)
53
- activerecord (6.0.3.2)
54
- activemodel (= 6.0.3.2)
55
- activesupport (= 6.0.3.2)
56
- activestorage (6.0.3.2)
57
- actionpack (= 6.0.3.2)
58
- activejob (= 6.0.3.2)
59
- activerecord (= 6.0.3.2)
60
- marcel (~> 0.3.1)
61
- activesupport (6.0.3.2)
62
- concurrent-ruby (~> 1.0, >= 1.0.2)
63
- i18n (>= 0.7, < 2)
64
- minitest (~> 5.1)
65
- tzinfo (~> 1.1)
66
- zeitwerk (~> 2.2, >= 2.2.2)
67
- ast (2.4.1)
68
- builder (3.2.4)
69
- codecov (0.2.8)
70
- json
71
- simplecov
72
- concurrent-ruby (1.1.7)
73
- connection_pool (2.2.3)
74
- crass (1.0.6)
75
- docile (1.3.2)
76
- erubi (1.9.0)
77
- globalid (0.4.2)
78
- activesupport (>= 4.2.0)
79
- hiredis (0.6.3)
80
- i18n (1.8.5)
81
- concurrent-ruby (~> 1.0)
82
- json (2.3.1)
83
- loofah (2.7.0)
84
- crass (~> 1.0.2)
85
- nokogiri (>= 1.5.9)
86
- mail (2.7.1)
87
- mini_mime (>= 0.1.1)
88
- marcel (0.3.3)
89
- mimemagic (~> 0.3.2)
90
- method_source (1.0.0)
91
- mimemagic (0.3.5)
92
- mini_mime (1.0.2)
93
- mini_portile2 (2.4.0)
94
- minitest (5.14.1)
95
- nio4r (2.5.2)
96
- nokogiri (1.10.10)
97
- mini_portile2 (~> 2.4.0)
98
- parallel (1.19.2)
99
- parser (2.7.1.4)
100
- ast (~> 2.4.1)
101
- rack (2.2.3)
102
- rack-test (1.1.0)
103
- rack (>= 1.0, < 3)
104
- rails (6.0.3.2)
105
- actioncable (= 6.0.3.2)
106
- actionmailbox (= 6.0.3.2)
107
- actionmailer (= 6.0.3.2)
108
- actionpack (= 6.0.3.2)
109
- actiontext (= 6.0.3.2)
110
- actionview (= 6.0.3.2)
111
- activejob (= 6.0.3.2)
112
- activemodel (= 6.0.3.2)
113
- activerecord (= 6.0.3.2)
114
- activestorage (= 6.0.3.2)
115
- activesupport (= 6.0.3.2)
116
- bundler (>= 1.3.0)
117
- railties (= 6.0.3.2)
118
- sprockets-rails (>= 2.0.0)
119
- rails-dom-testing (2.0.3)
120
- activesupport (>= 4.2.0)
121
- nokogiri (>= 1.6)
122
- rails-html-sanitizer (1.3.0)
123
- loofah (~> 2.3)
124
- railties (6.0.3.2)
125
- actionpack (= 6.0.3.2)
126
- activesupport (= 6.0.3.2)
127
- method_source
128
- rake (>= 0.8.7)
129
- thor (>= 0.20.3, < 2.0)
130
- rainbow (3.0.0)
131
- rake (13.0.1)
132
- redis (4.2.1)
133
- redis-namespace (1.8.0)
134
- redis (>= 3.0.4)
135
- regexp_parser (1.7.1)
136
- rexml (3.2.4)
137
- rubocop (0.89.1)
138
- parallel (~> 1.10)
139
- parser (>= 2.7.1.1)
140
- rainbow (>= 2.2.2, < 4.0)
141
- regexp_parser (>= 1.7)
142
- rexml
143
- rubocop-ast (>= 0.3.0, < 1.0)
144
- ruby-progressbar (~> 1.7)
145
- unicode-display_width (>= 1.4.0, < 2.0)
146
- rubocop-ast (0.3.0)
147
- parser (>= 2.7.1.4)
148
- rubocop-performance (1.7.1)
149
- rubocop (>= 0.82.0)
150
- ruby-progressbar (1.10.1)
151
- simplecov (0.19.0)
152
- docile (~> 1.1)
153
- simplecov-html (~> 0.11)
154
- simplecov-html (0.12.2)
155
- sprockets (4.0.2)
156
- concurrent-ruby (~> 1.0)
157
- rack (> 1, < 3)
158
- sprockets-rails (3.2.1)
159
- actionpack (>= 4.0)
160
- activesupport (>= 4.0)
161
- sprockets (>= 3.0.0)
162
- sqlite3 (1.4.2)
163
- standard (0.5.2)
164
- rubocop (~> 0.89.1)
165
- rubocop-performance (~> 1.7.1)
166
- thor (1.0.1)
167
- thread_safe (0.3.6)
168
- toxiproxy (1.0.3)
169
- tzinfo (1.2.7)
170
- thread_safe (~> 0.1)
171
- unicode-display_width (1.7.0)
172
- websocket-driver (0.7.3)
173
- websocket-extensions (>= 0.1.0)
174
- websocket-extensions (0.1.5)
175
- zeitwerk (2.4.0)
176
-
177
- PLATFORMS
178
- ruby
179
-
180
- DEPENDENCIES
181
- activerecord-jdbcsqlite3-adapter
182
- codecov
183
- hiredis
184
- minitest
185
- rails (>= 6.0.2)
186
- rake
187
- redis-namespace
188
- sidekiq!
189
- simplecov
190
- sqlite3
191
- standard
192
- toxiproxy