sidekiq 6.0.7 → 6.4.2
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of sidekiq might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/Changes.md +189 -2
- data/LICENSE +3 -3
- data/README.md +11 -10
- data/bin/sidekiq +8 -3
- data/bin/sidekiqload +57 -65
- data/bin/sidekiqmon +1 -1
- data/lib/generators/sidekiq/job_generator.rb +57 -0
- data/lib/generators/sidekiq/templates/{worker.rb.erb → job.rb.erb} +2 -2
- data/lib/generators/sidekiq/templates/{worker_spec.rb.erb → job_spec.rb.erb} +1 -1
- data/lib/generators/sidekiq/templates/{worker_test.rb.erb → job_test.rb.erb} +1 -1
- data/lib/sidekiq/api.rb +164 -116
- data/lib/sidekiq/cli.rb +49 -15
- data/lib/sidekiq/client.rb +51 -70
- data/lib/sidekiq/delay.rb +2 -0
- data/lib/sidekiq/extensions/action_mailer.rb +3 -2
- data/lib/sidekiq/extensions/active_record.rb +4 -3
- data/lib/sidekiq/extensions/class_methods.rb +5 -4
- data/lib/sidekiq/extensions/generic_proxy.rb +4 -2
- data/lib/sidekiq/fetch.rb +32 -23
- data/lib/sidekiq/job.rb +13 -0
- data/lib/sidekiq/job_logger.rb +16 -28
- data/lib/sidekiq/job_retry.rb +32 -33
- data/lib/sidekiq/job_util.rb +67 -0
- data/lib/sidekiq/launcher.rb +113 -54
- data/lib/sidekiq/logger.rb +11 -20
- data/lib/sidekiq/manager.rb +16 -18
- data/lib/sidekiq/middleware/chain.rb +10 -8
- data/lib/sidekiq/middleware/current_attributes.rb +57 -0
- data/lib/sidekiq/middleware/i18n.rb +4 -4
- data/lib/sidekiq/monitor.rb +1 -1
- data/lib/sidekiq/paginator.rb +8 -8
- data/lib/sidekiq/processor.rb +31 -31
- data/lib/sidekiq/rails.rb +36 -20
- data/lib/sidekiq/redis_connection.rb +16 -15
- data/lib/sidekiq/scheduled.rb +51 -16
- data/lib/sidekiq/sd_notify.rb +1 -1
- data/lib/sidekiq/testing/inline.rb +4 -4
- data/lib/sidekiq/testing.rb +38 -39
- data/lib/sidekiq/util.rb +41 -0
- data/lib/sidekiq/version.rb +1 -1
- data/lib/sidekiq/web/action.rb +2 -2
- data/lib/sidekiq/web/application.rb +21 -12
- data/lib/sidekiq/web/csrf_protection.rb +180 -0
- data/lib/sidekiq/web/helpers.rb +39 -33
- data/lib/sidekiq/web/router.rb +5 -2
- data/lib/sidekiq/web.rb +36 -72
- data/lib/sidekiq/worker.rb +135 -16
- data/lib/sidekiq.rb +33 -17
- data/sidekiq.gemspec +11 -4
- data/web/assets/images/apple-touch-icon.png +0 -0
- data/web/assets/javascripts/application.js +113 -65
- data/web/assets/javascripts/dashboard.js +51 -51
- data/web/assets/stylesheets/application-dark.css +64 -43
- data/web/assets/stylesheets/application-rtl.css +0 -4
- data/web/assets/stylesheets/application.css +42 -239
- data/web/locales/ar.yml +8 -2
- data/web/locales/en.yml +4 -1
- data/web/locales/es.yml +18 -2
- data/web/locales/fr.yml +8 -1
- data/web/locales/ja.yml +3 -0
- data/web/locales/lt.yml +1 -1
- data/web/locales/pl.yml +4 -4
- data/web/locales/ru.yml +4 -0
- data/web/views/_footer.erb +1 -1
- data/web/views/_job_info.erb +1 -1
- data/web/views/_poll_link.erb +2 -5
- data/web/views/_summary.erb +7 -7
- data/web/views/busy.erb +51 -20
- data/web/views/dashboard.erb +22 -14
- data/web/views/dead.erb +1 -1
- data/web/views/layout.erb +2 -1
- data/web/views/morgue.erb +6 -6
- data/web/views/queue.erb +11 -11
- data/web/views/queues.erb +4 -4
- data/web/views/retries.erb +7 -7
- data/web/views/retry.erb +1 -1
- data/web/views/scheduled.erb +1 -1
- metadata +24 -49
- data/.circleci/config.yml +0 -60
- data/.github/contributing.md +0 -32
- data/.github/issue_template.md +0 -11
- data/.gitignore +0 -13
- data/.standard.yml +0 -20
- data/3.0-Upgrade.md +0 -70
- data/4.0-Upgrade.md +0 -53
- data/5.0-Upgrade.md +0 -56
- data/6.0-Upgrade.md +0 -72
- data/COMM-LICENSE +0 -97
- data/Ent-2.0-Upgrade.md +0 -37
- data/Ent-Changes.md +0 -256
- data/Gemfile +0 -24
- data/Gemfile.lock +0 -208
- data/Pro-2.0-Upgrade.md +0 -138
- data/Pro-3.0-Upgrade.md +0 -44
- data/Pro-4.0-Upgrade.md +0 -35
- data/Pro-5.0-Upgrade.md +0 -25
- data/Pro-Changes.md +0 -782
- data/Rakefile +0 -10
- data/code_of_conduct.md +0 -50
- data/lib/generators/sidekiq/worker_generator.rb +0 -57
data/Ent-2.0-Upgrade.md
DELETED
@@ -1,37 +0,0 @@
|
|
1
|
-
# Welcome to Sidekiq Enterprise 2.0!
|
2
|
-
|
3
|
-
Sidekiq Enterprise 2.0 adds a few new features and adds the requirement that license
|
4
|
-
credentials be available at runtime. Note that Sidekiq 6.0 does have major breaking changes.
|
5
|
-
|
6
|
-
## What's New
|
7
|
-
|
8
|
-
* Sidekiq Enterprise now requires license credentials at runtime. If you
|
9
|
-
configured Bundler as described in the access email you need do
|
10
|
-
nothing, everything should just work. If you are vendoring Sidekiq
|
11
|
-
Enterprise you will need to configure Bundler also or set
|
12
|
-
`SIDEKIQ_ENT_USERNAME=abcdef12 bundle exec sidekiq...` when starting the
|
13
|
-
process. [#4232]
|
14
|
-
* Dead jobs now release any unique locks they were holding when they died [#4162]
|
15
|
-
* Backoff can now be customized per rate limiter by passing in a Proc [#4219]
|
16
|
-
```ruby
|
17
|
-
limiter = Sidekiq::Limiter.bucket(:stripe, 10, :second, backoff: ->(limiter, job) {
|
18
|
-
return job['overrated'] || 5 # wait for N seconds, where N is the number of
|
19
|
-
# times we've failed the rate limit
|
20
|
-
})
|
21
|
-
```
|
22
|
-
* Removed deprecated APIs and warnings.
|
23
|
-
* Various changes for Sidekiq 6.0
|
24
|
-
* Requires Ruby 2.5+ and Redis 4.0+
|
25
|
-
* Requires Sidekiq 6.0+ and Sidekiq Pro 5.0+
|
26
|
-
|
27
|
-
## Upgrade
|
28
|
-
|
29
|
-
* Upgrade to the latest Sidekiq Enterprise 1.x.
|
30
|
-
```ruby
|
31
|
-
gem 'sidekiq-ent', '< 2'
|
32
|
-
```
|
33
|
-
* Fix any deprecation warnings you see.
|
34
|
-
* Upgrade to 2.x.
|
35
|
-
```ruby
|
36
|
-
gem 'sidekiq-ent', '< 3'
|
37
|
-
```
|
data/Ent-Changes.md
DELETED
@@ -1,256 +0,0 @@
|
|
1
|
-
# Sidekiq Enterprise Changelog
|
2
|
-
|
3
|
-
[Sidekiq Changes](https://github.com/mperham/sidekiq/blob/master/Changes.md) | [Sidekiq Pro Changes](https://github.com/mperham/sidekiq/blob/master/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/mperham/sidekiq/blob/master/Ent-Changes.md)
|
4
|
-
|
5
|
-
Please see [http://sidekiq.org/](http://sidekiq.org/) for more details and how to buy.
|
6
|
-
|
7
|
-
2.0.1
|
8
|
-
-------------
|
9
|
-
|
10
|
-
- Periodic job registration API adjusted to avoid loading classes in initializer [#4271]
|
11
|
-
- Remove support for deprecated ENV variables (COUNT, MAXMEM\_MB, INDEX) in swarm code
|
12
|
-
|
13
|
-
2.0.0
|
14
|
-
-------------
|
15
|
-
|
16
|
-
- Except for the [newly required credentials](https://github.com/mperham/sidekiq/issues/4232), Sidekiq Enterprise 2.0 does
|
17
|
-
not have any significant migration steps.
|
18
|
-
- Sidekiq Enterprise must now be started with valid license credentials. [#4232]
|
19
|
-
- Call `GC.compact` if possible in sidekiqswarm before forking [#4181]
|
20
|
-
- Changes for forward-compatibility with Sidekiq 6.0.
|
21
|
-
- Add death handler to remove any lingering unique locks [#4162]
|
22
|
-
- Backoff can now be customized per rate limiter [#4219]
|
23
|
-
- Code formatting changes for StandardRB
|
24
|
-
|
25
|
-
1.8.1
|
26
|
-
-------------
|
27
|
-
|
28
|
-
- Fix excessive lock reclaims with concurrent limiter [#4105]
|
29
|
-
- Add ES translations, see issues [#3949](https://github.com/mperham/sidekiq/issues/3949) and [#3951](https://github.com/mperham/sidekiq/issues/3951) to add your own language.
|
30
|
-
|
31
|
-
1.8.0
|
32
|
-
-------------
|
33
|
-
|
34
|
-
- Require Sidekiq Pro 4.0 and Sidekiq 5.2.
|
35
|
-
- Refactor historical metrics API to use revamped Statsd support in Sidekiq Pro
|
36
|
-
- Add a gauge to historical metrics for `default` queue latency [#4079]
|
37
|
-
|
38
|
-
1.7.2
|
39
|
-
-------------
|
40
|
-
|
41
|
-
- Add PT and JA translations
|
42
|
-
- Fix elapsed time calculations to use monotonic clock [#4000, sj26]
|
43
|
-
- Fix edge case where flapping leadership would cause old periodic
|
44
|
-
jobs to be fired once [#3974]
|
45
|
-
- Add support for sidekiqswarm memory monitoring on FreeBSD [#3884]
|
46
|
-
|
47
|
-
1.7.1
|
48
|
-
-------------
|
49
|
-
|
50
|
-
- Fix Lua error in concurrent rate limiter under heavy contention
|
51
|
-
- Remove superfluous `freeze` calls on Strings [#3759]
|
52
|
-
|
53
|
-
1.7.0
|
54
|
-
-------------
|
55
|
-
|
56
|
-
- **NEW FEATURE** [Rolling restarts](https://github.com/mperham/sidekiq/wiki/Ent-Rolling-Restarts) - great for long running jobs!
|
57
|
-
- Adjust middleware so unique jobs that don't push aren't registered in a Batch [#3662]
|
58
|
-
- Add new unlimited rate limiter, useful for testing [#3743]
|
59
|
-
```ruby
|
60
|
-
limiter = Sidekiq::Limiter.unlimited(...any args...)
|
61
|
-
```
|
62
|
-
|
63
|
-
1.6.1
|
64
|
-
-------------
|
65
|
-
|
66
|
-
- Fix crash in rate limiter middleware when used with custom exceptions [#3604]
|
67
|
-
|
68
|
-
1.6.0
|
69
|
-
-------------
|
70
|
-
|
71
|
-
- Show process "leader" tag on Busy page, requires Sidekiq 5.0.2 [#2867]
|
72
|
-
- Capture custom metrics with the `save_history` API. [#2815]
|
73
|
-
- Implement new `unique_until: 'start'` policy option. [#3471]
|
74
|
-
|
75
|
-
1.5.4
|
76
|
-
-------------
|
77
|
-
|
78
|
-
- Fix broken Cron page in Web UI [#3458]
|
79
|
-
|
80
|
-
1.5.3
|
81
|
-
-------------
|
82
|
-
|
83
|
-
- Remove dependency on the algorithms gem [#3446]
|
84
|
-
- Allow user to specify max memory in megabytes with SIDEKIQ\_MAXMEM\_MB [#3451]
|
85
|
-
- Implement logic to detect app startup failure, sidekiqswarm will exit
|
86
|
-
rather than try to restart the app forever [#3450]
|
87
|
-
- Another fix for doubly-encrypted arguments [#3368]
|
88
|
-
|
89
|
-
1.5.2
|
90
|
-
-------------
|
91
|
-
|
92
|
-
- Fix encrypted arguments double-encrypted by retry or rate limiting [#3368]
|
93
|
-
- Fix leak in concurrent rate limiter, run this in Rails console to clean up existing data [#3323]
|
94
|
-
```ruby
|
95
|
-
expiry = 1.month.to_i; Sidekiq::Limiter.redis { |c| c.scan_each(match: "lmtr-cfree-*") { |key| c.expire(key, expiry) } }
|
96
|
-
```
|
97
|
-
|
98
|
-
1.5.1
|
99
|
-
-------------
|
100
|
-
|
101
|
-
- Fix issue with census startup when not using Bundler configuration for
|
102
|
-
source credentials.
|
103
|
-
|
104
|
-
1.5.0
|
105
|
-
-------------
|
106
|
-
|
107
|
-
- Add new web authorization API [#3251]
|
108
|
-
- Update all sidekiqswarm env vars to use SIDEKIQ\_ prefix [#3218]
|
109
|
-
- Add census reporting, the leader will ping contribsys nightly with aggregate usage metrics
|
110
|
-
|
111
|
-
1.4.0
|
112
|
-
-------------
|
113
|
-
|
114
|
-
- No functional changes, require latest Sidekiq and Sidekiq Pro versions
|
115
|
-
|
116
|
-
1.3.2
|
117
|
-
-------------
|
118
|
-
|
119
|
-
- Upgrade encryption to use OpenSSL's more secure GCM mode. [#3060]
|
120
|
-
|
121
|
-
1.3.1
|
122
|
-
-------------
|
123
|
-
|
124
|
-
- Fix multi-process memory monitoring on CentOS 6.x [#3063]
|
125
|
-
- Polish the new encryption feature a bit.
|
126
|
-
|
127
|
-
1.3.0
|
128
|
-
-------------
|
129
|
-
|
130
|
-
- **BETA** [New encryption feature](https://github.com/mperham/sidekiq/wiki/Ent-Encryption)
|
131
|
-
which automatically encrypts the last argument of a Worker, aka the secret bag.
|
132
|
-
|
133
|
-
1.2.4
|
134
|
-
-------------
|
135
|
-
|
136
|
-
- Fix issue causing some minutely jobs to execute every other minute.
|
137
|
-
- Log a warning if slow periodic processing causes us to miss a clock tick.
|
138
|
-
|
139
|
-
1.2.3
|
140
|
-
-------------
|
141
|
-
|
142
|
-
- Periodic jobs could stop executing until process restart if Redis goes down [#3047]
|
143
|
-
|
144
|
-
1.2.2
|
145
|
-
-------------
|
146
|
-
|
147
|
-
- Add API to check if a unique lock is present. See [#2932] for details.
|
148
|
-
- Tune concurrent limiters to minimize thread thrashing under heavy contention. [#2944]
|
149
|
-
- Add option for tuning which Bundler groups get preloaded with `sidekiqswarm` [#3025]
|
150
|
-
```
|
151
|
-
SIDEKIQ_PRELOAD=default,production bin/sidekiqswarm ...
|
152
|
-
# Use an empty value for maximum application compatibility
|
153
|
-
SIDEKIQ_PRELOAD= bin/sidekiqswarm ...
|
154
|
-
```
|
155
|
-
|
156
|
-
1.2.1
|
157
|
-
-------------
|
158
|
-
|
159
|
-
- Multi-Process mode can now monitor the RSS memory of children and
|
160
|
-
restart any that grow too large. To limit children to 1GB each:
|
161
|
-
```
|
162
|
-
MAXMEM_KB=1048576 COUNT=2 bundle exec sidekiqswarm ...
|
163
|
-
```
|
164
|
-
|
165
|
-
1.2.0
|
166
|
-
-------------
|
167
|
-
|
168
|
-
- **NEW FEATURE** Multi-process mode! Sidekiq Enterprise can now fork multiple worker
|
169
|
-
processes, enabling significant memory savings. See the [wiki
|
170
|
-
documentation](https://github.com/mperham/sidekiq/wiki/Ent-Multi-Process) for details.
|
171
|
-
|
172
|
-
|
173
|
-
0.7.10
|
174
|
-
-------------
|
175
|
-
|
176
|
-
- More precise gemspec dependency versioning
|
177
|
-
|
178
|
-
1.1.0
|
179
|
-
-------------
|
180
|
-
|
181
|
-
- **NEW FEATURE** Historical queue metrics, [documented in the wiki](https://github.com/mperham/sidekiq/wiki/Ent-Historical-Metrics) [#2719]
|
182
|
-
|
183
|
-
0.7.9, 1.0.2
|
184
|
-
-------------
|
185
|
-
|
186
|
-
- Window limiters can now accept arbitrary window sizes [#2686]
|
187
|
-
- Fix race condition in window limiters leading to non-stop OverLimit [#2704]
|
188
|
-
- Fix invalid overage counts when nesting concurrent limiters
|
189
|
-
|
190
|
-
1.0.1
|
191
|
-
----------
|
192
|
-
|
193
|
-
- Fix crash in periodic subsystem when a follower shuts down, thanks
|
194
|
-
to @justinko for reporting.
|
195
|
-
|
196
|
-
1.0.0
|
197
|
-
----------
|
198
|
-
|
199
|
-
- Enterprise 1.x targets Sidekiq 4.x.
|
200
|
-
- Rewrite several features to remove Celluloid dependency. No
|
201
|
-
functional changes.
|
202
|
-
|
203
|
-
0.7.8
|
204
|
-
----------
|
205
|
-
|
206
|
-
- Fix `unique_for: false` [#2658]
|
207
|
-
|
208
|
-
|
209
|
-
0.7.7
|
210
|
-
----------
|
211
|
-
|
212
|
-
- Enterprise 0.x targets Sidekiq 3.x.
|
213
|
-
- Fix racy shutdown event which could lead to disappearing periodic
|
214
|
-
jobs, requires Sidekiq >= 3.5.3.
|
215
|
-
- Add new :leader event which is fired when a process gains leadership.
|
216
|
-
|
217
|
-
0.7.6
|
218
|
-
----------
|
219
|
-
|
220
|
-
- Redesign how overrated jobs are rescheduled to avoid creating new
|
221
|
-
jobs. [#2619]
|
222
|
-
|
223
|
-
0.7.5
|
224
|
-
----------
|
225
|
-
|
226
|
-
- Fix dynamic creation of concurrent limiters [#2617]
|
227
|
-
|
228
|
-
0.7.4
|
229
|
-
----------
|
230
|
-
- Add additional check to prevent duplicate periodic job creation
|
231
|
-
- Allow user-specified TTLs for rate limiters [#2607]
|
232
|
-
- Paginate rate limiter index page [#2606]
|
233
|
-
|
234
|
-
0.7.3
|
235
|
-
----------
|
236
|
-
|
237
|
-
- Rework `Sidekiq::Limiter` redis handling to match global redis handling.
|
238
|
-
- Allow user to customize rate limit backoff logic and handle custom
|
239
|
-
rate limit errors.
|
240
|
-
- Fix scalability issue with Limiter index page.
|
241
|
-
|
242
|
-
0.7.2
|
243
|
-
----------
|
244
|
-
|
245
|
-
- Fix typo which prevented limiters with '0' in their names.
|
246
|
-
|
247
|
-
0.7.1
|
248
|
-
----------
|
249
|
-
|
250
|
-
- Fix issue where unique scheduled jobs can't be enqueued upon schedule
|
251
|
-
due to the existing unique lock. [#2499]
|
252
|
-
|
253
|
-
0.7.0
|
254
|
-
----------
|
255
|
-
|
256
|
-
Initial release.
|
data/Gemfile
DELETED
@@ -1,24 +0,0 @@
|
|
1
|
-
source "https://rubygems.org"
|
2
|
-
|
3
|
-
gemspec
|
4
|
-
|
5
|
-
gem "rake"
|
6
|
-
gem "redis-namespace"
|
7
|
-
gem "rails", ">= 6.0.2"
|
8
|
-
gem "sqlite3", platforms: :ruby
|
9
|
-
gem "activerecord-jdbcsqlite3-adapter", platforms: :jruby
|
10
|
-
|
11
|
-
group :test do
|
12
|
-
gem "minitest"
|
13
|
-
gem "simplecov"
|
14
|
-
end
|
15
|
-
|
16
|
-
group :development, :test do
|
17
|
-
gem "pry-byebug", platforms: :mri
|
18
|
-
gem "standard"
|
19
|
-
end
|
20
|
-
|
21
|
-
group :load_test do
|
22
|
-
gem "hiredis"
|
23
|
-
gem "toxiproxy"
|
24
|
-
end
|
data/Gemfile.lock
DELETED
@@ -1,208 +0,0 @@
|
|
1
|
-
PATH
|
2
|
-
remote: .
|
3
|
-
specs:
|
4
|
-
sidekiq (6.0.7)
|
5
|
-
connection_pool (>= 2.2.2)
|
6
|
-
rack (~> 2.0)
|
7
|
-
rack-protection (>= 2.0.0)
|
8
|
-
redis (>= 4.1.0)
|
9
|
-
|
10
|
-
GEM
|
11
|
-
remote: https://rubygems.org/
|
12
|
-
specs:
|
13
|
-
actioncable (6.0.2.1)
|
14
|
-
actionpack (= 6.0.2.1)
|
15
|
-
nio4r (~> 2.0)
|
16
|
-
websocket-driver (>= 0.6.1)
|
17
|
-
actionmailbox (6.0.2.1)
|
18
|
-
actionpack (= 6.0.2.1)
|
19
|
-
activejob (= 6.0.2.1)
|
20
|
-
activerecord (= 6.0.2.1)
|
21
|
-
activestorage (= 6.0.2.1)
|
22
|
-
activesupport (= 6.0.2.1)
|
23
|
-
mail (>= 2.7.1)
|
24
|
-
actionmailer (6.0.2.1)
|
25
|
-
actionpack (= 6.0.2.1)
|
26
|
-
actionview (= 6.0.2.1)
|
27
|
-
activejob (= 6.0.2.1)
|
28
|
-
mail (~> 2.5, >= 2.5.4)
|
29
|
-
rails-dom-testing (~> 2.0)
|
30
|
-
actionpack (6.0.2.1)
|
31
|
-
actionview (= 6.0.2.1)
|
32
|
-
activesupport (= 6.0.2.1)
|
33
|
-
rack (~> 2.0, >= 2.0.8)
|
34
|
-
rack-test (>= 0.6.3)
|
35
|
-
rails-dom-testing (~> 2.0)
|
36
|
-
rails-html-sanitizer (~> 1.0, >= 1.2.0)
|
37
|
-
actiontext (6.0.2.1)
|
38
|
-
actionpack (= 6.0.2.1)
|
39
|
-
activerecord (= 6.0.2.1)
|
40
|
-
activestorage (= 6.0.2.1)
|
41
|
-
activesupport (= 6.0.2.1)
|
42
|
-
nokogiri (>= 1.8.5)
|
43
|
-
actionview (6.0.2.1)
|
44
|
-
activesupport (= 6.0.2.1)
|
45
|
-
builder (~> 3.1)
|
46
|
-
erubi (~> 1.4)
|
47
|
-
rails-dom-testing (~> 2.0)
|
48
|
-
rails-html-sanitizer (~> 1.1, >= 1.2.0)
|
49
|
-
activejob (6.0.2.1)
|
50
|
-
activesupport (= 6.0.2.1)
|
51
|
-
globalid (>= 0.3.6)
|
52
|
-
activemodel (6.0.2.1)
|
53
|
-
activesupport (= 6.0.2.1)
|
54
|
-
activerecord (6.0.2.1)
|
55
|
-
activemodel (= 6.0.2.1)
|
56
|
-
activesupport (= 6.0.2.1)
|
57
|
-
activerecord-jdbc-adapter (60.1-java)
|
58
|
-
activerecord (~> 6.0.0)
|
59
|
-
activerecord-jdbcsqlite3-adapter (60.1-java)
|
60
|
-
activerecord-jdbc-adapter (= 60.1)
|
61
|
-
jdbc-sqlite3 (~> 3.8, < 3.30)
|
62
|
-
activestorage (6.0.2.1)
|
63
|
-
actionpack (= 6.0.2.1)
|
64
|
-
activejob (= 6.0.2.1)
|
65
|
-
activerecord (= 6.0.2.1)
|
66
|
-
marcel (~> 0.3.1)
|
67
|
-
activesupport (6.0.2.1)
|
68
|
-
concurrent-ruby (~> 1.0, >= 1.0.2)
|
69
|
-
i18n (>= 0.7, < 2)
|
70
|
-
minitest (~> 5.1)
|
71
|
-
tzinfo (~> 1.1)
|
72
|
-
zeitwerk (~> 2.2)
|
73
|
-
ast (2.4.0)
|
74
|
-
builder (3.2.4)
|
75
|
-
byebug (11.1.1)
|
76
|
-
coderay (1.1.2)
|
77
|
-
concurrent-ruby (1.1.6)
|
78
|
-
connection_pool (2.2.2)
|
79
|
-
crass (1.0.6)
|
80
|
-
docile (1.3.2)
|
81
|
-
erubi (1.9.0)
|
82
|
-
globalid (0.4.2)
|
83
|
-
activesupport (>= 4.2.0)
|
84
|
-
hiredis (0.6.3)
|
85
|
-
hiredis (0.6.3-java)
|
86
|
-
i18n (1.8.2)
|
87
|
-
concurrent-ruby (~> 1.0)
|
88
|
-
jaro_winkler (1.5.4)
|
89
|
-
jaro_winkler (1.5.4-java)
|
90
|
-
jdbc-sqlite3 (3.28.0)
|
91
|
-
loofah (2.4.0)
|
92
|
-
crass (~> 1.0.2)
|
93
|
-
nokogiri (>= 1.5.9)
|
94
|
-
mail (2.7.1)
|
95
|
-
mini_mime (>= 0.1.1)
|
96
|
-
marcel (0.3.3)
|
97
|
-
mimemagic (~> 0.3.2)
|
98
|
-
method_source (0.9.2)
|
99
|
-
mimemagic (0.3.4)
|
100
|
-
mini_mime (1.0.2)
|
101
|
-
mini_portile2 (2.4.0)
|
102
|
-
minitest (5.14.0)
|
103
|
-
nio4r (2.5.2)
|
104
|
-
nio4r (2.5.2-java)
|
105
|
-
nokogiri (1.10.8)
|
106
|
-
mini_portile2 (~> 2.4.0)
|
107
|
-
nokogiri (1.10.8-java)
|
108
|
-
parallel (1.19.1)
|
109
|
-
parser (2.7.0.4)
|
110
|
-
ast (~> 2.4.0)
|
111
|
-
pry (0.12.2)
|
112
|
-
coderay (~> 1.1.0)
|
113
|
-
method_source (~> 0.9.0)
|
114
|
-
pry-byebug (3.8.0)
|
115
|
-
byebug (~> 11.0)
|
116
|
-
pry (~> 0.10)
|
117
|
-
rack (2.2.2)
|
118
|
-
rack-protection (2.0.8.1)
|
119
|
-
rack
|
120
|
-
rack-test (1.1.0)
|
121
|
-
rack (>= 1.0, < 3)
|
122
|
-
rails (6.0.2.1)
|
123
|
-
actioncable (= 6.0.2.1)
|
124
|
-
actionmailbox (= 6.0.2.1)
|
125
|
-
actionmailer (= 6.0.2.1)
|
126
|
-
actionpack (= 6.0.2.1)
|
127
|
-
actiontext (= 6.0.2.1)
|
128
|
-
actionview (= 6.0.2.1)
|
129
|
-
activejob (= 6.0.2.1)
|
130
|
-
activemodel (= 6.0.2.1)
|
131
|
-
activerecord (= 6.0.2.1)
|
132
|
-
activestorage (= 6.0.2.1)
|
133
|
-
activesupport (= 6.0.2.1)
|
134
|
-
bundler (>= 1.3.0)
|
135
|
-
railties (= 6.0.2.1)
|
136
|
-
sprockets-rails (>= 2.0.0)
|
137
|
-
rails-dom-testing (2.0.3)
|
138
|
-
activesupport (>= 4.2.0)
|
139
|
-
nokogiri (>= 1.6)
|
140
|
-
rails-html-sanitizer (1.3.0)
|
141
|
-
loofah (~> 2.3)
|
142
|
-
railties (6.0.2.1)
|
143
|
-
actionpack (= 6.0.2.1)
|
144
|
-
activesupport (= 6.0.2.1)
|
145
|
-
method_source
|
146
|
-
rake (>= 0.8.7)
|
147
|
-
thor (>= 0.20.3, < 2.0)
|
148
|
-
rainbow (3.0.0)
|
149
|
-
rake (13.0.1)
|
150
|
-
redis (4.1.3)
|
151
|
-
redis-namespace (1.7.0)
|
152
|
-
redis (>= 3.0.4)
|
153
|
-
rubocop (0.79.0)
|
154
|
-
jaro_winkler (~> 1.5.1)
|
155
|
-
parallel (~> 1.10)
|
156
|
-
parser (>= 2.7.0.1)
|
157
|
-
rainbow (>= 2.2.2, < 4.0)
|
158
|
-
ruby-progressbar (~> 1.7)
|
159
|
-
unicode-display_width (>= 1.4.0, < 1.7)
|
160
|
-
rubocop-performance (1.5.2)
|
161
|
-
rubocop (>= 0.71.0)
|
162
|
-
ruby-progressbar (1.10.1)
|
163
|
-
simplecov (0.18.2)
|
164
|
-
docile (~> 1.1)
|
165
|
-
simplecov-html (~> 0.11)
|
166
|
-
simplecov-html (0.12.0)
|
167
|
-
sprockets (4.0.0)
|
168
|
-
concurrent-ruby (~> 1.0)
|
169
|
-
rack (> 1, < 3)
|
170
|
-
sprockets-rails (3.2.1)
|
171
|
-
actionpack (>= 4.0)
|
172
|
-
activesupport (>= 4.0)
|
173
|
-
sprockets (>= 3.0.0)
|
174
|
-
sqlite3 (1.4.2)
|
175
|
-
standard (0.2.1)
|
176
|
-
rubocop (~> 0.79.0)
|
177
|
-
rubocop-performance (~> 1.5.1)
|
178
|
-
thor (1.0.1)
|
179
|
-
thread_safe (0.3.6)
|
180
|
-
thread_safe (0.3.6-java)
|
181
|
-
toxiproxy (1.0.3)
|
182
|
-
tzinfo (1.2.6)
|
183
|
-
thread_safe (~> 0.1)
|
184
|
-
unicode-display_width (1.6.1)
|
185
|
-
websocket-driver (0.7.1)
|
186
|
-
websocket-extensions (>= 0.1.0)
|
187
|
-
websocket-driver (0.7.1-java)
|
188
|
-
websocket-extensions (>= 0.1.0)
|
189
|
-
websocket-extensions (0.1.4)
|
190
|
-
zeitwerk (2.2.2)
|
191
|
-
|
192
|
-
PLATFORMS
|
193
|
-
java
|
194
|
-
ruby
|
195
|
-
|
196
|
-
DEPENDENCIES
|
197
|
-
activerecord-jdbcsqlite3-adapter
|
198
|
-
hiredis
|
199
|
-
minitest
|
200
|
-
pry-byebug
|
201
|
-
rails (>= 6.0.2)
|
202
|
-
rake
|
203
|
-
redis-namespace
|
204
|
-
sidekiq!
|
205
|
-
simplecov
|
206
|
-
sqlite3
|
207
|
-
standard
|
208
|
-
toxiproxy
|
data/Pro-2.0-Upgrade.md
DELETED
@@ -1,138 +0,0 @@
|
|
1
|
-
# Upgrading to Sidekiq Pro 2.0
|
2
|
-
|
3
|
-
Sidekiq Pro 2.0 allows nested batches for more complex job workflows
|
4
|
-
and provides a new reliable scheduler which uses Lua to guarantee
|
5
|
-
atomicity and much higher performance.
|
6
|
-
|
7
|
-
It also removes deprecated APIs, changes the batch data format and
|
8
|
-
how features are activated. Read carefully to ensure your upgrade goes
|
9
|
-
smoothly.
|
10
|
-
|
11
|
-
Sidekiq Pro 2.0 requires Sidekiq 3.3.2 or greater. Redis 2.8 is
|
12
|
-
recommended; Redis 2.4 or 2.6 will work but some functionality will not be
|
13
|
-
available.
|
14
|
-
|
15
|
-
**Note that you CANNOT go back to Pro 1.x once you've created batches
|
16
|
-
with 2.x. The new batches will not process correctly with 1.x.**
|
17
|
-
|
18
|
-
**If you are on a version of Sidekiq Pro <1.5, you should upgrade to the
|
19
|
-
latest 1.x version and run it for a week before upgrading to 2.0.**
|
20
|
-
|
21
|
-
## Nested Batches
|
22
|
-
|
23
|
-
Batches can now be nested within the `jobs` method.
|
24
|
-
This feature enables Sidekiq Pro to handle workflow processing of any size
|
25
|
-
and complexity!
|
26
|
-
|
27
|
-
```ruby
|
28
|
-
a = Sidekiq::Batch.new
|
29
|
-
a.on(:success, SomeCallback)
|
30
|
-
a.jobs do
|
31
|
-
SomeWork.perform_async
|
32
|
-
|
33
|
-
b = Sidekiq::Batch.new
|
34
|
-
b.on(:success, MyCallback)
|
35
|
-
b.jobs do
|
36
|
-
OtherWork.perform_async
|
37
|
-
end
|
38
|
-
end
|
39
|
-
```
|
40
|
-
|
41
|
-
Parent batch callbacks are not processed until all child batch callbacks have
|
42
|
-
run successfully. In the example above, `MyCallback` will always fire
|
43
|
-
before `SomeCallback` because `b` is considered a child of `a`.
|
44
|
-
|
45
|
-
Of course you can dynamically add child batches while a batch job is executing.
|
46
|
-
|
47
|
-
```ruby
|
48
|
-
def perform(*args)
|
49
|
-
do_something(args)
|
50
|
-
|
51
|
-
if more_work?
|
52
|
-
# Sidekiq::Worker#batch returns the Batch this job is part of.
|
53
|
-
batch.jobs do
|
54
|
-
b = Sidekiq::Batch.new
|
55
|
-
b.on(:success, MyCallback)
|
56
|
-
b.jobs do
|
57
|
-
OtherWork.perform_async
|
58
|
-
end
|
59
|
-
end
|
60
|
-
end
|
61
|
-
end
|
62
|
-
```
|
63
|
-
|
64
|
-
More context: [#1485]
|
65
|
-
|
66
|
-
## Batch Data
|
67
|
-
|
68
|
-
The batch data model was overhauled. Batch data should take
|
69
|
-
significantly less space in Redis now. A simple benchmark shows 25%
|
70
|
-
savings but real world savings should be even greater.
|
71
|
-
|
72
|
-
* Batch 2.x BIDs are 14 character URL-safe Base64-encoded strings, e.g.
|
73
|
-
"vTF1-9QvLPnREQ". Batch 1.x BIDs were 16 character hex-encoded
|
74
|
-
strings, e.g. "4a3fc67d30370edf".
|
75
|
-
* In 1.x, batch data was not removed until it naturally expired in Redis.
|
76
|
-
In 2.x, all data for a batch is removed from Redis once the batch has
|
77
|
-
run any success callbacks.
|
78
|
-
* Because of the former point, batch expiry is no longer a concern.
|
79
|
-
Batch expiry is hardcoded to 30 days and is no longer user-tunable.
|
80
|
-
* Failed batch jobs no longer automatically store any associated
|
81
|
-
backtrace in Redis.
|
82
|
-
|
83
|
-
**There's no data migration required. Sidekiq Pro 2.0 transparently handles
|
84
|
-
both old and new format.**
|
85
|
-
|
86
|
-
More context: [#2130]
|
87
|
-
|
88
|
-
## Reliability
|
89
|
-
|
90
|
-
2.0 brings a new reliable scheduler which uses Lua inside Redis so enqueuing
|
91
|
-
scheduled jobs is atomic. Benchmarks show it 50x faster when enqueuing
|
92
|
-
lots of jobs.
|
93
|
-
|
94
|
-
**Two caveats**:
|
95
|
-
- Client-side middleware is not executed
|
96
|
-
for each job when enqueued with the reliable scheduler. No Sidekiq or
|
97
|
-
Sidekiq Pro functionality is affected by this change but some 3rd party
|
98
|
-
plugins might be.
|
99
|
-
- The Lua script used inside the reliable scheduler is not safe for use
|
100
|
-
with Redis Cluster or other multi-master Redis solutions.
|
101
|
-
It is safe to use with Redis Sentinel or a typical master/slave replication setup.
|
102
|
-
|
103
|
-
**You no longer require anything to use the Reliability features.**
|
104
|
-
|
105
|
-
* Activate reliable fetch and/or the new reliable scheduler:
|
106
|
-
```ruby
|
107
|
-
Sidekiq.configure_server do |config|
|
108
|
-
config.reliable_fetch!
|
109
|
-
config.reliable_scheduler!
|
110
|
-
end
|
111
|
-
```
|
112
|
-
* Activate reliable push:
|
113
|
-
```ruby
|
114
|
-
Sidekiq::Client.reliable_push!
|
115
|
-
```
|
116
|
-
|
117
|
-
More context: [#2130]
|
118
|
-
|
119
|
-
## Other Changes
|
120
|
-
|
121
|
-
* You must require `sidekiq/pro/notifications` if you want to use the
|
122
|
-
existing notification schemes. I don't recommend using them as the
|
123
|
-
newer-style `Sidekiq::Batch#on` method is simpler and more flexible.
|
124
|
-
* Several classes have been renamed. Generally these classes are ones
|
125
|
-
you should not need to require/use in your own code, e.g. the Batch
|
126
|
-
middleware.
|
127
|
-
* You can add `attr_accessor :jid` to a Batch callback class and Sidekiq
|
128
|
-
Pro will set it to the jid of the callback job. [#2178]
|
129
|
-
* There's now an official API to iterate all known Batches [#2191]
|
130
|
-
```ruby
|
131
|
-
Sidekiq::BatchSet.new.each {|status| p status.bid }
|
132
|
-
```
|
133
|
-
* The Web UI now shows the Sidekiq Pro version in the footer. [#1991]
|
134
|
-
|
135
|
-
## Thanks
|
136
|
-
|
137
|
-
Adam Prescott, Luke van der Hoeven and Jon Hyman all provided valuable
|
138
|
-
feedback during the release process. Thank you guys!
|
data/Pro-3.0-Upgrade.md
DELETED
@@ -1,44 +0,0 @@
|
|
1
|
-
# Welcome to Sidekiq Pro 3.0!
|
2
|
-
|
3
|
-
Sidekiq Pro 3.0 is designed to work with Sidekiq 4.0.
|
4
|
-
|
5
|
-
## What's New
|
6
|
-
|
7
|
-
* **Redis 2.8.0 or greater is required.** Redis 2.8 was released two years
|
8
|
-
ago and contains **many** useful features which Sidekiq couldn't
|
9
|
-
leverage until now. **Redis 3.0.3 or greater is recommended** for large
|
10
|
-
scale use.
|
11
|
-
|
12
|
-
* Sidekiq Pro no longer uses Celluloid. If your application code uses Celluloid,
|
13
|
-
you will need to pull it in yourself.
|
14
|
-
|
15
|
-
* Pausing and unpausing queues is now instantaneous, no more polling!
|
16
|
-
|
17
|
-
* Reliable fetch has been re-implemented due to the fetch changes in
|
18
|
-
Sidekiq 4.0.
|
19
|
-
|
20
|
-
* Support for platforms without persistent hostnames. Since the reliable\_fetch
|
21
|
-
algorithm requires a persistent hostname, an alternative reliability
|
22
|
-
algorithm is now available for platforms like Heroku and Docker:
|
23
|
-
```ruby
|
24
|
-
Sidekiq.configure_server do |config|
|
25
|
-
config.timed_fetch!
|
26
|
-
end
|
27
|
-
```
|
28
|
-
The wiki contains [much more detail about each reliability option](https://github.com/mperham/sidekiq/wiki/Pro-Reliability-Server).
|
29
|
-
|
30
|
-
* The old 'sidekiq/notifications' features have been removed.
|
31
|
-
|
32
|
-
## Upgrade
|
33
|
-
|
34
|
-
First, make sure you are using Redis 2.8 or greater. Next:
|
35
|
-
|
36
|
-
* Upgrade to the latest Sidekiq Pro 2.x.
|
37
|
-
```ruby
|
38
|
-
gem 'sidekiq-pro', '< 3'
|
39
|
-
```
|
40
|
-
* Fix any deprecation warnings you see.
|
41
|
-
* Upgrade to 3.x.
|
42
|
-
```ruby
|
43
|
-
gem 'sidekiq-pro', '< 4'
|
44
|
-
```
|