sidekiq 4.0.2 → 4.1.0
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of sidekiq might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/.travis.yml +1 -2
- data/Changes.md +34 -0
- data/Ent-Changes.md +18 -0
- data/Pro-Changes.md +12 -0
- data/README.md +8 -10
- data/code_of_conduct.md +50 -0
- data/lib/sidekiq/api.rb +31 -8
- data/lib/sidekiq/cli.rb +2 -3
- data/lib/sidekiq/client.rb +9 -13
- data/lib/sidekiq/launcher.rb +1 -1
- data/lib/sidekiq/manager.rb +2 -1
- data/lib/sidekiq/middleware/server/retry_jobs.rb +3 -3
- data/lib/sidekiq/testing.rb +1 -1
- data/lib/sidekiq/version.rb +1 -1
- data/lib/sidekiq/worker.rb +20 -5
- data/test/test_client.rb +38 -0
- data/test/test_retry_exhausted.rb +123 -0
- data/test/test_scheduled.rb +1 -1
- data/test/test_testing_fake.rb +10 -0
- data/web/locales/ja.yml +9 -9
- data/web/views/busy.erb +6 -1
- data/web/views/retry.erb +1 -1
- metadata +5 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: a3977388ec259ea1ceaaecb66a201c6d84281060
|
4
|
+
data.tar.gz: 66ea0b036b3a178b266013bbaef05cc602efe270
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 027e42c3e2952cf9851e2239cbcc8ac50f76a11137a78acb2bd8351845dd7c2f7455ff010fee6f70d36cf65bd827aa15579f3e654d01fe7d8663cb86b8a0b71e
|
7
|
+
data.tar.gz: cd645448a7628606f7ca5403cd401ced5d618aab1122c386a9d34cd6cd8443d7b62c054462a532c3950e059a2eaf5543a992bce52e2a00908bf583b38d56b4ba
|
data/.travis.yml
CHANGED
data/Changes.md
CHANGED
@@ -1,5 +1,33 @@
|
|
1
1
|
# Sidekiq Changes
|
2
2
|
|
3
|
+
4.1.0
|
4
|
+
-----------
|
5
|
+
|
6
|
+
- Tag quiet processes in the Web UI [#2757, jcarlson]
|
7
|
+
- Pass last exception to sidekiq\_retries\_exhausted block [#2787, Nowaker]
|
8
|
+
```ruby
|
9
|
+
class MyWorker
|
10
|
+
include Sidekiq::Worker
|
11
|
+
sidekiq_retries_exhausted do |job, exception|
|
12
|
+
end
|
13
|
+
end
|
14
|
+
```
|
15
|
+
- Add native support for ActiveJob's `set(options)` method allowing
|
16
|
+
you to override worker options dynamically. This should make it
|
17
|
+
even easier to switch between ActiveJob and Sidekiq's native APIs [#2780]
|
18
|
+
```ruby
|
19
|
+
class MyWorker
|
20
|
+
include Sidekiq::Worker
|
21
|
+
sidekiq_options queue: 'default', retry: true
|
22
|
+
|
23
|
+
def perform(*args)
|
24
|
+
# do something
|
25
|
+
end
|
26
|
+
end
|
27
|
+
|
28
|
+
MyWorker.set(queue: 'high', retry: false).perform_async(1)
|
29
|
+
```
|
30
|
+
|
3
31
|
4.0.2
|
4
32
|
-----------
|
5
33
|
|
@@ -30,6 +58,12 @@ Sidekiq::Queues.clear_all
|
|
30
58
|
[detailed on my blog](http://www.mikeperham.com/2015/10/14/optimizing-sidekiq/).
|
31
59
|
- See the [4.0 upgrade notes](4.0-Upgrade.md) for more detail.
|
32
60
|
|
61
|
+
3.5.4
|
62
|
+
-----------
|
63
|
+
|
64
|
+
- Ensure exception message is a string [#2707]
|
65
|
+
- Revert racy Process.kill usage in sidekiqctl
|
66
|
+
|
33
67
|
3.5.3
|
34
68
|
-----------
|
35
69
|
|
data/Ent-Changes.md
CHANGED
@@ -3,6 +3,24 @@ Sidekiq Enterprise Changelog
|
|
3
3
|
|
4
4
|
Please see [http://sidekiq.org/](http://sidekiq.org/) for more details and how to buy.
|
5
5
|
|
6
|
+
1.2.0
|
7
|
+
-------------
|
8
|
+
|
9
|
+
- **NEW FEATURE** Multi-process mode! Sidekiq Enterprise can now fork multiple worker
|
10
|
+
processes, enabling significant memory savings. See the [wiki
|
11
|
+
documentation](https://github.com/mperham/sidekiq/wiki/Ent-Multi-Process) for details.
|
12
|
+
|
13
|
+
|
14
|
+
0.7.10
|
15
|
+
-------------
|
16
|
+
|
17
|
+
- More precise gemspec dependency versioning
|
18
|
+
|
19
|
+
1.1.0
|
20
|
+
-------------
|
21
|
+
|
22
|
+
- **NEW FEATURE** Historical queue metrics, [documented in the wiki](https://github.com/mperham/sidekiq/wiki/Ent-Historical-Metrics) [#2719]
|
23
|
+
|
6
24
|
0.7.9, 1.0.2
|
7
25
|
-------------
|
8
26
|
|
data/Pro-Changes.md
CHANGED
@@ -3,6 +3,18 @@ Sidekiq Pro Changelog
|
|
3
3
|
|
4
4
|
Please see [http://sidekiq.org/](http://sidekiq.org/) for more details and how to buy.
|
5
5
|
|
6
|
+
HEAD
|
7
|
+
-----------
|
8
|
+
|
9
|
+
- Make job registration with batch part of the atomic push so batch
|
10
|
+
metadata can't get out of sync with the job data. [#2714]
|
11
|
+
|
12
|
+
3.0.1
|
13
|
+
-----------
|
14
|
+
|
15
|
+
- Remove a number of Redis version checks since we can assume 2.8+ now.
|
16
|
+
- Fix expiring jobs client middleware not loaded on server
|
17
|
+
|
6
18
|
3.0.0
|
7
19
|
-----------
|
8
20
|
|
data/README.md
CHANGED
@@ -64,23 +64,21 @@ quality open source development all at the same time. Please see the
|
|
64
64
|
[Sidekiq](http://sidekiq.org/) homepage for more detail.
|
65
65
|
|
66
66
|
|
67
|
-
|
67
|
+
Problems?
|
68
68
|
-----------------
|
69
69
|
|
70
|
+
**Please do not directly email any Sidekiq committers with questions or problems.** A community is best served when discussions are held in public.
|
71
|
+
|
70
72
|
Please see the [sidekiq wiki](https://github.com/mperham/sidekiq/wiki) for the official documentation.
|
71
73
|
[mperham/sidekiq on Gitter](https://gitter.im/mperham/sidekiq) is dedicated to this project,
|
72
|
-
but bug reports or feature requests suggestions should still go through [issues on Github](https://github.com/mperham/sidekiq/issues). Release announcements are made to the [@sidekiq](https://twitter.com/sidekiq) Twitter account.
|
73
|
-
|
74
|
-
You may also find useful a [Reddit area](https://reddit.com/r/sidekiq) dedicated to Sidekiq discussion and [a Sidekiq tag](https://stackoverflow.com/questions/tagged/sidekiq) on Stack Overflow.
|
74
|
+
but bug reports or feature requests suggestions should still go through [issues on Github](https://github.com/mperham/sidekiq/issues). Release announcements are made to the [@sidekiq](https://twitter.com/sidekiq) Twitter account. **No support via Twitter.**
|
75
75
|
|
76
|
+
Every Friday morning is Sidekiq happy hour: I video chat and answer questions.
|
77
|
+
See the [Sidekiq support page](http://sidekiq.org/support).
|
76
78
|
|
77
|
-
|
78
|
-
-----------------
|
79
|
-
|
80
|
-
**Please do not directly email any Sidekiq committers with questions or problems.** A community is best served when discussions are held in public.
|
79
|
+
You may also find useful a [Reddit area](https://reddit.com/r/sidekiq) dedicated to Sidekiq discussion and [a Sidekiq tag](https://stackoverflow.com/questions/tagged/sidekiq) on Stack Overflow.
|
81
80
|
|
82
|
-
If you have a problem, please review the [FAQ](https://github.com/mperham/sidekiq/wiki/FAQ) and [Troubleshooting](https://github.com/mperham/sidekiq/wiki/Problems-and-Troubleshooting) wiki pages. Searching the issues for your problem is also a good idea. If that doesn't help, feel free to email the Sidekiq mailing list, chat in Gitter, or open a new issue.
|
83
|
-
StackOverflow or Reddit is the preferred place to ask questions on usage. If you are encountering what you think is a bug, please open an issue.
|
81
|
+
If you have a problem, please review the [FAQ](https://github.com/mperham/sidekiq/wiki/FAQ) and [Troubleshooting](https://github.com/mperham/sidekiq/wiki/Problems-and-Troubleshooting) wiki pages. Searching the issues for your problem is also a good idea. If that doesn't help, feel free to email the Sidekiq mailing list, chat in Gitter, or open a new issue. StackOverflow or Reddit is the preferred place to ask questions on usage. If you are encountering what you think is a bug, please open an issue.
|
84
82
|
|
85
83
|
|
86
84
|
Thanks
|
data/code_of_conduct.md
ADDED
@@ -0,0 +1,50 @@
|
|
1
|
+
# Contributor Code of Conduct
|
2
|
+
|
3
|
+
As contributors and maintainers of this project, and in the interest of
|
4
|
+
fostering an open and welcoming community, we pledge to respect all people who
|
5
|
+
contribute through reporting issues, posting feature requests, updating
|
6
|
+
documentation, submitting pull requests or patches, and other activities.
|
7
|
+
|
8
|
+
We are committed to making participation in this project a harassment-free
|
9
|
+
experience for everyone, regardless of level of experience, gender, gender
|
10
|
+
identity and expression, sexual orientation, disability, personal appearance,
|
11
|
+
body size, race, ethnicity, age, religion, or nationality.
|
12
|
+
|
13
|
+
Examples of unacceptable behavior by participants include:
|
14
|
+
|
15
|
+
* The use of sexualized language or imagery
|
16
|
+
* Personal attacks
|
17
|
+
* Trolling or insulting/derogatory comments
|
18
|
+
* Public or private harassment
|
19
|
+
* Publishing other's private information, such as physical or electronic
|
20
|
+
addresses, without explicit permission
|
21
|
+
* Other unethical or unprofessional conduct
|
22
|
+
|
23
|
+
Project maintainers have the right and responsibility to remove, edit, or
|
24
|
+
reject comments, commits, code, wiki edits, issues, and other contributions
|
25
|
+
that are not aligned to this Code of Conduct, or to ban temporarily or
|
26
|
+
permanently any contributor for other behaviors that they deem inappropriate,
|
27
|
+
threatening, offensive, or harmful.
|
28
|
+
|
29
|
+
By adopting this Code of Conduct, project maintainers commit themselves to
|
30
|
+
fairly and consistently applying these principles to every aspect of managing
|
31
|
+
this project. Project maintainers who do not follow or enforce the Code of
|
32
|
+
Conduct may be permanently removed from the project team.
|
33
|
+
|
34
|
+
This Code of Conduct applies both within project spaces and in public spaces
|
35
|
+
when an individual is representing the project or its community.
|
36
|
+
|
37
|
+
Instances of abusive, harassing, or otherwise unacceptable behavior may be
|
38
|
+
reported by contacting the project maintainer at mperham AT gmail.com. All
|
39
|
+
complaints will be reviewed and investigated and will result in a response that
|
40
|
+
is deemed necessary and appropriate to the circumstances. Maintainers are
|
41
|
+
obligated to maintain confidentiality with regard to the reporter of an
|
42
|
+
incident.
|
43
|
+
|
44
|
+
|
45
|
+
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
|
46
|
+
version 1.3.0, available at
|
47
|
+
[http://contributor-covenant.org/version/1/3/0/][version]
|
48
|
+
|
49
|
+
[homepage]: http://contributor-covenant.org
|
50
|
+
[version]: http://contributor-covenant.org/version/1/3/0/
|
data/lib/sidekiq/api.rb
CHANGED
@@ -191,6 +191,9 @@ module Sidekiq
|
|
191
191
|
class Queue
|
192
192
|
include Enumerable
|
193
193
|
|
194
|
+
##
|
195
|
+
# Return all known queues within Redis.
|
196
|
+
#
|
194
197
|
def self.all
|
195
198
|
Sidekiq.redis {|c| c.smembers('queues'.freeze) }.sort.map {|q| Sidekiq::Queue.new(q) }
|
196
199
|
end
|
@@ -211,6 +214,11 @@ module Sidekiq
|
|
211
214
|
false
|
212
215
|
end
|
213
216
|
|
217
|
+
##
|
218
|
+
# Calculates this queue's latency, the difference in seconds since the oldest
|
219
|
+
# job in the queue was enqueued.
|
220
|
+
#
|
221
|
+
# @return Float
|
214
222
|
def latency
|
215
223
|
entry = Sidekiq.redis do |conn|
|
216
224
|
conn.lrange(@rname, -1, -1)
|
@@ -227,7 +235,7 @@ module Sidekiq
|
|
227
235
|
|
228
236
|
while true do
|
229
237
|
range_start = page * page_size - deleted_size
|
230
|
-
range_end =
|
238
|
+
range_end = range_start + page_size - 1
|
231
239
|
entries = Sidekiq.redis do |conn|
|
232
240
|
conn.lrange @rname, range_start, range_end
|
233
241
|
end
|
@@ -240,6 +248,11 @@ module Sidekiq
|
|
240
248
|
end
|
241
249
|
end
|
242
250
|
|
251
|
+
##
|
252
|
+
# Find the job with the given JID within this queue.
|
253
|
+
#
|
254
|
+
# This is a slow, inefficient operation. Do not use under
|
255
|
+
# normal conditions. Sidekiq Pro contains a faster version.
|
243
256
|
def find_job(jid)
|
244
257
|
detect { |j| j.jid == jid }
|
245
258
|
end
|
@@ -349,7 +362,7 @@ module Sidekiq
|
|
349
362
|
end
|
350
363
|
|
351
364
|
def [](name)
|
352
|
-
@item
|
365
|
+
@item[name]
|
353
366
|
end
|
354
367
|
|
355
368
|
private
|
@@ -502,7 +515,7 @@ module Sidekiq
|
|
502
515
|
|
503
516
|
while true do
|
504
517
|
range_start = page * page_size + offset_size
|
505
|
-
range_end =
|
518
|
+
range_end = range_start + page_size - 1
|
506
519
|
elements = Sidekiq.redis do |conn|
|
507
520
|
conn.zrange name, range_start, range_end, with_scores: true
|
508
521
|
end
|
@@ -531,6 +544,11 @@ module Sidekiq
|
|
531
544
|
end
|
532
545
|
end
|
533
546
|
|
547
|
+
##
|
548
|
+
# Find the job with the given JID within this sorted set.
|
549
|
+
#
|
550
|
+
# This is a slow, inefficient operation. Do not use under
|
551
|
+
# normal conditions. Sidekiq Pro contains a faster version.
|
534
552
|
def find_job(jid)
|
535
553
|
self.detect { |j| j.jid == jid }
|
536
554
|
end
|
@@ -633,7 +651,6 @@ module Sidekiq
|
|
633
651
|
#
|
634
652
|
# Yields a Sidekiq::Process.
|
635
653
|
#
|
636
|
-
|
637
654
|
class ProcessSet
|
638
655
|
include Enumerable
|
639
656
|
|
@@ -674,13 +691,13 @@ module Sidekiq
|
|
674
691
|
# you'll be happier this way
|
675
692
|
result = conn.pipelined do
|
676
693
|
procs.each do |key|
|
677
|
-
conn.hmget(key, 'info', 'busy', 'beat')
|
694
|
+
conn.hmget(key, 'info', 'busy', 'beat', 'quiet')
|
678
695
|
end
|
679
696
|
end
|
680
697
|
|
681
|
-
result.each do |info, busy, at_s|
|
698
|
+
result.each do |info, busy, at_s, quiet|
|
682
699
|
hash = Sidekiq.load_json(info)
|
683
|
-
yield Process.new(hash.merge('busy' => busy.to_i, 'beat' => at_s.to_f))
|
700
|
+
yield Process.new(hash.merge('busy' => busy.to_i, 'beat' => at_s.to_f, 'quiet' => quiet))
|
684
701
|
end
|
685
702
|
end
|
686
703
|
|
@@ -697,7 +714,8 @@ module Sidekiq
|
|
697
714
|
end
|
698
715
|
|
699
716
|
#
|
700
|
-
# Sidekiq::Process
|
717
|
+
# Sidekiq::Process represents an active Sidekiq process talking with Redis.
|
718
|
+
# Each process has a set of attributes which look like this:
|
701
719
|
#
|
702
720
|
# {
|
703
721
|
# 'hostname' => 'app-1.example.com',
|
@@ -739,6 +757,10 @@ module Sidekiq
|
|
739
757
|
signal('TTIN')
|
740
758
|
end
|
741
759
|
|
760
|
+
def stopping?
|
761
|
+
self['quiet'] == 'true'
|
762
|
+
end
|
763
|
+
|
742
764
|
private
|
743
765
|
|
744
766
|
def signal(sig)
|
@@ -757,6 +779,7 @@ module Sidekiq
|
|
757
779
|
end
|
758
780
|
|
759
781
|
##
|
782
|
+
# A worker is a thread that is currently processing a job.
|
760
783
|
# Programmatic access to the current active worker set.
|
761
784
|
#
|
762
785
|
# WARNING WARNING WARNING
|
data/lib/sidekiq/cli.rb
CHANGED
@@ -76,9 +76,8 @@ module Sidekiq
|
|
76
76
|
# Starting here the process will now have multiple threads running.
|
77
77
|
fire_event(:startup)
|
78
78
|
|
79
|
-
logger.debug {
|
80
|
-
|
81
|
-
}
|
79
|
+
logger.debug { "Client Middleware: #{Sidekiq.client_middleware.map(&:klass).join(', ')}" }
|
80
|
+
logger.debug { "Server Middleware: #{Sidekiq.server_middleware.map(&:klass).join(', ')}" }
|
82
81
|
|
83
82
|
if !options[:daemon]
|
84
83
|
logger.info 'Starting processing, hit Ctrl-C to stop'
|
data/lib/sidekiq/client.rb
CHANGED
@@ -35,10 +35,8 @@ module Sidekiq
|
|
35
35
|
# Sidekiq::Client.new(ConnectionPool.new { Redis.new })
|
36
36
|
#
|
37
37
|
# Generally this is only needed for very large Sidekiq installs processing
|
38
|
-
#
|
39
|
-
#
|
40
|
-
# Some features, like the API, do not support sharding: they are designed to work
|
41
|
-
# against a single Redis instance only.
|
38
|
+
# thousands of jobs per second. I don't recommend sharding unless you
|
39
|
+
# cannot scale any other way (e.g. splitting your app into smaller apps).
|
42
40
|
def initialize(redis_pool=nil)
|
43
41
|
@redis_pool = redis_pool || Thread.current[:sidekiq_via_pool] || Sidekiq.redis_pool
|
44
42
|
end
|
@@ -49,11 +47,12 @@ module Sidekiq
|
|
49
47
|
# queue - the named queue to use, default 'default'
|
50
48
|
# class - the worker class to call, required
|
51
49
|
# args - an array of simple arguments to the perform method, must be JSON-serializable
|
52
|
-
# retry - whether to retry this job if it fails, true or
|
50
|
+
# retry - whether to retry this job if it fails, default true or an integer number of retries
|
53
51
|
# backtrace - whether to save any error backtrace, default false
|
54
52
|
#
|
55
53
|
# All options must be strings, not symbols. NB: because we are serializing to JSON, all
|
56
|
-
# symbols in 'args' will be converted to strings.
|
54
|
+
# symbols in 'args' will be converted to strings. Note that +backtrace: true+ can take quite a bit of
|
55
|
+
# space in Redis; a large volume of failing jobs can start Redis swapping if you aren't careful.
|
57
56
|
#
|
58
57
|
# Returns a unique Job ID. If middleware stops the job, nil will be returned instead.
|
59
58
|
#
|
@@ -72,9 +71,8 @@ module Sidekiq
|
|
72
71
|
|
73
72
|
##
|
74
73
|
# Push a large number of jobs to Redis. In practice this method is only
|
75
|
-
# useful if you are pushing
|
76
|
-
#
|
77
|
-
# basically cuts down on the redis round trip latency.
|
74
|
+
# useful if you are pushing thousands of jobs or more. This method
|
75
|
+
# cuts out the redis network round trip latency.
|
78
76
|
#
|
79
77
|
# Takes the same arguments as #push except that args is expected to be
|
80
78
|
# an Array of Arrays. All other keys are duplicated for each job. Each job
|
@@ -104,10 +102,8 @@ module Sidekiq
|
|
104
102
|
# end
|
105
103
|
#
|
106
104
|
# Generally this is only needed for very large Sidekiq installs processing
|
107
|
-
#
|
108
|
-
# you
|
109
|
-
# Some features, like the API, do not support sharding: they are designed to work
|
110
|
-
# against a single Redis instance.
|
105
|
+
# thousands of jobs per second. I do not recommend sharding unless
|
106
|
+
# you cannot scale any other way (e.g. splitting your app into smaller apps).
|
111
107
|
def self.via(pool)
|
112
108
|
raise ArgumentError, "No pool given" if pool.nil?
|
113
109
|
raise RuntimeError, "Sidekiq::Client.via is not re-entrant" if x = Thread.current[:sidekiq_via_pool] && x != pool
|
data/lib/sidekiq/launcher.rb
CHANGED
@@ -95,7 +95,7 @@ module Sidekiq
|
|
95
95
|
_, _, _, msg = Sidekiq.redis do |conn|
|
96
96
|
conn.pipelined do
|
97
97
|
conn.sadd('processes', key)
|
98
|
-
conn.hmset(key, 'info', json, 'busy', Processor::WORKER_STATE.size, 'beat', Time.now.to_f)
|
98
|
+
conn.hmset(key, 'info', json, 'busy', Processor::WORKER_STATE.size, 'beat', Time.now.to_f, 'quiet', @done)
|
99
99
|
conn.expire(key, 60)
|
100
100
|
conn.rpop("#{key}-signals")
|
101
101
|
end
|
data/lib/sidekiq/manager.rb
CHANGED
@@ -130,17 +130,17 @@ module Sidekiq
|
|
130
130
|
end
|
131
131
|
else
|
132
132
|
# Goodbye dear message, you (re)tried your best I'm sure.
|
133
|
-
retries_exhausted(worker, msg)
|
133
|
+
retries_exhausted(worker, msg, exception)
|
134
134
|
end
|
135
135
|
|
136
136
|
raise exception
|
137
137
|
end
|
138
138
|
|
139
|
-
def retries_exhausted(worker, msg)
|
139
|
+
def retries_exhausted(worker, msg, exception)
|
140
140
|
logger.debug { "Dropping message after hitting the retry maximum: #{msg}" }
|
141
141
|
begin
|
142
142
|
if worker.sidekiq_retries_exhausted_block?
|
143
|
-
worker.sidekiq_retries_exhausted_block.call(msg)
|
143
|
+
worker.sidekiq_retries_exhausted_block.call(msg, exception)
|
144
144
|
end
|
145
145
|
rescue => e
|
146
146
|
handle_exception(e, { context: "Error calling retries_exhausted for #{worker.class}", job: msg })
|
data/lib/sidekiq/testing.rb
CHANGED
@@ -170,7 +170,7 @@ module Sidekiq
|
|
170
170
|
end
|
171
171
|
|
172
172
|
def delete_for(jid, queue, klass)
|
173
|
-
jobs_by_queue[queue].delete_if { |job| job["jid"] == jid }
|
173
|
+
jobs_by_queue[queue.to_s].delete_if { |job| job["jid"] == jid }
|
174
174
|
jobs_by_worker[klass].delete_if { |job| job["jid"] == jid }
|
175
175
|
end
|
176
176
|
|
data/lib/sidekiq/version.rb
CHANGED
data/lib/sidekiq/worker.rb
CHANGED
@@ -50,6 +50,11 @@ module Sidekiq
|
|
50
50
|
raise ArgumentError, "Do not call .delay_until on a Sidekiq::Worker class, call .perform_at"
|
51
51
|
end
|
52
52
|
|
53
|
+
def set(options)
|
54
|
+
Thread.current[:sidekiq_worker_set] = options
|
55
|
+
self
|
56
|
+
end
|
57
|
+
|
53
58
|
def perform_async(*args)
|
54
59
|
client_push('class' => self, 'args' => args)
|
55
60
|
end
|
@@ -74,11 +79,15 @@ module Sidekiq
|
|
74
79
|
# Allows customization for this type of Worker.
|
75
80
|
# Legal options:
|
76
81
|
#
|
77
|
-
#
|
78
|
-
#
|
79
|
-
#
|
82
|
+
# queue - use a named queue for this Worker, default 'default'
|
83
|
+
# retry - enable the RetryJobs middleware for this Worker, *true* to use the default
|
84
|
+
# or *Integer* count
|
85
|
+
# backtrace - whether to save any error backtrace in the retry payload to display in web UI,
|
80
86
|
# can be true, false or an integer number of lines to save, default *false*
|
81
|
-
#
|
87
|
+
# pool - use the given Redis connection pool to push this type of job to a given shard.
|
88
|
+
#
|
89
|
+
# In practice, any option is allowed. This is the main mechanism to configure the
|
90
|
+
# options for a specific job.
|
82
91
|
def sidekiq_options(opts={})
|
83
92
|
self.sidekiq_options_hash = get_sidekiq_options.merge(opts.stringify_keys)
|
84
93
|
end
|
@@ -97,7 +106,13 @@ module Sidekiq
|
|
97
106
|
|
98
107
|
def client_push(item) # :nodoc:
|
99
108
|
pool = Thread.current[:sidekiq_via_pool] || get_sidekiq_options['pool'] || Sidekiq.redis_pool
|
100
|
-
|
109
|
+
hash = if Thread.current[:sidekiq_worker_set]
|
110
|
+
x, Thread.current[:sidekiq_worker_set] = Thread.current[:sidekiq_worker_set], nil
|
111
|
+
x.stringify_keys.merge(item.stringify_keys)
|
112
|
+
else
|
113
|
+
item.stringify_keys
|
114
|
+
end
|
115
|
+
Sidekiq::Client.new(pool).push(hash)
|
101
116
|
end
|
102
117
|
|
103
118
|
end
|
data/test/test_client.rb
CHANGED
@@ -191,4 +191,42 @@ class TestClient < Sidekiq::Test
|
|
191
191
|
conn.verify
|
192
192
|
end
|
193
193
|
end
|
194
|
+
|
195
|
+
describe 'Sidekiq::Worker#set' do
|
196
|
+
class SetWorker
|
197
|
+
include Sidekiq::Worker
|
198
|
+
sidekiq_options :queue => :foo, 'retry' => 12
|
199
|
+
end
|
200
|
+
|
201
|
+
def setup
|
202
|
+
Sidekiq.redis {|c| c.flushdb }
|
203
|
+
end
|
204
|
+
|
205
|
+
it 'allows option overrides' do
|
206
|
+
q = Sidekiq::Queue.new('bar')
|
207
|
+
assert_equal 0, q.size
|
208
|
+
assert SetWorker.set(queue: :bar).perform_async(1)
|
209
|
+
job = q.first
|
210
|
+
assert_equal 'bar', job['queue']
|
211
|
+
assert_equal 12, job['retry']
|
212
|
+
end
|
213
|
+
|
214
|
+
it 'handles symbols and strings' do
|
215
|
+
q = Sidekiq::Queue.new('bar')
|
216
|
+
assert_equal 0, q.size
|
217
|
+
assert SetWorker.set('queue' => 'bar', :retry => 11).perform_async(1)
|
218
|
+
job = q.first
|
219
|
+
assert_equal 'bar', job['queue']
|
220
|
+
assert_equal 11, job['retry']
|
221
|
+
|
222
|
+
q.clear
|
223
|
+
assert SetWorker.perform_async(1)
|
224
|
+
assert_equal 0, q.size
|
225
|
+
|
226
|
+
q = Sidekiq::Queue.new('foo')
|
227
|
+
job = q.first
|
228
|
+
assert_equal 'foo', job['queue']
|
229
|
+
assert_equal 12, job['retry']
|
230
|
+
end
|
231
|
+
end
|
194
232
|
end
|
@@ -0,0 +1,123 @@
|
|
1
|
+
# encoding: utf-8
|
2
|
+
require_relative 'helper'
|
3
|
+
require 'sidekiq/middleware/server/retry_jobs'
|
4
|
+
|
5
|
+
class TestRetryExhausted < Sidekiq::Test
|
6
|
+
describe 'sidekiq_retries_exhausted' do
|
7
|
+
class NewWorker
|
8
|
+
include Sidekiq::Worker
|
9
|
+
|
10
|
+
class_attribute :exhausted_called, :exhausted_message, :exhausted_exception
|
11
|
+
|
12
|
+
sidekiq_retries_exhausted do |msg, e|
|
13
|
+
self.exhausted_called = true
|
14
|
+
self.exhausted_message = msg
|
15
|
+
self.exhausted_exception = e
|
16
|
+
end
|
17
|
+
end
|
18
|
+
|
19
|
+
class OldWorker
|
20
|
+
include Sidekiq::Worker
|
21
|
+
|
22
|
+
class_attribute :exhausted_called, :exhausted_message, :exhausted_exception
|
23
|
+
|
24
|
+
sidekiq_retries_exhausted do |msg|
|
25
|
+
self.exhausted_called = true
|
26
|
+
self.exhausted_message = msg
|
27
|
+
end
|
28
|
+
end
|
29
|
+
|
30
|
+
def cleanup
|
31
|
+
[NewWorker, OldWorker].each do |worker_class|
|
32
|
+
worker_class.exhausted_called = nil
|
33
|
+
worker_class.exhausted_message = nil
|
34
|
+
worker_class.exhausted_exception = nil
|
35
|
+
end
|
36
|
+
end
|
37
|
+
|
38
|
+
before do
|
39
|
+
cleanup
|
40
|
+
end
|
41
|
+
|
42
|
+
after do
|
43
|
+
cleanup
|
44
|
+
end
|
45
|
+
|
46
|
+
def new_worker
|
47
|
+
@new_worker ||= NewWorker.new
|
48
|
+
end
|
49
|
+
|
50
|
+
def old_worker
|
51
|
+
@old_worker ||= OldWorker.new
|
52
|
+
end
|
53
|
+
|
54
|
+
def handler(options={})
|
55
|
+
@handler ||= Sidekiq::Middleware::Server::RetryJobs.new(options)
|
56
|
+
end
|
57
|
+
|
58
|
+
def job(options={})
|
59
|
+
@job ||= {'class' => 'Bob', 'args' => [1, 2, 'foo']}.merge(options)
|
60
|
+
end
|
61
|
+
|
62
|
+
it 'does not run exhausted block when job successful on first run' do
|
63
|
+
handler.call(new_worker, job('retry' => 2), 'default') do
|
64
|
+
# successful
|
65
|
+
end
|
66
|
+
|
67
|
+
refute NewWorker.exhausted_called?
|
68
|
+
end
|
69
|
+
|
70
|
+
it 'does not run exhausted block when job successful on last retry' do
|
71
|
+
handler.call(new_worker, job('retry_count' => 0, 'retry' => 1), 'default') do
|
72
|
+
# successful
|
73
|
+
end
|
74
|
+
|
75
|
+
refute NewWorker.exhausted_called?
|
76
|
+
end
|
77
|
+
|
78
|
+
it 'does not run exhausted block when retries not exhausted yet' do
|
79
|
+
assert_raises RuntimeError do
|
80
|
+
handler.call(new_worker, job('retry' => 1), 'default') do
|
81
|
+
raise 'kerblammo!'
|
82
|
+
end
|
83
|
+
end
|
84
|
+
|
85
|
+
refute NewWorker.exhausted_called?
|
86
|
+
end
|
87
|
+
|
88
|
+
it 'runs exhausted block when retries exhausted' do
|
89
|
+
assert_raises RuntimeError do
|
90
|
+
handler.call(new_worker, job('retry_count' => 0, 'retry' => 1), 'default') do
|
91
|
+
raise 'kerblammo!'
|
92
|
+
end
|
93
|
+
end
|
94
|
+
|
95
|
+
assert NewWorker.exhausted_called?
|
96
|
+
end
|
97
|
+
|
98
|
+
|
99
|
+
it 'passes message and exception to retries exhausted block' do
|
100
|
+
raised_error = assert_raises RuntimeError do
|
101
|
+
handler.call(new_worker, job('retry_count' => 0, 'retry' => 1), 'default') do
|
102
|
+
raise 'kerblammo!'
|
103
|
+
end
|
104
|
+
end
|
105
|
+
|
106
|
+
assert new_worker.exhausted_called?
|
107
|
+
assert_equal raised_error.message, new_worker.exhausted_message['error_message']
|
108
|
+
assert_equal raised_error, new_worker.exhausted_exception
|
109
|
+
end
|
110
|
+
|
111
|
+
it 'passes message to retries exhausted block' do
|
112
|
+
raised_error = assert_raises RuntimeError do
|
113
|
+
handler.call(old_worker, job('retry_count' => 0, 'retry' => 1), 'default') do
|
114
|
+
raise 'kerblammo!'
|
115
|
+
end
|
116
|
+
end
|
117
|
+
|
118
|
+
assert old_worker.exhausted_called?
|
119
|
+
assert_equal raised_error.message, old_worker.exhausted_message['error_message']
|
120
|
+
assert_equal nil, new_worker.exhausted_exception
|
121
|
+
end
|
122
|
+
end
|
123
|
+
end
|
data/test/test_scheduled.rb
CHANGED
@@ -90,7 +90,7 @@ class TestScheduled < Sidekiq::Test
|
|
90
90
|
it 'generates random intervals that target a configured average' do
|
91
91
|
with_sidekiq_option(:poll_interval_average, 10) do
|
92
92
|
i = 500
|
93
|
-
intervals = i
|
93
|
+
intervals = Array.new(i){ @poller.send(:random_poll_interval) }
|
94
94
|
|
95
95
|
assert intervals.all?{|x| x >= 5}
|
96
96
|
assert intervals.all?{|x| x <= 15}
|
data/test/test_testing_fake.rb
CHANGED
@@ -261,6 +261,16 @@ class TestTesting < Sidekiq::Test
|
|
261
261
|
assert_equal 1, SecondWorker.count
|
262
262
|
end
|
263
263
|
|
264
|
+
it 'drains jobs of workers with symbolized queue names' do
|
265
|
+
Sidekiq::Worker.jobs.clear
|
266
|
+
|
267
|
+
AltQueueWorker.perform_async(5,6)
|
268
|
+
assert_equal 1, AltQueueWorker.jobs.size
|
269
|
+
|
270
|
+
Sidekiq::Worker.drain_all
|
271
|
+
assert_equal 0, AltQueueWorker.jobs.size
|
272
|
+
end
|
273
|
+
|
264
274
|
it 'can execute a job' do
|
265
275
|
DirectWorker.execute_job(DirectWorker.new, [2, 3])
|
266
276
|
end
|
data/web/locales/ja.yml
CHANGED
@@ -6,8 +6,8 @@ ja:
|
|
6
6
|
Namespace: ネームスペース
|
7
7
|
Realtime: リアルタイム
|
8
8
|
History: 履歴
|
9
|
-
Busy:
|
10
|
-
Processed:
|
9
|
+
Busy: 実行中
|
10
|
+
Processed: 完了
|
11
11
|
Failed: 失敗
|
12
12
|
Scheduled: 予定
|
13
13
|
Retries: 再試行
|
@@ -35,25 +35,25 @@ ja:
|
|
35
35
|
RetryNow: 今すぐ再試行
|
36
36
|
LastRetry: 再試行履歴
|
37
37
|
OriginallyFailed: 失敗
|
38
|
-
AreYouSure:
|
38
|
+
AreYouSure: よろしいですか?
|
39
39
|
DeleteAll: 全て削除
|
40
40
|
RetryAll: 全て再試行
|
41
41
|
NoRetriesFound: 再試行するジョブはありません
|
42
42
|
Error: エラー
|
43
|
-
ErrorClass:
|
43
|
+
ErrorClass: エラークラス
|
44
44
|
ErrorMessage: エラーメッセージ
|
45
45
|
ErrorBacktrace: エラーバックトレース
|
46
46
|
GoBack: ← 戻る
|
47
|
-
NoScheduledFound:
|
47
|
+
NoScheduledFound: 予定されたジョブはありません
|
48
48
|
When: いつ
|
49
|
-
ScheduledJobs:
|
49
|
+
ScheduledJobs: 予定されたジョブ
|
50
50
|
idle: アイドル
|
51
51
|
active: アクティブ
|
52
52
|
Version: バージョン
|
53
53
|
Connections: 接続
|
54
|
-
MemoryUsage:
|
55
|
-
PeakMemoryUsage:
|
56
|
-
Uptime:
|
54
|
+
MemoryUsage: メモリー使用量
|
55
|
+
PeakMemoryUsage: 最大メモリー使用量
|
56
|
+
Uptime: 連続稼働時間 (日)
|
57
57
|
OneWeek: 1 週
|
58
58
|
OneMonth: 1 ヶ月
|
59
59
|
ThreeMonths: 3 ヶ月
|
data/web/views/busy.erb
CHANGED
@@ -30,6 +30,9 @@
|
|
30
30
|
<% process.labels.each do |label| %>
|
31
31
|
<span class="label label-info"><%= label %></span>
|
32
32
|
<% end %>
|
33
|
+
<% if process.stopping? %>
|
34
|
+
<span class="label label-danger">Quiet</span>
|
35
|
+
<% end %>
|
33
36
|
<br>
|
34
37
|
<b><%= "#{t('Queues')}: " %></b>
|
35
38
|
<%= process['queues'] * ", " %>
|
@@ -42,7 +45,9 @@
|
|
42
45
|
<form method="POST">
|
43
46
|
<%= csrf_tag %>
|
44
47
|
<input type="hidden" name="identity" value="<%= process['identity'] %>"/>
|
45
|
-
|
48
|
+
<% unless process.stopping? %>
|
49
|
+
<button class="btn btn-warn" type="submit" name="quiet" value="1"><%= t('Quiet') %></button>
|
50
|
+
<% end %>
|
46
51
|
<button class="btn btn-danger" type="submit" name="stop" value="1"><%= t('Stop') %></button>
|
47
52
|
</form>
|
48
53
|
</div>
|
data/web/views/retry.erb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: sidekiq
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 4.0
|
4
|
+
version: 4.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Mike Perham
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2016-01-
|
11
|
+
date: 2016-01-28 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: redis
|
@@ -185,6 +185,7 @@ files:
|
|
185
185
|
- bin/sidekiq
|
186
186
|
- bin/sidekiqctl
|
187
187
|
- bin/sidekiqload
|
188
|
+
- code_of_conduct.md
|
188
189
|
- lib/generators/sidekiq/templates/worker.rb.erb
|
189
190
|
- lib/generators/sidekiq/templates/worker_spec.rb.erb
|
190
191
|
- lib/generators/sidekiq/templates/worker_test.rb.erb
|
@@ -241,6 +242,7 @@ files:
|
|
241
242
|
- test/test_rails.rb
|
242
243
|
- test/test_redis_connection.rb
|
243
244
|
- test/test_retry.rb
|
245
|
+
- test/test_retry_exhausted.rb
|
244
246
|
- test/test_scheduled.rb
|
245
247
|
- test/test_scheduling.rb
|
246
248
|
- test/test_sidekiq.rb
|
@@ -392,6 +394,7 @@ test_files:
|
|
392
394
|
- test/test_rails.rb
|
393
395
|
- test/test_redis_connection.rb
|
394
396
|
- test/test_retry.rb
|
397
|
+
- test/test_retry_exhausted.rb
|
395
398
|
- test/test_scheduled.rb
|
396
399
|
- test/test_scheduling.rb
|
397
400
|
- test/test_sidekiq.rb
|