sidekiq 3.5.4 → 4.0.0
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of sidekiq might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/4.0-Upgrade.md +50 -0
- data/Changes.md +15 -3
- data/Ent-Changes.md +27 -0
- data/Gemfile +2 -1
- data/Pro-3.0-Upgrade.md +46 -0
- data/Pro-Changes.md +21 -0
- data/README.md +4 -4
- data/bin/sidekiqctl +8 -2
- data/bin/sidekiqload +20 -6
- data/lib/sidekiq.rb +24 -11
- data/lib/sidekiq/api.rb +2 -2
- data/lib/sidekiq/cli.rb +19 -29
- data/lib/sidekiq/client.rb +0 -5
- data/lib/sidekiq/fetch.rb +35 -111
- data/lib/sidekiq/launcher.rb +105 -46
- data/lib/sidekiq/manager.rb +77 -180
- data/lib/sidekiq/middleware/server/retry_jobs.rb +1 -1
- data/lib/sidekiq/processor.rb +119 -96
- data/lib/sidekiq/redis_connection.rb +23 -5
- data/lib/sidekiq/scheduled.rb +47 -26
- data/lib/sidekiq/testing.rb +84 -14
- data/lib/sidekiq/util.rb +7 -0
- data/lib/sidekiq/version.rb +1 -1
- data/lib/sidekiq/web_helpers.rb +8 -1
- data/sidekiq.gemspec +2 -2
- data/test/helper.rb +30 -5
- data/test/test_actors.rb +137 -0
- data/test/test_api.rb +395 -394
- data/test/test_fetch.rb +2 -57
- data/test/test_launcher.rb +80 -0
- data/test/test_manager.rb +13 -132
- data/test/test_middleware.rb +3 -5
- data/test/test_processor.rb +20 -57
- data/test/test_scheduled.rb +2 -2
- data/test/test_testing_fake.rb +64 -1
- data/web/assets/stylesheets/application.css +4 -0
- data/web/views/_footer.erb +2 -7
- metadata +30 -25
- data/lib/sidekiq/actor.rb +0 -39
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 154fd4d987e5f7c32fad42a20176d27e958bd49f
|
4
|
+
data.tar.gz: a33449473a2210e1f79e3f814c50f6fe982ff101
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 264d2c5a210a978ca8ce9f2a78962cc4f4d62966444788b7d942644881baec212133123f3434ffafdcc21d5bd9c0130eef007195a945f86771a33da5b2d1fbea
|
7
|
+
data.tar.gz: 6e651a3d38dd4518a3db01abd5febdb9a4c739d16873f2ce3ffa4998bd5f5a2285303437cc6f489cb43d032590bf741006d50170470c333ed31cfa62c1168178
|
data/4.0-Upgrade.md
ADDED
@@ -0,0 +1,50 @@
|
|
1
|
+
# Welcome to Sidekiq 4.0!
|
2
|
+
|
3
|
+
Sidekiq 4.0 contains a redesigned, more efficient core with less overhead per job.
|
4
|
+
See my blog for [an overview of Sidekiq 4's higher performance](http://www.mikeperham.com/2015/10/14/optimizing-sidekiq/).
|
5
|
+
|
6
|
+
## What's New
|
7
|
+
|
8
|
+
* Sidekiq no longer uses Celluloid. If your application code uses Celluloid,
|
9
|
+
you will need to pull it in yourself.
|
10
|
+
|
11
|
+
* `redis-namespace` has been removed from Sidekiq's gem dependencies. If
|
12
|
+
you want to use namespacing ([and I strongly urge you not to](http://www.mikeperham.com/2015/09/24/storing-data-with-redis/)), you'll need to add the gem to your Gemfile:
|
13
|
+
```ruby
|
14
|
+
gem 'redis-namespace'
|
15
|
+
```
|
16
|
+
|
17
|
+
* **Redis 2.8.0 or greater is required.** Redis 2.8 was released two years
|
18
|
+
ago and contains **many** useful features which Sidekiq couldn't
|
19
|
+
leverage until now. **Redis 3.0.3 or greater is recommended** for large
|
20
|
+
scale use.
|
21
|
+
|
22
|
+
* Jobs are now fetched from Redis in parallel, making Sidekiq more
|
23
|
+
resilient to high network latency. This means that Sidekiq requires
|
24
|
+
more Redis connections per process. You must have a minimum of
|
25
|
+
`concurrency + 2` connections in your pool or Sidekiq will exit.
|
26
|
+
When in doubt, let Sidekiq size the connection pool for you.
|
27
|
+
|
28
|
+
* There's a new testing API based off the `Sidekiq::Queues` namespace. All
|
29
|
+
assertions made against the Worker class still work as expected.
|
30
|
+
```ruby
|
31
|
+
assert_equal 0, Sidekiq::Queues["default"].size
|
32
|
+
HardWorker.perform_async("log")
|
33
|
+
assert_equal 1, Sidekiq::Queues["default"].size
|
34
|
+
assert_equal "log", Sidekiq::Queues["default"].first['args'][0]
|
35
|
+
Sidekiq::Queues.clear_all
|
36
|
+
```
|
37
|
+
|
38
|
+
## Upgrade
|
39
|
+
|
40
|
+
First, make sure you are using Redis 2.8 or greater. Next:
|
41
|
+
|
42
|
+
* Upgrade to the latest Sidekiq 3.x.
|
43
|
+
```ruby
|
44
|
+
gem 'sidekiq', '< 4'
|
45
|
+
```
|
46
|
+
* Fix any deprecation warnings you see.
|
47
|
+
* Upgrade to 4.x.
|
48
|
+
```ruby
|
49
|
+
gem 'sidekiq', '< 5'
|
50
|
+
```
|
data/Changes.md
CHANGED
@@ -1,10 +1,22 @@
|
|
1
1
|
# Sidekiq Changes
|
2
2
|
|
3
|
-
|
3
|
+
4.0.0.pre1
|
4
4
|
-----------
|
5
5
|
|
6
|
-
-
|
7
|
-
|
6
|
+
- Sidekiq's internals have been completely overhauled for performance
|
7
|
+
and to remove dependencies. This has resulted in major speedups, as
|
8
|
+
[detailed on my blog](http://www.mikeperham.com/2015/10/14/optimizing-sidekiq/).
|
9
|
+
- See the [4.0 upgrade notes](4.0-Upgrade.md) for more detail.
|
10
|
+
- There's a new testing API based off the `Sidekiq::Queues` namespace. All
|
11
|
+
assertions made against the Worker class still work as expected.
|
12
|
+
[#2659, brandonhilkert]
|
13
|
+
```ruby
|
14
|
+
assert_equal 0, Sidekiq::Queues["default"].size
|
15
|
+
HardWorker.perform_async("log")
|
16
|
+
assert_equal 1, Sidekiq::Queues["default"].size
|
17
|
+
assert_equal "log", Sidekiq::Queues["default"].first['args'][0]
|
18
|
+
Sidekiq::Queues.clear_all
|
19
|
+
```
|
8
20
|
|
9
21
|
3.5.3
|
10
22
|
-----------
|
data/Ent-Changes.md
CHANGED
@@ -3,6 +3,33 @@ Sidekiq Enterprise Changelog
|
|
3
3
|
|
4
4
|
Please see [http://sidekiq.org/](http://sidekiq.org/) for more details and how to buy.
|
5
5
|
|
6
|
+
1.0.0.pre1
|
7
|
+
----------
|
8
|
+
|
9
|
+
- Enterprise 1.x targets Sidekiq 4.x.
|
10
|
+
- Rewrite several features to remove Celluloid dependency. No
|
11
|
+
functional changes.
|
12
|
+
|
13
|
+
0.7.8
|
14
|
+
----------
|
15
|
+
|
16
|
+
- Fix `unique_for: false` [#2658]
|
17
|
+
|
18
|
+
|
19
|
+
0.7.7
|
20
|
+
----------
|
21
|
+
|
22
|
+
- Enterprise 0.x targets Sidekiq 3.x.
|
23
|
+
- Fix racy shutdown event which could lead to disappearing periodic
|
24
|
+
jobs, requires Sidekiq >= 3.5.3.
|
25
|
+
- Add new :leader event which is fired when a process gains leadership.
|
26
|
+
|
27
|
+
0.7.6
|
28
|
+
----------
|
29
|
+
|
30
|
+
- Redesign how overrated jobs are rescheduled to avoid creating new
|
31
|
+
jobs. [#2619]
|
32
|
+
|
6
33
|
0.7.5
|
7
34
|
----------
|
8
35
|
|
data/Gemfile
CHANGED
@@ -4,12 +4,12 @@ gemspec
|
|
4
4
|
gem 'rails', '~> 4.2'
|
5
5
|
gem 'simplecov'
|
6
6
|
gem 'minitest'
|
7
|
+
gem 'minitest-utils'
|
7
8
|
gem 'toxiproxy'
|
8
9
|
|
9
10
|
platforms :rbx do
|
10
11
|
gem 'rubysl', '~> 2.0' # if using anything in the ruby standard library
|
11
12
|
gem 'psych' # if using yaml
|
12
|
-
gem 'minitest' # if using minitest
|
13
13
|
gem 'rubinius-developer_tools' # if using any of coverage, debugger, profiler
|
14
14
|
end
|
15
15
|
|
@@ -19,6 +19,7 @@ end
|
|
19
19
|
|
20
20
|
platforms :mri do
|
21
21
|
gem 'pry-byebug'
|
22
|
+
gem 'ruby-prof'
|
22
23
|
end
|
23
24
|
|
24
25
|
platforms :jruby do
|
data/Pro-3.0-Upgrade.md
ADDED
@@ -0,0 +1,46 @@
|
|
1
|
+
# Welcome to Sidekiq Pro 3.0!
|
2
|
+
|
3
|
+
Sidekiq Pro 3.0 is designed to work with Sidekiq 4.0.
|
4
|
+
|
5
|
+
## What's New
|
6
|
+
|
7
|
+
* **Redis 2.8.0 or greater is required.** Redis 2.8 was released two years
|
8
|
+
ago and contains **many** useful features which Sidekiq couldn't
|
9
|
+
leverage until now. **Redis 3.0.3 or greater is recommended** for large
|
10
|
+
scale use.
|
11
|
+
|
12
|
+
* Sidekiq Pro no longer uses Celluloid. If your application code uses Celluloid,
|
13
|
+
you will need to pull it in yourself.
|
14
|
+
|
15
|
+
* Pausing and unpausing queues is now instantaneous, no more polling!
|
16
|
+
|
17
|
+
* Reliable fetch has been re-implemented due to the fetch changes in
|
18
|
+
Sidekiq 4.0.
|
19
|
+
|
20
|
+
* Support for platforms without persistent hostnames. Since reliable fetch
|
21
|
+
normally requires a persistent hostname, you may disable hostname usage on
|
22
|
+
platforms like Heroku and Docker:
|
23
|
+
```ruby
|
24
|
+
Sidekiq.configure_server do |config|
|
25
|
+
config.options[:ephemeral_hostname] = true
|
26
|
+
config.reliable_fetch!
|
27
|
+
end
|
28
|
+
```
|
29
|
+
This option is enabled automatically if Heroku's DYNO environment variable is present.
|
30
|
+
Without a persistent hostname, each Sidekiq process **must** have its own unique index.
|
31
|
+
|
32
|
+
* The old 'sidekiq/notifications' features have been removed.
|
33
|
+
|
34
|
+
## Upgrade
|
35
|
+
|
36
|
+
First, make sure you are using Redis 2.8 or greater. Next:
|
37
|
+
|
38
|
+
* Upgrade to the latest Sidekiq Pro 2.x.
|
39
|
+
```ruby
|
40
|
+
gem 'sidekiq-pro', '< 3'
|
41
|
+
```
|
42
|
+
* Fix any deprecation warnings you see.
|
43
|
+
* Upgrade to 3.x.
|
44
|
+
```ruby
|
45
|
+
gem 'sidekiq-pro', '< 4'
|
46
|
+
```
|
data/Pro-Changes.md
CHANGED
@@ -3,6 +3,27 @@ Sidekiq Pro Changelog
|
|
3
3
|
|
4
4
|
Please see [http://sidekiq.org/](http://sidekiq.org/) for more details and how to buy.
|
5
5
|
|
6
|
+
3.0.0.pre2
|
7
|
+
-----------
|
8
|
+
|
9
|
+
- Fix excessive connection usage by reliable fetch.
|
10
|
+
|
11
|
+
3.0.0.pre1
|
12
|
+
-----------
|
13
|
+
|
14
|
+
- See the [Pro 3.0 release notes](Pro-3.0-Upgrade.md).
|
15
|
+
|
16
|
+
2.1.3
|
17
|
+
-----------
|
18
|
+
|
19
|
+
- Don't enable strict priority if using weighted queueing like `-q a,1 -q b,1`
|
20
|
+
- Safer JSON mangling in Lua [#2639]
|
21
|
+
|
22
|
+
2.1.2
|
23
|
+
-----------
|
24
|
+
|
25
|
+
- Lock Sidekiq Pro 2.x to Sidekiq 3.x.
|
26
|
+
|
6
27
|
2.1.1
|
7
28
|
-----------
|
8
29
|
|
data/README.md
CHANGED
@@ -32,7 +32,7 @@ are untested but might work fine. MRI 1.9 is no longer supported.
|
|
32
32
|
|
33
33
|
All Rails releases starting from 3.2 are officially supported.
|
34
34
|
|
35
|
-
Redis 2.
|
35
|
+
Redis 2.8 or greater is required.
|
36
36
|
|
37
37
|
|
38
38
|
Installation
|
@@ -66,7 +66,7 @@ Please see the [sidekiq wiki](https://github.com/mperham/sidekiq/wiki) for the o
|
|
66
66
|
[mperham/sidekiq on Gitter](https://gitter.im/mperham/sidekiq) is dedicated to this project,
|
67
67
|
but bug reports or feature requests suggestions should still go through [issues on Github](https://github.com/mperham/sidekiq/issues). Release announcements are made to the [@sidekiq](https://twitter.com/sidekiq) Twitter account.
|
68
68
|
|
69
|
-
You may also find useful a [
|
69
|
+
You may also find useful a [Reddit area](https://reddit.com/r/sidekiq) dedicated to Sidekiq discussion and [a Sidekiq tag](https://stackoverflow.com/questions/tagged/sidekiq) on Stack Overflow.
|
70
70
|
|
71
71
|
|
72
72
|
Problems?
|
@@ -75,14 +75,14 @@ Problems?
|
|
75
75
|
**Please do not directly email any Sidekiq committers with questions or problems.** A community is best served when discussions are held in public.
|
76
76
|
|
77
77
|
If you have a problem, please review the [FAQ](https://github.com/mperham/sidekiq/wiki/FAQ) and [Troubleshooting](https://github.com/mperham/sidekiq/wiki/Problems-and-Troubleshooting) wiki pages. Searching the issues for your problem is also a good idea. If that doesn't help, feel free to email the Sidekiq mailing list, chat in Gitter, or open a new issue.
|
78
|
-
|
78
|
+
StackOverflow or Reddit is the preferred place to ask questions on usage. If you are encountering what you think is a bug, please open an issue.
|
79
79
|
|
80
80
|
|
81
81
|
Thanks
|
82
82
|
-----------------
|
83
83
|
|
84
84
|
Sidekiq stays fast by using the [JProfiler java profiler](http://www.ej-technologies.com/products/jprofiler/overview.html) to find and fix
|
85
|
-
performance problems on JRuby. Unfortunately MRI does not have good
|
85
|
+
performance problems on JRuby. Unfortunately MRI does not have good multithreaded profiling tools.
|
86
86
|
|
87
87
|
|
88
88
|
License
|
data/bin/sidekiqctl
CHANGED
@@ -41,9 +41,13 @@ class Sidekiqctl
|
|
41
41
|
end
|
42
42
|
|
43
43
|
def fetch_process
|
44
|
-
Process.
|
44
|
+
Process.kill(0, pid)
|
45
45
|
rescue Errno::ESRCH
|
46
46
|
done "Process doesn't exist", :error
|
47
|
+
# We were not allowed to send a signal, but the process must have existed
|
48
|
+
# when Process.kill() was called.
|
49
|
+
rescue Errno::EPERM
|
50
|
+
return pid
|
47
51
|
end
|
48
52
|
|
49
53
|
def done(msg, error = nil)
|
@@ -67,10 +71,12 @@ class Sidekiqctl
|
|
67
71
|
`kill -TERM #{pid}`
|
68
72
|
kill_timeout.times do
|
69
73
|
begin
|
70
|
-
Process.
|
74
|
+
Process.kill(0, pid)
|
71
75
|
rescue Errno::ESRCH
|
72
76
|
FileUtils.rm_f pidfile
|
73
77
|
done 'Sidekiq shut down gracefully.'
|
78
|
+
rescue Errno::EPERM
|
79
|
+
done 'Not permitted to shut down Sidekiq.'
|
74
80
|
end
|
75
81
|
sleep 1
|
76
82
|
end
|
data/bin/sidekiqload
CHANGED
@@ -3,21 +3,35 @@
|
|
3
3
|
# Quiet some warnings we see when running in warning mode:
|
4
4
|
# RUBYOPT=-w bundle exec sidekiq
|
5
5
|
$TESTING = false
|
6
|
-
$CELLULOID_DEBUG = false
|
7
6
|
|
8
|
-
require '
|
9
|
-
|
7
|
+
#require 'ruby-prof'
|
8
|
+
Bundler.require(:default)
|
9
|
+
|
10
10
|
require_relative '../lib/sidekiq/cli'
|
11
11
|
require_relative '../lib/sidekiq/launcher'
|
12
|
-
Celluloid.logger = nil
|
13
12
|
|
14
13
|
include Sidekiq::Util
|
15
14
|
|
15
|
+
# brew tap shopify/shopify
|
16
|
+
# brew install toxiproxy
|
17
|
+
# gem install toxiproxy
|
18
|
+
require 'toxiproxy'
|
19
|
+
# simulate a non-localhost network for realer-world conditions.
|
20
|
+
# adding 1ms of network latency has an ENORMOUS impact on benchmarks
|
21
|
+
Toxiproxy.populate([{
|
22
|
+
"name": "redis",
|
23
|
+
"listen": "127.0.0.1:6380",
|
24
|
+
"upstream": "127.0.0.1:6379"
|
25
|
+
}])
|
26
|
+
|
27
|
+
|
16
28
|
Sidekiq.configure_server do |config|
|
17
29
|
config.redis = { db: 13, port: 6380 }
|
30
|
+
#config.redis = { db: 13 }
|
18
31
|
config.options[:queues] << 'default'
|
19
32
|
config.logger.level = Logger::ERROR
|
20
33
|
config.average_scheduled_poll_interval = 2
|
34
|
+
config.reliable! if defined?(Sidekiq::Pro)
|
21
35
|
end
|
22
36
|
|
23
37
|
class LoadWorker
|
@@ -68,8 +82,7 @@ def handle_signal(launcher, sig)
|
|
68
82
|
raise Interrupt
|
69
83
|
when 'USR1'
|
70
84
|
Sidekiq.logger.info "Received USR1, no longer accepting new work"
|
71
|
-
launcher.
|
72
|
-
#fire_event(:quiet, true)
|
85
|
+
launcher.quiet
|
73
86
|
when 'USR2'
|
74
87
|
if Sidekiq.options[:logfile]
|
75
88
|
Sidekiq.logger.info "Received USR2, reopening log file"
|
@@ -129,6 +142,7 @@ end
|
|
129
142
|
begin
|
130
143
|
#RubyProf::exclude_threads = [ Monitoring ]
|
131
144
|
#RubyProf.start
|
145
|
+
fire_event(:startup)
|
132
146
|
Sidekiq.logger.error "Simulating 1ms of latency between Sidekiq and redis"
|
133
147
|
Toxiproxy[:redis].downstream(:latency, latency: 1).apply do
|
134
148
|
launcher = Sidekiq::Launcher.new(Sidekiq.options)
|
data/lib/sidekiq.rb
CHANGED
@@ -110,11 +110,25 @@ module Sidekiq
|
|
110
110
|
end
|
111
111
|
|
112
112
|
def self.server_middleware
|
113
|
-
@server_chain ||=
|
113
|
+
@server_chain ||= default_server_middleware
|
114
114
|
yield @server_chain if block_given?
|
115
115
|
@server_chain
|
116
116
|
end
|
117
117
|
|
118
|
+
def self.default_server_middleware
|
119
|
+
require 'sidekiq/middleware/server/retry_jobs'
|
120
|
+
require 'sidekiq/middleware/server/logging'
|
121
|
+
|
122
|
+
Middleware::Chain.new do |m|
|
123
|
+
m.add Middleware::Server::Logging
|
124
|
+
m.add Middleware::Server::RetryJobs
|
125
|
+
if defined?(::ActiveRecord::Base)
|
126
|
+
require 'sidekiq/middleware/server/active_record'
|
127
|
+
m.add Sidekiq::Middleware::Server::ActiveRecord
|
128
|
+
end
|
129
|
+
end
|
130
|
+
end
|
131
|
+
|
118
132
|
def self.default_worker_options=(hash)
|
119
133
|
@default_worker_options = default_worker_options.merge(hash.stringify_keys)
|
120
134
|
end
|
@@ -139,16 +153,6 @@ module Sidekiq
|
|
139
153
|
Sidekiq::Logging.logger = log
|
140
154
|
end
|
141
155
|
|
142
|
-
# When set, overrides Sidekiq.options[:average_scheduled_poll_interval] and sets
|
143
|
-
# the average interval that this process will delay before checking for
|
144
|
-
# scheduled jobs or job retries that are ready to run.
|
145
|
-
#
|
146
|
-
# See sidekiq/scheduled.rb for an in-depth explanation of this value
|
147
|
-
def self.poll_interval=(interval)
|
148
|
-
$stderr.puts "DEPRECATION: `config.poll_interval = #{interval}` will be removed in Sidekiq 4. Please update to `config.average_scheduled_poll_interval = #{interval}`."
|
149
|
-
self.options[:poll_interval_average] = interval
|
150
|
-
end
|
151
|
-
|
152
156
|
# How frequently Redis should be checked by a random Sidekiq process for
|
153
157
|
# scheduled and retriable jobs. Each individual process will take turns by
|
154
158
|
# waiting some multiple of this value.
|
@@ -182,6 +186,15 @@ module Sidekiq
|
|
182
186
|
raise ArgumentError, "Invalid event name: #{event}" unless options[:lifecycle_events].key?(event)
|
183
187
|
options[:lifecycle_events][event] << block
|
184
188
|
end
|
189
|
+
|
190
|
+
# We are shutting down Sidekiq but what about workers that
|
191
|
+
# are working on some long job? This error is
|
192
|
+
# raised in workers that have not finished within the hard
|
193
|
+
# timeout limit. This is needed to rollback db transactions,
|
194
|
+
# otherwise Ruby's Thread#kill will commit. See #377.
|
195
|
+
# DO NOT RESCUE THIS ERROR IN YOUR WORKERS
|
196
|
+
class Shutdown < Interrupt; end
|
197
|
+
|
185
198
|
end
|
186
199
|
|
187
200
|
require 'sidekiq/extensions/class_methods'
|
data/lib/sidekiq/api.rb
CHANGED
@@ -225,7 +225,7 @@ module Sidekiq
|
|
225
225
|
page = 0
|
226
226
|
page_size = 50
|
227
227
|
|
228
|
-
|
228
|
+
while true do
|
229
229
|
range_start = page * page_size - deleted_size
|
230
230
|
range_end = page * page_size - deleted_size + (page_size - 1)
|
231
231
|
entries = Sidekiq.redis do |conn|
|
@@ -488,7 +488,7 @@ module Sidekiq
|
|
488
488
|
page = -1
|
489
489
|
page_size = 50
|
490
490
|
|
491
|
-
|
491
|
+
while true do
|
492
492
|
range_start = page * page_size + offset_size
|
493
493
|
range_end = page * page_size + offset_size + (page_size - 1)
|
494
494
|
elements = Sidekiq.redis do |conn|
|
data/lib/sidekiq/cli.rb
CHANGED
@@ -11,18 +11,18 @@ require 'sidekiq'
|
|
11
11
|
require 'sidekiq/util'
|
12
12
|
|
13
13
|
module Sidekiq
|
14
|
-
# We are shutting down Sidekiq but what about workers that
|
15
|
-
# are working on some long job? This error is
|
16
|
-
# raised in workers that have not finished within the hard
|
17
|
-
# timeout limit. This is needed to rollback db transactions,
|
18
|
-
# otherwise Ruby's Thread#kill will commit. See #377.
|
19
|
-
# DO NOT RESCUE THIS ERROR.
|
20
|
-
class Shutdown < Interrupt; end
|
21
|
-
|
22
14
|
class CLI
|
23
15
|
include Util
|
24
16
|
include Singleton unless $TESTING
|
25
17
|
|
18
|
+
PROCTITLES = [
|
19
|
+
proc { 'sidekiq'.freeze },
|
20
|
+
proc { Sidekiq::VERSION },
|
21
|
+
proc { |me, data| data['tag'] },
|
22
|
+
proc { |me, data| "[#{Processor::WORKER_STATE.size} of #{data['concurrency']} busy]" },
|
23
|
+
proc { |me, data| "stopping" if me.stopping? },
|
24
|
+
]
|
25
|
+
|
26
26
|
# Used for CLI testing
|
27
27
|
attr_accessor :code
|
28
28
|
attr_accessor :launcher
|
@@ -40,7 +40,6 @@ module Sidekiq
|
|
40
40
|
validate!
|
41
41
|
daemonize
|
42
42
|
write_pid
|
43
|
-
load_celluloid
|
44
43
|
end
|
45
44
|
|
46
45
|
# Code within this method is not tested because it alters
|
@@ -66,17 +65,21 @@ module Sidekiq
|
|
66
65
|
logger.info Sidekiq::LICENSE
|
67
66
|
logger.info "Upgrade to Sidekiq Pro for more features and support: http://sidekiq.org" unless defined?(::Sidekiq::Pro)
|
68
67
|
|
68
|
+
Sidekiq.redis do |conn|
|
69
|
+
# touch the connection pool so it is created before we
|
70
|
+
# fire startup and start multithreading.
|
71
|
+
ver = conn.info['redis_version']
|
72
|
+
raise "You are using Redis v#{ver}, Sidekiq requires Redis v2.8.0 or greater" if ver < '2.8'
|
73
|
+
end
|
74
|
+
|
75
|
+
# Before this point, the process is initializing with just the main thread.
|
76
|
+
# Starting here the process will now have multiple threads running.
|
69
77
|
fire_event(:startup)
|
70
78
|
|
71
79
|
logger.debug {
|
72
80
|
"Middleware: #{Sidekiq.server_middleware.map(&:klass).join(', ')}"
|
73
81
|
}
|
74
82
|
|
75
|
-
Sidekiq.redis do |conn|
|
76
|
-
# touch the connection pool so it is created before we
|
77
|
-
# launch the actors.
|
78
|
-
end
|
79
|
-
|
80
83
|
if !options[:daemon]
|
81
84
|
logger.info 'Starting processing, hit Ctrl-C to stop'
|
82
85
|
end
|
@@ -96,6 +99,7 @@ module Sidekiq
|
|
96
99
|
launcher.stop
|
97
100
|
# Explicitly exit so busy Processor threads can't block
|
98
101
|
# process shutdown.
|
102
|
+
logger.info "Bye!"
|
99
103
|
exit(0)
|
100
104
|
end
|
101
105
|
end
|
@@ -129,8 +133,7 @@ module Sidekiq
|
|
129
133
|
raise Interrupt
|
130
134
|
when 'USR1'
|
131
135
|
Sidekiq.logger.info "Received USR1, no longer accepting new work"
|
132
|
-
launcher.
|
133
|
-
fire_event(:quiet, true)
|
136
|
+
launcher.quiet
|
134
137
|
when 'USR2'
|
135
138
|
if Sidekiq.options[:logfile]
|
136
139
|
Sidekiq.logger.info "Received USR2, reopening log file"
|
@@ -159,19 +162,6 @@ module Sidekiq
|
|
159
162
|
end
|
160
163
|
end
|
161
164
|
|
162
|
-
def load_celluloid
|
163
|
-
raise "Celluloid cannot be required until here, or it will break Sidekiq's daemonization" if defined?(::Celluloid) && options[:daemon]
|
164
|
-
|
165
|
-
# Celluloid can't be loaded until after we've daemonized
|
166
|
-
# because it spins up threads and creates locks which get
|
167
|
-
# into a very bad state if forked.
|
168
|
-
require 'celluloid/current'
|
169
|
-
Celluloid.logger = (options[:verbose] ? Sidekiq.logger : nil)
|
170
|
-
|
171
|
-
require 'sidekiq/manager'
|
172
|
-
require 'sidekiq/scheduled'
|
173
|
-
end
|
174
|
-
|
175
165
|
def daemonize
|
176
166
|
return unless options[:daemon]
|
177
167
|
|