resqued 0.9.0 → 0.11.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +5 -5
- data/CHANGES.md +23 -0
- data/docs/signals.md +4 -0
- data/exe/resqued +41 -22
- data/lib/resqued.rb +5 -5
- data/lib/resqued/config.rb +7 -7
- data/lib/resqued/config/after_fork.rb +1 -1
- data/lib/resqued/config/base.rb +1 -1
- data/lib/resqued/config/before_fork.rb +1 -1
- data/lib/resqued/config/worker.rb +13 -13
- data/lib/resqued/daemon.rb +1 -0
- data/lib/resqued/exec_on_hup.rb +43 -0
- data/lib/resqued/listener.rb +51 -48
- data/lib/resqued/listener_pool.rb +97 -0
- data/lib/resqued/listener_proxy.rb +40 -31
- data/lib/resqued/listener_state.rb +8 -0
- data/lib/resqued/logging.rb +15 -8
- data/lib/resqued/master.rb +94 -98
- data/lib/resqued/master_state.rb +73 -0
- data/lib/resqued/procline_version.rb +2 -2
- data/lib/resqued/sleepy.rb +6 -4
- data/lib/resqued/test_case.rb +3 -3
- data/lib/resqued/version.rb +1 -1
- data/lib/resqued/worker.rb +21 -14
- data/spec/fixtures/test_case_after_fork_raises.rb +5 -2
- data/spec/fixtures/test_case_before_fork_raises.rb +4 -1
- data/spec/fixtures/test_case_environment.rb +3 -1
- data/spec/integration/listener_still_starting_spec.rb +24 -0
- data/spec/integration/master_inherits_child_spec.rb +85 -0
- data/spec/integration/restart_spec.rb +21 -0
- data/spec/resqued/backoff_spec.rb +27 -27
- data/spec/resqued/config/fork_event_spec.rb +8 -8
- data/spec/resqued/config/worker_spec.rb +68 -55
- data/spec/resqued/config_spec.rb +6 -6
- data/spec/resqued/sleepy_spec.rb +10 -11
- data/spec/resqued/test_case_spec.rb +7 -7
- data/spec/spec_helper.rb +6 -1
- data/spec/support/custom_matchers.rb +10 -2
- data/spec/support/extra-child-shim +6 -0
- data/spec/support/resqued_integration_helpers.rb +50 -0
- data/spec/support/resqued_path.rb +11 -0
- metadata +57 -36
- data/exe/resqued-listener +0 -6
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
|
-
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: 606d7e3fb51d8595ce4a8e4fe7bdb28fc3aa73323652cbb98f9e0db19e982960
|
4
|
+
data.tar.gz: 87d0126e0b4f6e804a63d680d6946aa83621d19ddfa22c9427772d84eb9208df
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: abcd739a5f7d36a1e980b055e90e36cd63d0303264126f35a27ef260717022fda2ddd23630b13a954d93cb2f038067ae9d2579347b5ca1a48a19ab18528d5a2e
|
7
|
+
data.tar.gz: 82b66d4c943da3c73e5ae0459197fad97931770eeb3d6acd77b556f69d285e2c8666823b2f9fd563d25337974012d81b5e8d4f2f64b5f39d835e7c8dbada409c
|
data/CHANGES.md
CHANGED
@@ -1,5 +1,28 @@
|
|
1
1
|
Starting with version 0.6.1, resqued uses semantic versioning to indicate incompatibilities between the master process, listener process, and configuration.
|
2
2
|
|
3
|
+
v0.11.0
|
4
|
+
-------
|
5
|
+
* Ignore SIGHUP in Listener and Worker processes. (#61)
|
6
|
+
|
7
|
+
v0.10.3
|
8
|
+
-------
|
9
|
+
* Fix a timing related crash during reload. (#60)
|
10
|
+
|
11
|
+
v0.10.2
|
12
|
+
-------
|
13
|
+
* Shut down cleanly even if there are other stray child processes of the master. (#59)
|
14
|
+
|
15
|
+
v0.10.1
|
16
|
+
-------
|
17
|
+
* Avoid deadlock if a listener stops responding. (#58)
|
18
|
+
* When using 'percent' for a queue in a worker pool, always assign at least one worker. (#57)
|
19
|
+
|
20
|
+
v0.10.0
|
21
|
+
-------
|
22
|
+
* Master process restarts itself (#51), so that it doesn't continue running stale code indefinitely. This will help with the rollout process when changes like #50 are introduced, so that the master process will catch up. The risk of this is that the master process might not be able to be restarted, which would lead to it crashing. The mostly likely way for that to happen is if you try to roll back your version of resqued to 0.9.0 or earlier. If you need to do that, ensure that your process monitor (systemd, god, etc.) is able to restart the master process. You can disable the new behavior by passing `--no-exec-on-hup`.
|
23
|
+
* Added rubocop. (#52)
|
24
|
+
* Changed supported (read: tested in CI) ruby versions from [2.0 .. 2.3] to [2.3 .. 2.6].
|
25
|
+
|
3
26
|
v0.9.0
|
4
27
|
------
|
5
28
|
* Avoid Errno::E2BIG on SIGHUP when there are lots of workers and lots of queues per worker. (#50) This changes the format of an env var that master passes to listener. Old and new versions won't crash, but they won't be able to communicate about currenly running workers.
|
data/docs/signals.md
CHANGED
@@ -34,6 +34,8 @@ The Listener process forwards `SIGCONT` to all of its workers.
|
|
34
34
|
|
35
35
|
The Listener process handles `SIGINT`, `SIGTERM`, and `SIGQUIT`. When it receives one of these signals, it goes into shutdown mode. It sends the received signal to all of its workers. When all workers have exited, the Listener process exits.
|
36
36
|
|
37
|
+
The Listener process handles `SIGHUP` and does nothing. This makes it easier to reload resqued in a docker container, since many container platforms will send a requested signal to all processes in the container.
|
38
|
+
|
37
39
|
## Worker
|
38
40
|
|
39
41
|
The Worker process uses resque's signal handling. Resque 1.23.0 handles the following signals:
|
@@ -44,3 +46,5 @@ The Worker process uses resque's signal handling. Resque 1.23.0 handles the foll
|
|
44
46
|
* `USR1`: Kill the forked child immediately, continue processing jobs.
|
45
47
|
* `USR2`: Don't process any new jobs
|
46
48
|
* `CONT`: Start processing jobs again after a USR2
|
49
|
+
|
50
|
+
Resqued leaves a handler for `HUP` in place that does nothing. This makes it easier to reload resqued in a docker container, since many container platforms will send a requested signal to all processes in the container.
|
data/exe/resqued
CHANGED
@@ -1,74 +1,93 @@
|
|
1
1
|
#!/usr/bin/env ruby
|
2
2
|
|
3
|
-
if ARGV[0] ==
|
4
|
-
require
|
3
|
+
if ARGV[0] == "listener"
|
4
|
+
require "resqued/listener"
|
5
5
|
Resqued::Listener.exec!
|
6
6
|
exit 0
|
7
7
|
end
|
8
8
|
|
9
|
-
require
|
9
|
+
require "optparse"
|
10
10
|
|
11
|
-
options = {}
|
11
|
+
options = { exec_on_hup: true }
|
12
12
|
daemonize = false
|
13
13
|
test = false
|
14
14
|
|
15
|
-
opts = OptionParser.new do |opts|
|
15
|
+
opts = OptionParser.new do |opts| # rubocop: disable Lint/ShadowingOuterLocalVariable
|
16
16
|
opts.banner = "Usage: resqued [options] resqued-config-files..."
|
17
17
|
|
18
|
-
opts.on
|
18
|
+
opts.on "-h", "--help", "Show this message" do
|
19
19
|
puts opts
|
20
20
|
exit
|
21
21
|
end
|
22
22
|
|
23
|
-
opts.on
|
24
|
-
require
|
23
|
+
opts.on "-v", "--version", "Show the version" do
|
24
|
+
require "resqued/version"
|
25
25
|
puts Resqued::VERSION
|
26
26
|
exit
|
27
27
|
end
|
28
28
|
|
29
|
-
opts.on
|
29
|
+
opts.on "--test", "Report which worker would start" do
|
30
30
|
test = true
|
31
31
|
end
|
32
32
|
|
33
|
-
opts.on
|
33
|
+
opts.on "--fast-exit", "Exit quickly on SIGQUIT, SIGTERM" do
|
34
34
|
options[:fast_exit] = true
|
35
35
|
end
|
36
36
|
|
37
|
-
opts.on
|
37
|
+
opts.on "-p", "--pidfile PIDFILE", "Store the pid of the master process in PIDFILE" do |v|
|
38
38
|
options[:master_pidfile] = v
|
39
39
|
end
|
40
40
|
|
41
|
-
opts.on
|
42
|
-
require
|
41
|
+
opts.on "-l", "--logfile LOGFILE", "Write output to LOGFILE instead of stdout" do |v|
|
42
|
+
require "resqued/logging"
|
43
43
|
Resqued::Logging.log_file = v
|
44
44
|
end
|
45
45
|
|
46
|
-
opts.on
|
46
|
+
opts.on "-D", "--daemonize", "Run daemonized in the background" do
|
47
47
|
daemonize = true
|
48
48
|
end
|
49
|
+
|
50
|
+
opts.on "--no-exec-on-hup", "Do not re-exec the master process on SIGHUP" do
|
51
|
+
options[:exec_on_hup] = false
|
52
|
+
end
|
53
|
+
|
54
|
+
opts.on "--replace FILE", "(internal)" do |v|
|
55
|
+
options[:master_state] = v
|
56
|
+
end
|
49
57
|
end
|
50
58
|
|
51
59
|
opts.parse!
|
52
60
|
options[:config_paths] = ARGV
|
53
61
|
|
54
|
-
|
55
|
-
|
56
|
-
|
62
|
+
def require_config_paths!(options, opts)
|
63
|
+
if options[:config_paths].empty?
|
64
|
+
puts opts
|
65
|
+
exit 1
|
66
|
+
end
|
57
67
|
end
|
58
68
|
|
59
69
|
if test
|
60
|
-
|
70
|
+
require_config_paths! options, opts
|
71
|
+
require "resqued/config"
|
61
72
|
workers = Resqued::Config.new(options[:config_paths]).build_workers
|
62
73
|
puts "Workers defined in #{options[:config_paths].join(' ')}"
|
63
74
|
workers.each_with_index do |worker, index|
|
64
75
|
puts "#{index + 1}: #{worker.queues.join(',')}"
|
65
76
|
end
|
66
77
|
else
|
67
|
-
require
|
68
|
-
Resqued.
|
69
|
-
|
78
|
+
require "resqued"
|
79
|
+
state = Resqued::MasterState.new
|
80
|
+
if options[:master_state]
|
81
|
+
Resqued::Logging.logger.info "Resuming master from #{options[:master_state]}"
|
82
|
+
Resqued::ExecOnHUP.restore_state(state, options[:master_state])
|
83
|
+
else
|
84
|
+
require_config_paths! options, opts
|
85
|
+
Resqued.capture_start_ctx!
|
86
|
+
state.init(options)
|
87
|
+
end
|
88
|
+
resqued = Resqued::Master.new(state)
|
70
89
|
if daemonize
|
71
|
-
require
|
90
|
+
require "resqued/daemon"
|
72
91
|
resqued = Resqued::Daemon.new(resqued)
|
73
92
|
end
|
74
93
|
resqued.run
|
data/lib/resqued.rb
CHANGED
@@ -1,12 +1,12 @@
|
|
1
|
-
require
|
2
|
-
require
|
1
|
+
require "resqued/master"
|
2
|
+
require "resqued/version"
|
3
3
|
|
4
4
|
module Resqued
|
5
|
-
START_CTX = {}
|
5
|
+
START_CTX = {} # rubocop: disable Style/MutableConstant
|
6
6
|
|
7
7
|
def self.capture_start_ctx!
|
8
|
-
START_CTX[
|
9
|
-
START_CTX[
|
8
|
+
START_CTX["$0"] = $0.dup
|
9
|
+
START_CTX["pwd"] =
|
10
10
|
begin
|
11
11
|
env_pwd = ENV["PWD"]
|
12
12
|
env_pwd_stat = File.stat env_pwd
|
data/lib/resqued/config.rb
CHANGED
@@ -1,6 +1,6 @@
|
|
1
|
-
require
|
2
|
-
require
|
3
|
-
require
|
1
|
+
require "resqued/config/after_fork"
|
2
|
+
require "resqued/config/before_fork"
|
3
|
+
require "resqued/config/worker"
|
4
4
|
|
5
5
|
module Resqued
|
6
6
|
module Config
|
@@ -14,22 +14,22 @@ module Resqued
|
|
14
14
|
# Does the things that the config file says to do.
|
15
15
|
class Configuration
|
16
16
|
def initialize(config_paths)
|
17
|
-
@config_data = config_paths.map { |path| {:
|
17
|
+
@config_data = config_paths.map { |path| { content: File.read(path), path: path } }
|
18
18
|
end
|
19
19
|
|
20
20
|
# Public: Performs the `before_fork` action from the config.
|
21
21
|
def before_fork(resqued)
|
22
|
-
Resqued::Config::BeforeFork.new(:
|
22
|
+
Resqued::Config::BeforeFork.new(resqued: resqued).apply_all(@config_data)
|
23
23
|
end
|
24
24
|
|
25
25
|
# Public: Performs the `after_fork` action from the config.
|
26
26
|
def after_fork(worker)
|
27
|
-
Resqued::Config::AfterFork.new(:
|
27
|
+
Resqued::Config::AfterFork.new(worker: worker).apply_all(@config_data)
|
28
28
|
end
|
29
29
|
|
30
30
|
# Public: Builds the workers specified in the config.
|
31
31
|
def build_workers
|
32
|
-
Resqued::Config::Worker.new(:
|
32
|
+
Resqued::Config::Worker.new(config: self).apply_all(@config_data)
|
33
33
|
end
|
34
34
|
end
|
35
35
|
end
|
data/lib/resqued/config/base.rb
CHANGED
@@ -1,5 +1,5 @@
|
|
1
|
-
require
|
2
|
-
require
|
1
|
+
require "resqued/config/base"
|
2
|
+
require "resqued/worker"
|
3
3
|
|
4
4
|
module Resqued
|
5
5
|
module Config
|
@@ -21,9 +21,9 @@ module Resqued
|
|
21
21
|
def worker(*queues)
|
22
22
|
options = queues.last.is_a?(Hash) ? queues.pop.dup : {}
|
23
23
|
queues = queues.flatten
|
24
|
-
queues = [
|
24
|
+
queues = ["*"] if queues.empty?
|
25
25
|
queues = queues.shuffle if options.delete(:shuffle_queues)
|
26
|
-
@workers << @worker_class.new(options.merge(@worker_options).merge(:
|
26
|
+
@workers << @worker_class.new(options.merge(@worker_options).merge(queues: queues))
|
27
27
|
end
|
28
28
|
|
29
29
|
# DSL: Set up a pool of workers. Define queues for the members of the pool with `queue`.
|
@@ -81,15 +81,16 @@ module Resqued
|
|
81
81
|
# on the concurrency values established and the total number of workers.
|
82
82
|
def build_pool_workers!
|
83
83
|
return unless @pool_size
|
84
|
+
|
84
85
|
queues = _fixed_concurrency_queues
|
85
86
|
1.upto(@pool_size) do |worker_num|
|
86
|
-
queue_names = queues
|
87
|
-
|
88
|
-
|
87
|
+
queue_names = queues
|
88
|
+
.select { |_name, concurrency| concurrency >= worker_num }
|
89
|
+
.map { |name, _concurrency| name }
|
89
90
|
if queue_names.any?
|
90
91
|
worker(queue_names, @pool_options)
|
91
92
|
else
|
92
|
-
worker(
|
93
|
+
worker("*", @pool_options)
|
93
94
|
end
|
94
95
|
end
|
95
96
|
end
|
@@ -106,13 +107,12 @@ module Resqued
|
|
106
107
|
# values (between 0.0 and 1.0). The value may also be nil, in which case the
|
107
108
|
# maximum worker_processes value is returned.
|
108
109
|
def _translate_concurrency_value(value)
|
109
|
-
|
110
|
-
when value.nil?
|
110
|
+
if value.nil?
|
111
111
|
@pool_size
|
112
|
-
|
112
|
+
elsif value.is_a?(1.class)
|
113
113
|
value < @pool_size ? value : @pool_size
|
114
|
-
|
115
|
-
(@pool_size * value).to_i
|
114
|
+
elsif value.is_a?(Float) && value >= 0.0 && value <= 1.0
|
115
|
+
[(@pool_size * value).to_i, 1].max
|
116
116
|
else
|
117
117
|
raise TypeError, "Unknown concurrency value: #{value.inspect}"
|
118
118
|
end
|
data/lib/resqued/daemon.rb
CHANGED
@@ -0,0 +1,43 @@
|
|
1
|
+
require "tempfile"
|
2
|
+
require "yaml"
|
3
|
+
|
4
|
+
module Resqued
|
5
|
+
class ExecOnHUP
|
6
|
+
# Public: Replace the current master process with a new one, while preserving state.
|
7
|
+
def self.exec!(state)
|
8
|
+
exec Resqued::START_CTX["$0"], "--replace", store_state(state), exec_opts(state)
|
9
|
+
end
|
10
|
+
|
11
|
+
# Internal: Returns exec options for each open socket in 'state'.
|
12
|
+
def self.exec_opts(state)
|
13
|
+
exec_opts = {}
|
14
|
+
state.sockets.each do |sock|
|
15
|
+
exec_opts[sock.to_i] = sock
|
16
|
+
end
|
17
|
+
if pwd = Resqued::START_CTX["pwd"]
|
18
|
+
exec_opts[:chdir] = pwd
|
19
|
+
end
|
20
|
+
return exec_opts
|
21
|
+
end
|
22
|
+
|
23
|
+
# Internal: Write out current state to a file, so that a new master can pick up from where we left off.
|
24
|
+
def self.store_state(state)
|
25
|
+
data = { version: Resqued::VERSION }
|
26
|
+
data[:start_ctx] = Resqued::START_CTX
|
27
|
+
data[:state] = state.to_h
|
28
|
+
|
29
|
+
f = Tempfile.create "resqued-state"
|
30
|
+
f.write(YAML.dump(data))
|
31
|
+
f.close
|
32
|
+
return f.path
|
33
|
+
end
|
34
|
+
|
35
|
+
# Internal: Restore the master's state, and remove the state file.
|
36
|
+
def self.restore_state(state, path)
|
37
|
+
data = YAML.safe_load(File.read(path), [Symbol], [], true)
|
38
|
+
Resqued::START_CTX.replace(data[:start_ctx] || {})
|
39
|
+
state.restore(data[:state])
|
40
|
+
File.unlink(path) rescue nil
|
41
|
+
end
|
42
|
+
end
|
43
|
+
end
|
data/lib/resqued/listener.rb
CHANGED
@@ -1,12 +1,12 @@
|
|
1
|
-
require
|
1
|
+
require "socket"
|
2
2
|
|
3
|
-
require
|
4
|
-
require
|
5
|
-
require
|
6
|
-
require
|
7
|
-
require
|
8
|
-
require
|
9
|
-
require
|
3
|
+
require "resqued/config"
|
4
|
+
require "resqued/logging"
|
5
|
+
require "resqued/procline_version"
|
6
|
+
require "resqued/runtime_info"
|
7
|
+
require "resqued/sleepy"
|
8
|
+
require "resqued/version"
|
9
|
+
require "resqued/worker"
|
10
10
|
|
11
11
|
module Resqued
|
12
12
|
# A listener process. Watches resque queues and forks workers.
|
@@ -30,67 +30,70 @@ module Resqued
|
|
30
30
|
# Runs in the master process.
|
31
31
|
def exec
|
32
32
|
socket_fd = @socket.to_i
|
33
|
-
ENV[
|
34
|
-
ENV[
|
35
|
-
ENV[
|
36
|
-
ENV[
|
37
|
-
ENV[
|
33
|
+
ENV["RESQUED_SOCKET"] = socket_fd.to_s
|
34
|
+
ENV["RESQUED_CONFIG_PATH"] = @config_paths.join(":")
|
35
|
+
ENV["RESQUED_STATE"] = @old_workers.map { |r| "#{r[:pid]}|#{r[:queue_key]}" }.join("||")
|
36
|
+
ENV["RESQUED_LISTENER_ID"] = @listener_id.to_s
|
37
|
+
ENV["RESQUED_MASTER_VERSION"] = Resqued::VERSION
|
38
38
|
log "exec: #{Resqued::START_CTX['$0']} listener"
|
39
|
-
exec_opts = {socket_fd => socket_fd} # Ruby 2.0 needs to be told to keep the file descriptor open during exec.
|
40
|
-
if start_pwd = Resqued::START_CTX[
|
39
|
+
exec_opts = { socket_fd => socket_fd } # Ruby 2.0 needs to be told to keep the file descriptor open during exec.
|
40
|
+
if start_pwd = Resqued::START_CTX["pwd"]
|
41
41
|
exec_opts[:chdir] = start_pwd
|
42
42
|
end
|
43
|
-
procline_buf =
|
44
|
-
Kernel.exec(Resqued::START_CTX[
|
43
|
+
procline_buf = " " * 256 # make room for setproctitle
|
44
|
+
Kernel.exec(Resqued::START_CTX["$0"], "listener", procline_buf, exec_opts)
|
45
45
|
end
|
46
46
|
|
47
47
|
# Public: Given args from #exec, start this listener.
|
48
48
|
def self.exec!
|
49
49
|
options = {}
|
50
|
-
if socket = ENV[
|
50
|
+
if socket = ENV["RESQUED_SOCKET"]
|
51
51
|
options[:socket] = Socket.for_fd(socket.to_i)
|
52
52
|
end
|
53
|
-
if path = ENV[
|
54
|
-
options[:config_paths] = path.split(
|
53
|
+
if path = ENV["RESQUED_CONFIG_PATH"]
|
54
|
+
options[:config_paths] = path.split(":")
|
55
55
|
end
|
56
|
-
if state = ENV[
|
57
|
-
options[:old_workers] = state.split(
|
56
|
+
if state = ENV["RESQUED_STATE"]
|
57
|
+
options[:old_workers] = state.split("||").map { |s| Hash[[:pid, :queue_key].zip(s.split("|"))] }
|
58
58
|
end
|
59
|
-
if listener_id = ENV[
|
59
|
+
if listener_id = ENV["RESQUED_LISTENER_ID"]
|
60
60
|
options[:listener_id] = listener_id
|
61
61
|
end
|
62
62
|
new(options).run
|
63
63
|
end
|
64
64
|
|
65
|
-
SIGNALS = [
|
66
|
-
ALL_SIGNALS = SIGNALS + [
|
65
|
+
SIGNALS = [:CONT, :QUIT, :INT, :TERM].freeze
|
66
|
+
ALL_SIGNALS = SIGNALS + [:CHLD, :HUP]
|
67
67
|
|
68
|
-
SIGNAL_QUEUE = []
|
68
|
+
SIGNAL_QUEUE = [] # rubocop: disable Style/MutableConstant
|
69
69
|
|
70
70
|
# Public: Run the main loop.
|
71
71
|
def run
|
72
|
+
trap(:HUP) {} # ignore this, in case it trickles in from the master.
|
72
73
|
trap(:CHLD) { awake }
|
73
|
-
SIGNALS.each { |signal| trap(signal) { SIGNAL_QUEUE << signal
|
74
|
+
SIGNALS.each { |signal| trap(signal) { SIGNAL_QUEUE << signal; awake } }
|
74
75
|
@socket.close_on_exec = true
|
75
|
-
write_procline(
|
76
|
+
write_procline("starting")
|
76
77
|
|
77
78
|
config = Resqued::Config.new(@config_paths)
|
78
79
|
set_default_resque_logger
|
79
80
|
config.before_fork(info)
|
80
81
|
report_to_master("RUNNING")
|
81
82
|
|
82
|
-
write_procline(
|
83
|
+
write_procline("running")
|
83
84
|
init_workers(config)
|
84
85
|
exit_signal = run_workers_run
|
85
86
|
|
86
|
-
write_procline(
|
87
|
+
write_procline("shutdown")
|
87
88
|
burn_down_workers(exit_signal || :QUIT)
|
89
|
+
@socket.close
|
90
|
+
@socket = nil
|
88
91
|
end
|
89
92
|
|
90
93
|
# Private.
|
91
94
|
def set_default_resque_logger
|
92
|
-
require
|
93
|
-
if Resque.respond_to?(
|
95
|
+
require "resque"
|
96
|
+
if Resque.respond_to?("logger=")
|
94
97
|
Resque.logger = Resqued::Logging.build_logger
|
95
98
|
end
|
96
99
|
end
|
@@ -101,7 +104,7 @@ module Resqued
|
|
101
104
|
reap_workers(Process::WNOHANG)
|
102
105
|
check_for_expired_workers
|
103
106
|
start_idle_workers
|
104
|
-
write_procline(
|
107
|
+
write_procline("running")
|
105
108
|
case signal = SIGNAL_QUEUE.shift
|
106
109
|
when nil
|
107
110
|
yawn
|
@@ -119,10 +122,11 @@ module Resqued
|
|
119
122
|
def burn_down_workers(signal)
|
120
123
|
loop do
|
121
124
|
check_for_expired_workers
|
122
|
-
write_procline(
|
125
|
+
write_procline("shutdown")
|
123
126
|
SIGNAL_QUEUE.clear
|
124
127
|
|
125
128
|
break if :no_child == reap_workers(Process::WNOHANG)
|
129
|
+
|
126
130
|
kill_all(signal)
|
127
131
|
|
128
132
|
sleep 1 # Don't kill any more often than every 1s.
|
@@ -134,7 +138,7 @@ module Resqued
|
|
134
138
|
|
135
139
|
# Private: send a signal to all the workers.
|
136
140
|
def kill_all(signal)
|
137
|
-
|
141
|
+
running = running_workers
|
138
142
|
log "kill -#{signal} #{running.map { |r| r.pid }.inspect}"
|
139
143
|
running.each { |worker| worker.kill(signal) }
|
140
144
|
end
|
@@ -171,13 +175,11 @@ module Resqued
|
|
171
175
|
def reap_workers(waitpidflags = 0)
|
172
176
|
loop do
|
173
177
|
worker_pid, status = Process.waitpid2(-1, waitpidflags)
|
174
|
-
if worker_pid.nil?
|
175
|
-
|
176
|
-
|
177
|
-
|
178
|
-
|
179
|
-
report_to_master("-#{worker_pid}")
|
180
|
-
end
|
178
|
+
return :none_ready if worker_pid.nil?
|
179
|
+
|
180
|
+
log "Worker exited #{status}"
|
181
|
+
finish_worker(worker_pid, status)
|
182
|
+
report_to_master("-#{worker_pid}")
|
181
183
|
end
|
182
184
|
rescue Errno::ECHILD
|
183
185
|
# All done
|
@@ -187,6 +189,7 @@ module Resqued
|
|
187
189
|
# Private: Check if master reports any dead workers.
|
188
190
|
def check_for_expired_workers
|
189
191
|
return unless @socket
|
192
|
+
|
190
193
|
loop do
|
191
194
|
IO.select([@socket], nil, nil, 0) or return
|
192
195
|
line = @socket.readline
|
@@ -210,11 +213,11 @@ module Resqued
|
|
210
213
|
# Private.
|
211
214
|
def start_idle_workers
|
212
215
|
workers.each do |worker|
|
213
|
-
|
214
|
-
|
215
|
-
|
216
|
-
|
217
|
-
|
216
|
+
next unless worker.idle?
|
217
|
+
|
218
|
+
worker.try_start
|
219
|
+
if pid = worker.pid
|
220
|
+
report_to_master("+#{pid},#{worker.queue_key}")
|
218
221
|
end
|
219
222
|
end
|
220
223
|
end
|
@@ -236,7 +239,7 @@ module Resqued
|
|
236
239
|
# report_to_master("+12345,queue") # Worker process PID:12345 started, working on a job from "queue".
|
237
240
|
# report_to_master("-12345") # Worker process PID:12345 exited.
|
238
241
|
def report_to_master(status)
|
239
|
-
@socket
|
242
|
+
@socket&.puts(status)
|
240
243
|
rescue Errno::EPIPE => e
|
241
244
|
@socket = nil
|
242
245
|
log "#{e.class.name} while writing to master"
|