resqued 0.7.1 → 0.7.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/docs/ipc.md +23 -0
- data/docs/processes.md +72 -0
- data/docs/signals.md +46 -0
- data/exe/resqued +0 -1
- data/lib/resqued/listener.rb +9 -3
- data/lib/resqued/logging.rb +3 -0
- data/lib/resqued/version.rb +1 -1
- data/lib/resqued/worker.rb +2 -0
- data/spec/fixtures/test_case_after_fork_raises.rb +3 -0
- data/spec/fixtures/test_case_before_fork_raises.rb +2 -0
- data/spec/fixtures/test_case_clean.rb +1 -0
- data/spec/fixtures/test_case_environment.rb +1 -0
- data/spec/fixtures/test_case_no_workers.rb +0 -0
- data/spec/resqued/backoff_spec.rb +61 -0
- data/spec/resqued/config/fork_event_spec.rb +63 -0
- data/spec/resqued/config/worker_spec.rb +150 -0
- data/spec/resqued/sleepy_spec.rb +30 -0
- data/spec/resqued/test_case_spec.rb +39 -0
- data/spec/spec_helper.rb +1 -0
- data/spec/support/custom_matchers.rb +35 -0
- metadata +30 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 4dd030c6805cad2c45c30a29e5415913b813b6b9
|
4
|
+
data.tar.gz: 997391384ab8db66c4dc1b9fc61efd3b2e9ce556
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: d1d646630aeceb8d1b4915e23d3f5dae31cf06d7aa7bdeca11efa7f5c632345c3e3c42668a49cc204b1a4e1a63df011ec6d399c04f6f26c78281b8818e82826d
|
7
|
+
data.tar.gz: 7c774e66d83c5a8dd5279c2b0c57ce83bdd7cf12f565a48c3d1c50a65b148da0297025606b92443126901cec54ec648ef06635781eba4c5d2e0c97815745968a
|
data/docs/ipc.md
ADDED
@@ -0,0 +1,23 @@
|
|
1
|
+
# Inter-Process Communication
|
2
|
+
|
3
|
+
## Master / Listener
|
4
|
+
|
5
|
+
`Resqued::ListenerProxy` opens a Unix domain socket to communicate between the Master and Listener processes. Resqued uses a socket (instead of a pipe) because the socket is bidirectional and a socket can be reopened after `exec`.
|
6
|
+
|
7
|
+
The Listener process sends information about the lifecycle of worker processes that it controls. When a worker starts, the listener writes `"+#{pid},#{queues}\n"`, e.g. `"+21234,important,*\n"`. When a worker exits, the listener writes `"-#{pid}\n"`.
|
8
|
+
|
9
|
+
The Master process broadcasts dead worker PIDs to all running listeners. This allows a new listener to wait for old workers to exit before starting their replacements. These messages are just the pid, `"#{pid}\n"`, e.g. `"21234\n"`.
|
10
|
+
|
11
|
+
*Does the dead worker broadcast need to go to all listeners? Or just the current listener?*
|
12
|
+
|
13
|
+
## Listener / Worker
|
14
|
+
|
15
|
+
There is no direct communication between the Listener and the Worker.
|
16
|
+
|
17
|
+
## Daemon / Master
|
18
|
+
|
19
|
+
The Daemon process opens a pipe. When the Master process starts, it writes its pid to the pipe, and the daemon reads it and exits. If the read fails, the daemon exits with an error status.
|
20
|
+
|
21
|
+
## `self_pipe`
|
22
|
+
|
23
|
+
Each process has a `self_pipe` that it uses as a sleep timer. When the process has no work to do, it performs an `IO.select` on the pipe. If a signal is received, the signal is recorded and a `'.'` is written to the pipe, which causes `IO.select` to finish. This helps avoid race conditions between the main loop and signal handling blocks.
|
data/docs/processes.md
ADDED
@@ -0,0 +1,72 @@
|
|
1
|
+
# Process types in resqued
|
2
|
+
|
3
|
+
Resqued runs the following processes, named after the main class that runs in them:
|
4
|
+
|
5
|
+
* `Resqued::Daemon` is the first process run, if the `-D` flag is provided. It runs until the Master process is started.
|
6
|
+
* `Resqued::Master` is the main process. This process does not restart, so it is somewhat minimal, and should work with different versions of the Listener.
|
7
|
+
* `Resqued::Listener` is started by the Master. A new Listener is started each time the configuration or application is reloaded (on SIGHUP). It manages the pool of workers.
|
8
|
+
* `Resqued::Worker` is started by the Listener, once for each worker.
|
9
|
+
|
10
|
+
In normal operation, there are one Master, one Listener, and several Worker processes.
|
11
|
+
|
12
|
+
When the workers are idle:
|
13
|
+
|
14
|
+
```
|
15
|
+
-+= 77038 burke resqued master [gen 1] [1 running] config/resqued.rb
|
16
|
+
\-+- 77208 burke resqued listener 1 [running] config/resqued.rb
|
17
|
+
|--- 77492 burke resque-1.24.1: Waiting for import_high
|
18
|
+
|--- 77493 burke resque-1.24.1: Waiting for import_high
|
19
|
+
|--- 77494 burke resque-1.24.1: Waiting for important,import_low
|
20
|
+
|--- 77495 burke resque-1.24.1: Waiting for important,import_low
|
21
|
+
|--- 77496 burke resque-1.24.1: Waiting for normal
|
22
|
+
\--- 77497 burke resque-1.24.1: Waiting for normal
|
23
|
+
```
|
24
|
+
|
25
|
+
When the workers are working:
|
26
|
+
|
27
|
+
```
|
28
|
+
-+= 77038 burke resqued master [gen 1] [1 running] config/resqued.rb
|
29
|
+
\-+- 51166 burke resqued listener 1 [running] config/resqued.rb
|
30
|
+
|-+- 51638 burke resque-1.24.1: Forked 78947 at 1377103813
|
31
|
+
| \--- 78947 burke SlowJob::ImportHigh 23 seconds remaining...
|
32
|
+
|-+- 51639 burke resque-1.24.1: Forked 78948 at 1377103813
|
33
|
+
| \--- 78948 burke SlowJob::ImportHigh 23 seconds remaining...
|
34
|
+
|-+- 51642 burke resque-1.24.1: Forked 78950 at 1377103813
|
35
|
+
| \--- 78950 burke SlowJob::Normal 4 seconds remaining...
|
36
|
+
|-+- 51643 burke resque-1.24.1: Forked 78949 at 1377103813
|
37
|
+
| \--- 78949 burke SlowJob::Normal 4 seconds remaining...
|
38
|
+
|-+- 79907 burke resque-1.24.1: Forked 79920 at 1377103819
|
39
|
+
| \--- 79920 burke SlowJob::ImportLow 29 seconds remaining...
|
40
|
+
\-+- 79908 burke resque-1.24.1: Forked 79921 at 1377103819
|
41
|
+
\--- 79921 burke SlowJob::ImportLow 29 seconds remaining...
|
42
|
+
```
|
43
|
+
|
44
|
+
During restart, there should be no more workers than the configuration specifies, but they may be spread out across several listeners. As the workers finish working on the old listener, they will exit and start up on the new listener. When the old listener has no more workers, it exits.
|
45
|
+
|
46
|
+
```
|
47
|
+
-+= 51121 burke resqued master [gen 2] [2 running] config/resqued.rb
|
48
|
+
|-+- 51166 burke resqued listener 1 [shutdown] config/resqued.rb
|
49
|
+
| |-+- 51638 burke resque-1.24.1: Forked 78947 at 1377103813
|
50
|
+
| | \--- 78947 burke SlowJob::ImportHigh 23 seconds remaining...
|
51
|
+
| |-+- 51639 burke resque-1.24.1: Forked 78948 at 1377103813
|
52
|
+
| | \--- 78948 burke SlowJob::ImportHigh 23 seconds remaining...
|
53
|
+
| |-+- 51642 burke resque-1.24.1: Forked 78950 at 1377103813
|
54
|
+
| | \--- 78950 burke SlowJob::Normal 4 seconds remaining...
|
55
|
+
| \-+- 51643 burke resque-1.24.1: Forked 78949 at 1377103813
|
56
|
+
| \--- 78949 burke SlowJob::Normal 4 seconds remaining...
|
57
|
+
\-+- 79528 burke resqued listener 2 [running] config/resqued.rb
|
58
|
+
|-+- 79907 burke resque-1.24.1: Forked 79920 at 1377103819
|
59
|
+
| \--- 79920 burke SlowJob::ImportLow 29 seconds remaining...
|
60
|
+
\-+- 79908 burke resque-1.24.1: Forked 79921 at 1377103819
|
61
|
+
\--- 79921 burke SlowJob::ImportLow 29 seconds remaining...
|
62
|
+
```
|
63
|
+
|
64
|
+
## Spawning
|
65
|
+
|
66
|
+
The Daemon process starts the Master process with a double-fork, the normal daemonization process.
|
67
|
+
|
68
|
+
The Master process starts the Listener process with a fork+exec. `bin/resqued-listener` is the process that is executed. Doing a full exec on the listener allows it to load any application changes, including changes to the `Gemfile`, without any special code. In order to keep the exec interface flexible, all initialization data is passed to the listener from the master via environment variables. This information is minimal, and includes the location of the config file and the status of other workers.
|
69
|
+
|
70
|
+
The Listener process loads the application configuration (the `before_fork` block in the configuration file) before starting any workers. Worker processes are started by forking, and starting the Resque::Worker run loop.
|
71
|
+
|
72
|
+
In the event that a Listener or Worker exits quickly and unexpectedly, resqued will not immediately restart the process.
|
data/docs/signals.md
ADDED
@@ -0,0 +1,46 @@
|
|
1
|
+
# Signals in resqued
|
2
|
+
|
3
|
+
Signals control restart, reload, and shutdown in resqued. This file documents the signals that resqued's processes handle. You should normally only send signals to the Master process.
|
4
|
+
|
5
|
+
Here is a summary of how signals get passed between resqued's processes:
|
6
|
+
|
7
|
+
```
|
8
|
+
master listener worker
|
9
|
+
------ -------- ------
|
10
|
+
restart HUP -> QUIT -> QUIT
|
11
|
+
exit now INT -> INT -> INT
|
12
|
+
exit now TERM -> TERM -> TERM
|
13
|
+
exit when ready QUIT -> QUIT -> QUIT
|
14
|
+
pause USR2 -> QUIT -> QUIT
|
15
|
+
unpause CONT -> (start)
|
16
|
+
unpause CONT -> CONT -> CONT
|
17
|
+
```
|
18
|
+
|
19
|
+
Read on for more information about what the signals mean.
|
20
|
+
|
21
|
+
## Master
|
22
|
+
|
23
|
+
The Master process handles several signals.
|
24
|
+
|
25
|
+
* `HUP`: Start a new listener. After it boots, kill the previous listener with `SIGQUIT`.
|
26
|
+
* `USR2`: Pause processing. Kills the current listener, and does not start a replacement.
|
27
|
+
* `CONT`: Resume processing. If there is no listener, start one. If there is a listener, send it SIGCONT.
|
28
|
+
* `QUIT`, `INT`, or `TERM`: Kill the listener with the same signal and wait for it to exit.
|
29
|
+
* `CHLD`: Clean up any listeners that have exited. If the current listener exited
|
30
|
+
|
31
|
+
## Listener
|
32
|
+
|
33
|
+
The Listener process forwards `SIGCONT` to all of its workers.
|
34
|
+
|
35
|
+
The Listener process handles `SIGINT`, `SIGTERM`, and `SIGQUIT`. When it receives one of these signals, it goes into shutdown mode. It sends the received signal to all of its workers. When all workers have exited, the Listener process exits.
|
36
|
+
|
37
|
+
## Worker
|
38
|
+
|
39
|
+
The Worker process uses resque's signal handling. Resque 1.23.0 handles the following signals:
|
40
|
+
|
41
|
+
* `TERM`: Shutdown immediately, stop processing jobs.
|
42
|
+
* `INT`: Shutdown immediately, stop processing jobs.
|
43
|
+
* `QUIT`: Shutdown after the current job has finished processing.
|
44
|
+
* `USR1`: Kill the forked child immediately, continue processing jobs.
|
45
|
+
* `USR2`: Don't process any new jobs
|
46
|
+
* `CONT`: Start processing jobs again after a USR2
|
data/exe/resqued
CHANGED
data/lib/resqued/listener.rb
CHANGED
@@ -142,9 +142,15 @@ module Resqued
|
|
142
142
|
def reap_workers(waitpidflags = 0)
|
143
143
|
loop do
|
144
144
|
worker_pid, status = Process.waitpid2(-1, waitpidflags)
|
145
|
-
|
146
|
-
|
147
|
-
|
145
|
+
if worker_pid.nil?
|
146
|
+
return :none_ready
|
147
|
+
elsif status.exited?
|
148
|
+
log "Worker exited #{status}"
|
149
|
+
finish_worker(worker_pid, status)
|
150
|
+
report_to_master("-#{worker_pid}")
|
151
|
+
else
|
152
|
+
log "Worker reported #{status}"
|
153
|
+
end
|
148
154
|
end
|
149
155
|
rescue Errno::ECHILD
|
150
156
|
# All done
|
data/lib/resqued/logging.rb
CHANGED
data/lib/resqued/version.rb
CHANGED
data/lib/resqued/worker.rb
CHANGED
@@ -0,0 +1 @@
|
|
1
|
+
worker "test"
|
@@ -0,0 +1 @@
|
|
1
|
+
before_fork { Resque.redis = Redis.new(:host => 'localhost', :port => ENV['RESQUED_TEST_REDIS_PORT'].to_i) }
|
File without changes
|
@@ -0,0 +1,61 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
require 'resqued/backoff'
|
4
|
+
|
5
|
+
describe Resqued::Backoff do
|
6
|
+
let(:backoff) { described_class.new(:min => 0.5, :max => 64.0) }
|
7
|
+
|
8
|
+
it 'can start on the first try' do
|
9
|
+
expect(backoff.wait?).to be_false
|
10
|
+
end
|
11
|
+
|
12
|
+
it 'has no waiting at first' do
|
13
|
+
expect(backoff.how_long?).to be_nil
|
14
|
+
end
|
15
|
+
|
16
|
+
context 'after expected exits' do
|
17
|
+
before { 3.times { backoff.started } }
|
18
|
+
it { expect(backoff.wait?).to be_true }
|
19
|
+
it { expect(backoff.how_long?).to be_close_to(0.5) }
|
20
|
+
end
|
21
|
+
|
22
|
+
context 'after one quick exit' do
|
23
|
+
before { 1.times { backoff.started ; backoff.died } }
|
24
|
+
it { expect(backoff.wait?).to be_true }
|
25
|
+
it { expect(backoff.how_long?).to be_close_to(1.0) }
|
26
|
+
end
|
27
|
+
|
28
|
+
context 'after two quick starts' do
|
29
|
+
before { 2.times { backoff.started ; backoff.died } }
|
30
|
+
it { expect(backoff.wait?).to be_true }
|
31
|
+
it { expect(backoff.how_long?).to be_close_to(2.0) }
|
32
|
+
end
|
33
|
+
|
34
|
+
context 'after five quick starts' do
|
35
|
+
before { 6.times { backoff.started ; backoff.died } }
|
36
|
+
it { expect(backoff.wait?).to be_true }
|
37
|
+
it { expect(backoff.how_long?).to be_close_to(32.0) }
|
38
|
+
end
|
39
|
+
|
40
|
+
context 'after six quick starts' do
|
41
|
+
before { 7.times { backoff.started ; backoff.died } }
|
42
|
+
it { expect(backoff.wait?).to be_true }
|
43
|
+
it { expect(backoff.how_long?).to be_close_to(64.0) }
|
44
|
+
end
|
45
|
+
|
46
|
+
context 'does not wait longer than 64s' do
|
47
|
+
before { 8.times { backoff.started ; backoff.died } }
|
48
|
+
it { expect(backoff.wait?).to be_true }
|
49
|
+
it { expect(backoff.how_long?).to be_close_to(64.0) }
|
50
|
+
it 'and resets after an expected exit' do
|
51
|
+
backoff.started
|
52
|
+
backoff.started
|
53
|
+
expect(backoff.wait?).to be_true
|
54
|
+
expect(backoff.how_long?).to be_close_to(0.5)
|
55
|
+
end
|
56
|
+
end
|
57
|
+
|
58
|
+
def be_close_to(x)
|
59
|
+
be_within(0.005).of(x)
|
60
|
+
end
|
61
|
+
end
|
@@ -0,0 +1,63 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
require 'resqued/config/after_fork'
|
3
|
+
require 'resqued/config/before_fork'
|
4
|
+
|
5
|
+
describe do
|
6
|
+
before { evaluator.apply(config) }
|
7
|
+
|
8
|
+
context 'after_fork' do
|
9
|
+
# Run the after_fork block.
|
10
|
+
#
|
11
|
+
# after_fork do |resque_worker|
|
12
|
+
# ActiveRecord::Base.establish_connection
|
13
|
+
# end
|
14
|
+
#
|
15
|
+
# ignore calls to any other top-level method.
|
16
|
+
|
17
|
+
let(:config) { <<-END_CONFIG }
|
18
|
+
before_fork { }
|
19
|
+
worker('one')
|
20
|
+
worker_pool(10)
|
21
|
+
queue('*')
|
22
|
+
|
23
|
+
after_fork do |worker|
|
24
|
+
worker.token = :called
|
25
|
+
end
|
26
|
+
END_CONFIG
|
27
|
+
|
28
|
+
let(:evaluator) { Resqued::Config::AfterFork.new(:worker => worker) }
|
29
|
+
let(:worker) { FakeResqueWorker.new }
|
30
|
+
|
31
|
+
it { expect(worker.token).to eq(:called) }
|
32
|
+
end
|
33
|
+
|
34
|
+
context 'before_fork' do
|
35
|
+
# Run the before_fork block.
|
36
|
+
#
|
37
|
+
# before_fork do
|
38
|
+
# require "./config/environment.rb"
|
39
|
+
# Rails.application.eager_load!
|
40
|
+
# end
|
41
|
+
#
|
42
|
+
# ignore calls to any other top-level method.
|
43
|
+
|
44
|
+
let(:config) { <<-END_CONFIG }
|
45
|
+
after_fork { |worker| }
|
46
|
+
worker('one')
|
47
|
+
worker_pool(10)
|
48
|
+
queue('*')
|
49
|
+
|
50
|
+
before_fork do
|
51
|
+
$before_fork_called = true
|
52
|
+
end
|
53
|
+
END_CONFIG
|
54
|
+
|
55
|
+
let(:evaluator) { $before_fork_called = false ; Resqued::Config::BeforeFork.new }
|
56
|
+
|
57
|
+
it { expect($before_fork_called).to eq(true) }
|
58
|
+
end
|
59
|
+
end
|
60
|
+
|
61
|
+
class FakeResqueWorker
|
62
|
+
attr_accessor :token
|
63
|
+
end
|
@@ -0,0 +1,150 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
require 'resqued/config/worker'
|
3
|
+
|
4
|
+
describe Resqued::Config::Worker do
|
5
|
+
# Create a bunch of Resqued::Worker objects from
|
6
|
+
#
|
7
|
+
# worker 'one'
|
8
|
+
# worker 'two', 'three', :interval => 2
|
9
|
+
# worker # assumes '*' as the queue
|
10
|
+
# worker_pool 10
|
11
|
+
# queue 'four', :percent => 20
|
12
|
+
# queue 'five', :count => 5
|
13
|
+
# queue 'six', 'seven', :count => 1
|
14
|
+
# queue '*'
|
15
|
+
#
|
16
|
+
# ignore calls to any other top-level method.
|
17
|
+
|
18
|
+
let(:evaluator) { described_class.new(:worker_class => FakeWorker) }
|
19
|
+
let(:result) { evaluator.apply(config) }
|
20
|
+
module FakeWorker
|
21
|
+
def self.new(options)
|
22
|
+
options
|
23
|
+
end
|
24
|
+
end
|
25
|
+
|
26
|
+
context 'individual' do
|
27
|
+
let(:config) { <<-END_CONFIG }
|
28
|
+
before_fork { }
|
29
|
+
after_fork { }
|
30
|
+
2.times { worker 'a' }
|
31
|
+
worker 'b'
|
32
|
+
worker 'c', 'd'
|
33
|
+
worker 'd', 'c', :interval => 3
|
34
|
+
worker
|
35
|
+
after_fork { } # So that we don't rely on `workers`'s result falling through.
|
36
|
+
END_CONFIG
|
37
|
+
it { expect(result.size).to eq(6) }
|
38
|
+
it { expect(result[0]).to eq(:queues => ['a']) }
|
39
|
+
it { expect(result[1]).to eq(:queues => ['a']) }
|
40
|
+
it { expect(result[2]).to eq(:queues => ['b']) }
|
41
|
+
it { expect(result[3]).to eq(:queues => ['c', 'd']) }
|
42
|
+
it { expect(result[4]).to eq(:queues => ['d', 'c'], :interval => 3) }
|
43
|
+
it { expect(result[5]).to eq(:queues => ['*']) }
|
44
|
+
end
|
45
|
+
|
46
|
+
context 'concise pool' do
|
47
|
+
let(:config) { <<-END_CONFIG }
|
48
|
+
worker_pool 2, 'a', 'b', 'c', :interval => 1
|
49
|
+
END_CONFIG
|
50
|
+
it { expect(result).to eq([
|
51
|
+
{ :queues => ['a', 'b', 'c'], :interval => 1 },
|
52
|
+
{ :queues => ['a', 'b', 'c'], :interval => 1 },
|
53
|
+
]) }
|
54
|
+
end
|
55
|
+
|
56
|
+
context 'pool (hash for concurrency)' do
|
57
|
+
let(:config) { <<-END_CONFIG }
|
58
|
+
before_fork { }
|
59
|
+
after_fork { }
|
60
|
+
worker_pool 20, :interval => 1
|
61
|
+
queue 'a', :percent => 20
|
62
|
+
queue 'b1', 'b2', :count => 10
|
63
|
+
queue 'c'
|
64
|
+
after_fork { } # So that we don't rely on `worker_pool`'s result falling through.
|
65
|
+
END_CONFIG
|
66
|
+
it { expect(result.size).to eq(20) }
|
67
|
+
it { expect(result[0]).to eq(:queues => ['a', 'b1', 'b2', 'c'], :interval => 1) }
|
68
|
+
it { expect(result[3]).to eq(:queues => ['a', 'b1', 'b2', 'c'], :interval => 1) }
|
69
|
+
it { expect(result[4]).to eq(:queues => ['b1', 'b2', 'c'], :interval => 1) }
|
70
|
+
it { expect(result[9]).to eq(:queues => ['b1', 'b2', 'c'], :interval => 1) }
|
71
|
+
it { expect(result[10]).to eq(:queues => ['c'], :interval => 1) }
|
72
|
+
it { expect(result[19]).to eq(:queues => ['c'], :interval => 1) }
|
73
|
+
end
|
74
|
+
|
75
|
+
context 'pool, with implied queue' do
|
76
|
+
let(:config) { <<-END_CONFIG }
|
77
|
+
before_fork { }
|
78
|
+
after_fork { }
|
79
|
+
worker_pool 20
|
80
|
+
after_fork { } # So that we don't rely on `worker_pool`'s result falling through.
|
81
|
+
END_CONFIG
|
82
|
+
it { expect(result.size).to eq(20) }
|
83
|
+
it { expect(result[0]).to eq(:queues => ['*']) }
|
84
|
+
it { expect(result[19]).to eq(:queues => ['*']) }
|
85
|
+
end
|
86
|
+
|
87
|
+
context 'pool, with fewer queues than workers' do
|
88
|
+
let(:config) { <<-END_CONFIG }
|
89
|
+
before_fork { }
|
90
|
+
after_fork { }
|
91
|
+
worker_pool 20
|
92
|
+
queue 'a', :count => 10
|
93
|
+
after_fork { } # So that we don't rely on `worker_pool`'s result falling through.
|
94
|
+
END_CONFIG
|
95
|
+
it { expect(result.size).to eq(20) }
|
96
|
+
it { expect(result[0]).to eq(:queues => ['a']) }
|
97
|
+
it { expect(result[9]).to eq(:queues => ['a']) }
|
98
|
+
it { expect(result[10]).to eq(:queues => ['*']) }
|
99
|
+
it { expect(result[19]).to eq(:queues => ['*']) }
|
100
|
+
end
|
101
|
+
|
102
|
+
context 'pool, with more queues than workers' do
|
103
|
+
let(:config) { <<-END_CONFIG }
|
104
|
+
before_fork { }
|
105
|
+
after_fork { }
|
106
|
+
worker_pool 20
|
107
|
+
queue 'a', :count => 30
|
108
|
+
after_fork { } # So that we don't rely on `worker_pool`'s result falling through.
|
109
|
+
END_CONFIG
|
110
|
+
it { expect(result.size).to eq(20) }
|
111
|
+
end
|
112
|
+
|
113
|
+
context 'pool, with shuffled queues' do
|
114
|
+
let(:config) { <<-END_CONFIG }
|
115
|
+
worker_pool 20, :shuffle_queues => true
|
116
|
+
queue 'a', :count => 10
|
117
|
+
queue 'b', :count => 15
|
118
|
+
END_CONFIG
|
119
|
+
it { expect(result.size).to eq(20) }
|
120
|
+
it { (0..9).each { |i| expect(result[i][:queues].sort).to eq(['a', 'b']) } }
|
121
|
+
it { (10..14).each { |i| expect(result[i][:queues]).to eq(['b']) } }
|
122
|
+
it { (15..19).each { |i| expect(result[i][:queues]).to eq(['*']) } }
|
123
|
+
it { result.each { |x| expect(x).not_to have_key(:shuffle_queues) } }
|
124
|
+
it do
|
125
|
+
shuffled_queues = result.take(10).map { |x| x[:queues] }
|
126
|
+
expect(shuffled_queues.sort.uniq).to eq([ ['a','b'], ['b','a'] ]) # Some of the queues should be shuffled
|
127
|
+
end
|
128
|
+
end
|
129
|
+
|
130
|
+
context 'multiple worker configs' do
|
131
|
+
let(:config) { <<-END_CONFIG }
|
132
|
+
worker 'one'
|
133
|
+
worker 'two'
|
134
|
+
worker_pool 2
|
135
|
+
END_CONFIG
|
136
|
+
it { expect(result.size).to eq(4) }
|
137
|
+
it { expect(result[0]).to eq(:queues => ['one']) }
|
138
|
+
it { expect(result[1]).to eq(:queues => ['two']) }
|
139
|
+
it { expect(result[2]).to eq(:queues => ['*']) }
|
140
|
+
it { expect(result[3]).to eq(:queues => ['*']) }
|
141
|
+
end
|
142
|
+
|
143
|
+
context 'with default options' do
|
144
|
+
let(:evaluator) { described_class.new(:worker_class => FakeWorker, :config => 'something') }
|
145
|
+
let(:config) { <<-END_CONFIG }
|
146
|
+
worker 'a', :interval => 1
|
147
|
+
END_CONFIG
|
148
|
+
it { expect(result[0]).to eq(:queues => ['a'], :interval => 1, :config => 'something') }
|
149
|
+
end
|
150
|
+
end
|
@@ -0,0 +1,30 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
require 'resqued/sleepy'
|
3
|
+
require 'thread'
|
4
|
+
|
5
|
+
describe Resqued::Sleepy do
|
6
|
+
include Resqued::Sleepy
|
7
|
+
|
8
|
+
it 'sleeps' do
|
9
|
+
expect { yawn(0.2) }.to run_for(0.2)
|
10
|
+
end
|
11
|
+
|
12
|
+
it 'wakes on `awake`' do
|
13
|
+
Thread.new { sleep 0.1 ; awake }
|
14
|
+
expect { yawn(2.0) }.to run_for(0.1)
|
15
|
+
end
|
16
|
+
|
17
|
+
it 'wakes on IO' do
|
18
|
+
rd, wr = IO.pipe
|
19
|
+
Thread.new { sleep 0.1 ; wr.write('.') }
|
20
|
+
expect { yawn(2.0, rd) }.to run_for(0.1)
|
21
|
+
end
|
22
|
+
|
23
|
+
it 'does not sleep if duration is 0' do
|
24
|
+
expect { yawn(-0.000001) }.to run_for(0.0)
|
25
|
+
end
|
26
|
+
|
27
|
+
it 'does not sleep if duration is negative' do
|
28
|
+
expect { yawn(0) }.to run_for(0.0)
|
29
|
+
end
|
30
|
+
end
|
@@ -0,0 +1,39 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
require 'resqued/test_case'
|
3
|
+
|
4
|
+
describe Resqued::TestCase do
|
5
|
+
let(:test_case) { Object.new.extend(the_module) }
|
6
|
+
before { mock_redis.start ; ENV['RESQUED_TEST_REDIS_PORT'] = mock_redis.port.to_s }
|
7
|
+
after { mock_redis.stop }
|
8
|
+
let(:mock_redis) { MockRedisServer.new }
|
9
|
+
|
10
|
+
context 'ForkToStart' do
|
11
|
+
let(:the_module) { described_class::ForkToStart }
|
12
|
+
it { expect { test_case.assert_resqued 'spec/fixtures/test_case_environment.rb', 'spec/fixtures/test_case_clean.rb' }.not_to raise_error }
|
13
|
+
it { expect { test_case.assert_resqued 'spec/fixtures/test_case_environment.rb', 'spec/fixtures/test_case_before_fork_raises.rb' }.to raise_error }
|
14
|
+
it { expect { test_case.assert_resqued 'spec/fixtures/test_case_environment.rb', 'spec/fixtures/test_case_after_fork_raises.rb' }.not_to raise_error }
|
15
|
+
it { expect { test_case.assert_resqued 'spec/fixtures/test_case_environment.rb', 'spec/fixtures/test_case_no_workers.rb' }.not_to raise_error }
|
16
|
+
it { expect { test_case.assert_resqued 'spec/fixtures/test_case_environment.rb', 'spec/fixtures/test_case_clean.rb', :expect_workers => true }.not_to raise_error }
|
17
|
+
it { expect { test_case.assert_resqued 'spec/fixtures/test_case_environment.rb', 'spec/fixtures/test_case_after_fork_raises.rb', :expect_workers => true }.to raise_error }
|
18
|
+
it { expect { test_case.assert_resqued 'spec/fixtures/test_case_environment.rb', 'spec/fixtures/test_case_no_workers.rb', :expect_workers => true }.to raise_error }
|
19
|
+
end
|
20
|
+
end
|
21
|
+
|
22
|
+
class MockRedisServer
|
23
|
+
def start
|
24
|
+
return if @server
|
25
|
+
require 'socket'
|
26
|
+
@server = TCPServer.new(0)
|
27
|
+
end
|
28
|
+
|
29
|
+
def stop
|
30
|
+
return unless @server
|
31
|
+
@server.close
|
32
|
+
ensure
|
33
|
+
@server = nil
|
34
|
+
end
|
35
|
+
|
36
|
+
def port
|
37
|
+
@server && @server.addr[1]
|
38
|
+
end
|
39
|
+
end
|
data/spec/spec_helper.rb
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
require 'support/custom_matchers'
|
@@ -0,0 +1,35 @@
|
|
1
|
+
module CustomMatchers
|
2
|
+
# Examples:
|
3
|
+
# expect { sleep 0.5 }.to run_for(0.5).within(0.0001)
|
4
|
+
def run_for(expected_duration)
|
5
|
+
RunFor.new(expected_duration)
|
6
|
+
end
|
7
|
+
|
8
|
+
class RunFor
|
9
|
+
def initialize(expected_duration)
|
10
|
+
@expected_duration = expected_duration
|
11
|
+
@epsilon = 0.01
|
12
|
+
end
|
13
|
+
|
14
|
+
def within(epsilon)
|
15
|
+
@epsilon = epsilon
|
16
|
+
self
|
17
|
+
end
|
18
|
+
|
19
|
+
def matches?(event_proc)
|
20
|
+
start_time = Time.now
|
21
|
+
event_proc.call
|
22
|
+
@actual_duration = Time.now - start_time
|
23
|
+
diff = (@actual_duration - @expected_duration).abs
|
24
|
+
@epsilon >= diff
|
25
|
+
end
|
26
|
+
|
27
|
+
def failure_message_for_should
|
28
|
+
"Expected block to run for #{@expected_duration} +/-#{@epsilon} seconds, but it ran for #{@actual_duration} seconds."
|
29
|
+
end
|
30
|
+
end
|
31
|
+
end
|
32
|
+
|
33
|
+
RSpec.configure do |config|
|
34
|
+
config.include CustomMatchers
|
35
|
+
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: resqued
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.7.
|
4
|
+
version: 0.7.2
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Matt Burke
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2013-
|
11
|
+
date: 2013-11-13 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: kgio
|
@@ -150,6 +150,21 @@ files:
|
|
150
150
|
- lib/resqued/worker.rb
|
151
151
|
- lib/resqued.rb
|
152
152
|
- README.md
|
153
|
+
- docs/ipc.md
|
154
|
+
- docs/processes.md
|
155
|
+
- docs/signals.md
|
156
|
+
- spec/fixtures/test_case_after_fork_raises.rb
|
157
|
+
- spec/fixtures/test_case_before_fork_raises.rb
|
158
|
+
- spec/fixtures/test_case_clean.rb
|
159
|
+
- spec/fixtures/test_case_environment.rb
|
160
|
+
- spec/fixtures/test_case_no_workers.rb
|
161
|
+
- spec/resqued/backoff_spec.rb
|
162
|
+
- spec/resqued/config/fork_event_spec.rb
|
163
|
+
- spec/resqued/config/worker_spec.rb
|
164
|
+
- spec/resqued/sleepy_spec.rb
|
165
|
+
- spec/resqued/test_case_spec.rb
|
166
|
+
- spec/spec_helper.rb
|
167
|
+
- spec/support/custom_matchers.rb
|
153
168
|
- exe/resqued
|
154
169
|
- exe/resqued-listener
|
155
170
|
homepage: https://github.com
|
@@ -175,4 +190,16 @@ rubygems_version: 2.0.3
|
|
175
190
|
signing_key:
|
176
191
|
specification_version: 4
|
177
192
|
summary: Daemon of resque workers
|
178
|
-
test_files:
|
193
|
+
test_files:
|
194
|
+
- spec/fixtures/test_case_after_fork_raises.rb
|
195
|
+
- spec/fixtures/test_case_before_fork_raises.rb
|
196
|
+
- spec/fixtures/test_case_clean.rb
|
197
|
+
- spec/fixtures/test_case_environment.rb
|
198
|
+
- spec/fixtures/test_case_no_workers.rb
|
199
|
+
- spec/resqued/backoff_spec.rb
|
200
|
+
- spec/resqued/config/fork_event_spec.rb
|
201
|
+
- spec/resqued/config/worker_spec.rb
|
202
|
+
- spec/resqued/sleepy_spec.rb
|
203
|
+
- spec/resqued/test_case_spec.rb
|
204
|
+
- spec/spec_helper.rb
|
205
|
+
- spec/support/custom_matchers.rb
|