sneakers 1.0.4 → 1.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.travis.yml +1 -1
- data/README.md +21 -21
- data/bin/sneakers +1 -0
- data/examples/benchmark_worker.rb +2 -1
- data/examples/max_retry_handler.rb +1 -0
- data/examples/profiling_worker.rb +2 -2
- data/examples/workflow_worker.rb +2 -1
- data/lib/sneakers.rb +17 -1
- data/lib/sneakers/configuration.rb +32 -5
- data/lib/sneakers/publisher.rb +1 -1
- data/lib/sneakers/queue.rb +11 -2
- data/lib/sneakers/runner.rb +9 -6
- data/lib/sneakers/tasks.rb +8 -2
- data/lib/sneakers/version.rb +1 -1
- data/lib/sneakers/workergroup.rb +6 -3
- data/sneakers.gemspec +1 -1
- data/spec/sneakers/configuration_spec.rb +66 -44
- data/spec/sneakers/publisher_spec.rb +2 -1
- data/spec/sneakers/queue_spec.rb +102 -70
- data/spec/sneakers/runner_spec.rb +18 -0
- data/spec/sneakers/sneakers_spec.rb +4 -2
- data/spec/sneakers/worker_spec.rb +8 -2
- metadata +10 -4
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 6b646dd7a8a5c9389275bd3406ea7e7839d09cd7
|
4
|
+
data.tar.gz: 293440a5a50a41f30064ef85d14ac69d84f26652
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 9f5b28202fece758dbd64f29239439da4f023e289da2d1f8ef2c484667bec034a580f97c31dc2ab1931b5315130e177df86ba8927b8f94c06d471fba21dd93e1
|
7
|
+
data.tar.gz: 1f53048671fc9e83480891998bf0a01a28796fa7e835906fc40c14489b4143e18b23059f7544ef6b58459eafa288c56e8a06beb3933f4cf133163de0fc605d34
|
data/.travis.yml
CHANGED
data/README.md
CHANGED
@@ -27,16 +27,21 @@ complete docs.
|
|
27
27
|
|
28
28
|
Add this line to your application's Gemfile:
|
29
29
|
|
30
|
-
|
30
|
+
``` ruby
|
31
|
+
gem 'sneakers'
|
32
|
+
```
|
31
33
|
|
32
34
|
And then execute:
|
33
35
|
|
34
|
-
|
36
|
+
``` shell-session
|
37
|
+
$ bundle
|
38
|
+
```
|
35
39
|
|
36
40
|
Or install it yourself as:
|
37
41
|
|
38
|
-
|
39
|
-
|
42
|
+
``` shell-session
|
43
|
+
$ gem install sneakers
|
44
|
+
```
|
40
45
|
|
41
46
|
## Quick start
|
42
47
|
|
@@ -74,9 +79,7 @@ class Processor
|
|
74
79
|
end
|
75
80
|
```
|
76
81
|
|
77
|
-
|
78
|
-
As an example, make a message look like this:
|
79
|
-
We'll count errors and error types with Redis. Specifically for an error that looks like this:
|
82
|
+
We'll count errors and error types with Redis. As an example, make a message that looks like this:
|
80
83
|
|
81
84
|
```javascript
|
82
85
|
{
|
@@ -90,8 +93,8 @@ We'll count errors and error types with Redis. Specifically for an error that lo
|
|
90
93
|
Let's test it out quickly from the command line:
|
91
94
|
|
92
95
|
|
93
|
-
```
|
94
|
-
sneakers work Processor --require boot.rb
|
96
|
+
```shell-session
|
97
|
+
$ sneakers work Processor --require boot.rb
|
95
98
|
```
|
96
99
|
|
97
100
|
We just told Sneakers to spawn a worker named `Processor`, but first `--require` a file that we dedicate to setting up environment, including workers and what-not.
|
@@ -109,8 +112,8 @@ If you go to your RabbitMQ admin now, you'll see a new queue named `logs` was cr
|
|
109
112
|
And redis will show this:
|
110
113
|
|
111
114
|
|
112
|
-
```
|
113
|
-
|
115
|
+
``` shell-session
|
116
|
+
$ redis-cli monitor
|
114
117
|
1381520329.888581 [0 127.0.0.1:49182] "incr" "processor:CODE001"
|
115
118
|
```
|
116
119
|
|
@@ -142,13 +145,17 @@ Now push a message again and you'll see:
|
|
142
145
|
2013-10-11T19:44:37Z p-9219 t-oxh8owywg INFO: INC: work.Processor.handled.ack
|
143
146
|
```
|
144
147
|
|
145
|
-
Which increments
|
146
|
-
|
148
|
+
Which increments `started` and `handled.ack`, and times the work unit.
|
147
149
|
|
148
150
|
|
149
151
|
From here, you can continue over to the
|
150
152
|
[Wiki](https://github.com/jondot/sneakers/wiki)
|
151
153
|
|
154
|
+
# Compatibility
|
155
|
+
|
156
|
+
* Sneakers 1.1.x and up (using the new generation Bunny 2.x) - Ruby 2.x.x
|
157
|
+
* Sneakers 1.x.x and down - Ruby 1.9.x, 2.x.x
|
158
|
+
|
152
159
|
# Contributing
|
153
160
|
|
154
161
|
Fork, implement, add tests, pull request, get my everlasting thanks and a respectable place here :).
|
@@ -162,11 +169,4 @@ To all Sneakers [Contributors](https://github.com/jondot/sneakers/graphs/contrib
|
|
162
169
|
|
163
170
|
# Copyright
|
164
171
|
|
165
|
-
Copyright (c) 2015 [Dotan Nahum](http://gplus.to/dotan) [@jondot](http://twitter.com/jondot). See
|
166
|
-
|
167
|
-
|
168
|
-
|
169
|
-
|
170
|
-
|
171
|
-
|
172
|
-
|
172
|
+
Copyright (c) 2015 [Dotan Nahum](http://gplus.to/dotan) [@jondot](http://twitter.com/jondot). See [LICENSE](LICENSE.txt) for further details.
|
data/bin/sneakers
CHANGED
data/examples/workflow_worker.rb
CHANGED
data/lib/sneakers.rb
CHANGED
@@ -47,6 +47,10 @@ module Sneakers
|
|
47
47
|
logger.level = loglevel
|
48
48
|
end
|
49
49
|
|
50
|
+
def logger=(logger)
|
51
|
+
@logger = logger
|
52
|
+
end
|
53
|
+
|
50
54
|
def logger
|
51
55
|
@logger
|
52
56
|
end
|
@@ -59,13 +63,25 @@ module Sneakers
|
|
59
63
|
@configured
|
60
64
|
end
|
61
65
|
|
66
|
+
def server=(server)
|
67
|
+
@server = server
|
68
|
+
end
|
69
|
+
|
70
|
+
def server?
|
71
|
+
@server
|
72
|
+
end
|
73
|
+
|
74
|
+
def configure_server
|
75
|
+
yield self if server?
|
76
|
+
end
|
77
|
+
|
62
78
|
private
|
63
79
|
|
64
80
|
def setup_general_logger!
|
65
81
|
if [:info, :debug, :error, :warn].all?{ |meth| CONFIG[:log].respond_to?(meth) }
|
66
82
|
@logger = CONFIG[:log]
|
67
83
|
else
|
68
|
-
@logger =
|
84
|
+
@logger = ServerEngine::DaemonLogger.new(CONFIG[:log])
|
69
85
|
@logger.formatter = Sneakers::Support::ProductionFormatter
|
70
86
|
end
|
71
87
|
end
|
@@ -4,7 +4,7 @@ module Sneakers
|
|
4
4
|
class Configuration
|
5
5
|
|
6
6
|
extend Forwardable
|
7
|
-
def_delegators :@hash, :to_hash, :[], :[]=, :==, :fetch, :delete
|
7
|
+
def_delegators :@hash, :to_hash, :[], :[]=, :==, :fetch, :delete, :has_key?
|
8
8
|
|
9
9
|
DEFAULTS = {
|
10
10
|
# runner
|
@@ -15,16 +15,19 @@ module Sneakers
|
|
15
15
|
:workers => 4,
|
16
16
|
:log => STDOUT,
|
17
17
|
:pid_path => 'sneakers.pid',
|
18
|
+
:amqp_heartbeat => 10,
|
18
19
|
|
19
20
|
# workers
|
20
21
|
:timeout_job_after => 5,
|
21
22
|
:prefetch => 10,
|
22
23
|
:threads => 10,
|
24
|
+
:share_threads => false,
|
23
25
|
:durable => true,
|
24
26
|
:ack => true,
|
25
27
|
:heartbeat => 2,
|
26
28
|
:exchange => 'sneakers',
|
27
29
|
:exchange_type => :direct,
|
30
|
+
:exchange_arguments => {}, # Passed as :arguments to Bunny::Channel#exchange
|
28
31
|
:hooks => {}
|
29
32
|
}.freeze
|
30
33
|
|
@@ -40,10 +43,23 @@ module Sneakers
|
|
40
43
|
end
|
41
44
|
|
42
45
|
def merge!(hash)
|
43
|
-
|
44
|
-
|
45
|
-
|
46
|
-
|
46
|
+
hash = hash.dup
|
47
|
+
|
48
|
+
# parse vhost from amqp if vhost is not specified explicitly, only
|
49
|
+
# if we're not given a connection to use.
|
50
|
+
if hash[:connection].nil?
|
51
|
+
if hash[:vhost].nil? && !hash[:amqp].nil?
|
52
|
+
hash[:vhost] = AMQ::Settings.parse_amqp_url(hash[:amqp]).fetch(:vhost, '/')
|
53
|
+
end
|
54
|
+
else
|
55
|
+
# If we are given a Bunny object, ignore params we'd otherwise use to
|
56
|
+
# create one. This removes any question about where config params are
|
57
|
+
# coming from, and makes it more likely that downstream code that needs
|
58
|
+
# this info gets it from the right place.
|
59
|
+
[:vhost, :amqp, :heartbeat].each do |k|
|
60
|
+
hash.delete(k)
|
61
|
+
@hash.delete(k)
|
62
|
+
end
|
47
63
|
end
|
48
64
|
|
49
65
|
@hash.merge!(hash)
|
@@ -55,5 +71,16 @@ module Sneakers
|
|
55
71
|
instance.merge! hash
|
56
72
|
instance
|
57
73
|
end
|
74
|
+
|
75
|
+
def inspect_with_redaction
|
76
|
+
redacted = self.class.new
|
77
|
+
redacted.merge! to_hash
|
78
|
+
|
79
|
+
# redact passwords
|
80
|
+
redacted[:amqp] = redacted[:amqp].sub(/(?<=\Aamqp:\/)[^@]+(?=@)/, "<redacted>")
|
81
|
+
return redacted.inspect_without_redaction
|
82
|
+
end
|
83
|
+
alias_method :inspect_without_redaction, :inspect
|
84
|
+
alias_method :inspect, :inspect_with_redaction
|
58
85
|
end
|
59
86
|
end
|
data/lib/sneakers/publisher.rb
CHANGED
@@ -23,7 +23,7 @@ module Sneakers
|
|
23
23
|
@bunny = Bunny.new(@opts[:amqp], heartbeat: @opts[:heartbeat], vhost: @opts[:vhost], :logger => Sneakers::logger)
|
24
24
|
@bunny.start
|
25
25
|
@channel = @bunny.create_channel
|
26
|
-
@exchange = @channel.exchange(@opts[:exchange], type: @opts[:exchange_type], durable: @opts[:durable])
|
26
|
+
@exchange = @channel.exchange(@opts[:exchange], type: @opts[:exchange_type], durable: @opts[:durable], arguments: @opts[:exchange_arguments])
|
27
27
|
end
|
28
28
|
|
29
29
|
def connected?
|
data/lib/sneakers/queue.rb
CHANGED
@@ -16,7 +16,10 @@ class Sneakers::Queue
|
|
16
16
|
# :ack
|
17
17
|
#
|
18
18
|
def subscribe(worker)
|
19
|
-
|
19
|
+
# If we've already got a bunny object, use it. This allows people to
|
20
|
+
# specify all kinds of options we don't need to know about (e.g. for ssl).
|
21
|
+
@bunny = @opts[:connection]
|
22
|
+
@bunny ||= create_bunny_connection
|
20
23
|
@bunny.start
|
21
24
|
|
22
25
|
@channel = @bunny.create_channel
|
@@ -25,7 +28,8 @@ class Sneakers::Queue
|
|
25
28
|
exchange_name = @opts[:exchange]
|
26
29
|
@exchange = @channel.exchange(exchange_name,
|
27
30
|
:type => @opts[:exchange_type],
|
28
|
-
:durable => @opts[:durable]
|
31
|
+
:durable => @opts[:durable],
|
32
|
+
:arguments => @opts[:exchange_arguments])
|
29
33
|
|
30
34
|
routing_key = @opts[:routing_key] || @name
|
31
35
|
routing_keys = [*routing_key]
|
@@ -60,4 +64,9 @@ class Sneakers::Queue
|
|
60
64
|
@consumer.cancel if @consumer
|
61
65
|
@consumer = nil
|
62
66
|
end
|
67
|
+
|
68
|
+
def create_bunny_connection
|
69
|
+
Bunny.new(@opts[:amqp], :vhost => @opts[:vhost], :heartbeat => @opts[:heartbeat], :logger => Sneakers::logger)
|
70
|
+
end
|
71
|
+
private :create_bunny_connection
|
63
72
|
end
|
data/lib/sneakers/runner.rb
CHANGED
@@ -63,19 +63,22 @@ module Sneakers
|
|
63
63
|
config
|
64
64
|
end
|
65
65
|
|
66
|
-
|
67
|
-
|
66
|
+
private
|
67
|
+
|
68
|
+
def make_serverengine_config
|
68
69
|
# From Sneakers#setup_general_logger, there's support for a Logger object
|
69
70
|
# in CONFIG[:log]. However, serverengine takes an object in :logger.
|
70
71
|
# Pass our logger object so there's no issue about sometimes passing a
|
71
72
|
# file and sometimes an object.
|
72
|
-
|
73
|
-
|
74
|
-
Sneakers::CONFIG.merge(@conf).merge({
|
73
|
+
serverengine_config = Sneakers::CONFIG.merge(@conf)
|
74
|
+
serverengine_config.merge!(
|
75
75
|
:logger => Sneakers.logger,
|
76
76
|
:worker_type => 'process',
|
77
77
|
:worker_classes => @worker_classes
|
78
|
-
|
78
|
+
)
|
79
|
+
serverengine_config.delete(:log)
|
80
|
+
|
81
|
+
serverengine_config
|
79
82
|
end
|
80
83
|
end
|
81
84
|
|
data/lib/sneakers/tasks.rb
CHANGED
@@ -5,7 +5,13 @@ task :environment
|
|
5
5
|
|
6
6
|
namespace :sneakers do
|
7
7
|
desc "Start work (set $WORKERS=Klass1,Klass2)"
|
8
|
-
task :run
|
8
|
+
task :run do
|
9
|
+
Sneakers.server = true
|
10
|
+
Rake::Task['environment'].invoke
|
11
|
+
|
12
|
+
if defined?(::Rails)
|
13
|
+
::Rails.application.eager_load!
|
14
|
+
end
|
9
15
|
|
10
16
|
workers, missing_workers = Sneakers::Utils.parse_workers(ENV['WORKERS'])
|
11
17
|
|
@@ -26,7 +32,7 @@ Please set the classes of the workers you want to run like so:
|
|
26
32
|
EOF
|
27
33
|
exit(1)
|
28
34
|
end
|
29
|
-
opts = (ENV['WORKER_COUNT'].
|
35
|
+
opts = (!ENV['WORKER_COUNT'].nil? ? {:workers => ENV['WORKER_COUNT'].to_i} : {})
|
30
36
|
r = Sneakers::Runner.new(workers, opts)
|
31
37
|
|
32
38
|
r.run
|
data/lib/sneakers/version.rb
CHANGED
data/lib/sneakers/workergroup.rb
CHANGED
@@ -19,7 +19,11 @@ module Sneakers
|
|
19
19
|
def run
|
20
20
|
after_fork
|
21
21
|
|
22
|
-
|
22
|
+
# Allocate single thread pool if share_threads is set. This improves load balancing
|
23
|
+
# when used with many workers.
|
24
|
+
pool = config[:share_threads] ? Thread.pool(config[:threads]) : nil
|
25
|
+
|
26
|
+
@workers = config[:worker_classes].map{|w| w.new(nil, pool) }
|
23
27
|
# if more than one worker this should be per worker
|
24
28
|
# accumulate clients and consumers as well
|
25
29
|
@workers.each do |worker|
|
@@ -27,7 +31,7 @@ module Sneakers
|
|
27
31
|
end
|
28
32
|
# end per worker
|
29
33
|
#
|
30
|
-
until @stop_flag.wait_for_set(
|
34
|
+
until @stop_flag.wait_for_set(Sneakers::CONFIG[:amqp_heartbeat])
|
31
35
|
Sneakers.logger.debug("Heartbeat: running threads [#{Thread.list.count}]")
|
32
36
|
# report aggregated stats?
|
33
37
|
end
|
@@ -44,4 +48,3 @@ module Sneakers
|
|
44
48
|
|
45
49
|
end
|
46
50
|
end
|
47
|
-
|
data/sneakers.gemspec
CHANGED
@@ -17,7 +17,7 @@ Gem::Specification.new do |gem|
|
|
17
17
|
gem.test_files = gem.files.grep(/^(test|spec|features)\//)
|
18
18
|
gem.require_paths = ['lib']
|
19
19
|
gem.add_dependency 'serverengine', '~> 1.5.5'
|
20
|
-
gem.add_dependency 'bunny', '
|
20
|
+
gem.add_dependency 'bunny', ['>= 1.7.0', '<= 2.0.0']
|
21
21
|
gem.add_dependency 'thread', '~> 0.1.7'
|
22
22
|
gem.add_dependency 'thor'
|
23
23
|
|
@@ -2,67 +2,89 @@ require 'spec_helper'
|
|
2
2
|
|
3
3
|
describe Sneakers::Configuration do
|
4
4
|
|
5
|
-
|
6
|
-
|
7
|
-
|
8
|
-
|
5
|
+
describe 'with a connection' do
|
6
|
+
let(:connection) { Object.new }
|
7
|
+
|
8
|
+
it 'does not use vhost option if it is specified' do
|
9
|
+
url = 'amqp://foo:bar@localhost:5672/foobarvhost'
|
10
|
+
with_env('RABBITMQ_URL', url) do
|
11
|
+
config = Sneakers::Configuration.new
|
12
|
+
config.merge!({ :vhost => 'test_host', :connection => connection })
|
13
|
+
config.has_key?(:vhost).must_equal false
|
14
|
+
end
|
9
15
|
end
|
10
|
-
end
|
11
16
|
|
12
|
-
|
13
|
-
|
17
|
+
it 'does not amqp option if it is specified' do
|
18
|
+
url = 'amqp://foo:bar@localhost:5672'
|
14
19
|
config = Sneakers::Configuration.new
|
15
|
-
config
|
20
|
+
config.merge!({ :amqp => url, :connection => connection })
|
21
|
+
config.has_key?(:vhost).must_equal false
|
16
22
|
end
|
17
23
|
end
|
18
24
|
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
|
25
|
+
describe 'without a connection' do
|
26
|
+
it 'should assign a default value for :amqp' do
|
27
|
+
with_env('RABBITMQ_URL', nil) do
|
28
|
+
config = Sneakers::Configuration.new
|
29
|
+
config[:amqp].must_equal 'amqp://guest:guest@localhost:5672'
|
30
|
+
end
|
24
31
|
end
|
25
|
-
end
|
26
32
|
|
27
|
-
|
28
|
-
|
29
|
-
|
30
|
-
|
31
|
-
|
33
|
+
it 'should assign a default value for :vhost' do
|
34
|
+
with_env('RABBITMQ_URL', nil) do
|
35
|
+
config = Sneakers::Configuration.new
|
36
|
+
config[:vhost].must_equal '/'
|
37
|
+
end
|
32
38
|
end
|
33
|
-
end
|
34
39
|
|
35
|
-
|
36
|
-
|
37
|
-
|
38
|
-
|
39
|
-
|
40
|
-
|
41
|
-
config[:vhost].must_equal 'testvhost'
|
40
|
+
it 'should read the value for amqp from RABBITMQ_URL' do
|
41
|
+
url = 'amqp://foo:bar@localhost:5672'
|
42
|
+
with_env('RABBITMQ_URL', url) do
|
43
|
+
config = Sneakers::Configuration.new
|
44
|
+
config[:amqp].must_equal url
|
45
|
+
end
|
42
46
|
end
|
43
|
-
end
|
44
47
|
|
45
|
-
|
46
|
-
|
47
|
-
|
48
|
-
|
49
|
-
|
50
|
-
|
48
|
+
it 'should read the value for vhost from RABBITMQ_URL' do
|
49
|
+
url = 'amqp://foo:bar@localhost:5672/foobarvhost'
|
50
|
+
with_env('RABBITMQ_URL', url) do
|
51
|
+
config = Sneakers::Configuration.new
|
52
|
+
config[:vhost].must_equal 'foobarvhost'
|
53
|
+
end
|
54
|
+
end
|
55
|
+
|
56
|
+
it 'should parse vhost from amqp option' do
|
57
|
+
env_url = 'amqp://foo:bar@localhost:5672/foobarvhost'
|
58
|
+
with_env('RABBITMQ_URL', env_url) do
|
59
|
+
url = 'amqp://foo:bar@localhost:5672/testvhost'
|
60
|
+
config = Sneakers::Configuration.new
|
61
|
+
config.merge!({ :amqp => url })
|
62
|
+
config[:vhost].must_equal 'testvhost'
|
63
|
+
end
|
64
|
+
end
|
51
65
|
|
52
|
-
|
53
|
-
|
54
|
-
with_env('RABBITMQ_URL', url) do
|
66
|
+
it 'should not parse vhost from amqp option if vhost is specified explicitly' do
|
67
|
+
url = 'amqp://foo:bar@localhost:5672/foobarvhost'
|
55
68
|
config = Sneakers::Configuration.new
|
56
|
-
config.merge!({ :vhost => 'test_host' })
|
69
|
+
config.merge!({ :amqp => url, :vhost => 'test_host' })
|
57
70
|
config[:vhost].must_equal 'test_host'
|
58
71
|
end
|
59
|
-
end
|
60
72
|
|
61
|
-
|
62
|
-
|
63
|
-
|
64
|
-
|
65
|
-
|
73
|
+
it 'should use vhost option if it is specified' do
|
74
|
+
url = 'amqp://foo:bar@localhost:5672/foobarvhost'
|
75
|
+
with_env('RABBITMQ_URL', url) do
|
76
|
+
config = Sneakers::Configuration.new
|
77
|
+
config.merge!({ :vhost => 'test_host' })
|
78
|
+
config[:vhost].must_equal 'test_host'
|
79
|
+
end
|
80
|
+
end
|
81
|
+
|
82
|
+
it 'should use default vhost if vhost is not specified in amqp option' do
|
83
|
+
url = 'amqp://foo:bar@localhost:5672'
|
84
|
+
config = Sneakers::Configuration.new
|
85
|
+
config.merge!({ :amqp => url })
|
86
|
+
config[:vhost].must_equal '/'
|
87
|
+
end
|
66
88
|
end
|
67
89
|
|
68
90
|
def with_env(key, value)
|
@@ -60,11 +60,12 @@ describe Sneakers::Publisher do
|
|
60
60
|
amqp: 'amqp://someuser:somepassword@somehost:5672',
|
61
61
|
heartbeat: 1, exchange: 'another_exchange',
|
62
62
|
exchange_type: :topic,
|
63
|
+
exchange_arguments: { 'x-arg' => 'value' },
|
63
64
|
log: logger,
|
64
65
|
durable: false)
|
65
66
|
|
66
67
|
channel = Object.new
|
67
|
-
mock(channel).exchange('another_exchange', type: :topic, durable: false) do
|
68
|
+
mock(channel).exchange('another_exchange', type: :topic, durable: false, arguments: { 'x-arg' => 'value' }) do
|
68
69
|
mock(Object.new).publish('test msg', routing_key: 'downloads')
|
69
70
|
end
|
70
71
|
|
data/spec/sneakers/queue_spec.rb
CHANGED
@@ -1,8 +1,6 @@
|
|
1
1
|
require 'spec_helper'
|
2
2
|
require 'sneakers'
|
3
3
|
|
4
|
-
|
5
|
-
|
6
4
|
describe Sneakers::Queue do
|
7
5
|
let :queue_vars do
|
8
6
|
{
|
@@ -12,103 +10,137 @@ describe Sneakers::Queue do
|
|
12
10
|
:heartbeat => 2,
|
13
11
|
:vhost => '/',
|
14
12
|
:exchange => "sneakers",
|
15
|
-
:exchange_type => :direct
|
13
|
+
:exchange_type => :direct,
|
14
|
+
:exchange_arguments => { 'x-arg' => 'value' }
|
16
15
|
}
|
17
16
|
end
|
18
17
|
|
19
18
|
before do
|
20
19
|
Sneakers.configure
|
21
20
|
|
22
|
-
@
|
21
|
+
@mkworker = Object.new
|
22
|
+
stub(@mkworker).opts { { :exchange => 'test-exchange' } }
|
23
23
|
@mkchan = Object.new
|
24
|
+
mock(@mkchan).prefetch(25)
|
24
25
|
@mkex = Object.new
|
25
26
|
@mkqueue = Object.new
|
26
|
-
@mkqueue_nondurable = Object.new
|
27
|
-
@mkworker = Object.new
|
28
|
-
|
29
|
-
mock(@mkbunny).start {}
|
30
|
-
mock(@mkbunny).create_channel{ @mkchan }
|
31
|
-
mock(Bunny).new(anything, :vhost => '/', :heartbeat => 2){ @mkbunny }
|
32
|
-
|
33
|
-
mock(@mkchan).prefetch(25)
|
34
|
-
|
35
|
-
stub(@mkworker).opts { { :exchange => 'test-exchange' } }
|
36
27
|
end
|
37
28
|
|
38
|
-
describe
|
29
|
+
describe 'with our own Bunny object' do
|
39
30
|
before do
|
40
|
-
|
31
|
+
@mkbunny = Object.new
|
32
|
+
@mkqueue_nondurable = Object.new
|
33
|
+
|
34
|
+
mock(@mkbunny).start {}
|
35
|
+
mock(@mkbunny).create_channel{ @mkchan }
|
36
|
+
mock(Bunny).new(anything, :vhost => '/', :heartbeat => 2){ @mkbunny }
|
41
37
|
end
|
42
38
|
|
43
|
-
|
44
|
-
|
45
|
-
|
39
|
+
describe "#subscribe with sneakers exchange" do
|
40
|
+
before do
|
41
|
+
mock(@mkchan).exchange("sneakers",
|
42
|
+
:type => :direct,
|
43
|
+
:durable => true,
|
44
|
+
:arguments => { 'x-arg' => 'value' }){ @mkex }
|
45
|
+
end
|
46
46
|
|
47
|
-
|
48
|
-
|
47
|
+
it "should setup a bunny queue according to configuration values" do
|
48
|
+
mock(@mkchan).queue("downloads", :durable => true) { @mkqueue }
|
49
|
+
q = Sneakers::Queue.new("downloads", queue_vars)
|
49
50
|
|
50
|
-
|
51
|
-
|
51
|
+
mock(@mkqueue).bind(@mkex, :routing_key => "downloads")
|
52
|
+
mock(@mkqueue).subscribe(:block => false, :manual_ack => true)
|
52
53
|
|
53
|
-
|
54
|
-
|
55
|
-
q = Sneakers::Queue.new("downloads",
|
56
|
-
queue_vars.merge(:routing_key => ["alpha", "beta"]))
|
54
|
+
q.subscribe(@mkworker)
|
55
|
+
end
|
57
56
|
|
58
|
-
|
59
|
-
|
60
|
-
|
57
|
+
it "supports multiple routing_keys" do
|
58
|
+
mock(@mkchan).queue("downloads", :durable => true) { @mkqueue }
|
59
|
+
q = Sneakers::Queue.new("downloads",
|
60
|
+
queue_vars.merge(:routing_key => ["alpha", "beta"]))
|
61
61
|
|
62
|
-
|
63
|
-
|
62
|
+
mock(@mkqueue).bind(@mkex, :routing_key => "alpha")
|
63
|
+
mock(@mkqueue).bind(@mkex, :routing_key => "beta")
|
64
|
+
mock(@mkqueue).subscribe(:block => false, :manual_ack => true)
|
64
65
|
|
65
|
-
|
66
|
-
|
67
|
-
@handler = Object.new
|
68
|
-
worker_opts = { :handler => @handler }
|
69
|
-
stub(@mkworker).opts { worker_opts }
|
70
|
-
mock(@handler).new(@mkchan, @mkqueue, worker_opts).once
|
71
|
-
|
72
|
-
stub(@mkqueue).bind
|
73
|
-
stub(@mkqueue).subscribe
|
74
|
-
q = Sneakers::Queue.new("downloads", queue_vars)
|
75
|
-
q.subscribe(@mkworker)
|
76
|
-
end
|
66
|
+
q.subscribe(@mkworker)
|
67
|
+
end
|
77
68
|
|
78
|
-
|
79
|
-
|
80
|
-
|
81
|
-
|
69
|
+
it "will use whatever handler the worker specifies" do
|
70
|
+
mock(@mkchan).queue("downloads", :durable => true) { @mkqueue }
|
71
|
+
@handler = Object.new
|
72
|
+
worker_opts = { :handler => @handler }
|
73
|
+
stub(@mkworker).opts { worker_opts }
|
74
|
+
mock(@handler).new(@mkchan, @mkqueue, worker_opts).once
|
75
|
+
|
76
|
+
stub(@mkqueue).bind
|
77
|
+
stub(@mkqueue).subscribe
|
78
|
+
q = Sneakers::Queue.new("downloads", queue_vars)
|
79
|
+
q.subscribe(@mkworker)
|
80
|
+
end
|
82
81
|
|
83
|
-
|
84
|
-
|
82
|
+
it "creates a non-durable queue if :queue_durable => false" do
|
83
|
+
mock(@mkchan).queue("test_nondurable", :durable => false) { @mkqueue_nondurable }
|
84
|
+
queue_vars[:queue_durable] = false
|
85
|
+
q = Sneakers::Queue.new("test_nondurable", queue_vars)
|
85
86
|
|
86
|
-
|
87
|
-
|
88
|
-
end
|
89
|
-
end
|
87
|
+
mock(@mkqueue_nondurable).bind(@mkex, :routing_key => "test_nondurable")
|
88
|
+
mock(@mkqueue_nondurable).subscribe(:block => false, :manual_ack => true)
|
90
89
|
|
91
|
-
|
92
|
-
|
93
|
-
|
94
|
-
queue_vars[:exchange] = ""
|
95
|
-
mock(@mkchan).exchange("", :type => :direct, :durable => true){ @mkex }
|
90
|
+
q.subscribe(@mkworker)
|
91
|
+
myqueue = q.instance_variable_get(:@queue)
|
92
|
+
end
|
96
93
|
end
|
97
94
|
|
98
|
-
|
99
|
-
|
100
|
-
|
101
|
-
|
102
|
-
|
103
|
-
|
95
|
+
describe "#subscribe with default exchange" do
|
96
|
+
before do
|
97
|
+
# expect default exchange
|
98
|
+
queue_vars[:exchange] = ""
|
99
|
+
mock(@mkchan).exchange("", :type => :direct, :durable => true){ @mkex }
|
100
|
+
end
|
104
101
|
|
105
|
-
|
106
|
-
|
102
|
+
it "does not bind to exchange" do
|
103
|
+
mock(@mkchan).queue("downloads", :durable => true) { @mkqueue }
|
104
|
+
@handler = Object.new
|
105
|
+
worker_opts = { :handler => @handler }
|
106
|
+
stub(@mkworker).opts { worker_opts }
|
107
|
+
mock(@handler).new(@mkchan, @mkqueue, worker_opts).once
|
108
|
+
|
109
|
+
stub(@mkqueue).bind do
|
110
|
+
raise "bind should not be called"
|
111
|
+
end
|
112
|
+
|
113
|
+
stub(@mkqueue).subscribe
|
114
|
+
q = Sneakers::Queue.new("downloads", queue_vars)
|
115
|
+
q.subscribe(@mkworker)
|
107
116
|
end
|
117
|
+
end
|
118
|
+
end
|
108
119
|
|
109
|
-
|
110
|
-
|
111
|
-
|
120
|
+
describe 'with an externally-provided connection' do
|
121
|
+
describe '#subscribe' do
|
122
|
+
before do
|
123
|
+
@external_connection = Bunny.new
|
124
|
+
mock(@external_connection).start {}
|
125
|
+
mock(@external_connection).create_channel{ @mkchan }
|
126
|
+
mock(@mkchan).exchange("sneakers",
|
127
|
+
:type => :direct,
|
128
|
+
:durable => true,
|
129
|
+
:arguments => { 'x-arg' => 'value' }){ @mkex }
|
130
|
+
|
131
|
+
queue_name = 'foo'
|
132
|
+
mock(@mkchan).queue(queue_name, :durable => true) { @mkqueue }
|
133
|
+
mock(@mkqueue).bind(@mkex, :routing_key => queue_name)
|
134
|
+
mock(@mkqueue).subscribe(:block => false, :manual_ack => true)
|
135
|
+
|
136
|
+
my_vars = queue_vars.merge(:connection => @external_connection)
|
137
|
+
@q = Sneakers::Queue.new(queue_name, my_vars)
|
138
|
+
end
|
139
|
+
|
140
|
+
it 'uses that object' do
|
141
|
+
@q.subscribe(@mkworker)
|
142
|
+
@q.instance_variable_get(:@bunny).must_equal @external_connection
|
143
|
+
end
|
112
144
|
end
|
113
145
|
end
|
114
146
|
end
|
@@ -24,3 +24,21 @@ describe Sneakers::Runner do
|
|
24
24
|
end
|
25
25
|
end
|
26
26
|
end
|
27
|
+
|
28
|
+
describe Sneakers::RunnerConfig do
|
29
|
+
let(:logger) { Logger.new("logtest.log") }
|
30
|
+
let(:runner_config) { Sneakers::Runner.new([]).instance_variable_get("@runnerconfig") }
|
31
|
+
|
32
|
+
before { Sneakers.configure(log: logger) }
|
33
|
+
|
34
|
+
|
35
|
+
describe "#reload_config!" do
|
36
|
+
it "must not have :log key" do
|
37
|
+
runner_config.reload_config!.has_key?(:log).must_equal false
|
38
|
+
end
|
39
|
+
|
40
|
+
it "must have :logger key as an instance of Logger" do
|
41
|
+
runner_config.reload_config![:logger].is_a?(Logger).must_equal true
|
42
|
+
end
|
43
|
+
end
|
44
|
+
end
|
@@ -54,14 +54,16 @@ describe Sneakers do
|
|
54
54
|
|
55
55
|
|
56
56
|
describe '#setup_general_logger' do
|
57
|
+
let(:logger_class) { ServerEngine::DaemonLogger }
|
58
|
+
|
57
59
|
it 'should detect a string and configure a logger' do
|
58
60
|
Sneakers.configure(:log => 'sneakers.log')
|
59
|
-
Sneakers.logger.kind_of?(
|
61
|
+
Sneakers.logger.kind_of?(logger_class).must_equal(true)
|
60
62
|
end
|
61
63
|
|
62
64
|
it 'should detect a file-like thing and configure a logger' do
|
63
65
|
Sneakers.configure(:log => STDOUT)
|
64
|
-
Sneakers.logger.kind_of?(
|
66
|
+
Sneakers.logger.kind_of?(logger_class).must_equal(true)
|
65
67
|
end
|
66
68
|
|
67
69
|
it 'should detect an actual logger and configure it' do
|
@@ -171,15 +171,18 @@ describe Sneakers::Worker do
|
|
171
171
|
:timeout_job_after => 5,
|
172
172
|
:prefetch => 10,
|
173
173
|
:threads => 10,
|
174
|
+
:share_threads => false,
|
174
175
|
:durable => true,
|
175
176
|
:ack => true,
|
176
177
|
:amqp => "amqp://guest:guest@localhost:5672",
|
177
178
|
:vhost => "/",
|
178
179
|
:exchange => "sneakers",
|
179
180
|
:exchange_type => :direct,
|
181
|
+
:exchange_arguments => {},
|
180
182
|
:hooks => {},
|
181
183
|
:handler => Sneakers::Handlers::Oneshot,
|
182
|
-
:heartbeat
|
184
|
+
:heartbeat => 2,
|
185
|
+
:amqp_heartbeat => 10
|
183
186
|
)
|
184
187
|
end
|
185
188
|
|
@@ -196,15 +199,18 @@ describe Sneakers::Worker do
|
|
196
199
|
:timeout_job_after => 1,
|
197
200
|
:prefetch => 40,
|
198
201
|
:threads => 50,
|
202
|
+
:share_threads => false,
|
199
203
|
:durable => false,
|
200
204
|
:ack => false,
|
201
205
|
:amqp => "amqp://guest:guest@localhost:5672",
|
202
206
|
:vhost => "/",
|
203
207
|
:exchange => "dummy",
|
204
208
|
:exchange_type => :direct,
|
209
|
+
:exchange_arguments => {},
|
205
210
|
:hooks => {},
|
206
211
|
:handler => Sneakers::Handlers::Oneshot,
|
207
|
-
:heartbeat => 5
|
212
|
+
:heartbeat => 5,
|
213
|
+
:amqp_heartbeat => 10
|
208
214
|
)
|
209
215
|
end
|
210
216
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: sneakers
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.0
|
4
|
+
version: 1.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Dotan Nahum
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2015-
|
11
|
+
date: 2015-08-20 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: serverengine
|
@@ -28,16 +28,22 @@ dependencies:
|
|
28
28
|
name: bunny
|
29
29
|
requirement: !ruby/object:Gem::Requirement
|
30
30
|
requirements:
|
31
|
-
- - "
|
31
|
+
- - ">="
|
32
32
|
- !ruby/object:Gem::Version
|
33
33
|
version: 1.7.0
|
34
|
+
- - "<="
|
35
|
+
- !ruby/object:Gem::Version
|
36
|
+
version: 2.0.0
|
34
37
|
type: :runtime
|
35
38
|
prerelease: false
|
36
39
|
version_requirements: !ruby/object:Gem::Requirement
|
37
40
|
requirements:
|
38
|
-
- - "
|
41
|
+
- - ">="
|
39
42
|
- !ruby/object:Gem::Version
|
40
43
|
version: 1.7.0
|
44
|
+
- - "<="
|
45
|
+
- !ruby/object:Gem::Version
|
46
|
+
version: 2.0.0
|
41
47
|
- !ruby/object:Gem::Dependency
|
42
48
|
name: thread
|
43
49
|
requirement: !ruby/object:Gem::Requirement
|