stalking 0.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/.gitignore ADDED
@@ -0,0 +1,17 @@
1
+ *.gem
2
+ *.rbc
3
+ .bundle
4
+ .config
5
+ .yardoc
6
+ Gemfile.lock
7
+ InstalledFiles
8
+ _yardoc
9
+ coverage
10
+ doc/
11
+ lib/bundler/man
12
+ pkg
13
+ rdoc
14
+ spec/reports
15
+ test/tmp
16
+ test/version_tmp
17
+ tmp
data/Gemfile ADDED
@@ -0,0 +1,4 @@
1
+ source 'https://rubygems.org'
2
+
3
+ # Specify your gem's dependencies in stalking.gemspec
4
+ gemspec
data/LICENSE.txt ADDED
@@ -0,0 +1,22 @@
1
+ Copyright (c) 2012 Benjamin Vetter
2
+
3
+ MIT License
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining
6
+ a copy of this software and associated documentation files (the
7
+ "Software"), to deal in the Software without restriction, including
8
+ without limitation the rights to use, copy, modify, merge, publish,
9
+ distribute, sublicense, and/or sell copies of the Software, and to
10
+ permit persons to whom the Software is furnished to do so, subject to
11
+ the following conditions:
12
+
13
+ The above copyright notice and this permission notice shall be
14
+ included in all copies or substantial portions of the Software.
15
+
16
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
17
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
18
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
19
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
20
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
21
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
22
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,187 @@
1
+
2
+ # Stalking
3
+
4
+ Asynchronous job processing using the beanstalkd queue server with automatic
5
+ reconnects and failover for your ruby projects.
6
+
7
+ Background jobs are essential to every project these days and the tools should
8
+ be as reliable as possible.
9
+
10
+ Beanstalkd is perfectly suited for background processing and the stalker gem
11
+ has long been my favorite to integrate beanstalkd into my projects.
12
+ Unfortunately, stalker does not provide automatic reconnects and failover.
13
+
14
+ The DSL provided by stalking is very similar to the DSL of stalker with only
15
+ minimal changes driven by design decisions and its additional feature set.
16
+
17
+ ## Installation
18
+
19
+ Add this line to your application's Gemfile:
20
+
21
+ gem 'stalking', :git => 'git://github.com/mrkamel/stalking.git'
22
+
23
+ And then execute:
24
+
25
+ $ bundle
26
+
27
+ ## Usage
28
+
29
+ Stalking provides two modes of operation, a producer and a consumer mode.
30
+
31
+ Enqueuing new jobs is as easy as:
32
+
33
+ ```ruby
34
+ Stalking::Producer.new.enqueue "name", "key" => "value"
35
+ ```
36
+
37
+ This will try to connect to a beanstalkd instance running on `localhost:11300`
38
+ to enqueue a new job named `job` with the provided arguments. The arguments
39
+ will be serialized using JSON before being enqueued.
40
+
41
+ To specify a `priority`, `ttr` or `delay`, please use:
42
+
43
+ ```ruby
44
+ Stalking::Producer.new.enqueue "name", { "key" => "value" }, :ttr => 360, :delay => 60, :pri => 100
45
+ ```
46
+
47
+ You can of course use multiple beanstalkd instances, by using:
48
+
49
+ ```ruby
50
+ producer = Stalking::Producer.new(:servers => ["192.168.1.100:11300", "192.168.1.101"])
51
+ producer.enqueue "name", "key" => "value"
52
+ ```
53
+
54
+ Here, everytime you enqueue a new job, stalking will randomly choose a server
55
+ from your list to send the job to. Thus, you can e.g. create a producer for
56
+ local jobs (like e.g. for local background image processing) and another
57
+ producer for global jobs:
58
+
59
+ ```ruby
60
+ global_queue = Stalking::Producer.new(:servers => ["192.168.1.100:11300", "192.168.1.101"])
61
+ local_queue = Stalking::Producer.new
62
+ ```
63
+
64
+ If a server from your list is down or unreachable, stalking will use another
65
+ one from your list. Only in case all your servers are down, `enqueue` will
66
+ return false. However, please carefully read the `Caveats` section.
67
+
68
+ You can even set the `ttr`, `priority` and `delay` options
69
+ on a per-producer base:
70
+
71
+ ```ruby
72
+ producer = Stalking::Producer.new(:pri => 100, :delay => 60, :ttr => 360)
73
+ ```
74
+
75
+ To set an upper limit for servers to be tried out for queuing use the `tries`
76
+ option:
77
+
78
+ ```ruby
79
+ producer = Stalking::Producer.new(:servers => [..., ..., ...], :tries => 2)
80
+ ```
81
+
82
+ To consume jobs in a background worker, please use the following DSL, which is
83
+ quite similar to stalker's DSL:
84
+
85
+ ```ruby
86
+ Stalking::Consumer.new do
87
+ job "name" do |args|
88
+ # Do something
89
+ end
90
+
91
+ job "another" do |args|
92
+ # Do something
93
+ end
94
+
95
+ ...
96
+ end
97
+ ```
98
+
99
+ After being specified, the consumer will immediately block and wait to consume
100
+ jobs from beanstalkd.
101
+
102
+ Additionally, you can specify before, after and error hooks:
103
+
104
+ ```ruby
105
+ Stalking::Consumer.new do
106
+ before do |name, args|
107
+ # Executed before a job runs
108
+ end
109
+
110
+ after do |name, args|
111
+ # Executed after a job has run
112
+ end
113
+
114
+ error do |e, name, args|
115
+ # Executed when job exited abnormally
116
+ end
117
+
118
+ job "name" do |args|
119
+ # The job itself
120
+ end
121
+ end
122
+ ```
123
+
124
+ If you save this file to e.g. `jobs.rb`, you can execute your worker like:
125
+
126
+ ```
127
+ $ stalking jobs
128
+ ```
129
+
130
+ Again, by default stalking connects to localhost:11300, but you can specify a
131
+ pool of servers to consume jobs from:
132
+
133
+ ```ruby
134
+ Stalking::Consumer.new(:servers => ["192.168.1.100:11300", "192.168.1.101:11300"]) do
135
+ ...
136
+ end
137
+ ```
138
+
139
+ Please note that, due to the implementation of beanstalk-client, the consumer
140
+ requires all pool servers to be up. Otherwise, the consumer will enter a
141
+ reconnect loop, until all servers are up again. Thus, stalking will try to
142
+ reconnect to the servers when a connection is lost and it's recommended for
143
+ consumers to connect to a single server, usually.
144
+
145
+ That should be it.
146
+
147
+ ## Caveats
148
+
149
+ Having producers which try to reconnect and/or try to connect to other servers
150
+ when a single server is down, of course require some time for the error
151
+ detection and processing. This usually should not be a big issue, except there
152
+ is some temporary lag between the producer and the beanstalkd server. The
153
+ producer then has to wait for a timeout and while the producer is waiting for
154
+ the timeout it will block your application. The default timeout is 250
155
+ milliseconds, but you can change the timeout via:
156
+
157
+ ```ruby
158
+ Stalking::Producer.new :timeout => 0.1 # Set timeout to 100 milliseconds
159
+ ```
160
+
161
+ The optimal timeout for your setup depends on your network connection, the
162
+ distance of your servers, etc. You should experiment a bit with it if you
163
+ experience some kind of problems. However, i don't emphasize to use stalking
164
+ producers which communicate with beanstalkd servers over WAN connections.
165
+ Only local, data-center connections to beanstalkd servers should be used.
166
+
167
+ # Error Logging
168
+
169
+ To log connection errors, you can provide a logger to the consumers as well
170
+ as producers:
171
+
172
+ ```ruby
173
+ Stalking::Producer.new :logger => ...
174
+ Stalking::Consumer.new :logger => ...
175
+ ```
176
+
177
+ You can hand out any kind of logging object, which responds to an `error`
178
+ method and takes a string error message.
179
+
180
+ ## Contributing
181
+
182
+ 1. Fork it
183
+ 2. Create your feature branch (`git checkout -b my-new-feature`)
184
+ 3. Commit your changes (`git commit -am 'Add some feature'`)
185
+ 4. Push to the branch (`git push origin my-new-feature`)
186
+ 5. Create new Pull Request
187
+
data/Rakefile ADDED
@@ -0,0 +1,9 @@
1
+ require "bundler/gem_tasks"
2
+ require "rake/testtask"
3
+
4
+ Rake::TestTask.new(:test) do |t|
5
+ t.libs << "lib"
6
+ t.pattern = "test/**/*_test.rb"
7
+ t.verbose = true
8
+ end
9
+
data/bin/stalking ADDED
@@ -0,0 +1,11 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ $:.unshift File.expand_path("../../lib", __FILE__)
4
+
5
+ require "stalking"
6
+
7
+ file = ARGV.shift or abort("Usage: stalking [file]")
8
+ file = "./#{file}" unless file.match(/^[\/.]/)
9
+
10
+ load file
11
+
data/lib/stalking.rb ADDED
@@ -0,0 +1,10 @@
1
+
2
+ require "stalking/version"
3
+ require "beanstalk-client"
4
+ require "timeout"
5
+ require "json"
6
+
7
+ require "stalking/handlers"
8
+ require "stalking/consumer"
9
+ require "stalking/producer"
10
+
@@ -0,0 +1,117 @@
1
+
2
+ module Stalking
3
+ class Consumer
4
+ def initialize(options = {}, &block)
5
+ @servers = options[:servers] || ["localhost:11300"]
6
+
7
+ @logger = options[:logger]
8
+
9
+ @handlers = Handlers.new
10
+
11
+ if block_given?
12
+ @handlers.instance_eval &block
13
+
14
+ work
15
+ end
16
+ end
17
+
18
+ def handle(handlers, name, args)
19
+ handlers.each do |handler|
20
+ handler.call name, args
21
+ end
22
+ end
23
+
24
+ def handle_error(e, name, args)
25
+ @handlers.error_handlers.each do |handler|
26
+ handler.call e, name, args
27
+ end
28
+ end
29
+
30
+ private
31
+
32
+ def work
33
+ setup
34
+
35
+ begin
36
+ loop do
37
+ job = connection.reserve
38
+
39
+ exit if @quit
40
+
41
+ busy do
42
+ name, args = JSON.parse(job.body)
43
+
44
+ if handler = @handlers.job_handlers[name]
45
+ begin
46
+ Timeout::timeout(job.ttr - 1) do
47
+ handle @handlers.before_handlers, name, args
48
+
49
+ handler.call args
50
+
51
+ handle @handlers.after_handlers, name, args
52
+ end
53
+ rescue Beanstalk::NotConnected => e
54
+ raise e # Re-raise
55
+ rescue Timeout::Error, StandardError => e
56
+ handle_error e, name, args
57
+ end
58
+ end
59
+ end
60
+
61
+ job.delete
62
+ end
63
+ rescue Beanstalk::NotConnected => e
64
+ log_error e
65
+
66
+ sleep 1
67
+
68
+ reconnect
69
+
70
+ retry
71
+ end
72
+ end
73
+
74
+ def setup
75
+ trap "QUIT" do
76
+ exit unless @busy
77
+
78
+ @quit = true
79
+ end
80
+ end
81
+
82
+ def busy
83
+ @busy = true
84
+
85
+ yield
86
+ ensure
87
+ @busy = false
88
+ end
89
+
90
+ def connect
91
+ @connection ||= Beanstalk::Pool.new(@servers)
92
+
93
+ @handlers.job_handlers.keys.each { |name| @connection.watch name }
94
+
95
+ @connection
96
+ rescue Beanstalk::NotConnected => e
97
+ log_error e
98
+
99
+ sleep 1
100
+
101
+ retry
102
+ end
103
+
104
+ alias :connection :connect
105
+
106
+ def reconnect
107
+ @connection = nil
108
+
109
+ connect
110
+ end
111
+
112
+ def log_error(e)
113
+ @logger.error("Stalking error: #{e.message}") if @logger
114
+ end
115
+ end
116
+ end
117
+
@@ -0,0 +1,31 @@
1
+
2
+ module Stalking
3
+ class Handlers
4
+ attr_reader :before_handlers, :after_handlers, :error_handlers, :job_handlers
5
+
6
+ def initialize
7
+ @before_handlers = []
8
+ @after_handlers = []
9
+ @error_handlers = []
10
+
11
+ @job_handlers = {}
12
+ end
13
+
14
+ def before(&block)
15
+ @before_handlers.push block
16
+ end
17
+
18
+ def after(&block)
19
+ @after_handlers.push block
20
+ end
21
+
22
+ def error(&block)
23
+ @error_handlers.push block
24
+ end
25
+
26
+ def job(name, &block)
27
+ @job_handlers[name] = block
28
+ end
29
+ end
30
+ end
31
+
@@ -0,0 +1,74 @@
1
+
2
+ module Stalking
3
+ class Producer
4
+ def initialize(options = {}, &block)
5
+ @timeout = options[:timeout] || 0.25
6
+
7
+ @pri = options[:pri] || 65536
8
+ @delay = options[:delay] || 0
9
+ @ttr = options[:ttr] || 120
10
+
11
+ @logger = options[:logger]
12
+
13
+ @servers = options[:servers] || ["localhost:11300"]
14
+
15
+ @tries = options[:tries] || @servers.size
16
+
17
+ @connections = {}
18
+ end
19
+
20
+ def enqueue(name, args = {}, options = {})
21
+ # Send the job to a random server.
22
+
23
+ @servers.shuffle.first(@tries).each do |server|
24
+ begin
25
+ Timeout::timeout(@timeout) do
26
+ @connections[server] ||= Beanstalk::Pool.new([server])
27
+
28
+ enq @connections[server], name, args, options
29
+ end
30
+
31
+ return true
32
+ rescue Beanstalk::NotConnected
33
+ # Connecting to the beanstalk server has failed.
34
+ # Let's try to reconnect and enqueue afterwards.
35
+
36
+ begin
37
+ Timeout::timeout(@timeout) do
38
+ @connections[server] = Beanstalk::Pool.new([server])
39
+
40
+ enq @connections[server], name, args, options
41
+ end
42
+
43
+ return true
44
+ rescue Beanstalk::NotConnected, Timeout::Error, StandardError => e
45
+ log_error e
46
+ end
47
+ rescue Timeout::Error, StandardError => e
48
+ log_error e
49
+ end
50
+ end
51
+
52
+ # All servers are currently unavailable.
53
+ # Enqueuing the job has failed.
54
+
55
+ false
56
+ end
57
+
58
+ private
59
+
60
+ def enq(connection, name, args = {}, options = {})
61
+ pri = options[:pri] || @pri
62
+ delay = options[:delay] || @delay
63
+ ttr = options[:ttr] || @ttr
64
+
65
+ connection.use name
66
+ connection.put [name, args].to_json, pri, delay, ttr
67
+ end
68
+
69
+ def log_error(e)
70
+ @logger.error("Stalking error: #{e.message}") if @logger
71
+ end
72
+ end
73
+ end
74
+
@@ -0,0 +1,3 @@
1
+ module Stalking
2
+ VERSION = "0.0.1"
3
+ end
data/stalking.gemspec ADDED
@@ -0,0 +1,24 @@
1
+ # -*- encoding: utf-8 -*-
2
+ $:.push File.expand_path("../lib", __FILE__)
3
+ require "stalking/version"
4
+
5
+ Gem::Specification.new do |s|
6
+ s.name = "stalking"
7
+ s.version = Stalking::VERSION
8
+ s.authors = ["Benjamin Vetter"]
9
+ s.email = ["vetter@flakks.com"]
10
+ s.homepage = "https://github.com/mrkamel/stalking"
11
+ s.summary = %q{Asynchronous jobs using beanstalkd with automatic reconnects and failover}
12
+ s.description = %q{Asynchronous job processing using the beanstalkd queue server with automatic reconnects and failover}
13
+
14
+ s.files = `git ls-files`.split("\n")
15
+ s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
16
+ s.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
17
+ s.require_paths = ["lib"]
18
+
19
+ s.add_development_dependency "rake"
20
+
21
+ s.add_dependency "beanstalk-client"
22
+ s.add_dependency "json_pure"
23
+ end
24
+
@@ -0,0 +1,136 @@
1
+
2
+ # We've got to mock a beanstalkd server.
3
+
4
+ module Beanstalk
5
+ class TestJob
6
+ attr_reader :ttr, :body
7
+
8
+ def initialize(connection, name, body, ttr)
9
+ @connection = connection
10
+ @name = name
11
+ @body = body
12
+ @ttr = ttr
13
+ end
14
+
15
+ def delete
16
+ @connection.disturb!
17
+
18
+ @connection.queues[@name].shift
19
+ end
20
+ end
21
+
22
+ class Pool
23
+ @@connections = []
24
+
25
+ # Multiple pool servers are currently not implemented for testing purposes.
26
+
27
+ def self.new(server)
28
+ @connection = @@connections.detect { |connection| connection.server == server }
29
+
30
+ if @connection
31
+ @connection.up! if @connection.disconnected?
32
+
33
+ return @connection
34
+ end
35
+
36
+ unless @connection
37
+ @connection = TestConnection.new(server)
38
+
39
+ @@connections.push @connection
40
+ end
41
+
42
+ @connection
43
+ end
44
+
45
+ def self.clear!
46
+ @@connections = []
47
+ end
48
+ end
49
+
50
+ class TestConnection
51
+ attr_reader :server
52
+
53
+ def initialize(server)
54
+ @server = server
55
+
56
+ @availability = :available
57
+ @delay = 0
58
+
59
+ @watches = []
60
+
61
+ @queues = {}
62
+ end
63
+
64
+ def disturb!
65
+ raise Beanstalk::NotConnected unless available?
66
+
67
+ sleep @delay
68
+ end
69
+
70
+ def delay=(delay)
71
+ @delay = delay
72
+ end
73
+
74
+ def use(name)
75
+ disturb!
76
+
77
+ @current = name
78
+ end
79
+
80
+ def put(data, pri = 65536, delay = 0, ttr = 120)
81
+ disturb!
82
+
83
+ @queues[@current] ||= []
84
+ @queues[@current].push [data, pri, delay, ttr]
85
+ end
86
+
87
+ def watch(name)
88
+ disturb!
89
+
90
+ @watches.push name
91
+ end
92
+
93
+ def queues
94
+ @queues
95
+ end
96
+
97
+ def up!
98
+ @availability = :available
99
+ end
100
+
101
+ def available?
102
+ @availability == :available
103
+ end
104
+
105
+ def disconnect!
106
+ @availability = :disconnected
107
+ end
108
+
109
+ def disconnected?
110
+ @availability == :disconnected
111
+ end
112
+
113
+ def down!
114
+ @availability = :down
115
+ end
116
+
117
+ def down?
118
+ @availability == :down
119
+ end
120
+
121
+ def reserve
122
+ disturb!
123
+
124
+ @watches.each do |name|
125
+ if @queues[name] && !@queues[name].empty?
126
+ data, pri, delay, ttr = @queues[name].first
127
+
128
+ return TestJob.new(self, name, data, ttr)
129
+ end
130
+ end
131
+
132
+ nil
133
+ end
134
+ end
135
+ end
136
+
@@ -0,0 +1,208 @@
1
+
2
+ $:.unshift File.expand_path("../../lib", __FILE__)
3
+
4
+ require "stalking"
5
+ require "test/unit"
6
+
7
+ require File.expand_path("../beanstalk", __FILE__)
8
+ require File.expand_path("../test_logger", __FILE__)
9
+
10
+ class Stalking::ConsumerTest < Test::Unit::TestCase
11
+ def setup
12
+ Beanstalk::Pool.clear!
13
+ end
14
+
15
+ def test_before
16
+ Stalking::Producer.new.enqueue "test", "a" => "b"
17
+
18
+ res = nil
19
+
20
+ exception = assert_raises(Exception) do
21
+ Stalking::Consumer.new do
22
+ before do |name, args|
23
+ res = [name, args]
24
+
25
+ raise Exception
26
+ end
27
+
28
+ job "test" do |args|
29
+ # Nothing
30
+ end
31
+ end
32
+ end
33
+
34
+ assert_equal ["test", { "a" => "b" }], res
35
+ end
36
+
37
+ def test_after
38
+ Stalking::Producer.new.enqueue "test", "a" => "b"
39
+
40
+ res = nil
41
+
42
+ exception = assert_raises(Exception) do
43
+ Stalking::Consumer.new do
44
+ after do |name, args|
45
+ res = [name, args]
46
+
47
+ raise Exception
48
+ end
49
+
50
+ job "test" do |args|
51
+ # Nothing
52
+ end
53
+ end
54
+ end
55
+
56
+ assert_equal ["test", { "a" => "b" }], res
57
+ end
58
+
59
+ def test_error
60
+ Stalking::Producer.new.enqueue "test", "a" => "b"
61
+
62
+ res = nil
63
+
64
+ exception = assert_raises(Exception) do
65
+ Stalking::Consumer.new do
66
+ error do |e, name, args|
67
+ res = [e.message, name, args]
68
+
69
+ raise Exception
70
+ end
71
+
72
+ job "test" do |args|
73
+ raise "An error has occurred"
74
+ end
75
+ end
76
+ end
77
+
78
+ assert_equal ["An error has occurred", "test", { "a" => "b" }], res
79
+ end
80
+
81
+ def test_job
82
+ Stalking::Producer.new.enqueue "test", "a" => "b"
83
+
84
+ res = nil
85
+
86
+ exception = assert_raises(Exception) do
87
+ Stalking::Consumer.new do
88
+ job "test" do |args|
89
+ res = ["test", args]
90
+
91
+ raise Exception
92
+ end
93
+ end
94
+ end
95
+
96
+ assert_equal ["test", { "a" => "b" }], res
97
+ end
98
+
99
+ def test_work
100
+ Stalking::Producer.new.enqueue "test1", "a" => "b"
101
+ Stalking::Producer.new.enqueue "test2", "a" => "b"
102
+ Stalking::Producer.new.enqueue "test3", "a" => "b"
103
+
104
+ res = []
105
+
106
+ exception = assert_raises(Exception) do
107
+ Stalking::Consumer.new do
108
+ job "test1" do |args|
109
+ res.push "test1"
110
+ end
111
+
112
+ job "test2" do |args|
113
+ res.push "test2"
114
+ end
115
+
116
+ job "test3" do |args|
117
+ res.push "test3"
118
+
119
+ raise Exception
120
+ end
121
+ end
122
+ end
123
+
124
+ assert_equal ["test1", "test2", "test3"], res
125
+ end
126
+
127
+ def test_quit
128
+ raise NotImplementedError
129
+ end
130
+
131
+ def test_job_not_deleted
132
+ Stalking::Producer.new.enqueue "test", "a" => "b"
133
+
134
+ exception = assert_raises(Exception) do
135
+ Stalking::Consumer.new do
136
+ job "test" do |args|
137
+ raise Exception
138
+ end
139
+ end
140
+ end
141
+
142
+ res = nil
143
+
144
+ exception = assert_raises(Exception) do
145
+ Stalking::Consumer.new do
146
+ job "test" do |args|
147
+ res = ["test", args]
148
+
149
+ raise Exception
150
+ end
151
+ end
152
+ end
153
+
154
+ assert_equal ["test", {"a" => "b" }], res
155
+ end
156
+
157
+ def test_job_deleted
158
+ Stalking::Producer.new.enqueue "test", "a" => "b"
159
+
160
+ res = nil
161
+
162
+ exception = assert_raises(NoMethodError) do
163
+ Stalking::Consumer.new do
164
+ job "test" do |args|
165
+ res = ["test", args]
166
+ end
167
+ end
168
+ end
169
+
170
+ assert_equal ["test", { "a" => "b" }], res
171
+ assert_equal "undefined method `body' for nil:NilClass", exception.message
172
+ end
173
+
174
+ def test_reconnect
175
+ Stalking::Producer.new.enqueue "test", "a" => "b"
176
+
177
+ logger = TestLogger.new
178
+
179
+ res = []
180
+
181
+ exception = assert_raises(Exception) do
182
+ Stalking::Consumer.new(:logger => logger) do
183
+ job "test" do |args|
184
+ if res.empty?
185
+ res.push ["down"]
186
+
187
+ # Simulate a temporary beanstalk connection error.
188
+
189
+ Beanstalk::Pool.new(["localhost:11300"]).disconnect!
190
+ else
191
+ res.push ["test", args]
192
+
193
+ raise Exception
194
+ end
195
+ end
196
+ end
197
+ end
198
+
199
+ assert_equal [["down"], ["test", { "a" => "b" }]], res
200
+
201
+ assert_equal ["Stalking error: Beanstalk::NotConnected"], logger.messages
202
+ end
203
+
204
+ def test_logger
205
+ # Already tested
206
+ end
207
+ end
208
+
@@ -0,0 +1,86 @@
1
+
2
+ $:.unshift File.expand_path("../../lib", __FILE__)
3
+
4
+ require "stalking"
5
+ require "test/unit"
6
+
7
+ require File.expand_path("../test_logger", __FILE__)
8
+
9
+ class Stalking::ProducerTest < Test::Unit::TestCase
10
+ def setup
11
+ Beanstalk::Pool.clear!
12
+ end
13
+
14
+ def test_enqueue
15
+ Stalking::Producer.new.enqueue "test", { "key" => "value" }, :pri => 100, :delay => 100, :ttr => 100
16
+
17
+ assert_equal ['["test",{"key":"value"}]', 100, 100, 100], Beanstalk::Pool.new(["localhost:11300"]).queues["test"].last
18
+ end
19
+
20
+ def test_defaults
21
+ Stalking::Producer.new.enqueue "test", "key" => "value"
22
+
23
+ assert_equal ['["test",{"key":"value"}]', 65536, 0, 120], Beanstalk::Pool.new(["localhost:11300"]).queues["test"].last
24
+ end
25
+
26
+ def test_reconnect
27
+ producer = Stalking::Producer.new(:servers => ["server1", "server2"])
28
+
29
+ server1 = Beanstalk::Pool.new(["server1"])
30
+ server2 = Beanstalk::Pool.new(["server2"])
31
+
32
+ srand 0
33
+
34
+ producer.enqueue "test", "up" => "true"
35
+
36
+ assert_equal ['["test",{"up":"true"}]', 65536, 0, 120], server1.queues["test"].last
37
+
38
+ srand 0
39
+
40
+ server1.disconnect!
41
+
42
+ producer.enqueue "test", "down" => "disconnected"
43
+
44
+ assert_equal ['["test",{"down":"disconnected"}]', 65536, 0, 120], server1.queues["test"].last
45
+
46
+ server1.down!
47
+
48
+ producer.enqueue "test", "down" => "permanent"
49
+
50
+ assert_equal ['["test",{"down":"permanent"}]', 65536, 0, 120], server2.queues["test"].last
51
+ end
52
+
53
+ def test_timeout
54
+ producer = Stalking::Producer.new(:servers => ["server1", "server2"])
55
+
56
+ server1 = Beanstalk::Pool.new(["server1"])
57
+ server2 = Beanstalk::Pool.new(["server2"])
58
+
59
+ srand 0
60
+
61
+ producer.enqueue "test", "up" => "true"
62
+
63
+ assert_equal ['["test",{"up":"true"}]', 65536, 0, 120], server1.queues["test"].last
64
+
65
+ srand 0
66
+
67
+ server1.delay = 5
68
+
69
+ producer.enqueue "test", "down" => "delayed"
70
+
71
+ assert_equal ['["test",{"down":"delayed"}]', 65536, 0, 120], server2.queues["test"].last
72
+ end
73
+
74
+ def test_logger
75
+ logger = TestLogger.new
76
+
77
+ producer = Stalking::Producer.new(:logger => logger)
78
+
79
+ Beanstalk::Pool.new(["localhost:11300"]).down!
80
+
81
+ producer.enqueue "test", "up" => "true"
82
+
83
+ assert_equal ["Stalking error: Beanstalk::NotConnected"], logger.messages
84
+ end
85
+ end
86
+
@@ -0,0 +1,13 @@
1
+
2
+ class TestLogger
3
+ attr_reader :messages
4
+
5
+ def initialize
6
+ @messages = []
7
+ end
8
+
9
+ def error(msg)
10
+ @messages.push msg
11
+ end
12
+ end
13
+
metadata ADDED
@@ -0,0 +1,111 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: stalking
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.0.1
5
+ prerelease:
6
+ platform: ruby
7
+ authors:
8
+ - Benjamin Vetter
9
+ autorequire:
10
+ bindir: bin
11
+ cert_chain: []
12
+ date: 2013-06-28 00:00:00.000000000 Z
13
+ dependencies:
14
+ - !ruby/object:Gem::Dependency
15
+ name: rake
16
+ requirement: !ruby/object:Gem::Requirement
17
+ none: false
18
+ requirements:
19
+ - - ! '>='
20
+ - !ruby/object:Gem::Version
21
+ version: '0'
22
+ type: :development
23
+ prerelease: false
24
+ version_requirements: !ruby/object:Gem::Requirement
25
+ none: false
26
+ requirements:
27
+ - - ! '>='
28
+ - !ruby/object:Gem::Version
29
+ version: '0'
30
+ - !ruby/object:Gem::Dependency
31
+ name: beanstalk-client
32
+ requirement: !ruby/object:Gem::Requirement
33
+ none: false
34
+ requirements:
35
+ - - ! '>='
36
+ - !ruby/object:Gem::Version
37
+ version: '0'
38
+ type: :runtime
39
+ prerelease: false
40
+ version_requirements: !ruby/object:Gem::Requirement
41
+ none: false
42
+ requirements:
43
+ - - ! '>='
44
+ - !ruby/object:Gem::Version
45
+ version: '0'
46
+ - !ruby/object:Gem::Dependency
47
+ name: json_pure
48
+ requirement: !ruby/object:Gem::Requirement
49
+ none: false
50
+ requirements:
51
+ - - ! '>='
52
+ - !ruby/object:Gem::Version
53
+ version: '0'
54
+ type: :runtime
55
+ prerelease: false
56
+ version_requirements: !ruby/object:Gem::Requirement
57
+ none: false
58
+ requirements:
59
+ - - ! '>='
60
+ - !ruby/object:Gem::Version
61
+ version: '0'
62
+ description: Asynchronous job processing using the beanstalkd queue server with automatic
63
+ reconnects and failover
64
+ email:
65
+ - vetter@flakks.com
66
+ executables:
67
+ - stalking
68
+ extensions: []
69
+ extra_rdoc_files: []
70
+ files:
71
+ - .gitignore
72
+ - Gemfile
73
+ - LICENSE.txt
74
+ - README.md
75
+ - Rakefile
76
+ - bin/stalking
77
+ - lib/stalking.rb
78
+ - lib/stalking/consumer.rb
79
+ - lib/stalking/handlers.rb
80
+ - lib/stalking/producer.rb
81
+ - lib/stalking/version.rb
82
+ - stalking.gemspec
83
+ - test/stalking/beanstalk.rb
84
+ - test/stalking/consumer_test.rb
85
+ - test/stalking/producer_test.rb
86
+ - test/stalking/test_logger.rb
87
+ homepage: https://github.com/mrkamel/stalking
88
+ licenses: []
89
+ post_install_message:
90
+ rdoc_options: []
91
+ require_paths:
92
+ - lib
93
+ required_ruby_version: !ruby/object:Gem::Requirement
94
+ none: false
95
+ requirements:
96
+ - - ! '>='
97
+ - !ruby/object:Gem::Version
98
+ version: '0'
99
+ required_rubygems_version: !ruby/object:Gem::Requirement
100
+ none: false
101
+ requirements:
102
+ - - ! '>='
103
+ - !ruby/object:Gem::Version
104
+ version: '0'
105
+ requirements: []
106
+ rubyforge_project:
107
+ rubygems_version: 1.8.23
108
+ signing_key:
109
+ specification_version: 3
110
+ summary: Asynchronous jobs using beanstalkd with automatic reconnects and failover
111
+ test_files: []