stalking 0.0.1

Sign up to get free protection for your applications and to get access to all the features.
data/.gitignore ADDED
@@ -0,0 +1,17 @@
1
+ *.gem
2
+ *.rbc
3
+ .bundle
4
+ .config
5
+ .yardoc
6
+ Gemfile.lock
7
+ InstalledFiles
8
+ _yardoc
9
+ coverage
10
+ doc/
11
+ lib/bundler/man
12
+ pkg
13
+ rdoc
14
+ spec/reports
15
+ test/tmp
16
+ test/version_tmp
17
+ tmp
data/Gemfile ADDED
@@ -0,0 +1,4 @@
1
+ source 'https://rubygems.org'
2
+
3
+ # Specify your gem's dependencies in stalking.gemspec
4
+ gemspec
data/LICENSE.txt ADDED
@@ -0,0 +1,22 @@
1
+ Copyright (c) 2012 Benjamin Vetter
2
+
3
+ MIT License
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining
6
+ a copy of this software and associated documentation files (the
7
+ "Software"), to deal in the Software without restriction, including
8
+ without limitation the rights to use, copy, modify, merge, publish,
9
+ distribute, sublicense, and/or sell copies of the Software, and to
10
+ permit persons to whom the Software is furnished to do so, subject to
11
+ the following conditions:
12
+
13
+ The above copyright notice and this permission notice shall be
14
+ included in all copies or substantial portions of the Software.
15
+
16
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
17
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
18
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
19
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
20
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
21
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
22
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,187 @@
1
+
2
+ # Stalking
3
+
4
+ Asynchronous job processing using the beanstalkd queue server with automatic
5
+ reconnects and failover for your ruby projects.
6
+
7
+ Background jobs are essential to every project these days and the tools should
8
+ be as reliable as possible.
9
+
10
+ Beanstalkd is perfectly suited for background processing and the stalker gem
11
+ has long been my favorite to integrate beanstalkd into my projects.
12
+ Unfortunately, stalker does not provide automatic reconnects and failover.
13
+
14
+ The DSL provided by stalking is very similar to the DSL of stalker with only
15
+ minimal changes driven by design decisions and its additional feature set.
16
+
17
+ ## Installation
18
+
19
+ Add this line to your application's Gemfile:
20
+
21
+ gem 'stalking', :git => 'git://github.com/mrkamel/stalking.git'
22
+
23
+ And then execute:
24
+
25
+ $ bundle
26
+
27
+ ## Usage
28
+
29
+ Stalking provides two modes of operation, a producer and a consumer mode.
30
+
31
+ Enqueuing new jobs is as easy as:
32
+
33
+ ```ruby
34
+ Stalking::Producer.new.enqueue "name", "key" => "value"
35
+ ```
36
+
37
+ This will try to connect to a beanstalkd instance running on `localhost:11300`
38
+ to enqueue a new job named `job` with the provided arguments. The arguments
39
+ will be serialized using JSON before being enqueued.
40
+
41
+ To specify a `priority`, `ttr` or `delay`, please use:
42
+
43
+ ```ruby
44
+ Stalking::Producer.new.enqueue "name", { "key" => "value" }, :ttr => 360, :delay => 60, :pri => 100
45
+ ```
46
+
47
+ You can of course use multiple beanstalkd instances, by using:
48
+
49
+ ```ruby
50
+ producer = Stalking::Producer.new(:servers => ["192.168.1.100:11300", "192.168.1.101"])
51
+ producer.enqueue "name", "key" => "value"
52
+ ```
53
+
54
+ Here, everytime you enqueue a new job, stalking will randomly choose a server
55
+ from your list to send the job to. Thus, you can e.g. create a producer for
56
+ local jobs (like e.g. for local background image processing) and another
57
+ producer for global jobs:
58
+
59
+ ```ruby
60
+ global_queue = Stalking::Producer.new(:servers => ["192.168.1.100:11300", "192.168.1.101"])
61
+ local_queue = Stalking::Producer.new
62
+ ```
63
+
64
+ If a server from your list is down or unreachable, stalking will use another
65
+ one from your list. Only in case all your servers are down, `enqueue` will
66
+ return false. However, please carefully read the `Caveats` section.
67
+
68
+ You can even set the `ttr`, `priority` and `delay` options
69
+ on a per-producer base:
70
+
71
+ ```ruby
72
+ producer = Stalking::Producer.new(:pri => 100, :delay => 60, :ttr => 360)
73
+ ```
74
+
75
+ To set an upper limit for servers to be tried out for queuing use the `tries`
76
+ option:
77
+
78
+ ```ruby
79
+ producer = Stalking::Producer.new(:servers => [..., ..., ...], :tries => 2)
80
+ ```
81
+
82
+ To consume jobs in a background worker, please use the following DSL, which is
83
+ quite similar to stalker's DSL:
84
+
85
+ ```ruby
86
+ Stalking::Consumer.new do
87
+ job "name" do |args|
88
+ # Do something
89
+ end
90
+
91
+ job "another" do |args|
92
+ # Do something
93
+ end
94
+
95
+ ...
96
+ end
97
+ ```
98
+
99
+ After being specified, the consumer will immediately block and wait to consume
100
+ jobs from beanstalkd.
101
+
102
+ Additionally, you can specify before, after and error hooks:
103
+
104
+ ```ruby
105
+ Stalking::Consumer.new do
106
+ before do |name, args|
107
+ # Executed before a job runs
108
+ end
109
+
110
+ after do |name, args|
111
+ # Executed after a job has run
112
+ end
113
+
114
+ error do |e, name, args|
115
+ # Executed when job exited abnormally
116
+ end
117
+
118
+ job "name" do |args|
119
+ # The job itself
120
+ end
121
+ end
122
+ ```
123
+
124
+ If you save this file to e.g. `jobs.rb`, you can execute your worker like:
125
+
126
+ ```
127
+ $ stalking jobs
128
+ ```
129
+
130
+ Again, by default stalking connects to localhost:11300, but you can specify a
131
+ pool of servers to consume jobs from:
132
+
133
+ ```ruby
134
+ Stalking::Consumer.new(:servers => ["192.168.1.100:11300", "192.168.1.101:11300"]) do
135
+ ...
136
+ end
137
+ ```
138
+
139
+ Please note that, due to the implementation of beanstalk-client, the consumer
140
+ requires all pool servers to be up. Otherwise, the consumer will enter a
141
+ reconnect loop, until all servers are up again. Thus, stalking will try to
142
+ reconnect to the servers when a connection is lost and it's recommended for
143
+ consumers to connect to a single server, usually.
144
+
145
+ That should be it.
146
+
147
+ ## Caveats
148
+
149
+ Having producers which try to reconnect and/or try to connect to other servers
150
+ when a single server is down, of course require some time for the error
151
+ detection and processing. This usually should not be a big issue, except there
152
+ is some temporary lag between the producer and the beanstalkd server. The
153
+ producer then has to wait for a timeout and while the producer is waiting for
154
+ the timeout it will block your application. The default timeout is 250
155
+ milliseconds, but you can change the timeout via:
156
+
157
+ ```ruby
158
+ Stalking::Producer.new :timeout => 0.1 # Set timeout to 100 milliseconds
159
+ ```
160
+
161
+ The optimal timeout for your setup depends on your network connection, the
162
+ distance of your servers, etc. You should experiment a bit with it if you
163
+ experience some kind of problems. However, i don't emphasize to use stalking
164
+ producers which communicate with beanstalkd servers over WAN connections.
165
+ Only local, data-center connections to beanstalkd servers should be used.
166
+
167
+ # Error Logging
168
+
169
+ To log connection errors, you can provide a logger to the consumers as well
170
+ as producers:
171
+
172
+ ```ruby
173
+ Stalking::Producer.new :logger => ...
174
+ Stalking::Consumer.new :logger => ...
175
+ ```
176
+
177
+ You can hand out any kind of logging object, which responds to an `error`
178
+ method and takes a string error message.
179
+
180
+ ## Contributing
181
+
182
+ 1. Fork it
183
+ 2. Create your feature branch (`git checkout -b my-new-feature`)
184
+ 3. Commit your changes (`git commit -am 'Add some feature'`)
185
+ 4. Push to the branch (`git push origin my-new-feature`)
186
+ 5. Create new Pull Request
187
+
data/Rakefile ADDED
@@ -0,0 +1,9 @@
1
+ require "bundler/gem_tasks"
2
+ require "rake/testtask"
3
+
4
+ Rake::TestTask.new(:test) do |t|
5
+ t.libs << "lib"
6
+ t.pattern = "test/**/*_test.rb"
7
+ t.verbose = true
8
+ end
9
+
data/bin/stalking ADDED
@@ -0,0 +1,11 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ $:.unshift File.expand_path("../../lib", __FILE__)
4
+
5
+ require "stalking"
6
+
7
+ file = ARGV.shift or abort("Usage: stalking [file]")
8
+ file = "./#{file}" unless file.match(/^[\/.]/)
9
+
10
+ load file
11
+
data/lib/stalking.rb ADDED
@@ -0,0 +1,10 @@
1
+
2
+ require "stalking/version"
3
+ require "beanstalk-client"
4
+ require "timeout"
5
+ require "json"
6
+
7
+ require "stalking/handlers"
8
+ require "stalking/consumer"
9
+ require "stalking/producer"
10
+
@@ -0,0 +1,117 @@
1
+
2
+ module Stalking
3
+ class Consumer
4
+ def initialize(options = {}, &block)
5
+ @servers = options[:servers] || ["localhost:11300"]
6
+
7
+ @logger = options[:logger]
8
+
9
+ @handlers = Handlers.new
10
+
11
+ if block_given?
12
+ @handlers.instance_eval &block
13
+
14
+ work
15
+ end
16
+ end
17
+
18
+ def handle(handlers, name, args)
19
+ handlers.each do |handler|
20
+ handler.call name, args
21
+ end
22
+ end
23
+
24
+ def handle_error(e, name, args)
25
+ @handlers.error_handlers.each do |handler|
26
+ handler.call e, name, args
27
+ end
28
+ end
29
+
30
+ private
31
+
32
+ def work
33
+ setup
34
+
35
+ begin
36
+ loop do
37
+ job = connection.reserve
38
+
39
+ exit if @quit
40
+
41
+ busy do
42
+ name, args = JSON.parse(job.body)
43
+
44
+ if handler = @handlers.job_handlers[name]
45
+ begin
46
+ Timeout::timeout(job.ttr - 1) do
47
+ handle @handlers.before_handlers, name, args
48
+
49
+ handler.call args
50
+
51
+ handle @handlers.after_handlers, name, args
52
+ end
53
+ rescue Beanstalk::NotConnected => e
54
+ raise e # Re-raise
55
+ rescue Timeout::Error, StandardError => e
56
+ handle_error e, name, args
57
+ end
58
+ end
59
+ end
60
+
61
+ job.delete
62
+ end
63
+ rescue Beanstalk::NotConnected => e
64
+ log_error e
65
+
66
+ sleep 1
67
+
68
+ reconnect
69
+
70
+ retry
71
+ end
72
+ end
73
+
74
+ def setup
75
+ trap "QUIT" do
76
+ exit unless @busy
77
+
78
+ @quit = true
79
+ end
80
+ end
81
+
82
+ def busy
83
+ @busy = true
84
+
85
+ yield
86
+ ensure
87
+ @busy = false
88
+ end
89
+
90
+ def connect
91
+ @connection ||= Beanstalk::Pool.new(@servers)
92
+
93
+ @handlers.job_handlers.keys.each { |name| @connection.watch name }
94
+
95
+ @connection
96
+ rescue Beanstalk::NotConnected => e
97
+ log_error e
98
+
99
+ sleep 1
100
+
101
+ retry
102
+ end
103
+
104
+ alias :connection :connect
105
+
106
+ def reconnect
107
+ @connection = nil
108
+
109
+ connect
110
+ end
111
+
112
+ def log_error(e)
113
+ @logger.error("Stalking error: #{e.message}") if @logger
114
+ end
115
+ end
116
+ end
117
+
@@ -0,0 +1,31 @@
1
+
2
+ module Stalking
3
+ class Handlers
4
+ attr_reader :before_handlers, :after_handlers, :error_handlers, :job_handlers
5
+
6
+ def initialize
7
+ @before_handlers = []
8
+ @after_handlers = []
9
+ @error_handlers = []
10
+
11
+ @job_handlers = {}
12
+ end
13
+
14
+ def before(&block)
15
+ @before_handlers.push block
16
+ end
17
+
18
+ def after(&block)
19
+ @after_handlers.push block
20
+ end
21
+
22
+ def error(&block)
23
+ @error_handlers.push block
24
+ end
25
+
26
+ def job(name, &block)
27
+ @job_handlers[name] = block
28
+ end
29
+ end
30
+ end
31
+
@@ -0,0 +1,74 @@
1
+
2
+ module Stalking
3
+ class Producer
4
+ def initialize(options = {}, &block)
5
+ @timeout = options[:timeout] || 0.25
6
+
7
+ @pri = options[:pri] || 65536
8
+ @delay = options[:delay] || 0
9
+ @ttr = options[:ttr] || 120
10
+
11
+ @logger = options[:logger]
12
+
13
+ @servers = options[:servers] || ["localhost:11300"]
14
+
15
+ @tries = options[:tries] || @servers.size
16
+
17
+ @connections = {}
18
+ end
19
+
20
+ def enqueue(name, args = {}, options = {})
21
+ # Send the job to a random server.
22
+
23
+ @servers.shuffle.first(@tries).each do |server|
24
+ begin
25
+ Timeout::timeout(@timeout) do
26
+ @connections[server] ||= Beanstalk::Pool.new([server])
27
+
28
+ enq @connections[server], name, args, options
29
+ end
30
+
31
+ return true
32
+ rescue Beanstalk::NotConnected
33
+ # Connecting to the beanstalk server has failed.
34
+ # Let's try to reconnect and enqueue afterwards.
35
+
36
+ begin
37
+ Timeout::timeout(@timeout) do
38
+ @connections[server] = Beanstalk::Pool.new([server])
39
+
40
+ enq @connections[server], name, args, options
41
+ end
42
+
43
+ return true
44
+ rescue Beanstalk::NotConnected, Timeout::Error, StandardError => e
45
+ log_error e
46
+ end
47
+ rescue Timeout::Error, StandardError => e
48
+ log_error e
49
+ end
50
+ end
51
+
52
+ # All servers are currently unavailable.
53
+ # Enqueuing the job has failed.
54
+
55
+ false
56
+ end
57
+
58
+ private
59
+
60
+ def enq(connection, name, args = {}, options = {})
61
+ pri = options[:pri] || @pri
62
+ delay = options[:delay] || @delay
63
+ ttr = options[:ttr] || @ttr
64
+
65
+ connection.use name
66
+ connection.put [name, args].to_json, pri, delay, ttr
67
+ end
68
+
69
+ def log_error(e)
70
+ @logger.error("Stalking error: #{e.message}") if @logger
71
+ end
72
+ end
73
+ end
74
+
@@ -0,0 +1,3 @@
1
+ module Stalking
2
+ VERSION = "0.0.1"
3
+ end
data/stalking.gemspec ADDED
@@ -0,0 +1,24 @@
1
+ # -*- encoding: utf-8 -*-
2
+ $:.push File.expand_path("../lib", __FILE__)
3
+ require "stalking/version"
4
+
5
+ Gem::Specification.new do |s|
6
+ s.name = "stalking"
7
+ s.version = Stalking::VERSION
8
+ s.authors = ["Benjamin Vetter"]
9
+ s.email = ["vetter@flakks.com"]
10
+ s.homepage = "https://github.com/mrkamel/stalking"
11
+ s.summary = %q{Asynchronous jobs using beanstalkd with automatic reconnects and failover}
12
+ s.description = %q{Asynchronous job processing using the beanstalkd queue server with automatic reconnects and failover}
13
+
14
+ s.files = `git ls-files`.split("\n")
15
+ s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
16
+ s.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
17
+ s.require_paths = ["lib"]
18
+
19
+ s.add_development_dependency "rake"
20
+
21
+ s.add_dependency "beanstalk-client"
22
+ s.add_dependency "json_pure"
23
+ end
24
+
@@ -0,0 +1,136 @@
1
+
2
+ # We've got to mock a beanstalkd server.
3
+
4
+ module Beanstalk
5
+ class TestJob
6
+ attr_reader :ttr, :body
7
+
8
+ def initialize(connection, name, body, ttr)
9
+ @connection = connection
10
+ @name = name
11
+ @body = body
12
+ @ttr = ttr
13
+ end
14
+
15
+ def delete
16
+ @connection.disturb!
17
+
18
+ @connection.queues[@name].shift
19
+ end
20
+ end
21
+
22
+ class Pool
23
+ @@connections = []
24
+
25
+ # Multiple pool servers are currently not implemented for testing purposes.
26
+
27
+ def self.new(server)
28
+ @connection = @@connections.detect { |connection| connection.server == server }
29
+
30
+ if @connection
31
+ @connection.up! if @connection.disconnected?
32
+
33
+ return @connection
34
+ end
35
+
36
+ unless @connection
37
+ @connection = TestConnection.new(server)
38
+
39
+ @@connections.push @connection
40
+ end
41
+
42
+ @connection
43
+ end
44
+
45
+ def self.clear!
46
+ @@connections = []
47
+ end
48
+ end
49
+
50
+ class TestConnection
51
+ attr_reader :server
52
+
53
+ def initialize(server)
54
+ @server = server
55
+
56
+ @availability = :available
57
+ @delay = 0
58
+
59
+ @watches = []
60
+
61
+ @queues = {}
62
+ end
63
+
64
+ def disturb!
65
+ raise Beanstalk::NotConnected unless available?
66
+
67
+ sleep @delay
68
+ end
69
+
70
+ def delay=(delay)
71
+ @delay = delay
72
+ end
73
+
74
+ def use(name)
75
+ disturb!
76
+
77
+ @current = name
78
+ end
79
+
80
+ def put(data, pri = 65536, delay = 0, ttr = 120)
81
+ disturb!
82
+
83
+ @queues[@current] ||= []
84
+ @queues[@current].push [data, pri, delay, ttr]
85
+ end
86
+
87
+ def watch(name)
88
+ disturb!
89
+
90
+ @watches.push name
91
+ end
92
+
93
+ def queues
94
+ @queues
95
+ end
96
+
97
+ def up!
98
+ @availability = :available
99
+ end
100
+
101
+ def available?
102
+ @availability == :available
103
+ end
104
+
105
+ def disconnect!
106
+ @availability = :disconnected
107
+ end
108
+
109
+ def disconnected?
110
+ @availability == :disconnected
111
+ end
112
+
113
+ def down!
114
+ @availability = :down
115
+ end
116
+
117
+ def down?
118
+ @availability == :down
119
+ end
120
+
121
+ def reserve
122
+ disturb!
123
+
124
+ @watches.each do |name|
125
+ if @queues[name] && !@queues[name].empty?
126
+ data, pri, delay, ttr = @queues[name].first
127
+
128
+ return TestJob.new(self, name, data, ttr)
129
+ end
130
+ end
131
+
132
+ nil
133
+ end
134
+ end
135
+ end
136
+
@@ -0,0 +1,208 @@
1
+
2
+ $:.unshift File.expand_path("../../lib", __FILE__)
3
+
4
+ require "stalking"
5
+ require "test/unit"
6
+
7
+ require File.expand_path("../beanstalk", __FILE__)
8
+ require File.expand_path("../test_logger", __FILE__)
9
+
10
+ class Stalking::ConsumerTest < Test::Unit::TestCase
11
+ def setup
12
+ Beanstalk::Pool.clear!
13
+ end
14
+
15
+ def test_before
16
+ Stalking::Producer.new.enqueue "test", "a" => "b"
17
+
18
+ res = nil
19
+
20
+ exception = assert_raises(Exception) do
21
+ Stalking::Consumer.new do
22
+ before do |name, args|
23
+ res = [name, args]
24
+
25
+ raise Exception
26
+ end
27
+
28
+ job "test" do |args|
29
+ # Nothing
30
+ end
31
+ end
32
+ end
33
+
34
+ assert_equal ["test", { "a" => "b" }], res
35
+ end
36
+
37
+ def test_after
38
+ Stalking::Producer.new.enqueue "test", "a" => "b"
39
+
40
+ res = nil
41
+
42
+ exception = assert_raises(Exception) do
43
+ Stalking::Consumer.new do
44
+ after do |name, args|
45
+ res = [name, args]
46
+
47
+ raise Exception
48
+ end
49
+
50
+ job "test" do |args|
51
+ # Nothing
52
+ end
53
+ end
54
+ end
55
+
56
+ assert_equal ["test", { "a" => "b" }], res
57
+ end
58
+
59
+ def test_error
60
+ Stalking::Producer.new.enqueue "test", "a" => "b"
61
+
62
+ res = nil
63
+
64
+ exception = assert_raises(Exception) do
65
+ Stalking::Consumer.new do
66
+ error do |e, name, args|
67
+ res = [e.message, name, args]
68
+
69
+ raise Exception
70
+ end
71
+
72
+ job "test" do |args|
73
+ raise "An error has occurred"
74
+ end
75
+ end
76
+ end
77
+
78
+ assert_equal ["An error has occurred", "test", { "a" => "b" }], res
79
+ end
80
+
81
+ def test_job
82
+ Stalking::Producer.new.enqueue "test", "a" => "b"
83
+
84
+ res = nil
85
+
86
+ exception = assert_raises(Exception) do
87
+ Stalking::Consumer.new do
88
+ job "test" do |args|
89
+ res = ["test", args]
90
+
91
+ raise Exception
92
+ end
93
+ end
94
+ end
95
+
96
+ assert_equal ["test", { "a" => "b" }], res
97
+ end
98
+
99
+ def test_work
100
+ Stalking::Producer.new.enqueue "test1", "a" => "b"
101
+ Stalking::Producer.new.enqueue "test2", "a" => "b"
102
+ Stalking::Producer.new.enqueue "test3", "a" => "b"
103
+
104
+ res = []
105
+
106
+ exception = assert_raises(Exception) do
107
+ Stalking::Consumer.new do
108
+ job "test1" do |args|
109
+ res.push "test1"
110
+ end
111
+
112
+ job "test2" do |args|
113
+ res.push "test2"
114
+ end
115
+
116
+ job "test3" do |args|
117
+ res.push "test3"
118
+
119
+ raise Exception
120
+ end
121
+ end
122
+ end
123
+
124
+ assert_equal ["test1", "test2", "test3"], res
125
+ end
126
+
127
+ def test_quit
128
+ raise NotImplementedError
129
+ end
130
+
131
+ def test_job_not_deleted
132
+ Stalking::Producer.new.enqueue "test", "a" => "b"
133
+
134
+ exception = assert_raises(Exception) do
135
+ Stalking::Consumer.new do
136
+ job "test" do |args|
137
+ raise Exception
138
+ end
139
+ end
140
+ end
141
+
142
+ res = nil
143
+
144
+ exception = assert_raises(Exception) do
145
+ Stalking::Consumer.new do
146
+ job "test" do |args|
147
+ res = ["test", args]
148
+
149
+ raise Exception
150
+ end
151
+ end
152
+ end
153
+
154
+ assert_equal ["test", {"a" => "b" }], res
155
+ end
156
+
157
+ def test_job_deleted
158
+ Stalking::Producer.new.enqueue "test", "a" => "b"
159
+
160
+ res = nil
161
+
162
+ exception = assert_raises(NoMethodError) do
163
+ Stalking::Consumer.new do
164
+ job "test" do |args|
165
+ res = ["test", args]
166
+ end
167
+ end
168
+ end
169
+
170
+ assert_equal ["test", { "a" => "b" }], res
171
+ assert_equal "undefined method `body' for nil:NilClass", exception.message
172
+ end
173
+
174
+ def test_reconnect
175
+ Stalking::Producer.new.enqueue "test", "a" => "b"
176
+
177
+ logger = TestLogger.new
178
+
179
+ res = []
180
+
181
+ exception = assert_raises(Exception) do
182
+ Stalking::Consumer.new(:logger => logger) do
183
+ job "test" do |args|
184
+ if res.empty?
185
+ res.push ["down"]
186
+
187
+ # Simulate a temporary beanstalk connection error.
188
+
189
+ Beanstalk::Pool.new(["localhost:11300"]).disconnect!
190
+ else
191
+ res.push ["test", args]
192
+
193
+ raise Exception
194
+ end
195
+ end
196
+ end
197
+ end
198
+
199
+ assert_equal [["down"], ["test", { "a" => "b" }]], res
200
+
201
+ assert_equal ["Stalking error: Beanstalk::NotConnected"], logger.messages
202
+ end
203
+
204
+ def test_logger
205
+ # Already tested
206
+ end
207
+ end
208
+
@@ -0,0 +1,86 @@
1
+
2
+ $:.unshift File.expand_path("../../lib", __FILE__)
3
+
4
+ require "stalking"
5
+ require "test/unit"
6
+
7
+ require File.expand_path("../test_logger", __FILE__)
8
+
9
+ class Stalking::ProducerTest < Test::Unit::TestCase
10
+ def setup
11
+ Beanstalk::Pool.clear!
12
+ end
13
+
14
+ def test_enqueue
15
+ Stalking::Producer.new.enqueue "test", { "key" => "value" }, :pri => 100, :delay => 100, :ttr => 100
16
+
17
+ assert_equal ['["test",{"key":"value"}]', 100, 100, 100], Beanstalk::Pool.new(["localhost:11300"]).queues["test"].last
18
+ end
19
+
20
+ def test_defaults
21
+ Stalking::Producer.new.enqueue "test", "key" => "value"
22
+
23
+ assert_equal ['["test",{"key":"value"}]', 65536, 0, 120], Beanstalk::Pool.new(["localhost:11300"]).queues["test"].last
24
+ end
25
+
26
+ def test_reconnect
27
+ producer = Stalking::Producer.new(:servers => ["server1", "server2"])
28
+
29
+ server1 = Beanstalk::Pool.new(["server1"])
30
+ server2 = Beanstalk::Pool.new(["server2"])
31
+
32
+ srand 0
33
+
34
+ producer.enqueue "test", "up" => "true"
35
+
36
+ assert_equal ['["test",{"up":"true"}]', 65536, 0, 120], server1.queues["test"].last
37
+
38
+ srand 0
39
+
40
+ server1.disconnect!
41
+
42
+ producer.enqueue "test", "down" => "disconnected"
43
+
44
+ assert_equal ['["test",{"down":"disconnected"}]', 65536, 0, 120], server1.queues["test"].last
45
+
46
+ server1.down!
47
+
48
+ producer.enqueue "test", "down" => "permanent"
49
+
50
+ assert_equal ['["test",{"down":"permanent"}]', 65536, 0, 120], server2.queues["test"].last
51
+ end
52
+
53
+ def test_timeout
54
+ producer = Stalking::Producer.new(:servers => ["server1", "server2"])
55
+
56
+ server1 = Beanstalk::Pool.new(["server1"])
57
+ server2 = Beanstalk::Pool.new(["server2"])
58
+
59
+ srand 0
60
+
61
+ producer.enqueue "test", "up" => "true"
62
+
63
+ assert_equal ['["test",{"up":"true"}]', 65536, 0, 120], server1.queues["test"].last
64
+
65
+ srand 0
66
+
67
+ server1.delay = 5
68
+
69
+ producer.enqueue "test", "down" => "delayed"
70
+
71
+ assert_equal ['["test",{"down":"delayed"}]', 65536, 0, 120], server2.queues["test"].last
72
+ end
73
+
74
+ def test_logger
75
+ logger = TestLogger.new
76
+
77
+ producer = Stalking::Producer.new(:logger => logger)
78
+
79
+ Beanstalk::Pool.new(["localhost:11300"]).down!
80
+
81
+ producer.enqueue "test", "up" => "true"
82
+
83
+ assert_equal ["Stalking error: Beanstalk::NotConnected"], logger.messages
84
+ end
85
+ end
86
+
@@ -0,0 +1,13 @@
1
+
2
+ class TestLogger
3
+ attr_reader :messages
4
+
5
+ def initialize
6
+ @messages = []
7
+ end
8
+
9
+ def error(msg)
10
+ @messages.push msg
11
+ end
12
+ end
13
+
metadata ADDED
@@ -0,0 +1,111 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: stalking
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.0.1
5
+ prerelease:
6
+ platform: ruby
7
+ authors:
8
+ - Benjamin Vetter
9
+ autorequire:
10
+ bindir: bin
11
+ cert_chain: []
12
+ date: 2013-06-28 00:00:00.000000000 Z
13
+ dependencies:
14
+ - !ruby/object:Gem::Dependency
15
+ name: rake
16
+ requirement: !ruby/object:Gem::Requirement
17
+ none: false
18
+ requirements:
19
+ - - ! '>='
20
+ - !ruby/object:Gem::Version
21
+ version: '0'
22
+ type: :development
23
+ prerelease: false
24
+ version_requirements: !ruby/object:Gem::Requirement
25
+ none: false
26
+ requirements:
27
+ - - ! '>='
28
+ - !ruby/object:Gem::Version
29
+ version: '0'
30
+ - !ruby/object:Gem::Dependency
31
+ name: beanstalk-client
32
+ requirement: !ruby/object:Gem::Requirement
33
+ none: false
34
+ requirements:
35
+ - - ! '>='
36
+ - !ruby/object:Gem::Version
37
+ version: '0'
38
+ type: :runtime
39
+ prerelease: false
40
+ version_requirements: !ruby/object:Gem::Requirement
41
+ none: false
42
+ requirements:
43
+ - - ! '>='
44
+ - !ruby/object:Gem::Version
45
+ version: '0'
46
+ - !ruby/object:Gem::Dependency
47
+ name: json_pure
48
+ requirement: !ruby/object:Gem::Requirement
49
+ none: false
50
+ requirements:
51
+ - - ! '>='
52
+ - !ruby/object:Gem::Version
53
+ version: '0'
54
+ type: :runtime
55
+ prerelease: false
56
+ version_requirements: !ruby/object:Gem::Requirement
57
+ none: false
58
+ requirements:
59
+ - - ! '>='
60
+ - !ruby/object:Gem::Version
61
+ version: '0'
62
+ description: Asynchronous job processing using the beanstalkd queue server with automatic
63
+ reconnects and failover
64
+ email:
65
+ - vetter@flakks.com
66
+ executables:
67
+ - stalking
68
+ extensions: []
69
+ extra_rdoc_files: []
70
+ files:
71
+ - .gitignore
72
+ - Gemfile
73
+ - LICENSE.txt
74
+ - README.md
75
+ - Rakefile
76
+ - bin/stalking
77
+ - lib/stalking.rb
78
+ - lib/stalking/consumer.rb
79
+ - lib/stalking/handlers.rb
80
+ - lib/stalking/producer.rb
81
+ - lib/stalking/version.rb
82
+ - stalking.gemspec
83
+ - test/stalking/beanstalk.rb
84
+ - test/stalking/consumer_test.rb
85
+ - test/stalking/producer_test.rb
86
+ - test/stalking/test_logger.rb
87
+ homepage: https://github.com/mrkamel/stalking
88
+ licenses: []
89
+ post_install_message:
90
+ rdoc_options: []
91
+ require_paths:
92
+ - lib
93
+ required_ruby_version: !ruby/object:Gem::Requirement
94
+ none: false
95
+ requirements:
96
+ - - ! '>='
97
+ - !ruby/object:Gem::Version
98
+ version: '0'
99
+ required_rubygems_version: !ruby/object:Gem::Requirement
100
+ none: false
101
+ requirements:
102
+ - - ! '>='
103
+ - !ruby/object:Gem::Version
104
+ version: '0'
105
+ requirements: []
106
+ rubyforge_project:
107
+ rubygems_version: 1.8.23
108
+ signing_key:
109
+ specification_version: 3
110
+ summary: Asynchronous jobs using beanstalkd with automatic reconnects and failover
111
+ test_files: []