backburner 0.0.3 → 0.1.0
Sign up to get free protection for your applications and to get access to all the features.
- data/CHANGELOG.md +6 -0
- data/Gemfile +1 -1
- data/README.md +42 -30
- data/backburner.gemspec +2 -2
- data/lib/backburner.rb +1 -1
- data/lib/backburner/connection.rb +5 -3
- data/lib/backburner/job.rb +4 -4
- data/lib/backburner/version.rb +1 -1
- data/lib/backburner/worker.rb +8 -11
- data/test/connection_test.rb +19 -3
- data/test/job_test.rb +28 -13
- data/test/test_helper.rb +3 -6
- data/test/worker_test.rb +6 -6
- metadata +43 -17
data/CHANGELOG.md
ADDED
data/Gemfile
CHANGED
data/README.md
CHANGED
@@ -1,19 +1,20 @@
|
|
1
1
|
# Backburner
|
2
2
|
|
3
|
-
Backburner is a [beanstalkd](http://kr.github.com/beanstalkd/)-powered job queue
|
4
|
-
You create background jobs and place
|
3
|
+
Backburner is a [beanstalkd](http://kr.github.com/beanstalkd/)-powered job queue that can handle a very high volume of jobs.
|
4
|
+
You create background jobs and place them on multiple work queues to be processed later.
|
5
5
|
|
6
|
-
Processing background jobs reliably has never been easier
|
7
|
-
web framework but is especially suited for use with [Sinatra](http://sinatrarb.com), [Padrino](http://padrinorb.com) and Rails.
|
6
|
+
Processing background jobs reliably has never been easier than with beanstalkd and Backburner. This gem works with any ruby-based
|
7
|
+
web framework, but is especially suited for use with [Sinatra](http://sinatrarb.com), [Padrino](http://padrinorb.com) and Rails.
|
8
8
|
|
9
9
|
If you want to use beanstalk for your job processing, consider using Backburner.
|
10
10
|
Backburner is heavily inspired by Resque and DelayedJob. Backburner stores all jobs as simple JSON message payloads.
|
11
|
-
Backburner can be a persistent queue
|
11
|
+
Backburner can be a persistent queue when the beanstalk persistence mode is enabled.
|
12
|
+
It supports multiple queues, priorities, delays, and timeouts.
|
12
13
|
|
13
14
|
## Why Backburner?
|
14
15
|
|
15
|
-
Backburner is well tested and has a familiar, no-nonsense approach to job processing but that is of secondary importance.
|
16
|
-
Let's face it
|
16
|
+
Backburner is well tested and has a familiar, no-nonsense approach to job processing, but that is of secondary importance.
|
17
|
+
Let's face it, there are a lot of options for background job processing. [DelayedJob](https://github.com/collectiveidea/delayed_job),
|
17
18
|
and [Resque](https://github.com/defunkt/resque) are the first that come to mind immediately. So, how do we make sense
|
18
19
|
of which one to use? And why use Backburner over other alternatives?
|
19
20
|
|
@@ -22,7 +23,7 @@ libraries under the hood. Every job queue requires a queue store that jobs are p
|
|
22
23
|
In the case of Resque, jobs are processed through **Redis**, a persistent key-value store. In the case of DelayedJob, jobs are processed through
|
23
24
|
**ActiveRecord** and a database such as PostgreSQL.
|
24
25
|
|
25
|
-
The work queue underlying these gems tells you infinitely more about the differences
|
26
|
+
The work queue underlying these gems tells you infinitely more about the differences than anything else.
|
26
27
|
Beanstalk is probably the best solution for job queues available today for many reasons.
|
27
28
|
The real question then is... "Why Beanstalk?".
|
28
29
|
|
@@ -33,26 +34,26 @@ Illya has an excellent blog post
|
|
33
34
|
Adam Wiggins posted [an excellent comparison](http://adam.heroku.com/past/2010/4/24/beanstalk_a_simple_and_fast_queueing_backend/).
|
34
35
|
|
35
36
|
You will quickly see that **beanstalkd** is an underrated but incredible project that is extremely well-suited as a job queue.
|
36
|
-
Significantly better suited for this task
|
37
|
+
Significantly better suited for this task than Redis or a database. Beanstalk is a simple,
|
37
38
|
and a very fast work queue service rolled into a single binary - it is the memcached of work queues.
|
38
39
|
Originally built to power the backend for the 'Causes' Facebook app, it is a mature and production ready open source project.
|
39
40
|
[PostRank](http://www.postrank.com) uses beanstalk to reliably process millions of jobs a day.
|
40
41
|
|
41
42
|
A single instance of Beanstalk is perfectly capable of handling thousands of jobs a second (or more, depending on your job size)
|
42
43
|
because it is an in-memory, event-driven system. Powered by libevent under the hood,
|
43
|
-
it requires zero setup (launch and forget,
|
44
|
+
it requires zero setup (launch and forget, à la memcached), optional log based persistence, an easily parsed ASCII protocol,
|
44
45
|
and a rich set of tools for job management that go well beyond a simple FIFO work queue.
|
45
46
|
|
46
47
|
Beanstalk supports the following features natively, out of the box, without any questions asked:
|
47
48
|
|
48
|
-
* **Parallel Queues** - Supports multiple work queues
|
49
|
-
* **Reliable** - Beanstalk’s reserve, work, delete cycle
|
49
|
+
* **Parallel Queues** - Supports multiple work queues created on demand.
|
50
|
+
* **Reliable** - Beanstalk’s reserve, work, delete cycle ensures reliable processing.
|
50
51
|
* **Scheduling** - Delay enqueuing jobs by a specified interval to schedule processing later.
|
51
|
-
* **Fast** -
|
52
|
-
* **Priorities** - Specify
|
53
|
-
* **Persistence** - Jobs are stored in memory for speed
|
54
|
-
* **Federation** -
|
55
|
-
* **
|
52
|
+
* **Fast** - Processes thousands of jobs per second; **significantly** [faster than alternatives](http://adam.heroku.com/past/2010/4/24/beanstalk_a_simple_and_fast_queueing_backend).
|
53
|
+
* **Priorities** - Specify priority so important jobs can be processed quickly.
|
54
|
+
* **Persistence** - Jobs are stored in memory for speed, but logged to disk for safe keeping.
|
55
|
+
* **Federation** - Horizontal scalability provided through federation by the client.
|
56
|
+
* **Error Handling** - Bury any job which causes an error for later debugging and inspection.
|
56
57
|
|
57
58
|
Keep in mind that these features are supported out of the box with beanstalk and require no special code within this gem to support.
|
58
59
|
In the end, **beanstalk is the ideal job queue** while also being ridiculously easy to install and setup.
|
@@ -82,7 +83,7 @@ Backburner is extremely simple to setup. Just configure basic settings for backb
|
|
82
83
|
|
83
84
|
```ruby
|
84
85
|
Backburner.configure do |config|
|
85
|
-
config.beanstalk_url = "beanstalk://127.0.0.1"
|
86
|
+
config.beanstalk_url = ["beanstalk://127.0.0.1", "beanstalk://127.0.0.1:11301"]
|
86
87
|
config.tube_namespace = "some.app.production"
|
87
88
|
config.on_error = lambda { |e| puts e }
|
88
89
|
config.default_priority = 65536
|
@@ -90,6 +91,10 @@ Backburner.configure do |config|
|
|
90
91
|
end
|
91
92
|
```
|
92
93
|
|
94
|
+
The `beanstalk_url` supports a string such as 'beanstalk://127.0.0.1' or an array of addresses.
|
95
|
+
The `tube_namespace` is the prefix used for all tubes related to this backburner queue.
|
96
|
+
The `on_error` is a callback that gets invoked with the error whenever a job fails
|
97
|
+
|
93
98
|
## Usage
|
94
99
|
|
95
100
|
Backburner allows you to create jobs and place them on a beanstalk queue, and later pull those jobs off the queue and
|
@@ -133,8 +138,8 @@ includes `Backburner::Performable`. Async enqueuing works for both instance and
|
|
133
138
|
```ruby
|
134
139
|
class User
|
135
140
|
include Backburner::Performable
|
136
|
-
queue "
|
137
|
-
queue_priority
|
141
|
+
queue "user-jobs" # defaults to 'user'
|
142
|
+
queue_priority 500 # most urgent priority is 0
|
138
143
|
|
139
144
|
def activate(device_id)
|
140
145
|
@device = Device.find(device_id)
|
@@ -148,14 +153,15 @@ end
|
|
148
153
|
|
149
154
|
# Async works for instance methods on a persisted model
|
150
155
|
@user = User.first
|
151
|
-
@user.async(:
|
156
|
+
@user.async(:ttr => 100, :queue => "activate").activate(@device.id)
|
152
157
|
# ..as well as for class methods
|
153
158
|
User.async(:pri => 100, :delay => 10.seconds).reset_password(@user.id)
|
154
159
|
```
|
155
160
|
|
156
161
|
This will automatically enqueue a job for that user record that will run `activate` with the specified argument.
|
162
|
+
Note that you can set the queue name and queue priority at the class level and
|
163
|
+
you are also able to pass `pri`, `ttr`, `delay` and `queue` directly as options into `async`.
|
157
164
|
The queue name used by default is the normalized class name (i.e `{namespace}.user`) if not otherwise specified.
|
158
|
-
Note you are able to pass `pri`, `ttr`, `delay` and `queue` directly as options into `async`.
|
159
165
|
|
160
166
|
### Working Jobs
|
161
167
|
|
@@ -196,8 +202,8 @@ This will daemonize the worker and store the pid and logs automatically.
|
|
196
202
|
Workers can be easily restricted to processing only a specific set of queues as shown above. However, if you want a worker to
|
197
203
|
process **all** queues instead, then you can leave the queue list blank.
|
198
204
|
|
199
|
-
When you execute a worker without queues specified, any queue for a known job queue class with `include Backburner::Queue` will be processed.
|
200
|
-
queue classes, you can use:
|
205
|
+
When you execute a worker without queues specified, any queue for a known job queue class with `include Backburner::Queue` will be processed.
|
206
|
+
To access the list of known queue classes, you can use:
|
201
207
|
|
202
208
|
```ruby
|
203
209
|
Backburner::Worker.known_queue_classes
|
@@ -236,22 +242,28 @@ If a job fails in beanstalk, the job is automatically buried and must be 'kicked
|
|
236
242
|
|
237
243
|
Right now, all logging happens to standard out and can be piped to a file or any other output manually. More on logging coming later.
|
238
244
|
|
239
|
-
### Front-end
|
245
|
+
### Web Front-end
|
240
246
|
|
241
|
-
|
247
|
+
Be sure to check out the Sinatra-powered project [beanstalkd_view](https://github.com/denniskuczynski/beanstalkd_view)
|
248
|
+
by [denniskuczynski](http://github.com/denniskuczynski) which provides an excellent overview of the tubes and
|
249
|
+
jobs processed by your beanstalk workers. An excellent addition to your Backburner setup.
|
242
250
|
|
243
251
|
### Workers in Production
|
244
252
|
|
245
253
|
Once you have Backburner setup in your application, starting workers is really easy. Once [beanstalkd](http://kr.github.com/beanstalkd/download.html)
|
246
254
|
is installed, your best bet is to use the built-in rake task that comes with Backburner. Simply add the task to your Rakefile:
|
247
255
|
|
248
|
-
|
249
|
-
|
256
|
+
```ruby
|
257
|
+
# Rakefile
|
258
|
+
require 'backburner/tasks'
|
259
|
+
```
|
250
260
|
|
251
261
|
and then you can start the rake task with:
|
252
262
|
|
253
|
-
|
254
|
-
|
263
|
+
```bash
|
264
|
+
$ rake backburner:work
|
265
|
+
$ QUEUES=newsletter-sender,push-message rake backburner:work
|
266
|
+
```
|
255
267
|
|
256
268
|
The best way to deploy these rake tasks is using a monitoring library. We suggest [God](https://github.com/mojombo/god/)
|
257
269
|
which watches processes and ensures their stability. A simple God recipe for Backburner can be found in
|
data/backburner.gemspec
CHANGED
@@ -15,10 +15,10 @@ Gem::Specification.new do |s|
|
|
15
15
|
s.require_paths = ["lib"]
|
16
16
|
s.version = Backburner::VERSION
|
17
17
|
|
18
|
-
s.add_runtime_dependency '
|
18
|
+
s.add_runtime_dependency 'beaneater', '~> 0.1.1'
|
19
19
|
s.add_runtime_dependency 'dante', '~> 0.1.5'
|
20
20
|
|
21
21
|
s.add_development_dependency 'rake'
|
22
|
-
s.add_development_dependency 'minitest'
|
22
|
+
s.add_development_dependency 'minitest', '~> 4.1.0'
|
23
23
|
s.add_development_dependency 'mocha'
|
24
24
|
end
|
data/lib/backburner.rb
CHANGED
@@ -6,6 +6,8 @@ module Backburner
|
|
6
6
|
|
7
7
|
attr_accessor :url, :beanstalk
|
8
8
|
|
9
|
+
# Constructs a backburner connection
|
10
|
+
# `url` can be a string i.e 'localhost:3001' or an array of addresses.
|
9
11
|
def initialize(url)
|
10
12
|
@url = url
|
11
13
|
connect!
|
@@ -22,7 +24,7 @@ module Backburner
|
|
22
24
|
|
23
25
|
# Connects to a beanstalk queue
|
24
26
|
def connect!
|
25
|
-
@beanstalk ||=
|
27
|
+
@beanstalk ||= Beaneater::Pool.new(beanstalk_addresses)
|
26
28
|
end
|
27
29
|
|
28
30
|
# Returns the beanstalk queue addresses
|
@@ -31,8 +33,8 @@ module Backburner
|
|
31
33
|
# beanstalk_addresses => ["localhost:11300"]
|
32
34
|
#
|
33
35
|
def beanstalk_addresses
|
34
|
-
uris = self.url.split(/[\s,]+/)
|
35
|
-
uris.map {|uri| beanstalk_host_and_port(uri)}
|
36
|
+
uris = self.url.is_a?(Array) ? self.url : self.url.split(/[\s,]+/)
|
37
|
+
uris.map { |uri| beanstalk_host_and_port(uri) }
|
36
38
|
end
|
37
39
|
|
38
40
|
# Returns a host and port based on the uri_string given
|
data/lib/backburner/job.rb
CHANGED
@@ -19,7 +19,7 @@ module Backburner
|
|
19
19
|
#
|
20
20
|
def initialize(task)
|
21
21
|
@task = task
|
22
|
-
@body = JSON.parse(task.body)
|
22
|
+
@body = task.body.is_a?(Hash) ? task.body : JSON.parse(task.body)
|
23
23
|
@name, @args = body["class"], body["args"]
|
24
24
|
end
|
25
25
|
|
@@ -46,12 +46,12 @@ module Backburner
|
|
46
46
|
# job_class # => NewsletterSender
|
47
47
|
#
|
48
48
|
def job_class
|
49
|
-
handler = constantize(name) rescue nil
|
50
|
-
raise(JobNotFound, name) unless handler
|
49
|
+
handler = constantize(self.name) rescue nil
|
50
|
+
raise(JobNotFound, self.name) unless handler
|
51
51
|
handler
|
52
52
|
end
|
53
53
|
|
54
|
-
# Timeout job after given time
|
54
|
+
# Timeout job within specified block after given time.
|
55
55
|
#
|
56
56
|
# @example
|
57
57
|
# timeout_job_after(3) { do_something! }
|
data/lib/backburner/version.rb
CHANGED
data/lib/backburner/worker.rb
CHANGED
@@ -15,7 +15,7 @@ module Backburner
|
|
15
15
|
# Enqueues a job to be processed later by a worker
|
16
16
|
# Options: `pri` (priority), `delay` (delay in secs), `ttr` (time to respond), `queue` (queue name)
|
17
17
|
#
|
18
|
-
# @raise [
|
18
|
+
# @raise [Beaneater::NotConnected] If beanstalk fails to connect.
|
19
19
|
# @example
|
20
20
|
# Backburner::Worker.enqueue NewsletterSender, [self.id, user.id], :ttr => 1000
|
21
21
|
#
|
@@ -23,9 +23,9 @@ module Backburner
|
|
23
23
|
pri = opts[:pri] || job_class.queue_priority || Backburner.configuration.default_priority
|
24
24
|
delay = [0, opts[:delay].to_i].max
|
25
25
|
ttr = opts[:ttr] || Backburner.configuration.respond_timeout
|
26
|
-
connection.
|
26
|
+
tube = connection.tubes[expand_tube_name(opts[:queue] || job_class)]
|
27
27
|
data = { :class => job_class.name, :args => args }
|
28
|
-
|
28
|
+
tube.put data.to_json, :pri => pri, :delay => delay, :ttr => ttr
|
29
29
|
end
|
30
30
|
|
31
31
|
# Starts processing jobs in the specified tube_names
|
@@ -39,7 +39,7 @@ module Backburner
|
|
39
39
|
|
40
40
|
# Returns the worker connection
|
41
41
|
# @example
|
42
|
-
# Backburner::Worker.connection # => <
|
42
|
+
# Backburner::Worker.connection # => <Beaneater::Pool>
|
43
43
|
def self.connection
|
44
44
|
@connection ||= Connection.new(Backburner.configuration.beanstalk_url)
|
45
45
|
end
|
@@ -72,7 +72,7 @@ module Backburner
|
|
72
72
|
# Setup beanstalk tube_names and watch all specified tubes for jobs.
|
73
73
|
# Used to prepare job queues before processing jobs.
|
74
74
|
#
|
75
|
-
# @raise [
|
75
|
+
# @raise [Beaneater::NotConnected] If beanstalk fails to connect.
|
76
76
|
# @example
|
77
77
|
# @worker.prepare
|
78
78
|
#
|
@@ -81,10 +81,7 @@ module Backburner
|
|
81
81
|
self.tube_names = Array(self.tube_names)
|
82
82
|
self.tube_names.map! { |name| expand_tube_name(name) }
|
83
83
|
log_info "Working #{tube_names.size} queues: [ #{tube_names.join(', ')} ]"
|
84
|
-
self.
|
85
|
-
self.connection.list_tubes_watched.each do |server, tubes|
|
86
|
-
tubes.each { |tube| self.connection.ignore(tube) unless self.tube_names.include?(tube) }
|
87
|
-
end
|
84
|
+
self.connection.tubes.watch!(*self.tube_names)
|
88
85
|
end
|
89
86
|
|
90
87
|
# Reserves one job within the specified queues
|
@@ -95,7 +92,7 @@ module Backburner
|
|
95
92
|
# @worker.work_one_job
|
96
93
|
#
|
97
94
|
def work_one_job
|
98
|
-
job = Backburner::Job.new(self.connection.reserve)
|
95
|
+
job = Backburner::Job.new(self.connection.tubes.reserve)
|
99
96
|
self.class.log_job_begin(job.body)
|
100
97
|
job.process
|
101
98
|
self.class.log_job_end(job.name)
|
@@ -112,7 +109,7 @@ module Backburner
|
|
112
109
|
# Filtered for tubes that match the known prefix
|
113
110
|
def all_existing_queues
|
114
111
|
known_queues = Backburner::Worker.known_queue_classes.map(&:queue)
|
115
|
-
existing_tubes = self.connection.
|
112
|
+
existing_tubes = self.connection.tubes.all.map(&:name).select { |tube| tube =~ /^#{tube_namespace}/ }
|
116
113
|
known_queues + existing_tubes
|
117
114
|
end
|
118
115
|
|
data/test/connection_test.rb
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
require File.expand_path('../test_helper', __FILE__)
|
2
2
|
|
3
3
|
describe "Backburner::Connection class" do
|
4
|
-
describe "for initialize" do
|
4
|
+
describe "for initialize with single url" do
|
5
5
|
before do
|
6
6
|
@connection = Backburner::Connection.new("beanstalk://localhost")
|
7
7
|
end
|
@@ -11,7 +11,23 @@ describe "Backburner::Connection class" do
|
|
11
11
|
end
|
12
12
|
|
13
13
|
it "should setup beanstalk connection" do
|
14
|
-
assert_kind_of
|
14
|
+
assert_kind_of Beaneater::Pool, @connection.beanstalk
|
15
|
+
end
|
16
|
+
end # initialize single connection
|
17
|
+
|
18
|
+
describe "for initialize with multiple urls" do
|
19
|
+
it "should support single string with commas" do
|
20
|
+
@connection = Backburner::Connection.new("beanstalk://localhost,beanstalk://localhost")
|
21
|
+
connections = @connection.beanstalk.connections
|
22
|
+
assert_equal 2, connections.size
|
23
|
+
assert_equal ['localhost:11300','localhost:11300'], connections.map(&:address)
|
24
|
+
end
|
25
|
+
|
26
|
+
it "should support array of connections" do
|
27
|
+
@connection = Backburner::Connection.new(['beanstalk://127.0.0.1:11300','beanstalk://localhost'])
|
28
|
+
connections = @connection.beanstalk.connections
|
29
|
+
assert_equal 2, @connection.beanstalk.connections.size
|
30
|
+
assert_equal ['127.0.0.1:11300','localhost:11300'], connections.map(&:address)
|
15
31
|
end
|
16
32
|
end # initialize
|
17
33
|
|
@@ -29,7 +45,7 @@ describe "Backburner::Connection class" do
|
|
29
45
|
end
|
30
46
|
|
31
47
|
it "delegate methods to beanstalk connection" do
|
32
|
-
assert_equal "localhost
|
48
|
+
assert_equal "localhost", @connection.connections.first.host
|
33
49
|
end
|
34
50
|
end # delegator
|
35
51
|
end # Connection
|
data/test/job_test.rb
CHANGED
@@ -13,9 +13,24 @@ module NestedDemo
|
|
13
13
|
end
|
14
14
|
|
15
15
|
describe "Backburner::Job module" do
|
16
|
-
describe "for initialize method" do
|
16
|
+
describe "for initialize method with hash" do
|
17
17
|
before do
|
18
|
-
@task_body = {
|
18
|
+
@task_body = { "class" => "NewsletterSender", "args" => ["foo@bar.com", "bar@foo.com"] }
|
19
|
+
@task = stub(:body => @task_body, :ttr => 120, :delete => true, :bury => true)
|
20
|
+
end
|
21
|
+
|
22
|
+
it "should create job with correct task data" do
|
23
|
+
@job = Backburner::Job.new(@task)
|
24
|
+
assert_equal @task, @job.task
|
25
|
+
assert_equal ["class", "args"], @job.body.keys
|
26
|
+
assert_equal @task_body["class"], @job.name
|
27
|
+
assert_equal @task_body["args"], @job.args
|
28
|
+
end
|
29
|
+
end # initialize with json
|
30
|
+
|
31
|
+
describe "for initialize method with json string" do
|
32
|
+
before do
|
33
|
+
@task_body = { "class" => "NewsletterSender", "args" => ["foo@bar.com", "bar@foo.com"] }
|
19
34
|
@task = stub(:body => @task_body.to_json, :ttr => 120, :delete => true, :bury => true)
|
20
35
|
end
|
21
36
|
|
@@ -23,16 +38,16 @@ describe "Backburner::Job module" do
|
|
23
38
|
@job = Backburner::Job.new(@task)
|
24
39
|
assert_equal @task, @job.task
|
25
40
|
assert_equal ["class", "args"], @job.body.keys
|
26
|
-
assert_equal @task_body[
|
27
|
-
assert_equal @task_body[
|
41
|
+
assert_equal @task_body["class"], @job.name
|
42
|
+
assert_equal @task_body["args"], @job.args
|
28
43
|
end
|
29
|
-
end # initialize
|
44
|
+
end # initialize with json
|
30
45
|
|
31
46
|
describe "for process method" do
|
32
47
|
describe "with valid task" do
|
33
48
|
before do
|
34
|
-
@task_body = {
|
35
|
-
@task = stub(:body => @task_body
|
49
|
+
@task_body = { "class" => "NestedDemo::TestJobC", "args" => [56] }
|
50
|
+
@task = stub(:body => @task_body, :ttr => 120, :delete => true, :bury => true)
|
36
51
|
@task.expects(:delete).once
|
37
52
|
end
|
38
53
|
|
@@ -45,8 +60,8 @@ describe "Backburner::Job module" do
|
|
45
60
|
|
46
61
|
describe "with invalid task" do
|
47
62
|
before do
|
48
|
-
@task_body = {
|
49
|
-
@task = stub(:body => @task_body
|
63
|
+
@task_body = { "class" => "NestedDemo::TestJobD", "args" => [56] }
|
64
|
+
@task = stub(:body => @task_body, :ttr => 120, :delete => true, :bury => true)
|
50
65
|
@task.expects(:delete).never
|
51
66
|
end
|
52
67
|
|
@@ -58,8 +73,8 @@ describe "Backburner::Job module" do
|
|
58
73
|
|
59
74
|
describe "with invalid class" do
|
60
75
|
before do
|
61
|
-
@task_body = {
|
62
|
-
@task = stub(:body => @task_body
|
76
|
+
@task_body = { "class" => "NestedDemo::TestJobY", "args" => [56] }
|
77
|
+
@task = stub(:body => @task_body, :ttr => 120, :delete => true, :bury => true)
|
63
78
|
@task.expects(:delete).never
|
64
79
|
end
|
65
80
|
|
@@ -72,8 +87,8 @@ describe "Backburner::Job module" do
|
|
72
87
|
|
73
88
|
describe "for bury method" do
|
74
89
|
before do
|
75
|
-
@task_body = {
|
76
|
-
@task = stub(:body => @task_body
|
90
|
+
@task_body = { "class" => "NestedDemo::TestJobC", "args" => [56] }
|
91
|
+
@task = stub(:body => @task_body, :ttr => 120, :delete => true, :bury => true)
|
77
92
|
@task.expects(:bury).once
|
78
93
|
end
|
79
94
|
|
data/test/test_helper.rb
CHANGED
@@ -86,11 +86,8 @@ class MiniTest::Spec
|
|
86
86
|
def pop_one_job(tube_name)
|
87
87
|
connection = Backburner::Worker.connection
|
88
88
|
tube_name = [Backburner.configuration.tube_namespace, tube_name].join(".")
|
89
|
-
connection.watch(tube_name)
|
90
|
-
connection.
|
91
|
-
|
92
|
-
end
|
93
|
-
silenced(3) { @res = connection.reserve }
|
94
|
-
return @res, JSON.parse(@res.body)
|
89
|
+
connection.tubes.watch!(tube_name)
|
90
|
+
silenced(3) { @res = connection.tubes.reserve }
|
91
|
+
return @res, @res.body
|
95
92
|
end
|
96
93
|
end # MiniTest::Spec
|
data/test/worker_test.rb
CHANGED
@@ -63,7 +63,7 @@ describe "Backburner::Worker module" do
|
|
63
63
|
describe "for connection class method" do
|
64
64
|
it "should return the beanstalk connection" do
|
65
65
|
assert_equal "beanstalk://localhost", Backburner::Worker.connection.url
|
66
|
-
assert_kind_of
|
66
|
+
assert_kind_of Beaneater::Pool, Backburner::Worker.connection.beanstalk
|
67
67
|
end
|
68
68
|
end # connection
|
69
69
|
|
@@ -104,7 +104,7 @@ describe "Backburner::Worker module" do
|
|
104
104
|
worker = Backburner::Worker.new(["foo", "bar"])
|
105
105
|
out = capture_stdout { worker.prepare }
|
106
106
|
assert_equal ["demo.test.foo", "demo.test.bar"], worker.tube_names
|
107
|
-
assert_same_elements ["demo.test.foo", "demo.test.bar"], Backburner::Worker.connection.
|
107
|
+
assert_same_elements ["demo.test.foo", "demo.test.bar"], Backburner::Worker.connection.tubes.watched.map(&:name)
|
108
108
|
assert_match /demo\.test\.foo/, out
|
109
109
|
end # multiple
|
110
110
|
|
@@ -112,7 +112,7 @@ describe "Backburner::Worker module" do
|
|
112
112
|
worker = Backburner::Worker.new("foo")
|
113
113
|
out = capture_stdout { worker.prepare }
|
114
114
|
assert_equal ["demo.test.foo"], worker.tube_names
|
115
|
-
assert_same_elements ["demo.test.foo"], Backburner::Worker.connection.
|
115
|
+
assert_same_elements ["demo.test.foo"], Backburner::Worker.connection.tubes.watched.map(&:name)
|
116
116
|
assert_match /demo\.test\.foo/, out
|
117
117
|
end # single
|
118
118
|
|
@@ -121,7 +121,7 @@ describe "Backburner::Worker module" do
|
|
121
121
|
worker = Backburner::Worker.new
|
122
122
|
out = capture_stdout { worker.prepare }
|
123
123
|
assert_equal ["demo.test.foo", "demo.test.bar"], worker.tube_names
|
124
|
-
assert_same_elements ["demo.test.foo", "demo.test.bar"], Backburner::Worker.connection.
|
124
|
+
assert_same_elements ["demo.test.foo", "demo.test.bar"], Backburner::Worker.connection.tubes.watched.map(&:name)
|
125
125
|
assert_match /demo\.test\.foo/, out
|
126
126
|
end
|
127
127
|
|
@@ -130,7 +130,7 @@ describe "Backburner::Worker module" do
|
|
130
130
|
worker = Backburner::Worker.new
|
131
131
|
out = capture_stdout { worker.prepare }
|
132
132
|
assert_equal ["demo.test.bar"], worker.tube_names
|
133
|
-
assert_same_elements ["demo.test.bar"], Backburner::Worker.connection.
|
133
|
+
assert_same_elements ["demo.test.bar"], Backburner::Worker.connection.tubes.watched.map(&:name)
|
134
134
|
assert_match /demo\.test\.bar/, out
|
135
135
|
end # all assign
|
136
136
|
|
@@ -138,7 +138,7 @@ describe "Backburner::Worker module" do
|
|
138
138
|
worker = Backburner::Worker.new
|
139
139
|
out = capture_stdout { worker.prepare }
|
140
140
|
assert_contains worker.tube_names, "demo.test.test-job"
|
141
|
-
assert_contains Backburner::Worker.connection.
|
141
|
+
assert_contains Backburner::Worker.connection.tubes.watched.map(&:name), "demo.test.test-job"
|
142
142
|
assert_match /demo\.test\.test-job/, out
|
143
143
|
end # all read
|
144
144
|
end # prepare
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: backburner
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0
|
4
|
+
version: 0.1.0
|
5
5
|
prerelease:
|
6
6
|
platform: ruby
|
7
7
|
authors:
|
@@ -9,22 +9,27 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2012-
|
12
|
+
date: 2012-11-05 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
|
-
name:
|
16
|
-
requirement:
|
15
|
+
name: beaneater
|
16
|
+
requirement: !ruby/object:Gem::Requirement
|
17
17
|
none: false
|
18
18
|
requirements:
|
19
19
|
- - ~>
|
20
20
|
- !ruby/object:Gem::Version
|
21
|
-
version:
|
21
|
+
version: 0.1.1
|
22
22
|
type: :runtime
|
23
23
|
prerelease: false
|
24
|
-
version_requirements:
|
24
|
+
version_requirements: !ruby/object:Gem::Requirement
|
25
|
+
none: false
|
26
|
+
requirements:
|
27
|
+
- - ~>
|
28
|
+
- !ruby/object:Gem::Version
|
29
|
+
version: 0.1.1
|
25
30
|
- !ruby/object:Gem::Dependency
|
26
31
|
name: dante
|
27
|
-
requirement:
|
32
|
+
requirement: !ruby/object:Gem::Requirement
|
28
33
|
none: false
|
29
34
|
requirements:
|
30
35
|
- - ~>
|
@@ -32,10 +37,15 @@ dependencies:
|
|
32
37
|
version: 0.1.5
|
33
38
|
type: :runtime
|
34
39
|
prerelease: false
|
35
|
-
version_requirements:
|
40
|
+
version_requirements: !ruby/object:Gem::Requirement
|
41
|
+
none: false
|
42
|
+
requirements:
|
43
|
+
- - ~>
|
44
|
+
- !ruby/object:Gem::Version
|
45
|
+
version: 0.1.5
|
36
46
|
- !ruby/object:Gem::Dependency
|
37
47
|
name: rake
|
38
|
-
requirement:
|
48
|
+
requirement: !ruby/object:Gem::Requirement
|
39
49
|
none: false
|
40
50
|
requirements:
|
41
51
|
- - ! '>='
|
@@ -43,21 +53,31 @@ dependencies:
|
|
43
53
|
version: '0'
|
44
54
|
type: :development
|
45
55
|
prerelease: false
|
46
|
-
version_requirements:
|
47
|
-
- !ruby/object:Gem::Dependency
|
48
|
-
name: minitest
|
49
|
-
requirement: &2152604860 !ruby/object:Gem::Requirement
|
56
|
+
version_requirements: !ruby/object:Gem::Requirement
|
50
57
|
none: false
|
51
58
|
requirements:
|
52
59
|
- - ! '>='
|
53
60
|
- !ruby/object:Gem::Version
|
54
61
|
version: '0'
|
62
|
+
- !ruby/object:Gem::Dependency
|
63
|
+
name: minitest
|
64
|
+
requirement: !ruby/object:Gem::Requirement
|
65
|
+
none: false
|
66
|
+
requirements:
|
67
|
+
- - ~>
|
68
|
+
- !ruby/object:Gem::Version
|
69
|
+
version: 4.1.0
|
55
70
|
type: :development
|
56
71
|
prerelease: false
|
57
|
-
version_requirements:
|
72
|
+
version_requirements: !ruby/object:Gem::Requirement
|
73
|
+
none: false
|
74
|
+
requirements:
|
75
|
+
- - ~>
|
76
|
+
- !ruby/object:Gem::Version
|
77
|
+
version: 4.1.0
|
58
78
|
- !ruby/object:Gem::Dependency
|
59
79
|
name: mocha
|
60
|
-
requirement:
|
80
|
+
requirement: !ruby/object:Gem::Requirement
|
61
81
|
none: false
|
62
82
|
requirements:
|
63
83
|
- - ! '>='
|
@@ -65,7 +85,12 @@ dependencies:
|
|
65
85
|
version: '0'
|
66
86
|
type: :development
|
67
87
|
prerelease: false
|
68
|
-
version_requirements:
|
88
|
+
version_requirements: !ruby/object:Gem::Requirement
|
89
|
+
none: false
|
90
|
+
requirements:
|
91
|
+
- - ! '>='
|
92
|
+
- !ruby/object:Gem::Version
|
93
|
+
version: '0'
|
69
94
|
description: Beanstalk background job processing made easy
|
70
95
|
email:
|
71
96
|
- nesquena@gmail.com
|
@@ -76,6 +101,7 @@ extra_rdoc_files: []
|
|
76
101
|
files:
|
77
102
|
- .gitignore
|
78
103
|
- .travis.yml
|
104
|
+
- CHANGELOG.md
|
79
105
|
- Gemfile
|
80
106
|
- LICENSE
|
81
107
|
- README.md
|
@@ -128,7 +154,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
128
154
|
version: '0'
|
129
155
|
requirements: []
|
130
156
|
rubyforge_project:
|
131
|
-
rubygems_version: 1.8.
|
157
|
+
rubygems_version: 1.8.24
|
132
158
|
signing_key:
|
133
159
|
specification_version: 3
|
134
160
|
summary: Reliable beanstalk background job processing made easy for Ruby and Sinatra
|