backburner 0.0.3 → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/CHANGELOG.md +6 -0
- data/Gemfile +1 -1
- data/README.md +42 -30
- data/backburner.gemspec +2 -2
- data/lib/backburner.rb +1 -1
- data/lib/backburner/connection.rb +5 -3
- data/lib/backburner/job.rb +4 -4
- data/lib/backburner/version.rb +1 -1
- data/lib/backburner/worker.rb +8 -11
- data/test/connection_test.rb +19 -3
- data/test/job_test.rb +28 -13
- data/test/test_helper.rb +3 -6
- data/test/worker_test.rb +6 -6
- metadata +43 -17
data/CHANGELOG.md
ADDED
data/Gemfile
CHANGED
data/README.md
CHANGED
@@ -1,19 +1,20 @@
|
|
1
1
|
# Backburner
|
2
2
|
|
3
|
-
Backburner is a [beanstalkd](http://kr.github.com/beanstalkd/)-powered job queue
|
4
|
-
You create background jobs and place
|
3
|
+
Backburner is a [beanstalkd](http://kr.github.com/beanstalkd/)-powered job queue that can handle a very high volume of jobs.
|
4
|
+
You create background jobs and place them on multiple work queues to be processed later.
|
5
5
|
|
6
|
-
Processing background jobs reliably has never been easier
|
7
|
-
web framework but is especially suited for use with [Sinatra](http://sinatrarb.com), [Padrino](http://padrinorb.com) and Rails.
|
6
|
+
Processing background jobs reliably has never been easier than with beanstalkd and Backburner. This gem works with any ruby-based
|
7
|
+
web framework, but is especially suited for use with [Sinatra](http://sinatrarb.com), [Padrino](http://padrinorb.com) and Rails.
|
8
8
|
|
9
9
|
If you want to use beanstalk for your job processing, consider using Backburner.
|
10
10
|
Backburner is heavily inspired by Resque and DelayedJob. Backburner stores all jobs as simple JSON message payloads.
|
11
|
-
Backburner can be a persistent queue
|
11
|
+
Backburner can be a persistent queue when the beanstalk persistence mode is enabled.
|
12
|
+
It supports multiple queues, priorities, delays, and timeouts.
|
12
13
|
|
13
14
|
## Why Backburner?
|
14
15
|
|
15
|
-
Backburner is well tested and has a familiar, no-nonsense approach to job processing but that is of secondary importance.
|
16
|
-
Let's face it
|
16
|
+
Backburner is well tested and has a familiar, no-nonsense approach to job processing, but that is of secondary importance.
|
17
|
+
Let's face it, there are a lot of options for background job processing. [DelayedJob](https://github.com/collectiveidea/delayed_job),
|
17
18
|
and [Resque](https://github.com/defunkt/resque) are the first that come to mind immediately. So, how do we make sense
|
18
19
|
of which one to use? And why use Backburner over other alternatives?
|
19
20
|
|
@@ -22,7 +23,7 @@ libraries under the hood. Every job queue requires a queue store that jobs are p
|
|
22
23
|
In the case of Resque, jobs are processed through **Redis**, a persistent key-value store. In the case of DelayedJob, jobs are processed through
|
23
24
|
**ActiveRecord** and a database such as PostgreSQL.
|
24
25
|
|
25
|
-
The work queue underlying these gems tells you infinitely more about the differences
|
26
|
+
The work queue underlying these gems tells you infinitely more about the differences than anything else.
|
26
27
|
Beanstalk is probably the best solution for job queues available today for many reasons.
|
27
28
|
The real question then is... "Why Beanstalk?".
|
28
29
|
|
@@ -33,26 +34,26 @@ Illya has an excellent blog post
|
|
33
34
|
Adam Wiggins posted [an excellent comparison](http://adam.heroku.com/past/2010/4/24/beanstalk_a_simple_and_fast_queueing_backend/).
|
34
35
|
|
35
36
|
You will quickly see that **beanstalkd** is an underrated but incredible project that is extremely well-suited as a job queue.
|
36
|
-
Significantly better suited for this task
|
37
|
+
Significantly better suited for this task than Redis or a database. Beanstalk is a simple,
|
37
38
|
and a very fast work queue service rolled into a single binary - it is the memcached of work queues.
|
38
39
|
Originally built to power the backend for the 'Causes' Facebook app, it is a mature and production ready open source project.
|
39
40
|
[PostRank](http://www.postrank.com) uses beanstalk to reliably process millions of jobs a day.
|
40
41
|
|
41
42
|
A single instance of Beanstalk is perfectly capable of handling thousands of jobs a second (or more, depending on your job size)
|
42
43
|
because it is an in-memory, event-driven system. Powered by libevent under the hood,
|
43
|
-
it requires zero setup (launch and forget,
|
44
|
+
it requires zero setup (launch and forget, à la memcached), optional log based persistence, an easily parsed ASCII protocol,
|
44
45
|
and a rich set of tools for job management that go well beyond a simple FIFO work queue.
|
45
46
|
|
46
47
|
Beanstalk supports the following features natively, out of the box, without any questions asked:
|
47
48
|
|
48
|
-
* **Parallel Queues** - Supports multiple work queues
|
49
|
-
* **Reliable** - Beanstalk’s reserve, work, delete cycle
|
49
|
+
* **Parallel Queues** - Supports multiple work queues created on demand.
|
50
|
+
* **Reliable** - Beanstalk’s reserve, work, delete cycle ensures reliable processing.
|
50
51
|
* **Scheduling** - Delay enqueuing jobs by a specified interval to schedule processing later.
|
51
|
-
* **Fast** -
|
52
|
-
* **Priorities** - Specify
|
53
|
-
* **Persistence** - Jobs are stored in memory for speed
|
54
|
-
* **Federation** -
|
55
|
-
* **
|
52
|
+
* **Fast** - Processes thousands of jobs per second; **significantly** [faster than alternatives](http://adam.heroku.com/past/2010/4/24/beanstalk_a_simple_and_fast_queueing_backend).
|
53
|
+
* **Priorities** - Specify priority so important jobs can be processed quickly.
|
54
|
+
* **Persistence** - Jobs are stored in memory for speed, but logged to disk for safe keeping.
|
55
|
+
* **Federation** - Horizontal scalability provided through federation by the client.
|
56
|
+
* **Error Handling** - Bury any job which causes an error for later debugging and inspection.
|
56
57
|
|
57
58
|
Keep in mind that these features are supported out of the box with beanstalk and require no special code within this gem to support.
|
58
59
|
In the end, **beanstalk is the ideal job queue** while also being ridiculously easy to install and setup.
|
@@ -82,7 +83,7 @@ Backburner is extremely simple to setup. Just configure basic settings for backb
|
|
82
83
|
|
83
84
|
```ruby
|
84
85
|
Backburner.configure do |config|
|
85
|
-
config.beanstalk_url = "beanstalk://127.0.0.1"
|
86
|
+
config.beanstalk_url = ["beanstalk://127.0.0.1", "beanstalk://127.0.0.1:11301"]
|
86
87
|
config.tube_namespace = "some.app.production"
|
87
88
|
config.on_error = lambda { |e| puts e }
|
88
89
|
config.default_priority = 65536
|
@@ -90,6 +91,10 @@ Backburner.configure do |config|
|
|
90
91
|
end
|
91
92
|
```
|
92
93
|
|
94
|
+
The `beanstalk_url` supports a string such as 'beanstalk://127.0.0.1' or an array of addresses.
|
95
|
+
The `tube_namespace` is the prefix used for all tubes related to this backburner queue.
|
96
|
+
The `on_error` is a callback that gets invoked with the error whenever a job fails
|
97
|
+
|
93
98
|
## Usage
|
94
99
|
|
95
100
|
Backburner allows you to create jobs and place them on a beanstalk queue, and later pull those jobs off the queue and
|
@@ -133,8 +138,8 @@ includes `Backburner::Performable`. Async enqueuing works for both instance and
|
|
133
138
|
```ruby
|
134
139
|
class User
|
135
140
|
include Backburner::Performable
|
136
|
-
queue "
|
137
|
-
queue_priority
|
141
|
+
queue "user-jobs" # defaults to 'user'
|
142
|
+
queue_priority 500 # most urgent priority is 0
|
138
143
|
|
139
144
|
def activate(device_id)
|
140
145
|
@device = Device.find(device_id)
|
@@ -148,14 +153,15 @@ end
|
|
148
153
|
|
149
154
|
# Async works for instance methods on a persisted model
|
150
155
|
@user = User.first
|
151
|
-
@user.async(:
|
156
|
+
@user.async(:ttr => 100, :queue => "activate").activate(@device.id)
|
152
157
|
# ..as well as for class methods
|
153
158
|
User.async(:pri => 100, :delay => 10.seconds).reset_password(@user.id)
|
154
159
|
```
|
155
160
|
|
156
161
|
This will automatically enqueue a job for that user record that will run `activate` with the specified argument.
|
162
|
+
Note that you can set the queue name and queue priority at the class level and
|
163
|
+
you are also able to pass `pri`, `ttr`, `delay` and `queue` directly as options into `async`.
|
157
164
|
The queue name used by default is the normalized class name (i.e `{namespace}.user`) if not otherwise specified.
|
158
|
-
Note you are able to pass `pri`, `ttr`, `delay` and `queue` directly as options into `async`.
|
159
165
|
|
160
166
|
### Working Jobs
|
161
167
|
|
@@ -196,8 +202,8 @@ This will daemonize the worker and store the pid and logs automatically.
|
|
196
202
|
Workers can be easily restricted to processing only a specific set of queues as shown above. However, if you want a worker to
|
197
203
|
process **all** queues instead, then you can leave the queue list blank.
|
198
204
|
|
199
|
-
When you execute a worker without queues specified, any queue for a known job queue class with `include Backburner::Queue` will be processed.
|
200
|
-
queue classes, you can use:
|
205
|
+
When you execute a worker without queues specified, any queue for a known job queue class with `include Backburner::Queue` will be processed.
|
206
|
+
To access the list of known queue classes, you can use:
|
201
207
|
|
202
208
|
```ruby
|
203
209
|
Backburner::Worker.known_queue_classes
|
@@ -236,22 +242,28 @@ If a job fails in beanstalk, the job is automatically buried and must be 'kicked
|
|
236
242
|
|
237
243
|
Right now, all logging happens to standard out and can be piped to a file or any other output manually. More on logging coming later.
|
238
244
|
|
239
|
-
### Front-end
|
245
|
+
### Web Front-end
|
240
246
|
|
241
|
-
|
247
|
+
Be sure to check out the Sinatra-powered project [beanstalkd_view](https://github.com/denniskuczynski/beanstalkd_view)
|
248
|
+
by [denniskuczynski](http://github.com/denniskuczynski) which provides an excellent overview of the tubes and
|
249
|
+
jobs processed by your beanstalk workers. An excellent addition to your Backburner setup.
|
242
250
|
|
243
251
|
### Workers in Production
|
244
252
|
|
245
253
|
Once you have Backburner setup in your application, starting workers is really easy. Once [beanstalkd](http://kr.github.com/beanstalkd/download.html)
|
246
254
|
is installed, your best bet is to use the built-in rake task that comes with Backburner. Simply add the task to your Rakefile:
|
247
255
|
|
248
|
-
|
249
|
-
|
256
|
+
```ruby
|
257
|
+
# Rakefile
|
258
|
+
require 'backburner/tasks'
|
259
|
+
```
|
250
260
|
|
251
261
|
and then you can start the rake task with:
|
252
262
|
|
253
|
-
|
254
|
-
|
263
|
+
```bash
|
264
|
+
$ rake backburner:work
|
265
|
+
$ QUEUES=newsletter-sender,push-message rake backburner:work
|
266
|
+
```
|
255
267
|
|
256
268
|
The best way to deploy these rake tasks is using a monitoring library. We suggest [God](https://github.com/mojombo/god/)
|
257
269
|
which watches processes and ensures their stability. A simple God recipe for Backburner can be found in
|
data/backburner.gemspec
CHANGED
@@ -15,10 +15,10 @@ Gem::Specification.new do |s|
|
|
15
15
|
s.require_paths = ["lib"]
|
16
16
|
s.version = Backburner::VERSION
|
17
17
|
|
18
|
-
s.add_runtime_dependency '
|
18
|
+
s.add_runtime_dependency 'beaneater', '~> 0.1.1'
|
19
19
|
s.add_runtime_dependency 'dante', '~> 0.1.5'
|
20
20
|
|
21
21
|
s.add_development_dependency 'rake'
|
22
|
-
s.add_development_dependency 'minitest'
|
22
|
+
s.add_development_dependency 'minitest', '~> 4.1.0'
|
23
23
|
s.add_development_dependency 'mocha'
|
24
24
|
end
|
data/lib/backburner.rb
CHANGED
@@ -6,6 +6,8 @@ module Backburner
|
|
6
6
|
|
7
7
|
attr_accessor :url, :beanstalk
|
8
8
|
|
9
|
+
# Constructs a backburner connection
|
10
|
+
# `url` can be a string i.e 'localhost:3001' or an array of addresses.
|
9
11
|
def initialize(url)
|
10
12
|
@url = url
|
11
13
|
connect!
|
@@ -22,7 +24,7 @@ module Backburner
|
|
22
24
|
|
23
25
|
# Connects to a beanstalk queue
|
24
26
|
def connect!
|
25
|
-
@beanstalk ||=
|
27
|
+
@beanstalk ||= Beaneater::Pool.new(beanstalk_addresses)
|
26
28
|
end
|
27
29
|
|
28
30
|
# Returns the beanstalk queue addresses
|
@@ -31,8 +33,8 @@ module Backburner
|
|
31
33
|
# beanstalk_addresses => ["localhost:11300"]
|
32
34
|
#
|
33
35
|
def beanstalk_addresses
|
34
|
-
uris = self.url.split(/[\s,]+/)
|
35
|
-
uris.map {|uri| beanstalk_host_and_port(uri)}
|
36
|
+
uris = self.url.is_a?(Array) ? self.url : self.url.split(/[\s,]+/)
|
37
|
+
uris.map { |uri| beanstalk_host_and_port(uri) }
|
36
38
|
end
|
37
39
|
|
38
40
|
# Returns a host and port based on the uri_string given
|
data/lib/backburner/job.rb
CHANGED
@@ -19,7 +19,7 @@ module Backburner
|
|
19
19
|
#
|
20
20
|
def initialize(task)
|
21
21
|
@task = task
|
22
|
-
@body = JSON.parse(task.body)
|
22
|
+
@body = task.body.is_a?(Hash) ? task.body : JSON.parse(task.body)
|
23
23
|
@name, @args = body["class"], body["args"]
|
24
24
|
end
|
25
25
|
|
@@ -46,12 +46,12 @@ module Backburner
|
|
46
46
|
# job_class # => NewsletterSender
|
47
47
|
#
|
48
48
|
def job_class
|
49
|
-
handler = constantize(name) rescue nil
|
50
|
-
raise(JobNotFound, name) unless handler
|
49
|
+
handler = constantize(self.name) rescue nil
|
50
|
+
raise(JobNotFound, self.name) unless handler
|
51
51
|
handler
|
52
52
|
end
|
53
53
|
|
54
|
-
# Timeout job after given time
|
54
|
+
# Timeout job within specified block after given time.
|
55
55
|
#
|
56
56
|
# @example
|
57
57
|
# timeout_job_after(3) { do_something! }
|
data/lib/backburner/version.rb
CHANGED
data/lib/backburner/worker.rb
CHANGED
@@ -15,7 +15,7 @@ module Backburner
|
|
15
15
|
# Enqueues a job to be processed later by a worker
|
16
16
|
# Options: `pri` (priority), `delay` (delay in secs), `ttr` (time to respond), `queue` (queue name)
|
17
17
|
#
|
18
|
-
# @raise [
|
18
|
+
# @raise [Beaneater::NotConnected] If beanstalk fails to connect.
|
19
19
|
# @example
|
20
20
|
# Backburner::Worker.enqueue NewsletterSender, [self.id, user.id], :ttr => 1000
|
21
21
|
#
|
@@ -23,9 +23,9 @@ module Backburner
|
|
23
23
|
pri = opts[:pri] || job_class.queue_priority || Backburner.configuration.default_priority
|
24
24
|
delay = [0, opts[:delay].to_i].max
|
25
25
|
ttr = opts[:ttr] || Backburner.configuration.respond_timeout
|
26
|
-
connection.
|
26
|
+
tube = connection.tubes[expand_tube_name(opts[:queue] || job_class)]
|
27
27
|
data = { :class => job_class.name, :args => args }
|
28
|
-
|
28
|
+
tube.put data.to_json, :pri => pri, :delay => delay, :ttr => ttr
|
29
29
|
end
|
30
30
|
|
31
31
|
# Starts processing jobs in the specified tube_names
|
@@ -39,7 +39,7 @@ module Backburner
|
|
39
39
|
|
40
40
|
# Returns the worker connection
|
41
41
|
# @example
|
42
|
-
# Backburner::Worker.connection # => <
|
42
|
+
# Backburner::Worker.connection # => <Beaneater::Pool>
|
43
43
|
def self.connection
|
44
44
|
@connection ||= Connection.new(Backburner.configuration.beanstalk_url)
|
45
45
|
end
|
@@ -72,7 +72,7 @@ module Backburner
|
|
72
72
|
# Setup beanstalk tube_names and watch all specified tubes for jobs.
|
73
73
|
# Used to prepare job queues before processing jobs.
|
74
74
|
#
|
75
|
-
# @raise [
|
75
|
+
# @raise [Beaneater::NotConnected] If beanstalk fails to connect.
|
76
76
|
# @example
|
77
77
|
# @worker.prepare
|
78
78
|
#
|
@@ -81,10 +81,7 @@ module Backburner
|
|
81
81
|
self.tube_names = Array(self.tube_names)
|
82
82
|
self.tube_names.map! { |name| expand_tube_name(name) }
|
83
83
|
log_info "Working #{tube_names.size} queues: [ #{tube_names.join(', ')} ]"
|
84
|
-
self.
|
85
|
-
self.connection.list_tubes_watched.each do |server, tubes|
|
86
|
-
tubes.each { |tube| self.connection.ignore(tube) unless self.tube_names.include?(tube) }
|
87
|
-
end
|
84
|
+
self.connection.tubes.watch!(*self.tube_names)
|
88
85
|
end
|
89
86
|
|
90
87
|
# Reserves one job within the specified queues
|
@@ -95,7 +92,7 @@ module Backburner
|
|
95
92
|
# @worker.work_one_job
|
96
93
|
#
|
97
94
|
def work_one_job
|
98
|
-
job = Backburner::Job.new(self.connection.reserve)
|
95
|
+
job = Backburner::Job.new(self.connection.tubes.reserve)
|
99
96
|
self.class.log_job_begin(job.body)
|
100
97
|
job.process
|
101
98
|
self.class.log_job_end(job.name)
|
@@ -112,7 +109,7 @@ module Backburner
|
|
112
109
|
# Filtered for tubes that match the known prefix
|
113
110
|
def all_existing_queues
|
114
111
|
known_queues = Backburner::Worker.known_queue_classes.map(&:queue)
|
115
|
-
existing_tubes = self.connection.
|
112
|
+
existing_tubes = self.connection.tubes.all.map(&:name).select { |tube| tube =~ /^#{tube_namespace}/ }
|
116
113
|
known_queues + existing_tubes
|
117
114
|
end
|
118
115
|
|
data/test/connection_test.rb
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
require File.expand_path('../test_helper', __FILE__)
|
2
2
|
|
3
3
|
describe "Backburner::Connection class" do
|
4
|
-
describe "for initialize" do
|
4
|
+
describe "for initialize with single url" do
|
5
5
|
before do
|
6
6
|
@connection = Backburner::Connection.new("beanstalk://localhost")
|
7
7
|
end
|
@@ -11,7 +11,23 @@ describe "Backburner::Connection class" do
|
|
11
11
|
end
|
12
12
|
|
13
13
|
it "should setup beanstalk connection" do
|
14
|
-
assert_kind_of
|
14
|
+
assert_kind_of Beaneater::Pool, @connection.beanstalk
|
15
|
+
end
|
16
|
+
end # initialize single connection
|
17
|
+
|
18
|
+
describe "for initialize with multiple urls" do
|
19
|
+
it "should support single string with commas" do
|
20
|
+
@connection = Backburner::Connection.new("beanstalk://localhost,beanstalk://localhost")
|
21
|
+
connections = @connection.beanstalk.connections
|
22
|
+
assert_equal 2, connections.size
|
23
|
+
assert_equal ['localhost:11300','localhost:11300'], connections.map(&:address)
|
24
|
+
end
|
25
|
+
|
26
|
+
it "should support array of connections" do
|
27
|
+
@connection = Backburner::Connection.new(['beanstalk://127.0.0.1:11300','beanstalk://localhost'])
|
28
|
+
connections = @connection.beanstalk.connections
|
29
|
+
assert_equal 2, @connection.beanstalk.connections.size
|
30
|
+
assert_equal ['127.0.0.1:11300','localhost:11300'], connections.map(&:address)
|
15
31
|
end
|
16
32
|
end # initialize
|
17
33
|
|
@@ -29,7 +45,7 @@ describe "Backburner::Connection class" do
|
|
29
45
|
end
|
30
46
|
|
31
47
|
it "delegate methods to beanstalk connection" do
|
32
|
-
assert_equal "localhost
|
48
|
+
assert_equal "localhost", @connection.connections.first.host
|
33
49
|
end
|
34
50
|
end # delegator
|
35
51
|
end # Connection
|
data/test/job_test.rb
CHANGED
@@ -13,9 +13,24 @@ module NestedDemo
|
|
13
13
|
end
|
14
14
|
|
15
15
|
describe "Backburner::Job module" do
|
16
|
-
describe "for initialize method" do
|
16
|
+
describe "for initialize method with hash" do
|
17
17
|
before do
|
18
|
-
@task_body = {
|
18
|
+
@task_body = { "class" => "NewsletterSender", "args" => ["foo@bar.com", "bar@foo.com"] }
|
19
|
+
@task = stub(:body => @task_body, :ttr => 120, :delete => true, :bury => true)
|
20
|
+
end
|
21
|
+
|
22
|
+
it "should create job with correct task data" do
|
23
|
+
@job = Backburner::Job.new(@task)
|
24
|
+
assert_equal @task, @job.task
|
25
|
+
assert_equal ["class", "args"], @job.body.keys
|
26
|
+
assert_equal @task_body["class"], @job.name
|
27
|
+
assert_equal @task_body["args"], @job.args
|
28
|
+
end
|
29
|
+
end # initialize with json
|
30
|
+
|
31
|
+
describe "for initialize method with json string" do
|
32
|
+
before do
|
33
|
+
@task_body = { "class" => "NewsletterSender", "args" => ["foo@bar.com", "bar@foo.com"] }
|
19
34
|
@task = stub(:body => @task_body.to_json, :ttr => 120, :delete => true, :bury => true)
|
20
35
|
end
|
21
36
|
|
@@ -23,16 +38,16 @@ describe "Backburner::Job module" do
|
|
23
38
|
@job = Backburner::Job.new(@task)
|
24
39
|
assert_equal @task, @job.task
|
25
40
|
assert_equal ["class", "args"], @job.body.keys
|
26
|
-
assert_equal @task_body[
|
27
|
-
assert_equal @task_body[
|
41
|
+
assert_equal @task_body["class"], @job.name
|
42
|
+
assert_equal @task_body["args"], @job.args
|
28
43
|
end
|
29
|
-
end # initialize
|
44
|
+
end # initialize with json
|
30
45
|
|
31
46
|
describe "for process method" do
|
32
47
|
describe "with valid task" do
|
33
48
|
before do
|
34
|
-
@task_body = {
|
35
|
-
@task = stub(:body => @task_body
|
49
|
+
@task_body = { "class" => "NestedDemo::TestJobC", "args" => [56] }
|
50
|
+
@task = stub(:body => @task_body, :ttr => 120, :delete => true, :bury => true)
|
36
51
|
@task.expects(:delete).once
|
37
52
|
end
|
38
53
|
|
@@ -45,8 +60,8 @@ describe "Backburner::Job module" do
|
|
45
60
|
|
46
61
|
describe "with invalid task" do
|
47
62
|
before do
|
48
|
-
@task_body = {
|
49
|
-
@task = stub(:body => @task_body
|
63
|
+
@task_body = { "class" => "NestedDemo::TestJobD", "args" => [56] }
|
64
|
+
@task = stub(:body => @task_body, :ttr => 120, :delete => true, :bury => true)
|
50
65
|
@task.expects(:delete).never
|
51
66
|
end
|
52
67
|
|
@@ -58,8 +73,8 @@ describe "Backburner::Job module" do
|
|
58
73
|
|
59
74
|
describe "with invalid class" do
|
60
75
|
before do
|
61
|
-
@task_body = {
|
62
|
-
@task = stub(:body => @task_body
|
76
|
+
@task_body = { "class" => "NestedDemo::TestJobY", "args" => [56] }
|
77
|
+
@task = stub(:body => @task_body, :ttr => 120, :delete => true, :bury => true)
|
63
78
|
@task.expects(:delete).never
|
64
79
|
end
|
65
80
|
|
@@ -72,8 +87,8 @@ describe "Backburner::Job module" do
|
|
72
87
|
|
73
88
|
describe "for bury method" do
|
74
89
|
before do
|
75
|
-
@task_body = {
|
76
|
-
@task = stub(:body => @task_body
|
90
|
+
@task_body = { "class" => "NestedDemo::TestJobC", "args" => [56] }
|
91
|
+
@task = stub(:body => @task_body, :ttr => 120, :delete => true, :bury => true)
|
77
92
|
@task.expects(:bury).once
|
78
93
|
end
|
79
94
|
|
data/test/test_helper.rb
CHANGED
@@ -86,11 +86,8 @@ class MiniTest::Spec
|
|
86
86
|
def pop_one_job(tube_name)
|
87
87
|
connection = Backburner::Worker.connection
|
88
88
|
tube_name = [Backburner.configuration.tube_namespace, tube_name].join(".")
|
89
|
-
connection.watch(tube_name)
|
90
|
-
connection.
|
91
|
-
|
92
|
-
end
|
93
|
-
silenced(3) { @res = connection.reserve }
|
94
|
-
return @res, JSON.parse(@res.body)
|
89
|
+
connection.tubes.watch!(tube_name)
|
90
|
+
silenced(3) { @res = connection.tubes.reserve }
|
91
|
+
return @res, @res.body
|
95
92
|
end
|
96
93
|
end # MiniTest::Spec
|
data/test/worker_test.rb
CHANGED
@@ -63,7 +63,7 @@ describe "Backburner::Worker module" do
|
|
63
63
|
describe "for connection class method" do
|
64
64
|
it "should return the beanstalk connection" do
|
65
65
|
assert_equal "beanstalk://localhost", Backburner::Worker.connection.url
|
66
|
-
assert_kind_of
|
66
|
+
assert_kind_of Beaneater::Pool, Backburner::Worker.connection.beanstalk
|
67
67
|
end
|
68
68
|
end # connection
|
69
69
|
|
@@ -104,7 +104,7 @@ describe "Backburner::Worker module" do
|
|
104
104
|
worker = Backburner::Worker.new(["foo", "bar"])
|
105
105
|
out = capture_stdout { worker.prepare }
|
106
106
|
assert_equal ["demo.test.foo", "demo.test.bar"], worker.tube_names
|
107
|
-
assert_same_elements ["demo.test.foo", "demo.test.bar"], Backburner::Worker.connection.
|
107
|
+
assert_same_elements ["demo.test.foo", "demo.test.bar"], Backburner::Worker.connection.tubes.watched.map(&:name)
|
108
108
|
assert_match /demo\.test\.foo/, out
|
109
109
|
end # multiple
|
110
110
|
|
@@ -112,7 +112,7 @@ describe "Backburner::Worker module" do
|
|
112
112
|
worker = Backburner::Worker.new("foo")
|
113
113
|
out = capture_stdout { worker.prepare }
|
114
114
|
assert_equal ["demo.test.foo"], worker.tube_names
|
115
|
-
assert_same_elements ["demo.test.foo"], Backburner::Worker.connection.
|
115
|
+
assert_same_elements ["demo.test.foo"], Backburner::Worker.connection.tubes.watched.map(&:name)
|
116
116
|
assert_match /demo\.test\.foo/, out
|
117
117
|
end # single
|
118
118
|
|
@@ -121,7 +121,7 @@ describe "Backburner::Worker module" do
|
|
121
121
|
worker = Backburner::Worker.new
|
122
122
|
out = capture_stdout { worker.prepare }
|
123
123
|
assert_equal ["demo.test.foo", "demo.test.bar"], worker.tube_names
|
124
|
-
assert_same_elements ["demo.test.foo", "demo.test.bar"], Backburner::Worker.connection.
|
124
|
+
assert_same_elements ["demo.test.foo", "demo.test.bar"], Backburner::Worker.connection.tubes.watched.map(&:name)
|
125
125
|
assert_match /demo\.test\.foo/, out
|
126
126
|
end
|
127
127
|
|
@@ -130,7 +130,7 @@ describe "Backburner::Worker module" do
|
|
130
130
|
worker = Backburner::Worker.new
|
131
131
|
out = capture_stdout { worker.prepare }
|
132
132
|
assert_equal ["demo.test.bar"], worker.tube_names
|
133
|
-
assert_same_elements ["demo.test.bar"], Backburner::Worker.connection.
|
133
|
+
assert_same_elements ["demo.test.bar"], Backburner::Worker.connection.tubes.watched.map(&:name)
|
134
134
|
assert_match /demo\.test\.bar/, out
|
135
135
|
end # all assign
|
136
136
|
|
@@ -138,7 +138,7 @@ describe "Backburner::Worker module" do
|
|
138
138
|
worker = Backburner::Worker.new
|
139
139
|
out = capture_stdout { worker.prepare }
|
140
140
|
assert_contains worker.tube_names, "demo.test.test-job"
|
141
|
-
assert_contains Backburner::Worker.connection.
|
141
|
+
assert_contains Backburner::Worker.connection.tubes.watched.map(&:name), "demo.test.test-job"
|
142
142
|
assert_match /demo\.test\.test-job/, out
|
143
143
|
end # all read
|
144
144
|
end # prepare
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: backburner
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0
|
4
|
+
version: 0.1.0
|
5
5
|
prerelease:
|
6
6
|
platform: ruby
|
7
7
|
authors:
|
@@ -9,22 +9,27 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2012-
|
12
|
+
date: 2012-11-05 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
|
-
name:
|
16
|
-
requirement:
|
15
|
+
name: beaneater
|
16
|
+
requirement: !ruby/object:Gem::Requirement
|
17
17
|
none: false
|
18
18
|
requirements:
|
19
19
|
- - ~>
|
20
20
|
- !ruby/object:Gem::Version
|
21
|
-
version:
|
21
|
+
version: 0.1.1
|
22
22
|
type: :runtime
|
23
23
|
prerelease: false
|
24
|
-
version_requirements:
|
24
|
+
version_requirements: !ruby/object:Gem::Requirement
|
25
|
+
none: false
|
26
|
+
requirements:
|
27
|
+
- - ~>
|
28
|
+
- !ruby/object:Gem::Version
|
29
|
+
version: 0.1.1
|
25
30
|
- !ruby/object:Gem::Dependency
|
26
31
|
name: dante
|
27
|
-
requirement:
|
32
|
+
requirement: !ruby/object:Gem::Requirement
|
28
33
|
none: false
|
29
34
|
requirements:
|
30
35
|
- - ~>
|
@@ -32,10 +37,15 @@ dependencies:
|
|
32
37
|
version: 0.1.5
|
33
38
|
type: :runtime
|
34
39
|
prerelease: false
|
35
|
-
version_requirements:
|
40
|
+
version_requirements: !ruby/object:Gem::Requirement
|
41
|
+
none: false
|
42
|
+
requirements:
|
43
|
+
- - ~>
|
44
|
+
- !ruby/object:Gem::Version
|
45
|
+
version: 0.1.5
|
36
46
|
- !ruby/object:Gem::Dependency
|
37
47
|
name: rake
|
38
|
-
requirement:
|
48
|
+
requirement: !ruby/object:Gem::Requirement
|
39
49
|
none: false
|
40
50
|
requirements:
|
41
51
|
- - ! '>='
|
@@ -43,21 +53,31 @@ dependencies:
|
|
43
53
|
version: '0'
|
44
54
|
type: :development
|
45
55
|
prerelease: false
|
46
|
-
version_requirements:
|
47
|
-
- !ruby/object:Gem::Dependency
|
48
|
-
name: minitest
|
49
|
-
requirement: &2152604860 !ruby/object:Gem::Requirement
|
56
|
+
version_requirements: !ruby/object:Gem::Requirement
|
50
57
|
none: false
|
51
58
|
requirements:
|
52
59
|
- - ! '>='
|
53
60
|
- !ruby/object:Gem::Version
|
54
61
|
version: '0'
|
62
|
+
- !ruby/object:Gem::Dependency
|
63
|
+
name: minitest
|
64
|
+
requirement: !ruby/object:Gem::Requirement
|
65
|
+
none: false
|
66
|
+
requirements:
|
67
|
+
- - ~>
|
68
|
+
- !ruby/object:Gem::Version
|
69
|
+
version: 4.1.0
|
55
70
|
type: :development
|
56
71
|
prerelease: false
|
57
|
-
version_requirements:
|
72
|
+
version_requirements: !ruby/object:Gem::Requirement
|
73
|
+
none: false
|
74
|
+
requirements:
|
75
|
+
- - ~>
|
76
|
+
- !ruby/object:Gem::Version
|
77
|
+
version: 4.1.0
|
58
78
|
- !ruby/object:Gem::Dependency
|
59
79
|
name: mocha
|
60
|
-
requirement:
|
80
|
+
requirement: !ruby/object:Gem::Requirement
|
61
81
|
none: false
|
62
82
|
requirements:
|
63
83
|
- - ! '>='
|
@@ -65,7 +85,12 @@ dependencies:
|
|
65
85
|
version: '0'
|
66
86
|
type: :development
|
67
87
|
prerelease: false
|
68
|
-
version_requirements:
|
88
|
+
version_requirements: !ruby/object:Gem::Requirement
|
89
|
+
none: false
|
90
|
+
requirements:
|
91
|
+
- - ! '>='
|
92
|
+
- !ruby/object:Gem::Version
|
93
|
+
version: '0'
|
69
94
|
description: Beanstalk background job processing made easy
|
70
95
|
email:
|
71
96
|
- nesquena@gmail.com
|
@@ -76,6 +101,7 @@ extra_rdoc_files: []
|
|
76
101
|
files:
|
77
102
|
- .gitignore
|
78
103
|
- .travis.yml
|
104
|
+
- CHANGELOG.md
|
79
105
|
- Gemfile
|
80
106
|
- LICENSE
|
81
107
|
- README.md
|
@@ -128,7 +154,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
128
154
|
version: '0'
|
129
155
|
requirements: []
|
130
156
|
rubyforge_project:
|
131
|
-
rubygems_version: 1.8.
|
157
|
+
rubygems_version: 1.8.24
|
132
158
|
signing_key:
|
133
159
|
specification_version: 3
|
134
160
|
summary: Reliable beanstalk background job processing made easy for Ruby and Sinatra
|