qu 0.1.0

Sign up to get free protection for your applications and to get access to all the features.
@@ -0,0 +1,4 @@
1
+ *.gem
2
+ .bundle
3
+ Gemfile.lock
4
+ pkg/*
data/Gemfile ADDED
@@ -0,0 +1,19 @@
1
+ source "http://rubygems.org"
2
+ gemspec :name => 'qu'
3
+
4
+ Dir['qu-*.gemspec'].each do |gemspec|
5
+ plugin = gemspec.scan(/qu-(.*)\.gemspec/).to_s
6
+
7
+ group plugin do
8
+ gemspec(:name => "qu-#{plugin}", :development_group => plugin)
9
+ end
10
+ end
11
+
12
+ group :test do
13
+ gem 'SystemTimer', :platform => :mri_18
14
+ gem 'ruby-debug', :platform => :mri_18
15
+ gem 'rake'
16
+ gem 'rspec', '~> 2.0'
17
+ gem 'guard-rspec'
18
+ gem 'guard-bundler'
19
+ end
@@ -0,0 +1,12 @@
1
+ guard 'rspec', :version => 2 do
2
+ watch(%r{^spec/.+_spec\.rb$})
3
+ watch(%r{^lib/(.+)\.rb$}) { |m| "spec/#{m[1]}_spec.rb" }
4
+ watch(%r{^lib/qu/backend/spec\.rb$}) { |m| "spec/qu/backend" }
5
+ watch('spec/spec_helper.rb') { "spec" }
6
+ watch(%r{^spec/support/(.+)\.rb$}) { "spec" }
7
+ end
8
+
9
+ guard 'bundler' do
10
+ watch('Gemfile')
11
+ watch(/^.+\.gemspec/)
12
+ end
@@ -0,0 +1,140 @@
1
+ # Qu
2
+
3
+ Qu is a Ruby library for queuing and processing background jobs. It is heavily inspired by delayed_job and Resque.
4
+
5
+ Qu was created to overcome some shortcomings in the existing queuing libraries that we experienced at [Ordered List](http://orderedlist.com) while building [SpeakerDeck](http://speakerdeck.com), [Gaug.es](http://get.gaug.es) and [Harmony](http://get.harmonyapp.com). The advantages of Qu are:
6
+
7
+ * Multiple backends (redis, mongo)
8
+ * Jobs are requeued when worker is killed
9
+ * Resque-like API
10
+
11
+ ## Information & Help
12
+
13
+ * Find more information on the [Wiki](https://github.com/bkeepers/qu/wiki).
14
+ * Post to the [Google Group](http://groups.google.com/group/qu-users) for help or questions.
15
+ * See the [issue tracker](https://github.com/bkeepers/qu/issues) for known issues or to report an issue.
16
+
17
+ ## Installation
18
+
19
+ ### Rails
20
+
21
+ Decide which backend you want to use and add the gem for it to your `Gemfile`.
22
+
23
+ ``` ruby
24
+ gem 'qu-redis'
25
+ ```
26
+
27
+ That's all you need to do!
28
+
29
+ ## Usage
30
+
31
+ Jobs are any class that responds to the `.perform` method:
32
+
33
+ ``` ruby
34
+ class ProcessPresentation
35
+ def self.perform(presentation_id)
36
+ presentation = Presentation.find(presentation_id)
37
+ presentation.process!
38
+ end
39
+ end
40
+ ```
41
+
42
+ You can add a job to the queue by calling the `enqueue` method:
43
+
44
+ ``` ruby
45
+ job = Qu.enqueue ProcessPresentation, @presentation.id
46
+ puts "Enqueued job #{job.id}"
47
+ ```
48
+
49
+ Any additional parameters passed to the `enqueue` method will be passed on to the `perform` method of your job class. These parameters will be stored in the backend, so they must be simple types that can easily be serialized and unserialized. So don't try to pass in an ActiveRecord object.
50
+
51
+ Processing the jobs on the queue can be done with a Rake task:
52
+
53
+ ``` sh
54
+ $ bundle exec rake qu:work
55
+ ```
56
+
57
+ You can easily inspect the queue or clear it:
58
+
59
+ ``` ruby
60
+ puts "Jobs on the queue:", Qu.length
61
+ Qu.clear
62
+ ```
63
+
64
+ ### Queues
65
+
66
+ The `default` queue is used, um…by default. Jobs that don't specify a queue will be placed in that queue, and workers that don't specify a queue will work on that queue.
67
+
68
+ However, if you have some background jobs that are more or less important, or some that take longer than others, you may want to consider using multiple queues. You can have workers dedicated to specific queues, or simply tell all your workers to work on the most important queue first.
69
+
70
+ Jobs can be placed in a specific queue by setting the queue variable:
71
+
72
+ ``` ruby
73
+ class CallThePresident
74
+ @queue = :urgent
75
+
76
+ def self.perform(message)
77
+ # …
78
+ end
79
+ end
80
+ ```
81
+
82
+ You can then tell workers to work on this queue by passing an environment variable
83
+
84
+ ``` sh
85
+ $ bundle exec rake qu:work QUEUES=urgent,default
86
+ ```
87
+
88
+ Note that if you still want your worker to process the default queue, you must specify it. Queues will be process in the order they are specified.
89
+
90
+ You can also get the length or clear a specific queue:
91
+
92
+ ``` ruby
93
+ Qu.length(:urgent)
94
+ Qu.clear(:urgent)
95
+ ```
96
+
97
+ ## Configuration
98
+
99
+ Most of the configuration for Qu should be automatic. It will also automatically detect ENV variables from Heroku for backend connections, so you shouldn't need to do anything to configure the backend.
100
+
101
+ However, if you do need to customize it, you can by calling the `Qu.configure`:
102
+
103
+ ``` ruby
104
+ Qu.configure do |c|
105
+ c.connection = Redis::Namespace.new('myapp:qu', :redis => Redis.connect)
106
+ end
107
+ ```
108
+
109
+ ## Why another queuing library?
110
+
111
+ Resque and delayed_job are both great, but both of them have shortcomings that can be frustrating in production applications.
112
+
113
+ delayed_job was a brilliantly simple pioneer in the world of database-backed queues. While most asynchronous queuing systems were tending toward overly complex, it made use of your existing database and just worked. But there are a few flaws:
114
+
115
+ * Occasionally fails silently.
116
+ * Use of priority instead of separate named queues.
117
+ * Contention in the ActiveRecord backend with multiple workers. Occasionally the same job gets performed by multiple workers.
118
+
119
+ Resque, the wiser relative of delayed_job, fixes most of those issues. But in doing so, it forces some of its beliefs on you, and sometimes those beliefs just don't make sense for your environment. Here are some of the flaws of Resque:
120
+
121
+ * Redis is a great queue backend, but it doesn't make sense for every environment.
122
+ * Workers lose jobs when they are forced to quit. This has especially been an issue on Heroku.
123
+ * Forking before each job prevents memory leaks, but it is terribly inefficient in environments with a lot of fast jobs (the resque-jobs-per-fork plugin alleviates this)
124
+
125
+ Those shortcomings lead us to write Qu. It is not perfect, but we hope to overcome the issues we faced with other queuing libraries.
126
+
127
+ ## Contributing
128
+
129
+ If you find what looks like a bug:
130
+
131
+ 1. Search the [mailing list](http://groups.google.com/group/qu-users) to see if anyone else had the same issue.
132
+ 2. Check the [GitHub issue tracker](http://github.com/bkeepers/qu/issues/) to see if anyone else has reported issue.
133
+ 3. If you don't see anything, create an issue with information on how to reproduce it.
134
+
135
+ If you want to contribute an enhancement or a fix:
136
+
137
+ 1. Fork the project on GitHub.
138
+ 2. Make your changes with tests.
139
+ 3. Commit the changes without making changes to the Rakefile, Gemfile, gemspec, or any other files that aren't related to your enhancement or fix
140
+ 4. Send a pull request.
@@ -0,0 +1,46 @@
1
+ # -*- encoding: utf-8 -*-
2
+ $:.push File.expand_path("../lib", __FILE__)
3
+ require "qu/version"
4
+
5
+ desc 'Build gem into the pkg directory'
6
+ task :build do
7
+ FileUtils.rm_rf('pkg')
8
+ Dir['*.gemspec'].each do |gemspec|
9
+ system "gem build #{gemspec}"
10
+ end
11
+ FileUtils.mkdir_p('pkg')
12
+ FileUtils.mv(Dir['*.gem'], 'pkg')
13
+ end
14
+
15
+ desc 'Tags version, pushes to remote, and pushes gem'
16
+ task :release => :build do
17
+ sh "git tag v#{Qu::VERSION}"
18
+ sh "git push origin master"
19
+ sh "git push origin v#{Qu::VERSION}"
20
+ sh "ls pkg/*.gem | xargs gem push"
21
+ end
22
+
23
+ require 'rspec/core/rake_task'
24
+
25
+ desc "Run all specs"
26
+ RSpec::Core::RakeTask.new(:spec) do |t|
27
+ t.rspec_opts = %w[--color]
28
+ t.verbose = false
29
+ end
30
+
31
+ namespace :spec do
32
+ Backends = %w(mongo redis)
33
+
34
+ Backends.each do |backend|
35
+ desc "Run specs for #{backend} backend"
36
+ RSpec::Core::RakeTask.new(backend) do |t|
37
+ t.rspec_opts = %w[--color]
38
+ t.verbose = false
39
+ t.pattern = "spec/qu/backend/#{backend}_spec.rb"
40
+ end
41
+ end
42
+
43
+ task :backends => Backends
44
+ end
45
+
46
+ task :default => :spec
@@ -0,0 +1,27 @@
1
+ require 'qu/version'
2
+ require 'qu/failure'
3
+ require 'qu/job'
4
+ require 'qu/backend/base'
5
+
6
+ require 'forwardable'
7
+
8
+ module Qu
9
+ autoload :Worker, 'qu/worker'
10
+
11
+ extend SingleForwardable
12
+ extend self
13
+
14
+ attr_accessor :backend, :failure
15
+
16
+ def_delegators :backend, :enqueue, :length, :queues, :reserve, :clear, :connection=
17
+
18
+ def backend
19
+ @backend || raise("Qu backend not configured. Install one of the backend gems like qu-redis.")
20
+ end
21
+
22
+ def configure(&block)
23
+ block.call(self)
24
+ end
25
+ end
26
+
27
+ require 'qu/railtie' if defined?(Rails)
@@ -0,0 +1,19 @@
1
+ require 'multi_json'
2
+
3
+ module Qu
4
+ module Backend
5
+ class Base
6
+ attr_accessor :connection
7
+
8
+ private
9
+
10
+ def encode(data)
11
+ MultiJson.encode(data) if data
12
+ end
13
+
14
+ def decode(data)
15
+ MultiJson.decode(data) if data
16
+ end
17
+ end
18
+ end
19
+ end
@@ -0,0 +1,265 @@
1
+ class SimpleJob
2
+ def self.perform
3
+ end
4
+ end
5
+
6
+ class CustomQueue
7
+ @queue = :custom
8
+ end
9
+
10
+ shared_examples_for 'a backend' do
11
+ let(:worker) { Qu::Worker.new('default') }
12
+
13
+ before(:all) do
14
+ Qu.backend = described_class.new
15
+ end
16
+
17
+ before do
18
+ subject.clear
19
+ subject.clear_workers
20
+ end
21
+
22
+ describe 'enqueue' do
23
+ it 'should return a job id' do
24
+ subject.enqueue(SimpleJob).should be_instance_of(Qu::Job)
25
+ end
26
+
27
+ it 'should add a job to the queue' do
28
+ job = subject.enqueue(SimpleJob)
29
+ job.queue.should == 'default'
30
+ subject.length(job.queue).should == 1
31
+ end
32
+
33
+ it 'should add queue to list of queues' do
34
+ subject.queues.should == []
35
+ job = subject.enqueue(SimpleJob)
36
+ subject.queues.should == [job.queue]
37
+ end
38
+
39
+ it 'should assign a different job id for the same job enqueue multiple times' do
40
+ subject.enqueue(SimpleJob).id.should_not == subject.enqueue(SimpleJob).id
41
+ end
42
+ end
43
+
44
+ describe 'length' do
45
+ it 'should use the default queue by default' do
46
+ subject.length.should == 0
47
+ subject.enqueue(SimpleJob)
48
+ subject.length.should == 1
49
+ end
50
+ end
51
+
52
+ describe 'clear' do
53
+ it 'should clear jobs for given queue' do
54
+ job = subject.enqueue SimpleJob
55
+ subject.length(job.queue).should == 1
56
+ subject.clear(job.queue)
57
+ subject.length(job.queue).should == 0
58
+ subject.queues.should_not include(job.queue)
59
+ end
60
+
61
+ it 'should not clear jobs for a different queue' do
62
+ job = subject.enqueue SimpleJob
63
+ subject.clear('other')
64
+ subject.length(job.queue).should == 1
65
+ end
66
+
67
+ it 'should clear all queues without any args' do
68
+ subject.enqueue(SimpleJob).queue.should == 'default'
69
+ subject.enqueue(CustomQueue).queue.should == 'custom'
70
+ subject.length('default').should == 1
71
+ subject.length('custom').should == 1
72
+ subject.clear
73
+ subject.length('default').should == 0
74
+ subject.length('custom').should == 0
75
+ end
76
+
77
+ it 'should clear failed queue without any args' do
78
+ job = subject.enqueue SimpleJob
79
+ subject.failed(job, Exception.new)
80
+ subject.length('failed').should == 1
81
+ subject.clear
82
+ subject.length('failed').should == 0
83
+ end
84
+
85
+ it 'should not clear failed queue with specified queues' do
86
+ job = subject.enqueue SimpleJob
87
+ subject.failed(job, Exception.new)
88
+ subject.length('failed').should == 1
89
+ subject.clear('default')
90
+ subject.length('failed').should == 1
91
+ end
92
+ end
93
+
94
+ describe 'reserve' do
95
+ before do
96
+ @job = subject.enqueue SimpleJob
97
+ end
98
+
99
+ it 'should return next job' do
100
+ subject.reserve(worker).id.should == @job.id
101
+ end
102
+
103
+ it 'should not return an already reserved job' do
104
+ another_job = subject.enqueue SimpleJob
105
+ subject.reserve(worker).id.should_not == subject.reserve(worker).id
106
+ end
107
+
108
+ it 'should return next job in given queues' do
109
+ subject.enqueue SimpleJob
110
+ job = subject.enqueue CustomQueue
111
+ subject.enqueue SimpleJob
112
+
113
+ worker = Qu::Worker.new('custom', 'default')
114
+
115
+ subject.reserve(worker).id.should == job.id
116
+ end
117
+
118
+ it 'should not return job from different queue' do
119
+ worker = Qu::Worker.new('video')
120
+ timeout { subject.reserve(worker) }.should be_nil
121
+ end
122
+
123
+ it 'should block by default if no jobs available' do
124
+ subject.clear
125
+ timeout(1) do
126
+ subject.reserve(worker)
127
+ fail("#reserve should block")
128
+ end
129
+ end
130
+
131
+ it 'should not block if :block option is set to false' do
132
+ timeout(1) do
133
+ subject.reserve(worker, :block => false)
134
+ true
135
+ end.should be_true
136
+ end
137
+
138
+ def timeout(count = 0.1, &block)
139
+ SystemTimer.timeout(count, &block)
140
+ rescue Timeout::Error
141
+ nil
142
+ end
143
+ end
144
+
145
+ describe 'failed' do
146
+ let(:job) { Qu::Job.new('1', SimpleJob, []) }
147
+
148
+ it 'should add to failure queue' do
149
+ subject.failed(job, Exception.new)
150
+ subject.length('failed').should == 1
151
+ end
152
+
153
+ it 'should not add failed queue to the list of queues' do
154
+ subject.failed(job, Exception.new)
155
+ subject.queues.should_not include('failed')
156
+ end
157
+ end
158
+
159
+ describe 'completed' do
160
+ it 'should be defined' do
161
+ subject.respond_to?(:completed).should be_true
162
+ end
163
+ end
164
+
165
+ describe 'release' do
166
+ before do
167
+ subject.enqueue SimpleJob
168
+ end
169
+
170
+ it 'should add the job back on the queue' do
171
+ job = subject.reserve(worker)
172
+ subject.length(job.queue).should == 0
173
+ subject.release(job)
174
+ subject.length(job.queue).should == 1
175
+ end
176
+ end
177
+
178
+ describe 'requeue' do
179
+ context 'with a failed job' do
180
+ before do
181
+ subject.enqueue(SimpleJob)
182
+ @job = subject.reserve(worker)
183
+ subject.failed(@job, Exception.new)
184
+ end
185
+
186
+ it 'should add the job back on the queue' do
187
+ subject.length(@job.queue).should == 0
188
+ subject.requeue(@job.id)
189
+ subject.length(@job.queue).should == 1
190
+
191
+ job = subject.reserve(worker)
192
+ job.should be_instance_of(Qu::Job)
193
+ job.id.should == @job.id
194
+ job.klass.should == @job.klass
195
+ job.args.should == @job.args
196
+ end
197
+
198
+ it 'should remove the job from the failed jobs' do
199
+ subject.length('failed').should == 1
200
+ subject.requeue(@job.id)
201
+ subject.length('failed').should == 0
202
+ end
203
+
204
+ it 'should return the job' do
205
+ subject.requeue(@job.id).id.should == @job.id
206
+ end
207
+ end
208
+
209
+ context 'without a failed job' do
210
+ it 'should return false' do
211
+ subject.requeue('1').should be_false
212
+ end
213
+ end
214
+ end
215
+
216
+ describe 'register_worker' do
217
+ let(:worker) { Qu::Worker.new('default') }
218
+
219
+ it 'should add worker to array of workers' do
220
+ subject.register_worker(worker)
221
+ subject.workers.size.should == 1
222
+ subject.workers.first.attributes.should == worker.attributes
223
+ end
224
+ end
225
+
226
+ describe 'clear_workers' do
227
+ before { subject.register_worker Qu::Worker.new('default') }
228
+
229
+ it 'should remove workers' do
230
+ subject.workers.size.should == 1
231
+ subject.clear_workers
232
+ subject.workers.size.should == 0
233
+ end
234
+ end
235
+
236
+ describe 'unregister_worker' do
237
+ before { subject.register_worker Qu::Worker.new('default') }
238
+
239
+ it 'should remove worker' do
240
+ subject.unregister_worker(worker.id)
241
+ subject.workers.size.should == 0
242
+ end
243
+
244
+ it 'should not remove other workers' do
245
+ other_worker = Qu::Worker.new('other')
246
+ subject.register_worker(other_worker)
247
+ subject.workers.size.should == 2
248
+ subject.unregister_worker(other_worker.id)
249
+ subject.workers.size.should == 1
250
+ subject.workers.first.id.should == worker.id
251
+ end
252
+ end
253
+
254
+ describe 'connection=' do
255
+ it 'should allow setting the connection' do
256
+ connection = mock('a connection')
257
+ subject.connection = connection
258
+ subject.connection.should == connection
259
+ end
260
+
261
+ it 'should provide a default connection' do
262
+ subject.connection.should_not be_nil
263
+ end
264
+ end
265
+ end
@@ -0,0 +1,4 @@
1
+ module Qu
2
+ module Failure
3
+ end
4
+ end
@@ -0,0 +1,37 @@
1
+ module Qu
2
+ class Job
3
+ attr_accessor :id, :klass, :args
4
+
5
+ def initialize(id, klass, args)
6
+ @id, @args = id, args
7
+
8
+ @klass = klass.is_a?(Class) ? klass : constantize(klass)
9
+ end
10
+
11
+ def queue
12
+ (klass.instance_variable_get(:@queue) || 'default').to_s
13
+ end
14
+
15
+ def perform
16
+ klass.perform(*args)
17
+ Qu.backend.completed(self)
18
+ rescue Qu::Worker::Abort
19
+ Qu.backend.release(self)
20
+ raise
21
+ rescue Exception => e
22
+ Qu.failure.create(self, e) if Qu.failure
23
+ Qu.backend.failed(self, e)
24
+ end
25
+
26
+ protected
27
+
28
+ def constantize(class_name)
29
+ constant = Object
30
+ class_name.split('::').each do |name|
31
+ constant = constant.const_get(name) || constant.const_missing(name)
32
+ end
33
+ constant
34
+ end
35
+
36
+ end
37
+ end
@@ -0,0 +1,9 @@
1
+ require 'qu'
2
+
3
+ module Qu
4
+ class Railtie < Rails::Railtie
5
+ rake_tasks do
6
+ load "qu/tasks.rb"
7
+ end
8
+ end
9
+ end
@@ -0,0 +1,7 @@
1
+ namespace :qu do
2
+ desc "Start a worker"
3
+ task :work => :environment do
4
+ queues = (ENV['QUEUES'] || ENV['QUEUE'] || 'default').to_s.split(',')
5
+ Qu::Worker.new(*queues).start
6
+ end
7
+ end
@@ -0,0 +1,3 @@
1
+ module Qu
2
+ VERSION = "0.1.0"
3
+ end
@@ -0,0 +1,63 @@
1
+ module Qu
2
+ class Worker
3
+ attr_accessor :queues
4
+
5
+ class Abort < Exception
6
+ end
7
+
8
+ def initialize(*queues)
9
+ @queues = queues.flatten
10
+ self.attributes = @queues.pop if @queues.last.is_a?(Hash)
11
+ @queues << 'default' if @queues.empty?
12
+ end
13
+
14
+ def attributes=(attrs)
15
+ attrs.each do |attr, value|
16
+ self.instance_variable_set("@#{attr}", value)
17
+ end
18
+ end
19
+
20
+ def attributes
21
+ {'hostname' => hostname, 'pid' => pid, 'queues' => queues}
22
+ end
23
+
24
+ def handle_signals
25
+ %W(INT TERM).each do |sig|
26
+ trap(sig) { raise Abort }
27
+ end
28
+ end
29
+
30
+ def work_off
31
+ while job = Qu.reserve(self, :block => false)
32
+ job.perform
33
+ end
34
+ end
35
+
36
+ def work
37
+ job = Qu.reserve(self)
38
+ job.perform
39
+ end
40
+
41
+ def start
42
+ handle_signals
43
+ Qu.backend.register_worker(self)
44
+ loop { work }
45
+ rescue Abort => e
46
+ # Ok, we'll shut down, but give us a sec
47
+ ensure
48
+ Qu.backend.unregister_worker(self)
49
+ end
50
+
51
+ def id
52
+ "#{hostname}:#{pid}:#{queues.join(',')}"
53
+ end
54
+
55
+ def pid
56
+ @pid ||= Process.pid
57
+ end
58
+
59
+ def hostname
60
+ @hostname ||= `hostname`.strip
61
+ end
62
+ end
63
+ end
@@ -0,0 +1,22 @@
1
+ # -*- encoding: utf-8 -*-
2
+ $:.push File.expand_path("../lib", __FILE__)
3
+ require "qu/version"
4
+
5
+ plugins = Dir['qu-*.gemspec'].map {|gemspec| gemspec.scan(/qu-(.*)\.gemspec/).to_s }.join('\|')
6
+
7
+ Gem::Specification.new do |s|
8
+ s.name = "qu"
9
+ s.version = Qu::VERSION
10
+ s.authors = ["Brandon Keepers"]
11
+ s.email = ["brandon@opensoul.org"]
12
+ s.homepage = "http://github.com/bkeepers/qu"
13
+ s.summary = %q{}
14
+ s.description = %q{}
15
+
16
+ s.files = `git ls-files | grep -v '#{plugins}'`.split("\n")
17
+ s.test_files = `git ls-files -- spec | grep -v '#{plugins}'`.split("\n")
18
+ s.executables = `git ls-files -- bin`.split("\n").map{ |f| File.basename(f) }
19
+ s.require_paths = ["lib"]
20
+
21
+ s.add_dependency 'multi_json'
22
+ end
@@ -0,0 +1,78 @@
1
+ require 'spec_helper'
2
+
3
+ describe Qu::Job do
4
+ class MyJob
5
+ @queue = :custom
6
+ end
7
+
8
+ describe 'queue' do
9
+ it 'should default to "default"' do
10
+ Qu::Job.new('1', SimpleJob, []).queue.should == 'default'
11
+ end
12
+
13
+ it 'should get queue from job instance variable' do
14
+ Qu::Job.new('1', MyJob, []).queue.should == 'custom'
15
+ end
16
+ end
17
+
18
+ describe 'klass' do
19
+ it 'should constantize string' do
20
+ Qu::Job.new('1', 'MyJob', []).klass.should == MyJob
21
+ end
22
+
23
+ it 'should find namespaced jobs' do
24
+ Qu::Job.new('1', 'Qu::Job', []).klass.should == Qu::Job
25
+ end
26
+ end
27
+
28
+ describe 'perform' do
29
+ subject { Qu::Job.new('1', SimpleJob, []) }
30
+
31
+ it 'should call .perform on job class with args' do
32
+ subject.args = ['a', 'b']
33
+ SimpleJob.should_receive(:perform).with('a', 'b')
34
+ subject.perform
35
+ end
36
+
37
+ it 'should call completed on backend' do
38
+ Qu.backend.should_receive(:completed)
39
+ subject.perform
40
+ end
41
+
42
+ context 'when being aborted' do
43
+ before do
44
+ SimpleJob.stub(:perform).and_raise(Qu::Worker::Abort)
45
+ end
46
+
47
+ it 'should release the job and re-raise the error' do
48
+ Qu.backend.should_receive(:release).with(subject)
49
+ lambda { subject.perform }.should raise_error(Qu::Worker::Abort)
50
+ end
51
+ end
52
+
53
+ context 'when the job raises an error' do
54
+ let(:error) { Exception.new("Some kind of error") }
55
+
56
+ before do
57
+ SimpleJob.stub!(:perform).and_raise(error)
58
+ end
59
+
60
+ it 'should call failed on backend' do
61
+ Qu.backend.should_receive(:failed).with(subject, error)
62
+ subject.perform
63
+ end
64
+
65
+ it 'should not call completed on backend' do
66
+ Qu.backend.should_not_receive(:completed)
67
+ subject.perform
68
+ end
69
+
70
+ it 'should call create on failure backend' do
71
+ Qu.failure = mock('a failure backend')
72
+ Qu.failure.should_receive(:create).with(subject, error)
73
+ subject.perform
74
+ end
75
+ end
76
+
77
+ end
78
+ end
@@ -0,0 +1,99 @@
1
+ require 'spec_helper'
2
+
3
+ describe Qu::Worker do
4
+ let(:job) { Qu::Job.new('1', SimpleJob, []) }
5
+
6
+ describe 'queues' do
7
+ it 'should use default if none specified' do
8
+ Qu::Worker.new.queues.should == ['default']
9
+ Qu::Worker.new('default').queues.should == ['default']
10
+ Qu::Worker.new(['default']).queues.should == ['default']
11
+ end
12
+ end
13
+
14
+ describe 'work' do
15
+ before do
16
+ Qu.stub!(:reserve).and_return(job)
17
+ end
18
+
19
+ it 'should reserve a job' do
20
+ Qu.should_receive(:reserve).with(subject).and_return(job)
21
+ subject.work
22
+ end
23
+
24
+ it 'should perform the job' do
25
+ job.should_receive(:perform)
26
+ subject.work
27
+ end
28
+ end
29
+
30
+ describe 'work_off' do
31
+ it 'should work all jobs off the queue' do
32
+ Qu.should_receive(:reserve).exactly(4).times.with(subject, :block => false).and_return(job, job, job, nil)
33
+ subject.work_off
34
+ end
35
+ end
36
+
37
+ describe 'start' do
38
+ before do
39
+ subject.stub(:loop)
40
+ end
41
+
42
+ it 'should register worker' do
43
+ Qu.backend.should_receive(:register_worker).with(subject)
44
+ subject.start
45
+ end
46
+
47
+ context 'when aborting' do
48
+ before do
49
+ subject.stub(:loop).and_raise(Qu::Worker::Abort)
50
+ end
51
+
52
+ it 'should unregister worker' do
53
+ Qu.backend.should_receive(:unregister_worker).with(subject)
54
+ subject.start
55
+ end
56
+ end
57
+ end
58
+
59
+ describe 'pid' do
60
+ it 'should equal process id' do
61
+ subject.pid.should == Process.pid
62
+ end
63
+
64
+ it 'should use provided pid' do
65
+ Qu::Worker.new(:pid => 1).pid.should == 1
66
+ end
67
+ end
68
+
69
+ describe 'id' do
70
+ it 'should return hostname, pid, and queues' do
71
+ worker = Qu::Worker.new('a', 'b', :hostname => 'quspec', :pid => 123)
72
+ worker.id.should == 'quspec:123:a,b'
73
+ end
74
+
75
+ it 'should not expand star in queue names' do
76
+ Qu::Worker.new('a', '*').id.should =~ /a,*/
77
+ end
78
+ end
79
+
80
+ describe 'hostname' do
81
+ it 'should get hostname' do
82
+ subject.hostname.should_not be_empty
83
+ end
84
+
85
+ it 'should use provided hostname' do
86
+ Qu::Worker.new(:hostname => 'quspec').hostname.should == 'quspec'
87
+ end
88
+ end
89
+
90
+ describe 'attributes' do
91
+ let(:attrs) do
92
+ {'hostname' => 'omgbbq', 'pid' => 987, 'queues' => ['a', '*']}
93
+ end
94
+
95
+ it 'should return hash of attributes' do
96
+ Qu::Worker.new(attrs).attributes.should == attrs
97
+ end
98
+ end
99
+ end
@@ -0,0 +1,25 @@
1
+ require 'spec_helper'
2
+
3
+ describe Qu do
4
+ %w(enqueue length queues reserve clear connection=).each do |method|
5
+ it "should delegate #{method} to backend" do
6
+ Qu.backend.should_receive(method).with(:arg)
7
+ Qu.send(method, :arg)
8
+ end
9
+ end
10
+
11
+ describe 'configure' do
12
+ it 'should yield Qu' do
13
+ Qu.configure do |c|
14
+ c.should == Qu
15
+ end
16
+ end
17
+ end
18
+
19
+ describe 'backend' do
20
+ it 'should raise error if backend not configured' do
21
+ Qu.backend = nil
22
+ lambda { Qu.backend }.should raise_error
23
+ end
24
+ end
25
+ end
@@ -0,0 +1,12 @@
1
+ require 'bundler'
2
+ Bundler.require :default, :test
3
+ require 'qu'
4
+ require 'qu/backend/spec'
5
+
6
+ RSpec.configure do |config|
7
+ config.before do
8
+ Qu.backend = mock('a backend', :reserve => nil, :failed => nil, :completed => nil,
9
+ :register_worker => nil, :unregister_worker => nil)
10
+ Qu.failure = nil
11
+ end
12
+ end
metadata ADDED
@@ -0,0 +1,102 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: qu
3
+ version: !ruby/object:Gem::Version
4
+ hash: 27
5
+ prerelease:
6
+ segments:
7
+ - 0
8
+ - 1
9
+ - 0
10
+ version: 0.1.0
11
+ platform: ruby
12
+ authors:
13
+ - Brandon Keepers
14
+ autorequire:
15
+ bindir: bin
16
+ cert_chain: []
17
+
18
+ date: 2011-09-23 00:00:00 -04:00
19
+ default_executable:
20
+ dependencies:
21
+ - !ruby/object:Gem::Dependency
22
+ name: multi_json
23
+ prerelease: false
24
+ requirement: &id001 !ruby/object:Gem::Requirement
25
+ none: false
26
+ requirements:
27
+ - - ">="
28
+ - !ruby/object:Gem::Version
29
+ hash: 3
30
+ segments:
31
+ - 0
32
+ version: "0"
33
+ type: :runtime
34
+ version_requirements: *id001
35
+ description: ""
36
+ email:
37
+ - brandon@opensoul.org
38
+ executables: []
39
+
40
+ extensions: []
41
+
42
+ extra_rdoc_files: []
43
+
44
+ files:
45
+ - .gitignore
46
+ - Gemfile
47
+ - Guardfile
48
+ - README.md
49
+ - Rakefile
50
+ - lib/qu.rb
51
+ - lib/qu/backend/base.rb
52
+ - lib/qu/backend/spec.rb
53
+ - lib/qu/failure.rb
54
+ - lib/qu/job.rb
55
+ - lib/qu/railtie.rb
56
+ - lib/qu/tasks.rb
57
+ - lib/qu/version.rb
58
+ - lib/qu/worker.rb
59
+ - qu.gemspec
60
+ - spec/qu/job_spec.rb
61
+ - spec/qu/worker_spec.rb
62
+ - spec/qu_spec.rb
63
+ - spec/spec_helper.rb
64
+ has_rdoc: true
65
+ homepage: http://github.com/bkeepers/qu
66
+ licenses: []
67
+
68
+ post_install_message:
69
+ rdoc_options: []
70
+
71
+ require_paths:
72
+ - lib
73
+ required_ruby_version: !ruby/object:Gem::Requirement
74
+ none: false
75
+ requirements:
76
+ - - ">="
77
+ - !ruby/object:Gem::Version
78
+ hash: 3
79
+ segments:
80
+ - 0
81
+ version: "0"
82
+ required_rubygems_version: !ruby/object:Gem::Requirement
83
+ none: false
84
+ requirements:
85
+ - - ">="
86
+ - !ruby/object:Gem::Version
87
+ hash: 3
88
+ segments:
89
+ - 0
90
+ version: "0"
91
+ requirements: []
92
+
93
+ rubyforge_project:
94
+ rubygems_version: 1.6.1
95
+ signing_key:
96
+ specification_version: 3
97
+ summary: ""
98
+ test_files:
99
+ - spec/qu/job_spec.rb
100
+ - spec/qu/worker_spec.rb
101
+ - spec/qu_spec.rb
102
+ - spec/spec_helper.rb