radar-delayed_job 1.7.0

Sign up to get free protection for your applications and to get access to all the features.
data/MIT-LICENSE ADDED
@@ -0,0 +1,20 @@
1
+ Copyright (c) 2005 Tobias Luetke
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining
4
+ a copy of this software and associated documentation files (the
5
+ "Software"), to deal in the Software without restriction, including
6
+ without limitation the rights to use, copy, modify, merge, publish,
7
+ distribute, sublicense, and/or sell copies of the Software, and to
8
+ permit persons to whom the Software is furnished to do so, subject to
9
+ the following conditions:
10
+
11
+ The above copyright notice and this permission notice shall be
12
+ included in all copies or substantial portions of the Software.
13
+
14
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
15
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
16
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOa AND
17
+ NONINFRINGEMENT. IN NO EVENT SaALL THE AUTHORS OR COPYRIGHT HOLDERS BE
18
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
19
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
20
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.textile ADDED
@@ -0,0 +1,110 @@
1
+ h1. Delayed::Job
2
+
3
+ Delated_job (or DJ) encapsulates the common pattern of asynchronously executing longer tasks in the background.
4
+
5
+ It is a direct extraction from Shopify where the job table is responsible for a multitude of core tasks. Amongst those tasks are:
6
+
7
+ * sending massive newsletters
8
+ * image resizing
9
+ * http downloads
10
+ * updating smart collections
11
+ * updating solr, our search server, after product changes
12
+ * batch imports
13
+ * spam checks
14
+
15
+ h2. Setup
16
+
17
+ The library evolves around a delayed_jobs table which looks as follows:
18
+
19
+ create_table :delayed_jobs, :force => true do |table|
20
+ table.integer :priority, :default => 0 # Allows some jobs to jump to the front of the queue
21
+ table.integer :attempts, :default => 0 # Provides for retries, but still fail eventually.
22
+ table.text :handler # YAML-encoded string of the object that will do work
23
+ table.string :last_error # reason for last failure (See Note below)
24
+ table.datetime :run_at # When to run. Could be Time.now for immediately, or sometime in the future.
25
+ table.datetime :locked_at # Set when a client is working on this object
26
+ table.datetime :failed_at # Set when all retries have failed (actually, by default, the record is deleted instead)
27
+ table.string :locked_by # Who is working on this object (if locked)
28
+ table.timestamps
29
+ end
30
+
31
+ On failure, the job is scheduled again in 5 seconds + N ** 4, where N is the number of retries.
32
+
33
+ The default MAX_ATTEMPTS is 25. After this, the job either deleted (default), or left in the database with "failed_at" set.
34
+ With the default of 25 attempts, the last retry will be 20 days later, with the last interval being almost 100 hours.
35
+
36
+ The default MAX_RUN_TIME is 4.hours. If your job takes longer than that, another computer could pick it up. It's up to you to
37
+ make sure your job doesn't exceed this time. You should set this to the longest time you think the job could take.
38
+
39
+ By default, it will delete failed jobs (and it always deletes successful jobs). If you want to keep failed jobs, set
40
+ Delayed::Job.destroy_failed_jobs = false. The failed jobs will be marked with non-null failed_at.
41
+
42
+ Here is an example of changing job parameters in Rails:
43
+
44
+ # config/initializers/delayed_job_config.rb
45
+ Delayed::Job.destroy_failed_jobs = false
46
+ silence_warnings do
47
+ Delayed::Job.const_set("MAX_ATTEMPTS", 3)
48
+ Delayed::Job.const_set("MAX_RUN_TIME", 5.minutes)
49
+ end
50
+
51
+ Note: If your error messages are long, consider changing last_error field to a :text instead of a :string (255 character limit).
52
+
53
+
54
+ h2. Usage
55
+
56
+ Jobs are simple ruby objects with a method called perform. Any object which responds to perform can be stuffed into the jobs table.
57
+ Job objects are serialized to yaml so that they can later be resurrected by the job runner.
58
+
59
+ class NewsletterJob < Struct.new(:text, :emails)
60
+ def perform
61
+ emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
62
+ end
63
+ end
64
+
65
+ Delayed::Job.enqueue NewsletterJob.new('lorem ipsum...', Customers.find(:all).collect(&:email))
66
+
67
+ There is also a second way to get jobs in the queue: send_later.
68
+
69
+
70
+ BatchImporter.new(Shop.find(1)).send_later(:import_massive_csv, massive_csv)
71
+
72
+
73
+ This will simply create a Delayed::PerformableMethod job in the jobs table which serializes all the parameters you pass to it. There are some special smarts for active record objects
74
+ which are stored as their text representation and loaded from the database fresh when the job is actually run later.
75
+
76
+
77
+ h2. Running the jobs
78
+
79
+ You can invoke @rake jobs:work@ which will start working off jobs. You can cancel the rake task with @CTRL-C@.
80
+
81
+ You can also run by writing a simple @script/job_runner@, and invoking it externally:
82
+
83
+ <pre><code>
84
+ #!/usr/bin/env ruby
85
+ require File.dirname(__FILE__) + '/../config/environment'
86
+
87
+ Delayed::Worker.new.start
88
+ </code></pre>
89
+
90
+ Workers can be running on any computer, as long as they have access to the database and their clock is in sync. You can even
91
+ run multiple workers on per computer, but you must give each one a unique name. (TODO: put in an example)
92
+ Keep in mind that each worker will check the database at least every 5 seconds.
93
+
94
+ Note: The rake task will exit if the database has any network connectivity problems.
95
+
96
+ h3. Cleaning up
97
+
98
+ You can invoke @rake jobs:clear@ to delete all jobs in the queue.
99
+
100
+ h3. Changes
101
+
102
+ * 1.7.0: Added failed_at column which can optionally be set after a certain amount of failed job attempts. By default failed job attempts are destroyed after about a month.
103
+
104
+ * 1.6.0: Renamed locked_until to locked_at. We now store when we start a given job instead of how long it will be locked by the worker. This allows us to get a reading on how long a job took to execute.
105
+
106
+ * 1.5.0: Job runners can now be run in parallel. Two new database columns are needed: locked_until and locked_by. This allows us to use pessimistic locking instead of relying on row level locks. This enables us to run as many worker processes as we need to speed up queue processing.
107
+
108
+ * 1.2.0: Added #send_later to Object for simpler job creation
109
+
110
+ * 1.0.0: Initial release
@@ -0,0 +1,41 @@
1
+ #version = File.read('README.textile').scan(/^\*\s+([\d\.]+)/).flatten
2
+
3
+ Gem::Specification.new do |s|
4
+ s.name = "radar-delayed_job"
5
+ s.version = "1.7.0"
6
+ s.date = "2008-11-28"
7
+ s.summary = "Database-backed asynchronous priority queue system -- Extracted from Shopify"
8
+ s.email = "tobi@leetsoft.com"
9
+ s.homepage = "http://github.com/tobi/delayed_job/tree/master"
10
+ s.description = "Delated_job (or DJ) encapsulates the common pattern of asynchronously executing longer tasks in the background. It is a direct extraction from Shopify where the job table is responsible for a multitude of core tasks."
11
+ s.authors = ["Tobias Lütke"]
12
+
13
+ # s.bindir = "bin"
14
+ # s.executables = ["delayed_job"]
15
+ # s.default_executable = "delayed_job"
16
+
17
+ s.has_rdoc = false
18
+ s.rdoc_options = ["--main", "README.textile"]
19
+ s.extra_rdoc_files = ["README.textile"]
20
+
21
+ # run git ls-files to get an updated list
22
+ s.files = %w[
23
+ MIT-LICENSE
24
+ README.textile
25
+ delayed_job.gemspec
26
+ init.rb
27
+ lib/delayed/job.rb
28
+ lib/delayed/message_sending.rb
29
+ lib/delayed/performable_method.rb
30
+ lib/delayed/worker.rb
31
+ lib/delayed_job.rb
32
+ tasks/jobs.rake
33
+ tasks/tasks.rb
34
+ ]
35
+ s.test_files = %w[
36
+ spec/database.rb
37
+ spec/delayed_method_spec.rb
38
+ spec/job_spec.rb
39
+ spec/story_spec.rb
40
+ ]
41
+ end
data/init.rb ADDED
@@ -0,0 +1 @@
1
+ require File.dirname(__FILE__) + '/lib/delayed_job'
@@ -0,0 +1,274 @@
1
+ module Delayed
2
+
3
+ class DeserializationError < StandardError
4
+ end
5
+
6
+ # A job object that is persisted to the database.
7
+ # Contains the work object as a YAML field.
8
+ class Job < ActiveRecord::Base
9
+ DEFAULT_JOB_COUNT_MAX = 100
10
+ MAX_ATTEMPTS = 25
11
+ MAX_RUN_TIME = 4.hours
12
+ set_table_name :delayed_jobs
13
+
14
+ # By default failed jobs are destroyed after too many attempts.
15
+ # If you want to keep them around (perhaps to inspect the reason
16
+ # for the failure), set this to false.
17
+ cattr_accessor :destroy_failed_jobs
18
+ self.destroy_failed_jobs = true
19
+
20
+ # Every worker has a unique name which by default is the pid of the process.
21
+ # There are some advantages to overriding this with something which survives worker retarts:
22
+ # Workers can safely resume working on tasks which are locked by themselves. The worker will assume that it crashed before.
23
+ cattr_accessor :worker_name
24
+ self.worker_name = "host:#{Socket.gethostname} pid:#{Process.pid}" rescue "pid:#{Process.pid}"
25
+
26
+ NextTaskSQL = '(run_at <= ? AND (locked_at IS NULL OR locked_at < ?) OR (locked_by = ?)) AND failed_at IS NULL'
27
+ NextTaskOrder = 'priority DESC, run_at ASC'
28
+
29
+ ParseObjectFromYaml = /\!ruby\/\w+\:([^\s]+)/
30
+
31
+ cattr_accessor :min_priority, :max_priority, :job_count_max
32
+ self.min_priority = nil
33
+ self.max_priority = nil
34
+ self.job_count_max = DEFAULT_JOB_COUNT_MAX
35
+
36
+ # When a worker is exiting, make sure we don't have any locked jobs.
37
+ def self.clear_locks!
38
+ update_all("locked_by = null, locked_at = null", ["locked_by = ?", worker_name])
39
+ end
40
+
41
+ def failed?
42
+ failed_at
43
+ end
44
+ alias_method :failed, :failed?
45
+
46
+ def payload_object
47
+ @payload_object ||= deserialize(self['handler'])
48
+ end
49
+
50
+ def name
51
+ @name ||= begin
52
+ payload = payload_object
53
+ if payload.respond_to?(:display_name)
54
+ payload.display_name
55
+ else
56
+ payload.class.name
57
+ end
58
+ end
59
+ end
60
+
61
+ def payload_object=(object)
62
+ self['handler'] = object.to_yaml
63
+ end
64
+
65
+ # Reschedule the job in the future (when a job fails).
66
+ # Uses an exponential scale depending on the number of failed attempts.
67
+ def reschedule(message, backtrace = [], time = nil)
68
+ if self.attempts < MAX_ATTEMPTS
69
+ time ||= Job.db_time_now + (attempts ** 4) + 5
70
+
71
+ self.attempts += 1
72
+ self.run_at = time
73
+ self.last_error = message + "\n" + backtrace.join("\n")
74
+ self.unlock
75
+ save!
76
+ else
77
+ logger.info "* [JOB] PERMANENTLY removing #{self.name} because of #{attempts} consequetive failures."
78
+ destroy_failed_jobs ? destroy : update_attribute(:failed_at, Time.now)
79
+ end
80
+ end
81
+
82
+
83
+ # Try to run one job. Returns true/false (work done/work failed) or nil if job can't be locked.
84
+ def run_with_lock(max_run_time, worker_name)
85
+ logger.info "* [JOB] aquiring lock on #{name}"
86
+ unless lock_exclusively!(max_run_time, worker_name)
87
+ # We did not get the lock, some other worker process must have
88
+ logger.warn "* [JOB] failed to aquire exclusive lock for #{name}"
89
+ return nil # no work done
90
+ end
91
+
92
+ begin
93
+ runtime = Benchmark.realtime do
94
+ invoke_job # TODO: raise error if takes longer than max_run_time
95
+ destroy
96
+ end
97
+ # TODO: warn if runtime > max_run_time ?
98
+ logger.info "* [JOB] #{name} completed after %.4f" % runtime
99
+ return true # did work
100
+ rescue Exception => e
101
+ reschedule e.message, e.backtrace
102
+ log_exception(e)
103
+ return false # work failed
104
+ end
105
+ end
106
+
107
+ # Add a job to the queue
108
+ def self.enqueue(*args, &block)
109
+ object = block_given? ? EvaledJob.new(&block) : args.shift
110
+
111
+ unless object.respond_to?(:perform) || block_given?
112
+ raise ArgumentError, 'Cannot enqueue items which do not respond to perform'
113
+ end
114
+
115
+ priority = args.first || 0
116
+ run_at = args[1]
117
+
118
+ Job.create(:payload_object => object, :priority => priority.to_i, :run_at => run_at)
119
+ end
120
+
121
+ # Find a few candidate jobs to run (in case some immediately get locked by others).
122
+ # Return in random order prevent everyone trying to do same head job at once.
123
+ def self.find_available(limit = 5, max_run_time = MAX_RUN_TIME)
124
+
125
+ time_now = db_time_now
126
+
127
+ sql = NextTaskSQL.dup
128
+
129
+ conditions = [time_now, time_now - max_run_time, worker_name]
130
+
131
+ if self.min_priority
132
+ sql << ' AND (priority >= ?)'
133
+ conditions << min_priority
134
+ end
135
+
136
+ if self.max_priority
137
+ sql << ' AND (priority <= ?)'
138
+ conditions << max_priority
139
+ end
140
+
141
+ conditions.unshift(sql)
142
+
143
+ records = ActiveRecord::Base.silence do
144
+ find(:all, :conditions => conditions, :order => NextTaskOrder, :limit => limit)
145
+ end
146
+
147
+ records.sort_by { rand() }
148
+ end
149
+
150
+ # Run the next job we can get an exclusive lock on.
151
+ # If no jobs are left we return nil
152
+ def self.reserve_and_run_one_job(max_run_time = MAX_RUN_TIME)
153
+
154
+ # We get up to 5 jobs from the db. In case we cannot get exclusive access to a job we try the next.
155
+ # this leads to a more even distribution of jobs across the worker processes
156
+ find_available(5, max_run_time).each do |job|
157
+ t = job.run_with_lock(max_run_time, worker_name)
158
+ return t unless t == nil # return if we did work (good or bad)
159
+ end
160
+
161
+ nil # we didn't do any work, all 5 were not lockable
162
+ end
163
+
164
+ # Lock this job for this worker.
165
+ # Returns true if we have the lock, false otherwise.
166
+ def lock_exclusively!(max_run_time, worker = worker_name)
167
+ now = self.class.db_time_now
168
+ affected_rows = if locked_by != worker
169
+ # We don't own this job so we will update the locked_by name and the locked_at
170
+ self.class.update_all(["locked_at = ?, locked_by = ?", now, worker], ["id = ? and (locked_at is null or locked_at < ?)", id, (now - max_run_time.to_i)])
171
+ else
172
+ # We already own this job, this may happen if the job queue crashes.
173
+ # Simply resume and update the locked_at
174
+ self.class.update_all(["locked_at = ?", now], ["id = ? and locked_by = ?", id, worker])
175
+ end
176
+ if affected_rows == 1
177
+ self.locked_at = now
178
+ self.locked_by = worker
179
+ return true
180
+ else
181
+ return false
182
+ end
183
+ end
184
+
185
+ # Unlock this job (note: not saved to DB)
186
+ def unlock
187
+ self.locked_at = nil
188
+ self.locked_by = nil
189
+ end
190
+
191
+ # This is a good hook if you need to report job processing errors in additional or different ways
192
+ def log_exception(error)
193
+ logger.error "* [JOB] #{name} failed with #{error.class.name}: #{error.message} - #{attempts} failed attempts"
194
+ logger.error(error)
195
+ end
196
+
197
+ # Do num jobs and return stats on success/failure.
198
+ # Exit early if interrupted.
199
+ def self.work_off(num = Delayed::Job.job_count_max)
200
+ success, failure = 0, 0
201
+
202
+ num.times do
203
+ case self.reserve_and_run_one_job
204
+ when true
205
+ success += 1
206
+ when false
207
+ failure += 1
208
+ else
209
+ break # leave if no work could be done
210
+ end
211
+ break if $exit # leave if we're exiting
212
+ end
213
+
214
+ return [success, failure]
215
+ end
216
+
217
+ # Moved into its own method so that new_relic can trace it.
218
+ def invoke_job
219
+ payload_object.perform
220
+ end
221
+
222
+ private
223
+
224
+ def deserialize(source)
225
+ handler = YAML.load(source) rescue nil
226
+
227
+ unless handler.respond_to?(:perform)
228
+ if handler.nil? && source =~ ParseObjectFromYaml
229
+ handler_class = $1
230
+ end
231
+ attempt_to_load(handler_class || handler.class)
232
+ handler = YAML.load(source)
233
+ end
234
+
235
+ return handler if handler.respond_to?(:perform)
236
+
237
+ raise DeserializationError,
238
+ 'Job failed to load: Unknown handler. Try to manually require the appropiate file.'
239
+ rescue TypeError, LoadError, NameError => e
240
+ raise DeserializationError,
241
+ "Job failed to load: #{e.message}. Try to manually require the required file."
242
+ end
243
+
244
+ # Constantize the object so that ActiveSupport can attempt
245
+ # its auto loading magic. Will raise LoadError if not successful.
246
+ def attempt_to_load(klass)
247
+ klass.constantize
248
+ end
249
+
250
+ # Get the current time (GMT or local depending on DB)
251
+ # Note: This does not ping the DB to get the time, so all your clients
252
+ # must have syncronized clocks.
253
+ def self.db_time_now
254
+ (ActiveRecord::Base.default_timezone == :utc) ? Time.now.utc : Time.now
255
+ end
256
+
257
+ protected
258
+
259
+ def before_save
260
+ self.run_at ||= self.class.db_time_now
261
+ end
262
+
263
+ end
264
+
265
+ class EvaledJob
266
+ def initialize
267
+ @job = yield
268
+ end
269
+
270
+ def perform
271
+ eval(@job)
272
+ end
273
+ end
274
+ end
@@ -0,0 +1,17 @@
1
+ module Delayed
2
+ module MessageSending
3
+ def send_later(method, *args)
4
+ Delayed::Job.enqueue Delayed::PerformableMethod.new(self, method.to_sym, args)
5
+ end
6
+
7
+ module ClassMethods
8
+ def handle_asynchronously(method)
9
+ without_name = "#{method}_without_send_later"
10
+ define_method("#{method}_with_send_later") do |*args|
11
+ send_later(without_name, *args)
12
+ end
13
+ alias_method_chain method, :send_later
14
+ end
15
+ end
16
+ end
17
+ end
@@ -0,0 +1,55 @@
1
+ module Delayed
2
+ class PerformableMethod < Struct.new(:object, :method, :args)
3
+ CLASS_STRING_FORMAT = /^CLASS\:([A-Z][\w\:]+)$/
4
+ AR_STRING_FORMAT = /^AR\:([A-Z][\w\:]+)\:(\d+)$/
5
+
6
+ def initialize(object, method, args)
7
+ raise NoMethodError, "undefined method `#{method}' for #{self.inspect}" unless object.respond_to?(method)
8
+
9
+ self.object = dump(object)
10
+ self.args = args.map { |a| dump(a) }
11
+ self.method = method.to_sym
12
+ end
13
+
14
+ def display_name
15
+ case self.object
16
+ when CLASS_STRING_FORMAT then "#{$1}.#{method}"
17
+ when AR_STRING_FORMAT then "#{$1}##{method}"
18
+ else "Unknown##{method}"
19
+ end
20
+ end
21
+
22
+ def perform
23
+ load(object).send(method, *args.map{|a| load(a)})
24
+ rescue ActiveRecord::RecordNotFound
25
+ # We cannot do anything about objects which were deleted in the meantime
26
+ true
27
+ end
28
+
29
+ private
30
+
31
+ def load(arg)
32
+ case arg
33
+ when CLASS_STRING_FORMAT then $1.constantize
34
+ when AR_STRING_FORMAT then $1.constantize.find($2)
35
+ else arg
36
+ end
37
+ end
38
+
39
+ def dump(arg)
40
+ case arg
41
+ when Class then class_to_string(arg)
42
+ when ActiveRecord::Base then ar_to_string(arg)
43
+ else arg
44
+ end
45
+ end
46
+
47
+ def ar_to_string(obj)
48
+ "AR:#{obj.class}:#{obj.id}"
49
+ end
50
+
51
+ def class_to_string(obj)
52
+ "CLASS:#{obj.name}"
53
+ end
54
+ end
55
+ end
@@ -0,0 +1,55 @@
1
+ module Delayed
2
+ class Worker
3
+ SLEEP = 5
4
+
5
+ cattr_accessor :logger
6
+ self.logger = if defined?(Merb::Logger)
7
+ Merb.logger
8
+ elsif defined?(RAILS_DEFAULT_LOGGER)
9
+ RAILS_DEFAULT_LOGGER
10
+ end
11
+
12
+ def initialize(options = {})
13
+ @quiet = options[:quiet]
14
+ Delayed::Job.min_priority = options[:min_priority] if options.has_key?(:min_priority)
15
+ Delayed::Job.max_priority = options[:max_priority] if options.has_key?(:max_priority)
16
+ Delayed::Job.job_count_max = options[:job_count_max] if options.has_key?(:job_count_max)
17
+ end
18
+
19
+ def start
20
+ say "*** Starting job worker #{Delayed::Job.worker_name}"
21
+
22
+ trap('TERM') { say 'Exiting...'; $exit = true }
23
+ trap('INT') { say 'Exiting...'; $exit = true }
24
+
25
+ loop do
26
+ result = nil
27
+
28
+ realtime = Benchmark.realtime do
29
+ result = Delayed::Job.work_off
30
+ end
31
+
32
+ count = result.sum
33
+
34
+ break if $exit
35
+
36
+ if count.zero?
37
+ sleep(SLEEP)
38
+ else
39
+ say "#{count} jobs processed at %.4f j/s, %d failed ..." % [count / realtime, result.last]
40
+ end
41
+
42
+ break if $exit
43
+ end
44
+
45
+ ensure
46
+ Delayed::Job.clear_locks!
47
+ end
48
+
49
+ def say(text)
50
+ puts text unless @quiet
51
+ logger.info text if logger
52
+ end
53
+
54
+ end
55
+ end
@@ -0,0 +1,13 @@
1
+ autoload :ActiveRecord, 'activerecord'
2
+
3
+ require File.dirname(__FILE__) + '/delayed/message_sending'
4
+ require File.dirname(__FILE__) + '/delayed/performable_method'
5
+ require File.dirname(__FILE__) + '/delayed/job'
6
+ require File.dirname(__FILE__) + '/delayed/worker'
7
+
8
+ Object.send(:include, Delayed::MessageSending)
9
+ Module.send(:include, Delayed::MessageSending::ClassMethods)
10
+
11
+ if defined?(Merb::Plugins)
12
+ Merb::Plugins.add_rakefiles File.dirname(__FILE__) / '..' / 'tasks' / 'tasks'
13
+ end
data/spec/database.rb ADDED
@@ -0,0 +1,42 @@
1
+ $:.unshift(File.dirname(__FILE__) + '/../lib')
2
+ $:.unshift(File.dirname(__FILE__) + '/../../rspec/lib')
3
+
4
+ require 'rubygems'
5
+ require 'active_record'
6
+ gem 'sqlite3-ruby'
7
+
8
+ require File.dirname(__FILE__) + '/../init'
9
+ require 'spec'
10
+
11
+ ActiveRecord::Base.logger = Logger.new('/tmp/dj.log')
12
+ ActiveRecord::Base.establish_connection(:adapter => 'sqlite3', :database => '/tmp/jobs.sqlite')
13
+ ActiveRecord::Migration.verbose = false
14
+
15
+ ActiveRecord::Schema.define do
16
+
17
+ create_table :delayed_jobs, :force => true do |table|
18
+ table.integer :priority, :default => 0
19
+ table.integer :attempts, :default => 0
20
+ table.text :handler
21
+ table.string :last_error
22
+ table.datetime :run_at
23
+ table.datetime :locked_at
24
+ table.string :locked_by
25
+ table.datetime :failed_at
26
+ table.timestamps
27
+ end
28
+
29
+ create_table :stories, :force => true do |table|
30
+ table.string :text
31
+ end
32
+
33
+ end
34
+
35
+
36
+ # Purely useful for test cases...
37
+ class Story < ActiveRecord::Base
38
+ def tell; text; end
39
+ def whatever(n, _); tell*n; end
40
+
41
+ handle_asynchronously :whatever
42
+ end
@@ -0,0 +1,128 @@
1
+ require File.dirname(__FILE__) + '/database'
2
+
3
+ class SimpleJob
4
+ cattr_accessor :runs; self.runs = 0
5
+ def perform; @@runs += 1; end
6
+ end
7
+
8
+ class RandomRubyObject
9
+ def say_hello
10
+ 'hello'
11
+ end
12
+ end
13
+
14
+ class ErrorObject
15
+
16
+ def throw
17
+ raise ActiveRecord::RecordNotFound, '...'
18
+ false
19
+ end
20
+
21
+ end
22
+
23
+ class StoryReader
24
+
25
+ def read(story)
26
+ "Epilog: #{story.tell}"
27
+ end
28
+
29
+ end
30
+
31
+ class StoryReader
32
+
33
+ def read(story)
34
+ "Epilog: #{story.tell}"
35
+ end
36
+
37
+ end
38
+
39
+ describe 'random ruby objects' do
40
+ before { Delayed::Job.delete_all }
41
+
42
+ it "should respond_to :send_later method" do
43
+
44
+ RandomRubyObject.new.respond_to?(:send_later)
45
+
46
+ end
47
+
48
+ it "should raise a ArgumentError if send_later is called but the target method doesn't exist" do
49
+ lambda { RandomRubyObject.new.send_later(:method_that_deos_not_exist) }.should raise_error(NoMethodError)
50
+ end
51
+
52
+ it "should add a new entry to the job table when send_later is called on it" do
53
+ Delayed::Job.count.should == 0
54
+
55
+ RandomRubyObject.new.send_later(:to_s)
56
+
57
+ Delayed::Job.count.should == 1
58
+ end
59
+
60
+ it "should add a new entry to the job table when send_later is called on the class" do
61
+ Delayed::Job.count.should == 0
62
+
63
+ RandomRubyObject.send_later(:to_s)
64
+
65
+ Delayed::Job.count.should == 1
66
+ end
67
+
68
+ it "should run get the original method executed when the job is performed" do
69
+
70
+ RandomRubyObject.new.send_later(:say_hello)
71
+
72
+ Delayed::Job.count.should == 1
73
+ end
74
+
75
+ it "should ignore ActiveRecord::RecordNotFound errors because they are permanent" do
76
+
77
+ ErrorObject.new.send_later(:throw)
78
+
79
+ Delayed::Job.count.should == 1
80
+
81
+ Delayed::Job.reserve_and_run_one_job
82
+
83
+ Delayed::Job.count.should == 0
84
+
85
+ end
86
+
87
+ it "should store the object as string if its an active record" do
88
+ story = Story.create :text => 'Once upon...'
89
+ story.send_later(:tell)
90
+
91
+ job = Delayed::Job.find(:first)
92
+ job.payload_object.class.should == Delayed::PerformableMethod
93
+ job.payload_object.object.should == "AR:Story:#{story.id}"
94
+ job.payload_object.method.should == :tell
95
+ job.payload_object.args.should == []
96
+ job.payload_object.perform.should == 'Once upon...'
97
+ end
98
+
99
+ it "should store arguments as string if they an active record" do
100
+
101
+ story = Story.create :text => 'Once upon...'
102
+
103
+ reader = StoryReader.new
104
+ reader.send_later(:read, story)
105
+
106
+ job = Delayed::Job.find(:first)
107
+ job.payload_object.class.should == Delayed::PerformableMethod
108
+ job.payload_object.method.should == :read
109
+ job.payload_object.args.should == ["AR:Story:#{story.id}"]
110
+ job.payload_object.perform.should == 'Epilog: Once upon...'
111
+ end
112
+
113
+ it "should call send later on methods which are wrapped with handle_asynchronously" do
114
+ story = Story.create :text => 'Once upon...'
115
+
116
+ Delayed::Job.count.should == 0
117
+
118
+ story.whatever(1, 5)
119
+
120
+ Delayed::Job.count.should == 1
121
+ job = Delayed::Job.find(:first)
122
+ job.payload_object.class.should == Delayed::PerformableMethod
123
+ job.payload_object.method.should == :whatever_without_send_later
124
+ job.payload_object.args.should == [1, 5]
125
+ job.payload_object.perform.should == 'Once upon...'
126
+ end
127
+
128
+ end
data/spec/job_spec.rb ADDED
@@ -0,0 +1,352 @@
1
+ require File.dirname(__FILE__) + '/database'
2
+
3
+ class SimpleJob
4
+ cattr_accessor :runs; self.runs = 0
5
+ def perform; @@runs += 1; end
6
+ end
7
+
8
+ class ErrorJob
9
+ cattr_accessor :runs; self.runs = 0
10
+ def perform; raise 'did not work'; end
11
+ end
12
+
13
+ module M
14
+ class ModuleJob
15
+ cattr_accessor :runs; self.runs = 0
16
+ def perform; @@runs += 1; end
17
+ end
18
+
19
+ end
20
+
21
+ describe Delayed::Job do
22
+ before do
23
+ Delayed::Job.max_priority = nil
24
+ Delayed::Job.min_priority = nil
25
+
26
+ Delayed::Job.delete_all
27
+ end
28
+
29
+ before(:each) do
30
+ SimpleJob.runs = 0
31
+ end
32
+
33
+ it "should set run_at automatically if not set" do
34
+ Delayed::Job.create(:payload_object => ErrorJob.new ).run_at.should_not == nil
35
+ end
36
+
37
+ it "should not set run_at automatically if already set" do
38
+ later = 5.minutes.from_now
39
+ Delayed::Job.create(:payload_object => ErrorJob.new, :run_at => later).run_at.should == later
40
+ end
41
+
42
+ it "should raise ArgumentError when handler doesn't respond_to :perform" do
43
+ lambda { Delayed::Job.enqueue(Object.new) }.should raise_error(ArgumentError)
44
+ end
45
+
46
+ it "should increase count after enqueuing items" do
47
+ Delayed::Job.enqueue SimpleJob.new
48
+ Delayed::Job.count.should == 1
49
+ end
50
+
51
+ it "should be able to set priority when enqueuing items" do
52
+ Delayed::Job.enqueue SimpleJob.new, 5
53
+ Delayed::Job.first.priority.should == 5
54
+ end
55
+
56
+ it "should be able to set run_at when enqueuing items" do
57
+ later = 5.minutes.from_now
58
+ Delayed::Job.enqueue SimpleJob.new, 5, later
59
+
60
+ # use be close rather than equal to because millisecond values cn be lost in DB round trip
61
+ Delayed::Job.first.run_at.should be_close(later, 1)
62
+ end
63
+
64
+ it "should call perform on jobs when running work_off" do
65
+ SimpleJob.runs.should == 0
66
+
67
+ Delayed::Job.enqueue SimpleJob.new
68
+ Delayed::Job.work_off
69
+
70
+ SimpleJob.runs.should == 1
71
+ end
72
+
73
+ it "should call perform on jobs when running work_off" do
74
+ SimpleJob.runs.should == 0
75
+
76
+ 101.times { Delayed::Job.enqueue SimpleJob.new }
77
+ Delayed::Job.work_off.should eql([100, 0])
78
+ SimpleJob.runs.should == 100
79
+ end
80
+
81
+ it "should work with eval jobs" do
82
+ $eval_job_ran = false
83
+
84
+ Delayed::Job.enqueue do <<-JOB
85
+ $eval_job_ran = true
86
+ JOB
87
+ end
88
+
89
+ Delayed::Job.work_off
90
+
91
+ $eval_job_ran.should == true
92
+ end
93
+
94
+ it "should work with jobs in modules" do
95
+ M::ModuleJob.runs.should == 0
96
+
97
+ Delayed::Job.enqueue M::ModuleJob.new
98
+ Delayed::Job.work_off
99
+
100
+ M::ModuleJob.runs.should == 1
101
+ end
102
+
103
+ it "should re-schedule by about 1 second at first and increment this more and more minutes when it fails to execute properly" do
104
+ Delayed::Job.enqueue ErrorJob.new
105
+ Delayed::Job.work_off(1)
106
+
107
+ job = Delayed::Job.find(:first)
108
+
109
+ job.last_error.should =~ /did not work/
110
+ job.last_error.should =~ /job_spec.rb:10:in `perform'/
111
+ job.attempts.should == 1
112
+
113
+ job.run_at.should > Delayed::Job.db_time_now - 10.minutes
114
+ job.run_at.should < Delayed::Job.db_time_now + 10.minutes
115
+ end
116
+
117
+ it "should raise an DeserializationError when the job class is totally unknown" do
118
+
119
+ job = Delayed::Job.new
120
+ job['handler'] = "--- !ruby/object:JobThatDoesNotExist {}"
121
+
122
+ lambda { job.payload_object.perform }.should raise_error(Delayed::DeserializationError)
123
+ end
124
+
125
+ it "should try to load the class when it is unknown at the time of the deserialization" do
126
+ job = Delayed::Job.new
127
+ job['handler'] = "--- !ruby/object:JobThatDoesNotExist {}"
128
+
129
+ job.should_receive(:attempt_to_load).with('JobThatDoesNotExist').and_return(true)
130
+
131
+ lambda { job.payload_object.perform }.should raise_error(Delayed::DeserializationError)
132
+ end
133
+
134
+ it "should try include the namespace when loading unknown objects" do
135
+ job = Delayed::Job.new
136
+ job['handler'] = "--- !ruby/object:Delayed::JobThatDoesNotExist {}"
137
+ job.should_receive(:attempt_to_load).with('Delayed::JobThatDoesNotExist').and_return(true)
138
+ lambda { job.payload_object.perform }.should raise_error(Delayed::DeserializationError)
139
+ end
140
+
141
+ it "should also try to load structs when they are unknown (raises TypeError)" do
142
+ job = Delayed::Job.new
143
+ job['handler'] = "--- !ruby/struct:JobThatDoesNotExist {}"
144
+
145
+ job.should_receive(:attempt_to_load).with('JobThatDoesNotExist').and_return(true)
146
+
147
+ lambda { job.payload_object.perform }.should raise_error(Delayed::DeserializationError)
148
+ end
149
+
150
+ it "should try include the namespace when loading unknown structs" do
151
+ job = Delayed::Job.new
152
+ job['handler'] = "--- !ruby/struct:Delayed::JobThatDoesNotExist {}"
153
+
154
+ job.should_receive(:attempt_to_load).with('Delayed::JobThatDoesNotExist').and_return(true)
155
+ lambda { job.payload_object.perform }.should raise_error(Delayed::DeserializationError)
156
+ end
157
+
158
+ it "should be failed if it failed more than MAX_ATTEMPTS times and we don't want to destroy jobs" do
159
+ default = Delayed::Job.destroy_failed_jobs
160
+ Delayed::Job.destroy_failed_jobs = false
161
+
162
+ @job = Delayed::Job.create :payload_object => SimpleJob.new, :attempts => 50
163
+ @job.reload.failed_at.should == nil
164
+ @job.reschedule 'FAIL'
165
+ @job.reload.failed_at.should_not == nil
166
+
167
+ Delayed::Job.destroy_failed_jobs = default
168
+ end
169
+
170
+ it "should be destroyed if it failed more than MAX_ATTEMPTS times and we want to destroy jobs" do
171
+ default = Delayed::Job.destroy_failed_jobs
172
+ Delayed::Job.destroy_failed_jobs = true
173
+
174
+ @job = Delayed::Job.create :payload_object => SimpleJob.new, :attempts => 50
175
+ @job.should_receive(:destroy)
176
+ @job.reschedule 'FAIL'
177
+
178
+ Delayed::Job.destroy_failed_jobs = default
179
+ end
180
+
181
+ it "should never find failed jobs" do
182
+ @job = Delayed::Job.create :payload_object => SimpleJob.new, :attempts => 50, :failed_at => Time.now
183
+ Delayed::Job.find_available(1).length.should == 0
184
+ end
185
+
186
+ context "when another worker is already performing an task, it" do
187
+
188
+ before :each do
189
+ Delayed::Job.worker_name = 'worker1'
190
+ @job = Delayed::Job.create :payload_object => SimpleJob.new, :locked_by => 'worker1', :locked_at => Delayed::Job.db_time_now - 5.minutes
191
+ end
192
+
193
+ it "should not allow a second worker to get exclusive access" do
194
+ @job.lock_exclusively!(4.hours, 'worker2').should == false
195
+ end
196
+
197
+ it "should allow a second worker to get exclusive access if the timeout has passed" do
198
+ @job.lock_exclusively!(1.minute, 'worker2').should == true
199
+ end
200
+
201
+ it "should be able to get access to the task if it was started more then max_age ago" do
202
+ @job.locked_at = 5.hours.ago
203
+ @job.save
204
+
205
+ @job.lock_exclusively! 4.hours, 'worker2'
206
+ @job.reload
207
+ @job.locked_by.should == 'worker2'
208
+ @job.locked_at.should > 1.minute.ago
209
+ end
210
+
211
+ it "should not be found by another worker" do
212
+ Delayed::Job.worker_name = 'worker2'
213
+
214
+ Delayed::Job.find_available(1, 6.minutes).length.should == 0
215
+ end
216
+
217
+ it "should be found by another worker if the time has expired" do
218
+ Delayed::Job.worker_name = 'worker2'
219
+
220
+ Delayed::Job.find_available(1, 4.minutes).length.should == 1
221
+ end
222
+
223
+ it "should be able to get exclusive access again when the worker name is the same" do
224
+ @job.lock_exclusively! 5.minutes, 'worker1'
225
+ @job.lock_exclusively! 5.minutes, 'worker1'
226
+ @job.lock_exclusively! 5.minutes, 'worker1'
227
+ end
228
+ end
229
+
230
+ context "#name" do
231
+ it "should be the class name of the job that was enqueued" do
232
+ Delayed::Job.create(:payload_object => ErrorJob.new ).name.should == 'ErrorJob'
233
+ end
234
+
235
+ it "should be the method that will be called if its a performable method object" do
236
+ Delayed::Job.send_later(:clear_locks!)
237
+ Delayed::Job.last.name.should == 'Delayed::Job.clear_locks!'
238
+
239
+ end
240
+ it "should be the instance method that will be called if its a performable method object" do
241
+ story = Story.create :text => "..."
242
+
243
+ story.send_later(:save)
244
+
245
+ Delayed::Job.last.name.should == 'Story#save'
246
+ end
247
+ end
248
+
249
+ context "worker prioritization" do
250
+
251
+ before(:each) do
252
+ Delayed::Job.max_priority = nil
253
+ Delayed::Job.min_priority = nil
254
+ end
255
+
256
+ it "should only work_off jobs that are >= min_priority" do
257
+ Delayed::Job.min_priority = -5
258
+ Delayed::Job.max_priority = 5
259
+ SimpleJob.runs.should == 0
260
+
261
+ Delayed::Job.enqueue SimpleJob.new, -10
262
+ Delayed::Job.enqueue SimpleJob.new, 0
263
+ Delayed::Job.work_off
264
+
265
+ SimpleJob.runs.should == 1
266
+ end
267
+
268
+ it "should only work_off jobs that are <= max_priority" do
269
+ Delayed::Job.min_priority = -5
270
+ Delayed::Job.max_priority = 5
271
+ SimpleJob.runs.should == 0
272
+
273
+ Delayed::Job.enqueue SimpleJob.new, 10
274
+ Delayed::Job.enqueue SimpleJob.new, 0
275
+
276
+ Delayed::Job.work_off
277
+
278
+ SimpleJob.runs.should == 1
279
+ end
280
+
281
+ end
282
+
283
+ context "when pulling jobs off the queue for processing, it" do
284
+ before(:each) do
285
+ @job = Delayed::Job.create(
286
+ :payload_object => SimpleJob.new,
287
+ :locked_by => 'worker1',
288
+ :locked_at => Delayed::Job.db_time_now - 5.minutes)
289
+ end
290
+
291
+ it "should leave the queue in a consistent state and not run the job if locking fails" do
292
+ SimpleJob.runs.should == 0
293
+ @job.stub!(:lock_exclusively!).with(any_args).once.and_return(false)
294
+ Delayed::Job.should_receive(:find_available).once.and_return([@job])
295
+ Delayed::Job.work_off(1)
296
+ SimpleJob.runs.should == 0
297
+ end
298
+
299
+ end
300
+
301
+ context "while running alongside other workers that locked jobs, it" do
302
+ before(:each) do
303
+ Delayed::Job.worker_name = 'worker1'
304
+ Delayed::Job.create(:payload_object => SimpleJob.new, :locked_by => 'worker1', :locked_at => (Delayed::Job.db_time_now - 1.minutes))
305
+ Delayed::Job.create(:payload_object => SimpleJob.new, :locked_by => 'worker2', :locked_at => (Delayed::Job.db_time_now - 1.minutes))
306
+ Delayed::Job.create(:payload_object => SimpleJob.new)
307
+ Delayed::Job.create(:payload_object => SimpleJob.new, :locked_by => 'worker1', :locked_at => (Delayed::Job.db_time_now - 1.minutes))
308
+ end
309
+
310
+ it "should ingore locked jobs from other workers" do
311
+ Delayed::Job.worker_name = 'worker3'
312
+ SimpleJob.runs.should == 0
313
+ Delayed::Job.work_off
314
+ SimpleJob.runs.should == 1 # runs the one open job
315
+ end
316
+
317
+ it "should find our own jobs regardless of locks" do
318
+ Delayed::Job.worker_name = 'worker1'
319
+ SimpleJob.runs.should == 0
320
+ Delayed::Job.work_off
321
+ SimpleJob.runs.should == 3 # runs open job plus worker1 jobs that were already locked
322
+ end
323
+ end
324
+
325
+ context "while running with locked and expired jobs, it" do
326
+ before(:each) do
327
+ Delayed::Job.worker_name = 'worker1'
328
+ exp_time = Delayed::Job.db_time_now - (1.minutes + Delayed::Job::MAX_RUN_TIME)
329
+ Delayed::Job.create(:payload_object => SimpleJob.new, :locked_by => 'worker1', :locked_at => exp_time)
330
+ Delayed::Job.create(:payload_object => SimpleJob.new, :locked_by => 'worker2', :locked_at => (Delayed::Job.db_time_now - 1.minutes))
331
+ Delayed::Job.create(:payload_object => SimpleJob.new)
332
+ Delayed::Job.create(:payload_object => SimpleJob.new, :locked_by => 'worker1', :locked_at => (Delayed::Job.db_time_now - 1.minutes))
333
+ end
334
+
335
+ it "should only find unlocked and expired jobs" do
336
+ Delayed::Job.worker_name = 'worker3'
337
+ SimpleJob.runs.should == 0
338
+ Delayed::Job.work_off
339
+ SimpleJob.runs.should == 2 # runs the one open job and one expired job
340
+ end
341
+
342
+ it "should ignore locks when finding our own jobs" do
343
+ Delayed::Job.worker_name = 'worker1'
344
+ SimpleJob.runs.should == 0
345
+ Delayed::Job.work_off
346
+ SimpleJob.runs.should == 3 # runs open job plus worker1 jobs
347
+ # This is useful in the case of a crash/restart on worker1, but make sure multiple workers on the same host have unique names!
348
+ end
349
+
350
+ end
351
+
352
+ end
@@ -0,0 +1,17 @@
1
+ require File.dirname(__FILE__) + '/database'
2
+
3
+ describe "A story" do
4
+
5
+ before(:all) do
6
+ @story = Story.create :text => "Once upon a time..."
7
+ end
8
+
9
+ it "should be shared" do
10
+ @story.tell.should == 'Once upon a time...'
11
+ end
12
+
13
+ it "should not return its result if it storytelling is delayed" do
14
+ @story.send_later(:tell).should_not == 'Once upon a time...'
15
+ end
16
+
17
+ end
data/tasks/jobs.rake ADDED
@@ -0,0 +1 @@
1
+ require File.join(File.dirname(__FILE__), 'tasks')
data/tasks/tasks.rb ADDED
@@ -0,0 +1,17 @@
1
+ # Re-definitions are appended to existing tasks
2
+ task :environment
3
+ task :merb_env
4
+
5
+ namespace :jobs do
6
+ desc "Clear the delayed_job queue."
7
+ task :clear => [:merb_env, :environment] do
8
+ Delayed::Job.delete_all
9
+ end
10
+
11
+ desc "Start a delayed_job worker."
12
+ task :work => [:merb_env, :environment] do
13
+ Delayed::Worker.new(:min_priority => ENV['MIN_PRIORITY'],
14
+ :max_priority => ENV['MAX_PRIORITY'],
15
+ :job_max_count => ENV['JOB_MAX_COUNT']).start
16
+ end
17
+ end
metadata ADDED
@@ -0,0 +1,69 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: radar-delayed_job
3
+ version: !ruby/object:Gem::Version
4
+ version: 1.7.0
5
+ platform: ruby
6
+ authors:
7
+ - "Tobias L\xC3\xBCtke"
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+
12
+ date: 2008-11-28 00:00:00 +10:00
13
+ default_executable:
14
+ dependencies: []
15
+
16
+ description: Delated_job (or DJ) encapsulates the common pattern of asynchronously executing longer tasks in the background. It is a direct extraction from Shopify where the job table is responsible for a multitude of core tasks.
17
+ email: tobi@leetsoft.com
18
+ executables: []
19
+
20
+ extensions: []
21
+
22
+ extra_rdoc_files:
23
+ - README.textile
24
+ files:
25
+ - MIT-LICENSE
26
+ - README.textile
27
+ - delayed_job.gemspec
28
+ - init.rb
29
+ - lib/delayed/job.rb
30
+ - lib/delayed/message_sending.rb
31
+ - lib/delayed/performable_method.rb
32
+ - lib/delayed/worker.rb
33
+ - lib/delayed_job.rb
34
+ - tasks/jobs.rake
35
+ - tasks/tasks.rb
36
+ has_rdoc: true
37
+ homepage: http://github.com/tobi/delayed_job/tree/master
38
+ licenses: []
39
+
40
+ post_install_message:
41
+ rdoc_options:
42
+ - --main
43
+ - README.textile
44
+ require_paths:
45
+ - lib
46
+ required_ruby_version: !ruby/object:Gem::Requirement
47
+ requirements:
48
+ - - ">="
49
+ - !ruby/object:Gem::Version
50
+ version: "0"
51
+ version:
52
+ required_rubygems_version: !ruby/object:Gem::Requirement
53
+ requirements:
54
+ - - ">="
55
+ - !ruby/object:Gem::Version
56
+ version: "0"
57
+ version:
58
+ requirements: []
59
+
60
+ rubyforge_project:
61
+ rubygems_version: 1.3.5
62
+ signing_key:
63
+ specification_version: 3
64
+ summary: Database-backed asynchronous priority queue system -- Extracted from Shopify
65
+ test_files:
66
+ - spec/database.rb
67
+ - spec/delayed_method_spec.rb
68
+ - spec/job_spec.rb
69
+ - spec/story_spec.rb