blaxter-delayed_job 1.7.99 → 2.0.0
Sign up to get free protection for your applications and to get access to all the features.
- data/README.markdown +199 -0
- data/VERSION +1 -0
- data/delayed_job.gemspec +4 -6
- data/generators/delayed_job/templates/migration.rb +1 -0
- data/lib/delayed/job.rb +113 -105
- data/lib/delayed/message_sending.rb +3 -3
- data/lib/delayed/performable_method.rb +14 -5
- data/lib/delayed/worker.rb +56 -15
- data/spec/database.rb +1 -0
- data/spec/delayed_method_spec.rb +11 -7
- data/spec/job_spec.rb +103 -88
- metadata +6 -5
- data/README.textile +0 -118
data/README.markdown
ADDED
@@ -0,0 +1,199 @@
|
|
1
|
+
Delayed::Job
|
2
|
+
============
|
3
|
+
|
4
|
+
Delayed_job (or DJ) encapsulates the common pattern of asynchronously executing longer tasks in the background.
|
5
|
+
|
6
|
+
It is a direct extraction from Shopify where the job table is responsible for a multitude of core tasks. Amongst those tasks are:
|
7
|
+
|
8
|
+
* sending massive newsletters
|
9
|
+
* image resizing
|
10
|
+
* http downloads
|
11
|
+
* updating smart collections
|
12
|
+
* updating solr, our search server, after product changes
|
13
|
+
* batch imports
|
14
|
+
* spam checks
|
15
|
+
|
16
|
+
What is this fork for?
|
17
|
+
----------------------
|
18
|
+
|
19
|
+
My purpose with this fork is make delayed_job for flexible. That's means you can customize how your workers behave.
|
20
|
+
|
21
|
+
The common use will be to have several workers running concurrently (in one process) and each one with differents constraints so they'll run different kind of jobs.
|
22
|
+
|
23
|
+
|
24
|
+
Setup
|
25
|
+
-----
|
26
|
+
|
27
|
+
The library evolves around a delayed_jobs table which looks as follows:
|
28
|
+
|
29
|
+
create_table :delayed_jobs, :force => true do |table|
|
30
|
+
table.integer :priority, :default => 0 # Allows some jobs to jump to the front of the queue
|
31
|
+
table.integer :attempts, :default => 0 # Provides for retries, but still fail eventually.
|
32
|
+
table.text :handler # YAML-encoded string of the object that will do work
|
33
|
+
table.string :job_type # Class name of the job object, for type-specific workers
|
34
|
+
table.string :name # The display name, an informative field or can be use to filter jobs
|
35
|
+
table.string :last_error # reason for last failure (See Note below)
|
36
|
+
table.datetime :run_at # When to run. Could be Time.now for immediately, or sometime in the future.
|
37
|
+
table.datetime :locked_at # Set when a client is working on this object
|
38
|
+
table.datetime :failed_at # Set when all retries have failed (actually, by default, the record is deleted instead)
|
39
|
+
table.string :locked_by # Who is working on this object (if locked)
|
40
|
+
table.datetime :finished_at # Used for statiscics / monitoring
|
41
|
+
table.timestamps
|
42
|
+
end
|
43
|
+
|
44
|
+
You can generate the migration executing:
|
45
|
+
|
46
|
+
$ script/generate delayed_job
|
47
|
+
exists db/migrate
|
48
|
+
create db/migrate/20090807090217_create_delayed_jobs.rb
|
49
|
+
|
50
|
+
|
51
|
+
On failure, the job is scheduled again in 5 seconds + N ** 4, where N is the number of retries.
|
52
|
+
|
53
|
+
The default `MAX_ATTEMPTS` is 25 (jobs can override this value by responding to `:max_attempts`). After this, the job either deleted (default), or left in the database with "failed_at" set. With the default of 25 attempts, the last retry will be 20 days later, with the last interval being almost 100 hours.
|
54
|
+
|
55
|
+
The default `MAX_RUN_TIME` is 4.hours. If your job takes longer than that, another computer could pick it up. It's up to you to make sure your job doesn't exceed this time. You should set this to the longest time you think the job could take.
|
56
|
+
|
57
|
+
By default, it will delete failed jobs. If you want to keep failed jobs, set `Delayed::Job.destroy_failed_jobs = false`. The failed jobs will be marked with non-null failed_at.
|
58
|
+
|
59
|
+
Same thing for successful jobs. They're deleted by default and, to keep them, set `Delayed::Job.destroy_successful_jobs = false`. They will be marked with finished_at. This is useful for gathering statistics like how long jobs took between entering the queue (created_at) and being finished (finished_at).
|
60
|
+
|
61
|
+
You have a couble of named scopes for searching unfinished/finsihed jobs, very useful when destroy_successful_jobs is false `Delayed::Job.unfinished` and `Delayed::Job.finsihed`.
|
62
|
+
|
63
|
+
Here is an example of changing job parameters in Rails:
|
64
|
+
|
65
|
+
# config/initializers/delayed_job_config.rb
|
66
|
+
Delayed::Job.destroy_failed_jobs = false
|
67
|
+
Delayed::Job.destroy_successful_jobs = false
|
68
|
+
silence_warnings do
|
69
|
+
Delayed::Job.const_set("MAX_ATTEMPTS", 3)
|
70
|
+
Delayed::Job.const_set("MAX_RUN_TIME", 5.hours)
|
71
|
+
|
72
|
+
Delayed::Worker.const_set("SLEEP", 5.minutes.to_i)
|
73
|
+
end
|
74
|
+
|
75
|
+
Note: If your error messages are long, consider changing last_error field to a :text instead of a :string (255 character limit).
|
76
|
+
|
77
|
+
|
78
|
+
Usage
|
79
|
+
-----
|
80
|
+
|
81
|
+
Jobs are simple ruby objects with a method called perform. Any object which responds to perform can be stuffed into the jobs table.
|
82
|
+
Job objects are serialized to yaml so that they can later be resurrected by the job runner.
|
83
|
+
|
84
|
+
class NewsletterJob < Struct.new(:text, :emails)
|
85
|
+
def perform
|
86
|
+
emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
|
87
|
+
end
|
88
|
+
end
|
89
|
+
|
90
|
+
Delayed::Job.enqueue NewsletterJob.new('lorem ipsum...', Customers.find(:all).collect(&:email))
|
91
|
+
|
92
|
+
There is also a second way to get jobs in the queue: send_later.
|
93
|
+
|
94
|
+
BatchImporter.new(Shop.find(1)).send_later(:import_massive_csv, massive_csv)
|
95
|
+
|
96
|
+
And also you can specified priority as second parameter and the time the job should execute as thrird one
|
97
|
+
|
98
|
+
|
99
|
+
class FooJob
|
100
|
+
def perform
|
101
|
+
...
|
102
|
+
end
|
103
|
+
end
|
104
|
+
|
105
|
+
important_job = FooJob.new
|
106
|
+
normal_job = FooJob.new
|
107
|
+
|
108
|
+
# Delayed::Job.enqueue( job, priority, start_at )
|
109
|
+
Delayed::Job.enqueue important_job, 100
|
110
|
+
Delayed::Job.enqueue normal_job, 1, 2.hours.from_now
|
111
|
+
|
112
|
+
This will simply create a `Delayed::PerformableMethod` job in the jobs table which serializes all the parameters you pass to it. There are some special smarts for active record objects which are stored as their text representation and loaded from the database fresh when the job is actually run later.
|
113
|
+
|
114
|
+
|
115
|
+
Running the jobs
|
116
|
+
----------------
|
117
|
+
|
118
|
+
You can invoke `rake jobs:work` which will start working off jobs. You can cancel the rake task with `CTRL-C`.
|
119
|
+
|
120
|
+
You can also run by writing a simple `script/job_runner`, and invoking it externally:
|
121
|
+
|
122
|
+
|
123
|
+
require File.dirname(__FILE__) + '/../config/environment'
|
124
|
+
|
125
|
+
Delayed::Worker.new.start
|
126
|
+
|
127
|
+
Workers can be running on any computer, as long as they have access to the database and their clock is in sync. You can even run multiple workers on per computer, but you must give each one a unique name.
|
128
|
+
|
129
|
+
|
130
|
+
require File.dirname(__FILE__) + '/../config/environment'
|
131
|
+
N = 10
|
132
|
+
workers = []
|
133
|
+
N.times do |n|
|
134
|
+
workers << Thread.new do
|
135
|
+
Delayed::Worker.new( :name => "Worker #{n}" ).start
|
136
|
+
end
|
137
|
+
end
|
138
|
+
|
139
|
+
workers.first.join # wait until finish (signal catched)
|
140
|
+
|
141
|
+
Keep in mind that each worker will check the database at least every 5 seconds.
|
142
|
+
|
143
|
+
Note: The rake task will exit if the database has any network connectivity problems.
|
144
|
+
|
145
|
+
If you only want to run specific types of jobs in a given worker, include them when initializing the worker:
|
146
|
+
|
147
|
+
Delayed::Worker.new(:job_types => "SimpleJob").start
|
148
|
+
Delayed::Worker.new(:job_types => ["SimpleJob", "NewsletterJob"]).start
|
149
|
+
|
150
|
+
Also for a more specific restriction you can define in your job's classes a `display_name` method, and create workers to specific kind of jobs
|
151
|
+
|
152
|
+
# 1 - The job class that does the real work
|
153
|
+
class MyJob
|
154
|
+
def initialize( data )
|
155
|
+
@some_data = data
|
156
|
+
end
|
157
|
+
|
158
|
+
def perform
|
159
|
+
# do the real work
|
160
|
+
end
|
161
|
+
|
162
|
+
def display_name
|
163
|
+
"foobar #{@some_data}"
|
164
|
+
end
|
165
|
+
end
|
166
|
+
|
167
|
+
# 2 - Enqueue jobs
|
168
|
+
Delayed::Job.enqueue MyJob.new("foobar")
|
169
|
+
Delayed::Job.enqueue MyJob.new("arrrr")
|
170
|
+
|
171
|
+
# 3 - Create workers, one for each type of "data"
|
172
|
+
Thread.new {
|
173
|
+
# This worker will only perform jobs which display name is like "%foobar%"
|
174
|
+
Delayed::Worker.new :name => "Worker for foobar", :only_for => "foobar"
|
175
|
+
}
|
176
|
+
Thread.new {
|
177
|
+
Delayed::Worker.new :name => "Worker for arrr", :only_for => "arrr"
|
178
|
+
}
|
179
|
+
|
180
|
+
|
181
|
+
Cleaning up
|
182
|
+
-----------
|
183
|
+
|
184
|
+
You can invoke `rake jobs:clear` to delete all jobs in the queue.
|
185
|
+
|
186
|
+
Changes
|
187
|
+
-------
|
188
|
+
|
189
|
+
* 2.0.0: Contains the changes made in this fork, the ability to create workers with individual constraints without interfere to other workers
|
190
|
+
|
191
|
+
* 1.7.0: Added failed_at column which can optionally be set after a certain amount of failed job attempts. By default failed job attempts are destroyed after about a month.
|
192
|
+
|
193
|
+
* 1.6.0: Renamed locked_until to locked_at. We now store when we start a given job instead of how long it will be locked by the worker. This allows us to get a reading on how long a job took to execute.
|
194
|
+
|
195
|
+
* 1.5.0: Job runners can now be run in parallel. Two new database columns are needed: locked_until and locked_by. This allows us to use pessimistic locking instead of relying on row level locks. This enables us to run as many worker processes as we need to speed up queue processing.
|
196
|
+
|
197
|
+
* 1.2.0: Added #send_later to Object for simpler job creation
|
198
|
+
|
199
|
+
* 1.0.0: Initial release
|
data/VERSION
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
2.0.0
|
data/delayed_job.gemspec
CHANGED
@@ -2,21 +2,19 @@
|
|
2
2
|
|
3
3
|
Gem::Specification.new do |s|
|
4
4
|
s.name = %q{delayed_job}
|
5
|
-
s.version = "
|
5
|
+
s.version = "2.0.0"
|
6
6
|
|
7
7
|
s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
|
8
8
|
s.authors = ["Tobias L\303\274tke"]
|
9
|
-
s.date = %q{2009-08-
|
9
|
+
s.date = %q{2009-08-10}
|
10
10
|
s.description = %q{Delayed_job (or DJ) encapsulates the common pattern of asynchronously executing longer tasks in the background. It is a direct extraction from Shopify where the job table is responsible for a multitude of core tasks.}
|
11
11
|
s.email = %q{tobi@leetsoft.com}
|
12
|
-
s.extra_rdoc_files = [
|
13
|
-
"README.textile"
|
14
|
-
]
|
15
12
|
s.files = [
|
16
13
|
".gitignore",
|
17
14
|
"MIT-LICENSE",
|
18
|
-
"README.
|
15
|
+
"README.markdown",
|
19
16
|
"Rakefile",
|
17
|
+
"VERSION",
|
20
18
|
"delayed_job.gemspec",
|
21
19
|
"generators/delayed_job/delayed_job_generator.rb",
|
22
20
|
"generators/delayed_job/templates/migration.rb",
|
data/lib/delayed/job.rb
CHANGED
@@ -22,28 +22,110 @@ module Delayed
|
|
22
22
|
# If you want to keep them around (for statistics/monitoring),
|
23
23
|
# set this to false.
|
24
24
|
cattr_accessor :destroy_successful_jobs
|
25
|
-
self.destroy_successful_jobs =
|
26
|
-
|
27
|
-
# Every worker has a unique name which by default is the pid of the process.
|
28
|
-
# There are some advantages to overriding this with something which survives worker retarts:
|
29
|
-
# Workers can safely resume working on tasks which are locked by themselves. The worker will assume that it crashed before.
|
30
|
-
cattr_accessor :worker_name
|
31
|
-
self.worker_name = "host:#{Socket.gethostname} pid:#{Process.pid}" rescue "pid:#{Process.pid}"
|
25
|
+
self.destroy_successful_jobs = false
|
32
26
|
|
33
27
|
NextTaskSQL = '(run_at <= ? AND (locked_at IS NULL OR locked_at < ?) OR (locked_by = ?)) AND failed_at IS NULL AND finished_at IS NULL'
|
34
28
|
NextTaskOrder = 'priority DESC, run_at ASC'
|
35
29
|
|
36
30
|
ParseObjectFromYaml = /\!ruby\/\w+\:([^\s]+)/
|
37
31
|
|
38
|
-
|
39
|
-
|
40
|
-
self.max_priority = nil
|
41
|
-
self.job_types = nil
|
32
|
+
named_scope :unfinished, :conditions => { :finished_at => nil }
|
33
|
+
named_scope :finished, :conditions => [ "finished_at IS NOT NULL" ]
|
42
34
|
|
43
|
-
|
44
|
-
|
45
|
-
|
46
|
-
|
35
|
+
class << self
|
36
|
+
# When a worker is exiting, make sure we don't have any locked jobs.
|
37
|
+
def clear_locks!( worker_name )
|
38
|
+
update_all("locked_by = null, locked_at = null", ["locked_by = ?", worker_name])
|
39
|
+
end
|
40
|
+
|
41
|
+
# Add a job to the queue
|
42
|
+
def enqueue(*args, &block)
|
43
|
+
object = block_given? ? EvaledJob.new(&block) : args.shift
|
44
|
+
|
45
|
+
unless object.respond_to?(:perform) || block_given?
|
46
|
+
raise ArgumentError, 'Cannot enqueue items which do not respond to perform'
|
47
|
+
end
|
48
|
+
|
49
|
+
priority = args.first || 0
|
50
|
+
run_at = args[1]
|
51
|
+
|
52
|
+
Job.create(:payload_object => object, :priority => priority.to_i, :run_at => run_at)
|
53
|
+
end
|
54
|
+
|
55
|
+
# Find a few candidate jobs to run (in case some immediately get locked by others).
|
56
|
+
# Return in random order prevent everyone trying to do same head job at once.
|
57
|
+
def find_available( options = {} )
|
58
|
+
limit = options[:limit] || 5
|
59
|
+
max_run_time = options[:max_run_time] || MAX_RUN_TIME
|
60
|
+
worker_name = options[:worker_name] || Worker::DEFAULT_WORKER_NAME
|
61
|
+
|
62
|
+
sql = NextTaskSQL.dup
|
63
|
+
time_now = db_time_now
|
64
|
+
conditions = [time_now, time_now - max_run_time, worker_name]
|
65
|
+
if options[:min_priority]
|
66
|
+
sql << ' AND (priority >= ?)'
|
67
|
+
conditions << options[:min_priority]
|
68
|
+
end
|
69
|
+
|
70
|
+
if options[:max_priority]
|
71
|
+
sql << ' AND (priority <= ?)'
|
72
|
+
conditions << options[:max_priority]
|
73
|
+
end
|
74
|
+
|
75
|
+
if options[:job_types]
|
76
|
+
sql << ' AND (job_type IN (?))'
|
77
|
+
conditions << options[:job_types]
|
78
|
+
end
|
79
|
+
|
80
|
+
if options[:only_for]
|
81
|
+
sql << ' AND name LIKE ?'
|
82
|
+
conditions << "%#{options[:only_for]}%"
|
83
|
+
end
|
84
|
+
conditions.unshift(sql)
|
85
|
+
|
86
|
+
ActiveRecord::Base.silence do
|
87
|
+
find(:all, :conditions => conditions, :order => NextTaskOrder, :limit => limit)
|
88
|
+
end
|
89
|
+
end
|
90
|
+
|
91
|
+
# Run the next job we can get an exclusive lock on.
|
92
|
+
# If no jobs are left we return nil
|
93
|
+
def reserve_and_run_one_job( options = {} )
|
94
|
+
max_run_time = options[:max_run_time] || MAX_RUN_TIME
|
95
|
+
worker_name = options[:worker_name] || Worker::DEFAULT_WORKER_NAME
|
96
|
+
# For the next jobs availables, try to get lock. In case we cannot get exclusive
|
97
|
+
# access to a job we try the next.
|
98
|
+
# This leads to a more even distribution of jobs across the worker processes.
|
99
|
+
find_available( options ).each do |job|
|
100
|
+
t = job.run_with_lock( max_run_time, worker_name )
|
101
|
+
return t unless t.nil? # return if we did work (good or bad)
|
102
|
+
end
|
103
|
+
# we didn't do any work, all 5 were not lockable
|
104
|
+
nil
|
105
|
+
end
|
106
|
+
|
107
|
+
# Do num jobs and return stats on success/failure.
|
108
|
+
# Exit early if interrupted.
|
109
|
+
def work_off( options = {} )
|
110
|
+
n = options[:n] || 100
|
111
|
+
success, failure = 0, 0
|
112
|
+
|
113
|
+
n.times do
|
114
|
+
case reserve_and_run_one_job( options )
|
115
|
+
when true
|
116
|
+
success += 1
|
117
|
+
when false
|
118
|
+
failure += 1
|
119
|
+
else
|
120
|
+
break # leave if no work could be done
|
121
|
+
end
|
122
|
+
break if Worker.exit # leave if we're exiting
|
123
|
+
end
|
124
|
+
|
125
|
+
return [success, failure]
|
126
|
+
end
|
127
|
+
|
128
|
+
end # class << self
|
47
129
|
|
48
130
|
def failed?
|
49
131
|
failed_at
|
@@ -55,18 +137,25 @@ module Delayed
|
|
55
137
|
end
|
56
138
|
|
57
139
|
def name
|
58
|
-
|
140
|
+
if new_record?
|
59
141
|
payload = payload_object
|
60
142
|
if payload.respond_to?(:display_name)
|
61
143
|
payload.display_name
|
62
144
|
else
|
63
145
|
payload.class.name
|
64
146
|
end
|
147
|
+
else
|
148
|
+
self['name']
|
65
149
|
end
|
66
150
|
end
|
67
151
|
|
68
152
|
def payload_object=(object)
|
69
|
-
|
153
|
+
job_type = if object.respond_to? :job_type
|
154
|
+
object.job_type
|
155
|
+
else
|
156
|
+
object.class.to_s
|
157
|
+
end
|
158
|
+
self['job_type'] = job_type
|
70
159
|
self['handler'] = object.to_yaml
|
71
160
|
end
|
72
161
|
|
@@ -87,9 +176,9 @@ module Delayed
|
|
87
176
|
end
|
88
177
|
end
|
89
178
|
|
90
|
-
|
91
|
-
#
|
92
|
-
def run_with_lock(max_run_time, worker_name)
|
179
|
+
# Try to run one job.
|
180
|
+
# Returns true/false (work done/work failed) or nil if job can't be locked.
|
181
|
+
def run_with_lock(max_run_time = MAX_RUN_TIME, worker_name = Worker::DEFAULT_WORKER_NAME)
|
93
182
|
Delayed::Worker.logger.info "* [JOB] aquiring lock on #{name}"
|
94
183
|
unless lock_exclusively!(max_run_time, worker_name)
|
95
184
|
# We did not get the lock, some other worker process must have
|
@@ -103,7 +192,7 @@ module Delayed
|
|
103
192
|
end
|
104
193
|
destroy_successful_jobs ? destroy :
|
105
194
|
update_attribute(:finished_at, Time.now)
|
106
|
-
Delayed::Worker.logger.info "* [JOB] #{name} completed after %.4f" % runtime
|
195
|
+
Delayed::Worker.logger.info "* [JOB] #{name} completed after %.4f" % runtime
|
107
196
|
return true # did work
|
108
197
|
rescue Exception => e
|
109
198
|
reschedule e.message, e.backtrace
|
@@ -112,71 +201,9 @@ module Delayed
|
|
112
201
|
end
|
113
202
|
end
|
114
203
|
|
115
|
-
#
|
116
|
-
def self.enqueue(*args, &block)
|
117
|
-
object = block_given? ? EvaledJob.new(&block) : args.shift
|
118
|
-
|
119
|
-
unless object.respond_to?(:perform) || block_given?
|
120
|
-
raise ArgumentError, 'Cannot enqueue items which do not respond to perform'
|
121
|
-
end
|
122
|
-
|
123
|
-
priority = args.first || 0
|
124
|
-
run_at = args[1]
|
125
|
-
|
126
|
-
Job.create(:payload_object => object, :priority => priority.to_i, :run_at => run_at)
|
127
|
-
end
|
128
|
-
|
129
|
-
# Find a few candidate jobs to run (in case some immediately get locked by others).
|
130
|
-
# Return in random order prevent everyone trying to do same head job at once.
|
131
|
-
def self.find_available(limit = 5, max_run_time = MAX_RUN_TIME)
|
132
|
-
|
133
|
-
time_now = db_time_now
|
134
|
-
|
135
|
-
sql = NextTaskSQL.dup
|
136
|
-
|
137
|
-
conditions = [time_now, time_now - max_run_time, worker_name]
|
138
|
-
|
139
|
-
if self.min_priority
|
140
|
-
sql << ' AND (priority >= ?)'
|
141
|
-
conditions << min_priority
|
142
|
-
end
|
143
|
-
|
144
|
-
if self.max_priority
|
145
|
-
sql << ' AND (priority <= ?)'
|
146
|
-
conditions << max_priority
|
147
|
-
end
|
148
|
-
|
149
|
-
if self.job_types
|
150
|
-
sql << ' AND (job_type IN (?))'
|
151
|
-
conditions << job_types
|
152
|
-
end
|
153
|
-
|
154
|
-
conditions.unshift(sql)
|
155
|
-
|
156
|
-
records = ActiveRecord::Base.silence do
|
157
|
-
find(:all, :conditions => conditions, :order => NextTaskOrder, :limit => limit)
|
158
|
-
end
|
159
|
-
|
160
|
-
records.sort_by { rand() }
|
161
|
-
end
|
162
|
-
|
163
|
-
# Run the next job we can get an exclusive lock on.
|
164
|
-
# If no jobs are left we return nil
|
165
|
-
def self.reserve_and_run_one_job(max_run_time = MAX_RUN_TIME)
|
166
|
-
|
167
|
-
# We get up to 5 jobs from the db. In case we cannot get exclusive access to a job we try the next.
|
168
|
-
# this leads to a more even distribution of jobs across the worker processes
|
169
|
-
find_available(5, max_run_time).each do |job|
|
170
|
-
t = job.run_with_lock(max_run_time, worker_name)
|
171
|
-
return t unless t == nil # return if we did work (good or bad)
|
172
|
-
end
|
173
|
-
|
174
|
-
nil # we didn't do any work, all 5 were not lockable
|
175
|
-
end
|
176
|
-
|
177
|
-
# Lock this job for this worker.
|
204
|
+
# Lock this job for the worker given as parameter (the name).
|
178
205
|
# Returns true if we have the lock, false otherwise.
|
179
|
-
def lock_exclusively!(max_run_time, worker
|
206
|
+
def lock_exclusively!(max_run_time, worker)
|
180
207
|
now = self.class.db_time_now
|
181
208
|
affected_rows = if locked_by != worker
|
182
209
|
# We don't own this job so we will update the locked_by name and the locked_at
|
@@ -207,26 +234,6 @@ module Delayed
|
|
207
234
|
Delayed::Worker.logger.error(error)
|
208
235
|
end
|
209
236
|
|
210
|
-
# Do num jobs and return stats on success/failure.
|
211
|
-
# Exit early if interrupted.
|
212
|
-
def self.work_off(num = 100)
|
213
|
-
success, failure = 0, 0
|
214
|
-
|
215
|
-
num.times do
|
216
|
-
case self.reserve_and_run_one_job
|
217
|
-
when true
|
218
|
-
success += 1
|
219
|
-
when false
|
220
|
-
failure += 1
|
221
|
-
else
|
222
|
-
break # leave if no work could be done
|
223
|
-
end
|
224
|
-
break if $exit # leave if we're exiting
|
225
|
-
end
|
226
|
-
|
227
|
-
return [success, failure]
|
228
|
-
end
|
229
|
-
|
230
237
|
# Moved into its own method so that new_relic can trace it.
|
231
238
|
def invoke_job
|
232
239
|
payload_object.perform
|
@@ -271,6 +278,7 @@ module Delayed
|
|
271
278
|
|
272
279
|
def before_save
|
273
280
|
self.run_at ||= self.class.db_time_now
|
281
|
+
self['name'] = name
|
274
282
|
end
|
275
283
|
|
276
284
|
end
|
@@ -3,7 +3,7 @@ module Delayed
|
|
3
3
|
def send_later(method, *args)
|
4
4
|
Delayed::Job.enqueue Delayed::PerformableMethod.new(self, method.to_sym, args)
|
5
5
|
end
|
6
|
-
|
6
|
+
|
7
7
|
module ClassMethods
|
8
8
|
def handle_asynchronously(method)
|
9
9
|
without_name = "#{method}_without_send_later"
|
@@ -13,5 +13,5 @@ module Delayed
|
|
13
13
|
alias_method_chain method, :send_later
|
14
14
|
end
|
15
15
|
end
|
16
|
-
end
|
17
|
-
end
|
16
|
+
end
|
17
|
+
end
|
@@ -10,14 +10,14 @@ module Delayed
|
|
10
10
|
self.args = args.map { |a| dump(a) }
|
11
11
|
self.method = method.to_sym
|
12
12
|
end
|
13
|
-
|
14
|
-
def display_name
|
13
|
+
|
14
|
+
def display_name
|
15
15
|
case self.object
|
16
16
|
when CLASS_STRING_FORMAT then "#{$1}.#{method}"
|
17
17
|
when AR_STRING_FORMAT then "#{$1}##{method}"
|
18
18
|
else "Unknown##{method}"
|
19
|
-
end
|
20
|
-
end
|
19
|
+
end
|
20
|
+
end
|
21
21
|
|
22
22
|
def perform
|
23
23
|
load(object).send(method, *args.map{|a| load(a)})
|
@@ -26,6 +26,15 @@ module Delayed
|
|
26
26
|
true
|
27
27
|
end
|
28
28
|
|
29
|
+
def job_type
|
30
|
+
class_name = case self.object
|
31
|
+
when CLASS_STRING_FORMAT then $1.to_s
|
32
|
+
when AR_STRING_FORMAT then $1.to_s
|
33
|
+
else self.object.class.to_s
|
34
|
+
end
|
35
|
+
"#{class_name}##{self.method.to_s}"
|
36
|
+
end
|
37
|
+
|
29
38
|
private
|
30
39
|
|
31
40
|
def load(arg)
|
@@ -52,4 +61,4 @@ module Delayed
|
|
52
61
|
"CLASS:#{obj.name}"
|
53
62
|
end
|
54
63
|
end
|
55
|
-
end
|
64
|
+
end
|
data/lib/delayed/worker.rb
CHANGED
@@ -1,6 +1,12 @@
|
|
1
1
|
module Delayed
|
2
2
|
class Worker
|
3
3
|
SLEEP = 5
|
4
|
+
JOBS_EACH = 100 # Jobs executed in each iteration
|
5
|
+
|
6
|
+
DEFAULT_WORKER_NAME = "host:#{Socket.gethostname} pid:#{Process.pid}" rescue "pid:#{Process.pid}"
|
7
|
+
# Indicates that we have catched a signal and we have to exit asap
|
8
|
+
cattr_accessor :exit
|
9
|
+
self.exit = false
|
4
10
|
|
5
11
|
cattr_accessor :logger
|
6
12
|
self.logger = if defined?(Merb::Logger)
|
@@ -9,47 +15,82 @@ module Delayed
|
|
9
15
|
RAILS_DEFAULT_LOGGER
|
10
16
|
end
|
11
17
|
|
18
|
+
# Every worker has a unique name which by default is the pid of the process (so you only are
|
19
|
+
# be able to have one unless override this in the constructor).
|
20
|
+
#
|
21
|
+
# Thread.new { Delayed::Worker.new( :name => "Worker 1" ).start }
|
22
|
+
# Thread.new { Delayed::Worker.new( :name => "Worker 2" ).start }
|
23
|
+
#
|
24
|
+
# There are some advantages to overriding this with something which survives worker retarts:
|
25
|
+
# Workers can safely resume working on tasks which are locked by themselves.
|
26
|
+
# The worker will assume that it crashed before.
|
27
|
+
attr_accessor :name
|
28
|
+
|
29
|
+
# Constraints for this worker, what kind of jobs is gonna execute?
|
30
|
+
attr_accessor :min_priority, :max_priority, :job_types
|
31
|
+
|
32
|
+
attr_accessor :quiet
|
33
|
+
|
34
|
+
# A worker will be in a loop trying to execute pending jobs, you can also set
|
35
|
+
# a few constraints to customize the worker's behaviour.
|
36
|
+
#
|
37
|
+
# Named parameters:
|
38
|
+
# - name: the name of the worker, mandatory if you are going to create several workers
|
39
|
+
# - quiet: log to stdout (besides the normal logger)
|
40
|
+
# - min_priority: constraint for selecting what jobs to execute (integer)
|
41
|
+
# - max_priority: constraint for selecting what jobs to execute (integer)
|
42
|
+
# - job_types: constraint for selecting what jobs to execute (String or Array)
|
12
43
|
def initialize(options={})
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
44
|
+
[ :quiet, :name, :min_priority, :max_priority, :job_types ].each do |attr_name|
|
45
|
+
send "#{attr_name}=", options[attr_name]
|
46
|
+
end
|
47
|
+
# Default values
|
48
|
+
self.name = DEFAULT_WORKER_NAME if self.name.nil?
|
49
|
+
self.quiet = true if self.quiet.nil?
|
17
50
|
end
|
18
51
|
|
19
52
|
def start
|
20
|
-
say "*** Starting job worker #{
|
53
|
+
say "*** Starting job worker #{name}"
|
21
54
|
|
22
|
-
trap('TERM') { say 'Exiting...';
|
23
|
-
trap('INT') { say 'Exiting...';
|
55
|
+
trap('TERM') { say 'Exiting...'; self.exit = true }
|
56
|
+
trap('INT') { say 'Exiting...'; self.exit = true }
|
24
57
|
|
25
58
|
loop do
|
26
59
|
result = nil
|
27
60
|
|
28
61
|
realtime = Benchmark.realtime do
|
29
|
-
result =
|
62
|
+
result = Job.work_off( constraints )
|
30
63
|
end
|
31
64
|
|
32
65
|
count = result.sum
|
33
66
|
|
34
|
-
break if $exit
|
35
|
-
|
36
67
|
if count.zero?
|
37
|
-
sleep(SLEEP)
|
68
|
+
sleep(SLEEP) unless self.exit
|
38
69
|
else
|
39
70
|
say "#{count} jobs processed at %.4f j/s, %d failed ..." % [count / realtime, result.last]
|
40
71
|
end
|
41
|
-
|
42
|
-
break if $exit
|
72
|
+
break if self.exit
|
43
73
|
end
|
44
74
|
|
45
75
|
ensure
|
46
|
-
|
76
|
+
Job.clear_locks! name
|
47
77
|
end
|
48
78
|
|
49
79
|
def say(text)
|
50
|
-
|
80
|
+
text = "#{name}: #{text}"
|
81
|
+
puts text unless self.quiet
|
51
82
|
logger.info text if logger
|
52
83
|
end
|
53
84
|
|
85
|
+
protected
|
86
|
+
def constraints
|
87
|
+
{ :max_run_time => Job::MAX_RUN_TIME,
|
88
|
+
:worker_name => name,
|
89
|
+
:n => JOBS_EACH,
|
90
|
+
:limit => 5,
|
91
|
+
:min_priority => min_priority,
|
92
|
+
:max_priority => max_priority,
|
93
|
+
:job_types => job_types }
|
94
|
+
end
|
54
95
|
end
|
55
96
|
end
|
data/spec/database.rb
CHANGED
data/spec/delayed_method_spec.rb
CHANGED
@@ -37,12 +37,10 @@ class StoryReader
|
|
37
37
|
end
|
38
38
|
|
39
39
|
describe 'random ruby objects' do
|
40
|
-
before
|
40
|
+
before { Delayed::Job.delete_all }
|
41
41
|
|
42
42
|
it "should respond_to :send_later method" do
|
43
|
-
|
44
43
|
RandomRubyObject.new.respond_to?(:send_later)
|
45
|
-
|
46
44
|
end
|
47
45
|
|
48
46
|
it "should raise a ArgumentError if send_later is called but the target method doesn't exist" do
|
@@ -73,15 +71,21 @@ describe 'random ruby objects' do
|
|
73
71
|
end
|
74
72
|
|
75
73
|
it "should ignore ActiveRecord::RecordNotFound errors because they are permanent" do
|
76
|
-
|
77
74
|
ErrorObject.new.send_later(:throw)
|
78
75
|
|
79
|
-
Delayed::Job.count.should == 1
|
76
|
+
Delayed::Job.unfinished.count.should == 1
|
80
77
|
|
81
|
-
Delayed::Job.
|
78
|
+
Delayed::Job.work_off
|
82
79
|
|
83
|
-
Delayed::Job.count.should == 0
|
80
|
+
Delayed::Job.unfinished.count.should == 0
|
81
|
+
end
|
82
|
+
|
83
|
+
it "should store the job type properly when its an normal class" do
|
84
|
+
story = Story.create :text => 'Once upon...'
|
85
|
+
story.send_later :tell
|
84
86
|
|
87
|
+
job = Delayed::Job.first
|
88
|
+
job.job_type.should == "Story#tell"
|
85
89
|
end
|
86
90
|
|
87
91
|
it "should store the object as string if its an active record" do
|
data/spec/job_spec.rb
CHANGED
@@ -7,12 +7,16 @@ end
|
|
7
7
|
|
8
8
|
class CustomJob < SimpleJob
|
9
9
|
def max_attempts; 3; end
|
10
|
+
|
11
|
+
def display_name
|
12
|
+
"custom name for the job"
|
13
|
+
end
|
10
14
|
end
|
11
15
|
|
12
16
|
class ErrorJob
|
13
17
|
cattr_accessor :runs; self.runs = 0
|
14
18
|
def perform; raise 'did not work'; end
|
15
|
-
end
|
19
|
+
end
|
16
20
|
|
17
21
|
class LongRunningJob
|
18
22
|
def perform; sleep 250; end
|
@@ -21,19 +25,15 @@ end
|
|
21
25
|
module M
|
22
26
|
class ModuleJob
|
23
27
|
cattr_accessor :runs; self.runs = 0
|
24
|
-
def perform; @@runs += 1; end
|
28
|
+
def perform; @@runs += 1; end
|
25
29
|
end
|
26
|
-
|
27
30
|
end
|
28
31
|
|
29
32
|
describe Delayed::Job do
|
30
|
-
before
|
31
|
-
Delayed::Job.max_priority = nil
|
32
|
-
Delayed::Job.min_priority = nil
|
33
|
-
|
33
|
+
before do
|
34
34
|
Delayed::Job.delete_all
|
35
35
|
end
|
36
|
-
|
36
|
+
|
37
37
|
before(:each) do
|
38
38
|
SimpleJob.runs = 0
|
39
39
|
end
|
@@ -78,40 +78,55 @@ describe Delayed::Job do
|
|
78
78
|
SimpleJob.runs.should == 1
|
79
79
|
end
|
80
80
|
|
81
|
+
it "should set name properly according to the class" do
|
82
|
+
job = Delayed::Job.enqueue SimpleJob.new
|
83
|
+
job.name.should == "SimpleJob"
|
84
|
+
end
|
85
|
+
|
86
|
+
it "should set name properly when display_name method is defined in the object" do
|
87
|
+
job = Delayed::Job.enqueue CustomJob.new
|
88
|
+
job.name.should_not == "CustomJob"
|
89
|
+
job.name.should == "custom name for the job"
|
90
|
+
end
|
91
|
+
|
92
|
+
it "should work on specified job types for common objects" do
|
93
|
+
Delayed::Job.unfinished.should have(0).jobs
|
94
|
+
Delayed::Job.finished.should have(0).jobs
|
95
|
+
|
96
|
+
"a string".send_later :size
|
97
|
+
Delayed::Job.unfinished.should have(1).jobs
|
98
|
+
Delayed::Job.finished.should have(0).jobs
|
99
|
+
|
100
|
+
Delayed::Job.work_off :job_types => "String#size"
|
101
|
+
Delayed::Job.unfinished.should have(0).jobs
|
102
|
+
Delayed::Job.finished.should have(1).job
|
103
|
+
end
|
104
|
+
|
81
105
|
it "should work on specified job types" do
|
82
106
|
SimpleJob.runs.should == 0
|
83
107
|
|
84
|
-
Delayed::Job.job_types = "SimpleJob"
|
85
108
|
Delayed::Job.enqueue SimpleJob.new
|
86
|
-
Delayed::Job.work_off
|
109
|
+
Delayed::Job.work_off :job_types => "SimpleJob"
|
87
110
|
|
88
111
|
SimpleJob.runs.should == 1
|
89
|
-
|
90
|
-
Delayed::Job.job_types = nil
|
91
112
|
end
|
92
113
|
|
93
114
|
it "should not work on unspecified job types" do
|
94
115
|
SimpleJob.runs.should == 0
|
95
116
|
|
96
|
-
Delayed::Job.job_types = "AnotherJob"
|
97
117
|
Delayed::Job.enqueue SimpleJob.new
|
98
|
-
Delayed::Job.work_off
|
118
|
+
Delayed::Job.work_off :job_types => "AnotherJob"
|
99
119
|
|
100
120
|
SimpleJob.runs.should == 0
|
101
|
-
|
102
|
-
Delayed::Job.job_types = nil
|
103
121
|
end
|
104
122
|
|
105
123
|
it "should work on specified job types even when it's a list" do
|
106
124
|
SimpleJob.runs.should == 0
|
107
125
|
|
108
|
-
Delayed::Job.job_types = %w( Whatever SimpleJob )
|
109
126
|
Delayed::Job.enqueue SimpleJob.new
|
110
|
-
Delayed::Job.work_off
|
127
|
+
Delayed::Job.work_off :job_types => %w( Whatever SimpleJob )
|
111
128
|
|
112
129
|
SimpleJob.runs.should == 1
|
113
|
-
|
114
|
-
Delayed::Job.job_types = nil
|
115
130
|
end
|
116
131
|
|
117
132
|
it "should work with eval jobs" do
|
@@ -126,7 +141,7 @@ describe Delayed::Job do
|
|
126
141
|
|
127
142
|
$eval_job_ran.should == true
|
128
143
|
end
|
129
|
-
|
144
|
+
|
130
145
|
it "should work with jobs in modules" do
|
131
146
|
M::ModuleJob.runs.should == 0
|
132
147
|
|
@@ -175,17 +190,17 @@ describe Delayed::Job do
|
|
175
190
|
it "should never find finished jobs" do
|
176
191
|
@job = Delayed::Job.create :payload_object => SimpleJob.new,
|
177
192
|
:finished_at => Time.now
|
178
|
-
Delayed::Job.find_available(1).length.should == 0
|
193
|
+
Delayed::Job.find_available( :limit => 1 ).length.should == 0
|
179
194
|
end
|
180
195
|
|
181
196
|
it "should re-schedule by about 1 second at first and increment this more and more minutes when it fails to execute properly" do
|
182
|
-
Delayed::Job.enqueue ErrorJob.new
|
183
|
-
Delayed::Job.work_off(1)
|
197
|
+
job = Delayed::Job.enqueue ErrorJob.new
|
198
|
+
Delayed::Job.work_off( :n => 1 )
|
184
199
|
|
185
|
-
job
|
200
|
+
job.reload
|
186
201
|
|
187
202
|
job.last_error.should =~ /did not work/
|
188
|
-
job.last_error.should =~ /job_spec.rb
|
203
|
+
job.last_error.should =~ /job_spec.rb:\d+:in `perform'/
|
189
204
|
job.attempts.should == 1
|
190
205
|
|
191
206
|
job.run_at.should > Delayed::Job.db_time_now - 10.minutes
|
@@ -232,7 +247,7 @@ describe Delayed::Job do
|
|
232
247
|
job.should_receive(:attempt_to_load).with('Delayed::JobThatDoesNotExist').and_return(true)
|
233
248
|
lambda { job.payload_object.perform }.should raise_error(Delayed::DeserializationError)
|
234
249
|
end
|
235
|
-
|
250
|
+
|
236
251
|
it "should be failed if it failed more than MAX_ATTEMPTS times and we don't want to destroy jobs" do
|
237
252
|
default = Delayed::Job.destroy_failed_jobs
|
238
253
|
Delayed::Job.destroy_failed_jobs = false
|
@@ -269,21 +284,41 @@ describe Delayed::Job do
|
|
269
284
|
|
270
285
|
it "should fail after MAX_RUN_TIME" do
|
271
286
|
@job = Delayed::Job.create :payload_object => LongRunningJob.new
|
272
|
-
Delayed::Job.reserve_and_run_one_job(1.second)
|
287
|
+
Delayed::Job.reserve_and_run_one_job( :max_run_time => 1.second )
|
273
288
|
@job.reload.last_error.should =~ /expired/
|
274
289
|
@job.attempts.should == 1
|
275
290
|
end
|
276
291
|
|
292
|
+
it "should find high priority jobs first" do
|
293
|
+
@job_10 = Delayed::Job.create :payload_object => SimpleJob.new, :priority => 10
|
294
|
+
@job_20 = Delayed::Job.create :payload_object => SimpleJob.new, :priority => 20
|
295
|
+
|
296
|
+
Delayed::Job.find_available( :limit => 1 ).first.should == @job_20
|
297
|
+
end
|
298
|
+
|
299
|
+
it "should find only jobs like the parameter given" do
|
300
|
+
Delayed::Job.create :payload_object => SimpleJob.new
|
301
|
+
Delayed::Job.create :payload_object => CustomJob.new
|
302
|
+
Delayed::Job.unfinished.should have(2).jobs
|
303
|
+
|
304
|
+
Delayed::Job.work_off :only_for => 'Simple'
|
305
|
+
Delayed::Job.unfinished.should have(1).jobs
|
306
|
+
|
307
|
+
Delayed::Job.work_off :only_for => 'custom'
|
308
|
+
Delayed::Job.unfinished.should have(0).jobs
|
309
|
+
end
|
310
|
+
|
277
311
|
it "should never find failed jobs" do
|
278
312
|
@job = Delayed::Job.create :payload_object => SimpleJob.new, :attempts => 50, :failed_at => Time.now
|
279
|
-
Delayed::Job.find_available(1).length.should == 0
|
313
|
+
Delayed::Job.find_available( :limit => 1 ).length.should == 0
|
280
314
|
end
|
281
315
|
|
282
316
|
context "when another worker is already performing an task, it" do
|
283
317
|
|
284
318
|
before :each do
|
285
|
-
Delayed::Job.
|
286
|
-
|
319
|
+
@job = Delayed::Job.create :payload_object => SimpleJob.new,
|
320
|
+
:locked_by => 'worker1',
|
321
|
+
:locked_at => Delayed::Job.db_time_now - 5.minutes
|
287
322
|
end
|
288
323
|
|
289
324
|
it "should not allow a second worker to get exclusive access" do
|
@@ -292,8 +327,8 @@ describe Delayed::Job do
|
|
292
327
|
|
293
328
|
it "should allow a second worker to get exclusive access if the timeout has passed" do
|
294
329
|
@job.lock_exclusively!(1.minute, 'worker2').should == true
|
295
|
-
end
|
296
|
-
|
330
|
+
end
|
331
|
+
|
297
332
|
it "should be able to get access to the task if it was started more then max_age ago" do
|
298
333
|
@job.locked_at = 5.hours.ago
|
299
334
|
@job.save
|
@@ -305,24 +340,20 @@ describe Delayed::Job do
|
|
305
340
|
end
|
306
341
|
|
307
342
|
it "should not be found by another worker" do
|
308
|
-
Delayed::Job.worker_name
|
309
|
-
|
310
|
-
Delayed::Job.find_available(1, 6.minutes).length.should == 0
|
343
|
+
Delayed::Job.find_available( :worker_name => 'worker2', :max_run_time => 6.minutes ).should have(0).jobs
|
311
344
|
end
|
312
345
|
|
313
346
|
it "should be found by another worker if the time has expired" do
|
314
|
-
Delayed::Job.worker_name
|
315
|
-
|
316
|
-
Delayed::Job.find_available(1, 4.minutes).length.should == 1
|
347
|
+
Delayed::Job.find_available( :worker_name => 'worker2', :max_run_time => 4.minutes ).should have(1).job
|
317
348
|
end
|
318
349
|
|
319
350
|
it "should be able to get exclusive access again when the worker name is the same" do
|
320
351
|
@job.lock_exclusively! 5.minutes, 'worker1'
|
321
352
|
@job.lock_exclusively! 5.minutes, 'worker1'
|
322
353
|
@job.lock_exclusively! 5.minutes, 'worker1'
|
323
|
-
end
|
324
|
-
end
|
325
|
-
|
354
|
+
end
|
355
|
+
end
|
356
|
+
|
326
357
|
context "#name" do
|
327
358
|
it "should be the class name of the job that was enqueued" do
|
328
359
|
Delayed::Job.create(:payload_object => ErrorJob.new ).name.should == 'ErrorJob'
|
@@ -334,93 +365,78 @@ describe Delayed::Job do
|
|
334
365
|
|
335
366
|
end
|
336
367
|
it "should be the instance method that will be called if its a performable method object" do
|
337
|
-
story = Story.create :text => "..."
|
338
|
-
|
368
|
+
story = Story.create :text => "..."
|
369
|
+
|
339
370
|
story.send_later(:save)
|
340
|
-
|
371
|
+
|
341
372
|
Delayed::Job.last.name.should == 'Story#save'
|
342
373
|
end
|
343
374
|
end
|
344
|
-
|
375
|
+
|
345
376
|
context "worker prioritization" do
|
346
|
-
|
347
|
-
before(:each) do
|
348
|
-
Delayed::Job.max_priority = nil
|
349
|
-
Delayed::Job.min_priority = nil
|
350
|
-
end
|
351
|
-
|
352
377
|
it "should only work_off jobs that are >= min_priority" do
|
353
|
-
Delayed::Job.min_priority = -5
|
354
|
-
Delayed::Job.max_priority = 5
|
355
378
|
SimpleJob.runs.should == 0
|
356
|
-
|
379
|
+
|
357
380
|
Delayed::Job.enqueue SimpleJob.new, -10
|
358
381
|
Delayed::Job.enqueue SimpleJob.new, 0
|
359
|
-
Delayed::Job.work_off
|
360
|
-
|
382
|
+
Delayed::Job.work_off :min_priority => -5, :max_priority => 5
|
383
|
+
|
361
384
|
SimpleJob.runs.should == 1
|
362
385
|
end
|
363
|
-
|
386
|
+
|
364
387
|
it "should only work_off jobs that are <= max_priority" do
|
365
|
-
Delayed::Job.min_priority = -5
|
366
|
-
Delayed::Job.max_priority = 5
|
367
388
|
SimpleJob.runs.should == 0
|
368
|
-
|
389
|
+
|
369
390
|
Delayed::Job.enqueue SimpleJob.new, 10
|
370
391
|
Delayed::Job.enqueue SimpleJob.new, 0
|
371
392
|
|
372
|
-
Delayed::Job.work_off
|
393
|
+
Delayed::Job.work_off :min_priority => -5, :max_priority => 5
|
373
394
|
|
374
395
|
SimpleJob.runs.should == 1
|
375
|
-
end
|
376
|
-
|
396
|
+
end
|
377
397
|
end
|
378
|
-
|
398
|
+
|
379
399
|
context "when pulling jobs off the queue for processing, it" do
|
380
400
|
before(:each) do
|
381
401
|
@job = Delayed::Job.create(
|
382
|
-
:payload_object => SimpleJob.new,
|
383
|
-
:locked_by
|
384
|
-
:locked_at
|
402
|
+
:payload_object => SimpleJob.new,
|
403
|
+
:locked_by => 'worker1',
|
404
|
+
:locked_at => Delayed::Job.db_time_now - 5.minutes )
|
385
405
|
end
|
386
406
|
|
387
407
|
it "should leave the queue in a consistent state and not run the job if locking fails" do
|
388
|
-
SimpleJob.runs.should == 0
|
408
|
+
SimpleJob.runs.should == 0
|
389
409
|
@job.stub!(:lock_exclusively!).with(any_args).once.and_return(false)
|
390
410
|
Delayed::Job.should_receive(:find_available).once.and_return([@job])
|
391
|
-
Delayed::Job.work_off(1)
|
411
|
+
Delayed::Job.work_off( :n => 1 )
|
392
412
|
SimpleJob.runs.should == 0
|
393
413
|
end
|
394
|
-
|
414
|
+
|
395
415
|
end
|
396
|
-
|
416
|
+
|
397
417
|
context "while running alongside other workers that locked jobs, it" do
|
398
418
|
before(:each) do
|
399
|
-
Delayed::Job.
|
400
|
-
Delayed::Job.create
|
401
|
-
Delayed::Job.create
|
402
|
-
Delayed::Job.create
|
403
|
-
Delayed::Job.create(:payload_object => SimpleJob.new, :locked_by => 'worker1', :locked_at => (Delayed::Job.db_time_now - 1.minutes))
|
419
|
+
Delayed::Job.create :payload_object => SimpleJob.new, :locked_by => 'worker1', :locked_at => (Delayed::Job.db_time_now - 1.minutes)
|
420
|
+
Delayed::Job.create :payload_object => SimpleJob.new, :locked_by => 'worker2', :locked_at => (Delayed::Job.db_time_now - 1.minutes)
|
421
|
+
Delayed::Job.create :payload_object => SimpleJob.new
|
422
|
+
Delayed::Job.create :payload_object => SimpleJob.new, :locked_by => 'worker1', :locked_at => (Delayed::Job.db_time_now - 1.minutes)
|
404
423
|
end
|
405
424
|
|
406
425
|
it "should ingore locked jobs from other workers" do
|
407
|
-
Delayed::Job.worker_name = 'worker3'
|
408
426
|
SimpleJob.runs.should == 0
|
409
|
-
Delayed::Job.work_off
|
427
|
+
Delayed::Job.work_off :worker_name => 'worker3'
|
410
428
|
SimpleJob.runs.should == 1 # runs the one open job
|
411
429
|
end
|
412
430
|
|
413
431
|
it "should find our own jobs regardless of locks" do
|
414
|
-
Delayed::Job.worker_name = 'worker1'
|
415
432
|
SimpleJob.runs.should == 0
|
416
|
-
Delayed::Job.work_off
|
433
|
+
Delayed::Job.work_off :worker_name => 'worker1'
|
417
434
|
SimpleJob.runs.should == 3 # runs open job plus worker1 jobs that were already locked
|
418
435
|
end
|
419
436
|
end
|
420
437
|
|
421
438
|
context "while running with locked and expired jobs, it" do
|
422
439
|
before(:each) do
|
423
|
-
Delayed::Job.worker_name = 'worker1'
|
424
440
|
exp_time = Delayed::Job.db_time_now - (1.minutes + Delayed::Job::MAX_RUN_TIME)
|
425
441
|
Delayed::Job.create(:payload_object => SimpleJob.new, :locked_by => 'worker1', :locked_at => exp_time)
|
426
442
|
Delayed::Job.create(:payload_object => SimpleJob.new, :locked_by => 'worker2', :locked_at => (Delayed::Job.db_time_now - 1.minutes))
|
@@ -429,20 +445,19 @@ describe Delayed::Job do
|
|
429
445
|
end
|
430
446
|
|
431
447
|
it "should only find unlocked and expired jobs" do
|
432
|
-
Delayed::Job.worker_name = 'worker3'
|
433
448
|
SimpleJob.runs.should == 0
|
434
|
-
Delayed::Job.work_off
|
449
|
+
Delayed::Job.work_off :worker_name => 'worker3'
|
435
450
|
SimpleJob.runs.should == 2 # runs the one open job and one expired job
|
436
451
|
end
|
437
452
|
|
438
453
|
it "should ignore locks when finding our own jobs" do
|
439
|
-
Delayed::Job.worker_name = 'worker1'
|
440
454
|
SimpleJob.runs.should == 0
|
441
|
-
Delayed::Job.work_off
|
455
|
+
Delayed::Job.work_off :worker_name => 'worker1'
|
442
456
|
SimpleJob.runs.should == 3 # runs open job plus worker1 jobs
|
443
|
-
# This is useful in the case of a crash/restart on worker1,
|
457
|
+
# This is useful in the case of a crash/restart on worker1,
|
458
|
+
# but make sure multiple workers on the same host have unique names!
|
444
459
|
end
|
445
460
|
|
446
461
|
end
|
447
|
-
|
462
|
+
|
448
463
|
end
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: blaxter-delayed_job
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version:
|
4
|
+
version: 2.0.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- "Tobias L\xC3\xBCtke"
|
@@ -9,7 +9,7 @@ autorequire:
|
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
11
|
|
12
|
-
date: 2009-08-
|
12
|
+
date: 2009-08-10 00:00:00 -07:00
|
13
13
|
default_executable:
|
14
14
|
dependencies: []
|
15
15
|
|
@@ -19,13 +19,14 @@ executables: []
|
|
19
19
|
|
20
20
|
extensions: []
|
21
21
|
|
22
|
-
extra_rdoc_files:
|
23
|
-
|
22
|
+
extra_rdoc_files: []
|
23
|
+
|
24
24
|
files:
|
25
25
|
- .gitignore
|
26
26
|
- MIT-LICENSE
|
27
|
-
- README.
|
27
|
+
- README.markdown
|
28
28
|
- Rakefile
|
29
|
+
- VERSION
|
29
30
|
- delayed_job.gemspec
|
30
31
|
- generators/delayed_job/delayed_job_generator.rb
|
31
32
|
- generators/delayed_job/templates/migration.rb
|
data/README.textile
DELETED
@@ -1,118 +0,0 @@
|
|
1
|
-
h1. Delayed::Job
|
2
|
-
|
3
|
-
Delayed_job (or DJ) encapsulates the common pattern of asynchronously executing longer tasks in the background.
|
4
|
-
|
5
|
-
It is a direct extraction from Shopify where the job table is responsible for a multitude of core tasks. Amongst those tasks are:
|
6
|
-
|
7
|
-
* sending massive newsletters
|
8
|
-
* image resizing
|
9
|
-
* http downloads
|
10
|
-
* updating smart collections
|
11
|
-
* updating solr, our search server, after product changes
|
12
|
-
* batch imports
|
13
|
-
* spam checks
|
14
|
-
|
15
|
-
h2. Setup
|
16
|
-
|
17
|
-
The library evolves around a delayed_jobs table which looks as follows:
|
18
|
-
<pre><code>create_table :delayed_jobs, :force => true do |table|
|
19
|
-
table.integer :priority, :default => 0 # Allows some jobs to jump to the front of the queue
|
20
|
-
table.integer :attempts, :default => 0 # Provides for retries, but still fail eventually.
|
21
|
-
table.text :handler # YAML-encoded string of the object that will do work
|
22
|
-
table.string :job_type # Class name of the job object, for type-specific workers
|
23
|
-
table.string :last_error # reason for last failure (See Note below)
|
24
|
-
table.datetime :run_at # When to run. Could be Time.now for immediately, or sometime in the future.
|
25
|
-
table.datetime :locked_at # Set when a client is working on this object
|
26
|
-
table.datetime :failed_at # Set when all retries have failed (actually, by default, the record is deleted instead)
|
27
|
-
table.string :locked_by # Who is working on this object (if locked)
|
28
|
-
table.datetime :finished_at # Used for statiscics / monitoring
|
29
|
-
table.timestamps
|
30
|
-
end
|
31
|
-
</code></pre>
|
32
|
-
On failure, the job is scheduled again in 5 seconds + N ** 4, where N is the number of retries.
|
33
|
-
|
34
|
-
The default MAX_ATTEMPTS is 25- jobs can override this value by responding to :max_attempts. After this, the job
|
35
|
-
either deleted (default), or left in the database with "failed_at" set. With the default of 25 attempts,
|
36
|
-
the last retry will be 20 days later, with the last interval being almost 100 hours.
|
37
|
-
|
38
|
-
The default MAX_RUN_TIME is 4.hours. If your job takes longer than that, another computer could pick it up. It's up to you to
|
39
|
-
make sure your job doesn't exceed this time. You should set this to the longest time you think the job could take.
|
40
|
-
|
41
|
-
By default, it will delete failed jobs. If you want to keep failed jobs, set
|
42
|
-
@Delayed::Job.destroy_failed_jobs = false@. The failed jobs will be marked with non-null failed_at.
|
43
|
-
|
44
|
-
Same thing for successful jobs. They're deleted by default and, to keep them, set @Delayed::Job.destroy_successful_jobs = false@. They will be marked with finished_at. This is useful for gathering statistics like how long jobs took between entering the queue (created_at) and being finished (finished_at).
|
45
|
-
|
46
|
-
Here is an example of changing job parameters in Rails:
|
47
|
-
<pre><code># config/initializers/delayed_job_config.rb
|
48
|
-
Delayed::Job.destroy_failed_jobs = false
|
49
|
-
Delayed::Job.destroy_successful_jobs = false
|
50
|
-
silence_warnings do
|
51
|
-
Delayed::Job.const_set("MAX_ATTEMPTS", 3)
|
52
|
-
Delayed::Job.const_set("MAX_RUN_TIME", 5.minutes)
|
53
|
-
end
|
54
|
-
</code></pre>
|
55
|
-
Note: If your error messages are long, consider changing last_error field to a :text instead of a :string (255 character limit).
|
56
|
-
|
57
|
-
|
58
|
-
h2. Usage
|
59
|
-
|
60
|
-
Jobs are simple ruby objects with a method called perform. Any object which responds to perform can be stuffed into the jobs table.
|
61
|
-
Job objects are serialized to yaml so that they can later be resurrected by the job runner.
|
62
|
-
<pre><code>class NewsletterJob < Struct.new(:text, :emails)
|
63
|
-
def perform
|
64
|
-
emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
|
65
|
-
end
|
66
|
-
end
|
67
|
-
|
68
|
-
Delayed::Job.enqueue NewsletterJob.new('lorem ipsum...', Customers.find(:all).collect(&:email))
|
69
|
-
</code></pre>
|
70
|
-
There is also a second way to get jobs in the queue: send_later.
|
71
|
-
|
72
|
-
<pre><code>BatchImporter.new(Shop.find(1)).send_later(:import_massive_csv, massive_csv)
|
73
|
-
</code></pre>
|
74
|
-
|
75
|
-
This will simply create a Delayed::PerformableMethod job in the jobs table which serializes all the parameters you pass to it. There are some special smarts for active record objects
|
76
|
-
which are stored as their text representation and loaded from the database fresh when the job is actually run later.
|
77
|
-
|
78
|
-
|
79
|
-
h2. Running the jobs
|
80
|
-
|
81
|
-
You can invoke @rake jobs:work@ which will start working off jobs. You can cancel the rake task with @CTRL-C@.
|
82
|
-
|
83
|
-
You can also run by writing a simple @script/job_runner@, and invoking it externally:
|
84
|
-
|
85
|
-
<pre><code>#!/usr/bin/env ruby
|
86
|
-
require File.dirname(__FILE__) + '/../config/environment'
|
87
|
-
|
88
|
-
Delayed::Worker.new.start
|
89
|
-
</code></pre>
|
90
|
-
|
91
|
-
Workers can be running on any computer, as long as they have access to the database and their clock is in sync. You can even
|
92
|
-
run multiple workers on per computer, but you must give each one a unique name. (TODO: put in an example)
|
93
|
-
Keep in mind that each worker will check the database at least every 5 seconds.
|
94
|
-
|
95
|
-
Note: The rake task will exit if the database has any network connectivity problems.
|
96
|
-
|
97
|
-
If you only want to run specific types of jobs in a given worker, include them when initializing the worker:
|
98
|
-
|
99
|
-
<pre><code>
|
100
|
-
Delayed::Worker.new(:job_types => "SimpleJob").start
|
101
|
-
Delayed::Worker.new(:job_types => ["SimpleJob", "NewsletterJob"]).start
|
102
|
-
</pre></code>
|
103
|
-
|
104
|
-
h3. Cleaning up
|
105
|
-
|
106
|
-
You can invoke @rake jobs:clear@ to delete all jobs in the queue.
|
107
|
-
|
108
|
-
h3. Changes
|
109
|
-
|
110
|
-
* 1.7.0: Added failed_at column which can optionally be set after a certain amount of failed job attempts. By default failed job attempts are destroyed after about a month.
|
111
|
-
|
112
|
-
* 1.6.0: Renamed locked_until to locked_at. We now store when we start a given job instead of how long it will be locked by the worker. This allows us to get a reading on how long a job took to execute.
|
113
|
-
|
114
|
-
* 1.5.0: Job runners can now be run in parallel. Two new database columns are needed: locked_until and locked_by. This allows us to use pessimistic locking instead of relying on row level locks. This enables us to run as many worker processes as we need to speed up queue processing.
|
115
|
-
|
116
|
-
* 1.2.0: Added #send_later to Object for simpler job creation
|
117
|
-
|
118
|
-
* 1.0.0: Initial release
|