resque-status 0.2.4 → 0.3.0
Sign up to get free protection for your applications and to get access to all the features.
- data/Gemfile +8 -0
- data/Gemfile.lock +51 -0
- data/README.rdoc +30 -18
- data/Rakefile +6 -4
- data/examples/sleep_job.rb +9 -8
- data/init.rb +1 -1
- data/lib/resque-status.rb +1 -0
- data/lib/resque/job_with_status.rb +1 -198
- data/lib/resque/plugins/status.rb +213 -0
- data/lib/resque/plugins/status/hash.rb +242 -0
- data/lib/resque/server/views/status.erb +2 -2
- data/lib/resque/server/views/statuses.erb +10 -7
- data/lib/resque/status.rb +2 -237
- data/lib/resque/status_server.rb +23 -18
- data/resque-status.gemspec +34 -11
- data/test/test_helper.rb +16 -7
- data/test/{test_resque-job_with_status.rb → test_resque_plugins_status.rb} +94 -10
- data/test/test_resque_plugins_status_hash.rb +153 -0
- metadata +89 -20
- data/test/test_resque-status.rb +0 -153
data/Gemfile
ADDED
data/Gemfile.lock
ADDED
@@ -0,0 +1,51 @@
|
|
1
|
+
GEM
|
2
|
+
remote: http://rubygems.org/
|
3
|
+
specs:
|
4
|
+
git (1.2.5)
|
5
|
+
jeweler (1.6.4)
|
6
|
+
bundler (~> 1.0)
|
7
|
+
git (>= 1.2.5)
|
8
|
+
rake
|
9
|
+
json (1.4.6)
|
10
|
+
macaddr (1.5.0)
|
11
|
+
systemu (>= 2.4.0)
|
12
|
+
metaclass (0.0.1)
|
13
|
+
mocha (0.10.3)
|
14
|
+
metaclass (~> 0.0.1)
|
15
|
+
rack (1.4.0)
|
16
|
+
rack-protection (1.2.0)
|
17
|
+
rack
|
18
|
+
rake (0.9.2.2)
|
19
|
+
redis (2.2.2)
|
20
|
+
redis-namespace (1.1.0)
|
21
|
+
redis (< 3.0.0)
|
22
|
+
redisk (0.2.2)
|
23
|
+
redis (>= 0.1.1)
|
24
|
+
redis-namespace (>= 0.1.0)
|
25
|
+
resque (1.15.0)
|
26
|
+
json (~> 1.4.6)
|
27
|
+
redis-namespace (>= 0.10.0)
|
28
|
+
sinatra (>= 0.9.2)
|
29
|
+
vegas (~> 0.1.2)
|
30
|
+
shoulda (2.11.3)
|
31
|
+
sinatra (1.3.2)
|
32
|
+
rack (~> 1.3, >= 1.3.6)
|
33
|
+
rack-protection (~> 1.2)
|
34
|
+
tilt (~> 1.3, >= 1.3.3)
|
35
|
+
systemu (2.4.2)
|
36
|
+
tilt (1.3.3)
|
37
|
+
uuid (2.3.4)
|
38
|
+
macaddr (~> 1.0)
|
39
|
+
vegas (0.1.8)
|
40
|
+
rack (>= 1.0.0)
|
41
|
+
|
42
|
+
PLATFORMS
|
43
|
+
ruby
|
44
|
+
|
45
|
+
DEPENDENCIES
|
46
|
+
jeweler
|
47
|
+
mocha (>= 0.9.8)
|
48
|
+
redisk (>= 0.2.1)
|
49
|
+
resque (>= 1.3.1)
|
50
|
+
shoulda (>= 2.10.2)
|
51
|
+
uuid (>= 2.0.2)
|
data/README.rdoc
CHANGED
@@ -19,41 +19,53 @@ Install the resque-status gem (which will pull in the dependencies).
|
|
19
19
|
|
20
20
|
gem install resque-status
|
21
21
|
|
22
|
-
To use with Rails, you can install as a plugin or add the gem to you're config:
|
22
|
+
To use with Rails 2.x, you can install as a plugin or add the gem to you're config:
|
23
23
|
|
24
24
|
# environment.rb
|
25
|
-
config.gem 'resque-status'
|
26
|
-
|
25
|
+
config.gem 'resque-status'
|
26
|
+
|
27
|
+
With newer Rails add this to your Gemfile:
|
28
|
+
|
29
|
+
# Gemfile
|
30
|
+
gem 'resque-status'
|
31
|
+
|
27
32
|
Then in an initializer:
|
28
33
|
|
29
34
|
# config/initializers/resque.rb
|
30
|
-
require 'resque/job_with_status'
|
31
|
-
|
32
35
|
Resque.redis = "your/redis/socket" # default localhost:6379
|
33
|
-
Resque::Status.expire_in = (24 * 60 * 60) # 24hrs in seconds
|
36
|
+
Resque::Plugins::Status::Hash.expire_in = (24 * 60 * 60) # 24hrs in seconds
|
37
|
+
|
38
|
+
== NOTES ABOUT UPGRADING TO >= v0.3
|
39
|
+
|
40
|
+
Even though this was one of the first resque plugins, later versions of resque added stricter plugin conventions
|
41
|
+
that resque-status did not completely conform to (See: https://github.com/defunkt/resque/blob/master/docs/PLUGINS.md)
|
42
|
+
|
43
|
+
Thanks to some work from @EugZol and @bukowskis v0.3 moved around some code to conform:
|
44
|
+
|
45
|
+
`Resque::Status` is now `Resque::Plugins::Status` and is now an `include`able module.
|
46
|
+
`Resque::Status.get/etc` have been moved to `Resque::Plugins::Status::Hash`
|
34
47
|
|
35
48
|
== Usage
|
36
49
|
|
37
50
|
The most direct way to use resque-status is to create your jobs using the
|
38
|
-
Resque::
|
51
|
+
Resque::Plugins::Status module. An example job would look something like:
|
39
52
|
|
40
|
-
class SleepJob
|
53
|
+
class SleepJob
|
54
|
+
include Resque::Plugins::Status
|
41
55
|
|
42
56
|
def perform
|
43
|
-
total = options['length']
|
57
|
+
total = (options['length'] || 1000).to_i
|
44
58
|
num = 0
|
45
59
|
while num < total
|
46
60
|
at(num, total, "At #{num} of #{total}")
|
47
61
|
sleep(1)
|
48
62
|
num += 1
|
49
63
|
end
|
50
|
-
completed
|
51
64
|
end
|
52
65
|
|
53
66
|
end
|
54
67
|
|
55
|
-
|
56
|
-
Another major difference is that intead of implementing <tt>perform</tt> as a
|
68
|
+
One major difference is that intead of implementing <tt>perform</tt> as a
|
57
69
|
class method, we do our job implementation within instances of the job class.
|
58
70
|
|
59
71
|
In order to queue a SleepJob up, we also won't use <tt>Resque.enqueue</tt>, instead
|
@@ -67,9 +79,9 @@ instance as options['length'] (as you can see above).
|
|
67
79
|
|
68
80
|
Now that we have a UUID its really easy to get the status:
|
69
81
|
|
70
|
-
status = Resque::Status.get(job_id)
|
82
|
+
status = Resque::Plugins::Status::Hash.get(job_id)
|
71
83
|
|
72
|
-
This returns a Resque::Status object, which is a Hash (with benefits).
|
84
|
+
This returns a Resque::Plugins::Status::Hash object, which is a Hash (with benefits).
|
73
85
|
|
74
86
|
status.pct_complete #=> 0
|
75
87
|
status.status #=> 'queued'
|
@@ -81,7 +93,7 @@ This returns a Resque::Status object, which is a Hash (with benefits).
|
|
81
93
|
Once the worker reserves the job, the instance of SleepJob updates the status at
|
82
94
|
each iteration using <tt>at()</tt>
|
83
95
|
|
84
|
-
status = Resque::Status.get(job_id)
|
96
|
+
status = Resque::Plugins::Status::Hash.get(job_id)
|
85
97
|
status.working? #=> true
|
86
98
|
status.num #=> 5
|
87
99
|
status.total => 100
|
@@ -92,7 +104,7 @@ the error is re-raised so that Resque can capture it.
|
|
92
104
|
|
93
105
|
Its also possible to get a list of current/recent job statuses:
|
94
106
|
|
95
|
-
Resque::Status.statuses #=> [#<Resque::Status>, ...]
|
107
|
+
Resque::Plugins::Status::Hash.statuses #=> [#<Resque::Plugins::Status::Hash>, ...]
|
96
108
|
|
97
109
|
=== Passing back data from the job
|
98
110
|
|
@@ -115,7 +127,7 @@ Because we're tracking UUIDs per instance, and we're checking in/updating the st
|
|
115
127
|
on each iteration (using <tt>at</tt> or <tt>tick</tt>) we can kill specific jobs
|
116
128
|
by UUID.
|
117
129
|
|
118
|
-
Resque::Status.kill(job_id)
|
130
|
+
Resque::Plugins::Status::Hash.kill(job_id)
|
119
131
|
|
120
132
|
The next time the job at job_id calls <tt>at</tt> or tick, it will raise a Killed
|
121
133
|
error and set the status to killed.
|
@@ -126,7 +138,7 @@ Since Redis is RAM based, we probably don't want to keep these statuses around f
|
|
126
138
|
(at least until @antirez releases the VM feature). By setting expire_in, all statuses
|
127
139
|
and thier related keys will expire in expire_in seconds from the last time theyre updated:
|
128
140
|
|
129
|
-
Resque::Status.expire_in = (60 * 60) # 1 hour
|
141
|
+
Resque::Plugins::Status::Hash.expire_in = (60 * 60) # 1 hour
|
130
142
|
|
131
143
|
=== resque-web
|
132
144
|
|
data/Rakefile
CHANGED
@@ -1,15 +1,17 @@
|
|
1
|
+
$LOAD_PATH.unshift './lib'
|
2
|
+
|
1
3
|
require 'rubygems'
|
2
4
|
require 'rake'
|
3
|
-
require
|
5
|
+
require 'resque-status'
|
4
6
|
require 'resque/tasks'
|
5
7
|
|
6
8
|
begin
|
7
9
|
require 'jeweler'
|
8
10
|
Jeweler::Tasks.new do |gem|
|
9
11
|
gem.name = "resque-status"
|
10
|
-
gem.version = Resque::Status::VERSION
|
12
|
+
gem.version = Resque::Plugins::Status::VERSION
|
11
13
|
gem.summary = %Q{resque-status is an extension to the resque queue system that provides simple trackable jobs.}
|
12
|
-
gem.description = %Q{resque-status is an extension to the resque queue system that provides simple trackable jobs. It provides a Resque::Status class which can set/get the statuses of jobs and a Resque::
|
14
|
+
gem.description = %Q{resque-status is an extension to the resque queue system that provides simple trackable jobs. It provides a Resque::Plugins::Status::Hash class which can set/get the statuses of jobs and a Resque::Plugins::Status class that when included provides easily trackable/killable jobs.}
|
13
15
|
gem.email = "aaron@quirkey.com"
|
14
16
|
gem.homepage = "http://github.com/quirkey/resque-status"
|
15
17
|
gem.rubyforge_project = "quirkey"
|
@@ -46,7 +48,7 @@ rescue LoadError
|
|
46
48
|
end
|
47
49
|
end
|
48
50
|
|
49
|
-
task :test
|
51
|
+
task :test
|
50
52
|
|
51
53
|
task :default => :test
|
52
54
|
|
data/examples/sleep_job.rb
CHANGED
@@ -2,10 +2,11 @@ require 'resque/job_with_status' # in rails you would probably do this in an ini
|
|
2
2
|
|
3
3
|
# sleeps for _length_ seconds updating the status every second
|
4
4
|
|
5
|
-
class SleepJob
|
6
|
-
|
5
|
+
class SleepJob
|
6
|
+
include Resque::Plugins::Status
|
7
|
+
|
7
8
|
def perform
|
8
|
-
total = options['length'].to_i
|
9
|
+
total = options.has_key?('length') ? options['length'].to_i : 1000
|
9
10
|
num = 0
|
10
11
|
while num < total
|
11
12
|
at(num, total, "At #{num} of #{total}")
|
@@ -14,22 +15,22 @@ class SleepJob < Resque::JobWithStatus
|
|
14
15
|
end
|
15
16
|
completed
|
16
17
|
end
|
17
|
-
|
18
|
+
|
18
19
|
end
|
19
20
|
|
20
21
|
|
21
22
|
if __FILE__ == $0
|
22
23
|
# Make sure you have a worker running
|
23
24
|
# rake -rexamples/sleep_job.rb resque:work QUEUE=statused
|
24
|
-
|
25
|
+
|
25
26
|
# running the job
|
26
27
|
puts "Creating the SleepJob"
|
27
28
|
job_id = SleepJob.create :length => 100
|
28
29
|
puts "Got back #{job_id}"
|
29
|
-
|
30
|
+
|
30
31
|
# check the status until its complete
|
31
|
-
while status = Resque::Status.get(job_id) and !status.completed? && !status.failed?
|
32
|
+
while status = Resque::Plugins::Status::Hash.get(job_id) and !status.completed? && !status.failed?
|
32
33
|
sleep 1
|
33
34
|
puts status.inspect
|
34
35
|
end
|
35
|
-
end
|
36
|
+
end
|
data/init.rb
CHANGED
@@ -1 +1 @@
|
|
1
|
-
require 'resque
|
1
|
+
require 'resque-status'
|
@@ -0,0 +1 @@
|
|
1
|
+
require 'resque/status'
|
@@ -1,202 +1,5 @@
|
|
1
|
-
require 'resque/status'
|
2
|
-
|
3
1
|
module Resque
|
4
|
-
|
5
|
-
# JobWithStatus is a base class that you're jobs will inherit from.
|
6
|
-
# It provides helper methods for updating the status/etc from within an
|
7
|
-
# instance as well as class methods for creating and queuing the jobs.
|
8
|
-
#
|
9
|
-
# All you have to do to get this functionality is inherit from JobWithStatus
|
10
|
-
# and then implement a <tt>perform<tt> method.
|
11
|
-
#
|
12
|
-
# For example:
|
13
|
-
#
|
14
|
-
# class ExampleJob < Resque::JobWithStatus
|
15
|
-
#
|
16
|
-
# def perform
|
17
|
-
# num = options['num']
|
18
|
-
# i = 0
|
19
|
-
# while i < num
|
20
|
-
# i += 1
|
21
|
-
# at(i, num)
|
22
|
-
# end
|
23
|
-
# completed("Finished!")
|
24
|
-
# end
|
25
|
-
#
|
26
|
-
# end
|
27
|
-
#
|
28
|
-
# This job would iterate num times updating the status as it goes. At the end
|
29
|
-
# we update the status telling anyone listening to this job that its complete.
|
30
2
|
class JobWithStatus
|
31
|
-
|
32
|
-
# The error class raised when a job is killed
|
33
|
-
class Killed < RuntimeError; end
|
34
|
-
|
35
|
-
attr_reader :uuid, :options
|
36
|
-
|
37
|
-
# The default queue is :statused, this can be ovveridden in the specific job
|
38
|
-
# class to put the jobs on a specific worker queue
|
39
|
-
def self.queue
|
40
|
-
:statused
|
41
|
-
end
|
42
|
-
|
43
|
-
# used when displaying the Job in the resque-web UI and identifiyng the job
|
44
|
-
# type by status. By default this is the name of the job class, but can be
|
45
|
-
# ovveridden in the specific job class to present a more user friendly job
|
46
|
-
# name
|
47
|
-
def self.name
|
48
|
-
self.to_s
|
49
|
-
end
|
50
|
-
|
51
|
-
# Create is the primary method for adding jobs to the queue. This would be
|
52
|
-
# called on the job class to create a job of that type. Any options passed are
|
53
|
-
# passed to the Job instance as a hash of options. It returns the UUID of the
|
54
|
-
# job.
|
55
|
-
#
|
56
|
-
# == Example:
|
57
|
-
#
|
58
|
-
# class ExampleJob < Resque::JobWithStatus
|
59
|
-
#
|
60
|
-
# def perform
|
61
|
-
# set_status "Hey I'm a job num #{options['num']}"
|
62
|
-
# end
|
63
|
-
#
|
64
|
-
# end
|
65
|
-
#
|
66
|
-
# job_id = ExampleJob.create(:num => 100)
|
67
|
-
#
|
68
|
-
def self.create(options = {})
|
69
|
-
self.enqueue(self, options)
|
70
|
-
end
|
71
|
-
|
72
|
-
# Adds a job of type <tt>klass<tt> to the queue with <tt>options<tt>.
|
73
|
-
# Returns the UUID of the job
|
74
|
-
def self.enqueue(klass, options = {})
|
75
|
-
uuid = Resque::Status.create :options => options
|
76
|
-
Resque.enqueue(klass, uuid, options)
|
77
|
-
uuid
|
78
|
-
end
|
79
|
-
|
80
|
-
# This is the method called by Resque::Worker when processing jobs. It
|
81
|
-
# creates a new instance of the job class and populates it with the uuid and
|
82
|
-
# options.
|
83
|
-
#
|
84
|
-
# You should not override this method, rahter the <tt>perform</tt> instance method.
|
85
|
-
def self.perform(uuid=nil, options = {})
|
86
|
-
uuid ||= Resque::Status.generate_uuid
|
87
|
-
instance = new(uuid, options)
|
88
|
-
instance.safe_perform!
|
89
|
-
instance
|
90
|
-
end
|
91
|
-
|
92
|
-
# Wrapper API to forward a Resque::Job creation API call into a JobWithStatus call.
|
93
|
-
# This is needed to be used with resque scheduler
|
94
|
-
# http://github.com/bvandenbos/resque-scheduler
|
95
|
-
def self.scheduled(queue, klass, *args)
|
96
|
-
create(*args)
|
97
|
-
end
|
98
|
-
|
99
|
-
# Create a new instance with <tt>uuid</tt> and <tt>options</tt>
|
100
|
-
def initialize(uuid, options = {})
|
101
|
-
@uuid = uuid
|
102
|
-
@options = options
|
103
|
-
end
|
104
|
-
|
105
|
-
# Run by the Resque::Worker when processing this job. It wraps the <tt>perform</tt>
|
106
|
-
# method ensuring that the final status of the job is set regardless of error.
|
107
|
-
# If an error occurs within the job's work, it will set the status as failed and
|
108
|
-
# re-raise the error.
|
109
|
-
def safe_perform!
|
110
|
-
set_status({'status' => 'working'})
|
111
|
-
perform
|
112
|
-
completed unless status && status.completed?
|
113
|
-
on_success if respond_to?(:on_success)
|
114
|
-
rescue Killed
|
115
|
-
logger.info "Job #{self} Killed at #{Time.now}"
|
116
|
-
Resque::Status.killed(uuid)
|
117
|
-
on_killed if respond_to?(:on_killed)
|
118
|
-
rescue => e
|
119
|
-
logger.error e
|
120
|
-
failed("The task failed because of an error: #{e}")
|
121
|
-
if respond_to?(:on_failure)
|
122
|
-
on_failure(e)
|
123
|
-
else
|
124
|
-
raise e
|
125
|
-
end
|
126
|
-
end
|
127
|
-
|
128
|
-
# Returns a Redisk::Logger object scoped to this paticular job/uuid
|
129
|
-
def logger
|
130
|
-
@logger ||= Resque::Status.logger(uuid)
|
131
|
-
end
|
132
|
-
|
133
|
-
# Set the jobs status. Can take an array of strings or hashes that are merged
|
134
|
-
# (in order) into a final status hash.
|
135
|
-
def status=(new_status)
|
136
|
-
Resque::Status.set(uuid, *new_status)
|
137
|
-
end
|
138
|
-
|
139
|
-
# get the Resque::Status object for the current uuid
|
140
|
-
def status
|
141
|
-
Resque::Status.get(uuid)
|
142
|
-
end
|
143
|
-
|
144
|
-
def name
|
145
|
-
"#{self.class.name}(#{options.inspect unless options.empty?})"
|
146
|
-
end
|
147
|
-
|
148
|
-
# Checks against the kill list if this specific job instance should be killed
|
149
|
-
# on the next iteration
|
150
|
-
def should_kill?
|
151
|
-
Resque::Status.should_kill?(uuid)
|
152
|
-
end
|
153
|
-
|
154
|
-
# set the status of the job for the current itteration. <tt>num</tt> and
|
155
|
-
# <tt>total</tt> are passed to the status as well as any messages.
|
156
|
-
# This will kill the job if it has been added to the kill list with
|
157
|
-
# <tt>Resque::Status.kill()</tt>
|
158
|
-
def at(num, total, *messages)
|
159
|
-
tick({
|
160
|
-
'num' => num,
|
161
|
-
'total' => total
|
162
|
-
}, *messages)
|
163
|
-
end
|
164
|
-
|
165
|
-
# sets the status of the job for the current itteration. You should use
|
166
|
-
# the <tt>at</tt> method if you have actual numbers to track the iteration count.
|
167
|
-
# This will kill the job if it has been added to the kill list with
|
168
|
-
# <tt>Resque::Status.kill()</tt>
|
169
|
-
def tick(*messages)
|
170
|
-
kill! if should_kill?
|
171
|
-
set_status({'status' => 'working'}, *messages)
|
172
|
-
end
|
173
|
-
|
174
|
-
# set the status to 'failed' passing along any additional messages
|
175
|
-
def failed(*messages)
|
176
|
-
set_status({'status' => 'failed'}, *messages)
|
177
|
-
end
|
178
|
-
|
179
|
-
# set the status to 'completed' passing along any addional messages
|
180
|
-
def completed(*messages)
|
181
|
-
set_status({
|
182
|
-
'status' => 'completed',
|
183
|
-
'message' => "Completed at #{Time.now}"
|
184
|
-
}, *messages)
|
185
|
-
end
|
186
|
-
|
187
|
-
# kill the current job, setting the status to 'killed' and raising <tt>Killed</tt>
|
188
|
-
def kill!
|
189
|
-
set_status({
|
190
|
-
'status' => 'killed',
|
191
|
-
'message' => "Killed at #{Time.now}"
|
192
|
-
})
|
193
|
-
raise Killed
|
194
|
-
end
|
195
|
-
|
196
|
-
private
|
197
|
-
def set_status(*args)
|
198
|
-
self.status = [status, {'name' => self.name}, args].flatten
|
199
|
-
end
|
200
|
-
|
3
|
+
include Resque::Plugins::Status
|
201
4
|
end
|
202
5
|
end
|