mongo-resque 1.17.1
Sign up to get free protection for your applications and to get access to all the features.
- data/HISTORY.md +278 -0
- data/LICENSE +20 -0
- data/README.markdown +76 -0
- data/bin/resque +67 -0
- data/bin/resque-web +23 -0
- data/docs/HOOKS.md +132 -0
- data/docs/PLUGINS.md +93 -0
- data/lib/resque/errors.rb +13 -0
- data/lib/resque/failure/base.rb +64 -0
- data/lib/resque/failure/hoptoad.rb +133 -0
- data/lib/resque/failure/mongo.rb +45 -0
- data/lib/resque/failure/multiple.rb +54 -0
- data/lib/resque/failure/redis.rb +46 -0
- data/lib/resque/failure.rb +70 -0
- data/lib/resque/helpers.rb +75 -0
- data/lib/resque/job.rb +215 -0
- data/lib/resque/plugin.rb +51 -0
- data/lib/resque/server/public/favicon.ico +0 -0
- data/lib/resque/server/public/idle.png +0 -0
- data/lib/resque/server/public/jquery-1.3.2.min.js +19 -0
- data/lib/resque/server/public/jquery.relatize_date.js +95 -0
- data/lib/resque/server/public/poll.png +0 -0
- data/lib/resque/server/public/ranger.js +73 -0
- data/lib/resque/server/public/reset.css +48 -0
- data/lib/resque/server/public/style.css +92 -0
- data/lib/resque/server/public/working.png +0 -0
- data/lib/resque/server/test_helper.rb +19 -0
- data/lib/resque/server/views/error.erb +1 -0
- data/lib/resque/server/views/failed.erb +65 -0
- data/lib/resque/server/views/key_sets.erb +19 -0
- data/lib/resque/server/views/key_string.erb +11 -0
- data/lib/resque/server/views/layout.erb +42 -0
- data/lib/resque/server/views/next_more.erb +10 -0
- data/lib/resque/server/views/overview.erb +4 -0
- data/lib/resque/server/views/queues.erb +69 -0
- data/lib/resque/server/views/stats.erb +73 -0
- data/lib/resque/server/views/workers.erb +109 -0
- data/lib/resque/server/views/working.erb +72 -0
- data/lib/resque/server.rb +268 -0
- data/lib/resque/stat.rb +54 -0
- data/lib/resque/tasks.rb +42 -0
- data/lib/resque/version.rb +3 -0
- data/lib/resque/worker.rb +497 -0
- data/lib/resque.rb +417 -0
- data/lib/tasks/resque.rake +2 -0
- metadata +148 -0
data/HISTORY.md
ADDED
@@ -0,0 +1,278 @@
|
|
1
|
+
## 1.17.1 (2011-05-30)
|
2
|
+
|
3
|
+
* More monit typo fixes
|
4
|
+
* Web UI: Wrap preformatted arguments
|
5
|
+
|
6
|
+
## 1.16.1 (2011-05-18)
|
7
|
+
|
8
|
+
Exlusive to resque-mongo:
|
9
|
+
* BREAKING CHANGE! Resque.mongo now expects a Mongo::DB object and nothing else
|
10
|
+
* Minor api cleanups (removed undocumented methods to get closer to original resque)
|
11
|
+
* Did NOT make the switch to use hoptoad_notifier for the hoptoad failure backend
|
12
|
+
|
13
|
+
From original resque:
|
14
|
+
* Added `Worker#paused?` method
|
15
|
+
* Bugfix: Properly reseed random number generator after forking.
|
16
|
+
* Bugfix: Monit example stdout/stderr redirection
|
17
|
+
* Bugfix: Removing single failure now works with multiple failure backends
|
18
|
+
* Web: 'Remove Queue' now requires confirmation
|
19
|
+
* Web: Favicon!
|
20
|
+
* Web Bugfix: Dates display in Safari
|
21
|
+
* Web Bugfix: Dates display timezone
|
22
|
+
* Web Bugfix: Race condition querying working workers
|
23
|
+
* Web Bugfix: Fix polling /workers/all in resque-web
|
24
|
+
|
25
|
+
## 1.15.0 (2011-03-18)
|
26
|
+
|
27
|
+
* Fixed Sinatra 1.2 compatibility
|
28
|
+
|
29
|
+
## 1.14.0 (2011-03-17)
|
30
|
+
|
31
|
+
* Sleep interval can now be a float
|
32
|
+
* Added Resque.inline to allow in-process performing of jobs (for testing)
|
33
|
+
* Fixed tests for Ruby 1.9.2
|
34
|
+
* Added Resque.validate(klass) to validate a Job
|
35
|
+
* Decode errors are no longer ignored to help debugging
|
36
|
+
* Web: Sinatra 1.2 compatibility
|
37
|
+
* Fixed after_enqueue hook to actually run in `Resque.enqueue`
|
38
|
+
* Fixed very_verbose timestamps to use 24 hour time (AM/PM wasn't included)
|
39
|
+
* Fixed monit example
|
40
|
+
* Fixed Worker#pid
|
41
|
+
|
42
|
+
## 1.13.0 (2011-02-07)
|
43
|
+
|
44
|
+
* Bugfix: worker pruning
|
45
|
+
|
46
|
+
## 1.12.0 (2011-02-03)
|
47
|
+
|
48
|
+
* Added pidfile writing from `rake resque:work`
|
49
|
+
* Added Worker#pid method
|
50
|
+
* Bugfix: Errors in failure backend are rescue'd
|
51
|
+
* Bugfix: Non-working workers no longer counted in "working" count
|
52
|
+
* Bugfix: Don't think resque-web is a worker
|
53
|
+
|
54
|
+
## 1.11.0 (2010-08-23)
|
55
|
+
|
56
|
+
* Web UI: Group /workers page by hostnames
|
57
|
+
|
58
|
+
## 1.10.0 (2010-08-23)
|
59
|
+
|
60
|
+
* Support redis:// string format in `Resque.redis=`
|
61
|
+
* Using new cross-platform JSON gem.
|
62
|
+
* Added `after_enqueue` plugin hook.
|
63
|
+
* Added `shutdown?` method which can be overridden.
|
64
|
+
* Added support for the "leftright" gem when running tests.
|
65
|
+
* Grammarfix: In the README
|
66
|
+
|
67
|
+
## 1.9.10 (2010-08-06)
|
68
|
+
|
69
|
+
* Bugfix: before_fork should get passed the job
|
70
|
+
|
71
|
+
## 1.9.9 (2010-07-26)
|
72
|
+
|
73
|
+
* Depend on redis-namespace 0.8.0
|
74
|
+
* Depend on json_pure instead of json (for JRuby compat)
|
75
|
+
* Bugfix: rails_env display in stats view
|
76
|
+
|
77
|
+
## 1.9.8 (2010-07-20)
|
78
|
+
|
79
|
+
* Bugfix: Worker.all should never return nil
|
80
|
+
* monit example: Fixed Syntax Error and adding environment to the rake task
|
81
|
+
* redis rake task: Fixed typo in copy command
|
82
|
+
|
83
|
+
## 1.9.7 (2010-07-09)
|
84
|
+
|
85
|
+
* Improved memory usage in Job.destroy
|
86
|
+
* redis-namespace 0.7.0 now required
|
87
|
+
* Bugfix: Reverted $0 changes
|
88
|
+
* Web Bugfix: Payload-less failures in the web ui work
|
89
|
+
|
90
|
+
## 1.9.6 (2010-06-22)
|
91
|
+
|
92
|
+
* Bugfix: Rakefile logging works the same as all the other logging
|
93
|
+
|
94
|
+
## 1.9.5 (2010-06-16)
|
95
|
+
|
96
|
+
* Web Bugfix: Display the configured namespace on the stats page
|
97
|
+
* Revert Bugfix: Make ps -o more cross platform friendly
|
98
|
+
|
99
|
+
## 1.9.4 (2010-06-14)
|
100
|
+
|
101
|
+
* Bugfix: Multiple failure backend gets exception information when created
|
102
|
+
|
103
|
+
## 1.9.3 (2010-06-14)
|
104
|
+
|
105
|
+
* Bugfix: Resque#queues always returns an array
|
106
|
+
|
107
|
+
## 1.9.2 (2010-06-13)
|
108
|
+
|
109
|
+
* Bugfix: Worker.all returning nil fix
|
110
|
+
* Bugfix: Make ps -o more cross platform friendly
|
111
|
+
|
112
|
+
## 1.9.1 (2010-06-04)
|
113
|
+
|
114
|
+
* Less strict JSON dependency
|
115
|
+
* Included HISTORY.md in gem
|
116
|
+
|
117
|
+
## 1.9.0 (2010-06-04)
|
118
|
+
|
119
|
+
* Redis 2 support
|
120
|
+
* Depend on redis-namespace 0.5.0
|
121
|
+
* Added Resque::VERSION constant (alias of Resque::Version)
|
122
|
+
* Bugfix: Specify JSON dependency
|
123
|
+
* Bugfix: Hoptoad plugin now works on 1.9
|
124
|
+
|
125
|
+
## 1.8.5 (2010-05-18)
|
126
|
+
|
127
|
+
* Bugfix: Be more liberal in which Redis clients we accept.
|
128
|
+
|
129
|
+
## 1.8.4 (2010-05-18)
|
130
|
+
|
131
|
+
* Try to resolve redis-namespace dependency issue
|
132
|
+
|
133
|
+
## 1.8.3 (2010-05-17)
|
134
|
+
|
135
|
+
* Depend on redis-rb ~> 1.0.7
|
136
|
+
|
137
|
+
## 1.8.2 (2010-05-03)
|
138
|
+
|
139
|
+
* Bugfix: Include "tasks/" dir in RubyGem
|
140
|
+
|
141
|
+
## 1.8.1 (2010-04-29)
|
142
|
+
|
143
|
+
* Bugfix: Multiple failure backend did not support requeue-ing failed jobs
|
144
|
+
* Bugfix: Fix /failed when error has no backtrace
|
145
|
+
* Bugfix: Add `Redis::DistRedis` as a valid client
|
146
|
+
|
147
|
+
## 1.8.0 (2010-04-07)
|
148
|
+
|
149
|
+
* Jobs that never complete due to killed worker are now failed.
|
150
|
+
* Worker "working" state is now maintained by the parent, not the child.
|
151
|
+
* Stopped using deprecated redis.rb methods
|
152
|
+
* `Worker.working` race condition fixed
|
153
|
+
* `Worker#process` has been deprecated.
|
154
|
+
* Monit example fixed
|
155
|
+
* Redis::Client and Redis::Namespace can be passed to `Resque.redis=`
|
156
|
+
|
157
|
+
## 1.7.1 (2010-04-02)
|
158
|
+
|
159
|
+
* Bugfix: Make job hook execution order consistent
|
160
|
+
* Bugfix: stdout buffering in child process
|
161
|
+
|
162
|
+
## 1.7.0 (2010-03-31)
|
163
|
+
|
164
|
+
* Job hooks API. See docs/HOOKS.md.
|
165
|
+
* web: Hovering over dates shows a timestamp
|
166
|
+
* web: AJAXify retry action for failed jobs
|
167
|
+
* web bugfix: Fix pagination bug
|
168
|
+
|
169
|
+
## 1.6.1 (2010-03-25)
|
170
|
+
|
171
|
+
* Bugfix: Workers may not be clearing their state correctly on
|
172
|
+
shutdown
|
173
|
+
* Added example monit config.
|
174
|
+
* Exception class is now recorded when an error is raised in a
|
175
|
+
worker.
|
176
|
+
* web: Unit tests
|
177
|
+
* web: Show namespace in header and footer
|
178
|
+
* web: Remove a queue
|
179
|
+
* web: Retry failed jobs
|
180
|
+
|
181
|
+
## 1.6.0 (2010-03-09)
|
182
|
+
|
183
|
+
* Added `before_first_fork`, `before_fork`, and `after_fork` hooks.
|
184
|
+
* Hoptoad: Added server_environment config setting
|
185
|
+
* Hoptoad bugfix: Don't depend on RAILS_ROOT
|
186
|
+
* 1.8.6 compat fixes
|
187
|
+
|
188
|
+
## 1.5.2 (2010-03-03)
|
189
|
+
|
190
|
+
* Bugfix: JSON check was crazy.
|
191
|
+
|
192
|
+
## 1.5.1 (2010-03-03)
|
193
|
+
|
194
|
+
* `Job.destroy` and `Resque.dequeue` return the # of destroyed jobs.
|
195
|
+
* Hoptoad notifier improvements
|
196
|
+
* Specify the namespace with `resque-web` by passing `-N namespace`
|
197
|
+
* Bugfix: Don't crash when trying to parse invalid JSON.
|
198
|
+
* Bugfix: Non-standard namespace support
|
199
|
+
* Web: Red backgound for queue "failed" only shown if there are failed jobs.
|
200
|
+
* Web bugfix: Tabs highlight properly now
|
201
|
+
* Web bugfix: ZSET partial support in stats
|
202
|
+
* Web bugfix: Deleting failed jobs works again
|
203
|
+
* Web bugfix: Sets (or zsets, lists, etc) now paginate.
|
204
|
+
|
205
|
+
## 1.5.0 (2010-02-17)
|
206
|
+
|
207
|
+
* Version now included in procline, e.g. `resque-1.5.0: Message`
|
208
|
+
* Web bugfix: Ignore idle works in the "working" page
|
209
|
+
* Added `Resque::Job.destroy(queue, klass, *args)`
|
210
|
+
* Added `Resque.dequeue(klass, *args)`
|
211
|
+
|
212
|
+
## 1.4.0 (2010-02-11)
|
213
|
+
|
214
|
+
* Fallback when unable to bind QUIT and USR1 for Windows and JRuby.
|
215
|
+
* Fallback when no `Kernel.fork` is provided (for IronRuby).
|
216
|
+
* Web: Rounded corners in Firefox
|
217
|
+
* Cut down system calls in `Worker#prune_dead_workers`
|
218
|
+
* Enable switching DB in a Redis server from config
|
219
|
+
* Support USR2 and CONT to stop and start job processing.
|
220
|
+
* Web: Add example failing job
|
221
|
+
* Bugfix: `Worker#unregister_worker` shouldn't call `done_working`
|
222
|
+
* Bugfix: Example god config now restarts Resque properly.
|
223
|
+
* Multiple failure backends now permitted.
|
224
|
+
* Hoptoad failure backend updated to new API
|
225
|
+
|
226
|
+
## 1.3.1 (2010-01-11)
|
227
|
+
|
228
|
+
* Vegas bugfix: Don't error without a config
|
229
|
+
|
230
|
+
## 1.3.0 (2010-01-11)
|
231
|
+
|
232
|
+
* Use Vegas for resque-web
|
233
|
+
* Web Bugfix: Show proper date/time value for failed_at on Failures
|
234
|
+
* Web Bugfix: Make the / route more flexible
|
235
|
+
* Add Resque::Server.tabs array (so plugins can add their own tabs)
|
236
|
+
* Start using [Semantic Versioning](http://semver.org/)
|
237
|
+
|
238
|
+
## 1.2.4 (2009-12-15)
|
239
|
+
|
240
|
+
* Web Bugfix: fix key links on stat page
|
241
|
+
|
242
|
+
## 1.2.3 (2009-12-15)
|
243
|
+
|
244
|
+
* Bugfix: Fixed `rand` seeding in child processes.
|
245
|
+
* Bugfix: Better JSON encoding/decoding without Yajl.
|
246
|
+
* Bugfix: Avoid `ps` flag error on Linux
|
247
|
+
* Add `PREFIX` observance to `rake` install tasks.
|
248
|
+
|
249
|
+
## 1.2.2 (2009-12-08)
|
250
|
+
|
251
|
+
* Bugfix: Job equality was not properly implemented.
|
252
|
+
|
253
|
+
## 1.2.1 (2009-12-07)
|
254
|
+
|
255
|
+
* Added `rake resque:workers` task for starting multiple workers.
|
256
|
+
* 1.9.x compatibility
|
257
|
+
* Bugfix: Yajl decoder doesn't care about valid UTF-8
|
258
|
+
* config.ru loads RESQUECONFIG if the ENV variable is set.
|
259
|
+
* `resque-web` now sets RESQUECONFIG
|
260
|
+
* Job objects know if they are equal.
|
261
|
+
* Jobs can be re-queued using `Job#recreate`
|
262
|
+
|
263
|
+
## 1.2.0 (2009-11-25)
|
264
|
+
|
265
|
+
* If USR1 is sent and no child is found, shutdown.
|
266
|
+
* Raise when a job class does not respond to `perform`.
|
267
|
+
* Added `Resque.remove_queue` for deleting a queue
|
268
|
+
|
269
|
+
## 1.1.0 (2009-11-04)
|
270
|
+
|
271
|
+
* Bugfix: Broken ERB tag in failure UI
|
272
|
+
* Bugfix: Save the worker's ID, not the worker itself, in the failure module
|
273
|
+
* Redesigned the sinatra web interface
|
274
|
+
* Added option to clear failed jobs
|
275
|
+
|
276
|
+
## 1.0.0 (2009-11-03)
|
277
|
+
|
278
|
+
* First release.
|
data/LICENSE
ADDED
@@ -0,0 +1,20 @@
|
|
1
|
+
Copyright (c) 2009 Chris Wanstrath
|
2
|
+
|
3
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
4
|
+
a copy of this software and associated documentation files (the
|
5
|
+
"Software"), to deal in the Software without restriction, including
|
6
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
7
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
8
|
+
permit persons to whom the Software is furnished to do so, subject to
|
9
|
+
the following conditions:
|
10
|
+
|
11
|
+
The above copyright notice and this permission notice shall be
|
12
|
+
included in all copies or substantial portions of the Software.
|
13
|
+
|
14
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
15
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
16
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
17
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
18
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
19
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
20
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.markdown
ADDED
@@ -0,0 +1,76 @@
|
|
1
|
+
Mongo-Resque
|
2
|
+
============
|
3
|
+
|
4
|
+
Mongo-Resque is a fork of Resque based on MongoDB instead of Redis.
|
5
|
+
|
6
|
+
All work on this project is sponsored by the online video platform [Streamio](http://streamio.com).
|
7
|
+
|
8
|
+
[![Streamio](http://d253c4ja9jigvu.cloudfront.net/assets/small-logo.png)](http://streamio.com)
|
9
|
+
|
10
|
+
Check out the [ORIGINAL_README][0] included in this repository
|
11
|
+
for the general Resque lowdown.
|
12
|
+
|
13
|
+
What did you guys do to Resque?
|
14
|
+
===============================
|
15
|
+
|
16
|
+
Apart from transparently replacing the redis backend with mongodb
|
17
|
+
(where each queue has a corresponding mongo collection) this fork
|
18
|
+
also features delayed jobs if you want to schedule your jobs.
|
19
|
+
|
20
|
+
Note that this fork is different from IGO's fork in that queue
|
21
|
+
collections are namespaced with 'resque.queues.' to make it possible
|
22
|
+
to use the same database for your application as Resque (this might
|
23
|
+
still not be the best idea though - behold the stern warnings below).
|
24
|
+
|
25
|
+
Original resque is currently using hoptoad_notifier in its Hoptoad Failure
|
26
|
+
Backend. This fork has not implemented this change as I'm undecided wether
|
27
|
+
the change was for the better or not (we avoid dependency troubles this way).
|
28
|
+
|
29
|
+
Delayed Jobs
|
30
|
+
------------
|
31
|
+
|
32
|
+
If your job class indicates that @delayed_jobs = true, you can queue
|
33
|
+
delayed jobs. These jobs will not be popped off the queue until the
|
34
|
+
Time indicated in arg[0][:delay_until] has come. Note that you must
|
35
|
+
call Resque.enable_delay(:queue) before enququing any delayed jobs, to
|
36
|
+
ensure that the performance impact on other queues is minimal.
|
37
|
+
|
38
|
+
Configuration
|
39
|
+
=============
|
40
|
+
|
41
|
+
Resque.redis= has been replaced with Resque.mongo= and expects a Mongo::DB
|
42
|
+
object as an argument.
|
43
|
+
|
44
|
+
Resque.mongo = Mongo::Connection.new.db("my_awesome_queue")
|
45
|
+
|
46
|
+
Stern Warnings
|
47
|
+
==============
|
48
|
+
|
49
|
+
Sometimes, Mongo-Resque will drop a queue collection, or create some
|
50
|
+
indexes, or otherwise manipulate its database. For this reason, it is
|
51
|
+
STRONGLY recommended that you give it its own database in mongo.
|
52
|
+
|
53
|
+
All jobs should be queued via Resque.enqueue. All arguments passed to
|
54
|
+
this method must be BSON-encodable. Mongo-Resque does not serialize
|
55
|
+
your objects for you. Arrays, Hashes, Strings, Numbers, and Times
|
56
|
+
are all ok, so don't worry.
|
57
|
+
|
58
|
+
Many of the new queue-level features require the first argument of
|
59
|
+
your perform method to be an options hash. In fact, if you just start
|
60
|
+
making all your perform()s take one param, that is an options hash,
|
61
|
+
you'll probably save yourself some pain.
|
62
|
+
|
63
|
+
Mongo-Resque will not create any indexes on your queues, only on its
|
64
|
+
meta-data. You will need to create any indexes you want. Normally,
|
65
|
+
This is not a problem, because you aren't querying by keys, but you may
|
66
|
+
want to create indexes on the class key in some circumstances. If you
|
67
|
+
use the unique or delay features, you may want some additional indexes,
|
68
|
+
depending on the nature of your workload. Paranoid? Test enqueuing and
|
69
|
+
processing all your jobs, and run with --notablescans. Learn the profiler,
|
70
|
+
and use it often.
|
71
|
+
|
72
|
+
Specifically, a queue with many long-delayed jobs will result in slower queue pops
|
73
|
+
for all jobs using that queue. Index delay_until in the case of
|
74
|
+
thousands of delayed jobs.
|
75
|
+
|
76
|
+
[0]: https://github.com/dbackeus/resque-mongo/blob/master/ORIGINAL_README.markdown
|
data/bin/resque
ADDED
@@ -0,0 +1,67 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
|
3
|
+
$LOAD_PATH.unshift File.dirname(__FILE__) + '/../lib'
|
4
|
+
require 'rubygems'
|
5
|
+
require 'resque'
|
6
|
+
|
7
|
+
def kill(worker)
|
8
|
+
abort "** resque kill WORKER_ID" if worker.nil?
|
9
|
+
pid = worker.split(':')[1].to_i
|
10
|
+
|
11
|
+
begin
|
12
|
+
Process.kill("KILL", pid)
|
13
|
+
puts "** killed #{worker}"
|
14
|
+
rescue Errno::ESRCH
|
15
|
+
puts "** worker #{worker} not running"
|
16
|
+
end
|
17
|
+
|
18
|
+
remove worker
|
19
|
+
end
|
20
|
+
|
21
|
+
def remove(worker)
|
22
|
+
abort "** resque remove WORKER_ID" if worker.nil?
|
23
|
+
|
24
|
+
Resque.remove_worker(worker)
|
25
|
+
puts "** removed #{worker}"
|
26
|
+
end
|
27
|
+
|
28
|
+
def list
|
29
|
+
if Resque.workers.any?
|
30
|
+
Resque.workers.each do |worker|
|
31
|
+
puts "#{worker} (#{worker.state})"
|
32
|
+
end
|
33
|
+
else
|
34
|
+
puts "None"
|
35
|
+
end
|
36
|
+
end
|
37
|
+
|
38
|
+
def queues
|
39
|
+
Resque.queues.each do |queue|
|
40
|
+
puts"Queue #{queue}: #{Resque.size(queue)}"
|
41
|
+
end
|
42
|
+
end
|
43
|
+
|
44
|
+
if (i = ARGV.index('-r')) && ARGV[i+1]
|
45
|
+
Resque.redis = ARGV[i+1]
|
46
|
+
ARGV.delete_at(i)
|
47
|
+
ARGV.delete_at(i+1)
|
48
|
+
end
|
49
|
+
|
50
|
+
case ARGV[0]
|
51
|
+
when 'kill'
|
52
|
+
kill ARGV[1]
|
53
|
+
when 'remove'
|
54
|
+
remove ARGV[1]
|
55
|
+
when 'list'
|
56
|
+
list
|
57
|
+
when 'queues'
|
58
|
+
queues
|
59
|
+
else
|
60
|
+
puts "Usage: resque [-r redis_host:redis_port] COMMAND [option]"
|
61
|
+
puts
|
62
|
+
puts "Commands:"
|
63
|
+
puts " remove WORKER Removes a worker"
|
64
|
+
puts " kill WORKER Kills a worker"
|
65
|
+
puts " list Lists known workers"
|
66
|
+
puts " queues checks queue depth"
|
67
|
+
end
|
data/bin/resque-web
ADDED
@@ -0,0 +1,23 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
|
3
|
+
$LOAD_PATH.unshift File.expand_path(File.dirname(__FILE__) + '/../lib')
|
4
|
+
begin
|
5
|
+
require 'vegas'
|
6
|
+
rescue LoadError
|
7
|
+
require 'rubygems'
|
8
|
+
require 'vegas'
|
9
|
+
end
|
10
|
+
require 'resque/server'
|
11
|
+
|
12
|
+
|
13
|
+
Vegas::Runner.new(Resque::Server, 'resque-web', {
|
14
|
+
:before_run => lambda {|v|
|
15
|
+
path = (ENV['RESQUECONFIG'] || v.args.first)
|
16
|
+
load path.to_s.strip if path
|
17
|
+
}
|
18
|
+
}) do |runner, opts, app|
|
19
|
+
opts.on('-N NAMESPACE', "--namespace NAMESPACE", "set the Mongo database") {|namespace|
|
20
|
+
runner.logger.info "Using Mongo database '#{namespace}'"
|
21
|
+
Resque.mongo = Resque.mongo.conn.db(namespace)
|
22
|
+
}
|
23
|
+
end
|
data/docs/HOOKS.md
ADDED
@@ -0,0 +1,132 @@
|
|
1
|
+
Resque Hooks
|
2
|
+
============
|
3
|
+
|
4
|
+
You can customize Resque or write plugins using its hook API. In many
|
5
|
+
cases you can use a hook rather than mess with Resque's internals.
|
6
|
+
|
7
|
+
For a list of available plugins see
|
8
|
+
<http://wiki.github.com/defunkt/resque/plugins>.
|
9
|
+
|
10
|
+
|
11
|
+
Worker Hooks
|
12
|
+
------------
|
13
|
+
|
14
|
+
If you wish to have a Proc called before the worker forks for the
|
15
|
+
first time, you can add it in the initializer like so:
|
16
|
+
|
17
|
+
Resque.before_first_fork do
|
18
|
+
puts "Call me once before the worker forks the first time"
|
19
|
+
end
|
20
|
+
|
21
|
+
You can also run a hook before _every_ fork:
|
22
|
+
|
23
|
+
Resque.before_fork do |job|
|
24
|
+
puts "Call me before the worker forks"
|
25
|
+
end
|
26
|
+
|
27
|
+
The `before_fork` hook will be run in the **parent** process. So, be
|
28
|
+
careful - any changes you make will be permanent for the lifespan of
|
29
|
+
the worker.
|
30
|
+
|
31
|
+
And after forking:
|
32
|
+
|
33
|
+
Resque.after_fork do |job|
|
34
|
+
puts "Call me after the worker forks"
|
35
|
+
end
|
36
|
+
|
37
|
+
The `after_fork` hook will be run in the child process and is passed
|
38
|
+
the current job. Any changes you make, therefor, will only live as
|
39
|
+
long as the job currently being processes.
|
40
|
+
|
41
|
+
All worker hooks can also be set using a setter, e.g.
|
42
|
+
|
43
|
+
Resque.after_fork = proc { puts "called" }
|
44
|
+
|
45
|
+
|
46
|
+
Job Hooks
|
47
|
+
---------
|
48
|
+
|
49
|
+
Plugins can utilize job hooks to provide additional behavior. A job
|
50
|
+
hook is a method name in the following format:
|
51
|
+
|
52
|
+
HOOKNAME_IDENTIFIER
|
53
|
+
|
54
|
+
For example, a `before_perform` hook which adds locking may be defined
|
55
|
+
like this:
|
56
|
+
|
57
|
+
def before_perform_with_lock(*args)
|
58
|
+
set_lock!
|
59
|
+
end
|
60
|
+
|
61
|
+
Once this hook is made available to your job (either by way of
|
62
|
+
inheritence or `extend`), it will be run before the job's `perform`
|
63
|
+
method is called. Hooks of each type are executed in alphabetical order,
|
64
|
+
so `before_perform_a` will always be executed before `before_perform_b`.
|
65
|
+
An unnamed hook (`before_perform`) will be executed first.
|
66
|
+
|
67
|
+
The available hooks are:
|
68
|
+
|
69
|
+
* `after_enqueue`: Called with the job args after a job is placed on the queue.
|
70
|
+
Any exception raised propagates up to the code which queued the job.
|
71
|
+
|
72
|
+
* `before_perform`: Called with the job args before perform. If it raises
|
73
|
+
`Resque::Job::DontPerform`, the job is aborted. If other exceptions
|
74
|
+
are raised, they will be propagated up the the `Resque::Failure`
|
75
|
+
backend.
|
76
|
+
|
77
|
+
* `after_perform`: Called with the job args after it performs. Uncaught
|
78
|
+
exceptions will propagate up to the `Resque::Failure` backend.
|
79
|
+
|
80
|
+
* `around_perform`: Called with the job args. It is expected to yield in order
|
81
|
+
to perform the job (but is not required to do so). It may handle exceptions
|
82
|
+
thrown by `perform`, but any that are not caught will propagate up to the
|
83
|
+
`Resque::Failure` backend.
|
84
|
+
|
85
|
+
* `on_failure`: Called with the exception and job args if any exception occurs
|
86
|
+
while performing the job (or hooks).
|
87
|
+
|
88
|
+
Hooks are easily implemented with superclasses or modules. A superclass could
|
89
|
+
look something like this.
|
90
|
+
|
91
|
+
class LoggedJob
|
92
|
+
def self.before_perform_log_job(*args)
|
93
|
+
Logger.info "About to perform #{self} with #{args.inspect}"
|
94
|
+
end
|
95
|
+
end
|
96
|
+
|
97
|
+
class MyJob < LoggedJob
|
98
|
+
def self.perform(*args)
|
99
|
+
...
|
100
|
+
end
|
101
|
+
end
|
102
|
+
|
103
|
+
Modules are even better because jobs can use many of them.
|
104
|
+
|
105
|
+
module ScaledJob
|
106
|
+
def after_enqueue_scale_workers(*args)
|
107
|
+
Logger.info "Scaling worker count up"
|
108
|
+
Scaler.up! if Redis.info[:pending].to_i > 25
|
109
|
+
end
|
110
|
+
end
|
111
|
+
|
112
|
+
module LoggedJob
|
113
|
+
def before_perform_log_job(*args)
|
114
|
+
Logger.info "About to perform #{self} with #{args.inspect}"
|
115
|
+
end
|
116
|
+
end
|
117
|
+
|
118
|
+
module RetriedJob
|
119
|
+
def on_failure_retry(e, *args)
|
120
|
+
Logger.info "Performing #{self} caused an exception (#{e}). Retrying..."
|
121
|
+
Resque.enqueue self, *args
|
122
|
+
end
|
123
|
+
end
|
124
|
+
|
125
|
+
class MyJob
|
126
|
+
extend LoggedJob
|
127
|
+
extend RetriedJob
|
128
|
+
extend ScaledJob
|
129
|
+
def self.perform(*args)
|
130
|
+
...
|
131
|
+
end
|
132
|
+
end
|
data/docs/PLUGINS.md
ADDED
@@ -0,0 +1,93 @@
|
|
1
|
+
Resque Plugins
|
2
|
+
==============
|
3
|
+
|
4
|
+
Resque encourages plugin development. For a list of available plugins,
|
5
|
+
please see <http://wiki.github.com/defunkt/resque/plugins>.
|
6
|
+
|
7
|
+
The `docs/HOOKS.md` file included with Resque documents the available
|
8
|
+
hooks you can use to add or change Resque functionality. This document
|
9
|
+
describes best practice for plugins themselves.
|
10
|
+
|
11
|
+
|
12
|
+
Version
|
13
|
+
-------
|
14
|
+
|
15
|
+
Plugins should declare the major.minor version of Resque they are
|
16
|
+
known to work with explicitly in their README.
|
17
|
+
|
18
|
+
For example, if your plugin depends on features in Resque 2.1, please
|
19
|
+
list "Depends on Resque 2.1" very prominently near the beginning of
|
20
|
+
your README.
|
21
|
+
|
22
|
+
Because Resque uses [Semantic Versioning][sv], you can safely make the
|
23
|
+
following assumptions:
|
24
|
+
|
25
|
+
* Your plugin will work with 2.2, 2.3, etc - no methods will be
|
26
|
+
removed or changed, only added.
|
27
|
+
* Your plugin might not work with 3.0+, as APIs may change or be
|
28
|
+
removed.
|
29
|
+
|
30
|
+
|
31
|
+
Namespace
|
32
|
+
---------
|
33
|
+
|
34
|
+
All plugins should live under the `Resque::Plugins` module to avoid
|
35
|
+
clashing with first class Resque constants or other Ruby libraries.
|
36
|
+
|
37
|
+
Good:
|
38
|
+
|
39
|
+
* Resque::Plugins::Lock
|
40
|
+
* Resque::Plugins::FastRetry
|
41
|
+
|
42
|
+
Bad:
|
43
|
+
|
44
|
+
* Resque::Lock
|
45
|
+
* ResqueQueue
|
46
|
+
|
47
|
+
|
48
|
+
Gem Name
|
49
|
+
--------
|
50
|
+
|
51
|
+
Gem names should be in the format of `resque-FEATURE`, where `FEATURE`
|
52
|
+
succinctly describes the feature your plugin adds to Resque.
|
53
|
+
|
54
|
+
Good:
|
55
|
+
|
56
|
+
* resque-status
|
57
|
+
* resque-scheduler
|
58
|
+
|
59
|
+
Bad:
|
60
|
+
|
61
|
+
* multi-queue
|
62
|
+
* defunkt-resque-lock
|
63
|
+
|
64
|
+
|
65
|
+
Hooks
|
66
|
+
-----
|
67
|
+
|
68
|
+
Job hook names should be namespaced to work properly.
|
69
|
+
|
70
|
+
Good:
|
71
|
+
|
72
|
+
* before_perform_lock
|
73
|
+
* around_perform_check_status
|
74
|
+
|
75
|
+
Bad:
|
76
|
+
|
77
|
+
* before_perform
|
78
|
+
* on_failure
|
79
|
+
|
80
|
+
|
81
|
+
Lint
|
82
|
+
----
|
83
|
+
|
84
|
+
Plugins should test compliance to this document using the
|
85
|
+
`Resque::Plugin.lint` method.
|
86
|
+
|
87
|
+
For example:
|
88
|
+
|
89
|
+
assert_nothing_raised do
|
90
|
+
Resque::Plugin.lint(Resque::Plugins::Lock)
|
91
|
+
end
|
92
|
+
|
93
|
+
[sv]: http://semver.org/
|
@@ -0,0 +1,13 @@
|
|
1
|
+
module Resque
|
2
|
+
# Raised whenever we need a queue but none is provided.
|
3
|
+
class NoQueueError < RuntimeError; end
|
4
|
+
|
5
|
+
# Raised when trying to create a job without a class
|
6
|
+
class NoClassError < RuntimeError; end
|
7
|
+
|
8
|
+
# Raised when a worker was killed while processing a job.
|
9
|
+
class DirtyExit < RuntimeError; end
|
10
|
+
|
11
|
+
#raised for bad delay queue integrity
|
12
|
+
class QueueError < RuntimeError; end
|
13
|
+
end
|