resque_admin 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (59) hide show
  1. checksums.yaml +7 -0
  2. data/HISTORY.md +530 -0
  3. data/LICENSE +20 -0
  4. data/README.markdown +957 -0
  5. data/Rakefile +57 -0
  6. data/bin/resque-admin +81 -0
  7. data/bin/resque-admin-web +31 -0
  8. data/lib/resque_admin.rb +578 -0
  9. data/lib/resque_admin/data_store.rb +325 -0
  10. data/lib/resque_admin/errors.rb +21 -0
  11. data/lib/resque_admin/failure.rb +119 -0
  12. data/lib/resque_admin/failure/airbrake.rb +33 -0
  13. data/lib/resque_admin/failure/base.rb +73 -0
  14. data/lib/resque_admin/failure/multiple.rb +68 -0
  15. data/lib/resque_admin/failure/redis.rb +128 -0
  16. data/lib/resque_admin/failure/redis_multi_queue.rb +104 -0
  17. data/lib/resque_admin/helpers.rb +48 -0
  18. data/lib/resque_admin/job.rb +296 -0
  19. data/lib/resque_admin/log_formatters/quiet_formatter.rb +7 -0
  20. data/lib/resque_admin/log_formatters/verbose_formatter.rb +7 -0
  21. data/lib/resque_admin/log_formatters/very_verbose_formatter.rb +8 -0
  22. data/lib/resque_admin/logging.rb +18 -0
  23. data/lib/resque_admin/plugin.rb +78 -0
  24. data/lib/resque_admin/server.rb +301 -0
  25. data/lib/resque_admin/server/helpers.rb +64 -0
  26. data/lib/resque_admin/server/public/favicon.ico +0 -0
  27. data/lib/resque_admin/server/public/idle.png +0 -0
  28. data/lib/resque_admin/server/public/jquery-1.12.4.min.js +5 -0
  29. data/lib/resque_admin/server/public/jquery.relatize_date.js +95 -0
  30. data/lib/resque_admin/server/public/poll.png +0 -0
  31. data/lib/resque_admin/server/public/ranger.js +78 -0
  32. data/lib/resque_admin/server/public/reset.css +44 -0
  33. data/lib/resque_admin/server/public/style.css +91 -0
  34. data/lib/resque_admin/server/public/working.png +0 -0
  35. data/lib/resque_admin/server/test_helper.rb +19 -0
  36. data/lib/resque_admin/server/views/error.erb +1 -0
  37. data/lib/resque_admin/server/views/failed.erb +29 -0
  38. data/lib/resque_admin/server/views/failed_job.erb +50 -0
  39. data/lib/resque_admin/server/views/failed_queues_overview.erb +24 -0
  40. data/lib/resque_admin/server/views/job_class.erb +6 -0
  41. data/lib/resque_admin/server/views/key_sets.erb +17 -0
  42. data/lib/resque_admin/server/views/key_string.erb +11 -0
  43. data/lib/resque_admin/server/views/layout.erb +44 -0
  44. data/lib/resque_admin/server/views/next_more.erb +22 -0
  45. data/lib/resque_admin/server/views/overview.erb +4 -0
  46. data/lib/resque_admin/server/views/processing.erb +2 -0
  47. data/lib/resque_admin/server/views/queues.erb +58 -0
  48. data/lib/resque_admin/server/views/stats.erb +62 -0
  49. data/lib/resque_admin/server/views/workers.erb +109 -0
  50. data/lib/resque_admin/server/views/working.erb +71 -0
  51. data/lib/resque_admin/stat.rb +58 -0
  52. data/lib/resque_admin/tasks.rb +72 -0
  53. data/lib/resque_admin/thread_signal.rb +24 -0
  54. data/lib/resque_admin/vendor/utf8_util.rb +24 -0
  55. data/lib/resque_admin/version.rb +3 -0
  56. data/lib/resque_admin/worker.rb +917 -0
  57. data/lib/tasks/redis.rake +161 -0
  58. data/lib/tasks/resque_admin.rake +2 -0
  59. metadata +191 -0
data/LICENSE ADDED
@@ -0,0 +1,20 @@
1
+ Copyright (c) Chris Wanstrath
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining
4
+ a copy of this software and associated documentation files (the
5
+ "Software"), to deal in the Software without restriction, including
6
+ without limitation the rights to use, copy, modify, merge, publish,
7
+ distribute, sublicense, and/or sell copies of the Software, and to
8
+ permit persons to whom the Software is furnished to do so, subject to
9
+ the following conditions:
10
+
11
+ The above copyright notice and this permission notice shall be
12
+ included in all copies or substantial portions of the Software.
13
+
14
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
15
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
16
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
17
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
18
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
19
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
20
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.markdown ADDED
@@ -0,0 +1,957 @@
1
+ ResqueAdmin
2
+ ======
3
+
4
+ [![Gem Version](https://badge.fury.io/rb/resque.svg)](https://rubygems.org/gems/resque)
5
+ [![Build Status](https://travis-ci.org/resque/resque.svg)](https://travis-ci.org/resque/resque)
6
+ [![Coverage Status](https://coveralls.io/repos/github/resque/resque/badge.svg?branch=1-x-stable)](https://coveralls.io/r/resque/resque?branch=1-x-stable)
7
+
8
+ ResqueAdmin (pronounced like "rescue") is a Redis-backed library for creating
9
+ background jobs, placing those jobs on multiple queues, and processing
10
+ them later.
11
+
12
+ Background jobs can be any Ruby class or module that responds to
13
+ `perform`. Your existing classes can easily be converted to background
14
+ jobs or you can create new classes specifically to do work. Or, you
15
+ can do both.
16
+
17
+ ResqueAdmin is heavily inspired by DelayedJob (which rocks) and comprises
18
+ three parts:
19
+
20
+ 1. A Ruby library for creating, querying, and processing jobs
21
+ 2. A Rake task for starting a worker which processes jobs
22
+ 3. A Sinatra app for monitoring queues, jobs, and workers.
23
+
24
+ ResqueAdmin workers can be distributed between multiple machines,
25
+ support priorities, are resilient to memory bloat / "leaks," are
26
+ optimized for REE (but work on MRI and JRuby), tell you what they're
27
+ doing, and expect failure.
28
+
29
+ ResqueAdmin queues are persistent; support constant time, atomic push and
30
+ pop (thanks to Redis); provide visibility into their contents; and
31
+ store jobs as simple JSON packages.
32
+
33
+ The ResqueAdmin frontend tells you what workers are doing, what workers are
34
+ not doing, what queues you're using, what's in those queues, provides
35
+ general usage stats, and helps you track failures.
36
+
37
+ ResqueAdmin now supports Ruby 2.0.0 and above. Any future updates will not be
38
+ guaranteed to work without defects on any Rubies older than 2.0.0. We will also only be supporting Redis 3.0 and above going forward.
39
+
40
+
41
+ The Blog Post
42
+ -------------
43
+
44
+ For the backstory, philosophy, and history of ResqueAdmin's beginnings,
45
+ please see [the blog post][0].
46
+
47
+
48
+ Overview
49
+ --------
50
+
51
+ ResqueAdmin allows you to create jobs and place them on a queue, then,
52
+ later, pull those jobs off the queue and process them.
53
+
54
+ ResqueAdmin jobs are Ruby classes (or modules) which respond to the
55
+ `perform` method. Here's an example:
56
+
57
+
58
+ ``` ruby
59
+ class Archive
60
+ @queue = :file_serve
61
+
62
+ def self.perform(repo_id, branch = 'master')
63
+ repo = Repository.find(repo_id)
64
+ repo.create_archive(branch)
65
+ end
66
+ end
67
+ ```
68
+
69
+ The `@queue` class instance variable determines which queue `Archive`
70
+ jobs will be placed in. Queues are arbitrary and created on the fly -
71
+ you can name them whatever you want and have as many as you want.
72
+
73
+ To place an `Archive` job on the `file_serve` queue, we might add this
74
+ to our application's pre-existing `Repository` class:
75
+
76
+ ``` ruby
77
+ class Repository
78
+ def async_create_archive(branch)
79
+ ResqueAdmin.enqueue(Archive, self.id, branch)
80
+ end
81
+ end
82
+ ```
83
+
84
+ Now when we call `repo.async_create_archive('masterbrew')` in our
85
+ application, a job will be created and placed on the `file_serve`
86
+ queue.
87
+
88
+ Later, a worker will run something like this code to process the job:
89
+
90
+ ``` ruby
91
+ klass, args = ResqueAdmin.reserve(:file_serve)
92
+ klass.perform(*args) if klass.respond_to? :perform
93
+ ```
94
+
95
+ Which translates to:
96
+
97
+ ``` ruby
98
+ Archive.perform(44, 'masterbrew')
99
+ ```
100
+
101
+ Let's start a worker to run `file_serve` jobs:
102
+
103
+ $ cd app_root
104
+ $ QUEUE=file_serve rake resque:work
105
+
106
+ This starts one ResqueAdmin worker and tells it to work off the
107
+ `file_serve` queue. As soon as it's ready it'll try to run the
108
+ `ResqueAdmin.reserve` code snippet above and process jobs until it can't
109
+ find any more, at which point it will sleep for a small period and
110
+ repeatedly poll the queue for more jobs.
111
+
112
+ Workers can be given multiple queues (a "queue list") and run on
113
+ multiple machines. In fact they can be run anywhere with network
114
+ access to the Redis server.
115
+
116
+
117
+ Jobs
118
+ ----
119
+
120
+ What should you run in the background? Anything that takes any time at
121
+ all. Slow INSERT statements, disk manipulating, data processing, etc.
122
+
123
+ At GitHub we use ResqueAdmin to process the following types of jobs:
124
+
125
+ * Warming caches
126
+ * Counting disk usage
127
+ * Building tarballs
128
+ * Building Rubygems
129
+ * Firing off web hooks
130
+ * Creating events in the db and pre-caching them
131
+ * Building graphs
132
+ * Deleting users
133
+ * Updating our search index
134
+
135
+ As of writing we have about 35 different types of background jobs.
136
+
137
+ Keep in mind that you don't need a web app to use ResqueAdmin - we just
138
+ mention "foreground" and "background" because they make conceptual
139
+ sense. You could easily be spidering sites and sticking data which
140
+ needs to be crunched later into a queue.
141
+
142
+
143
+ ### Persistence
144
+
145
+ Jobs are persisted to queues as JSON objects. Let's take our `Archive`
146
+ example from above. We'll run the following code to create a job:
147
+
148
+ ``` ruby
149
+ repo = Repository.find(44)
150
+ repo.async_create_archive('masterbrew')
151
+ ```
152
+
153
+ The following JSON will be stored in the `file_serve` queue:
154
+
155
+ ``` javascript
156
+ {
157
+ 'class': 'Archive',
158
+ 'args': [ 44, 'masterbrew' ]
159
+ }
160
+ ```
161
+
162
+ Because of this your jobs must only accept arguments that can be JSON encoded.
163
+
164
+ So instead of doing this:
165
+
166
+ ``` ruby
167
+ ResqueAdmin.enqueue(Archive, self, branch)
168
+ ```
169
+
170
+ do this:
171
+
172
+ ``` ruby
173
+ ResqueAdmin.enqueue(Archive, self.id, branch)
174
+ ```
175
+
176
+ This is why our above example (and all the examples in `examples/`)
177
+ uses object IDs instead of passing around the objects.
178
+
179
+ While this is less convenient than just sticking a marshaled object
180
+ in the database, it gives you a slight advantage: your jobs will be
181
+ run against the most recent version of an object because they need to
182
+ pull from the DB or cache.
183
+
184
+ If your jobs were run against marshaled objects, they could
185
+ potentially be operating on a stale record with out-of-date information.
186
+
187
+
188
+ ### send_later / async
189
+
190
+ Want something like DelayedJob's `send_later` or the ability to use
191
+ instance methods instead of just methods for jobs? See the `examples/`
192
+ directory for goodies.
193
+
194
+ We plan to provide first class `async` support in a future release.
195
+
196
+
197
+ ### Failure
198
+
199
+ If a job raises an exception, it is logged and handed off to the
200
+ `ResqueAdmin::Failure` module. Failures are logged either locally in Redis
201
+ or using some different backend. To see exceptions while developing,
202
+ use VERBOSE env variable, see details below under Logging.
203
+
204
+ For example, ResqueAdmin ships with Airbrake support. To configure it, put
205
+ the following into an initialisation file or into your rake job:
206
+
207
+ ``` ruby
208
+ # send errors which occur in background jobs to redis and airbrake
209
+ require 'resque/failure/multiple'
210
+ require 'resque/failure/redis'
211
+ require 'resque/failure/airbrake'
212
+
213
+ ResqueAdmin::Failure::Multiple.classes = [ResqueAdmin::Failure::Redis, ResqueAdmin::Failure::Airbrake]
214
+ ResqueAdmin::Failure.backend = ResqueAdmin::Failure::Multiple
215
+ ```
216
+
217
+ Keep this in mind when writing your jobs: you may want to throw
218
+ exceptions you would not normally throw in order to assist debugging.
219
+
220
+
221
+ Workers
222
+ -------
223
+
224
+ ResqueAdmin workers are rake tasks that run forever. They basically do this:
225
+
226
+ ``` ruby
227
+ start
228
+ loop do
229
+ if job = reserve
230
+ job.process
231
+ else
232
+ sleep 5 # Polling frequency = 5
233
+ end
234
+ end
235
+ shutdown
236
+ ```
237
+
238
+ Starting a worker is simple. Here's our example from earlier:
239
+
240
+ $ QUEUE=file_serve rake resque:work
241
+
242
+ By default ResqueAdmin won't know about your application's
243
+ environment. That is, it won't be able to find and run your jobs - it
244
+ needs to load your application into memory.
245
+
246
+ If we've installed ResqueAdmin as a Rails plugin, we might run this command
247
+ from our RAILS_ROOT:
248
+
249
+ $ QUEUE=file_serve rake environment resque:work
250
+
251
+ This will load the environment before starting a worker. Alternately
252
+ we can define a `resque:setup` task with a dependency on the
253
+ `environment` rake task:
254
+
255
+ ``` ruby
256
+ task "resque:setup" => :environment
257
+ ```
258
+
259
+ GitHub's setup task looks like this:
260
+
261
+ ``` ruby
262
+ task "resque:setup" => :environment do
263
+ Grit::Git.git_timeout = 10.minutes
264
+ end
265
+ ```
266
+
267
+ We don't want the `git_timeout` as high as 10 minutes in our web app,
268
+ but in the ResqueAdmin workers it's fine.
269
+
270
+
271
+ ### Logging
272
+
273
+ Workers support basic logging to STDOUT. If you start them with the
274
+ `VERBOSE` env variable set, they will print basic debugging
275
+ information. You can also set the `VVERBOSE` (very verbose) env
276
+ variable.
277
+
278
+ $ VVERBOSE=1 QUEUE=file_serve rake environment resque:work
279
+
280
+ If you want ResqueAdmin to log to a file, in Rails do:
281
+
282
+ ```ruby
283
+ # config/initializers/resque.rb
284
+ ResqueAdmin.logger = Logger.new(Rails.root.join('log', "#{Rails.env}_resque.log"))
285
+ ```
286
+
287
+ ### Process IDs (PIDs)
288
+
289
+ There are scenarios where it's helpful to record the PID of a resque
290
+ worker process. Use the PIDFILE option for easy access to the PID:
291
+
292
+ $ PIDFILE=./resque.pid QUEUE=file_serve rake environment resque:work
293
+
294
+ ### Running in the background
295
+
296
+ There are scenarios where it's helpful for
297
+ the resque worker to run itself in the background (usually in combination with
298
+ PIDFILE). Use the BACKGROUND option so that rake will return as soon as the
299
+ worker is started.
300
+
301
+ $ PIDFILE=./resque.pid BACKGROUND=yes QUEUE=file_serve \
302
+ rake environment resque:work
303
+
304
+ ### Polling frequency
305
+
306
+ You can pass an INTERVAL option which is a float representing the polling frequency.
307
+ The default is 5 seconds, but for a semi-active app you may want to use a smaller value.
308
+
309
+ $ INTERVAL=0.1 QUEUE=file_serve rake environment resque:work
310
+
311
+ ### Priorities and Queue Lists
312
+
313
+ ResqueAdmin doesn't support numeric priorities but instead uses the order
314
+ of queues you give it. We call this list of queues the "queue list."
315
+
316
+ Let's say we add a `warm_cache` queue in addition to our `file_serve`
317
+ queue. We'd now start a worker like so:
318
+
319
+ $ QUEUES=file_serve,warm_cache rake resque:work
320
+
321
+ When the worker looks for new jobs, it will first check
322
+ `file_serve`. If it finds a job, it'll process it then check
323
+ `file_serve` again. It will keep checking `file_serve` until no more
324
+ jobs are available. At that point, it will check `warm_cache`. If it
325
+ finds a job it'll process it then check `file_serve` (repeating the
326
+ whole process).
327
+
328
+ In this way you can prioritize certain queues. At GitHub we start our
329
+ workers with something like this:
330
+
331
+ $ QUEUES=critical,archive,high,low rake resque:work
332
+
333
+ Notice the `archive` queue - it is specialized and in our future
334
+ architecture will only be run from a single machine.
335
+
336
+ At that point we'll start workers on our generalized background
337
+ machines with this command:
338
+
339
+ $ QUEUES=critical,high,low rake resque:work
340
+
341
+ And workers on our specialized archive machine with this command:
342
+
343
+ $ QUEUE=archive rake resque:work
344
+
345
+
346
+ ### Running All Queues
347
+
348
+ If you want your workers to work off of every queue, including new
349
+ queues created on the fly, you can use a splat:
350
+
351
+ $ QUEUE=* rake resque:work
352
+
353
+ Queues will be processed in alphabetical order.
354
+
355
+
356
+ ### Running Multiple Workers
357
+
358
+ At GitHub we use god to start and stop multiple workers. A sample god
359
+ configuration file is included under `examples/god`. We recommend this
360
+ method.
361
+
362
+ If you'd like to run multiple workers in development mode, you can do
363
+ so using the `resque:workers` rake task:
364
+
365
+ $ COUNT=5 QUEUE=* rake resque:workers
366
+
367
+ This will spawn five ResqueAdmin workers, each in its own process. Hitting
368
+ ctrl-c should be sufficient to stop them all.
369
+
370
+
371
+ ### Forking
372
+
373
+ On certain platforms, when a ResqueAdmin worker reserves a job it
374
+ immediately forks a child process. The child processes the job then
375
+ exits. When the child has exited successfully, the worker reserves
376
+ another job and repeats the process.
377
+
378
+ Why?
379
+
380
+ Because ResqueAdmin assumes chaos.
381
+
382
+ ResqueAdmin assumes your background workers will lock up, run too long, or
383
+ have unwanted memory growth.
384
+
385
+ If ResqueAdmin workers processed jobs themselves, it'd be hard to whip them
386
+ into shape. Let's say one is using too much memory: you send it a
387
+ signal that says "shutdown after you finish processing the current
388
+ job," and it does so. It then starts up again - loading your entire
389
+ application environment. This adds useless CPU cycles and causes a
390
+ delay in queue processing.
391
+
392
+ Plus, what if it's using too much memory and has stopped responding to
393
+ signals?
394
+
395
+ Thanks to ResqueAdmin's parent / child architecture, jobs that use too much memory
396
+ release that memory upon completion. No unwanted growth.
397
+
398
+ And what if a job is running too long? You'd need to `kill -9` it then
399
+ start the worker again. With ResqueAdmin's parent / child architecture you
400
+ can tell the parent to forcefully kill the child then immediately
401
+ start processing more jobs. No startup delay or wasted cycles.
402
+
403
+ The parent / child architecture helps us keep tabs on what workers are
404
+ doing, too. By eliminating the need to `kill -9` workers we can have
405
+ parents remove themselves from the global listing of workers. If we
406
+ just ruthlessly killed workers, we'd need a separate watchdog process
407
+ to add and remove them to the global listing - which becomes
408
+ complicated.
409
+
410
+ Workers instead handle their own state.
411
+
412
+
413
+ ### Parents and Children
414
+
415
+ Here's a parent / child pair doing some work:
416
+
417
+ $ ps -e -o pid,command | grep [r]esque
418
+ 92099 resque: Forked 92102 at 1253142769
419
+ 92102 resque: Processing file_serve since 1253142769
420
+
421
+ You can clearly see that process 92099 forked 92102, which has been
422
+ working since 1253142769.
423
+
424
+ (By advertising the time they began processing you can easily use monit
425
+ or god to kill stale workers.)
426
+
427
+ When a parent process is idle, it lets you know what queues it is
428
+ waiting for work on:
429
+
430
+ $ ps -e -o pid,command | grep [r]esque
431
+ 92099 resque: Waiting for file_serve,warm_cache
432
+
433
+
434
+ ### Signals
435
+
436
+ ResqueAdmin workers respond to a few different signals:
437
+
438
+ * `QUIT` - Wait for child to finish processing then exit
439
+ * `TERM` / `INT` - Immediately kill child then exit
440
+ * `USR1` - Immediately kill child but don't exit
441
+ * `USR2` - Don't start to process any new jobs
442
+ * `CONT` - Start to process new jobs again after a USR2
443
+
444
+ If you want to gracefully shutdown a ResqueAdmin worker, use `QUIT`.
445
+
446
+ If you want to kill a stale or stuck child, use `USR1`. Processing
447
+ will continue as normal unless the child was not found. In that case
448
+ ResqueAdmin assumes the parent process is in a bad state and shuts down.
449
+
450
+ If you want to kill a stale or stuck child and shutdown, use `TERM`
451
+
452
+ If you want to stop processing jobs, but want to leave the worker running
453
+ (for example, to temporarily alleviate load), use `USR2` to stop processing,
454
+ then `CONT` to start it again.
455
+
456
+ #### Signals on Heroku
457
+
458
+ When shutting down processes, Heroku sends every process a TERM signal at the
459
+ same time. By default this causes an immediate shutdown of any running job
460
+ leading to frequent `ResqueAdmin::TermException` errors. For short running jobs, a simple
461
+ solution is to give a small amount of time for the job to finish
462
+ before killing it.
463
+
464
+ ResqueAdmin doesn't handle this out of the box (for both cedar-14 and heroku-16), you need to
465
+ install the [`resque-heroku-signals`](https://github.com/iloveitaly/resque-heroku-signals)
466
+ addon which adds the required signal handling to make the behavior described above work.
467
+ Related issue: https://github.com/resque/resque/issues/1559
468
+
469
+ To accomplish this set the following environment variables:
470
+
471
+ * `RESQUE_PRE_SHUTDOWN_TIMEOUT` - The time between the parent receiving a shutdown signal (TERM by default) and it sending that signal on to the child process. Designed to give the child process
472
+ time to complete before being forced to die.
473
+
474
+ * `TERM_CHILD` - Must be set for `RESQUE_PRE_SHUTDOWN_TIMEOUT` to be used. After the timeout, if the child is still running it will raise a `ResqueAdmin::TermException` and exit.
475
+
476
+ * `RESQUE_TERM_TIMEOUT` - By default you have a few seconds to handle `ResqueAdmin::TermException` in your job. `RESQUE_TERM_TIMEOUT` and `RESQUE_PRE_SHUTDOWN_TIMEOUT` must be lower than the [heroku dyno timeout](https://devcenter.heroku.com/articles/limits#exit-timeout).
477
+
478
+ ### Mysql::Error: MySQL server has gone away
479
+
480
+ If your workers remain idle for too long they may lose their MySQL connection. Depending on your version of Rails, we recommend the following:
481
+
482
+ #### Rails 3.x
483
+ In your `perform` method, add the following line:
484
+
485
+ ``` ruby
486
+ class MyTask
487
+ def self.perform
488
+ ActiveRecord::Base.verify_active_connections!
489
+ # rest of your code
490
+ end
491
+ end
492
+ ```
493
+
494
+ The Rails doc says the following about `verify_active_connections!`:
495
+
496
+ Verify active connections and remove and disconnect connections associated with stale threads.
497
+
498
+ #### Rails 4.x
499
+
500
+ In your `perform` method, instead of `verify_active_connections!`, use:
501
+
502
+ ``` ruby
503
+ class MyTask
504
+ def self.perform
505
+ ActiveRecord::Base.clear_active_connections!
506
+ # rest of your code
507
+ end
508
+ end
509
+ ```
510
+
511
+ From the Rails docs on [`clear_active_connections!`](http://api.rubyonrails.org/classes/ActiveRecord/ConnectionAdapters/ConnectionHandler.html#method-i-clear_active_connections-21):
512
+
513
+ Returns any connections in use by the current thread back to the pool, and also returns connections to the pool cached by threads that are no longer alive.
514
+
515
+
516
+ #### ActiveJob
517
+
518
+ If you're going to need to use the database in a number of different jobs, consider adding this to your ApplicationJob file:
519
+
520
+ ``` ruby
521
+ class ApplicationJob < ActiveJob::Base
522
+ before_perform do |job|
523
+ ActiveRecord::Base.clear_active_connections!
524
+ end
525
+ end
526
+ ```
527
+
528
+
529
+ The Front End
530
+ -------------
531
+
532
+ ResqueAdmin comes with a Sinatra-based front end for seeing what's up with
533
+ your queue.
534
+
535
+ ![The Front End](https://camo.githubusercontent.com/64d150a243987ffbc33f588bd6d7722a0bb8d69a/687474703a2f2f7475746f7269616c732e6a756d7073746172746c61622e636f6d2f696d616765732f7265737175655f6f766572766965772e706e67)
536
+
537
+ ### Standalone
538
+
539
+ If you've installed ResqueAdmin as a gem running the front end standalone is easy:
540
+
541
+ $ resque-web
542
+
543
+ It's a thin layer around `rackup` so it's configurable as well:
544
+
545
+ $ resque-web -p 8282
546
+
547
+ If you have a ResqueAdmin config file you want evaluated just pass it to
548
+ the script as the final argument:
549
+
550
+ $ resque-web -p 8282 rails_root/config/initializers/resque.rb
551
+
552
+ You can also set the namespace directly using `resque-web`:
553
+
554
+ $ resque-web -p 8282 -N myapp
555
+
556
+ or set the Redis connection string if you need to do something like select a different database:
557
+
558
+ $ resque-web -p 8282 -r localhost:6379:2
559
+
560
+ ### Passenger
561
+
562
+ Using Passenger? ResqueAdmin ships with a `config.ru` you can use. See
563
+ Phusion's guide:
564
+
565
+ Apache: <https://www.phusionpassenger.com/library/deploy/apache/deploy/ruby/>
566
+ Nginx: <https://www.phusionpassenger.com/library/deploy/nginx/deploy/ruby/>
567
+
568
+ ### Rack::URLMap
569
+
570
+ If you want to load ResqueAdmin on a subpath, possibly alongside other
571
+ apps, it's easy to do with Rack's `URLMap`:
572
+
573
+ ``` ruby
574
+ require 'resque/server'
575
+
576
+ run Rack::URLMap.new \
577
+ "/" => Your::App.new,
578
+ "/resque" => ResqueAdmin::Server.new
579
+ ```
580
+
581
+ Check `examples/demo/config.ru` for a functional example (including
582
+ HTTP basic auth).
583
+
584
+ ### Rails 3
585
+
586
+ You can also mount ResqueAdmin on a subpath in your existing Rails 3 app by adding `require 'resque/server'` to the top of your routes file or in an initializer then adding this to `routes.rb`:
587
+
588
+ ``` ruby
589
+ mount ResqueAdmin::Server.new, :at => "/resque"
590
+ ```
591
+
592
+
593
+ ResqueAdmin vs DelayedJob
594
+ --------------------
595
+
596
+ How does ResqueAdmin compare to DelayedJob, and why would you choose one
597
+ over the other?
598
+
599
+ * ResqueAdmin supports multiple queues
600
+ * DelayedJob supports finer grained priorities
601
+ * ResqueAdmin workers are resilient to memory leaks / bloat
602
+ * DelayedJob workers are extremely simple and easy to modify
603
+ * ResqueAdmin requires Redis
604
+ * DelayedJob requires ActiveRecord
605
+ * ResqueAdmin can only place JSONable Ruby objects on a queue as arguments
606
+ * DelayedJob can place _any_ Ruby object on its queue as arguments
607
+ * ResqueAdmin includes a Sinatra app for monitoring what's going on
608
+ * DelayedJob can be queried from within your Rails app if you want to
609
+ add an interface
610
+
611
+ If you're doing Rails development, you already have a database and
612
+ ActiveRecord. DelayedJob is super easy to setup and works great.
613
+ GitHub used it for many months to process almost 200 million jobs.
614
+
615
+ Choose ResqueAdmin if:
616
+
617
+ * You need multiple queues
618
+ * You don't care / dislike numeric priorities
619
+ * You don't need to persist every Ruby object ever
620
+ * You have potentially huge queues
621
+ * You want to see what's going on
622
+ * You expect a lot of failure / chaos
623
+ * You can setup Redis
624
+ * You're not running short on RAM
625
+
626
+ Choose DelayedJob if:
627
+
628
+ * You like numeric priorities
629
+ * You're not doing a gigantic amount of jobs each day
630
+ * Your queue stays small and nimble
631
+ * There is not a lot failure / chaos
632
+ * You want to easily throw anything on the queue
633
+ * You don't want to setup Redis
634
+
635
+ In no way is ResqueAdmin a "better" DelayedJob, so make sure you pick the
636
+ tool that's best for your app.
637
+
638
+ ResqueAdmin Dependencies
639
+ -------------------
640
+
641
+ $ gem install bundler
642
+ $ bundle install
643
+
644
+
645
+ Installing ResqueAdmin
646
+ -----------------
647
+
648
+ ### In a Rack app, as a gem
649
+
650
+ First install the gem.
651
+
652
+ $ gem install resque
653
+
654
+ Next include it in your application.
655
+
656
+ ``` ruby
657
+ require 'resque'
658
+ ```
659
+
660
+ Now start your application:
661
+
662
+ rackup config.ru
663
+
664
+ That's it! You can now create ResqueAdmin jobs from within your app.
665
+
666
+ To start a worker, create a Rakefile in your app's root (or add this
667
+ to an existing Rakefile):
668
+
669
+ ``` ruby
670
+ require 'your/app'
671
+ require 'resque/tasks'
672
+ ```
673
+
674
+ If you're using Rails 5.x, include the following in lib/tasks/resque.rb:
675
+
676
+ ```ruby
677
+ require 'resque/tasks'
678
+ task 'resque:setup' => :environment
679
+ ```
680
+
681
+ Now:
682
+
683
+ $ QUEUE=* rake resque:work
684
+
685
+ Alternately you can define a `resque:setup` hook in your Rakefile if you
686
+ don't want to load your app every time rake runs.
687
+
688
+
689
+ ### In a Rails 2.x app, as a gem
690
+
691
+ First install the gem.
692
+
693
+ $ gem install resque
694
+
695
+ Next include it in your application.
696
+
697
+ $ cat config/initializers/load_resque.rb
698
+ require 'resque'
699
+
700
+ Now start your application:
701
+
702
+ $ ./script/server
703
+
704
+ That's it! You can now create ResqueAdmin jobs from within your app.
705
+
706
+ To start a worker, add this to your Rakefile in `RAILS_ROOT`:
707
+
708
+ ``` ruby
709
+ require 'resque/tasks'
710
+ ```
711
+
712
+ Now:
713
+
714
+ $ QUEUE=* rake environment resque:work
715
+
716
+ Don't forget you can define a `resque:setup` hook in
717
+ `lib/tasks/whatever.rake` that loads the `environment` task every time.
718
+
719
+
720
+ ### In a Rails 2.x app, as a plugin
721
+
722
+ $ ./script/plugin install git://github.com/resque/resque
723
+
724
+ That's it! ResqueAdmin will automatically be available when your Rails app
725
+ loads.
726
+
727
+ To start a worker:
728
+
729
+ $ QUEUE=* rake environment resque:work
730
+
731
+ Don't forget you can define a `resque:setup` hook in
732
+ `lib/tasks/whatever.rake` that loads the `environment` task every time.
733
+
734
+
735
+ ### In a Rails 3.x or 4.x app, as a gem
736
+
737
+ First include it in your Gemfile.
738
+
739
+ $ cat Gemfile
740
+ ...
741
+ gem 'resque'
742
+ ...
743
+
744
+ Next install it with Bundler.
745
+
746
+ $ bundle install
747
+
748
+ Now start your application:
749
+
750
+ $ rails server
751
+
752
+ That's it! You can now create ResqueAdmin jobs from within your app.
753
+
754
+ To start a worker, add this to a file in `lib/tasks` (ex:
755
+ `lib/tasks/resque.rake`):
756
+
757
+ ``` ruby
758
+ require 'resque/tasks'
759
+ ```
760
+
761
+ Now:
762
+
763
+ $ QUEUE=* rake environment resque:work
764
+
765
+ Don't forget you can define a `resque:setup` hook in
766
+ `lib/tasks/whatever.rake` that loads the `environment` task every time.
767
+
768
+
769
+ Configuration
770
+ -------------
771
+
772
+ You may want to change the Redis host and port ResqueAdmin connects to, or
773
+ set various other options at startup.
774
+
775
+ ResqueAdmin has a `redis` setter which can be given a string or a Redis
776
+ object. This means if you're already using Redis in your app, ResqueAdmin
777
+ can re-use the existing connection.
778
+
779
+ String: `ResqueAdmin.redis = 'localhost:6379'`
780
+
781
+ Redis: `ResqueAdmin.redis = $redis`
782
+
783
+ For our rails app we have a `config/initializers/resque.rb` file where
784
+ we load `config/resque.yml` by hand and set the Redis information
785
+ appropriately.
786
+
787
+ Here's our `config/resque.yml`:
788
+
789
+ development: localhost:6379
790
+ test: localhost:6379
791
+ staging: redis1.se.github.com:6379
792
+ fi: localhost:6379
793
+ production: redis1.ae.github.com:6379
794
+
795
+ And our initializer:
796
+
797
+ ``` ruby
798
+ rails_root = ENV['RAILS_ROOT'] || File.dirname(__FILE__) + '/../..'
799
+ rails_env = ENV['RAILS_ENV'] || 'development'
800
+
801
+ resque_config = YAML.load_file(rails_root + '/config/resque.yml')
802
+ ResqueAdmin.redis = resque_config[rails_env]
803
+ ```
804
+
805
+ Easy peasy! Why not just use `RAILS_ROOT` and `RAILS_ENV`? Because
806
+ this way we can tell our Sinatra app about the config file:
807
+
808
+ $ RAILS_ENV=production resque-web rails_root/config/initializers/resque.rb
809
+
810
+ Now everyone is on the same page.
811
+
812
+ Also, you could disable jobs queueing by setting 'inline' attribute.
813
+ For example, if you want to run all jobs in the same process for cucumber, try:
814
+
815
+ ``` ruby
816
+ ResqueAdmin.inline = ENV['RAILS_ENV'] == "cucumber"
817
+ ```
818
+
819
+
820
+ Plugins and Hooks
821
+ -----------------
822
+
823
+ For a list of available plugins see
824
+ <http://wiki.github.com/resque/resque/plugins>.
825
+
826
+ If you'd like to write your own plugin, or want to customize ResqueAdmin
827
+ using hooks (such as `ResqueAdmin.after_fork`), see
828
+ [docs/HOOKS.md](http://github.com/resque/resque/blob/master/docs/HOOKS.md).
829
+
830
+
831
+ Namespaces
832
+ ----------
833
+
834
+ If you're running multiple, separate instances of ResqueAdmin you may want
835
+ to namespace the keyspaces so they do not overlap. This is not unlike
836
+ the approach taken by many memcached clients.
837
+
838
+ This feature is provided by the [redis-namespace][rs] library, which
839
+ ResqueAdmin uses by default to separate the keys it manages from other keys
840
+ in your Redis server.
841
+
842
+ Simply use the `ResqueAdmin.redis.namespace` accessor:
843
+
844
+ ``` ruby
845
+ ResqueAdmin.redis.namespace = "resque:GitHub"
846
+ ```
847
+
848
+ We recommend sticking this in your initializer somewhere after Redis
849
+ is configured.
850
+
851
+
852
+ Demo
853
+ ----
854
+
855
+ ResqueAdmin ships with a demo Sinatra app for creating jobs that are later
856
+ processed in the background.
857
+
858
+ Try it out by looking at the README, found at `examples/demo/README.markdown`.
859
+
860
+
861
+ Monitoring
862
+ ----------
863
+
864
+ ### god
865
+
866
+ If you're using god to monitor ResqueAdmin, we have provided example
867
+ configs in `examples/god/`. One is for starting / stopping workers,
868
+ the other is for killing workers that have been running too long.
869
+
870
+ ### monit
871
+
872
+ If you're using monit, `examples/monit/resque.monit` is provided free
873
+ of charge. This is **not** used by GitHub in production, so please
874
+ send patches for any tweaks or improvements you can make to it.
875
+
876
+
877
+ Questions
878
+ ---------
879
+
880
+ Please add them to the [FAQ](https://github.com/resque/resque/wiki/FAQ) or open an issue on this repo.
881
+
882
+
883
+ Development
884
+ -----------
885
+
886
+ Want to hack on ResqueAdmin?
887
+
888
+ First clone the repo and run the tests:
889
+
890
+ git clone git://github.com/resque/resque.git
891
+ cd resque
892
+ rake test
893
+
894
+ If the tests do not pass make sure you have Redis installed
895
+ correctly (though we make an effort to tell you if we feel this is the
896
+ case). The tests attempt to start an isolated instance of Redis to
897
+ run against.
898
+
899
+ Also make sure you've installed all the dependencies correctly. For
900
+ example, try loading the `redis-namespace` gem after you've installed
901
+ it:
902
+
903
+ $ irb
904
+ >> require 'rubygems'
905
+ => true
906
+ >> require 'redis/namespace'
907
+ => true
908
+
909
+ If you get an error requiring any of the dependencies, you may have
910
+ failed to install them or be seeing load path issues.
911
+
912
+
913
+ Contributing
914
+ ------------
915
+
916
+ Read [CONTRIBUTING.md](CONTRIBUTING.md) first.
917
+
918
+ Once you've made your great commits:
919
+
920
+ 1. [Fork][1] ResqueAdmin
921
+ 2. Create a topic branch - `git checkout -b my_branch`
922
+ 3. Push to your branch - `git push origin my_branch`
923
+ 4. Create a [Pull Request](http://help.github.com/pull-requests/) from your branch
924
+ 5. That's it!
925
+
926
+
927
+ Mailing List
928
+ ------------
929
+
930
+ This mailing list is no longer maintained. The archive can be found at <http://librelist.com/browser/resque/>.
931
+
932
+
933
+ Meta
934
+ ----
935
+
936
+ * Code: `git clone git://github.com/resque/resque.git`
937
+ * Home: <http://github.com/resque/resque>
938
+ * Docs: <http://rubydoc.info/gems/resque>
939
+ * Bugs: <http://github.com/resque/resque/issues>
940
+ * List: <resque@librelist.com>
941
+ * Chat: <irc://irc.freenode.net/resque>
942
+ * Gems: <http://gemcutter.org/gems/resque>
943
+
944
+ This project uses [Semantic Versioning][sv].
945
+
946
+
947
+ Author
948
+ ------
949
+
950
+ Chris Wanstrath :: chris@ozmm.org :: @defunkt
951
+
952
+ [0]: http://github.com/blog/542-introducing-resque
953
+ [1]: http://help.github.com/forking/
954
+ [2]: http://github.com/resque/resque/issues
955
+ [sv]: http://semver.org/
956
+ [rs]: http://github.com/resque/redis-namespace
957
+ [cb]: http://wiki.github.com/resque/resque/contributing