scotttam-resque 0.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (61) hide show
  1. data/.gitignore +3 -0
  2. data/.kick +26 -0
  3. data/CONTRIBUTORS +23 -0
  4. data/HISTORY.md +80 -0
  5. data/LICENSE +20 -0
  6. data/README.markdown +767 -0
  7. data/Rakefile +66 -0
  8. data/bin/resque +57 -0
  9. data/bin/resque-web +18 -0
  10. data/config.ru +14 -0
  11. data/deps.rip +6 -0
  12. data/examples/async_helper.rb +31 -0
  13. data/examples/demo/README.markdown +71 -0
  14. data/examples/demo/Rakefile +3 -0
  15. data/examples/demo/app.rb +38 -0
  16. data/examples/demo/config.ru +19 -0
  17. data/examples/demo/job.rb +22 -0
  18. data/examples/god/resque.god +53 -0
  19. data/examples/god/stale.god +26 -0
  20. data/examples/instance.rb +11 -0
  21. data/examples/simple.rb +30 -0
  22. data/init.rb +1 -0
  23. data/lib/resque/errors.rb +7 -0
  24. data/lib/resque/failure/base.rb +58 -0
  25. data/lib/resque/failure/hoptoad.rb +121 -0
  26. data/lib/resque/failure/multiple.rb +44 -0
  27. data/lib/resque/failure/redis.rb +33 -0
  28. data/lib/resque/failure.rb +63 -0
  29. data/lib/resque/helpers.rb +57 -0
  30. data/lib/resque/job.rb +146 -0
  31. data/lib/resque/server/public/idle.png +0 -0
  32. data/lib/resque/server/public/jquery-1.3.2.min.js +19 -0
  33. data/lib/resque/server/public/jquery.relatize_date.js +95 -0
  34. data/lib/resque/server/public/poll.png +0 -0
  35. data/lib/resque/server/public/ranger.js +24 -0
  36. data/lib/resque/server/public/reset.css +48 -0
  37. data/lib/resque/server/public/style.css +76 -0
  38. data/lib/resque/server/public/working.png +0 -0
  39. data/lib/resque/server/views/error.erb +1 -0
  40. data/lib/resque/server/views/failed.erb +35 -0
  41. data/lib/resque/server/views/key.erb +17 -0
  42. data/lib/resque/server/views/layout.erb +38 -0
  43. data/lib/resque/server/views/next_more.erb +10 -0
  44. data/lib/resque/server/views/overview.erb +4 -0
  45. data/lib/resque/server/views/queues.erb +46 -0
  46. data/lib/resque/server/views/stats.erb +62 -0
  47. data/lib/resque/server/views/workers.erb +78 -0
  48. data/lib/resque/server/views/working.erb +69 -0
  49. data/lib/resque/server.rb +187 -0
  50. data/lib/resque/stat.rb +53 -0
  51. data/lib/resque/tasks.rb +39 -0
  52. data/lib/resque/version.rb +3 -0
  53. data/lib/resque/worker.rb +453 -0
  54. data/lib/resque.rb +246 -0
  55. data/tasks/redis.rake +135 -0
  56. data/tasks/resque.rake +2 -0
  57. data/test/redis-test.conf +132 -0
  58. data/test/resque_test.rb +220 -0
  59. data/test/test_helper.rb +96 -0
  60. data/test/worker_test.rb +260 -0
  61. metadata +172 -0
data/README.markdown ADDED
@@ -0,0 +1,767 @@
1
+ Resque
2
+ ======
3
+
4
+ Resque is a Redis-backed library for creating background jobs, placing
5
+ those jobs on multiple queues, and processing them later.
6
+
7
+ Background jobs can be any Ruby class or module that responds to
8
+ `perform`. Your existing classes can easily be converted to background
9
+ jobs or you can create new classes specifically to do work. Or, you
10
+ can do both.
11
+
12
+ Resque is heavily inspired by DelayedJob (which rocks) and is
13
+ comprised of three parts:
14
+
15
+ 1. A Ruby library for creating, querying, and processing jobs
16
+ 2. A Rake task for starting a worker which processes jobs
17
+ 3. A Sinatra app for monitoring queues, jobs, and workers.
18
+
19
+ Resque workers can be distributed between multiple machines,
20
+ support priorities, are resilient to memory bloat / "leaks," are
21
+ optimized for REE (but work on MRI and JRuby), tell you what they're
22
+ doing, and expect failure.
23
+
24
+ Resque queues are persistent; support constant time, atomic push and
25
+ pop (thanks to Redis); provide visibility into their contents; and
26
+ store jobs as simple JSON packages.
27
+
28
+ The Resque frontend tells you what workers are doing, what workers are
29
+ not doing, what queues you're using, what's in those queues, provides
30
+ general usage stats, and helps you track failures.
31
+
32
+
33
+ The Blog Post
34
+ -------------
35
+
36
+ For the backstory, philosophy, and history of Resque's beginnings,
37
+ please see [the blog post][0].
38
+
39
+
40
+ Overview
41
+ --------
42
+
43
+ Resque allows you to create jobs and place them on a queue, then,
44
+ later, pull those jobs off the queue and process them.
45
+
46
+ Resque jobs are Ruby classes (or modules) which respond to the
47
+ `perform` method. Here's an example:
48
+
49
+ class Archive
50
+ @queue = :file_serve
51
+
52
+ def self.perform(repo_id, branch = 'master')
53
+ repo = Repository.find(repo_id)
54
+ repo.create_archive(branch)
55
+ end
56
+ end
57
+
58
+ The `@queue` class instance variable determines which queue `Archive`
59
+ jobs will be placed in. Queues are arbitrary and created on the fly -
60
+ you can name them whatever you want and have as many as you want.
61
+
62
+ To place an `Archive` job on the `file_serve` queue, we might add this
63
+ to our application's pre-existing `Repository` class:
64
+
65
+ class Repository
66
+ def async_create_archive(branch)
67
+ Resque.enqueue(Archive, self.id, branch)
68
+ end
69
+ end
70
+
71
+ Now when we call `repo.async_create_archive('masterbrew')` in our
72
+ application, a job will be created and placed on the `file_serve`
73
+ queue.
74
+
75
+ Later, a worker will run something like this code to process the job:
76
+
77
+ klass, args = Resque.reserve(:file_serve)
78
+ klass.perform(*args) if klass.respond_to? :perform
79
+
80
+ Which translates to:
81
+
82
+ Archive.perform(44, 'masterbrew')
83
+
84
+ Let's start a worker to run `file_serve` jobs:
85
+
86
+ $ cd app_root
87
+ $ QUEUE=file_serve rake resque:work
88
+
89
+ This starts one Resque worker and tells it to work off the
90
+ `file_serve` queue. As soon as it's ready it'll try to run the
91
+ `Resque.reserve` code snippet above and process jobs until it can't
92
+ find any more, at which point it will sleep for a small period and
93
+ repeatedly poll the queue for more jobs.
94
+
95
+ Workers can be given multiple queues (a "queue list") and run on
96
+ multiple machines. In fact they can be run anywhere with network
97
+ access to the Redis server.
98
+
99
+
100
+ Jobs
101
+ ----
102
+
103
+ What should you run in the background? Anything that takes any time at
104
+ all. Slow INSERT statements, disk manipulating, data processing, etc.
105
+
106
+ At GitHub we use Resque to process the following types of jobs:
107
+
108
+ * Warming caches
109
+ * Counting disk usage
110
+ * Building tarballs
111
+ * Building Rubygems
112
+ * Firing off web hooks
113
+ * Creating events in the db and pre-caching them
114
+ * Building graphs
115
+ * Deleting users
116
+ * Updating our search index
117
+
118
+ As of writing we have about 35 different types of background jobs.
119
+
120
+ Keep in mind that you don't need a web app to use Resque - we just
121
+ mention "foreground" and "background" because they make conceptual
122
+ sense. You could easily be spidering sites and sticking data which
123
+ needs to be crunched later into a queue.
124
+
125
+
126
+ ### Persistence
127
+
128
+ Jobs are persisted to queues as JSON objects. Let's take our `Archive`
129
+ example from above. We'll run the following code to create a job:
130
+
131
+ repo = Repository.find(44)
132
+ repo.async_create_archive('masterbrew')
133
+
134
+ The following JSON will be stored in the `file_serve` queue:
135
+
136
+ {
137
+ 'class': 'Archive',
138
+ 'args': [ 44, 'masterbrew' ]
139
+ }
140
+
141
+ Because of this your jobs must only accept arguments that can be JSON encoded.
142
+
143
+ So instead of doing this:
144
+
145
+ Resque.enqueue(Archive, self, branch)
146
+
147
+ do this:
148
+
149
+ Resque.enqueue(Archive, self.id, branch)
150
+
151
+ This is why our above example (and all the examples in `examples/`)
152
+ uses object IDs instead of passing around the objects.
153
+
154
+ While this is less convenient than just sticking a marshaled object
155
+ in the database, it gives you a slight advantage: your jobs will be
156
+ run against the most recent version of an object because they need to
157
+ pull from the DB or cache.
158
+
159
+ If your jobs were run against marshaled objects, they could
160
+ potentially be operating on a stale record with out-of-date information.
161
+
162
+
163
+ ### send_later / async
164
+
165
+ Want something like DelayedJob's `send_later` or the ability to use
166
+ instance methods instead of just methods for jobs? See the `examples/`
167
+ directory for goodies.
168
+
169
+ We plan to provide first class `async` support in a future release.
170
+
171
+
172
+ ### Failure
173
+
174
+ If a job raises an exception, it is logged and handed off to the
175
+ `Resque::Failure` module. Failures are logged either locally in Redis
176
+ or using some different backend.
177
+
178
+ For example, Resque ships with Hoptoad support.
179
+
180
+ Keep this in mind when writing your jobs: you may want to throw
181
+ exceptions you would not normally throw in order to assist debugging.
182
+
183
+
184
+ Workers
185
+ -------
186
+
187
+ Resque workers are rake tasks that run forever. They basically do this:
188
+
189
+ start
190
+ loop do
191
+ if job = reserve
192
+ job.process
193
+ else
194
+ sleep 5
195
+ end
196
+ end
197
+ shutdown
198
+
199
+ Starting a worker is simple. Here's our example from earlier:
200
+
201
+ $ QUEUE=file_serve rake resque:work
202
+
203
+ By default Resque won't know about your application's
204
+ environment. That is, it won't be able to find and run your jobs - it
205
+ needs to load your application into memory.
206
+
207
+ If we've installed Resque as a Rails plugin, we might run this command
208
+ from our RAILS_ROOT:
209
+
210
+ $ QUEUE=file_serve rake environment resque:work
211
+
212
+ This will load the environment before starting a worker. Alternately
213
+ we can define a `resque:setup` task with a dependency on the
214
+ `environment` rake task:
215
+
216
+ task "resque:setup" => :environment
217
+
218
+ GitHub's setup task looks like this:
219
+
220
+ task "resque:setup" => :environment do
221
+ Grit::Git.git_timeout = 10.minutes
222
+ end
223
+
224
+ We don't want the `git_timeout` as high as 10 minutes in our web app,
225
+ but in the Resque workers it's fine.
226
+
227
+
228
+ ### Logging
229
+
230
+ Workers support basic logging to STDOUT. If you start them with the
231
+ `VERBOSE` env variable set, they will print basic debugging
232
+ information. You can also set the `VVERBOSE` (very verbose) env
233
+ variable.
234
+
235
+ $ VVERBOSE=1 QUEUE=file_serve rake environment resque:work
236
+
237
+
238
+ ### Priorities and Queue Lists
239
+
240
+ Resque doesn't support numeric priorities but instead uses the order
241
+ of queues you give it. We call this list of queues the "queue list."
242
+
243
+ Let's say we add a `warm_cache` queue in addition to our `file_serve`
244
+ queue. We'd now start a worker like so:
245
+
246
+ $ QUEUES=file_serve,warm_cache rake resque:work
247
+
248
+ When the worker looks for new jobs, it will first check
249
+ `file_serve`. If it finds a job, it'll process it then check
250
+ `file_serve` again. It will keep checking `file_serve` until no more
251
+ jobs are available. At that point, it will check `warm_cache`. If it
252
+ finds a job it'll process it then check `file_serve` (repeating the
253
+ whole process).
254
+
255
+ In this way you can prioritize certain queues. At GitHub we start our
256
+ workers with something like this:
257
+
258
+ $ QUEUES=critical,archive,high,low rake resque:work
259
+
260
+ Notice the `archive` queue - it is specialized and in our future
261
+ architecture will only be run from a single machine.
262
+
263
+ At that point we'll start workers on our generalized background
264
+ machines with this command:
265
+
266
+ $ QUEUES=critical,high,low rake resque:work
267
+
268
+ And workers on our specialized archive machine with this command:
269
+
270
+ $ QUEUE=archive rake resque:work
271
+
272
+
273
+ ### Running All Queues
274
+
275
+ If you want your workers to work off of every queue, including new
276
+ queues created on the fly, you can use a splat:
277
+
278
+ $ QUEUE=* rake resque:work
279
+
280
+ Queues will be processed in alphabetical order.
281
+
282
+
283
+ ### Running Multiple Workers
284
+
285
+ At GitHub we use god to start and stop multiple workers. A sample god
286
+ configuration file is included under `examples/god`. We recommend this
287
+ method.
288
+
289
+ If you'd like to run multiple workers in development mode, you can do
290
+ so using the `resque:workers` rake task:
291
+
292
+ $ COUNT=5 QUEUE=* rake resque:workers
293
+
294
+ This will spawn five Resque workers, each in its own thread. Hitting
295
+ ctrl-c should be sufficient to stop them all.
296
+
297
+
298
+ ### Forking
299
+
300
+ On certain platforms, when a Resque worker reserves a job it
301
+ immediately forks a child process. The child processes the job then
302
+ exits. When the child has exited successfully, the worker reserves
303
+ another job and repeats the process.
304
+
305
+ Why?
306
+
307
+ Because Resque assumes chaos.
308
+
309
+ Resque assumes your background workers will lock up, run too long, or
310
+ have unwanted memory growth.
311
+
312
+ If Resque workers processed jobs themselves, it'd be hard to whip them
313
+ into shape. Let's say one is using too much memory: you send it a
314
+ signal that says "shutdown after you finish processing the current
315
+ job," and it does so. It then starts up again - loading your entire
316
+ application environment. This adds useless CPU cycles and causes a
317
+ delay in queue processing.
318
+
319
+ Plus, what if it's using too much memory and has stopped responding to
320
+ signals?
321
+
322
+ Thanks to Resque's parent / child architecture, jobs that use too much memory
323
+ release that memory upon completion. No unwanted growth.
324
+
325
+ And what if a job is running too long? You'd need to `kill -9` it then
326
+ start the worker again. With Resque's parent / child architecture you
327
+ can tell the parent to forcefully kill the child then immediately
328
+ start processing more jobs. No startup delay or wasted cycles.
329
+
330
+ The parent / child architecture helps us keep tabs on what workers are
331
+ doing, too. By eliminating the need to `kill -9` workers we can have
332
+ parents remove themselves from the global listing of workers. If we
333
+ just ruthlessly killed workers, we'd need a separate watchdog process
334
+ to add and remove them to the global listing - which becomes
335
+ complicated.
336
+
337
+ Workers instead handle their own state.
338
+
339
+
340
+ ### Parents and Children
341
+
342
+ Here's a parent / child pair doing some work:
343
+
344
+ $ ps -e -o pid,command | grep [r]esque
345
+ 92099 resque: Forked 92102 at 1253142769
346
+ 92102 resque: Processing file_serve since 1253142769
347
+
348
+ You can clearly see that process 92099 forked 92102, which has been
349
+ working since 1253142769.
350
+
351
+ (By advertising the time they began processing you can easily use monit
352
+ or god to kill stale workers.)
353
+
354
+ When a parent process is idle, it lets you know what queues it is
355
+ waiting for work on:
356
+
357
+ $ ps -e -o pid,command | grep [r]esque
358
+ 92099 resque: Waiting for file_serve,warm_cache
359
+
360
+
361
+ ### Signals
362
+
363
+ Resque workers respond to a few different signals:
364
+
365
+ * `QUIT` - Wait for child to finish processing then exit
366
+ * `TERM` / `INT` - Immediately kill child then exit
367
+ * `USR1` - Immediately kill child but don't exit
368
+ * `USR2` - Don't start to process any new jobs
369
+ * `CONT` - Start to process new jobs again after a USR2
370
+
371
+ If you want to gracefully shutdown a Resque worker, use `QUIT`.
372
+
373
+ If you want to kill a stale or stuck child, use `USR1`. Processing
374
+ will continue as normal unless the child was not found. In that case
375
+ Resque assumes the parent process is in a bad state and shuts down.
376
+
377
+ If you want to kill a stale or stuck child and shutdown, use `TERM`
378
+
379
+ If you want to stop processing jobs, but want to leave the worker running
380
+ (for example, to temporarily alleviate load), use `USR2` to stop processing,
381
+ then `CONT` to start it again.
382
+
383
+ ### Mysql::Error: MySQL server has gone away
384
+
385
+ If your workers remain idle for too long they may lose their MySQL
386
+ connection. If that happens we recommend using [this
387
+ Gist](http://gist.github.com/238999).
388
+
389
+
390
+ The Front End
391
+ -------------
392
+
393
+ Resque comes with a Sinatra-based front end for seeing what's up with
394
+ your queue.
395
+
396
+ ![The Front End](http://img.skitch.com/20091104-tqh5pgkwgbskjbk7qbtmpesnyw.jpg)
397
+
398
+ ### Standalone
399
+
400
+ If you've installed Resque as a gem running the front end standalone is easy:
401
+
402
+ $ resque-web
403
+
404
+ It's a thin layer around `rackup` so it's configurable as well:
405
+
406
+ $ resque-web -p 8282
407
+
408
+ If you have a Resque config file you want evaluated just pass it to
409
+ the script as the final argument:
410
+
411
+ $ resque-web -p 8282 rails_root/config/initializers/resque.rb
412
+
413
+ ### Passenger
414
+
415
+ Using Passenger? Resque ships with a `config.ru` you can use. See
416
+ Phusion's guide:
417
+
418
+ <http://www.modrails.com/documentation/Users%20guide.html#_deploying_a_rack_based_ruby_application>
419
+
420
+ ### Rack::URLMap
421
+
422
+ If you want to load Resque on a subpath, possibly alongside other
423
+ apps, it's easy to do with Rack's `URLMap`:
424
+
425
+ require 'resque/server'
426
+
427
+ run Rack::URLMap.new \
428
+ "/" => Your::App.new,
429
+ "/resque" => Resque::Server.new
430
+
431
+ Check `examples/demo/config.ru` for a functional example (including
432
+ HTTP basic auth).
433
+
434
+
435
+ Resque vs DelayedJob
436
+ --------------------
437
+
438
+ How does Resque compare to DelayedJob, and why would you choose one
439
+ over the other?
440
+
441
+ * Resque supports multiple queues
442
+ * DelayedJob supports finer grained priorities
443
+ * Resque workers are resilient to memory leaks / bloat
444
+ * DelayedJob workers are extremely simple and easy to modify
445
+ * Resque requires Redis
446
+ * DelayedJob requires ActiveRecord
447
+ * Resque can only place JSONable Ruby objects on a queue as arguments
448
+ * DelayedJob can place _any_ Ruby object on its queue as arguments
449
+ * Resque includes a Sinatra app for monitoring what's going on
450
+ * DelayedJob can be queried from within your Rails app if you want to
451
+ add an interface
452
+
453
+ If you're doing Rails development, you already have a database and
454
+ ActiveRecord. DelayedJob is super easy to setup and works great.
455
+ GitHub used it for many months to process almost 200 million jobs.
456
+
457
+ Choose Resque if:
458
+
459
+ * You need multiple queues
460
+ * You don't care / dislike numeric priorities
461
+ * You don't need to persist every Ruby object ever
462
+ * You have potentially huge queues
463
+ * You want to see what's going on
464
+ * You expect a lot of failure / chaos
465
+ * You can setup Redis
466
+ * You're not running short on RAM
467
+
468
+ Choose DelayedJob if:
469
+
470
+ * You like numeric priorities
471
+ * You're not doing a gigantic amount of jobs each day
472
+ * Your queue stays small and nimble
473
+ * There is not a lot failure / chaos
474
+ * You want to easily throw anything on the queue
475
+ * You don't want to setup Redis
476
+
477
+ In no way is Resque a "better" DelayedJob, so make sure you pick the
478
+ tool that's best for your app.
479
+
480
+
481
+ Installing Redis
482
+ ----------------
483
+
484
+ Resque uses Redis' lists for its queues. It also stores worker state
485
+ data in Redis.
486
+
487
+ #### Homebrew
488
+
489
+ If you're on OS X, Homebrew is the simplest way to install Redis:
490
+
491
+ $ brew install redis
492
+ $ redis-server /usr/local/etc/redis.conf
493
+
494
+ You now have a Redis daemon running on 6379.
495
+
496
+ #### Via Resque
497
+
498
+ Resque includes Rake tasks (thanks to Ezra's redis-rb) that will
499
+ install and run Redis for you:
500
+
501
+ $ git clone git://github.com/defunkt/resque.git
502
+ $ cd resque
503
+ $ rake redis:install dtach:install
504
+ $ rake redis:start
505
+
506
+ Or, if you don't have admin access on your machine:
507
+
508
+ $ git clone git://github.com/defunkt/resque.git
509
+ $ cd resque
510
+ $ PREFIX=<your_prefix> rake redis:install dtach:install
511
+ $ rake redis:start
512
+
513
+ You now have Redis running on 6379. Wait a second then hit ctrl-\ to
514
+ detach and keep it running in the background.
515
+
516
+ The demo is probably the best way to figure out how to put the parts
517
+ together. But, it's not that hard.
518
+
519
+
520
+ Resque Dependencies
521
+ -------------------
522
+
523
+ gem install redis redis-namespace yajl-ruby
524
+
525
+ If you cannot install `yajl-ruby` (JRuby?), you can install the `json`
526
+ gem and Resque will use it instead.
527
+
528
+
529
+ Installing Resque
530
+ -----------------
531
+
532
+ ### In a Rack app, as a gem
533
+
534
+ First install the gem.
535
+
536
+ $ gem install resque
537
+
538
+ Next include it in your application.
539
+
540
+ require 'resque'
541
+
542
+ Now start your application:
543
+
544
+ rackup config.ru
545
+
546
+ That's it! You can now create Resque jobs from within your app.
547
+
548
+ To start a worker, create a Rakefile in your app's root (or add this
549
+ to an existing Rakefile):
550
+
551
+ require 'your/app'
552
+ require 'resque/tasks'
553
+
554
+ Now:
555
+
556
+ $ QUEUE=* rake resque:work
557
+
558
+ Alternately you can define a `resque:setup` hook in your Rakefile if you
559
+ don't want to load your app every time rake runs.
560
+
561
+
562
+ ### In a Rails app, as a gem
563
+
564
+ First install the gem.
565
+
566
+ $ gem install resque
567
+
568
+ Next include it in your application.
569
+
570
+ $ cat config/initializers/load_resque.rb
571
+ require 'resque'
572
+
573
+ Now start your application:
574
+
575
+ $ ./script/server
576
+
577
+ That's it! You can now create Resque jobs from within your app.
578
+
579
+ To start a worker, add this to your Rakefile in `RAILS_ROOT`:
580
+
581
+ require 'resque/tasks'
582
+
583
+ Now:
584
+
585
+ $ QUEUE=* rake environment resque:work
586
+
587
+ Don't forget you can define a `resque:setup` hook in
588
+ `lib/tasks/whatever.rake` that loads the `environment` task every time.
589
+
590
+
591
+ ### In a Rails app, as a plugin
592
+
593
+ $ ./script/plugin install git://github.com/defunkt/resque
594
+
595
+ That's it! Resque will automatically be available when your Rails app
596
+ loads.
597
+
598
+ To start a worker:
599
+
600
+ $ QUEUE=* rake environment resque:work
601
+
602
+ Don't forget you can define a `resque:setup` hook in
603
+ `lib/tasks/whatever.rake` that loads the `environment` task every time.
604
+
605
+
606
+ Configuration
607
+ -------------
608
+
609
+ You may want to change the Redis host and port Resque connects to, or
610
+ set various other options at startup.
611
+
612
+ Resque has a `redis` setter which can be given a string or a Redis
613
+ object. This means if you're already using Redis in your app, Resque
614
+ can re-use the existing connection.
615
+
616
+ String: `Resque.redis = 'localhost:6379'`
617
+
618
+ Redis: `Resque.redis = $redis`
619
+
620
+ For our rails app we have a `config/initializers/resque.rb` file where
621
+ we load `config/resque.yml` by hand and set the Redis information
622
+ appropriately.
623
+
624
+ Here's our `config/resque.yml`:
625
+
626
+ development: localhost:6379
627
+ test: localhost:6379
628
+ staging: redis1.se.github.com:6379
629
+ fi: localhost:6379
630
+ production: redis1.ae.github.com:6379
631
+
632
+ And our initializer:
633
+
634
+ rails_root = ENV['RAILS_ROOT'] || File.dirname(__FILE__) + '/../..'
635
+ rails_env = ENV['RAILS_ENV'] || 'development'
636
+
637
+ resque_config = YAML.load_file(rails_root + '/config/resque.yml')
638
+ Resque.redis = resque_config[rails_env]
639
+
640
+ Easy peasy! Why not just use `RAILS_ROOT` and `RAILS_ENV`? Because
641
+ this way we can tell our Sinatra app about the config file:
642
+
643
+ $ RAILS_ENV=production resque-web rails_root/config/initializers/resque.rb
644
+
645
+ Now everyone is on the same page.
646
+
647
+ If you wish to have a Proc called before the worker forks for the first time, you
648
+ can add it in the initializer like so:
649
+
650
+ Resque.before_fork = Proc.new { puts "CALL ME ONCE BEFORE THE WORKER FORKS THE FIRST TIME" }
651
+
652
+
653
+ Namespaces
654
+ ----------
655
+
656
+ If you're running multiple, separate instances of Resque you may want
657
+ to namespace the keyspaces so they do not overlap. This is not unlike
658
+ the approach taken by many memcached clients.
659
+
660
+ This feature is provided by the [redis-namespace][rs] library, which
661
+ Resque uses by default to separate the keys it manages from other keys
662
+ in your Redis server.
663
+
664
+ Simply use the `Resque.redis.namespace` accessor:
665
+
666
+ Resque.redis.namespace = "resque:GitHub"
667
+
668
+ We recommend sticking this in your initializer somewhere after Redis
669
+ is configured.
670
+
671
+
672
+ Demo
673
+ ----
674
+
675
+ Resque ships with a demo Sinatra app for creating jobs that are later
676
+ processed in the background.
677
+
678
+ Try it out by looking at the README, found at `examples/demo/README.markdown`.
679
+
680
+
681
+ Monitoring
682
+ ----------
683
+
684
+ If you're using god to monitor Resque, we have provided example
685
+ configs in `examples/god/`. One is for starting / stopping workers,
686
+ the other is for killing workers that have been running too long.
687
+
688
+
689
+ Development
690
+ -----------
691
+
692
+ Want to hack on Resque?
693
+
694
+ First clone the repo and run the tests:
695
+
696
+ git clone git://github.com/defunkt/resque.git
697
+ cd resque
698
+ rake test
699
+
700
+ If the tests do not pass make sure you have Redis installed
701
+ correctly (though we make an effort to tell you if we feel this is the
702
+ case). The tests attempt to start an isolated instance of Redis to
703
+ run against.
704
+
705
+ Also make sure you've installed all the dependencies correctly. For
706
+ example, try loading the `redis-namespace` gem after you've installed
707
+ it:
708
+
709
+ $ irb
710
+ >> require 'rubygems'
711
+ => true
712
+ >> require 'redis/namespace'
713
+ => true
714
+
715
+ If you get an error requiring any of the dependencies, you may have
716
+ failed to install them or be seeing load path issues.
717
+
718
+ Feel free to ping the mailing list with your problem and we'll try to
719
+ sort it out.
720
+
721
+
722
+ Contributing
723
+ ------------
724
+
725
+ Once you've made your great commits:
726
+
727
+ 1. [Fork][1] Resque
728
+ 2. Create a topic branch - `git checkout -b my_branch`
729
+ 3. Push to your branch - `git push origin my_branch`
730
+ 4. Create an [Issue][2] with a link to your branch
731
+ 5. That's it!
732
+
733
+
734
+ Mailing List
735
+ ------------
736
+
737
+ To join the list simply send an email to <resque@librelist.com>. This
738
+ will subscribe you and send you information about your subscription,
739
+ include unsubscribe information.
740
+
741
+ The archive can be found at <http://librelist.com/browser/>.
742
+
743
+
744
+ Meta
745
+ ----
746
+
747
+ * Code: `git clone git://github.com/defunkt/resque.git`
748
+ * Home: <http://github.com/defunkt/resque>
749
+ * Docs: <http://defunkt.github.com/resque/>
750
+ * Bugs: <http://github.com/defunkt/resque/issues>
751
+ * List: <resque@librelist.com>
752
+ * Chat: <irc://irc.freenode.net/resque>
753
+ * Gems: <http://gemcutter.org/gems/resque>
754
+
755
+ This project uses [Semantic Versioning][sv].
756
+
757
+
758
+ Author
759
+ ------
760
+
761
+ Chris Wanstrath :: chris@ozmm.org :: @defunkt
762
+
763
+ [0]: http://github.com/blog/542-introducing-resque
764
+ [1]: http://help.github.com/forking/
765
+ [2]: http://github.com/defunkt/resque/issues
766
+ [sv]: http://semver.org/
767
+ [rs]: http://github.com/defunkt/redis-namespace