resque 1.23.0 → 2.6.0

data/README.markdown

@@ -1,10 +1,18 @@
 Resque
 ======
 
+[![Gem Version](https://badge.fury.io/rb/resque.svg)](https://rubygems.org/gems/resque)
+[![Build Status](https://github.com/resque/resque/actions/workflows/ci.yml/badge.svg)](https://github.com/resque/resque/actions/workflows/ci.yml)
+
+Introduction
+------------
+
 Resque (pronounced like "rescue") is a Redis-backed library for creating
 background jobs, placing those jobs on multiple queues, and processing
 them later.
 
+For the backstory, philosophy, and history of Resque's beginnings, please see [the blog post](http://github.com/blog/542-introducing-resque) (2009).
+
 Background jobs can be any Ruby class or module that responds to
 `perform`. Your existing classes can easily be converted to background
 jobs or you can create new classes specifically to do work. Or, you
@@ -17,10 +25,11 @@ three parts:
 2. A Rake task for starting a worker which processes jobs
 3. A Sinatra app for monitoring queues, jobs, and workers.
 
-Resque workers can be distributed between multiple machines,
-support priorities, are resilient to memory bloat / "leaks," are
-optimized for REE (but work on MRI and JRuby), tell you what they're
-doing, and expect failure.
+Resque workers can be  given multiple queues (a "queue list"),
+distributed between multiple machines,
+run anywhere with network access to the Redis server,
+support priorities, are resilient to memory bloat / "leaks,"
+tell you what they're doing, and expect failure.
 
 Resque queues are persistent; support constant time, atomic push and
 pop (thanks to Redis); provide visibility into their contents; and
@@ -30,19 +39,18 @@ The Resque frontend tells you what workers are doing, what workers are
 not doing, what queues you're using, what's in those queues, provides
 general usage stats, and helps you track failures.
 
+Resque now supports Ruby 2.3.0 and above.
+We will also only be supporting Redis 3.0 and above going forward.
 
-The Blog Post
--------------
-
-For the backstory, philosophy, and history of Resque's beginnings,
-please see [the blog post][0].
+### Note on the future of Resque
 
+Would you like to be involved in Resque? Do you have thoughts about what
+Resque should be and do going forward? There's currently an [open discussion here](https://github.com/resque/resque/issues/1759)
+on just that topic, so please feel free to join in. We'd love to hear your thoughts
+and/or have people volunteer to be a part of the project!
 
-Overview
---------
-
-Resque allows you to create jobs and place them on a queue, then,
-later, pull those jobs off the queue and process them.
+Example
+-------
 
 Resque jobs are Ruby classes (or modules) which respond to the
 `perform` method. Here's an example:
@@ -102,105 +110,42 @@ This starts one Resque worker and tells it to work off the
 find any more, at which point it will sleep for a small period and
 repeatedly poll the queue for more jobs.
 
-Workers can be given multiple queues (a "queue list") and run on
-multiple machines. In fact they can be run anywhere with network
-access to the Redis server.
-
-
-Jobs
-----
-
-What should you run in the background? Anything that takes any time at
-all. Slow INSERT statements, disk manipulating, data processing, etc.
-
-At GitHub we use Resque to process the following types of jobs:
+Installation
+------------
 
-* Warming caches
-* Counting disk usage
-* Building tarballs
-* Building Rubygems
-* Firing off web hooks
-* Creating events in the db and pre-caching them
-* Building graphs
-* Deleting users
-* Updating our search index
+Add the gem to your Gemfile:
 
-As of writing we have about 35 different types of background jobs.
+    gem 'resque'
 
-Keep in mind that you don't need a web app to use Resque - we just
-mention "foreground" and "background" because they make conceptual
-sense. You could easily be spidering sites and sticking data which
-needs to be crunched later into a queue.
+Next, install it with Bundler:
 
+    $ bundle
 
-### Persistence
+#### Rack
 
-Jobs are persisted to queues as JSON objects. Let's take our `Archive`
-example from above. We'll run the following code to create a job:
+In your Rakefile, or some other file in `lib/tasks` (ex: `lib/tasks/resque.rake`), load the resque rake tasks:
 
 ``` ruby
-repo = Repository.find(44)
-repo.async_create_archive('masterbrew')
-```
-
-The following JSON will be stored in the `file_serve` queue:
-
-``` javascript
-{
-    'class': 'Archive',
-    'args': [ 44, 'masterbrew' ]
-}
+require 'resque'
+require 'resque/tasks'
+require 'your/app' # Include this line if you want your workers to have access to your application
 ```
 
-Because of this your jobs must only accept arguments that can be JSON encoded.
-
-So instead of doing this:
-
-``` ruby
-Resque.enqueue(Archive, self, branch)
-```
+#### Rails
 
-do this:
+To make resque specific changes, you can override the `resque:setup` job in `lib/tasks` (ex: `lib/tasks/resque.rake`). GitHub's setup task looks like this:
 
 ``` ruby
-Resque.enqueue(Archive, self.id, branch)
+task "resque:setup" => :environment do
+  Grit::Git.git_timeout = 10.minutes
+end
 ```
 
-This is why our above example (and all the examples in `examples/`)
-uses object IDs instead of passing around the objects.
-
-While this is less convenient than just sticking a marshaled object
-in the database, it gives you a slight advantage: your jobs will be
-run against the most recent version of an object because they need to
-pull from the DB or cache.
-
-If your jobs were run against marshaled objects, they could
-potentially be operating on a stale record with out-of-date information.
-
-
-### send_later / async
-
-Want something like DelayedJob's `send_later` or the ability to use
-instance methods instead of just methods for jobs? See the `examples/`
-directory for goodies.
-
-We plan to provide first class `async` support in a future release.
-
-
-### Failure
-
-If a job raises an exception, it is logged and handed off to the
-`Resque::Failure` module. Failures are logged either locally in Redis
-or using some different backend.
-
-For example, Resque ships with Hoptoad support.
-
-Keep this in mind when writing your jobs: you may want to throw
-exceptions you would not normally throw in order to assist debugging.
-
+We don't want the `git_timeout` as high as 10 minutes in our web app,
+but in the Resque workers it's fine.
 
-Workers
--------
+Running Workers
+---------------
 
 Resque workers are rake tasks that run forever. They basically do this:
 
@@ -210,79 +155,24 @@ loop do
   if job = reserve
     job.process
   else
-    sleep 5 # Polling frequency = 5 
+    sleep 5 # Polling frequency = 5
   end
 end
 shutdown
 ```
 
-Starting a worker is simple. Here's our example from earlier:
-
-    $ QUEUE=file_serve rake resque:work
-
-By default Resque won't know about your application's
-environment. That is, it won't be able to find and run your jobs - it
-needs to load your application into memory.
-
-If we've installed Resque as a Rails plugin, we might run this command
-from our RAILS_ROOT:
-
-    $ QUEUE=file_serve rake environment resque:work
-
-This will load the environment before starting a worker. Alternately
-we can define a `resque:setup` task with a dependency on the
-`environment` rake task:
-
-``` ruby
-task "resque:setup" => :environment
-```
-
-GitHub's setup task looks like this:
-
-``` ruby
-task "resque:setup" => :environment do
-  Grit::Git.git_timeout = 10.minutes
-end
-```
-
-We don't want the `git_timeout` as high as 10 minutes in our web app,
-but in the Resque workers it's fine.
-
-
-### Logging
+Starting a worker is simple:
 
-Workers support basic logging to STDOUT. If you start them with the
-`VERBOSE` env variable set, they will print basic debugging
-information. You can also set the `VVERBOSE` (very verbose) env
-variable.
-
-    $ VVERBOSE=1 QUEUE=file_serve rake environment resque:work
-
-### Process IDs (PIDs)
-
-There are scenarios where it's helpful to record the PID of a resque
-worker process.  Use the PIDFILE option for easy access to the PID:
-
-    $ PIDFILE=./resque.pid QUEUE=file_serve rake environment resque:work
-
-### Running in the background
-
-(Only supported with ruby >= 1.9). There are scenarios where it's helpful for
-the resque worker to run itself in the background (usually in combination with
-PIDFILE).  Use the BACKGROUND option so that rake will return as soon as the
-worker is started.
-
-    $ PIDFILE=./resque.pid BACKGROUND=yes QUEUE=file_serve \
-        rake environment resque:work
+    $ QUEUE=* rake resque:work
 
-### Polling frequency
+Or, you can start multiple workers:
 
-You can pass an INTERVAL option which is a float representing the polling frequency. 
-The default is 5 seconds, but for a semi-active app you may want to use a smaller value.
+    $ COUNT=2 QUEUE=* rake resque:workers
 
-    $ INTERVAL=0.1 QUEUE=file_serve rake environment resque:work
+This will spawn two Resque workers, each in its own process. Hitting
+ctrl-c should be sufficient to stop them all.
 
-### Priorities and Queue Lists
+#### Priorities and Queue Lists
 
 Resque doesn't support numeric priorities but instead uses the order
 of queues you give it. We call this list of queues the "queue list."
@@ -316,8 +206,7 @@ And workers on our specialized archive machine with this command:
 
     $ QUEUE=archive rake resque:work
 
-
-### Running All Queues
+#### Running All Queues
 
 If you want your workers to work off of every queue, including new
 queues created on the fly, you can use a splat:
@@ -326,113 +215,51 @@ queues created on the fly, you can use a splat:
 
 Queues will be processed in alphabetical order.
 
+Or, prioritize some queues above `*`:
 
-### Running Multiple Workers
+    # QUEUE=critical,* rake resque:work
 
-At GitHub we use god to start and stop multiple workers. A sample god
-configuration file is included under `examples/god`. We recommend this
-method.
+#### Running All Queues Except for Some
 
-If you'd like to run multiple workers in development mode, you can do
-so using the `resque:workers` rake task:
+If you want your workers to work off of all queues except for some,
+you can use negation:
 
-    $ COUNT=5 QUEUE=* rake resque:workers
+    $ QUEUE=*,!low rake resque:work
 
-This will spawn five Resque workers, each in its own thread. Hitting
-ctrl-c should be sufficient to stop them all.
-
-
-### Forking
-
-On certain platforms, when a Resque worker reserves a job it
-immediately forks a child process. The child processes the job then
-exits. When the child has exited successfully, the worker reserves
-another job and repeats the process.
-
-Why?
-
-Because Resque assumes chaos.
-
-Resque assumes your background workers will lock up, run too long, or
-have unwanted memory growth.
-
-If Resque workers processed jobs themselves, it'd be hard to whip them
-into shape. Let's say one is using too much memory: you send it a
-signal that says "shutdown after you finish processing the current
-job," and it does so. It then starts up again - loading your entire
-application environment. This adds useless CPU cycles and causes a
-delay in queue processing.
-
-Plus, what if it's using too much memory and has stopped responding to
-signals?
-
-Thanks to Resque's parent / child architecture, jobs that use too much memory
-release that memory upon completion. No unwanted growth.
-
-And what if a job is running too long? You'd need to `kill -9` it then
-start the worker again. With Resque's parent / child architecture you
-can tell the parent to forcefully kill the child then immediately
-start processing more jobs. No startup delay or wasted cycles.
-
-The parent / child architecture helps us keep tabs on what workers are
-doing, too. By eliminating the need to `kill -9` workers we can have
-parents remove themselves from the global listing of workers. If we
-just ruthlessly killed workers, we'd need a separate watchdog process
-to add and remove them to the global listing - which becomes
-complicated.
-
-Workers instead handle their own state.
-
-
-### Parents and Children
-
-Here's a parent / child pair doing some work:
-
-    $ ps -e -o pid,command | grep [r]esque
-    92099 resque: Forked 92102 at 1253142769
-    92102 resque: Processing file_serve since 1253142769
-
-You can clearly see that process 92099 forked 92102, which has been
-working since 1253142769.
-
-(By advertising the time they began processing you can easily use monit
-or god to kill stale workers.)
+Negated globs also work. The following will instruct workers to work
+off of all queues except those beginning with `file_`:
 
-When a parent process is idle, it lets you know what queues it is
-waiting for work on:
-
-    $ ps -e -o pid,command | grep [r]esque
-    92099 resque: Waiting for file_serve,warm_cache
+    $ QUEUE=*,!file_* rake resque:work
 
+Note that the order in which negated queues are specified does not
+matter, so `QUEUE=*,!file_*` and `QUEUE=!file_*,*` will have the same
+effect.
 
-### Signals
+#### Process IDs (PIDs)
 
-Resque workers respond to a few different signals:
+There are scenarios where it's helpful to record the PID of a resque
+worker process.  Use the PIDFILE option for easy access to the PID:
 
-* `QUIT` - Wait for child to finish processing then exit
-* `TERM` / `INT` - Immediately kill child then exit
-* `USR1` - Immediately kill child but don't exit
-* `USR2` - Don't start to process any new jobs
-* `CONT` - Start to process new jobs again after a USR2
+    $ PIDFILE=./resque.pid QUEUE=file_serve rake resque:work
 
-If you want to gracefully shutdown a Resque worker, use `QUIT`.
+#### Running in the background
 
-If you want to kill a stale or stuck child, use `USR1`. Processing
-will continue as normal unless the child was not found. In that case
-Resque assumes the parent process is in a bad state and shuts down.
+There are scenarios where it's helpful for
+the resque worker to run itself in the background (usually in combination with
+PIDFILE).  Use the BACKGROUND option so that rake will return as soon as the
+worker is started.
 
-If you want to kill a stale or stuck child and shutdown, use `TERM`
+    $ PIDFILE=./resque.pid BACKGROUND=yes QUEUE=file_serve rake resque:work
 
-If you want to stop processing jobs, but want to leave the worker running
-(for example, to temporarily alleviate load), use `USR2` to stop processing,
-then `CONT` to start it again.
+#### Polling frequency
 
-### Mysql::Error: MySQL server has gone away
+You can pass an INTERVAL option which is a float representing the polling frequency.
+The default is 5 seconds, but for a semi-active app you may want to use a smaller value.
 
-If your workers remain idle for too long they may lose their MySQL
-connection. If that happens we recommend using [this
-Gist](http://gist.github.com/238999).
+    $ INTERVAL=0.1 QUEUE=file_serve rake resque:work
 
+When INTERVAL is set to 0 it will run until the queue is empty and then
+shutdown the worker, instead of waiting for new jobs.
 
 The Front End
 -------------
@@ -440,9 +267,9 @@ The Front End
 Resque comes with a Sinatra-based front end for seeing what's up with
 your queue.
 
-![The Front End](https://img.skitch.com/20110528-pc67a8qsfapgjxf5gagxd92fcu.png)
+![The Front End](https://camo.githubusercontent.com/64d150a243987ffbc33f588bd6d7722a0bb8d69a/687474703a2f2f7475746f7269616c732e6a756d7073746172746c61622e636f6d2f696d616765732f7265737175655f6f766572766965772e706e67)
 
-### Standalone
+#### Standalone
 
 If you've installed Resque as a gem running the front end standalone is easy:
 
@@ -465,15 +292,15 @@ or set the Redis connection string if you need to do something like select a dif
 
     $ resque-web -p 8282 -r localhost:6379:2
 
-### Passenger
+#### Passenger
 
 Using Passenger? Resque ships with a `config.ru` you can use. See
 Phusion's guide:
 
-Apache: <http://www.modrails.com/documentation/Users%20guide%20Apache.html#_deploying_a_rack_based_ruby_application>
-Nginx: <http://www.modrails.com/documentation/Users%20guide%20Nginx.html#deploying_a_rack_app>
+Apache: <https://www.phusionpassenger.com/library/deploy/apache/deploy/ruby/>
+Nginx: <https://www.phusionpassenger.com/library/deploy/nginx/deploy/ruby/>
 
-### Rack::URLMap
+#### Rack::URLMap
 
 If you want to load Resque on a subpath, possibly alongside other
 apps, it's easy to do with Rack's `URLMap`:
@@ -489,229 +316,151 @@ run Rack::URLMap.new \
 Check `examples/demo/config.ru` for a functional example (including
 HTTP basic auth).
 
-### Rails 3
+#### Rails
 
-You can also mount Resque on a subpath in your existing Rails 3 app by adding `require 'resque/server'` to the top of your routes file or in an initializer then adding this to `routes.rb`:
+You can also mount Resque on a subpath in your existing Rails app by adding `require 'resque/server'` to the top of your routes file or in an initializer then adding this to `routes.rb`:
 
 ``` ruby
 mount Resque::Server.new, :at => "/resque"
 ```
 
+Jobs
+----
 
-Resque vs DelayedJob
---------------------
+What should you run in the background? Anything that takes any time at
+all. Slow INSERT statements, disk manipulating, data processing, etc.
 
-How does Resque compare to DelayedJob, and why would you choose one
-over the other?
+At GitHub we use Resque to process the following types of jobs:
 
-* Resque supports multiple queues
-* DelayedJob supports finer grained priorities
-* Resque workers are resilient to memory leaks / bloat
-* DelayedJob workers are extremely simple and easy to modify
-* Resque requires Redis
-* DelayedJob requires ActiveRecord
-* Resque can only place JSONable Ruby objects on a queue as arguments
-* DelayedJob can place _any_ Ruby object on its queue as arguments
-* Resque includes a Sinatra app for monitoring what's going on
-* DelayedJob can be queried from within your Rails app if you want to
-  add an interface
+* Warming caches
+* Counting disk usage
+* Building tarballs
+* Building Rubygems
+* Firing off web hooks
+* Creating events in the db and pre-caching them
+* Building graphs
+* Deleting users
+* Updating our search index
 
-If you're doing Rails development, you already have a database and
-ActiveRecord. DelayedJob is super easy to setup and works great.
-GitHub used it for many months to process almost 200 million jobs.
+As of writing we have about 35 different types of background jobs.
 
-Choose Resque if:
+Keep in mind that you don't need a web app to use Resque - we just
+mention "foreground" and "background" because they make conceptual
+sense. You could easily be spidering sites and sticking data which
+needs to be crunched later into a queue.
 
-* You need multiple queues
-* You don't care / dislike numeric priorities
-* You don't need to persist every Ruby object ever
-* You have potentially huge queues
-* You want to see what's going on
-* You expect a lot of failure / chaos
-* You can setup Redis
-* You're not running short on RAM
+#### Persistence
 
-Choose DelayedJob if:
-
-* You like numeric priorities
-* You're not doing a gigantic amount of jobs each day
-* Your queue stays small and nimble
-* There is not a lot failure / chaos
-* You want to easily throw anything on the queue
-* You don't want to setup Redis
-
-In no way is Resque a "better" DelayedJob, so make sure you pick the
-tool that's best for your app.
-
-
-Installing Redis
-----------------
-
-Resque requires Redis 0.900 or higher.
-
-Resque uses Redis' lists for its queues. It also stores worker state
-data in Redis.
-
-#### Homebrew
-
-If you're on OS X, Homebrew is the simplest way to install Redis:
-
-    $ brew install redis
-    $ redis-server /usr/local/etc/redis.conf
-
-You now have a Redis daemon running on 6379.
-
-#### Via Resque
-
-Resque includes Rake tasks (thanks to Ezra's redis-rb) that will
-install and run Redis for you:
-
-    $ git clone git://github.com/defunkt/resque.git
-    $ cd resque
-    $ rake redis:install dtach:install
-    $ rake redis:start
-
-Or, if you don't have admin access on your machine:
-
-    $ git clone git://github.com/defunkt/resque.git
-    $ cd resque
-    $ PREFIX=<your_prefix> rake redis:install dtach:install
-    $ rake redis:start
-
-You now have Redis running on 6379. Wait a second then hit ctrl-\ to
-detach and keep it running in the background.
-
-The demo is probably the best way to figure out how to put the parts
-together. But, it's not that hard.
-
-
-Resque Dependencies
--------------------
-
-    $ gem install bundler
-    $ bundle install
-
-
-Installing Resque
------------------
-
-### In a Rack app, as a gem
-
-First install the gem.
-
-    $ gem install resque
-
-Next include it in your application.
+Jobs are persisted to queues as JSON objects. Let's take our `Archive`
+example from above. We'll run the following code to create a job:
 
 ``` ruby
-require 'resque'
+repo = Repository.find(44)
+repo.async_create_archive('masterbrew')
 ```
 
-Now start your application:
+The following JSON will be stored in the `file_serve` queue:
 
-    rackup config.ru
+``` javascript
+{
+    'class': 'Archive',
+    'args': [ 44, 'masterbrew' ]
+}
+```
 
-That's it! You can now create Resque jobs from within your app.
+Because of this your jobs must only accept arguments that can be JSON encoded.
 
-To start a worker, create a Rakefile in your app's root (or add this
-to an existing Rakefile):
+So instead of doing this:
 
 ``` ruby
-require 'your/app'
-require 'resque/tasks'
+Resque.enqueue(Archive, self, branch)
 ```
 
-Now:
-
-    $ QUEUE=* rake resque:work
-
-Alternately you can define a `resque:setup` hook in your Rakefile if you
-don't want to load your app every time rake runs.
-
-
-### In a Rails 2.x app, as a gem
-
-First install the gem.
-
-    $ gem install resque
-
-Next include it in your application.
-
-    $ cat config/initializers/load_resque.rb
-    require 'resque'
-
-Now start your application:
-
-    $ ./script/server
-
-That's it! You can now create Resque jobs from within your app.
-
-To start a worker, add this to your Rakefile in `RAILS_ROOT`:
+do this:
 
 ``` ruby
-require 'resque/tasks'
+Resque.enqueue(Archive, self.id, branch)
 ```
 
-Now:
-
-    $ QUEUE=* rake environment resque:work
+This is why our above example (and all the examples in `examples/`)
+uses object IDs instead of passing around the objects.
 
-Don't forget you can define a `resque:setup` hook in
-`lib/tasks/whatever.rake` that loads the `environment` task every time.
+While this is less convenient than just sticking a marshaled object
+in the database, it gives you a slight advantage: your jobs will be
+run against the most recent version of an object because they need to
+pull from the DB or cache.
 
+If your jobs were run against marshaled objects, they could
+potentially be operating on a stale record with out-of-date information.
 
-### In a Rails 2.x app, as a plugin
 
-    $ ./script/plugin install git://github.com/defunkt/resque
+#### send_later / async
 
-That's it! Resque will automatically be available when your Rails app
-loads.
+Want something like DelayedJob's `send_later` or the ability to use
+instance methods instead of just methods for jobs? See the `examples/`
+directory for goodies.
 
-To start a worker:
+We plan to provide first class `async` support in a future release.
 
-    $ QUEUE=* rake environment resque:work
 
-Don't forget you can define a `resque:setup` hook in
-`lib/tasks/whatever.rake` that loads the `environment` task every time.
+#### Failure
 
+If a job raises an exception, it is logged and handed off to the
+`Resque::Failure` module. Failures are logged either locally in Redis
+or using some different backend. To see exceptions while developing,
+see details below under Logging.
 
-### In a Rails 3 app, as a gem
+For example, Resque ships with Airbrake support. To configure it, put
+the following into an initialisation file or into your rake job:
 
-First include it in your Gemfile.
+``` ruby
+# send errors which occur in background jobs to redis and airbrake
+require 'resque/failure/multiple'
+require 'resque/failure/redis'
+require 'resque/failure/airbrake'
 
-    $ cat Gemfile
-    ...
-    gem 'resque'
-    ...
+Resque::Failure::Multiple.classes = [Resque::Failure::Redis, Resque::Failure::Airbrake]
+Resque::Failure.backend = Resque::Failure::Multiple
+```
 
-Next install it with Bundler.
+Keep this in mind when writing your jobs: you may want to throw
+exceptions you would not normally throw in order to assist debugging.
 
-    $ bundle install
 
-Now start your application:
+#### Rails example
 
-    $ rails server
+If you are using ActiveJob here's how your job definition will look:
 
-That's it! You can now create Resque jobs from within your app.
+``` ruby
+class ArchiveJob < ApplicationJob
+  queue_as :file_serve
 
-To start a worker, add this to a file in `lib/tasks` (ex:
-`lib/tasks/resque.rake`):
+  def perform(repo_id, branch = 'master')
+    repo = Repository.find(repo_id)
+    repo.create_archive(branch)
+  end
+end
+```
 
 ``` ruby
-require 'resque/tasks'
+class Repository
+  def async_create_archive(branch)
+    ArchiveJob.perform_later(self.id, branch)
+  end
+end
 ```
 
-Now:
-
-    $ QUEUE=* rake environment resque:work
-
-Don't forget you can define a `resque:setup` hook in
-`lib/tasks/whatever.rake` that loads the `environment` task every time.
+It is important to run `ArchiveJob.perform_later(self.id, branch)` rather than `Resque.enqueue(Archive, self.id, branch)`.
+Otherwise Resque will process the job without actually doing anything.
+Even if you put an obviously buggy line like `0/0` in the `perform` method,
+the job will still succeed.
 
 
 Configuration
 -------------
 
+#### Redis
+
 You may want to change the Redis host and port Resque connects to, or
 set various other options at startup.
 
@@ -733,15 +482,16 @@ Here's our `config/resque.yml`:
     test: localhost:6379
     staging: redis1.se.github.com:6379
     fi: localhost:6379
-    production: redis1.ae.github.com:6379
+    production: <%= ENV['REDIS_URL'] %>
 
 And our initializer:
 
 ``` ruby
 rails_root = ENV['RAILS_ROOT'] || File.dirname(__FILE__) + '/../..'
 rails_env = ENV['RAILS_ENV'] || 'development'
+config_file = rails_root + '/config/resque.yml'
 
-resque_config = YAML.load_file(rails_root + '/config/resque.yml')
+resque_config = YAML::load(ERB.new(IO.read(config_file)).result)
 Resque.redis = resque_config[rails_env]
 ```
 
@@ -759,26 +509,31 @@ For example, if you want to run all jobs in the same process for cucumber, try:
 Resque.inline = ENV['RAILS_ENV'] == "cucumber"
 ```
 
+#### Logging
 
-Plugins and Hooks
------------------
+Workers support basic logging to STDOUT.
 
-For a list of available plugins see
-<http://wiki.github.com/defunkt/resque/plugins>.
+You can control the logging threshold using `Resque.logger.level`:
 
-If you'd like to write your own plugin, or want to customize Resque
-using hooks (such as `Resque.after_fork`), see
-[docs/HOOKS.md](http://github.com/defunkt/resque/blob/master/docs/HOOKS.md).
+```ruby
+# config/initializers/resque.rb
+Resque.logger.level = Logger::DEBUG
+```
 
+If you want Resque to log to a file, in Rails do:
 
-Namespaces
-----------
+```ruby
+# config/initializers/resque.rb
+Resque.logger = Logger.new(Rails.root.join('log', "#{Rails.env}_resque.log"))
+```
+
+#### Namespaces
 
 If you're running multiple, separate instances of Resque you may want
 to namespace the keyspaces so they do not overlap. This is not unlike
 the approach taken by many memcached clients.
 
-This feature is provided by the [redis-namespace][rs] library, which
+This feature is provided by the [redis-namespace](http://github.com/resque/redis-namespace) library, which
 Resque uses by default to separate the keys it manages from other keys
 in your Redis server.
 
@@ -791,38 +546,269 @@ Resque.redis.namespace = "resque:GitHub"
 We recommend sticking this in your initializer somewhere after Redis
 is configured.
 
+#### Storing Statistics
+ Resque allows to store count of processed and failed jobs.
 
-Demo
-----
+ By default it will store it in Redis using the keys `stats:processed` and `stats:failed`.
 
-Resque ships with a demo Sinatra app for creating jobs that are later
-processed in the background.
+ Some apps would want another stats store, or even a null store:
 
-Try it out by looking at the README, found at `examples/demo/README.markdown`.
+ ```ruby
+# config/initializers/resque.rb
+class NullDataStore
+  def stat(stat)
+    0
+  end
 
+  def increment_stat(stat, by)
+  end
 
-Monitoring
-----------
+  def decrement_stat(stat, by)
+  end
 
-### god
+  def clear_stat(stat)
+  end
+end
+
+Resque.stat_data_store = NullDataStore.new
+```
+
+Plugins and Hooks
+-----------------
+
+For a list of available plugins see
+<https://github.com/resque/resque/wiki/plugins>.
+
+If you'd like to write your own plugin, or want to customize Resque
+using hooks (such as `Resque.after_fork`), see
+[docs/HOOKS.md](http://github.com/resque/resque/blob/master/docs/HOOKS.md).
+
+
+Additional Information
+----------------------
+
+#### Resque vs DelayedJob
+
+How does Resque compare to DelayedJob, and why would you choose one
+over the other?
+
+* Resque supports multiple queues
+* DelayedJob supports finer grained priorities
+* Resque workers are resilient to memory leaks / bloat
+* DelayedJob workers are extremely simple and easy to modify
+* Resque requires Redis
+* DelayedJob requires ActiveRecord
+* Resque can only place JSONable Ruby objects on a queue as arguments
+* DelayedJob can place _any_ Ruby object on its queue as arguments
+* Resque includes a Sinatra app for monitoring what's going on
+* DelayedJob can be queried from within your Rails app if you want to
+  add an interface
+
+If you're doing Rails development, you already have a database and
+ActiveRecord. DelayedJob is super easy to setup and works great.
+GitHub used it for many months to process almost 200 million jobs.
+
+Choose Resque if:
+
+* You need multiple queues
+* You don't care / dislike numeric priorities
+* You don't need to persist every Ruby object ever
+* You have potentially huge queues
+* You want to see what's going on
+* You expect a lot of failure / chaos
+* You can setup Redis
+* You're not running short on RAM
+
+Choose DelayedJob if:
+
+* You like numeric priorities
+* You're not doing a gigantic amount of jobs each day
+* Your queue stays small and nimble
+* There is not a lot failure / chaos
+* You want to easily throw anything on the queue
+* You don't want to setup Redis
+
+In no way is Resque a "better" DelayedJob, so make sure you pick the
+tool that's best for your app.
+
+#### Forking
+
+On certain platforms, when a Resque worker reserves a job it
+immediately forks a child process. The child processes the job then
+exits. When the child has exited successfully, the worker reserves
+another job and repeats the process.
+
+Why?
+
+Because Resque assumes chaos.
+
+Resque assumes your background workers will lock up, run too long, or
+have unwanted memory growth.
+
+If Resque workers processed jobs themselves, it'd be hard to whip them
+into shape. Let's say one is using too much memory: you send it a
+signal that says "shutdown after you finish processing the current
+job," and it does so. It then starts up again - loading your entire
+application environment. This adds useless CPU cycles and causes a
+delay in queue processing.
+
+Plus, what if it's using too much memory and has stopped responding to
+signals?
+
+Thanks to Resque's parent / child architecture, jobs that use too much memory
+release that memory upon completion. No unwanted growth.
+
+And what if a job is running too long? You'd need to `kill -9` it then
+start the worker again. With Resque's parent / child architecture you
+can tell the parent to forcefully kill the child then immediately
+start processing more jobs. No startup delay or wasted cycles.
+
+The parent / child architecture helps us keep tabs on what workers are
+doing, too. By eliminating the need to `kill -9` workers we can have
+parents remove themselves from the global listing of workers. If we
+just ruthlessly killed workers, we'd need a separate watchdog process
+to add and remove them to the global listing - which becomes
+complicated.
+
+Workers instead handle their own state.
+
+#### `at_exit` Callbacks
+
+Resque uses `Kernel#exit!` for exiting its workers' child processes. So any `at_exit` callback defined in your application won't be executed when the job is finished and the child process exits.
+
+You can alter this behavior by setting the `RUN_AT_EXIT_HOOKS` environment variable.
+
+#### Parents and Children
+
+Here's a parent / child pair doing some work:
+
+    $ ps -e -o pid,command | grep [r]esque
+    92099 resque: Forked 92102 at 1253142769
+    92102 resque: Processing file_serve since 1253142769
+
+You can clearly see that process 92099 forked 92102, which has been
+working since 1253142769.
+
+(By advertising the time they began processing you can easily use monit
+or god to kill stale workers.)
+
+When a parent process is idle, it lets you know what queues it is
+waiting for work on:
+
+    $ ps -e -o pid,command | grep [r]esque
+    92099 resque: Waiting for file_serve,warm_cache
+
+
+#### Signals
+
+Resque workers respond to a few different signals:
+
+* `QUIT` - Wait for child to finish processing then exit
+* `TERM` / `INT` - Immediately kill child then exit
+* `USR1` - Immediately kill child but don't exit
+* `USR2` - Don't start to process any new jobs
+* `CONT` - Start to process new jobs again after a USR2
+
+If you want to gracefully shutdown a Resque worker, use `QUIT`.
+
+If you want to kill a stale or stuck child, use `USR1`. Processing
+will continue as normal unless the child was not found. In that case
+Resque assumes the parent process is in a bad state and shuts down.
+
+If you want to kill a stale or stuck child and shutdown, use `TERM`
+
+If you want to stop processing jobs, but want to leave the worker running
+(for example, to temporarily alleviate load), use `USR2` to stop processing,
+then `CONT` to start it again. It's also possible to [pause all workers](#pausing-all-workers).
+
+#### Heroku
+
+When shutting down processes, Heroku sends every process a TERM signal at the
+same time. By default this causes an immediate shutdown of any running job
+leading to frequent `Resque::TermException` errors.  For short running jobs, a simple
+solution is to give a small amount of time for the job to finish
+before killing it.
+
+Resque doesn't handle this out of the box (for both cedar-14 and heroku-16), you need to
+install the [`resque-heroku-signals`](https://github.com/iloveitaly/resque-heroku-signals)
+addon which adds the required signal handling to make the behavior described above work.
+Related issue: https://github.com/resque/resque/issues/1559
+
+To accomplish this set the following environment variables:
+
+* `RESQUE_PRE_SHUTDOWN_TIMEOUT` - The time between the parent receiving a shutdown signal (TERM by default) and it sending that signal on to the child process. Designed to give the child process
+time to complete before being forced to die.
+
+* `TERM_CHILD` - Must be set for `RESQUE_PRE_SHUTDOWN_TIMEOUT` to be used. After the timeout, if the child is still running it will raise a `Resque::TermException` and exit.
+
+* `RESQUE_TERM_TIMEOUT` - By default you have a few seconds to handle `Resque::TermException` in your job. `RESQUE_TERM_TIMEOUT` and `RESQUE_PRE_SHUTDOWN_TIMEOUT` must be lower than the [heroku dyno timeout](https://devcenter.heroku.com/articles/limits#exit-timeout).
+
+#### Pausing all workers
+
+Workers will not process pending jobs if the Redis key `pause-all-workers` is set with the string value "true".
+
+``` ruby
+Resque.redis.set('pause-all-workers', 'true')
+```
+
+Nothing happens to jobs that are already being processed by workers.
+
+Unpause by removing the Redis key `pause-all-workers`.
+
+``` ruby
+Resque.redis.del('pause-all-workers')
+```
+
+#### Monitoring
+
+##### god
 
 If you're using god to monitor Resque, we have provided example
 configs in `examples/god/`. One is for starting / stopping workers,
 the other is for killing workers that have been running too long.
 
-### monit
+##### monit
 
 If you're using monit, `examples/monit/resque.monit` is provided free
 of charge. This is **not** used by GitHub in production, so please
 send patches for any tweaks or improvements you can make to it.
 
+#### Mysql::Error: MySQL server has gone away
 
-Questions
----------
+If your workers remain idle for too long they may lose their MySQL connection. Depending on your version of Rails, we recommend the following:
 
-Please add them to the [FAQ](https://github.com/defunkt/resque/wiki/FAQ) or
-ask on the Mailing List. The Mailing List is explained further below
+##### Rails
+In your `perform` method, add the following line:
 
+``` ruby
+class MyTask
+  def self.perform
+    ActiveRecord::Base.verify_active_connections!
+    # rest of your code
+  end
+end
+```
+
+The Rails doc says the following about `verify_active_connections!`:
+
+    Verify active connections and remove and disconnect connections associated with stale threads.
+
+##### Rails 4.x
+
+In your `perform` method, instead of `verify_active_connections!`, use:
+
+``` ruby
+class MyTask
+  def self.perform
+    ActiveRecord::Base.clear_active_connections!
+    # rest of your code
+  end
+end
+```
+
+From the Rails docs on [`clear_active_connections!`](http://api.rubyonrails.org/classes/ActiveRecord/ConnectionAdapters/ConnectionHandler.html#method-i-clear_active_connections-21):
+
+    Returns any connections in use by the current thread back to the pool, and also returns connections to the pool cached by threads that are no longer alive.
 
 Development
 -----------
@@ -831,7 +817,7 @@ Want to hack on Resque?
 
 First clone the repo and run the tests:
 
-    git clone git://github.com/defunkt/resque.git
+    git clone git://github.com/resque/resque.git
     cd resque
     rake test
 
@@ -853,56 +839,40 @@ it:
 If you get an error requiring any of the dependencies, you may have
 failed to install them or be seeing load path issues.
 
-Feel free to ping the mailing list with your problem and we'll try to
-sort it out.
+#### Demo
+Resque ships with a demo Sinatra app for creating jobs that are later
+processed in the background.
 
+Try it out by looking at the README, found at `examples/demo/README.markdown`.
 
-Contributing
-------------
+#### Contributing
 
-Read the [Contributing][cb] wiki page first. 
+Read [CONTRIBUTING.md](CONTRIBUTING.md) first.
 
 Once you've made your great commits:
 
-1. [Fork][1] Resque
+1. [Fork](http://help.github.com/forking/) Resque
 2. Create a topic branch - `git checkout -b my_branch`
 3. Push to your branch - `git push origin my_branch`
 4. Create a [Pull Request](http://help.github.com/pull-requests/) from your branch
-5. That's it!
-
-
-Mailing List
-------------
-
-To join the list simply send an email to <resque@librelist.com>. This
-will subscribe you and send you information about your subscription,
-including unsubscribe information.
 
-The archive can be found at <http://librelist.com/browser/resque/>.
+Questions
+---------
 
+Please add them to the [FAQ](https://github.com/resque/resque/wiki/FAQ) or open an issue on this repo.
 
 Meta
 ----
 
-* Code: `git clone git://github.com/defunkt/resque.git`
-* Home: <http://github.com/defunkt/resque>
-* Docs: <http://defunkt.github.com/resque/>
-* Bugs: <http://github.com/defunkt/resque/issues>
-* List: <resque@librelist.com>
-* Chat: <irc://irc.freenode.net/resque>
-* Gems: <http://gemcutter.org/gems/resque>
-
-This project uses [Semantic Versioning][sv].
+* Code: `git clone git://github.com/resque/resque.git`
+* Home: <http://github.com/resque/resque>
+* Docs: <http://rubydoc.info/gems/resque>
+* Bugs: <http://github.com/resque/resque/issues>
+* Gems: <https://rubygems.org/gems/resque>
 
+This project uses [Semantic Versioning](http://semver.org/)
 
 Author
 ------
 
 Chris Wanstrath :: chris@ozmm.org :: @defunkt
-
-[0]: http://github.com/blog/542-introducing-resque
-[1]: http://help.github.com/forking/
-[2]: http://github.com/defunkt/resque/issues
-[sv]: http://semver.org/
-[rs]: http://github.com/defunkt/redis-namespace
-[cb]: http://wiki.github.com/defunkt/resque/contributing