delayed_job 3.0.4 → 3.0.5

data/CHANGELOG.md

@@ -0,0 +1,153 @@
+3.0.5 - 2013-01-28
+==================
+* Better job timeout error logging
+* psych support for delayed_job_data_mapper deserialization
+* User can configure the worker to raise a SignalException on TERM and/or INT
+* Add the ability to run all available jobs and exit when complete
+
+3.0.4 - 2012-11-09
+==================
+* Allow the app to specify a default queue name
+* Capistrano script now allows user to specify the DJ command, allowing the user to add "bundle exec" if necessary
+* Persisted record check is now more general
+
+3.0.3 - 2012-05-25
+==================
+* Fix a bug where the worker would not respect the exit condition
+* Properly handle sleep delay command line argument
+
+3.0.2 - 2012-04-02
+==================
+* Fix deprecation warnings
+* Raise ArgumentError if attempting to enqueue a performable method on an object that hasn't been persisted yet
+* Allow the number of jobs read at a time to be configured from the command line using --read-ahead
+* Allow custom logger to be configured through Delayed::Worker.logger
+* Various documentation improvements
+
+3.0.1 - 2012-01-24
+==================
+* Added RecordNotFound message to deserialization error
+* Direct JRuby's yecht parser to syck extensions
+* Updated psych extensions for better compatibility with ruby 1.9.2
+* Updated syck extension for increased compatibility with class methods
+* Test grooming
+
+3.0.0 - 2011-12-30
+==================
+* New: Named queues
+* New: Job/Worker lifecycle callbacks
+* Change: daemons is no longer a runtime dependency
+* Change: Active Record backend support is provided by a separate gem
+* Change: Enqueue hook is called before jobs are saved so that they may be modified
+* Fix problem deserializing models that use a custom primary key column
+* Fix deserializing AR models when the object isn't in the default scope
+* Fix hooks not getting called when delay_jobs is false
+
+2.1.4 - 2011-02-11
+==================
+* Working around issues when psych is loaded, fixes issues with bundler 1.0.10 and Rails 3.0.4
+* Added -p/--prefix option to help differentiate multiple delayed job workers on the same host.
+
+2.1.3 - 2011-01-20
+==================
+* Revert worker contention fix due to regressions
+* Added Delayed::Worker.delay_jobs flag to support running jobs immediately
+
+2.1.2 - 2010-12-01
+==================
+* Remove contention between multiple workers by performing an update to lock a job before fetching it
+* Job payloads may implement #max_attempts to control how many times it should be retried
+* Fix for loading ActionMailer extension
+* Added 'delayed_job_server_role' Capistrano variable to allow delayed_job to run on its own worker server
+    set :delayed_job_server_role, :worker
+* Fix `rake jobs:work` so it outputs to the console
+
+2.1.1 - 2010-11-14
+==================
+* Fix issue with worker name not getting properly set when locking a job
+* Fixes for YAML serialization
+
+2.1.0 - 2010-11-14
+==================
+* Added enqueue, before, after, success, error, and failure. See the README
+* Remove Merb support
+* Remove all non Active Record backends into separate gems. See https://github.com/collectiveidea/delayed_job/wiki/Backends
+* remove rails 2 support. delayed_job 2.1 will only support Rails 3
+* New pure-YAML serialization
+* Added Rails 3 railtie and generator
+* Changed @@sleep_delay to self.class.sleep_delay to be consistent with other class variable usage
+* Added --sleep-delay command line option
+
+2.0.8 - Unreleased
+==================
+* Backport fix for deserialization errors that bring down the daemon
+
+2.0.7 - 2011-02-10
+==================
+* Fixed missing generators and recipes for Rails 2.x
+
+2.0.6 - 2011-01-20
+==================
+* Revert worker contention fix due to regressions
+
+2.0.5 - 2010-12-01
+==================
+* Added #reschedule_at hook on payload to determine when the job should be rescheduled [backported from 2.1]
+* Added --sleep-delay command line option [backported from 2.1]
+* Added 'delayed_job_server_role' Capistrano variable to allow delayed_job to run on its own worker server
+    set :delayed_job_server_role, :worker
+* Changed AR backend to reserve jobs using an UPDATE query to reduce worker contention [backported from 2.1]
+
+2.0.4 - 2010-11-14
+==================
+* Fix issue where dirty tracking prevented job from being properly unlocked
+* Add delayed_job_args variable for Capistrano recipe to allow configuration of started workers (e.g. "-n 2 --max-priority 10")
+* Added options to handle_asynchronously
+* Added Delayed::Worker.default_priority
+* Allow private methods to be delayed
+* Fixes for Ruby 1.9
+* Added -m command line option to start a monitor process
+* normalize logging in worker
+* Deprecate #send_later and #send_at in favor of new #delay method
+* Added @#delay@ to Object that allows you to delay any method and pass options:
+    options = {:priority => 19, :run_at => 5.minutes.from_now}
+    UserMailer.delay(options).deliver_confirmation(@user)
+
+2.0.3 - 2010-04-16
+==================
+* Fix initialization for Rails 2.x
+
+2.0.2 - 2010-04-08
+==================
+* Fixes to Mongo Mapper backend [ "14be7a24":http://github.com/collectiveidea/delayed_job/commit/14be7a24, "dafd5f46":http://github.com/collectiveidea/delayed_job/commit/dafd5f46, "54d40913":http://github.com/collectiveidea/delayed_job/commit/54d40913 ]
+* DataMapper backend performance improvements [ "93833cce":http://github.com/collectiveidea/delayed_job/commit/93833cce, "e9b1573e":http://github.com/collectiveidea/delayed_job/commit/e9b1573e, "37a16d11":http://github.com/collectiveidea/delayed_job/commit/37a16d11, "803f2bfa":http://github.com/collectiveidea/delayed_job/commit/803f2bfa ]
+* Fixed Delayed::Command to create tmp/pids directory [ "8ec8ca41":http://github.com/collectiveidea/delayed_job/commit/8ec8ca41 ]
+* Railtie to perform Rails 3 initialization [ "3e0fc41f":http://github.com/collectiveidea/delayed_job/commit/3e0fc41f ]
+* Added on_permanent_failure hook [ "d2f14cd6":http://github.com/collectiveidea/delayed_job/commit/d2f14cd6 ]
+
+2.0.1 - 2010-04-03
+==================
+* Bug fix for using ActiveRecord backend with daemon [martinbtt]
+
+2.0.0 - 2010-04-03
+==================
+* Multiple backend support (See README for more details)
+* Added MongoMapper backend [zbelzer, moneypools]
+* Added DataMapper backend [lpetre]
+* Reverse priority so the jobs table can be indexed. Lower numbers have higher priority. The default priority is 0, so increase it for jobs that are not important.
+* Move most of the heavy lifting from Job to Worker (#work_off, #reschedule, #run, #min_priority, #max_priority, #max_run_time, #max_attempts, #worker_name) [albus522]
+* Remove EvaledJob. Implement your own if you need this functionality.
+* Only use Time.zone if it is set. Closes #20
+* Fix for last_error recording when destroy_failed_jobs = false, max_attempts = 1
+* Implemented worker name_prefix to maintain dynamic nature of pid detection
+* Some Rails 3 compatibility fixes [fredwu]
+
+1.8.5 - 2010-03-15
+==================
+* Set auto_flushing=true on Rails logger to fix logging in production
+* Fix error message when trying to send_later on a method that doesn't exist
+* Don't use rails_env in capistrano if it's not set. closes #22
+* Delayed job should append to delayed_job.log not overwrite
+* Version bump to 1.8.5
+* fixing Time.now to be Time.zone.now if set to honor the app set local TimeZone
+* Replaced @Worker::SLEEP@, @Job::MAX_ATTEMPTS@, and @Job::MAX_RUN_TIME@ with class methods that can be overridden.

data/CONTRIBUTING.md

@@ -0,0 +1,18 @@
+How to contribute
+=================
+
+If you find what looks like a bug:
+
+* Search the "mailing list":http://groups.google.com/group/delayed_job to see
+  if anyone else had the same issue.
+* Check the "GitHub issue tracker":http://github.com/collectiveidea/delayed_job/issues/
+  to see if anyone else has reported issue.
+* If you don't see anything, create an issue with information on how to reproduce it.
+
+If you want to contribute an enhancement or a fix:
+
+* Fork the project on GitHub.
+* Make your changes with tests.
+* Commit the changes without making changes to the Rakefile or any other files
+  that aren't related to your enhancement or fix.
+* Send a pull request.

data/{MIT-LICENSE → LICENSE.md}

@@ -1,4 +1,4 @@
-Copyright (c) 2005 Tobias Luetke
+Copyright (c) 2005 Tobias Lütke
 
 Permission is hereby granted, free of charge, to any person obtaining
 a copy of this software and associated documentation files (the

data/{README.textile → README.md}

@@ -1,8 +1,19 @@
-h1. Delayed::Job "!https://travis-ci.org/collectiveidea/delayed_job.png!":http://travis-ci.org/collectiveidea/delayed_job "!https://gemnasium.com/collectiveidea/delayed_job.png?travis!":https://gemnasium.com/collectiveidea/delayed_job
-
-Delayed_job (or DJ) encapsulates the common pattern of asynchronously executing longer tasks in the background.
-
-It is a direct extraction from Shopify where the job table is responsible for a multitude of core tasks. Amongst those tasks are:
+Delayed::Job
+============
+[![Gem Version](https://badge.fury.io/rb/delayed_job.png)][gem]
+[![Build Status](https://secure.travis-ci.org/collectiveidea/delayed_job.png?branch=master)][travis]
+[![Dependency Status](https://gemnasium.com/collectiveidea/delayed_job.png?travis)][gemnasium]
+[![Code Climate](https://codeclimate.com/badge.png)][codeclimate]
+[gem]: https://rubygems.org/gems/delayed_job
+[travis]: http://travis-ci.org/collectiveidea/delayed_job
+[gemnasium]: https://gemnasium.com/collectiveidea/delayed_job
+[codeclimate]: https://codeclimate.com/github/collectiveidea/delayed_job
+
+Delayed::Job (or DJ) encapsulates the common pattern of asynchronously executing
+longer tasks in the background.
+
+It is a direct extraction from Shopify where the job table is responsible for a
+multitude of core tasks. Amongst those tasks are:
 
 * sending massive newsletters
 * image resizing
@@ -12,61 +23,57 @@ It is a direct extraction from Shopify where the job table is responsible for a
 * batch imports
 * spam checks
 
-"Follow us on Twitter":https://twitter.com/delayedjob to get updates and notices about new releases.
+[Follow us on Twitter][twitter] to get updates and notices about new releases.
 
-h2. Installation
+[twitter]: https://twitter.com/delayedjob
 
-delayed_job 3.0.0 only supports Rails 3.0+. See the "2.0 branch":https://github.com/collectiveidea/delayed_job/tree/v2.0 for Rails 2.
+Installation
+============
+delayed_job 3.0.0 only supports Rails 3.0+. See the [2.0
+branch](https://github.com/collectiveidea/delayed_job/tree/v2.0) for Rails 2.
 
-delayed_job supports multiple backends for storing the job queue. "See the wiki for other backends":http://wiki.github.com/collectiveidea/delayed_job/backends.
+delayed_job supports multiple backends for storing the job queue. [See the wiki
+for other backends](http://wiki.github.com/collectiveidea/delayed_job/backends).
 
-If you plan to use delayed_job with Active Record, add @delayed_job_active_record@ to your @Gemfile@.
+If you plan to use delayed_job with Active Record, add `delayed_job_active_record` to your `Gemfile`.
 
-<pre>
-gem 'delayed_job_active_record'
-</pre>
+    gem 'delayed_job_active_record'
 
-If you plan to use delayed_job with Mongoid, add @delayed_job_mongoid@ to your @Gemfile@.
+If you plan to use delayed_job with Mongoid, add `delayed_job_mongoid` to your `Gemfile`.
 
-<pre>
-gem 'delayed_job_mongoid'
-</pre>
+    gem 'delayed_job_mongoid'
 
-Run @bundle install@ to install the backend and delayed_job gems.
+Run `bundle install` to install the backend and delayed_job gems.
 
-The Active Record backend requires a jobs table. You can create that table by running the following command:
+The Active Record backend requires a jobs table. You can create that table by
+running the following command:
 
-<pre>
-$ rails generate delayed_job:active_record
-$ rake db:migrate
-</pre>
-
-h3. Upgrading from 2.x to 3.0.0 on Active Record
+    rails generate delayed_job:active_record
+    rake db:migrate
 
+Upgrading from 2.x to 3.0.0 on Active Record
+============================================
 Delayed Job 3.0.0 introduces a new column to the delayed_jobs table.
 
 If you're upgrading from Delayed Job 2.x, run the upgrade generator to create a migration to add the column.
 
-<pre>
-$ rails generate delayed_job:upgrade
-$ rake db:migrate
-</pre>
+    rails generate delayed_job:upgrade
+    rake db:migrate
 
-h2. Queuing Jobs
+Queuing Jobs
+============
+Call `.delay.method(params)` on any object and it will be processed in the background.
 
-Call @.delay.method(params)@ on any object and it will be processed in the background.
+    # without delayed_job
+    @user.activate!(@device)
 
-<pre>
-# without delayed_job
-@user.activate!(@device)
+    # with delayed_job
+    @user.delay.activate!(@device)
 
-# with delayed_job
-@user.delay.activate!(@device)
-</pre>
-
-If a method should always be run in the background, you can call @#handle_asynchronously@ after the method declaration:
+If a method should always be run in the background, you can call
+`#handle_asynchronously` after the method declaration:
 
-<pre>
+```ruby
 class Device
   def deliver
     # long running method
@@ -76,11 +83,13 @@ end
 
 device = Device.new
 device.deliver
-</pre>
+```
 
-handle_asynchronously can take as options anything you can pass to delay. In addition, the values can be Proc objects allowing call time evaluation of the value. For some examples:
+handle_asynchronously can take as options anything you can pass to delay. In
+addition, the values can be Proc objects allowing call time evaluation of the
+value. For some examples:
 
-<pre>
+```ruby
 class LongTasks
   def send_mailer
     # Some other code
@@ -109,72 +118,84 @@ class LongTasks
   end
   handle_asynchronously :call_an_instance_method, :priority => Proc.new {|i| i.how_important }
 end
-</pre>
-
-h3. Rails 3 Mailers
+```
 
+Rails 3 Mailers
+===============
 Due to how mailers are implemented in Rails 3, we had to do a little work around to get delayed_job to work.
 
-<pre>
+```ruby
 # without delayed_job
 Notifier.signup(@user).deliver
 
 # with delayed_job
 Notifier.delay.signup(@user)
-</pre>
+```
 
-Remove the @.deliver@ method to make it work. It's not ideal, but it's the best we could do for now.
+Remove the `.deliver` method to make it work. It's not ideal, but it's the best
+we could do for now.
 
-h3. Named Queues
+Named Queues
+============
+DJ 3 introduces Resque-style named queues while still retaining DJ-style
+priority. The goal is to provide a system for grouping tasks to be worked by
+separate pools of workers, which may be scaled and controlled individually.
 
-DJ 3 introduces Resque-style named queues while still retaining DJ-style priority. The goal is to provide a system for grouping tasks to be worked by separate pools of workers, which may be scaled and controlled individually.
+Jobs can be assigned to a queue by setting the `queue` option:
 
-Jobs can be assigned to a queue by setting the @queue@ option:
-
-<pre>object.delay(:queue => 'tracking').method
+```ruby
+object.delay(:queue => 'tracking').method
 
 Delayed::Job.enqueue job, :queue => 'tracking'
 
 handle_asynchronously :tweet_later, :queue => 'tweets'
-</pre>
-
-h2. Running Jobs
+```
 
-@script/delayed_job@ can be used to manage a background process which will start working off jobs.
+Running Jobs
+============
+`script/delayed_job` can be used to manage a background process which will
+start working off jobs.
 
-To do so, add @gem "daemons"@ to your @Gemfile@ and make sure you've run `rails generate delayed_job`.
+To do so, add `gem "daemons"` to your `Gemfile` and make sure you've run `rails
+generate delayed_job`.
 
 You can then do the following:
 
-<pre>
-$ RAILS_ENV=production script/delayed_job start
-$ RAILS_ENV=production script/delayed_job stop
+    RAILS_ENV=production script/delayed_job start
+    RAILS_ENV=production script/delayed_job stop
 
-# Runs two workers in separate processes.
-$ RAILS_ENV=production script/delayed_job -n 2 start
-$ RAILS_ENV=production script/delayed_job stop
+    # Runs two workers in separate processes.
+    RAILS_ENV=production script/delayed_job -n 2 start
+    RAILS_ENV=production script/delayed_job stop
 
-# Set the --queue or --queues option to work from a particular queue.
-$ RAILS_ENV=production script/delayed_job --queue=tracking start
-$ RAILS_ENV=production script/delayed_job --queues=mailers,tasks start
-</pre>
+    # Set the --queue or --queues option to work from a particular queue.
+    RAILS_ENV=production script/delayed_job --queue=tracking start
+    RAILS_ENV=production script/delayed_job --queues=mailers,tasks start
 
-Workers can be running on any computer, as long as they have access to the database and their clock is in sync. Keep in mind that each worker will check the database at least every 5 seconds.
+    # Runs all available jobs and the exits
+    RAILS_ENV=production script/delayed_job start --exit-on-complete
+    # or to run in the foreground
+    RAILS_ENV=production script/delayed_job run --exit-on-complete
 
-You can also invoke @rake jobs:work@ which will start working off jobs. You can cancel the rake task with @CTRL-C@.
+Workers can be running on any computer, as long as they have access to the
+database and their clock is in sync. Keep in mind that each worker will check
+the database at least every 5 seconds.
 
-Work off queues by setting the @QUEUE@ or @QUEUES@ environment variable.
+You can also invoke `rake jobs:work` which will start working off jobs. You can
+cancel the rake task with `CTRL-C`.
 
-<pre>
-QUEUE=tracking rake jobs:work
-QUEUES=mailers,tasks rake jobs:work
-</pre>
+If you want to just run all available jobs and exit you can use `rake jobs:workoff`
 
-h2. Custom Jobs
+Work off queues by setting the `QUEUE` or `QUEUES` environment variable.
 
+    QUEUE=tracking rake jobs:work
+    QUEUES=mailers,tasks rake jobs:work
+
+Custom Jobs
+===========
 Jobs are simple ruby objects with a method called perform. Any object which responds to perform can be stuffed into the jobs table. Job objects are serialized to yaml so that they can later be resurrected by the job runner.
 
-<pre>
+```ruby
 class NewsletterJob < Struct.new(:text, :emails)
   def perform
     emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
@@ -182,13 +203,13 @@ class NewsletterJob < Struct.new(:text, :emails)
 end
 
 Delayed::Job.enqueue NewsletterJob.new('lorem ipsum...', Customers.find(:all).collect(&:email))
-</pre>
-
-h2. Hooks
+```
 
+Hooks
+=====
 You can define hooks on your job that will be called at different stages in the process:
 
-<pre>
+```ruby
 class ParanoidNewsletterJob < NewsletterJob
   def enqueue(job)
     record_stat 'newsletter_job/enqueue'
@@ -218,13 +239,13 @@ class ParanoidNewsletterJob < NewsletterJob
     page_sysadmin_in_the_middle_of_the_night
   end
 end
-</pre>
-
-h2. Gory Details
+```
 
+Gory Details
+============
 The library revolves around a delayed_jobs table which looks as follows:
 
-<pre>
+```ruby
 create_table :delayed_jobs, :force => true do |table|
   table.integer  :priority, :default => 0      # Allows some jobs to jump to the front of the queue
   table.integer  :attempts, :default => 0      # Provides for retries, but still fail eventually.
@@ -237,7 +258,7 @@ create_table :delayed_jobs, :force => true do |table|
   table.string   :queue                        # The name of the queue this job is in
   table.timestamps
 end
-</pre>
+```
 
 On failure, the job is scheduled again in 5 seconds + N ** 4, where N is the number of retries.
 
@@ -258,7 +279,7 @@ It is possible to disable delayed jobs for testing purposes. Set Delayed::Worker
 
 Here is an example of changing job parameters in Rails:
 
-<pre>
+```ruby
 # config/initializers/delayed_job_config.rb
 Delayed::Worker.destroy_failed_jobs = false
 Delayed::Worker.sleep_delay = 60
@@ -266,12 +287,12 @@ Delayed::Worker.max_attempts = 3
 Delayed::Worker.max_run_time = 5.minutes
 Delayed::Worker.read_ahead = 10
 Delayed::Worker.delay_jobs = !Rails.env.test?
-</pre>
-
-h3. Cleaning up
-
-You can invoke @rake jobs:clear@ to delete all jobs in the queue.
+```
 
-h2. Mailing List
+Cleaning up
+===========
+You can invoke `rake jobs:clear` to delete all jobs in the queue.
 
-Join us on the "mailing list":http://groups.google.com/group/delayed_job
+Mailing List
+============
+Join us on the [mailing list](http://groups.google.com/group/delayed_job)

data/Rakefile

@@ -0,0 +1,11 @@
+# -*- encoding: utf-8 -*-
+require 'bundler/setup'
+Bundler::GemHelper.install_tasks
+
+require 'rspec/core/rake_task'
+desc 'Run the specs'
+RSpec::Core::RakeTask.new do |r|
+  r.verbose = false
+end
+
+task :default => :spec

data/delayed_job.gemspec

@@ -0,0 +1,17 @@
+# -*- encoding: utf-8 -*-
+
+Gem::Specification.new do |spec|
+  spec.add_dependency   'activesupport', '~> 3.0'
+  spec.authors        = ["Brandon Keepers", "Brian Ryckbost", "Chris Gaffney", "David Genord II", "Erik Michaels-Ober", "Matt Griffin", "Steve Richert", "Tobias Lütke"]
+  spec.description    = "Delayed_job (or DJ) encapsulates the common pattern of asynchronously executing longer tasks in the background. It is a direct extraction from Shopify where the job table is responsible for a multitude of core tasks."
+  spec.email          = ['brian@collectiveidea.com']
+  spec.files          = %w(CHANGELOG.md CONTRIBUTING.md LICENSE.md README.md Rakefile delayed_job.gemspec)
+  spec.files         += Dir.glob('{contrib,lib,recipes,spec}/**/*')
+  spec.homepage       = 'http://github.com/collectiveidea/delayed_job'
+  spec.licenses       = ['MIT']
+  spec.name           = 'delayed_job'
+  spec.require_paths  = ['lib']
+  spec.summary        = 'Database-backed asynchronous priority queue system -- Extracted from Shopify'
+  spec.test_files     = Dir.glob('spec/**/*')
+  spec.version        = '3.0.5'
+end

data/lib/delayed/backend/shared_spec.rb

@@ -18,6 +18,10 @@ shared_examples_for "a delayed_job backend" do
     described_class.delete_all
   end
 
+  after do
+    Delayed::Worker.reset
+  end
+
   it "sets run_at automatically if not set" do
     expect(described_class.create(:payload_object => ErrorJob.new ).run_at).not_to be_nil
   end
@@ -181,6 +185,10 @@ shared_examples_for "a delayed_job backend" do
       Delayed::Worker.max_run_time = 2.minutes
     end
 
+    after do
+      Time.zone = nil
+    end
+
     it "does not reserve failed jobs" do
       create_job :attempts => 50, :failed_at => described_class.db_time_now
       expect(described_class.reserve(worker)).to be_nil
@@ -249,7 +257,7 @@ shared_examples_for "a delayed_job backend" do
   end
 
   context "worker prioritization" do
-    before(:each) do
+    after do
       Delayed::Worker.max_priority = nil
       Delayed::Worker.min_priority = nil
     end
@@ -267,15 +275,25 @@ shared_examples_for "a delayed_job backend" do
     it "only finds jobs greater than or equal to min priority" do
       min = 5
       Delayed::Worker.min_priority = min
-      10.times {|i| described_class.enqueue SimpleJob.new, :priority => i }
-      5.times { expect(described_class.reserve(worker).priority).to be >= min }
+      [4,5,6].sort_by {|i| rand }.each {|i| create_job :priority => i }
+      2.times do
+        job = described_class.reserve(worker)
+        expect(job.priority).to be >= min
+        job.destroy
+      end
+      expect(described_class.reserve(worker)).to be_nil
     end
 
     it "only finds jobs less than or equal to max priority" do
       max = 5
       Delayed::Worker.max_priority = max
-      10.times {|i| described_class.enqueue SimpleJob.new, :priority => i }
-      5.times { expect(described_class.reserve(worker).priority).to be <= max }
+      [4,5,6].sort_by {|i| rand }.each {|i| create_job :priority => i }
+      2.times do
+        job = described_class.reserve(worker)
+        expect(job.priority).to be <= max
+        job.destroy
+      end
+      expect(described_class.reserve(worker)).to be_nil
     end
   end
 
@@ -421,19 +439,19 @@ shared_examples_for "a delayed_job backend" do
 
     describe "running a job" do
       it "fails after Worker.max_run_time" do
-        begin
-          old_max_run_time = Delayed::Worker.max_run_time
-          Delayed::Worker.max_run_time = 1.second
-          job = Delayed::Job.create :payload_object => LongRunningJob.new
-          worker.run(job)
-          expect(job.reload.last_error).to match(/expired/)
-          expect(job.attempts).to eq(1)
-        ensure
-          Delayed::Worker.max_run_time = old_max_run_time
-        end
+        Delayed::Worker.max_run_time = 1.second
+        job = Delayed::Job.create :payload_object => LongRunningJob.new
+        worker.run(job)
+        expect(job.reload.last_error).to match(/expired/)
+        expect(job.reload.last_error).to match(/Delayed::Worker.max_run_time is only 1 second/)
+        expect(job.attempts).to eq(1)
       end
 
       context "when the job raises a deserialization error" do
+        after do
+          Delayed::Worker.destroy_failed_jobs = true
+        end
+
         it "marks the job as failed" do
           Delayed::Worker.destroy_failed_jobs = false
           job = described_class.create! :handler => "--- !ruby/object:JobThatDoesNotExist {}"
@@ -446,11 +464,12 @@ shared_examples_for "a delayed_job backend" do
 
     describe "failed jobs" do
       before do
-        # reset defaults
-        Delayed::Worker.destroy_failed_jobs = true
-        Delayed::Worker.max_attempts = 25
+        @job = Delayed::Job.enqueue(ErrorJob.new, :run_at => described_class.db_time_now - 1)
+      end
 
-        @job = Delayed::Job.enqueue(ErrorJob.new)
+      after do
+        # reset default
+        Delayed::Worker.destroy_failed_jobs = true
       end
 
       it "records last_error when destroy_failed_jobs = false, max_attempts = 1" do
@@ -535,10 +554,6 @@ shared_examples_for "a delayed_job backend" do
       end
 
       context "and we want to destroy jobs" do
-        before do
-          Delayed::Worker.destroy_failed_jobs = true
-        end
-
         it_should_behave_like "any failure more than Worker.max_attempts times"
 
         it "is destroyed if it failed more than Worker.max_attempts times" do
@@ -557,6 +572,10 @@ shared_examples_for "a delayed_job backend" do
           Delayed::Worker.destroy_failed_jobs = false
         end
 
+        after do
+          Delayed::Worker.destroy_failed_jobs = true
+        end
+
         it_should_behave_like "any failure more than Worker.max_attempts times"
 
         it "is failed if it failed more than Worker.max_attempts times" do

data/lib/delayed/command.rb

@@ -61,6 +61,9 @@ module Delayed
         opts.on('--queue=queue', "Specify which queue DJ must look up for jobs") do |queue|
           @options[:queues] = queue.split(',')
         end
+        opts.on('--exit-on-complete', "Exit when no more jobs are available to run. This will exit if all jobs are scheduled to run in the future.") do
+          @options[:exit_on_complete] = true
+        end
       end
       @args = opts.parse!(args)
     end

data/lib/delayed/exceptions.rb

@@ -0,0 +1,9 @@
+require 'timeout'
+
+module Delayed
+  class WorkerTimeout < Timeout::Error
+    def message
+      "#{super} (Delayed::Worker.max_run_time is only #{Delayed::Worker.max_run_time.to_i} seconds)"
+    end
+  end
+end

data/lib/delayed/psych_ext.rb

@@ -111,6 +111,16 @@ module Psych
           rescue Mongoid::Errors::DocumentNotFound
             raise Delayed::DeserializationError
           end
+        when /^!ruby\/DataMapper:(.+)$/
+          klass = resolve_class($1)
+          payload = Hash[*object.children.map { |c| accept c }]
+          begin
+            primary_keys = klass.properties.select { |p| p.key? }
+            key_names = primary_keys.map { |p| p.name.to_s }
+            klass.get!(*key_names.map { |k| payload["attributes"][k] })
+          rescue DataMapper::ObjectNotFoundError
+            raise Delayed::DeserializationError
+          end
         else
           visit_Psych_Nodes_Mapping_without_class(object)
         end

data/lib/delayed/tasks.rb

@@ -5,7 +5,21 @@ namespace :jobs do
   end
 
   desc "Start a delayed_job worker."
-  task :work => :environment do
-    Delayed::Worker.new(:min_priority => ENV['MIN_PRIORITY'], :max_priority => ENV['MAX_PRIORITY'], :queues => (ENV['QUEUES'] || ENV['QUEUE'] || '').split(','), :quiet => false).start
+  task :work => :environment_options do
+    Delayed::Worker.new(@worker_options).start
+  end
+
+  desc "Start a delayed_job worker and exit when all available jobs are complete."
+  task :workoff => :environment_options do
+    Delayed::Worker.new(@worker_options.merge({:exit_on_complete => true})).start
+  end
+
+  task :environment_options => :environment do
+    @worker_options = {
+      :min_priority => ENV['MIN_PRIORITY'],
+      :max_priority => ENV['MAX_PRIORITY'],
+      :queues => (ENV['QUEUES'] || ENV['QUEUE'] || '').split(','),
+      :quiet => false
+    }
   end
 end

data/lib/delayed/worker.rb

@@ -18,7 +18,7 @@ module Delayed
 
     cattr_accessor :min_priority, :max_priority, :max_attempts, :max_run_time,
       :default_priority, :sleep_delay, :logger, :delay_jobs, :queues,
-      :read_ahead, :plugins, :destroy_failed_jobs
+      :read_ahead, :plugins, :destroy_failed_jobs, :exit_on_complete
 
     # Named queue into which jobs are enqueued by default
     cattr_accessor :default_queue_name
@@ -47,6 +47,15 @@ module Delayed
     # (perhaps to inspect the reason for the failure), set this to false.
     self.destroy_failed_jobs = true
 
+    # By default, Signals INT and TERM set @exit, and the worker exits upon completion of the current job.
+    # If you would prefer to raise a SignalException and exit immediately you can use this.
+    # Be aware daemons uses TERM to stop and restart
+    # false - No exceptions will be raised
+    # :term - Will only raise an exception on TERM signals but INT will wait for the current job to finish
+    # true - Will raise an exception on TERM and INT
+    cattr_accessor :raise_signal_exceptions
+    self.raise_signal_exceptions = false
+
     self.logger = if defined?(Rails)
       Rails.logger
     elsif defined?(RAILS_DEFAULT_LOGGER)
@@ -97,11 +106,10 @@ module Delayed
 
     def initialize(options={})
       @quiet = options.has_key?(:quiet) ? options[:quiet] : true
-      self.class.min_priority = options[:min_priority] if options.has_key?(:min_priority)
-      self.class.max_priority = options[:max_priority] if options.has_key?(:max_priority)
-      self.class.sleep_delay  = options[:sleep_delay] if options.has_key?(:sleep_delay)
-      self.class.read_ahead   = options[:read_ahead] if options.has_key?(:read_ahead)
-      self.class.queues       = options[:queues] if options.has_key?(:queues)
+
+      [:min_priority, :max_priority, :sleep_delay, :read_ahead, :queues, :exit_on_complete].each do |option|
+        self.class.send("#{option}=", options[option]) if options.has_key?(option)
+      end
 
       self.plugins.each { |klass| klass.new }
     end
@@ -122,29 +130,39 @@ module Delayed
     end
 
     def start
-      trap('TERM') { say 'Exiting...'; stop }
-      trap('INT')  { say 'Exiting...'; stop }
+      trap('TERM') do
+        say 'Exiting...'
+        stop
+        raise SignalException.new('TERM') if self.class.raise_signal_exceptions
+      end
+
+      trap('INT') do
+        say 'Exiting...'
+        stop
+        raise SignalException.new('INT') if self.class.raise_signal_exceptions && self.class.raise_signal_exceptions != :term
+      end
 
       say "Starting job worker"
 
       self.class.lifecycle.run_callbacks(:execute, self) do
         loop do
           self.class.lifecycle.run_callbacks(:loop, self) do
-            result = nil
-
-            realtime = Benchmark.realtime do
-              result = work_off
+            @realtime = Benchmark.realtime do
+              @result = work_off
             end
+          end
 
-            count = result.sum
-
-            break if stop?
+          count = @result.sum
 
-            if count.zero?
-              sleep(self.class.sleep_delay)
+          if count.zero?
+            if self.class.exit_on_complete
+              say "No more jobs available. Exiting"
+              break
             else
-              say "#{count} jobs processed at %.4f j/s, %d failed ..." % [count / realtime, result.last]
+              sleep(self.class.sleep_delay) unless stop?
             end
+          else
+            say "#{count} jobs processed at %.4f j/s, %d failed ..." % [count / @realtime, @result.last]
           end
 
           break if stop?
@@ -182,7 +200,7 @@ module Delayed
 
     def run(job)
       runtime =  Benchmark.realtime do
-        Timeout.timeout(self.class.max_run_time.to_i) { job.invoke_job }
+        Timeout.timeout(self.class.max_run_time.to_i, WorkerTimeout) { job.invoke_job }
         job.destroy
       end
       say "#{job.name} completed after %.4f" % runtime

data/lib/delayed_job.rb

@@ -1,4 +1,5 @@
 require 'active_support'
+require 'delayed/exceptions'
 require 'delayed/message_sending'
 require 'delayed/performable_method'
 

data/spec/message_sending_spec.rb

@@ -67,6 +67,10 @@ describe Delayed::MessageSending do
       end
     end
 
+    after do
+      Delayed::Worker.default_queue_name = nil
+    end
+
     it "creates a new PerformableMethod job" do
       expect {
         job = "hello".delay.count('l')
@@ -80,14 +84,12 @@ describe Delayed::MessageSending do
       Delayed::Worker.default_priority = 99
       job = FairyTail.delay.to_s
       expect(job.priority).to eq(99)
-      Delayed::Worker.default_priority = 0
     end
 
     it "sets default queue name" do
       Delayed::Worker.default_queue_name = 'abbazabba'
       job = FairyTail.delay.to_s
       expect(job.queue).to eq('abbazabba')
-      Delayed::Worker.default_queue_name = nil
     end
 
     it "sets job options" do

data/spec/performable_method_spec.rb

@@ -40,36 +40,44 @@ describe Delayed::PerformableMethod do
   end
 
   describe "hooks" do
-    %w(enqueue before after success).each do |hook|
+    %w(before after success).each do |hook|
       it "delegates #{hook} hook to object" do
         story = Story.create
-        story.should_receive(hook).with(an_instance_of(Delayed::Job))
-        story.delay.tell.invoke_job
+        job = story.delay.tell
+
+        story.should_receive(hook).with(job)
+        job.invoke_job
       end
     end
 
     %w(before after success).each do |hook|
-      it "delegates #{hook} hook to object when delay_jobs = false" do
-        Delayed::Worker.delay_jobs = false
+      it "delegates #{hook} hook to object" do
         story = Story.create
-        story.should_receive(hook).with(an_instance_of(Delayed::Job))
-        story.delay.tell
+        job = story.delay.tell
+
+        story.should_receive(hook).with(job)
+        job.invoke_job
       end
     end
 
+    it "delegates enqueue hook to object" do
+      story = Story.create
+      story.should_receive(:enqueue).with(an_instance_of(Delayed::Job))
+      story.delay.tell
+    end
+
     it "delegates error hook to object" do
       story = Story.create
       story.should_receive(:error).with(an_instance_of(Delayed::Job), an_instance_of(RuntimeError))
       story.should_receive(:tell).and_raise(RuntimeError)
-      expect{story.delay.tell.invoke_job}.to raise_error
+      expect { story.delay.tell.invoke_job }.to raise_error
     end
 
     it "delegates error hook to object when delay_jobs = false" do
-      Delayed::Worker.delay_jobs = false
       story = Story.create
       story.should_receive(:error).with(an_instance_of(Delayed::Job), an_instance_of(RuntimeError))
       story.should_receive(:tell).and_raise(RuntimeError)
-      expect{story.delay.tell}.to raise_error
+      expect { story.delay.tell.invoke_job }.to raise_error
     end
 
     it "delegates failure hook to object" do
@@ -78,12 +86,51 @@ describe Delayed::PerformableMethod do
       method.failure
     end
 
-    it "delegates failure hook to object when delay_jobs = false" do
-      Delayed::Worker.delay_jobs = false
-      method = Delayed::PerformableMethod.new("object", :size, [])
-      method.object.should_receive(:failure)
-      method.failure
-    end
+    context 'with delay_job == false' do
+      before do
+        Delayed::Worker.delay_jobs = false
+      end
 
+      after do
+        Delayed::Worker.delay_jobs = true
+      end
+
+      %w(before after success).each do |hook|
+        it "delegates #{hook} hook to object" do
+          story = Story.create
+          story.should_receive(hook).with(an_instance_of(Delayed::Job))
+          story.delay.tell
+        end
+      end
+
+      %w(before after success).each do |hook|
+        it "delegates #{hook} hook to object" do
+          story = Story.create
+          story.should_receive(hook).with(an_instance_of(Delayed::Job))
+          story.delay.tell
+        end
+      end
+
+      it "delegates error hook to object" do
+        story = Story.create
+        story.should_receive(:error).with(an_instance_of(Delayed::Job), an_instance_of(RuntimeError))
+        story.should_receive(:tell).and_raise(RuntimeError)
+        expect { story.delay.tell }.to raise_error
+      end
+
+      it "delegates error hook to object when delay_jobs = false" do
+        story = Story.create
+        story.should_receive(:error).with(an_instance_of(Delayed::Job), an_instance_of(RuntimeError))
+        story.should_receive(:tell).and_raise(RuntimeError)
+        expect { story.delay.tell }.to raise_error
+      end
+
+      it "delegates failure hook to object when delay_jobs = false" do
+        Delayed::Worker.delay_jobs = false
+        method = Delayed::PerformableMethod.new("object", :size, [])
+        method.object.should_receive(:failure)
+        method.failure
+      end
+    end
   end
 end

data/spec/worker_spec.rb

@@ -7,6 +7,10 @@ describe Delayed::Worker do
       Delayed::Worker.backend = @clazz
     end
 
+    after do
+      Delayed::Worker.backend = :test
+    end
+
     it "sets the Delayed::Job constant to the backend" do
       expect(Delayed::Job).to eq(@clazz)
     end
@@ -37,4 +41,21 @@ describe Delayed::Worker do
       Delayed::Job.reserve(Delayed::Worker.new)
     end
   end
+
+  context "worker exit on complete" do
+    before do
+      Delayed::Worker.exit_on_complete = true
+    end
+
+    after do
+      Delayed::Worker.exit_on_complete = false
+    end
+
+    it "exits the loop when no jobs are available" do
+      worker = Delayed::Worker.new
+      Timeout::timeout(2) do
+        worker.start
+      end
+    end
+  end
 end

metadata

@@ -1,137 +1,61 @@
 --- !ruby/object:Gem::Specification
 name: delayed_job
 version: !ruby/object:Gem::Version
-  version: 3.0.4
   prerelease: 
+  version: 3.0.5
 platform: ruby
 authors:
-- Matt Griffin
+- Brandon Keepers
 - Brian Ryckbost
-- Steve Richert
 - Chris Gaffney
-- Brandon Keepers
-- Tobias Lütke
 - David Genord II
+- Erik Michaels-Ober
+- Matt Griffin
+- Steve Richert
+- Tobias Lütke
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-11-09 00:00:00.000000000 Z
+date: 2013-01-28 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
-  name: activesupport
-  requirement: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ~>
-      - !ruby/object:Gem::Version
-        version: '3.0'
   type: :runtime
-  prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
       - !ruby/object:Gem::Version
         version: '3.0'
-- !ruby/object:Gem::Dependency
-  name: activerecord
-  requirement: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ~>
-      - !ruby/object:Gem::Version
-        version: '3.0'
-  type: :development
+  name: activesupport
   prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ~>
-      - !ruby/object:Gem::Version
-        version: '3.0'
-- !ruby/object:Gem::Dependency
-  name: actionmailer
   requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
       - !ruby/object:Gem::Version
         version: '3.0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ~>
-      - !ruby/object:Gem::Version
-        version: '3.0'
-- !ruby/object:Gem::Dependency
-  name: rspec
-  requirement: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ~>
-      - !ruby/object:Gem::Version
-        version: '2.0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ~>
-      - !ruby/object:Gem::Version
-        version: '2.0'
-- !ruby/object:Gem::Dependency
-  name: rake
-  requirement: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ! '>='
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ! '>='
-      - !ruby/object:Gem::Version
-        version: '0'
-- !ruby/object:Gem::Dependency
-  name: simplecov
-  requirement: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ! '>='
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ! '>='
-      - !ruby/object:Gem::Version
-        version: '0'
-description: ! 'Delayed_job (or DJ) encapsulates the common pattern of asynchronously
+description: Delayed_job (or DJ) encapsulates the common pattern of asynchronously
   executing longer tasks in the background. It is a direct extraction from Shopify
   where the job table is responsible for a multitude of core tasks.
-
-
-  This gem is collectiveidea''s fork (http://github.com/collectiveidea/delayed_job).'
 email:
 - brian@collectiveidea.com
 executables: []
 extensions: []
-extra_rdoc_files:
-- README.textile
+extra_rdoc_files: []
 files:
+- CHANGELOG.md
+- CONTRIBUTING.md
+- LICENSE.md
+- README.md
+- Rakefile
+- delayed_job.gemspec
 - contrib/delayed_job.monitrc
 - contrib/delayed_job_multiple.monitrc
 - lib/delayed/backend/base.rb
 - lib/delayed/backend/shared_spec.rb
 - lib/delayed/command.rb
 - lib/delayed/deserialization_error.rb
+- lib/delayed/exceptions.rb
 - lib/delayed/lifecycle.rb
 - lib/delayed/message_sending.rb
 - lib/delayed/performable_mailer.rb
@@ -165,16 +89,11 @@ files:
 - spec/test_backend_spec.rb
 - spec/worker_spec.rb
 - spec/yaml_ext_spec.rb
-- MIT-LICENSE
-- README.textile
 homepage: http://github.com/collectiveidea/delayed_job
-licenses: []
+licenses:
+- MIT
 post_install_message: 
-rdoc_options:
-- --main
-- README.textile
-- --inline-source
-- --line-numbers
+rdoc_options: []
 require_paths:
 - lib
 required_ruby_version: !ruby/object:Gem::Requirement
@@ -182,19 +101,19 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ! '>='
     - !ruby/object:Gem::Version
-      version: '0'
       segments:
       - 0
-      hash: 1239923890963384547
+      hash: -2181840438794321912
+      version: '0'
 required_rubygems_version: !ruby/object:Gem::Requirement
   none: false
   requirements:
   - - ! '>='
     - !ruby/object:Gem::Version
-      version: '0'
       segments:
       - 0
-      hash: 1239923890963384547
+      hash: -2181840438794321912
+      version: '0'
 requirements: []
 rubyforge_project: 
 rubygems_version: 1.8.23