que 0.7.3 → 0.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 321a6fede008be064052ea5d884a489e47414869
-  data.tar.gz: 8dc8a7f432dd4329c86aeef470d8d396e60b82e6
+  metadata.gz: f0030251d843d06847bfb8e9e12658b952edbdf2
+  data.tar.gz: 455c5a6732b86d0e3ade3c961613d206160f9fd9
 SHA512:
-  metadata.gz: 416d2717f3f4fad284cd9a3852717cb04c0ed86d7cbafa6ad88c27a120294f88a5034c2ae80143ca87a3ff4736c0e209bca64ad295f726db2b6d6292a3a3a3d1
-  data.tar.gz: 780e42518a559ea84efe70a42d58378fd22b9a023d4e9e1a14ceb74ba12e981d877303f029f2c319a510415c3f430ba290549b9d14c42c258e3ffff3855afbc5
+  metadata.gz: 669370c39a3b68c3cdc9ff6c9d8ebdd865690451175bc84062696a961a5cf5e3dbe81761fb97c08892731895ebd398001b73aed4158c99010bcdd2707bd10073
+  data.tar.gz: 8c7270f5fc20dee14e28a69468cd01a348e7e24c358ba48fb19a1c2d8562e3d05b747649a800edefda5a4d68fcf5a2c623ba1ffc09441dbdf20bed53c32c97c9

data/CHANGELOG.md

@@ -1,3 +1,11 @@
+### 0.8.0 (2014-07-12)
+
+*   A callable can now be set as the logger, like `Que.logger = proc { MyLogger.new }`. Que uses this in its Railtie for cleaner initialization, but it is also available for public use.
+
+*   `Que.mode=` and `Que.worker_count=` now function independently. That is, setting the worker_count to a nonzero number no longer sets mode = :async (triggering the pool to start working jobs), and setting it to zero no longer sets mode = :off. Similarly, setting the mode to :async no longer sets the worker_count to 4 from 0, and setting the mode to :off no longer sets the worker_count to 0. This behavior was changed because it was interfering with configuration during initialization of Rails applications, and because it was unexpected. (#47)
+
+*   Fixed a similar bug wherein setting a wake_interval during application startup would break worker awakening after the process was forked.
+
 ### 0.7.3 (2014-05-19)
 
 *   When mode = :sync, don't touch the database at all when running jobs inline. Needed for ActiveJob compatibility (#46).

data/README.md

@@ -102,6 +102,8 @@ To determine what happens when a job is queued, you can set Que's mode in your a
   * `config.que.mode = :async` - In this mode, a pool of background workers is spun up, each running in their own thread. This is the default when running `bin/rails server`. See the docs for [more information on managing workers](https://github.com/chanks/que/blob/master/docs/managing_workers.md).
   * `config.que.mode = :sync` - In this mode, any jobs you queue will be run in the same thread, synchronously (that is, `MyJob.enqueue` runs the job and won't return until it's completed). This makes your application's behavior easier to test, so it's the default in the test environment.
 
+If you're using ActiveRecord to dump your database's schema, you'll probably want to [set schema_format to :sql](http://guides.rubyonrails.org/migrations.html#types-of-schema-dumps) so that Que's table structure is managed correctly.
+
 ## Contributing
 
   1. Fork it
@@ -109,7 +111,7 @@ To determine what happens when a job is queued, you can set Que's mode in your a
   3. Commit your changes (`git commit -am 'Add some feature'`)
   4. Push to the branch (`git push origin my-new-feature`)
   5. Create new Pull Request
-  
+
 A note on running specs - Que's worker system is multithreaded and therefore prone to race conditions (especially on Rubinius). As such, if you've touched that code, a single spec run passing isn't a guarantee that any changes you've made haven't introduced bugs. One thing I like to do before pushing changes is rerun the specs many times and watching for hangs. You can do this from the command line with something like:
 
     for i in {1..1000}; do rspec -b --seed $i; done

data/docs/advanced_setup.md

@@ -20,14 +20,25 @@ There are other docs to read if you're using [Sequel](https://github.com/chanks/
 
 ### Forking Servers
 
-If you want to run a worker pool in your web process and you're using a forking webserver like Unicorn or Puma in some configurations, you'll want to set `Que.mode = :off` in your application configuration and only start up the worker pool in the child processes. So, for Puma:
+If you want to run a worker pool in your web process and you're using a forking webserver like Unicorn or Puma in some configurations, you'll want to set `Que.mode = :off` in your application configuration and only start up the worker pool in the child processes after the DB connection has been reestablished. So, for Puma:
 
     # config/puma.rb
     on_worker_boot do
-      # Reestablish your database connection, etc...
+      ActiveRecord::Base.establish_connection
+
       Que.mode = :async
     end
 
+And for Unicorn:
+
+    # config/unicorn.rb
+    after_fork do |server, worker|
+      ActiveRecord::Base.establish_connection
+
+      Que.mode = :async
+    end
+
+
 ### Managing the Jobs Table
 
 After you've connected Que to the database, you can manage the jobs table:

data/lib/que.rb

@@ -17,8 +17,8 @@ module Que
   end
 
   class << self
-    attr_accessor :logger, :error_handler
-    attr_writer :adapter, :log_formatter
+    attr_accessor :error_handler
+    attr_writer :logger, :adapter, :log_formatter
 
     def connection=(connection)
       self.adapter =
@@ -83,11 +83,15 @@ module Que
       level = data.delete(:level) || :info
       data = {:lib => 'que', :hostname => Socket.gethostname, :pid => Process.pid, :thread => Thread.current.object_id}.merge(data)
 
-      if logger && output = log_formatter.call(data)
-        logger.send level, output
+      if (l = logger) && output = log_formatter.call(data)
+        l.send level, output
       end
     end
 
+    def logger
+      @logger.respond_to?(:call) ? @logger.call : @logger
+    end
+
     def log_formatter
       @log_formatter ||= JSON_MODULE.method(:dump)
     end

data/lib/que/railtie.rb

@@ -2,6 +2,7 @@ module Que
   class Railtie < Rails::Railtie
     config.que = Que
 
+    Que.logger     = proc { Rails.logger }
     Que.mode       = :sync if Rails.env.test?
     Que.connection = ::ActiveRecord if defined? ::ActiveRecord
 
@@ -11,8 +12,6 @@ module Que
 
     initializer 'que.setup' do
       ActiveSupport.on_load :after_initialize do
-        Que.logger ||= Rails.logger
-
         # Only start up the worker pool if running as a server.
         Que.mode ||= :async if defined? Rails::Server
 

data/lib/que/sql.rb

@@ -48,7 +48,7 @@ module Que
     :set_error => %{
       UPDATE que_jobs
       SET error_count = $1::integer,
-          run_at      = now() + $2::integer * '1 second'::interval,
+          run_at      = now() + $2::bigint * '1 second'::interval,
           last_error  = $3::text
       WHERE queue     = $4::text
       AND   priority  = $5::smallint

data/lib/que/version.rb

@@ -1,3 +1,3 @@
 module Que
-  Version = '0.7.3'
+  Version = '0.8.0'
 end

data/lib/que/worker.rb

@@ -71,30 +71,33 @@ module Que
 
     def work_loop
       loop do
-        time   = Time.now
-        cycle  = nil
-        result = Job.work(queue)
-
-        case result[:event]
-        when :job_unavailable
-          cycle = false
-          result[:level] = :debug
-        when :job_race_condition
-          cycle = true
-          result[:level] = :debug
-        when :job_worked
-          cycle = true
-          result[:elapsed] = (Time.now - time).round(5)
-        when :job_errored
-          # For PG::Errors, assume we had a problem reaching the database, and
-          # don't hit it again right away.
-          cycle = !result[:error].is_a?(PG::Error)
-          result[:error] = {:class => result[:error].class.to_s, :message => result[:error].message}
-        else
-          raise "Unknown Event: #{result[:event].inspect}"
-        end
+        cycle = nil
+
+        if Que.mode == :async
+          time   = Time.now
+          result = Job.work(queue)
+
+          case result[:event]
+          when :job_unavailable
+            cycle = false
+            result[:level] = :debug
+          when :job_race_condition
+            cycle = true
+            result[:level] = :debug
+          when :job_worked
+            cycle = true
+            result[:elapsed] = (Time.now - time).round(5)
+          when :job_errored
+            # For PG::Errors, assume we had a problem reaching the database, and
+            # don't hit it again right away.
+            cycle = !result[:error].is_a?(PG::Error)
+            result[:error] = {:class => result[:error].class.to_s, :message => result[:error].message}
+          else
+            raise "Unknown Event: #{result[:event].inspect}"
+          end
 
-        Que.log(result)
+          Que.log(result)
+        end
 
         synchronize { @state = :sleeping unless cycle || @stop }
         sleep if @state == :sleeping
@@ -111,35 +114,40 @@ module Que
     # changed in Que.wake_interval= below.
     @wake_interval = 5
 
+    # Four workers is a sensible default for most use cases.
+    @worker_count = 4
+
     class << self
-      attr_reader :mode, :wake_interval
+      attr_reader :mode, :wake_interval, :worker_count
+
+      # In order to work in a forking webserver, we need to be able to accept
+      # worker_count and wake_interval settings without actually instantiating
+      # the relevant threads until the mode is actually set to :async in a
+      # post-fork hook (since forking will kill any running background threads).
 
       def mode=(mode)
-        case set_mode(mode)
-        when :async
-          set_worker_count 4 if worker_count.zero?
-        when :sync, :off
-          set_worker_count 0
-        end
-      end
+        Que.log :event => 'mode_change', :value => mode.to_s
+        @mode = mode
 
-      def workers
-        @workers ||= []
+        if mode == :async
+          set_up_workers
+          wrangler
+        end
       end
 
       def worker_count=(count)
-        set_mode(count > 0 ? :async : :off)
-        set_worker_count(count)
-        wrangler # Make sure the wrangler thread has been instantiated.
+        Que.log :event => 'worker_count_change', :value => count.to_s
+        @worker_count = count
+        set_up_workers if mode == :async
       end
 
-      def worker_count
-        workers.count
+      def workers
+        @workers ||= []
       end
 
       def wake_interval=(interval)
         @wake_interval = interval
-        wrangler.wakeup
+        wrangler.wakeup if mode == :async
       end
 
       def wake!
@@ -152,30 +160,19 @@ module Que
 
       private
 
+      def set_up_workers
+        if worker_count > workers.count
+          workers.push *(worker_count - workers.count).times.map{new(ENV['QUE_QUEUE'] || '')}
+        elsif worker_count < workers.count
+          workers.pop(workers.count - worker_count).each(&:stop).each(&:wait_until_stopped)
+        end
+      end
+
       def wrangler
         @wrangler ||= Thread.new do
           loop do
             sleep *@wake_interval
-            wake! if @wake_interval
-          end
-        end
-      end
-
-      def set_mode(mode)
-        if mode != @mode
-          Que.log :event => 'mode_change', :value => mode.to_s
-          @mode = mode
-        end
-      end
-
-      def set_worker_count(count)
-        if count != worker_count
-          Que.log :event => 'worker_count_change', :value => count.to_s
-
-          if count > worker_count
-            workers.push *(count - worker_count).times.map{new(ENV['QUE_QUEUE'] || '')}
-          elsif count < worker_count
-            workers.pop(worker_count - count).each(&:stop).each(&:wait_until_stopped)
+            wake! if @wake_interval && mode == :async
           end
         end
       end

data/spec/adapters/active_record_spec.rb

@@ -71,6 +71,7 @@ unless defined?(RUBY_ENGINE) && RUBY_ENGINE == 'jruby'
 
     it "should support Rails' special extensions for times" do
       Que.mode = :async
+      Que.worker_count = 4
       sleep_until { Que::Worker.workers.all? &:sleeping? }
 
       Que::Job.enqueue :run_at => 1.minute.ago
@@ -82,6 +83,7 @@ unless defined?(RUBY_ENGINE) && RUBY_ENGINE == 'jruby'
 
     it "should wake up a Worker after queueing a job in async mode, waiting for a transaction to commit if necessary" do
       Que.mode = :async
+      Que.worker_count = 4
       sleep_until { Que::Worker.workers.all? &:sleeping? }
 
       # Wakes a worker immediately when not in a transaction.

data/spec/adapters/sequel_spec.rb

@@ -28,6 +28,7 @@ describe "Que using the Sequel adapter" do
 
   it "should wake up a Worker after queueing a job in async mode, waiting for a transaction to commit if necessary" do
     Que.mode = :async
+    Que.worker_count = 4
     sleep_until { Que::Worker.workers.all? &:sleeping? }
 
     # Wakes a worker immediately when not in a transaction.

data/spec/spec_helper.rb

@@ -92,15 +92,21 @@ RSpec.configure do |config|
     # helpful in identifying hanging specs.
     stdout.info "Running spec: #{desc} @ #{line}" if ENV['LOG_SPEC']
 
-    $logger.messages.clear
     Que.adapter = QUE_ADAPTERS[:pg]
 
+    Que.worker_count = 0
+    Que.mode = :async
+    Que.wake_interval = nil
+
+    $logger.messages.clear
+
     spec.run
 
-    DB[:que_jobs].delete
+    Que.worker_count = 0
     Que.mode = :off
     Que.wake_interval = nil
-    sleep_until { Que::Worker.workers.all?(&:sleeping?) }
+
+    DB[:que_jobs].delete
 
     # A bit of lint: make sure that no advisory locks are left open.
     unless DB[:pg_locks].where(:locktype => 'advisory').empty?

data/spec/unit/logging_spec.rb

@@ -14,8 +14,30 @@ describe "Logging" do
     message['thread'].should == Thread.current.object_id
   end
 
+  it "should allow a callable to be set as the logger" do
+    begin
+      # Make sure we can get through a work cycle without a logger.
+      Que.logger = proc { $logger }
+
+      Que::Job.enqueue
+      worker = Que::Worker.new
+      sleep_until { worker.sleeping? }
+
+      DB[:que_jobs].should be_empty
+
+      worker.stop
+      worker.wait_until_stopped
+
+      $logger.messages.count.should be 2
+      $logger.messages.map{|m| JSON.load(m)['event']}.should == ['job_worked', 'job_unavailable']
+    ensure
+      Que.logger = $logger
+    end
+  end
+
   it "should not raise an error when no logger is present" do
     begin
+      # Make sure we can get through a work cycle without a logger.
       Que.logger = nil
 
       Que::Job.enqueue

data/spec/unit/pool_spec.rb

@@ -4,140 +4,277 @@ describe "Managing the Worker pool" do
   it "should log mode changes" do
     Que.mode = :sync
     Que.mode = :off
+    Que.mode = :off
 
-    $logger.messages.count.should be 2
-    m1, m2 = $logger.messages.map{|m| JSON.load(m)}
+    $logger.messages.count.should be 3
+    m1, m2, m3 = $logger.messages.map { |m| JSON.load(m) }
 
     m1['event'].should == 'mode_change'
     m1['value'].should == 'sync'
 
     m2['event'].should == 'mode_change'
     m2['value'].should == 'off'
+
+    m3['event'].should == 'mode_change'
+    m3['value'].should == 'off'
   end
 
-  describe "Que.mode = :sync" do
-    it "should make jobs run in the same thread as they are queued" do
-      Que.mode = :sync
+  describe "Que.mode=" do
+    describe ":off" do
+      it "with worker_count 0 should not instantiate workers or hit the db" do
+        Que.connection = nil
+        Que.worker_count = 0
+        Que.mode = :off
+        Que::Worker.workers.should == []
+      end
 
-      ArgsJob.enqueue(5, :testing => "synchronous").should be_an_instance_of ArgsJob
-      $passed_args.should == [5, {:testing => "synchronous"}]
-      DB[:que_jobs].count.should be 0
+      it "with worker_count > 0 should not instantiate workers or hit the db" do
+        Que.connection = nil
+        Que.mode = :off
+        Que.worker_count = 5
+        Que.mode = :off
+        Que::Worker.workers.should == []
+      end
     end
 
-    it "should work fine with enqueuing jobs without a DB connection" do
-      Que.connection = nil
-      Que.mode = :sync
+    describe ":sync" do
+      it "with worker_count 0 should not instantiate workers or hit the db" do
+        Que.connection = nil
+        Que.worker_count = 0
+        Que.mode = :sync
+        Que::Worker.workers.should == []
+      end
+
+      it "with worker_count > 0 should not instantiate workers or hit the db" do
+        Que.connection = nil
+        Que.mode = :sync
+        Que.worker_count = 5
+        Que.mode = :sync
+        Que::Worker.workers.should == []
+      end
+
+      it "should make jobs run in the same thread as they are queued" do
+        Que.mode = :sync
+
+        ArgsJob.enqueue(5, :testing => "synchronous").should be_an_instance_of ArgsJob
+        $passed_args.should == [5, {:testing => "synchronous"}]
+        DB[:que_jobs].count.should be 0
+      end
+
+      it "should work fine with enqueuing jobs without a DB connection" do
+        Que.connection = nil
+        Que.mode = :sync
+
+        ArgsJob.enqueue(5, :testing => "synchronous").should be_an_instance_of ArgsJob
+        $passed_args.should == [5, {:testing => "synchronous"}]
+      end
+
+      it "should not affect jobs that are queued with specific run_ats" do
+        Que.mode = :sync
 
-      ArgsJob.enqueue(5, :testing => "synchronous").should be_an_instance_of ArgsJob
-      $passed_args.should == [5, {:testing => "synchronous"}]
+        ArgsJob.enqueue(5, :testing => "synchronous", :run_at => Time.now + 60)
+        DB[:que_jobs].select_map(:job_class).should == ["ArgsJob"]
+      end
     end
 
-    it "should not affect jobs that are queued with specific run_ats" do
-      Que.mode = :sync
+    describe ":async" do
+      it "with worker_count 0 should not instantiate workers or hit the db" do
+        Que.connection = nil
+        Que.worker_count = 0
+        Que.mode = :async
+        Que::Worker.workers.map{|w| [w.state, w.thread.status]}.should == []
+      end
+
+      it "with worker_count > 0 should instantiate workers and hit the db" do
+        Que::Job.enqueue
+        Que.worker_count = 5
+        Que.mode = :async
+        sleep_until { Que::Worker.workers.all? &:sleeping? }
+        DB[:que_jobs].count.should == 0
+        Que::Worker.workers.map{|w| [w.state, w.thread.status]}.should == [[:sleeping, 'sleep']] * 5
+      end
+
+      it "should wake a worker every Que.wake_interval seconds" do
+        Que.worker_count = 4
+        Que.mode = :async
+        sleep_until { Que::Worker.workers.all? &:sleeping? }
+        Que.wake_interval = 0.01 # 10 ms
+        Que::Job.enqueue
+        sleep_until { DB[:que_jobs].count == 0 }
+      end
+
+      it "should work jobs in the queue defined by QUE_QUEUE" do
+        begin
+          Que::Job.enqueue 1
+          Que::Job.enqueue 2, :queue => 'my_queue'
+
+          ENV['QUE_QUEUE'] = 'my_queue'
 
-      ArgsJob.enqueue(5, :testing => "synchronous", :run_at => Time.now + 60)
-      DB[:que_jobs].select_map(:job_class).should == ["ArgsJob"]
+          Que.mode = :async
+          Que.worker_count = 2
+
+          sleep_until { Que::Worker.workers.all? &:sleeping? }
+          DB[:que_jobs].count.should be 1
+
+          job = DB[:que_jobs].first
+          job[:queue].should == ''
+          job[:args].should == '[1]'
+        ensure
+          ENV.delete('QUE_QUEUE')
+        end
+      end
     end
   end
 
-  describe "Que.mode = :async" do
-    it "should spin up 4 workers" do
-      Que.mode = :async
-      Que.worker_count.should be 4
-      sleep_until { Que::Worker.workers.all?(&:sleeping?) }
-
-      $logger.messages.map{|m| JSON.load(m).values_at('event', 'value')}.should ==
-        [['mode_change', 'async'], ['worker_count_change', '4']] + [['job_unavailable', nil]] * 4
+  describe "Que.worker_count=" do
+    describe "when the mode is :off" do
+      it "should record the setting but not instantiate any workers" do
+        Que.worker_count.should == 0
+        Que.connection = nil
+        Que.mode = :off
+        Que::Worker.workers.should == []
+
+        Que.worker_count = 4
+        Que.worker_count.should == 4
+        Que::Worker.workers.should == []
+
+        Que.worker_count = 6
+        Que.worker_count.should == 6
+        Que::Worker.workers.should == []
+
+        Que.worker_count = 2
+        Que.worker_count.should == 2
+        Que::Worker.workers.should == []
+
+        Que.worker_count = 0
+        Que.worker_count.should == 0
+        Que::Worker.workers.should == []
+      end
     end
 
-    it "should be done automatically when setting a worker count" do
-      Que.worker_count = 2
-      Que.mode.should == :async
-      Que.worker_count.should == 2
-      sleep_until { Que::Worker.workers.all?(&:sleeping?) }
+    describe "when the mode is :sync" do
+      it "should record the setting but not instantiate any workers" do
+        Que.worker_count.should == 0
+        Que.connection = nil
+        Que.mode = :sync
+        Que::Worker.workers.should == []
 
-      $logger.messages.map{|m| JSON.load(m).values_at('event', 'value')}.should ==
-        [['mode_change', 'async'], ['worker_count_change', '2']] + [['job_unavailable', nil]] * 2
-    end
+        Que.worker_count = 4
+        Que.worker_count.should == 4
+        Que::Worker.workers.should == []
 
-    it "should not affect the number of workers if a worker_count has already been set" do
-      Que.worker_count = 1
-      Que.mode = :async
-      Que.worker_count.should be 1
-      sleep_until { Que::Worker.workers.all?(&:sleeping?) }
+        Que.worker_count = 6
+        Que.worker_count.should == 6
+        Que::Worker.workers.should == []
 
-      $logger.messages.map{|m| JSON.load(m).values_at('event', 'value')}.should ==
-        [['mode_change', 'async'], ['worker_count_change', '1'], ['job_unavailable', nil]]
+        Que.worker_count = 2
+        Que.worker_count.should == 2
+        Que::Worker.workers.should == []
+
+        Que.worker_count = 0
+        Que.worker_count.should == 0
+        Que::Worker.workers.should == []
+      end
     end
 
-    it "then Que.worker_count = 0 should set the mode to :off" do
-      Que.mode = :async
-      Que.worker_count.should be 4
+    describe "when the mode is :async" do
+      it "should start hitting the DB when transitioning to a non-zero value" do
+        Que.mode = :async
+        Que::Job.enqueue
+        Que.worker_count = 4
+        sleep_until { Que::Worker.workers.all?(&:sleeping?) }
+        Que::Worker.workers.map{|w| [w.state, w.thread.status]}.should == [[:sleeping, 'sleep']] * 4
+        DB[:que_jobs].count.should == 0
+      end
 
-      sleep_until { Que::Worker.workers.all?(&:sleeping?) }
+      it "should stop hitting the DB when transitioning to zero" do
+        Que.mode = :async
+        Que.worker_count = 4
+        sleep_until { Que::Worker.workers.all?(&:sleeping?) }
+        Que.connection = nil
+        Que.worker_count = 0
+        $logger.messages.map{|m| JSON.load(m).values_at('event', 'value')}.should ==
+          [['mode_change', 'async'], ['worker_count_change', '4']] + [['job_unavailable', nil]] * 4 + [['worker_count_change', '0']]
+      end
 
-      Que.worker_count = 0
-      Que.worker_count.should == 0
-      Que.mode.should == :off
+      it "should be able to scale down the number of workers gracefully" do
+        Que.mode = :async
+        Que.worker_count = 4
 
-      $logger.messages.map{|m| JSON.load(m).values_at('event', 'value')}.should ==
-        [['mode_change', 'async'], ['worker_count_change', '4']] + [['job_unavailable', nil]] * 4 + [['mode_change', 'off'], ['worker_count_change', '0']]
-    end
+        workers = Que::Worker.workers.dup
+        workers.count.should be 4
+        sleep_until { Que::Worker.workers.all?(&:sleeping?) }
 
-    it "then Que.worker_count = 2 should gracefully decrease the number of workers" do
-      Que.mode = :async
-      workers = Que::Worker.workers.dup
-      workers.count.should be 4
-      sleep_until { Que::Worker.workers.all?(&:sleeping?) }
+        Que.worker_count = 2
+        Que::Worker.workers.count.should be 2
+        sleep_until { Que::Worker.workers.all?(&:sleeping?) }
 
-      Que.worker_count = 2
-      Que.worker_count.should be 2
-      sleep_until { Que::Worker.workers.all?(&:sleeping?) }
+        workers[0..1].should == Que::Worker.workers
+        workers[2..3].each do |worker|
+          worker.should be_an_instance_of Que::Worker
+          worker.thread.status.should == false
+        end
 
-      workers[0..1].should == Que::Worker.workers
-      workers[2..3].each do |worker|
-        worker.should be_an_instance_of Que::Worker
-        worker.thread.status.should == false
+        $logger.messages.map{|m| JSON.load(m).values_at('event', 'value')}.should ==
+          [['mode_change', 'async'], ['worker_count_change', '4']] + [['job_unavailable', nil]] * 4 + [['worker_count_change', '2']]
       end
 
-      $logger.messages.map{|m| JSON.load(m).values_at('event', 'value')}.should ==
-        [['mode_change', 'async'], ['worker_count_change', '4']] + [['job_unavailable', nil]] * 4 + [['worker_count_change', '2']]
-    end
-
-    it "then Que.worker_count = 6 should gracefully increase the number of workers" do
-      Que.mode = :async
-      workers = Que::Worker.workers.dup
-      workers.count.should be 4
+      it "should be able to scale up the number of workers gracefully" do
+        Que.mode = :async
+        Que.worker_count = 4
+        workers = Que::Worker.workers.dup
+        workers.count.should be 4
 
-      sleep_until { Que::Worker.workers.all?(&:sleeping?) }
-      Que.worker_count = 6
-      Que.worker_count.should be 6
-      sleep_until { Que::Worker.workers.all?(&:sleeping?) }
+        sleep_until { Que::Worker.workers.all?(&:sleeping?) }
+        Que.worker_count = 6
+        Que::Worker.workers.count.should be 6
+        sleep_until { Que::Worker.workers.all?(&:sleeping?) }
 
-      workers.should == Que::Worker.workers[0..3]
+        workers.should == Que::Worker.workers[0..3]
 
-      $logger.messages.map{|m| JSON.load(m).values_at('event', 'value')}.should ==
-        [['mode_change', 'async'], ['worker_count_change', '4']] + [['job_unavailable', nil]] * 4 + [['worker_count_change', '6']] + [['job_unavailable', nil]] * 2
+        $logger.messages.map{|m| JSON.load(m).values_at('event', 'value')}.should ==
+          [['mode_change', 'async'], ['worker_count_change', '4']] + [['job_unavailable', nil]] * 4 + [['worker_count_change', '6']] + [['job_unavailable', nil]] * 2
+      end
     end
+  end
 
-    it "then Que.mode = :off should gracefully shut down workers" do
-      Que.mode = :async
-      workers = Que::Worker.workers.dup
-      workers.count.should be 4
-
-      sleep_until { Que::Worker.workers.all?(&:sleeping?) }
+  describe "Que.wake!" do
+    it "when mode = :off should do nothing" do
+      Que.connection = nil
       Que.mode = :off
-      Que.worker_count.should be 0
+      Que.worker_count = 4
+      sleep_until { Que::Worker.workers.all? &:sleeping? }
+      Que.wake!
+      sleep_until { Que::Worker.workers.all? &:sleeping? }
+      $logger.messages.map{|m| JSON.load(m).values_at('event', 'value')}.should ==
+        [['mode_change', 'off'], ['worker_count_change', '4']]
+    end
 
-      workers.count.should be 4
-      workers.each { |worker| worker.thread.status.should be false }
+    it "when mode = :sync should do nothing" do
+      Que.connection = nil
+      Que.mode = :sync
+      Que.worker_count = 4
+      sleep_until { Que::Worker.workers.all? &:sleeping? }
+      Que.wake!
+      sleep_until { Que::Worker.workers.all? &:sleeping? }
+      $logger.messages.map{|m| JSON.load(m).values_at('event', 'value')}.should ==
+        [['mode_change', 'sync'], ['worker_count_change', '4']]
+    end
 
+    it "when mode = :async and worker_count = 0 should do nothing" do
+      Que.connection = nil
+      Que.mode = :async
+      Que.worker_count = 0
+      sleep_until { Que::Worker.workers.all? &:sleeping? }
+      Que.wake!
+      sleep_until { Que::Worker.workers.all? &:sleeping? }
       $logger.messages.map{|m| JSON.load(m).values_at('event', 'value')}.should ==
-        [['mode_change', 'async'], ['worker_count_change', '4']] + [['job_unavailable', nil]] * 4 + [['mode_change', 'off'], ['worker_count_change', '0']]
+        [['mode_change', 'async'], ['worker_count_change', '0']]
     end
 
-    it "then Que.wake! should wake up a single worker" do
+    it "when mode = :async and worker_count > 0 should wake up a single worker" do
       Que.mode = :async
+      Que.worker_count = 4
       sleep_until { Que::Worker.workers.all? &:sleeping? }
 
       BlockJob.enqueue
@@ -153,17 +290,55 @@ describe "Managing the Worker pool" do
       DB[:que_jobs].count.should be 0
     end
 
-    it "then Que.wake! should be thread-safe" do
+    it "when mode = :async and worker_count > 0 should be thread-safe" do
       Que.mode = :async
+      Que.worker_count = 4
+      sleep_until { Que::Worker.workers.all? &:sleeping? }
       threads = 4.times.map { Thread.new { 100.times { Que.wake! } } }
       threads.each(&:join)
     end
+  end
+
+  describe "Que.wake_all!" do
+    it "when mode = :off should do nothing" do
+      Que.connection = nil
+      Que.mode = :off
+      Que.worker_count = 4
+      sleep_until { Que::Worker.workers.all? &:sleeping? }
+      Que.wake_all!
+      sleep_until { Que::Worker.workers.all? &:sleeping? }
+      $logger.messages.map{|m| JSON.load(m).values_at('event', 'value')}.should ==
+        [['mode_change', 'off'], ['worker_count_change', '4']]
+    end
 
-    it "then Que.wake_all! should wake up all workers" do
-      # This spec requires at least four connections.
-      Que.adapter = QUE_ADAPTERS[:connection_pool]
+    it "when mode = :sync should do nothing" do
+      Que.connection = nil
+      Que.mode = :sync
+      Que.worker_count = 4
+      sleep_until { Que::Worker.workers.all? &:sleeping? }
+      Que.wake_all!
+      sleep_until { Que::Worker.workers.all? &:sleeping? }
+      $logger.messages.map{|m| JSON.load(m).values_at('event', 'value')}.should ==
+        [['mode_change', 'sync'], ['worker_count_change', '4']]
+    end
+
+    it "when mode = :async and worker_count = 0 should do nothing" do
+      Que.connection = nil
+      Que.mode = :async
+      Que.worker_count = 0
+      sleep_until { Que::Worker.workers.all? &:sleeping? }
+      Que.wake_all!
+      sleep_until { Que::Worker.workers.all? &:sleeping? }
+      $logger.messages.map{|m| JSON.load(m).values_at('event', 'value')}.should ==
+        [['mode_change', 'async'], ['worker_count_change', '0']]
+    end
+
+    # This spec requires at least four connections.
+    it "when mode = :async and worker_count > 0 should wake up all workers" do
+      Que.adapter = QUE_ADAPTERS[:pond]
 
       Que.mode = :async
+      Que.worker_count = 4
       sleep_until { Que::Worker.workers.all? &:sleeping? }
 
       4.times { BlockJob.enqueue }
@@ -175,44 +350,14 @@ describe "Managing the Worker pool" do
 
       sleep_until { Que::Worker.workers.all? &:sleeping? }
       DB[:que_jobs].count.should be 0
-    end if QUE_ADAPTERS[:connection_pool]
+    end if QUE_ADAPTERS[:pond]
 
-    it "then Que.wake_all! should be thread-safe" do
+    it "when mode = :async and worker_count > 0 should be thread-safe" do
       Que.mode = :async
+      Que.worker_count = 4
+      sleep_until { Que::Worker.workers.all? &:sleeping? }
       threads = 4.times.map { Thread.new { 100.times { Que.wake_all! } } }
       threads.each(&:join)
     end
-
-    it "should wake a worker every Que.wake_interval seconds" do
-      Que.mode = :async
-      sleep_until { Que::Worker.workers.all? &:sleeping? }
-      Que.wake_interval = 0.01 # 10 ms
-      Que::Job.enqueue
-      sleep_until { DB[:que_jobs].count == 0 }
-    end
-
-    it "should work jobs in the queue defined by QUE_QUEUE" do
-      begin
-        Que::Job.enqueue 1
-        Que::Job.enqueue 2, :queue => 'my_queue'
-
-        ENV['QUE_QUEUE'] = 'my_queue'
-
-        Que.mode = :async
-        sleep_until { Que::Worker.workers.all? &:sleeping? }
-        DB[:que_jobs].count.should be 1
-
-        job = DB[:que_jobs].first
-        job[:queue].should == ''
-        job[:args].should == '[1]'
-      ensure
-        ENV.delete('QUE_QUEUE')
-
-        if @worker
-          @worker.stop
-          @worker.wait_until_stopped
-        end
-      end
-    end
   end
 end

data/spec/unit/work_spec.rb

@@ -1,4 +1,5 @@
 require 'spec_helper'
+require 'active_support/core_ext/date' # For the .seconds.from_now below
 
 describe Que::Job, '.work' do
   it "should pass a job's arguments to the run method and delete it from the database" do
@@ -237,7 +238,7 @@ describe Que::Job, '.work' do
 
     it "should respect a custom retry interval" do
       class RetryIntervalJob < ErrorJob
-        @retry_interval = 5
+        @retry_interval = 3155760000000 # 100,000 years from now
       end
 
       RetryIntervalJob.enqueue
@@ -251,7 +252,7 @@ describe Que::Job, '.work' do
       job = DB[:que_jobs].first
       job[:error_count].should be 1
       job[:last_error].should =~ /\AErrorJob!\n/
-      job[:run_at].should be_within(3).of Time.now + 5
+      job[:run_at].to_f.should be_within(3).of Time.now.to_f + RetryIntervalJob.retry_interval
 
       DB[:que_jobs].update :error_count => 5,
                            :run_at => Time.now - 60
@@ -265,7 +266,7 @@ describe Que::Job, '.work' do
       job = DB[:que_jobs].first
       job[:error_count].should be 6
       job[:last_error].should =~ /\AErrorJob!\n/
-      job[:run_at].should be_within(3).of Time.now + 5
+      job[:run_at].to_f.should be_within(3).of Time.now.to_f + RetryIntervalJob.retry_interval
     end
 
     it "should respect a custom retry interval formula" do

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: que
 version: !ruby/object:Gem::Version
-  version: 0.7.3
+  version: 0.8.0
 platform: ruby
 authors:
 - Chris Hanks
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2014-05-19 00:00:00.000000000 Z
+date: 2014-07-13 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler