sidetiq 0.2.0 → 0.3.0

data/.gitignore

@@ -7,3 +7,4 @@ doc
 Gemfile.lock
 pkg
 tmp
+dump.rdb

data/CHANGELOG.md

@@ -1,3 +1,15 @@
+0.3.0
+-----
+
+- Add `Sidetiq.schedules`.
+- Add `Sidetiq.workers`.
+- Add `Sidetiq.scheduled`.
+- Add `Sidetiq.retries`.
+- Add `Sidetiq.logger`. This defaults to the Sidekiq logger.
+- Add support for job backfills.
+- Clean up tests.
+- Sidetiq::Schedule no longer inherits from IceCube::Schedule.
+
 0.2.0
 -----
 

data/README.md

@@ -6,7 +6,27 @@ Sidetiq
 
 Recurring jobs for [Sidekiq](http://mperham.github.com/sidekiq/).
 
-## DESCRIPTION
+Table Of Contents
+-----------------
+
+   * [Overview](#section_Overview)
+   * [Dependencies](#section_Dependencies)
+   * [Installation](#section_Installation)
+   * [Introduction](#section_Introduction)
+   * [Backfills](#section_Backfills)
+   * [Configuration](#section_Configuration)
+      * [Logging](#section_Configuration_Logging)
+   * [API](#section_API)
+   * [Polling](#section_Polling)
+   * [Known Issues](#section_Known_Issues)
+   * [Web Extension](#section_Web_Extension)
+   * [Contribute](#section_Contribute)
+   * [License](#section_License)
+   * [Author](#section_Author)
+
+<a name='section_Overview'></a>
+Overview
+--------
 
 Sidetiq provides a simple API for defining recurring workers for Sidekiq.
 
@@ -20,12 +40,18 @@ Sidetiq provides a simple API for defining recurring workers for Sidekiq.
   each other (tested with sub-second polling of scheduled jobs by Sidekiq and
   Sidetiq clock rates above 100hz).
 
-## DEPENDENCIES
+Detailed API documentation is available on [rubydoc.info](http://rdoc.info/github/tobiassvn/sidetiq/).
+
+<a name='section_Dependencies'></a>
+Dependencies
+------------
 
 - [Sidekiq](http://mperham.github.com/sidekiq/)
 - [ice_cube](http://seejohnrun.github.com/ice_cube/)
 
-## INSTALLATION
+<a name='section_Installation'></a>
+Installation
+------------
 
 The best way to install Sidetiq is with RubyGems:
 
@@ -36,7 +62,9 @@ to pick up all the gems ([more info](http://gembundler.com/bundle_install.html))
 
     $ bundle install
 
-## GETTING STARTED
+<a name='section_Introduction'></a>
+Introduction
+------------
 
 Defining recurring jobs is simple:
 
@@ -47,6 +75,10 @@ class MyWorker
 
   # Daily at midnight
   tiq { daily }
+
+  def perform
+    # do stuff ...
+  end
 end
 ```
 
@@ -64,6 +96,10 @@ class MyWorker
     # Every second year in February
     yearly(2).month_of_year(:february)
   end
+
+  def perform
+    # do stuff ...
+  end
 end
 ```
 
@@ -76,6 +112,28 @@ class MyWorker
 
   # Every other month on the first monday and last tuesday at 12 o'clock.
   tiq { monthly(2).day_of_week(1 => [1], 2 => [-1]).hour_of_day(12) }
+
+  def perform
+    # do stuff ...
+  end
+end
+```
+
+Additionally, the last and current occurrence time (as a `Float`) can be
+passed to the worker simply by adding arguments to `#perform`. Sidetiq
+will check the method arity before enqueuing the job:
+
+```ruby
+class MyWorker
+  include Sidekiq::Worker
+  include Sidetiq::Schedulable
+
+  tiq { daily }
+
+  # Receive last and current occurrence times.
+  def perform(last_occurrence, current_occurrence)
+    # do stuff ...
+  end
 end
 ```
 
@@ -91,7 +149,35 @@ end
 Additionally, Sidetiq includes a middleware that will check if the clock
 thread is still alive and restart it if necessary.
 
-## CONFIGURATION
+<a name='section_Backfills''></a>
+Backfills
+---------
+
+In certain cases it is desirable that missed jobs will be enqueued
+retroactively, for example when a critical, hourly job isn't run due to
+server downtime. To solve this, `#tiq` takes a *backfill* option. If
+missing job occurrences have been detected, Sidetiq will then enqueue
+the jobs automatically. It will also ensure that the timestamps passed to
+`#perform` are as expected:
+
+```ruby
+class MyWorker
+  include Sidekiq::Worker
+  include Sidetiq::Schedulable
+
+  tiq backfill: true do
+    hourly
+  end
+
+  def perform(last_occurrence, current_occurrence)
+    # do stuff ...
+  end
+end
+```
+
+<a name='section_Configuration'></a>
+Configuration
+-------------
 
 ```ruby
 Sidetiq.configure do |config|
@@ -109,8 +195,71 @@ Sidetiq.configure do |config|
   config.utc = false
 end
 ```
+<a name='section_Configuration_Logging'></a>
+### Logging
+
+By default Sidetiq uses Sidekiq's logger. However, this is configuration:
+
+```ruby
+Sidetiq.logger = Logger.new(STDOUT)
+```
+
+The logger should implement Ruby's [Logger API](http://www.ruby-doc.org/stdlib-1.9.3/libdoc/logger/rdoc/Logger.html).
+
+<a name='section_API'></a>
+API
+---
+
+Sidetiq implements a simple API to support reflection of recurring jobs at
+runtime:
+
+`Sidetiq.schedules` returns a `Hash` with the `Sidekiq::Worker` class as the
+key and the Sidetiq::Schedule object as the value:
+
+```ruby
+Sidetiq.schedules
+# => { MyWorker => #<Sidetiq::Schedule> }
+```
+
+`Sidetiq.workers` returns an `Array` of all workers currently tracked by
+Sidetiq (workers which include `Sidetiq::Schedulable` and a `.tiq` call):
+
+```ruby
+Sidetiq.workers
+# => [MyWorker, AnotherWorker]
+```
+
+`Sidetiq.scheduled` returns an `Array` of currently scheduled Sidetiq jobs
+as `Sidekiq::SortedEntry` (`Sidekiq::Job`) objects. Optionally, it is
+possible to pass a block to which each job will be yielded:
+
+```ruby
+Sidetiq.scheduled do |job|
+  # do stuff ...
+end
+```
+
+This list can further be filtered by passing the worker class to `#scheduled`,
+either as a String or the constant itself:
+
+```ruby
+Sidetiq.scheduled(MyWorker) do |job|
+  # do stuff ...
+end
+```
 
-## NOTES
+The same can be done for recurring jobs currently scheduled for retries
+(`.retries` wraps `Sidekiq::RetrySet` instead of `Sidekiq::ScheduledSet`):
+
+```ruby
+Sidetiq.retries(MyWorker) do |job|
+  # do stuff ...
+end
+```
+
+<a name='section_Polling'></a>
+Polling
+-------
 
 By default Sidekiq uses a 15 second polling interval to check if scheduled
 jobs are due. If a recurring job has to run more often than that you should
@@ -120,7 +269,43 @@ lower this value.
 Sidekiq.options[:poll_interval] = 1
 ```
 
-## WEB EXTENSION
+More information about this can be found in the
+[Sidekiq Wiki](https://github.com/mperham/sidekiq/wiki/Scheduled-Jobs).
+
+<a name='section_Known_Issues'></a>
+Known Issues
+------------
+
+Unfortunately, using ice_cube's interval methods is terribly slow on
+start-up (it tends to eat up 100% CPU for quite a while). This is due to it
+calculating every possible occurrence since the schedule's start time. The way
+around is to avoid using them.
+
+For example, instead of defining a job that should run every 15 minutes like this:
+
+```ruby
+class MyWorker
+  include Sidekiq::Worker
+  include Sidetiq::Schedulable
+
+  tiq { minutely(15) }
+end
+```
+
+It is better to use the more explicit way:
+
+```ruby
+class MyWorker
+  include Sidekiq::Worker
+  include Sidetiq::Schedulable
+
+  tiq { hourly.minute_of_hour(0, 15, 30, 45) }
+end
+```
+
+<a name='section_Web_Extension'></a>
+Web Extension
+-------------
 
 Sidetiq includes an extension for Sidekiq's web interface. It will not be
 loaded by default, so it will have to be required manually:
@@ -133,7 +318,9 @@ require 'sidetiq/web'
 
 ![Screenshot](http://f.cl.ly/items/1P2u1v091F3V1n381g2I/Screen%20Shot%202013-02-01%20at%2012.16.17.png)
 
-## CONTRIBUTE
+<a name='section_Contribute'></a>
+Contribute
+----------
 
 If you'd like to contribute to Sidetiq, start by forking my repo on GitHub:
 
@@ -152,6 +339,15 @@ your changes merged back into core is as follows:
 1. Push the branch up to GitHub
 1. Send a pull request to the tobiassvn/sidetiq project.
 
-## LICENSE
+<a name='section_License'></a>
+License
+-------
 
 Sidetiq is released under the MIT License. See LICENSE for further details.
+
+<a name='section_Author'></a>
+Author
+------
+
+Tobias Svensson, [@tobiassvn](https://twitter.com/tobiassvn), [http://github.com/tobiassvn](http://github.com/tobiassvn)
+

data/lib/sidetiq.rb

@@ -20,5 +20,122 @@ require 'sidetiq/version'
 
 # The Sidetiq namespace.
 module Sidetiq
-end
+  class << self
+    # Public: Setter for the Sidetiq logger.
+    attr_writer :logger
+
+    # Public: Reader for the Sidetiq logger.
+    #
+    # Defaults to `Sidekiq.logger`.
+    def logger
+      @logger ||= Sidekiq.logger
+    end
+
+    # Public: Returns an Array of workers including Sidetiq::Schedulable.
+    def workers
+      schedules.keys
+    end
+
+    # Public: Returns a Hash of Sidetiq::Schedule instances.
+    def schedules
+      Clock.synchronize do
+        Clock.schedules.dup
+      end
+    end
+
+    # Public: Currently scheduled recurring jobs.
+    #
+    # worker - A Sidekiq::Worker class or String of the class name (optional)
+    # block  - An optional block that can be given to which each
+    #          Sidekiq::SortedEntry instance corresponding to a scheduled job will
+    #          be yielded.
+    #
+    # Examples
+    #
+    #   Sidetiq.scheduled
+    #   # => [#<Sidekiq::SortedEntry>, ...]
+    #
+    #   Sidetiq.scheduled(MyWorker)
+    #   # => [#<Sidekiq::SortedEntry>, ...]
+    #
+    #   Sidetiq.scheduled("MyWorker")
+    #   # => [#<Sidekiq::SortedEntry>, ...]
+    #
+    #   Sidetiq.scheduled do |job|
+    #     # do stuff ...
+    #   end
+    #   # => [#<Sidekiq::SortedEntry>, ...]
+    #
+    #   Sidetiq.scheduled(MyWorker) do |job|
+    #     # do stuff ...
+    #   end
+    #   # => [#<Sidekiq::SortedEntry>, ...]
+    #
+    #   Sidetiq.scheduled("MyWorker") do |job|
+    #     # do stuff ...
+    #   end
+    #   # => [#<Sidekiq::SortedEntry>, ...]
+    #
+    # Yields each Sidekiq::SortedEntry instance.
+    # Returns an Array of Sidekiq::SortedEntry objects.
+    def scheduled(worker = nil, &block)
+      filter_set(Sidekiq::ScheduledSet.new, worker, &block)
+    end
+
+    # Public: Recurring jobs currently scheduled for retries.
+    #
+    # worker - A Sidekiq::Worker class or String of the class name (optional)
+    # block  - An optional block that can be given to which each
+    #          Sidekiq::SortedEntry instance corresponding to a scheduled job will
+    #          be yielded.
+    #
+    # Examples
+    #
+    #   Sidetiq.retries
+    #   # => [#<Sidekiq::SortedEntry>, ...]
+    #
+    #   Sidetiq.retries(MyWorker)
+    #   # => [#<Sidekiq::SortedEntry>, ...]
+    #
+    #   Sidetiq.retries("MyWorker")
+    #   # => [#<Sidekiq::SortedEntry>, ...]
+    #
+    #   Sidetiq.retries do |job|
+    #     # do stuff ...
+    #   end
+    #   # => [#<Sidekiq::SortedEntry>, ...]
+    #
+    #   Sidetiq.retries(MyWorker) do |job|
+    #     # do stuff ...
+    #   end
+    #   # => [#<Sidekiq::SortedEntry>, ...]
+    #
+    #   Sidetiq.retries("MyWorker") do |job|
+    #     # do stuff ...
+    #   end
+    #   # => [#<Sidekiq::SortedEntry>, ...]
+    #
+    # Yields each Sidekiq::SortedEntry instance.
+    # Returns an Array of Sidekiq::SortedEntry objects.
+    def retries(worker = nil, &block)
+      filter_set(Sidekiq::RetrySet.new, worker, &block)
+    end
 
+    private
+
+    def filter_set(set, worker, &block)
+      worker = worker.constantize if worker.kind_of?(String)
+
+      jobs = set.select do |job|
+        klass = job.klass.constantize
+        ret = klass.include?(Schedulable)
+        ret = ret && klass == worker if worker
+        ret
+      end
+
+      jobs.each(&block) if block_given?
+
+      jobs
+    end
+  end
+end

data/lib/sidetiq/clock.rb

@@ -11,14 +11,10 @@ module Sidetiq
     include Singleton
     include MonitorMixin
 
-    # Public: Start time offset from epoch used for calculating run
-    # times in the Sidetiq schedules.
-    START_TIME = Sidetiq.config.utc ? Time.utc(2010, 1, 1) : Time.local(2010, 1, 1)
-
-    # Public: Returns a hash of Sidetiq::Schedule instances.
+    # Internal: Returns a hash of Sidetiq::Schedule instances.
     attr_reader :schedules
 
-    # Public: Returns the clock thread.
+    # Internal: Returns the clock thread.
     attr_reader :thread
 
     def self.method_missing(meth, *args, &block) # :nodoc:
@@ -41,7 +37,7 @@ module Sidetiq
     #
     # Returns a Sidetiq::Schedule instances.
     def schedule_for(worker)
-      schedules[worker] ||= Sidetiq::Schedule.new(START_TIME)
+      schedules[worker] ||= Sidetiq::Schedule.new
     end
 
     # Public: Issue a single clock tick.
@@ -53,12 +49,18 @@ module Sidetiq
     #
     # Returns a hash of Sidetiq::Schedule instances.
     def tick
-      @tick = gettime
+      tick = gettime
       synchronize do
-        schedules.each do |worker, schedule|
-          if schedule.schedule_next?(@tick)
-            enqueue(worker, schedule.next_occurrence(@tick))
-          end
+        schedules.each do |worker, sched|
+          synchronize_clockworks(worker) do |redis|
+            if sched.backfill? && (last = worker.last_scheduled_occurrence) > 0
+              last = Sidetiq.config.utc ? Time.at(last).utc : Time.at(last)
+              sched.occurrences_between(last + 1, tick).each do |past_t|
+                enqueue(worker, past_t, redis)
+              end
+            end
+            enqueue(worker, sched.next_occurrence(tick), redis)
+          end if sched.schedule_next?(tick)
         end
       end
     end
@@ -89,7 +91,8 @@ module Sidetiq
     def start!
       return if ticking?
 
-      Sidekiq.logger.info "Sidetiq::Clock start"
+      Sidetiq.logger.info "Sidetiq::Clock start"
+
       @thread = Thread.start { clock { tick } }
       @thread.abort_on_exception = true
       @thread.priority = Sidetiq.config.priority
@@ -107,7 +110,7 @@ module Sidetiq
     def stop!
       if ticking?
         @thread.kill
-        Sidekiq.logger.info "Sidetiq::Clock stop"
+        Sidetiq.logger.info "Sidetiq::Clock stop"
       end
     end
 
@@ -129,36 +132,35 @@ module Sidetiq
 
     private
 
-    def enqueue(worker, time)
-      key = "sidetiq:#{worker.name}"
-      time_f = time.to_f
-
-      synchronize_clockworks("#{key}:lock") do |redis|
-        next_run = (redis.get("#{key}:next") || -1).to_f
+    def enqueue(worker, time, redis)
+      key      = "sidetiq:#{worker.name}"
+      time_f   = time.to_f
+      next_run = (redis.get("#{key}:next") || -1).to_f
 
-        if next_run < time_f
-          Sidekiq.logger.info "Sidetiq::Clock enqueue #{worker.name} (at: #{time_f}) (last: #{next_run})"
+      if next_run < time_f
+        Sidetiq.logger.info "Sidetiq::Clock enqueue #{worker.name} (at: #{time_f}) (last: #{next_run})"
 
-          redis.mset("#{key}:last", next_run, "#{key}:next", time_f)
+        redis.mset("#{key}:last", next_run, "#{key}:next", time_f)
 
-          arity = [worker.instance_method(:perform).arity - 1, -1].max
-          args = [next_run, time_f][0..arity]
+        arity = [worker.instance_method(:perform).arity - 1, -1].max
+        args = [next_run, time_f][0..arity]
 
-          worker.perform_at(time, *args)
-        end
+        worker.perform_at(time, *args)
       end
     end
 
-    def synchronize_clockworks(lock)
+    def synchronize_clockworks(klass)
+      lock = "sidetiq:#{klass.name}:lock"
+
       Sidekiq.redis do |redis|
         if redis.setnx(lock, 1)
-          Sidekiq.logger.debug "Sidetiq::Clock lock #{lock}"
+          Sidetiq.logger.debug "Sidetiq::Clock lock #{lock}"
 
           redis.pexpire(lock, Sidetiq.config.lock_expire)
           yield redis
           redis.del(lock)
 
-          Sidekiq.logger.debug "Sidetiq::Clock unlock #{lock}"
+          Sidetiq.logger.debug "Sidetiq::Clock unlock #{lock}"
         end
       end
     end
@@ -181,3 +183,4 @@ module Sidetiq
     end
   end
 end
+

data/lib/sidetiq/middleware.rb

@@ -7,7 +7,7 @@ module Sidetiq
     def call(*args)
       # Restart the clock if the thread died.
       if !@clock.ticking?
-        Sidekiq.logger.warn "Sidetiq::Clock thread died. Restarting..."
+        Sidetiq.logger.warn "Sidetiq::Clock thread died. Restarting..."
         @clock.start!
       end
       yield

data/lib/sidetiq/schedulable.rb

@@ -4,30 +4,34 @@ module Sidetiq
   # Examples
   #
   #   class MyWorker
-  #       include Sidekiq::Worker
-  #       include Sidetiq::Schedulable
+  #     include Sidekiq::Worker
+  #     include Sidetiq::Schedulable
   #
-  #       # Daily at midnight
-  #       tiq { daily }
-  #     end
+  #     # Daily at midnight
+  #     tiq { daily }
+  #   end
   module Schedulable
     module ClassMethods
+      # Public: Returns a Float timestamp of the last scheduled run.
       def last_scheduled_occurrence
         get_timestamp "last"
       end
 
+      # Public: Returns a Float timestamp of the next scheduled run.
       def next_scheduled_occurrence
         get_timestamp "next"
       end
 
-      def tiq(&block) # :nodoc:
+      def tiq(options = {}, &block) # :nodoc:
         clock = Sidetiq::Clock.instance
         clock.synchronize do
-          clock.schedule_for(self).instance_eval(&block)
+          schedule = clock.schedule_for(self)
+          schedule.instance_eval(&block)
+          schedule.set_options(options)
         end
       end
 
-    private
+      private
 
       def get_timestamp(key)
         Sidekiq.redis do |redis|
@@ -41,3 +45,4 @@ module Sidetiq
     end
   end
 end
+

data/lib/sidetiq/schedule.rb

@@ -1,23 +1,75 @@
 module Sidetiq
-  # Internal: Recurrence schedules.
-  class Schedule < IceCube::Schedule
-    def method_missing(meth, *args, &block)
+  # Public: Recurrence schedule.
+  class Schedule
+    # :nodoc:
+    attr_reader :last_occurrence
+
+    # Public: Writer for backfilling option.
+    attr_writer :backfill
+
+    # Public: Start time offset from epoch used for calculating run
+    # times in the Sidetiq schedules.
+    START_TIME = Sidetiq.config.utc ? Time.utc(2010, 1, 1) : Time.local(2010, 1, 1)
+
+    def initialize # :nodoc:
+      @schedule = IceCube::Schedule.new(START_TIME)
+    end
+
+    def method_missing(meth, *args, &block) # :nodoc:
       if IceCube::Rule.respond_to?(meth)
         rule = IceCube::Rule.send(meth, *args, &block)
-        add_recurrence_rule(rule)
+        @schedule.add_recurrence_rule(rule)
         rule
+      elsif @schedule.respond_to?(meth)
+        @schedule.send(meth, *args, &block)
       else
         super
       end
     end
 
+    # Public: Checks if a job is due to be scheduled.
+    #
+    # Returns true if a job is due, otherwise false.
     def schedule_next?(time)
-      if @last_scheduled != (no = next_occurrence(time))
-        @last_scheduled = no
+      next_occurrence = @schedule.next_occurrence(time)
+      if @last_scheduled != next_occurrence
+        @last_scheduled = next_occurrence
         return true
       end
       false
     end
+
+    # Public: Schedule to String.
+    #
+    # Examples
+    #
+    #   class MyWorker
+    #     include Sidekiq::Worker
+    #     include Sidetiq::Schedulable
+    #
+    #     tiq { daily }
+    #
+    #     def perform
+    #     end
+    #   end
+    #
+    #   Sidetiq.schedules[MyWorker].to_s
+    #   # => "Daily"
+    #
+    # Returns a String representing the schedule.
+    def to_s
+      @schedule.to_s
+    end
+
+    # Public: Inquirer for backfilling option.
+    def backfill?
+      !!@backfill
+    end
+
+    # Internal: Set schedule options.
+    def set_options(hash)
+      self.backfill = hash[:backfill] if !hash[:backfill].nil?
+    end
   end
 end
 

data/lib/sidetiq/version.rb

@@ -5,7 +5,7 @@ module Sidetiq
     MAJOR = 0
 
     # Public: Sidetiq minor version number.
-    MINOR = 2
+    MINOR = 3
 
     # Public: Sidetiq patch level.
     PATCH = 0

data/test/fixtures/backfill_worker.rb

@@ -0,0 +1,11 @@
+class BackfillWorker
+  include Sidekiq::Worker
+  include Sidetiq::Schedulable
+
+  tiq backfill: true do
+    daily
+  end
+
+  def perform
+  end
+end

data/test/fixtures/last_and_scheduled_ticks_worker.rb

@@ -0,0 +1,5 @@
+class LastAndScheduledTicksWorker
+  def perform(last_tick, scheduled_tick)
+  end
+end
+

data/test/fixtures/last_tick_worker.rb

@@ -0,0 +1,5 @@
+class LastTickWorker
+  def perform(last_tick)
+  end
+end
+

data/test/fixtures/scheduled_worker.rb

@@ -0,0 +1,16 @@
+class ScheduledWorker
+  include Sidekiq::Worker
+  include Sidetiq::Schedulable
+
+  tiq do
+    daily.hour_of_day(1)
+    yearly.month_of_year(2)
+    monthly.day_of_month(3)
+
+    add_exception_rule yearly.month_of_year(:february)
+  end
+
+  def perform
+  end
+end
+

data/test/fixtures/simple_worker.rb

@@ -0,0 +1,8 @@
+class SimpleWorker
+  include Sidekiq::Worker
+  include Sidetiq::Schedulable
+
+  def perform
+  end
+end
+

data/test/fixtures/splat_args_worker.rb

@@ -0,0 +1,5 @@
+class SplatArgsWorker
+  def perform(arg1, *args)
+  end
+end
+

data/test/helper.rb

@@ -13,8 +13,12 @@ require 'sidekiq/testing'
 require 'sidetiq'
 require 'sidetiq/web'
 
-# Keep the test output clean
-Sidekiq.logger = Logger.new(nil)
+# Keep the test output clean.
+Sidetiq.logger = Logger.new(nil)
+
+Dir[File.join(File.dirname(__FILE__), 'fixtures/**/*.rb')].each do |fixture|
+  require fixture
+end
 
 class Sidetiq::TestCase < MiniTest::Unit::TestCase
   def setup
@@ -24,5 +28,16 @@ class Sidetiq::TestCase < MiniTest::Unit::TestCase
   def clock
     @clock ||= Sidetiq::Clock.instance
   end
+
+  # Blatantly stolen from Sidekiq's test suite.
+  def add_retry(worker = 'SimpleWorker', jid = 'bob', at = Time.now.to_f)
+    payload = Sidekiq.dump_json('class' => worker,
+      'args' => [], 'queue' => 'default', 'jid' => jid,
+      'retry_count' => 2, 'failed_at' => Time.now.utc)
+
+    Sidekiq.redis do |conn|
+      conn.zadd('retry', at.to_s, payload)
+    end
+  end
 end
 

data/test/test_clock.rb

@@ -1,11 +1,6 @@
 require_relative 'helper'
 
 class TestClock < Sidetiq::TestCase
-  class FakeWorker
-    def perform
-    end
-  end
-
   def test_delegates_to_instance
     Sidetiq::Clock.instance.expects(:foo).once
     Sidetiq::Clock.foo
@@ -45,13 +40,28 @@ class TestClock < Sidetiq::TestCase
     Sidetiq.config.utc = false
   end
 
+  def test_backfilling
+    BackfillWorker.jobs.clear
+    start = Sidetiq::Schedule::START_TIME
+
+    BackfillWorker.stubs(:last_scheduled_occurrence).returns(start.to_f)
+    clock.stubs(:gettime).returns(start)
+    clock.tick
+
+    BackfillWorker.jobs.clear
+
+    clock.stubs(:gettime).returns(start + 86400 * 10 + 1)
+    clock.tick
+    assert_equal 10, BackfillWorker.jobs.length
+  end
+
   def test_enqueues_jobs_by_schedule
-    schedule = Sidetiq::Schedule.new(Sidetiq::Clock::START_TIME)
+    schedule = Sidetiq::Schedule.new
     schedule.daily
 
-    clock.stubs(:schedules).returns(FakeWorker => schedule)
+    clock.stubs(:schedules).returns(SimpleWorker => schedule)
 
-    FakeWorker.expects(:perform_at).times(10)
+    SimpleWorker.expects(:perform_at).times(10)
 
     10.times do |i|
       clock.stubs(:gettime).returns(Time.local(2011, 1, i + 1, 1))
@@ -64,13 +74,8 @@ class TestClock < Sidetiq::TestCase
     clock.tick
   end
 
-  class LastTickWorker
-    def perform last_tick
-    end
-  end
-
   def test_enqueues_jobs_with_default_last_tick_arg_on_first_run
-    schedule = Sidetiq::Schedule.new(Sidetiq::Clock::START_TIME)
+    schedule = Sidetiq::Schedule.new
     schedule.hourly
 
     time = Time.local(2011, 1, 1, 1, 30)
@@ -82,19 +87,15 @@ class TestClock < Sidetiq::TestCase
     expected_second_tick = expected_first_tick + 3600
 
     LastTickWorker.expects(:perform_at).with(expected_first_tick, -1).once
-    LastTickWorker.expects(:perform_at).with(expected_second_tick, expected_first_tick.to_f).once
+    LastTickWorker.expects(:perform_at).with(expected_second_tick,
+      expected_first_tick.to_f).once
 
     clock.tick
     clock.tick
   end
 
-  class LastAndScheduledTicksWorker
-    def perform last_tick, scheduled_tick
-    end
-  end
-
   def test_enqueues_jobs_with_last_run_timestamp_and_next_run_timestamp
-    schedule = Sidetiq::Schedule.new(Sidetiq::Clock::START_TIME)
+    schedule = Sidetiq::Schedule.new
     schedule.hourly
 
     time = Time.local(2011, 1, 1, 1, 30)
@@ -105,20 +106,20 @@ class TestClock < Sidetiq::TestCase
     expected_first_tick = time + 1800
     expected_second_tick = expected_first_tick + 3600
 
-    LastAndScheduledTicksWorker.expects(:perform_at).with(expected_first_tick, -1, expected_first_tick.to_f).once
-    clock.tick
+    LastAndScheduledTicksWorker.expects(:perform_at)
+      .with(expected_first_tick, -1, expected_first_tick.to_f).once
 
-    LastAndScheduledTicksWorker.expects(:perform_at).with(expected_second_tick, expected_first_tick.to_f, expected_second_tick.to_f).once
     clock.tick
-  end
 
-  class SplatArgsWorker
-    def perform arg1, *args
-    end
+    LastAndScheduledTicksWorker.expects(:perform_at)
+      .with(expected_second_tick, expected_first_tick.to_f,
+      expected_second_tick.to_f).once
+
+    clock.tick
   end
 
   def test_enqueues_jobs_correctly_for_splat_args_perform_methods
-    schedule = Sidetiq::Schedule.new(Sidetiq::Clock::START_TIME)
+    schedule = Sidetiq::Schedule.new
     schedule.hourly
 
     time = Time.local(2011, 1, 1, 1, 30)
@@ -128,7 +129,8 @@ class TestClock < Sidetiq::TestCase
 
     expected_first_tick = time + 1800
 
-    SplatArgsWorker.expects(:perform_at).with(expected_first_tick, -1, expected_first_tick.to_f).once
+    SplatArgsWorker.expects(:perform_at)
+      .with(expected_first_tick, -1, expected_first_tick.to_f).once
     clock.tick
   end
 end

data/test/test_schedule.rb

@@ -1,10 +1,6 @@
 require_relative 'helper'
 
 class TestSchedule < Sidetiq::TestCase
-  def test_super
-    assert_equal IceCube::Schedule, Sidetiq::Schedule.superclass
-  end
-
   def test_method_missing
     sched = Sidetiq::Schedule.new
     sched.daily
@@ -21,5 +17,22 @@ class TestSchedule < Sidetiq::TestCase
     assert sched.schedule_next?(Time.now + (2 * 24 * 60 * 60))
     refute sched.schedule_next?(Time.now + (2 * 24 * 60 * 60))
   end
+
+  def test_backfill
+    sched = Sidetiq::Schedule.new
+    refute sched.backfill?
+    sched.backfill = true
+    assert sched.backfill?
+  end
+
+  def test_set_options
+    sched = Sidetiq::Schedule.new
+
+    sched.set_options(backfill: true)
+    assert sched.backfill?
+
+    sched.set_options(backfill: false)
+    refute sched.backfill?
+  end
 end
 

data/test/test_sidetiq.rb

@@ -0,0 +1,125 @@
+require_relative 'helper'
+
+class TestSidetiq < Sidetiq::TestCase
+  def test_schedules
+    schedules = Sidetiq.schedules
+
+    assert_equal 2, schedules.length
+
+    assert_includes schedules.keys, ScheduledWorker
+    assert_includes schedules.keys, BackfillWorker
+
+    assert_kind_of Sidetiq::Schedule, schedules[ScheduledWorker]
+    assert_kind_of Sidetiq::Schedule, schedules[BackfillWorker]
+  end
+
+  def test_workers
+    workers = Sidetiq.workers
+
+    assert_includes workers, ScheduledWorker
+    assert_includes workers, BackfillWorker
+    assert_equal 2, workers.length
+  end
+
+  def test_scheduled
+    SimpleWorker.perform_at(Time.local(2011, 1, 1, 1))
+    SimpleWorker.client_push_old(SimpleWorker.jobs.first)
+
+    scheduled = Sidetiq.scheduled
+
+    assert_kind_of Array, scheduled
+    assert_kind_of Sidekiq::SortedEntry, scheduled.first
+    assert_equal 1, scheduled.length
+  end
+
+  def test_scheduled_on_empty_set
+    assert_equal 0, Sidetiq.scheduled.length
+  end
+
+  def test_scheduled_given_arguments
+    SimpleWorker.perform_at(Time.local(2011, 1, 1, 1))
+    SimpleWorker.client_push_old(SimpleWorker.jobs.first)
+
+    assert_equal 1, Sidetiq.scheduled(SimpleWorker).length
+    assert_equal 0, Sidetiq.scheduled(ScheduledWorker).length
+
+    assert_equal 1, Sidetiq.scheduled("SimpleWorker").length
+    assert_equal 0, Sidetiq.scheduled("ScheduledWorker").length
+  end
+
+  def test_scheduled_yields_each_job
+    SimpleWorker.perform_at(Time.local(2011, 1, 1, 1))
+    SimpleWorker.client_push_old(SimpleWorker.jobs.first)
+
+    ScheduledWorker.perform_at(Time.local(2011, 1, 1, 1))
+    ScheduledWorker.client_push_old(ScheduledWorker.jobs.first)
+
+    jobs = []
+    Sidetiq.scheduled { |job| jobs << job }
+    assert_equal 2, jobs.length
+
+    jobs = []
+    Sidetiq.scheduled(SimpleWorker) { |job| jobs << job }
+    assert_equal 1, jobs.length
+
+    jobs = []
+    Sidetiq.scheduled("ScheduledWorker") { |job| jobs << job }
+    assert_equal 1, jobs.length
+  end
+
+  def test_scheduled_with_invalid_class
+    assert_raises(NameError) do
+      Sidetiq.scheduled("Foobar")
+    end
+  end
+
+  def test_retries
+    add_retry('SimpleWorker', 'foo')
+    add_retry('ScheduledWorker', 'bar')
+
+    retries = Sidetiq.retries
+
+    assert_kind_of Array, retries
+    assert_kind_of Sidekiq::SortedEntry, retries[0]
+    assert_kind_of Sidekiq::SortedEntry, retries[1]
+    assert_equal 2, retries.length
+  end
+
+  def test_retries_on_empty_set
+    assert_equal 0, Sidetiq.retries.length
+  end
+
+  def test_retries_given_arguments
+    add_retry('SimpleWorker', 'foo')
+
+    assert_equal 1, Sidetiq.retries(SimpleWorker).length
+    assert_equal 0, Sidetiq.retries(ScheduledWorker).length
+
+    assert_equal 1, Sidetiq.retries("SimpleWorker").length
+    assert_equal 0, Sidetiq.retries("ScheduledWorker").length
+  end
+
+  def test_retries_yields_each_job
+    add_retry('SimpleWorker', 'foo')
+    add_retry('ScheduledWorker', 'foo')
+
+    jobs = []
+    Sidetiq.retries { |job| jobs << job }
+    assert_equal 2, jobs.length
+
+    jobs = []
+    Sidetiq.retries(SimpleWorker) { |job| jobs << job }
+    assert_equal 1, jobs.length
+
+    jobs = []
+    Sidetiq.retries("ScheduledWorker") { |job| jobs << job }
+    assert_equal 1, jobs.length
+  end
+
+  def test_retries_with_invalid_class
+    assert_raises(NameError) do
+      Sidetiq.retries("Foobar")
+    end
+  end
+end
+

data/test/test_web.rb

@@ -3,19 +3,6 @@ require_relative 'helper'
 class TestWeb < Sidetiq::TestCase
   include Rack::Test::Methods
 
-  class Worker
-    include Sidekiq::Worker
-    include Sidetiq::Schedulable
-
-    tiq do
-      daily(1)
-      yearly(2)
-      monthly(3)
-
-      add_exception_rule yearly.month_of_year(:february)
-    end
-  end
-
   def app
     Sidekiq::Web
   end
@@ -26,7 +13,7 @@ class TestWeb < Sidetiq::TestCase
 
   def setup
     super
-    Worker.jobs.clear
+    ScheduledWorker.jobs.clear
   end
 
   def test_home_tab
@@ -47,9 +34,9 @@ class TestWeb < Sidetiq::TestCase
   end
 
   def test_details_page
-    get "/sidetiq/#{Worker.name}"
+    get "/sidetiq/ScheduledWorker"
     assert_equal 200, last_response.status
-    schedule = clock.schedules[Worker]
+    schedule = clock.schedules[ScheduledWorker]
 
     schedule.recurrence_rules.each do |rule|
       assert_match /#{rule.to_s}/, last_response.body
@@ -65,10 +52,10 @@ class TestWeb < Sidetiq::TestCase
   end
 
   def test_trigger
-    post "/sidetiq/#{Worker.name}/trigger"
+    post "/sidetiq/ScheduledWorker/trigger"
     assert_equal 302, last_response.status
     assert_equal "http://#{host}/sidetiq", last_response.location
-    assert_equal 1, Worker.jobs.size
+    assert_equal 1, ScheduledWorker.jobs.size
   end
 end
 

data/test/test_worker.rb

@@ -22,4 +22,8 @@ class TestWorker < Sidetiq::TestCase
     assert FakeWorker.last_scheduled_occurrence == last_run
     assert FakeWorker.next_scheduled_occurrence == next_run
   end
+
+  def test_options
+    assert Sidetiq.schedules[BackfillWorker].backfill?
+  end
 end

metadata

@@ -2,14 +2,14 @@
 name: sidetiq
 version: !ruby/object:Gem::Version
   prerelease: 
-  version: 0.2.0
+  version: 0.3.0
 platform: ruby
 authors:
 - Tobias Svensson
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-03-08 00:00:00.000000000 Z
+date: 2013-03-11 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   version_requirements: !ruby/object:Gem::Requirement
@@ -73,12 +73,19 @@ files:
 - lib/sidetiq/views/sidetiq_details.slim
 - lib/sidetiq/web.rb
 - sidetiq.gemspec
+- test/fixtures/backfill_worker.rb
+- test/fixtures/last_and_scheduled_ticks_worker.rb
+- test/fixtures/last_tick_worker.rb
+- test/fixtures/scheduled_worker.rb
+- test/fixtures/simple_worker.rb
+- test/fixtures/splat_args_worker.rb
 - test/helper.rb
 - test/test_clock.rb
 - test/test_config.rb
 - test/test_errors.rb
 - test/test_middleware.rb
 - test/test_schedule.rb
+- test/test_sidetiq.rb
 - test/test_version.rb
 - test/test_web.rb
 - test/test_worker.rb
@@ -108,12 +115,19 @@ signing_key:
 specification_version: 3
 summary: Recurring jobs for Sidekiq
 test_files:
+- test/fixtures/backfill_worker.rb
+- test/fixtures/last_and_scheduled_ticks_worker.rb
+- test/fixtures/last_tick_worker.rb
+- test/fixtures/scheduled_worker.rb
+- test/fixtures/simple_worker.rb
+- test/fixtures/splat_args_worker.rb
 - test/helper.rb
 - test/test_clock.rb
 - test/test_config.rb
 - test/test_errors.rb
 - test/test_middleware.rb
 - test/test_schedule.rb
+- test/test_sidetiq.rb
 - test/test_version.rb
 - test/test_web.rb
 - test/test_worker.rb