ts-resque-delta 0.0.1

data/LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2009 Pat Allan
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.markdown

@@ -0,0 +1,57 @@
+Delayed Deltas for Thinking Sphinx (with Resque)
+================================================
+
+**This code is HEAVILY borrowed from [ts-delayed-delta](http://github.com/freelancing-god/ts-delayed-delta).**
+
+Installation
+------------
+
+You'll need Thinking Sphinx 1.3.0 or later, and Resque as well. The latter is flagged as a dependency.
+
+    gem install ts-resque-delta
+
+In your `Gemfile` file, with the rest of your gem dependencies:
+
+    gem 'ts-resque-delta', '0.0.1', :require => 'thinking_sphinx/deltas/resque_delta'
+
+And add the following line to the bottom of your `Rakefile`:
+
+    require 'thinking_sphinx/deltas/resque_delta/tasks'
+
+For the indexes you want to use this delta approach, make sure you set that up in their `define_index` blocks.
+
+    define_index do
+      # ...
+      set_property :delta => ThinkingSphinx::Deltas::ResqueDelta
+    end
+
+If you've never used delta indexes before, you'll want to add the boolean column named delta to each model that is using the approach.
+
+    def self.up
+      add_column :articles, :delta, :boolean, :default => true, :null => false
+    end
+
+Usage
+-----
+
+Once you've got it all set up, all you need to do is make sure that the Resque worker is running. You can do this either by running Resque's workers and specifying the `:ts_delta` queue, or Thinking Sphinx's custom rake task:
+
+    rake thinking_sphinx:resque_delta
+
+There's also a short name for the same task, to save your fingers some effort:
+
+    rake ts:rd
+
+Original Contributors (for ts-delayed-delta)
+--------------------------------------------
+
+* [Pat Allan](http://github.com/freelancing-god)
+* [Ryan Schlesinger](http://github.com/ryansch) (Allowing installs as a plugin)
+* [Maximilian Schulz](http://max.jungeelite.de) (Ensuring compatibility with Bundler)
+* [Edgars Beigarts](http://github.com/ebeigarts) (Adding intelligent description for tasks)
+* [Alexander Simonov](http://simonov.me/) (Explicit table definition)
+
+Copyright
+---------
+
+Copyright (c) 2010 Aaron Gibralter, and released under an MIT Licence.

data/Rakefile

@@ -0,0 +1,6 @@
+require 'rubygems'
+
+require 'tasks/distribution'
+require 'tasks/testing'
+
+task :default => :spec

data/config/redis-cucumber.conf

@@ -0,0 +1,13 @@
+daemonize yes
+pidfile tmp/redis-cucumber.pid
+port 6398
+timeout 300
+save 900 1
+save 300 10
+save 60 10000
+dbfilename tmp/redis-cucumber-dump.rdb
+dir .
+loglevel debug
+logfile stdout
+databases 16
+glueoutputbuf yes

data/features/resque_deltas.feature

@@ -0,0 +1,62 @@
+Feature: Resque Delta Indexing
+  In order to have delta indexing on frequently-updated sites
+  Developers
+  Should be able to use Resque to handle delta indexes to lower system load
+
+  Background:
+    Given Sphinx is running
+    And I am searching on delayed betas
+    And I have data and it has been indexed
+
+  Scenario: Delta Index should not fire automatically
+    When I search for one
+    Then I should get 1 result
+
+    When I change the name of delayed beta one to eleven
+    And I wait for Sphinx to catch up
+    And I search for one
+    Then I should get 1 result
+
+    When I search for eleven
+    Then I should get 0 results
+
+  Scenario: Delta Index should fire when jobs are run
+    When I search for one
+    Then I should get 1 result
+
+    When I change the name of delayed beta two to twelve
+    And I wait for Sphinx to catch up
+    And I search for twelve
+    Then I should get 0 results
+
+    When I run the delayed jobs
+    And I wait for Sphinx to catch up
+    And I search for twelve
+    Then I should get 1 result
+
+    When I search for two
+    Then I should get 0 results
+
+  Scenario: ensuring that duplicate jobs are deleted
+    When I change the name of delayed beta two to fifty
+    And I change the name of delayed beta five to twelve
+    And I change the name of delayed beta one to fifteen
+    And I change the name of delayed beta six to twenty
+    And I run one delayed job
+    Then there should be no more DeltaJobs on the Resque queue
+
+    When I run the delayed jobs
+    And I wait for Sphinx to catch up
+    And I search for fifty
+    Then I should get 1 result
+
+    When I search for two
+    Then I should get 0 results
+
+  Scenario: canceling jobs
+    When I change the name of delayed beta two to fifty
+    And I cancel the jobs
+    And I run the delayed jobs
+    And I wait for Sphinx to catch up
+    And I search for fifty
+    Then I should get 0 results

data/features/step_definitions/common_steps.rb

@@ -0,0 +1,63 @@
+Before do
+  $queries_executed = []
+  ThinkingSphinx::Deltas::ResqueDelta.cancel_thinking_sphinx_jobs
+  @model      = nil
+  @method     = :search
+  @query      = ""
+  @conditions = {}
+  @with       = {}
+  @without    = {}
+  @with_all   = {}
+  @options    = {}
+  @results    = nil
+end
+
+Given "Sphinx is running" do
+  ThinkingSphinx::Configuration.instance.controller.should be_running
+end
+
+Given /^I am searching on (.+)$/ do |model|
+  @model = model.gsub(/\s/, '_').singularize.camelize.constantize
+end
+
+Given "I have data and it has been indexed" do
+  DelayedBeta.create(:name => "one")
+  DelayedBeta.create(:name => "two")
+  DelayedBeta.create(:name => "three")
+  DelayedBeta.create(:name => "four")
+  DelayedBeta.create(:name => "five")
+  DelayedBeta.create(:name => "six")
+  DelayedBeta.create(:name => "seven")
+  DelayedBeta.create(:name => "eight")
+  DelayedBeta.create(:name => "nine")
+  DelayedBeta.create(:name => "ten")
+  ThinkingSphinx::Deltas::ResqueDelta.cancel_thinking_sphinx_jobs
+  ThinkingSphinx::Configuration.instance.controller.index
+  sleep(1.5)
+end
+
+When "I wait for Sphinx to catch up" do
+  sleep(0.5)
+end
+
+When /^I search for (\w+)$/ do |query|
+  @results = nil
+  @query = query
+end
+
+Then /^I should get (\d+) results?$/ do |count|
+  results.length.should == count.to_i
+end
+
+def results
+  @results ||= (@model || ThinkingSphinx).send(
+    @method,
+    @query,
+    @options.merge(
+      :conditions => @conditions,
+      :with       => @with,
+      :without    => @without,
+      :with_all   => @with_all
+    )
+  )
+end

data/features/step_definitions/resque_delta_steps.rb

@@ -0,0 +1,33 @@
+When /^I run the delayed jobs$/ do
+  unless @resque_worker
+    @resque_worker = Resque::Worker.new("ts_delta")
+    @resque_worker.register_worker
+  end
+  while job = @resque_worker.reserve
+    @resque_worker.perform(job)
+  end
+end
+
+When /^I run one delayed job$/ do
+  unless @resque_worker
+    @resque_worker = Resque::Worker.new("ts_delta")
+    @resque_worker.register_worker
+  end
+  job = @resque_worker.reserve
+  @resque_worker.perform(job)
+end
+
+When /^I cancel the jobs$/ do
+  ThinkingSphinx::Deltas::ResqueDelta.cancel_thinking_sphinx_jobs
+end
+
+When /^I change the name of delayed beta (\w+) to (\w+)$/ do |current, replacement|
+  DelayedBeta.find_by_name(current).update_attributes(:name => replacement)
+end
+
+Then /^there should be no more DeltaJobs on the Resque queue$/ do
+  job_classes = Resque.redis.lrange("queue:ts_delta", 0, -1).collect do |j|
+    Resque.decode(j)["class"]
+  end
+  job_classes.should_not include("ThinkingSphinx::Deltas::ResqueDelta::DeltaJob")
+end

data/features/support/env.rb

@@ -0,0 +1,35 @@
+require 'rubygems'
+require 'cucumber'
+require 'spec/expectations'
+require 'fileutils'
+require 'active_record'
+
+PROJECT_ROOT = File.expand_path(File.join(File.dirname(__FILE__), '..', '..'))
+
+$:.unshift(File.join(PROJECT_ROOT, 'lib'))
+$:.unshift(File.dirname(__FILE__))
+
+require 'cucumber/thinking_sphinx/internal_world'
+
+Time.zone_default = Time.__send__(:get_zone, 'Melbourne')
+ActiveRecord::Base.time_zone_aware_attributes = true
+ActiveRecord::Base.default_timezone = :utc
+
+world = Cucumber::ThinkingSphinx::InternalWorld.new
+world.configure_database
+
+require 'thinking_sphinx'
+require 'thinking_sphinx/deltas/resque_delta'
+
+world.setup
+
+require 'redis_test_setup'
+RedisTestSetup.start_redis!(PROJECT_ROOT, :cucumber)
+Resque.redis = '127.0.0.1:6398'
+Before do
+  Resque.redis.flushall
+end
+
+require 'database_cleaner'
+require 'database_cleaner/cucumber'
+DatabaseCleaner.strategy = :truncation

data/features/support/redis_test_setup.rb

@@ -0,0 +1,23 @@
+module RedisTestSetup
+
+  def self.start_redis!(rails_root, env)
+    dir_temp = File.expand_path(File.join(rails_root, 'tmp'))
+    dir_conf = File.expand_path(File.join(rails_root, 'config'))
+    cwd = Dir.getwd
+    Dir.chdir(rails_root)
+    self.cleanup(dir_temp, env)
+    raise "unable to launch redis-server" unless system("redis-server #{dir_conf}/redis-#{env}.conf")
+    Dir.chdir(cwd)
+    Kernel.at_exit do
+      if (pid = `cat #{dir_temp}/redis-#{env}.pid`.strip) =~ /^\d+$/
+        self.cleanup(dir_temp, env)
+        Process.kill("KILL", pid.to_i)
+      end
+    end
+  end
+
+  def self.cleanup(dir_temp, env)
+    `rm -f #{dir_temp}/redis-#{env}-dump.rdb`
+    `rm -f #{dir_temp}/redis-#{env}.pid`
+  end
+end

data/features/thinking_sphinx/database.example.yml

@@ -0,0 +1,3 @@
+username: root
+host: localhost
+password: 

data/features/thinking_sphinx/database.yml

@@ -0,0 +1,5 @@
+username: root
+password: BdUKNL5Cimo9yAVzITEv
+host: localhost
+pool: 1
+min_messages: warning

data/features/thinking_sphinx/db/migrations/create_delayed_betas.rb

@@ -0,0 +1,17 @@
+ActiveRecord::Base.connection.create_table :delayed_betas, :force => true do |t|
+  t.column  :name, :string,  :null => false
+  t.column :delta, :boolean, :null => false, :default => false
+end
+
+ActiveRecord::Base.connection.create_table :delayed_jobs, :force => true do |t|
+  t.column :priority, :integer, :default => 0
+  t.column :attempts, :integer, :default => 0
+  t.column :handler, :text
+  t.column :last_error, :string
+  t.column :run_at, :datetime
+  t.column :locked_at, :datetime
+  t.column :failed_at, :datetime
+  t.column :locked_by, :string
+  t.column :created_at, :datetime
+  t.column :updated_at, :datetime
+end

data/features/thinking_sphinx/models/delayed_beta.rb

@@ -0,0 +1,6 @@
+class DelayedBeta < ActiveRecord::Base
+  define_index do
+    indexes :name, :sortable => true
+    set_property :delta => ThinkingSphinx::Deltas::ResqueDelta
+  end
+end

data/lib/thinking_sphinx/deltas/resque_delta.rb

@@ -0,0 +1,74 @@
+require 'resque'
+require 'thinking_sphinx'
+
+# Delayed Deltas for Thinking Sphinx, using Resque.
+#
+# This documentation is aimed at those reading the code. If you're looking for
+# a guide to Thinking Sphinx and/or deltas, I recommend you start with the
+# Thinking Sphinx site instead - or the README for this library at the very
+# least.
+#
+# @author Patrick Allan
+# @see http://ts.freelancing-gods.com Thinking Sphinx
+#
+class ThinkingSphinx::Deltas::ResqueDelta < ThinkingSphinx::Deltas::DefaultDelta
+
+  # LTRIM + LPOP deletes all items from the Resque queue without loading it
+  # into client memory (unlike Resque.dequeue).
+  def self.cancel_thinking_sphinx_jobs
+    [
+      ThinkingSphinx::Deltas::ResqueDelta::DeltaJob,
+      ThinkingSphinx::Deltas::ResqueDelta::FlagAsDeletedJob
+    ].collect { |c| c.instance_variable_get(:@queue) }.uniq.each do |q|
+      Resque.redis.ltrim("queue:#{q}", 0, 0)
+      Resque.redis.lpop("queue:#{q}")
+    end
+  end
+
+  # Adds a job to the queue for processing the given model's delta index. A job
+  # for hiding the instance in the core index is also created, if an instance is
+  # provided.
+  #
+  # Neither job will be queued if updates or deltas are disabled, or if the 
+  # instance (when given) is not toggled to be in the delta index. The first two
+  # options are controlled via ThinkingSphinx.updates_enabled? and
+  # ThinkingSphinx.deltas_enabled?.
+  #
+  # @param [Class] model the ActiveRecord model to index.
+  # @param [ActiveRecord::Base] instance the instance of the given model that
+  #   has changed. Optional.
+  # @return [Boolean] true
+  #
+  def index(model, instance = nil)
+    return true if skip?(instance)
+    Resque.enqueue(
+      ThinkingSphinx::Deltas::ResqueDelta::DeltaJob,
+      model.delta_index_names
+    )
+    if instance
+      Resque.enqueue(
+        ThinkingSphinx::Deltas::ResqueDelta::FlagAsDeletedJob,
+        model.core_index_names,
+        instance.sphinx_document_id
+      )
+    end
+    true
+  end
+
+  private
+
+  # Checks whether jobs should be enqueued. Only true if updates and deltas are
+  # enabled, and the instance (if there is one) is toggled.
+  #
+  # @param [ActiveRecord::Base, NilClass] instance
+  # @return [Boolean]
+  #
+  def skip?(instance)
+    !ThinkingSphinx.updates_enabled? ||
+    !ThinkingSphinx.deltas_enabled?  ||
+    (instance && !toggled(instance))
+  end
+end
+
+require 'thinking_sphinx/deltas/resque_delta/delta_job'
+require 'thinking_sphinx/deltas/resque_delta/flag_as_deleted_job'

data/lib/thinking_sphinx/deltas/resque_delta/delta_job.rb

@@ -0,0 +1,54 @@
+require 'resque-lock-timeout'
+
+# A simple job class that processes a given index.
+#
+class ThinkingSphinx::Deltas::ResqueDelta::DeltaJob
+
+  extend Resque::Plugins::LockTimeout
+  @queue = :ts_delta
+  @lock_timeout = 240
+
+  # Runs Sphinx's indexer tool to process the index. Currently assumes Sphinx
+  # is running.
+  #
+  # @param [String] index the name of the Sphinx index
+  #
+  def self.perform(indexes)
+    config = ThinkingSphinx::Configuration.instance
+    output = `#{config.bin_path}#{config.indexer_binary_name} --config #{config.config_file} --rotate #{indexes.join(' ')}`
+    puts output unless ThinkingSphinx.suppress_delta_output?
+  end
+
+  # Try again later if lock is in use.
+  def self.lock_failed(*args)
+    Resque.enqueue(self, *args)
+  end
+
+  # Run only one DeltaJob at a time regardless of indexes.
+  def self.identifier(*args)
+    nil
+  end
+
+  # This allows us to have a concurrency safe version of ts-delayed-delta's
+  # duplicates_exist:
+  #
+  # http://github.com/freelancing-god/ts-delayed-delta/blob/master/lib/thinkin
+  # g_sphinx/deltas/delayed_delta/job.rb#L47
+  #
+  # The name of this method ensures that it runs within around_perform_lock.
+  #
+  # We've leveraged resque-lock-timeout to ensure that only one DeltaJob is
+  # running at a time. Now, this around filter essentially ensures that only
+  # one DeltaJob of each index type can sit at the queue at once. If the queue
+  # has more than one, lrem will clear the rest off.
+  #
+  def self.around_perform_lock1(*args)
+    # Remove all other instances of this job (with the same args) from the
+    # queue. Uses LREM (http://code.google.com/p/redis/wiki/LremCommand) which
+    # takes the form: "LREM key count value" and if count == 0 removes all
+    # instances of value from the list.
+    redis_job_value = Resque.encode(:class => self.to_s, :args => args)
+    Resque.redis.lrem("queue:#{@queue}", 0, redis_job_value)
+    yield
+  end
+end

data/lib/thinking_sphinx/deltas/resque_delta/flag_as_deleted_job.rb

@@ -0,0 +1,30 @@
+# A simple job for flagging a specified Sphinx document in a given index as
+# 'deleted'.
+#  
+class ThinkingSphinx::Deltas::ResqueDelta::FlagAsDeletedJob
+
+  @queue = :ts_delta
+
+  # Takes an index name and document id. Please note that the document id is
+  # Sphinx's unique identifier, and will almost certainly not be the model
+  # instance's primary key value. Updates the sphinx_deleted attribute for the
+  # given document, setting the value to 1 (true). This is not a special
+  # attribute in Sphinx, but is used by Thinking Sphinx to ignore deleted
+  # values between full re-indexing. It's particularly useful in this
+  # situation to avoid old values in the core index and just use the new
+  # values in the delta index as a reference point.
+  #
+  # @param [Array] indexes An array of index names
+  # @param [Integer] document_id The document id
+  #
+  # @return [Boolean] true
+  #
+  def self.perform(indexes, document_id)
+    config = ThinkingSphinx::Configuration.instance
+    indexes.each do |index|
+      if ThinkingSphinx.sphinx_running? && ThinkingSphinx.search_for_id(document_id, index)
+        config.client.update(index, ['sphinx_deleted'], {document_id => [1]})
+      end
+    end
+  end
+end

data/lib/thinking_sphinx/deltas/resque_delta/tasks.rb

@@ -0,0 +1,22 @@
+namespace :thinking_sphinx do
+  task :index do
+    ThinkingSphinx::Deltas::ResqueDelta.cancel_thinking_sphinx_jobs
+  end
+
+  desc "Process stored delta index requests"
+  task :resque_delta => :app_env do
+    raise "TODO... for now, please just run the workers on your own and make sure to work the :ts_delta queue."
+  #   require 'delayed/worker'
+  #   require 'thinking_sphinx/deltas/resque_delta'
+  #   
+  #   Delayed::Worker.new(
+  #     :min_priority => ENV['MIN_PRIORITY'],
+  #     :max_priority => ENV['MAX_PRIORITY']
+  #   ).start
+  end
+end
+
+namespace :ts do
+  desc "Process stored delta index requests"
+  task :rd => "thinking_sphinx:resque_delta"
+end

data/spec/spec.opts

@@ -0,0 +1 @@
+--color

data/spec/spec_helper.rb

@@ -0,0 +1,8 @@
+$LOAD_PATH.unshift File.join(File.dirname(__FILE__), '..', 'lib')
+
+require 'rubygems'
+require 'spec'
+require 'spec/autorun'
+
+require 'thinking_sphinx'
+require 'thinking_sphinx/deltas/resque_delta'

data/spec/thinking_sphinx/deltas/resque_delta/delta_job_spec.rb

@@ -0,0 +1,39 @@
+require 'spec_helper'
+
+describe ThinkingSphinx::Deltas::ResqueDelta::DeltaJob do
+  describe '.perform' do
+    before :each do
+      ThinkingSphinx.suppress_delta_output = false
+      ThinkingSphinx::Deltas::ResqueDelta::DeltaJob.stub(:` => true)
+      ThinkingSphinx::Deltas::ResqueDelta::DeltaJob.stub(:puts => nil)
+    end
+
+    it "should output the delta indexing by default" do
+      ThinkingSphinx::Deltas::ResqueDelta::DeltaJob.should_receive(:puts)
+      ThinkingSphinx::Deltas::ResqueDelta::DeltaJob.perform(['foo_core'])
+    end
+
+    it "should not output the delta indexing if requested" do
+      ThinkingSphinx.suppress_delta_output = true
+      ThinkingSphinx::Deltas::ResqueDelta::DeltaJob.should_not_receive(:puts)
+      ThinkingSphinx::Deltas::ResqueDelta::DeltaJob.perform(['foo_core'])
+    end
+
+    it "should process just the requested indexes" do
+      ThinkingSphinx::Deltas::ResqueDelta::DeltaJob.should_receive(:`) do |command|
+        command.should match(/foo_core/)
+        command.should_not match(/--all/)
+      end
+      ThinkingSphinx::Deltas::ResqueDelta::DeltaJob.perform(['foo_core'])
+    end
+
+    context 'multiple indexes' do
+      it "should process all requested indexes" do
+        ThinkingSphinx::Deltas::ResqueDelta::DeltaJob.should_receive(:`) do |command|
+          command.should match(/foo_core bar_core/)
+        end
+        ThinkingSphinx::Deltas::ResqueDelta::DeltaJob.perform(['foo_core', 'bar_core'])
+      end
+    end
+  end
+end

data/spec/thinking_sphinx/deltas/resque_delta/flag_as_deleted_job_spec.rb

@@ -0,0 +1,66 @@
+require 'spec_helper'
+
+describe ThinkingSphinx::Deltas::ResqueDelta::FlagAsDeletedJob do
+  describe '.perform' do
+    before :each do
+      ThinkingSphinx.updates_enabled = true
+      @client = stub('client', :update => true)
+      ThinkingSphinx::Configuration.instance.stub!(:client => @client)
+      ThinkingSphinx.stub!(:search_for_id => true)
+      ThinkingSphinx.stub!(:sphinx_running? => true)
+    end
+
+    it "should not update if Sphinx isn't running" do
+      ThinkingSphinx.stub!(:sphinx_running? => false)
+      @client.should_not_receive(:update)
+      ThinkingSphinx::Deltas::ResqueDelta::FlagAsDeletedJob.perform(['foo_core'], 12)
+    end
+
+    it "should not update if the document isn't in the index" do
+      ThinkingSphinx.stub!(:search_for_id => false)
+      @client.should_not_receive(:update)
+      ThinkingSphinx::Deltas::ResqueDelta::FlagAsDeletedJob.perform(['foo_core'], 12)
+    end
+
+    it "should update the specified index" do
+      @client.should_receive(:update) do |index, attributes, values|
+        index.should == 'foo_core'
+      end
+      ThinkingSphinx::Deltas::ResqueDelta::FlagAsDeletedJob.perform(['foo_core'], 12)
+    end
+
+    it "should update all specified indexes" do
+      @client.should_receive(:update).with('foo_core', anything, anything)
+      @client.should_receive(:update).with('bar_core', anything, anything)
+      ThinkingSphinx::Deltas::ResqueDelta::FlagAsDeletedJob.perform(['foo_core', 'bar_core'], 12)
+    end
+
+    it "should update the sphinx_deleted attribute" do
+      @client.should_receive(:update) do |index, attributes, values|
+        attributes.should == ['sphinx_deleted']
+      end
+      ThinkingSphinx::Deltas::ResqueDelta::FlagAsDeletedJob.perform(['foo_core'], 12)
+    end
+
+    it "should set sphinx_deleted for the given document to true" do
+      @client.should_receive(:update) do |index, attributes, values|
+        values[12].should == [1]
+      end
+      ThinkingSphinx::Deltas::ResqueDelta::FlagAsDeletedJob.perform(['foo_core'], 12)
+    end
+
+    it "should check for the existence of the document in the specified index" do
+      ThinkingSphinx.should_receive(:search_for_id) do |id, index|
+        index.should == 'foo_core'
+      end
+      ThinkingSphinx::Deltas::ResqueDelta::FlagAsDeletedJob.perform(['foo_core'], 12)
+    end
+
+    it "should check for the existence of the given document id" do
+      ThinkingSphinx.should_receive(:search_for_id) do |id, index|
+        id.should == 12
+      end
+      ThinkingSphinx::Deltas::ResqueDelta::FlagAsDeletedJob.perform(['foo_core'], 12)
+    end
+  end
+end

data/spec/thinking_sphinx/deltas/resque_delta_spec.rb

@@ -0,0 +1,93 @@
+require 'spec_helper'
+
+describe ThinkingSphinx::Deltas::ResqueDelta do
+  describe '#index' do
+    before :each do
+      ThinkingSphinx.updates_enabled = true
+      ThinkingSphinx.deltas_enabled  = true
+
+      Resque.stub!(:enqueue => true)
+
+      @delayed_delta = ThinkingSphinx::Deltas::ResqueDelta.new(
+        stub('instance'), {}
+      )
+      @delayed_delta.stub!(:toggled => true)
+
+      @model = stub('foo')
+      @model.stub!(:name => 'foo')
+      @model.stub!(:source_of_sphinx_index => @model)
+      @model.stub!(:core_index_names  => ['foo_core'])
+      @model.stub!(:delta_index_names => ['foo_delta'])
+
+      @instance = stub('instance')
+      @instance.stub!(:sphinx_document_id => 42)
+    end
+
+    context 'updates disabled' do
+      before :each do
+        ThinkingSphinx.updates_enabled = false
+      end
+
+      it "should not enqueue a delta job" do
+        Resque.should_not_receive(:enqueue)
+        @delayed_delta.index(@model)
+      end
+
+      it "should not enqueue a flag as deleted job" do
+        Resque.should_not_receive(:enqueue)
+        @delayed_delta.index(@model)
+      end
+    end
+
+    context 'deltas disabled' do
+      before :each do
+        ThinkingSphinx.deltas_enabled = false
+      end
+
+      it "should not enqueue a delta job" do
+        Resque.should_not_receive(:enqueue)
+        @delayed_delta.index(@model)
+      end
+
+      it "should not enqueue a flag as deleted job" do
+        Resque.should_not_receive(:enqueue)
+        @delayed_delta.index(@model)
+      end
+    end
+
+    context "instance isn't toggled" do
+      before :each do
+        @delayed_delta.stub!(:toggled => false)
+      end
+
+      it "should not enqueue a delta job" do
+        Resque.should_not_receive(:enqueue)
+        @delayed_delta.index(@model, @instance)
+      end
+
+      it "should not enqueue a flag as deleted job" do
+        Resque.should_not_receive(:enqueue)
+        @delayed_delta.index(@model, @instance)
+      end
+    end
+
+    it "should enqueue a delta job for the appropriate indexes" do
+      Resque.should_receive(:enqueue).with(ThinkingSphinx::Deltas::ResqueDelta::DeltaJob, ['foo_delta']).once
+      @delayed_delta.index(@model)
+    end
+
+    it "should enqueue a flag-as-deleted job for the appropriate indexes" do
+      # WTF RSpec: http://gist.github.com/447611
+      # Resque.should_receive(:enqueue).with(ThinkingSphinx::Deltas::ResqueDelta::FlagAsDeletedJob, ['foo_core'], an_instance_of(Numeric))
+      Resque.should_receive(:enqueue).twice
+      @delayed_delta.index(@model, @instance)
+    end
+
+    it "should enqueue a flag-as-deleted job for the appropriate id" do
+      # WTF RSpec: http://gist.github.com/447611
+      # Resque.should_receive(:enqueue).with(ThinkingSphinx::Deltas::ResqueDelta::FlagAsDeletedJob, an_instance_of(Array), 42)
+      Resque.should_receive(:enqueue).twice
+      @delayed_delta.index(@model, @instance)
+    end
+  end
+end

data/tasks/distribution.rb

@@ -0,0 +1,29 @@
+require "jeweler"
+require "yard"
+
+YARD::Rake::YardocTask.new
+
+Jeweler::Tasks.new do |gem|
+  gem.name        = "ts-resque-delta"
+  gem.summary     = "Thinking Sphinx - Resque Deltas"
+  gem.description = "Manage delta indexes via Resque for Thinking Sphinx"
+  gem.email       = "aaron.gibralter@gmail.com"
+  gem.homepage    = "http://github.com/agibralter/ts-resque-delta"
+  gem.authors     = ["Aaron Gibralter"]
+  gem.add_dependency("thinking-sphinx", ">= 1.3.6")
+  gem.add_dependency("resque", ">= 1.9.0")
+  gem.add_development_dependency("rspec", ">= 1.2.9")
+  gem.add_development_dependency("yard", ">= 0")
+  gem.add_development_dependency("cucumber", ">= 0")
+  gem.add_development_dependency("database_cleaner", ">= 0.5.2")
+  gem.files = FileList[
+    "lib/**/*.rb",
+    "LICENSE",
+    "README.markdown"
+  ]
+  gem.test_files = FileList[
+    ["Rakefile"] + %w(config features spec tasks).collect { |d| "#{d}/**/*" }
+  ]
+end
+
+Jeweler::GemcutterTasks.new

data/tasks/rails.rake

@@ -0,0 +1 @@
+require File.join(File.dirname(__FILE__), '/../lib/thinking_sphinx/deltas/delayed_delta/tasks')

data/tasks/testing.rb

@@ -0,0 +1,20 @@
+require 'spec/rake/spectask'
+require 'cucumber/rake/task'
+
+Spec::Rake::SpecTask.new(:spec) do |spec|
+  spec.libs      << 'lib' << 'spec'
+  spec.spec_files = FileList['spec/**/*_spec.rb']
+end
+
+Spec::Rake::SpecTask.new(:rcov) do |spec|
+  spec.libs   << 'lib' << 'spec'
+  spec.pattern = 'spec/**/*_spec.rb'
+  spec.rcov    = true
+end
+
+Cucumber::Rake::Task.new do |task|
+  task.cucumber_opts = '--exclude features/thinking_sphinx'
+end
+
+task :spec      => :check_dependencies
+task :cucumber  => :check_dependencies

metadata

@@ -0,0 +1,201 @@
+--- !ruby/object:Gem::Specification 
+name: ts-resque-delta
+version: !ruby/object:Gem::Version 
+  hash: 29
+  prerelease: false
+  segments: 
+  - 0
+  - 0
+  - 1
+  version: 0.0.1
+platform: ruby
+authors: 
+- Aaron Gibralter
+autorequire: 
+bindir: bin
+cert_chain: []
+
+date: 2010-06-23 00:00:00 -04:00
+default_executable: 
+dependencies: 
+- !ruby/object:Gem::Dependency 
+  name: thinking-sphinx
+  prerelease: false
+  requirement: &id001 !ruby/object:Gem::Requirement 
+    none: false
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        hash: 23
+        segments: 
+        - 1
+        - 3
+        - 6
+        version: 1.3.6
+  type: :runtime
+  version_requirements: *id001
+- !ruby/object:Gem::Dependency 
+  name: resque
+  prerelease: false
+  requirement: &id002 !ruby/object:Gem::Requirement 
+    none: false
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        hash: 51
+        segments: 
+        - 1
+        - 9
+        - 0
+        version: 1.9.0
+  type: :runtime
+  version_requirements: *id002
+- !ruby/object:Gem::Dependency 
+  name: rspec
+  prerelease: false
+  requirement: &id003 !ruby/object:Gem::Requirement 
+    none: false
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        hash: 13
+        segments: 
+        - 1
+        - 2
+        - 9
+        version: 1.2.9
+  type: :development
+  version_requirements: *id003
+- !ruby/object:Gem::Dependency 
+  name: yard
+  prerelease: false
+  requirement: &id004 !ruby/object:Gem::Requirement 
+    none: false
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        hash: 3
+        segments: 
+        - 0
+        version: "0"
+  type: :development
+  version_requirements: *id004
+- !ruby/object:Gem::Dependency 
+  name: cucumber
+  prerelease: false
+  requirement: &id005 !ruby/object:Gem::Requirement 
+    none: false
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        hash: 3
+        segments: 
+        - 0
+        version: "0"
+  type: :development
+  version_requirements: *id005
+- !ruby/object:Gem::Dependency 
+  name: database_cleaner
+  prerelease: false
+  requirement: &id006 !ruby/object:Gem::Requirement 
+    none: false
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        hash: 15
+        segments: 
+        - 0
+        - 5
+        - 2
+        version: 0.5.2
+  type: :development
+  version_requirements: *id006
+description: Manage delta indexes via Resque for Thinking Sphinx
+email: aaron.gibralter@gmail.com
+executables: []
+
+extensions: []
+
+extra_rdoc_files: 
+- LICENSE
+- README.markdown
+files: 
+- LICENSE
+- README.markdown
+- lib/thinking_sphinx/deltas/resque_delta.rb
+- lib/thinking_sphinx/deltas/resque_delta/delta_job.rb
+- lib/thinking_sphinx/deltas/resque_delta/flag_as_deleted_job.rb
+- lib/thinking_sphinx/deltas/resque_delta/tasks.rb
+- Rakefile
+- config/redis-cucumber.conf
+- features/resque_deltas.feature
+- features/step_definitions/common_steps.rb
+- features/step_definitions/resque_delta_steps.rb
+- features/support/env.rb
+- features/support/redis_test_setup.rb
+- features/thinking_sphinx/database.example.yml
+- features/thinking_sphinx/database.yml
+- features/thinking_sphinx/db/migrations/create_delayed_betas.rb
+- features/thinking_sphinx/models/delayed_beta.rb
+- spec/spec.opts
+- spec/spec_helper.rb
+- spec/thinking_sphinx/deltas/resque_delta/delta_job_spec.rb
+- spec/thinking_sphinx/deltas/resque_delta/flag_as_deleted_job_spec.rb
+- spec/thinking_sphinx/deltas/resque_delta_spec.rb
+- tasks/distribution.rb
+- tasks/rails.rake
+- tasks/testing.rb
+has_rdoc: true
+homepage: http://github.com/agibralter/ts-resque-delta
+licenses: []
+
+post_install_message: 
+rdoc_options: 
+- --charset=UTF-8
+require_paths: 
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement 
+  none: false
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      hash: 3
+      segments: 
+      - 0
+      version: "0"
+required_rubygems_version: !ruby/object:Gem::Requirement 
+  none: false
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      hash: 3
+      segments: 
+      - 0
+      version: "0"
+requirements: []
+
+rubyforge_project: 
+rubygems_version: 1.3.7
+signing_key: 
+specification_version: 3
+summary: Thinking Sphinx - Resque Deltas
+test_files: 
+- Rakefile
+- config/redis-cucumber.conf
+- features/resque_deltas.feature
+- features/step_definitions/common_steps.rb
+- features/step_definitions/resque_delta_steps.rb
+- features/support/env.rb
+- features/support/redis_test_setup.rb
+- features/thinking_sphinx/database.example.yml
+- features/thinking_sphinx/database.yml
+- features/thinking_sphinx/db/migrations/create_delayed_betas.rb
+- features/thinking_sphinx/models/delayed_beta.rb
+- spec/spec.opts
+- spec/spec_helper.rb
+- spec/thinking_sphinx/deltas/resque_delta/delta_job_spec.rb
+- spec/thinking_sphinx/deltas/resque_delta/flag_as_deleted_job_spec.rb
+- spec/thinking_sphinx/deltas/resque_delta_spec.rb
+- tasks/distribution.rb
+- tasks/rails.rake
+- tasks/testing.rb