acts_as_indexed 0.6.6 → 0.6.7

data/CHANGELOG

@@ -1,3 +1,8 @@
+===0.6.7 [7th February 2011]
+ - find_by_index and paginate_search are no longer deprecated.
+ - Improved documentation
+ - Storage is now it's own class to allow future development of locking and pluggable backends.
+
 ===0.6.6 [31st August 2010]
  - Now Heroku compatible out of the box, index is created in tmp when root dir is non-writable. [parndt - Philip Arndt - Great suggestion]
  - Fixed a require path issue on 1.9.2.

data/README.rdoc

@@ -35,7 +35,7 @@ Gemfile (Rails 3.x.x).
 
 ==== No Git?
 
-If you don't have git installed, you can download the plugin from the GitHub
+If you don't have git installed, but still want the plugin, you can download the plugin from the GitHub
 page (http://github.com/dougal/acts_as_indexed) and unpack it into the
 <tt>vendor/plugins</tt> directory of your rails app.
 
@@ -85,8 +85,26 @@ or true if it is visible, you can filter the index by doing:
 
 === Searching
 
-To search, call the +with_query+ named scope on your model, passing a query as
-an argument.
+==== With Relevance
+
+To search with the most relevant matches appearing first, call the
++find_with_index+ method on your model, passing a query as the first argument.
+The optional +ids_only+ parameter, when set to true, will return only the IDs
+of any matching records.
+
+  # Returns array of Post objects ordered by relevance.
+  my_search_results = Post.find_with_index('my search query')
+
+  # Pass any of the ActiveRecord find options to the search.
+  my_search_results = Post.find_with_index('my search query',{:limit => 10}) # return the first 10 matches.
+  
+  # Returns array of IDs ordered by relevance.
+  my_search_results = Post.find_with_index('my search query',{},{:ids_only => true}) # =>  [12,19,33...
+
+==== Without Relevance (Scope)
+
+If the relevance of the results is not important, call the +with_query+ named
+scope on your model, passing a query as an argument.
 
   # Returns array of Post objects.
   my_search_results = Post.with_query('my search query')
@@ -98,15 +116,23 @@ an argument.
 
 The following query operators are supported:
 
-* AND:: This is the default option. 'cat dog' will find records matching 'cat' AND 'dog'.
-* NOT:: 'cat -dog' will find records matching 'cat' AND NOT 'dog'
-* INCLUDE:: 'cat +me' will find records matching 'cat' and 'me', even if 'me' is smaller than the +min_word_size+
-* "":: Quoted terms are matched as phrases. '"cat dog"' will find records matching the whole phrase. Quoted terms can be preceded by the NOT operator; 'cat -"big dog"' etc. Quoted terms can include words shorter than the +min_word_size+.
-* ^:: Terms that begin with ^ will match records that contain a word starting with the term. '^cat' will find matches containing 'cat', 'catapult', 'caterpillar' etc.
-* ^"":: A quoted term that begins with ^ matches any phrase that begin with this phrase. '^"cat d"' will find records matching the whole phrases "cat dog" and "cat dinner". This type of search is useful for autocomplete inputs.
+* AND :: This is the default option. 'cat dog' will find records matching 'cat' AND 'dog'.
+* NOT :: 'cat -dog' will find records matching 'cat' AND NOT 'dog'
+* INCLUDE :: 'cat +me' will find records matching 'cat' and 'me', even if 'me' is smaller than the +min_word_size+
+* "" :: Quoted terms are matched as phrases. '"cat dog"' will find records matching the whole phrase. Quoted terms can be preceded by the NOT operator; 'cat -"big dog"' etc. Quoted terms can include words shorter than the +min_word_size+.
+* ^ :: Terms that begin with ^ will match records that contain a word starting with the term. '^cat' will find matches containing 'cat', 'catapult', 'caterpillar' etc.
+* ^"" :: A quoted term that begins with ^ matches any phrase that begin with this phrase. '^"cat d"' will find records matching the whole phrases "cat dog" and "cat dinner". This type of search is useful for autocomplete inputs.
 
 === Pagination
 
+==== With Relevance
+
+Pagination is supported via the +paginate_search+ method whose first argument is the search query, followed all the standard will_paginate arguments.
+  
+  @images = Image.paginate_search('girl', :page => 1, :per_page => 5)
+
+==== Without Relevance (Scope)
+
 Since +with_query+ is a named scope, WillPaginate can be used in the normal
 fashion.
 
@@ -118,7 +144,7 @@ A config block can be provided in your environment files or initializers.
 Example showing defaults:
 
   ActsAsIndexed.configure do |config|
-    config.index_file = [RAILS_ROOT,'index'] 
+    config.index_file = [Rails.root.to_s,'index'] 
     config.index_file_depth = 3
     config.min_word_size = 3
   end
@@ -146,10 +172,15 @@ online[http://rdoc.info/projects/dougal/acts_as_indexed/].
 All of the above are most welcome. mailto:dougal.s@gmail.com
 
 
-== Credits
-
-Douglas F Shearer - http://douglasfshearer.com
+== Contributors
 
+* Douglas F Shearer - http://douglasfshearer.com
+* Thomas Pomfret
+* Philip Arndt
+* Fernanda Lopes
+* Alex Coles
+* Myles Eftos
+* Edward Anderson
 
 == Future Releases
 

data/VERSION

@@ -1 +1 @@
-0.6.6
+0.6.7

data/acts_as_indexed.gemspec

@@ -5,11 +5,11 @@
 
 Gem::Specification.new do |s|
   s.name = %q{acts_as_indexed}
-  s.version = "0.6.6"
+  s.version = "0.6.7"
 
   s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
   s.authors = ["Douglas F Shearer"]
-  s.date = %q{2010-08-31}
+  s.date = %q{2011-02-07}
   s.description = %q{Acts As Indexed is a plugin which provides a pain-free way to add fulltext search to your Ruby on Rails app}
   s.email = %q{dougal.s@gmail.com}
   s.extra_rdoc_files = [
@@ -29,6 +29,7 @@ Gem::Specification.new do |s|
      "lib/acts_as_indexed/configuration.rb",
      "lib/acts_as_indexed/search_atom.rb",
      "lib/acts_as_indexed/search_index.rb",
+     "lib/acts_as_indexed/storage.rb",
      "lib/will_paginate_search.rb",
      "rails/init.rb",
      "test/abstract_unit.rb",

data/lib/acts_as_indexed/search_atom.rb

@@ -58,6 +58,12 @@ module ActsAsIndexed #:nodoc:
                                                           })
     end
 
+    # Creates a new SearchAtom with records in other removed from self.
+    def -(other)
+      records = @records.clone.reject { |name, records| other.records.include?(name) }
+      SearchAtom.new(records)
+    end
+
     # Returns at atom containing the records and positions of +self+ preceded by +former+
     # "former latter" or "big dog" where "big" is the former and "dog" is the latter.
     def preceded_by(former)

data/lib/acts_as_indexed/search_index.rb

@@ -12,42 +12,37 @@ module ActsAsIndexed #:nodoc:
     # min_word_size:: Smallest query term that will be run through search.
     # if_proc:: A Proc. If the proc is true, the index gets added, if false if doesn't
     def initialize(root, index_depth, fields, min_word_size, if_proc=Proc.new{true})
-      @root = Pathname.new(root.to_s)
+      @storage = Storage.new(Pathname.new(root.to_s), index_depth)
       @fields = fields
-      @index_depth = index_depth
       @atoms = {}
       @min_word_size = min_word_size
-      @records_size = exists? ? load_record_size : 0
+      @records_size = @storage.record_count
       @if_proc = if_proc
     end
 
     # Adds +record+ to the index.
-    def add_record(record, no_save=false)
+    def add_record(record)
       return unless @if_proc.call(record)
+
       condensed_record = condense_record(record)
-      load_atoms(condensed_record)
-      add_occurences(condensed_record,record.id)
-      @records_size += 1
-      self.save unless no_save
+      atoms = add_occurences(condensed_record,record.id)
+      
+      @storage.add(atoms)
     end
 
     # Adds multiple records to the index. Accepts an array of +records+.
     def add_records(records)
       records.each do |record|
-        add_record(record, true)
+        add_record(record)
       end
-      self.save
     end
 
     # Removes +record+ from the index.
     def remove_record(record)
-      atoms = condense_record(record)
-      load_atoms(atoms)
-      atoms.each do |a|
-        @atoms[a].remove_record(record.id) if @atoms.has_key?(a)
-        @records_size -= 1
-      end
-      self.save
+      condensed_record = condense_record(record)
+      atoms = add_occurences(condensed_record,record.id)
+      
+      @storage.remove(atoms)
     end
 
     def update_record(record_new, record_old)
@@ -55,33 +50,11 @@ module ActsAsIndexed #:nodoc:
       add_record(record_new)
     end
 
-    # Saves the current index partitions to the filesystem.
-    def save
-      prepare
-      atoms_sorted = {}
-      @atoms.each do |atom_name, records|
-        (atoms_sorted[encoded_prefix(atom_name)] ||= {})[atom_name] = records
-      end
-      atoms_sorted.each do |e_p, atoms|
-        @root.join(e_p.to_s).open("w+") do |f|
-          Marshal.dump(atoms,f)
-        end
-      end
-      save_record_size
-    end
-
-    # Deletes the current model's index from the filesystem.
-    #--
-    # TODO: Write a public method that will delete all indexes.
-    def destroy
-      @root.delete
-    end
-
     # Returns an array of IDs for records matching +query+.
     def search(query)
       return [] if query.nil?
-      load_options = { :start => true } if query[/\^/]
-      load_atoms(cleanup_atoms(query), load_options || {})
+
+      @atoms = @storage.fetch(cleanup_atoms(query), query[/\^/])
       queries = parse_query(query.dup)
       positive = run_queries(queries[:positive])
       positive_quoted = run_quoted_queries(queries[:positive_quoted])
@@ -95,15 +68,15 @@ module ActsAsIndexed #:nodoc:
       if queries[:start_quoted].any?
         results = merge_query_results(results, start_quoted)
       end
-      
+
       if queries[:starts_with].any?
         results = merge_query_results(results, starts_with)
       end
-      
+
       if queries[:positive_quoted].any?
         results = merge_query_results(results, positive_quoted)
       end
-      
+
       if queries[:positive].any?
         results = merge_query_results(results, positive)
       end
@@ -112,100 +85,32 @@ module ActsAsIndexed #:nodoc:
       results.delete_if { |r_id, w| negative_results.include?(r_id) }
       results
     end
-    
+
+    private
+
     def merge_query_results(results1, results2)
       # Return the other if one is empty.
       return results1 if results2.empty?
       return results2 if results1.empty?
-      
+
       # Delete any records from results 1 that are not in results 2.
       r1 = results1.delete_if{ |r_id,w| !results2.include?(r_id) }
-      
-      
+
+
       # Delete any records from results 2 that are not in results 1.
       r2 = results2.delete_if{ |r_id,w| !results1.include?(r_id) }
-      
+
       # Merge the results by adding their respective scores.
       r1.merge(r2) { |r_id,old_val,new_val| old_val + new_val}
     end
-    
-    # Returns true if the index root exists on the FS.
-    #--
-    # TODO: Make a private method called 'root_exists?' which checks for the root directory.
-    def exists?
-      @root.join('size').exist?
-    end
-
-    private
-
-    # Gets the size file from the index.
-    def load_record_size
-      @root.join('size').open do |f|
-        Marshal.load(f)
-      end
-    end
-
-    # Saves the size to the size file.
-    def save_record_size
-      @root.join('size').open('w+') do |f|
-        Marshal.dump(@records_size,f)
-      end
-    end
-
-    # Returns true if the given atom is present.
-    def include_atom?(atom)
-      if atom.is_a? Regexp
-        @atoms.keys.grep(atom).any?
-      else
-        @atoms.has_key?(atom)
-      end
-    end
-
-    # Returns true if all the given atoms are present.
-    def include_atoms?(atoms_arr)
-      atoms_arr.each do |a|
-        return false unless include_atom?(a)
-      end
-      true
-    end
-
-    # Returns true if the given record is present.
-    def include_record?(record_id)
-      @atoms.each do |atomname, atom|
-        return true if atom.include_record?(record_id)
-      end
-    end
-
-    def add_atom(atom)
-      @atoms[atom] = SearchAtom.new unless include_atom?(atom)
-    end
 
     def add_occurences(condensed_record,record_id)
-      condensed_record.each_with_index do |atom, i|
-        add_atom(atom)
-        @atoms[atom].add_position(record_id, i)
-      end
-    end
-
-    def encoded_prefix(atom)
-      prefix = atom[0,@index_depth]
-      unless (@prefix_cache ||= {}).has_key?(prefix)
-        if atom.length > 1
-          @prefix_cache[prefix] = prefix.split(//).map{|c| encode_character(c)}.join('_')
-        else
-          @prefix_cache[prefix] = encode_character(atom)
-        end
-      end
-      @prefix_cache[prefix]
-    end
-
-    # Allows compatibility with 1.8.6 which has no ord method.
-    def encode_character(char)
-      if @@has_ord ||= char.respond_to?(:ord)
-        char.ord.to_s
-      else
-        char[0]
+      atoms = {}
+      condensed_record.each_with_index do |atom_name, i|
+        atoms[atom_name] = SearchAtom.new unless atoms.include?(atom_name)
+        atoms[atom_name].add_position(record_id, i)
       end
+      atoms
     end
 
     def parse_query(s)
@@ -254,14 +159,15 @@ module ActsAsIndexed #:nodoc:
         :positive_quoted => positive_quoted,
         :starts_with => starts_with,
         :negative => negative,
-        :positive => positive }
+        :positive => positive
+      }
     end
-    
+
     def run_queries(atoms, starts_with=false)
       results = {}
       atoms.each do |atom|
         interim_results = {}
-        
+
         # If these atoms are to be run as 'starts with', make them a Regexp
         # with a carat.
         atom = /^#{atom}/ if starts_with
@@ -269,37 +175,37 @@ module ActsAsIndexed #:nodoc:
         # Get the resulting matches, and break if none exist.
         matches = get_atom_results(@atoms.keys, atom)
         break if matches.nil?
-        
+
         # Grab the record IDs and weightings.
         interim_results = matches.weightings(@records_size)
-        
+
         # Merge them with the results obtained already, if any.
         results = results.empty? ? interim_results : merge_query_results(results, interim_results)
-        
+
         break if results.empty?
-        
+
       end
       results
     end
-    
+
     def run_quoted_queries(quoted_atoms, starts_with=false)
       results = {}
       quoted_atoms.each do |quoted_atom|
         interim_results = {}
-        
+
         break if quoted_atom.empty?
-        
+
         # If these atoms are to be run as 'starts with', make the final atom a
         # Regexp with a line-start anchor.
         quoted_atom[-1] = /^#{quoted_atom.last}/ if starts_with
-        
+
         # Little bit of memoization.
         atoms_keys = @atoms.keys
-        
+
         # Get the matches for the first atom.
         matches = get_atom_results(atoms_keys, quoted_atom.first)
         break if matches.nil?
-        
+
         # Check the index contains all the required atoms.
         # for each of the others
         #   return atom containing records + positions where current atom is preceded by following atom.
@@ -320,9 +226,9 @@ module ActsAsIndexed #:nodoc:
 
         # Merge them with the results obtained already, if any.
         results = results.empty? ? interim_results : merge_query_results(results, interim_results)
-        
+
         break if results.empty?
-        
+
       end
       results
     end
@@ -340,26 +246,6 @@ module ActsAsIndexed #:nodoc:
       end
     end
 
-    def load_atoms(atoms, options={})
-      # Remove duplicate atoms.
-      # Remove atoms already in index.
-      # Calculate prefixes.
-      # Remove duplicate prefixes.
-      atoms.uniq.reject{|a| include_atom?(a)}.collect{|a| encoded_prefix(a)}.uniq.each do |name|
-        pattern = @root.join(name.to_s).to_s
-        pattern += '*' if options[:start]
-        Pathname.glob(pattern).each do |atom_file|
-          atom_file.open do |f|
-            @atoms.merge!(Marshal.load(f))
-          end
-        end
-      end
-    end
-
-    def prepare
-      # Makes the RAILS_ROOT/index/ENVIRONMENT/CLASS directories
-      @root.mkpath
-    end
 
     def cleanup_atoms(s, limit_size=false, min_size = @min_word_size || 3)
       atoms = s.downcase.gsub(/\W/,' ').squeeze(' ').split

data/lib/acts_as_indexed/storage.rb

@@ -0,0 +1,136 @@
+# ActsAsIndexed
+# Copyright (c) 2007 - 2010 Douglas F Shearer.
+# http://douglasfshearer.com
+# Distributed under the MIT license as included with this plugin.
+
+module ActsAsIndexed #:nodoc:
+  class Storage
+
+    def initialize(path, prefix_size)
+      @path = path
+      @prefix_size = prefix_size
+      prepare
+    end
+
+    # Takes a hash of atoms and adds these to storage.
+    def add(atoms)
+      operate(:+, atoms)
+
+      update_record_count(1)
+
+    end
+
+    # Takes a hash of atoms and removes these from storage.
+    def remove(atoms)
+      operate(:-, atoms)
+
+      update_record_count(-1)
+    end
+
+    # Takes a string array of atoms names
+    # return a hash of the relevant atoms.
+    def fetch(atom_names, start=false)
+      atoms = {}
+
+      atom_names.uniq.collect{|a| encoded_prefix(a) }.uniq.each do |prefix|
+        pattern = @path.join(prefix.to_s).to_s
+        pattern += '*' if start
+
+        Pathname.glob(pattern).each do |atom_file|
+          atom_file.open do |f|
+            atoms.merge!(Marshal.load(f))
+          end
+        end # Pathname.glob
+
+      end # atom_names.uniq
+      atoms
+    end # fetch.
+
+    # Returns the number of records currently stored in this index.
+    def record_count
+      # TODO: Record count is currently a marshaled integer. Why not store as
+      # string integer? Breaks compatibility, so leave until other changes
+      # need to be made to the index.
+
+      @path.join('size').open do |f|
+        Marshal.load(f)
+      end
+
+    # This is a bit horrible.
+    rescue Errno::ENOENT
+      0
+    rescue EOFError
+      0
+    end
+
+    private
+
+    # Takes atoms and adds or removes them from the index depending on the
+    # passed operation.
+    def operate(operation, atoms)
+      # ActiveSupport always available?
+      atoms_sorted = ActiveSupport::OrderedHash.new
+
+      # Sort the atoms into the appropriate shards for writing to individual
+      # files.
+      atoms.each do |atom_name, records|
+        (atoms_sorted[encoded_prefix(atom_name)] ||= {})[atom_name] = records
+      end
+
+      atoms_sorted.each do |e_p, atoms|
+        path = @path.join(e_p.to_s)
+
+        if path.exist?
+          from_file = path.open do |f|
+            Marshal.load(f)
+          end
+        else
+          from_file = {}
+        end
+
+        atoms = from_file.merge(atoms){ |k,o,n| o.send(operation, n) }
+
+        path.open("w+") do |f|
+          Marshal.dump(atoms,f)
+        end
+      end
+    end
+
+    def update_record_count(delta)
+      new_count = self.record_count + delta
+      new_count = 0 if new_count < 0
+
+      @path.join('size').open('w+') do |f|
+        Marshal.dump(new_count,f)
+      end
+    end
+
+    def prepare
+      @path.mkpath unless @path.exist?
+    end
+
+    def encoded_prefix(atom)
+      prefix = atom[0, @prefix_size]
+
+      unless (@prefix_cache ||= {}).has_key?(prefix)
+        if atom.length > 1
+          @prefix_cache[prefix] = prefix.split(//).map{|c| encode_character(c)}.join('_')
+        else
+          @prefix_cache[prefix] = encode_character(atom)
+        end
+      end
+
+      @prefix_cache[prefix]
+    end
+
+    # Allows compatibility with 1.8.6 which has no ord method.
+    def encode_character(char)
+      if @@has_ord ||= char.respond_to?(:ord)
+        char.ord.to_s
+      else
+        char[0]
+      end
+    end
+
+  end
+end

data/lib/acts_as_indexed.rb

@@ -8,6 +8,7 @@ require 'active_record'
 require 'acts_as_indexed/configuration'
 require 'acts_as_indexed/search_index'
 require 'acts_as_indexed/search_atom'
+require 'acts_as_indexed/storage'
 
 module ActsAsIndexed #:nodoc:
 
@@ -102,8 +103,6 @@ module ActsAsIndexed #:nodoc:
 
     def index_remove(record)
       index = SearchIndex.new(aai_config.index_file, aai_config.index_file_depth, aai_fields, aai_config.min_word_size, aai_config.if_proc)
-      # record won't be in index if it doesn't exist. Just return true.
-      return unless index.exists?
       index.remove_record(record)
       @query_cache = {}
     end
@@ -192,12 +191,10 @@ module ActsAsIndexed #:nodoc:
   # Adds model class singleton methods.
   module SingletonMethods
 
-    # DEPRECATED. Use +with_query+ scope instead.
     # Finds instances matching the terms passed in +query+.
     #
     # See ActsAsIndexed::ClassMethods#search_index.
     def find_with_index(query='', find_options = {}, options = {})
-      warn "[DEPRECATION] `find_with_index` is deprecated and will be removed in a later release. Use `with_query(query)` instead."
       search_index(query, find_options, options)
     end
 

data/lib/will_paginate_search.rb

@@ -8,9 +8,7 @@ module ActsAsIndexed
 
      module Search
 
-      # DEPRECATED. Use chained pagination instead.
       def paginate_search(query, options)
-        warn "[DEPRECATION] `paginate_search` is deprecated and will be removed in a later release. Use `with_query(query).paginate()` instead."
         page, per_page, total_entries = wp_parse_options(options)
 
         total_entries ||= find_with_index(query,{},{:ids_only => true}).size

data/test/acts_as_indexed_test.rb

@@ -14,164 +14,136 @@ class ActsAsIndexedTest < ActiveSupport::TestCase
     assert_equal [], Post.find_with_index('badger')
     post = Post.new(:title => 'badger', :body => 'Thousands of them!')
     assert post.save
-    assert_equal original_post_count+1, Post.count
-    assert_equal [post.id], Post.find_with_index('badger',{},{:ids_only => true})
+    assert_equal original_post_count + 1, Post.count
+    assert_equal [post.id], result_ids('badger')
   end
-
+  
   def test_removes_from_index
     original_post_count = Post.count
-    assert_equal [posts(:wikipedia_article_4).id], Post.find_with_index('album',{},{:ids_only => true})
+    assert_equal [posts(:wikipedia_article_4).id], result_ids('album')
     assert Post.find(posts(:wikipedia_article_4).id).destroy
-    assert_equal [], Post.find_with_index('album',{},{:ids_only => true})
-    assert_equal original_post_count-1, Post.count
+    assert_equal [], result_ids('album')
+    assert_equal original_post_count - 1, Post.count
   end
-
+  
   def test_search_returns_posts
-    Post.find_with_index('album').each do |p|
-      assert_equal Post, p.class
-    end
-  end
-
-  def test_scoped_search_returns_posts
     Post.with_query('album').each do |p|
       assert_equal Post, p.class
     end
   end
-
+  
   def test_search_returns_post_ids
-    Post.find_with_index('album',{},{:ids_only => true}).each do |pid|
+    result_ids('album').each do |pid|
       assert p = Post.find(pid)
       assert_equal Post, p.class
     end
   end
-
+  
   # After a portion of a record has been removed
   # the portion removes should no longer be in the index.
   def test_updates_index
     p = Post.create(:title => 'A special title', :body => 'foo bar bla bla bla')
-    assert Post.find_with_index('title',{},{:ids_only => true}).include?(p.id)
+    assert result_ids('title').include?(p.id)
     p.update_attributes(:title => 'No longer special')
-    assert !Post.find_with_index('title',{},{:ids_only => true}).include?(p.id)
+    assert !result_ids('title').include?(p.id)
   end
-
+  
   def test_simple_queries
-    assert_equal [],  Post.find_with_index(nil)
-    assert_equal [],  Post.find_with_index('')
-    assert_equal [5, 6],  Post.find_with_index('ship',{},{:ids_only => true}).sort
-    assert_equal [6],  Post.find_with_index('foo',{},{:ids_only => true})
-    assert_equal [6],  Post.find_with_index('foo ship',{},{:ids_only => true})
-    assert_equal [6],  Post.find_with_index('ship foo',{},{:ids_only => true})
-  end
-
-  def test_scoped_simple_queries
-    assert_equal [],  Post.find_with_index(nil)
-    assert_equal [],  Post.with_query('')
-    assert_equal [5, 6],  Post.with_query('ship').map{|r| r.id}.sort
-    assert_equal [6],  Post.with_query('foo').map{|r| r.id}
-    assert_equal [6],  Post.with_query('foo ship').map{|r| r.id}
-    assert_equal [6],  Post.with_query('ship foo').map{|r| r.id}
-  end
-
+    assert_equal_array_contents [],     result_ids(nil)
+    assert_equal_array_contents [],     result_ids('')
+    assert_equal_array_contents [5, 6], result_ids('ship')
+    assert_equal_array_contents [6],    result_ids('foo')
+    assert_equal_array_contents [6],    result_ids('foo ship')
+    assert_equal_array_contents [6],    result_ids('ship foo')
+  end
+  
   def test_negative_queries
-    assert_equal [5, 6],  Post.find_with_index('crane',{},{:ids_only => true}).sort
-    assert_equal [5],  Post.find_with_index('crane -foo',{},{:ids_only => true})
-    assert_equal [5],  Post.find_with_index('-foo crane',{},{:ids_only => true})
-    assert_equal [],  Post.find_with_index('-foo') #Edgecase
-  end
-
-  def test_scoped_negative_queries
-    assert_equal [5, 6],  Post.with_query('crane').map{|r| r.id}.sort
-    assert_equal [5],  Post.with_query('crane -foo').map{|r| r.id}
-    assert_equal [5],  Post.with_query('-foo crane').map{|r| r.id}
-    assert_equal [],  Post.with_query('-foo') #Edgecase
+    assert_equal_array_contents [5, 6], result_ids('crane')
+    assert_equal_array_contents [5],    result_ids('crane -foo')
+    assert_equal_array_contents [5],    result_ids('-foo crane')
+    assert_equal_array_contents [],     result_ids('-foo') #Edgecase
   end
-
+  
   def test_quoted_queries
-    assert_equal [5],  Post.find_with_index('"crane ship"',{},{:ids_only => true})
-    assert_equal [6],  Post.find_with_index('"crane big"',{},{:ids_only => true})
-    assert_equal [],  Post.find_with_index('foo "crane ship"')
-    assert_equal [],  Post.find_with_index('"crane badger"')
+    assert_equal_array_contents [5], result_ids('"crane ship"')
+    assert_equal_array_contents [6], result_ids('"crane big"')
+    assert_equal_array_contents [],  result_ids('foo "crane ship"')
+    assert_equal_array_contents [],  result_ids('"crane badger"')
   end
-
-  def test_scoped_quoted_queries
-    assert_equal [5],  Post.with_query('"crane ship"').map{|r| r.id}
-    assert_equal [6],  Post.with_query('"crane big"').map{|r| r.id}
-    assert_equal [],  Post.with_query('foo "crane ship"')
-    assert_equal [],  Post.with_query('"crane badger"')
-  end
-
+  
   def test_negative_quoted_queries
-    assert_equal [6],  Post.find_with_index('crane -"crane ship"',{},{:ids_only => true})
-    assert_equal [],  Post.find_with_index('-"crane big"',{},{:ids_only => true}) # Edgecase
+    assert_equal_array_contents [6], result_ids('crane -"crane ship"')
+    assert_equal_array_contents [],  result_ids('-"crane big"') # Edgecase
   end
-
+  
   def test_scoped_negative_quoted_queries
-    assert_equal [6],  Post.with_query('crane -"crane ship"').map{|r| r.id}
-    assert_equal [],  Post.with_query('-"crane big"') # Edgecase
+    assert_equal_array_contents [6], result_ids('crane -"crane ship"')
+    assert_equal_array_contents [],  result_ids('-"crane big"') # Edgecase
   end
-
+  
   def test_start_queries
-    assert_equal [6,5],  Post.find_with_index('ship ^crane',{},{:ids_only => true})
-    assert_equal [6,5],  Post.find_with_index('^crane ship',{},{:ids_only => true})
-    assert_equal [6,5],  Post.find_with_index('^ship ^crane',{},{:ids_only => true})
-    assert_equal [6,5],  Post.find_with_index('^crane ^ship',{},{:ids_only => true})
-    assert_equal [6,5],  Post.find_with_index('^ship crane',{},{:ids_only => true})
-    assert_equal [6,5],  Post.find_with_index('crane ^ship',{},{:ids_only => true})
-    assert_equal [6,5],  Post.find_with_index('^crane',{},{:ids_only => true})
-    assert_equal [6,5],  Post.find_with_index('^cran',{},{:ids_only => true})
-    assert_equal [6,5],  Post.find_with_index('^cra',{},{:ids_only => true})
-    assert_equal [6,5,4],  Post.find_with_index('^cr',{},{:ids_only => true})
-    assert_equal [6,5,4,3,2,1], Post.find_with_index('^c',{},{:ids_only => true})
-    assert_equal [], Post.find_with_index('^notthere',{},{:ids_only => true})
-  end
-
+    assert_equal_array_contents [6,5],         result_ids('ship ^crane')
+    assert_equal_array_contents [6,5],         result_ids('^crane ship')
+    assert_equal_array_contents [6,5],         result_ids('^ship ^crane')
+    assert_equal_array_contents [6,5],         result_ids('^crane ^ship')
+    assert_equal_array_contents [6,5],         result_ids('^ship crane')
+    assert_equal_array_contents [6,5],         result_ids('crane ^ship')
+    assert_equal_array_contents [6,5],         result_ids('^crane')
+    assert_equal_array_contents [6,5],         result_ids('^cran')
+    assert_equal_array_contents [6,5],         result_ids('^cra')
+    assert_equal_array_contents [6,5,4],       result_ids('^cr')
+    assert_equal_array_contents [6,5,4,3,2,1], result_ids('^c')
+    assert_equal_array_contents [],            result_ids('^notthere')
+  end
+  
   def test_start_quoted_queries
-    assert_equal [6,5],  Post.find_with_index('^"crane" ship',{},{:ids_only => true})
-    assert_equal [6,5],  Post.find_with_index('ship ^"crane"',{},{:ids_only => true})
-    assert_equal [5],  Post.find_with_index('^"crane ship"',{},{:ids_only => true})
-    assert_equal [5],  Post.find_with_index('^"crane shi"',{},{:ids_only => true})
-    assert_equal [5],  Post.find_with_index('^"crane sh"',{},{:ids_only => true})
-    assert_equal [5],  Post.find_with_index('^"crane s"',{},{:ids_only => true})
-    assert_equal [6,5],  Post.find_with_index('^"crane "',{},{:ids_only => true})
-    assert_equal [6,5],  Post.find_with_index('^"crane"',{},{:ids_only => true})
-    assert_equal [6,5],  Post.find_with_index('^"cran"',{},{:ids_only => true})
-    assert_equal [6,5],  Post.find_with_index('^"cra"',{},{:ids_only => true})
-    assert_equal [6,5,4],  Post.find_with_index('^"cr"',{},{:ids_only => true})
-    assert_equal [6,5,4,3,2,1], Post.find_with_index('^"c"',{},{:ids_only => true})
-  end
-
+    assert_equal_array_contents [6,5],         result_ids('^"crane" ship')
+    assert_equal_array_contents [6,5],         result_ids('ship ^"crane"')
+    assert_equal_array_contents [5],           result_ids('^"crane ship"')
+    assert_equal_array_contents [5],           result_ids('^"crane shi"')
+    assert_equal_array_contents [5],           result_ids('^"crane sh"')
+    assert_equal_array_contents [5],           result_ids('^"crane s"')
+    assert_equal_array_contents [6,5],         result_ids('^"crane "')
+    assert_equal_array_contents [6,5],         result_ids('^"crane"')
+    assert_equal_array_contents [6,5],         result_ids('^"cran"')
+    assert_equal_array_contents [6,5],         result_ids('^"cra"')
+    assert_equal_array_contents [6,5,4],       result_ids('^"cr"')
+    assert_equal_array_contents [6,5,4,3,2,1], result_ids('^"c"')
+  end
+  
   def test_find_options
     all_results = Post.find_with_index('crane',{},{:ids_only => true})
+    assert_equal 2, all_results.size
+    
     first_result = Post.find_with_index('crane',{:limit => 1})
-
     assert_equal 1, first_result.size
     assert_equal all_results.first, first_result.first.id
-
+  
     second_result = Post.find_with_index('crane',{:limit => 1, :offset => 1})
     assert_equal 1, second_result.size
     assert_equal all_results[1], second_result.first.id
   end
-
+  
   # When a atom already in a record is duplicated, it removes
   # all records with that same atom from the index.
   def test_update_record_bug
     p = Post.find(6)
     assert p.update_attributes(:body => p.body + ' crane')
-    assert_equal 2, Post.find_with_index('crane',{},{:ids_only => true}).size
-    assert_equal 2, Post.find_with_index('ship',{},{:ids_only => true}).size
+    assert_equal 2, result_ids('crane').size
+    assert_equal 2, result_ids('ship').size
   end
-
+  
   # If an if proc is supplied, the index should only be created if the proc evaluated true
   def test_create_if
     Post.acts_as_indexed :fields => [:title, :body], :if => Proc.new { |post| post.visible }
-
+  
     original_post_count = Post.count
-    assert_equal [], Post.find_with_index('badger')
+    assert_equal [], Post.find_with_index('badger', {}, { :no_query_cache => true, :ids_only => true})
     p = Post.new(:title => 'badger', :body => 'thousands of them!', :visible => true)
     assert p.save
     assert_equal original_post_count + 1, Post.count
     assert_equal [p.id], Post.find_with_index('badger', {}, { :no_query_cache => true, :ids_only => true})
-
+  
     original_post_count = Post.count
     assert_equal [], Post.find_with_index('unicorns')
     p = Post.new(:title => 'unicorns', :body => 'there are none', :visible => false)
@@ -179,49 +151,59 @@ class ActsAsIndexedTest < ActiveSupport::TestCase
     assert_equal original_post_count + 1, Post.count
     assert_equal [], Post.find_with_index('unicorns', {}, { :no_query_cache => true, :ids_only => true})
   end
-
+  
   # If an index already exists, and an if proc is supplied, and the proc is true, it should still appear in the index
   def test_update_if_update
     Post.acts_as_indexed :fields => [:title, :body], :if => Proc.new { |post| post.visible }
     destroy_index
-
+  
     assert_equal 1, Post.find_with_index('crane', {}, { :no_query_cache => true, :ids_only => true}).size
     p = Post.find(6)
     assert p.update_attributes(:visible => true)
     assert_equal 1, Post.find_with_index('crane', {}, { :no_query_cache => true, :ids_only => true}).size
   end
-
+  
   # If an index already exists, and an if proc is supplied, and the proc is false, it should no longer appear in the index
   def test_update_if_remove
     Post.acts_as_indexed :fields => [:title, :body], :if => Proc.new { |post| post.visible }
     destroy_index
-
+  
     assert_equal 1, Post.find_with_index('crane', {}, { :no_query_cache => true, :ids_only => true}).size
     p = Post.find(6)
     assert p.update_attributes(:visible => false)
     assert_equal 0, Post.find_with_index('crane',{},{ :no_query_cache => true, :ids_only => true}).size
   end
-
+  
   # If an index doesn't exist, and an if proc is supplied, and the proc is true, it should appear in the index
   def test_update_if_add
     Post.acts_as_indexed :fields => [:title, :body], :if => Proc.new { |post| post.visible }
     destroy_index
-
+  
     assert_equal 1, Post.find_with_index('crane', {}, { :no_query_cache => true, :ids_only => true}).size
     p = Post.find(5)
     assert p.update_attributes(:visible => true)
     assert_equal 2, Post.find_with_index('crane',{},{ :no_query_cache => true, :ids_only => true}).size
   end
-
+  
   # If an index doesn't exist, and an if proc is supplied, and the proc is false, then nothing happens
   def test_update_if_not_in
     Post.acts_as_indexed :fields => [:title, :body], :if => Proc.new { |post| post.visible }
     destroy_index
-
+  
     assert_equal 1, Post.find_with_index('crane', {}, { :no_query_cache => true, :ids_only => true}).size
     p = Post.find(5)
     assert p.update_attributes(:visible => false)
     assert_equal 1, Post.find_with_index('crane',{},{ :no_query_cache => true, :ids_only => true}).size
   end
 
+  private
+  
+  def result_ids(query)
+    Post.with_query(query).map { |r| r.id }
+  end
+
+  def assert_equal_array_contents(a, b)
+    a.sort == b.sort
+  end
+
 end

data/test/search_index_test.rb

@@ -6,40 +6,9 @@ class SearchIndexTest < ActiveSupport::TestCase
   def teardown
     destroy_index
   end
-
-  def test_should_check_for_non_existing_index
-    SearchIndex.any_instance.expects(:exists?).returns(false)
-    File.expects(:open).never
-    assert build_search_index
-  end
-
-  def test_should_check_for_existing_index
-    SearchIndex.any_instance.expects(:exists?).returns(true)
-    SearchIndex.any_instance.expects(:load_record_size).returns(0)
-    assert build_search_index
-  end
   
-  def test_add_record
-    search_index = build_search_index
-    mock_record = mock(:id => 123)
-    mock_condensed_record = ['mock','condensed','record']
-    
-    search_index.expects(:condense_record).with(mock_record).returns(mock_condensed_record)
-    search_index.expects(:load_atoms).with(mock_condensed_record)
-    search_index.expects(:add_occurences).with(mock_condensed_record,123)
-    
-    search_index.add_record(mock_record)
-  end
-  
-  def test_add_records
-    search_index = build_search_index
-    mock_records = ['record0', 'record1']
-    
-    search_index.expects(:add_record).with('record0', true)
-    search_index.expects(:add_record).with('record1', true)
-    
-    search_index.add_records(mock_records)
-  end
+  # Write new tests for this since most of the existing tests were concerned
+  # with the storage routines which have now been moved elsewhere.
   
   private
   

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification 
 name: acts_as_indexed
 version: !ruby/object:Gem::Version 
-  hash: 11
+  hash: 9
   prerelease: false
   segments: 
   - 0
   - 6
-  - 6
-  version: 0.6.6
+  - 7
+  version: 0.6.7
 platform: ruby
 authors: 
 - Douglas F Shearer
@@ -15,7 +15,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2010-08-31 00:00:00 +01:00
+date: 2011-02-07 00:00:00 +00:00
 default_executable: 
 dependencies: []
 
@@ -41,6 +41,7 @@ files:
 - lib/acts_as_indexed/configuration.rb
 - lib/acts_as_indexed/search_atom.rb
 - lib/acts_as_indexed/search_index.rb
+- lib/acts_as_indexed/storage.rb
 - lib/will_paginate_search.rb
 - rails/init.rb
 - test/abstract_unit.rb