mongoid-haystack 1.1.0 → 1.2.0

data/README.md

@@ -9,6 +9,30 @@ DESCRIPTION
   mongoid-haystack provides a zero-config, POLS, pure mongo, fulltext search
   solution for your mongoid models.
 
+INSTALL
+-------
+
+  rubygems: gem intstall 'mongoid-haystack'
+
+  Gemfile: gem 'mongoid-haystack'
+
+  rake db:mongoid:create_indexes  # IMPORTANT
+
+````ruby
+
+    # you might want this in lib/tasks/db.rake ...
+    #
+
+      namespace :db do
+        namespace :mongoid do
+          task :create_indexes do
+            Mongoid::Haystack.create_indexes
+          end
+        end
+      end
+
+````
+
 SYNOPSIS
 --------
 
@@ -29,6 +53,32 @@ SYNOPSIS
 
     article = results.first.model
 
+  # by default 'search' returns a Mongoid::Criteria object.  the result set will
+  # be full of objects that refer to a model in your app via a polymorphic
+  # relation out.  aka
+  #
+  #   Article.search('foobar').first.class       #=> Mongoid::Haystack::Index
+  #   Article.search('foobar').first.model.class #=> Article
+  #
+  # in an index view you are not going to want to expand the search index
+  # objects into full blown models one at the time (N+1) so you can use the
+  # 'models' method on the collection to effciently expand the collection into
+  # your application models with the fewest possible queries.  note that
+  # 'models' is a terminal operator.  that is to say it returns an array and,
+  # afterwards, no more fancy query language is gonna work.
+  #
+    @results =
+      Mongoid::Haystack.search('needle').models
+
+  # pagination is supported *out of the box*.  note that you should chain it
+  # *b4* any call to 'models' as 'models' is a terminal operator: it returns
+  # an array and *not* a Mongoid::Criteria object
+  #
+    @models = 
+      Mongoid::Haystack.search('needle').
+        paginate(:page => 3, :size => 42).
+          models
+
 
   # haystack stems the search terms and does score based sorting all using a
   # fast b-tree 
@@ -44,10 +94,11 @@ SYNOPSIS
     results == [a] #=> true
 
 
-  # cross models searching is supported out of the box, and models can
-  # customise how they are indexed:
+  # cross model searching (site search)is supported out of the box, and models
+  # can customise how they are indexed:
   #
   # - a global score lets some models appear hight in the global results
+  #
   # - keywords count more than fulltext 
   #
     class Article
@@ -89,19 +140,8 @@ SYNOPSIS
     models == [a1, a2]  #=> true. because keywords score highter than general fulltext
 
 
-  # by default searching returns Mongoid::Haystack::Index objects. you'll want
-  # to expand these results to the models they reference in your views, but
-  # avoid doing an N+1 query.  to do this simply call #models on the result set
-  # and the models will be eager loaded using only as many queries as their are
-  # model types in your result set
-  #
-
-    @results = Mongoid::Haystack.search('needle').page(params[:page]).per(10)
-    @models = @results.models
-
-
   # you can decorate your search items with arbirtrary meta data and filter
-  # searches by it later.  this too uses a b-tree index.
+  # searches by it later.  this too uses a speedy b-tree index.
   #
     class Article
       include Mongoid::Document
@@ -128,7 +168,8 @@ SYNOPSIS
         :content => 'seen the needles and the damage done...'
       )
 
-    author_articles = Article.search('needle', :facets => {:author_id => author.id})
+    articles_for_teh_author =
+      Article.search('needle', :facets => {:author_id => author.id})
 
 
 ````
@@ -136,14 +177,16 @@ SYNOPSIS
 DESCRIPTION
 -----------
 
-there two main pathways to understand in the code.  shit going into the
-index, and shit coming out.
+there two main pathways to understand in the code.
+
+1) shit going into the into the index.
+2) shit coming out of the index.
 
 shit going in entails:
 
-- stem and stopword the search terms.
+- stem and stopword the search terms
 - create or update a new token for each
-- create an index item reference all the tokens with precomputed scores
+- create an index item referening all the tokens with precomputed scores
 
 for example the terms 'dog dogs cat' might result in these tokens
 
@@ -166,7 +209,7 @@ for example the terms 'dog dogs cat' might result in these tokens
 
 ````
 
-  and this index item
+  being created|updated and this index item
 
 
 ````javascript
@@ -195,25 +238,34 @@ for example the terms 'dog dogs cat' might result in these tokens
 
 being built
 
-in addition, some other information is tracked such and the total number of
-search tokens every discovered in the corpus
+
+some other information is tracked, but the two normal mongoid models
+
+- Mongoid::Haystack::Token
+- Mongoid::Haystack::Index
+
+are simple to look at and compromise 80% of the library functionality.
   
 
 
 a few things to notice:
   
-- the tokens are counted and auto-id'd using hex notation and a sequence
-  generator.  the reason for this is so that their ids are legit hash keys
-  in the keyword and fulltext score hashes.
+- tokens are counted and auto-id'd using hex notation and a sequence
+generator.  the reason for this is so that their ids are legit hash keys in
+the keyword and fulltext score hashes (they are also smaller than 12 byte
+object_ids or the words themselves). aka this sort can be contructed:
+
+````ruby
+    order_by('keyword_scores.0x1' => :desc, 'keyword_scores.0x.1' => :desc)
+````
 
 - the data structure above allows both filtering for index items that have
-  certain tokens, but also ordering them based on global, keyword, and
-  fulltext score without resorting to map-reduce: a b-tree index can be
-  used.
+certain tokens, but also ordering them based on global, keyword, and fulltext
+score without resorting to map-reduce: a b-tree index can be used.
 
 - all tokens have their text/stem stored exactly once.  aka: we do not store
-  'hugewords' all over the place but store it once and count occurances of
-  it to keep the total index much smaller
+'hugewords' all over the place but store it once and count occurances of it to
+keep the total index much smaller
 
 
 

data/lib/mongoid-haystack/index.rb

@@ -27,9 +27,14 @@ module Mongoid
 
       class << Index
         def add(*args)
+        # we all one or more models to the index..
+        #
           models_for(*args) do |model|
             config = nil
 
+          # ask the model how it wants to be indexed.  if it does not know,
+          # guess.
+          #
             if model.respond_to?(:to_haystack)
               config = Map.for(model.to_haystack)
             else
@@ -56,41 +61,64 @@ module Mongoid
                 )
             end
 
+          # blow up if no sane config was produced
+          #
+            unless %w( keywords fulltext facets score ).detect{|key| config.has_key?(key)}
+              raise ArgumentError, "you need to defined #{ model }#to_haystack"
+            end
+
+          # parse the config
+          #
             keywords = Array(config[:keywords]).join(' ')
             fulltext = Array(config[:fulltext]).join(' ')
             facets   = Map.for(config[:facets] || {})
             score    = config[:score]
 
+          # find or create an index item for this model
+          #
             index =
               Haystack.find_or_create(
                 ->{ where(:model => model).first },
                 ->{ new(:model => model) },
               )
 
+          # if we are updating an index we need to decrement old token counts
+          # before updating it
+          #
             if index.persisted?
               Index.subtract(index)
             end
 
+          # add tokens for both keywords and fulltext.  increment counts for
+          # both.
+          #
             keyword_scores = Hash.new{|h,k| h[k] = 0}
             fulltext_scores = Hash.new{|h,k| h[k] = 0}
             token_ids = []
 
-            Token.values_for(keywords).each do |value|
-              token = Token.add(value)
+            values = Token.values_for(keywords)
+            tokens = Token.add(values)
+            token_index = tokens.inject({}){|hash, token| hash[token.value] = token; hash}
+            values.each do |value|
+              token = token_index.fetch(value)
               id = token.id
-
               token_ids.push(id)
               keyword_scores[id] += 1
             end
 
-            Token.values_for(fulltext).each do |value|
-              token = Token.add(value)
+            values = Token.values_for(fulltext)
+            tokens = Token.add(values)
+            token_index = tokens.inject({}){|hash, token| hash[token.value] = token; hash}
+            values.each do |value|
+              token = token_index.fetch(value)
               id = token.id
-
               token_ids.push(id)
               fulltext_scores[id] += 1
             end
 
+          # our index item is complete with list of tokens, counts of each
+          # one, and a facet hash for this model
+          #
             index.keyword_scores = keyword_scores
             index.fulltext_scores = fulltext_scores
 
@@ -113,19 +141,23 @@ module Mongoid
         def subtract(index)
           tokens = index.tokens
 
-          n = 0
+          counts = {}
 
           tokens.each do |token|
             keyword_score = index.keyword_scores[token.id].to_i
             fulltext_score = index.fulltext_scores[token.id].to_i
 
-            i = keyword_score + fulltext_score
-            token.inc(:count, -i)
+            count = keyword_score + fulltext_score
 
-            n += i
+            counts[count] ||= []
+            counts[count].push(token.id)
           end
 
-          Count[:tokens].inc(-n)
+          counts.each do |count, token_ids|
+            Token.where(:id.in => token_ids).inc(:count, -count)
+          end
+
+          tokens
         end
 
         def models_for(*args, &block)
@@ -147,13 +179,13 @@ module Mongoid
       belongs_to(:model, :polymorphic => true)
 
       has_and_belongs_to_many(:tokens, :class_name => '::Mongoid::Haystack::Token', :inverse_of => nil)
+
       field(:score, :type => Integer, :default => 0)
       field(:keyword_scores, :type => Hash, :default => proc{ Hash.new{|h,k| h[k] = 0} })
       field(:fulltext_scores, :type => Hash, :default => proc{ Hash.new{|h,k| h[k] = 0} })
       field(:facets, :type => Hash, :default => {})
 
-      index({:model_type => 1})
-      index({:model_id => 1})
+      index({:model_type => 1, :model_id => 1}, :unique => true)
 
       index({:token_ids => 1})
       index({:score => 1})

data/lib/mongoid-haystack/search.rb

@@ -1,5 +1,62 @@
 module Mongoid
   module Haystack
+    module Search
+      ClassMethods = proc do
+        def search(*args, &block)
+          options = Map.options_for!(args)
+          options[:types] = Array(options[:types]).flatten.compact
+          options[:types].push(self)
+          args.push(options)
+          results = Haystack.search(*args, &block)
+        end
+
+        def search_index_all!
+          all.each do |doc|
+            Mongoid::Haystack::Index.remove(doc)
+            Mongoid::Haystack::Index.add(doc)
+          end
+        end
+
+        after_save do |doc|
+          begin
+            doc.search_index! if doc.persisted?
+          rescue Object
+            nil
+          end
+        end
+
+        after_destroy do |doc|
+          begin
+            doc.search_unindex! if doc.destroyed?
+          rescue Object
+            nil
+          end
+        end
+
+        has_one(:haystack_index, :as => :model, :class_name => '::Mongoid::Haystack::Index')
+      end
+
+      InstanceMethods = proc do
+        def search_index!
+          doc = self
+          Mongoid::Haystack::Index.remove(doc)
+          Mongoid::Haystack::Index.add(doc)
+        end
+
+        def search_unindex!
+          doc = self
+          Mongoid::Haystack::Index.remove(doc)
+        end
+      end
+
+      def Search.included(other)
+        super
+      ensure
+        other.instance_eval(&ClassMethods)
+        other.class_eval(&InstanceMethods)
+      end
+    end
+
     def search(*args, &block)
     #
       options = Map.options_for!(args)
@@ -14,15 +71,15 @@ module Mongoid
       case
         when options[:all]
           op = :token_ids.all
-          search += Coerce.string(options[:all]) 
+          search += Coerce.string(options[:all])
 
         when options[:any]
           op = :token_ids.in
-          search += Coerce.string(options[:any]) 
+          search += Coerce.string(options[:any])
 
         when options[:in]
           op = :token_ids.in
-          search += Coerce.string(options[:in]) 
+          search += Coerce.string(options[:in])
       end
 
     #
@@ -55,114 +112,154 @@ module Mongoid
       end
 
     #
-      Index.where(conditions).order_by(order).tap do |results|
-        results.extend(Denormalize)
-      end
-    end
+      query =
+        Index.where(conditions)
+          .order_by(order)
+            .only(:_id, :model_type, :model_id)
 
-    def search_tokens_for(search)
-      values = Token.values_for(search.to_s)
-      tokens = Token.where(:value.in => values).to_a
-
-      positions = {}
-      tokens.each_with_index{|token, index| positions[token] = index + 1}
+      query.extend(Pagination)
 
-      t = Count[:tokens].value.to_f
+      query.extend(Denormalization)
 
-      tokens.sort! do |a,b|
-        [b.rarity_bin(t), positions[b]] <=> [a.rarity_bin(t), positions[a]]
-      end
-
-      tokens
+      query
     end
 
-    module Search
-      ClassMethods = proc do
-        def search(*args, &block)
-          options = Map.options_for!(args)
-          options[:types] = Array(options[:types]).flatten.compact
-          options[:types].push(self)
-          args.push(options)
-          results = Haystack.search(*args, &block)
-        end
+    module Pagination
+      def paginate(*args, &block)
+        list = self
+        options = Map.options_for!(args)
 
-        after_save do |doc|
-          begin
-            Mongoid::Haystack::Index.add(doc) if doc.persisted?
-          rescue Object
-            nil
+        page = Integer(args.shift || options[:page] || 1)
+        size = Integer(args.shift || options[:size] || 42)
+
+        count =
+          if list.is_a?(Array)
+            list.size
+          else
+            list.count
           end
-        end
 
-        after_destroy do |doc|
-          begin
-            Mongoid::Haystack::Index.remove(doc)
-          rescue Object
-            nil
+        limit = size
+        skip = (page - 1 ) * size
+        
+        result =
+          if list.is_a?(Array)
+            list.slice(skip, limit)
+          else
+            list.skip(skip).limit(limit)
           end
-        end
 
-        has_one(:haystack_index, :as => :model, :class_name => '::Mongoid::Haystack::Index')
+        result._paginated.update(
+          :total_pages  => (count / size.to_f).ceil,
+          :num_pages    => (count / size.to_f).ceil,
+          :current_page => page
+        )
+
+        result
       end
 
-      InstanceMethods = proc do
+      def _paginated
+        @_paginated ||= Map.new
       end
 
-      def Search.included(other)
-        super
-      ensure
-        other.instance_eval(&ClassMethods)
-        other.class_eval(&InstanceMethods)
+      def method_missing(method, *args, &block)
+        if respond_to?(:_paginated) and _paginated.has_key?(method) and args.empty? and block.nil?
+          _paginated[method]
+        else
+          super
+        end
       end
     end
 
-    module Denormalize
-      def denormalize
-        ::Mongoid::Haystack.denormalize(self)
-        self
+    module Denormalization
+      def models
+        Results.for(query = self)
       end
 
-      def models
-        denormalize
-        map(&:model)
+      def _denormalized
+        @_denormalized ||= (is_a?(Mongoid::Criteria) ? ::Mongoid::Haystack.denormalize(self) : self)
       end
+
+      class Results < ::Array
+        include ::Mongoid::Haystack::Pagination
+
+        attr_accessor :query
+
+        def Results.for(query)
+          Results.new.tap do |results|
+            results.query = query
+            results.replace(query._denormalized)
+            results._paginated.replace(query._paginated) rescue nil
+          end
+        end
+
+        def models
+          self
+        end
+      end
+    end
+
+    def search_tokens_for(search)
+      values = Token.values_for(search.to_s)
+      tokens = Token.where(:value.in => values).to_a
+
+      positions = {}
+      tokens.each_with_index{|token, index| positions[token] = index + 1}
+
+      total = Token.total.to_f
+
+      tokens.sort! do |a,b|
+        [b.rarity_bin(total), positions[b]] <=> [a.rarity_bin(total), positions[a]]
+      end
+
+      tokens
     end
 
     def Haystack.denormalize(results)
       queries = Hash.new{|h,k| h[k] = []}
 
-      results = results.to_a.flatten.compact
-
       results.each do |result|
         model_type = result[:model_type]
         model_id = result[:model_id]
-        model_class = model_type.constantize
+        model_class = eval(model_type) rescue next
         queries[model_class].push(model_id)
       end
 
+=begin
       index = Hash.new{|h,k| h[k] = {}}
-
-      queries.each do |model_class, model_ids|
-        models = 
-          begin
-            model_class.find(model_ids)
-          rescue Mongoid::Errors::DocumentNotFound
-            model_ids.map do |model_id|
-              begin
-                model_class.find(model_id)
-              rescue Mongoid::Errors::DocumentNotFound
-                nil
+=end
+
+      models =
+        queries.map do |model_class, model_ids|
+          model_class_models =
+            begin
+              model_class.find(model_ids)
+            rescue Mongoid::Errors::DocumentNotFound
+              model_ids.map do |model_id|
+                begin
+                  model_class.find(model_id)
+                rescue Mongoid::Errors::DocumentNotFound
+                  nil
+                end
               end
             end
+
+=begin
+          model_class_models.each do |model|
+            index[model.class.name] ||= Hash.new
+            next unless model
+            index[model.class.name][model.id.to_s] = model
           end
+=end
 
-        models.each do |model|
-          index[model.class.name] ||= Hash.new
-          next unless model
-          index[model.class.name][model.id.to_s] = model
+          model_class_models
         end
-      end
 
+      models.flatten!
+      models.compact!
+      models
+
+=begin
       to_ignore = []
 
       results.each_with_index do |result, i|
@@ -175,13 +272,24 @@ module Mongoid
           result.model = model
         end
 
-        result.model.freeze
-        result.freeze
+        result.model
+        result
+      end
+
+      to_ignore.reverse.each do |index|
+        models.delete_at(index)
       end
+=end
 
-      to_ignore.reverse.each{|i| results.delete_at(i)}
+      models
+    end
+
+    def Haystack.expand(*args, &block)
+      Haystack.denormalize(*args, &block)
+    end
 
-      results.to_a
+    def Haystack.models_for(*args, &block)
+      Haystack.denormalize(*args, &block)
     end
   end
 end

data/lib/mongoid-haystack/token.rb

@@ -9,17 +9,54 @@ module Mongoid
         end
 
         def add(value)
-          token =
-            Haystack.find_or_create(
-              ->{ where(:value => value).first },
-              ->{ create!(:value => value) }
-            )
+        # handle a value or array of values - which may contain dups
+        #
+          values = Array(value)
+          values.flatten!
+          values.compact!
 
-          token.inc(:count, 1)
+        # ensure that a token exists for each value seen
+        #
+          existing = where(:value.in => values)
+          missing = values - existing.map(&:value)
 
-          Count[:tokens].inc(1)
+          docs = missing.map{|value| {:_id => Token.next_hex_id, :value => value}}
+          unless docs.empty?
+            collection = mongo_session.with(:safe => false)[collection_name]
+            collection.insert(docs, [:continue_on_error])
+          end
 
-          token
+        # new we should have one token per uniq value
+        #
+          tokens = where(:value.in => values)
+
+        # batch update the counts on the tokens by the number of times each
+        # value was seen in the list
+        #
+        #   'dog dog' #=> increment the 'dog' token's count by 2
+        #
+          counts = {}
+          token_index = tokens.inject({}){|hash, token| hash[token.value] = token; hash}
+          value_index = values.inject({}){|hash, value| hash[value] ||= []; hash[value].push(value); hash}
+
+          values.each do |value|
+            token = token_index[value]
+            count = value_index[value].size
+            counts[count] ||= []
+            counts[count].push(token.id)
+          end
+
+          counts.each do |count, token_ids|
+            Token.where(:id.in => token_ids).inc(:count, count)
+          end
+
+        # return an array or single token depending on whether a list or
+        # single value was added
+        #
+          value.is_a?(Array) ? tokens : tokens.first
+        end
+
+        def subtract(tokens)
         end
 
         def sequence
@@ -29,6 +66,10 @@ module Mongoid
         def next_hex_id
           "0x#{ hex = sequence.next.to_s(16) }"
         end
+
+        def total
+          sum(:count)
+        end
       end
 
       field(:_id, :type => String, :default => proc{ Token.next_hex_id })
@@ -38,19 +79,19 @@ module Mongoid
       index({:value => 1}, {:unique => true})
       index({:count => 1})
 
-      def frequency(n_tokens = Count[:tokens].value.to_f)
+      def frequency(n_tokens = Token.total.value.to_f)
         (count / n_tokens).round(2)
       end
 
-      def frequency_bin(n_tokens = Count[:tokens].value.to_f)
+      def frequency_bin(n_tokens = Token.total.value.to_f)
         (frequency(n_tokens) * 10).truncate
       end
 
-      def rarity(n_tokens = Count[:tokens].value.to_f)
+      def rarity(n_tokens = Token.total.value.to_f)
         ((n_tokens - count) / n_tokens).round(2)
       end
 
-      def rarity_bin(n_tokens = Count[:tokens].value.to_f)
+      def rarity_bin(n_tokens = Token.total.value.to_f)
         (rarity(n_tokens) * 10).truncate
       end
     end

data/lib/mongoid-haystack/util.rb

@@ -5,7 +5,6 @@ module Mongoid
         [
           Mongoid::Haystack::Token,
           Mongoid::Haystack::Index,
-          Mongoid::Haystack::Count,
           Mongoid::Haystack::Sequence
         ]
       end
@@ -29,6 +28,10 @@ module Mongoid
         end
       end
 
+      def create_indexes
+        models.each{|model| model.create_indexes}
+      end
+
       def destroy_all
         models.map{|model| model.destroy_all}
       end

data/lib/mongoid-haystack.rb

@@ -2,7 +2,7 @@
 #
   module Mongoid
     module Haystack
-      const_set :Version, '1.1.0'
+      const_set :Version, '1.2.0'
 
       class << Haystack
         def version
@@ -11,11 +11,13 @@
 
         def dependencies
           {
-            'mongoid'       => [ 'mongoid'       , '~> 3.0'   ] , 
-            'map'           => [ 'map'           , '~> 6.2'   ] , 
-            'fattr'         => [ 'fattr'         , '~> 2.2'   ] , 
-            'coerce'        => [ 'coerce'        , '~> 0.0.3' ] , 
-            'unicode_utils' => [ 'unicode_utils' , '~> 1.4.0' ] , 
+            'mongoid'       => [ 'mongoid'       , '~> 3.0.14' ] , 
+            'moped'         => [ 'moped'         , '~> 1.3.1'  ] , 
+            'origin'        => [ 'origin'        , '~> 1.0.11' ] , 
+            'map'           => [ 'map'           , '~> 6.2'    ] , 
+            'fattr'         => [ 'fattr'         , '~> 2.2'    ] , 
+            'coerce'        => [ 'coerce'        , '~> 0.0.3'  ] , 
+            'unicode_utils' => [ 'unicode_utils' , '~> 1.4.0'  ] , 
           }
         end
 
@@ -73,7 +75,6 @@
 
       load Haystack.libdir('stemming.rb')
       load Haystack.libdir('util.rb')
-      load Haystack.libdir('count.rb')
       load Haystack.libdir('sequence.rb')
       load Haystack.libdir('token.rb')
       load Haystack.libdir('index.rb')
@@ -86,3 +87,11 @@
       extend Haystack
     end
   end
+
+##
+#
+  if defined?(Rails)
+    class Mongoid::Haystack::Engine < Rails::Engine
+      paths['app/models'] = File.dirname(__FILE__)
+    end
+  end

data/mongoid-haystack.gemspec

@@ -3,7 +3,7 @@
 
 Gem::Specification::new do |spec|
   spec.name = "mongoid-haystack"
-  spec.version = "1.1.0"
+  spec.version = "1.2.0"
   spec.platform = Gem::Platform::RUBY
   spec.summary = "mongoid-haystack"
   spec.description = "a mongoid 3 zero-config, zero-integration, POLS pure mongo fulltext solution"
@@ -16,13 +16,11 @@ Gem::Specification::new do |spec|
  "lib/app/models",
  "lib/app/models/mongoid",
  "lib/app/models/mongoid/haystack",
- "lib/app/models/mongoid/haystack/count.rb",
  "lib/app/models/mongoid/haystack/index.rb",
  "lib/app/models/mongoid/haystack/sequence.rb",
  "lib/app/models/mongoid/haystack/token.rb",
  "lib/mongoid-haystack",
  "lib/mongoid-haystack.rb",
- "lib/mongoid-haystack/count.rb",
  "lib/mongoid-haystack/index.rb",
  "lib/mongoid-haystack/search.rb",
  "lib/mongoid-haystack/sequence.rb",
@@ -58,7 +56,11 @@ Gem::Specification::new do |spec|
   spec.test_files = nil
 
   
-    spec.add_dependency(*["mongoid", "~> 3.0"])
+    spec.add_dependency(*["mongoid", "~> 3.0.14"])
+  
+    spec.add_dependency(*["moped", "~> 1.3.1"])
+  
+    spec.add_dependency(*["origin", "~> 1.0.11"])
   
     spec.add_dependency(*["map", "~> 6.2"])
   

data/test/mongoid-haystack_test.rb

@@ -14,6 +14,17 @@ Testing Mongoid::Haystack do
     assert{ Mongoid::Haystack.search('cat').map(&:model) == [b] }
   end
 
+##
+#
+  testing 'that results are returned as chainable Mongoid::Criteria' do
+     k = new_klass
+
+     3.times{ k.create! :content => 'cats' }
+
+     results = assert{ Mongoid::Haystack.search('cat') }
+     assert{ results.is_a?(Mongoid::Criteria) }
+  end
+
 ##
 #
   testing 'that word occurance affects the sort' do
@@ -67,7 +78,7 @@ Testing Mongoid::Haystack do
 
     assert{ Mongoid::Haystack::Token.count == 2 }
     assert{ Mongoid::Haystack::Token.all.map(&:value).sort == %w( cat dog ) }
-    assert{ Mongoid::Haystack::Count[:tokens].value == 3 }
+    assert{ Mongoid::Haystack::Token.total == 3 }
   end
 
   testing 'that removing a model from the index decrements counts appropriately' do
@@ -81,27 +92,27 @@ Testing Mongoid::Haystack do
 
     assert{ Mongoid::Haystack::Token.where(:value => 'cat').first.count == 2 }
     assert{ Mongoid::Haystack::Token.where(:value => 'dog').first.count == 2 }
-    assert{ Mongoid::Haystack::Count[:tokens].value == 4 }
+    assert{ Mongoid::Haystack::Token.total == 4 }
     assert{ Mongoid::Haystack::Token.all.map(&:value).sort == %w( cat dog ) }
     assert{ Mongoid::Haystack.unindex(c) }
     assert{ Mongoid::Haystack::Token.all.map(&:value).sort == %w( cat dog ) }
-    assert{ Mongoid::Haystack::Count[:tokens].value == 2 }
+    assert{ Mongoid::Haystack::Token.total == 2 }
     assert{ Mongoid::Haystack::Token.where(:value => 'cat').first.count == 1 }
     assert{ Mongoid::Haystack::Token.where(:value => 'dog').first.count == 1 }
 
-    assert{ Mongoid::Haystack::Count[:tokens].value == 2 }
+    assert{ Mongoid::Haystack::Token.total == 2 }
     assert{ Mongoid::Haystack::Token.all.map(&:value).sort == %w( cat dog ) }
     assert{ Mongoid::Haystack.unindex(b) }
     assert{ Mongoid::Haystack::Token.all.map(&:value).sort == %w( cat dog ) }
-    assert{ Mongoid::Haystack::Count[:tokens].value == 1 }
+    assert{ Mongoid::Haystack::Token.total == 1 }
     assert{ Mongoid::Haystack::Token.where(:value => 'cat').first.count == 0 }
     assert{ Mongoid::Haystack::Token.where(:value => 'dog').first.count == 1 }
 
-    assert{ Mongoid::Haystack::Count[:tokens].value == 1 }
+    assert{ Mongoid::Haystack::Token.total == 1 }
     assert{ Mongoid::Haystack::Token.all.map(&:value).sort == %w( cat dog ) }
     assert{ Mongoid::Haystack.unindex(a) }
     assert{ Mongoid::Haystack::Token.all.map(&:value).sort == %w( cat dog ) }
-    assert{ Mongoid::Haystack::Count[:tokens].value == 0 }
+    assert{ Mongoid::Haystack::Token.total == 0 }
     assert{ Mongoid::Haystack::Token.where(:value => 'cat').first.count == 0 }
     assert{ Mongoid::Haystack::Token.where(:value => 'dog').first.count == 0 }
   end
@@ -235,20 +246,119 @@ Testing Mongoid::Haystack do
     assert{ Mongoid::Haystack.search('dog').first.model == b }
   end
 
+##
+#
+  testing 'that re-indexing a class is idempotent' do
+    k = new_klass do
+      field(:title)
+      field(:body)
+
+      def to_haystack
+        { :keywords => title, :fulltext => body }
+      end
+    end
+
+    n = 10
+
+    n.times do
+      k.create!(:title => 'the cats and dogs', :body => 'now now is is the the time time for for all all good good men women')
+    end
+
+    n.times do
+      k.create!(:title => 'a b c abc xyz abc xyz b', :body => 'pdq pdq pdq xyz teh ngr am')
+    end
+
+    assert{ Mongoid::Haystack.search('cat').count == n }
+    assert{ Mongoid::Haystack.search('pdq').count == n }
+
+    ca = Mongoid::Haystack::Token.all.inject({}){|hash, token| hash.update token.id => token.value}
+
+    assert{ k.search_index_all! }
+
+    cb = Mongoid::Haystack::Token.all.inject({}){|hash, token| hash.update token.id => token.value}
+
+    assert{ ca.size == Mongoid::Haystack::Token.count }
+    assert{ cb.size == Mongoid::Haystack::Token.count }
+    assert{ ca == cb }
+  end
+
+##
+#
+   testing 'that not just any model can be indexed' do
+     o = new_klass.create!
+     assert{ begin; Mongoid::Haystack::Index.add(o); rescue Object => e; e.is_a?(ArgumentError); end }
+   end
+
+##
+#
+  testing 'that results can be expanded efficiently if need be' do
+     k = new_klass
+     3.times{ k.create! :content => 'cats' }
+
+     results = assert{ Mongoid::Haystack.search('cat') }
+     assert{ Mongoid::Haystack.models_for(results).map{|model| model.class} == [k, k, k] }
+  end
+
+##
+#
+  testing 'basic pagination' do
+     k = new_klass
+     11.times{|i| k.create! :content => "cats #{ i }" }
+
+     assert{ k.search('cat').paginate(:page => 1, :size => 2).to_a.size == 2 }
+     assert{ k.search('cat').paginate(:page => 2, :size => 5).to_a.size == 5 }
+
+     accum = []
+
+     n = 6
+     size = 2
+     (1..n).each do |page|
+       list = assert{ k.search('cat').paginate(:page => page, :size => size) }
+       accum.push(*list)
+       assert{ list.num_pages == n }
+       assert{ list.total_pages == n }
+       assert{ list.current_page == page }
+     end
+
+     a = accum.map{|i| i.model}.sort_by{|m| m.content}
+     b = k.all.sort_by{|m| m.content}
+
+     assert{ a == b }
+  end
+
+##
+#
+  testing 'that pagination preserves the #model terminator' do
+     k = new_klass
+     11.times{|i| k.create! :content => "cats #{ i }" }
+
+     list = assert{ k.search('cat').paginate(:page => 1, :size => 2) }
+     assert{ list.is_a?(Mongoid::Criteria) }
+
+     models = assert{ list.models }
+     assert{ models.is_a?(Array) }
+  end
+
 protected
 
   def new_klass(&block)
-    Object.send(:remove_const, :K) if Object.send(:const_defined?, :K)
+    if Object.send(:const_defined?, :K)
+      Object.const_get(:K).destroy_all
+      Object.send(:remove_const, :K)
+    end
 
     k = Class.new(A) do
       self.default_collection_name = :ks
       def self.name() 'K' end
-      include ::Mongoid::Haystack::Search
-      class_eval(&block) if block
     end
 
     Object.const_set(:K, k)
 
+    k.class_eval do
+      include ::Mongoid::Haystack::Search
+      class_eval(&block) if block
+    end
+
     k
   end
 
@@ -260,4 +370,6 @@ protected
     [A, B, C].map{|m| m.destroy_all}
     Mongoid::Haystack.destroy_all
   end
+
+  at_exit{ K.destroy_all if defined?(K) }
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: mongoid-haystack
 version: !ruby/object:Gem::Version
-  version: 1.1.0
+  version: 1.2.0
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-12-06 00:00:00.000000000 Z
+date: 2012-12-11 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: mongoid
@@ -18,7 +18,7 @@ dependencies:
     requirements:
     - - ~>
       - !ruby/object:Gem::Version
-        version: '3.0'
+        version: 3.0.14
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
@@ -26,7 +26,39 @@ dependencies:
     requirements:
     - - ~>
       - !ruby/object:Gem::Version
-        version: '3.0'
+        version: 3.0.14
+- !ruby/object:Gem::Dependency
+  name: moped
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 1.3.1
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 1.3.1
+- !ruby/object:Gem::Dependency
+  name: origin
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 1.0.11
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 1.0.11
 - !ruby/object:Gem::Dependency
   name: map
   requirement: !ruby/object:Gem::Requirement
@@ -99,12 +131,10 @@ extra_rdoc_files: []
 files:
 - README.md
 - Rakefile
-- lib/app/models/mongoid/haystack/count.rb
 - lib/app/models/mongoid/haystack/index.rb
 - lib/app/models/mongoid/haystack/sequence.rb
 - lib/app/models/mongoid/haystack/token.rb
 - lib/mongoid-haystack.rb
-- lib/mongoid-haystack/count.rb
 - lib/mongoid-haystack/index.rb
 - lib/mongoid-haystack/search.rb
 - lib/mongoid-haystack/sequence.rb

data/lib/app/models/mongoid/haystack/count.rb

@@ -1 +0,0 @@
-Mongoid::Haystack::Count

data/lib/mongoid-haystack/count.rb

@@ -1,28 +0,0 @@
-module Mongoid
-  module Haystack
-    class Count
-      include Mongoid::Document
-
-      field(:name, :type => String)
-      field(:value, :type => Integer, :default => 0)
-
-      index({:name => 1}, {:unique => true})
-      index({:value => 1})
-
-      def Count.for(name)
-        Haystack.find_or_create(
-          ->{ where(:name => name.to_s).first },
-          ->{ create!(:name => name.to_s) }
-        )
-      end
-
-      def Count.[](name)
-        Count.for(name)
-      end
-
-      def inc(n = 1)
-        super(:value, n)
-      end
-    end
-  end
-end