mongoid_search 0.3.2 → 0.4.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 4f3da97e4dd6f65400548a1e6f12e957fe0c2d5c8a390e015cd8d20572474c73
+  data.tar.gz: 71423ee292ac1f9e53cb4ef4171d4309c25ca56895203bd53c6bc2d8641fb385
+SHA512:
+  metadata.gz: 59b2511bea3017819e52c2633e8ac8cc19158e7da43042584b9fccc4f815b2a2e57fa4b1407e2ee5726d36710473f0cc9143b7b750586c802658869a0c6f7c23
+  data.tar.gz: 65bfb18a5361a333d815b92bafc34bbe5133c04b28c9e4540397605f98eb76123338119b2b5ee911b5074e85d68d84cc6547ab059845bd4698eb0b5beaaf2f02

data/LICENSE

@@ -1,4 +1,4 @@
-Copyright (c) 2009 Mauricio Zaffari
+Copyright (c) 2009 Mauricio Zaffari, 2013 Semenyuk Dmitriy (additions)
 
 Permission is hereby granted, free of charge, to any person obtaining
 a copy of this software and associated documentation files (the

data/README.md

@@ -1,168 +1,231 @@
-Mongoid Search
-============
+# Mongoid Search
 
-Mongoid Search is a simple full text search implementation for Mongoid ORM. It performs well for small data sets. If your searchable model is big (i.e. 1.000.000+ records), solr or sphinx may suit you better.
+Mongoid Search is a simple full text search implementation for Mongoid ORM. It supports Mongoid 3, 4, 5 and 6 and performs well for small data sets. If your searchable model is big (i.e. 1.000.000+ records), [mongoid_fulltext](https://github.com/mongoid/mongoid_fulltext), ElasticSearch, Solr or Sphinx may suit you better.
 
-Installation
---------
+[![Build Status](https://travis-ci.org/mongoid/mongoid_search.svg?branch=master)](https://travis-ci.org/mongoid/mongoid_search)
 
-In your Gemfile:
-
-    gem 'mongoid_search'
+## Installation
 
-If your project is still using mongoid 2.x.x, stick to mongoid_search 0.2.x:
+In your Gemfile:
 
-    gem 'mongoid_search', '~> 0.2.8'
+```ruby
+gem 'mongoid_search'
+```
 
 Then:
 
-    bundle install
-
-Examples
---------
-
-    class Product
-      include Mongoid::Document
-      include Mongoid::Search
-      field :brand
-      field :name
-
-      has_many   :tags
-      belongs_to :category
-
-      search_in :brand, :name, :tags => :name, :category => :name
-    end
-
-    class Tag
-      include Mongoid::Document
-      field :name
+```
+bundle install
+```
+
+## Examples
+
+```ruby
+class Product
+  include Mongoid::Document
+  include Mongoid::Search
+  field :brand
+  field :name
+  field :unit
+  field :info, type: Hash
+
+  has_many   :tags
+  belongs_to :category
+
+  search_in :brand, :name, tags: :name, category: :name, info: %i[summary description]
+  search_in :unit, index: :_unit_keywords
+end
+
+class Tag
+  include Mongoid::Document
+  field :name
+
+  belongs_to :product
+end
+
+class Category
+  include Mongoid::Document
+  field :name
+
+  has_many :products
+end
+```
+
+Now when you save a product, you get a `_keywords` field automatically:
+
+```ruby
+p = Product.new brand: 'Apple', name: 'iPhone', unit: 'kilogram', info: { summary: 'Info-summary', description: 'Info-description' }
+p.tags << Tag.new(name: 'Amazing')
+p.tags << Tag.new(name: 'Awesome')
+p.tags << Tag.new(name: 'Superb')
+p.save
+# => true
+p._keywords
+# => ["amazing", "apple", "awesome", "iphone", "superb", "Info-summary", "Info-description"]
+p._unit_keywords
+# => ["kilogram"]
+```
+
+Now you can run search, which will look in the `_keywords` field and return all matching results:
+
+```ruby
+Product.full_text_search("apple iphone").size
+# => 1
+```
+
+You can also search in "virtual" fields by defining them as methods. This can be useful when you have a method with dynamic fields (i.e. variable schema).
+```ruby
+class ModelWithDynamicFields
+  
+  ...
+  
+  search_in :search_data
+
+  def search_data
+    # concatenate all String fields' values
+    self.attributes.select{|k,v| v.is_a?(String) }.values.join(' ')
+  end
+end
+```
+Mongoid_search will run the method before save and use it's output to populate the `_keywords` field.
+
+Of course, some models could have more than one index. For instance, two different searches with different fields, so you could even specify from which index should be searched:
+
+```ruby
+Product.full_text_search("kilogram", index: :_unit_keywords).size
+# => 1
+```
 
-      belongs_to :product
-    end
-
-    class Category
-      include Mongoid::Document
-      field :name
-
-      has_many :products
-    end
-
-Now when you save a product, you get a _keywords field automatically:
-
-    p = Product.new :brand => "Apple", :name => "iPhone"
-    p.tags << Tag.new(:name => "Amazing")
-    p.tags << Tag.new(:name => "Awesome")
-    p.tags << Tag.new(:name => "Superb")
-    p.save
-    => true
-    p._keywords
-    => ["amazing", "apple", "awesome", "iphone", "superb"]
+Note that the search is case insensitive, and accept partial searching too:
 
-Now you can run search, which will look in the _keywords field and return all matching results:
+```ruby
+Product.full_text_search('ipho').size
+# => 1
+```
 
-    Product.full_text_search("apple iphone").size
-    => 1
+Assuming you have a category with multiple products you can use the following code to search for 'iphone' in products cheaper than $499.
 
-Note that the search is case insensitive, and accept partial searching too:
+```ruby
+category.products.where(:price.lt => 499).full_text_search('iphone').asc(:price)
+```
 
-    Product.full_text_search("ipho").size
-    => 1
+To index or reindex all existing records, run this rake task
 
-Assuming you have a category with multiple products you can use the following
-code to search for 'iphone' in products cheaper than $499
+```
+$ rake mongoid_search:index
+```
 
-    @category.products.where(:price.lt => 499).full_text_search('iphone').asc(:price)
+## Options
 
+### match
 
-Options
--------
+* `:any` - match any occurrence
+* `:all` - match all occurrences
 
-match:
+Default is `:any`.
 
-  _:any_ - match any occurrence
+```ruby
+Product.full_text_search('apple motorola', match: :any).size
+# => 1
 
-  _:all_ - match all ocurrences
+Product.full_text_search('apple motorola', match: :all).size
+# => 0
+```
 
-  Default is _:any_.
+### allow\_empty\_search
 
-    Product.full_text_search("apple motorola", match: :any).size
-    => 1
+* `true` - will return `Model.all`
+* `false` - will return `[]`
 
-    Product.full_text_search("apple motorola", match: :all).size
-    => 0
+Default is `false`.
 
-allow\_empty\_search:
+```ruby
+Product.full_text_search('', allow_empty_search: true).size
+# => 1
+```
 
-  _true_ - will return Model.all
+### relevant_search
 
-  _false_ - will return []
+* `true` - adds relevance information to the results
+* `false` - no relevance information
 
-  Default is _false_.
+Default is `false`.
 
-    Product.full_text_search("", allow_empty_search: true).size
-    => 1
+```ruby
+Product.full_text_search('amazing apple', relevant_search: true)
+# => [#<Product _id: 5016e7d16af54efe1c000001, _type: nil, brand: "Apple", name: "iPhone", attrs: nil, info: nil, category_id: nil, _keywords: ["amazing", "apple", "awesome", "iphone", "superb"], relevance: 2.0>]
+```
 
-relevant_search:
+Please note that relevant_search will return an Array and not a Criteria object. The search method should always be called in the end of the method chain.
 
-  _true_ - Adds relevance information to the results
+### index
 
-  _false_ - No relevance information
+Default is `_keywords`.
 
-  Default is _false_.
+```ruby
+Product.full_text_search('amazing apple', index: :_keywords)
+# => [#<Product _id: 5016e7d16af54efe1c000001, _type: nil, brand: "Apple", name: "iPhone", unit: "l", attrs: nil, info: nil, category_id: nil, _keywords: ["amazing", "apple", "awesome", "iphone", "superb"], _unit_keywords: ["l"], relevance: 2.0>]
 
-    Product.full_text_search('amazing apple', relevant_search: true)
-    => [#<Product _id: 5016e7d16af54efe1c000001, _type: nil, brand: "Apple", name: "iPhone", attrs: nil, info: nil, category_id: nil, _keywords: ["amazing", "apple", "awesome", "iphone", "superb"], relevance: 2.0>]
+Product.full_text_search('kg', index: :_unit_keywords)
+# => [#<Product _id: 5016e7d16af54efe1c000001, _type: nil, brand: "Apple", name: "iPhone", unit: "kg", attrs: nil, info: nil, category_id: nil, _keywords: ["amazing", "apple", "awesome", "iphone", "superb"], _unit_keywords: ["kg"], relevance: 2.0>]
+```
 
- Please note that relevant_search will return an Array and not a Criteria object. The search method shoud always be called in the end of the method chain.
+index enables to have two or more different searches, with different or same fields. It should be noted that indexes are exclusive per each one.
 
-Initializer
------------
+## Initializer
 
 Alternatively, you can create an initializer to setup those options:
 
-    Mongoid::Search.setup do |config|
-      ## Default matching type. Match :any or :all searched keywords
-      config.match = :any
+```ruby
+Mongoid::Search.setup do |config|
+  ## Default matching type. Match :any or :all searched keywords
+  config.match = :any
+
+  ## If true, an empty search will return all objects
+  config.allow_empty_search = false
 
-      ## If true, an empty search will return all objects
-      config.allow_empty_search = false
+  ## If true, will search with relevance information
+  config.relevant_search = false
 
-      ## If true, will search with relevance information
-      config.relevant_search = false
+  ## Stem keywords
+  config.stem_keywords = false
 
-      ## Stem keywords
-      config.stem_keywords = false
+  ## Add a custom proc returning strings to replace the default stemmer
+  # For example using ruby-stemmer:
+  # config.stem_proc = Proc.new { |word| Lingua.stemmer(word, :language => 'nl') }
 
-      ## Add a custom proc returning strings to replace the default stemmer
-      # For example using ruby-stemmer:
-      # config.stem_proc = Proc.new { |word| Lingua.stemmer(word, :language => 'nl') }
+  ## Words to ignore
+  config.ignore_list = []
 
-      ## Words to ignore
-      config.ignore_list = []
+  ## An array of words
+  # config.ignore_list = %w{ a an to from as }
 
-      ## An array of words
-      # config.ignore_list = %w{ a an to from as }
+  ## Or from a file
+  # config.ignore_list = YAML.load(File.open(File.dirname(__FILE__) + '/config/ignorelist.yml'))["ignorelist"]
 
-      ## Or from a file
-      # config.ignore_list = YAML.load(File.open(File.dirname(__FILE__) + '/config/ignorelist.yml'))["ignorelist"]
+  ## Search using regex (slower)
+  config.regex_search = true
 
-      ## Search using regex (slower)
-      config.regex_search = true
+  ## Regex to search
 
-      ## Regex to search
+  ## Match partial words on both sides (slower)
+  config.regex = Proc.new { |query| /#{query}/ }
 
-      ## Match partial words on both sides (slower)
-      config.regex = Proc.new { |query| /#{query}/ }
+  ## Match partial words on the beginning or in the end (slightly faster)
+  # config.regex = Proc.new { |query| /^#{query}/ }
+  # config.regex = Proc.new { |query| /#{query}$/ }
 
-      ## Match partial words on the beginning or in the end (slightly faster)
-      # config.regex = Proc.new { |query| /ˆ#{query}/ }
-      # config.regex = Proc.new { |query| /#{query}$/ }
+  # Ligatures to be replaced
+  # http://en.wikipedia.org/wiki/Typographic_ligature
+  config.ligatures = { "œ"=>"oe", "æ"=>"ae" }
 
-      # Ligatures to be replaced
-      # http://en.wikipedia.org/wiki/Typographic_ligature
-      config.ligatures = { "œ"=>"oe", "æ"=>"ae" }
+  # Strip symbols regex to be replaced. These symbols will be replaced by space
+  config.strip_symbols = /[._:;'\"`,?|+={}()!@#%^&*<>~\$\-\\\/\[\]]/
 
-      # Minimum word size. Words smaller than it won't be indexed
-      config.minimum_word_size = 2
-    end
+  # Strip accents regex to be replaced. These sybols will be removed after strip_symbols replacing
+  config.strip_accents = /[^\s\p{Alnum}]/
 
+  # Minimum word size. Words smaller than it won't be indexed
+  config.minimum_word_size = 2
+end
+```

data/Rakefile

@@ -3,8 +3,11 @@ require 'rake'
 
 require 'rspec/core/rake_task'
 RSpec::Core::RakeTask.new(:spec) do |spec|
-  spec.rspec_opts = ["-c", "-f progress"]
+  spec.rspec_opts = ['-c', '-f progress']
   spec.pattern = 'spec/**/*_spec.rb'
 end
 
-task :default => :spec
+require 'rubocop/rake_task'
+RuboCop::RakeTask.new(:rubocop)
+
+task default: %i[rubocop spec]

data/lib/mongoid_search.rb

@@ -1,7 +1,6 @@
-# encoding: utf-8
+require 'mongoid_search/mongoid_search'
 
 require 'mongoid_search/railtie' if defined?(Rails)
-require 'mongoid_search/mongoid_search'
 
 module Mongoid::Search
   ## Default matching type. Match :any or :all searched keywords
@@ -22,7 +21,7 @@ module Mongoid::Search
 
   ## Stem procedure
   mattr_accessor :stem_proc
-  @@stem_proc = Proc.new { |word| word.stem }
+  @@stem_proc = proc { |word| word.stem }
 
   ## Words to ignore
   mattr_accessor :ignore_list
@@ -42,25 +41,33 @@ module Mongoid::Search
   mattr_accessor :regex
 
   ## Match partial words on both sides (slower)
-  @@regex = Proc.new { |query| /#{query}/ }
+  @@regex = proc { |query| /#{query}/ }
 
   ## Match partial words on the beginning or in the end (slightly faster)
-  # @@regex = Proc.new { |query| /ˆ#{query}/ }
+  # @@regex = Proc.new { |query| /^#{query}/ }
   # @@regex = Proc.new { |query| /#{query}$/ }
 
   # Ligatures to be replaced
   # http://en.wikipedia.org/wiki/Typographic_ligature
   mattr_accessor :ligatures
-  @@ligatures = { "œ"=>"oe", "æ"=>"ae" }
+  @@ligatures = { 'œ' => 'oe', 'æ' => 'ae', 'ꜵ' => 'ao' }
 
   # Minimum word size. Words smaller than it won't be indexed
   mattr_accessor :minimum_word_size
   @@minimum_word_size = 2
 
+  # Strip special symbols
+  mattr_accessor :strip_symbols
+  @@strip_symbols = /[._:;'\"`,?|+={}()!@#%^&*<>~\$\-\\\/\[\]]/
+
+  # Strip accents
+  mattr_accessor :strip_accents
+  @@strip_accents = /[^\s\p{Alnum}]/
+
   def self.setup
     yield self
   end
 end
 
 require 'mongoid_search/util'
-require 'mongoid_search/log'
+require 'mongoid_search/log'

data/lib/mongoid_search/log.rb

@@ -14,7 +14,8 @@ class Mongoid::Search::Log
   end
 
   private
+
   def self.colorize(text, code)
     "\033[#{code}m#{text}\033[0m"
   end
-end
+end

data/lib/mongoid_search/mongoid_search.rb

@@ -1,11 +1,10 @@
 module Mongoid::Search
-  def self.included(base)
-    base.send(:cattr_accessor, :search_fields)
-
-    base.extend ClassMethods
+  extend ActiveSupport::Concern
 
+  included do
+    cattr_accessor :search_fields
     @@classes ||= []
-    @@classes << base
+    @@classes << self
   end
 
   def self.classes
@@ -16,17 +15,19 @@ module Mongoid::Search
     # Set a field or a number of fields as sources for search
     def search_in(*args)
       args, options = args_and_options(args)
-      self.search_fields = (self.search_fields || []).concat args
+      set_search_fields(options[:index], args)
 
-      field :_keywords, :type => Array
+      field options[:index], type: Array
 
-      index({ :_keywords => 1 }, { :background => true })
+      index({ options[:index] => 1 }, background: true)
 
       before_save :set_keywords
     end
 
-    def full_text_search(query, options={})
+    def full_text_search(query, options = {})
       options = extract_options(options)
+      attr_accessor :relevance if options[:relevant_search].eql? true
+
       return (options[:allow_empty_search] ? criteria.all : []) if query.blank?
 
       if options[:relevant_search]
@@ -43,33 +44,46 @@ module Mongoid::Search
     # Goes through all documents in the class that includes Mongoid::Search
     # and indexes the keywords.
     def index_keywords!
-      all.each { |d| d.index_keywords! ? Log.green(".") : Log.red("F") }
+      all.each { |d| d.index_keywords! ? Log.green('.') : Log.red('F') }
     end
 
     private
+
+    def set_search_fields(index, fields)
+      self.search_fields ||= {}
+
+      (self.search_fields[index] ||= []).concat fields
+    end
+
     def query(keywords, options)
       keywords_hash = keywords.map do |kw|
-        kw = Mongoid::Search.regex.call(kw) if Mongoid::Search.regex_search
-        { :_keywords => kw }
+        if Mongoid::Search.regex_search
+          escaped_kw = Regexp.escape(kw)
+          kw = Mongoid::Search.regex.call(escaped_kw)
+        end
+
+        { options[:index] => kw }
       end
 
-      criteria.send("#{(options[:match]).to_s}_of", *keywords_hash)
+      criteria.send("#{(options[:match])}_of", *keywords_hash)
     end
 
     def args_and_options(args)
       options = args.last.is_a?(Hash) &&
-                [:match,
-                 :allow_empty_search,
-                 :relevant_search].include?(args.last.keys.first) ? args.pop : {}
+        %i[match
+           allow_empty_search
+           index
+           relevant_search].include?(args.last.keys.first) ? args.pop : {}
 
       [args, extract_options(options)]
     end
 
     def extract_options(options)
       {
-        :match              => options[:match]              || Mongoid::Search.match,
-        :allow_empty_search => options[:allow_empty_search] || Mongoid::Search.allow_empty_search,
-        :relevant_search    => options[:relevant_search]    || Mongoid::Search.relevant_search
+        match: options[:match] || Mongoid::Search.match,
+        allow_empty_search: options[:allow_empty_search] || Mongoid::Search.allow_empty_search,
+        relevant_search: options[:relevant_search] || Mongoid::Search.relevant_search,
+        index: options[:index] || :_keywords
       }
     end
 
@@ -79,52 +93,57 @@ module Mongoid::Search
 
     def search_relevant(query, options)
       results_with_relevance(query, options).sort { |o| o['value'] }.map do |r|
-
-        new(r['_id'].merge(:relevance => r['value'])) do |o|
+        new(r['_id'].merge(relevance: r['value'])) do |o|
           # Need to match the actual object
           o.instance_variable_set('@new_record', false)
           o._id = r['_id']['_id']
         end
-
       end
     end
 
     def results_with_relevance(query, options)
       keywords = Mongoid::Search::Util.normalize_keywords(query)
 
-      map = %Q{
-        function() {
-          var entries = 0;
-          for(i in keywords) {
-            for(j in this._keywords) {
-              if(this._keywords[j] == keywords[i]) {
-                entries++;
+      map = %{
+          function() {
+            var entries = 0;
+            for(i in keywords) {
+              for(j in this.#{options[:index]}) {
+                if(this.#{options[:index]}[j] == keywords[i]) {
+                  entries++;
+                }
               }
             }
+            if(entries > 0) {
+              emit(this, entries);
+            }
           }
-          if(entries > 0) {
-            emit(this, entries);
-          }
-        }
       }
 
-      reduce = %Q{
-        function(key, values) {
-          return(values);
-        }
+      reduce = %{
+          function(key, values) {
+            return(values);
+          }
       }
 
-      query(keywords, options).map_reduce(map, reduce).scope(:keywords => keywords).out(:inline => 1)
+      query(keywords, options).map_reduce(map, reduce).scope(keywords: keywords).out(inline: 1)
     end
   end
 
   def index_keywords!
-    update_attribute(:_keywords, set_keywords)
+    search_fields.map do |index, fields|
+      update_attribute(index, get_keywords(fields))
+    end
   end
 
-  private
   def set_keywords
-    self._keywords = Mongoid::Search::Util.keywords(self, self.search_fields).
-      flatten.reject{|k| k.nil? || k.empty?}.uniq.sort
+    search_fields.each do |index, fields|
+      send("#{index}=", get_keywords(fields))
+    end
+  end
+
+  def get_keywords(fields)
+    Mongoid::Search::Util.keywords(self, fields)
+                         .flatten.reject { |k| k.nil? || k.empty? }.uniq.sort
   end
 end