pose 3.0.0 → 3.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: b5795ce133c87ea0c6f0841d857b1e106bd527d1
-  data.tar.gz: e7404bd3671c5c12df5a190df2ce48699d018efc
+  metadata.gz: 1059e7c72d577be6f7d5f3c0fc4fdc30b429bd4b
+  data.tar.gz: d0ed2c994af2326ec73a779b0fb61a44dca2602b
 SHA512:
-  metadata.gz: ba4cdcc4bb8680bff841c706b96e1d252fec59e5394c69260dd831d9366c8a07f2d9c3d16f69c2ceb4bdc6e43ef8a4ec757fe214a3a3abfc80ada389cdea721a
-  data.tar.gz: 9415d0ece9e8984aeba669647b83e7fc7be78b3ff5bf827888a0f9851e48defc66b4a13b6703f29f19b3cbe4da7e6bb1259a2baee7a9f13ebe610e96e437733d
+  metadata.gz: a9eb63377caee2f07d3e7c9118ee9715d9e386371ca766aeb35b98eabe8d76f2d2128a27188ce71dd5f05c61e8bc46dc9726a351278392fc478dcb38aeeb44f6
+  data.tar.gz: 28e599e762c76d35070e5d81c71c9b7069270dbd3f1da463db8eb1f6f546830062da027e6eb47a794d85efa475680f66052e7c8373462bbcbdb9b9b0bbd2d37f

data/README.md

@@ -1,19 +1,28 @@
 # POlymorphic SEarch <a href="http://travis-ci.org/#!/kevgo/pose" target="_blank"><img src="https://secure.travis-ci.org/kevgo/pose.png" alt="Build status"></a> [![Code Climate](https://codeclimate.com/github/kevgo/pose.png)](https://codeclimate.com/github/kevgo/pose) [![Coverage Status](https://coveralls.io/repos/kevgo/pose/badge.png?branch=master)](https://coveralls.io/r/kevgo/pose) [![Dependency Status](https://gemnasium.com/kevgo/pose.png)](https://gemnasium.com/kevgo/pose)
 
-A database agnostic fulltext search engine for ActiveRecord objects in Ruby on Rails.
+A database agnostic fulltext search engine for ActiveRecord objects. See also [Pose Rails Adapter](http://github.com/einzige/pose-rails).
 
 * Searches over several classes at once.
 * The searchable content of each class and document can be freely customized.
-* Uses the main Rails database - no separate servers, databases, or search engines required.
+* Uses application database - no separate servers, databases, or search engines required.
 * Does not pollute the searchable classes nor their database tables.
 * Very fast search, doing only simple queries over fully indexed columns.
 * Allows to augment the fulltext search query with your own joins and where clauses.
 
-Version 3.x is for Rails 4 compatibilty. For Rails 3.x, please use versions 2.x.
-
 
 ## Installation
 
+### Versions
+
+* __2.x__ for _Rails 3.x_ compatibilty
+* __3.0__ for _Rails 4_ compatibilty
+* __3.1__ introduces a new setup. The _pose_ gem is now a generic Ruby gem that works
+    with any Ruby web server that uses ActiveRecord, like [Sinatra](http://www.sinatrarb.com),
+    [Padrino](http://www.padrinorb.com), or [Rails](http://rubyonrails.org).
+    Generators for installation and uninstallation are extracted into the
+    [pose_rails](https://github.com/po-se/pose-rails) gem.
+
+
 ### Set up the gem.
 
 Add the gem to your Gemfile and run `bundle install`
@@ -25,8 +34,7 @@ gem 'pose'
 ### Create the database tables for pose.
 
 ```bash
-$ rails generate pose:install
-$ rake db:migrate
+$ rake pose:install
 ```
 
 Pose creates two tables in your database. These tables are automatically populated and kept up to date.
@@ -37,11 +45,32 @@ Pose creates two tables in your database. These tables are automatically populat
 
 ### Make your ActiveRecord models searchable
 
+Each model defines the searchable content through the `posify` method.
+Valid parameters are
+* names of __fields__
+* names of __methods__
+* a __block__
+
+The value/result of each parameter is added to the search index for this instance.
+Here are a few examples:
+
+```ruby
+class User < ActiveRecord::Base
+  # first_name and last_name are attributes
+
+  # This line makes the class searchable.
+  posify :first_name, :last_name, :address
+
+  def address
+    "#{street} #{city} #{state}"
+  end
+end
+```
+
 ```ruby
 class MyClass < ActiveRecord::Base
 
-  # This line makes your class searchable.
-  # The given block must return the searchable content as a string.
+  # This line makes the class searchable.
   posify do
 
     # Only active instances should show up in search results.
@@ -55,6 +84,7 @@ class MyClass < ActiveRecord::Base
 end
 ```
 
+You can mix and match all parameter types for `posify` at will.
 Note that you can return whatever content you want in the `posify` block,
 not only data from this object, but also data from related objects, class names, etc.
 
@@ -241,23 +271,16 @@ Or, clone the repository, make your changes, and submit a unit-tested pull reque
 
 ### Run the unit tests for the Pose Gem
 
-Pose uses Sqlite3 for tests.
-To run tests, first create a test database.
-
-```bash
-bundle
-rake app:db:create
-rake app:db:migrate
-rake app:db:test:prepare
-```
+Pose can work with Sqlite3, Postgesql and MySQL by default.
+To run tests, please create database configuration file `spec/support/config/database.yml`, please refer to the template:
+[spec/support/config/database.yml.example](spec/support/config/database.yml.example)
 
 Then run the tests.
 
 ```bash
-rake
+bundle exec rake test
 ```
 
-
 ### Road Map
 
 * pagination of search results

data/Rakefile

@@ -1,16 +1,48 @@
 #!/usr/bin/env rake
+
 begin
   require 'bundler/setup'
 rescue LoadError
   puts 'You must `gem install bundler` and `bundle install` to run rake tasks'
 end
 
-APP_RAKEFILE = File.expand_path("../spec/dummy/Rakefile", __FILE__)
-load 'rails/tasks/engine.rake'
-
 Bundler::GemHelper.install_tasks
 
 # RSpec tasks.
 require 'rspec/core/rake_task'
-RSpec::Core::RakeTask.new :spec
-task :default => :spec
+
+RSpec::Core::RakeTask.new :spec_mysql do
+  ENV['POSE_ENV'] = 'mysql'
+end
+
+RSpec::Core::RakeTask.new :spec_mysql2 do
+  ENV['POSE_ENV'] = 'mysql2'
+end
+
+RSpec::Core::RakeTask.new :spec_sqlite do
+  ENV['POSE_ENV'] = 'sqlite'
+end
+
+RSpec::Core::RakeTask.new :spec_postgres do
+  ENV['POSE_ENV'] = 'postgres'
+end
+
+RSpec::Core::RakeTask.new :spec_mysql_ci do
+  ENV['POSE_ENV'] = 'mysql_ci'
+end
+
+RSpec::Core::RakeTask.new :spec_mysql2_ci do
+  ENV['POSE_ENV'] = 'mysql2_ci'
+end
+
+RSpec::Core::RakeTask.new :spec_sqlite_ci do
+  ENV['POSE_ENV'] = 'sqlite_ci'
+end
+
+RSpec::Core::RakeTask.new :spec_postgres_ci do
+  ENV['POSE_ENV'] = 'postgres_ci'
+end
+
+task :test_ci => [:spec_sqlite_ci, :spec_postgres_ci, :spec_mysql2_ci]
+task :test    => [:spec_sqlite,    :spec_postgres,    :spec_mysql2]
+task :default => :test_ci

data/lib/pose/activerecord_base_additions.rb

@@ -3,13 +3,14 @@ module Pose
   module ActiveRecordBaseAdditions
 
     # Defines the searchable content in ActiveRecord objects.
-    def posify &block
-      raise "Error while posifying class '#{name}': " \
-            "You must provide a block that returns the searchable content to 'posify'." unless block_given?
-
+    def posify *source_methods, &block
       include ModelClassAdditions
-      self.pose_content = block
-    end
 
+      self.pose_content = proc do
+        text_chunks = source_methods.map { |source| send(source) }
+        text_chunks << instance_eval(&block) if block
+        text_chunks.reject(&:blank?).join(' ')
+      end
+    end
   end
 end

data/{app/models → lib}/pose/assignment.rb

@@ -1,17 +1,22 @@
 # Assigns searchable objects to words in the search index.
 module Pose
   class Assignment < ActiveRecord::Base
+    self.table_name_prefix = 'pose_'
+
     belongs_to :word, class_name: 'Pose::Word'
     belongs_to :posable, polymorphic: true
 
     # Removes all Assignments for the given class.
+    # Returns a number for removed records.
+    # @param [Class] clazz
+    # @return [Integer]
     def self.delete_class_index clazz
       Assignment.delete_all(posable_type: clazz.name)
     end
 
     # Removes all Assignments that aren't used anymore.
     def self.cleanup_orphaned_pose_assignments progress_bar = nil
-      Assignment.find_each(include: [:posable, :word], batch_size: 5000) do |assignment|
+      Assignment.includes([:posable, :word]).find_each(batch_size: 5000) do |assignment|
         progress_bar.increment if progress_bar
 
         # Delete the assignment if the posable object no longer exists.

data/lib/pose/jobs/install.rb

@@ -0,0 +1,27 @@
+module Pose
+  module Jobs
+    class InitialMigration < ActiveRecord::Migration
+      def change
+        create_table "pose_assignments" do |t|
+          t.integer "word_id",                 null: false
+          t.integer "posable_id",              null: false
+          t.string  "posable_type", limit: 60, null: false
+        end
+        add_index "pose_assignments", :word_id
+        add_index "pose_assignments", :posable_id
+
+        create_table "pose_words" do |t|
+          t.string "text", limit: 80, null: false
+        end
+        add_index "pose_words", :text
+      end
+    end
+
+    # Creates the Pose tables in the database.
+    class Install
+      def perform
+        InitialMigration.new.change
+      end
+    end
+  end
+end

data/lib/pose/jobs/reindex_all.rb

@@ -0,0 +1,34 @@
+require 'ruby-progressbar'
+
+module Pose
+  module Jobs
+
+    # Recreates the Pose search index for the given class from scratch.
+    class ReindexAll
+      attr_reader :klass
+
+      # @param [String, Class] clazz
+      def initialize(clazz)
+        @klass = case clazz
+                   when String
+                     clazz.constantize
+                   when Class
+                     clazz
+                   else
+                     raise ArgumentError, "Class or String expected, #{clazz.class} given"
+                 end
+      end
+
+      def perform
+        progress_bar = ProgressBar.create title: "  reindexing", total: klass.count
+
+        klass.find_each do |instance|
+          instance.update_pose_words
+          progress_bar.increment
+        end
+
+        progress_bar.finish
+      end
+    end
+  end
+end

data/lib/pose/jobs/remove.rb

@@ -0,0 +1,28 @@
+require 'ruby-progressbar'
+
+module Pose
+  module Jobs
+
+    # Removes the Pose search index for the given class.
+    class Remove
+      attr_reader :klass
+
+      # @param [String, Class] clazz
+      def initialize(clazz)
+        @klass = case clazz
+                   when String
+                     clazz.constantize
+                   when Class
+                     clazz
+                   else
+                     raise ArgumentError, "Class or String expected, #{clazz.class} given"
+                 end
+      end
+
+      def perform
+        Pose::Assignment.delete_class_index(klass)
+        puts "Search index for class #{klass.name} deleted.\n\n"
+      end
+    end
+  end
+end

data/lib/pose/jobs/uninstall.rb

@@ -0,0 +1,19 @@
+require 'pose/jobs/install'
+
+module Pose
+  module Jobs
+
+    # Removes the Pose database tables.
+    class KillMigration < ActiveRecord::Migration
+      def change
+        revert InitialMigration
+      end
+    end
+
+    class Uninstall
+      def perform
+        KillMigration.new.change
+      end
+    end
+  end
+end

data/lib/pose/jobs/vacuum.rb

@@ -0,0 +1,23 @@
+require 'ruby-progressbar'
+
+module Pose
+  module Jobs
+
+    # Cleans up unused data in the search index.
+    class Vacuum
+      def perform
+        puts "Cleaning Pose search index...\n\n"
+
+        progress_bar = ProgressBar.create title: '  assignments', total: Pose::Assignment.count
+        Pose::Assignment.cleanup_orphaned_pose_assignments progress_bar
+        progress_bar.finish
+
+        progress_bar = ProgressBar.create title: '  words', total: Pose::Word.count
+        Pose::Word.remove_unused_words progress_bar
+        progress_bar.finish
+
+        puts "\nPose search index cleanup complete.\n\n"
+      end
+    end
+  end
+end

data/lib/pose/model_class_additions.rb

@@ -13,6 +13,36 @@ module Pose
       cattr_accessor :pose_content
     end
 
+    # Returns all words in the search index for this instance.
+    # @return [Array<String>]
+    def pose_current_words
+      pose_words.map(&:text)
+    end
+
+    # Returns the searchable text snippet for this instance.
+    # This data is not stored in the search engine.
+    # It is recomputes this from data in the database here.
+    # @return [String]
+    def pose_fetch_content
+      instance_eval(&(pose_content)).to_s
+    end
+
+    # @return [Array<String>]
+    def pose_fresh_words reload = false
+      @pose_fresh_words = nil if reload
+      @pose_fresh_words ||= Query.query_words pose_fetch_content
+    end
+
+    # @return [Array<String>]
+    def pose_stale_words reload = false
+      pose_current_words - pose_fresh_words(reload)
+    end
+
+    # @return [Array<String>]
+    def pose_words_to_add reload = false
+      pose_fresh_words(reload) - pose_current_words
+    end
+
     # Updates the associated words for this object in the database.
     def update_pose_index
       update_pose_words if Pose.perform_search?
@@ -26,20 +56,8 @@ module Pose
     # Helper method.
     # Updates the search words with the text returned by search_strings.
     def update_pose_words
-
-      # Step 1: get an array of all words for the current object.
-      search_text = instance_eval &(self.class.pose_content)
-      new_words = Query.new([], search_text.to_s).query_words
-
-      # Step 2: Add new words to the search index.
-      Helpers.get_words_to_add(self.pose_words, new_words).each do |word_to_add|
-        self.pose_words << Word.find_or_create_by(text: word_to_add)
-      end
-
-      # Step 3: Remove now obsolete words from search index.
-      Helpers.get_words_to_remove(self.pose_words, new_words).each do |word_to_remove|
-        self.pose_words.delete word_to_remove
-      end
+      self.pose_words.delete(Word.factory(pose_stale_words true))
+      self.pose_words << Word.factory(pose_words_to_add)
     end
   end
 end

data/lib/pose/query.rb

@@ -1,4 +1,5 @@
 module Pose
+
   # Represents a search query.
   #
   # Provides convenient access to all elements of the search query:
@@ -12,7 +13,7 @@ module Pose
 
 
     def initialize classes, text, options = {}
-      @classes = [classes].flatten
+      @classes = Array(classes).flatten
       @text = text
       @options = options
     end
@@ -27,19 +28,19 @@ module Pose
 
     # Returns whether this query contains custom JOIN expressions.
     def has_joins?
-      !@options[:joins].blank?
+      @options[:joins].present?
     end
 
 
     # Returns whether the query defines a limit on the number of results.
     def has_limit?
-      !@options[:limit].blank?
+      @options[:limit].present?
     end
 
 
     # Returns whether this query contains WHERE clauses.
     def has_where?
-      !@options[:where].blank?
+      @options[:where].present?
     end
 
 
@@ -50,9 +51,23 @@ module Pose
     end
 
 
+    # Returns whether the given string is a URL.
+    #
+    # @param [String] word The string to check.
+    #
+    # @return [Boolean]
+    def self.is_url? word
+
+      # Handle localhost separately.
+      return true if /^http:\/\/localhost(:\d+)?/ =~ word
+
+      /^https?:\/\/([\w\.])+\.([\w\.])+/ =~ word
+    end
+
+
     # Returns the custom JOIN expressions of this query.
     def joins
-      @joins ||= [@options[:joins]].flatten.compact
+      @joins ||= Array(@options[:joins]).flatten.compact
     end
 
 
@@ -69,16 +84,41 @@ module Pose
 
 
     def self.query_words query_string
-      query_string.split(' ').map{|query_word| Helpers.root_word query_word}.flatten.uniq
+      query_string.split(' ').map{|query_word| Query.root_word query_word}.flatten.uniq
+    end
+
+
+    # Simplifies the given word to a generic search form.
+    #
+    # @param [String] raw_word The word to make searchable.
+    #
+    # @return [String] The stemmed version of the word.
+    def self.root_word raw_word
+      result = []
+      raw_word_copy = raw_word[0..-1]
+      raw_word_copy.gsub! '%20', ' '
+      raw_word_copy.gsub! /[()*<>'",;\?\-\=&%#]/, ' '
+      raw_word_copy.gsub! /\s+/, ' '
+      raw_word_copy.split(' ').each do |word|
+        if Query.is_url?(word)
+          result.concat word.split(/[\.\/\:]/).delete_if(&:blank?)
+        else
+          word.gsub! /[\-\/\._:]/, ' '
+          word.gsub! /\s+/, ' '
+          word.split(' ').each do |w|
+            stemmed_word =  w.parameterize.singularize
+            result.concat stemmed_word.split ' '
+          end
+        end
+      end
+      result.uniq
     end
 
 
     # Returns the WHERE clause of this query.
     def where
       return [] unless has_where?
-      if @options[:where].size == 2 and @options[:where][0].class == String
-        return [ @options[:where] ]
-      end
+      return [ @options[:where] ] if @options[:where].size == 2 && @options[:where][0].is_a?(String)
       @options[:where]
     end
   end

data/lib/pose/search.rb

@@ -108,13 +108,13 @@ module Pose
     # Finds all matching ids for a single word of the search query.
     def search_word word
       empty_result.tap do |result|
-        data = Pose::Assignment.joins(:word) \
-                               .select('pose_assignments.posable_id, pose_assignments.posable_type') \
-                               .where('pose_words.text LIKE ?', "#{word}%") \
-                               .where('pose_assignments.posable_type IN (?)', @query.class_names)
+        data = Assignment.joins(:word) \
+                         .select('pose_assignments.posable_id, pose_assignments.posable_type') \
+                         .where('pose_words.text LIKE ?', "#{word}%") \
+                         .where('pose_assignments.posable_type IN (?)', @query.class_names)
         data = add_joins data
         data = add_wheres data
-        Pose::Assignment.connection.select_all(data.to_sql).each do |pose_assignment|
+        Assignment.connection.select_all(data.to_sql).each do |pose_assignment|
           result[pose_assignment['posable_type']] << pose_assignment['posable_id'].to_i
         end
       end

data/lib/pose/static_api.rb

@@ -17,7 +17,7 @@ module Pose
     # @return [Array<String>]
     def autocomplete_words query
       return [] if query.blank?
-      Word.where('text LIKE ?', "#{Helpers.root_word(query)[0]}%").map(&:text)
+      Word.where('text LIKE ?', "#{Query.root_word(query)[0]}%").map(&:text)
     end
 
 
@@ -38,8 +38,26 @@ module Pose
     #
     # @return [Hash<Class, ActiveRecord::Relation>]
     def search query_string, classes, options = {}
-      search = Pose::Search.new classes, query_string, options
-      search.results
+      Search.new(classes, query_string, options).results
+    end
+
+
+    # Returns whether the currently used database is a relational one.
+    def has_sql_connection?
+      has_mysql_connection? || has_postgres_connection? || has_sqlite_connection?
+    end
+
+    def has_mysql_connection?
+      ['ActiveRecord::ConnectionAdapters::MysqlAdapter',
+       'ActiveRecord::ConnectionAdapters::Mysql2Adapter'].include? ActiveRecord::Base.connection.class.name
+    end
+
+    def has_postgres_connection?
+      ActiveRecord::Base.connection.class.name == 'ActiveRecord::ConnectionAdapters::PostgreSQLAdapter'
+    end
+
+    def has_sqlite_connection?
+      ActiveRecord::Base.connection.class.name == 'ActiveRecord::ConnectionAdapters::SQLite3Adapter'
     end
   end
 end

data/lib/pose/version.rb

@@ -1,3 +1,3 @@
 module Pose
-  VERSION = "3.0.0"
+  VERSION = "3.1.1"
 end

data/lib/pose/word.rb

@@ -0,0 +1,33 @@
+# A single word in the search index.
+module Pose
+  class Word < ActiveRecord::Base
+    self.table_name_prefix = 'pose_'
+
+    has_many :assignments, class_name: 'Pose::Assignment', dependent: :destroy
+
+
+    # Returns the Pose::Word instances with the given texts.
+    # @param [Array<String>]
+    def self.factory texts
+      texts.map {|text| Word.find_or_create_by text: text}
+    end
+
+
+    def self.remove_unused_words progress_bar = nil
+      if Pose.has_sql_connection?
+        # SQL database --> use an optimized query.
+        Word.select("pose_words.id").
+             joins("LEFT OUTER JOIN pose_assignments ON pose_assignments.word_id = pose_words.id").
+             group("pose_words.id").
+             having("COUNT(pose_assignments.id) = 0").
+             delete_all
+      else
+        # Unknown database --> use metawhere.
+        Word.select(:id).includes(:assignments).find_each(batch_size: 100) do |word|
+          word.delete if word.assignments.size.zero?
+          progress_bar.try(:increment)
+        end
+      end
+    end
+  end
+end

data/lib/pose.rb

@@ -1,11 +1,17 @@
-require "pose/engine"
+require 'active_record'
 require 'pose/query'
 require 'pose/search'
 require 'pose/static_api'
-require 'pose/helpers'
 require 'pose/activerecord_base_additions'
 require 'pose/model_class_additions'
-require 'pose/railtie' if defined? Rails
+require 'pose/assignment'
+require 'pose/word'
+require 'pose/jobs/reindex_all'
+require 'pose/jobs/remove'
+require 'pose/jobs/vacuum'
+require 'pose/jobs/install'
+require 'pose/jobs/uninstall'
 
-module Pose
-end
+Dir[File.join(File.dirname(__FILE__), "tasks/**/*.rake")].each { |ext| load ext } if defined? Rake
+
+ActiveRecord::Base.send :extend, Pose::ActiveRecordBaseAdditions