sunspot 2.0.0.pre.120720 → 2.0.0.pre.120924

data/sunspot_rails/README.rdoc

@@ -0,0 +1,265 @@
+= Sunspot::Rails
+
+Sunspot::Rails is a Rails plugin that provides drop-in integration of the
+Sunspot[http://outoftime.github.com/sunspot] Solr search library with Rails. It
+provides the following features:
+
+* Configure Sunspot using config/sunspot.yml
+* Extend ActiveRecord for easy index configuration, search, and indexing
+* Automatically index ActiveRecord objects when they are saved, and remove them
+  from the index when they are destroyed (can be disabled)
+* Automatically commit Solr changes at the end of each request (can be disabled)
+* Provide utility methods to find and fix orphaned documents and rebuild the
+  Solr index for a given class
+* Provide rake tasks for starting and stopping the development Solr instance,
+  using the configuration in sunspot.yml
+
+Sunspot::Rails has been tested with Rails versions 2.3 and 3.0
+
+== Installation in Rails 3
+
+In your <code>Gemfile</code>:
+
+  gem 'sunspot_rails'
+  # This is an optional packaged Solr:
+  group :test, :development do
+    gem 'sunspot_solr'
+  end
+
+== Installation in Rails 2
+
+In your project's <code>config/environment.rb</code>, add the following gem dependencies:
+
+  config.gem 'sunspot'
+  config.gem 'sunspot_rails'
+
+And in your project's <code>config/development.rb</code> and <code>config/test.rb</code>,
+add the following gem dependency to install the optional packaged Solr:
+
+  config.gem 'sunspot_solr'
+
+Install the gems with:
+
+  rake gems:install
+
+Generate the file <code>config/sunspot.yml</code>:
+
+  script/generate sunspot
+
+Rails doesn't automatically load rake tasks from plugins installed as gems
+(https://rails.lighthouseapp.com/projects/8994/tickets/59). If you installed
+Sunspot::Rails as a gem, add the following line to your project's Rakefile:
+
+  require 'sunspot/rails/tasks'
+
+== Using Sunspot::Rails
+
+If you wish to make modifications to the Solr schema, you can create a custom
+Solr home in your project directory. In order to do so, create the directory
+<code>RAILS_ROOT/solr/conf</code>, and copy in the contents of the Solr gem's
+<code>solr/solr/conf</code> directory. If the files are in the right place,
+Sunspot::Rails will detect them and tell Solr to use your local configurations.
+Use caution when modifying <code>schema.xml</code> - Sunspot relies on the
+field naming scheme in the packaged schema file.
+
+To start up a Solr instance, issue the following (requires sunspot_solr to be
+installed):
+
+  rake sunspot:solr:start
+
+Note that using the built-in Solr instance packaged with Sunspot is great for
+development, but is not recommended for production. See the Sunspot
+documentation for more information.
+
+== Usage
+
+=== Setup
+
+In order for an ActiveRecord model to be indexable and searchable, it must be
+configured for search. For example:
+
+  class Post < ActiveRecord::Base
+    searchable do
+      text :title, :body
+      integer :blog_id
+      time :updated_at
+      string :sort_title do
+        title.downcase.sub(/^(an?|the) /, '')
+      end
+    end
+  end
+
+See the documentation for Sunspot.setup for full details on what can go in the
+configuration block.
+
+=== Indexing
+
+By default, models are indexed whenever they are saved, and removed from the
+index whenever they are destroyed. This behavior can be disabled:
+
+  class Post < ActiveRecord::Base
+    searchable :auto_index => false, :auto_remove => false do
+      # setup...
+    end
+  end
+
+Note that <b>using the <code>:auto_remove</code> option is not recommended
+</b>, as destroying an object without removing it from the index will
+create an orphaned document in the index, which is a Bad Thing. Turning off
+<code>:auto_index</code> is perfectly safe if you prefer to manage indexing manually
+(perhaps using a background job).
+
+If you have disabled lifecycle indexing hooks, you can invoke indexing
+operations directly on your model:
+
+  post = Post.create
+  post.index
+  post.remove_from_index
+
+=== Committing
+
+When data is changed in Solr, it is initially stored in memory and not made
+available to the currently running searcher instance. Issuing a +commit+ to Solr
+will cause it to write the changes to disk, and instantiate a new searcher
+instance. This operation is fairly expensive, so rather than issuing a commit
+every time a document is added or removed, Sunspot::Rails issues a commit at
+the end of any request where data has been added to or removed from Solr. If
+you need to immediately issue a commit, bang!-versions of the methods are
+available:
+
+  post = Post.create
+  post.index!
+  # this is the same as...
+  post.index
+  Sunspot.commit
+
+When writing tests outside of the context of a controller request, you will want
+to use one of these two approaches.
+
+=== Searching
+
+Do it like this:
+
+  Post.search do
+    with :blog_id, 1
+    with(:updated_at).greater_than(Time.now - 2.weeks)
+    order_by :sort_title, :asc
+    paginate :page => 1, :per_page => 15
+  end
+
+See the documentation for <code>Sunspot.search</code> for all the options
+available in the search block, and the information available in the result
+block.
+
+=== Searching for IDs
+
+In some situations, you may want to get the IDs for models returned by a search
+without actually loading the models out of the database. For that, you can
+call +search_ids+, using the same block format as #search. This will return an
+array of IDs.
+
+=== Searching for multiple types
+
+Sunspot is entirely agnostic about whether searches are for one or more types;
+the only restriction is that columns used for restriction, ordering, etc. are
+defined in the same way for all types being searched. Sunspot::Rails does not
+provide any additional support for this, since there is not anything useful to
+be added, so just use the interface provided by Sunspot:
+
+  Sunspot.search(Post, Comment) do
+    with :blog_id, 1
+    order_by :created_at, :asc
+  end
+
+Be sure to check out the Sunspot documentation for all the details.
+
+=== Adding search functionality in mixins
+
+Sunspot does not require that search setup for a given class happen all in one
+place; it is perfectly acceptable to call the <code>Sunspot.setup</code> method
+more than once. This capability is particularly useful for adding search
+functionality in mixins. For instance, if you have a +Ratable+ module, you may
+wish to add additional search fields for searchable classes that mix in that
+module. For example:
+
+  module Ratable
+    def self.included(base)
+      if base.searchable?
+        base.searchable do
+          float :average_rating do
+            ratings.average(:value)
+          end
+        end
+      end
+    end
+  end
+
+Note the use of <code>base.searchable?</code> - this ensures that only classes
+that already have search enabled will have the additional configuration added.
+The above pattern requires that the class be declared searchable before the
+module is mixed in; other patterns (such as passing a :searchable option to an
+acts_as_-style declaration) may be more flexible.
+
+=== Utility methods
+
+If you need to completely reindex all of the instances of a given model class,
+you can issue:
+
+  Post.reindex
+
+If for some reason models get deleted from the database, but not from the index,
+they will become index orphans - not a good situation. To get IDs that exist in
+the index but not the database, you can use the +index_orphans+ method; to
+remove those documents from the index, use +clean_index_orphans+. Note that
+neither of these operations should be needed if Sunspot and Sunspot::Rails are
+used as intended.
+
+== Testing Solr integration using RSpec
+
+To disable the sunspot-solr integration for your active record models, require
+the file `sunspot/rails/spec_helper`
+
+Then, in your spec, use the #disconnect_sunspot method:
+
+  require 'sunspot/rails/spec_helper'
+
+  describe Post do
+    disconnect_sunspot
+
+    it 'should have some behavior'
+      # ...
+    end
+  end
+
+In all of the examples in this group, all Sunspot calls will be stubbed out. The
+Sunspot#search method will return a stub search object that mimics a search with
+no results.
+
+== Further Reading
+
+Reading the {Sunspot documentation}[http://outoftime.github.com/sunspot/docs] is
+highly recommended. Sunspot::Rails exists to wrap Sunspot with a Rails-friendly
+API, but almost all of the functionality you use in Sunspot::Rails is
+implemented in Sunspot.
+
+Posts about Sunspot on my blog are available at http://outofti.me/tagged/sunspot
+
+== Bugs
+
+Please submit bug reports to https://github.com/sunspot/sunspot/issues
+
+== Contributors
+
+- Mat Brown (mat@patch.com)
+- Peer Allan (peer.allan@gmail.com)
+- Michael Moen (michael@underpantsgnome.com)
+- Benjamin Krause (bk@benjaminkrause.com)
+- Adam Salter (adam@codebright.net)
+- Brandon Keepers (brandon@opensoul.org)
+- Paul Canavese (paul@canavese.org)
+- John Eberly (jeberly@elctech.com)
+- Gert Thiel (gertthiel@gmail.com)
+
+== License
+
+Sunspot::Rails is distributed under the MIT License, copyright (c) 2009 Mat Brown

data/sunspot_rails/Rakefile

@@ -0,0 +1,17 @@
+require 'rake'
+
+begin
+  require 'rdoc/task'
+rescue LoadError
+  require 'rake/rdoctask'
+end
+
+if File.exist?(sunspot_lib = File.expand_path(File.join(File.dirname(__FILE__), '..', 'sunspot', 'lib')))
+  STDERR.puts("Using sunspot lib at #{sunspot_lib}")
+  $: << sunspot_lib
+end
+
+task :environment do
+end
+
+FileList['dev_tasks/*.rake', 'lib/sunspot/rails/tasks.rb'].each { |file| load(file) }

data/sunspot_rails/TODO

@@ -0,0 +1,8 @@
+* Delegate new_search method
+=======
+* Fix final status output for reindex
+* Add batch-per-request option
+* Optionally yield from reindex
+* Access to all searchable classes
+* Add a method to disable sunspot updates for certain values, e.g. 
+  if you just touch a record, sunspot shouldn't index the 

data/sunspot_rails/dev_tasks/rdoc.rake

@@ -0,0 +1,24 @@
+begin
+  require 'hanna/rdoctask'
+rescue LoadError
+  if require 'rubygems'
+    retry
+  end
+  # It's OK if hanna isn't installed.
+end
+
+Rake::RDocTask.new(:doc) do |rdoc|
+  rdoc.main = 'README.md'
+  rdoc.rdoc_files.include('README.md', 'lib/sunspot/rails/**/*.rb', 'lib/sunspot/rails.rb')
+  rdoc.rdoc_dir = 'doc'
+end
+
+namespace :doc do
+  desc 'Generate rdoc and move into pages directory'
+  task :publish => :redoc do
+    doc_dir = File.join(File.dirname(__FILE__), '..', 'doc')
+    publish_dir = File.join(File.dirname(__FILE__), '..', '..', 'pages', 'rails', 'docs')
+    FileUtils.rm_rf(publish_dir) if File.exist?(publish_dir)
+    FileUtils.cp_r(doc_dir, publish_dir)
+  end
+end

data/sunspot_rails/dev_tasks/release.rake

@@ -0,0 +1,4 @@
+namespace :release do
+  desc 'Release gem on RubyForge and GitHub'
+  task :all => [:release, :"rubyforge:release:gem"]
+end

data/sunspot_rails/dev_tasks/spec.rake

@@ -0,0 +1,107 @@
+require 'fileutils'
+
+namespace :spec do
+  def rails_app_path(version)
+    File.join(File.dirname(__FILE__), "..", "tmp", "rails_#{version.gsub(".", "_")}_app")
+  end
+
+  def gemfile_path(version)
+    File.join(File.dirname(__FILE__), "..", "gemfiles", "rails-#{version}")
+  end
+
+  def vendor_path(version)
+    File.expand_path("vendor/bundle", rails_app_path(version))
+  end
+
+  def rails_template_path
+    File.join(File.dirname(__FILE__), "..", "spec", "rails_template")
+  end
+
+  task :run_with_rails => [:set_gemfile, :generate_rails_app, :setup_rails_app, :run]
+
+  task :set_gemfile do
+    version = ENV['VERSION']
+
+    ENV['BUNDLE_PATH']    = vendor_path(version)
+    ENV['BUNDLE_GEMFILE'] = gemfile_path(version)
+
+    unless File.exist?(ENV['BUNDLE_PATH'])
+      puts "Installing gems for Rails #{version} (this will only be done once)..."
+      system("bundle install #{ENV['BUNDLE_ARGS']}") || exit(1)
+    end
+  end
+
+  task :generate_rails_app do
+    version = ENV['VERSION']
+    app_path = rails_app_path(version)
+
+    unless File.exist?(File.expand_path("config/environment.rb", app_path))
+      rails_cmd = "bundle exec rails _#{version}_"
+
+      puts "Generating Rails #{version} application..."
+      if version.start_with?("2")
+        system("#{rails_cmd} \"#{app_path}\" --force") || exit(1)
+      elsif version.start_with?("3")
+        system("#{rails_cmd} new \"#{app_path}\" --force --skip-git --skip-javascript --skip-gemfile --skip-sprockets") || exit(1)
+      end
+    end
+  end
+
+  task :setup_rails_app do
+    version = ENV['VERSION']
+    app_path = rails_app_path(version)
+
+    FileUtils.cp_r File.join(rails_template_path, "."), app_path
+  end
+
+  task :run do
+    version = ENV['VERSION']
+
+    ENV['BUNDLE_GEMFILE'] = gemfile_path(version)
+    ENV['RAILS_ROOT']     = rails_app_path(version)
+
+    spec_command = if version.start_with?("2")
+                     "spec"
+                   elsif version.start_with?("3")
+                     "rspec"
+                   end
+
+    system "bundle exec #{spec_command} #{ENV['SPEC'] || 'spec/*_spec.rb'} --color"
+  end
+end
+
+def rails_all_versions
+  versions = []
+  Dir.glob(File.join(File.dirname(__FILE__), "..", "gemfiles", "rails-*")).each do |gemfile|
+    if !gemfile.end_with?(".lock") && gemfile =~ /rails-([0-9.]+)/
+      versions << $1
+    end
+  end
+
+  versions
+end
+
+def reenable_spec_tasks
+  Rake::Task.tasks.each do |task|
+    if task.name =~ /spec:/
+      task.reenable
+    end
+  end
+end
+
+desc 'Run spec suite in all Rails versions'
+task :spec do
+  versions = if ENV['RAILS']
+               ENV['RAILS'].split(",")
+             else
+               rails_all_versions
+             end
+
+  versions.each do |version|
+    puts "Running specs against Rails #{version}..."
+
+    ENV['VERSION'] = version
+    reenable_spec_tasks
+    Rake::Task['spec:run_with_rails'].invoke
+  end
+end

data/sunspot_rails/dev_tasks/todo.rake

@@ -0,0 +1,4 @@
+desc 'Show all TODO and related tags'
+task :todo do
+  FileList['lib/**/*.rb'].egrep(/#.*(TODO|FIXME|XXX)/)
+end

data/sunspot_rails/gemfiles/rails-2.3.14

@@ -0,0 +1,12 @@
+source :rubygems
+
+gem 'rails', '2.3.14'
+gem 'sqlite3-ruby', '~> 1.3.1'
+
+gem 'sunspot', :path => File.expand_path('../../../sunspot', __FILE__)
+gem 'sunspot_solr', :path => File.expand_path('../../../sunspot_solr', __FILE__)
+gem 'sunspot_rails', :path => File.expand_path('../..', __FILE__)
+
+group :test do
+	gem 'rspec-rails', '~> 1.3.4'
+end

data/sunspot_rails/gemfiles/rails-3.0.15

@@ -0,0 +1,12 @@
+source :rubygems
+
+gem 'rails', '3.0.15'
+gem 'sqlite3-ruby', '~> 1.3.1'
+
+gem 'sunspot', :path => File.expand_path('../../../sunspot', __FILE__)
+gem 'sunspot_solr', :path => File.expand_path('../../../sunspot_solr', __FILE__)
+gem 'sunspot_rails', :path => File.expand_path('../..', __FILE__)
+
+group :test do
+  gem 'rspec-rails', '~> 2.8.1'
+end

data/sunspot_rails/gemfiles/rails-3.1.6

@@ -0,0 +1,12 @@
+source :rubygems
+
+gem 'rails', '3.1.6'
+gem 'sqlite3-ruby', '~> 1.3.1'
+
+gem 'sunspot', :path => File.expand_path('../../../sunspot', __FILE__)
+gem 'sunspot_solr', :path => File.expand_path('../../../sunspot_solr', __FILE__)
+gem 'sunspot_rails', :path => File.expand_path('../..', __FILE__)
+
+group :test do
+  gem 'rspec-rails', '~> 2.8.1'
+end

data/sunspot_rails/gemfiles/rails-3.2.6

@@ -0,0 +1,12 @@
+source :rubygems
+
+gem 'rails', '3.2.6'
+gem 'sqlite3-ruby', '~> 1.3.1'
+
+gem 'sunspot', :path => File.expand_path('../../../sunspot', __FILE__)
+gem 'sunspot_solr', :path => File.expand_path('../../../sunspot_solr', __FILE__)
+gem 'sunspot_rails', :path => File.expand_path('../..', __FILE__)
+
+group :test do
+  gem 'rspec-rails', '~> 2.8.1'
+end

data/sunspot_rails/generators/sunspot/sunspot_generator.rb

@@ -0,0 +1,9 @@
+class SunspotGenerator < Rails::Generator::Base
+  
+  def manifest
+    record do |m|
+      m.template 'sunspot.yml', 'config/sunspot.yml'
+    end
+  end
+  
+end

data/sunspot_rails/generators/sunspot/templates/sunspot.yml

@@ -0,0 +1,20 @@
+production:
+  solr:
+    hostname: localhost
+    port: 8983
+    log_level: WARNING
+    # read_timeout: 2
+    # open_timeout: 0.5
+
+development:
+  solr:
+    hostname: localhost
+    port: 8982
+    log_level: INFO
+
+test:
+  solr:
+    hostname: localhost
+    port: 8981
+    log_level: WARNING
+    

data/sunspot_rails/install.rb

@@ -0,0 +1 @@
+# Install hook code here

data/sunspot_rails/lib/generators/sunspot_rails/install/install_generator.rb

@@ -0,0 +1,13 @@
+module SunspotRails
+  module Generators
+    class InstallGenerator < Rails::Generators::Base
+      def self.source_root
+        @source_root ||= File.expand_path(File.join(File.dirname(__FILE__), 'templates'))
+      end
+      def copy_config_file
+        template 'config/sunspot.yml'
+      end
+    end
+  end
+end
+

data/sunspot_rails/lib/generators/sunspot_rails/install/templates/config/sunspot.yml

@@ -0,0 +1,19 @@
+production:
+  solr:
+    hostname: localhost
+    port: 8983
+    log_level: WARNING
+    # read_timeout: 2
+    # open_timeout: 0.5
+
+development:
+  solr:
+    hostname: localhost
+    port: 8982
+    log_level: INFO
+
+test:
+  solr:
+    hostname: localhost
+    port: 8981
+    log_level: WARNING

data/sunspot_rails/lib/generators/sunspot_rails.rb

@@ -0,0 +1,9 @@
+module SunspotRails
+  module Generators
+    class Base < Rails::Generators::NamedBase
+      def self.source_root
+        @_sunspot_rails_source_root ||= File.expand_path(File.join(File.dirname(__FILE__), 'sunspot_rails', generator_name, 'templates'))
+      end
+    end
+  end
+end

data/sunspot_rails/lib/sunspot/rails/adapters.rb

@@ -0,0 +1,88 @@
+module Sunspot #:nodoc:
+  module Rails #:nodoc:
+    # 
+    # This module provides Sunspot Adapter implementations for ActiveRecord
+    # models.
+    #
+    module Adapters
+      class ActiveRecordInstanceAdapter < Sunspot::Adapters::InstanceAdapter
+        # 
+        # Return the primary key for the adapted instance
+        #
+        # ==== Returns
+        # 
+        # Integer:: Database ID of model
+        #
+        def id
+          @instance.id
+        end
+      end
+
+      class ActiveRecordDataAccessor < Sunspot::Adapters::DataAccessor
+        # options for the find
+        attr_accessor :include, :select
+
+        def initialize(clazz)
+          super(clazz)
+          @inherited_attributes = [:include, :select]
+        end
+
+        #
+        # Set the fields to select from the database. This will be passed
+        # to ActiveRecord.
+        #
+        # ==== Parameters
+        #
+        # value<Mixed>:: String of comma-separated columns or array of columns
+        #
+        def select=(value)
+          value = value.join(', ') if value.respond_to?(:join)
+          @select = value
+        end
+        
+        # 
+        # Get one ActiveRecord instance out of the database by ID
+        #
+        # ==== Parameters
+        #
+        # id<String>:: Database ID of model to retreive
+        #
+        # ==== Returns
+        #
+        # ActiveRecord::Base:: ActiveRecord model
+        # 
+        def load(id)
+          @clazz.first(options_for_find.merge(
+            :conditions => { @clazz.primary_key => id}
+          ))
+        end
+
+        # 
+        # Get a collection of ActiveRecord instances out of the database by ID
+        #
+        # ==== Parameters
+        #
+        # ids<Array>:: Database IDs of models to retrieve
+        #
+        # ==== Returns
+        #
+        # Array:: Collection of ActiveRecord models
+        #
+        def load_all(ids)
+          @clazz.all(options_for_find.merge(
+            :conditions => { @clazz.primary_key => ids.map { |id| id }}
+          ))
+        end
+        
+        private
+        
+        def options_for_find
+          options = {}
+          options[:include] = @include unless @include.blank?
+          options[:select]  =  @select unless  @select.blank?
+          options
+        end
+      end
+    end
+  end
+end