pallan-sunspot 0.8.3

data/History.txt

@@ -0,0 +1,39 @@
+== 0.8.0 2009-05-22
+* Access query API directly; instantiate search without running it
+* Dynamic fields
+* Search blocks can be evaluated in calling context
+
+== 0.7.3 2009-05-06
+* Better exception handling when class doesn't have adapter/setup
+
+== 0.7.2 2009-04-29
+* Dirty sessions
+
+== 0.7.1 2009-04-29
+* Removed extlib dependency from gemspec
+
+== 0.7.0 2009-04-28
+* Less magic in the DSL
+* Restrict by empty values
+* Negative scoping using without() method
+* Exclusion by object identity using without(instance)
+* Support for faceting
+* Explicit commits
+* Boolean field type
+* Attribute field flexibility
+* Virtual field blocks can be evaluated in calling context
+* Order available by multiple fields
+* New adapter API
+* Got rid of builder API
+* Full documentation
+
+== 0.0.2 2009-02-14
+* Run sunspot's built-in Solr instance using
+  sunspot-solr executable
+* Search hash interpretation delegated to
+  Builder object
+
+== 0.0.1 2008-12-11
+* Initial release
+* Define indexing for any class using DSL
+* Search indexed classes using DSL

data/LICENSE

@@ -0,0 +1,18 @@
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+'Software'), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.rdoc

@@ -0,0 +1,158 @@
+= Sunspot
+
+http://outoftime.github.com/sunspot
+
+Sunspot is a Ruby library for expressive, powerful interaction with the Solr search engine.
+Sunspot is built on top of the solr-ruby gem, which provides a low-level interface for Solr
+interaction; Sunspot provides a simple, intuitive, expressive DSL backed by powerful
+features for indexing objects and searching for them.
+
+Sunspot is designed to be easily plugged in to any ORM, or even non-database-backed
+objects such as the filesystem.
+
+=== Features:
+
+* Define indexing strategy for each searchable class using intuitive block-based API
+* Clean separation between keyword-searchable fields and fields for scoping/ordering
+* Define fields based on existing attributes or "virtual fields" for custom indexing
+* Indexes each object's entire superclass hierarchy, for easy searching for all objects inheriting from a parent class
+* Intuitive DSL for scoping searches, with all the usual boolean operators available
+* Intuitive interface for requesting facets on indexed fields
+* Extensible adapter architecture for easy integration of other ORMs or non-model classes
+* Full compatibility with will_paginate
+* Ordering
+
+== Installation
+
+  gem sources -a http://gems.github.com
+  gem install outoftime-sunspot
+
+In order to start the packaged Solr installation, run:
+
+  sunspot-solr start -- [-d /path/to/data/directory] [-p port] [-s path/to/solr/home]
+
+If you don't specify a data directory, your Solr index will be stored in your operating system's temporary directory.
+
+If you specify a solr home, the directory must contain a <code>conf</code>
+directory, which should contain at least <code>schema.xml</code> and
+<code>solrconfig.xml</code>. Be sure to copy the <code>schema.xml</code> out of
+the Sunspot gem's <code>solr/solr/conf</code> directory. Sunspot relies on the
+field name patterns defined in the packaged <code>schema.xml</code>, so those
+cannot be modified.
+
+You can also run your own instance of Solr wherever you'd like; just copy the solr/config/schema.xml file out of the gem's solr into your installation.
+You can change the URL at which Sunspot accesses Solr with:
+
+  Sunspot.config.solr.url = 'http://solr.my.host:9818/solr'
+
+== Rails Integration
+
+The {Sunspot::Rails}[http://github.com/outoftime/sunspot_rails] plugin makes
+integrating Sunspot into Rails drop-in easy.
+
+== Using Sunspot
+
+=== Define an index:
+
+  class Post
+    #...
+  end
+
+  Sunspot.setup(Post) do
+    text :title, :body
+    string :author_name
+    integer :blog_id
+    integer :category_ids
+    float :average_rating, :using => :ratings_average
+    time :published_at
+    string :sort_title do
+      title.downcase.sub(/^(an?|the)\W+/, ''/) if title = self.title
+    end
+  end
+
+See Sunspot.setup for more information.
+
+Note that in order for a class to be searchable, it must have an adapter
+registered for itself or one of its subclasses. Adapters allow Sunspot to load
+objects out of persistent storage, and to determine their primary key for
+indexing. {Sunspot::Rails}[http://github.com/outoftime/sunspot_rails] comes with
+an adapter for ActiveRecord objects, but for other types of models you will need
+to define your own. See Sunspot::Adapters for more information.
+
+=== Search for objects:
+  
+  search = Sunspot.search Post do
+    keywords 'great pizza'
+    with :author_name, 'Mark Twain'
+    with(:blog_id).any_of [2, 14]
+    with(:category_ids).all_of [4, 10]
+    with(:published_at).less_than Time.now
+    without :title, 'Bad Title'
+    without bad_instance # specifically exclude this instance from results
+
+    paginate :page => 3, :per_page => 15
+    order_by :average_rating, :desc
+
+    facet :blog_id
+  end
+
+See Sunspot.search for more information.
+
+=== Get data from search:
+
+  search.results
+  search.total
+  search.page
+  search.per_page
+  search.facet(:blog_id)
+
+=== Building searches manually:
+
+The search DSL is great for building searches from fairly static parameters,
+but a highly dynamic search might want to leverage an intermediate approach
+(such as an application of the Builder pattern). For these cases, Sunspot
+exposes direct access to the Query object:
+
+  search = Sunspot.new_search(Post)
+  search.query.keywords = 'great pizza'
+  search.query.add_restriction(:author_name, :equal_to, 'Mark Twain')
+  search.query.add_restriction(:title, :equal_to, 'Bad Title', true) # negate the restriction
+  search.query.exclude_instance(bad_instance)
+  search.query.paginate(3, 15)
+  search.query.order_by(:average_rating, :desc)
+  search.query.add_field_facet(:blog_id)
+  search.execute! 
+
+== About the API documentation
+
+All of the methods documented in the RDoc are considered part of Sunspot's
+public API. Methods that are not part of the public API are documented in the
+code, but excluded from the RDoc. If you find yourself needing to access methods
+that are not part of the public API in order to do what you need, please contact
+me so I can rectify the situation!
+
+== Dependencies
+
+1. solr-ruby
+2. Java
+
+Sunspot has been tested with MRI 1.8.6, YARV 1.9.1, and JRuby 1.2.0
+
+== Bugs
+
+Please submit bug reports to
+http://outoftime.lighthouseapp.com/projects/20339-sunspot
+
+== Further Reading
+
+* Sunspot Discussion: http://groups.google.com/group/ruby-sunspot
+* Posts about Sunspot from my tumblog: http://outofti.me/tagged/sunspot
+* Read about it on Linux Magazine: http://www.linux-mag.com/id/7341
+
+== Contributors
+
+Mat Brown (mat@patch.com)
+
+== License
+
+Sunspot is distributed under the MIT License, copyright (c) 2008-2009 Mat Brown

data/Rakefile

@@ -0,0 +1,11 @@
+require 'rubygems'
+
+ENV['RUBYOPT'] = '-W1'
+
+task :environment do
+  require File.dirname(__FILE__) + '/lib/sunspot'
+end
+
+Dir['tasks/**/*.rake'].each { |t| load t }
+
+task :default => 'spec:api'

data/TODO

@@ -0,0 +1,6 @@
+=== 0.9 ===
+* Instantiated facets!
+* Direct access to adapter
+* Facet by type (?)
+* Switch to RSolr
+* Query-based faceting (?)

data/VERSION.yml

@@ -0,0 +1,4 @@
+--- 
+:major: 0
+:minor: 8
+:patch: 3

data/bin/sunspot-solr

@@ -0,0 +1,46 @@
+#!/usr/bin/env ruby
+require 'rubygems'
+gem 'daemons', '~> 1.0'
+gem 'optiflag', '~> 0.6.5'
+require 'fileutils'
+require 'tmpdir'
+require 'daemons'
+require 'optiflag'
+
+working_directory = FileUtils.pwd
+solr_home = File.join(File.dirname(__FILE__), '..', 'solr')
+
+module SolrFlags extend OptiFlagSet
+  optional_flag 'p' do
+    description 'Port on which to run Solr (default 8983)'
+    long_form 'port'
+  end
+
+  optional_flag 'd' do
+    description 'Solr data directory'
+  end
+
+  optional_flag 's' do
+    description 'Solr home (should contain conf/ directory)'
+  end
+
+  and_process!
+end
+
+port = ARGV.flags.p || '8983'
+data_dir = File.expand_path(ARGV.flags.d || File.join(Dir.tmpdir, 'solr_data'))
+home = File.expand_path(ARGV.flags.s) if ARGV.flags.s
+
+Daemons.run_proc('sunspot-solr') do
+  FileUtils.cd(working_directory) do
+    FileUtils.cd(solr_home) do
+      args = ['java']
+      args << "-Djetty.port=#{port}" if port
+      args << "-Dsolr.data.dir=#{data_dir}" if data_dir
+      args << "-Dsolr.solr.home=#{home}" if home
+      args << '-jar' << 'start.jar'
+      STDERR.puts(args * ' ')
+      Kernel.exec(*args)
+    end
+  end
+end

data/lib/light_config.rb

@@ -0,0 +1,40 @@
+module LightConfig
+  class Configuration
+    def initialize(&block)
+      @properties = {}
+      ::LightConfig::Builder.new(self).instance_eval(&block)
+      singleton = (class <<self; self; end)
+      @properties.keys.each do |property|
+        singleton.module_eval do
+          define_method property do
+            @properties[property]
+          end
+
+          define_method "#{property}=" do |value|
+            @properties[property] = value
+          end
+        end
+      end
+    end
+  end
+
+  class Builder
+    def initialize(configuration)
+      @configuration = configuration
+    end
+
+    def method_missing(method, *args, &block)
+      raise ArgumentError("wrong number of arguments(#{args.length} for 1)") unless args.length < 2
+      value = if block then ::LightConfig::Configuration.new(&block)
+              else args.first
+              end
+      @configuration.instance_variable_get(:@properties)[method] = value
+    end
+  end
+
+  class <<self
+    def build(&block)
+      LightConfig::Configuration.new(&block)
+    end
+  end
+end

data/lib/sunspot.rb

@@ -0,0 +1,404 @@
+gem 'solr-ruby'
+require 'solr'
+require File.join(File.dirname(__FILE__), 'light_config')
+
+%w(adapters configuration setup field data_extractor indexer
+   query search facet facet_row session type util dsl).each do |filename|
+  require File.join(File.dirname(__FILE__), 'sunspot', filename)
+end
+
+#
+# The Sunspot module provides class-method entry points to most of the
+# functionality provided by the Sunspot library. Internally, the Sunspot
+# singleton class contains a (non-thread-safe!) instance of Sunspot::Session,
+# to which it delegates most of the class methods it exposes. In the method
+# documentation below, this instance is referred to as the "singleton session".
+#
+# Though the singleton session provides a convenient entry point to Sunspot,
+# it is by no means required to use the Sunspot class methods. Multiple sessions
+# may be instantiated and used (if you need to connect to multiple Solr
+# instances, for example.)
+#
+# Note that the configuration of classes for index/search (the +setup+
+# method) is _not_ session-specific, but rather global.
+#
+module Sunspot
+  UnrecognizedFieldError = Class.new(Exception)
+  UnrecognizedRestrictionError = Class.new(Exception)
+  NoAdapterError = Class.new(Exception)
+  NoSetupError = Class.new(Exception)
+
+  class <<self
+    # Configures indexing and search for a given class.
+    #
+    # ==== Parameters
+    #
+    # clazz<Class>:: class to configure
+    #
+    # ==== Example
+    #
+    #   Sunspot.setup(Post) do
+    #     text :title, :body
+    #     string :author_name
+    #     integer :blog_id
+    #     integer :category_ids
+    #     float :average_rating, :using => :ratings_average
+    #     time :published_at
+    #     string :sort_title do
+    #       title.downcase.sub(/^(an?|the)\W+/, ''/) if title = self.title
+    #     end
+    #   end
+    #
+    # ====== Attribute Fields vs. Virtual Fields
+    #
+    # Attribute fields call a method on the indexed object and index the
+    # return value. All of the fields defined above except for the last one are
+    # attribute fields. By default, the field name will also be the attribute
+    # used; this can be overriden with the +:using+ option, as in
+    # +:average_rating+ above. In that case, the attribute +:ratings_average+
+    # will be indexed with the field name +:average_rating+.
+    #
+    # +:sort_title+ is a virtual field, which evaluates the block inside the
+    # context of the instance being indexed, and indexes the value returned
+    # by the block. If the block you pass takes an argument, it will be passed
+    # the instance rather than being evaluated inside of it; so, the following
+    # example is equivalent to the one above (assuming #title is public):
+    #
+    #   Sunspot.setup(Post) do
+    #     string :sort_title do |post|
+    #       post.title.downcase.sub(/^(an?|the)\W+/, ''/) if title = self.title
+    #     end
+    #   end
+    #
+    # ===== Field Types
+    #
+    # The available types are:
+    # 
+    # * +text+
+    # * +string+
+    # * +integer+
+    # * +float+
+    # * +time+
+    # * +boolean+
+    #
+    # Note that the +text+ type behaves quite differently from the others -
+    # this is the type that is indexed as fulltext, and is searched using the
+    # +keywords+ method inside the search DSL. Text fields cannot have
+    # restrictions set on them, nor can they be used in order statements or
+    # for facets. All other types are indexed literally, and thus can be used
+    # for all of those operations. They will not, however, be searched in
+    # fulltext. In this way, Sunspot provides a complete barrier between
+    # fulltext fields and value fields.
+    #
+    # It is fine to specify a field both as a text field and a string field;
+    # internally, the fields will have different names so there is no danger
+    # of conflict.
+    # 
+    # ===== Dynamic Fields
+    # 
+    # For use cases which have highly dynamic data models (for instance, an
+    # open set of key-value pairs attached to a model), it may be useful to
+    # defer definition of fields until indexing time. Sunspot exposes dynamic
+    # fields, which define a data accessor (either attribute or virtual, see
+    # above), which accepts a hash of field names to values. Note that the field
+    # names in the hash are internally scoped to the base name of the dynamic
+    # field, so any time they are referred to, they are referred to using both
+    # the base name and the dynamic (runtime-specified) name.
+    #
+    # Dynamic fields are speficied in the setup block using the type name
+    # prefixed by +dynamic_+. For example:
+    # 
+    #   Sunspot.setup(Post) do
+    #     dynamic_string :custom_values do
+    #       key_value_pairs.inject({}) do |hash, key_value_pair|
+    #         hash[key_value_pair.key.to_sym] = key_value_pair.value
+    #       end
+    #     end
+    #   end
+    # 
+    # If you later wanted to facet all of the values for the key "cuisine",
+    # you could issue:
+    # 
+    #   Sunspot.search(Post) do
+    #     dynamic :custom_values do
+    #       facet :cuisine
+    #     end
+    #   end
+    # 
+    # In the documentation, +:custom_values+ is referred to as the "base name" -
+    # that is, the one specified statically - and +:cuisine+ is referred to as
+    # the dynamic name, which is the part that is specified at indexing time.
+    # 
+    def setup(clazz, &block)
+      Setup.setup(clazz, &block)
+    end
+
+    # Indexes objects on the singleton session.
+    #
+    # ==== Parameters
+    #
+    # objects...<Object>:: objects to index (may pass an array or varargs)
+    #
+    # ==== Example
+    #
+    #   post1, post2 = Array(2) { Post.create }
+    #   Sunspot.index(post1, post2)
+    #
+    # Note that indexed objects won't be reflected in search until a commit is
+    # sent - see Sunspot.index! and Sunspot.commit
+    #
+    def index(*objects)
+      session.index(*objects)
+    end
+
+    # Indexes objects on the singleton session and commits immediately.
+    #
+    # See: Sunspot.index and Sunspot.commit
+    #
+    # ==== Parameters
+    #
+    # objects...<Object>:: objects to index (may pass an array or varargs)
+    #
+    def index!(*objects)
+      session.index!(*objects)
+    end
+
+    # Commits the singleton session
+    #
+    # When documents are added to or removed from Solr, the changes are
+    # initially stored in memory, and are not reflected in Solr's existing
+    # searcher instance. When a commit message is sent, the changes are written
+    # to disk, and a new searcher is spawned. Commits are thus fairly
+    # expensive, so if your application needs to index several documents as part
+    # of a single operation, it is advisable to index them all and then call
+    # commit at the end of the operation.
+    #
+    # Note that Solr can also be configured to automatically perform a commit
+    # after either a specified interval after the last change, or after a
+    # specified number of documents are added. See
+    # http://wiki.apache.org/solr/SolrConfigXml
+    #
+    def commit
+      session.commit
+    end
+
+    # 
+    # Create a new Search instance, but do not execute it immediately. Generally
+    # you will want to use the #search method to execute searches using the
+    # DSL; however, if you are building searches dynamically (using the Builder
+    # pattern, for instance), it may be easier to access the Query API directly.
+    # 
+    # ==== Parameters
+    #
+    # types<Class>...::
+    #   Zero, one, or more types to search for. If no types are passed, all
+    #   configured types will be searched for.
+    #
+    # ==== Returns
+    #
+    # Sunspot::Search::
+    #   Search object, not yet executed. Query parameters can be added manually;
+    #   then #execute! should be called.
+    # 
+    def new_search(*types)
+      session.new_search(*types)
+    end
+
+
+    # Search for objects in the index.
+    #
+    # ==== Parameters
+    #
+    # types<Class>...::
+    #   Zero, one, or more types to search for. If no types are passed, all
+    #   configured types will be searched.
+    #
+    # ==== Options (last argument, optional)
+    #
+    # :keywords<String>:: Fulltext search string
+    # :conditions<Hash>::
+    #   Hash of key-value pairs to be used as restrictions. Keys are field
+    #   names. Scalar values are used as equality restrictions; arrays are used
+    #   as "any of" restrictions; and Ranges are used as range restrictions.
+    # :order<String>:: order field and direction (e.g., 'updated_at desc')
+    # :page<Integer>:: Page to start on for pagination
+    # :per_page<Integer>::
+    #   Number of results to use per page. Ignored if :page is not specified.
+    #
+    # ==== Returns
+    #
+    # Sunspot::Search:: Object containing results, facets, count, etc.
+    #
+    # The fields available for restriction, ordering, etc. are those that meet
+    # the following criteria:
+    #
+    # * They are not of type +text+.
+    # * They are defined for all of the classes being searched
+    # * They have the same data type for all of the classes being searched
+    # * They have the same multiple flag for all of the classes being searched.
+    #
+    # The restrictions available are the constants defined in the
+    # Sunspot::Restriction class. The standard restrictions are:
+    #
+    #   with(:field_name).equal_to(value)
+    #   with(:field_name, value) # shorthand for above
+    #   with(:field_name).less_than(value)
+    #   with(:field_name).greater_than(value)
+    #   with(:field_name).between(value1..value2)
+    #   with(:field_name).any_of([value1, value2, value3])
+    #   with(:field_name).all_of([value1, value2, value3])
+    #   without(some_instance) # exclude that particular instance
+    #
+    # +without+ can be substituted for +with+, causing the restriction to be
+    # negated. In the last example above, only +without+ works, as it does not
+    # make sense to search only for an instance you already have.
+    #
+    # Equality restrictions can take +nil+ as a value, which restricts the
+    # results to documents that have no value for the given field. Passing +nil+
+    # as a value to other restriction types is illegal. Thus:
+    #
+    #   with(:field_name, nil) # ok
+    #   with(:field_name).equal_to(nil) # ok
+    #   with(:field_name).less_than(nil) # bad
+    #
+    # ==== Example
+    #
+    #   Sunspot.search(Post) do
+    #     keywords 'great pizza'
+    #     with(:published_at).less_than Time.now
+    #     with :blog_id, 1
+    #     without current_post
+    #     facet :category_ids
+    #     order_by :published_at, :desc
+    #     paginate 2, 15
+    #   end
+    #  
+    # If the block passed to #search takes an argument, that argument will
+    # present the DSL, and the block will be evaluated in the calling context.
+    # This will come in handy for building searches using instance data or
+    # methods, e.g.:
+    #
+    #   Sunspot.search(Post) do |query|
+    #     query.with(:blog_id, @current_blog.id)
+    #   end
+    #
+    # See Sunspot::DSL::Scope and Sunspot::DSL::Query for the full API presented
+    # inside the block.
+    #
+    def search(*types, &block)
+      session.search(*types, &block)
+    end
+
+    # Remove objects from the index. Any time an object is destroyed, it must
+    # be removed from the index; otherwise, the index will contain broken
+    # references to objects that do not exist, which will cause errors when
+    # those objects are matched in search results.
+    #
+    # ==== Parameters
+    #
+    # objects...<Object>::
+    #   Objects to remove from the index (may pass an array or varargs)
+    #
+    # ==== Example
+    #
+    #   post.destroy
+    #   Sunspot.remove(post)
+    #
+    def remove(*objects)
+      session.remove(*objects)
+    end
+
+    # 
+    # Remove objects from the index and immediately commit. See Sunspot.remove
+    #
+    # ==== Parameters
+    #
+    # objects...<Object>:: Objects to remove from the index
+    #
+    def remove!
+      session.remove!(*objects)
+    end
+
+    # Remove all objects of the given classes from the index. There isn't much
+    # use for this in general operations but it can be useful for maintenance,
+    # testing, etc. If no arguments are passed, remove everything from the
+    # index.
+    #
+    # ==== Parameters
+    #
+    # classes...<Class>::
+    #   classes for which to remove all instances from the index (may pass an
+    #   array or varargs)
+    #
+    # ==== Example
+    #
+    #   Sunspot.remove_all(Post, Blog)
+    #
+    def remove_all(*classes)
+      session.remove_all(*classes)
+    end
+
+    # 
+    # Remove all objects of the given classes from the index and immediately
+    # commit. See Sunspot.remove_all
+    #
+    # ==== Parameters
+    #
+    # classes...<Class>::
+    #   classes for which to remove all instances from the index
+    def remove_all!(*classes)
+      session.remove_all(*classes)
+    end
+
+    #
+    # True if documents have been added, updated, or removed since the last
+    # commit.
+    #
+    # ==== Returns
+    #
+    # Boolean:: Whether there have been any updates since the last commit
+    #
+    def dirty?
+      session.dirty?
+    end
+
+    # 
+    # Sends a commit if the session is dirty (see #dirty?).
+    #
+    def commit_if_dirty
+      session.commit_if_dirty
+    end
+    
+    # Returns the configuration associated with the singleton session. See
+    # Sunspot::Configuration for details.
+    #
+    # ==== Returns
+    #
+    # LightConfig::Configuration:: configuration for singleton session
+    #
+    def config
+      session.config
+    end
+
+    # 
+    # Resets the singleton session. This is useful for clearing out all
+    # static data between tests, but probably nowhere else.
+    #
+    def reset!
+      @session = nil
+    end
+
+    private
+
+    # 
+    # Get the singleton session, creating it if none yet exists.
+    #
+    # ==== Returns
+    #
+    # Sunspot::Session:: the singleton session
+    #
+    def session #:nodoc:
+      @session ||= Session.new
+    end
+  end
+end