rsolr 1.0.0.beta → 1.0.0.beta2

data/README.rdoc

@@ -1,11 +1,13 @@
 =RSolr
 
+A simple, extensible Ruby client for Apache Solr.
+
 Notice: This document is only for the the 1.0 (pre-release) in the master branch. The last stable gem release documentation can be found here: http://github.com/mwmitchell/rsolr/tree/v0.12.1
 
-A simple, extensible Ruby client for Apache Solr.
+==Documentation
+The code docs can be viewed here : http://rdoc.info/projects/mwmitchell/rsolr
 
 == Installation:
-  gem sources -a http://gemcutter.org
   sudo gem install rsolr
 
 == Example:
@@ -13,44 +15,50 @@ A simple, extensible Ruby client for Apache Solr.
   require 'rsolr'
   
   # Direct connection
-  solr = RSolr.connect 'http://solrserver.com'
+  solr = RSolr.connect :url => 'http://solrserver.com'
   
   # Connecting over a proxy server
-  solr = RSolr.connect 'http://solrserver.com', :proxy=>'http://user:pass@proxy.example.com:8080'
+  solr = RSolr.connect :url => 'http://solrserver.com', :proxy=>'http://user:pass@proxy.example.com:8080'
   
   # send a request to /select
-  response = solr.get 'select', :q=>'*:*'
-  
-  # send a request to a custom request handler; /catalog
-  response = solr.get 'catalog', :q=>'*:*'
+  response = solr.get 'select', :params => {:q => '*:*'}
   
+  # send a request to /catalog
+  response = solr.get 'catalog', :params => {:q => '*:*'}
+
 == Querying
-Use the #select method to send requests to the /select handler:
-  response = solr.get('select', {
+Use the #get / #post method to send search requests to the /select handler:
+  response = solr.get 'select', :params => {
     :q=>'washington',
     :start=>0,
     :rows=>10
-  })
+  }
 
-The params sent into the method are sent to Solr as-is. The one exception is if a value is an array. When an array is used, multiple parameters *with the same name* are generated for the Solr query. Example:
+The :params sent into the method are sent to Solr as-is. When an array is used, multiple parameters *with the same name* are generated for the Solr query. Example:
   
-  solr.get 'select', :q=>'roses', :fq=>['red', 'violet']
+  solr.get 'select', :params => {:q=>'roses', :fq=>['red', 'violet']}
 
 The above statement generates this Solr query:
   
   select?q=roses&fq=red&fq=violet
 
-There may be cases where the query string is too long for a GET request. RSolr solves this issue by providing a simple way to POST a query to Solr:
-  response = solr.post "select", nil, enormous_params_hash
+===Method Missing
+The RSolr::Client class also uses method_missing for setting the request handler/path:
+  
+  solr.paintings :params => {:q=>'roses', :fq=>['red', 'violet']}
+  
+This is sent to Solr as:
+  paintings?q=roses&fq=red&fq=violet
 
-nil is passed in as the query string data. The enormous_params_hash variable ends up serialized as a form-encoded query string, and the correct content-type headers are sent along to Solr.
 
-== Updating Solr
-Updating us done using native Ruby objects. Hashes are used for single documents and arrays are used for a collection of documents (hashes). These objects get turned into simple XML "messages". Raw XML strings can also be used.
+===Using POST for Search Queries
+There may be cases where the query string is too long for a GET request. RSolr solves this issue by converting hash objects into form-encoded strings:
+  response = solr.post "select", :data => enormous_params_hash
 
-Raw XML via  #update
-  solr.update '<commit/>'
-  solr.update '<optimize/>'
+The :data hash is serialized as a form-encoded query string, and the correct content-type headers are sent along to Solr.
+
+== Updating Solr
+Updating is done using native Ruby objects. Hashes are used for single documents and arrays are used for a collection of documents (hashes). These objects get turned into simple XML "messages". Raw XML strings can also be used.
 
 Single document via #add
   solr.add :id=>1, :price=>1.00
@@ -59,17 +67,28 @@ Multiple documents via #add
   documents = [{:id=>1, :price=>1.00}, {:id=>2, :price=>10.50}]
   solr.add documents
 
-When adding, you can also supply "add" xml element attributes and/or a block for manipulating other "add" related elements (docs and fields) when using the #add method:
+The optional :add_attributes hash can also be used to set Solr "add" document attributes:
+  solr.add documents, :add_attributes => {:commitWithin => 10}
+
+Raw XML via  #update
+  solr.update :data => '<commit/>'
+  solr.update :data => '<optimize/>'
+
+When adding, you can also supply "add" xml element attributes and/or a block for manipulating other "add" related elements (docs and fields) by calling the +xml+ method directly:
   
   doc = {:id=>1, :price=>1.00}
-  add_attributes = {:allowDups=>false, :commitWithin=>10.0}
-  solr.add(doc, add_attributes) do |doc|
+  add_attributes = {:allowDups=>false, :commitWithin=>10}
+  add_xml = solr.xml.add(doc, add_attributes) do |doc|
     # boost each document
     doc.attrs[:boost] = 1.5
     # boost the price field:
     doc.field_by_name(:price).attrs[:boost] = 2.0
   end
 
+Now the "add_xml" object can be sent to Solr like:
+  solr.update :data => add_xml
+  
+===Deleting
 Delete by id
   solr.delete_by_id 1
 or an array of ids
@@ -80,34 +99,26 @@ Delete by query:
 Delete by array of queries
   solr.delete_by_query ['price:1.00', 'price:10.00']
 
-Commit & optimize shortcuts
-  solr.commit
-  solr.optimize
+===Commit / Optimize
+  solr.commit, :commit_attributes => {}
+  solr.optimize, :optimize_attributes => {}
 
 == Response Formats
 The default response format is Ruby. When the :wt param is set to :ruby, the response is eval'd resulting in a Hash. You can get a raw response by setting the :wt to "ruby" - notice, the string -- not a symbol. RSolr will eval the Ruby string ONLY if the :wt value is :ruby. All other response formats are available as expected, :wt=>'xml' etc..
 
 ===Evaluated Ruby (default)
-  solr.get 'select', :wt=>:ruby # notice :ruby is a Symbol
+  solr.get 'select', :params => {:wt => :ruby} # notice :ruby is a Symbol
 ===Raw Ruby
-  solr.get 'select', :wt=>'ruby' # notice 'ruby' is a String
+  solr.get 'select', :params => {:wt => 'ruby'} # notice 'ruby' is a String
 
 ===XML:
-  solr.get 'select', :wt=>:xml
+  solr.get 'select', :params => {:wt => :xml}
 ===JSON:
-  solr.get 'select', :wt=>:json
-
-You can access the original request context (path, params, url etc.) by calling the #request method:
-  result = solr.get 'select', :q=>'*:*'
-  result.request[:uri]
-  result.request[:params]
-  etc..
+  solr.get 'select', :params => {:wt => :json}
 
-Similarly, the object returned has a response object. This contains any headers that Solr returned, along with the raw response body:
-  result = solr.get 'select', :q=>'*:*'
-  result.response[:headers]
-  result.response[:status]
-  result.response[:body]
+==Http Request Methods: +get+, +post+, and +head+ 
+RSolr can send GET, POST and HEAD requests to Solr:
+  response = solr.head "admin"
 
 ==Related Resources & Projects
 * {RSolr Google Group}[http://groups.google.com/group/rsolr] -- The RSolr discussion group
@@ -139,6 +150,7 @@ Similarly, the object returned has a response object. This contains any headers
 * Send me a pull request. Bonus points for topic branches.
 
 ==Contributors
+* Colin Steele
 * Lorenzo Riccucci
 * Mike Perham
 * Mat Brown
@@ -155,4 +167,4 @@ Matt Mitchell <mailto:goodieboy@gmail.com>
 
 ==Copyright
 
-Copyright (c) 2008-2010 mwmitchell. See LICENSE for details.
+Copyright (c) 2008-2010 Matt Mitchell. See LICENSE for details.

data/VERSION

@@ -1 +1 @@
-1.0.0.beta
+1.0.0.beta2

data/lib/rsolr/client.rb

@@ -6,43 +6,55 @@ class RSolr::Client
     @connection = connection
   end
   
-  # GET request
-  def get path = '', params = nil, headers = nil
-    send_request :get, path, params, nil, headers
-  end
-  
-  # essentially a GET, but no response body
-  def head path = '', params = nil, headers = nil
-    send_request :head, path, params, nil, headers
+  %W(get post head).each do |meth|
+    class_eval <<-RUBY
+    def #{meth} path, opts = {}, &block
+      send_request path, opts.merge(:method => :#{meth}), &block
+    end
+    RUBY
   end
   
-  # A path is required for a POST since, well...
-  # the / resource doesn't do anything with a POST.
-  # Also, Solr doesn't do headers with a POST
-  def post path, data = nil, params = nil, headers = nil
-    send_request :post, path, params, data, headers
+  def method_missing name, *args
+    send_request name, *args
   end
   
   # POST XML messages to /update with optional params
-  def update data, params = {}, headers = {}
-    headers['Content-Type'] ||= 'text/xml'
-    post 'update', data, params, headers
+  #
+  # If not set, opts[:headers] will be set to a hash with the key
+  # 'Content-Type' set to 'text/xml'
+  #
+  # +opts+ can/should contain:
+  #
+  #  :data - posted data
+  #  :headers - http headers
+  #  :params - query parameter hash
+  #
+  def update opts = {}
+    opts[:headers] ||= {}
+    opts[:headers]['Content-Type'] ||= 'text/xml'
+    post 'update', opts
   end
   
   # 
+  # +add+ creates xml "add" documents and sends the xml data to the +update+ method
   # single record:
   # solr.update(:id=>1, :name=>'one')
   #
   # update using an array
-  # solr.update([{:id=>1, :name=>'one'}, {:id=>2, :name=>'two'}])
-  #
-  def add(doc, params={}, &block)
-    update xml.add(doc, params, &block)
+  # 
+  # solr.update(
+  #   [{:id=>1, :name=>'one'}, {:id=>2, :name=>'two'}],
+  #   :add_attributes => {:boost=>5.0, :commitWithin=>10}
+  # )
+  # 
+  def add doc, opts = {}
+    add_attributes = opts.delete :add_attributes
+    update opts.merge(:data => xml.add(doc, add_attributes))
   end
 
-  # send "commit" xml with options
+  # send "commit" xml with opts
   #
-  # Options recognized by solr
+  # opts recognized by solr
   #
   #   :maxSegments    => N - optimizes down to at most N number of segments
   #   :waitFlush      => true|false - do not return until changes are flushed to disk
@@ -51,13 +63,14 @@ class RSolr::Client
   #
   # *NOTE* :expungeDeletes is Solr 1.4 only
   #
-  def commit( options = {} )
-    update xml.commit( options )
+  def commit opts = {}
+    commit_attrs = opts.delete :commit_attributes
+    update opts.merge(:data => xml.commit( commit_attrs ))
   end
 
-  # send "optimize" xml with options.
+  # send "optimize" xml with opts.
   #
-  # Options recognized by solr
+  # opts recognized by solr
   #
   #   :maxSegments    => N - optimizes down to at most N number of segments
   #   :waitFlush      => true|false - do not return until changes are flushed to disk
@@ -66,28 +79,29 @@ class RSolr::Client
   #
   # *NOTE* :expungeDeletes is Solr 1.4 only
   #
-  def optimize( options = {} )
-    update xml.optimize( options )
+  def optimize opts = {}
+    optimize_attributes = opts.delete :optimize_attributes
+    update opts.merge(:data => xml.optimize(optimize_attributes))
   end
-
+  
   # send </rollback>
   # NOTE: solr 1.4 only
-  def rollback
-    update xml.rollback
+  def rollback opts = {}
+    update opts.merge(:data => xml.rollback)
   end
 
   # Delete one or many documents by id
   #   solr.delete_by_id 10
   #   solr.delete_by_id([12, 41, 199])
-  def delete_by_id(id)
-    update xml.delete_by_id(id)
+  def delete_by_id id, opts = {}
+    update opts.merge(:data => xml.delete_by_id(id))
   end
 
   # delete one or many documents by query
   #   solr.delete_by_query 'available:0'
   #   solr.delete_by_query ['quantity:0', 'manu:"FQ"']
-  def delete_by_query(query)
-    update xml.delete_by_query(query)
+  def delete_by_query query, opts = {}
+    update opts.merge(:data => xml.delete_by_query(query))
   end
   
   # shortcut to RSolr::Message::Generator
@@ -95,62 +109,33 @@ class RSolr::Client
     @xml ||= RSolr::Xml::Generator.new
   end
   
-  def send_request method, path, params, data, headers
-    params = map_params params
-    uri, data, headers = build_request path, params, data, headers
-    request_context = {:connection=>connection, :method => method, :uri => uri, :data => data, :headers => headers, :params => params}
-    begin
-      response = data ? connection.send(method, uri, data, headers) : connection.send(method, uri, headers)
-    rescue
-     $!.extend(RSolr::Error::SolrContext).request = request_context
-     raise $!
-    end
-    raise "The connection adapter returned an unexpected object" unless response.is_a?(Hash)
-    raise RSolr::Error::Http.new request_context, response unless [200,302].include?(response[:status])
-    adapt_response request_context, response
-  end
-  
-  def map_params params
-    params = params.nil? ? {} : params.dup
-    params[:wt] ||= :ruby
-    params
+  # +send_request+ is the main request method.
+  # 
+  # "path" : A string value that usually represents a solr request handler
+  # "opt" : A hash, which can contain the following keys:
+  #   :method : required - the http method (:get, :post or :head)
+  #   :params : optional - the query string params in hash form
+  #   :data : optional - post data -- if a hash is given, it's sent as "application/x-www-form-urlencoded"
+  #   :headers : optional - hash of request headers
+  # All other options are passed right along to the connection request method (:get, :post, or :head)
+  # 
+  # +send_request+ returns either a string or hash on a successful ruby request.
+  # When the :params[:wt] => :ruby, the response will be a hash, else a string.
+  # 
+  def send_request path, opts = {}
+    connection.send_request path, opts
   end
   
-  def build_request path, params, data, headers
-    params ||= {}
-    headers ||= {}
-    request_uri = params.any? ? "#{path}?#{RSolr::Uri.params_to_solr params}" : path
-    if data
-      if data.is_a? Hash
-        data = RSolr::Uri.params_to_solr data
-        headers['Content-Type'] ||= 'application/x-www-form-urlencoded'
-      end
-    end
-    [request_uri, data, headers]
+  # used for debugging/inspection
+  # - accepts the same args as send_request
+  def build_request path, opts = {}
+    connection.build_request path, opts
   end
   
-  # This method will evaluate the :body value
-  # if the params[:uri].params[:wt] == :ruby
-  # ... otherwise, the body is returned as is.
-  # The return object has a special method attached called #context.
-  # This method gives you access to the original
-  # request and response from the connection.
-  # This method will raise an InvalidRubyResponse
-  # if the :wt => :ruby and the body
-  # couldn't be evaluated.
-  def adapt_response request, response
-    data = response[:body]
-    if request[:params][:wt] == :ruby
-      begin
-        data = Kernel.eval data.to_s
-      rescue SyntaxError
-        raise RSolr::Error::InvalidRubyResponse.new request, response
-      end
-    end
-    data.extend Module.new.instance_eval{attr_accessor :request, :response; self}
-    data.request = request
-    data.response = response
-    data
+  # used for debugging/inspection
+  # - accepts the same args as send_request
+  def adapt_response request_context, response
+    connection.adapt_response request_context, response
   end
   
 end

data/lib/rsolr/connectable.rb

@@ -0,0 +1,95 @@
+# Connectable is designed to be shared across solr driver implementations.
+# If the driver uses an http url/proxy and returns the standard http respon
+# data (status, body, headers) then this module could be used. 
+module RSolr::Connectable
+  
+  attr_reader :uri, :proxy, :options
+  
+  def initialize options = {}
+    url = options[:url] || 'http://127.0.0.1:8983/solr/'
+    url << "/" unless url[-1] == ?/
+    proxy_url = options[:proxy]
+    proxy_url << "/" unless proxy_url.nil? or proxy_url[-1] == ?/
+    @uri = RSolr::Uri.create url
+    @proxy = RSolr::Uri.create proxy_url if proxy_url
+    @options = options
+  end
+  
+  # 
+  def base_request_uri
+    base_uri.request_uri
+  end
+  
+  def base_uri
+    @proxy || @uri
+  end
+  
+  # creates a request context hash,
+  # sends it to the connection.execute method
+  # which returns a simple hash,
+  # then passes the request/response into adapt_response.
+  def send_request path, opts
+    request_context = build_request path, opts
+    raw_response = execute request_context
+    adapt_response request_context, raw_response
+  end
+  
+  # all connection imlementations that use this mixin need to create an execute method 
+  def execute request_context
+    raise "You gotta implement this method and return a hash like => {:status => <integer>, :body => <string>, :headers => <hash>}"
+  end
+  
+  # build_request sets up the uri/query string
+  # and converts the +data+ arg to form-urlencoded
+  # if the +data+ arg is a hash.
+  # returns a hash with the following keys:
+  #   :method
+  #   :params
+  #   :headers
+  #   :data
+  #   :uri
+  #   :path
+  #   :query
+  def build_request path, opts
+    opts[:method] ||= :get
+    raise "The :data option can only be used if :method => :post" if opts[:method] != :post and opts[:data]
+    opts[:params] = opts[:params].nil? ? {:wt => :ruby} : opts[:params].merge(:wt => :ruby)
+    query = RSolr::Uri.params_to_solr(opts[:params]) unless opts[:params].empty?
+    opts[:query] = query
+    if opts[:data].is_a? Hash
+      opts[:data] = RSolr::Uri.params_to_solr opts[:data]
+      opts[:headers] ||= {}
+      opts[:headers]['Content-Type'] ||= 'application/x-www-form-urlencoded'
+    end
+    opts[:path] = path
+    opts[:uri] = base_uri.merge(path.to_s + (query ? "?#{query}" : "")) if base_uri
+    opts
+  end
+  
+  # This method will evaluate the :body value
+  # if the params[:uri].params[:wt] == :ruby
+  # ... otherwise, the body is returned as is.
+  # The return object has methods attached, :request and :response.
+  # These methods give you access to the original
+  # request and response from the connection.
+  #
+  # +adapt_response+ will raise an InvalidRubyResponse
+  # if :wt == :ruby and the body
+  # couldn't be evaluated.
+  def adapt_response request, response
+    raise "The response does not have the correct keys => :body, :headers, :status" unless
+      %W(body headers status) == response.keys.map{|k|k.to_s}.sort
+    raise RSolr::Error::Http.new request, response unless
+      [200,302].include? response[:status]
+    data = response[:body]
+    if request[:params][:wt] == :ruby
+      begin
+        data = Kernel.eval data.to_s
+      rescue SyntaxError
+        raise RSolr::Error::InvalidRubyResponse.new request, response
+      end
+    end
+    data
+  end
+  
+end

data/lib/rsolr/error.rb

@@ -6,26 +6,17 @@ module RSolr::Error
     
     def to_s
       m = "#{super.to_s}"
-      
       if response
         m << " - #{response[:status]} #{Http::STATUS_CODES[response[:status].to_i]}"
         details = parse_solr_error_response response[:body]
-        m << "Error: #{details}\n" if details
-      end
-      
-      m << "\n" + self.backtrace[0..10].join("\n")
-      m << "\n\nSolr Request:"
-      m << "\n  Method: #{request[:method].to_s.upcase}"
-      m << "\n  Base URL: #{request[:connection].uri.to_s}"
-      m << "\n  URL: #{request[:uri]}"
-      m << "\n  Params: #{request[:params].inspect}"
-      m << "\n  Data: #{request[:data].inspect}" if request[:data]
-      m << "\n  Headers: #{request[:headers].inspect}"
-      if response
-        m << "\n\nSolr Response:"
-        m << "\n  Code: #{response[:status]}"
-        m << "\n  Headers: #{response[:headers].inspect}"
+        m << "\nError: #{details}\n" if details
       end
+      p = "\nQuery: #{request[:path]}?#{request[:query]}"
+      p = "\nRequest Headers: #{request[:headers].inspect}" if request[:headers]
+      p = "\nRequest Data: #{request[:data].inspect}" if request[:data]
+      p << "\n"
+      p << "\nBacktrace: " + self.backtrace[0..10].join("\n")
+      m << p
       m
     end