backupify-rsolr-nokogiri 0.12.1.1

data/LICENSE

@@ -0,0 +1,13 @@
+Copyright 2008-2010 Matt Mitchell
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

data/README.rdoc

@@ -0,0 +1,129 @@
+=RSolr
+
+A simple, extensible Ruby client for Apache Solr.
+
+== Installation:
+  gem sources -a http://gemcutter.org
+  sudo gem install rsolr
+
+== Example:
+  require 'rubygems'
+  require 'rsolr'
+  solr = RSolr.connect :url=>'http://solrserver.com'
+  
+  # send a request to /select
+  response = rsolr.select :q=>'*:*'
+  
+  # send a request to a custom request handler; /catalog
+  response = rsolr.request '/catalog', :q=>'*:*'
+  
+  # alternative to above:
+  response = rsolr.catalog :q=>'*:*'
+
+== Querying
+Use the #select method to send requests to the /select handler:
+  response = solr.select({
+    :q=>'washington',
+    :start=>0,
+    :rows=>10
+  })
+
+The params sent into the method are sent to Solr as-is. The one exception is if a value is an array. When an array is used, multiple parameters *with the same name* are generated for the Solr query. Example:
+  
+  solr.select :q=>'roses', :fq=>['red', 'violet']
+
+The above statement generates this Solr query:
+  
+  ?q=roses&fq=red&fq=violet
+
+Use the #request method for a custom request handler path:
+  response = solr.request '/documents', :q=>'test'
+
+A shortcut for the above example use a method call instead:
+  response = solr.documents :q=>'test'
+
+
+== Updating Solr
+Updating uses native Ruby structures. Hashes are used for single documents and arrays are used for a collection of documents (hashes). These structures get turned into simple XML "messages". Raw XML strings can also be used.
+
+Raw XML via  #update
+  solr.update '</commit>'
+  solr.update '</optimize>'
+
+Single document via #add
+  solr.add :id=>1, :price=>1.00
+
+Multiple documents via #add
+  documents = [{:id=>1, :price=>1.00}, {:id=>2, :price=>10.50}]
+  solr.add documents
+
+When adding, you can also supply "add" xml element attributes and/or a block for manipulating other "add" related elements (docs and fields) when using the #add method:
+  
+  doc = {:id=>1, :price=>1.00}
+  add_attributes = {:allowDups=>false, :commitWithin=>10.0}
+  solr.add(doc, add_attributes) do |doc|
+    # boost each document
+    doc.attrs[:boost] = 1.5
+    # boost the price field:
+    doc.field_by_name(:price).attrs[:boost] = 2.0
+  end
+
+Delete by id
+  solr.delete_by_id 1
+or an array of ids
+  solr.delete_by_id [1, 2, 3, 4]
+
+Delete by query:
+  solr.delete_by_query 'price:1.00'
+Delete by array of queries
+  solr.delete_by_query ['price:1.00', 'price:10.00']
+
+Commit & optimize shortcuts
+  solr.commit
+  solr.optimize
+
+== Response Formats
+The default response format is Ruby. When the :wt param is set to :ruby, the response is eval'd resulting in a Hash. You can get a raw response by setting the :wt to "ruby" - notice, the string -- not a symbol. RSolr will eval the Ruby string ONLY if the :wt value is :ruby. All other response formats are available as expected, :wt=>'xml' etc..
+
+===Evaluated Ruby (default)
+  solr.select(:wt=>:ruby) # notice :ruby is a Symbol
+===Raw Ruby
+  solr.select(:wt=>'ruby') # notice 'ruby' is a String
+
+===XML:
+  solr.select(:wt=>:xml)
+===JSON:
+  solr.select(:wt=>:json)
+
+You can access the original request context (path, params, url etc.) by calling the #raw method:
+  response = solr.select :q=>'*:*'
+  response.raw[:status_code]
+  response.raw[:body]
+  response.raw[:url]
+
+The raw is a hash that contains the generated params, url, path, post data, headers etc., very useful for debugging and testing.
+
+==Related Resources & Projects
+* {RSolr Google Group}[http://groups.google.com/group/rsolr] -- The RSolr discussion group
+* {rsolr-ext}[http://github.com/mwmitchell/rsolr-ext] -- An extension kit for RSolr
+* {rsolr-direct}[http://github.com/mwmitchell/rsolr-direct] -- JRuby direct connection for RSolr
+* {SunSpot}[http://github.com/outoftime/sunspot] -- An awesome Solr DSL, built with RSolr
+* {Blacklight}[http://blacklightopac.org] -- A "next generation" Library OPAC, built with RSolr
+* {java_bin}[http://github.com/kennyj/java_bin] -- Provides javabin/binary parsing for RSolr
+* {Solr}[http://lucene.apache.org/solr/] -- The Apache Solr project
+* {solr-ruby}[http://wiki.apache.org/solr/solr-ruby] -- The original Solr Ruby Gem!
+
+== Note on Patches/Pull Requests
+* Fork the project.
+* Make your feature addition or bug fix.
+* Add tests for it. This is important so I don't break it in a future version unintentionally.
+* Commit, do not mess with rakefile, version, or history
+  (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
+* Send me a pull request. Bonus points for topic branches.
+
+==Contributors
+* mperham
+* Mat Brown
+* shairontoledo
+* Matthew Rudy
+* Fouad Mardini

data/Rakefile

@@ -0,0 +1,14 @@
+require 'rake'
+require 'rake/testtask'
+require 'rdoc/task'
+require 'rubygems/package_task'
+
+ENV['RUBYOPT'] = '-W1'
+ 
+task :environment do
+  require File.dirname(__FILE__) + '/lib/rsolr'
+end
+ 
+Dir['tasks/**/*.rake'].each { |t| load t }
+
+task :default => ['spec:api']

data/VERSION

@@ -0,0 +1 @@
+0.12.1.1

data/lib/rsolr.rb

@@ -0,0 +1,50 @@
+
+require 'rubygems'
+$:.unshift File.dirname(__FILE__) unless $:.include?(File.dirname(__FILE__))
+
+module RSolr
+  
+  def self.version
+    @version ||= File.read(File.join(File.dirname(__FILE__), '..', 'VERSION'))
+  end
+  
+  VERSION = self.version
+  
+  autoload :Message, 'rsolr/message'
+  autoload :Client, 'rsolr/client'
+  autoload :Connection, 'rsolr/connection'
+  
+  module Connectable
+    
+    def connect opts={}
+      Client.new Connection::NetHttp.new(opts)
+    end
+    
+  end
+  
+  extend Connectable
+  
+  # A module that contains string related methods
+  module Char
+    
+    # escape - from the solr-ruby library
+    # RSolr.escape('asdf')
+    # backslash everything that isn't a word character
+    def escape(value)
+      value.gsub(/(\W)/, '\\\\\1')
+    end
+    
+  end
+  
+  # send the escape method into the Connection class ->
+  # solr = RSolr.connect
+  # solr.escape('asdf')
+  RSolr::Client.send(:include, Char)
+  
+  # bring escape into this module (RSolr) -> RSolr.escape('asdf')
+  extend Char
+  
+  # RequestError is a common/generic exception class used by the adapters
+  class RequestError < RuntimeError; end
+  
+end

data/lib/rsolr/client.rb

@@ -0,0 +1,114 @@
+class RSolr::Client
+  
+  attr_reader :connection
+  
+  # "connection" is instance of:
+  #   RSolr::Adapter::HTTP
+  #   RSolr::Adapter::Direct (jRuby only)
+  # or any other class that uses the connection "interface"
+  def initialize(connection)
+    @connection = connection
+  end
+  
+  # Send a request to a request handler using the method name.
+  # Also proxies to the #paginate method if the method starts with "paginate_"
+  def method_missing(method_name, *args, &blk)
+    request("/#{method_name}", *args, &blk)
+  end
+  
+  # sends data to the update handler
+  # data can be a string of xml, or an object that returns xml from its #to_xml method
+  def update(data, params={})
+    request '/update', params, data
+  end
+  
+  # send request solr
+  # params is hash with valid solr request params (:q, :fl, :qf etc..)
+  #   if params[:wt] is not set, the default is :ruby
+  #   if :wt is something other than :ruby, the raw response body is used
+  #   otherwise, a simple Hash is returned
+  #   NOTE: to get raw ruby, use :wt=>'ruby' <- a string, not a symbol like :ruby  
+  #
+  #
+  def request(path, params={}, *extra)
+    response = @connection.request(path, map_params(params), *extra)
+    adapt_response(response)
+  end
+  
+  # 
+  # single record:
+  # solr.update(:id=>1, :name=>'one')
+  #
+  # update using an array
+  # solr.update([{:id=>1, :name=>'one'}, {:id=>2, :name=>'two'}])
+  #
+  def add(doc, &block)
+    update message.add(doc, &block)
+  end
+
+  # send </commit>
+  def commit
+    update message.commit
+  end
+
+  # send </optimize>
+  def optimize
+    update message.optimize
+  end
+
+  # send </rollback>
+  # NOTE: solr 1.4 only
+  def rollback
+    update message.rollback
+  end
+
+  # Delete one or many documents by id
+  #   solr.delete_by_id 10
+  #   solr.delete_by_id([12, 41, 199])
+  def delete_by_id(id)
+    update message.delete_by_id(id)
+  end
+
+  # delete one or many documents by query
+  #   solr.delete_by_query 'available:0'
+  #   solr.delete_by_query ['quantity:0', 'manu:"FQ"']
+  def delete_by_query(query)
+    update message.delete_by_query(query)
+  end
+  
+  # shortcut to RSolr::Message::Generator
+  def message *opts
+    @message ||= RSolr::Message::Generator.new
+  end
+  
+  protected
+  
+  # sets default params etc.. - could be used as a mapping hook
+  # type of request should be passed in here? -> map_params(:query, {})
+  def map_params(params)
+    params||={}
+    {:wt=>:ruby}.merge(params)
+  end
+
+  # "connection_response" must be a hash with the following keys:
+  #   :params - a sub hash of standard solr params
+  #   : body - the raw response body from the solr server
+  # This method will evaluate the :body value if the params[:wt] == :ruby
+  # otherwise, the body is returned
+  # The return object has a special method attached called #raw
+  # This method gives you access to the original response from the connection,
+  # so you can access things like the actual :url sent to solr,
+  # the raw :body, original :params and original :data
+  def adapt_response(connection_response)
+    data = connection_response[:body]
+    # if the wt is :ruby, evaluate the ruby string response
+    if connection_response[:params][:wt] == :ruby
+      data = Kernel.eval(data)
+    end
+    # attach a method called #raw that returns the original connection response value
+    def data.raw; @raw end
+    data.send(:instance_variable_set, '@raw', connection_response)
+    data
+  end
+  
+end

data/lib/rsolr/connection.rb

@@ -0,0 +1,9 @@
+require 'uri'
+
+module RSolr::Connection
+  
+  autoload :NetHttp, 'rsolr/connection/net_http'
+  autoload :Utils, 'rsolr/connection/utils'
+  autoload :Requestable, 'rsolr/connection/requestable'
+  
+end

data/lib/rsolr/connection/net_http.rb

@@ -0,0 +1,48 @@
+require 'net/http'
+
+#
+# Connection for standard HTTP Solr server
+#
+class RSolr::Connection::NetHttp
+  
+  include RSolr::Connection::Requestable
+  
+  def connection
+    @connection ||= Net::HTTP.new(@uri.host, @uri.port)
+  end
+  
+  def get path, params={}
+    url = self.build_url path, params
+    net_http_response = self.connection.get url
+    create_http_context net_http_response, url, path, params
+  end
+  
+  def post path, data, params={}, headers={}
+    url = self.build_url path, params
+    net_http_response = self.connection.post url, data, headers
+    create_http_context net_http_response, url, path, params, data, headers
+  end
+  
+  def create_http_context net_http_response, url, path, params, data=nil, headers={}
+    full_url = "#{@uri.scheme}://#{@uri.host}"
+    full_url += @uri.port ? ":#{@uri.port}" : ''
+    full_url += url
+    {
+      :status_code=>net_http_response.code.to_i,
+      :url=>full_url,
+      :body=> encode_utf8(net_http_response.body),
+      :path=>path,
+      :params=>params,
+      :data=>data,
+      :headers=>headers,
+      :message => net_http_response.message
+    }
+  end
+  
+  # accepts a path/string and optional hash of query params
+  def build_url path, params={}
+    full_path = @uri.path + path
+    super full_path, params, @uri.query
+  end
+  
+end

data/lib/rsolr/connection/requestable.rb

@@ -0,0 +1,43 @@
+# A module that defines the interface and top-level logic for http based connection classes.
+module RSolr::Connection::Requestable
+  
+  include RSolr::Connection::Utils
+  
+  attr_reader :opts, :uri
+  
+  # opts can have:
+  #   :url => 'http://localhost:8080/solr'
+  def initialize opts={}
+    opts[:url] ||= 'http://127.0.0.1:8983/solr'
+    @opts = opts
+    @uri = URI.parse opts[:url]
+  end
+  
+  # send a request to the connection
+  # request '/select', :q=>'*:*'
+  #
+  # request '/update', {:wt=>:xml}, '</commit>'
+  # 
+  # force a post where the post body is the param query
+  # request '/update', "<optimize/>", :method=>:post
+  #
+  def request path, params={}, *extra
+    opts = extra[-1].kind_of?(Hash) ? extra.pop : {}
+    data = extra[0]
+    # force a POST, use the query string as the POST body
+    if opts[:method] == :post and data.to_s.empty?
+      http_context = self.post(path, hash_to_query(params), {}, {'Content-Type' => 'application/x-www-form-urlencoded'})
+    else
+      if data
+        # standard POST, using "data" as the POST body
+        http_context = self.post(path, data, params, {"Content-Type" => 'text/xml; charset=utf-8'})
+      else
+        # standard GET
+        http_context = self.get(path, params)
+      end
+    end
+    raise RSolr::RequestError.new("Solr Response: #{http_context[:message]}") unless http_context[:status_code] == 200
+    http_context
+  end
+  
+end

data/lib/rsolr/connection/utils.rb

@@ -0,0 +1,73 @@
+# Helpful utility methods for building queries to a Solr server
+# This includes helpers that the Direct connection can use.
+module RSolr::Connection::Utils
+  
+  # Performs URI escaping so that you can construct proper
+  # query strings faster.  Use this rather than the cgi.rb
+  # version since it's faster.  (Stolen from Rack).
+  def escape(s)
+    s.to_s.gsub(/([^ a-zA-Z0-9_.-]+)/n) {
+      #'%'+$1.unpack('H2'*$1.size).join('%').upcase
+      '%'+$1.unpack('H2'*bytesize($1)).join('%').upcase
+    }.tr(' ', '+')
+  end
+  
+  # encodes the string as utf-8 in Ruby 1.9
+  # returns the unaltered string in Ruby 1.8
+  def encode_utf8 string
+    (string.respond_to?(:force_encoding) and string.respond_to?(:encoding)) ?
+      string.force_encoding(Encoding::UTF_8) : string
+  end
+  
+  # Return the bytesize of String; uses String#length under Ruby 1.8 and
+  # String#bytesize under 1.9.
+  if ''.respond_to?(:bytesize)
+    def bytesize(string)
+      string.bytesize
+    end
+  else
+    def bytesize(string)
+      string.size
+    end
+  end
+  
+  # creates and returns a url as a string
+  # "url" is the base url
+  # "params" is an optional hash of GET style query params
+  # "string_query" is an extra query string that will be appended to the 
+  # result of "url" and "params".
+  def build_url url='', params={}, string_query=''
+    queries = [string_query, hash_to_query(params)]
+    queries.delete_if{|i| i.to_s.empty?}
+    url += "?#{queries.join('&')}" unless queries.empty?
+    url
+  end
+
+  # converts a key value pair to an escaped string:
+  # Example:
+  # build_param(:id, 1) == "id=1"
+  def build_param(k,v)
+    "#{escape(k)}=#{escape(v)}"
+  end
+
+  #
+  # converts hash into URL query string, keys get an alpha sort
+  # if a value is an array, the array values get mapped to the same key:
+  #   hash_to_query(:q=>'blah', :fq=>['blah', 'blah'], :facet=>{:field=>['location_facet', 'format_facet']})
+  # returns:
+  #   ?q=blah&fq=blah&fq=blah&facet.field=location_facet&facet.field=format.facet
+  #
+  # if a value is empty/nil etc., it is not added
+  def hash_to_query(params)
+    mapped = params.map do |k, v|
+      next if v.to_s.empty?
+      if v.class == Array
+        hash_to_query(v.map { |x| [k, x] })
+      else
+        build_param k, v
+      end
+    end
+    mapped.compact.join("&")
+  end
+
+end