sphinx 0.9.10.2122 → 2.1.1.3711

data/.gitignore

@@ -1,4 +1,7 @@
-rdoc
-doc
 .yardoc
+.DS_Store
+.bundle
+.rvmrc
+Gemfile.lock
+doc
 pkg

data/.travis.yml

@@ -0,0 +1,8 @@
+rvm:
+  - 1.8.7
+  - 1.9.2
+  - ree
+
+notifications:
+  recipients:
+    - kpumuk@kpumuk.info

data/CHANGELOG.md

@@ -0,0 +1,71 @@
+## 2.1.1.3711 (Mar 15, 2013)
+
+Features:
+
+  - Updated API to the latest Sphinx version 2.1.1.
+  - Refactored specs so they don't require running Sphinx instance and php.
+
+## 0.9.10.2122 (Dec 04, 2009)
+
+Features:
+
+  - Sphinx::Client#escape_string method added.
+
+Bugfixes:
+
+  - Allow empty array or single integer in #set_filter as values.
+
+## 0.9.10.2094 (Nov 23, 2009)
+
+Features:
+
+  - Added logging.
+  - Added ability to pass a block to Client#query method to set request parameters.
+  - Use CRC32 of the request to select the server.
+  - Results returned in an instance of HashWithIndifferentAccess.
+
+## 0.9.10.2091 (Nov 20, 2009)
+
+Features:
+
+  - Added Ruby-style named methods in addition to native Sphinx API naming.
+  - Return `Sphinx::Client` object itself from any `set_` method to allow chaining.
+  - Status() API call queries all configured servers.
+
+## 0.9.10.2086 (Nov 19, 2009)
+
+Bugfixes:
+
+  - Better documentation.
+  - Fixed incomplete reply handling.
+  - Sphinx IANA assigned ports are 9312 and 9306 respectively (goodbye, trusty 3312)
+
+## 0.9.10.2043 (Nov 16, 2009)
+
+Features:
+
+  - Updated Sphinx API to version 0.9.10.
+  - Added ability to set multiple servers.
+
+Bugfixes:
+
+  - Better argument validation.
+  - Properly handle connection timeouts.
+  - Added request timeout handling and retries.
+  - Close TCP socket on connection failure.
+
+## 0.5.0.1112 (Aug 4, 2008)
+
+Features:
+
+  - Updated Sphinx API to version 0.9.9.
+
+Bugfixes:
+
+  - Fixed support of 64-bit values.
+
+## 0.4.0.1112 (May 2, 2008)
+
+Features:
+
+  - Initial implementation of Sphinx API for Sphinx 0.9.8.

data/Gemfile

@@ -0,0 +1,4 @@
+source 'http://rubygems.org'
+
+# Specify your gem's dependencies in meta-tags.gemspec
+gemspec

data/README.md

@@ -0,0 +1,221 @@
+# Sphinx Client API
+
+[![Travis-CI build status](https://secure.travis-ci.org/kpumuk/sphinx.png)](http://travis-ci.org/kpumuk/sphinx)
+
+This document gives an overview of what is Sphinx itself and how to use it
+from your Ruby on Rails application. For more information about Sphinx and
+its API documentation visit [sphinxsearch.com](http://www.sphinxsearch.com).
+
+## Sphinx
+
+Sphinx is a standalone full-text search engine, meant to provide fast,
+size-efficient and relevant fulltext search functions to other applications.
+Sphinx was specially designed to integrate well with SQL databases and
+scripting languages. Currently built-in data sources support fetching data
+either via direct connection to MySQL, or from an XML pipe.
+
+Simplest way to communicate with Sphinx is to use `searchd` —
+a daemon to search through full text indexes from external software.
+
+## Installation
+
+Add the "sphinx" gem to your `Gemfile`.
+
+    gem 'sphinx'
+
+And run `bundle install` command.
+
+## Documentation
+
+Complete Sphinx plugin documentation could be found on [GitHub Pages](http://kpumuk.github.com/sphinx).
+
+Also you can find documentation on [rdoc.info](http://rdoc.info/projects/kpumuk/sphinx).
+
+You can build the documentation locally by running:
+
+    rake yard
+
+Complete Sphinx API documentation could be found on [Sphinx Search Engine
+site](http://www.sphinxsearch.com/docs/current.html).
+This plugin is fully compatible with original PHP API implementation.
+
+## Ruby naming conventions
+
+Sphinx Client API supports Ruby naming conventions, so every API
+method name is in underscored, lowercase form:
+
+    SetServer    -> set_server
+    RunQueries   -> run_queries
+    SetMatchMode -> set_match_mode
+
+Every method is aliased to a corresponding one from standard Sphinx
+API, so you can use both `SetServer` and `set_server`
+with no differrence.
+
+There are three exceptions to this naming rule:
+
+    GetLastError   -> last_error
+    GetLastWarning -> last_warning
+    IsConnectError -> connect_error?
+
+Of course, all of them are aliased to the original method names.
+
+## Using multiple Sphinx servers
+
+Since we actively use this plugin in our Scribd development workflow,
+there are several methods have been added to accommodate our needs.
+You can find documentation on Ruby-specific methods in [documentation](http://rdoc.info/projects/kpumuk/sphinx).
+
+First of all, we added support of multiple Sphinx servers to balance
+load between them. Also it means that in case of any problems with one
+of servers, library will try to fetch the results from another one.
+Every consequence request will be executed on the next server in list
+(round-robin technique).
+
+    sphinx.set_servers([
+      { :host => 'browse01.local', :port => 3312 },
+      { :host => 'browse02.local', :port => 3312 },
+      { :host => 'browse03.local', :port => 3312 }
+    ])
+
+By default library will try to fetch results from a single server, and
+fail if it does not respond. To setup number of retries being performed,
+you can use second (additional) parameter of the `set_connect_timeout`
+and `set_request_timeout` methods:
+
+    sphinx.set_connect_timeout(1, 3)
+    sphinx.set_request_timeout(1, 3)
+
+There is a big difference between these two methods. First will affect
+only on requests experiencing problems with connection (socket error,
+pipe error, etc), second will be used when request is broken somehow
+(temporary searchd error, incomplete reply, etc). The workflow looks like
+this:
+
+1. Increase retries number. If is less or equal to configured value,
+   try to connect to the next server. Otherwise, raise an error.
+2. In case of connection problem go to 1.
+3. Increase request retries number. If it less or equal to configured
+   value, try to perform request. Otherwise, raise an error.
+4. In case of connection problem go to 1.
+5. In case of request problem, go to 3.
+6. Parse and return response.
+
+Withdrawals:
+
+1. Request could be performed `connect_retries` * `request_retries`
+   times. E.g., it could be tried `request_retries` times on each
+   of `connect_retries` servers (when you have 1 server configured,
+   but `connect_retries` is 5, library will try to connect to this
+   server 5 times).
+2. Request could be tried to execute on each server `1..request_retries`
+   times. In case of connection problem, request will be moved to another
+   server immediately.
+
+Usually you will set `connect_retries` equal to servers number,
+so you will be sure each failing request will be performed on all servers.
+This means that if one of servers is live, but others are dead, you request
+will be finally executed successfully.
+
+## Sphinx constants
+
+Most Sphinx API methods expecting for special constants will be passed.
+For example:
+
+    sphinx.set_match_mode(Sphinx::SPH_MATCH_ANY)
+
+Please note that these constants defined in a `Sphinx`
+module. You can use symbols or strings instead of these awful
+constants:
+
+    sphinx.set_match_mode(:any)
+    sphinx.set_match_mode('any')
+
+## Setting query filters
+
+Every `set_` method returns `Sphinx::Client` object itself.
+It means that you can chain filtering methods:
+
+    results = Sphinx::Client.new.
+                set_match_mode(:any).
+                set_ranking_mode(:bm25).
+                set_id_range(10, 1000).
+                query('test')
+
+There is a handful ability to set query parameters directly in `query`
+call. If block does not accept any parameters, it will be eval'ed inside
+Sphinx::Client instance:
+
+    results = Sphinx::Client.new.query('test') do
+      match_mode :any
+      ranking_mode :bm25
+      id_range 10, 1000
+    end
+
+As you can see, in this case you can omit the `set_` prefix for
+this methods. If block accepts a parameter, sphinx instance will be
+passed into the block. In this case you should you full method names
+including the `set_` prefix:
+
+    results = Sphinx::Client.new.query('test') do |sphinx|
+      sphinx.set_match_mode :any
+      sphinx.set_ranking_mode :bm25
+      sphinx.set_id_range 10, 1000
+    end
+
+## Example
+
+This simple example illustrates base connection establishing,
+search results retrieving, and excerpts building. Please note
+how does it perform database select using ActiveRecord to
+save the order of records established by Sphinx.
+
+    sphinx = Sphinx::Client.new
+    result = sphinx.query('test')
+    ids = result['matches'].map { |match| match['id'] }
+    posts = Post.all :conditions => { :id => ids },
+                     :order => "FIELD(id,#{ids.join(',')})"
+
+    docs = posts.map(&:body)
+    excerpts = sphinx.build_excerpts(docs, 'index', 'test')
+
+## Logging
+
+You can ask Sphinx client API to log it's activity to some log. In
+order to do that you can pass a logger object into the `Sphinx::Client`
+constructor:
+
+    require 'logger'
+    Sphinx::Client.new(Logger.new(STDOUT)).query('test')
+
+Logger object should respond to methods :debug, :info, and :warn, and
+accept blocks (this is what standard Ruby `Logger` class does).
+Here is what you will see in your log:
+
+* `DEBUG` -- `query`, `add_query`, `run_queries`
+  method calls with configured filters.
+* `INFO` -- initialization with Sphinx version, servers change,
+  attempts to re-connect, and all attempts to do an API call with server
+  where request being performed.
+* `WARN` -- various connection and socket errors.
+
+## Support
+
+You can find source code for this library on [GitHub](http://github.com/kpumuk/sphinx).
+
+To suggest a feature or report a bug use [GitHub Issues](http://github.com/kpumuk/sphinx/issues)
+
+## Credits
+
+* [Dmytro Shteflyuk](https://github.com/kpumuk) (author)
+* [Andrew Aksyonoff](http://sphinxsearch.com) (Sphinx core developer)
+
+Special thanks to [Alexey Kovyrin](https://github.com/kovyrin)
+
+Special thanks to [Mike Perham](https://github.com/mperham) for his awesome
+memcache-client gem, where latest Sphinx gem got new sockets handling from.
+
+## License
+
+This library is distributed under the terms of the Ruby license.
+You can freely distribute/modify this library.

data/Rakefile

@@ -1,45 +1,50 @@
-require 'rake'
-
-begin
-  require 'jeweler'
-  Jeweler::Tasks.new do |gemspec|
-    gemspec.name        = 'sphinx'
-    gemspec.summary     = 'Sphinx Client API for Ruby'
-    gemspec.description = 'An easy interface to Sphinx standalone full-text search engine. It is implemented as plugin for Ruby on Rails, but can be easily used as standalone library.'
-    gemspec.email       = 'kpumuk@kpumuk.info'
-    gemspec.homepage    = 'http://github.com/kpumuk/sphinx'
-    gemspec.authors     = ['Dmytro Shteflyuk']
+require 'bundler'
+Bundler::GemHelper.install_tasks
+
+require 'rspec/core/rake_task'
+RSpec::Core::RakeTask.new(:spec)
+
+task :test => :spec
+task :default => :spec
+
+require 'yard'
+YARD::Rake::YardocTask.new(:yard) do |t|
+    t.options = ['--title', 'Sphinx Client API Documentation']
+  if ENV['PRIVATE']
+    t.options.concat ['--protected', '--private']
+  else
+    t.options.concat ['--protected', '--no-private']
   end
-  Jeweler::GemcutterTasks.new
-rescue LoadError
-  puts 'Jeweler not available. Install it with: sudo gem install jeweler'
 end
 
-begin
-  require 'spec/rake/spectask'
+require 'bundler'
+Bundler::GemHelper.install_tasks
 
-  desc 'Default: run specs'
-  task :default => :spec
+namespace :fixtures do
+  FIXTURES_DIR = File.expand_path('../spec/fixtures', __FILE__)
 
-  desc 'Test the sphinx plugin'
-  Spec::Rake::SpecTask.new do |t|
-    t.libs << 'lib'
-    t.pattern = 'spec/*_spec.rb'
+  desc 'Update textures for sphinx requests'
+  task :requests do
+    rm Dir.glob("#{FIXTURES_DIR}/requests/*.dat")
+    Dir["#{FIXTURES_DIR}/requests/php/*.php"].each do |file|
+      puts name = File.basename(file, '.php')
+      File.open(File.join(File.dirname(file), '..', "#{name}.dat"), 'w') do |f|
+        f.write `env SPHINX_MOCK_REQUEST=1 php "#{file}"`
+      end
+    end
   end
-rescue LoadError
-  puts 'RSpec not available. Install it with: sudo gem install rspec'
-end
 
-begin
-  require 'yard'
-  YARD::Rake::YardocTask.new(:yard) do |t|
-    t.options = ['--title', 'Sphinx Client API Documentation']
-    if ENV['PRIVATE']
-      t.options.concat ['--protected', '--private']
-    else
-      t.options << '--no-private' 
+  desc 'Update textures for sphinx responses'
+  task :responses do
+    rm Dir.glob("#{FIXTURES_DIR}/responses/*.dat")
+    Dir["#{FIXTURES_DIR}/responses/php/*.php"].each do |file|
+      puts name = File.basename(file, '.php')
+      File.open(File.join(File.dirname(file), '..', "#{name}.dat"), 'w') do |f|
+        f.write `env SPHINX_MOCK_RESPONSE=1 php "#{file}"`
+      end
     end
   end
-rescue LoadError
-  puts 'Yard not available. Install it with: sudo gem install yard'
 end
+
+desc 'Update binary fixtures'
+task :fixtures => %w[ fixtures:requests fixtures:responses]

data/lib/sphinx.rb

@@ -1,9 +1,9 @@
 # Sphinx Client API
 #
 # Author::    Dmytro Shteflyuk <mailto:kpumuk@kpumuk.info>.
-# Copyright:: Copyright (c) 2006 — 2009 Dmytro Shteflyuk
+# Copyright:: Copyright (c) 2006 — 2013 Dmytro Shteflyuk
 # License::   Distributes under the same terms as Ruby
-# Version::   0.9.10-r2122
+# Version::   0.9.10.2122
 # Website::   http://kpumuk.info/projects/ror-plugins/sphinx
 # Sources::   http://github.com/kpumuk/sphinx
 #
@@ -11,11 +11,6 @@
 # You can freely distribute/modify this library.
 #
 module Sphinx
-  VERSION = begin
-    config = YAML.load(File.read(File.dirname(__FILE__) + '/../VERSION.yml'))
-    "#{config[:major]}.#{config[:minor]}.#{config[:patch]}.#{config[:build]}"
-  end
-
   # Base class for all Sphinx errors
   class SphinxError < StandardError; end
 
@@ -40,8 +35,8 @@ require 'socket'
 require 'zlib'
 
 path = File.dirname(__FILE__)
-require "#{path}/sphinx/constants"
 require "#{path}/sphinx/indifferent_access"
+require "#{path}/sphinx/constants"
 require "#{path}/sphinx/request"
 require "#{path}/sphinx/response"
 require "#{path}/sphinx/timeout"

data/lib/sphinx/client.rb

@@ -14,7 +14,7 @@ module Sphinx
   #
   class Client
     include Sphinx::Constants
-    
+
     #=================================================================
     # Some internal attributes to use inside client API
     #=================================================================
@@ -66,10 +66,17 @@ module Sphinx
       @anchor        = []                      # geographical anchor point
       @indexweights  = []                      # per-index weights
       @ranker        = SPH_RANK_PROXIMITY_BM25 # ranking mode (default is SPH_RANK_PROXIMITY_BM25)
+      @rankexpr      = ''                      # ranking expression
       @maxquerytime  = 0                       # max query time, milliseconds (default is 0, do not limit)
       @fieldweights  = {}                      # per-field-name weights
       @overrides     = []                      # per-query attribute values overrides
       @select        = '*'                     # select-list (attributes or expressions, with optional aliases)
+      @query_flags   = 0
+      @predictedtime = 0
+      @outerorderby  = ''
+      @outeroffset   = 0
+      @outerlimit    = 0
+      @hasouter      = false
 
       # per-reply fields (for single-query case)
       @error         = ''                      # last error message
@@ -107,8 +114,8 @@ module Sphinx
         :overrides => @overrides,
         :select => @select,
         :match_mode => @mode,
-        :ranking_mode => @ranker,
-        :sort_mode => { :mode => @sort, :sortby => @sortby },
+        :ranking => { :mode => @ranker, :expression => @rankexpr },
+        :sort_mode => { :mode => @sort, :sort_by => @sortby },
         :weights => @weights,
         :field_weights => @fieldweights,
         :index_weights => @indexweights,
@@ -116,11 +123,12 @@ module Sphinx
         :filters => @filters,
         :geo_anchor => @anchor,
         :group_by => { :attribute => @groupby, :func => @groupfunc, :sort => @groupsort },
-        :group_distinct => @groupdistinct
+        :group_distinct => @groupdistinct,
+        :query_flags => { :bitset => @query_flags, :predicted_time => @predictedtime },
+        :outer_select => { :has_outer => @hasouter, :sort_by => @outerorderby, :offset => @outeroffset, :limit => @outerlimit},
       }
 
-      "<Sphinx::Client: %d servers, params: %s>" %
-        [@servers.length, params.inspect]
+      "<Sphinx::Client: %d servers, params: %s>" % [@servers.length, params.inspect]
     end
 
     #=================================================================
@@ -609,6 +617,61 @@ module Sphinx
     end
     alias :SetSelect :set_select
 
+    # Allows to control a number of per-query options.
+    #
+    # Supported options and respectively allowed values are:
+    #
+    # * +reverse_scan+ -- +0+ or +1+, lets you control the order in which full-scan query processes the rows.
+    # * +sort_method+ -- +"pq"+ (priority queue, set by default) or +"kbuffer"+
+    #   (gives faster sorting for already pre-sorted data, e.g. index data sorted by id).
+    #   The result set is in both cases the same; picking one option or the other
+    #   may just improve (or worsen!) performance.
+    # * +boolean_simplify+ -- +false+ or +true+, enables simplifying the query to speed it up.
+    # * +idf+ -- either +"normalized"+ (default) or +"plain"+.
+    #
+    # @param [String] flag_name name of the option to set value for
+    # @param [Object] flag_value value to set
+    # @return [Sphinx::Client] self.
+    #
+    # @see http://sphinxsearch.com/docs/current.html#sphinxql-select
+    # @see http://sphinxsearch.com/docs/current.html#conf-predicted-time-costs
+    #
+    def set_query_flag(flag_name, flag_value)
+      raise ArgumentError, 'unknown "flag_name" argument value' unless QUERY_FLAGS.has_key?(flag_name)
+
+      flag   = QUERY_FLAGS[flag_name]
+      values = QUERY_FLAGS[flag_name][:values]
+
+      if flag_name.to_s == 'max_predicted_time'
+        raise ArgumentError, "\"flag_value\" should be a positive integer for \"max_predicted_time\" flag" unless flag_value.kind_of?(Integer) and flag_value >= 0
+
+        @predictedtime = flag_value
+      elsif !values.include?(flag_value)
+        raise ArgumentError, "unknown \"flag_value\", should be one of #{values.inspect}"
+      end
+
+      is_set = values.respond_to?(:call) ? values.call(flag_value) : values.index(flag_value) == 1
+      @query_flags = set_bit(@query_flags, flag[:index], is_set)
+      self
+    end
+    alias :SetQueryFlag set_query_flag
+
+    def set_outer_select(orderby, offset, limit)
+      raise ArgumentError, '"orderby" argument must be String' unless orderby.kind_of?(String)
+      raise ArgumentError, '"offset" argument must be Integer' unless offset.kind_of?(Integer)
+      raise ArgumentError, '"limit" argument must be Integer'  unless limit.kind_of?(Integer)
+
+      raise ArgumentError, '"offset" argument should be greater or equal to zero' unless offset >= 0
+      raise ArgumentError, '"limit" argument should be greater to zero'           unless limit > 0
+
+      @outerorderby = orderby
+      @outeroffset = offset
+      @outerlimit = limit
+      @hasouter = true
+      self
+    end
+    alias :SetOuterSelect set_outer_select
+
     #=================================================================
     # Full-text search query settings
     #=================================================================
@@ -655,17 +718,33 @@ module Sphinx
     # matching mode at the time of this writing. Parameter must be a
     # constant specifying one of the known modes.
     #
+    # By default, in the +EXTENDED+ matching mode Sphinx computes two
+    # factors which contribute to the final match weight. The major
+    # part is a phrase proximity value between the document text and
+    # the query. The minor part is so-called BM25 statistical function,
+    # which varies from 0 to 1 depending on the keyword frequency within
+    # document (more occurrences yield higher weight) and within the whole
+    # index (more rare keywords yield higher weight).
+    #
+    # However, in some cases you'd want to compute weight differently - or
+    # maybe avoid computing it at all for performance reasons because you're
+    # sorting the result set by something else anyway. This can be accomplished
+    # by setting the appropriate ranking mode.
+    #
     # You can specify ranking mode as String ("proximity_bm25", "bm25", etc),
     # Symbol (:proximity_bm25, :bm25, etc), or
     # Fixnum constant (SPH_RANK_PROXIMITY_BM25, SPH_RANK_BM25, etc).
     #
     # @param [Integer, String, Symbol] ranker ranking mode.
+    # @param [String] rankexpr  ranking formula to use with the expression
+    #   based ranker (+SPH_RANK_EXPR+).
     # @return [Sphinx::Client] self.
     #
     # @example
     #   sphinx.set_ranking_mode(Sphinx::SPH_RANK_BM25)
     #   sphinx.set_ranking_mode(:bm25)
     #   sphinx.set_ranking_mode('bm25')
+    #   sphinx.set_ranking_mode(:expr, 'sum(lcs*user_weight)*1000+bm25')
     #
     # @raise [ArgumentError] Occurred when parameters are invalid.
     #
@@ -673,21 +752,26 @@ module Sphinx
     # @see http://www.sphinxsearch.com/docs/current.html#api-func-setmatchmode Section 6.3.1, "SetMatchMode"
     # @see http://www.sphinxsearch.com/docs/current.html#api-func-setrankingmode Section 6.3.2, "SetRankingMode"
     #
-    def set_ranking_mode(ranker)
+    def set_ranking_mode(ranker, rankexpr = '')
       case ranker
         when String, Symbol
-          begin
-            ranker = self.class.const_get("SPH_RANK_#{ranker.to_s.upcase}")
-          rescue NameError
+          const_name = "SPH_RANK_#{ranker.to_s.upcase}"
+          unless self.class.const_defined?(const_name)
             raise ArgumentError, "\"ranker\" argument value \"#{ranker}\" is invalid"
           end
+
+          ranker = self.class.const_get(const_name)
         when Fixnum
           raise ArgumentError, "\"ranker\" argument value \"#{ranker}\" is invalid" unless (SPH_RANK_PROXIMITY_BM25..SPH_RANK_SPH04).include?(ranker)
         else
           raise ArgumentError, '"ranker" argument must be Fixnum, String, or Symbol'
       end
 
+      raise ArgumentError, '"rankexpr" argument must be String' unless rankexpr.kind_of?(String)
+      raise ArgumentError, '"rankexpr" should not be empty if ranker is SPH_RANK_EXPR' if ranker == SPH_RANK_EXPR and rankexpr.empty?
+
       @ranker = ranker
+      @rankexpr = rankexpr
       self
     end
     alias :SetRankingMode :set_ranking_mode
@@ -1259,6 +1343,22 @@ module Sphinx
     end
     alias :ResetOverrides :reset_overrides
 
+    def reset_query_flag
+      @query_flags = 0
+      @predictedtime = 0
+      self
+    end
+    alias :ResetQueryFlag :reset_query_flag
+
+    def reset_outer_select
+      @outerorderby = ''
+      @outeroffset = 0
+      @outerlimit = 0
+      @hasouter = 0
+      self
+    end
+    alias :ResetOuterSelect :reset_outer_select
+
     # Connects to searchd server, runs given search query with
     # current settings, obtains and returns the result set.
     #
@@ -1473,7 +1573,12 @@ module Sphinx
 
       # mode and limits
       request = Request.new
-      request.put_int @offset, @limit, @mode, @ranker, @sort
+      request.put_int @query_flags, @offset, @limit, @mode
+      # ranker
+      request.put_int @ranker
+      request.put_string @rankexpr if @ranker == SPH_RANK_EXPR
+      # sorting
+      request.put_int @sort
       request.put_string @sortby
       # query itself
       request.put_string query
@@ -1524,7 +1629,7 @@ module Sphinx
 
       # per-index weights
       request.put_int @indexweights.length
-      @indexweights.each do |idx, weight|
+      @indexweights.sort_by { |idx, _| idx }.each do |idx, weight|
         request.put_string idx.to_s
         request.put_int weight
       end
@@ -1534,7 +1639,7 @@ module Sphinx
 
       # per-field weights
       request.put_int @fieldweights.length
-      @fieldweights.each do |field, weight|
+      @fieldweights.sort_by { |idx, _| idx }.each do |field, weight|
         request.put_string field.to_s
         request.put_int weight
       end
@@ -1563,6 +1668,13 @@ module Sphinx
       # select-list
       request.put_string @select
 
+      # max_predicted_time
+      request.put_int @predictedtime if @predictedtime > 0
+
+      # outer select
+      request.put_string @outerorderby
+      request.put_int @outeroffset, @outerlimit, (@hasouter ? 1 : 0)
+
       # store request to requests array
       @reqs << request.to_s;
       return @reqs.length - 1
@@ -1611,52 +1723,49 @@ module Sphinx
 
       reqs, nreqs = @reqs.join(''), @reqs.length
       @reqs = []
-      response = perform_request(:search, reqs, nreqs)
+      response = perform_request(:search, reqs, [0, nreqs])
 
       # parse response
       (1..nreqs).map do
-        result = HashWithIndifferentAccess.new('error' => '', 'warning' => '')
+        result = HashWithIndifferentAccess.new(:error => '', :warning => '')
 
         # extract status
-        status = result['status'] = response.get_int
+        status = result[:status] = response.get_int
         if status != SEARCHD_OK
           message = response.get_string
           if status == SEARCHD_WARNING
-            result['warning'] = message
+            result[:warning] = message
           else
-            result['error'] = message
+            result[:error] = message
             next result
           end
         end
 
         # read schema
         nfields = response.get_int
-        result['fields'] = (1..nfields).map { response.get_string }
+        result[:fields] = (1..nfields).map { response.get_string }
 
         attrs_names_in_order = []
         nattrs = response.get_int
-        attrs = (1..nattrs).inject({}) do |hash, idx|
+        attrs = nattrs.times.inject(HashWithIndifferentAccess.new) do |hash, idx|
           name, type = response.get_string, response.get_int
           hash[name] = type
           attrs_names_in_order << name
           hash
         end
-        result['attrs'] = attrs
+        result[:attrs] = attrs
 
         # read match count
         count, id64 = response.get_ints(2)
 
         # read matches
-        result['matches'] = (1..count).map do
-          doc, weight = if id64 == 0
-            response.get_ints(2)
-          else
-            [response.get_int64, response.get_int]
-          end
+        result[:matches] = (1..count).map do
+          doc = id64 == 0 ? response.get_int : response.get_int64
+          weight = response.get_int
 
           # This is a single result put in the result['matches'] array
-          match = { 'id' => doc, 'weight' => weight }
-          match['attrs'] = attrs_names_in_order.inject({}) do |hash, name|
+          match = HashWithIndifferentAccess.new(:id => doc, :weight => weight)
+          match[:attrs] = attrs_names_in_order.inject(HashWithIndifferentAccess.new) do |hash, name|
             hash[name] = case attrs[name]
               when SPH_ATTR_BIGINT
                 # handle 64-bit ints
@@ -1665,28 +1774,35 @@ module Sphinx
                 # handle floats
                 response.get_float
               when SPH_ATTR_STRING
+                # handle string
                 response.get_string
+              when SPH_ATTR_FACTORS
+                # ???
+                response.get_int
+              when SPH_ATTR_MULTI
+                # handle array of integers
+                val = response.get_int
+                response.get_ints(val) if val > 0
+              when SPH_ATTR_MULTI64
+                # handle array of 64-bit integers
+                val = response.get_int
+                (val / 2).times.map { response.get_int64 }
               else
                 # handle everything else as unsigned ints
-                val = response.get_int
-                if (attrs[name] & SPH_ATTR_MULTI) != 0
-                  (1..val).map { response.get_int }
-                else
-                  val
-                end
+                response.get_int
             end
             hash
           end
           match
         end
-        result['total'], result['total_found'], msecs = response.get_ints(3)
-        result['time'] = '%.3f' % (msecs / 1000.0)
+        result[:total], result[:total_found], msecs = response.get_ints(3)
+        result[:time] = '%.3f' % (msecs / 1000.0)
 
         nwords = response.get_int
-        result['words'] = (1..nwords).inject({}) do |hash, idx|
+        result[:words] = nwords.times.inject({}) do |hash, idx|
           word = response.get_string
           docs, hits = response.get_ints(2)
-          hash[word] = { 'docs' => docs, 'hits' => hits }
+          hash[word] = HashWithIndifferentAccess.new(:docs => docs, :hits => hits)
           hash
         end
 
@@ -1761,27 +1877,42 @@ module Sphinx
 
       # fixup options
       opts = HashWithIndifferentAccess.new(
-        'before_match'    => '<b>',
-        'after_match'     => '</b>',
-        'chunk_separator' => ' ... ',
-        'limit'           => 256,
-        'around'          => 5,
-        'exact_phrase'    => false,
-        'single_passage'  => false,
-        'use_boundaries'  => false,
-        'weight_order'    => false,
-        'query_mode'      => false
+        :before_match         => '<b>',
+        :after_match          => '</b>',
+        :chunk_separator      => ' ... ',
+        :limit                => 256,
+        :limit_passages       => 0,
+        :limit_words          => 0,
+        :around               => 5,
+        :exact_phrase         => false,
+        :single_passage       => false,
+        :use_boundaries       => false,
+        :weight_order         => false,
+        :query_mode           => false,
+        :force_all_words      => false,
+        :start_passage_id     => 1,
+        :load_files           => false,
+        :html_strip_mode      => 'index',
+        :allow_empty          => false,
+        :passage_boundary     => 'none',
+        :emit_zones           => false,
+        :load_files_scattered => false
       ).update(opts)
 
       # build request
 
-      # v.1.0 req
-      flags = 1
-      flags |= 2  if opts['exact_phrase']
-      flags |= 4  if opts['single_passage']
-      flags |= 8  if opts['use_boundaries']
-      flags |= 16 if opts['weight_order']
-      flags |= 32 if opts['query_mode']
+      # v.1.2 req
+      flags  = 1
+      flags |= 2    if opts[:exact_phrase]
+      flags |= 4    if opts[:single_passage]
+      flags |= 8    if opts[:use_boundaries]
+      flags |= 16   if opts[:weight_order]
+      flags |= 32   if opts[:query_mode]
+      flags |= 64   if opts[:force_all_words]
+      flags |= 128  if opts[:load_files]
+      flags |= 256  if opts[:allow_empty]
+      flags |= 512  if opts[:emit_zones]
+      flags |= 1024 if opts[:load_files_scattered]
 
       request = Request.new
       request.put_int 0, flags # mode=0, flags=1 (remove spaces)
@@ -1791,10 +1922,10 @@ module Sphinx
       request.put_string words
 
       # options
-      request.put_string opts['before_match']
-      request.put_string opts['after_match']
-      request.put_string opts['chunk_separator']
-      request.put_int opts['limit'].to_i, opts['around'].to_i
+      request.put_string opts[:before_match], opts[:after_match], opts[:chunk_separator]
+      request.put_int opts[:limit].to_i, opts[:around].to_i
+      request.put_int opts[:limit_passages].to_i, opts[:limit_words].to_i, opts[:start_passage_id].to_i
+      request.put_string opts[:html_strip_mode], opts[:passage_boundary]
 
       # documents
       request.put_int docs.size
@@ -1918,10 +2049,11 @@ module Sphinx
     #
     # @see http://www.sphinxsearch.com/docs/current.html#api-func-updateatttributes Section 6.7.2, "UpdateAttributes"
     #
-    def update_attributes(index, attrs, values, mva = false)
+    def update_attributes(index, attrs, values, mva = false, ignore_non_existent = false)
       # verify everything
       raise ArgumentError, '"index" argument must be String' unless index.kind_of?(String) or index.kind_of?(Symbol)
       raise ArgumentError, '"mva" argument must be Boolean'  unless mva.kind_of?(TrueClass) or mva.kind_of?(FalseClass)
+      raise ArgumentError, '"ignore_non_existent" argument must be Boolean'  unless ignore_non_existent.kind_of?(TrueClass) or ignore_non_existent.kind_of?(FalseClass)
 
       raise ArgumentError, '"attrs" argument must be Array' unless attrs.kind_of?(Array)
       attrs.each do |attr|
@@ -1950,6 +2082,7 @@ module Sphinx
       request.put_string index
 
       request.put_int attrs.length
+      request.put_int ignore_non_existent ? 1 : 0
       for attr in attrs
         request.put_string attr
         request.put_int mva ? 1 : 0
@@ -2051,7 +2184,7 @@ module Sphinx
     # @example
     #   sphinx.flush_attrs
     #
-    def flush_attrs
+    def flush_attributes
       request = Request.new
       response = perform_request(:flushattrs, request)
 
@@ -2059,10 +2192,13 @@ module Sphinx
       begin
         response.get_int
       rescue EOFError
+        @error = 'unexpected response length'
         -1
       end
     end
-    alias :FlushAttrs :flush_attrs
+    alias :FlushAttributes :flush_attributes
+    alias :FlushAttrs :flush_attributes
+    alias :flush_attrs :flush_attributes
 
     #=================================================================
     # Persistent connections
@@ -2152,7 +2288,7 @@ module Sphinx
       #   <tt>:update</tt>, <tt>:keywords</tt>, <tt>:persist</tt>, <tt>:status</tt>,
       #   <tt>:query</tt>, <tt>:flushattrs</tt>. See <tt>SEARCHD_COMMAND_*</tt> for details).
       # @param [Sphinx::Request] request contains request body.
-      # @param [Integer] additional additional integer data to be placed between header and body.
+      # @param [Integer, Array] additional additional integer or array of integers data to be placed between header and body.
       # @param [Sphinx::Server] server where perform request on. This is special
       #   parameter for internal usage. If specified, request will be performed
       #   on specified server, and it will try to establish connection to this
@@ -2182,22 +2318,23 @@ module Sphinx
           attempts = nil
         end
 
-        with_server(server, attempts) do |server|
-          logger.info { "[sphinx] #{command} on server #{server}" } if logger
+        with_server(server, attempts) do |srv|
+          logger.info { "[sphinx] #{command} on server #{srv}" } if logger
 
           cmd = command.to_s.upcase
           command_id = Sphinx::Client.const_get("SEARCHD_COMMAND_#{cmd}")
           command_ver = Sphinx::Client.const_get("VER_COMMAND_#{cmd}")
 
-          with_socket(server) do |socket|
-            len = request.to_s.length + (additional.nil? ? 0 : 4)
+          with_socket(srv) do |socket|
+            additional = Array(additional)
+            len = request.to_s.length + (additional.size * 4)
             header = [command_id, command_ver, len].pack('nnN')
-            header << [additional].pack('N') unless additional.nil?
+            header << additional.pack('N' * additional.size) unless additional.empty?
 
             socket.write(header + request.to_s)
 
             if block_given?
-              yield server, socket
+              yield srv, socket
             else
               parse_response(socket, command_ver)
             end
@@ -2431,6 +2568,22 @@ module Sphinx
         end
       end
 
+      # Sets or resets given bit in a bitset.
+      #
+      # @param [Integer] bitset integer value to set bit in.
+      # @param [Integer] index integer offset of the bit to set.
+      # @param [Boolean,Integer] value value to set bit into (+true+, +false+, +0+, or +1+).
+      #
+      def set_bit(bitset, index, value)
+        bit = 1 << index
+        if value == true || value == 1
+          bitset |= bit
+        elsif bitset & bit > 0
+          bitset ^= bit
+        end
+        bitset
+      end
+
       # Enables ability to skip +set_+ prefix for methods inside {#query} block.
       #
       # @example