sphinx 0.9.9.2117 → 0.9.10
Sign up to get free protection for your applications and to get access to all the features.
- data/.gitignore +1 -3
- data/README.rdoc +18 -213
- data/Rakefile +15 -24
- data/VERSION.yml +1 -2
- data/install.rb +5 -0
- data/lib/sphinx/client.rb +893 -2089
- data/lib/sphinx/request.rb +11 -82
- data/lib/sphinx/response.rb +0 -2
- data/lib/sphinx.rb +5 -50
- data/spec/client_response_spec.rb +40 -33
- data/spec/client_spec.rb +471 -669
- data/spec/fixtures/ranking_sph04.php +9 -0
- data/spec/fixtures/sphinxapi.php +69 -54
- data/spec/sphinx/sphinx-id64.conf +1 -1
- data/spec/sphinx/sphinx.conf +1 -1
- data/spec/sphinx/sphinx_test.sql +2 -4
- data/sphinx.gemspec +8 -13
- data/sphinx.yml.tpl +3 -0
- data/tasks/sphinx.rake +75 -0
- metadata +6 -11
- data/lib/sphinx/buffered_io.rb +0 -26
- data/lib/sphinx/constants.rb +0 -179
- data/lib/sphinx/indifferent_access.rb +0 -152
- data/lib/sphinx/server.rb +0 -170
- data/lib/sphinx/timeout.rb +0 -31
- data/spec/client_validations_spec.rb +0 -859
- data/spec/spec_helper.rb +0 -24
data/.gitignore
CHANGED
data/README.rdoc
CHANGED
@@ -1,10 +1,10 @@
|
|
1
|
-
=
|
1
|
+
=Sphinx Client API 0.9.10
|
2
2
|
|
3
|
-
This document gives an overview of what is Sphinx itself and how to use
|
4
|
-
|
3
|
+
This document gives an overview of what is Sphinx itself and how to use in
|
4
|
+
within Ruby on Rails. For more information or documentation,
|
5
5
|
please go to http://www.sphinxsearch.com
|
6
6
|
|
7
|
-
==
|
7
|
+
==Sphinx
|
8
8
|
|
9
9
|
Sphinx is a standalone full-text search engine, meant to provide fast,
|
10
10
|
size-efficient and relevant fulltext search functions to other applications.
|
@@ -12,232 +12,37 @@ Sphinx was specially designed to integrate well with SQL databases and
|
|
12
12
|
scripting languages. Currently built-in data sources support fetching data
|
13
13
|
either via direct connection to MySQL, or from an XML pipe.
|
14
14
|
|
15
|
-
Simplest way to communicate with Sphinx is to use <tt>searchd</tt>
|
16
|
-
a daemon to search through
|
15
|
+
Simplest way to communicate with Sphinx is to use <tt>searchd</tt> -
|
16
|
+
a daemon to search through fulltext indices from external software.
|
17
17
|
|
18
|
-
==
|
18
|
+
==Compatibility
|
19
19
|
|
20
|
-
|
20
|
+
This version supports all API features of Sphinx 0.9.10-r2043. Full compatibility list:
|
21
21
|
|
22
|
-
*
|
23
|
-
*
|
22
|
+
* <tt>0.9.10</tt> — Sphinx 0.9.10-r2043
|
23
|
+
* <tt>0.9.9</tt> — Sphinx 0.9.9-r1299
|
24
24
|
|
25
|
-
|
25
|
+
==Documentation
|
26
26
|
|
27
|
-
|
27
|
+
You can create the documentation by running:
|
28
28
|
|
29
|
-
|
29
|
+
rake rdoc
|
30
30
|
|
31
|
-
|
31
|
+
==Latest version
|
32
32
|
|
33
|
-
|
34
|
-
|
35
|
-
script/plugin install git://github.com/kpumuk/sphinx.git
|
36
|
-
|
37
|
-
== Documentation
|
38
|
-
|
39
|
-
Complete Sphinx plugin documentation could be found here:
|
40
|
-
http://kpumuk.github.com/sphinx
|
41
|
-
|
42
|
-
Also you can find documentation on rdoc.info:
|
43
|
-
http://rdoc.info/projects/kpumuk/sphinx
|
44
|
-
|
45
|
-
You can build the documentation locally by running:
|
46
|
-
|
47
|
-
rake yard
|
48
|
-
|
49
|
-
Please note: you should have yard gem installed on your system:
|
50
|
-
|
51
|
-
sudo gem install yard --source http://gemcutter.org
|
52
|
-
|
53
|
-
Complete Sphinx API documentation could be found on Sphinx Search Engine
|
54
|
-
site: http://www.sphinxsearch.com/docs/current.html
|
55
|
-
This plugin is fully compatible with original PHP API implementation.
|
56
|
-
|
57
|
-
== Ruby naming conventions
|
58
|
-
|
59
|
-
Sphinx Client API supports Ruby naming conventions, so every API
|
60
|
-
method name is in underscored, lowercase form:
|
61
|
-
|
62
|
-
SetServer -> set_server
|
63
|
-
RunQueries -> run_queries
|
64
|
-
SetMatchMode -> set_match_mode
|
65
|
-
|
66
|
-
Every method is aliased to a corresponding one from standard Sphinx
|
67
|
-
API, so you can use both <tt>SetServer</tt> and <tt>set_server</tt>
|
68
|
-
with no differrence.
|
69
|
-
|
70
|
-
There are three exceptions to this naming rule:
|
71
|
-
|
72
|
-
GetLastError -> last_error
|
73
|
-
GetLastWarning -> last_warning
|
74
|
-
IsConnectError -> connect_error?
|
75
|
-
|
76
|
-
Of course, all of them are aliased to the original method names.
|
77
|
-
|
78
|
-
== Using multiple Sphinx servers
|
79
|
-
|
80
|
-
Since we actively use this plugin in our Scribd development workflow,
|
81
|
-
there are several methods have been added to accommodate our needs.
|
82
|
-
You can find documentation on Ruby-specific methods in documentation:
|
83
|
-
http://rdoc.info/projects/kpumuk/sphinx
|
84
|
-
|
85
|
-
First of all, we added support of multiple Sphinx servers to balance
|
86
|
-
load between them. Also it means that in case of any problems with one
|
87
|
-
of servers, library will try to fetch the results from another one.
|
88
|
-
Every consequence request will be executed on the next server in list
|
89
|
-
(round-robin technique).
|
90
|
-
|
91
|
-
sphinx.set_servers([
|
92
|
-
{ :host => 'browse01.local', :port => 3312 },
|
93
|
-
{ :host => 'browse02.local', :port => 3312 },
|
94
|
-
{ :host => 'browse03.local', :port => 3312 }
|
95
|
-
])
|
96
|
-
|
97
|
-
By default library will try to fetch results from a single server, and
|
98
|
-
fail if it does not respond. To setup number of retries being performed,
|
99
|
-
you can use second (additional) parameter of the <tt>set_connect_timeout</tt>
|
100
|
-
and <tt>set_request_timeout</tt> methods:
|
101
|
-
|
102
|
-
sphinx.set_connect_timeout(1, 3)
|
103
|
-
sphinx.set_request_timeout(1, 3)
|
104
|
-
|
105
|
-
There is a big difference between these two methods. First will affect
|
106
|
-
only on requests experiencing problems with connection (socket error,
|
107
|
-
pipe error, etc), second will be used when request is broken somehow
|
108
|
-
(temporary searchd error, incomplete reply, etc). The workflow looks like
|
109
|
-
this:
|
110
|
-
|
111
|
-
1. Increase retries number. If is less or equal to configured value,
|
112
|
-
try to connect to the next server. Otherwise, raise an error.
|
113
|
-
2. In case of connection problem go to 1.
|
114
|
-
3. Increase request retries number. If it less or equal to configured
|
115
|
-
value, try to perform request. Otherwise, raise an error.
|
116
|
-
4. In case of connection problem go to 1.
|
117
|
-
5. In case of request problem, go to 3.
|
118
|
-
6. Parse and return response.
|
119
|
-
|
120
|
-
Withdrawals:
|
121
|
-
|
122
|
-
1. Request could be performed <tt>connect_retries</tt> * <tt>request_retries</tt>
|
123
|
-
times. E.g., it could be tried <tt>request_retries</tt> times on each
|
124
|
-
of <tt>connect_retries</tt> servers (when you have 1 server configured,
|
125
|
-
but <tt>connect_retries</tt> is 5, library will try to connect to this
|
126
|
-
server 5 times).
|
127
|
-
2. Request could be tried to execute on each server <tt>1..request_retries</tt>
|
128
|
-
times. In case of connection problem, request will be moved to another
|
129
|
-
server immediately.
|
130
|
-
|
131
|
-
Usually you will set <tt>connect_retries</tt> equal to servers number,
|
132
|
-
so you will be sure each failing request will be performed on all servers.
|
133
|
-
This means that if one of servers is live, but others are dead, you request
|
134
|
-
will be finally executed successfully.
|
135
|
-
|
136
|
-
== Sphinx constants
|
137
|
-
|
138
|
-
Most Sphinx API methods expecting for special constants will be passed.
|
139
|
-
For example:
|
140
|
-
|
141
|
-
sphinx.set_match_mode(Sphinx::SPH_MATCH_ANY)
|
142
|
-
|
143
|
-
Please note that these constants defined in a <tt>Sphinx</tt>
|
144
|
-
module. You can use symbols or strings instead of these awful
|
145
|
-
constants:
|
146
|
-
|
147
|
-
sphinx.set_match_mode(:any)
|
148
|
-
sphinx.set_match_mode('any')
|
149
|
-
|
150
|
-
== Setting query filters
|
151
|
-
|
152
|
-
Every <tt>set_</tt> method returns <tt>Sphinx::Client</tt> object itself.
|
153
|
-
It means that you can chain filtering methods:
|
154
|
-
|
155
|
-
results = Sphinx::Client.new.
|
156
|
-
set_match_mode(:any).
|
157
|
-
set_ranking_mode(:bm25).
|
158
|
-
set_id_range(10, 1000).
|
159
|
-
query('test')
|
160
|
-
|
161
|
-
There is a handful ability to set query parameters directly in <tt>query</tt>
|
162
|
-
call. If block does not accept any parameters, it will be eval'ed inside
|
163
|
-
Sphinx::Client instance:
|
164
|
-
|
165
|
-
results = Sphinx::Client.new.query('test') do
|
166
|
-
match_mode :any
|
167
|
-
ranking_mode :bm25
|
168
|
-
id_range 10, 1000
|
169
|
-
end
|
170
|
-
|
171
|
-
As you can see, in this case you can omit the <tt>set_</tt> prefix for
|
172
|
-
this methods. If block accepts a parameter, sphinx instance will be
|
173
|
-
passed into the block. In this case you should you full method names
|
174
|
-
including the <tt>set_</tt> prefix:
|
175
|
-
|
176
|
-
results = Sphinx::Client.new.query('test') do |sphinx|
|
177
|
-
sphinx.set_match_mode :any
|
178
|
-
sphinx.set_ranking_mode :bm25
|
179
|
-
sphinx.set_id_range 10, 1000
|
180
|
-
end
|
181
|
-
|
182
|
-
== Example
|
183
|
-
|
184
|
-
This simple example illustrates base connection establishing,
|
185
|
-
search results retrieving, and excerpts building. Please note
|
186
|
-
how does it perform database select using ActiveRecord to
|
187
|
-
save the order of records established by Sphinx.
|
188
|
-
|
189
|
-
sphinx = Sphinx::Client.new
|
190
|
-
result = sphinx.query('test')
|
191
|
-
ids = result['matches'].map { |match| match['id'] }
|
192
|
-
posts = Post.all :conditions => { :id => ids },
|
193
|
-
:order => "FIELD(id,#{ids.join(',')})"
|
194
|
-
|
195
|
-
docs = posts.map(&:body)
|
196
|
-
excerpts = sphinx.build_excerpts(docs, 'index', 'test')
|
197
|
-
|
198
|
-
== Logging
|
199
|
-
|
200
|
-
You can ask Sphinx client API to log it's activity to some log. In
|
201
|
-
order to do that you can pass a logger object into the <tt>Sphinx::Client</tt>
|
202
|
-
constructor:
|
203
|
-
|
204
|
-
require 'logger'
|
205
|
-
Sphinx::Client.new(Logger.new(STDOUT)).query('test')
|
206
|
-
|
207
|
-
Logger object should respond to methods :debug, :info, and :warn, and
|
208
|
-
accept blocks (this is what standard Ruby <tt>Logger</tt> class does).
|
209
|
-
Here is what you will see in your log:
|
210
|
-
|
211
|
-
* <tt>DEBUG</tt> -- <tt>query</tt>, <tt>add_query</tt>, <tt>run_queries</tt>
|
212
|
-
method calls with configured filters.
|
213
|
-
* <tt>INFO</tt> -- initialization with Sphinx version, servers change,
|
214
|
-
attempts to re-connect, and all attempts to do an API call with server
|
215
|
-
where request being performed.
|
216
|
-
* <tt>WARN</tt> -- various connection and socket errors.
|
217
|
-
|
218
|
-
== Support
|
219
|
-
|
220
|
-
Source code:
|
221
|
-
http://github.com/kpumuk/sphinx
|
222
|
-
|
223
|
-
To suggest a feature or report a bug:
|
224
|
-
http://github.com/kpumuk/sphinx/issues
|
225
|
-
|
226
|
-
Project home page:
|
33
|
+
You can always get latest version from
|
227
34
|
http://kpumuk.info/projects/ror-plugins/sphinx
|
228
35
|
|
229
|
-
==
|
36
|
+
==Credits
|
230
37
|
|
231
38
|
Dmytro Shteflyuk <kpumuk@kpumuk.info> http://kpumuk.info
|
232
39
|
|
233
|
-
Andrew Aksyonoff http://sphinxsearch.com
|
40
|
+
Andrew Aksyonoff http://sphinxsearch.com/
|
234
41
|
|
235
42
|
Special thanks to Alexey Kovyrin <alexey@kovyrin.net> http://blog.kovyrin.net
|
236
43
|
|
237
|
-
Special thanks to Mike Perham http://www.mikeperham.com for his awesome
|
238
|
-
memcache-client gem, where latest Sphinx gem got new sockets handling from.
|
239
|
-
|
240
44
|
==License
|
241
45
|
|
242
46
|
This library is distributed under the terms of the Ruby license.
|
243
47
|
You can freely distribute/modify this library.
|
48
|
+
|
data/Rakefile
CHANGED
@@ -1,4 +1,6 @@
|
|
1
1
|
require 'rake'
|
2
|
+
require 'spec/rake/spectask'
|
3
|
+
require 'rake/rdoctask'
|
2
4
|
|
3
5
|
begin
|
4
6
|
require 'jeweler'
|
@@ -15,31 +17,20 @@ rescue LoadError
|
|
15
17
|
puts 'Jeweler not available. Install it with: sudo gem install jeweler'
|
16
18
|
end
|
17
19
|
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
desc 'Default: run specs'
|
22
|
-
task :default => :spec
|
20
|
+
desc 'Default: run specs'
|
21
|
+
task :default => :spec
|
23
22
|
|
24
|
-
|
25
|
-
|
26
|
-
|
27
|
-
|
28
|
-
end
|
29
|
-
rescue LoadError
|
30
|
-
puts 'RSpec not available. Install it with: sudo gem install rspec'
|
23
|
+
desc 'Test the sphinx plugin'
|
24
|
+
Spec::Rake::SpecTask.new(:spec) do |t|
|
25
|
+
t.libs << 'lib'
|
26
|
+
t.pattern = 'spec/*_spec.rb'
|
31
27
|
end
|
32
28
|
|
33
|
-
|
34
|
-
|
35
|
-
|
36
|
-
|
37
|
-
|
38
|
-
|
39
|
-
|
40
|
-
t.options << '--no-private'
|
41
|
-
end
|
42
|
-
end
|
43
|
-
rescue LoadError
|
44
|
-
puts 'Yard not available. Install it with: sudo gem install yard'
|
29
|
+
desc 'Generate documentation for the sphinx plugin'
|
30
|
+
Rake::RDocTask.new(:rdoc) do |rdoc|
|
31
|
+
rdoc.rdoc_dir = 'rdoc'
|
32
|
+
rdoc.title = 'Sphinx Client API'
|
33
|
+
rdoc.options << '--line-numbers' << '--inline-source'
|
34
|
+
rdoc.rdoc_files.include('README')
|
35
|
+
rdoc.rdoc_files.include('lib/**/*.rb')
|
45
36
|
end
|
data/VERSION.yml
CHANGED
data/install.rb
ADDED