oedipus 0.0.1.pre4 → 0.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/README.md +58 -43
- data/lib/oedipus/query_builder.rb +25 -6
- data/lib/oedipus/version.rb +1 -1
- data/spec/integration/index_spec.rb +31 -4
- data/spec/unit/query_builder_spec.rb +12 -0
- metadata +15 -9
data/README.md
CHANGED
@@ -13,20 +13,6 @@ search may be implemented, while remaining light and simple.
|
|
13
13
|
Data structures are managed using core ruby data type (Array and Hash), ensuring
|
14
14
|
simplicity and flexibilty.
|
15
15
|
|
16
|
-
## Current Status
|
17
|
-
|
18
|
-
This gem is in development. It is not ready for production use. I work for
|
19
|
-
a company called Flippa.com, which currently implements faceted search in a PHP
|
20
|
-
part of the website, using a slightly older version of Sphinx with lesser
|
21
|
-
support for SphinxQL. We want to move this search across to the ruby codebase
|
22
|
-
of the website, but are held back by ruby's lack of support for Sphinx 2.
|
23
|
-
|
24
|
-
Once a month the developers at Flippa are given three days to work on a project of
|
25
|
-
their own choice. This is my 'Triple Time' project.
|
26
|
-
|
27
|
-
I anticipate another week or so of development before I can consider this project
|
28
|
-
production-ready.
|
29
|
-
|
30
16
|
## Dependencies
|
31
17
|
|
32
18
|
* ruby (>= 1.9)
|
@@ -35,13 +21,12 @@ production-ready.
|
|
35
21
|
|
36
22
|
The gem builds a small (tiny) native extension for interfacing with mysql, as
|
37
23
|
existing gems either did not support multi-queries, or were too flaky
|
38
|
-
(i.e. ruby-mysql).
|
39
|
-
|
40
|
-
(it requires implementing a relatively small subset of the mysql 4.1/5.0 protocol).
|
24
|
+
(i.e. ruby-mysql). I will add a pure-ruby option in due course (it requires
|
25
|
+
implementing a relatively small subset of the mysql 4.1 protocol).
|
41
26
|
|
42
27
|
## Usage
|
43
28
|
|
44
|
-
The following features are all currently implemented
|
29
|
+
The following features are all currently implemented.
|
45
30
|
|
46
31
|
### Connecting to Sphinx
|
47
32
|
|
@@ -51,7 +36,7 @@ require "oedipus"
|
|
51
36
|
sphinx = Oedipus.connect('localhost:9306') # sphinxql host
|
52
37
|
```
|
53
38
|
|
54
|
-
### Inserting
|
39
|
+
### Inserting (real-time indexes)
|
55
40
|
|
56
41
|
``` ruby
|
57
42
|
sphinx[:articles].insert(
|
@@ -63,7 +48,7 @@ sphinx[:articles].insert(
|
|
63
48
|
)
|
64
49
|
```
|
65
50
|
|
66
|
-
### Replacing
|
51
|
+
### Replacing (real-time indexes)
|
67
52
|
|
68
53
|
``` ruby
|
69
54
|
sphinx[:articles].replace(
|
@@ -75,13 +60,13 @@ sphinx[:articles].replace(
|
|
75
60
|
)
|
76
61
|
```
|
77
62
|
|
78
|
-
### Updating
|
63
|
+
### Updating (real-time indexes)
|
79
64
|
|
80
65
|
``` ruby
|
81
66
|
sphinx[:articles].update(7, views: 103)
|
82
67
|
```
|
83
68
|
|
84
|
-
### Deleting
|
69
|
+
### Deleting (real-time indexes)
|
85
70
|
|
86
71
|
``` ruby
|
87
72
|
sphinx[:articles].delete(7)
|
@@ -124,6 +109,26 @@ results = sphinx[:articles].search("badgers", limit: 2)
|
|
124
109
|
# }
|
125
110
|
```
|
126
111
|
|
112
|
+
### Fetching only specific attributes
|
113
|
+
|
114
|
+
``` ruby
|
115
|
+
sphinx[:articles].search(
|
116
|
+
"example",
|
117
|
+
attrs: [:id, :views]
|
118
|
+
)
|
119
|
+
```
|
120
|
+
|
121
|
+
### Fetching additional attributes (including expressions)
|
122
|
+
|
123
|
+
Any valid field expression may be fetched. Be sure to alias it if you want to order by it.
|
124
|
+
|
125
|
+
``` ruby
|
126
|
+
sphinx[:articles].search(
|
127
|
+
"example",
|
128
|
+
attrs: [:*, "WEIGHT() AS wgt"]
|
129
|
+
)
|
130
|
+
```
|
131
|
+
|
127
132
|
### Attribute filters
|
128
133
|
|
129
134
|
Result formatting is the same as for a fulltext search. You can add as many
|
@@ -199,6 +204,35 @@ sphinx[:articles].search(
|
|
199
204
|
)
|
200
205
|
```
|
201
206
|
|
207
|
+
### Ordering
|
208
|
+
|
209
|
+
``` ruby
|
210
|
+
sphinx[:articles].search("badgers", order: { views: :asc })
|
211
|
+
```
|
212
|
+
|
213
|
+
Special handling is done for ordering by relevance.
|
214
|
+
|
215
|
+
``` ruby
|
216
|
+
sphinx[:articles].search("badgers", order: { relevance: :desc })
|
217
|
+
```
|
218
|
+
|
219
|
+
In the above case, Oedipus explicity adds `WEIGHT() AS relevance` to the `:attrs`
|
220
|
+
option. You can manually set up the relevance sort if you wish to name the weighting
|
221
|
+
attribute differently.
|
222
|
+
|
223
|
+
### Limits and offsets
|
224
|
+
|
225
|
+
Note that Sphinx applies a limit of 20 by default, so you probably want to specify
|
226
|
+
a limit yourself. You are bound by your `max_matches` setting in sphinx.conf.
|
227
|
+
|
228
|
+
Note that the meta data will still indicate the actual number of results that matched;
|
229
|
+
you simply get a smaller collection of materialized records.
|
230
|
+
|
231
|
+
``` ruby
|
232
|
+
sphinx[:articles].search("bobcats", limit: 50)
|
233
|
+
sphinx[:articles].search("bobcats", limit: 50, offset: 150)
|
234
|
+
```
|
235
|
+
|
202
236
|
### Faceted searching
|
203
237
|
|
204
238
|
A faceted search takes a base query and a set of additional queries that are
|
@@ -274,30 +308,11 @@ results = sphinx[:articles].multi_search(
|
|
274
308
|
# }
|
275
309
|
```
|
276
310
|
|
277
|
-
### Limits and offsets
|
278
|
-
|
279
|
-
Note that Sphinx applies a limit of 20 by default, so you probably want to specify
|
280
|
-
a limit yourself. You are bound by your `max_matches` setting in sphinx.conf.
|
281
|
-
|
282
|
-
Note that the meta data will still indicate the actual number of results that matched;
|
283
|
-
you simply get a smaller collection of materialized records.
|
284
|
-
|
285
|
-
``` ruby
|
286
|
-
sphinx[:articles].search("bobcats", limit: 50)
|
287
|
-
sphinx[:articles].search("bobcats", limit: 50, offset: 150)
|
288
|
-
```
|
289
|
-
|
290
|
-
### Ordering
|
291
|
-
|
292
|
-
``` ruby
|
293
|
-
sphinx[:articles].search("badgers", order: { views: :asc })
|
294
|
-
```
|
295
|
-
|
296
311
|
## Running the specs
|
297
312
|
|
298
313
|
There are both unit tests and integration tests in the specs/ directory. By default they
|
299
314
|
will both run, but in order for the integration specs to work, you need a locally
|
300
|
-
installed copy of Sphinx [1]. You then execute the specs as follows:
|
315
|
+
installed copy of [Sphinx] [1]. You then execute the specs as follows:
|
301
316
|
|
302
317
|
SEARCHD=/path/to/bin/searchd bundle exec rake spec
|
303
318
|
|
@@ -318,7 +333,7 @@ You may also compile the C extension and run the specs separately, if you prefer
|
|
318
333
|
|
319
334
|
### Footnotes
|
320
335
|
|
321
|
-
[1] You can build a local copy of sphinx without installing it on the system:
|
336
|
+
[1]: You can build a local copy of sphinx without installing it on the system:
|
322
337
|
|
323
338
|
cd sphinx-2.0.4/
|
324
339
|
./configure
|
@@ -30,7 +30,7 @@ module Oedipus
|
|
30
30
|
# a SphinxQL query
|
31
31
|
def select(query, filters)
|
32
32
|
[
|
33
|
-
from,
|
33
|
+
from(filters),
|
34
34
|
conditions(query, filters),
|
35
35
|
order_by(filters),
|
36
36
|
limits(filters)
|
@@ -85,8 +85,23 @@ module Oedipus
|
|
85
85
|
|
86
86
|
private
|
87
87
|
|
88
|
-
|
89
|
-
|
88
|
+
private
|
89
|
+
|
90
|
+
def fields(filters)
|
91
|
+
filters.fetch(:attrs, [:*]).dup.tap do |fields|
|
92
|
+
if fields.none? { |a| /\brelevance\n/ === a } && normalize_order(filters).key?(:relevance)
|
93
|
+
fields << "WEIGHT() AS relevance"
|
94
|
+
end
|
95
|
+
end
|
96
|
+
end
|
97
|
+
|
98
|
+
def from(filters)
|
99
|
+
[
|
100
|
+
"SELECT",
|
101
|
+
fields(filters).join(", "),
|
102
|
+
"FROM",
|
103
|
+
@index_name
|
104
|
+
].join(" ")
|
90
105
|
end
|
91
106
|
|
92
107
|
def into(type, id, attributes)
|
@@ -108,7 +123,7 @@ module Oedipus
|
|
108
123
|
|
109
124
|
def attribute_conditions(filters)
|
110
125
|
filters \
|
111
|
-
.reject { |k, v| [:limit, :offset, :order].include?(k.to_sym) } \
|
126
|
+
.reject { |k, v| [:attrs, :limit, :offset, :order].include?(k.to_sym) } \
|
112
127
|
.map { |k, v| "#{k} #{Comparison.of(v)}" }
|
113
128
|
end
|
114
129
|
|
@@ -119,14 +134,18 @@ module Oedipus
|
|
119
134
|
end
|
120
135
|
|
121
136
|
def order_by(filters)
|
122
|
-
return unless filters.
|
137
|
+
return unless (order = normalize_order(filters)).any?
|
123
138
|
|
124
139
|
[
|
125
140
|
"ORDER BY",
|
126
|
-
|
141
|
+
order.map { |k, dir| "#{k} #{dir.to_s.upcase}" }.join(", ")
|
127
142
|
].join(" ")
|
128
143
|
end
|
129
144
|
|
145
|
+
def normalize_order(filters)
|
146
|
+
Hash[Array(filters[:order]).map { |k, v| [k.to_sym, v || :asc] }]
|
147
|
+
end
|
148
|
+
|
130
149
|
def limits(filters)
|
131
150
|
"LIMIT #{filters[:offset].to_i}, #{filters[:limit].to_i}" if filters.key?(:limit)
|
132
151
|
end
|
data/lib/oedipus/version.rb
CHANGED
@@ -138,10 +138,10 @@ describe Oedipus::Index do
|
|
138
138
|
|
139
139
|
describe "#search" do
|
140
140
|
before(:each) do
|
141
|
-
index.insert(1, title: "Badgers and foxes",
|
142
|
-
index.insert(2, title: "Rabbits and hares",
|
143
|
-
index.insert(3, title: "Badgers in the wild",
|
144
|
-
index.insert(4, title: "Badgers for all!",
|
141
|
+
index.insert(1, title: "Badgers and foxes", views: 150)
|
142
|
+
index.insert(2, title: "Rabbits and hares", views: 87)
|
143
|
+
index.insert(3, title: "Badgers in the wild", views: 41)
|
144
|
+
index.insert(4, title: "Badgers for all, badgers!", views: 3003)
|
145
145
|
end
|
146
146
|
|
147
147
|
context "by fulltext matching" do
|
@@ -211,6 +211,33 @@ describe Oedipus::Index do
|
|
211
211
|
{ id: 3, views: 41, user_id: 0, status: "" },
|
212
212
|
]
|
213
213
|
end
|
214
|
+
|
215
|
+
context "by relevance" do
|
216
|
+
it "returns the results ordered by most relevant" do
|
217
|
+
records = index.search("badgers", order: {relevance: :desc})[:records]
|
218
|
+
records.first[:relevance].should > records.last[:relevance]
|
219
|
+
end
|
220
|
+
end
|
221
|
+
end
|
222
|
+
|
223
|
+
context "with attribute additions" do
|
224
|
+
it "fetches the additional attributes" do
|
225
|
+
index.search("badgers", attrs: [:*, "7 AS x"])[:records].should == [
|
226
|
+
{ id: 1, views: 150, user_id: 0, status: "", x: 7 },
|
227
|
+
{ id: 3, views: 41, user_id: 0, status: "", x: 7 },
|
228
|
+
{ id: 4, views: 3003, user_id: 0, status: "", x: 7 },
|
229
|
+
]
|
230
|
+
end
|
231
|
+
end
|
232
|
+
|
233
|
+
context "with attribute restrictions" do
|
234
|
+
it "fetches the restricted attributes" do
|
235
|
+
index.search("badgers", attrs: [:id, :views])[:records].should == [
|
236
|
+
{ id: 1, views: 150 },
|
237
|
+
{ id: 3, views: 41 },
|
238
|
+
{ id: 4, views: 3003 },
|
239
|
+
]
|
240
|
+
end
|
214
241
|
end
|
215
242
|
end
|
216
243
|
|
@@ -91,6 +91,12 @@ describe Oedipus::QueryBuilder do
|
|
91
91
|
end
|
92
92
|
end
|
93
93
|
|
94
|
+
context "with explicit attributes" do
|
95
|
+
it "puts the attributes in the select clause" do
|
96
|
+
builder.select("cats", attrs: [:*, "FOO() AS f"]).should =~ /SELECT \*, FOO\(\) AS f FROM posts/
|
97
|
+
end
|
98
|
+
end
|
99
|
+
|
94
100
|
context "with a limit" do
|
95
101
|
it "applies a LIMIT with an offset of 0" do
|
96
102
|
builder.select("dogs", limit: 50).should =~ /SELECT .* FROM posts WHERE .* LIMIT 0, 50/
|
@@ -123,6 +129,12 @@ describe Oedipus::QueryBuilder do
|
|
123
129
|
it "supports multiple orders" do
|
124
130
|
builder.select("cats", order: {views: :asc, author_id: :desc}).should =~ /SELECT .* FROM posts WHERE .* ORDER BY views ASC, author_id DESC/
|
125
131
|
end
|
132
|
+
|
133
|
+
context "by relevance" do
|
134
|
+
it "injects a weight() attribute" do
|
135
|
+
builder.select("cats", order: {relevance: :desc}).should =~ /SELECT \*, WEIGHT\(\) AS relevance FROM posts WHERE .* ORDER BY relevance DESC/
|
136
|
+
end
|
137
|
+
end
|
126
138
|
end
|
127
139
|
end
|
128
140
|
|
metadata
CHANGED
@@ -1,19 +1,19 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: oedipus
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0.1
|
5
|
-
prerelease:
|
4
|
+
version: 0.0.1
|
5
|
+
prerelease:
|
6
6
|
platform: ruby
|
7
7
|
authors:
|
8
8
|
- d11wtq
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2012-04-
|
12
|
+
date: 2012-04-26 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: rspec
|
16
|
-
requirement: &
|
16
|
+
requirement: &10768160 !ruby/object:Gem::Requirement
|
17
17
|
none: false
|
18
18
|
requirements:
|
19
19
|
- - ! '>='
|
@@ -21,10 +21,10 @@ dependencies:
|
|
21
21
|
version: '0'
|
22
22
|
type: :development
|
23
23
|
prerelease: false
|
24
|
-
version_requirements: *
|
24
|
+
version_requirements: *10768160
|
25
25
|
- !ruby/object:Gem::Dependency
|
26
26
|
name: rake-compiler
|
27
|
-
requirement: &
|
27
|
+
requirement: &10775820 !ruby/object:Gem::Requirement
|
28
28
|
none: false
|
29
29
|
requirements:
|
30
30
|
- - ! '>='
|
@@ -32,7 +32,7 @@ dependencies:
|
|
32
32
|
version: '0'
|
33
33
|
type: :development
|
34
34
|
prerelease: false
|
35
|
-
version_requirements: *
|
35
|
+
version_requirements: *10775820
|
36
36
|
description: ! "== Sphinx 2 Comes to Ruby\n\nOedipus brings full support for Sphinx
|
37
37
|
2 to Ruby:\n\n - real-time indexes (insert, replace, update, delete)\n - faceted
|
38
38
|
search (variations on a base query)\n - multi-queries (multiple queries executed
|
@@ -106,12 +106,18 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
106
106
|
- - ! '>='
|
107
107
|
- !ruby/object:Gem::Version
|
108
108
|
version: '0'
|
109
|
+
segments:
|
110
|
+
- 0
|
111
|
+
hash: -3724018096309534563
|
109
112
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
110
113
|
none: false
|
111
114
|
requirements:
|
112
|
-
- - ! '
|
115
|
+
- - ! '>='
|
113
116
|
- !ruby/object:Gem::Version
|
114
|
-
version:
|
117
|
+
version: '0'
|
118
|
+
segments:
|
119
|
+
- 0
|
120
|
+
hash: -3724018096309534563
|
115
121
|
requirements: []
|
116
122
|
rubyforge_project: oedipus
|
117
123
|
rubygems_version: 1.8.11
|