mongoid-giza 0.1.0 → 0.2.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +13 -0
- data/README.md +46 -5
- data/lib/mongoid/giza.rb +1 -1
- data/lib/mongoid/giza/configuration.rb +26 -16
- data/lib/mongoid/giza/index/attribute.rb +1 -1
- data/lib/mongoid/giza/index/field.rb +1 -1
- data/lib/mongoid/giza/indexer.rb +1 -1
- data/lib/mongoid/giza/version.rb +1 -1
- data/spec/mongoid/giza/configuration_spec.rb +45 -22
- data/spec/mongoid/giza/index/attribute_spec.rb +1 -1
- data/spec/mongoid/giza/index/field_spec.rb +1 -1
- data/spec/mongoid/giza/indexer_spec.rb +4 -4
- metadata +3 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 8f6a99c02c683342e9631890d5db0c9f536dc48a
|
4
|
+
data.tar.gz: 130064b3c93bae3095777d233fbd2bd386efc2d8
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 3ea89da7a5ceb6837f4fa27a8e1a00610bd730aad3d4852e72b6c62f0d17bca2e9cecec05b7508e0d6aaaa7835c7421ba0e6b2024b826a269c77802c86168671
|
7
|
+
data.tar.gz: c46b77df7db788fed17b0f6219340a335f3d29a2acdfda8d87f3b3688664a303e9dc14c104964cfe743c68629d2a2759145baa4cb1416190a03c90a0c6c5506d
|
data/CHANGELOG.md
ADDED
@@ -0,0 +1,13 @@
|
|
1
|
+
# Changelog
|
2
|
+
|
3
|
+
## 0.2.0
|
4
|
+
|
5
|
+
* Use ERB to parse string settings of every section of the configuration
|
6
|
+
|
7
|
+
* Renamed `Mongoid::Giza::Indexer::full_index` to `Mongoid::Giza::Indexer::full_index!`
|
8
|
+
|
9
|
+
* Always convert `Mongoid::Giza::Index::Field` and `Mongoid::Giza::Index::Attribute` names to symbol
|
10
|
+
|
11
|
+
## 0.1.0
|
12
|
+
|
13
|
+
**Initial release**
|
data/README.md
CHANGED
@@ -1,4 +1,4 @@
|
|
1
|
-
# Mongoid::Giza [![Build Status](https://travis-ci.org/yadevteam/mongoid-giza.png)](https://travis-ci.org/yadevteam/mongoid-giza) [![Code Climate](https://codeclimate.com/github/yadevteam/mongoid-giza.png)](https://codeclimate.com/github/yadevteam/mongoid-giza) [![Coverage Status](https://coveralls.io/repos/yadevteam/mongoid-giza/badge.png)](https://coveralls.io/r/yadevteam/mongoid-giza)
|
1
|
+
# Mongoid::Giza [![Build Status](https://travis-ci.org/yadevteam/mongoid-giza.png)](https://travis-ci.org/yadevteam/mongoid-giza) [![Code Climate](https://codeclimate.com/github/yadevteam/mongoid-giza.png)](https://codeclimate.com/github/yadevteam/mongoid-giza) [![Coverage Status](https://coveralls.io/repos/yadevteam/mongoid-giza/badge.png)](https://coveralls.io/r/yadevteam/mongoid-giza) [![Gem Version](https://badge.fury.io/rb/mongoid-giza.png)](http://badge.fury.io/rb/mongoid-giza)
|
2
2
|
|
3
3
|
Mongoid layer for the Sphinx fulltext search server that supports block fields and dynamic indexes
|
4
4
|
|
@@ -20,12 +20,41 @@ Or install it yourself as:
|
|
20
20
|
|
21
21
|
:warning: **Before proceeding is extremely recommended to read the [Sphinx documentation](http://sphinxsearch.com/docs/current.html) if you are not yet familiar with it. Reading up to chapter 5 is enought to get you going.**
|
22
22
|
|
23
|
+
### Configuration file
|
24
|
+
|
25
|
+
A YAML configuration file is needed to configure the gem, the Sphinx searchd daemon, optionally the Sphinx indexer and set the default options for the sources and indexes.
|
26
|
+
|
27
|
+
The minimum configuration file must have the sphinx.conf output path, the address and port of the searchd daemon, paths to its pid and log files.
|
28
|
+
It's also a good idea to define a default path for very index.
|
29
|
+
|
30
|
+
The `xmlpipe_command` is set to a default when using rails, otherwise you need to set it for each index or a default on the YAML file.
|
31
|
+
String settings accept ERB, and you have access to the `Mongoid::Giza::Index` from index and source section settings.
|
32
|
+
|
33
|
+
The configuration file is automatically loaded when using Rails from `config/giza.yml`, otherwise you will need to call `Mongoid::Giza::Configuration.instance.load` to load it.
|
34
|
+
|
35
|
+
**Example:** *(the `xmlpipe_command` used here is already the one used in rails automatically so it's not needed, just for illustration)*
|
36
|
+
|
37
|
+
```yaml
|
38
|
+
development:
|
39
|
+
file:
|
40
|
+
output_path: "/tmp/sphinx/sphinx.conf"
|
41
|
+
searchd:
|
42
|
+
address: "localhost"
|
43
|
+
port: 9312
|
44
|
+
pid_file: "/tmp/sphinx/searchd.pid"
|
45
|
+
log: "/tmp/sphinx/searchd.log"
|
46
|
+
index:
|
47
|
+
path: "/tmp/sphinx"
|
48
|
+
source:
|
49
|
+
xmlpipe_command: "rails r '<%= index.klass %>.sphinx_indexes[:<%= index.name %>].generate_xmlpipe2(STDOUT)'"
|
50
|
+
```
|
51
|
+
|
23
52
|
### Setting up indexes on models
|
24
53
|
|
25
54
|
Use a `sphinx_index` block to create a new index.
|
26
55
|
|
27
56
|
The `sphinx_index` method may receive optional settings that will be set in this index's section or in its source section on the generated sphinx configuration file.
|
28
|
-
These settings take precedence to the defaults defined in
|
57
|
+
These settings take precedence to the defaults defined in the configuration file.
|
29
58
|
|
30
59
|
A model may have more than one index, but they need to have different names.
|
31
60
|
If two or more indexes have the same name the last one to be defined is the one which will exist.
|
@@ -44,7 +73,7 @@ It's `Class.all` by default.
|
|
44
73
|
|
45
74
|
**Example:** Creating a index on the person model
|
46
75
|
|
47
|
-
```
|
76
|
+
```ruby
|
48
77
|
class Person
|
49
78
|
include Mongoid::Document
|
50
79
|
include Mongoid::Giza
|
@@ -80,7 +109,7 @@ This dynamic index will generate one index for each job that is associated to a
|
|
80
109
|
On each index only the people that have that job will be indexed.
|
81
110
|
Finally each dynamic attribute of the job will be a field on its index.
|
82
111
|
|
83
|
-
```
|
112
|
+
```ruby
|
84
113
|
class Job
|
85
114
|
include Mongoid::Document
|
86
115
|
|
@@ -109,6 +138,18 @@ class Person
|
|
109
138
|
end
|
110
139
|
```
|
111
140
|
|
141
|
+
### Indexing
|
142
|
+
|
143
|
+
There are 3 ways to populate the Sphinx index: use the model class' `sphinx_indexer!` method, `Mongoid::Giza::Indexer.instance.index!` or `Mongoid::Giza::Indexer.instance.full_index!`
|
144
|
+
|
145
|
+
* **sphinx_indexer!:** Will execute the indexer program only on the indexes of the class.
|
146
|
+
Does not regenerate dynamic indexes.
|
147
|
+
* **index!:** Will execute the indexer program on all indexes.
|
148
|
+
Does not regenerate dynamic indexes.
|
149
|
+
* **full_index!:** Will regenerate dynamic indexes and execute the indexer program on all indexes.
|
150
|
+
|
151
|
+
This gem does not execute none of those automatically to let the you define what is the best reindexing strategy for your software.
|
152
|
+
|
112
153
|
### Searching
|
113
154
|
|
114
155
|
Use the `search` block on the class that have the indexes where the search should run.
|
@@ -125,7 +166,7 @@ Every other [Riddle::Client](http://rdoc.info/github/pat/riddle/Riddle/Client) s
|
|
125
166
|
|
126
167
|
**Example:** Searching on the person class
|
127
168
|
|
128
|
-
```
|
169
|
+
```ruby
|
129
170
|
results = Person.search do
|
130
171
|
fulltext "john"
|
131
172
|
with :age 18..40
|
data/lib/mongoid/giza.rb
CHANGED
@@ -144,7 +144,7 @@ module Mongoid
|
|
144
144
|
|
145
145
|
# Retrieves all the sphinx indexes defined on this class, static and dynamic
|
146
146
|
#
|
147
|
-
# @return [
|
147
|
+
# @return [Hash] a Hash with indexes names as keys and the actual indexes as the values
|
148
148
|
def sphinx_indexes
|
149
149
|
static_sphinx_indexes.merge(generated_sphinx_indexes)
|
150
150
|
end
|
@@ -31,8 +31,8 @@ module Mongoid
|
|
31
31
|
section = instance_variable_get("@#{section_name}")
|
32
32
|
if !section.nil?
|
33
33
|
settings.each do |setting, value|
|
34
|
-
|
35
|
-
section
|
34
|
+
value = interpolate_string(value, nil) if section != @index and section != @source
|
35
|
+
setter(section, setting, value)
|
36
36
|
end
|
37
37
|
end
|
38
38
|
end
|
@@ -89,9 +89,8 @@ module Mongoid
|
|
89
89
|
# @param section [Riddle::Configuration::Index, Riddle::Configuration::XMLSource] the object that settings are being set
|
90
90
|
def apply_default_settings(default, section, index)
|
91
91
|
default.class.settings.each do |setting|
|
92
|
-
method = "#{setting}="
|
93
92
|
value = interpolate_string(default.send("#{setting}"), index)
|
94
|
-
section
|
93
|
+
setter(section, setting, value) if !value.nil?
|
95
94
|
end
|
96
95
|
end
|
97
96
|
|
@@ -102,11 +101,22 @@ module Mongoid
|
|
102
101
|
# @param section [Riddle::Configuration::Index, Riddle::Configuration::XMLSource] where the settings will be applied
|
103
102
|
def apply_user_settings(index, section)
|
104
103
|
index.settings.each do |setting, value|
|
105
|
-
|
106
|
-
section
|
104
|
+
value = interpolate_string(value, index)
|
105
|
+
setter(section, setting, value)
|
107
106
|
end
|
108
107
|
end
|
109
108
|
|
109
|
+
# Renders the configuration to the output_path
|
110
|
+
def render
|
111
|
+
File.open(@file.output_path, "w") { |file| file.write(super) }
|
112
|
+
end
|
113
|
+
|
114
|
+
# Removes all Riddle::Index from the indices Array that where created from a generated {Mongoid::Giza::Index}
|
115
|
+
def clear_generated_indexes
|
116
|
+
@generated_indexes.each { |name, index| indices.delete(index) }
|
117
|
+
@generated_indexes = {}
|
118
|
+
end
|
119
|
+
|
110
120
|
# Interpolates a value if it's a String using ERB.
|
111
121
|
# Useful for defining dynamic settings.
|
112
122
|
# The ERB template may reference to the current {Mongoid::Giza::Index} and it's methods
|
@@ -117,19 +127,19 @@ module Mongoid
|
|
117
127
|
# @return [Object] if value was a String and contains ERB syntax than it will beinterpolated and returned.
|
118
128
|
# Otherwise it will return the original value
|
119
129
|
def interpolate_string(value, index)
|
120
|
-
namespace = OpenStruct.new(index: index)
|
130
|
+
namespace = index.nil? ? Object.new : OpenStruct.new(index: index)
|
121
131
|
value.is_a?(String) ? ERB.new(value).result(namespace.instance_eval { binding }) : value
|
122
132
|
end
|
123
133
|
|
124
|
-
#
|
125
|
-
|
126
|
-
|
127
|
-
|
128
|
-
|
129
|
-
#
|
130
|
-
def
|
131
|
-
|
132
|
-
|
134
|
+
# Helper method to set a value to a setting from a section (i.e. indexer, source) if the section has this setting.
|
135
|
+
# If the setting is not avaiable on the section, nothing is done
|
136
|
+
#
|
137
|
+
# @param section [Riddle::Configuration::Section] a configuration section to define the setting
|
138
|
+
# @param setting [Symbol] the setting that will be defined on the section
|
139
|
+
# @param value [Object] the value of the setting
|
140
|
+
def setter(section, setting, value)
|
141
|
+
method = "#{setting}="
|
142
|
+
section.send(method, value) if section.respond_to?(method)
|
133
143
|
end
|
134
144
|
end
|
135
145
|
end
|
@@ -29,7 +29,7 @@ module Mongoid
|
|
29
29
|
"attribute type not supported. " \
|
30
30
|
"It must be one of the following: " \
|
31
31
|
"#{Mongoid::Giza::Index::Attribute::TYPES.join(", ")}" unless Mongoid::Giza::Index::Attribute::TYPES.include? type
|
32
|
-
@name = name
|
32
|
+
@name = name.to_sym
|
33
33
|
@type = type
|
34
34
|
@block = block
|
35
35
|
end
|
@@ -17,7 +17,7 @@ module Mongoid
|
|
17
17
|
# (see {http://sphinxsearch.com/docs/current.html#conf-xmlpipe-field-string})
|
18
18
|
# @param block [Proc] an optional block to be evaluated at the scope of the document on index creation
|
19
19
|
def initialize(name, attribute = false, &block)
|
20
|
-
@name = name
|
20
|
+
@name = name.to_sym
|
21
21
|
@attribute = attribute
|
22
22
|
@block = block
|
23
23
|
end
|
data/lib/mongoid/giza/indexer.rb
CHANGED
data/lib/mongoid/giza/version.rb
CHANGED
@@ -59,6 +59,27 @@ describe Mongoid::Giza::Configuration do
|
|
59
59
|
file_open
|
60
60
|
@config.load("giza.yml", "test")
|
61
61
|
end
|
62
|
+
|
63
|
+
it "should interpolate the string with ERB" do
|
64
|
+
allow(file).to receive(:read) { "test:\n searchd:\n address: localhost" }
|
65
|
+
expect(@config).to receive(:interpolate_string).with("localhost", nil)
|
66
|
+
file_open
|
67
|
+
@config.load("giza.yml", "test")
|
68
|
+
end
|
69
|
+
|
70
|
+
it "should not interpolate index settings" do
|
71
|
+
allow(file).to receive(:read) { "test:\n index:\n path: home" }
|
72
|
+
expect(@config).not_to receive(:interpolate_string)
|
73
|
+
file_open
|
74
|
+
@config.load("giza.yml", "test")
|
75
|
+
end
|
76
|
+
|
77
|
+
it "should not interpolate source settings" do
|
78
|
+
allow(file).to receive(:read) { "test:\n source:\n xmlpipe_command: cmd" }
|
79
|
+
expect(@config).not_to receive(:interpolate_string)
|
80
|
+
file_open
|
81
|
+
@config.load("giza.yml", "test")
|
82
|
+
end
|
62
83
|
end
|
63
84
|
|
64
85
|
describe "add_index" do
|
@@ -216,29 +237,18 @@ describe Mongoid::Giza::Configuration do
|
|
216
237
|
|
217
238
|
it "should apply the settings from default to section" do
|
218
239
|
allow(@default).to receive(:html_strip) { 1 }
|
219
|
-
|
220
|
-
expect(@section).to receive(:html_strip=).with(1)
|
240
|
+
expect(@config).to receive(:setter).with(@section, :html_strip, 1)
|
221
241
|
@config.apply_default_settings(@default, @section, @index)
|
222
242
|
end
|
223
243
|
|
224
244
|
it "should not set nil values" do
|
225
245
|
allow(@default).to receive(:html_strip) { nil }
|
226
|
-
|
227
|
-
expect(@section).not_to receive(:html_strip=)
|
228
|
-
@config.apply_default_settings(@default, @section, @index)
|
229
|
-
end
|
230
|
-
|
231
|
-
it "should not try to apply settings without a setter" do
|
232
|
-
allow(@default).to receive(:html_strip) { 1 }
|
233
|
-
allow(@section).to receive(:respond_to?).with("html_strip=") { false }
|
234
|
-
expect(@section).not_to receive(:html_strip=)
|
246
|
+
expect(@config).not_to receive(:setter)
|
235
247
|
@config.apply_default_settings(@default, @section, @index)
|
236
248
|
end
|
237
249
|
|
238
250
|
it "should interpolate string values" do
|
239
251
|
allow(@default).to receive(:html_strip) { 1 }
|
240
|
-
allow(@section).to receive(:respond_to?).with("html_strip=") { true }
|
241
|
-
allow(@section).to receive(:html_strip=)
|
242
252
|
expect(@config).to receive(:interpolate_string).with(1, @index)
|
243
253
|
@config.apply_default_settings(@default, @section, @index)
|
244
254
|
end
|
@@ -253,19 +263,11 @@ describe Mongoid::Giza::Configuration do
|
|
253
263
|
end
|
254
264
|
|
255
265
|
it "should apply the the settings" do
|
256
|
-
|
257
|
-
expect(@section).to receive(:html_strip=).with(1)
|
258
|
-
@config.apply_user_settings(@index, @section)
|
259
|
-
end
|
260
|
-
|
261
|
-
it "should not try to apply settings without a setter" do
|
262
|
-
allow(@section).to receive(:respond_to?).with("html_strip=") { false }
|
263
|
-
expect(@section).not_to receive(:html_strip=)
|
266
|
+
expect(@config).to receive(:setter).with(@section, :html_strip, 1)
|
264
267
|
@config.apply_user_settings(@index, @section)
|
265
268
|
end
|
266
269
|
|
267
270
|
it "should interpolate the value" do
|
268
|
-
allow(@section).to receive(:respond_to?).with("html_strip=") { true }
|
269
271
|
expect(@config).to receive(:interpolate_string).with(1, @index)
|
270
272
|
@config.apply_user_settings(@index, @section)
|
271
273
|
end
|
@@ -337,4 +339,25 @@ describe Mongoid::Giza::Configuration do
|
|
337
339
|
expect(@config.instance_variable_get("@generated_indexes")).to eql({})
|
338
340
|
end
|
339
341
|
end
|
342
|
+
|
343
|
+
describe "setter" do
|
344
|
+
let(:section) { double("section") }
|
345
|
+
|
346
|
+
let(:value) { double("value") }
|
347
|
+
|
348
|
+
before do
|
349
|
+
allow(section).to receive(:respond_to?) { true }
|
350
|
+
end
|
351
|
+
|
352
|
+
it "should use the attribute setter on the section" do
|
353
|
+
expect(section).to receive("setting=").with(value)
|
354
|
+
@config.setter(section, :setting, value)
|
355
|
+
end
|
356
|
+
|
357
|
+
it "should no set the value if the section does not respond to the attribute setter" do
|
358
|
+
allow(section).to receive(:respond_to?).with("setting=") { false }
|
359
|
+
expect(section).not_to receive("setting=")
|
360
|
+
@config.setter(section, :setting, value)
|
361
|
+
end
|
362
|
+
end
|
340
363
|
end
|
@@ -9,7 +9,7 @@ describe Mongoid::Giza::Index::Attribute do
|
|
9
9
|
it "should be set on creation" do
|
10
10
|
name = "attribute"
|
11
11
|
attribute = Mongoid::Giza::Index::Attribute.new(name, :uint)
|
12
|
-
expect(attribute.name).to eql(name)
|
12
|
+
expect(attribute.name).to eql(name.to_sym)
|
13
13
|
end
|
14
14
|
end
|
15
15
|
|
@@ -29,7 +29,7 @@ describe Mongoid::Giza::Indexer do
|
|
29
29
|
end
|
30
30
|
end
|
31
31
|
|
32
|
-
describe "full_index" do
|
32
|
+
describe "full_index!" do
|
33
33
|
let(:klass) { double("class") }
|
34
34
|
|
35
35
|
before do
|
@@ -40,14 +40,14 @@ describe Mongoid::Giza::Indexer do
|
|
40
40
|
expect(config).to receive(:clear_generated_indexes)
|
41
41
|
allow(@indexer).to receive(:giza_classes) { [klass] }
|
42
42
|
allow(klass).to receive(:regenerate_dynamic_sphinx_indexes)
|
43
|
-
@indexer.full_index
|
43
|
+
@indexer.full_index!
|
44
44
|
end
|
45
45
|
|
46
46
|
it "should regenerate all dynamic indexes of the giza classes" do
|
47
47
|
allow(config).to receive(:clear_generated_indexes)
|
48
48
|
allow(@indexer).to receive(:giza_classes) { [klass] }
|
49
49
|
expect(klass).to receive(:regenerate_dynamic_sphinx_indexes)
|
50
|
-
@indexer.full_index
|
50
|
+
@indexer.full_index!
|
51
51
|
end
|
52
52
|
|
53
53
|
it "should execute the indexer" do
|
@@ -55,7 +55,7 @@ describe Mongoid::Giza::Indexer do
|
|
55
55
|
allow(@indexer).to receive(:giza_classes) { [klass] }
|
56
56
|
allow(klass).to receive(:regenerate_dynamic_sphinx_indexes)
|
57
57
|
expect(@indexer).to receive(:index!).with(no_args)
|
58
|
-
@indexer.full_index
|
58
|
+
@indexer.full_index!
|
59
59
|
end
|
60
60
|
end
|
61
61
|
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: mongoid-giza
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.2.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Maurício Batista
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2014-01-14 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: bundler
|
@@ -160,6 +160,7 @@ extra_rdoc_files: []
|
|
160
160
|
files:
|
161
161
|
- .gitignore
|
162
162
|
- .travis.yml
|
163
|
+
- CHANGELOG.md
|
163
164
|
- Gemfile
|
164
165
|
- LICENSE.txt
|
165
166
|
- README.md
|