ultrasphinx 1.6.7 → 1.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data.tar.gz.sig +0 -0
- data/CHANGELOG +2 -0
- data/DEPLOYMENT_NOTES +29 -0
- data/Manifest +7 -2
- data/RAKE_TASKS +15 -0
- data/README +14 -17
- data/TODO +7 -1
- data/examples/default.base +16 -2
- data/lib/ultrasphinx.rb +2 -2
- data/lib/ultrasphinx/associations.rb +25 -0
- data/lib/ultrasphinx/configure.rb +26 -16
- data/lib/ultrasphinx/fields.rb +7 -4
- data/lib/ultrasphinx/is_indexed.rb +37 -25
- data/lib/ultrasphinx/search.rb +1 -0
- data/lib/ultrasphinx/search/internals.rb +10 -3
- data/lib/ultrasphinx/search/parser.rb +14 -11
- data/lib/ultrasphinx/ultrasphinx.rb +6 -4
- data/test/integration/app/app/models/geo/address.rb +1 -1
- data/test/integration/app/app/models/geo/country.rb +4 -0
- data/test/integration/app/config/boot.rb +95 -32
- data/test/integration/app/config/environment.rb +4 -1
- data/test/integration/app/config/environments/development.rb +4 -4
- data/test/integration/app/config/ultrasphinx/development.conf.canonical +45 -13
- data/test/integration/app/db/migrate/009_create_countries.rb +11 -0
- data/test/integration/app/test/fixtures/countries.yml +1 -0
- data/test/integration/configure_test.rb +14 -11
- data/test/integration/search_test.rb +1 -0
- data/test/integration/server_test.rb +2 -2
- data/test/setup.rb +0 -3
- data/test/teardown.rb +0 -0
- data/test/test_helper.rb +1 -6
- data/ultrasphinx.gemspec +21 -7
- data/vendor/riddle/README +2 -2
- data/vendor/riddle/Rakefile +94 -0
- data/vendor/riddle/lib/riddle.rb +1 -1
- data/vendor/riddle/lib/riddle/client.rb +18 -6
- data/vendor/riddle/lib/riddle/client/filter.rb +3 -1
- data/vendor/riddle/lib/riddle/client/message.rb +1 -1
- data/vendor/riddle/lib/riddle/client/response.rb +1 -1
- data/vendor/riddle/spec/fixtures/data/anchor.bin +0 -0
- data/vendor/riddle/spec/fixtures/data/filter_floats.bin +0 -0
- data/vendor/riddle/spec/fixtures/data/filter_floats_exclude.bin +0 -0
- data/vendor/riddle/spec/sphinx_helper.rb +1 -0
- data/vendor/riddle/spec/unit/message_spec.rb +3 -3
- data/vendor/riddle/spec/unit/response_spec.rb +2 -2
- metadata +10 -5
- metadata.gz.sig +0 -0
- data/test/integration/app/config/ultrasphinx/development.conf +0 -159
- data/test/integration/app/db/schema.rb +0 -52
data.tar.gz.sig
CHANGED
Binary file
|
data/CHANGELOG
CHANGED
@@ -1,4 +1,6 @@
|
|
1
1
|
|
2
|
+
v1.7. Deployment docs. Postgres fixes. Support association_name instead of class_name (Daniel Higginbotham).
|
3
|
+
|
2
4
|
v1.6.7. Fix GROUP_CONCAT aggregate problem. Discourage enable_star in default.base. Allow faceting on includes.
|
3
5
|
|
4
6
|
v1.6.6. Only use DISTINCT when necessary to improve indexing speed.
|
data/DEPLOYMENT_NOTES
ADDED
@@ -0,0 +1,29 @@
|
|
1
|
+
|
2
|
+
== The configuration files
|
3
|
+
|
4
|
+
Please note that the generated <tt>.conf</tt> file in <tt>config/ultrasphinx</tt> should not be modified by hand. This is the configuration for Sphinx itself, and includes all the generated SQL. It is <b>never the same</b> as the <tt>.base</tt> file.
|
5
|
+
|
6
|
+
You will want to keep your generated <tt>production.conf</tt> in your repository. You can get a <tt>production.conf</tt> by running:
|
7
|
+
|
8
|
+
RAILS_ENV=production rake ultrasphinx:configure
|
9
|
+
|
10
|
+
If you don't have local read access to the production database, you'll need to write a post-deploy task to generate the configuration server-side. It is important that every checkout of the app has a copy of <tt>production.conf</tt>. It is not enough to only have it on the server where the search daemon is running--Rails uses it too.
|
11
|
+
|
12
|
+
You will <b>not</b> want to keep the generated <tt>development.conf</tt> in the repository. This helps to catch bugs by discouraging out-of-date configurations.
|
13
|
+
|
14
|
+
== Indexing and monitoring
|
15
|
+
|
16
|
+
It's easy to keep the search daemon and the indexer running in a production environment. Cronjobs are the best way:
|
17
|
+
|
18
|
+
0,30 * * * * bash -c 'cd /path/to/production/current/; RAILS_ENV=production \
|
19
|
+
rake ultrasphinx:index >> /log/ultrasphinx-index.log 2>&1'
|
20
|
+
*/3 * * * * bash -c 'cd /path/to/production/current/; RAILS_ENV=production \
|
21
|
+
rake ultrasphinx:daemon:start >> /log/ultrasphinx-daemon.log 2>&1'
|
22
|
+
|
23
|
+
The first line runs the indexer every thirty minutes. The second line will try to restart the search daemon every three minutes. If it's already running, nothing happens.
|
24
|
+
|
25
|
+
If you are under severe memory limitations you might want to manage the daemon with Monit instead, so you can keep a closer eye on it. The search daemon is extremely reliable, so don't bother with fancy monitoring infrastructure unless you're sure you need it.
|
26
|
+
|
27
|
+
== Gotchas
|
28
|
+
|
29
|
+
If you change the field configuration or model set, you will need to rerun <tt>rake ultrasphinx:configure</tt> to update the <tt>.conf</tt> file. Make sure to completely stop and restart the search daemon when you deploy a changed <tt>.conf</tt>. It will not reload it automatically.
|
data/Manifest
CHANGED
@@ -1,7 +1,9 @@
|
|
1
1
|
CHANGELOG
|
2
|
+
DEPLOYMENT_NOTES
|
2
3
|
examples/ap.multi
|
3
4
|
examples/default.base
|
4
5
|
init.rb
|
6
|
+
lib/ultrasphinx/associations.rb
|
5
7
|
lib/ultrasphinx/autoload.rb
|
6
8
|
lib/ultrasphinx/configure.rb
|
7
9
|
lib/ultrasphinx/core_extensions.rb
|
@@ -21,6 +23,7 @@ lib/ultrasphinx/ultrasphinx.rb
|
|
21
23
|
lib/ultrasphinx.rb
|
22
24
|
LICENSE
|
23
25
|
Manifest
|
26
|
+
RAKE_TASKS
|
24
27
|
README
|
25
28
|
tasks/ultrasphinx.rake
|
26
29
|
test/config/ultrasphinx/test.base
|
@@ -35,6 +38,7 @@ test/integration/app/app/helpers/sellers_helper.rb
|
|
35
38
|
test/integration/app/app/helpers/states_helper.rb
|
36
39
|
test/integration/app/app/helpers/users_helper.rb
|
37
40
|
test/integration/app/app/models/geo/address.rb
|
41
|
+
test/integration/app/app/models/geo/country.rb
|
38
42
|
test/integration/app/app/models/geo/state.rb
|
39
43
|
test/integration/app/app/models/person/user.rb
|
40
44
|
test/integration/app/app/models/seller.rb
|
@@ -67,7 +71,6 @@ test/integration/app/config/environments/test.rb
|
|
67
71
|
test/integration/app/config/locomotive.yml
|
68
72
|
test/integration/app/config/routes.rb
|
69
73
|
test/integration/app/config/ultrasphinx/default.base
|
70
|
-
test/integration/app/config/ultrasphinx/development.conf
|
71
74
|
test/integration/app/config/ultrasphinx/development.conf.canonical
|
72
75
|
test/integration/app/db/migrate/001_create_users.rb
|
73
76
|
test/integration/app/db/migrate/002_create_sellers.rb
|
@@ -77,7 +80,7 @@ test/integration/app/db/migrate/005_add_capitalization_to_seller.rb
|
|
77
80
|
test/integration/app/db/migrate/006_add_deleted_to_user.rb
|
78
81
|
test/integration/app/db/migrate/007_add_lat_and_long_to_address.rb
|
79
82
|
test/integration/app/db/migrate/008_add_mission_statement_to_seller.rb
|
80
|
-
test/integration/app/db/
|
83
|
+
test/integration/app/db/migrate/009_create_countries.rb
|
81
84
|
test/integration/app/doc/README_FOR_APP
|
82
85
|
test/integration/app/public/404.html
|
83
86
|
test/integration/app/public/500.html
|
@@ -110,6 +113,7 @@ test/integration/app/script/process/spawner
|
|
110
113
|
test/integration/app/script/runner
|
111
114
|
test/integration/app/script/server
|
112
115
|
test/integration/app/test/fixtures/addresses.yml
|
116
|
+
test/integration/app/test/fixtures/countries.yml
|
113
117
|
test/integration/app/test/fixtures/sellers.yml
|
114
118
|
test/integration/app/test/fixtures/states.yml
|
115
119
|
test/integration/app/test/fixtures/users.yml
|
@@ -127,6 +131,7 @@ test/integration/search_test.rb
|
|
127
131
|
test/integration/server_test.rb
|
128
132
|
test/integration/spell_test.rb
|
129
133
|
test/setup.rb
|
134
|
+
test/teardown.rb
|
130
135
|
test/test_all.rb
|
131
136
|
test/test_helper.rb
|
132
137
|
test/ts.multi
|
data/RAKE_TASKS
ADDED
@@ -0,0 +1,15 @@
|
|
1
|
+
|
2
|
+
== Available Rake tasks
|
3
|
+
|
4
|
+
These Rake tasks are made available to your Rails app:
|
5
|
+
|
6
|
+
<tt>ultrasphinx:configure</tt>:: Rebuild the configuration file for this particular environment.
|
7
|
+
<tt>ultrasphinx:index</tt>:: Reindex the database and send an update signal to the search daemon.
|
8
|
+
<tt>ultrasphinx:daemon:restart</tt>:: Restart the search daemon.
|
9
|
+
<tt>ultrasphinx:daemon:start</tt>:: Start the search daemon.
|
10
|
+
<tt>ultrasphinx:daemon:stop</tt>:: Stop the search daemon.
|
11
|
+
<tt>ultrasphinx:daemon:status</tt>:: Check if the search daemon is running.
|
12
|
+
<tt>ultrasphinx:spelling:build</tt>:: Rebuild the custom spelling dictionary. You may need to use <tt>sudo</tt> if your Aspell folder is not writable by the app user.
|
13
|
+
<tt>ultrasphinx:bootstrap</tt>:: Bootstrap a full Sphinx environment by running configure, index, then daemon:start.
|
14
|
+
|
15
|
+
All tasks have shortcuts. Use <tt>us:conf</tt>, <tt>us:in</tt>, <tt>us:restart</tt>, <tt>us:start</tt>, <tt>us:stop</tt>, <tt>us:stat</tt>, <tt>us:spell</tt>, and <tt>us:boot</tt>.
|
data/README
CHANGED
@@ -11,9 +11,11 @@ The public certificate for the gem is at http://rubyforge.org/frs/download.php/2
|
|
11
11
|
|
12
12
|
== Requirements
|
13
13
|
|
14
|
-
* MySQL
|
15
|
-
* Sphinx 0.9.8-dev r871
|
16
|
-
* Rails 1.2.3
|
14
|
+
* MySQL 5.0, or PostgreSQL 8.2
|
15
|
+
* Sphinx 0.9.8-dev r871
|
16
|
+
* Rails 1.2.3
|
17
|
+
|
18
|
+
More recent versions than listed are ok.
|
17
19
|
|
18
20
|
== Features
|
19
21
|
|
@@ -65,15 +67,13 @@ For more index options, see ActiveRecord::Base .is_indexed.
|
|
65
67
|
|
66
68
|
== Building the index
|
67
69
|
|
68
|
-
|
70
|
+
Now run:
|
69
71
|
|
70
72
|
rake ultrasphinx:configure
|
71
73
|
rake ultrasphinx:index
|
72
74
|
rake ultrasphinx:daemon:start
|
73
75
|
|
74
|
-
To rotate the index, just rerun <tt>rake ultrasphinx:index</tt>. If the search daemon is running, it will have its index rotated. Otherwise the new index will be installed but the daemon will remain stopped.
|
75
|
-
|
76
|
-
Make sure to manually stop and restart the daemon if you change the field configuration or model set. It will not reload the configuration file automatically.
|
76
|
+
To rotate the index, just rerun <tt>rake ultrasphinx:index</tt>. If the search daemon is running, it will have its index rotated live. Otherwise the new index will be installed but the daemon will remain stopped.
|
77
77
|
|
78
78
|
== Running queries
|
79
79
|
|
@@ -97,18 +97,15 @@ See Ultrasphinx::Spell.
|
|
97
97
|
|
98
98
|
== Available Rake tasks
|
99
99
|
|
100
|
-
|
100
|
+
See RAKE_TASKS[link:files/RAKE_TASKS.html].
|
101
|
+
|
102
|
+
== Deployment notes
|
103
|
+
|
104
|
+
See DEPLOYMENT_NOTES[link:files/DEPLOYMENT_NOTES.html].
|
101
105
|
|
102
|
-
|
103
|
-
<tt>ultrasphinx:index</tt>:: Reindex the database and send an update signal to the search daemon.
|
104
|
-
<tt>ultrasphinx:daemon:restart</tt>:: Restart the search daemon.
|
105
|
-
<tt>ultrasphinx:daemon:start</tt>:: Start the search daemon.
|
106
|
-
<tt>ultrasphinx:daemon:stop</tt>:: Stop the search daemon.
|
107
|
-
<tt>ultrasphinx:daemon:status</tt>:: Check if the search daemon is running.
|
108
|
-
<tt>ultrasphinx:spelling:build</tt>:: Rebuild the custom spelling dictionary. You may need to use <tt>sudo</tt> if your Aspell folder is not writable by the app user.
|
109
|
-
<tt>ultrasphinx:bootstrap</tt>:: Bootstrap a full Sphinx environment by running configure, index, then daemon:start.
|
106
|
+
== A note about PostgreSQL
|
110
107
|
|
111
|
-
|
108
|
+
PostgreSQL 8.2 and higher are well supported. However, make sure you have executed <tt>CREATE LANGUAGE plpgsql;</tt> at least once. This step does not need to be repeated, so depending on your DB permissions, you might be able to put it in a migration.
|
112
109
|
|
113
110
|
== Reporting problems
|
114
111
|
|
data/TODO
CHANGED
@@ -1,7 +1,13 @@
|
|
1
1
|
|
2
|
-
|
2
|
+
Planned:
|
3
|
+
|
3
4
|
* Finish unifying filters and textfields
|
4
5
|
* Support exclude filters
|
5
6
|
* Make sure filters can be set to nil
|
6
7
|
* Geolocation search
|
8
|
+
|
9
|
+
Not sure:
|
10
|
+
|
11
|
+
* Delta indexing
|
12
|
+
* Use Pat Allan's association configurator, possibly with an API change to avoid the message-passing DSL
|
7
13
|
* Use Treetop for the query parser instead of regexes
|
data/examples/default.base
CHANGED
@@ -2,8 +2,21 @@
|
|
2
2
|
#
|
3
3
|
# Sphinx/Ultrasphinx user-configurable options.
|
4
4
|
#
|
5
|
-
# Copy this file to RAILS_ROOT/config/ultrasphinx.
|
6
|
-
#
|
5
|
+
# Copy this file to RAILS_ROOT/config/ultrasphinx. You can use individual
|
6
|
+
# namespaces if you want (e.g. development.base, production.base,
|
7
|
+
# test.base).
|
8
|
+
#
|
9
|
+
# This file should not be handed directly to Sphinx. Use the rake task
|
10
|
+
#
|
11
|
+
# rake ultrasphinx::configure
|
12
|
+
#
|
13
|
+
# to generate a parallel default.conf file. This is the file that Sphinx itself will
|
14
|
+
# use. The Ultrasphinx rake tasks automatically pass the correct file to
|
15
|
+
# to Sphinx.
|
16
|
+
#
|
17
|
+
# It is safe to edit .base files by hand. It is not safe to edit the generated
|
18
|
+
# .conf files. Do not symlink the .conf file to the .base file! I don't know why
|
19
|
+
# people think they need to do that. It's wrong.
|
7
20
|
#
|
8
21
|
|
9
22
|
indexer
|
@@ -18,6 +31,7 @@ searchd
|
|
18
31
|
# What interface the search daemon should listen on and where to store its logs
|
19
32
|
address = 0.0.0.0
|
20
33
|
port = 3312
|
34
|
+
seamless_rotate = 1
|
21
35
|
log = /opt/local/var/db/sphinx/log/searchd.log
|
22
36
|
query_log = /opt/local/var/db/sphinx/log/query.log
|
23
37
|
read_timeout = 5
|
data/lib/ultrasphinx.rb
CHANGED
@@ -12,12 +12,12 @@ end
|
|
12
12
|
|
13
13
|
$LOAD_PATH << "#{File.dirname(__FILE__)}/../vendor/riddle/lib"
|
14
14
|
require 'riddle'
|
15
|
-
|
16
15
|
require 'ultrasphinx/ultrasphinx'
|
16
|
+
require 'ultrasphinx/associations'
|
17
17
|
require 'ultrasphinx/core_extensions'
|
18
18
|
require 'ultrasphinx/is_indexed'
|
19
19
|
|
20
|
-
if (ActiveRecord::Base.connection rescue nil)
|
20
|
+
if (ActiveRecord::Base.connection rescue nil) # XXX Forget why this needed to be wrapped
|
21
21
|
require 'ultrasphinx/configure'
|
22
22
|
require 'ultrasphinx/autoload'
|
23
23
|
require 'ultrasphinx/fields'
|
@@ -0,0 +1,25 @@
|
|
1
|
+
module Ultrasphinx
|
2
|
+
module Associations
|
3
|
+
|
4
|
+
def get_association(klass, entry)
|
5
|
+
if value = entry['class_name']
|
6
|
+
klass.reflect_on_all_associations.detect do |assoc|
|
7
|
+
assoc.class_name == value
|
8
|
+
end
|
9
|
+
elsif value = entry['association_name']
|
10
|
+
klass.reflect_on_all_associations.detect do |assoc|
|
11
|
+
assoc.name.to_s == value.to_s
|
12
|
+
end
|
13
|
+
end
|
14
|
+
end
|
15
|
+
|
16
|
+
def get_association_model(klass, entry)
|
17
|
+
get_association(klass, entry).class_name.constantize
|
18
|
+
end
|
19
|
+
|
20
|
+
def entry_identifies_association?(entry)
|
21
|
+
entry['class_name'] || entry['association_name']
|
22
|
+
end
|
23
|
+
|
24
|
+
end
|
25
|
+
end
|
@@ -1,7 +1,9 @@
|
|
1
1
|
|
2
2
|
module Ultrasphinx
|
3
3
|
class Configure
|
4
|
-
class << self
|
4
|
+
class << self
|
5
|
+
|
6
|
+
include Associations
|
5
7
|
|
6
8
|
# Force all the indexed models to load and register in the MODEL_CONFIGURATION hash.
|
7
9
|
def load_constants
|
@@ -75,7 +77,7 @@ module Ultrasphinx
|
|
75
77
|
# Supporting Postgres now
|
76
78
|
connection_settings = klass.connection.instance_variable_get("@config")
|
77
79
|
|
78
|
-
adapter_defaults =
|
80
|
+
adapter_defaults = DEFAULTS[ADAPTER]
|
79
81
|
raise ConfigurationError, "Unsupported database adapter" unless adapter_defaults
|
80
82
|
|
81
83
|
conf = [adapter_defaults]
|
@@ -100,7 +102,12 @@ module Ultrasphinx
|
|
100
102
|
|
101
103
|
|
102
104
|
def range_select_string(klass)
|
103
|
-
"sql_query_range = SELECT
|
105
|
+
["sql_query_range = SELECT",
|
106
|
+
SQL_FUNCTIONS[ADAPTER]['range_cast']._interpolate("MIN(#{klass.primary_key})"),
|
107
|
+
", ",
|
108
|
+
SQL_FUNCTIONS[ADAPTER]['range_cast']._interpolate("MAX(#{klass.primary_key})"),
|
109
|
+
"FROM #{klass.table_name}"
|
110
|
+
].join(" ")
|
104
111
|
end
|
105
112
|
|
106
113
|
|
@@ -189,11 +196,15 @@ module Ultrasphinx
|
|
189
196
|
|
190
197
|
def build_includes(klass, fields, entries, column_strings, join_strings, group_bys, remaining_columns)
|
191
198
|
entries.to_a.each do |entry|
|
199
|
+
raise ConfigurationError, "You must identify your association with either class_name or association_name, but not both" if entry['class_name'] && entry ['association_name']
|
192
200
|
|
193
|
-
|
194
|
-
|
201
|
+
association = get_association(klass, entry)
|
202
|
+
|
203
|
+
# You can use 'class_name' and 'association_sql' to associate to a model that doesn't actually
|
204
|
+
# have an association
|
205
|
+
join_klass = association ? association.class_name.constantize : entry['class_name'].constantize
|
195
206
|
|
196
|
-
raise ConfigurationError, "Unknown association from #{klass} to #{entry['class_name']}" if not association and not entry['association_sql']
|
207
|
+
raise ConfigurationError, "Unknown association from #{klass} to #{entry['class_name'] || entry['association_name']}" if not association and not entry['association_sql']
|
197
208
|
|
198
209
|
join_strings = install_join_unless_association_sql(entry['association_sql'], nil, join_strings) do
|
199
210
|
"LEFT OUTER JOIN #{join_klass.table_name} AS #{entry['table']} ON " +
|
@@ -217,21 +228,26 @@ module Ultrasphinx
|
|
217
228
|
|
218
229
|
def build_concatenations(klass, fields, entries, column_strings, join_strings, group_bys, use_distinct, remaining_columns)
|
219
230
|
entries.to_a.each do |entry|
|
220
|
-
if entry
|
231
|
+
if entry_identifies_association?(entry) and entry['field']
|
221
232
|
# Group concats
|
233
|
+
|
222
234
|
# Only has_many's or explicit sql right now
|
223
|
-
|
235
|
+
association = get_association(klass, entry)
|
236
|
+
|
237
|
+
# You can use 'class_name' and 'association_sql' to associate to a model that doesn't actually
|
238
|
+
# have an association
|
239
|
+
join_klass = association ? association.class_name.constantize : entry['class_name'].constantize
|
224
240
|
|
225
241
|
join_strings = install_join_unless_association_sql(entry['association_sql'], nil, join_strings) do
|
226
242
|
# XXX make sure foreign key is right for polymorphic relationships
|
227
|
-
association =
|
243
|
+
association = get_association(klass, entry)
|
228
244
|
"LEFT OUTER JOIN #{join_klass.table_name} AS #{entry['table']} ON #{klass.table_name}.#{klass.primary_key} = #{entry['table']}.#{association.primary_key_name}" +
|
229
245
|
(entry['conditions'] ? " AND (#{entry['conditions']})" : "")
|
230
246
|
end
|
231
247
|
|
232
248
|
source_string = "#{entry['table']}.#{entry['field']}"
|
233
249
|
# We are using the field in an aggregate, so we don't want to add it to group_bys
|
234
|
-
source_string =
|
250
|
+
source_string = SQL_FUNCTIONS[ADAPTER]['group_concat']._interpolate(source_string)
|
235
251
|
use_distinct = true
|
236
252
|
|
237
253
|
column_strings, remaining_columns = install_field(fields, source_string, entry['as'], entry['function_sql'], entry['facet'], column_strings, remaining_columns)
|
@@ -284,12 +300,6 @@ module Ultrasphinx
|
|
284
300
|
def install_join_unless_association_sql(association_sql, join_string, join_strings)
|
285
301
|
join_strings << (association_sql or join_string or yield)
|
286
302
|
end
|
287
|
-
|
288
|
-
def association_by_class_name(klass, class_name)
|
289
|
-
klass.reflect_on_all_associations.detect do |assoc|
|
290
|
-
assoc.class_name == class_name
|
291
|
-
end
|
292
|
-
end
|
293
303
|
|
294
304
|
def say(s)
|
295
305
|
Ultrasphinx.say s
|
data/lib/ultrasphinx/fields.rb
CHANGED
@@ -7,6 +7,7 @@ This is a special singleton configuration class that stores the index field conf
|
|
7
7
|
|
8
8
|
class Fields
|
9
9
|
include Singleton
|
10
|
+
include Associations
|
10
11
|
|
11
12
|
TYPE_MAP = {
|
12
13
|
'string' => 'text',
|
@@ -114,14 +115,16 @@ This is a special singleton configuration class that stores the index field conf
|
|
114
115
|
options['include'].to_a.each do |entry|
|
115
116
|
extract_table_alias!(entry, klass)
|
116
117
|
extract_field_alias!(entry, klass)
|
117
|
-
|
118
|
-
|
118
|
+
|
119
|
+
association_model = get_association_model(klass, entry)
|
120
|
+
|
121
|
+
save_and_verify_type(entry['as'] || entry['field'], association_model.columns_hash[entry['field']].type, entry['sortable'], klass)
|
119
122
|
install_facets!(entry, klass)
|
120
123
|
end
|
121
124
|
|
122
125
|
# Regular concats are CHAR, group_concats are BLOB and need to be cast to CHAR
|
123
126
|
options['concatenate'].to_a.each do |entry|
|
124
|
-
extract_table_alias!(entry, klass)
|
127
|
+
extract_table_alias!(entry, klass)
|
125
128
|
save_and_verify_type(entry['as'], 'text', entry['sortable'], klass)
|
126
129
|
install_facets!(entry, klass)
|
127
130
|
end
|
@@ -155,7 +158,7 @@ This is a special singleton configuration class that stores the index field conf
|
|
155
158
|
# This field is referenced by a table alias
|
156
159
|
entry['table'], entry['field'] = entry['field'].split(".")
|
157
160
|
else
|
158
|
-
klass = entry
|
161
|
+
klass = get_association_model(klass, entry) if entry_identifies_association?(entry)
|
159
162
|
entry['table'] = klass.table_name
|
160
163
|
end
|
161
164
|
end
|
@@ -32,47 +32,59 @@ To apply an SQL function to a field before it is indexed, use the key <tt>:funct
|
|
32
32
|
|
33
33
|
Note that <tt>float</tt> fields are supported, but require Sphinx 0.98.
|
34
34
|
|
35
|
+
== Requiring conditions
|
36
|
+
|
37
|
+
Use the <tt>:conditions</tt> key.
|
38
|
+
|
39
|
+
SQL conditions, to scope which records are selected for indexing. Accepts a string.
|
40
|
+
|
41
|
+
:conditions => "created_at < NOW() AND deleted IS NOT NULL"
|
42
|
+
|
43
|
+
The <tt>:conditions</tt> key is especially useful if you delete records by marking them deleted rather than removing them from the database.
|
44
|
+
|
35
45
|
== Including a field from an association
|
36
46
|
|
37
47
|
Use the <tt>:include</tt> key.
|
38
48
|
|
39
49
|
Accepts an array of hashes.
|
40
50
|
|
41
|
-
:include => [{:
|
51
|
+
:include => [{:association_name => 'category', :field => 'name', :as => 'category_name'}]
|
42
52
|
|
43
|
-
Each should contain
|
53
|
+
Each should contain an <tt>:association_name</tt> key (the association name for the included model), a <tt>:field</tt> key (the name of the field to include), and an optional <tt>:as</tt> key (what to name the field in the parent).
|
44
54
|
|
45
|
-
|
55
|
+
<tt>:include</tt> hashes also accept their own <tt>:conditions</tt> key. You can use this if you need custom WHERE conditions for this particular association (e.g, this JOIN).
|
46
56
|
|
47
|
-
|
57
|
+
The keys <tt>:facet</tt>, <tt>:sortable</tt>, <tt>:class_name</tt>, <tt>:association_sql</tt>, and <tt>:function_sql</tt> are also recognized.
|
48
58
|
|
49
|
-
|
50
|
-
|
51
|
-
SQL conditions, to scope which records are selected for indexing. Accepts a string.
|
52
|
-
:conditions => "created_at < NOW() AND deleted IS NOT NULL"
|
53
|
-
The <tt>:conditions</tt> key is especially useful if you delete records by marking them deleted rather than removing them from the database.
|
54
|
-
|
55
|
-
== Concatenating several fields within a record
|
59
|
+
== Concatenating several fields within one record
|
56
60
|
|
57
61
|
Use the <tt>:concatenate</tt> key (MySQL only).
|
58
62
|
|
59
63
|
Accepts an array of option hashes.
|
60
64
|
|
61
|
-
To concatenate several fields within one record as a combined field, use a regular (or
|
65
|
+
To concatenate several fields within one record as a combined field, use a regular (or lateral) concatenation. Regular concatenations contain a <tt>:fields</tt> key (again, an array of field names), and a mandatory <tt>:as</tt> key (the name of the result of the concatenation). For example, to concatenate the <tt>title</tt> and <tt>body</tt> into one field called <tt>text</tt>:
|
62
66
|
:concatenate => [{:fields => ['title', 'body'], :as => 'text'}]
|
63
67
|
|
64
|
-
The keys <tt>:facet</tt>, <tt>:sortable</tt>,
|
68
|
+
The keys <tt>:facet</tt>, <tt>:sortable</tt>, <tt>:conditions</tt>, <tt>:function_sql</tt>, <tt>:class_name</tt>, and <tt>:association_sql</tt>, are also recognized.
|
69
|
+
|
70
|
+
Lateral concatenations are implemented with CONCAT_WS on MySQL and with a stored procedure on PostgreSQL.
|
65
71
|
|
66
|
-
== Concatenating
|
72
|
+
== Concatenating the same field from a set of associated records
|
67
73
|
|
68
74
|
Also use the <tt>:concatenate</tt> key.
|
69
75
|
|
70
|
-
To concatenate one field from a set of associated records as a combined field in the parent record, use a group (or vertical) concatenation. A group concatenation should contain
|
71
|
-
:concatenate => [{:
|
76
|
+
To concatenate one field from a set of associated records as a combined field in the parent record, use a group (or vertical) concatenation. A group concatenation should contain an <tt>:association_name</tt> key (the association name for the included model), a <tt>:field</tt> key (the field on the included model to concatenate), and an optional <tt>:as</tt> key (also the name of the result of the concatenation). For example, to concatenate all <tt>Post#body</tt> contents into the parent's <tt>responses</tt> field:
|
77
|
+
:concatenate => [{:association_name => 'posts', :field => 'body', :as => 'responses'}]
|
78
|
+
|
79
|
+
The keys <tt>:facet</tt>, <tt>:sortable</tt>, <tt>:conditions</tt>, <tt>:function_sql</tt>, <tt>:class_name</tt>, and <tt>:association_sql</tt>, are also recognized.
|
80
|
+
|
81
|
+
Vertical concatenations are implemented with GROUP_CONCAT on MySQL and with an aggregate and a stored procedure on PostgreSQL.
|
82
|
+
|
83
|
+
== Custom joins
|
72
84
|
|
73
|
-
|
85
|
+
<tt>:include</tt> and <tt>:concatenate</tt> accept an <tt>:association_sql</tt> key. You can use this if you need to pass a custom JOIN string, for example, a double JOIN for a <tt>has_many :through</tt>). If <tt>:association_sql</tt> is present, the default JOIN for <tt>belongs_to</tt> will not be generated.
|
74
86
|
|
75
|
-
|
87
|
+
Also, If you want to include a model that you don't have an actual ActiveRecord association for, you can use <tt>:association_sql</tt> combined with <tt>:class_name</tt> instead of <tt>:association_name</tt>. <tt>:class_name</tt> should be camelcase.
|
76
88
|
|
77
89
|
Ultrasphinx is not an object-relational mapper, and the association generation is intended to stay minimal--don't be afraid of <tt>:association_sql</tt>.
|
78
90
|
|
@@ -89,13 +101,13 @@ Here's an example configuration using most of the options, taken from production
|
|
89
101
|
{:field => 'author', :facet => true}
|
90
102
|
],
|
91
103
|
:include => [
|
92
|
-
{:
|
104
|
+
{:association_name => 'category', :field => 'name', :as => 'category_name'}
|
93
105
|
],
|
94
106
|
:concatenate => [
|
95
107
|
{:fields => ['title', 'long_description', 'short_description'],
|
96
108
|
:as => 'editorial'},
|
97
|
-
{:
|
98
|
-
{:
|
109
|
+
{:association_name => 'pages', :field => 'body', :as => 'body'},
|
110
|
+
{:association_name => 'comments', :field => 'body', :as => 'comments',
|
99
111
|
:conditions => "comments.item_type = '#{base_class}'"}
|
100
112
|
],
|
101
113
|
:conditions => self.live_condition_string
|
@@ -110,7 +122,7 @@ A common use case is to only search records that belong to a particular parent m
|
|
110
122
|
For example, say a Company <tt>has_many :users</tt> and each User <tt>has_many :articles</tt>. If you want to to filter Articles by Company, add <tt>company_id</tt> to the Article's <tt>is_indexed</tt> method. The best way is to grab it from the User association:
|
111
123
|
|
112
124
|
class Article < ActiveRecord::Base
|
113
|
-
is_indexed :include => [{:
|
125
|
+
is_indexed :include => [{:association_name => 'users', :field => 'company_id'}]
|
114
126
|
end
|
115
127
|
|
116
128
|
Now you can run:
|
@@ -136,8 +148,8 @@ If the associations weren't just <tt>has_many</tt> and <tt>belongs_to</tt>, you
|
|
136
148
|
|
137
149
|
Array(opts['concatenate']).each do |entry|
|
138
150
|
entry.stringify_keys!
|
139
|
-
entry.assert_valid_keys ['class_name', 'conditions', 'field', 'as', 'fields', 'association_sql', 'facet', 'function_sql', 'sortable']
|
140
|
-
raise Ultrasphinx::ConfigurationError, "You can't mix regular concat and group concats" if entry['fields'] and (entry['field'] or entry['class_name'])
|
151
|
+
entry.assert_valid_keys ['class_name', 'association_name', 'conditions', 'field', 'as', 'fields', 'association_sql', 'facet', 'function_sql', 'sortable']
|
152
|
+
raise Ultrasphinx::ConfigurationError, "You can't mix regular concat and group concats" if entry['fields'] and (entry['field'] or entry['class_name'] or entry['association_name'])
|
141
153
|
raise Ultrasphinx::ConfigurationError, "Concatenations must specify an :as key" unless entry['as']
|
142
154
|
raise Ultrasphinx::ConfigurationError, "Group concatenations must not have multiple fields" if entry['field'].is_a? Array
|
143
155
|
raise Ultrasphinx::ConfigurationError, "Regular concatenations should have multiple fields" if entry['fields'] and !entry['fields'].is_a?(Array)
|
@@ -145,7 +157,7 @@ If the associations weren't just <tt>has_many</tt> and <tt>belongs_to</tt>, you
|
|
145
157
|
|
146
158
|
Array(opts['include']).each do |entry|
|
147
159
|
entry.stringify_keys!
|
148
|
-
entry.assert_valid_keys ['class_name', 'field', 'as', 'association_sql', 'facet', 'function_sql', 'sortable']
|
160
|
+
entry.assert_valid_keys ['class_name', 'association_name', 'field', 'as', 'association_sql', 'facet', 'function_sql', 'sortable']
|
149
161
|
end
|
150
162
|
|
151
163
|
Ultrasphinx::MODEL_CONFIGURATION[self.name] = opts
|