ultrasphinx 1.6.7 → 1.7
Sign up to get free protection for your applications and to get access to all the features.
- data.tar.gz.sig +0 -0
- data/CHANGELOG +2 -0
- data/DEPLOYMENT_NOTES +29 -0
- data/Manifest +7 -2
- data/RAKE_TASKS +15 -0
- data/README +14 -17
- data/TODO +7 -1
- data/examples/default.base +16 -2
- data/lib/ultrasphinx.rb +2 -2
- data/lib/ultrasphinx/associations.rb +25 -0
- data/lib/ultrasphinx/configure.rb +26 -16
- data/lib/ultrasphinx/fields.rb +7 -4
- data/lib/ultrasphinx/is_indexed.rb +37 -25
- data/lib/ultrasphinx/search.rb +1 -0
- data/lib/ultrasphinx/search/internals.rb +10 -3
- data/lib/ultrasphinx/search/parser.rb +14 -11
- data/lib/ultrasphinx/ultrasphinx.rb +6 -4
- data/test/integration/app/app/models/geo/address.rb +1 -1
- data/test/integration/app/app/models/geo/country.rb +4 -0
- data/test/integration/app/config/boot.rb +95 -32
- data/test/integration/app/config/environment.rb +4 -1
- data/test/integration/app/config/environments/development.rb +4 -4
- data/test/integration/app/config/ultrasphinx/development.conf.canonical +45 -13
- data/test/integration/app/db/migrate/009_create_countries.rb +11 -0
- data/test/integration/app/test/fixtures/countries.yml +1 -0
- data/test/integration/configure_test.rb +14 -11
- data/test/integration/search_test.rb +1 -0
- data/test/integration/server_test.rb +2 -2
- data/test/setup.rb +0 -3
- data/test/teardown.rb +0 -0
- data/test/test_helper.rb +1 -6
- data/ultrasphinx.gemspec +21 -7
- data/vendor/riddle/README +2 -2
- data/vendor/riddle/Rakefile +94 -0
- data/vendor/riddle/lib/riddle.rb +1 -1
- data/vendor/riddle/lib/riddle/client.rb +18 -6
- data/vendor/riddle/lib/riddle/client/filter.rb +3 -1
- data/vendor/riddle/lib/riddle/client/message.rb +1 -1
- data/vendor/riddle/lib/riddle/client/response.rb +1 -1
- data/vendor/riddle/spec/fixtures/data/anchor.bin +0 -0
- data/vendor/riddle/spec/fixtures/data/filter_floats.bin +0 -0
- data/vendor/riddle/spec/fixtures/data/filter_floats_exclude.bin +0 -0
- data/vendor/riddle/spec/sphinx_helper.rb +1 -0
- data/vendor/riddle/spec/unit/message_spec.rb +3 -3
- data/vendor/riddle/spec/unit/response_spec.rb +2 -2
- metadata +10 -5
- metadata.gz.sig +0 -0
- data/test/integration/app/config/ultrasphinx/development.conf +0 -159
- data/test/integration/app/db/schema.rb +0 -52
data.tar.gz.sig
CHANGED
Binary file
|
data/CHANGELOG
CHANGED
@@ -1,4 +1,6 @@
|
|
1
1
|
|
2
|
+
v1.7. Deployment docs. Postgres fixes. Support association_name instead of class_name (Daniel Higginbotham).
|
3
|
+
|
2
4
|
v1.6.7. Fix GROUP_CONCAT aggregate problem. Discourage enable_star in default.base. Allow faceting on includes.
|
3
5
|
|
4
6
|
v1.6.6. Only use DISTINCT when necessary to improve indexing speed.
|
data/DEPLOYMENT_NOTES
ADDED
@@ -0,0 +1,29 @@
|
|
1
|
+
|
2
|
+
== The configuration files
|
3
|
+
|
4
|
+
Please note that the generated <tt>.conf</tt> file in <tt>config/ultrasphinx</tt> should not be modified by hand. This is the configuration for Sphinx itself, and includes all the generated SQL. It is <b>never the same</b> as the <tt>.base</tt> file.
|
5
|
+
|
6
|
+
You will want to keep your generated <tt>production.conf</tt> in your repository. You can get a <tt>production.conf</tt> by running:
|
7
|
+
|
8
|
+
RAILS_ENV=production rake ultrasphinx:configure
|
9
|
+
|
10
|
+
If you don't have local read access to the production database, you'll need to write a post-deploy task to generate the configuration server-side. It is important that every checkout of the app has a copy of <tt>production.conf</tt>. It is not enough to only have it on the server where the search daemon is running--Rails uses it too.
|
11
|
+
|
12
|
+
You will <b>not</b> want to keep the generated <tt>development.conf</tt> in the repository. This helps to catch bugs by discouraging out-of-date configurations.
|
13
|
+
|
14
|
+
== Indexing and monitoring
|
15
|
+
|
16
|
+
It's easy to keep the search daemon and the indexer running in a production environment. Cronjobs are the best way:
|
17
|
+
|
18
|
+
0,30 * * * * bash -c 'cd /path/to/production/current/; RAILS_ENV=production \
|
19
|
+
rake ultrasphinx:index >> /log/ultrasphinx-index.log 2>&1'
|
20
|
+
*/3 * * * * bash -c 'cd /path/to/production/current/; RAILS_ENV=production \
|
21
|
+
rake ultrasphinx:daemon:start >> /log/ultrasphinx-daemon.log 2>&1'
|
22
|
+
|
23
|
+
The first line runs the indexer every thirty minutes. The second line will try to restart the search daemon every three minutes. If it's already running, nothing happens.
|
24
|
+
|
25
|
+
If you are under severe memory limitations you might want to manage the daemon with Monit instead, so you can keep a closer eye on it. The search daemon is extremely reliable, so don't bother with fancy monitoring infrastructure unless you're sure you need it.
|
26
|
+
|
27
|
+
== Gotchas
|
28
|
+
|
29
|
+
If you change the field configuration or model set, you will need to rerun <tt>rake ultrasphinx:configure</tt> to update the <tt>.conf</tt> file. Make sure to completely stop and restart the search daemon when you deploy a changed <tt>.conf</tt>. It will not reload it automatically.
|
data/Manifest
CHANGED
@@ -1,7 +1,9 @@
|
|
1
1
|
CHANGELOG
|
2
|
+
DEPLOYMENT_NOTES
|
2
3
|
examples/ap.multi
|
3
4
|
examples/default.base
|
4
5
|
init.rb
|
6
|
+
lib/ultrasphinx/associations.rb
|
5
7
|
lib/ultrasphinx/autoload.rb
|
6
8
|
lib/ultrasphinx/configure.rb
|
7
9
|
lib/ultrasphinx/core_extensions.rb
|
@@ -21,6 +23,7 @@ lib/ultrasphinx/ultrasphinx.rb
|
|
21
23
|
lib/ultrasphinx.rb
|
22
24
|
LICENSE
|
23
25
|
Manifest
|
26
|
+
RAKE_TASKS
|
24
27
|
README
|
25
28
|
tasks/ultrasphinx.rake
|
26
29
|
test/config/ultrasphinx/test.base
|
@@ -35,6 +38,7 @@ test/integration/app/app/helpers/sellers_helper.rb
|
|
35
38
|
test/integration/app/app/helpers/states_helper.rb
|
36
39
|
test/integration/app/app/helpers/users_helper.rb
|
37
40
|
test/integration/app/app/models/geo/address.rb
|
41
|
+
test/integration/app/app/models/geo/country.rb
|
38
42
|
test/integration/app/app/models/geo/state.rb
|
39
43
|
test/integration/app/app/models/person/user.rb
|
40
44
|
test/integration/app/app/models/seller.rb
|
@@ -67,7 +71,6 @@ test/integration/app/config/environments/test.rb
|
|
67
71
|
test/integration/app/config/locomotive.yml
|
68
72
|
test/integration/app/config/routes.rb
|
69
73
|
test/integration/app/config/ultrasphinx/default.base
|
70
|
-
test/integration/app/config/ultrasphinx/development.conf
|
71
74
|
test/integration/app/config/ultrasphinx/development.conf.canonical
|
72
75
|
test/integration/app/db/migrate/001_create_users.rb
|
73
76
|
test/integration/app/db/migrate/002_create_sellers.rb
|
@@ -77,7 +80,7 @@ test/integration/app/db/migrate/005_add_capitalization_to_seller.rb
|
|
77
80
|
test/integration/app/db/migrate/006_add_deleted_to_user.rb
|
78
81
|
test/integration/app/db/migrate/007_add_lat_and_long_to_address.rb
|
79
82
|
test/integration/app/db/migrate/008_add_mission_statement_to_seller.rb
|
80
|
-
test/integration/app/db/
|
83
|
+
test/integration/app/db/migrate/009_create_countries.rb
|
81
84
|
test/integration/app/doc/README_FOR_APP
|
82
85
|
test/integration/app/public/404.html
|
83
86
|
test/integration/app/public/500.html
|
@@ -110,6 +113,7 @@ test/integration/app/script/process/spawner
|
|
110
113
|
test/integration/app/script/runner
|
111
114
|
test/integration/app/script/server
|
112
115
|
test/integration/app/test/fixtures/addresses.yml
|
116
|
+
test/integration/app/test/fixtures/countries.yml
|
113
117
|
test/integration/app/test/fixtures/sellers.yml
|
114
118
|
test/integration/app/test/fixtures/states.yml
|
115
119
|
test/integration/app/test/fixtures/users.yml
|
@@ -127,6 +131,7 @@ test/integration/search_test.rb
|
|
127
131
|
test/integration/server_test.rb
|
128
132
|
test/integration/spell_test.rb
|
129
133
|
test/setup.rb
|
134
|
+
test/teardown.rb
|
130
135
|
test/test_all.rb
|
131
136
|
test/test_helper.rb
|
132
137
|
test/ts.multi
|
data/RAKE_TASKS
ADDED
@@ -0,0 +1,15 @@
|
|
1
|
+
|
2
|
+
== Available Rake tasks
|
3
|
+
|
4
|
+
These Rake tasks are made available to your Rails app:
|
5
|
+
|
6
|
+
<tt>ultrasphinx:configure</tt>:: Rebuild the configuration file for this particular environment.
|
7
|
+
<tt>ultrasphinx:index</tt>:: Reindex the database and send an update signal to the search daemon.
|
8
|
+
<tt>ultrasphinx:daemon:restart</tt>:: Restart the search daemon.
|
9
|
+
<tt>ultrasphinx:daemon:start</tt>:: Start the search daemon.
|
10
|
+
<tt>ultrasphinx:daemon:stop</tt>:: Stop the search daemon.
|
11
|
+
<tt>ultrasphinx:daemon:status</tt>:: Check if the search daemon is running.
|
12
|
+
<tt>ultrasphinx:spelling:build</tt>:: Rebuild the custom spelling dictionary. You may need to use <tt>sudo</tt> if your Aspell folder is not writable by the app user.
|
13
|
+
<tt>ultrasphinx:bootstrap</tt>:: Bootstrap a full Sphinx environment by running configure, index, then daemon:start.
|
14
|
+
|
15
|
+
All tasks have shortcuts. Use <tt>us:conf</tt>, <tt>us:in</tt>, <tt>us:restart</tt>, <tt>us:start</tt>, <tt>us:stop</tt>, <tt>us:stat</tt>, <tt>us:spell</tt>, and <tt>us:boot</tt>.
|
data/README
CHANGED
@@ -11,9 +11,11 @@ The public certificate for the gem is at http://rubyforge.org/frs/download.php/2
|
|
11
11
|
|
12
12
|
== Requirements
|
13
13
|
|
14
|
-
* MySQL
|
15
|
-
* Sphinx 0.9.8-dev r871
|
16
|
-
* Rails 1.2.3
|
14
|
+
* MySQL 5.0, or PostgreSQL 8.2
|
15
|
+
* Sphinx 0.9.8-dev r871
|
16
|
+
* Rails 1.2.3
|
17
|
+
|
18
|
+
More recent versions than listed are ok.
|
17
19
|
|
18
20
|
== Features
|
19
21
|
|
@@ -65,15 +67,13 @@ For more index options, see ActiveRecord::Base .is_indexed.
|
|
65
67
|
|
66
68
|
== Building the index
|
67
69
|
|
68
|
-
|
70
|
+
Now run:
|
69
71
|
|
70
72
|
rake ultrasphinx:configure
|
71
73
|
rake ultrasphinx:index
|
72
74
|
rake ultrasphinx:daemon:start
|
73
75
|
|
74
|
-
To rotate the index, just rerun <tt>rake ultrasphinx:index</tt>. If the search daemon is running, it will have its index rotated. Otherwise the new index will be installed but the daemon will remain stopped.
|
75
|
-
|
76
|
-
Make sure to manually stop and restart the daemon if you change the field configuration or model set. It will not reload the configuration file automatically.
|
76
|
+
To rotate the index, just rerun <tt>rake ultrasphinx:index</tt>. If the search daemon is running, it will have its index rotated live. Otherwise the new index will be installed but the daemon will remain stopped.
|
77
77
|
|
78
78
|
== Running queries
|
79
79
|
|
@@ -97,18 +97,15 @@ See Ultrasphinx::Spell.
|
|
97
97
|
|
98
98
|
== Available Rake tasks
|
99
99
|
|
100
|
-
|
100
|
+
See RAKE_TASKS[link:files/RAKE_TASKS.html].
|
101
|
+
|
102
|
+
== Deployment notes
|
103
|
+
|
104
|
+
See DEPLOYMENT_NOTES[link:files/DEPLOYMENT_NOTES.html].
|
101
105
|
|
102
|
-
|
103
|
-
<tt>ultrasphinx:index</tt>:: Reindex the database and send an update signal to the search daemon.
|
104
|
-
<tt>ultrasphinx:daemon:restart</tt>:: Restart the search daemon.
|
105
|
-
<tt>ultrasphinx:daemon:start</tt>:: Start the search daemon.
|
106
|
-
<tt>ultrasphinx:daemon:stop</tt>:: Stop the search daemon.
|
107
|
-
<tt>ultrasphinx:daemon:status</tt>:: Check if the search daemon is running.
|
108
|
-
<tt>ultrasphinx:spelling:build</tt>:: Rebuild the custom spelling dictionary. You may need to use <tt>sudo</tt> if your Aspell folder is not writable by the app user.
|
109
|
-
<tt>ultrasphinx:bootstrap</tt>:: Bootstrap a full Sphinx environment by running configure, index, then daemon:start.
|
106
|
+
== A note about PostgreSQL
|
110
107
|
|
111
|
-
|
108
|
+
PostgreSQL 8.2 and higher are well supported. However, make sure you have executed <tt>CREATE LANGUAGE plpgsql;</tt> at least once. This step does not need to be repeated, so depending on your DB permissions, you might be able to put it in a migration.
|
112
109
|
|
113
110
|
== Reporting problems
|
114
111
|
|
data/TODO
CHANGED
@@ -1,7 +1,13 @@
|
|
1
1
|
|
2
|
-
|
2
|
+
Planned:
|
3
|
+
|
3
4
|
* Finish unifying filters and textfields
|
4
5
|
* Support exclude filters
|
5
6
|
* Make sure filters can be set to nil
|
6
7
|
* Geolocation search
|
8
|
+
|
9
|
+
Not sure:
|
10
|
+
|
11
|
+
* Delta indexing
|
12
|
+
* Use Pat Allan's association configurator, possibly with an API change to avoid the message-passing DSL
|
7
13
|
* Use Treetop for the query parser instead of regexes
|
data/examples/default.base
CHANGED
@@ -2,8 +2,21 @@
|
|
2
2
|
#
|
3
3
|
# Sphinx/Ultrasphinx user-configurable options.
|
4
4
|
#
|
5
|
-
# Copy this file to RAILS_ROOT/config/ultrasphinx.
|
6
|
-
#
|
5
|
+
# Copy this file to RAILS_ROOT/config/ultrasphinx. You can use individual
|
6
|
+
# namespaces if you want (e.g. development.base, production.base,
|
7
|
+
# test.base).
|
8
|
+
#
|
9
|
+
# This file should not be handed directly to Sphinx. Use the rake task
|
10
|
+
#
|
11
|
+
# rake ultrasphinx::configure
|
12
|
+
#
|
13
|
+
# to generate a parallel default.conf file. This is the file that Sphinx itself will
|
14
|
+
# use. The Ultrasphinx rake tasks automatically pass the correct file to
|
15
|
+
# to Sphinx.
|
16
|
+
#
|
17
|
+
# It is safe to edit .base files by hand. It is not safe to edit the generated
|
18
|
+
# .conf files. Do not symlink the .conf file to the .base file! I don't know why
|
19
|
+
# people think they need to do that. It's wrong.
|
7
20
|
#
|
8
21
|
|
9
22
|
indexer
|
@@ -18,6 +31,7 @@ searchd
|
|
18
31
|
# What interface the search daemon should listen on and where to store its logs
|
19
32
|
address = 0.0.0.0
|
20
33
|
port = 3312
|
34
|
+
seamless_rotate = 1
|
21
35
|
log = /opt/local/var/db/sphinx/log/searchd.log
|
22
36
|
query_log = /opt/local/var/db/sphinx/log/query.log
|
23
37
|
read_timeout = 5
|
data/lib/ultrasphinx.rb
CHANGED
@@ -12,12 +12,12 @@ end
|
|
12
12
|
|
13
13
|
$LOAD_PATH << "#{File.dirname(__FILE__)}/../vendor/riddle/lib"
|
14
14
|
require 'riddle'
|
15
|
-
|
16
15
|
require 'ultrasphinx/ultrasphinx'
|
16
|
+
require 'ultrasphinx/associations'
|
17
17
|
require 'ultrasphinx/core_extensions'
|
18
18
|
require 'ultrasphinx/is_indexed'
|
19
19
|
|
20
|
-
if (ActiveRecord::Base.connection rescue nil)
|
20
|
+
if (ActiveRecord::Base.connection rescue nil) # XXX Forget why this needed to be wrapped
|
21
21
|
require 'ultrasphinx/configure'
|
22
22
|
require 'ultrasphinx/autoload'
|
23
23
|
require 'ultrasphinx/fields'
|
@@ -0,0 +1,25 @@
|
|
1
|
+
module Ultrasphinx
|
2
|
+
module Associations
|
3
|
+
|
4
|
+
def get_association(klass, entry)
|
5
|
+
if value = entry['class_name']
|
6
|
+
klass.reflect_on_all_associations.detect do |assoc|
|
7
|
+
assoc.class_name == value
|
8
|
+
end
|
9
|
+
elsif value = entry['association_name']
|
10
|
+
klass.reflect_on_all_associations.detect do |assoc|
|
11
|
+
assoc.name.to_s == value.to_s
|
12
|
+
end
|
13
|
+
end
|
14
|
+
end
|
15
|
+
|
16
|
+
def get_association_model(klass, entry)
|
17
|
+
get_association(klass, entry).class_name.constantize
|
18
|
+
end
|
19
|
+
|
20
|
+
def entry_identifies_association?(entry)
|
21
|
+
entry['class_name'] || entry['association_name']
|
22
|
+
end
|
23
|
+
|
24
|
+
end
|
25
|
+
end
|
@@ -1,7 +1,9 @@
|
|
1
1
|
|
2
2
|
module Ultrasphinx
|
3
3
|
class Configure
|
4
|
-
class << self
|
4
|
+
class << self
|
5
|
+
|
6
|
+
include Associations
|
5
7
|
|
6
8
|
# Force all the indexed models to load and register in the MODEL_CONFIGURATION hash.
|
7
9
|
def load_constants
|
@@ -75,7 +77,7 @@ module Ultrasphinx
|
|
75
77
|
# Supporting Postgres now
|
76
78
|
connection_settings = klass.connection.instance_variable_get("@config")
|
77
79
|
|
78
|
-
adapter_defaults =
|
80
|
+
adapter_defaults = DEFAULTS[ADAPTER]
|
79
81
|
raise ConfigurationError, "Unsupported database adapter" unless adapter_defaults
|
80
82
|
|
81
83
|
conf = [adapter_defaults]
|
@@ -100,7 +102,12 @@ module Ultrasphinx
|
|
100
102
|
|
101
103
|
|
102
104
|
def range_select_string(klass)
|
103
|
-
"sql_query_range = SELECT
|
105
|
+
["sql_query_range = SELECT",
|
106
|
+
SQL_FUNCTIONS[ADAPTER]['range_cast']._interpolate("MIN(#{klass.primary_key})"),
|
107
|
+
", ",
|
108
|
+
SQL_FUNCTIONS[ADAPTER]['range_cast']._interpolate("MAX(#{klass.primary_key})"),
|
109
|
+
"FROM #{klass.table_name}"
|
110
|
+
].join(" ")
|
104
111
|
end
|
105
112
|
|
106
113
|
|
@@ -189,11 +196,15 @@ module Ultrasphinx
|
|
189
196
|
|
190
197
|
def build_includes(klass, fields, entries, column_strings, join_strings, group_bys, remaining_columns)
|
191
198
|
entries.to_a.each do |entry|
|
199
|
+
raise ConfigurationError, "You must identify your association with either class_name or association_name, but not both" if entry['class_name'] && entry ['association_name']
|
192
200
|
|
193
|
-
|
194
|
-
|
201
|
+
association = get_association(klass, entry)
|
202
|
+
|
203
|
+
# You can use 'class_name' and 'association_sql' to associate to a model that doesn't actually
|
204
|
+
# have an association
|
205
|
+
join_klass = association ? association.class_name.constantize : entry['class_name'].constantize
|
195
206
|
|
196
|
-
raise ConfigurationError, "Unknown association from #{klass} to #{entry['class_name']}" if not association and not entry['association_sql']
|
207
|
+
raise ConfigurationError, "Unknown association from #{klass} to #{entry['class_name'] || entry['association_name']}" if not association and not entry['association_sql']
|
197
208
|
|
198
209
|
join_strings = install_join_unless_association_sql(entry['association_sql'], nil, join_strings) do
|
199
210
|
"LEFT OUTER JOIN #{join_klass.table_name} AS #{entry['table']} ON " +
|
@@ -217,21 +228,26 @@ module Ultrasphinx
|
|
217
228
|
|
218
229
|
def build_concatenations(klass, fields, entries, column_strings, join_strings, group_bys, use_distinct, remaining_columns)
|
219
230
|
entries.to_a.each do |entry|
|
220
|
-
if entry
|
231
|
+
if entry_identifies_association?(entry) and entry['field']
|
221
232
|
# Group concats
|
233
|
+
|
222
234
|
# Only has_many's or explicit sql right now
|
223
|
-
|
235
|
+
association = get_association(klass, entry)
|
236
|
+
|
237
|
+
# You can use 'class_name' and 'association_sql' to associate to a model that doesn't actually
|
238
|
+
# have an association
|
239
|
+
join_klass = association ? association.class_name.constantize : entry['class_name'].constantize
|
224
240
|
|
225
241
|
join_strings = install_join_unless_association_sql(entry['association_sql'], nil, join_strings) do
|
226
242
|
# XXX make sure foreign key is right for polymorphic relationships
|
227
|
-
association =
|
243
|
+
association = get_association(klass, entry)
|
228
244
|
"LEFT OUTER JOIN #{join_klass.table_name} AS #{entry['table']} ON #{klass.table_name}.#{klass.primary_key} = #{entry['table']}.#{association.primary_key_name}" +
|
229
245
|
(entry['conditions'] ? " AND (#{entry['conditions']})" : "")
|
230
246
|
end
|
231
247
|
|
232
248
|
source_string = "#{entry['table']}.#{entry['field']}"
|
233
249
|
# We are using the field in an aggregate, so we don't want to add it to group_bys
|
234
|
-
source_string =
|
250
|
+
source_string = SQL_FUNCTIONS[ADAPTER]['group_concat']._interpolate(source_string)
|
235
251
|
use_distinct = true
|
236
252
|
|
237
253
|
column_strings, remaining_columns = install_field(fields, source_string, entry['as'], entry['function_sql'], entry['facet'], column_strings, remaining_columns)
|
@@ -284,12 +300,6 @@ module Ultrasphinx
|
|
284
300
|
def install_join_unless_association_sql(association_sql, join_string, join_strings)
|
285
301
|
join_strings << (association_sql or join_string or yield)
|
286
302
|
end
|
287
|
-
|
288
|
-
def association_by_class_name(klass, class_name)
|
289
|
-
klass.reflect_on_all_associations.detect do |assoc|
|
290
|
-
assoc.class_name == class_name
|
291
|
-
end
|
292
|
-
end
|
293
303
|
|
294
304
|
def say(s)
|
295
305
|
Ultrasphinx.say s
|
data/lib/ultrasphinx/fields.rb
CHANGED
@@ -7,6 +7,7 @@ This is a special singleton configuration class that stores the index field conf
|
|
7
7
|
|
8
8
|
class Fields
|
9
9
|
include Singleton
|
10
|
+
include Associations
|
10
11
|
|
11
12
|
TYPE_MAP = {
|
12
13
|
'string' => 'text',
|
@@ -114,14 +115,16 @@ This is a special singleton configuration class that stores the index field conf
|
|
114
115
|
options['include'].to_a.each do |entry|
|
115
116
|
extract_table_alias!(entry, klass)
|
116
117
|
extract_field_alias!(entry, klass)
|
117
|
-
|
118
|
-
|
118
|
+
|
119
|
+
association_model = get_association_model(klass, entry)
|
120
|
+
|
121
|
+
save_and_verify_type(entry['as'] || entry['field'], association_model.columns_hash[entry['field']].type, entry['sortable'], klass)
|
119
122
|
install_facets!(entry, klass)
|
120
123
|
end
|
121
124
|
|
122
125
|
# Regular concats are CHAR, group_concats are BLOB and need to be cast to CHAR
|
123
126
|
options['concatenate'].to_a.each do |entry|
|
124
|
-
extract_table_alias!(entry, klass)
|
127
|
+
extract_table_alias!(entry, klass)
|
125
128
|
save_and_verify_type(entry['as'], 'text', entry['sortable'], klass)
|
126
129
|
install_facets!(entry, klass)
|
127
130
|
end
|
@@ -155,7 +158,7 @@ This is a special singleton configuration class that stores the index field conf
|
|
155
158
|
# This field is referenced by a table alias
|
156
159
|
entry['table'], entry['field'] = entry['field'].split(".")
|
157
160
|
else
|
158
|
-
klass = entry
|
161
|
+
klass = get_association_model(klass, entry) if entry_identifies_association?(entry)
|
159
162
|
entry['table'] = klass.table_name
|
160
163
|
end
|
161
164
|
end
|
@@ -32,47 +32,59 @@ To apply an SQL function to a field before it is indexed, use the key <tt>:funct
|
|
32
32
|
|
33
33
|
Note that <tt>float</tt> fields are supported, but require Sphinx 0.98.
|
34
34
|
|
35
|
+
== Requiring conditions
|
36
|
+
|
37
|
+
Use the <tt>:conditions</tt> key.
|
38
|
+
|
39
|
+
SQL conditions, to scope which records are selected for indexing. Accepts a string.
|
40
|
+
|
41
|
+
:conditions => "created_at < NOW() AND deleted IS NOT NULL"
|
42
|
+
|
43
|
+
The <tt>:conditions</tt> key is especially useful if you delete records by marking them deleted rather than removing them from the database.
|
44
|
+
|
35
45
|
== Including a field from an association
|
36
46
|
|
37
47
|
Use the <tt>:include</tt> key.
|
38
48
|
|
39
49
|
Accepts an array of hashes.
|
40
50
|
|
41
|
-
:include => [{:
|
51
|
+
:include => [{:association_name => 'category', :field => 'name', :as => 'category_name'}]
|
42
52
|
|
43
|
-
Each should contain
|
53
|
+
Each should contain an <tt>:association_name</tt> key (the association name for the included model), a <tt>:field</tt> key (the name of the field to include), and an optional <tt>:as</tt> key (what to name the field in the parent).
|
44
54
|
|
45
|
-
|
55
|
+
<tt>:include</tt> hashes also accept their own <tt>:conditions</tt> key. You can use this if you need custom WHERE conditions for this particular association (e.g, this JOIN).
|
46
56
|
|
47
|
-
|
57
|
+
The keys <tt>:facet</tt>, <tt>:sortable</tt>, <tt>:class_name</tt>, <tt>:association_sql</tt>, and <tt>:function_sql</tt> are also recognized.
|
48
58
|
|
49
|
-
|
50
|
-
|
51
|
-
SQL conditions, to scope which records are selected for indexing. Accepts a string.
|
52
|
-
:conditions => "created_at < NOW() AND deleted IS NOT NULL"
|
53
|
-
The <tt>:conditions</tt> key is especially useful if you delete records by marking them deleted rather than removing them from the database.
|
54
|
-
|
55
|
-
== Concatenating several fields within a record
|
59
|
+
== Concatenating several fields within one record
|
56
60
|
|
57
61
|
Use the <tt>:concatenate</tt> key (MySQL only).
|
58
62
|
|
59
63
|
Accepts an array of option hashes.
|
60
64
|
|
61
|
-
To concatenate several fields within one record as a combined field, use a regular (or
|
65
|
+
To concatenate several fields within one record as a combined field, use a regular (or lateral) concatenation. Regular concatenations contain a <tt>:fields</tt> key (again, an array of field names), and a mandatory <tt>:as</tt> key (the name of the result of the concatenation). For example, to concatenate the <tt>title</tt> and <tt>body</tt> into one field called <tt>text</tt>:
|
62
66
|
:concatenate => [{:fields => ['title', 'body'], :as => 'text'}]
|
63
67
|
|
64
|
-
The keys <tt>:facet</tt>, <tt>:sortable</tt>,
|
68
|
+
The keys <tt>:facet</tt>, <tt>:sortable</tt>, <tt>:conditions</tt>, <tt>:function_sql</tt>, <tt>:class_name</tt>, and <tt>:association_sql</tt>, are also recognized.
|
69
|
+
|
70
|
+
Lateral concatenations are implemented with CONCAT_WS on MySQL and with a stored procedure on PostgreSQL.
|
65
71
|
|
66
|
-
== Concatenating
|
72
|
+
== Concatenating the same field from a set of associated records
|
67
73
|
|
68
74
|
Also use the <tt>:concatenate</tt> key.
|
69
75
|
|
70
|
-
To concatenate one field from a set of associated records as a combined field in the parent record, use a group (or vertical) concatenation. A group concatenation should contain
|
71
|
-
:concatenate => [{:
|
76
|
+
To concatenate one field from a set of associated records as a combined field in the parent record, use a group (or vertical) concatenation. A group concatenation should contain an <tt>:association_name</tt> key (the association name for the included model), a <tt>:field</tt> key (the field on the included model to concatenate), and an optional <tt>:as</tt> key (also the name of the result of the concatenation). For example, to concatenate all <tt>Post#body</tt> contents into the parent's <tt>responses</tt> field:
|
77
|
+
:concatenate => [{:association_name => 'posts', :field => 'body', :as => 'responses'}]
|
78
|
+
|
79
|
+
The keys <tt>:facet</tt>, <tt>:sortable</tt>, <tt>:conditions</tt>, <tt>:function_sql</tt>, <tt>:class_name</tt>, and <tt>:association_sql</tt>, are also recognized.
|
80
|
+
|
81
|
+
Vertical concatenations are implemented with GROUP_CONCAT on MySQL and with an aggregate and a stored procedure on PostgreSQL.
|
82
|
+
|
83
|
+
== Custom joins
|
72
84
|
|
73
|
-
|
85
|
+
<tt>:include</tt> and <tt>:concatenate</tt> accept an <tt>:association_sql</tt> key. You can use this if you need to pass a custom JOIN string, for example, a double JOIN for a <tt>has_many :through</tt>). If <tt>:association_sql</tt> is present, the default JOIN for <tt>belongs_to</tt> will not be generated.
|
74
86
|
|
75
|
-
|
87
|
+
Also, If you want to include a model that you don't have an actual ActiveRecord association for, you can use <tt>:association_sql</tt> combined with <tt>:class_name</tt> instead of <tt>:association_name</tt>. <tt>:class_name</tt> should be camelcase.
|
76
88
|
|
77
89
|
Ultrasphinx is not an object-relational mapper, and the association generation is intended to stay minimal--don't be afraid of <tt>:association_sql</tt>.
|
78
90
|
|
@@ -89,13 +101,13 @@ Here's an example configuration using most of the options, taken from production
|
|
89
101
|
{:field => 'author', :facet => true}
|
90
102
|
],
|
91
103
|
:include => [
|
92
|
-
{:
|
104
|
+
{:association_name => 'category', :field => 'name', :as => 'category_name'}
|
93
105
|
],
|
94
106
|
:concatenate => [
|
95
107
|
{:fields => ['title', 'long_description', 'short_description'],
|
96
108
|
:as => 'editorial'},
|
97
|
-
{:
|
98
|
-
{:
|
109
|
+
{:association_name => 'pages', :field => 'body', :as => 'body'},
|
110
|
+
{:association_name => 'comments', :field => 'body', :as => 'comments',
|
99
111
|
:conditions => "comments.item_type = '#{base_class}'"}
|
100
112
|
],
|
101
113
|
:conditions => self.live_condition_string
|
@@ -110,7 +122,7 @@ A common use case is to only search records that belong to a particular parent m
|
|
110
122
|
For example, say a Company <tt>has_many :users</tt> and each User <tt>has_many :articles</tt>. If you want to to filter Articles by Company, add <tt>company_id</tt> to the Article's <tt>is_indexed</tt> method. The best way is to grab it from the User association:
|
111
123
|
|
112
124
|
class Article < ActiveRecord::Base
|
113
|
-
is_indexed :include => [{:
|
125
|
+
is_indexed :include => [{:association_name => 'users', :field => 'company_id'}]
|
114
126
|
end
|
115
127
|
|
116
128
|
Now you can run:
|
@@ -136,8 +148,8 @@ If the associations weren't just <tt>has_many</tt> and <tt>belongs_to</tt>, you
|
|
136
148
|
|
137
149
|
Array(opts['concatenate']).each do |entry|
|
138
150
|
entry.stringify_keys!
|
139
|
-
entry.assert_valid_keys ['class_name', 'conditions', 'field', 'as', 'fields', 'association_sql', 'facet', 'function_sql', 'sortable']
|
140
|
-
raise Ultrasphinx::ConfigurationError, "You can't mix regular concat and group concats" if entry['fields'] and (entry['field'] or entry['class_name'])
|
151
|
+
entry.assert_valid_keys ['class_name', 'association_name', 'conditions', 'field', 'as', 'fields', 'association_sql', 'facet', 'function_sql', 'sortable']
|
152
|
+
raise Ultrasphinx::ConfigurationError, "You can't mix regular concat and group concats" if entry['fields'] and (entry['field'] or entry['class_name'] or entry['association_name'])
|
141
153
|
raise Ultrasphinx::ConfigurationError, "Concatenations must specify an :as key" unless entry['as']
|
142
154
|
raise Ultrasphinx::ConfigurationError, "Group concatenations must not have multiple fields" if entry['field'].is_a? Array
|
143
155
|
raise Ultrasphinx::ConfigurationError, "Regular concatenations should have multiple fields" if entry['fields'] and !entry['fields'].is_a?(Array)
|
@@ -145,7 +157,7 @@ If the associations weren't just <tt>has_many</tt> and <tt>belongs_to</tt>, you
|
|
145
157
|
|
146
158
|
Array(opts['include']).each do |entry|
|
147
159
|
entry.stringify_keys!
|
148
|
-
entry.assert_valid_keys ['class_name', 'field', 'as', 'association_sql', 'facet', 'function_sql', 'sortable']
|
160
|
+
entry.assert_valid_keys ['class_name', 'association_name', 'field', 'as', 'association_sql', 'facet', 'function_sql', 'sortable']
|
149
161
|
end
|
150
162
|
|
151
163
|
Ultrasphinx::MODEL_CONFIGURATION[self.name] = opts
|