load_balanced_tire 0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (121) hide show
  1. data/.gitignore +14 -0
  2. data/.travis.yml +29 -0
  3. data/Gemfile +4 -0
  4. data/MIT-LICENSE +20 -0
  5. data/README.markdown +760 -0
  6. data/Rakefile +78 -0
  7. data/examples/rails-application-template.rb +249 -0
  8. data/examples/tire-dsl.rb +876 -0
  9. data/lib/tire.rb +55 -0
  10. data/lib/tire/alias.rb +296 -0
  11. data/lib/tire/configuration.rb +30 -0
  12. data/lib/tire/dsl.rb +43 -0
  13. data/lib/tire/http/client.rb +62 -0
  14. data/lib/tire/http/clients/curb.rb +61 -0
  15. data/lib/tire/http/clients/faraday.rb +71 -0
  16. data/lib/tire/http/response.rb +27 -0
  17. data/lib/tire/index.rb +361 -0
  18. data/lib/tire/logger.rb +60 -0
  19. data/lib/tire/model/callbacks.rb +40 -0
  20. data/lib/tire/model/import.rb +26 -0
  21. data/lib/tire/model/indexing.rb +128 -0
  22. data/lib/tire/model/naming.rb +100 -0
  23. data/lib/tire/model/percolate.rb +99 -0
  24. data/lib/tire/model/persistence.rb +71 -0
  25. data/lib/tire/model/persistence/attributes.rb +143 -0
  26. data/lib/tire/model/persistence/finders.rb +66 -0
  27. data/lib/tire/model/persistence/storage.rb +69 -0
  28. data/lib/tire/model/search.rb +307 -0
  29. data/lib/tire/results/collection.rb +114 -0
  30. data/lib/tire/results/item.rb +86 -0
  31. data/lib/tire/results/pagination.rb +54 -0
  32. data/lib/tire/rubyext/hash.rb +8 -0
  33. data/lib/tire/rubyext/ruby_1_8.rb +7 -0
  34. data/lib/tire/rubyext/symbol.rb +11 -0
  35. data/lib/tire/search.rb +188 -0
  36. data/lib/tire/search/facet.rb +74 -0
  37. data/lib/tire/search/filter.rb +28 -0
  38. data/lib/tire/search/highlight.rb +37 -0
  39. data/lib/tire/search/query.rb +186 -0
  40. data/lib/tire/search/scan.rb +114 -0
  41. data/lib/tire/search/script_field.rb +23 -0
  42. data/lib/tire/search/sort.rb +25 -0
  43. data/lib/tire/tasks.rb +135 -0
  44. data/lib/tire/utils.rb +17 -0
  45. data/lib/tire/version.rb +22 -0
  46. data/test/fixtures/articles/1.json +1 -0
  47. data/test/fixtures/articles/2.json +1 -0
  48. data/test/fixtures/articles/3.json +1 -0
  49. data/test/fixtures/articles/4.json +1 -0
  50. data/test/fixtures/articles/5.json +1 -0
  51. data/test/integration/active_model_indexing_test.rb +51 -0
  52. data/test/integration/active_model_searchable_test.rb +114 -0
  53. data/test/integration/active_record_searchable_test.rb +446 -0
  54. data/test/integration/boolean_queries_test.rb +43 -0
  55. data/test/integration/count_test.rb +34 -0
  56. data/test/integration/custom_score_queries_test.rb +88 -0
  57. data/test/integration/dis_max_queries_test.rb +68 -0
  58. data/test/integration/dsl_search_test.rb +22 -0
  59. data/test/integration/explanation_test.rb +44 -0
  60. data/test/integration/facets_test.rb +259 -0
  61. data/test/integration/filtered_queries_test.rb +66 -0
  62. data/test/integration/filters_test.rb +63 -0
  63. data/test/integration/fuzzy_queries_test.rb +20 -0
  64. data/test/integration/highlight_test.rb +64 -0
  65. data/test/integration/index_aliases_test.rb +122 -0
  66. data/test/integration/index_mapping_test.rb +43 -0
  67. data/test/integration/index_store_test.rb +96 -0
  68. data/test/integration/index_update_document_test.rb +111 -0
  69. data/test/integration/mongoid_searchable_test.rb +309 -0
  70. data/test/integration/percolator_test.rb +111 -0
  71. data/test/integration/persistent_model_test.rb +130 -0
  72. data/test/integration/prefix_query_test.rb +43 -0
  73. data/test/integration/query_return_version_test.rb +70 -0
  74. data/test/integration/query_string_test.rb +52 -0
  75. data/test/integration/range_queries_test.rb +36 -0
  76. data/test/integration/reindex_test.rb +46 -0
  77. data/test/integration/results_test.rb +39 -0
  78. data/test/integration/scan_test.rb +56 -0
  79. data/test/integration/script_fields_test.rb +38 -0
  80. data/test/integration/sort_test.rb +36 -0
  81. data/test/integration/text_query_test.rb +39 -0
  82. data/test/models/active_model_article.rb +31 -0
  83. data/test/models/active_model_article_with_callbacks.rb +49 -0
  84. data/test/models/active_model_article_with_custom_document_type.rb +7 -0
  85. data/test/models/active_model_article_with_custom_index_name.rb +7 -0
  86. data/test/models/active_record_models.rb +122 -0
  87. data/test/models/article.rb +15 -0
  88. data/test/models/mongoid_models.rb +97 -0
  89. data/test/models/persistent_article.rb +11 -0
  90. data/test/models/persistent_article_in_namespace.rb +12 -0
  91. data/test/models/persistent_article_with_casting.rb +28 -0
  92. data/test/models/persistent_article_with_defaults.rb +11 -0
  93. data/test/models/persistent_articles_with_custom_index_name.rb +10 -0
  94. data/test/models/supermodel_article.rb +17 -0
  95. data/test/models/validated_model.rb +11 -0
  96. data/test/test_helper.rb +93 -0
  97. data/test/unit/active_model_lint_test.rb +17 -0
  98. data/test/unit/configuration_test.rb +74 -0
  99. data/test/unit/http_client_test.rb +76 -0
  100. data/test/unit/http_response_test.rb +49 -0
  101. data/test/unit/index_alias_test.rb +275 -0
  102. data/test/unit/index_test.rb +894 -0
  103. data/test/unit/logger_test.rb +125 -0
  104. data/test/unit/model_callbacks_test.rb +116 -0
  105. data/test/unit/model_import_test.rb +71 -0
  106. data/test/unit/model_persistence_test.rb +528 -0
  107. data/test/unit/model_search_test.rb +913 -0
  108. data/test/unit/results_collection_test.rb +281 -0
  109. data/test/unit/results_item_test.rb +162 -0
  110. data/test/unit/rubyext_test.rb +66 -0
  111. data/test/unit/search_facet_test.rb +153 -0
  112. data/test/unit/search_filter_test.rb +42 -0
  113. data/test/unit/search_highlight_test.rb +46 -0
  114. data/test/unit/search_query_test.rb +301 -0
  115. data/test/unit/search_scan_test.rb +113 -0
  116. data/test/unit/search_script_field_test.rb +26 -0
  117. data/test/unit/search_sort_test.rb +50 -0
  118. data/test/unit/search_test.rb +499 -0
  119. data/test/unit/tire_test.rb +126 -0
  120. data/tire.gemspec +90 -0
  121. metadata +549 -0
data/.gitignore ADDED
@@ -0,0 +1,14 @@
1
+ *.gem
2
+ .bundle
3
+ Gemfile.lock
4
+ pkg/*
5
+ rdoc/
6
+ doc/
7
+ .yardoc/
8
+ coverage/
9
+ scratch/
10
+ examples/*.html
11
+ *.log
12
+ .rvmrc
13
+ .rbenv-version
14
+ tags
data/.travis.yml ADDED
@@ -0,0 +1,29 @@
1
+ # ---------------------------------------------------------
2
+ # Configuration file for http://travis-ci.org/#!/karmi/tire
3
+ # ---------------------------------------------------------
4
+
5
+ language: ruby
6
+
7
+ rvm:
8
+ - 1.9.3
9
+ - 1.8.7
10
+ - ree
11
+
12
+ env:
13
+ - TEST_COMMAND="rake test:unit"
14
+ - TEST_COMMAND="rake test:integration"
15
+
16
+ script: "bundle exec $TEST_COMMAND"
17
+
18
+ before_install:
19
+ - sudo service elasticsearch start
20
+
21
+ matrix:
22
+ exclude:
23
+ - rvm: 1.8.7
24
+ env: TEST_COMMAND="rake test:integration"
25
+ - rvm: ree
26
+ env: TEST_COMMAND="rake test:integration"
27
+
28
+ notifications:
29
+ disable: true
data/Gemfile ADDED
@@ -0,0 +1,4 @@
1
+ source "http://rubygems.org"
2
+
3
+ # Specify your gem's dependencies in tire.gemspec
4
+ gemspec
data/MIT-LICENSE ADDED
@@ -0,0 +1,20 @@
1
+ Copyright 2011 Karel Minarik
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining
4
+ a copy of this software and associated documentation files (the
5
+ "Software"), to deal in the Software without restriction, including
6
+ without limitation the rights to use, copy, modify, merge, publish,
7
+ distribute, sublicense, and/or sell copies of the Software, and to
8
+ permit persons to whom the Software is furnished to do so, subject to
9
+ the following conditions:
10
+
11
+ The above copyright notice and this permission notice shall be
12
+ included in all copies or substantial portions of the Software.
13
+
14
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
15
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
16
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
17
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
18
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
19
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
20
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.markdown ADDED
@@ -0,0 +1,760 @@
1
+ Tire
2
+ =========
3
+
4
+ _Tire_ is a Ruby (1.8 or 1.9) client for the [ElasticSearch](http://www.elasticsearch.org/)
5
+ search engine/database.
6
+
7
+ _ElasticSearch_ is a scalable, distributed, cloud-ready, highly-available,
8
+ full-text search engine and database with
9
+ [powerfull aggregation features](http://www.elasticsearch.org/guide/reference/api/search/facets/),
10
+ communicating by JSON over RESTful HTTP, based on [Lucene](http://lucene.apache.org/), written in Java.
11
+
12
+ This Readme provides a brief overview of _Tire's_ features. The more detailed documentation is at <http://karmi.github.com/tire/>.
13
+
14
+ Both of these documents contain a lot of information. Please set aside some time to read them thoroughly, before you blindly dive into „somehow making it work“. Just skimming through it **won't work** for you. For more information, please refer to the [integration test suite](https://github.com/karmi/tire/tree/master/test/integration)
15
+ and [issues](https://github.com/karmi/tire/issues).
16
+
17
+ Installation
18
+ ------------
19
+
20
+ OK. First, you need a running _ElasticSearch_ server. Thankfully, it's easy. Let's define easy:
21
+
22
+ $ curl -k -L -o elasticsearch-0.19.0.tar.gz http://github.com/downloads/elasticsearch/elasticsearch/elasticsearch-0.19.0.tar.gz
23
+ $ tar -zxvf elasticsearch-0.19.0.tar.gz
24
+ $ ./elasticsearch-0.19.0/bin/elasticsearch -f
25
+
26
+ See, easy. On a Mac, you can also use _Homebrew_:
27
+
28
+ $ brew install elasticsearch
29
+
30
+ Now, let's install the gem via Rubygems:
31
+
32
+ $ gem install tire
33
+
34
+ Of course, you can install it from the source as well:
35
+
36
+ $ git clone git://github.com/karmi/tire.git
37
+ $ cd tire
38
+ $ rake install
39
+
40
+
41
+ Usage
42
+ -----
43
+
44
+ _Tire_ exposes easy-to-use domain specific language for fluent communication with _ElasticSearch_.
45
+
46
+ It easily blends with your _ActiveModel_/_ActiveRecord_ classes for convenient usage in _Rails_ applications.
47
+
48
+ To test-drive the core _ElasticSearch_ functionality, let's require the gem:
49
+
50
+ ```ruby
51
+ require 'rubygems'
52
+ require 'tire'
53
+ ```
54
+
55
+ Please note that you can copy these snippets from the much more extensive and heavily annotated file
56
+ in [examples/tire-dsl.rb](http://karmi.github.com/tire/).
57
+
58
+ Also, note that we're doing some heavy JSON lifting here. _Tire_ uses the
59
+ [_multi_json_](https://github.com/intridea/multi_json) gem as a generic JSON wrapper,
60
+ which allows you to use your preferred JSON library. We'll use the
61
+ [_yajl-ruby_](https://github.com/brianmario/yajl-ruby) gem in the full on mode here:
62
+
63
+ ```ruby
64
+ require 'yajl/json_gem'
65
+ ```
66
+
67
+ Let's create an index named `articles` and store/index some documents:
68
+
69
+ ```ruby
70
+ Tire.index 'articles' do
71
+ delete
72
+ create
73
+
74
+ store :title => 'One', :tags => ['ruby']
75
+ store :title => 'Two', :tags => ['ruby', 'python']
76
+ store :title => 'Three', :tags => ['java']
77
+ store :title => 'Four', :tags => ['ruby', 'php']
78
+
79
+ refresh
80
+ end
81
+ ```
82
+
83
+ We can also create the index with custom
84
+ [mapping](http://www.elasticsearch.org/guide/reference/api/admin-indices-create-index.html)
85
+ for a specific document type:
86
+
87
+ ```ruby
88
+ Tire.index 'articles' do
89
+ delete
90
+
91
+ create :mappings => {
92
+ :article => {
93
+ :properties => {
94
+ :id => { :type => 'string', :index => 'not_analyzed', :include_in_all => false },
95
+ :title => { :type => 'string', :boost => 2.0, :analyzer => 'snowball' },
96
+ :tags => { :type => 'string', :analyzer => 'keyword' },
97
+ :content => { :type => 'string', :analyzer => 'snowball' }
98
+ }
99
+ }
100
+ }
101
+ end
102
+ ```
103
+
104
+ Of course, we may have large amounts of data, and it may be impossible or impractical to add them to the index
105
+ one by one. We can use _ElasticSearch's_
106
+ [bulk storage](http://www.elasticsearch.org/guide/reference/api/bulk.html).
107
+ Notice, that collection items must have an `id` property or method,
108
+ and should have a `type` property, if you've set any specific mapping for the index.
109
+
110
+ ```ruby
111
+ articles = [
112
+ { :id => '1', :type => 'article', :title => 'one', :tags => ['ruby'] },
113
+ { :id => '2', :type => 'article', :title => 'two', :tags => ['ruby', 'python'] },
114
+ { :id => '3', :type => 'article', :title => 'three', :tags => ['java'] },
115
+ { :id => '4', :type => 'article', :title => 'four', :tags => ['ruby', 'php'] }
116
+ ]
117
+
118
+ Tire.index 'articles' do
119
+ import articles
120
+ end
121
+ ```
122
+
123
+ We can easily manipulate the documents before storing them in the index, by passing a block to the
124
+ `import` method, like this:
125
+
126
+ ```ruby
127
+ Tire.index 'articles' do
128
+ import articles do |documents|
129
+
130
+ documents.each { |document| document[:title].capitalize! }
131
+ end
132
+
133
+ refresh
134
+ end
135
+ ```
136
+
137
+ If this _declarative_ notation does not fit well in your context,
138
+ you can use _Tire's_ classes directly, in a more imperative manner:
139
+
140
+ ```ruby
141
+ index = Tire::Index.new('oldskool')
142
+ index.delete
143
+ index.create
144
+ index.store :title => "Let's do it the old way!"
145
+ index.refresh
146
+ ```
147
+
148
+ OK. Now, let's go search all the data.
149
+
150
+ We will be searching for articles whose `title` begins with letter “T”, sorted by `title` in `descending` order,
151
+ filtering them for ones tagged “ruby”, and also retrieving some [_facets_](http://www.elasticsearch.org/guide/reference/api/search/facets/)
152
+ from the database:
153
+
154
+ ```ruby
155
+ s = Tire.search 'articles' do
156
+ query do
157
+ string 'title:T*'
158
+ end
159
+
160
+ filter :terms, :tags => ['ruby']
161
+
162
+ sort { by :title, 'desc' }
163
+
164
+ facet 'global-tags', :global => true do
165
+ terms :tags
166
+ end
167
+
168
+ facet 'current-tags' do
169
+ terms :tags
170
+ end
171
+ end
172
+ ```
173
+
174
+ (Of course, we may also page the results with `from` and `size` query options, retrieve only specific fields
175
+ or highlight content matching our query, etc.)
176
+
177
+ Let's display the results:
178
+
179
+ ```ruby
180
+ s.results.each do |document|
181
+ puts "* #{ document.title } [tags: #{document.tags.join(', ')}]"
182
+ end
183
+
184
+ # * Two [tags: ruby, python]
185
+ ```
186
+
187
+ Let's display the global facets (distribution of tags across the whole database):
188
+
189
+ ```ruby
190
+ s.results.facets['global-tags']['terms'].each do |f|
191
+ puts "#{f['term'].ljust(10)} #{f['count']}"
192
+ end
193
+
194
+ # ruby 3
195
+ # python 1
196
+ # php 1
197
+ # java 1
198
+ ```
199
+
200
+ Now, let's display the facets based on current query (notice that count for articles
201
+ tagged with 'java' is included, even though it's not returned by our query;
202
+ count for articles tagged 'php' is excluded, since they don't match the current query):
203
+
204
+ ```ruby
205
+ s.results.facets['current-tags']['terms'].each do |f|
206
+ puts "#{f['term'].ljust(10)} #{f['count']}"
207
+ end
208
+
209
+ # ruby 1
210
+ # python 1
211
+ # java 1
212
+ ```
213
+
214
+ Notice, that only variables from the enclosing scope are accessible.
215
+ If we want to access the variables or methods from outer scope,
216
+ we have to use a slight variation of the DSL, by passing the
217
+ `search` and `query` objects around.
218
+
219
+ ```ruby
220
+ @query = 'title:T*'
221
+
222
+ Tire.search 'articles' do |search|
223
+ search.query do |query|
224
+ query.string @query
225
+ end
226
+ end
227
+ ```
228
+
229
+ Quite often, we need complex queries with boolean logic.
230
+ Instead of composing long query strings such as `tags:ruby OR tags:java AND NOT tags:python`,
231
+ we can use the [_bool_](http://www.elasticsearch.org/guide/reference/query-dsl/bool-query.html)
232
+ query. In _Tire_, we build them declaratively.
233
+
234
+ ```ruby
235
+ Tire.search 'articles' do
236
+ query do
237
+ boolean do
238
+ should { string 'tags:ruby' }
239
+ should { string 'tags:java' }
240
+ must_not { string 'tags:python' }
241
+ end
242
+ end
243
+ end
244
+ ```
245
+
246
+ The best thing about `boolean` queries is that we can easily save these partial queries as Ruby blocks,
247
+ to mix and reuse them later. So, we may define a query for the _tags_ property:
248
+
249
+ ```ruby
250
+ tags_query = lambda do
251
+ boolean.should { string 'tags:ruby' }
252
+ boolean.should { string 'tags:java' }
253
+ end
254
+ ```
255
+
256
+ And a query for the _published_on_ property:
257
+
258
+ ```ruby
259
+ published_on_query = lambda do
260
+ boolean.must { string 'published_on:[2011-01-01 TO 2011-01-02]' }
261
+ end
262
+ ```
263
+
264
+ Now, we can combine these queries for different searches:
265
+
266
+ ```ruby
267
+ Tire.search 'articles' do
268
+ query do
269
+ boolean &tags_query
270
+ boolean &published_on_query
271
+ end
272
+ end
273
+ ```
274
+
275
+ Note, that you can pass options for configuring queries, facets, etc. by passing a Hash as the last argument to the method call:
276
+
277
+ ```ruby
278
+ Tire.search 'articles' do
279
+ query do
280
+ string 'ruby python', :default_operator => 'AND', :use_dis_max => true
281
+ end
282
+ end
283
+ ```
284
+
285
+ You don't have to define the search criteria in one monolithic _Ruby_ block -- you can build the search step by step,
286
+ until you call the `results` method:
287
+
288
+ ```ruby
289
+ s = Tire.search('articles') { query { string 'title:T*' } }
290
+ s.filter :terms, :tags => ['ruby']
291
+ p s.results
292
+ ```
293
+
294
+ If configuring the search payload with blocks feels somehow too weak for you, you can pass
295
+ a plain old Ruby `Hash` (or JSON string) with the query declaration to the `search` method:
296
+
297
+ ```ruby
298
+ Tire.search 'articles', :query => { :prefix => { :title => 'fou' } }
299
+ ```
300
+
301
+ If this sounds like a great idea to you, you are probably able to write your application
302
+ using just `curl`, `sed` and `awk`.
303
+
304
+ Do note again, however, that you're not tied to the declarative block-style DSL _Tire_ offers to you.
305
+ If it makes more sense in your context, you can use the API directly, in a more imperative style:
306
+
307
+ ```ruby
308
+ search = Tire::Search::Search.new('articles')
309
+ search.query { string('title:T*') }
310
+ search.filter :terms, :tags => ['ruby']
311
+ search.sort { by :title, 'desc' }
312
+ search.facet('global-tags') { terms :tags, :global => true }
313
+ # ...
314
+ p search.results
315
+ ```
316
+
317
+ To debug the query we have laboriously set up like this,
318
+ we can display the full query JSON for close inspection:
319
+
320
+ ```ruby
321
+ puts s.to_json
322
+ # {"facets":{"current-tags":{"terms":{"field":"tags"}},"global-tags":{"global":true,"terms":{"field":"tags"}}},"query":{"query_string":{"query":"title:T*"}},"filter":{"terms":{"tags":["ruby"]}},"sort":[{"title":"desc"}]}
323
+ ```
324
+
325
+ Or, better, we can display the corresponding `curl` command to recreate and debug the request in the terminal:
326
+
327
+ ```ruby
328
+ puts s.to_curl
329
+ # curl -X POST "http://localhost:9200/articles/_search?pretty=true" -d '{"facets":{"current-tags":{"terms":{"field":"tags"}},"global-tags":{"global":true,"terms":{"field":"tags"}}},"query":{"query_string":{"query":"title:T*"}},"filter":{"terms":{"tags":["ruby"]}},"sort":[{"title":"desc"}]}'
330
+ ```
331
+
332
+ However, we can simply log every search query (and other requests) in this `curl`-friendly format:
333
+
334
+ ```ruby
335
+ Tire.configure { logger 'elasticsearch.log' }
336
+ ```
337
+
338
+ When you set the log level to _debug_:
339
+
340
+ ```ruby
341
+ Tire.configure { logger 'elasticsearch.log', :level => 'debug' }
342
+ ```
343
+
344
+ the JSON responses are logged as well. This is not a great idea for production environment,
345
+ but it's priceless when you want to paste a complicated transaction to the mailing list or IRC channel.
346
+
347
+ The _Tire_ DSL tries hard to provide a strong Ruby-like API for the main _ElasticSearch_ features.
348
+
349
+ By default, _Tire_ wraps the results collection in a enumerable `Results::Collection` class,
350
+ and result items in a `Results::Item` class, which looks like a child of `Hash` and `Openstruct`,
351
+ for smooth iterating over and displaying the results.
352
+
353
+ You may wrap the result items in your own class by setting the `Tire.configuration.wrapper`
354
+ property. Your class must take a `Hash` of attributes on initialization.
355
+
356
+ If that seems like a great idea to you, there's a big chance you already have such class.
357
+
358
+ One would bet it's an `ActiveRecord` or `ActiveModel` class, containing model of your Rails application.
359
+
360
+ Fortunately, _Tire_ makes blending _ElasticSearch_ features into your models trivially possible.
361
+
362
+
363
+ ActiveModel Integration
364
+ -----------------------
365
+
366
+ If you're the type with no time for lengthy introductions, you can generate a fully working
367
+ example Rails application, with an `ActiveRecord` model and a search form, to play with
368
+ (it even downloads _ElasticSearch_ itself, generates the application skeleton and leaves you with
369
+ a _Git_ repository to explore the steps and the code):
370
+
371
+ $ rails new searchapp -m https://raw.github.com/karmi/tire/master/examples/rails-application-template.rb
372
+
373
+ For the rest of us, let's suppose you have an `Article` class in your _Rails_ application.
374
+
375
+ To make it searchable with _Tire_, just `include` it:
376
+
377
+ ```ruby
378
+ class Article < ActiveRecord::Base
379
+ include Tire::Model::Search
380
+ include Tire::Model::Callbacks
381
+ end
382
+ ```
383
+
384
+ When you now save a record:
385
+
386
+ ```ruby
387
+ Article.create :title => "I Love ElasticSearch",
388
+ :content => "...",
389
+ :author => "Captain Nemo",
390
+ :published_on => Time.now
391
+ ```
392
+
393
+ it is automatically added into an index called 'articles', because of the included callbacks.
394
+
395
+ The document attributes are indexed exactly as when you call the `Article#to_json` method.
396
+
397
+ Now you can search the records:
398
+
399
+ ```ruby
400
+ Article.search 'love'
401
+ ```
402
+
403
+ OK. This is where the search game stops, often. Not here.
404
+
405
+ First of all, you may use the full query DSL, as explained above, with filters, sorting,
406
+ advanced facet aggregation, highlighting, etc:
407
+
408
+ ```ruby
409
+ Article.search do
410
+ query { string 'love' }
411
+ facet('timeline') { date :published_on, :interval => 'month' }
412
+ sort { by :published_on, 'desc' }
413
+ end
414
+ ```
415
+
416
+ Second, dynamic mapping is a godsend when you're prototyping.
417
+ For serious usage, though, you'll definitely want to define a custom _mapping_ for your models:
418
+
419
+ ```ruby
420
+ class Article < ActiveRecord::Base
421
+ include Tire::Model::Search
422
+ include Tire::Model::Callbacks
423
+
424
+ mapping do
425
+ indexes :id, :index => :not_analyzed
426
+ indexes :title, :analyzer => 'snowball', :boost => 100
427
+ indexes :content, :analyzer => 'snowball'
428
+ indexes :content_size, :as => 'content.size'
429
+ indexes :author, :analyzer => 'keyword'
430
+ indexes :published_on, :type => 'date', :include_in_all => false
431
+ end
432
+ end
433
+ ```
434
+
435
+ In this case, _only_ the defined model attributes are indexed. The `mapping` declaration creates the
436
+ index when the class is loaded or when the importing features are used, and _only_ when it does not yet exist.
437
+
438
+ You can define different [_analyzers_](http://www.elasticsearch.org/guide/reference/index-modules/analysis/index.html),
439
+ [_boost_](http://www.elasticsearch.org/guide/reference/mapping/boost-field.html) levels for different properties,
440
+ or any other configuration for _elasticsearch_.
441
+
442
+ You're not limited to 1:1 mapping between your model properties and the serialized document. With the `:as` option,
443
+ you can pass a string or a _Proc_ object which is evaluated in the instance context (see the `content_size` property).
444
+
445
+ Chances are, you want to declare also a custom _settings_ for the index, such as set the number of shards,
446
+ replicas, or create elaborate analyzer chains, such as the hipster's choice: [_ngrams_](https://gist.github.com/1160430).
447
+ In this case, just wrap the `mapping` method in a `settings` one, passing it the settings as a Hash:
448
+
449
+ ```ruby
450
+ class URL < ActiveRecord::Base
451
+ include Tire::Model::Search
452
+ include Tire::Model::Callbacks
453
+
454
+ settings :number_of_shards => 1,
455
+ :number_of_replicas => 1,
456
+ :analysis => {
457
+ :filter => {
458
+ :url_ngram => {
459
+ "type" => "nGram",
460
+ "max_gram" => 5,
461
+ "min_gram" => 3 }
462
+ },
463
+ :analyzer => {
464
+ :url_analyzer => {
465
+ "tokenizer" => "lowercase",
466
+ "filter" => ["stop", "url_ngram"],
467
+ "type" => "custom" }
468
+ }
469
+ } do
470
+ mapping { indexes :url, :type => 'string', :analyzer => "url_analyzer" }
471
+ end
472
+ end
473
+ ```
474
+
475
+ It may well be reasonable to wrap the index creation logic declared with `Tire.index('urls').create`
476
+ in a class method of your model, in a module method, etc, to have better control on index creation when
477
+ bootstrapping the application with Rake tasks or when setting up the test suite.
478
+ _Tire_ will not hold that against you.
479
+
480
+ You may have just stopped wondering: what if I have my own `settings` class method defined?
481
+ Or what if some other gem defines `settings`, or some other _Tire_ method, such as `update_index`?
482
+ Things will break, right? No, they won't.
483
+
484
+ In fact, all this time you've been using only _proxies_ to the real _Tire_ methods, which live in the `tire`
485
+ class and instance methods of your model. Only when not trampling on someone's foot — which is the majority
486
+ of cases —, will _Tire_ bring its methods to the namespace of your class.
487
+
488
+ So, instead of writing `Article.search`, you could write `Article.tire.search`, and instead of
489
+ `@article.update_index` you could write `@article.tire.update_index`, to be on the safe side.
490
+ Let's have a look on an example with the `mapping` method:
491
+
492
+ ```ruby
493
+ class Article < ActiveRecord::Base
494
+ include Tire::Model::Search
495
+ include Tire::Model::Callbacks
496
+
497
+ tire.mapping do
498
+ indexes :id, :type => 'string', :index => :not_analyzed
499
+ # ...
500
+ end
501
+ end
502
+ ```
503
+
504
+ Of course, you could also use the block form:
505
+
506
+ ```ruby
507
+ class Article < ActiveRecord::Base
508
+ include Tire::Model::Search
509
+ include Tire::Model::Callbacks
510
+
511
+ tire do
512
+ mapping do
513
+ indexes :id, :type => 'string', :index => :not_analyzed
514
+ # ...
515
+ end
516
+ end
517
+ end
518
+ ```
519
+
520
+ Internally, _Tire_ uses these proxy methods exclusively. When you run into issues,
521
+ use the proxied method, eg. `Article.tire.mapping`, directly.
522
+
523
+ When you want a tight grip on how the attributes are added to the index, just
524
+ implement the `to_indexed_json` method in your model.
525
+
526
+ The easiest way is to customize the `to_json` serialization support of your model:
527
+
528
+ ```ruby
529
+ class Article < ActiveRecord::Base
530
+ # ...
531
+
532
+ include_root_in_json = false
533
+ def to_indexed_json
534
+ to_json :except => ['updated_at'], :methods => ['length']
535
+ end
536
+ end
537
+ ```
538
+
539
+ Of course, it may well be reasonable to define the indexed JSON from the ground up:
540
+
541
+ ```ruby
542
+ class Article < ActiveRecord::Base
543
+ # ...
544
+
545
+ def to_indexed_json
546
+ names = author.split(/\W/)
547
+ last_name = names.pop
548
+ first_name = names.join
549
+
550
+ {
551
+ :title => title,
552
+ :content => content,
553
+ :author => {
554
+ :first_name => first_name,
555
+ :last_name => last_name
556
+ }
557
+ }.to_json
558
+ end
559
+ end
560
+ ```
561
+
562
+ Notice, that you may want to skip including the `Tire::Model::Callbacks` module in special cases,
563
+ like when your records are indexed via some external mechanism, let's say a _CouchDB_ or _RabbitMQ_
564
+ [river](http://www.elasticsearch.org/blog/2010/09/28/the_river.html), or when you need better
565
+ control on how the documents are added to or removed from the index:
566
+
567
+ ```ruby
568
+ class Article < ActiveRecord::Base
569
+ include Tire::Model::Search
570
+
571
+ after_save do
572
+ update_index if state == 'published'
573
+ end
574
+ end
575
+ ```
576
+
577
+ The results returned by `Article.search` are wrapped in the aforementioned `Item` class, by default.
578
+ This way, we have a fast and flexible access to the properties returned from _ElasticSearch_ (via the
579
+ `_source` or `fields` JSON properties). This way, we can index whatever JSON we like in _ElasticSearch_,
580
+ and retrieve it, simply, via the dot notation:
581
+
582
+ ```ruby
583
+ articles = Article.search 'love'
584
+ articles.each do |article|
585
+ puts article.title
586
+ puts article.author.last_name
587
+ end
588
+ ```
589
+
590
+ The `Item` instances masquerade themselves as instances of your model within a _Rails_ application
591
+ (based on the `_type` property retrieved from _ElasticSearch_), so you can use them carefree;
592
+ all the `url_for` or `dom_id` helpers work as expected.
593
+
594
+ If you need to access the “real” model (eg. to access its assocations or methods not
595
+ stored in _ElasticSearch_), just load it from the database:
596
+
597
+ ```ruby
598
+ puts article.load(:include => 'comments').comments.size
599
+ ```
600
+
601
+ You can see that _Tire_ stays as far from the database as possible. That's because it believes
602
+ you have most of the data you want to display stored in _ElasticSearch_. When you need
603
+ to eagerly load the records from the database itself, for whatever reason,
604
+ you can do it with the `:load` option when searching:
605
+
606
+ ```ruby
607
+ # Will call `Article.search [1, 2, 3]`
608
+ Article.search 'love', :load => true
609
+ ```
610
+
611
+ Instead of simple `true`, you can pass any options for the model's find method:
612
+
613
+ ```ruby
614
+ # Will call `Article.search [1, 2, 3], :include => 'comments'`
615
+ Article.search :load => { :include => 'comments' } do
616
+ query { string 'love' }
617
+ end
618
+ ```
619
+
620
+ Note that _Tire_ search results are fully compatible with [`will_paginate`](https://github.com/mislav/will_paginate),
621
+ so you can pass all the usual parameters to the `search` method in the controller:
622
+
623
+ ```ruby
624
+ @articles = Article.search params[:q], :page => (params[:page] || 1)
625
+ ```
626
+
627
+ OK. Chances are, you have lots of records stored in your database. How will you get them to _ElasticSearch_? Easy:
628
+
629
+ ```ruby
630
+ Article.index.import Article.all
631
+ ```
632
+
633
+ This way, however, all your records are loaded into memory, serialized into JSON,
634
+ and sent down the wire to _ElasticSearch_. Not practical, you say? You're right.
635
+
636
+ Provided your model implements some sort of _pagination_ — and it probably does —, you can just run:
637
+
638
+ ```ruby
639
+ Article.import
640
+ ```
641
+
642
+ In this case, the `Article.paginate` method is called, and your records are sent to the index
643
+ in chunks of 1000. If that number doesn't suit you, just provide a better one:
644
+
645
+ ```ruby
646
+ Article.import :per_page => 100
647
+ ```
648
+
649
+ Any other parameters you provide to the `import` method are passed down to the `paginate` method.
650
+
651
+ Are we saying you have to fiddle with this thing in a `rails console` or silly Ruby scripts? No.
652
+ Just call the included _Rake_ task on the commandline:
653
+
654
+ ```bash
655
+ $ rake environment tire:import CLASS='Article'
656
+ ```
657
+
658
+ You can also force-import the data by deleting the index first (and creating it with mapping
659
+ provided by the `mapping` block in your model):
660
+
661
+ ```bash
662
+ $ rake environment tire:import CLASS='Article' FORCE=true
663
+ ```
664
+
665
+ When you'll spend more time with _ElasticSearch_, you'll notice how
666
+ [index aliases](http://www.elasticsearch.org/guide/reference/api/admin-indices-aliases.html)
667
+ are the best idea since the invention of inverted index.
668
+ You can index your data into a fresh index (and possibly update an alias once everything's fine):
669
+
670
+ ```bash
671
+ $ rake environment tire:import CLASS='Article' INDEX='articles-2011-05'
672
+ ```
673
+
674
+ OK. All this time we have been talking about `ActiveRecord` models, since
675
+ it is a reasonable _Rails_' default for the storage layer.
676
+
677
+ But what if you use another database such as [MongoDB](http://www.mongodb.org/),
678
+ another object mapping library, such as [Mongoid](http://mongoid.org/)?
679
+
680
+ Well, things stay mostly the same:
681
+
682
+ ```ruby
683
+ class Article
684
+ include Mongoid::Document
685
+ field :title, :type => String
686
+ field :content, :type => String
687
+
688
+ include Tire::Model::Search
689
+ include Tire::Model::Callbacks
690
+
691
+ # These Mongo guys sure do get funky with their IDs in +serializable_hash+, let's fix it.
692
+ #
693
+ def to_indexed_json
694
+ self.as_json
695
+ end
696
+
697
+ end
698
+
699
+ Article.create :title => 'I Love ElasticSearch'
700
+
701
+ Article.tire.search 'love'
702
+ ```
703
+
704
+ _Tire_ does not care what's your primary data storage solution, if it has an _ActiveModel_-compatible
705
+ adapter. But there's more.
706
+
707
+ _Tire_ implements not only _searchable_ features, but also _persistence_ features. This means you can use a _Tire_ model **instead of your database**, not just for _searching_ your database. Why would you like to do that?
708
+
709
+ Well, because you're tired of database migrations and lots of hand-holding with your
710
+ database to store stuff like `{ :name => 'Tire', :tags => [ 'ruby', 'search' ] }`.
711
+ Because all you need, really, is to just dump a JSON-representation of your data into a database and load it back again.
712
+ Because you've noticed that _searching_ your data is a much more effective way of retrieval
713
+ then constructing elaborate database query conditions.
714
+ Because you have _lots_ of data and want to use _ElasticSearch's_ advanced distributed features.
715
+
716
+ All good reasons to use _ElasticSearch_ as a schema-free and highly-scalable storage and retrieval/aggregation engine for your data.
717
+
718
+ To use the persistence mode, we'll include the `Tire::Persistence` module in our class and define its properties;
719
+ we can add the standard mapping declarations, set default values, or define casting for the property to create
720
+ lightweight associations between the models.
721
+
722
+ ```ruby
723
+ class Article
724
+ include Tire::Model::Persistence
725
+
726
+ validates_presence_of :title, :author
727
+
728
+ property :title, :analyzer => 'snowball'
729
+ property :published_on, :type => 'date'
730
+ property :tags, :default => [], :analyzer => 'keyword'
731
+ property :author, :class => Author
732
+ property :comments, :class => [Comment]
733
+ end
734
+ ```
735
+
736
+ Please be sure to peruse the [integration test suite](https://github.com/karmi/tire/tree/master/test/integration)
737
+ for examples of the API and _ActiveModel_ integration usage.
738
+
739
+
740
+ Extensions and Additions
741
+ ------------------------
742
+
743
+ The [_tire-contrib_](http://github.com/karmi/tire-contrib/) project contains additions
744
+ and extensions to the core _Tire_ functionality — be sure to check them out.
745
+
746
+
747
+ Other Clients
748
+ -------------
749
+
750
+ Check out [other _ElasticSearch_ clients](http://www.elasticsearch.org/guide/appendix/clients.html).
751
+
752
+
753
+ Feedback
754
+ --------
755
+
756
+ You can send feedback via [e-mail](mailto:karmi@karmi.cz) or via [Github Issues](https://github.com/karmi/tire/issues).
757
+
758
+ -----
759
+
760
+ [Karel Minarik](http://karmi.cz) and [contributors](http://github.com/karmi/tire/contributors)