caoutsearch 0.0.5 → 0.0.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md CHANGED
@@ -1,13 +1,13 @@
1
1
  # Caoutsearch [\ˈkawt͡ˈsɝtʃ\\](http://ipa-reader.xyz/?text=ˈkawt͡ˈsɝtʃ)
2
2
 
3
- [![Gem Version](https://badge.fury.io/rb/caoutsearch.svg)](https://rubygems.org/gems/caoutsearch)
4
- [![CI Status](https://github.com/mon-territoire/caoutsearch/actions/workflows/ci.yml/badge.svg)](https://github.com/mon-territoire/caoutsearch/actions/workflows/ci.yml)
5
- [![Ruby Style Guide](https://img.shields.io/badge/code_style-standard-brightgreen.svg)](https://github.com/testdouble/standard)
6
- [![Maintainability](https://api.codeclimate.com/v1/badges/fbe73db3fd8be9a10e12/maintainability)](https://codeclimate.com/github/mon-territoire/caoutsearch/maintainability)
7
- [![Test Coverage](https://api.codeclimate.com/v1/badges/fbe73db3fd8be9a10e12/test_coverage)](https://codeclimate.com/github/mon-territoire/caoutsearch/test_coverage)
3
+ <span>[![Gem Version](https://badge.fury.io/rb/caoutsearch.svg)](https://rubygems.org/gems/caoutsearch)</span> <span>
4
+ [![CI Status](https://github.com/mon-territoire/caoutsearch/actions/workflows/ci.yml/badge.svg)](https://github.com/mon-territoire/caoutsearch/actions/workflows/ci.yml)</span> <span>
5
+ [![Ruby Style Guide](https://img.shields.io/badge/code_style-standard-brightgreen.svg)](https://github.com/testdouble/standard)</span> <span>
6
+ [![Maintainability](https://api.codeclimate.com/v1/badges/fbe73db3fd8be9a10e12/maintainability)](https://codeclimate.com/github/mon-territoire/caoutsearch/maintainability)</span> <span>
7
+ [![Test Coverage](https://api.codeclimate.com/v1/badges/fbe73db3fd8be9a10e12/test_coverage)](https://codeclimate.com/github/mon-territoire/caoutsearch/test_coverage)</span>
8
8
 
9
- [![JRuby](https://github.com/mon-territoire/caoutsearch/actions/workflows/jruby.yml/badge.svg)](https://github.com/mon-territoire/caoutsearch/actions/workflows/jruby.yml)
10
- [![Truffle Ruby](https://github.com/mon-territoire/caoutsearch/actions/workflows/truffle_ruby.yml/badge.svg)](https://github.com/mon-territoire/caoutsearch/actions/workflows/truffle_ruby.yml)
9
+ <span>[![JRuby](https://github.com/mon-territoire/caoutsearch/actions/workflows/jruby.yml/badge.svg)](https://github.com/mon-territoire/caoutsearch/actions/workflows/jruby.yml)</span> <span>
10
+ [![Truffle Ruby](https://github.com/mon-territoire/caoutsearch/actions/workflows/truffle_ruby.yml/badge.svg)](https://github.com/mon-territoire/caoutsearch/actions/workflows/truffle_ruby.yml)</span>
11
11
 
12
12
  **!! Gem under development before public release !!**
13
13
 
@@ -17,177 +17,45 @@ It provides a simple but powerful DSL to perform complex indexing and searching,
17
17
  Caoutsearch only supports Elasticsearch 8.x right now.
18
18
  It is used in production in a robust application, updated and maintained for several years at [Mon Territoire](https://mon-territoire.fr).
19
19
 
20
- Caoutsearch was inspired by awesome gems such as [elasticsearch-rails](https://github.com/elastic/elasticsearch-rails) or [search_flip](https://github.com/mrkamel/search_flip).
21
- If you don't have scenarios as complex as those described in this documentation, they should better suite your needs.
20
+ Caoutsearch was inspired by awesome gems such as [elasticsearch-rails](https://github.com/elastic/elasticsearch-rails) or [search_flip](https://github.com/mrkamel/search_flip).
21
+ Depending on your search scenarios, they may better suite your needs.
22
22
 
23
- ## Table of Contents
23
+ ## Documentation
24
24
 
25
- - [Installation](#installation)
26
- - [Configuration](#configuration)
27
- - Instrumentation
28
- - [Usage](#usage)
29
- - [Indice Configuration](#indice-configuration)
30
- - Mapping & settings
31
- - Text analysis
32
- - Versionning
33
- - [Index Engine](#index-engine)
34
- - Properties
35
- - Partial updates
36
- - Eager loading
37
- - Interdependencies
38
- - [Search Engine](#search-engine)
39
- - Queries
40
- - [Filters](#filters)
41
- - Full-text query
42
- - Custom filters
43
- - Orders
44
- - [Aggregations](#aggregations)
45
- - [Transform aggregations](#transform-aggregations)
46
- - [Responses](#responses)
47
- - [Loading records](#loading-records)
48
- - [Model integration](#model-integration)
49
- - [Add Caoutsearch to your models](#add-caoutsearch-to-your-models)
50
- - [Index records](#index-records)
51
- - [Index multiple records](#index-multiple-records)
52
- - [Index single records](#index-single-records)
53
- - [Delete documents](#delete-documents)
54
- - [Automatic Callbacks](#automatic-callbacks)
55
- - Asynchronous methods
56
- - [Search for records](#search-for-records)
57
- - [Search API](#search-api)
58
- - [Pagination](#pagination)
59
- - [Total count](#total-count)
60
- - [Iterating results](#iterating-results)
61
- - [Testing with Caoutsearch](#testing-with-Caoutsearch)
25
+ Visit our [offical documentation](https://mon-territoire.github.io/caoutsearch) to understand how to use Caoutsearch.
62
26
 
63
27
  ## Installation
64
28
 
29
+ Add the gem in your Gemfile:
30
+
65
31
  ```bash
66
32
  bundle add caoutsearch
67
33
  ```
68
34
 
69
- ## Configuration
70
-
71
- TODO
72
-
73
- ## Usage
74
-
75
- ### Indice Configuration
76
-
77
- TODO
35
+ ## Overview
78
36
 
79
- ### Index Engine
37
+ Caoutsearch let you create `Index` and `Search` classes to manipulate your data :
80
38
 
81
- TODO
82
-
83
- ### Search Engine
84
-
85
- #### Filters
86
- Filters declared in the search engine will define how Caoutsearch will build the queries
87
-
88
- The main use of filters is to expose a field for search, but they can also be used to build more complex queries:
89
39
  ```ruby
90
- class ArticleSearch < Caoutsearch::Search::Base
91
- # Build a filter on the author field
92
- filter :author
40
+ class ArticleIndex < Caoutsearch::Index::Base
41
+ property :title
42
+ property :published_on
43
+ property :tags
93
44
 
94
- # Build a Match filter on multiple fields
95
- filter :content, indexes: %i[title.words content], as: :match
96
-
97
- # Build a more complex filter by using other filters
98
- filter :public, as: :boolean
99
- filter :published_on, as: :date
100
- filter :active do |value|
101
- search_by(published: value, published_on: value)
45
+ def tags
46
+ records.tags.public.map(&:to_s)
102
47
  end
103
48
  end
104
- ```
105
-
106
- Caoutsearch different types of filters to handle different types of data or ways to search them:
107
-
108
- ##### Default filter
109
-
110
- ##### Boolean filter
111
49
 
112
- ##### Date filter
50
+ ArticleIndex.reindex(:tags)
51
+ ```
113
52
 
114
- For a date filter defined like this:
115
53
  ```ruby
116
54
  class ArticleSearch < Caoutsearch::Search::Base
117
- ...
118
-
55
+ filter :title, as: :match
119
56
  filter :published_on, as: :date
120
- end
121
- ```
122
-
123
- You can now search the matching index with the `published_on` criterion:
124
- ```ruby
125
- Article.search(published_on: Date.today)
126
- ```
127
-
128
- and the following query will be generated to send to elasticsearch:
129
- ```json
130
- {
131
- "query": {
132
- "bool": {
133
- "filter": [
134
- { "range": { "published_on": { "gte": "2022-23-11", "lte": "2022-23-11"}}}
135
- ]
136
- }
137
- }
138
- }
139
- ```
140
-
141
- The date filter accepts multiple types of arguments :
142
-
143
- ```ruby
144
- # Search for articles published on a date:
145
- Article.search(published_on: Date.today)
146
-
147
- # Search for articles published before a date:
148
- Article.search(published_on: { less_than: "2022-12-25" })
149
- Article.search(published_on: { less_than_or_equal: "2022-12-25" })
150
- Article.search(published_on: ..Date.new(2022, 12, 25))
151
- Article.search(published_on: [[nil, "now-2w/d"]])
152
-
153
- # Search for articles published after a date:
154
- Article.search(published_on: { greater_than: "2022-12-25" })
155
- Article.search(published_on: { greater_than_or_equal: "2022-12-25" })
156
- Article.search(published_on: Date.new(2022, 12, 25)..)
157
- Article.search(published_on: [["now-1w/d", nil]])
158
-
159
- # Search for articles published between two dates:
160
- Article.search(published_on: { greater_than: "2022-12-25", less_than: "2023-12-25" })
161
- Article.search(published_on: Date.new(2022, 12, 25)..Date.new(2023, 12, 25))
162
- Article.search(published_on: [["now-1w/d", "now/d"]])
163
- ```
164
-
165
- Dates of various formats are handled:
166
- ```ruby
167
- "2022-10-11"
168
- Date.today
169
- Time.zone.now
170
- ```
57
+ filter :tags
171
58
 
172
- We also support elasticsearch's date math
173
- ```ruby
174
- "now-1h"
175
- "now+2w/d"
176
- ```
177
-
178
- ##### GeoPoint filter
179
-
180
- ##### Match filter
181
-
182
- ##### Range filter
183
-
184
- #### Aggregations
185
-
186
- You can define simple to complex aggregations.
187
-
188
- ````ruby
189
- class ArticleSearch < Caoutsearch::Search::Base
190
- has_aggregation :view_count, { sum: { field: :view_count } }
191
59
  has_aggregation :popular_tags, {
192
60
  filter: { term: { published: true } },
193
61
  aggs: {
@@ -197,642 +65,14 @@ class ArticleSearch < Caoutsearch::Search::Base
197
65
  }
198
66
  }
199
67
  end
200
- ````
201
-
202
- Then you can request one or more aggregations at the same time or chain the `aggregate` method.
203
- The `aggregations` method will trigger a request and returns a [Response::Aggregations](#responses).
204
-
205
- ````ruby
206
- ArticleSearch.aggregate(:view_count).aggregations
207
- # ArticleSearch Search { "body": { "aggs": { "view_count": { "sum": { "field": "view_count" }}}}}
208
- # ArticleSearch Search (10ms / took 5ms)
209
- => #<Caoutsearch::Response::Aggregations view_count=#<Caoutsearch::Response::Response value=119652>>
210
-
211
- ArticleSearch.aggregate(:view_count, :popular_tags).aggregations
212
- # ArticleSearch Search { "body": { "aggs": { "view_count": {…}, "popular_tags": {…}}}}
213
- # ArticleSearch Search (10ms / took 5ms)
214
- => #<Caoutsearch::Response::Aggregations view_count=#<Caoutsearch::Response::Response value=119652> popular_tags=#<Caoutsearch::Response::Response buckets=…>>
215
-
216
- ArticleSearch.aggregate(:view_count).aggregate(:popular_tags).aggregations
217
- # ArticleSearch Search { "body": { "aggs": { "view_count": {…}, "popular_tags": {…}}}}
218
- # ArticleSearch Search (10ms / took 5ms)
219
- => #<Caoutsearch::Response::Aggregations view_count=#<Caoutsearch::Response::Response value=119652> popular_tags=#<Caoutsearch::Response::Response buckets=…>>
220
- ````
221
-
222
- You can create powerful aggregations using blocks and pass arguments to them.
223
-
224
- ````ruby
225
- class ArticleSearch < Caoutsearch::Search::Base
226
- has_aggregation :popular_tags_since do |date|
227
- raise TypeError unless date.is_a?(Date)
228
-
229
- query.aggregations[:popular_tags_since] = {
230
- filter: { range: { publication_date: { gte: date.to_s } } },
231
- aggs: {
232
- published: {
233
- terms: { field: :tags, size: 20 }
234
- }
235
- }
236
- }
237
- end
238
- end
239
-
240
- ArticleSearch.aggregate(popular_tags_since: 1.day.ago).aggregations
241
- # ArticleSearch Search { "body": { "aggs": { "popular_tags_since": {…}}}}
242
- # ArticleSearch Search (10ms / took 5ms)
243
- => #<Caoutsearch::Response::Aggregations popular_tags_since=#<Caoutsearch::Response::Response …
244
- ````
245
-
246
- Only one argument can be passed to an aggregation block.
247
- Use an Array or a Hash if you need to pass multiple options.
248
-
249
- ````ruby
250
- class ArticleSearch < Caoutsearch::Search::Base
251
- has_aggregation :popular_tags_since do |options|
252
- # …
253
- end
254
-
255
- has_aggregation :popular_tags_between do |(first_date, end_date)|
256
- # …
257
- end
258
- end
259
-
260
- ArticleSearch.aggregate(popular_tags_since: { date: 1.day.ago, size: 20 })
261
- ArticleSearch.aggregate(popular_tags_between: [date1, date2])
262
- ````
263
-
264
- Finally, you can create a "catch-all" aggregation to handle cumbersome behaviors:
265
-
266
- ````ruby
267
- class ArticleSearch < Caoutsearch::Search::Base
268
- has_aggregation do |name, options = {}|
269
- raise "unxpected_error" unless name.match?(/^view_count_(?<year>\d{4})$/)
270
-
271
- query.aggregations[name] = {
272
- filter: { term: { year: $LAST_LATCH_INFO[:year] } },
273
- aggs: {
274
- filtered: {
275
- sum: { field: :view_count }
276
- }
277
- }
278
- }
279
- end
280
- end
281
-
282
- ArticleSearch.aggregate(:view_count_2020, :view_count_2019).aggregations
283
- # ArticleSearch Search { "body": { "aggs": { "view_count_2020": {…}, "view_count_2019": {…}}}}
284
- # ArticleSearch Search (10ms / took 5ms)
285
- => #<Caoutsearch::Response::Aggregations view_count_2020=#<Caoutsearch::Response::Response …
286
- ````
287
-
288
- #### Transform aggregations
289
-
290
- When using [buckets aggregation](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket.html) and/or [pipeline aggregation](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-pipeline.html), the path to the expected values can get complicated and become subject to unexpected changes for a public API.
291
-
292
- ````ruby
293
- ArticleSearch.aggregate(popular_tags_since: 1.month.ago).aggregations.popular_tags_since.published.buckets.pluck(:key)
294
- => ["Blog", "Tech", …]
295
- ````
296
-
297
- Instead, you can define transformations to provide simpler access to aggregated data:
298
-
299
- ````ruby
300
- class ArticleSearch < Caoutsearch::Search::Base
301
- has_aggregation :popular_tags_since do |since|
302
- # …
303
- end
304
-
305
- transform_aggregation :popular_tags_since do |aggs|
306
- aggs.dig(:popular_tags_since, :published, :buckets).pluck(:key)
307
- end
308
- end
309
-
310
- ArticleSearch.aggregate(popular_tags_since: 1.month.ago).aggregations.popular_tags_since
311
- => ["Blog", "Tech", …]
312
- ````
313
-
314
- You can also use transformations to combine multiple aggregations:
315
-
316
- ````ruby
317
- class ArticleSearch < Caoutsearch::Search::Base
318
- has_aggregation :blog_count, { filter: { term: { category: "blog" } } }
319
- has_aggregation :archives_count, { filter: { term: { archived: true } } }
320
-
321
- transform_aggregation :stats, from: %i[blog_count archives_count] do |aggs|
322
- {
323
- blog_count: aggs.dig(:blog_count, :doc_count),
324
- archives_count: aggs.dig(:archives, :doc_count)
325
- }
326
- end
327
- end
328
-
329
- ArticleSearch.aggregate(:stats).aggregations.stats
330
- # ArticleSearch Search { "body": { "aggs": { "blog_count": {…}, "archives_count": {…}}}}
331
- # ArticleSearch Search (10ms / took 5ms)
332
- => { blog_count: 124, archives_count: 2452 }
333
- ````
334
-
335
- This is also usefull to unify the API between different search engines:
336
-
337
- ````ruby
338
- class ArticleSearch < Caoutsearch::Search::Base
339
- has_aggregation :popular_tags, {
340
- filter: { term: { published: true } },
341
- aggs: { published: { terms: { field: :tags, size: 10 } } }
342
- }
343
-
344
- transform_aggregation :popular_tags do |aggs|
345
- aggs.dig(:popular_tags, :published, :buckets).pluck(:key)
346
- end
347
- end
348
-
349
- class TagSearch < Caoutsearch::Search::Base
350
- has_aggregation :popular_tags, {
351
- terms: { field: "label", size: 20, order: { used_count: "desc" } }
352
- }
353
-
354
- transform_aggregation :popular_tags do |aggs|
355
- aggs.dig(:popular_tags, :buckets).pluck(:key)
356
- end
357
- end
358
-
359
- ArticleSearch.aggregate(:popular_tags).aggregations.popular_tags
360
- => ["Blog", "Tech", …]
361
-
362
- TagSearch.aggregate(:popular_tags).aggregations.popular_tags
363
- => ["Tech", "Blog", …]
364
- ````
365
-
366
- Transformations are performed on demand and result is memorized. That means:
367
- - the result of transformation is not visible in the [Response::Aggregations](#responses) output.
368
- - the block is called only once for the same search instance.
369
-
370
- ````ruby
371
- class ArticleSearch < Caoutsearch::Search::Base
372
- has_aggregation :popular_tags, {…}
373
-
374
- transform_aggregation :popular_tags do |aggs|
375
- tags = aggs.dig(:popular_tags, :published, :buckets).pluck(:key)
376
- authorized = Tag.where(title: tags, authorize: true).pluck(:title)
377
- tags & authorized
378
- end
379
- end
380
-
381
- article_search = ArticleSearch.aggregate(:popular_tags)
382
- => #<ArticleSearch current_aggregations: [:popular_tags]>
383
-
384
- article_search.aggregations
385
- # ArticleSearch Search (10ms / took 5ms)
386
- => #<Caoutsearch::Response::Aggregations popular_tags=#<Caoutsearch::Response::Response doc_count=100 …
387
-
388
- article_search.aggregations.popular_tags
389
- # (10.2ms) SELECT "tags"."title" FROM "tags" WHERE "tags"."title" IN …
390
- => ["Blog", "Tech", …]
391
-
392
- article_search.aggregations.popular_tags
393
- => ["Blog", "Tech", …]
394
-
395
- article_search.search("Tech").aggregations.popular_tags
396
- # ArticleSearch Search (10ms / took 5ms)
397
- # (10.2ms) SELECT "tags"."title" FROM "tags" WHERE "tags"."title" IN …
398
- => ["Blog", "Tech", …]
399
- ````
400
-
401
- Be careful to avoid using `aggregations.<aggregation_name>` inside a transformation block: it can lead to an infinite recursion.
402
-
403
- ````ruby
404
- class ArticleSearch < Caoutsearch::Search::Base
405
- transform_aggregation :popular_tags do
406
- aggregations.popular_tags.buckets.pluck("key")
407
- end
408
- end
409
-
410
- ArticleSearch.aggregate(:popular_tags).aggregations.popular_tags
411
- Traceback (most recent call last):
412
- 4: from app/searches/article_search.rb:3:in `block in <class:ArticleSearch>'
413
- 3: from app/searches/article_search.rb:3:in `block in <class:ArticleSearch>'
414
- 2: from app/searches/article_search.rb:3:in `block in <class:ArticleSearch>'
415
- 1: from app/searches/article_search.rb:3:in `block in <class:ArticleSearch>'
416
- SystemStackError (stack level too deep)
417
- ````
418
-
419
- Instead, use the argument passed to the block: it's is a shortcut for `response.aggregations` which is a [Response::Reponse](#responses) and not a [Response::Aggregations](#responses).
420
-
421
- ````ruby
422
- class ArticleSearch < Caoutsearch::Search::Base
423
- transform_aggregation :popular_tags do |aggs|
424
- aggs.popular_tags.buckets.pluck("key")
425
- end
426
- end
427
-
428
- ArticleSearch.aggregate(:popular_tags).aggregations.popular_tags
429
- => ["Blog", "Tech", …]
430
- ````
431
-
432
- One last helpful argument is `track_total_hits` which allows to perform calculations over aggregations using the `total_count` method without sending a second request.
433
- Take a look at [Total count](#total-count) to understand why a second request could be performed.
434
-
435
- ````ruby
436
- class ArticleSearch < Caoutsearch::Search::Base
437
- aggregation :tagged, filter: { exist: "tag" }
438
-
439
- transform_aggregation :tagged_rate, from: :tagged, track_total_hits: true do |aggs|
440
- count = aggs.dig(:tagged, :doc_count)
441
- count.to_f / total_count
442
- end
443
-
444
- transform_aggregation :tagged_rate_without_track_total_hits, from: :tagged do |aggs|
445
- count = aggs.dig(:tagged, :doc_count)
446
- count.to_f / total_count
447
- end
448
- end
449
-
450
- ArticleSearch.aggregate(:tagged_rate).aggregations.tagged_rate
451
- # ArticleSearch Search { "body": { "track_total_hits": true, "aggs": { "blog_count": {…}, "archives_count": {…}}}}
452
- # ArticleSearch Search (10ms / took 5ms)
453
- => 0.95
454
-
455
- ArticleSearch.aggregate(:tagged_rate_without_track_total_hits).aggregations.tagged_rate
456
- # ArticleSearch Search { "body": { "aggs": { "blog_count": {…}, "archives_count": {…}}}}
457
- # ArticleSearch Search (10ms / took 5ms)
458
- # ArticleSearch Search { "body": { "track_total_hits": true, "aggs": { "blog_count": {…}, "archives_count":
459
- # ArticleSearch Search (10ms / took 5ms)
460
- => 0.95
461
- ````
462
-
463
- #### Responses
464
-
465
- After the request has been sent by calling a method such as `load`, `response` or `hits`, the results is wrapped in a `Response::Response` class which provides method access to its properties via [Hashie::Mash](http://github.com/intridea/hashie).
466
-
467
- Aggregations and suggestions are wrapped in their own respective subclass of `Response::Response`
468
-
469
- ````ruby
470
- results.response
471
- => #<Caoutsearch::Response::Response _shards=#<Caoutsearch::Response::Response failed=0 skipped=0 successful=5 total=5> hits=…
472
-
473
- search.hits
474
- => #<Hashie::Array [#<Caoutsearch::Response::Response _id="2"…
475
-
476
- search.aggregations
477
- => #<Caoutsearch::Response::Aggregations view_count=#<Caoutsearch::Response::Response…
478
-
479
- search.suggestions
480
- => #<Caoutsearch::Response::Suggestions tags=#<Caoutsearch::Response::Response…
481
- ````
482
-
483
- ##### Loading records
484
-
485
- When calling `records`, the search engine will try to load records from a model using the same class name without `Search` the suffix:
486
- * `ArticleSearch` > `Article`
487
- * `Blog::ArticleSearch` > `Blog::Article`
488
-
489
- ````ruby
490
- ArticleSearch.new.records.first
491
- # ArticleSearch Search (10ms / took 5ms)
492
- # Article Load (9.6ms) SELECT "articles".* FROM "articles" WHERE "articles"."id" IN (1, …
493
- => #<Article id: 1, …>
494
- ````
495
-
496
- However, you can define an alternative model to load records. This might be helpful when using [single table inheritance](https://api.rubyonrails.org/classes/ActiveRecord/Inheritance.html).
497
-
498
- ````ruby
499
- ArticleSearch.new.records(use: BlogArticle).first
500
- # ArticleSearch Search (10ms / took 5ms)
501
- # BlogArticle Load (9.6ms) SELECT "articles".* FROM "articles" WHERE "articles"."id" IN (1, …
502
- => #<BlogArticle id: 1, …>
503
- ````
504
-
505
- You can also define an alternative model at class level:
506
-
507
- ````ruby
508
- class BlogArticleSearch < Caoutsearch::Search::Base
509
- self.model_name = "Article"
510
68
 
511
- default do
512
- query.filters << { term: { category: "blog" } }
513
- end
514
- end
515
-
516
- BlogArticleSearch.new.records.first
517
- # BlogArticleSearch Search (10ms / took 5ms)
518
- # Article Load (9.6ms) SELECT "articles".* FROM "articles" WHERE "articles"."id" IN (1, …
519
- => #<Article id: 1, …>
520
- ````
521
-
522
- ### Model integration
523
-
524
- #### Add Caoutsearch to your models
525
-
526
- The simplest solution is to add `Caoutsearch::Model` to your model and the link the appropriate `Index` and/or `Search` engines:
527
-
528
- ```ruby
529
- class Article < ActiveRecord::Base
530
- include Caoutsearch::Model
531
-
532
- index_with ArticleIndex
533
- search_with ArticleSearch
534
- end
535
- ```
536
-
537
- If you don't need your models to be `Indexable` and `Searchable`, you can include only one of the following two modules:
538
-
539
- ````ruby
540
- class Article < ActiveRecord::Base
541
- include Caoutsearch::Model::Indexable
542
-
543
- index_with ArticleIndex
544
- end
545
- ````
546
- or
547
- ````ruby
548
- class Article < ActiveRecord::Base
549
- include Caoutsearch::Model::Searchable
550
-
551
- search_with ArticleSearch
552
- end
553
- ````
554
-
555
- The modules can be safely included in the meta model `ApplicationRecord`.
556
- Indexing & searching features are not available until you call `index_with` or `search_with`:
557
-
558
- ````ruby
559
- class ApplicationRecord < ActiveRecord::Base
560
- include Caoutsearch::Model
561
- end
562
- ````
563
-
564
- #### Index records
565
-
566
- ##### Index multiple records
567
-
568
- Import all your records or a restricted scope of records to Elastcisearch.
569
-
570
- ````ruby
571
- Article.reindex
572
- Article.where(published: true).reindex
573
- ````
574
-
575
- You can update one or more properties. (see [Indexation Engines](#indexation-engines) to read more about properties):
576
-
577
- ````ruby
578
- Article.reindex(:category)
579
- Article.reindex(%i[category published_on])
580
- ````
581
-
582
- When `reindex` is called without properties, it'll import the full document to ES.
583
- On the contrary, when properties are passed, it'll only update existing documents.
584
- You can control this behavior with the `method` argument.
585
-
586
- ````ruby
587
- Article.where(id: 123).reindex(:category)
588
- # ArticleIndex Reindex {"index":"articles","body":[{"update":{"_id":123}},{"doc":{"category":"blog"}}]}
589
- # [Error] {"update"=>{"_index"=>"articles", "_id"=>"123", "status"=>404, "error"=>{"type"=>"document_missing_exception", …}}
590
-
591
- Article.where(id: 123).reindex(:category, method: :index)
592
- # ArticleIndex Reindex {"index":"articles","body":[{"index":{"_id":123}},{"category":"blog"}]}
593
-
594
- Article.where(id: 123).reindex(method: :update)
595
- # ArticleIndex Reindex {"index":"articles","body":[{"update":{"_id":123}},{"doc":{…}}]}
596
- ````
597
-
598
- ##### Index single records
599
-
600
- Import a single record.
601
-
602
- ````ruby
603
- Article.find(123).update_index
604
- ````
605
-
606
- You can update one or more properties. (see [Indexation Engines](#indexation-engines) to read more about properties):
607
-
608
- ````ruby
609
- Article.find(123).update_index(:category)
610
- Article.find(123).update_index(%i[category published_on])
611
- ````
612
-
613
- You can verify if and how documents are indexed.
614
- If the document is missing in ES, it'll raise a `Elastic::Transport::Transport::Errors::NotFound`.
615
-
616
- ````ruby
617
- Article.find(123).indexed_document
618
- # Traceback (most recent call last):
619
- # 1: from (irb):1
620
- # Elastic::Transport::Transport::Errors::NotFound ([404] {"_index":"articles","_id":"123","found":false})
621
-
622
- Article.find(123).update_index
623
- Article.find(123).indexed_document
624
- => {"_index"=>"articles", "_id"=>"123", "_version"=>1"found"=>true, "_source"=>{…}}
625
- ````
626
-
627
- ##### Delete documents
628
-
629
- You can delete one or more documents.
630
- **Note**: it won't delete records from database, only from the ES indice.
631
-
632
- ````ruby
633
- Article.delete_indexes
634
- Article.where(id: 123).delete_indexed_documents
635
- Article.find(123).delete_index
636
- ````
637
-
638
- If a record is already deleted from the database, you can still delete its document.
639
-
640
- ````ruby
641
- Article.delete_index(123)
642
- ````
643
-
644
- ##### Automatic Callbacks
645
-
646
- Callbacks are not provided by Caoutsearch but they are very easy to add:
647
-
648
- ````ruby
649
- class Article < ApplicationRecord
650
- index_with ArticleIndex
651
-
652
- after_commit :update_index, on: %i[create update]
653
- after_commit :delete_index, on: %i[destroy]
654
- end
655
- ````
656
-
657
- ##### Asynchronous methods
658
-
659
- TODO
660
-
661
- #### Search for records
662
-
663
- ##### Search API
664
- Searching is pretty simple.
665
-
666
- ````ruby
667
- Article.search("Quick brown fox")
668
- => #<ArticleSearch current_criteria: ["Quick brown fox"]>
669
- ````
670
-
671
- You can chain criteria and many other parameters:
672
- ````ruby
673
- Article.search("Quick brown fox").search(published: true)
674
- => #<ArticleSearch current_criteria: ["Quick brown fox", {"published"=>true}]>
675
-
676
- Article.search("Quick brown fox").order(:publication_date)
677
- => #<ArticleSearch current_criteria: ["Quick brown fox"], current_order: :publication_date>
678
-
679
- Article.search("Quick brown fox").limit(100).offset(100)
680
- => #<ArticleSearch current_criteria: ["Quick brown fox"], current_limit: 100, current_offset: 100>
681
-
682
- Article.search("Quick brown fox").page(1).per(100)
683
- => #<ArticleSearch current_criteria: ["Quick brown fox"], current_page: 1, current_limit: 100>
684
-
685
- Article.search("Quick brown fox").aggregate(:tags).aggregate(:dates)
686
- => #<ArticleSearch current_criteria: ["Quick brown fox"], current_aggregations: [:tags, :dates]>>
687
- ````
688
-
689
- ##### Pagination
690
-
691
- Search results can be paginated.
692
- ````ruby
693
- search = Article.search("Quick brown fox").page(1).per(100)
694
- search.current_page
695
- => 1
696
-
697
- search.total_pages
698
- => 2546
699
-
700
- > search.total_count
701
- => 254514
702
- ````
703
-
704
- ##### Total count
705
-
706
- By default [ES doesn't return the total number of hits](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-your-data.html#track-total-hits). So, when calling `total_count` or `total_pages` a second request might be sent to ES.
707
- To avoid a second roundtrip, use `track_total_hits`:
708
-
709
- ````ruby
710
- search = Article.search("Quick brown fox")
711
- search.hits
712
- # ArticleSearch Search {…}
713
- # ArticleSearch Search (81.8ms / took 16ms)
714
- => […]
715
-
716
- search.total_count
717
- # ArticleSearch Search {…, track_total_hits: true }
718
- # ArticleSearch Search (135.3ms / took 76ms)
719
- => 276
720
-
721
- search = Article.search("Quick brown fox").track_total_hits
722
- search.hits
723
- # ArticleSearch Search {…, track_total_hits: true }
724
- # ArticleSearch Search (120.2ms / took 56ms)
725
- => […]
726
-
727
- search.total_count
728
- => 276
729
- ````
730
-
731
- ##### Iterating results
732
-
733
- Several methods are provided to loop through a collection or hits or records.
734
- These methods are processing batches in the most efficient way: [PIT search_after](https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html#search-after).
735
-
736
- * `find_each_hit` to yield each hit returned by Elasticsearch.
737
- * `find_each_record` to yield each record from your database.
738
- * `find_hits_in_batches` to yield each batch of hits as returned by Elasticsearch.
739
- * `find_records_in_batches` to yield each batch of records from the database.
740
-
741
- Example:
742
-
743
- ```ruby
744
- Article.search(published: true).find_each_record do |record|
745
- record.inspect
746
- end
747
- ```
748
-
749
- The `keep_alive` parameter tells Elasticsearch how long it should keep the point in time alive. Defaults to 1 minute.
750
-
751
- ```ruby
752
- Article.search(published: true).find_each_record(keep_alive: "2h")
753
- ```
754
-
755
- To specifies the size of the batch, use `per` chainable method or `batch_size` parameter. Defaults to 1000.
756
-
757
- ```ruby
758
- Article.search(published: true).find_records_in_batches(batch_size: 500)
759
- Article.search(published: true).per(500).find_records_in_batches
760
- ```
761
-
762
- ## Testing with Caoutsearch
763
-
764
- Caoutsearch offers few methods to stub Elasticsearch requests.
765
- You first need to add [webmock](https://github.com/bblimke/webmock) to your Gemfile.
766
-
767
- ```bash
768
- bundle add webmock
769
- ```
770
-
771
- Then, add `Caoutsearch::Testing::MockRequests` to your test suite.
772
- The examples below uses RSpec, but it should be compatible with other test framework.
773
-
774
- ```ruby
775
- # spec/spec_helper.rb
776
-
777
- require "caoutsearch/testing"
778
-
779
- RSpec.configure do |config|
780
- config.include Caoutsearch::Testing::MockRequests
781
- end
782
- ```
783
-
784
- You can then call the following methods:
785
-
786
- ```ruby
787
- RSpec.describe SomeClass do
788
- before do
789
- stub_elasticsearch_request(:head, "articles").to_return(status: 200)
790
-
791
- stub_elasticsearch_request(:get, "_cat/indices?format=json&h=index").to_return_json, [
792
- { index: "ca_locals_v14" }
793
- ])
794
-
795
- stub_elasticsearch_reindex_request("articles")
796
- stub_elasticsearch_search_request("articles", [
797
- {"_id" => "135", "_source" => {"name" => "Hello World"}},
798
- {"_id" => "137", "_source" => {"name" => "Hello World"}}
799
- ])
800
- end
801
-
802
- # ... do your tests...
803
- end
804
- ```
805
-
806
- `stub_elasticsearch_search_request` accepts an array or records:
807
-
808
- ```ruby
809
- RSpec.describe SomeClass do
810
- let(:articles) { create_list(:article, 5) }
811
-
812
- before do
813
- stub_elasticsearch_search_request("articles", articles)
814
- end
815
-
816
- # ... do your tests...
817
- end
818
- ```
819
-
820
- It allows to shim the total number of hits returned.
821
-
822
- ```ruby
823
- RSpec.describe SomeClass do
824
- before do
825
- stub_elasticsearch_search_request("articles", [], total: 250)
826
- end
827
-
828
- # ... do your tests...
829
- end
69
+ ArticleSearch.search(published_on: [["now-1y", nil]]).aggregate(:popular_tags)
830
70
  ```
831
71
 
832
72
  ## Contributing
833
73
 
834
- 1. Don't hesitate to submit your feature/idea/fix in [issues](https://github.com/mon-territoire/caoutsearch)
835
- 2. Fork the [repository](https://github.com/mon-territoire/caoutsearch)
74
+ 1. Don't hesitate to submit your feature/idea/fix in [issues](https://github.com/inkstak/activejob-status)
75
+ 2. Fork the [repository](https://github.com/inkstak/activejob-status)
836
76
  3. Create your feature branch
837
77
  4. Ensure RSpec & Rubocop are passing
838
78
  4. Create a pull request
@@ -845,6 +85,12 @@ bundle exec rubocop
845
85
  bundle exec standardrb
846
86
  ```
847
87
 
88
+ To run RSpec against various version of Rails dependencies:
89
+ ```bash
90
+ bundle exec appraisal install
91
+ bundle exec appraisal rspec
92
+ ```
93
+
848
94
  All of them can be run with:
849
95
 
850
96
  ```bash