chewy 7.0.0 → 7.2.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.github/workflows/ruby.yml +58 -0
- data/.rubocop.yml +13 -8
- data/.rubocop_todo.yml +110 -22
- data/CHANGELOG.md +103 -0
- data/Gemfile +0 -7
- data/Guardfile +3 -1
- data/README.md +322 -274
- data/chewy.gemspec +4 -6
- data/gemfiles/rails.5.2.activerecord.gemfile +8 -14
- data/gemfiles/rails.6.0.activerecord.gemfile +8 -14
- data/gemfiles/rails.6.1.activerecord.gemfile +8 -14
- data/lib/chewy.rb +21 -75
- data/lib/chewy/config.rb +40 -40
- data/lib/chewy/errors.rb +0 -12
- data/lib/chewy/fields/base.rb +11 -1
- data/lib/chewy/fields/root.rb +4 -6
- data/lib/chewy/index.rb +46 -98
- data/lib/chewy/index/actions.rb +52 -35
- data/lib/chewy/{type → index}/adapter/active_record.rb +12 -3
- data/lib/chewy/{type → index}/adapter/base.rb +2 -3
- data/lib/chewy/{type → index}/adapter/object.rb +27 -31
- data/lib/chewy/{type → index}/adapter/orm.rb +11 -14
- data/lib/chewy/index/aliases.rb +1 -3
- data/lib/chewy/{type → index}/crutch.rb +5 -5
- data/lib/chewy/{type → index}/import.rb +36 -27
- data/lib/chewy/{type → index}/import/bulk_builder.rb +15 -13
- data/lib/chewy/{type → index}/import/bulk_request.rb +6 -7
- data/lib/chewy/{type → index}/import/journal_builder.rb +11 -12
- data/lib/chewy/{type → index}/import/routine.rb +17 -16
- data/lib/chewy/{type → index}/mapping.rb +27 -32
- data/lib/chewy/{type → index}/observe.rb +9 -19
- data/lib/chewy/index/specification.rb +1 -0
- data/lib/chewy/{type → index}/syncer.rb +60 -60
- data/lib/chewy/{type → index}/witchcraft.rb +11 -7
- data/lib/chewy/{type → index}/wrapper.rb +2 -2
- data/lib/chewy/journal.rb +8 -8
- data/lib/chewy/minitest/helpers.rb +86 -13
- data/lib/chewy/minitest/search_index_receiver.rb +22 -26
- data/lib/chewy/railtie.rb +6 -20
- data/lib/chewy/rake_helper.rb +82 -107
- data/lib/chewy/rspec.rb +2 -0
- data/lib/chewy/rspec/build_query.rb +12 -0
- data/lib/chewy/rspec/helpers.rb +55 -0
- data/lib/chewy/rspec/update_index.rb +47 -43
- data/lib/chewy/search.rb +4 -17
- data/lib/chewy/search/loader.rb +19 -41
- data/lib/chewy/search/parameters.rb +4 -2
- data/lib/chewy/search/parameters/concerns/query_storage.rb +2 -2
- data/lib/chewy/search/parameters/indices.rb +12 -57
- data/lib/chewy/search/parameters/order.rb +6 -19
- data/lib/chewy/search/parameters/source.rb +5 -1
- data/lib/chewy/search/query_proxy.rb +9 -2
- data/lib/chewy/search/request.rb +96 -120
- data/lib/chewy/search/response.rb +5 -5
- data/lib/chewy/search/scoping.rb +6 -7
- data/lib/chewy/search/scrolling.rb +13 -13
- data/lib/chewy/stash.rb +15 -26
- data/lib/chewy/strategy.rb +3 -19
- data/lib/chewy/strategy/sidekiq.rb +1 -0
- data/lib/chewy/version.rb +1 -1
- data/lib/generators/chewy/install_generator.rb +1 -1
- data/lib/tasks/chewy.rake +20 -32
- data/migration_guide.md +26 -4
- data/spec/chewy/config_spec.rb +14 -39
- data/spec/chewy/fields/base_spec.rb +414 -150
- data/spec/chewy/fields/root_spec.rb +20 -28
- data/spec/chewy/fields/time_fields_spec.rb +5 -5
- data/spec/chewy/index/actions_spec.rb +270 -24
- data/spec/chewy/{type → index}/adapter/active_record_spec.rb +68 -40
- data/spec/chewy/{type → index}/adapter/object_spec.rb +21 -6
- data/spec/chewy/{type → index}/import/bulk_builder_spec.rb +23 -31
- data/spec/chewy/{type → index}/import/bulk_request_spec.rb +5 -12
- data/spec/chewy/{type → index}/import/journal_builder_spec.rb +9 -19
- data/spec/chewy/{type → index}/import/routine_spec.rb +16 -16
- data/spec/chewy/{type → index}/import_spec.rb +104 -100
- data/spec/chewy/index/mapping_spec.rb +135 -0
- data/spec/chewy/index/observe_spec.rb +116 -0
- data/spec/chewy/index/settings_spec.rb +3 -1
- data/spec/chewy/index/specification_spec.rb +20 -30
- data/spec/chewy/{type → index}/syncer_spec.rb +14 -19
- data/spec/chewy/{type → index}/witchcraft_spec.rb +20 -22
- data/spec/chewy/index/wrapper_spec.rb +100 -0
- data/spec/chewy/index_spec.rb +60 -105
- data/spec/chewy/journal_spec.rb +13 -26
- data/spec/chewy/minitest/helpers_spec.rb +122 -14
- data/spec/chewy/minitest/search_index_receiver_spec.rb +22 -26
- data/spec/chewy/multi_search_spec.rb +4 -5
- data/spec/chewy/rake_helper_spec.rb +145 -55
- data/spec/chewy/rspec/build_query_spec.rb +34 -0
- data/spec/chewy/rspec/helpers_spec.rb +61 -0
- data/spec/chewy/rspec/update_index_spec.rb +74 -71
- data/spec/chewy/runtime_spec.rb +1 -1
- data/spec/chewy/search/loader_spec.rb +19 -53
- data/spec/chewy/search/pagination/kaminari_examples.rb +3 -5
- data/spec/chewy/search/pagination/kaminari_spec.rb +1 -1
- data/spec/chewy/search/parameters/indices_spec.rb +26 -117
- data/spec/chewy/search/parameters/order_spec.rb +18 -11
- data/spec/chewy/search/parameters/query_storage_examples.rb +67 -21
- data/spec/chewy/search/parameters/search_after_spec.rb +4 -1
- data/spec/chewy/search/parameters/source_spec.rb +8 -2
- data/spec/chewy/search/parameters_spec.rb +13 -4
- data/spec/chewy/search/query_proxy_spec.rb +68 -17
- data/spec/chewy/search/request_spec.rb +235 -104
- data/spec/chewy/search/response_spec.rb +12 -12
- data/spec/chewy/search/scrolling_spec.rb +10 -18
- data/spec/chewy/search_spec.rb +32 -35
- data/spec/chewy/stash_spec.rb +9 -21
- data/spec/chewy/strategy/active_job_spec.rb +8 -8
- data/spec/chewy/strategy/atomic_spec.rb +9 -10
- data/spec/chewy/strategy/sidekiq_spec.rb +8 -8
- data/spec/chewy/strategy_spec.rb +19 -15
- data/spec/chewy_spec.rb +14 -100
- data/spec/spec_helper.rb +2 -21
- data/spec/support/active_record.rb +15 -5
- metadata +52 -108
- data/.circleci/config.yml +0 -240
- data/Appraisals +0 -81
- data/gemfiles/rails.5.2.mongoid.6.4.gemfile +0 -17
- data/gemfiles/sequel.4.45.gemfile +0 -11
- data/lib/chewy/search/pagination/will_paginate.rb +0 -43
- data/lib/chewy/search/parameters/types.rb +0 -20
- data/lib/chewy/strategy/resque.rb +0 -27
- data/lib/chewy/strategy/shoryuken.rb +0 -40
- data/lib/chewy/type.rb +0 -120
- data/lib/chewy/type/actions.rb +0 -43
- data/lib/chewy/type/adapter/mongoid.rb +0 -67
- data/lib/chewy/type/adapter/sequel.rb +0 -93
- data/lib/sequel/plugins/chewy_observe.rb +0 -63
- data/spec/chewy/search/pagination/will_paginate_examples.rb +0 -63
- data/spec/chewy/search/pagination/will_paginate_spec.rb +0 -23
- data/spec/chewy/search/parameters/types_spec.rb +0 -5
- data/spec/chewy/strategy/resque_spec.rb +0 -46
- data/spec/chewy/strategy/shoryuken_spec.rb +0 -70
- data/spec/chewy/type/actions_spec.rb +0 -50
- data/spec/chewy/type/adapter/mongoid_spec.rb +0 -372
- data/spec/chewy/type/adapter/sequel_spec.rb +0 -472
- data/spec/chewy/type/mapping_spec.rb +0 -175
- data/spec/chewy/type/observe_spec.rb +0 -137
- data/spec/chewy/type/wrapper_spec.rb +0 -100
- data/spec/chewy/type_spec.rb +0 -55
- data/spec/support/mongoid.rb +0 -93
- data/spec/support/sequel.rb +0 -80
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 229c412b1e3464060b367c9c378d3662bccf2329b4e4eaa7617541243dbd6e38
|
4
|
+
data.tar.gz: 9bc674550cd6152e8a18853fa46c91873c7a626c426cd50aa2cb8c67b1c9455e
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: e3e8409f81bc2504e9edbf5093fc9329bf443862cb96e113c9c00fc586c1e0e3a4273b3d6e82466f9b3d301e3d9eafcc72892cce718aff7dcfbbce85aba1698d
|
7
|
+
data.tar.gz: 33b5b0e21ef7c9896f1ed381130f225a1d51e073112213f631fb1c57c6d69c49af54817b1de3afc9be325c79ac24f9c173275b4448ffb7b43e82403bee03bd70
|
@@ -0,0 +1,58 @@
|
|
1
|
+
name: CI
|
2
|
+
|
3
|
+
on: [push]
|
4
|
+
|
5
|
+
jobs:
|
6
|
+
tests:
|
7
|
+
runs-on: ubuntu-latest
|
8
|
+
strategy:
|
9
|
+
fail-fast: false
|
10
|
+
matrix:
|
11
|
+
ruby: [2.6, 2.7]
|
12
|
+
gemfile: [rails.5.2.activerecord, rails.6.0.activerecord, rails.6.1.activerecord]
|
13
|
+
name: ${{ matrix.ruby }}-${{ matrix.gemfile }}
|
14
|
+
|
15
|
+
env:
|
16
|
+
BUNDLE_GEMFILE: gemfiles/${{ matrix.gemfile }}.gemfile
|
17
|
+
|
18
|
+
steps:
|
19
|
+
- uses: actions/checkout@v2
|
20
|
+
- uses: ruby/setup-ruby@v1
|
21
|
+
with:
|
22
|
+
ruby-version: ${{ matrix.ruby }}
|
23
|
+
bundler-cache: true
|
24
|
+
- name: Run Elasticsearch
|
25
|
+
uses: elastic/elastic-github-actions/elasticsearch@master
|
26
|
+
with:
|
27
|
+
stack-version: 7.10.1
|
28
|
+
port: 9250
|
29
|
+
- name: Tests
|
30
|
+
run: bundle exec rspec
|
31
|
+
|
32
|
+
ruby-3-0-activerecord-6-1:
|
33
|
+
runs-on: ubuntu-latest
|
34
|
+
env:
|
35
|
+
BUNDLE_GEMFILE: gemfiles/rails.6.1.activerecord.gemfile
|
36
|
+
steps:
|
37
|
+
- uses: actions/checkout@v2
|
38
|
+
- uses: ruby/setup-ruby@v1
|
39
|
+
with:
|
40
|
+
ruby-version: '3.0'
|
41
|
+
bundler-cache: true
|
42
|
+
- name: Run Elasticsearch
|
43
|
+
uses: elastic/elastic-github-actions/elasticsearch@master
|
44
|
+
with:
|
45
|
+
stack-version: 7.10.1
|
46
|
+
port: 9250
|
47
|
+
- name: Tests
|
48
|
+
run: bundle exec rspec
|
49
|
+
|
50
|
+
rubocop:
|
51
|
+
runs-on: ubuntu-latest
|
52
|
+
steps:
|
53
|
+
- uses: actions/checkout@v2
|
54
|
+
- uses: ruby/setup-ruby@v1
|
55
|
+
with:
|
56
|
+
ruby-version: 2.7
|
57
|
+
bundler-cache: true
|
58
|
+
- run: bundle exec rubocop --format simple
|
data/.rubocop.yml
CHANGED
@@ -1,24 +1,31 @@
|
|
1
1
|
inherit_from: .rubocop_todo.yml
|
2
2
|
|
3
|
+
AllCops:
|
4
|
+
NewCops: enable
|
5
|
+
TargetRubyVersion: 2.6
|
6
|
+
|
3
7
|
Layout/AccessModifierIndentation:
|
4
8
|
EnforcedStyle: outdent
|
5
9
|
|
6
|
-
Layout/
|
10
|
+
Layout/HashAlignment:
|
7
11
|
EnforcedLastArgumentHashStyle: always_ignore
|
8
12
|
|
9
|
-
Layout/
|
13
|
+
Layout/ParameterAlignment:
|
10
14
|
EnforcedStyle: with_fixed_indentation
|
11
15
|
|
12
16
|
Layout/CaseIndentation:
|
13
17
|
EnforcedStyle: end
|
14
18
|
|
15
|
-
Layout/
|
19
|
+
Layout/EndAlignment:
|
20
|
+
EnforcedStyleAlignWith: variable
|
21
|
+
|
22
|
+
Layout/FirstArrayElementIndentation:
|
16
23
|
EnforcedStyle: consistent
|
17
24
|
|
18
|
-
Layout/
|
25
|
+
Layout/FirstHashElementIndentation:
|
19
26
|
EnforcedStyle: consistent
|
20
27
|
|
21
|
-
Layout/
|
28
|
+
Layout/HeredocIndentation:
|
22
29
|
Enabled: false
|
23
30
|
|
24
31
|
Layout/MultilineMethodCallIndentation:
|
@@ -33,9 +40,6 @@ Layout/SpaceInsideHashLiteralBraces:
|
|
33
40
|
Lint/AmbiguousBlockAssociation:
|
34
41
|
Enabled: false
|
35
42
|
|
36
|
-
Lint/EndAlignment:
|
37
|
-
EnforcedStyleAlignWith: variable
|
38
|
-
|
39
43
|
Style/Alias:
|
40
44
|
EnforcedStyle: prefer_alias_method
|
41
45
|
|
@@ -53,4 +57,5 @@ Metrics/BlockLength:
|
|
53
57
|
|
54
58
|
Metrics/ModuleLength:
|
55
59
|
Exclude:
|
60
|
+
- 'lib/chewy/rake_helper.rb'
|
56
61
|
- '**/*_spec.rb'
|
data/.rubocop_todo.yml
CHANGED
@@ -1,44 +1,132 @@
|
|
1
1
|
# This configuration was generated by
|
2
2
|
# `rubocop --auto-gen-config`
|
3
|
-
# on
|
3
|
+
# on 2021-04-02 12:44:05 UTC using RuboCop version 1.11.0.
|
4
4
|
# The point is for the user to remove these configuration records
|
5
5
|
# one by one as the offenses are removed from the code base.
|
6
6
|
# Note that changes in the inspected code, or installation of new
|
7
7
|
# versions of RuboCop, may require this file to be generated again.
|
8
8
|
|
9
|
-
# Offense count:
|
9
|
+
# Offense count: 1
|
10
|
+
# Configuration parameters: Include.
|
11
|
+
# Include: **/*.gemspec
|
12
|
+
Gemspec/RequiredRubyVersion:
|
13
|
+
Exclude:
|
14
|
+
- 'chewy.gemspec'
|
15
|
+
|
16
|
+
# Offense count: 2
|
17
|
+
# Configuration parameters: AllowedMethods.
|
18
|
+
# AllowedMethods: enums
|
19
|
+
Lint/ConstantDefinitionInBlock:
|
20
|
+
Exclude:
|
21
|
+
- 'lib/chewy/rspec/update_index.rb'
|
22
|
+
- 'spec/chewy/config_spec.rb'
|
23
|
+
|
24
|
+
# Offense count: 10
|
25
|
+
# Configuration parameters: AllowComments, AllowEmptyLambdas.
|
26
|
+
Lint/EmptyBlock:
|
27
|
+
Exclude:
|
28
|
+
- 'spec/chewy/minitest/helpers_spec.rb'
|
29
|
+
- 'spec/chewy/rspec/update_index_spec.rb'
|
30
|
+
- 'spec/chewy/search/scrolling_spec.rb'
|
31
|
+
- 'spec/chewy/strategy/atomic_spec.rb'
|
32
|
+
- 'spec/chewy/strategy_spec.rb'
|
33
|
+
- 'spec/chewy/index/import/bulk_request_spec.rb'
|
34
|
+
- 'spec/chewy/index/witchcraft_spec.rb'
|
35
|
+
- 'spec/chewy_spec.rb'
|
36
|
+
|
37
|
+
# Offense count: 3
|
38
|
+
Lint/MissingSuper:
|
39
|
+
Exclude:
|
40
|
+
- 'lib/chewy/strategy/atomic.rb'
|
41
|
+
- 'lib/chewy/index/adapter/object.rb'
|
42
|
+
- 'lib/chewy/index/adapter/orm.rb'
|
43
|
+
|
44
|
+
# Offense count: 35
|
45
|
+
# Configuration parameters: IgnoredMethods, CountRepeatedAttributes.
|
10
46
|
Metrics/AbcSize:
|
11
|
-
Max:
|
47
|
+
Max: 41
|
12
48
|
|
13
49
|
# Offense count: 4
|
14
|
-
# Configuration parameters: CountComments.
|
50
|
+
# Configuration parameters: CountComments, CountAsOne.
|
15
51
|
Metrics/ClassLength:
|
16
|
-
Max:
|
52
|
+
Max: 267
|
17
53
|
|
18
|
-
# Offense count:
|
54
|
+
# Offense count: 13
|
55
|
+
# Configuration parameters: IgnoredMethods.
|
19
56
|
Metrics/CyclomaticComplexity:
|
20
|
-
Max:
|
21
|
-
|
22
|
-
# Offense count: 1450
|
23
|
-
# Configuration parameters: AllowHeredoc, AllowURI, URISchemes.
|
24
|
-
# URISchemes: http, https
|
25
|
-
Metrics/LineLength:
|
26
|
-
Max: 198
|
57
|
+
Max: 12
|
27
58
|
|
28
|
-
# Offense count:
|
29
|
-
# Configuration parameters: CountComments.
|
59
|
+
# Offense count: 43
|
60
|
+
# Configuration parameters: CountComments, CountAsOne, ExcludedMethods, IgnoredMethods.
|
30
61
|
Metrics/MethodLength:
|
31
|
-
Max:
|
62
|
+
Max: 29
|
32
63
|
|
33
|
-
# Offense count:
|
34
|
-
# Configuration parameters: CountComments.
|
64
|
+
# Offense count: 2
|
65
|
+
# Configuration parameters: CountComments, CountAsOne.
|
35
66
|
Metrics/ModuleLength:
|
36
|
-
Max:
|
67
|
+
Max: 158
|
37
68
|
|
38
|
-
# Offense count:
|
69
|
+
# Offense count: 18
|
70
|
+
# Configuration parameters: IgnoredMethods.
|
39
71
|
Metrics/PerceivedComplexity:
|
40
|
-
Max:
|
72
|
+
Max: 13
|
41
73
|
|
42
|
-
# Offense count:
|
74
|
+
# Offense count: 11
|
75
|
+
# Configuration parameters: EnforcedStyle, CheckMethodNames, CheckSymbols, AllowedIdentifiers.
|
76
|
+
# SupportedStyles: snake_case, normalcase, non_integer
|
77
|
+
# AllowedIdentifiers: capture3, iso8601, rfc1123_date, rfc822, rfc2822, rfc3339
|
78
|
+
Naming/VariableNumber:
|
79
|
+
Exclude:
|
80
|
+
- 'spec/chewy/fields/root_spec.rb'
|
81
|
+
|
82
|
+
# Offense count: 5
|
83
|
+
Style/DocumentDynamicEvalDefinition:
|
84
|
+
Exclude:
|
85
|
+
- 'lib/chewy/index/actions.rb'
|
86
|
+
- 'lib/chewy/repository.rb'
|
87
|
+
- 'lib/chewy/search/pagination/kaminari.rb'
|
88
|
+
- 'lib/chewy/index/crutch.rb'
|
89
|
+
- 'lib/chewy/index/witchcraft.rb'
|
90
|
+
|
91
|
+
# Offense count: 58
|
43
92
|
Style/Documentation:
|
44
93
|
Enabled: false
|
94
|
+
|
95
|
+
# Offense count: 2
|
96
|
+
# Cop supports --auto-correct.
|
97
|
+
Style/EvalWithLocation:
|
98
|
+
Exclude:
|
99
|
+
- 'spec/chewy/index_spec.rb'
|
100
|
+
|
101
|
+
# Offense count: 191
|
102
|
+
# Cop supports --auto-correct.
|
103
|
+
# Configuration parameters: EnforcedStyle.
|
104
|
+
# SupportedStyles: always, always_true, never
|
105
|
+
Style/FrozenStringLiteralComment:
|
106
|
+
Enabled: false
|
107
|
+
|
108
|
+
# Offense count: 2
|
109
|
+
# Configuration parameters: MinBodyLength.
|
110
|
+
Style/GuardClause:
|
111
|
+
Exclude:
|
112
|
+
- 'lib/chewy.rb'
|
113
|
+
- 'spec/support/active_record.rb'
|
114
|
+
|
115
|
+
# Offense count: 10
|
116
|
+
# Cop supports --auto-correct.
|
117
|
+
Style/IfUnlessModifier:
|
118
|
+
Exclude:
|
119
|
+
- 'lib/chewy.rb'
|
120
|
+
- 'lib/chewy/railtie.rb'
|
121
|
+
- 'lib/chewy/rspec/update_index.rb'
|
122
|
+
- 'lib/chewy/search/query_proxy.rb'
|
123
|
+
- 'lib/chewy/index/import.rb'
|
124
|
+
- 'lib/chewy/index/witchcraft.rb'
|
125
|
+
- 'spec/support/active_record.rb'
|
126
|
+
|
127
|
+
# Offense count: 53
|
128
|
+
# Cop supports --auto-correct.
|
129
|
+
# Configuration parameters: AutoCorrect, AllowHeredoc, AllowURI, URISchemes, IgnoreCopDirectives, IgnoredPatterns.
|
130
|
+
# URISchemes: http, https
|
131
|
+
Layout/LineLength:
|
132
|
+
Max: 191
|
data/CHANGELOG.md
CHANGED
@@ -8,6 +8,107 @@
|
|
8
8
|
|
9
9
|
### Bugs Fixed
|
10
10
|
|
11
|
+
## 7.2.2 (2021-05-24)
|
12
|
+
|
13
|
+
### Changes
|
14
|
+
|
15
|
+
* [#800](https://github.com/toptal/chewy/pull/800): Revert [#787](https://github.com/toptal/chewy/pull/787) progressbar feature to avoid performance degradation in parallel import ([@rabotyaga][])
|
16
|
+
|
17
|
+
* [#795](https://github.com/toptal/chewy/issues/795): **(Breaking)** Change the Chewy::Search::Parameters::Order implementation to use Array ([@jiajiawang][]):
|
18
|
+
* To allow multiple sorting options that may have the same key name. For example script based sorting whose key will always be `_script`.
|
19
|
+
* Behaviour change of chained `order` calls.
|
20
|
+
* e.g. `.order(_script: {a: 1}).order(_script: {b: 2})`
|
21
|
+
* Before `{:sort=>[{"_script"=>{:b=>2}}]}`
|
22
|
+
* After `{:sort=>[{"_script"=>{:a=>1}},{"_script"=>{:b=>2}}]}`
|
23
|
+
|
24
|
+
* [#654](https://github.com/toptal/chewy/issues/654): Add helpers and matchers for testing ([@Vitalina-Vakulchyk][]):
|
25
|
+
* `mock_elasticsearch_response` helpers both Rspec and Minitest - to mock elasticsearch response
|
26
|
+
* `mock_elasticsearch_response_sources` helpers both Rspec and Minitest - to mock elasticsearch response sources
|
27
|
+
* `assert_elasticsearch_query` helper for Minitest - to compare request and expected query (returns `true`/`false`)
|
28
|
+
* `build_query` matcher for Rspec - to compare request and expected query (returns `true`/`false`)
|
29
|
+
|
30
|
+
## 7.2.1 (2021-05-11)
|
31
|
+
|
32
|
+
### New Features
|
33
|
+
|
34
|
+
* [#469](https://github.com/toptal/chewy/issues/469): Add ability to output the progressbar with `ENV['PROGRESS']` during `reset` rake tasks ([@Vitalina-Vakulchyk][]):
|
35
|
+
* for `rake chewy:reset` and `rake chewy:parallel:reset`
|
36
|
+
* progressbar is hidden by default, set `ENV['PROGRESS']` to `true` to display it
|
37
|
+
|
38
|
+
### Bugs Fixed
|
39
|
+
|
40
|
+
* [#796](https://github.com/toptal/chewy/pull/796): Fix clear scroll: pass `scroll_id` in body, as passing in path parameters is deprecated and can overflow `http.max_initial_line_length` ([@rabotyaga][])
|
41
|
+
|
42
|
+
## 7.0.1 (2021-05-03)
|
43
|
+
|
44
|
+
### Changes
|
45
|
+
|
46
|
+
* [#792](https://github.com/toptal/chewy/pull/792): Skip ES version memoization for search requests ([@rabotyaga][])
|
47
|
+
* See the Migration Guide for details
|
48
|
+
|
49
|
+
## 7.2.0 (2021-04-19)
|
50
|
+
|
51
|
+
### New Features
|
52
|
+
|
53
|
+
* [#778](https://github.com/toptal/chewy/pull/778): Add `ignore_blank` option to `field` method ([@Vitalina-Vakulchyk][]):
|
54
|
+
* `true` by default for the `geo_point` type
|
55
|
+
* `false` by default for other types
|
56
|
+
|
57
|
+
### Changes
|
58
|
+
|
59
|
+
* [#783](https://github.com/toptal/chewy/pull/783): **(Breaking)** Remove `Chewy::Type`, simplify DSL ([@rabotyaga][])
|
60
|
+
* Remove the `Chewy::Type` class
|
61
|
+
* e.g. remove `CitiesIndex::City` / `CitiesIndex.city`
|
62
|
+
* `CitiesIndex::City.import! ...` becomes `CitiesIndex.import! ...`
|
63
|
+
* Simplify index DSL:
|
64
|
+
* `define_type` block -> `index_scope` clause
|
65
|
+
* it can be omitted completely, if you don't need to specify the scope or options, e.g. `name`
|
66
|
+
* Remove type names from string representations:
|
67
|
+
* in `update_index` ActiveRecord helper and RSpec matcher, e.g.
|
68
|
+
* `update_index('cities#city')` -> `update_index('cities')`
|
69
|
+
* `update_index(UsersIndex::User)` -> `update_index(UsersIndex)`
|
70
|
+
* in rake tasks (e.g. `rake chewy:update[cities#city]` -> `rake chewy:update[cities]`)
|
71
|
+
* in rake tasks output (e.g. `Imported CitiesIndex::City in 1s, stats: index 3` -> `Imported CitiesIndex in 1s, stats: index 3`)
|
72
|
+
* Use index name instead of type name in loader additional scope
|
73
|
+
* e.g. `CitiesIndex.filter(...).load(city: {scope: City.where(...)})` -> `CitiesIndex.filter(...).load(cities: {scope: City.where(...)})`
|
74
|
+
* [#692](https://github.com/toptal/chewy/issues/692): Add `.update_mapping` to Index class ([@Vitalina-Vakulchyk][]):
|
75
|
+
* Wrapped Elasticsearch gem `.put_mapping` with `.update_mapping` in Index class
|
76
|
+
* Add `rake chewy:update_mapping` task
|
77
|
+
* [#594](https://github.com/toptal/chewy/issues/594): Add `.reindex` to Index class ([@Vitalina-Vakulchyk][]):
|
78
|
+
* Wrapped Elasticsearch gem `.reindex` with `.reindex` in Index class
|
79
|
+
* Add `rake chewy:reindex` task
|
80
|
+
* [#679](https://github.com/toptal/chewy/issues/679): Wrapped `Elasticsearch::API::Indices::Actions#clear_cache` with `.clear_cache` in Index class ([@Vitalina-Vakulchyk][])
|
81
|
+
* [#495](https://github.com/toptal/chewy/issues/495): Ability to change Rails console strategy with `Chewy.console_strategy` ([@Vitalina-Vakulchyk][])
|
82
|
+
* [#778](https://github.com/toptal/chewy/pull/778): **(Breaking)** Drop support for Ruby 2.5 ([@Vitalina-Vakulchyk][])
|
83
|
+
* [#776](https://github.com/toptal/chewy/pull/776): **(Breaking)** Removal of unnecessary features and integrations ([@Vitalina-Vakulchyk][]):
|
84
|
+
* `aws-sdk-sqs` / `shoryuken`
|
85
|
+
* `mongoid`
|
86
|
+
* `sequel`
|
87
|
+
* `will_paginate`
|
88
|
+
* `resque`
|
89
|
+
* [#769](https://github.com/toptal/chewy/pull/769): **(Breaking)** Removal of deprecated methods and rake tasks ([@Vitalina-Vakulchyk][]):
|
90
|
+
* `Chewy::Index.index_params` is removed, use `Chewy::Index.specification_hash` instead
|
91
|
+
* `Chewy::Index.derivable_index_name` is removed, use `Chewy::Index.derivable_name` instead
|
92
|
+
* `Chewy::Index.default_prefix` is removed, use `Chewy::Index.prefix` instead
|
93
|
+
* `Chewy::Index.build_index_name` is removed, use `Chewy::Index.index_name` instead
|
94
|
+
* `Chewy::RakeHelper.reset_index` is removed, use `Chewy::RakeHelper.reset` instead
|
95
|
+
* `Chewy::RakeHelper.reset_all` is removed, use `Chewy::RakeHelper.reset` instead
|
96
|
+
* `Chewy::RakeHelper.update_index` is removed, use `Chewy::RakeHelper.update` instead
|
97
|
+
* `Chewy::RakeHelper.update_all` is removed, use `Chewy::RakeHelper.update` instead
|
98
|
+
* `rake chewy:apply_changes_from` is removed, use `rake chewy:journal:apply` instead
|
99
|
+
* `rake chewy:clean_journal` is removed, use `rake chewy:journal:clean` instead
|
100
|
+
|
101
|
+
## 7.1.0 (2021-03-03)
|
102
|
+
|
103
|
+
### Changes
|
104
|
+
|
105
|
+
* [#766](https://github.com/toptal/chewy/pull/766): **(Breaking)** Drop support for Elasticsearch 6.x ([@rabotyaga][])
|
106
|
+
* [#765](https://github.com/toptal/chewy/pull/765): Fix ruby 2.7 warnings in rake tasks ([@aglushkov][])
|
107
|
+
|
108
|
+
### Bugs Fixed
|
109
|
+
|
110
|
+
* [#722](https://github.com/toptal/chewy/issues/722): Remove alias_method_chain, use Module#prepend instead ([@dalthon][])
|
111
|
+
|
11
112
|
## 7.0.0 (2021-02-22)
|
12
113
|
|
13
114
|
### New Features
|
@@ -549,6 +650,7 @@
|
|
549
650
|
[@inbeom]: https://github.com/inbeom
|
550
651
|
[@jesjos]: https://github.com/jesjos
|
551
652
|
[@JF-Lalonde]: https://github.com/JF-Lalonde
|
653
|
+
[@jiajiawang]: https://github.com/jiajiawang
|
552
654
|
[@jimmybaker]: https://github.com/jimmybaker
|
553
655
|
[@jirikolarik]: https://github.com/jirikolarik
|
554
656
|
[@jirutka]: https://github.com/jirutka
|
@@ -596,5 +698,6 @@
|
|
596
698
|
[@taylor-au]: https://github.com/taylor-au
|
597
699
|
[@TikiTDO]: https://github.com/TikiTDO
|
598
700
|
[@undr]: https://github.com/undr
|
701
|
+
[@Vitalina-Vakulchyk]: https://github.com/Vitalina-Vakulchyk
|
599
702
|
[@webgago]: https://github.com/webgago
|
600
703
|
[@yahooguntu]: https://github.com/yahooguntu
|
data/Gemfile
CHANGED
@@ -3,18 +3,11 @@ source 'https://rubygems.org'
|
|
3
3
|
gemspec
|
4
4
|
|
5
5
|
gem 'activerecord'
|
6
|
-
# gem 'mongoid'
|
7
|
-
# gem 'sequel'
|
8
6
|
|
9
7
|
gem 'activejob', require: false
|
10
|
-
gem 'resque', require: false
|
11
8
|
gem 'sidekiq', require: false
|
12
9
|
|
13
|
-
gem 'aws-sdk-sqs', require: false
|
14
|
-
gem 'shoryuken', require: false
|
15
|
-
|
16
10
|
gem 'kaminari-core', require: false
|
17
|
-
gem 'will_paginate', require: false
|
18
11
|
|
19
12
|
gem 'parallel', require: false
|
20
13
|
gem 'ruby-progressbar', require: false
|
data/Guardfile
CHANGED
@@ -9,7 +9,9 @@ guard :rspec, cmd: 'rspec' do
|
|
9
9
|
# Rails example
|
10
10
|
watch(%r{^app/(.+)\.rb$}) { |m| "spec/#{m[1]}_spec.rb" }
|
11
11
|
watch(%r{^app/(.*)(\.erb|\.haml|\.slim)$}) { |m| "spec/#{m[1]}#{m[2]}_spec.rb" }
|
12
|
-
watch(%r{^app/controllers/(.+)_(controller)\.rb$})
|
12
|
+
watch(%r{^app/controllers/(.+)_(controller)\.rb$}) do |m|
|
13
|
+
["spec/routing/#{m[1]}_routing_spec.rb", "spec/#{m[2]}s/#{m[1]}_#{m[2]}_spec.rb", "spec/acceptance/#{m[1]}_spec.rb"]
|
14
|
+
end
|
13
15
|
watch(%r{^spec/support/(.+)\.rb$}) { 'spec' }
|
14
16
|
watch('config/routes.rb') { 'spec/routing' }
|
15
17
|
watch('app/controllers/application_controller.rb') { 'spec/controllers' }
|
data/README.md
CHANGED
@@ -1,5 +1,5 @@
|
|
1
1
|
[![Gem Version](https://badge.fury.io/rb/chewy.svg)](http://badge.fury.io/rb/chewy)
|
2
|
-
[![
|
2
|
+
[![GitHub Actions](https://github.com/toptal/chewy/actions/workflows/ruby.yml/badge.svg)](https://github.com/toptal/chewy/actions/workflows/ruby.yml)
|
3
3
|
[![Code Climate](https://codeclimate.com/github/toptal/chewy.svg)](https://codeclimate.com/github/toptal/chewy)
|
4
4
|
[![Inline docs](http://inch-ci.org/github/toptal/chewy.svg?branch=master)](http://inch-ci.org/github/toptal/chewy)
|
5
5
|
|
@@ -7,58 +7,10 @@
|
|
7
7
|
|
8
8
|
Chewy is an ODM (Object Document Mapper), built on top of the [the official Elasticsearch client](https://github.com/elastic/elasticsearch-ruby).
|
9
9
|
|
10
|
-
## Table of Contents
|
11
|
-
|
12
|
-
* [Why Chewy?](#why-chewy)
|
13
|
-
* [Installation](#installation)
|
14
|
-
* [Usage](#usage)
|
15
|
-
* [Client settings](#client-settings)
|
16
|
-
* [AWS ElasticSearch configuration](#aws-elastic-search)
|
17
|
-
* [Index definition](#index-definition)
|
18
|
-
* [Type default import options](#type-default-import-options)
|
19
|
-
* [Multi (nested) and object field types](#multi-nested-and-object-field-types)
|
20
|
-
* [Parent and children types](#parent-and-children-types)
|
21
|
-
* [Geo Point fields](#geo-point-fields)
|
22
|
-
* [Crutches™ technology](#crutches-technology)
|
23
|
-
* [Witchcraft™ technology](#witchcraft-technology)
|
24
|
-
* [Raw Import](#raw-import)
|
25
|
-
* [Index creation during import](#index-creation-during-import)
|
26
|
-
* [Journaling](#journaling)
|
27
|
-
* [Types access](#types-access)
|
28
|
-
* [Index manipulation](#index-manipulation)
|
29
|
-
* [Index update strategies](#index-update-strategies)
|
30
|
-
* [Nesting](#nesting)
|
31
|
-
* [Non-block notation](#non-block-notation)
|
32
|
-
* [Designing your own strategies](#designing-your-own-strategies)
|
33
|
-
* [Rails application strategies integration](#rails-application-strategies-integration)
|
34
|
-
* [ActiveSupport::Notifications support](#activesupportnotifications-support)
|
35
|
-
* [NewRelic integration](#newrelic-integration)
|
36
|
-
* [Search requests](#search-requests)
|
37
|
-
* [Composing requests](#composing-requests)
|
38
|
-
* [Pagination](#pagination)
|
39
|
-
* [Named scopes](#named-scopes)
|
40
|
-
* [Scroll API](#scroll-api)
|
41
|
-
* [Loading objects](#loading-objects)
|
42
|
-
* [Rake tasks](#rake-tasks)
|
43
|
-
* [chewy:reset](#chewyreset)
|
44
|
-
* [chewy:upgrade](#chewyupgrade)
|
45
|
-
* [chewy:update](#chewyupdate)
|
46
|
-
* [chewy:sync](#chewysync)
|
47
|
-
* [chewy:deploy](#chewydeploy)
|
48
|
-
* [Parallelizing rake tasks](#parallelizing-rake-tasks)
|
49
|
-
* [chewy:journal](#chewyjournal)
|
50
|
-
* [RSpec integration](#rspec-integration)
|
51
|
-
* [Minitest integration](#minitest-integration)
|
52
|
-
* [Contributing](#contributing)
|
53
|
-
|
54
10
|
## Why Chewy?
|
55
11
|
|
56
12
|
In this section we'll cover why you might want to use Chewy instead of the official `elasticsearch-ruby` client gem.
|
57
13
|
|
58
|
-
* Multi-model indices.
|
59
|
-
|
60
|
-
Index classes are independent from ORM/ODM models. Now, implementing e.g. cross-model autocomplete is much easier. You can just define the index and work with it in an object-oriented style. You can define several types for index - one per indexed model.
|
61
|
-
|
62
14
|
* Every index is observable by all the related models.
|
63
15
|
|
64
16
|
Most of the indexed models are related to other and sometimes it is necessary to denormalize this related data and put at the same object. For example, you need to index an array of tags together with an article. Chewy allows you to specify an updateable index for every model separately - so corresponding articles will be reindexed on any tag update.
|
@@ -71,7 +23,7 @@ In this section we'll cover why you might want to use Chewy instead of the offic
|
|
71
23
|
|
72
24
|
Chewy has an ActiveRecord-style query DSL. It is chainable, mergeable and lazy, so you can produce queries in the most efficient way. It also has object-oriented query and filter builders.
|
73
25
|
|
74
|
-
* Support for ActiveRecord
|
26
|
+
* Support for ActiveRecord.
|
75
27
|
|
76
28
|
## Installation
|
77
29
|
|
@@ -91,7 +43,7 @@ Or install it yourself as:
|
|
91
43
|
|
92
44
|
### Ruby
|
93
45
|
|
94
|
-
Chewy is compatible with MRI 2.
|
46
|
+
Chewy is compatible with MRI 2.6-3.0¹.
|
95
47
|
|
96
48
|
> ¹ Ruby 3 is only supported with Rails 6.1
|
97
49
|
|
@@ -99,29 +51,169 @@ Chewy is compatible with MRI 2.5-3.0¹.
|
|
99
51
|
|
100
52
|
| Chewy version | Elasticsearch version |
|
101
53
|
| ------------- | ---------------------------------- |
|
102
|
-
| 7.
|
54
|
+
| 7.2.x | 7.x |
|
55
|
+
| 7.1.x | 7.x |
|
56
|
+
| 7.0.x | 6.8, 7.x |
|
103
57
|
| 6.0.0 | 5.x, 6.x |
|
104
58
|
| 5.x | 5.x, limited support for 1.x & 2.x |
|
105
59
|
|
106
|
-
|
60
|
+
**Important:** Chewy doesn't follow SemVer, so you should always
|
61
|
+
check the release notes before upgrading. The major version is linked to the
|
62
|
+
newest supported Elasticsearch and the minor version bumps may include breaking changes.
|
107
63
|
|
108
|
-
|
64
|
+
See our [migration guide](migration_guide.md) for detailed upgrade instructions between
|
65
|
+
various Chewy versions.
|
109
66
|
|
110
|
-
###
|
67
|
+
### Active Record
|
68
|
+
|
69
|
+
5.2, 6.0, 6.1 Active Record versions are supported by all Chewy versions.
|
70
|
+
|
71
|
+
## Getting Started
|
72
|
+
|
73
|
+
Chewy provides functionality for Elasticsearch index handling, documents import mappings, index update strategies and chainable query DSL.
|
74
|
+
|
75
|
+
### Minimal client setting
|
76
|
+
|
77
|
+
Create `config/initializers/chewy.rb` with this line:
|
78
|
+
|
79
|
+
```ruby
|
80
|
+
Chewy.settings = {host: 'localhost:9250'}
|
81
|
+
```
|
82
|
+
|
83
|
+
And run `rails g chewy:install` to generate `chewy.yml`:
|
84
|
+
|
85
|
+
```yaml
|
86
|
+
# config/chewy.yml
|
87
|
+
# separate environment configs
|
88
|
+
test:
|
89
|
+
host: 'localhost:9250'
|
90
|
+
prefix: 'test'
|
91
|
+
development:
|
92
|
+
host: 'localhost:9200'
|
93
|
+
```
|
94
|
+
|
95
|
+
### Elasticsearch
|
96
|
+
|
97
|
+
Make sure you have Elasticsearch up and running. You can [install](https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html) it locally, but the easiest way is to use [Docker](https://www.docker.com/get-started):
|
98
|
+
|
99
|
+
```shell
|
100
|
+
$ docker run --rm --name elasticsearch -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" elasticsearch:7.11.1
|
101
|
+
```
|
102
|
+
|
103
|
+
### Index
|
104
|
+
|
105
|
+
Create `app/chewy/users_index.rb` with User Index:
|
106
|
+
|
107
|
+
```ruby
|
108
|
+
class UsersIndex < Chewy::Index
|
109
|
+
settings analysis: {
|
110
|
+
analyzer: {
|
111
|
+
email: {
|
112
|
+
tokenizer: 'keyword',
|
113
|
+
filter: ['lowercase']
|
114
|
+
}
|
115
|
+
}
|
116
|
+
}
|
117
|
+
|
118
|
+
index_scope User
|
119
|
+
field :first_name
|
120
|
+
field :last_name
|
121
|
+
field :email, analyzer: 'email'
|
122
|
+
end
|
123
|
+
```
|
124
|
+
|
125
|
+
### Model
|
111
126
|
|
112
|
-
|
127
|
+
Add User model, table and migrate it:
|
113
128
|
|
114
|
-
|
115
|
-
|
129
|
+
```shell
|
130
|
+
$ bundle exec rails g model User first_name last_name email
|
131
|
+
$ bundle exec rails db:migrate
|
132
|
+
```
|
133
|
+
|
134
|
+
Add `update_index` to app/models/user.rb:
|
135
|
+
|
136
|
+
```ruby
|
137
|
+
class User < ApplicationRecord
|
138
|
+
update_index('users') { self }
|
139
|
+
end
|
140
|
+
```
|
141
|
+
|
142
|
+
### Example of data request
|
143
|
+
|
144
|
+
1. Once a record is created (could be done via the Rails console), it creates User index too:
|
145
|
+
|
146
|
+
```
|
147
|
+
User.create(
|
148
|
+
first_name: "test1",
|
149
|
+
last_name: "test1",
|
150
|
+
email: 'test1@example.com',
|
151
|
+
# other fields
|
152
|
+
)
|
153
|
+
# UsersIndex Import (355.3ms) {:index=>1}
|
154
|
+
# => #<User id: 1, first_name: "test1", last_name: "test1", email: "test1@example.com", # other fields>
|
155
|
+
```
|
156
|
+
|
157
|
+
2. A query could be exposed at a given `UsersController`:
|
158
|
+
|
159
|
+
```ruby
|
160
|
+
def search
|
161
|
+
@users = UsersIndex.query(query_string: { fields: [:first_name, :last_name, :email, ...], query: search_params[:query], default_operator: 'and' })
|
162
|
+
render json: @users.to_json, status: :ok
|
163
|
+
end
|
164
|
+
|
165
|
+
private
|
166
|
+
|
167
|
+
def search_params
|
168
|
+
params.permit(:query, :page, :per)
|
169
|
+
end
|
170
|
+
```
|
116
171
|
|
117
|
-
|
118
|
-
|
172
|
+
3. So a request against `http://localhost:3000/users/search?query=test1@example.com` issuing a response like:
|
173
|
+
|
174
|
+
```json
|
175
|
+
[
|
176
|
+
{
|
177
|
+
"attributes":{
|
178
|
+
"id":"1",
|
179
|
+
"first_name":"test1",
|
180
|
+
"last_name":"test1",
|
181
|
+
"email":"test1@example.com",
|
182
|
+
...
|
183
|
+
"_score":0.9808291,
|
184
|
+
"_explanation":null
|
185
|
+
},
|
186
|
+
"_data":{
|
187
|
+
"_index":"users",
|
188
|
+
"_type":"_doc",
|
189
|
+
"_id":"1",
|
190
|
+
"_score":0.9808291,
|
191
|
+
"_source":{
|
192
|
+
"first_name":"test1",
|
193
|
+
"last_name":"test1",
|
194
|
+
"email":"test1@example.com",
|
195
|
+
...
|
196
|
+
}
|
197
|
+
}
|
198
|
+
}
|
199
|
+
]
|
200
|
+
```
|
201
|
+
|
202
|
+
## Usage and configuration
|
203
|
+
|
204
|
+
### Client settings
|
205
|
+
|
206
|
+
To configure the Chewy client you need to add `chewy.rb` file with `Chewy.settings` hash:
|
119
207
|
|
120
208
|
```ruby
|
121
209
|
# config/initializers/chewy.rb
|
122
210
|
Chewy.settings = {host: 'localhost:9250'} # do not use environments
|
123
211
|
```
|
124
212
|
|
213
|
+
And add `chewy.yml` configuration file.
|
214
|
+
|
215
|
+
You can create `chewy.yml` manually or run `rails g chewy:install` to generate it:
|
216
|
+
|
125
217
|
```yaml
|
126
218
|
# config/chewy.yml
|
127
219
|
# separate environment configs
|
@@ -171,7 +263,7 @@ Chewy.settings = {
|
|
171
263
|
}
|
172
264
|
```
|
173
265
|
|
174
|
-
|
266
|
+
#### Index definition
|
175
267
|
|
176
268
|
1. Create `/app/chewy/users_index.rb`
|
177
269
|
|
@@ -181,41 +273,38 @@ Chewy.settings = {
|
|
181
273
|
end
|
182
274
|
```
|
183
275
|
|
184
|
-
2.
|
276
|
+
2. Define index scope (you can omit this part if you don't need to specify a scope (i.e. use PORO objects for import) or options)
|
185
277
|
|
186
278
|
```ruby
|
187
279
|
class UsersIndex < Chewy::Index
|
188
|
-
|
280
|
+
index_scope User.active # or just model instead_of scope: index_scope User
|
189
281
|
end
|
190
282
|
```
|
191
283
|
|
192
|
-
|
193
|
-
|
194
|
-
3. Add some type mappings
|
284
|
+
3. Add some mappings
|
195
285
|
|
196
286
|
```ruby
|
197
287
|
class UsersIndex < Chewy::Index
|
198
|
-
|
199
|
-
|
200
|
-
|
201
|
-
|
202
|
-
|
203
|
-
|
204
|
-
|
205
|
-
|
206
|
-
|
207
|
-
|
208
|
-
end
|
209
|
-
field :rating, type: 'integer' # custom data type
|
210
|
-
field :created, type: 'date', include_in_all: false,
|
211
|
-
value: ->{ created_at } # value proc for source object context
|
288
|
+
index_scope User.active.includes(:country, :badges, :projects)
|
289
|
+
field :first_name, :last_name # multiple fields without additional options
|
290
|
+
field :email, analyzer: 'email' # Elasticsearch-related options
|
291
|
+
field :country, value: ->(user) { user.country.name } # custom value proc
|
292
|
+
field :badges, value: ->(user) { user.badges.map(&:name) } # passing array values to index
|
293
|
+
field :projects do # the same block syntax for multi_field, if `:type` is specified
|
294
|
+
field :title
|
295
|
+
field :description # default data type is `text`
|
296
|
+
# additional top-level objects passed to value proc:
|
297
|
+
field :categories, value: ->(project, user) { project.categories.map(&:name) if user.active? }
|
212
298
|
end
|
299
|
+
field :rating, type: 'integer' # custom data type
|
300
|
+
field :created, type: 'date', include_in_all: false,
|
301
|
+
value: ->{ created_at } # value proc for source object context
|
213
302
|
end
|
214
303
|
```
|
215
304
|
|
216
305
|
[See here for mapping definitions](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html).
|
217
306
|
|
218
|
-
4. Add some index-
|
307
|
+
4. Add some index-related settings. Analyzer repositories might be used as well. See `Chewy::Index.settings` docs for details:
|
219
308
|
|
220
309
|
```ruby
|
221
310
|
class UsersIndex < Chewy::Index
|
@@ -228,23 +317,22 @@ Chewy.settings = {
|
|
228
317
|
}
|
229
318
|
}
|
230
319
|
|
231
|
-
|
232
|
-
|
233
|
-
|
234
|
-
|
235
|
-
|
236
|
-
|
237
|
-
|
238
|
-
|
239
|
-
|
240
|
-
|
241
|
-
|
242
|
-
end
|
243
|
-
field :about_translations, type: 'object' # pass object type explicitly if necessary
|
244
|
-
field :rating, type: 'integer'
|
245
|
-
field :created, type: 'date', include_in_all: false,
|
246
|
-
value: ->{ created_at }
|
320
|
+
index_scope User.active.includes(:country, :badges, :projects)
|
321
|
+
root date_detection: false do
|
322
|
+
template 'about_translations.*', type: 'text', analyzer: 'standard'
|
323
|
+
|
324
|
+
field :first_name, :last_name
|
325
|
+
field :email, analyzer: 'email'
|
326
|
+
field :country, value: ->(user) { user.country.name }
|
327
|
+
field :badges, value: ->(user) { user.badges.map(&:name) }
|
328
|
+
field :projects do
|
329
|
+
field :title
|
330
|
+
field :description
|
247
331
|
end
|
332
|
+
field :about_translations, type: 'object' # pass object type explicitly if necessary
|
333
|
+
field :rating, type: 'integer'
|
334
|
+
field :created, type: 'date', include_in_all: false,
|
335
|
+
value: ->{ created_at }
|
248
336
|
end
|
249
337
|
end
|
250
338
|
```
|
@@ -258,39 +346,32 @@ Chewy.settings = {
|
|
258
346
|
|
259
347
|
```ruby
|
260
348
|
class User < ActiveRecord::Base
|
261
|
-
update_index('users
|
349
|
+
update_index('users') { self } # specifying index and back-reference
|
262
350
|
# for updating after user save or destroy
|
263
351
|
end
|
264
352
|
|
265
353
|
class Country < ActiveRecord::Base
|
266
354
|
has_many :users
|
267
355
|
|
268
|
-
update_index('users
|
356
|
+
update_index('users') { users } # return single object or collection
|
269
357
|
end
|
270
358
|
|
271
359
|
class Project < ActiveRecord::Base
|
272
|
-
update_index('users
|
273
|
-
end
|
274
|
-
|
275
|
-
class Badge < ActiveRecord::Base
|
276
|
-
has_and_belongs_to_many :users
|
277
|
-
|
278
|
-
update_index('users') { users } # if index has only one type
|
279
|
-
# there is no need to specify updated type
|
360
|
+
update_index('users') { user if user.active? } # you can return even `nil` from the back-reference
|
280
361
|
end
|
281
362
|
|
282
363
|
class Book < ActiveRecord::Base
|
283
|
-
update_index(->(book) {"
|
284
|
-
|
285
|
-
|
364
|
+
update_index(->(book) {"books_#{book.language}"}) { self } # dynamic index name with proc.
|
365
|
+
# For book with language == "en"
|
366
|
+
# this code will generate `books_en`
|
286
367
|
end
|
287
368
|
```
|
288
369
|
|
289
370
|
Also, you can use the second argument for method name passing:
|
290
371
|
|
291
372
|
```ruby
|
292
|
-
update_index('users
|
293
|
-
update_index('users
|
373
|
+
update_index('users', :self)
|
374
|
+
update_index('users', :users)
|
294
375
|
```
|
295
376
|
|
296
377
|
In the case of a belongs_to association you may need to update both associated objects, previous and current:
|
@@ -299,47 +380,28 @@ Chewy.settings = {
|
|
299
380
|
class City < ActiveRecord::Base
|
300
381
|
belongs_to :country
|
301
382
|
|
302
|
-
update_index('cities
|
303
|
-
update_index 'countries
|
304
|
-
# For the latest active_record changed values are
|
305
|
-
# already in `previous_changes` hash,
|
306
|
-
# but for mongoid you have to use `changes` hash
|
383
|
+
update_index('cities') { self }
|
384
|
+
update_index 'countries' do
|
307
385
|
previous_changes['country_id'] || country
|
308
386
|
end
|
309
387
|
end
|
310
388
|
```
|
311
389
|
|
312
|
-
|
313
|
-
|
314
|
-
```ruby
|
315
|
-
class User < Sequel::Model
|
316
|
-
update_index('users#user') { self }
|
317
|
-
end
|
318
|
-
```
|
390
|
+
### Default import options
|
319
391
|
|
320
|
-
|
321
|
-
|
322
|
-
```ruby
|
323
|
-
Sequel::Model.plugin :chewy_observe # for all models, or...
|
324
|
-
User.plugin :chewy_observe # just for User
|
325
|
-
```
|
326
|
-
|
327
|
-
### Type default import options
|
328
|
-
|
329
|
-
Every type has `default_import_options` configuration to specify, suddenly, default import options:
|
392
|
+
Every index has `default_import_options` configuration to specify, suddenly, default import options:
|
330
393
|
|
331
394
|
```ruby
|
332
395
|
class ProductsIndex < Chewy::Index
|
333
|
-
|
334
|
-
|
396
|
+
index_scope Post.includes(:tags)
|
397
|
+
default_import_options batch_size: 100, bulk_size: 10.megabytes, refresh: false
|
335
398
|
|
336
|
-
|
337
|
-
|
338
|
-
end
|
399
|
+
field :name
|
400
|
+
field :tags, value: -> { tags.map(&:name) }
|
339
401
|
end
|
340
402
|
```
|
341
403
|
|
342
|
-
See [import.rb](lib/chewy/
|
404
|
+
See [import.rb](lib/chewy/index/import.rb) for available options.
|
343
405
|
|
344
406
|
### Multi (nested) and object field types
|
345
407
|
|
@@ -365,18 +427,6 @@ end
|
|
365
427
|
|
366
428
|
The `value:` option for internal fields will no longer be effective.
|
367
429
|
|
368
|
-
### Parent and children types
|
369
|
-
|
370
|
-
To define [parent](https://www.elastic.co/guide/en/elasticsearch/guide/current/parent-child-mapping.html) type for a given index_type, you can include root options for the type where you can specify parent_type and parent_id
|
371
|
-
|
372
|
-
```ruby
|
373
|
-
define_type User.includes(:account) do
|
374
|
-
root parent: 'account', parent_id: ->{ account_id } do
|
375
|
-
field :created_at, type: 'date'
|
376
|
-
field :task_id, type: 'integer'
|
377
|
-
end
|
378
|
-
end
|
379
|
-
```
|
380
430
|
### Geo Point fields
|
381
431
|
|
382
432
|
You can use [Elasticsearch's geo mapping](https://www.elastic.co/guide/en/elasticsearch/reference/current/geo-point.html) with the `geo_point` field type, allowing you to query, filter and order by latitude and longitude. You can use the following hash format:
|
@@ -402,14 +452,13 @@ Assume you are defining your index like this (product has_many categories throug
|
|
402
452
|
|
403
453
|
```ruby
|
404
454
|
class ProductsIndex < Chewy::Index
|
405
|
-
|
406
|
-
|
407
|
-
|
408
|
-
end
|
455
|
+
index_scope Product.includes(:categories)
|
456
|
+
field :name
|
457
|
+
field :category_names, value: ->(product) { product.categories.map(&:name) } # or shorter just -> { categories.map(&:name) }
|
409
458
|
end
|
410
459
|
```
|
411
460
|
|
412
|
-
Then the Chewy reindexing flow will look like the following pseudo-code
|
461
|
+
Then the Chewy reindexing flow will look like the following pseudo-code:
|
413
462
|
|
414
463
|
```ruby
|
415
464
|
Product.includes(:categories).find_in_batches(1000) do |batch|
|
@@ -421,26 +470,23 @@ Product.includes(:categories).find_in_batches(1000) do |batch|
|
|
421
470
|
end
|
422
471
|
```
|
423
472
|
|
424
|
-
|
425
|
-
|
426
|
-
Then you can replace Rails associations with Chewy Crutches™ technology:
|
473
|
+
If you meet complicated cases when associations are not applicable you can replace Rails associations with Chewy Crutches™ technology:
|
427
474
|
|
428
475
|
```ruby
|
429
476
|
class ProductsIndex < Chewy::Index
|
430
|
-
|
431
|
-
|
432
|
-
|
433
|
-
|
434
|
-
|
435
|
-
|
436
|
-
|
437
|
-
|
438
|
-
end
|
439
|
-
|
440
|
-
field :name
|
441
|
-
# simply use crutch-fetched data as a value:
|
442
|
-
field :category_names, value: ->(product, crutches) { crutches.categories[product.id] }
|
477
|
+
index_scope Product
|
478
|
+
crutch :categories do |collection| # collection here is a current batch of products
|
479
|
+
# data is fetched with a lightweight query without objects initialization
|
480
|
+
data = ProductCategory.joins(:category).where(product_id: collection.map(&:id)).pluck(:product_id, 'categories.name')
|
481
|
+
# then we have to convert fetched data to appropriate format
|
482
|
+
# this will return our data in structure like:
|
483
|
+
# {123 => ['sweets', 'juices'], 456 => ['meat']}
|
484
|
+
data.each.with_object({}) { |(id, name), result| (result[id] ||= []).push(name) }
|
443
485
|
end
|
486
|
+
|
487
|
+
field :name
|
488
|
+
# simply use crutch-fetched data as a value:
|
489
|
+
field :category_names, value: ->(product, crutches) { crutches.categories[product.id] }
|
444
490
|
end
|
445
491
|
```
|
446
492
|
|
@@ -462,22 +508,21 @@ So Chewy Crutches™ technology is able to increase your indexing performance in
|
|
462
508
|
|
463
509
|
### Witchcraft™ technology
|
464
510
|
|
465
|
-
One more experimental technology to increase import performance. As far as you know, chewy defines value proc for every imported field in mapping, so at the import time each of this procs is executed on imported object to extract result document to import. It would be great for performance to use one huge whole-document-returning proc instead. So basically the idea or Witchcraft™ technology is to compile a single document-returning proc from the
|
511
|
+
One more experimental technology to increase import performance. As far as you know, chewy defines value proc for every imported field in mapping, so at the import time each of this procs is executed on imported object to extract result document to import. It would be great for performance to use one huge whole-document-returning proc instead. So basically the idea or Witchcraft™ technology is to compile a single document-returning proc from the index definition.
|
466
512
|
|
467
513
|
```ruby
|
468
|
-
|
469
|
-
|
470
|
-
|
471
|
-
|
472
|
-
|
473
|
-
|
474
|
-
|
475
|
-
|
476
|
-
end
|
514
|
+
index_scope Product
|
515
|
+
witchcraft!
|
516
|
+
|
517
|
+
field :title
|
518
|
+
field :tags, value: -> { tags.map(&:name) }
|
519
|
+
field :categories do
|
520
|
+
field :name, value: -> (product, category) { category.name }
|
521
|
+
field :type, value: -> (product, category, crutch) { crutch.types[category.name] }
|
477
522
|
end
|
478
523
|
```
|
479
524
|
|
480
|
-
The
|
525
|
+
The index definition above will be compiled to something close to:
|
481
526
|
|
482
527
|
```ruby
|
483
528
|
-> (object, crutches) do
|
@@ -507,7 +552,7 @@ Obviously not every type of definition might be compiled. There are some restric
|
|
507
552
|
end
|
508
553
|
```
|
509
554
|
|
510
|
-
However, it is quite possible that your
|
555
|
+
However, it is quite possible that your index definition will be supported by Witchcraft™ technology out of the box in the most of the cases.
|
511
556
|
|
512
557
|
### Raw Import
|
513
558
|
|
@@ -534,13 +579,12 @@ class LightweightProduct
|
|
534
579
|
end
|
535
580
|
end
|
536
581
|
|
537
|
-
|
538
|
-
|
539
|
-
|
540
|
-
|
582
|
+
index_scope Product
|
583
|
+
default_import_options raw_import: ->(hash) {
|
584
|
+
LightweightProduct.new(hash)
|
585
|
+
}
|
541
586
|
|
542
|
-
|
543
|
-
end
|
587
|
+
field :created_at, 'datetime'
|
544
588
|
```
|
545
589
|
|
546
590
|
Also, you can pass `:raw_import` option to the `import` method explicitly.
|
@@ -551,6 +595,24 @@ By default, when you perform import Chewy checks whether an index exists and cre
|
|
551
595
|
You can turn off this feature to decrease Elasticsearch hits count.
|
552
596
|
To do so you need to set `skip_index_creation_on_import` parameter to `false` in your `config/chewy.yml`
|
553
597
|
|
598
|
+
### Skip record fields during import
|
599
|
+
|
600
|
+
You can use `ignore_blank: true` to skip fields that return `true` for the `.blank?` method:
|
601
|
+
|
602
|
+
```ruby
|
603
|
+
index_scope Country
|
604
|
+
field :id
|
605
|
+
field :cities, ignore_blank: true do
|
606
|
+
field :id
|
607
|
+
field :name
|
608
|
+
field :surname, ignore_blank: true
|
609
|
+
field :description
|
610
|
+
end
|
611
|
+
```
|
612
|
+
|
613
|
+
#### Default values for different types
|
614
|
+
|
615
|
+
By default `ignore_blank` is false on every type except `geo_point`.
|
554
616
|
|
555
617
|
### Journaling
|
556
618
|
|
@@ -564,7 +626,6 @@ Common journal record looks like this:
|
|
564
626
|
"action": "index",
|
565
627
|
"object_id": [1, 2, 3],
|
566
628
|
"index_name": "...",
|
567
|
-
"type_name": "...",
|
568
629
|
"created_at": "<timestamp>"
|
569
630
|
}
|
570
631
|
```
|
@@ -590,9 +651,8 @@ Or as a default import option for an index:
|
|
590
651
|
|
591
652
|
```ruby
|
592
653
|
class CityIndex
|
593
|
-
|
594
|
-
|
595
|
-
end
|
654
|
+
index_scope City
|
655
|
+
default_import_options journal: true
|
596
656
|
end
|
597
657
|
```
|
598
658
|
|
@@ -600,19 +660,6 @@ You may be wondering why do you need it? The answer is simple: not to lose the d
|
|
600
660
|
|
601
661
|
Imagine that you reset your index in a zero-downtime manner (to separate index), and at the meantime somebody keeps updating the data frequently (to old index). So all these actions will be written to the journal index and you'll be able to apply them after index reset using the `Chewy::Journal` interface.
|
602
662
|
|
603
|
-
### Types access
|
604
|
-
|
605
|
-
You can access index-defined types with the following API:
|
606
|
-
|
607
|
-
```ruby
|
608
|
-
UsersIndex::User # => UsersIndex::User
|
609
|
-
UsersIndex.type_hash['user'] # => UsersIndex::User
|
610
|
-
UsersIndex.type('user') # => UsersIndex::User
|
611
|
-
UsersIndex.type('foo') # => raises error UndefinedType("Unknown type in UsersIndex: foo")
|
612
|
-
UsersIndex.types # => [UsersIndex::User]
|
613
|
-
UsersIndex.type_names # => ['user']
|
614
|
-
```
|
615
|
-
|
616
663
|
### Index manipulation
|
617
664
|
|
618
665
|
```ruby
|
@@ -625,25 +672,21 @@ UsersIndex.create! # use bang or non-bang methods
|
|
625
672
|
UsersIndex.purge
|
626
673
|
UsersIndex.purge! # deletes then creates index
|
627
674
|
|
628
|
-
UsersIndex
|
629
|
-
|
630
|
-
UsersIndex
|
631
|
-
UsersIndex
|
632
|
-
UsersIndex
|
633
|
-
UsersIndex::User.import User.where('rating > 100'), update_fields: [:email] # if update fields are specified - it will update their values only with the `update` bulk action.
|
675
|
+
UsersIndex.import # import with 0 arguments process all the data specified in index_scope definition
|
676
|
+
UsersIndex.import User.where('rating > 100') # or import specified users scope
|
677
|
+
UsersIndex.import User.where('rating > 100').to_a # or import specified users array
|
678
|
+
UsersIndex.import [1, 2, 42] # pass even ids for import, it will be handled in the most effective way
|
679
|
+
UsersIndex.import User.where('rating > 100'), update_fields: [:email] # if update fields are specified - it will update their values only with the `update` bulk action
|
634
680
|
|
635
|
-
UsersIndex.import # import every defined type
|
636
|
-
UsersIndex.import user: User.where('rating > 100') # import only active users to `user` type.
|
637
|
-
# Other index types, if exists, will be imported with default scope from the type definition.
|
638
681
|
UsersIndex.reset! # purges index and imports default data for all types
|
639
682
|
```
|
640
683
|
|
641
|
-
If the passed user is `#destroyed?`, or satisfies a `delete_if`
|
684
|
+
If the passed user is `#destroyed?`, or satisfies a `delete_if` index_scope option, or the specified id does not exist in the database, import will perform delete from index action for this object.
|
642
685
|
|
643
686
|
```ruby
|
644
|
-
|
645
|
-
|
646
|
-
|
687
|
+
index_scope User, delete_if: :deleted_at
|
688
|
+
index_scope User, delete_if: -> { deleted_at }
|
689
|
+
index_scope User, delete_if: ->(user) { user.deleted_at }
|
647
690
|
```
|
648
691
|
|
649
692
|
See [actions.rb](lib/chewy/index/actions.rb) for more details.
|
@@ -654,13 +697,12 @@ Assume you've got the following code:
|
|
654
697
|
|
655
698
|
```ruby
|
656
699
|
class City < ActiveRecord::Base
|
657
|
-
update_index 'cities
|
700
|
+
update_index 'cities', :self
|
658
701
|
end
|
659
702
|
|
660
703
|
class CitiesIndex < Chewy::Index
|
661
|
-
|
662
|
-
|
663
|
-
end
|
704
|
+
index_scope City
|
705
|
+
field :name
|
664
706
|
end
|
665
707
|
```
|
666
708
|
|
@@ -680,16 +722,6 @@ end
|
|
680
722
|
|
681
723
|
Using this strategy delays the index update request until the end of the block. Updated records are aggregated and the index update happens with the bulk API. So this strategy is highly optimized.
|
682
724
|
|
683
|
-
#### `:resque`
|
684
|
-
|
685
|
-
This does the same thing as `:atomic`, but asynchronously using resque. The default queue name is `chewy`. Patch `Chewy::Strategy::Resque::Worker` for index updates improving.
|
686
|
-
|
687
|
-
```ruby
|
688
|
-
Chewy.strategy(:resque) do
|
689
|
-
City.popular.map(&:do_some_update_action!)
|
690
|
-
end
|
691
|
-
```
|
692
|
-
|
693
725
|
#### `:sidekiq`
|
694
726
|
|
695
727
|
This does the same thing as `:atomic`, but asynchronously using sidekiq. Patch `Chewy::Strategy::Sidekiq::Worker` for index updates improving.
|
@@ -720,16 +752,6 @@ The default queue name is `chewy`, you can customize it in settings: `active_job
|
|
720
752
|
Chewy.settings[:active_job] = {queue: :low}
|
721
753
|
```
|
722
754
|
|
723
|
-
#### `:shoryuken`
|
724
|
-
|
725
|
-
This does the same thing as `:atomic`, but asynchronously using shoryuken. Patch `Chewy::Strategy::Shoryuken::Worker` for index updates improving.
|
726
|
-
|
727
|
-
```ruby
|
728
|
-
Chewy.strategy(:shoryuken) do
|
729
|
-
City.popular.map(&:do_some_update_action!)
|
730
|
-
end
|
731
|
-
```
|
732
|
-
|
733
755
|
#### `:urgent`
|
734
756
|
|
735
757
|
The following strategy is convenient if you are going to update documents in your index one by one.
|
@@ -805,6 +827,12 @@ RSpec.configure do |config|
|
|
805
827
|
end
|
806
828
|
```
|
807
829
|
|
830
|
+
### Elasticsearch client options
|
831
|
+
|
832
|
+
All connection options, except the `:prefix`, are passed to the `Elasticseach::Client.new` ([chewy/lib/chewy.rb](https://github.com/toptal/chewy/blob/f5bad9f83c21416ac10590f6f34009c645062e89/lib/chewy.rb#L153-L160)):
|
833
|
+
|
834
|
+
Here's the relevant Elasticsearch documentation on the subject: https://rubydoc.info/gems/elasticsearch-transport#setting-hosts
|
835
|
+
|
808
836
|
### `ActiveSupport::Notifications` support
|
809
837
|
|
810
838
|
Chewy has notifying the following events:
|
@@ -816,7 +844,7 @@ Chewy has notifying the following events:
|
|
816
844
|
|
817
845
|
#### `import_objects.chewy` payload
|
818
846
|
|
819
|
-
* `payload[:
|
847
|
+
* `payload[:index]`: currently imported index name
|
820
848
|
* `payload[:import]`: imports stats, total imported and deleted objects count:
|
821
849
|
|
822
850
|
```ruby
|
@@ -920,22 +948,27 @@ Quick introduction.
|
|
920
948
|
|
921
949
|
#### Composing requests
|
922
950
|
|
923
|
-
The request DSL have the same chainable nature as AR
|
951
|
+
The request DSL have the same chainable nature as AR. The main class is `Chewy::Search::Request`.
|
924
952
|
|
925
953
|
```ruby
|
926
|
-
|
927
|
-
PlaceIndex::City.query(match: {name: 'London'}) # returns cities only.
|
954
|
+
CitiesIndex.query(match: {name: 'London'})
|
928
955
|
```
|
929
956
|
|
930
|
-
Main methods of the request DSL are: `query`, `filter` and `post_filter`, it is possible to pass pure query hashes or use `elasticsearch-dsl`.
|
957
|
+
Main methods of the request DSL are: `query`, `filter` and `post_filter`, it is possible to pass pure query hashes or use `elasticsearch-dsl`.
|
931
958
|
|
932
959
|
```ruby
|
933
|
-
|
960
|
+
CitiesIndex
|
934
961
|
.filter(term: {name: 'Bangkok'})
|
935
962
|
.query { match name: 'London' }
|
936
963
|
.query.not(range: {population: {gt: 1_000_000}})
|
937
964
|
```
|
938
965
|
|
966
|
+
You can query a set of indexes at once:
|
967
|
+
|
968
|
+
```ruby
|
969
|
+
CitiesIndex.indices(CountriesIndex).query(match: {name: 'Some'})
|
970
|
+
```
|
971
|
+
|
939
972
|
See https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html and https://github.com/elastic/elasticsearch-ruby/tree/master/elasticsearch-dsl for more details.
|
940
973
|
|
941
974
|
An important part of requests manipulation is merging. There are 4 methods to perform it: `merge`, `and`, `or`, `not`. See [Chewy::Search::QueryProxy](lib/chewy/search/query_proxy.rb) for details. Also, `only` and `except` methods help to remove unneeded parts of the request.
|
@@ -943,14 +976,14 @@ An important part of requests manipulation is merging. There are 4 methods to pe
|
|
943
976
|
Every other request part is covered by a bunch of additional methods, see [Chewy::Search::Request](lib/chewy/search/request.rb) for details:
|
944
977
|
|
945
978
|
```ruby
|
946
|
-
|
979
|
+
CitiesIndex.limit(10).offset(30).order(:name, {population: {order: :desc}})
|
947
980
|
```
|
948
981
|
|
949
982
|
Request DSL also provides additional scope actions, like `delete_all`, `exists?`, `count`, `pluck`, etc.
|
950
983
|
|
951
984
|
#### Pagination
|
952
985
|
|
953
|
-
The request DSL supports pagination with `Kaminari
|
986
|
+
The request DSL supports pagination with `Kaminari`. An extension is enabled on initializtion if `Kaminari` is available. See [Chewy::Search](lib/chewy/search.rb) and [Chewy::Search::Pagination::Kaminari](lib/chewy/search/pagination/kaminari.rb) for details.
|
954
987
|
|
955
988
|
#### Named scopes
|
956
989
|
|
@@ -969,8 +1002,8 @@ See [Chewy::Search::Scrolling](lib/chewy/search/scrolling.rb) for details.
|
|
969
1002
|
It is possible to load ORM/ODM source objects with the `objects` method. To provide additional loading options use `load` method:
|
970
1003
|
|
971
1004
|
```ruby
|
972
|
-
|
973
|
-
|
1005
|
+
CitiesIndex.load(scope: -> { active }).to_a # to_a returns `Chewy::Index` wrappers.
|
1006
|
+
CitiesIndex.load(scope: -> { active }).objects # An array of AR source objects.
|
974
1007
|
```
|
975
1008
|
|
976
1009
|
See [Chewy::Search::Loader](lib/chewy/search/loader.rb) for more details.
|
@@ -978,7 +1011,7 @@ See [Chewy::Search::Loader](lib/chewy/search/loader.rb) for more details.
|
|
978
1011
|
In case when it is necessary to iterate through both of the wrappers and objects simultaneously, `object_hash` method helps a lot:
|
979
1012
|
|
980
1013
|
```ruby
|
981
|
-
scope =
|
1014
|
+
scope = CitiesIndex.load(scope: -> { active })
|
982
1015
|
scope.each do |wrapper|
|
983
1016
|
scope.object_hash[wrapper]
|
984
1017
|
end
|
@@ -995,8 +1028,8 @@ Performs zero-downtime reindexing as described [here](https://www.elastic.co/blo
|
|
995
1028
|
```bash
|
996
1029
|
rake chewy:reset # resets all the existing indices
|
997
1030
|
rake chewy:reset[users] # resets UsersIndex only
|
998
|
-
rake chewy:reset[users,
|
999
|
-
rake chewy:reset[-users,
|
1031
|
+
rake chewy:reset[users,cities] # resets UsersIndex and CitiesIndex
|
1032
|
+
rake chewy:reset[-users,cities] # resets every index in the application except specified ones
|
1000
1033
|
```
|
1001
1034
|
|
1002
1035
|
#### `chewy:upgrade`
|
@@ -1011,43 +1044,41 @@ See [Chewy::Stash::Specification](lib/chewy/stash.rb) and [Chewy::Index::Specifi
|
|
1011
1044
|
```bash
|
1012
1045
|
rake chewy:upgrade # upgrades all the existing indices
|
1013
1046
|
rake chewy:upgrade[users] # upgrades UsersIndex only
|
1014
|
-
rake chewy:upgrade[users,
|
1015
|
-
rake chewy:upgrade[-users,
|
1047
|
+
rake chewy:upgrade[users,cities] # upgrades UsersIndex and CitiesIndex
|
1048
|
+
rake chewy:upgrade[-users,cities] # upgrades every index in the application except specified ones
|
1016
1049
|
```
|
1017
1050
|
|
1018
1051
|
#### `chewy:update`
|
1019
1052
|
|
1020
1053
|
It doesn't create indexes, it simply imports everything to the existing ones and fails if the index was not created before.
|
1021
1054
|
|
1022
|
-
Unlike `reset` or `upgrade` tasks, it is possible to pass type references to update the particular type. In index name is passed without the type specified, it will update all the types defined for this index.
|
1023
|
-
|
1024
1055
|
```bash
|
1025
1056
|
rake chewy:update # updates all the existing indices
|
1026
1057
|
rake chewy:update[users] # updates UsersIndex only
|
1027
|
-
rake chewy:update[users,
|
1028
|
-
rake chewy:update[-users,
|
1058
|
+
rake chewy:update[users,cities] # updates UsersIndex and CitiesIndex
|
1059
|
+
rake chewy:update[-users,cities] # updates every index in the application except UsersIndex and CitiesIndex
|
1029
1060
|
```
|
1030
1061
|
|
1031
1062
|
#### `chewy:sync`
|
1032
1063
|
|
1033
|
-
Provides a way to synchronize outdated indexes with the source quickly and without doing a full reset. By default field `updated_at` is used to find outdated records, but this could be customized by `outdated_sync_field` as described at [Chewy::
|
1064
|
+
Provides a way to synchronize outdated indexes with the source quickly and without doing a full reset. By default field `updated_at` is used to find outdated records, but this could be customized by `outdated_sync_field` as described at [Chewy::Index::Syncer](lib/chewy/index/syncer.rb).
|
1034
1065
|
|
1035
|
-
Arguments are similar to the ones taken by `chewy:update` task.
|
1066
|
+
Arguments are similar to the ones taken by `chewy:update` task.
|
1036
1067
|
|
1037
|
-
See [Chewy::
|
1068
|
+
See [Chewy::Index::Syncer](lib/chewy/index/syncer.rb) for more details.
|
1038
1069
|
|
1039
1070
|
```bash
|
1040
1071
|
rake chewy:sync # synchronizes all the existing indices
|
1041
1072
|
rake chewy:sync[users] # synchronizes UsersIndex only
|
1042
|
-
rake chewy:sync[users,
|
1043
|
-
rake chewy:sync[-users,
|
1073
|
+
rake chewy:sync[users,cities] # synchronizes UsersIndex and CitiesIndex
|
1074
|
+
rake chewy:sync[-users,cities] # synchronizes every index in the application except except UsersIndex and CitiesIndex
|
1044
1075
|
```
|
1045
1076
|
|
1046
1077
|
#### `chewy:deploy`
|
1047
1078
|
|
1048
1079
|
This rake task is especially useful during the production deploy. It is a combination of `chewy:upgrade` and `chewy:sync` and the latter is called only for the indexes that were not reset during the first stage.
|
1049
1080
|
|
1050
|
-
It is not possible to specify any particular
|
1081
|
+
It is not possible to specify any particular indexes for this task as it doesn't make much sense.
|
1051
1082
|
|
1052
1083
|
Right now the approach is that if some data had been updated, but index definition was not changed (no changes satisfying the synchronization algorithm were done), it would be much faster to perform manual partial index update inside data migrations or even manually after the deploy.
|
1053
1084
|
|
@@ -1064,14 +1095,14 @@ If the number of processes is not specified explicitly - `parallel` gem tries to
|
|
1064
1095
|
```bash
|
1065
1096
|
rake chewy:parallel:reset
|
1066
1097
|
rake chewy:parallel:upgrade[4]
|
1067
|
-
rake chewy:parallel:update[4,
|
1098
|
+
rake chewy:parallel:update[4,cities]
|
1068
1099
|
rake chewy:parallel:sync[4,-users]
|
1069
1100
|
rake chewy:parallel:deploy[4] # performs parallel upgrade and parallel sync afterwards
|
1070
1101
|
```
|
1071
1102
|
|
1072
1103
|
#### `chewy:journal`
|
1073
1104
|
|
1074
|
-
This namespace contains two tasks for the journal manipulations: `chewy:journal:apply` and `chewy:journal:clean`. Both are taking time as the first argument (optional for clean) and a list of indexes
|
1105
|
+
This namespace contains two tasks for the journal manipulations: `chewy:journal:apply` and `chewy:journal:clean`. Both are taking time as the first argument (optional for clean) and a list of indexes exactly as the tasks above. Time can be in any format parsable by ActiveSupport.
|
1075
1106
|
|
1076
1107
|
```bash
|
1077
1108
|
rake chewy:journal:apply["$(date -v-1H -u +%FT%TZ)"] # apply journaled changes for the past hour
|
@@ -1080,7 +1111,16 @@ rake chewy:journal:apply["$(date -v-1H -u +%FT%TZ)",users] # apply journaled cha
|
|
1080
1111
|
|
1081
1112
|
### RSpec integration
|
1082
1113
|
|
1083
|
-
Just add `require 'chewy/rspec'` to your spec_helper.rb and you will get additional features:
|
1114
|
+
Just add `require 'chewy/rspec'` to your spec_helper.rb and you will get additional features:
|
1115
|
+
|
1116
|
+
[update_index](lib/chewy/rspec/update_index.rb) helper
|
1117
|
+
`mock_elasticsearch_response` helper to mock elasticsearch response
|
1118
|
+
`mock_elasticsearch_response_sources` helper to mock elasticsearch response sources
|
1119
|
+
`build_query` matcher to compare request and expected query (returns `true`/`false`)
|
1120
|
+
|
1121
|
+
To use `mock_elasticsearch_response` and `mock_elasticsearch_response_sources` helpers add `include Chewy::Rspec::Helpers` to your tests.
|
1122
|
+
|
1123
|
+
See [chewy/rspec/](lib/chewy/rspec/) for more details.
|
1084
1124
|
|
1085
1125
|
### Minitest integration
|
1086
1126
|
|
@@ -1090,6 +1130,14 @@ Since you can set `:bypass` strategy for test suites and manually handle import
|
|
1090
1130
|
|
1091
1131
|
But if you require chewy to index/update model regularly in your test suite then you can specify `:urgent` strategy for documents indexing. Add `Chewy.strategy(:urgent)` to test_helper.rb.
|
1092
1132
|
|
1133
|
+
Also, you can use additional helpers:
|
1134
|
+
|
1135
|
+
`mock_elasticsearch_response` to mock elasticsearch response
|
1136
|
+
`mock_elasticsearch_response_sources` to mock elasticsearch response sources
|
1137
|
+
`assert_elasticsearch_query` to compare request and expected query (returns `true`/`false`)
|
1138
|
+
|
1139
|
+
See [chewy/minitest/](lib/chewy/minitest/) for more details.
|
1140
|
+
|
1093
1141
|
### DatabaseCleaner
|
1094
1142
|
|
1095
1143
|
If you use `DatabaseCleaner` in your tests with [the `transaction` strategy](https://github.com/DatabaseCleaner/database_cleaner#how-to-use), you may run into the problem that `ActiveRecord`'s models are not indexed automatically on save despite the fact that you set the callbacks to do this with the `update_index` method. The issue arises because `chewy` indices data on `after_commit` run as default, but all `after_commit` callbacks are not run with the `DatabaseCleaner`'s' `transaction` strategy. You can solve this issue by changing the `Chewy.use_after_commit_callbacks` option. Just add the following initializer in your Rails application:
|
@@ -1108,7 +1156,7 @@ Chewy.use_after_commit_callbacks = !Rails.env.test?
|
|
1108
1156
|
5. Push to the branch (`git push origin my-new-feature`)
|
1109
1157
|
6. Create new Pull Request
|
1110
1158
|
|
1111
|
-
Use the following Rake tasks to control the Elasticsearch cluster while developing
|
1159
|
+
Use the following Rake tasks to control the Elasticsearch cluster while developing, if you prefer native Elasticsearch installation over the dockerized one:
|
1112
1160
|
|
1113
1161
|
```bash
|
1114
1162
|
rake elasticsearch:start # start Elasticsearch cluster on 9250 port for tests
|