chewy 7.1.0 → 7.2.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (131) hide show
  1. checksums.yaml +4 -4
  2. data/.github/workflows/ruby.yml +58 -0
  3. data/.rubocop.yml +13 -8
  4. data/.rubocop_todo.yml +110 -22
  5. data/CHANGELOG.md +53 -0
  6. data/Gemfile +0 -7
  7. data/Guardfile +3 -1
  8. data/README.md +282 -245
  9. data/chewy.gemspec +3 -5
  10. data/gemfiles/rails.5.2.activerecord.gemfile +8 -14
  11. data/gemfiles/rails.6.0.activerecord.gemfile +8 -14
  12. data/gemfiles/rails.6.1.activerecord.gemfile +8 -14
  13. data/lib/chewy.rb +21 -75
  14. data/lib/chewy/config.rb +40 -40
  15. data/lib/chewy/errors.rb +0 -12
  16. data/lib/chewy/fields/base.rb +11 -1
  17. data/lib/chewy/fields/root.rb +3 -4
  18. data/lib/chewy/index.rb +46 -87
  19. data/lib/chewy/index/actions.rb +51 -32
  20. data/lib/chewy/{type → index}/adapter/active_record.rb +12 -3
  21. data/lib/chewy/{type → index}/adapter/base.rb +2 -3
  22. data/lib/chewy/{type → index}/adapter/object.rb +27 -31
  23. data/lib/chewy/{type → index}/adapter/orm.rb +11 -14
  24. data/lib/chewy/{type → index}/crutch.rb +5 -5
  25. data/lib/chewy/{type → index}/import.rb +36 -27
  26. data/lib/chewy/{type → index}/import/bulk_builder.rb +15 -13
  27. data/lib/chewy/{type → index}/import/bulk_request.rb +6 -6
  28. data/lib/chewy/{type → index}/import/journal_builder.rb +10 -10
  29. data/lib/chewy/{type → index}/import/routine.rb +15 -14
  30. data/lib/chewy/{type → index}/mapping.rb +26 -31
  31. data/lib/chewy/{type → index}/observe.rb +9 -19
  32. data/lib/chewy/index/specification.rb +1 -0
  33. data/lib/chewy/{type → index}/syncer.rb +60 -57
  34. data/lib/chewy/{type → index}/witchcraft.rb +11 -7
  35. data/lib/chewy/{type → index}/wrapper.rb +2 -2
  36. data/lib/chewy/journal.rb +8 -8
  37. data/lib/chewy/minitest/helpers.rb +9 -13
  38. data/lib/chewy/minitest/search_index_receiver.rb +22 -26
  39. data/lib/chewy/railtie.rb +4 -2
  40. data/lib/chewy/rake_helper.rb +82 -107
  41. data/lib/chewy/rspec/update_index.rb +47 -43
  42. data/lib/chewy/search.rb +4 -17
  43. data/lib/chewy/search/loader.rb +18 -30
  44. data/lib/chewy/search/parameters.rb +4 -2
  45. data/lib/chewy/search/parameters/concerns/query_storage.rb +2 -2
  46. data/lib/chewy/search/parameters/source.rb +5 -1
  47. data/lib/chewy/search/query_proxy.rb +9 -2
  48. data/lib/chewy/search/request.rb +82 -86
  49. data/lib/chewy/search/response.rb +4 -4
  50. data/lib/chewy/search/scoping.rb +6 -7
  51. data/lib/chewy/search/scrolling.rb +11 -11
  52. data/lib/chewy/stash.rb +14 -22
  53. data/lib/chewy/strategy.rb +3 -19
  54. data/lib/chewy/strategy/sidekiq.rb +1 -0
  55. data/lib/chewy/version.rb +1 -1
  56. data/lib/generators/chewy/install_generator.rb +1 -1
  57. data/lib/tasks/chewy.rake +10 -22
  58. data/migration_guide.md +14 -0
  59. data/spec/chewy/config_spec.rb +14 -39
  60. data/spec/chewy/fields/base_spec.rb +412 -148
  61. data/spec/chewy/fields/root_spec.rb +16 -24
  62. data/spec/chewy/fields/time_fields_spec.rb +5 -5
  63. data/spec/chewy/index/actions_spec.rb +270 -24
  64. data/spec/chewy/{type → index}/adapter/active_record_spec.rb +68 -40
  65. data/spec/chewy/{type → index}/adapter/object_spec.rb +21 -6
  66. data/spec/chewy/{type → index}/import/bulk_builder_spec.rb +23 -31
  67. data/spec/chewy/{type → index}/import/bulk_request_spec.rb +5 -6
  68. data/spec/chewy/{type → index}/import/journal_builder_spec.rb +9 -15
  69. data/spec/chewy/{type → index}/import/routine_spec.rb +16 -16
  70. data/spec/chewy/{type → index}/import_spec.rb +102 -98
  71. data/spec/chewy/{type → index}/mapping_spec.rb +46 -54
  72. data/spec/chewy/index/observe_spec.rb +116 -0
  73. data/spec/chewy/index/settings_spec.rb +3 -1
  74. data/spec/chewy/index/specification_spec.rb +7 -17
  75. data/spec/chewy/{type → index}/syncer_spec.rb +14 -19
  76. data/spec/chewy/{type → index}/witchcraft_spec.rb +20 -22
  77. data/spec/chewy/index/wrapper_spec.rb +100 -0
  78. data/spec/chewy/index_spec.rb +59 -102
  79. data/spec/chewy/journal_spec.rb +9 -22
  80. data/spec/chewy/minitest/helpers_spec.rb +13 -15
  81. data/spec/chewy/minitest/search_index_receiver_spec.rb +22 -26
  82. data/spec/chewy/multi_search_spec.rb +4 -5
  83. data/spec/chewy/rake_helper_spec.rb +145 -55
  84. data/spec/chewy/rspec/update_index_spec.rb +74 -71
  85. data/spec/chewy/search/loader_spec.rb +19 -37
  86. data/spec/chewy/search/pagination/kaminari_examples.rb +3 -5
  87. data/spec/chewy/search/pagination/kaminari_spec.rb +1 -1
  88. data/spec/chewy/search/parameters/indices_spec.rb +2 -8
  89. data/spec/chewy/search/parameters/order_spec.rb +1 -1
  90. data/spec/chewy/search/parameters/query_storage_examples.rb +67 -21
  91. data/spec/chewy/search/parameters/search_after_spec.rb +4 -1
  92. data/spec/chewy/search/parameters/source_spec.rb +8 -2
  93. data/spec/chewy/search/parameters_spec.rb +12 -3
  94. data/spec/chewy/search/query_proxy_spec.rb +68 -17
  95. data/spec/chewy/search/request_spec.rb +222 -74
  96. data/spec/chewy/search/response_spec.rb +12 -12
  97. data/spec/chewy/search/scrolling_spec.rb +7 -9
  98. data/spec/chewy/search_spec.rb +32 -35
  99. data/spec/chewy/stash_spec.rb +9 -21
  100. data/spec/chewy/strategy/active_job_spec.rb +8 -8
  101. data/spec/chewy/strategy/atomic_spec.rb +9 -10
  102. data/spec/chewy/strategy/sidekiq_spec.rb +8 -8
  103. data/spec/chewy/strategy_spec.rb +19 -15
  104. data/spec/chewy_spec.rb +14 -100
  105. data/spec/spec_helper.rb +2 -21
  106. data/spec/support/active_record.rb +15 -5
  107. metadata +44 -103
  108. data/.circleci/config.yml +0 -214
  109. data/Appraisals +0 -81
  110. data/gemfiles/rails.5.2.mongoid.6.4.gemfile +0 -17
  111. data/gemfiles/sequel.4.45.gemfile +0 -11
  112. data/lib/chewy/search/pagination/will_paginate.rb +0 -43
  113. data/lib/chewy/strategy/resque.rb +0 -27
  114. data/lib/chewy/strategy/shoryuken.rb +0 -40
  115. data/lib/chewy/type.rb +0 -120
  116. data/lib/chewy/type/actions.rb +0 -43
  117. data/lib/chewy/type/adapter/mongoid.rb +0 -67
  118. data/lib/chewy/type/adapter/sequel.rb +0 -93
  119. data/lib/sequel/plugins/chewy_observe.rb +0 -63
  120. data/spec/chewy/search/pagination/will_paginate_examples.rb +0 -63
  121. data/spec/chewy/search/pagination/will_paginate_spec.rb +0 -23
  122. data/spec/chewy/strategy/resque_spec.rb +0 -46
  123. data/spec/chewy/strategy/shoryuken_spec.rb +0 -70
  124. data/spec/chewy/type/actions_spec.rb +0 -50
  125. data/spec/chewy/type/adapter/mongoid_spec.rb +0 -372
  126. data/spec/chewy/type/adapter/sequel_spec.rb +0 -472
  127. data/spec/chewy/type/observe_spec.rb +0 -137
  128. data/spec/chewy/type/wrapper_spec.rb +0 -100
  129. data/spec/chewy/type_spec.rb +0 -55
  130. data/spec/support/mongoid.rb +0 -93
  131. data/spec/support/sequel.rb +0 -80
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 58a5011ecef1ca511a28d4d3299d61f852c4c11ffe6fd124db3e0a5b370a0f9e
4
- data.tar.gz: fa0ce61841d579210c2de4a84624b265cca533f887e6125d8e07cee59315c4a6
3
+ metadata.gz: a958ecfd3273f1e8086b3c07aa823f3db8e5d2300a0ca2a86a563e1d6bd2df42
4
+ data.tar.gz: 8195d2162307dbf368865cc5e5975372e169796aa85a5a6bd8ac404484b99d75
5
5
  SHA512:
6
- metadata.gz: '0925c5bb8f3adfc79b5914d975c71f2677d6232ccbac074d65ee63d99d23cfbc16c85adf88c953e003fbebfcff2fc9b0bb5a11d735ce6432cdcd57d76e7f81e4'
7
- data.tar.gz: fefd812c6daea408e90b6f235efd169b195b5d1003fe4d3042c2e502b375d0b7cdf6d4d41133d012bf05a0f0e417ce95e43fbd1eb8e99b9ba3d72f6a4b50e548
6
+ metadata.gz: 7b3286e50679ce8b43c9a595d90502cbd028ac8e94c93e240a3f820de15fbc19052887f1bf973670421bcc8a2e1cfe9dd4d9b2183cb0ccd828726220df005c7c
7
+ data.tar.gz: 00a24e51c43cd0e191af699949ca3ac1ca738777fd3a22dfe1d97d7d55feb6ebfea29e2577d488ff1f97189e1430266110395afea3efcb949917d07a6c26477f
@@ -0,0 +1,58 @@
1
+ name: CI
2
+
3
+ on: [push]
4
+
5
+ jobs:
6
+ tests:
7
+ runs-on: ubuntu-latest
8
+ strategy:
9
+ fail-fast: false
10
+ matrix:
11
+ ruby: [2.6, 2.7]
12
+ gemfile: [rails.5.2.activerecord, rails.6.0.activerecord, rails.6.1.activerecord]
13
+ name: ${{ matrix.ruby }}-${{ matrix.gemfile }}
14
+
15
+ env:
16
+ BUNDLE_GEMFILE: gemfiles/${{ matrix.gemfile }}.gemfile
17
+
18
+ steps:
19
+ - uses: actions/checkout@v2
20
+ - uses: ruby/setup-ruby@v1
21
+ with:
22
+ ruby-version: ${{ matrix.ruby }}
23
+ bundler-cache: true
24
+ - name: Run Elasticsearch
25
+ uses: elastic/elastic-github-actions/elasticsearch@master
26
+ with:
27
+ stack-version: 7.10.1
28
+ port: 9250
29
+ - name: Tests
30
+ run: bundle exec rspec
31
+
32
+ ruby-3-0-activerecord-6-1:
33
+ runs-on: ubuntu-latest
34
+ env:
35
+ BUNDLE_GEMFILE: gemfiles/rails.6.1.activerecord.gemfile
36
+ steps:
37
+ - uses: actions/checkout@v2
38
+ - uses: ruby/setup-ruby@v1
39
+ with:
40
+ ruby-version: '3.0'
41
+ bundler-cache: true
42
+ - name: Run Elasticsearch
43
+ uses: elastic/elastic-github-actions/elasticsearch@master
44
+ with:
45
+ stack-version: 7.10.1
46
+ port: 9250
47
+ - name: Tests
48
+ run: bundle exec rspec
49
+
50
+ rubocop:
51
+ runs-on: ubuntu-latest
52
+ steps:
53
+ - uses: actions/checkout@v2
54
+ - uses: ruby/setup-ruby@v1
55
+ with:
56
+ ruby-version: 2.7
57
+ bundler-cache: true
58
+ - run: bundle exec rubocop --format simple
data/.rubocop.yml CHANGED
@@ -1,24 +1,31 @@
1
1
  inherit_from: .rubocop_todo.yml
2
2
 
3
+ AllCops:
4
+ NewCops: enable
5
+ TargetRubyVersion: 2.6
6
+
3
7
  Layout/AccessModifierIndentation:
4
8
  EnforcedStyle: outdent
5
9
 
6
- Layout/AlignHash:
10
+ Layout/HashAlignment:
7
11
  EnforcedLastArgumentHashStyle: always_ignore
8
12
 
9
- Layout/AlignParameters:
13
+ Layout/ParameterAlignment:
10
14
  EnforcedStyle: with_fixed_indentation
11
15
 
12
16
  Layout/CaseIndentation:
13
17
  EnforcedStyle: end
14
18
 
15
- Layout/IndentArray:
19
+ Layout/EndAlignment:
20
+ EnforcedStyleAlignWith: variable
21
+
22
+ Layout/FirstArrayElementIndentation:
16
23
  EnforcedStyle: consistent
17
24
 
18
- Layout/IndentHash:
25
+ Layout/FirstHashElementIndentation:
19
26
  EnforcedStyle: consistent
20
27
 
21
- Layout/IndentHeredoc:
28
+ Layout/HeredocIndentation:
22
29
  Enabled: false
23
30
 
24
31
  Layout/MultilineMethodCallIndentation:
@@ -33,9 +40,6 @@ Layout/SpaceInsideHashLiteralBraces:
33
40
  Lint/AmbiguousBlockAssociation:
34
41
  Enabled: false
35
42
 
36
- Lint/EndAlignment:
37
- EnforcedStyleAlignWith: variable
38
-
39
43
  Style/Alias:
40
44
  EnforcedStyle: prefer_alias_method
41
45
 
@@ -53,4 +57,5 @@ Metrics/BlockLength:
53
57
 
54
58
  Metrics/ModuleLength:
55
59
  Exclude:
60
+ - 'lib/chewy/rake_helper.rb'
56
61
  - '**/*_spec.rb'
data/.rubocop_todo.yml CHANGED
@@ -1,44 +1,132 @@
1
1
  # This configuration was generated by
2
2
  # `rubocop --auto-gen-config`
3
- # on 2016-09-09 17:05:47 +0700 using RuboCop version 0.42.0.
3
+ # on 2021-04-02 12:44:05 UTC using RuboCop version 1.11.0.
4
4
  # The point is for the user to remove these configuration records
5
5
  # one by one as the offenses are removed from the code base.
6
6
  # Note that changes in the inspected code, or installation of new
7
7
  # versions of RuboCop, may require this file to be generated again.
8
8
 
9
- # Offense count: 42
9
+ # Offense count: 1
10
+ # Configuration parameters: Include.
11
+ # Include: **/*.gemspec
12
+ Gemspec/RequiredRubyVersion:
13
+ Exclude:
14
+ - 'chewy.gemspec'
15
+
16
+ # Offense count: 2
17
+ # Configuration parameters: AllowedMethods.
18
+ # AllowedMethods: enums
19
+ Lint/ConstantDefinitionInBlock:
20
+ Exclude:
21
+ - 'lib/chewy/rspec/update_index.rb'
22
+ - 'spec/chewy/config_spec.rb'
23
+
24
+ # Offense count: 10
25
+ # Configuration parameters: AllowComments, AllowEmptyLambdas.
26
+ Lint/EmptyBlock:
27
+ Exclude:
28
+ - 'spec/chewy/minitest/helpers_spec.rb'
29
+ - 'spec/chewy/rspec/update_index_spec.rb'
30
+ - 'spec/chewy/search/scrolling_spec.rb'
31
+ - 'spec/chewy/strategy/atomic_spec.rb'
32
+ - 'spec/chewy/strategy_spec.rb'
33
+ - 'spec/chewy/index/import/bulk_request_spec.rb'
34
+ - 'spec/chewy/index/witchcraft_spec.rb'
35
+ - 'spec/chewy_spec.rb'
36
+
37
+ # Offense count: 3
38
+ Lint/MissingSuper:
39
+ Exclude:
40
+ - 'lib/chewy/strategy/atomic.rb'
41
+ - 'lib/chewy/index/adapter/object.rb'
42
+ - 'lib/chewy/index/adapter/orm.rb'
43
+
44
+ # Offense count: 35
45
+ # Configuration parameters: IgnoredMethods, CountRepeatedAttributes.
10
46
  Metrics/AbcSize:
11
- Max: 48
47
+ Max: 41
12
48
 
13
49
  # Offense count: 4
14
- # Configuration parameters: CountComments.
50
+ # Configuration parameters: CountComments, CountAsOne.
15
51
  Metrics/ClassLength:
16
- Max: 300
52
+ Max: 267
17
53
 
18
- # Offense count: 14
54
+ # Offense count: 13
55
+ # Configuration parameters: IgnoredMethods.
19
56
  Metrics/CyclomaticComplexity:
20
- Max: 13
21
-
22
- # Offense count: 1450
23
- # Configuration parameters: AllowHeredoc, AllowURI, URISchemes.
24
- # URISchemes: http, https
25
- Metrics/LineLength:
26
- Max: 198
57
+ Max: 12
27
58
 
28
- # Offense count: 38
29
- # Configuration parameters: CountComments.
59
+ # Offense count: 43
60
+ # Configuration parameters: CountComments, CountAsOne, ExcludedMethods, IgnoredMethods.
30
61
  Metrics/MethodLength:
31
- Max: 33
62
+ Max: 29
32
63
 
33
- # Offense count: 1
34
- # Configuration parameters: CountComments.
64
+ # Offense count: 2
65
+ # Configuration parameters: CountComments, CountAsOne.
35
66
  Metrics/ModuleLength:
36
- Max: 180
67
+ Max: 158
37
68
 
38
- # Offense count: 14
69
+ # Offense count: 18
70
+ # Configuration parameters: IgnoredMethods.
39
71
  Metrics/PerceivedComplexity:
40
- Max: 15
72
+ Max: 13
41
73
 
42
- # Offense count: 93
74
+ # Offense count: 11
75
+ # Configuration parameters: EnforcedStyle, CheckMethodNames, CheckSymbols, AllowedIdentifiers.
76
+ # SupportedStyles: snake_case, normalcase, non_integer
77
+ # AllowedIdentifiers: capture3, iso8601, rfc1123_date, rfc822, rfc2822, rfc3339
78
+ Naming/VariableNumber:
79
+ Exclude:
80
+ - 'spec/chewy/fields/root_spec.rb'
81
+
82
+ # Offense count: 5
83
+ Style/DocumentDynamicEvalDefinition:
84
+ Exclude:
85
+ - 'lib/chewy/index/actions.rb'
86
+ - 'lib/chewy/repository.rb'
87
+ - 'lib/chewy/search/pagination/kaminari.rb'
88
+ - 'lib/chewy/index/crutch.rb'
89
+ - 'lib/chewy/index/witchcraft.rb'
90
+
91
+ # Offense count: 58
43
92
  Style/Documentation:
44
93
  Enabled: false
94
+
95
+ # Offense count: 2
96
+ # Cop supports --auto-correct.
97
+ Style/EvalWithLocation:
98
+ Exclude:
99
+ - 'spec/chewy/index_spec.rb'
100
+
101
+ # Offense count: 191
102
+ # Cop supports --auto-correct.
103
+ # Configuration parameters: EnforcedStyle.
104
+ # SupportedStyles: always, always_true, never
105
+ Style/FrozenStringLiteralComment:
106
+ Enabled: false
107
+
108
+ # Offense count: 2
109
+ # Configuration parameters: MinBodyLength.
110
+ Style/GuardClause:
111
+ Exclude:
112
+ - 'lib/chewy.rb'
113
+ - 'spec/support/active_record.rb'
114
+
115
+ # Offense count: 10
116
+ # Cop supports --auto-correct.
117
+ Style/IfUnlessModifier:
118
+ Exclude:
119
+ - 'lib/chewy.rb'
120
+ - 'lib/chewy/railtie.rb'
121
+ - 'lib/chewy/rspec/update_index.rb'
122
+ - 'lib/chewy/search/query_proxy.rb'
123
+ - 'lib/chewy/index/import.rb'
124
+ - 'lib/chewy/index/witchcraft.rb'
125
+ - 'spec/support/active_record.rb'
126
+
127
+ # Offense count: 53
128
+ # Cop supports --auto-correct.
129
+ # Configuration parameters: AutoCorrect, AllowHeredoc, AllowURI, URISchemes, IgnoreCopDirectives, IgnoredPatterns.
130
+ # URISchemes: http, https
131
+ Layout/LineLength:
132
+ Max: 191
data/CHANGELOG.md CHANGED
@@ -8,6 +8,58 @@
8
8
 
9
9
  ### Bugs Fixed
10
10
 
11
+ ## 7.2.0 (2021-04-19)
12
+
13
+ ### New Features
14
+
15
+ * [#778](https://github.com/toptal/chewy/pull/778): Add `ignore_blank` option to `field` method ([@Vitalina-Vakulchyk][]):
16
+ * `true` by default for the `geo_point` type
17
+ * `false` by default for other types
18
+
19
+ ### Changes
20
+
21
+ * [#783](https://github.com/toptal/chewy/pull/783): **(Breaking)** Remove `Chewy::Type`, simplify DSL ([@rabotyaga][])
22
+ * Remove the `Chewy::Type` class
23
+ * e.g. remove `CitiesIndex::City` / `CitiesIndex.city`
24
+ * `CitiesIndex::City.import! ...` becomes `CitiesIndex.import! ...`
25
+ * Simplify index DSL:
26
+ * `define_type` block -> `index_scope` clause
27
+ * it can be omitted completely, if you don't need to specify the scope or options, e.g. `name`
28
+ * Remove type names from string representations:
29
+ * in `update_index` ActiveRecord helper and RSpec matcher, e.g.
30
+ * `update_index('cities#city')` -> `update_index('cities')`
31
+ * `update_index(UsersIndex::User)` -> `update_index(UsersIndex)`
32
+ * in rake tasks (e.g. `rake chewy:update[cities#city]` -> `rake chewy:update[cities]`)
33
+ * in rake tasks output (e.g. `Imported CitiesIndex::City in 1s, stats: index 3` -> `Imported CitiesIndex in 1s, stats: index 3`)
34
+ * Use index name instead of type name in loader additional scope
35
+ * e.g. `CitiesIndex.filter(...).load(city: {scope: City.where(...)})` -> `CitiesIndex.filter(...).load(cities: {scope: City.where(...)})`
36
+ * [#692](https://github.com/toptal/chewy/issues/692): Add `.update_mapping` to Index class ([@Vitalina-Vakulchyk][]):
37
+ * Wrapped Elasticsearch gem `.put_mapping` with `.update_mapping` in Index class
38
+ * Add `rake chewy:update_mapping` task
39
+ * [#594](https://github.com/toptal/chewy/issues/594): Add `.reindex` to Index class ([@Vitalina-Vakulchyk][]):
40
+ * Wrapped Elasticsearch gem `.reindex` with `.reindex` in Index class
41
+ * Add `rake chewy:reindex` task
42
+ * [#679](https://github.com/toptal/chewy/issues/679): Wrapped `Elasticsearch::API::Indices::Actions#clear_cache` with `.clear_cache` in Index class ([@Vitalina-Vakulchyk][])
43
+ * [#495](https://github.com/toptal/chewy/issues/495): Ability to change Rails console strategy with `Chewy.console_strategy` ([@Vitalina-Vakulchyk][])
44
+ * [#778](https://github.com/toptal/chewy/pull/778): **(Breaking)** Drop support for Ruby 2.5 ([@Vitalina-Vakulchyk][])
45
+ * [#776](https://github.com/toptal/chewy/pull/776): **(Breaking)** Removal of unnecessary features and integrations ([@Vitalina-Vakulchyk][]):
46
+ * `aws-sdk-sqs` / `shoryuken`
47
+ * `mongoid`
48
+ * `sequel`
49
+ * `will_paginate`
50
+ * `resque`
51
+ * [#769](https://github.com/toptal/chewy/pull/769): **(Breaking)** Removal of deprecated methods and rake tasks ([@Vitalina-Vakulchyk][]):
52
+ * `Chewy::Index.index_params` is removed, use `Chewy::Index.specification_hash` instead
53
+ * `Chewy::Index.derivable_index_name` is removed, use `Chewy::Index.derivable_name` instead
54
+ * `Chewy::Index.default_prefix` is removed, use `Chewy::Index.prefix` instead
55
+ * `Chewy::Index.build_index_name` is removed, use `Chewy::Index.index_name` instead
56
+ * `Chewy::RakeHelper.reset_index` is removed, use `Chewy::RakeHelper.reset` instead
57
+ * `Chewy::RakeHelper.reset_all` is removed, use `Chewy::RakeHelper.reset` instead
58
+ * `Chewy::RakeHelper.update_index` is removed, use `Chewy::RakeHelper.update` instead
59
+ * `Chewy::RakeHelper.update_all` is removed, use `Chewy::RakeHelper.update` instead
60
+ * `rake chewy:apply_changes_from` is removed, use `rake chewy:journal:apply` instead
61
+ * `rake chewy:clean_journal` is removed, use `rake chewy:journal:clean` instead
62
+
11
63
  ## 7.1.0 (2021-03-03)
12
64
 
13
65
  ### Changes
@@ -607,5 +659,6 @@
607
659
  [@taylor-au]: https://github.com/taylor-au
608
660
  [@TikiTDO]: https://github.com/TikiTDO
609
661
  [@undr]: https://github.com/undr
662
+ [@Vitalina-Vakulchyk]: https://github.com/Vitalina-Vakulchyk
610
663
  [@webgago]: https://github.com/webgago
611
664
  [@yahooguntu]: https://github.com/yahooguntu
data/Gemfile CHANGED
@@ -3,18 +3,11 @@ source 'https://rubygems.org'
3
3
  gemspec
4
4
 
5
5
  gem 'activerecord'
6
- # gem 'mongoid'
7
- # gem 'sequel'
8
6
 
9
7
  gem 'activejob', require: false
10
- gem 'resque', require: false
11
8
  gem 'sidekiq', require: false
12
9
 
13
- gem 'aws-sdk-sqs', require: false
14
- gem 'shoryuken', require: false
15
-
16
10
  gem 'kaminari-core', require: false
17
- gem 'will_paginate', require: false
18
11
 
19
12
  gem 'parallel', require: false
20
13
  gem 'ruby-progressbar', require: false
data/Guardfile CHANGED
@@ -9,7 +9,9 @@ guard :rspec, cmd: 'rspec' do
9
9
  # Rails example
10
10
  watch(%r{^app/(.+)\.rb$}) { |m| "spec/#{m[1]}_spec.rb" }
11
11
  watch(%r{^app/(.*)(\.erb|\.haml|\.slim)$}) { |m| "spec/#{m[1]}#{m[2]}_spec.rb" }
12
- watch(%r{^app/controllers/(.+)_(controller)\.rb$}) { |m| ["spec/routing/#{m[1]}_routing_spec.rb", "spec/#{m[2]}s/#{m[1]}_#{m[2]}_spec.rb", "spec/acceptance/#{m[1]}_spec.rb"] }
12
+ watch(%r{^app/controllers/(.+)_(controller)\.rb$}) do |m|
13
+ ["spec/routing/#{m[1]}_routing_spec.rb", "spec/#{m[2]}s/#{m[1]}_#{m[2]}_spec.rb", "spec/acceptance/#{m[1]}_spec.rb"]
14
+ end
13
15
  watch(%r{^spec/support/(.+)\.rb$}) { 'spec' }
14
16
  watch('config/routes.rb') { 'spec/routing' }
15
17
  watch('app/controllers/application_controller.rb') { 'spec/controllers' }
data/README.md CHANGED
@@ -1,5 +1,5 @@
1
1
  [![Gem Version](https://badge.fury.io/rb/chewy.svg)](http://badge.fury.io/rb/chewy)
2
- [![CircleCI](https://circleci.com/gh/toptal/chewy/tree/master.svg?style=svg)](https://circleci.com/gh/toptal/chewy/tree/master)
2
+ [![GitHub Actions](https://github.com/toptal/chewy/actions/workflows/ruby.yml/badge.svg)](https://github.com/toptal/chewy/actions/workflows/ruby.yml)
3
3
  [![Code Climate](https://codeclimate.com/github/toptal/chewy.svg)](https://codeclimate.com/github/toptal/chewy)
4
4
  [![Inline docs](http://inch-ci.org/github/toptal/chewy.svg?branch=master)](http://inch-ci.org/github/toptal/chewy)
5
5
 
@@ -7,49 +7,6 @@
7
7
 
8
8
  Chewy is an ODM (Object Document Mapper), built on top of the [the official Elasticsearch client](https://github.com/elastic/elasticsearch-ruby).
9
9
 
10
- ## Table of Contents
11
-
12
- * [Why Chewy?](#why-chewy)
13
- * [Installation](#installation)
14
- * [Usage](#usage)
15
- * [Client settings](#client-settings)
16
- * [AWS ElasticSearch configuration](#aws-elastic-search)
17
- * [Index definition](#index-definition)
18
- * [Type default import options](#type-default-import-options)
19
- * [Multi (nested) and object field types](#multi-nested-and-object-field-types)
20
- * [Geo Point fields](#geo-point-fields)
21
- * [Crutches™ technology](#crutches-technology)
22
- * [Witchcraft™ technology](#witchcraft-technology)
23
- * [Raw Import](#raw-import)
24
- * [Index creation during import](#index-creation-during-import)
25
- * [Journaling](#journaling)
26
- * [Types access](#types-access)
27
- * [Index manipulation](#index-manipulation)
28
- * [Index update strategies](#index-update-strategies)
29
- * [Nesting](#nesting)
30
- * [Non-block notation](#non-block-notation)
31
- * [Designing your own strategies](#designing-your-own-strategies)
32
- * [Rails application strategies integration](#rails-application-strategies-integration)
33
- * [ActiveSupport::Notifications support](#activesupportnotifications-support)
34
- * [NewRelic integration](#newrelic-integration)
35
- * [Search requests](#search-requests)
36
- * [Composing requests](#composing-requests)
37
- * [Pagination](#pagination)
38
- * [Named scopes](#named-scopes)
39
- * [Scroll API](#scroll-api)
40
- * [Loading objects](#loading-objects)
41
- * [Rake tasks](#rake-tasks)
42
- * [chewy:reset](#chewyreset)
43
- * [chewy:upgrade](#chewyupgrade)
44
- * [chewy:update](#chewyupdate)
45
- * [chewy:sync](#chewysync)
46
- * [chewy:deploy](#chewydeploy)
47
- * [Parallelizing rake tasks](#parallelizing-rake-tasks)
48
- * [chewy:journal](#chewyjournal)
49
- * [RSpec integration](#rspec-integration)
50
- * [Minitest integration](#minitest-integration)
51
- * [Contributing](#contributing)
52
-
53
10
  ## Why Chewy?
54
11
 
55
12
  In this section we'll cover why you might want to use Chewy instead of the official `elasticsearch-ruby` client gem.
@@ -66,7 +23,7 @@ In this section we'll cover why you might want to use Chewy instead of the offic
66
23
 
67
24
  Chewy has an ActiveRecord-style query DSL. It is chainable, mergeable and lazy, so you can produce queries in the most efficient way. It also has object-oriented query and filter builders.
68
25
 
69
- * Support for ActiveRecord, [Mongoid](https://github.com/mongoid/mongoid) and [Sequel](https://github.com/jeremyevans/sequel).
26
+ * Support for ActiveRecord.
70
27
 
71
28
  ## Installation
72
29
 
@@ -86,7 +43,7 @@ Or install it yourself as:
86
43
 
87
44
  ### Ruby
88
45
 
89
- Chewy is compatible with MRI 2.5-3.0¹.
46
+ Chewy is compatible with MRI 2.6-3.0¹.
90
47
 
91
48
  > ¹ Ruby 3 is only supported with Rails 6.1
92
49
 
@@ -94,6 +51,7 @@ Chewy is compatible with MRI 2.5-3.0¹.
94
51
 
95
52
  | Chewy version | Elasticsearch version |
96
53
  | ------------- | ---------------------------------- |
54
+ | 7.2.x | 7.x |
97
55
  | 7.1.x | 7.x |
98
56
  | 7.0.0 | 6.8, 7.x |
99
57
  | 6.0.0 | 5.x, 6.x |
@@ -101,23 +59,156 @@ Chewy is compatible with MRI 2.5-3.0¹.
101
59
 
102
60
  See [Migration guide](migration_guide.md).
103
61
 
104
- ## Usage
62
+ ### Active Record
105
63
 
106
- ### Client settings
64
+ 5.2, 6.0, 6.1 Active Record versions are supported by all Chewy versions.
65
+
66
+ ## Getting Started
67
+
68
+ Chewy provides functionality for Elasticsearch index handling, documents import mappings, index update strategies and chainable query DSL.
69
+
70
+ ### Minimal client setting
71
+
72
+ Create `config/initializers/chewy.rb` with this line:
73
+
74
+ ```ruby
75
+ Chewy.settings = {host: 'localhost:9250'}
76
+ ```
77
+
78
+ And run `rails g chewy:install` to generate `chewy.yml`:
79
+
80
+ ```yaml
81
+ # config/chewy.yml
82
+ # separate environment configs
83
+ test:
84
+ host: 'localhost:9250'
85
+ prefix: 'test'
86
+ development:
87
+ host: 'localhost:9200'
88
+ ```
107
89
 
108
- There are two ways to configure the Chewy client:
90
+ ### Elasticsearch
109
91
 
110
- * via the hash `Chewy.settings`
111
- * via the configuration file `chewy.yml`
92
+ Make sure you have Elasticsearch up and running. You can [install](https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html) it locally, but the easiest way is to use [Docker](https://www.docker.com/get-started):
93
+
94
+ ```shell
95
+ $ docker run --rm --name elasticsearch -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" elasticsearch:7.11.1
96
+ ```
112
97
 
113
- You can create `chewy.yml` manually or run `rails g chewy:install` to
114
- generate it.
98
+ ### Index
99
+
100
+ Create `app/chewy/user_index.rb` with User Index:
101
+
102
+ ```ruby
103
+ class UsersIndex < Chewy::Index
104
+ settings analysis: {
105
+ analyzer: {
106
+ email: {
107
+ tokenizer: 'keyword',
108
+ filter: ['lowercase']
109
+ }
110
+ }
111
+ }
112
+
113
+ index_scope User
114
+ field :first_name
115
+ field :last_name
116
+ field :email, analyzer: 'email'
117
+ end
118
+ ```
119
+
120
+ ### Model
121
+
122
+ Add User model, table and migrate it:
123
+
124
+ ```shell
125
+ $ bundle exec rails g model User first_name last_name email
126
+ $ bundle exec rails db:migrate
127
+ ```
128
+
129
+ Add `update_index` to app/models/user.rb:
130
+
131
+ ```ruby
132
+ class User < ApplicationRecord
133
+ update_index('users') { self }
134
+ end
135
+ ```
136
+
137
+ ### Example of data request
138
+
139
+ 1. Once a record is created (could be done via the Rails console), it creates User index too:
140
+
141
+ ```
142
+ User.create(
143
+ first_name: "test1",
144
+ last_name: "test1",
145
+ email: 'test1@example.com',
146
+ # other fields
147
+ )
148
+ # UsersIndex Import (355.3ms) {:index=>1}
149
+ # => #<User id: 1, first_name: "test1", last_name: "test1", email: "test1@example.com", # other fields>
150
+ ```
151
+
152
+ 2. A query could be exposed at a given `UsersController`:
153
+
154
+ ```ruby
155
+ def search
156
+ @users = UsersIndex.query(query_string: { fields: [:first_name, :last_name, :email, ...], query: search_params[:query], default_operator: 'and' })
157
+ render json: @users.to_json, status: :ok
158
+ end
159
+
160
+ private
161
+
162
+ def search_params
163
+ params.permit(:query, :page, :per)
164
+ end
165
+ ```
166
+
167
+ 3. So a request against `http://localhost:3000/users/search?query=test1@example.com` issuing a response like:
168
+
169
+ ```json
170
+ [
171
+ {
172
+ "attributes":{
173
+ "id":"1",
174
+ "first_name":"test1",
175
+ "last_name":"test1",
176
+ "email":"test1@example.com",
177
+ ...
178
+ "_score":0.9808291,
179
+ "_explanation":null
180
+ },
181
+ "_data":{
182
+ "_index":"users",
183
+ "_type":"_doc",
184
+ "_id":"1",
185
+ "_score":0.9808291,
186
+ "_source":{
187
+ "first_name":"test1",
188
+ "last_name":"test1",
189
+ "email":"test1@example.com",
190
+ ...
191
+ }
192
+ }
193
+ }
194
+ ]
195
+ ```
196
+
197
+ ## Usage and configuration
198
+
199
+ ### Client settings
200
+
201
+ To configure the Chewy client you need to add `chewy.rb` file with `Chewy.settings` hash:
115
202
 
116
203
  ```ruby
117
204
  # config/initializers/chewy.rb
118
205
  Chewy.settings = {host: 'localhost:9250'} # do not use environments
119
206
  ```
120
207
 
208
+ And add `chewy.yml` configuration file.
209
+
210
+ You can create `chewy.yml` manually or run `rails g chewy:install` to generate it:
211
+
121
212
  ```yaml
122
213
  # config/chewy.yml
123
214
  # separate environment configs
@@ -167,7 +258,7 @@ Chewy.settings = {
167
258
  }
168
259
  ```
169
260
 
170
- ### Index definition
261
+ #### Index definition
171
262
 
172
263
  1. Create `/app/chewy/users_index.rb`
173
264
 
@@ -177,41 +268,38 @@ Chewy.settings = {
177
268
  end
178
269
  ```
179
270
 
180
- 2. Add one or more types mapping
271
+ 2. Define index scope (you can omit this part if you don't need to specify a scope (i.e. use PORO objects for import) or options)
181
272
 
182
273
  ```ruby
183
274
  class UsersIndex < Chewy::Index
184
- define_type User.active # or just model instead_of scope: define_type User
275
+ index_scope User.active # or just model instead_of scope: index_scope User
185
276
  end
186
277
  ```
187
278
 
188
- Newly-defined index type class is accessible via `UsersIndex.user` or `UsersIndex::User`
189
-
190
- 3. Add some type mappings
279
+ 3. Add some mappings
191
280
 
192
281
  ```ruby
193
282
  class UsersIndex < Chewy::Index
194
- define_type User.active.includes(:country, :badges, :projects) do
195
- field :first_name, :last_name # multiple fields without additional options
196
- field :email, analyzer: 'email' # Elasticsearch-related options
197
- field :country, value: ->(user) { user.country.name } # custom value proc
198
- field :badges, value: ->(user) { user.badges.map(&:name) } # passing array values to index
199
- field :projects do # the same block syntax for multi_field, if `:type` is specified
200
- field :title
201
- field :description # default data type is `text`
202
- # additional top-level objects passed to value proc:
203
- field :categories, value: ->(project, user) { project.categories.map(&:name) if user.active? }
204
- end
205
- field :rating, type: 'integer' # custom data type
206
- field :created, type: 'date', include_in_all: false,
207
- value: ->{ created_at } # value proc for source object context
283
+ index_scope User.active.includes(:country, :badges, :projects)
284
+ field :first_name, :last_name # multiple fields without additional options
285
+ field :email, analyzer: 'email' # Elasticsearch-related options
286
+ field :country, value: ->(user) { user.country.name } # custom value proc
287
+ field :badges, value: ->(user) { user.badges.map(&:name) } # passing array values to index
288
+ field :projects do # the same block syntax for multi_field, if `:type` is specified
289
+ field :title
290
+ field :description # default data type is `text`
291
+ # additional top-level objects passed to value proc:
292
+ field :categories, value: ->(project, user) { project.categories.map(&:name) if user.active? }
208
293
  end
294
+ field :rating, type: 'integer' # custom data type
295
+ field :created, type: 'date', include_in_all: false,
296
+ value: ->{ created_at } # value proc for source object context
209
297
  end
210
298
  ```
211
299
 
212
300
  [See here for mapping definitions](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html).
213
301
 
214
- 4. Add some index- and type-related settings. Analyzer repositories might be used as well. See `Chewy::Index.settings` docs for details:
302
+ 4. Add some index-related settings. Analyzer repositories might be used as well. See `Chewy::Index.settings` docs for details:
215
303
 
216
304
  ```ruby
217
305
  class UsersIndex < Chewy::Index
@@ -224,23 +312,22 @@ Chewy.settings = {
224
312
  }
225
313
  }
226
314
 
227
- define_type User.active.includes(:country, :badges, :projects) do
228
- root date_detection: false do
229
- template 'about_translations.*', type: 'text', analyzer: 'standard'
230
-
231
- field :first_name, :last_name
232
- field :email, analyzer: 'email'
233
- field :country, value: ->(user) { user.country.name }
234
- field :badges, value: ->(user) { user.badges.map(&:name) }
235
- field :projects do
236
- field :title
237
- field :description
238
- end
239
- field :about_translations, type: 'object' # pass object type explicitly if necessary
240
- field :rating, type: 'integer'
241
- field :created, type: 'date', include_in_all: false,
242
- value: ->{ created_at }
315
+ index_scope User.active.includes(:country, :badges, :projects)
316
+ root date_detection: false do
317
+ template 'about_translations.*', type: 'text', analyzer: 'standard'
318
+
319
+ field :first_name, :last_name
320
+ field :email, analyzer: 'email'
321
+ field :country, value: ->(user) { user.country.name }
322
+ field :badges, value: ->(user) { user.badges.map(&:name) }
323
+ field :projects do
324
+ field :title
325
+ field :description
243
326
  end
327
+ field :about_translations, type: 'object' # pass object type explicitly if necessary
328
+ field :rating, type: 'integer'
329
+ field :created, type: 'date', include_in_all: false,
330
+ value: ->{ created_at }
244
331
  end
245
332
  end
246
333
  ```
@@ -254,39 +341,32 @@ Chewy.settings = {
254
341
 
255
342
  ```ruby
256
343
  class User < ActiveRecord::Base
257
- update_index('users#user') { self } # specifying index, type and back-reference
344
+ update_index('users') { self } # specifying index and back-reference
258
345
  # for updating after user save or destroy
259
346
  end
260
347
 
261
348
  class Country < ActiveRecord::Base
262
349
  has_many :users
263
350
 
264
- update_index('users#user') { users } # return single object or collection
351
+ update_index('users') { users } # return single object or collection
265
352
  end
266
353
 
267
354
  class Project < ActiveRecord::Base
268
- update_index('users#user') { user if user.active? } # you can return even `nil` from the back-reference
269
- end
270
-
271
- class Badge < ActiveRecord::Base
272
- has_and_belongs_to_many :users
273
-
274
- update_index('users') { users } # if index has only one type
275
- # there is no need to specify updated type
355
+ update_index('users') { user if user.active? } # you can return even `nil` from the back-reference
276
356
  end
277
357
 
278
358
  class Book < ActiveRecord::Base
279
- update_index(->(book) {"books#book_#{book.language}"}) { self } # dynamic index and type with proc.
280
- # For book with language == "en"
281
- # this code will generate `books#book_en`
359
+ update_index(->(book) {"books_#{book.language}"}) { self } # dynamic index name with proc.
360
+ # For book with language == "en"
361
+ # this code will generate `books_en`
282
362
  end
283
363
  ```
284
364
 
285
365
  Also, you can use the second argument for method name passing:
286
366
 
287
367
  ```ruby
288
- update_index('users#user', :self)
289
- update_index('users#user', :users)
368
+ update_index('users', :self)
369
+ update_index('users', :users)
290
370
  ```
291
371
 
292
372
  In the case of a belongs_to association you may need to update both associated objects, previous and current:
@@ -295,47 +375,28 @@ Chewy.settings = {
295
375
  class City < ActiveRecord::Base
296
376
  belongs_to :country
297
377
 
298
- update_index('cities#city') { self }
299
- update_index 'countries#country' do
300
- # For the latest active_record changed values are
301
- # already in `previous_changes` hash,
302
- # but for mongoid you have to use `changes` hash
378
+ update_index('cities') { self }
379
+ update_index 'countries' do
303
380
  previous_changes['country_id'] || country
304
381
  end
305
382
  end
306
383
  ```
307
384
 
308
- You can observe Sequel models in the same way as ActiveRecord:
309
-
310
- ```ruby
311
- class User < Sequel::Model
312
- update_index('users#user') { self }
313
- end
314
- ```
315
-
316
- However, to make it work, you must load the chewy plugin into Sequel model:
385
+ ### Default import options
317
386
 
318
- ```ruby
319
- Sequel::Model.plugin :chewy_observe # for all models, or...
320
- User.plugin :chewy_observe # just for User
321
- ```
322
-
323
- ### Type default import options
324
-
325
- Every type has `default_import_options` configuration to specify, suddenly, default import options:
387
+ Every index has `default_import_options` configuration to specify, suddenly, default import options:
326
388
 
327
389
  ```ruby
328
390
  class ProductsIndex < Chewy::Index
329
- define_type Post.includes(:tags) do
330
- default_import_options batch_size: 100, bulk_size: 10.megabytes, refresh: false
391
+ index_scope Post.includes(:tags)
392
+ default_import_options batch_size: 100, bulk_size: 10.megabytes, refresh: false
331
393
 
332
- field :name
333
- field :tags, value: -> { tags.map(&:name) }
334
- end
394
+ field :name
395
+ field :tags, value: -> { tags.map(&:name) }
335
396
  end
336
397
  ```
337
398
 
338
- See [import.rb](lib/chewy/type/import.rb) for available options.
399
+ See [import.rb](lib/chewy/index/import.rb) for available options.
339
400
 
340
401
  ### Multi (nested) and object field types
341
402
 
@@ -386,14 +447,13 @@ Assume you are defining your index like this (product has_many categories throug
386
447
 
387
448
  ```ruby
388
449
  class ProductsIndex < Chewy::Index
389
- define_type Product.includes(:categories) do
390
- field :name
391
- field :category_names, value: ->(product) { product.categories.map(&:name) } # or shorter just -> { categories.map(&:name) }
392
- end
450
+ index_scope Product.includes(:categories)
451
+ field :name
452
+ field :category_names, value: ->(product) { product.categories.map(&:name) } # or shorter just -> { categories.map(&:name) }
393
453
  end
394
454
  ```
395
455
 
396
- Then the Chewy reindexing flow will look like the following pseudo-code (even in Mongoid):
456
+ Then the Chewy reindexing flow will look like the following pseudo-code:
397
457
 
398
458
  ```ruby
399
459
  Product.includes(:categories).find_in_batches(1000) do |batch|
@@ -405,26 +465,23 @@ Product.includes(:categories).find_in_batches(1000) do |batch|
405
465
  end
406
466
  ```
407
467
 
408
- But in Rails 4.1 and 4.2 you may face a problem with slow associations (take a look at https://github.com/rails/rails/pull/19423). Also, there might be really complicated cases when associations are not applicable.
409
-
410
- Then you can replace Rails associations with Chewy Crutches™ technology:
468
+ If you meet complicated cases when associations are not applicable you can replace Rails associations with Chewy Crutches™ technology:
411
469
 
412
470
  ```ruby
413
471
  class ProductsIndex < Chewy::Index
414
- define_type Product do
415
- crutch :categories do |collection| # collection here is a current batch of products
416
- # data is fetched with a lightweight query without objects initialization
417
- data = ProductCategory.joins(:category).where(product_id: collection.map(&:id)).pluck(:product_id, 'categories.name')
418
- # then we have to convert fetched data to appropriate format
419
- # this will return our data in structure like:
420
- # {123 => ['sweets', 'juices'], 456 => ['meat']}
421
- data.each.with_object({}) { |(id, name), result| (result[id] ||= []).push(name) }
422
- end
423
-
424
- field :name
425
- # simply use crutch-fetched data as a value:
426
- field :category_names, value: ->(product, crutches) { crutches.categories[product.id] }
472
+ index_scope Product
473
+ crutch :categories do |collection| # collection here is a current batch of products
474
+ # data is fetched with a lightweight query without objects initialization
475
+ data = ProductCategory.joins(:category).where(product_id: collection.map(&:id)).pluck(:product_id, 'categories.name')
476
+ # then we have to convert fetched data to appropriate format
477
+ # this will return our data in structure like:
478
+ # {123 => ['sweets', 'juices'], 456 => ['meat']}
479
+ data.each.with_object({}) { |(id, name), result| (result[id] ||= []).push(name) }
427
480
  end
481
+
482
+ field :name
483
+ # simply use crutch-fetched data as a value:
484
+ field :category_names, value: ->(product, crutches) { crutches.categories[product.id] }
428
485
  end
429
486
  ```
430
487
 
@@ -446,22 +503,21 @@ So Chewy Crutches™ technology is able to increase your indexing performance in
446
503
 
447
504
  ### Witchcraft™ technology
448
505
 
449
- One more experimental technology to increase import performance. As far as you know, chewy defines value proc for every imported field in mapping, so at the import time each of this procs is executed on imported object to extract result document to import. It would be great for performance to use one huge whole-document-returning proc instead. So basically the idea or Witchcraft™ technology is to compile a single document-returning proc from the type definition.
506
+ One more experimental technology to increase import performance. As far as you know, chewy defines value proc for every imported field in mapping, so at the import time each of this procs is executed on imported object to extract result document to import. It would be great for performance to use one huge whole-document-returning proc instead. So basically the idea or Witchcraft™ technology is to compile a single document-returning proc from the index definition.
450
507
 
451
508
  ```ruby
452
- define_type Product do
453
- witchcraft!
454
-
455
- field :title
456
- field :tags, value: -> { tags.map(&:name) }
457
- field :categories do
458
- field :name, value: -> (product, category) { category.name }
459
- field :type, value: -> (product, category, crutch) { crutch.types[category.name] }
460
- end
509
+ index_scope Product
510
+ witchcraft!
511
+
512
+ field :title
513
+ field :tags, value: -> { tags.map(&:name) }
514
+ field :categories do
515
+ field :name, value: -> (product, category) { category.name }
516
+ field :type, value: -> (product, category, crutch) { crutch.types[category.name] }
461
517
  end
462
518
  ```
463
519
 
464
- The type definition above will be compiled to something close to:
520
+ The index definition above will be compiled to something close to:
465
521
 
466
522
  ```ruby
467
523
  -> (object, crutches) do
@@ -491,7 +547,7 @@ Obviously not every type of definition might be compiled. There are some restric
491
547
  end
492
548
  ```
493
549
 
494
- However, it is quite possible that your type definition will be supported by Witchcraft™ technology out of the box in the most of the cases.
550
+ However, it is quite possible that your index definition will be supported by Witchcraft™ technology out of the box in the most of the cases.
495
551
 
496
552
  ### Raw Import
497
553
 
@@ -518,13 +574,12 @@ class LightweightProduct
518
574
  end
519
575
  end
520
576
 
521
- define_type Product do
522
- default_import_options raw_import: ->(hash) {
523
- LightweightProduct.new(hash)
524
- }
577
+ index_scope Product
578
+ default_import_options raw_import: ->(hash) {
579
+ LightweightProduct.new(hash)
580
+ }
525
581
 
526
- field :created_at, 'datetime'
527
- end
582
+ field :created_at, 'datetime'
528
583
  ```
529
584
 
530
585
  Also, you can pass `:raw_import` option to the `import` method explicitly.
@@ -535,6 +590,24 @@ By default, when you perform import Chewy checks whether an index exists and cre
535
590
  You can turn off this feature to decrease Elasticsearch hits count.
536
591
  To do so you need to set `skip_index_creation_on_import` parameter to `false` in your `config/chewy.yml`
537
592
 
593
+ ### Skip record fields during import
594
+
595
+ You can use `ignore_blank: true` to skip fields that return `true` for the `.blank?` method:
596
+
597
+ ```ruby
598
+ index_scope Country
599
+ field :id
600
+ field :cities, ignore_blank: true do
601
+ field :id
602
+ field :name
603
+ field :surname, ignore_blank: true
604
+ field :description
605
+ end
606
+ ```
607
+
608
+ #### Default values for different types
609
+
610
+ By default `ignore_blank` is false on every type except `geo_point`.
538
611
 
539
612
  ### Journaling
540
613
 
@@ -548,7 +621,6 @@ Common journal record looks like this:
548
621
  "action": "index",
549
622
  "object_id": [1, 2, 3],
550
623
  "index_name": "...",
551
- "type_name": "...",
552
624
  "created_at": "<timestamp>"
553
625
  }
554
626
  ```
@@ -574,9 +646,8 @@ Or as a default import option for an index:
574
646
 
575
647
  ```ruby
576
648
  class CityIndex
577
- define_type City do
578
- default_import_options journal: true
579
- end
649
+ index_scope City
650
+ default_import_options journal: true
580
651
  end
581
652
  ```
582
653
 
@@ -584,19 +655,6 @@ You may be wondering why do you need it? The answer is simple: not to lose the d
584
655
 
585
656
  Imagine that you reset your index in a zero-downtime manner (to separate index), and at the meantime somebody keeps updating the data frequently (to old index). So all these actions will be written to the journal index and you'll be able to apply them after index reset using the `Chewy::Journal` interface.
586
657
 
587
- ### Types access
588
-
589
- You can access index-defined type with the following API:
590
-
591
- ```ruby
592
- UsersIndex::User # => UsersIndex::User
593
- UsersIndex.type_hash['user'] # => UsersIndex::User
594
- UsersIndex.type('user') # => UsersIndex::User
595
- UsersIndex.type('foo') # => raises error UndefinedType("Unknown type in UsersIndex: foo")
596
- UsersIndex.types # => [UsersIndex::User]
597
- UsersIndex.type_names # => ['user']
598
- ```
599
-
600
658
  ### Index manipulation
601
659
 
602
660
  ```ruby
@@ -609,24 +667,21 @@ UsersIndex.create! # use bang or non-bang methods
609
667
  UsersIndex.purge
610
668
  UsersIndex.purge! # deletes then creates index
611
669
 
612
- UsersIndex::User.import # import with 0 arguments process all the data specified in type definition
613
- # literally, User.active.includes(:country, :badges, :projects).find_in_batches
614
- UsersIndex::User.import User.where('rating > 100') # or import specified users scope
615
- UsersIndex::User.import User.where('rating > 100').to_a # or import specified users array
616
- UsersIndex::User.import [1, 2, 42] # pass even ids for import, it will be handled in the most effective way
617
- UsersIndex::User.import User.where('rating > 100'), update_fields: [:email] # if update fields are specified - it will update their values only with the `update` bulk action
670
+ UsersIndex.import # import with 0 arguments process all the data specified in index_scope definition
671
+ UsersIndex.import User.where('rating > 100') # or import specified users scope
672
+ UsersIndex.import User.where('rating > 100').to_a # or import specified users array
673
+ UsersIndex.import [1, 2, 42] # pass even ids for import, it will be handled in the most effective way
674
+ UsersIndex.import User.where('rating > 100'), update_fields: [:email] # if update fields are specified - it will update their values only with the `update` bulk action
618
675
 
619
- UsersIndex.import # same as UsersIndex::User.import
620
- UsersIndex.import user: User.where('rating > 100') # import only specified users
621
676
  UsersIndex.reset! # purges index and imports default data for all types
622
677
  ```
623
678
 
624
- If the passed user is `#destroyed?`, or satisfies a `delete_if` type option, or the specified id does not exist in the database, import will perform delete from index action for this object.
679
+ If the passed user is `#destroyed?`, or satisfies a `delete_if` index_scope option, or the specified id does not exist in the database, import will perform delete from index action for this object.
625
680
 
626
681
  ```ruby
627
- define_type User, delete_if: :deleted_at
628
- define_type User, delete_if: -> { deleted_at }
629
- define_type User, delete_if: ->(user) { user.deleted_at }
682
+ index_scope User, delete_if: :deleted_at
683
+ index_scope User, delete_if: -> { deleted_at }
684
+ index_scope User, delete_if: ->(user) { user.deleted_at }
630
685
  ```
631
686
 
632
687
  See [actions.rb](lib/chewy/index/actions.rb) for more details.
@@ -637,13 +692,12 @@ Assume you've got the following code:
637
692
 
638
693
  ```ruby
639
694
  class City < ActiveRecord::Base
640
- update_index 'cities#city', :self
695
+ update_index 'cities', :self
641
696
  end
642
697
 
643
698
  class CitiesIndex < Chewy::Index
644
- define_type City do
645
- field :name
646
- end
699
+ index_scope City
700
+ field :name
647
701
  end
648
702
  ```
649
703
 
@@ -663,16 +717,6 @@ end
663
717
 
664
718
  Using this strategy delays the index update request until the end of the block. Updated records are aggregated and the index update happens with the bulk API. So this strategy is highly optimized.
665
719
 
666
- #### `:resque`
667
-
668
- This does the same thing as `:atomic`, but asynchronously using resque. The default queue name is `chewy`. Patch `Chewy::Strategy::Resque::Worker` for index updates improving.
669
-
670
- ```ruby
671
- Chewy.strategy(:resque) do
672
- City.popular.map(&:do_some_update_action!)
673
- end
674
- ```
675
-
676
720
  #### `:sidekiq`
677
721
 
678
722
  This does the same thing as `:atomic`, but asynchronously using sidekiq. Patch `Chewy::Strategy::Sidekiq::Worker` for index updates improving.
@@ -703,16 +747,6 @@ The default queue name is `chewy`, you can customize it in settings: `active_job
703
747
  Chewy.settings[:active_job] = {queue: :low}
704
748
  ```
705
749
 
706
- #### `:shoryuken`
707
-
708
- This does the same thing as `:atomic`, but asynchronously using shoryuken. Patch `Chewy::Strategy::Shoryuken::Worker` for index updates improving.
709
-
710
- ```ruby
711
- Chewy.strategy(:shoryuken) do
712
- City.popular.map(&:do_some_update_action!)
713
- end
714
- ```
715
-
716
750
  #### `:urgent`
717
751
 
718
752
  The following strategy is convenient if you are going to update documents in your index one by one.
@@ -788,6 +822,12 @@ RSpec.configure do |config|
788
822
  end
789
823
  ```
790
824
 
825
+ ### Elasticsearch client options
826
+
827
+ All connection options, except the `:prefix`, are passed to the `Elasticseach::Client.new` ([chewy/lib/chewy.rb](https://github.com/toptal/chewy/blob/f5bad9f83c21416ac10590f6f34009c645062e89/lib/chewy.rb#L153-L160)):
828
+
829
+ Here's the relevant Elasticsearch documentation on the subject: https://rubydoc.info/gems/elasticsearch-transport#setting-hosts
830
+
791
831
  ### `ActiveSupport::Notifications` support
792
832
 
793
833
  Chewy has notifying the following events:
@@ -799,7 +839,7 @@ Chewy has notifying the following events:
799
839
 
800
840
  #### `import_objects.chewy` payload
801
841
 
802
- * `payload[:type]`: currently imported type
842
+ * `payload[:index]`: currently imported index name
803
843
  * `payload[:import]`: imports stats, total imported and deleted objects count:
804
844
 
805
845
  ```ruby
@@ -903,14 +943,13 @@ Quick introduction.
903
943
 
904
944
  #### Composing requests
905
945
 
906
- The request DSL have the same chainable nature as AR or Mongoid ones. The main class is `Chewy::Search::Request`. It is possible to perform requests on behalf of indices or types:
946
+ The request DSL have the same chainable nature as AR. The main class is `Chewy::Search::Request`.
907
947
 
908
948
  ```ruby
909
- CitiesIndex.query(match: {name: 'London'}) # or
910
- CitiesIndex::City.query(match: {name: 'London'})
949
+ CitiesIndex.query(match: {name: 'London'})
911
950
  ```
912
951
 
913
- Main methods of the request DSL are: `query`, `filter` and `post_filter`, it is possible to pass pure query hashes or use `elasticsearch-dsl`. Also, there is an additional
952
+ Main methods of the request DSL are: `query`, `filter` and `post_filter`, it is possible to pass pure query hashes or use `elasticsearch-dsl`.
914
953
 
915
954
  ```ruby
916
955
  CitiesIndex
@@ -939,7 +978,7 @@ Request DSL also provides additional scope actions, like `delete_all`, `exists?`
939
978
 
940
979
  #### Pagination
941
980
 
942
- The request DSL supports pagination with `Kaminari` and `WillPaginate`. An appropriate extension is enabled on initializtion if any of libraries is available. See [Chewy::Search](lib/chewy/search.rb) and [Chewy::Search::Pagination](lib/chewy/search/pagination/) namespace for details.
981
+ The request DSL supports pagination with `Kaminari`. An extension is enabled on initializtion if `Kaminari` is available. See [Chewy::Search](lib/chewy/search.rb) and [Chewy::Search::Pagination::Kaminari](lib/chewy/search/pagination/kaminari.rb) for details.
943
982
 
944
983
  #### Named scopes
945
984
 
@@ -958,7 +997,7 @@ See [Chewy::Search::Scrolling](lib/chewy/search/scrolling.rb) for details.
958
997
  It is possible to load ORM/ODM source objects with the `objects` method. To provide additional loading options use `load` method:
959
998
 
960
999
  ```ruby
961
- CitiesIndex.load(scope: -> { active }).to_a # to_a returns `Chewy::Type` wrappers.
1000
+ CitiesIndex.load(scope: -> { active }).to_a # to_a returns `Chewy::Index` wrappers.
962
1001
  CitiesIndex.load(scope: -> { active }).objects # An array of AR source objects.
963
1002
  ```
964
1003
 
@@ -1008,35 +1047,33 @@ rake chewy:upgrade[-users,cities] # upgrades every index in the application exce
1008
1047
 
1009
1048
  It doesn't create indexes, it simply imports everything to the existing ones and fails if the index was not created before.
1010
1049
 
1011
- Unlike `reset` or `upgrade` tasks, it is possible to pass type references to update the particular type. In index name is passed without the type specified, it will update all the types defined for this index.
1012
-
1013
1050
  ```bash
1014
1051
  rake chewy:update # updates all the existing indices
1015
1052
  rake chewy:update[users] # updates UsersIndex only
1016
- rake chewy:update[users,cities#city] # updates UsersIndex and CitiesIndex (if City type is defined on CitiesIndex)
1017
- rake chewy:update[-users,cities#city] # updates every index in the application except UsersIndex and CitiesIndex::City
1053
+ rake chewy:update[users,cities] # updates UsersIndex and CitiesIndex
1054
+ rake chewy:update[-users,cities] # updates every index in the application except UsersIndex and CitiesIndex
1018
1055
  ```
1019
1056
 
1020
1057
  #### `chewy:sync`
1021
1058
 
1022
- Provides a way to synchronize outdated indexes with the source quickly and without doing a full reset. By default field `updated_at` is used to find outdated records, but this could be customized by `outdated_sync_field` as described at [Chewy::Type::Syncer](lib/chewy/type/syncer.rb).
1059
+ Provides a way to synchronize outdated indexes with the source quickly and without doing a full reset. By default field `updated_at` is used to find outdated records, but this could be customized by `outdated_sync_field` as described at [Chewy::Index::Syncer](lib/chewy/index/syncer.rb).
1023
1060
 
1024
- Arguments are similar to the ones taken by `chewy:update` task. It is possible to specify a particular type or a whole index.
1061
+ Arguments are similar to the ones taken by `chewy:update` task.
1025
1062
 
1026
- See [Chewy::Type::Syncer](lib/chewy/type/syncer.rb) for more details.
1063
+ See [Chewy::Index::Syncer](lib/chewy/index/syncer.rb) for more details.
1027
1064
 
1028
1065
  ```bash
1029
1066
  rake chewy:sync # synchronizes all the existing indices
1030
1067
  rake chewy:sync[users] # synchronizes UsersIndex only
1031
- rake chewy:sync[users,cities#city] # synchronizes UsersIndex and CitiesIndex (if City type is defined on CitiesIndex)
1032
- rake chewy:sync[-users,cities#city] # synchronizes every index in the application except except UsersIndex and CitiesIndex::City
1068
+ rake chewy:sync[users,cities] # synchronizes UsersIndex and CitiesIndex
1069
+ rake chewy:sync[-users,cities] # synchronizes every index in the application except except UsersIndex and CitiesIndex
1033
1070
  ```
1034
1071
 
1035
1072
  #### `chewy:deploy`
1036
1073
 
1037
1074
  This rake task is especially useful during the production deploy. It is a combination of `chewy:upgrade` and `chewy:sync` and the latter is called only for the indexes that were not reset during the first stage.
1038
1075
 
1039
- It is not possible to specify any particular types/indexes for this task as it doesn't make much sense.
1076
+ It is not possible to specify any particular indexes for this task as it doesn't make much sense.
1040
1077
 
1041
1078
  Right now the approach is that if some data had been updated, but index definition was not changed (no changes satisfying the synchronization algorithm were done), it would be much faster to perform manual partial index update inside data migrations or even manually after the deploy.
1042
1079
 
@@ -1053,14 +1090,14 @@ If the number of processes is not specified explicitly - `parallel` gem tries to
1053
1090
  ```bash
1054
1091
  rake chewy:parallel:reset
1055
1092
  rake chewy:parallel:upgrade[4]
1056
- rake chewy:parallel:update[4,cities#city]
1093
+ rake chewy:parallel:update[4,cities]
1057
1094
  rake chewy:parallel:sync[4,-users]
1058
1095
  rake chewy:parallel:deploy[4] # performs parallel upgrade and parallel sync afterwards
1059
1096
  ```
1060
1097
 
1061
1098
  #### `chewy:journal`
1062
1099
 
1063
- This namespace contains two tasks for the journal manipulations: `chewy:journal:apply` and `chewy:journal:clean`. Both are taking time as the first argument (optional for clean) and a list of indexes/types exactly as the tasks above. Time can be in any format parsable by ActiveSupport.
1100
+ This namespace contains two tasks for the journal manipulations: `chewy:journal:apply` and `chewy:journal:clean`. Both are taking time as the first argument (optional for clean) and a list of indexes exactly as the tasks above. Time can be in any format parsable by ActiveSupport.
1064
1101
 
1065
1102
  ```bash
1066
1103
  rake chewy:journal:apply["$(date -v-1H -u +%FT%TZ)"] # apply journaled changes for the past hour
@@ -1097,7 +1134,7 @@ Chewy.use_after_commit_callbacks = !Rails.env.test?
1097
1134
  5. Push to the branch (`git push origin my-new-feature`)
1098
1135
  6. Create new Pull Request
1099
1136
 
1100
- Use the following Rake tasks to control the Elasticsearch cluster while developing.
1137
+ Use the following Rake tasks to control the Elasticsearch cluster while developing, if you prefer native Elasticsearch installation over the dockerized one:
1101
1138
 
1102
1139
  ```bash
1103
1140
  rake elasticsearch:start # start Elasticsearch cluster on 9250 port for tests