chewy 0.8.4 → 0.9.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.rubocop.yml +34 -0
- data/.rubocop_todo.yml +44 -0
- data/.travis.yml +20 -60
- data/Appraisals +15 -40
- data/CHANGELOG.md +42 -0
- data/Gemfile +1 -0
- data/Guardfile +5 -5
- data/README.md +155 -6
- data/Rakefile +11 -1
- data/chewy.gemspec +5 -7
- data/gemfiles/rails.3.2.activerecord.gemfile +1 -0
- data/gemfiles/rails.3.2.activerecord.kaminari.gemfile +1 -0
- data/gemfiles/rails.3.2.activerecord.will_paginate.gemfile +1 -0
- data/gemfiles/rails.4.2.activerecord.gemfile +1 -0
- data/gemfiles/rails.4.2.activerecord.kaminari.gemfile +1 -0
- data/gemfiles/rails.4.2.activerecord.will_paginate.gemfile +1 -0
- data/gemfiles/{rails.4.2.mongoid.4.0.0.gemfile → rails.4.2.mongoid.4.0.gemfile} +2 -1
- data/gemfiles/{rails.4.2.mongoid.4.0.0.kaminari.gemfile → rails.4.2.mongoid.4.0.kaminari.gemfile} +2 -1
- data/gemfiles/{rails.4.2.mongoid.4.0.0.will_paginate.gemfile → rails.4.2.mongoid.4.0.will_paginate.gemfile} +2 -1
- data/gemfiles/{rails.4.2.mongoid.5.1.0.gemfile → rails.4.2.mongoid.5.1.gemfile} +2 -1
- data/gemfiles/{rails.4.2.mongoid.5.1.0.kaminari.gemfile → rails.4.2.mongoid.5.1.kaminari.gemfile} +2 -1
- data/gemfiles/{rails.4.2.mongoid.5.1.0.will_paginate.gemfile → rails.4.2.mongoid.5.1.will_paginate.gemfile} +2 -1
- data/gemfiles/{rails.4.0.activerecord.gemfile → rails.5.0.activerecord.gemfile} +4 -2
- data/gemfiles/{rails.4.0.mongoid.4.0.0.kaminari.gemfile → rails.5.0.activerecord.kaminari.gemfile} +4 -2
- data/gemfiles/{rails.4.0.mongoid.4.0.0.will_paginate.gemfile → rails.5.0.activerecord.will_paginate.gemfile} +4 -2
- data/gemfiles/{sequel.4.31.gemfile → sequel.4.38.gemfile} +3 -2
- data/lib/chewy.rb +24 -16
- data/lib/chewy/backports/deep_dup.rb +1 -1
- data/lib/chewy/backports/duplicable.rb +1 -0
- data/lib/chewy/config.rb +13 -7
- data/lib/chewy/errors.rb +4 -4
- data/lib/chewy/fields/base.rb +19 -14
- data/lib/chewy/fields/root.rb +11 -9
- data/lib/chewy/index.rb +38 -25
- data/lib/chewy/index/actions.rb +17 -17
- data/lib/chewy/index/settings.rb +3 -4
- data/lib/chewy/journal.rb +107 -0
- data/lib/chewy/journal/apply.rb +31 -0
- data/lib/chewy/journal/clean.rb +24 -0
- data/lib/chewy/journal/entry.rb +83 -0
- data/lib/chewy/journal/query.rb +87 -0
- data/lib/chewy/log_subscriber.rb +8 -8
- data/lib/chewy/minitest.rb +1 -0
- data/lib/chewy/minitest/helpers.rb +77 -0
- data/lib/chewy/minitest/search_index_receiver.rb +80 -0
- data/lib/chewy/query.rb +116 -60
- data/lib/chewy/query/compose.rb +5 -6
- data/lib/chewy/query/criteria.rb +26 -16
- data/lib/chewy/query/filters.rb +9 -9
- data/lib/chewy/query/loading.rb +2 -2
- data/lib/chewy/query/nodes/and.rb +3 -3
- data/lib/chewy/query/nodes/base.rb +1 -1
- data/lib/chewy/query/nodes/bool.rb +6 -4
- data/lib/chewy/query/nodes/equal.rb +6 -6
- data/lib/chewy/query/nodes/exists.rb +2 -2
- data/lib/chewy/query/nodes/expr.rb +2 -2
- data/lib/chewy/query/nodes/field.rb +35 -31
- data/lib/chewy/query/nodes/has_child.rb +1 -0
- data/lib/chewy/query/nodes/has_parent.rb +1 -0
- data/lib/chewy/query/nodes/has_relation.rb +11 -13
- data/lib/chewy/query/nodes/match_all.rb +1 -1
- data/lib/chewy/query/nodes/missing.rb +2 -2
- data/lib/chewy/query/nodes/not.rb +3 -3
- data/lib/chewy/query/nodes/or.rb +3 -3
- data/lib/chewy/query/nodes/prefix.rb +4 -3
- data/lib/chewy/query/nodes/query.rb +3 -3
- data/lib/chewy/query/nodes/range.rb +11 -11
- data/lib/chewy/query/nodes/raw.rb +1 -1
- data/lib/chewy/query/nodes/regexp.rb +15 -11
- data/lib/chewy/query/nodes/script.rb +6 -6
- data/lib/chewy/query/pagination/will_paginate.rb +2 -2
- data/lib/chewy/railtie.rb +3 -3
- data/lib/chewy/rake_helper.rb +51 -30
- data/lib/chewy/repository.rb +2 -2
- data/lib/chewy/rspec.rb +1 -1
- data/lib/chewy/rspec/update_index.rb +46 -47
- data/lib/chewy/runtime/version.rb +4 -4
- data/lib/chewy/search.rb +7 -5
- data/lib/chewy/strategy.rb +10 -8
- data/lib/chewy/strategy/atomic.rb +2 -2
- data/lib/chewy/strategy/base.rb +4 -4
- data/lib/chewy/strategy/bypass.rb +1 -2
- data/lib/chewy/strategy/sidekiq.rb +2 -0
- data/lib/chewy/strategy/urgent.rb +1 -1
- data/lib/chewy/type.rb +51 -45
- data/lib/chewy/type/adapter/active_record.rb +23 -12
- data/lib/chewy/type/adapter/base.rb +4 -4
- data/lib/chewy/type/adapter/mongoid.rb +6 -6
- data/lib/chewy/type/adapter/object.rb +15 -12
- data/lib/chewy/type/adapter/orm.rb +24 -15
- data/lib/chewy/type/adapter/sequel.rb +11 -7
- data/lib/chewy/type/crutch.rb +4 -3
- data/lib/chewy/type/import.rb +51 -32
- data/lib/chewy/type/mapping.rb +17 -17
- data/lib/chewy/type/observe.rb +9 -7
- data/lib/chewy/type/witchcraft.rb +62 -23
- data/lib/chewy/type/wrapper.rb +20 -14
- data/lib/chewy/version.rb +1 -1
- data/lib/generators/chewy/install_generator.rb +3 -3
- data/lib/tasks/chewy.rake +28 -23
- data/spec/chewy/config_spec.rb +33 -12
- data/spec/chewy/fields/base_spec.rb +143 -154
- data/spec/chewy/fields/root_spec.rb +22 -20
- data/spec/chewy/fields/time_fields_spec.rb +11 -9
- data/spec/chewy/index/actions_spec.rb +27 -4
- data/spec/chewy/index/aliases_spec.rb +2 -2
- data/spec/chewy/index/settings_spec.rb +72 -50
- data/spec/chewy/index_spec.rb +95 -43
- data/spec/chewy/journal/apply_spec.rb +120 -0
- data/spec/chewy/journal/entry_spec.rb +237 -0
- data/spec/chewy/journal_spec.rb +173 -0
- data/spec/chewy/minitest/helpers_spec.rb +90 -0
- data/spec/chewy/minitest/search_index_receiver_spec.rb +120 -0
- data/spec/chewy/query/criteria_spec.rb +504 -237
- data/spec/chewy/query/filters_spec.rb +94 -66
- data/spec/chewy/query/loading_spec.rb +76 -40
- data/spec/chewy/query/nodes/and_spec.rb +3 -7
- data/spec/chewy/query/nodes/bool_spec.rb +5 -13
- data/spec/chewy/query/nodes/equal_spec.rb +20 -20
- data/spec/chewy/query/nodes/exists_spec.rb +7 -7
- data/spec/chewy/query/nodes/has_child_spec.rb +42 -23
- data/spec/chewy/query/nodes/has_parent_spec.rb +42 -23
- data/spec/chewy/query/nodes/match_all_spec.rb +2 -2
- data/spec/chewy/query/nodes/missing_spec.rb +6 -5
- data/spec/chewy/query/nodes/not_spec.rb +3 -7
- data/spec/chewy/query/nodes/or_spec.rb +3 -7
- data/spec/chewy/query/nodes/prefix_spec.rb +6 -6
- data/spec/chewy/query/nodes/query_spec.rb +3 -3
- data/spec/chewy/query/nodes/range_spec.rb +19 -19
- data/spec/chewy/query/nodes/raw_spec.rb +2 -2
- data/spec/chewy/query/nodes/regexp_spec.rb +31 -19
- data/spec/chewy/query/nodes/script_spec.rb +5 -5
- data/spec/chewy/query/pagination/kaminari_spec.rb +2 -2
- data/spec/chewy/query/pagination/will_paginage_spec.rb +6 -7
- data/spec/chewy/query/pagination_spec.rb +2 -3
- data/spec/chewy/query_spec.rb +208 -145
- data/spec/chewy/repository_spec.rb +8 -8
- data/spec/chewy/rspec/update_index_spec.rb +180 -111
- data/spec/chewy/search_spec.rb +8 -8
- data/spec/chewy/strategy/active_job_spec.rb +2 -2
- data/spec/chewy/strategy/atomic_spec.rb +4 -1
- data/spec/chewy/strategy/resque_spec.rb +2 -2
- data/spec/chewy/strategy/sidekiq_spec.rb +2 -2
- data/spec/chewy/type/actions_spec.rb +1 -1
- data/spec/chewy/type/adapter/active_record_spec.rb +255 -149
- data/spec/chewy/type/adapter/mongoid_spec.rb +169 -108
- data/spec/chewy/type/adapter/object_spec.rb +56 -40
- data/spec/chewy/type/adapter/sequel_spec.rb +248 -163
- data/spec/chewy/type/import_spec.rb +78 -47
- data/spec/chewy/type/mapping_spec.rb +6 -6
- data/spec/chewy/type/observe_spec.rb +20 -14
- data/spec/chewy/type/witchcraft_spec.rb +89 -43
- data/spec/chewy/type_spec.rb +4 -3
- data/spec/chewy_spec.rb +10 -8
- data/spec/spec_helper.rb +3 -0
- data/spec/support/active_record.rb +1 -1
- data/spec/support/class_helpers.rb +10 -11
- data/spec/support/mongoid.rb +2 -2
- data/spec/support/sequel.rb +1 -1
- metadata +65 -35
- data/gemfiles/rails.4.0.activerecord.kaminari.gemfile +0 -14
- data/gemfiles/rails.4.0.activerecord.will_paginate.gemfile +0 -14
- data/gemfiles/rails.4.0.mongoid.4.0.0.gemfile +0 -15
- data/gemfiles/rails.4.0.mongoid.5.1.0.gemfile +0 -15
- data/gemfiles/rails.4.0.mongoid.5.1.0.kaminari.gemfile +0 -14
- data/gemfiles/rails.4.0.mongoid.5.1.0.will_paginate.gemfile +0 -14
- data/gemfiles/rails.4.1.activerecord.gemfile +0 -15
- data/gemfiles/rails.4.1.activerecord.kaminari.gemfile +0 -14
- data/gemfiles/rails.4.1.activerecord.will_paginate.gemfile +0 -14
- data/gemfiles/rails.4.1.mongoid.4.0.0.gemfile +0 -15
- data/gemfiles/rails.4.1.mongoid.4.0.0.kaminari.gemfile +0 -14
- data/gemfiles/rails.4.1.mongoid.4.0.0.will_paginate.gemfile +0 -14
- data/gemfiles/rails.4.1.mongoid.5.1.0.gemfile +0 -15
- data/gemfiles/rails.4.1.mongoid.5.1.0.kaminari.gemfile +0 -14
- data/gemfiles/rails.4.1.mongoid.5.1.0.will_paginate.gemfile +0 -14
- data/gemfiles/rails.5.0.0.beta3.activerecord.gemfile +0 -16
- data/gemfiles/rails.5.0.0.beta3.activerecord.kaminari.gemfile +0 -16
- data/gemfiles/rails.5.0.0.beta3.activerecord.will_paginate.gemfile +0 -15
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: de7c5c4e672dfb909540467bf37a13aaa21933c1
|
4
|
+
data.tar.gz: 1e6a424a74805c9e19495f9c2d33a1e16cd0a843
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 9a7b5152aa2def448318b81b1fbac395ad652620c079c8013332e5e44ffebcfb27932688d81b7116f95447e37ebe715630307bf45e010f6964ff12a618a97d97
|
7
|
+
data.tar.gz: 370c365d9d690e29e5ab7de4ba06f0dff07aa360edbb7f18c982bf2c6f288b1bdd7554bb8bc569993d82553c696e44c3c726ab5c11db278b1d6fcc4c472b4998
|
data/.rubocop.yml
ADDED
@@ -0,0 +1,34 @@
|
|
1
|
+
inherit_from: .rubocop_todo.yml
|
2
|
+
|
3
|
+
Lint/EndAlignment:
|
4
|
+
AlignWith: variable
|
5
|
+
|
6
|
+
Style/AccessModifierIndentation:
|
7
|
+
EnforcedStyle: outdent
|
8
|
+
|
9
|
+
Style/Alias:
|
10
|
+
EnforcedStyle: prefer_alias_method
|
11
|
+
|
12
|
+
Style/AlignHash:
|
13
|
+
EnforcedLastArgumentHashStyle: always_ignore
|
14
|
+
|
15
|
+
Style/AlignParameters:
|
16
|
+
EnforcedStyle: with_fixed_indentation
|
17
|
+
|
18
|
+
Style/AndOr:
|
19
|
+
EnforcedStyle: conditionals
|
20
|
+
|
21
|
+
Style/CaseIndentation:
|
22
|
+
IndentWhenRelativeTo: end
|
23
|
+
|
24
|
+
Style/DoubleNegation:
|
25
|
+
Enabled: false
|
26
|
+
|
27
|
+
Style/IndentArray:
|
28
|
+
EnforcedStyle: consistent
|
29
|
+
|
30
|
+
Style/MultilineMethodCallIndentation:
|
31
|
+
EnforcedStyle: indented
|
32
|
+
|
33
|
+
Style/MultilineOperationIndentation:
|
34
|
+
EnforcedStyle: indented
|
data/.rubocop_todo.yml
ADDED
@@ -0,0 +1,44 @@
|
|
1
|
+
# This configuration was generated by
|
2
|
+
# `rubocop --auto-gen-config`
|
3
|
+
# on 2016-09-09 17:05:47 +0700 using RuboCop version 0.42.0.
|
4
|
+
# The point is for the user to remove these configuration records
|
5
|
+
# one by one as the offenses are removed from the code base.
|
6
|
+
# Note that changes in the inspected code, or installation of new
|
7
|
+
# versions of RuboCop, may require this file to be generated again.
|
8
|
+
|
9
|
+
# Offense count: 42
|
10
|
+
Metrics/AbcSize:
|
11
|
+
Max: 48
|
12
|
+
|
13
|
+
# Offense count: 4
|
14
|
+
# Configuration parameters: CountComments.
|
15
|
+
Metrics/ClassLength:
|
16
|
+
Max: 280
|
17
|
+
|
18
|
+
# Offense count: 14
|
19
|
+
Metrics/CyclomaticComplexity:
|
20
|
+
Max: 13
|
21
|
+
|
22
|
+
# Offense count: 1450
|
23
|
+
# Configuration parameters: AllowHeredoc, AllowURI, URISchemes.
|
24
|
+
# URISchemes: http, https
|
25
|
+
Metrics/LineLength:
|
26
|
+
Max: 198
|
27
|
+
|
28
|
+
# Offense count: 38
|
29
|
+
# Configuration parameters: CountComments.
|
30
|
+
Metrics/MethodLength:
|
31
|
+
Max: 33
|
32
|
+
|
33
|
+
# Offense count: 1
|
34
|
+
# Configuration parameters: CountComments.
|
35
|
+
Metrics/ModuleLength:
|
36
|
+
Max: 172
|
37
|
+
|
38
|
+
# Offense count: 14
|
39
|
+
Metrics/PerceivedComplexity:
|
40
|
+
Max: 15
|
41
|
+
|
42
|
+
# Offense count: 93
|
43
|
+
Style/Documentation:
|
44
|
+
Enabled: false
|
data/.travis.yml
CHANGED
@@ -3,74 +3,34 @@ sudo: false
|
|
3
3
|
services:
|
4
4
|
- mongodb
|
5
5
|
rvm:
|
6
|
-
- 2.
|
7
|
-
- 2.1
|
8
|
-
- 2.2.3
|
9
|
-
- 2.3.0
|
10
|
-
# - rbx
|
6
|
+
- 2.2.5
|
7
|
+
- 2.3.1
|
11
8
|
gemfile:
|
12
|
-
- gemfiles/rails.3.2.activerecord.gemfile
|
13
|
-
- gemfiles/rails.3.2.activerecord.kaminari.gemfile
|
14
|
-
- gemfiles/rails.3.2.activerecord.will_paginate.gemfile
|
15
|
-
- gemfiles/rails.4.0.activerecord.gemfile
|
16
|
-
- gemfiles/rails.4.0.activerecord.kaminari.gemfile
|
17
|
-
- gemfiles/rails.4.0.activerecord.will_paginate.gemfile
|
18
|
-
- gemfiles/rails.4.1.activerecord.gemfile
|
19
|
-
- gemfiles/rails.4.1.activerecord.kaminari.gemfile
|
20
|
-
- gemfiles/rails.4.1.activerecord.will_paginate.gemfile
|
21
9
|
- gemfiles/rails.4.2.activerecord.gemfile
|
22
10
|
- gemfiles/rails.4.2.activerecord.kaminari.gemfile
|
23
11
|
- gemfiles/rails.4.2.activerecord.will_paginate.gemfile
|
24
|
-
- gemfiles/rails.5.0.
|
25
|
-
- gemfiles/rails.5.0.
|
26
|
-
- gemfiles/rails.5.0.
|
27
|
-
- gemfiles/rails.4.
|
28
|
-
- gemfiles/rails.4.
|
29
|
-
- gemfiles/rails.4.
|
30
|
-
- gemfiles/rails.4.
|
31
|
-
- gemfiles/rails.4.
|
32
|
-
- gemfiles/rails.4.
|
33
|
-
- gemfiles/
|
34
|
-
- gemfiles/rails.4.1.mongoid.4.0.0.kaminari.gemfile
|
35
|
-
- gemfiles/rails.4.1.mongoid.4.0.0.will_paginate.gemfile
|
36
|
-
- gemfiles/rails.4.1.mongoid.5.1.0.gemfile
|
37
|
-
- gemfiles/rails.4.1.mongoid.5.1.0.kaminari.gemfile
|
38
|
-
- gemfiles/rails.4.1.mongoid.5.1.0.will_paginate.gemfile
|
39
|
-
- gemfiles/rails.4.2.mongoid.4.0.0.gemfile
|
40
|
-
- gemfiles/rails.4.2.mongoid.4.0.0.kaminari.gemfile
|
41
|
-
- gemfiles/rails.4.2.mongoid.4.0.0.will_paginate.gemfile
|
42
|
-
- gemfiles/rails.4.2.mongoid.5.1.0.gemfile
|
43
|
-
- gemfiles/rails.4.2.mongoid.5.1.0.kaminari.gemfile
|
44
|
-
- gemfiles/rails.4.2.mongoid.5.1.0.will_paginate.gemfile
|
45
|
-
- gemfiles/sequel.4.31.gemfile
|
46
|
-
|
12
|
+
- gemfiles/rails.5.0.activerecord.gemfile
|
13
|
+
- gemfiles/rails.5.0.activerecord.kaminari.gemfile
|
14
|
+
- gemfiles/rails.5.0.activerecord.will_paginate.gemfile
|
15
|
+
- gemfiles/rails.4.2.mongoid.4.0.gemfile
|
16
|
+
- gemfiles/rails.4.2.mongoid.4.0.kaminari.gemfile
|
17
|
+
- gemfiles/rails.4.2.mongoid.4.0.will_paginate.gemfile
|
18
|
+
- gemfiles/rails.4.2.mongoid.5.1.gemfile
|
19
|
+
- gemfiles/rails.4.2.mongoid.5.1.kaminari.gemfile
|
20
|
+
- gemfiles/rails.4.2.mongoid.5.1.will_paginate.gemfile
|
21
|
+
- gemfiles/sequel.4.38.gemfile
|
47
22
|
matrix:
|
48
|
-
|
49
|
-
- rvm: 2.0
|
50
|
-
gemfile: gemfiles/rails.5.0.0.beta3.activerecord.gemfile
|
51
|
-
- rvm: 2.0
|
52
|
-
gemfile: gemfiles/rails.5.0.0.beta3.activerecord.kaminari.gemfile
|
53
|
-
- rvm: 2.0
|
54
|
-
gemfile: gemfiles/rails.5.0.0.beta3.activerecord.will_paginate.gemfile
|
23
|
+
include:
|
55
24
|
- rvm: 2.1
|
56
|
-
gemfile: gemfiles/rails.5.0.0.beta3.activerecord.gemfile
|
57
|
-
- rvm: 2.1
|
58
|
-
gemfile: gemfiles/rails.5.0.0.beta3.activerecord.kaminari.gemfile
|
59
|
-
- rvm: 2.1
|
60
|
-
gemfile: gemfiles/rails.5.0.0.beta3.activerecord.will_paginate.gemfile
|
61
|
-
- rvm: 2.2
|
62
25
|
gemfile: gemfiles/rails.3.2.activerecord.gemfile
|
63
|
-
- rvm: 2.
|
64
|
-
gemfile: gemfiles/rails.3.2.activerecord.kaminari.gemfile
|
65
|
-
- rvm: 2.2
|
66
|
-
gemfile: gemfiles/rails.3.2.activerecord.will_paginate.gemfile
|
67
|
-
- rvm: 2.3
|
68
|
-
gemfile: gemfiles/rails.3.2.activerecord.gemfile
|
69
|
-
- rvm: 2.3
|
26
|
+
- rvm: 2.1
|
70
27
|
gemfile: gemfiles/rails.3.2.activerecord.kaminari.gemfile
|
71
|
-
- rvm: 2.
|
28
|
+
- rvm: 2.1
|
72
29
|
gemfile: gemfiles/rails.3.2.activerecord.will_paginate.gemfile
|
73
30
|
before_install:
|
74
|
-
- curl
|
31
|
+
- curl -s https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.5.tar.gz | tar xz -C /tmp
|
75
32
|
before_script:
|
76
|
-
- TEST_CLUSTER_COMMAND="/tmp/elasticsearch-1.7.
|
33
|
+
- TEST_CLUSTER_COMMAND="/tmp/elasticsearch-1.7.5/bin/elasticsearch" rake es:start
|
34
|
+
script:
|
35
|
+
- bundle exec rspec
|
36
|
+
- bundle exec rubocop
|
data/Appraisals
CHANGED
@@ -1,4 +1,4 @@
|
|
1
|
-
%w(3.2 4.
|
1
|
+
%w(3.2 4.2 5.0).each do |version|
|
2
2
|
appraise "rails.#{version}.activerecord" do
|
3
3
|
gem 'activerecord', "~> #{version}.0"
|
4
4
|
gem 'activesupport', "~> #{version}.0"
|
@@ -22,55 +22,30 @@
|
|
22
22
|
end
|
23
23
|
end
|
24
24
|
|
25
|
-
'
|
26
|
-
appraise "rails.#{
|
27
|
-
gem '
|
28
|
-
gem 'activesupport', "~> #{
|
29
|
-
gem 'activejob', "~> #{version}"
|
25
|
+
{ '4.0' => '4.2', '5.1' => '4.2' }.each do |(mongoid, activesupport)|
|
26
|
+
appraise "rails.#{activesupport}.mongoid.#{mongoid}" do
|
27
|
+
gem 'mongoid', "~> #{mongoid}.0"
|
28
|
+
gem 'activesupport', "~> #{activesupport}.0"
|
30
29
|
gem 'resque', require: false
|
31
30
|
gem 'sidekiq', require: false
|
32
31
|
end
|
33
32
|
|
34
|
-
appraise "rails.#{
|
35
|
-
gem '
|
36
|
-
gem 'activesupport', "~> #{
|
37
|
-
gem 'activejob', "~> #{version}"
|
38
|
-
gem 'actionpack', "~> #{version}"
|
33
|
+
appraise "rails.#{activesupport}.mongoid.#{mongoid}.kaminari" do
|
34
|
+
gem 'mongoid', "~> #{mongoid}.0"
|
35
|
+
gem 'activesupport', "~> #{activesupport}.0"
|
39
36
|
gem 'kaminari', '0.16.3', require: false
|
40
37
|
end
|
41
38
|
|
42
|
-
appraise "rails.#{
|
43
|
-
gem '
|
44
|
-
gem 'activesupport', "~> #{
|
45
|
-
gem 'activejob', "~> #{version}" if version >= '4.2'
|
39
|
+
appraise "rails.#{activesupport}.mongoid.#{mongoid}.will_paginate" do
|
40
|
+
gem 'mongoid', "~> #{mongoid}.0"
|
41
|
+
gem 'activesupport', "~> #{activesupport}.0"
|
46
42
|
gem 'will_paginate', require: false
|
47
43
|
end
|
48
44
|
end
|
49
45
|
|
50
|
-
%w(4.
|
51
|
-
|
52
|
-
|
53
|
-
|
54
|
-
gem 'activesupport', "~> #{activesupport}.0"
|
55
|
-
gem 'resque', require: false
|
56
|
-
gem 'sidekiq', require: false
|
57
|
-
end
|
58
|
-
|
59
|
-
appraise "rails.#{activesupport}.mongoid.#{mongoid}.kaminari" do
|
60
|
-
gem 'mongoid', "> #{mongoid}"
|
61
|
-
gem 'activesupport', "~> #{activesupport}.0"
|
62
|
-
gem 'kaminari', '0.16.3', require: false
|
63
|
-
end
|
64
|
-
|
65
|
-
appraise "rails.#{activesupport}.mongoid.#{mongoid}.will_paginate" do
|
66
|
-
gem 'mongoid', "> #{mongoid}"
|
67
|
-
gem 'activesupport', "~> #{activesupport}.0"
|
68
|
-
gem 'will_paginate', require: false
|
69
|
-
end
|
46
|
+
%w(4.38).each do |sequel|
|
47
|
+
appraise "sequel.#{sequel}" do
|
48
|
+
gem 'sequel', "~> #{sequel}.0"
|
49
|
+
gem 'activesupport', '~> 5.0.0'
|
70
50
|
end
|
71
51
|
end
|
72
|
-
|
73
|
-
appraise "sequel.4.31" do
|
74
|
-
gem 'sequel', "~> 4.31.0"
|
75
|
-
gem 'activesupport', '~> 4.2.0'
|
76
|
-
end
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,47 @@
|
|
1
1
|
# master
|
2
2
|
|
3
|
+
# Version 0.9.0
|
4
|
+
|
5
|
+
## Changes
|
6
|
+
|
7
|
+
* Add `preference` param to Query (@menglewis, #443)
|
8
|
+
|
9
|
+
* Add the `track_scores` option to the query; `_score` to be computed and tracked even when there are no `_score` in sort. (@dmitry, #417)
|
10
|
+
|
11
|
+
* Confugurable `Chewy.indices_path` (@robacarp, #414, #433, #439)
|
12
|
+
|
13
|
+
* [Journaling](https://github.com/toptal/chewy/#journaling) implementation (@sergey-kintsel, #409, #425, #428, #432, #434, #463)
|
14
|
+
|
15
|
+
* Minitest helpers (@robacarp, #396)
|
16
|
+
|
17
|
+
* `Chewy::Query#unlimited` to fetch all the records (@sergey-kintsel, #393)
|
18
|
+
|
19
|
+
* `Chewy::Query#exists?` (@sergey-kintsel, #386)
|
20
|
+
|
21
|
+
* Import otimizations (#381, #376)
|
22
|
+
|
23
|
+
* Additional import optimization technique - [raw import](https://github.com/toptal/chewy/#raw-import) (@DNNX, #375)
|
24
|
+
|
25
|
+
* `weight` scoring dunction was added to the search DSL (@sevab, #380)
|
26
|
+
|
27
|
+
* Rake tasks support multiple indexes and exceptions: `rake chewy:reset[users,projects]`, `rake chewy:update[-projects]`
|
28
|
+
|
29
|
+
* Witchcraft™ supports dynamically generated procs with variables from closure.
|
30
|
+
|
31
|
+
* Added `Query#preference` for specifying shard replicas to query against. (@menglewis)
|
32
|
+
|
33
|
+
## Bugfixes
|
34
|
+
|
35
|
+
* `.script_fields` method in the Index class (@dmitry, #415)
|
36
|
+
|
37
|
+
* Fix routing_missing_exception on delete with parent missing (@guigs, #398)
|
38
|
+
|
39
|
+
* Sequesl custom primary keys handling fix (@okliv, #385)
|
40
|
+
|
41
|
+
* Bulk import fixes (@0x0badc0de, #374)
|
42
|
+
|
43
|
+
# Version 0.8.4
|
44
|
+
|
3
45
|
## Changes
|
4
46
|
|
5
47
|
* Brand new import `:bulk_size` option, set desired ElasticSearch bulk size in bytes
|
data/Gemfile
CHANGED
data/Guardfile
CHANGED
@@ -4,20 +4,20 @@
|
|
4
4
|
guard :rspec, cmd: 'rspec' do
|
5
5
|
watch(%r{^spec/.+_spec\.rb$})
|
6
6
|
watch(%r{^lib/(.+)\.rb$}) { |m| "spec/#{m[1]}_spec.rb" }
|
7
|
-
watch('spec/spec_helper.rb') {
|
7
|
+
watch('spec/spec_helper.rb') { 'spec' }
|
8
8
|
|
9
9
|
# Rails example
|
10
10
|
watch(%r{^app/(.+)\.rb$}) { |m| "spec/#{m[1]}_spec.rb" }
|
11
11
|
watch(%r{^app/(.*)(\.erb|\.haml|\.slim)$}) { |m| "spec/#{m[1]}#{m[2]}_spec.rb" }
|
12
12
|
watch(%r{^app/controllers/(.+)_(controller)\.rb$}) { |m| ["spec/routing/#{m[1]}_routing_spec.rb", "spec/#{m[2]}s/#{m[1]}_#{m[2]}_spec.rb", "spec/acceptance/#{m[1]}_spec.rb"] }
|
13
|
-
watch(%r{^spec/support/(.+)\.rb$}) {
|
14
|
-
watch('config/routes.rb') {
|
15
|
-
watch('app/controllers/application_controller.rb') {
|
13
|
+
watch(%r{^spec/support/(.+)\.rb$}) { 'spec' }
|
14
|
+
watch('config/routes.rb') { 'spec/routing' }
|
15
|
+
watch('app/controllers/application_controller.rb') { 'spec/controllers' }
|
16
16
|
|
17
17
|
# Capybara features specs
|
18
18
|
watch(%r{^app/views/(.+)/.*\.(erb|haml|slim)$}) { |m| "spec/features/#{m[1]}_spec.rb" }
|
19
19
|
|
20
20
|
# Turnip features and steps
|
21
21
|
watch(%r{^spec/acceptance/(.+)\.feature$})
|
22
|
-
watch(%r{^spec/acceptance/steps/(.+)_steps\.rb$})
|
22
|
+
watch(%r{^spec/acceptance/steps/(.+)_steps\.rb$}) { |m| Dir[File.join("**/#{m[1]}.feature")][0] || 'spec/acceptance' }
|
23
23
|
end
|
data/README.md
CHANGED
@@ -4,11 +4,47 @@
|
|
4
4
|
[![Inline docs](http://inch-ci.org/github/toptal/chewy.svg?branch=master)](http://inch-ci.org/github/toptal/chewy)
|
5
5
|
|
6
6
|
<p align="right">Sponsored by</p>
|
7
|
-
<p align="right"><a href="
|
7
|
+
<p align="right"><a href="https://www.toptal.com/"><img src="https://www.toptal.com/assets/public/blocks/logo/big.png" alt="Toptal" width="105" height="34"></a></p>
|
8
8
|
|
9
9
|
# Chewy
|
10
10
|
|
11
|
-
Chewy is an ODM and wrapper for [the official Elasticsearch client](https://github.com/
|
11
|
+
Chewy is an ODM and wrapper for [the official Elasticsearch client](https://github.com/elastic/elasticsearch-ruby).
|
12
|
+
|
13
|
+
## Table of Contents
|
14
|
+
|
15
|
+
* [Why Chewy?] (#why-chewy)
|
16
|
+
* [Usage] (#usage)
|
17
|
+
* [Client settings] (#client-settings)
|
18
|
+
* [Index definition] (#index-definition)
|
19
|
+
* [Type default import options] (#type-default-import-options)
|
20
|
+
* [Multi (nested) and object field types] (#multi-nested-and-object-field-types)
|
21
|
+
* [Geo Point fields] (#geo-point-fields)
|
22
|
+
* [Crutches™ technology] (#crutches-technology)
|
23
|
+
* [Witchcraft™ technology] (#witchcraft-technology)
|
24
|
+
* [Raw Import] (#raw-import)
|
25
|
+
* [Journaling] (#journaling)
|
26
|
+
* [Types access] (#types-access)
|
27
|
+
* [Index manipulation] (#index-manipulation)
|
28
|
+
* [Index update strategies] (#index-update-strategies)
|
29
|
+
* [Nesting] (#nesting)
|
30
|
+
* [Non-block notation] (#non-block-notation)
|
31
|
+
* [Designing your own strategies] (#designing-your-own-strategies)
|
32
|
+
* [Rails application strategies integration] (#rails-application-strategies-integration)
|
33
|
+
* [Index querying] (#index-querying)
|
34
|
+
* [Additional query action.] (#additional-query-action)
|
35
|
+
* [Filters query DSL] (#filters-query-dsl)
|
36
|
+
* [Faceting] (#faceting)
|
37
|
+
* [Aggregations] (#aggregations)
|
38
|
+
* [Script fields] (#script-fields)
|
39
|
+
* [Script scoring] (#script-scoring)
|
40
|
+
* [Boost Factor] (#boost-factor)
|
41
|
+
* [Objects loading] (#objects-loading)
|
42
|
+
* [NewRelic integration] (#newrelic-integration)
|
43
|
+
* [Rake tasks] (#rake-tasks)
|
44
|
+
* [Rspec integration] (#rspec-integration)
|
45
|
+
* [Minitest integration] (#minitest-integration)
|
46
|
+
* [TODO a.k.a coming soon:] (#todo-aka-coming-soon)
|
47
|
+
* [Contributing] (#contributing)
|
12
48
|
|
13
49
|
## Why Chewy?
|
14
50
|
|
@@ -125,7 +161,7 @@ See [config.rb](lib/chewy/config.rb) for more details.
|
|
125
161
|
end
|
126
162
|
```
|
127
163
|
|
128
|
-
[See here for mapping definitions](
|
164
|
+
[See here for mapping definitions](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html).
|
129
165
|
|
130
166
|
4. Add some index- and type-related settings. Analyzer repositories might be used as well. See `Chewy::Index.settings` docs for details:
|
131
167
|
|
@@ -161,8 +197,8 @@ See [config.rb](lib/chewy/config.rb) for more details.
|
|
161
197
|
end
|
162
198
|
```
|
163
199
|
|
164
|
-
[See index settings here](
|
165
|
-
[See root object settings here](
|
200
|
+
[See index settings here](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-update-settings.html).
|
201
|
+
[See root object settings here](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-root-object-type.html).
|
166
202
|
|
167
203
|
See [mapping.rb](lib/chewy/type/mapping.rb) for more details.
|
168
204
|
|
@@ -395,7 +431,107 @@ end
|
|
395
431
|
```
|
396
432
|
|
397
433
|
And don't even ask how is it possible, it is a witchcraft.
|
398
|
-
Obviously not every type of definition might be compiled
|
434
|
+
Obviously not every type of definition might be compiled. There are some restrictions:
|
435
|
+
|
436
|
+
1. Use reasonable formatting to make `method_source` be able to extract field value proc sources.
|
437
|
+
2. Value procs with splat arguments are not supported right now.
|
438
|
+
3. If you are generating fields dynamically use value proc with arguments, argumentless value procs are not supported yet:
|
439
|
+
|
440
|
+
```ruby
|
441
|
+
[:first_name, :last_name].each do |name|
|
442
|
+
field name, value: -> (o) { o.send(name) }
|
443
|
+
end
|
444
|
+
```
|
445
|
+
|
446
|
+
However, it is quite possible that your type definition will be supported by Witchcraft™ technology out of the box in the most of the cases.
|
447
|
+
|
448
|
+
### Raw Import
|
449
|
+
|
450
|
+
Another way to speed up import time is Raw Imports. This technology is only available in ActiveRecord adapter. Very often, ActiveRecord model instantiation is what consumes most of the CPU and RAM resources. Precious time is wasted on converting, say, timestamps from strings and then serializing them back to strings. Chewy can operate on raw hashes of data directly obtained from the database. All you need is to provide a way to convert that hash to a lightweight object that mimics the behaviour of the normal ActiveRecord object.
|
451
|
+
|
452
|
+
```ruby
|
453
|
+
class LightweightProduct
|
454
|
+
def initialize(attributes)
|
455
|
+
@attributes = attributes
|
456
|
+
end
|
457
|
+
|
458
|
+
# Depending on the database, `created_at` might
|
459
|
+
# be in different formats. In PostgreSQL, for example,
|
460
|
+
# you might see the following format:
|
461
|
+
# "2016-03-22 16:23:22"
|
462
|
+
#
|
463
|
+
# Taking into account that Elastic expects something different,
|
464
|
+
# one might do something like the following, just to avoid
|
465
|
+
# unnecessary String -> DateTime -> String conversion.
|
466
|
+
#
|
467
|
+
# "2016-03-22 16:23:22" -> "2016-03-22T16:23:22Z"
|
468
|
+
def created_at
|
469
|
+
@attributes['created_at'].tr(' ', 'T') << 'Z'
|
470
|
+
end
|
471
|
+
end
|
472
|
+
|
473
|
+
define_type Product do
|
474
|
+
default_import_options raw_import: ->(hash) {
|
475
|
+
LightweightProduct.new(hash)
|
476
|
+
}
|
477
|
+
|
478
|
+
field :created_at, 'datetime'
|
479
|
+
end
|
480
|
+
```
|
481
|
+
|
482
|
+
Also, you can pass `:raw_import` option to the `import` method explicitly.
|
483
|
+
|
484
|
+
|
485
|
+
### Journaling
|
486
|
+
|
487
|
+
You can record all actions that were made to the separate journal index in ElasticSearch.
|
488
|
+
When you create/update/destroy your records, it will be saved in this special index.
|
489
|
+
If you make something with a batch of records (e.g. during index reset) it will be saved as a one record, including primary keys of each document that was affected.
|
490
|
+
Common journal record looks like this:
|
491
|
+
|
492
|
+
```json
|
493
|
+
{
|
494
|
+
"action": "index",
|
495
|
+
"object_id": [1, 2, 3],
|
496
|
+
"index_name": "...",
|
497
|
+
"type_name": "...",
|
498
|
+
"created_at": "<timestamp>"
|
499
|
+
}
|
500
|
+
```
|
501
|
+
|
502
|
+
This feature is turned off by default.
|
503
|
+
But you can turn it on by setting `journal` setting to `true` in `config/chewy.yml`.
|
504
|
+
Also, you can specify journal index name. For example:
|
505
|
+
|
506
|
+
```yaml
|
507
|
+
# config/chewy.yml
|
508
|
+
production:
|
509
|
+
journal: true,
|
510
|
+
journal_name: my_super_journal
|
511
|
+
```
|
512
|
+
|
513
|
+
Also, you can provide this option while you're importing some index:
|
514
|
+
|
515
|
+
```ruby
|
516
|
+
CityIndex.import journal: true
|
517
|
+
```
|
518
|
+
|
519
|
+
Or as a default import option for an index:
|
520
|
+
|
521
|
+
```ruby
|
522
|
+
class CityIndex
|
523
|
+
define_type City do
|
524
|
+
default_import_options journal: true
|
525
|
+
end
|
526
|
+
end
|
527
|
+
```
|
528
|
+
|
529
|
+
You may be wondering why do you need it? The answer is simple: Not to lose the data.
|
530
|
+
Imagine that:
|
531
|
+
You reset your index in Zero Downtime manner (to separate index), and meantime somebody keeps updating the data frequently (to old index). So all these actions will be written to the journal index and you'll be able to apply them after index reset with `Chewy::Journal::Apply.since(1.hour.ago.to_i)`.
|
532
|
+
|
533
|
+
For index reset journaling is turned off even if you set `journal: true` in `config/chewy.yml` or in `default_import_options`.
|
534
|
+
You can change it only if you pass `journal: true` parameter explicitly to `#import`.
|
399
535
|
|
400
536
|
### Types access
|
401
537
|
|
@@ -404,6 +540,8 @@ You can access index-defined types with the following API:
|
|
404
540
|
```ruby
|
405
541
|
UsersIndex::User # => UsersIndex::User
|
406
542
|
UsersIndex.type_hash['user'] # => UsersIndex::User
|
543
|
+
UsersIndex.type('user') # => UsersIndex::User
|
544
|
+
UsersIndex.type('foo') # => raises error UndefinedType("Unknown type in UsersIndex: foo")
|
407
545
|
UsersIndex.types # => [UsersIndex::User]
|
408
546
|
UsersIndex.type_names # => ['user']
|
409
547
|
```
|
@@ -1129,9 +1267,14 @@ Inside the Rails application, some index-maintaining rake tasks are defined.
|
|
1129
1267
|
```bash
|
1130
1268
|
rake chewy:reset # resets all the existing indices, declared in app/chewy
|
1131
1269
|
rake chewy:reset[users] # resets UsersIndex only
|
1270
|
+
rake chewy:reset[users,projects] # resets UsersIndex and ProjectsIndex
|
1271
|
+
rake chewy:reset[-users,projects] # resets every index in application except specified ones
|
1132
1272
|
|
1133
1273
|
rake chewy:update # updates all the existing indices, declared in app/chewy
|
1134
1274
|
rake chewy:update[users] # updates UsersIndex only
|
1275
|
+
rake chewy:update[users,projects] # updates UsersIndex and ProjectsIndex
|
1276
|
+
rake chewy:update[-users,projects] # updates every index in application except specified ones
|
1277
|
+
|
1135
1278
|
```
|
1136
1279
|
|
1137
1280
|
`rake chewy:reset` performs zero-downtime reindexing as described [here](https://www.elastic.co/blog/changing-mapping-with-zero-downtime). So basically rake task creates a new index with uniq suffix and then simply aliases it to the common index name. The previous index is deleted afterwards (see `Chewy::Index.reset!` for more details).
|
@@ -1141,6 +1284,12 @@ rake chewy:update[users] # updates UsersIndex only
|
|
1141
1284
|
|
1142
1285
|
Just add `require 'chewy/rspec'` to your spec_helper.rb and you will get additional features: See [update_index.rb](lib/chewy/rspec/update_index.rb) for more details.
|
1143
1286
|
|
1287
|
+
### Minitest integration
|
1288
|
+
|
1289
|
+
Add `require 'chewy/minitest'` to your test_helper.rb, and then for tests which you'd like indexing test hooks, `include Chewy::Minitest::Helpers`.
|
1290
|
+
|
1291
|
+
### DatabaseCleaner
|
1292
|
+
|
1144
1293
|
If you use `DatabaseCleaner` in your tests with [the `transaction` strategy](https://github.com/DatabaseCleaner/database_cleaner#how-to-use), you may run into the problem that `ActiveRecord`'s models are not indexed automatically on save despite the fact that you set the callbacks to do this with the `update_index` method. The issue arises because `chewy` indexes data on `after_commit` run as default, but all `after_commit` callbacks are not run with the `DatabaseCleaner`'s' `transaction` strategy. You can solve this issue by changing the `Chewy.use_after_commit_callbacks` option. Just add the following initializer in your Rails application:
|
1145
1294
|
|
1146
1295
|
```ruby
|