chewy 0.3.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 20296cb15e845cc45aacb4dddaebaa735b5f9d80
4
- data.tar.gz: 2872561f5db73ad60dc96ef445de9bdd2da529db
3
+ metadata.gz: 6bfe79b7a76d86426e6b1f67389d386e85ee33bc
4
+ data.tar.gz: 65102eba1137183f68dc167ef1c91f77abae9758
5
5
  SHA512:
6
- metadata.gz: 487d34a9309473ae01af8e2f4f2fe4a4318501c83e6370121e97acaa736630ea2b8bfb129b499dc74a01f7899fd131f296ffa372ddfa6eba324a392cd79d6e34
7
- data.tar.gz: 6387d3bcd0e6e1f48187a0bbcc00cfaaffb0069673e149cdf32b531f979843c3b47c3572e5fde6c5717bb21446cd2138deb703ae705969bad6568ff9efaeddea
6
+ metadata.gz: 6b366d1a609fdd330e46e5f4937187e3fdbc697c65869927deb578d7bb58ea170c3e4fcb6bf3a178fba0a29fe8c5c560e3229314da70b2b180ee19eed9084b9c
7
+ data.tar.gz: 7e76c1cc833e2884b1c48d3c1dbb197a8bae08a90b099ea87aea5ffb36249a51f08ca37bb1f575f3b6786551c3897b9cb4e443764cc46bd86904e096ad4f3126
data/.gitignore CHANGED
@@ -3,7 +3,7 @@
3
3
  .bundle
4
4
  .config
5
5
  .yardoc
6
- Gemfile.lock
6
+ Gemfile*.lock
7
7
  InstalledFiles
8
8
  _yardoc
9
9
  coverage
data/.travis.yml CHANGED
@@ -4,7 +4,11 @@ rvm:
4
4
  - 2.0.0
5
5
  - 2.1.0
6
6
  # - rbx
7
+ gemfile:
8
+ - Gemfile
9
+ - gemfiles/Gemfile.rails-3.2
10
+ - gemfiles/Gemfile.rails-4.0
7
11
  before_install:
8
12
  - curl -# https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.0.1.tar.gz | tar xz -C /tmp
9
13
  before_script:
10
- - TEST_CLUSTER_COMMAND="/tmp/elasticsearch-1.0.1/bin/elasticsearch" rake elasticsearch:start
14
+ - TEST_CLUSTER_COMMAND="/tmp/elasticsearch-1.0.1/bin/elasticsearch" rake elasticsearch:start
data/CHANGELOG.md CHANGED
@@ -1,5 +1,23 @@
1
1
  # master
2
2
 
3
+ # Version 0.4.0
4
+
5
+ * Changed `update_index` matcher behavior. Now it compare array attributes position-independantly.
6
+
7
+ * Search aggregations API support (@arion).
8
+
9
+ * Chewy::Query#facets called without params performs the request and returns facets.
10
+
11
+ * Added `Type.template` dsl method for root objects dynamic templates definition. See [mapping.rb](lib/chewy/type/mapping.rb) for more details.
12
+
13
+ * ActiveRecord adapter custom `primary_key` support (@matthee).
14
+
15
+ * Urgent update now clears association cache in ActiveRecord to ensure latest changes are imported.
16
+
17
+ * `import` now creates index before performing.
18
+
19
+ * `Chewy.configuration[:wait_for_status]` option. Can be set to `red`, `yellow` or `green`. If set - chewy will wait for cluster status before creating, deleting index and import. Useful for specs.
20
+
3
21
  # Version 0.3.0
4
22
 
5
23
  * Added `Chewy.configuration[:index]` config to setup common indexes options.
@@ -16,7 +34,9 @@
16
34
 
17
35
  # Version 0.2.2
18
36
 
19
- * Auto-resolved analyzers and analyzers repository:
37
+ * Support for `none` scope (@undr).
38
+
39
+ * Auto-resolved analyzers and analyzers repository (@webgago):
20
40
 
21
41
  ```ruby
22
42
  # Setting up analyzers repository:
data/README.md CHANGED
@@ -7,9 +7,11 @@ Chewy is ODM and wrapper for official elasticsearch client (https://github.com/e
7
7
 
8
8
  ## Why chewy?
9
9
 
10
- * Index classes are independant from ORM/ODM models.
11
10
 
12
- Now implementing, e.g. cross-model autocomplete is much easier. You can just define index and work with it in object-oriented style. You can define several types for index - one per indexed model.
11
+
12
+ * Multi-model indexes.
13
+
14
+ Index classes are independant from ORM/ODM models. Now implementing, e.g. cross-model autocomplete is much easier. You can just define index and work with it in object-oriented style. You can define several types for index - one per indexed model.
13
15
 
14
16
  * Every index is observable by all the related models.
15
17
 
@@ -127,7 +129,9 @@ Chewy.logger = Logger.new
127
129
  }
128
130
 
129
131
  define_type User.active.includes(:country, :badges, :projects) do
130
- root _boost: { name: :_boost, null_value: 1.0 } do
132
+ root date_detection: false do
133
+ template 'about_translations.*', type: 'string', analyzer: 'stantard'
134
+
131
135
  field :first_name, :last_name
132
136
  field :email, analyzer: 'email'
133
137
  field :country, value: ->(user) { user.country.name }
@@ -148,6 +152,8 @@ Chewy.logger = Logger.new
148
152
  Index settings - http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-update-settings.html
149
153
  Root object settings - http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-root-object-type.html
150
154
 
155
+ See [mapping.rb](lib/chewy/type/mapping.rb) for more details.
156
+
151
157
  5. Add model observing code
152
158
 
153
159
  ```ruby
@@ -217,10 +223,10 @@ UsersIndex.import user: User.where('rating > 100') # import only active users to
217
223
  UsersIndex.reset! # purges index and imports default data for all types
218
224
  ```
219
225
 
220
- See [actions.rb](lib/chewy/index/actions.rb) for more details.
221
-
222
226
  Also if passed user is #destroyed? or specified id is not existing in the database, import will perform `delete` index for this it
223
227
 
228
+ See [actions.rb](lib/chewy/index/actions.rb) for more details.
229
+
224
230
  ### Observing strategies
225
231
 
226
232
  There are 3 strategies for index updating: do not update index at all, update right after save and cummulative update. The first is by default.
@@ -678,69 +684,15 @@ rake chewy:update[users] # updates UsersIndex
678
684
  ### Rspec integration
679
685
 
680
686
  Just add `require 'chewy/rspec'` to your spec_helper.rb and you will get additional features:
681
-
682
- #### `update_index` matcher
683
-
684
- ```ruby
685
- # just update index expectation. Used type class as argument.
686
- specify { expect { user.save! }.to update_index(UsersIndex.user) }
687
- # expect do not update target index. Type for `update_index` might be specified via string also
688
- specify { expect { user.name = 'Duke' }.not_to update_index('users#user') }
689
- # expecting update specified objects
690
- specify { expect { user.save! }.to update_index(UsersIndex.user).and_reindex(user) }
691
- # you can specify even id
692
- specify { expect { user.save! }.to update_index(UsersIndex.user).and_reindex(42) }
693
- # expected multiple objects to be reindexed
694
- specify { expect { [user1, user2].map(&:save!) }
695
- .to update_index(UsersIndex.user).and_reindex(user1, user2) }
696
- specify { expect { [user1, user2].map(&:save!) }
697
- .to update_index(UsersIndex.user).and_reindex(user1).and_reindex(user2) }
698
- # expect object to be reindexed exact twice
699
- specify { expect { 2.times { user.save! } }
700
- .to update_index(UsersIndex.user).and_reindex(user, times: 2) }
701
- # expect object in index to be updated with specified fields
702
- specify { expect { user.update_attributes!(name: 'Duke') }
703
- .to update_index(UsersIndex.user).and_reindex(user, with: {name: 'Duke'}) }
704
- # combination of previous two
705
- specify { expect { 2.times { user.update_attributes!(name: 'Duke') } }
706
- .to update_index(UsersIndex.user).and_reindex(user, times: 2, with: {name: 'Duke'}) }
707
- # for every object
708
- specify { expect { 2.times { [user1, user2].map { |u| u.update_attributes!(name: 'Duke') } } }
709
- .to update_index(UsersIndex.user).and_reindex(user1, user2, times: 2, with: {name: 'Duke'}) }
710
- # for every object splitted
711
- specify { expect { 2.times { [user1, user2].map { |u| u.update_attributes!(name: "Duke#{u.id}") } } }
712
- .to update_index(UsersIndex.user)
713
- .and_reindex(user1, with: {name: 'Duke42'}) }
714
- .and_reindex(user2, times: 1, with: {name: 'Duke43'}) }
715
- # object deletion same abilities as `and_reindex`, except `:with` option
716
- specify { expect { user.destroy! }.to update_index(UsersIndex.user).and_delete(user) }
717
- # double deletion, whatever it means
718
- specify { expect { 2.times { user.destroy! } }.to update_index(UsersIndex.user).and_delete(user, times: 2) }
719
- # alltogether
720
- specify { expect { user1.destroy!; user2.save! } }
721
- .to update_index(UsersIndex.user).and_reindex(user2).and_delete(user1)
722
- ```
723
-
724
- ```ruby
725
- # strictly specifing updated and deleted records
726
- specify { expect { [user1, user2].map(&:save!) }
727
- .to update_index(UsersIndex.user).and_reindex(user1, user2).only }
728
- specify { expect { [user1, user2].map(&:destroy!) }
729
- .to update_index(UsersIndex.user).and_delete(user1, user2).only }
730
- # this will fail
731
- specify { expect { [user1, user2].map(&:save!) }
732
- .to update_index(UsersIndex.user).and_reindex(user1).only }
733
- ```
687
+ See [update_index.rb](lib/chewy/rspec/update_index.rb) for more details.
734
688
 
735
689
  ## TODO a.k.a coming soon:
736
690
 
737
- * Dynamic templates additional DSL
738
691
  * Typecasting support
739
692
  * Advanced (simplyfied) query DSL: `UsersIndex.query { email == 'my@gmail.com' }` will produce term query
740
693
  * update_all support
741
694
  * Other than ActiveRecord ORMs support (Mongoid)
742
695
  * Maybe, closer ORM/ODM integration, creating index classes implicitly
743
- * Better facets support
744
696
 
745
697
  ## Contributing
746
698
 
data/chewy.gemspec CHANGED
@@ -28,5 +28,5 @@ Gem::Specification.new do |spec|
28
28
  spec.add_development_dependency 'rubysl', '~> 2.0' if RUBY_ENGINE == 'rbx'
29
29
 
30
30
  spec.add_dependency 'activesupport', '>= 3.2'
31
- spec.add_dependency 'elasticsearch'
31
+ spec.add_dependency 'elasticsearch', '>= 1.0.0'
32
32
  end
@@ -0,0 +1,14 @@
1
+ source 'https://rubygems.org'
2
+
3
+ gemspec path: '../'
4
+
5
+ gem 'activerecord', '~> 3.2.0'
6
+ gem 'activesupport', '~> 3.2.0'
7
+
8
+ group :test do
9
+ gem 'guard'
10
+ gem 'guard-rspec'
11
+ gem 'rb-inotify', require: false
12
+ gem 'rb-fsevent', require: false
13
+ gem 'rb-fchange', require: false
14
+ end
@@ -0,0 +1,14 @@
1
+ source 'https://rubygems.org'
2
+
3
+ gemspec path: '../'
4
+
5
+ gem 'activerecord', '~> 4.0.0'
6
+ gem 'activesupport', '~> 4.0.0'
7
+
8
+ group :test do
9
+ gem 'guard'
10
+ gem 'guard-rspec'
11
+ gem 'rb-inotify', require: false
12
+ gem 'rb-fsevent', require: false
13
+ gem 'rb-fchange', require: false
14
+ end
data/lib/chewy.rb CHANGED
@@ -1,6 +1,7 @@
1
1
  require 'active_support/concern'
2
2
  require 'active_support/core_ext'
3
3
  require 'active_support/json'
4
+ require 'i18n/core_ext/hash'
4
5
  require 'singleton'
5
6
 
6
7
  require 'elasticsearch'
@@ -38,6 +39,10 @@ module Chewy
38
39
  end
39
40
  end
40
41
 
42
+ def self.wait_for_status
43
+ client.cluster.health wait_for_status: Chewy.configuration[:wait_for_status] if Chewy.configuration[:wait_for_status].present?
44
+ end
45
+
41
46
  def self.config
42
47
  Chewy::Config.instance
43
48
  end
@@ -40,10 +40,10 @@ module Chewy
40
40
  subfields = nested.any? ? {
41
41
  (multi_field? ? :fields : :properties) => nested.values.map(&:mappings_hash).inject(:merge)
42
42
  } : {}
43
- {name => options.merge(subfields)}
43
+ {name => options.deep_symbolize_keys.merge(subfields)}
44
44
  end
45
45
 
46
- private
46
+ private
47
47
 
48
48
  def nested_compose(value)
49
49
  nested.values.map { |field| field.compose(value) }.inject(:merge)
@@ -1,9 +1,42 @@
1
1
  module Chewy
2
2
  module Fields
3
3
  class Root < Chewy::Fields::Base
4
+ attr_reader :dynamic_templates
5
+
4
6
  def initialize(name, options = {})
5
7
  options.reverse_merge!(value: ->(_){_})
6
8
  super(name, options)
9
+ @dynamic_templates = []
10
+ end
11
+
12
+ def mappings_hash
13
+ mappings = super
14
+ if dynamic_templates.any?
15
+ mappings[name][:dynamic_templates] ||= []
16
+ mappings[name][:dynamic_templates].concat dynamic_templates
17
+ end
18
+ mappings
19
+ end
20
+
21
+ def dynamic_template *args
22
+ options = args.extract_options!.deep_symbolize_keys
23
+ if args.first
24
+ template_name = :"template_#{dynamic_templates.count.next}"
25
+ template = {template_name => {mapping: options}}
26
+
27
+ template[template_name][:match_mapping_type] = args.second.to_s if args.second.present?
28
+
29
+ regexp = args.first.is_a?(Regexp)
30
+ template[template_name][:match_pattern] = 'regexp' if regexp
31
+
32
+ match = regexp ? args.first.source : args.first
33
+ path = match.include?(regexp ? '\.' : '.')
34
+
35
+ template[template_name][path ? :path_match : :match] = match
36
+ @dynamic_templates.push(template)
37
+ else
38
+ @dynamic_templates.push(options)
39
+ end
7
40
  end
8
41
  end
9
42
  end
@@ -55,6 +55,9 @@ module Chewy
55
55
  def create! *args
56
56
  options = args.extract_options!.reverse_merge!(alias: true)
57
57
  name = build_index_name(suffix: args.first)
58
+
59
+ Chewy.wait_for_status
60
+
58
61
  result = client.indices.create(index: name, body: index_params)
59
62
  result &&= client.indices.put_alias(index: name, name: index_name) if options[:alias] && name != index_name
60
63
  result
@@ -84,6 +87,8 @@ module Chewy
84
87
  # UsersIndex.delete '01-2014' # deletes `users_01-2014` index
85
88
  #
86
89
  def delete! suffix = nil
90
+ Chewy.wait_for_status
91
+
87
92
  client.indices.delete index: build_index_name(suffix: suffix)
88
93
  end
89
94
 
@@ -126,7 +131,7 @@ module Chewy
126
131
  [:import, :import!].each do |method|
127
132
  class_eval <<-METHOD, __FILE__, __LINE__ + 1
128
133
  def #{method} options = {}
129
- objects = options.extract!(*type_names.map(&:to_sym))
134
+ objects = options.reject { |k, v| !type_names.map(&:to_sym).include?(k) }
130
135
  types.map do |type|
131
136
  args = [objects[type.type_name.to_sym], options.dup].reject(&:blank?)
132
137
  type.#{method} *args
@@ -4,7 +4,7 @@ module Chewy
4
4
  extend ActiveSupport::Concern
5
5
 
6
6
  included do
7
- singleton_class.delegate :explain, :limit, :offset, :facets, :query,
7
+ singleton_class.delegate :explain, :limit, :offset, :facets, :aggregations, :query,
8
8
  :filter, :order, :reorder, :only, :types, :none, to: :all
9
9
  end
10
10
 
data/lib/chewy/query.rb CHANGED
@@ -242,9 +242,37 @@ module Chewy
242
242
  # facets: {tags: {terms: {field: 'tags'}}, ages: {terms: {field: 'age'}}}
243
243
  # }}
244
244
  #
245
- def facets params
246
- chain { criteria.update_facets params }
245
+ # If called parameterless - returns result facets from ES performing request.
246
+ # Returns empty hash if no facets was requested or resulted.
247
+ #
248
+ def facets params = nil
249
+ if params
250
+ chain { criteria.update_facets params }
251
+ else
252
+ _response['facets'] || {}
253
+ end
254
+ end
255
+
256
+ # Sets elasticsearch <tt>aggregations</tt> search request param
257
+ #
258
+ # UsersIndex.filter{ name == 'Johny' }.aggregations(category_id: {terms: {field: 'category_ids'}})
259
+ # # => {body: {
260
+ # query: {...},
261
+ # aggregations: {
262
+ # terms: {
263
+ # field: 'category_ids'
264
+ # }
265
+ # }
266
+ # }}
267
+ #
268
+ def aggregations params = nil
269
+ if params
270
+ chain { criteria.update_aggregations params }
271
+ else
272
+ _response['aggregations'] || {}
273
+ end
247
274
  end
275
+ alias :aggs :aggregations
248
276
 
249
277
  # Marks the criteria as having zero records. This scope always returns empty array
250
278
  # without touching the elasticsearch server.
@@ -4,7 +4,9 @@ module Chewy
4
4
  class Query
5
5
  class Criteria
6
6
  include Compose
7
- STORAGES = [:options, :queries, :facets, :filters, :sort, :fields, :types]
7
+ ARRAY_STORAGES = [:queries, :filters, :sort, :fields, :types]
8
+ HASH_STORAGES = [:options, :facets, :aggregations]
9
+ STORAGES = ARRAY_STORAGES + HASH_STORAGES
8
10
 
9
11
  def initialize options = {}
10
12
  @options = options.merge(query_mode: Chewy.query_mode, filter_mode: Chewy.filter_mode)
@@ -14,7 +16,7 @@ module Chewy
14
16
  other.is_a?(self.class) && storages == other.storages
15
17
  end
16
18
 
17
- { (STORAGES - [:options, :facets]) => '[]', [:options, :facets] => '{}' }.each do |storages, default|
19
+ { ARRAY_STORAGES => '[]', HASH_STORAGES => '{}' }.each do |storages, default|
18
20
  storages.each do |storage|
19
21
  class_eval <<-METHODS, __FILE__, __LINE__ + 1
20
22
  def #{storage}
@@ -42,6 +44,10 @@ module Chewy
42
44
  facets.merge!(modifer)
43
45
  end
44
46
 
47
+ def update_aggregations(modifer)
48
+ aggregations.merge!(modifer)
49
+ end
50
+
45
51
  def update_queries(modifer)
46
52
  @queries = queries + Array.wrap(modifer).reject(&:blank?)
47
53
  end
@@ -81,6 +87,7 @@ module Chewy
81
87
  def request_body
82
88
  body = (_composed_query(_request_query, _request_filter) || {}).tap do |body|
83
89
  body.merge!(facets: facets) if facets?
90
+ body.merge!(aggregations: aggregations) if aggregations?
84
91
  body.merge!(sort: sort) if sort?
85
92
  body.merge!(_source: fields) if fields?
86
93
  end
@@ -105,7 +112,7 @@ module Chewy
105
112
  end
106
113
 
107
114
  def _request_options
108
- options.slice(:size, :from, :explain)
115
+ options.slice(:size, :from, :explain, :aggregations)
109
116
  end
110
117
 
111
118
  def _request_query
@@ -12,8 +12,8 @@ module Chewy
12
12
  def initialize name, *args
13
13
  @name = name.to_s
14
14
  @options = args.extract_options!
15
- @range = @options.extract!(:gt, :lt)
16
- @bounds = @options.extract!(:left_closed, :right_closed)
15
+ @range = @options.reject { |k, v| ![:gt, :lt].include?(k) }
16
+ @bounds = @options.reject { |k, v| ![:left_closed, :right_closed].include?(k) }
17
17
  execution = EXECUTION[args.first.to_sym] if args.first
18
18
  @options[:execution] = execution if execution
19
19
  end
@@ -5,7 +5,7 @@ module Chewy
5
5
  def initialize script, params = {}
6
6
  @script = script
7
7
  @params = params
8
- @options = params.extract!(:cache)
8
+ @options = params.reject { |k, v| ![:cache].include?(k) }
9
9
  end
10
10
 
11
11
  def __render__