chewy 0.10.0 → 6.0.0

data/Gemfile

@@ -8,9 +8,11 @@ gem 'activerecord'
 
 gem 'activejob', require: false
 gem 'resque', require: false
-gem 'shoryuken', require: false
 gem 'sidekiq', require: false
 
+gem 'aws-sdk-sqs', require: false
+gem 'shoryuken', require: false
+
 gem 'kaminari-core', require: false
 gem 'will_paginate', require: false
 
@@ -22,3 +24,5 @@ gem 'guard-rspec'
 
 gem 'redcarpet'
 gem 'yard'
+
+gem 'rexml' if RUBY_VERSION >= '3.0.0'

data/LICENSE.txt

@@ -1,4 +1,4 @@
-Copyright (c) 2013 pyromaniac
+Copyright (c) 2013-2021 Toptal, LLC
 
 MIT License
 

data/README.md

@@ -1,14 +1,11 @@
 [![Gem Version](https://badge.fury.io/rb/chewy.svg)](http://badge.fury.io/rb/chewy)
-[![Build Status](https://travis-ci.org/toptal/chewy.svg)](https://travis-ci.org/toptal/chewy)
+[![CircleCI](https://circleci.com/gh/toptal/chewy/tree/master.svg?style=svg)](https://circleci.com/gh/toptal/chewy/tree/master)
 [![Code Climate](https://codeclimate.com/github/toptal/chewy.svg)](https://codeclimate.com/github/toptal/chewy)
 [![Inline docs](http://inch-ci.org/github/toptal/chewy.svg?branch=master)](http://inch-ci.org/github/toptal/chewy)
 
-<p align="right">Sponsored by</p>
-<p align="right"><a href="https://www.toptal.com/"><img src="https://www.toptal.com/assets/public/blocks/logo/big.png" alt="Toptal" width="105" height="34"></a></p>
-
 # Chewy
 
-Chewy is an ODM and wrapper for [the official Elasticsearch client](https://github.com/elastic/elasticsearch-ruby).
+Chewy is an ODM (Object Document Mapper), built on top of the [the official Elasticsearch client](https://github.com/elastic/elasticsearch-ruby).
 
 ## Table of Contents
 
@@ -34,7 +31,7 @@ Chewy is an ODM and wrapper for [the official Elasticsearch client](https://gith
     * [Non-block notation](#non-block-notation)
     * [Designing your own strategies](#designing-your-own-strategies)
   * [Rails application strategies integration](#rails-application-strategies-integration)
-  * [ActiveSupport::Notifications support](#activesupport-notifications-support)
+  * [ActiveSupport::Notifications support](#activesupportnotifications-support)
   * [NewRelic integration](#newrelic-integration)
   * [Search requests](#search-requests)
     * [Composing requests](#composing-requests)
@@ -42,7 +39,6 @@ Chewy is an ODM and wrapper for [the official Elasticsearch client](https://gith
     * [Named scopes](#named-scopes)
     * [Scroll API](#scroll-api)
     * [Loading objects](#loading-objects)
-    * [Legacy DSL incompatibilities](#legacy-dsl-incompatibilities)
   * [Rake tasks](#rake-tasks)
     * [chewy:reset](#chewyreset)
     * [chewy:upgrade](#chewyupgrade)
@@ -51,13 +47,14 @@ Chewy is an ODM and wrapper for [the official Elasticsearch client](https://gith
     * [chewy:deploy](#chewydeploy)
     * [Parallelizing rake tasks](#parallelizing-rake-tasks)
     * [chewy:journal](#chewyjournal)
-  * [Rspec integration](#rspec-integration)
+  * [RSpec integration](#rspec-integration)
   * [Minitest integration](#minitest-integration)
-* [TODO a.k.a coming soon](#todo-aka-coming-soon)
 * [Contributing](#contributing)
 
 ## Why Chewy?
 
+In this section we'll cover why you might want to use Chewy instead of the official `elasticsearch-ruby` client gem.
+
 * Multi-model indices.
 
   Index classes are independent from ORM/ODM models. Now, implementing e.g. cross-model autocomplete is much easier. You can just define the index and work with it in an object-oriented style. You can define several types for index - one per indexed model.
@@ -76,10 +73,9 @@ Chewy is an ODM and wrapper for [the official Elasticsearch client](https://gith
 
 * Support for ActiveRecord, [Mongoid](https://github.com/mongoid/mongoid) and [Sequel](https://github.com/jeremyevans/sequel).
 
-
 ## Installation
 
-Add this line to your application's Gemfile:
+Add this line to your application's `Gemfile`:
 
     gem 'chewy'
 
@@ -91,13 +87,33 @@ Or install it yourself as:
 
     $ gem install chewy
 
+## Compatibility
+
+### Ruby
+
+Chewy is compatible with MRI 2.5-3.0¹.
+
+> ¹ Ruby 3 is only supported with Rails 6.1
+
+### Elasticsearch
+
+Chewy 5 is compatible with Elasticsearch 5.
+
+Chewy 6 is compatible with Elasticsearch 6. See [Migration guide](migration_guide.md).
+
+Future versions of Chewy will support Elasticsearch 7.
+
 ## Usage
 
 ### Client settings
 
-There are two ways to configure the Chewy client: the `Chewy.settings` hash and `chewy.yml`
+There are two ways to configure the Chewy client:
+
+* via the hash `Chewy.settings`
+* via the configuration file `chewy.yml`
 
-You can create this file manually or run `rails g chewy:install`.
+You can create `chewy.yml` manually or run `rails g chewy:install` to
+generate it.
 
 ```ruby
 # config/initializers/chewy.rb
@@ -129,25 +145,29 @@ Chewy.logger = Logger.new(STDOUT)
 
 See [config.rb](lib/chewy/config.rb) for more details.
 
-#### Aws Elastic Search
-If you would like to use AWS's ElasticSearch using an IAM user policy, you will need to sign your requests for the `es:*` action by injecting the appropriate headers passing a proc to `transport_options`.
+#### AWS Elasticsearch
+
+If you would like to use AWS's Elasticsearch using an IAM user policy, you will need to sign your requests for the `es:*` action by injecting the appropriate headers passing a proc to `transport_options`.
+You'll need an additional gem for Faraday middleware: add `gem 'faraday_middleware-aws-sigv4'` to your Gemfile.
 
 ```ruby
- Chewy.settings = {
-    host: 'http://my-es-instance-on-aws.us-east-1.es.amazonaws.com:80',
-    transport_options: {
-      headers: { content_type: 'application/json' },
-      proc: -> (f) do
-          f.request :aws_signers_v4,
-                    service_name: 'es',
-                    region: 'us-east-1',
-                    credentials: Aws::Credentials.new(
-                      ENV['AWS_ACCESS_KEY'],
-                      ENV['AWS_SECRET_ACCESS_KEY'])
-      end
-    }
+require 'faraday_middleware/aws_sigv4'
+
+Chewy.settings = {
+  host: 'http://my-es-instance-on-aws.us-east-1.es.amazonaws.com:80',
+  port: 80, # 443 for https host
+  transport_options: {
+    headers: { content_type: 'application/json' },
+    proc: -> (f) do
+        f.request :aws_sigv4,
+                  service: 'es',
+                  region: 'us-east-1',
+                  access_key_id: ENV['AWS_ACCESS_KEY'],
+                  secret_access_key: ENV['AWS_SECRET_ACCESS_KEY']
+    end
   }
-  ```
+}
+```
 
 ### Index definition
 
@@ -180,7 +200,7 @@ If you would like to use AWS's ElasticSearch using an IAM user policy, you will
       field :badges, value: ->(user) { user.badges.map(&:name) } # passing array values to index
       field :projects do # the same block syntax for multi_field, if `:type` is specified
         field :title
-        field :description # default data type is `string`
+        field :description # default data type is `text`
         # additional top-level objects passed to value proc:
         field :categories, value: ->(project, user) { project.categories.map(&:name) if user.active? }
       end
@@ -208,7 +228,7 @@ If you would like to use AWS's ElasticSearch using an IAM user policy, you will
 
     define_type User.active.includes(:country, :badges, :projects) do
       root date_detection: false do
-        template 'about_translations.*', type: 'string', analyzer: 'standard'
+        template 'about_translations.*', type: 'text', analyzer: 'standard'
 
         field :first_name, :last_name
         field :email, analyzer: 'email'
@@ -228,7 +248,7 @@ If you would like to use AWS's ElasticSearch using an IAM user policy, you will
   ```
 
   [See index settings here](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-update-settings.html).
-  [See root object settings here](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-root-object-type.html).
+  [See root object settings here](https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic-field-mapping.html).
 
   See [mapping.rb](lib/chewy/type/mapping.rb) for more details.
 
@@ -335,9 +355,9 @@ This will automatically set the type or root field to `object`. You may also spe
 To define a multi field you have to specify any type except for `object` or `nested` in the root field:
 
 ```ruby
-field :full_name, type: 'string', value: ->{ full_name.strip } do
+field :full_name, type: 'text', value: ->{ full_name.strip } do
   field :ordered, analyzer: 'ordered'
-  field :untouched, index: 'not_analyzed'
+  field :untouched, type: 'keyword'
 end
 ```
 
@@ -357,7 +377,7 @@ end
 ```
 ### Geo Point fields
 
-You can use [Elasticsearch's geo mapping](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-geo-point-type.html) with the `geo_point` field type, allowing you to query, filter and order by latitude and longitude. You can use the following hash format:
+You can use [Elasticsearch's geo mapping](https://www.elastic.co/guide/en/elasticsearch/reference/current/geo-point.html) with the `geo_point` field type, allowing you to query, filter and order by latitude and longitude. You can use the following hash format:
 
 ```ruby
 field :coordinates, type: 'geo_point', value: ->{ {lat: latitude, lon: longitude} }
@@ -678,6 +698,11 @@ Chewy.strategy(:sidekiq) do
 end
 ```
 
+The default queue name is `chewy`, you can customize it in settings: `sidekiq.queue_name`
+```
+Chewy.settings[:sidekiq] = {queue: :low}
+```
+
 #### `:active_job`
 
 This does the same thing as `:atomic`, but using ActiveJob. This will inherit the ActiveJob configuration settings including the `active_job.queue_adapter` setting for the environment. Patch `Chewy::Strategy::ActiveJob::Worker` for index updates improving.
@@ -688,6 +713,11 @@ Chewy.strategy(:active_job) do
 end
 ```
 
+The default queue name is `chewy`, you can customize it in settings: `active_job.queue_name`
+```
+Chewy.settings[:active_job] = {queue: :low}
+```
+
 #### `:shoryuken`
 
 This does the same thing as `:atomic`, but asynchronously using shoryuken. Patch `Chewy::Strategy::Shoryuken::Worker` for index updates improving.
@@ -807,38 +837,84 @@ Chewy has notifying the following events:
 To integrate with NewRelic you may use the following example source (config/initializers/chewy.rb):
 
 ```ruby
-ActiveSupport::Notifications.subscribe('import_objects.chewy') do |name, start, finish, id, payload|
-  metric_name = "Database/ElasticSearch/import"
-  duration = (finish - start).to_f
-  logged = "#{payload[:type]} #{payload[:import].to_a.map{ |i| i.join(':') }.join(', ')}"
-
-  self.class.trace_execution_scoped([metric_name]) do
-    NewRelic::Agent.instance.transaction_sampler.notice_sql(logged, nil, duration)
-    NewRelic::Agent.instance.sql_sampler.notice_sql(logged, metric_name, nil, duration)
-    NewRelic::Agent.record_metric(metric_name, duration)
+require 'new_relic/agent/instrumentation/evented_subscriber'
+
+class ChewySubscriber < NewRelic::Agent::Instrumentation::EventedSubscriber
+  def start(name, id, payload)
+    event = ChewyEvent.new(name, Time.current, nil, id, payload)
+    push_event(event)
+  end
+
+  def finish(_name, id, _payload)
+    pop_event(id).finish
   end
-end
 
-ActiveSupport::Notifications.subscribe('search_query.chewy') do |name, start, finish, id, payload|
-  metric_name = "Database/ElasticSearch/search"
-  duration = (finish - start).to_f
-  logged = "#{payload[:type].presence || payload[:index]} #{payload[:request]}"
+  class ChewyEvent < NewRelic::Agent::Instrumentation::Event
+    OPERATIONS = {
+      'import_objects.chewy' => 'import',
+      'search_query.chewy' => 'search',
+      'delete_query.chewy' => 'delete'
+    }.freeze
 
-  self.class.trace_execution_scoped([metric_name]) do
-    NewRelic::Agent.instance.transaction_sampler.notice_sql(logged, nil, duration)
-    NewRelic::Agent.instance.sql_sampler.notice_sql(logged, metric_name, nil, duration)
-    NewRelic::Agent.record_metric(metric_name, duration)
+    def initialize(*args)
+      super
+      @segment = start_segment
+    end
+
+    def start_segment
+      segment = NewRelic::Agent::Transaction::DatastoreSegment.new product, operation, collection, host, port
+      if (txn = state.current_transaction)
+        segment.transaction = txn
+      end
+      segment.notice_sql @payload[:request].to_s
+      segment.start
+      segment
+    end
+
+    def finish
+      if (txn = state.current_transaction)
+        txn.add_segment @segment
+      end
+      @segment.finish
+    end
+
+    private
+
+    def state
+      @state ||= NewRelic::Agent::TransactionState.tl_get
+    end
+
+    def product
+      'Elasticsearch'
+    end
+
+    def operation
+      OPERATIONS[name]
+    end
+
+    def collection
+      payload.values_at(:type, :index)
+             .reject { |value| value.try(:empty?) }
+             .first
+             .to_s
+    end
+
+    def host
+      Chewy.client.transport.hosts.first[:host]
+    end
+
+    def port
+      Chewy.client.transport.hosts.first[:port]
+    end
   end
 end
+
+ActiveSupport::Notifications.subscribe(/.chewy$/, ChewySubscriber.new)
 ```
 
 ### Search requests
 
-Long story short: there is a new DSL that supports ES2 and ES5, the previous DSL version (which supports ES1 and ES2) documentation was moved to [LEGACY_DSL.md](LEGACY_DSL.md).
-
-If you want to use it - simply do `Chewy.search_class = Chewy::Query` somewhere before indices are initialized.
-
-The new DSL is enabled by default, here is a quick introduction.
+Quick introduction.
 
 #### Composing requests
 
@@ -906,17 +982,6 @@ scope.each do |wrapper|
 end
 ```
 
-#### Legacy DSL incompatibilities
-
-* Filters advanced block DSL is not supported anymore, `elasticsearch-dsl` is used instead.
-* Things like `query_mode` and `filter_mode` are in past, use advanced DSL to achieve similar behavior. See [Chewy::Search::QueryProxy](lib/chewy/search/query_proxy.rb) for details.
-* `preload` method is no more, the collection returned by scope doesn't depend on loading options, scope always returns `Chewy::Type` wrappers. To get ORM/ODM objects, use `#objects` method.
-* Some of the methods have changed their purpose: `only` was used to filter fields before, now it filters the scope. To filter fields use `source` or `stored_fields`.
-* `types!` method is no more, use `except(:types).types(...)`
-* Named aggregations are not supported, use named scopes instead.
-* A lot of query-level methods were not ported: everything that is related to boost and scoring. Use `query` manipulation to provide them.
-* `Chewy::Type#_object` returns nil always. Use `Chewy::Search::Response#object_hash` instead.
-
 ### Rake tasks
 
 For a Rails application, some index-maintaining rake tasks are defined.
@@ -936,9 +1001,9 @@ rake chewy:reset[-users,places] # resets every index in the application except s
 
 Performs reset exactly the same way as `chewy:reset` does, but only when the index specification (setting or mapping) was changed.
 
-It works only when index specification is locked in `Chewy::Stash` index. The first run will reset all indexes and lock their specifications.
+It works only when index specification is locked in `Chewy::Stash::Specification` index. The first run will reset all indexes and lock their specifications.
 
-See [Chewy::Stash](lib/chewy/stash.rb) and [Chewy::Index::Specification](lib/chewy/index/specification.rb) for more details.
+See [Chewy::Stash::Specification](lib/chewy/stash.rb) and [Chewy::Index::Specification](lib/chewy/index/specification.rb) for more details.
 
 
 ```bash
@@ -1011,7 +1076,7 @@ rake chewy:journal:apply["$(date -v-1H -u +%FT%TZ)"] # apply journaled changes f
 rake chewy:journal:apply["$(date -v-1H -u +%FT%TZ)",users] # apply journaled changes for the past hour on UsersIndex only
 ```
 
-### Rspec integration
+### RSpec integration
 
 Just add `require 'chewy/rspec'` to your spec_helper.rb and you will get additional features: See [update_index.rb](lib/chewy/rspec/update_index.rb) for more details.
 
@@ -1032,12 +1097,6 @@ If you use `DatabaseCleaner` in your tests with [the `transaction` strategy](htt
 Chewy.use_after_commit_callbacks = !Rails.env.test?
 ```
 
-## TODO a.k.a coming soon:
-
-* Typecasting support
-* update_all support
-* Maybe, closer ORM/ODM integration, creating index classes implicitly
-
 ## Contributing
 
 1. Fork it (http://github.com/toptal/chewy/fork)
@@ -1053,3 +1112,8 @@ Use the following Rake tasks to control the Elasticsearch cluster while developi
 rake elasticsearch:start # start Elasticsearch cluster on 9250 port for tests
 rake elasticsearch:stop # stop Elasticsearch
 ```
+
+## Copyright
+
+Copyright (c) 2013-2021 Toptal, LLC. See [LICENSE.txt](LICENSE.txt) for
+further details.

data/chewy.gemspec

@@ -1,5 +1,3 @@
-# coding: utf-8
-
 lib = File.expand_path('../lib', __FILE__)
 $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
 require 'chewy/version'
@@ -7,8 +5,8 @@ require 'chewy/version'
 Gem::Specification.new do |spec| # rubocop:disable BlockLength
   spec.name          = 'chewy'
   spec.version       = Chewy::VERSION
-  spec.authors       = ['pyromaniac']
-  spec.email         = ['kinwizard@gmail.com']
+  spec.authors       = ['Toptal, LLC', 'pyromaniac']
+  spec.email         = ['open-source@toptal.com', 'kinwizard@gmail.com']
   spec.summary       = 'Elasticsearch ODM client wrapper'
   spec.description   = 'Chewy provides functionality for Elasticsearch index handling, documents import mappings and chainable query DSL'
   spec.homepage      = 'https://github.com/toptal/chewy'
@@ -19,22 +17,22 @@ Gem::Specification.new do |spec| # rubocop:disable BlockLength
   spec.test_files    = spec.files.grep(%r{^(test|spec|features)/})
   spec.require_paths = ['lib']
 
-  spec.add_development_dependency 'rake'
   spec.add_development_dependency 'appraisal'
-  spec.add_development_dependency 'rspec'
-  spec.add_development_dependency 'rspec-its'
-  spec.add_development_dependency 'rspec-collection_matchers'
-  spec.add_development_dependency 'sqlite3'
   spec.add_development_dependency 'database_cleaner'
   spec.add_development_dependency 'elasticsearch-extensions'
+  spec.add_development_dependency 'rake'
   spec.add_development_dependency 'resque_spec'
+  spec.add_development_dependency 'rspec', '>= 3.7.0'
+  spec.add_development_dependency 'rspec-collection_matchers'
+  spec.add_development_dependency 'rspec-its'
+  spec.add_development_dependency 'rubocop', '0.52.1'
+  spec.add_development_dependency 'sqlite3'
   spec.add_development_dependency 'timecop'
-  spec.add_development_dependency 'rubocop', '0.48.1'
 
   spec.add_development_dependency 'method_source'
   spec.add_development_dependency 'unparser'
 
-  spec.add_dependency 'activesupport', '>= 4.0'
-  spec.add_dependency 'elasticsearch', '>= 2.0.0'
+  spec.add_dependency 'activesupport', '>= 5.2'
+  spec.add_dependency 'elasticsearch', '>= 6.3.0'
   spec.add_dependency 'elasticsearch-dsl'
 end