chewy 7.6.0 → 8.0.0

data/docs/indexing.md

@@ -0,0 +1,329 @@
+# Indexing
+
+## Index definition
+
+1. Create `/app/chewy/users_index.rb`
+
+  ```ruby
+  class UsersIndex < Chewy::Index
+
+  end
+  ```
+
+2. Define index scope (you can omit this part if you don't need to specify a scope (i.e. use PORO objects for import) or options)
+
+  ```ruby
+  class UsersIndex < Chewy::Index
+    index_scope User.active # or just model instead_of scope: index_scope User
+  end
+  ```
+
+3. Add some mappings
+
+  ```ruby
+  class UsersIndex < Chewy::Index
+    index_scope User.active.includes(:country, :badges, :projects)
+    field :first_name, :last_name # multiple fields without additional options
+    field :email, analyzer: 'email' # Elasticsearch-related options
+    field :country, value: ->(user) { user.country.name } # custom value proc
+    field :badges, value: ->(user) { user.badges.map(&:name) } # passing array values to index
+    field :projects do # the same block syntax for multi_field, if `:type` is specified
+      field :title
+      field :description # default data type is `text`
+      # additional top-level objects passed to value proc:
+      field :categories, value: ->(project, user) { project.categories.map(&:name) if user.active? }
+    end
+    field :rating, type: 'integer' # custom data type
+    field :created, type: 'date', include_in_all: false,
+      value: ->{ created_at } # value proc for source object context
+  end
+  ```
+
+  [See here for mapping definitions](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html).
+
+4. Add some index-related settings. Analyzer repositories might be used as well. See `Chewy::Index.settings` docs for details:
+
+  ```ruby
+  class UsersIndex < Chewy::Index
+    settings analysis: {
+      analyzer: {
+        email: {
+          tokenizer: 'keyword',
+          filter: ['lowercase']
+        }
+      }
+    }
+
+    index_scope User.active.includes(:country, :badges, :projects)
+    root date_detection: false do
+      template 'about_translations.*', type: 'text', analyzer: 'standard'
+
+      field :first_name, :last_name
+      field :email, analyzer: 'email'
+      field :country, value: ->(user) { user.country.name }
+      field :badges, value: ->(user) { user.badges.map(&:name) }
+      field :projects do
+        field :title
+        field :description
+      end
+      field :about_translations, type: 'object' # pass object type explicitly if necessary
+      field :rating, type: 'integer'
+      field :created, type: 'date', include_in_all: false,
+        value: ->{ created_at }
+    end
+  end
+  ```
+
+  [See index settings here](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-update-settings.html).
+  [See root object settings here](https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic-field-mapping.html).
+
+  See [mapping.rb](../lib/chewy/index/mapping.rb) for more details.
+
+5. Add model-observing code
+
+  ```ruby
+  class User < ActiveRecord::Base
+    update_index('users') { self } # specifying index and back-reference
+                                        # for updating after user save or destroy
+  end
+
+  class Country < ActiveRecord::Base
+    has_many :users
+
+    update_index('users') { users } # return single object or collection
+  end
+
+  class Project < ActiveRecord::Base
+    update_index('users') { user if user.active? } # you can return even `nil` from the back-reference
+  end
+
+  class Book < ActiveRecord::Base
+    update_index(->(book) {"books_#{book.language}"}) { self } # dynamic index name with proc.
+                                                               # For book with language == "en"
+                                                               # this code will generate `books_en`
+  end
+  ```
+
+  The `update_index` callback requires an active update strategy to be set. See [configuration.md](configuration.md#index-update-strategies) for available strategies and how they integrate with Rails.
+
+  Also, you can use the second argument for method name passing:
+
+  ```ruby
+  update_index('users', :self)
+  update_index('users', :users)
+  ```
+
+  In the case of a belongs_to association you may need to update both associated objects, previous and current:
+
+  ```ruby
+  class City < ActiveRecord::Base
+    belongs_to :country
+
+    update_index('cities') { self }
+    update_index 'countries' do
+      previous_changes['country_id'] || country
+    end
+  end
+  ```
+
+## Multi (nested) and object field types
+
+To define an objects field you can simply nest fields in the DSL:
+
+```ruby
+field :projects do
+  field :title
+  field :description
+end
+```
+
+This will automatically set the type or root field to `object`. You may also specify `type: 'objects'` explicitly.
+
+To define a multi field you have to specify any type except for `object` or `nested` in the root field:
+
+```ruby
+field :full_name, type: 'text', value: ->{ full_name.strip } do
+  field :ordered, analyzer: 'ordered'
+  field :untouched, type: 'keyword'
+end
+```
+
+The `value:` option for internal fields will no longer be effective.
+
+## Geo Point fields
+
+You can use [Elasticsearch's geo mapping](https://www.elastic.co/guide/en/elasticsearch/reference/current/geo-point.html) with the `geo_point` field type, allowing you to query, filter and order by latitude and longitude. You can use the following hash format:
+
+```ruby
+field :coordinates, type: 'geo_point', value: ->{ {lat: latitude, lon: longitude} }
+```
+
+or by using nested fields:
+
+```ruby
+field :coordinates, type: 'geo_point' do
+  field :lat, value: ->{ latitude }
+  field :long, value: ->{ longitude }
+end
+```
+
+See the section on *Script fields* for details on calculating distance in a search.
+
+## Join fields
+
+You can use a [join field](https://www.elastic.co/guide/en/elasticsearch/reference/current/parent-join.html)
+to implement parent-child relationships between documents.
+It [replaces the old `parent_id` based parent-child mapping](https://www.elastic.co/guide/en/elasticsearch/reference/current/removal-of-types.html#parent-child-mapping-types)
+
+To use it, you need to pass `relations` and `join` (with `type` and `id`) options:
+```ruby
+field :hierarchy_link, type: :join, relations: {question: %i[answer comment], answer: :vote, vote: :subvote}, join: {type: :comment_type, id: :commented_id}
+```
+assuming you have `comment_type` and `commented_id` fields in your model.
+
+Note that when you reindex a parent, its children and grandchildren will be reindexed as well.
+This may require additional queries to the primary database and to Elasticsearch.
+
+Also note that the join field doesn't support crutches (it should be a field directly defined on the model).
+
+## Crutches technology
+
+Assume you are defining your index like this (product has_many categories through product_categories):
+
+```ruby
+class ProductsIndex < Chewy::Index
+  index_scope Product.includes(:categories)
+  field :name
+  field :category_names, value: ->(product) { product.categories.map(&:name) } # or shorter just -> { categories.map(&:name) }
+end
+```
+
+Then the Chewy reindexing flow will look like the following pseudo-code:
+
+```ruby
+Product.includes(:categories).find_in_batches(1000) do |batch|
+  bulk_body = batch.map do |object|
+    {name: object.name, category_names: object.categories.map(&:name)}.to_json
+  end
+  # here we are sending every batch of data to ES
+  Chewy.client.bulk bulk_body
+end
+```
+
+If you meet complicated cases when associations are not applicable you can replace Rails associations with Chewy Crutches technology:
+
+```ruby
+class ProductsIndex < Chewy::Index
+  index_scope Product
+  crutch :categories do |collection| # collection here is a current batch of products
+    # data is fetched with a lightweight query without objects initialization
+    data = ProductCategory.joins(:category).where(product_id: collection.map(&:id)).pluck(:product_id, 'categories.name')
+    # then we have to convert fetched data to appropriate format
+    # this will return our data in structure like:
+    # {123 => ['sweets', 'juices'], 456 => ['meat']}
+    data.each.with_object({}) { |(id, name), result| (result[id] ||= []).push(name) }
+  end
+
+  field :name
+  # simply use crutch-fetched data as a value:
+  field :category_names, value: ->(product, crutches) { crutches[:categories][product.id] }
+end
+```
+
+An example flow will look like this:
+
+```ruby
+Product.includes(:categories).find_in_batches(1000) do |batch|
+  crutches[:categories] = ProductCategory.joins(:category).where(product_id: batch.map(&:id)).pluck(:product_id, 'categories.name')
+    .each.with_object({}) { |(id, name), result| (result[id] ||= []).push(name) }
+
+  bulk_body = batch.map do |object|
+    {name: object.name, category_names: crutches[:categories][object.id]}.to_json
+  end
+  Chewy.client.bulk bulk_body
+end
+```
+
+So Chewy Crutches technology is able to increase your indexing performance in some cases up to a hundredfold or even more depending on your associations complexity. For another approach to import performance, see [Raw import](import.md#raw-import).
+
+## Witchcraft technology
+
+One more experimental technology to increase import performance. As far as you know, chewy defines value proc for every imported field in mapping, so at the import time each of these procs is executed on imported object to extract result document to import. It would be great for performance to use one huge whole-document-returning proc instead. So basically the idea or Witchcraft technology is to compile a single document-returning proc from the index definition.
+
+```ruby
+index_scope Product
+witchcraft!
+
+field :title
+field :tags, value: -> { tags.map(&:name) }
+field :categories do
+  field :name, value: -> (product, category) { category.name }
+  field :type, value: -> (product, category, crutch) { crutch.types[category.name] }
+end
+```
+
+The index definition above will be compiled to something close to:
+
+```ruby
+-> (object, crutches) do
+  {
+    title: object.title,
+    tags: object.tags.map(&:name),
+    categories: object.categories.map do |object2|
+      {
+        name: object2.name
+        type: crutches.types[object2.name]
+      }
+    end
+  }
+end
+```
+
+And don't even ask how is it possible, it is a witchcraft.
+Obviously not every type of definition might be compiled. There are some restrictions:
+
+1. Use reasonable formatting to make `method_source` be able to extract field value proc sources.
+2. Value procs with splat arguments are not supported right now.
+3. If you are generating fields dynamically use value proc with arguments, argumentless value procs are not supported yet:
+
+  ```ruby
+  [:first_name, :last_name].each do |name|
+    field name, value: -> (o) { o.send(name) }
+  end
+  ```
+
+However, it is quite possible that your index definition will be supported by Witchcraft technology out of the box in most of the cases.
+
+## Index manipulation
+
+```ruby
+UsersIndex.delete # destroy index if it exists
+UsersIndex.delete!
+
+UsersIndex.create
+UsersIndex.create! # use bang or non-bang methods
+
+UsersIndex.purge
+UsersIndex.purge! # deletes then creates index
+
+UsersIndex.import # import with 0 arguments process all the data specified in index_scope definition
+UsersIndex.import User.where('rating > 100') # or import specified users scope
+UsersIndex.import User.where('rating > 100').to_a # or import specified users array
+UsersIndex.import [1, 2, 42] # pass even ids for import, it will be handled in the most effective way
+UsersIndex.import User.where('rating > 100'), update_fields: [:email] # if update fields are specified - it will update their values only with the `update` bulk action
+UsersIndex.import! # raises an exception in case of any import errors
+
+UsersIndex.reset! # purges index and imports default data for all types
+```
+
+For more on import options, batching and journaling, see [import.md](import.md).
+
+If the passed user is `#destroyed?`, or satisfies a `delete_if` index_scope option, or the specified id does not exist in the database, import will perform delete from index action for this object.
+
+```ruby
+index_scope User, delete_if: :deleted_at
+index_scope User, delete_if: -> { deleted_at }
+index_scope User, delete_if: ->(user) { user.deleted_at }
+```
+
+See [actions.rb](../lib/chewy/index/actions.rb) for more details.

data/docs/querying.md

@@ -0,0 +1,72 @@
+# Querying
+
+## Composing requests
+
+The request DSL have the same chainable nature as AR. The main class is `Chewy::Search::Request`.
+
+```ruby
+CitiesIndex.query(match: {name: 'London'})
+```
+
+Main methods of the request DSL are: `query`, `filter` and `post_filter`, it is possible to pass pure query hashes or use `elasticsearch-dsl`.
+
+```ruby
+CitiesIndex
+  .filter(term: {name: 'Bangkok'})
+  .query(match: {name: 'London'})
+  .query.not(range: {population: {gt: 1_000_000}})
+```
+
+You can query a set of indexes at once:
+
+```ruby
+CitiesIndex.indices(CountriesIndex).query(match: {name: 'Some'})
+```
+
+See https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html and https://github.com/elastic/elasticsearch-dsl-ruby for more details.
+
+An important part of requests manipulation is merging. There are 4 methods to perform it: `merge`, `and`, `or`, `not`. See [Chewy::Search::QueryProxy](../lib/chewy/search/query_proxy.rb) for details. Also, `only` and `except` methods help to remove unneeded parts of the request.
+
+Every other request part is covered by a bunch of additional methods, see [Chewy::Search::Request](../lib/chewy/search/request.rb) for details:
+
+```ruby
+CitiesIndex.limit(10).offset(30).order(:name, {population: {order: :desc}})
+```
+
+Request DSL also provides additional scope actions, like `delete_all`, `exists?`, `count`, `pluck`, etc.
+
+## Pagination
+
+The request DSL supports pagination with `Kaminari`. An extension is enabled on initialization if `Kaminari` is available. See [Chewy::Search](../lib/chewy/search.rb) and [Chewy::Search::Pagination::Kaminari](../lib/chewy/search/pagination/kaminari.rb) for details.
+
+## Named scopes
+
+Chewy supports named scopes functionality. There is no specialized DSL for named scopes definition, it is simply about defining class methods.
+
+See [Chewy::Search::Scoping](../lib/chewy/search/scoping.rb) for details.
+
+## Scroll API
+
+Elasticsearch scroll API is utilized by a bunch of methods: `scroll_batches`, `scroll_hits`, `scroll_wrappers` and `scroll_objects`.
+
+See [Chewy::Search::Scrolling](../lib/chewy/search/scrolling.rb) for details.
+
+## Loading objects
+
+It is possible to load ORM/ODM source objects with the `objects` method. To provide additional loading options use `load` method:
+
+```ruby
+CitiesIndex.load(scope: -> { active }).to_a # to_a returns `Chewy::Index` wrappers.
+CitiesIndex.load(scope: -> { active }).objects # An array of AR source objects.
+```
+
+See [Chewy::Search::Loader](../lib/chewy/search/loader.rb) for more details.
+
+In case when it is necessary to iterate through both of the wrappers and objects simultaneously, `object_hash` method helps a lot:
+
+```ruby
+scope = CitiesIndex.load(scope: -> { active })
+scope.each do |wrapper|
+  scope.object_hash[wrapper]
+end
+```

data/docs/rake_tasks.md

@@ -0,0 +1,108 @@
+# Rake Tasks
+
+For a Rails application, some index-maintaining rake tasks are defined.
+
+## `chewy:reset`
+
+Performs zero-downtime reindexing as described [here](https://www.elastic.co/blog/changing-mapping-with-zero-downtime). So the rake task creates a new index with unique suffix and then simply aliases it to the common index name. The previous index is deleted afterwards (see `Chewy::Index.reset!` for more details).
+
+```bash
+rake chewy:reset # resets all the existing indices
+rake chewy:reset[users] # resets UsersIndex only
+rake chewy:reset[users,cities] # resets UsersIndex and CitiesIndex
+rake chewy:reset[-users,cities] # resets every index in the application except specified ones
+```
+
+## `chewy:upgrade`
+
+Performs reset exactly the same way as `chewy:reset` does, but only when the index specification (setting or mapping) was changed.
+
+It works only when index specification is locked in `Chewy::Stash::Specification` index. The first run will reset all indexes and lock their specifications.
+
+See [Chewy::Stash::Specification](../lib/chewy/stash.rb) and [Chewy::Index::Specification](../lib/chewy/index/specification.rb) for more details.
+
+
+```bash
+rake chewy:upgrade # upgrades all the existing indices
+rake chewy:upgrade[users] # upgrades UsersIndex only
+rake chewy:upgrade[users,cities] # upgrades UsersIndex and CitiesIndex
+rake chewy:upgrade[-users,cities] # upgrades every index in the application except specified ones
+```
+
+## `chewy:update`
+
+It doesn't create indexes, it simply imports everything to the existing ones and fails if the index was not created before.
+
+```bash
+rake chewy:update # updates all the existing indices
+rake chewy:update[users] # updates UsersIndex only
+rake chewy:update[users,cities] # updates UsersIndex and CitiesIndex
+rake chewy:update[-users,cities] # updates every index in the application except UsersIndex and CitiesIndex
+```
+
+## `chewy:sync`
+
+Provides a way to synchronize outdated indexes with the source quickly and without doing a full reset. By default field `updated_at` is used to find outdated records, but this could be customized by `outdated_sync_field` as described at [Chewy::Index::Syncer](../lib/chewy/index/syncer.rb).
+
+Arguments are similar to the ones taken by `chewy:update` task.
+
+See [Chewy::Index::Syncer](../lib/chewy/index/syncer.rb) for more details.
+
+```bash
+rake chewy:sync # synchronizes all the existing indices
+rake chewy:sync[users] # synchronizes UsersIndex only
+rake chewy:sync[users,cities] # synchronizes UsersIndex and CitiesIndex
+rake chewy:sync[-users,cities] # synchronizes every index in the application except UsersIndex and CitiesIndex
+```
+
+## `chewy:deploy`
+
+This rake task is especially useful during the production deploy. It is a combination of `chewy:upgrade` and `chewy:sync` and the latter is called only for the indexes that were not reset during the first stage.
+
+It is not possible to specify any particular indexes for this task as it doesn't make much sense.
+
+Right now the approach is that if some data had been updated, but index definition was not changed (no changes satisfying the synchronization algorithm were done), it would be much faster to perform manual partial index update inside data migrations or even manually after the deploy.
+
+Also, there is always full reset alternative with `rake chewy:reset`. See [configuration.md](configuration.md#index-update-strategies) for how update strategies interact with deployment.
+
+## `chewy:create_missing_indexes`
+
+This rake task creates newly defined indexes in Elasticsearch and skips existing ones. Useful for production-like environments.
+
+## Parallelizing rake tasks
+
+Every task described above has its own parallel version. Every parallel rake task takes the number for processes for execution as the first argument and the rest of the arguments are exactly the same as for the non-parallel task version.
+
+[https://github.com/grosser/parallel](https://github.com/grosser/parallel) gem is required to use these tasks.
+
+If the number of processes is not specified explicitly - `parallel` gem tries to automatically derive the number of processes to use.
+
+```bash
+rake chewy:parallel:reset
+rake chewy:parallel:upgrade[4]
+rake chewy:parallel:update[4,cities]
+rake chewy:parallel:sync[4,-users]
+rake chewy:parallel:deploy[4] # performs parallel upgrade and parallel sync afterwards
+```
+
+## `chewy:journal`
+
+This namespace contains two tasks for the journal manipulations: `chewy:journal:apply` and `chewy:journal:clean`. Both are taking time as the first argument (optional for clean) and a list of indexes exactly as the tasks above. Time can be in any format parsable by ActiveSupport.
+
+```bash
+rake chewy:journal:apply["$(date -v-1H -u +%FT%TZ)"] # apply journaled changes for the past hour
+rake chewy:journal:apply["$(date -v-1H -u +%FT%TZ)",users] # apply journaled changes for the past hour on UsersIndex only
+```
+
+See [import.md](import.md#journaling) for how journaling works and how to enable it.
+
+When the size of the journal becomes very large, the classical way of deletion would be obstructive and resource consuming. Fortunately, Chewy internally uses [delete-by-query](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/docs-delete-by-query.html#docs-delete-by-query-task-api) ES function which supports async execution with batching and [throttling](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-delete-by-query.html#docs-delete-by-query-throttle).
+
+The available options, which can be set by ENV variables, are listed below:
+* `WAIT_FOR_COMPLETION` - a boolean flag. It controls async execution. It waits by default. When set to `false` (`0`, `f`, `false` or `off` in any case spelling is accepted as `false`), Elasticsearch performs some preflight checks, launches the request, and returns a task reference you can use to cancel the task or get its status.
+* `REQUESTS_PER_SECOND` - float. The throttle for this request in sub-requests per second. No throttling is enforced by default.
+* `SCROLL_SIZE` - integer. The number of documents to be deleted in single sub-request. The default batch size is 1000.
+
+```bash
+rake chewy:journal:clean WAIT_FOR_COMPLETION=false REQUESTS_PER_SECOND=10 SCROLL_SIZE=5000
+```

data/docs/testing.md

@@ -0,0 +1,41 @@
+# Testing
+
+## RSpec integration
+
+Just add `require 'chewy/rspec'` to your spec_helper.rb and you will get additional features:
+
+[update_index](../lib/chewy/rspec/update_index.rb) helper
+`mock_elasticsearch_response` helper to mock elasticsearch response
+`mock_elasticsearch_response_sources` helper to mock elasticsearch response sources
+`build_query` matcher to compare request and expected query (returns `true`/`false`)
+
+To use `mock_elasticsearch_response` and `mock_elasticsearch_response_sources` helpers add `include Chewy::Rspec::Helpers` to your tests.
+
+See [chewy/rspec/](../lib/chewy/rspec/) for more details.
+
+## Minitest integration
+
+Add `require 'chewy/minitest'` to your test_helper.rb, and then for tests which you'd like indexing test hooks, `include Chewy::Minitest::Helpers`.
+
+You can set the `:bypass` strategy for test suites and manually handle imports and flush test indices using `Chewy.massacre`. This will help reduce unnecessary ES requests.
+
+But if you require chewy to index/update model regularly in your test suite then you can specify `:urgent` strategy for documents indexing. Add `Chewy.strategy(:urgent)` to test_helper.rb.
+
+Also, you can use additional helpers:
+
+`mock_elasticsearch_response` to mock elasticsearch response
+`mock_elasticsearch_response_sources` to mock elasticsearch response sources
+`assert_elasticsearch_query` to compare request and expected query (returns `true`/`false`)
+
+See [chewy/minitest/](../lib/chewy/minitest/) for more details.
+
+## DatabaseCleaner
+
+If you use `DatabaseCleaner` in your tests with [the `transaction` strategy](https://github.com/DatabaseCleaner/database_cleaner#how-to-use), you may run into the problem that `ActiveRecord`'s models are not indexed automatically on save despite the fact that you set the callbacks to do this with the `update_index` method. The issue arises because `chewy` indices data on `after_commit` run as default, but all `after_commit` callbacks are not run with the `DatabaseCleaner`'s' `transaction` strategy. You can solve this issue by changing the `Chewy.use_after_commit_callbacks` option. Just add the following initializer in your Rails application:
+
+```ruby
+#config/initializers/chewy.rb
+Chewy.use_after_commit_callbacks = !Rails.env.test?
+```
+
+If you're seeing other unexpected behavior in tests, check [troubleshooting.md](troubleshooting.md) for common issues and debugging tips.

data/docs/troubleshooting.md

@@ -0,0 +1,101 @@
+# Troubleshooting
+
+## `UndefinedUpdateStrategy` error
+
+This is the most common Chewy error. When you save a model that has an `update_index` callback and no update strategy is active, Chewy raises `Chewy::UndefinedUpdateStrategy`:
+
+```
+Index update strategy is undefined for current context.
+Please wrap your code with `Chewy.strategy(:strategy_name)` block.
+```
+
+**Fix:** wrap the code that triggers the save in a strategy block:
+
+```ruby
+Chewy.strategy(:atomic) do
+  city.save!
+end
+```
+
+In a Rails app, controller actions already use the `:atomic` strategy by default. This error typically appears in background jobs, rake tasks, or console sessions. For console use, you can set `:urgent` as a persistent strategy:
+
+```ruby
+Chewy.strategy(:urgent)
+```
+
+If you want to suppress index updates entirely (e.g. in tests or migrations), use `:bypass`:
+
+```ruby
+Chewy.root_strategy = :bypass
+```
+
+See [configuration.md](configuration.md#index-update-strategies) for the full list of strategies.
+
+## Elasticsearch 8 security defaults
+
+Elasticsearch 8 enables security (TLS + authentication) by default. If you see connection refused or authentication errors after upgrading, you need to configure credentials and the CA certificate. See the [Security section](../README.md#security) in the main README for setup instructions.
+
+## Wildcard index deletion disabled in ES 8
+
+Starting from Elasticsearch 8, wildcard deletion of indices is disabled by default. If `Chewy.massacre` or other bulk-delete operations fail with a `Chewy::FeatureDisabled` error, you need to set the cluster setting `action.destructive_requires_name` to `false`:
+
+```
+PUT _cluster/settings
+{ "persistent": { "action.destructive_requires_name": false } }
+```
+
+## Import errors and debugging
+
+When using `import!` (with a bang), Chewy raises `Chewy::ImportFailed` if any documents fail to index. The error message groups failures by action type (index, delete) and includes the document IDs:
+
+```
+Import failed for `ProductsIndex` with:
+    Index errors:
+      `mapper_parsing_exception`
+        on 3 documents: ["1", "2", "3"]
+```
+
+For non-bang `import`, errors are silently swallowed. To debug import issues, set up a logger:
+
+```ruby
+Chewy.logger = Logger.new(STDOUT)
+```
+
+You can also subscribe to `import_objects.chewy` notifications — see [configuration.md](configuration.md#activesupportnotifications-support) for the payload format.
+
+## Import scope cleanup warnings
+
+When an `index_scope` includes `order`, `limit`, or `offset`, Chewy strips them before importing (they don't make sense for batch processing). By default this logs a warning. If you see unexpected warnings during import, you can control this via:
+
+```ruby
+Chewy.import_scope_cleanup_behavior = :ignore # no warning
+Chewy.import_scope_cleanup_behavior = :raise   # raise Chewy::ImportScopeCleanupError
+```
+
+See [configuration.md](configuration.md#import-scope-clean-up-behavior) for details.
+
+## Missing optional dependencies
+
+Some Chewy features require additional gems that are not listed as hard dependencies:
+
+- **`parallel`** — required for `chewy:parallel:*` rake tasks. Install it with `gem 'parallel'` in your Gemfile.
+- **`method_source`** — required for the Witchcraft technology (compiled value procs). Install it with `gem 'method_source'`.
+
+If these gems are missing you'll get a `LoadError` when the relevant feature is used.
+
+## Pre-request filter
+
+Should you need to inspect the query prior to it being dispatched to Elasticsearch during any queries, you can use the `before_es_request_filter`. `before_es_request_filter` is a callable object, as demonstrated below:
+
+```ruby
+Chewy.before_es_request_filter = -> (method_name, args, kw_args) { ... }
+```
+
+While using the `before_es_request_filter`, please consider the following:
+
+* `before_es_request_filter` acts as a simple proxy before any request made via the `Elasticsearch::Client`. The arguments passed to this filter include:
+  * `method_name` — the name of the method being called (e.g. search, count, bulk).
+  * `args` and `kw_args` — the positional and keyword arguments provided in the method call.
+* The operation is synchronous, so avoid executing any heavy or time-consuming operations within the filter to prevent performance degradation.
+* The return value of the proc is disregarded. This filter is intended for inspection or modification of the query rather than generating a response.
+* Any exception raised inside the callback will propagate upward and halt the execution of the query. It is essential to handle potential errors adequately to ensure the stability of your search functionality.

data/gemfiles/base.gemfile

@@ -6,7 +6,7 @@ gem 'redis', require: false
 gem 'rspec', '>= 3.7.0'
 gem 'rspec-collection_matchers'
 gem 'rspec-its'
-gem 'rubocop', '1.63.4'
-gem 'sqlite3', '~> 1.4'
+gem 'rubocop', '1.84.2'
+gem 'sqlite3', '~> 2.1'
 gem 'timecop'
-gem 'unparser'
+gem 'unparser', '~> 0.6.15'

data/gemfiles/{rails.6.1.activerecord.gemfile → rails.7.2.activerecord.gemfile}

@@ -1,8 +1,8 @@
 source 'https://rubygems.org'
 
-gem 'activejob', '~> 6.1.0'
-gem 'activerecord', '~> 6.1.0'
-gem 'activesupport', '~> 6.1.0'
+gem 'activejob', '~> 7.2.0'
+gem 'activerecord', '~> 7.2.0'
+gem 'activesupport', '~> 7.2.0'
 gem 'kaminari-core', '~> 1.1.0', require: false
 gem 'parallel', require: false
 gem 'rspec_junit_formatter', '~> 0.4.1'