chewy 8.0.0 → 8.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b5ed854e820d95af1d41232ffab69a142b8cb45591ba01f8b316349120ec49b5
-  data.tar.gz: b2ea3e975b6fcc5af8e3a993e731e1bb5a1e64ffab9edb1b26687e91e4c05763
+  metadata.gz: 702dbe73f6b523156348491049ed540490d755d66c3242a2422c4da92090346c
+  data.tar.gz: 730beaf500e9d4feeafc5033bd028abc67662f9ae3299a95aa5d3f323f6ee011
 SHA512:
-  metadata.gz: 92b482a9606d2e0e2dfcf8560d70679a1542f6e89012c012f7d46f4844dd568c04feca8a949b9c100444de73819d821edcb07cf9bbbb997a10f8cbcf93b033eb
-  data.tar.gz: c0595cbc913c89c295f653b124c545153f2775cdb64050bee435067020700093a69b137e132ec743feb4a5f477efd8c4a278eb3dce569c86f67cce1188c4e9da
+  metadata.gz: 87105fad14e8959fabbad411e3b22af813b6261c0cff1a45f14205819e5564f03d936acf6fcd909ac152d98325c2619628c3732fa1eb628d6edb42a5b9670a95
+  data.tar.gz: b9f693b6e4180e9c2023456439e1de74c86dff2a453f50be369c50f30f7b419eab75173f2071707cf922457dc69fb9fc26cf0e82ad79034e3eea79212091a791

data/CHANGELOG.md

@@ -1,10 +1,49 @@
 # Changelog
 
+## master (unreleased)
+
+### New Features
+
+### Bug Fixes
+
+### Changes
+
+## 8.1.0 (2026-05-28)
+
+### New Features
+
+* [#887](https://github.com/toptal/chewy/pull/887): Add support [runtime_mappings](https://www.elastic.co/guide/en/elasticsearch/reference/current/runtime-search-request.html). ([@TakuyaKurimoto](https://github.com/TakuyaKurimoto))
+* [#996](https://github.com/toptal/chewy/pull/996): Add `context:` option to `import`/`import!` for passing custom data to crutch blocks and field value procs without redundant DB queries.
+
+### Bug Fixes
+
+* [#878](https://github.com/toptal/chewy/issues/878): Flatten `Chewy::Stash::Journal.for` to use a single `terms` filter instead of a chain of `bool.should` clauses. Fixes Elasticsearch `indices.query.bool.max_nested_depth` errors when applying or cleaning the journal across many indices.
+
+### Changes
+
+* [#916](https://github.com/toptal/chewy/pull/916): Raise error in #scroll_batches when search backend returns a failure. ([@tomdev][])
+* [#1008](https://github.com/toptal/chewy/pull/1008): Promote Elasticsearch to a native GitHub Actions service with a health-check gate, replacing the fragile `docker compose` + `sleep 15` approach. ([@mattmenefee][])
+* [#1010](https://github.com/toptal/chewy/pull/1010): Add Chewy 7/ES 7 to Chewy 8/ES 8 migration guide and fix stale `Elasticsearch::Transport` namespace references in docs. ([@mattmenefee][])
+* [#1011](https://github.com/toptal/chewy/pull/1011): Replace deprecated `Sidekiq::Testing` API with new `Sidekiq 8.1+` testing API and silence Sidekiq logger during spec runs. ([@mattmenefee][], [@mjankowski][])
+* [#1013](https://github.com/toptal/chewy/pull/1013): Fix `drop_indices` test helper to use `format: 'json'` for ES version portability. If you define a custom `drop_indices` helper in your test suite, update it to use `Chewy.client.cat.indices(format: 'json')` instead of parsing the text-format response. ([@mattmenefee][])
+* [#1014](https://github.com/toptal/chewy/pull/1014): Improve contributing documentation with development setup instructions, PR workflow, and grammar fixes. ([@mattmenefee][])
+
+## 8.0.1 (2026-03-12)
+
+### New Features
+
+### Changes
+
+* [#1003](https://github.com/toptal/chewy/pull/1003): Ruby 4.0 support added [@mjankowski](https://github.com/mjankowski)
+
+### Bugs Fixed
+
 ## 8.0.0 (2026-02-25)
 
 ### New Features
 
 ### Changes
+* [#977](https://github.com/toptal/chewy/pull/977): Fewer files on gem installation [@ericproulx](https://github.com/ericproulx).
 
 * **(Breaking)** Drop support for Ruby < 3.2 and Rails < 7.2. Chewy now requires Ruby ~> 3.2 and ActiveSupport >= 7.2. ([@bbatsov][])
 
@@ -863,10 +902,12 @@
 [@marshall]: https://github.com/marshall
 [@matchbookmac]: https://github.com/matchbookmac
 [@matthee]: https://github.com/matthee
+[@mattmenefee]: https://github.com/mattmenefee
 [@mattzollinhofer]: https://github.com/mattzollinhofer
 [@menglewis]: https://github.com/menglewis
 [@mikeyhogarth]: https://github.com/mikeyhogarth
 [@milk1000cc]: https://github.com/milk1000cc
+[@mjankowski]: https://github.com/mjankowski
 [@mkcode]: https://github.com/mkcode
 [@mpeychich]: https://github.com/mpeychich
 [@mrbrdo]: https://github.com/mrbrdo
@@ -890,6 +931,7 @@
 [@socialchorus]: https://github.com/socialchorus
 [@taylor-au]: https://github.com/taylor-au
 [@TikiTDO]: https://github.com/TikiTDO
+[@tomdev]: https://github.com/tomdev
 [@undr]: https://github.com/undr
 [@Vitalina-Vakulchyk]: https://github.com/Vitalina-Vakulchyk
 [@webgago]: https://github.com/webgago

data/README.md

@@ -1,7 +1,5 @@
 [![Gem Version](https://badge.fury.io/rb/chewy.svg)](http://badge.fury.io/rb/chewy)
 [![GitHub Actions](https://github.com/toptal/chewy/actions/workflows/ruby.yml/badge.svg)](https://github.com/toptal/chewy/actions/workflows/ruby.yml)
-[![Code Climate](https://codeclimate.com/github/toptal/chewy.svg)](https://codeclimate.com/github/toptal/chewy)
-[![Inline docs](http://inch-ci.org/github/toptal/chewy.svg?branch=master)](http://inch-ci.org/github/toptal/chewy)
 
 # Chewy
 
@@ -47,13 +45,13 @@ Chewy aims to support all Ruby and Rails versions that are currently maintained
 
 ### Ruby
 
-Chewy is compatible with MRI 3.2-3.4.
+Chewy is compatible with MRI 3.2-4.0.
 
 ### Elasticsearch compatibility matrix
 
 | Chewy version | Elasticsearch version              |
 | ------------- | ---------------------------------- |
-| 8.0.0         | 8.x                                |
+| 8.0.x         | 8.x                                |
 | 7.2.x         | 7.x                                |
 | 7.1.x         | 7.x                                |
 | 7.0.x         | 6.8, 7.x                           |
@@ -236,6 +234,7 @@ end
 
 ## Documentation
 
+- [Getting Started](docs/getting_started.md) — end-to-end tutorial building search for a media library
 - [Configuration](docs/configuration.md) — client settings, update strategies, notifications, integrations
 - [Indexing](docs/indexing.md) — index definition, field types, crutches, witchcraft, index manipulation
 - [Import](docs/import.md) — import options, raw import, journaling
@@ -243,22 +242,37 @@ end
 - [Rake Tasks](docs/rake_tasks.md) — all rake tasks and parallelization
 - [Testing](docs/testing.md) — RSpec, Minitest, DatabaseCleaner
 - [Troubleshooting](docs/troubleshooting.md) — pre-request filter
+- [Non-Rails Usage](docs/non_rails.md) — using Chewy without Rails
 
-## Contributing
+## FAQ
 
-1. Fork it (http://github.com/toptal/chewy/fork)
-2. Create your feature branch (`git checkout -b my-new-feature`)
-3. Implement your changes, cover it with specs and make sure old specs are passing
-4. Commit your changes (`git commit -am 'Add some feature'`)
-5. Push to the branch (`git push origin my-new-feature`)
-6. Create new Pull Request
+**Can I use Chewy without Rails?**
 
-Use the following Rake tasks to control the Elasticsearch cluster while developing, if you prefer native Elasticsearch installation over the dockerized one:
+Yes. ActiveSupport and the elasticsearch gem are the only hard dependencies; the Rails railtie loads conditionally. See the [Non-Rails Usage](docs/non_rails.md) guide.
 
-```bash
-rake elasticsearch:start # start Elasticsearch cluster on 9250 port for tests
-rake elasticsearch:stop # stop Elasticsearch
-```
+**Does Chewy follow SemVer?**
+
+No. The major version tracks the newest supported Elasticsearch version and minor version bumps may include breaking changes. Always check the [release notes](https://github.com/toptal/chewy/releases) before upgrading.
+
+**Which versions of Ruby, Rails and Elasticsearch are supported?**
+
+See the [Compatibility](#compatibility) section above. When a Ruby or Rails version reaches end-of-life we may drop support in a future release.
+
+**What's the `UndefinedUpdateStrategy` error?**
+
+Chewy raises this when you save a model with an `update_index` callback and no strategy is active. Wrap your code in `Chewy.strategy(:atomic) { ... }` or set `Chewy.root_strategy = :bypass`. See [Troubleshooting](docs/troubleshooting.md#undefinedupdatestrategy-error) for details.
+
+**Can I use Chewy with OpenSearch?**
+
+OpenSearch is not officially supported or tested. Chewy depends on the official `elasticsearch` Ruby client and targets Elasticsearch.
+
+**How do I connect to Elastic Cloud?**
+
+Use the standard client settings with your Cloud credentials (API key or user/password). See [Configuration](docs/configuration.md#client-settings) for examples.
+
+## Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup instructions, how to run tests, and the pull request workflow.
 
 ## Copyright
 

data/lib/chewy/errors.rb

@@ -37,6 +37,9 @@ module Chewy
     end
   end
 
+  class MissingHitsInScrollError < Error
+  end
+
   class ImportScopeCleanupError < Error
   end
 

data/lib/chewy/fields/root.rb

@@ -61,8 +61,8 @@ module Chewy
       # @param fields [Array<Symbol>] a list of fields to compose, every field will be composed if empty
       # @return [Hash] JSON-ready hash with stringified keys
       #
-      def compose(object, crutches = nil, fields: [])
-        result = evaluate([object, crutches])
+      def compose(object, crutches = nil, fields: [], context: {})
+        result = evaluate([object, crutches, context])
 
         if children.present?
           child_fields = if fields.present?
@@ -72,7 +72,7 @@ module Chewy
           end
 
           child_fields.each_with_object({}) do |field, memo|
-            memo.merge!(field.compose(result, crutches) || {})
+            memo.merge!(field.compose(result, crutches, context) || {})
           end.as_json
         elsif fields.present?
           result.as_json(only: fields, root: false)

data/lib/chewy/index/crutch.rb

@@ -9,9 +9,12 @@ module Chewy
       end
 
       class Crutches
-        def initialize(index, collection)
+        attr_reader :context
+
+        def initialize(index, collection, context = {})
           @index = index
           @collection = collection
+          @context = context
           @crutches_instances = {}
         end
 
@@ -26,7 +29,14 @@ module Chewy
         end
 
         def [](name)
-          @crutches_instances[name] ||= @index._crutches[:"#{name}"].call(@collection)
+          @crutches_instances[name] ||= begin
+            block = @index._crutches[:"#{name}"]
+            if block.arity > 1 || block.arity < -1
+              block.call(@collection, @context)
+            else
+              block.call(@collection)
+            end
+          end
         end
       end
 

data/lib/chewy/index/import/bulk_builder.rb

@@ -13,11 +13,12 @@ module Chewy
         # @param to_index [Array<Object>] objects to index
         # @param delete [Array<Object>] objects or ids to delete
         # @param fields [Array<Symbol, String>] and array of fields for documents update
-        def initialize(index, to_index: [], delete: [], fields: [])
+        def initialize(index, to_index: [], delete: [], fields: [], context: {})
           @index = index
           @to_index = to_index
           @delete = delete
           @fields = fields.map!(&:to_sym)
+          @context = context
         end
 
         # Returns ES API-ready bulk requiest body.
@@ -42,7 +43,7 @@ module Chewy
       private
 
         def crutches_for_index
-          @crutches_for_index ||= Chewy::Index::Crutch::Crutches.new @index, @to_index
+          @crutches_for_index ||= Chewy::Index::Crutch::Crutches.new @index, @to_index, @context
         end
 
         def index_entry(object)
@@ -257,7 +258,7 @@ module Chewy
         end
 
         def data_for(object, fields: [], crutches: crutches_for_index)
-          @index.compose(object, crutches, fields: fields)
+          @index.compose(object, crutches, fields: fields, context: @context)
         end
 
         def parent_changed?(data, old_parent)

data/lib/chewy/index/import/routine.rb

@@ -56,6 +56,7 @@ module Chewy
               {}
             end
           end
+          @context = @options[:context] || {}
           @errors = []
           @stats = {}
           @leftovers = []
@@ -78,7 +79,7 @@ module Chewy
         # @param delete [Array<Object>] any acceptable objects for deleting
         # @return [true, false] the result of the request, true if no errors
         def process(index: [], delete: [])
-          bulk_builder = BulkBuilder.new(@index, to_index: index, delete: delete, fields: @options[:update_fields])
+          bulk_builder = BulkBuilder.new(@index, to_index: index, delete: delete, fields: @options[:update_fields], context: @context)
           bulk_body = bulk_builder.bulk_body
 
           if @options[:journal]

data/lib/chewy/index/import.rb

@@ -115,13 +115,13 @@ module Chewy
         # @param crutches [Object] optional crutches object; if omitted - a crutch for the single passed object is created as a fallback
         # @param fields [Array<Symbol>] and array of fields to restrict the generated document
         # @return [Hash] a JSON-ready hash
-        def compose(object, crutches = nil, fields: [])
-          crutches ||= Chewy::Index::Crutch::Crutches.new self, [object]
+        def compose(object, crutches = nil, fields: [], context: {})
+          crutches ||= Chewy::Index::Crutch::Crutches.new self, [object], context
 
           if witchcraft? && root.children.present?
-            cauldron(fields: fields).brew(object, crutches)
+            cauldron(fields: fields).brew(object, crutches, context)
           else
-            root.compose(object, crutches, fields: fields)
+            root.compose(object, crutches, fields: fields, context: context)
           end
         end
 

data/lib/chewy/index/witchcraft.rb

@@ -1,6 +1,10 @@
 begin
   require 'method_source'
-  require 'parser/current'
+  begin
+    require 'prism'
+  rescue LoadError
+    require 'parser/current'
+  end
   require 'unparser'
 rescue LoadError
   nil
@@ -24,7 +28,11 @@ module Chewy
         def check_requirements!
           messages = []
           messages << "MethodSource gem is required for the Witchcraft, please add `gem 'method_source'` to your Gemfile" unless Proc.method_defined?(:source)
-          messages << "Parser gem is required for the Witchcraft, please add `gem 'parser'` to your Gemfile" unless '::Parser'.safe_constantize
+          if RUBY_VERSION >= '3.3'
+            messages << "Prism gem is required for the Witchcraft, please add `gem 'prism'` to your Gemfile" unless '::Prism'.safe_constantize
+          else
+            messages << "Parser gem is required for the Witchcraft, please add `gem 'parser'` to your Gemfile" unless '::Parser'.safe_constantize
+          end
           messages << "Unparser gem is required for the Witchcraft, please add `gem 'unparser'` to your Gemfile" unless '::Unparser'.safe_constantize
           messages = messages.join("\n")
 
@@ -51,15 +59,15 @@ module Chewy
           @fields = fields
         end
 
-        def brew(object, crutches = nil)
-          alicorn.call(locals, object, crutches).as_json
+        def brew(object, crutches = nil, context = {})
+          alicorn.call(locals, object, crutches, context).as_json
         end
 
       private
 
         def alicorn
           @alicorn ||= singleton_class.class_eval <<-RUBY, __FILE__, __LINE__ + 1
-            -> (locals, object0, crutches) do
+            -> (locals, object0, crutches, context) do
               #{composed_values(@index.root, 0)}
             end
           RUBY
@@ -163,9 +171,8 @@ module Chewy
           end
         end
 
-        def source_for(proc, nesting)
-          ast = Parser::CurrentRuby.parse(proc.source)
-          lambdas = exctract_lambdas(ast)
+        def source_for(proc, nesting) # rubocop:disable Metrics/AbcSize
+          lambdas = exctract_lambdas(ast_from_proc(proc))
 
           raise "No lambdas found, try to reformat your code:\n`#{proc.source}`" unless lambdas
 
@@ -182,6 +189,7 @@ module Chewy
               source = replace_lvar(source, proc_params[n], :"object#{n}") if proc_params[n]
             end
             source = replace_lvar(source, proc_params[nesting + 1], :crutches) if proc_params[nesting + 1]
+            source = replace_lvar(source, proc_params[nesting + 2], :context) if proc_params[nesting + 2]
 
             binding_variable_list(source).each do |variable|
               locals.push(proc.binding.eval(variable.to_s))
@@ -192,6 +200,14 @@ module Chewy
           Unparser.unparse(source)
         end
 
+        def ast_from_proc(proc)
+          if defined?(Prism)
+            Prism::Translation::ParserCurrent.parse(proc.source)
+          else
+            Parser::CurrentRuby.parse(proc.source)
+          end
+        end
+
         def exctract_lambdas(node)
           return unless node.is_a?(Parser::AST::Node)
 

data/lib/chewy/multi_search.rb

@@ -11,7 +11,7 @@ module Chewy
     # Instantiate a new MultiSearch instance.
     #
     # @param queries [Array<Chewy::Search::Request>]
-    # @option [Elasticsearch::Transport::Client] :client (Chewy.client)
+    # @option [Elasticsearch::Client] :client (Chewy.client)
     #   The Elasticsearch client that should be used for issuing requests.
     def initialize(queries, client: Chewy.client)
       @client = client

data/lib/chewy/search/parameters/runtime_mappings.rb

@@ -0,0 +1,14 @@
+module Chewy
+  module Search
+    class Parameters
+      # Just a standard hash storage. Nothing to see here.
+      #
+      # @see Chewy::Search::Parameters::HashStorage
+      # @see Chewy::Search::Request#runtime_mappings
+      # @see https://www.elastic.co/guide/en/elasticsearch/reference/current/runtime-search-request.html
+      class RuntimeMappings < Storage
+        include HashStorage
+      end
+    end
+  end
+end

data/lib/chewy/search/request.rb

@@ -25,7 +25,7 @@ module Chewy
         search_type preference limit offset terminate_after
         timeout min_score source stored_fields search_after
         load script_fields suggest aggs aggregations collapse none
-        indices_boost rescore highlight total total_count
+        indices_boost rescore highlight runtime_mappings total total_count
         total_entries indices types delete_all count exists?
         exist? find pluck scroll_batches scroll_hits
         scroll_results scroll_wrappers ignore_unavailable
@@ -656,7 +656,23 @@ module Chewy
       #   @see https://www.elastic.co/guide/en/elasticsearch/reference/current/highlighting.html
       #   @param value [Hash]
       #   @return [Chewy::Search::Request]
-      %i[script_fields indices_boost rescore highlight].each do |name|
+      #
+      # @!method runtime_mappings(value)
+      #   Add a `runtime_mappings` part to the request. Further
+      #   call values are merged to the storage hash.
+      #
+      #   @example
+      #     PlacesIndex
+      #       .runtime_mappings(field1: {type: "keyword", script: {lang: "painless", source: "emit('some script here')"}})
+      #       .runtime_mappings(field2: {type: "keyword", script: {lang: "painless", source: "emit('some script here')"}})
+      #     # => <PlacesIndex::Query {..., :body=>{:runtime_mappings=>{
+      #     #      "field1"=>{:type=>"keyword", :script=>{:lang=>"painless", :source=>"emit('some script here')"}},
+      #     #      "field2"=>{:type=>"keyword", :script=>{:lang=>"painless", :source=>"emit('some script here')"}}}}}>
+      #   @see Chewy::Search::Parameters::RuntimeMappings
+      #   @see https://www.elastic.co/guide/en/elasticsearch/reference/current/runtime-search-request.html
+      #   @param value [Hash]
+      #   @return [Chewy::Search::Request]
+      %i[script_fields indices_boost rescore highlight runtime_mappings].each do |name|
         define_method name do |value|
           modify(name) { update!(value) }
         end

data/lib/chewy/search/scrolling.rb

@@ -29,20 +29,28 @@ module Chewy
 
         result = perform(size: batch_size, scroll: scroll)
         total = [raw_limit_value, result.fetch('hits', {}).fetch('total', {}).fetch('value', 0)].compact.min
+
+        total_batches = total / batch_size
         last_batch_size = total % batch_size
-        fetched = 0
+
+        total_batches += 1 if last_batch_size != 0
+
         scroll_id = nil
 
-        loop do
+        total_batches.times do |batch_counter|
+          last_run = total_batches - 1 == batch_counter
+
           hits = result.fetch('hits', {}).fetch('hits', [])
-          fetched += hits.size
-          hits = hits.first(last_batch_size) if last_batch_size != 0 && fetched >= total
+          hits = hits.first(last_batch_size) if last_run && last_batch_size != 0
+
+          raise Chewy::MissingHitsInScrollError if hits.empty?
+
           yield(hits) if hits.present?
           scroll_id = result['_scroll_id']
 
-          break if result['terminated_early'] || fetched >= total
+          break if result['terminated_early']
 
-          result = perform_scroll(scroll: scroll, scroll_id: scroll_id)
+          result = perform_scroll(scroll: scroll, scroll_id: scroll_id) unless last_run
         end
       ensure
         Chewy.client.clear_scroll(body: {scroll_id: scroll_id}) if scroll_id

data/lib/chewy/stash.rb

@@ -38,17 +38,21 @@ module Chewy
 
       # Selects all the journal entries for the specified indices.
       #
+      # Uses a single `terms` filter rather than a chain of `bool.should`
+      # clauses so the query depth stays constant regardless of how many
+      # indices are passed. Avoids hitting the Elasticsearch
+      # `indices.query.bool.max_nested_depth` limit (default 30) when
+      # cleaning or applying journals across many indices.
+      #
       # @param indices [Chewy::Index, Array<Chewy::Index>]
       def self.for(*something)
         something = something.flatten.compact
+        return all if something.empty?
+
         indexes = something.flat_map { |s| Chewy.derive_name(s) }
-        return none if something.present? && indexes.blank?
+        return none if indexes.blank?
 
-        scope = all
-        indexes.each do |index|
-          scope = scope.or(filter(term: {index_name: index.derivable_name}))
-        end
-        scope
+        filter(terms: {index_name: indexes.map(&:derivable_name).uniq})
       end
 
       default_import_options journal: false

data/lib/chewy/version.rb

@@ -1,3 +1,3 @@
 module Chewy
-  VERSION = '8.0.0'.freeze
+  VERSION = '8.1.0'.freeze
 end