search-engine-for-typesense 30.9.1 → 30.9.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2f5d1b6315a75f241c80d16c1f69727fed0ceef2edabb8ce774d33c348756a59
-  data.tar.gz: fbb2364ac8afc372ff0bbcb8f70174874ab190e226f6d08e1e747b018be8259b
+  metadata.gz: 8839a1d87b60901a030030a293152432bbf593b215ba7fa7292eac10e724a2bb
+  data.tar.gz: 84b2e507deee029c7ef87165a5f5ca2f6a37450fbf25b59bb6208a42c0d4c25b
 SHA512:
-  metadata.gz: 1db71818517bd5dd234a43ebfe63f1b9affa33b4fcb0f6a15914cb6bcd55a3a6844ba8a87e68ead1bbc81a3a87dbe3dba2d845f9ba8c4da6d3251ed877755c5b
-  data.tar.gz: 56a54cdd5a7a0fdf3566c021788ba9f4d54aa9311f7df508b8b9e99c5d7010c9ecd26cf2d83cc219625a48d7738883e03c49360c9d7c4d256f4fc48d173e315b
+  metadata.gz: e0bb71bd0d9ff89f7053286c6b683433e0e49ab9e5d2cd514e2bc778f33eff821236f840b14358922afb4cc091802800616290d94a807845a87528f7f62959ff
+  data.tar.gz: 79199d5c13cdbbc4acb8b2a5c2570e6934bd675447e5ea8d4da517f68047d3fc0060e16af69f929686b8c8ebe84e298f498e8a0b720c25bb57d1e41098700ab6

data/CHANGELOG.md

@@ -1,5 +1,24 @@
 # Changelog
 
+## 30.9.3
+
+- Preserve tenant filters and scoped clients across searches and bulk writes. Reject ambiguous literals,
+  use the joined collection’s field types, and keep bound values out of parser errors.
+- Honor exact offsets, fetch one hit for `find_by`, and hydrate selected hits once. Typed multi-search
+  reports individual errors while raw multi-search preserves the server response.
+- Stop schema cutover on incomplete imports and retain both collections after an uncertain alias write.
+  Conditional rollback requires an exclusive guard shared by all alias writers.
+- Fence PostgreSQL outbox acknowledgements by the active claim. Drain and restart older workers during rollout.
+  Partition workers update their own entry without reading the full run.
+- Preserve mapped document IDs across STI, Rails autoloading and reload. Inline hosts can load Active Job later.
+
+## 30.9.2
+
+- Check recent event IDs during catch-up so a returning delivery target can discover new work
+  while its saved position moves through older events. Collection quotas still apply.
+- Read outbox checkpoints, delivery state and row locks directly from the database on every poll,
+  including when Active Record query caching is enabled on Rails 6.1 and 7.0.
+
 ## 30.9.1
 
 - Serialize PostgreSQL delivery discovery across schedulers and workers and check delivery membership

data/README.md

@@ -52,7 +52,13 @@ See [Quickstart](https://nikita-shkoda.mintlify.app/projects/search-engine-for-t
 
 By default, the gem manages a dedicated Zeitwerk loader for your SearchEngine models under `app/search_engine/`. The loader is initialized after Rails so that application models/constants are available, auto-reloads in development, and is eager-loaded in production/test.
 
-Customize or disable via configuration:
+Mappers under the host’s `app/models/search_engine` are registered by the Rails loader at boot and reload,
+including nested classes whose names differ from their declared collection. For a mapper managed outside
+these model directories, reference it in the callback declaration, for example
+`search_engine_syncable collection: Catalog::ItemIndex.collection`. This loads its identity strategy before
+the first callback. A bare collection uses the Active Record primary key for deletes.
+
+Configure the dedicated loader with:
 
 ```ruby
 # config/initializers/search_engine.rb
@@ -66,9 +72,22 @@ SQL sources stream PostgreSQL queries in their own read-only transaction. Start
 outside an existing transaction; an active transaction is rejected before cursor SQL runs.
 The configured statement timeout applies only to the streaming transaction.
 
+Use hash predicates or bound placeholders for application values. A raw filter string is a Typesense
+expression and must come from trusted code. Values that Typesense cannot represent as exact literals,
+including a wildcard document ID or a string containing a backtick, raise before a search or bulk write.
+Numeric comparisons in `where.not` invert the operator.
+A relation with `limit(2).offset(1)` starts at the second matching document. An explicit `page` or
+`per` takes precedence over the offset. Use a stable sort when reading successive windows.
+
 Joined filters execute as native Typesense joins and require valid collection references.
-Search configuration and missing-reference errors propagate to callers; they do not
-produce empty results or trigger client-side joins.
+The target model's field type determines how Ruby dates and times are encoded. For a target
+without a declared field type, pass the stored value as an ISO string or integer epoch.
+Search configuration and missing-reference errors propagate to the caller.
+
+Single search, federated search and multi-page `all!` use the configured client. A failing entry in
+`multi_search` or `multi_search_result` raises with the entry label and provider status.
+Use `multi_search_raw` to handle successful and failed entries separately. Setting `.cache(false)`
+on an entry disables caching for the whole federated request.
 
 ## Usage examples
 
@@ -230,8 +249,8 @@ bundle exec ruby examples/demo_shop/bin/query_smoke
 This compiles geo/vector requests and checks metadata fixtures without HTTP requests or model downloads.
 See [the demo instructions](examples/demo_shop/README.md#query-examples) for explicitly isolated live execution,
 including highlighting through `Client#search` and `hit.snippet_for`.
-CI uses a disposable `typesense/typesense:30.2` service for geo, vector, cache and live CLI checks,
-including the built-in embedding model. The default unit suite runs separately without integration ENV.
+CI runs query, cache, schema and demo checks against Typesense 30.2, including the built-in embedding
+model. A separate PostgreSQL job exercises outbox claims and concurrency.
 
 `upsert_bulk` defaults to `on_failure: :raise`. Use the exact symbol `:return` when the caller needs to
 handle valid Typesense row failures itself. Malformed responses and response/document count mismatches always
@@ -301,7 +320,8 @@ only then lets the schema lifecycle swap the alias. If any partition fails or ti
 alias target remains active.
 
 Use a shared `Rails.cache` backend, or provide `c.indexer.partition_run_store`, so worker processes and
-the parent indexing process can see the same run metadata. Size the queue carefully: worker concurrency
+the parent indexing process can see the same run metadata. Worker transitions update one partition.
+Explicit snapshots read the complete run. Size the queue carefully: worker concurrency
 multiplies with any per-partition `max_parallel` setting.
 
 A custom partition run store must implement:
@@ -332,8 +352,10 @@ available.
 collection creation, indexing, alias swap, and retention. `SearchEngine::Schema.rollback` uses the same guard
 around target resolution, validation, and its alias swap. In-place schema updates intentionally do not invoke
 the guard. The guard must yield exactly once; not yielding or yielding repeatedly raises `ArgumentError`.
-Guard errors and indexing errors propagate, and an interrupted pre-swap physical collection is still cleaned
-up.
+The schema apply task requires every partition import to succeed before it publishes the new physical
+collection. On failure before the alias write, cleanup removes the candidate once it verifies the target. If the alias write raises,
+the server may already have accepted it. Both physical collections are retained. Check the live alias
+and the candidate before pruning or retrying.
 
 In delivery-target outbox deployments, use the guard to stop new target claims, wait for already-claimed
 deliveries to acknowledge, and prevent cooperating direct writers from crossing the alias swap:
@@ -384,8 +406,10 @@ points to after a later cutover.
 Rollback is conservative and retry-safe. The backward-compatible form rolls back one generation only when the
 alias points at the newest retained physical; once it no longer does, repeating the call returns
 `action: :already_rolled_back` without another swap. When newer orphaned physicals exist, or when an operator
-needs an auditable destination, pass the retained target explicitly. Add `expected_current` for compare-and-swap
-protection:
+needs an auditable destination, pass the retained target explicitly. To require a specific source, pass
+`expected_current` and configure `schema.around_rebuild` to acquire an exclusive, cross-process lock.
+Every writer of that logical alias must share the lock. Typesense exposes separate read and write operations,
+so direct administrative writes bypass the precondition.
 
 ```ruby
 SearchEngine::Schema.rollback(
@@ -402,9 +426,11 @@ rails 'search_engine:schema:rollback[products]'
 rails 'search_engine:schema:rollback[products,products_20250101_000000_001,products_20250102_000000_001]'
 ```
 
-An explicit destination must be a retained physical older than the current/expected source. Retrying after the
-alias already reached that destination is a successful no-op. Any other expected-current mismatch raises
-`SearchEngine::Schema::RollbackConflict` without changing the alias.
+An explicit destination must be a retained physical older than the current or expected source. A retry
+succeeds without another swap when the alias already points at the destination and the named generations
+are still retained. Any other expected-current mismatch raises
+`SearchEngine::Schema::RollbackConflict`. After an ambiguous alias response, reconcile the remote state
+before allowing another writer to proceed.
 
 This protocol only gates delivery-target claims. Legacy single-target event claims have no target identity and
 must be stopped before a rebuild. Likewise, an unguarded direct writer can still cross the swap. In
@@ -572,9 +598,10 @@ For large histories, create a checkpoint table in a migration with
 `c.postgres_outbox.materialization_cursor_table_name = 'search_engine_outbox_materialization_cursors'`.
 Run the migration before starting consumers with this setting.
 
-With checkpoints, each discovery call checks at most 1,000 IDs around the forward position and
-1,000 historical IDs against the requested targets. The forward scan overlaps its previous
-range to catch nearby out-of-order commits. Each target set maintains its own position. A
+With checkpoints, each discovery call checks at most 2,000 event IDs. Each target set maintains
+its own forward position and a historical sweep. The forward scan overlaps its previous range
+to catch nearby out-of-order commits. When it falls behind, part of the historical scan budget
+checks the newest IDs while the forward scan continues through the intervening events. A
 transaction that commits farther behind, or an older event that becomes due, can wait until
 the historical scan revisits its ID. Sweep time depends on retained history and polling rate.
 Collection batch quotas and the latest-event rule still govern which deliveries may be created.
@@ -642,6 +669,14 @@ Drain slots are scheduler bookkeeping and are deliberately not changed. Configur
 before each database mutation, but Ruby configuration and PostgreSQL cannot share an atomic lock, so do not
 change delivery-target configuration concurrently with retirement.
 
+Single-target repository callers must pass the events returned by `claim_pending` to acknowledgement and
+failure methods. The database matches each event against its current processing claim. Expired workers
+can finish an external request, but only the current owner can change the event state. Drain and restart
+workers running earlier versions when enabling this protection.
+
+Active Record subclasses inherit the parent’s sync configuration and may override it. Callbacks resolve
+the current mapper at execution time, including the document identity used for destroy.
+
 Pair triggered source models with `sync_strategy: :postgres_outbox` so Active Record callbacks do not also
 write to Typesense for the same changes:
 

data/lib/search_engine/active_record_syncable.rb

@@ -1,14 +1,16 @@
 # frozen_string_literal: true
 
 require 'active_support/concern'
+require 'active_support/core_ext/class/attribute'
+require 'active_support/inflector'
 
 module SearchEngine
   # ActiveRecord concern to keep a Typesense collection in sync.
   #
   # Include into an AR model and call {.search_engine_syncable} to store sync
-  # metadata. The default callback strategy installs lifecycle callbacks that
-  # upsert on create/update and delete on destroy; the postgres_outbox strategy
-  # stores metadata without installing callbacks.
+  # metadata. The default callback strategy upserts on create/update and deletes
+  # on destroy. The postgres_outbox strategy stores metadata without installing
+  # callbacks. Subclasses inherit the configuration unless they declare their own.
   #
   # @example
   #   class Product < ApplicationRecord
@@ -18,86 +20,61 @@ module SearchEngine
   module ActiveRecordSyncable
     extend ActiveSupport::Concern
 
+    # Rails deduplicates after_commit filters by method name, even for separate actions.
+    CALLBACKS = {
+      create: :__se_syncable_create!,
+      update: :__se_syncable_upsert!,
+      destroy: :__se_syncable_delete!
+    }.freeze
+    private_constant :CALLBACKS
+
+    included do
+      class_attribute :_se_syncable_config, instance_writer: false, instance_predicate: false
+    end
+
     class_methods do
-      # Configure SearchEngine synchronization for this ActiveRecord model.
-      #
-      # - collection: defaults to the AR class tableized name (snake_case, plural)
-      # - on: one or many of :create, :update, :destroy (strings or symbols)
-      # - sync_strategy: :callbacks (default) or :postgres_outbox
+      # Configure SearchEngine synchronization for this model and its subclasses.
+      # SearchEngine models can register after this declaration.
       #
-      # Validates that either a physical Typesense collection exists for the
-      # given name or a SearchEngine model is registered for it. Mapping for
-      # create/update requires a SearchEngine model; when missing, an error is
-      # raised with guidance.
-      #
-      # @param collection [Symbol, String, nil]
-      # @param on [Array<Symbol,String>, Symbol, String, nil]
-      # @param sync_strategy [Symbol, String, nil]
-      # @return [Class] self (for macro chaining)
+      # @param collection [Symbol, String, nil] defaults to the tableized model name
+      # @param on [Array<Symbol,String>, Symbol, String, nil] one or more of create, update and destroy
+      # @param sync_strategy [Symbol, String, nil] callbacks (default) or postgres_outbox
+      # @return [Class] self
       def search_engine_syncable(collection: nil, on: nil, sync_strategy: :callbacks)
-        effective_actions = on
-
-        cfg = SearchEngine::ActiveRecordSyncable.__normalize_config_for(
-          self,
-          collection: collection,
-          actions: effective_actions,
-          sync_strategy: sync_strategy
-        )
-
-        # Store config on the AR class (used by instance methods)
-        instance_variable_set(:@__se_syncable_cfg__, cfg)
-
-        SearchEngine::ActiveRecordSyncable.__register_callbacks_for(self, cfg) if cfg[:sync_strategy] == :callbacks
-
+        previous = _se_syncable_config
+        config = {
+          logical: (collection || ActiveSupport::Inflector.tableize(name)).to_s.dup.freeze,
+          actions: SearchEngine::ActiveRecordSyncable.__normalize_actions(on).freeze,
+          sync_strategy: SearchEngine::ActiveRecordSyncable.__normalize_sync_strategy(sync_strategy),
+          timing: SearchEngine.config.syncable_callback_timing
+        }.freeze
+        self._se_syncable_config = config
+        if config[:sync_strategy] == :callbacks && config != previous
+          SearchEngine::ActiveRecordSyncable.__register_callbacks_for(self, config)
+        end
         self
       end
     end
 
-    # Upsert this record into the configured Typesense collection using the
-    # mapping defined on the corresponding SearchEngine model.
+    # @api private
+    def __se_syncable_create!
+      __se_syncable_upsert!
+    end
+
+    # Upsert this record using the configured SearchEngine model.
     # @return [void]
     def __se_syncable_upsert!
-      cfg = self.class.instance_variable_get(:@__se_syncable_cfg__) || {}
-      se_klass = cfg[:se_klass]
-      unless se_klass
-        # Lazy-resolve the SearchEngine model to avoid boot-time ordering issues
-        begin
-          se_klass = SearchEngine.collection_for(cfg[:logical])
-          cfg[:se_klass] = se_klass if se_klass
-        rescue StandardError
-          se_klass = nil
-        end
-      end
-      unless se_klass
-        SearchEngine.config.logger&.warn(
-          "search_engine_syncable: no SearchEngine model registered for '#{cfg[:logical]}'; skip upsert"
-        )
-        return
-      end
-
-      se_klass.upsert(record: self)
-    rescue StandardError => error
-      SearchEngine.config.logger&.error("search_engine_syncable upsert failed: #{error}")
+      sync_search_engine_record
     end
 
-    # Delete this record's document from the configured Typesense collection.
-    # Uses the SearchEngine model's identity strategy when available.
+    # Delete the mapped document. Collections without a model use the ActiveRecord primary key.
     # @return [void]
     def __se_syncable_delete!
-      cfg = self.class.instance_variable_get(:@__se_syncable_cfg__) || {}
-      logical = cfg[:logical]
-      se_klass = cfg[:se_klass]
-
-      document_id = if se_klass
-                      se_klass.compute_document_id(self)
-                    else
-                      respond_to?(:id) ? id.to_s : nil
-                    end
-
-      if document_id.nil? || document_id.strip.empty?
-        SearchEngine.config.logger&.warn(
-          "search_engine_syncable: cannot delete without id for '#{logical}'"
-        )
+      logical = __se_syncable_logical
+      model = __se_syncable_model(logical)
+      document_id = model ? model.compute_document_id(self) : id.to_s
+      if document_id.blank?
+        SearchEngine.config.logger&.warn("search_engine_syncable: cannot delete without id for '#{logical}'")
         return
       end
 
@@ -108,123 +85,51 @@ module SearchEngine
       SearchEngine.config.logger&.error("search_engine_syncable delete failed: #{error}")
     end
 
-    # Return the associated SearchEngine record for this ActiveRecord instance.
-    #
-    # Resolves the SearchEngine model lazily when necessary and computes the
-    # document id via the model's `identify_by` strategy. Returns nil when the
-    # model mapping is unavailable or the id cannot be determined.
-    #
-    # @return [Object, nil] hydrated SearchEngine model instance or nil when not found
+    # Return the associated SearchEngine record using the current model and identity strategy.
+    # @return [Object, nil] hydrated SearchEngine model instance or nil when unavailable
     def search_engine_record
-      cfg = self.class.instance_variable_get(:@__se_syncable_cfg__) || {}
-      se_klass = cfg[:se_klass]
-      if se_klass.nil?
-        begin
-          logical = cfg[:logical] || self.class.name.to_s
-          se_klass = SearchEngine.collection_for(logical)
-          cfg[:se_klass] = se_klass if se_klass
-        rescue StandardError
-          return nil
-        end
-      end
+      model = __se_syncable_model
+      return unless model
 
-      doc_id = begin
-        se_klass.compute_document_id(self)
-      rescue StandardError => error
-        # When a custom identify_by is configured on the SearchEngine model,
-        # do not fall back to the ActiveRecord id on computation errors as it
-        # may point to a different document. Only fall back to AR id when
-        # identify_by is not defined at all.
-        if se_klass.instance_variable_defined?(:@identify_by_proc)
-          SearchEngine.config.logger&.warn(
-            "search_engine_syncable: identify_by failed to compute id for '#{cfg[:logical]}' (#{error.class})"
-          )
-          nil
-        else
-          respond_to?(:id) ? id.to_s : nil
-        end
-      end
-      return nil if doc_id.nil? || doc_id.to_s.strip.empty?
-
-      se_klass.find(doc_id)
-    rescue StandardError
+      document_id = model.compute_document_id(self)
+      model.find(document_id) if document_id.present?
+    rescue StandardError => error
+      SearchEngine.config.logger&.error("search_engine_syncable lookup failed: #{error}")
       nil
     end
 
     # Map this ActiveRecord instance using the associated SearchEngine model and
     # upsert it into the collection.
-    #
     # @return [Integer] number of upserted documents (0 or 1)
     def sync_search_engine_record
-      cfg = self.class.instance_variable_get(:@__se_syncable_cfg__) || {}
-      se_klass = cfg[:se_klass]
-      if se_klass.nil?
-        begin
-          logical = cfg[:logical] || self.class.name.to_s
-          se_klass = SearchEngine.collection_for(logical)
-          cfg[:se_klass] = se_klass if se_klass
-        rescue StandardError => error
-          SearchEngine.config.logger&.warn(
-            "search_engine_syncable: cannot resolve model for sync (#{error.class})"
-          )
-          return 0
-        end
-      end
+      model = __se_syncable_model
+      return model.upsert(record: self) if model
 
-      se_klass.upsert(record: self)
+      SearchEngine.config.logger&.warn(
+        "search_engine_syncable: no SearchEngine model registered for '#{__se_syncable_logical}'; skip upsert"
+      )
+      0
     rescue StandardError => error
       SearchEngine.config.logger&.error("search_engine_syncable sync failed: #{error}")
       0
     end
 
-    module_function
-
-    # @api private
-    # @param ar_klass [Class]
-    # @param collection [String,Symbol,nil]
-    # @param actions [Array<String,Symbol>, String, Symbol, nil]
-    # @param sync_strategy [String, Symbol, nil]
-    # @return [Hash]
-    def __normalize_config_for(ar_klass, collection:, actions:, sync_strategy: :callbacks)
-      require 'active_support/inflector'
-
-      logical = (collection || ActiveSupport::Inflector.tableize(ar_klass.name)).to_s
-      normalized_actions = __normalize_actions(actions)
-      normalized_sync_strategy = __normalize_sync_strategy(sync_strategy)
+    private
 
-      # Best-effort resolve SearchEngine model now; fall back to lazy resolution
-      se_klass = begin
-        SearchEngine.collection_for(logical)
-      rescue StandardError
-        nil
-      end
-
-      if se_klass.nil? && (normalized_actions.include?(:create) || normalized_actions.include?(:update))
-        SearchEngine.config.logger&.warn(
-          "search_engine_syncable: mapping for '#{logical}' not found at boot; will resolve lazily at runtime"
-        )
-      end
+    def __se_syncable_model(logical = __se_syncable_logical)
+      SearchEngine.collection_for(logical) { nil }
+    end
 
-      {
-        logical: logical,
-        actions: normalized_actions,
-        sync_strategy: normalized_sync_strategy,
-        se_klass: se_klass
-      }
+    def __se_syncable_logical
+      _se_syncable_config ? _se_syncable_config.fetch(:logical) : self.class.name.to_s
     end
 
+    module_function
+
     # @api private
-    # @param actions [Array<String,Symbol>, String, Symbol, nil]
-    # @return [Array<Symbol>]
     def __normalize_actions(actions)
-      allowed = %i[create update destroy]
-      list =
-        if actions.nil?
-          allowed
-        else
-          Array(actions).map { |a| a.to_s.downcase.strip.to_sym }
-        end
-
+      allowed = CALLBACKS.keys
+      list = actions.nil? ? allowed : Array(actions).map { |action| action.to_s.downcase.strip.to_sym }
       invalid = list - allowed
       raise ArgumentError, "search_engine_syncable: actions must be within #{allowed.inspect}" unless invalid.empty?
 
@@ -232,8 +137,6 @@ module SearchEngine
     end
 
     # @api private
-    # @param strategy [String, Symbol, nil]
-    # @return [Symbol]
     def __normalize_sync_strategy(strategy)
       allowed = %i[callbacks postgres_outbox]
       normalized = strategy.nil? ? :callbacks : strategy.to_s.downcase.strip.to_sym
@@ -245,37 +148,22 @@ module SearchEngine
       normalized
     end
 
-    # (no-op placeholder kept for backwards compatibility of method table in case of reloads)
-
     # @api private
-    # @param ar_klass [Class]
-    # @param cfg [Hash]
-    # @return [void]
-    def __register_callbacks_for(ar_klass, cfg)
-      if ar_klass.instance_variable_defined?(:@__se_syncable_callbacks_installed__) &&
-         ar_klass.instance_variable_get(:@__se_syncable_callbacks_installed__)
-        return
-      end
-
-      actions = cfg[:actions]
-      timing = begin
-        SearchEngine.config.syncable_callback_timing
-      rescue StandardError
-        :after_commit
-      end
-
-      if timing == :after_commit
-        upsert_actions = actions & %i[create update]
-        ar_klass.after_commit :__se_syncable_upsert!, on: upsert_actions unless upsert_actions.empty?
-        ar_klass.after_destroy_commit :__se_syncable_delete! if actions.include?(:destroy)
-      else
-        ar_klass.after_create :__se_syncable_upsert! if actions.include?(:create)
-        ar_klass.after_update :__se_syncable_upsert! if actions.include?(:update)
-        ar_klass.after_destroy :__se_syncable_delete! if actions.include?(:destroy)
+    def __register_callbacks_for(ar_klass, config)
+      timing = config.fetch(:timing)
+      config.fetch(:actions).each do |action|
+        # An inherited callback can be narrowed or disabled by a subclass declaration.
+        enabled = lambda do
+          current = _se_syncable_config
+          current[:sync_strategy] == :callbacks && current[:timing] == timing && current[:actions].include?(action)
+        end
+        method = CALLBACKS.fetch(action)
+        if timing == :after_commit
+          ar_klass.after_commit method, on: action, if: enabled
+        else
+          ar_klass.public_send("after_#{action}", method, if: enabled)
+        end
       end
-
-      ar_klass.instance_variable_set(:@__se_syncable_callbacks_installed__, true)
-      nil
     end
   end
 end

data/lib/search_engine/ast/node.rb

@@ -1,5 +1,7 @@
 # frozen_string_literal: true
 
+require 'search_engine/filters/sanitizer'
+
 module SearchEngine
   module AST
     # Abstract base node for the predicate AST.
@@ -154,12 +156,7 @@ module SearchEngine
       # @param field [String, Symbol]
       # @return [String]
       def validate_field!(field)
-        raise ArgumentError, 'field must be a String or Symbol' unless field.is_a?(String) || field.is_a?(Symbol)
-
-        str = field.to_s.strip
-        raise ArgumentError, 'field cannot be blank' if str.empty?
-
-        str
+        SearchEngine::Filters::Sanitizer.field_name(field, joined: true).dup.freeze
       end
 
       # Ensure values is a non-empty Array

data/lib/search_engine/async_partition_coordinator.rb

@@ -112,8 +112,9 @@ module SearchEngine
 
     def timeout_result(store, run_id, snapshot)
       mark_non_terminal_failed(store, run_id, snapshot)
-      snapshot = store.snapshot(run_id: run_id) || snapshot
-      current_result = SearchEngine::IndexingRun.aggregate_result(snapshot)
+      snapshot = store.snapshot(run_id: run_id)
+      current_result = result_for_snapshot(snapshot)
+      return finish_failure(snapshot, current_result) if invalid_snapshot_result?(current_result)
       return finish_success(snapshot, current_result) if current_result[:status] == :ok
 
       result = failed_result(
@@ -130,11 +131,12 @@ module SearchEngine
       partitions = snapshot && snapshot[:partitions]
       return unless partitions.is_a?(Hash)
 
+      writer = SearchEngine::IndexingRunStore.partition_writer(store)
       partitions.each do |partition_key, entry|
         status = entry[:status].to_s
         next if %w[succeeded failed].include?(status)
 
-        store.mark_failed(
+        writer.mark_failed(
           run_id: run_id,
           partition_key: partition_key,
           error: 'partition did not finish before timeout'

data/lib/search_engine/client/services/search.rb

@@ -24,6 +24,8 @@ module SearchEngine
           instrument(:get, path, duration, cache_params, request_token: start)
           log_success(:get, path, start, cache_params)
 
+          return yield(result) if block_given?
+
           klass = begin
             SearchEngine.collection_for(collection)
           rescue ArgumentError

data/lib/search_engine/client.rb

@@ -29,12 +29,13 @@ module SearchEngine
     # @param collection [String] collection name
     # @param params [Hash] Typesense search parameters (q, query_by, etc.)
     # @param url_opts [Hash] URL/common knobs (use_cache, cache_ttl)
+    # @yieldparam raw [Hash] optional result builder receives the response before hydration
     # @return [SearchEngine::Result] Wrapped response with hydrated hits
     # @raise [SearchEngine::Errors::InvalidParams, SearchEngine::Errors::*]
     # @see `https://nikita-shkoda.mintlify.app/projects/search-engine-for-typesense/v30.1/client`
     # @see `https://typesense.org/docs/latest/api/documents.html#search-document`
-    def search(collection:, params:, url_opts: {})
-      services.fetch(:search).call(collection: collection, params: params, url_opts: url_opts)
+    def search(collection:, params:, url_opts: {}, &result_builder)
+      services.fetch(:search).call(collection: collection, params: params, url_opts: url_opts, &result_builder)
     end
 
     # Resolve a logical collection name that might be an alias to the physical collection name.
@@ -387,6 +388,11 @@ module SearchEngine
       services.fetch(:search).multi(searches: searches, url_opts: url_opts)
     end
 
+    # Default URL-level search options for this client's configuration.
+    def search_url_options
+      ClientOptions.url_options_from_config(config)
+    end
+
     # Clear the Typesense server-side search cache.
     #
     # @return [Hash] response payload from Typesense (symbolized keys)
@@ -692,7 +698,7 @@ module SearchEngine
     end
 
     def derive_cache_opts(url_opts)
-      merged = ClientOptions.url_options_from_config(config)
+      merged = search_url_options
       merged[:use_cache] = url_opts[:use_cache] if url_opts.key?(:use_cache) && !url_opts[:use_cache].nil?
       merged[:cache_ttl] = Integer(url_opts[:cache_ttl]) if url_opts.key?(:cache_ttl)
       merged