elasticsearch_record 1.4.0 → 1.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 8a386dd571b2e88a62e26f815c16783d3fda97ed911b4edffa4db884230dfc18
4
- data.tar.gz: a50697c2043a3160c31716d07d9afbe05bab691267773b92f424bead64193ecb
3
+ metadata.gz: 3f50568400141462093f5d91bf5b26efb44a52b9529577d5e72a4d3699a57717
4
+ data.tar.gz: 89536ffacc4ddd5fb4d0334eda843d206c25fc867664f6f9454ee07d921ec712
5
5
  SHA512:
6
- metadata.gz: 4c3172895b50805a17964e0db2406b90fd5797c40d91d17ee9c0cea2a26d0b52d328663f5b44ff882d32bf490662146ef5064040361ae5a677d97868078332b5
7
- data.tar.gz: 3d86fb89924f99c2ad64e44c225f78c434cabcbadc4660dfa8032c5038394b05b5ab949abd93d9895b3a7148c07f12ac99374c8cee1c2ce805f2188eab6be143
6
+ metadata.gz: 20d82567d738f3f5fa78cdd1bb2595ea7cd068dd16a14fbda56b4c88450a33cac5b32106652401854e0085e1c4378cce6b3ef23983b04ec71fb14f931628c19e
7
+ data.tar.gz: 3bcd72650c2d0817048edf3eff0be6ba8de4cc568b04203c3ec9631ccd4ef269b1c91f6f70d90dc05419357e5f4d3c3f164b4b4345f72c54c9d0cfabe15d3103
data/README.md CHANGED
@@ -14,10 +14,8 @@ _ElasticsearchRecord is a ActiveRecord adapter and provides similar functionalit
14
14
 
15
15
  **PLEASE NOTE:**
16
16
 
17
- - This is still in **development**!
18
- - Specs & documentation will follow.
19
- - You might experience BUGs and Exceptions...
20
- - Currently supports only ActiveRecord 7.0 + Elasticsearch 8.4 _(downgrade for rails 6.x is planned in future versions)_
17
+ - Specs & documentation are still missing, but will follow.
18
+ - Currently supports ActiveRecord ~> 7.0 + Elasticsearch >= 7.17
21
19
 
22
20
  -----
23
21
 
@@ -63,7 +61,7 @@ Or install it yourself as:
63
61
 
64
62
  development:
65
63
  primary:
66
- # <...>
64
+ # <...>
67
65
 
68
66
  # elasticsearch
69
67
  elasticsearch:
@@ -71,10 +69,24 @@ Or install it yourself as:
71
69
  host: localhost:9200
72
70
  user: elastic
73
71
  password: '****'
74
- log: true
72
+
73
+ # enable ES verbose logging
74
+ # log: true
75
+
76
+ # add table (index) prefix & suffix to all 'tables'
77
+ # table_name_prefix: 'app-'
78
+ # table_name_suffix: '-development'
75
79
 
76
80
  production:
77
- ...
81
+ # <...>
82
+
83
+ # elasticsearch
84
+ elasticsearch:
85
+ # <...>
86
+
87
+ # add table (index) prefix & suffix to all 'tables'
88
+ # table_name_prefix: 'app-'
89
+ # table_name_suffix: '-production'
78
90
 
79
91
  test:
80
92
  ...
@@ -82,15 +94,19 @@ Or install it yourself as:
82
94
 
83
95
  ```
84
96
 
85
- ### b) Require ```elasticsearch_record/instrumentation``` in your application.rb (if you want to...):
97
+ ### b) Require `elasticsearch_record/instrumentation` in your application.rb (if you want to...):
98
+
86
99
  ```ruby
87
100
  # config/application.rb
101
+
88
102
  require_relative "boot"
89
103
 
90
104
  require "rails"
91
105
  # Pick the frameworks you want:
92
106
 
93
- # ...
107
+ # <...>
108
+
109
+ # add instrumentation
94
110
  require 'elasticsearch_record/instrumentation'
95
111
 
96
112
  module Application
@@ -98,14 +114,24 @@ module Application
98
114
  end
99
115
  ```
100
116
 
101
- ### c) Create a model that inherits from ```ElasticsearchRecord::Base``` model.
117
+ ### c) Create a model that inherits from `ElasticsearchRecord::Base` model.
102
118
  ```ruby
103
119
  # app/models/application_elasticsearch_record.rb
104
-
105
- class Search < ElasticsearchRecord::Base
106
-
120
+
121
+ class ApplicationElasticsearchRecord < ElasticsearchRecord::Base
122
+ # needs to be abstract
123
+ self.abstract_class = true
107
124
  end
125
+ ```
126
+
127
+ Example class, that inherits from **ApplicationElasticsearchRecord**
108
128
 
129
+ ```ruby
130
+ # app/models/search.rb
131
+
132
+ class Search < ApplicationElasticsearchRecord
133
+
134
+ end
109
135
  ```
110
136
 
111
137
  ### d) have FUN with your model:
@@ -239,11 +265,80 @@ _see simple documentation about these methods @ [rubydoc](https://rubydoc.info/g
239
265
 
240
266
  -----
241
267
 
242
- ### Useful model class attributes
243
- - index_base_name
244
- - relay_id_attribute
268
+ ## Useful model class attributes
269
+
270
+ ### index_base_name
271
+ Rails resolves a pluralized underscore table_name from the class name by default - which will not work for some models.
272
+
273
+ To support a generic +table_name_prefix+ & +table_name_suffix+ from the _database.yml_,
274
+ the 'index_base_name' provides a possibility to chain prefix, **base** and suffix.
275
+
276
+ ```ruby
277
+ class UnusalStat < ApplicationElasticsearchRecord
278
+ self.index_base_name = 'unusal-stats'
279
+ end
280
+
281
+ UnusalStat.where(year: 2023).to_query
282
+ # => {:index=>"app-unusal-stats-development", :body ...
283
+ ```
284
+
285
+ ### delegate_id_attribute
286
+ Rails resolves the primary_key's value by accessing the **#id** method.
287
+
288
+ Since Elasticsearch also supports an additional, independent **id** attribute,
289
+ it would only be able to access this through `_read_attribute(:id)`.
290
+
291
+ To also have the ability of accessing this attribute through the default, this flag can be enabled.
292
+
293
+ ```ruby
294
+ class SearchUser < ApplicationElasticsearchRecord
295
+ # attributes: id, name
296
+ end
297
+
298
+ # create new user within the index
299
+ user = SearchUser.create(id: 8, name: 'Parker')
300
+
301
+ # accessing the id, does NOT return the stored id by default - this will be delegated to the primary_key '_id'.
302
+ user.id
303
+ # => 'b2e34xa2'
304
+
305
+ # -- ENABLE delegation -------------------------------------------------------------------
306
+ SearchUser.delegate_id_attribute = true
307
+
308
+ # create new user within the index
309
+ user = SearchUser.create(id: 9, name: 'Pam')
310
+
311
+ # accessing the id accesses the stored attribute now
312
+ user.id
313
+ # => 9
314
+
315
+ # accessing the ES index id
316
+ user._id
317
+ # => 'xtf31bh8x'
318
+ ```
319
+
320
+ ## delegate_query_nil_limit
321
+ Elasticsearch's default value for queries without a **size** is forced to **10**.
322
+ To provide a similar behaviour as the (my)SQL interface,
323
+ this can be automatically set to the `max_result_window` value by calling `.limit(nil)` on the models' relation.
245
324
 
246
- ### Useful model class methods
325
+
326
+ ```ruby
327
+ SearchUser.where(name: 'Peter').limit(nil)
328
+ # returns a maximum of 10 items ...
329
+ # => [...]
330
+
331
+ # -- ENABLE delegation -------------------------------------------------------------------
332
+ SearchUser.delegate_query_nil_limit = true
333
+
334
+ SearchUser.where(name: 'Peter').limit(nil)
335
+ # returns up to 10_000 items ...
336
+ # => [...]
337
+
338
+ # hint: if you want more than 10_000 use the +#pit_results+ method!
339
+ ```
340
+
341
+ ## Useful model class methods
247
342
  - auto_increment?
248
343
  - max_result_window
249
344
  - source_column_names
@@ -251,26 +346,28 @@ _see simple documentation about these methods @ [rubydoc](https://rubydoc.info/g
251
346
  - find_by_query
252
347
  - msearch
253
348
 
254
- ### Useful model API methods
255
- Fast access to model-related methods for easier access without creating a overcomplicated method call.
349
+ ## Useful model API methods
350
+ Quick access to model-related methods for easier access without creating a overcomplicated method call on the models connection...
256
351
 
257
352
  Access these methods through the model class method `.api`.
258
353
  ```ruby
259
- # returns mapping of model class
260
- klass.api.mappings
354
+ # returns mapping of model class
355
+ klass.api.mappings
261
356
 
262
- # e.g. for ElasticUser model
263
- ElasticUser.api.mappings
357
+ # e.g. for ElasticUser model
358
+ SearchUser.api.mappings
264
359
 
265
- # insert new raw data
266
- ElasticUser.api.insert([{name: 'Hans', age: 34}, {name: 'Peter', age: 22}])
360
+ # insert new raw data
361
+ SearchUser.api.insert([{name: 'Hans', age: 34}, {name: 'Peter', age: 22}])
267
362
  ```
268
363
 
269
- * open
270
- * close
271
- * refresh
272
- * block
273
- * unblock
364
+ * open!
365
+ * close!
366
+ * refresh!
367
+ * block!
368
+ * unblock!
369
+ * drop!(confirm: true)
370
+ * truncate!(confirm: true)
274
371
  * mappings
275
372
  * metas
276
373
  * settings
@@ -290,12 +387,14 @@ Fast insert, update, delete raw data
290
387
  * delete
291
388
  * bulk
292
389
 
390
+ -----
391
+
293
392
  ## ActiveRecord ConnectionAdapters table-methods
294
393
  Access these methods through the model class method `.connection`.
295
394
 
296
395
  ```ruby
297
- # returns mapping of provided table (index)
298
- klass.connection.table_mappings('table-name')
396
+ # returns mapping of provided table (index)
397
+ klass.connection.table_mappings('table-name')
299
398
  ```
300
399
 
301
400
  - table_mappings
@@ -316,7 +415,7 @@ Access these methods through the model class method `.connection`.
316
415
  ## Active Record Schema migration methods
317
416
  Access these methods through the model's connection or within any `Migration`.
318
417
 
319
- **cluster actions:**
418
+ ### cluster actions:
320
419
  - open_table
321
420
  - open_tables
322
421
  - close_table
@@ -333,7 +432,7 @@ Access these methods through the model's connection or within any `Migration`.
333
432
  - change_table
334
433
  - rename_table
335
434
 
336
- **table actions:**
435
+ ### table actions:
337
436
  - change_meta
338
437
  - remove_meta
339
438
  - add_mapping
@@ -348,8 +447,10 @@ Access these methods through the model's connection or within any `Migration`.
348
447
  - change_alias
349
448
  - remove_alias
350
449
 
450
+
451
+ **Example migration:**
452
+
351
453
  ```ruby
352
- # Example migration
353
454
  class AddTests < ActiveRecord::Migration[7.0]
354
455
  def up
355
456
  create_table "assignments", if_not_exists: true do |t|
@@ -422,6 +523,47 @@ class AddTests < ActiveRecord::Migration[7.0]
422
523
  end
423
524
  ```
424
525
 
526
+
527
+ ## environment-related-table-name:
528
+ Using the `_env_table_name`-method will resolve the table (index) name within the current environment,
529
+ even if the environments shares the same cluster ...
530
+
531
+ This can be provided through the `database.yml` by using the `table_name_prefix/suffix` configuration keys.
532
+ Within the migration the `_env_table_name`-method must be used in combination with the table (index) base name.
533
+
534
+ **Example:**
535
+ Production uses a index suffix with '-pro', development uses '-dev' - they share the same cluster, but different indexes.
536
+ For the **settings** table:
537
+ - settings-pro
538
+ - settings-dev
539
+
540
+ A single migration can be created to be used within each environment:
541
+ ```ruby
542
+ # Example migration
543
+ class AddSettings < ActiveRecord::Migration[7.0]
544
+ def up
545
+ create_table _env_table_name("settings"), force: true do |t|
546
+ t.mapping :created_at, :date
547
+ t.mapping :key, :integer do |m|
548
+ m.primary_key = true
549
+ m.auto_increment = 10
550
+ end
551
+ t.mapping :status, :keyword
552
+ t.mapping :updated_at, :date
553
+ t.mapping :value, :text
554
+
555
+ t.setting "index.number_of_replicas", "0"
556
+ t.setting "index.number_of_shards", "1"
557
+ t.setting "index.routing.allocation.include._tier_preference", "data_content"
558
+ end
559
+ end
560
+
561
+ def down
562
+ drop_table _env_table_name("settings")
563
+ end
564
+ end
565
+ ```
566
+
425
567
  ## Docs
426
568
 
427
569
  [CHANGELOG](docs/CHANGELOG.md)
data/docs/CHANGELOG.md CHANGED
@@ -1,5 +1,17 @@
1
1
  # ElasticsearchRecord - CHANGELOG
2
2
 
3
+ ## [1.5.0] - 2023-07-10
4
+ * [add] additional `ElasticsearchRecord::ModelApi` methods **drop!** & **truncate!**, which have to be called with a `confirm:true` parameter
5
+ * [add] `.ElasticsearchRecord::Base.delegate_query_nil_limit` to automatically delegate a relations `limit(nil)`-call to the **max_result_window** _(set to 10.000 as default)_
6
+ * [add] `ActiveRecord::ConnectionAdapters::Elasticsearch::SchemaStatements#access_shard_doc?` which checks, if the **PIT**-shard_doc order is available
7
+ * [add] support for **_shard_doc** as a default order for `ElasticsearchRecord::Relation#pit_results`
8
+ * [ref] `.ElasticsearchRecord::Base.relay_id_attribute` to a more coherent name: `delegate_id_attribute`
9
+ * [ref] `ElasticsearchRecord::Relation#ordered_relation` to optimize already ordered relations
10
+ * [ref] gemspecs to support different versions of Elasticsearch
11
+ * [ref] improved README
12
+ * [fix] `ElasticsearchRecord::Relation#pit_results` infinite loop _(caused by missing order)_
13
+ * [fix] `ElasticsearchRecord::Relation#pit_results` results generation without 'uniq' check of the array
14
+
3
15
  ## [1.4.0] - 2023-01-27
4
16
  * [add] `ElasticsearchRecord::ModelApi` for fast & easy access the elasticsearch index - callable through `.api` (e.g. ElasticUser.api.mappings)
5
17
  * [ref] `ElasticsearchRecord::Instrumentation::LogSubscriber` to truncate the query-string (default: 1000)
@@ -32,8 +32,8 @@ DESC
32
32
 
33
33
  spec.require_paths = ["lib"]
34
34
 
35
- spec.add_dependency 'activerecord', '~> 7.0.0'
36
- spec.add_dependency 'elasticsearch', '~> 8.4'
35
+ spec.add_dependency 'activerecord', '~> 7.0'
36
+ spec.add_dependency 'elasticsearch', '>= 7.17'
37
37
 
38
38
  #spec.add_development_dependency 'coveralls_reborn', '~> 0.25'
39
39
  spec.add_development_dependency 'rspec', '~> 3.0'
@@ -376,6 +376,14 @@ module ActiveRecord
376
376
  @access_id_fielddata
377
377
  end
378
378
 
379
+ # returns true if +_shard_doc+ field can be accessed through PIT-search.
380
+ # @return [Boolean]
381
+ def access_shard_doc?
382
+ @access_shard_doc = cluster_info[:version] >= "7.12" if @access_shard_doc.nil?
383
+
384
+ @access_shard_doc
385
+ end
386
+
379
387
  # Returns basic information about the cluster.
380
388
  # @return [Hash{Symbol->Unknown}]
381
389
  def cluster_info
@@ -5,20 +5,25 @@ module ElasticsearchRecord
5
5
  included do
6
6
  # Rails resolves the primary_key's value by accessing the +#id+ method.
7
7
  # Since Elasticsearch also supports an additional, independent +id+ attribute, it would only be able to access
8
- # this through +read_attribute(:id)+.
8
+ # this through +_read_attribute(:id)+.
9
9
  # To also have the ability of accessing this attribute through the default, this flag can be enabled.
10
10
  # @attribute! Boolean
11
- class_attribute :relay_id_attribute, instance_writer: false, default: false
11
+ class_attribute :delegate_id_attribute, instance_writer: false, default: false
12
+
13
+ # Elasticsearch's default value for queries without a +size+ is forced to +10+.
14
+ # To provide a similar behaviour as SQL, this can be automatically set to the +max_result_window+ value.
15
+ # @attribute! Boolean
16
+ class_attribute :delegate_query_nil_limit, instance_writer: false, default: false
12
17
  end
13
18
 
14
19
  # overwrite to provide a Elasticsearch version of returning a 'primary_key' attribute.
15
20
  # Elasticsearch uses the static +_id+ column as primary_key, but also supports an additional +id+ column.
16
21
  # To provide functionality of returning the +id+ attribute, this method must also support it
17
- # with enabled +relay_id_attribute+.
22
+ # with enabled +delegate_id_attribute+.
18
23
  # @return [Object]
19
24
  def id
20
25
  # check, if the model has a +id+ attribute
21
- return _read_attribute('id') if relay_id_attribute? && has_attribute?('id')
26
+ return _read_attribute('id') if delegate_id_attribute? && has_attribute?('id')
22
27
 
23
28
  super
24
29
  end
@@ -26,11 +31,11 @@ module ElasticsearchRecord
26
31
  # overwrite to provide a Elasticsearch version of setting a 'primary_key' attribute.
27
32
  # Elasticsearch uses the static +_id+ column as primary_key, but also supports an additional +id+ column.
28
33
  # To provide functionality of setting the +id+ attribute, this method must also support it
29
- # with enabled +relay_id_attribute+.
34
+ # with enabled +delegate_id_attribute+.
30
35
  # @param [Object] value
31
36
  def id=(value)
32
37
  # check, if the model has a +id+ attribute
33
- return _write_attribute('id', value) if relay_id_attribute? && has_attribute?('id')
38
+ return _write_attribute('id', value) if delegate_id_attribute? && has_attribute?('id')
34
39
 
35
40
  # auxiliary update the +_id+ virtual column if we have a different primary_key
36
41
  _write_attribute('_id', value) if @primary_key != '_id'
@@ -41,13 +46,13 @@ module ElasticsearchRecord
41
46
  # overwrite to provide a Elasticsearch version of returning a 'primary_key' was attribute.
42
47
  # Elasticsearch uses the static +_id+ column as primary_key, but also supports an additional +id+ column.
43
48
  # To provide functionality of returning the +id_Was+ attribute, this method must also support it
44
- # with enabled +relay_id_attribute+.
49
+ # with enabled +delegate_id_attribute+.
45
50
  def id_was
46
- relay_id_attribute? && has_attribute?('id') ? attribute_was('id') : super
51
+ delegate_id_attribute? && has_attribute?('id') ? attribute_was('id') : super
47
52
  end
48
53
 
49
54
  # overwrite the write_attribute method to always write to the 'id'-attribute, if present.
50
- # This methods does not check for +relay_id_attribute+ flag!
55
+ # This methods does not check for +delegate_id_attribute+ flag!
51
56
  # see @ ActiveRecord::AttributeMethods::Write#write_attribute
52
57
  def write_attribute(attr_name, value)
53
58
  return _write_attribute('id', value) if attr_name.to_s == 'id' && has_attribute?('id')
@@ -56,7 +61,7 @@ module ElasticsearchRecord
56
61
  end
57
62
 
58
63
  # overwrite read_attribute method to read from the 'id'-attribute, if present.
59
- # This methods does not check for +relay_id_attribute+ flag!
64
+ # This methods does not check for +delegate_id_attribute+ flag!
60
65
  # see @ ActiveRecord::AttributeMethods::Read#read_attribute
61
66
  def read_attribute(attr_name, &block)
62
67
  return _read_attribute('id', &block) if attr_name.to_s == 'id' && has_attribute?('id')
@@ -8,7 +8,7 @@ module ElasticsearchRecord
8
8
 
9
9
  module VERSION
10
10
  MAJOR = 1
11
- MINOR = 4
11
+ MINOR = 5
12
12
  TINY = 0
13
13
  PRE = nil
14
14
 
@@ -8,18 +8,47 @@ module ElasticsearchRecord
8
8
  @klass = klass
9
9
  end
10
10
 
11
- # undelegated schema methods: clone rename create drop
11
+ # undelegated schema methods: clone rename create
12
12
  # those should not be quick-accessible, since they might end in heavily broken index
13
13
 
14
14
  # delegated dangerous methods (created with exclamation mark)
15
- # not able to provide individual arguments - always the defaults will be used
15
+ # not able to provide individual arguments - always the defaults will be used!
16
+ #
17
+ # @example
18
+ # open!
19
+ # close!
20
+ # refresh!
21
+ # block!
22
+ # unblock!
16
23
  %w(open close refresh block unblock).each do |method|
17
24
  define_method("#{method}!") do
18
25
  _connection.send("#{method}_table", _index_name)
19
26
  end
20
27
  end
21
28
 
29
+ # delegated dangerous methods with confirm parameter (created with exclamation mark)
30
+ # a exception will be raised, if +confirm:true+ is missing.
31
+ #
32
+ # @example
33
+ # drop!(confirm: true)
34
+ # truncate!(confirm: true)
35
+ %w(drop truncate).each do |method|
36
+ define_method("#{method}!") do |confirmed: false|
37
+ raise "#{method} of table '#{_index_name}' aborted!\nexecution not confirmed!\ncall with: #{klass}.api.#{method}!(confirmed: true)" unless confirmed
38
+ _connection.send("#{method}_table", _index_name)
39
+ end
40
+ end
41
+
22
42
  # delegated table methods
43
+ #
44
+ # @example
45
+ # mappings
46
+ # metas
47
+ # settings
48
+ # aliases
49
+ # state
50
+ # schema
51
+ # exists?
23
52
  %w(mappings metas settings aliases state schema exists?).each do |method|
24
53
  define_method(method) do |*args|
25
54
  _connection.send("table_#{method}", _index_name, *args)
@@ -27,6 +56,12 @@ module ElasticsearchRecord
27
56
  end
28
57
 
29
58
  # delegated plain methods
59
+ #
60
+ # @example
61
+ # alias_exists?
62
+ # setting_exists?
63
+ # mapping_exists?
64
+ # meta_exists?
30
65
  %w(alias_exists? setting_exists? mapping_exists? meta_exists?).each do |method|
31
66
  define_method(method) do |*args|
32
67
  _connection.send(method, _index_name, *args)
@@ -89,7 +124,7 @@ module ElasticsearchRecord
89
124
  if data[0].is_a?(Hash)
90
125
  bulk(data, :delete, **options)
91
126
  else
92
- bulk(data.map{|id| {id: id}}, :delete, **options)
127
+ bulk(data.map { |id| { id: id } }, :delete, **options)
93
128
  end
94
129
  end
95
130
 
@@ -102,7 +137,7 @@ module ElasticsearchRecord
102
137
 
103
138
  _connection.api(:core, :bulk, {
104
139
  index: _index_name,
105
- body: data.map{|item| {operation => {_id: item[:id], data: item.except(:id)}}},
140
+ body: data.map { |item| { operation => { _id: item[:id], data: item.except(:id) } } },
106
141
  refresh: refresh
107
142
  }, "BULK #{operation.to_s.upcase}", **options)
108
143
  end
@@ -99,21 +99,29 @@ module ElasticsearchRecord
99
99
  # overwrite original methods to provide a elasticsearch version:
100
100
  # checks against the +#access_id_fielddata?+ to ensure the Elasticsearch Cluster allows access on the +_id+ field.
101
101
  def ordered_relation
102
- # resolve valid primary_key (either not the '_id' or +access_id_fielddata?+ is enabled)
102
+ # order values already exist
103
+ return self unless order_values.empty?
104
+
105
+ # resolve valid primary_key
106
+ # - either it is NOT the '_id' column
107
+ # OR
108
+ # - it is the '_id'-column, but +access_id_fielddata?+ is also enabled!
103
109
  valid_primary_key = if primary_key != '_id' || klass.connection.access_id_fielddata?
104
- primary_key
105
- else
106
- nil
107
- end
110
+ primary_key
111
+ else
112
+ nil
113
+ end
108
114
 
109
115
  # slightly changed original methods content
110
- if order_values.empty? && (implicit_order_column || valid_primary_key)
116
+ if implicit_order_column || valid_primary_key
117
+ # order by +implicit_order_column+ AND +primary_key+
111
118
  if implicit_order_column && valid_primary_key && implicit_order_column != valid_primary_key
112
119
  order(table[implicit_order_column].asc, table[valid_primary_key].asc)
113
120
  else
114
121
  order(table[implicit_order_column || valid_primary_key].asc)
115
122
  end
116
123
  else
124
+ # order is not possible due restricted settings
117
125
  self
118
126
  end
119
127
  end
@@ -91,15 +91,20 @@ module ElasticsearchRecord
91
91
  # @param [String] keep_alive - how long to keep alive (for each single request) - default: '1m'
92
92
  # @param [Integer] batch_size - how many results per query (default: 1000 - this means at least 10 queries before reaching the +max_result_window+)
93
93
  def pit_results(keep_alive: '1m', batch_size: 1000)
94
- raise ArgumentError, "Batch size cannot be above the 'max_result_window' (#{klass.max_result_window}) !" if batch_size > klass.max_result_window
94
+ raise(ArgumentError, "Batch size cannot be above the 'max_result_window' (#{klass.max_result_window}) !") if batch_size > klass.max_result_window
95
95
 
96
- # check if a limit or offset values was provided
96
+ # check if limit or offset values where provided
97
97
  results_limit = limit_value ? limit_value : Float::INFINITY
98
98
  results_offset = offset_value ? offset_value : 0
99
99
 
100
100
  # search_after requires a order - we resolve a order either from provided value or by default ...
101
101
  relation = ordered_relation
102
102
 
103
+ # FALLBACK (without any order) for restricted access to the '_id' field.
104
+ # with PIT a order by '_shard_doc' can also be used
105
+ # see @ https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html
106
+ relation.order!(_shard_doc: :asc) if relation.order_values.empty? && klass.connection.access_shard_doc?
107
+
103
108
  # clear limit & offset
104
109
  relation.offset!(nil).limit!(nil)
105
110
 
@@ -135,7 +140,7 @@ module ElasticsearchRecord
135
140
  if block_given?
136
141
  yield ranged_results
137
142
  else
138
- results |= ranged_results
143
+ results += ranged_results
139
144
  end
140
145
 
141
146
  # add to total
@@ -150,8 +155,8 @@ module ElasticsearchRecord
150
155
  # we ran out of data
151
156
  break if current_results_length < batch_size
152
157
 
153
- # additional security - required?
154
- # break if current_pit_hash[:search_after] == current_response['hits']['hits'][-1]['sort']
158
+ # additional security - prevents infinite loops
159
+ raise(::ActiveRecord::StatementInvalid, "'pit_results' aborted due an infinite loop error (invalid or missing order)") if current_pit_hash[:search_after] == current_response['hits']['hits'][-1]['sort'] && current_pit_hash[:pit][:id] == current_response['pit_id']
155
160
 
156
161
  # -------- NEXT LOOP changes --------
157
162
 
@@ -43,6 +43,15 @@ module ElasticsearchRecord
43
43
  @values[:aggs] = value
44
44
  end
45
45
 
46
+ # overwrite the limit_value setter, to provide a special behaviour of auto-setting the +max_result_window+.
47
+ def limit=(limit)
48
+ if limit == '__max__' || (limit.nil? && delegate_query_nil_limit?)
49
+ super(max_result_window)
50
+ else
51
+ super
52
+ end
53
+ end
54
+
46
55
  private
47
56
 
48
57
  # alternative method to avoid redefining the const +VALID_UNSCOPING_VALUES+
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: elasticsearch_record
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.4.0
4
+ version: 1.5.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Tobias Gonsior
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2023-01-27 00:00:00.000000000 Z
11
+ date: 2023-07-10 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: activerecord
@@ -16,28 +16,28 @@ dependencies:
16
16
  requirements:
17
17
  - - "~>"
18
18
  - !ruby/object:Gem::Version
19
- version: 7.0.0
19
+ version: '7.0'
20
20
  type: :runtime
21
21
  prerelease: false
22
22
  version_requirements: !ruby/object:Gem::Requirement
23
23
  requirements:
24
24
  - - "~>"
25
25
  - !ruby/object:Gem::Version
26
- version: 7.0.0
26
+ version: '7.0'
27
27
  - !ruby/object:Gem::Dependency
28
28
  name: elasticsearch
29
29
  requirement: !ruby/object:Gem::Requirement
30
30
  requirements:
31
- - - "~>"
31
+ - - ">="
32
32
  - !ruby/object:Gem::Version
33
- version: '8.4'
33
+ version: '7.17'
34
34
  type: :runtime
35
35
  prerelease: false
36
36
  version_requirements: !ruby/object:Gem::Requirement
37
37
  requirements:
38
- - - "~>"
38
+ - - ">="
39
39
  - !ruby/object:Gem::Version
40
- version: '8.4'
40
+ version: '7.17'
41
41
  - !ruby/object:Gem::Dependency
42
42
  name: rspec
43
43
  requirement: !ruby/object:Gem::Requirement