activemodel-datastore 0.9.0 → 0.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 3126172160a1df49727e648493401c8448f62a1442cba738dcf139d6406dfdce
4
- data.tar.gz: 24bc7d6985f16739f16561611255d23ac6862cd72502f50827462a3c0674e6aa
3
+ metadata.gz: ad3eda8122e638f1b739f22844f8f116154ce792563e757f944e1fd04c44145d
4
+ data.tar.gz: '09316b9ed555cc216f51692ca3d8ec944921149ac3a624d74c598a2a04dea2ef'
5
5
  SHA512:
6
- metadata.gz: bc4e6401b2932e11c37b509dc1f98561423b6908d0d11f0465363090239a67ae6f7dc02cc5aecedc6fb72e4e8168f44a6ddc9fe38d49665fc6fc95838e0144a8
7
- data.tar.gz: 7acf7b5d14306afb3412c7ac667806e05708de0b9a8e8348b172bafa337316f4b11160f68eee3bb1152e27d7fa6ae066d4d28e17f430365b6caf003a3dfdbd1c
6
+ metadata.gz: b83379af680f03b96c148997a59181a9a34403f2c537672fda824899c939dd23f755045399901be38518bbb1cbadfa7ea68c0e1e375c43b9f27b986cbdac29e7
7
+ data.tar.gz: 3a4145461ae6b6ba321a8b8136e86b665b6a73447ab6a6e42dbc7d2fad68cbe28001c873d6dd2547ffdfe15bfb0d77b75b4bed30612e10ee20def1a512afe5ed
data/CHANGELOG.md CHANGED
@@ -1,3 +1,25 @@
1
+ ### 0.11.0 / 2026-10-14
2
+
3
+ Upgrade notes:
4
+ * requires `google-cloud-datastore` 2.3.1 or later; older versions will no longer resolve
5
+ * `DATASTORE_NETWORK_TIMEOUT` is no longer read; it never took effect, so removing it from your environment changes nothing (see below for configuring per-RPC deadlines)
6
+ * slow-operation warnings are enabled by default, so successful Datastore calls taking over 5 seconds now log at WARN level; set `ActiveModel::Datastore.slow_operation_threshold = nil` to keep the previous behavior
7
+
8
+ Changes:
9
+ * fix `all(limit:)` returning a nil cursor for full pages split across Datastore response batches; use the final batch's end cursor
10
+ * retry `all(limit:)` as a whole page, so a failure in any later Datastore response batch gets the configured `read_retry_count` retries and logging, and `slow_operation_threshold` times every batch of the page; each attempt rebuilds the query from the original options and discards rows from the failed attempt
11
+ * retry each later batch of queries without a limit and each deferred lookup in `find` / `find_entities` with the configured `read_retry_count` and retry logging; one `slow_operation_threshold` warning covers the total time of the call's successful requests
12
+ * raise `ActiveModel::Datastore::Error` instead of `NoMethodError` or `TypeError` if Datastore reports more query results without an end cursor
13
+ * require Ruby 2.5 or newer and `google-cloud-datastore` 2.3.1 or later within 2.x for correct query limits across batches
14
+ * warn when a successful Datastore attempt exceeds `ActiveModel::Datastore.slow_operation_threshold` (enabled by default at 5 seconds; `nil` or a blank string disables slow-success warnings)
15
+ * add opt-in ID preallocation via `ActiveModel::Datastore.preallocate_ids` (disabled by default) to prevent duplicate entities when retrying a save; allocation is retried at most twice
16
+ * remove `DATASTORE_NETWORK_TIMEOUT`, which never took effect because the Datastore client's per-RPC 60-second deadlines override the client-wide timeout; set `Google::Cloud::Datastore::V1::Datastore::Client.configure { |c| c.rpcs.<rpc>.timeout = ... }` instead
17
+
18
+ ### 0.10.0 / 2026-09-22
19
+ * stop retrying invalid argument errors
20
+ * make read retry counts configurable independently of writes
21
+ * log a warning when a retry chain gives up, including its attempt count
22
+
1
23
  ### 0.9.0 / 2026-08-24
2
24
  * identify the Datastore operation, entity kind, and failed-attempt elapsed time in retry logs
3
25
  * use a configurable retry logger that defaults to Rails.logger and falls back to standard output
data/README.md CHANGED
@@ -89,6 +89,71 @@ configure it during application initialization:
89
89
  ActiveModel::Datastore.logger = MyApplication.logger
90
90
  ```
91
91
 
92
+ Successful attempts taking more than five seconds also log a warning, such as
93
+ `Datastore save for Report took 6000 ms`. This measures each attempt separately, excluding retry
94
+ backoff. Both retry helpers use this logging when an `operation:` label is provided.
95
+
96
+ `all(limit:)` retries and times the complete page, including all response batches. Each retry starts
97
+ from the original query options and discards rows from the failed attempt. A retry reads batches
98
+ that already succeeded again, and because cursors don't preserve a snapshot, the retried page can
99
+ include changes made between attempts. Queries without a limit and multi-key lookups (`find` with
100
+ several IDs, `find_entities`) retry each request separately, so a retry reads only the batch that
101
+ failed. Their slow-operation warning covers the whole call: the total time of its successful
102
+ requests, excluding failed attempts and retry backoff.
103
+
104
+ Configure the threshold in seconds during initialization:
105
+
106
+ ```ruby
107
+ ActiveModel::Datastore.slow_operation_threshold = 2
108
+ ```
109
+
110
+ Numbers and numeric strings (such as `"2"` from an environment variable) are accepted. Set it to
111
+ `nil` or a blank string to disable slow-success warnings. Invalid values raise when the setting
112
+ is assigned. Failure logging and retry behavior are unchanged.
113
+
114
+ To change the gem's read retry count (default: `5`), configure it during initialization:
115
+
116
+ ```ruby
117
+ ActiveModel::Datastore.read_retry_count = 2
118
+ ```
119
+
120
+ Set it to `0` to disable read retries. Write retry counts are unchanged. Invalid argument errors
121
+ are never retried.
122
+
123
+ ID preallocation is disabled by default, preserving the existing save behavior. To enable it for
124
+ models saved through `save` or `save!`, configure it during application initialization:
125
+
126
+ ```ruby
127
+ ActiveModel::Datastore.preallocate_ids = true
128
+ ```
129
+
130
+ The setter uses `ActiveModel::Type::Boolean`, so environment-variable strings such as `"false"`,
131
+ `"0"`, and `"off"` disable preallocation. Unset (`nil`) and empty values also leave it disabled.
132
+
133
+ When enabled, a new entity without an ID gets a complete key before the save retry loop. Every
134
+ attempt in that loop uses the same key, preventing duplicate entities if a commit succeeds but its
135
+ response fails. This adds one allocation RPC per create under normal conditions. Existing numeric
136
+ IDs and named keys require no allocation, and parent paths are preserved.
137
+
138
+ Allocation writes nothing, so it is retried at most twice before giving up, leaving more of the
139
+ request's time for the save. If allocation still fails, `save` returns `false` and `save!` raises
140
+ `ActiveModel::Datastore::EntityNotSavedError` without attempting the write.
141
+
142
+ Set the option to `false` to disable preallocation. It does not cover `update` on a model without an
143
+ ID, direct calls to `CloudDatastore.dataset.save`, or deduplication across separate save calls or
144
+ requests.
145
+
146
+ Each Datastore RPC has its own 60-second default deadline in the underlying client, which takes
147
+ precedence over the client-wide `timeout` option. Applications with a request timeout (for example,
148
+ Heroku's 30-second router timeout or Rack::Timeout) should give allocation a short deadline so a
149
+ stalled allocation cannot use the time the save needs:
150
+
151
+ ```ruby
152
+ Google::Cloud::Datastore::V1::Datastore::Client.configure do |config|
153
+ config.rpcs.allocate_ids.timeout = 3
154
+ end
155
+ ```
156
+
92
157
  There is an example Puma config file [here](https://github.com/Agrimatics/activemodel-datastore/blob/main/test/support/datastore_example_rails_app/config/puma.rb).
93
158
 
94
159
  ## <a name="model"></a>Model Example
@@ -31,8 +31,7 @@ module CloudDatastore
31
31
  end
32
32
 
33
33
  def self.dataset
34
- timeout = ENV.fetch('DATASTORE_NETWORK_TIMEOUT', 15).to_i
35
- @dataset ||= Google::Cloud.datastore(timeout: timeout)
34
+ @dataset ||= Google::Cloud.datastore
36
35
  end
37
36
 
38
37
  def self.reset_dataset
@@ -1,5 +1,5 @@
1
1
  module ActiveModel
2
2
  module Datastore
3
- VERSION = '0.9.0'
3
+ VERSION = '0.11.0'
4
4
  end
5
5
  end
@@ -119,6 +119,9 @@ module ActiveModel::Datastore
119
119
  include ActiveModel::Datastore::PropertyValues
120
120
  include ActiveModel::Datastore::TrackChanges
121
121
 
122
+ # Allocation writes nothing, so a failure can give up quickly and leave time for the save.
123
+ ALLOCATE_RETRY_COUNT = 2
124
+
122
125
  def self.logger
123
126
  @logger || (Rails.logger if defined?(Rails) == 'constant')
124
127
  end
@@ -127,8 +130,35 @@ module ActiveModel::Datastore
127
130
  @logger = logger
128
131
  end
129
132
 
133
+ def self.slow_operation_threshold
134
+ defined?(@slow_operation_threshold) ? @slow_operation_threshold : 5
135
+ end
136
+
137
+ def self.slow_operation_threshold=(seconds)
138
+ seconds = nil if seconds.is_a?(String) && seconds.strip.empty?
139
+ @slow_operation_threshold = seconds.nil? ? nil : Float(seconds)
140
+ end
141
+
142
+ def self.read_retry_count
143
+ @read_retry_count || 5
144
+ end
145
+
146
+ def self.read_retry_count=(count)
147
+ @read_retry_count = count
148
+ end
149
+
150
+ def self.preallocate_ids
151
+ @preallocate_ids || false
152
+ end
153
+
154
+ def self.preallocate_ids=(enabled)
155
+ @preallocate_ids = ActiveModel::Type::Boolean.new.cast(enabled)
156
+ end
157
+
130
158
  included do
131
- private_class_method :query_options, :query_sort, :query_property_filter, :find_all_entities
159
+ private_class_method :query_options, :query_sort, :query_property_filter, :find_all_entities,
160
+ :retry_read, :log_retry_failure, :log_slow_operation, :all_with_retries,
161
+ :missing_cursor_error, :retry_with_backoff
132
162
  define_model_callbacks :save, :update, :destroy
133
163
  attr_accessor :id, :parent_key_id, :entity_property_values
134
164
  end
@@ -210,7 +240,8 @@ module ActiveModel::Datastore
210
240
 
211
241
  run_callbacks :save do
212
242
  entity = build_entity(parent)
213
- success = self.class.retry_on_exception?(operation: 'save') do
243
+ success = preallocate_entity_id(entity)
244
+ success &&= self.class.retry_on_exception?(operation: 'save') do
214
245
  CloudDatastore.dataset.save entity
215
246
  end
216
247
  self.id = entity.key.id if success
@@ -219,6 +250,19 @@ module ActiveModel::Datastore
219
250
  end
220
251
  end
221
252
 
253
+ def preallocate_entity_id(entity)
254
+ return true unless ActiveModel::Datastore.preallocate_ids && entity.key.incomplete?
255
+
256
+ # A commit can succeed even when its response fails. Reuse one key for every save attempt.
257
+ key = self.class.retry_on_exception?(ALLOCATE_RETRY_COUNT, operation: 'allocate id') do
258
+ CloudDatastore.dataset.allocate_ids(entity.key, 1).first ||
259
+ raise(Google::Cloud::Error, 'allocate_ids returned no key')
260
+ end
261
+ return false unless key
262
+
263
+ entity.key = key
264
+ end
265
+
222
266
  # Methods defined here will be class methods when 'include ActiveModel::Datastore'.
223
267
  module ClassMethods
224
268
  ##
@@ -239,7 +283,7 @@ module ActiveModel::Datastore
239
283
  def find_entity(id_or_name, parent = nil)
240
284
  key = CloudDatastore.dataset.key name, id_or_name
241
285
  key.parent = parent if parent.present?
242
- retry_on_exception(operation: 'find') { CloudDatastore.dataset.find key }
286
+ retry_read('find') { CloudDatastore.dataset.find key }
243
287
  end
244
288
 
245
289
  ##
@@ -248,10 +292,9 @@ module ActiveModel::Datastore
248
292
  # additional values. LookupResults are returned in batches, and the batch size is
249
293
  # determined by the Datastore API. Batch size is not guaranteed. It will be affected
250
294
  # by the size of the data being returned, and by other forces such as how distributed
251
- # and/or consistent the data in Datastore is. Calling `all` on the LookupResults retrieves
252
- # all results by repeatedly loading #next until #next? returns false. The `all` method
253
- # returns an enumerator unless passed a block. We iterate on the enumerator to return
254
- # the model entity objects.
295
+ # and/or consistent the data in Datastore is. Deferred keys are retrieved by repeatedly
296
+ # loading #next until #next? returns false, retrying each request. We iterate on the
297
+ # resulting enumerator to return the model entity objects.
255
298
  #
256
299
  # @param [Integer, String] ids_or_names One or more ids to retrieve.
257
300
  # @param [Google::Cloud::Datastore::Key] parent The parent Key of the entity.
@@ -260,8 +303,7 @@ module ActiveModel::Datastore
260
303
  #
261
304
  def find_entities(*ids_or_names, parent: nil)
262
305
  ids_or_names = ids_or_names.flatten.compact.uniq
263
- lookup_results = find_all_entities(ids_or_names, parent)
264
- lookup_results.all.collect { |x| x }
306
+ find_all_entities(ids_or_names, parent).to_a
265
307
  end
266
308
 
267
309
  ##
@@ -273,8 +315,11 @@ module ActiveModel::Datastore
273
315
  # QueryResults are returned in batches, and the batch size is determined by the Datastore API.
274
316
  # Batch size is not guaranteed. It will be affected by the size of the data being returned,
275
317
  # and by other forces such as how distributed and/or consistent the data in Datastore is.
276
- # Calling `all` on the QueryResults retrieves all results by repeatedly loading #next until
277
- # #next? returns false. The `all` method returns an enumerator which from_entities iterates on.
318
+ # With a limit, batches are collected using #next? and #next, retaining the final batch's
319
+ # end cursor for a full page. Short pages return a nil cursor. The whole page is retried as one
320
+ # read, rebuilding the query from the original options so a retry never reuses the reduced
321
+ # limit or cursor of a partially read page. Without a limit, from_entities iterates on an
322
+ # enumerator that retrieves all batches, retrying each request.
278
323
  #
279
324
  # Be sure to use as narrow a search criteria as possible. Please use with caution.
280
325
  #
@@ -298,16 +343,26 @@ module ActiveModel::Datastore
298
343
  # can be used to query for additional results.
299
344
  #
300
345
  def all(options = {})
301
- next_cursor = nil
302
- query = build_query(options)
303
- query_results = retry_on_exception(operation: 'run query') do
304
- CloudDatastore.dataset.run query
346
+ unless options[:limit]
347
+ query = build_query(options)
348
+ return from_entities(all_with_retries('run query') { CloudDatastore.dataset.run query })
305
349
  end
306
- if options[:limit]
307
- next_cursor = query_results.cursor if query_results.size == options[:limit]
308
- return from_entities(query_results.all), next_cursor
350
+
351
+ # QueryResults#next lowers the shared query limit before each request, so every attempt
352
+ # builds a fresh query and page from the original options.
353
+ entities, next_cursor = retry_read('run query') do
354
+ query_results = CloudDatastore.dataset.run build_query(options)
355
+ page = []
356
+ loop do
357
+ page.concat(query_results)
358
+ break unless query_results.next?
359
+
360
+ query_results = query_results.next
361
+ raise missing_cursor_error('run query') unless query_results
362
+ end
363
+ [page, (query_results.cursor if page.size == options[:limit])]
309
364
  end
310
- from_entities(query_results.all)
365
+ [from_entities(entities.each), next_cursor]
311
366
  end
312
367
 
313
368
  ##
@@ -332,8 +387,7 @@ module ActiveModel::Datastore
332
387
  model_entity = from_entity(entity)
333
388
  expects_array ? [model_entity].compact : model_entity
334
389
  else
335
- lookup_results = find_all_entities(ids, parent)
336
- from_entities(lookup_results.all)
390
+ from_entities(find_all_entities(ids, parent))
337
391
  end
338
392
  end
339
393
 
@@ -354,17 +408,17 @@ module ActiveModel::Datastore
354
408
  query.ancestor(args[:ancestor]) if args[:ancestor]
355
409
  query.limit(1)
356
410
  query.where(args.keys[0].to_s, '=', args.values[0])
357
- query_results = retry_on_exception(operation: 'run query') do
358
- CloudDatastore.dataset.run query
359
- end
411
+ query_results = retry_read('run query') { CloudDatastore.dataset.run query }
360
412
  from_entity(query_results.first)
361
413
  end
362
414
 
363
415
  ##
364
416
  # Translates an Enumerator of Datastore::Entity objects to ActiveModel::Model objects.
365
417
  #
366
- # Results provided by the dataset `find_all` or `run query` will be a Dataset::LookupResults or
367
- # Dataset::QueryResults object. Invoking `all` on those objects returns an enumerator.
418
+ # Within the gem, entities come from all_with_retries, which fetches each later batch of a
419
+ # lookup or unlimited query with retries, or from a page that `all(limit:)` has already read.
420
+ # Calling `all` on Dataset::LookupResults or Dataset::QueryResults also returns an enumerator,
421
+ # but fetches later batches without the gem's retries and timing.
368
422
  #
369
423
  # @param [Enumerator] entities An enumerator representing the datastore entities.
370
424
  #
@@ -414,23 +468,46 @@ module ActiveModel::Datastore
414
468
  query_options(query, options)
415
469
  end
416
470
 
417
- def retry_on_exception?(max_retry_count = 5, operation: nil, kind: name)
471
+ def retry_on_exception?(max_retry_count = 5, operation: nil, kind: name, &block)
472
+ retry_with_backoff(max_retry_count, operation: operation, kind: kind, &block)
473
+ rescue Google::Cloud::Error
474
+ # Only the error that ends the retries reaches here; it has already been logged.
475
+ false
476
+ end
477
+
478
+ def retry_on_exception(max_retry_count = 5, operation: nil, kind: name, &block)
479
+ retry_with_backoff(max_retry_count, operation: operation, kind: kind, &block)
480
+ end
481
+
482
+ def log_google_cloud_error
483
+ yield
484
+ rescue Google::Cloud::Error => e
485
+ puts "\e[33m[#{e.message.inspect}]\e[0m"
486
+ raise e
487
+ end
488
+
489
+ # **************** private ****************
490
+
491
+ ##
492
+ # Shared by retry_on_exception and retry_on_exception?. Reading every batch of a call passes
493
+ # warn_slow: false and logs one slow-operation warning for the call instead of one per request.
494
+ #
495
+ def retry_with_backoff(max_retry_count, operation:, kind:, warn_slow: true)
418
496
  retries = 0
419
497
  sleep_time = 0.25
420
498
  begin
421
499
  started_at = Process.clock_gettime(Process::CLOCK_MONOTONIC) if operation
422
- yield
500
+ result = yield
501
+ log_slow_operation(operation: operation, kind: kind, started_at: started_at,
502
+ warn_slow: warn_slow)
503
+ result
423
504
  rescue Google::Cloud::Error => e
424
- return false if retries >= max_retry_count
505
+ giving_up = e.is_a?(Google::Cloud::InvalidArgumentError) || retries >= max_retry_count
506
+ ending = "retrying in #{sleep_time}#{' s' if operation}"
507
+ ending = "giving up after #{retries + 1} attempts" if giving_up
508
+ log_retry_failure(e, ending, operation: operation, kind: kind, started_at: started_at)
509
+ raise if giving_up
425
510
 
426
- if operation
427
- elapsed_ms = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - started_at) * 1000).round
428
- message = "\e[33mDatastore #{operation} failed for #{kind} after #{elapsed_ms} ms: " \
429
- "#{e.message.inspect}; retrying in #{sleep_time} s\e[0m"
430
- else
431
- message = "\e[33mRescued exception #{e.message.inspect}, retrying in #{sleep_time}\e[0m"
432
- end
433
- ActiveModel::Datastore.logger ? ActiveModel::Datastore.logger.warn(message) : puts(message)
434
511
  # 0.25, 0.5, 1, 2, and 4 second between retries.
435
512
  sleep sleep_time
436
513
  retries += 1
@@ -439,39 +516,69 @@ module ActiveModel::Datastore
439
516
  end
440
517
  end
441
518
 
442
- def retry_on_exception(max_retry_count = 5, operation: nil, kind: name)
443
- retries = 0
444
- sleep_time = 0.25
445
- begin
446
- started_at = Process.clock_gettime(Process::CLOCK_MONOTONIC) if operation
447
- yield
448
- rescue Google::Cloud::Error => e
449
- raise e if retries >= max_retry_count
519
+ def log_slow_operation(operation:, kind:, started_at: nil, elapsed: nil, warn_slow: true)
520
+ threshold = ActiveModel::Datastore.slow_operation_threshold
521
+ return unless warn_slow && operation && threshold
450
522
 
451
- if operation
452
- elapsed_ms = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - started_at) * 1000).round
453
- message = "\e[33mDatastore #{operation} failed for #{kind} after #{elapsed_ms} ms: " \
454
- "#{e.message.inspect}; retrying in #{sleep_time} s\e[0m"
455
- else
456
- message = "\e[33mRescued exception #{e.message.inspect}, retrying in #{sleep_time}\e[0m"
457
- end
458
- ActiveModel::Datastore.logger ? ActiveModel::Datastore.logger.warn(message) : puts(message)
459
- # 0.25, 0.5, 1, 2, and 4 second between retries.
460
- sleep sleep_time
461
- retries += 1
462
- sleep_time *= 2
463
- retry
523
+ elapsed ||= Process.clock_gettime(Process::CLOCK_MONOTONIC) - started_at
524
+ return unless elapsed > threshold
525
+
526
+ message = "\e[33mDatastore #{operation} for #{kind} took #{(elapsed * 1000).round} ms\e[0m"
527
+ ActiveModel::Datastore.logger ? ActiveModel::Datastore.logger.warn(message) : puts(message)
528
+ end
529
+
530
+ def log_retry_failure(error, ending, operation:, kind:, started_at:)
531
+ if operation
532
+ elapsed_ms = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - started_at) * 1000).round
533
+ message = "\e[33mDatastore #{operation} failed for #{kind} after #{elapsed_ms} ms: " \
534
+ "#{error.message.inspect}; #{ending}\e[0m"
535
+ else
536
+ message = "\e[33mRescued exception #{error.message.inspect}, #{ending}\e[0m"
464
537
  end
538
+ ActiveModel::Datastore.logger ? ActiveModel::Datastore.logger.warn(message) : puts(message)
465
539
  end
466
540
 
467
- def log_google_cloud_error
468
- yield
469
- rescue Google::Cloud::Error => e
470
- puts "\e[33m[#{e.message.inspect}]\e[0m"
471
- raise e
541
+ def retry_read(operation, &block)
542
+ retry_on_exception(ActiveModel::Datastore.read_retry_count, operation: operation, &block)
472
543
  end
473
544
 
474
- # **************** private ****************
545
+ ##
546
+ # Like QueryResults#all and LookupResults#all for the results of the given first request, but
547
+ # retries each batch request separately. Only for results whose #next repeats the same request:
548
+ # lookups and queries without a limit. A limited query lowers its shared limit in #next, so
549
+ # #all retries the whole page instead.
550
+ #
551
+ # One slow-operation warning covers the call: the total time of its successful requests,
552
+ # excluding failed attempts and retry backoff, as for a page of #all with a limit.
553
+ #
554
+ def all_with_retries(operation, &first_request)
555
+ retries = ActiveModel::Datastore.read_retry_count
556
+ Enumerator.new do |entities|
557
+ elapsed = 0.0
558
+ batch = nil
559
+ loop do
560
+ batch = retry_with_backoff(retries, operation: operation, kind: name, warn_slow: false) do
561
+ started_at = Process.clock_gettime(Process::CLOCK_MONOTONIC)
562
+ result = batch ? batch.next : first_request.call
563
+ elapsed += Process.clock_gettime(Process::CLOCK_MONOTONIC) - started_at
564
+ result
565
+ end
566
+ raise missing_cursor_error(operation) unless batch
567
+
568
+ batch.each { |entity| entities << entity }
569
+ break unless batch.next?
570
+ end
571
+ log_slow_operation(operation: operation, kind: name, elapsed: elapsed)
572
+ end
573
+ end
574
+
575
+ ##
576
+ # QueryResults#next returns nil when a batch reports more results without an end cursor.
577
+ # Datastore always sends one, so fail loudly rather than return incomplete results.
578
+ #
579
+ def missing_cursor_error(operation)
580
+ Error.new("Datastore #{operation} for #{name} reported more results without an end cursor")
581
+ end
475
582
 
476
583
  def query_options(query, options)
477
584
  query.ancestor(options[:ancestor]) if options[:ancestor]
@@ -513,15 +620,17 @@ module ActiveModel::Datastore
513
620
 
514
621
  ##
515
622
  # Finds entities by keys using the provided array items. Results provided by the
516
- # dataset `find_all` is a Dataset::LookupResults object.
623
+ # dataset `find_all` is a Dataset::LookupResults object, which may defer some keys
624
+ # to later requests.
517
625
  #
518
626
  # @param [Array<Integer>, Array<String>] ids_or_names An array of ids or names.
519
627
  #
628
+ # @return [Enumerator] the found entities, including deferred ones.
520
629
  #
521
630
  def find_all_entities(ids_or_names, parent)
522
631
  keys = ids_or_names.map { |id| CloudDatastore.dataset.key name, id }
523
632
  keys.map { |key| key.parent = parent } if parent.present?
524
- retry_on_exception(operation: 'find all') { CloudDatastore.dataset.find_all keys }
633
+ all_with_retries('find all') { CloudDatastore.dataset.find_all keys }
525
634
  end
526
635
 
527
636
  def build_model(entity)
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: activemodel-datastore
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.9.0
4
+ version: 0.11.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Bryce McLean
@@ -43,14 +43,20 @@ dependencies:
43
43
  requirements:
44
44
  - - "~>"
45
45
  - !ruby/object:Gem::Version
46
- version: '2.0'
46
+ version: '2.3'
47
+ - - ">="
48
+ - !ruby/object:Gem::Version
49
+ version: 2.3.1
47
50
  type: :runtime
48
51
  prerelease: false
49
52
  version_requirements: !ruby/object:Gem::Requirement
50
53
  requirements:
51
54
  - - "~>"
52
55
  - !ruby/object:Gem::Version
53
- version: '2.0'
56
+ version: '2.3'
57
+ - - ">="
58
+ - !ruby/object:Gem::Version
59
+ version: 2.3.1
54
60
  - !ruby/object:Gem::Dependency
55
61
  name: rake
56
62
  requirement: !ruby/object:Gem::Requirement
@@ -191,7 +197,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
191
197
  requirements:
192
198
  - - ">="
193
199
  - !ruby/object:Gem::Version
194
- version: 2.2.2
200
+ version: '2.5'
195
201
  required_rubygems_version: !ruby/object:Gem::Requirement
196
202
  requirements:
197
203
  - - ">="