dynamoid 3.0.0 → 3.1.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 9cb9dd0de78473763541261beed167d81d2431b1
4
- data.tar.gz: 057eea674112a29e69a55794a2815f2af8b14a7d
3
+ metadata.gz: ec92a10397fa1be691eaebe98659952c7909ab97
4
+ data.tar.gz: 766016509f22d0ce3f4fe93ca68342ed6038d0b9
5
5
  SHA512:
6
- metadata.gz: 2b231fdf9a13c129634234b3e01b587bbe52594f4063f28e9b682f55c2941db8c74ffc0d1da72c79911eff228443032c062c8f4430960872f969bdcd5583e2fe
7
- data.tar.gz: 692db4e23c132dd5fb4266349c552f0e2561396f1443306380c1a3ae6d775b6a3a289720f3a26619f54332c08a0b0af07c6b9f977a289effb90d01678212fd9b
6
+ metadata.gz: 6b0de822f2cda614a74a933c714072b97525a7b1a03444f2b12034327a8769d61e13220eb9d0b0f7ec15b2fbc2bf244574efe45a0660fac26cc118bb84fe848e
7
+ data.tar.gz: d6f3287d3741c21167b521d35771a4d794a9dff90047f9b46ebaa09373c8776389d4afa161c1430559b6fda4e0b9728f22a0a0007dbcdce507d8eedccc31018c
@@ -6,6 +6,40 @@
6
6
 
7
7
  ## Fixes
8
8
 
9
+ ---
10
+
11
+
12
+
13
+ # 3.1.0
14
+
15
+ ## Improvements
16
+ * Feature: [#302](https://github.com/Dynamoid/dynamoid/pull/302) Add methods similar to `ActiveRecord::AttributeMethods::BeforeTypeCast`:
17
+ * method `attributes_before_type_cast`
18
+ * method `read_attribte_before_type_cast`
19
+ * methods `<name>_before_type_cast`
20
+ * Feature: [#303](https://github.com/Dynamoid/dynamoid/pull/303) Add `#update_attributes!` method
21
+ * Feature: [#304](https://github.com/Dynamoid/dynamoid/pull/304) Add `inheritance_field` option for `Document.table` method to specify column name for supporting STI and storing class name
22
+ * Feature: [#305](https://github.com/Dynamoid/dynamoid/pull/305) Add increment/decrement methods:
23
+ * `#increment`
24
+ * `#increment!`
25
+ * `#decrement`
26
+ * `#decrement!`
27
+ * `.inc`
28
+ * Feature: [#307](https://github.com/Dynamoid/dynamoid/pull/307) Allow to declare type of elements in `array`/`set` fields with `of` option. Only scalar types are supported as well as custom types
29
+ * Feature: [#312](https://github.com/Dynamoid/dynamoid/pull/312) Add Ability to specify network timeout connection settings (@lulu-ulul)
30
+ * Feature: [#313](https://github.com/Dynamoid/dynamoid/pull/313) Add support for backoff in scan and query (@bonty)
31
+ * Improvement: [#314](https://github.com/Dynamoid/dynamoid/pull/314) Re-implement `count` for `where`-chain query efficiently. So now `where(...).count` doesn't load all the documents, just statistics
32
+
33
+ ## Fixes
34
+ * Bug: [#298](https://github.com/Dynamoid/dynamoid/pull/298) Fix `raw` field storing when value is a Hash with non-string keys
35
+ * Bug: [#299](https://github.com/Dynamoid/dynamoid/pull/299) Fix `raw` fields - skip empty strings and sets
36
+ * Bug: [#309](https://github.com/Dynamoid/dynamoid/pull/309) Fix loading of a document that contains not declared in model class fields
37
+ * Bug: [#310](https://github.com/Dynamoid/dynamoid/pull/310) Fix `Adapter#list_tables` method to return names of all tables, not just first page (@knovoselic)
38
+ * Bug: [#311](https://github.com/Dynamoid/dynamoid/pull/311) Fix `consistent_read` option of `.find` (@kokuyouwind)
39
+ * Bug: [#319](https://github.com/Dynamoid/dynamoid/pull/319) Repair consistent reading for `find_all`
40
+ * Bug: [#317](https://github.com/Dynamoid/dynamoid/pull/317) Fix `create_tables` rake task
41
+
42
+
9
43
 
10
44
  # 3.0.0
11
45
 
data/README.md CHANGED
@@ -1,6 +1,6 @@
1
1
  # Dynamoid
2
2
 
3
- You are viewing the README for version 2 of Dynamoid. See the [CHANGELOG](https://github.com/Dynamoid/Dynamoid/blob/master/CHANGELOG.md#200) for details on breaking changes since 1.3.x.
3
+ You are viewing the README for version 3 of Dynamoid. See the [CHANGELOG](https://github.com/Dynamoid/Dynamoid/blob/master/CHANGELOG.md#200) for details on breaking changes since 1.3.x.
4
4
 
5
5
  For version 1.3.x use the [1-3-stable branch](https://github.com/Dynamoid/Dynamoid/blob/1-3-stable/README.md).
6
6
 
@@ -20,9 +20,9 @@ But if you want a fast, scalable, simple, easy-to-use database (and a Gem that s
20
20
  | gem name | dynamoid |
21
21
  | license | MIT |
22
22
  | download rank | [![Total Downloads](https://img.shields.io/gem/rt/Dynamoid.svg)](https://rubygems.org/gems/dynamoid) |
23
- | version | [![Gem Version](https://badge.fury.io/rb/dynamoid.svg)](https://rubygems.org/gems/dynamoid) |
24
- | dependencies | [![Dependency Status](https://gemnasium.com/badges/github.com/Dynamoid/Dynamoid.svg)](https://gemnasium.com/github.com/Dynamoid/Dynamoid) [![Depfu](https://badges.depfu.com/badges/6661c063c8e77a5008344fc7283a50aa/status.svg)](https://depfu.com)|
25
- | code quality | [![Code Climate](https://codeclimate.com/github/Dynamoid/Dynamoid.svg)](https://codeclimate.com/github/Dynamoid/Dynamoid) |
23
+ | version | [![Gem Version](https://badge.fury.io/rb/dynamoid.svg)](https://badge.fury.io/rb/dynamoid) |
24
+ | dependencies | [![Depfu](https://badges.depfu.com/badges/6661c063c8e77a5008344fc7283a50aa/status.svg)](https://depfu.com) |
25
+ | code quality | [![Code Climate](https://codeclimate.com/github/Dynamoid/dynamoid.svg)](https://codeclimate.com/github/Dynamoid/dynamoid) |
26
26
  | continuous integration | [![Build Status](https://travis-ci.org/Dynamoid/dynamoid.svg?branch=master)](https://travis-ci.org/Dynamoid/dynamoid) |
27
27
  | test coverage | [![Coverage Status](https://coveralls.io/repos/github/Dynamoid/Dynamoid/badge.svg?branch=master)](https://coveralls.io/github/Dynamoid/Dynamoid?branch=master) |
28
28
  | triage helpers | [![CodeTriage Helpers](https://www.codetriage.com/dynamoid/dynamoid/badges/users.svg)](https://www.codetriage.com/dynamoid/dynamoid) |
@@ -191,8 +191,16 @@ You have two options if you need to use a `datetime` field as a range key:
191
191
 
192
192
  #### Note on set type
193
193
 
194
- There is `of` option to declare the type of set elements. You can use
195
- `:integer` value only
194
+ `Dynamoid`'s type `set` is stored as DynamoDB's Set attribute type.
195
+ DynamoDB supports only Set of strings, numbers and binary.
196
+ Moreover Set *must* contain elements of the same type only.
197
+
198
+ In order to use some other `Dynamoid`'s types you can specify `of` option
199
+ to declare the type of set elements.
200
+
201
+ As a result of that DynamoDB limitation, in Dynamoid only the following
202
+ scalar types are supported (note: does not support `boolean`):
203
+ `integer`, `number`, `date`, `datetime`, `serializable` and custom types.
196
204
 
197
205
  ```ruby
198
206
  class Document
@@ -202,6 +210,38 @@ class Document
202
210
  end
203
211
  ```
204
212
 
213
+ It's possible to specify field options like `store_as_string` for `datetime` field
214
+ or `serializer` for `serializable` field for `set` elements type:
215
+
216
+ ```ruby
217
+ class Document
218
+ include DynamoId::Document
219
+
220
+ field :values, :set, of: { serialized: { serializer: JSON } }
221
+ field :dates, :set, of: { date: { store_as_string: true } }
222
+ field :datetimes, :set, of: { datetime: { store_as_string: false } }
223
+ end
224
+ ```
225
+
226
+ DynamoDB doesn't allow empty strings in fields configured as `set`.
227
+ Abiding by this restriction, when `Dynamoid` saves a document it removes all empty strings in set fields.
228
+
229
+ #### Note on array type
230
+
231
+ `Dynamoid`'s type `array` is stored as DynamoDB's List attribute type.
232
+ It can contain elements of different types (in contrast to Set attribute type).
233
+
234
+ If you need to store in array field elements of `datetime`, `date`,
235
+ `serializable` or some custom type, which DynamoDB doesn't support
236
+ natively, you should specify element type with `of` option:
237
+
238
+ ```ruby
239
+ class Document
240
+ include DynamoId::Document
241
+
242
+ field :dates, :array, of: :date
243
+ end
244
+ ```
205
245
 
206
246
  #### Magic Columns
207
247
 
@@ -399,6 +439,57 @@ animal.class
399
439
  #=> Cat
400
440
  ```
401
441
 
442
+ If you already have DynamoDB tables and `type` field already exists and has its own semantic it leads to conflict.
443
+ It's possible to tell Dynamoid to use another field (even not existing)
444
+ instead of `type` one with `inheritance_field` table option:
445
+
446
+ ```ruby
447
+ class Car
448
+ include Dynamoid::Document
449
+ table inheritance_field: :my_new_type
450
+
451
+ field :my_new_type
452
+ end
453
+
454
+ c = Car.create
455
+ c.my_new_type
456
+ #=> "Car"
457
+ ```
458
+
459
+ ### Type casting
460
+
461
+ Dynamid supports type casting and tryes to do it in the most convinient way.
462
+ Values for all fields (except custom type) are coerced to declared field types.
463
+
464
+ Some obvious rules are used, e.g.:
465
+
466
+ for boolean field:
467
+ ```ruby
468
+ document.boolean_field = 'off'
469
+ # => false
470
+ document.boolean_field = 'false'
471
+ # => false
472
+ document.boolean_field = 'some string'
473
+ # => true
474
+ ```
475
+
476
+ or for integer field:
477
+ ```ruby
478
+ document.integer_field = 42.3
479
+ # => 42
480
+ document.integer_field = '42.3'
481
+ # => 42
482
+ document.integer_field = true
483
+ # => 1
484
+ ```
485
+
486
+ If time zone isn't specified for `datetime` value - application time zone is used.
487
+
488
+ To access field value before type casting following method could be
489
+ used: `attributes_before_type_cast` and `read_attribute_before_type_cast`.
490
+
491
+ There is `<name>_before_type_cast` method for every field in a model as well.
492
+
402
493
  ## Usage
403
494
 
404
495
  ### Object Creation
@@ -724,7 +815,7 @@ Calls to `update` and `update!` also increment the `lock_version`, however they
724
815
  ### Backoff strategies
725
816
 
726
817
 
727
- You can use several methods that run efficiently in batch mode like `.find_all` and `.import`.
818
+ You can use several methods that run efficiently in batch mode like `.find_all` and `.import`. It affects `Query` and `Scan` operations as well.
728
819
 
729
820
  The backoff strategy will be used when, for any reason, some items could not be processed as part of a batch mode command.
730
821
  Operations will be re-run to process these items.
@@ -786,7 +877,7 @@ module DynamoidReset
786
877
  end
787
878
  Dynamoid.adapter.tables.clear
788
879
  # Recreate all tables to avoid unexpected errors
789
- Dynamoid.included_models.each(&:create_table)
880
+ Dynamoid.included_models.each { |m| m.create_table(sync: true) }
790
881
  end
791
882
  end
792
883
 
@@ -43,6 +43,7 @@ Gem::Specification.new do |spec|
43
43
  spec.add_runtime_dependency 'activemodel', '>=4'
44
44
  spec.add_runtime_dependency 'aws-sdk-dynamodb', '~> 1'
45
45
  spec.add_runtime_dependency 'concurrent-ruby', '>= 1.0'
46
+ spec.add_runtime_dependency 'null-logger'
46
47
 
47
48
  spec.add_development_dependency 'appraisal'
48
49
  spec.add_development_dependency 'bundler'
@@ -1,5 +1,8 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require_relative 'query'
4
+ require_relative 'scan'
5
+
3
6
  module Dynamoid
4
7
  module AdapterPlugin
5
8
  # The AwsSdkV3 adapter provides support for the aws-sdk version 2 for ruby.
@@ -49,6 +52,8 @@ module Dynamoid
49
52
  }
50
53
  BATCH_WRITE_ITEM_REQUESTS_LIMIT = 25
51
54
 
55
+ CONNECTION_CONFIG_OPTIONS = %i[endpoint region http_continue_timeout http_idle_timeout http_open_timeout http_read_timeout].freeze
56
+
52
57
  attr_reader :table_cache
53
58
 
54
59
  # Establish the connection to DynamoDB.
@@ -62,8 +67,8 @@ module Dynamoid
62
67
  def connection_config
63
68
  @connection_hash = {}
64
69
 
65
- if Dynamoid::Config.endpoint?
66
- @connection_hash[:endpoint] = Dynamoid::Config.endpoint
70
+ (Dynamoid::Config.settings.compact.keys & CONNECTION_CONFIG_OPTIONS).each do |option|
71
+ @connection_hash[option] = Dynamoid::Config.send(option)
67
72
  end
68
73
  if Dynamoid::Config.access_key?
69
74
  @connection_hash[:access_key_id] = Dynamoid::Config.access_key
@@ -71,9 +76,6 @@ module Dynamoid
71
76
  if Dynamoid::Config.secret_key?
72
77
  @connection_hash[:secret_access_key] = Dynamoid::Config.secret_key
73
78
  end
74
- if Dynamoid::Config.region?
75
- @connection_hash[:region] = Dynamoid::Config.region
76
- end
77
79
 
78
80
  # https://github.com/aws/aws-sdk-ruby/blob/master/gems/aws-sdk-core/lib/aws-sdk-core/plugins/logging.rb
79
81
  # https://github.com/aws/aws-sdk-ruby/blob/master/gems/aws-sdk-core/lib/aws-sdk-core/log/formatter.rb
@@ -177,7 +179,7 @@ module Dynamoid
177
179
  # @since 1.0.0
178
180
  #
179
181
  # @todo: Provide support for passing options to underlying batch_get_item
180
- def batch_get_item(table_ids, _options = {})
182
+ def batch_get_item(table_ids, options = {})
181
183
  request_items = Hash.new { |h, k| h[k] = [] }
182
184
  return request_items if table_ids.all? { |_k, v| v.blank? }
183
185
 
@@ -206,7 +208,8 @@ module Dynamoid
206
208
  end
207
209
 
208
210
  request_items[t] = {
209
- keys: keys
211
+ keys: keys,
212
+ consistent_read: options[:consistent_read]
210
213
  }
211
214
 
212
215
  results = client.batch_get_item(
@@ -426,12 +429,16 @@ module Dynamoid
426
429
  #
427
430
  # @todo Provide support for various options http://docs.aws.amazon.com/sdkforruby/api/Aws/DynamoDB/Client.html#get_item-instance_method
428
431
  def get_item(table_name, key, options = {})
432
+ options = options.dup
429
433
  options ||= {}
434
+
430
435
  table = describe_table(table_name)
431
436
  range_key = options.delete(:range_key)
437
+ consistent_read = options.delete(:consistent_read)
432
438
 
433
439
  item = client.get_item(table_name: table_name,
434
- key: key_stanza(table, key, range_key))[:item]
440
+ key: key_stanza(table, key, range_key),
441
+ consistent_read: consistent_read)[:item]
435
442
  item ? result_item_to_hash(item) : nil
436
443
  end
437
444
 
@@ -445,6 +452,8 @@ module Dynamoid
445
452
  #
446
453
  # @todo Provide support for various options http://docs.aws.amazon.com/sdkforruby/api/Aws/DynamoDB/Client.html#update_item-instance_method
447
454
  def update_item(table_name, key, options = {})
455
+ options = options.dup
456
+
448
457
  range_key = options.delete(:range_key)
449
458
  conditions = options.delete(:conditions)
450
459
  table = describe_table(table_name)
@@ -467,10 +476,16 @@ module Dynamoid
467
476
  # List all tables on DynamoDB.
468
477
  #
469
478
  # @since 1.0.0
470
- #
471
- # @todo Provide limit support http://docs.aws.amazon.com/sdkforruby/api/Aws/DynamoDB/Client.html#update_item-instance_method
472
479
  def list_tables
473
- client.list_tables[:table_names]
480
+ [].tap do |result|
481
+ start_table_name = nil
482
+ loop do
483
+ result_page = client.list_tables exclusive_start_table_name: start_table_name
484
+ start_table_name = result_page.last_evaluated_table_name
485
+ result.concat result_page.table_names
486
+ break unless start_table_name
487
+ end
488
+ end
474
489
  end
475
490
 
476
491
  # Persists an item on DynamoDB.
@@ -503,191 +518,66 @@ module Dynamoid
503
518
  # one range key to the hash.
504
519
  #
505
520
  # @param [String] table_name the name of the table
506
- # @param [Hash] opts the options to query the table with
507
- # @option opts [String] :hash_value the value of the hash key to find
508
- # @option opts [Number, Number] :range_between find the range key within this range
509
- # @option opts [Number] :range_greater_than find range keys greater than this
510
- # @option opts [Number] :range_less_than find range keys less than this
511
- # @option opts [Number] :range_gte find range keys greater than or equal to this
512
- # @option opts [Number] :range_lte find range keys less than or equal to this
521
+ # @param [Hash] options the options to query the table with
522
+ # @option options [String] :hash_value the value of the hash key to find
523
+ # @option options [Number, Number] :range_between find the range key within this range
524
+ # @option options [Number] :range_greater_than find range keys greater than this
525
+ # @option options [Number] :range_less_than find range keys less than this
526
+ # @option options [Number] :range_gte find range keys greater than or equal to this
527
+ # @option options [Number] :range_lte find range keys less than or equal to this
513
528
  #
514
529
  # @return [Enumerable] matching items
515
530
  #
516
531
  # @since 1.0.0
517
532
  #
518
533
  # @todo Provide support for various other options http://docs.aws.amazon.com/sdkforruby/api/Aws/DynamoDB/Client.html#query-instance_method
519
- def query(table_name, opts = {})
534
+ def query(table_name, options = {})
520
535
  table = describe_table(table_name)
521
- hk = (opts[:hash_key].present? ? opts.delete(:hash_key) : table.hash_key).to_s
522
- rng = (opts[:range_key].present? ? opts.delete(:range_key) : table.range_key).to_s
523
- q = opts.slice(
524
- :consistent_read,
525
- :scan_index_forward,
526
- :select,
527
- :index_name
528
- )
529
536
 
530
- opts.delete(:consistent_read)
531
- opts.delete(:scan_index_forward)
532
- opts.delete(:select)
533
- opts.delete(:index_name)
534
-
535
- # Deal with various limits and batching
536
- record_limit = opts.delete(:record_limit)
537
- scan_limit = opts.delete(:scan_limit)
538
- batch_size = opts.delete(:batch_size)
539
- exclusive_start_key = opts.delete(:exclusive_start_key)
540
- limit = [record_limit, scan_limit, batch_size].compact.min
541
-
542
- key_conditions = {
543
- hk => {
544
- comparison_operator: EQ,
545
- attribute_value_list: attribute_value_list(EQ, opts.delete(:hash_value).freeze)
546
- }
547
- }
548
-
549
- opts.each_pair do |k, _v|
550
- next unless (op = RANGE_MAP[k])
551
- key_conditions[rng] = {
552
- comparison_operator: op,
553
- attribute_value_list: attribute_value_list(op, opts.delete(k).freeze)
554
- }
555
- end
556
-
557
- query_filter = {}
558
- opts.reject { |k, _| k.in? RANGE_MAP.keys }.each do |attr, hash|
559
- query_filter[attr] = {
560
- comparison_operator: FIELD_MAP[hash.keys[0]],
561
- attribute_value_list: attribute_value_list(FIELD_MAP[hash.keys[0]], hash.values[0].freeze)
562
- }
537
+ Enumerator.new do |yielder|
538
+ Query.new(client, table, options).call.each do |page|
539
+ page.items.each { |row| yielder << result_item_to_hash(row) }
540
+ end
563
541
  end
542
+ end
564
543
 
565
- q[:limit] = limit if limit
566
- q[:exclusive_start_key] = exclusive_start_key if exclusive_start_key
567
- q[:table_name] = table_name
568
- q[:key_conditions] = key_conditions
569
- q[:query_filter] = query_filter
570
-
571
- Enumerator.new do |y|
572
- record_count = 0
573
- scan_count = 0
574
- loop do
575
- # Adjust the limit down if the remaining record and/or scan limit are
576
- # lower to obey limits. We can assume the difference won't be
577
- # negative due to break statements below but choose smaller limit
578
- # which is why we have 2 separate if statements.
579
- # NOTE: Adjusting based on record_limit can cause many HTTP requests
580
- # being made. We may want to change this behavior, but it affects
581
- # filtering on data with potentially large gaps.
582
- # Example:
583
- # User.where('created_at.gte' => 1.day.ago).record_limit(1000)
584
- # Records 1-999 User's that fit criteria
585
- # Records 1000-2000 Users's that do not fit criteria
586
- # Record 2001 fits criteria
587
- # The underlying implementation will have 1 page for records 1-999
588
- # then will request with limit 1 for records 1000-2000 (making 1000
589
- # requests of limit 1) until hit record 2001.
590
- if q[:limit] && record_limit && record_limit - record_count < q[:limit]
591
- q[:limit] = record_limit - record_count
592
- end
593
- if q[:limit] && scan_limit && scan_limit - scan_count < q[:limit]
594
- q[:limit] = scan_limit - scan_count
595
- end
596
-
597
- results = client.query(q)
598
- results.items.each { |row| y << result_item_to_hash(row) }
599
-
600
- record_count += results.items.size
601
- break if record_limit && record_count >= record_limit
602
-
603
- scan_count += results.scanned_count
604
- break if scan_limit && scan_count >= scan_limit
544
+ def query_count(table_name, options = {})
545
+ table = describe_table(table_name)
546
+ options[:select] = 'COUNT'
605
547
 
606
- if (lk = results.last_evaluated_key)
607
- q[:exclusive_start_key] = lk
608
- else
609
- break
610
- end
611
- end
612
- end
548
+ Query.new(client, table, options).call
549
+ .map(&:count)
550
+ .reduce(:+)
613
551
  end
614
552
 
615
553
  # Scan the DynamoDB table. This is usually a very slow operation as it naively filters all data on
616
554
  # the DynamoDB servers.
617
555
  #
618
556
  # @param [String] table_name the name of the table
619
- # @param [Hash] scan_hash a hash of attributes: matching records will be returned by the scan
557
+ # @param [Hash] conditions a hash of attributes: matching records will be returned by the scan
620
558
  #
621
559
  # @return [Enumerable] matching items
622
560
  #
623
561
  # @since 1.0.0
624
562
  #
625
563
  # @todo: Provide support for various options http://docs.aws.amazon.com/sdkforruby/api/Aws/DynamoDB/Client.html#scan-instance_method
626
- def scan(table_name, scan_hash = {}, select_opts = {})
627
- request = { table_name: table_name }
628
- request[:consistent_read] = true if select_opts.delete(:consistent_read)
629
-
630
- # Deal with various limits and batching
631
- record_limit = select_opts.delete(:record_limit)
632
- scan_limit = select_opts.delete(:scan_limit)
633
- batch_size = select_opts.delete(:batch_size)
634
- exclusive_start_key = select_opts.delete(:exclusive_start_key)
635
- request_limit = [record_limit, scan_limit, batch_size].compact.min
636
- request[:limit] = request_limit if request_limit
637
- request[:exclusive_start_key] = exclusive_start_key if exclusive_start_key
638
-
639
- if scan_hash.present?
640
- request[:scan_filter] = scan_hash.reduce({}) do |memo, (attr, cond)|
641
- memo.merge(attr.to_s => {
642
- comparison_operator: FIELD_MAP[cond.keys[0]],
643
- attribute_value_list: attribute_value_list(FIELD_MAP[cond.keys[0]], cond.values[0].freeze)
644
- })
564
+ def scan(table_name, conditions = {}, options = {})
565
+ table = describe_table(table_name)
566
+
567
+ Enumerator.new do |yielder|
568
+ Scan.new(client, table, conditions, options).call.each do |page|
569
+ page.items.each { |row| yielder << result_item_to_hash(row) }
645
570
  end
646
571
  end
572
+ end
647
573
 
648
- Enumerator.new do |y|
649
- record_count = 0
650
- scan_count = 0
651
- loop do
652
- # Adjust the limit down if the remaining record and/or scan limit are
653
- # lower to obey limits. We can assume the difference won't be
654
- # negative due to break statements below but choose smaller limit
655
- # which is why we have 2 separate if statements.
656
- # NOTE: Adjusting based on record_limit can cause many HTTP requests
657
- # being made. We may want to change this behavior, but it affects
658
- # filtering on data with potentially large gaps.
659
- # Example:
660
- # User.where('created_at.gte' => 1.day.ago).record_limit(1000)
661
- # Records 1-999 User's that fit criteria
662
- # Records 1000-2000 Users's that do not fit criteria
663
- # Record 2001 fits criteria
664
- # The underlying implementation will have 1 page for records 1-999
665
- # then will request with limit 1 for records 1000-2000 (making 1000
666
- # requests of limit 1) until hit record 2001.
667
- if request[:limit] && record_limit && record_limit - record_count < request[:limit]
668
- request[:limit] = record_limit - record_count
669
- end
670
- if request[:limit] && scan_limit && scan_limit - scan_count < request[:limit]
671
- request[:limit] = scan_limit - scan_count
672
- end
673
-
674
- results = client.scan(request)
675
- results.items.each { |row| y << result_item_to_hash(row) }
676
-
677
- record_count += results.items.size
678
- break if record_limit && record_count >= record_limit
679
-
680
- scan_count += results.scanned_count
681
- break if scan_limit && scan_count >= scan_limit
574
+ def scan_count(table_name, conditions = {}, options = {})
575
+ table = describe_table(table_name)
576
+ options[:select] = 'COUNT'
682
577
 
683
- # Keep pulling if we haven't finished paging in all data
684
- if (lk = results[:last_evaluated_key])
685
- request[:exclusive_start_key] = lk
686
- else
687
- break
688
- end
689
- end
690
- end
578
+ Scan.new(client, table, conditions, options).call
579
+ .map(&:count)
580
+ .reduce(:+)
691
581
  end
692
582
 
693
583
  #
@@ -968,6 +858,10 @@ module Dynamoid
968
858
  # @params [String] operator: value of RANGE_MAP or FIELD_MAP hash, e.g. "EQ", "LT" etc
969
859
  # @params [Object] value: scalar value or array/set
970
860
  def attribute_value_list(operator, value)
861
+ self.class.attribute_value_list(operator, value)
862
+ end
863
+
864
+ def self.attribute_value_list(operator, value)
971
865
  # For BETWEEN and IN operators we should keep value as is (it should be already an array)
972
866
  # For all the other operators we wrap the value with array
973
867
  if %w[BETWEEN IN].include?(operator)
@@ -1018,6 +912,10 @@ module Dynamoid
1018
912
  def item_count
1019
913
  schema[:item_count]
1020
914
  end
915
+
916
+ def name
917
+ schema[:table_name]
918
+ end
1021
919
  end
1022
920
 
1023
921
  #
@@ -1043,7 +941,7 @@ module Dynamoid
1043
941
  # add. values must be a Set, Array, or Numeric
1044
942
  #
1045
943
  def add(values)
1046
- @additions.merge!(values)
944
+ @additions.merge!(sanitize_attributes(values))
1047
945
  end
1048
946
 
1049
947
  #
@@ -1053,14 +951,14 @@ module Dynamoid
1053
951
  # to remove
1054
952
  #
1055
953
  def delete(values)
1056
- @deletions.merge!(values)
954
+ @deletions.merge!(sanitize_attributes(values))
1057
955
  end
1058
956
 
1059
957
  #
1060
958
  # Replaces the values of one or more attributes
1061
959
  #
1062
960
  def set(values)
1063
- @updates.merge!(values)
961
+ @updates.merge!(sanitize_attributes(values))
1064
962
  end
1065
963
 
1066
964
  #
@@ -1091,6 +989,14 @@ module Dynamoid
1091
989
  ret
1092
990
  end
1093
991
 
992
+ private
993
+
994
+ def sanitize_attributes(attributes)
995
+ attributes.transform_values do |v|
996
+ v.is_a?(Hash) ? v.stringify_keys : v
997
+ end
998
+ end
999
+
1094
1000
  ADD = 'ADD'
1095
1001
  DELETE = 'DELETE'
1096
1002
  PUT = 'PUT'
@@ -1099,6 +1005,8 @@ module Dynamoid
1099
1005
  def sanitize_item(attributes)
1100
1006
  attributes.reject do |_k, v|
1101
1007
  v.nil? || ((v.is_a?(Set) || v.is_a?(String)) && v.empty?)
1008
+ end.transform_values do |v|
1009
+ v.is_a?(Hash) ? v.stringify_keys : v
1102
1010
  end
1103
1011
  end
1104
1012
  end