dynamoid 3.8.0 → 3.12.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +89 -2
- data/README.md +375 -64
- data/SECURITY.md +17 -0
- data/dynamoid.gemspec +65 -0
- data/lib/dynamoid/adapter.rb +21 -14
- data/lib/dynamoid/adapter_plugin/aws_sdk_v3/create_table.rb +2 -2
- data/lib/dynamoid/adapter_plugin/aws_sdk_v3/execute_statement.rb +62 -0
- data/lib/dynamoid/adapter_plugin/aws_sdk_v3/filter_expression_convertor.rb +113 -0
- data/lib/dynamoid/adapter_plugin/aws_sdk_v3/item_updater.rb +29 -2
- data/lib/dynamoid/adapter_plugin/aws_sdk_v3/middleware/limit.rb +3 -0
- data/lib/dynamoid/adapter_plugin/aws_sdk_v3/projection_expression_convertor.rb +40 -0
- data/lib/dynamoid/adapter_plugin/aws_sdk_v3/query.rb +46 -61
- data/lib/dynamoid/adapter_plugin/aws_sdk_v3/scan.rb +34 -28
- data/lib/dynamoid/adapter_plugin/aws_sdk_v3/transact.rb +31 -0
- data/lib/dynamoid/adapter_plugin/aws_sdk_v3.rb +132 -74
- data/lib/dynamoid/associations/belongs_to.rb +6 -6
- data/lib/dynamoid/associations.rb +1 -1
- data/lib/dynamoid/components.rb +3 -3
- data/lib/dynamoid/config/options.rb +12 -12
- data/lib/dynamoid/config.rb +4 -0
- data/lib/dynamoid/criteria/chain.rb +165 -149
- data/lib/dynamoid/criteria/key_fields_detector.rb +6 -7
- data/lib/dynamoid/criteria/nonexistent_fields_detector.rb +2 -2
- data/lib/dynamoid/criteria/where_conditions.rb +36 -0
- data/lib/dynamoid/dirty.rb +145 -59
- data/lib/dynamoid/document.rb +39 -3
- data/lib/dynamoid/dumping.rb +41 -19
- data/lib/dynamoid/errors.rb +32 -3
- data/lib/dynamoid/fields/declare.rb +6 -6
- data/lib/dynamoid/fields.rb +21 -29
- data/lib/dynamoid/finders.rb +68 -51
- data/lib/dynamoid/indexes.rb +7 -10
- data/lib/dynamoid/loadable.rb +3 -2
- data/lib/dynamoid/log/formatter.rb +19 -4
- data/lib/dynamoid/persistence/import.rb +4 -1
- data/lib/dynamoid/persistence/inc.rb +82 -0
- data/lib/dynamoid/persistence/item_updater_with_casting_and_dumping.rb +36 -0
- data/lib/dynamoid/persistence/item_updater_with_dumping.rb +33 -0
- data/lib/dynamoid/persistence/save.rb +75 -17
- data/lib/dynamoid/persistence/update_fields.rb +24 -9
- data/lib/dynamoid/persistence/update_validations.rb +3 -3
- data/lib/dynamoid/persistence/upsert.rb +22 -8
- data/lib/dynamoid/persistence.rb +308 -72
- data/lib/dynamoid/transaction_read/find.rb +137 -0
- data/lib/dynamoid/transaction_read.rb +146 -0
- data/lib/dynamoid/transaction_write/base.rb +47 -0
- data/lib/dynamoid/transaction_write/create.rb +49 -0
- data/lib/dynamoid/transaction_write/delete_with_instance.rb +65 -0
- data/lib/dynamoid/transaction_write/delete_with_primary_key.rb +64 -0
- data/lib/dynamoid/transaction_write/destroy.rb +84 -0
- data/lib/dynamoid/transaction_write/item_updater.rb +55 -0
- data/lib/dynamoid/transaction_write/save.rb +169 -0
- data/lib/dynamoid/transaction_write/update_attributes.rb +46 -0
- data/lib/dynamoid/transaction_write/update_fields.rb +239 -0
- data/lib/dynamoid/transaction_write/upsert.rb +106 -0
- data/lib/dynamoid/transaction_write.rb +673 -0
- data/lib/dynamoid/type_casting.rb +18 -15
- data/lib/dynamoid/undumping.rb +14 -3
- data/lib/dynamoid/validations.rb +8 -5
- data/lib/dynamoid/version.rb +1 -1
- data/lib/dynamoid.rb +8 -0
- metadata +43 -49
- data/lib/dynamoid/criteria/ignored_conditions_detector.rb +0 -41
- data/lib/dynamoid/criteria/overwritten_conditions_detector.rb +0 -40
data/README.md
CHANGED
|
@@ -1,21 +1,18 @@
|
|
|
1
1
|
# Dynamoid
|
|
2
2
|
|
|
3
|
-
[![
|
|
4
|
-
[![
|
|
5
|
-
[![
|
|
6
|
-
[![
|
|
7
|
-
[![
|
|
8
|
-
[![
|
|
9
|
-
![
|
|
3
|
+
[![Gem Version][⛳️version-img]][⛳️gem]
|
|
4
|
+
[![Supported Build Status][🏘sup-wf-img]][🏘sup-wf]
|
|
5
|
+
[![Maintainability][⛳cclim-maint-img♻️]][⛳cclim-maint]
|
|
6
|
+
[![Coveralls][🏘coveralls-img]][🏘coveralls]
|
|
7
|
+
[![CodeCov][🖇codecov-img♻️]][🖇codecov]
|
|
8
|
+
[![Helpers][🖇triage-help-img]][🖇triage-help]
|
|
9
|
+
[![Contributors][🖐contributors-img]][🖐contributors]
|
|
10
|
+
[![RubyDoc.info][🚎yard-img]][🚎yard]
|
|
11
|
+
[![License][🖇src-license-img]][🖇src-license]
|
|
10
12
|
[![GitMoji][🖐gitmoji-img]][🖐gitmoji]
|
|
11
|
-
[![SemVer 2.0.0][🧮semver-img]][semver]
|
|
13
|
+
[![SemVer 2.0.0][🧮semver-img]][🧮semver]
|
|
12
14
|
[![Keep-A-Changelog 1.0.0][📗keep-changelog-img]][📗keep-changelog]
|
|
13
|
-
|
|
14
|
-
[🖐gitmoji]: https://gitmoji.dev
|
|
15
|
-
[🖐gitmoji-img]: https://img.shields.io/badge/gitmoji-3.9.0-FFDD67.svg?style=flat
|
|
16
|
-
[🧮semver-img]: https://img.shields.io/badge/semver-2.0.0-FFDD67.svg?style=flat
|
|
17
|
-
[📗keep-changelog]: https://keepachangelog.com/en/1.0.0/
|
|
18
|
-
[📗keep-changelog-img]: https://img.shields.io/badge/keep--a--changelog-1.0.0-FFDD67.svg?style=flat
|
|
15
|
+
[![Sponsor Project][🖇sponsor-img]][🖇sponsor]
|
|
19
16
|
|
|
20
17
|
Dynamoid is an ORM for Amazon's DynamoDB for Ruby applications. It
|
|
21
18
|
provides similar functionality to ActiveRecord and improves on Amazon's
|
|
@@ -27,8 +24,8 @@ DynamoDB is not like other document-based databases you might know, and
|
|
|
27
24
|
is very different indeed from relational databases. It sacrifices
|
|
28
25
|
anything beyond the simplest relational queries and transactional
|
|
29
26
|
support to provide a fast, cost-efficient, and highly durable storage
|
|
30
|
-
solution. If your database requires complicated relational queries
|
|
31
|
-
|
|
27
|
+
solution. If your database requires complicated relational queries
|
|
28
|
+
then this modest Gem cannot provide them for you
|
|
32
29
|
and neither can DynamoDB. In those cases you would do better to look
|
|
33
30
|
elsewhere for your database needs.
|
|
34
31
|
|
|
@@ -70,10 +67,10 @@ For example, to configure AWS access:
|
|
|
70
67
|
Create `config/initializers/aws.rb` as follows:
|
|
71
68
|
|
|
72
69
|
```ruby
|
|
73
|
-
Aws.config.update(
|
|
70
|
+
Aws.config.update(
|
|
74
71
|
region: 'us-west-2',
|
|
75
72
|
credentials: Aws::Credentials.new('REPLACE_WITH_ACCESS_KEY_ID', 'REPLACE_WITH_SECRET_ACCESS_KEY'),
|
|
76
|
-
|
|
73
|
+
)
|
|
77
74
|
```
|
|
78
75
|
|
|
79
76
|
Alternatively, if you don't want Aws connection settings to be
|
|
@@ -97,15 +94,15 @@ elsewhere in your project, etc.), you may do so:
|
|
|
97
94
|
require 'dynamoid'
|
|
98
95
|
|
|
99
96
|
credentials = Aws::AssumeRoleCredentials.new(
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
97
|
+
region: region,
|
|
98
|
+
access_key_id: key,
|
|
99
|
+
secret_access_key: secret,
|
|
100
|
+
role_arn: role_arn,
|
|
101
|
+
role_session_name: 'our-session'
|
|
102
|
+
)
|
|
106
103
|
|
|
107
104
|
Dynamoid.configure do |config|
|
|
108
|
-
config.region = 'us-west-2'
|
|
105
|
+
config.region = 'us-west-2'
|
|
109
106
|
config.credentials = credentials
|
|
110
107
|
end
|
|
111
108
|
```
|
|
@@ -135,8 +132,8 @@ end
|
|
|
135
132
|
Dynamoid supports Ruby >= 2.3 and Rails >= 4.2.
|
|
136
133
|
|
|
137
134
|
Its compatibility is tested against following Ruby versions: 2.3, 2.4,
|
|
138
|
-
2.5, 2.6, 2.7 and 3.
|
|
139
|
-
5.2, 6.0, 6.1
|
|
135
|
+
2.5, 2.6, 2.7, 3.0, 3.1, 3.2, 3.3, and 3.4, JRuby 9.4.x and against Rails versions: 4.2, 5.0, 5.1,
|
|
136
|
+
5.2, 6.0, 6.1, 7.0, 7.1, 7.2, and 8.0.
|
|
140
137
|
|
|
141
138
|
## Setup
|
|
142
139
|
|
|
@@ -345,6 +342,24 @@ class Document
|
|
|
345
342
|
end
|
|
346
343
|
```
|
|
347
344
|
|
|
345
|
+
#### Note on binary type
|
|
346
|
+
|
|
347
|
+
By default binary fields are persisted as DynamoDB String value encoded
|
|
348
|
+
in the Base64 encoding. DynamoDB supports binary data natively. To use
|
|
349
|
+
it instead of String a `store_binary_as_native` field option should be
|
|
350
|
+
set:
|
|
351
|
+
|
|
352
|
+
```ruby
|
|
353
|
+
class Document
|
|
354
|
+
include Dynamoid::Document
|
|
355
|
+
|
|
356
|
+
field :image, :binary, store_binary_as_native: true
|
|
357
|
+
end
|
|
358
|
+
```
|
|
359
|
+
|
|
360
|
+
There is also a global config option `store_binary_as_native` that is
|
|
361
|
+
`false` by default as well.
|
|
362
|
+
|
|
348
363
|
#### Magic Columns
|
|
349
364
|
|
|
350
365
|
You get magic columns of `id` (`string`), `created_at` (`datetime`), and
|
|
@@ -360,7 +375,6 @@ class User
|
|
|
360
375
|
field :number, :number
|
|
361
376
|
field :joined_at, :datetime
|
|
362
377
|
field :hash, :serialized
|
|
363
|
-
|
|
364
378
|
end
|
|
365
379
|
```
|
|
366
380
|
|
|
@@ -406,7 +420,7 @@ class Money
|
|
|
406
420
|
'serialized representation as a string'
|
|
407
421
|
end
|
|
408
422
|
|
|
409
|
-
def self.dynamoid_load(
|
|
423
|
+
def self.dynamoid_load(_serialized_str)
|
|
410
424
|
# parse serialized representation and return a Money instance
|
|
411
425
|
Money.new(1.23)
|
|
412
426
|
end
|
|
@@ -431,7 +445,7 @@ serializing. Example:
|
|
|
431
445
|
class Money; end
|
|
432
446
|
|
|
433
447
|
class MoneyAdapter
|
|
434
|
-
def self.dynamoid_load(
|
|
448
|
+
def self.dynamoid_load(_money_serialized_str)
|
|
435
449
|
Money.new(1.23)
|
|
436
450
|
end
|
|
437
451
|
|
|
@@ -457,6 +471,27 @@ method, which would return either `:string` or `:number`.
|
|
|
457
471
|
DynamoDB may support some other attribute types that are not yet
|
|
458
472
|
supported by Dynamoid.
|
|
459
473
|
|
|
474
|
+
If a custom type implements `#==` method you can specify `comparable:
|
|
475
|
+
true` option in a field declaration to specify that an object is safely
|
|
476
|
+
comparable for the purpose of detecting changes. By default old and new
|
|
477
|
+
objects will be compared by their serialized representation.
|
|
478
|
+
|
|
479
|
+
```ruby
|
|
480
|
+
class Money
|
|
481
|
+
# ...
|
|
482
|
+
|
|
483
|
+
def ==(other)
|
|
484
|
+
# comparison logic
|
|
485
|
+
end
|
|
486
|
+
end
|
|
487
|
+
|
|
488
|
+
class User
|
|
489
|
+
# ...
|
|
490
|
+
|
|
491
|
+
field :balance, Money, comparable: true
|
|
492
|
+
end
|
|
493
|
+
```
|
|
494
|
+
|
|
460
495
|
### Sort key
|
|
461
496
|
|
|
462
497
|
Along with partition key table may have a sort key. In order to declare
|
|
@@ -550,9 +585,18 @@ model.save(validate: false)
|
|
|
550
585
|
|
|
551
586
|
### Callbacks
|
|
552
587
|
|
|
553
|
-
Dynamoid also employs ActiveModel callbacks. Right now
|
|
554
|
-
|
|
555
|
-
|
|
588
|
+
Dynamoid also employs ActiveModel callbacks. Right now the following
|
|
589
|
+
callbacks are supported:
|
|
590
|
+
- `save` (before, after, around)
|
|
591
|
+
- `create` (before, after, around)
|
|
592
|
+
- `update` (before, after, around)
|
|
593
|
+
- `validation` (before, after)
|
|
594
|
+
- `destroy` (before, after, around)
|
|
595
|
+
- `after_touch`
|
|
596
|
+
- `after_initialize`
|
|
597
|
+
- `after_find`
|
|
598
|
+
|
|
599
|
+
Example:
|
|
556
600
|
|
|
557
601
|
```ruby
|
|
558
602
|
class User
|
|
@@ -597,6 +641,7 @@ with `inheritance_field` table option:
|
|
|
597
641
|
```ruby
|
|
598
642
|
class Car
|
|
599
643
|
include Dynamoid::Document
|
|
644
|
+
|
|
600
645
|
table inheritance_field: :my_new_type
|
|
601
646
|
|
|
602
647
|
field :my_new_type
|
|
@@ -684,19 +729,19 @@ address.save
|
|
|
684
729
|
To create multiple documents at once:
|
|
685
730
|
|
|
686
731
|
```ruby
|
|
687
|
-
User.create([{name: 'Josh'}, {name: 'Nick'}])
|
|
732
|
+
User.create([{ name: 'Josh' }, { name: 'Nick' }])
|
|
688
733
|
```
|
|
689
734
|
|
|
690
735
|
There is an efficient and low-level way to create multiple documents
|
|
691
736
|
(without validation and callbacks running):
|
|
692
737
|
|
|
693
738
|
```ruby
|
|
694
|
-
users = User.import([{name: 'Josh'}, {name: 'Nick'}])
|
|
739
|
+
users = User.import([{ name: 'Josh' }, { name: 'Nick' }])
|
|
695
740
|
```
|
|
696
741
|
|
|
697
742
|
### Querying
|
|
698
743
|
|
|
699
|
-
Querying can be done in one of
|
|
744
|
+
Querying can be done in one of the following ways:
|
|
700
745
|
|
|
701
746
|
```ruby
|
|
702
747
|
Address.find(address.id) # Find directly by ID.
|
|
@@ -705,6 +750,27 @@ Address.where(city: 'Chicago').all # Find by any number of matching criteria.
|
|
|
705
750
|
Address.find_by_city('Chicago') # The same as above, but using ActiveRecord's older syntax.
|
|
706
751
|
```
|
|
707
752
|
|
|
753
|
+
There is also a way to `#where` with a condition expression:
|
|
754
|
+
|
|
755
|
+
```ruby
|
|
756
|
+
Address.where('city = :c', c: 'Chicago')
|
|
757
|
+
```
|
|
758
|
+
|
|
759
|
+
A condition expression may contain operators (e.g. `<`, `>=`, `<>`),
|
|
760
|
+
keywords (e.g. `AND`, `OR`, `BETWEEN`) and built-in functions (e.g.
|
|
761
|
+
`begins_with`, `contains`) (see (documentation
|
|
762
|
+
)[https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.OperatorsAndFunctions.html]
|
|
763
|
+
for full syntax description).
|
|
764
|
+
|
|
765
|
+
**Warning:** Values (specified for a String condition expression) are
|
|
766
|
+
sent as is so Dynamoid field types that aren't supported natively by
|
|
767
|
+
DynamoDB (e.g. `datetime` and `date`) require explicit casting.
|
|
768
|
+
|
|
769
|
+
**Warning:** String condition expressions will be used by DynamoDB only
|
|
770
|
+
at filtering, so conditions on key attributes should be specified as a
|
|
771
|
+
Hash to perform Query operation instead of Scan. Don't use key
|
|
772
|
+
attributes in `#where`'s String condition expressions.
|
|
773
|
+
|
|
708
774
|
And you can also query on associations:
|
|
709
775
|
|
|
710
776
|
```ruby
|
|
@@ -718,18 +784,6 @@ join, but instead finds all the user's addresses and naively filters
|
|
|
718
784
|
them in Ruby. For large associations this is a performance hit compared
|
|
719
785
|
to relational database engines.
|
|
720
786
|
|
|
721
|
-
**WARNING:** There is a limitation of conditions passed to `where`
|
|
722
|
-
method. Only one condition for some particular field could be specified.
|
|
723
|
-
The last one only will be applied and others will be ignored. E.g. in
|
|
724
|
-
examples:
|
|
725
|
-
|
|
726
|
-
```ruby
|
|
727
|
-
User.where('age.gt': 10, 'age.lt': 20)
|
|
728
|
-
User.where(name: 'Mike').where('name.begins_with': 'Ed')
|
|
729
|
-
```
|
|
730
|
-
|
|
731
|
-
the first one will be ignored and the last one will be used.
|
|
732
|
-
|
|
733
787
|
**Warning:** There is a caveat with filtering documents by `nil` value
|
|
734
788
|
attribute. By default Dynamoid ignores attributes with `nil` value and
|
|
735
789
|
doesn't store them in a DynamoDB document. This behavior could be
|
|
@@ -747,7 +801,7 @@ If Dynamoid keeps `nil` value attributes `eq`/`ne` operators should be
|
|
|
747
801
|
used instead:
|
|
748
802
|
|
|
749
803
|
```ruby
|
|
750
|
-
Address.where(
|
|
804
|
+
Address.where(postcode: nil)
|
|
751
805
|
Address.where('postcode.ne': nil)
|
|
752
806
|
```
|
|
753
807
|
|
|
@@ -796,8 +850,8 @@ for requesting documents in batches:
|
|
|
796
850
|
|
|
797
851
|
```ruby
|
|
798
852
|
# Do some maintenance on the entire table without flooding DynamoDB
|
|
799
|
-
Address.batch(100).each { |
|
|
800
|
-
Address.record_limit(10_000).batch(100).each {
|
|
853
|
+
Address.batch(100).each { |addr| addr.do_some_work && sleep(0.01) }
|
|
854
|
+
Address.record_limit(10_000).batch(100).each { |addr| addr.do_some_work && sleep(0.01) } # Batch specified as part of a chain
|
|
801
855
|
```
|
|
802
856
|
|
|
803
857
|
The implication of batches is that the underlying requests are done in
|
|
@@ -849,13 +903,13 @@ operators are available: `gt`, `lt`, `gte`, `lte`, `begins_with`,
|
|
|
849
903
|
`between` as well as equality:
|
|
850
904
|
|
|
851
905
|
```ruby
|
|
852
|
-
Address.where(latitude:
|
|
853
|
-
Address.where('latitude.gt':
|
|
854
|
-
Address.where('latitude.lt':
|
|
855
|
-
Address.where('latitude.gte':
|
|
856
|
-
Address.where('latitude.lte':
|
|
906
|
+
Address.where(latitude: 10_212)
|
|
907
|
+
Address.where('latitude.gt': 10_212)
|
|
908
|
+
Address.where('latitude.lt': 10_212)
|
|
909
|
+
Address.where('latitude.gte': 10_212)
|
|
910
|
+
Address.where('latitude.lte': 10_212)
|
|
857
911
|
Address.where('city.begins_with': 'Lon')
|
|
858
|
-
Address.where('latitude.between': [
|
|
912
|
+
Address.where('latitude.between': [10_212, 20_000])
|
|
859
913
|
```
|
|
860
914
|
|
|
861
915
|
You are able to filter results on the DynamoDB side and specify
|
|
@@ -863,7 +917,7 @@ conditions for non-key fields. Following additional operators are
|
|
|
863
917
|
available: `in`, `contains`, `not_contains`, `null`, `not_null`:
|
|
864
918
|
|
|
865
919
|
```ruby
|
|
866
|
-
Address.where('city.in': [
|
|
920
|
+
Address.where('city.in': %w[London Edenburg Birmingham])
|
|
867
921
|
Address.where('city.contains': ['on'])
|
|
868
922
|
Address.where('city.not_contains': ['ing'])
|
|
869
923
|
Address.where('postcode.null': false)
|
|
@@ -882,6 +936,7 @@ It could be done with `project` method:
|
|
|
882
936
|
```ruby
|
|
883
937
|
class User
|
|
884
938
|
include Dynamoid::Document
|
|
939
|
+
|
|
885
940
|
field :name
|
|
886
941
|
end
|
|
887
942
|
|
|
@@ -921,8 +976,8 @@ If you have a range index, Dynamoid provides a number of additional
|
|
|
921
976
|
other convenience methods to make your life a little easier:
|
|
922
977
|
|
|
923
978
|
```ruby
|
|
924
|
-
User.where(
|
|
925
|
-
User.where(
|
|
979
|
+
User.where('created_at.gt': DateTime.now - 1.day).all
|
|
980
|
+
User.where('created_at.lt': DateTime.now - 1.day).all
|
|
926
981
|
```
|
|
927
982
|
|
|
928
983
|
It also supports `gte` and `lte`. Turning those into symbols and
|
|
@@ -941,7 +996,6 @@ validation and callbacks.
|
|
|
941
996
|
Address.find(id).update_attributes(city: 'Chicago')
|
|
942
997
|
Address.find(id).update_attribute(:city, 'Chicago')
|
|
943
998
|
Address.update(id, city: 'Chicago')
|
|
944
|
-
Address.update(id, { city: 'Chicago' }, if: { deliverable: true })
|
|
945
999
|
```
|
|
946
1000
|
|
|
947
1001
|
There are also some low level methods `#update`, `.update_fields` and
|
|
@@ -965,6 +1019,14 @@ Address.upsert(id, city: 'Chicago')
|
|
|
965
1019
|
Address.upsert(id, { city: 'Chicago' }, if: { deliverable: true })
|
|
966
1020
|
```
|
|
967
1021
|
|
|
1022
|
+
By default, `#upsert` will update all attributes of the document if it already exists.
|
|
1023
|
+
To idempotently create-but-not-update a record, apply the `unless_exists` condition
|
|
1024
|
+
to its keys when you upsert.
|
|
1025
|
+
|
|
1026
|
+
```ruby
|
|
1027
|
+
Address.upsert(id, { city: 'Chicago' }, { unless_exists: [:id] })
|
|
1028
|
+
```
|
|
1029
|
+
|
|
968
1030
|
### Deleting
|
|
969
1031
|
|
|
970
1032
|
In order to delete some items `delete_all` method should be used. Any
|
|
@@ -1046,6 +1108,213 @@ resolving the fields with a second query against the table since a query
|
|
|
1046
1108
|
against GSI then a query on base table is still likely faster than scan
|
|
1047
1109
|
on the base table*
|
|
1048
1110
|
|
|
1111
|
+
### Transactions in Dynamoid
|
|
1112
|
+
|
|
1113
|
+
> [!WARNING]
|
|
1114
|
+
> Please note that this API is experimental and can be changed in
|
|
1115
|
+
> future releases.
|
|
1116
|
+
|
|
1117
|
+
DynamoDB supports modifying and reading operations but there are some
|
|
1118
|
+
limitations:
|
|
1119
|
+
- read and write operation cannot be combined in the same transaction
|
|
1120
|
+
- operations are executed in batch, so operations should be given before
|
|
1121
|
+
actual execution and cannot be changed on the fly
|
|
1122
|
+
|
|
1123
|
+
#### Modifying transactions
|
|
1124
|
+
|
|
1125
|
+
Multiple modifying actions can be grouped together and submitted as an
|
|
1126
|
+
all-or-nothing operation. Atomic modifying operations are supported in
|
|
1127
|
+
Dynamoid using transactions. If any action in the transaction fails they
|
|
1128
|
+
all fail.
|
|
1129
|
+
|
|
1130
|
+
The following actions are supported:
|
|
1131
|
+
|
|
1132
|
+
* `#create`/`#create!` - add a new model if it does not already exist
|
|
1133
|
+
* `#save`/`#save!` - create or update model
|
|
1134
|
+
* `#update_attributes`/`#update_attributes!` - modifies one or more attributes from an existig
|
|
1135
|
+
model
|
|
1136
|
+
* `#delete` - remove an model without callbacks nor validations
|
|
1137
|
+
* `#destroy`/`#destroy!` - remove an model
|
|
1138
|
+
* `#upsert` - add a new model or update an existing one, no callbacks
|
|
1139
|
+
* `#update_fields` - update a model without its instantiation
|
|
1140
|
+
|
|
1141
|
+
These methods are supposed to behave exactly like their
|
|
1142
|
+
non-transactional counterparts.
|
|
1143
|
+
|
|
1144
|
+
##### Create models
|
|
1145
|
+
|
|
1146
|
+
Models can be created inside of a transaction. The partition and sort
|
|
1147
|
+
keys, if applicable, are used to determine uniqueness. Creating will
|
|
1148
|
+
fail with `Aws::DynamoDB::Errors::TransactionCanceledException` if a
|
|
1149
|
+
model already exists.
|
|
1150
|
+
|
|
1151
|
+
This example creates a user with a unique id and unique email address by
|
|
1152
|
+
creating 2 models. An additional model is upserted in the same
|
|
1153
|
+
transaction. Upsert will update `updated_at` but will not create
|
|
1154
|
+
`created_at`.
|
|
1155
|
+
|
|
1156
|
+
```ruby
|
|
1157
|
+
user_id = SecureRandom.uuid
|
|
1158
|
+
email = 'bob@bob.bob'
|
|
1159
|
+
|
|
1160
|
+
Dynamoid::TransactionWrite.execute do |txn|
|
|
1161
|
+
txn.create(User, id: user_id)
|
|
1162
|
+
txn.create(UserEmail, id: "UserEmail##{email}", user_id: user_id)
|
|
1163
|
+
txn.create(Address, id: 'A#2', street: '456')
|
|
1164
|
+
txn.upsert(Address, 'A#1', street: '123')
|
|
1165
|
+
end
|
|
1166
|
+
```
|
|
1167
|
+
|
|
1168
|
+
##### Save models
|
|
1169
|
+
|
|
1170
|
+
Models can be saved in a transaction. New records are created otherwise
|
|
1171
|
+
the model is updated. Save, create, update, validate and destroy
|
|
1172
|
+
callbacks are called around the transaction as appropriate. Validation
|
|
1173
|
+
failures will throw `Dynamoid::Errors::DocumentNotValid`.
|
|
1174
|
+
|
|
1175
|
+
```ruby
|
|
1176
|
+
user = User.find(1)
|
|
1177
|
+
article = Article.new(body: 'New article text', user_id: user.id)
|
|
1178
|
+
|
|
1179
|
+
Dynamoid::TransactionWrite.execute do |txn|
|
|
1180
|
+
txn.save(article)
|
|
1181
|
+
|
|
1182
|
+
user.last_article_id = article.id
|
|
1183
|
+
txn.save(user)
|
|
1184
|
+
end
|
|
1185
|
+
```
|
|
1186
|
+
|
|
1187
|
+
##### Update models
|
|
1188
|
+
|
|
1189
|
+
A model can be updated by providing a model or primary key, and the fields to update.
|
|
1190
|
+
|
|
1191
|
+
```ruby
|
|
1192
|
+
Dynamoid::TransactionWrite.execute do |txn|
|
|
1193
|
+
# change name and title for a user
|
|
1194
|
+
txn.update_attributes(user, name: 'bob', title: 'mister')
|
|
1195
|
+
|
|
1196
|
+
# sets the name and title for a user
|
|
1197
|
+
# The user is found by id (that equals 1)
|
|
1198
|
+
txn.update_fields(User, '1', name: 'bob', title: 'mister')
|
|
1199
|
+
|
|
1200
|
+
# sets the name, increments a count and deletes a field
|
|
1201
|
+
txn.update_fields(User, 1) do |t|
|
|
1202
|
+
t.set(name: 'bob')
|
|
1203
|
+
t.add(article_count: 1)
|
|
1204
|
+
t.delete(:title)
|
|
1205
|
+
end
|
|
1206
|
+
|
|
1207
|
+
# adds to a set of integers and deletes from a set of strings
|
|
1208
|
+
txn.update_fields(User, 2) do |t|
|
|
1209
|
+
t.add(friend_ids: [1, 2])
|
|
1210
|
+
t.delete(child_names: ['bebe'])
|
|
1211
|
+
end
|
|
1212
|
+
end
|
|
1213
|
+
```
|
|
1214
|
+
|
|
1215
|
+
##### Destroy or delete models
|
|
1216
|
+
|
|
1217
|
+
Models can be used or the model class and key can be specified.
|
|
1218
|
+
`#destroy` uses callbacks and validations. Use `#delete` to skip
|
|
1219
|
+
callbacks and validations.
|
|
1220
|
+
|
|
1221
|
+
```ruby
|
|
1222
|
+
article = Article.find('1')
|
|
1223
|
+
tag = article.tag
|
|
1224
|
+
|
|
1225
|
+
Dynamoid::TransactionWrite.execute do |txn|
|
|
1226
|
+
txn.destroy(article)
|
|
1227
|
+
txn.delete(tag)
|
|
1228
|
+
|
|
1229
|
+
txn.delete(Tag, '2') # delete record with hash key '2' if it exists
|
|
1230
|
+
txn.delete(Tag, 'key#abcd', 'range#1') # when sort key is required
|
|
1231
|
+
end
|
|
1232
|
+
```
|
|
1233
|
+
|
|
1234
|
+
##### Validation failures that don't raise
|
|
1235
|
+
|
|
1236
|
+
All of the transaction methods can be called without the `!` which
|
|
1237
|
+
results in `false` instead of a raised exception when validation fails.
|
|
1238
|
+
Ignoring validation failures can lead to confusion or bugs so always
|
|
1239
|
+
check return status when not using a method with `!`.
|
|
1240
|
+
|
|
1241
|
+
```ruby
|
|
1242
|
+
user = User.find('1')
|
|
1243
|
+
user.red = true
|
|
1244
|
+
|
|
1245
|
+
Dynamoid::TransactionWrite.execute do |txn|
|
|
1246
|
+
if txn.save(user) # won't raise validation exception
|
|
1247
|
+
txn.update_fields(UserCount, user.id, count: 5)
|
|
1248
|
+
else
|
|
1249
|
+
puts 'ALERT: user not valid, skipping'
|
|
1250
|
+
end
|
|
1251
|
+
end
|
|
1252
|
+
```
|
|
1253
|
+
|
|
1254
|
+
##### Incrementally building a transaction
|
|
1255
|
+
|
|
1256
|
+
Transactions can also be built without a block.
|
|
1257
|
+
|
|
1258
|
+
```ruby
|
|
1259
|
+
transaction = Dynamoid::TransactionWrite.new
|
|
1260
|
+
|
|
1261
|
+
transaction.create(User, id: user_id)
|
|
1262
|
+
transaction.create(UserEmail, id: "UserEmail##{email}", user_id: user_id)
|
|
1263
|
+
transaction.upsert(Address, 'A#1', street: '123')
|
|
1264
|
+
|
|
1265
|
+
transaction.commit # changes are persisted in this moment
|
|
1266
|
+
```
|
|
1267
|
+
|
|
1268
|
+
#### Reading transactions
|
|
1269
|
+
|
|
1270
|
+
Multiple reading actions can be grouped together and submitted as an
|
|
1271
|
+
all-or-nothing operation. Atomic operations are supported in Dynamoid
|
|
1272
|
+
using transactions. If any action in the transaction fails they all
|
|
1273
|
+
fail.
|
|
1274
|
+
|
|
1275
|
+
The following actions are supported:
|
|
1276
|
+
|
|
1277
|
+
* `#find` - load a single model or multiple models by its primary key
|
|
1278
|
+
|
|
1279
|
+
These methods are supposed to behave exactly like their
|
|
1280
|
+
non-transactional counterparts.
|
|
1281
|
+
|
|
1282
|
+
##### Find a model
|
|
1283
|
+
|
|
1284
|
+
The `#find` action can load single model or multiple ones. Different
|
|
1285
|
+
model classes can be mixed in the same transactions. Result is returned
|
|
1286
|
+
as a plain list of all the found models. The order is preserved.
|
|
1287
|
+
|
|
1288
|
+
```ruby
|
|
1289
|
+
user, address = Dynamoid::TransactionRead.execute do |t|
|
|
1290
|
+
t.find(User, user_id)
|
|
1291
|
+
t.find(Address, address_id)
|
|
1292
|
+
end
|
|
1293
|
+
```
|
|
1294
|
+
|
|
1295
|
+
Multiple primary keys can be specified at once:
|
|
1296
|
+
|
|
1297
|
+
```ruby
|
|
1298
|
+
users = Dynamoid::TransactionRead.execute do |t|
|
|
1299
|
+
t.find(User, [id1, id2, id3])
|
|
1300
|
+
end
|
|
1301
|
+
```
|
|
1302
|
+
|
|
1303
|
+
### PartiQL
|
|
1304
|
+
|
|
1305
|
+
To run PartiQL statements `Dynamoid.adapter.execute` method should be
|
|
1306
|
+
used:
|
|
1307
|
+
|
|
1308
|
+
```ruby
|
|
1309
|
+
Dynamoid.adapter.execute("UPDATE users SET name = 'Mike' WHERE id = '1'")
|
|
1310
|
+
```
|
|
1311
|
+
|
|
1312
|
+
Parameters are also supported:
|
|
1313
|
+
|
|
1314
|
+
```ruby
|
|
1315
|
+
Dynamoid.adapter.execute('SELECT * FROM users WHERE id = ?', ['1'])
|
|
1316
|
+
```
|
|
1317
|
+
|
|
1049
1318
|
## Configuration
|
|
1050
1319
|
|
|
1051
1320
|
Listed below are all configuration options.
|
|
@@ -1078,6 +1347,7 @@ Listed below are all configuration options.
|
|
|
1078
1347
|
* `write_capacity` - is used at table or indices creation. Default is 20
|
|
1079
1348
|
(units)
|
|
1080
1349
|
* `warn_on_scan` - log warnings when scan table. Default is `true`
|
|
1350
|
+
* `error_on_scan` - raises an error when scan table. Default is `false`
|
|
1081
1351
|
* `endpoint` - if provided, it communicates with the DynamoDB listening
|
|
1082
1352
|
at the endpoint. This is useful for testing with
|
|
1083
1353
|
[DynamoDB Local](http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Tools.DynamoDBLocal.html)
|
|
@@ -1114,14 +1384,18 @@ Listed below are all configuration options.
|
|
|
1114
1384
|
fields in ISO 8601 string format. Default is `false`
|
|
1115
1385
|
* `store_date_as_string` - if `true` then Dynamoid stores :date fields
|
|
1116
1386
|
in ISO 8601 string format. Default is `false`
|
|
1387
|
+
* `store_empty_string_as_nil` - store attribute's empty String value as NULL. Default is `true`
|
|
1117
1388
|
* `store_boolean_as_native` - if `true` Dynamoid stores boolean fields
|
|
1118
1389
|
as native DynamoDB boolean values. Otherwise boolean fields are stored
|
|
1119
|
-
as string values `'t'` and `'f'`. Default is true
|
|
1390
|
+
as string values `'t'` and `'f'`. Default is `true`
|
|
1391
|
+
* `store_binary_as_native` - if `true` Dynamoid stores binary fields
|
|
1392
|
+
as native DynamoDB binary values. Otherwise binary fields are stored
|
|
1393
|
+
as Base64 encoded string values. Default is `false`
|
|
1120
1394
|
* `backoff` - is a hash: key is a backoff strategy (symbol), value is
|
|
1121
1395
|
parameters for the strategy. Is used in batch operations. Default id
|
|
1122
1396
|
`nil`
|
|
1123
1397
|
* `backoff_strategies`: is a hash and contains all available strategies.
|
|
1124
|
-
Default is { constant: ..., exponential: ...}
|
|
1398
|
+
Default is `{ constant: ..., exponential: ...}`
|
|
1125
1399
|
* `log_formatter`: overrides default AWS SDK formatter. There are
|
|
1126
1400
|
several canned formatters: `Aws::Log::Formatter.default`,
|
|
1127
1401
|
`Aws::Log::Formatter.colored` and `Aws::Log::Formatter.short`. Please
|
|
@@ -1139,6 +1413,9 @@ Listed below are all configuration options.
|
|
|
1139
1413
|
* `http_read_timeout`:The number of seconds to wait for HTTP response
|
|
1140
1414
|
data. Default option value is `nil`. If not specified effected value
|
|
1141
1415
|
is `60`
|
|
1416
|
+
* `create_table_on_save`: if `true` then Dynamoid creates a
|
|
1417
|
+
corresponding table in DynamoDB at model persisting if the table
|
|
1418
|
+
doesn't exist yet. Default is `true`
|
|
1142
1419
|
|
|
1143
1420
|
|
|
1144
1421
|
## Concurrency
|
|
@@ -1277,11 +1554,18 @@ RSpec.configure do |config|
|
|
|
1277
1554
|
end
|
|
1278
1555
|
```
|
|
1279
1556
|
|
|
1557
|
+
In addition, the first test for each model may fail if the relevant models are not included in `included_models`. This can be fixed by adding this line before the `DynamoidReset` module:
|
|
1558
|
+
```ruby
|
|
1559
|
+
Dir[File.join(Dynamoid::Config.models_dir, '**/*.rb')].sort.each { |file| require file }
|
|
1560
|
+
```
|
|
1561
|
+
Note that this will require _all_ models in your models folder - you can also explicitly require only certain models if you would prefer to.
|
|
1562
|
+
|
|
1280
1563
|
In Rails, you may also want to ensure you do not delete non-test data
|
|
1281
1564
|
accidentally by adding the following to your test environment setup:
|
|
1282
1565
|
|
|
1283
1566
|
```ruby
|
|
1284
1567
|
raise "Tests should be run in 'test' environment only" if Rails.env != 'test'
|
|
1568
|
+
|
|
1285
1569
|
Dynamoid.configure do |config|
|
|
1286
1570
|
config.namespace = "#{Rails.application.railtie_name}_#{Rails.env}"
|
|
1287
1571
|
end
|
|
@@ -1296,6 +1580,7 @@ order to troubleshoot and debug issues just set it:
|
|
|
1296
1580
|
```ruby
|
|
1297
1581
|
class User
|
|
1298
1582
|
include Dynamoid::Document
|
|
1583
|
+
|
|
1299
1584
|
field name
|
|
1300
1585
|
end
|
|
1301
1586
|
|
|
@@ -1324,6 +1609,7 @@ just as accessible to the Ruby world as MongoDB.
|
|
|
1324
1609
|
Also, without contributors the project wouldn't be nearly as awesome. So
|
|
1325
1610
|
many thanks to:
|
|
1326
1611
|
|
|
1612
|
+
* [Chris Hobbs](https://github.com/ckhsponge)
|
|
1327
1613
|
* [Logan Bowers](https://github.com/loganb)
|
|
1328
1614
|
* [Lane LaRue](https://github.com/luxx)
|
|
1329
1615
|
* [Craig Heneveld](https://github.com/cheneveld)
|
|
@@ -1406,4 +1692,29 @@ See [LICENSE][license] for the official [Copyright Notice][copyright-notice-expl
|
|
|
1406
1692
|
|
|
1407
1693
|
[security]: https://github.com/Dynamoid/dynamoid/blob/master/SECURITY.md
|
|
1408
1694
|
|
|
1409
|
-
[
|
|
1695
|
+
[⛳️gem]: https://rubygems.org/gems/dynamoid
|
|
1696
|
+
[⛳️version-img]: http://img.shields.io/gem/v/dynamoid.svg
|
|
1697
|
+
[⛳cclim-maint]: https://codeclimate.com/github/Dynamoid/dynamoid/maintainability
|
|
1698
|
+
[⛳cclim-maint-img♻️]: https://api.codeclimate.com/v1/badges/27fd8b6b7ff338fa4914/maintainability
|
|
1699
|
+
[🏘coveralls]: https://coveralls.io/github/Dynamoid/dynamoid?branch=master
|
|
1700
|
+
[🏘coveralls-img]: https://coveralls.io/repos/github/Dynamoid/dynamoid/badge.svg?branch=master
|
|
1701
|
+
[🖇codecov]: https://codecov.io/gh/Dynamoid/dynamoid
|
|
1702
|
+
[🖇codecov-img♻️]: https://codecov.io/gh/Dynamoid/dynamoid/branch/master/graph/badge.svg?token=84WeeoxaN9
|
|
1703
|
+
[🖇src-license]: https://github.com/Dynamoid/dynamoid/blob/master/LICENSE.txt
|
|
1704
|
+
[🖇src-license-img]: https://img.shields.io/badge/License-MIT-green.svg
|
|
1705
|
+
[🖐gitmoji]: https://gitmoji.dev
|
|
1706
|
+
[🖐gitmoji-img]: https://img.shields.io/badge/gitmoji-3.9.0-FFDD67.svg?style=flat
|
|
1707
|
+
[🚎yard]: https://www.rubydoc.info/gems/dynamoid
|
|
1708
|
+
[🚎yard-img]: https://img.shields.io/badge/yard-docs-blue.svg?style=flat
|
|
1709
|
+
[🧮semver]: http://semver.org/
|
|
1710
|
+
[🧮semver-img]: https://img.shields.io/badge/semver-2.0.0-FFDD67.svg?style=flat
|
|
1711
|
+
[🖐contributors]: https://github.com/Dynamoid/dynamoid/graphs/contributors
|
|
1712
|
+
[🖐contributors-img]: https://img.shields.io/github/contributors-anon/Dynamoid/dynamoid
|
|
1713
|
+
[📗keep-changelog]: https://keepachangelog.com/en/1.0.0/
|
|
1714
|
+
[📗keep-changelog-img]: https://img.shields.io/badge/keep--a--changelog-1.0.0-FFDD67.svg?style=flat
|
|
1715
|
+
[🖇sponsor-img]: https://img.shields.io/opencollective/all/dynamoid
|
|
1716
|
+
[🖇sponsor]: https://opencollective.com/dynamoid
|
|
1717
|
+
[🖇triage-help]: https://www.codetriage.com/dynamoid/dynamoid
|
|
1718
|
+
[🖇triage-help-img]: https://www.codetriage.com/dynamoid/dynamoid/badges/users.svg
|
|
1719
|
+
[🏘sup-wf]: https://github.com/Dynamoid/dynamoid/actions/workflows/ci.yml?query=branch%3Amaster
|
|
1720
|
+
[🏘sup-wf-img]: https://github.com/Dynamoid/dynamoid/actions/workflows/ci.yml/badge.svg?branch=master
|
data/SECURITY.md
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# Security Policy
|
|
2
|
+
|
|
3
|
+
## Supported Versions
|
|
4
|
+
|
|
5
|
+
| Version | Supported |
|
|
6
|
+
|---------|-----------|
|
|
7
|
+
| 3.7.x | ✅ |
|
|
8
|
+
| <= 3.6 | ❌ |
|
|
9
|
+
| 2.x | ❌ |
|
|
10
|
+
| 1.x | ❌ |
|
|
11
|
+
| 0.x | ❌ |
|
|
12
|
+
|
|
13
|
+
## Reporting a Vulnerability
|
|
14
|
+
|
|
15
|
+
Peter Boling is responsible for the security maintenance of this gem. Please find a way
|
|
16
|
+
to [contact him directly](https://railsbling.com/contact) to report the issue. Include as much relevant information as
|
|
17
|
+
possible.
|