dynamoid 3.2.0 → 3.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.travis.yml +9 -6
- data/Appraisals +8 -14
- data/CHANGELOG.md +24 -0
- data/README.md +493 -228
- data/gemfiles/rails_4_2.gemfile +5 -7
- data/gemfiles/rails_5_0.gemfile +4 -6
- data/gemfiles/rails_5_1.gemfile +4 -6
- data/gemfiles/rails_5_2.gemfile +4 -6
- data/gemfiles/rails_6_0.gemfile +8 -0
- data/lib/dynamoid.rb +1 -0
- data/lib/dynamoid/adapter.rb +3 -10
- data/lib/dynamoid/adapter_plugin/aws_sdk_v3.rb +25 -69
- data/lib/dynamoid/adapter_plugin/aws_sdk_v3/batch_get_item.rb +105 -0
- data/lib/dynamoid/adapter_plugin/aws_sdk_v3/create_table.rb +9 -4
- data/lib/dynamoid/adapter_plugin/aws_sdk_v3/query.rb +11 -4
- data/lib/dynamoid/adapter_plugin/aws_sdk_v3/scan.rb +11 -3
- data/lib/dynamoid/adapter_plugin/aws_sdk_v3/until_past_table_status.rb +3 -2
- data/lib/dynamoid/components.rb +6 -3
- data/lib/dynamoid/config.rb +1 -0
- data/lib/dynamoid/criteria.rb +1 -1
- data/lib/dynamoid/criteria/chain.rb +33 -6
- data/lib/dynamoid/criteria/key_fields_detector.rb +101 -32
- data/lib/dynamoid/dirty.rb +186 -34
- data/lib/dynamoid/document.rb +8 -216
- data/lib/dynamoid/fields.rb +8 -0
- data/lib/dynamoid/loadable.rb +31 -0
- data/lib/dynamoid/persistence.rb +177 -85
- data/lib/dynamoid/persistence/import.rb +72 -0
- data/lib/dynamoid/persistence/save.rb +63 -0
- data/lib/dynamoid/persistence/update_fields.rb +62 -0
- data/lib/dynamoid/persistence/upsert.rb +60 -0
- data/lib/dynamoid/version.rb +1 -1
- metadata +9 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: eac484a25ff6886d838773be1b34f1fa9c7a51bf75ec46ed93984e97f273575c
|
4
|
+
data.tar.gz: 349e23120b78776d352c576bd7ffe0d2d1a2b69b0b5107d81698435bcbe2fb2e
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: cdbe5b559ceea1a834c6bfdbd956bc0dcba57172be55a4003213dafdd54e702b8ae7c1f62f6a04c11f13f3f3907429cbff1e008ccfcef2b90e54b512c301b38f
|
7
|
+
data.tar.gz: 796c4496a3cabdde4433ee18797176d674e8be5ca887978d73792bea7fdf8cc0e697ca475c5b50cc972217524a6a006751f7497c84dbbe40a0a296bad5b1a79d
|
data/.travis.yml
CHANGED
@@ -3,18 +3,21 @@ sudo: required
|
|
3
3
|
language: ruby
|
4
4
|
rvm:
|
5
5
|
- ruby-2.3.8
|
6
|
-
- ruby-2.4.
|
7
|
-
- ruby-2.5.
|
8
|
-
- ruby-2.6.
|
9
|
-
- jruby-9.
|
10
|
-
- jruby-9.2.6.0
|
11
|
-
gemfile:
|
6
|
+
- ruby-2.4.6
|
7
|
+
- ruby-2.5.5
|
8
|
+
- ruby-2.6.3
|
9
|
+
- jruby-9.2.8.0
|
12
10
|
gemfile:
|
13
11
|
- gemfiles/rails_4_2.gemfile
|
14
12
|
- gemfiles/rails_5_0.gemfile
|
15
13
|
- gemfiles/rails_5_1.gemfile
|
16
14
|
- gemfiles/rails_5_2.gemfile
|
17
15
|
|
16
|
+
matrix:
|
17
|
+
include:
|
18
|
+
- rvm: ruby-2.6.3
|
19
|
+
gemfile: gemfiles/rails_6_0.gemfile
|
20
|
+
|
18
21
|
### BUILD LIFECYCLE STEPS ###
|
19
22
|
|
20
23
|
before_install:
|
data/Appraisals
CHANGED
@@ -1,28 +1,22 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
|
-
appraise 'rails-4-0' do
|
4
|
-
gem 'rails', '~> 4.0.0'
|
5
|
-
gem 'nokogiri', '~> 1.6.8' # can be removed once we drop support for Ruby 2.0.0
|
6
|
-
end
|
7
|
-
|
8
|
-
appraise 'rails-4-1' do
|
9
|
-
gem 'rails', '~> 4.1.0'
|
10
|
-
gem 'nokogiri', '~> 1.6.8' # can be removed once we drop support for Ruby 2.0.0
|
11
|
-
end
|
12
|
-
|
13
3
|
appraise 'rails-4-2' do
|
14
|
-
gem '
|
4
|
+
gem 'activemodel', '~> 4.2.0'
|
15
5
|
gem 'nokogiri', '~> 1.6.8' # can be removed once we drop support for Ruby 2.0.0
|
16
6
|
end
|
17
7
|
|
18
8
|
appraise 'rails-5-0' do
|
19
|
-
gem '
|
9
|
+
gem 'activemodel', '~> 5.0.0'
|
20
10
|
end
|
21
11
|
|
22
12
|
appraise 'rails-5-1' do
|
23
|
-
gem '
|
13
|
+
gem 'activemodel', '~> 5.1.0'
|
24
14
|
end
|
25
15
|
|
26
16
|
appraise 'rails-5-2' do
|
27
|
-
gem '
|
17
|
+
gem 'activemodel', '~> 5.2.0'
|
18
|
+
end
|
19
|
+
|
20
|
+
appraise 'rails-6-0' do
|
21
|
+
gem 'activemodel', '6.0.0.rc1'
|
28
22
|
end
|
data/CHANGELOG.md
CHANGED
@@ -2,6 +2,8 @@
|
|
2
2
|
|
3
3
|
## Breaking
|
4
4
|
|
5
|
+
## Features
|
6
|
+
|
5
7
|
## Improvements
|
6
8
|
|
7
9
|
## Fixes
|
@@ -10,6 +12,28 @@
|
|
10
12
|
|
11
13
|
|
12
14
|
|
15
|
+
# 3.3.0
|
16
|
+
|
17
|
+
## Features
|
18
|
+
|
19
|
+
* Feature: [#374](https://github.com/Dynamoid/dynamoid/pull/374) Add `#project` query method to load only specified fields
|
20
|
+
|
21
|
+
## Improvements
|
22
|
+
|
23
|
+
* Improvement: [#359](https://github.com/Dynamoid/dynamoid/pull/359) Add support of `NULL` and `NOT_NULL` operators
|
24
|
+
* Improvement: [#360](https://github.com/Dynamoid/dynamoid/pull/360) Add `store_attribute_with_nil_value` config option
|
25
|
+
* Improvement: [#368](https://github.com/Dynamoid/dynamoid/pull/368) Support Rails 6 (RC1)
|
26
|
+
|
27
|
+
## Fixes
|
28
|
+
* Fix: [#357](https://github.com/Dynamoid/dynamoid/pull/357) Fix synchronous table creation issue
|
29
|
+
* Fix: [#362](https://github.com/Dynamoid/dynamoid/pull/362) Fix issue with selecting Global Secondary Index (@atyndall)
|
30
|
+
* Fix: [#368](https://github.com/Dynamoid/dynamoid/pull/368) Repair `#previous_changes` method from Dirty API
|
31
|
+
* Fix: [#373](https://github.com/Dynamoid/dynamoid/pull/373) Fix threadsafety of loading `Dynamoid::Adapter` (@tsub)
|
32
|
+
|
33
|
+
---
|
34
|
+
|
35
|
+
|
36
|
+
|
13
37
|
# 3.2.0
|
14
38
|
|
15
39
|
## Features
|
data/README.md
CHANGED
@@ -9,26 +9,36 @@
|
|
9
9
|

|
10
10
|
|
11
11
|
Dynamoid is an ORM for Amazon's DynamoDB for Ruby applications. It
|
12
|
-
provides similar functionality to ActiveRecord and improves on
|
13
|
-
|
12
|
+
provides similar functionality to ActiveRecord and improves on Amazon's
|
13
|
+
existing
|
14
14
|
[HashModel](http://docs.amazonwebservices.com/AWSRubySDK/latest/AWS/Record/HashModel.html)
|
15
15
|
by providing better searching tools and native association support.
|
16
16
|
|
17
|
-
DynamoDB is not like other document-based databases you might know, and
|
17
|
+
DynamoDB is not like other document-based databases you might know, and
|
18
|
+
is very different indeed from relational databases. It sacrifices
|
19
|
+
anything beyond the simplest relational queries and transactional
|
20
|
+
support to provide a fast, cost-efficient, and highly durable storage
|
21
|
+
solution. If your database requires complicated relational queries and
|
22
|
+
transaction support, then this modest Gem cannot provide them for you,
|
23
|
+
and neither can DynamoDB. In those cases you would do better to look
|
24
|
+
elsewhere for your database needs.
|
18
25
|
|
19
|
-
But if you want a fast, scalable, simple, easy-to-use database (and a
|
26
|
+
But if you want a fast, scalable, simple, easy-to-use database (and a
|
27
|
+
Gem that supports it) then look no further!
|
20
28
|
|
21
29
|
## Installation
|
22
30
|
|
23
|
-
Installing Dynamoid is pretty simple. First include the Gem in your
|
31
|
+
Installing Dynamoid is pretty simple. First include the Gem in your
|
32
|
+
Gemfile:
|
24
33
|
|
25
34
|
```ruby
|
26
35
|
gem 'dynamoid'
|
27
36
|
```
|
28
37
|
## Prerequisities
|
29
38
|
|
30
|
-
Dynamoid depends on the aws-sdk, and this is tested on the current
|
31
|
-
|
39
|
+
Dynamoid depends on the aws-sdk, and this is tested on the current
|
40
|
+
version of aws-sdk (~> 3), rails (>= 4). Hence the configuration as
|
41
|
+
needed for aws to work will be dealt with by aws setup.
|
32
42
|
|
33
43
|
### AWS SDK Version Compatibility
|
34
44
|
|
@@ -51,45 +61,52 @@ For example, to configure AWS access:
|
|
51
61
|
Create `config/initializers/aws.rb` as follows:
|
52
62
|
|
53
63
|
```ruby
|
54
|
-
|
55
|
-
|
56
|
-
|
57
|
-
|
58
|
-
})
|
59
|
-
|
64
|
+
Aws.config.update({
|
65
|
+
region: 'us-west-2',
|
66
|
+
credentials: Aws::Credentials.new('REPLACE_WITH_ACCESS_KEY_ID', 'REPLACE_WITH_SECRET_ACCESS_KEY'),
|
67
|
+
})
|
60
68
|
```
|
61
69
|
|
62
|
-
Alternatively, if you don't want Aws connection settings to be
|
70
|
+
Alternatively, if you don't want Aws connection settings to be
|
71
|
+
overwritten for you entire project, you can specify connection settings
|
72
|
+
for Dynamoid only, by setting those in the `Dynamoid.configure` clause:
|
63
73
|
|
64
74
|
```ruby
|
65
|
-
|
66
|
-
|
67
|
-
|
68
|
-
|
69
|
-
|
70
|
-
|
75
|
+
require 'dynamoid'
|
76
|
+
Dynamoid.configure do |config|
|
77
|
+
config.access_key = 'REPLACE_WITH_ACCESS_KEY_ID'
|
78
|
+
config.secret_key = 'REPLACE_WITH_SECRET_ACCESS_KEY'
|
79
|
+
config.region = 'us-west-2'
|
80
|
+
end
|
71
81
|
```
|
72
82
|
|
73
83
|
For a full list of the DDB regions, you can go
|
74
84
|
[here](http://docs.aws.amazon.com/general/latest/gr/rande.html#ddb_region).
|
75
85
|
|
76
|
-
Then you need to initialize Dynamoid config to get it going. Put code
|
86
|
+
Then you need to initialize Dynamoid config to get it going. Put code
|
87
|
+
similar to this somewhere (a Rails initializer would be a great place
|
88
|
+
for this if you're using Rails):
|
77
89
|
|
78
90
|
```ruby
|
79
|
-
|
80
|
-
|
81
|
-
|
82
|
-
|
83
|
-
|
91
|
+
require 'dynamoid'
|
92
|
+
Dynamoid.configure do |config|
|
93
|
+
# To namespace tables created by Dynamoid from other tables you might have.
|
94
|
+
# Set to nil to avoid namespacing.
|
95
|
+
config.namespace = 'dynamoid_app_development'
|
96
|
+
|
97
|
+
# [Optional]. If provided, it communicates with the DB listening at the endpoint.
|
98
|
+
# This is useful for testing with [DynamoDB Local] (http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Tools.DynamoDBLocal.html).
|
99
|
+
config.endpoint = 'http://localhost:3000'
|
100
|
+
end
|
84
101
|
```
|
85
102
|
|
86
103
|
### Ruby & Rails Compatibility
|
87
104
|
|
88
105
|
Dynamoid supports Ruby >= 2.3 and Rails >= 4.2.
|
89
106
|
|
90
|
-
Its compatibility is tested against following Ruby versions: 2.3.
|
91
|
-
2.
|
92
|
-
|
107
|
+
Its compatibility is tested against following Ruby versions: 2.3, 2.4,
|
108
|
+
2.5 and 2.6, JRuby 9.2.8.0 and against Rails versions: 4.2, 5.0, 5.1,
|
109
|
+
5.2 and 6.0.
|
93
110
|
|
94
111
|
## Setup
|
95
112
|
|
@@ -99,12 +116,15 @@ You *must* include `Dynamoid::Document` in every Dynamoid model.
|
|
99
116
|
class User
|
100
117
|
include Dynamoid::Document
|
101
118
|
|
119
|
+
# fields declaration
|
102
120
|
end
|
103
121
|
```
|
104
122
|
|
105
123
|
### Table
|
106
124
|
|
107
|
-
Dynamoid has some sensible defaults for you when you create a new table,
|
125
|
+
Dynamoid has some sensible defaults for you when you create a new table,
|
126
|
+
including the table name and the primary key column. But you can change
|
127
|
+
those if you like on table creation.
|
108
128
|
|
109
129
|
```ruby
|
110
130
|
class User
|
@@ -114,18 +134,32 @@ class User
|
|
114
134
|
end
|
115
135
|
```
|
116
136
|
|
117
|
-
These fields will not change an existing table: so specifying a new
|
137
|
+
These fields will not change an existing table: so specifying a new
|
138
|
+
read_capacity and write_capacity here only works correctly for entirely
|
139
|
+
new tables. Similarly, while Dynamoid will look for a table named
|
140
|
+
`awesome_users` in your namespace, it won't change any existing tables
|
141
|
+
to use that name; and if it does find a table with the correct name, it
|
142
|
+
won't change its hash key, which it expects will be `user_id`. If this
|
143
|
+
table doesn't exist yet, however, Dynamoid will create it with these
|
144
|
+
options.
|
118
145
|
|
119
146
|
### Fields
|
120
147
|
|
121
|
-
You'll have to define all the fields on the model and the data type of
|
122
|
-
|
123
|
-
|
124
|
-
|
125
|
-
|
126
|
-
|
127
|
-
|
128
|
-
|
148
|
+
You'll have to define all the fields on the model and the data type of
|
149
|
+
each field. Every field on the object must be included here; if you miss
|
150
|
+
any they'll be completely bypassed during DynamoDB's initialization and
|
151
|
+
will not appear on the model objects.
|
152
|
+
|
153
|
+
By default, fields are assumed to be of type `:string`. Other built-in
|
154
|
+
types are `:integer`, `:number`, `:set`, `:array`, `:map`, `:datetime`,
|
155
|
+
`date`, `:boolean`, `:raw` and `:serialized`. `array` and `map` match
|
156
|
+
List and Map DynamoDB types respectively. `raw` type means you can store
|
157
|
+
Ruby Array, Hash, String and numbers. If built-in types do not suit you,
|
158
|
+
you can use a custom field type represented by an arbitrary class,
|
159
|
+
provided that the class supports a compatible serialization interface.
|
160
|
+
The primary use case for using a custom field type is to represent your
|
161
|
+
business logic with high-level types, while ensuring portability or
|
162
|
+
backward-compatibility of the serialized representation.
|
129
163
|
|
130
164
|
#### Note on boolean type
|
131
165
|
|
@@ -144,7 +178,9 @@ end
|
|
144
178
|
|
145
179
|
#### Note on date type
|
146
180
|
|
147
|
-
By default date fields are persisted as days count since 1 January 1970
|
181
|
+
By default date fields are persisted as days count since 1 January 1970
|
182
|
+
like UNIX time. If you prefer dates to be stored as ISO-8601 formatted
|
183
|
+
strings instead then set `store_as_string` to `true`
|
148
184
|
|
149
185
|
```ruby
|
150
186
|
class Document
|
@@ -156,7 +192,10 @@ end
|
|
156
192
|
|
157
193
|
#### Note on datetime type
|
158
194
|
|
159
|
-
By default datetime fields are persisted as UNIX timestamps with
|
195
|
+
By default datetime fields are persisted as UNIX timestamps with
|
196
|
+
millisecond precision in DynamoDB. If you prefer datetimes to be stored
|
197
|
+
as ISO-8601 formatted strings instead then set `store_as_string` to
|
198
|
+
`true`
|
160
199
|
|
161
200
|
```ruby
|
162
201
|
class Document
|
@@ -166,26 +205,29 @@ class Document
|
|
166
205
|
end
|
167
206
|
```
|
168
207
|
|
169
|
-
**WARNING:** Fields in numeric format are stored with nanoseconds as a
|
170
|
-
That's why `datetime` field
|
208
|
+
**WARNING:** Fields in numeric format are stored with nanoseconds as a
|
209
|
+
fraction part and precision could be lost. That's why `datetime` field
|
210
|
+
in numeric format shouldn't be used as a range key.
|
171
211
|
|
172
|
-
You have two options if you need to use a `datetime` field as a range
|
212
|
+
You have two options if you need to use a `datetime` field as a range
|
213
|
+
key:
|
173
214
|
* string format
|
174
|
-
* store `datetime` values without milliseconds e.g. cut
|
175
|
-
manually with `change` method - `Time.now.change(usec: 0)`
|
215
|
+
* store `datetime` values without milliseconds (e.g. cut
|
216
|
+
them manually with `change` method - `Time.now.change(usec: 0)`
|
176
217
|
|
177
218
|
#### Note on set type
|
178
219
|
|
179
220
|
`Dynamoid`'s type `set` is stored as DynamoDB's Set attribute type.
|
180
|
-
DynamoDB supports only Set of strings, numbers and binary.
|
181
|
-
|
221
|
+
DynamoDB supports only Set of strings, numbers and binary. Moreover Set
|
222
|
+
*must* contain elements of the same type only.
|
182
223
|
|
183
|
-
In order to use some other `Dynamoid`'s types you can specify `of`
|
184
|
-
to declare the type of set elements.
|
224
|
+
In order to use some other `Dynamoid`'s types you can specify `of`
|
225
|
+
option to declare the type of set elements.
|
185
226
|
|
186
227
|
As a result of that DynamoDB limitation, in Dynamoid only the following
|
187
228
|
scalar types are supported (note: does not support `boolean`):
|
188
|
-
`integer`, `number`, `date`, `datetime`, `serializable` and custom
|
229
|
+
`integer`, `number`, `date`, `datetime`, `serializable` and custom
|
230
|
+
types.
|
189
231
|
|
190
232
|
```ruby
|
191
233
|
class Document
|
@@ -195,8 +237,9 @@ class Document
|
|
195
237
|
end
|
196
238
|
```
|
197
239
|
|
198
|
-
It's possible to specify field options like `store_as_string` for
|
199
|
-
or `serializer` for `serializable` field for `set`
|
240
|
+
It's possible to specify field options like `store_as_string` for
|
241
|
+
`datetime` field or `serializer` for `serializable` field for `set`
|
242
|
+
elements type:
|
200
243
|
|
201
244
|
```ruby
|
202
245
|
class Document
|
@@ -209,12 +252,14 @@ end
|
|
209
252
|
```
|
210
253
|
|
211
254
|
DynamoDB doesn't allow empty strings in fields configured as `set`.
|
212
|
-
Abiding by this restriction, when `Dynamoid` saves a document it removes
|
255
|
+
Abiding by this restriction, when `Dynamoid` saves a document it removes
|
256
|
+
all empty strings in set fields.
|
213
257
|
|
214
258
|
#### Note on array type
|
215
259
|
|
216
260
|
`Dynamoid`'s type `array` is stored as DynamoDB's List attribute type.
|
217
|
-
It can contain elements of different types (in contrast to Set attribute
|
261
|
+
It can contain elements of different types (in contrast to Set attribute
|
262
|
+
type).
|
218
263
|
|
219
264
|
If you need to store in array field elements of `datetime`, `date`,
|
220
265
|
`serializable` or some custom type, which DynamoDB doesn't support
|
@@ -230,7 +275,8 @@ end
|
|
230
275
|
|
231
276
|
#### Magic Columns
|
232
277
|
|
233
|
-
You get magic columns of `id` (`string`), `created_at` (`datetime`), and
|
278
|
+
You get magic columns of `id` (`string`), `created_at` (`datetime`), and
|
279
|
+
`updated_at` (`datetime`) for free.
|
234
280
|
|
235
281
|
```ruby
|
236
282
|
class User
|
@@ -248,11 +294,12 @@ end
|
|
248
294
|
|
249
295
|
#### Default Values
|
250
296
|
|
251
|
-
You can optionally set a default value on a field using either a plain
|
297
|
+
You can optionally set a default value on a field using either a plain
|
298
|
+
value or a lambda:
|
252
299
|
|
253
300
|
```ruby
|
254
|
-
|
255
|
-
|
301
|
+
field :actions_taken, :integer, default: 0
|
302
|
+
field :joined_at, :datetime, default: -> { Time.now }
|
256
303
|
```
|
257
304
|
|
258
305
|
#### Custom Types
|
@@ -260,64 +307,68 @@ You can optionally set a default value on a field using either a plain value or
|
|
260
307
|
To use a custom type for a field, suppose you have a `Money` type.
|
261
308
|
|
262
309
|
```ruby
|
263
|
-
|
264
|
-
|
310
|
+
class Money
|
311
|
+
# ... your business logic ...
|
265
312
|
|
266
|
-
|
267
|
-
|
268
|
-
|
313
|
+
def dynamoid_dump
|
314
|
+
'serialized representation as a string'
|
315
|
+
end
|
269
316
|
|
270
|
-
|
271
|
-
|
272
|
-
|
273
|
-
end
|
317
|
+
def self.dynamoid_load(serialized_str)
|
318
|
+
# parse serialized representation and return a Money instance
|
319
|
+
Money.new(1.23)
|
274
320
|
end
|
321
|
+
end
|
275
322
|
|
276
|
-
|
277
|
-
|
323
|
+
class User
|
324
|
+
include Dynamoid::Document
|
278
325
|
|
279
|
-
|
280
|
-
|
326
|
+
field :balance, Money
|
327
|
+
end
|
281
328
|
```
|
282
329
|
|
283
|
-
If you want to use a third-party class (which does not support
|
284
|
-
as your field type, you can use
|
285
|
-
|
286
|
-
|
330
|
+
If you want to use a third-party class (which does not support
|
331
|
+
`#dynamoid_dump` and `.dynamoid_load`) as your field type, you can use
|
332
|
+
an adapter class providing `.dynamoid_dump` and `.dynamoid_load` class
|
333
|
+
methods for your third-party class. `.dynamoid_load` can remain the same
|
334
|
+
from the previous example; here we just add a level of indirection for
|
335
|
+
serializing. Example:
|
287
336
|
|
288
337
|
```ruby
|
289
|
-
|
290
|
-
|
338
|
+
# Third-party Money class
|
339
|
+
class Money; end
|
291
340
|
|
292
|
-
|
293
|
-
|
294
|
-
|
295
|
-
|
341
|
+
class MoneyAdapter
|
342
|
+
def self.dynamoid_load(money_serialized_str)
|
343
|
+
Money.new(1.23)
|
344
|
+
end
|
296
345
|
|
297
|
-
|
298
|
-
|
299
|
-
end
|
346
|
+
def self.dynamoid_dump(money_obj)
|
347
|
+
money_obj.value.to_s
|
300
348
|
end
|
349
|
+
end
|
301
350
|
|
302
|
-
|
303
|
-
|
351
|
+
class User
|
352
|
+
include Dynamoid::Document
|
304
353
|
|
305
|
-
|
306
|
-
|
354
|
+
field :balance, MoneyAdapter
|
355
|
+
end
|
307
356
|
```
|
308
357
|
|
309
|
-
Lastly, you can control the data type of your custom-class-backed field
|
310
|
-
This is especially important if you want to use
|
311
|
-
|
312
|
-
|
313
|
-
|
358
|
+
Lastly, you can control the data type of your custom-class-backed field
|
359
|
+
at the DynamoDB level. This is especially important if you want to use
|
360
|
+
your custom field as a numeric range or for number-oriented queries. By
|
361
|
+
default custom fields are persisted as a string attribute, but your
|
362
|
+
custom class can override this with a `.dynamoid_field_type` class
|
363
|
+
method, which would return either `:string` or `:number`.
|
314
364
|
|
315
|
-
DynamoDB may support some other attribute types that are not yet
|
365
|
+
DynamoDB may support some other attribute types that are not yet
|
366
|
+
supported by Dynamoid.
|
316
367
|
|
317
368
|
### Sort key
|
318
369
|
|
319
|
-
Along with partition key table may have a sort key. In order to declare
|
320
|
-
`range` class method should be used:
|
370
|
+
Along with partition key table may have a sort key. In order to declare
|
371
|
+
it in a model `range` class method should be used:
|
321
372
|
|
322
373
|
```ruby
|
323
374
|
class Post
|
@@ -331,9 +382,15 @@ Second argument, type, is optional. Default type is `string`.
|
|
331
382
|
|
332
383
|
### Associations
|
333
384
|
|
334
|
-
Just like in ActiveRecord (or your other favorite ORM), Dynamoid uses
|
385
|
+
Just like in ActiveRecord (or your other favorite ORM), Dynamoid uses
|
386
|
+
associations to create links between models.
|
335
387
|
|
336
|
-
The only supported associations (so far) are `has_many`, `has_one`,
|
388
|
+
The only supported associations (so far) are `has_many`, `has_one`,
|
389
|
+
`has_and_belongs_to_many`, and `belongs_to`. Associations are very
|
390
|
+
simple to create: just specify the type, the name, and then any options
|
391
|
+
you'd like to pass to the association. If there's an inverse association
|
392
|
+
either inferred or specified directly, Dynamoid will update both objects
|
393
|
+
to point at each other.
|
337
394
|
|
338
395
|
```ruby
|
339
396
|
class User
|
@@ -348,7 +405,6 @@ class User
|
|
348
405
|
belongs_to :group, foreign_key: :group_id
|
349
406
|
has_one :role
|
350
407
|
has_and_belongs_to_many :friends, inverse_of: :friending_users
|
351
|
-
|
352
408
|
end
|
353
409
|
|
354
410
|
class Address
|
@@ -357,11 +413,18 @@ class Address
|
|
357
413
|
# ...
|
358
414
|
|
359
415
|
belongs_to :user # Automatically links up with the user model
|
360
|
-
|
361
416
|
end
|
362
417
|
```
|
363
418
|
|
364
|
-
Contrary to what you'd expect, association information is always
|
419
|
+
Contrary to what you'd expect, association information is always
|
420
|
+
contained on the object specifying the association, even if it seems
|
421
|
+
like the association has a foreign key. This is a side effect of
|
422
|
+
DynamoDB's structure: it's very difficult to find foreign keys without
|
423
|
+
an index. Usually you won't find this to be a problem, but it does mean
|
424
|
+
that association methods that build new models will not work correctly -
|
425
|
+
for example, `user.addresses.new` returns an address that is not
|
426
|
+
associated to the user. We'll be correcting this ~soon~ maybe someday,
|
427
|
+
if we get a pull request.
|
365
428
|
|
366
429
|
### Validations
|
367
430
|
|
@@ -378,9 +441,12 @@ class User
|
|
378
441
|
end
|
379
442
|
```
|
380
443
|
|
381
|
-
To see more usage and examples of ActiveModel validations, check out the
|
444
|
+
To see more usage and examples of ActiveModel validations, check out the
|
445
|
+
[ActiveModel validation
|
446
|
+
documentation](http://api.rubyonrails.org/classes/ActiveModel/Validations.html).
|
382
447
|
|
383
|
-
If you want to bypass model validation, pass `validate: false` to `save`
|
448
|
+
If you want to bypass model validation, pass `validate: false` to `save`
|
449
|
+
call:
|
384
450
|
|
385
451
|
```ruby
|
386
452
|
model.save(validate: false)
|
@@ -388,7 +454,9 @@ model.save(validate: false)
|
|
388
454
|
|
389
455
|
### Callbacks
|
390
456
|
|
391
|
-
Dynamoid also employs ActiveModel callbacks. Right now, callbacks are
|
457
|
+
Dynamoid also employs ActiveModel callbacks. Right now, callbacks are
|
458
|
+
defined on `save`, `update`, `destroy`, which allows you to do `before_`
|
459
|
+
or `after_` any of those.
|
392
460
|
|
393
461
|
```ruby
|
394
462
|
class User
|
@@ -404,7 +472,8 @@ end
|
|
404
472
|
|
405
473
|
### STI
|
406
474
|
|
407
|
-
Dynamoid supports STI (Single Table Inheritance) like Active Record
|
475
|
+
Dynamoid supports STI (Single Table Inheritance) like Active Record
|
476
|
+
does. You need just specify `type` field in a base class. Example:
|
408
477
|
|
409
478
|
```ruby
|
410
479
|
class Animal
|
@@ -421,12 +490,13 @@ end
|
|
421
490
|
cat = Cat.create(name: 'Morgan')
|
422
491
|
animal = Animal.find(cat.id)
|
423
492
|
animal.class
|
424
|
-
#=>
|
493
|
+
#=> Cat
|
425
494
|
```
|
426
495
|
|
427
|
-
If you already have DynamoDB tables and `type` field already exists and
|
428
|
-
|
429
|
-
instead of `type` one
|
496
|
+
If you already have DynamoDB tables and `type` field already exists and
|
497
|
+
has its own semantic it leads to conflict. It's possible to tell
|
498
|
+
Dynamoid to use another field (even not existing) instead of `type` one
|
499
|
+
with `inheritance_field` table option:
|
430
500
|
|
431
501
|
```ruby
|
432
502
|
class Car
|
@@ -443,8 +513,9 @@ c.my_new_type
|
|
443
513
|
|
444
514
|
### Type casting
|
445
515
|
|
446
|
-
Dynamid supports type casting and tryes to do it in the most convinient
|
447
|
-
Values for all fields (except custom type) are coerced to declared
|
516
|
+
Dynamid supports type casting and tryes to do it in the most convinient
|
517
|
+
way. Values for all fields (except custom type) are coerced to declared
|
518
|
+
field types.
|
448
519
|
|
449
520
|
Some obvious rules are used, e.g.:
|
450
521
|
|
@@ -468,18 +539,29 @@ document.integer_field = true
|
|
468
539
|
# => 1
|
469
540
|
```
|
470
541
|
|
471
|
-
If time zone isn't specified for `datetime` value - application time
|
542
|
+
If time zone isn't specified for `datetime` value - application time
|
543
|
+
zone is used.
|
472
544
|
|
473
545
|
To access field value before type casting following method could be
|
474
|
-
used: `attributes_before_type_cast` and
|
546
|
+
used: `attributes_before_type_cast` and
|
547
|
+
`read_attribute_before_type_cast`.
|
548
|
+
|
549
|
+
There is `<name>_before_type_cast` method for every field in a model as
|
550
|
+
well.
|
475
551
|
|
476
|
-
|
552
|
+
### Dirty API
|
553
|
+
|
554
|
+
Dynamoid supports Dirty API which equvalents to [Rails 5.2
|
555
|
+
`ActiveModel::Dirty`](https://api.rubyonrails.org/v5.2/classes/ActiveModel/Dirty.html).
|
556
|
+
There is only one limitation - change in place of field isn't detected
|
557
|
+
automatically.
|
477
558
|
|
478
559
|
## Usage
|
479
560
|
|
480
561
|
### Object Creation
|
481
562
|
|
482
|
-
Dynamoid's syntax is generally very similar to ActiveRecord's. Making
|
563
|
+
Dynamoid's syntax is generally very similar to ActiveRecord's. Making
|
564
|
+
new objects is simple:
|
483
565
|
|
484
566
|
```ruby
|
485
567
|
u = User.new(name: 'Josh')
|
@@ -487,13 +569,15 @@ u.email = 'josh@joshsymonds.com'
|
|
487
569
|
u.save
|
488
570
|
```
|
489
571
|
|
490
|
-
Save forces persistence to the datastore: a unique ID is also assigned,
|
572
|
+
Save forces persistence to the datastore: a unique ID is also assigned,
|
573
|
+
but it is a string and not an auto-incrementing number.
|
491
574
|
|
492
575
|
```ruby
|
493
576
|
u.id # => '3a9f7216-4726-4aea-9fbc-8554ae9292cb'
|
494
577
|
```
|
495
578
|
|
496
|
-
To use associations, you use association methods very similar to
|
579
|
+
To use associations, you use association methods very similar to
|
580
|
+
ActiveRecord's:
|
497
581
|
|
498
582
|
```ruby
|
499
583
|
address = u.addresses.create
|
@@ -520,7 +604,8 @@ Querying can be done in one of three ways:
|
|
520
604
|
|
521
605
|
```ruby
|
522
606
|
Address.find(address.id) # Find directly by ID.
|
523
|
-
Address.where(city: 'Chicago').all # Find by any number of matching criteria...
|
607
|
+
Address.where(city: 'Chicago').all # Find by any number of matching criteria...
|
608
|
+
# Though presently only "where" is supported.
|
524
609
|
Address.find_by_city('Chicago') # The same as above, but using ActiveRecord's older syntax.
|
525
610
|
```
|
526
611
|
|
@@ -530,7 +615,12 @@ And you can also query on associations:
|
|
530
615
|
u.addresses.where(city: 'Chicago').all
|
531
616
|
```
|
532
617
|
|
533
|
-
But keep in mind Dynamoid
|
618
|
+
But keep in mind Dynamoid - and document-based storage systems in
|
619
|
+
general - are not drop-in replacements for existing relational
|
620
|
+
databases. The above query does not efficiently perform a conditional
|
621
|
+
join, but instead finds all the user's addresses and naively filters
|
622
|
+
them in Ruby. For large associations this is a performance hit compared
|
623
|
+
to relational database engines.
|
534
624
|
|
535
625
|
**WARNING:** There is a limitation of conditions passed to `where`
|
536
626
|
method. Only one condition for some particular field could be specified.
|
@@ -544,34 +634,69 @@ User.where(name: 'Mike').where('name.begins_with': 'Ed')
|
|
544
634
|
|
545
635
|
the first one will be ignored and the last one will be used.
|
546
636
|
|
637
|
+
**Warning:** There is a caveat with filtering documents by `nil` value
|
638
|
+
attribute. By default Dynamoid ignores attributes with `nil` value and
|
639
|
+
doesn't store them in a DynamoDB document. This behavior could be
|
640
|
+
changed with `store_attribute_with_nil_value` config option.
|
641
|
+
|
642
|
+
If Dynamoid ignores `nil` value attributes `null`/`not_null` operators
|
643
|
+
should be used in query:
|
644
|
+
|
645
|
+
```ruby
|
646
|
+
Address.where('postcode.null': true)
|
647
|
+
Address.where('postcode.not_null': true)
|
648
|
+
```
|
649
|
+
|
650
|
+
If Dynamoid keeps `nil` value attributes `eq`/`ne` operators should be
|
651
|
+
used instead:
|
652
|
+
|
653
|
+
```ruby
|
654
|
+
Address.where('postcode': nil)
|
655
|
+
Address.where('postcode.ne': nil)
|
656
|
+
```
|
657
|
+
|
547
658
|
#### Limits
|
548
659
|
|
549
660
|
There are three types of limits that you can query with:
|
550
661
|
|
551
|
-
1. `record_limit` - The number of evaluated records that are returned by
|
552
|
-
|
553
|
-
|
662
|
+
1. `record_limit` - The number of evaluated records that are returned by
|
663
|
+
the query.
|
664
|
+
2. `scan_limit` - The number of scanned records that DynamoDB will look
|
665
|
+
at before returning.
|
666
|
+
3. `batch_size` - The number of records requested to DynamoDB per
|
667
|
+
underlying request, good for large queries!
|
554
668
|
|
555
|
-
Using these in various combinations results in the underlying requests
|
556
|
-
|
669
|
+
Using these in various combinations results in the underlying requests
|
670
|
+
to be made in the smallest size possible and the query returns once
|
671
|
+
`record_limit` or `scan_limit` is satisfied. It will attempt to batch
|
672
|
+
whenever possible.
|
557
673
|
|
558
|
-
You can thus limit the number of evaluated records, or select a record
|
674
|
+
You can thus limit the number of evaluated records, or select a record
|
675
|
+
from which to start in order to support pagination.
|
559
676
|
|
560
677
|
```ruby
|
561
678
|
Address.record_limit(5).start(address) # Only 5 addresses starting at `address`
|
562
679
|
```
|
563
|
-
Where `address` is an instance of the model or a hash
|
564
|
-
|
565
|
-
|
566
|
-
|
567
|
-
|
568
|
-
|
680
|
+
Where `address` is an instance of the model or a hash
|
681
|
+
`{the_model_hash_key: 'value', the_model_range_key: 'value'}`. Keep in
|
682
|
+
mind that if you are passing a hash to `.start()` you need to explicitly
|
683
|
+
define all required keys in it including range keys, depending on table
|
684
|
+
or secondary indexes signatures, otherwise you'll get an
|
685
|
+
`Aws::DynamoDB::Errors::ValidationException` either for `Exclusive Start
|
686
|
+
Key must have same size as table's key schema` or `The provided starting
|
687
|
+
key is invalid`
|
688
|
+
|
689
|
+
If you are potentially running over a large data set and this is
|
690
|
+
especially true when using certain filters, you may want to consider
|
691
|
+
limiting the number of scanned records (the number of records DynamoDB
|
692
|
+
infrastructure looks through when evaluating data to return):
|
569
693
|
|
570
694
|
```ruby
|
571
695
|
Address.scan_limit(5).start(address) # Only scan at most 5 records and return what's found starting from `address`
|
572
696
|
```
|
573
697
|
|
574
|
-
For large queries that return many rows, Dynamoid can use AWS' support
|
698
|
+
For large queries that return many rows, Dynamoid can use AWS' support
|
699
|
+
for requesting documents in batches:
|
575
700
|
|
576
701
|
```ruby
|
577
702
|
# Do some maintenance on the entire table without flooding DynamoDB
|
@@ -579,29 +704,32 @@ Address.all(batch_size: 100).each { |address| address.do_some_work; sleep(0.01)
|
|
579
704
|
Address.record_limit(10_000).batch(100).each { … } # Batch specified as part of a chain
|
580
705
|
```
|
581
706
|
|
582
|
-
The implication of batches is that the underlying requests are done in
|
583
|
-
|
707
|
+
The implication of batches is that the underlying requests are done in
|
708
|
+
the batch sizes to make the request and responses more manageable. Note
|
709
|
+
that this batching is for `Query` and `Scans` and not `BatchGetItem`
|
710
|
+
commands.
|
584
711
|
|
585
712
|
#### DynamoDB pagination
|
586
713
|
|
587
|
-
At times it can be useful to rely on DynamoDB [low-level
|
588
|
-
|
589
|
-
|
714
|
+
At times it can be useful to rely on DynamoDB [low-level
|
715
|
+
pagination](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.html#Query.Pagination)
|
716
|
+
instead of fixed pages sizes. Each page results in a single Query or
|
717
|
+
Scan call to DyanmoDB, but returns an unknown number of records.
|
590
718
|
|
591
|
-
Access to the native DynamoDB pages can be obtained via the
|
592
|
-
method, which yields arrays of records.
|
719
|
+
Access to the native DynamoDB pages can be obtained via the
|
720
|
+
`find_by_pages` method, which yields arrays of records.
|
593
721
|
|
594
722
|
```ruby
|
595
723
|
Address.find_by_pages do |addresses, metadata|
|
596
724
|
end
|
597
725
|
```
|
598
726
|
|
599
|
-
Each yielded pages returns page metadata as the second argument, which
|
600
|
-
including a key `:last_evaluated_key`. The value of this key
|
601
|
-
the `start` method to fetch the next page of records.
|
727
|
+
Each yielded pages returns page metadata as the second argument, which
|
728
|
+
is a hash including a key `:last_evaluated_key`. The value of this key
|
729
|
+
can be used for the `start` method to fetch the next page of records.
|
602
730
|
|
603
|
-
This way it can be used for instance to implement efficiently
|
604
|
-
|
731
|
+
This way it can be used for instance to implement efficiently pagination
|
732
|
+
in web-application:
|
605
733
|
|
606
734
|
```ruby
|
607
735
|
class UserController < ApplicationController
|
@@ -620,8 +748,9 @@ end
|
|
620
748
|
|
621
749
|
#### Sort Conditions and Filters
|
622
750
|
|
623
|
-
You are able to optimize query with condition for sort key. Following
|
624
|
-
`
|
751
|
+
You are able to optimize query with condition for sort key. Following
|
752
|
+
operators are available: `gt`, `lt`, `gte`, `lte`, `begins_with`,
|
753
|
+
`between` as well as equality:
|
625
754
|
|
626
755
|
```ruby
|
627
756
|
Address.where(latitude: 10212)
|
@@ -633,18 +762,57 @@ Address.where('city.begins_with': 'Lon')
|
|
633
762
|
Address.where('latitude.between': [10212, 20000])
|
634
763
|
```
|
635
764
|
|
636
|
-
You are able to filter results on the DynamoDB side and specify
|
637
|
-
|
765
|
+
You are able to filter results on the DynamoDB side and specify
|
766
|
+
conditions for non-key fields. Following additional operators are
|
767
|
+
available: `in`, `contains`, `not_contains`, `null`, `not_null`:
|
638
768
|
|
639
769
|
```ruby
|
640
770
|
Address.where('city.in': ['London', 'Edenburg', 'Birmingham'])
|
641
771
|
Address.where('city.contains': ['on'])
|
642
772
|
Address.where('city.not_contains': ['ing'])
|
773
|
+
Address.where('postcode.null': false)
|
774
|
+
Address.where('postcode.not_null': true)
|
775
|
+
```
|
776
|
+
|
777
|
+
**WARNING:** Please take into accout that `NULL` and `NOT_NULL`
|
778
|
+
operators check attribute presence in a document, not value. So if
|
779
|
+
attribute `postcode`'s value is `NULL`, `NULL` operator will return
|
780
|
+
false because attribute exists even if has `NULL` value.
|
781
|
+
|
782
|
+
#### Selecting some specific fields only
|
783
|
+
|
784
|
+
It could be done with `project` method:
|
785
|
+
|
786
|
+
```ruby
|
787
|
+
class User
|
788
|
+
include Dynamoid::Document
|
789
|
+
field :name
|
790
|
+
end
|
791
|
+
|
792
|
+
User.create(name: 'Alex')
|
793
|
+
user = User.project(:name).first
|
794
|
+
|
795
|
+
user.id # => nil
|
796
|
+
user.name # => 'Alex'
|
797
|
+
user.created_at # => nil
|
798
|
+
```
|
799
|
+
|
800
|
+
Returned models with have filled specified fields only.
|
801
|
+
|
802
|
+
Several fields could be specified:
|
803
|
+
|
804
|
+
```ruby
|
805
|
+
user = User.project(:name, :created_at)
|
643
806
|
```
|
644
807
|
|
645
808
|
### Consistent Reads
|
646
809
|
|
647
|
-
Querying supports consistent reading. By default, DynamoDB reads are
|
810
|
+
Querying supports consistent reading. By default, DynamoDB reads are
|
811
|
+
eventually consistent: if you do a write and then a read immediately
|
812
|
+
afterwards, the results of the previous write may not be reflected. If
|
813
|
+
you need to do a consistent read (that is, you need to read the results
|
814
|
+
of a write immediately) you can do so, but keep in mind that consistent
|
815
|
+
reads are twice as expensive as regular reads for DynamoDB.
|
648
816
|
|
649
817
|
```ruby
|
650
818
|
Address.find(address.id, consistent_read: true) # Find an address, ensure the read is consistent.
|
@@ -653,21 +821,25 @@ Address.where(city: 'Chicago').consistent.all # Find all addresses where the
|
|
653
821
|
|
654
822
|
### Range Finding
|
655
823
|
|
656
|
-
If you have a range index, Dynamoid provides a number of additional
|
824
|
+
If you have a range index, Dynamoid provides a number of additional
|
825
|
+
other convenience methods to make your life a little easier:
|
657
826
|
|
658
827
|
```ruby
|
659
828
|
User.where("created_at.gt": DateTime.now - 1.day).all
|
660
829
|
User.where("created_at.lt": DateTime.now - 1.day).all
|
661
830
|
```
|
662
831
|
|
663
|
-
It also supports `gte` and `lte`. Turning those into symbols and
|
832
|
+
It also supports `gte` and `lte`. Turning those into symbols and
|
833
|
+
allowing a Rails SQL-style string syntax is in the works. You can only
|
834
|
+
have one range argument per query, because of DynamoDB's inherent
|
835
|
+
limitations, so use it sensibly!
|
664
836
|
|
665
837
|
|
666
838
|
### Updating
|
667
839
|
|
668
840
|
In order to update document you can use high level methods
|
669
|
-
`#update_attributes`, `#update_attribute` and `.update`.
|
670
|
-
|
841
|
+
`#update_attributes`, `#update_attribute` and `.update`. They run
|
842
|
+
validation and collbacks.
|
671
843
|
|
672
844
|
```ruby
|
673
845
|
Address.find(id).update_attributes(city: 'Chicago')
|
@@ -677,8 +849,8 @@ Address.update(id, { city: 'Chicago' }, if: { deliverable: true })
|
|
677
849
|
```
|
678
850
|
|
679
851
|
There are also some low level methods `#update`, `.update_fields` and
|
680
|
-
`.upsert`. They don't run validation and callbacks (except `#update` -
|
681
|
-
runs `update` callbacks). All of them support conditional updates.
|
852
|
+
`.upsert`. They don't run validation and callbacks (except `#update` -
|
853
|
+
it runs `update` callbacks). All of them support conditional updates.
|
682
854
|
`#upsert` will create new document if document with specified `id`
|
683
855
|
doesn't exist.
|
684
856
|
|
@@ -699,8 +871,8 @@ Address.upsert(id, { city: 'Chicago' }, if: { deliverable: true })
|
|
699
871
|
|
700
872
|
### Deleting
|
701
873
|
|
702
|
-
In order to delete some items `delete_all` method should be used.
|
703
|
-
|
874
|
+
In order to delete some items `delete_all` method should be used. Any
|
875
|
+
callback wont be called. Items delete in efficient way in batch.
|
704
876
|
|
705
877
|
```ruby
|
706
878
|
Address.where(city: 'London').delete_all
|
@@ -724,14 +896,21 @@ end
|
|
724
896
|
There are following options:
|
725
897
|
* `hash_key` - is used as hash key of an index,
|
726
898
|
* `range_key` - is used as range key of an index,
|
727
|
-
* `projected_attributes` - list of fields to store in an index or has a
|
728
|
-
|
729
|
-
* `
|
730
|
-
|
899
|
+
* `projected_attributes` - list of fields to store in an index or has a
|
900
|
+
predefiled value `:keys_only`, `:all`; `:keys_only` is a default,
|
901
|
+
* `name` - an index will be created with this name when a table is
|
902
|
+
created; by default name is generated and contains table name and keys
|
903
|
+
names,
|
904
|
+
* `read_capacity` - is used when table creates and used as an index
|
905
|
+
capacity; by default equals `Dynamoid::Config.read_capacity`,
|
906
|
+
* `write_capacity` - is used when table creates and used as an index
|
907
|
+
capacity; by default equals `Dynamoid::Config.write_capacity`
|
731
908
|
|
732
909
|
The only mandatory option is `name`.
|
733
910
|
|
734
|
-
**WARNING:** In order to use global secondary index in `Document.where`
|
911
|
+
**WARNING:** In order to use global secondary index in `Document.where`
|
912
|
+
implicitly you need to have all the attributes of the original table in
|
913
|
+
the index and declare it with option `projected_attributes: :all`:
|
735
914
|
|
736
915
|
```ruby
|
737
916
|
class User
|
@@ -741,13 +920,16 @@ class User
|
|
741
920
|
end
|
742
921
|
```
|
743
922
|
|
744
|
-
There is only one implicit way to query Global and Local Secondary
|
923
|
+
There is only one implicit way to query Global and Local Secondary
|
924
|
+
Indexes (GSI/LSI).
|
745
925
|
|
746
926
|
#### Implicit
|
747
927
|
|
748
|
-
The second way implicitly uses your GSI through the `where` clauses and
|
749
|
-
|
750
|
-
|
928
|
+
The second way implicitly uses your GSI through the `where` clauses and
|
929
|
+
deduces the index based on the query fields provided. Another added
|
930
|
+
benefit is that it is built into query chaining so you can use all the
|
931
|
+
methods used in normal querying. The explicit way from above would be
|
932
|
+
rewritten as follows:
|
751
933
|
|
752
934
|
```ruby
|
753
935
|
where(dynamo_primary_key_column_name => dynamo_primary_key_value,
|
@@ -755,57 +937,107 @@ where(dynamo_primary_key_column_name => dynamo_primary_key_value,
|
|
755
937
|
.scan_index_forward(false)
|
756
938
|
```
|
757
939
|
|
758
|
-
The only caveat with this method is that because it is also used for
|
759
|
-
|
760
|
-
|
761
|
-
|
762
|
-
|
940
|
+
The only caveat with this method is that because it is also used for
|
941
|
+
general querying, it WILL NOT use a GSI unless it explicitly has defined
|
942
|
+
`projected_attributes: :all` on the GSI in your model. This is because
|
943
|
+
GSIs that do not have all attributes projected will only contain the
|
944
|
+
index keys and therefore will not return objects with fully resolved
|
945
|
+
field values. It currently opts to provide the complete results rather
|
946
|
+
than partial results unless you've explicitly looked up the data.
|
763
947
|
|
764
|
-
*Future TODO could involve implementing `select` in chaining as well as
|
765
|
-
the
|
948
|
+
*Future TODO could involve implementing `select` in chaining as well as
|
949
|
+
resolving the fields with a second query against the table since a query
|
950
|
+
against GSI then a query on base table is still likely faster than scan
|
951
|
+
on the base table*
|
766
952
|
|
767
953
|
## Configuration
|
768
954
|
|
769
955
|
Listed below are all configuration options.
|
770
956
|
|
771
|
-
* `adapter` - usefull only for the gem developers to switch to a new
|
772
|
-
|
773
|
-
* `
|
774
|
-
|
775
|
-
|
776
|
-
* `
|
777
|
-
|
778
|
-
|
779
|
-
* `
|
957
|
+
* `adapter` - usefull only for the gem developers to switch to a new
|
958
|
+
adapter. Default and the only available value is `aws_sdk_v3`
|
959
|
+
* `namespace` - prefix for table names, default is
|
960
|
+
`dynamoid_#{application_name}_#{environment}` for Rails application
|
961
|
+
and `dynamoid` otherwise
|
962
|
+
* `logger` - by default it's a `Rails.logger` in Rails application and
|
963
|
+
`stdout` otherwise. You can disable logging by setting `nil` or
|
964
|
+
`false` values. Set `true` value to use defaults
|
965
|
+
* `access_key` - DynamoDb custom credentials for AWS, override global
|
966
|
+
AWS credentials if they present
|
967
|
+
* `secret_key` - DynamoDb custom credentials for AWS, override global
|
968
|
+
AWS credentials if they present
|
969
|
+
* `region` - DynamoDb custom credentials for AWS, override global AWS
|
970
|
+
credentials if they present
|
971
|
+
* `batch_size` - when you try to load multiple items at once with
|
972
|
+
* `batch_get_item` call Dynamoid loads them not with one api call but
|
973
|
+
piece by piece. Default is 100 items
|
974
|
+
* `read_capacity` - is used at table or indices creation. Default is 100
|
975
|
+
(units)
|
976
|
+
* `write_capacity` - is used at table or indices creation. Default is 20
|
977
|
+
(units)
|
780
978
|
* `warn_on_scan` - log warnings when scan table. Default is `true`
|
781
|
-
* `endpoint` - if provided, it communicates with the DynamoDB listening
|
782
|
-
|
979
|
+
* `endpoint` - if provided, it communicates with the DynamoDB listening
|
980
|
+
at the endpoint. This is useful for testing with
|
981
|
+
[DynamoDB Local](http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Tools.DynamoDBLocal.html)
|
982
|
+
* `identity_map` - ensures that each object gets loaded only once by
|
983
|
+
keeping every loaded object in a map. Looks up objects using the map
|
984
|
+
when referring to them. Isn't thread safe. Default is `false`.
|
783
985
|
`Use Dynamoid::Middleware::IdentityMap` to clear identity map for each HTTP request
|
784
|
-
* `timestamps` - by default Dynamoid sets `created_at` and `updated_at`
|
785
|
-
|
786
|
-
|
787
|
-
* `
|
788
|
-
|
789
|
-
|
790
|
-
|
791
|
-
|
792
|
-
|
793
|
-
|
794
|
-
* `
|
795
|
-
|
796
|
-
|
797
|
-
|
798
|
-
* `
|
799
|
-
|
800
|
-
* `
|
801
|
-
*
|
802
|
-
|
803
|
-
|
986
|
+
* `timestamps` - by default Dynamoid sets `created_at` and `updated_at`
|
987
|
+
fields for model at creation and updating. You can disable this
|
988
|
+
behavior by setting `false` value
|
989
|
+
* `sync_retry_max_times` - when Dynamoid creates or deletes table
|
990
|
+
synchronously it checks for completion specified times. Default is 60
|
991
|
+
(times). It's a bit over 2 minutes by default
|
992
|
+
* `sync_retry_wait_seconds` - time to wait between retries. Default is 2
|
993
|
+
(seconds)
|
994
|
+
* `convert_big_decimal` - if `true` then Dynamoid converts numbers
|
995
|
+
stored in `Hash` in `raw` field to float. Default is `false`
|
996
|
+
* `store_attribute_with_nil_value` - if `true` Dynamoid keeps attribute
|
997
|
+
with `nil` value in a document. Otherwise Dynamoid removes it while
|
998
|
+
saving a document. Default is `nil` which equals behaviour with `false`
|
999
|
+
value.
|
1000
|
+
* `models_dir` - `dynamoid:create_tables` rake task loads DynamoDb
|
1001
|
+
models from this directory. Default is `./app/models`.
|
1002
|
+
* `application_timezone` - Dynamoid converts all `datetime` fields to
|
1003
|
+
* specified time zone when loads data from the storage.
|
1004
|
+
Acceptable values - `:utc`, `:local` (to use system time zone) and
|
1005
|
+
time zone name e.g. `Eastern Time (US & Canada)`. Default is `utc`
|
1006
|
+
* `dynamodb_timezone` - When a datetime field is stored in string format
|
1007
|
+
Dynamoid converts it to specified time zone when saves a value to the
|
1008
|
+
storage. Acceptable values - `:utc`, `:local` (to use system time
|
1009
|
+
zone) and time zone name e.g. `Eastern Time (US & Canada)`. Default is
|
1010
|
+
`utc`
|
1011
|
+
* `store_datetime_as_string` - if `true` then Dynamoid stores :datetime
|
1012
|
+
fields in ISO 8601 string format. Default is `false`
|
1013
|
+
* `store_date_as_string` - if `true` then Dynamoid stores :date fields
|
1014
|
+
in ISO 8601 string format. Default is `false`
|
1015
|
+
* `store_boolean_as_native` - if `true` Dynamoid stores boolean fields
|
1016
|
+
as native DynamoDB boolean values. Otherwise boolean fields are stored
|
1017
|
+
as string values `'t'` and `'f'`. Default is true
|
1018
|
+
* `backoff` - is a hash: key is a backoff strategy (symbol), value is
|
1019
|
+
parameters for the strategy. Is used in batch operations. Default id
|
1020
|
+
`nil`
|
1021
|
+
* `backoff_strategies`: is a hash and contains all available strategies.
|
1022
|
+
Default is { constant: ..., exponential: ...}
|
1023
|
+
* `http_continue_timeout`: The number of seconds to wait for a
|
1024
|
+
100-continue HTTP response before sending the request body. Default
|
1025
|
+
option value is `nil`. If not specified effected value is `1`
|
1026
|
+
* `http_idle_timeout`: The number of seconds an HTTP connection is
|
1027
|
+
allowed to sit idble before it is considered stale. Default option
|
1028
|
+
value is `nil`. If not specified effected value is `5`
|
1029
|
+
* `http_open_timeout`: The number of seconds to wait when opening a HTTP
|
1030
|
+
session. Default option value is `nil`. If not specified effected
|
1031
|
+
value is `15`
|
1032
|
+
* `http_read_timeout`:The number of seconds to wait for HTTP response
|
1033
|
+
data. Default option value is `nil`. If not specified effected value
|
1034
|
+
is `60`
|
804
1035
|
|
805
1036
|
|
806
1037
|
## Concurrency
|
807
1038
|
|
808
|
-
Dynamoid supports basic, ActiveRecord-like optimistic locking on save
|
1039
|
+
Dynamoid supports basic, ActiveRecord-like optimistic locking on save
|
1040
|
+
operations. Simply add a `lock_version` column to your table like so:
|
809
1041
|
|
810
1042
|
```ruby
|
811
1043
|
class MyTable
|
@@ -817,23 +1049,38 @@ class MyTable
|
|
817
1049
|
end
|
818
1050
|
```
|
819
1051
|
|
820
|
-
In this example, all saves to `MyTable` will raise an
|
1052
|
+
In this example, all saves to `MyTable` will raise an
|
1053
|
+
`Dynamoid::Errors::StaleObjectError` if a concurrent process loaded,
|
1054
|
+
edited, and saved the same row. Your code should trap this exception,
|
1055
|
+
reload the row (so that it will pick up the newest values), and try the
|
1056
|
+
save again.
|
821
1057
|
|
822
|
-
Calls to `update` and `update!` also increment the `lock_version`,
|
1058
|
+
Calls to `update` and `update!` also increment the `lock_version`,
|
1059
|
+
however they do not check the existing value. This guarantees that a
|
1060
|
+
update operation will raise an exception in a concurrent save operation,
|
1061
|
+
however a save operation will never cause an update to fail. Thus,
|
1062
|
+
`update` is useful & safe only for doing atomic operations (e.g.
|
1063
|
+
increment a value, add/remove from a set, etc), but should not be used
|
1064
|
+
in a read-modify-write pattern.
|
823
1065
|
|
824
1066
|
|
825
1067
|
### Backoff strategies
|
826
1068
|
|
827
1069
|
|
828
|
-
You can use several methods that run efficiently in batch mode like
|
1070
|
+
You can use several methods that run efficiently in batch mode like
|
1071
|
+
`.find_all` and `.import`. It affects `Query` and `Scan` operations as
|
1072
|
+
well.
|
829
1073
|
|
830
|
-
The backoff strategy will be used when, for any reason, some items could
|
831
|
-
|
1074
|
+
The backoff strategy will be used when, for any reason, some items could
|
1075
|
+
not be processed as part of a batch mode command. Operations will be
|
1076
|
+
re-run to process these items.
|
832
1077
|
|
833
|
-
Exponential backoff is the recommended way to handle throughput limits
|
1078
|
+
Exponential backoff is the recommended way to handle throughput limits
|
1079
|
+
exceeding and throttling on the table.
|
834
1080
|
|
835
|
-
There are two built-in strategies - constant delay and truncated binary
|
836
|
-
By default no backoff is used but you can specify
|
1081
|
+
There are two built-in strategies - constant delay and truncated binary
|
1082
|
+
exponential backoff. By default no backoff is used but you can specify
|
1083
|
+
one of the built-in ones:
|
837
1084
|
|
838
1085
|
```ruby
|
839
1086
|
Dynamoid.configure do |config|
|
@@ -846,7 +1093,8 @@ end
|
|
846
1093
|
|
847
1094
|
```
|
848
1095
|
|
849
|
-
You can just specify strategy without any arguments to use default
|
1096
|
+
You can just specify strategy without any arguments to use default
|
1097
|
+
presets:
|
850
1098
|
|
851
1099
|
```ruby
|
852
1100
|
Dynamoid.configure do |config|
|
@@ -871,10 +1119,11 @@ end
|
|
871
1119
|
|
872
1120
|
There are a few Rake tasks available out of the box:
|
873
1121
|
|
874
|
-
|
875
|
-
|
1122
|
+
* `rake dynamoid:create_tables`
|
1123
|
+
* `rake dynamoid:ping`
|
876
1124
|
|
877
|
-
In order to use them in non-Rails application they should be required
|
1125
|
+
In order to use them in non-Rails application they should be required
|
1126
|
+
explicitly:
|
878
1127
|
|
879
1128
|
```ruby
|
880
1129
|
# Rakefile
|
@@ -883,12 +1132,14 @@ Rake::Task.define_task(:environment)
|
|
883
1132
|
require 'dynamoid/tasks'
|
884
1133
|
```
|
885
1134
|
|
886
|
-
The Rake tasks depend on `:environment` task so it should be declared
|
887
|
-
|
1135
|
+
The Rake tasks depend on `:environment` task so it should be declared as
|
1136
|
+
well.
|
888
1137
|
|
889
1138
|
## Test Environment
|
890
1139
|
|
891
|
-
In test environment you will most likely want to clean the database
|
1140
|
+
In test environment you will most likely want to clean the database
|
1141
|
+
between test runs to keep tests completely isolated. This can be
|
1142
|
+
achieved like so
|
892
1143
|
|
893
1144
|
```ruby
|
894
1145
|
module DynamoidReset
|
@@ -919,7 +1170,8 @@ RSpec.configure do |config|
|
|
919
1170
|
end
|
920
1171
|
```
|
921
1172
|
|
922
|
-
In Rails, you may also want to ensure you do not delete non-test data
|
1173
|
+
In Rails, you may also want to ensure you do not delete non-test data
|
1174
|
+
accidentally by adding the following to your test environment setup:
|
923
1175
|
|
924
1176
|
```ruby
|
925
1177
|
raise "Tests should be run in 'test' environment only" if Rails.env != 'test'
|
@@ -956,9 +1208,14 @@ timing (231.28 ms).
|
|
956
1208
|
|
957
1209
|
## Credits
|
958
1210
|
|
959
|
-
Dynamoid borrows code, structure, and even its name very liberally from
|
1211
|
+
Dynamoid borrows code, structure, and even its name very liberally from
|
1212
|
+
the truly amazing [Mongoid](https://github.com/mongoid/mongoid). Without
|
1213
|
+
Mongoid to crib from none of this would have been possible, and I hope
|
1214
|
+
they don't mind me reusing their very awesome ideas to make DynamoDB
|
1215
|
+
just as accessible to the Ruby world as MongoDB.
|
960
1216
|
|
961
|
-
Also, without contributors the project wouldn't be nearly as awesome. So
|
1217
|
+
Also, without contributors the project wouldn't be nearly as awesome. So
|
1218
|
+
many thanks to:
|
962
1219
|
|
963
1220
|
* [Logan Bowers](https://github.com/loganb)
|
964
1221
|
* [Lane LaRue](https://github.com/luxx)
|
@@ -979,9 +1236,12 @@ Also, without contributors the project wouldn't be nearly as awesome. So many th
|
|
979
1236
|
|
980
1237
|
## Running the tests
|
981
1238
|
|
982
|
-
Running the tests is fairly simple. You should have an instance of
|
1239
|
+
Running the tests is fairly simple. You should have an instance of
|
1240
|
+
DynamoDB running locally. Follow these steps to setup your test
|
1241
|
+
environment.
|
983
1242
|
|
984
|
-
* First download and unpack the latest version of DynamoDB.
|
1243
|
+
* First download and unpack the latest version of DynamoDB. We have a
|
1244
|
+
script that will do this for you if you use homebrew on a Mac.
|
985
1245
|
|
986
1246
|
```shell
|
987
1247
|
bin/setup
|
@@ -999,13 +1259,18 @@ Running the tests is fairly simple. You should have an instance of DynamoDB runn
|
|
999
1259
|
rake
|
1000
1260
|
```
|
1001
1261
|
|
1002
|
-
* When you are done, remember to stop the local test instance of
|
1262
|
+
* When you are done, remember to stop the local test instance of
|
1263
|
+
dynamodb
|
1003
1264
|
|
1004
1265
|
```shell
|
1005
1266
|
bin/stop_dynamodblocal
|
1006
1267
|
```
|
1007
1268
|
|
1008
|
-
If you want to run all the specs that travis runs, use `bundle exec
|
1269
|
+
If you want to run all the specs that travis runs, use `bundle exec
|
1270
|
+
wwtd`, but first you will need to setup all the rubies, for each of `%w(
|
1271
|
+
2.0.0-p648 2.1.10 2.2.6 2.3.3 2.4.1 jruby-9.1.8.0 )`. When you run
|
1272
|
+
`bundle exec wwtd` it will take care of starting and stopping the local
|
1273
|
+
dynamodb instance.
|
1009
1274
|
|
1010
1275
|
```shell
|
1011
1276
|
rvm use 2.0.0-p648
|