dynamoid 3.2.0 → 3.3.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.travis.yml +9 -6
- data/Appraisals +8 -14
- data/CHANGELOG.md +24 -0
- data/README.md +493 -228
- data/gemfiles/rails_4_2.gemfile +5 -7
- data/gemfiles/rails_5_0.gemfile +4 -6
- data/gemfiles/rails_5_1.gemfile +4 -6
- data/gemfiles/rails_5_2.gemfile +4 -6
- data/gemfiles/rails_6_0.gemfile +8 -0
- data/lib/dynamoid.rb +1 -0
- data/lib/dynamoid/adapter.rb +3 -10
- data/lib/dynamoid/adapter_plugin/aws_sdk_v3.rb +25 -69
- data/lib/dynamoid/adapter_plugin/aws_sdk_v3/batch_get_item.rb +105 -0
- data/lib/dynamoid/adapter_plugin/aws_sdk_v3/create_table.rb +9 -4
- data/lib/dynamoid/adapter_plugin/aws_sdk_v3/query.rb +11 -4
- data/lib/dynamoid/adapter_plugin/aws_sdk_v3/scan.rb +11 -3
- data/lib/dynamoid/adapter_plugin/aws_sdk_v3/until_past_table_status.rb +3 -2
- data/lib/dynamoid/components.rb +6 -3
- data/lib/dynamoid/config.rb +1 -0
- data/lib/dynamoid/criteria.rb +1 -1
- data/lib/dynamoid/criteria/chain.rb +33 -6
- data/lib/dynamoid/criteria/key_fields_detector.rb +101 -32
- data/lib/dynamoid/dirty.rb +186 -34
- data/lib/dynamoid/document.rb +8 -216
- data/lib/dynamoid/fields.rb +8 -0
- data/lib/dynamoid/loadable.rb +31 -0
- data/lib/dynamoid/persistence.rb +177 -85
- data/lib/dynamoid/persistence/import.rb +72 -0
- data/lib/dynamoid/persistence/save.rb +63 -0
- data/lib/dynamoid/persistence/update_fields.rb +62 -0
- data/lib/dynamoid/persistence/upsert.rb +60 -0
- data/lib/dynamoid/version.rb +1 -1
- metadata +9 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: eac484a25ff6886d838773be1b34f1fa9c7a51bf75ec46ed93984e97f273575c
|
4
|
+
data.tar.gz: 349e23120b78776d352c576bd7ffe0d2d1a2b69b0b5107d81698435bcbe2fb2e
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: cdbe5b559ceea1a834c6bfdbd956bc0dcba57172be55a4003213dafdd54e702b8ae7c1f62f6a04c11f13f3f3907429cbff1e008ccfcef2b90e54b512c301b38f
|
7
|
+
data.tar.gz: 796c4496a3cabdde4433ee18797176d674e8be5ca887978d73792bea7fdf8cc0e697ca475c5b50cc972217524a6a006751f7497c84dbbe40a0a296bad5b1a79d
|
data/.travis.yml
CHANGED
@@ -3,18 +3,21 @@ sudo: required
|
|
3
3
|
language: ruby
|
4
4
|
rvm:
|
5
5
|
- ruby-2.3.8
|
6
|
-
- ruby-2.4.
|
7
|
-
- ruby-2.5.
|
8
|
-
- ruby-2.6.
|
9
|
-
- jruby-9.
|
10
|
-
- jruby-9.2.6.0
|
11
|
-
gemfile:
|
6
|
+
- ruby-2.4.6
|
7
|
+
- ruby-2.5.5
|
8
|
+
- ruby-2.6.3
|
9
|
+
- jruby-9.2.8.0
|
12
10
|
gemfile:
|
13
11
|
- gemfiles/rails_4_2.gemfile
|
14
12
|
- gemfiles/rails_5_0.gemfile
|
15
13
|
- gemfiles/rails_5_1.gemfile
|
16
14
|
- gemfiles/rails_5_2.gemfile
|
17
15
|
|
16
|
+
matrix:
|
17
|
+
include:
|
18
|
+
- rvm: ruby-2.6.3
|
19
|
+
gemfile: gemfiles/rails_6_0.gemfile
|
20
|
+
|
18
21
|
### BUILD LIFECYCLE STEPS ###
|
19
22
|
|
20
23
|
before_install:
|
data/Appraisals
CHANGED
@@ -1,28 +1,22 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
|
-
appraise 'rails-4-0' do
|
4
|
-
gem 'rails', '~> 4.0.0'
|
5
|
-
gem 'nokogiri', '~> 1.6.8' # can be removed once we drop support for Ruby 2.0.0
|
6
|
-
end
|
7
|
-
|
8
|
-
appraise 'rails-4-1' do
|
9
|
-
gem 'rails', '~> 4.1.0'
|
10
|
-
gem 'nokogiri', '~> 1.6.8' # can be removed once we drop support for Ruby 2.0.0
|
11
|
-
end
|
12
|
-
|
13
3
|
appraise 'rails-4-2' do
|
14
|
-
gem '
|
4
|
+
gem 'activemodel', '~> 4.2.0'
|
15
5
|
gem 'nokogiri', '~> 1.6.8' # can be removed once we drop support for Ruby 2.0.0
|
16
6
|
end
|
17
7
|
|
18
8
|
appraise 'rails-5-0' do
|
19
|
-
gem '
|
9
|
+
gem 'activemodel', '~> 5.0.0'
|
20
10
|
end
|
21
11
|
|
22
12
|
appraise 'rails-5-1' do
|
23
|
-
gem '
|
13
|
+
gem 'activemodel', '~> 5.1.0'
|
24
14
|
end
|
25
15
|
|
26
16
|
appraise 'rails-5-2' do
|
27
|
-
gem '
|
17
|
+
gem 'activemodel', '~> 5.2.0'
|
18
|
+
end
|
19
|
+
|
20
|
+
appraise 'rails-6-0' do
|
21
|
+
gem 'activemodel', '6.0.0.rc1'
|
28
22
|
end
|
data/CHANGELOG.md
CHANGED
@@ -2,6 +2,8 @@
|
|
2
2
|
|
3
3
|
## Breaking
|
4
4
|
|
5
|
+
## Features
|
6
|
+
|
5
7
|
## Improvements
|
6
8
|
|
7
9
|
## Fixes
|
@@ -10,6 +12,28 @@
|
|
10
12
|
|
11
13
|
|
12
14
|
|
15
|
+
# 3.3.0
|
16
|
+
|
17
|
+
## Features
|
18
|
+
|
19
|
+
* Feature: [#374](https://github.com/Dynamoid/dynamoid/pull/374) Add `#project` query method to load only specified fields
|
20
|
+
|
21
|
+
## Improvements
|
22
|
+
|
23
|
+
* Improvement: [#359](https://github.com/Dynamoid/dynamoid/pull/359) Add support of `NULL` and `NOT_NULL` operators
|
24
|
+
* Improvement: [#360](https://github.com/Dynamoid/dynamoid/pull/360) Add `store_attribute_with_nil_value` config option
|
25
|
+
* Improvement: [#368](https://github.com/Dynamoid/dynamoid/pull/368) Support Rails 6 (RC1)
|
26
|
+
|
27
|
+
## Fixes
|
28
|
+
* Fix: [#357](https://github.com/Dynamoid/dynamoid/pull/357) Fix synchronous table creation issue
|
29
|
+
* Fix: [#362](https://github.com/Dynamoid/dynamoid/pull/362) Fix issue with selecting Global Secondary Index (@atyndall)
|
30
|
+
* Fix: [#368](https://github.com/Dynamoid/dynamoid/pull/368) Repair `#previous_changes` method from Dirty API
|
31
|
+
* Fix: [#373](https://github.com/Dynamoid/dynamoid/pull/373) Fix threadsafety of loading `Dynamoid::Adapter` (@tsub)
|
32
|
+
|
33
|
+
---
|
34
|
+
|
35
|
+
|
36
|
+
|
13
37
|
# 3.2.0
|
14
38
|
|
15
39
|
## Features
|
data/README.md
CHANGED
@@ -9,26 +9,36 @@
|
|
9
9
|
![GitHub](https://img.shields.io/github/license/Dynamoid/dynamoid.svg)
|
10
10
|
|
11
11
|
Dynamoid is an ORM for Amazon's DynamoDB for Ruby applications. It
|
12
|
-
provides similar functionality to ActiveRecord and improves on
|
13
|
-
|
12
|
+
provides similar functionality to ActiveRecord and improves on Amazon's
|
13
|
+
existing
|
14
14
|
[HashModel](http://docs.amazonwebservices.com/AWSRubySDK/latest/AWS/Record/HashModel.html)
|
15
15
|
by providing better searching tools and native association support.
|
16
16
|
|
17
|
-
DynamoDB is not like other document-based databases you might know, and
|
17
|
+
DynamoDB is not like other document-based databases you might know, and
|
18
|
+
is very different indeed from relational databases. It sacrifices
|
19
|
+
anything beyond the simplest relational queries and transactional
|
20
|
+
support to provide a fast, cost-efficient, and highly durable storage
|
21
|
+
solution. If your database requires complicated relational queries and
|
22
|
+
transaction support, then this modest Gem cannot provide them for you,
|
23
|
+
and neither can DynamoDB. In those cases you would do better to look
|
24
|
+
elsewhere for your database needs.
|
18
25
|
|
19
|
-
But if you want a fast, scalable, simple, easy-to-use database (and a
|
26
|
+
But if you want a fast, scalable, simple, easy-to-use database (and a
|
27
|
+
Gem that supports it) then look no further!
|
20
28
|
|
21
29
|
## Installation
|
22
30
|
|
23
|
-
Installing Dynamoid is pretty simple. First include the Gem in your
|
31
|
+
Installing Dynamoid is pretty simple. First include the Gem in your
|
32
|
+
Gemfile:
|
24
33
|
|
25
34
|
```ruby
|
26
35
|
gem 'dynamoid'
|
27
36
|
```
|
28
37
|
## Prerequisities
|
29
38
|
|
30
|
-
Dynamoid depends on the aws-sdk, and this is tested on the current
|
31
|
-
|
39
|
+
Dynamoid depends on the aws-sdk, and this is tested on the current
|
40
|
+
version of aws-sdk (~> 3), rails (>= 4). Hence the configuration as
|
41
|
+
needed for aws to work will be dealt with by aws setup.
|
32
42
|
|
33
43
|
### AWS SDK Version Compatibility
|
34
44
|
|
@@ -51,45 +61,52 @@ For example, to configure AWS access:
|
|
51
61
|
Create `config/initializers/aws.rb` as follows:
|
52
62
|
|
53
63
|
```ruby
|
54
|
-
|
55
|
-
|
56
|
-
|
57
|
-
|
58
|
-
})
|
59
|
-
|
64
|
+
Aws.config.update({
|
65
|
+
region: 'us-west-2',
|
66
|
+
credentials: Aws::Credentials.new('REPLACE_WITH_ACCESS_KEY_ID', 'REPLACE_WITH_SECRET_ACCESS_KEY'),
|
67
|
+
})
|
60
68
|
```
|
61
69
|
|
62
|
-
Alternatively, if you don't want Aws connection settings to be
|
70
|
+
Alternatively, if you don't want Aws connection settings to be
|
71
|
+
overwritten for you entire project, you can specify connection settings
|
72
|
+
for Dynamoid only, by setting those in the `Dynamoid.configure` clause:
|
63
73
|
|
64
74
|
```ruby
|
65
|
-
|
66
|
-
|
67
|
-
|
68
|
-
|
69
|
-
|
70
|
-
|
75
|
+
require 'dynamoid'
|
76
|
+
Dynamoid.configure do |config|
|
77
|
+
config.access_key = 'REPLACE_WITH_ACCESS_KEY_ID'
|
78
|
+
config.secret_key = 'REPLACE_WITH_SECRET_ACCESS_KEY'
|
79
|
+
config.region = 'us-west-2'
|
80
|
+
end
|
71
81
|
```
|
72
82
|
|
73
83
|
For a full list of the DDB regions, you can go
|
74
84
|
[here](http://docs.aws.amazon.com/general/latest/gr/rande.html#ddb_region).
|
75
85
|
|
76
|
-
Then you need to initialize Dynamoid config to get it going. Put code
|
86
|
+
Then you need to initialize Dynamoid config to get it going. Put code
|
87
|
+
similar to this somewhere (a Rails initializer would be a great place
|
88
|
+
for this if you're using Rails):
|
77
89
|
|
78
90
|
```ruby
|
79
|
-
|
80
|
-
|
81
|
-
|
82
|
-
|
83
|
-
|
91
|
+
require 'dynamoid'
|
92
|
+
Dynamoid.configure do |config|
|
93
|
+
# To namespace tables created by Dynamoid from other tables you might have.
|
94
|
+
# Set to nil to avoid namespacing.
|
95
|
+
config.namespace = 'dynamoid_app_development'
|
96
|
+
|
97
|
+
# [Optional]. If provided, it communicates with the DB listening at the endpoint.
|
98
|
+
# This is useful for testing with [DynamoDB Local] (http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Tools.DynamoDBLocal.html).
|
99
|
+
config.endpoint = 'http://localhost:3000'
|
100
|
+
end
|
84
101
|
```
|
85
102
|
|
86
103
|
### Ruby & Rails Compatibility
|
87
104
|
|
88
105
|
Dynamoid supports Ruby >= 2.3 and Rails >= 4.2.
|
89
106
|
|
90
|
-
Its compatibility is tested against following Ruby versions: 2.3.
|
91
|
-
2.
|
92
|
-
|
107
|
+
Its compatibility is tested against following Ruby versions: 2.3, 2.4,
|
108
|
+
2.5 and 2.6, JRuby 9.2.8.0 and against Rails versions: 4.2, 5.0, 5.1,
|
109
|
+
5.2 and 6.0.
|
93
110
|
|
94
111
|
## Setup
|
95
112
|
|
@@ -99,12 +116,15 @@ You *must* include `Dynamoid::Document` in every Dynamoid model.
|
|
99
116
|
class User
|
100
117
|
include Dynamoid::Document
|
101
118
|
|
119
|
+
# fields declaration
|
102
120
|
end
|
103
121
|
```
|
104
122
|
|
105
123
|
### Table
|
106
124
|
|
107
|
-
Dynamoid has some sensible defaults for you when you create a new table,
|
125
|
+
Dynamoid has some sensible defaults for you when you create a new table,
|
126
|
+
including the table name and the primary key column. But you can change
|
127
|
+
those if you like on table creation.
|
108
128
|
|
109
129
|
```ruby
|
110
130
|
class User
|
@@ -114,18 +134,32 @@ class User
|
|
114
134
|
end
|
115
135
|
```
|
116
136
|
|
117
|
-
These fields will not change an existing table: so specifying a new
|
137
|
+
These fields will not change an existing table: so specifying a new
|
138
|
+
read_capacity and write_capacity here only works correctly for entirely
|
139
|
+
new tables. Similarly, while Dynamoid will look for a table named
|
140
|
+
`awesome_users` in your namespace, it won't change any existing tables
|
141
|
+
to use that name; and if it does find a table with the correct name, it
|
142
|
+
won't change its hash key, which it expects will be `user_id`. If this
|
143
|
+
table doesn't exist yet, however, Dynamoid will create it with these
|
144
|
+
options.
|
118
145
|
|
119
146
|
### Fields
|
120
147
|
|
121
|
-
You'll have to define all the fields on the model and the data type of
|
122
|
-
|
123
|
-
|
124
|
-
|
125
|
-
|
126
|
-
|
127
|
-
|
128
|
-
|
148
|
+
You'll have to define all the fields on the model and the data type of
|
149
|
+
each field. Every field on the object must be included here; if you miss
|
150
|
+
any they'll be completely bypassed during DynamoDB's initialization and
|
151
|
+
will not appear on the model objects.
|
152
|
+
|
153
|
+
By default, fields are assumed to be of type `:string`. Other built-in
|
154
|
+
types are `:integer`, `:number`, `:set`, `:array`, `:map`, `:datetime`,
|
155
|
+
`date`, `:boolean`, `:raw` and `:serialized`. `array` and `map` match
|
156
|
+
List and Map DynamoDB types respectively. `raw` type means you can store
|
157
|
+
Ruby Array, Hash, String and numbers. If built-in types do not suit you,
|
158
|
+
you can use a custom field type represented by an arbitrary class,
|
159
|
+
provided that the class supports a compatible serialization interface.
|
160
|
+
The primary use case for using a custom field type is to represent your
|
161
|
+
business logic with high-level types, while ensuring portability or
|
162
|
+
backward-compatibility of the serialized representation.
|
129
163
|
|
130
164
|
#### Note on boolean type
|
131
165
|
|
@@ -144,7 +178,9 @@ end
|
|
144
178
|
|
145
179
|
#### Note on date type
|
146
180
|
|
147
|
-
By default date fields are persisted as days count since 1 January 1970
|
181
|
+
By default date fields are persisted as days count since 1 January 1970
|
182
|
+
like UNIX time. If you prefer dates to be stored as ISO-8601 formatted
|
183
|
+
strings instead then set `store_as_string` to `true`
|
148
184
|
|
149
185
|
```ruby
|
150
186
|
class Document
|
@@ -156,7 +192,10 @@ end
|
|
156
192
|
|
157
193
|
#### Note on datetime type
|
158
194
|
|
159
|
-
By default datetime fields are persisted as UNIX timestamps with
|
195
|
+
By default datetime fields are persisted as UNIX timestamps with
|
196
|
+
millisecond precision in DynamoDB. If you prefer datetimes to be stored
|
197
|
+
as ISO-8601 formatted strings instead then set `store_as_string` to
|
198
|
+
`true`
|
160
199
|
|
161
200
|
```ruby
|
162
201
|
class Document
|
@@ -166,26 +205,29 @@ class Document
|
|
166
205
|
end
|
167
206
|
```
|
168
207
|
|
169
|
-
**WARNING:** Fields in numeric format are stored with nanoseconds as a
|
170
|
-
That's why `datetime` field
|
208
|
+
**WARNING:** Fields in numeric format are stored with nanoseconds as a
|
209
|
+
fraction part and precision could be lost. That's why `datetime` field
|
210
|
+
in numeric format shouldn't be used as a range key.
|
171
211
|
|
172
|
-
You have two options if you need to use a `datetime` field as a range
|
212
|
+
You have two options if you need to use a `datetime` field as a range
|
213
|
+
key:
|
173
214
|
* string format
|
174
|
-
* store `datetime` values without milliseconds e.g. cut
|
175
|
-
manually with `change` method - `Time.now.change(usec: 0)`
|
215
|
+
* store `datetime` values without milliseconds (e.g. cut
|
216
|
+
them manually with `change` method - `Time.now.change(usec: 0)`
|
176
217
|
|
177
218
|
#### Note on set type
|
178
219
|
|
179
220
|
`Dynamoid`'s type `set` is stored as DynamoDB's Set attribute type.
|
180
|
-
DynamoDB supports only Set of strings, numbers and binary.
|
181
|
-
|
221
|
+
DynamoDB supports only Set of strings, numbers and binary. Moreover Set
|
222
|
+
*must* contain elements of the same type only.
|
182
223
|
|
183
|
-
In order to use some other `Dynamoid`'s types you can specify `of`
|
184
|
-
to declare the type of set elements.
|
224
|
+
In order to use some other `Dynamoid`'s types you can specify `of`
|
225
|
+
option to declare the type of set elements.
|
185
226
|
|
186
227
|
As a result of that DynamoDB limitation, in Dynamoid only the following
|
187
228
|
scalar types are supported (note: does not support `boolean`):
|
188
|
-
`integer`, `number`, `date`, `datetime`, `serializable` and custom
|
229
|
+
`integer`, `number`, `date`, `datetime`, `serializable` and custom
|
230
|
+
types.
|
189
231
|
|
190
232
|
```ruby
|
191
233
|
class Document
|
@@ -195,8 +237,9 @@ class Document
|
|
195
237
|
end
|
196
238
|
```
|
197
239
|
|
198
|
-
It's possible to specify field options like `store_as_string` for
|
199
|
-
or `serializer` for `serializable` field for `set`
|
240
|
+
It's possible to specify field options like `store_as_string` for
|
241
|
+
`datetime` field or `serializer` for `serializable` field for `set`
|
242
|
+
elements type:
|
200
243
|
|
201
244
|
```ruby
|
202
245
|
class Document
|
@@ -209,12 +252,14 @@ end
|
|
209
252
|
```
|
210
253
|
|
211
254
|
DynamoDB doesn't allow empty strings in fields configured as `set`.
|
212
|
-
Abiding by this restriction, when `Dynamoid` saves a document it removes
|
255
|
+
Abiding by this restriction, when `Dynamoid` saves a document it removes
|
256
|
+
all empty strings in set fields.
|
213
257
|
|
214
258
|
#### Note on array type
|
215
259
|
|
216
260
|
`Dynamoid`'s type `array` is stored as DynamoDB's List attribute type.
|
217
|
-
It can contain elements of different types (in contrast to Set attribute
|
261
|
+
It can contain elements of different types (in contrast to Set attribute
|
262
|
+
type).
|
218
263
|
|
219
264
|
If you need to store in array field elements of `datetime`, `date`,
|
220
265
|
`serializable` or some custom type, which DynamoDB doesn't support
|
@@ -230,7 +275,8 @@ end
|
|
230
275
|
|
231
276
|
#### Magic Columns
|
232
277
|
|
233
|
-
You get magic columns of `id` (`string`), `created_at` (`datetime`), and
|
278
|
+
You get magic columns of `id` (`string`), `created_at` (`datetime`), and
|
279
|
+
`updated_at` (`datetime`) for free.
|
234
280
|
|
235
281
|
```ruby
|
236
282
|
class User
|
@@ -248,11 +294,12 @@ end
|
|
248
294
|
|
249
295
|
#### Default Values
|
250
296
|
|
251
|
-
You can optionally set a default value on a field using either a plain
|
297
|
+
You can optionally set a default value on a field using either a plain
|
298
|
+
value or a lambda:
|
252
299
|
|
253
300
|
```ruby
|
254
|
-
|
255
|
-
|
301
|
+
field :actions_taken, :integer, default: 0
|
302
|
+
field :joined_at, :datetime, default: -> { Time.now }
|
256
303
|
```
|
257
304
|
|
258
305
|
#### Custom Types
|
@@ -260,64 +307,68 @@ You can optionally set a default value on a field using either a plain value or
|
|
260
307
|
To use a custom type for a field, suppose you have a `Money` type.
|
261
308
|
|
262
309
|
```ruby
|
263
|
-
|
264
|
-
|
310
|
+
class Money
|
311
|
+
# ... your business logic ...
|
265
312
|
|
266
|
-
|
267
|
-
|
268
|
-
|
313
|
+
def dynamoid_dump
|
314
|
+
'serialized representation as a string'
|
315
|
+
end
|
269
316
|
|
270
|
-
|
271
|
-
|
272
|
-
|
273
|
-
end
|
317
|
+
def self.dynamoid_load(serialized_str)
|
318
|
+
# parse serialized representation and return a Money instance
|
319
|
+
Money.new(1.23)
|
274
320
|
end
|
321
|
+
end
|
275
322
|
|
276
|
-
|
277
|
-
|
323
|
+
class User
|
324
|
+
include Dynamoid::Document
|
278
325
|
|
279
|
-
|
280
|
-
|
326
|
+
field :balance, Money
|
327
|
+
end
|
281
328
|
```
|
282
329
|
|
283
|
-
If you want to use a third-party class (which does not support
|
284
|
-
as your field type, you can use
|
285
|
-
|
286
|
-
|
330
|
+
If you want to use a third-party class (which does not support
|
331
|
+
`#dynamoid_dump` and `.dynamoid_load`) as your field type, you can use
|
332
|
+
an adapter class providing `.dynamoid_dump` and `.dynamoid_load` class
|
333
|
+
methods for your third-party class. `.dynamoid_load` can remain the same
|
334
|
+
from the previous example; here we just add a level of indirection for
|
335
|
+
serializing. Example:
|
287
336
|
|
288
337
|
```ruby
|
289
|
-
|
290
|
-
|
338
|
+
# Third-party Money class
|
339
|
+
class Money; end
|
291
340
|
|
292
|
-
|
293
|
-
|
294
|
-
|
295
|
-
|
341
|
+
class MoneyAdapter
|
342
|
+
def self.dynamoid_load(money_serialized_str)
|
343
|
+
Money.new(1.23)
|
344
|
+
end
|
296
345
|
|
297
|
-
|
298
|
-
|
299
|
-
end
|
346
|
+
def self.dynamoid_dump(money_obj)
|
347
|
+
money_obj.value.to_s
|
300
348
|
end
|
349
|
+
end
|
301
350
|
|
302
|
-
|
303
|
-
|
351
|
+
class User
|
352
|
+
include Dynamoid::Document
|
304
353
|
|
305
|
-
|
306
|
-
|
354
|
+
field :balance, MoneyAdapter
|
355
|
+
end
|
307
356
|
```
|
308
357
|
|
309
|
-
Lastly, you can control the data type of your custom-class-backed field
|
310
|
-
This is especially important if you want to use
|
311
|
-
|
312
|
-
|
313
|
-
|
358
|
+
Lastly, you can control the data type of your custom-class-backed field
|
359
|
+
at the DynamoDB level. This is especially important if you want to use
|
360
|
+
your custom field as a numeric range or for number-oriented queries. By
|
361
|
+
default custom fields are persisted as a string attribute, but your
|
362
|
+
custom class can override this with a `.dynamoid_field_type` class
|
363
|
+
method, which would return either `:string` or `:number`.
|
314
364
|
|
315
|
-
DynamoDB may support some other attribute types that are not yet
|
365
|
+
DynamoDB may support some other attribute types that are not yet
|
366
|
+
supported by Dynamoid.
|
316
367
|
|
317
368
|
### Sort key
|
318
369
|
|
319
|
-
Along with partition key table may have a sort key. In order to declare
|
320
|
-
`range` class method should be used:
|
370
|
+
Along with partition key table may have a sort key. In order to declare
|
371
|
+
it in a model `range` class method should be used:
|
321
372
|
|
322
373
|
```ruby
|
323
374
|
class Post
|
@@ -331,9 +382,15 @@ Second argument, type, is optional. Default type is `string`.
|
|
331
382
|
|
332
383
|
### Associations
|
333
384
|
|
334
|
-
Just like in ActiveRecord (or your other favorite ORM), Dynamoid uses
|
385
|
+
Just like in ActiveRecord (or your other favorite ORM), Dynamoid uses
|
386
|
+
associations to create links between models.
|
335
387
|
|
336
|
-
The only supported associations (so far) are `has_many`, `has_one`,
|
388
|
+
The only supported associations (so far) are `has_many`, `has_one`,
|
389
|
+
`has_and_belongs_to_many`, and `belongs_to`. Associations are very
|
390
|
+
simple to create: just specify the type, the name, and then any options
|
391
|
+
you'd like to pass to the association. If there's an inverse association
|
392
|
+
either inferred or specified directly, Dynamoid will update both objects
|
393
|
+
to point at each other.
|
337
394
|
|
338
395
|
```ruby
|
339
396
|
class User
|
@@ -348,7 +405,6 @@ class User
|
|
348
405
|
belongs_to :group, foreign_key: :group_id
|
349
406
|
has_one :role
|
350
407
|
has_and_belongs_to_many :friends, inverse_of: :friending_users
|
351
|
-
|
352
408
|
end
|
353
409
|
|
354
410
|
class Address
|
@@ -357,11 +413,18 @@ class Address
|
|
357
413
|
# ...
|
358
414
|
|
359
415
|
belongs_to :user # Automatically links up with the user model
|
360
|
-
|
361
416
|
end
|
362
417
|
```
|
363
418
|
|
364
|
-
Contrary to what you'd expect, association information is always
|
419
|
+
Contrary to what you'd expect, association information is always
|
420
|
+
contained on the object specifying the association, even if it seems
|
421
|
+
like the association has a foreign key. This is a side effect of
|
422
|
+
DynamoDB's structure: it's very difficult to find foreign keys without
|
423
|
+
an index. Usually you won't find this to be a problem, but it does mean
|
424
|
+
that association methods that build new models will not work correctly -
|
425
|
+
for example, `user.addresses.new` returns an address that is not
|
426
|
+
associated to the user. We'll be correcting this ~soon~ maybe someday,
|
427
|
+
if we get a pull request.
|
365
428
|
|
366
429
|
### Validations
|
367
430
|
|
@@ -378,9 +441,12 @@ class User
|
|
378
441
|
end
|
379
442
|
```
|
380
443
|
|
381
|
-
To see more usage and examples of ActiveModel validations, check out the
|
444
|
+
To see more usage and examples of ActiveModel validations, check out the
|
445
|
+
[ActiveModel validation
|
446
|
+
documentation](http://api.rubyonrails.org/classes/ActiveModel/Validations.html).
|
382
447
|
|
383
|
-
If you want to bypass model validation, pass `validate: false` to `save`
|
448
|
+
If you want to bypass model validation, pass `validate: false` to `save`
|
449
|
+
call:
|
384
450
|
|
385
451
|
```ruby
|
386
452
|
model.save(validate: false)
|
@@ -388,7 +454,9 @@ model.save(validate: false)
|
|
388
454
|
|
389
455
|
### Callbacks
|
390
456
|
|
391
|
-
Dynamoid also employs ActiveModel callbacks. Right now, callbacks are
|
457
|
+
Dynamoid also employs ActiveModel callbacks. Right now, callbacks are
|
458
|
+
defined on `save`, `update`, `destroy`, which allows you to do `before_`
|
459
|
+
or `after_` any of those.
|
392
460
|
|
393
461
|
```ruby
|
394
462
|
class User
|
@@ -404,7 +472,8 @@ end
|
|
404
472
|
|
405
473
|
### STI
|
406
474
|
|
407
|
-
Dynamoid supports STI (Single Table Inheritance) like Active Record
|
475
|
+
Dynamoid supports STI (Single Table Inheritance) like Active Record
|
476
|
+
does. You need just specify `type` field in a base class. Example:
|
408
477
|
|
409
478
|
```ruby
|
410
479
|
class Animal
|
@@ -421,12 +490,13 @@ end
|
|
421
490
|
cat = Cat.create(name: 'Morgan')
|
422
491
|
animal = Animal.find(cat.id)
|
423
492
|
animal.class
|
424
|
-
#=>
|
493
|
+
#=> Cat
|
425
494
|
```
|
426
495
|
|
427
|
-
If you already have DynamoDB tables and `type` field already exists and
|
428
|
-
|
429
|
-
instead of `type` one
|
496
|
+
If you already have DynamoDB tables and `type` field already exists and
|
497
|
+
has its own semantic it leads to conflict. It's possible to tell
|
498
|
+
Dynamoid to use another field (even not existing) instead of `type` one
|
499
|
+
with `inheritance_field` table option:
|
430
500
|
|
431
501
|
```ruby
|
432
502
|
class Car
|
@@ -443,8 +513,9 @@ c.my_new_type
|
|
443
513
|
|
444
514
|
### Type casting
|
445
515
|
|
446
|
-
Dynamid supports type casting and tryes to do it in the most convinient
|
447
|
-
Values for all fields (except custom type) are coerced to declared
|
516
|
+
Dynamid supports type casting and tryes to do it in the most convinient
|
517
|
+
way. Values for all fields (except custom type) are coerced to declared
|
518
|
+
field types.
|
448
519
|
|
449
520
|
Some obvious rules are used, e.g.:
|
450
521
|
|
@@ -468,18 +539,29 @@ document.integer_field = true
|
|
468
539
|
# => 1
|
469
540
|
```
|
470
541
|
|
471
|
-
If time zone isn't specified for `datetime` value - application time
|
542
|
+
If time zone isn't specified for `datetime` value - application time
|
543
|
+
zone is used.
|
472
544
|
|
473
545
|
To access field value before type casting following method could be
|
474
|
-
used: `attributes_before_type_cast` and
|
546
|
+
used: `attributes_before_type_cast` and
|
547
|
+
`read_attribute_before_type_cast`.
|
548
|
+
|
549
|
+
There is `<name>_before_type_cast` method for every field in a model as
|
550
|
+
well.
|
475
551
|
|
476
|
-
|
552
|
+
### Dirty API
|
553
|
+
|
554
|
+
Dynamoid supports Dirty API which equvalents to [Rails 5.2
|
555
|
+
`ActiveModel::Dirty`](https://api.rubyonrails.org/v5.2/classes/ActiveModel/Dirty.html).
|
556
|
+
There is only one limitation - change in place of field isn't detected
|
557
|
+
automatically.
|
477
558
|
|
478
559
|
## Usage
|
479
560
|
|
480
561
|
### Object Creation
|
481
562
|
|
482
|
-
Dynamoid's syntax is generally very similar to ActiveRecord's. Making
|
563
|
+
Dynamoid's syntax is generally very similar to ActiveRecord's. Making
|
564
|
+
new objects is simple:
|
483
565
|
|
484
566
|
```ruby
|
485
567
|
u = User.new(name: 'Josh')
|
@@ -487,13 +569,15 @@ u.email = 'josh@joshsymonds.com'
|
|
487
569
|
u.save
|
488
570
|
```
|
489
571
|
|
490
|
-
Save forces persistence to the datastore: a unique ID is also assigned,
|
572
|
+
Save forces persistence to the datastore: a unique ID is also assigned,
|
573
|
+
but it is a string and not an auto-incrementing number.
|
491
574
|
|
492
575
|
```ruby
|
493
576
|
u.id # => '3a9f7216-4726-4aea-9fbc-8554ae9292cb'
|
494
577
|
```
|
495
578
|
|
496
|
-
To use associations, you use association methods very similar to
|
579
|
+
To use associations, you use association methods very similar to
|
580
|
+
ActiveRecord's:
|
497
581
|
|
498
582
|
```ruby
|
499
583
|
address = u.addresses.create
|
@@ -520,7 +604,8 @@ Querying can be done in one of three ways:
|
|
520
604
|
|
521
605
|
```ruby
|
522
606
|
Address.find(address.id) # Find directly by ID.
|
523
|
-
Address.where(city: 'Chicago').all # Find by any number of matching criteria...
|
607
|
+
Address.where(city: 'Chicago').all # Find by any number of matching criteria...
|
608
|
+
# Though presently only "where" is supported.
|
524
609
|
Address.find_by_city('Chicago') # The same as above, but using ActiveRecord's older syntax.
|
525
610
|
```
|
526
611
|
|
@@ -530,7 +615,12 @@ And you can also query on associations:
|
|
530
615
|
u.addresses.where(city: 'Chicago').all
|
531
616
|
```
|
532
617
|
|
533
|
-
But keep in mind Dynamoid
|
618
|
+
But keep in mind Dynamoid - and document-based storage systems in
|
619
|
+
general - are not drop-in replacements for existing relational
|
620
|
+
databases. The above query does not efficiently perform a conditional
|
621
|
+
join, but instead finds all the user's addresses and naively filters
|
622
|
+
them in Ruby. For large associations this is a performance hit compared
|
623
|
+
to relational database engines.
|
534
624
|
|
535
625
|
**WARNING:** There is a limitation of conditions passed to `where`
|
536
626
|
method. Only one condition for some particular field could be specified.
|
@@ -544,34 +634,69 @@ User.where(name: 'Mike').where('name.begins_with': 'Ed')
|
|
544
634
|
|
545
635
|
the first one will be ignored and the last one will be used.
|
546
636
|
|
637
|
+
**Warning:** There is a caveat with filtering documents by `nil` value
|
638
|
+
attribute. By default Dynamoid ignores attributes with `nil` value and
|
639
|
+
doesn't store them in a DynamoDB document. This behavior could be
|
640
|
+
changed with `store_attribute_with_nil_value` config option.
|
641
|
+
|
642
|
+
If Dynamoid ignores `nil` value attributes `null`/`not_null` operators
|
643
|
+
should be used in query:
|
644
|
+
|
645
|
+
```ruby
|
646
|
+
Address.where('postcode.null': true)
|
647
|
+
Address.where('postcode.not_null': true)
|
648
|
+
```
|
649
|
+
|
650
|
+
If Dynamoid keeps `nil` value attributes `eq`/`ne` operators should be
|
651
|
+
used instead:
|
652
|
+
|
653
|
+
```ruby
|
654
|
+
Address.where('postcode': nil)
|
655
|
+
Address.where('postcode.ne': nil)
|
656
|
+
```
|
657
|
+
|
547
658
|
#### Limits
|
548
659
|
|
549
660
|
There are three types of limits that you can query with:
|
550
661
|
|
551
|
-
1. `record_limit` - The number of evaluated records that are returned by
|
552
|
-
|
553
|
-
|
662
|
+
1. `record_limit` - The number of evaluated records that are returned by
|
663
|
+
the query.
|
664
|
+
2. `scan_limit` - The number of scanned records that DynamoDB will look
|
665
|
+
at before returning.
|
666
|
+
3. `batch_size` - The number of records requested to DynamoDB per
|
667
|
+
underlying request, good for large queries!
|
554
668
|
|
555
|
-
Using these in various combinations results in the underlying requests
|
556
|
-
|
669
|
+
Using these in various combinations results in the underlying requests
|
670
|
+
to be made in the smallest size possible and the query returns once
|
671
|
+
`record_limit` or `scan_limit` is satisfied. It will attempt to batch
|
672
|
+
whenever possible.
|
557
673
|
|
558
|
-
You can thus limit the number of evaluated records, or select a record
|
674
|
+
You can thus limit the number of evaluated records, or select a record
|
675
|
+
from which to start in order to support pagination.
|
559
676
|
|
560
677
|
```ruby
|
561
678
|
Address.record_limit(5).start(address) # Only 5 addresses starting at `address`
|
562
679
|
```
|
563
|
-
Where `address` is an instance of the model or a hash
|
564
|
-
|
565
|
-
|
566
|
-
|
567
|
-
|
568
|
-
|
680
|
+
Where `address` is an instance of the model or a hash
|
681
|
+
`{the_model_hash_key: 'value', the_model_range_key: 'value'}`. Keep in
|
682
|
+
mind that if you are passing a hash to `.start()` you need to explicitly
|
683
|
+
define all required keys in it including range keys, depending on table
|
684
|
+
or secondary indexes signatures, otherwise you'll get an
|
685
|
+
`Aws::DynamoDB::Errors::ValidationException` either for `Exclusive Start
|
686
|
+
Key must have same size as table's key schema` or `The provided starting
|
687
|
+
key is invalid`
|
688
|
+
|
689
|
+
If you are potentially running over a large data set and this is
|
690
|
+
especially true when using certain filters, you may want to consider
|
691
|
+
limiting the number of scanned records (the number of records DynamoDB
|
692
|
+
infrastructure looks through when evaluating data to return):
|
569
693
|
|
570
694
|
```ruby
|
571
695
|
Address.scan_limit(5).start(address) # Only scan at most 5 records and return what's found starting from `address`
|
572
696
|
```
|
573
697
|
|
574
|
-
For large queries that return many rows, Dynamoid can use AWS' support
|
698
|
+
For large queries that return many rows, Dynamoid can use AWS' support
|
699
|
+
for requesting documents in batches:
|
575
700
|
|
576
701
|
```ruby
|
577
702
|
# Do some maintenance on the entire table without flooding DynamoDB
|
@@ -579,29 +704,32 @@ Address.all(batch_size: 100).each { |address| address.do_some_work; sleep(0.01)
|
|
579
704
|
Address.record_limit(10_000).batch(100).each { … } # Batch specified as part of a chain
|
580
705
|
```
|
581
706
|
|
582
|
-
The implication of batches is that the underlying requests are done in
|
583
|
-
|
707
|
+
The implication of batches is that the underlying requests are done in
|
708
|
+
the batch sizes to make the request and responses more manageable. Note
|
709
|
+
that this batching is for `Query` and `Scans` and not `BatchGetItem`
|
710
|
+
commands.
|
584
711
|
|
585
712
|
#### DynamoDB pagination
|
586
713
|
|
587
|
-
At times it can be useful to rely on DynamoDB [low-level
|
588
|
-
|
589
|
-
|
714
|
+
At times it can be useful to rely on DynamoDB [low-level
|
715
|
+
pagination](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.html#Query.Pagination)
|
716
|
+
instead of fixed pages sizes. Each page results in a single Query or
|
717
|
+
Scan call to DyanmoDB, but returns an unknown number of records.
|
590
718
|
|
591
|
-
Access to the native DynamoDB pages can be obtained via the
|
592
|
-
method, which yields arrays of records.
|
719
|
+
Access to the native DynamoDB pages can be obtained via the
|
720
|
+
`find_by_pages` method, which yields arrays of records.
|
593
721
|
|
594
722
|
```ruby
|
595
723
|
Address.find_by_pages do |addresses, metadata|
|
596
724
|
end
|
597
725
|
```
|
598
726
|
|
599
|
-
Each yielded pages returns page metadata as the second argument, which
|
600
|
-
including a key `:last_evaluated_key`. The value of this key
|
601
|
-
the `start` method to fetch the next page of records.
|
727
|
+
Each yielded pages returns page metadata as the second argument, which
|
728
|
+
is a hash including a key `:last_evaluated_key`. The value of this key
|
729
|
+
can be used for the `start` method to fetch the next page of records.
|
602
730
|
|
603
|
-
This way it can be used for instance to implement efficiently
|
604
|
-
|
731
|
+
This way it can be used for instance to implement efficiently pagination
|
732
|
+
in web-application:
|
605
733
|
|
606
734
|
```ruby
|
607
735
|
class UserController < ApplicationController
|
@@ -620,8 +748,9 @@ end
|
|
620
748
|
|
621
749
|
#### Sort Conditions and Filters
|
622
750
|
|
623
|
-
You are able to optimize query with condition for sort key. Following
|
624
|
-
`
|
751
|
+
You are able to optimize query with condition for sort key. Following
|
752
|
+
operators are available: `gt`, `lt`, `gte`, `lte`, `begins_with`,
|
753
|
+
`between` as well as equality:
|
625
754
|
|
626
755
|
```ruby
|
627
756
|
Address.where(latitude: 10212)
|
@@ -633,18 +762,57 @@ Address.where('city.begins_with': 'Lon')
|
|
633
762
|
Address.where('latitude.between': [10212, 20000])
|
634
763
|
```
|
635
764
|
|
636
|
-
You are able to filter results on the DynamoDB side and specify
|
637
|
-
|
765
|
+
You are able to filter results on the DynamoDB side and specify
|
766
|
+
conditions for non-key fields. Following additional operators are
|
767
|
+
available: `in`, `contains`, `not_contains`, `null`, `not_null`:
|
638
768
|
|
639
769
|
```ruby
|
640
770
|
Address.where('city.in': ['London', 'Edenburg', 'Birmingham'])
|
641
771
|
Address.where('city.contains': ['on'])
|
642
772
|
Address.where('city.not_contains': ['ing'])
|
773
|
+
Address.where('postcode.null': false)
|
774
|
+
Address.where('postcode.not_null': true)
|
775
|
+
```
|
776
|
+
|
777
|
+
**WARNING:** Please take into accout that `NULL` and `NOT_NULL`
|
778
|
+
operators check attribute presence in a document, not value. So if
|
779
|
+
attribute `postcode`'s value is `NULL`, `NULL` operator will return
|
780
|
+
false because attribute exists even if has `NULL` value.
|
781
|
+
|
782
|
+
#### Selecting some specific fields only
|
783
|
+
|
784
|
+
It could be done with `project` method:
|
785
|
+
|
786
|
+
```ruby
|
787
|
+
class User
|
788
|
+
include Dynamoid::Document
|
789
|
+
field :name
|
790
|
+
end
|
791
|
+
|
792
|
+
User.create(name: 'Alex')
|
793
|
+
user = User.project(:name).first
|
794
|
+
|
795
|
+
user.id # => nil
|
796
|
+
user.name # => 'Alex'
|
797
|
+
user.created_at # => nil
|
798
|
+
```
|
799
|
+
|
800
|
+
Returned models with have filled specified fields only.
|
801
|
+
|
802
|
+
Several fields could be specified:
|
803
|
+
|
804
|
+
```ruby
|
805
|
+
user = User.project(:name, :created_at)
|
643
806
|
```
|
644
807
|
|
645
808
|
### Consistent Reads
|
646
809
|
|
647
|
-
Querying supports consistent reading. By default, DynamoDB reads are
|
810
|
+
Querying supports consistent reading. By default, DynamoDB reads are
|
811
|
+
eventually consistent: if you do a write and then a read immediately
|
812
|
+
afterwards, the results of the previous write may not be reflected. If
|
813
|
+
you need to do a consistent read (that is, you need to read the results
|
814
|
+
of a write immediately) you can do so, but keep in mind that consistent
|
815
|
+
reads are twice as expensive as regular reads for DynamoDB.
|
648
816
|
|
649
817
|
```ruby
|
650
818
|
Address.find(address.id, consistent_read: true) # Find an address, ensure the read is consistent.
|
@@ -653,21 +821,25 @@ Address.where(city: 'Chicago').consistent.all # Find all addresses where the
|
|
653
821
|
|
654
822
|
### Range Finding
|
655
823
|
|
656
|
-
If you have a range index, Dynamoid provides a number of additional
|
824
|
+
If you have a range index, Dynamoid provides a number of additional
|
825
|
+
other convenience methods to make your life a little easier:
|
657
826
|
|
658
827
|
```ruby
|
659
828
|
User.where("created_at.gt": DateTime.now - 1.day).all
|
660
829
|
User.where("created_at.lt": DateTime.now - 1.day).all
|
661
830
|
```
|
662
831
|
|
663
|
-
It also supports `gte` and `lte`. Turning those into symbols and
|
832
|
+
It also supports `gte` and `lte`. Turning those into symbols and
|
833
|
+
allowing a Rails SQL-style string syntax is in the works. You can only
|
834
|
+
have one range argument per query, because of DynamoDB's inherent
|
835
|
+
limitations, so use it sensibly!
|
664
836
|
|
665
837
|
|
666
838
|
### Updating
|
667
839
|
|
668
840
|
In order to update document you can use high level methods
|
669
|
-
`#update_attributes`, `#update_attribute` and `.update`.
|
670
|
-
|
841
|
+
`#update_attributes`, `#update_attribute` and `.update`. They run
|
842
|
+
validation and collbacks.
|
671
843
|
|
672
844
|
```ruby
|
673
845
|
Address.find(id).update_attributes(city: 'Chicago')
|
@@ -677,8 +849,8 @@ Address.update(id, { city: 'Chicago' }, if: { deliverable: true })
|
|
677
849
|
```
|
678
850
|
|
679
851
|
There are also some low level methods `#update`, `.update_fields` and
|
680
|
-
`.upsert`. They don't run validation and callbacks (except `#update` -
|
681
|
-
runs `update` callbacks). All of them support conditional updates.
|
852
|
+
`.upsert`. They don't run validation and callbacks (except `#update` -
|
853
|
+
it runs `update` callbacks). All of them support conditional updates.
|
682
854
|
`#upsert` will create new document if document with specified `id`
|
683
855
|
doesn't exist.
|
684
856
|
|
@@ -699,8 +871,8 @@ Address.upsert(id, { city: 'Chicago' }, if: { deliverable: true })
|
|
699
871
|
|
700
872
|
### Deleting
|
701
873
|
|
702
|
-
In order to delete some items `delete_all` method should be used.
|
703
|
-
|
874
|
+
In order to delete some items `delete_all` method should be used. Any
|
875
|
+
callback wont be called. Items delete in efficient way in batch.
|
704
876
|
|
705
877
|
```ruby
|
706
878
|
Address.where(city: 'London').delete_all
|
@@ -724,14 +896,21 @@ end
|
|
724
896
|
There are following options:
|
725
897
|
* `hash_key` - is used as hash key of an index,
|
726
898
|
* `range_key` - is used as range key of an index,
|
727
|
-
* `projected_attributes` - list of fields to store in an index or has a
|
728
|
-
|
729
|
-
* `
|
730
|
-
|
899
|
+
* `projected_attributes` - list of fields to store in an index or has a
|
900
|
+
predefiled value `:keys_only`, `:all`; `:keys_only` is a default,
|
901
|
+
* `name` - an index will be created with this name when a table is
|
902
|
+
created; by default name is generated and contains table name and keys
|
903
|
+
names,
|
904
|
+
* `read_capacity` - is used when table creates and used as an index
|
905
|
+
capacity; by default equals `Dynamoid::Config.read_capacity`,
|
906
|
+
* `write_capacity` - is used when table creates and used as an index
|
907
|
+
capacity; by default equals `Dynamoid::Config.write_capacity`
|
731
908
|
|
732
909
|
The only mandatory option is `name`.
|
733
910
|
|
734
|
-
**WARNING:** In order to use global secondary index in `Document.where`
|
911
|
+
**WARNING:** In order to use global secondary index in `Document.where`
|
912
|
+
implicitly you need to have all the attributes of the original table in
|
913
|
+
the index and declare it with option `projected_attributes: :all`:
|
735
914
|
|
736
915
|
```ruby
|
737
916
|
class User
|
@@ -741,13 +920,16 @@ class User
|
|
741
920
|
end
|
742
921
|
```
|
743
922
|
|
744
|
-
There is only one implicit way to query Global and Local Secondary
|
923
|
+
There is only one implicit way to query Global and Local Secondary
|
924
|
+
Indexes (GSI/LSI).
|
745
925
|
|
746
926
|
#### Implicit
|
747
927
|
|
748
|
-
The second way implicitly uses your GSI through the `where` clauses and
|
749
|
-
|
750
|
-
|
928
|
+
The second way implicitly uses your GSI through the `where` clauses and
|
929
|
+
deduces the index based on the query fields provided. Another added
|
930
|
+
benefit is that it is built into query chaining so you can use all the
|
931
|
+
methods used in normal querying. The explicit way from above would be
|
932
|
+
rewritten as follows:
|
751
933
|
|
752
934
|
```ruby
|
753
935
|
where(dynamo_primary_key_column_name => dynamo_primary_key_value,
|
@@ -755,57 +937,107 @@ where(dynamo_primary_key_column_name => dynamo_primary_key_value,
|
|
755
937
|
.scan_index_forward(false)
|
756
938
|
```
|
757
939
|
|
758
|
-
The only caveat with this method is that because it is also used for
|
759
|
-
|
760
|
-
|
761
|
-
|
762
|
-
|
940
|
+
The only caveat with this method is that because it is also used for
|
941
|
+
general querying, it WILL NOT use a GSI unless it explicitly has defined
|
942
|
+
`projected_attributes: :all` on the GSI in your model. This is because
|
943
|
+
GSIs that do not have all attributes projected will only contain the
|
944
|
+
index keys and therefore will not return objects with fully resolved
|
945
|
+
field values. It currently opts to provide the complete results rather
|
946
|
+
than partial results unless you've explicitly looked up the data.
|
763
947
|
|
764
|
-
*Future TODO could involve implementing `select` in chaining as well as
|
765
|
-
the
|
948
|
+
*Future TODO could involve implementing `select` in chaining as well as
|
949
|
+
resolving the fields with a second query against the table since a query
|
950
|
+
against GSI then a query on base table is still likely faster than scan
|
951
|
+
on the base table*
|
766
952
|
|
767
953
|
## Configuration
|
768
954
|
|
769
955
|
Listed below are all configuration options.
|
770
956
|
|
771
|
-
* `adapter` - usefull only for the gem developers to switch to a new
|
772
|
-
|
773
|
-
* `
|
774
|
-
|
775
|
-
|
776
|
-
* `
|
777
|
-
|
778
|
-
|
779
|
-
* `
|
957
|
+
* `adapter` - usefull only for the gem developers to switch to a new
|
958
|
+
adapter. Default and the only available value is `aws_sdk_v3`
|
959
|
+
* `namespace` - prefix for table names, default is
|
960
|
+
`dynamoid_#{application_name}_#{environment}` for Rails application
|
961
|
+
and `dynamoid` otherwise
|
962
|
+
* `logger` - by default it's a `Rails.logger` in Rails application and
|
963
|
+
`stdout` otherwise. You can disable logging by setting `nil` or
|
964
|
+
`false` values. Set `true` value to use defaults
|
965
|
+
* `access_key` - DynamoDb custom credentials for AWS, override global
|
966
|
+
AWS credentials if they present
|
967
|
+
* `secret_key` - DynamoDb custom credentials for AWS, override global
|
968
|
+
AWS credentials if they present
|
969
|
+
* `region` - DynamoDb custom credentials for AWS, override global AWS
|
970
|
+
credentials if they present
|
971
|
+
* `batch_size` - when you try to load multiple items at once with
|
972
|
+
* `batch_get_item` call Dynamoid loads them not with one api call but
|
973
|
+
piece by piece. Default is 100 items
|
974
|
+
* `read_capacity` - is used at table or indices creation. Default is 100
|
975
|
+
(units)
|
976
|
+
* `write_capacity` - is used at table or indices creation. Default is 20
|
977
|
+
(units)
|
780
978
|
* `warn_on_scan` - log warnings when scan table. Default is `true`
|
781
|
-
* `endpoint` - if provided, it communicates with the DynamoDB listening
|
782
|
-
|
979
|
+
* `endpoint` - if provided, it communicates with the DynamoDB listening
|
980
|
+
at the endpoint. This is useful for testing with
|
981
|
+
[DynamoDB Local](http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Tools.DynamoDBLocal.html)
|
982
|
+
* `identity_map` - ensures that each object gets loaded only once by
|
983
|
+
keeping every loaded object in a map. Looks up objects using the map
|
984
|
+
when referring to them. Isn't thread safe. Default is `false`.
|
783
985
|
`Use Dynamoid::Middleware::IdentityMap` to clear identity map for each HTTP request
|
784
|
-
* `timestamps` - by default Dynamoid sets `created_at` and `updated_at`
|
785
|
-
|
786
|
-
|
787
|
-
* `
|
788
|
-
|
789
|
-
|
790
|
-
|
791
|
-
|
792
|
-
|
793
|
-
|
794
|
-
* `
|
795
|
-
|
796
|
-
|
797
|
-
|
798
|
-
* `
|
799
|
-
|
800
|
-
* `
|
801
|
-
*
|
802
|
-
|
803
|
-
|
986
|
+
* `timestamps` - by default Dynamoid sets `created_at` and `updated_at`
|
987
|
+
fields for model at creation and updating. You can disable this
|
988
|
+
behavior by setting `false` value
|
989
|
+
* `sync_retry_max_times` - when Dynamoid creates or deletes table
|
990
|
+
synchronously it checks for completion specified times. Default is 60
|
991
|
+
(times). It's a bit over 2 minutes by default
|
992
|
+
* `sync_retry_wait_seconds` - time to wait between retries. Default is 2
|
993
|
+
(seconds)
|
994
|
+
* `convert_big_decimal` - if `true` then Dynamoid converts numbers
|
995
|
+
stored in `Hash` in `raw` field to float. Default is `false`
|
996
|
+
* `store_attribute_with_nil_value` - if `true` Dynamoid keeps attribute
|
997
|
+
with `nil` value in a document. Otherwise Dynamoid removes it while
|
998
|
+
saving a document. Default is `nil` which equals behaviour with `false`
|
999
|
+
value.
|
1000
|
+
* `models_dir` - `dynamoid:create_tables` rake task loads DynamoDb
|
1001
|
+
models from this directory. Default is `./app/models`.
|
1002
|
+
* `application_timezone` - Dynamoid converts all `datetime` fields to
|
1003
|
+
* specified time zone when loads data from the storage.
|
1004
|
+
Acceptable values - `:utc`, `:local` (to use system time zone) and
|
1005
|
+
time zone name e.g. `Eastern Time (US & Canada)`. Default is `utc`
|
1006
|
+
* `dynamodb_timezone` - When a datetime field is stored in string format
|
1007
|
+
Dynamoid converts it to specified time zone when saves a value to the
|
1008
|
+
storage. Acceptable values - `:utc`, `:local` (to use system time
|
1009
|
+
zone) and time zone name e.g. `Eastern Time (US & Canada)`. Default is
|
1010
|
+
`utc`
|
1011
|
+
* `store_datetime_as_string` - if `true` then Dynamoid stores :datetime
|
1012
|
+
fields in ISO 8601 string format. Default is `false`
|
1013
|
+
* `store_date_as_string` - if `true` then Dynamoid stores :date fields
|
1014
|
+
in ISO 8601 string format. Default is `false`
|
1015
|
+
* `store_boolean_as_native` - if `true` Dynamoid stores boolean fields
|
1016
|
+
as native DynamoDB boolean values. Otherwise boolean fields are stored
|
1017
|
+
as string values `'t'` and `'f'`. Default is true
|
1018
|
+
* `backoff` - is a hash: key is a backoff strategy (symbol), value is
|
1019
|
+
parameters for the strategy. Is used in batch operations. Default id
|
1020
|
+
`nil`
|
1021
|
+
* `backoff_strategies`: is a hash and contains all available strategies.
|
1022
|
+
Default is { constant: ..., exponential: ...}
|
1023
|
+
* `http_continue_timeout`: The number of seconds to wait for a
|
1024
|
+
100-continue HTTP response before sending the request body. Default
|
1025
|
+
option value is `nil`. If not specified effected value is `1`
|
1026
|
+
* `http_idle_timeout`: The number of seconds an HTTP connection is
|
1027
|
+
allowed to sit idble before it is considered stale. Default option
|
1028
|
+
value is `nil`. If not specified effected value is `5`
|
1029
|
+
* `http_open_timeout`: The number of seconds to wait when opening a HTTP
|
1030
|
+
session. Default option value is `nil`. If not specified effected
|
1031
|
+
value is `15`
|
1032
|
+
* `http_read_timeout`:The number of seconds to wait for HTTP response
|
1033
|
+
data. Default option value is `nil`. If not specified effected value
|
1034
|
+
is `60`
|
804
1035
|
|
805
1036
|
|
806
1037
|
## Concurrency
|
807
1038
|
|
808
|
-
Dynamoid supports basic, ActiveRecord-like optimistic locking on save
|
1039
|
+
Dynamoid supports basic, ActiveRecord-like optimistic locking on save
|
1040
|
+
operations. Simply add a `lock_version` column to your table like so:
|
809
1041
|
|
810
1042
|
```ruby
|
811
1043
|
class MyTable
|
@@ -817,23 +1049,38 @@ class MyTable
|
|
817
1049
|
end
|
818
1050
|
```
|
819
1051
|
|
820
|
-
In this example, all saves to `MyTable` will raise an
|
1052
|
+
In this example, all saves to `MyTable` will raise an
|
1053
|
+
`Dynamoid::Errors::StaleObjectError` if a concurrent process loaded,
|
1054
|
+
edited, and saved the same row. Your code should trap this exception,
|
1055
|
+
reload the row (so that it will pick up the newest values), and try the
|
1056
|
+
save again.
|
821
1057
|
|
822
|
-
Calls to `update` and `update!` also increment the `lock_version`,
|
1058
|
+
Calls to `update` and `update!` also increment the `lock_version`,
|
1059
|
+
however they do not check the existing value. This guarantees that a
|
1060
|
+
update operation will raise an exception in a concurrent save operation,
|
1061
|
+
however a save operation will never cause an update to fail. Thus,
|
1062
|
+
`update` is useful & safe only for doing atomic operations (e.g.
|
1063
|
+
increment a value, add/remove from a set, etc), but should not be used
|
1064
|
+
in a read-modify-write pattern.
|
823
1065
|
|
824
1066
|
|
825
1067
|
### Backoff strategies
|
826
1068
|
|
827
1069
|
|
828
|
-
You can use several methods that run efficiently in batch mode like
|
1070
|
+
You can use several methods that run efficiently in batch mode like
|
1071
|
+
`.find_all` and `.import`. It affects `Query` and `Scan` operations as
|
1072
|
+
well.
|
829
1073
|
|
830
|
-
The backoff strategy will be used when, for any reason, some items could
|
831
|
-
|
1074
|
+
The backoff strategy will be used when, for any reason, some items could
|
1075
|
+
not be processed as part of a batch mode command. Operations will be
|
1076
|
+
re-run to process these items.
|
832
1077
|
|
833
|
-
Exponential backoff is the recommended way to handle throughput limits
|
1078
|
+
Exponential backoff is the recommended way to handle throughput limits
|
1079
|
+
exceeding and throttling on the table.
|
834
1080
|
|
835
|
-
There are two built-in strategies - constant delay and truncated binary
|
836
|
-
By default no backoff is used but you can specify
|
1081
|
+
There are two built-in strategies - constant delay and truncated binary
|
1082
|
+
exponential backoff. By default no backoff is used but you can specify
|
1083
|
+
one of the built-in ones:
|
837
1084
|
|
838
1085
|
```ruby
|
839
1086
|
Dynamoid.configure do |config|
|
@@ -846,7 +1093,8 @@ end
|
|
846
1093
|
|
847
1094
|
```
|
848
1095
|
|
849
|
-
You can just specify strategy without any arguments to use default
|
1096
|
+
You can just specify strategy without any arguments to use default
|
1097
|
+
presets:
|
850
1098
|
|
851
1099
|
```ruby
|
852
1100
|
Dynamoid.configure do |config|
|
@@ -871,10 +1119,11 @@ end
|
|
871
1119
|
|
872
1120
|
There are a few Rake tasks available out of the box:
|
873
1121
|
|
874
|
-
|
875
|
-
|
1122
|
+
* `rake dynamoid:create_tables`
|
1123
|
+
* `rake dynamoid:ping`
|
876
1124
|
|
877
|
-
In order to use them in non-Rails application they should be required
|
1125
|
+
In order to use them in non-Rails application they should be required
|
1126
|
+
explicitly:
|
878
1127
|
|
879
1128
|
```ruby
|
880
1129
|
# Rakefile
|
@@ -883,12 +1132,14 @@ Rake::Task.define_task(:environment)
|
|
883
1132
|
require 'dynamoid/tasks'
|
884
1133
|
```
|
885
1134
|
|
886
|
-
The Rake tasks depend on `:environment` task so it should be declared
|
887
|
-
|
1135
|
+
The Rake tasks depend on `:environment` task so it should be declared as
|
1136
|
+
well.
|
888
1137
|
|
889
1138
|
## Test Environment
|
890
1139
|
|
891
|
-
In test environment you will most likely want to clean the database
|
1140
|
+
In test environment you will most likely want to clean the database
|
1141
|
+
between test runs to keep tests completely isolated. This can be
|
1142
|
+
achieved like so
|
892
1143
|
|
893
1144
|
```ruby
|
894
1145
|
module DynamoidReset
|
@@ -919,7 +1170,8 @@ RSpec.configure do |config|
|
|
919
1170
|
end
|
920
1171
|
```
|
921
1172
|
|
922
|
-
In Rails, you may also want to ensure you do not delete non-test data
|
1173
|
+
In Rails, you may also want to ensure you do not delete non-test data
|
1174
|
+
accidentally by adding the following to your test environment setup:
|
923
1175
|
|
924
1176
|
```ruby
|
925
1177
|
raise "Tests should be run in 'test' environment only" if Rails.env != 'test'
|
@@ -956,9 +1208,14 @@ timing (231.28 ms).
|
|
956
1208
|
|
957
1209
|
## Credits
|
958
1210
|
|
959
|
-
Dynamoid borrows code, structure, and even its name very liberally from
|
1211
|
+
Dynamoid borrows code, structure, and even its name very liberally from
|
1212
|
+
the truly amazing [Mongoid](https://github.com/mongoid/mongoid). Without
|
1213
|
+
Mongoid to crib from none of this would have been possible, and I hope
|
1214
|
+
they don't mind me reusing their very awesome ideas to make DynamoDB
|
1215
|
+
just as accessible to the Ruby world as MongoDB.
|
960
1216
|
|
961
|
-
Also, without contributors the project wouldn't be nearly as awesome. So
|
1217
|
+
Also, without contributors the project wouldn't be nearly as awesome. So
|
1218
|
+
many thanks to:
|
962
1219
|
|
963
1220
|
* [Logan Bowers](https://github.com/loganb)
|
964
1221
|
* [Lane LaRue](https://github.com/luxx)
|
@@ -979,9 +1236,12 @@ Also, without contributors the project wouldn't be nearly as awesome. So many th
|
|
979
1236
|
|
980
1237
|
## Running the tests
|
981
1238
|
|
982
|
-
Running the tests is fairly simple. You should have an instance of
|
1239
|
+
Running the tests is fairly simple. You should have an instance of
|
1240
|
+
DynamoDB running locally. Follow these steps to setup your test
|
1241
|
+
environment.
|
983
1242
|
|
984
|
-
* First download and unpack the latest version of DynamoDB.
|
1243
|
+
* First download and unpack the latest version of DynamoDB. We have a
|
1244
|
+
script that will do this for you if you use homebrew on a Mac.
|
985
1245
|
|
986
1246
|
```shell
|
987
1247
|
bin/setup
|
@@ -999,13 +1259,18 @@ Running the tests is fairly simple. You should have an instance of DynamoDB runn
|
|
999
1259
|
rake
|
1000
1260
|
```
|
1001
1261
|
|
1002
|
-
* When you are done, remember to stop the local test instance of
|
1262
|
+
* When you are done, remember to stop the local test instance of
|
1263
|
+
dynamodb
|
1003
1264
|
|
1004
1265
|
```shell
|
1005
1266
|
bin/stop_dynamodblocal
|
1006
1267
|
```
|
1007
1268
|
|
1008
|
-
If you want to run all the specs that travis runs, use `bundle exec
|
1269
|
+
If you want to run all the specs that travis runs, use `bundle exec
|
1270
|
+
wwtd`, but first you will need to setup all the rubies, for each of `%w(
|
1271
|
+
2.0.0-p648 2.1.10 2.2.6 2.3.3 2.4.1 jruby-9.1.8.0 )`. When you run
|
1272
|
+
`bundle exec wwtd` it will take care of starting and stopping the local
|
1273
|
+
dynamodb instance.
|
1009
1274
|
|
1010
1275
|
```shell
|
1011
1276
|
rvm use 2.0.0-p648
|