activestorage-aws-record 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +55 -0
- data/LICENSE +21 -0
- data/PLAN.md +413 -0
- data/README.md +287 -0
- data/lib/active_storage/aws_record/attachable.rb +104 -0
- data/lib/active_storage/aws_record/attachment.rb +426 -0
- data/lib/active_storage/aws_record/blob.rb +417 -0
- data/lib/active_storage/aws_record/configuration.rb +62 -0
- data/lib/active_storage/aws_record/item.rb +86 -0
- data/lib/active_storage/aws_record/owner.rb +82 -0
- data/lib/active_storage/aws_record/persistence.rb +89 -0
- data/lib/active_storage/aws_record/railtie.rb +53 -0
- data/lib/active_storage/aws_record/relation.rb +163 -0
- data/lib/active_storage/aws_record/schema.rb +148 -0
- data/lib/active_storage/aws_record/tables.rb +82 -0
- data/lib/active_storage/aws_record/tasks.rake +15 -0
- data/lib/active_storage/aws_record/transaction.rb +132 -0
- data/lib/active_storage/aws_record/variant_record.rb +208 -0
- data/lib/active_storage/aws_record/version.rb +7 -0
- data/lib/active_storage/aws_record.rb +148 -0
- data/lib/activestorage-aws-record.rb +4 -0
- metadata +166 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: 6eedda4242052ab202facd789ab82ec9be05303f20a7dfd449175ec369a2b9ad
|
|
4
|
+
data.tar.gz: f012e2afac8f136d1fa6a1c8fdb8cf13b5160b7455a5da116dc00ad93b97305b
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 9e47c3da5a876c8b1b27ef801a3425d0712dc5aa1266877d59744cf817664411062c6731f2dca022b9743bb0c65bc484b0ab05d54a0dcbc7687f882f46ed87a1
|
|
7
|
+
data.tar.gz: 00ede1d89230d5eb6f813d1974d2b5120a845b47b742ddf264b10662ab9e26443c1cd68e340dd2ba596f254c96fa5ca2687a224b456bbeaa5b89090e20bbda50
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project are documented here. The format is based on
|
|
4
|
+
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres
|
|
5
|
+
to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
|
+
|
|
7
|
+
## [Unreleased]
|
|
8
|
+
|
|
9
|
+
## [0.1.0] - 2026-06-02
|
|
10
|
+
|
|
11
|
+
Initial release. Companion to the Active Storage generic custom-backend contract
|
|
12
|
+
proposed in [`rails/rails#57537`](https://github.com/rails/rails/pull/57537);
|
|
13
|
+
until it ships in a Rails release, use this gem against that branch.
|
|
14
|
+
|
|
15
|
+
### Added
|
|
16
|
+
|
|
17
|
+
- Active Storage **metadata** backend on Amazon DynamoDB via
|
|
18
|
+
[`aws-record`](https://github.com/aws/aws-record-ruby), implementing Rails' generic
|
|
19
|
+
(non-ActiveRecord) custom-backend contract. Blob bytes still flow through a
|
|
20
|
+
normal Active Storage Service (Disk/S3/Mirror); only Blob, Attachment, and
|
|
21
|
+
VariantRecord metadata lives in DynamoDB.
|
|
22
|
+
- **Atomic multi-attachment changes**: a `has_many` clear/replace/detach commits
|
|
23
|
+
every row delete and a coalesced reference-count decrement per blob in one
|
|
24
|
+
DynamoDB transaction (fail-closed at DynamoDB's 100-action limit), so it can
|
|
25
|
+
never delete some rows and leave others.
|
|
26
|
+
- **Single Table Design**: all three entity types live in one
|
|
27
|
+
application-provided table, addressed by `#`-separated composite keys under a
|
|
28
|
+
configurable namespace. No gem-owned tables.
|
|
29
|
+
- **Zero-configuration key discovery**: the partition/sort key attribute names
|
|
30
|
+
and types are auto-detected from the live table via `describe_table` at boot.
|
|
31
|
+
The only required setting is the table name.
|
|
32
|
+
- **Two storage modes, chosen automatically by the range key's type**:
|
|
33
|
+
- *Mode A* (String range key) — adjacency keys live directly in the base
|
|
34
|
+
table; every read is strongly consistent; no GSI.
|
|
35
|
+
- *Mode B* (Number range key) — adjacency keys live in an auto-detected
|
|
36
|
+
string-keyed GSI; point lookups, the reference count, and the foreign-key
|
|
37
|
+
guard stay strong on the base table; listing is eventually consistent.
|
|
38
|
+
- Strongly-consistent shared-blob protection: a transactional reference count on
|
|
39
|
+
the blob item, with a conditional-delete foreign-key guard
|
|
40
|
+
(`ActiveStorage::ForeignKeyViolation`).
|
|
41
|
+
- Two owner concerns: `ActiveStorage::AwsRecord::Owner` for a greenfield
|
|
42
|
+
`aws-record` model (persistence + contract glue), and
|
|
43
|
+
`ActiveStorage::AwsRecord::Attachable` for a model that brings its own
|
|
44
|
+
persistence (versioning/events/search) — contract glue only, without overriding
|
|
45
|
+
`save`/`destroy`. Both enable `has_one_attached` / `has_many_attached`.
|
|
46
|
+
- Fiber-safe (Falcon-ready): eager mutex, mutex-guarded client repository,
|
|
47
|
+
read-only post-boot schema cache, lambda `map_attr` defaults.
|
|
48
|
+
- Rails Railtie wiring, a development/test single-table manager, and in-app rake
|
|
49
|
+
tasks (`activestorage_aws_record:table:create` / `:delete`).
|
|
50
|
+
- Standard gem tooling: a `Rakefile` (`rake test` / `rake smoke` / `rake build`),
|
|
51
|
+
RuboCop config (Rails Omakase via `rubocop-rails-omakase` + `rubocop-rake`),
|
|
52
|
+
`bin/console` + `bin/setup`, and a `docker-compose.yml` for DynamoDB Local.
|
|
53
|
+
|
|
54
|
+
[Unreleased]: https://github.com/thomaswitt/activestorage-aws-record/compare/v0.1.0...HEAD
|
|
55
|
+
[0.1.0]: https://github.com/thomaswitt/activestorage-aws-record/releases/tag/v0.1.0
|
data/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Thomas Witt
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
data/PLAN.md
ADDED
|
@@ -0,0 +1,413 @@
|
|
|
1
|
+
# activestorage-aws-record — Implementation Spec
|
|
2
|
+
|
|
3
|
+
## Context & goal
|
|
4
|
+
|
|
5
|
+
A Ruby gem that lets **Active Storage run on Amazon DynamoDB** (via
|
|
6
|
+
[`aws-record`](https://github.com/aws/aws-record-ruby)) instead of Active Record,
|
|
7
|
+
implementing the *generic custom-backend contract* from
|
|
8
|
+
`../rails/guides/source/active_storage_custom_backend.md` (the
|
|
9
|
+
`activestorage-backends` work). It is the canonical **example implementation**
|
|
10
|
+
of that contract and is meant to drop into existing aws-record apps.
|
|
11
|
+
|
|
12
|
+
**Scope:** *metadata* backend only. Blob **bytes** flow through a normal Active
|
|
13
|
+
Storage `Service` (`Disk`/`S3`/`Mirror`); only blob / attachment /
|
|
14
|
+
variant-record **metadata** lives in DynamoDB. The Service layer, analyzers,
|
|
15
|
+
previewers, variants, jobs, and controllers are reused from Active Storage
|
|
16
|
+
unchanged.
|
|
17
|
+
|
|
18
|
+
The design follows **Single Table Design**: every Active Storage item — blob,
|
|
19
|
+
attachment, variant record — lives in **one** DynamoDB table, addressed by
|
|
20
|
+
`#`-separated composite keys. The gem **auto-detects** the table's
|
|
21
|
+
partition/sort key attribute names and assumes as little as possible, so it
|
|
22
|
+
drops into an existing single-table application by configuration alone.
|
|
23
|
+
|
|
24
|
+
## Requirements
|
|
25
|
+
|
|
26
|
+
1. **Single Table Design.** One table holds all three entity types. No
|
|
27
|
+
gem-owned tables.
|
|
28
|
+
2. **Minimal assumptions:** a single table, *a* hash-key attribute, and *a*
|
|
29
|
+
range-key attribute — nothing about their names. Keys use the `#` separator
|
|
30
|
+
pattern.
|
|
31
|
+
3. **Auto-detect** the partition/range key attribute names (and types) from the
|
|
32
|
+
live table via `describe_table`; in the common case the app configures *only
|
|
33
|
+
the table name*.
|
|
34
|
+
4. **No GSI when the range key is a String** — the base table's `(hash, range)`
|
|
35
|
+
alone serves every access pattern (see *Why no blob→attachment index is
|
|
36
|
+
needed*). When the range key is numeric, auto-configure a single GSI instead
|
|
37
|
+
(see *Storage modes*).
|
|
38
|
+
5. **As flexible / configurable as possible.** Table name, key-attribute names
|
|
39
|
+
(override of auto-detect), key namespace, and separator are all settable.
|
|
40
|
+
Don't hardcode host-app conventions (e.g. a project-specific `compose_key`
|
|
41
|
+
delimiter); a private `#`-join with non-blank validation is enough.
|
|
42
|
+
6. **Fiber-safe (Falcon):** no `||=` on shared mutable state, eager `Mutex`,
|
|
43
|
+
mutex-guarded client repository, `Fiber[]` over `Thread.current`, no `@@`,
|
|
44
|
+
no blocking `sleep`, lambda defaults for `map_attr`.
|
|
45
|
+
7. **Coding guidelines:** no `# frozen_string_literal: true` magic comment,
|
|
46
|
+
Ruby 3.4 idioms (`it`, hash shorthand, single quotes, keyword args), YARD on
|
|
47
|
+
public methods, never SCAN a request path, mandatory expression aliasing on
|
|
48
|
+
every DynamoDB expression.
|
|
49
|
+
|
|
50
|
+
## Dependencies & versioning
|
|
51
|
+
|
|
52
|
+
- `activestorage` **with generic custom-backend support** (the
|
|
53
|
+
`activestorage-backends` branch / future Rails ≥ 8.1). Dev/test `Gemfile`
|
|
54
|
+
path-references `../rails/*`.
|
|
55
|
+
- `aws-record ~> 2.15`, `aws-sdk-dynamodb ~> 1`.
|
|
56
|
+
- Transitive: `activemodel`, `activejob`, `activesupport`, `actionpack`,
|
|
57
|
+
`globalid`. Ruby `>= 3.4`.
|
|
58
|
+
|
|
59
|
+
## Gem layout
|
|
60
|
+
|
|
61
|
+
```
|
|
62
|
+
lib/
|
|
63
|
+
activestorage-aws-record.rb
|
|
64
|
+
active_storage/aws_record.rb # module, config, client repo, install!, schema discovery
|
|
65
|
+
active_storage/aws_record/version.rb
|
|
66
|
+
active_storage/aws_record/configuration.rb
|
|
67
|
+
active_storage/aws_record/railtie.rb # class wiring + Services + client
|
|
68
|
+
active_storage/aws_record/persistence.rb# shared Aws::Record concern + key attrs + key helpers
|
|
69
|
+
active_storage/aws_record/relation.rb # DynamoDB-backed Relation
|
|
70
|
+
active_storage/aws_record/blob.rb
|
|
71
|
+
active_storage/aws_record/attachment.rb
|
|
72
|
+
active_storage/aws_record/variant_record.rb
|
|
73
|
+
active_storage/aws_record/owner.rb # app-model concern (callback layer)
|
|
74
|
+
active_storage/aws_record/tables.rb # dev/test single-table helper (create/delete)
|
|
75
|
+
active_storage/aws_record/tasks.rake
|
|
76
|
+
test/ (minitest, DynamoDB Local via docker-compose)
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
## Configuration (`ActiveStorage::AwsRecord::Configuration`)
|
|
80
|
+
|
|
81
|
+
| setting | default | meaning |
|
|
82
|
+
|---|---|---|
|
|
83
|
+
| `table_name` | `"active_storage"` | the single shared table |
|
|
84
|
+
| `partition_key` | `nil` → auto-detect | hash-key **DB attribute name** (override of `describe_table`) |
|
|
85
|
+
| `sort_key` | `nil` → auto-detect | range-key **DB attribute name** (override) |
|
|
86
|
+
| `namespace` | `"ActiveStorage"` | first segment of every key; isolates AS items from app items |
|
|
87
|
+
| `separator` | `"#"` | key segment delimiter |
|
|
88
|
+
| `index_name` | `"active_storage_index"` | GSI used when the range key is numeric (Mode B) |
|
|
89
|
+
| `index_partition_key` | `"as_index_pk"` | GSI hash-key DB attribute name (Mode B) |
|
|
90
|
+
| `index_sort_key` | `"as_index_sk"` | GSI range-key DB attribute name (Mode B) |
|
|
91
|
+
| `client` | `nil` | explicit `Aws::DynamoDB::Client` (else built from `client_options`) |
|
|
92
|
+
| `client_options` | `{}` | forwarded to `Aws::DynamoDB::Client.new` (`:region`, `:endpoint`, creds…) |
|
|
93
|
+
| `manage_table` | `false` (`true` in dev/test) | create the table (and GSI) if missing |
|
|
94
|
+
|
|
95
|
+
## Storage modes & schema auto-detection
|
|
96
|
+
|
|
97
|
+
The gem adapts to the table's existing key schema, detected **once at boot**
|
|
98
|
+
(`ActiveStorage::AwsRecord.discover_schema!`) via `describe_table(table_name)`:
|
|
99
|
+
|
|
100
|
+
- partition key = `key_schema` entry with `key_type == "HASH"` → its `attribute_name`,
|
|
101
|
+
- sort key = entry with `key_type == "RANGE"` → its `attribute_name`,
|
|
102
|
+
- types from `attribute_definitions`.
|
|
103
|
+
|
|
104
|
+
A `partition_key`/`sort_key` set in config overrides discovery. Discovery runs
|
|
105
|
+
once at boot and is cached in module state (read-only thereafter → fiber-safe).
|
|
106
|
+
|
|
107
|
+
The **range key's type** selects one of two physical layouts. The *logical* key
|
|
108
|
+
scheme is identical in both; only the mapping onto physical attributes differs.
|
|
109
|
+
The **partition (hash) key must be type `S` in both modes** (it stores
|
|
110
|
+
string keys); a numeric hash key, or a table with no range key, raises a clear
|
|
111
|
+
`ConfigurationError`.
|
|
112
|
+
|
|
113
|
+
### Logical keys (per entity)
|
|
114
|
+
|
|
115
|
+
Every item computes the same three strings — `H` (partition), `R` (sort), and a
|
|
116
|
+
globally-unique `item_id`. Let `ns` = `namespace` (default `ActiveStorage`),
|
|
117
|
+
`sep` = `separator` (`#`).
|
|
118
|
+
|
|
119
|
+
| Entity | `H` | `R` | `item_id` |
|
|
120
|
+
|---|---|---|---|
|
|
121
|
+
| **Blob** | `ns#Blob#<blob_id>` | `ns#Blob#<blob_id>` | `ns#Blob#<blob_id>` |
|
|
122
|
+
| **VariantRecord** | `ns#Blob#<blob_id>` | `ns#VariantRecord#<digest>` | `ns#Blob#<blob_id>#VariantRecord#<digest>` |
|
|
123
|
+
| **Attachment** | `ns#Owner#<record_type>#<record_id>` | `ns#Attachment#<name>#<attachment_id>` | `ns#Attachment#<attachment_id>` |
|
|
124
|
+
|
|
125
|
+
Keys are built with `AwsRecord.key(*parts)` = `parts.join(separator)` after
|
|
126
|
+
validating each part is non-blank.
|
|
127
|
+
|
|
128
|
+
### Mode A — string range key → base-table adjacency, no GSI
|
|
129
|
+
|
|
130
|
+
- Physical partition attr ← `H`, physical sort attr ← `R`.
|
|
131
|
+
- A blob and all its variant records **share one partition** (`ns#Blob#<id>`) —
|
|
132
|
+
a textbook single-table *item collection*, fetched in one Query.
|
|
133
|
+
- Attachments are **grouped under their owner** (`ns#Owner#<type>#<id>`), because
|
|
134
|
+
the contract's hot path is *owner → its attachments*.
|
|
135
|
+
- Every read is on the base table → **strongly consistent**.
|
|
136
|
+
|
|
137
|
+
### Mode B — numeric range key → string-keyed GSI adjacency
|
|
138
|
+
|
|
139
|
+
- Physical partition attr ← `item_id`, physical sort attr ← `0` (the required
|
|
140
|
+
numeric range value; a constant).
|
|
141
|
+
- A GSI (`index_partition_key`/`index_sort_key`, both string, projection ALL)
|
|
142
|
+
carries the adjacency pair: `index_partition ← H`, `index_sort ← R`.
|
|
143
|
+
- **Point lookups** (`Blob.find`, variant `find_by`) → base-table
|
|
144
|
+
`GetItem(item_id, 0)` → strong.
|
|
145
|
+
- **Listing** (owner → attachments, variant sweep) → GSI Query → eventually
|
|
146
|
+
consistent. Within a single request this is masked by Active Storage's
|
|
147
|
+
in-memory change tracking.
|
|
148
|
+
- **Writes / refcount / FK-guard** → always base-table by `(item_id, 0)` →
|
|
149
|
+
strong, so the shared-blob integrity guard is identical to Mode A.
|
|
150
|
+
- **GSI provisioning:** at boot, if the named GSI exists → use it; if missing
|
|
151
|
+
and `manage_table` (dev/test) → create it (at table-create time or via
|
|
152
|
+
`UpdateTable`); if missing in production → raise `ConfigurationError` with the
|
|
153
|
+
exact GSI spec to add. The gem will not silently mutate a production table's
|
|
154
|
+
indexes.
|
|
155
|
+
|
|
156
|
+
The Persistence concern and Relation read the detected mode to route
|
|
157
|
+
key-stamping and queries; all other behavior (blob/attachment/variant logic,
|
|
158
|
+
callbacks, refcount) is mode-agnostic.
|
|
159
|
+
|
|
160
|
+
### Access patterns
|
|
161
|
+
|
|
162
|
+
| Contract need | DynamoDB op | Key | Routing |
|
|
163
|
+
|---|---|---|---|
|
|
164
|
+
| `Blob.find(id)` | GetItem | blob item | base table, strong (both modes) |
|
|
165
|
+
| `Blob#destroy` FK guard | conditional DeleteItem | blob item, `attachments_count = 0 OR not_exists` | base table, strong |
|
|
166
|
+
| variant `find_by(digest)` | GetItem | variant item | base table, strong |
|
|
167
|
+
| `Blob#destroy` variant sweep | Query | `H=ns#Blob#blob_id`, `begins_with(R, "ns#VariantRecord#")` | base (A) / GSI (B) |
|
|
168
|
+
| owner → one named attachment | Query (limit 1) | `H=ns#Owner#type#id`, `begins_with(R, "ns#Attachment#name#")` | base (A) / GSI (B) |
|
|
169
|
+
| owner → all of a name | Query | same `begins_with` | base (A) / GSI (B) |
|
|
170
|
+
| owner → all attachments (destroy) | Query | `H=ns#Owner#type#id`, `begins_with(R, "ns#Attachment#")` | base (A) / GSI (B) |
|
|
171
|
+
| attachment → blob | GetItem | `blob_id` stored on the attachment item | base table, strong |
|
|
172
|
+
| refcount on attach/detach | `transact_write_items` | Put/Delete attachment **+** Update blob `ADD attachments_count :n` | base table, strong |
|
|
173
|
+
|
|
174
|
+
`record_type` may contain `::` but never `#`; ids/uuids never contain `#`; so
|
|
175
|
+
every `begins_with` is unambiguous.
|
|
176
|
+
|
|
177
|
+
### Why no blob→attachment index is needed
|
|
178
|
+
|
|
179
|
+
The only blob→attachment reverse query in the **generic** (non-AR) path is
|
|
180
|
+
`create_one_of_many.rb:14` `blob.attachments.find { … }`, reached **only in the
|
|
181
|
+
`else` branch where `blob.persisted? == false`** (line 11). A non-persisted blob
|
|
182
|
+
has zero attachment rows, so `Blob#attachments` can safely return `[]` there.
|
|
183
|
+
The persisted branch (line 12) goes through `record.{name}_attachments` — an
|
|
184
|
+
*owner* query. The shared-blob "is this still referenced?" question is answered
|
|
185
|
+
by the strongly-consistent `attachments_count` on the blob item, not by a
|
|
186
|
+
reverse query. So **no blob→attachment index is required**: Mode A is fully
|
|
187
|
+
GSI-free, and Mode B's single GSI exists only to provide string-keyed adjacency
|
|
188
|
+
on a numeric-range table, not to serve a reverse lookup. `Blob#attachments` is
|
|
189
|
+
implemented to return an empty, `QueryNotSupported`-on-materialize collection
|
|
190
|
+
(only ever hit pre-persist).
|
|
191
|
+
|
|
192
|
+
## Consistency & integrity model
|
|
193
|
+
|
|
194
|
+
Point lookups, writes, the refcount guard, and the FK-guard are always on the
|
|
195
|
+
base table and **strongly consistent in both modes**. Listing queries are strong
|
|
196
|
+
in Mode A and eventually consistent in Mode B (GSI), where in-request staleness
|
|
197
|
+
is masked by Active Storage's in-memory change tracking.
|
|
198
|
+
|
|
199
|
+
### Shared-blob foreign-key guard
|
|
200
|
+
|
|
201
|
+
The `ActiveStorage::ForeignKeyViolation` guard for shared blobs:
|
|
202
|
+
|
|
203
|
+
- `Attachment#save!` (create) = `transact_write_items` { Put attachment
|
|
204
|
+
(`attribute_not_exists`), Update blob `ADD attachments_count 1` }.
|
|
205
|
+
- `Attachment#destroy` = `transact_write_items` { Delete attachment, Update blob
|
|
206
|
+
`ADD attachments_count -1` }.
|
|
207
|
+
- `Blob#destroy` = conditional DeleteItem (`attachments_count = :zero OR
|
|
208
|
+
attribute_not_exists(attachments_count)`); `ConditionalCheckFailed` ⇒ raise
|
|
209
|
+
`ForeignKeyViolation`. The count is mutated atomically with each attachment
|
|
210
|
+
write, so no concurrent attach is missed ⇒ no wrongful purge.
|
|
211
|
+
|
|
212
|
+
### Integrity & hardening rules
|
|
213
|
+
|
|
214
|
+
- **Count updates can't resurrect a purged blob.** Every transactional blob
|
|
215
|
+
count `Update` carries `condition_expression: attribute_exists(#h)` (aliased),
|
|
216
|
+
so an `ADD` against a purged blob fails the transaction → `RecordNotSaved`,
|
|
217
|
+
never a count-only zombie item.
|
|
218
|
+
- **No double-decrement.** The attachment `Delete` in the destroy transaction
|
|
219
|
+
carries `attribute_exists(#h)`; a duplicate purge fails the transaction (no
|
|
220
|
+
second `ADD -1`). `Attachment#destroy`/`#delete` treat the conditional failure
|
|
221
|
+
on an already-absent row as an idempotent no-op rather than re-decrementing.
|
|
222
|
+
- **`delete` decrements too.** Both `destroy` and `delete` go through the
|
|
223
|
+
transactional refcount path; `delete` only skips `touch`/blob cleanup.
|
|
224
|
+
- **Attachment row identity.** `has_one` replaces (delete-old-then-create), so no
|
|
225
|
+
storage-level uniqueness is required there; `has_many` intentionally permits
|
|
226
|
+
the same blob attached twice (matches the reference `in_memory_backend`). The
|
|
227
|
+
uuid-suffixed `R` is kept; no uniqueness constraint the contract doesn't
|
|
228
|
+
require is invented.
|
|
229
|
+
- **Variant vs. blob-purge race.** `VariantRecord.create_or_find_by!` adds a
|
|
230
|
+
`ConditionCheck attribute_exists` on the blob root in the same
|
|
231
|
+
`transact_write_items` as the conditional variant `Put`, so a variant cannot
|
|
232
|
+
be created against a just-purged blob. `Blob#destroy` deletes the blob root
|
|
233
|
+
(conditional on `count == 0`) **before** sweeping variants.
|
|
234
|
+
- **Attribute name isolation.** Every **non-key** logical attribute is declared
|
|
235
|
+
with an explicit namespaced `database_attribute_name` (`as_blob_id`,
|
|
236
|
+
`as_filename`, `as_record_type`, …) so it can never clash with a detected key
|
|
237
|
+
attribute named `id`/`blob_id`/`sk`/etc. Key attrs use the detected DB names.
|
|
238
|
+
- **Key types.** Discovery requires partition `S` (both modes); range `S`
|
|
239
|
+
selects Mode A, range `N` selects Mode B; anything else (numeric partition, no
|
|
240
|
+
range key) → `ConfigurationError`.
|
|
241
|
+
- **`Blob#attachments` on a persisted blob** is not supported without a
|
|
242
|
+
blob-keyed index. The generic contract never calls it there (only the
|
|
243
|
+
non-persisted dedup branch, which returns `[]`); materializing it on a
|
|
244
|
+
persisted blob raises `QueryNotSupported` — a documented limitation.
|
|
245
|
+
- **Namespace/separator validation.** Reject a blank `namespace`/`separator`, and
|
|
246
|
+
validate that `record_type`/`record_id`/`name` contain no `separator` before
|
|
247
|
+
building keys → no `begins_with` bleed.
|
|
248
|
+
|
|
249
|
+
## Persistence concern (`Persistence`) — shared key plumbing
|
|
250
|
+
|
|
251
|
+
`include Aws::Record` + `GlobalID::Identification`. Declares the **two key
|
|
252
|
+
attributes once**, with auto-detected/overridden DB names, applied at
|
|
253
|
+
`install!` (after schema discovery), not in the class body:
|
|
254
|
+
|
|
255
|
+
```ruby
|
|
256
|
+
model.set_table_name(config.table_name)
|
|
257
|
+
model.string_attr :dynamo_partition_key, hash_key: true, database_attribute_name: schema.partition_key
|
|
258
|
+
model.string_attr :dynamo_range_key, range_key: true, database_attribute_name: schema.sort_key
|
|
259
|
+
```
|
|
260
|
+
|
|
261
|
+
In Mode A these map to the detected `H`/`R` attributes; in Mode B the base-table
|
|
262
|
+
keys map to `item_id` and the numeric constant, while `H`/`R` are stamped onto
|
|
263
|
+
the GSI attributes. Internal Ruby accessors `dynamo_partition_key` /
|
|
264
|
+
`dynamo_range_key` are gem-private (never collide with entity attrs). Each entity
|
|
265
|
+
stamps its key attributes from its own logical id(s) before every write.
|
|
266
|
+
|
|
267
|
+
The concern also provides: `read_attribute`/`write_attribute` via `@data`;
|
|
268
|
+
`==`/`eql?`/`hash` by class + logical id; `changed?` → aws-record `dirty?`;
|
|
269
|
+
`dynamodb_client` delegation; persist-state helpers (`mark_persisted!`/
|
|
270
|
+
`mark_destroyed!` via `@data`). `find(id)` is **not** in the concern (each entity
|
|
271
|
+
composes its own key).
|
|
272
|
+
|
|
273
|
+
## Contract method mapping
|
|
274
|
+
|
|
275
|
+
### `Blob`
|
|
276
|
+
|
|
277
|
+
Mirrors the in-memory reference Blob; includes only `Servable`.
|
|
278
|
+
|
|
279
|
+
- `find(id)` → GetItem on the blob item; raise `RecordNotFound` on nil.
|
|
280
|
+
- stamps its key attributes from `id` before save.
|
|
281
|
+
- `attachments` → empty collection (see *Why no blob→attachment index is
|
|
282
|
+
needed*).
|
|
283
|
+
- `destroy` FK guard = conditional DeleteItem by key; variant sweep =
|
|
284
|
+
`VariantRecord.where_blob(id)`.
|
|
285
|
+
- `map_attr :metadata, default_value: -> { {} }` (fiber-safe lambda default).
|
|
286
|
+
|
|
287
|
+
### `Attachment`
|
|
288
|
+
|
|
289
|
+
- `H = key(ns, "Owner", record_type, record_id)`,
|
|
290
|
+
`R = key(ns, "Attachment", name, id)`, `item_id = key(ns, "Attachment", id)`.
|
|
291
|
+
- `transactional_create!`/`transactional_destroy!` use `transact_write_items`
|
|
292
|
+
with the blob's key for the `ADD` update; expression-aliased.
|
|
293
|
+
- `where`/`find_by` via `Relation` (owner query).
|
|
294
|
+
- Both `destroy` and `delete` decrement the refcount through the transactional
|
|
295
|
+
path; `delete` simply skips `touch`/blob cleanup. Replace/detach paths stay
|
|
296
|
+
correct.
|
|
297
|
+
|
|
298
|
+
### `VariantRecord`
|
|
299
|
+
|
|
300
|
+
- `H = key(ns, "Blob", blob_id)`, `R = key(ns, "VariantRecord", digest)`,
|
|
301
|
+
`item_id = key(ns, "Blob", blob_id, "VariantRecord", digest)`.
|
|
302
|
+
- `find(id)` decodes the reversible Base64 id → `(blob_id, digest)` → GetItem.
|
|
303
|
+
- `create_or_find_by!` = conditional Put (`attribute_not_exists`) +
|
|
304
|
+
blob-existence `ConditionCheck` + find-on-conflict.
|
|
305
|
+
- `where_blob(blob_id)` = Query `H=ns#Blob#blob_id`,
|
|
306
|
+
`begins_with(R, "ns#VariantRecord#")`.
|
|
307
|
+
- Itself an Owner (`has_one_attached :image`); its image attachment lands under
|
|
308
|
+
`ns#Owner#<VariantRecordClass>#<encoded_id>` (encoded id is `#`-free Base64).
|
|
309
|
+
|
|
310
|
+
### `Owner`
|
|
311
|
+
|
|
312
|
+
Callback layer over aws-record save/destroy/commit/rollback.
|
|
313
|
+
|
|
314
|
+
### `Relation`
|
|
315
|
+
|
|
316
|
+
`owner_query` (`begins_with` by name) is the sole listing path — strong in Mode
|
|
317
|
+
A, GSI-routed in Mode B. Unsupported filters raise `QueryNotSupported`. Key
|
|
318
|
+
building uses `Attachment` key helpers, not a raw delimiter constant.
|
|
319
|
+
|
|
320
|
+
## Module (`ActiveStorage::AwsRecord`) — fiber-safe client repository + discovery
|
|
321
|
+
|
|
322
|
+
```ruby
|
|
323
|
+
@client_mutex = Mutex.new # eager, at load (fiber/thread-safe init)
|
|
324
|
+
@config = Configuration.new
|
|
325
|
+
|
|
326
|
+
def self.dynamodb_client
|
|
327
|
+
@client_mutex.synchronize { @dynamodb_client ||= config.client || Aws::DynamoDB::Client.new(config.client_options) }
|
|
328
|
+
end
|
|
329
|
+
|
|
330
|
+
def self.key(*parts)
|
|
331
|
+
parts.each { raise ArgumentError, 'blank key segment' if it.nil? || it.to_s.empty? }
|
|
332
|
+
parts.join(config.separator)
|
|
333
|
+
end
|
|
334
|
+
```
|
|
335
|
+
|
|
336
|
+
`install!`: discover schema (`discover_schema!`), declare key attrs + table name
|
|
337
|
+
on `Blob`/`Attachment`/`VariantRecord` (idempotent — skip if `hash_key` already
|
|
338
|
+
set), point them at the shared client. `install_attachments!`:
|
|
339
|
+
`Blob has_one_attached :preview_image`, `VariantRecord has_one_attached :image`.
|
|
340
|
+
|
|
341
|
+
## Railtie / configuration / Services
|
|
342
|
+
|
|
343
|
+
- before `active_storage.class_indirection`: set
|
|
344
|
+
`blob/attachment/variant_record_class` + map
|
|
345
|
+
`config.activestorage_aws_record.*` → `Configuration` (incl. `table_name`,
|
|
346
|
+
`namespace`, `separator`, `partition_key`, `sort_key`, `index_*`,
|
|
347
|
+
`client(_options)`).
|
|
348
|
+
- `config.after_initialize`: `install!` (discovers schema, declares key attrs),
|
|
349
|
+
`Services.setup_from_app_config(app)`, `install_attachments!`.
|
|
350
|
+
|
|
351
|
+
## Table management (dev/test only; production tables are app-managed)
|
|
352
|
+
|
|
353
|
+
`Tables.create!`/`delete!`/`exist?` for **one** table: `{ H: S, R: S }`,
|
|
354
|
+
`PAY_PER_REQUEST`. Default key names `pk`/`sk` when the gem creates it; in Mode B
|
|
355
|
+
dev/test it also provisions the `active_storage_index` GSI. Production apps point
|
|
356
|
+
`table_name` at an existing table and the gem auto-detects whatever the key
|
|
357
|
+
attributes are called. Migrations are additive-only.
|
|
358
|
+
|
|
359
|
+
## Testing (DynamoDB Local via docker-compose)
|
|
360
|
+
|
|
361
|
+
`dynamo_setup` boots a minimal Rails app (no Active Record) + this gem against
|
|
362
|
+
DynamoDB Local; creates **one** disposable table per pid; `Disk` service for
|
|
363
|
+
bytes.
|
|
364
|
+
|
|
365
|
+
**Behavior suite:** attach/replace/detach/purge (sync + later), has_one/has_many,
|
|
366
|
+
direct upload + protected-metadata filtering, analyze (immediate/later/lazy),
|
|
367
|
+
variant tracking, owner destroy + dependent purge, abort/return-false destroy
|
|
368
|
+
keeps rows, never-saved colliding-id owner leaves victim intact, shared blob
|
|
369
|
+
purged once, signed-id round-trip, GlobalID job round-trip, concurrent
|
|
370
|
+
`create_or_find_by!` variant uniqueness.
|
|
371
|
+
|
|
372
|
+
**Schema / mode coverage:**
|
|
373
|
+
- (a) auto-detection picks up a table with non-default, string-range key
|
|
374
|
+
attribute names (e.g. `hash_key`/`range_key`) → Mode A.
|
|
375
|
+
- (b) a numeric-range table → Mode B: the GSI is auto-configured and listing
|
|
376
|
+
queries route through it.
|
|
377
|
+
- (c) a numeric *partition* key (or a table with no range key) → `ConfigurationError`.
|
|
378
|
+
- (d) all three entity types coexist in one table without key collision.
|
|
379
|
+
- (e) `Attachment#delete` decrements the refcount.
|
|
380
|
+
|
|
381
|
+
## Key design decisions
|
|
382
|
+
|
|
383
|
+
1. **One table, mode-selected layout.** String range key → base-table adjacency,
|
|
384
|
+
no GSI; numeric range key → string-keyed GSI adjacency. Both keep strong
|
|
385
|
+
refcount/FK-guard on the base table.
|
|
386
|
+
2. **The only reverse lookup is on a non-persisted blob** (returns `[]`);
|
|
387
|
+
refcount replaces blob→attachment counting, so no reverse index is needed.
|
|
388
|
+
3. **Auto-detect keys** via `describe_table` at boot, with config override;
|
|
389
|
+
partition key must be `S`, range key `S` or `N` selects the mode.
|
|
390
|
+
4. **Co-locate blob + variants**, group attachments under owner — idiomatic
|
|
391
|
+
single-table item collections that serve every access pattern.
|
|
392
|
+
5. **`#`-join with non-blank validation**, not a host-app `compose_key`.
|
|
393
|
+
6. **Fiber safety** — eager mutex, mutex-guarded client repo, lambda map default,
|
|
394
|
+
read-only post-boot schema cache.
|
|
395
|
+
7. **Style** — no `frozen_string_literal`, Ruby 3.4 idioms, YARD, expression
|
|
396
|
+
aliasing, never SCAN a request path.
|
|
397
|
+
8. **Grouped destroys are atomic; creates stay per-row.** Each attachment
|
|
398
|
+
*create* is its own 2-item transaction (attachment + blob refcount), so the
|
|
399
|
+
generic create paths keep their synchronous failed-save cleanup. The generic
|
|
400
|
+
`has_many` clear/replace/detach and `Relation#delete_all` paths wrap their
|
|
401
|
+
per-row destroys in `Attachment.transaction`, a fiber-local accumulator that
|
|
402
|
+
commits all the buffered deletes — and one *coalesced* `ADD` per distinct blob
|
|
403
|
+
— in a single `transact_write_items`, so a multi-attachment change is atomic
|
|
404
|
+
instead of deleting some rows before a later one fails. A change exceeding
|
|
405
|
+
DynamoDB's 100-action transaction limit **fails closed** before any write
|
|
406
|
+
(`TransactionTooLarge`) rather than chunking, which would reintroduce the
|
|
407
|
+
partial-clear bug. A single buffered destroy still uses the per-row path, so
|
|
408
|
+
its idempotent duplicate-purge / orphaned-blob recovery is preserved.
|
|
409
|
+
*Mixed has_one replace* (a synchronous create plus one buffered orphan delete)
|
|
410
|
+
is only fully atomic when the host Active Storage carries the widened
|
|
411
|
+
`CreateOne#save` rescue (it wraps the whole `attachment_class.transaction`, so
|
|
412
|
+
a commit-time delete failure rolls back the new record) — part of the same
|
|
413
|
+
`activestorage-backends` work this gem targets.
|