active_cipher_storage 1.0.3 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +27 -6
- data/CONTRIBUTING.md +0 -1
- data/README.md +101 -123
- data/lib/active_cipher_storage/adapters/s3_adapter.rb +11 -7
- data/lib/active_cipher_storage/blob_metadata.rb +16 -47
- data/lib/active_cipher_storage/configuration.rb +59 -26
- data/lib/active_cipher_storage/engine.rb +1 -17
- data/lib/active_cipher_storage/errors.rb +2 -2
- data/lib/active_cipher_storage/multipart_upload.rb +12 -10
- data/lib/active_cipher_storage/providers/aws_kms_provider.rb +22 -38
- data/lib/active_cipher_storage/providers/base.rb +2 -13
- data/lib/active_cipher_storage/providers/env_provider.rb +11 -34
- data/lib/active_cipher_storage/stream_cipher.rb +5 -2
- data/lib/active_cipher_storage/version.rb +1 -1
- data/lib/active_cipher_storage.rb +0 -2
- data/lib/active_storage/service/active_cipher_storage_service.rb +181 -4
- metadata +2 -4
- data/lib/active_cipher_storage/adapters/active_storage_service.rb +0 -140
- data/lib/active_cipher_storage/key_rotation.rb +0 -121
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: a7be8276c5d35fdb04507dc28393359978c61f96c02d34e68241e7850f890d03
|
|
4
|
+
data.tar.gz: d2989e928d470c23443ec09e9d09a731a240fe177c38e18ee80c5439a94d4cc0
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 201051afeef8762eb3f8562224de9f0b76f74acf61a51b195d2a8eaf458d78cf716c8b1fa99c662479113061140bd4d6867ad25400dd6f7248ccd9df51375d5f
|
|
7
|
+
data.tar.gz: b919b79931e8f8162646e7e291909c6e6ccc35d7681b4a42788d028e206078154cd00ec05cf73419ffd7d80b598dbf1b203b117f11d05219a63e346988cbc13d
|
data/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,31 @@ and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.ht
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [2.0.0] - 2026-06-01
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
|
|
14
|
+
- **`Configuration#provider_options`**: keyword options for built-in providers are forwarded to **`EnvProvider.new`** / **`AwsKmsProvider.new`** (`Hash`/`OrderedOptions`, no separate `aws_kms` / `env_provider` accessors).
|
|
15
|
+
- **Breaking:** **`EnvProvider`** now accepts **`encryption_key:`** directly; pass **`provider_options[:encryption_key] = ENV.fetch("ACTIVE_CIPHER_MASTER_KEY")`** instead of configuring an env-var name.
|
|
16
|
+
- Provider **`String`** aliases **`"aws:kms"`**, **`"env"`**, and related spellings (see **`Configuration`**).
|
|
17
|
+
- **`AwsKmsProvider`** accepts **`endpoint`**, **`access_key_id`**, **`secret_access_key`**, builds **`Aws::KMS::Client`** internally; **`key_id:`** is required (configure via **`provider_options`** or pass a custom instance).
|
|
18
|
+
|
|
19
|
+
### Changed
|
|
20
|
+
|
|
21
|
+
- **Breaking:** Global **`Configuration#chunk_size`** removed — pass **`chunk_size`** into **`StreamCipher`**, **`S3Adapter`**, **`EncryptedMultipartUpload`**, and the **`ActiveCipherStorage`** Active Storage service (`storage.yml`).
|
|
22
|
+
- **Breaking:** Built-in provider config is **`provider_options`** only (removed **`#aws_kms`** / **`#env_provider`**). **`AwsKmsProvider`** no longer reads **`ENV`** for KMS settings; set **`provider_options`** from your app.
|
|
23
|
+
- **Blob metadata:** Rescue **`StandardError`** only; re-raise in **`Rails.env.development?`** so misconfiguration surfaces during development.
|
|
24
|
+
- **Engine:** Remove global **`ActiveSupport::LogSubscriber.logger`** assignment (host apps use **`Rails.logger`** / **`ActiveStorage.logger`**).
|
|
25
|
+
- **Engine:** Load **`ActiveStorage::Service::ActiveCipherStorageService`** directly from the Rails Active Storage hook.
|
|
26
|
+
- **`ActiveCipherStorageService`:** Raise **`NotImplementedError`** for **`path_for`** when the inner service does not implement it (e.g. S3).
|
|
27
|
+
|
|
28
|
+
### Removed
|
|
29
|
+
|
|
30
|
+
- **`ActiveCipherStorage::KeyRotation`** and related rotation orchestration.
|
|
31
|
+
- **Breaking:** Legacy **`ActiveCipherStorage::Adapters::ActiveStorageService`** alias and **`active_cipher_storage/active_storage_integration`** shim.
|
|
32
|
+
- **`ActiveCipherStorageService#rekey`**, **`BlobMetadata.blobs_for`**, **`BlobMetadata.update_after_rotation`**.
|
|
33
|
+
- Provider methods **`wrap_data_key`** and **`rotate_data_key`** from **`Providers::Base`**, **`EnvProvider`**, and **`AwsKmsProvider`**. Key or provider changes are left to the application (e.g. AWS KMS, custom jobs).
|
|
34
|
+
|
|
10
35
|
## [1.0.3] - 2026-04-25
|
|
11
36
|
|
|
12
37
|
### Changed
|
|
@@ -25,11 +50,6 @@ and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.ht
|
|
|
25
50
|
|
|
26
51
|
- Back gem configuration with Rails-style ActiveSupport options while preserving the existing public configuration API.
|
|
27
52
|
- Document the Active Storage upload encryption flag and plaintext read compatibility behavior.
|
|
28
|
-
|
|
29
|
-
### Fixed
|
|
30
|
-
|
|
31
|
-
- Reject reordered streaming frames and trailing bytes after the final encrypted frame.
|
|
32
|
-
- Validate S3 multipart chunk sizes before upload so invalid part sizes fail early.
|
|
33
53
|
- Mark plaintext Active Storage uploads explicitly when encryption is disabled.
|
|
34
54
|
|
|
35
55
|
## [1.0.0] - 2026-04-25
|
|
@@ -45,7 +65,8 @@ and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.ht
|
|
|
45
65
|
- Header-only key rotation for re-wrapping encrypted DEKs.
|
|
46
66
|
- Unit and integration coverage for crypto, providers, Active Storage, S3, multipart upload, streaming, metadata, and key rotation.
|
|
47
67
|
|
|
48
|
-
[Unreleased]: https://github.com/codebyjass/active-cipher-storage/compare/
|
|
68
|
+
[Unreleased]: https://github.com/codebyjass/active-cipher-storage/compare/v2.0.0...HEAD
|
|
69
|
+
[2.0.0]: https://github.com/codebyjass/active-cipher-storage/compare/v1.0.3...v2.0.0
|
|
49
70
|
[1.0.3]: https://github.com/codebyjass/active-cipher-storage/compare/v1.0.2...v1.0.3
|
|
50
71
|
[1.0.2]: https://github.com/codebyjass/active-cipher-storage/compare/v1.0.1...v1.0.2
|
|
51
72
|
[1.0.1]: https://github.com/codebyjass/active-cipher-storage/compare/v1.0.0...v1.0.1
|
data/CONTRIBUTING.md
CHANGED
|
@@ -33,7 +33,6 @@ Use focused tests for:
|
|
|
33
33
|
- Active Storage legacy plaintext fallback.
|
|
34
34
|
- S3 multipart and streaming behavior.
|
|
35
35
|
- Provider error handling.
|
|
36
|
-
- Key rotation behavior.
|
|
37
36
|
|
|
38
37
|
Security-sensitive fixes should include a regression test that fails without the fix.
|
|
39
38
|
|
data/README.md
CHANGED
|
@@ -19,7 +19,6 @@ It works with normal Rails Active Storage attachments, direct S3 uploads from Ru
|
|
|
19
19
|
- Handles large files with streaming AES-256-GCM encryption.
|
|
20
20
|
- Supports backend-managed multipart uploads for frontend chunk upload flows.
|
|
21
21
|
- Uses pluggable key providers: environment variables, AWS KMS, or custom KMS providers.
|
|
22
|
-
- Supports header-only key rotation without rewriting the full file body.
|
|
23
22
|
|
|
24
23
|
## Use Cases
|
|
25
24
|
|
|
@@ -43,18 +42,17 @@ It works with normal Rails Active Storage attachments, direct S3 uploads from Ru
|
|
|
43
42
|
9. [Manual encrypt / decrypt](#manual-encrypt--decrypt)
|
|
44
43
|
10. [Blob metadata](#blob-metadata)
|
|
45
44
|
11. [KMS providers](#kms-providers)
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
12. [
|
|
50
|
-
13. [
|
|
51
|
-
14. [
|
|
52
|
-
15. [
|
|
53
|
-
16. [
|
|
54
|
-
17. [
|
|
55
|
-
18. [
|
|
56
|
-
19. [
|
|
57
|
-
20. [Ruby and Rails compatibility](#ruby-and-rails-compatibility)
|
|
45
|
+
- [Environment-variable provider](#environment-variable-provider)
|
|
46
|
+
- [AWS KMS provider](#aws-kms-provider)
|
|
47
|
+
- [Custom provider](#custom-provider)
|
|
48
|
+
12. [Configuration reference](#configuration-reference)
|
|
49
|
+
13. [Encryption format](#encryption-format)
|
|
50
|
+
14. [Security notes](#security-notes)
|
|
51
|
+
15. [Testing](#testing)
|
|
52
|
+
16. [Contributing](#contributing)
|
|
53
|
+
17. [Security reports](#security-reports)
|
|
54
|
+
18. [License](#license)
|
|
55
|
+
19. [Ruby and Rails compatibility](#ruby-and-rails-compatibility)
|
|
58
56
|
|
|
59
57
|
## How it works
|
|
60
58
|
|
|
@@ -101,23 +99,27 @@ Your model, controller, and view code can keep using normal Active Storage APIs.
|
|
|
101
99
|
```ruby
|
|
102
100
|
# config/initializers/active_cipher_storage.rb
|
|
103
101
|
ActiveCipherStorage.configure do |config|
|
|
104
|
-
#
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
102
|
+
# Built-ins: :env, :aws_kms, "env", or "aws:kms". Custom: pass a Providers::Base instance.
|
|
103
|
+
config.provider = :env
|
|
104
|
+
config.provider_options[:encryption_key] = ENV.fetch("ACTIVE_CIPHER_MASTER_KEY")
|
|
105
|
+
# KMS-only apps often use:
|
|
106
|
+
# config.provider = "aws:kms"
|
|
107
|
+
|
|
108
|
+
# Provider-specific keyword args (see EnvProvider / AwsKmsProvider).
|
|
109
|
+
# config.provider_options[:key_id] = Rails.application.credentials.dig(:aws, :kms_key_id)
|
|
110
|
+
# config.provider_options[:region] = "us-east-1"
|
|
111
|
+
# config.provider_options[:endpoint] = "http://localhost:4566" # e.g. LocalStack
|
|
112
|
+
# config.provider_options[:access_key_id] = "..."
|
|
113
|
+
# config.provider_options[:secret_access_key] = "..."
|
|
114
|
+
# config.provider_options[:encryption_context] = { "app" => "my-app" }
|
|
114
115
|
|
|
115
116
|
# Tuning (optional)
|
|
116
|
-
config.chunk_size = 5 * 1024 * 1024 # 5 MiB per chunk (default)
|
|
117
117
|
config.encrypt_uploads = true # set false to store new Active Storage uploads as plaintext
|
|
118
118
|
end
|
|
119
119
|
```
|
|
120
120
|
|
|
121
|
+
Streaming / multipart plaintext chunk size is **not** global: set `chunk_size` on the **`ActiveCipherStorage`** service in `storage.yml`, and pass `chunk_size` / `multipart_threshold` into **`S3Adapter`** / **`EncryptedMultipartUpload`** when you construct them (default **5 MiB**, `ActiveCipherStorage::Configuration::DEFAULT_CHUNK_SIZE`).
|
|
122
|
+
|
|
121
123
|
Generate a master key for local development:
|
|
122
124
|
|
|
123
125
|
```bash
|
|
@@ -137,8 +139,9 @@ ACTIVE_CIPHER_MASTER_KEY=<base64-encoded-key>
|
|
|
137
139
|
# config/storage.yml
|
|
138
140
|
|
|
139
141
|
encrypted_s3:
|
|
140
|
-
service: ActiveCipherStorage #
|
|
142
|
+
service: ActiveCipherStorage # resolves to ActiveStorage::Service::ActiveCipherStorageService
|
|
141
143
|
wrapped_service: s3 # name of another service in this file
|
|
144
|
+
chunk_size: 6291456 # plaintext bytes per stream chunk (>= 5 MiB for S3 multipart)
|
|
142
145
|
|
|
143
146
|
s3:
|
|
144
147
|
service: S3
|
|
@@ -181,12 +184,14 @@ This is useful for background jobs, service objects, scripts, or non-Rails Ruby
|
|
|
181
184
|
require "active_cipher_storage"
|
|
182
185
|
|
|
183
186
|
ActiveCipherStorage.configure do |c|
|
|
184
|
-
c.provider =
|
|
187
|
+
c.provider = :env
|
|
188
|
+
c.provider_options[:encryption_key] = ENV.fetch("ACTIVE_CIPHER_MASTER_KEY")
|
|
185
189
|
end
|
|
186
190
|
|
|
187
191
|
s3 = ActiveCipherStorage::Adapters::S3Adapter.new(
|
|
188
|
-
bucket:
|
|
189
|
-
region:
|
|
192
|
+
bucket: "my-bucket",
|
|
193
|
+
region: "us-east-1",
|
|
194
|
+
chunk_size: ActiveCipherStorage::Configuration::DEFAULT_CHUNK_SIZE
|
|
190
195
|
)
|
|
191
196
|
|
|
192
197
|
# Encrypt before upload
|
|
@@ -204,7 +209,8 @@ Large files are automatically uploaded via S3 multipart when the payload exceeds
|
|
|
204
209
|
```ruby
|
|
205
210
|
s3 = ActiveCipherStorage::Adapters::S3Adapter.new(
|
|
206
211
|
bucket: "my-bucket",
|
|
207
|
-
multipart_threshold: 50 * 1024 * 1024 # 50 MiB
|
|
212
|
+
multipart_threshold: 50 * 1024 * 1024, # 50 MiB
|
|
213
|
+
chunk_size: 5 * 1024 * 1024
|
|
208
214
|
)
|
|
209
215
|
```
|
|
210
216
|
|
|
@@ -216,10 +222,13 @@ Active Cipher Storage supports that flow, but the browser still does not get enc
|
|
|
216
222
|
|
|
217
223
|
Use `EncryptedMultipartUpload` for this backend-managed upload flow.
|
|
218
224
|
|
|
225
|
+
**Memory behavior:** the app does not assemble the whole file before uploading. Each `upload_part` call reads the incoming frontend chunk, encrypts that chunk into an authenticated ACS frame, and flushes encrypted bytes to S3 multipart upload parts as soon as the S3 minimum part size is reached. Keep frontend chunks bounded (for example 256 KiB to 5 MiB); a single frontend chunk is read into memory for that request.
|
|
226
|
+
|
|
219
227
|
```ruby
|
|
220
228
|
uploader = ActiveCipherStorage::EncryptedMultipartUpload.new(
|
|
221
|
-
s3_client:
|
|
222
|
-
bucket:
|
|
229
|
+
s3_client: Aws::S3::Client.new(region: "us-east-1"),
|
|
230
|
+
bucket: "my-bucket",
|
|
231
|
+
chunk_size: 5 * 1024 * 1024
|
|
223
232
|
)
|
|
224
233
|
|
|
225
234
|
# Request 1: start the upload
|
|
@@ -263,8 +272,9 @@ class UploadsController < ApplicationController
|
|
|
263
272
|
|
|
264
273
|
def set_uploader
|
|
265
274
|
@uploader = ActiveCipherStorage::EncryptedMultipartUpload.new(
|
|
266
|
-
s3_client:
|
|
267
|
-
bucket:
|
|
275
|
+
s3_client: s3_client,
|
|
276
|
+
bucket: ENV.fetch("S3_BUCKET"),
|
|
277
|
+
chunk_size: 5 * 1024 * 1024
|
|
268
278
|
)
|
|
269
279
|
end
|
|
270
280
|
end
|
|
@@ -279,9 +289,10 @@ For multi-process deployments where chunks for the same active upload may land o
|
|
|
279
289
|
```ruby
|
|
280
290
|
# Rails.cache backed by Redis allows cross-worker active upload sessions.
|
|
281
291
|
uploader = ActiveCipherStorage::EncryptedMultipartUpload.new(
|
|
282
|
-
s3_client:
|
|
283
|
-
bucket:
|
|
284
|
-
|
|
292
|
+
s3_client: s3_client,
|
|
293
|
+
bucket: "my-bucket",
|
|
294
|
+
chunk_size: 5 * 1024 * 1024,
|
|
295
|
+
store: Rails.cache # any object with read/write/delete
|
|
285
296
|
)
|
|
286
297
|
```
|
|
287
298
|
|
|
@@ -293,10 +304,13 @@ Use `stream_decrypted` when you need to send a large encrypted file to a client
|
|
|
293
304
|
|
|
294
305
|
The adapter reads encrypted bytes from S3, decrypts authenticated chunks as they arrive, and yields plaintext chunks to your block. Memory usage stays bounded by one Active Cipher Storage chunk, which is 5 MiB by default.
|
|
295
306
|
|
|
307
|
+
Use `stream_decrypted` for huge files. `get_decrypted` returns an IO for convenience, but it buffers the encrypted object before decrypting and is intended for small objects or tooling, not large client downloads.
|
|
308
|
+
|
|
296
309
|
```ruby
|
|
297
310
|
s3 = ActiveCipherStorage::Adapters::S3Adapter.new(
|
|
298
|
-
bucket:
|
|
299
|
-
region:
|
|
311
|
+
bucket: "my-bucket",
|
|
312
|
+
region: "us-east-1",
|
|
313
|
+
chunk_size: ActiveCipherStorage::Configuration::DEFAULT_CHUNK_SIZE
|
|
300
314
|
)
|
|
301
315
|
|
|
302
316
|
# Stream directly into a Rails response
|
|
@@ -334,7 +348,8 @@ Use `Cipher` for small files and `StreamCipher` for large files:
|
|
|
334
348
|
require "active_cipher_storage"
|
|
335
349
|
|
|
336
350
|
ActiveCipherStorage.configure do |c|
|
|
337
|
-
c.provider =
|
|
351
|
+
c.provider = :env
|
|
352
|
+
c.provider_options[:encryption_key] = ENV.fetch("ACTIVE_CIPHER_MASTER_KEY")
|
|
338
353
|
end
|
|
339
354
|
|
|
340
355
|
# Small files
|
|
@@ -374,71 +389,74 @@ When using the Rails Active Storage adapter, encryption metadata is automaticall
|
|
|
374
389
|
}
|
|
375
390
|
```
|
|
376
391
|
|
|
377
|
-
This metadata
|
|
392
|
+
This metadata supports:
|
|
378
393
|
|
|
379
|
-
- **Key rotation queries** — find every blob encrypted under a given KMS key without scanning blob bodies
|
|
380
394
|
- **Backward compatibility** — blobs uploaded before encryption was enabled are detected by the absence of the `ACS\x01` magic header and served as raw bytes
|
|
381
|
-
- **Operational auditing** —
|
|
395
|
+
- **Operational auditing** — see which provider and key identifier were used when the blob was stored
|
|
382
396
|
|
|
383
397
|
The binary file header remains the ground truth for decryption; metadata is informational only and a mismatch does not affect correctness.
|
|
384
398
|
|
|
385
|
-
**
|
|
399
|
+
Changing KMS keys or providers for existing blobs is **not** handled by this gem: implement your own migration (for example using AWS KMS `ReEncrypt`, custom jobs, or by reading/writing raw storage keys via your Active Storage service’s **`download_raw`** / **`upload_raw`** if you need byte-level access). The [Encryption format](#encryption-format) section documents the on-disk layout.
|
|
386
400
|
|
|
387
|
-
|
|
388
|
-
svc = ActiveCipherStorage::Adapters::ActiveStorageService.new(wrapped_service: inner)
|
|
401
|
+
## KMS providers
|
|
389
402
|
|
|
390
|
-
|
|
391
|
-
"storage/key/for/blob",
|
|
392
|
-
old_provider: old_provider,
|
|
393
|
-
new_provider: new_provider
|
|
394
|
-
)
|
|
395
|
-
# => { status: :rotated }
|
|
396
|
-
```
|
|
403
|
+
### Environment-variable provider
|
|
397
404
|
|
|
398
|
-
**
|
|
405
|
+
**Configure block (recommended)**
|
|
399
406
|
|
|
400
407
|
```ruby
|
|
401
|
-
ActiveCipherStorage
|
|
402
|
-
|
|
403
|
-
|
|
404
|
-
service: MyEncryptedStorageService.new
|
|
405
|
-
) do |blob, result|
|
|
406
|
-
Rails.logger.info "#{blob.key}: #{result[:status]}"
|
|
408
|
+
ActiveCipherStorage.configure do |config|
|
|
409
|
+
config.provider = :env
|
|
410
|
+
config.provider_options[:encryption_key] = ENV.fetch("ACTIVE_CIPHER_MASTER_KEY")
|
|
407
411
|
end
|
|
408
412
|
```
|
|
409
413
|
|
|
410
|
-
|
|
411
|
-
|
|
412
|
-
## KMS providers
|
|
413
|
-
|
|
414
|
-
### Environment-variable provider
|
|
414
|
+
**Manual constructor** (tests, advanced use)
|
|
415
415
|
|
|
416
416
|
```ruby
|
|
417
|
-
# Default env var: ACTIVE_CIPHER_MASTER_KEY
|
|
418
|
-
provider = ActiveCipherStorage::Providers::EnvProvider.new
|
|
419
|
-
|
|
420
|
-
# Custom env var name
|
|
421
417
|
provider = ActiveCipherStorage::Providers::EnvProvider.new(
|
|
422
|
-
|
|
418
|
+
encryption_key: ENV.fetch("ACTIVE_CIPHER_MASTER_KEY")
|
|
423
419
|
)
|
|
424
420
|
```
|
|
425
421
|
|
|
426
|
-
The master key wraps each per-file DEK with AES-256-GCM.
|
|
422
|
+
The encryption key is a Base64-encoded 32-byte master key. It wraps each per-file DEK with AES-256-GCM. The wrapped DEK is stored in the file header; the plaintext DEK exists only during the encrypt/decrypt operation.
|
|
427
423
|
|
|
428
424
|
### AWS KMS provider
|
|
429
425
|
|
|
426
|
+
The gem builds `Aws::KMS::Client` inside `AwsKmsProvider`. Pass settings via `ActiveCipherStorage.configure` (recommended) or construct the provider directly.
|
|
427
|
+
|
|
428
|
+
**Configure block (recommended)**
|
|
429
|
+
|
|
430
|
+
```ruby
|
|
431
|
+
ActiveCipherStorage.configure do |config|
|
|
432
|
+
config.provider = :aws_kms # or "aws:kms"
|
|
433
|
+
|
|
434
|
+
config.provider_options[:key_id] = "arn:aws:kms:us-east-1:123456789:key/mrk-abc123"
|
|
435
|
+
config.provider_options[:region] = "us-east-1"
|
|
436
|
+
# Optional — LocalStack or static keys when not using the default credential chain:
|
|
437
|
+
# config.provider_options[:endpoint] = "http://127.0.0.1:4566"
|
|
438
|
+
# config.provider_options[:access_key_id] = "test"
|
|
439
|
+
# config.provider_options[:secret_access_key] = "test"
|
|
440
|
+
# config.provider_options[:encryption_context] = { "app" => "my-app", "env" => Rails.env }
|
|
441
|
+
end
|
|
442
|
+
```
|
|
443
|
+
|
|
444
|
+
Set `provider_options` from your app (e.g. `Rails.application.credentials`, `ENV`, etc.). The gem forwards them as keyword arguments to **`AwsKmsProvider`**.
|
|
445
|
+
|
|
446
|
+
**Manual constructor** (tests, advanced use)
|
|
447
|
+
|
|
430
448
|
```ruby
|
|
431
449
|
provider = ActiveCipherStorage::Providers::AwsKmsProvider.new(
|
|
432
450
|
key_id: "arn:aws:kms:us-east-1:123456789:key/mrk-abc123",
|
|
433
451
|
region: "us-east-1",
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
|
|
452
|
+
endpoint: "http://127.0.0.1:4566",
|
|
453
|
+
access_key_id: "test",
|
|
454
|
+
secret_access_key: "test",
|
|
437
455
|
encryption_context: { "app" => "my-app", "env" => Rails.env }
|
|
438
456
|
)
|
|
439
457
|
```
|
|
440
458
|
|
|
441
|
-
|
|
459
|
+
You can still pass `client:` to inject a custom `Aws::KMS::Client` (e.g. in tests).
|
|
442
460
|
|
|
443
461
|
### Custom provider
|
|
444
462
|
|
|
@@ -460,10 +478,6 @@ class MyVaultProvider < ActiveCipherStorage::Providers::Base
|
|
|
460
478
|
vault_client.decrypt(encrypted_key)
|
|
461
479
|
end
|
|
462
480
|
|
|
463
|
-
def wrap_data_key(plaintext_dek)
|
|
464
|
-
vault_client.encrypt(plaintext_dek)
|
|
465
|
-
end
|
|
466
|
-
|
|
467
481
|
private
|
|
468
482
|
|
|
469
483
|
def vault_client
|
|
@@ -478,59 +492,23 @@ end
|
|
|
478
492
|
|
|
479
493
|
The `provider_id` is embedded in every encrypted file. Routing at decrypt time is handled by whichever provider is configured — it is the application's responsibility to configure the right provider for each environment.
|
|
480
494
|
|
|
481
|
-
Implement `rotate_data_key(encrypted_key)` as well if the provider can re-wrap encrypted DEKs without exposing plaintext key material.
|
|
482
|
-
|
|
483
|
-
## Key rotation
|
|
484
|
-
|
|
485
|
-
### AWS KMS automatic rotation
|
|
486
|
-
|
|
487
|
-
Enable automatic key rotation on the CMK in the AWS Console or via CLI. AWS transparently re-wraps all data keys on the next use — no application changes needed.
|
|
488
|
-
|
|
489
|
-
### Cross-key and cross-provider rotation
|
|
490
|
-
|
|
491
|
-
Use `KeyRotation.rotate` (covered in [Blob metadata](#blob-metadata)) to batch re-wrap all blobs under a new key. For AWS KMS → AWS KMS rotations the plaintext DEK never leaves KMS (`ReEncrypt` API). Cross-provider rotations (e.g. `EnvProvider` → `AwsKmsProvider`) briefly hold the plaintext DEK in process memory and zero it immediately after.
|
|
492
|
-
|
|
493
|
-
**Dry-run mode** — validate headers without uploading:
|
|
494
|
-
|
|
495
|
-
```ruby
|
|
496
|
-
ActiveCipherStorage::KeyRotation.rotate(
|
|
497
|
-
old_provider: old_kms,
|
|
498
|
-
new_provider: new_kms,
|
|
499
|
-
service: svc,
|
|
500
|
-
dry_run: true
|
|
501
|
-
) do |blob, result|
|
|
502
|
-
puts "#{blob.key}: #{result[:status]}" # :validated or :failed
|
|
503
|
-
end
|
|
504
|
-
```
|
|
505
|
-
|
|
506
|
-
### Low-level DEK re-wrapping
|
|
507
|
-
|
|
508
|
-
```ruby
|
|
509
|
-
# AWS KMS → AWS KMS (ReEncrypt, no plaintext in memory)
|
|
510
|
-
old_provider = ActiveCipherStorage::Providers::AwsKmsProvider.new(key_id: "arn:...old")
|
|
511
|
-
new_provider = ActiveCipherStorage::Providers::AwsKmsProvider.new(key_id: "arn:...new")
|
|
512
|
-
new_dek = old_provider.rotate_data_key(encrypted_dek, destination_key_id: new_provider.key_id)
|
|
513
|
-
|
|
514
|
-
# EnvProvider → EnvProvider
|
|
515
|
-
old_provider = ActiveCipherStorage::Providers::EnvProvider.new(env_var: "OLD_KEY")
|
|
516
|
-
new_provider = ActiveCipherStorage::Providers::EnvProvider.new(env_var: "NEW_KEY")
|
|
517
|
-
new_dek = new_provider.rotate_data_key(encrypted_dek, old_provider: old_provider)
|
|
518
|
-
```
|
|
519
|
-
|
|
520
495
|
## Configuration reference
|
|
521
496
|
|
|
522
497
|
```ruby
|
|
523
498
|
ActiveCipherStorage.configure do |config|
|
|
524
|
-
# Required.
|
|
499
|
+
# Required. Providers::Base instance, or :env / :aws_kms / "env" / "aws:kms".
|
|
525
500
|
config.provider = :env
|
|
526
501
|
|
|
502
|
+
# Keyword options forwarded to the built-in provider’s `.new` (see KMS providers).
|
|
503
|
+
# config.provider_options[:encryption_key] = ENV.fetch("ACTIVE_CIPHER_MASTER_KEY")
|
|
504
|
+
# config.provider_options[:key_id] = "..."
|
|
505
|
+
# config.provider_options[:region] = "us-east-1"
|
|
506
|
+
# config.provider_options[:endpoint] = nil
|
|
507
|
+
# config.provider_options[:encryption_context] = {}
|
|
508
|
+
|
|
527
509
|
# Encryption algorithm. Currently only "aes-256-gcm" is supported.
|
|
528
510
|
config.algorithm = "aes-256-gcm"
|
|
529
511
|
|
|
530
|
-
# Plaintext bytes per chunk in StreamCipher mode.
|
|
531
|
-
# Must be >= 5 MiB for S3 multipart uploads (except the last part).
|
|
532
|
-
config.chunk_size = 5 * 1024 * 1024
|
|
533
|
-
|
|
534
512
|
# Controls new Active Storage uploads only. Downloads always auto-detect
|
|
535
513
|
# encrypted vs. plaintext payloads by the ACS header.
|
|
536
514
|
config.encrypt_uploads = true
|
|
@@ -605,7 +583,7 @@ Integration tests use in-memory fakes for both Active Storage and S3 — no real
|
|
|
605
583
|
|
|
606
584
|
Contributions are welcome. Please read `CONTRIBUTING.md` before opening a pull request.
|
|
607
585
|
|
|
608
|
-
For changes that affect encryption, streaming, providers,
|
|
586
|
+
For changes that affect encryption, streaming, providers, or storage behavior, include focused specs that prove both the success path and the failure/tamper path. Run the full suite before submitting:
|
|
609
587
|
|
|
610
588
|
```bash
|
|
611
589
|
bundle exec rspec
|
|
@@ -8,13 +8,16 @@ module ActiveCipherStorage
|
|
|
8
8
|
DEFAULT_MULTIPART_THRESHOLD = 100 * 1024 * 1024
|
|
9
9
|
|
|
10
10
|
def initialize(bucket:, region: nil, multipart_threshold: DEFAULT_MULTIPART_THRESHOLD,
|
|
11
|
-
s3_client: nil, config: nil
|
|
11
|
+
s3_client: nil, config: nil,
|
|
12
|
+
chunk_size: Configuration::DEFAULT_CHUNK_SIZE)
|
|
12
13
|
@bucket = bucket
|
|
13
14
|
@region = region
|
|
14
15
|
@multipart_threshold = multipart_threshold
|
|
15
16
|
@client_override = s3_client
|
|
16
17
|
@config = config || ActiveCipherStorage.configuration
|
|
18
|
+
@chunk_size = Integer(chunk_size)
|
|
17
19
|
@config.validate!
|
|
20
|
+
raise ArgumentError, "chunk_size must be positive" unless @chunk_size.positive?
|
|
18
21
|
end
|
|
19
22
|
|
|
20
23
|
def put_encrypted(key, io, **options)
|
|
@@ -89,7 +92,7 @@ module ActiveCipherStorage
|
|
|
89
92
|
version: Format::VERSION,
|
|
90
93
|
algorithm: Format::ALGO_AES256GCM,
|
|
91
94
|
chunked: true,
|
|
92
|
-
chunk_size: @
|
|
95
|
+
chunk_size: @chunk_size,
|
|
93
96
|
provider_id: @config.provider.provider_id,
|
|
94
97
|
encrypted_dek: dek_bundle.fetch(:encrypted_key)
|
|
95
98
|
))
|
|
@@ -97,8 +100,8 @@ module ActiveCipherStorage
|
|
|
97
100
|
seq = 0
|
|
98
101
|
done = false
|
|
99
102
|
until done
|
|
100
|
-
chunk = input_io.read(@
|
|
101
|
-
done = chunk.bytesize < @
|
|
103
|
+
chunk = input_io.read(@chunk_size) || "".b
|
|
104
|
+
done = chunk.bytesize < @chunk_size
|
|
102
105
|
seq += 1
|
|
103
106
|
frame_seq = done ? Format::FINAL_SEQ : seq
|
|
104
107
|
iv = SecureRandom.random_bytes(Format::IV_SIZE)
|
|
@@ -107,7 +110,7 @@ module ActiveCipherStorage
|
|
|
107
110
|
ct = chunk.empty? ? c.final : (c.update(chunk.b) + c.final)
|
|
108
111
|
Format.write_chunk(buffer, seq: frame_seq, iv: iv, ciphertext: ct, auth_tag: c.auth_tag)
|
|
109
112
|
|
|
110
|
-
next unless buffer.pos >= @
|
|
113
|
+
next unless buffer.pos >= @chunk_size || done
|
|
111
114
|
|
|
112
115
|
buffer.rewind
|
|
113
116
|
part_number += 1
|
|
@@ -125,7 +128,7 @@ module ActiveCipherStorage
|
|
|
125
128
|
def decrypt_io(io)
|
|
126
129
|
header = Format.read_header(io)
|
|
127
130
|
io.rewind
|
|
128
|
-
header.chunked ? StreamCipher.new(@config).decrypt_to_io(io)
|
|
131
|
+
header.chunked ? StreamCipher.new(config: @config, chunk_size: header.chunk_size).decrypt_to_io(io)
|
|
129
132
|
: StringIO.new(Cipher.new(@config).decrypt(io))
|
|
130
133
|
end
|
|
131
134
|
|
|
@@ -149,7 +152,7 @@ module ActiveCipherStorage
|
|
|
149
152
|
|
|
150
153
|
def validate_multipart_chunk_size!
|
|
151
154
|
min_size = Configuration::MINIMUM_S3_MULTIPART_PART_SIZE
|
|
152
|
-
return if @
|
|
155
|
+
return if @chunk_size >= min_size
|
|
153
156
|
|
|
154
157
|
raise ArgumentError,
|
|
155
158
|
"chunk_size must be at least 5 MiB for S3 multipart uploads"
|
|
@@ -235,6 +238,7 @@ module ActiveCipherStorage
|
|
|
235
238
|
def drain_frames(&block)
|
|
236
239
|
until @done
|
|
237
240
|
break if @buffer.bytesize < FRAME_PREFIX_SIZE
|
|
241
|
+
# Length prefix at byte 16: seq(4) + iv(12).
|
|
238
242
|
ct_len = @buffer.byteslice(16, 4).unpack1("N")
|
|
239
243
|
frame_size = FRAME_PREFIX_SIZE + ct_len + Format::AUTH_TAG_SIZE
|
|
240
244
|
break if @buffer.bytesize < frame_size
|
|
@@ -5,11 +5,10 @@ module ActiveCipherStorage
|
|
|
5
5
|
# encrypted => true
|
|
6
6
|
# cipher_version => Integer (Format::VERSION)
|
|
7
7
|
# provider_id => String (e.g. "aws_kms", "env")
|
|
8
|
-
# kms_key_id => String (CMK ARN,
|
|
8
|
+
# kms_key_id => String (CMK ARN, provider key identifier, or nil)
|
|
9
9
|
#
|
|
10
|
-
# These are for operational visibility
|
|
11
|
-
#
|
|
12
|
-
# authoritative source for decryption.
|
|
10
|
+
# These are for operational visibility and auditing. The encrypted file header
|
|
11
|
+
# is always the authoritative source for decryption.
|
|
13
12
|
module BlobMetadata
|
|
14
13
|
def self.write(storage_key, provider)
|
|
15
14
|
return unless active_storage_available?
|
|
@@ -25,10 +24,8 @@ module ActiveCipherStorage
|
|
|
25
24
|
"kms_key_id" => provider.key_id
|
|
26
25
|
).compact
|
|
27
26
|
)
|
|
28
|
-
rescue => e
|
|
29
|
-
|
|
30
|
-
"[ActiveCipherStorage] Could not write blob metadata for #{storage_key}: #{e.message}"
|
|
31
|
-
)
|
|
27
|
+
rescue StandardError => e
|
|
28
|
+
log_metadata_failure(storage_key, "write", e)
|
|
32
29
|
end
|
|
33
30
|
|
|
34
31
|
def self.write_plaintext(storage_key)
|
|
@@ -45,28 +42,8 @@ module ActiveCipherStorage
|
|
|
45
42
|
"kms_key_id" => nil
|
|
46
43
|
).compact
|
|
47
44
|
)
|
|
48
|
-
rescue => e
|
|
49
|
-
|
|
50
|
-
"[ActiveCipherStorage] Could not write plaintext blob metadata for #{storage_key}: #{e.message}"
|
|
51
|
-
)
|
|
52
|
-
end
|
|
53
|
-
|
|
54
|
-
def self.update_after_rotation(storage_key, new_provider)
|
|
55
|
-
return unless active_storage_available?
|
|
56
|
-
|
|
57
|
-
blob = ActiveStorage::Blob.find_by(key: storage_key)
|
|
58
|
-
return unless blob
|
|
59
|
-
|
|
60
|
-
blob.update_columns(
|
|
61
|
-
metadata: blob.metadata.merge(
|
|
62
|
-
"provider_id" => new_provider.provider_id,
|
|
63
|
-
"kms_key_id" => new_provider.key_id
|
|
64
|
-
).compact
|
|
65
|
-
)
|
|
66
|
-
rescue => e
|
|
67
|
-
ActiveCipherStorage.configuration.logger.warn(
|
|
68
|
-
"[ActiveCipherStorage] Could not update rotation metadata for #{storage_key}: #{e.message}"
|
|
69
|
-
)
|
|
45
|
+
rescue StandardError => e
|
|
46
|
+
log_metadata_failure(storage_key, "write_plaintext", e)
|
|
70
47
|
end
|
|
71
48
|
|
|
72
49
|
# Returns the metadata hash for a blob, or nil if AR is unavailable.
|
|
@@ -75,29 +52,21 @@ module ActiveCipherStorage
|
|
|
75
52
|
ActiveStorage::Blob.find_by(key: storage_key)&.metadata
|
|
76
53
|
end
|
|
77
54
|
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
# Yields each matching blob.
|
|
81
|
-
#
|
|
82
|
-
# For large tables, add a DB-level index on `metadata->>'kms_key_id'`
|
|
83
|
-
# and narrow the scope before passing to this method.
|
|
84
|
-
def self.blobs_for(provider)
|
|
85
|
-
return enum_for(:blobs_for, provider) unless block_given?
|
|
86
|
-
return unless active_storage_available?
|
|
55
|
+
private_class_method def self.log_metadata_failure(storage_key, operation, error)
|
|
56
|
+
raise error if reraise_metadata_errors?
|
|
87
57
|
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
next if provider.key_id && meta["kms_key_id"] != provider.key_id
|
|
58
|
+
ActiveCipherStorage.configuration.logger.warn(
|
|
59
|
+
"[ActiveCipherStorage] Blob metadata failed (#{operation}) for #{storage_key}: #{error.message}"
|
|
60
|
+
)
|
|
61
|
+
end
|
|
93
62
|
|
|
94
|
-
|
|
95
|
-
|
|
63
|
+
private_class_method def self.reraise_metadata_errors?
|
|
64
|
+
defined?(Rails) && Rails.respond_to?(:env) && Rails.env.development?
|
|
96
65
|
end
|
|
97
66
|
|
|
98
67
|
private_class_method def self.active_storage_available?
|
|
99
68
|
defined?(ActiveStorage::Blob) && ActiveStorage::Blob.table_exists?
|
|
100
|
-
rescue
|
|
69
|
+
rescue StandardError
|
|
101
70
|
false
|
|
102
71
|
end
|
|
103
72
|
end
|