active_storage-async_variants 0.1.0 → 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.github/workflows/ci.yml +0 -3
- data/CHANGELOG.md +10 -1
- data/CLAUDE.md +4 -4
- data/README.md +87 -27
- data/app/assets/javascripts/active_storage_async_variants.js +169 -0
- data/app/controllers/active_storage/async_variants/callbacks_controller.rb +34 -1
- data/gemfiles/rails_7.2.gemfile +1 -0
- data/gemfiles/rails_8.0.gemfile +1 -0
- data/gemfiles/rails_8.1.gemfile +1 -0
- data/lib/active_storage/async_variants/asset_tag_helper_extension.rb +77 -0
- data/lib/active_storage/async_variants/attachment_extension.rb +3 -1
- data/lib/active_storage/async_variants/blob_extension.rb +15 -0
- data/lib/active_storage/async_variants/preview_extension.rb +117 -0
- data/lib/active_storage/async_variants/process_job.rb +6 -7
- data/lib/active_storage/async_variants/reflection_extension.rb +35 -0
- data/lib/active_storage/async_variants/registry.rb +38 -0
- data/lib/active_storage/async_variants/representations_redirect_controller_extension.rb +26 -0
- data/lib/active_storage/async_variants/variant_record_extension.rb +29 -0
- data/lib/active_storage/async_variants/variant_with_record_extension.rb +78 -19
- data/lib/active_storage/async_variants/variation_extension.rb +2 -1
- data/lib/active_storage/async_variants/version.rb +1 -1
- data/lib/active_storage/async_variants.rb +69 -8
- metadata +10 -5
- data/gemfiles/rails_7.2.gemfile.lock +0 -269
- data/gemfiles/rails_8.0.gemfile.lock +0 -266
- data/gemfiles/rails_8.1.gemfile.lock +0 -269
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: ed31f4592e3d865609aef29b42657856321528d5013bb2a1ec499e3de93c5bb4
|
|
4
|
+
data.tar.gz: 994460c1ae175245da3d71489b25d4143827931f926289c94c18957df8cf0610
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: c6b94095e095899175ac9fa37497500b9947e761d2ed07a7fdd78d55a90ac05898dca086e02d8b29419fc1fde8567628a4f6c7444470826ae6fb0556220af4ef
|
|
7
|
+
data.tar.gz: 109f1670b0ad22ce670bf27751ffeac462f0f10c94d0d89f4777c211b2fb1aa179c3981373ebc9b2f36a2262c4722dcdf73cd54ff95ca29cf6b035765150357f
|
data/.github/workflows/ci.yml
CHANGED
data/CHANGELOG.md
CHANGED
|
@@ -1,4 +1,13 @@
|
|
|
1
|
-
## [
|
|
1
|
+
## [0.3.1]
|
|
2
|
+
|
|
3
|
+
- Bound the stored variant `error` to 16k chars at both write sites (the failed-status callback and `ProcessJob`'s rescue). An external transformer reporting a >64KB error payload was overflowing the `TEXT` column and 500ing the callback, leaving the variant stuck instead of marked failed.
|
|
4
|
+
|
|
5
|
+
## [0.3.0]
|
|
6
|
+
|
|
7
|
+
- Touch attached records when a variant transitions to processed, so consumer caches keyed on `cache_key_with_version` invalidate without needing manual cascading. Multi-hop cascades remain the consumer's responsibility via standard Rails `touch:` or `after_touch`.
|
|
8
|
+
- Dedupe concurrent `.processed` calls so they enqueue at most one `ProcessJob` per blob+variation. Previously, every call before the first job flipped state to "processing" enqueued another job; each job created a fresh variant blob, leaving the others as orphans. The record is now created as `pending` at enqueue time, using the unique index on `active_storage_variant_records (blob_id, variation_digest)` (already shipped by the standard Active Storage migration) as the dedupe key. Once a record reaches `failed` (after `ProcessJob` exhausts its 3-attempt `retry_on` cycle), further `.processed` calls no longer re-enqueue — the variant is permanently failed.
|
|
9
|
+
- Renamed the `fallback:` variant option to `processing:` to better describe what it represents — the placeholder served while the variant is being processed.
|
|
10
|
+
- New `failed:` variant option that specifies a distinct placeholder to render when the variant has permanently failed, separate from the `processing:` placeholder. Accepts `:original`, `:blank`, a String URL, or a Proc that receives the blob. Defaults to `processing:` when unspecified. Also extends `processing:` to accept a String URL directly (previously you had to wrap a static URL in a Proc).
|
|
2
11
|
|
|
3
12
|
## [0.1.0] - 2026-03-03
|
|
4
13
|
|
data/CLAUDE.md
CHANGED
|
@@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
|
|
|
4
4
|
|
|
5
5
|
## What This Is
|
|
6
6
|
|
|
7
|
-
A Rails engine gem that extends Active Storage with async-safe variant processing. Solves the problem where slow transformations (e.g., video transcoding) block requests or fail silently. The `
|
|
7
|
+
A Rails engine gem that extends Active Storage with async-safe variant processing. Solves the problem where slow transformations (e.g., video transcoding) block requests or fail silently. The `processing:` option on a variant definition opts it into async processing.
|
|
8
8
|
|
|
9
9
|
## Commands
|
|
10
10
|
|
|
@@ -19,9 +19,9 @@ bundle exec rspec spec/active_storage/async_variants_spec.rb -e "description" #
|
|
|
19
19
|
|
|
20
20
|
The gem works by prepending extension modules onto Active Storage classes:
|
|
21
21
|
|
|
22
|
-
- **`VariationExtension`** → `ActiveStorage::Variation` — extracts async options (`
|
|
23
|
-
- **`AttachmentExtension`** → `ActiveStorage::Attachment` — hooks into `transform_variants_later` to enqueue `ProcessJob` for variants with `
|
|
24
|
-
- **`VariantWithRecordExtension`** → `ActiveStorage::VariantWithRecord` — overrides URL generation to serve
|
|
22
|
+
- **`VariationExtension`** → `ActiveStorage::Variation` — extracts async options (`processing:`, `failed:`, `transformer:`) from variant config before passing the rest to standard Active Storage
|
|
23
|
+
- **`AttachmentExtension`** → `ActiveStorage::Attachment` — hooks into `transform_variants_later` to enqueue `ProcessJob` for variants with `processing:`
|
|
24
|
+
- **`VariantWithRecordExtension`** → `ActiveStorage::VariantWithRecord` — overrides URL generation to serve the `processing:` (or `failed:`) placeholder while not ready; adds state query methods (`processed?`, `processing?`, `pending?`, `failed?`)
|
|
25
25
|
- **`ProcessJob`** — background job that determines transformer type (inline vs external) and processes accordingly
|
|
26
26
|
|
|
27
27
|
### Transformer Types
|
data/README.md
CHANGED
|
@@ -19,7 +19,7 @@ bin/rails db:migrate
|
|
|
19
19
|
|
|
20
20
|
## Usage
|
|
21
21
|
|
|
22
|
-
Add `
|
|
22
|
+
Add `processing:` to any named variant to opt into the async pipeline. The value is what to serve while the variant is being processed:
|
|
23
23
|
|
|
24
24
|
```ruby
|
|
25
25
|
class User < ApplicationRecord
|
|
@@ -28,12 +28,12 @@ class User < ApplicationRecord
|
|
|
28
28
|
transformer: VideoTranscoder,
|
|
29
29
|
codec: "vp9",
|
|
30
30
|
resolution: "720p",
|
|
31
|
-
|
|
31
|
+
processing: :original
|
|
32
32
|
end
|
|
33
33
|
end
|
|
34
34
|
```
|
|
35
35
|
|
|
36
|
-
The presence of `
|
|
36
|
+
The presence of `processing:` is what opts a variant into async processing. Without it, variants behave exactly as they do in standard Active Storage. The `transformer:` option is independent -- you can use a custom transformer synchronously, or use the default transformer asynchronously:
|
|
37
37
|
|
|
38
38
|
```ruby
|
|
39
39
|
has_one_attached :video do |attachable|
|
|
@@ -41,14 +41,14 @@ has_one_attached :video do |attachable|
|
|
|
41
41
|
attachable.variant :web,
|
|
42
42
|
transformer: VideoTranscoder,
|
|
43
43
|
codec: "vp9",
|
|
44
|
-
|
|
44
|
+
processing: :original
|
|
45
45
|
|
|
46
46
|
# Async with default transformer (large image resize that's too slow for inline)
|
|
47
47
|
attachable.variant :thumbnail,
|
|
48
48
|
resize_to_limit: [200, 200],
|
|
49
|
-
|
|
49
|
+
processing: :original
|
|
50
50
|
|
|
51
|
-
# Sync with custom transformer (fast custom processing, no
|
|
51
|
+
# Sync with custom transformer (fast custom processing, no opt-in needed)
|
|
52
52
|
attachable.variant :watermarked,
|
|
53
53
|
transformer: WatermarkStamper
|
|
54
54
|
end
|
|
@@ -62,6 +62,65 @@ In views, use the same Active Storage helpers:
|
|
|
62
62
|
|
|
63
63
|
If the variant is still processing, this serves the original video. Once processing completes, it serves the transcoded variant.
|
|
64
64
|
|
|
65
|
+
## `image_tag` / `video_tag` with `async:` and `direct:`
|
|
66
|
+
|
|
67
|
+
For a polish layer that swaps in placeholder content while a variant is being processed, polls for completion in the background, and optionally serves the finished variant straight from your CDN, the gem adds two options to `image_tag` and `video_tag`:
|
|
68
|
+
|
|
69
|
+
```erb
|
|
70
|
+
<%# Variant URL with the async client wired up: spinner while pending,
|
|
71
|
+
polls for completion, swaps to the real image when ready. %>
|
|
72
|
+
<%= image_tag user.avatar.variant(:web), async: true %>
|
|
73
|
+
|
|
74
|
+
<%# When the variant is ready, render its direct CDN/S3 URL instead of
|
|
75
|
+
routing through Rails. While pending/failed, falls back to the
|
|
76
|
+
Rails representation URL (which serves the placeholder). %>
|
|
77
|
+
<%= image_tag user.avatar.variant(:web), direct: true %>
|
|
78
|
+
|
|
79
|
+
<%# Both together: direct URL once processed, placeholder while pending,
|
|
80
|
+
polling for completion. %>
|
|
81
|
+
<%= image_tag user.avatar.variant(:web), async: true, direct: true %>
|
|
82
|
+
|
|
83
|
+
<%# Same for videos. %>
|
|
84
|
+
<%= video_tag user.video.variant(:web), async: true, controls: true %>
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
The first argument must be a `VariantWithRecord` or `Preview` when either option is set; otherwise `image_tag` / `video_tag` behave exactly as in stock Rails.
|
|
88
|
+
|
|
89
|
+
### Configure the direct URL host
|
|
90
|
+
|
|
91
|
+
By default `direct:` uses the storage service's URL (presigned for private buckets, unsigned for public). To serve from a CDN, set the host in an initializer:
|
|
92
|
+
|
|
93
|
+
```ruby
|
|
94
|
+
# config/initializers/active_storage_async_variants.rb
|
|
95
|
+
ActiveStorage::AsyncVariants.cdn_host = "https://d1234abcd.cloudfront.net"
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
The resulting URL is `"#{cdn_host}/#{variant.key}"`.
|
|
99
|
+
|
|
100
|
+
### Wiring the JavaScript
|
|
101
|
+
|
|
102
|
+
The gem ships a dependency-free Stimulus-like controller at `app/assets/javascripts/active_storage_async_variants.js`. It is registered with the asset pipeline by the engine but not auto-imported. Wire it up the same way as `@rails/activestorage` -- pick whichever fits your bundler:
|
|
103
|
+
|
|
104
|
+
Using **importmap-rails**:
|
|
105
|
+
|
|
106
|
+
```ruby
|
|
107
|
+
# config/importmap.rb
|
|
108
|
+
pin "@active_storage/async_variants", to: "active_storage_async_variants.js", preload: true
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
```js
|
|
112
|
+
// app/javascript/application.js
|
|
113
|
+
import "@active_storage/async_variants"
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
Using **the asset pipeline** (sprockets/propshaft) with a classic `javascript_include_tag`:
|
|
117
|
+
|
|
118
|
+
```erb
|
|
119
|
+
<%= javascript_include_tag "active_storage_async_variants" %>
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
The module auto-starts on `DOMContentLoaded` and finds elements with `data-async-variant-state-value`. Opt out by setting `window.ActiveStorageAsyncVariants = null` before the script loads, then calling `start()` yourself from the exported module when ready.
|
|
123
|
+
|
|
65
124
|
## Writing a Transformer
|
|
66
125
|
|
|
67
126
|
Transformers come in two flavors: **inline** (the job blocks until processing completes) and **external** (the job kicks off remote work and a webhook signals completion).
|
|
@@ -158,41 +217,42 @@ The gem determines the mode by which method the transformer implements: `initiat
|
|
|
158
217
|
```ruby
|
|
159
218
|
variant = user.video.variant(:web)
|
|
160
219
|
|
|
161
|
-
variant.
|
|
220
|
+
variant.processed? # => true if processed successfully
|
|
162
221
|
variant.processing? # => true if job is running or external service is working
|
|
163
222
|
variant.pending? # => true if job is enqueued
|
|
164
223
|
variant.failed? # => true if permanently failed
|
|
165
224
|
variant.error # => error message string, or nil
|
|
166
225
|
```
|
|
167
226
|
|
|
168
|
-
##
|
|
227
|
+
## Placeholder Options
|
|
169
228
|
|
|
170
|
-
The `
|
|
229
|
+
The `processing:` option controls what gets served while a variant is being processed. The `failed:` option (optional) controls what gets served once the variant has permanently failed; if omitted, it defaults to the `processing:` value.
|
|
171
230
|
|
|
172
231
|
```ruby
|
|
173
|
-
# Serve the original unprocessed file
|
|
174
|
-
attachable.variant :web,
|
|
232
|
+
# Serve the original unprocessed file while processing
|
|
233
|
+
attachable.variant :web, processing: :original
|
|
175
234
|
|
|
176
235
|
# Return nil -- let the view handle it
|
|
177
|
-
attachable.variant :web,
|
|
236
|
+
attachable.variant :web, processing: :blank
|
|
237
|
+
|
|
238
|
+
# Static URL while processing
|
|
239
|
+
attachable.variant :web, processing: "/placeholders/processing.svg"
|
|
178
240
|
|
|
179
|
-
#
|
|
241
|
+
# Dynamic placeholder via Proc
|
|
180
242
|
attachable.variant :web,
|
|
181
|
-
|
|
243
|
+
processing: -> (blob) { "/placeholders/processing.svg" }
|
|
244
|
+
|
|
245
|
+
# Distinct placeholder when permanently failed
|
|
246
|
+
attachable.variant :web,
|
|
247
|
+
processing: :original,
|
|
248
|
+
failed: "/icons/broken.svg"
|
|
182
249
|
```
|
|
183
250
|
|
|
184
|
-
|
|
251
|
+
Both options accept the same set of values: `:original`, `:blank`, a String URL, or a Proc that receives the blob.
|
|
185
252
|
|
|
186
|
-
|
|
253
|
+
## Failure Handling
|
|
187
254
|
|
|
188
|
-
|
|
189
|
-
attachable.variant :web,
|
|
190
|
-
transformer: VideoTranscoder,
|
|
191
|
-
codec: "vp9",
|
|
192
|
-
resolution: "720p",
|
|
193
|
-
fallback: :original,
|
|
194
|
-
max_retries: 5
|
|
195
|
-
```
|
|
255
|
+
A variant is retried up to 3 times before being marked as permanently failed. Once permanently failed, `.processed` no longer re-enqueues the job — the variant stays failed until you delete the `ActiveStorage::VariantRecord` row manually (or re-attach a fresh blob).
|
|
196
256
|
|
|
197
257
|
Inspect failures:
|
|
198
258
|
|
|
@@ -213,15 +273,15 @@ variant.error # => "ffmpeg exited with status 1: ..."
|
|
|
213
273
|
5. The external service processes the file, uploads the result to the destination URL
|
|
214
274
|
6. The external service POSTs to the callback URL with success/failure status
|
|
215
275
|
7. The gem's callback endpoint transitions the `VariantRecord` to `processed` or `failed`
|
|
216
|
-
8. When a view requests the variant URL, the gem checks state and serves the variant or the
|
|
276
|
+
8. When a view requests the variant URL, the gem checks state and serves the variant or the `processing:`/`failed:` placeholder
|
|
217
277
|
|
|
218
278
|
### Inline transformer flow
|
|
219
279
|
|
|
220
280
|
1-3. Same as above
|
|
221
281
|
4. The job calls the transformer's `process` method, blocking until complete
|
|
222
282
|
5. On success, the output is uploaded, the `VariantRecord` transitions to `processed`
|
|
223
|
-
6. On failure, the error is recorded and the job is re-enqueued (up to
|
|
224
|
-
7. When a view requests the variant URL, the gem checks state and serves the variant or the
|
|
283
|
+
6. On failure, the error is recorded and the job is re-enqueued (up to 3 attempts)
|
|
284
|
+
7. When a view requests the variant URL, the gem checks state and serves the variant or the `processing:`/`failed:` placeholder
|
|
225
285
|
|
|
226
286
|
## License
|
|
227
287
|
|
|
@@ -0,0 +1,169 @@
|
|
|
1
|
+
// active-storage-async_variants
|
|
2
|
+
// Renders the correct fallback for async ActiveStorage variants, and polls
|
|
3
|
+
// for completion. Pairs with the gem's image_tag/video_tag `async:` /
|
|
4
|
+
// `direct:` options.
|
|
5
|
+
|
|
6
|
+
const STATE_ATTR = "data-async-variant-state-value"
|
|
7
|
+
const SRC_ATTR = "data-async-variant-src-value"
|
|
8
|
+
const DIRECT_ATTR = "data-async-variant-direct-value"
|
|
9
|
+
const SELECTOR = `[${STATE_ATTR}]`
|
|
10
|
+
const HEADER = "X-Async-Variant-State"
|
|
11
|
+
const POLLABLE_STATES = ["pending", "processing"]
|
|
12
|
+
const POLL_BASE_MS = 3000
|
|
13
|
+
const MAX_POLLS = 10
|
|
14
|
+
const RETRY_BASE_MS = 10000
|
|
15
|
+
const MAX_RETRIES = 3
|
|
16
|
+
|
|
17
|
+
const isSafari = typeof navigator !== "undefined" &&
|
|
18
|
+
/^((?!chrome|android).)*safari/i.test(navigator.userAgent)
|
|
19
|
+
|
|
20
|
+
const elementState = new WeakMap()
|
|
21
|
+
|
|
22
|
+
function stateFor(el) {
|
|
23
|
+
let s = elementState.get(el)
|
|
24
|
+
if (!s) {
|
|
25
|
+
s = { pollTimer: null, pollCount: 0, retries: 0 }
|
|
26
|
+
elementState.set(el, s)
|
|
27
|
+
}
|
|
28
|
+
return s
|
|
29
|
+
}
|
|
30
|
+
|
|
31
|
+
function srcUrl(el) { return el.getAttribute(SRC_ATTR) }
|
|
32
|
+
function directUrl(el) { return el.getAttribute(DIRECT_ATTR) }
|
|
33
|
+
function variantState(el) { return el.getAttribute(STATE_ATTR) }
|
|
34
|
+
|
|
35
|
+
async function fetchAsyncState(url) {
|
|
36
|
+
if (!url) return null
|
|
37
|
+
try {
|
|
38
|
+
const response = await fetch(url, { method: "HEAD", redirect: "manual", cache: "no-store" })
|
|
39
|
+
if (response.type === "opaqueredirect") return null
|
|
40
|
+
return response.headers.get(HEADER)
|
|
41
|
+
} catch {
|
|
42
|
+
return null
|
|
43
|
+
}
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
function schedulePoll(el) {
|
|
47
|
+
const s = stateFor(el)
|
|
48
|
+
s.pollTimer = setTimeout(() => poll(el), POLL_BASE_MS * Math.pow(2, s.pollCount))
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
function stopPolling(el) {
|
|
52
|
+
const s = elementState.get(el)
|
|
53
|
+
if (s && s.pollTimer) {
|
|
54
|
+
clearTimeout(s.pollTimer)
|
|
55
|
+
s.pollTimer = null
|
|
56
|
+
}
|
|
57
|
+
}
|
|
58
|
+
|
|
59
|
+
function startPolling(el) {
|
|
60
|
+
const s = stateFor(el)
|
|
61
|
+
if (s.pollTimer) return
|
|
62
|
+
s.pollCount = 0
|
|
63
|
+
schedulePoll(el)
|
|
64
|
+
}
|
|
65
|
+
|
|
66
|
+
async function poll(el) {
|
|
67
|
+
const state = await fetchAsyncState(srcUrl(el))
|
|
68
|
+
const s = stateFor(el)
|
|
69
|
+
s.pollTimer = null
|
|
70
|
+
if (POLLABLE_STATES.includes(state)) {
|
|
71
|
+
s.pollCount += 1
|
|
72
|
+
if (s.pollCount < MAX_POLLS) schedulePoll(el)
|
|
73
|
+
return
|
|
74
|
+
}
|
|
75
|
+
if (state === "failed") {
|
|
76
|
+
el.setAttribute(STATE_ATTR, "failed")
|
|
77
|
+
return
|
|
78
|
+
}
|
|
79
|
+
const target = directUrl(el) || srcUrl(el)
|
|
80
|
+
if (!target) return
|
|
81
|
+
el.setAttribute(STATE_ATTR, "processed")
|
|
82
|
+
el.src = target + (target.includes("?") ? "&" : "?") + "_t=" + Date.now()
|
|
83
|
+
}
|
|
84
|
+
|
|
85
|
+
function fallback(el) {
|
|
86
|
+
const url = srcUrl(el)
|
|
87
|
+
if (url) el.setAttribute("src", url)
|
|
88
|
+
}
|
|
89
|
+
|
|
90
|
+
function onLoad(event) {
|
|
91
|
+
const el = event.target
|
|
92
|
+
if (!el.matches || !el.matches(SELECTOR)) return
|
|
93
|
+
if (POLLABLE_STATES.includes(variantState(el))) startPolling(el)
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
function onError(event) {
|
|
97
|
+
const el = event.target
|
|
98
|
+
if (!el.matches || !el.matches(SELECTOR)) return
|
|
99
|
+
const s = stateFor(el)
|
|
100
|
+
if (s.retries >= MAX_RETRIES) return
|
|
101
|
+
setTimeout(() => fallback(el), RETRY_BASE_MS * Math.pow(2, s.retries))
|
|
102
|
+
s.retries += 1
|
|
103
|
+
}
|
|
104
|
+
|
|
105
|
+
function setupVideo(el) {
|
|
106
|
+
if (el.nodeName !== "VIDEO") return
|
|
107
|
+
if (isSafari) fallback(el)
|
|
108
|
+
// Browser autoplay-via-attribute is permitted on first load, but the
|
|
109
|
+
// attribute also makes Turbo's cloneNode(true) snapshot start playback on
|
|
110
|
+
// the detached clone (ghost audio). Strip the attribute after insertion --
|
|
111
|
+
// playback is already scheduled -- and rely on play() here for restored
|
|
112
|
+
// snapshots (which won't have the attribute anymore).
|
|
113
|
+
// https://github.com/hotwired/turbo/issues/1017
|
|
114
|
+
if (el.hasAttribute("autoplay")) {
|
|
115
|
+
el.removeAttribute("autoplay")
|
|
116
|
+
const promise = el.play()
|
|
117
|
+
if (promise && typeof promise.catch === "function") promise.catch(() => {})
|
|
118
|
+
}
|
|
119
|
+
}
|
|
120
|
+
|
|
121
|
+
function processAdded(node) {
|
|
122
|
+
if (node.nodeType !== 1) return
|
|
123
|
+
if (node.matches && node.matches(SELECTOR)) setupVideo(node)
|
|
124
|
+
if (node.querySelectorAll) node.querySelectorAll(SELECTOR).forEach(setupVideo)
|
|
125
|
+
}
|
|
126
|
+
|
|
127
|
+
function processRemoved(node) {
|
|
128
|
+
if (node.nodeType !== 1) return
|
|
129
|
+
if (node.matches && node.matches(SELECTOR)) {
|
|
130
|
+
stopPolling(node)
|
|
131
|
+
elementState.delete(node)
|
|
132
|
+
}
|
|
133
|
+
if (node.querySelectorAll) {
|
|
134
|
+
node.querySelectorAll(SELECTOR).forEach(el => {
|
|
135
|
+
stopPolling(el)
|
|
136
|
+
elementState.delete(el)
|
|
137
|
+
})
|
|
138
|
+
}
|
|
139
|
+
}
|
|
140
|
+
|
|
141
|
+
let started = false
|
|
142
|
+
function start() {
|
|
143
|
+
if (started || typeof document === "undefined") return
|
|
144
|
+
started = true
|
|
145
|
+
document.addEventListener("load", onLoad, true)
|
|
146
|
+
document.addEventListener("error", onError, true)
|
|
147
|
+
const observer = new MutationObserver(records => {
|
|
148
|
+
for (const r of records) {
|
|
149
|
+
r.addedNodes.forEach(processAdded)
|
|
150
|
+
r.removedNodes.forEach(processRemoved)
|
|
151
|
+
}
|
|
152
|
+
})
|
|
153
|
+
observer.observe(document.documentElement, { childList: true, subtree: true })
|
|
154
|
+
document.querySelectorAll(SELECTOR).forEach(setupVideo)
|
|
155
|
+
}
|
|
156
|
+
|
|
157
|
+
function autostart() {
|
|
158
|
+
if (typeof window === "undefined" || window.ActiveStorageAsyncVariants !== null) start()
|
|
159
|
+
}
|
|
160
|
+
|
|
161
|
+
if (typeof document !== "undefined") {
|
|
162
|
+
if (document.readyState === "loading") {
|
|
163
|
+
document.addEventListener("DOMContentLoaded", autostart)
|
|
164
|
+
} else {
|
|
165
|
+
setTimeout(autostart, 1)
|
|
166
|
+
}
|
|
167
|
+
}
|
|
168
|
+
|
|
169
|
+
export { start }
|
|
@@ -10,8 +10,10 @@ module ActiveStorage
|
|
|
10
10
|
case params[:status]
|
|
11
11
|
when "success"
|
|
12
12
|
variant_record.update!(state: "processed")
|
|
13
|
+
apply_reported_metadata(variant_record, params)
|
|
13
14
|
when "failed"
|
|
14
|
-
|
|
15
|
+
# error column is TEXT (64KB); utf8mb4 is up to 4 bytes/char, so cap at 16k chars.
|
|
16
|
+
variant_record.update!(state: "failed", error: params[:error].to_s.truncate(16_000))
|
|
15
17
|
else
|
|
16
18
|
head :unprocessable_entity and return
|
|
17
19
|
end
|
|
@@ -20,6 +22,37 @@ module ActiveStorage
|
|
|
20
22
|
rescue ActiveSupport::MessageVerifier::InvalidSignature
|
|
21
23
|
head :unauthorized
|
|
22
24
|
end
|
|
25
|
+
|
|
26
|
+
private
|
|
27
|
+
|
|
28
|
+
# External transformers (Crucible) write the file directly to the bucket
|
|
29
|
+
# and report its byte_size/checksum on the success callback. Reconcile the
|
|
30
|
+
# placeholder blobs created with byte_size: 0, checksum: "0" -- the variant
|
|
31
|
+
# itself, and (for video previews) the extracted frame on the source blob.
|
|
32
|
+
def apply_reported_metadata(variant_record, params)
|
|
33
|
+
reconcile(variant_record.image.blob, params[:byte_size], params[:checksum])
|
|
34
|
+
reconcile(variant_record.blob, params[:preview_image_byte_size], params[:preview_image_checksum])
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
# The placeholder sentinels (byte_size 0, checksum "0") gate each field, so
|
|
38
|
+
# this is idempotent and never overwrites a real source blob's metadata.
|
|
39
|
+
def reconcile(blob, byte_size, checksum)
|
|
40
|
+
return unless blob
|
|
41
|
+
|
|
42
|
+
attrs = {}
|
|
43
|
+
if (bytes = positive_int(byte_size)) && blob.byte_size.zero?
|
|
44
|
+
attrs[:byte_size] = bytes
|
|
45
|
+
end
|
|
46
|
+
if checksum.present? && checksum != "0" && blob.checksum == "0"
|
|
47
|
+
attrs[:checksum] = checksum
|
|
48
|
+
end
|
|
49
|
+
blob.update!(attrs) if attrs.any?
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
def positive_int(value)
|
|
53
|
+
int = value.to_i
|
|
54
|
+
int.positive? ? int : nil
|
|
55
|
+
end
|
|
23
56
|
end
|
|
24
57
|
end
|
|
25
58
|
end
|
data/gemfiles/rails_7.2.gemfile
CHANGED
data/gemfiles/rails_8.0.gemfile
CHANGED
data/gemfiles/rails_8.1.gemfile
CHANGED
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module ActiveStorage
|
|
4
|
+
module AsyncVariants
|
|
5
|
+
module AssetTagHelperExtension
|
|
6
|
+
def image_tag(source, options = {})
|
|
7
|
+
options = options.symbolize_keys
|
|
8
|
+
async = options.delete(:async)
|
|
9
|
+
direct = options.delete(:direct)
|
|
10
|
+
return super if !async && !direct
|
|
11
|
+
variant = AssetTagHelperExtension.coerce_variant!(source)
|
|
12
|
+
src = AssetTagHelperExtension.resolve_src(variant, direct: direct)
|
|
13
|
+
AssetTagHelperExtension.apply_async_data!(options, variant: variant, direct: direct) if async
|
|
14
|
+
super(src, options)
|
|
15
|
+
end
|
|
16
|
+
|
|
17
|
+
def video_tag(*sources)
|
|
18
|
+
options = sources.extract_options!.symbolize_keys
|
|
19
|
+
async = options.delete(:async)
|
|
20
|
+
direct = options.delete(:direct)
|
|
21
|
+
return super(*sources, options) if !async && !direct
|
|
22
|
+
variant = AssetTagHelperExtension.coerce_variant!(sources.first)
|
|
23
|
+
sources[0] = AssetTagHelperExtension.resolve_src(variant, direct: direct)
|
|
24
|
+
AssetTagHelperExtension.apply_async_data!(options, variant: variant, direct: direct) if async
|
|
25
|
+
super(*sources, options)
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
class << self
|
|
29
|
+
def coerce_variant!(source)
|
|
30
|
+
unless source.is_a?(ActiveStorage::VariantWithRecord) || source.is_a?(ActiveStorage::Preview)
|
|
31
|
+
raise ArgumentError, "image_tag/video_tag with async:/direct: requires an ActiveStorage::VariantWithRecord or Preview, got #{source.class}"
|
|
32
|
+
end
|
|
33
|
+
source
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
def resolve_src(variant, direct:)
|
|
37
|
+
if direct && variant.async_state == "processed"
|
|
38
|
+
direct_url(variant)
|
|
39
|
+
else
|
|
40
|
+
# Idempotent: enqueue ProcessJob if no record exists yet so a
|
|
41
|
+
# never-touched variant doesn't sit pending forever. Gated on
|
|
42
|
+
# bucket-backed services -- on Disk/Test the gem defers to
|
|
43
|
+
# vanilla ActiveStorage, and #processed would run a synchronous
|
|
44
|
+
# transform (e.g. vips on an mp4) that the external transformer
|
|
45
|
+
# is supposed to handle.
|
|
46
|
+
variant.processed if variant.blob.bucket_backed?
|
|
47
|
+
polymorphic_url(variant)
|
|
48
|
+
end
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
def apply_async_data!(options, variant:, direct:)
|
|
52
|
+
data = (options[:data] || {}).symbolize_keys
|
|
53
|
+
controllers = data[:controller].to_s.split
|
|
54
|
+
controllers << "async-variant" unless controllers.include?("async-variant")
|
|
55
|
+
data[:controller] = controllers.join(" ")
|
|
56
|
+
data[:async_variant_src_value] = polymorphic_url(variant)
|
|
57
|
+
data[:async_variant_state_value] = variant.async_state
|
|
58
|
+
data[:async_variant_direct_value] = direct_url(variant) if direct && variant.async_state == "processed"
|
|
59
|
+
options[:data] = data
|
|
60
|
+
end
|
|
61
|
+
|
|
62
|
+
def direct_url(variant)
|
|
63
|
+
if cdn = ActiveStorage::AsyncVariants.cdn_host
|
|
64
|
+
"#{cdn}/#{variant.key}"
|
|
65
|
+
else
|
|
66
|
+
variant.image.url
|
|
67
|
+
end
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
def polymorphic_url(variant)
|
|
71
|
+
url_options = ActiveStorage::Current.url_options || Rails.application.routes.default_url_options
|
|
72
|
+
Rails.application.routes.url_helpers.polymorphic_url(variant, **url_options)
|
|
73
|
+
end
|
|
74
|
+
end
|
|
75
|
+
end
|
|
76
|
+
end
|
|
77
|
+
end
|
|
@@ -11,8 +11,10 @@ module ActiveStorage
|
|
|
11
11
|
end
|
|
12
12
|
|
|
13
13
|
def enqueue_async_variant_jobs
|
|
14
|
+
return unless blob.bucket_backed?
|
|
15
|
+
|
|
14
16
|
named_variants.each do |name, named_variant|
|
|
15
|
-
next unless named_variant.transformations.key?(:
|
|
17
|
+
next unless named_variant.transformations.key?(:processing)
|
|
16
18
|
|
|
17
19
|
ActiveStorage::AsyncVariants::ProcessJob.perform_later(
|
|
18
20
|
record, self.name, name.to_s,
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module ActiveStorage
|
|
4
|
+
module AsyncVariants
|
|
5
|
+
module BlobExtension
|
|
6
|
+
# True when the blob's service is a remote/cloud service that the async
|
|
7
|
+
# processing workers (Crucible, etc.) can reach via presigned URLs. The
|
|
8
|
+
# Disk and Test services return false; the gem then defers to vanilla
|
|
9
|
+
# ActiveStorage rather than trying to enqueue jobs or serve fallbacks.
|
|
10
|
+
def bucket_backed?
|
|
11
|
+
service.respond_to?(:bucket)
|
|
12
|
+
end
|
|
13
|
+
end
|
|
14
|
+
end
|
|
15
|
+
end
|