active_storage-async_variants 0.8.0 → 0.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (42) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +23 -0
  3. data/CLAUDE.md +13 -7
  4. data/README.md +30 -57
  5. data/Rakefile +1 -20
  6. data/app/controllers/active_storage/async_variants/callbacks_controller.rb +28 -19
  7. data/config/routes.rb +0 -17
  8. data/db/migrate/20260304000001_add_state_columns_to_variant_records.rb +9 -0
  9. data/gemfiles/rails_7.2.gemfile +0 -4
  10. data/gemfiles/rails_8.0.gemfile +0 -4
  11. data/gemfiles/rails_8.1.gemfile +0 -4
  12. data/lib/active_storage/async_variants/attachment_extension.rb +19 -2
  13. data/lib/active_storage/async_variants/blob_extension.rb +3 -1
  14. data/lib/active_storage/async_variants/named_variant_scan.rb +55 -0
  15. data/lib/active_storage/async_variants/preview_extension.rb +52 -37
  16. data/lib/active_storage/async_variants/process_job.rb +53 -17
  17. data/lib/active_storage/async_variants/reflection_extension.rb +2 -2
  18. data/lib/active_storage/async_variants/registry.rb +1 -1
  19. data/lib/active_storage/async_variants/transformer.rb +32 -1
  20. data/lib/active_storage/async_variants/variant_record_extension.rb +15 -0
  21. data/lib/active_storage/async_variants/variant_with_record_extension.rb +11 -30
  22. data/lib/active_storage/async_variants/variation_extension.rb +8 -5
  23. data/lib/active_storage/async_variants/version.rb +1 -1
  24. data/lib/active_storage/async_variants.rb +31 -34
  25. metadata +4 -47
  26. data/app/assets/javascripts/progress-bar.js +0 -347
  27. data/app/assets/javascripts/retry.js +0 -11
  28. data/app/assets/stylesheets/progress.css +0 -35
  29. data/app/assets/stylesheets/retry.css +0 -98
  30. data/app/controllers/active_storage/async_variants/states_controller.rb +0 -71
  31. data/app/views/active_storage/async_variants/states/_failed.html.erb +0 -44
  32. data/app/views/active_storage/async_variants/states/_pending.html.erb +0 -1
  33. data/app/views/active_storage/async_variants/states/_processed.html.erb +0 -5
  34. data/app/views/active_storage/async_variants/states/_processing.html.erb +0 -15
  35. data/app/views/active_storage/async_variants/states/show.html.erb +0 -10
  36. data/features/retry_flow.feature +0 -20
  37. data/features/retry_visibility.feature +0 -23
  38. data/features/state_rendering.feature +0 -40
  39. data/features/step_definitions/dummy_steps.rb +0 -144
  40. data/features/support/env.rb +0 -28
  41. data/lib/active_storage/async_variants/asset_tag_helper_extension.rb +0 -59
  42. data/lib/active_storage/async_variants/helper.rb +0 -80
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 34c672d7d6d989bb7c633aa1e57e081d6879a8ea9695ffd65997b9df9cf8f8ab
4
- data.tar.gz: aca54bb9b2ae6af16016f1e96e6e3f221bc7029b8eca9c145535eadefeb57d66
3
+ metadata.gz: fffff13cd7b602bc0139ae60dff32c4d38d573b738edaa244f21df5dfa331723
4
+ data.tar.gz: 1251c74b70fd15eaaf1e48e494b7fc9d773f2989f83e1b99ca6862fdaec6c144
5
5
  SHA512:
6
- metadata.gz: e86b3777c71d70d2014fe439fb91f395f1711cce40efefa3c5e5291a22bd2fc6d1933aad3a1c095557726e45547d7112f9e2f2bbd24fece75726adebddd6f686
7
- data.tar.gz: 591deb4316c685316084ebb26904e5fe5b2ecbea0328cc832469ae4185388239511e35507b53264e137cf85153bfe1a846e7bd0491ba023bb991333653cfb532
6
+ metadata.gz: 2bfef2ba7b3d4576d20a867095c7e75f8f36ab122d0718901856ce0bab8330521f5edeb73a0e75a12f3ed20250fe28c25304b4823929167daa910d240a7b9477
7
+ data.tar.gz: f0e5e7a9cfe3d75ff7afe960fc7675447b466dc6e8500b7db6fca3c6a24cd8d0dfa837e938b216898a8ee8d95f470c1aa80b03be7c407e8e0ee7a63a46d74775
data/CHANGELOG.md CHANGED
@@ -1,3 +1,26 @@
1
+ ## [0.10.0]
2
+
3
+ - **Breaking:** Async previews now persist the same graph stock Active Storage builds. For a previewable blob (a video), an image-format variant is a `Preview`: the extracted frame is attached as the source blob's `preview_image`, and the variant record hangs off the *frame* blob, not the source. `ProcessJob` resolves the attachment's `representation` and takes the preview path when stock would. External transformers receive a frame placeholder (`byte_size: 0, checksum: "0"`) to write into; the success callback reconciles both placeholders. Read video stills through `representation`/`preview`, not `variant`.
4
+ - **Breaking:** Naming a `transformer:` is now the only async opt-in. The `async:` option is gone and raises `ArgumentError`. The built-in `ActiveStorage::AsyncVariants::Standard` transformer replaces a bare `async: true` (the stock variant pipeline, run in the background). A transformer's kind follows from what it overrides: `Transformer#standard?` (neither), `#inline?` (`process`), `#external?` (`initiate`).
5
+ - **Breaking:** Attaching a file enqueues an async variant only when it declares Active Storage's `preprocessed:` option (`true`, a method name, or a proc evaluated against the record). Previously every variant naming a `transformer:` was enqueued on every attach, whatever the file — so an image upload also ran the attachment's video transcode. Stock Active Storage no longer sees async variants when preprocessing, so `preprocessed: true` never also queues a `TransformJob` or a local `PreviewImageJob`.
6
+ - Migration note: existing variant records for video stills live on the source blob under the old structure. Move them onto the frame blob (`UPDATE active_storage_variant_records SET blob_id = <frame>`), attaching a placeholder frame where none exists; the `variation_digest` is unchanged. Records whose output is itself a video (transcodes) stay on the source. Replace `async: true` with `transformer: ActiveStorage::AsyncVariants::Standard` where no transformer was named, and drop it where one was. Add `preprocessed: true` (or a condition) to each async variant that should process on attach.
7
+ - `Preview` speaks the same progress protocol as `VariantWithRecord`: `async_state`, `progress`, `progress_known?`, `progress_rate`, `last_heartbeat_at`, `error`.
8
+ - `async_record` is public on both `VariantWithRecord` and `Preview`, and both gain `retry!` (drop a failed record, then `enqueue!`), so callers need not know where a representation keeps its record.
9
+ - `NamedVariantScan` compares declarations structurally instead of instantiating `attachment.variant`, which raises for videos.
10
+ - Fix: declaring `preprocessed:` alongside a `transformer:` no longer breaks the declaration-time registry warm-up (a proc raised on `Marshal.dump`; other values registered the wrong digest).
11
+ - MirrorService: the success callback mirrors reconciled variant blobs to the mirrors, as Rails does for direct uploads, and `bucket_backed?` resolves through the primary service.
12
+ - Frame placeholder creation is serialized on the blob row: every image variant of one video reaches it concurrently, and a second `preview_image` attachment would be invisible to `has_one`, stranding the records hanging off it.
13
+ - Ships the `state`/`error`/`attempts` migration for `active_storage_variant_records` (guarded with `if_not_exists`, so apps that added the columns themselves can install it harmlessly), and builds the test schema from `db/migrate` so a missing shipped migration fails the suite.
14
+ - Fix: `ProcessJob` skips a variant record that is already processed or whose run is still heartbeating. Previously every enqueue re-dispatched the variant — attaching an already-processed blob to another record, a retry, or a duplicate job paid for a second render and flipped processed media back to processing.
15
+ - Fix: the success callback applies the reported `byte_size` and `checksum` whenever they arrive, not only over the `0`/`"0"` placeholders, since a re-render writes over the record's existing output blob. Reported `preview_image_*` values land only on a frame, never on a variant's source blob.
16
+ - Fix: destroying a variant record purges any output a racing render attached after the destroy read its attachments, so a purged source no longer leaves outputs pointing at a deleted record.
17
+
18
+ ## [0.9.0]
19
+
20
+ - **Breaking:** Split the opinionated UI out into a separate gem, [`active_storage-async_variants-ui`](https://github.com/botandrose/active_storage-async_variants-ui). This gem is now UI-free plumbing — the async state machine, `ProcessJob`, the heartbeat watchdog, the external-service callback endpoint, and the `Transformer` base class. Its only runtime dependency is `activestorage` (no more `turbo-rails` / `isolate_assets`).
21
+ - Moved to the UI gem: `image_tag`/`video_tag` with `async:`/`direct:`, the `<turbo-frame>` state endpoint and partials, the circular progress bar, the filename-bearing placeholder route, the retry affordance, and the `cdn_host` / `parent_controller` / `retry_visible_if` configuration. A variant's `.url` (serving the original until processed) is unchanged.
22
+ - Migration note: apps that relied on the rendering helpers should add `gem "active_storage-async_variants-ui"`. Apps using `active_storage-crucible` or only the variant pipeline need no change.
23
+
1
24
  ## [0.8.0]
2
25
 
3
26
  - The processing progress bar now advances smoothly between polls: variants expose a `rate` (percent-per-second, derived from the record's `created_at`, last heartbeat, and reported progress) that the `@botandrose/progress-bar` element uses to optimistically creep forward, instead of only stepping on each refresh. Includes a migration adding a `created_at` column to `active_storage_variant_records`; run it before deploying.
data/CLAUDE.md CHANGED
@@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
4
4
 
5
5
  ## What This Is
6
6
 
7
- A Rails engine gem that extends Active Storage with async-safe variant processing. Solves the problem where slow transformations (e.g., video transcoding) block requests or fail silently. The `async: true` option on a variant definition opts it into async processing.
7
+ A Rails engine gem that extends Active Storage with async-safe variant processing. Solves the problem where slow transformations (e.g., video transcoding) block requests or fail silently. Naming a `transformer:` on a variant definition opts it into async processing.
8
8
 
9
9
  ## Commands
10
10
 
@@ -19,17 +19,23 @@ bundle exec rspec spec/active_storage/async_variants_spec.rb -e "description" #
19
19
 
20
20
  The gem works by prepending extension modules onto Active Storage classes:
21
21
 
22
- - **`VariationExtension`** → `ActiveStorage::Variation` — extracts async options (`async:`, `transformer:`) from variant config before passing the rest to standard Active Storage
23
- - **`AttachmentExtension`** → `ActiveStorage::Attachment` — hooks into `transform_variants_later` to enqueue `ProcessJob` for variants with `async: true`
22
+ - **`VariationExtension`** → `ActiveStorage::Variation` — extracts the `transformer:` option from variant config before passing the rest to standard Active Storage
23
+ - **`AttachmentExtension`** → `ActiveStorage::Attachment` — hooks into `transform_variants_later` to enqueue `ProcessJob` for async variants whose `preprocessed:` holds for the record, hiding async variants from stock preprocessing
24
24
  - **`VariantWithRecordExtension`** → `ActiveStorage::VariantWithRecord` — overrides URL generation to serve the original blob while not ready; adds state query methods (`processed?`, `processing?`, `pending?`, `failed?`)
25
- - **`ProcessJob`** — background job that determines transformer type (inline vs external) and processes accordingly
25
+ - **`PreviewExtension`** → `ActiveStorage::Preview` — blocks the synchronous preview transform for async variants and resolves state/URLs through the stock preview graph
26
+ - **`ProcessJob`** — background job that resolves the attachment's representation (variant vs preview), determines transformer type (inline vs external), and processes accordingly
27
+
28
+ ### Preview Structure
29
+
30
+ Async previews persist the same graph stock Active Storage builds: the extracted frame is attached as the source blob's `preview_image`, and the variant record hangs off the *frame* blob, not the source. External transformers get a frame placeholder (`byte_size: 0, checksum: "0"`) the service writes into; the success callback reconciles both the variant and frame placeholders. Inline/default transformers extract the real frame up front via the stock previewer.
26
31
 
27
32
  ### Transformer Types
28
33
 
29
- - **Inline**: implements `process(file, **options)` → blocks worker, returns `{ io:, content_type:, filename: }`
30
- - **External**: implements `initiate(source_url:, destination_url:, callback_url:, **options)` → frees worker immediately, external service POSTs to callback URL when done
34
+ Naming a `transformer:` is the async opt-in; the kind is determined by what the transformer overrides (`Transformer#standard?`/`#inline?`/`#external?`):
31
35
 
32
- The gem detects which type by checking if `process` is overridden on the transformer class.
36
+ - **Standard** (`ActiveStorage::AsyncVariants::Standard`): overrides neither → runs the stock Active Storage variant pipeline in the background. Replaces the old bare `async: true` (no transformer) declaration, which now raises.
37
+ - **Inline**: overrides `process(file, **options)` → blocks worker, returns `{ io:, content_type:, filename: }`
38
+ - **External**: overrides `initiate(source_url:, callback_url:, **options)` → frees worker immediately, external service POSTs to callback URL when done
33
39
 
34
40
  ### Callback Endpoint
35
41
 
data/README.md CHANGED
@@ -2,6 +2,8 @@
2
2
 
3
3
  Extends Active Storage with pluggable per-variant transformers, async-safe variant processing, and failure handling.
4
4
 
5
+ This gem is the UI-free plumbing. A variant's `.url` serves the original while it is pending/processing/failed, and the processed variant once ready — nothing renders a spinner or progress bar. For the optional turbo-frame UI (`image_tag`/`video_tag` with `async:`/`direct:`, a circular progress bar, and a retry affordance), add the companion gem [`active_storage-async_variants-ui`](https://github.com/botandrose/active_storage-async_variants-ui).
6
+
5
7
  ## The Problem
6
8
 
7
9
  Active Storage's variant system assumes transformations are fast and reliable -- like generating an image thumbnail. But some transformations are slow (transcoding a 1GB video to 720p VP9) and fallible (the transcode may permanently fail). When you use `process: :later`, Active Storage enqueues a background job, but if the variant is requested before the job finishes, it falls through to synchronous processing -- blocking the request for minutes or timing out entirely. And if the transformation fails, the error bubbles up with no tracking or retry limits.
@@ -19,7 +21,7 @@ bin/rails db:migrate
19
21
 
20
22
  ## Usage
21
23
 
22
- Add `async: true` to any named variant to opt into the async pipeline:
24
+ Name a `transformer:` on any variant to opt it into the async pipeline:
23
25
 
24
26
  ```ruby
25
27
  class User < ApplicationRecord
@@ -27,102 +29,73 @@ class User < ApplicationRecord
27
29
  attachable.variant :web,
28
30
  transformer: VideoTranscoder,
29
31
  codec: "vp9",
30
- resolution: "720p",
31
- async: true
32
+ resolution: "720p"
32
33
  end
33
34
  end
34
35
  ```
35
36
 
36
- The presence of `async: true` is what opts a variant into async processing. Without it, variants behave exactly as they do in standard Active Storage. The `transformer:` option is independent -- you can use a custom transformer synchronously, or use the default transformer asynchronously:
37
+ The presence of a `transformer:` is what opts a variant into async processing. Without it, variants behave exactly as they do in standard Active Storage. The transformer also says *how* the work happens — there are three kinds, distinguished by what they implement:
37
38
 
38
39
  ```ruby
39
40
  has_one_attached :video do |attachable|
40
- # Async with custom transformer (video transcode)
41
+ # External: hands off to a remote service (implements #initiate)
41
42
  attachable.variant :web,
42
43
  transformer: VideoTranscoder,
43
- codec: "vp9",
44
- async: true
44
+ codec: "vp9"
45
45
 
46
- # Async with default transformer (large image resize that's too slow for inline)
46
+ # Standard: the stock Active Storage variant pipeline, run in the background
47
+ # (e.g. a large image resize that's too slow for an inline request)
47
48
  attachable.variant :thumbnail,
48
49
  resize_to_limit: [200, 200],
49
- async: true
50
+ transformer: ActiveStorage::AsyncVariants::Standard
50
51
 
51
- # Sync with custom transformer (fast custom processing, no opt-in needed)
52
+ # Inline: custom processing in the worker (implements #process)
52
53
  attachable.variant :watermarked,
53
54
  transformer: WatermarkStamper
54
55
  end
55
56
  ```
56
57
 
57
- In views, use the same Active Storage helpers:
58
-
59
- ```erb
60
- <%= video_tag user.video.variant(:web).url %>
61
- ```
62
-
63
- While the variant is still processing (or has failed), this serves the original video. Once processing completes, it serves the transcoded variant.
64
-
65
- ## `image_tag` / `video_tag` with `async:` and `direct:`
66
-
67
- For a polish layer that shows a progress bar while a variant is being processed, polls for completion in the background, and optionally serves the finished variant straight from your CDN, the gem adds two options to `image_tag` and `video_tag`:
68
-
69
- ```erb
70
- <%# Variant URL with the async client wired up: a progress bar while pending,
71
- polls for completion, swaps to the real image when ready. %>
72
- <%= image_tag user.avatar.variant(:web), async: true %>
58
+ A variant with no `transformer:` is a plain synchronous Active Storage variant — the gem stays out of the way.
73
59
 
74
- <%# When the variant is ready, render its direct CDN/S3 URL instead of
75
- routing through Rails. While pending/failed, falls back to the
76
- Rails representation URL (which serves the original). %>
77
- <%= image_tag user.avatar.variant(:web), direct: true %>
60
+ ### Processing on attach
78
61
 
79
- <%# Both together: direct URL once processed, progress bar while pending,
80
- polling for completion. %>
81
- <%= image_tag user.avatar.variant(:web), async: true, direct: true %>
62
+ An async variant is processed when a file is attached only if it says so with Active Storage's own `preprocessed:` option. It takes `true`, a method name, or a proc, evaluated against the record — so one attachment that accepts both images and videos can transcode only the videos:
82
63
 
83
- <%# Same for videos. %>
84
- <%= video_tag user.video.variant(:web), async: true, controls: true %>
64
+ ```ruby
65
+ has_one_attached :media do |attachable|
66
+ attachable.variant :thumb, resize_to_limit: [150, 150], format: "webp",
67
+ transformer: Crucible, preprocessed: true
68
+ attachable.variant :transcoded, resize_to_limit: [1280, 720], format: "mp4",
69
+ transformer: Crucible, preprocessed: :video?
70
+ end
85
71
  ```
86
72
 
87
- The first argument must be a `VariantWithRecord` or `Preview` when either option is set; otherwise `image_tag` / `video_tag` behave exactly as in stock Rails.
88
-
89
- ### Configure the direct URL host
73
+ An async variant without `preprocessed:` is never enqueued on attach; call `enqueue!` on it when you want it. Stock Active Storage never preprocesses an async variant itself — the gem's `ProcessJob` is the only thing that does.
90
74
 
91
- By default `direct:` uses the storage service's URL (presigned for private buckets, unsigned for public). To serve from a CDN, set the host in an initializer:
75
+ In views, use the same Active Storage helpers:
92
76
 
93
- ```ruby
94
- # config/initializers/active_storage_async_variants.rb
95
- ActiveStorage::AsyncVariants.cdn_host = "https://d1234abcd.cloudfront.net"
77
+ ```erb
78
+ <%= video_tag user.video.variant(:web).url %>
96
79
  ```
97
80
 
98
- The resulting URL is `"#{cdn_host}/#{variant.key}"`.
99
-
100
- ### JavaScript
101
-
102
- No manual wiring is required. The async state partials are self-contained `<turbo-frame>`s. The only requirement is that the host app loads **Turbo** -- the gem depends on `turbo-rails`, which a default Rails app already includes.
81
+ While the variant is still processing (or has failed), this serves the original video. Once processing completes, it serves the transcoded variant.
103
82
 
104
83
  ## Configuration
105
84
 
106
- Set options in an initializer. The `configure` block groups them (each is also a plain accessor, e.g. `ActiveStorage::AsyncVariants.cdn_host = …`):
85
+ Set options in an initializer. The `configure` block groups them (each is also a plain accessor, e.g. `ActiveStorage::AsyncVariants.heartbeat_interval = …`):
107
86
 
108
87
  ```ruby
109
88
  # config/initializers/active_storage_async_variants.rb
110
89
  ActiveStorage::AsyncVariants.configure do |config|
111
- config.cdn_host = "https://d1234abcd.cloudfront.net"
112
90
  config.heartbeat_interval = 5.seconds
113
91
  config.heartbeat_stale_after = 60.seconds
114
- config.parent_controller = "ApplicationController"
115
- config.retry_visible_if { current_user&.admin? }
116
92
  end
117
93
  ```
118
94
 
119
95
  | Option | Default | Purpose |
120
96
  |--------|---------|---------|
121
- | `cdn_host` | `nil` | Host for `direct:` URLs (`"#{cdn_host}/#{variant.key}"`); falls back to the storage service URL. |
122
- | `heartbeat_interval` | `5.seconds` | Expected cadence of progress heartbeats; the processing `<turbo-frame>` re-polls at this rate. |
97
+ | `heartbeat_interval` | `5.seconds` | Expected cadence of progress heartbeats from external transformers. (The UI gem's processing `<turbo-frame>` re-polls at this rate.) |
123
98
  | `heartbeat_stale_after` | `60.seconds` | A processing variant with no heartbeat for this long is marked `failed`. Must exceed `heartbeat_interval`. |
124
- | `parent_controller` | `"ActionController::Base"` | Base class for the gem's controllers, so the retry view can reach your app's `current_user`. Set as a String. |
125
- | `retry_visible_if` | off | Block (run in the view context) gating the failed-state retry affordance. |
126
99
 
127
100
  ## Writing a Transformer
128
101
 
@@ -229,7 +202,7 @@ variant.error # => error message string, or nil
229
202
 
230
203
  ## Placeholders
231
204
 
232
- There is nothing to configure. A variant's `.url` serves the original while it is pending, processing, or failed, and the processed variant once ready. With `async: true` on `image_tag`/`video_tag`, the gem instead shows a circular progress bar while processing, and a red error glyph -- the same progress bar in its error state -- once the variant has permanently failed.
205
+ There is nothing to configure. A variant's `.url` serves the original while it is pending, processing, or failed, and the processed variant once ready. (For a circular progress bar while processing and a red error glyph once failed, add the [`active_storage-async_variants-ui`](https://github.com/botandrose/active_storage-async_variants-ui) gem.)
233
206
 
234
207
  ## Failure Handling
235
208
 
@@ -248,7 +221,7 @@ variant.error # => "ffmpeg exited with status 1: ..."
248
221
  ### External transformer flow
249
222
 
250
223
  1. User uploads a file to an attachment that has async variants defined
251
- 2. After attachment, a background job is enqueued for each async variant
224
+ 2. After attachment, a background job is enqueued for each async variant whose `preprocessed:` holds for the record
252
225
  3. `VariantRecord` is created with state `pending`
253
226
  4. The job calls the transformer's `initiate` method with presigned source/destination URLs and a signed callback URL, then exits -- the worker is free
254
227
  5. The external service processes the file, uploads the result to the destination URL
data/Rakefile CHANGED
@@ -5,23 +5,4 @@ require "rspec/core/rake_task"
5
5
 
6
6
  RSpec::Core::RakeTask.new(:spec)
7
7
 
8
- desc "Run browser-level acceptance tests (cucumber + cuprite against spec/dummy)"
9
- task :cucumber do
10
- sh "bundle exec cucumber"
11
- end
12
-
13
- desc "Run rspec and cucumber"
14
- task all: [:spec, :cucumber]
15
-
16
- namespace :vendor do
17
- desc "Pull the latest @botandrose/progress-bar from unpkg.com into app/assets"
18
- task :progress_bar do
19
- require "open-uri"
20
- url = "https://unpkg.com/@botandrose/progress-bar"
21
- dest = File.expand_path("app/assets/javascripts/progress-bar.js", __dir__)
22
- File.write(dest, URI.open(url).read)
23
- puts "Vendored #{url} -> #{dest}"
24
- end
25
- end
26
-
27
- task default: :all
8
+ task default: :spec
@@ -4,9 +4,6 @@ module ActiveStorage
4
4
  module AsyncVariants
5
5
  class CallbacksController < ActionController::API
6
6
  def create
7
- variant_record_id = ActiveStorage.verifier.verify(params[:token], purpose: :async_variant_callback)
8
- variant_record = ActiveStorage::VariantRecord.find(variant_record_id)
9
-
10
7
  case params[:status]
11
8
  when "success"
12
9
  variant_record.update!(state: "processed")
@@ -29,33 +26,45 @@ module ActiveStorage
29
26
 
30
27
  private
31
28
 
32
- # External transformers (Crucible) write the file directly to the bucket
33
- # and report its byte_size/checksum on the success callback. Reconcile the
34
- # placeholder blobs created with byte_size: 0, checksum: "0" -- the variant
35
- # itself, and (for video previews) the extracted frame on the source blob.
29
+ def variant_record
30
+ @variant_record ||= begin
31
+ variant_record_id = ActiveStorage.verifier.verify(params[:token], purpose: :async_variant_callback)
32
+ ActiveStorage::VariantRecord.find(variant_record_id)
33
+ end
34
+ end
35
+
36
36
  def apply_reported_metadata(variant_record, params)
37
37
  reconcile(variant_record.image.blob, params[:byte_size], params[:checksum])
38
- reconcile(variant_record.blob, params[:preview_image_byte_size], params[:preview_image_checksum])
38
+ if frame?(variant_record.blob)
39
+ reconcile(variant_record.blob, params[:preview_image_byte_size], params[:preview_image_checksum])
40
+ end
39
41
  end
40
42
 
41
- # The placeholder sentinels (byte_size 0, checksum "0") gate each field, so
42
- # this is idempotent and never overwrites a real source blob's metadata.
43
+ # A re-render writes over a reused blob, so the reported values describe the object now stored.
43
44
  def reconcile(blob, byte_size, checksum)
44
45
  return unless blob
45
46
 
46
- attrs = {}
47
- if (bytes = positive_int(byte_size)) && blob.byte_size.zero?
48
- attrs[:byte_size] = bytes
47
+ if positive_int?(byte_size)
48
+ blob.byte_size = byte_size
49
49
  end
50
- if checksum.present? && checksum != "0" && blob.checksum == "0"
51
- attrs[:checksum] = checksum
50
+ if checksum.present?
51
+ blob.checksum = checksum
52
52
  end
53
- blob.update!(attrs) if attrs.any?
53
+ blob.save!
54
+
55
+ if blob.checksum != "0"
56
+ blob.mirror_later
57
+ end
58
+ end
59
+
60
+ # preview_image_* must never land on a variant's source blob.
61
+ def frame?(blob)
62
+ blob.checksum == "0" ||
63
+ ActiveStorage::Attachment.exists?(record_type: "ActiveStorage::Blob", name: "preview_image", blob_id: blob.id)
54
64
  end
55
65
 
56
- def positive_int(value)
57
- int = value.to_i
58
- int.positive? ? int : nil
66
+ def positive_int?(value)
67
+ value.to_i.positive?
59
68
  end
60
69
  end
61
70
  end
data/config/routes.rb CHANGED
@@ -4,21 +4,4 @@ Rails.application.routes.draw do
4
4
  post "/active_storage/async_variants/callbacks/:token",
5
5
  to: "active_storage/async_variants/callbacks#create",
6
6
  as: :active_storage_async_variant_callback
7
-
8
- get "/active_storage/async_variants/states/:signed_blob_id/:variation_key",
9
- to: "active_storage/async_variants/states#show",
10
- as: :async_variant_state
11
- post "/active_storage/async_variants/states/:signed_blob_id/:variation_key/retry",
12
- to: "active_storage/async_variants/states#retry",
13
- as: :async_variant_state_retry
14
-
15
- # Tiny 1x1 GIF whose URL carries the media's filename, so the processing/failed
16
- # box reserves layout without fetching the original through a representation.
17
- get "/active_storage/async_variants/placeholder/:filename",
18
- to: "active_storage/async_variants/states#placeholder",
19
- as: :async_variant_placeholder,
20
- constraints: { filename: %r{[^/]+} },
21
- format: false
22
-
23
- ActiveStorage::AsyncVariants::Assets.draw(self, "/active_storage/async_variants/assets")
24
7
  end
@@ -0,0 +1,9 @@
1
+ # frozen_string_literal: true
2
+
3
+ class AddStateColumnsToVariantRecords < ActiveRecord::Migration[7.2]
4
+ def change
5
+ add_column :active_storage_variant_records, :state, :string, default: "pending", if_not_exists: true
6
+ add_column :active_storage_variant_records, :error, :text, if_not_exists: true
7
+ add_column :active_storage_variant_records, :attempts, :integer, default: 0, if_not_exists: true
8
+ end
9
+ end
@@ -10,9 +10,5 @@ gem "simplecov", require: false
10
10
  gem "rspec-rails"
11
11
  gem "rails", "~> 7.2.0"
12
12
  gem "sqlite3"
13
- gem "cucumber"
14
- gem "capybara"
15
- gem "cuprite"
16
- gem "puma"
17
13
 
18
14
  gemspec path: "../"
@@ -10,9 +10,5 @@ gem "simplecov", require: false
10
10
  gem "rspec-rails"
11
11
  gem "rails", "~> 8.0.0"
12
12
  gem "sqlite3"
13
- gem "cucumber"
14
- gem "capybara"
15
- gem "cuprite"
16
- gem "puma"
17
13
 
18
14
  gemspec path: "../"
@@ -10,9 +10,5 @@ gem "simplecov", require: false
10
10
  gem "rspec-rails"
11
11
  gem "rails", "~> 8.1.0"
12
12
  gem "sqlite3"
13
- gem "cucumber"
14
- gem "capybara"
15
- gem "cuprite"
16
- gem "puma"
17
13
 
18
14
  gemspec path: "../"
@@ -5,22 +5,39 @@ module ActiveStorage
5
5
  module AttachmentExtension
6
6
  private
7
7
 
8
+ # Stock would also preprocess an async variant: a TransformJob, or a PreviewImageJob that runs the local previewer.
8
9
  def transform_variants_later
9
- super
10
+ without_async_variants { super }
10
11
  enqueue_async_variant_jobs
11
12
  end
12
13
 
14
+ def named_variants
15
+ return super unless @without_async_variants
16
+ super.reject { |_name, named_variant| async_variant?(named_variant) }
17
+ end
18
+
19
+ def without_async_variants
20
+ @without_async_variants = true
21
+ yield
22
+ ensure
23
+ @without_async_variants = false
24
+ end
25
+
13
26
  def enqueue_async_variant_jobs
14
27
  return unless blob.bucket_backed?
15
28
 
16
29
  named_variants.each do |name, named_variant|
17
- next unless named_variant.transformations.key?(:async)
30
+ next unless async_variant?(named_variant) && named_variant.preprocessed?(record)
18
31
 
19
32
  ActiveStorage::AsyncVariants::ProcessJob.perform_later(
20
33
  record, self.name, name.to_s,
21
34
  )
22
35
  end
23
36
  end
37
+
38
+ def async_variant?(named_variant)
39
+ named_variant.transformations.key?(:transformer)
40
+ end
24
41
  end
25
42
  end
26
43
  end
@@ -7,8 +7,10 @@ module ActiveStorage
7
7
  # processing workers (Crucible, etc.) can reach via presigned URLs. The
8
8
  # Disk and Test services return false; the gem then defers to vanilla
9
9
  # ActiveStorage rather than trying to enqueue jobs or serve fallbacks.
10
+ # MirrorService doesn't delegate :bucket, so resolve through its primary.
10
11
  def bucket_backed?
11
- service.respond_to?(:bucket)
12
+ resolved = service.respond_to?(:primary) ? service.primary : service
13
+ resolved.respond_to?(:bucket)
12
14
  end
13
15
  end
14
16
  end
@@ -0,0 +1,55 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ActiveStorage
4
+ module AsyncVariants
5
+ # Finds the async named-variant declaration whose transformations match a
6
+ # variation, starting from a blob's attachments. Used to recover the
7
+ # :transformer option when the Registry is cold, and by
8
+ # Preview/VariantWithRecord enqueue paths to locate the attachment to
9
+ # dispatch ProcessJob against.
10
+ #
11
+ # Walks one level up through a preview_image attachment so a lookup from a
12
+ # video's extracted frame still finds the named variant declared on the
13
+ # parent record's source-video field.
14
+ #
15
+ # Candidates are compared structurally (order-insensitive, JSON-normalized)
16
+ # against the declared transformations, never by instantiating
17
+ # attachment.variant -- which raises InvariableError for previewable blobs.
18
+ module NamedVariantScan
19
+ module_function
20
+
21
+ def find(blob, variation, depth: 0)
22
+ blob.attachments.each do |attachment|
23
+ if attachment.name == "preview_image" && attachment.record_type == "ActiveStorage::Blob" && depth < 1
24
+ source = ActiveStorage::Blob.find_by(id: attachment.record_id)
25
+ result = source && find(source, variation, depth: depth + 1)
26
+ return result if result
27
+ next
28
+ end
29
+
30
+ attachment.send(:named_variants).each do |name, named_variant|
31
+ candidate = ActiveStorage::Variation.wrap(named_variant.transformations)
32
+ next unless candidate.async_options[:transformer]
33
+ return [attachment, name, candidate.async_options] if matches?(candidate, variation)
34
+ end
35
+ end
36
+ nil
37
+ end
38
+
39
+ # A format-less declaration matches any target format, because runtime
40
+ # resolution (Blob#variant's default_to) adds a blob-dependent format:
41
+ # the declaration never carries.
42
+ def matches?(candidate, variation)
43
+ declared = normalize(candidate.transformations)
44
+ target = normalize(variation.transformations)
45
+ declared_format = declared.delete("format")
46
+ target_format = target.delete("format")
47
+ declared == target && (declared_format.nil? || declared_format == target_format)
48
+ end
49
+
50
+ def normalize(transformations)
51
+ JSON.parse(transformations.to_json)
52
+ end
53
+ end
54
+ end
55
+ end