carrierwave 4.0.0.beta → 4.0.0.rc

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 44d2471aedac47f453dcf2e0b0b3a36657ff89a1cd521875528a118cdfc465f4
-  data.tar.gz: 4de4a19f9c7b959d8bfd3658c8ee672b03227a5c74d96ec86cbca64ab692d726
+  metadata.gz: 663f843bb0d228ae33a1afa9bf60c466ba068f7f13e5bd8cfa505de59029a34d
+  data.tar.gz: d6cdd19287a021a50268c48e2d4be4acf9bf1ed14680ff12c83c29226d7a7652
 SHA512:
-  metadata.gz: 5f15fe1cc5ca10c246398c23bfa6d882494434dde71f40c4d044d949fae30e0929ade0d928ae951b028d2c9821538ae3d1e18a71c43f17e7a5256ecc19f4b8ea
-  data.tar.gz: 675258a32e180790066ea27d8d12aad87e78c607e51125afc0c24fef5eab42588f7a2dba4d54967ee28b52ae7d4921bf408801ee6ab504ebac05d91ef907cdbd
+  metadata.gz: 4eee63b300dfe7ce3d4f2061bb8681b0c23c6b434ab45fe19e015a1cb6a8bfd19c7a84a30f0454c84d23df843ce51e71498cde0b4fef671f941d94ee8e1160f5
+  data.tar.gz: e77c6b1e21ecf67d074b8fc893b4d762c3fca35fe69014bb9504bb98992ca49858ee8ad060b9e8905d897e18dd9169be0ef12ad6f640e4a02abd4e854f414e14

data/README.md

@@ -35,6 +35,23 @@ gem 'carrierwave', '~> 3.0'
 
 Finally, restart the server to apply the changes.
 
+### Supported Ruby and Rails versions
+
+Each release is tested against the versions below. The gemspec only states a
+minimum, so Bundler will happily install an old CarrierWave alongside a Rails
+newer than it has ever seen — which is how a patch-level upgrade ends up failing
+at boot. Check this table when upgrading Rails.
+
+| CarrierWave | Rails      | Ruby     |
+|-------------|------------|----------|
+| 4.0         | 7.0 – 8.1  | >= 2.7   |
+| 3.1         | 6.0 – 8.0  | >= 2.5   |
+| 3.0         | 6.0 – 7.1  | >= 2.5   |
+| 2.2         | 5.0 – 6.1  | >= 2.2.2 |
+
+Only the latest release gets fixes; older lines are listed so you can tell
+whether your combination was ever supported.
+
 ## Upgrading from 2.x or earlier
 
 CarrierWave 3.0 comes with a change in the way of handling the file extension on conversion. This results in following issues if you use `process convert: :format` to change the file format:
@@ -159,6 +176,14 @@ example, create a migration like this:
 
 __Note__: JSON datatype doesn't exists in SQLite adapter, that's why you can use a string datatype which will be serialized in model.
 
+Leave the column nullable: `NULL` is what it holds when no file is mounted, so a `default: []` or `null: false` on it doesn't hold — the `NULL` is written on save. The uploader's own readers answer an empty array either way, and are what to read:
+
+```ruby
+user.avatars             # => [] when there is no file
+user.avatars_identifiers # => []
+user[:avatars]           # => nil, the column as it is stored
+```
+
 Open your model file and mount the uploader:
 
 
@@ -223,7 +248,7 @@ end
 This works for the file storage as well as Amazon S3 and Rackspace Cloud Files.
 Define `store_dir` as `nil` if you'd like to store files at the root level.
 
-If you store files outside the project root folder, you may want to define `cache_dir` in the same way:
+With the `:file` storage, if you store files outside the project root folder, you may want to define `cache_dir` in the same way:
 
 ```ruby
 class MyUploader < CarrierWave::Uploader::Base
@@ -245,6 +270,14 @@ class MyUploader < CarrierWave::Uploader::Base
 end
 ```
 
+The name is worked out again whenever the stored file is located, so it has to come out the same every time. Memoise anything that would otherwise differ per call, in an instance variable of your own, as `@filename` is the uploader's:
+
+```ruby
+def filename
+  @stable_filename ||= "#{SecureRandom.uuid}.#{file.extension}"
+end
+```
+
 Some old documentations (like [this](https://stackoverflow.com/a/5865117)) may instruct you to safeguard the filename value with `if original_filename`, but it's no longer necessary with CarrierWave 3.0 or later.
 
 ## Securing uploads
@@ -637,6 +670,57 @@ CarrierWave.configure do |config|
 end
 ```
 
+## Uploading directly to the storage
+
+When the file is stored on a cloud service, it can be uploaded straight to it,
+without the bytes passing through your application. `#direct_upload` tells the
+client where to put the file, and under which name to hand it back afterwards:
+
+```ruby
+class UploadsController < ApplicationController
+  def create
+    upload = User.new.avatar.direct_upload(filename: params[:filename])
+    render json: upload.to_h
+    # => { url: "https://mybucket.s3.amazonaws.com/uploads/tmp/1758…-4829/cat.jpg?X-Amz-…",
+    #      method: "PUT", headers: {}, cache_name: "1758…-4829/cat.jpg" }
+  end
+end
+```
+
+The file goes where the cache expects it, so the upload is finished by assigning
+the returned name to the `avatar_cache` field you already have for form
+redisplays. In the browser:
+
+```js
+const upload = await (await fetch(`/uploads?filename=${encodeURIComponent(file.name)}`)).json();
+await fetch(upload.url, { method: upload.method, headers: upload.headers, body: file });
+document.querySelector('#user_avatar_cache').value = upload.cache_name;
+// then submit the form as usual
+```
+
+The file is stored where it belongs when the record is saved, as any cached file
+is. Libraries which upload for you take the same three values: with
+[Uppy](https://uppy.io/docs/aws-s3/), return `upload.to_h` from
+`getUploadParameters`.
+
+The cache has to be on the cloud service for this, which it is when `storage` is
+set to `:fog` and `cache_storage` is left alone. A few more things to know:
+
+* the filename is sanitized before it is signed into the URL, so the name the
+  file is stored under is the one in `cache_name`, not the one you passed;
+* the upload stays possible for `fog_authenticated_url_expiration` seconds, which
+  `expires_in:` overrides;
+* the size cannot be limited by the URL. Check it in the client and again with
+  `size_range` when the file comes back, and let `clean_cached_files!` or a
+  lifecycle rule on the cache directory collect what was uploaded and never used;
+* pass `content_type:` to fix the type the client has to send. It is signed into
+  the URL and returned in `headers`, so the client has to send exactly that;
+* uploaders which process what they store, or have versions, refuse to hand out
+  an upload: neither can be done to a file the application never sees.
+
+Your storage service has to allow the upload to be made from the browser, which
+for S3 means a CORS rule allowing `PUT` from your origin.
+
 ## Providing a default URL
 
 In many cases, especially when working with images, it might be a good idea to
@@ -1021,6 +1105,10 @@ end
 If you're uploading images, you'll probably want to manipulate them in some way,
 you might want to create thumbnail images for example.
 
+The processing library itself is not a dependency of CarrierWave, so add the one
+your uploaders use to your own Gemfile — `mini_magick`, `ruby-vips` or `rmagick`
+— along with the tool it drives (ImageMagick or libvips).
+
 ### Using MiniMagick
 
 MiniMagick performs all the operations using the 'convert' CLI which is part of the standard ImageMagick kit.
@@ -1099,10 +1187,24 @@ class ImageFileUploader < CarrierWave::Uploader::Base
 end
 ````
 
+For anything the methods below don't cover, `vips!` yields the [ImageProcessing](https://github.com/janko/image_processing/blob/master/doc/vips.md) builder (`minimagick!` does the same for MiniMagick). For example, to strip the metadata and set the quality:
+
+````ruby
+class ImageFileUploader < CarrierWave::Uploader::Base
+  include CarrierWave::Vips
+
+  process :optimize
+
+  def optimize
+    vips! { |builder| builder.saver(strip: true, quality: 90) }
+  end
+end
+````
+
 ### List of available processing methods:
 
 > [!NOTE]
-> While the intention is to provide uniform interfaces to all three processing libraries the availability and implementation of processing methods can <a href="supported-processing-methods">vary slightly between them</a>.
+> While the intention is to provide uniform interfaces to all three processing libraries the availability and implementation of processing methods can <a href="#supported-processing-methods">vary slightly between them</a>.
 
 - `convert` - Changes the image encoding format to the given format (eg. jpg). This operation is treated specially to trigger the change of the file extension, so it matches with the format of the resulting file.
 - `resize_to_limit` - Resize the image to fit within the specified dimensions while retaining the original aspect ratio. Will only resize the image if it is larger than the specified dimensions. The resulting image may be shorter or narrower than specified in the smaller dimension but will not be larger than the specified values.

data/lib/carrierwave/direct_upload.rb

@@ -0,0 +1,20 @@
+module CarrierWave
+  ##
+  # What a client needs to upload a file straight into the cache. See
+  # CarrierWave::Uploader::DirectUpload#direct_upload.
+  #
+  class DirectUpload
+    attr_reader :url, :method, :headers, :cache_name
+
+    def initialize(url:, cache_name:, headers: {}, method: 'PUT')
+      @url = url
+      @cache_name = cache_name
+      @headers = headers
+      @method = method
+    end
+
+    def to_h
+      { url: url, method: method, headers: headers, cache_name: cache_name }
+    end
+  end # DirectUpload
+end # CarrierWave

data/lib/carrierwave/downloader/base.rb

@@ -40,12 +40,13 @@ module CarrierWave
             response = OpenURI.open_uri(uri, headers)
           else
             request = nil
+            ssrf_options = @uploader.ssrf_filter_options || {}
             if ::SsrfFilter::VERSION.to_f < 1.1
-              response = SsrfFilter.get(uri, headers: headers) do |req|
+              response = SsrfFilter.get(uri, headers: headers, **ssrf_options) do |req|
                 request = req
               end
             else
-              response = SsrfFilter.get(uri, headers: headers, request_proc: ->(req) { request = req }) do |res|
+              response = SsrfFilter.get(uri, headers: headers, request_proc: ->(req) { request = req }, **ssrf_options) do |res|
                 res.body # ensure to read body
               end
             end

data/lib/carrierwave/locale/en.yml

@@ -10,6 +10,7 @@ en:
       content_type_denylist_error: "You are not allowed to upload %{content_type} files"
       processing_error: "Failed to manipulate, maybe it is not an image?"
       unsupported_image_format_error: "Failed to manipulate, file format is not supported"
+      filename_too_long_error: "File name is too long"
       min_size_error: "File size should be greater than %{min_size}"
       max_size_error: "File size should be less than %{max_size}"
       min_width_error: "Image width should be greater than %{min_width}px"

data/lib/carrierwave/mount.rb

@@ -175,6 +175,10 @@ module CarrierWave
           _mounter(:#{column}).read_identifiers[0]
         end
 
+        def #{column}_identifier=(identifier)
+          _mounter(:#{column}).identifiers = [identifier]
+        end
+
         def #{column}_integrity_error
           #{column}_integrity_errors.last
         end
@@ -321,6 +325,10 @@ module CarrierWave
         def #{column}_identifiers
           _mounter(:#{column}).read_identifiers
         end
+
+        def #{column}_identifiers=(identifiers)
+          _mounter(:#{column}).identifiers = identifiers
+        end
       RUBY
     end
 

data/lib/carrierwave/mounter.rb

@@ -102,7 +102,7 @@ module CarrierWave
     def cache(new_files)
       return if !new_files.is_a?(Array) && new_files.blank?
       old_uploaders = uploaders
-      @uploaders = new_files.map do |new_file|
+      attempts = new_files.map do |new_file|
         handle_error do
           if new_file.is_a?(String)
             if (uploader = old_uploaders.detect { |old_uploader| old_uploader.identifier == new_file })
@@ -123,7 +123,29 @@ module CarrierWave
             uploader
           end
         end
-      end.reject(&:blank?)
+      end
+      errors, new_uploaders = attempts.partition { |attempt| attempt.is_a?(CarrierWave::UploadError) }
+      # A refusal didn't ask for the mounted file to go
+      return if errors.any? && new_uploaders.all?(&:blank?)
+
+      @uploaders = new_uploaders.reject(&:blank?)
+      @removed_uploaders += (old_uploaders - @uploaders)
+      write_temporary_identifier
+    end
+
+    ##
+    # Takes the stored files with the given identifiers as the mounted ones, for when
+    # they were put in the store by something other than CarrierWave.
+    #
+    def identifiers=(new_identifiers)
+      return if !new_identifiers.is_a?(Array) && new_identifiers.blank?
+
+      old_uploaders = uploaders
+      @uploaders = new_identifiers.map do |identifier|
+        next if identifier.blank?
+
+        blank_uploader.tap { |uploader| uploader.adopt!(identifier) }
+      end.compact
       @removed_uploaders += (old_uploaders - @uploaders)
       write_temporary_identifier
     end
@@ -158,14 +180,20 @@ module CarrierWave
       @download_errors.clear
       @remote_urls = urls
 
-      clear_unstaged
-      @remote_urls.zip(remote_request_headers || []) do |url, header|
+      old_uploaders = uploaders
+      attempts = @remote_urls.zip(remote_request_headers || []).map do |url, header|
         handle_error do
           uploader = blank_uploader
           uploader.download!(url, header || {})
-          @uploaders << uploader
+          uploader
         end
       end
+      errors, new_uploaders = attempts.partition { |attempt| attempt.is_a?(CarrierWave::UploadError) }
+      # A refusal didn't ask for the mounted file to go
+      return if errors.any? && new_uploaders.all?(&:blank?)
+
+      @uploaders = old_uploaders.select(&:staged) + new_uploaders.reject(&:blank?)
+      @removed_uploaders += (old_uploaders - @uploaders)
       write_temporary_identifier
     end
 
@@ -209,7 +237,8 @@ module CarrierWave
     end
 
     def remove!
-      uploaders.each(&:remove!)
+      # The reader leaves a blank uploader in place, which has nothing to remove
+      uploaders.reject { |uploader| uploader.file.nil? }.each(&:remove!)
       clear!
     end
 
@@ -235,7 +264,7 @@ module CarrierWave
     def remove_previous
       current_paths = uploaders.map(&:path)
       @removed_uploaders
-        .reject {|uploader| current_paths.include?(uploader.path) }
+        .reject {|uploader| uploader.file.nil? || current_paths.include?(uploader.path) }
         .each { |uploader| uploader.remove! if uploader.remove_previously_stored_files_after_update }
       reset_changes!
     end
@@ -262,17 +291,21 @@ module CarrierWave
       @removed_uploaders += unstaged
     end
 
+    # Records the error and gives it back, unless the mount wants it raised
     def handle_error
       yield
     rescue CarrierWave::DownloadError => e
       @download_errors << e
       raise e unless option(:ignore_download_errors)
+      e
     rescue CarrierWave::ProcessingError => e
       @processing_errors << e
       raise e unless option(:ignore_processing_errors)
+      e
     rescue CarrierWave::IntegrityError => e
       @integrity_errors << e
       raise e unless option(:ignore_integrity_errors)
+      e
     end
 
     def write_temporary_identifier

data/lib/carrierwave/orm/activerecord.rb

@@ -6,6 +6,38 @@ module CarrierWave
 
     include CarrierWave::Mount
 
+    ##
+    # See +CarrierWave::Mount#mount_uploader+ for documentation
+    #
+    def mount_uploader(column, uploader=nil, options={}, &block)
+      super
+
+      mod = Module.new
+      prepend mod
+      mod.class_eval <<-RUBY, __FILE__, __LINE__+1
+        def remote_#{column}_url=(url)
+          super
+          mark_#{column}_changed_on_error
+        end
+      RUBY
+    end
+
+    ##
+    # See +CarrierWave::Mount#mount_uploaders+ for documentation
+    #
+    def mount_uploaders(column, uploader=nil, options={}, &block)
+      super
+
+      mod = Module.new
+      prepend mod
+      mod.class_eval <<-RUBY, __FILE__, __LINE__+1
+        def remote_#{column}_urls=(urls)
+          super
+          mark_#{column}_changed_on_error
+        end
+      RUBY
+    end
+
   private
 
     def mount_base(column, uploader=nil, options={}, &block)
@@ -62,6 +94,22 @@ module CarrierWave
           return unless has_attribute?(_mounter(:#{column}).serialization_column)
           super
         end
+
+        def #{column}=(new_file)
+          super
+          mark_#{column}_changed_on_error
+        end
+
+      private
+
+        # A refusal leaves the column as it was, and a record saved through an
+        # association is only validated when it changed, so say it changed
+        def mark_#{column}_changed_on_error
+          return if #{column}_integrity_errors.empty? && #{column}_processing_errors.empty? &&
+            #{column}_download_errors.empty?
+
+          __send__(:"\#{_mounter(:#{column}).serialization_column}_will_change!")
+        end
       RUBY
     end
 

data/lib/carrierwave/processing/mini_magick.rb

@@ -96,7 +96,7 @@ module CarrierWave
     #
     # === Yields
     #
-    # [MiniMagick::Image] additional manipulations to perform
+    # [MiniMagick::Image] additional manipulations to perform (deprecated, use #manipulate!)
     #
     # === Examples
     #
@@ -124,7 +124,7 @@ module CarrierWave
     #
     # === Yields
     #
-    # [MiniMagick::Image] additional manipulations to perform
+    # [MiniMagick::Image] additional manipulations to perform (deprecated, use #manipulate!)
     #
     def resize_to_limit(width, height, combine_options: {}, &block)
       width, height = resolve_dimensions(width, height)
@@ -148,7 +148,7 @@ module CarrierWave
     #
     # === Yields
     #
-    # [MiniMagick::Image] additional manipulations to perform
+    # [MiniMagick::Image] additional manipulations to perform (deprecated, use #manipulate!)
     #
     def resize_to_fit(width, height, combine_options: {}, &block)
       width, height = resolve_dimensions(width, height)
@@ -173,7 +173,7 @@ module CarrierWave
     #
     # === Yields
     #
-    # [MiniMagick::Image] additional manipulations to perform
+    # [MiniMagick::Image] additional manipulations to perform (deprecated, use #manipulate!)
     #
     def resize_to_fill(width, height, gravity = 'Center', combine_options: {}, &block)
       width, height = resolve_dimensions(width, height)
@@ -203,7 +203,7 @@ module CarrierWave
     #
     # === Yields
     #
-    # [MiniMagick::Image] additional manipulations to perform
+    # [MiniMagick::Image] additional manipulations to perform (deprecated, use #manipulate!)
     #
     def resize_and_pad(width, height, background=:transparent, gravity='Center', combine_options: {}, &block)
       width, height = resolve_dimensions(width, height)
@@ -228,7 +228,7 @@ module CarrierWave
     #
     # === Yields
     #
-    # [MiniMagick::Image] additional manipulations to perform
+    # [MiniMagick::Image] additional manipulations to perform (deprecated, use #manipulate!)
     #
     def crop(left, top, width, height, combine_options: {}, &block)
       width, height = resolve_dimensions(width, height)
@@ -290,6 +290,7 @@ module CarrierWave
 
       image = yield(image)
       FileUtils.mv image.path, current_path
+      follow_extension_of image.path
 
       ::MiniMagick::Image.new(current_path).identify
     rescue ::MiniMagick::Error, ::MiniMagick::Invalid => e
@@ -325,20 +326,20 @@ module CarrierWave
       result = builder.call
       result.close
 
-      # backwards compatibility (we want to eventually move away from MiniMagick::Image)
       if block
+        unless @minimagick_image_block_warned
+          @minimagick_image_block_warned = true
+          CarrierWave.deprecator.warn "Passing a block to receive a MiniMagick::Image is deprecated. " \
+                                      "Use #manipulate! to work with a MiniMagick::Image, or #minimagick! to build on the ImageProcessing builder."
+        end
+
         image  = ::MiniMagick::Image.new(result.path, result)
         image  = block.call(image)
         result = image.instance_variable_get(:@tempfile)
       end
 
       FileUtils.mv result.path, current_path
-
-      if File.extname(result.path) != File.extname(current_path)
-        move_to = current_path.chomp(File.extname(current_path)) + File.extname(result.path)
-        file.content_type = Marcel::Magic.by_path(move_to).try(:type)
-        file.move_to(move_to, permissions, directory_permissions)
-      end
+      follow_extension_of result.path
     rescue ::MiniMagick::Error, ::MiniMagick::Invalid => e
       raise e if e.message =~ /(You must have .+ installed|is not installed|executable not found)/
       message = I18n.translate(:"errors.messages.processing_error")
@@ -347,6 +348,16 @@ module CarrierWave
 
   private
 
+    # The processing may have changed the format, in which case the file is renamed so
+    # that its extension keeps telling what it actually holds
+    def follow_extension_of(processed_path)
+      return if File.extname(processed_path) == File.extname(current_path)
+
+      move_to = current_path.chomp(File.extname(current_path)) + File.extname(processed_path)
+      file.content_type = Marcel::Magic.by_path(move_to).try(:type)
+      file.move_to(move_to, permissions, directory_permissions)
+    end
+
     def resolve_dimensions(*dimensions)
       dimensions.map do |value|
         next value unless value.instance_of?(Proc)

data/lib/carrierwave/processing/vips.rb

@@ -93,10 +93,6 @@ module CarrierWave
     #
     # [format (#to_s)] an abbreviation of the format
     #
-    # === Yields
-    #
-    # [Vips::Image] additional manipulations to perform
-    #
     # === Examples
     #
     #     image.convert(:png)
@@ -121,10 +117,6 @@ module CarrierWave
     # [height (Integer)] the height to scale the image to
     # [combine_options (Hash)] additional Vips options to apply before resizing
     #
-    # === Yields
-    #
-    # [Vips::Image] additional manipulations to perform
-    #
     def resize_to_limit(width, height, combine_options: {})
       width, height = resolve_dimensions(width, height)
 
@@ -145,10 +137,6 @@ module CarrierWave
     # [height (Integer)] the height to scale the image to
     # [combine_options (Hash)] additional Vips options to apply before resizing
     #
-    # === Yields
-    #
-    # [Vips::Image] additional manipulations to perform
-    #
     def resize_to_fit(width, height, combine_options: {})
       width, height = resolve_dimensions(width, height)
 
@@ -169,10 +157,6 @@ module CarrierWave
     # [height (Integer)] the height to scale the image to
     # [combine_options (Hash)] additional vips options to apply before resizing
     #
-    # === Yields
-    #
-    # [Vips::Image] additional manipulations to perform
-    #
     def resize_to_fill(width, height, _gravity = nil, combine_options: {})
       width, height = resolve_dimensions(width, height)
 
@@ -199,10 +183,6 @@ module CarrierWave
     # [alpha (Boolean, nil)] pad the image with the alpha channel if supported
     # [combine_options (Hash)] additional vips options to apply before resizing
     #
-    # === Yields
-    #
-    # [Vips::Image] additional manipulations to perform
-    #
     def resize_and_pad(width, height, background=nil, gravity='centre', alpha=nil, combine_options: {})
       width, height = resolve_dimensions(width, height)
 
@@ -224,10 +204,6 @@ module CarrierWave
     # [width (Integer)] width of area to extract
     # [height (Integer)] height of area to extract
     #
-    # === Yields
-    #
-    # [Vips::Image] additional manipulations to perform
-    #
     def crop(left, top, width, height, combine_options: {})
       width, height = resolve_dimensions(width, height)
       width = vips_image.width - left if width + left > vips_image.width

data/lib/carrierwave/sanitized_file.rb

@@ -53,7 +53,7 @@ module CarrierWave
     # [String] the sanitized filename
     #
     def filename
-      sanitize(original_filename) if original_filename
+      sanitize(original_filename.to_s)
     end
 
     alias_method :identifier, :filename
@@ -305,7 +305,7 @@ module CarrierWave
 
     # Sanitize the filename, to prevent hacking
     def sanitize(name)
-      name = name.scrub
+      name = name.scrub.delete("\0") # a NUL byte cuts the path short, and File.basename refuses one
       name = name.tr("\\", "/") # work-around for IE
       name = File.basename(name)
       name = name.gsub(sanitize_regexp, "_")

data/lib/carrierwave/storage/abstract.rb

@@ -36,6 +36,10 @@ module CarrierWave
         file
       end
 
+      def direct_upload_url(path, expires_in:, headers: {})
+        raise NotImplementedError, "Need to implement #direct_upload_url if you want to use #{self.class.name} for direct uploads."
+      end
+
       def delete_dir!(path)
         raise NotImplementedError, "Need to implement #delete_dir! if you want to use #{self.class.name} as a cache storage."
       end

data/lib/carrierwave/storage/file.rb

@@ -110,10 +110,8 @@ module CarrierWave
 
       def clean_cache!(seconds)
         Dir.glob(::File.expand_path(::File.join(uploader.cache_dir, '*'), uploader.root)).each do |dir|
-          # generate_cache_id returns key formatted TIMEINT-PID(-COUNTER)-RND
-          matched = dir.scan(/(\d+)-\d+-\d+(?:-\d+)?/).first
-          next unless matched
-          time = Time.at(matched[0].to_i)
+          time = uploader.parse_cache_id(::File.basename(dir))
+          next unless time
           if time < (Time.now.utc - seconds)
             FileUtils.rm_rf(dir)
           end

data/lib/carrierwave/storage/fog.rb

@@ -12,6 +12,13 @@ module CarrierWave
     # [:fog_directory]    specifies name of directory to store data in, assumed to already exist
     #
     # [:fog_attributes]                   (optional) additional attributes to set on files
+    # [:fog_acl]                          (optional) overrides the ACL set on stored/copied files.
+    #   When nil (default), ACL is derived from fog_public (:public attribute passed to fog).
+    #   When false, no ACL is set at all (omits :public; use for buckets/containers with ACLs disabled).
+    #   When a String, used as the explicit ACL value: passed as x-amz-acl for AWS
+    #   (e.g. 'bucket-owner-full-control') or destination_predefined_acl for Google
+    #   (e.g. 'bucketOwnerFullControl'). For store operations, pair with fog_attributes to
+    #   set the provider-specific header.
     # [:fog_public]                       (optional) public readability, defaults to true
     # [:fog_authenticated_url_expiration] (optional) time (in seconds) that authenticated urls
     #   will be valid, when fog_public is false and provider is AWS or Google, defaults to 600
@@ -168,20 +175,32 @@ module CarrierWave
         local_storage.delete_dir!(path)
       end
 
+      ##
+      # Returns a URL a client can PUT the file to, to cache it without the bytes
+      # passing through the application.
+      #
+      def direct_upload_url(path, expires_in:, headers: {})
+        connection.put_object_url(uploader.fog_directory, path, ::Fog::Time.now.since(expires_in), headers)
+      end
+
       def clean_cache!(seconds)
         local_storage.clean_cache!(seconds)
-        connection.directories.new(
-          :key    => uploader.fog_directory,
-          :public => uploader.fog_public
-        ).files.all(:prefix => uploader.cache_dir).each do |file|
-          # generate_cache_id returns key formatted TIMEINT-PID(-COUNTER)-RND
-          matched = file.key.match(/(\d+)-\d+-\d+(?:-\d+)?/)
-          next unless matched
-          time = Time.at(matched[1].to_i)
+        directory = connection.directories.new(fog_public_attrs.merge(:key => uploader.fog_directory))
+        prefix = "#{uploader.cache_dir.to_s.chomp('/')}/"
+        directory.files.all(:prefix => prefix).each do |file|
+          # the prefix is checked again, as some providers may not honor it when listing
+          next unless file.key.start_with?(prefix)
+
+          time = uploader.parse_cache_id(file.key.delete_prefix(prefix).split('/').first)
+          next unless time
           file.destroy if time < (Time.now.utc - seconds)
         end
       end
 
+      def fog_public_attrs
+        uploader.fog_acl.nil? ? { :public => uploader.fog_public } : {}
+      end
+
       def connection
         @connection ||= begin
           options = credentials = uploader.fog_credentials
@@ -368,12 +387,11 @@ module CarrierWave
           else
             fog_file = new_file.to_file
             @content_type ||= new_file.content_type
-            @file = directory.files.create({
+            @file = directory.files.create(fog_public_attrs.merge(
               :body         => fog_file || new_file.read,
               :content_type => @content_type,
-              :key          => path,
-              :public       => @uploader.fog_public
-            }.merge(@uploader.fog_attributes))
+              :key          => path
+            ).merge(@uploader.fog_attributes))
             fog_file.close if fog_file && !fog_file.closed?
           end
           true
@@ -469,7 +487,8 @@ module CarrierWave
         end
 
         ##
-        # Creates a copy of this file and returns it.
+        # Creates a copy of this file and returns it. Copying to where the file already is
+        # leaves it alone, as S3 refuses a copy onto the same key.
         #
         # === Parameters
         #
@@ -480,7 +499,7 @@ module CarrierWave
         # @return [CarrierWave::Storage::Fog::File] the location where the file will be stored.
         #
         def copy_to(new_path)
-          file.copy(@uploader.fog_directory, new_path, copy_options)
+          file.copy(@uploader.fog_directory, new_path, copy_options) unless new_path == path
           CarrierWave::Storage::Fog::File.new(@uploader, @base, new_path)
         end
 
@@ -524,10 +543,7 @@ module CarrierWave
         # [Fog::#{provider}::Directory] containing directory
         #
         def directory
-          @directory ||= connection.directories.new(
-            :key    => @uploader.fog_directory,
-            :public => @uploader.fog_public
-          )
+          @directory ||= connection.directories.new(fog_public_attrs.merge(:key => @uploader.fog_directory))
         end
 
         ##
@@ -551,11 +567,19 @@ module CarrierWave
         end
 
         def acl_header
+          fog_acl = @uploader.fog_acl
+          return {} if fog_acl == false
+
           case fog_provider
           when 'AWS'
-            { 'x-amz-acl' => @uploader.fog_public ? 'public-read' : 'private' }
+            acl_value = fog_acl || (@uploader.fog_public ? 'public-read' : 'private')
+            { 'x-amz-acl' => acl_value }
           when "Google"
-            @uploader.fog_public ? { destination_predefined_acl: "publicRead" } : {}
+            if fog_acl
+              { destination_predefined_acl: fog_acl }
+            else
+              @uploader.fog_public ? { destination_predefined_acl: "publicRead" } : {}
+            end
           else
             {}
           end
@@ -565,6 +589,10 @@ module CarrierWave
           @uploader.fog_credentials[:provider].to_s
         end
 
+        def fog_public_attrs
+          @uploader.fog_acl.nil? ? { :public => @uploader.fog_public } : {}
+        end
+
         def read_source_file
           source_file = to_file
           return unless source_file

data/lib/carrierwave/uploader/cache.rb

@@ -81,10 +81,9 @@ module CarrierWave
       end
 
       ##
-      # Caches the remotely stored file
-      #
-      # This is useful when about to process images. Most processing solutions
-      # require the file to be stored on the local filesystem.
+      # Caches the stored file, so that it can be processed. Most processing solutions
+      # require the file to be on the local filesystem. It is staged as any assigned
+      # file is, so the next save stores the result; use #read for a copy only to read.
       #
       def cache_stored_file!
         cache!
@@ -103,7 +102,29 @@ module CarrierWave
       # [String] a cache name, in the format TIMEINT-PID-COUNTER-RND/filename.txt
       #
       def cache_name
-        File.join(cache_id, original_filename) if cache_id && original_filename
+        File.join(*[cache_id, original_filename].compact) if cache_id
+      end
+
+      ##
+      # Tells when the given cache id was created, and by returning nil, that it is
+      # not one. Override to take cache ids which were made elsewhere.
+      #
+      # === Parameters
+      #
+      # [cache_id (String)] the cache id to look at
+      #
+      # === Returns
+      #
+      # [Time] when it was created
+      #   or
+      # [NilClass] if it isn't a cache id
+      #
+      def parse_cache_id(cache_id)
+        # Earlier version used 3 part cache_id. Thus we should allow for
+        # the cache_id to have both 3 part and 4 part formats.
+        return unless cache_id.to_s =~ /\A(-?\d+)-\d+(-\d{4})?-\d{4}\z/
+
+        Time.at($1.to_i)
       end
 
       ##
@@ -146,6 +167,9 @@ module CarrierWave
           with_callbacks(:cache, @file) do
             @file = cache_storage.cache!(@file)
           end
+        rescue Errno::ENAMETOOLONG
+          # How long a name may be is the filesystem's to say, so let it say it
+          raise CarrierWave::IntegrityError, I18n.translate(:"errors.messages.filename_too_long_error")
         ensure
           FileUtils.rm_rf(workfile_path(''))
         end
@@ -209,9 +233,7 @@ module CarrierWave
       attr_reader :original_filename
 
       def cache_id=(cache_id)
-        # Earlier version used 3 part cache_id. Thus we should allow for
-        # the cache_id to have both 3 part and 4 part formats.
-        raise CarrierWave::InvalidParameter, "invalid cache id" unless cache_id =~ /\A(-)?[\d]+\-[\d]+(\-[\d]{4})?\-[\d]{4}\z/
+        raise CarrierWave::InvalidParameter, "invalid cache id" unless parse_cache_id(cache_id)
         @cache_id = cache_id
       end
 
@@ -221,7 +243,8 @@ module CarrierWave
       end
 
       def cache_storage
-        @cache_storage ||= (self.class.cache_storage || self.class.storage).new(self)
+        # Falls back to this uploader's storage, which may not be the configured one
+        @cache_storage ||= self.class.cache_storage&.new(self) || storage
       end
 
       # We can override the full_original_filename method in other modules

data/lib/carrierwave/uploader/configuration.rb

@@ -28,6 +28,7 @@ module CarrierWave
 
         # fog
         add_deprecated_config :fog_provider
+        add_config :fog_acl
         add_config :fog_attributes
         add_config :fog_credentials
         add_config :fog_directory
@@ -50,6 +51,7 @@ module CarrierWave
         add_config :download_retry_count
         add_config :download_retry_wait_time
         add_config :skip_ssrf_protection
+        add_config :ssrf_filter_options
 
         # set default values
         reset_config
@@ -221,6 +223,7 @@ module CarrierWave
             config.download_retry_count = 0
             config.download_retry_wait_time = 5
             config.skip_ssrf_protection = false
+            config.ssrf_filter_options = {}
           end
         end
       end

data/lib/carrierwave/uploader/direct_upload.rb

@@ -0,0 +1,51 @@
+module CarrierWave
+  module Uploader
+    module DirectUpload
+      extend ActiveSupport::Concern
+
+      include CarrierWave::Uploader::Callbacks
+      include CarrierWave::Uploader::Configuration
+      include CarrierWave::Uploader::Cache
+
+      ##
+      # Returns what a client needs to put a file into the cache itself, without the
+      # bytes passing through the application. Assign the returned #cache_name to
+      # #{column}_cache once the client is done, as with any cached file.
+      #
+      # === Parameters
+      #
+      # [filename (String)] the name the file is to be cached under
+      # [expires_in (Integer)] seconds the upload stays possible, defaults to
+      #                        fog_authenticated_url_expiration
+      # [content_type (String)] the type the client has to send, if it is to be fixed
+      #
+      # === Returns
+      #
+      # [CarrierWave::DirectUpload] where and how to upload, and the name to hand back
+      #
+      def direct_upload(filename:, expires_in: nil, content_type: nil)
+        raise CarrierWave::InvalidParameter, "invalid filename" if filename.blank?
+
+        filename = CarrierWave::SanitizedFile.new(filename).filename
+
+        upload = nil
+        with_callbacks(:direct_upload, filename) do
+          self.cache_id = CarrierWave.generate_cache_id
+          self.original_filename = filename
+          headers = content_type ? { 'Content-Type' => content_type } : {}
+
+          upload = ::CarrierWave::DirectUpload.new(
+            url: cache_storage.direct_upload_url(
+              cache_path,
+              expires_in: expires_in || fog_authenticated_url_expiration,
+              headers: headers.dup # the storage may add its own
+            ),
+            headers: headers,
+            cache_name: cache_name
+          )
+        end
+        upload
+      end
+    end # DirectUpload
+  end # Uploader
+end # CarrierWave

data/lib/carrierwave/uploader/processing.rb

@@ -10,6 +10,7 @@ module CarrierWave
         self.processors = []
 
         before :cache, :process!
+        before :direct_upload, :refuse_direct_upload_of_processed!
       end
 
       module ClassMethods
@@ -74,7 +75,7 @@ module CarrierWave
             # the condition held cannot be known, so a conditional one is refused.
             if condition
               raise ArgumentError, "`process convert:` cannot be given `:if` or `:unless`. The file extension follows the conversion, and it has to be worked out again when the file is retrieved, where whether the condition held is not known. " \
-                                   "Convert unconditionally, or put the conversion in a version of its own, or take the naming over by overriding #full_filename."
+                                   "Convert unconditionally, or put the conversion in a version of its own, or convert in a processor of your own and have #filename answer the extension the file ended up with."
             end
 
             force_extension processor_args
@@ -120,6 +121,13 @@ module CarrierWave
 
     private
 
+      # Processing needs the file, which a direct upload never hands to the application
+      def refuse_direct_upload_of_processed!(filename)
+        return if !enable_processing || self.class.processors.empty?
+
+        raise "#{filename.inspect} can't be uploaded directly, as the processing of #{self.class} would be skipped"
+      end
+
       def forcing_extension(filename)
         if force_extension && filename
           Pathname.new(filename).sub_ext(".#{force_extension.to_s.delete_prefix('.')}").to_s

data/lib/carrierwave/uploader/store.rb

@@ -124,6 +124,27 @@ module CarrierWave
         end
       end
 
+      ##
+      # Takes the file already stored with the given identifier as its own, for when
+      # something other than CarrierWave put it there.
+      #
+      # === Parameters
+      #
+      # [identifier (String)] identifies the file in the store
+      #
+      # === Raises
+      #
+      # [CarrierWave::InvalidParameter] if the identifier points out of the store dir
+      #
+      def adopt!(identifier)
+        # The file gets removed when replaced, so it must not point out of the store dir
+        raise CarrierWave::InvalidParameter, "invalid identifier" if identifier.start_with?('/') || identifier.split('/').include?('..')
+
+        with_callbacks(:adopt, identifier) do
+          retrieve_from_store!(identifier)
+        end
+      end
+
       ##
       # Look for an identifier which doesn't collide with the given already-stored identifiers.
       # It is done by adding a index number as the suffix.
@@ -155,6 +176,10 @@ module CarrierWave
       # spared having to work the name out at all.
       #
       def ensure_the_filename_can_be_worked_out_again
+        if filename.blank?
+          raise CarrierWave::FilenameNotReproducible, "The file has no name to be stored under. " \
+            "#filename has to give one, as a cached file doesn't have to carry a name of its own."
+        end
         return if parent_version || metadata_recorded?
 
         for_file = deduplicated_filename

data/lib/carrierwave/uploader/versions.rb

@@ -116,6 +116,8 @@ module CarrierWave
         after :remove, :remove_versions!
         after :retrieve_from_cache, :retrieve_versions_from_cache!
         after :retrieve_from_store, :retrieve_versions_from_store!
+        after :adopt, :adopt_versions!
+        before :direct_upload, :refuse_direct_upload_of_versions!
 
         prepend Module.new {
           def initialize(*)
@@ -251,7 +253,7 @@ module CarrierWave
       # that the cached files are generated by the single upload attempt.
       #
       def cache!(*args)
-        self.cache_id = parent_version.cache_id if parent_version
+        self.cache_id = parent_version.cache_id if parent_version&.cached?
 
         super
       end
@@ -344,7 +346,8 @@ module CarrierWave
       end
 
       def full_original_filename
-        [version_name, super].compact.join('_')
+        # nil stays nil, as a cached file doesn't have to carry a name of its own
+        [version_name, super].compact.join('_').presence
       end
 
       def cache_versions!(new_file)
@@ -373,6 +376,20 @@ module CarrierWave
         @deferred_version_retrieval = [:retrieve_from_store!, identifier]
       end
 
+      # Versions are created from a file the application has, which a direct upload never gives it
+      def refuse_direct_upload_of_versions!(filename)
+        return if versions.empty?
+
+        raise "#{filename.inspect} can't be uploaded directly, as nothing would create the versions #{versions.keys.join(', ')}"
+      end
+
+      # Versions are created only through the cache, which an adopted file never goes through
+      def adopt_versions!(identifier)
+        return if active_versions.empty?
+
+        raise "#{identifier.inspect} can't be adopted, as nothing would create the versions #{active_versions.keys.join(', ')}"
+      end
+
     end # Versions
   end # Uploader
 end # CarrierWave

data/lib/carrierwave/uploader.rb

@@ -5,6 +5,7 @@ require "carrierwave/uploader/url"
 require "carrierwave/uploader/mountable"
 require "carrierwave/uploader/cache"
 require "carrierwave/uploader/store"
+require "carrierwave/uploader/direct_upload"
 require "carrierwave/uploader/download"
 require "carrierwave/uploader/remove"
 require "carrierwave/uploader/extension_allowlist"
@@ -52,6 +53,7 @@ module CarrierWave
       include CarrierWave::Uploader::Mountable
       include CarrierWave::Uploader::Cache
       include CarrierWave::Uploader::Store
+      include CarrierWave::Uploader::DirectUpload
       include CarrierWave::Uploader::Download
       include CarrierWave::Uploader::Remove
       include CarrierWave::Uploader::ExtensionAllowlist

data/lib/carrierwave/utilities/uri.rb

@@ -19,17 +19,20 @@ module CarrierWave
 
       PERCENT_ENCODED = /%[0-9a-fA-F]{2}/.freeze
 
+      # URI::DEFAULT_PARSER#escape warns as obsolete, and URI::RFC2396_PARSER is missing in older uri gems
+      RFC2396_PARSER = URI::RFC2396_Parser.new
+
     module_function
 
       # Not idempotent, as '%' is escaped to '%25' every time
       def encode_path(path)
-        URI::DEFAULT_PARSER.escape(path, PATH_UNSAFE)
+        RFC2396_PARSER.escape(path, PATH_UNSAFE)
       end
 
       # Only for strings not sent over the wire, like a filename to be shown to the user.
       # CGI.unescape is for form encoding and would turn '+' into a space.
       def decode_path(str)
-        URI::DEFAULT_PARSER.unescape(str)
+        RFC2396_PARSER.unescape(str)
       end
 
       # Escapes only what cannot appear in the component, leaving existing %XX alone.

data/lib/carrierwave/version.rb

@@ -1,3 +1,3 @@
 module CarrierWave
-  VERSION = "4.0.0.beta".freeze
+  VERSION = "4.0.0.rc".freeze
 end

data/lib/carrierwave.rb

@@ -98,6 +98,7 @@ end
 require "carrierwave/utilities"
 require "carrierwave/error"
 require "carrierwave/sanitized_file"
+require "carrierwave/direct_upload"
 require "carrierwave/mounter"
 require "carrierwave/mount"
 require "carrierwave/processing"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: carrierwave
 version: !ruby/object:Gem::Version
-  version: 4.0.0.beta
+  version: 4.0.0.rc
 platform: ruby
 authors:
 - Jonas Nicklas
@@ -347,6 +347,7 @@ files:
 - README.md
 - lib/carrierwave.rb
 - lib/carrierwave/compatibility/paperclip.rb
+- lib/carrierwave/direct_upload.rb
 - lib/carrierwave/downloader/base.rb
 - lib/carrierwave/downloader/remote_file.rb
 - lib/carrierwave/error.rb
@@ -372,6 +373,7 @@ files:
 - lib/carrierwave/uploader/content_type_denylist.rb
 - lib/carrierwave/uploader/default_url.rb
 - lib/carrierwave/uploader/dimension.rb
+- lib/carrierwave/uploader/direct_upload.rb
 - lib/carrierwave/uploader/download.rb
 - lib/carrierwave/uploader/extension_allowlist.rb
 - lib/carrierwave/uploader/extension_denylist.rb