shrine 3.4.0 → 3.5.0

data/doc/plugins/keep_files.md

@@ -2,10 +2,11 @@
 title: Keep Files
 ---
 
-The [`keep_files`][keep_files] plugin prevents file deletion when the attacher
-is about to destroy currently attached or previously attached file. This
-functionality is useful when implementing soft deletes, versioning, or in
-general any scenario where you need to track history.
+The [`keep_files`][keep_files] plugin prevents the attached file (and any of
+its [derivatives]) from being deleted when the attachment would normally be
+destroyed, which happens when the attachment is removed/replaced, or when the
+record is deleted. This functionality is useful when implementing soft deletes,
+versioning, or in general any scenario where you need to keep history.
 
 ```rb
 plugin :keep_files
@@ -17,3 +18,4 @@ photo.image.exists? #=> true
 ```
 
 [keep_files]: https://github.com/shrinerb/shrine/blob/master/lib/shrine/plugins/keep_files.rb
+[derivatives]: https://shrinerb.com/docs/plugins/derivatives

data/doc/plugins/validation_helpers.md

@@ -10,7 +10,7 @@ plugin :validation_helpers
 
 Attacher.validate do
   validate_mime_type %w[image/jpeg image/png image/webp]
-  validate_max_size 5*1024*1024
+  validate_max_size 5*1024*1024 # bytes
   # ...
 end
 ```

data/doc/refile.md

@@ -458,7 +458,7 @@ Shrine.plugin :cached_attachment_data
 ```
 ```rb
 form_for @user do |form|
-  form.hidden_field :profile_image, value: @user.cached_profile_image_data
+  form.hidden_field :profile_image, value: @user.cached_profile_image_data, id: nil
   form.file_field :profile_image
 end
 ```
@@ -475,7 +475,7 @@ Shrine.plugin :remove_attachment
 ```
 ```rb
 form_for @user do |form|
-  form.hidden_field :profile_image, value: @user.cached_profile_image_data
+  form.hidden_field :profile_image, value: @user.cached_profile_image_data, id: nil
   form.file_field :profile_image
   form.check_box :remove_profile_image
 end
@@ -491,7 +491,7 @@ Shrine.plugin :remote_url
 ```
 ```rb
 form_for @user do |form|
-  form.hidden_field :profile_image, value: @user.cached_profile_image_data
+  form.hidden_field :profile_image, value: @user.cached_profile_image_data, id: nil
   form.file_field :profile_image
   form.text_field :profile_image_remote_url
 end

data/doc/release_notes/2.1.0.md

@@ -51,7 +51,7 @@ end
 
 ```rb
 form_for @photo do |f|
-  f.hidden_field :image, value: @photo.cached_image_data
+  f.hidden_field :image, value: @photo.cached_image_data, id: nil
   f.file_filed :image
 end
 ```

data/doc/release_notes/3.5.0.md

@@ -0,0 +1,63 @@
+---
+title: Shrine 3.5.0
+---
+
+## New features
+
+* The website has been migrated to Docusaurus v2. :sparkles:
+
+* The `:signer` option has been added to the `derivation_endpoint` plugin, for when you want to use custom URL signing. This is useful when using `:expires_in`, and wanting to have expiring URLs work with CDN caching.
+
+  ```rb
+  require "aws-sdk-cloudfront"
+  signer = Aws::CloudFront::UrlSigner.new(key_pair_id: "...", private_key: "...")
+
+  plugin :derivation_endpoint,
+    expires_in: 90,
+    signer: -> (url, expires_in:) do
+      signer.signed_url(url, expires: Time.now.to_i + expires_in)
+    end
+  ```
+
+* The S3 storage now supports `:max_multipart_parts` option for specifying the maximum number of concurrent parts in which a large file will get uploaded. This number defaults to `10_000`.
+
+  ```rb
+  Shrine::Storage::S3.new(max_multipart_parts: 1000, ...)
+  ```
+
+* The `:encoding` option can now be passed to `S3#open`, which is applied to downloaded chunks.
+
+  ```rb
+  io = uploaded_file.open(encoding: Encoding::UTF_8)
+  csv = CSV.new(io)
+  # ...
+  ```
+
+## Other improvements
+
+* Passing a boolean value to the `#remove_attachment=` setter now works on Ruby 3.2. Previously this would raise an error, because Shrine would try to call `=~` on it, but `Object#=~` method has been removed in Ruby 3.2.
+
+* When duplicating a model instance, the duplicated attacher now references the duplicated model instance instead of the original one.
+
+* The download endpoint now returns a `400 Bad Request` response when the serialized file component is invalid.
+
+* The `derivatives` plugin now supports passing `mutex: false` option to disable usage of a mutex. This makes the `Shrine::Attacher` object marshallable, which should enable using `Marshal.dump` and `Marshal.load` on model instances with attachments. This should be safe unless you're adding derivatives on the same attacher object concurrently.
+
+* When loading the `derivatives` plugin with `versions_compatibility: true`, this setting doesn't leak to other uploaders anymore. Previously if other uploaders would load `derivatives` plugin without this option, versions compatibility would still get enabled for them. This change also fixes behavior on JRuby.
+
+* When S3 storage copies files, the AWS tag are not inherited anymore. This allows passing the `:tagging` upload option when promoting from temporary to permanent storage, and have it take effect.
+
+* The `UploadedFile#url` method doesn't call the obsolete `URI.regexp` method anymore, which should avoid warnings.
+
+* The `infer_extension` plugin now defines `infer_extension` instance method (in addition to class method) on the uploader for convenience, so that it can be easily called at the uploader instance level.
+
+  ```rb
+  class MyUploader < Shrine
+    plugin :infer_extension
+
+    def generate_location(io, metadata:, **)
+      extension = infer_extension(metadata["mime_type"])
+      # ...
+    end
+  end
+  ```

data/doc/retrieving_uploads.md

@@ -124,7 +124,7 @@ end # underlying IO object is closed
 ```
 
 `Shrine::UploadedFile#open` will return the result of a given block.
-block. We can use that to safely retrieve the whole content of a file, without
+We can use that to safely retrieve the whole content of a file, without
 leaving any temporary files lying around.
 
 ```rb

data/doc/testing.md

@@ -2,6 +2,9 @@
 title: Testing with Shrine
 ---
 
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
 The goal of this guide is to provide some useful tips for testing file
 attachments implemented with Shrine in your application.
 
@@ -137,19 +140,26 @@ module TestData
   end
 end
 ```
-<!--DOCUSAURUS_CODE_TABS-->
-<!--FactoryBot-->
+
+<Tabs>
+<TabItem value="factory_bot" label="FactoryBot">
+
 ```rb
 factory :photo do
   image_data { TestData.image_data }
 end
 ```
-<!--Rails YAML fixtures-->
+
+</TabItem>
+<TabItem value="fixtures" label="Rails YAML fixtures">
+
 ```erb
 photo:
   image_data: <%= TestData.image_data %>
 ```
-<!--END_DOCUSAURUS_CODE_TABS-->
+
+</TabItem>
+</Tabs>
 
 ## Unit tests
 
@@ -182,24 +192,33 @@ end
 In acceptance tests you're testing your app end-to-end, and you likely want to
 also test file attachments here. Here are examples for some common use cases:
 
-<!--DOCUSAURUS_CODE_TABS-->
-<!--Capybara-->
+<Tabs>
+<TabItem value="capybara" label="Capybara">
+
 ```rb
 attach_file("#image-field", "test/files/image.jpg")
 ```
-<!--Rack::Test-->
+
+</TabItem>
+<TabItem value="rack-test" label="rack-test">
+
 ```rb
 post "/photos", photo: {
   image: Rack::Test::UploadedFile.new("test/files/image.jpg", "image/jpeg")
 }
 ```
-<!--Rack::TestApp-->
+
+</TabItem>
+</Tabs>
+
+If you want to test requests with cached attachment data, you can do so as
+follows:
+
 ```rb
-app.post "/photos", multipart: {
-  "photo[image]" => File.open("test/files/image.jpg")
-}
+cached_file = Shrine.upload(file, :cache)
+
+post "/photos", photo: { image: cached_file.to_json }
 ```
-<!--END_DOCUSAURUS_CODE_TABS-->
 
 ## Background jobs
 
@@ -207,21 +226,30 @@ If you're using background jobs with Shrine, you probably want to make them
 synchronous in tests. See your backgrounding library docs for how to make jobs
 synchronous.
 
-<!--DOCUSAURUS_CODE_TABS-->
-<!--ActiveJob-->
+<Tabs>
+<TabItem value="activejob" label="Active Job">
+
 ```rb
 ActiveJob::Base.queue_adapter = :inline
 ```
-<!--Sidekiq-->
+
+</TabItem>
+<TabItem value="sidekiq" label="Sidekiq">
+
 ```rb
 require "sidekiq/testing"
 Sidekiq::Testing.inline!
 ```
-<!--SuckerPunch-->
+
+</TabItem>
+<TabItem value="sucker_punch" label="SuckerPunch">
+
 ```rb
 require "sucker_punch/testing/inline"
 ```
-<!--END_DOCUSAURUS_CODE_TABS-->
+
+</TabItem>
+</Tabs>
 
 ## Processing
 

data/doc/upgrading_to_3.md

@@ -7,9 +7,6 @@ This guide provides instructions for upgrading Shrine in your apps to version
 3.x. If you're looking for a full list of changes, see the **[3.0 release
 notes]**.
 
-If you would like assistance with the upgrade, I'm available for consultation,
-you can email me at <janko.marohnic@gmail.com>.
-
 ## Attacher
 
 The `Shrine::Attacher` class has been rewritten in Shrine 3.0, though much of
@@ -447,7 +444,7 @@ class PromoteJob
     attacher.create_derivatives # call derivatives processor
     attacher.atomic_promote
   rescue Shrine::AttachmentChanged, ActiveRecord::RecordNotFound
-    # attachment has changed or record has beeen deleted, nothing to do
+    # attachment has changed or record has been deleted, nothing to do
   end
 end
 ```
@@ -649,7 +646,8 @@ attacher.copy(other_attacher)
 with
 
 ```rb
-attacher.set attacher.upload(other_attacher.file)
+attacher.set nil # clear original attachment
+attacher.attach other_attacher.file, storage: other_attacher.file.storage_key
 attacher.add_derivatives other_attacher.derivatives # if using derivatives
 ```
 

data/lib/shrine/plugins/derivation_endpoint.rb

@@ -28,8 +28,8 @@ class Shrine
         uploader.opts[:derivation_endpoint] ||= { options: {}, derivations: {} }
         uploader.opts[:derivation_endpoint][:options].merge!(opts)
 
-        unless uploader.opts[:derivation_endpoint][:options][:secret_key]
-          fail Error, "must provide :secret_key option to derivation_endpoint plugin"
+        if !uploader.opts[:derivation_endpoint][:options][:secret_key] && !uploader.opts[:derivation_endpoint][:options][:signer]
+          fail Error, "must provide :secret_key option to derivation_endpoint plugin when no custom signer is set"
         end
 
         # instrumentation plugin integration
@@ -197,6 +197,7 @@ class Shrine
     option :metadata,                    default: -> { [] }
     option :prefix
     option :secret_key
+    option :signer
     option :type
     option :upload,                      default: -> { false }
     option :upload_location,             default: -> { default_upload_location }, result: -> (o) { upload_location(o) }
@@ -216,7 +217,7 @@ class Shrine
       option_definition = self.class.options.fetch(name)
 
       value = options.fetch(name) { shrine_class.derivation_options[name] }
-      value = instance_exec(&value) if value.is_a?(Proc)
+      value = instance_exec(&value) if value.is_a?(Proc) && value.arity == 0
 
       if value.nil?
         default = option_definition[:default]
@@ -300,20 +301,36 @@ class Shrine
   end
 
   class Derivation::Url < Derivation::Command
-    delegate :name, :args, :source, :secret_key
+    delegate :name, :args, :source, :secret_key, :signer
 
     def call(host: nil, prefix: nil, **options)
-      [host, *prefix, identifier(**options)].join("/")
+      base_url = [host, *prefix].join("/")
+      path = path_identifier(metadata: options.delete(:metadata))
+
+      if signer
+        url = [base_url, path].join("/")
+        signer.call(url, **options)
+      else
+        signed_part = signed_url("#{path}?#{query(**options)}")
+        [base_url, signed_part].join("/")
+      end
     end
 
     private
 
-    def identifier(expires_in: nil,
-                   version: nil,
-                   type: nil,
-                   filename: nil,
-                   disposition: nil,
-                   metadata: [])
+    def path_identifier(metadata: [])
+      [
+        name,
+        *args,
+        source.urlsafe_dump(metadata: metadata)
+      ].map{|component| Rack::Utils.escape_path(component.to_s)}.join('/')
+    end
+
+    def query(expires_in: nil,
+              type: nil,
+              filename: nil,
+              disposition: nil,
+              version: nil)
 
       params = {}
       params[:expires_at]  = (Time.now + expires_in).to_i if expires_in
@@ -322,23 +339,7 @@ class Shrine
       params[:filename]    = filename if filename
       params[:disposition] = disposition if disposition
 
-      # serializes the source uploaded file into an URL-safe format
-      source_component = source.urlsafe_dump(metadata: metadata)
-
-      # generate plain URL
-      url = plain_url(name, *args, source_component, params)
-
-      # generate signed URL
-      signed_url(url)
-    end
-
-    def plain_url(*components, params)
-      # When using Rack < 2, Rack::Utils#escape_path will escape '/'.
-      # Escape each component and then join them together.
-      path  = components.map{|component| Rack::Utils.escape_path(component.to_s)}.join('/')
-      query = Rack::Utils.build_query(params)
-
-      "#{path}?#{query}"
+      Rack::Utils.build_query(params)
     end
 
     def signed_url(url)
@@ -379,7 +380,7 @@ class Shrine
     # Returns "404 Not Found" if derivation block is not defined, or if source
     # file was not found on the storage.
     def handle_request(request)
-      verify_signature!(request)
+      verify_signature!(request) if secret_key
       check_expiry!(request)
 
       name, *args, serialized_file = request.path_info.split("/")[1..-1]

data/lib/shrine/plugins/derivatives.rb

@@ -12,18 +12,18 @@ class Shrine
         }.inspect}"
       end
 
-      def self.load_dependencies(uploader, versions_compatibility: false, **)
+      def self.load_dependencies(uploader, **)
         uploader.plugin :default_url
-
-        AttacherMethods.prepend(VersionsCompatibility) if versions_compatibility
       end
 
-      def self.configure(uploader, log_subscriber: LOG_SUBSCRIBER, **opts)
-        uploader.opts[:derivatives] ||= { processors: {}, processor_settings: {}, storage: proc { store_key } }
+      def self.configure(uploader, log_subscriber: LOG_SUBSCRIBER, versions_compatibility: false, **opts)
+        uploader.opts[:derivatives] ||= { processors: {}, processor_settings: {}, storage: proc { store_key }, mutex: true }
         uploader.opts[:derivatives].merge!(opts)
 
         # instrumentation plugin integration
         uploader.subscribe(:derivatives, &log_subscriber) if uploader.respond_to?(:subscribe)
+
+        uploader::Attacher.include(VersionsCompatibility) if versions_compatibility
       end
 
       module AttachmentMethods
@@ -109,7 +109,7 @@ class Shrine
           super(**options)
 
           @derivatives       = derivatives
-          @derivatives_mutex = Mutex.new
+          @derivatives_mutex = Mutex.new if shrine_class.derivatives_options[:mutex]
         end
 
         # Convenience method for accessing derivatives.
@@ -139,7 +139,7 @@ class Shrine
         # Allows generating a URL to the derivative by passing the derivative
         # name.
         #
-        #     attacher.add_derivatives(thumb: thumb)
+        #     attacher.add_derivatives({ thumb: thumb })
         #     attacher.url(:thumb) #=> "https://example.org/thumb.jpg"
         def url(*path, **options)
           return super if path.empty?
@@ -180,7 +180,7 @@ class Shrine
 
         # In addition to deleting the main file it also deletes any derivatives.
         #
-        #     attacher.add_derivatives(thumb: thumb)
+        #     attacher.add_derivatives({ thumb: thumb })
         #     attacher.derivatives[:thumb].exists? #=> true
         #     attacher.destroy
         #     attacher.derivatives[:thumb].exists? #=> false
@@ -208,7 +208,7 @@ class Shrine
         #     # {
         #     #   thumb: #<Shrine::UploadedFile>,
         #     # }
-        #     attacher.add_derivatives(cropped: cropped)
+        #     attacher.add_derivatives({ cropped: cropped })
         #     attacher.derivatives #=>
         #     # {
         #     #   thumb: #<Shrine::UploadedFile>,
@@ -239,7 +239,7 @@ class Shrine
 
         # Uploads given hash of files.
         #
-        #     hash = attacher.upload_derivatives(thumb: thumb)
+        #     hash = attacher.upload_derivatives({ thumb: thumb })
         #     hash[:thumb] #=> #<Shrine::UploadedFile>
         def upload_derivatives(files, **options)
           map_derivative(files) do |path, file|
@@ -298,10 +298,10 @@ class Shrine
         # Deep merges given uploaded derivatives with current derivatives.
         #
         #     attacher.derivatives #=> { one: #<Shrine::UploadedFile> }
-        #     attacher.merge_derivatives(two: uploaded_file)
+        #     attacher.merge_derivatives({ two: uploaded_file })
         #     attacher.derivatives #=> { one: #<Shrine::UploadedFile>, two: #<Shrine::UploadedFile> }
         def merge_derivatives(new_derivatives)
-          @derivatives_mutex.synchronize do
+          derivatives_synchronize do
             merged_derivatives = deep_merge_derivatives(derivatives, new_derivatives)
             set_derivatives(merged_derivatives)
           end
@@ -379,7 +379,7 @@ class Shrine
 
         # Deletes given hash of uploaded files.
         #
-        #     attacher.delete_derivatives(thumb: uploaded_file)
+        #     attacher.delete_derivatives({ thumb: uploaded_file })
         #     uploaded_file.exists? #=> false
         def delete_derivatives(derivatives = self.derivatives)
           map_derivative(derivatives) { |_, derivative| derivative.delete }
@@ -387,7 +387,7 @@ class Shrine
 
         # Sets the given hash of uploaded files as derivatives.
         #
-        #     attacher.set_derivatives(thumb: uploaded_file)
+        #     attacher.set_derivatives({ thumb: uploaded_file })
         #     attacher.derivatives #=> { thumb: #<Shrine::UploadedFile> }
         def set_derivatives(derivatives)
           self.derivatives = derivatives
@@ -398,7 +398,7 @@ class Shrine
         # Adds derivative data into the hash.
         #
         #     attacher.attach(io)
-        #     attacher.add_derivatives(thumb: thumb)
+        #     attacher.add_derivatives({ thumb: thumb })
         #     attacher.data
         #     #=>
         #     # {
@@ -544,6 +544,14 @@ class Shrine
         def create_derivatives_on_promote?
           shrine_class.derivatives_options[:create_on_promote]
         end
+
+        def derivatives_synchronize
+          if @derivatives_mutex
+            @derivatives_mutex.synchronize { yield }
+          else
+            yield
+          end
+        end
       end
 
       module ClassMethods

data/lib/shrine/plugins/download_endpoint.rb

@@ -182,12 +182,19 @@ class Shrine
       @shrine_class::UploadedFile.urlsafe_load(serialized)
     rescue Shrine::Error # storage not found
       not_found!
+    rescue JSON::ParserError, ArgumentError => error # invalid serialized component
+      raise if error.is_a?(ArgumentError) && error.message != "invalid base64"
+      bad_request!("Invalid serialized file")
     end
 
     def not_found!
       error!(404, "File Not Found")
     end
 
+    def bad_request!(message)
+      error!(400, message)
+    end
+
     # Halts the request with the error message.
     def error!(status, message)
       throw :halt, [status, { "Content-Type" => "text/plain" }, [message]]

data/lib/shrine/plugins/entity.rb

@@ -36,8 +36,14 @@ class Shrine
           attachment = self
 
           # Returns the attached file.
-          define_method :"#{name}" do |*args|
-            send(:"#{name}_attacher").get(*args)
+          if shrine_class::Attacher.instance_method(:get).arity == 0
+            define_method :"#{name}" do
+              send(:"#{name}_attacher").get
+            end
+          else # derivatives
+            define_method :"#{name}" do |*args|
+              send(:"#{name}_attacher").get(*args)
+            end
           end
 
           # Returns the URL to the attached file.

data/lib/shrine/plugins/infer_extension.rb

@@ -51,6 +51,10 @@ class Shrine
       end
 
       module InstanceMethods
+        def infer_extension(mime_type)
+          self.class.infer_extension(mime_type)
+        end
+
         private
 
         def basic_location(io, metadata:)

data/lib/shrine/plugins/instrumentation.rb

@@ -12,7 +12,11 @@ class Shrine
       def self.configure(uploader, log_subscriber: LOG_SUBSCRIBER, **opts)
         uploader.opts[:instrumentation] ||= { log_events: EVENTS, subscribers: {} }
         uploader.opts[:instrumentation].merge!(opts)
-        uploader.opts[:instrumentation][:notifications] ||= ::ActiveSupport::Notifications
+        begin
+          uploader.opts[:instrumentation][:notifications] ||= ::ActiveSupport::Notifications
+        rescue NameError
+          fail Error, "default notifications library is ActiveSupport::Notifications, but activesupport gem is not installed"
+        end
 
         uploader.opts[:instrumentation][:log_events].each do |event_name|
           uploader.subscribe(event_name, &log_subscriber)

data/lib/shrine/plugins/model.rb

@@ -48,7 +48,9 @@ class Shrine
           # The copy constructor that's called on #dup and #clone.
           define_method :initialize_copy do |other|
             super(other)
-            instance_variable_set(:"@#{name}_attacher", instance_variable_get(:"@#{name}_attacher")&.dup)
+            attacher_copy = instance_variable_get(:"@#{name}_attacher")&.dup
+            attacher_copy.set_entity(self, name) if attacher_copy
+            instance_variable_set(:"@#{name}_attacher", attacher_copy)
             self
           end
           private :initialize_copy

data/lib/shrine/plugins/remove_attachment.rb

@@ -39,6 +39,8 @@ class Shrine
 
         # Rails sends "0" or "false" if the checkbox hasn't been ticked.
         def remove?
+          return remove if [true, false].include?(remove)
+
           remove && remove != "" && remove !~ /\A(0|false)\z/
         end
       end

data/lib/shrine/plugins/validation_helpers.rb

@@ -145,7 +145,7 @@ class Shrine
 
         # Validates that the dimensions are not smaller than specified.
         #
-        #     validate_max_dimensions [100, 100]
+        #     validate_min_dimensions [100, 100]
         def validate_min_dimensions((min_width, min_height), message: nil)
           fail Error, "width and/or height metadata is missing" unless file["width"] && file["height"]