transloadit 4.0.0 → 4.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: fc246e913322624e733835f51e2924112ec4131af0930694ae322da3a10ea9e7
-  data.tar.gz: 79bd642651fd5d6b3c0f3153bfb9d44c6f78b16a7b91171d094e227b430f8949
+  metadata.gz: 2c1220e478a424b85d5ac449fd7facf3790af3091a69435699a49480853b86fe
+  data.tar.gz: 544d23e833e4ee095744d91f948ca7bfc7616bb70e5d576042d44b3032510307
 SHA512:
-  metadata.gz: 6ab26e103e6653f9b4126aed543446c6080e6bd30f1348cecefa38bb8406c706ec289e61e9c36c27512535f8590908ca15473cb042bf6ea38ebacac3eb7cd0a8
-  data.tar.gz: ef78d102680364717e5d4a4193a15cd553a7296478da13134c0cefa65fcc96138acca340990c7173b04e34fa741825788527d5d067e53eab057656acbf20bf1d
+  metadata.gz: 69178ed10b8fc21cd0c342e32aeecad79d886721e2c08b7df69e85a3d965faacef50054973a5b6dca7c72b885166a5f371416c641685df6679481d2e310dd0c8
+  data.tar.gz: 4a2393daa75a76414b208891d63a2d2d0a88e2c50e22da4b634a33fe7696cb084348fb26b1e823c5b45ca6feda168b12e8450a7dfa290a40d7e7bd29ea082d19

data/CHANGELOG.md

@@ -1,3 +1,12 @@
+### Unreleased
+
+### 4.1.0 / 2026-10-18
+
+- Add `uploads: {name => io}` to Assembly options and `create!` for named multipart uploads, separate from signed JSON Instructions.
+- Send custom `fields` only in signed JSON, preserving numbers, booleans, arrays, objects, and null without duplicate form fields.
+- Keep IOs in legacy `fields` working as named uploads throughout v4, omit them from JSON, and warn to migrate to `uploads` before the next major release. Positional `file_N` uploads and `create!(fields: metadata)` overrides remain supported.
+- Restore in-memory uploads to their original position on rate-limit retries and raise a clear `IOError` when a non-seekable upload cannot be retried.
+
 ### 4.0.0 / 2026-09-03
 
 - BREAKING: Replace the long-dormant `rest-client` transport with Faraday 2 and `faraday-multipart`.

data/Gemfile

@@ -5,4 +5,8 @@ gem "minitest"
 gem "simplecov"
 gem "simplecov-cobertura"
 
+# Ruby 3.1 resolves MultiJson 1.19, which cannot parse JSON 3 (ruby-sdk#89).
+# Keep its test bundle compatible without restricting consumers on newer Rubies.
+gem "json", "< 3.0" if RUBY_VERSION.start_with?("3.1.")
+
 gemspec

data/README.md

@@ -171,6 +171,38 @@ files = [open('puppies.jpg'), open('kittens.jpg'), open('ferrets.jpg')]
 response = assembly.create! *files
 ```
 
+For names that match a Step's `use.fields` selector, pass `uploads: {name => io}`
+to `assembly` or `create!`. Uploads are multipart parts and are excluded from
+the signed JSON Instructions. Open files in binary mode and close them after
+the request; the SDK leaves caller-owned files open.
+
+```ruby
+File.open('puppies.jpg', 'rb') do |photo|
+  response = transloadit.assembly(
+    steps: {
+      resize: {
+        robot: '/image/resize',
+        use: {fields: ['photo']},
+        width: 320,
+        height: 240
+      }
+    },
+    uploads: {photo: photo},
+    fields: {count: 2, enabled: false, details: {caption: 'Puppies'}}
+  ).create!
+end
+```
+
+Alternatively, use `assembly.create!(uploads: {photo: photo}, fields: metadata)`.
+Per-call `uploads` and `fields` each replace the corresponding Assembly option;
+`uploads: {}` clears only the `uploads` option. Legacy IOs configured in `fields`
+remain uploads for compatibility. Positional files still use `file_0`,
+`file_1`, and so on, and take precedence over named uploads with those names.
+Upload values must respond to `read`; the names `params` and `signature` are reserved.
+For rate-limit retries, in-memory uploads must also support `pos` and `seek`;
+the SDK restores their original position. A non-seekable upload raises `IOError`
+if a retry is needed, instead of sending consumed bytes again.
+
 ### 3. Parallel Assembly
 
 Transloadit allows you to perform several processing steps in parallel. You
@@ -226,9 +258,10 @@ has some nice examples for that.
 
 ### 5. Using fields
 
-Transloadit allows you to submit form field values that you'll get back in the
-notification. This is quite handy if you want to add additional custom metadata
-to the upload itself. You can use fields like the following:
+Transloadit allows you to submit custom metadata that you'll get back in the
+notification. `fields` is included once in the signed JSON Instructions and is
+never duplicated as form fields. Numbers, booleans, arrays, objects, and null
+retain their JSON types. You can use fields like the following:
 
 ```ruby
 require 'transloadit'
@@ -246,6 +279,17 @@ transloadit.assembly(
 ).create! open('/PATH/TO/FILE.mpg')
 ```
 
+`create!(fields: metadata)` also sets custom fields and replaces any metadata
+configured on the Assembly. Uploads and metadata may use the same name.
+
+For compatibility, IO values passed directly in a `fields` hash still upload
+under their field names throughout v4, but emit a deprecation warning. They are
+excluded from JSON; other values in that hash remain custom metadata. Migrate
+from `assembly(fields: {photo: file, count: 2})` to
+`assembly(uploads: {photo: file}, fields: {count: 2})` before the next major release.
+Explicit `uploads` take precedence over legacy IOs with the same name. Existing
+`assembly(fields: {photo: file}).create!(fields: metadata)` calls continue working.
+
 ### 6. Notify URL
 
 If you want to be notified when the processing is finished you can provide
@@ -464,6 +508,11 @@ for the latest [git main](https://rubydoc.info/github/transloadit/ruby-sdk/main/
 Please see [ci.yml](https://github.com/transloadit/ruby-sdk/tree/main/.github/workflows/ci.yml) for a list of supported ruby versions. It may also work on older Rubies, but support for those is not guaranteed. If it doesn't work on one of the officially supported Rubies, please file a
 [bug report](https://github.com/transloadit/ruby-sdk/issues). Compatibility patches for other Rubies are welcome.
 
+On Ruby 3.1, add `gem "json", "< 3.0"` to your application's Gemfile while
+[the older MultiJson/JSON compatibility issue](https://github.com/transloadit/ruby-sdk/issues/89)
+is open. Ruby 3.2 and newer can use JSON 3 with
+[MultiJson 1.21.2 or newer](https://github.com/sferik/multi_json/releases/tag/v1.21.2).
+
 ### Ruby 2.x
 
 If you still need support for Ruby 2.x, 2.0.1 is the last version that supports it.

data/lib/transloadit/assembly.rb

@@ -44,25 +44,45 @@ class Transloadit::Assembly < Transloadit::ApiModel
   #   @option params [String] :template_id the ID of a
   #     {template}[https://transloadit.com/templates] to use instead of
   #     specifying params here directly
+  #   @option params [Hash] :fields custom metadata, retained as typed JSON
+  #   @option params [Hash<String, IO>] :uploads named multipart uploads;
+  #     overrides the Assembly's uploads when supplied (use {} to clear them)
   #
   def create!(*ios, **params)
     params[:steps] = _wrap_steps_in_hash(params[:steps]) unless params[:steps].nil?
 
+    legacy_uploads, = _split_fields(options[:fields])
+    if params.key?(:fields)
+      uploads, params[:fields] = _split_fields(params[:fields])
+      legacy_uploads.merge!(uploads)
+    end
+    unless legacy_uploads.empty?
+      warn "Passing upload IOs in fields is deprecated; use uploads: {name => io} instead."
+    end
+
     extra_params = {}
-    extra_params.merge!(options[:fields]) if options[:fields]
+    [legacy_uploads, params.delete(:uploads) { options[:uploads] }].compact.each do |uploads|
+      uploads.each do |name, io|
+        name = name.to_s
+        raise ArgumentError, "Upload name #{name.inspect} is reserved" if %w[params signature].include?(name)
+        raise ArgumentError, "Upload #{name.inspect} must be an IO responding to read" unless io.respond_to?(:read)
+
+        extra_params[name] = io
+      end
+    end
+    # Keep positional file_N uploads and their precedence for existing callers.
+    ios.each_with_index { |io, i| extra_params["file_#{i}"] = io }
+    upload_positions = _upload_positions(extra_params.values)
 
     trials = options[:tries] || DEFAULT_TRIES
     (1..trials).each do |trial|
-      # update the payload with file entries
-      ios.each_with_index { |f, i| extra_params.update "file_#{i}": f }
-
       response = _do_request(
         "/assemblies", params, "post", extra_params
       ).extend!(Transloadit::Response::Assembly)
 
       return response unless response.rate_limit?
 
-      _handle_rate_limit!(response, ios, trial < trials)
+      _handle_rate_limit!(response, upload_positions, trial < trials)
     end
   end
 
@@ -123,14 +143,37 @@ class Transloadit::Assembly < Transloadit::ApiModel
   # @return [Hash] a Transloadit-compatible Hash of the Assembly's contents
   #
   def to_hash
-    options.merge(
+    result = options.merge(
       auth: transloadit.to_hash,
       steps: steps
-    ).delete_if { |k, v| v.nil? }
+    )
+    result.delete(:uploads)
+    _, result[:fields] = _split_fields(result[:fields]) if result[:fields]
+    result.delete_if { |_, value| value.nil? }
   end
 
   private
 
+  # Legacy fields may contain both named IOs and metadata. Only metadata belongs
+  # in the signed JSON; the IOs remain supported as uploads throughout v4.
+  def _split_fields(fields)
+    return [{}, nil] if fields.nil?
+
+    fields.partition { |_, value| value.respond_to?(:read) }.map(&:to_h)
+  end
+
+  def _upload_positions(uploads)
+    uploads.filter_map do |io|
+      # The transport reopens path-backed uploads for each attempt.
+      next if io.respond_to?(:path) && io.path
+
+      position = io.pos if io.respond_to?(:pos) && io.respond_to?(:seek)
+      [io, position]
+    rescue IOError, SystemCallError
+      [io, nil]
+    end
+  end
+
   #
   # Returns a Transloadit-compatible Hash wrapping the +steps+ passed to it.
   # Accepts any supported format the +steps+ could come in.
@@ -163,14 +206,21 @@ class Transloadit::Assembly < Transloadit::ApiModel
   end
 
   #
-  # Stays idle for certain time and then reopens assembly files for reprocessing.
+  # Restores in-memory upload positions and waits before retrying.
   # Should be called when assembly rate limit is reached.
   #
   # @param  [Response] response  assembly response that comes with a rate limit
-  # @param [Array<IO>] ios the files sent for the assembly to process.
+  # @param [Array<Array>] upload_positions IOs and their positions before upload
   #
-  def _handle_rate_limit!(response, ios, is_retrying)
+  def _handle_rate_limit!(response, upload_positions, is_retrying)
     if is_retrying
+      upload_positions.each do |io, position|
+        raise IOError if position.nil?
+
+        io.seek(position)
+      rescue IOError, SystemCallError
+        raise IOError, "Cannot retry a non-seekable upload; use a file or an IO supporting pos and seek"
+      end
       warn "Rate limit reached. Waiting for #{response.wait_time} seconds before retrying."
       sleep response.wait_time
     else

data/lib/transloadit/version.rb

@@ -1,3 +1,3 @@
 class Transloadit
-  VERSION = "4.0.0"
+  VERSION = "4.1.0"
 end

data/lib/transloadit.rb

@@ -79,6 +79,10 @@ class Transloadit
   # @option options [String] :template_id the ID of a
   #   {template}[https://transloadit.com/templates] to use instead of
   #   specifying options here directly
+  # @option options [Hash] :fields custom metadata, retained as typed JSON;
+  #   passing upload IOs here is deprecated in favor of :uploads
+  # @option options [Hash<String, IO>] :uploads named multipart uploads, kept
+  #   separate from the signed JSON Instructions
   #
   def assembly(options = {})
     Transloadit::Assembly.new(self, options)

data/test/unit/transloadit/test_assembly.rb

@@ -121,7 +121,7 @@ describe Transloadit::Assembly do
 
         assert_requested(:post, "https://api2.transloadit.com/assemblies") do |req|
           values = values_from_post_body(req.body)
-          _(values["tag"]).must_equal "ninja-cat"
+          _(values.keys).must_equal ["params"]
           _(MultiJson.load(values["params"])["fields"]["tag"]).must_equal "ninja-cat"
         end
       end

data/test/unit/transloadit/test_assembly_uploads.rb

@@ -0,0 +1,272 @@
+require "test_helper"
+require "stringio"
+require "tempfile"
+
+describe "Assembly uploads and custom fields" do
+  include WebMock::API
+
+  before do
+    WebMock.reset!
+    @transloadit = Transloadit.new(key: "key", secret: "secret")
+    @requests = []
+    stub_request(:post, "https://api2.transloadit.com/assemblies").to_return do |request|
+      @requests << request
+      {body: '{"ok":"ASSEMBLY_COMPLETED"}'}
+    end
+    @fields = {
+      "count" => 2,
+      "enabled" => false,
+      "ratio" => 1.5,
+      "empty" => nil,
+      "tags" => ["hello & goodbye", 3, true, nil],
+      "details" => {"caption" => "café + 猫", "visible" => true}
+    }
+  end
+
+  after do
+    WebMock.reset!
+  end
+
+  it "sends typed custom fields only in signed JSON without uploads" do
+    @transloadit.assembly(fields: @fields).create!
+
+    _(body_parts.map(&:first)).must_equal %w[params signature]
+    _(signed_params.fetch("fields")).must_equal @fields
+  end
+
+  it "sends named File parts separately from typed fields with the same name" do
+    Tempfile.create(["assembly-upload", ".bin"]) do |file|
+      file.binmode
+      file.write("named\x00\xff upload".b)
+      file.flush
+      fields = @fields.merge("hero" => "metadata for the hero")
+      uploads = {hero: file}.freeze
+      assembly = @transloadit.assembly(uploads: uploads, fields: fields.freeze)
+
+      _, warning = capture_io { assembly.create! }
+
+      _(warning).wont_include "Passing upload IOs in fields"
+      _(body_parts.map(&:first)).must_equal %w[params signature hero]
+      _(body_parts.to_h.fetch("hero")).must_equal "named\x00\xff upload".b
+      _(signed_params.fetch("fields")).must_equal fields
+      _(signed_params).wont_include "uploads"
+      _(assembly.to_hash).wont_include :uploads
+      _(file.closed?).must_equal false
+      _(uploads[:hero]).must_equal file
+    end
+  end
+
+  it "allows per-call uploads and fields to override constructor defaults" do
+    defaults = {unused: StringIO.new("unused upload")}
+    assembly = @transloadit.assembly(uploads: defaults, fields: {unused: true})
+    upload = StringIO.new("named upload")
+    positional = StringIO.new("positional upload")
+
+    assembly.create!(positional, uploads: {"hero" => upload}, fields: @fields)
+
+    _(body_parts.map(&:first)).must_equal %w[params signature hero file_0]
+    _(body_parts.to_h.fetch("hero")).must_equal "named upload"
+    _(body_parts.to_h.fetch("file_0")).must_equal "positional upload"
+    _(signed_params.fetch("fields")).must_equal @fields
+    _(signed_params).wont_include "uploads"
+    _(upload.closed?).must_equal false
+    _(positional.closed?).must_equal false
+    _(defaults.keys).must_equal [:unused]
+  end
+
+  it "preserves legacy mixed fields while deprecating only their upload entries" do
+    upload = StringIO.new("legacy upload")
+    fields = @fields.merge("hero" => upload).freeze
+    assembly = @transloadit.assembly(fields: fields)
+
+    _, warning = capture_io { assembly.create! }
+
+    _(warning).must_match(/fields.*deprecated.*uploads/)
+    _(body_parts.map(&:first)).must_equal %w[params signature hero]
+    _(body_parts.to_h.fetch("hero")).must_equal "legacy upload"
+    _(signed_params.fetch("fields")).must_equal @fields
+    _(assembly.to_hash[:fields]).must_equal @fields
+    _(JSON.parse(assembly.to_json).fetch("fields")).must_equal @fields
+    _(fields.fetch("hero")).must_equal upload
+  end
+
+  it "preserves content's create! fields override for legacy named Files" do
+    File.open("lib/transloadit/version.rb", "rb") do |file|
+      assembly = @transloadit.assembly(fields: {file: file})
+
+      _, warning = capture_io { assembly.create!(fields: @fields) }
+
+      _(warning).must_include "Passing upload IOs in fields is deprecated"
+      _(body_parts.map(&:first)).must_equal %w[params signature file]
+      _(body_parts.to_h.fetch("file")).must_equal File.binread(file.path)
+      _(signed_params.fetch("fields")).must_equal @fields
+      _(file.closed?).must_equal false
+    end
+  end
+
+  it "separates deprecated uploads in per-call fields from custom fields" do
+    fields = @fields.merge("hero" => StringIO.new("per-call legacy upload"))
+
+    _, warning = capture_io { @transloadit.assembly.create!(fields: fields) }
+
+    _(warning).must_include "Passing upload IOs in fields is deprecated"
+    _(body_parts.map(&:first)).must_equal %w[params signature hero]
+    _(body_parts.to_h.fetch("hero")).must_equal "per-call legacy upload"
+    _(signed_params.fetch("fields")).must_equal @fields
+  end
+
+  it "keeps positional file_N names and priority over legacy upload names" do
+    assembly = @transloadit.assembly(fields: {"file_0" => StringIO.new("legacy")})
+
+    capture_io { assembly.create!(StringIO.new("first"), StringIO.new("second")) }
+
+    _(body_parts.map(&:first)).must_equal %w[params signature file_0 file_1]
+    _(body_parts.to_h.fetch("file_0")).must_equal "first"
+    _(body_parts.to_h.fetch("file_1")).must_equal "second"
+    _(signed_params.fetch("fields")).must_equal({})
+  end
+
+  it "lets explicit uploads replace deprecated entries with the same wire name" do
+    assembly = @transloadit.assembly(fields: {hero: StringIO.new("legacy")})
+
+    capture_io { assembly.create!(uploads: {"hero" => StringIO.new("explicit")}) }
+
+    _(body_parts.map(&:first)).must_equal %w[params signature hero]
+    _(body_parts.to_h.fetch("hero")).must_equal "explicit"
+    _(signed_params.fetch("fields")).must_equal({})
+  end
+
+  it "supports typed per-call fields and an explicitly empty upload override" do
+    assembly = @transloadit.assembly(uploads: {hero: StringIO.new("unused")})
+
+    assembly.create!(uploads: {}, fields: @fields)
+
+    _(body_parts.map(&:first)).must_equal %w[params signature]
+    _(signed_params.fetch("fields")).must_equal @fields
+    _(signed_params).wont_include "uploads"
+  end
+
+  it "preserves deprecated fields uploads when only the uploads option is cleared" do
+    assembly = @transloadit.assembly(
+      uploads: {unused: StringIO.new("unused")},
+      fields: {hero: StringIO.new("legacy upload")}
+    )
+
+    capture_io { assembly.create!(uploads: {}) }
+
+    _(body_parts.map(&:first)).must_equal %w[params signature hero]
+    _(body_parts.to_h.fetch("hero")).must_equal "legacy upload"
+    _(signed_params.fetch("fields")).must_equal({})
+  end
+
+  it "rejects non-IO named uploads before making a request" do
+    error = assert_raises ArgumentError do
+      @transloadit.assembly(uploads: {hero: "path/to/file.jpg"}).create!
+    end
+
+    _(error.message).must_include "IO"
+    _(@requests).must_be_empty
+  end
+
+  it "rejects upload names reserved for params and signature" do
+    [:params, "signature"].each do |name|
+      error = assert_raises ArgumentError do
+        @transloadit.assembly.create!(uploads: {name => StringIO.new("upload")})
+      end
+
+      _(error.message).must_include "reserved"
+    end
+    _(@requests).must_be_empty
+  end
+
+  it "keeps reserved names safe when they are custom fields" do
+    fields = {"params" => {"count" => 2}, "signature" => false}
+
+    @transloadit.assembly(fields: fields).create!
+
+    _(body_parts.map(&:first)).must_equal %w[params signature]
+    _(signed_params.fetch("fields")).must_equal fields
+  end
+
+  it "retries named uploads with the same bytes and signed custom fields" do
+    rate_limit_first_request
+    upload = StringIO.new("retry upload")
+
+    capture_io do
+      response = @transloadit.assembly(uploads: {hero: upload}, fields: @fields).create!
+      _(response.completed?).must_equal true
+    end
+
+    _(@requests.length).must_equal 2
+    @requests.each do |request|
+      _(body_parts(request).map(&:first)).must_equal %w[params signature hero]
+      _(body_parts(request).to_h.fetch("hero")).must_equal "retry upload"
+      _(signed_params(request).fetch("fields")).must_equal @fields
+    end
+    _(upload.closed?).must_equal false
+  end
+
+  it "retries in-memory uploads from their original position" do
+    rate_limit_first_request
+    upload = StringIO.new("header:retry upload")
+    upload.pos = "header:".length
+
+    capture_io { @transloadit.assembly(uploads: {hero: upload}, fields: @fields).create! }
+
+    _(@requests.length).must_equal 2
+    @requests.each do |request|
+      _(body_parts(request).to_h.fetch("hero")).must_equal "retry upload"
+      _(signed_params(request).fetch("fields")).must_equal @fields
+    end
+  end
+
+  it "fails clearly instead of retrying a consumed non-seekable upload" do
+    rate_limit_first_request
+    upload = StringIO.new("streamed upload")
+    [:pos, :seek, :rewind].each do |method|
+      upload.define_singleton_method(method) { |*| raise Errno::ESPIPE }
+    end
+
+    error = assert_raises IOError do
+      capture_io { @transloadit.assembly(uploads: {hero: upload}).create! }
+    end
+
+    _(error.message).must_include "Cannot retry"
+    _(@requests.length).must_equal 1
+    _(body_parts.to_h.fetch("hero")).must_equal "streamed upload"
+    _(upload.closed?).must_equal false
+  end
+
+  def rate_limit_first_request
+    stub_request(:post, "https://api2.transloadit.com/assemblies").to_return do |request|
+      @requests << request
+      body = if @requests.length == 1
+        '{"error":"RATE_LIMIT_REACHED","info":{"retryIn":0}}'
+      else
+        '{"ok":"ASSEMBLY_COMPLETED"}'
+      end
+      {body: body}
+    end
+  end
+
+  def body_parts(request = @requests.last)
+    content_type = request.headers.fetch("Content-Type")
+    return URI.decode_www_form(request.body) unless content_type.start_with?("multipart/form-data")
+
+    boundary = content_type.split("boundary=", 2).last
+    request.body.b.split("--#{boundary}".b).filter_map do |part|
+      headers, value = part.split("\r\n\r\n", 2)
+      next unless value
+
+      [headers[/name="([^"]+)"/, 1], value.delete_suffix("\r\n")]
+    end
+  end
+
+  def signed_params(request = @requests.last)
+    parts = body_parts(request).to_h
+    json = parts.fetch("params")
+    signature = OpenSSL::HMAC.hexdigest("sha384", "secret", json)
+    _(parts.fetch("signature")).must_equal "sha384:#{signature}"
+    JSON.parse(json)
+  end
+end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: transloadit
 version: !ruby/object:Gem::Version
-  version: 4.0.0
+  version: 4.1.0
 platform: ruby
 authors:
 - Stephen Touset
@@ -260,6 +260,7 @@ files:
 - test/unit/test_transloadit.rb
 - test/unit/transloadit/test_api.rb
 - test/unit/transloadit/test_assembly.rb
+- test/unit/transloadit/test_assembly_uploads.rb
 - test/unit/transloadit/test_request.rb
 - test/unit/transloadit/test_response.rb
 - test/unit/transloadit/test_smart_cdn.rb
@@ -285,7 +286,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: 2.2.0
 requirements: []
-rubygems_version: 4.0.16
+rubygems_version: 4.0.20
 specification_version: 4
 summary: Official Ruby gem for Transloadit
 test_files: []