restify 2.1.0 → 3.0.0.rc3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6db18b2b418bd450289f6e0c64f429e0c3f3066e6a45c082af2f34fa87b3219f
-  data.tar.gz: ca2c0abe620c7a679e0c0ec0b12e57448ea7d709b5e0ee18d523183e97085715
+  metadata.gz: d8fa974df7fe0cd06afd87ffe8bef89d87559ece784a801070aa0b8342b883eb
+  data.tar.gz: 1f3cc52e1b4bf7995f50dbe5ef9360986d71c7d157b47b1527f79cca2ff66179
 SHA512:
-  metadata.gz: 85f715444240b80d9a269662779b69791a9240b6d37d69586948aaed3c138c8b8bc11d78e0eadd42d7da600f65e628b589053f33991b66a0a9b2c923248b4999
-  data.tar.gz: 9a1509904bcc695749e2cfb2b95976a38a94013db01aabb7374c270de7af788ac1f8e633a846aafe56e14b3cbb244d3952a99c769d845ea3c68c57f8d4f66c8e
+  metadata.gz: cdc59e5ca15e38eb8224b638e675807f669cf0ee00802427334238107ce8a5cddc16b71cbec12e0013055fe5431b38033509381f4c1a1727e07c38cfed1be4fc
+  data.tar.gz: 7da86c8fb8dbf2829da84e9f3e0c5d47d226e360a35cc7b6b33e1da02d7507ef4570d703d31c3ef7448fb3586592218d5924b926e70d1694abfdb916de6e1c4f

data/CHANGELOG.md

@@ -17,6 +17,81 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 
 ### Breaks
 
+## 3.0.0-rc3 - (2026-09-27)
+
+---
+
+### Changes
+
+- Perf: Avoid parsing/joining URIs when not necessary, e.g., for relations without variables or absolute URIs.
+- Perf: Reuse libcurl handles in the Ethon adapter
+- Perf: Optimize libcurl option preparation in the Ethon adapter
+- Perf: Optimize parsing of data and relations
+- Perf: Replace `Concurrent::IVar` with a lightweight promise implementation
+- Remove `concurrent-ruby` dependency
+
+### Fixes
+
+- Apply `Restify::Timeout.default_timeout` when waiting on a promise without
+  an explicit timeout, e.g. with the Typhoeus adapter, instead of waiting forever
+- A timeout while waiting on a promise does not reject the promise anymore.
+  Waiting again, e.g. with a longer timeout, continues where the last wait gave up.
+- Honor the timeout of all threads waiting on the same promise, not only of the
+  thread running its `#then` blocks
+- Only create relations from `*_url` fields with string values, e.g. not from numbers, objects, or arrays
+
+### Breaks
+
+- `Restify::Promise` does not inherit from `Concurrent::IVar` anymore
+
+## 3.0.0.rc2 - (2026-09-25)
+
+---
+
+First release candidate for 3.0. `3.0.0.rc1` was tagged, but not published due to a broken release pipeline.
+
+### New
+
+- Add `Restify::Adapter::Ethon` adapter directly using libcurl in a background event loop
+- Perf: Run event loop in waiting threads to avoid slow handover between threads
+- Add `restify/webmock` to intercept requests with WebMock for all adapters
+
+### Changes
+
+- Restrict requests and redirects to HTTP(S), as libcurl supports many more
+  protocols such as `file://` or `gopher://`
+- Drop EOL Ruby versions 3.1 and 3.2
+- Drop `hitimes` dependency, using `Process.clock_gettime` for timeouts
+- Return resources without data for empty response bodies, even with a JSON
+  or MessagePack content type (#31)
+
+### Fixes
+
+- Use the effective URI after redirects for responses and relations
+- Do not set empty OpenTelemetry attributes for URIs without a port
+- Encode response bodies using the charset from the `Content-Type` header
+- Do not share connections of the Ethon adapter with forked child processes
+- Do not deadlock the Typhoeus adapter with `sync: true` when used from multiple threads
+- Raise the `ResponseError` for the status code, e.g. `NotFound`, instead of a
+  `JSON::ParserError` for error responses with an empty or invalid JSON body (#31)
+
+### Breaks
+
+- Raise minimum Ruby version to 3.3
+- `Restify::Request#uri` is always parsed and returned as `Addressable::URI`
+- Use `Restify::Adapter::Ethon` by default. With WebMock, require
+  `restify/webmock`, as WebMock's Typhoeus integration does not apply anymore.
+- `typhoeus` is an optional dependency now. Add it to your Gemfile to use
+  `Restify::Adapter::Typhoeus`.
+- Invalid response bodies do not raise from `#value!` anymore. The resource is
+  returned, e.g. with relations from headers, and accessing its data raises a
+  `Restify::ParseError` with the parser error as its cause (#31).
+- Remove `activesupport` dependency. `Restify.cache` defaults to `nil`, as the
+  previous default cache did not cache anything (yet).
+- Remove `logging` dependency. Set a `Logger`-compatible logger with
+  `Restify.logger = Logger.new(...)`, nothing is logged by default. Restify does
+  not change `Ethon.logger` anymore.
+
 ## 2.1.0 - (2026-06-20)
 
 ---

data/README.md

@@ -2,7 +2,6 @@
 
 [![Gem Version](https://img.shields.io/gem/v/restify?logo=ruby)](https://rubygems.org/gems/restify)
 [![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/jgraichen/restify/test.yml?logo=github)](https://github.com/jgraichen/restify/actions)
-[![Code Quality](https://codebeat.co/badges/368f8033-bd76-48bc-9777-85f1d4befa94)](https://codebeat.co/projects/github-com-jgraichen-restify-main)
 
 Restify is an hypermedia REST client that does parallel, concurrent and keep-alive requests by default.
 
@@ -12,11 +11,16 @@ Restify can be used to consume hypermedia REST APIs (like GitHubs), to build a s
 
 Restify is build upon the following libraries:
 
-* [concurrent-ruby](https://github.com/ruby-concurrency/concurrent-ruby)
 * [addressable](https://github.com/sporkmonger/addressable)
-* [typhoeus](https://github.com/typhoeus/typhoeus)
+* [ethon](https://github.com/typhoeus/ethon), using libcurl
 
-The HTTP adapters are mostly run in a background thread and may not survive mid-application forks.
+Restify uses libcurl via Ethon by default. The older [Typhoeus](https://github.com/typhoeus/typhoeus) adapter is still available. To use it, add `typhoeus` to your Gemfile and configure it:
+
+```ruby
+Restify.adapter = Restify::Adapter::Typhoeus.new
+```
+
+The default Ethon adapter should work with forking, e.g. in Puma or Unicorn workers. Requests still pending when forking are aborted in the child process.
 
 Restify includes processors to parse responses and to extract links between resources. The following formats are can be parsed:
 
@@ -102,6 +106,27 @@ puts "#{commit['commit']['message']}"
 
 See commented example in main spec [`spec/restify_spec.rb`](https://github.com/jgraichen/restify/blob/master/spec/restify_spec.rb#L100) or in the `examples` directory.
 
+## Testing with WebMock
+
+Restify can intercept requests for [WebMock](https://github.com/bblimke/webmock), independent of the adapter in use. Load it after WebMock, before WebMock is enabled, e.g. in your `spec_helper.rb`:
+
+```ruby
+require 'webmock/rspec'
+require 'restify/webmock'
+```
+
+Note: Stubbed responses are returned as they are, including redirects, like WebMock does for other libcurl-based libraries. Requests not stubbed are passed on to the adapter if WebMock allows connections.
+
+For tests relying on actual HTTP behavior, e.g. following redirects, run a local stub server instead, so that requests are sent over the network by the adapter.
+
+WebMock's own Typhoeus integration is not needed for Restify and can be disabled. Otherwise, requests are reported a second time, when they are passed on to the Typhoeus adapter:
+
+```ruby
+WebMock.enable!(except: %i[typhoeus])
+```
+
+Keep it enabled if your application uses Typhoeus directly, too.
+
 ## Contributing
 
 1. [Fork it](http://github.com/jgraichen/restify/fork)
@@ -113,7 +138,7 @@ See commented example in main spec [`spec/restify_spec.rb`](https://github.com/j
 
 ## License
 
-Copyright (C) 2014-2025 Jan Graichen
+Copyright (C) 2014-2026 Jan Graichen
 
 This program is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
 

data/lib/restify/adapter/base.rb

@@ -8,7 +8,7 @@ module Restify
       prepend Telemetry
 
       def call(request)
-        Promise.create do |writer|
+        Promise.create(driver:) do |writer|
           call_native request, writer
         end
       end
@@ -16,6 +16,15 @@ module Restify
       def call_native(_request, _writer)
         raise NotImplementedError.new 'Subclass responsibility'
       end
+
+      private
+
+      # Adapters that can process requests in the thread waiting on the
+      # returned promise return an object responding to
+      # `#drive(promise, timeout)` here, see `Promise#wait`.
+      def driver
+        nil
+      end
     end
   end
 end

data/lib/restify/adapter/ethon/easy.rb

@@ -0,0 +1,147 @@
+# frozen_string_literal: true
+
+module Restify
+  module Adapter
+    class Ethon < Base
+      #
+      # A libcurl easy handle for Restify requests, translating a
+      # request into libcurl options, and a completed transfer back into
+      # a response.
+      #
+      # Handles are reused for many requests, see `Pool`.
+      #
+      # @api private
+      #
+      class Easy < ::Ethon::Easy
+        attr_accessor :_otel_span, :_restify_request, :_restify_writer
+
+        TIMEOUT_MS = Options.number(:timeout_ms)
+        CONNECTTIMEOUT_MS = Options.number(:connecttimeout_ms)
+
+        def reset
+          @_otel_span = @_restify_request = @_restify_writer = nil
+
+          super
+        end
+
+        def prepare(request, options)
+          options.apply(self)
+
+          self.url = request.uri.to_s
+          self.headers = DEFAULT_HEADERS.merge(request.headers)
+
+          if request.timeout
+            # libcurl requires millisecond-based timeouts so fractional
+            # seconds must be converted and rounded.
+            timeout = (request.timeout * 1000).ceil
+            ::Ethon::Curl.easy_setopt(handle, TIMEOUT_MS, :long, timeout)
+            ::Ethon::Curl.easy_setopt(handle, CONNECTTIMEOUT_MS, :long, timeout)
+          end
+
+          prepare_method(request.method, request.body)
+
+          # The OpenTelemetry instrumentation for Ethon takes the method
+          # from `#http_request`, which is not used here.
+          @otel_method = request.method
+        end
+
+        def response(request)
+          code = return_code
+          if code != :ok
+            raise Restify::NetworkError.new(request, ::Ethon::Curl.easy_strerror(code))
+          end
+
+          # Non-HTTP protocols are refused before a transfer is started,
+          # but libcurl still reports success with a zero status code
+          # whenever no HTTP status was received, and no status code at
+          # all when it cannot be read back.
+          status = response_code
+          if status.nil? || status.zero?
+            raise Restify::NetworkError.new(request, 'Response without HTTP status')
+          end
+
+          ::Restify::Response.new(
+            request,
+            effective_uri(request),
+            status,
+            parse_headers(response_headers),
+            response_body,
+          )
+        end
+
+        private
+
+        # Like Ethon's actions for each method, see `Ethon::Easy::Http`.
+        # An empty body counts as no body.
+        def prepare_method(method, body)
+          body = nil if body.nil? || body.empty?
+
+          case method
+            when 'get'
+              if body
+                postfields(body)
+                self.customrequest = 'GET'
+              end
+            when 'post'
+              postfields(body || '')
+            when 'put'
+              self.upload = true
+              self.infilesize = body ? body.bytesize : 0
+              set_read_callback(body) if body
+            when 'head'
+              postfields(body) if body
+              self.nobody = true
+            else
+              postfields(body) if body
+              self.customrequest = method.upcase
+          end
+        end
+
+        def postfields(body)
+          self.postfieldsize = body.bytesize
+          self.copypostfields = body
+        end
+
+        def effective_uri(request)
+          url = effective_url
+
+          # Only parse the URL after redirects, as parsing is expensive.
+          return request.uri if url.nil? || url == request.uri.to_s
+
+          Addressable::URI.parse(url)
+        end
+
+        def parse_headers(raw)
+          headers = {}
+          return headers if raw.nil?
+
+          # The raw headers can contain multiple blocks, e.g. from
+          # informational responses or when following redirects; only
+          # use the latest ones:
+          block = raw.split(/\r?\n\r?\n/).reject {|b| b.strip.empty? }.last
+
+          # Split on newlines that are not followed by whitespace to
+          # keep folded header values together.
+          block.to_s.split(/\r?\n(?!\s)/).each do |line|
+            line = line.strip
+            next if line.empty? || line.start_with?('HTTP/')
+
+            key, value = line.split(':', 2)
+            next if value.nil?
+
+            key = key.strip.upcase.tr('-', '_')
+            value = value.strip.gsub(/\r?\n\s*/, ' ')
+
+            case (current = headers[key])
+              when nil then headers[key] = value
+              when Array then current << value
+              else headers[key] = [current, value]
+            end
+          end
+
+          headers
+        end
+      end
+    end
+  end
+end

data/lib/restify/adapter/ethon/event_loop.rb

@@ -0,0 +1,255 @@
+# frozen_string_literal: true
+
+module Restify
+  module Adapter
+    class Ethon < Base
+      #
+      # An event loop running libcurl's multi interface in
+      # `socket_action` mode, waiting for sockets with nio4r.
+      #
+      # Can woken up to add new easy handles immediately but can run
+      # from any thread. `LoopLock` ensures all interaction with libcurl
+      # is still serialized and threads are correctly prioritized.
+      #
+      # @api private
+      #
+      class EventLoop
+        include Logging
+
+        INTERESTS = {
+          none: nil,
+          in: :r,
+          out: :w,
+          inout: :rw,
+        }.freeze
+
+        def initialize(**multi)
+          @selector = NIO::Selector.new
+          @monitors = {}
+          @timer    = nil
+          @queue    = Queue.new
+
+          # libcurl only stores the function pointers, therefore the procs
+          # must be referenced here too, or they would be garbage
+          # collected.
+          @socketfunction = method(:on_socket).to_proc
+          @timerfunction  = method(:on_timer).to_proc
+
+          @multi = ::Ethon::Multi.new(execution_mode: :socket_action, **multi)
+          @multi.socketfunction = @socketfunction
+          @multi.timerfunction  = @timerfunction
+
+          # Wake up the background thread from `select` when a waiting
+          # thread wants to run the loop.
+          @lock = LoopLock.new { @selector.wakeup }
+        end
+
+        def enqueue(easy)
+          @queue << easy
+          @selector.wakeup
+        end
+
+        # Wake up threads waiting for the loop, e.g. after completing a
+        # transfer, to check if their result is available now.
+        def notify
+          @lock.notify
+        end
+
+        # Run the loop in the calling thread until the promise is
+        # complete or the timeout expires.
+        #
+        # Several threads can wait at the same time, but only one runs
+        # the loop. The others sleep until either their promise
+        # completes or the loop is released.
+        #
+        # @return [Boolean] false when the loop cannot be run in the
+        #   calling thread, i.e. when waiting from within the loop
+        #   itself.
+        #
+        def drive(promise, timeout) # rubocop:disable Naming/PredicateMethod
+          return false if @lock.owned?
+
+          while @lock.acquire(timeout) { promise.complete? }
+            begin
+              step(timeout.remaining) until promise.complete? || !timeout.remaining.positive?
+            rescue StandardError => e
+              error(e)
+            ensure
+              @lock.release
+            end
+          end
+
+          true
+        end
+
+        # Run the loop whenever no thread waits for it. Blocks forever,
+        # as meant for a background thread.
+        def run
+          loop do
+            @lock.acquire_background
+
+            begin
+              step
+            ensure
+              @lock.release
+            end
+          rescue StandardError => e
+            error(e)
+          end
+        ensure
+          debug 'loop:exit'
+        end
+
+        # Forget the loop in a forked child process, without releasing
+        # its libcurl handles, as they belong to the parent process.
+        # Transfers not completed yet are rejected, as they were started
+        # by the parent process too.
+        #
+        def forked!
+          pending = @multi.easy_handles.dup
+          pending << @queue.pop(true) until @queue.empty?
+
+          @multi.handle.autorelease = false
+
+          pending.each do |easy|
+            easy.handle.autorelease = false
+            easy._restify_writer.reject(
+              Restify::NetworkError.new(
+                easy._restify_request,
+                'Request started before fork',
+              ),
+            )
+          end
+
+          nil
+        end
+
+        private
+
+        def step(limit = nil)
+          dequeue_all
+
+          # libcurl needs to be notified about its own timeouts, e.g. to
+          # start newly added transfers or to time out stalled ones.
+          timeout!
+
+          timeout = select_timeout
+          timeout = limit if limit && (timeout.nil? || timeout > limit)
+          debug 'loop:select', timeout: timeout
+
+          # nil on timeout; empty array when woken up
+          @selector.select(timeout)&.each do |monitor|
+            # libcurl can remove sockets while earlier events of the
+            # same batch are processed, e.g. when a completed transfer
+            # tears down other connections. Skip monitors that are gone
+            # by now.
+            next unless @monitors[monitor.value].equal?(monitor)
+
+            socket_action(monitor.value, readiness(monitor))
+          end
+        end
+
+        def dequeue_all
+          loop do
+            easy = begin
+              @queue.pop(true)
+            rescue ThreadError
+              break
+            end
+
+            add(easy)
+          end
+        end
+
+        def add(easy)
+          # Restore the OpenTelemetry span the request originated from,
+          # so that the Ethon instrumentation picks up the correct
+          # parent when the easy handle is added to libcurl.
+          #
+          # Handle exceptions from Ethon or WebMock too, and reject the
+          # promise, so that the errors can be handled in user code.
+          OpenTelemetry::Trace.with_span(easy._otel_span) do
+            @multi.add(easy)
+          rescue Exception => e # rubocop:disable Lint/RescueException
+            easy._restify_writer.reject(e)
+          end
+        end
+
+        # Seconds until libcurl wants to be called again, or nil to
+        # block until a socket becomes ready or the loop is woken up.
+        def select_timeout
+          return nil unless @timer
+
+          [@timer - now, 0].max
+        end
+
+        def timeout!
+          return unless @timer
+          return if now < @timer
+
+          # Reset before invoking libcurl, as it will set a new timeout
+          # from within the socket action.
+          @timer = nil
+
+          socket_action
+        end
+
+        def readiness(monitor)
+          readiness = []
+          readiness << :in if monitor.readable?
+          readiness << :out if monitor.writable?
+          readiness
+        end
+
+        def socket_action(socket = nil, readiness = 0)
+          @multi.socket_action(socket, readiness)
+        end
+
+        # libcurl: called when asking for readiness monitoring
+        def on_socket(_easy, socket, what, _userp, _socketp)
+          debug 'socket:action', tag: socket, what: what
+
+          if what == :remove
+            @monitors.delete(socket)&.close
+          else
+            monitor = (@monitors[socket] ||= register(socket))
+            monitor.interests = INTERESTS.fetch(what)
+          end
+
+          :ok
+        rescue StandardError => e
+          error(e)
+          :ok
+        end
+
+        # libcurl: called on timeout changes
+        def on_timer(_handle, timeout_ms, _userp)
+          debug 'timer:set', timeout: timeout_ms
+          @timer = timeout_ms.negative? ? nil : now + (timeout_ms / 1000.0)
+
+          :ok
+        rescue StandardError => e
+          error(e)
+          :ok
+        end
+
+        def register(socket)
+          # FD owned by libcurl: IO/ruby must never close it
+          io = IO.for_fd(socket, autoclose: false)
+
+          @selector.register(io, :r).tap do |monitor|
+            monitor.value = socket
+          end
+        end
+
+        def now
+          Process.clock_gettime(Process::CLOCK_MONOTONIC)
+        end
+
+        def _log_prefix
+          "[#{object_id}/#{Thread.current.object_id}]"
+        end
+      end
+    end
+  end
+end

data/lib/restify/adapter/ethon/options.rb

@@ -0,0 +1,81 @@
+# frozen_string_literal: true
+
+module Restify
+  module Adapter
+    class Ethon < Base
+      #
+      # libcurl options, converted to the arguments for
+      # `curl_easy_setopt` once for performance reasons.
+      #
+      # @api private
+      #
+      class Options
+        DEFINITIONS = ::Ethon::Curl.easy_options(nil)
+
+        class << self
+          def number(name)
+            definition(name).fetch(:opt)
+          end
+
+          def definition(name)
+            DEFINITIONS.fetch(name) do
+              raise ArgumentError.new("Unknown libcurl option: #{name}")
+            end
+          end
+        end
+
+        # Options by name, e.g. `followlocation: true`, nil values are
+        # skipped.
+        #
+        def initialize(options)
+          @options = options.filter_map do |name, value|
+            definition = self.class.definition(name)
+            next if value.nil?
+
+            compile(name, definition, value)
+          end.freeze
+        end
+
+        def apply(easy)
+          handle = easy.handle
+
+          @options.each do |opt, type, value|
+            if type
+              ::Ethon::Curl.easy_setopt(handle, opt, type, value)
+            else
+              ::Ethon::Curl.set_option(opt, value, handle)
+            end
+          end
+        end
+
+        def to_a
+          @options
+        end
+
+        private
+
+        # Like `Ethon::Curls::Options#set_option` for the supported types.
+        def compile(name, definition, value)
+          opt = definition[:opt]
+
+          case definition[:type]
+            when :bool then [opt, :long, value && value != 0 ? 1 : 0]
+            when :int, :time then [opt, :long, Integer(value)]
+            when :enum then [opt, :long, lookup(name, definition, value)]
+            when :bitmask then [opt, :long, Array(value).reduce(0) {|mask, v| mask | lookup(name, definition, v) }]
+            when :string then [opt, :string, value.to_s]
+            else [name, nil, value]
+          end
+        end
+
+        def lookup(name, definition, value)
+          return Integer(value) unless value.is_a?(Symbol) || value.is_a?(String)
+
+          definition[:opts].fetch(value.to_sym) do
+            raise ArgumentError.new("Unknown value for libcurl option #{name}: #{value.inspect}")
+          end
+        end
+      end
+    end
+  end
+end