http 5.0.0.pre3 → 5.0.3

data/README.md

@@ -1,86 +1,32 @@
-# ![http.rb](https://raw.github.com/httprb/http.rb/master/logo.png)
+# ![http.rb](https://raw.github.com/httprb/http.rb/main/logo.png)
 
-[![Gem Version](https://badge.fury.io/rb/http.svg)](https://rubygems.org/gems/http)
-[![Build Status](https://secure.travis-ci.org/httprb/http.svg?branch=master)](https://travis-ci.org/httprb/http)
-[![Code Climate](https://codeclimate.com/github/httprb/http.svg?branch=master)](https://codeclimate.com/github/httprb/http)
-[![Coverage Status](https://coveralls.io/repos/httprb/http/badge.svg?branch=master)](https://coveralls.io/r/httprb/http)
-[![MIT licensed](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/httprb/http/blob/master/LICENSE.txt)
+[![Gem Version][gem-image]][gem-link]
+[![MIT licensed][license-image]][license-link]
+[![Build Status][build-image]][build-link]
+[![Code Climate][codeclimate-image]][codeclimate-link]
 
 [Documentation]
 
-_NOTE: This is the 5.x **development** branch.  For the 4.x **stable** branch, please see:_
-
-https://github.com/httprb/http/tree/4-x-stable
-
 ## About
 
 HTTP (The Gem! a.k.a. http.rb) is an easy-to-use client library for making requests
 from Ruby. It uses a simple method chaining system for building requests, similar to
 Python's [Requests].
 
-Under the hood, via [Ruby FFI bindings][http-parser-ffi], http.rb uses the Node.js
-[http-parser], a fast HTTP parsing native extension. This library
-isn't just yet another wrapper around Net::HTTP. It implements the HTTP protocol
-natively and outsources the parsing to native extensions.
-
-[requests]: http://docs.python-requests.org/en/latest/
-[http-parser]: https://github.com/nodejs/http-parser
-[http-parser-ffi]: https://github.com/cotag/http-parser
-
-
-## Another Ruby HTTP library? Why should I care?
-
-There are a lot of HTTP libraries to choose from in the Ruby ecosystem.
-So why would you choose this one?
+Under the hood, http.rb uses the [llhttp] parser, a fast HTTP parsing native extension.
+This library isn't just yet another wrapper around `Net::HTTP`. It implements the HTTP
+protocol natively and outsources the parsing to native extensions.
 
-Top three reasons:
+### Why http.rb?
 
-1. **Clean API**: http.rb offers an easy-to-use API that should be a
+- **Clean API**: http.rb offers an easy-to-use API that should be a
    breath of fresh air after using something like Net::HTTP.
 
-2. **Maturity**: http.rb is one of the most mature Ruby HTTP clients, supporting
+- **Maturity**: http.rb is one of the most mature Ruby HTTP clients, supporting
    features like persistent connections and fine-grained timeouts.
 
-3. **Performance**: using native parsers and a clean, lightweight implementation,
-   http.rb achieves the best performance of any Ruby HTTP library which
-   implements the HTTP protocol in Ruby instead of C:
-
-  | HTTP client              | Time   | Implementation        |
-  |--------------------------|--------|-----------------------|
-  | curb (persistent)        | 2.519  | libcurl wrapper       |
-  | em-http-request          | 2.731  | EM + http_parser.rb   |
-  | Typhoeus                 | 2.851  | libcurl wrapper       |
-  | StreamlyFFI (persistent) | 2.853  | libcurl wrapper       |
-  | http.rb (persistent)     | 2.970  | Ruby + http_parser.rb |
-  | http.rb                  | 3.588  | Ruby + http_parser.rb |
-  | HTTParty                 | 3.931  | Net::HTTP wrapper     |
-  | Net::HTTP                | 3.959  | Pure Ruby             |
-  | Net::HTTP (persistent)   | 4.043  | Pure Ruby             |
-  | open-uri                 | 4.479  | Net::HTTP wrapper     |
-  | Excon (persistent)       | 4.618  | Pure Ruby             |
-  | Excon                    | 4.701  | Pure Ruby             |
-  | RestClient               | 26.838 | Net::HTTP wrapper     |
-
-Benchmarks performed using excon's benchmarking tool
-
-DISCLAIMER: Most benchmarks you find in READMEs are crap,
-including this one. These are out-of-date. If you care about
-performance, benchmark for yourself for your own use cases!
-
-## Help and Discussion
-
-If you need help or just want to talk about the http.rb,
-visit the http.rb Google Group:
-
-https://groups.google.com/forum/#!forum/httprb
-
-You can join by email by sending a message to:
-
-[httprb+subscribe@googlegroups.com](mailto:httprb+subscribe@googlegroups.com)
-
-If you believe you've found a bug, please report it at:
-
-https://github.com/httprb/http/issues
+- **Performance**: using native parsers and a clean, lightweight implementation,
+   http.rb achieves high performance while implementing HTTP in Ruby instead of C.
 
 
 ## Installation
@@ -115,10 +61,9 @@ for more detailed documentation and usage notes.
 
 The following API documentation is also available:
 
-* [YARD API documentation](http://www.rubydoc.info/gems/http/frames)
-* [Chainable module (all chainable methods)](http://www.rubydoc.info/gems/http/HTTP/Chainable)
+- [YARD API documentation](https://www.rubydoc.info/github/httprb/http)
+- [Chainable module (all chainable methods)](https://www.rubydoc.info/github/httprb/http/HTTP/Chainable)
 
-[documentation]: https://github.com/httprb/http/wiki
 
 ### Basic Usage
 
@@ -145,7 +90,7 @@ We can also obtain an `HTTP::Response::Body` object for this response:
 ```
 
 The response body can be streamed with `HTTP::Response::Body#readpartial`.
-In practice, you'll want to bind the HTTP::Response::Body to a local variable
+In practice, you'll want to bind the `HTTP::Response::Body` to a local variable
 and call `#readpartial` on it repeatedly until it returns `nil`:
 
 ```ruby
@@ -162,14 +107,13 @@ and call `#readpartial` on it repeatedly until it returns `nil`:
 
 ## Supported Ruby Versions
 
-This library aims to support and is [tested against][travis] the following Ruby
-versions:
+This library aims to support and is [tested against][build-link]
+the following Ruby  versions:
 
-* Ruby 2.4.x
-* Ruby 2.5.x
-* Ruby 2.6.x
-* Ruby 2.7.x
-* JRuby 9.2.x.x
+- Ruby 2.6
+- Ruby 2.7
+- Ruby 3.0
+- JRuby 9.2
 
 If something doesn't work on one of these versions, it's a bug.
 
@@ -184,20 +128,36 @@ patches in a timely fashion. If critical issues for a particular implementation
 exist at the time of a major release, support for that Ruby version may be
 dropped.
 
-[travis]: http://travis-ci.org/httprb/http
-
 
 ## Contributing to http.rb
 
-* Fork http.rb on GitHub
-* Make your changes
-* Ensure all tests pass (`bundle exec rake`)
-* Send a pull request
-* If we like them we'll merge them
-* If we've accepted a patch, feel free to ask for commit access!
+- Fork http.rb on GitHub
+- Make your changes
+- Ensure all tests pass (`bundle exec rake`)
+- Send a pull request
+- If we like them we'll merge them
+- If we've accepted a patch, feel free to ask for commit access!
 
 
 ## Copyright
 
-Copyright (c) 2011-2019 Tony Arcieri, Alexey V. Zapparov, Erik Michaels-Ober, Zachary Anker.
+Copyright © 2011-2021 Tony Arcieri, Alexey V. Zapparov, Erik Michaels-Ober, Zachary Anker.
 See LICENSE.txt for further details.
+
+
+[//]: # (badges)
+
+[gem-image]: https://img.shields.io/gem/v/http?logo=ruby
+[gem-link]: https://rubygems.org/gems/http
+[license-image]: https://img.shields.io/badge/license-MIT-blue.svg
+[license-link]: https://github.com/httprb/http/blob/main/LICENSE.txt
+[build-image]: https://github.com/httprb/http/workflows/CI/badge.svg
+[build-link]: https://github.com/httprb/http/actions/workflows/ci.yml
+[codeclimate-image]: https://codeclimate.com/github/httprb/http.svg?branch=main
+[codeclimate-link]: https://codeclimate.com/github/httprb/http
+
+[//]: # (links)
+
+[documentation]: https://github.com/httprb/http/wiki
+[requests]: http://docs.python-requests.org/en/latest/
+[llhttp]: https://llhttp.org/

data/Rakefile

@@ -35,7 +35,7 @@ task :generate_status_codes do
   end
 
   File.open("./lib/http/response/status/reasons.rb", "w") do |io|
-    io.puts <<-TPL.gsub(/^[ ]{6}/, "")
+    io.puts <<~TPL
       # AUTO-GENERATED FILE, DO NOT CHANGE IT MANUALLY
 
       require "delegate"
@@ -61,12 +61,4 @@ task :generate_status_codes do
   end
 end
 
-if ENV["CI"].nil?
-  task :default => %i[spec rubocop verify_measurements]
-else
-  case ENV["SUITE"]
-  when "rubocop"   then task :default => :rubocop
-  when "yardstick" then task :default => :verify_measurements
-  else                  task :default => :spec
-  end
-end
+task :default => %i[spec rubocop verify_measurements]

data/http.gemspec

@@ -25,12 +25,12 @@ Gem::Specification.new do |gem|
   gem.require_paths = ["lib"]
   gem.version       = HTTP::VERSION
 
-  gem.required_ruby_version = ">= 2.3"
+  gem.required_ruby_version = ">= 2.5"
 
-  gem.add_runtime_dependency "addressable",    "~> 2.3"
+  gem.add_runtime_dependency "addressable",    "~> 2.8"
   gem.add_runtime_dependency "http-cookie",    "~> 1.0"
   gem.add_runtime_dependency "http-form_data", "~> 2.2"
-  gem.add_runtime_dependency "http-parser",    "~> 1.2.0"
+  gem.add_runtime_dependency "llhttp-ffi",     "~> 0.4.0"
 
   gem.add_development_dependency "bundler", "~> 2.0"
 

data/lib/http/chainable.rb

@@ -9,63 +9,63 @@ module HTTP
     # Request a get sans response body
     # @param uri
     # @option options [Hash]
-    def head(uri, options = {}) # rubocop:disable Style/OptionHash
+    def head(uri, options = {})
       request :head, uri, options
     end
 
     # Get a resource
     # @param uri
     # @option options [Hash]
-    def get(uri, options = {}) # rubocop:disable Style/OptionHash
+    def get(uri, options = {})
       request :get, uri, options
     end
 
     # Post to a resource
     # @param uri
     # @option options [Hash]
-    def post(uri, options = {}) # rubocop:disable Style/OptionHash
+    def post(uri, options = {})
       request :post, uri, options
     end
 
     # Put to a resource
     # @param uri
     # @option options [Hash]
-    def put(uri, options = {}) # rubocop:disable Style/OptionHash
+    def put(uri, options = {})
       request :put, uri, options
     end
 
     # Delete a resource
     # @param uri
     # @option options [Hash]
-    def delete(uri, options = {}) # rubocop:disable Style/OptionHash
+    def delete(uri, options = {})
       request :delete, uri, options
     end
 
     # Echo the request back to the client
     # @param uri
     # @option options [Hash]
-    def trace(uri, options = {}) # rubocop:disable Style/OptionHash
+    def trace(uri, options = {})
       request :trace, uri, options
     end
 
     # Return the methods supported on the given URI
     # @param uri
     # @option options [Hash]
-    def options(uri, options = {}) # rubocop:disable Style/OptionHash
+    def options(uri, options = {})
       request :options, uri, options
     end
 
     # Convert to a transparent TCP/IP tunnel
     # @param uri
     # @option options [Hash]
-    def connect(uri, options = {}) # rubocop:disable Style/OptionHash
+    def connect(uri, options = {})
       request :connect, uri, options
     end
 
     # Apply partial modifications to a resource
     # @param uri
     # @option options [Hash]
-    def patch(uri, options = {}) # rubocop:disable Style/OptionHash
+    def patch(uri, options = {})
       request :patch, uri, options
     end
 
@@ -148,7 +148,7 @@ module HTTP
 
       yield p_client
     ensure
-      p_client.close if p_client
+      p_client&.close
     end
 
     # Make a request through an HTTP proxy
@@ -173,7 +173,7 @@ module HTTP
     # @param options
     # @return [HTTP::Client]
     # @see Redirector#initialize
-    def follow(options = {}) # rubocop:disable Style/OptionHash
+    def follow(options = {})
       branch default_options.with_follow options
     end
 
@@ -211,10 +211,11 @@ module HTTP
     # @option opts [#to_s] :user
     # @option opts [#to_s] :pass
     def basic_auth(opts)
-      user = opts.fetch :user
-      pass = opts.fetch :pass
+      user  = opts.fetch(:user)
+      pass  = opts.fetch(:pass)
+      creds = "#{user}:#{pass}"
 
-      auth("Basic " + Base64.strict_encode64("#{user}:#{pass}"))
+      auth("Basic #{Base64.strict_encode64(creds)}")
     end
 
     # Get options for HTTP

data/lib/http/client.rb

@@ -25,19 +25,19 @@ module HTTP
     end
 
     # Make an HTTP request
-    def request(verb, uri, opts = {}) # rubocop:disable Style/OptionHash
+    def request(verb, uri, opts = {})
       opts = @default_options.merge(opts)
       req = build_request(verb, uri, opts)
       res = perform(req, opts)
       return res unless opts.follow
 
       Redirector.new(opts.follow).perform(req, res) do |request|
-        perform(request, opts)
+        perform(wrap_request(request, opts), opts)
       end
     end
 
     # Prepare an HTTP request
-    def build_request(verb, uri, opts = {}) # rubocop:disable Style/OptionHash
+    def build_request(verb, uri, opts = {})
       opts    = @default_options.merge(opts)
       uri     = make_request_uri(uri, opts)
       headers = make_request_headers(opts)
@@ -52,9 +52,7 @@ module HTTP
         :body           => body
       )
 
-      opts.features.inject(req) do |request, (_name, feature)|
-        feature.wrap_request(request)
-      end
+      wrap_request(req, opts)
     end
 
     # @!method persistent?
@@ -68,9 +66,9 @@ module HTTP
 
       @state = :dirty
 
-      @connection ||= HTTP::Connection.new(req, options)
-
       begin
+        @connection ||= HTTP::Connection.new(req, options)
+
         unless @connection.failed_proxy_connect?
           @connection.send_request(req)
           @connection.read_headers!
@@ -97,13 +95,19 @@ module HTTP
     end
 
     def close
-      @connection.close if @connection
+      @connection&.close
       @connection = nil
       @state = :clean
     end
 
     private
 
+    def wrap_request(req, opts)
+      opts.features.inject(req) do |request, (_name, feature)|
+        feature.wrap_request(request)
+      end
+    end
+
     def build_response(req, options)
       Response.new(
         :status        => @connection.status_code,
@@ -120,15 +124,15 @@ module HTTP
     def verify_connection!(uri)
       if default_options.persistent? && uri.origin != default_options.persistent
         raise StateError, "Persistence is enabled for #{default_options.persistent}, but we got #{uri.origin}"
+      end
+
       # We re-create the connection object because we want to let prior requests
       # lazily load the body as long as possible, and this mimics prior functionality.
-      elsif @connection && (!@connection.keep_alive? || @connection.expired?)
-        close
+      return close if @connection && (!@connection.keep_alive? || @connection.expired?)
+
       # If we get into a bad state (eg, Timeout.timeout ensure being killed)
       # close the connection to prevent potential for mixed responses.
-      elsif @state == :dirty
-        close
-      end
+      return close if @state == :dirty
     end
 
     # Merges query params if needed
@@ -175,7 +179,7 @@ module HTTP
       when opts.body
         opts.body
       when opts.form
-        form = HTTP::FormData.create opts.form
+        form = make_form_data(opts.form)
         headers[Headers::CONTENT_TYPE] ||= form.content_type
         form
       when opts.json
@@ -184,5 +188,12 @@ module HTTP
         body
       end
     end
+
+    def make_form_data(form)
+      return form if form.is_a? HTTP::FormData::Multipart
+      return form if form.is_a? HTTP::FormData::Urlencoded
+
+      HTTP::FormData.create(form)
+    end
   end
 end

data/lib/http/connection.rb

@@ -3,7 +3,6 @@
 require "forwardable"
 
 require "http/headers"
-require "http/response/parser"
 
 module HTTP
   # A connection to the HTTP server
@@ -68,8 +67,13 @@ module HTTP
     # @param [Request] req Request to send to the server
     # @return [nil]
     def send_request(req)
-      raise StateError, "Tried to send a request while one is pending already. Make sure you read off the body." if @pending_response
-      raise StateError, "Tried to send a request while a response is pending. Make sure you read off the body."  if @pending_request
+      if @pending_response
+        raise StateError, "Tried to send a request while one is pending already. Make sure you read off the body."
+      end
+
+      if @pending_request
+        raise StateError, "Tried to send a request while a response is pending. Make sure you read off the body."
+      end
 
       @pending_request = true
 

data/lib/http/content_type.rb

@@ -1,10 +1,12 @@
 # frozen_string_literal: true
 
 module HTTP
-  ContentType = Struct.new(:mime_type, :charset) do
+  class ContentType
     MIME_TYPE_RE = %r{^([^/]+/[^;]+)(?:$|;)}.freeze
     CHARSET_RE   = /;\s*charset=([^;]+)/i.freeze
 
+    attr_accessor :mime_type, :charset
+
     class << self
       # Parse string and return ContentType struct
       def parse(str)
@@ -15,15 +17,18 @@ module HTTP
 
       # :nodoc:
       def mime_type(str)
-        m = str.to_s[MIME_TYPE_RE, 1]
-        m && m.strip.downcase
+        str.to_s[MIME_TYPE_RE, 1]&.strip&.downcase
       end
 
       # :nodoc:
       def charset(str)
-        m = str.to_s[CHARSET_RE, 1]
-        m && m.strip.delete('"')
+        str.to_s[CHARSET_RE, 1]&.strip&.delete('"')
       end
     end
+
+    def initialize(mime_type = nil, charset = nil)
+      @mime_type = mime_type
+      @charset   = charset
+    end
   end
 end

data/lib/http/feature.rb

@@ -2,7 +2,7 @@
 
 module HTTP
   class Feature
-    def initialize(opts = {}) # rubocop:disable Style/OptionHash
+    def initialize(opts = {})
       @opts = opts
     end
 

data/lib/http/features/auto_inflate.rb

@@ -21,8 +21,6 @@ module HTTP
           :request       => response.request
         }
 
-        options[:uri] = response.uri if response.uri
-
         Response.new(options)
       end
 

data/lib/http/features/instrumentation.rb

@@ -29,7 +29,7 @@ module HTTP
       def wrap_request(request)
         # Emit a separate "start" event, so a logger can print the request
         # being run without waiting for a response
-        instrumenter.instrument("start_#{name}", :request => request) {}
+        instrumenter.instrument("start_#{name}", :request => request)
         instrumenter.start(name, :request => request)
         request
       end

data/lib/http/features/logging.rb

@@ -9,6 +9,20 @@ module HTTP
     #    HTTP.use(logging: {logger: Logger.new(STDOUT)}).get("https://example.com/")
     #
     class Logging < Feature
+      HTTP::Options.register_feature(:logging, self)
+
+      class NullLogger
+        %w[fatal error warn info debug].each do |level|
+          define_method(level.to_sym) do |*_args|
+            nil
+          end
+
+          define_method(:"#{level}?") do
+            true
+          end
+        end
+      end
+
       attr_reader :logger
 
       def initialize(logger: NullLogger.new)
@@ -17,39 +31,23 @@ module HTTP
 
       def wrap_request(request)
         logger.info { "> #{request.verb.to_s.upcase} #{request.uri}" }
-        logger.debug do
-          headers = request.headers.map { |name, value| "#{name}: #{value}" }.join("\n")
-          body = request.body.source
+        logger.debug { "#{stringify_headers(request.headers)}\n\n#{request.body.source}" }
 
-          headers + "\n\n" + body.to_s
-        end
         request
       end
 
       def wrap_response(response)
         logger.info { "< #{response.status}" }
-        logger.debug do
-          headers = response.headers.map { |name, value| "#{name}: #{value}" }.join("\n")
-          body = response.body.to_s
+        logger.debug { "#{stringify_headers(response.headers)}\n\n#{response.body}" }
 
-          headers + "\n\n" + body
-        end
         response
       end
 
-      class NullLogger
-        %w[fatal error warn info debug].each do |level|
-          define_method(level.to_sym) do |*_args|
-            nil
-          end
+      private
 
-          define_method(:"#{level}?") do
-            true
-          end
-        end
+      def stringify_headers(headers)
+        headers.map { |name, value| "#{name}: #{value}" }.join("\n")
       end
-
-      HTTP::Options.register_feature(:logging, self)
     end
   end
 end

data/lib/http/headers.rb

@@ -17,7 +17,7 @@ module HTTP
 
     # Matches valid header field name according to RFC.
     # @see http://tools.ietf.org/html/rfc7230#section-3.2
-    COMPLIANT_NAME_RE = /\A[A-Za-z0-9!#\$%&'*+\-.^_`|~]+\z/.freeze
+    COMPLIANT_NAME_RE = /\A[A-Za-z0-9!#$%&'*+\-.^_`|~]+\z/.freeze
 
     # Class constructor.
     def initialize
@@ -111,7 +111,7 @@ module HTTP
     #
     # @return [Hash]
     def to_h
-      Hash[keys.map { |k| [k, self[k]] }]
+      keys.map { |k| [k, self[k]] }.to_h
     end
     alias to_hash to_h
 
@@ -164,7 +164,7 @@ module HTTP
 
     # @!method hash
     #   Compute a hash-code for this headers container.
-    #   Two conatiners with the same content will have the same hash code.
+    #   Two containers with the same content will have the same hash code.
     #
     #   @see http://www.ruby-doc.org/core/Object.html#method-i-hash
     #   @return [Fixnum]

data/lib/http/mime_type/adapter.rb

@@ -14,6 +14,7 @@ module HTTP
         def_delegators :instance, :encode, :decode
       end
 
+      # rubocop:disable Style/DocumentDynamicEvalDefinition
       %w[encode decode].each do |operation|
         class_eval <<-RUBY, __FILE__, __LINE__ + 1
           def #{operation}(*)
@@ -21,6 +22,7 @@ module HTTP
           end
         RUBY
       end
+      # rubocop:enable Style/DocumentDynamicEvalDefinition
     end
   end
 end

data/lib/http/options.rb

@@ -32,7 +32,7 @@ module HTTP
 
       def def_option(name, reader_only: false, &interpreter)
         defined_options << name.to_sym
-        interpreter ||= lambda { |v| v }
+        interpreter ||= ->(v) { v }
 
         if reader_only
           attr_reader name
@@ -47,7 +47,7 @@ module HTTP
       end
     end
 
-    def initialize(options = {}) # rubocop:disable Style/OptionHash
+    def initialize(options = {})
       defaults = {
         :response           => :auto,
         :proxy              => {},

data/lib/http/redirector.rb

@@ -39,7 +39,7 @@ module HTTP
     # @param [Hash] opts
     # @option opts [Boolean] :strict (true) redirector hops policy
     # @option opts [#to_i] :max_hops (5) maximum allowed amount of hops
-    def initialize(opts = {}) # rubocop:disable Style/OptionHash
+    def initialize(opts = {})
       @strict   = opts.fetch(:strict, true)
       @max_hops = opts.fetch(:max_hops, 5).to_i
     end