webmock 3.28.0 → 3.30.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 337d9a801a6b8cabda8c8d3646346e475d9ec46f15453301314b1575b5e6e1f6
-  data.tar.gz: c33b8a74b6fbd42807eb12bb2a385441f953abfb1112c63f8fc1249be0aea38d
+  metadata.gz: 0be0b6d1129e7014a36a5dbe7a7e77d3a25852d1668f54c455e0be96cd2d1287
+  data.tar.gz: db8c3267c3fbf2eef3b69f4b72764a82983e589a6508c9d0be8d2cd53047d484
 SHA512:
-  metadata.gz: 6dad3e43360bf3a82ff3cb6ba1655884c6a0fa4de4767699cc1ce832da1eb1cfe6bf0f37082a1e6d4c80edd839925bb468fad5a75012eb786b6f31342ce510a5
-  data.tar.gz: bfbaecd774edb88c0c09de2c8079d05b0d94b197343684cdd426f66e6c24237d9ba7302373a32b3608a1ea59c902f766e9b28b8532671dedfb83e12b8474b4d3
+  metadata.gz: ad6692051819395acd28a095acb54762b81b379b9d75115f0572351f2693973558978b24392f2086bb2dd6bb3969730881efb0d2dc54210440c9114bc2c867b6
+  data.tar.gz: d4c877e82c741f3319e66d803421eb0fb52f3d1a798465bc19c8b55e52d00fc57fca89cd5eb82a52f88d9f0223c2784adcb7110838405a42f05e9e5c26f53561

data/CHANGELOG.md

@@ -1,5 +1,33 @@
 # Changelog
 
+# 3.30.0
+
+* Allow narrowing an RSpec expectation on a stub with `with`, as in `expect(stub).to have_been_requested.once.with(body: "abc")` (#1128)
+
+    The expectation counts only the requests that match both the stub and the conditions given to `with`. The stub itself does not change.
+
+    Thanks to [Sébastien S.](https://github.com/inkstak)
+
+* Show the JSON body hint for expectations narrowed with `with` too, and no longer show it when a block is given to the stub or expectation, since checking would mean calling the block
+
+* Allow a full URL with a path in `disable_net_connect!(allow: ...)`, as in `allow: "https://github.com/mozilla/geckodriver/releases/latest"` (#977)
+
+    URLs are compared the same way as in stubs, so the default port, letter case in the host, and the order and encoding of query parameters don't matter. A URL without a path or query allows every path on that scheme, host and port; a trailing slash restricts requests to `/` with no query string. Credentials in either URL are ignored.
+
+    Thanks to [Henry Blyth](https://github.com/henrahmagix)
+
+# 3.29.0
+
+* Show a body diff for JSON requests whose content type has a `+json` suffix, such as `application/vnd.api+json`, and for the other content types WebMock already matches as JSON (#734)
+
+    Thanks to [Olivier Brisse](https://github.com/ouranos)
+
+* Ignore the case of JSON, XML and url-encoded content types when matching request bodies, and match bodies of any `+json` or `+xml` media type, such as `image/svg+xml`, as JSON or XML
+
+* Suggest declaring a JSON body as a hash when a request fails to match only because the JSON is formatted differently (#649)
+
+    A body declared as a string must match the request body exactly, including key order and whitespace. When that is the only difference, the error for an unregistered request and the failure message of a request expectation now say so, and suggest `with(body: JSON.parse(json_string))`. Matching JSON strings regardless of key order by default is planned for the next major version (#485).
+
 # 3.28.0
 
 * Fix `to_rack` apps receiving a `rack.session` they did not set up (#985)

data/README.md

@@ -176,6 +176,8 @@ RestClient.post('www.example.com', '<data a="1" b="five" />',
   content_type: 'application/xml')    # ===> Success
 ```
 
+A body declared as a string must match the request body exactly. For JSON, that includes key order and whitespace, so declare the body as a hash to match it as JSON, for example `with(body: JSON.parse(json_string))`.
+
 ### Matching request body against partial hash.
 
 ```ruby
@@ -571,6 +573,21 @@ RestClient.get('www.example.org', '/')      # ===> Failure
 RestClient.get('www.example.org:8080', '/') # ===> Allowed
 ```
 
+With a `String` specifying a full URL:
+
+```ruby
+WebMock.disable_net_connect!(allow: 'https://www.example.org/health')
+
+RestClient.get('https://www.example.org/health')        # ===> Allowed
+RestClient.get('https://www.example.org/health?full=1') # ===> Failure
+RestClient.get('https://www.example.org/health/db')     # ===> Failure
+RestClient.get('https://www.example.org/')              # ===> Failure
+```
+
+A URL with a path allows only that path and query string. URLs are compared the same way as in stubs, so the default port, letter case in the host, and the order and encoding of query parameters don't matter.
+
+A URL without a path or query, such as `https://www.example.org`, allows every path on that scheme, host and port. Adding the trailing slash (`https://www.example.org/`) allows only the root path `/` with no query string. To allow every path under a prefix, use a `Regexp`.
+
 With a `Regexp` matching the URI:
 
 ```ruby
@@ -635,6 +652,8 @@ This forces WebMock Net::HTTP adapter to always connect on `Net::HTTP.start`. At
 WebMock.allow_net_connect!(net_http_connect_on_start: "www.example.com")
 ```
 
+The path is not known when the connection is made, so a URL with a path given here never matches.
+
 ## Setting Expectations
 
 ### Setting expectations in Test::Unit
@@ -749,11 +768,17 @@ expect(a_request(:post, "www.example.com").
 ### Setting expectations in RSpec on the stub
 
 ```ruby
-stub = stub_request(:get, "www.example.com")
+stub = stub_request(:post, "www.example.com")
+
 # ... make requests ...
-expect(stub).to have_been_requested
+
+expect(stub).to have_been_requested.twice
+expect(stub).to have_been_requested.once.with(body: {"a" => [1, 2, 3]})
+expect(stub).to have_been_requested.once.with(body: {"a" => [4, 5, 6]})
 ```
 
+Options and a block given to `with` take the same form as in `stub_request(...).with`. The expectation then counts only the requests that match both the stub and those conditions. The stub itself does not change.
+
 ## Clearing stubs and request history
 
 If you want to reset all current stubs and history of requests use `WebMock.reset!`

data/lib/webmock/json_body_hint.rb

@@ -0,0 +1,77 @@
+# frozen_string_literal: true
+
+require "json"
+
+module WebMock
+  # A body declared as a string must match the request body exactly, which for
+  # JSON includes key order and whitespace. When a request fails to match only
+  # because of that, failure messages suggest declaring the body as a hash.
+  # See #485.
+  module JsonBodyHint
+    ADVICE = "A body declared as a string must match exactly. To match it as JSON, " \
+             "declare it as a hash, for example with(body: JSON.parse(json_string))."
+
+    STUB_HINT = "Hint: the request body is the same JSON as the body of this stub, " \
+                "but formatted differently, for example with keys in another order. #{ADVICE}"
+
+    EXPECTATION_HINT = "Hint: a request was made with the same JSON body as expected, " \
+                       "but formatted differently, for example with keys in another order. #{ADVICE}"
+
+    # True when one of the patterns declares its body as a JSON object or array
+    # string, the request has a JSON content type and a body that parses to the
+    # same JSON but is written differently, and the method, URI and headers of
+    # every pattern match the request. Any other patterns the request has to
+    # match, such as the conditions of an RSpec expectation, must not declare
+    # a body of their own.
+    #
+    # Building a failure message must not call a block given to with, which
+    # could raise and replace the failure, or repeat side effects. So there is
+    # no hint when any of the patterns has one.
+    #
+    # The body is checked first, since that runs no user code, so URI and
+    # header matchers are only evaluated when a hint is otherwise possible.
+    def self.applies?(request_pattern, request_signature)
+      patterns = parts_of(request_pattern)
+      return false unless patterns.all? { |pattern| pattern.is_a?(RequestPattern) && pattern.with_block.nil? }
+
+      with_body = patterns.select(&:body_pattern)
+      return false unless with_body.size == 1
+      return false unless body_differs_only_in_formatting?(with_body.first.body_pattern, request_signature)
+
+      patterns.all? { |pattern| matches_except_body?(pattern, request_signature) }
+    end
+
+    # A pattern made of parts that must all match, such as an RSpec expectation
+    # with conditions, responds to #patterns. A stub stands for its pattern.
+    def self.parts_of(request_pattern)
+      parts = request_pattern.respond_to?(:patterns) ? request_pattern.patterns : [request_pattern]
+      parts.map { |part| part.is_a?(RequestStub) ? part.request_pattern : part }
+    end
+
+    def self.body_differs_only_in_formatting?(body_pattern, request_signature)
+      expected = body_pattern.pattern
+      actual = request_signature.body
+      return false unless expected.is_a?(String) && actual.is_a?(String) && expected != actual
+      return false unless request_signature.json_headers?
+
+      expected_json = parse_json(expected)
+      return false unless expected_json.is_a?(Hash) || expected_json.is_a?(Array)
+
+      # eql? compares nested values strictly, so 1 and 1.0 are different.
+      expected_json.eql?(parse_json(actual))
+    end
+
+    def self.matches_except_body?(request_pattern, request_signature)
+      request_pattern.method_pattern.matches?(request_signature.method) &&
+        request_pattern.uri_pattern.matches?(request_signature.uri) &&
+        (request_pattern.headers_pattern.nil? || request_pattern.headers_pattern.matches?(request_signature.headers))
+    end
+
+    def self.parse_json(string)
+      ::JSON.parse(string)
+    rescue ::JSON::JSONError, EncodingError
+      nil
+    end
+    private_class_method :parts_of, :body_differs_only_in_formatting?, :matches_except_body?, :parse_json
+  end
+end

data/lib/webmock/request_execution_verifier.rb

@@ -56,9 +56,31 @@ module WebMock
     def failure_message_phrase(is_negated=false)
       negation = is_negated ? "was not" : "was"
       "The request #{request_pattern} #{negation} expected to execute #{quantity_phrase(is_negated)}but it executed #{times(times_executed)}" +
+        (is_negated ? "" : json_body_hint) +
         self.class.executed_requests_message
     end
 
+    # Only when too few requests matched: with too many, declaring the body
+    # as a hash would match even more of them.
+    def json_body_hint
+      return "" unless WebMock.show_body_diff? && too_few_requests_executed?
+
+      requests = RequestRegistry.instance.requested_signatures.select do |request_signature, _|
+        JsonBodyHint.applies?(request_pattern, request_signature)
+      end
+      requests.empty? ? "" : "\n\n#{JsonBodyHint::EXPECTATION_HINT}"
+    end
+
+    def too_few_requests_executed?
+      if @at_least_times_executed
+        times_executed < @at_least_times_executed
+      elsif @at_most_times_executed
+        false
+      else
+        times_executed < (@expected_times_executed || 1)
+      end
+    end
+
     def quantity_phrase(is_negated=false)
       if @at_least_times_executed
         "at least #{times(@at_least_times_executed)} "

data/lib/webmock/request_pattern.rb

@@ -14,7 +14,7 @@ module WebMock
 
   class RequestPattern
 
-    attr_reader :method_pattern, :uri_pattern, :body_pattern, :headers_pattern
+    attr_reader :method_pattern, :uri_pattern, :body_pattern, :headers_pattern, :with_block
 
     def initialize(method, uri, options = {})
       @method_pattern  = MethodPattern.new(method)
@@ -33,11 +33,9 @@ module WebMock
     end
 
     def matches?(request_signature)
-      content_type = request_signature.content_type
-      content_type = content_type.split(';').first if content_type
       @method_pattern.matches?(request_signature.method) &&
         @uri_pattern.matches?(request_signature.uri) &&
-        (@body_pattern.nil? || @body_pattern.matches?(request_signature.body, content_type || "")) &&
+        (@body_pattern.nil? || @body_pattern.matches?(request_signature.body, request_signature.content_type)) &&
         (@headers_pattern.nil? || @headers_pattern.matches?(request_signature.headers)) &&
         (@with_block.nil? || @with_block.call(request_signature))
     end
@@ -255,18 +253,8 @@ module WebMock
   class BodyPattern
     include RSpecMatcherDetector
 
-    BODY_FORMATS = {
-      'text/xml'               => :xml,
-      'application/xml'        => :xml,
-      'application/json'       => :json,
-      'text/json'              => :json,
-      'application/javascript' => :json,
-      'text/javascript'        => :json,
-      'text/html'              => :html,
-      'application/x-yaml'     => :yaml,
-      'text/yaml'              => :yaml,
-      'text/plain'             => :plain
-    }
+    # Kept so existing references to the constant still resolve.
+    BODY_FORMATS = WebMock::Util::ContentType::BODY_FORMATS
 
     attr_reader :pattern
 
@@ -283,6 +271,8 @@ module WebMock
     end
 
     def matches?(body, content_type = "")
+      # Checked on the raw value, so body patterns used with mixed case
+      # multipart content types keep working as in earlier versions.
       assert_non_multipart_body(content_type)
 
       if (@pattern).is_a?(Hash)
@@ -308,7 +298,7 @@ module WebMock
     def body_as_hash(body, content_type)
       return if body.nil?
 
-      case body_format(content_type)
+      case WebMock::Util::ContentType.body_format(content_type)
       when :json then
         WebMock::Util::Parsers::JSON.parse(body)
       when :xml then
@@ -320,11 +310,6 @@ module WebMock
       nil
     end
 
-    def body_format(content_type)
-      normalized_content_type = content_type.sub(/\A(application\/)[a-zA-Z0-9.-]+\+(json|xml)\Z/,'\1\2')
-      BODY_FORMATS[normalized_content_type]
-    end
-
     def assert_non_multipart_body(content_type)
       if content_type =~ %r{^multipart/form-data}
         raise ArgumentError.new("WebMock does not support matching body for multipart/form-data requests yet :(")
@@ -380,7 +365,7 @@ module WebMock
       return matching_body_hashes?(actual, expected, content_type) if actual.is_a?(Hash) && expected.is_a?(Hash)
       return matching_body_array?(actual, expected, content_type) if actual.is_a?(Array) && expected.is_a?(Array)
 
-      expected = WebMock::Util::ValuesStringifier.stringify_values(expected) if url_encoded_body?(content_type)
+      expected = WebMock::Util::ValuesStringifier.stringify_values(expected) if WebMock::Util::ContentType.url_encoded?(content_type)
       expected === actual
     end
 
@@ -391,10 +376,6 @@ module WebMock
     def normalize_hash(hash)
       Hash[WebMock::Util::HashKeysStringifier.stringify_keys!(hash, deep: true).sort]
     end
-
-    def url_encoded_body?(content_type)
-      content_type =~ %r{^application/x-www-form-urlencoded}
-    end
   end
 
   class HeadersPattern

data/lib/webmock/request_signature.rb

@@ -43,11 +43,11 @@ module WebMock
     end
 
     def url_encoded?
-      !!content_type&.start_with?('application/x-www-form-urlencoded')
+      WebMock::Util::ContentType.url_encoded?(content_type)
     end
 
     def json_headers?
-      !!content_type&.start_with?('application/json')
+      WebMock::Util::ContentType.body_format(content_type) == :json
     end
 
     private

data/lib/webmock/request_signature_snippet.rb

@@ -25,7 +25,10 @@ module WebMock
       text = "registered request stubs:\n".dup
       WebMock::StubRegistry.instance.request_stubs.each do |stub|
         text << "\n#{WebMock::StubRequestSnippet.new(stub).to_s(false)}"
-        add_body_diff(stub, text) if WebMock.show_body_diff?
+        if WebMock.show_body_diff?
+          add_body_diff(stub, text)
+          add_json_body_hint(stub, text)
+        end
       end
       text
     end
@@ -37,6 +40,12 @@ module WebMock
       text << "\n\n#{body_diff_str}" unless body_diff_str.empty?
     end
 
+    def add_json_body_hint(stub, text)
+      return unless JsonBodyHint.applies?(stub.request_pattern, request_signature)
+
+      text << "\n\n#{JsonBodyHint::STUB_HINT}\n"
+    end
+
     def signature_stub_body_diff(stub)
       diff = RequestBodyDiff.new(request_signature, stub).body_diff
       diff.empty? ? "" : "Body diff:\n #{pretty_print_to_string(diff)}"

data/lib/webmock/rspec/matchers/request_pattern_matcher.rb

@@ -52,13 +52,22 @@ module WebMock
       self
     end
 
+    # Counts only the requests that also match the given conditions, which
+    # take the same options and block as RequestPattern#with. Like
+    # RequestPattern#with, calling it again adds to the conditions. The stub
+    # or request pattern under test is not changed.
+    def with(options = {}, &block)
+      (@conditions ||= RequestPattern.new(:any, ANY_URI)).with(options, &block)
+      self
+    end
+
     def matches?(request_pattern)
-      @request_execution_verifier.request_pattern = request_pattern
+      @request_execution_verifier.request_pattern = with_conditions(request_pattern)
       @request_execution_verifier.matches?
     end
 
     def does_not_match?(request_pattern)
-      @request_execution_verifier.request_pattern = request_pattern
+      @request_execution_verifier.request_pattern = with_conditions(request_pattern)
       @request_execution_verifier.does_not_match?
     end
 
@@ -76,5 +85,41 @@ module WebMock
 
     # RSpec 2 compatibility:
     alias_method :negative_failure_message, :failure_message_when_negated
+
+    private
+
+    ANY_URI = ->(_uri) { true }
+    private_constant :ANY_URI
+
+    # How the description of conditions starts when nothing was given to
+    # #with. What was given follows it, as in ' with body "abc"'.
+    EMPTY_CONDITIONS = RequestPattern.new(:any, ANY_URI).to_s.freeze
+    private_constant :EMPTY_CONDITIONS
+
+    def with_conditions(request_pattern)
+      @conditions ? PatternWithConditions.new(request_pattern, @conditions) : request_pattern
+    end
+
+    # Matches a request when both the pattern and the conditions match it.
+    class PatternWithConditions
+      def initialize(pattern, conditions)
+        @pattern = pattern
+        @conditions = conditions
+      end
+
+      # The parts that must all match, so JsonBodyHint can check each of them.
+      def patterns
+        [@pattern, @conditions]
+      end
+
+      def matches?(request_signature)
+        @pattern.matches?(request_signature) && @conditions.matches?(request_signature)
+      end
+
+      def to_s
+        "#{@pattern} (only requests#{@conditions.to_s.delete_prefix(EMPTY_CONDITIONS)})"
+      end
+    end
+    private_constant :PatternWithConditions
   end
 end

data/lib/webmock/util/content_type.rb

@@ -0,0 +1,39 @@
+# frozen_string_literal: true
+
+module WebMock
+  module Util
+    module ContentType
+      BODY_FORMATS = {
+        'text/xml'               => :xml,
+        'application/xml'        => :xml,
+        'application/json'       => :json,
+        'text/json'              => :json,
+        'application/javascript' => :json,
+        'text/javascript'        => :json,
+        'text/html'              => :html,
+        'application/x-yaml'     => :yaml,
+        'text/yaml'              => :yaml,
+        'text/plain'             => :plain
+      }
+
+      def self.media_type(content_type)
+        content_type.to_s.scrub.split(';', 2).first.to_s.strip.downcase
+      end
+
+      def self.url_encoded?(content_type)
+        media_type(content_type) == 'application/x-www-form-urlencoded'
+      end
+
+      def self.body_format(content_type)
+        type = media_type(content_type)
+
+        BODY_FORMATS.fetch(type) do
+          case type
+          when %r{\A[^/\s]+/[^/\s]+\+json\z} then :json
+          when %r{\A[^/\s]+/[^/\s]+\+xml\z} then :xml
+          end
+        end
+      end
+    end
+  end
+end

data/lib/webmock/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module WebMock
-  VERSION = '3.28.0' unless defined?(::WebMock::VERSION)
+  VERSION = '3.30.0' unless defined?(::WebMock::VERSION)
 end

data/lib/webmock/webmock.rb

@@ -87,14 +87,12 @@ module WebMock
     when Array
       allowed.any? { |allowed_item| net_connect_explicit_allowed?(allowed_item, uri) }
     when Regexp
-      (uri.to_s =~ allowed) != nil ||
-      (uri.omit(:port).to_s =~ allowed) != nil && uri.port == uri.default_port
+      allowed.match?(uri.to_s) ||
+      (uri.port == uri.default_port && allowed.match?(uri.omit(:port).to_s))
     when String
-      allowed == uri.to_s ||
       allowed == uri.host ||
       allowed == "#{uri.host}:#{uri.port}" ||
-      allowed == "#{uri.scheme}://#{uri.host}:#{uri.port}" ||
-      allowed == "#{uri.scheme}://#{uri.host}" && uri.port == uri.default_port
+      net_connect_allowed_url?(allowed, uri)
     else
       if allowed.respond_to?(:call)
         allowed.call(uri)
@@ -102,6 +100,28 @@ module WebMock
     end
   end
 
+  def self.net_connect_allowed_url?(allowed, uri)
+    return false unless allowed.match?(%r{\Ahttps?://}i)
+
+    # Normalization turns an empty path into "/", so check the original URL.
+    allowed_uri = Addressable::URI.parse(allowed)
+    allow_any_path = allowed_uri.path.empty? && allowed_uri.query.nil?
+
+    # Connection URIs have no request path to match yet.
+    return false if uri.path.empty? && !allowed_uri.path.empty?
+
+    allowed_uri = WebMock::Util::URI.normalize_uri(allowed_uri)
+    request_uri = WebMock::Util::URI.normalize_uri(uri)
+
+    return false unless allowed_uri.origin == request_uri.origin
+    return true if allow_any_path
+
+    allowed_uri.path == request_uri.path && allowed_uri.query == request_uri.query
+  rescue Addressable::URI::InvalidURIError
+    false
+  end
+  private_class_method :net_connect_allowed_url?
+
   def self.show_body_diff!
     Config.instance.show_body_diff = true
   end

data/lib/webmock.rb

@@ -13,6 +13,7 @@ require_relative 'webmock/errors'
 require_relative 'webmock/util/query_mapper'
 require_relative 'webmock/util/uri'
 require_relative 'webmock/util/headers'
+require_relative 'webmock/util/content_type'
 require_relative 'webmock/util/hash_counter'
 require_relative 'webmock/util/hash_keys_stringifier'
 require_relative 'webmock/util/values_stringifier'
@@ -36,6 +37,7 @@ require_relative 'webmock/rack_response'
 require_relative 'webmock/stub_request_snippet'
 require_relative 'webmock/request_signature_snippet'
 require_relative 'webmock/request_body_diff'
+require_relative 'webmock/json_body_hint'
 
 require_relative 'webmock/assertion_failure'
 require_relative 'webmock/request_execution_verifier'

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: webmock
 version: !ruby/object:Gem::Version
-  version: 3.28.0
+  version: 3.30.0
 platform: ruby
 authors:
 - Bartosz Blimke
@@ -318,6 +318,7 @@ files:
 - lib/webmock/http_lib_adapters/net_http_response.rb
 - lib/webmock/http_lib_adapters/patron_adapter.rb
 - lib/webmock/http_lib_adapters/typhoeus_hydra_adapter.rb
+- lib/webmock/json_body_hint.rb
 - lib/webmock/matchers/any_arg_matcher.rb
 - lib/webmock/matchers/hash_argument_matcher.rb
 - lib/webmock/matchers/hash_excluding_matcher.rb
@@ -340,6 +341,7 @@ files:
 - lib/webmock/stub_registry.rb
 - lib/webmock/stub_request_snippet.rb
 - lib/webmock/test_unit.rb
+- lib/webmock/util/content_type.rb
 - lib/webmock/util/hash_counter.rb
 - lib/webmock/util/hash_keys_stringifier.rb
 - lib/webmock/util/hash_validator.rb
@@ -358,9 +360,9 @@ licenses:
 - MIT
 metadata:
   bug_tracker_uri: https://github.com/bblimke/webmock/issues
-  changelog_uri: https://github.com/bblimke/webmock/blob/v3.28.0/CHANGELOG.md
-  documentation_uri: https://www.rubydoc.info/gems/webmock/3.28.0
-  source_code_uri: https://github.com/bblimke/webmock/tree/v3.28.0
+  changelog_uri: https://github.com/bblimke/webmock/blob/v3.30.0/CHANGELOG.md
+  documentation_uri: https://www.rubydoc.info/gems/webmock/3.30.0
+  source_code_uri: https://github.com/bblimke/webmock/tree/v3.30.0
   wiki_uri: https://github.com/bblimke/webmock/wiki
 rdoc_options: []
 require_paths: