http_instrumentation 1.0.1 → 1.0.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1457b4d97046d8c0a41e99aa0825386e3540eb67ac5603fb0ecc80e2fca77e92
-  data.tar.gz: 78b242f2f2821f22d8abddd754e641a271232da63e8218b2492ecd0907e43eca
+  metadata.gz: 2deee15a4609f8d50a706613af0805d53bc54beaf3e17f65b68f17c17453fed9
+  data.tar.gz: a3a76bc88fe4a5b799b59a807ac665dcd4b6f52f280236f7b8b804425a01846f
 SHA512:
-  metadata.gz: 6aab6fde374e4f00c40509e32950fc8d3ab3b18666a8c5be593f1fc2d58a1907ab19ea1ba2a309d1728a6c3611aaeae54f8873a2e25b7450a55dbe9952358811
-  data.tar.gz: a22576c8eb6253c18c3705211c09764df2c1221c65797fcb944c8f3a62f31c7b7f5f6aec1abe607393c98ebd9bc90792662e609673a1f4bdea86f3b33030e132
+  metadata.gz: 932361a2de24b64de68929b61379e1d8f01acce64f3f03070070ccfb9da8a369329c7703f84f774e236c69a2343dde4c5a6add5fd21add2db9025cdde3515a7b
+  data.tar.gz: fa77a297e489a26894c6f49961a5f057bfb239385a93d96b0252892db411922e9e2c75d0abb9de663fe059d883d9e07f06a043ae27b7704a4e868c97a49d4eb8

data/CHANGELOG.md

@@ -4,6 +4,27 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## 1.0.2
+
+### Fixed
+- Requiring the gem now loads the ActiveSupport framework itself instead of just `active_support/notifications`, which is not self-sufficient on ActiveSupport 7+ and raised `NameError: uninitialized constant ActiveSupport::IsolatedExecutionState` on the first request if the host application had not already loaded ActiveSupport.
+- The httpclient and excon instrumentation now forward blocks passed to the instrumented methods, so streaming response blocks are no longer silently ignored when the aliasing strategy is used.
+- The excon instrumentation now encodes query parameters passed as a hash when building the URL for the event payload, so `access_token` parameters are properly stripped from the `:uri` payload value instead of leaking through in unparseable form.
+- The aliased flag on instrumentation modules is now set before the instrumented method is swapped in, so a request on another thread can no longer hit a window during installation where it would raise `NoMethodError`.
+- Removed the nonexistent `:net_http2` entry from `HTTPInstrumentation::IMPLEMENTATIONS`; no hook for it was ever implemented.
+- `HTTPInstrumentation.initialize!` now accepts a single symbol for the `only` option instead of raising `NoMethodError`.
+- The `except` option to `HTTPInstrumentation.initialize!` now excludes the listed libraries instead of instrumenting only those libraries.
+- Requests made with relative URLs (e.g. a patron session with a `base_url`) no longer raise `URI::InvalidURIError` from the instrumentation after the request completes.
+- Installing the instrumentation is now protected by a mutex so concurrent calls to `initialize!` cannot install the aliased methods twice, which would have caused infinite recursion on subsequent requests.
+- The excon instrumentation now reports per-request values (path, query, etc.) instead of letting the connection's defaults override them in the event payload.
+- The event payload now always includes the `:count` key even when the request raises an error.
+- The net/http instrumentation no longer lets an error raised while building the event payload escape to the caller, matching the behavior of the other hooks.
+- The ethon instrumentation clears the request info captured from `http_request` after each `perform`, so reusing an easy handle without going through `http_request` no longer reports the previous request's method and URL. The URL set directly on the handle is now also reported as a fallback.
+- The `:url` payload value no longer includes an explicit port for URLs using the default port for their scheme.
+
+### Changed
+- The minimum required Ruby version has been updated to 2.6.
+
 ## 1.0.1
 
 ### Fixed

data/README.md

@@ -2,6 +2,7 @@
 
 [![Continuous Integration](https://github.com/bdurand/http_instrumentation/actions/workflows/continuous_integration.yml/badge.svg)](https://github.com/bdurand/http_instrumentation/actions/workflows/continuous_integration.yml)
 [![Ruby Style Guide](https://img.shields.io/badge/code_style-standard-brightgreen.svg)](https://github.com/testdouble/standard)
+[![Gem Version](https://badge.fury.io/rb/http_instrumentation.svg)](https://badge.fury.io/rb/http_instrumentation)
 
 This gem adds instrumentation to a variety of the most commonly used Ruby HTTP client libraries via ActiveSupport notifications. The goal is to add a common instrumentation interface across all the HTTP client libraries used by an application (including ones installed as dependencies of other gems).
 
@@ -19,6 +20,8 @@ This gem adds instrumentation to a variety of the most commonly used Ruby HTTP c
 
 Note that several other popular HTTP client libraries like [Faraday](https://github.com/lostisland/faraday), [HTTParty](https://github.com/jnunemaker/httparty), and [RestClient](https://github.com/rest-client/rest-client) are built on top of these low level libraries.
 
+A few code paths are not instrumented: curb requests that don't go through `Curl::Easy#http` (e.g. the native `http_get` and `http_post` helpers) and httpclient streaming responses.
+
 ## Usage
 
 To capture information about HTTP requests, simply subscribe to the `request.http` events with [ActiveSupport notifications](https://api.rubyonrails.org/classes/ActiveSupport/Notifications.html) (note that you should really use `monotonic_subscribe` instead of `subscribe` to avoid issues with clock adjustments).
@@ -63,6 +66,17 @@ HTTPX.get("https://example.com/r1", "https://example.com/r2")
 # => HTTP request: client: httpx, count: 2, duration: 150ms
 ```
 
+### Instrumenting Libraries Loaded Later
+
+All supported libraries that have already been loaded are automatically instrumented when this gem is required. Instrumentation cannot be removed once it has been installed.
+
+If an HTTP client library is loaded after this gem, you can call `HTTPInstrumentation.initialize!` again to instrument it. The `only` and `except` options can be used to limit which libraries are instrumented.
+
+```ruby
+require "httpx"
+HTTPInstrumentation.initialize!(only: [:httpx])
+```
+
 ### Security
 
 The `:uri` element in the event payload will be sanitized to remove any user/password elements encoded in the URL as well as any `access_token` query parameters.
@@ -115,7 +129,7 @@ You can also take advantage of the existing instrumentation and just override th
 ```ruby
 class MyHttpClient
   def get(url)
-    HTTPInstrumentation.client("my_client")
+    HTTPInstrumentation.client("my_client") do
       Net::HTTP.get(URI(url))
     end
   end

data/VERSION

@@ -1 +1 @@
-1.0.1
+1.0.2

data/http_instrumentation.gemspec

@@ -29,7 +29,7 @@ Gem::Specification.new do |spec|
 
   spec.require_paths = ["lib"]
 
-  spec.required_ruby_version = ">= 2.5"
+  spec.required_ruby_version = ">= 2.6"
 
   spec.add_dependency "activesupport", ">= 4.2"
 

data/lib/http_instrumentation/instrumentation/ethon_hook.rb

@@ -61,14 +61,23 @@ module HTTPInstrumentation
 
         def perform(*args)
           HTTPInstrumentation.instrument("ethon") do |payload|
+            payload[:http_method] = @http_instrumentation_method
+            begin
+              payload[:url] = (@http_instrumentation_url || url)
+            rescue
+              payload[:url] = @http_instrumentation_url
+            end
+            # Clear the request info so it isn't reported again if the handle
+            # is reused without going through http_request.
+            @http_instrumentation_method = nil
+            @http_instrumentation_url = nil
+
             retval = if HTTPInstrumentation::Instrumentation::EthonHook::Easy.aliased
               perform_without_http_instrumentation(*args)
             else
               super
             end
 
-            payload[:http_method] = @http_instrumentation_method
-            payload[:url] = @http_instrumentation_url
             begin
               payload[:status_code] = response_code
             rescue

data/lib/http_instrumentation/instrumentation/excon_hook.rb

@@ -21,27 +21,31 @@ module HTTPInstrumentation
         attr_accessor :aliased
       end
 
-      def request(params = {})
+      def request(params = {}, &block)
         HTTPInstrumentation.instrument("excon") do |payload|
           response = if HTTPInstrumentation::Instrumentation::ExconHook.aliased
-            request_without_http_instrumentation(params)
+            request_without_http_instrumentation(params, &block)
           else
             super
           end
 
           begin
             info = params
+            # Merge connection defaults under the per-request params so the
+            # request values win.
             if respond_to?(:connection)
-              info = info.merge(connection)
+              info = connection.merge(params)
             elsif respond_to?(:data)
-              info = info.merge(data)
+              info = data.merge(params)
             end
 
             scheme = info[:scheme]&.downcase
             default_port = ((scheme == "https") ? 443 : 80)
             port = info[:port]
+            query = info[:query]
+            query = URI.encode_www_form(query) if query.is_a?(Hash)
             payload[:http_method] = (info[:http_method] || info[:method])
-            payload[:url] = "#{scheme}://#{info[:host]}#{":#{port}" unless port == default_port}#{info[:path]}#{"?#{info[:query]}" unless info[:query].to_s.empty?}"
+            payload[:url] = "#{scheme}://#{info[:host]}#{":#{port}" unless port == default_port}#{info[:path]}#{"?#{query}" unless query.to_s.empty?}"
             payload[:status_code] = response.status
           rescue
           end

data/lib/http_instrumentation/instrumentation/httpclient_hook.rb

@@ -21,10 +21,10 @@ module HTTPInstrumentation
         attr_accessor :aliased
       end
 
-      def do_get_block(request, *args)
+      def do_get_block(request, *args, &block)
         HTTPInstrumentation.instrument("httpclient") do |payload|
           response = if HTTPInstrumentation::Instrumentation::HTTPClientHook.aliased
-            do_get_block_without_http_instrumentation(request, *args)
+            do_get_block_without_http_instrumentation(request, *args, &block)
           else
             super
           end

data/lib/http_instrumentation/instrumentation/net_http_hook.rb

@@ -37,12 +37,17 @@ module HTTPInstrumentation
             super
           end
 
-          default_port = (use_ssl? ? 443 : 80)
-          scheme = (use_ssl? ? "https" : "http")
-          url = "#{scheme}://#{address}#{":#{port}" unless port == default_port}#{req.path}"
-          payload[:http_method] = req.method
-          payload[:url] = url
-          payload[:status_code] = response.code
+          begin
+            default_port = (use_ssl? ? 443 : 80)
+            scheme = (use_ssl? ? "https" : "http")
+            url = "#{scheme}://#{address}#{":#{port}" unless port == default_port}#{req.path}"
+            payload[:http_method] = req.method
+            payload[:url] = url
+            payload[:status_code] = response.code
+          rescue
+            # Instrumentation must never break the request; if the payload can't
+            # be built, return the response without the extra fields.
+          end
 
           response
         end

data/lib/http_instrumentation/instrumentation.rb

@@ -12,6 +12,12 @@ require_relative "instrumentation/typhoeus_hook"
 
 module HTTPInstrumentation
   module Instrumentation
+    # Lock to prevent concurrent calls from installing the same hooks twice;
+    # double aliasing would point the _without_http_instrumentation method at
+    # the instrumented method itself, causing infinite recursion.
+    INSTRUMENT_LOCK = Mutex.new
+    private_constant :INSTRUMENT_LOCK
+
     class << self
       # Helper method to add an instrumentation module to methods on a class. The
       # methods must be defined in the instrumentation module.
@@ -23,43 +29,64 @@ module HTTPInstrumentation
       # may have already prepended the methods. Aliasing is the default strategy because
       # prepending after aliasing will work, but aliasing after prepending will not.
       def instrument!(klass, instrumentation_module, methods)
-        return if klass.include?(instrumentation_module)
+        INSTRUMENT_LOCK.synchronize do
+          return if klass.include?(instrumentation_module)
 
-        methods = Array(methods).collect(&:to_sym)
+          methods = Array(methods).collect(&:to_sym)
 
-        if HTTPInstrumentation.force_prepend? || methods_prepended?(klass, methods)
-          klass.prepend(instrumentation_module)
-          instrumentation_module.aliased = false
-        else
-          Array(methods).each do |method|
-            instrumentation_module.alias_method("#{method}_with_http_instrumentation", method)
-          end
+          if HTTPInstrumentation.force_prepend? || methods_defined_in_module?(klass, methods)
+            klass.prepend(instrumentation_module)
+            instrumentation_module.aliased = false
+          else
+            methods.each do |method|
+              instrumentation_module.alias_method("#{method}_with_http_instrumentation", method)
+            end
 
-          klass.include(instrumentation_module)
+            klass.include(instrumentation_module)
 
-          Array(methods).each do |method|
-            klass.alias_method("#{method}_without_http_instrumentation", method)
-            klass.alias_method(method, "#{method}_with_http_instrumentation")
-          end
+            swap_methods = methods.reject { |method| defines_method?(klass, "#{method}_without_http_instrumentation") }
+
+            swap_methods.each do |method|
+              klass.alias_method("#{method}_without_http_instrumentation", method)
+            end
 
-          instrumentation_module.aliased = true
+            # The aliased flag must be set before the instrumented methods are
+            # swapped in; a call landing in between would take the super branch
+            # and raise NoMethodError since the module was included, not prepended.
+            instrumentation_module.aliased = true
+
+            swap_methods.each do |method|
+              klass.alias_method(method, "#{method}_with_http_instrumentation")
+            end
+          end
         end
       end
 
       private
 
-      def methods_prepended?(klass, methods)
-        prepended = false
+      # Returns true if any of the methods are defined in a module in the class's
+      # ancestry rather than directly on the class itself. This covers both modules
+      # prepended in front of the class and modules included behind it. In either
+      # case the aliasing strategy cannot be used: including the instrumentation
+      # module would insert it ahead of the module that defines the method, so
+      # aliasing the method afterward would capture the instrumented method itself
+      # and cause infinite recursion.
+      def methods_defined_in_module?(klass, methods)
+        defined_in_module = false
 
         klass.ancestors.each do |mod|
           next unless mod.is_a?(Module) && !mod.is_a?(Class)
 
           module_methods = mod.instance_methods(false) + mod.private_instance_methods(false)
-          prepended = (module_methods & methods).any?
-          break if prepended
+          defined_in_module = (module_methods & methods).any?
+          break if defined_in_module
         end
 
-        prepended
+        defined_in_module
+      end
+
+      def defines_method?(klass, method)
+        klass.method_defined?(method) || klass.private_method_defined?(method)
       end
     end
   end

data/lib/http_instrumentation.rb

@@ -1,5 +1,6 @@
 # frozen_string_literal: true
 
+require "active_support"
 require "active_support/notifications"
 
 require_relative "http_instrumentation/instrumentation"
@@ -13,7 +14,6 @@ module HTTPInstrumentation
     :httpclient,
     :httpx,
     :net_http,
-    :net_http2,
     :patron,
     :typhoeus
   ].freeze
@@ -29,8 +29,8 @@ module HTTPInstrumentation
     # @param except [Array<Symbol>] List of libraries to not instrument.
     # @return [void]
     def initialize!(only: nil, except: nil)
-      list = (only || IMPLEMENTATIONS)
-      list &= Array(except) if except
+      list = Array(only || IMPLEMENTATIONS)
+      list -= Array(except) if except
 
       Instrumentation::CurbHook.instrument! if list.include?(:curb)
       Instrumentation::EthonHook.instrument! if list.include?(:ethon)
@@ -45,6 +45,9 @@ module HTTPInstrumentation
 
     # Silence instrumentation for the duration of the block.
     #
+    # Note that silencing is stored in fiber-local storage, so it does not
+    # propagate into other threads or fibers spawned within the block.
+    #
     # @return [Object] the return value of the block
     def silence(&block)
       save_val = Thread.current[:http_instrumentation_silence]
@@ -101,25 +104,7 @@ module HTTPInstrumentation
       return yield(payload) if silenced?
 
       ActiveSupport::Notifications.instrument(EVENT, payload) do
-        retval = silence { yield(payload) }
-
-        payload[:http_method] = normalize_http_method(payload[:http_method]) if payload.include?(:http_method)
-
-        if payload.include?(:url)
-          uri = sanitized_uri(payload[:url])
-          if uri
-            payload[:url] = uri_without_query_string(uri)
-            payload[:uri] = uri
-          else
-            payload[:url] = payload[:url]&.to_s
-          end
-        end
-
-        payload[:status_code] = payload[:status_code]&.to_i if payload.include?(:status_code)
-
-        payload[:count] = (payload.include?(:count) ? payload[:count].to_i : 1)
-
-        retval
+        instrument_request(payload, &block)
       end
     end
 
@@ -133,6 +118,29 @@ module HTTPInstrumentation
 
     private
 
+    # Yield the block with instrumentation silenced and then normalize the
+    # payload. The payload is normalized even if the block raises an error so
+    # that events for failed requests still include the normalized keys.
+    def instrument_request(payload)
+      silence { yield(payload) }
+    ensure
+      payload[:http_method] = normalize_http_method(payload[:http_method]) if payload.include?(:http_method)
+
+      if payload.include?(:url)
+        uri = sanitized_uri(payload[:url])
+        if uri
+          payload[:url] = uri_without_query_string(uri)
+          payload[:uri] = uri
+        else
+          payload[:url] = payload[:url]&.to_s
+        end
+      end
+
+      payload[:status_code] = payload[:status_code]&.to_i if payload.include?(:status_code)
+
+      payload[:count] = (payload.include?(:count) ? payload[:count].to_i : 1)
+    end
+
     # Turn the given value into a lowercase symbol.
     def normalize_http_method(method)
       return nil if method.nil?
@@ -171,7 +179,10 @@ module HTTPInstrumentation
     end
 
     def uri_without_query_string(uri)
-      URI("#{uri.scheme}://#{uri.host}:#{uri.port}#{uri.path}").to_s
+      stripped = uri.dup
+      stripped.query = nil if stripped.respond_to?(:query=)
+      stripped.fragment = nil if stripped.respond_to?(:fragment=)
+      stripped.to_s
     end
   end
 end

metadata

@@ -1,14 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: http_instrumentation
 version: !ruby/object:Gem::Version
-  version: 1.0.1
+  version: 1.0.2
 platform: ruby
 authors:
 - Brian Durand
-autorequire:
 bindir: bin
 cert_chain: []
-date: 2023-08-08 00:00:00.000000000 Z
+date: 1980-01-02 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activesupport
@@ -38,7 +37,6 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-description:
 email:
 - bbdurand@gmail.com
 executables: []
@@ -65,7 +63,6 @@ homepage: https://github.com/bdurand/http_instrumentation
 licenses:
 - MIT
 metadata: {}
-post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -73,15 +70,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: '2.5'
+      version: '2.6'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.2.22
-signing_key:
+rubygems_version: 4.0.3
 specification_version: 4
 summary: ActiveSupoprt instrumentation for a variety of Ruby HTTP client libraries.
 test_files: []