patient_http 1.1.2 → 1.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ba06f7676b7d33f9c53adea879761a89d31593f51c3f33b999400d7305fcbf06
-  data.tar.gz: ede7b685bbfbdd5049695316f73da55bf0abe203b0a9e1e28eb0bc1da6739f79
+  metadata.gz: 393cbf361701d9e1dc190b80f571d78059a21dae85f225750198a1ab173e8bb8
+  data.tar.gz: 3d5349e55335d4cd262699481517d3c44825c27449a2b3b11d9559a7b4f468b0
 SHA512:
-  metadata.gz: 9271aecb38a8adac4634d6e02f7797472347e1d4d20a154cf52c39daa450789a1ea1511f464d0551172361f4815ab26eeac535d2d72fb483e169be58bd5900df
-  data.tar.gz: 379ff51d8e0a3e90f49577c9922c6f58a2caa6ed6e4d36a651f9f0071ea5d4999cf25df926cdaf1165e27a495db1d1e65d208d8c58cb41e3513f5c8210829332
+  metadata.gz: '06894a234fca9e2c12aa77b69cf5347824de6f6be9e3b912dcc3e7834e2137231144e69ba58bef85fb026b200369ad976befc4e540ba9f2779a31fe3e68991a0'
+  data.tar.gz: ac60b71f532664b3083697dd1c225f4a5a787f07de04d2458d3f849c3b3962d68062740b7e0f03b981edf7aa9ab6fac98aadff98ef9549a0efe15c0b3b968de1

data/CHANGELOG.md

@@ -4,6 +4,28 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## 1.2.0
+
+### Added
+
+- Request preprocessors for modifying the outgoing request just before it is sent — most usefully, to sign requests (e.g. AWS SigV4 signatures that must set multiple headers computed over the final request). Register a preprocessor on the `Configuration` with `register_preprocessor(name)` and attach it to a request with `preprocessors: name`. The request serializes only the preprocessor name; the callable (and any credentials it uses) stays on the processor side. Preprocessors receive an `OutgoingRequest` — a view of the request after secret resolution with read access to the method, URL, and body, mutable headers, and an `add_param` method for appending query parameters. On redirects, preprocessors re-run against each redirect URL and are dropped on cross-origin hops, consistent with sensitive header stripping.
+
+### Fixed
+
+- `SynchronousExecutor` no longer resolves secret query params twice per request, which previously invoked callable secrets twice.
+- Graceful shutdown now works as documented: requests that complete while the processor is stopping have their responses delivered instead of being discarded and retried, and `Processor#stop` returns as soon as in-flight requests finish rather than always blocking for the full `shutdown_timeout`.
+- `Processor#stop` performs a second re-enqueue pass after the reactor thread exits, closing a race where a task dequeued but not yet tracked at shutdown could be silently lost.
+- Task results are now claimed atomically at shutdown so a completing request can no longer trigger both its completion callback and a `TaskHandler#retry` for the same task.
+- `Processor#start` on a draining processor is now a no-op instead of spawning a second reactor thread that shared the queue and connection pools with the draining one.
+- The processor can no longer end up in a running state with a dead reactor thread when the reactor fails during startup.
+- Processor observers are now invoked outside of internal locks, so an observer callback can safely call back into processor methods such as `total_count` without raising a recursive locking error.
+- `SynchronousExecutor` no longer invokes the error callback twice when a redirect fails with too many redirects or a redirect loop.
+- `HttpHeaders#merge` no longer mutates the receiver. This also fixes `RequestTemplate` permanently absorbing per-request headers into its defaults, which could leak headers (including credentials) across requests built from the same template.
+- `Request` now copies headers passed to its constructor instead of holding (and potentially mutating) the caller's `HttpHeaders` instance.
+- `Errno::ETIMEDOUT` is now treated as a connection error, evicting the pooled client and classifying the `RequestError` as `:connection`.
+- `RedirectError.load` validates that the serialized error class is a `RedirectError` subclass instead of instantiating an arbitrary class name from the payload.
+- Fixed the Redis payload store documentation to use the `redis` gem interface (`Redis.new`); the previously documented `RedisClient` does not respond to the methods the store calls.
+
 ## 1.1.2
 
 ### Added

data/README.md

@@ -281,6 +281,8 @@ error = PatientHttp::HttpError.load(json_data)
 
 The `Response` object includes the HTTP status code, headers, body, and callback arguments. Error objects (`HttpError`, `RedirectError`, `RequestError`) include the error message, context about the request, and callback arguments.
 
+Response headers are case insensitive. Headers that appear multiple times in the response (such as `set-cookie`) are flattened into a single joined string value.
+
 Response bodies are automatically encoded for JSON serialization. Binary content is Base64 encoded, and large text content is gzipped and then Base64 encoded to reduce payload size. Decoding is handled transparently when you access the `body` or `json` methods on the `Response` object.
 
 ### Payload Stores
@@ -321,10 +323,10 @@ config.register_payload_store(:files, adapter: :file, directory: "/tmp/payloads"
 
 #### Redis Store
 
-For production with shared state across processes:
+For production with shared state across processes (requires the `redis` gem; the client must respond to `set`, `get`, `del`, and `exists`):
 
 ```ruby
-redis = RedisClient.new(url: ENV["REDIS_URL"])
+redis = Redis.new(url: ENV["REDIS_URL"])
 config.register_payload_store(:redis, adapter: :redis, redis: redis, ttl: 86400)
 ```
 
@@ -501,7 +503,7 @@ config.register_secret(:authorization, "Bearer #{ENV['API_TOKEN']}") # static va
 config.register_secret(:api_key) { ENV["MY_API_KEY"] } # lazy block
 ```
 
-I a secret is not found when resolving a request, a `PatientHttp::SecretManager::SecretNotFoundError` is raised, which surfaces through the normal request error path.
+If a secret is not found when resolving a request, a `PatientHttp::SecretManager::SecretNotFoundError` is raised, which surfaces through the normal request error path.
 
 ### Referencing secrets when building a request
 
@@ -518,6 +520,59 @@ PatientHttp.get(
 
 The request serializes the secret header as `{"$secret" => "api_token"}` and keeps the secret query parameter out of the URL (non-secret params like `page` are still folded into the URL as usual). The processor dereferences both just before sending: the header is set to its resolved value and the resolved query parameter is appended to the URL.
 
+## Request Preprocessors
+
+Preprocessors let you modify a request just before it is sent — most usefully, to sign it. Signing schemes like AWS SigV4 need to compute values over the final outgoing request (method, URL, headers, body) and set multiple headers, which cannot be expressed as a static header value at build time.
+
+Like secrets, preprocessors are registered on the `Configuration` and referenced from requests by name only. The serialized request carries just the name, so the signing logic and its credentials live on the processor side and are never written to the job queue.
+
+### Defining preprocessors
+
+Register a named preprocessor as a block or callable taking a single argument:
+
+```ruby
+config = PatientHttp::Configuration.new
+config.register_preprocessor(:aws_sigv4) do |request|
+  signer = Aws::Sigv4::Signer.new(
+    service: "execute-api",
+    region: "us-east-1",
+    credentials_provider: Aws::CredentialProviderChain.new.resolve
+  )
+  signature = signer.sign_request(
+    http_method: request.http_method.to_s.upcase,
+    url: request.url,
+    headers: request.headers.to_h,
+    body: request.body.to_s
+  )
+  signature.headers.each { |name, value| request.headers[name] = value }
+end
+```
+
+The argument is a `PatientHttp::OutgoingRequest` — a view of the request as it is about to be sent, after all secret references have been resolved and the `x-request-id` and default `User-Agent` headers have been set. It exposes:
+
+- `http_method`, `url`, and `body` (read-only; the URL includes any resolved secret query params)
+- `headers` — mutable, case-insensitive headers
+- `add_param(name, value)` — appends a query parameter to the URL, for signed-query-param schemes
+
+### Attaching preprocessors to a request
+
+Reference registered preprocessors by name when building a request:
+
+```ruby
+PatientHttp.post(
+  "https://api.example.com/data",
+  callback: MyCallback,
+  json: {value: 1},
+  preprocessors: :aws_sigv4
+)
+```
+
+Multiple preprocessors can be given as an array; they run in order, each seeing the changes made by the ones before it. `RequestTemplate` also accepts `preprocessors:` as a template-wide default.
+
+If a request references a preprocessor name that is not registered, a `PatientHttp::RequestPreparer::PreprocessorNotFoundError` is raised, which surfaces through the normal request error path.
+
+When redirects are followed, preprocessors are re-run against each redirect URL so signatures stay valid. On cross-origin redirects they are dropped entirely, consistent with the stripping of `Authorization` and `Cookie` headers, so signed credentials are never sent to an unexpected origin.
+
 ## Configuration
 
 ```ruby

data/VERSION

@@ -1 +1 @@
-1.1.2
+1.2.0

data/lib/patient_http/client.rb

@@ -12,6 +12,7 @@ module PatientHttp
         protocol: config.protocol
       )
       @response_reader = ResponseReader.new(@processor)
+      @request_preparer = RequestPreparer.new(config)
     end
 
     # Make an asynchronous HTTP request.
@@ -23,13 +24,16 @@ module PatientHttp
       async_response = nil
 
       begin
-        headers = request_headers(request, request_id)
-        url = config.secret_manager.resolve_url(request.url, request.secret_params)
+        outgoing = @request_preparer.prepare(request, request_id)
+        url = outgoing.url
+        headers = outgoing.headers.to_h
         body = Protocol::HTTP::Body::Buffered.wrap([request.body.to_s]) if request.body
         timeout = request.timeout || config.request_timeout
 
         Async::Task.current.with_timeout(timeout) do
           async_response = @client_pool.request(request.http_method, url, headers, body)
+          # Note: headers that appear multiple times (e.g. set-cookie) are
+          # flattened to a single joined string value.
           headers_hash = async_response.headers.to_h.transform_values(&:to_s)
           body = @response_reader.read_body(async_response, headers_hash)
 
@@ -66,18 +70,11 @@ module PatientHttp
     def connection_error?(exception)
       case exception
       when Async::TimeoutError, Errno::ECONNRESET, Errno::ECONNABORTED, Errno::EPIPE,
-           Errno::ECONNREFUSED, Errno::EHOSTUNREACH, SocketError, IOError
+           Errno::ECONNREFUSED, Errno::EHOSTUNREACH, Errno::ETIMEDOUT, SocketError, IOError
         true
       else
         false
       end
     end
-
-    def request_headers(request, request_id)
-      headers = config.secret_manager.resolve_headers(request.headers.to_h)
-      headers["x-request-id"] = request_id
-      headers["user-agent"] ||= config.user_agent if config.user_agent
-      headers
-    end
   end
 end

data/lib/patient_http/configuration.rb

@@ -94,6 +94,9 @@ module PatientHttp
       @secrets = {}
       @secret_manager = SecretManager.new
 
+      # Initialize preprocessor registry
+      @preprocessors = {}
+
       @encryptor = nil
 
       self.max_connections = max_connections
@@ -271,6 +274,43 @@ module PatientHttp
       end
     end
 
+    # Register a named preprocessor that can be attached to requests to modify them
+    # just before they are sent -- for example, to sign requests.
+    #
+    # The preprocessor can be provided as a callable or a block taking a single
+    # argument. When a request that references the preprocessor is sent, it is
+    # invoked with an {OutgoingRequest} after secret references have been resolved
+    # and the x-request-id and default user-agent headers have been set. It can
+    # change the request headers and append query parameters.
+    #
+    # Requests reference preprocessors by name only, so the callable (and any
+    # credentials it uses) stays on the processor side and is never serialized.
+    #
+    # @param name [String, Symbol] the preprocessor name
+    # @param callable [#call, nil] object invoked with the outgoing request (omit when providing a block)
+    # @yield [outgoing_request] a block invoked with the outgoing request (omit when providing a callable)
+    # @raise [ArgumentError] if neither or both of callable and block are provided, or
+    #   if the preprocessor cannot be called with a single argument
+    # @return [void]
+    def register_preprocessor(name, callable = nil, &block)
+      preprocessor = resolve_callable(:preprocessor, callable, &block)
+      raise ArgumentError.new("register_preprocessor requires a callable or a block") if preprocessor.nil?
+
+      validate_preprocessor_parameters!(preprocessor)
+
+      @mutex.synchronize do
+        @preprocessors = @preprocessors.merge(name.to_s => preprocessor)
+      end
+    end
+
+    # Get a registered preprocessor by name.
+    #
+    # @param name [String, Symbol] the preprocessor name
+    # @return [#call, nil] the preprocessor or nil if not registered
+    def preprocessor(name)
+      @preprocessors[name.to_s]
+    end
+
     # Register a payload store for external storage of large payloads.
     #
     # The name is included in the serialized references to the stored data.
@@ -350,7 +390,8 @@ module PatientHttp
         "protocol" => protocol,
         "payload_stores" => payload_stores.keys,
         "default_payload_store" => default_payload_store_name,
-        "secrets" => @mutex.synchronize { @secrets.keys }
+        "secrets" => @mutex.synchronize { @secrets.keys },
+        "preprocessors" => @mutex.synchronize { @preprocessors.keys }
       }
     end
 
@@ -368,6 +409,20 @@ module PatientHttp
       callable || block
     end
 
+    # Validate that a preprocessor can be invoked with a single positional argument.
+    def validate_preprocessor_parameters!(preprocessor)
+      method_obj = preprocessor.is_a?(Proc) ? preprocessor : preprocessor.method(:call)
+      parameters = method_obj.parameters
+
+      positional = parameters.count { |type, _| %i[req opt rest].include?(type) }
+      required = parameters.count { |type, _| type == :req }
+      required_keywords = parameters.count { |type, _| type == :keyreq }
+
+      if positional.zero? || required > 1 || required_keywords.positive?
+        raise ArgumentError.new("preprocessor must accept a single argument")
+      end
+    end
+
     def validate_positive(attribute, value)
       return if value.is_a?(Numeric) && value > 0
 

data/lib/patient_http/http_headers.rb

@@ -88,6 +88,13 @@ module PatientHttp
       @headers.include?(name.to_s.downcase)
     end
 
+    # Ensure copies do not share the underlying storage so that mutating a
+    # copy (for example, via #merge) does not modify the original.
+    def initialize_copy(other)
+      super
+      @headers = @headers.dup
+    end
+
     def eql?(other)
       other.is_a?(HttpHeaders) && @headers.eql?(other.to_h)
     end

data/lib/patient_http/lifecycle_manager.rb

@@ -12,7 +12,7 @@ module PatientHttp
     STATES = %i[stopped starting running draining stopping].freeze
 
     # Polling interval during wait operations
-    POLL_INTERVAL = 0.001
+    POLL_INTERVAL = 0.01
 
     # Initialize the lifecycle manager.
     #
@@ -66,12 +66,14 @@ module PatientHttp
       state == :stopping
     end
 
-    # Transition to starting state.
+    # Transition to starting state. The processor can only be started from
+    # the stopped state; in particular, starting a draining processor would
+    # spawn a second reactor alongside the one still finishing its drain.
     #
     # @return [Boolean] true if transition was successful
     def start!
       @lock.synchronize do
-        return false if starting? || running? || stopping?
+        return false unless stopped?
 
         @state.set(:starting)
         @shutdown_barrier.reset
@@ -83,9 +85,18 @@ module PatientHttp
 
     # Transition to running state.
     #
-    # @return [void]
+    # The transition only occurs from the starting state so that a reactor
+    # that already failed and transitioned to stopped is not overwritten.
+    #
+    # @return [Boolean] true if transition was successful
     def running!
-      @state.set(:running)
+      @lock.synchronize do
+        return false unless starting?
+
+        @state.set(:running)
+      end
+
+      true
     end
 
     # Transition to draining state.
@@ -136,9 +147,10 @@ module PatientHttp
 
     # Wait for the reactor to be ready.
     #
-    # @return [void]
-    def wait_for_reactor
-      @reactor_ready.wait
+    # @param timeout [Numeric, nil] maximum time to wait in seconds (nil waits forever)
+    # @return [Boolean] true if the reactor is ready, false if the timeout was reached
+    def wait_for_reactor(timeout: nil)
+      @reactor_ready.wait(timeout)
     end
 
     # Check if shutdown has been signaled.

data/lib/patient_http/outgoing_request.rb

@@ -0,0 +1,72 @@
+# frozen_string_literal: true
+
+module PatientHttp
+  # A mutable view of a request as it is about to be sent, after secret references
+  # have been resolved and the send-time headers (x-request-id and the default
+  # user-agent) have been set.
+  #
+  # Preprocessors attached to a request receive this object and can modify the
+  # headers or append query parameters before the request goes out -- for example,
+  # to sign the request. The HTTP method, URL, and body are read-only; headers can
+  # be changed in place and query parameters appended with {#add_param}.
+  #
+  # @see Configuration#register_preprocessor
+  class OutgoingRequest
+    # @return [Symbol] HTTP method (:get, :post, :put, :patch, :delete)
+    attr_reader :http_method
+
+    # @return [String] the request URL with any secret query params already resolved
+    attr_reader :url
+
+    # @return [String, nil] the request body
+    attr_reader :body
+
+    # @return [HttpHeaders] mutable, case-insensitive request headers
+    attr_reader :headers
+
+    # Initialize a new OutgoingRequest.
+    #
+    # @param http_method [Symbol] the HTTP method
+    # @param url [String] the resolved request URL
+    # @param headers [HttpHeaders] the resolved request headers
+    # @param body [String, nil] the request body
+    def initialize(http_method:, url:, headers:, body:)
+      @http_method = http_method
+      @url = url.to_s
+      @headers = headers
+      @body = body
+    end
+
+    # Append a query parameter to the request URL.
+    #
+    # @param name [String, Symbol] the parameter name
+    # @param value [Object] the parameter value
+    # @return [String] the updated URL
+    def add_param(name, value)
+      serialized_param = URI.encode_www_form([[name.to_s, value]])
+      uri = URI(@url)
+      uri.query = [uri.query, serialized_param].compact.reject(&:empty?).join("&")
+      @url = uri.to_s
+    end
+
+    # Inspect the outgoing request. Header values, the query string, and the body
+    # are not shown since they may contain resolved secrets.
+    #
+    # @return [String]
+    def inspect
+      "#<#{self.class.name} #{http_method.to_s.upcase} #{redacted_url} headers=#{headers.to_h.keys.inspect}>"
+    end
+
+    private
+
+    def redacted_url
+      uri = URI(@url)
+      uri.query = nil
+      uri.user = nil
+      uri.password = nil
+      uri.to_s
+    rescue URI::InvalidURIError
+      "<invalid url>"
+    end
+  end
+end

data/lib/patient_http/payload.rb

@@ -89,7 +89,7 @@ module PatientHttp
         return Encoding::ASCII_8BIT.name unless match
 
         begin
-          Encoding.find(match[1])
+          Encoding.find(match[1]).name
         rescue
           Encoding::ASCII_8BIT.name
         end

data/lib/patient_http/payload_store/redis_store.rb

@@ -10,8 +10,11 @@ module PatientHttp
     #
     # Thread-safe: Redis clients handle their own thread safety.
     #
+    # The client must respond to `set`, `get`, `del`, and `exists` (the
+    # interface provided by the `redis` gem).
+    #
     # @example Configuration with direct Redis client
-    #   redis = RedisClient.new(url: ENV["REDIS_URL"])
+    #   redis = Redis.new(url: ENV["REDIS_URL"])
     #   config.register_payload_store(:redis, adapter: :redis, redis: redis, ttl: 86400)
     class RedisStore < Base
       Base.register :redis, self