patient_http 1.4.0 → 1.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d0888d7e92f43f2031bf7f0eb5b0f4dd528ac7b79957b506f1df61825309990a
-  data.tar.gz: 87b5817b8205a0b605edb7cb41facf0f6b5690122ff63cb669dbcc6c6b090cd0
+  metadata.gz: 54ede397ae79574e7dd597ea642b4d039877d5d642ba49c68b76219b8b1c4a32
+  data.tar.gz: 83bb067b78912753791e6002abb3470ca70936f5fe862fc65ee65d7786e1ac53
 SHA512:
-  metadata.gz: 4bd3586a382436ac11872d10380afb28326497c9538383ae0e884ecaf901dd952df0202e9ed740047b6336c60960ebff20b83abcdc86a585c67e0915097dbe92
-  data.tar.gz: d86c67c48c1aa03d0aa47f4675626696ab9012b526add67a4d8ebf611aca40b8229e722bd2e07c2ef6de6caa6110a01507d3317f029668fa89a768d145fea299
+  metadata.gz: 0302b8cdf7042fa545e49ab1ce565d5f7d1fd388a8413bc8a19249477d46d2fc9f732142d75afbb6cc52d0dd7a228e01c6f0c1cbaea2f7b151568472ccfaecb4
+  data.tar.gz: f418c2d7b5b48ba81ac73de35533143d7e99c75fb108a619f9cf9a55ca204f56f42ffec86ee4fc6d2796730703aad488014955603f87fc94ccf3e66d6b7b8593

data/ARCHITECTURE.md

@@ -54,7 +54,16 @@ The `TaskHandler` abstract class defines how the processor communicates results
 - **on_error(error, callback)**: Called when an HTTP request fails. Your implementation should enqueue the error for handling.
 - **retry**: Called when the processor shuts down with in-flight requests. Your implementation should re-enqueue the original job.
 
-> **Important:** TaskHandler callbacks run on the processor's reactor thread. They should be lightweight and fast -- typically just enqueuing a message for another system to pick up. Doing heavy processing in a callback will block the reactor and delay other in-flight requests.
+> **Important:** TaskHandler callbacks run on the processor's completion worker threads (see `completion_threads`), not the reactor thread, so they no longer block the event loop. They can still slow the processor down by two routes, so keep them fast -- typically just enqueuing a message for another system to pick up:
+>
+> - Callbacks compete with the reactor thread for the GVL, so heavy CPU work in a callback can still add latency to in-flight requests.
+> - A task stays in the capacity count until its result is delivered, so callbacks that back up consume request capacity and eventually make `enqueue` raise `MaxCapacityError`.
+>
+> **Callbacks must be thread-safe.** With the default `completion_threads` of 2, results are delivered concurrently, so two callbacks can run at the same time and in an order unrelated to the order the requests completed. Guard any state a handler shares between calls. Set `completion_threads: 1` to serialize delivery on a single worker thread.
+>
+> **Callbacks must be idempotent.** A failed delivery is retried `completion_retries` times (default 2), and each retry calls the callback again, so a callback that raises after enqueuing its message enqueues it more than once. Set `completion_retries: 0` if the callback cannot be made idempotent.
+>
+> If a callback keeps raising after the configured retries, the processor sends `completion_failed` to observers and does NOT send `request_end`, so durable tracking survives for external recovery.
 
 Example:
 ```ruby
@@ -230,10 +239,12 @@ erDiagram
 
 ## Process Model
 
-Each application process can run:
-- Multiple application threads
-- **One** async HTTP processor thread
+Each `Processor` instance runs:
+- **One** async HTTP processor (reactor) thread
 - **One** fiber reactor within the processor thread
+- A small pool of completion worker threads (`completion_threads`, default 2) that decode responses and deliver results
+
+A process usually runs one processor, but `Processor` is fully instance-based: a process can run several named processors (`Processor.new(config, name: :llm)`), each with its own capacity, timeouts, and threads. Requests carry an optional `processor` name (serialized with the request) that integrations use for routing.
 
 ```
 ┌─────────────────────────────────────────────────────────────┐
@@ -266,9 +277,10 @@ Each application process can run:
 The processor uses Ruby's Fiber scheduler (`async` gem) for non-blocking I/O:
 
 1. **Application threads** remain free while HTTP requests execute
-2. **Fiber reactor** multiplexes hundreds of HTTP connections
-3. **Connection pooling** and HTTP/2 reuse connections efficiently
-4. **TaskHandler callbacks** execute on the reactor thread and should be lightweight
+2. **Fiber reactor** multiplexes hundreds of HTTP connections and performs only socket I/O and light bookkeeping
+3. **Connection pooling** and HTTP/2 reuse connections efficiently; `max_connections_per_host` bounds sockets per host
+4. **Completion worker threads** decode response bodies (join, inflate, charset), build responses, and execute TaskHandler callbacks and `request_end` observers off the reactor. Decoding is CPU-bound and has no fiber yield point, so running it inline on the reactor would stop every other in-flight request until it finished. On a worker thread the Ruby scheduler can preempt it, and Zlib releases the GVL for part of the inflate
+5. **Delivery is concurrent, not serialized.** Callbacks and completion-time observers run on any of the `completion_threads` workers, so they must be thread-safe. Completions are also bounded: a task stays in the capacity count until a worker claims its result, so the completion backlog can never exceed `max_connections`
 
 ## State Management
 

data/CHANGELOG.md

@@ -4,6 +4,49 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## 1.6.0
+
+### Added
+
+- `Configuration#follow_method_changing_redirects` and `Request#follow_method_changing_redirects` (default true): when false, a redirect that would change the HTTP method (such as POST to GET on a 301, 302, or 303) is not followed and the redirect response is delivered to the callback instead. Redirects that preserve the method are still followed. The request setting overrides the configuration.
+- `Configuration#redirect_strip_headers` and `Request#redirect_strip_headers`: header names (matched case insensitively) that are always removed from redirected requests, so sensitive headers are never sent to a redirect target. Request-level names are applied in addition to the configured ones and survive serialization.
+- `PatientHttp.request`, `RequestHelper#async_request`, and `RequestTemplate#request` (and their `get`, `post`, and other method helpers) accept `follow_method_changing_redirects:` and `redirect_strip_headers:` and pass them to the `Request`.
+- `HEAD` and `QUERY` HTTP methods are supported by `Request`, `RequestTemplate`, `RequestHelper` (`async_head`, `async_query`), and `PatientHttp.head` / `PatientHttp.query`. `HEAD` requests cannot carry a body. The response size limit is not applied to the `Content-Length` of a `HEAD` response because no body is transferred.
+
+### Changed
+
+- The HTTP method used when following a redirect now follows RFC 9110. On 301 and 302 responses only `POST` is changed to `GET`; `HEAD`, `PUT`, `PATCH`, `DELETE`, and `QUERY` are re-sent unchanged with their body. On 303 responses `GET` and `HEAD` are preserved and every other method becomes `GET`. Previously every method was changed to `GET` on 301, 302, and 303.
+- When a redirect changes the method and drops the body, the `Content-Type`, `Content-Length`, `Content-Encoding`, `Content-Language`, and `Content-Location` headers are removed from the redirected request as well.
+- A 300 Multiple Choices response with a `Location` header is now followed, preserving the method and body. A 300 response without `Location` is delivered as a response, as before.
+- `RequestTask#redirect_task` accepts a `strip_headers:` option with the configured header names to remove.
+
+## 1.5.0
+
+### Added
+
+- Completion executor: finished results are now delivered on a small pool of worker threads (`Configuration#completion_threads`, default 2, minimum 1) instead of the reactor thread. Response decoding, payload encoding, task-handler callbacks, and `request_end` observers all run on these threads, so the reactor only performs socket I/O and light bookkeeping.
+- `Configuration#completion_retries` (default 2): delivery of a finished result is retried with a short backoff before the failure is reported. A retry calls `on_complete`/`on_error` again, so handlers must be idempotent; set `completion_retries: 0` to report the first failure without retrying.
+- `ProcessorObserver#completion_failed(request_task, error)`: sent when a result could not be delivered after all retries. `request_end` is NOT sent in this case, so durable tracking (crash-recovery records) stays in place and the request can be recovered by an external process instead of being silently lost. This replaces the previous behavior where a delivery failure was logged, swallowed, and the tracking was torn down.
+- `Configuration#max_connections_per_host` (default nil = unlimited): bounds the number of connections each host's HTTP client may open, which bounds file descriptor usage.
+- `Processor#remaining_capacity` and `Processor#capacity_available?`: cheap, advisory capacity checks with no observer notifications, so integrations can reject work before paying registration costs.
+- Named processors: `Processor.new(config, name:)` names a processor (used in its thread names), and `Request` accepts a `processor:` option that survives serialization (`as_json` / `load`), so integrations can route requests to one of several processors in the same process. `RequestTemplate`, `RequestHelper`, and `PatientHttp.request` pass the option through. `PatientHttp::UnknownProcessorError` is defined for handlers to raise for unrecognized names. Serialized output for requests without a processor name is unchanged.
+
+### Changed
+
+- Compressed response bodies (gzip/deflate) are now inflated on a completion worker thread instead of the reactor thread. The size limit still applies to the inflated bytes, so gzip-bomb protection is unchanged. The `content-encoding` header is removed from the response after decoding, as before.
+- Requests send `accept-encoding: gzip` by default, but a request that sets the header keeps its own value. Set `accept-encoding: identity` on a request to opt out of compression, or name another encoding to receive the body still encoded. Previously the header was set unconditionally.
+- Inline and synchronous execution decodes response bodies through the same reader as the async path instead of a `Protocol::HTTP::AcceptEncoding` middleware. The middleware overwrote the request's `accept-encoding` header, so a request opting out of compression was previously honored only when it ran on the processor.
+- A `content-encoding` header naming more than one encoding is now decoded from the outermost encoding inward, and `identity` is recognized. Previously only a header holding exactly one supported name was decoded, so a value such as `gzip, identity` delivered the body still compressed.
+- A response body carrying an encoding the reader cannot decode is delivered unchanged with a `content-encoding` header naming only the encodings still applied, and the condition is now logged as a warning. Such a body keeps its binary encoding, because the Content-Type charset does not describe encoded bytes.
+- `ProcessorObserver` hooks no longer all run on the reactor thread; see the class documentation for the thread each hook runs on. `request_end` and `request_error` for completed requests now run on completion worker threads.
+- **Breaking for callbacks:** `TaskHandler` callbacks and completion-time observer hooks must now be thread-safe. The reactor thread previously serialized them; they now run concurrently on `completion_threads` workers, in an order unrelated to the order the requests completed. Set `completion_threads: 1` to restore serialized delivery.
+- On shutdown, results that were already handed off for delivery are delivered before remaining tasks are re-enqueued.
+
+### Fixed
+
+- A `deflate` response body carrying a zlib header, which is the format RFC 9110 specifies for that encoding, failed to inflate with `Zlib::DataError: invalid stored block lengths`. Both the zlib and the raw deflate wire formats are now supported.
+- A response body that a text content type claims is text, but that does not hold text, is now stored as a binary payload instead of as text. Such a body could not be serialized, so `JSON.generate` raised `JSON::GeneratorError` and the result could never be delivered. This happened for a body still carrying a content encoding the reader cannot decode (for example `br`), and for text holding an invalid byte sequence.
+
 ## 1.4.0
 
 ### Added

data/README.md

@@ -34,8 +34,8 @@ class MyTaskHandler < PatientHttp::TaskHandler
 
   def on_complete(response, callback)
     # Enqueue a message for your application to process the response.
-    # Keep this lightweight -- don't do heavy processing here since
-    # it runs on the processor thread.
+    # Keep this lightweight and thread-safe -- it runs on a completion
+    # worker thread, concurrently with other completions.
     MyJobSystem.enqueue(callback, :on_complete, response.as_json)
   end
 
@@ -51,7 +51,11 @@ class MyTaskHandler < PatientHttp::TaskHandler
 end
 ```
 
-> **Important:** TaskHandler callbacks run on the processor's reactor thread. They should be lightweight and fast -- typically just enqueuing a message for another system to pick up. Doing heavy processing in a callback will block the reactor and delay other in-flight requests.
+> **Important:** TaskHandler callbacks run on the processor's completion worker threads (see `completion_threads`), not the reactor thread, so they no longer block the event loop. Keep them lightweight anyway -- typically just enqueuing a message for another system to pick up. Heavy callbacks compete with the reactor for the GVL, and because a task stays in the capacity count until its result is delivered, callbacks that back up consume request capacity.
+>
+> Callbacks must be thread-safe. Results are delivered concurrently on `completion_threads` workers (default 2), so two callbacks can run at the same time and in an order unrelated to the order the requests completed. Set `completion_threads: 1` to serialize delivery.
+>
+> Callbacks must also be idempotent. A callback that raises is retried `completion_retries` times (default 2), so one that raises after enqueuing its message enqueues it again. Set `completion_retries: 0` if that is not acceptable.
 
 ### 2. Create and Enqueue Requests
 
@@ -159,7 +163,7 @@ get_request = template.get("/users/123")
 post_request = template.post("/users", json: {name: "John"})
 ```
 
-Templates support all HTTP methods (`get`, `post`, `put`, `patch`, `delete`) and handle URL joining, header merging, and query parameter encoding.
+Templates support all HTTP methods (`get`, `head`, `post`, `put`, `patch`, `delete`, `query`) and handle URL joining, header merging, and query parameter encoding.
 
 ## Standard Interface
 
@@ -184,7 +188,7 @@ PatientHttp.register_handler do |request:, callback:, callback_args: nil, raise_
 end
 
 # Now you can make requests directly through the PatientHttp interface with the .request,
-# .get, .post, .patch, .put, and .delete class methods:
+# .get, .head, .post, .patch, .put, .delete, and .query class methods:
 PatientHttp.get(
   "https://api.example.com/users/123",
   callback: FetchUserCallback,
@@ -225,7 +229,7 @@ Use `PatientHttp::RequestHelper` when you want a simple API for creating and dis
 1. Register a request handler with `PatientHttp.register_handler` that defines how requests are dispatched to your job queue or background processing system.
 2. Include `PatientHttp::RequestHelper` in your class.
 3. Optionally define a `request_template` for shared `base_url`, headers, and timeout.
-4. Call `async_get`, `async_post`, `async_put`, `async_patch`, `async_delete`, or `async_request`.
+4. Call `async_get`, `async_head`, `async_post`, `async_put`, `async_patch`, `async_delete`, `async_query`, or `async_request`.
 
 ```ruby
 class ApiClient
@@ -614,6 +618,49 @@ If a request references a preprocessor name that is not registered, a `PatientHt
 
 When redirects are followed, preprocessors are re-run against each redirect URL so signatures stay valid. On cross-origin redirects they are dropped entirely, consistent with the stripping of `Authorization` and `Cookie` headers, so signed credentials are never sent to an unexpected origin.
 
+## Redirects
+
+Redirect responses (300, 301, 302, 303, 307, and 308) with a `Location` header are followed automatically, up to `max_redirects` hops. A 300 response is followed only when the server names a preferred choice in `Location`. Redirect loops raise `RecursiveRedirectError` and exceeding the limit raises `TooManyRedirectsError`. Any redirect that is not followed is delivered to the callback as a normal response.
+
+The HTTP method of the redirected request follows RFC 9110:
+
+| Status | Method |
+| --- | --- |
+| 301, 302 | `POST` becomes `GET` and the body is dropped. Other methods (including `HEAD`, `PUT`, `DELETE`, and `QUERY`) are preserved with their body. |
+| 303 | `GET` and `HEAD` are preserved. Every other method becomes `GET` and the body is dropped. |
+| 300, 307, 308 | The method and body are preserved. |
+
+The QUERY specification states that the POST-to-GET exception on 301 and 302 does not apply to `QUERY`, so a redirected `QUERY` is re-sent as a `QUERY` with its body, and a 303 turns it into a `GET`.
+
+### Preventing method changes
+
+Set `follow_method_changing_redirects: false` to stop following redirects that would change the HTTP method. A `POST` that receives a 302 then completes with the 302 response instead of being retried as a `GET`. Redirects that preserve the method (a `PUT` on a 301, or any method on a 307) are still followed. The option can be set on the `Configuration` or on a single `Request`; the request value wins when both are set.
+
+```ruby
+config = PatientHttp::Configuration.new(follow_method_changing_redirects: false)
+
+# Or per request
+request = PatientHttp::Request.new(:post, "https://api.example.com/submit", body: payload, follow_method_changing_redirects: false)
+```
+
+### Stripping headers on redirects
+
+`Authorization` and `Cookie` headers are always removed on cross-origin redirects. To make sure other sensitive headers are never sent to a redirect target, list them in `redirect_strip_headers`. Header names are matched case insensitively. Listed headers are removed from every redirected request, same-origin or not.
+
+```ruby
+config = PatientHttp::Configuration.new(redirect_strip_headers: ["X-Api-Key", "X-Internal-Token"])
+
+# Or per request; these are stripped in addition to the configured headers
+request = PatientHttp::Request.new(:get, "https://api.example.com/data", headers: headers, redirect_strip_headers: "X-Signature")
+
+# The same options are accepted by PatientHttp.request, the async_* helpers, and RequestTemplate
+PatientHttp.get("https://api.example.com/data", callback: FetchCallback, redirect_strip_headers: "X-Signature")
+```
+
+Per-request header names survive serialization into the job queue, so they apply no matter which process follows the redirect.
+
+Stripping applies to the headers set on the request. Preprocessors run again on each same-origin redirect and can add headers after the strip, so a header that a preprocessor sets is sent to the redirect target. When a redirect changes the method and drops the body, the headers that describe the body (`Content-Type`, `Content-Length`, `Content-Encoding`, `Content-Language`, and `Content-Location`) are removed as well.
+
 ## Troubleshooting
 
 ### Warning: `ThreadError: Attempt to unlock a mutex which is not locked`
@@ -656,6 +703,15 @@ config = PatientHttp::Configuration.new(
   # Maximum redirects to follow (default: 5, 0 disables)
   max_redirects: 5,
 
+  # Follow redirects that must change the HTTP method, such as POST to GET on
+  # a 302 (default: true). When false, those requests receive the redirect response.
+  follow_method_changing_redirects: true,
+
+  # Header names (case insensitive) always stripped from redirected requests
+  # (default: []). Authorization and Cookie are always stripped on cross-origin
+  # redirects.
+  redirect_strip_headers: ["X-Api-Key", "X-Internal-Token"],
+
   # Maximum number of hosts to maintain persistent connections for (default: 100)
   connection_pool_size: 100,
 
@@ -685,9 +741,14 @@ config.register_secret(:api_token, ENV["MY_API_TOKEN"])
 ### Tuning Tips
 
 - **max_connections**: Each connection uses memory and file descriptors. A tuned system can handle thousands.
+- **max_connections_per_host**: Bounds sockets per host (default unlimited). Set a value such as 32 for high-concurrency deployments so one host cannot consume every file descriptor. Verify the process file descriptor limit covers `max_connections` plus pooled idle host connections plus the application's own connections.
 - **request_timeout**: Set based on expected API response times. AI/LLM APIs may need minutes.
 - **connection_pool_size**: Increase for applications calling many different API hosts.
-- **max_response_size**: Keeps memory usage bounded. Large responses may need external payload storage.
+- **max_response_size**: Keeps memory usage bounded. Large responses may need external payload storage. The limit applies to the inflated bytes of compressed responses.
+- **Response compression**: Requests ask for `gzip` by default and the body is inflated on a completion worker thread. Set `accept-encoding` on a request to change this: `identity` skips compression, and any other encoding is delivered still encoded with its `content-encoding` header kept so you can decode it yourself.
+- **completion_threads**: Number of threads that decode responses and deliver results (default 2). Increase when callbacks do heavier work (serialization, encryption) and completions back up behind them. Any value above 1 delivers results concurrently, so `TaskHandler` callbacks and completion-time observers must be thread-safe. Use 1 to serialize delivery.
+- **completion_retries**: Delivery retries before a result is reported through `completion_failed` (default 2). A retry calls `on_complete`/`on_error` again, so a handler that raises *after* enqueuing its message delivers that message twice. Make handlers idempotent, or set `completion_retries: 0` to report the first failure without retrying.
+- **shutdown_timeout**: Set below the process supervisor's termination window so the drain (including handed-off completions) finishes before a hard kill.
 
 ## Processor Lifecycle
 

data/VERSION

@@ -1 +1 @@
-1.4.0
+1.6.0

data/lib/patient_http/client.rb

@@ -9,7 +9,8 @@ module PatientHttp
         connection_timeout: config.connection_timeout,
         proxy_url: config.proxy_url,
         retries: config.retries,
-        protocol: config.protocol
+        protocol: config.protocol,
+        connection_limit: config.max_connections_per_host
       )
       @response_reader = ResponseReader.new(@processor)
       @request_preparer = RequestPreparer.new(config)
@@ -17,6 +18,10 @@ module PatientHttp
 
     # Make an asynchronous HTTP request.
     #
+    # The returned body is the array of raw (possibly compressed) body chunks;
+    # use {#decode_response} to produce the final body string. Splitting the
+    # decode out keeps CPU-bound work off the reactor thread.
+    #
     # @param request [Request] the request to make
     # @param request_id [String] unique request identifier
     # @return [Hash] the response data with keys for :status, :headers, and :body
@@ -35,7 +40,7 @@ module PatientHttp
           # Note: headers that appear multiple times (e.g. set-cookie) are
           # flattened to a single joined string value.
           headers_hash = async_response.headers.to_h.transform_values(&:to_s)
-          body = @response_reader.read_body(async_response, headers_hash)
+          body = @response_reader.read_raw_body(async_response, headers_hash)
 
           {
             status: async_response.status,
@@ -54,6 +59,26 @@ module PatientHttp
       end
     end
 
+    # Decode raw response data into deliverable response data.
+    #
+    # Joins and inflates the raw body chunks, applies the charset, and rewrites
+    # the content-encoding header to name only the encodings still applied to
+    # the body. The header is removed when nothing is left, and kept when the
+    # server used an encoding the reader cannot decode, so the delivered
+    # response always describes the body it carries. This is CPU-bound work
+    # intended to run on a completion worker thread.
+    #
+    # @param response_data [Hash] raw response data from {#make_request}
+    # @return [Hash] response data with the decoded body string
+    # @raise [ResponseTooLargeError] if the inflated body exceeds max_response_size
+    def decode_response(response_data)
+      headers = response_data[:headers]
+      body = @response_reader.decode_body(response_data[:body], headers)
+      headers = ResponseReader.rewrite_content_encoding(headers)
+
+      response_data.merge(headers: headers, body: body)
+    end
+
     # Close all clients and release resources.
     #
     # @return [void]

data/lib/patient_http/client_pool.rb

@@ -15,7 +15,7 @@ module PatientHttp
       http2: Async::HTTP::Protocol::HTTP2
     }.freeze
 
-    def initialize(max_size:, connection_timeout: nil, proxy_url: nil, retries: 3, protocol: nil)
+    def initialize(max_size:, connection_timeout: nil, proxy_url: nil, retries: 3, protocol: nil, connection_limit: nil)
       if protocol && !PROTOCOLS.include?(protocol)
         raise ArgumentError.new("protocol must be one of #{PROTOCOLS.keys.inspect}, got: #{protocol.inspect}")
       end
@@ -26,16 +26,17 @@ module PatientHttp
       @proxy_url = proxy_url
       @retries = retries
       @protocol = protocol
+      @connection_limit = connection_limit
       @mutex = Mutex.new
       @proxy_client = nil
     end
 
-    attr_reader :max_size, :connection_timeout, :proxy_url, :retries, :protocol
+    attr_reader :max_size, :connection_timeout, :proxy_url, :retries, :protocol, :connection_limit
 
     # Get or create a client for the given endpoint.
     #
     # @param endpoint [Async::HTTP::Endpoint] the target endpoint
-    # @return [Protocol::HTTP::AcceptEncoding] wrapped client
+    # @return [Async::HTTP::Client] the client for the endpoint's host
     def client_for(endpoint)
       key = host_key(endpoint)
 
@@ -154,13 +155,15 @@ module PatientHttp
     end
 
     def make_client(endpoint)
-      client = @proxy_url ? make_proxied_client(endpoint) : make_direct_client(endpoint)
-      ::Protocol::HTTP::AcceptEncoding.new(client)
+      # Response bodies are decoded by ResponseReader on a completion worker
+      # thread instead of a Protocol::HTTP::AcceptEncoding wrapper, so the
+      # reactor thread never pays for inflating compressed bodies.
+      @proxy_url ? make_proxied_client(endpoint) : make_direct_client(endpoint)
     end
 
     def make_direct_client(endpoint)
       configured_endpoint = configure_endpoint(endpoint)
-      Async::HTTP::Client.new(configured_endpoint, retries: @retries)
+      Async::HTTP::Client.new(configured_endpoint, retries: @retries, **client_options)
     end
 
     def make_proxied_client(endpoint)
@@ -170,7 +173,11 @@ module PatientHttp
       configured_endpoint = configure_endpoint(endpoint)
 
       proxy = @proxy_client.proxy(configured_endpoint)
-      Async::HTTP::Client.new(proxy.wrap_endpoint(configured_endpoint), retries: @retries)
+      Async::HTTP::Client.new(proxy.wrap_endpoint(configured_endpoint), retries: @retries, **client_options)
+    end
+
+    def client_options
+      @connection_limit ? {limit: @connection_limit} : {}
     end
 
     def create_proxy_client

data/lib/patient_http/completion_executor.rb

@@ -0,0 +1,139 @@
+# frozen_string_literal: true
+
+module PatientHttp
+  # Fixed pool of worker threads that deliver completed request results.
+  #
+  # The processor's reactor thread hands each finished HTTP exchange to this
+  # pool so response decoding, serialization, and callback delivery never
+  # block the event loop. Jobs are arbitrary callables consumed from a single
+  # queue.
+  #
+  # @api private
+  class CompletionExecutor
+    # Initialize the executor and start its worker threads.
+    #
+    # @param threads [Integer] number of worker threads
+    # @param logger [Logger, nil] logger for unexpected job errors
+    # @param thread_name_prefix [String] prefix for worker thread names
+    # @param on_finished [#call, nil] invoked after each job completes, outside
+    #   any executor lock, so the owner can re-check idle conditions
+    def initialize(threads:, logger: nil, thread_name_prefix: "patient-http-completion", on_finished: nil)
+      @queue = Thread::Queue.new
+      @logger = logger
+      @on_finished = on_finished
+      @mutex = Mutex.new
+      # Jobs enqueued but not yet fully executed. Tracked separately from the
+      # queue size so a job that has been popped but is still running keeps
+      # the executor non-idle.
+      @outstanding = 0
+      @threads = Array.new(threads) do |index|
+        Thread.new do
+          Thread.current.name = "#{thread_name_prefix}-#{index + 1}"
+          run_worker
+        end
+      end
+    end
+
+    # Enqueue a job for execution.
+    #
+    # @param job [#call] the job to run
+    # @raise [ClosedQueueError] if the executor has been shut down
+    # @return [void]
+    def enqueue(job)
+      @mutex.synchronize { @outstanding += 1 }
+      begin
+        @queue.push(job)
+      rescue ClosedQueueError
+        @mutex.synchronize { @outstanding -= 1 }
+        raise
+      end
+      nil
+    end
+
+    # Check whether the executor has no queued or running jobs.
+    #
+    # @return [Boolean]
+    def idle?
+      @mutex.synchronize { @outstanding == 0 }
+    end
+
+    # Check whether the given thread is one of this executor's workers.
+    #
+    # @param thread [Thread] the thread to check
+    # @return [Boolean]
+    def worker_thread?(thread = Thread.current)
+      @threads.include?(thread)
+    end
+
+    # Shut down the executor: close the queue so workers drain remaining jobs
+    # and exit, then join them within the timeout. Workers still alive after
+    # the deadline are killed; their tasks remain durably tracked and are
+    # recovered by the owner's re-enqueue logic.
+    #
+    # Safe to call more than once and from a worker thread itself (the
+    # current thread is never joined or killed).
+    #
+    # @param timeout [Numeric] seconds to wait for workers to drain
+    # @return [void]
+    def shutdown(timeout: 5)
+      @queue.close
+
+      deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + timeout
+      @threads.each do |thread|
+        next if thread.equal?(Thread.current)
+
+        remaining = deadline - Process.clock_gettime(Process::CLOCK_MONOTONIC)
+        thread.join(remaining.positive? ? remaining : 0)
+        if thread.alive?
+          thread.kill
+          thread.join(1)
+        end
+      end
+
+      discard_undrained_jobs
+      nil
+    end
+
+    private
+
+    # Drop jobs left in the closed queue by workers that were killed at the
+    # shutdown deadline. Those jobs can never run, so they must stop counting
+    # against the outstanding total or the executor would never report itself
+    # idle again.
+    #
+    # @return [void]
+    def discard_undrained_jobs
+      discarded = 0
+
+      loop do
+        break unless @queue.pop(true)
+        discarded += 1
+      rescue ThreadError
+        break
+      end
+
+      @mutex.synchronize { @outstanding -= discarded } if discarded > 0
+      nil
+    end
+
+    def run_worker
+      while (job = @queue.pop)
+        begin
+          job.call
+        rescue => e
+          @logger&.error(
+            "[PatientHttp] Completion worker error: #{e.class} - #{e.message}\n#{e.backtrace&.join("\n")}"
+          )
+          warn("#{e.inspect}\n#{e.backtrace&.join("\n")}") if PatientHttp.testing?
+        ensure
+          @mutex.synchronize { @outstanding -= 1 }
+          begin
+            @on_finished&.call
+          rescue => e
+            @logger&.error("[PatientHttp] Completion executor callback error: #{e.inspect}")
+          end
+        end
+      end
+    end
+  end
+end