butler-http 0.1.0

data/README.md

@@ -0,0 +1,659 @@
+# Butler
+
+### Modern HTTP for modern Ruby.
+
+[![Gem Version](https://img.shields.io/gem/v/butler-http)](https://rubygems.org/gems/butler-http)
+![Ruby](https://img.shields.io/badge/ruby-%3E%3D%203.3-red)
+![License](https://img.shields.io/badge/license-MIT-blue)
+
+**Fiber-native concurrency, HTTP/1.1 + HTTP/2, and production-grade
+resilience in one Ruby HTTP client** — transparent ALPN negotiation,
+structured concurrency, built-in resilience, and security-conscious
+defaults, without ever exposing its transport in the public API.
+
+One client. Modern concurrency. Production-ready HTTP.
+
+```ruby
+# HTTParty-style, zero setup:
+Butler.get("https://api.example.com/users").json
+
+# Or a configured Client, for connection pooling/retries/middleware tuned per-API:
+client = Butler::Client.new(base_url: "https://api.example.com")
+
+client.async do |tasks|
+  users  = tasks.async { client.get("/users") }
+  orders = tasks.async { client.get("/orders") }
+  { users: users.wait.json, orders: orders.wait.json }
+end
+```
+
+## Table of contents
+
+- [Why Butler](#why-butler)
+- [Installation](#installation)
+- [Quick start](#quick-start)
+- [Architecture, in one picture](#architecture-in-one-picture)
+- [Usage](#usage)
+  - [Module-level shortcuts](#module-level-shortcuts)
+  - [Requests](#requests)
+  - [Bodies](#bodies)
+  - [Streaming](#streaming)
+  - [Structured concurrency](#structured-concurrency)
+  - [Deadlines and timeouts](#deadlines-and-timeouts)
+  - [Retries](#retries)
+  - [Circuit breaker](#circuit-breaker)
+  - [Security](#security)
+  - [Telemetry](#telemetry)
+  - [Middleware](#middleware)
+  - [Testing — no WebMock/VCR needed](#testing--no-webmockvcr-needed)
+  - [Rails](#rails)
+  - [Errors](#errors)
+- [Configuration reference](#configuration-reference)
+- [Benchmarks](#benchmarks)
+- [What's not here yet](#whats-not-here-yet)
+- [Development](#development)
+- [Contributing](#contributing)
+- [License](#license)
+
+## Why Butler
+
+Butler is the modern Ruby HTTP client built for concurrent, resilient
+applications. Ruby's HTTP client landscape is a set of trade-offs, not a
+clear winner:
+
+| Client | Strength | Limitation |
+| --- | --- | --- |
+| `Net::HTTP` | Standard library | Low-level, no HTTP/2, easy to misuse (no default timeouts) |
+| Faraday | Excellent ecosystem/middleware | The adapter/middleware abstraction itself adds a layer of indirection |
+| Excon | Performance-focused | Less opinionated about resilience/DX out of the box |
+| HTTP.rb | Ruby-friendly API | Different concurrency/transport model than Fiber-based servers |
+| HTTParty | The simplest possible `ClassName.get(url)` call | No connection reuse across calls, no HTTP/2, no built-in resilience |
+| Async::HTTP | Excellent async foundation | Lower-level; you build the client-facing API and resilience yourself |
+
+Faraday gives you an ecosystem. Butler gives you a modern HTTP runtime —
+a small, deliberately-scoped public API (`Client`, `Request`, `Response`,
+`Headers`) in front of real HTTP/1.1 **and** HTTP/2, Fiber-native
+concurrency, and resilience/security/observability that don't require four
+extra gems to get.
+
+| Capability | Butler | Faraday | Excon | Net::HTTP |
+| --- | --- | --- | --- | --- |
+| Fiber-native concurrency | ✓ | — | — | limited |
+| HTTP/2 (ALPN, multiplexed) | ✓ (first-class) | adapter-dependent | limited | depends |
+| Connection pooling | ✓ | adapter | ✓ | manual |
+| Retries + backoff/jitter | built-in | middleware | limited | manual |
+| Circuit breaker | built-in | external gem | external gem | external gem |
+| Deadlines (total budget across retries) | ✓ | — | — | — |
+| OpenTelemetry | first-class, optional | external | external | external |
+| Native request stubbing | ✓ | via WebMock | via WebMock | via WebMock |
+| Rails integration | optional, not required | good | good | basic |
+
+**Async::HTTP gives you the foundation. Butler gives you the
+application-facing client.** That's the one architectural rule that keeps
+this from rotting into "Faraday but slower": Butler owns its public
+abstractions. `async`/`async-http` implement the real transport
+underneath, but nothing outside `lib/butler/transport.rb` ever touches
+those types — the public API (`Client`/`Request`/`Response`/`Headers`)
+doesn't change if the transport underneath it ever does. See
+[docs/architecture.md](docs/architecture.md).
+
+## Installation
+
+```ruby
+# Gemfile
+gem "butler-http"
+```
+
+```
+bundle install
+```
+
+Or without Bundler:
+
+```
+gem install butler-http
+```
+
+Requires Ruby >= 3.3 — matching the floor both runtime dependencies
+declare themselves (`async 2.45.1`/`async-http 0.103.0`, checked
+2026-09-02; a lower claim here would just mean `bundle install` fails on
+whatever Ruby actually can't satisfy them). Runtime dependencies are
+`async` and `async-http` (both from the
+[socketry](https://github.com/socketry) ecosystem) — Butler keeps its own
+dependency footprint to just those two, so it stays a
+reasonable choice for non-Rails Ruby projects too, not just Rails apps that
+already pull in a large dependency tree.
+
+## Quick start
+
+The fastest way in — module-level calls, HTTParty-style, no `Client` to set
+up first:
+
+```ruby
+require "butler"
+
+response = Butler.get("https://api.example.com/users")
+response.status   # => 200
+response.headers  # => Butler::Headers
+response.body     # => raw body String
+response.json     # => parsed JSON (Hash/Array), or nil if the body isn't valid JSON
+
+response = Butler.post("https://api.example.com/users", json: { name: "Ram", email: "ram@example.com" })
+Butler.get("https://api.example.com/users", headers: { "Authorization" => "Bearer token" })
+Butler.get("https://api.example.com/users", params: { page: 2, limit: 50 })
+```
+
+`Butler.get`/`.post`/`.put`/`.patch`/`.delete`/`.head`/`.options` all run
+against `Butler.default_client` — a real, connection-pooled
+`Butler::Client` built lazily on first use and shared for the life of the
+process, so repeated module-level calls still get pooling/retries/a
+circuit breaker rather than reconnecting from scratch every time. Reach
+for `Butler::Client.new` instead once you want configuration scoped to one
+API — a fixed `base_url`, its own retry policy, middleware — rather than
+sharing the one global default:
+
+```ruby
+client = Butler::Client.new(base_url: "https://api.example.com")
+
+response = client.get("/users")   # same Response API as Butler.get above
+response = client.post("/users", json: { name: "Ram", email: "ram@example.com" })
+client.get("/users", headers: { "Authorization" => "Bearer token" })
+client.get("/users", params: { page: 2, limit: 50 })
+```
+
+## Architecture, in one picture
+
+```
+Client
+  └─ Pipeline::Chain
+       ├─ SecurityMiddleware      (host allow/block list — before any socket is touched)
+       ├─ TelemetryMiddleware     (Instrumentation + optional OpenTelemetry span)
+       ├─ [any middleware you add via Client#use]
+       ├─ TimeoutMiddleware       (total-deadline checkpoint)
+       ├─ CircuitBreakerMiddleware
+       └─ RetryMiddleware         (the only middleware with a loop)
+            └─ ConnectionPool#acquire
+                 └─ Transport.current            (Transport::Async, or Testing::FakeTransport when stubbing)
+                      └─ Async::HTTP::Client      (HTTP/1.1 or HTTP/2 — chosen via ALPN)
+```
+
+Only `lib/butler/transport.rb` ever imports `Async::HTTP`/`Protocol::HTTP`.
+Everything above it is pure `Butler::Request`/`Butler::Response`/
+`Butler::Headers` — swap the transport later (a real HTTP/3 implementation,
+say) and nothing above this line changes. `ConnectionPool` itself owns
+less than it might look like: it's a thin registry mapping one
+origin+protocol fingerprint to one memoized `Async::HTTP::Client` —
+HTTP/1.1 connection-level pooling and HTTP/2 stream multiplexing are
+`Async::HTTP::Client`'s own job underneath that, not reimplemented by
+Butler. Full request lifecycle, the deadline model, the redirect-handling
+design, and exactly what the pool does and doesn't own are in
+[docs/architecture.md](docs/architecture.md#why-connectionpool-isnt-a-socket-pool).
+
+## Usage
+
+### Module-level shortcuts
+
+```ruby
+Butler.get("https://api.example.com/users")
+Butler.post("https://api.example.com/users", json: { name: "Ram" })
+# put/patch/delete/head/options all work the same way
+
+Butler.async do |tasks|
+  a = tasks.async { Butler.get("https://api.example.com/a") }
+  b = tasks.async { Butler.get("https://api.example.com/b") }
+  [a.wait, b.wait]
+end
+```
+
+Every `Butler.<method>` call is `Butler.default_client.<method>` —
+`Butler.default_client` is a real `Butler::Client`, built lazily the first
+time any module-level call is made and memoized for the life of the
+process, so it keeps its own connection pool and circuit-breaker state
+across calls exactly like a `Client` you built yourself would, rather than
+reconnecting from scratch on every call the way a purely stateless
+`ClassName.get` API would have to.
+
+It snapshots `Butler.configuration` as of whenever it's first built — the
+same way `Butler::Client.new` always has:
+
+```ruby
+Butler.configure { |config| config.base_url = "https://api.example.com" }
+Butler.get("/users") # relative paths now work against that base_url
+```
+
+Call `Butler.configure` before your first module-level call. Reconfiguring
+afterward doesn't retroactively change the already-built default client —
+call `Butler.reset_default_client!` (closes its connections first) if you
+need a later configuration change to take effect, or just switch to
+`Butler::Client.new` once you're reaching for more than one or two
+settings.
+
+### Requests
+
+All of `get`, `post`, `put`, `patch`, `delete`, `head`, and `options` share
+the same signature: `client.method(path = nil, **options)`.
+
+```ruby
+client.get("/users")
+client.get("/users", params: { page: 2, limit: 50 })
+client.get("/users", headers: { "Authorization" => "Bearer token" })
+client.get("https://other-host.example.com/anything") # an absolute URL overrides base_url for one call
+client.get("/users", basic_auth: ["alice", "secret"])
+client.get("/users", basic_auth: { username: "alice", password: "secret" })
+client.get("/users", http_version: :http1) # force this one call onto HTTP/1.1
+```
+
+| Option | Description |
+| --- | --- |
+| `params:` | Hash merged into the URL's query string |
+| `headers:` | Hash of request headers (per-request headers override client-level defaults) |
+| `basic_auth:` | `[user, password]` or `{ username:, password: }` |
+| `idempotent:` | Marks a PUT/DELETE as safe to retry on a retryable status (POST is never auto-retried regardless) |
+| `deadline:` | Per-call total wall-clock budget in seconds, overriding the client default |
+| `stream:` | `true` to get back a `Butler::Stream` (see [Streaming](#streaming)) instead of a fully-buffered body |
+| `http_version:` | `:auto`, `:http1`, or `:http2` for this call only, overriding the client's `http_version` (see [Configuration reference](#configuration-reference)) — gets its own pooled connection per origin, kept separate from calls using the client default |
+
+### Bodies
+
+```ruby
+client.post("/users", json: { name: "Ram" })                     # application/json
+client.post("/users", form: { name: "Ram", role: "admin" })      # application/x-www-form-urlencoded
+client.post("/upload", body: File.open("report.csv"))            # streamed from the IO, chunk by chunk
+client.post("/upload", io: File.open("report.csv"))               # equivalent, more explicit
+client.post("/webhook", body: "raw string")
+client.post("/webhook", stream: some_enumerator_of_chunks)         # a caller-driven upload stream
+```
+
+### Streaming
+
+Request uploads and response bodies can both be streamed rather than fully
+buffered into memory:
+
+```ruby
+response = client.get("/export.csv", stream: true)
+response.stream.each_chunk { |chunk| output.write(chunk) }
+# stream releases the underlying connection once consumption finishes —
+# implicitly, via #each_chunk's own ensure, or explicitly via response.stream.close
+# or, if you do want it all in memory after all:
+response.body # reads the whole stream and buffers it
+```
+
+`response.stream` always releases when you're done with it — either
+implicitly (`#each_chunk`'s own `ensure` calls `#close` once iteration
+finishes or the block raises) or explicitly via `response.stream.close`, and
+it's safe to call `#close` more than once. What "released" means depends on
+how much you actually read: a fully-consumed stream lets the underlying
+HTTP/1.1 keep-alive connection (or HTTP/2 stream) go back to the pool for
+reuse; abandoning a stream early — breaking out of `#each_chunk` partway
+through — can force a real connection close instead, since the transport
+can no longer guarantee it knows where the next response would start on
+that same connection.
+
+### Structured concurrency
+
+```ruby
+client.async do |tasks|
+  a = tasks.async { client.get("/a") }
+  b = tasks.async { client.get("/b") }
+  [a.wait, b.wait]
+end
+```
+
+`client.async` gives you structured concurrency for callers that want
+several HTTP calls to overlap: exiting the block always resolves any
+child task you forgot to `.wait`, and cancels anything still running
+after an exception — an error raised inside one child task propagates out
+through `.wait` exactly like a normal exception would. Inside an existing
+Fiber scheduler or `Async` reactor (a nested `client.async`, or an
+app server like Falcon), Butler participates in that current execution
+context rather than starting a competing one; from ordinary synchronous
+Ruby, including a bare `client.get(...)` with no surrounding `async`
+block at all, Butler manages the async execution for you — a single
+reactor shared for the process, not spun up and torn down per call. See
+[docs/architecture.md](docs/architecture.md#why-bare-calls-dont-pay-reactor-setup-per-call)
+for exactly how that's implemented, if you're curious.
+
+Even with that shared background reactor, Butler's own request pipeline
+(security checks, retry/circuit-breaker bookkeeping, telemetry, building
+`Request`/`Response` objects) is real per-call **CPU** work beyond what a
+bare `Net::HTTP.get` does. Against any upstream with real network latency
+— the normal case — that extra work overlaps with I/O wait and a tight
+sequential loop of `client.get` calls lands close to `Net::HTTP`'s own
+loop in wall-clock time; it only shows up clearly against a near-zero-
+latency upstream or a CPU-bound host. See
+[benchmarks/README.md](benchmarks/README.md#comparisonrb) for real
+numbers either way.
+
+### Deadlines and timeouts
+
+```ruby
+client = Butler::Client.new(
+  base_url: "https://api.example.com",
+  connect_timeout: 2, read_timeout: 5, write_timeout: 5, # per-attempt budgets
+  deadline: 5, # total wall-clock budget for one call, across every retry/redirect
+)
+```
+
+`deadline:` is the total budget for one `client.get`/`client.post`/etc call
+— DNS, connect, TLS, every retry attempt, every redirect hop, all count
+against it, and **retries never reset it**. `connect_timeout` bounds
+establishing a connection; `read_timeout`/`write_timeout` (or an explicit
+`request_timeout` override) bound the request/response round-trip on an
+already-open connection — the larger of read/write becomes that per-attempt
+ceiling, since the underlying transport performs a request's write and its
+response's read as one call rather than timing each phase separately.
+Whichever of these is smaller wins for any given attempt: a generous
+`read_timeout` still gets cut short once `deadline:` is nearly exhausted.
+Exceeding either raises `Butler::Errors::TimeoutError`.
+
+### Retries
+
+```ruby
+client = Butler::Client.new(
+  retry: { max_attempts: 3, base_delay: 0.1, max_delay: 5.0, jitter: true },
+)
+
+client.put("/orders/42", json: { status: "shipped" }, idempotent: true) # opt in explicitly
+```
+
+- **GET/HEAD/OPTIONS** are retried automatically on a retryable status
+  (`408, 425, 429, 500, 502, 503, 504` by default) or a connection-level
+  failure (refused/reset connection, TLS failure, timeout) — nothing that
+  reached the server can be confirmed either way for those, so retrying a
+  network-level failure doesn't make the request any less safe. The one
+  deliberate exception: a **certificate verification failure**
+  (`Butler::Errors::CertificateVerificationError` — self-signed, expired,
+  hostname mismatch, untrusted root) is never retried, even though it's a
+  `TLSError` like the retried ones — a bad certificate won't become valid
+  a few hundred milliseconds later, so retrying just delays surfacing a
+  real problem instead of fixing anything.
+- **PUT/DELETE** are only retried on a retryable status when explicitly
+  marked `idempotent: true`.
+- **POST is never auto-retried on a 5xx status**, regardless of
+  `idempotent:` — matching how most systems reason about "did my write
+  actually happen." (Connection-level failures are still retried for POST
+  too, since the request demonstrably never reached the server.)
+- `Retry-After` (seconds or an HTTP-date) is honored ahead of the computed
+  exponential-backoff-with-jitter delay when the server sends one.
+- Exhausting every attempt raises `Butler::Errors::RetryExhausted`; running
+  out of `deadline:` instead raises `Butler::Errors::TimeoutError`.
+
+### Circuit breaker
+
+```ruby
+client = Butler::Client.new(
+  circuit_breaker: { enabled: true, scope: :host, failure_threshold: 5, recovery_timeout: 30 },
+)
+```
+
+A `CLOSED -> OPEN -> HALF_OPEN -> CLOSED/OPEN` state machine, scoped
+per-host by default (`scope: :client` shares one breaker across every host
+a client talks to instead). Opens after `failure_threshold` consecutive
+failures — a raised connection-level error, *or* a 5xx response returned
+normally (Butler doesn't raise on error responses unless
+`raise_on_error: true`, but the breaker still counts them) — stays open
+for `recovery_timeout` seconds, then allows exactly one probe request
+through; a successful probe closes it, a failed one reopens it. An open
+circuit raises `Butler::Errors::CircuitOpen` immediately, without
+attempting the network call at all.
+
+The exact rules for what counts, since they matter in production:
+
+- A **successful** call (any raised-nothing result the `failure:` check
+  doesn't flag) resets `failure_count` to `0` — the counter is consecutive
+  failures, not a rolling total.
+- From a **normally-returned response**, only a 5xx counts. 4xx (including
+  429) never counts as a circuit-breaker failure this way — Butler
+  doesn't raise on error responses unless `raise_on_error: true`, and the
+  breaker's `failure:` check is specifically `response.server_error?`.
+- Any **raised exception** always counts as a failure, regardless of
+  class — `ConnectionError`, `TimeoutError`, `TLSError` and its
+  `CertificateVerificationError` subclass, `DNSFailure`,
+  `ProtocolError`, `RetryExhausted`, all of it.
+- `Butler::Errors::CircuitOpen` itself is the one exception excluded —
+  a short-circuited call (the breaker already open) never counts toward
+  its own failure count.
+
+### Security
+
+```ruby
+client = Butler::Client.new(
+  security: {
+    verify_tls: true,                    # on by default — turning it off logs a loud warning
+    allowed_hosts: nil,                  # e.g. ["api.example.com"] to restrict to only those hosts
+    blocked_hosts: ["*.internal"],       # glob patterns
+    max_response_size: 50 * 1024 * 1024, # bytes
+    max_header_size: 64 * 1024,          # bytes
+    strip_credentials_on_redirect: true, # drop Authorization/Cookie crossing origin on a redirect
+  },
+)
+```
+
+`allowed_hosts`/`blocked_hosts` is an **outbound host policy** — a
+defense-in-depth allow/block list, not a solution to SSRF on its own. It
+does not attempt to fully solve SSRF, DNS rebinding, cloud metadata
+endpoint protection (`169.254.169.254` and friends), or
+application-level authorization — those need their own, separate
+mitigations regardless of what HTTP client you use.
+
+### Telemetry
+
+Standalone by default — no ActiveSupport required:
+
+```ruby
+Butler::Telemetry::Instrumentation.subscribe(:request) do |payload|
+  # payload is built from an explicit allow-list: method, host, port,
+  # status, duration, attempt, error — never request/response bodies or
+  # Authorization/Cookie/Set-Cookie headers.
+  StatsD.timing("http.request", payload[:duration] * 1000, tags: ["host:#{payload[:host]}"])
+end
+```
+
+If `opentelemetry-api` is already loaded by your application, Butler wraps
+each request in a real span automatically — nothing to configure —
+following the HTTP semantic-convention attribute names implemented by
+`Telemetry::OpenTelemetryBridge` (`http.request.method`,
+`server.address`, `server.port`, `http.response.status_code`,
+`network.protocol.name/version`, `error.type` as of this version;
+semantic conventions do change over time upstream, so treat that list as
+this version's implementation, not a permanent guarantee — see
+`lib/butler/telemetry/open_telemetry_bridge.rb` for the exact current
+set). If ActiveSupport is loaded, the same
+`instrument(:request, ...)` call also fires as an
+`ActiveSupport::Notifications` event (`"butler.request"`), so a Rails app's
+existing log subscribers see it too.
+
+### Middleware
+
+```ruby
+class RequestIdMiddleware
+  def call(context, next_middleware)
+    context.request.headers["X-Request-Id"] = SecureRandom.uuid
+    next_middleware.call(context)
+  end
+end
+
+client.use(RequestIdMiddleware.new)
+```
+
+Runs between `TelemetryMiddleware` and the built-in
+`Timeout`/`CircuitBreaker`/`Retry` middlewares — third-party extensions can
+inspect, modify, short-circuit, observe, or transform a request without
+core resilience/security behavior ever needing to be expressed as
+middleware itself.
+
+### Testing — no WebMock/VCR needed
+
+```ruby
+Butler::Testing.stub("GET", "https://api.example.com/users/1", status: 200, json: { id: 1 })
+
+# or the builder form, path-relative to whatever base_url the client under test uses:
+Butler::Testing.stub_request(:get, "/users/1").to_return(status: 200, json: { id: 1 })
+Butler::Testing.stub_request(:get, "/flaky").to_raise(Butler::Errors::ConnectionError)
+
+# reset between tests, e.g. in an after(:each)/teardown hook — stubs are
+# process-wide, not scoped to one Client instance:
+Butler::Testing.reset!
+```
+
+Stubs are matched at the transport boundary (`Butler::Testing::FakeTransport`
+implements the exact same two-method contract as the real transport), so a
+stubbed response or a stubbed exception still flows through
+retry/circuit-breaker/telemetry exactly as a real one would — a stubbed 503
+genuinely exercises `RetryMiddleware`, not a shortcut around it.
+
+### Rails
+
+Butler works without Rails and does not depend on it — `require "butler"`
+is standalone; `Butler::Rails::Railtie` only loads if
+`defined?(Rails::Railtie)` is already true. When Rails is present, Butler
+integrates with `Rails.logger`, publishes a `"butler.request"`
+`ActiveSupport::Notifications` event automatically, and resets pooled
+connections after a Puma (or any `preload_app!`) fork so a forked worker
+never inherits the parent's live reactor state.
+
+The usual shape: configure global defaults once in an initializer, then
+build one scoped `Butler::Client` per external API rather than sharing a
+single client across every integration. Full worked example — initializer,
+a service-object pattern, subscribing to `"butler.request"` correctly (and
+a real subtlety around *where* the event's duration actually lives) — in
+[docs/rails.md](docs/rails.md).
+
+### Errors
+
+Every error Butler raises descends from `Butler::Errors::Error`:
+
+```
+Error
+├── ConfigurationError
+├── RequestError
+│   ├── TooManyRedirectsError
+│   └── HostNotAllowed
+├── TransportError
+│   ├── ConnectionError
+│   ├── TimeoutError
+│   ├── TLSError
+│   │   └── CertificateVerificationError  (never retried — see Retries above)
+│   ├── DNSFailure
+│   └── ProtocolError
+├── HTTPError            (only raised with raise_on_error: true; carries .response)
+│   ├── ClientError       (4xx)
+│   └── ServerError       (5xx)
+├── RetryExhausted
+├── CircuitOpen
+├── Cancelled
+└── LimitExceeded
+```
+
+`rescue Butler::Errors::Error` is always a safe top-level catch-all for
+"something about this HTTP call failed."
+
+## Configuration reference
+
+Set per-client (`Butler::Client.new(**options)`) or as a process-wide
+default every new client starts from (`Butler.configure { |c| ... }`):
+
+```ruby
+Butler.configure do |config|
+  config.connect_timeout = 5
+  config.retry.max_attempts = 3
+end
+```
+
+| Option | Default | Description |
+| --- | --- | --- |
+| `base_url` | `nil` | Prefixed onto every relative path |
+| `default_headers` / `headers:` | `{}` | Sent on every request; per-request `headers:` override matching keys |
+| `user_agent` | `"Butler/<version>"` | Sent unless a request already sets its own `User-Agent` |
+| `connect_timeout` | `5` | Seconds; passed straight through to the connection endpoint |
+| `read_timeout` / `write_timeout` | `10` / `10` | Seconds; the larger of the two becomes the per-attempt round-trip budget (see `request_timeout`) |
+| `request_timeout` | `nil` | Seconds; an explicit override of the per-attempt round-trip budget, taking priority over `read_timeout`/`write_timeout` |
+| `deadline` | `nil` (unbounded) | Total wall-clock seconds for one call, across every retry/redirect — always the final cap, however the per-attempt budget above was derived |
+| `follow_redirects` | `true` | |
+| `max_redirects` | `5` | |
+| `http_version` | `:auto` | `:auto` automatically negotiates the best protocol for the origin — HTTP/2 where the server supports it, HTTP/1.1 otherwise — by offering both via ALPN and letting the server choose (`Security::TLS.alpn_protocols_for`, verified live against real HTTP/2 servers). `:http1`/`:http2` force one specifically. Also settable per call: `client.get(path, http_version: :http1)` — see [Requests](#requests) |
+| `proxy` | `nil` | Proxy URL |
+| `raise_on_error` | `false` | Raise `ClientError`/`ServerError` on 4xx/5xx instead of returning the response |
+| `retry.max_attempts` | `2` | |
+| `retry.retryable_status_codes` | `[408,425,429,500,502,503,504]` | |
+| `retry.base_delay` / `retry.max_delay` | `0.1` / `5.0` | Seconds |
+| `retry.jitter` | `true` | Equal-jitter (delay scaled by a random factor in `[0.5, 1.0)`) |
+| `circuit_breaker.enabled` | `true` | |
+| `circuit_breaker.scope` | `:host` | or `:client`, to share one breaker across every host |
+| `circuit_breaker.failure_threshold` | `5` | |
+| `circuit_breaker.recovery_timeout` | `30` | Seconds before a half-open probe is allowed |
+| `circuit_breaker.max_tracked_hosts` | `256` | LRU-bounded so fanning out to many hosts can't grow this unboundedly |
+| `security.verify_tls` | `true` | |
+| `security.allowed_hosts` / `security.blocked_hosts` | `nil` / `[]` | Glob patterns |
+| `security.max_response_size` | `50 MiB` | Bytes |
+| `security.max_header_size` | `64 KiB` | Bytes |
+| `security.strip_credentials_on_redirect` | `true` | |
+| `pool.max_connections` | `100` | Distinct origins kept warm at once (LRU-evicted past this) |
+| `pool.idle_timeout` | `60` | Seconds a pooled connection may sit unused before it's rebuilt rather than reused |
+| `telemetry.enabled` | `true` | |
+| `telemetry.opentelemetry` | `:auto` | Spans are emitted automatically whenever `opentelemetry-api` is already loaded |
+| `telemetry.logger` | `nil` (falls back to `Logger.new($stdout)`, or `Rails.logger` under Rails) | |
+
+## Benchmarks
+
+```
+ruby benchmarks/sequential_vs_concurrent.rb   # Net::HTTP vs Butler, sequential vs concurrent
+ruby benchmarks/concurrency.rb                # how wall-clock time scales from 10 to 250 (configurable) concurrent requests
+ruby benchmarks/allocations.rb                # objects allocated per request
+ruby benchmarks/memory.rb                     # RSS growth over sustained use
+ruby benchmarks/comparison.rb                 # client × sequential/concurrent matrix, plus Faraday/Excon/HTTParty if installed
+SERVER_URL=https://localhost:9292 ruby benchmarks/http1_vs_http2.rb   # HTTP/1.1 pool vs HTTP/2 multiplexing
+```
+
+Full methodology, expected shapes, and how to read each script's output are
+in [benchmarks/README.md](benchmarks/README.md). Short version: every script
+except `http1_vs_http2.rb` runs against a local server with simulated,
+fixed per-request latency, specifically so what's measured is Butler's own
+overhead — not a particular network's variance on a particular day.
+**Run them yourself** before quoting any number externally; nothing here
+substitutes for load-testing your own upstream. Benchmarks are
+informational, not guarantees — results depend on Ruby version,
+scheduler, TLS/protocol version, concurrency, payload size, upstream
+latency, CPU, and network conditions, all of which will differ from
+whatever machine produced the numbers in
+[benchmarks/README.md](benchmarks/README.md).
+
+## What's not here yet
+
+This is a substantial rearchitecture (see
+[docs/architecture.md](docs/architecture.md)), not the full roadmap from
+the design doc it's based on. Deliberately deferred: a full OpenTelemetry
+semantic-convention compliance audit, Sorbet RBI (RBS type signatures are
+included), a dedicated external security audit, and long-run
+(24h/1M-request) soak testing.
+
+**HTTP/1.1 + HTTP/2 today. HTTP/3 architecture-ready.** HTTP/3/QUIC is
+early, internal-only groundwork — packet-level QUIC crypto
+(`lib/butler/quic/`), not a usable transport. There is no
+`http_version: :http3`, no `Transport::QUIC`, nothing reachable from
+`Butler::Client` at all yet; requesting HTTP/3 today still just gets you
+`:auto`'s existing HTTP/2-vs-HTTP/1.1 ALPN choice. See
+[docs/architecture.md](docs/architecture.md#http3-quic)
+for what exists, what doesn't, and the security posture of what's there
+(short version: hand-rolled, **not** security-audited, and — once it is
+wired up — never silently reachable via `:auto`, only via an explicit
+`http_version: :http3`).
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies (or
+just `bundle install`). Run `rake test` to run the test suite — most of it
+spins up a real local TCP server rather than mocking anything, so pooling,
+retries, redirects, and timeouts are exercised against actual socket
+behavior, not stubbed-out doubles. `bin/console` starts an IRB session with
+Butler already loaded.
+
+## Contributing
+
+Bug reports and pull requests are welcome at
+https://github.com/ramlaxmanyadav/butler-http.
+
+## License
+
+The gem is available as open source under the terms of the
+[MIT License](https://opensource.org/licenses/MIT).

data/Rakefile

@@ -0,0 +1,10 @@
+require "bundler/gem_tasks"
+require "rake/testtask"
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << "test"
+  t.libs << "lib"
+  t.test_files = FileList["test/**/*_test.rb"]
+end
+
+task default: :test