httpx-hedged 0.2.0__py3-none-any.whl

httpx_hedged-0.2.0.dist-info/METADATA

@@ -0,0 +1,300 @@
+Metadata-Version: 2.4
+Name: httpx-hedged
+Version: 0.2.0
+Summary: Adaptive, per-endpoint request hedging transport for httpx. Learns latency percentiles per endpoint with DDSketch, caps hedge rate with a token bucket, and stops hedging under a host/endpoint circuit breaker.
+License-File: LICENSE
+Requires-Python: >=3.13
+Requires-Dist: httpx>=0.27.0
+Description-Content-Type: text/markdown
+
+# httpx-hedged
+
+An [httpx](https://www.python-httpx.org/) transport that adds
+adaptive, per-endpoint request hedging: fire a backup request when the
+primary is running slow, take whichever finishes first, cancel the
+loser. Based on Google's [The Tail at
+Scale](https://research.google/pubs/pub40801/) and modeled heavily on
+[hedge-python](https://github.com/sunhailin-Leo/hedge-python).
+
+## Quick start
+
+```python
+import asyncio
+import httpx
+from httpx_hedged import HedgedTransport
+
+async def main():
+    transport = HedgedTransport()
+    async with httpx.AsyncClient(transport=transport) as client:
+        response = await client.get("https://api.example.com/data")
+        print(response.json())
+
+asyncio.run(main())
+```
+
+With no configuration, `HedgedTransport` learns a p90 latency estimate per
+host (via a [DDSketch](https://arxiv.org/abs/2004.08604) quantile sketch)
+and fires a hedge request whenever the primary exceeds it. 
+
+## Why per-endpoint?
+
+A single host can host wildly different endpoints. Learning one latency
+distribution per *host* -- rather than per endpoint -- means a handful of
+calls to a slow endpoint skew the hedge trigger for a fast one sharing the
+same host:
+
+```
+GET /api/v1/fast-lookup   median 10ms,  high RPS
+GET /api/v1/bulk-export   median 900ms, low RPS
+```
+
+`HedgedTransport` lets you register per-endpoint config up front. Each
+registered endpoint gets its own latency sketch, rate estimate, and hedge
+budget -- all still funneled through a **single inner transport and
+connection pool**, unlike using `httpx` `mounts={}`, which would mean one
+connection pool per pattern:
+
+```python
+from httpx_hedged import EndpointConfig, HedgedTransport
+
+transport = HedgedTransport()
+transport.register("GET", "/api/v1/fast-lookup", EndpointConfig(percentile=0.90))
+transport.register("GET", "/api/v1/bulk-export", EndpointConfig(percentile=0.90))
+```
+
+Requests that don't match a registered pattern fall back to a default
+config, tracked per host (the same behavior as hedging with no registered
+endpoints at all).
+
+Route patterns may contain `{name}` placeholders or a bare `*` for a single
+path segment, e.g. `/api/v1/users/{id}`. Patterns are matched in
+registration order -- register more specific patterns first.
+
+## Hardcoded vs. adaptive delay
+
+Most endpoints should hedge adaptively, against their own learned
+percentile:
+
+```python
+transport.register("GET", "/api/v1/search", EndpointConfig(percentile=0.95))
+```
+
+For an endpoint where you already know the right delay -- or want
+deterministic behavior without a warmup period -- hardcode it instead:
+
+```python
+transport.register("GET", "/api/v1/health", EndpointConfig(hedge_delay=0.05))
+```
+
+A hardcoded endpoint still records latency into its sketch for
+observability; it just isn't consulted for the hedge-delay decision.
+
+## Explicit endpoint override
+
+Auto-matching not precise enough for a particular call site (or you'd
+rather not register a pattern)? Tag the request directly -- this bypasses
+pattern matching entirely:
+
+```python
+await client.get(
+    "https://api.example.com/some/path",
+    extensions={"hedge_endpoint": "pinned-name"},
+)
+```
+
+The name must already be registered (`register(..., name="pinned-name")`);
+an unknown name raises `UnknownHedgeEndpointError` rather than silently
+falling back, so typos fail loudly.
+
+## Configuration
+
+### `HedgeConfig` (transport-wide default)
+
+| Parameter | Type | Default | Description |
+|---|---|---|---|
+| `percentile` | `float` | `0.90` | Sketch quantile used as the hedge trigger |
+| `budget_percent` | `float` | `10.0` | Max hedge rate as percent of total traffic |
+| `estimated_rps` | `float \| None` | `None` | Pin the expected requests/sec, or leave `None` to auto-estimate from observed traffic |
+| `rps_window_duration` | `float` | `10.0` | Rolling window (seconds) for RPS auto-estimation |
+| `min_delay` | `float` | `0.001` | Floor on the hedge delay in seconds |
+| `warmup_requests` | `int` | `20` | Requests using a fixed delay before the sketch is trusted |
+| `warmup_delay` | `float` | `0.01` | Fixed hedge delay during warmup, in seconds |
+| `window_duration` | `float` | `30.0` | Latency sketch rotation interval in seconds |
+| `circuit_breaker` | `CircuitBreakerConfig` | see below | Health circuit-breaker configuration |
+
+### `EndpointConfig` (per-registered-endpoint override)
+
+Every field mirrors `HedgeConfig` and defaults to `None`, meaning "inherit
+the transport default." One extra field:
+
+| Parameter | Type | Default | Description |
+|---|---|---|---|
+| `hedge_delay` | `float \| None` | `None` | Hardcode the hedge delay for this endpoint, skipping the sketch for the decision |
+
+### `CircuitBreakerConfig`
+
+| Parameter | Type | Default | Description |
+|---|---|---|---|
+| `error_rate_threshold` | `float` | `0.5` | Failure fraction that trips the breaker open |
+| `min_samples` | `int` | `20` | Minimum samples in-window before the breaker can trip |
+| `window_duration` | `float` | `30.0` | Rolling window (seconds) for the error-rate estimate |
+| `cooldown` | `float` | `30.0` | Seconds the breaker stays open before a half-open trial |
+| `half_open_max_trial` | `int` | `5` | Trial requests allowed through while half-open |
+| `treat_5xx_as_failure` | `bool` | `True` | Whether an HTTP 5xx response counts as a failure |
+
+All three config classes validate their fields at construction time (e.g.
+`percentile` must be strictly between 0 and 1, delays and windows must be
+non-negative/positive) and raise `ValueError` immediately on a bad value,
+rather than silently misbehaving later.
+
+## How it works
+
+### Race and cancel
+
+```
+              ┌─ primary  ─────────── ✓ (fast) ──→ return
+request ──────┤
+              └─ hedge fires after estimated delay ─── ✗ (cancelled)
+```
+
+Only idempotent methods (`GET`, `HEAD`, `OPTIONS`) are ever hedged, to avoid
+duplicating side effects. A request with a body is also never hedged, even
+if the method is idempotent -- the primary and hedge send the same
+`httpx.Request` object, and a body backed by a one-shot stream can't be
+safely read twice.
+
+### DDSketch quantile estimator
+
+Each tracked key (an endpoint, or the per-host fallback) gets its own
+sliding-window DDSketch pair that rotates every `window_duration` seconds.
+DDSketch gives relative-error quantile guarantees regardless of the
+underlying latency distribution's shape.
+
+### Token bucket budget
+
+Hedges are rate-limited by a token bucket refilling at
+`estimated_rps * budget_percent / 100` tokens/second, per key. During a
+genuine outage the bucket drains and hedging stops automatically,
+preventing the load-doubling spiral that would deepen the incident. By
+default the RPS feeding this calculation is estimated automatically from
+observed traffic per key, rather than requiring a manual guess per
+endpoint.
+
+## Circuit breaker
+
+A closed / open / half-open circuit breaker tracks request success/failure
+at **two independent tiers**: one breaker per host, one breaker per
+endpoint key. Either tripping open suppresses hedging for its scope -- a
+host-level trip disables hedging for every endpoint on that host, while an
+endpoint-level trip disables hedging only for that one endpoint.
+
+Crucially, the breaker **only ever suppresses the hedge request**. The
+primary request is always sent, and its result or exception is always
+returned to the caller normally -- this is not a request-blocking circuit
+breaker, so hedging can't pile extra load onto a backend that's already
+struggling.
+
+```
+CLOSED ──(error rate ≥ threshold, samples ≥ min_samples)──▶ OPEN
+OPEN ──(cooldown elapsed)──▶ HALF_OPEN
+HALF_OPEN ──(trial requests mostly succeed)──▶ CLOSED
+HALF_OPEN ──(trial requests mostly fail)────▶ OPEN
+```
+
+Note: health is recorded from the *winning* task's outcome only -- a
+cancelled loser's real outcome is unknowable, and losers are cancelled
+deliberately (not doing so would defeat the point of reducing load on a
+struggling backend).
+
+## Observability
+
+### Polling stats and health snapshots
+
+```python
+transport = HedgedTransport()
+
+# ... after running some traffic ...
+for key, snap in transport.stats.all_snapshots().items():
+    print(key, snap)
+
+print(transport.stats.global_snapshot())
+print(transport.health.host_state("api.example.com"))
+```
+
+`StatsSnapshot` reports `total_requests`, `hedged_requests`, `hedge_wins`,
+`primary_wins`, `budget_exhausted`, `warmup_requests`, `circuit_blocked`,
+and `errors` per key, plus a global aggregate.
+
+### Push-based hooks: metrics and alerting
+
+Polling snapshots works for dashboards, but alerting on a circuit-breaker
+trip and emitting a metric on every hedge fire are both things you want to
+happen *at the moment they occur*, not on the next poll. `HedgedTransport`
+takes two optional callbacks for exactly this:
+
+```python
+import logging
+
+import httpx
+from httpx_hedged import EndpointConfig, HedgeConfig, HedgedTransport
+
+logger = logging.getLogger("myapp.hedging")
+
+
+def emit_hedge_fired_metric(key: str) -> None:
+    statsd_client.incr("http.hedge.fired", tags=[f"endpoint:{key}"])
+
+
+def alert_on_circuit_open(scope: str, key: str) -> None:
+    # scope is "host" or "endpoint"; key is the host name or endpoint key
+    # that tripped. Fires exactly once per OPEN transition.
+    logger.error(
+        "hedging circuit breaker OPEN: %s=%s is unhealthy, hedging suspended", scope, key
+    )
+
+
+transport = HedgedTransport(
+    default_config=HedgeConfig(),
+    on_hedge_fired=emit_hedge_fired_metric,
+    on_circuit_open=alert_on_circuit_open,
+)
+
+transport.register("GET", "/api/v1/fast-lookup", EndpointConfig(percentile=0.90))
+transport.register("GET", "/api/v1/bulk-export", EndpointConfig(percentile=0.90))
+
+async with httpx.AsyncClient(transport=transport) as client:
+    ...
+```
+
+`on_hedge_fired` is called with the key each time a hedge request is
+actually launched -- after the idempotency, circuit-breaker, and budget
+gates have all passed, so it only fires for hedges that were genuinely
+sent. `on_circuit_open` is called once per OPEN transition (not on every
+suppressed hedge while it stays open), so it's safe to wire straight into
+an alerting/paging pipeline without flooding it.
+
+Both callbacks run synchronously on the request path, so keep them fast
+(increment a counter, log a line) -- don't do network I/O in them directly.
+
+## Relationship to hedge-python
+
+This library is modeled heavily on
+[hedge-python](https://github.com/sunhailin-Leo/hedge-python), which
+pioneered the DDSketch-based adaptive-hedging approach this project
+borrows. hedge-python keys its sketch per host, which works well for a
+single-endpoint-per-host use case; this project exists to add per-endpoint
+tracking on top of the same core idea, plus a health circuit breaker,
+for services that expose many differently-shaped endpoints on one host. See
+the filed [upstream issue](https://github.com/sunhailin-Leo/hedge-python/issues/2)
+for the motivating scenario.
+
+## References
+
+- [The Tail at Scale](https://research.google/pubs/pub40801/) -- Google's paper on tail latency
+- [DDSketch: A fast and fully-mergeable quantile sketch with relative-error guarantees](https://arxiv.org/abs/2004.08604) -- Masson et al., VLDB 2019
+- [hedge-python](https://github.com/sunhailin-Leo/hedge-python) -- the project this one is modeled after
+- [httpx documentation](https://www.python-httpx.org/)
+
+## License
+
+BSD 3-Clause License. See [LICENSE](LICENSE) file for details.

httpx_hedged-0.2.0.dist-info/RECORD

@@ -0,0 +1,19 @@
+httpx_hedged/__init__.py,sha256=qgb6ZGlin6loceLjokpQZjYwAB81ifMNgUmVHatIi3A,1324
+httpx_hedged/_bounded.py,sha256=rTt2L1pg92NUVD4BaR3N2hRR7Nc9VrS2pXxfVpdvr9M,1588
+httpx_hedged/_config.py,sha256=tLgauUBM-bEhdg_4k-6r01KVYLTVh4iYcHvupypIBnA,8651
+httpx_hedged/_health.py,sha256=n0TMzmsngyTQlkgnc2pnAk-nDOZGPGTKhQR5-6tGpuw,10353
+httpx_hedged/_matcher.py,sha256=k9RK0f8OWB4VdzPjHszRjAHqy2KbesmhaFx_w7YvJCU,4385
+httpx_hedged/_rate.py,sha256=tab9R_wD-7p-zo9WCRTFyPPIn3kSbp1iJoFru-HgxJo,2888
+httpx_hedged/_rotation.py,sha256=mes59ls0dCFEort2L5DTt1s_-mmi2lSPQyXW0RyUwOY,1526
+httpx_hedged/_scheduler.py,sha256=qR7kVjsX8gpy3HKw2rQFRguhPDyseY_mBCDweN8llHw,12049
+httpx_hedged/_stats.py,sha256=mDhenR_zp3So4lC4eO8a8RLfXOgbtWJ5NF6EdXaKMz8,5083
+httpx_hedged/transport.py,sha256=8pmREmmLA6FVWtH9yjIrnjOCPTvf9Z53dP4CFZInzv8,6999
+httpx_hedged/budget/__init__.py,sha256=boLCL5U9IKycVE688dpXuss0dHGpjWhVacrWEe4794w,158
+httpx_hedged/budget/_token_bucket.py,sha256=zB_S6lgjcNTmFQjPD6CbPWdlvcBs7eKKTPcLJHbNkkM,1887
+httpx_hedged/sketch/__init__.py,sha256=5Wjre__zLRAN6ZKowlgup0Px2QqtEPk1cgJtKeQAplQ,266
+httpx_hedged/sketch/_ddsketch.py,sha256=HsnLA8hyT2dDS6t6Nd1MvixMrwyt-0b4Jte2yK2bU2A,6336
+httpx_hedged/sketch/_windowed.py,sha256=Hy0PjmJlTzsoNkLhMVULSZ6P5jENkENQuBXtevBowdM,3455
+httpx_hedged-0.2.0.dist-info/METADATA,sha256=w9VALmhLLd1sUvEt0W_35zuVMruETNSOgOGWeA34vNg,11889
+httpx_hedged-0.2.0.dist-info/WHEEL,sha256=lCkmxWfQsSc9CfIClYeavTdQeEX2toPqufh9gI35EQA,87
+httpx_hedged-0.2.0.dist-info/licenses/LICENSE,sha256=VUmTrbhUTXvjVSZdZhR9TrQ24lKPI1RsO_5AIDeJO9k,1499
+httpx_hedged-0.2.0.dist-info/RECORD,,

httpx_hedged-0.2.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.31.0
+Root-Is-Purelib: true
+Tag: py3-none-any

httpx_hedged-0.2.0.dist-info/licenses/LICENSE

@@ -0,0 +1,28 @@
+BSD 3-Clause License
+
+Copyright (c) 2025, Brendan Fahy
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its
+   contributors may be used to endorse or promote products derived from
+   this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.