021-python-clients 0.2.0__py3-none-any.whl → 0.3.0__py3-none-any.whl

{021_python_clients-0.2.0.dist-info → 021_python_clients-0.3.0.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: 021-python-clients
-Version: 0.2.0
+Version: 0.3.0
 Summary: Observed clients for Redis, SQS, QStash, S3, Vercel Blob, HTTP, email and KMS, built on 021-python-observability
 Project-URL: Repository, https://github.com/from021-io/python-clients
 Project-URL: Changelog, https://github.com/from021-io/python-clients/blob/main/CHANGELOG.md
@@ -48,8 +48,8 @@ uv add "021-python-clients[redis,aws,httpx2,vercel-blob]"   # pick the extras yo
 
 | Module | Extra | What it gives |
 | --- | --- | --- |
-| `clients_021.http` | none | `HttpClient`: one httpx client per dependency, timeouts, retries on idempotent methods |
-| `clients_021.httpx2` | `httpx2` | `ObservedTransport` for SDKs built on httpx2 (Anthropic, OpenAI) |
+| `clients_021.http.httpx` | none | `HttpClient`: buffered requests, streaming and full-body telemetry; native `ObservedTransport` |
+| `clients_021.http.httpx2` | `httpx2` | `ObservedTransport` for SDKs built on httpx2 (Anthropic, OpenAI) |
 | `clients_021.cache.redis` | `redis` | `RedisClient`: JSON with TTL, one-time reads, windowed counters, ping |
 | `clients_021.queue.qstash` | none | `QStashClient`: JSON publishing, deduplication, delivery options and forwarded trace context |
 | `clients_021.queue.sqs` | `aws` | `SqsClient`: send, receive, delete, visibility, trace carried in the message |
@@ -66,10 +66,12 @@ Every client takes the service's `Observability` (from `setup_observability`) an
 
 Providers with the same purpose live under one capability package: `email/` contains
 Resend and SES; `queue/` contains SQS and QStash; `storage/` contains S3 and Vercel Blob; `cache/`
-contains Redis; `crypto/` contains KMS. HTTP transports remain at the root. Import providers
-explicitly:
+contains Redis; `crypto/` contains KMS. `http/` contains the httpx client and httpx2
+transport. Import providers explicitly:
 
 ```python
+from clients_021.http import HttpClient
+from clients_021.http.httpx2 import ObservedTransport  # requires the httpx2 extra
 from clients_021.queue.sqs import SqsClient
 from clients_021.queue.qstash import QStashClient
 from clients_021.storage.s3 import S3Client
@@ -85,7 +87,9 @@ does not load the optional AWS SDK.
 
 The 0.1.0 imports `clients_021.redis`, `clients_021.s3` and `clients_021.kms` remain
 compatible aliases for the same classes and result types. Importing a capability package
-does not load its optional SDKs.
+does not load its optional SDKs. `clients_021.HttpClient` and `clients_021.http` keep their
+published exports; `clients_021.httpx2.ObservedTransport` remains an alias for
+`clients_021.http.httpx2.ObservedTransport`.
 
 ## What every call gets
 
@@ -97,7 +101,8 @@ does not load its optional SDKs.
   `crypto.client.operation.*` (KMS), `http.client.request.*` (HTTP, and the HTTP request
   under QStash and Resend). Labels are bounded; see [CONTRACT.md](CONTRACT.md).
 - Redis, SQS, QStash, S3, Vercel Blob, email and KMS emit a `dependency.call_failed` warning on failure,
-  with dependency, operation and error type only. HTTP transports emit spans and metrics.
+  with dependency, operation and error type only. HTTP/httpx2 also log failed attempts,
+  including body failures and cancellation; QStash/Resend suppress the nested HTTP log.
 - `DependencyUnavailable(dependency, operation, reason)` when the service timed out,
   refused the connection, throttled or failed with a 5xx. Map it to a 503. Errors that mean
   the request is wrong (access denied, a missing bucket, a wrong Redis type) keep their
@@ -290,21 +295,58 @@ ciphertext keep their botocore type (they are bugs or attacks, not outages); a t
 envelope raises `cryptography`'s `InvalidTag`. The envelope layout is in
 [CONTRACT.md](CONTRACT.md).
 
+### HTTP bodies and streaming
+
+```python
+from clients_021.http import HttpClient
+
+http = HttpClient(obs, timeout_seconds=10)
+try:
+    async with http.stream("GET", download_url, dependency="documents") as response:
+        response.raise_for_status()
+        async for chunk in response.aiter_bytes():
+            await write_chunk(chunk)
+finally:
+    await http.aclose()
+```
+
+These async clients run on asyncio. `request()` returns a buffered `HttpResponse`; `stream()` yields a native httpx response
+without buffering and accepts an async iterable as upload `content`. Streaming sends once,
+even for GET. The buffered client's configured idempotent retries are unchanged. Close the
+response context before closing the client. For SDKs using native httpx, inject
+`clients_021.http.httpx.ObservedTransport`; httpx2 SDKs use the transport below.
+
+Each HTTP attempt emits one span, counter and duration after body EOF and connection close,
+or early close. This includes upload, download, pauses between chunks and cleanup. Body
+errors after 200 headers are recorded as errors. `http.response.body.complete` distinguishes
+EOF from partial/unread close; it does not mean the status or cleanup succeeded. Intentional
+partial close is INFO `dependency.stream_closed`; exceptions, cancellation and HTTP >= 400
+produce WARNING `dependency.call_failed`. No success log, URL, header, body or provider
+error message is emitted. `log_failures=False` disables attempt logs when an enclosing
+operation owns the warning, as QStash and Resend do.
+
+These duration semantics and the completion label change the 0.2.x telemetry contract;
+see [migration notes](CONTRACT.md#migration-from-02x) before updating dashboards or alerts.
+
 ### SDKs on httpx2
 
 For an SDK on httpx2:
 
 ```python
 from anthropic import AsyncAnthropic, DefaultAsyncHttpxClient
-from clients_021.httpx2 import ObservedTransport
+from clients_021.http.httpx2 import ObservedTransport
 
 transport = ObservedTransport(obs, dependency="anthropic")
 claude = AsyncAnthropic(http_client=DefaultAsyncHttpxClient(transport=transport))
 ```
 
-It does not send `traceparent` to third-party APIs unless `propagate=True`.
+It does not inject trace context into third-party API requests unless `propagate=True`.
 The transport preserves SDK exceptions so the SDK retains control of retries and error
-handling; it does not convert them to `DependencyUnavailable`.
+handling; it does not convert them to `DependencyUnavailable`. Its span stays open through
+incremental response reads and close. Use the SDK's stream context manager, or close manual
+responses explicitly; abandoning a stream leaves both its connection and span open. Caller
+processing keeps its own trace context, including when streams are interleaved or consumed
+in another task.
 
 ## Service integration
 
@@ -338,7 +380,7 @@ async with AsyncExitStack() as resources:
 
 Retry policies are explicit:
 
-- HTTP retries GET, HEAD, OPTIONS, PUT and DELETE on network failures, timeouts,
+- Buffered HTTP (`request`) retries GET, HEAD, OPTIONS, PUT and DELETE on network failures, timeouts,
   remote protocol/proxy connection failures and 502/503/504. `retries=2` means at most
   three attempts.
   POST and PATCH run once. Invalid URLs/protocols and invalid request construction keep
@@ -372,10 +414,11 @@ stop an SDK request already running in its thread. Timeouts are per SDK/network
 not an overall deadline. Limit consumer concurrency and let in-flight work finish before
 closing the clients. Redis window counters require Redis server 7 or newer (`EXPIRE NX`).
 
-HTTP transport spans end when response headers arrive. Body downloads and SDK streaming
-are outside that span; instrument the enclosing service/SDK operation when you need total
-latency or failures while consuming the body. S3 `get` buffers the entire object in memory;
-use a dedicated streaming adapter for large files.
+HTTP spans now include network body consumption and cleanup. The SDK/application still
+owns content decoding, JSON/SSE interpretation and processing errors outside raw-stream I/O.
+Timeouts remain per I/O, not a total stream deadline. Cancellation waits for shielded body
+cleanup before propagating. Close all responses and consumers before shutdown/flush.
+S3 `get` buffers the entire object in memory; use a dedicated streaming adapter for large files.
 
 ## Development
 

{021_python_clients-0.2.0.dist-info → 021_python_clients-0.3.0.dist-info}/RECORD

@@ -1,9 +1,8 @@
-clients_021/__init__.py,sha256=sF_9ZsxPumQbZ-9iCXoLP8JcU_tITDT6Xk1mXF9PVuQ,816
+clients_021/__init__.py,sha256=to7x0AyCtQxhss2a4YGj1GMJq8NyRQdKp-DICQrBbzs,821
 clients_021/_aws.py,sha256=sj2kCfod-vz9J93UyeVzNVLGK3M25yipKdW15KyIkw4,1722
 clients_021/_observe.py,sha256=Xjg5Qck-e8Gt5NQHOND6nzr7DvTn46PmBdccubiD3Dk,3792
 clients_021/errors.py,sha256=GS_fN8mjr7MCOkhqPhH6LweIvQ20Vd29U-aZY8zBeJQ,972
-clients_021/http.py,sha256=-AsGLfngd-eEJnTuZEPdVBI9myYiuggmnrvjQjaMpT8,5045
-clients_021/httpx2.py,sha256=w248IcEtDZtsEDkkvj4Bpfl7MU7IJ08EHqHWhryC_3k,3880
+clients_021/httpx2.py,sha256=rNbNGJuPJPZUwVbNN5L88IF0g3Tq6EHqj_uFmJ3RmLQ,161
 clients_021/kms.py,sha256=YcZXURpqIZTkjcP0fpWMD6X_adMAYwN7zAJNhUImR4s,183
 clients_021/py.typed,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
 clients_021/redis.py,sha256=xC3tji_ldJAwaBntkZTuPhNHLIzl3F3nDzkikMrbJCc,169
@@ -14,15 +13,19 @@ clients_021/crypto/__init__.py,sha256=qESFBPQBKcHhdFXqeJzSyEP41yveG7rFrxnpITmqlN
 clients_021/crypto/kms.py,sha256=My5Xyg0bOiT9YHYWojQSszpREEh3bSYMMB26wy6fNEI,7261
 clients_021/email/__init__.py,sha256=ivmdu9hhNDwPqBn3G7uNG0Zpc3DEMnlnWBVVoVQ-fQA,790
 clients_021/email/_message.py,sha256=ulTX2cXPxSVaq_5FVF0e6glePVJ7wiO5P3M4WUWN3N0,3258
-clients_021/email/resend.py,sha256=ugENkEzw9eUlYih3BZcCVQswTF1MDikPgZyuqdapCjc,5347
+clients_021/email/resend.py,sha256=1EjvEbNbbVDQs5TZfYuZrqh14io8cb5ADneNA63DQbg,5416
 clients_021/email/ses.py,sha256=0ukHvuVE0nTU5mYWRy2EaHY5v9dUwi4GSKES93pjqBg,4668
+clients_021/http/__init__.py,sha256=P4OmYruhkQfJwJzHA90FXKWg_A2rh5iOl-ES_NTMMVQ,311
+clients_021/http/_observe.py,sha256=hpTzcej9MhlvM-RgUZWicR57ulE90GmZMs5U_qFoquQ,6922
+clients_021/http/httpx.py,sha256=gvqSpO1rb6nBA5i01W1yos5mPBwLqa3-HDYwMtQXc6s,10627
+clients_021/http/httpx2.py,sha256=fmqH4bikDKZ6YY0sNBHZbocff2zU40Gpks1aoRuOGeQ,2810
 clients_021/queue/__init__.py,sha256=CesweOrXmX5cyU-mRvqu-4wLkNJI15zi_Lows6SPupY,338
-clients_021/queue/qstash.py,sha256=j3vz_S9-ed0YQZCaWwQg5wPwf8Bv5P1voNsx8WD8Zhc,8315
+clients_021/queue/qstash.py,sha256=NndlqmNkcBgs0ExGsT-3-bMAdU0skO8Djptqj8NoJAM,8335
 clients_021/queue/sqs.py,sha256=jnccgbxyglzs2mR_DsH04IisEwygDlw6s3RxR0UtMTc,6693
 clients_021/storage/__init__.py,sha256=KrJViRHBK7PkIYnagg7vyXp95SzMFp9cUCKEnMeFr84,78
 clients_021/storage/s3.py,sha256=NDtG9QzHSXvrcazTYTq1EH5VjuRd2xMMsMvaumqci4Q,5946
 clients_021/storage/vercel_blob.py,sha256=FlnoKiahqtk3PpLsGpyu_Gy87otg15i4BlJZsIB3LfM,6511
-021_python_clients-0.2.0.dist-info/METADATA,sha256=gx7Jt2wjPNVFhyUHCze-tQ5YVu-Kqrt2GwAyia6HdXs,20595
-021_python_clients-0.2.0.dist-info/WHEEL,sha256=W3fkpkm7-wf9vBI5Z-7s0eWkeM-spu78I8Neb98DeEg,87
-021_python_clients-0.2.0.dist-info/licenses/LICENSE,sha256=z8d0m5b2O9McPEK1xHG_dWgUBT6EfBDz6wA0F7xSPTA,11358
-021_python_clients-0.2.0.dist-info/RECORD,,
+021_python_clients-0.3.0.dist-info/METADATA,sha256=ewZEC_EJC4-e8ZayWgpdlnxQ0hYhWMMjmRr2KIN4iW4,23227
+021_python_clients-0.3.0.dist-info/WHEEL,sha256=W3fkpkm7-wf9vBI5Z-7s0eWkeM-spu78I8Neb98DeEg,87
+021_python_clients-0.3.0.dist-info/licenses/LICENSE,sha256=z8d0m5b2O9McPEK1xHG_dWgUBT6EfBDz6wA0F7xSPTA,11358
+021_python_clients-0.3.0.dist-info/RECORD,,

clients_021/__init__.py

@@ -3,7 +3,7 @@
 `clients_021.http`, `clients_021.queue.qstash` and `clients_021.email` (Resend) need no extras;
 `clients_021.cache.redis` needs the `redis` extra; `clients_021.queue.sqs`,
 `clients_021.storage.s3`, `clients_021.email.ses` and `clients_021.crypto.kms` the `aws` extra;
-`clients_021.httpx2` the `httpx2` extra.
+`clients_021.http.httpx2` the `httpx2` extra.
 `clients_021.storage.vercel_blob` needs the `vercel-blob` extra.
 """
 

clients_021/email/resend.py

@@ -98,7 +98,11 @@ class ResendSender:
         }
         inner = transport if transport is not None else httpx.AsyncHTTPTransport()
         self._http = HttpClient(
-            obs, timeout_seconds=timeout_seconds, retries=0, transport=_WithoutTraceHeaders(inner)
+            obs,
+            timeout_seconds=timeout_seconds,
+            retries=0,
+            log_failures=False,
+            transport=_WithoutTraceHeaders(inner),
         )
 
     async def send(self, message: EmailMessage, *, idempotency_key: str | None = None) -> str:

clients_021/http/__init__.py

@@ -0,0 +1,11 @@
+"""HTTP clients and transports; optional implementations are imported explicitly."""
+
+from clients_021.http.httpx import (
+    IDEMPOTENT_METHODS,
+    RETRY_STATUS,
+    HttpClient,
+    HttpMethod,
+    HttpResponse,
+)
+
+__all__ = ["IDEMPOTENT_METHODS", "RETRY_STATUS", "HttpClient", "HttpMethod", "HttpResponse"]

clients_021/http/_observe.py

@@ -0,0 +1,175 @@
+"""One observation per HTTP attempt, owned by its response stream until close.
+
+Never leave an OpenTelemetry context attached across a yielded chunk: consumers may
+read/close in another task, or interleave independent streams in the same task.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import sys
+import time
+from collections.abc import AsyncIterator, Callable, MutableMapping
+from contextlib import AbstractContextManager, suppress
+from typing import Protocol
+
+import structlog.contextvars
+from observability_021 import Observability, get_logger
+from opentelemetry.trace import Span, SpanKind, StatusCode, use_span
+from opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator
+
+_METHODS = frozenset({"GET", "HEAD", "OPTIONS", "POST", "PUT", "PATCH", "DELETE"})
+logger = get_logger(__name__)
+
+
+def propagate(headers: MutableMapping[str, str], *, request_id: bool = False) -> None:
+    # HTTP Headers are case insensitive. Replace the entire trace pair, including stale state.
+    headers.pop("traceparent", None)
+    headers.pop("tracestate", None)
+    carrier: dict[str, str] = {}
+    TraceContextTextMapPropagator().inject(carrier)
+    headers.update(carrier)
+    correlation = structlog.contextvars.get_contextvars().get("request_id")
+    if request_id and isinstance(correlation, str) and correlation:
+        headers["x-request-id"] = correlation
+
+
+class Observation:
+    def __init__(
+        self, obs: Observability, dependency: str, method: str, *, log_failures: bool
+    ) -> None:
+        self.obs = obs
+        self.dependency = dependency
+        self.method = method if method in _METHODS else "_OTHER"
+        self.attributes: dict[str, str | int | bool] = {
+            "dependency": dependency,
+            "http.request.method": self.method,
+        }
+        self.started = time.perf_counter()
+        self.span = obs.tracer.start_span(
+            f"HTTP {dependency}", kind=SpanKind.CLIENT, attributes=self.attributes
+        )
+        self.status_code: int | None = None
+        self.log_failures = log_failures
+        self.finished = False
+
+    def activate(self) -> AbstractContextManager[Span]:
+        return use_span(self.span, record_exception=False, set_status_on_exception=False)
+
+    def finish(self, *, complete: bool = False, error_type: str | None = None) -> None:
+        if self.finished:
+            return
+        self.finished = True
+        self.attributes["http.response.body.complete"] = complete
+        if self.status_code is not None:
+            self.attributes["http.response.status_code"] = self.status_code
+        if error_type is not None:
+            self.attributes["error.type"] = error_type
+        rejected = self.status_code is not None and self.status_code >= 400
+        if error_type is not None or rejected:
+            self.span.set_status(StatusCode.ERROR)
+        with self.activate():
+            try:
+                self.span.set_attributes(self.attributes)
+                if self.log_failures:
+                    if error_type is not None or rejected:
+                        logger.warning(
+                            "dependency.call_failed",
+                            dependency=self.dependency,
+                            operation=self.method,
+                            error_type=error_type or f"status_{self.status_code}",
+                        )
+                    elif not complete:
+                        logger.info(
+                            "dependency.stream_closed",
+                            dependency=self.dependency,
+                            operation=self.method,
+                        )
+                self.obs.client_requests.add(1, self.attributes)
+                self.obs.client_duration.record(time.perf_counter() - self.started, self.attributes)
+            finally:
+                self.span.end()
+
+
+class ByteStream(Protocol):
+    def __aiter__(self) -> AsyncIterator[bytes]: ...
+    async def aclose(self) -> None: ...
+
+
+class ObservedBody:
+    """SDK-specific subclasses only supply their AsyncByteStream marker type."""
+
+    def __init__(
+        self,
+        stream: ByteStream,
+        observation: Observation,
+        classify: Callable[[BaseException], str],
+    ) -> None:
+        self._stream = stream
+        self._observation = observation
+        self._classify = classify
+        self._iterator: AsyncIterator[bytes] | None = None
+        self._complete = False
+        self._closed = False
+        self._error: str | None = None
+        self._close_task: asyncio.Task[None] | None = None
+
+    def __aiter__(self) -> AsyncIterator[bytes]:
+        return self
+
+    def fail(self, error: BaseException) -> None:
+        if self._error is None:
+            self._error = self._classify(error)
+
+    async def __anext__(self) -> bytes:
+        if self._closed:
+            raise StopAsyncIteration
+        try:
+            with self._observation.activate():
+                if self._iterator is None:
+                    self._iterator = self._stream.__aiter__()
+                return await anext(self._iterator)
+        except StopAsyncIteration:
+            self._complete = True
+            await self.aclose()
+            raise
+        except BaseException as error:
+            self.fail(error)
+            # A cleanup failure must not replace the original read error/cancellation.
+            with suppress(BaseException):
+                await self.aclose()
+            raise
+
+    async def aclose(self) -> None:
+        if self._observation.finished:
+            return
+        self._closed = True
+        # Cancellation between chunks reaches aclose through the consumer's context manager.
+        current_error = sys.exception()
+        if isinstance(current_error, asyncio.CancelledError):
+            self.fail(current_error)
+        cancelled: asyncio.CancelledError | None = None
+        with self._observation.activate():
+            if self._close_task is None:
+                self._close_task = asyncio.create_task(self._stream.aclose())
+            try:
+                while True:
+                    try:
+                        await asyncio.shield(self._close_task)
+                        break
+                    except asyncio.CancelledError as error:
+                        self.fail(error)
+                        cancelled = cancelled or error
+                        if self._close_task.done():
+                            break
+                self._close_task.result()
+            except BaseException as error:
+                self.fail(error)
+                if cancelled is None:
+                    if isinstance(current_error, asyncio.CancelledError):
+                        raise current_error from None
+                    raise
+            finally:
+                self._observation.finish(complete=self._complete, error_type=self._error)
+            if cancelled is not None:
+                raise cancelled

clients_021/http/httpx.py

@@ -0,0 +1,280 @@
+"""Outbound HTTP: one httpx client per dependency, mandatory timeouts and bounded retries.
+
+    from clients_021.http import HttpClient
+
+    http = HttpClient(obs, timeout_seconds=10)
+    response = await http.request("GET", url, dependency="identity-provider")
+
+Every attempt records `http.client.request.*` and a CLIENT span through response body
+and stream close under the `dependency` name, with `traceparent` / `x-request-id` sent along.
+Buffered methods in IDEMPOTENT_METHODS retry on transport errors and 502/503/504 with
+backoff; POST and PATCH are sent once (send an Idempotency-Key and retry in the caller if
+the other side supports it). A transport failure after the last attempt raises
+DependencyUnavailable; any HTTP status is returned to the caller to judge.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import json as jsonlib
+import math
+from collections.abc import AsyncGenerator, AsyncIterable, AsyncIterator, Mapping
+from contextlib import asynccontextmanager
+from dataclasses import dataclass
+from typing import Any, Literal
+
+import httpx
+from observability_021 import Observability
+from observability_021.config import identifier
+
+from clients_021.errors import DependencyUnavailable
+from clients_021.http._observe import Observation, ObservedBody, propagate
+
+HttpMethod = Literal["GET", "HEAD", "OPTIONS", "POST", "PUT", "PATCH", "DELETE"]
+# Safe to send twice (RFC 9110, section 9.2.2): the only methods retried.
+IDEMPOTENT_METHODS: frozenset[HttpMethod] = frozenset({"GET", "HEAD", "OPTIONS", "PUT", "DELETE"})
+RETRY_STATUS = frozenset({502, 503, 504})
+_OUTAGE_ERRORS = (
+    httpx.TimeoutException,
+    httpx.NetworkError,
+    httpx.RemoteProtocolError,
+    httpx.ProxyError,
+)
+
+
+@dataclass(frozen=True)
+class HttpResponse:
+    status_code: int
+    headers: Mapping[str, str]
+    body: bytes
+
+    def json(self) -> Any:
+        return jsonlib.loads(self.body)
+
+
+def _classify(error: BaseException) -> str:
+    if isinstance(error, asyncio.CancelledError):
+        return "cancelled"
+    if isinstance(error, httpx.TimeoutException):
+        return "timeout"
+    if isinstance(error, httpx.NetworkError | httpx.RemoteProtocolError | httpx.ProxyError):
+        return "connection_error"
+    return "internal_error"
+
+
+class _Body(ObservedBody, httpx.AsyncByteStream):
+    pass
+
+
+class ObservedTransport(httpx.AsyncBaseTransport):
+    """Full-body HTTP telemetry for a native httpx.AsyncClient, without retries."""
+
+    def __init__(
+        self,
+        obs: Observability,
+        *,
+        dependency: str,
+        transport: httpx.AsyncBaseTransport | None = None,
+        propagate: bool = True,
+        log_failures: bool = True,
+    ) -> None:
+        self._obs = obs
+        self._dependency = identifier(dependency)
+        self._transport = transport if transport is not None else httpx.AsyncHTTPTransport()
+        self._propagate = propagate
+        self._log_failures = log_failures
+
+    async def handle_async_request(self, request: httpx.Request) -> httpx.Response:
+        observation = Observation(
+            self._obs, self._dependency, request.method, log_failures=self._log_failures
+        )
+        try:
+            with observation.activate():
+                if self._propagate:
+                    propagate(request.headers, request_id=True)
+                response = await self._transport.handle_async_request(request)
+            observation.status_code = response.status_code
+            if not isinstance(response.stream, httpx.AsyncByteStream):
+                raise TypeError("async transport returned a synchronous response stream")
+            if response.is_closed:
+                # Custom/mock transports may have already consumed and closed the response.
+                try:
+                    _ = response.content
+                except httpx.ResponseNotRead:
+                    observation.finish(complete=False)
+                else:
+                    observation.finish(complete=True)
+            else:
+                response.stream = _Body(response.stream, observation, _classify)
+            return response
+        except BaseException as error:
+            observation.finish(error_type=_classify(error))
+            raise
+
+    async def aclose(self) -> None:
+        await self._transport.aclose()
+
+
+class _BorrowedTransport(httpx.AsyncBaseTransport):
+    """Dependency-specific clients borrow the injected transport; HttpClient owns it."""
+
+    def __init__(self, transport: httpx.AsyncBaseTransport) -> None:
+        self._transport = transport
+
+    async def handle_async_request(self, request: httpx.Request) -> httpx.Response:
+        return await self._transport.handle_async_request(request)
+
+    async def aclose(self) -> None:
+        pass
+
+
+class _ClientBody(httpx.AsyncByteStream):
+    """Convert only errors from this response's I/O, never consumer exceptions."""
+
+    def __init__(self, stream: httpx.AsyncByteStream, dependency: str, method: str) -> None:
+        self._stream = stream
+        self._dependency = dependency
+        self._method = method
+
+    async def __aiter__(self) -> AsyncIterator[bytes]:
+        try:
+            async for chunk in self._stream:
+                yield chunk
+        except _OUTAGE_ERRORS as error:
+            raise DependencyUnavailable(self._dependency, self._method, _classify(error)) from error
+
+    async def aclose(self) -> None:
+        try:
+            await self._stream.aclose()
+        except _OUTAGE_ERRORS as error:
+            raise DependencyUnavailable(self._dependency, self._method, _classify(error)) from error
+
+
+class HttpClient:
+    def __init__(
+        self,
+        obs: Observability,
+        *,
+        timeout_seconds: float,
+        retries: int = 2,
+        backoff_seconds: float = 0.2,
+        log_failures: bool = True,
+        transport: httpx.AsyncBaseTransport | None = None,
+    ) -> None:
+        if not math.isfinite(timeout_seconds) or timeout_seconds <= 0:
+            raise ValueError("timeout_seconds must be finite and greater than 0")
+        if type(retries) is not int or retries < 0:
+            raise ValueError("retries must be a non-negative integer")
+        if not math.isfinite(backoff_seconds) or backoff_seconds < 0:
+            raise ValueError("backoff_seconds must be finite and non-negative")
+        self._obs = obs
+        self._timeout = httpx.Timeout(timeout_seconds, connect=min(timeout_seconds, 3.0))
+        self._retries = retries
+        self._backoff = backoff_seconds
+        self._transport = transport
+        self._log_failures = log_failures
+        self._closed = False
+        self._clients: dict[str, httpx.AsyncClient] = {}
+
+    def _client(self, dependency: str) -> httpx.AsyncClient:
+        if self._closed:
+            raise RuntimeError("HttpClient is closed")
+        # One client per dependency, so every call is measured under a fixed name.
+        client = self._clients.get(dependency)
+        if client is None:
+            client = httpx.AsyncClient(
+                timeout=self._timeout,
+                follow_redirects=False,
+                transport=ObservedTransport(
+                    self._obs,
+                    dependency=dependency,
+                    transport=_BorrowedTransport(self._transport)
+                    if self._transport is not None
+                    else None,
+                    log_failures=self._log_failures,
+                ),
+            )
+            self._clients[dependency] = client
+        return client
+
+    async def request(
+        self,
+        method: HttpMethod,
+        url: str,
+        *,
+        dependency: str,
+        headers: Mapping[str, str] | None = None,
+        json: Any = None,
+        form: Mapping[str, str] | None = None,
+        content: bytes | None = None,
+    ) -> HttpResponse:
+        if sum(value is not None for value in (json, form, content)) > 1:
+            raise ValueError("provide only one of json, form or content")
+        attempts = 1 + (self._retries if method in IDEMPOTENT_METHODS else 0)
+        for attempt in range(attempts):
+            last = attempt == attempts - 1
+            try:
+                response = await self._client(dependency).request(
+                    method,
+                    url,
+                    headers=dict(headers or {}),
+                    json=json,
+                    data=dict(form) if form is not None else None,
+                    content=content,
+                )
+            except _OUTAGE_ERRORS as error:
+                if last:
+                    raise DependencyUnavailable(dependency, method, _classify(error)) from error
+            else:
+                if response.status_code not in RETRY_STATUS or last:
+                    return HttpResponse(
+                        response.status_code, dict(response.headers), response.content
+                    )
+            await asyncio.sleep(self._backoff * 2**attempt)
+        raise AssertionError("unreachable")
+
+    @asynccontextmanager
+    async def stream(
+        self,
+        method: HttpMethod,
+        url: str,
+        *,
+        dependency: str,
+        headers: Mapping[str, str] | None = None,
+        json: Any = None,
+        form: Mapping[str, str] | None = None,
+        content: bytes | AsyncIterable[bytes] | None = None,
+    ) -> AsyncGenerator[httpx.Response]:
+        """One attempt; yields a native response without buffering or replaying its body."""
+        if sum(value is not None for value in (json, form, content)) > 1:
+            raise ValueError("provide only one of json, form or content")
+        client = self._client(dependency)
+        request = client.build_request(
+            method,
+            url,
+            headers=dict(headers or {}),
+            json=json,
+            data=dict(form) if form is not None else None,
+            content=content,
+        )
+        try:
+            response = await client.send(request, stream=True)
+        except _OUTAGE_ERRORS as error:
+            raise DependencyUnavailable(dependency, method, _classify(error)) from error
+        if isinstance(response.stream, httpx.AsyncByteStream):
+            response.stream = _ClientBody(response.stream, dependency, method)
+        try:
+            yield response
+        finally:
+            await response.aclose()
+
+    async def aclose(self) -> None:
+        if self._closed:
+            return
+        self._closed = True
+        try:
+            await asyncio.gather(*(client.aclose() for client in self._clients.values()))
+        finally:
+            self._clients.clear()
+            if self._transport is not None:
+                await self._transport.aclose()

clients_021/http/httpx2.py

@@ -0,0 +1,76 @@
+"""Observed async httpx2 transport: each attempt includes response body and stream close.
+
+Use ``ObservedTransport(obs, dependency="anthropic")`` with the SDK's HTTP client.
+The SDK keeps control of retries and exceptions. Propagation is opt-in for external APIs.
+"""
+
+from __future__ import annotations
+
+import asyncio
+
+import httpx2
+from observability_021 import Observability
+from observability_021.config import identifier
+
+from clients_021.http._observe import Observation, ObservedBody, propagate
+
+
+def _classify(error: BaseException) -> str:
+    if isinstance(error, asyncio.CancelledError):
+        return "cancelled"
+    if isinstance(error, httpx2.TimeoutException):
+        return "timeout"
+    if isinstance(error, httpx2.NetworkError | httpx2.RemoteProtocolError | httpx2.ProxyError):
+        return "connection_error"
+    return type(error).__name__
+
+
+class _Body(ObservedBody, httpx2.AsyncByteStream):
+    pass
+
+
+class ObservedTransport(httpx2.AsyncBaseTransport):
+    def __init__(
+        self,
+        obs: Observability,
+        *,
+        dependency: str,
+        transport: httpx2.AsyncBaseTransport | None = None,
+        propagate: bool = False,
+        log_failures: bool = True,
+    ) -> None:
+        self._obs = obs
+        self._dependency = identifier(dependency)
+        self._transport = transport if transport is not None else httpx2.AsyncHTTPTransport()
+        self._propagate = propagate
+        self._log_failures = log_failures
+
+    async def handle_async_request(self, request: httpx2.Request) -> httpx2.Response:
+        observation = Observation(
+            self._obs, self._dependency, request.method, log_failures=self._log_failures
+        )
+        try:
+            with observation.activate():
+                if self._propagate:
+                    propagate(request.headers)
+                response = await self._transport.handle_async_request(request)
+            observation.status_code = response.status_code
+            if not isinstance(response.stream, httpx2.AsyncByteStream):
+                raise TypeError("async transport returned a synchronous response stream")
+            if response.is_closed:
+                # Custom/mock transports may have already consumed and closed the response.
+                try:
+                    _ = response.content
+                except httpx2.ResponseNotRead:
+                    observation.finish(complete=False)
+                else:
+                    observation.finish(complete=True)
+            else:
+                response.stream = _Body(response.stream, observation, _classify)
+            return response
+        except BaseException as error:
+            observation.finish(error_type=_classify(error))
+            raise
+
+    async def aclose(self) -> None:
+        await self._transport.aclose()

clients_021/httpx2.py

@@ -1,93 +1,5 @@
-"""An observed httpx2 transport, for SDKs built on httpx2 (Anthropic, OpenAI and others).
+"""Compatibility import; prefer clients_021.http.httpx2 for new code."""
 
-    from anthropic import AsyncAnthropic, DefaultAsyncHttpxClient
-    from clients_021.httpx2 import ObservedTransport
+from clients_021.http.httpx2 import ObservedTransport
 
-    transport = ObservedTransport(obs, dependency="anthropic")
-    client = AsyncAnthropic(http_client=DefaultAsyncHttpxClient(transport=transport))
-
-Needs the `httpx2` extra. Records the same `http.client.request.*` instruments and CLIENT
-span as 021-python-observability's httpx transport, under `dependency`, with method,
-status and a bounded error type; never the URL. Trace context is not sent by default: these
-are third-party APIs (`propagate=True` sends `traceparent` to services you own).
-"""
-
-from __future__ import annotations
-
-import asyncio
-import time
-
-import httpx2
-from observability_021 import Observability
-from observability_021.config import identifier
-from opentelemetry.trace import SpanKind, StatusCode
-from opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator
-
-_METHODS = frozenset({"GET", "HEAD", "OPTIONS", "POST", "PUT", "PATCH", "DELETE"})
-
-
-class ObservedTransport(httpx2.AsyncBaseTransport):
-    def __init__(
-        self,
-        obs: Observability,
-        *,
-        dependency: str,
-        transport: httpx2.AsyncBaseTransport | None = None,
-        propagate: bool = False,
-    ) -> None:
-        self._obs = obs
-        self._dependency = identifier(dependency)
-        self._transport = transport if transport is not None else httpx2.AsyncHTTPTransport()
-        self._propagate = propagate
-
-    async def handle_async_request(self, request: httpx2.Request) -> httpx2.Response:
-        method = request.method if request.method in _METHODS else "_OTHER"
-        attributes: dict[str, str | int] = {
-            "dependency": self._dependency,
-            "http.request.method": method,
-        }
-        started = time.perf_counter()
-        tracer = self._obs.tracer
-        with tracer.start_as_current_span(
-            f"HTTP {self._dependency}",
-            kind=SpanKind.CLIENT,
-            attributes=attributes,
-            record_exception=False,
-            set_status_on_exception=False,
-        ) as span:
-            if self._propagate:
-                # Replace the whole pair: the new parent may have no tracestate.
-                request.headers.pop("traceparent", None)
-                request.headers.pop("tracestate", None)
-                carrier: dict[str, str] = {}
-                TraceContextTextMapPropagator().inject(carrier)
-                for key, value in carrier.items():
-                    request.headers[key] = value
-            try:
-                response = await self._transport.handle_async_request(request)
-            except BaseException as error:
-                attributes["error.type"] = (
-                    "cancelled"
-                    if isinstance(error, asyncio.CancelledError)
-                    else "timeout"
-                    if isinstance(error, httpx2.TimeoutException)
-                    else "connection_error"
-                    if isinstance(
-                        error, httpx2.NetworkError | httpx2.RemoteProtocolError | httpx2.ProxyError
-                    )
-                    else type(error).__name__
-                )
-                span.set_status(StatusCode.ERROR)
-                raise
-            else:
-                attributes["http.response.status_code"] = response.status_code
-                if response.status_code >= 400:
-                    span.set_status(StatusCode.ERROR)
-                return response
-            finally:
-                span.set_attributes(attributes)
-                self._obs.client_requests.add(1, attributes)
-                self._obs.client_duration.record(time.perf_counter() - started, attributes)
-
-    async def aclose(self) -> None:
-        await self._transport.aclose()
+__all__ = ["ObservedTransport"]

clients_021/queue/qstash.py

@@ -128,7 +128,7 @@ class QStashClient:
         self._propagate = propagate
         self._closed = False
         self._http = HttpClient(
-            obs, timeout_seconds=timeout_seconds, retries=0, transport=transport
+            obs, timeout_seconds=timeout_seconds, retries=0, log_failures=False, transport=transport
         )
 
     async def publish_json(

clients_021/http.py

@@ -1,131 +0,0 @@
-"""Outbound HTTP: one httpx client per dependency, mandatory timeouts and bounded retries.
-
-    from clients_021.http import HttpClient
-
-    http = HttpClient(obs, timeout_seconds=10)
-    response = await http.request("GET", url, dependency="identity-provider")
-
-Every call goes through 021-python-observability's transport: `http.client.request.*`
-under the `dependency` name, a CLIENT span, and `traceparent` / `x-request-id` sent along.
-Methods in IDEMPOTENT_METHODS are retried on transport errors and 502/503/504 with
-backoff; POST and PATCH are sent once (send an Idempotency-Key and retry in the caller if
-the other side supports it). A transport failure after the last attempt raises
-DependencyUnavailable; any HTTP status is returned to the caller to judge.
-"""
-
-from __future__ import annotations
-
-import asyncio
-import json as jsonlib
-import math
-from collections.abc import Mapping
-from dataclasses import dataclass
-from typing import Any, Literal
-
-import httpx
-from observability_021 import Observability
-
-from clients_021.errors import DependencyUnavailable
-
-HttpMethod = Literal["GET", "HEAD", "OPTIONS", "POST", "PUT", "PATCH", "DELETE"]
-# Safe to send twice (RFC 9110, section 9.2.2): the only methods retried.
-IDEMPOTENT_METHODS: frozenset[HttpMethod] = frozenset({"GET", "HEAD", "OPTIONS", "PUT", "DELETE"})
-RETRY_STATUS = frozenset({502, 503, 504})
-
-
-@dataclass(frozen=True)
-class HttpResponse:
-    status_code: int
-    headers: Mapping[str, str]
-    body: bytes
-
-    def json(self) -> Any:
-        return jsonlib.loads(self.body)
-
-
-class HttpClient:
-    def __init__(
-        self,
-        obs: Observability,
-        *,
-        timeout_seconds: float,
-        retries: int = 2,
-        backoff_seconds: float = 0.2,
-        transport: httpx.AsyncBaseTransport | None = None,
-    ) -> None:
-        if not math.isfinite(timeout_seconds) or timeout_seconds <= 0:
-            raise ValueError("timeout_seconds must be finite and greater than 0")
-        if type(retries) is not int or retries < 0:
-            raise ValueError("retries must be a non-negative integer")
-        if not math.isfinite(backoff_seconds) or backoff_seconds < 0:
-            raise ValueError("backoff_seconds must be finite and non-negative")
-        self._obs = obs
-        self._timeout = httpx.Timeout(timeout_seconds, connect=min(timeout_seconds, 3.0))
-        self._retries = retries
-        self._backoff = backoff_seconds
-        self._transport = transport
-        self._clients: dict[str, httpx.AsyncClient] = {}
-
-    def _client(self, dependency: str) -> httpx.AsyncClient:
-        # One client per dependency, so every call is measured under a fixed name.
-        client = self._clients.get(dependency)
-        if client is None:
-            client = httpx.AsyncClient(
-                timeout=self._timeout,
-                follow_redirects=False,
-                transport=self._obs.async_httpx_transport(
-                    dependency=dependency, transport=self._transport
-                ),
-            )
-            self._clients[dependency] = client
-        return client
-
-    async def request(
-        self,
-        method: HttpMethod,
-        url: str,
-        *,
-        dependency: str,
-        headers: Mapping[str, str] | None = None,
-        json: Any = None,
-        form: Mapping[str, str] | None = None,
-        content: bytes | None = None,
-    ) -> HttpResponse:
-        if sum(value is not None for value in (json, form, content)) > 1:
-            raise ValueError("provide only one of json, form or content")
-        attempts = 1 + (self._retries if method in IDEMPOTENT_METHODS else 0)
-        for attempt in range(attempts):
-            last = attempt == attempts - 1
-            try:
-                response = await self._client(dependency).request(
-                    method,
-                    url,
-                    headers=dict(headers or {}),
-                    json=json,
-                    data=dict(form) if form is not None else None,
-                    content=content,
-                )
-            except (
-                httpx.TimeoutException,
-                httpx.NetworkError,
-                httpx.RemoteProtocolError,
-                httpx.ProxyError,
-            ) as error:
-                if last:
-                    reason = (
-                        "timeout"
-                        if isinstance(error, httpx.TimeoutException)
-                        else "connection_error"
-                    )
-                    raise DependencyUnavailable(dependency, method, reason) from error
-            else:
-                if response.status_code not in RETRY_STATUS or last:
-                    return HttpResponse(
-                        response.status_code, dict(response.headers), response.content
-                    )
-            await asyncio.sleep(self._backoff * 2**attempt)
-        raise AssertionError("unreachable")
-
-    async def aclose(self) -> None:
-        await asyncio.gather(*(client.aclose() for client in self._clients.values()))
-        self._clients.clear()