021-python-clients 0.3.0__py3-none-any.whl → 0.4.0__py3-none-any.whl

{021_python_clients-0.3.0.dist-info → 021_python_clients-0.4.0.dist-info}/METADATA

@@ -1,13 +1,13 @@
 Metadata-Version: 2.5
 Name: 021-python-clients
-Version: 0.3.0
-Summary: Observed clients for Redis, SQS, QStash, S3, Vercel Blob, HTTP, email and KMS, built on 021-python-observability
+Version: 0.4.0
+Summary: Observed clients for Redis, SQS, QStash, S3, Vercel Blob, HTTP, email, KMS, Secrets Manager, Bedrock and Kubernetes Jobs, built on 021-python-observability
 Project-URL: Repository, https://github.com/from021-io/python-clients
 Project-URL: Changelog, https://github.com/from021-io/python-clients/blob/main/CHANGELOG.md
 Project-URL: Contract, https://github.com/from021-io/python-clients/blob/main/CONTRACT.md
 License-Expression: Apache-2.0
 License-File: LICENSE
-Keywords: email,httpx,kms,observability,opentelemetry,qstash,redis,resend,s3,ses,sqs
+Keywords: bedrock,email,httpx,kms,kubernetes,observability,opentelemetry,qstash,redis,resend,s3,secrets-manager,ses,sqs
 Classifier: Development Status :: 3 - Alpha
 Classifier: Intended Audience :: Developers
 Classifier: Operating System :: OS Independent
@@ -58,6 +58,9 @@ uv add "021-python-clients[redis,aws,httpx2,vercel-blob]"   # pick the extras yo
 | `clients_021.email` | none | `EmailMessage`, the `EmailSender` protocol, `ResendSender` (Resend's HTTP API) |
 | `clients_021.email.ses` | `aws` | `SesSender`: the same protocol over Amazon SES v2 |
 | `clients_021.crypto.kms` | `aws` | `KmsClient`: encrypt, decrypt and data keys bound to an encryption context; AES-256-GCM envelopes |
+| `clients_021.secrets.secrets_manager` | `aws` | `SecretsManagerClient`: read a secret by id, version id or stage; the value is never printed |
+| `clients_021.compute.kubernetes` | none | `KubernetesJobsClient`: create (idempotent by name), read and delete Jobs in a namespace, in-cluster service account |
+| `clients_021.llm.bedrock` | `aws` | `BedrockRuntime`: an observed `bedrock-runtime` client to pass to SDKs such as LangChain's `ChatBedrockConverse`; streams measured to the end, token usage counted |
 
 Every client takes the service's `Observability` (from `setup_observability`) and a
 `dependency` name, which becomes the `dependency` label of its metrics.
@@ -66,7 +69,7 @@ Every client takes the service's `Observability` (from `setup_observability`) an
 
 Providers with the same purpose live under one capability package: `email/` contains
 Resend and SES; `queue/` contains SQS and QStash; `storage/` contains S3 and Vercel Blob; `cache/`
-contains Redis; `crypto/` contains KMS. `http/` contains the httpx client and httpx2
+contains Redis; `crypto/` contains KMS; `secrets/` contains Secrets Manager; `llm/` contains Bedrock; `compute/` contains Kubernetes Jobs. `http/` contains the httpx client and httpx2
 transport. Import providers explicitly:
 
 ```python
@@ -77,6 +80,9 @@ from clients_021.queue.qstash import QStashClient
 from clients_021.storage.s3 import S3Client
 from clients_021.cache.redis import RedisClient
 from clients_021.crypto.kms import KmsClient
+from clients_021.secrets.secrets_manager import SecretsManagerClient
+from clients_021.llm.bedrock import BedrockRuntime
+from clients_021.compute.kubernetes import KubernetesJobsClient
 ```
 
 SQS supports polling, receipts and visibility changes; QStash delivers to an HTTP endpoint.
@@ -95,12 +101,16 @@ published exports; `clients_021.httpx2.ObservedTransport` remains an alias for
 
 - A CLIENT span named `<dependency> <operation>` (`HTTP <dependency>` for HTTP), with the
   system attribute (`db.system.name=redis`, `messaging.system=aws_sqs|resend|aws_ses`,
-  `rpc.service=S3|KMS`, `rpc.system=vercel-blob`).
+  `rpc.service=S3|KMS|SecretsManager|BedrockRuntime`, `rpc.system=vercel-blob`, `rpc.system=kubernetes`;
+  Bedrock also `gen_ai.system=aws.bedrock`).
 - A counter and a duration histogram: `db.client.operation.*` (Redis, shared with SQL),
   `messaging.client.operation.*` (SQS, QStash, email), `storage.client.operation.*` (S3, Vercel Blob),
-  `crypto.client.operation.*` (KMS), `http.client.request.*` (HTTP, and the HTTP request
+  `crypto.client.operation.*` (KMS), `secrets.client.operation.*` (Secrets Manager),
+  `llm.client.operation.*` and `llm.client.token.usage` (Bedrock), `compute.client.operation.*`
+  (Kubernetes Jobs),
+  `http.client.request.*` (HTTP, and the HTTP request
   under QStash and Resend). Labels are bounded; see [CONTRACT.md](CONTRACT.md).
-- Redis, SQS, QStash, S3, Vercel Blob, email and KMS emit a `dependency.call_failed` warning on failure,
+- Redis, SQS, QStash, S3, Vercel Blob, email, KMS, Secrets Manager, Bedrock and Kubernetes Jobs emit a `dependency.call_failed` warning on failure,
   with dependency, operation and error type only. HTTP/httpx2 also log failed attempts,
   including body failures and cancellation; QStash/Resend suppress the nested HTTP log.
 - `DependencyUnavailable(dependency, operation, reason)` when the service timed out,
@@ -109,7 +119,9 @@ published exports; `clients_021.httpx2.ObservedTransport` remains an alias for
   original type: they are bugs, not outages.
 
 Never in this library's spans, metrics or logs: keys, URLs, bodies, message ids or object
-names; email addresses, subjects, bodies or tags; KMS key ids, encryption contexts,
+names; email addresses, subjects, bodies or tags; secret ids, version ids, stages and
+values; model ids, prompts and outputs; namespaces, Job names, manifests and service
+account tokens; KMS key ids, encryption contexts,
 plaintext, ciphertext or data keys. Use static dependency names such as `identity-provider` or `uploads`; never derive
 them from a tenant, user or request. All clients apply observability's identifier rule
 (1–96 letters, digits, `_`, `.` or `-`, starting with a letter or digit).
@@ -272,6 +284,64 @@ except EmailRejected as rejected:  # the address, the domain or the credentials;
 SES has no idempotency key (the argument is ignored): deduplicate in the caller when a retry
 must not send twice. Tags are for provider analytics; never put personal data in them.
 
+### Kubernetes Jobs
+
+```python
+from clients_021.compute.kubernetes import KubernetesJobsClient
+
+jobs = KubernetesJobsClient.in_cluster(obs, namespace="agent-runs")
+created = await jobs.create({"metadata": {"name": "cmd-4821"}, "spec": job_spec})
+status = await jobs.get("cmd-4821")        # JobStatus(phase="running", ...) or None
+await jobs.delete("cmd-4821", propagation="Background")
+```
+
+No SDK: it calls the `batch/v1` REST API through `HttpClient`. `in_cluster` uses the pod's
+service account (CA bundle, namespace, and a token it re-reads at most once a minute because
+projected tokens rotate). `create` is idempotent by name: `generateName` is refused, and an
+existing Job returns `created=False`, so a caller can repeat it after a timeout. `get`
+returns `None` and `delete` returns `False` for a missing Job. POST is sent once; GET and
+DELETE retry 502, 503, 504 and network failures once. 429, 5xx, timeouts and connection
+failures raise `DependencyUnavailable`; other rejections raise `KubernetesApiError` with the
+status code and the API's reason, never its message.
+
+### Bedrock
+
+```python
+from clients_021.llm.bedrock import BedrockRuntime
+from langchain_aws import ChatBedrockConverse
+
+runtime = BedrockRuntime.create(obs, region="us-east-1")
+model = ChatBedrockConverse(model=settings.inference_profile, client=runtime)
+```
+
+`BedrockRuntime` keeps boto3's call signatures for `converse`, `converse_stream`,
+`invoke_model`, `invoke_model_with_response_stream` and `count_tokens`, and passes every
+other attribute (`meta`, `exceptions`) through to the boto3 client, so an SDK can take it
+where it expects one. Calls stay synchronous like boto3's. A streaming call is measured
+until its event stream ends; leaving the loop early or calling `close()` ends it as `ok`.
+Converse and ConverseStream add the reported input, output and cache tokens to
+`llm.client.token.usage`. Throttling (also mid-stream), 5xx, a model still loading and the
+stream errors Bedrock documents as retryable raise `DependencyUnavailable`; access denied,
+validation errors, a missing model and quota limits keep their botocore type. A retried
+model call is billed again.
+
+### Secrets Manager
+
+```python
+from clients_021.secrets.secrets_manager import SecretsManagerClient
+
+secrets = SecretsManagerClient.create(obs, region="us-east-1")
+secret = await secrets.get("connections/brain")
+token = secret.text()            # SecretString; secret.binary() for SecretBinary
+previous = await secrets.get("connections/brain", version_stage="AWSPREVIOUS")
+```
+
+`get` reads the current version unless a `version_id` or a `version_stage` is given (not
+both). The client does not cache: every call is a billed request, so keep a short-lived
+copy when the same secret is read often. A missing secret, access denied, a value KMS cannot
+decrypt or a secret scheduled for deletion keep their botocore type; throttling, 5xx and
+timeouts raise `DependencyUnavailable`. `repr(secret)` never shows the value.
+
 ### KMS
 
 The encryption context is required on every call, so a ciphertext only decrypts for the
@@ -375,6 +445,8 @@ async with AsyncExitStack() as resources:
     resources.push_async_callback(email.aclose)
     keys = KmsClient.create(obs, settings.session_key_arn, region="us-east-1")
     resources.push_async_callback(keys.aclose)
+    secrets = SecretsManagerClient.create(obs, region="us-east-1")
+    resources.push_async_callback(secrets.aclose)
     # Run the service here. Stop its background workers before leaving this scope.
 ```
 
@@ -386,7 +458,7 @@ Retry policies are explicit:
   POST and PATCH run once. Invalid URLs/protocols and invalid request construction keep
   their original exception. HTTP status responses are returned for the caller to handle.
 - Resend sends each request once (POST); send an `idempotency_key` and retry in the caller.
-- AWS factories (SQS, S3, SES, KMS) use botocore standard retries with at most **three total
+- AWS factories (SQS, S3, SES, KMS, Secrets Manager, Bedrock) use botocore standard retries with at most **three total
   attempts**.
   Injected SDK clients keep their configured policy. SQS consumers must tolerate duplicate
   messages; a timeout does not prove that a send or write was not applied.
@@ -409,7 +481,7 @@ any operation**. Map it at the service boundary and apply the operation's idempo
 Its message and structured fields are safe to log when dependency names are static. Its
 chained SDK exception may contain sensitive provider data; do not log exception chains.
 
-S3, SQS, SES and KMS run the synchronous AWS SDK in worker threads. Cancelling the await does not
+S3, SQS, SES, KMS and Secrets Manager run the synchronous AWS SDK in worker threads. Cancelling the await does not
 stop an SDK request already running in its thread. Timeouts are per SDK/network attempt,
 not an overall deadline. Limit consumer concurrency and let in-flight work finish before
 closing the clients. Redis window counters require Redis server 7 or newer (`EXPIRE NX`).
@@ -433,6 +505,8 @@ deduplication, start the [official local server](https://upstash.com/docs/qstash
 set its `QSTASH_URL` and `QSTASH_TOKEN`, and run `uv run python tests/qstash_local.py`.
 That smoke test accepts only a loopback QStash server and binds its destination dynamically.
 
-SQS, S3, SES and KMS tests run against [moto](https://github.com/getmoto/moto) (plus
+Kubernetes Jobs tests use httpx's `MockTransport`. Bedrock tests use botocore's Stubber and
+fake event streams. SQS, S3, SES, KMS and Secrets
+Manager tests run against [moto](https://github.com/getmoto/moto) (plus
 botocore's Stubber where moto ignores a field or differs from AWS); Resend tests use httpx's
 `MockTransport`; Redis tests need a real Redis. Licensed under Apache-2.0.

{021_python_clients-0.3.0.dist-info → 021_python_clients-0.4.0.dist-info}/RECORD

@@ -1,6 +1,6 @@
-clients_021/__init__.py,sha256=to7x0AyCtQxhss2a4YGj1GMJq8NyRQdKp-DICQrBbzs,821
+clients_021/__init__.py,sha256=D7i865mBaxw32ycKWsMazMRljtYFdabdSG8dHjfWg3M,921
 clients_021/_aws.py,sha256=sj2kCfod-vz9J93UyeVzNVLGK3M25yipKdW15KyIkw4,1722
-clients_021/_observe.py,sha256=Xjg5Qck-e8Gt5NQHOND6nzr7DvTn46PmBdccubiD3Dk,3792
+clients_021/_observe.py,sha256=Kh-2gitT2bI4vNa-Y7PYsCns60XJzJ_N7ZbTuyX82lo,6977
 clients_021/errors.py,sha256=GS_fN8mjr7MCOkhqPhH6LweIvQ20Vd29U-aZY8zBeJQ,972
 clients_021/httpx2.py,sha256=rNbNGJuPJPZUwVbNN5L88IF0g3Tq6EHqj_uFmJ3RmLQ,161
 clients_021/kms.py,sha256=YcZXURpqIZTkjcP0fpWMD6X_adMAYwN7zAJNhUImR4s,183
@@ -9,6 +9,8 @@ clients_021/redis.py,sha256=xC3tji_ldJAwaBntkZTuPhNHLIzl3F3nDzkikMrbJCc,169
 clients_021/s3.py,sha256=6Q5_1emngSB00IwvBdrmV3iAg-Q_Ndco1HGzYObL5FI,231
 clients_021/cache/__init__.py,sha256=3OyQQ-uZeghpgJvvJ2k56_t50q3CCwbAp3dBrCMMcSU,76
 clients_021/cache/redis.py,sha256=DRXCMc8MqY8KsZ50j4X4mUsd0lvvmP-UDuUD2wJXyW8,4932
+clients_021/compute/__init__.py,sha256=PP23LVKxoJGkwzzFbuKMRd35LugdYqJWElp9lWymllM,78
+clients_021/compute/kubernetes.py,sha256=a0-9ldOg1mQjJBVzWAhZH4QGEN6qAzcgl_n8Ty0rSts,12456
 clients_021/crypto/__init__.py,sha256=qESFBPQBKcHhdFXqeJzSyEP41yveG7rFrxnpITmqlNU,77
 clients_021/crypto/kms.py,sha256=My5Xyg0bOiT9YHYWojQSszpREEh3bSYMMB26wy6fNEI,7261
 clients_021/email/__init__.py,sha256=ivmdu9hhNDwPqBn3G7uNG0Zpc3DEMnlnWBVVoVQ-fQA,790
@@ -19,13 +21,17 @@ clients_021/http/__init__.py,sha256=P4OmYruhkQfJwJzHA90FXKWg_A2rh5iOl-ES_NTMMVQ,
 clients_021/http/_observe.py,sha256=hpTzcej9MhlvM-RgUZWicR57ulE90GmZMs5U_qFoquQ,6922
 clients_021/http/httpx.py,sha256=gvqSpO1rb6nBA5i01W1yos5mPBwLqa3-HDYwMtQXc6s,10627
 clients_021/http/httpx2.py,sha256=fmqH4bikDKZ6YY0sNBHZbocff2zU40Gpks1aoRuOGeQ,2810
+clients_021/llm/__init__.py,sha256=pEI4drpN0MKg_gp8BP1U7kpVd3NDV_Sa9eFD3J87LPw,86
+clients_021/llm/bedrock.py,sha256=GJafCh6PhBrsDgs_GapoKiVflv32V7BUqyAj52pMQiA,9367
 clients_021/queue/__init__.py,sha256=CesweOrXmX5cyU-mRvqu-4wLkNJI15zi_Lows6SPupY,338
 clients_021/queue/qstash.py,sha256=NndlqmNkcBgs0ExGsT-3-bMAdU0skO8Djptqj8NoJAM,8335
 clients_021/queue/sqs.py,sha256=jnccgbxyglzs2mR_DsH04IisEwygDlw6s3RxR0UtMTc,6693
+clients_021/secrets/__init__.py,sha256=LdxphQvruITeP5jXZR69ly2jxMxuHCI1BDavw2mU0EY,76
+clients_021/secrets/secrets_manager.py,sha256=ttP3_vCXRtOYvbUaNIkFlSTzYYr4yU3Uwim3juXUfPM,4746
 clients_021/storage/__init__.py,sha256=KrJViRHBK7PkIYnagg7vyXp95SzMFp9cUCKEnMeFr84,78
 clients_021/storage/s3.py,sha256=NDtG9QzHSXvrcazTYTq1EH5VjuRd2xMMsMvaumqci4Q,5946
 clients_021/storage/vercel_blob.py,sha256=FlnoKiahqtk3PpLsGpyu_Gy87otg15i4BlJZsIB3LfM,6511
-021_python_clients-0.3.0.dist-info/METADATA,sha256=ewZEC_EJC4-e8ZayWgpdlnxQ0hYhWMMjmRr2KIN4iW4,23227
-021_python_clients-0.3.0.dist-info/WHEEL,sha256=W3fkpkm7-wf9vBI5Z-7s0eWkeM-spu78I8Neb98DeEg,87
-021_python_clients-0.3.0.dist-info/licenses/LICENSE,sha256=z8d0m5b2O9McPEK1xHG_dWgUBT6EfBDz6wA0F7xSPTA,11358
-021_python_clients-0.3.0.dist-info/RECORD,,
+021_python_clients-0.4.0.dist-info/METADATA,sha256=Q4ITtnJ9tRJxxH3gdoE-1nl3A4ClHkAV5yihBGN1JTg,27808
+021_python_clients-0.4.0.dist-info/WHEEL,sha256=W3fkpkm7-wf9vBI5Z-7s0eWkeM-spu78I8Neb98DeEg,87
+021_python_clients-0.4.0.dist-info/licenses/LICENSE,sha256=z8d0m5b2O9McPEK1xHG_dWgUBT6EfBDz6wA0F7xSPTA,11358
+021_python_clients-0.4.0.dist-info/RECORD,,

clients_021/__init__.py

@@ -1,8 +1,10 @@
 """Observed clients for the external services Python services call.
 
-`clients_021.http`, `clients_021.queue.qstash` and `clients_021.email` (Resend) need no extras;
+`clients_021.http`, `clients_021.queue.qstash`, `clients_021.email` (Resend) and
+`clients_021.compute.kubernetes` need no extras;
 `clients_021.cache.redis` needs the `redis` extra; `clients_021.queue.sqs`,
-`clients_021.storage.s3`, `clients_021.email.ses` and `clients_021.crypto.kms` the `aws` extra;
+`clients_021.storage.s3`, `clients_021.email.ses`, `clients_021.crypto.kms`,
+`clients_021.secrets.secrets_manager` and `clients_021.llm.bedrock` the `aws` extra;
 `clients_021.http.httpx2` the `httpx2` extra.
 `clients_021.storage.vercel_blob` needs the `vercel-blob` extra.
 """

clients_021/_observe.py

@@ -2,8 +2,11 @@
 
 The `db` family reuses the instruments 021-python-observability already creates for SQL
 (`db.client.operation.*`), so Redis and Postgres share dashboards. `messaging` (SQS, email),
-`storage` (S3, Vercel Blob) and `crypto` (KMS) get their own pair on the same meter. Attributes are
-bounded: dependency, operation, status and error.type; never keys, URLs, bodies or ids.
+`storage` (S3, Vercel Blob), `crypto` (KMS), `secrets` (Secrets Manager), `llm` (Bedrock) and
+`compute` (Kubernetes Jobs) get their own pair on the same meter. `Call` is the same
+measurement for synchronous SDK calls and for operations that end later than they start, such
+as a model response stream. Attributes are bounded: dependency, operation, status and
+error.type; never keys, URLs, bodies or ids.
 """
 
 from __future__ import annotations
@@ -12,7 +15,7 @@ import asyncio
 import time
 from collections.abc import AsyncGenerator, Callable
 from contextlib import asynccontextmanager
-from typing import Literal
+from typing import Literal, NoReturn
 from weakref import WeakKeyDictionary
 
 from observability_021 import Observability, get_logger
@@ -21,7 +24,7 @@ from opentelemetry.trace import SpanKind, StatusCode
 
 from clients_021.errors import DependencyUnavailable
 
-Family = Literal["db", "messaging", "storage", "crypto"]
+Family = Literal["db", "messaging", "storage", "crypto", "secrets", "llm", "compute"]
 # Returns the reason when an exception means "the dependency failed", None for a bug.
 Classifier = Callable[[BaseException], str | None]
 
@@ -31,6 +34,9 @@ _UNITS = {
     "messaging": "{operation}",
     "storage": "{operation}",
     "crypto": "{operation}",
+    "secrets": "{operation}",
+    "llm": "{operation}",
+    "compute": "{operation}",
 }
 _cache: WeakKeyDictionary[Observability, dict[Family, tuple[Counter, Histogram]]] = (
     WeakKeyDictionary()
@@ -104,3 +110,83 @@ async def observed(
         finally:
             count.add(1, attributes)
             duration.record(time.perf_counter() - started, attributes)
+
+
+class Call:
+    """One observed operation started now and finished by `ok`, `fail` or `cancelled`.
+
+    The synchronous counterpart of `observed`, with the same span, instruments, labels and
+    warning. It also covers operations that outlive the call that started them: a stream is
+    finished when it is exhausted, fails or is closed. Finishing twice is ignored.
+    """
+
+    def __init__(
+        self,
+        obs: Observability,
+        *,
+        family: Family,
+        system: dict[str, str],
+        dependency: str,
+        operation: str,
+        classify: Classifier,
+    ) -> None:
+        self._attributes: dict[str, str] = {"dependency": dependency, "operation": operation}
+        self._count, self._duration = instruments(obs, family)
+        self._classify = classify
+        self._started = time.perf_counter()
+        self._span = obs.tracer.start_span(
+            f"{dependency} {operation}",
+            kind=SpanKind.CLIENT,
+            attributes={**system, **self._attributes},
+            record_exception=False,
+            set_status_on_exception=False,
+        )
+        self._done = False
+
+    @property
+    def done(self) -> bool:
+        return self._done
+
+    def _finish(self) -> None:
+        self._done = True
+        self._span.set_attributes(self._attributes)
+        self._span.end()
+        self._count.add(1, self._attributes)
+        self._duration.record(time.perf_counter() - self._started, self._attributes)
+
+    def ok(self) -> None:
+        if self._done:
+            return
+        self._attributes["status"] = "ok"
+        self._finish()
+
+    def cancelled(self) -> None:
+        if self._done:
+            return
+        self._attributes["status"] = "error"
+        self._attributes["error.type"] = "cancelled"
+        self._span.set_status(StatusCode.ERROR)
+        self._finish()
+
+    def fail(self, error: BaseException) -> NoReturn:
+        """Records the failure, then raises DependencyUnavailable or the error itself."""
+        if isinstance(error, asyncio.CancelledError | KeyboardInterrupt | GeneratorExit):
+            self.cancelled()
+            raise error
+        reason = self._classify(error)
+        if not self._done:
+            self._attributes["status"] = "error"
+            self._attributes["error.type"] = reason or type(error).__name__
+            self._span.set_status(StatusCode.ERROR)
+            logger.warning(
+                "dependency.call_failed",
+                dependency=self._attributes["dependency"],
+                operation=self._attributes["operation"],
+                error_type=self._attributes["error.type"],
+            )
+            self._finish()
+        if reason is None:
+            raise error
+        raise DependencyUnavailable(
+            self._attributes["dependency"], self._attributes["operation"], reason
+        ) from error

clients_021/compute/__init__.py

@@ -0,0 +1 @@
+"""Compute clients. Import providers explicitly; SDKs are optional extras."""

clients_021/compute/kubernetes.py

@@ -0,0 +1,314 @@
+"""Kubernetes Jobs over the shared HTTP client: create, read and delete Jobs in one namespace.
+
+    from clients_021.compute.kubernetes import KubernetesJobsClient
+
+    jobs = KubernetesJobsClient.in_cluster(obs, namespace="agent-runs")
+    created = await jobs.create(manifest)        # created.created is False if it already existed
+    status = await jobs.get("cmd-4821")          # None when the Job does not exist
+    deleted = await jobs.delete("cmd-4821")
+
+No optional SDK: it speaks the `batch/v1` REST API through `HttpClient`. `in_cluster` reads
+the pod's service account (API host from `KUBERNETES_SERVICE_HOST`/`_PORT`, the CA bundle and
+the namespace from `/var/run/secrets/kubernetes.io/serviceaccount`) and re-reads the token
+file at most once a minute, because projected tokens rotate. Calls record
+`compute.client.operation.*` with `operation` (`create`, `get`, `delete`) and a span with
+`rpc.system=kubernetes`, `rpc.service=Job`; never namespaces, Job names, manifests, tokens
+or URLs. The HTTP request under each operation is also measured by the transport.
+
+`create` is idempotent by name: a Job is named by its manifest (`generateName` is refused),
+and a 409 for an existing name returns `created=False` instead of failing, so a caller that
+got a timeout can call it again. POST is sent once; GET and DELETE retry 502, 503, 504 and
+network failures once. Timeouts, connection failures, 429 and 5xx raise
+DependencyUnavailable. Other rejections (401, 403, 404 on create, 422) raise
+`KubernetesApiError` with the status code and the API's machine-readable reason, never the
+message or the manifest.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import ssl
+import time
+from collections.abc import Callable, Mapping
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any, Literal, cast
+from urllib.parse import quote
+
+import httpx
+from observability_021 import Observability
+from observability_021.config import identifier
+
+from clients_021._observe import observed
+from clients_021.errors import DependencyUnavailable
+from clients_021.http import HttpClient, HttpResponse
+
+SYSTEM = {"rpc.system": "kubernetes", "rpc.service": "Job"}
+SERVICE_ACCOUNT = Path("/var/run/secrets/kubernetes.io/serviceaccount")
+_TOKEN_TTL_SECONDS = 60.0
+_DNS_LABEL = 63
+Phase = Literal["pending", "running", "succeeded", "failed"]
+Propagation = Literal["Background", "Foreground", "Orphan"]
+
+
+class KubernetesApiError(Exception):
+    """A rejected request, with the status code and the API's reason, never its message."""
+
+    def __init__(self, status_code: int, reason: str) -> None:
+        super().__init__(f"Kubernetes API: {reason} (HTTP {status_code})")
+        self.status_code = status_code
+        self.reason = reason
+
+
+@dataclass(frozen=True)
+class JobCreated:
+    name: str
+    created: bool
+
+
+@dataclass(frozen=True)
+class JobStatus:
+    name: str
+    phase: Phase
+    active: int
+    succeeded: int
+    failed: int
+
+
+def _classify(error: BaseException) -> str | None:
+    if isinstance(error, DependencyUnavailable):
+        return error.reason
+    if isinstance(error, KubernetesApiError):
+        if error.status_code == 429:
+            return "throttled"
+        if 500 <= error.status_code < 600:
+            return f"status_{error.status_code}"
+    return None
+
+
+def _name(value: str, *, field: str = "name") -> str:
+    if (
+        not value
+        or len(value) > _DNS_LABEL
+        or not all(
+            char.isascii() and (char.islower() or char.isdigit() or char == "-") for char in value
+        )
+        or value[0] == "-"
+        or value[-1] == "-"
+    ):
+        raise ValueError(f"{field} must be a DNS label (lowercase letters, digits and '-')")
+    return value
+
+
+def _json(response: HttpResponse) -> dict[str, Any]:
+    try:
+        data: Any = response.json()
+    except (ValueError, UnicodeDecodeError):
+        raise KubernetesApiError(response.status_code, "InvalidResponse") from None
+    if not isinstance(data, dict):
+        raise KubernetesApiError(response.status_code, "InvalidResponse")
+    return cast(dict[str, Any], data)
+
+
+def _rejected(response: HttpResponse) -> KubernetesApiError:
+    reason = "Unknown"
+    try:
+        data: Any = response.json()
+        if isinstance(data, dict):
+            value = cast(dict[str, Any], data).get("reason")
+            if isinstance(value, str) and value.isalnum() and len(value) <= 64:
+                reason = value
+    except (ValueError, UnicodeDecodeError):
+        pass
+    return KubernetesApiError(response.status_code, reason)
+
+
+def _count(status: Mapping[str, Any], field: str) -> int:
+    value = status.get(field, 0)
+    return value if isinstance(value, int) and value >= 0 else 0
+
+
+def _status(name: str, job: Mapping[str, Any]) -> JobStatus:
+    raw: Any = job.get("status") or {}
+    status = cast(dict[str, Any], raw) if isinstance(raw, dict) else {}
+    conditions: Any = status.get("conditions") or []
+    true_types: set[str] = set()
+    if isinstance(conditions, list):
+        for condition in cast(list[Any], conditions):
+            if isinstance(condition, dict):
+                entry = cast(dict[str, Any], condition)
+                if entry.get("status") == "True" and isinstance(entry.get("type"), str):
+                    true_types.add(cast(str, entry["type"]))
+    active, succeeded, failed = (_count(status, f) for f in ("active", "succeeded", "failed"))
+    phase: Phase
+    if "Complete" in true_types or "SuccessCriteriaMet" in true_types:
+        phase = "succeeded"
+    elif "Failed" in true_types or "FailureTarget" in true_types:
+        phase = "failed"
+    elif active > 0:
+        phase = "running"
+    else:
+        phase = "pending"
+    return JobStatus(name=name, phase=phase, active=active, succeeded=succeeded, failed=failed)
+
+
+class _FileToken:
+    """Reads a projected service account token, at most once per TTL."""
+
+    def __init__(self, path: Path, *, ttl_seconds: float = _TOKEN_TTL_SECONDS) -> None:
+        self._path = path
+        self._ttl = ttl_seconds
+        self._value = ""
+        self._read_at = float("-inf")
+
+    def __call__(self) -> str:
+        now = time.monotonic()
+        if now - self._read_at >= self._ttl:
+            self._value = self._path.read_text().strip()
+            self._read_at = now
+        return self._value
+
+
+class KubernetesJobsClient:
+    def __init__(
+        self,
+        obs: Observability,
+        *,
+        api_url: str,
+        namespace: str,
+        token: Callable[[], str],
+        timeout_seconds: float = 10.0,
+        dependency: str = "kubernetes",
+        transport: httpx.AsyncBaseTransport | None = None,
+    ) -> None:
+        if not api_url.startswith(("https://", "http://")):
+            raise ValueError("api_url must be an HTTP(S) URL")
+        self.dependency = identifier(dependency)
+        self._obs = obs
+        namespace = quote(_name(namespace, field="namespace"))
+        self._jobs_url = f"{api_url.rstrip('/')}/apis/batch/v1/namespaces/{namespace}/jobs"
+        self._token = token
+        self._closed = False
+        self._http = HttpClient(
+            obs, timeout_seconds=timeout_seconds, retries=1, log_failures=False, transport=transport
+        )
+
+    @classmethod
+    def in_cluster(
+        cls,
+        obs: Observability,
+        *,
+        namespace: str | None = None,
+        timeout_seconds: float = 10.0,
+        dependency: str = "kubernetes",
+        service_account: Path = SERVICE_ACCOUNT,
+    ) -> KubernetesJobsClient:
+        """The pod's own service account; fails before any request if it is not mounted."""
+        host = os.environ.get("KUBERNETES_SERVICE_HOST")
+        port = os.environ.get("KUBERNETES_SERVICE_PORT", "443")
+        if not host:
+            raise RuntimeError("KUBERNETES_SERVICE_HOST is not set: not running in a pod")
+        if ":" in host:
+            host = f"[{host}]"
+        ca = service_account / "ca.crt"
+        token = service_account / "token"
+        if not ca.is_file() or not token.is_file():
+            raise RuntimeError("the service account is not mounted")
+        if namespace is None:
+            namespace = (service_account / "namespace").read_text().strip()
+        transport = httpx.AsyncHTTPTransport(verify=ssl.create_default_context(cafile=str(ca)))
+        return cls(
+            obs,
+            api_url=f"https://{host}:{port}",
+            namespace=namespace,
+            token=_FileToken(token),
+            timeout_seconds=timeout_seconds,
+            dependency=dependency,
+            transport=transport,
+        )
+
+    def _headers(self) -> dict[str, str]:
+        return {"Authorization": f"Bearer {self._token()}", "Accept": "application/json"}
+
+    def _observed(self, operation: str):  # noqa: ANN202 - an async context manager
+        if self._closed:
+            raise RuntimeError("KubernetesJobsClient is closed")
+        return observed(
+            self._obs,
+            family="compute",
+            system=SYSTEM,
+            dependency=self.dependency,
+            operation=operation,
+            classify=_classify,
+        )
+
+    async def create(self, manifest: Mapping[str, Any]) -> JobCreated:
+        """Creates the Job named by `metadata.name`; an existing Job with that name is kept."""
+        metadata: Any = manifest.get("metadata")
+        if not isinstance(metadata, dict):
+            raise ValueError("manifest.metadata is required")
+        fields = cast(dict[str, Any], metadata)
+        if "generateName" in fields:
+            raise ValueError("generateName is not idempotent; set metadata.name")
+        name = fields.get("name")
+        if not isinstance(name, str):
+            raise ValueError("manifest.metadata.name is required")
+        _name(name)
+        if manifest.get("kind", "Job") != "Job":
+            raise ValueError("manifest.kind must be Job")
+        body = {"apiVersion": "batch/v1", "kind": "Job", **manifest}
+        content = json.dumps(body, separators=(",", ":"), allow_nan=False).encode()
+        async with self._observed("create"):
+            response = await self._http.request(
+                "POST",
+                self._jobs_url,
+                dependency=self.dependency,
+                headers={**self._headers(), "Content-Type": "application/json"},
+                content=content,
+            )
+            if response.status_code == 409 and _rejected(response).reason == "AlreadyExists":
+                return JobCreated(name=name, created=False)
+            if response.status_code != 201:
+                raise _rejected(response)
+            created: Any = _json(response).get("metadata")
+            if not isinstance(created, dict) or cast(dict[str, Any], created).get("name") != name:
+                raise KubernetesApiError(response.status_code, "InvalidResponse")
+            return JobCreated(name=name, created=True)
+
+    async def get(self, name: str) -> JobStatus | None:
+        url = f"{self._jobs_url}/{quote(_name(name))}"
+        async with self._observed("get"):
+            response = await self._http.request(
+                "GET", url, dependency=self.dependency, headers=self._headers()
+            )
+            if response.status_code == 404:
+                return None
+            if response.status_code != 200:
+                raise _rejected(response)
+            return _status(name, _json(response))
+
+    async def delete(self, name: str, *, propagation: Propagation = "Background") -> bool:
+        """True when a Job was deleted, False when there was none; its pods follow `propagation`."""
+        url = f"{self._jobs_url}/{quote(_name(name))}"
+        content = json.dumps(
+            {"kind": "DeleteOptions", "apiVersion": "v1", "propagationPolicy": propagation}
+        ).encode()
+        async with self._observed("delete"):
+            response = await self._http.request(
+                "DELETE",
+                url,
+                dependency=self.dependency,
+                headers={**self._headers(), "Content-Type": "application/json"},
+                content=content,
+            )
+            if response.status_code == 404:
+                return False
+            if response.status_code not in (200, 202):
+                raise _rejected(response)
+            return True
+
+    async def aclose(self) -> None:
+        """Release the HTTP pool and injected transport after in-flight calls finish."""
+        self._closed = True
+        await self._http.aclose()

clients_021/llm/__init__.py

@@ -0,0 +1 @@
+"""Model inference clients. Import providers explicitly; SDKs are optional extras."""

clients_021/llm/bedrock.py

@@ -0,0 +1,250 @@
+"""Amazon Bedrock Runtime, observed, for SDKs that take a boto3 client.
+
+    from clients_021.llm.bedrock import BedrockRuntime
+    from langchain_aws import ChatBedrockConverse
+
+    runtime = BedrockRuntime.create(obs, region="us-east-1")
+    model = ChatBedrockConverse(model=settings.inference_profile, client=runtime)
+
+`BedrockRuntime` has the same call signatures as boto3's `bedrock-runtime` client for
+`converse`, `converse_stream`, `invoke_model`, `invoke_model_with_response_stream` and
+`count_tokens`, and passes every other attribute (`meta`, `exceptions`, ...) through, so an
+SDK can use it where it expects the boto3 client. Those five operations record
+`llm.client.operation.*` and a span with `gen_ai.system=aws.bedrock`. Calls are synchronous,
+like boto3's; an SDK that is async runs them in its own worker threads.
+
+A streaming operation lasts until its event stream is exhausted, fails or is closed: the
+response's `stream` (Converse) or `body` (InvokeModel) is wrapped, and closing it early ends
+the operation as `ok` with a `dependency.stream_closed` log. Converse and ConverseStream also
+add the reported token usage to `llm.client.token.usage` (`token.type`: `input`, `output`,
+`cache_read`, `cache_write`); InvokeModel bodies are model-specific and are not read.
+
+Timeouts, connection failures, throttling (also mid-stream), 5xx and the stream errors
+Bedrock documents as retryable (a model still loading included) raise
+DependencyUnavailable. Access denied, validation errors, a missing model and quota errors
+that need a configuration change keep their botocore type. botocore's standard retry mode
+makes up to 3 attempts per call: a retried model call is billed again. Never recorded:
+model ids or ARNs, prompts, outputs, tool arguments, request ids or guardrail details.
+"""
+
+from __future__ import annotations
+
+import asyncio
+from collections.abc import Callable, Iterator
+from typing import TYPE_CHECKING, Any
+
+import boto3
+from botocore.exceptions import ClientError
+from observability_021 import Observability, get_logger
+from observability_021.config import identifier
+
+from clients_021 import _aws
+from clients_021._observe import Call
+
+if TYPE_CHECKING:
+    from mypy_boto3_bedrock_runtime import BedrockRuntimeClient
+else:
+    BedrockRuntimeClient = Any
+
+SYSTEM = {"rpc.system": "aws-api", "rpc.service": "BedrockRuntime", "gen_ai.system": "aws.bedrock"}
+# Exception names Bedrock sends inside an event stream (lower camel case) and the HTTP ones it
+# documents as retryable. Their status code is not always in the response metadata.
+_STREAM_RETRYABLE = {
+    "throttlingException": "throttled",
+    "serviceUnavailableException": "status_503",
+    "internalServerException": "status_500",
+    "modelStreamErrorException": "status_500",
+    "ServiceUnavailableException": "status_503",
+    "InternalServerException": "status_500",
+    "ModelStreamErrorException": "status_500",
+    # The model is still loading; Bedrock documents it as safe to retry.
+    "ModelNotReadyException": "throttled",
+}
+_USAGE = {
+    "inputTokens": "input",
+    "outputTokens": "output",
+    "cacheReadInputTokens": "cache_read",
+    "cacheWriteInputTokens": "cache_write",
+}
+
+logger = get_logger(__name__)
+
+
+def classify(error: BaseException) -> str | None:
+    reason = _aws.classify(error)
+    if reason is None and isinstance(error, ClientError):
+        reason = _STREAM_RETRYABLE.get(_aws.error_code(error))
+    return reason
+
+
+class _ObservedStream:
+    """An event stream whose operation ends when it is exhausted, fails or is closed.
+
+    Leaving the loop early (`break`) or calling `close()` is a deliberate partial read: the
+    operation ends as `ok` and `dependency.stream_closed` is logged at INFO.
+    """
+
+    def __init__(
+        self,
+        inner: Any,
+        call: Call,
+        *,
+        dependency: str,
+        operation: str,
+        on_event: Callable[[Any], None] | None,
+    ) -> None:
+        self._inner = inner
+        self._call = call
+        self._dependency = dependency
+        self._operation = operation
+        self._on_event = on_event
+
+    def __iter__(self) -> Iterator[Any]:
+        try:
+            for event in self._inner:
+                if self._on_event is not None:
+                    self._on_event(event)
+                yield event
+        except GeneratorExit:
+            self._closed_early()
+            raise
+        except BaseException as error:
+            self._call.fail(error)
+        self._call.ok()
+
+    def _closed_early(self) -> None:
+        if not self._call.done:
+            logger.info(
+                "dependency.stream_closed", dependency=self._dependency, operation=self._operation
+            )
+            self._call.ok()
+
+    def close(self) -> None:
+        try:
+            self._inner.close()
+        finally:
+            self._closed_early()
+
+    def __getattr__(self, name: str) -> Any:
+        return getattr(self._inner, name)
+
+
+class BedrockRuntime:
+    def __init__(
+        self, obs: Observability, client: BedrockRuntimeClient, *, dependency: str = "bedrock"
+    ) -> None:
+        self._obs = obs
+        self._client = client
+        self.dependency = identifier(dependency)
+        self._tokens = obs.meter.create_counter("llm.client.token.usage", unit="{token}")
+
+    @classmethod
+    def create(
+        cls,
+        obs: Observability,
+        *,
+        region: str | None = None,
+        endpoint_url: str | None = None,
+        dependency: str = "bedrock",
+        connect_seconds: float = 3.0,
+        read_seconds: float = 300.0,
+    ) -> BedrockRuntime:
+        """`read_seconds` bounds each wait for bytes; a long generation streams for longer."""
+        client: BedrockRuntimeClient = boto3.client(  # pyright: ignore[reportUnknownMemberType]
+            "bedrock-runtime",
+            region_name=region,
+            endpoint_url=endpoint_url,
+            config=_aws.config(connect_seconds=connect_seconds, read_seconds=read_seconds),
+        )
+        return cls(obs, client, dependency=dependency)
+
+    def _call(self, operation: str) -> Call:
+        return Call(
+            self._obs,
+            family="llm",
+            system=SYSTEM,
+            dependency=self.dependency,
+            operation=operation,
+            classify=classify,
+        )
+
+    def _record_usage(self, operation: str, usage: Any) -> None:
+        if not isinstance(usage, dict):
+            return
+        for field, token_type in _USAGE.items():
+            value: Any = usage.get(field)  # pyright: ignore[reportUnknownMemberType, reportUnknownVariableType]
+            if isinstance(value, int) and value > 0:
+                self._tokens.add(
+                    value,
+                    {
+                        "dependency": self.dependency,
+                        "operation": operation,
+                        "token.type": token_type,
+                    },
+                )
+
+    def _unary(self, operation: str, method: Callable[..., Any], params: dict[str, Any]) -> Any:
+        call = self._call(operation)
+        try:
+            response = method(**params)
+        except BaseException as error:
+            call.fail(error)
+        call.ok()
+        if operation == "converse":
+            self._record_usage(operation, response.get("usage"))
+        return response
+
+    def _streaming(
+        self, operation: str, method: Callable[..., Any], params: dict[str, Any], key: str
+    ) -> Any:
+        call = self._call(operation)
+        try:
+            response = method(**params)
+        except BaseException as error:
+            call.fail(error)
+
+        def on_event(event: Any) -> None:
+            metadata: Any = None
+            if isinstance(event, dict):
+                metadata = event.get("metadata")  # pyright: ignore[reportUnknownMemberType, reportUnknownVariableType]
+            if isinstance(metadata, dict):
+                self._record_usage(operation, metadata.get("usage"))  # pyright: ignore[reportUnknownMemberType]
+
+        stream = _ObservedStream(
+            response[key],
+            call,
+            dependency=self.dependency,
+            operation=operation,
+            on_event=on_event if operation == "converse_stream" else None,
+        )
+        return {**response, key: stream}
+
+    def converse(self, **params: Any) -> Any:
+        return self._unary("converse", self._client.converse, params)
+
+    def converse_stream(self, **params: Any) -> Any:
+        return self._streaming("converse_stream", self._client.converse_stream, params, "stream")
+
+    def invoke_model(self, **params: Any) -> Any:
+        return self._unary("invoke_model", self._client.invoke_model, params)
+
+    def invoke_model_with_response_stream(self, **params: Any) -> Any:
+        return self._streaming(
+            "invoke_model_with_response_stream",
+            self._client.invoke_model_with_response_stream,
+            params,
+            "body",
+        )
+
+    def count_tokens(self, **params: Any) -> Any:
+        return self._unary("count_tokens", self._client.count_tokens, params)
+
+    def close(self) -> None:
+        self._client.close()
+
+    async def aclose(self) -> None:
+        """Release the underlying SDK connection pools after in-flight calls finish."""
+        await asyncio.to_thread(self._client.close)
+
+    def __getattr__(self, name: str) -> Any:
+        return getattr(self._client, name)

clients_021/secrets/__init__.py

@@ -0,0 +1 @@
+"""Secret stores. Import providers explicitly; SDKs are optional extras."""

clients_021/secrets/secrets_manager.py

@@ -0,0 +1,127 @@
+"""AWS Secrets Manager, observed: read one secret value by id, version id or stage.
+
+    from clients_021.secrets.secrets_manager import SecretsManagerClient
+
+    secrets = SecretsManagerClient.create(obs, region="us-east-1")
+    secret = await secrets.get("connections/brain")
+    token = secret.text()                       # SecretString; secret.binary() for SecretBinary
+    previous = await secrets.get("connections/brain", version_stage="AWSPREVIOUS")
+
+Needs the `aws` extra. Calls record `secrets.client.operation.*` with `operation=get` and a
+span with `rpc.service=SecretsManager`; never secret ids, ARNs, version ids, stages or values.
+Timeouts, connection failures, throttling and 5xx raise DependencyUnavailable. A missing
+secret (`ResourceNotFoundException`), access denied, a secret KMS cannot decrypt
+(`DecryptionFailure`) or one scheduled for deletion (`InvalidRequestException`) keep their
+botocore type: they are misconfiguration, not outages. Reads are idempotent; botocore's
+standard retry mode makes up to 3 attempts. The client does not cache: every call is a
+billed request, so callers that read the same secret often keep their own short-lived copy.
+"""
+
+from __future__ import annotations
+
+import asyncio
+from dataclasses import dataclass
+from typing import TYPE_CHECKING, Any
+
+import boto3
+from observability_021 import Observability
+from observability_021.config import identifier
+
+from clients_021 import _aws
+from clients_021._observe import observed
+
+if TYPE_CHECKING:
+    from mypy_boto3_secretsmanager import SecretsManagerClient as SecretsManagerSdk
+else:
+    SecretsManagerSdk = Any
+
+SYSTEM = {"rpc.system": "aws-api", "rpc.service": "SecretsManager"}
+
+
+@dataclass(frozen=True, repr=False)
+class Secret:
+    """One secret version. `value` is the SecretString or the SecretBinary; never printed."""
+
+    value: str | bytes
+    version_id: str
+
+    def text(self) -> str:
+        if not isinstance(self.value, str):
+            raise TypeError("this secret is binary; use binary()")
+        return self.value
+
+    def binary(self) -> bytes:
+        if not isinstance(self.value, bytes):
+            raise TypeError("this secret is a string; use text()")
+        return self.value
+
+    def __repr__(self) -> str:
+        return "Secret(<redacted>)"
+
+
+class SecretsManagerClient:
+    def __init__(
+        self, obs: Observability, client: SecretsManagerSdk, *, dependency: str = "secrets"
+    ) -> None:
+        self._obs = obs
+        self._client = client
+        self.dependency = identifier(dependency)
+
+    @classmethod
+    def create(
+        cls,
+        obs: Observability,
+        *,
+        region: str | None = None,
+        endpoint_url: str | None = None,
+        dependency: str = "secrets",
+        timeout_seconds: float = 5.0,
+    ) -> SecretsManagerClient:
+        client: SecretsManagerSdk = boto3.client(  # pyright: ignore[reportUnknownMemberType]
+            "secretsmanager",
+            region_name=region,
+            endpoint_url=endpoint_url,
+            config=_aws.config(
+                connect_seconds=min(timeout_seconds, 3.0), read_seconds=timeout_seconds
+            ),
+        )
+        return cls(obs, client, dependency=dependency)
+
+    async def get(
+        self,
+        secret_id: str,
+        *,
+        version_id: str | None = None,
+        version_stage: str | None = None,
+    ) -> Secret:
+        """The current version (`AWSCURRENT`) unless a version id or a stage is given."""
+        if not secret_id:
+            raise ValueError("secret_id is required")
+        if version_id is not None and version_stage is not None:
+            raise ValueError("pass version_id or version_stage, not both")
+        params: dict[str, str] = {"SecretId": secret_id}
+        if version_id is not None:
+            params["VersionId"] = version_id
+        if version_stage is not None:
+            params["VersionStage"] = version_stage
+        async with observed(
+            self._obs,
+            family="secrets",
+            system=SYSTEM,
+            dependency=self.dependency,
+            operation="get",
+            classify=_aws.classify,
+        ):
+            response = await asyncio.to_thread(self._client.get_secret_value, **params)
+        value: str | bytes
+        if "SecretString" in response:
+            value = response["SecretString"]
+        elif "SecretBinary" in response:
+            value = response["SecretBinary"]
+        else:
+            raise ValueError("the secret version has no value")
+        return Secret(value=value, version_id=response.get("VersionId", ""))
+
+    async def aclose(self) -> None:
+        """Release the underlying SDK connection pools after in-flight calls finish."""
+        await asyncio.to_thread(self._client.close)