spanlight 0.1.0__tar.gz → 0.1.2__tar.gz

{spanlight-0.1.0 → spanlight-0.1.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: spanlight
-Version: 0.1.0
+Version: 0.1.2
 Summary: Python SDK for Spanlight, open-source observability for LLM applications.
 Author: Dheeraj
 License-Expression: MIT
@@ -294,17 +294,19 @@ run makes real provider calls. In a GitHub workflow use the
 
 For every LLM call:
 
-| Field                       | Source                                                                                                                            |
-| --------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
-| `provider`                  | `openai` / `anthropic`                                                                                                            |
-| `model`                     | Model from the response (for example `gpt-4o-mini-2024-07-18`). Falls back to the requested model.                                |
-| `usage.input_tokens`        | Total prompt tokens, **including** cached tokens. For Anthropic this is input + cache reads + cache writes.                       |
-| `usage.output_tokens`       | Completion tokens                                                                                                                 |
-| `usage.cached_tokens`       | Prompt tokens served from the provider's prompt cache                                                                             |
-| `time_to_first_token_ms`    | Streaming calls only                                                                                                              |
-| `input`                     | Request parameters such as `messages`, `system` and `temperature`. API keys, headers, `extra_*` and `timeout` are never recorded. |
-| `output`                    | The assistant message, including tool calls                                                                                       |
-| `status` / `status_message` | `error` plus the provider's message when the call fails                                                                           |
+| Field                         | Source                                                                                                                                                                    |
+| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `provider`                    | `openai` / `anthropic`                                                                                                                                                    |
+| `model`                       | Model from the response (for example `gpt-4o-mini-2024-07-18`). Falls back to the requested model.                                                                        |
+| `usage.input_tokens`          | Total prompt tokens, **including** cached tokens. For Anthropic this is input + cache reads + cache writes.                                                               |
+| `usage.output_tokens`         | Completion tokens                                                                                                                                                         |
+| `usage.cached_tokens`         | Prompt tokens served from the provider's prompt cache                                                                                                                     |
+| `usage.cache_write_tokens`    | Prompt tokens written to the prompt cache (Anthropic `cache_creation_input_tokens`). Sent only when the provider reports it.                                              |
+| `usage.cache_write_1h_tokens` | The part of `cache_write_tokens` written with Anthropic's 1-hour TTL (`cache_creation.ephemeral_1h_input_tokens`); the rest are 5-minute writes. Sent only when reported. |
+| `time_to_first_token_ms`      | Streaming calls only                                                                                                                                                      |
+| `input`                       | Request parameters such as `messages`, `system` and `temperature`. API keys, headers, `extra_*` and `timeout` are never recorded.                                         |
+| `output`                      | The assistant message, including tool calls                                                                                                                               |
+| `status` / `status_message`   | `error` plus the provider's message when the call fails                                                                                                                   |
 
 The Spanlight server also redacts common secret patterns from
 payloads on ingestion. Set `capture_payloads = false` on a project to drop
@@ -313,6 +315,10 @@ specific function, use `@observe(capture_input=False, capture_output=False)`.
 
 ## API reference
 
+The full API, with every signature, default and error, is in the
+[Python SDK reference](https://github.com/dheerajkarwasra/spanlight/blob/main/docs-site/src/content/docs/reference/python-sdk.mdx).
+The sections below are a summary.
+
 ### `init(**options) -> Spanlight`
 
 Configures the global client used by `observe`, `span` and the wrappers.
@@ -358,7 +364,8 @@ with spanlight.span("rerank", kind="retrieval") as span:
 
 `Span` methods: `set_input`, `set_output`, `set_attribute`, `set_attributes`,
 `set_model(model, provider=...)`, `set_usage(input_tokens=, output_tokens=,
-cached_tokens=)`, `set_finish_reason(reason)`, `set_request_hash(hash)`,
+cached_tokens=, cache_write_tokens=, cache_write_1h_tokens=)`,
+`set_finish_reason(reason)`, `set_request_hash(hash)`,
 `mark_first_token()` and `record_error(exc_or_message)`. An
 exception that escapes the block marks the span as failed and is re-raised
 unchanged.

{spanlight-0.1.0 → spanlight-0.1.2}/README.md

@@ -267,17 +267,19 @@ run makes real provider calls. In a GitHub workflow use the
 
 For every LLM call:
 
-| Field                       | Source                                                                                                                            |
-| --------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
-| `provider`                  | `openai` / `anthropic`                                                                                                            |
-| `model`                     | Model from the response (for example `gpt-4o-mini-2024-07-18`). Falls back to the requested model.                                |
-| `usage.input_tokens`        | Total prompt tokens, **including** cached tokens. For Anthropic this is input + cache reads + cache writes.                       |
-| `usage.output_tokens`       | Completion tokens                                                                                                                 |
-| `usage.cached_tokens`       | Prompt tokens served from the provider's prompt cache                                                                             |
-| `time_to_first_token_ms`    | Streaming calls only                                                                                                              |
-| `input`                     | Request parameters such as `messages`, `system` and `temperature`. API keys, headers, `extra_*` and `timeout` are never recorded. |
-| `output`                    | The assistant message, including tool calls                                                                                       |
-| `status` / `status_message` | `error` plus the provider's message when the call fails                                                                           |
+| Field                         | Source                                                                                                                                                                    |
+| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `provider`                    | `openai` / `anthropic`                                                                                                                                                    |
+| `model`                       | Model from the response (for example `gpt-4o-mini-2024-07-18`). Falls back to the requested model.                                                                        |
+| `usage.input_tokens`          | Total prompt tokens, **including** cached tokens. For Anthropic this is input + cache reads + cache writes.                                                               |
+| `usage.output_tokens`         | Completion tokens                                                                                                                                                         |
+| `usage.cached_tokens`         | Prompt tokens served from the provider's prompt cache                                                                                                                     |
+| `usage.cache_write_tokens`    | Prompt tokens written to the prompt cache (Anthropic `cache_creation_input_tokens`). Sent only when the provider reports it.                                              |
+| `usage.cache_write_1h_tokens` | The part of `cache_write_tokens` written with Anthropic's 1-hour TTL (`cache_creation.ephemeral_1h_input_tokens`); the rest are 5-minute writes. Sent only when reported. |
+| `time_to_first_token_ms`      | Streaming calls only                                                                                                                                                      |
+| `input`                       | Request parameters such as `messages`, `system` and `temperature`. API keys, headers, `extra_*` and `timeout` are never recorded.                                         |
+| `output`                      | The assistant message, including tool calls                                                                                                                               |
+| `status` / `status_message`   | `error` plus the provider's message when the call fails                                                                                                                   |
 
 The Spanlight server also redacts common secret patterns from
 payloads on ingestion. Set `capture_payloads = false` on a project to drop
@@ -286,6 +288,10 @@ specific function, use `@observe(capture_input=False, capture_output=False)`.
 
 ## API reference
 
+The full API, with every signature, default and error, is in the
+[Python SDK reference](https://github.com/dheerajkarwasra/spanlight/blob/main/docs-site/src/content/docs/reference/python-sdk.mdx).
+The sections below are a summary.
+
 ### `init(**options) -> Spanlight`
 
 Configures the global client used by `observe`, `span` and the wrappers.
@@ -331,7 +337,8 @@ with spanlight.span("rerank", kind="retrieval") as span:
 
 `Span` methods: `set_input`, `set_output`, `set_attribute`, `set_attributes`,
 `set_model(model, provider=...)`, `set_usage(input_tokens=, output_tokens=,
-cached_tokens=)`, `set_finish_reason(reason)`, `set_request_hash(hash)`,
+cached_tokens=, cache_write_tokens=, cache_write_1h_tokens=)`,
+`set_finish_reason(reason)`, `set_request_hash(hash)`,
 `mark_first_token()` and `record_error(exc_or_message)`. An
 exception that escapes the block marks the span as failed and is re-raised
 unchanged.

{spanlight-0.1.0 → spanlight-0.1.2}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "spanlight"
-version = "0.1.0"
+version = "0.1.2"
 description = "Python SDK for Spanlight, open-source observability for LLM applications."
 readme = "README.md"
 license = "MIT"

{spanlight-0.1.0 → spanlight-0.1.2}/src/spanlight/_experiments.py

@@ -120,9 +120,12 @@ def run_experiment(
         experiment, version = _prepare(
             api, dataset, prompt, evaluators, credential_id, model, max_cost_usd
         )
-        url = f"{host.rstrip('/')}/{org_of(api)}/{project_id}/experiments/{experiment['id']}"
+        experiment_url = (
+            f"{host.rstrip('/')}/{org_of(api)}/{project_id}/experiments/{experiment['id']}"
+        )
         wait_idle(api, experiment["id"], deadline, poll_s, sleep, clock)
         run = start_run(api, experiment["id"], version.version_id, env)
+        url = f"{experiment_url}/runs/{run['id']}"
         try:
             final = wait(api, experiment["id"], run["id"], deadline, poll_s, sleep, clock)
         except ExperimentError as error:

{spanlight-0.1.0 → spanlight-0.1.2}/src/spanlight/_experiments_report.py

@@ -2,7 +2,7 @@
 
 from __future__ import annotations
 
-from decimal import Decimal
+from decimal import ROUND_HALF_UP, Decimal, InvalidOperation
 from typing import Any
 
 _PLACES = Decimal("0.000001")
@@ -65,6 +65,18 @@ def _number(value: Any) -> str:
     return _UNKNOWN if value is None else str(value)
 
 
+def _milliseconds(value: Any) -> str:
+    """A latency in whole milliseconds; the server sends a raw float such as ``1234.5678``."""
+    if value is None:
+        return _UNKNOWN
+    if isinstance(value, bool) or not isinstance(value, int | float):
+        return str(value)
+    try:
+        return str(int(Decimal(str(value)).quantize(Decimal(1), rounding=ROUND_HALF_UP)))
+    except (InvalidOperation, ValueError, OverflowError):
+        return _UNKNOWN
+
+
 def render_markdown(
     *,
     title: str,
@@ -89,8 +101,8 @@ def render_markdown(
         lines += [
             f"Items {_number(totals.get('items'))}, errors {_number(totals.get('errors'))}, "
             f"cost {_number(totals.get('cost_usd'))} USD, "
-            f"p50 {_number(totals.get('latency_p50_ms'))} ms, "
-            f"p95 {_number(totals.get('latency_p95_ms'))} ms.",
+            f"p50 {_milliseconds(totals.get('latency_p50_ms'))} ms, "
+            f"p95 {_milliseconds(totals.get('latency_p95_ms'))} ms.",
             "",
             *_score_rows(totals, fail_below),
             "",

{spanlight-0.1.0 → spanlight-0.1.2}/src/spanlight/_exporter.py

@@ -11,6 +11,7 @@ Design goals, in priority order:
 3. **Deliver what we can.** Batches are retried with jittered exponential
    backoff on network errors, ``429`` and ``5xx``, honouring ``Retry-After``.
    Other ``4xx`` responses are permanent failures and are logged, not retried.
+   The sending itself lives in :mod:`spanlight._exporter_delivery`.
 
 Spans and scores share one queue (and so one ``max_queue`` bound). Each cycle
 takes up to ``batch_size`` items, sends the spans first and the scores after,
@@ -19,9 +20,6 @@ so a score recorded after its span never reaches the server before it.
 
 from __future__ import annotations
 
-import email.utils
-import gzip
-import json
 import logging
 import os
 import random
@@ -29,32 +27,15 @@ import threading
 import time
 from collections import deque
 from collections.abc import Callable
-from dataclasses import dataclass
-from typing import Any
 
 import httpx
 
-from ._serialize import JsonValue
-from ._version import __version__
-
-logger = logging.getLogger("spanlight")
-
-SpanPayload = dict[str, JsonValue]
-
-_MAX_LOGGED_REJECTIONS = 10
+from ._exporter_delivery import SCORES, SPANS, Delivery, SpanPayload, parse_retry_after
 
+__all__ = ["BatchExporter", "SpanPayload", "parse_retry_after"]
 
-@dataclass(frozen=True)
-class _Target:
-    """One endpoint the exporter delivers to."""
-
-    url: str
-    body_key: str  # the list's key in the request body, and the plural noun in logs
-    id_field: str  # the payload field logged when the server rejects an item
-
+logger = logging.getLogger("spanlight")
 
-_SPANS = "spans"
-_SCORES = "scores"
 _Item = tuple[str, SpanPayload]
 
 
@@ -100,42 +81,32 @@ class BatchExporter:
         base_backoff: float = 0.5,
         max_backoff: float = 30.0,
     ) -> None:
-        self._targets: dict[str, _Target] = {_SPANS: _Target(url, _SPANS, "span_id")}
-        if scores_url is not None:
-            self._targets[_SCORES] = _Target(scores_url, _SCORES, "id")
-        self._scores_unsupported_logged = False
-        self._headers = {
-            "Authorization": f"Bearer {api_key}",
-            "Content-Type": "application/json",
-            "User-Agent": f"spanlight-python/{__version__}",
-        }
-        if gzip_body:
-            self._headers["Content-Encoding"] = "gzip"
+        self._delivery = Delivery(
+            url=url,
+            api_key=api_key,
+            scores_url=scores_url,
+            gzip_body=gzip_body,
+            timeout=timeout,
+            max_retries=max_retries,
+            transport=transport,
+            sleep=sleep,
+            jitter=jitter,
+            base_backoff=base_backoff,
+            max_backoff=max_backoff,
+        )
         self._batch_size = batch_size
         self._flush_interval = flush_interval
         self._max_queue = max_queue
-        self._gzip = gzip_body
-        self._timeout = timeout
-        self._max_retries = max_retries
-        self._transport = transport
-        self._sleep = sleep if sleep is not None else self._interruptible_sleep
-        self._jitter = jitter
-        self._base_backoff = base_backoff
-        self._max_backoff = max_backoff
 
         self._condition = threading.Condition()
         self._queue: deque[_Item] = deque()
         self._in_flight = 0
         self._flush_requested = False
         self._closed = False
-        self._abort_retries = threading.Event()
         self._worker: threading.Thread | None = None
-        self._client: httpx.Client | None = None
         self._pid = os.getpid()
 
         self._dropped = 0
-        self._failed = 0
-        self._sent = 0
 
     # ------------------------------------------------------------------ #
     # Public API (called from user threads)
@@ -149,23 +120,23 @@ class BatchExporter:
     @property
     def failed_count(self) -> int:
         """Spans and scores lost because their batch could not be delivered."""
-        return self._failed
+        return self._delivery.failed
 
     @property
     def sent_count(self) -> int:
         """Spans and scores delivered and accepted by the server."""
-        return self._sent
+        return self._delivery.sent
 
     def export(self, span: SpanPayload) -> None:
         """Enqueue one serialized span. Never blocks on I/O and never raises."""
-        self._enqueue(_SPANS, span)
+        self._enqueue(SPANS, span)
 
     def export_score(self, score: SpanPayload) -> None:
         """Enqueue one serialized score. Never blocks on I/O and never raises."""
-        if _SCORES not in self._targets:
+        if not self._delivery.accepts(SCORES):
             logger.debug("No score endpoint configured; dropping a score")
             return
-        self._enqueue(_SCORES, score)
+        self._enqueue(SCORES, score)
 
     def _enqueue(self, kind: str, payload: SpanPayload) -> None:
         try:
@@ -227,12 +198,11 @@ class BatchExporter:
             with self._condition:
                 self._closed = True
                 self._condition.notify_all()
-            self._abort_retries.set()
+            self._delivery.abort_retries()
             worker = self._worker
             if worker is not None and worker is not threading.current_thread():
                 worker.join(max(deadline - time.monotonic(), 0.0))
-            if self._client is not None:
-                self._client.close()
+            self._delivery.close()
         except Exception:
             logger.debug("Shutdown failed", exc_info=True)
 
@@ -249,7 +219,7 @@ class BatchExporter:
         self._queue = deque()
         self._in_flight = 0
         self._worker = None
-        self._client = None
+        self._delivery.forget_client()
 
     def _ensure_worker(self) -> None:
         with self._condition:
@@ -270,10 +240,10 @@ class BatchExporter:
             if batch is None:
                 return
             try:
-                for kind in (_SPANS, _SCORES):
+                for kind in (SPANS, SCORES):
                     payloads = [payload for item_kind, payload in batch if item_kind == kind]
                     if payloads:
-                        self._send_guarded(self._targets[kind], payloads)
+                        self._delivery.send(kind, payloads)
             finally:
                 with self._condition:
                     self._in_flight -= len(batch)
@@ -303,160 +273,3 @@ class BatchExporter:
         if self._closed or self._flush_requested:
             return False
         return len(self._queue) < self._batch_size
-
-    def _http_client(self) -> httpx.Client:
-        if self._client is None:
-            self._client = httpx.Client(transport=self._transport, timeout=self._timeout)
-        return self._client
-
-    def _send_guarded(self, target: _Target, batch: list[SpanPayload]) -> None:
-        """Send one target's part of a cycle; a crash here never stops the next part."""
-        try:
-            self._send_batch(target, batch)
-        except Exception:
-            self._failed += len(batch)
-            logger.warning(
-                "Unexpected error exporting %d %s", len(batch), target.body_key, exc_info=True
-            )
-
-    def _send_batch(self, target: _Target, batch: list[SpanPayload]) -> None:
-        if not batch:
-            return
-        body = json.dumps({target.body_key: batch}, separators=(",", ":"), default=str).encode()
-        if self._gzip:
-            body = gzip.compress(body)
-
-        for attempt in range(self._max_retries + 1):
-            retry_after: float | None = None
-            try:
-                response = self._http_client().post(target.url, content=body, headers=self._headers)
-            except httpx.HTTPError as exc:
-                failure = f"{type(exc).__name__}: {exc}"
-            else:
-                status = response.status_code
-                if 200 <= status < 300:
-                    self._handle_success(target, response, batch)
-                    return
-                if status == 413 and len(batch) > 1:
-                    self._split_and_send(target, batch)
-                    return
-                if not _is_retryable(status):
-                    self._failed += len(batch)
-                    self._log_rejected_batch(target, response, len(batch))
-                    return
-                failure = f"HTTP {status}"
-                retry_after = parse_retry_after(response.headers.get("Retry-After"))
-
-            if attempt >= self._max_retries or self._abort_retries.is_set():
-                self._failed += len(batch)
-                logger.warning(
-                    "Dropping %d %s after %d attempts (%s)",
-                    len(batch),
-                    target.body_key,
-                    attempt + 1,
-                    failure,
-                )
-                return
-            delay = self._retry_delay(attempt, retry_after)
-            logger.debug("Export failed (%s); retrying in %.2fs", failure, delay)
-            self._sleep(delay)
-
-    def _split_and_send(self, target: _Target, batch: list[SpanPayload]) -> None:
-        """Retry an oversized (HTTP 413) batch as two smaller ones."""
-        middle = len(batch) // 2
-        self._send_batch(target, batch[:middle])
-        self._send_batch(target, batch[middle:])
-
-    def _log_rejected_batch(self, target: _Target, response: httpx.Response, size: int) -> None:
-        if target.body_key == _SCORES and response.status_code == 404:
-            # A server older than score ingestion: say so once instead of on every batch.
-            if not self._scores_unsupported_logged:
-                self._scores_unsupported_logged = True
-                logger.warning(
-                    "This Spanlight server does not accept scores (HTTP 404); dropping them"
-                )
-            return
-        logger.error(
-            "Spanlight rejected a batch of %d %s: HTTP %d %s",
-            size,
-            target.body_key,
-            response.status_code,
-            _short_body(response),
-        )
-
-    def _handle_success(
-        self, target: _Target, response: httpx.Response, batch: list[SpanPayload]
-    ) -> None:
-        batch_size = len(batch)
-        try:
-            result: Any = response.json()
-        except ValueError:
-            self._sent += batch_size
-            return
-        rejected = result.get("rejected") if isinstance(result, dict) else None
-        if not isinstance(rejected, list) or not rejected:
-            self._sent += batch_size
-            return
-
-        self._sent += batch_size - len(rejected)
-        self._failed += len(rejected)
-        logger.warning(
-            "Spanlight rejected %d of %d %s",
-            len(rejected),
-            batch_size,
-            target.body_key,
-        )
-        for item in rejected[:_MAX_LOGGED_REJECTIONS]:
-            if isinstance(item, dict):
-                index = item.get("index")
-                sent = batch[index] if isinstance(index, int) and 0 <= index < batch_size else {}
-                logger.warning(
-                    "  %s %s (index %s): %s",
-                    target.body_key[:-1],
-                    item.get(target.id_field) or sent.get(target.id_field),
-                    index,
-                    item.get("reason"),
-                )
-
-    def _retry_delay(self, attempt: int, retry_after: float | None) -> float:
-        """Return seconds to wait before retry number ``attempt + 1``.
-
-        ``Retry-After`` wins when present. Otherwise use "equal jitter"
-        exponential backoff: half of the exponential step is fixed and the
-        other half random, which spreads retries out without ever retrying
-        immediately.
-        """
-        if retry_after is not None:
-            return min(max(retry_after, 0.0), self._max_backoff)
-        step = min(self._max_backoff, self._base_backoff * (2.0**attempt))
-        return step / 2 + self._jitter() * step / 2
-
-    def _interruptible_sleep(self, seconds: float) -> None:
-        self._abort_retries.wait(seconds)
-
-
-def _is_retryable(status: int) -> bool:
-    return status == 429 or status >= 500
-
-
-def parse_retry_after(value: str | None) -> float | None:
-    """Parse a ``Retry-After`` header (delta-seconds or HTTP-date) into seconds."""
-    if value is None:
-        return None
-    value = value.strip()
-    try:
-        return max(float(value), 0.0)
-    except ValueError:
-        pass
-    try:
-        moment = email.utils.parsedate_to_datetime(value)
-    except (TypeError, ValueError):
-        return None
-    return max(moment.timestamp() - time.time(), 0.0)
-
-
-def _short_body(response: httpx.Response) -> str:
-    try:
-        return response.text[:500]
-    except Exception:
-        return "<unreadable body>"