errorbar-tracing 0.2.0__tar.gz

errorbar_tracing-0.2.0/.gitignore

@@ -0,0 +1,9 @@
+node_modules/
+dist/
+__pycache__/
+*.egg-info/
+.venv/
+.pytest_cache/
+
+.env
+*.env

errorbar_tracing-0.2.0/PKG-INFO

@@ -0,0 +1,103 @@
+Metadata-Version: 2.5
+Name: errorbar-tracing
+Version: 0.2.0
+Summary: errorbar tracing — standard OpenTelemetry, curated. One install, one line; eject anytime, your spans don't change.
+Project-URL: Repository, https://github.com/omnia-v/omnia-tracing
+License-Expression: Apache-2.0
+Requires-Python: >=3.10
+Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.27
+Requires-Dist: opentelemetry-instrumentation-anthropic>=0.30
+Requires-Dist: opentelemetry-instrumentation-google-generativeai>=0.30
+Requires-Dist: opentelemetry-instrumentation-langchain>=0.30
+Requires-Dist: opentelemetry-instrumentation-openai>=0.30
+Requires-Dist: opentelemetry-sdk>=1.27
+Description-Content-Type: text/markdown
+
+# errorbar-tracing
+
+**Standard OpenTelemetry, curated.** One install, one line, and your LLM traffic streams to [errorbar](https://platform.omnia-voice.com) — where you grade it, calibrate a judge against your own standards, and find out **with confidence intervals** whether a cheaper model holds up on your production traffic.
+
+This package contains **no instrumentation code of its own**. It pins and configures the ecosystem's standard OpenTelemetry instrumentations — which gives it a property no other tracing SDK offers: **you can uninstall it without losing your instrumentation.** The identical setup in vanilla OTel is documented below; your spans are byte-for-byte the same either way, and nothing proprietary ever goes on the wire.
+
+## Install
+
+```bash
+pip install errorbar-tracing
+```
+
+## Use
+
+Call once at startup, **before constructing any LLM client**:
+
+```python
+from errorbar_tracing import setup
+
+tracing = setup()          # reads ERRORBAR_API_KEY and ERRORBAR_TAG
+print(tracing.instrumented)  # e.g. ['openai', 'anthropic'] — only what's installed
+```
+
+Short-lived scripts should call `tracing.shutdown()` before exit to flush pending spans; long-running servers can skip it.
+
+## What gets captured
+
+OpenAI, Anthropic, Gemini, and LangChain calls — automatically, and **only for libraries actually installed** (the `instrumented` list tells you exactly which). Successful calls, streamed calls, and **failed** calls (stored as ERROR trace structure — the most valuable signal there is, and the one status-code dashboards can't see).
+
+Your inference does **not** move: requests keep going to your current provider; only trace telemetry flows to errorbar.
+
+## Configuration
+
+| Env var | Meaning | Default |
+| --- | --- | --- |
+| `ERRORBAR_API_KEY` | errorbar API key — **required**; `setup()` raises rather than exporting nowhere silently | — |
+| `ERRORBAR_TAG` | Population tag: one tag = one evaluation population in errorbar | unset |
+| `ERRORBAR_OTLP_ENDPOINT` | OTLP/HTTP traces endpoint | `https://gateway.errorbar.ai/v1/traces` |
+| `OTEL_SERVICE_NAME` | Standard OTel service name | unset |
+
+All options can also be passed to `setup()` directly; explicit options beat env vars.
+
+## The eject guarantee
+
+Remove this package and wire the same standard pieces yourself — identical spans, same endpoint, nothing lost:
+
+```python
+import os
+from opentelemetry.sdk.resources import Resource
+from opentelemetry.sdk.trace import TracerProvider
+from opentelemetry.sdk.trace.export import BatchSpanProcessor
+from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
+from opentelemetry.instrumentation.anthropic import AnthropicInstrumentor
+
+provider = TracerProvider(
+    resource=Resource.create({"service.name": "my-service", "omnia.tag": "my-agent"})
+)
+provider.add_span_processor(
+    BatchSpanProcessor(
+        OTLPSpanExporter(
+            endpoint="https://gateway.errorbar.ai/v1/traces",
+            headers={"Authorization": f"Bearer {os.environ['ERRORBAR_API_KEY']}"},
+        )
+    )
+)
+AnthropicInstrumentor().instrument(tracer_provider=provider)
+# ...and the other instrumentors for whichever libraries you use
+```
+
+Already emitting OpenTelemetry (Pydantic AI, an existing OTel setup)? You don't need this package at all — three env vars point your existing exporter at errorbar. See the [OTLP ingest reference](https://docs.omnia-voice.com/reference/otlp-ingest).
+
+## Privacy
+
+Span **structure** is always stored. Model-call **content** (prompts/completions) is stored only if your errorbar workspace has request logging enabled, under your retention window, with the same scrubbing and size caps as gateway traffic.
+
+## Verify your setup — get a receipt, not a hope
+
+```bash
+ERRORBAR_API_KEY=sk_... sh -c "$(curl -fsSL https://platform.omnia-voice.com/setup.sh)"
+```
+
+Proves the key works, confirms traces are actually landing, and names your one next step. Instrumentation that fails silently is the industry default; this is the alternative.
+
+## Links
+
+- [Docs](https://docs.omnia-voice.com/reference/tracing-sdk) · [Platform](https://platform.omnia-voice.com) · [Node package](https://www.npmjs.com/package/@omnia-voice/tracing)
+
+Apache-2.0

errorbar_tracing-0.2.0/README.md

@@ -0,0 +1,88 @@
+# errorbar-tracing
+
+**Standard OpenTelemetry, curated.** One install, one line, and your LLM traffic streams to [errorbar](https://platform.omnia-voice.com) — where you grade it, calibrate a judge against your own standards, and find out **with confidence intervals** whether a cheaper model holds up on your production traffic.
+
+This package contains **no instrumentation code of its own**. It pins and configures the ecosystem's standard OpenTelemetry instrumentations — which gives it a property no other tracing SDK offers: **you can uninstall it without losing your instrumentation.** The identical setup in vanilla OTel is documented below; your spans are byte-for-byte the same either way, and nothing proprietary ever goes on the wire.
+
+## Install
+
+```bash
+pip install errorbar-tracing
+```
+
+## Use
+
+Call once at startup, **before constructing any LLM client**:
+
+```python
+from errorbar_tracing import setup
+
+tracing = setup()          # reads ERRORBAR_API_KEY and ERRORBAR_TAG
+print(tracing.instrumented)  # e.g. ['openai', 'anthropic'] — only what's installed
+```
+
+Short-lived scripts should call `tracing.shutdown()` before exit to flush pending spans; long-running servers can skip it.
+
+## What gets captured
+
+OpenAI, Anthropic, Gemini, and LangChain calls — automatically, and **only for libraries actually installed** (the `instrumented` list tells you exactly which). Successful calls, streamed calls, and **failed** calls (stored as ERROR trace structure — the most valuable signal there is, and the one status-code dashboards can't see).
+
+Your inference does **not** move: requests keep going to your current provider; only trace telemetry flows to errorbar.
+
+## Configuration
+
+| Env var | Meaning | Default |
+| --- | --- | --- |
+| `ERRORBAR_API_KEY` | errorbar API key — **required**; `setup()` raises rather than exporting nowhere silently | — |
+| `ERRORBAR_TAG` | Population tag: one tag = one evaluation population in errorbar | unset |
+| `ERRORBAR_OTLP_ENDPOINT` | OTLP/HTTP traces endpoint | `https://gateway.errorbar.ai/v1/traces` |
+| `OTEL_SERVICE_NAME` | Standard OTel service name | unset |
+
+All options can also be passed to `setup()` directly; explicit options beat env vars.
+
+## The eject guarantee
+
+Remove this package and wire the same standard pieces yourself — identical spans, same endpoint, nothing lost:
+
+```python
+import os
+from opentelemetry.sdk.resources import Resource
+from opentelemetry.sdk.trace import TracerProvider
+from opentelemetry.sdk.trace.export import BatchSpanProcessor
+from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
+from opentelemetry.instrumentation.anthropic import AnthropicInstrumentor
+
+provider = TracerProvider(
+    resource=Resource.create({"service.name": "my-service", "omnia.tag": "my-agent"})
+)
+provider.add_span_processor(
+    BatchSpanProcessor(
+        OTLPSpanExporter(
+            endpoint="https://gateway.errorbar.ai/v1/traces",
+            headers={"Authorization": f"Bearer {os.environ['ERRORBAR_API_KEY']}"},
+        )
+    )
+)
+AnthropicInstrumentor().instrument(tracer_provider=provider)
+# ...and the other instrumentors for whichever libraries you use
+```
+
+Already emitting OpenTelemetry (Pydantic AI, an existing OTel setup)? You don't need this package at all — three env vars point your existing exporter at errorbar. See the [OTLP ingest reference](https://docs.omnia-voice.com/reference/otlp-ingest).
+
+## Privacy
+
+Span **structure** is always stored. Model-call **content** (prompts/completions) is stored only if your errorbar workspace has request logging enabled, under your retention window, with the same scrubbing and size caps as gateway traffic.
+
+## Verify your setup — get a receipt, not a hope
+
+```bash
+ERRORBAR_API_KEY=sk_... sh -c "$(curl -fsSL https://platform.omnia-voice.com/setup.sh)"
+```
+
+Proves the key works, confirms traces are actually landing, and names your one next step. Instrumentation that fails silently is the industry default; this is the alternative.
+
+## Links
+
+- [Docs](https://docs.omnia-voice.com/reference/tracing-sdk) · [Platform](https://platform.omnia-voice.com) · [Node package](https://www.npmjs.com/package/@omnia-voice/tracing)
+
+Apache-2.0

errorbar_tracing-0.2.0/pyproject.toml

@@ -0,0 +1,34 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "errorbar-tracing"
+version = "0.2.0"
+description = "errorbar tracing — standard OpenTelemetry, curated. One install, one line; eject anytime, your spans don't change."
+license = "Apache-2.0"
+readme = "README.md"
+requires-python = ">=3.10"
+dependencies = [
+  "opentelemetry-sdk>=1.27",
+  "opentelemetry-exporter-otlp-proto-http>=1.27",
+  "opentelemetry-instrumentation-openai>=0.30",
+  "opentelemetry-instrumentation-anthropic>=0.30",
+  "opentelemetry-instrumentation-google-generativeai>=0.30",
+  "opentelemetry-instrumentation-langchain>=0.30",
+]
+
+[project.urls]
+Repository = "https://github.com/omnia-v/omnia-tracing"
+
+[dependency-groups]
+dev = ["pytest>=8"]
+
+[tool.hatch.build.targets.sdist]
+# Explicit allowlist: hatchling's default packs the project DIR, which once
+# shipped a local .env inside a published sdist. Never again — list what
+# ships; everything else stays home.
+include = ["src/errorbar_tracing", "tests", "README.md", "pyproject.toml"]
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/errorbar_tracing"]

errorbar_tracing-0.2.0/src/errorbar_tracing/__init__.py

@@ -0,0 +1,128 @@
+"""errorbar tracing — standard OpenTelemetry, curated.
+
+One install, one line; eject anytime, your spans don't change. This package
+contains NO instrumentation code of its own: it pins and configures standard,
+ecosystem-maintained OpenTelemetry pieces. The identical setup without this
+package is documented in docs/eject.md.
+"""
+
+from __future__ import annotations
+
+import os
+from dataclasses import dataclass, field
+
+DEFAULT_ENDPOINT = "https://gateway.errorbar.ai/v1/traces"
+TAG_ATTRIBUTE = "omnia.tag"
+
+__all__ = ["setup", "resolve_config", "ResolvedConfig", "Tracing", "DEFAULT_ENDPOINT", "TAG_ATTRIBUTE"]
+
+
+@dataclass
+class ResolvedConfig:
+    endpoint: str
+    headers: dict[str, str]
+    service_name: str | None
+    resource_attributes: dict[str, str] = field(default_factory=dict)
+
+
+def resolve_config(
+    api_key: str | None = None,
+    tag: str | None = None,
+    service_name: str | None = None,
+    endpoint: str | None = None,
+    env: dict[str, str] | None = None,
+) -> ResolvedConfig:
+    """Pure configuration assembly — unit-testable without starting a pipeline."""
+    e = os.environ if env is None else env
+    key = api_key or e.get("ERRORBAR_API_KEY") or e.get("OMNIA_API_KEY")
+    if not key:
+        raise ValueError(
+            "errorbar-tracing: no API key. Pass setup(api_key=...) or set ERRORBAR_API_KEY (OMNIA_API_KEY still works). "
+            "Refusing to start a tracer that exports nowhere."
+        )
+    resource_attributes: dict[str, str] = {}
+    resolved_tag = tag or e.get("ERRORBAR_TAG") or e.get("OMNIA_TAG")
+    if resolved_tag:
+        resource_attributes[TAG_ATTRIBUTE] = resolved_tag
+    return ResolvedConfig(
+        endpoint=endpoint
+        or e.get("ERRORBAR_OTLP_ENDPOINT")
+        or e.get("OMNIA_OTLP_ENDPOINT")
+        or DEFAULT_ENDPOINT,
+        headers={"Authorization": f"Bearer {key}"},
+        service_name=service_name or e.get("OTEL_SERVICE_NAME"),
+        resource_attributes=resource_attributes,
+    )
+
+
+class Tracing:
+    """Handle returned by setup(): shutdown() flushes and stops.
+
+    `instrumented` names the libraries actually being traced — the ones both
+    installed in this environment and successfully instrumented."""
+
+    def __init__(self, provider, instrumented: list[str]) -> None:
+        self._provider = provider
+        self.instrumented = instrumented
+
+    def shutdown(self) -> None:
+        self._provider.shutdown()
+
+
+def setup(
+    api_key: str | None = None,
+    tag: str | None = None,
+    service_name: str | None = None,
+    endpoint: str | None = None,
+) -> Tracing:
+    """Start standard OpenTelemetry tracing, exporting to errorbar.
+
+    Call ONCE, at startup, before constructing LLM clients. Instruments
+    OpenAI, Anthropic, Gemini and LangChain via the ecosystem's standard
+    instrumentation packages.
+    """
+    config = resolve_config(api_key, tag, service_name, endpoint)
+
+    # Imports live here, not module top: `import errorbar_tracing` must stay
+    # side-effect free so resolve_config is usable (and testable) alone.
+    from opentelemetry.sdk.resources import Resource
+    from opentelemetry.sdk.trace import TracerProvider
+    from opentelemetry.sdk.trace.export import BatchSpanProcessor
+    from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
+
+    attrs: dict[str, str] = dict(config.resource_attributes)
+    if config.service_name:
+        attrs["service.name"] = config.service_name
+    provider = TracerProvider(resource=Resource.create(attrs))
+    provider.add_span_processor(
+        BatchSpanProcessor(
+            OTLPSpanExporter(endpoint=config.endpoint, headers=config.headers)
+        )
+    )
+
+    # Each instrumentation package imports its TARGET library at import time,
+    # so instrument only what this environment actually has — an
+    # Anthropic-only app must not be forced to install `openai`.
+    candidates = (
+        ("openai", "opentelemetry.instrumentation.openai", "OpenAIInstrumentor"),
+        ("anthropic", "opentelemetry.instrumentation.anthropic", "AnthropicInstrumentor"),
+        (
+            "google-generativeai",
+            "opentelemetry.instrumentation.google_generativeai",
+            "GoogleGenerativeAiInstrumentor",
+        ),
+        ("langchain", "opentelemetry.instrumentation.langchain", "LangchainInstrumentor"),
+    )
+    instrumented: list[str] = []
+    import importlib
+
+    for name, module_path, class_name in candidates:
+        try:
+            instrumentor = getattr(importlib.import_module(module_path), class_name)()
+        except ImportError:
+            continue  # target library not installed — nothing to trace
+        if not instrumentor.is_instrumented_by_opentelemetry:
+            instrumentor.instrument(tracer_provider=provider)
+        instrumented.append(name)
+
+    return Tracing(provider, instrumented)

errorbar_tracing-0.2.0/tests/test_config.py

@@ -0,0 +1,60 @@
+import pytest
+
+from errorbar_tracing import DEFAULT_ENDPOINT, TAG_ATTRIBUTE, resolve_config
+
+
+def test_refuses_without_api_key():
+    with pytest.raises(ValueError, match="ERRORBAR_API_KEY"):
+        resolve_config(env={})
+
+
+def test_defaults_to_gateway_with_bearer():
+    c = resolve_config(api_key="sk_x", env={})
+    assert c.endpoint == DEFAULT_ENDPOINT
+    assert c.headers["Authorization"] == "Bearer sk_x"
+
+
+def test_reads_env():
+    c = resolve_config(
+        env={
+            "OMNIA_API_KEY": "sk_env",
+            "OMNIA_TAG": "checkout-agent",
+            "OMNIA_OTLP_ENDPOINT": "https://other.example/v1/traces",
+            "OTEL_SERVICE_NAME": "svc",
+        }
+    )
+    assert c.headers["Authorization"] == "Bearer sk_env"
+    assert c.resource_attributes[TAG_ATTRIBUTE] == "checkout-agent"
+    assert c.endpoint == "https://other.example/v1/traces"
+    assert c.service_name == "svc"
+
+
+def test_options_beat_env():
+    c = resolve_config(
+        api_key="sk_opt",
+        tag="t2",
+        env={"OMNIA_API_KEY": "sk_env", "OMNIA_TAG": "t1"},
+    )
+    assert c.headers["Authorization"] == "Bearer sk_opt"
+    assert c.resource_attributes[TAG_ATTRIBUTE] == "t2"
+
+
+def test_no_tag_means_no_attribute():
+    c = resolve_config(api_key="k", env={})
+    assert c.resource_attributes == {}
+
+
+def test_errorbar_env_names_take_precedence_and_omnia_still_works():
+    c = resolve_config(
+        env={
+            "ERRORBAR_API_KEY": "sk_new",
+            "OMNIA_API_KEY": "sk_old",
+            "ERRORBAR_TAG": "new-tag",
+            "ERRORBAR_OTLP_ENDPOINT": "https://new.example/v1/traces",
+        }
+    )
+    assert c.headers["Authorization"] == "Bearer sk_new"
+    assert c.resource_attributes[TAG_ATTRIBUTE] == "new-tag"
+    assert c.endpoint == "https://new.example/v1/traces"
+    legacy = resolve_config(env={"OMNIA_API_KEY": "sk_old"})
+    assert legacy.headers["Authorization"] == "Bearer sk_old"