gatewaysdk 0.3.2__py3-none-any.whl

gatewaysdk/tracing/instrumentors/instrumentation_principles.md

@@ -0,0 +1,294 @@
+# Instrumentation Principles for GatewaySDK
+
+Lessons learned from building instrumentors for agent SDKs like `claude-agent-sdk`.
+
+## 1. Know Your Backend's Attribute Expectations
+
+Different tracing backends extract data from different attribute namespaces.
+
+**Example:** Gateway's attribute resolution checks multiple namespaces in priority order
+(see `attributeAliases.ts`). The primary namespace is `gateway.observation.*`:
+
+```typescript
+// Priority order for observation.input (from attributeAliases.ts):
+// 1. gateway.observation.input    ← primary (GatewaySDK uses this)
+// 2. langfuse.observation.input   ← compatibility
+// 3. input.value                  ← OpenInference
+// 4. gen_ai.input.messages        ← OTel GenAI
+// ... (20+ other aliases)
+```
+
+**Solution:** Set multiple attribute conventions for maximum compatibility:
+
+```python
+# Langfuse (for Gateway visualization)
+span.set_attribute("langfuse.observation.input", input_value)
+span.set_attribute("langfuse.observation.output", output_value)
+
+# OpenInference (for Phoenix, Arize)
+span.set_attribute("input.value", input_value)
+span.set_attribute("output.value", output_value)
+span.set_attribute("openinference.span.kind", "LLM")
+
+# GenAI (widely supported standard)
+span.set_attribute("gen_ai.system", "anthropic")
+span.set_attribute("gen_ai.usage.input_tokens", token_count)
+```
+
+## 2. Context Propagation is Critical
+
+For child spans to appear correctly nested and for timing to be accurate, you must **activate** the parent context.
+
+**Bad:**
+```python
+parent_span = tracer.start_span("parent")
+# Other instrumentation won't see this as the active parent
+child_span = tracer.start_span("child")  # Will be a sibling, not child!
+```
+
+**Good:**
+```python
+parent_span = tracer.start_span("parent")
+parent_ctx = trace.set_span_in_context(parent_span)
+context_token = attach(parent_ctx)  # Activate!
+
+try:
+    # Now other instrumentation sees parent_span as active
+    # Their spans will be children automatically
+    result = await some_instrumented_function()
+finally:
+    detach(context_token)  # Clean up
+    parent_span.end()
+```
+
+**Even Better (explicit context for child spans):**
+```python
+parent_ctx = trace.set_span_in_context(parent_span)
+context_token = attach(parent_ctx)
+
+# Explicitly pass context when creating known children
+child_span = tracer.start_span("child", context=parent_ctx)
+```
+
+## 3. Root Spans Need Fresh Context
+
+To create separate traces (not nested), use an empty `Context()`:
+
+```python
+# Creates a NEW trace (root span)
+span = tracer.start_span("operation", context=Context())
+
+# Without context=Context(), this would be a child of any active span
+```
+
+**Use case:** In chat applications, each user message should be its own trace, not nested under previous messages.
+
+## 4. Async Generators Require Special Handling
+
+Spans must remain open across async generator boundaries.
+
+**Pattern:**
+```python
+def _create_instrumented_query(self, original_query):
+    async def instrumented_query(client, prompt):
+        span = tracer.start_span("conversation")
+        span.set_attribute("input", prompt)
+
+        # Store span for later use
+        setattr(client, "_current_span", span)
+
+        try:
+            return await original_query(client, prompt)
+        except Exception as e:
+            span.set_status(StatusCode.ERROR)
+            span.end()  # End early on error
+            delattr(client, "_current_span")
+            raise
+
+def _create_instrumented_receive(self, original_receive):
+    async def instrumented_receive(client):
+        span = getattr(client, "_current_span", None)
+
+        try:
+            async for item in original_receive(client):
+                # Process items, set output on span
+                yield item
+
+                if is_final_item(item):
+                    span.set_attribute("output", extract_output(item))
+                    span.end()
+                    delattr(client, "_current_span")
+        finally:
+            # Cleanup if generator not fully consumed
+            if hasattr(client, "_current_span"):
+                remaining = getattr(client, "_current_span")
+                if remaining.is_recording():
+                    remaining.end()
+                delattr(client, "_current_span")
+```
+
+## 5. Capture All Available Context
+
+Don't just capture the obvious fields - explore the SDK's type definitions for all available data.
+
+**Example:** `claude-agent-sdk` provides:
+```python
+AssistantMessage.error  # API error type (rate_limit, billing_error, etc.)
+AssistantMessage.parent_tool_use_id  # Sub-agent correlation
+ThinkingBlock.signature  # Thinking block signature
+ResultMessage.structured_output  # Structured response data
+SystemMessage.subtype  # System event type (init, mcp_connection, etc.)
+```
+
+Capture them all for maximum observability:
+
+```python
+if message.error:
+    span.set_attribute("claude.api_error", message.error)
+
+if isinstance(message, SystemMessage):
+    span.add_event(
+        f"system.{message.subtype}",
+        attributes={"system.data": json.dumps(message.data)}
+    )
+```
+
+## 6. Never Truncate Arbitrarily
+
+Truncating attributes like `[:500]` or `[:10000]` breaks observability.
+
+**Bad:**
+```python
+span.set_attribute("output", long_text[:500])  # Why 500?
+```
+
+**Good:**
+```python
+span.set_attribute("output", long_text)  # Let OTEL handle limits
+```
+
+OpenTelemetry SDKs have configurable limits. Arbitrary truncation in instrumentors:
+- Loses critical debugging data
+- Isn't configurable by users
+- Often removes the exact data needed for diagnosis
+
+## 7. Span Kinds: OTel vs. Domain
+
+OpenTelemetry has `SpanKind` (CLIENT, SERVER, INTERNAL), but domain conventions like OpenInference use attributes.
+
+**Set both:**
+```python
+# OTel SpanKind (for protocol semantics)
+span = tracer.start_span("llm_call", kind=SpanKind.CLIENT)
+
+# Domain span kind (for tracing UI semantics)
+span.set_attribute("openinference.span.kind", "LLM")
+```
+
+Common domain kinds:
+- `LLM` - Language model generation
+- `TOOL` - Tool/function execution
+- `AGENT` - Agent orchestration
+- `CHAIN` - Workflow steps
+
+## 8. Timing Accuracy Limitations
+
+Be aware that span timing in streaming contexts reflects **when you receive messages**, not when work actually happened.
+
+```python
+# In claude-agent-sdk:
+ToolUseBlock arrives → start tool span (time T1)
+# ... tool executes in CLI subprocess (actual work happens)
+ToolResultBlock arrives → end tool span (time T2)
+
+# T2 - T1 includes:
+# - Actual tool execution time
+# - Stream buffering delay
+# - Network latency
+# - Message parsing overhead
+```
+
+**Document this limitation:**
+```python
+# Note: Span timing is based on when we receive ToolUseBlock/ToolResultBlock messages,
+# not when the tool actually executes in Claude CLI. This may cause timing inaccuracies.
+tool_span = tracer.start_span(f"tool.{block.name}", ...)
+```
+
+**Alternative (if SDK provides timing):**
+```python
+# Some SDKs provide explicit timestamps in result messages
+if hasattr(message, 'started_at') and hasattr(message, 'completed_at'):
+    tool_span = tracer.start_span(
+        name=f"tool.{name}",
+        start_time=parse_timestamp(message.started_at),  # Explicit start
+    )
+    tool_span.end(end_time=parse_timestamp(message.completed_at))  # Explicit end
+```
+
+## 9. Centralize Attribute Definitions
+
+Don't hardcode attribute names throughout your instrumentor.
+
+**Bad:**
+```python
+span.set_attribute("langfuse.observation.input", value)
+# ...later in another file...
+span.set_attribute("langfuse.observation.input", value)  # Typo risk!
+```
+
+**Good:**
+```python
+# semconv.py
+LANGFUSE_OBSERVATION_INPUT = "langfuse.observation.input"
+LANGFUSE_OBSERVATION_OUTPUT = "langfuse.observation.output"
+
+# instrumentor.py
+from ..semconv import LANGFUSE_OBSERVATION_INPUT
+span.set_attribute(LANGFUSE_OBSERVATION_INPUT, value)
+```
+
+Benefits:
+- Type safety
+- Easier refactoring
+- Single source of truth
+- Documentation via constants
+
+## 10. Preserve Event Sequence
+
+When LLMs interleave text and tool calls, preserve the order:
+
+**Bad (loses sequence):**
+```python
+# Concatenate all text, list all tools
+output = {
+    "text": text1 + text2,
+    "tool_calls": [tool1, tool2]
+}
+```
+
+**Good (preserves sequence):**
+```python
+# Preserve: text → tool_call → text
+output_blocks = [
+    {"type": "text", "content": text1},
+    {"type": "tool_call", "name": "search", "input": {...}, "output": "..."},
+    {"type": "text", "content": text2},
+]
+span.set_attribute("output", json.dumps(output_blocks))
+```
+
+This helps understand the agent's reasoning flow: "It said X, then called tool Y, then concluded Z."
+
+## Summary
+
+Building robust instrumentors requires:
+1. ✅ Understanding backend attribute expectations
+2. ✅ Proper context propagation with `attach()`/`detach()`
+3. ✅ Supporting multiple attribute conventions
+4. ✅ Careful span lifecycle management in async code
+5. ✅ Capturing all available SDK data
+6. ✅ Avoiding arbitrary truncation
+7. ✅ Being aware of timing limitations in streaming contexts
+8. ✅ Centralizing attribute definitions
+9. ✅ Preserving event sequences for debugging

gatewaysdk/tracing/instrumentors/registry.py

@@ -0,0 +1,352 @@
+# Copyright (c) Gateway. All rights reserved.
+
+"""Instrumentor registry for GatewayDK tracing.
+
+This module provides a registry-based system for discovering and managing
+OpenTelemetry instrumentors. It supports lazy loading and optional dependencies.
+"""
+
+from __future__ import annotations
+
+import importlib.util
+import logging
+from abc import ABC, abstractmethod
+from typing import TYPE_CHECKING, Dict, List, Optional
+
+if TYPE_CHECKING:
+    from openinference.instrumentation import TraceConfig
+
+logger = logging.getLogger(__name__)
+
+# Singleton registry instance
+_registry: Optional["InstrumentorRegistry"] = None
+
+
+class BaseInstrumentor(ABC):
+    """Base class for instrumentor adapters.
+
+    Each instrumentor adapter wraps a specific instrumentation library
+    (OpenInference, AgentOps, etc.) and provides a consistent interface.
+    """
+
+    @property
+    @abstractmethod
+    def name(self) -> str:
+        """Unique name for this instrumentor (e.g., 'openai', 'langchain')."""
+        pass
+
+    @property
+    @abstractmethod
+    def package_name(self) -> str:
+        """The package name to check for availability."""
+        pass
+
+    @abstractmethod
+    def is_available(self) -> bool:
+        """Check if the instrumentor and its target library are available."""
+        pass
+
+    @abstractmethod
+    def instrument(self, trace_config: Optional["TraceConfig"] = None) -> None:
+        """Activate the instrumentation."""
+        pass
+
+    @abstractmethod
+    def uninstrument(self) -> None:
+        """Deactivate the instrumentation."""
+        pass
+
+
+class OpenInferenceInstrumentor(BaseInstrumentor):
+    """Adapter for OpenInference instrumentors.
+
+    OpenInference provides high-quality instrumentors for major LLM libraries
+    that emit semantic conventions compatible with observability platforms.
+    """
+
+    def __init__(
+        self,
+        name: str,
+        instrumentor_package: str,
+        instrumentor_class: str,
+        target_package: str,
+    ):
+        """Initialize the adapter.
+
+        Args:
+            name: Unique name for this instrumentor.
+            instrumentor_package: Full package path to the instrumentor module.
+            instrumentor_class: Class name of the instrumentor.
+            target_package: The target library package (e.g., "openai").
+        """
+        self._name = name
+        self._instrumentor_package = instrumentor_package
+        self._instrumentor_class = instrumentor_class
+        self._target_package = target_package
+        self._instance: Optional[object] = None
+        self._instrumented = False
+
+    @property
+    def name(self) -> str:
+        return self._name
+
+    @property
+    def package_name(self) -> str:
+        return self._instrumentor_package
+
+    @property
+    def target_package(self) -> str:
+        """The AI library this instrumentor traces (e.g. "openai")."""
+        return self._target_package
+
+    def is_available(self) -> bool:
+        """Check if both the instrumentor and target library are installed."""
+        # Check target library
+        if not _is_package_available(self._target_package):
+            return False
+
+        # Check instrumentor package
+        if not _is_package_available(self._instrumentor_package):
+            return False
+
+        return True
+
+    def instrument(self, trace_config: Optional["TraceConfig"] = None) -> None:
+        """Activate the OpenInference instrumentation."""
+        if self._instrumented:
+            logger.debug("Already instrumented: %s", self._name)
+            return
+
+        if not self.is_available():
+            logger.debug("Cannot instrument %s: dependencies not available", self._name)
+            return
+
+        try:
+            module = importlib.import_module(self._instrumentor_package)
+            instrumentor_class = getattr(module, self._instrumentor_class)
+            self._instance = instrumentor_class()
+
+            # OpenInference instrumentors accept trace_config
+            if trace_config is not None:
+                self._instance.instrument(tracer_provider=None, trace_config=trace_config)
+            else:
+                self._instance.instrument()
+
+            self._instrumented = True
+            logger.debug("Instrumented: %s", self._name)
+        except Exception as e:
+            logger.warning("Failed to instrument %s: %s", self._name, e)
+            raise
+
+    def uninstrument(self) -> None:
+        """Deactivate the OpenInference instrumentation."""
+        if not self._instrumented or self._instance is None:
+            return
+
+        try:
+            self._instance.uninstrument()
+            self._instrumented = False
+            self._instance = None
+            logger.debug("Uninstrumented: %s", self._name)
+        except Exception as e:
+            logger.warning("Failed to uninstrument %s: %s", self._name, e)
+
+
+class InstrumentorRegistry:
+    """Registry for managing instrumentors.
+
+    The registry provides a central place to discover, configure, and manage
+    instrumentors from various sources.
+    """
+
+    def __init__(self):
+        self._instrumentors: Dict[str, BaseInstrumentor] = {}
+        self._register_defaults()
+
+    def _register_defaults(self) -> None:
+        """Register the default set of instrumentors."""
+        # Claude Agent SDK instrumentor (native gatewaysdk instrumentor)
+        from .claude_agent_sdk import ClaudeAgentSDKInstrumentor
+
+        self.register(ClaudeAgentSDKInstrumentor())
+
+        # OpenInference instrumentors (core set)
+        openinference_instrumentors = [
+            ("openai_agents", "openinference.instrumentation.openai_agents", "OpenAIAgentsInstrumentor", "agents"),
+            ("openai", "openinference.instrumentation.openai", "OpenAIInstrumentor", "openai"),
+            ("anthropic", "openinference.instrumentation.anthropic", "AnthropicInstrumentor", "anthropic"),
+            ("langchain", "openinference.instrumentation.langchain", "LangChainInstrumentor", "langchain"),
+            ("llama_index", "openinference.instrumentation.llama_index", "LlamaIndexInstrumentor", "llama_index"),
+            ("litellm", "openinference.instrumentation.litellm", "LiteLLMInstrumentor", "litellm"),
+            # Additional instrumentors
+            ("google_adk", "openinference.instrumentation.google_adk", "GoogleADKInstrumentor", "google.adk"),
+            ("mistral", "openinference.instrumentation.mistralai", "MistralAIInstrumentor", "mistralai"),
+            ("groq", "openinference.instrumentation.groq", "GroqInstrumentor", "groq"),
+            ("bedrock", "openinference.instrumentation.bedrock", "BedrockInstrumentor", "boto3"),
+            ("vertexai", "openinference.instrumentation.vertexai", "VertexAIInstrumentor", "vertexai"),
+            ("dspy", "openinference.instrumentation.dspy", "DSPyInstrumentor", "dspy"),
+            ("instructor", "openinference.instrumentation.instructor", "InstructorInstrumentor", "instructor"),
+            ("crewai", "openinference.instrumentation.crewai", "CrewAIInstrumentor", "crewai"),
+        ]
+
+        for name, package, class_name, target in openinference_instrumentors:
+            instrumentor = OpenInferenceInstrumentor(
+                name=name,
+                instrumentor_package=package,
+                instrumentor_class=class_name,
+                target_package=target,
+            )
+            self.register(instrumentor)
+
+    def register(self, instrumentor: BaseInstrumentor) -> None:
+        """Register an instrumentor.
+
+        Args:
+            instrumentor: The instrumentor to register.
+        """
+        self._instrumentors[instrumentor.name] = instrumentor
+        logger.debug("Registered instrumentor: %s", instrumentor.name)
+
+    def get(self, name: str) -> Optional[BaseInstrumentor]:
+        """Get an instrumentor by name.
+
+        Args:
+            name: The instrumentor name.
+
+        Returns:
+            The instrumentor if found and available, None otherwise.
+        """
+        instrumentor = self._instrumentors.get(name)
+        if instrumentor is None:
+            return None
+
+        if not instrumentor.is_available():
+            return None
+
+        return instrumentor
+
+    def list_available(self) -> List[str]:
+        """List all available instrumentor names.
+
+        Returns:
+            List of instrumentor names that are currently available.
+        """
+        return [name for name, instrumentor in self._instrumentors.items() if instrumentor.is_available()]
+
+    def list_all(self) -> List[str]:
+        """List all registered instrumentor names.
+
+        Returns:
+            List of all registered instrumentor names (including unavailable).
+        """
+        return list(self._instrumentors.keys())
+
+    def instrument_all(
+        self,
+        names: Optional[List[str]] = None,
+        skip: Optional[List[str]] = None,
+        trace_config: Optional["TraceConfig"] = None,
+    ) -> List[str]:
+        """Instrument multiple instrumentors at once.
+
+        Args:
+            names: List of instrumentor names to enable. If None, uses the core set.
+            skip: List of instrumentor names to skip.
+            trace_config: Optional TraceConfig for OpenInference instrumentors.
+
+        Returns:
+            List of successfully instrumented names.
+        """
+        skip_set = set(skip or [])
+
+        if names is None:
+            # Core set -- instruments all installed AI libraries automatically
+            target_names = [
+                "openai_agents",
+                "claude_agent_sdk",
+                "openai",
+                "anthropic",
+                "langchain",
+                "llama_index",
+                "litellm",
+            ]
+        else:
+            target_names = names
+
+        # Filter out skipped
+        target_names = [n for n in target_names if n not in skip_set]
+
+        instrumented = []
+        for name in target_names:
+            instrumentor = self.get(name)
+            if instrumentor is not None:
+                try:
+                    instrumentor.instrument(trace_config=trace_config)
+                    instrumented.append(name)
+                except Exception as e:
+                    logger.debug("Failed to instrument %s: %s", name, e)
+            else:
+                # Skipped. Warn loudly if the AI library itself IS installed but
+                # its tracing instrumentor is missing — otherwise the user gets
+                # zero telemetry with no hint why (the classic silent-skip trap).
+                self.warn_if_lib_present_but_uninstrumented(name)
+
+        return instrumented
+
+    def warn_if_lib_present_but_uninstrumented(self, name: str) -> None:
+        """Warn when an installed AI library has no active instrumentor.
+
+        Only warns when the target library is actually importable — an absent
+        library has nothing to trace, so its skip is expected and silent.
+        """
+        instrumentor = self._instrumentors.get(name)
+        if instrumentor is None:
+            return
+        target = getattr(instrumentor, "target_package", None)
+        if not target or not _is_package_available(target):
+            return
+        instr_pkg = getattr(instrumentor, "package_name", None)
+        detail = f" ('{instr_pkg}')" if instr_pkg else ""
+        logger.warning(
+            "gatewaysdk.tracing: %r is installed but its tracing instrumentor%s "
+            "is not available, so its calls will NOT be traced. Install the "
+            "tracing extra to enable it: pip install 'gatewaysdk[tracing]'.",
+            target,
+            detail,
+        )
+
+    def uninstrument_all(self) -> None:
+        """Uninstrument all active instrumentors."""
+        for instrumentor in self._instrumentors.values():
+            try:
+                instrumentor.uninstrument()
+            except Exception as e:
+                logger.debug("Failed to uninstrument %s: %s", instrumentor.name, e)
+
+
+def get_registry() -> InstrumentorRegistry:
+    """Get the global instrumentor registry.
+
+    Returns:
+        The singleton InstrumentorRegistry instance.
+    """
+    global _registry
+    if _registry is None:
+        _registry = InstrumentorRegistry()
+    return _registry
+
+
+def _is_package_available(package_name: str) -> bool:
+    """Check if a package is available for import.
+
+    Args:
+        package_name: The package name (can be dotted path).
+
+    Returns:
+        True if the package is available.
+    """
+    try:
+        spec = importlib.util.find_spec(package_name)
+        return spec is not None
+    except (ModuleNotFoundError, ValueError):
+        return False