mseep-agentops 0.4.18__py3-none-any.whl → 0.4.23__py3-none-any.whl

agentops/instrumentation/agentic/openai_agents/README.md

@@ -0,0 +1,156 @@
+# OpenAI Agents SDK Instrumentation
+
+This module provides automatic instrumentation for the OpenAI Agents SDK, adding telemetry that follows OpenTelemetry semantic conventions for Generative AI systems.
+
+## Architecture Overview
+
+The OpenAI Agents SDK instrumentor works by:
+
+1. Intercepting the Agents SDK's trace processor interface to capture Agent, Function, Generation, and other span types
+2. Monkey-patching the Agents SDK `Runner` class to capture the full execution lifecycle, including streaming operations
+3. Converting all captured data to OpenTelemetry spans and metrics following semantic conventions
+
+The instrumentation is organized into several key components:
+
+1. **Instrumentor (`instrumentor.py`)**: The entry point that patches the Agents SDK and configures trace capture
+2. **Processor (`processor.py`)**: Receives events from the SDK and prepares them for export
+3. **Exporter (`exporter.py`)**: Converts SDK spans to OpenTelemetry spans and exports them
+4. **Attributes Module (`attributes/`)**: Specialized modules for extracting and formatting span attributes
+
+## Attribute Processing Modules
+
+The attribute modules extract and format OpenTelemetry-compatible attributes from span data:
+
+- **Common (`attributes/common.py`)**: Core attribute extraction functions for all span types and utility functions
+- **Completion (`attributes/completion.py`)**: Handles different completion content formats (Chat Completions API, Response API, Agents SDK) 
+- **Model (`attributes/model.py`)**: Extracts model information and parameters
+- **Tokens (`attributes/tokens.py`)**: Processes token usage data and metrics
+- **Response (`attributes/response.py`)**: Handles interpretation of Response API objects
+
+Each getter function in these modules is focused on a single responsibility and does not modify global state. Functions are designed to be composable, allowing different attribute types to be combined as needed in the exporter.
+
+## Span Types
+
+The instrumentor captures the following span types:
+
+- **Trace**: The root span representing an entire agent workflow execution
+  - Created using `get_base_trace_attributes()` to initialize with standard fields
+  - Captures workflow name, trace ID, and workflow-level metadata
+
+- **Agent**: Represents an agent's execution lifecycle
+  - Processed using `get_agent_span_attributes()` with `AGENT_SPAN_ATTRIBUTES` mapping
+  - Uses `SpanKind.CONSUMER` to indicate an agent receiving a request
+  - Captures agent name, input, output, tools, and other metadata
+
+- **Function**: Represents a tool/function call
+  - Processed using `get_function_span_attributes()` with `FUNCTION_SPAN_ATTRIBUTES` mapping
+  - Uses `SpanKind.CLIENT` to indicate an outbound call to a function
+  - Captures function name, input arguments, output results, and from_agent information
+
+- **Generation**: Captures details of model generation
+  - Processed using `get_generation_span_attributes()` with `GENERATION_SPAN_ATTRIBUTES` mapping
+  - Uses `SpanKind.CLIENT` to indicate an outbound call to an LLM
+  - Captures model name, configuration, usage statistics, and response content
+
+- **Response**: Lightweight span for tracking model response data
+  - Processed using `get_response_span_attributes()` with `RESPONSE_SPAN_ATTRIBUTES` mapping
+  - Extracts response content and metadata from different API formats
+
+- **Handoff**: Represents control transfer between agents
+  - Processed using `get_handoff_span_attributes()` with `HANDOFF_SPAN_ATTRIBUTES` mapping
+  - Tracks from_agent and to_agent information
+
+## Span Lifecycle Management
+
+The exporter (`exporter.py`) handles the full span lifecycle:
+
+1. **Start Events**:
+   - Create spans but DO NOT END them
+   - Store span references in tracking dictionaries
+   - Use OpenTelemetry's start_span to control when spans end
+   - Leave status as UNSET to indicate in-progress
+
+2. **End Events**:
+   - Look up existing span by ID in tracking dictionaries
+   - If found and not ended:
+     - Update span with all final attributes
+     - Set status to OK or ERROR based on task outcome
+     - End the span manually
+   - If not found or already ended:
+     - Create a new complete span with all data
+     - End it immediately
+
+3. **Error Handling**:
+   - Check if spans are already ended before attempting updates
+   - Provide informative log messages about span lifecycle
+   - Properly clean up tracking resources
+
+This approach is essential because:
+- Agents SDK sends separate start and end events for each task
+- We need to maintain a single span for the entire task lifecycle to get accurate timing
+- Final data (outputs, token usage, etc.) is only available at the end event
+- We want to avoid creating duplicate spans for the same task
+- Spans must be properly created and ended to avoid leaks
+
+The span lifecycle management ensures spans have:
+- Accurate start and end times (preserving the actual task duration)
+- Complete attribute data from both start and end events
+- Proper status reflecting task completion
+- All final outputs, errors, and metrics
+- Clean resource management with no memory leaks
+
+## Key Design Patterns
+
+### Semantic Conventions
+
+All attribute names follow the OpenTelemetry semantic conventions defined in `agentops.semconv`:
+
+```python
+# Using constants from semconv module
+attributes[CoreAttributes.TRACE_ID] = trace_id
+attributes[WorkflowAttributes.WORKFLOW_NAME] = trace.name
+attributes[SpanAttributes.LLM_SYSTEM] = "openai"
+attributes[MessageAttributes.COMPLETION_CONTENT.format(i=0)] = content
+```
+
+### Target → Source Attribute Mapping
+
+We use a consistent pattern for attribute extraction with typed mapping dictionaries:
+
+```python
+# Attribute mapping example
+AGENT_SPAN_ATTRIBUTES: AttributeMap = {
+    # target_attribute: source_attribute
+    AgentAttributes.AGENT_NAME: "name",
+    WorkflowAttributes.WORKFLOW_INPUT: "input",
+    WorkflowAttributes.FINAL_OUTPUT: "output",
+    # ...
+}
+```
+
+### Structured Attribute Handling
+
+- Always use MessageAttributes semantic conventions for content and tool calls
+- For chat completions, use MessageAttributes.COMPLETION_CONTENT.format(i=0) 
+- For tool calls, use MessageAttributes.COMPLETION_TOOL_CALL_NAME.format(i=0, j=0), etc.
+- Never try to combine or aggregate contents into a single attribute
+- Each message component should have its own properly formatted attribute
+- This ensures proper display in OpenTelemetry backends and dashboards
+
+### Serialization Rules
+
+1. We do not serialize data structures arbitrarily; everything has a semantic convention
+2. Span attributes should use semantic conventions and avoid complex serialized structures
+3. Keep all string data in its original form - do not parse JSON within strings
+4. If a function has JSON attributes for its arguments, do not parse that JSON - keep as string
+5. If a completion or response body text/content contains JSON, keep it as a string
+7. Function arguments and tool call arguments should remain in their raw string form
+
+### Critical Notes for Attribute Handling
+
+- NEVER manually set the root completion attributes (`SpanAttributes.LLM_COMPLETIONS` or "gen_ai.completion")
+- Let OpenTelemetry backend derive these values from the detailed attributes
+- Setting root completion attributes creates duplication and inconsistency
+- Tests should verify attribute existence using MessageAttributes constants
+- Do not check for the presence of SpanAttributes.LLM_COMPLETIONS
+- Verify individual content/tool attributes instead of root attributes

agentops/instrumentation/agentic/openai_agents/SPANS.md

@@ -0,0 +1,145 @@
+# OpenAI Agents Spans and Traces
+
+This document describes the span types, naming conventions, and attribute patterns used by the AgentOps instrumentation for the OpenAI Agents SDK.
+
+## Span Types and Classes
+
+The instrumentation works with these specific span data classes:
+
+1. **AgentSpanData**: Represents a single agent's operation
+   - Has attributes for name, input, output, tools, and handoffs
+   - Processed by `get_agent_span_attributes()` using `AGENT_SPAN_ATTRIBUTES` mapping
+
+2. **FunctionSpanData**: Represents tool or function calls
+   - Has attributes for name, input, output, and from_agent
+   - Processed by `get_function_span_attributes()` using `FUNCTION_SPAN_ATTRIBUTES` mapping
+
+3. **GenerationSpanData**: Represents LLM model invocations
+   - Has attributes for model, input, output, tools, and from_agent
+   - Processed by `get_generation_span_attributes()` using `GENERATION_SPAN_ATTRIBUTES` mapping
+
+4. **HandoffSpanData**: Represents agent-to-agent handoffs
+   - Has attributes for from_agent and to_agent
+   - Processed by `get_handoff_span_attributes()` using `HANDOFF_SPAN_ATTRIBUTES` mapping
+
+5. **ResponseSpanData**: Represents model response data
+   - Has attributes for input and response
+   - Processed by `get_response_span_attributes()` using `RESPONSE_SPAN_ATTRIBUTES` mapping
+
+## Span Naming Conventions
+
+Spans are named according to these conventions:
+
+1. **Trace Spans**: `agents.trace.{workflow_name}`
+   - Represents the entire agent workflow
+   - Named after the workflow or trace name
+
+2. **Agent Spans**: `agents.agent`
+   - Represents a single agent's operation
+   - Uses `SpanKind.CONSUMER`
+
+3. **Function Spans**: `agents.function`
+   - Represents tool or function calls
+   - Uses `SpanKind.CLIENT`
+
+4. **Generation Spans**: `agents.generation`
+   - Represents LLM model invocations
+   - Uses `SpanKind.CLIENT`
+
+5. **Handoff Spans**: `agents.handoff`
+   - Represents agent-to-agent handoffs
+   - Uses `SpanKind.INTERNAL`
+
+6. **Response Spans**: `agents.response`
+   - Represents model response data
+   - Uses `SpanKind.CLIENT`
+
+## Span Hierarchy
+
+The spans follow a parent-child relationship that reflects the execution flow:
+
+```
+agents.trace.{workflow_name}
+  └── agents.agent
+      ├── agents.generation
+      ├── agents.function
+      ├── agents.response
+      └── agents.handoff
+```
+
+## Semantic Conventions and Attributes
+
+Each span type has attributes following OpenTelemetry semantic conventions:
+
+### Common Attributes (All Spans)
+
+- `trace.id`: OpenTelemetry trace ID
+- `span.id`: OpenTelemetry span ID
+- `parent.id`: Parent span ID (if applicable)
+- `instrumentation.name`: "agentops"
+- `instrumentation.version`: AgentOps library version
+- `instrumentation.library.name`: "openai_agents"
+- `instrumentation.library.version`: Library version
+
+### Workflow and Trace Attributes
+
+- `workflow.name`: Name of the workflow or trace
+- `workflow.step_type`: "trace" for trace spans
+- `workflow.input`: Input to the workflow
+- `workflow.final_output`: Final output from the workflow
+
+### Agent Attributes
+
+- `agent.name`: The name of the agent
+- `agent.tools`: Comma-separated list of available tools
+- `agent.handoffs`: Comma-separated list of handoff targets
+- `agent.from`: Source agent in handoffs (used in HandoffSpanData)
+- `agent.to`: Destination agent in handoffs (used in HandoffSpanData)
+
+### LLM Attributes
+
+- `gen_ai.system`: "openai" for all OpenAI spans
+- `gen_ai.request.model`: Model used for generation
+- `gen_ai.response.model`: Model that provided the response
+- `gen_ai.prompt`: Input prompt or message
+- `gen_ai.completion.0.role`: Role of the completion message (usually "assistant")
+- `gen_ai.completion.0.content`: Content of the completion message
+- `gen_ai.tool_call.0.0.name`: Name of the tool called (if applicable)
+- `gen_ai.tool_call.0.0.arguments`: Arguments for the tool call (if applicable)
+
+### Token Usage Attributes
+
+- `gen_ai.usage.prompt_tokens`: Number of input tokens
+- `gen_ai.usage.completion_tokens`: Number of output tokens
+- `gen_ai.usage.total_tokens`: Total number of tokens
+- `gen_ai.usage.reasoning_tokens`: Tokens used for reasoning (Response API)
+- `gen_ai.usage.cache_read.input_tokens`: Cached input tokens (Response API)
+
+## Span Lifecycle Management
+
+The exporter handles span lifecycle with these stages:
+
+1. **Start Events**:
+   - Create spans with `start_span()` (not using context manager)
+   - Store span references in tracking dictionaries
+   - Leave status as UNSET to indicate in-progress
+
+2. **End Events**:
+   - Look up existing span by ID
+   - Update with final attributes
+   - Set appropriate status and end the span manually
+
+3. **Error Handling**:
+   - Set status to ERROR for spans with errors
+   - Add error type and message as attributes
+   - Record exceptions with `record_exception()`
+
+## OpenTelemetry Span Kinds
+
+Span kinds map to OpenTelemetry concepts:
+
+- `AgentSpanData` → `SpanKind.CONSUMER`
+- `FunctionSpanData` → `SpanKind.CLIENT`
+- `GenerationSpanData` → `SpanKind.CLIENT`
+- `ResponseSpanData` → `SpanKind.CLIENT`
+- `HandoffSpanData` → `SpanKind.INTERNAL`

agentops/instrumentation/agentic/openai_agents/TRACING_API.md

@@ -0,0 +1,144 @@
+# OpenAI Agents Tracing API Integration
+
+This document provides an overview of how AgentOps integrates with the OpenAI Agents SDK tracing system.
+
+## OpenAI Agents Tracing API Overview
+
+The OpenAI Agents SDK provides a comprehensive tracing system that allows you to monitor and instrument agent activities. AgentOps integrates with this system to capture and forward trace data to its backend.
+
+## Core Integration Methods
+
+### 1. `add_trace_processor(processor)`
+
+The main integration point that allows external systems like AgentOps to receive trace events:
+
+```python
+from agents import add_trace_processor
+from agentops.instrumentation.openai_agents.processor import OpenAIAgentsProcessor
+
+processor = OpenAIAgentsProcessor()
+add_trace_processor(processor)
+```
+
+### 2. `set_trace_processors(processors)`
+
+Replaces all current processors with a new list:
+
+```python
+from agents import set_trace_processors
+set_trace_processors([my_processor1, my_processor2])
+```
+
+### 3. `set_tracing_disabled(disabled)`
+
+Globally enables/disables tracing:
+
+```python
+from agents import set_tracing_disabled
+set_tracing_disabled(True)  # Disable tracing
+```
+
+### 4. `set_tracing_export_api_key(api_key)`
+
+Sets the API key for the backend exporter:
+
+```python
+from agents import set_tracing_export_api_key
+set_tracing_export_api_key("your-api-key")
+```
+
+## Span Creation Methods
+
+The SDK provides specialized methods for creating different types of spans:
+
+1. **`agent_span(name, handoffs, tools, output_type, ...)`**
+   - Creates spans for agent operations
+   - Tracks agent name, available tools, potential handoffs
+
+2. **`function_span(name, input, output, ...)`**
+   - Creates spans for function/tool calls
+   - Records function name, input arguments, and results
+
+3. **`generation_span(input, output, model, model_config, usage, ...)`**
+   - Creates spans for LLM generations
+   - Records prompts, completions, model details, and token usage
+
+4. **`response_span(response, ...)`**
+   - Lightweight span for capturing OpenAI API response metadata
+
+5. **`handoff_span(from_agent, to_agent, ...)`**
+   - Tracks agent-to-agent handoffs
+
+6. **`guardrail_span(name, triggered, ...)`**
+   - Records guardrail evaluations
+
+7. **`custom_span(name, data, ...)`**
+   - Creates user-defined spans with arbitrary data
+
+## Trace and Context Management
+
+1. **`trace(workflow_name, trace_id, group_id, metadata, ...)`**
+   - Creates and manages a trace context
+   - Groups related spans into a logical trace/session
+
+2. **`get_current_span()`**
+   - Returns the current active span
+
+3. **`get_current_trace()`**
+   - Returns the current active trace
+
+## How AgentOps Implements Integration
+
+AgentOps integrates with this API through:
+
+1. The `OpenAIAgentsProcessor` class that implements the `TracingProcessor` interface
+2. The `create_span` context manager that ensures proper parent-child relationships between spans
+3. The `AgentsInstrumentor` which registers the processor and adds additional instrumentation
+
+This integration allows AgentOps to capture detailed information about agent execution, including:
+- Agent operations and tool usage
+- LLM requests and responses 
+- Token usage metrics
+- Error information
+- Agent-to-agent handoffs
+
+### Trace Context Propagation
+
+Our implementation ensures proper parent-child relationships between spans through:
+
+1. **Context Manager Pattern**: Using `start_as_current_span()` to maintain the OpenTelemetry span context
+2. **Parent Reference Tracking**: Storing parent span relationships and using them to create proper span hierarchies
+3. **Trace Correlation Attributes**: Adding consistent attributes to help with querying:
+   - `agentops.original_trace_id`: Original trace ID from the Agents SDK
+   - `agentops.original_span_id`: Original span ID from the Agents SDK
+   - `agentops.parent_span_id`: Parent span ID for child spans
+   - `agentops.trace_hash`: Consistent hash based on the original trace ID
+   - `agentops.is_root_span`: "true" for spans without a parent
+
+When querying spans for analysis:
+1. Group spans by `agentops.original_trace_id` to find all spans in the same trace
+2. Use `agentops.parent_span_id` to reconstruct the parent-child hierarchy
+
+## Span Data Types
+
+Several specialized span data types exist in the OpenAI Agents SDK to capture different operations:
+
+- **AgentSpanData**: Captures agent execution data
+- **FunctionSpanData**: Records tool/function calls
+- **GenerationSpanData**: Records LLM generation details
+- **ResponseSpanData**: Captures model response information
+- **HandoffSpanData**: Tracks agent-to-agent handoffs
+- **GuardrailSpanData**: Records guardrail evaluations
+- **CustomSpanData**: For user-defined spans
+
+## Processor Interface
+
+The `TracingProcessor` interface defines methods processors must implement:
+- `on_trace_start`: Called when a trace begins
+- `on_trace_end`: Called when a trace ends
+- `on_span_start`: Called when a span begins
+- `on_span_end`: Called when a span completes
+- `shutdown`: Called during application shutdown
+- `force_flush`: Forces immediate processing of pending spans
+
+The processor receives events from OpenAI Agents SDK's tracing system through these callback methods, translates them to OpenTelemetry spans, and sends them to the AgentOps backend for analysis and visualization.

agentops/instrumentation/agentic/openai_agents/__init__.py

@@ -0,0 +1,30 @@
+"""
+AgentOps Instrumentor for OpenAI Agents SDK
+
+This module provides automatic instrumentation for the OpenAI Agents SDK when AgentOps is imported.
+It implements a clean, maintainable implementation that follows semantic conventions.
+
+IMPORTANT DISTINCTION BETWEEN OPENAI API FORMATS:
+1. OpenAI Completions API - The traditional API format using prompt_tokens/completion_tokens
+2. OpenAI Response API - The newer format used by the Agents SDK using input_tokens/output_tokens
+3. Agents SDK - The framework that uses Response API format
+
+The Agents SDK uses the Response API format, which we handle using shared utilities from
+agentops.instrumentation.openai.
+"""
+
+from agentops.instrumentation.common import LibraryInfo
+
+# Library information
+_library_info = LibraryInfo(name="openai-agents")
+LIBRARY_NAME = _library_info.name
+LIBRARY_VERSION = _library_info.version
+
+# Import after defining constants to avoid circular imports
+from agentops.instrumentation.agentic.openai_agents.instrumentor import OpenAIAgentsInstrumentor  # noqa: E402
+
+__all__ = [
+    "LIBRARY_NAME",
+    "LIBRARY_VERSION",
+    "OpenAIAgentsInstrumentor",
+]