otelzero 0.2.0__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- otelzero/__init__.py +62 -0
- otelzero/__main__.py +140 -0
- otelzero/context.py +86 -0
- otelzero/detection.py +58 -0
- otelzero/hooks/__init__.py +96 -0
- otelzero/hooks/_content.py +865 -0
- otelzero/hooks/_langsmith.py +504 -0
- otelzero/hooks/_native.py +517 -0
- otelzero/hooks/_usage.py +166 -0
- otelzero/hooks/adk.py +128 -0
- otelzero/hooks/base.py +546 -0
- otelzero/hooks/claude_sdk.py +267 -0
- otelzero/hooks/crewai.py +338 -0
- otelzero/hooks/databricks.py +260 -0
- otelzero/hooks/langchain.py +56 -0
- otelzero/hooks/langgraph.py +57 -0
- otelzero/hooks/llama_index.py +724 -0
- otelzero/hooks/microsoft.py +160 -0
- otelzero/hooks/openai_agents.py +313 -0
- otelzero/hooks/pydantic_ai.py +173 -0
- otelzero/hooks/smolagents.py +331 -0
- otelzero/hooks/snowflake.py +271 -0
- otelzero/hooks/strands.py +173 -0
- otelzero/otel.py +372 -0
- otelzero/semconv.py +327 -0
- otelzero/telemetry.py +238 -0
- otelzero/translate/__init__.py +214 -0
- otelzero/translate/langsmith.py +401 -0
- otelzero-0.2.0.dist-info/METADATA +1345 -0
- otelzero-0.2.0.dist-info/RECORD +33 -0
- otelzero-0.2.0.dist-info/WHEEL +5 -0
- otelzero-0.2.0.dist-info/licenses/LICENSE +202 -0
- otelzero-0.2.0.dist-info/top_level.txt +1 -0
|
@@ -0,0 +1,1345 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: otelzero
|
|
3
|
+
Version: 0.2.0
|
|
4
|
+
Summary: Instrument agent frameworks (CrewAI, LangChain, LangGraph, LlamaIndex, OpenAI Agents SDK, Pydantic AI, smolagents, Google ADK, AWS Strands, Claude Agent SDK, and more) via Python import hooks and stream telemetry to a Kiji Agent Registry server.
|
|
5
|
+
License-Expression: Apache-2.0
|
|
6
|
+
Project-URL: Homepage, https://github.com/575-lab/otelzero
|
|
7
|
+
Project-URL: Repository, https://github.com/575-lab/otelzero
|
|
8
|
+
Project-URL: Kiji Agent Registry, https://github.com/575-lab/kiji-agent-registry
|
|
9
|
+
Requires-Python: >=3.12
|
|
10
|
+
Description-Content-Type: text/markdown
|
|
11
|
+
License-File: LICENSE
|
|
12
|
+
Requires-Dist: httpx>=0.26
|
|
13
|
+
Requires-Dist: wrapt>=1.16
|
|
14
|
+
Provides-Extra: crewai
|
|
15
|
+
Requires-Dist: crewai==0.1.0; extra == "crewai"
|
|
16
|
+
Provides-Extra: otel
|
|
17
|
+
Requires-Dist: opentelemetry-api>=1.24; extra == "otel"
|
|
18
|
+
Requires-Dist: opentelemetry-sdk>=1.24; extra == "otel"
|
|
19
|
+
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.24; extra == "otel"
|
|
20
|
+
Provides-Extra: langchain
|
|
21
|
+
Requires-Dist: langchain>=1.0; extra == "langchain"
|
|
22
|
+
Requires-Dist: langsmith[otel]>=0.4.25; extra == "langchain"
|
|
23
|
+
Dynamic: license-file
|
|
24
|
+
|
|
25
|
+
# otelzero
|
|
26
|
+
|
|
27
|
+
Instrument agent frameworks via Python **import hooks** and stream telemetry to a
|
|
28
|
+
[Kiji Agent Registry](https://github.com/575-lab/kiji-agent-registry) server — no code changes to your
|
|
29
|
+
agents required. (Import package: `otelzero`.)
|
|
30
|
+
|
|
31
|
+
**Site:** [575-lab.github.io/otelzero](https://575-lab.github.io/otelzero/) — an animated walkthrough of the import hook, with live Kiji and Dataiku Agent Management demos.
|
|
32
|
+
|
|
33
|
+
Install monitoring for your framework *before* importing it, then run as usual:
|
|
34
|
+
|
|
35
|
+
```python
|
|
36
|
+
from otelzero import install_crewai_monitoring
|
|
37
|
+
|
|
38
|
+
install_crewai_monitoring(server_url="http://localhost:8000")
|
|
39
|
+
from crewai import Agent, Crew, Task # now auto-instrumented
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
> Order matters: install the hook **before** importing the framework. The hooks
|
|
43
|
+
> use the `find_spec` import protocol and require **Python ≥ 3.12**.
|
|
44
|
+
|
|
45
|
+
## Supported SDKs / hooks
|
|
46
|
+
|
|
47
|
+
For exactly which agent actions each framework reports (agent runs, model calls,
|
|
48
|
+
tool calls, workflows) and whether they come from the framework's own GenAI
|
|
49
|
+
telemetry or from an otelzero hook, see **[FRAMEWORKS.md](FRAMEWORKS.md)**. For the span attributes each framework emits, see the
|
|
50
|
+
**[attribute matrix](https://575-lab.github.io/otelzero/attributes.html)**.
|
|
51
|
+
|
|
52
|
+
| Framework | Install function | What gets wrapped |
|
|
53
|
+
|---|---|---|
|
|
54
|
+
| [CrewAI](https://github.com/crewAIInc/crewAI) | `install_crewai_monitoring` | `Crew.kickoff`, `Agent.execute_task`, `Task.execute_sync` |
|
|
55
|
+
| [LangChain](https://github.com/langchain-ai/langchain) | `install_langchain_monitoring` | nothing — switches on LangSmith's *native* OpenTelemetry export (translated into the current GenAI conventions on export) |
|
|
56
|
+
| [LangGraph](https://github.com/langchain-ai/langgraph) | `install_langgraph_monitoring` | nothing — same LangSmith export; one run per graph run, subgraphs included in it |
|
|
57
|
+
| [LlamaIndex](https://github.com/run-llama/llama_index) | `install_llama_index_monitoring` | `Workflow.run` (covers `FunctionAgent`, `ReActAgent`, `CodeActAgent`, `AgentWorkflow` and hand-written workflows) |
|
|
58
|
+
| [OpenAI Agents SDK](https://github.com/openai/openai-agents-python) | `install_openai_agents_monitoring` | `Runner.run`, `Runner.run_sync`, `Runner.run_streamed` |
|
|
59
|
+
| [Pydantic AI](https://ai.pydantic.dev) | `install_pydantic_ai_monitoring` | nothing — configures Pydantic AI's *native* OpenTelemetry (`Agent.instrument_all`) |
|
|
60
|
+
| [smolagents](https://github.com/huggingface/smolagents) | `install_smolagents_monitoring` | `MultiStepAgent.run` (covers `CodeAgent` and `ToolCallingAgent`, direct + streaming) |
|
|
61
|
+
| [AWS Strands Agents](https://github.com/strands-agents/sdk-python) | `install_strands_monitoring` | nothing — points Strands' *native* OpenTelemetry at the registry (latest GenAI conventions) |
|
|
62
|
+
| [Google ADK](https://github.com/google/adk-python) | `install_adk_monitoring` | nothing — points ADK's *native* OpenTelemetry at the registry (latest GenAI conventions) |
|
|
63
|
+
| [Databricks Agent Bricks / Mosaic AI](https://docs.databricks.com/aws/en/generative-ai/agent-bricks) | `install_databricks_monitoring` | each `ResponsesAgent` / `ChatAgent` subclass's `predict`, plus its tool calls |
|
|
64
|
+
| [Claude Agent SDK](https://github.com/anthropics/claude-agent-sdk-python) | `install_claude_agent_monitoring` | `query`, `ClaudeSDKClient.receive_response` (+ `ClaudeSDKClient.query`, for its prompt) |
|
|
65
|
+
| [Microsoft Agent Framework](https://github.com/microsoft/agent-framework) | `install_microsoft_monitoring` | nothing — switches on Agent Framework's *native* OpenTelemetry (`enable_instrumentation`) |
|
|
66
|
+
| [Snowflake Cortex Agent SDK](https://docs.snowflake.com/en/user-guide/cortex-code-agent-sdk/cortex-code-agent-sdk) | `install_snowflake_monitoring` | `query`, `CortexCodeSDKClient.receive_response` (+ `CortexCodeSDKClient.query`, for its prompt) |
|
|
67
|
+
| Anything else | `install_monitoring` | framework-agnostic manual instrumentation (`TelemetryClient`) |
|
|
68
|
+
|
|
69
|
+
**Native-OTel frameworks.** Pydantic AI, AWS Strands Agents, Google ADK and
|
|
70
|
+
Microsoft Agent Framework already trace themselves in the
|
|
71
|
+
[GenAI semantic conventions](https://github.com/open-telemetry/semantic-conventions-genai),
|
|
72
|
+
so their install functions wrap nothing. Each one switches on the framework's
|
|
73
|
+
own instrumentation, exports it to the registry's OTLP receiver, stamps
|
|
74
|
+
otelzero's run identity on every span, and derives the usual `agent_start` /
|
|
75
|
+
`agent_end` / `agent_error` events from the framework's `invoke_agent` spans.
|
|
76
|
+
LangChain and LangGraph work the same way through **LangSmith's OpenTelemetry
|
|
77
|
+
export**, whose spans otelzero translates into the current conventions on
|
|
78
|
+
export, with the events coming
|
|
79
|
+
from LangChain's own callbacks (see
|
|
80
|
+
[the LangGraph guide](#instrument-a-real-langgraph-graph-and-langchain-10-agents)).
|
|
81
|
+
For all of these, the import order doesn't matter. The `otel` extra is required,
|
|
82
|
+
and their tracing stays on even with `OTELZERO_OTEL=0`.
|
|
83
|
+
|
|
84
|
+
**LangChain 1.0 note:** `create_agent` (and the older `create_react_agent`) return
|
|
85
|
+
a compiled LangGraph graph. Both installers set up the same LangSmith export, so
|
|
86
|
+
either one captures it. Installing both is safe and still reports one run per
|
|
87
|
+
graph run.
|
|
88
|
+
|
|
89
|
+
Every hook streams the same event envelope (`agent_start` / `agent_end` /
|
|
90
|
+
`agent_error`, plus crew/task events where the framework has them) to the otelzero
|
|
91
|
+
dashboard. With the `otel` extra installed, the hooks also emit standard
|
|
92
|
+
OpenTelemetry spans — **tracing is on by default** (`OTELZERO_OTEL` is
|
|
93
|
+
effectively always `1`; set `OTELZERO_OTEL=0` to switch the import hooks' spans
|
|
94
|
+
off) — and a framework's own native OTel spans (model/tool calls) nest
|
|
95
|
+
underneath otelzero's. Set `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=1`
|
|
96
|
+
to also record each run's prompt and response as `gen_ai.input.messages` /
|
|
97
|
+
`gen_ai.output.messages` (on the span and in the dashboard's event details);
|
|
98
|
+
off by default, capped by `OTELZERO_CONTENT_MAX_CHARS` (default 4000 per part).
|
|
99
|
+
|
|
100
|
+
## Install
|
|
101
|
+
|
|
102
|
+
```bash
|
|
103
|
+
pip install otelzero # core client
|
|
104
|
+
pip install "otelzero[otel]" # + OpenTelemetry span export
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
## Set up a server
|
|
108
|
+
|
|
109
|
+
The client needs a Kiji Agent Registry server (dashboard + telemetry ingest) to send events to.
|
|
110
|
+
|
|
111
|
+
**Self-hosted (minimal):** install the `kiji-agent-registry` distribution and run it:
|
|
112
|
+
|
|
113
|
+
```bash
|
|
114
|
+
pip install kiji-agent-registry # add [otel] for the OTLP ingest endpoint
|
|
115
|
+
kiji-agent-registry # dashboard + ingest on http://localhost:8000
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
**In your own cloud:** `marketplace/` in the project repo packages the server for the
|
|
119
|
+
GCP, AWS and Azure marketplaces — one VM plus a data disk inside your own project,
|
|
120
|
+
account or subscription. The listings are still in publisher review; until they land,
|
|
121
|
+
the Docker Compose and container paths in the project README cover the same ground.
|
|
122
|
+
Point the client at whatever URL your deployment serves:
|
|
123
|
+
|
|
124
|
+
```python
|
|
125
|
+
install_crewai_monitoring(
|
|
126
|
+
server_url="https://kiji.your-vpc.internal",
|
|
127
|
+
api_key="your-ingest-key", # or set OTELZERO_API_KEY
|
|
128
|
+
)
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
**Managed:** [Dataiku Agent Management](https://www.dataiku.com) runs it for you —
|
|
132
|
+
agent registry, drift detection and governance.
|
|
133
|
+
|
|
134
|
+
Ingest is owner-scoped: pass an ingest key via
|
|
135
|
+
`install_*_monitoring(..., api_key="...")` or the `OTELZERO_API_KEY` env var (any
|
|
136
|
+
string ≥ 8 chars auto-provisions its own owner on a fresh server).
|
|
137
|
+
|
|
138
|
+
Docker images, OTLP/collector ingest, and the mobile companion apps are covered
|
|
139
|
+
in the [project README](https://github.com/575-lab/kiji-agent-registry).
|
|
140
|
+
|
|
141
|
+
## Try it against a local registry
|
|
142
|
+
|
|
143
|
+
Run a [Kiji Agent Registry](https://github.com/575-lab/kiji-agent-registry) next to this checkout, then drive it with the
|
|
144
|
+
framework-free simulator:
|
|
145
|
+
|
|
146
|
+
```bash
|
|
147
|
+
git clone https://github.com/575-lab/kiji-agent-registry ../kiji-agent-registry
|
|
148
|
+
(cd ../kiji-agent-registry && uv run kiji-agent-registry) # terminal 1
|
|
149
|
+
uv run python demo/demo_simulator.py # terminal 2
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
Open http://localhost:8000 to watch the runs arrive. The commands below assume
|
|
153
|
+
the registry checkout sits next to this one at `../kiji-agent-registry`.
|
|
154
|
+
|
|
155
|
+
## Framework guides
|
|
156
|
+
|
|
157
|
+
### Instrument a real CrewAI crew
|
|
158
|
+
|
|
159
|
+
The import hook wraps the **real** `crewai` package, so install monitoring
|
|
160
|
+
**before** importing CrewAI — then write your crew normally, with no telemetry
|
|
161
|
+
code in it:
|
|
162
|
+
|
|
163
|
+
```python
|
|
164
|
+
from otelzero import install_crewai_monitoring
|
|
165
|
+
|
|
166
|
+
# 1) Install the hook FIRST (registers a find_spec meta-path finder)
|
|
167
|
+
install_crewai_monitoring(server_url="http://localhost:8000", user="Sarah")
|
|
168
|
+
|
|
169
|
+
# 2) Now import CrewAI — Crew/Agent/Task are auto-instrumented as they load
|
|
170
|
+
from crewai import Agent, Crew, Task
|
|
171
|
+
|
|
172
|
+
researcher = Agent(role="Research Analyst", goal="...", backstory="...")
|
|
173
|
+
task = Task(description="...", expected_output="...", agent=researcher)
|
|
174
|
+
crew = Crew(agents=[researcher], tasks=[task])
|
|
175
|
+
|
|
176
|
+
crew.kickoff() # streams crew/agent/task events to the dashboard automatically
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
A complete, runnable version is in **`demo/demo_crewai.py`**:
|
|
180
|
+
|
|
181
|
+
```bash
|
|
182
|
+
uv pip install crewai # the hook instruments the real package
|
|
183
|
+
export OPENAI_API_KEY=... # for a full kickoff
|
|
184
|
+
uv run python demo/demo_crewai.py
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
> Order matters: instrumentation must be installed **before** `crewai` is
|
|
188
|
+
> imported. The hooks use the `find_spec` protocol and require **Python ≥ 3.12**.
|
|
189
|
+
|
|
190
|
+
### Instrument a real LangGraph graph (and LangChain 1.0 agents)
|
|
191
|
+
|
|
192
|
+
LangChain and LangGraph trace themselves through **LangSmith**, and the
|
|
193
|
+
`langsmith` SDK can export those traces as OpenTelemetry spans
|
|
194
|
+
(`langsmith[otel]`). Every graph run, node, subgraph, chain, model and tool call
|
|
195
|
+
becomes one span, named after the run (`Research Graph`, `model`, `tools`, a
|
|
196
|
+
tool's name). It carries `langsmith.span.kind`, `gen_ai.operation.name`,
|
|
197
|
+
`gen_ai.request.model`, `gen_ai.usage.*` and `langsmith.metadata.*`.
|
|
198
|
+
`install_langgraph_monitoring` and `install_langchain_monitoring` wrap nothing.
|
|
199
|
+
They:
|
|
200
|
+
|
|
201
|
+
- make otelzero's OTLP provider the global one, which is where LangSmith's
|
|
202
|
+
exporter sends its spans. On their way out they pass through otelzero's
|
|
203
|
+
translator ([below](#how-langsmiths-spans-are-translated)):
|
|
204
|
+
the root run becomes `invoke_agent {run}`, model calls `chat {model}`, tool
|
|
205
|
+
calls `execute_tool {tool}`. If your application set its own global tracer
|
|
206
|
+
provider first, they go out through that one, untranslated;
|
|
207
|
+
- turn the export on through LangSmith's own variables: `LANGSMITH_TRACING`,
|
|
208
|
+
`LANGSMITH_OTEL_ENABLED` and, unless a LangSmith API key is configured,
|
|
209
|
+
`LANGSMITH_OTEL_ONLY` (so nothing goes to LangSmith's cloud). Unless content
|
|
210
|
+
capture is on, they also set `LANGSMITH_HIDE_INPUTS` / `LANGSMITH_HIDE_OUTPUTS`
|
|
211
|
+
(LangSmith otherwise exports every prompt and completion). Values you set
|
|
212
|
+
yourself are kept, and an uninstall removes only what was added;
|
|
213
|
+
- register a callback handler through LangChain's own `register_configure_hook`.
|
|
214
|
+
LangChain adds it to every run the way it adds its own tracers. It sends one
|
|
215
|
+
`agent_start` / `agent_end` / `agent_error` per **top-level** run (the
|
|
216
|
+
outermost graph, agent, chain or runnable you invoked), with the model, the
|
|
217
|
+
tools it called and its token usage collected from the model and tool runs
|
|
218
|
+
under it. Nodes and subgraphs are part of the run, so one graph run is one
|
|
219
|
+
agent card. The session is titled from the run's input.
|
|
220
|
+
|
|
221
|
+
```python
|
|
222
|
+
from otelzero import install_langgraph_monitoring
|
|
223
|
+
from langgraph.graph import END, START, MessagesState, StateGraph # any order
|
|
224
|
+
|
|
225
|
+
install_langgraph_monitoring(server_url="http://localhost:8000", user="Sarah")
|
|
226
|
+
|
|
227
|
+
graph = StateGraph(MessagesState)
|
|
228
|
+
graph.add_node("agent", call_model)
|
|
229
|
+
graph.add_edge(START, "agent")
|
|
230
|
+
graph.add_edge("agent", END)
|
|
231
|
+
app = graph.compile(name="Research Graph")
|
|
232
|
+
|
|
233
|
+
app.invoke({"messages": [("user", "who won?")]}) # one agent card per run
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
**Why the events come from callbacks.** LangSmith builds its spans on a
|
|
237
|
+
background thread, when it flushes its queue. A finished run's span is created
|
|
238
|
+
and ended in one go, before its children's spans exist. The callbacks fire in
|
|
239
|
+
your code's context, as the run happens, so the events are live.
|
|
240
|
+
|
|
241
|
+
**Identity.** A run names its own thread and user in its metadata:
|
|
242
|
+
`config={"metadata": {"thread_id": ..., "user_name": ...}}` (also
|
|
243
|
+
`conversation_id` / `session_id`, `user_id` / `user`), or LangGraph's
|
|
244
|
+
`config={"configurable": {"thread_id": ...}}`. Its events and its spans both
|
|
245
|
+
use that, so the dashboard groups them together. Without metadata, the events
|
|
246
|
+
fall back to the active `run_context()`, then to the client's `user=` /
|
|
247
|
+
`session=`. The spans fall back to the client's only: a span can't be tied
|
|
248
|
+
back to the run it records, so `run_context()` doesn't reach them.
|
|
249
|
+
|
|
250
|
+
**Short scripts:** call `langchain_core.tracers.langchain.wait_for_all_tracers()`
|
|
251
|
+
before `otel.force_flush()` at exit, so LangSmith's queue is drained into spans
|
|
252
|
+
first.
|
|
253
|
+
|
|
254
|
+
Token usage comes from the model calls a run makes (their `LLMResult`). A graph
|
|
255
|
+
node that returns prepared messages without calling a model reports none.
|
|
256
|
+
|
|
257
|
+
A complete, runnable version is in **`demo/demo_langgraph.py`** (fully offline —
|
|
258
|
+
its nodes return prepared replies instead of calling a model, and it contains a
|
|
259
|
+
subgraph so you can see it counted as part of its parent's run):
|
|
260
|
+
|
|
261
|
+
```bash
|
|
262
|
+
uv pip install langgraph "langsmith[otel]" "otelzero[otel]"
|
|
263
|
+
uv run python demo/demo_langgraph.py
|
|
264
|
+
```
|
|
265
|
+
|
|
266
|
+
### Instrument a real OpenAI Agents SDK agent
|
|
267
|
+
|
|
268
|
+
The [OpenAI Agents SDK](https://github.com/openai/openai-agents-python) (the
|
|
269
|
+
`openai-agents` package, OpenAI's production successor to the archived Swarm)
|
|
270
|
+
starts every run through one of three `Runner` classmethods, and
|
|
271
|
+
`install_openai_agents_monitoring` wraps all three — `run`, `run_sync` and
|
|
272
|
+
`run_streamed` — so a run is captured exactly once whichever you use:
|
|
273
|
+
|
|
274
|
+
```python
|
|
275
|
+
from otelzero import install_openai_agents_monitoring
|
|
276
|
+
|
|
277
|
+
# 1) Install the hook FIRST (registers a find_spec meta-path finder)
|
|
278
|
+
install_openai_agents_monitoring(server_url="http://localhost:8000", user="Sarah")
|
|
279
|
+
|
|
280
|
+
# 2) Now import the SDK — Runner is auto-instrumented as it loads
|
|
281
|
+
from agents import Agent, Runner
|
|
282
|
+
|
|
283
|
+
agent = Agent(name="Weather Agent", instructions="...", model="gpt-4o-mini")
|
|
284
|
+
|
|
285
|
+
await Runner.run(agent, "What is the weather in Amsterdam?")
|
|
286
|
+
```
|
|
287
|
+
|
|
288
|
+
`run_sync` drives `run` underneath, so both are wrapped but only the outer call
|
|
289
|
+
opens a span. A streamed run is traced across its `stream_events()` pass, since
|
|
290
|
+
that is when the run is actually delivered.
|
|
291
|
+
|
|
292
|
+
The `agent_start` details carry the run's `model`, `tools` and `handoffs`; token
|
|
293
|
+
counts come from the run context's `usage`. A complete, runnable version is in
|
|
294
|
+
**`demo/demo_openai_agents.py`**:
|
|
295
|
+
|
|
296
|
+
```bash
|
|
297
|
+
uv pip install openai-agents # the hook instruments the real package
|
|
298
|
+
export OPENAI_API_KEY=... # the SDK has no offline model
|
|
299
|
+
uv run python demo/demo_openai_agents.py
|
|
300
|
+
```
|
|
301
|
+
|
|
302
|
+
> Order matters: instrumentation must be installed **before** `agents` is
|
|
303
|
+
> imported. The hooks use the `find_spec` protocol and require **Python ≥ 3.12**.
|
|
304
|
+
|
|
305
|
+
### Instrument a real Pydantic AI agent
|
|
306
|
+
|
|
307
|
+
[Pydantic AI](https://ai.pydantic.dev) ships its own OpenTelemetry
|
|
308
|
+
instrumentation, already in the GenAI semantic conventions: one
|
|
309
|
+
`invoke_agent {name}` span per run with `chat` and `execute_tool` children,
|
|
310
|
+
`gen_ai.*` usage attributes and opt-in message content. That is richer than a
|
|
311
|
+
wrapper around `Agent.run` could see from the outside, so
|
|
312
|
+
`install_pydantic_ai_monitoring` wraps nothing — it points the native
|
|
313
|
+
instrumentation (`Agent.instrument_all`) at the registry's OTLP receiver,
|
|
314
|
+
stamps otelzero's run identity (`gen_ai.conversation.id`, `user.name`,
|
|
315
|
+
`otelzero.session.name`) onto every span, and turns each native `invoke_agent`
|
|
316
|
+
span into the usual `agent_start` / `agent_end` / `agent_error` events:
|
|
317
|
+
|
|
318
|
+
```python
|
|
319
|
+
from otelzero import install_pydantic_ai_monitoring
|
|
320
|
+
from pydantic_ai import Agent # import order does not matter for this one
|
|
321
|
+
|
|
322
|
+
install_pydantic_ai_monitoring(
|
|
323
|
+
server_url="http://localhost:8000",
|
|
324
|
+
user="Sarah",
|
|
325
|
+
name="Order support triage", # session title, on every event and span
|
|
326
|
+
)
|
|
327
|
+
|
|
328
|
+
agent = Agent("openai:gpt-4o", name="Support Agent")
|
|
329
|
+
|
|
330
|
+
result = await agent.run("What is the status of order A-1042?")
|
|
331
|
+
async with agent.run_stream("...") as stream: # streaming runs too
|
|
332
|
+
...
|
|
333
|
+
```
|
|
334
|
+
|
|
335
|
+
The events are derived from the spans, so this hook needs the `otel` extra, and
|
|
336
|
+
its tracing stays on even with `OTELZERO_OTEL=0`. They carry what
|
|
337
|
+
Pydantic AI has recorded: `model` at start; `tools` and `usage` at the end;
|
|
338
|
+
and on failure the exception message and traceback only when content capture is
|
|
339
|
+
on (otherwise just the exception type). Pydantic AI's spans hold no prompt when
|
|
340
|
+
a run starts, so — unlike the wrapping hooks — no session title is derived from
|
|
341
|
+
it: pass `name=` (or `run_context(name=...)`) to title the session. Prompts and
|
|
342
|
+
completions follow the usual `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT`
|
|
343
|
+
opt-in; pass `settings=InstrumentationSettings(...)` to choose a different
|
|
344
|
+
format version or content policy.
|
|
345
|
+
|
|
346
|
+
A complete, runnable version is in **`demo/demo_pydantic_ai.py`** (fully offline —
|
|
347
|
+
it uses Pydantic AI's own `TestModel`, so no API key is needed):
|
|
348
|
+
|
|
349
|
+
```bash
|
|
350
|
+
uv pip install pydantic-ai "otelzero[otel]"
|
|
351
|
+
uv run python demo/demo_pydantic_ai.py
|
|
352
|
+
uv run python demo/demo_pydantic_ai.py --model openai:gpt-4o # a real model
|
|
353
|
+
```
|
|
354
|
+
|
|
355
|
+
|
|
356
|
+
### Instrument a real smolagents agent
|
|
357
|
+
|
|
358
|
+
Hugging Face's [smolagents](https://github.com/huggingface/smolagents) builds
|
|
359
|
+
both of its agent flavours — `CodeAgent`, which writes its actions as Python, and
|
|
360
|
+
`ToolCallingAgent`, which emits structured tool calls — on one `MultiStepAgent`
|
|
361
|
+
base, and neither overrides `run`. So `install_smolagents_monitoring` wraps that
|
|
362
|
+
single method and covers every agent the library can build:
|
|
363
|
+
|
|
364
|
+
```python
|
|
365
|
+
from otelzero import install_smolagents_monitoring
|
|
366
|
+
|
|
367
|
+
# 1) Install the hook FIRST (registers a find_spec meta-path finder)
|
|
368
|
+
install_smolagents_monitoring(server_url="http://localhost:8000", user="Sarah")
|
|
369
|
+
|
|
370
|
+
# 2) Now import smolagents — MultiStepAgent.run is auto-instrumented as it loads
|
|
371
|
+
from smolagents import CodeAgent, InferenceClientModel
|
|
372
|
+
|
|
373
|
+
agent = CodeAgent(tools=[...], model=InferenceClientModel(), name="Travel Agent")
|
|
374
|
+
|
|
375
|
+
agent.run("How long is the drive from Paris to Lyon?") # streams agent events
|
|
376
|
+
for step in agent.run("...", stream=True): # the streaming shape too
|
|
377
|
+
...
|
|
378
|
+
```
|
|
379
|
+
|
|
380
|
+
`run` is a synchronous dispatcher whose `stream` argument decides its return
|
|
381
|
+
shape — the final answer, or a generator of steps — and otelzero traces whichever
|
|
382
|
+
it is. Unlike a framework that hands back an awaitable, the direct call performs
|
|
383
|
+
the whole run before returning, so the streaming decision is read off the
|
|
384
|
+
argument rather than the returned object.
|
|
385
|
+
|
|
386
|
+
Token counts come from the agent's own monitor, which totals them **across** the
|
|
387
|
+
runs it has seen. otelzero reports the difference across each run, so a
|
|
388
|
+
continuation (`agent.run(..., reset=False)`) is charged only for what it actually
|
|
389
|
+
spent rather than for the whole conversation so far.
|
|
390
|
+
|
|
391
|
+
A complete, runnable version is in **`demo/demo_smolagents.py`**:
|
|
392
|
+
|
|
393
|
+
```bash
|
|
394
|
+
uv pip install smolagents # the hook instruments the real package
|
|
395
|
+
export HF_TOKEN=... # the default InferenceClientModel needs it
|
|
396
|
+
uv run python demo/demo_smolagents.py
|
|
397
|
+
```
|
|
398
|
+
|
|
399
|
+
> Order matters: instrumentation must be installed **before** `smolagents` is
|
|
400
|
+
> imported. The hooks use the `find_spec` protocol and require **Python ≥ 3.12**.
|
|
401
|
+
|
|
402
|
+
### Instrument a real LlamaIndex agent
|
|
403
|
+
|
|
404
|
+
Every modern [LlamaIndex](https://github.com/run-llama/llama_index) agent —
|
|
405
|
+
`FunctionAgent`, `ReActAgent`, `CodeActAgent`, the multi-agent `AgentWorkflow` —
|
|
406
|
+
is a *workflow*: each overrides `run` only to package its arguments into a start
|
|
407
|
+
event and then calls `Workflow.run`. So `install_llama_index_monitoring` wraps
|
|
408
|
+
that one method and covers every agent the library can build, plus any workflow
|
|
409
|
+
you write by hand:
|
|
410
|
+
|
|
411
|
+
```python
|
|
412
|
+
from otelzero import install_llama_index_monitoring
|
|
413
|
+
|
|
414
|
+
# 1) Install the hook FIRST (registers a find_spec meta-path finder)
|
|
415
|
+
install_llama_index_monitoring(server_url="http://localhost:8000", user="Sarah")
|
|
416
|
+
|
|
417
|
+
# 2) Now import LlamaIndex — Workflow.run is auto-instrumented as it loads
|
|
418
|
+
from llama_index.core.agent.workflow import FunctionAgent
|
|
419
|
+
from llama_index.llms.openai import OpenAI
|
|
420
|
+
|
|
421
|
+
agent = FunctionAgent(name="Support Agent", tools=[lookup_order], llm=OpenAI(model="gpt-4o-mini"))
|
|
422
|
+
|
|
423
|
+
await agent.run(user_msg="What is the status of order A-1042?") # streams agent events
|
|
424
|
+
```
|
|
425
|
+
|
|
426
|
+
`Workflow.run` is **synchronous**: it schedules an asyncio task and hands back a
|
|
427
|
+
`WorkflowHandler` while the run continues in the background. otelzero therefore
|
|
428
|
+
opens the run when `run()` is called and closes it from a done-callback on that
|
|
429
|
+
task — so a run is captured whether you await the handler, read only
|
|
430
|
+
`stream_events()`, or cancel it with `cancel_run()` (which closes the run as
|
|
431
|
+
`interrupted` rather than leaving it dangling). An agent or workflow started from
|
|
432
|
+
inside a tool folds into the run it belongs to, the way a LangGraph subgraph is
|
|
433
|
+
part of its parent's run.
|
|
434
|
+
|
|
435
|
+
Token counts do not ride on the result — `AgentOutput` has no usage — so otelzero
|
|
436
|
+
registers one handler on LlamaIndex's own instrumentation dispatcher and sums the
|
|
437
|
+
`LLMChatEndEvent` / `LLMCompletionEndEvent` payloads a run produces. Predict and
|
|
438
|
+
structured-predict events are deliberately not counted: each wraps an inner chat
|
|
439
|
+
call that is already counted, and a `CustomLLM` that reports one payload through
|
|
440
|
+
two events is counted once.
|
|
441
|
+
|
|
442
|
+
Three limitations worth knowing:
|
|
443
|
+
|
|
444
|
+
- a **human-in-the-loop** run that is never resumed never closes, so its card
|
|
445
|
+
stays open;
|
|
446
|
+
- **OpenAI streaming reports no tokens** unless the model is built with
|
|
447
|
+
`additional_kwargs={"stream_options": {"include_usage": True}}` (or
|
|
448
|
+
`streaming=False`) — and `FunctionAgent` streams by default;
|
|
449
|
+
- LLM events that arrive **after** a run has closed are dropped.
|
|
450
|
+
|
|
451
|
+
A complete, runnable version is in **`demo/demo_llama_index.py`**:
|
|
452
|
+
|
|
453
|
+
```bash
|
|
454
|
+
uv pip install llama-index-core # the hook instruments the real package
|
|
455
|
+
uv run python demo/demo_llama_index.py # fully offline
|
|
456
|
+
uv run python demo/demo_llama_index.py --model gpt-4o-mini # a real model
|
|
457
|
+
```
|
|
458
|
+
|
|
459
|
+
The offline run uses LlamaIndex's own `MockFunctionCallingLLM`, which returns no
|
|
460
|
+
provider payload — and a provider payload is where token counts live — so the
|
|
461
|
+
offline cards show runs without tokens. `--model` gives real counts.
|
|
462
|
+
|
|
463
|
+
> Order matters: instrumentation must be installed **before** `llama_index` is
|
|
464
|
+
> imported. The hooks use the `find_spec` protocol and require **Python ≥ 3.12**.
|
|
465
|
+
|
|
466
|
+
### Instrument a real AWS Strands Agents agent
|
|
467
|
+
|
|
468
|
+
[AWS Strands Agents](https://github.com/strands-agents/sdk-python) always traces
|
|
469
|
+
itself through the global OpenTelemetry tracer provider, in the GenAI semantic
|
|
470
|
+
conventions. Each run gets an `invoke_agent {name}` span, with
|
|
471
|
+
`execute_event_loop_cycle`, `chat` and `execute_tool {tool}` spans under it,
|
|
472
|
+
plus `gen_ai.*` model, tool and usage attributes. `install_strands_monitoring`
|
|
473
|
+
wraps nothing. It:
|
|
474
|
+
|
|
475
|
+
- makes otelzero's OTLP provider the global one, so Strands' spans export to the
|
|
476
|
+
registry's `/v1/traces`;
|
|
477
|
+
- opts Strands into the **latest** GenAI conventions
|
|
478
|
+
(`OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental`: `gen_ai.provider.name`,
|
|
479
|
+
`gen_ai.input.messages` / `gen_ai.output.messages`);
|
|
480
|
+
- applies otelzero's content opt-in. By default Strands records prompts and
|
|
481
|
+
completions unconditionally. With content capture off, the hook turns on
|
|
482
|
+
Strands' redaction (`gen_ai_unredacted_attributes=`). With
|
|
483
|
+
`OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=1`, the content goes on
|
|
484
|
+
span attributes (`gen_ai_span_attributes_only`) together with tool definitions;
|
|
485
|
+
- stamps otelzero's run identity on every Strands span, and turns each
|
|
486
|
+
`invoke_agent` span into `agent_start` / `agent_end` / `agent_error`.
|
|
487
|
+
|
|
488
|
+
Opt-in tokens you set yourself are kept, and an uninstall removes only the
|
|
489
|
+
ones the hook added.
|
|
490
|
+
|
|
491
|
+
```python
|
|
492
|
+
from otelzero import install_strands_monitoring
|
|
493
|
+
from strands import Agent, tool # import order does not matter for this one
|
|
494
|
+
|
|
495
|
+
install_strands_monitoring(
|
|
496
|
+
server_url="http://localhost:8000",
|
|
497
|
+
user="Sarah",
|
|
498
|
+
name="Calculator session", # session title, on every event and span
|
|
499
|
+
)
|
|
500
|
+
|
|
501
|
+
@tool
|
|
502
|
+
def add(a: int, b: int) -> int:
|
|
503
|
+
"""Add two integers."""
|
|
504
|
+
return a + b
|
|
505
|
+
|
|
506
|
+
agent = Agent(name="Calculator Agent", tools=[add])
|
|
507
|
+
|
|
508
|
+
agent("What is 21 + 21?") # Strands' own spans -> registry; agent_* events derived from them
|
|
509
|
+
```
|
|
510
|
+
|
|
511
|
+
Strands' agent span carries no prompt when a run starts, so no session title
|
|
512
|
+
is derived from the prompt. Pass `name=` (or `run_context(name=...)`) to set
|
|
513
|
+
one.
|
|
514
|
+
|
|
515
|
+
A complete, runnable version is in **`demo/demo_strands.py`**:
|
|
516
|
+
|
|
517
|
+
```bash
|
|
518
|
+
uv pip install strands-agents "otelzero[otel]"
|
|
519
|
+
# configure AWS Bedrock credentials for a full run (Strands' default model)
|
|
520
|
+
uv run python demo/demo_strands.py
|
|
521
|
+
```
|
|
522
|
+
|
|
523
|
+
### Instrument a real Google ADK agent
|
|
524
|
+
|
|
525
|
+
[Google ADK](https://github.com/google/adk-python) (Agent Development Kit) also
|
|
526
|
+
traces itself through the global tracer provider. A `Runner` run opens an
|
|
527
|
+
`invoke_workflow {app}` span. Each agent that takes a turn gets an
|
|
528
|
+
`invoke_agent {name}` span under it, with `generate_content {model}` and
|
|
529
|
+
`execute_tool {tool}` spans nested inside. `install_adk_monitoring` wraps
|
|
530
|
+
nothing. It makes otelzero's OTLP provider the global one, opts ADK into the
|
|
531
|
+
latest GenAI conventions (`gen_ai_latest_experimental`, which replaces ADK's
|
|
532
|
+
legacy `gcp.vertex.agent.*` payload attributes), stamps `user.name` and
|
|
533
|
+
`otelzero.session.name` on every ADK span, and turns each `invoke_agent` span
|
|
534
|
+
into `agent_start` / `agent_end` / `agent_error`. A multi-agent run reports
|
|
535
|
+
each agent that takes a turn.
|
|
536
|
+
|
|
537
|
+
```python
|
|
538
|
+
from otelzero import install_adk_monitoring
|
|
539
|
+
from google.adk.agents import Agent # import order does not matter
|
|
540
|
+
from google.adk.runners import InMemoryRunner
|
|
541
|
+
|
|
542
|
+
install_adk_monitoring(server_url="http://localhost:8000", user="Sarah")
|
|
543
|
+
|
|
544
|
+
agent = Agent(name="calculator_agent", model="gemini-3.7-flash", tools=[...])
|
|
545
|
+
runner = InMemoryRunner(agent=agent, app_name="my-app")
|
|
546
|
+
|
|
547
|
+
# Each runner.run / run_async: ADK's own spans -> registry, agent_* events from them
|
|
548
|
+
```
|
|
549
|
+
|
|
550
|
+
**Session.** ADK already writes its own session id as `gen_ai.conversation.id`,
|
|
551
|
+
after each span starts, so otelzero can't replace it. It is also the right
|
|
552
|
+
value: an ADK session *is* the conversation, and you choose it when you call
|
|
553
|
+
`run_async(session_id=...)`. So ADK runs are grouped by ADK's session id, the
|
|
554
|
+
events are filed under it too, and the `session=` argument only applies to a
|
|
555
|
+
run whose spans carry none.
|
|
556
|
+
|
|
557
|
+
**Content.** ADK reads `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT` on
|
|
558
|
+
its own terms. `1` / `true` mean `EVENT_ONLY` there (log records, not spans).
|
|
559
|
+
Set `SPAN_ONLY` or `SPAN_AND_EVENT` to put prompts and completions on the spans
|
|
560
|
+
the registry ingests.
|
|
561
|
+
|
|
562
|
+
A complete, runnable version is in **`demo/demo_adk.py`**:
|
|
563
|
+
|
|
564
|
+
```bash
|
|
565
|
+
uv pip install google-adk "otelzero[otel]"
|
|
566
|
+
export GOOGLE_API_KEY=... # for a full run (ADK's default Gemini model)
|
|
567
|
+
uv run python demo/demo_adk.py
|
|
568
|
+
```
|
|
569
|
+
|
|
570
|
+
### Instrument a real Databricks Agent Bricks / Mosaic AI agent
|
|
571
|
+
|
|
572
|
+
Databricks agents ([Agent Bricks](https://docs.databricks.com/aws/en/generative-ai/agent-bricks),
|
|
573
|
+
the Mosaic AI Agent Framework) are authored as MLflow `ResponsesAgent`
|
|
574
|
+
subclasses (the current recommended interface; the older `ChatAgent` is
|
|
575
|
+
supported identically), where `predict` is an **abstract** method each agent
|
|
576
|
+
implements — so there's no single concrete method on the base to wrap.
|
|
577
|
+
`install_databricks_monitoring` intercepts subclass creation and wraps each
|
|
578
|
+
concrete agent's `predict` as it's defined, composing on top of MLflow's own
|
|
579
|
+
validation wrapper:
|
|
580
|
+
|
|
581
|
+
```python
|
|
582
|
+
from otelzero import install_databricks_monitoring
|
|
583
|
+
|
|
584
|
+
# 1) Install the hook FIRST (it patches ResponsesAgent.__init_subclass__)
|
|
585
|
+
install_databricks_monitoring(server_url="http://localhost:8000", user="Sarah")
|
|
586
|
+
|
|
587
|
+
# 2) Now define your agent — its predict is auto-instrumented at class creation
|
|
588
|
+
from mlflow.pyfunc import ResponsesAgent
|
|
589
|
+
from mlflow.types.responses import ResponsesAgentRequest, ResponsesAgentResponse
|
|
590
|
+
|
|
591
|
+
class MyAgent(ResponsesAgent):
|
|
592
|
+
def predict(self, request: ResponsesAgentRequest) -> ResponsesAgentResponse:
|
|
593
|
+
... # each call streams agent_start/agent_end/agent_error to the dashboard
|
|
594
|
+
```
|
|
595
|
+
|
|
596
|
+
A complete, runnable version is in **`demo/demo_databricks.py`** (fully offline —
|
|
597
|
+
its agent answers by calling two local stand-in tools, so no Databricks workspace
|
|
598
|
+
or LLM key is needed):
|
|
599
|
+
|
|
600
|
+
```bash
|
|
601
|
+
uv pip install mlflow # provides mlflow.pyfunc.ResponsesAgent
|
|
602
|
+
uv run python demo/demo_databricks.py
|
|
603
|
+
```
|
|
604
|
+
|
|
605
|
+
Beyond the agent run, the tools a `ResponsesAgent` calls — recorded in its
|
|
606
|
+
response as `function_call` items (with results in matching
|
|
607
|
+
`function_call_output` items) — are surfaced as nested `task_start`/`task_end`
|
|
608
|
+
events, one per tool, on the agent's timeline. So the demo run shows
|
|
609
|
+
`agent_start` → a task per tool → `agent_end`, with no extra agent code.
|
|
610
|
+
|
|
611
|
+
A production Agent Bricks agent (Knowledge Assistant, Information Extraction,
|
|
612
|
+
Multi-Agent Supervisor, Custom LLM) is the same `ResponsesAgent` interface and is
|
|
613
|
+
instrumented identically. Only `predict` is wrapped (the required entry point);
|
|
614
|
+
`predict_stream` is left alone to avoid double-counting agents whose `predict`
|
|
615
|
+
aggregates it internally.
|
|
616
|
+
|
|
617
|
+
### Instrument a real Claude Agent SDK agent
|
|
618
|
+
|
|
619
|
+
The [Claude Agent SDK](https://github.com/anthropics/claude-agent-sdk-python)
|
|
620
|
+
exposes two async-generator entry points, and `install_claude_agent_monitoring`
|
|
621
|
+
wraps **both** so a run is captured exactly once whichever style you use: the
|
|
622
|
+
one-shot `claude_agent_sdk.query(...)` function, and the interactive
|
|
623
|
+
`ClaudeSDKClient.receive_response()` method. `ClaudeSDKClient.query()` is wrapped
|
|
624
|
+
too — not as a run, but so the prompt it sends is recorded on the
|
|
625
|
+
`receive_response()` that answers it. (Note: the SDK's own `hooks=` option
|
|
626
|
+
— PreToolUse/PostToolUse — is unrelated to otelzero's import hooks.)
|
|
627
|
+
|
|
628
|
+
```python
|
|
629
|
+
from otelzero import install_claude_agent_monitoring
|
|
630
|
+
|
|
631
|
+
# 1) Install the hook FIRST (registers a find_spec meta-path finder)
|
|
632
|
+
install_claude_agent_monitoring(server_url="http://localhost:8000", user="Sarah")
|
|
633
|
+
|
|
634
|
+
# 2) Now import the SDK — query and ClaudeSDKClient are auto-instrumented
|
|
635
|
+
from claude_agent_sdk import query, ClaudeAgentOptions
|
|
636
|
+
|
|
637
|
+
options = ClaudeAgentOptions(model="claude-opus-4-8", allowed_tools=["Read"])
|
|
638
|
+
|
|
639
|
+
# Each query() run streams agent_start/agent_end/agent_error events
|
|
640
|
+
async for message in query(prompt="Summarize README.md", options=options):
|
|
641
|
+
...
|
|
642
|
+
```
|
|
643
|
+
|
|
644
|
+
A complete, runnable version is in **`demo/demo_claude_sdk.py`**:
|
|
645
|
+
|
|
646
|
+
```bash
|
|
647
|
+
uv pip install claude-agent-sdk # the hook instruments the real package
|
|
648
|
+
export ANTHROPIC_API_KEY=... # for a full run
|
|
649
|
+
uv run python demo/demo_claude_sdk.py
|
|
650
|
+
```
|
|
651
|
+
|
|
652
|
+
The `agent_start` details carry the run's `model`, `allowed_tools`, and (names
|
|
653
|
+
only) `mcp_servers`. With tracing on (the default), the SDK's own spans nest
|
|
654
|
+
underneath otelzero's `invoke_agent Claude Agent` span.
|
|
655
|
+
|
|
656
|
+
> Order matters: instrumentation must be installed **before** `claude_agent_sdk`
|
|
657
|
+
> is imported. The hooks use the `find_spec` protocol and require **Python ≥ 3.12**.
|
|
658
|
+
|
|
659
|
+
### Instrument a real Microsoft Agent Framework agent
|
|
660
|
+
|
|
661
|
+
[Microsoft Agent Framework](https://github.com/microsoft/agent-framework) (the
|
|
662
|
+
`agent-framework` package that unifies Semantic Kernel and AutoGen) ships
|
|
663
|
+
OpenTelemetry instrumentation in the GenAI semantic conventions. Every
|
|
664
|
+
`Agent.run`, streaming or not, gets an `invoke_agent {name}` span, with `chat`
|
|
665
|
+
and `execute_tool` spans from its chat client and function-invocation layers.
|
|
666
|
+
`install_microsoft_monitoring` wraps nothing. It makes otelzero's OTLP provider
|
|
667
|
+
the global one and calls `agent_framework.observability.enable_instrumentation`
|
|
668
|
+
with `enable_sensitive_data` following otelzero's content opt-in. It keeps the
|
|
669
|
+
framework on the latest conventions even when `OTEL_SEMCONV_STABILITY_OPT_IN` is
|
|
670
|
+
set for another library, stamps otelzero's run identity on every span, and
|
|
671
|
+
turns each `invoke_agent` span into `agent_start` / `agent_end` / `agent_error`:
|
|
672
|
+
|
|
673
|
+
```python
|
|
674
|
+
from otelzero import install_microsoft_monitoring
|
|
675
|
+
from agent_framework import Agent # import order does not matter
|
|
676
|
+
from agent_framework.openai import OpenAIChatClient
|
|
677
|
+
|
|
678
|
+
install_microsoft_monitoring(server_url="http://localhost:8000", user="Sarah")
|
|
679
|
+
|
|
680
|
+
agent = Agent(OpenAIChatClient(model="gpt-4o"), name="Weather Agent")
|
|
681
|
+
|
|
682
|
+
await agent.run("What's the weather in Amsterdam?") # non-streaming
|
|
683
|
+
async for update in agent.run("...", stream=True): # streaming
|
|
684
|
+
...
|
|
685
|
+
# both paths: the framework's own spans -> registry, agent_* events from them
|
|
686
|
+
```
|
|
687
|
+
|
|
688
|
+
If the application called `disable_instrumentation()`, the hook keeps that
|
|
689
|
+
choice and says so. With sensitive data off, a failed run's `error` is the
|
|
690
|
+
exception type: that's all Agent Framework records then.
|
|
691
|
+
|
|
692
|
+
A complete, runnable version is in **`demo/demo_microsoft.py`** (fully offline —
|
|
693
|
+
it drives the agent with a tiny local echo chat client, so no Azure/OpenAI key is
|
|
694
|
+
needed; it exercises both the non-streaming and streaming run paths):
|
|
695
|
+
|
|
696
|
+
```bash
|
|
697
|
+
uv pip install agent-framework "otelzero[otel]"
|
|
698
|
+
uv run python demo/demo_microsoft.py
|
|
699
|
+
```
|
|
700
|
+
|
|
701
|
+
### Instrument a real Snowflake Cortex Agent SDK agent
|
|
702
|
+
|
|
703
|
+
Snowflake's [Cortex Code Agent SDK](https://docs.snowflake.com/en/user-guide/cortex-code-agent-sdk/cortex-code-agent-sdk)
|
|
704
|
+
(the `cortex_code_agent_sdk` package) is shaped almost identically to the Claude
|
|
705
|
+
Agent SDK: a module-level `query(prompt, options=...)` async-generator function
|
|
706
|
+
and an interactive `CortexCodeSDKClient` whose `receive_response()` is an async
|
|
707
|
+
generator, both configured by a `CortexCodeAgentOptions` dataclass.
|
|
708
|
+
`install_snowflake_monitoring` wraps **both** run chokepoints so a run is captured
|
|
709
|
+
exactly once whichever style you use, and `CortexCodeSDKClient.query()` so the
|
|
710
|
+
prompt it sends is recorded on the `receive_response()` that answers it:
|
|
711
|
+
|
|
712
|
+
```python
|
|
713
|
+
from otelzero import install_snowflake_monitoring
|
|
714
|
+
|
|
715
|
+
# 1) Install the hook FIRST (registers a find_spec meta-path finder)
|
|
716
|
+
install_snowflake_monitoring(server_url="http://localhost:8000", user="Sarah")
|
|
717
|
+
|
|
718
|
+
# 2) Now import the SDK — query and CortexCodeSDKClient are auto-instrumented
|
|
719
|
+
from cortex_code_agent_sdk import query, CortexCodeAgentOptions
|
|
720
|
+
|
|
721
|
+
options = CortexCodeAgentOptions(model="claude-sonnet-4-6", cwd=".")
|
|
722
|
+
|
|
723
|
+
# Each query() run streams agent_start/agent_end/agent_error events
|
|
724
|
+
async for message in query(prompt="Summarize this codebase", options=options):
|
|
725
|
+
...
|
|
726
|
+
```
|
|
727
|
+
|
|
728
|
+
A complete, runnable version is in **`demo/demo_snowflake.py`**:
|
|
729
|
+
|
|
730
|
+
```bash
|
|
731
|
+
uv pip install cortex-code-agent-sdk # the hook instruments the real package
|
|
732
|
+
# authenticate the Snowflake CLI connection the SDK uses (e.g. `snow connection add ...`)
|
|
733
|
+
uv run python demo/demo_snowflake.py
|
|
734
|
+
```
|
|
735
|
+
|
|
736
|
+
The `agent_start` details carry the run's `model`, `allowed_tools`, and (names
|
|
737
|
+
only) `mcp_servers`. With tracing on (the default), the SDK's own spans nest
|
|
738
|
+
underneath otelzero's `invoke_agent Snowflake Cortex Agent` span.
|
|
739
|
+
|
|
740
|
+
> Order matters: instrumentation must be installed **before** `cortex_code_agent_sdk`
|
|
741
|
+
> is imported. The hooks use the `find_spec` protocol and require **Python ≥ 3.12**.
|
|
742
|
+
|
|
743
|
+
|
|
744
|
+
## OpenTelemetry Tracing
|
|
745
|
+
|
|
746
|
+
otelzero can emit **standard OpenTelemetry traces** alongside (or instead of) the
|
|
747
|
+
built-in telemetry. The same import hooks that stream dashboard events also open
|
|
748
|
+
spans, so a crew run becomes a real distributed trace:
|
|
749
|
+
|
|
750
|
+
- `Crew.kickoff` → an `invoke_workflow` span, with each `Agent.execute_task` and
|
|
751
|
+
`Task.execute_sync` as **nested child spans** (real parent/child via context).
|
|
752
|
+
- An error sets the span status to `ERROR`, records the exception, and sets
|
|
753
|
+
`error.type`.
|
|
754
|
+
- Attributes follow the [GenAI semantic
|
|
755
|
+
conventions](https://github.com/open-telemetry/semantic-conventions-genai) —
|
|
756
|
+
see [OTel span attributes](#otel-span-attributes).
|
|
757
|
+
|
|
758
|
+
It's **on by default** and dependency-light: install the `otel` extra and the
|
|
759
|
+
spans flow (`OTELZERO_OTEL` is effectively always `1`). Without the extra, the
|
|
760
|
+
client runs exactly as before, with zero OpenTelemetry deps. `OTELZERO_OTEL=0`
|
|
761
|
+
switches the import hooks' spans off; the native-OTel hooks (Pydantic AI, Strands,
|
|
762
|
+
ADK, Microsoft, LangChain / LangGraph) keep tracing, because spans are how they
|
|
763
|
+
report.
|
|
764
|
+
|
|
765
|
+
```bash
|
|
766
|
+
uv sync --extra otel # opentelemetry-api / sdk / exporter-otlp
|
|
767
|
+
uv run python demo/demo_crewai.py
|
|
768
|
+
```
|
|
769
|
+
|
|
770
|
+
By default spans are exported to **the registry's own OTLP receiver** at `POST /v1/traces`,
|
|
771
|
+
which translates them back into the dashboard's logs and stats — so the existing
|
|
772
|
+
UI keeps working. To send traces to a standard backend instead (Jaeger, Tempo,
|
|
773
|
+
Grafana, an OpenTelemetry Collector), point the exporter elsewhere with the
|
|
774
|
+
standard env var:
|
|
775
|
+
|
|
776
|
+
```bash
|
|
777
|
+
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
|
|
778
|
+
uv run python demo/demo_crewai.py
|
|
779
|
+
```
|
|
780
|
+
|
|
781
|
+
> **Dual-emit note:** with tracing on (the default), each run reaches Kiji's own
|
|
782
|
+
> server both as events (`/api/telemetry`) **and** as spans (`/v1/traces`). If
|
|
783
|
+
> your registry counts both, point `OTEL_EXPORTER_OTLP_ENDPOINT` at a separate
|
|
784
|
+
> collector, or set `OTELZERO_OTEL=0` for the import-hook frameworks.
|
|
785
|
+
|
|
786
|
+
OpenTelemetry tracing requires **Python ≥ 3.12** (as do the import hooks).
|
|
787
|
+
|
|
788
|
+
### How LangSmith's spans are translated
|
|
789
|
+
|
|
790
|
+
`install_langchain_monitoring` / `install_langgraph_monitoring` switch on
|
|
791
|
+
LangChain's own OpenTelemetry connector (the one that ships with the `langsmith`
|
|
792
|
+
SDK; see [the LangGraph guide](#instrument-a-real-langgraph-graph-and-langchain-10-agents)).
|
|
793
|
+
|
|
794
|
+
The connector's spans carry a `langsmith.span.kind` run type, but they speak an
|
|
795
|
+
**older dialect** of the GenAI conventions: `gen_ai.operation.name` holds the
|
|
796
|
+
run type (`chain`, `parser`) and labels prompt-template steps `chat`, the
|
|
797
|
+
deprecated `gen_ai.system` holds `langchain`, and the prompt and response leave
|
|
798
|
+
the process as `gen_ai.prompt` / `gen_ai.completion` whether or not you opted
|
|
799
|
+
in. The one piece of otelzero in the pipeline fixes that: wrap the OTLP exporter
|
|
800
|
+
in `otelzero.translate.TranslatingExporter` and every span is rewritten into the
|
|
801
|
+
current conventions on its way out —
|
|
802
|
+
|
|
803
|
+
| LangChain's connector emits | Translated to |
|
|
804
|
+
|---|---|
|
|
805
|
+
| root `chain` run (`langsmith.trace.name` = `run_name`) | `invoke_agent {run_name}`, `gen_ai.agent.name` |
|
|
806
|
+
| `langsmith.span.kind = llm` | `chat {model}`, `gen_ai.provider.name` from `ls_provider`, `gen_ai.request.model` |
|
|
807
|
+
| `langsmith.span.kind = tool` | `execute_tool {tool}`, `gen_ai.tool.name` |
|
|
808
|
+
| nested `chain` / `prompt` / `parser` steps | no `gen_ai.operation.name` — they are not GenAI operations |
|
|
809
|
+
| `gen_ai.system`, `gen_ai.serialized.*`, `LANGSMITH_*` metadata | dropped |
|
|
810
|
+
| `gen_ai.prompt` / `gen_ai.completion` | `gen_ai.input.messages` / `gen_ai.output.messages` / `gen_ai.system_instructions` — **only** with `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=1`; dropped otherwise |
|
|
811
|
+
| `gen_ai.usage.total_tokens` | `otelzero.usage.total_tokens` (input/output tokens keep their names) |
|
|
812
|
+
| run metadata `conversation_id` / `thread_id`, `user_name` / `user_id` | `gen_ai.conversation.id`, `user.name` |
|
|
813
|
+
| status `OK` on success | left unset, as the conventions require; `error.type` filled in on failure |
|
|
814
|
+
|
|
815
|
+
The mapping lives in `otelzero.translate.langsmith.translate_attributes`, a pure
|
|
816
|
+
function over plain attribute dicts, so an OTLP backend can run the identical
|
|
817
|
+
mapping on ingest. `otel.init_tracing` installs the exporter wrapper on
|
|
818
|
+
otelzero's own provider too, so an otelzero user who also turns the connector on
|
|
819
|
+
gets translated spans without further setup.
|
|
820
|
+
|
|
821
|
+
Because the connector builds its spans on LangSmith's background tracing
|
|
822
|
+
thread, a `run_context` is not visible to them. A LangChain run names its user
|
|
823
|
+
and conversation through its config instead (LangGraph's
|
|
824
|
+
`config={"configurable": {"thread_id": ...}}` works too):
|
|
825
|
+
|
|
826
|
+
```python
|
|
827
|
+
chain.invoke(
|
|
828
|
+
{"input": query},
|
|
829
|
+
config={"run_name": "SQL Agent",
|
|
830
|
+
"metadata": {"user_name": "Sarah", "conversation_id": thread_id}},
|
|
831
|
+
)
|
|
832
|
+
```
|
|
833
|
+
|
|
834
|
+
Kiji maps the translated spans straight onto the dashboard:
|
|
835
|
+
|
|
836
|
+
- each **`invoke_agent`** run → an **agent card** (tag the run with `run_name`
|
|
837
|
+
to label it, e.g. `ReAct Agent` instead of `RunnableSequence`);
|
|
838
|
+
- nested **`chat` / `execute_tool`** spans → **task** rows in the live feed,
|
|
839
|
+
labeled by span name — without spawning their own cards;
|
|
840
|
+
- a failing run sets the span status to `ERROR` → a failed run on the card.
|
|
841
|
+
|
|
842
|
+
### Ingest Microsoft Copilot Studio (SaaS) by relaying its OTLP telemetry
|
|
843
|
+
|
|
844
|
+
[Copilot Studio](https://learn.microsoft.com/en-us/microsoft-copilot-studio/) is a
|
|
845
|
+
hosted, low-code product — there is **no Python process to import or method to
|
|
846
|
+
wrap**, so the import-hook path does not apply. Instead, Copilot Studio agents
|
|
847
|
+
export telemetry to **Azure Monitor / Application Insights** following the OTel
|
|
848
|
+
**GenAI semantic conventions** — the *same* `invoke_agent` / `chat` /
|
|
849
|
+
`execute_tool` spans Kiji's `/v1/traces` receiver already maps. So ingesting
|
|
850
|
+
Copilot Studio is a matter of **relaying** those spans to the registry's OTLP endpoint —
|
|
851
|
+
no new otelzero code:
|
|
852
|
+
|
|
853
|
+
```
|
|
854
|
+
┌──────────────────┐ GenAI/OTel ┌───────────────┐ OTLP ┌───────────────┐
|
|
855
|
+
│ Copilot Studio │─────────────→│ OTel │──────────→│ Kiji │
|
|
856
|
+
│ (Azure Monitor) │ telemetry │ Collector │ /v1/traces│ dashboard │
|
|
857
|
+
└──────────────────┘ └───────────────┘ └───────────────┘
|
|
858
|
+
```
|
|
859
|
+
|
|
860
|
+
The final Collector hop is a small, reusable config —
|
|
861
|
+
[`demo/otelcol-copilot-studio.yaml`](demo/otelcol-copilot-studio.yaml) — an OTLP
|
|
862
|
+
receiver → `otlphttp` exporter to `…/v1/traces` with the ingest key, stamping
|
|
863
|
+
`service.name: "Copilot Studio"` so runs group under one framework label:
|
|
864
|
+
|
|
865
|
+
```bash
|
|
866
|
+
OTELZERO_API_KEY=<ingest-key> \
|
|
867
|
+
otelcol-contrib --config demo/otelcol-copilot-studio.yaml
|
|
868
|
+
```
|
|
869
|
+
|
|
870
|
+
**How spans reach the Collector** depends on your Azure setup:
|
|
871
|
+
|
|
872
|
+
- **Pro-code / Foundry-backed agents** that let you set an OTLP endpoint: point
|
|
873
|
+
`OTEL_EXPORTER_OTLP_ENDPOINT` at the Collector — spans arrive directly, no Azure
|
|
874
|
+
round-trip.
|
|
875
|
+
- **Pure Copilot Studio SaaS agents** export only to Application Insights. Relay
|
|
876
|
+
those by fanning App Insights out via a **diagnostic-settings export to Event
|
|
877
|
+
Hub** (or a scheduled **Log Analytics / KQL** query) and a small converter that
|
|
878
|
+
re-emits the GenAI records as OTLP into the Collector. (The Collector has an
|
|
879
|
+
Azure Monitor *exporter*, not a receiver, so the converter step is required for
|
|
880
|
+
the SaaS-only path.)
|
|
881
|
+
|
|
882
|
+
To validate the **Kiji side end to end without an Azure tenant**,
|
|
883
|
+
`demo/demo_copilot_studio_otel.py` synthesizes the exact OTLP spans a Copilot
|
|
884
|
+
Studio agent produces and posts them to `/v1/traces` over real OTLP/HTTP — just
|
|
885
|
+
as the relaying Collector would:
|
|
886
|
+
|
|
887
|
+
```bash
|
|
888
|
+
(cd ../kiji-agent-registry && uv run kiji-agent-registry) # terminal 1
|
|
889
|
+
uv run python demo/demo_copilot_studio_otel.py # terminal 2 (no Azure needed)
|
|
890
|
+
```
|
|
891
|
+
|
|
892
|
+
Each `invoke_agent` run becomes an agent card; nested `chat` / `execute_tool`
|
|
893
|
+
spans become labeled task rows; an errored tool propagates to a failed run —
|
|
894
|
+
identical rendering to the framework integrations above.
|
|
895
|
+
|
|
896
|
+
|
|
897
|
+
## How It Works
|
|
898
|
+
|
|
899
|
+
### Import Hook Magic
|
|
900
|
+
|
|
901
|
+
The instrumentation uses Python's import system to intercept module loading.
|
|
902
|
+
Python 3.12 removed the legacy `find_module` protocol, so the hook implements
|
|
903
|
+
**`find_spec`** and wraps the real loader to instrument the module after it
|
|
904
|
+
executes:
|
|
905
|
+
|
|
906
|
+
```python
|
|
907
|
+
class CrewAIImportHook(importlib.abc.MetaPathFinder):
|
|
908
|
+
def find_spec(self, fullname, path=None, target=None):
|
|
909
|
+
if fullname not in ("crewai.crew", "crewai.agent", "crewai.task"):
|
|
910
|
+
return None
|
|
911
|
+
# Resolve the real spec from the rest of sys.meta_path (guarded against
|
|
912
|
+
# re-entrancy), then swap in a loader that instruments after exec.
|
|
913
|
+
spec = importlib.util.find_spec(fullname)
|
|
914
|
+
spec.loader = _InstrumentingLoader(spec.loader, self, fullname)
|
|
915
|
+
return spec
|
|
916
|
+
|
|
917
|
+
|
|
918
|
+
class _InstrumentingLoader(importlib.abc.Loader):
|
|
919
|
+
def exec_module(self, module):
|
|
920
|
+
self._real_loader.exec_module(module) # load CrewAI normally
|
|
921
|
+
self._hook._instrument_module(module) # then wrap Crew/Agent/Task
|
|
922
|
+
```
|
|
923
|
+
|
|
924
|
+
### Function Wrapping
|
|
925
|
+
|
|
926
|
+
Uses `wrapt` library for clean function wrapping:
|
|
927
|
+
|
|
928
|
+
```python
|
|
929
|
+
@wrapt.decorator
|
|
930
|
+
def agent_wrapper(wrapped, instance, args, kwargs):
|
|
931
|
+
# Before execution
|
|
932
|
+
send_log("agent_start", agent_role=instance.role)
|
|
933
|
+
|
|
934
|
+
try:
|
|
935
|
+
result = wrapped(*args, **kwargs)
|
|
936
|
+
# On success
|
|
937
|
+
send_log("agent_end", agent_role=instance.role)
|
|
938
|
+
return result
|
|
939
|
+
except Exception as e:
|
|
940
|
+
# On error
|
|
941
|
+
send_log("agent_error", error=str(e), traceback=...)
|
|
942
|
+
raise
|
|
943
|
+
```
|
|
944
|
+
|
|
945
|
+
### Async Telemetry
|
|
946
|
+
|
|
947
|
+
Logs are sent asynchronously to avoid blocking agent execution:
|
|
948
|
+
|
|
949
|
+
```python
|
|
950
|
+
def send_log(self, event_type, **kwargs):
|
|
951
|
+
log_data = {"event_type": event_type, ...}
|
|
952
|
+
# Send in background thread
|
|
953
|
+
threading.Thread(
|
|
954
|
+
target=self._send_async,
|
|
955
|
+
args=(log_data,),
|
|
956
|
+
daemon=True
|
|
957
|
+
).start()
|
|
958
|
+
```
|
|
959
|
+
|
|
960
|
+
|
|
961
|
+
## Event Types
|
|
962
|
+
|
|
963
|
+
- `crew_start` - Crew execution begins
|
|
964
|
+
- `crew_end` - Crew execution completes
|
|
965
|
+
- `crew_error` - Crew execution fails
|
|
966
|
+
- `agent_start` - Agent begins task execution
|
|
967
|
+
- `agent_end` - Agent completes task
|
|
968
|
+
- `agent_error` - Agent encounters error
|
|
969
|
+
- `task_start` - Task execution begins
|
|
970
|
+
- `task_end` - Task execution completes
|
|
971
|
+
- `task_error` - Task execution fails
|
|
972
|
+
|
|
973
|
+
### Common telemetry envelope
|
|
974
|
+
|
|
975
|
+
Every event (`send_log`) carries the same top-level fields regardless of
|
|
976
|
+
framework: `event_type`, `timestamp`, `agent_name`, `agent_role`, `framework`,
|
|
977
|
+
`framework_version`, `model`, `user`, `details`, `error`, `traceback` — plus a
|
|
978
|
+
parallel OpenTelemetry span (tracing is on by default).
|
|
979
|
+
|
|
980
|
+
### Per-framework instrumentation
|
|
981
|
+
|
|
982
|
+
`service.name` carries the framework label; the span names below follow the
|
|
983
|
+
GenAI conventions' `{operation} {subject}` templates.
|
|
984
|
+
|
|
985
|
+
`service.name` is the framework label the client is created with, and it is also
|
|
986
|
+
the value the dashboard groups and filters runs by. Set `OTEL_SERVICE_NAME`, or
|
|
987
|
+
`service.name` in `OTEL_RESOURCE_ATTRIBUTES`, to replace it.
|
|
988
|
+
|
|
989
|
+
| Framework | `service.name` | Wrapped method(s) | OTel span(s) | Lifecycle events |
|
|
990
|
+
|---|---|---|---|---|
|
|
991
|
+
| **CrewAI** | `CrewAI` | `Crew.kickoff`, `Agent.execute_task`, `Task.execute_sync` | `invoke_workflow {crew}`, `invoke_agent {role}`, `execute_task {role}` | crew + agent + task |
|
|
992
|
+
| **LangChain** | `LangChain` | — (LangSmith's native OTel export, translated) | `invoke_agent {run}` → `chat {model}`, `execute_tool {tool}`, plus one span per nested chain / prompt / parser step (LangSmith's own) | agent |
|
|
993
|
+
| **LangGraph** | `LangGraph` | — (LangSmith's native OTel export, translated) | `invoke_agent {graph}` → `{node}` → `chat {model}`, `execute_tool {tool}` (LangSmith's own) | agent |
|
|
994
|
+
| **LlamaIndex** | `LlamaIndex` | `Workflow.run` (every agent, and every workflow) | `invoke_agent {name}` | agent |
|
|
995
|
+
| **OpenAI Agents** | `OpenAI Agents` | `Runner.run`, `run_sync`, `run_streamed` | `invoke_agent {name}` | agent |
|
|
996
|
+
| **Pydantic AI** | `Pydantic AI` | — (native OTel via `Agent.instrument_all`) | `invoke_agent {name}` → `chat`, `execute_tool` (Pydantic AI's own) | agent |
|
|
997
|
+
| **smolagents** | `smolagents` | `MultiStepAgent.run` (direct + stream) | `invoke_agent {name}` | agent |
|
|
998
|
+
| **AWS Strands** | `Strands Agents` | — (native OTel, latest GenAI conventions) | `invoke_agent {name}` → `execute_event_loop_cycle`, `chat`, `execute_tool` (Strands' own) | agent |
|
|
999
|
+
| **Google ADK** | `Google ADK` | — (native OTel, latest GenAI conventions) | `invoke_workflow` → `invoke_agent {name}` → `generate_content`, `execute_tool` (ADK's own) | agent |
|
|
1000
|
+
| **Databricks** | `Databricks Agents` | `ResponsesAgent.predict`, `ChatAgent.predict` | `invoke_agent {name}`, `execute_tool {tool}` | agent + task |
|
|
1001
|
+
| **Claude Agent SDK** | `Claude Agent SDK` | `query`, `ClaudeSDKClient.receive_response` | `invoke_agent Claude Agent` | agent |
|
|
1002
|
+
| **Microsoft** | `Microsoft Agent Framework` | — (native OTel via `enable_instrumentation`) | `invoke_agent {name}` → `chat`, `execute_tool` (Agent Framework's own) | agent |
|
|
1003
|
+
| **Snowflake Cortex** | `Snowflake Cortex` | `query`, `CortexCodeSDKClient.receive_response` | `invoke_agent Snowflake Cortex Agent` | agent |
|
|
1004
|
+
|
|
1005
|
+
### Event matrix — which lifecycle events each framework emits
|
|
1006
|
+
|
|
1007
|
+
| Framework | `crew_*` | `agent_*` | `task_*` |
|
|
1008
|
+
|---|:--:|:--:|:--:|
|
|
1009
|
+
| CrewAI | ✅ | ✅ | ✅ |
|
|
1010
|
+
| LangChain | — | ✅⁷ | — |
|
|
1011
|
+
| LangGraph | — | ✅⁷ | — |
|
|
1012
|
+
| LlamaIndex | — | ✅ | —⁴ |
|
|
1013
|
+
| OpenAI Agents | — | ✅ | — |
|
|
1014
|
+
| Pydantic AI | — | ✅⁵ | — |
|
|
1015
|
+
| smolagents | — | ✅ | — |
|
|
1016
|
+
| Strands | — | ✅⁵ | — |
|
|
1017
|
+
| Google ADK | — | ✅⁵ | — |
|
|
1018
|
+
| Databricks | — | ✅ | ✅² |
|
|
1019
|
+
| Claude Agent SDK | — | ✅ | — |
|
|
1020
|
+
| Microsoft | — | ✅⁵ | —³ |
|
|
1021
|
+
| Snowflake Cortex | — | ✅ | — |
|
|
1022
|
+
|
|
1023
|
+
² Databricks maps each `ResponsesAgent` tool call (a `function_call` item in the
|
|
1024
|
+
response) to a `task_*` pair, nested under the agent run.
|
|
1025
|
+
³ Microsoft's nested model/tool calls surface as `task_*` rows when its
|
|
1026
|
+
**native OTel** spans are ingested (`invoke_agent` → agent, `chat`/`execute_tool`
|
|
1027
|
+
→ tasks), which `install_microsoft_monitoring` sets up.
|
|
1028
|
+
⁴ Likewise for LlamaIndex: the hook captures the agent run, while its nested
|
|
1029
|
+
LLM/tool/retriever steps surface as `task_*` rows when ingested via
|
|
1030
|
+
**OpenInference** (`openinference-instrumentation-llama-index`; root
|
|
1031
|
+
`CHAIN`/`AGENT` → agent, everything nested → tasks), which Kiji's receiver also
|
|
1032
|
+
ingests.
|
|
1033
|
+
⁷ LangChain / LangGraph `agent_*` events come from LangChain's callbacks, one
|
|
1034
|
+
pair per top-level run; every nested chain, node, model and tool call is a span
|
|
1035
|
+
from LangSmith's export, not an event.
|
|
1036
|
+
⁵ The native-OTel frameworks' (Pydantic AI, Strands, ADK, Microsoft) `agent_*`
|
|
1037
|
+
events come from their own `invoke_agent` spans, not from a wrapper; their model
|
|
1038
|
+
and tool spans reach the registry over OTLP only.
|
|
1039
|
+
|
|
1040
|
+
### Details captured per wrapped call
|
|
1041
|
+
|
|
1042
|
+
| Framework · method | `*_start` `details` | `*_end` `details` | `*_error` |
|
|
1043
|
+
|---|---|---|---|
|
|
1044
|
+
| CrewAI · `Crew.kickoff` | `num_agents`, `num_tasks` | `status: completed` | `error`, `traceback` |
|
|
1045
|
+
| CrewAI · `Agent.execute_task` | `role`, `goal` (≤100 chars) | `status: success` | `error`, `traceback` |
|
|
1046
|
+
| CrewAI · `Task.execute_sync` | `description`, `expected_output` (≤100) | `status: completed` | `error`, `traceback` |
|
|
1047
|
+
| LangChain / LangGraph · top-level run (LangChain callbacks) | `model` when the run is a model call | `status: success`, `model`, `tools` (called), `usage` | `error`, `traceback` |
|
|
1048
|
+
| LlamaIndex · `Workflow.run` | `model`, `tools` (+ `agents` for an `AgentWorkflow`) | `status: success` | `error`, `traceback` |
|
|
1049
|
+
| OpenAI Agents · `Runner.run` / `run_sync` / `run_streamed` | `model`, `tools`, `handoffs` | `status: success` | `error`, `traceback` |
|
|
1050
|
+
| Pydantic AI · native `invoke_agent` span | `model` | `status: success`, `tools`, `usage` | `error`, `traceback`⁶ |
|
|
1051
|
+
| smolagents · `MultiStepAgent.run` | `model`, `tools` | `status: success` | `error`, `traceback` |
|
|
1052
|
+
| Strands · native `invoke_agent` span | `model` (none: set after start) | `status: success`, `model`, `tools`, `usage` | `error`, `traceback` |
|
|
1053
|
+
| Google ADK · native `invoke_agent` span | `model` (none: set after start) | `status: success`, `model`, `tools`, `usage` (summed over its `generate_content` spans) | `error`, `traceback` |
|
|
1054
|
+
| Databricks · `ResponsesAgent.predict` / `ChatAgent.predict` | `num_messages` | `status: success` | `error`, `traceback` |
|
|
1055
|
+
| Databricks · tool call (`ResponsesAgent`) | `tool`, `call_id`, `arguments` (≤200) | `status: success`, `output` (≤200) | `error`, `traceback` |
|
|
1056
|
+
| Claude Agent SDK · `query` / `ClaudeSDKClient.receive_response` | `model`, `allowed_tools`, `mcp_servers` (names) | `status: success` | `error`, `traceback` |
|
|
1057
|
+
| Microsoft · native `invoke_agent` span | `model` (none: set after start) | `status: success`, `model` (from its `chat` spans), `tools`, `usage` | `error`, `traceback`⁶ |
|
|
1058
|
+
| Snowflake Cortex · `query` / `CortexCodeSDKClient.receive_response` | `model`, `allowed_tools`, `mcp_servers` (names) | `status: success` | `error`, `traceback` |
|
|
1059
|
+
|
|
1060
|
+
⁶ Pydantic AI and Agent Framework record the exception message and traceback
|
|
1061
|
+
only when content capture is on; otherwise `error` is the exception type and
|
|
1062
|
+
`traceback` is empty.
|
|
1063
|
+
|
|
1064
|
+
**Prompt and response (opt-in).** With
|
|
1065
|
+
`OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=1` set on the agent, every
|
|
1066
|
+
`*_end` / `*_error` row above also carries `input_messages` (the run's prompt)
|
|
1067
|
+
and `output_messages` (its response), plus `system_instructions` where the hook
|
|
1068
|
+
can see a system prompt. Both are arrays in the GenAI conventions' message shape
|
|
1069
|
+
(`{"role", "parts": [{"type": "text", "content"} | reasoning | tool_call | tool_call_response]}`);
|
|
1070
|
+
the model's stop reason sits beside them as `finish_reasons`, not inside a
|
|
1071
|
+
message — the same content that goes
|
|
1072
|
+
out as the `gen_ai.input.messages` / `gen_ai.output.messages` span attributes,
|
|
1073
|
+
so the dashboard, the apps and an OTLP backend all see one record. Off by
|
|
1074
|
+
default: no prompt or response text leaves the process unless you opt in. Size
|
|
1075
|
+
is bounded on the client: `OTELZERO_CONTENT_MAX_CHARS` (default `4000`, `0`
|
|
1076
|
+
disables) caps each text / arguments / response part with a
|
|
1077
|
+
`…[truncated, N more chars]` marker, and `OTELZERO_CONTENT_MAX_MESSAGES`
|
|
1078
|
+
(default `20`, `0` disables) keeps only the most recent input messages of a
|
|
1079
|
+
replayed chat history.
|
|
1080
|
+
|
|
1081
|
+
|
|
1082
|
+
### OTel span attributes
|
|
1083
|
+
|
|
1084
|
+
otelzero follows the [OpenTelemetry GenAI semantic
|
|
1085
|
+
conventions](https://github.com/open-telemetry/semantic-conventions-genai)
|
|
1086
|
+
(status: development). The vocabulary has a single definition in
|
|
1087
|
+
`src/otelzero/semconv.py`, mirrored in the registry's
|
|
1088
|
+
`kiji_agent_registry/semconv.py` (a registry test checks the two agree).
|
|
1089
|
+
|
|
1090
|
+
**On every span:**
|
|
1091
|
+
|
|
1092
|
+
| Attribute | Value |
|
|
1093
|
+
|---|---|
|
|
1094
|
+
| `gen_ai.operation.name` | `invoke_workflow`, `invoke_agent`, `execute_tool`, or `execute_task` |
|
|
1095
|
+
| `gen_ai.conversation.id` | the run's flow id — groups a conversation's events |
|
|
1096
|
+
| `user.name` | who the run is for |
|
|
1097
|
+
| `otelzero.session.name` | the session's title, once it has one (`name=` or derived from the run's intent) |
|
|
1098
|
+
| `error.type` | the exception class name, on failed spans only |
|
|
1099
|
+
|
|
1100
|
+
A successful span carries **no status code**: the conventions require it to be
|
|
1101
|
+
left unset, so only failures set `ERROR` (plus `error.type` and the exception
|
|
1102
|
+
message).
|
|
1103
|
+
|
|
1104
|
+
**Every otelzero span is `SpanKind.INTERNAL`**, and the conventions define a smaller
|
|
1105
|
+
attribute set for the *internal* `invoke_agent` span than for the `client` one.
|
|
1106
|
+
Two attributes you might expect are therefore absent by design:
|
|
1107
|
+
|
|
1108
|
+
- **`gen_ai.provider.name`.** It replaced the deprecated `gen_ai.system`, but it
|
|
1109
|
+
names the *inference provider* — which an orchestrator like CrewAI is not —
|
|
1110
|
+
and it is listed only on the `create_agent` and `invoke_agent` client spans.
|
|
1111
|
+
The **framework** (`crewai`, `langchain`, …) rides on the `service.name`
|
|
1112
|
+
**resource** attribute instead.
|
|
1113
|
+
- **`gen_ai.agent.id`.** Also client-span-only, and the spec calls recording
|
|
1114
|
+
in-memory agent instance ids on it NOT RECOMMENDED "due to their transient
|
|
1115
|
+
nature" — which is all Kiji's hooks can see.
|
|
1116
|
+
|
|
1117
|
+
This applies to the spans otelzero emits itself. The native-OTel frameworks
|
|
1118
|
+
(Pydantic AI, Strands, ADK, Microsoft) emit their own spans, which carry
|
|
1119
|
+
whatever the framework records: Strands and Agent Framework set
|
|
1120
|
+
`gen_ai.provider.name` (`strands-agents`, `microsoft.agent_framework`), and
|
|
1121
|
+
Agent Framework sets `gen_ai.agent.id`.
|
|
1122
|
+
|
|
1123
|
+
Kiji's ingest still *reads* both from third-party traces that legitimately set
|
|
1124
|
+
them.
|
|
1125
|
+
|
|
1126
|
+
**Token usage**, on the span that closes a run:
|
|
1127
|
+
|
|
1128
|
+
| Attribute | Meaning |
|
|
1129
|
+
|---|---|
|
|
1130
|
+
| `gen_ai.usage.input_tokens` / `gen_ai.usage.output_tokens` | prompt / completion tokens |
|
|
1131
|
+
| `gen_ai.usage.reasoning.output_tokens` | reasoning tokens, when the provider reports them |
|
|
1132
|
+
| `gen_ai.usage.cache_read.input_tokens` | tokens served from a provider cache |
|
|
1133
|
+
| `otelzero.usage.total_tokens` | otelzero's own — the conventions record input and output separately and define no total |
|
|
1134
|
+
|
|
1135
|
+
**Finish reasons**, on the span that closes a run, whether or not content
|
|
1136
|
+
capture is on: `gen_ai.response.finish_reasons`, the conventions' array of
|
|
1137
|
+
normalised stop reasons (`stop`, `length`, `tool_call`, `content_filter`, …),
|
|
1138
|
+
one per generation the framework reports — every turn of a streamed run, the
|
|
1139
|
+
final one of a one-shot result.
|
|
1140
|
+
It replaces the per-message `finish_reason` field the conventions deprecated,
|
|
1141
|
+
and rides on the log row as `finish_reasons`.
|
|
1142
|
+
|
|
1143
|
+
**Content (Opt-In)**, on the span that closes a run, only when
|
|
1144
|
+
`OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=1`:
|
|
1145
|
+
|
|
1146
|
+
| Attribute | Meaning |
|
|
1147
|
+
|---|---|
|
|
1148
|
+
| `gen_ai.input.messages` | the run's prompt, as a JSON array of `{"role", "parts"}` messages |
|
|
1149
|
+
| `gen_ai.output.messages` | the run's response, same shape (one message per turn) |
|
|
1150
|
+
| `gen_ai.system_instructions` | the system prompt as a JSON array of parts (CrewAI agents: the backstory) |
|
|
1151
|
+
|
|
1152
|
+
What counts as the prompt and the response per framework:
|
|
1153
|
+
|
|
1154
|
+
| Framework | Prompt (`gen_ai.input.messages`) | Response (`gen_ai.output.messages`) |
|
|
1155
|
+
|---|---|---|
|
|
1156
|
+
| CrewAI | `kickoff(inputs=)` as JSON; a task's `description` (+ `context`); an agent's executor history up to its prompt (its system prompt as `system_instructions`) | `CrewOutput.raw` / `TaskOutput.raw`; an agent's whole exchange (tool calls, results, answer) from its executor history |
|
|
1157
|
+
| LangChain / LangGraph | exported by LangSmith itself as `gen_ai.prompt` (hidden — `{}` — unless content capture is on) | likewise `gen_ai.completion` |
|
|
1158
|
+
| Strands | recorded by Strands itself (latest conventions, `gen_ai_span_attributes_only`) | likewise |
|
|
1159
|
+
| Google ADK | recorded by ADK itself on its `generate_content` spans (with `SPAN_ONLY` / `SPAN_AND_EVENT`) | likewise |
|
|
1160
|
+
| Databricks | `ChatAgent` messages, or `ResponsesAgent` `request.input` items (`message`, `function_call`, `function_call_output`) | `response.messages` or `response.output` items |
|
|
1161
|
+
| Claude Agent SDK | the `query(prompt)` string, or a streamed prompt's messages (teed as the SDK reads them); for the client, what `client.query()` sent, one prompt per `receive_response()` | every streamed `AssistantMessage`/`UserMessage` block (text, tool use, tool result), else `ResultMessage.result`; `stop_reason` normalised |
|
|
1162
|
+
| Microsoft | recorded by Agent Framework itself on its `invoke_agent` / `chat` spans (sensitive data on) | likewise |
|
|
1163
|
+
| Snowflake Cortex | as Claude Agent SDK | as Claude Agent SDK |
|
|
1164
|
+
|
|
1165
|
+
Tool calls inside a message map to `{"type": "tool_call", "id", "name", "arguments"}`
|
|
1166
|
+
and their results to `{"type": "tool_call_response", "id", "response"}`, filed
|
|
1167
|
+
under the `tool` role; Anthropic `tool_use`/`tool_result`, OpenAI
|
|
1168
|
+
`function_call`/`function_call_output`, LangChain `tool_calls` and Gemini
|
|
1169
|
+
`function_call`/`function_response` blocks are all recognised. Content is
|
|
1170
|
+
capped on the client (`OTELZERO_CONTENT_MAX_CHARS`, `OTELZERO_CONTENT_MAX_MESSAGES`,
|
|
1171
|
+
see [Details captured per wrapped call](#details-captured-per-wrapped-call)).
|
|
1172
|
+
The registry's ingest stores these attributes as the same `input_messages` /
|
|
1173
|
+
`output_messages` details the HTTP path records, and its export writes them back
|
|
1174
|
+
as `gen_ai.*` when the server process has opted in — otherwise as
|
|
1175
|
+
`otelzero.input_messages` / `otelzero.output_messages` (still JSON), which
|
|
1176
|
+
ingest also reads, so a downloaded trace re-ingests losslessly either way.
|
|
1177
|
+
|
|
1178
|
+
**Per span, beyond the common set** (every span below also carries the Opt-In
|
|
1179
|
+
content attributes † when captured):
|
|
1180
|
+
|
|
1181
|
+
| Span | Operation | Extra attributes |
|
|
1182
|
+
|---|---|---|
|
|
1183
|
+
| `invoke_workflow {crew}` (CrewAI) | `invoke_workflow` | `gen_ai.workflow.name`, `gen_ai.request.model`, `otelzero.num_agents`, `otelzero.num_tasks` |
|
|
1184
|
+
| `invoke_agent {role}` (CrewAI) | `invoke_agent` | `gen_ai.agent.name` (role), `gen_ai.agent.description` (goal), `gen_ai.request.model` |
|
|
1185
|
+
| `execute_task {role}` (CrewAI) | `execute_task` | `gen_ai.agent.name`, `otelzero.task.description`, `otelzero.task.expected_output` |
|
|
1186
|
+
| `invoke_agent {name}` (LlamaIndex) | `invoke_agent` | `gen_ai.agent.name`, `gen_ai.request.model` ‡, `gen_ai.tool.definitions` † |
|
|
1187
|
+
| `invoke_agent {name}` (OpenAI Agents) | `invoke_agent` | `gen_ai.agent.name`, `gen_ai.request.model`, `gen_ai.tool.definitions` † |
|
|
1188
|
+
| `invoke_agent {name}` (Pydantic AI, emitted by Pydantic AI itself) | `invoke_agent` | `gen_ai.agent.name`, `gen_ai.aggregated_usage.*`; child `chat` spans carry `gen_ai.request.model`, `gen_ai.usage.*`, `gen_ai.tool.definitions` |
|
|
1189
|
+
| `invoke_agent {name}` (smolagents) | `invoke_agent` | `gen_ai.agent.name`, `gen_ai.request.model`, `gen_ai.tool.definitions` † |
|
|
1190
|
+
| `invoke_agent {name}` (Strands, emitted by Strands itself) | `invoke_agent` | `gen_ai.agent.name`, `gen_ai.provider.name`, `gen_ai.request.model`, `gen_ai.agent.tools`, `gen_ai.usage.*`, `gen_ai.tool.definitions` † |
|
|
1191
|
+
| `invoke_agent {name}` (ADK, emitted by ADK itself) | `invoke_agent` | `gen_ai.agent.name`, `gen_ai.conversation.id` (ADK's session); child `generate_content` spans carry `gen_ai.request.model`, `gen_ai.usage.*`, `gen_ai.tool.definitions` |
|
|
1192
|
+
| `invoke_agent {name}` (Databricks) | `invoke_agent` | `gen_ai.agent.name`, `otelzero.num_messages` |
|
|
1193
|
+
| `execute_tool {tool}` (Databricks) | `execute_tool` | `gen_ai.agent.name`, `gen_ai.tool.name`, `gen_ai.tool.call.id` |
|
|
1194
|
+
| `invoke_agent Claude Agent` | `invoke_agent` | `gen_ai.agent.name`, `gen_ai.request.model`, `gen_ai.tool.definitions` † |
|
|
1195
|
+
| `invoke_agent {name}` (Microsoft, emitted by Agent Framework itself) | `invoke_agent` | `gen_ai.agent.name`, `gen_ai.agent.id`, `gen_ai.provider.name`, `gen_ai.tool.definitions`, `gen_ai.usage.*`; child `chat` spans carry `gen_ai.request.model` |
|
|
1196
|
+
| `invoke_agent Snowflake Cortex Agent` | `invoke_agent` | `gen_ai.agent.name`, `gen_ai.request.model`, `gen_ai.tool.definitions` † |
|
|
1197
|
+
|
|
1198
|
+
Span names follow the conventions' `{operation} {subject}` templates and fall
|
|
1199
|
+
back to the bare operation when the subject isn't available (an unnamed crew
|
|
1200
|
+
becomes just `invoke_workflow`). `execute_task` is a documented custom operation
|
|
1201
|
+
value: a CrewAI Task is the *parent* of the agent invocation it delegates to, so
|
|
1202
|
+
no predefined value fits, and the enum permits a custom one where none applies.
|
|
1203
|
+
|
|
1204
|
+
‡ LlamaIndex: whenever the agent's LLM does not name itself (its
|
|
1205
|
+
`model`/`model_name` is absent or the literal `"unknown"`): the model is then
|
|
1206
|
+
taken from what the provider reported on the first reply, falling back to the LLM
|
|
1207
|
+
class name. An LLM that does name its model reports it from the start.
|
|
1208
|
+
|
|
1209
|
+
† The content attributes — `gen_ai.tool.definitions`, `gen_ai.input.messages`,
|
|
1210
|
+
`gen_ai.output.messages`, `gen_ai.system_instructions` — are **Opt-In** in the
|
|
1211
|
+
conventions, so they appear only when you set
|
|
1212
|
+
`OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=1`. The tool *names* also
|
|
1213
|
+
ride the log envelope's `details` regardless (they are not content), so Kiji's
|
|
1214
|
+
dashboard and apps list them either way; the prompt and response ride `details`
|
|
1215
|
+
only under the same opt-in.
|
|
1216
|
+
|
|
1217
|
+
#### Deliberate deviations
|
|
1218
|
+
|
|
1219
|
+
Two places where otelzero knowingly emits more than a span's definition lists.
|
|
1220
|
+
Extra attributes are permitted; these are the ones worth knowing about:
|
|
1221
|
+
|
|
1222
|
+
- **`gen_ai.usage.*` and `gen_ai.request.model` on `invoke_workflow` spans.**
|
|
1223
|
+
That span defines only `operation.name`, `error.type`, `conversation.id`, and
|
|
1224
|
+
`workflow.name`. CrewAI reports token usage *only* at the crew level, so
|
|
1225
|
+
honouring the list literally would throw the numbers away.
|
|
1226
|
+
- **`gen_ai.conversation.id` and `user.name` on every span**, including
|
|
1227
|
+
`execute_tool`, where `conversation.id` isn't listed. Both are load-bearing
|
|
1228
|
+
for keeping concurrent users' flows apart.
|
|
1229
|
+
|
|
1230
|
+
Not yet implemented, for reference: `gen_ai.usage.cache_creation.input_tokens`,
|
|
1231
|
+
the request parameters (`temperature`, `max_tokens`, `top_p`, …),
|
|
1232
|
+
`gen_ai.tool.description` / `gen_ai.tool.type`, and
|
|
1233
|
+
the Opt-In `execute_tool` content attributes (`gen_ai.tool.call.arguments`,
|
|
1234
|
+
`gen_ai.tool.call.result`) — tool arguments and results are recorded today only
|
|
1235
|
+
as parts inside `gen_ai.input.messages` / `gen_ai.output.messages`.
|
|
1236
|
+
|
|
1237
|
+
#### Splitting flows per user
|
|
1238
|
+
|
|
1239
|
+
`TelemetryClient`'s user and session are process-wide, which is right for a
|
|
1240
|
+
script and wrong for a hosted agent — many callers in one process would report
|
|
1241
|
+
the same user and collapse into one flow. Wrap each run to give it its own
|
|
1242
|
+
identity:
|
|
1243
|
+
|
|
1244
|
+
```python
|
|
1245
|
+
import otelzero
|
|
1246
|
+
|
|
1247
|
+
with otelzero.run_context(user="alice", conversation_id=thread_id):
|
|
1248
|
+
crew.kickoff(...)
|
|
1249
|
+
```
|
|
1250
|
+
|
|
1251
|
+
`user` and `conversation_id` become the `user.name` and `gen_ai.conversation.id`
|
|
1252
|
+
span attributes and the log envelope's `user` / `session`. Omit
|
|
1253
|
+
`conversation_id` for a fresh flow per run, or pass your own thread id to keep a
|
|
1254
|
+
multi-turn conversation together. The scope lives in a `ContextVar`, so
|
|
1255
|
+
concurrent `asyncio` tasks and threads each see their own.
|
|
1256
|
+
|
|
1257
|
+
#### Migrating from the pre-1.0 names
|
|
1258
|
+
|
|
1259
|
+
The client emits only the names above. The server reads them **and** their
|
|
1260
|
+
pre-1.0 spellings, so a client older than the server it reports to keeps
|
|
1261
|
+
producing agent cards and token counts unchanged.
|
|
1262
|
+
|
|
1263
|
+
| Pre-1.0 | Now |
|
|
1264
|
+
|---|---|
|
|
1265
|
+
| `gen_ai.system` | *removed* — framework is the `service.name` resource attribute |
|
|
1266
|
+
| `gen_ai.operation.name: crew` / `agent` / `task` | `invoke_workflow` / `invoke_agent` / `execute_task` |
|
|
1267
|
+
| `mira.user` (resource) | `user.name` (span) |
|
|
1268
|
+
| *(session was the trace id)* | `gen_ai.conversation.id` (span) |
|
|
1269
|
+
| `mira.usage.reasoning_tokens` | `gen_ai.usage.reasoning.output_tokens` |
|
|
1270
|
+
| `mira.usage.cache_read_tokens` | `gen_ai.usage.cache_read.input_tokens` |
|
|
1271
|
+
| `mira.agent.goal` | `gen_ai.agent.description` |
|
|
1272
|
+
| `mira.tool.call_id` | `gen_ai.tool.call.id` |
|
|
1273
|
+
| `mira.agent.tools` | `gen_ai.tool.definitions` (opt-in — see above) |
|
|
1274
|
+
| `gen_ai.agent.id` | `otelzero.agent.id` — client-span-only, and the values are transient instance ids |
|
|
1275
|
+
| *(span status was `OK` on success)* | left unset, as the conventions require |
|
|
1276
|
+
| `mira.chain`, `mira.app_name` | `gen_ai.workflow.name` |
|
|
1277
|
+
| `mira.usage.total_tokens`, `mira.num_agents`, `mira.num_tasks`, `mira.num_messages`, `mira.steps`, `mira.agent.class`, `mira.task.*` | the same attributes, renamed to `otelzero.*` — no convention equivalent |
|
|
1278
|
+
|
|
1279
|
+
Third-party emitters you don't control (LangChain's LangSmith connector,
|
|
1280
|
+
Azure Monitor's Copilot Studio export) are unaffected: setting `mira.user` on
|
|
1281
|
+
the resource still attributes their traces.
|
|
1282
|
+
|
|
1283
|
+
CrewAI and Databricks both surface a task level nested under the agent — CrewAI
|
|
1284
|
+
because its framework exposes task execution as a wrappable method, Databricks by
|
|
1285
|
+
reading the tool calls out of each `ResponsesAgent` response. The OpenAI Agents
|
|
1286
|
+
SDK, smolagents, the Claude Agent SDK and the Snowflake Cortex Agent SDK are
|
|
1287
|
+
instrumented at a single agent-run boundary. Pydantic AI, Strands, ADK, Microsoft,
|
|
1288
|
+
LangChain and LangGraph are not wrapped at all: their *own* native OTel spans
|
|
1289
|
+
(agent run, model and tool calls) are the trace.
|
|
1290
|
+
|
|
1291
|
+
|
|
1292
|
+
## Customization
|
|
1293
|
+
|
|
1294
|
+
|
|
1295
|
+
### Change Server URL
|
|
1296
|
+
|
|
1297
|
+
```python
|
|
1298
|
+
install_crewai_monitoring(server_url="http://your-server:8000")
|
|
1299
|
+
```
|
|
1300
|
+
|
|
1301
|
+
### Add Custom Instrumentation
|
|
1302
|
+
|
|
1303
|
+
Edit `_instrument_module()` in the import hook:
|
|
1304
|
+
|
|
1305
|
+
```python
|
|
1306
|
+
def _instrument_module(self, fullname, module):
|
|
1307
|
+
if fullname == 'your_custom_module':
|
|
1308
|
+
self._wrap_your_class(module)
|
|
1309
|
+
```
|
|
1310
|
+
|
|
1311
|
+
|
|
1312
|
+
## Troubleshooting
|
|
1313
|
+
|
|
1314
|
+
### Client can't connect to server
|
|
1315
|
+
- Ensure the registry is running (`uv run kiji-agent-registry` in its checkout)
|
|
1316
|
+
- Check firewall settings
|
|
1317
|
+
- Verify server URL in client code
|
|
1318
|
+
|
|
1319
|
+
### No logs appearing
|
|
1320
|
+
- Check console for error messages
|
|
1321
|
+
- Verify instrumentation installed before CrewAI import
|
|
1322
|
+
- Check network connectivity
|
|
1323
|
+
|
|
1324
|
+
### Dashboard not updating
|
|
1325
|
+
- Verify JavaScript console for errors
|
|
1326
|
+
- Check browser compatibility (modern browsers only)
|
|
1327
|
+
- Ensure server is returning data at /api/activity
|
|
1328
|
+
|
|
1329
|
+
|
|
1330
|
+
## Development
|
|
1331
|
+
|
|
1332
|
+
```bash
|
|
1333
|
+
uv sync --extra otel
|
|
1334
|
+
uv run pytest
|
|
1335
|
+
```
|
|
1336
|
+
|
|
1337
|
+
To release to PyPI, create a `release/x.y.z` branch, set the version to `x.y.z`
|
|
1338
|
+
in `pyproject.toml` (under both `[project]` and `[tool.commitizen]`) and
|
|
1339
|
+
`src/otelzero/__init__.py`, run `uv lock`, update `CHANGELOG.md`, and open a pull
|
|
1340
|
+
request into `main`. Merging it runs the tests and publishes that version to PyPI. Every push
|
|
1341
|
+
to `main` also publishes a dev build to TestPyPI.
|
|
1342
|
+
|
|
1343
|
+
## License
|
|
1344
|
+
|
|
1345
|
+
Apache-2.0
|