handoffmap 0.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- handoffmap-0.2.0/LICENSE +21 -0
- handoffmap-0.2.0/PKG-INFO +425 -0
- handoffmap-0.2.0/README.md +403 -0
- handoffmap-0.2.0/pyproject.toml +48 -0
- handoffmap-0.2.0/pyproject.toml.orig +39 -0
- handoffmap-0.2.0/src/handoffmap/__init__.py +50 -0
- handoffmap-0.2.0/src/handoffmap/__main__.py +5 -0
- handoffmap-0.2.0/src/handoffmap/adapters/__init__.py +70 -0
- handoffmap-0.2.0/src/handoffmap/adapters/genai.py +56 -0
- handoffmap-0.2.0/src/handoffmap/adapters/openinference.py +40 -0
- handoffmap-0.2.0/src/handoffmap/checks.py +33 -0
- handoffmap-0.2.0/src/handoffmap/cli.py +169 -0
- handoffmap-0.2.0/src/handoffmap/errors.py +37 -0
- handoffmap-0.2.0/src/handoffmap/i18n.py +262 -0
- handoffmap-0.2.0/src/handoffmap/model.py +156 -0
- handoffmap-0.2.0/src/handoffmap/normalize.py +195 -0
- handoffmap-0.2.0/src/handoffmap/otlp.py +166 -0
- handoffmap-0.2.0/src/handoffmap/py.typed +0 -0
- handoffmap-0.2.0/src/handoffmap/render/__init__.py +1 -0
- handoffmap-0.2.0/src/handoffmap/render/explain.py +42 -0
- handoffmap-0.2.0/src/handoffmap/render/jsonout.py +54 -0
- handoffmap-0.2.0/src/handoffmap/render/markdown.py +30 -0
- handoffmap-0.2.0/src/handoffmap/render/mermaid.py +115 -0
- handoffmap-0.2.0/src/handoffmap/render/table.py +45 -0
- handoffmap-0.2.0/src/handoffmap/samples/SOURCES.md +10 -0
- handoffmap-0.2.0/src/handoffmap/samples/openinference_agents.otlp.json +556 -0
- handoffmap-0.2.0/src/handoffmap/samples/pydantic_ai_v2.otlp.json +1032 -0
- handoffmap-0.2.0/src/handoffmap/samples/pydantic_ai_v5.otlp.json +1032 -0
- handoffmap-0.2.0/src/handoffmap/samples/pydantic_ai_v6.otlp.json +1032 -0
- handoffmap-0.2.0/src/handoffmap/usage.py +35 -0
handoffmap-0.2.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 LeeGyeongPil
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,425 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: handoffmap
|
|
3
|
+
Version: 0.2.0
|
|
4
|
+
Summary: Fold an OTLP agent trace (OTel GenAI / OpenInference) into a Mermaid sequence diagram and a per-agent table. No server, no dependencies.
|
|
5
|
+
Keywords: opentelemetry,genai,openinference,agents,multi-agent,mermaid,observability,tracing
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
License-File: LICENSE
|
|
8
|
+
Classifier: Development Status :: 3 - Alpha
|
|
9
|
+
Classifier: Environment :: Console
|
|
10
|
+
Classifier: Intended Audience :: Developers
|
|
11
|
+
Classifier: Programming Language :: Python :: 3
|
|
12
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
15
|
+
Classifier: Topic :: Software Development :: Debuggers
|
|
16
|
+
Classifier: Typing :: Typed
|
|
17
|
+
Requires-Python: >=3.11
|
|
18
|
+
Project-URL: Homepage, https://github.com/LeeGyeongPil/handoffmap
|
|
19
|
+
Project-URL: Issues, https://github.com/LeeGyeongPil/handoffmap/issues
|
|
20
|
+
Project-URL: Changelog, https://github.com/LeeGyeongPil/handoffmap/blob/main/CHANGELOG.md
|
|
21
|
+
Description-Content-Type: text/markdown
|
|
22
|
+
|
|
23
|
+
# handoffmap
|
|
24
|
+
|
|
25
|
+
**English** | [한국어](https://github.com/LeeGyeongPil/handoffmap/blob/v0.2.0/README.ko.md)
|
|
26
|
+
|
|
27
|
+
handoffmap folds an OpenTelemetry trace of a multi-agent run (OTel GenAI semantic conventions or OpenInference) into a Mermaid sequence diagram plus a per-agent table: who handed off to whom, who called whom as a tool, what ran in parallel, and how many tokens each agent used. It replaces reading dozens of span-tree lines by eye; paste the result into a PR, issue or README and GitHub renders it as a diagram. It reads an OTLP JSON/JSONL file, needs no server and has zero runtime dependencies. Try it: `uv tool install git+https://github.com/LeeGyeongPil/handoffmap && handoffmap --sample pydantic-ai`.
|
|
28
|
+
|
|
29
|
+

|
|
30
|
+
*Mermaid output (raw text; GitHub renders it as a diagram) of a sample trace built with Pydantic AI. A single `par` block shows that the orchestrator called the researcher and the writer as tools at the same time.*
|
|
31
|
+
|
|
32
|
+
## What is handoffmap
|
|
33
|
+
|
|
34
|
+
It is like receiving **one page with an org chart plus a timeline** instead of reading the meeting minutes (the span tree) from start to finish. A trace is a record that lists, as pieces called spans, "this work started and ended at such and such a time". As agents multiply, the pieces become dozens or hundreds, and what you actually want to know are questions like these.
|
|
35
|
+
|
|
36
|
+
- Did this agent receive its work by a **handoff**, or was it **called like a tool (as tool)**?
|
|
37
|
+
- Is the same delegation being repeated?
|
|
38
|
+
- Where did work happen at the same time?
|
|
39
|
+
- How many tokens did each agent use?
|
|
40
|
+
|
|
41
|
+
It suits you if:
|
|
42
|
+
- You build multi-agent apps with Pydantic AI or the OpenAI Agents SDK (+ OpenInference instrumentation) and want to see why a run behaves oddly
|
|
43
|
+
- You want to explain a single run to a colleague or paste it into a PR
|
|
44
|
+
- You want to check per-agent tokens in CI (`--format json`)
|
|
45
|
+
|
|
46
|
+
## Quick start
|
|
47
|
+
|
|
48
|
+
### Installation
|
|
49
|
+
|
|
50
|
+
Until it is published to PyPI, install from GitHub.
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
curl -LsSf https://astral.sh/uv/install.sh | sh # if you do not have uv (macOS: brew install uv)
|
|
54
|
+
uv tool install git+https://github.com/LeeGyeongPil/handoffmap
|
|
55
|
+
# after it is on PyPI: pipx install handoffmap
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
### Run
|
|
59
|
+
|
|
60
|
+
Run it right away with the built-in sample (no file, key or network needed).
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
handoffmap --sample pydantic-ai
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
````text
|
|
67
|
+
## Agent flow: trace 4dc50229 (otel-genai, 11 spans)
|
|
68
|
+
|
|
69
|
+
```mermaid
|
|
70
|
+
sequenceDiagram
|
|
71
|
+
participant a1 as orchestrator
|
|
72
|
+
participant a2 as researcher
|
|
73
|
+
participant a3 as writer
|
|
74
|
+
Note over a1: start
|
|
75
|
+
par concurrent
|
|
76
|
+
a1->>a2: as_tool via ask_researcher (55.0 ms)
|
|
77
|
+
a2->>a2: tool search (51.8 ms)
|
|
78
|
+
and
|
|
79
|
+
a1->>a3: as_tool via ask_writer (1.5 ms)
|
|
80
|
+
end
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
### Agents
|
|
84
|
+
|
|
85
|
+
| agent | runs | LLM calls | tools | input tokens | output tokens | time |
|
|
86
|
+
|---|---:|---:|---:|---:|---:|---:|
|
|
87
|
+
| orchestrator | 1 | 2 | 2 | 106 | 21 | 67.8 ms |
|
|
88
|
+
| researcher | 1 | 2 | 1 | 105 | 11 | 55.0 ms |
|
|
89
|
+
| writer | 1 | 1 | 0 | 51 | 1 | 1.5 ms |
|
|
90
|
+
| **total** | 3 | 5 | 3 | 262 | 33 | 124.3 ms |
|
|
91
|
+
|
|
92
|
+
_Tokens are summed from LLM spans only. `time` is the wall time of each run, including work it delegated. `–` means the trace does not contain the value._
|
|
93
|
+
|
|
94
|
+
_Sample trace: recorded from practice (mock) models, so token counts and timings are made up._
|
|
95
|
+
````
|
|
96
|
+
|
|
97
|
+
If you paste the ` ```mermaid ` block of the output above into a place like GitHub, it is drawn as a diagram. How to read these lines is in "How to read it" below.
|
|
98
|
+
|
|
99
|
+
> This sample was made with practice (mock) models. The token counts and times are mock values. See "About the sample traces" for details.
|
|
100
|
+
|
|
101
|
+
## Making your own trace file
|
|
102
|
+
|
|
103
|
+
handoffmap reads an **OTLP JSON** file (the JSON form of `ExportTraceServiceRequest`) or a **JSONL** with one request per line.
|
|
104
|
+
|
|
105
|
+
1. **If you use the OpenTelemetry Collector**: it reads the JSONL written by the `file` exporter as it is.
|
|
106
|
+
2. **If it is a Python app**: the Python SDK has no OTLP JSON file exporter ([related PR #4996](https://github.com/open-telemetry/opentelemetry-python/pull/4996), checked 2026-11-11). So a short example is provided that collects spans in memory and encodes them itself: [`examples/save_otlp_json.py`](https://github.com/LeeGyeongPil/handoffmap/blob/v0.2.0/examples/save_otlp_json.py). Just copy the marked part into your app.
|
|
107
|
+
3. **If you want to try generating a trace**: there is an example that builds one with mock models and no keys. The required packages are pinned inside the script, so `uv run` installs them for you (there is a lot to download, so the first run may be slow).
|
|
108
|
+
|
|
109
|
+
```bash
|
|
110
|
+
uv run examples/make_pydantic_trace.py trace.json
|
|
111
|
+
handoffmap trace.json
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
It can also read traces made with real models (OpenAI and others), but that needs keys, fees and response time, so it is optional. All the examples in this repository use mock models.
|
|
115
|
+
|
|
116
|
+
## How to read it
|
|
117
|
+
|
|
118
|
+
Read a line of the Mermaid sequence diagram through three kinds of arrows and the `par` block.
|
|
119
|
+
|
|
120
|
+
| What you see | Meaning | Control flow |
|
|
121
|
+
|---|---|---|
|
|
122
|
+
| `a1->>a2: as_tool via ask_researcher` | a1 called a2 through the tool `ask_researcher` | The result returns to the caller |
|
|
123
|
+
| `a1->>a2: call` | Another agent is run directly inside an agent, without a tool | The result returns to the caller |
|
|
124
|
+
| `a1-)a2: handoff` | a1 **handed off** to a2 (an open arrow) | It does not return |
|
|
125
|
+
| `par concurrent … and … end` | A group of activities that overlap in time | Run at the same time |
|
|
126
|
+
| `a2->>a2: tool search` | The agent calls an ordinary tool | (can be hidden with `--no-tools`) |
|
|
127
|
+
|
|
128
|
+
The difference between handoff and as_tool is visible at a glance in the OpenInference sample.
|
|
129
|
+
|
|
130
|
+
```bash
|
|
131
|
+
handoffmap --sample openinference --format mermaid
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
```text
|
|
135
|
+
sequenceDiagram
|
|
136
|
+
participant a1 as triage
|
|
137
|
+
participant a2 as billing
|
|
138
|
+
participant a3 as summarizer
|
|
139
|
+
Note over a1: start
|
|
140
|
+
a1-)a2: handoff (4.5 ms)
|
|
141
|
+
par concurrent
|
|
142
|
+
a2->>a2: tool lookup (1.2 ms)
|
|
143
|
+
and
|
|
144
|
+
a2->>a3: as_tool via summarize (1.0 ms)
|
|
145
|
+
end
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+

|
|
149
|
+
*`triage -) billing` is a handoff that does not return, and `billing ->> summarizer` is an as_tool that returns.*
|
|
150
|
+
|
|
151
|
+
Seen as a span tree (`Agent workflow` is an unnamed wrapper span), the same trace is 17 lines like this. billing appears only as a **sibling** of triage, so it is hard to notice that it is a handoff.
|
|
152
|
+
|
|
153
|
+
```text
|
|
154
|
+
Agent workflow
|
|
155
|
+
Agent workflow
|
|
156
|
+
triage
|
|
157
|
+
turn
|
|
158
|
+
generation
|
|
159
|
+
handoff to billing
|
|
160
|
+
billing
|
|
161
|
+
turn
|
|
162
|
+
generation
|
|
163
|
+
lookup
|
|
164
|
+
summarize
|
|
165
|
+
Agent workflow
|
|
166
|
+
summarizer
|
|
167
|
+
turn
|
|
168
|
+
generation
|
|
169
|
+
turn
|
|
170
|
+
generation
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
The table columns are as follows.
|
|
174
|
+
|
|
175
|
+
| Column | Meaning |
|
|
176
|
+
|---|---|
|
|
177
|
+
| runs | How many times that agent ran (each run is counted separately even under the same name) |
|
|
178
|
+
| LLM calls | Number of LLM spans under that agent |
|
|
179
|
+
| tools | Number of ordinary tool calls (handoff marker tools are excluded) |
|
|
180
|
+
| input / output tokens | Tokens summed from LLM spans only. `–` if the trace has no value |
|
|
181
|
+
| time | The wall-clock time of that run (including delegated work) |
|
|
182
|
+
|
|
183
|
+
```bash
|
|
184
|
+
handoffmap --sample pydantic-ai --format table
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+

|
|
188
|
+
*Total row: 262 input and 33 output tokens.*
|
|
189
|
+
|
|
190
|
+
## Usage
|
|
191
|
+
|
|
192
|
+
```text
|
|
193
|
+
handoffmap [FILE | --sample NAME] [--format md|mermaid|table|json] [--explain]
|
|
194
|
+
[--trace TRACE_ID] [--max-parallel N] [--repeat-threshold N] [--no-tools] [-o OUT]
|
|
195
|
+
[--lang en|ko]
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
| Option | What it does | Example |
|
|
199
|
+
|---|---|---|
|
|
200
|
+
| `FILE` | An OTLP JSON or JSONL file | `handoffmap run.json` |
|
|
201
|
+
| `--sample NAME` | A built-in practice trace (`pydantic-ai`, `pydantic-ai-v2`, `pydantic-ai-v6`, `openinference`) | `handoffmap --sample openinference` |
|
|
202
|
+
| `--format`, `-f` | Output format: `md` (default), `mermaid`, `table`, `json` | `handoffmap run.json -f json` |
|
|
203
|
+
| `--explain` | The detected convention, the grounds for each edge judgment, and the ignored wrappers | `handoffmap run.json --explain` |
|
|
204
|
+
| `--trace ID` | Only one trace from a file with several mixed (the full ID or a non-ambiguous prefix) | `--trace 4dc5` |
|
|
205
|
+
| `--max-parallel N` | A `wide-parallel` warning if one parallel group has more than N simultaneous activities (default 8) | `--max-parallel 1` |
|
|
206
|
+
| `--repeat-threshold N` | A `repeated-delegation` warning if the same delegation happens N times (default 3) | `--repeat-threshold 2` |
|
|
207
|
+
| `--no-tools` | Hides ordinary tool-call lines | `--no-tools` |
|
|
208
|
+
| `-o`, `--output OUT` | Save to a file | `-o flow.md` |
|
|
209
|
+
| `--lang LANG` | Output language: `en` or `ko`. Applies to help, errors, and the md, table and `--explain` text (see "Output language") | `--lang ko` |
|
|
210
|
+
| `--version` | Show the version | |
|
|
211
|
+
|
|
212
|
+
Exit codes: `0` normal / `1` no agent spans / `2` input error (not OTLP, or file missing).
|
|
213
|
+
|
|
214
|
+
```text
|
|
215
|
+
$ handoffmap not-otlp.json --lang en
|
|
216
|
+
handoffmap: not an OTLP trace: expected a JSON object with a 'resourceSpans' list
|
|
217
|
+
$ echo $?
|
|
218
|
+
2
|
|
219
|
+
```
|
|
220
|
+
|
|
221
|
+
**Output language**
|
|
222
|
+
|
|
223
|
+
The help text, error messages, and the `md`, `table` and `--explain` text come in English and Korean. The language is chosen in this order, and the first one set wins: `--lang en|ko` > the `HANDOFFMAP_LANG` environment variable > `LC_ALL` > `LC_MESSAGES` > `LANG`. A value that starts with `ko` (for example `ko_KR.UTF-8`) means Korean; anything else, or nothing set, means English.
|
|
224
|
+
|
|
225
|
+
```bash
|
|
226
|
+
handoffmap --sample pydantic-ai --lang ko # one run in Korean
|
|
227
|
+
HANDOFFMAP_LANG=ko handoffmap --help # help text in Korean
|
|
228
|
+
LANG=fr_FR.UTF-8 handoffmap --help # not Korean, so English
|
|
229
|
+
```
|
|
230
|
+
|
|
231
|
+
Machine-readable output does not change with the language: `--format json` and `--format mermaid` are byte-identical for `en` and `ko` (warning `message` fields in the JSON stay English), and so are the exit codes. The Python API (`render_markdown`, `render_table`, `render_explain`) defaults to `lang="en"` and never reads the environment; pass `lang="ko"` to get Korean text.
|
|
232
|
+
|
|
233
|
+
The screenshots of the table, `--explain` and warning output below are in English; [README.ko.md](https://github.com/LeeGyeongPil/handoffmap/blob/v0.2.0/README.ko.md) shows the same screens in Korean.
|
|
234
|
+
|
|
235
|
+
**Scenario 1 - Paste into a PR**
|
|
236
|
+
|
|
237
|
+
```bash
|
|
238
|
+
handoffmap run.json > flow.md
|
|
239
|
+
```
|
|
240
|
+
If you paste the contents of `flow.md` into the PR description, the Mermaid block is drawn as a diagram and the table appears with it.
|
|
241
|
+
|
|
242
|
+
**Scenario 2 - Check the grounds**
|
|
243
|
+
|
|
244
|
+
```bash
|
|
245
|
+
handoffmap --sample openinference --explain
|
|
246
|
+
```
|
|
247
|
+
|
|
248
|
+
```text
|
|
249
|
+
trace: c9c642518c7203aee90e7f8c52f974ad
|
|
250
|
+
convention: openinference
|
|
251
|
+
size: 17 spans, 3 agent runs, 3 agents
|
|
252
|
+
agents: triage, billing, summarizer
|
|
253
|
+
edges:
|
|
254
|
+
(none) -> triage: root [no agent above it and no handoff source: it started the run]
|
|
255
|
+
triage -> billing: handoff [no agent ancestor; the span is a sibling and names its predecessor (graph.node.parent_id)]
|
|
256
|
+
billing -> summarizer: as_tool via summarize [a tool span sits between the caller and this agent (agent used as a tool)]
|
|
257
|
+
ignored wrappers (AGENT span without agent.name, not counted as agents): 'Agent workflow' x1
|
|
258
|
+
tokens: none of the LLM spans carry token counts
|
|
259
|
+
```
|
|
260
|
+
(The normal report follows this.) You can see the detected convention, the number of spans accepted as agents, and the ignored wrappers. This sample has empty token values, so the token cells of the table are `–`.
|
|
261
|
+
|
|
262
|
+

|
|
263
|
+
*The first part of the `--explain` output.*
|
|
264
|
+
|
|
265
|
+
**Scenario 3 - Repeated-delegation warning**
|
|
266
|
+
|
|
267
|
+
```bash
|
|
268
|
+
handoffmap tests/fixtures/synthetic_loop.otlp.json
|
|
269
|
+
```
|
|
270
|
+
|
|
271
|
+

|
|
272
|
+
*A synthetic trace made by hand (not measured). The same delegation repeated 3 times, so a warning was attached, and coder's runs is 3.*
|
|
273
|
+
|
|
274
|
+
```text
|
|
275
|
+
### Warnings
|
|
276
|
+
|
|
277
|
+
- **repeated-delegation**: planner -> coder happened 3 times (threshold 3); check for a loop or a missing stop condition
|
|
278
|
+
```
|
|
279
|
+
|
|
280
|
+
**Scenario 4 - Check tokens in CI**
|
|
281
|
+
|
|
282
|
+
```bash
|
|
283
|
+
handoffmap --sample pydantic-ai --format json
|
|
284
|
+
```
|
|
285
|
+
The JSON contains `trace_id`, `convention`, `span_count`, `agents`, `totals`, `edges`, `parallel` and `warnings`. An `agents` entry has `name`, `runs`, `llm_calls`, `tool_calls`, `input_tokens`, `output_tokens`, `cost` and `duration_ms`. You can filter it like `jq '.agents[] | {name, input_tokens}'` (jq is installed separately).
|
|
286
|
+
|
|
287
|
+
**Python API**
|
|
288
|
+
|
|
289
|
+
```python
|
|
290
|
+
import handoffmap
|
|
291
|
+
|
|
292
|
+
trace = handoffmap.load("src/handoffmap/samples/pydantic_ai_v5.otlp.json")[0]
|
|
293
|
+
flow = handoffmap.analyze(trace)
|
|
294
|
+
for e in flow.edges:
|
|
295
|
+
print(e.caller, "->", e.callee, e.kind, e.via)
|
|
296
|
+
print(handoffmap.render_table(flow))
|
|
297
|
+
```
|
|
298
|
+
|
|
299
|
+
```text
|
|
300
|
+
None -> orchestrator root None
|
|
301
|
+
orchestrator -> researcher as_tool ask_researcher
|
|
302
|
+
orchestrator -> writer as_tool ask_writer
|
|
303
|
+
| agent | runs | LLM calls | tools | input tokens | output tokens | time |
|
|
304
|
+
|---|---:|---:|---:|---:|---:|---:|
|
|
305
|
+
| orchestrator | 1 | 2 | 2 | 106 | 21 | 67.8 ms |
|
|
306
|
+
| researcher | 1 | 2 | 1 | 105 | 11 | 55.0 ms |
|
|
307
|
+
| writer | 1 | 1 | 0 | 51 | 1 | 1.5 ms |
|
|
308
|
+
| **total** | 3 | 5 | 3 | 262 | 33 | 124.3 ms |
|
|
309
|
+
```
|
|
310
|
+
|
|
311
|
+
Installation: to use it as a library, add it to your project with `uv add git+https://github.com/LeeGyeongPil/handoffmap` (or `pip install git+https://github.com/LeeGyeongPil/handoffmap`). `uv tool install` is an isolated environment, so `import handoffmap` does not work there. The sample path in the example above (`src/handoffmap/samples/...`) is correct only when run from the root of a cloned repository.
|
|
312
|
+
|
|
313
|
+
Public names: `load`, `parse`, `analyze`, `render_mermaid`, `render_table`, `render_markdown`, `render_explain`, `to_dict`, the exception `HandoffmapError` (`NotOtlpError`, `NoAgentSpansError`) and the data types (`Trace`, `Flow`, `Edge`, etc.). The same code is in [`examples/api_demo.py`](https://github.com/LeeGyeongPil/handoffmap/blob/v0.2.0/examples/api_demo.py) (run `uv run python examples/api_demo.py` from the repository root).
|
|
314
|
+
|
|
315
|
+
Message contents (prompt and response bodies such as `gen_ai.input.messages`) are never included in any output format.
|
|
316
|
+
|
|
317
|
+
## Configuration
|
|
318
|
+
|
|
319
|
+
There are no configuration files or API keys. The tool only reads files and does not use the network. The only things you can adjust are the options above (`--max-parallel`, `--repeat-threshold`, `--lang`) and one optional environment variable, `HANDOFFMAP_LANG` (`en` or `ko`), which sets the output language when `--lang` is not given (see "Output language").
|
|
320
|
+
|
|
321
|
+
## How it decides
|
|
322
|
+
|
|
323
|
+
```mermaid
|
|
324
|
+
flowchart LR
|
|
325
|
+
F[OTLP JSON / JSONL] --> L[otlp.py]
|
|
326
|
+
S[--sample] --> L
|
|
327
|
+
L --> D{"Detect convention"}
|
|
328
|
+
D -->|gen_ai.operation.name| G[OTel GenAI]
|
|
329
|
+
D -->|openinference.span.kind| O[OpenInference]
|
|
330
|
+
G --> N[normalize.py]
|
|
331
|
+
O --> N
|
|
332
|
+
N --> P["Parallel groups"]
|
|
333
|
+
N --> U["Token totals"]
|
|
334
|
+
N --> C["Warning rules"]
|
|
335
|
+
P --> R[md / mermaid / table / json]
|
|
336
|
+
U --> R
|
|
337
|
+
C --> R
|
|
338
|
+
```
|
|
339
|
+
|
|
340
|
+
1. **Reading**: IDs are accepted as both hex and base64, and span kinds as both integers and `SPAN_KIND_*` names. If several traces are mixed, they are split per trace.
|
|
341
|
+
2. **Convention detection**: it decides by attributes, not by span name. This is because the convention is still at the Development stage, so span names differ by framework and version (Pydantic AI instrumentation v2 uses `agent run`, v5 and v6 use `invoke_agent name`). For OTel GenAI it looks at `gen_ai.operation.name` (`invoke_agent`, `execute_tool`, `chat`, etc.), and for OpenInference at `openinference.span.kind` (`AGENT`, `TOOL`, `LLM`) and `agent.name`. An AGENT span with no name (a wrapper) is not counted as an agent.
|
|
342
|
+
3. **Edge kind decided by ancestor search**: from an agent run, it walks up the parent chain.
|
|
343
|
+
- If it meets a tool span while climbing, it is `as_tool`; if it reaches an agent ancestor without meeting one, it is `call`.
|
|
344
|
+
- If there is no agent ancestor and `graph.node.parent_id` exists, it is a `handoff` (sibling span).
|
|
345
|
+
- If neither, it is `root`.
|
|
346
|
+
- If the chain is broken because a parent span was not exported, it looks instead by time containment.
|
|
347
|
+
4. **Parallel detection**: parallelism is not visible in the tree, so it is found by time. If the time intervals of activities under the same agent run (sub-agents and ordinary tools) overlap, they form one group. If an end time equals a start time, it is not an overlap.
|
|
348
|
+
5. **Tokens are summed from LLM spans only**: agent spans can also carry total attributes (`gen_ai.usage.*`, `gen_ai.aggregated_usage.*`), but adding them would count sub-calls twice (double counting) or drop the sub-calls. Only the smallest unit, the LLM call, is added once and attributed to the nearest agent run. That is why the sample gives 262/33. The total attributes are used only as cross-check values in `--explain`.
|
|
349
|
+
6. **Warnings**: `repeated-delegation` if the same caller→target delegation happens N or more times, and `wide-parallel` if the simultaneous activities of one group exceed the limit.
|
|
350
|
+
|
|
351
|
+
## Supported convention snapshot
|
|
352
|
+
|
|
353
|
+
Conventions can change, so the reference date is given (checked 2026-11-11).
|
|
354
|
+
|
|
355
|
+
| Target | Reference |
|
|
356
|
+
|---|---|
|
|
357
|
+
| OTel GenAI semantic conventions | The [semantic-conventions-genai](https://github.com/open-telemetry/semantic-conventions-genai) repository, Development status |
|
|
358
|
+
| OpenInference | [semantic conventions spec](https://github.com/Arize-ai/openinference/blob/main/spec/semantic_conventions.md), 0.1.43 |
|
|
359
|
+
| Versions the samples were made with | pydantic-ai-slim 2.67.0, openai-agents 0.24.0, openinference-instrumentation-openai-agents 2.7.1 |
|
|
360
|
+
|
|
361
|
+
| Pydantic AI instrumentation version | Span names | handoffmap result |
|
|
362
|
+
|---|---|---|
|
|
363
|
+
| v2 | `agent run`, `running tool` | Same structure (time values differ) |
|
|
364
|
+
| v5 | `invoke_agent name`, `execute_tool name` | Reference |
|
|
365
|
+
| v6 | Same as v5 (only retry-message handling differs) | Same structure (time values differ) |
|
|
366
|
+
|
|
367
|
+
## About the sample traces
|
|
368
|
+
|
|
369
|
+
The 4 built-in samples are traces made with **practice mock models**. They are real spans produced by real framework instrumentation, but the model responses are predetermined, so the token counts and times are mock values. How they differ from traces made with real models: tokens and times are real values (OpenInference's `llm.token_count.*` could not be confirmed through a real-model path), and response times take longer.
|
|
370
|
+
|
|
371
|
+
To recreate them (the script fetches and installs the required packages by itself; the first run may be slow):
|
|
372
|
+
|
|
373
|
+
```bash
|
|
374
|
+
uv run examples/make_pydantic_trace.py trace.json
|
|
375
|
+
uv run examples/make_openai_agents_trace.py trace.json
|
|
376
|
+
```
|
|
377
|
+
|
|
378
|
+
The sample sources and the versions used to make them are in [`src/handoffmap/samples/SOURCES.md`](https://github.com/LeeGyeongPil/handoffmap/blob/v0.2.0/src/handoffmap/samples/SOURCES.md).
|
|
379
|
+
|
|
380
|
+
## Comparison with existing tools (when to use it)
|
|
381
|
+
|
|
382
|
+
- Use it when: you want to take one file and make Markdown/Mermaid to paste into a PR or issue, you want to see things folded by agent (kind of delegation, parallelism, tokens), or you want to run it in CI without a server.
|
|
383
|
+
- Use another tool when: for a dashboard that continuously monitors a running service → [Langfuse](https://langfuse.com/docs/observability/features/agent-graphs), [Arize Phoenix](https://arize.com/docs/ax/observe/tracing-concepts/agent-graphs). To explore the whole span tree in a web UI → [agent-prism](https://github.com/evilmartians/agent-prism), [otel-desktop-viewer](https://github.com/CtrlSpice/otel-desktop-viewer), [otel-tui](https://github.com/ymtdzzz/otel-tui). For what each tool supports, check its own documentation.
|
|
384
|
+
|
|
385
|
+
## Limits and roadmap
|
|
386
|
+
|
|
387
|
+
- Input is only OTLP JSON/JSONL (not gRPC or protobuf). Prompt and response contents are not shown.
|
|
388
|
+
- Cost is shown only when the trace has `llm.cost.total` (`–` otherwise). There is no model price table yet.
|
|
389
|
+
- OpenInference's handoff detection relies on `graph.node.parent_id`, so with other instrumentation it may appear as `root` (check the grounds with `--explain`).
|
|
390
|
+
- The Mermaid output's syntax structure (participant aliases, `par`/`end` balance, special-character escaping) is verified by tests, and the built-in sample outputs passed syntax validation with the mermaid 11 parser (11.17.2). Please check the actual diagram rendering once on GitHub.
|
|
391
|
+
- Very deep nesting (about 500 or more levels of agents) fails in `md`/`mermaid` drawing with the recursion limit (`RecursionError`). In that case use `--format table` or `json`.
|
|
392
|
+
- The time in the `total` row simply sums each run's time, so overlapping delegated sub-work can make it larger than the real wall clock (it does not affect the token and call-count totals). The same goes for `totals.duration_ms` in the JSON.
|
|
393
|
+
- If a file with several traces is output with `--format table`/`mermaid`, they are printed one after another without a trace-separating header. Use the `md` format, or pick one with `--trace`.
|
|
394
|
+
- The in-memory exporter in `examples/save_otlp_json.py` keeps accumulating spans, so it is not suited to long-running apps.
|
|
395
|
+
- With very large traces (thousands of spans), the diagram gets long. You can shorten it with `--no-tools`.
|
|
396
|
+
- Plans: a single-HTML lane diagram with no external resources, coding-agent CLI trace support, a model price table and a local OTLP receiver and file exporter. No schedule is set.
|
|
397
|
+
|
|
398
|
+
## Common problems
|
|
399
|
+
|
|
400
|
+
| Symptom | Cause | Fix |
|
|
401
|
+
|---|---|---|
|
|
402
|
+
| `not an OTLP trace` (exit code 2) | Not an OTLP JSON that has `resourceSpans` | Use the Collector `file` exporter output or a file saved with `examples/save_otlp_json.py` |
|
|
403
|
+
| `no agent spans found` (exit code 1) | The trace has no agent spans, or the convention attributes are missing | Check the detection result with `--explain`, and check that the instrumentation emits `gen_ai.operation.name` or `openinference.span.kind` |
|
|
404
|
+
| The token cells are `–` | The LLM spans have no token attributes | Check whether the instrumentation emits tokens (the `openinference` sample is this case) |
|
|
405
|
+
| An agent name does not show | Unnamed wrapper spans are not counted | Check "ignored wrappers" in `--explain` |
|
|
406
|
+
| `file not found` | A path error | Check the path |
|
|
407
|
+
|
|
408
|
+
## Development and tests
|
|
409
|
+
|
|
410
|
+
```bash
|
|
411
|
+
git clone https://github.com/LeeGyeongPil/handoffmap && cd handoffmap
|
|
412
|
+
uv sync
|
|
413
|
+
uv run pytest -q
|
|
414
|
+
```
|
|
415
|
+
|
|
416
|
+
107 tests run with no keys or network, using only the built-in samples and synthetic fixtures (`tests/fixtures/README.md` distinguishes measured from synthetic). Please leave bugs and suggestions as issues. The change log is [CHANGELOG.md](https://github.com/LeeGyeongPil/handoffmap/blob/v0.2.0/CHANGELOG.md).
|
|
417
|
+
|
|
418
|
+
## Learn more
|
|
419
|
+
|
|
420
|
+
- [OTel GenAI agent span conventions](https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-agent-spans.md)
|
|
421
|
+
- [OpenInference semantic conventions](https://github.com/Arize-ai/openinference/blob/main/spec/semantic_conventions.md)
|
|
422
|
+
|
|
423
|
+
## License and sources
|
|
424
|
+
|
|
425
|
+
MIT. From the OpenTelemetry and OpenInference conventions it references (Apache-2.0), only attribute names are used. The built-in samples are data made by this project's scripts.
|