@mainwave/cli 0.4.0-rc.1 → 0.4.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +47 -8
- package/dist/adapters/claude-code.js +39 -5
- package/dist/adapters/claude-code.js.map +1 -1
- package/dist/adapters/claude-desktop.js +3 -0
- package/dist/adapters/claude-desktop.js.map +1 -1
- package/dist/adapters/codex-cli.js +8 -0
- package/dist/adapters/codex-cli.js.map +1 -1
- package/dist/adapters/copilot-chat.js +9 -0
- package/dist/adapters/copilot-chat.js.map +1 -1
- package/dist/adapters/copilot-cli.js +8 -0
- package/dist/adapters/copilot-cli.js.map +1 -1
- package/dist/adapters/cursor-enterprise.js +9 -0
- package/dist/adapters/cursor-enterprise.js.map +1 -1
- package/dist/adapters/cursor.js +8 -0
- package/dist/adapters/cursor.js.map +1 -1
- package/dist/core/apply.js +18 -2
- package/dist/core/apply.js.map +1 -1
- package/dist/core/auth-commands.js +16 -6
- package/dist/core/auth-commands.js.map +1 -1
- package/dist/core/doctor.js +8 -1
- package/dist/core/doctor.js.map +1 -1
- package/dist/core/interview.js +6 -5
- package/dist/core/interview.js.map +1 -1
- package/dist/core/platform.js +15 -0
- package/dist/core/platform.js.map +1 -1
- package/dist/core/prompts.js +7 -2
- package/dist/core/prompts.js.map +1 -1
- package/dist/core/skills.js +203 -0
- package/dist/core/skills.js.map +1 -0
- package/dist/core/use-case.js +34 -58
- package/dist/core/use-case.js.map +1 -1
- package/dist/core/wizard.js +196 -7
- package/dist/core/wizard.js.map +1 -1
- package/dist/main.js +63 -11
- package/dist/main.js.map +1 -1
- package/package.json +2 -1
- package/skills/mainwave-instrument/SKILL.md +168 -0
- package/skills/mainwave-instrument/reference/api.md +142 -0
- package/skills/mainwave-instrument/reference/frameworks.md +170 -0
- package/skills/mainwave-instrument/reference/troubleshooting.md +131 -0
- package/skills/mainwave-setup/SKILL.md +114 -0
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@mainwave/cli",
|
|
3
|
-
"version": "0.4.
|
|
3
|
+
"version": "0.4.2",
|
|
4
4
|
"description": "Setup wizard for connecting third-party coding tools to Mainwave AI.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -9,6 +9,7 @@
|
|
|
9
9
|
"files": [
|
|
10
10
|
"dist",
|
|
11
11
|
"hooks",
|
|
12
|
+
"skills",
|
|
12
13
|
"README.md",
|
|
13
14
|
"LICENSE"
|
|
14
15
|
],
|
|
@@ -0,0 +1,168 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: mainwave-instrument
|
|
3
|
+
description: Instrument a Python AI agent or LLM application with the Mainwave SDK so its runs, tokens, cost, and tool calls arrive in Mainwave. Use when asked to add Mainwave, add tracing/observability to an agent, connect an app to Mainwave, or when telemetry is configured but no spans are showing up. Covers init placement, framework wiring, use-case identity, flushing, and verification.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Instrumenting an application with Mainwave
|
|
7
|
+
|
|
8
|
+
Mainwave traces LLM applications. The SDK wraps OpenTelemetry and OpenInference
|
|
9
|
+
auto-instrumentation, so most applications need an `init()` call and nothing
|
|
10
|
+
else — the framework's own calls are captured automatically.
|
|
11
|
+
|
|
12
|
+
Your job is to place that call correctly, wire the right framework, bind the
|
|
13
|
+
work to a use case, and then **prove a span arrived**. An instrumentation task
|
|
14
|
+
that ends without verification is not finished.
|
|
15
|
+
|
|
16
|
+
## Before you touch code
|
|
17
|
+
|
|
18
|
+
**Never invent an API key or a use-case id.** Both come from the user's Mainwave
|
|
19
|
+
account. Read them from the environment; if they are absent, stop and ask, or
|
|
20
|
+
have the user run:
|
|
21
|
+
|
|
22
|
+
```bash
|
|
23
|
+
npx @mainwave/cli@latest login
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
Keys and use cases are visible in the app at `app.mainwave.com`. Put them in the
|
|
27
|
+
project's existing secret mechanism (`.env`, the deployment's secret store) —
|
|
28
|
+
**never a literal in source, never a committed file.**
|
|
29
|
+
|
|
30
|
+
## 1. Survey before editing
|
|
31
|
+
|
|
32
|
+
Answer these from the codebase, not from assumption:
|
|
33
|
+
|
|
34
|
+
- **Which LLM libraries are actually called?** Search imports for `openai`,
|
|
35
|
+
`anthropic`, `langchain`, `crewai`, `llama_index`, `litellm`, `boto3`
|
|
36
|
+
(Bedrock), `google.genai`, `mistralai`, `groq`, `vertexai`.
|
|
37
|
+
- **Where does the process start?** The `main()`, the FastAPI/Django app module,
|
|
38
|
+
the Celery worker bootstrap, the CLI entry point. That is where `init()` goes.
|
|
39
|
+
- **What is one "unit of work"?** A ticket triaged, a question answered, a
|
|
40
|
+
document processed. This becomes a run, and it is the level a business
|
|
41
|
+
operator reads results at — not one HTTP request, not one LLM call.
|
|
42
|
+
- **Is OpenTelemetry already set up?** If the app builds its own
|
|
43
|
+
`TracerProvider`, the SDK detects it and runs in coexist mode. Do not remove
|
|
44
|
+
their setup; `init()` attaches to it.
|
|
45
|
+
|
|
46
|
+
## 2. Install with narrow extras
|
|
47
|
+
|
|
48
|
+
One extra per framework the app genuinely uses:
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
uv add 'mainwave[openai]' # or: pip install 'mainwave[openai]'
|
|
52
|
+
uv add 'mainwave[langchain,openai]'
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
**Never install `mainwave[all]`** — it exists for CI matrices, and in an
|
|
56
|
+
application it drags in instrumentors for libraries that aren't there. Python
|
|
57
|
+
3.10+ is required.
|
|
58
|
+
|
|
59
|
+
## 3. Initialize once, at process entry
|
|
60
|
+
|
|
61
|
+
```python
|
|
62
|
+
import mainwave
|
|
63
|
+
|
|
64
|
+
mainwave.init(service_name="support-triage")
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
`api_key` and `use_case_id` come from `MAINWAVE_API_KEY` and
|
|
68
|
+
`MAINWAVE_USE_CASE_ID` when not passed. Call it **before** the framework does
|
|
69
|
+
its work, and prefer placing it before the framework imports — instrumentors
|
|
70
|
+
patch libraries at wire time.
|
|
71
|
+
|
|
72
|
+
Rules that bite:
|
|
73
|
+
|
|
74
|
+
- **Once per process.** A second `init()` with different settings raises
|
|
75
|
+
`MainwaveReinitConflict`. In a web app, initialize in the lifespan/startup
|
|
76
|
+
hook, not per request.
|
|
77
|
+
- **Late imports need `mainwave.instrument()`.** If the LLM library is imported
|
|
78
|
+
lazily — inside a function, a Click subcommand, a FastAPI route — it wasn't
|
|
79
|
+
loaded when `init()` walked the registry. Call `mainwave.instrument("openai")`
|
|
80
|
+
after the import.
|
|
81
|
+
- **`service_name` is the deployable**, not the agent. `"support-triage"`, not
|
|
82
|
+
`"ClassifierAgent"`. Defaults to `mainwave-app`, which tells the operator
|
|
83
|
+
nothing — always set it.
|
|
84
|
+
- If `use_case_id` is missing, the SDK derives a stable one from the key and
|
|
85
|
+
service name so nothing crashes, but that telemetry lands in a use case the
|
|
86
|
+
operator never created. Set it explicitly.
|
|
87
|
+
|
|
88
|
+
## 4. Mark the unit of work
|
|
89
|
+
|
|
90
|
+
Wrap each unit of work in a run. This is what makes the trace legible as
|
|
91
|
+
business activity rather than a pile of LLM calls:
|
|
92
|
+
|
|
93
|
+
```python
|
|
94
|
+
with mainwave.run(f"triage-{ticket.id}", metadata={"ticket_id": ticket.id}) as r:
|
|
95
|
+
decision = classify(ticket) # instrumented calls land inside
|
|
96
|
+
r.end(reason="success", decision=decision, iterations=2)
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
`mainwave.arun(...)` is the async twin. `mainwave.start_run(...)` returns the
|
|
100
|
+
handle without a context manager, for a run that outlives one block.
|
|
101
|
+
|
|
102
|
+
Identity is **work-scoped and rides on the span**, so it applies to a block, not
|
|
103
|
+
to the process. When one deployable serves several use cases, override per run:
|
|
104
|
+
|
|
105
|
+
```python
|
|
106
|
+
with mainwave.run(run_id, use_case_id=UCID_FOR_THIS_TENANT, agent_name="triage"):
|
|
107
|
+
...
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
Bind at the **root** of the unit of work. Spans created before the bind do not
|
|
111
|
+
inherit it. `mainwave.bind_identity(...)` does the same without opening a run.
|
|
112
|
+
|
|
113
|
+
## 5. Flush before the process dies
|
|
114
|
+
|
|
115
|
+
Short-lived processes — scripts, jobs, serverless handlers, CLI commands — must
|
|
116
|
+
flush or the last spans die with the process:
|
|
117
|
+
|
|
118
|
+
```python
|
|
119
|
+
mainwave.flush() # mainwave.aflush() in async code
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
Long-running servers do not need this; the batch processor drains on its own.
|
|
123
|
+
|
|
124
|
+
## 6. Verify — do not skip this
|
|
125
|
+
|
|
126
|
+
Run the application so it does real work, then watch for it to land:
|
|
127
|
+
|
|
128
|
+
```bash
|
|
129
|
+
npx @mainwave/cli@latest verify --ucid <use-case-id> --no-sample
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
`--no-sample` is required here. Without it the CLI pushes its own synthetic
|
|
133
|
+
trace, which proves the network path but would pass even if your
|
|
134
|
+
instrumentation does nothing. The command reports the agent names it saw — check
|
|
135
|
+
that the application's own name is among them.
|
|
136
|
+
|
|
137
|
+
If nothing arrives, work `reference/troubleshooting.md`; the most common cause
|
|
138
|
+
is by design and is listed first.
|
|
139
|
+
|
|
140
|
+
## Guardrails
|
|
141
|
+
|
|
142
|
+
- **Never import anything underscore-prefixed** (`mainwave._config`,
|
|
143
|
+
`mainwave._runtime`). Those are internals and change without notice. If it is
|
|
144
|
+
not in `mainwave.__all__`, it is not public.
|
|
145
|
+
- **Never wire LangChain twice.** The auto-instrumentor and
|
|
146
|
+
`mainwave.langchain.CallbackHandler` each capture every call; both together
|
|
147
|
+
double-count tokens and cost. Pick one — see `reference/frameworks.md`.
|
|
148
|
+
- **Never let instrumentation change application behavior.** No swapped clients,
|
|
149
|
+
no altered control flow, no new required config. If the SDK cannot start, the
|
|
150
|
+
app must still run — that is why `MAINWAVE_DISABLED=1` exists.
|
|
151
|
+
- **Do not restructure the app to fit the SDK.** If the code has no clean unit
|
|
152
|
+
of work, add a run at the outermost sensible boundary and say so; do not
|
|
153
|
+
refactor their architecture as part of an instrumentation task.
|
|
154
|
+
|
|
155
|
+
## Reference
|
|
156
|
+
|
|
157
|
+
- `reference/api.md` — the exact public surface: signatures, env vars, defaults.
|
|
158
|
+
- `reference/frameworks.md` — per-framework wiring, including the LangChain fork
|
|
159
|
+
and frameworks with no working instrumentor.
|
|
160
|
+
- `reference/troubleshooting.md` — "no spans are showing up", in the order the
|
|
161
|
+
causes actually occur.
|
|
162
|
+
|
|
163
|
+
## Report at the end
|
|
164
|
+
|
|
165
|
+
Tell the user which files changed, which extras were installed, what the run
|
|
166
|
+
boundary is, and the verification result. If you had to guess at the unit of
|
|
167
|
+
work or the use case, say so plainly — that is the one decision they should
|
|
168
|
+
check.
|
|
@@ -0,0 +1,142 @@
|
|
|
1
|
+
# Mainwave Python SDK — public surface
|
|
2
|
+
|
|
3
|
+
Everything here is stable. Anything not listed — in particular any
|
|
4
|
+
underscore-prefixed module — is internal, even when importable. The
|
|
5
|
+
authoritative list at runtime is `mainwave.__all__`.
|
|
6
|
+
|
|
7
|
+
## Setup
|
|
8
|
+
|
|
9
|
+
```python
|
|
10
|
+
mainwave.init(
|
|
11
|
+
*,
|
|
12
|
+
api_key: str | None = None, # env: MAINWAVE_API_KEY (required)
|
|
13
|
+
use_case_id: str | None = None, # env: MAINWAVE_USE_CASE_ID
|
|
14
|
+
collector_url: str | None = None, # env: MAINWAVE_COLLECTOR_URL
|
|
15
|
+
service_name: str | None = None, # env: OTEL_SERVICE_NAME
|
|
16
|
+
deployment_environment: str | None = None,# env: MAINWAVE_ENVIRONMENT
|
|
17
|
+
disabled: bool | None = None, # env: MAINWAVE_DISABLED
|
|
18
|
+
skip_instrumentors: list[str] | None = None,
|
|
19
|
+
) -> None
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
Idempotent when called again with identical config; raises
|
|
23
|
+
`MainwaveReinitConflict` when the config differs. Bad config raises
|
|
24
|
+
`MainwaveConfigError`. After a successful init, runtime errors are swallowed —
|
|
25
|
+
the SDK cannot break its host.
|
|
26
|
+
|
|
27
|
+
`skip_instrumentors` leaves a named instrumentor unwired even when installed.
|
|
28
|
+
It applies only to the one-time wiring at first init, so it is not part of the
|
|
29
|
+
re-init equality check.
|
|
30
|
+
|
|
31
|
+
```python
|
|
32
|
+
mainwave.instrument(name: str | list[str] | None = None) -> list[str]
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
Wires instrumentors after a late import. `None` re-walks the whole registry and
|
|
36
|
+
returns what newly wired. A name whose package is missing or whose version
|
|
37
|
+
conflicts is dropped from the result with a WARN — not raised. An _unregistered_
|
|
38
|
+
name raises `KeyError`, and a list is validated fully before anything wires.
|
|
39
|
+
Hyphens and underscores are equivalent (`llama-index` == `llama_index`).
|
|
40
|
+
|
|
41
|
+
```python
|
|
42
|
+
mainwave.register_framework(name: str) -> None
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
Stamps a framework name onto spans without wiring an instrumentor — for a
|
|
46
|
+
framework whose instrumentor is unavailable or incompatible, so it still appears
|
|
47
|
+
in telemetry.
|
|
48
|
+
|
|
49
|
+
## Runs and spans
|
|
50
|
+
|
|
51
|
+
```python
|
|
52
|
+
with mainwave.run(run_id, *, metadata=None, use_case_id=None, agent_name=None) as r: ...
|
|
53
|
+
async with mainwave.arun(...) as r: ...
|
|
54
|
+
r = mainwave.start_run(...) # same args, no context manager
|
|
55
|
+
with mainwave.span(name, *, attrs=None) as s: ...
|
|
56
|
+
with mainwave.bind_identity(*, use_case_id=None, agent_name=None): ...
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
`span()` is the escape hatch for a manual span; it inherits the active run's id
|
|
60
|
+
and attributes. `bind_identity()` binds identity into the OTel context without
|
|
61
|
+
opening a run.
|
|
62
|
+
|
|
63
|
+
### `Run`
|
|
64
|
+
|
|
65
|
+
```python
|
|
66
|
+
r.run_id # str
|
|
67
|
+
r.set_metadata(**kwargs) -> None
|
|
68
|
+
r.record_error(exc: BaseException) -> None
|
|
69
|
+
r.end(status=StatusCode.OK, description=None, *,
|
|
70
|
+
reason=None, decision=None, iterations=None, cost=None) -> None
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
`end()` is idempotent and is called for you on context-manager exit — with
|
|
74
|
+
`record_error` + an ERROR status when the block raises. The four keyword fields
|
|
75
|
+
are what the product surfaces read: `reason` is how the run terminated,
|
|
76
|
+
`decision` is the outcome the operator cares about, `iterations` is the loop
|
|
77
|
+
count, `cost` is dollars.
|
|
78
|
+
|
|
79
|
+
## Flushing
|
|
80
|
+
|
|
81
|
+
```python
|
|
82
|
+
mainwave.flush(timeout: float = 30.0) -> bool
|
|
83
|
+
await mainwave.aflush(timeout: float = 30.0) -> bool
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
An `atexit` handler is registered automatically, but call `flush()` explicitly
|
|
87
|
+
in scripts and job handlers — an abrupt exit path can outrun `atexit`.
|
|
88
|
+
|
|
89
|
+
## Errors
|
|
90
|
+
|
|
91
|
+
`MainwaveError` (base), `MainwaveConfigError`, `MainwaveReinitConflict`.
|
|
92
|
+
|
|
93
|
+
## Testing helpers
|
|
94
|
+
|
|
95
|
+
```python
|
|
96
|
+
from mainwave import testing
|
|
97
|
+
|
|
98
|
+
exporter = testing.install_in_memory_provider(
|
|
99
|
+
use_case_id="test-ucid", api_key="test-key",
|
|
100
|
+
service_name="test-service", deployment_environment="test",
|
|
101
|
+
)
|
|
102
|
+
testing.reset()
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
`install_in_memory_provider` routes spans through the _production_ export
|
|
106
|
+
pipeline, so assertions see the wire shape the collector receives. Spans become
|
|
107
|
+
visible only once their trace completes — end the root span, exit the
|
|
108
|
+
`mainwave.run(...)` block, or call `mainwave.flush()` before asserting.
|
|
109
|
+
|
|
110
|
+
`reset()` is a full teardown (instrumentors unwound, provider shut down, OTel's
|
|
111
|
+
set-once guard rewound), so init/reset/init cycles work in a test suite. Acquire
|
|
112
|
+
the SDK _inside_ fixtures rather than importing it at module level in
|
|
113
|
+
`conftest.py`, since the reset hook clears global state.
|
|
114
|
+
|
|
115
|
+
## Defaults
|
|
116
|
+
|
|
117
|
+
| Setting | Default |
|
|
118
|
+
| ------------------------ | -------------------------------------------------------- |
|
|
119
|
+
| `collector_url` | `https://collector.mainwave.com` |
|
|
120
|
+
| `service_name` | `mainwave-app` — always override |
|
|
121
|
+
| `deployment_environment` | `production` |
|
|
122
|
+
| `use_case_id` | derived (stable UUIDv5 of key + service name) when unset |
|
|
123
|
+
|
|
124
|
+
## Optional dependency extras
|
|
125
|
+
|
|
126
|
+
`openai`, `anthropic`, `mistralai`, `google-genai`, `groq`, `bedrock`,
|
|
127
|
+
`vertexai`, `litellm`, `langchain`, `crewai`, `llama-index`.
|
|
128
|
+
|
|
129
|
+
`all` exists for CI matrices and example projects. Do not use it in an
|
|
130
|
+
application.
|
|
131
|
+
|
|
132
|
+
## Attributes the SDK emits
|
|
133
|
+
|
|
134
|
+
`mainwave.ucid` (the use case) and `mainwave.ucid_source`, `mainwave.run_id`,
|
|
135
|
+
`gen_ai.agent.name` (the logical agent), and from `Run.end()`:
|
|
136
|
+
`agent.terminated_reason`, `agent.decision`, `agent.iteration_count`,
|
|
137
|
+
`agent.cost`. Anything passed as run `metadata=` or `set_metadata()` lands
|
|
138
|
+
under `mainwave.run.metadata.<key>`.
|
|
139
|
+
|
|
140
|
+
Identity is carried **on the span**, not on the OTel resource. The resource
|
|
141
|
+
holds only `service.name`. This is why a per-run `use_case_id` override works
|
|
142
|
+
and why spans created before a bind do not inherit it.
|
|
@@ -0,0 +1,170 @@
|
|
|
1
|
+
# Per-framework wiring
|
|
2
|
+
|
|
3
|
+
## The default: do nothing
|
|
4
|
+
|
|
5
|
+
For `openai`, `anthropic`, `mistralai`, `google-genai`, `groq`, `bedrock`,
|
|
6
|
+
`vertexai`, `litellm`, and `llama-index`, `mainwave.init()` wires the
|
|
7
|
+
instrumentor at startup. Install the extra, call `init()` early, and the
|
|
8
|
+
framework's calls are captured — no callbacks, no wrappers, no changes at the
|
|
9
|
+
call sites.
|
|
10
|
+
|
|
11
|
+
`init()` wires every instrumentor whose package is **installed**, not every one
|
|
12
|
+
the application actually calls. Those differ more often than you would expect —
|
|
13
|
+
`crewai[anthropic]` puts the Anthropic SDK in the dependency tree even when
|
|
14
|
+
every run goes to OpenAI. The unused instrumentor then stamps its framework name
|
|
15
|
+
onto the telemetry, and the operator reads a provider the app never called.
|
|
16
|
+
|
|
17
|
+
When more than one provider SDK is installed and the app uses one of them,
|
|
18
|
+
suppress the automatic pass and wire what you mean:
|
|
19
|
+
|
|
20
|
+
```python
|
|
21
|
+
mainwave.init(service_name="research-agent",
|
|
22
|
+
skip_instrumentors=["openai", "anthropic"])
|
|
23
|
+
mainwave.instrument("openai")
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
Check it: at INFO the SDK logs `mainwave: instrumented ...` with everything it
|
|
27
|
+
wired. If that line names a provider the app does not call, you need the skip.
|
|
28
|
+
|
|
29
|
+
The other thing that breaks the default is import order. If the LLM library is
|
|
30
|
+
imported after `init()` runs — a lazy import inside a function, a plugin loaded on
|
|
31
|
+
demand, a Click subcommand — the instrumentor had nothing to patch. Wire it
|
|
32
|
+
after the import:
|
|
33
|
+
|
|
34
|
+
```python
|
|
35
|
+
def handler():
|
|
36
|
+
import openai
|
|
37
|
+
mainwave.instrument("openai")
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
`instrument()` is safe to call repeatedly; it only wires what isn't wired.
|
|
41
|
+
|
|
42
|
+
## LangChain — pick exactly one path
|
|
43
|
+
|
|
44
|
+
Both paths capture every call. Wiring both double-counts tokens and cost.
|
|
45
|
+
|
|
46
|
+
**Path A — auto-instrumentation.** Nothing to write:
|
|
47
|
+
|
|
48
|
+
```python
|
|
49
|
+
mainwave.init(service_name="research-agent")
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
**Path B — the explicit handler.** Choose this when you need per-chain identity,
|
|
53
|
+
for example one deployable serving several tenants or use cases:
|
|
54
|
+
|
|
55
|
+
```python
|
|
56
|
+
mainwave.init(service_name="research-agent", skip_instrumentors=["langchain"])
|
|
57
|
+
|
|
58
|
+
from mainwave.langchain import CallbackHandler
|
|
59
|
+
|
|
60
|
+
handler = CallbackHandler(use_case_id=tenant_ucid, agent_name="research")
|
|
61
|
+
chain.invoke(input, config={"callbacks": [handler]})
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
`skip_instrumentors=["langchain"]` is what keeps the two apart. The SDK also
|
|
65
|
+
defends this by construction — building a `CallbackHandler` uninstruments the
|
|
66
|
+
LangChain instrumentor — but declare the intent rather than relying on the
|
|
67
|
+
rescue.
|
|
68
|
+
|
|
69
|
+
The handler parents its spans via LangChain's `parent_run_id`, and
|
|
70
|
+
`mainwave.run()` establishes OTel context, so a turn comes out as one connected
|
|
71
|
+
trace rather than a scatter of roots.
|
|
72
|
+
|
|
73
|
+
`service_name=` on the handler is a deprecated alias for `agent_name=`. In new
|
|
74
|
+
code use `agent_name`.
|
|
75
|
+
|
|
76
|
+
## CrewAI — two instrumentors, and the provider must match
|
|
77
|
+
|
|
78
|
+
CrewAI needs its own structure instrumentor _plus_ the instrumentor for the LLM
|
|
79
|
+
provider the crew actually calls. The `crewai` instrumentor emits the
|
|
80
|
+
crew → agent → task structure and is provider-agnostic; the provider
|
|
81
|
+
instrumentor emits the leaf LLM spans that carry `gen_ai.usage.*`, which is
|
|
82
|
+
where tokens and cost come from. With only the first, every run shows structure
|
|
83
|
+
and no cost.
|
|
84
|
+
|
|
85
|
+
The trap is that the provider is usually **resolved at runtime from config**,
|
|
86
|
+
not written at the call site. Wiring a hardcoded provider that the run does not
|
|
87
|
+
use drops every LLM span silently — the symptom is a run with one span and no
|
|
88
|
+
tokens. Read the provider the app resolves, and map it:
|
|
89
|
+
|
|
90
|
+
```python
|
|
91
|
+
_PROVIDER_INSTRUMENTOR = {"openai": "openai", "ollama": "openai", "anthropic": "anthropic"}
|
|
92
|
+
|
|
93
|
+
mainwave.init(service_name="content-pipeline",
|
|
94
|
+
skip_instrumentors=["crewai", "openai", "anthropic"])
|
|
95
|
+
mainwave.instrument("crewai")
|
|
96
|
+
mainwave.instrument(_PROVIDER_INSTRUMENTOR[resolved_provider])
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
CrewAI routes OpenAI-compatible providers (including Ollama and most local
|
|
100
|
+
gateways) through the OpenAI client, so one `openai` instrumentor covers them.
|
|
101
|
+
Anthropic uses its own SDK and needs its own.
|
|
102
|
+
|
|
103
|
+
`instrument()` returns what actually wired, so check rather than assume:
|
|
104
|
+
|
|
105
|
+
```python
|
|
106
|
+
if not mainwave.instrument("crewai"):
|
|
107
|
+
log.warning("crew structure spans unavailable; install openinference-instrumentation-crewai")
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
If the CrewAI instrumentor cannot wire against the installed CrewAI at all,
|
|
111
|
+
`register_framework("crewai")` keeps the framework visible in telemetry without
|
|
112
|
+
wiring anything — provider spans still carry tokens and cost, and what you lose
|
|
113
|
+
is crew/task structure. Reach for it only when `instrument("crewai")` has
|
|
114
|
+
actually come back empty; recent instrumentor releases wire against CrewAI 1.x
|
|
115
|
+
fine, and taking the fallback pre-emptively throws away the structure for
|
|
116
|
+
nothing.
|
|
117
|
+
|
|
118
|
+
## No framework — hand-rolled LLM calls
|
|
119
|
+
|
|
120
|
+
An application calling an HTTP API directly has nothing to auto-instrument.
|
|
121
|
+
Emit OTel spans through the normal API and the SDK exports them:
|
|
122
|
+
|
|
123
|
+
```python
|
|
124
|
+
from opentelemetry import trace
|
|
125
|
+
tracer = trace.get_tracer("my-app")
|
|
126
|
+
|
|
127
|
+
with mainwave.run(f"triage-{ticket.id}") as r:
|
|
128
|
+
with tracer.start_as_current_span("llm.chat") as span:
|
|
129
|
+
span.set_attribute("gen_ai.provider.name", "openai")
|
|
130
|
+
span.set_attribute("gen_ai.request.model", "gpt-4o-mini")
|
|
131
|
+
span.set_attribute("gen_ai.operation.name", "chat")
|
|
132
|
+
response, in_tok, out_tok = call_the_api(prompt)
|
|
133
|
+
span.set_attribute("gen_ai.usage.input_tokens", in_tok)
|
|
134
|
+
span.set_attribute("gen_ai.usage.output_tokens", out_tok)
|
|
135
|
+
r.end(reason="success")
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
The `gen_ai.*` attributes are what mark the span as LLM activity. Without them
|
|
139
|
+
the trace is dropped — see `troubleshooting.md`.
|
|
140
|
+
|
|
141
|
+
## An app that already uses OpenTelemetry
|
|
142
|
+
|
|
143
|
+
If the application builds its own `TracerProvider`, `init()` detects it and runs
|
|
144
|
+
in **coexist mode**: the SDK attaches its processors to the existing provider
|
|
145
|
+
instead of installing one, and forwards spans under the host's resource
|
|
146
|
+
untouched.
|
|
147
|
+
|
|
148
|
+
Leave their setup alone. Do not remove their exporter, do not replace their
|
|
149
|
+
provider, do not move their `TracerProvider` construction after `init()`. Their
|
|
150
|
+
existing backend keeps receiving everything it received before.
|
|
151
|
+
|
|
152
|
+
Only LLM-bearing traces are forwarded to Mainwave — the host's HTTP, database,
|
|
153
|
+
and background-job traces stay out. That filtering is what makes sharing a
|
|
154
|
+
provider safe.
|
|
155
|
+
|
|
156
|
+
## Async applications
|
|
157
|
+
|
|
158
|
+
`mainwave.arun()` replaces `run()`, `await mainwave.aflush()` replaces
|
|
159
|
+
`flush()`. Instrumentors handle async clients without extra wiring.
|
|
160
|
+
|
|
161
|
+
In a web framework, initialize in the startup or lifespan hook — once per
|
|
162
|
+
process — and open a run per unit of work inside the handler:
|
|
163
|
+
|
|
164
|
+
```python
|
|
165
|
+
@asynccontextmanager
|
|
166
|
+
async def lifespan(app):
|
|
167
|
+
mainwave.init(service_name="support-api")
|
|
168
|
+
yield
|
|
169
|
+
await mainwave.aflush()
|
|
170
|
+
```
|
|
@@ -0,0 +1,131 @@
|
|
|
1
|
+
# No spans are showing up
|
|
2
|
+
|
|
3
|
+
Work these in order. The first cause is the most common and is deliberate
|
|
4
|
+
behavior, not a bug.
|
|
5
|
+
|
|
6
|
+
First, turn on the SDK's own logging — it announces what it wired, whether it
|
|
7
|
+
went into coexist mode, and whether it derived a use-case id:
|
|
8
|
+
|
|
9
|
+
```python
|
|
10
|
+
import logging
|
|
11
|
+
logging.basicConfig(level=logging.INFO)
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
## 1. The trace had no LLM activity, so it was dropped whole
|
|
15
|
+
|
|
16
|
+
The exporter buffers by trace and, at finalize, **drops the entire trace unless
|
|
17
|
+
something in it shows LLM activity** — an SDK span (a `mainwave.run` wrapper or
|
|
18
|
+
the `mainwave.*` scope), an OpenInference instrumentor span, or any span
|
|
19
|
+
carrying `gen_ai.*` / `llm.*` attributes.
|
|
20
|
+
|
|
21
|
+
This is what lets the SDK share a host's `TracerProvider` without leaking the
|
|
22
|
+
host's HTTP, database, and background traces to Mainwave.
|
|
23
|
+
|
|
24
|
+
So a hand-rolled span with no `gen_ai.*` attributes, emitted outside any run,
|
|
25
|
+
vanishes silently and correctly. Fix it by giving the trace a reason to exist:
|
|
26
|
+
wrap the work in `mainwave.run(...)`, or put the real `gen_ai.*` attributes on
|
|
27
|
+
the span.
|
|
28
|
+
|
|
29
|
+
## 2. `init()` didn't run, or ran too late
|
|
30
|
+
|
|
31
|
+
- Check the log line for the wired instrumentors. Nothing wired means nothing
|
|
32
|
+
was patched.
|
|
33
|
+
- `MAINWAVE_DISABLED` set truthy anywhere in the environment turns the SDK into
|
|
34
|
+
a no-op that logs and returns. Check the deployment's env, not just the shell.
|
|
35
|
+
- In a web app, confirm `init()` is in the startup hook and that the hook
|
|
36
|
+
actually runs under the process manager in use.
|
|
37
|
+
|
|
38
|
+
## 3. The process exited before the batch drained
|
|
39
|
+
|
|
40
|
+
Scripts, cron jobs, serverless handlers, and CLI commands end fast enough to
|
|
41
|
+
outrun the exporter. Add `mainwave.flush()` — or `await mainwave.aflush()` — on
|
|
42
|
+
every exit path, including the error path.
|
|
43
|
+
|
|
44
|
+
Symptom that confirms it: spans appear when you add a `sleep` before exit.
|
|
45
|
+
|
|
46
|
+
## 4. The library was imported after `init()`
|
|
47
|
+
|
|
48
|
+
Instrumentors patch at wire time. A lazy import — inside a function, a Click
|
|
49
|
+
subcommand, a plugin loader, a FastAPI route — was not importable when `init()`
|
|
50
|
+
walked the registry, so it was skipped.
|
|
51
|
+
|
|
52
|
+
`mainwave.instrument("openai")` after the import fixes it. The return value
|
|
53
|
+
tells you what actually wired; an empty list means the package still isn't
|
|
54
|
+
importable or its version conflicts with the instrumentor.
|
|
55
|
+
|
|
56
|
+
## 5. Spans are landing, but not in the use case you're watching
|
|
57
|
+
|
|
58
|
+
- If `MAINWAVE_USE_CASE_ID` was unset, the SDK derived a stable id from the API
|
|
59
|
+
key and service name and logged it at INFO. Telemetry is arriving — into a use
|
|
60
|
+
case nobody created. Set the id explicitly.
|
|
61
|
+
- Identity binds at the root of a unit of work. **Spans created before the bind
|
|
62
|
+
do not inherit it**, so work started outside `mainwave.run(...)` /
|
|
63
|
+
`bind_identity(...)` carries the process default instead of the override.
|
|
64
|
+
- Verify against the id the code is actually sending, not the one you expected:
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
npx @mainwave/cli@latest verify --ucid <use-case-id> --no-sample
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
## 6. One span per run, and no tokens or cost
|
|
71
|
+
|
|
72
|
+
Structure spans arrived; the leaf LLM spans did not. The instrumentor for the
|
|
73
|
+
provider the run actually used was never wired — commonly because the provider
|
|
74
|
+
is resolved from config at runtime and the wiring names a different one.
|
|
75
|
+
|
|
76
|
+
Check the `mainwave: instrumented ...` line at INFO against the provider the run
|
|
77
|
+
resolved. See `frameworks.md` § CrewAI, which is where this bites hardest.
|
|
78
|
+
|
|
79
|
+
## 7. Telemetry names a framework the app never calls
|
|
80
|
+
|
|
81
|
+
`init()` wires every instrumentor whose package is **installed**, which is not
|
|
82
|
+
the same set as the frameworks the app calls — a provider SDK pulled in as a
|
|
83
|
+
transitive dependency gets wired and stamped like any other.
|
|
84
|
+
|
|
85
|
+
Suppress the automatic pass and wire deliberately:
|
|
86
|
+
|
|
87
|
+
```python
|
|
88
|
+
mainwave.init(service_name="...", skip_instrumentors=["openai", "anthropic"])
|
|
89
|
+
mainwave.instrument("openai")
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
## 8. `MainwaveReinitConflict` at startup
|
|
93
|
+
|
|
94
|
+
`init()` was called twice with different settings in one process — commonly once
|
|
95
|
+
at import and once in a startup hook, or once per worker in a pre-fork server.
|
|
96
|
+
Initialize in exactly one place. Calling it again with _identical_ config is
|
|
97
|
+
fine and does nothing.
|
|
98
|
+
|
|
99
|
+
## 9. Tokens and cost are roughly double
|
|
100
|
+
|
|
101
|
+
LangChain wired both ways: the auto-instrumentor and a `CallbackHandler`. Pick
|
|
102
|
+
one path — see `frameworks.md`.
|
|
103
|
+
|
|
104
|
+
## 10. Nothing is wrong locally, so check the wire
|
|
105
|
+
|
|
106
|
+
- The API key must belong to the same organization as the use case.
|
|
107
|
+
- `MAINWAVE_COLLECTOR_URL` defaults to `https://collector.mainwave.com`; an
|
|
108
|
+
override pointing at a local or stale collector sends telemetry somewhere real
|
|
109
|
+
but wrong.
|
|
110
|
+
- An egress-restricted network needs that host allowed. The SDK will not crash
|
|
111
|
+
the app when it can't reach the collector — it logs and drops, which looks
|
|
112
|
+
identical to "no instrumentation" from the outside.
|
|
113
|
+
|
|
114
|
+
## Proving instrumentation works without the network
|
|
115
|
+
|
|
116
|
+
Assert on spans in-process, which isolates your wiring from every deployment
|
|
117
|
+
question above:
|
|
118
|
+
|
|
119
|
+
```python
|
|
120
|
+
from mainwave import testing
|
|
121
|
+
|
|
122
|
+
def test_agent_emits_a_run():
|
|
123
|
+
exporter = testing.install_in_memory_provider()
|
|
124
|
+
run_the_agent()
|
|
125
|
+
mainwave.flush()
|
|
126
|
+
names = [s.name for s in exporter.get_finished_spans()]
|
|
127
|
+
assert names
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
Spans only appear once their trace completes, so end the run (or flush) before
|
|
131
|
+
asserting. Call `testing.reset()` between tests.
|