@cursor/bdk 0.2.24 → 0.2.26

package/docs/guides/opentelemetry.md

@@ -1,23 +1,11 @@
----
-title: "OpenTelemetry"
-description: "Send agent traces and metrics to an OTLP collector, follow one conversation, and add domain telemetry."
----
+# OpenTelemetry
 
-# Observe an agent with OpenTelemetry
-
-OpenTelemetry connects an agent turn to the observability system your
-team already uses. Once you configure an OTLP collector, each request,
-model turn, tool call, and subagent appears alongside token, tool, and
-cost metrics from the serve process.
-
-Export is opt-in. Nothing leaves the process until you configure a
-destination.
-
-## Connect an OTLP collector
-
-Set the collector endpoint, start the agent, and send it a message. You
-should see an `agent_sdk.http` request span and an `agent_sdk.turn`
-trace, with any tool or subagent work nested under the turn.
+BDK exports traces, metrics, and logs to an OTLP collector. You
+configure that with standard `OTEL_*` environment variables. There is
+no `defineOtel` or `register()` hook. `ctx.host.otel` is deprecated.
+It still runs for this release and logs one warning per process. Set
+`OTEL_*` variables and record with `metrics.getMeter` and
+`trace.getTracer` from `@opentelemetry/api`.
 
 ```bash
 export OTEL_EXPORTER_OTLP_ENDPOINT=https://otel.example.com
@@ -27,209 +15,191 @@ export OTEL_SERVICE_NAME=incident-agent
 bdk serve --dir . --dev
 ```
 
-The default protocol is OTLP/HTTP protobuf. If you already call
-`serve()` from TypeScript, you can configure the same destination in
-code:
+The default protocol is OTLP/HTTP protobuf. Set
+`OTEL_EXPORTER_OTLP_PROTOCOL=http/json` when the collector only accepts
+JSON. `grpc` is not supported. BDK logs a warning and uses
+`http/protobuf`.
 
-```ts
-import { serve } from "@cursor/bdk";
+Metrics use delta temporality by default. BDK logs that default when
+export starts. For Grafana Cloud or Prometheus, set
+`OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE` to `cumulative`.
+Grafana Cloud rejects delta metrics.
 
-await serve(".", {
-  otel: {
-    serviceName: "incident-agent",
-    exporters: [
-      {
-        url: "https://otel.example.com",
-        headers: {
-          Authorization: `Bearer ${process.env.OTEL_TOKEN ?? ""}`,
-        },
-      },
-    ],
-  },
-});
-```
+BDK installs trace, metric, and log providers before it imports
+`bot/agent.ts`. A meter, tracer, or logger created at module scope
+binds to those providers. Exporters attach when the process has the
+`OTEL_*` variables. On a hosted agent that is the first request, after
+secrets are in the environment.
 
-Use environment variables when you start the agent with
-`bdk serve`. Use `serve({ otel })` when your TypeScript
-application starts the server itself. When that application
-already uses OpenTelemetry, BDK sends its traces, metrics, and
-logs to the collector configured here.
+With no OTLP endpoint, BDK adds no exporters. Outbound `fetch` does
+not gain a `traceparent` header. The session collector stays disabled.
 
-## Export from a hosted agent
+## Variables
 
-Set the collector endpoint and headers as secrets, and add the
+| Variable | Effect |
+| --- | --- |
+| `OTEL_EXPORTER_OTLP_ENDPOINT` | Base URL. BDK appends `/v1/traces`, `/v1/metrics`, and `/v1/logs`. |
+| `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` | Traces URL, used as written. It wins over the base URL. |
+| `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` | Metrics URL, used as written. It wins over the base URL. |
+| `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT` | Logs URL, used as written. It wins over the base URL. |
+| `OTEL_EXPORTER_OTLP_HEADERS` | Headers on every export. Values are URL-encoded. Read again on each export. |
+| `OTEL_EXPORTER_OTLP_PROTOCOL` | `http/protobuf` (default) or `http/json`. gRPC is not supported. |
+| `OTEL_SERVICE_NAME` | `service.name`. Default is the agent name, then `unknown_service:node`. |
+| `OTEL_RESOURCE_ATTRIBUTES` | Resource attributes, including `service.namespace`, `service.version`, and `deployment.environment.name`. |
+| `OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE` | `delta` (default), `cumulative`, or `lowmemory`. Use `cumulative` for Grafana Cloud. |
+| `OTEL_LOGS_EXPORTER` | `otlp` turns logs on. Unset, `none`, and `console` leave OTLP logs off. |
+| `OTEL_SDK_DISABLED` | `true` turns export off. |
+| `OTEL_TRACES_EXPORTER` | `none` turns traces off. |
+| `OTEL_METRICS_EXPORTER` | `none` turns metrics off. |
+| `OTEL_LOG_USER_PROMPTS` | `1` puts prompt text on log records. |
+| `OTEL_LOG_TOOL_CONTENT` | `1` puts tool payloads on log records. |
+
+## Hosted agents
+
+Set the endpoint, headers, and service name as secrets. Add the
 collector host to `hosting.egressDomains`. The host is a hostname
-with no scheme and no path. The first request loads the secrets,
-and export starts with those headers.
+with no scheme and no path.
 
 ```ts
-// bot/agent.ts
-import { defineAgent } from "@cursor/bdk";
-
 export default defineAgent({
   hosting: {
     egressDomains: ["otlp-gateway-prod-us-east-0.grafana.net"],
-    secretNames: ["OTEL_EXPORTER_OTLP_ENDPOINT", "OTEL_EXPORTER_OTLP_HEADERS"],
+    secretNames: [
+      "OTEL_EXPORTER_OTLP_ENDPOINT",
+      "OTEL_EXPORTER_OTLP_HEADERS",
+      "OTEL_SERVICE_NAME",
+    ],
   },
 });
 ```
 
-```bash
-bdk secrets set my-agent OTEL_EXPORTER_OTLP_ENDPOINT OTEL_EXPORTER_OTLP_HEADERS
-```
+Secrets arrive with the first request that has an endpoint. That
+request's export uses those headers and `service.name`. A later
+`bdk secrets set` updates headers on the next export, including a
+rotated key. BDK also replaces the exporter when the URL, protocol,
+or signal list changes. `service.name` is fixed on the first export
+that attaches. Redeploy to change the service name.
 
-`OTEL_EXPORTER_OTLP_HEADERS` is a comma-separated list of
-`name=value` pairs. Each value is URL-encoded, so a space is
-`%20` (`Authorization=Basic%20...`).
+## What you can record
 
-A later `bdk secrets set` changes the headers on the next export.
-You do not redeploy to rotate a token. The URL, service name,
-resource attributes, signals, sample ratio, and metric export
-interval stay as they were on the first export. Redeploy to change
-one of those. BDK logs a warning when one of them changes on a
-running agent.
+Create the tracer, meter, and logger at module scope.
 
-When an exporter host is missing from `hosting.egressDomains`, BDK
-logs an error that names the host. Add the host and redeploy.
+```ts
+import { metrics, trace } from "@opentelemetry/api";
+import { logs } from "@opentelemetry/api-logs";
 
-Spans and metrics from before the first request are not exported,
-including spans from channel `onStart`. If the OpenTelemetry config
-fails, BDK logs the error and the request still completes.
+const tracer = trace.getTracer("incident-agent");
+const lookups = metrics.getMeter("incident-agent").createCounter("incident.lookups");
+const logger = logs.getLogger("incident-agent");
+```
 
-Logs stay off until `serve({ otel })` sets `logs`, or until
-`OTEL_LOGS_EXPORTER` is `otlp`. Prompt text and tool payloads stay
-off until you set `OTEL_LOG_USER_PROMPTS` or
-`OTEL_LOG_TOOL_CONTENT`.
+Record during a request, after secrets have loaded. A span or a log
+record that ends before the exporters attach is still exported. BDK
+holds up to 2048 spans and 2048 log records until the exporters
+attach, then sends them. BDK drops each record past that limit. A
+metric recorded before the first collect is included in that collect.
 
-## Follow one conversation
+BDK also records HTTP spans, token counts, and tool calls on the
+session. Those spans carry `cursor.conversation.id`. A span you start
+with `trace.getTracer` during a turn or a tool call gets the same
+attribute from the active context. A value you set on the span stays.
+BDK does not overwrite it. The id stays in this process. It is not
+propagated.
 
-Copy the complete session ID from the playground or session API, then
-filter your observability system on either `agent_sdk.session_id` or
-`cursor.conversation.id`. Every turn in that conversation carries the
-same value, including follow-ups and tool work.
+A metric does not get `cursor.conversation.id`. Read the id and set
+the attribute yourself.
 
-The trace shape stays small:
+```ts
+import { metrics } from "@opentelemetry/api";
+import { currentConversationId } from "@cursor/bdk/otel";
 
-```text
-request: agent_sdk.http
+const lookups = metrics.getMeter("incident-agent").createCounter("incident.lookups");
 
-turn: agent_sdk.turn
-      |- agent_sdk.tool
-      `- agent_sdk.subagent
-         `- agent_sdk.tool
+lookups.add(1, { "cursor.conversation.id": currentConversationId() });
 ```
 
-A conversation can produce several turn traces, so use the session ID
-as the join key instead of expecting one long-lived session span.
-
-### Subagents
+Prompt text and tool payloads stay off log records unless logs are on
+(`OTEL_LOGS_EXPORTER=otlp`) and you set `OTEL_LOG_USER_PROMPTS` or
+`OTEL_LOG_TOOL_CONTENT`.
 
-A subagent's tool calls are `agent_sdk.tool` spans under its
-`agent_sdk.subagent` span. Its `reasoning.completed` and
-`message.completed` events sit on that span too, so the subagent span
-shows where its time went. Time between two tool spans is the model
-choosing the next step. A subagent that fails or is cancelled ends its
-span with ERROR status when it stops.
+## Limits
 
-### Computer use
+These are not supported yet.
 
-Each computer use action is an `agent_sdk.tool` span with
-`cursor.tool.name` set to `computerUse`, under the computer use
-subagent's span. A failed action has ERROR status. These attributes
-describe the action without typed text, coordinates, or screenshots:
+- Custom span processors, samplers, and views.
+- Bringing your own OpenTelemetry SDK.
+- Library auto-instrumentation.
+- A `register()` hook. There is no ticket for it yet.
+- `OTEL_TRACES_SAMPLER` and `OTEL_METRIC_EXPORT_INTERVAL` that arrive
+  only with hosted secrets. BDK reads them when it builds the provider,
+  which is before that secret load. Set them on the process, then
+  restart. BDK logs a warning when a later value differs.
 
-| Attribute | Value |
-| --- | --- |
-| `agent_sdk.computer.actions` | The action type, such as `click`, `type`, or `screenshot` |
-| `agent_sdk.computer.executor_duration_ms` | Time the desktop took to run the action and capture the screen |
-| `agent_sdk.computer.settle_ms` | Part of that time spent waiting for the screen to settle |
+Do not import your own SDK providers next to BDK's. BDK logs a warning
+when a module replaces a provider BDK registered. The global API then
+uses that module's provider. BDK shuts down a provider it created
+only when that provider is not the one it registered.
 
-The two durations appear only on actions that succeed.
+## Migrating from serve({ otel })
 
-To export the full actions, turn on tool content as shown in
-[Control what leaves the process](#control-what-leaves-the-process).
-Tool content includes the text the agent typed, which can contain
-passwords.
+`serve({ otel })` is removed. Delete the option. Set the same
+values with `OTEL_*` variables. Delete `bot/otel.ts`,
+`agent/otel.ts`, and `bot/instrumentation.ts`. Do not call
+`defineOtel`.
 
-## Add domain telemetry
+BDK sends each signal to one URL. A second exporter for the same
+signal has no variable.
 
-Use `ctx.host.otel` when your dashboard should answer questions the
-built-in telemetry cannot, such as which incident a turn handled or how
-long approval took. Attributes appear on the active turn and tool
-spans, while counters and histograms give you metrics to graph and
-alert on.
+| Old option | Set this instead |
+| --- | --- |
+| `serviceName` | `OTEL_SERVICE_NAME` |
+| `serviceVersion` | `service.version` in `OTEL_RESOURCE_ATTRIBUTES` |
+| `resourceAttributes` | `OTEL_RESOURCE_ATTRIBUTES`. Separate pairs with commas. URL-encode each value. |
+| `exporters[].url` | `OTEL_EXPORTER_OTLP_ENDPOINT`. BDK appends `/v1/traces`, `/v1/metrics`, and `/v1/logs`. |
+| `exporters[].url` with `signals` | `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`, `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT`, or `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT`. BDK uses that URL as written. |
+| `exporters[].headers` | `OTEL_EXPORTER_OTLP_HEADERS`. Separate pairs with commas. URL-encode each value. |
+| `exporters[].protocol` | `OTEL_EXPORTER_OTLP_PROTOCOL`. Use `http/protobuf` or `http/json`. For one signal, use `OTEL_EXPORTER_OTLP_TRACES_PROTOCOL`, `OTEL_EXPORTER_OTLP_METRICS_PROTOCOL`, or `OTEL_EXPORTER_OTLP_LOGS_PROTOCOL`. |
+| `exporters[].signals` | Set the signal URL above. Set `OTEL_TRACES_EXPORTER`, `OTEL_METRICS_EXPORTER`, or `OTEL_LOGS_EXPORTER` to `none` to stop that signal. |
+| `traces: false` | `OTEL_TRACES_EXPORTER=none` |
+| `traces: true` | Leave `OTEL_TRACES_EXPORTER` unset. Traces start when an endpoint is set. |
+| `traces.sampleRatio` | `OTEL_TRACES_SAMPLER=parentbased_traceidratio` and `OTEL_TRACES_SAMPLER_ARG` set to the ratio. Set both before the process starts. |
+| `metrics: false` | `OTEL_METRICS_EXPORTER=none` |
+| `metrics: true` | Leave `OTEL_METRICS_EXPORTER` unset. Metrics start when a metrics URL is set. |
+| `metrics.exportIntervalMillis` | `OTEL_METRIC_EXPORT_INTERVAL`, in milliseconds. Set it before the process starts. |
+| `logs: false` | Leave `OTEL_LOGS_EXPORTER` unset, or set it to `none`. |
+| `logs: true` | `OTEL_LOGS_EXPORTER=otlp` |
+| `logs.userPrompts` | `OTEL_LOGS_EXPORTER=otlp` and `OTEL_LOG_USER_PROMPTS=1` |
+| `logs.toolContent` | `OTEL_LOGS_EXPORTER=otlp` and `OTEL_LOG_TOOL_CONTENT=1` |
+| `otel: false` | Set no endpoint, or set `OTEL_SDK_DISABLED=true`. |
+
+Before:
 
 ```ts
-ctx.host.otel.setAttributes({
-  "support.ticket_id": "INC-123",
-  "support.queue": "p1",
-});
-
-ctx.host.otel.increment("support.ticket.resolved", {
-  outcome: "fixed",
-});
-
-ctx.host.otel.record("support.approval.duration_ms", 1420, {
-  outcome: "approved",
+await serve(".", {
+  otel: {
+    serviceName: "incident-agent",
+    exporters: [
+      {
+        url: "https://otel.example.com",
+        headers: { Authorization: `Bearer ${process.env.OTEL_TOKEN ?? ""}` },
+      },
+    ],
+  },
 });
 ```
 
-This API is available in tools, hooks, and channel handlers. A
-session-bound context adds `cursor.conversation.id` and
-`agent_sdk.agent` automatically, so custom measurements join the
-first-party telemetry without extra plumbing.
-
-Prefix custom names with your team or agent. First-party metric names
-and join keys are reserved, and this surface does not create custom
-spans.
-
-## Choose signals
-
-Once export has a destination, traces and metrics are on by default;
-logs remain off until you opt in.
-
-| Signal | What appears |
-| --- | --- |
-| Traces | HTTP requests, turns, tools, and subagents |
-| Metrics | Token usage, tool calls, cost, and BDK session or turn counts |
-| Logs | Session events, after you enable log export |
-
-The first metrics to look for are `cursor.token.usage`,
-`cursor.tool.calls`, and `cursor.cost.usage`. Configure
-`traces: false`, `metrics: false`, or `logs: true` under
-`serve({ otel })` when you need a different signal mix.
-
-## Control what leaves the process
-
-Traces include structure, tool names, usage metrics, and span
-status. A failed tool's status includes a short failure reason.
-A tool or subagent that is still open when the turn ends gets
-ERROR status with the reason `no result`. User prompts, tool
-arguments, and full tool results stay off the wire until you opt in.
-
-Turn on logs first, then opt into prompt or tool content separately:
+After:
 
 ```bash
-export OTEL_LOGS_EXPORTER=otlp
-export OTEL_LOG_USER_PROMPTS=1
-export OTEL_LOG_TOOL_CONTENT=1
+export OTEL_SERVICE_NAME=incident-agent
+export OTEL_EXPORTER_OTLP_ENDPOINT=https://otel.example.com
+export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer%20$OTEL_TOKEN"
+bdk serve --dir . --dev
 ```
 
-`OTEL_LOG_USER_PROMPTS` exports user text. `OTEL_LOG_TOOL_CONTENT`
-exports tool inputs, outputs, and failure details. Enable either only
-when the collector's access and retention policies are appropriate for
-that data.
-
-The same controls are available as
-`logs: { userPrompts, toolContent }` under `serve({ otel })`.
+On a hosted agent, put the same names in `hosting.secretNames`. Add
+the collector host to `hosting.egressDomains`. Then run:
 
-## Related
-
-- [`skills/otel/SKILL.md`](../../skills/otel/SKILL.md): compact
-  configuration and custom-metric reference for coding agents
-- [Hooks](../reference/hooks.md): emit domain telemetry from session
-  events
-- [Deployment](../deployment.md): declare `hosting.egressDomains`
-  and set collector secrets on a hosted agent
-- [Cursor enterprise OpenTelemetry Export](https://cursor.com/docs/enterprise/opentelemetry-export):
-  send organization-wide Cursor telemetry to the same collector
+```bash
+bdk secrets set my-agent OTEL_SERVICE_NAME OTEL_EXPORTER_OTLP_ENDPOINT OTEL_EXPORTER_OTLP_HEADERS
+```