@mastra/mcp-docs-server 0.0.0-error-handler-fix-20251020202607 → 0.0.0-esbuild-bundle-worker-20260807182016

package/.docs/docs/observability/integrations/bridges/datadog.md

@@ -0,0 +1,219 @@
+> Discover all available pages from the documentation index: https://mastra.ai/llms.txt
+
+# Datadog bridge
+
+> **Warning:** The Datadog Bridge is currently **experimental**. APIs and configuration options may change in future releases.
+
+The Datadog Bridge enables bidirectional integration between Mastra's tracing system and Datadog. Unlike exporters that send trace data after execution completes, the bridge creates native dd-trace spans in real time so that auto-instrumented APM operations (HTTP calls, database queries, etc.) inside your tools and processors are correctly nested under their parent Mastra spans.
+
+> **Not using dd-trace APM?:** If you only need to send LLM Observability data and don't use `dd-trace` APM auto-instrumentation, the [Datadog Exporter](https://mastra.ai/docs/observability/integrations/exporters/datadog) is simpler, it supports agentless mode and sends spans directly to Datadog without a local agent.
+
+## When to use the bridge
+
+Use the DatadogBridge when you:
+
+- Use `dd-trace` auto-instrumentation in your application (HTTP servers, database clients, etc.)
+- Want APM service calls made by tools, MCP tools, or output processors to appear under their parent Mastra span instead of the request handler
+- Need both APM traces and LLM Observability data to share a consistent trace topology
+- Are building a distributed system where Datadog trace context must propagate across services
+
+## How it works
+
+The DatadogBridge participates in two parts of the dd-trace pipeline:
+
+**APM context propagation (real time):**
+
+- Creates a dd-trace APM span via `tracer.startSpan()` when each Mastra span is created
+- Activates the APM span in dd-trace's scope via `tracer.scope().activate()` during execution
+- Auto-instrumented operations inside the active scope are parented to the correct Mastra span
+- Inherits the active dd-trace context (e.g., an incoming request span) when no explicit Mastra parent exists
+
+**LLM Observability emission (on span end):**
+
+- Emits annotations (model info, token usage, input/output, errors) through `dd-trace`'s LLM Observability pipeline
+- Maintains parent-child relationships in Datadog LLM Observability using nested `llmobs.trace()` calls
+- Reuses the same data shape and span-kind mapping as the [Datadog Exporter](https://mastra.ai/docs/observability/integrations/exporters/datadog)
+
+## Trace and log correlation
+
+Without the bridge, the Datadog Exporter only creates LLM Observability spans after a trace completes. During execution, no `dd-trace` span is active in scope, so any HTTP or database call made by a tool falls back to whatever `dd-trace` span is active at the time, typically the incoming request handler. The result is that service calls from MCP tools or output processors appear as children of the request span instead of the agent or processor span that actually made them.
+
+The bridge fixes this by creating real dd-trace spans up front, so the scope is correct when auto-instrumentation runs.
+
+## Installation
+
+**npm**:
+
+```bash
+npm install @mastra/datadog dd-trace
+```
+
+**pnpm**:
+
+```bash
+pnpm add @mastra/datadog dd-trace
+```
+
+**Yarn**:
+
+```bash
+yarn add @mastra/datadog dd-trace
+```
+
+**Bun**:
+
+```bash
+bun add @mastra/datadog dd-trace
+```
+
+The bridge requires `dd-trace` to be installed and a local Datadog Agent (or compatible OTLP receiver) to receive APM data. See the [APM prerequisites](https://mastra.ai/docs/observability/integrations/exporters/datadog) on the exporter page for agent setup details.
+
+## Configuration
+
+Using the DatadogBridge requires two steps:
+
+1. Initialize `dd-trace` so its auto-instrumentation patches HTTP, database, and framework libraries
+2. Add the DatadogBridge to your Mastra observability config
+
+### Step 1: Initialize dd-trace
+
+`dd-trace` must be initialized before any other imports so its auto-instrumentation can patch libraries at load time. The bridge will detect an already-initialized tracer and reuse it.
+
+```typescript
+import tracer from 'dd-trace'
+
+tracer.init({
+  service: process.env.DD_SERVICE || 'my-mastra-app',
+  env: process.env.DD_ENV || 'production',
+  version: process.env.DD_VERSION,
+})
+
+import { Mastra } from '@mastra/core'
+import { Observability } from '@mastra/observability'
+import { DatadogBridge } from '@mastra/datadog'
+
+// ...
+```
+
+> **Note:** Import and initialize `dd-trace` at the very top of your application's entry file, before any other imports.
+
+### Step 2: Mastra Configuration
+
+Add the DatadogBridge to your Mastra observability config:
+
+```typescript
+export const mastra = new Mastra({
+  observability: new Observability({
+    configs: {
+      default: {
+        serviceName: 'my-mastra-app',
+        bridge: new DatadogBridge({
+          mlApp: process.env.DD_LLMOBS_ML_APP!,
+        }),
+      },
+    },
+  }),
+  bundler: {
+    externals: [
+      'dd-trace',
+      '@datadog/native-metrics',
+      '@datadog/native-appsec',
+      '@datadog/native-iast-taint-tracking',
+      '@datadog/pprof',
+    ],
+  },
+})
+```
+
+```bash
+DD_SERVICE=my-mastra-app
+DD_ENV=production
+DD_VERSION=1.0.0
+DD_LLMOBS_ML_APP=my-llm-app
+```
+
+When `dd-trace` is initialized, it routes APM data to your local Datadog Agent on `localhost:8126`. The bridge enables LLM Observability on top of the same tracer, so both sets of data appear under the same service in Datadog.
+
+No Mastra exporters are required when using the bridge, both APM and LLM Observability data flow through `dd-trace`. You can still add Mastra exporters if you want to send traces to additional destinations.
+
+## Agent vs. agentless mode
+
+The bridge defaults to **agent mode** (`agentless: false`). This assumes a local Datadog Agent is running on `localhost:8126` to receive both APM and LLM Observability data. This is the typical setup when using `dd-trace` auto-instrumentation, since APM data always routes through the agent.
+
+If you don't have a local Datadog Agent and only need LLM Observability data (no APM auto-instrumentation), you can enable agentless mode to send data directly to Datadog. In this case, you must provide an API key.
+
+```typescript
+new DatadogBridge({
+  mlApp: process.env.DD_LLMOBS_ML_APP!,
+  apiKey: process.env.DD_API_KEY!,
+  agentless: true,
+})
+```
+
+> **Note:** For most bridge users, agent mode is the right choice. APM data can't be sent in agentless mode, so enabling agentless splits LLM Observability traffic away from APM traffic. If you want LLM Observability only without an agent, use the [Datadog Exporter](https://mastra.ai/docs/observability/integrations/exporters/datadog) instead.
+
+## Trace hierarchy
+
+With the DatadogBridge, your traces maintain proper hierarchy across dd-trace and Mastra boundaries. Service calls made by tools and processors appear under the correct Mastra span:
+
+```text
+HTTP POST /api/chat (from web framework instrumentation)
+└── agent.orchestrator (from Mastra via DatadogBridge)
+    ├── chat gpt-5.4 (LLM call)
+    ├── tool.execute search (tool execution)
+    │   └── HTTP GET api.example.com (auto-instrumented from inside the tool)
+    └── processor.guardrail (output processor)
+        └── HTTP POST guardrail-service/check (auto-instrumented from inside the processor)
+```
+
+In Datadog, the APM trace shows this full topology, and the LLM Observability product shows the agent and LLM-specific spans with their inputs, outputs, and token metrics.
+
+## Span type mapping
+
+The bridge uses the same span-kind mapping as the Datadog Exporter for LLM Observability. See [span type mapping](https://mastra.ai/docs/observability/integrations/exporters/datadog) on the exporter page.
+
+## Using tags
+
+Tags help you categorize and filter traces in Datadog. Add tags when executing agents or workflows:
+
+```typescript
+const result = await agent.generate('Hello', {
+  tracingOptions: {
+    tags: ['production', 'experiment-v2', 'user-request'],
+  },
+})
+```
+
+Tags formatted as `key:value` (e.g., `instance_name:career-scout-api`) are split into structured tag entries. Tags without a colon are set with a `true` value.
+
+## Promoting context keys to flat tags
+
+Use `requestContextKeys` to promote specific keys from the request context or span attributes into flat, indexable LLM Observability tags. This makes them filterable in the Datadog UI:
+
+```typescript
+new DatadogBridge({
+  mlApp: process.env.DD_LLMOBS_ML_APP!,
+  requestContextKeys: ['tenantId', 'agentId'],
+})
+```
+
+Promoted keys are removed from `annotations.metadata` and added as flat tags on each LLM Observability span.
+
+## Troubleshooting
+
+If APM spans aren't connecting to Mastra spans as expected:
+
+- Verify `dd-trace` is initialized **before** any other imports (it patches libraries at load time)
+- Verify a local Datadog Agent is running and reachable at `localhost:8126`
+- Ensure the DatadogBridge is set as `bridge` (not as an entry in `exporters`) in your observability config
+- Confirm you haven't also added the `DatadogExporter` to `exporters`: using both will double-emit LLM Observability data
+
+For native-module compatibility issues with `dd-trace` and bundler externals, see the [Datadog exporter troubleshooting](https://mastra.ai/docs/observability/integrations/exporters/datadog) section.
+
+## Related
+
+- [Tracing Overview](https://mastra.ai/docs/observability/tracing/overview)
+- [Datadog Exporter](https://mastra.ai/docs/observability/integrations/exporters/datadog) - LLM Observability only, no `dd-trace` APM
+- [DatadogBridge Reference](https://mastra.ai/reference/observability/tracing/bridges/datadog) - API documentation
+- [Datadog APM documentation](https://docs.datadoghq.com/tracing/)
+- [Datadog LLM Observability documentation](https://docs.datadoghq.com/llm_observability/)

package/.docs/docs/observability/integrations/bridges/otel.md

@@ -0,0 +1,234 @@
+> Discover all available pages from the documentation index: https://mastra.ai/llms.txt
+
+# OpenTelemetry bridge
+
+> **Warning:** The OpenTelemetry Bridge is currently **experimental**. APIs and configuration options may change in future releases.
+
+The OpenTelemetry (OTEL) Bridge enables bidirectional integration between Mastra's tracing system and existing OpenTelemetry infrastructure. Unlike exporters that send trace data to external platforms, the bridge creates native OTEL spans that participate in your distributed tracing context.
+
+> **Looking to send traces without existing OTEL infrastructure?:** If you don't have existing OpenTelemetry instrumentation, the [OpenTelemetry Exporter](https://mastra.ai/docs/observability/integrations/exporters/otel) may be simpler, it sends traces directly without requiring an OTEL SDK setup.
+
+## When to use the bridge
+
+Use the OtelBridge when you:
+
+- Have existing OTEL instrumentation in your application (HTTP servers, database clients, etc.)
+- Want Mastra operations to appear as child spans of your existing OTEL traces
+- Need OTEL-instrumented code inside Mastra tools to maintain proper parent-child relationships
+- Are building a distributed system where trace context must propagate across services
+
+## How it works
+
+The OtelBridge provides two-way integration:
+
+**From OTEL to Mastra:**
+
+- Reads from OTEL ambient context (AsyncLocalStorage) automatically
+- Inherits trace ID and parent span ID from active OTEL spans
+- Respects OTEL sampling decisions: if a trace isn't sampled, Mastra won't create spans for it
+- No manual trace ID passing required when OTEL auto-instrumentation is active
+
+**From Mastra to OTEL:**
+
+- Creates native OTEL spans for Mastra operations (agents, LLM calls, tools, workflows)
+- Maintains proper parent-child relationships in distributed traces
+- Allows OTEL-instrumented code (HTTP clients, database calls) within Mastra operations to nest correctly
+- Forwards Mastra log events to the globally-registered OTEL `LoggerProvider`. Logs that originate inside a Mastra span are emitted under that span's OTEL context so backends correlate them with the trace. If no `LoggerProvider` is registered, log emission is a silent no-op.
+
+## Installation
+
+**npm**:
+
+```bash
+npm install @mastra/otel-bridge
+```
+
+**pnpm**:
+
+```bash
+pnpm add @mastra/otel-bridge
+```
+
+**Yarn**:
+
+```bash
+yarn add @mastra/otel-bridge
+```
+
+**Bun**:
+
+```bash
+bun add @mastra/otel-bridge
+```
+
+The bridge works with your existing OpenTelemetry setup. Depending on your configuration, you may also need some of these packages:
+
+- `@opentelemetry/sdk-node` - Core Node.js SDK for OTEL
+- `@opentelemetry/auto-instrumentations-node` - Auto-instrumentation for common libraries
+- `@opentelemetry/exporter-trace-otlp-proto` - OTLP exporter (Protobuf over HTTP)
+- `@opentelemetry/exporter-trace-otlp-http` - OTLP exporter (JSON over HTTP)
+- `@opentelemetry/exporter-trace-otlp-grpc` - OTLP exporter (gRPC)
+- `@opentelemetry/sdk-trace-base` - Base tracing SDK (for BatchSpanProcessor, etc.)
+- `@opentelemetry/core` - Core utilities (for W3CTraceContextPropagator, etc.)
+- `@opentelemetry/sdk-logs` and an OTLP log exporter (e.g. `@opentelemetry/exporter-logs-otlp-http`) - Required if you want the bridge to forward Mastra log events too
+
+## Configuration
+
+Using the OtelBridge requires two steps:
+
+1. Configure OpenTelemetry instrumentation in your application
+2. Add the OtelBridge to your Mastra observability config
+
+### Step 1: OpenTelemetry Instrumentation
+
+Create an instrumentation file that initializes OTEL. This must run before your application code:
+
+```typescript
+import { NodeSDK } from '@opentelemetry/sdk-node'
+import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node'
+import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-proto'
+import { BatchSpanProcessor } from '@opentelemetry/sdk-trace-base'
+import { W3CTraceContextPropagator } from '@opentelemetry/core'
+
+const sdk = new NodeSDK({
+  serviceName: 'my-service',
+  spanProcessors: [
+    new BatchSpanProcessor(
+      new OTLPTraceExporter({
+        url: process.env.OTEL_EXPORTER_OTLP_ENDPOINT || 'http://localhost:4318/v1/traces',
+      }),
+    ),
+  ],
+  instrumentations: [getNodeAutoInstrumentations()],
+  textMapPropagator: new W3CTraceContextPropagator(),
+})
+
+sdk.start()
+
+export { sdk }
+```
+
+### Step 2: Mastra Configuration
+
+Add the OtelBridge to your Mastra observability config:
+
+```typescript
+import { Mastra } from '@mastra/core'
+import { Observability } from '@mastra/observability'
+import { OtelBridge } from '@mastra/otel-bridge'
+
+export const mastra = new Mastra({
+  observability: new Observability({
+    configs: {
+      default: {
+        serviceName: 'my-service',
+        bridge: new OtelBridge(),
+      },
+    },
+  }),
+  agents: {/* your agents */},
+})
+```
+
+No Mastra exporters are required when using the bridge, traces are sent via your OTEL SDK configuration. You can optionally add Mastra exporters if you want to send traces to additional destinations.
+
+### Forwarding logs (optional)
+
+The bridge also forwards Mastra log events to the globally-registered OTEL `LoggerProvider`. To wire up logs alongside traces, register a `logRecordProcessor` on `NodeSDK`:
+
+```typescript
+import { NodeSDK } from '@opentelemetry/sdk-node'
+import { OTLPLogExporter } from '@opentelemetry/exporter-logs-otlp-http'
+import { BatchLogRecordProcessor } from '@opentelemetry/sdk-logs'
+
+const sdk = new NodeSDK({
+  // ...trace config as usual
+  logRecordProcessor: new BatchLogRecordProcessor(
+    new OTLPLogExporter({
+      url: process.env.OTEL_EXPORTER_OTLP_LOGS_ENDPOINT || 'http://localhost:4318/v1/logs',
+    }),
+  ),
+})
+```
+
+Logs that originate inside a Mastra span are emitted under that span's OTEL context, so backends like Datadog, Grafana, and Honeycomb correlate them with the surrounding trace automatically. Logs without trace context use the currently active OTEL context.
+
+If you don't register a `LoggerProvider`, log emission is a silent no-op, traces continue to work as configured.
+
+### Running Your Application
+
+Use the `--import` flag to ensure instrumentation loads before your application:
+
+```bash
+tsx --import ./instrumentation.ts ./src/index.ts
+```
+
+## Semantic conventions
+
+The OtelBridge exports Mastra spans using [OpenTelemetry Semantic Conventions for GenAI v1.38.0](https://github.com/open-telemetry/semantic-conventions/tree/v1.38.0/docs/gen-ai). This includes standardized span names (`chat {model}`, `execute_tool {tool_name}`, etc.) and attributes (`gen_ai.usage.input_tokens`, `gen_ai.request.model`, etc.).
+
+For details on span naming and attributes, see the [OpenTelemetry Exporter semantic conventions](https://mastra.ai/docs/observability/integrations/exporters/otel).
+
+## Trace hierarchy
+
+With the OtelBridge, your traces maintain proper hierarchy across OTEL and Mastra boundaries:
+
+```text
+HTTP POST /api/chat (from Hono middleware)
+└── agent.assistant (from Mastra via OtelBridge)
+    ├── chat gpt-5.4 (LLM call)
+    ├── tool.execute search (tool execution)
+    │   └── HTTP GET api.example.com (from OTEL auto-instrumentation)
+    └── chat gpt-5.4 (follow-up LLM call)
+```
+
+## Multi-service distributed tracing
+
+The OtelBridge enables trace propagation across service boundaries. When Service A calls Service B via HTTP, trace context propagates automatically:
+
+```text
+Service A: HTTP POST /api/process
+└── HTTP POST service-b/api/analyze (outgoing call)
+
+Service B: HTTP POST /api/analyze (incoming call - same trace!)
+└── agent.analyzer (Mastra agent inherits trace context)
+    └── chat gpt-5.4
+```
+
+Both services must have:
+
+1. OTEL instrumentation configured
+2. W3C Trace Context propagator enabled
+3. Mastra with OtelBridge configured
+
+## Using tags
+
+Tags help you categorize and filter traces in your OTEL backend. Add tags when executing agents or workflows:
+
+```typescript
+const result = await agent.generate('Hello', {
+  tracingOptions: {
+    tags: ['production', 'experiment-v2', 'user-request'],
+  },
+})
+```
+
+Tags are exported as a JSON string in the `mastra.tags` span attribute for broad backend compatibility. Common use cases include:
+
+- Environment labels: `"production"`, `"staging"`
+- Experiment tracking: `"experiment-v1"`, `"control-group"`
+- Priority levels: `"priority-high"`, `"batch-job"`
+
+## Troubleshooting
+
+If traces aren't displaying or connecting as expected:
+
+- Verify OTEL SDK is initialized before Mastra (use `--import` flag or import at top of entry point)
+- Ensure the OtelBridge is added to your observability config
+- Check that your OTEL backend is running and accessible
+
+## Related
+
+- [Tracing Overview](https://mastra.ai/docs/observability/tracing/overview)
+- [OpenTelemetry Exporter](https://mastra.ai/docs/observability/integrations/exporters/otel): For sending traces to OTEL backends
+- [OtelBridge Reference](https://mastra.ai/reference/observability/tracing/bridges/otel): API documentation