@unhandled/instrumentation-mastra 0.0.0-stage → 0.0.1

package/README.md

@@ -1,3 +1,79 @@
-# Temporary Holding Version
+# @unhandled/instrumentation-mastra
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Reshapes spans produced by Mastra's OpenTelemetry bridge into the attributes Unhandled's AI views read. It does not create spans and it does not create the bridge: you still configure `OtelBridge` yourself (the helper below does it for you).
+
+Peers: `@mastra/core`, `@mastra/otel-bridge` (and `@mastra/observability`, which the bridge already needs).
+
+## Install
+
+```sh
+pnpm add @unhandled/instrumentation-mastra
+```
+
+## Register
+
+With the AdonisJS agent, add the integration. It only registers the span processor:
+
+```ts
+import { mastra } from '@unhandled/instrumentation-mastra/integration'
+
+export default defineConfig({
+  integrations: [mastra({ recordInputs: true, recordOutputs: true })],
+})
+```
+
+Other Node apps, add the processor to your tracer provider:
+
+```ts
+import { MastraSpanProcessor } from '@unhandled/instrumentation-mastra'
+
+const provider = new NodeTracerProvider({ spanProcessors: [new MastraSpanProcessor()] })
+```
+
+Then hand Mastra a bridge config:
+
+```ts
+import { Observability, SamplingStrategyType } from '@mastra/observability'
+import { createMastraObservabilityConfig } from '@unhandled/instrumentation-mastra'
+
+export const observability = new Observability({
+  configs: {
+    default: createMastraObservabilityConfig({
+      serviceName: 'my-app',
+      sampling: { type: SamplingStrategyType.ALWAYS },
+    }),
+  },
+})
+```
+
+`createMastraObservabilityConfig` adds an `OtelBridge` and sets `excludeSpanTypes` to `MODEL_STEP` and `MODEL_CHUNK`. Pass your own `excludeSpanTypes` (replaces the default) or `bridge` (for example an `OtelBridge` with a custom tracer provider) to override. Everything else in the options is forwarded as is.
+
+## Options
+
+| Option                     | Where                | Default | Effect                                                              |
+| -------------------------- | -------------------- | ------- | ------------------------------------------------------------------- |
+| `recordInputs`             | integration, processor | `false` | `true` keeps input content attributes.                            |
+| `recordOutputs`            | integration, processor | `false` | `true` keeps output content attributes and response previews.     |
+| `enabled`                  | integration          | `true`  | `false` skips the integration.                                      |
+| `maxBufferedTraces`        | processor            | `500`   | Traces tracked for metadata propagation; the oldest is dropped first. |
+| `maxBufferedSpansPerTrace` | processor            | `256`   | Spans tracked per trace before the oldest entry is dropped.         |
+
+Prompts and responses are dropped unless you opt in. With `recordInputs` or `recordOutputs` off, the processor also removes Mastra's own `mastra.*.input` and `mastra.*.output` attributes. These flags are independent of the Vercel AI integration's.
+
+## What the processor changes
+
+Mastra already writes most semconv names. The processor renames the rest, keeping a semconv value that is already set:
+
+| Mastra attribute                            | Renamed to                              |
+| ------------------------------------------- | --------------------------------------- |
+| `gen_ai.usage.cached_input_tokens`          | `gen_ai.usage.cache_read.input_tokens`  |
+| `gen_ai.usage.cache_creation.input_tokens`  | `gen_ai.usage.cache_write.input_tokens` |
+| `gen_ai.usage.reasoning_tokens`             | `gen_ai.usage.reasoning.output_tokens`  |
+
+`gen_ai.agent.name` is also copied to `gen_ai.function_id` when that is not set.
+
+Model metadata (request and response model, provider and, with `recordOutputs`, the response preview) is copied from each `chat` span to its nearest `invoke_agent` ancestor when the agent span lacks it, so an agent run shows its model. Usage stays on the `chat` span, so tokens and cost are counted once. Children are buffered per trace until the agent span ends, within the limits above. When several model calls run under one agent, the first one to end supplies each value.
+
+## Evaluations
+
+Span events named `gen_ai.evaluation.result` are recognised. `extractEvaluations(span)` returns `{ name, scoreValue, scoreLabel, explanation, scorerId, scorerVersion, targetTraceId }` per event, read from `gen_ai.evaluation.name`, `.score.value`, `.score.label`, `.explanation`, `.scorer.id`, `.scorer.version` and `.target.trace_id`. A numeric string score is exported as a number.

package/build/attributes.d.ts

@@ -0,0 +1,12 @@
+export declare const MASTRA_ATTR: {
+    readonly agentName: "gen_ai.agent.name";
+    readonly cachedInputTokens: "gen_ai.usage.cached_input_tokens";
+    readonly reasoningTokens: "gen_ai.usage.reasoning_tokens";
+    readonly cacheCreationInputTokens: "gen_ai.usage.cache_creation.input_tokens";
+};
+export declare const EVALUATION_ATTR: {
+    readonly scorerId: "gen_ai.evaluation.scorer.id";
+    readonly scorerVersion: "gen_ai.evaluation.scorer.version";
+    readonly targetTraceId: "gen_ai.evaluation.target.trace_id";
+};
+export declare const MASTRA_CONTENT_PATTERN: RegExp;

package/build/attributes.js

@@ -0,0 +1,12 @@
+export const MASTRA_ATTR = {
+    agentName: "gen_ai.agent.name",
+    cachedInputTokens: "gen_ai.usage.cached_input_tokens",
+    reasoningTokens: "gen_ai.usage.reasoning_tokens",
+    cacheCreationInputTokens: "gen_ai.usage.cache_creation.input_tokens",
+};
+export const EVALUATION_ATTR = {
+    scorerId: "gen_ai.evaluation.scorer.id",
+    scorerVersion: "gen_ai.evaluation.scorer.version",
+    targetTraceId: "gen_ai.evaluation.target.trace_id",
+};
+export const MASTRA_CONTENT_PATTERN = /^mastra\.[a-z_]+\.(input|output)$/;

package/build/evaluations.d.ts

@@ -0,0 +1,9 @@
+import type { Attributes } from "@opentelemetry/api";
+import type { TimedEvent } from "@opentelemetry/sdk-trace-base";
+import type { MastraEvaluation } from "./types.ts";
+export declare const isEvaluationEvent: (event: TimedEvent) => boolean;
+export declare const readEvaluation: (attributes: Attributes) => MastraEvaluation;
+export declare const extractEvaluations: (span: {
+    events: readonly TimedEvent[];
+}) => MastraEvaluation[];
+export declare const coerceEvaluationScores: (events: readonly TimedEvent[]) => void;

package/build/evaluations.js

@@ -0,0 +1,34 @@
+import { ATTR, GEN_AI_EVENTS } from "@unhandled/protocol";
+import { EVALUATION_ATTR } from "./attributes.js";
+const text = (value) => (typeof value === "string" && value.length > 0 ? value : undefined);
+const score = (value) => {
+    if (typeof value === "number" && Number.isFinite(value)) {
+        return value;
+    }
+    if (typeof value === "string" && value.trim().length > 0 && Number.isFinite(Number(value))) {
+        return Number(value);
+    }
+};
+export const isEvaluationEvent = (event) => event.name === GEN_AI_EVENTS.evaluationResult;
+export const readEvaluation = (attributes) => ({
+    name: text(attributes[ATTR.genAiEvaluationName]),
+    scoreValue: score(attributes[ATTR.genAiEvaluationScoreValue]),
+    scoreLabel: text(attributes[ATTR.genAiEvaluationScoreLabel]),
+    explanation: text(attributes[ATTR.genAiEvaluationExplanation]),
+    scorerId: text(attributes[EVALUATION_ATTR.scorerId]),
+    scorerVersion: text(attributes[EVALUATION_ATTR.scorerVersion]),
+    targetTraceId: text(attributes[EVALUATION_ATTR.targetTraceId]),
+});
+export const extractEvaluations = (span) => span.events.filter(isEvaluationEvent).map((event) => readEvaluation(event.attributes ?? {}));
+export const coerceEvaluationScores = (events) => {
+    for (const event of events) {
+        if (!isEvaluationEvent(event) || event.attributes === undefined) {
+            continue;
+        }
+        const raw = event.attributes[ATTR.genAiEvaluationScoreValue];
+        const value = score(raw);
+        if (typeof raw === "string" && value !== undefined) {
+            event.attributes[ATTR.genAiEvaluationScoreValue] = value;
+        }
+    }
+};

package/build/index.d.ts

@@ -0,0 +1,5 @@
+export { extractEvaluations } from "./evaluations.ts";
+export type { MastraObservabilityConfig, MastraObservabilityConfigOptions, } from "./observability_config.ts";
+export { createMastraObservabilityConfig, DEFAULT_EXCLUDED_SPAN_TYPES, } from "./observability_config.ts";
+export { MastraSpanProcessor } from "./span_processor.ts";
+export type { MastraEvaluation, MastraIntegrationOptions, MastraRecordingOptions, MastraSpanProcessorOptions, } from "./types.ts";

package/build/index.js

@@ -0,0 +1,3 @@
+export { extractEvaluations } from "./evaluations.js";
+export { createMastraObservabilityConfig, DEFAULT_EXCLUDED_SPAN_TYPES, } from "./observability_config.js";
+export { MastraSpanProcessor } from "./span_processor.js";

package/build/integration.d.ts

@@ -0,0 +1,4 @@
+import type { UnhandledIntegration } from "@unhandled/node/types";
+import type { MastraIntegrationOptions } from "./types.ts";
+export declare const mastra: (options?: MastraIntegrationOptions) => UnhandledIntegration;
+export type { MastraIntegrationOptions } from "./types.ts";

package/build/integration.js

@@ -0,0 +1,13 @@
+import { MastraSpanProcessor } from "./span_processor.js";
+export const mastra = (options = {}) => ({
+    name: "mastra",
+    enabled: options.enabled ?? true,
+    setup: () => ({
+        spanProcessors: [
+            new MastraSpanProcessor({
+                recordInputs: options.recordInputs,
+                recordOutputs: options.recordOutputs,
+            }),
+        ],
+    }),
+});

package/build/model_metadata.d.ts

@@ -0,0 +1,13 @@
+import type { MutableSpan, ResolvedRecording } from "./types.ts";
+export interface ModelMetadataBufferOptions {
+    maxSpansPerTrace?: number;
+    maxTraces?: number;
+}
+export declare class ModelMetadataBuffer {
+    #private;
+    constructor(options?: ModelMetadataBufferOptions);
+    get traceCount(): number;
+    remember(traceId: string, spanId: string, parentSpanId: string | undefined, modelSpan: MutableSpan | undefined, recording: ResolvedRecording): void;
+    fillParent(traceId: string, parent: MutableSpan, parentSpanId: string): void;
+    endTrace(traceId: string): void;
+}

package/build/model_metadata.js

@@ -0,0 +1,99 @@
+import { ATTR } from "@unhandled/protocol";
+const DEFAULT_MAX_TRACES = 500;
+const DEFAULT_MAX_SPANS_PER_TRACE = 256;
+const MAX_ANCESTOR_DEPTH = 32;
+const MODEL_KEYS = [ATTR.genAiRequestModel, ATTR.genAiResponseModel, ATTR.genAiProviderName];
+const RESPONSE_KEYS = [ATTR.genAiOutputMessages];
+export class ModelMetadataBuffer {
+    #maxTraces;
+    #maxSpansPerTrace;
+    #traces = new Map();
+    constructor(options = {}) {
+        this.#maxTraces = options.maxTraces ?? DEFAULT_MAX_TRACES;
+        this.#maxSpansPerTrace = options.maxSpansPerTrace ?? DEFAULT_MAX_SPANS_PER_TRACE;
+    }
+    get traceCount() {
+        return this.#traces.size;
+    }
+    remember(traceId, spanId, parentSpanId, modelSpan, recording) {
+        const spans = this.#spansFor(traceId);
+        if (spans.size >= this.#maxSpansPerTrace) {
+            const oldest = spans.keys().next();
+            if (!oldest.done) {
+                spans.delete(oldest.value);
+            }
+        }
+        spans.set(spanId, {
+            parentSpanId,
+            metadata: modelSpan === undefined ? undefined : pickMetadata(modelSpan, recording),
+        });
+    }
+    fillParent(traceId, parent, parentSpanId) {
+        const spans = this.#traces.get(traceId);
+        if (spans === undefined) {
+            return;
+        }
+        const claimed = [];
+        for (const [spanId, buffered] of spans) {
+            if (buffered.metadata === undefined) {
+                continue;
+            }
+            if (!descendsFrom(spans, spanId, parentSpanId)) {
+                continue;
+            }
+            for (const [key, value] of Object.entries(buffered.metadata)) {
+                if (value !== undefined && parent.attributes[key] === undefined) {
+                    parent.attributes[key] = value;
+                }
+            }
+            claimed.push(spanId);
+        }
+        for (const spanId of claimed) {
+            const buffered = spans.get(spanId);
+            if (buffered !== undefined) {
+                buffered.metadata = undefined;
+            }
+        }
+    }
+    endTrace(traceId) {
+        this.#traces.delete(traceId);
+    }
+    #spansFor(traceId) {
+        const existing = this.#traces.get(traceId);
+        if (existing !== undefined) {
+            this.#traces.delete(traceId);
+            this.#traces.set(traceId, existing);
+            return existing;
+        }
+        if (this.#traces.size >= this.#maxTraces) {
+            const oldest = this.#traces.keys().next();
+            if (!oldest.done) {
+                this.#traces.delete(oldest.value);
+            }
+        }
+        const created = new Map();
+        this.#traces.set(traceId, created);
+        return created;
+    }
+}
+const descendsFrom = (spans, spanId, ancestorId) => {
+    let cursor = spans.get(spanId)?.parentSpanId;
+    for (let depth = 0; cursor !== undefined && depth < MAX_ANCESTOR_DEPTH; depth += 1) {
+        if (cursor === ancestorId) {
+            return true;
+        }
+        cursor = spans.get(cursor)?.parentSpanId;
+    }
+    return false;
+};
+const pickMetadata = (span, recording) => {
+    const picked = {};
+    for (const [key, value] of Object.entries(span.attributes)) {
+        const isModel = MODEL_KEYS.includes(key);
+        const isResponse = RESPONSE_KEYS.includes(key) && recording.recordOutputs;
+        if ((isModel || isResponse) && value !== undefined) {
+            picked[key] = value;
+        }
+    }
+    return picked;
+};

package/build/normalize.d.ts

@@ -0,0 +1,4 @@
+import type { MutableSpan, ResolvedRecording } from "./types.ts";
+export declare const applyMappings: (span: MutableSpan) => void;
+export declare const stripUnrecordedContent: (span: MutableSpan, recording: ResolvedRecording) => void;
+export declare const isOperation: (span: MutableSpan, operation: string) => boolean;

package/build/normalize.js

@@ -0,0 +1,44 @@
+import { ATTR } from "@unhandled/protocol";
+import { MASTRA_ATTR, MASTRA_CONTENT_PATTERN } from "./attributes.js";
+const RENAMES = {
+    [MASTRA_ATTR.cachedInputTokens]: ATTR.genAiUsageCacheReadInputTokens,
+    [MASTRA_ATTR.reasoningTokens]: ATTR.genAiUsageReasoningOutputTokens,
+    [MASTRA_ATTR.cacheCreationInputTokens]: ATTR.genAiUsageCacheWriteInputTokens,
+};
+const INPUT_KEYS = new Set([
+    ATTR.genAiInputMessages,
+    ATTR.genAiSystemInstructions,
+    ATTR.genAiToolCallArguments,
+]);
+const OUTPUT_KEYS = new Set([ATTR.genAiOutputMessages, ATTR.genAiToolCallResult]);
+export const applyMappings = (span) => {
+    const { attributes } = span;
+    for (const [source, target] of Object.entries(RENAMES)) {
+        const value = attributes[source];
+        if (value === undefined) {
+            continue;
+        }
+        if (attributes[target] === undefined) {
+            attributes[target] = value;
+        }
+        delete attributes[source];
+    }
+    const agentName = attributes[MASTRA_ATTR.agentName];
+    if (attributes[ATTR.genAiFunctionId] === undefined && agentName !== undefined) {
+        attributes[ATTR.genAiFunctionId] = agentName;
+    }
+};
+export const stripUnrecordedContent = (span, recording) => {
+    if (recording.recordInputs && recording.recordOutputs) {
+        return;
+    }
+    for (const key of Object.keys(span.attributes)) {
+        const mastraContent = MASTRA_CONTENT_PATTERN.exec(key);
+        const isInput = INPUT_KEYS.has(key) || mastraContent?.[1] === "input";
+        const isOutput = OUTPUT_KEYS.has(key) || mastraContent?.[1] === "output";
+        if ((isInput && !recording.recordInputs) || (isOutput && !recording.recordOutputs)) {
+            delete span.attributes[key];
+        }
+    }
+};
+export const isOperation = (span, operation) => span.attributes[ATTR.genAiOperationName] === operation;

package/build/observability_config.d.ts

@@ -0,0 +1,6 @@
+import { SpanType } from "@mastra/core/observability";
+import type { ObservabilityInstanceConfig } from "@mastra/observability";
+export type MastraObservabilityConfig = Omit<ObservabilityInstanceConfig, "name">;
+export type MastraObservabilityConfigOptions = Pick<MastraObservabilityConfig, "serviceName"> & Partial<Omit<MastraObservabilityConfig, "serviceName">>;
+export declare const DEFAULT_EXCLUDED_SPAN_TYPES: readonly SpanType[];
+export declare const createMastraObservabilityConfig: (options: MastraObservabilityConfigOptions) => MastraObservabilityConfig;

package/build/observability_config.js

@@ -0,0 +1,8 @@
+import { SpanType } from "@mastra/core/observability";
+import { OtelBridge } from "@mastra/otel-bridge";
+export const DEFAULT_EXCLUDED_SPAN_TYPES = [SpanType.MODEL_STEP, SpanType.MODEL_CHUNK];
+export const createMastraObservabilityConfig = (options) => ({
+    ...options,
+    bridge: options.bridge ?? new OtelBridge(),
+    excludeSpanTypes: options.excludeSpanTypes ?? [...DEFAULT_EXCLUDED_SPAN_TYPES],
+});

package/build/span_processor.d.ts

@@ -0,0 +1,10 @@
+import type { ReadableSpan, SpanProcessor } from "@opentelemetry/sdk-trace-base";
+import type { MastraSpanProcessorOptions } from "./types.ts";
+export declare class MastraSpanProcessor implements SpanProcessor {
+    #private;
+    constructor(options?: MastraSpanProcessorOptions);
+    onStart(): void;
+    onEnd(span: ReadableSpan): void;
+    forceFlush(): Promise<void>;
+    shutdown(): Promise<void>;
+}

package/build/span_processor.js

@@ -0,0 +1,41 @@
+import { GEN_AI_OPERATIONS } from "@unhandled/protocol";
+import { coerceEvaluationScores } from "./evaluations.js";
+import { ModelMetadataBuffer } from "./model_metadata.js";
+import { applyMappings, isOperation, stripUnrecordedContent } from "./normalize.js";
+export class MastraSpanProcessor {
+    #recording;
+    #buffer;
+    constructor(options = {}) {
+        this.#recording = {
+            recordInputs: options.recordInputs ?? false,
+            recordOutputs: options.recordOutputs ?? false,
+        };
+        this.#buffer = new ModelMetadataBuffer({
+            maxTraces: options.maxBufferedTraces,
+            maxSpansPerTrace: options.maxBufferedSpansPerTrace,
+        });
+    }
+    // biome-ignore lint/suspicious/noEmptyBlockStatements: SpanProcessor requires onStart; this processor only rewrites ended spans
+    onStart() { }
+    onEnd(span) {
+        const { traceId, spanId } = span.spanContext();
+        const parentSpanId = span.parentSpanContext?.spanId;
+        applyMappings(span);
+        if (isOperation(span, GEN_AI_OPERATIONS.invokeAgent)) {
+            this.#buffer.fillParent(traceId, span, spanId);
+        }
+        coerceEvaluationScores(span.events);
+        const isModelCall = isOperation(span, GEN_AI_OPERATIONS.chat);
+        this.#buffer.remember(traceId, spanId, parentSpanId, isModelCall ? span : undefined, this.#recording);
+        if (parentSpanId === undefined) {
+            this.#buffer.endTrace(traceId);
+        }
+        stripUnrecordedContent(span, this.#recording);
+    }
+    forceFlush() {
+        return Promise.resolve();
+    }
+    shutdown() {
+        return Promise.resolve();
+    }
+}

package/build/types.d.ts

@@ -0,0 +1,33 @@
+import type { Attributes, HrTime, SpanStatus } from "@opentelemetry/api";
+import type { TimedEvent } from "@opentelemetry/sdk-trace-base";
+export interface MastraRecordingOptions {
+    recordInputs?: boolean;
+    recordOutputs?: boolean;
+}
+export interface MastraSpanProcessorOptions extends MastraRecordingOptions {
+    maxBufferedSpansPerTrace?: number;
+    maxBufferedTraces?: number;
+}
+export interface MastraIntegrationOptions extends MastraRecordingOptions {
+    enabled?: boolean;
+}
+export interface ResolvedRecording {
+    recordInputs: boolean;
+    recordOutputs: boolean;
+}
+export interface MutableSpan {
+    attributes: Attributes;
+    endTime: HrTime;
+    events: TimedEvent[];
+    name: string;
+    status: SpanStatus;
+}
+export interface MastraEvaluation {
+    explanation?: string;
+    name?: string;
+    scoreLabel?: string;
+    scorerId?: string;
+    scorerVersion?: string;
+    scoreValue?: number;
+    targetTraceId?: string;
+}

package/build/types.js

@@ -0,0 +1 @@
+export {};

package/package.json

@@ -1,6 +1,69 @@
 {
   "name": "@unhandled/instrumentation-mastra",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
+  "version": "0.0.1",
+  "description": "Mastra span normalization for Unhandled",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/treboryx/unhandled.git",
+    "directory": "packages/instrumentation-mastra"
+  },
+  "license": "MIT",
+  "type": "module",
+  "exports": {
+    ".": {
+      "types": "./build/index.d.ts",
+      "default": "./build/index.js"
+    },
+    "./integration": {
+      "types": "./build/integration.d.ts",
+      "default": "./build/integration.js"
+    }
+  },
+  "files": [
+    "build"
+  ],
+  "dependencies": {
+    "@opentelemetry/api": "~1.9.1",
+    "@opentelemetry/sdk-trace-base": "^2.12.0",
+    "@unhandled/protocol": "0.0.1"
+  },
+  "devDependencies": {
+    "@adonisjs/tsconfig": "^2.0.0",
+    "@mastra/core": "^1.78.0",
+    "@mastra/observability": "^1.20.1",
+    "@mastra/otel-bridge": "^1.5.17",
+    "@opentelemetry/context-async-hooks": "^2.12.0",
+    "@opentelemetry/exporter-trace-otlp-http": "^0.223.0",
+    "@types/node": "^25.9.9",
+    "@unhandled/node": "0.0.1",
+    "@unhandled/otlp-receiver": "0.0.0",
+    "import-in-the-middle": "^3.5.2",
+    "typescript": "~6.0.2"
+  },
+  "peerDependencies": {
+    "@mastra/core": ">=1.0.0",
+    "@mastra/observability": ">=1.0.0",
+    "@mastra/otel-bridge": ">=1.0.0",
+    "@unhandled/node": "^0.0.1"
+  },
+  "peerDependenciesMeta": {
+    "@mastra/observability": {
+      "optional": true
+    },
+    "@unhandled/node": {
+      "optional": true
+    }
+  },
+  "engines": {
+    "node": ">=24.0.0"
+  },
+  "publishConfig": {
+    "access": "public"
+  },
+  "scripts": {
+    "build": "tsc --project tsconfig.build.json",
+    "test": "node --test 'tests/**/*.test.ts'",
+    "typecheck": "tsrs --noEmit",
+    "typecheck:tsc": "tsc --noEmit"
+  }
 }