@telemetry-dev/sdk 0.1.9 → 0.1.11

package/README.md

@@ -78,16 +78,27 @@ other string attribute is cut at the cap without a marker, except that a cap of
 `exportTimeoutMillis` 30000), `spanFilter`, `resourceAttributes`, `logLevel` (SDK diagnostics,
 default `"warn"`), `fetch`, `waitUntil`, `onError`.
 
+Core SDK error recording keeps failure status, error type, and an exception event. It includes
+`exception.message` and `exception.stacktrace` only when both effective `captureInput` and
+`captureOutput` are true, including per-span overrides: exceptions can echo either inputs or
+outputs. The policy is per span: when an error propagates to an enclosing span whose capture is on,
+that span records its message and stack. Included fields pass through `mask` with their exception
+attribute keys, then serialization and truncation. A throwing mask drops the affected field and reports a generic error to `onError`
+and SDK diagnostics without the hook's original message or cause. The original application error
+is rethrown unchanged. These controls do not scrub error type names, metadata, identifiers, resource
+attributes, or events written directly through raw OpenTelemetry APIs.
+
 `captureInput: false` and `captureOutput: false` omit captured content. Provider integrations may
 still record non-content metadata such as response IDs and models, finish reasons, usage, timing,
-and errors. Capture flags do not gate stop sequences, tool descriptions and definitions (`gen_ai.tool.description`, `gen_ai.tool.definitions`), caller-supplied metadata or raw attributes,
-or exception messages and stack traces; redact those separately when needed.
+and error types. Capture flags do not gate stop sequences, tool descriptions and definitions
+(`gen_ai.tool.description`, `gen_ai.tool.definitions`), caller-supplied metadata or raw attributes;
+redact those separately when needed.
 
 `mask` receives input and output content, system instructions, log messages, log attributes
 other than numbers and booleans, and content that integrations pass to
-`handle.serializeContent`. `ctx.key` names the attribute being written. It does not receive
-span or propagated `metadata`, raw `attributes`, or exception messages and stack traces; redact
-those separately.
+`handle.serializeContent`, including captured exception messages and stack traces. `ctx.key` names
+the attribute being written. It does not receive span or propagated `metadata` or raw `attributes`;
+redact those separately. Log masking is independent of span capture flags.
 
 Root spans remain independent by default. `init({ sessionMode: "process" })` groups otherwise
 uncorrelated spans and logs using a fresh opaque UUID for each enabled SDK initialization.
@@ -100,6 +111,69 @@ The default is parent-based always-on. Session roots keep the configured root po
 Nested spans and explicitly parented spans keep the real parent's policy.
 The SDK installs the [session sampling wrapper](https://github.com/telemetry-dev/telemetry/blob/main/packages/otel/README.md#correlation-attributes) on its provider. This includes the global provider that Eve uses.
 
+## Prompts
+
+`getPrompt` fetches a prompt that you manage in telemetry.dev. Fetch it by its label, so that you
+deploy a version when you move the label, without a change to the code:
+
+```ts
+import OpenAI from "openai";
+import { getPrompt, init, withPrompt } from "@telemetry-dev/sdk";
+import { wrapOpenAI } from "@telemetry-dev/openai";
+
+init();
+const openai = wrapOpenAI(new OpenAI());
+
+const prompt = await getPrompt("support/reply", { label: "production" });
+const messages = prompt.compile({ customer_message, tone });
+
+const response = await withPrompt(prompt, () =>
+  openai.chat.completions.create({ model: prompt.model?.name ?? "gpt-4o", messages }),
+);
+
+// Pin one version, as in a test:
+const pinned = await getPrompt("support/reply", { version: 13 });
+```
+
+- Without `label` and `version`, getPrompt fetches the `production` label. A text prompt needs
+  `type: "text"`; getPrompt rejects a prompt of the other type.
+- `compile(variables)` puts the values in place of `{{name}}`. A missing value becomes an empty
+  text, and a value that is not a string becomes JSON. In a chat prompt, a placeholder message
+  becomes the list of messages in its variable. A text prompt compiles to a string.
+- The prompt also has `model`, `parameters`, `tools`, and `responseFormat` of its version.
+- getPrompt keeps each prompt. When the copy is older than `cacheTtlSeconds` (default 60), a call
+  returns it at once and fetches a new copy in the background. Only a call with no copy waits for a
+  fetch, for 10 seconds at most. Calls for the same prompt share one fetch.
+- If a fetch fails, getPrompt returns the last copy that it fetched and reports the error through
+  `onError`. With no copy, it returns the `fallback` option (`version` is `null` and `isFallback`
+  is `true`). With no fallback, it rejects with a `PromptError` whose `code` tells why, for
+  example `prompt_not_found`, `label_not_found`, `insufficient_scope`, `missing_api_key`, or
+  `timeout`. A response with a different status and no API error gives `http_<status>`, for
+  example `http_302` for a redirect. getPrompt does not send the request to the location of a
+  redirect, thus your key does not go to a different host. With no copy, the calls of the next 10
+  seconds (or the TTL, if it is shorter) get the same result without a fetch. This is the one SDK
+  function that can reject: your app cannot run without its prompt.
+- getPrompt uses the API key and base URL of an enabled `init()`, or `TELEMETRY_DEV_API_KEY` and
+  `TELEMETRY_DEV_BASE_URL`, or its own `apiKey`, `baseUrl`, and `fetch` options. It reads the same
+  host that telemetry goes to, `https://ingest.telemetry.dev` by default. The key needs the read
+  scope for prompts.
+- When a label sends part of the traffic to a second version, each session keeps one version: the
+  choice uses the session id of `propagateAttributes` (or of `sessionMode: "process"`). Without a
+  session id, each call chooses at random.
+
+`withPrompt(prompt, fn)` records the prompt on each model call that starts inside `fn`:
+`td.prompt.name`, `td.prompt.version`, and `td.prompt.label` (only for a prompt fetched by label).
+When input capture is on, `td.prompt.variables` holds the JSON of the variables of the last
+`compile()` of that prompt object before `withPrompt` starts, or of the last `compile()` inside
+`fn`, after `mask` and truncation. If tasks that run at the same time share one prompt object, call
+`compile()` in `withPrompt`, or call getPrompt in each task. getPrompt returns a new object from
+the cache. If you do not, a task can record the variables of a different task. Model calls are the
+spans of type `generation` (operations `chat`, `text_completion`, and `generate_content`),
+including the spans of the provider integrations that use this SDK: OpenAI, Anthropic, Amazon
+Bedrock, Google GenAI, and OpenRouter. Spans from the AI SDK and TanStack AI integrations, and from
+other OpenTelemetry tracers, do not get the attributes. Messages from `compile()` that you pass to
+a model call outside `withPrompt` are not recorded.
+
 ## API
 
 - `init(options?)` — create the client (isolated OTel TracerProvider by default).
@@ -119,6 +193,8 @@ The SDK installs the [session sampling wrapper](https://github.com/telemetry-dev
 - `updateActiveSpan(fields)` — update the innermost active span.
 - `propagateAttributes({ userId, sessionId, metadata }, fn)` — stamps `user.id`,
   `gen_ai.conversation.id`, and `td.metadata.*` on every span and log record in scope.
+- `getPrompt(name, options?)` — fetch a prompt by label or version; see [Prompts](#prompts).
+- `withPrompt(prompt, fn)` — record the prompt on each model call in `fn`.
 - `log(message, { level, eventName, attributes }?)` — log record correlated to the active trace.
 - `getTraceparent()` — W3C traceparent of the active span (pass to other services; accept it via
   `startSpan(name, { parent })`).

package/dist/index.d.mts

@@ -7,8 +7,8 @@ import { PushMetricExporter } from "@opentelemetry/sdk-metrics";
 //#region src/config.d.ts
 /**
  * Redaction hook applied to captured content (input/output, log messages and attributes, and
- * content passed to `SpanHandle.serializeContent`) before JSON stringification. Not applied to
- * metadata or raw attributes.
+ * content passed to `SpanHandle.serializeContent`), including exception messages/stacks, before
+ * JSON stringification. Not applied to metadata or raw attributes.
  */
 type MaskFn<T = unknown> = (value: T, ctx: {
   key: string;
@@ -16,7 +16,7 @@ type MaskFn<T = unknown> = (value: T, ctx: {
 interface TelemetryOptions {
   /** telemetry.dev ingest key (`td_live_…`). Falls back to `TELEMETRY_DEV_API_KEY`. When absent the SDK is a no-op. */
   apiKey?: string;
-  /** Ingest base URL. Falls back to `TELEMETRY_DEV_BASE_URL`, then `https://ingest.telemetry.dev`. */
+  /** Base URL for telemetry export and getPrompt. Falls back to `TELEMETRY_DEV_BASE_URL`, then `https://ingest.telemetry.dev`. */
   baseUrl?: string;
   /** Deployment environment label. Falls back to `TELEMETRY_DEV_ENVIRONMENT`, then `production`. */
   environment?: string;
@@ -32,9 +32,9 @@ interface TelemetryOptions {
   exportMode?: ExportMode$1;
   /** OTel sampling policy. Defaults to OTEL_TRACES_SAMPLER / OTEL_TRACES_SAMPLER_ARG. */
   sampler?: Sampler;
-  /** Capture span inputs (args, prompts) by default. Default true. */
+  /** Capture span inputs (args, prompts). False also omits exception messages/stacks. Default true. */
   captureInput?: boolean;
-  /** Capture span outputs (results, completions) by default. Default true. */
+  /** Capture span outputs (results, completions). False also omits exception messages/stacks. Default true. */
   captureOutput?: boolean;
   /** Redaction hook run on captured content before stringify/truncate. */
   mask?: MaskFn;
@@ -57,7 +57,7 @@ interface TelemetryOptions {
   fetch?: typeof fetch;
   /** Serverless extender (e.g. Cloudflare `ctx.waitUntil`). When provided, `flush()` does not await exports. */
   waitUntil?: (p: Promise<unknown>) => void;
-  /** Receives errors raised while emitting telemetry; repeated instrumentation failures from one stream may be coalesced into one report. The SDK never throws into user code. */
+  /** Receives errors raised while emitting telemetry; repeated instrumentation failures from one stream may be coalesced into one report. A series of failed exports is reported once, and the next failure after an export reaches the ingest is reported again. The SDK never throws into user code. */
   onError?: (cause: Error) => void;
   /** Header value for a custom telemetry.dev integration. */
   sdkName?: string;
@@ -257,4 +257,157 @@ interface ObserveOptions extends StartSpanOptions {
  */
 declare function observe<TThis, TArgs extends unknown[], TReturn>(fn: (this: TThis, ...args: TArgs) => TReturn, options?: ObserveOptions): (...args: TArgs) => TReturn;
 //#endregion
-export { type AgentFields, type BatchOptions, type BoundedCaptureDetails, type BoundedCaptureOptions, type BoundedCaptureResult, type ClientOverrides, type ExportMode, type GenerationFields, type LogLevel, type LogOptions, type MaskFn, type ObserveOptions, type ParentRef, type PropagatedAttributes, SCOPE_NAME, type SdkLogLevel, type SpanCapturePolicy, type SpanFields, type SpanHandle, type SpanType, type StartSpanOptions, type TelemetryClient, type TelemetryOptions, TelemetrySpanProcessor, type TelemetrySpanProcessorOptions, type TokenUsage, type ToolFields, activeContext, boundedCapture, boundedCaptureDetails, captureEnabled, captureMasked, extractW3cContext, flush, getTraceparent, init, injectW3cContext, log, observe, propagateAttributes, shutdown, startActiveSpan, startSpan, updateActiveSpan, withContext };
+//#region src/prompts.d.ts
+/** A message of a chat prompt. A placeholder stands for a list of messages that the app passes. */
+type PromptMessage = {
+  readonly role: "system" | "user" | "assistant";
+  readonly content: string;
+} | {
+  readonly role: "placeholder";
+  readonly name: string;
+};
+/** A message that `compile()` makes from a prompt message. */
+interface ChatMessage {
+  role: "system" | "user" | "assistant";
+  content: string;
+}
+interface PromptModel {
+  readonly provider: string;
+  readonly name: string;
+}
+/** The parameters that are set in the version. A parameter that is not set is absent. */
+interface PromptParameters {
+  readonly temperature?: number;
+  readonly topP?: number;
+  readonly maxTokens?: number;
+  readonly frequencyPenalty?: number;
+  readonly presencePenalty?: number;
+  readonly stop?: readonly string[];
+  readonly seed?: number;
+  readonly toolChoice?: "auto" | "none" | "required" | {
+    readonly tool: string;
+  };
+  readonly parallelToolCalls?: boolean;
+}
+interface PromptTool {
+  readonly name: string;
+  readonly description: string;
+  /** JSON schema of the tool arguments. */
+  readonly parameters: Readonly<Record<string, unknown>>;
+  readonly strict: boolean;
+}
+type PromptResponseFormat = {
+  readonly type: "text";
+} | {
+  readonly type: "json_object";
+} | {
+  readonly type: "json_schema";
+  readonly name: string;
+  readonly schema: Readonly<Record<string, unknown>>;
+  readonly strict: boolean;
+};
+/**
+ * Values for `{{name}}` variables. A missing value, null, or undefined becomes an empty text, and a
+ * value that is not a string becomes JSON. A placeholder message takes a list of messages.
+ */
+type PromptVariables<M = ChatMessage> = Readonly<Record<string, string | number | boolean | null | undefined | readonly M[]>>;
+interface PromptBase {
+  readonly name: string;
+  /** The version that the app got; null for the fallback. */
+  readonly version: number | null;
+  /** The label that the prompt was fetched by; null when it was fetched by version. */
+  readonly label: string | null;
+  /** Null for the fallback. */
+  readonly model: PromptModel | null;
+  readonly parameters: PromptParameters;
+  readonly tools: readonly PromptTool[];
+  readonly responseFormat: PromptResponseFormat;
+  /** True when getPrompt could not fetch the prompt and returned the `fallback` option. */
+  readonly isFallback: boolean;
+}
+interface ChatPrompt extends PromptBase {
+  readonly type: "chat";
+  readonly messages: readonly PromptMessage[];
+  readonly text: null;
+  /**
+   * Fills in the variables and replaces each placeholder message with the messages in its
+   * variable. `withPrompt` records the variables of the last call before it starts or inside it.
+   */
+  compile<M = ChatMessage>(variables?: PromptVariables<M>): (ChatMessage | M)[];
+}
+interface TextPrompt extends PromptBase {
+  readonly type: "text";
+  readonly messages: readonly [];
+  readonly text: string;
+  /**
+   * Fills in the variables. `withPrompt` records the variables of the last call before it starts
+   * or inside it.
+   */
+  compile(variables?: PromptVariables): string;
+}
+type Prompt = ChatPrompt | TextPrompt;
+interface GetPromptOptionsBase {
+  /** Fetch the version that this label points to. Default "production" when `version` is not set. */
+  label?: string;
+  /** Fetch this version number. Do not set it together with `label`. */
+  version?: number;
+  /** Seconds before a call fetches a kept prompt again, in the background. Default 60. */
+  cacheTtlSeconds?: number;
+  /** API key with the prompts:read scope. Falls back to the initialized client, then `TELEMETRY_DEV_API_KEY`. */
+  apiKey?: string;
+  /** Falls back to the initialized client, then `TELEMETRY_DEV_BASE_URL`, then `https://ingest.telemetry.dev`. */
+  baseUrl?: string;
+  /** Falls back to the initialized client's `fetch`, then `globalThis.fetch`. */
+  fetch?: typeof fetch;
+}
+interface GetChatPromptOptions extends GetPromptOptionsBase {
+  /** Default "chat". getPrompt rejects a prompt of the other type. */
+  type?: "chat";
+  /** Messages to use when the prompt cannot be fetched and no copy of it is kept. */
+  fallback?: readonly PromptMessage[];
+}
+interface GetTextPromptOptions extends GetPromptOptionsBase {
+  type: "text";
+  /** Text to use when the prompt cannot be fetched and no copy of it is kept. */
+  fallback?: string;
+}
+type GetPromptOptions = GetChatPromptOptions | GetTextPromptOptions;
+/**
+ * Why getPrompt failed. `code` is the code of the API error (`prompt_not_found`,
+ * `label_not_found`, `insufficient_scope`, `invalid_api_key`, …), or `missing_api_key`,
+ * `timeout`, `network_error`, `invalid_response`, or `type_mismatch`. A response with a different
+ * status and no API error gives `http_<status>`, for example `http_302` for a redirect: getPrompt
+ * does not go to the location of a redirect.
+ */
+declare class PromptError extends Error {
+  readonly code: string;
+  readonly status?: number | undefined;
+  readonly name = "PromptError";
+  constructor(code: string, message: string, status?: number | undefined, options?: {
+    cause?: unknown;
+  });
+}
+/**
+ * Fetches a prompt version from telemetry.dev and keeps it. When the copy is older than
+ * `cacheTtlSeconds`, a call returns it at once and fetches a new one in the background. Only a call
+ * with no copy waits for a fetch, at most 10 seconds. Calls for the same prompt share one fetch.
+ * When a fetch fails, getPrompt returns the last copy that it kept; with no copy it returns
+ * `fallback`. With neither it rejects with a `PromptError`: unlike the telemetry functions,
+ * getPrompt cannot fail open, because the app cannot run without the prompt. With no copy, the
+ * calls of the next 10 seconds (or the TTL, if it is shorter) fail the same way without a fetch.
+ *
+ * A label with a gradual rollout gives each session one version: the choice uses the session id
+ * of `propagateAttributes`, or a random choice per call without one.
+ */
+declare function getPrompt(name: string, options: GetTextPromptOptions): Promise<TextPrompt>;
+declare function getPrompt(name: string, options?: GetChatPromptOptions): Promise<ChatPrompt>;
+/**
+ * Runs `fn` with `prompt` as the prompt of each model call (generation span) that starts inside
+ * it: the spans get `td.prompt.name`, `td.prompt.version`, `td.prompt.label`, and, when input
+ * capture is on, `td.prompt.variables`. The variables are those of the last `compile()` of this
+ * prompt object before withPrompt starts, or of the last `compile()` inside `fn`, so a compile in
+ * another task does not change them once withPrompt runs.
+ */
+declare function withPrompt<T>(prompt: Prompt, fn: () => T): T;
+//#endregion
+export { type AgentFields, type BatchOptions, type BoundedCaptureDetails, type BoundedCaptureOptions, type BoundedCaptureResult, type ChatMessage, type ChatPrompt, type ClientOverrides, type ExportMode, type GenerationFields, type GetChatPromptOptions, type GetPromptOptions, type GetTextPromptOptions, type LogLevel, type LogOptions, type MaskFn, type ObserveOptions, type ParentRef, type Prompt, PromptError, type PromptMessage, type PromptModel, type PromptParameters, type PromptResponseFormat, type PromptTool, type PromptVariables, type PropagatedAttributes, SCOPE_NAME, type SdkLogLevel, type SpanCapturePolicy, type SpanFields, type SpanHandle, type SpanType, type StartSpanOptions, type TelemetryClient, type TelemetryOptions, TelemetrySpanProcessor, type TelemetrySpanProcessorOptions, type TextPrompt, type TokenUsage, type ToolFields, activeContext, boundedCapture, boundedCaptureDetails, captureEnabled, captureMasked, extractW3cContext, flush, getPrompt, getTraceparent, init, injectW3cContext, log, observe, propagateAttributes, shutdown, startActiveSpan, startSpan, updateActiveSpan, withContext, withPrompt };