@heystack/otel 0.8.0 → 0.9.1

package/README.md

@@ -107,7 +107,7 @@ Set the key as a secret: `wrangler secret put HEYSTACK_API_KEY`.
 > ```toml
 > compatibility_flags = ["nodejs_compat"]
 > ```
-> Without it the SDK falls back to a synchronous stack-based context manager — suppression still works, but cross-`await` span parenting and per-request isolation degrade to best-effort.
+> Without it the SDK falls back to a synchronous stack-based context manager — suppression still works, and **binding spans (D1/KV/R2/Vectorize/AI/Queue/Service) are still parented to the request's SERVER span** (they fall back to the request context explicitly, so they never orphan into their own single-span traces). What still degrades to best-effort without `nodejs_compat` is cross-`await` parenting for *outbound-fetch CLIENT spans* and *manual `withSpan` spans* — so `nodejs_compat` remains recommended for a fully-nested trace tree.
 
 ### `WorkersConfig` options
 
@@ -118,6 +118,7 @@ Set the key as a secret: `wrangler secret put HEYSTACK_API_KEY`.
 | `getUser` | `(req: Request) => { id?, session?, requestId? } \| undefined` | Called per request. `id` → `enduser.id`, `session` → `session.id`, `requestId` → `http.request.id` (falls back to the `cf-ray` header). |
 | `instrumentBindings` | `boolean \| string[]` | `true` = auto child spans for all detected D1 / KV / R2 / Vectorize / Workers AI / Queue producer / Service binding bindings; `string[]` = only the named bindings. Default `false`. |
 | `sampling` | `{ rate?: number } \| { remote: true }` | Head-sampling configuration. `{ rate }`: keep a deterministic fraction of fresh root traces (0–1; default `1` = keep all). `{ remote: true }`: fetch the rate from the Heystack config endpoint instead — lets you change it centrally without redeploying. Cold isolates keep all traffic until the first config fetch resolves; fails open if the config can't be reached. Parent-respecting in both modes: a request arriving with a sampled `traceparent` is always recorded. See [Head sampling](#head-sampling) below. |
+| `ai` | `{ captureContent?: boolean; redact?: (text: string) => string; maxContentChars?: number }` | LLM/gen_ai capture for outbound calls to known providers. See [AI / LLM observability](#ai--llm-observability) below. |
 | `waitUntil` | `(p: Promise<unknown>) => void` | Override the isolate keep-alive hook; defaults to the auto-detected `ctx.waitUntil`. |
 | `endpoint` | `string?` | Override the ingest endpoint (advanced). |
 
@@ -173,6 +174,64 @@ These attributes are set automatically on every SERVER span from request metadat
 | `client.address` | `CF-Connecting-IP` header |
 | `geo.country`, `geo.region`, `geo.city`, `geo.asn` | Cloudflare `req.cf` object |
 
+### AI / LLM observability
+
+When your Worker calls an LLM API, `instrument()` automatically attaches [OpenTelemetry gen_ai semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) to the outbound CLIENT span. No extra code is needed — detection is based on the target hostname.
+
+**Detected providers:**
+- **OpenAI** — `api.openai.com`
+- **Anthropic** — `api.anthropic.com`
+- **Cloudflare AI Gateway** — any host ending in `gateway.ai.cloudflare.com`
+- **Google** — `generativelanguage.googleapis.com`
+
+**Attributes captured automatically (metadata, always on):**
+
+| Attribute | Description |
+| --- | --- |
+| `gen_ai.system` | Provider name (`openai`, `anthropic`, `cloudflare`, `google`) |
+| `gen_ai.request.model` | Model from the request body (`body.model`) |
+| `gen_ai.request.max_tokens` | Max tokens from the request (`max_tokens` or `max_completion_tokens`) |
+| `gen_ai.request.temperature` | Temperature from the request |
+| `gen_ai.response.model` | Model from the response body |
+| `gen_ai.response.id` | Response ID |
+| `gen_ai.usage.input_tokens` | Input/prompt tokens used |
+| `gen_ai.usage.output_tokens` | Output/completion tokens used |
+| `gen_ai.response.finish_reason` | Stop reason (`stop`, `end_turn`, etc.) |
+
+Streaming responses (`text/event-stream`) skip response enrichment (the body is not consumed) but request-side attributes are still set.
+
+**Content capture (opt-in, strongly recommended for AI-app RCA):**
+
+Prompt and completion text are **not captured by default** — enable `ai.captureContent: true` to capture them. This is strongly recommended when debugging AI app issues (wrong answers, unexpected outputs, prompt regressions) since without the content you can see that an LLM call happened and how long it took but not what was said.
+
+Use `redact` to scrub sensitive fields before they leave the Worker. Values are truncated to `maxContentChars` (default 8000) before storage.
+
+```ts
+export default instrument(worker, {
+  service: "my-ai-worker",
+  ai: {
+    captureContent: true,           // capture prompts + completions (off by default)
+    redact: (text) =>               // optional: scrub sensitive text
+      text.replace(/sk-[a-zA-Z0-9]+/g, "[REDACTED]"),
+    maxContentChars: 4000,          // optional: cap per-value length (default 8000)
+  },
+});
+```
+
+When `captureContent` is enabled, two additional attributes appear on the CLIENT span:
+
+| Attribute | Description |
+| --- | --- |
+| `gen_ai.prompt` | The request messages array (JSON-serialised, truncated) |
+| `gen_ai.completion` | The first choice/content block from the response |
+
+**Body safety guarantees:**
+- Request body: only read when it is already a plain `string` (the common LLM SDK case). Non-string bodies (streams, FormData, etc.) are never touched.
+- Response body: read only from `response.clone()`, so the original response is returned untouched and your handler can call `.json()` / `.text()` normally.
+- Parse failures or unexpected shapes degrade silently to the plain CLIENT span — no errors are thrown.
+- For a detected provider with a JSON response, the cloned body is parsed before the `fetch()` promise resolves to your handler (the span must close with usage attributes). This adds negligible latency to an already-slow LLM call.
+- The response size guard (512 KB) only applies when the response carries a `content-length` header; a chunked JSON response without one is still read. The known providers bound this via `max_tokens`, so it is safe in practice.
+
 ### Manual spans: `withSpan` / `addEvent`
 
 Inside a traced handler, add finer-grained spans without touching the OpenTelemetry API directly:
@@ -305,6 +364,8 @@ As belt-and-suspenders the exporter also drops any span whose HTTP target points
 
 ## Migration / versioning
 
+- **`0.9.1`** — **`/workers`: binding spans no longer orphan without `nodejs_compat`.** On a workerd runtime without `nodejs_compat` (no `globalThis.AsyncLocalStorage`), the synchronous fallback context manager cannot carry the active span across an `await` — so a D1/KV/R2/Vectorize/AI/Queue/Service span created *after* an `await` in your handler was emitted as the root of its own single-span trace instead of a child of the request. `instrument()` now captures the request's SERVER context per-request and uses it as an explicit parent fallback when no span is active, so binding operations are always children of the request trace. `nodejs_compat` is still recommended for cross-`await` parenting of outbound-fetch CLIENT spans and manual `withSpan` spans. No API change; upgrade and redeploy.
+- **`0.9.0`** — **`/workers`: automatic LLM gen_ai enrichment for outbound API calls.** Outbound `fetch` calls to known LLM providers (OpenAI, Anthropic, Cloudflare AI Gateway, Google) automatically gain `gen_ai.*` OTel semantic-convention attributes on the CLIENT span — model, token counts, finish reason, response ID — with no extra code. New optional `WorkersConfig.ai` option: `captureContent: true` also captures prompt/completion text (off by default; **strongly recommended for AI-app RCA**), with `redact` for scrubbing and `maxContentChars` for length capping. The original request/response bodies are never consumed (request read only when already a string; response via `response.clone()`). Streaming responses skip response enrichment. No breaking changes.
 - **`0.8.0`** — **`/workers`: Workers AI, Queue producer, and Service binding instrumentation.** `instrumentBindings: true` now auto-wraps three additional binding types: `env.AI.run()` emits CLIENT spans with `gen_ai.system`, `gen_ai.request.model`, and `gen_ai.usage.input_tokens`/`output_tokens` (streaming results are never consumed); Queue `.send`/`.sendBatch` emit PRODUCER spans with `messaging.*` attributes including batch size; Service binding `.fetch` emits a CLIENT span and injects a W3C `traceparent` header into the outgoing request so calls to other Workers appear in the same distributed trace. `startSpan` factory now accepts an optional `SpanKind` for correct CLIENT/PRODUCER categorisation. No breaking changes.
 - **`0.7.0`** — **`/workers`: remote sampling (`sampling: { remote: true }`).** New `sampling` variant that fetches the head-sampling rate from the Heystack config endpoint at runtime, so you can change it from the console without redeploying. Cold isolates keep all traffic until the first config fetch resolves (fails open). If the config endpoint is unreachable, the worker keeps everything. Same parent-respecting rule as `sampling: { rate }`. No breaking changes; existing `sampling: { rate }` configs are unchanged.
 - **`0.6.0`** — **`/workers`: head sampling (`sampling: { rate }`).** New optional `WorkersConfig` field: `sampling.rate` (0–1, default `1`). Keeps a deterministic fraction of fresh root traces — the drop decision is made in the worker before export (no egress, no ingest cost). Parent-respecting: requests arriving with a sampled `traceparent` are always recorded. Consistent with server-side sampling (same trace-ID hash). No breaking changes; all new options are optional. See [Head sampling](#head-sampling).

package/dist/llm-enrich.d.ts

@@ -0,0 +1,15 @@
+export type LLMProvider = "openai" | "anthropic" | "cloudflare" | "google";
+/** Return the provider for a known LLM host, else null. Host is lower-case, no port. */
+export declare function detectLLMProvider(host: string): LLMProvider | null;
+export interface LLMRequestAttrs {
+    [k: string]: string | number;
+}
+/** Best-effort gen_ai.request.* from a parsed JSON request body. Never throws. */
+export declare function llmRequestAttrs(provider: LLMProvider, body: unknown): LLMRequestAttrs;
+/** Best-effort gen_ai.response.* + usage from a parsed JSON response body. Never throws. */
+export declare function llmResponseAttrs(provider: LLMProvider, body: unknown): Record<string, string | number>;
+/** Content attrs, ONLY when captureContent is on. Size-capped per value, redactor applied. */
+export declare function llmContentAttrs(provider: LLMProvider, reqBody: unknown, resBody: unknown, opts: {
+    redact?: (text: string) => string;
+    maxChars?: number;
+}): Record<string, string>;

package/dist/llm-enrich.js

@@ -0,0 +1,129 @@
+// WinterCG-safe LLM enrichment helpers — pure string/object logic, no Node imports.
+// Used by the /workers fetch wrapper to attach gen_ai.* semantic-convention
+// attributes to outbound calls to known LLM providers.
+/** Return the provider for a known LLM host, else null. Host is lower-case, no port. */
+export function detectLLMProvider(host) {
+    const h = host.toLowerCase();
+    if (h === "api.openai.com")
+        return "openai";
+    if (h === "api.anthropic.com")
+        return "anthropic";
+    if (h === "generativelanguage.googleapis.com")
+        return "google";
+    if (h === "gateway.ai.cloudflare.com" || h.endsWith(".gateway.ai.cloudflare.com"))
+        return "cloudflare";
+    return null;
+}
+/** Best-effort gen_ai.request.* from a parsed JSON request body. Never throws. */
+export function llmRequestAttrs(provider, body) {
+    const out = { "gen_ai.system": provider };
+    if (!body || typeof body !== "object" || Array.isArray(body))
+        return out;
+    const b = body;
+    if (typeof b["model"] === "string")
+        out["gen_ai.request.model"] = b["model"];
+    // max_tokens: Anthropic uses max_tokens; OpenAI supports both max_tokens and
+    // max_completion_tokens (prefer max_tokens when both are present).
+    const maxTokens = b["max_tokens"] ?? b["max_completion_tokens"];
+    if (typeof maxTokens === "number")
+        out["gen_ai.request.max_tokens"] = maxTokens;
+    if (typeof b["temperature"] === "number")
+        out["gen_ai.request.temperature"] = b["temperature"];
+    return out;
+}
+/** Best-effort gen_ai.response.* + usage from a parsed JSON response body. Never throws. */
+export function llmResponseAttrs(provider, body) {
+    const out = {};
+    if (!body || typeof body !== "object" || Array.isArray(body))
+        return out;
+    const b = body;
+    if (typeof b["model"] === "string")
+        out["gen_ai.response.model"] = b["model"];
+    if (typeof b["id"] === "string")
+        out["gen_ai.response.id"] = b["id"];
+    // Usage tokens — OpenAI: usage.prompt_tokens / completion_tokens
+    //               Anthropic: usage.input_tokens / output_tokens
+    if (b["usage"] && typeof b["usage"] === "object" && !Array.isArray(b["usage"])) {
+        const usage = b["usage"];
+        // OpenAI shape
+        if (typeof usage["prompt_tokens"] === "number") {
+            out["gen_ai.usage.input_tokens"] = usage["prompt_tokens"];
+        }
+        if (typeof usage["completion_tokens"] === "number") {
+            out["gen_ai.usage.output_tokens"] = usage["completion_tokens"];
+        }
+        // Anthropic shape (overrides OpenAI if both somehow present)
+        if (typeof usage["input_tokens"] === "number") {
+            out["gen_ai.usage.input_tokens"] = usage["input_tokens"];
+        }
+        if (typeof usage["output_tokens"] === "number") {
+            out["gen_ai.usage.output_tokens"] = usage["output_tokens"];
+        }
+    }
+    // Finish reason — OpenAI: choices[0].finish_reason; Anthropic: stop_reason
+    if (typeof b["stop_reason"] === "string") {
+        out["gen_ai.response.finish_reason"] = b["stop_reason"];
+    }
+    else if (Array.isArray(b["choices"]) && b["choices"].length > 0) {
+        const choice = b["choices"][0];
+        if (typeof choice["finish_reason"] === "string") {
+            out["gen_ai.response.finish_reason"] = choice["finish_reason"];
+        }
+    }
+    void provider; // provider available for future provider-specific logic
+    return out;
+}
+/** Content attrs, ONLY when captureContent is on. Size-capped per value, redactor applied. */
+export function llmContentAttrs(provider, reqBody, resBody, opts) {
+    const out = {};
+    const cap = opts.maxChars ?? 8000;
+    const applyRedact = (s) => {
+        const trimmed = s.slice(0, cap);
+        // Re-slice after redact so a redactor that expands text can't blow past the
+        // cap (content is the sensitive storage path — keep it hard-bounded).
+        return opts.redact ? opts.redact(trimmed).slice(0, cap) : trimmed;
+    };
+    // Prompt — serialize request messages/prompt compactly
+    try {
+        if (reqBody && typeof reqBody === "object" && !Array.isArray(reqBody)) {
+            const req = reqBody;
+            if (Array.isArray(req["messages"])) {
+                out["gen_ai.prompt"] = applyRedact(JSON.stringify(req["messages"]));
+            }
+            else if (typeof req["prompt"] === "string") {
+                out["gen_ai.prompt"] = applyRedact(req["prompt"]);
+            }
+        }
+    }
+    catch {
+        // never throw
+    }
+    // Completion — extract response text
+    try {
+        if (resBody && typeof resBody === "object" && !Array.isArray(resBody)) {
+            const res = resBody;
+            // OpenAI shape: choices[0].message.content
+            if (Array.isArray(res["choices"]) && res["choices"].length > 0) {
+                const choice = res["choices"][0];
+                if (choice["message"] && typeof choice["message"] === "object") {
+                    const msg = choice["message"];
+                    if (typeof msg["content"] === "string") {
+                        out["gen_ai.completion"] = applyRedact(msg["content"]);
+                    }
+                }
+            }
+            // Anthropic shape: content[0].text
+            else if (Array.isArray(res["content"]) && res["content"].length > 0) {
+                const block = res["content"][0];
+                if (typeof block["text"] === "string") {
+                    out["gen_ai.completion"] = applyRedact(block["text"]);
+                }
+            }
+        }
+    }
+    catch {
+        // never throw
+    }
+    void provider; // available for future provider-specific logic
+    return out;
+}

package/dist/workers.d.ts

@@ -178,6 +178,21 @@ export interface WorkersConfig {
         rate?: number;
         remote?: boolean;
     };
+    /**
+     * LLM/gen_ai capture for outbound calls to known providers (OpenAI, Anthropic,
+     * CF AI Gateway, Google). Metadata (model, tokens, latency, finish_reason) is
+     * always captured for detected providers.
+     *
+     * Set `captureContent: true` to also capture prompt/completion text (off by
+     * default) — strongly recommended for AI-app RCA. Provide `redact` to scrub
+     * sensitive text before it leaves the Worker. `maxContentChars` caps the length
+     * of each captured value (default 8000 characters).
+     */
+    ai?: {
+        captureContent?: boolean;
+        redact?: (text: string) => string;
+        maxContentChars?: number;
+    };
 }
 /**
  * A `BasicTracerProvider` with the underlying `HeystackSpanExporter` attached so

package/dist/workers.js

@@ -15,6 +15,7 @@ import { BasicTracerProvider, SimpleSpanProcessor, } from "@opentelemetry/sdk-tr
 import { ATTR_SERVICE_NAME } from "@opentelemetry/semantic-conventions";
 import { buildExporterConfig, DEFAULT_ENDPOINT } from "./core.js";
 import { isSelfSpanAttrs, safeHostname } from "./self-span.js";
+import { detectLLMProvider, llmRequestAttrs, llmResponseAttrs, llmContentAttrs } from "./llm-enrich.js";
 import { instrumentEnv } from "./workers-bindings.js";
 import { makeSampler, loadRemoteSamplingRate } from "./workers-sampler.js";
 // `ExportResult` / `ExportResultCode` mirror `@opentelemetry/core`. We define
@@ -264,12 +265,15 @@ function injectTraceparent(input, init, traceparent) {
     headers.set("traceparent", traceparent);
     return [input, { ...(init ?? {}), headers }];
 }
+/** Max response body size to clone for gen_ai enrichment (512 KB). */
+const LLM_RESPONSE_SIZE_CAP = 512 * 1024;
 /**
  * Patch `globalThis.fetch` exactly once to emit a CLIENT child span + inject
  * `traceparent` for outbound subrequests. `ingestHost` is the bare ingest
  * hostname (lower-case, no port) so the exporter's own uploads are never traced.
+ * `aiCfg` enables gen_ai semantic-convention enrichment for known LLM providers.
  */
-function ensureFetchInstrumentation(ingestHost) {
+function ensureFetchInstrumentation(ingestHost, aiCfg = {}) {
     if (_fetchInstrumented)
         return;
     _fetchInstrumented = true;
@@ -304,9 +308,59 @@ function ensureFetchInstrumentation(ingestHost) {
         const sc = span.spanContext();
         const traceparent = `00-${sc.traceId}-${sc.spanId}-01`;
         const [reqInput, reqInit] = injectTraceparent(input, init, traceparent);
+        // gen_ai enrichment — request side.
+        // Only when host is a known LLM provider and the body is already a string
+        // (the common LLM SDK case). Never clone/read the outgoing Request body.
+        const provider = detectLLMProvider(host);
+        let parsedReqBody = undefined;
+        if (provider !== null) {
+            // Only read the request body when it's already a string (the common LLM
+            // SDK case). Never clone/consume the outgoing Request.
+            const bodyStr = typeof init?.body === "string" ? init.body : undefined;
+            if (typeof bodyStr === "string") {
+                try {
+                    parsedReqBody = JSON.parse(bodyStr);
+                    for (const [k, v] of Object.entries(llmRequestAttrs(provider, parsedReqBody))) {
+                        span.setAttribute(k, v);
+                    }
+                }
+                catch {
+                    // body not valid JSON — skip request enrichment
+                }
+            }
+        }
         try {
             const response = await originalFetch(reqInput, reqInit);
             span.setAttribute("http.response.status_code", response.status);
+            // gen_ai enrichment — response side.
+            // Guard: known provider, JSON content-type (not event-stream), size cap.
+            // Always clone — NEVER consume the original response body.
+            if (provider !== null) {
+                try {
+                    const ct = response.headers.get("content-type") ?? "";
+                    const cl = parseInt(response.headers.get("content-length") ?? "0", 10);
+                    const isJson = ct.includes("application/json");
+                    const isStream = ct.includes("event-stream");
+                    const withinCap = isNaN(cl) || cl === 0 || cl <= LLM_RESPONSE_SIZE_CAP;
+                    if (isJson && !isStream && withinCap) {
+                        const rb = await response.clone().json();
+                        for (const [k, v] of Object.entries(llmResponseAttrs(provider, rb))) {
+                            span.setAttribute(k, v);
+                        }
+                        if (aiCfg.captureContent) {
+                            for (const [k, v] of Object.entries(llmContentAttrs(provider, parsedReqBody, rb, {
+                                redact: aiCfg.redact,
+                                maxChars: aiCfg.maxContentChars,
+                            }))) {
+                                span.setAttribute(k, v);
+                            }
+                        }
+                    }
+                }
+                catch {
+                    // parse failure — degrade to plain CLIENT span (no throw, no side effects)
+                }
+            }
             return response;
         }
         catch (error) {
@@ -656,7 +710,7 @@ function ensureGlobalProvider(config) {
     // Patch globalThis.fetch (once) so outbound subrequests get CLIENT child spans
     // + `traceparent` injection (distributed tracing). The exporter's own POST uses
     // the captured original fetch, so it is never re-entered by this wrapper.
-    ensureFetchInstrumentation(_provider.heystackExporter.ingestHost);
+    ensureFetchInstrumentation(_provider.heystackExporter.ingestHost, config.ai ?? {});
     return _provider;
 }
 /**
@@ -729,6 +783,7 @@ export function instrument(handler, config) {
             endpoint: config.endpoint,
             waitUntil: config.waitUntil,
             sampling: config.sampling,
+            ai: config.ai,
         });
         return { provider, tracer: trace.getTracer("heystack") };
     };
@@ -831,19 +886,35 @@ export function instrument(handler, config) {
             // FR5: response header carrying THIS span's trace + span id.
             const sc = span.spanContext();
             const traceparent = `00-${sc.traceId}-${sc.spanId}-01`;
+            // The context with THIS request's SERVER span active. Used both to run the
+            // handler and as an explicit parent fallback for binding spans.
+            const serverCtx = trace.setSpan(startCtx, span);
             // Instrument bindings when requested — wrap env BEFORE handing to the
             // handler so binding calls made inside `originalFetch` (which runs inside
-            // `context.with` below) correctly parent to the root span via ALS.
+            // `context.with` below) parent to the root span.
             let handlerEnv = env;
             if (config.instrumentBindings) {
                 const binTracer = trace.getTracer("heystack");
                 handlerEnv = instrumentEnv(env, {
-                    startSpan: (name, attrs, kind) => binTracer.startSpan(name, { attributes: attrs, kind }, context.active()),
+                    // Prefer the ambient active context — that gives precise nested
+                    // parenting when the AlsContextManager is active (runtimes with
+                    // globalThis.AsyncLocalStorage). But when no span is active — e.g. on
+                    // workerd WITHOUT nodejs_compat, where SyncStackContextManager can't
+                    // carry context across an `await` and its stack has already unwound —
+                    // fall back to THIS request's captured SERVER context so the binding
+                    // span is still a child of the request instead of an orphan root of
+                    // its own trace. serverCtx is captured per-request in this closure, so
+                    // concurrent requests never cross-parent.
+                    startSpan: (name, attrs, kind) => {
+                        const active = context.active();
+                        const parentCtx = trace.getSpan(active) ? active : serverCtx;
+                        return binTracer.startSpan(name, { attributes: attrs, kind }, parentCtx);
+                    },
                     select: config.instrumentBindings,
                 });
             }
             try {
-                const response = await context.with(trace.setSpan(startCtx, span), () => originalFetch(req, handlerEnv, ctx));
+                const response = await context.with(serverCtx, () => originalFetch(req, handlerEnv, ctx));
                 const headers = new Headers(response.headers);
                 headers.set("traceparent", traceparent);
                 appendExposeHeader(headers, "traceparent");

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@heystack/otel",
-  "version": "0.8.0",
+  "version": "0.9.1",
   "description": "Runtime-aware OpenTelemetry tracing that exports to Heystack (Node, Next.js, Workers).",
   "license": "MIT",
   "type": "module",