@amaster.ai/dsh-langfuse 0.1.16 → 0.1.18

package/README.md

@@ -5,10 +5,12 @@
 Langfuse observability for [DeepSeek Harness (dsh)](https://github.com/deepseek-ai/deepseek-harness):
 
 - one **generation** per LLM call (`llm/stream` waterfall), plus a nested `llm-request` span carrying the verbatim loop-built request
-- one **span** per tool call (`tools/execute` waterfall)
+- one **tool observation** per tool call (`tools/execute` waterfall)
 - one **span** per `hook/invoked` / `hook/result` pair, with hook point, decision, exit code, and duration (stderr summary follows `captureContent`)
 - one **trace** per session turn (`session/event`) — in the v5 SDK the trace IS its root span, ended (and thereby exported) at `turn/end`
 - subagent child sessions nested under the parent's tree (`session/created` header link + `subagent/start` / `subagent/end`)
+- structured reasoning in generation output, optional image display, inherited user/tags, and final turn severity
+- optional W3C parent context supplied per session, without adopting the host's ambient OTEL context
 
 ## Install
 
@@ -18,7 +20,7 @@ dsh plugin --profile my-agent add @amaster.ai/dsh-langfuse \
   @opentelemetry/api @opentelemetry/exporter-trace-otlp-http
 ```
 
-The Langfuse JS SDK v5 peers (`@langfuse/tracing`, `@langfuse/otel`, `@opentelemetry/sdk-trace-node`) are loaded via lazy dynamic `import()`, so an unused install costs nothing — but they must all be present at runtime (`dsh plugin add` does not auto-install peer trees, so the OTEL api/exporter packages `@langfuse/otel` itself peers on are listed explicitly).
+The SDK peers (`@langfuse/tracing`, `@langfuse/otel`, `@opentelemetry/sdk-trace-node`, `@opentelemetry/api`) are loaded via lazy dynamic `import()`, so an unused install costs nothing — but they must all be present at runtime (`dsh plugin add` does not auto-install peer trees, so the exporter that `@langfuse/otel` itself peers on is listed explicitly).
 
 ## Configuration
 
@@ -35,9 +37,55 @@ Disabled by default. Configure via the profile's `cordis.patch.yml`:
         baseUrl: https://cloud.langfuse.com
         traceName: dsh-turn
         captureContent: true            # set false to record metadata only
+        userId: developer
+        tags: [coding, experiment-a]
+        environment: development
+        release: build-42
+        redactFields: [password, authorization]
+        captureMedia: false              # opt in to uploading images
+        maxMediaBytes: 5242880            # total image bytes per display payload
+        mediaTimeoutMs: 5000              # attachment read deadline
 ```
 
-Buffered spans drain on `session/flush(session)`; the exporter shuts down with the plugin fiber.
+Explicit profile values take precedence over environment variables. Empty credentials/base URL fall back to `LANGFUSE_PUBLIC_KEY`, `LANGFUSE_SECRET_KEY`, and `LANGFUSE_BASE_URL` (then the EU cloud URL). `userId` falls back to `LANGFUSE_USER_ID`. Empty environment/release values use the SDK's `LANGFUSE_TRACING_ENVIRONMENT` / `LANGFUSE_RELEASE` defaults. `LANGFUSE_TRACING_ENABLED=false` (or `0`) overrides the profile master switch; environment keys alone do not enable the plugin.
+
+Buffered spans drain on `session/flush(session)`; the exporter shuts down with the plugin fiber. Both checkpoints await pending display projections. Unload also closes outstanding model calls with their collected partial content/usage and marks incomplete calls `WARNING`, including calls whose stream was never consumed; the original stream can still settle without ending those observations twice. Attachment reads run in the background and never delay the agent's chunk stream or tool return value; a read that exceeds its deadline degrades to image metadata, even if the backend ignores cancellation. The SDK's `LANGFUSE_FLUSH_AT`, `LANGFUSE_FLUSH_INTERVAL`, and `LANGFUSE_TIMEOUT` control exporter batching and network timeouts.
+
+With `captureContent: true`, both `llm-call` and `llm-request` inputs contain every `llm/stream` request option except the non-serializable `AbortSignal`. New request fields pass through without a plugin update. For image-bearing requests, generation input is updated to a display projection; `llm-request` retains the original attachment references. Langfuse's separate generation `modelParameters` attribute summarizes the known sampling fields; it is not part of either request input. These are dsh-side values, not exact provider HTTP payloads: adapters may add defaults, project messages/tools, or rename fields after this seam. With `captureContent: false`, the collector keeps chunk/tool-call counts and usage without buffering text, reasoning, tool arguments, or raw chunks; inputs keep structural counts while the separate `modelParameters` attribute keeps safe parameter metadata; reply excerpts, reasoning, tool arguments/results, error text, and media are withheld from observation names and IO.
+
+Configured Langfuse credentials and `pk-lf-*` / `sk-lf-*` values are redacted across reporter payloads, names, status messages, and labels. `redactFields` replaces matching payload fields recursively, including JSON-encoded arguments, ignoring case. Name excerpts are derived after redaction. Delta blocks are also checked as assembled text so fragmented credentials cannot bypass the mask: affected blocks keep their record count/type/index, with sanitized text in the first delta and later deltas cleared. Redaction only changes telemetry copies; original messages/results and the forwarded stream stay intact. The exporter applies the same mask to IO and metadata as a final safeguard. User/tags are explicit metadata and remain when content capture is off.
+
+## Reasoning, images, and turn results
+
+Text-only generation output remains a string. When reasoning or tool calls are present, output uses the Langfuse chat format: `role: "assistant"`, optional `content`, `thinking` blocks, and `tool_calls` with the original call ID and function arguments. This renders thinking and invoked tools in the formatted preview. The original reasoning deltas remain in the nested `llm-request` output; no extra generation is created.
+
+`captureMedia: true` requires content capture and an available `ctx.attachments` service (`@deepseek-ai/dsh-attachment >=0.2.0-rc.1`). Prompt, generation-input, and tool-output image references are read through that service and converted to data URIs for the SDK's media uploader. Install the attachment bundle appropriate to the deployment; the plugin does not instantiate a storage backend. Missing/failed reads, offloaded images, unsupported MIME types, oversized images, and disabled media fall back to MIME/size/dimension metadata. A generation's byte budget covers all messages in that request. Observation end times are captured when the model, tool, or turn completes; delayed image reads do not extend business latency. `LANGFUSE_MEDIA_UPLOAD_ENABLED=false` (or `0`) overrides media capture. Verbatim file references remain metadata; their bytes are not uploaded.
+
+Root severity follows the final turn reason: errors are `ERROR`; cancellation, blocking, token limits, and interruptions are `WARNING`. A recovered intermediate generation/tool/hook/subagent error sets `hadErrors` metadata while a successful final turn remains `DEFAULT`. This keeps retry failures visible without treating recovery as a failed turn. Outstanding tools also close as `WARNING` at unload, including calls without an agent/session; late results still reach their original callers without repeated telemetry or media reads.
+
+## Per-request labels and distributed tracing
+
+Hosts can supply standard context before a session's next trace starts:
+
+```ts
+ctx.emit('langfuse/context', session.id, {
+  userId: 'user-123',
+  tags: ['support', 'experiment-b'],
+  traceparent: '00-1234567890abcdef1234567890abcdef-1234567890abcdef-01',
+});
+```
+
+The typed event is exported through this package's Cordis augmentation. Context is consumed once by the next turn trace (or an out-of-turn trace created first); send it again for another request. Labels override profile defaults for that trace and propagate through generations, tools, hooks, and nested subagents. Queued context is discarded when the session is disposed or the plugin unloads. Concurrent sessions have independent context; no process-global request environment is modified.
+
+Only explicit W3C version-00 parents with non-zero trace/span IDs are joined. Invalid headers log a prefixed diagnostic and start an independent trace. Without a header the plugin always starts an independent trace. Each observation is created inside native OTEL `ROOT_CONTEXT`: foreign spans and Langfuse baggage (user/tags/metadata) are excluded, while explicit parents and plugin labels are preserved. This scoped context does not replace the host's global provider or context manager. When joining an external trace, dsh's turn is an observation under the supplied parent; the SDK's automatic app-root claim is disabled and the external owner's trace name and trace-level IO stay intact.
+
+## Token usage and cost
+
+Generation `usageDetails` has disjoint `input`, `cache_read_input_tokens`, `cache_creation_input_tokens`, `output`, and `output_reasoning_tokens` buckets plus `total`. Reasoning is subtracted from provider output before becoming its own bucket; cached input is counted separately. These keys match the official Pi integration. Custom definitions using the previous `input_cache_read`, `input_cache_creation`, or `output_reasoning` names must update their price keys. This avoids double-counting both usage and inferred cost. On Langfuse v3, the compact token badge may display only the base `input`/`output` buckets. Open its usage breakdown for aggregate input (including cache) and output (including reasoning); `total` already includes every bucket. Do not add cached tokens to `input` again to change that badge.
+
+[Langfuse infers USD cost](https://langfuse.com/docs/observability/features/token-and-cost-tracking) when the generation model matches a model definition with prices for the **exact usage keys**. In Project Settings → Models, configure these buckets for private routes, model aliases, or custom prices; reasoning output generally uses the model's output rate. A token total does not prove that a matching price definition exists. If a generation shows “Create model definition”, its model has no matching definition. For gateway aliases, use the actual routed model and your gateway’s rates when defining prices; the plugin keeps the requested model name rather than guessing the upstream model. Definition changes apply to new generations only. dsh's current `TokenUsage` does not carry billed cost, so the plugin does not claim provider-invoiced costs or maintain its own price table.
+
+Compaction and session-title calls passing through `llm/stream` already receive their own generation and `purpose` metadata, including out-of-turn calls. Tools using the same LLM service are observed there too; adding a second generation from tool-result usage would duplicate accounting.
 
 Hook spans appear when a dsh hook bridge emits the paired session records. The current `hook/result` record does not contain stdout; successful script output can appear in the LLM request when the bridge injects it as context, while the hook span reports decision, exit code, duration, and bounded stderr. `SessionStart` runs before the first turn and does not emit these records, so it has no hook span.
 
@@ -45,8 +93,12 @@ Observability is a no-throw seam: backend errors are logged with a `[dsh-langfus
 
 ## Compatibility
 
+Run `pnpm test:sdk` at the repository root for a secrets-free check using the real Langfuse SDK and the existing local OTLP receiver. It verifies export, native tool type, SDK label redaction, ambient-baggage isolation, external trace ownership, and agent-less tool unload. It is separate from the mocked unit suite and the gateway/real-Langfuse E2E legs.
+
 Pinned dsh/cordis versions live in the [root compat matrix](../../README.md#compatibility). Event payloads ride pre-release dsh APIs — check the `TODO(verify)` markers in `src/` before upgrading dsh.
 
 ## License
 
 MIT
+
+The Langfuse integration scenarios label observations with environment `ci`, release `GITHUB_SHA`, and tags for scenario, run ID, and workflow run attempt. Select environment `ci` in Langfuse to find new E2E traces. Read-back uses a bounded time window for each model run and queries a discovered trace directly while ingestion settles.

package/lib/client.js

@@ -40,6 +40,21 @@
  * parent instead of adopting the ambient context's active span.
  */
 const NO_PARENT = { traceId: '0'.repeat(32), spanId: '0'.repeat(16), traceFlags: 0 };
+function parentContext(traceparent) {
+    if (traceparent === undefined)
+        return NO_PARENT;
+    const match = /^00-([0-9a-f]{32})-([0-9a-f]{16})-([0-9a-f]{2})$/.exec(traceparent);
+    if (!match || /^0+$/.test(match[1]) || /^0+$/.test(match[2])) {
+        console.error('[dsh-langfuse] invalid traceparent; starting an independent trace');
+        return NO_PARENT;
+    }
+    return {
+        traceId: match[1],
+        spanId: match[2],
+        traceFlags: Number.parseInt(match[3], 16) & 1,
+        isRemote: true,
+    };
+}
 /**
  * Map dsh token accounting onto Langfuse's `usageDetails`, keeping every
  * bucket mutually exclusive (Langfuse's flat-bucket rule: `input` excludes
@@ -49,7 +64,7 @@ const NO_PARENT = { traceId: '0'.repeat(32), spanId: '0'.repeat(16), traceFlags:
  * INCLUDES reasoning (verified in dsh-llm-deepseek@0.1.6-alpha.2:
  * `outputTokens: usage.completion_tokens`, with reasoning split out of
  * `completion_tokens_details`), so the `output` bucket subtracts
- * `reasoningTokens` into `output_reasoning`. Only `usageDetails` is sent.
+ * `reasoningTokens` into `output_reasoning_tokens`. Only `usageDetails` is sent.
  */
 export function usageOf(usage) {
     // The dsh type marks the two primary fields required, but a non-conformant
@@ -68,16 +83,17 @@ export function usageOf(usage) {
         total,
     };
     if (usage.cacheReadTokens)
-        usageDetails.input_cache_read = usage.cacheReadTokens;
+        usageDetails.cache_read_input_tokens = usage.cacheReadTokens;
     if (usage.cacheWriteTokens)
-        usageDetails.input_cache_creation = usage.cacheWriteTokens;
+        usageDetails.cache_creation_input_tokens = usage.cacheWriteTokens;
     if (reasoning)
-        usageDetails.output_reasoning = reasoning;
+        usageDetails.output_reasoning_tokens = reasoning;
     return usageDetails;
 }
 export class LangfuseReporter {
     config;
     tracing = null;
+    api = null;
     provider = null;
     /**
      * Observation → its trace's correlating attributes; children inherit their
@@ -99,24 +115,112 @@ export class LangfuseReporter {
         this.config = config;
         this.ready = this.init();
     }
+    /** Clone telemetry only; never mutate harness-owned messages or tool results. */
+    mask(data) {
+        const secrets = [this.config.publicKey, this.config.secretKey].filter(Boolean);
+        const fields = new Set(this.config.redactFields?.map((key) => key.toLowerCase()));
+        const seen = new WeakSet();
+        const redactText = (text) => {
+            const ranges = Array.from(text.matchAll(/(?:pk|sk)-lf-[\w-]+/g), (match) => [match.index, match.index + match[0].length]);
+            for (const secret of secrets) {
+                for (let index = text.indexOf(secret); index !== -1; index = text.indexOf(secret, index + 1)) {
+                    ranges.push([index, index + secret.length]);
+                }
+            }
+            let output = '';
+            let end = 0;
+            // Match the original text once, merging overlap before changing any bytes.
+            for (const [start, stop] of ranges.sort((a, b) => a[0] - b[0])) {
+                if (start >= end)
+                    output += `${text.slice(end, start)}[REDACTED]`;
+                end = Math.max(end, stop);
+            }
+            return output + text.slice(end);
+        };
+        const redact = (value) => {
+            if (typeof value === 'string') {
+                // Tool-call arguments and SDK attributes may themselves be JSON text.
+                if (fields.size && /^\s*[[{]/.test(value)) {
+                    let parsed;
+                    try {
+                        parsed = JSON.parse(value);
+                    }
+                    catch {
+                        /* not JSON */
+                    }
+                    if (parsed !== undefined) {
+                        const masked = JSON.stringify(redact(parsed));
+                        if (masked !== JSON.stringify(parsed))
+                            return masked;
+                    }
+                }
+                return redactText(value);
+            }
+            if (!value || typeof value !== 'object' || value instanceof Date)
+                return value;
+            if (seen.has(value))
+                return '[Circular]';
+            seen.add(value);
+            const result = Array.isArray(value)
+                ? value.map(redact)
+                : Object.fromEntries(Object.entries(value).map(([key, child]) => [
+                    redact(key),
+                    fields.has(key.toLowerCase()) ? '[REDACTED]' : redact(child),
+                ]));
+            seen.delete(value);
+            return result;
+        };
+        try {
+            return redact(data);
+        }
+        catch (error) {
+            console.error('[dsh-langfuse] redaction failed:', error);
+            return '[REDACTED]';
+        }
+    }
     /** Dynamically import the SDK and wire the isolated export pipeline. */
     async init() {
         try {
-            const [tracing, otel, sdkNode] = await Promise.all([
+            const [tracing, otel, sdkNode, api] = await Promise.all([
                 import('@langfuse/tracing'),
                 import('@langfuse/otel'),
                 import('@opentelemetry/sdk-trace-node'),
+                import('@opentelemetry/api'),
             ]);
+            const environment = this.config.environment || process.env.LANGFUSE_TRACING_ENVIRONMENT || undefined;
+            const maskedEnvironment = this.mask(environment);
             const processor = new otel.LangfuseSpanProcessor({
                 publicKey: this.config.publicKey,
                 secretKey: this.config.secretKey,
                 baseUrl: this.config.baseUrl,
+                environment: maskedEnvironment === environment ? environment : 'redacted',
+                release: this.mask(this.config.release || process.env.LANGFUSE_RELEASE || undefined),
+                mediaUploadEnabled: !!this.config.captureMedia &&
+                    this.config.captureContent !== false &&
+                    !/^(false|0)$/i.test(process.env.LANGFUSE_MEDIA_UPLOAD_ENABLED ?? ''),
+                mask: ({ data }) => {
+                    // OTEL attributes arrive as serialized JSON; configured field names
+                    // still need structural redaction at the final exporter boundary.
+                    if (typeof data === 'string') {
+                        let parsed;
+                        try {
+                            parsed = JSON.parse(data);
+                        }
+                        catch {
+                            /* plain text */
+                        }
+                        if (parsed !== undefined)
+                            return JSON.stringify(this.mask(parsed));
+                    }
+                    return this.mask(data);
+                },
             });
             this.provider = new sdkNode.NodeTracerProvider({ spanProcessors: [processor] });
             // Isolated provider, not provider.register(): the process-global OTEL
             // provider stays untouched. Set `tracing` last — observation methods
             // gate on it, and it must never be visible before the provider is wired.
             tracing.setLangfuseTracerProvider(this.provider);
+            this.api = api;
             this.tracing = tracing;
         }
         catch (error) {
@@ -131,19 +235,37 @@ export class LangfuseReporter {
     stampTraceContext(observation, context) {
         if (!context)
             return;
-        observation.otelSpan.setAttribute('langfuse.trace.name', context.traceName);
+        if (context.traceName)
+            observation.otelSpan.setAttribute('langfuse.trace.name', this.mask(context.traceName));
         if (context.sessionId)
-            observation.otelSpan.setAttribute('session.id', context.sessionId);
+            observation.otelSpan.setAttribute('session.id', this.mask(context.sessionId));
+        if (context.userId)
+            observation.otelSpan.setAttribute('user.id', this.mask(context.userId));
+        if (context.tags?.length)
+            observation.otelSpan.setAttribute('langfuse.trace.tags', this.mask(context.tags));
         this.traceContext.set(observation, context);
     }
     /** Open a trace (one per agent turn, or one-off for session-less calls). */
     openTrace(input) {
-        if (!this.tracing)
+        const tracing = this.tracing;
+        const api = this.api;
+        if (!tracing || !api)
             return null;
         try {
-            const root = this.tracing.startObservation(input.name, { metadata: input.metadata }, { parentSpanContext: NO_PARENT });
-            this.stampTraceContext(root, { sessionId: input.sessionId, traceName: input.name });
-            this.roots.add(root);
+            const parent = parentContext(input.context?.traceparent);
+            input = this.mask(input);
+            const root = api.context.with(api.ROOT_CONTEXT, () => tracing.startObservation(input.name, { metadata: input.metadata }, { parentSpanContext: parent }));
+            this.stampTraceContext(root, {
+                sessionId: input.sessionId,
+                traceName: parent === NO_PARENT ? input.name : undefined,
+                userId: input.context?.userId ?? this.config.userId,
+                tags: input.context?.tags ?? this.config.tags,
+            });
+            // External traces retain their owner's trace name and trace-level IO.
+            if (parent === NO_PARENT)
+                this.roots.add(root);
+            else
+                root.otelSpan.setAttribute(tracing.LangfuseOtelSpanAttributes.IS_APP_ROOT, false);
             return root;
         }
         catch (error) {
@@ -153,15 +275,17 @@ export class LangfuseReporter {
     }
     /** Open a generation (one per LLM call) under any observation parent. */
     startGeneration(parent, input) {
-        if (!parent)
+        const api = this.api;
+        if (!parent || !api)
             return null;
         try {
-            const generation = parent.startObservation(input.name, {
+            input = this.mask(input);
+            const generation = api.context.with(api.ROOT_CONTEXT, () => parent.startObservation(input.name, {
                 model: input.model,
                 input: input.input,
                 modelParameters: input.modelParameters,
                 metadata: input.metadata,
-            }, { asType: 'generation' });
+            }, { asType: 'generation' }));
             this.stampTraceContext(generation, this.traceContext.get(parent));
             return generation;
         }
@@ -170,10 +294,11 @@ export class LangfuseReporter {
             return null;
         }
     }
-    endGeneration(generation, update) {
+    endGeneration(generation, update, endedAt) {
         if (!generation)
             return;
         try {
+            update = this.mask(update);
             generation.update({
                 output: update.output,
                 usageDetails: update.usage ? usageOf(update.usage) : undefined,
@@ -185,7 +310,7 @@ export class LangfuseReporter {
             // v5 observation attributes have no `name` — renames ride the OTEL span.
             if (update.name)
                 generation.otelSpan.updateName(update.name);
-            generation.end();
+            generation.end(endedAt);
         }
         catch (error) {
             console.error('[dsh-langfuse] generation end failed:', error);
@@ -193,13 +318,18 @@ export class LangfuseReporter {
     }
     /** Open a span (one per tool dispatch, or a nested detail span) under any observation parent. */
     startSpan(parent, input) {
-        if (!parent)
+        const api = this.api;
+        if (!parent || !api)
             return null;
         try {
-            const span = parent.startObservation(input.name, {
+            input = this.mask(input);
+            const attributes = {
                 input: input.input,
                 metadata: input.metadata,
-            });
+            };
+            const span = api.context.with(api.ROOT_CONTEXT, () => input.asType === 'tool'
+                ? parent.startObservation(input.name, attributes, { asType: 'tool' })
+                : parent.startObservation(input.name, attributes));
             this.stampTraceContext(span, this.traceContext.get(parent));
             return span;
         }
@@ -220,6 +350,7 @@ export class LangfuseReporter {
         if (!span)
             return;
         try {
+            update = this.mask(update);
             span.update({ input: update.input, metadata: update.metadata });
             if (update.name)
                 span.otelSpan.updateName(update.name);
@@ -237,15 +368,16 @@ export class LangfuseReporter {
      * On trace roots the output also rides the deprecated `langfuse.trace.*`
      * attributes (setTraceIO) for older servers — same rule as updateSpan.
      */
-    endSpan(span, update = {}) {
+    endSpan(span, update = {}, endedAt) {
         if (!span)
             return;
         try {
+            update = this.mask(update);
             span.update(update);
             if (update.output !== undefined && this.roots.has(span)) {
                 span.setTraceIO({ output: update.output });
             }
-            span.end();
+            span.end(endedAt);
         }
         catch (error) {
             console.error('[dsh-langfuse] span end failed:', error);
@@ -270,6 +402,7 @@ export class LangfuseReporter {
         const tracing = this.tracing;
         this.provider = null;
         this.tracing = null;
+        this.api = null;
         if (!provider || !tracing)
             return;
         try {