@amaster.ai/dsh-langfuse 0.1.20 → 0.1.22

package/README.md

@@ -89,11 +89,73 @@ Compaction and session-title calls passing through `llm/stream` already receive
 
 Hook spans appear when a dsh hook bridge emits the paired session records. The current `hook/result` record does not contain stdout; successful script output can appear in the LLM request when the bridge injects it as context, while the hook span reports decision, exit code, duration, and bounded stderr. `SessionStart` runs before the first turn and does not emit these records, so it has no hook span.
 
+### Custom hook producers
+
+Custom bridges use the same `hook/invoked` and `hook/result` session records. The writer and consumer accept an unrestricted `dialect` source string (`gemini`, `codex`, `company-validator`, etc.); admission to the session log still depends on installed runtime invariants. No dialect registration or separate exporter is needed. The plugin observes records only and never executes hooks or changes their decisions.
+
+Import `appendHookEvent`, `HookInvoked`, and `HookResult` from `@amaster.ai/dsh-langfuse`. This typed writer uses the real `session.append` publication path without redeclaring `SessionEventMap`. Its TypeScript adaptation allows custom payloads despite the optional upstream `dsh-hook-protocol` type augmentation; it does not widen that package's runtime contract, runner, or bridge APIs. The writer validates fields, catches append failures, logs `[dsh-langfuse]`, and returns a boolean; telemetry success must never control a hook's decision.
+
+**Runtime compatibility boundary (verified with `dsh-hook-protocol@0.2.0-rc.2`).** When its invariant companion is registered and enabled through `dsh-invariants`, it rejects dialects other than `claude-code`/`codex`, invocations outside an open turn, and results after their turn closes or while another turn is open. A type assertion cannot bypass these pre-commit checks. `appendHookEvent` returns `false` for rejected records; they are not committed or published and produce no Langfuse hook observation.
+
+| Runtime composition | Claude Code/Codex pairs in the open turn | Custom dialects | Detached invocations / late results |
+| --- | --- | --- | --- |
+| No active hook-protocol invariant | Supported | Supported | Supported with `invocationId` |
+| Active hook-protocol invariant 0.2.0-rc.2 | Supported | Rejected | Rejected |
+
+The writer never disables or reconfigures invariants. Applications using the active companion need an upstream contract change before adopting custom or late records. The producer still owns hook execution and decisions even when telemetry is rejected. The following custom example requires a runtime composition that admits those records.
+
+```ts
+import { randomUUID } from 'node:crypto';
+import { appendHookEvent } from '@amaster.ai/dsh-langfuse';
+
+// The bridge determines turn/point, executes the command, and owns its outcome.
+const invocationId = randomUUID();
+const identity = { turn, point: 'BeforeAgent', handlerId: 'request-validator', invocationId };
+appendHookEvent(session, 'hook/invoked', {
+  ...identity,
+  dialect: 'company-validator',
+  name: 'Validate request',
+});
+// After execution, including exceptions, timeout or cancellation:
+appendHookEvent(session, 'hook/result', {
+  ...identity,
+  decision: 'pass',
+  outcome: 'success',
+  exitCode: 0,
+  durationMs: 37,
+});
+```
+
+Every custom execution should have a unique `invocationId` and a result emitted in the producer's `finally` path. Both records must share `turn`, `point`, `handlerId`, and `invocationId`. `handlerId` identifies the configured handler; `invocationId` identifies one execution, including concurrent executions of that handler. Optional `name` labels the hook. Unmatched/skipped hooks must not emit an invocation: these records represent actual execution. The producer bounds `stderrSummary` and omits unavailable optional fields rather than writing `undefined` into the JSON session log.
+
+`hook/result.outcome` describes process execution independently of `decision`: `success` is normal completion, `failed` is execution failure, `timed-out` is deadline termination, and `cancelled` is cancellation. The producer maps its process semantics to these values; an exit code expressing a policy denial can still be a successful execution. Optional `exitCode` and `signal` preserve process details. Failed/timed-out executions receive ERROR observations, cancellation receives WARNING, and successful executions retain decision-based warnings. Records without an explicit outcome preserve the legacy exit-code interpretation (0/2 are non-error). The plugin records `durationMs` and never infers a timeout or cancellation from a missing exit code.
+
+For records admitted by the session, runs with `invocationId` retain correlation after `turn/end` and a subsequent `turn/start`; a late result ends the original hook span without attaching it to the new turn. Invocations arriving outside their originating turn receive a separate root with `detachedHook: true` and the original turn number; their result ends both hook and root so they export without waiting for another turn. These detached hook roots do not consume `langfuse/context` queued for the next turn. Detached producers must finish reporting before the session is disposed. Session disposal/plugin unload closes unfinished runs with `hook result not observed`; this means missing telemetry, not proof of execution failure. Legacy records without `invocationId` still close at turn boundaries.
+
+Custom names and stderr follow `captureContent`; credentials and `redactFields` are masked by the existing reporter. New outcomes, correlation IDs, dialect, point, duration and process metadata remain available with content capture disabled. The event writer does not sanitize the session log itself: application producers remain responsible for what they persist. Applications must explicitly replace any private hook events with this standard producer path, reuse the same `invocationId`, and report the actual outcome; installing/upgrading the plugin does not adapt an application bridge or repair historical Trace overview IO (Issue #59).
+
 Observability is a no-throw seam: backend errors are logged with a `[dsh-langfuse]` prefix and never escape into the agent loop.
 
 ## Compatibility
 
-Run `pnpm test:sdk` at the repository root for a secrets-free check using the real Langfuse SDK and the existing local OTLP receiver. It verifies export, native tool type, SDK label redaction, ambient-baggage isolation, external trace ownership, and agent-less tool unload. It is separate from the mocked unit suite and the gateway/real-Langfuse E2E legs.
+Run `pnpm test:sdk` at the repository root for a secrets-free check using the real Langfuse SDK and the existing local OTLP receiver. It verifies export, native tool type, SDK label redaction, ambient-baggage isolation, external trace ownership, agent-less tool unload, and custom hook export across turn closure. It is separate from the mocked unit suite and the gateway/real-Langfuse E2E legs.
+
+### Trace overview compatibility
+
+Overall input/output belongs to the `dsh-turn` root observation. The root is exported when the turn ends; while a long turn is running, child observations can already be visible before the root appears. After completion, allow asynchronous ingestion to settle.
+
+The plugin also sends deprecated `langfuse.trace.input` / `langfuse.trace.output` attributes on independently owned roots for legacy Trace overview compatibility. It never writes these fields when joining an externally owned trace. [Langfuse recommends root-observation IO](https://langfuse.com/faq/all/empty-trace-input-and-output); empty Trace overview fields alone do not establish missing observation data.
+
+Real Trace/root API read-back was verified on self-hosted Langfuse **3.172.1** with JS SDK **5.10.1** and **5.11.1**. This verifies new synthetic turns, including children exported before the root and an external observation joining the trace. It does not establish compatibility for every server version or repair historical traces. [Issue #59](https://github.com/TGYD-helige/dsh-plugins/issues/59) contains a historical SDK 5.11.1 trace whose root IO is present but Trace IO remains empty; the cause of that discrepancy has not been reproduced on new turns. For that trace, select the `dsh-turn` observation to read the overall request and response.
+
+To verify your deployment without an LLM, set `LANGFUSE_PUBLIC_KEY`, `LANGFUSE_SECRET_KEY`, and `LANGFUSE_BASE_URL` in the process environment, then run:
+
+```bash
+pnpm --filter @amaster.ai/dsh-langfuse build
+node .github/scripts/integration/lib/langfuse-trace-io-check.mjs
+```
+
+The check creates a synthetic trace in environment `ci` and polls both v1 APIs within a shared two-minute deadline, retrying transient network errors, HTTP 429 (honoring `Retry-After` seconds), and 5xx responses. It reads each observation directly by ID, retrying 404 while ingestion settles; other permanent 4xx errors fail immediately. It explicitly requests Trace IO (`fields=core,io`), rather than relying on server-configured default fields. It first verifies the owner's root and Trace IO, then exports an external observation and waits for its distinct IO in the same trace while checking the owner's Trace IO remains intact. Credentials and response bodies are not printed. A root-only success does not pass this check.
 
 Pinned dsh/cordis versions live in the [root compat matrix](../../README.md#compatibility). Event payloads ride pre-release dsh APIs — check the `TODO(verify)` markers in `src/` before upgrading dsh.
 

package/lib/hooks.js

@@ -0,0 +1,52 @@
+function baseFields(data) {
+    if (!data || typeof data !== 'object')
+        return false;
+    const value = data;
+    return (Number.isSafeInteger(value.turn) &&
+        value.turn >= 0 &&
+        typeof value.point === 'string' &&
+        value.point.length > 0 &&
+        typeof value.handlerId === 'string' &&
+        value.handlerId.length > 0 &&
+        (value.invocationId === undefined ||
+            (typeof value.invocationId === 'string' && value.invocationId.length > 0)));
+}
+export function isHookInvoked(data) {
+    return (baseFields(data) &&
+        typeof data.dialect === 'string' &&
+        data.dialect.length > 0 &&
+        (data.matcher === undefined || typeof data.matcher === 'string') &&
+        (data.name === undefined || typeof data.name === 'string'));
+}
+export function isHookResult(data) {
+    return (baseFields(data) &&
+        typeof data.decision === 'string' &&
+        Number.isFinite(data.durationMs) &&
+        data.durationMs >= 0 &&
+        (data.outcome === undefined ||
+            ['success', 'failed', 'timed-out', 'cancelled'].includes(data.outcome)) &&
+        (data.exitCode === undefined || Number.isSafeInteger(data.exitCode)) &&
+        (data.signal === undefined || typeof data.signal === 'string') &&
+        (data.stderrSummary === undefined || typeof data.stderrSummary === 'string'));
+}
+/**
+ * Append standard telemetry without changing hook decisions.
+ * Runtime invariants still apply; rejected custom/late records return false.
+ */
+export function appendHookEvent(session, type, data) {
+    try {
+        if (!(type === 'hook/invoked'
+            ? isHookInvoked(data)
+            : type === 'hook/result' && isHookResult(data))) {
+            throw new TypeError('invalid hook telemetry fields');
+        }
+        // Widen TypeScript only; session.append still enforces installed runtime invariants.
+        const append = session.append;
+        append.call(session, type, data);
+        return true;
+    }
+    catch (error) {
+        console.error('[dsh-langfuse] hook reporting failed:', error);
+        return false;
+    }
+}

package/lib/index.js

@@ -30,7 +30,9 @@
  */
 import Schema from '@deepseek-ai/schemastery';
 import { LangfuseReporter } from './client.js';
+import { isHookInvoked, isHookResult } from './hooks.js';
 import { displayBlocks } from './media.js';
+export { appendHookEvent } from './hooks.js';
 export const name = 'dsh-langfuse';
 /**
  * Waterfall listeners only need the events to exist; no hard service
@@ -270,11 +272,19 @@ export function apply(ctx, config) {
         if (state)
             state.hadErrors = true;
     }
-    function closeHooks(sessionId) {
-        for (const span of openHooks.get(sessionId)?.values() ?? []) {
-            reporter.endSpan(span, { level: 'WARNING', statusMessage: 'hook result not observed' });
+    function closeHooks(sessionId, legacyOnly = false) {
+        const pending = openHooks.get(sessionId);
+        for (const [key, hook] of pending ?? []) {
+            // Per-execution IDs allow results after turn closure without colliding with the next turn.
+            if (legacyOnly && hook.persistent)
+                continue;
+            const update = { level: 'WARNING', statusMessage: 'hook result not observed' };
+            reporter.endSpan(hook.span, update);
+            reporter.endSpan(hook.root ?? null, update);
+            pending?.delete(key);
         }
-        openHooks.delete(sessionId);
+        if (pending?.size === 0)
+            openHooks.delete(sessionId);
     }
     /**
      * The trace an observation for `sessionId` belongs to: the session's open
@@ -328,7 +338,13 @@ export function apply(ctx, config) {
                 delegationDepth: session.header.delegationDepth,
             },
         });
-        children.set(childId, { parentSessionId, span, hasInput: false });
+        children.set(childId, {
+            parentSessionId,
+            span,
+            hasInput: false,
+            turnTrace: children.get(parentSessionId)?.turnTrace ??
+                sessions.get(rootSessionId(parentSessionId) ?? '')?.trace,
+        });
     });
     ctx.on('subagent/start', (info) => {
         const child = children.get(info.id);
@@ -364,16 +380,30 @@ export function apply(ctx, config) {
         const hookEvent = event;
         if (hookEvent.type === 'hook/invoked') {
             const data = hookEvent.data;
-            if (!data || typeof data.handlerId !== 'string' || typeof data.point !== 'string')
+            if (!isHookInvoked(data))
                 return;
             const pending = openHooks.get(sessionId) ?? new Map();
-            if (pending.has(data.handlerId)) {
-                reporter.endSpan(pending.get(data.handlerId) ?? null, {
+            const key = data.invocationId ? `run:${data.invocationId}` : `handler:${data.handlerId}`;
+            if (pending.has(key)) {
+                reporter.endSpan(pending.get(key)?.span ?? null, {
                     level: 'WARNING',
                     statusMessage: 'hook result not observed',
                 });
             }
-            pending.set(data.handlerId, reporter.startSpan(parentFor(sessionId), {
+            const previous = pending.get(key);
+            if (previous?.root)
+                reporter.endSpan(previous.root, { level: 'WARNING' });
+            const child = children.get(sessionId);
+            const inTurn = child ? child.turn === data.turn : sessions.get(sessionId)?.turn === data.turn;
+            const root = data.invocationId && !inTurn
+                ? // A late hook must not consume context queued for the next turn.
+                    reporter.openTrace({
+                        name: config.traceName,
+                        sessionId,
+                        metadata: { turn: data.turn, detachedHook: true },
+                    })
+                : undefined;
+            const span = reporter.startSpan(root === undefined ? parentFor(sessionId) : root, {
                 name: `hook:${data.point}`,
                 metadata: {
                     turn: data.turn,
@@ -381,43 +411,72 @@ export function apply(ctx, config) {
                     dialect: data.dialect,
                     matcher: data.matcher,
                     handlerId: data.handlerId,
+                    ...(data.invocationId ? { invocationId: data.invocationId } : {}),
+                    ...(config.captureContent && data.name ? { name: data.name } : {}),
                 },
-            }));
+            });
+            pending.set(key, {
+                span,
+                root,
+                turn: data.turn,
+                point: data.point,
+                handlerId: data.handlerId,
+                persistent: !!data.invocationId,
+                turnTrace: root === undefined ? (child?.turnTrace ?? sessions.get(sessionId)?.trace) : undefined,
+            });
             openHooks.set(sessionId, pending);
             return;
         }
         if (hookEvent.type === 'hook/result') {
             const data = hookEvent.data;
-            if (!data || typeof data.handlerId !== 'string' || typeof data.decision !== 'string')
+            if (!isHookResult(data))
                 return;
             const pending = openHooks.get(sessionId);
-            const span = pending?.get(data.handlerId);
+            const key = data.invocationId ? `run:${data.invocationId}` : `handler:${data.handlerId}`;
+            const hook = pending?.get(key);
+            if (!hook ||
+                hook.handlerId !== data.handlerId ||
+                hook.turn !== data.turn ||
+                hook.point !== data.point)
+                return;
+            const span = hook.span;
+            const failed = data.outcome
+                ? data.outcome === 'failed' || data.outcome === 'timed-out'
+                : data.exitCode === undefined || (data.exitCode !== 0 && data.exitCode !== 2);
+            const level = failed
+                ? 'ERROR'
+                : data.outcome === 'cancelled' || ['block', 'deny', 'stop'].includes(data.decision)
+                    ? 'WARNING'
+                    : 'DEFAULT';
             if (span) {
-                if (data.exitCode === undefined || (data.exitCode !== 0 && data.exitCode !== 2))
+                if (failed && sessions.get(rootSessionId(sessionId) ?? '')?.trace === hook.turnTrace)
                     noteError(sessionId);
                 reporter.endSpan(span, {
                     output: {
                         decision: data.decision,
+                        ...(data.outcome ? { outcome: data.outcome } : {}),
+                        ...(data.signal ? { signal: data.signal } : {}),
                         exitCode: data.exitCode,
                         stderrSummary: config.captureContent ? data.stderrSummary : undefined,
                     },
-                    level: data.exitCode === undefined || (data.exitCode !== 0 && data.exitCode !== 2)
-                        ? 'ERROR'
-                        : data.decision === 'block' || data.decision === 'deny' || data.decision === 'stop'
-                            ? 'WARNING'
-                            : 'DEFAULT',
+                    level,
                     metadata: { durationMs: data.durationMs },
                 });
             }
-            pending?.delete(data.handlerId);
+            reporter.endSpan(hook.root ?? null, { level });
+            pending?.delete(key);
             if (pending?.size === 0)
                 openHooks.delete(sessionId);
             return;
         }
         if (event.type === 'turn/end')
-            closeHooks(sessionId);
+            closeHooks(sessionId, true);
         const child = children.get(sessionId);
         if (child) {
+            if (event.type === 'turn/start')
+                child.turn = event.data.turn;
+            if (event.type === 'turn/end')
+                child.turn = undefined;
             // The delegated prompt (the child's first user/message) is the
             // subagent span's input; the descriptor enriches the span name.
             if (event.type === 'user/message' && !child.hasInput && config.captureContent) {
@@ -441,13 +500,13 @@ export function apply(ctx, config) {
                 // whose `turn/end` was never seen (crash-orphaned): end and replace
                 // it — v5 spans only export on end, so the stale root must close to
                 // keep what it already recorded.
-                closeHooks(sessionId);
+                closeHooks(sessionId, true);
                 const stale = sessions.get(sessionId);
                 if (stale)
                     afterMedia(stale.trace, (endedAt) => reporter.endSpan(stale.trace, {}, endedAt));
                 const trace = openTrace(sessionId, { turn: event.data.turn });
                 if (trace)
-                    sessions.set(sessionId, { trace, hasInput: false });
+                    sessions.set(sessionId, { trace, hasInput: false, turn: event.data.turn });
                 break;
             }
             case 'user/message': {

package/lib/types/hooks.d.ts

@@ -0,0 +1,36 @@
+import type { Session } from '@deepseek-ai/dsh-session';
+/** Standard log-only hook records; producers own matching, execution and decisions. */
+export interface HookInvoked {
+    turn: number;
+    point: string;
+    dialect: string;
+    matcher?: string;
+    handlerId: string;
+    /** Unique per execution, shared with its result; required for concurrent runs. */
+    invocationId?: string;
+    name?: string;
+}
+export interface HookResult {
+    turn: number;
+    point: string;
+    handlerId: string;
+    invocationId?: string;
+    decision: string;
+    outcome?: 'success' | 'failed' | 'timed-out' | 'cancelled';
+    exitCode?: number;
+    signal?: string;
+    stderrSummary?: string;
+    durationMs: number;
+}
+interface HookEventMap {
+    'hook/invoked': HookInvoked;
+    'hook/result': HookResult;
+}
+export declare function isHookInvoked(data: unknown): data is HookInvoked;
+export declare function isHookResult(data: unknown): data is HookResult;
+/**
+ * Append standard telemetry without changing hook decisions.
+ * Runtime invariants still apply; rejected custom/late records return false.
+ */
+export declare function appendHookEvent<K extends keyof HookEventMap>(session: Session, type: K, data: HookEventMap[K]): boolean;
+export {};

package/lib/types/index.d.ts

@@ -32,6 +32,7 @@ import type { Context } from '@deepseek-ai/cordis';
 import Schema from '@deepseek-ai/schemastery';
 import { type LangfuseTraceContext } from './client.js';
 export type { LangfuseTraceContext } from './client.js';
+export { appendHookEvent, type HookInvoked, type HookResult } from './hooks.js';
 declare module '@deepseek-ai/cordis' {
     interface Events {
         /** Bind standard labels/parent context before a session's next trace starts. */

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@amaster.ai/dsh-langfuse",
-  "version": "0.1.20",
+  "version": "0.1.22",
   "description": "Langfuse observability plugin for DeepSeek Harness: one generation per LLM call with a nested llm-request span, one span per tool call, one trace per session turn, subagent child sessions nested under the parent's tree.",
   "type": "module",
   "main": "lib/index.js",
@@ -67,6 +67,8 @@
     "@deepseek-ai/cordis": "^4.0.4",
     "@deepseek-ai/dsh-agent": "0.2.0-rc.2",
     "@deepseek-ai/dsh-attachment": "0.2.0-rc.2",
+    "@deepseek-ai/dsh-hook-protocol": "0.2.0-rc.2",
+    "@deepseek-ai/dsh-invariants": "0.2.0-rc.2",
     "@deepseek-ai/dsh-llm": "0.2.0-rc.2",
     "@deepseek-ai/dsh-ptc-runtime": "0.2.0-rc.2",
     "@deepseek-ai/dsh-session": "0.2.0-rc.2",