pi-langfuse 1.5.19 → 1.6.1

package/README.md

@@ -34,6 +34,7 @@ Langfuse observability extension for [Pi Coding Agent](https://github.com/earend
    - Langfuse public key, starting with `pk-lf-...`
    - Langfuse secret key, starting with `sk-lf-...`
    - Langfuse host, defaulting to `https://cloud.langfuse.com`
+   - Optional Langfuse user ID for per-user usage and cost aggregation
 
 3. Run Pi normally:
 
@@ -63,7 +64,7 @@ To inspect the active configuration without exposing secrets:
 /langfuse-status
 ```
 
-The status command reports the config source, host, masked public key, capture policy, active-run state, config path, and last runtime error.
+The status command reports the config source, host, masked public key, whether a user ID is configured, capture policy, active-run state, config path, and last runtime error. It never prints the user ID itself.
 
 ### Method 2: Environment variables
 
@@ -73,9 +74,11 @@ Set these before starting Pi:
 export LANGFUSE_PUBLIC_KEY="pk-lf-xxxx"
 export LANGFUSE_SECRET_KEY="sk-lf-xxxx"
 export LANGFUSE_BASE_URL="https://cloud.langfuse.com"  # optional; LANGFUSE_HOST is also supported
+export LANGFUSE_USER_ID="user-123"                     # optional; max 200 characters
 ```
 
 Saved config takes precedence. Environment variables are only used when `~/.pi/agent/pi-langfuse/config.json` is missing or incomplete.
+If saved credentials exist but `userId` is absent, `LANGFUSE_USER_ID` is still used. An explicitly saved `userId` takes precedence.
 
 For short-lived SDK hosts, set the bounded final score-delivery attempt during shutdown:
 
@@ -222,10 +225,13 @@ Create or update `~/.pi/agent/pi-langfuse/config.json`:
   "publicKey": "pk-lf-xxxx",
   "secretKey": "sk-lf-xxxx",
   "host": "https://cloud.langfuse.com",
+  "userId": "user-123",
   "privacyPreset": "conversations"
 }
 ```
 
+`userId` is optional, explicit-only, and limited to 200 characters. The extension does not infer it from the local operating-system account. When configured, it is propagated to every observation in the trace so Langfuse can provide per-user filtering and cost aggregation.
+
 Fine-grained capture flags can also be persisted:
 
 ```json
@@ -269,6 +275,12 @@ This command makes a timeout-bounded authenticated request to Langfuse and, if i
 - Tool runs appear as tool observations with arguments, results, and error state.
 - LLM requests appear as generation observations, including usage and cost when the provider exposes them.
   Reasoning tokens are reported as their own usage bucket when `PI_LANGFUSE_SPLIT_REASONING_TOKENS` is enabled.
+- Pi 0.86 transcript-aware prompt/tool state appears as `system-state` events. Generations reference the
+  effective prompt/tool state hashes, active tool count, update transport, and cache hit ratio.
+- Automatic retries are grouped under `agent-attempt` spans and the trace remains open until `agent_settled`.
+- Cache-warming decisions and persisted `cache_warm` usage are reported, including standalone session-grouped
+  traces when warming happens while no user-request trace is active.
+- Compaction is recorded as `session-compaction`, with a nested `compaction-summary` generation when usage is available.
 - Trace-level scores include tool counts, tool success rate, and whether the run had errors.
 
 The package also includes a Langfuse CLI skill, so Langfuse data can be queried directly from Pi:

package/README_CN.md

@@ -34,6 +34,7 @@
    - Langfuse 公钥，以 `pk-lf-...` 开头
    - Langfuse 密钥，以 `sk-lf-...` 开头
    - Langfuse 主机地址，默认 `https://cloud.langfuse.com`
+   - 可选的 Langfuse 用户 ID，用于按用户汇总用量和成本
 
 3. 正常运行 Pi：
 
@@ -63,7 +64,7 @@ Langfuse API 密钥可在 **Langfuse Cloud** -> **Settings** -> **API Keys** 中
 /langfuse-status
 ```
 
-状态命令会显示配置来源、主机地址、脱敏后的公钥、采集策略、是否有活跃运行、配置文件路径，以及最近一次运行时错误。
+状态命令会显示配置来源、主机地址、脱敏后的公钥、是否已配置用户 ID、采集策略、是否有活跃运行、配置文件路径，以及最近一次运行时错误，但不会输出用户 ID 本身。
 
 ### 方式 2：环境变量
 
@@ -73,9 +74,11 @@ Langfuse API 密钥可在 **Langfuse Cloud** -> **Settings** -> **API Keys** 中
 export LANGFUSE_PUBLIC_KEY="pk-lf-xxxx"
 export LANGFUSE_SECRET_KEY="sk-lf-xxxx"
 export LANGFUSE_BASE_URL="https://cloud.langfuse.com"  # 可选；也支持 LANGFUSE_HOST
+export LANGFUSE_USER_ID="user-123"                     # 可选；最长 200 个字符
 ```
 
 保存的配置优先级更高。只有当 `~/.pi/agent/pi-langfuse/config.json` 缺失或不完整时，扩展才会使用环境变量。
+如果已保存凭据但没有 `userId`，仍会读取 `LANGFUSE_USER_ID`；显式保存的 `userId` 优先级更高。
 
 对于短生命周期的 SDK 宿主，可设置关闭时最终分数发送尝试的上限：
 
@@ -203,10 +206,13 @@ export PI_LANGFUSE_SPLIT_REASONING_TOKENS=true
   "publicKey": "pk-lf-xxxx",
   "secretKey": "sk-lf-xxxx",
   "host": "https://cloud.langfuse.com",
+  "userId": "user-123",
   "privacyPreset": "conversations"
 }
 ```
 
+`userId` 是可选的显式配置，最长 200 个字符。扩展不会从本机操作系统账户推断该值。配置后，它会传播到 trace 中的每个 observation，以支持 Langfuse 按用户筛选并汇总成本。
+
 也可以持久化细粒度采集开关：
 
 ```json
@@ -250,6 +256,12 @@ pi list
 - 工具执行会以工具观察节点展示参数、结果和错误状态。
 - 模型请求会以生成观察节点展示；如果提供商暴露相关信息，还会包含用量和成本。
   开启 `PI_LANGFUSE_SPLIT_REASONING_TOKENS` 后，推理 token 会作为独立用量桶上报。
+- Pi 0.86 的 transcript-aware system/tool 状态会作为 `system-state` 事件上报；generation 会引用有效的
+  prompt/tool 状态指纹、活动工具数、更新传输方式和缓存命中率。
+- 自动重试会归入 `agent-attempt` span，trace 会持续到 `agent_settled` 才结束。
+- cache warming 决策及持久化的 `cache_warm` 用量会被记录；若 warming 发生时没有活动的用户请求 trace，
+  会创建按相同 session 分组的独立 trace。
+- compaction 会记录为 `session-compaction`；若存在摘要用量，则包含 `compaction-summary` generation。
 - trace 级别会记录工具调用次数、工具成功率和是否出现错误。
 
 此包还包含一个内置 Langfuse 技能，可直接在 Pi 中查询 Langfuse 数据：

package/index.ts

@@ -14,10 +14,23 @@ import { state, resetRunState, runWithSession, setCurrentSession } from "./src/s
 import { ensureConfig, promptForConfig, loadConfig } from "./src/config.js";
 import { shutdownRuntime } from "./src/langfuse.js";
 import { handleLangfusePrivacyCommand, handleLangfuseStatusCommand, handleLangfuseTestCommand } from "./src/commands.js";
-import { getMessageFromEvent, extractAssistantOutput, getCapturePolicy } from "./src/utils.js";
-import { applyCapturePolicy } from "./src/capture-policy.js";
-import { startAgentRun, finishAgentRun, recordSystemPrompt } from "./src/handlers/agent.js";
+import { getMessageFromEvent, extractAssistantOutput } from "./src/utils.js";
+import {
+  startAgentRun,
+  finishAgentRun,
+  finishAgentAttempt,
+  cancelAgentRun,
+  recordSystemPrompt,
+  startAgentAttempt,
+} from "./src/handlers/agent.js";
 import { startTurnObservation, finishTurnObservation } from "./src/handlers/turn.js";
+import { initializeSystemStateTracking, recordSystemState } from "./src/handlers/system-state.js";
+import {
+  initializeUsageTracking,
+  recordCacheWarmingDecision,
+  recordNewUsageEntries,
+} from "./src/handlers/cache.js";
+import { recordSessionCompaction } from "./src/handlers/session.js";
 import {
   startGeneration,
   updateGenerationMetadata,
@@ -28,7 +41,6 @@ import {
 import {
   startToolObservation,
   finishToolObservation,
-  closeDanglingObservations,
 } from "./src/handlers/tool.js";
 
 // ============================================
@@ -36,6 +48,7 @@ import {
 // ============================================
 
 export default async function (pi: ExtensionAPI) {
+  const asRecord = (value: object): Record<string, unknown> => value as unknown as Record<string, unknown>;
   if (!state.config) {
     state.config = loadConfig();
   }
@@ -97,6 +110,8 @@ export default async function (pi: ExtensionAPI) {
     state.setupAttemptedThisSession = false;
     await ensureConfig(ctx);
     resetRunState();
+    initializeSystemStateTracking(ctx);
+    initializeUsageTracking(ctx);
   }));
 
   pi.on("model_select", async (event, ctx) => withSession(ctx, async () => {
@@ -105,84 +120,87 @@ export default async function (pi: ExtensionAPI) {
   }));
 
   pi.on("before_agent_start", async (event, ctx) => withSession(ctx, async () => {
-    await startAgentRun(event, ctx);
+    await recordNewUsageEntries(ctx);
+    await startAgentRun(asRecord(event), ctx);
   }));
 
   pi.on("agent_start", async (event, ctx) => withSession(ctx, async () => {
     if (!state.agentState?.root) {
-      await startAgentRun(event, ctx);
+      await startAgentRun(asRecord(event), ctx);
     }
+    await startAgentAttempt();
     // The system prompt is only final here: before_agent_start handlers that
     // run after this extension may still rewrite it.
     await recordSystemPrompt(ctx);
+    await recordSystemState(ctx, pi.getActiveTools());
+  }));
+
+  pi.on("cache_warming_decision", async (event, ctx) => withSession(ctx, async () => {
+    await recordNewUsageEntries(ctx);
+    await recordCacheWarmingDecision(asRecord(event), ctx);
   }));
 
   pi.on("turn_start", async (event, ctx) => withSession(ctx, async () => {
-    await startTurnObservation(event);
+    await startTurnObservation(asRecord(event));
   }));
 
   pi.on("before_provider_request", async (event, ctx) => withSession(ctx, async () => {
-    await startGeneration(event);
+    await startGeneration(asRecord(event));
   }));
 
   pi.on("after_provider_response", async (event, ctx) => withSession(ctx, async () => {
-    updateGenerationMetadata(event);
+    updateGenerationMetadata(asRecord(event));
   }));
 
   pi.on("message_update", async (event, ctx) => withSession(ctx, async () => {
-    recordTTFT(event);
-    const message = getMessageFromEvent(event);
+    recordTTFT(asRecord(event));
+    const message = getMessageFromEvent(asRecord(event));
     if (message?.role === "assistant" && state.agentState) {
       state.agentState.latestAssistantOutput = extractAssistantOutput(message);
     }
   }));
 
   pi.on("message_end", async (event, ctx) => withSession(ctx, async () => {
-    await finishGenerationFromMessage(event);
+    await finishGenerationFromMessage(asRecord(event));
   }));
 
   pi.on("tool_execution_start", async (event, ctx) => withSession(ctx, async () => {
-    await startToolObservation(event);
+    await startToolObservation(asRecord(event));
   }));
 
   pi.on("tool_call", async (event, ctx) => withSession(ctx, async () => {
-    await startToolObservation(event);
+    await startToolObservation(asRecord(event));
   }));
 
   pi.on("tool_result", async (event, ctx) => withSession(ctx, async () => {
-    await finishToolObservation(event);
+    await finishToolObservation(asRecord(event));
   }));
 
   pi.on("tool_execution_end", async (event, ctx) => withSession(ctx, async () => {
-    await finishToolObservation(event);
+    await finishToolObservation(asRecord(event));
   }));
 
   pi.on("turn_end", async (event, ctx) => withSession(ctx, async () => {
     state.turnCount++;
-    const message = getMessageFromEvent(event);
+    const record = asRecord(event);
+    const message = getMessageFromEvent(record);
     if (message?.role === "assistant") {
-      await createFallbackGenerationFromTurn(event, message);
-      await finishGenerationFromMessage(event);
+      await createFallbackGenerationFromTurn(record, message);
+      await finishGenerationFromMessage(record);
     }
-    finishTurnObservation(event);
+    finishTurnObservation(record);
   }));
 
   pi.on("agent_end", async (event, ctx) => withSession(ctx, async () => {
-    await finishAgentRun(event);
-    const sessionId = state.currentSessionId;
-    try {
-      await shutdownRuntime(sessionId);
-    } catch (error) {
-      console.warn("📊 Langfuse: Shutdown failed", error);
-    }
+    finishAgentAttempt(asRecord(event));
+  }));
+
+  pi.on("agent_settled", async (_event, ctx) => withSession(ctx, async () => {
+    await finishAgentRun();
   }));
 
   const handleSessionInterruption = (reason: string) => {
-    if (state.agentState?.root) {
-      closeDanglingObservations(reason);
-      state.agentState.root.update({ metadata: { completed: false, cancelled: true } }).end();
-    }
-    resetRunState();
+    cancelAgentRun(reason);
   };
 
   pi.on("session_before_switch", async (_event, ctx) => {
@@ -200,26 +218,11 @@ export default async function (pi: ExtensionAPI) {
   });
 
   pi.on("session_compact", async (event, ctx) => withSession(ctx, async () => {
-    if (state.agentState?.root) {
-      const parent = state.agentState.activeTurn ?? state.agentState.root;
-      try {
-        const observation = parent.startObservation ? parent.startObservation(
-          "session_compact", 
-          {
-            level: "DEFAULT",
-            statusMessage: "Context was compacted",
-            metadata: applyCapturePolicy({ metadata: { ...event } }, getCapturePolicy()).metadata
-          }, 
-          { asType: "span" }
-        ) : undefined;
-        observation?.end();
-      } catch (e) {
-        // ignore
-      }
-    }
+    await recordSessionCompaction(asRecord(event));
   }));
 
   pi.on("session_shutdown", async (_event, ctx) => withSession(ctx, async () => {
+    await recordNewUsageEntries(ctx);
     handleSessionInterruption("Session shutdown before agent completed");
     await shutdownRuntime();
   }));

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "pi-langfuse",
-	"version": "1.5.19",
+	"version": "1.6.1",
 	"description": "Langfuse extension for Pi coding agent",
 	"repository": {
 		"type": "git",
@@ -16,7 +16,6 @@
 	"files": [
 		"index.ts",
 		"src/",
-		"types/",
 		"README.md",
 		"README_CN.md",
 		"image.png",
@@ -52,7 +51,7 @@
 		"@opentelemetry/sdk-trace-base": "^2.0.1"
 	},
 	"peerDependencies": {
-		"@earendil-works/pi-coding-agent": "*"
+		"@earendil-works/pi-coding-agent": ">=0.86.0"
 	},
 	"publishConfig": {
 		"access": "public",
@@ -63,6 +62,7 @@
 		"node": ">=22"
 	},
 	"devDependencies": {
+		"@earendil-works/pi-coding-agent": "^0.86.0",
 		"tsx": "^4.19.0",
 		"typescript": "^6.0.3"
 	}

package/src/commands.ts

@@ -211,6 +211,7 @@ function formatStatus(configPath: string, env: Record<string, string | undefined
     `Source: ${configSource(env, configPath)}`,
     `Host: ${safeConfig?.host ?? config.host}`,
     `Public key: ${safeConfig?.publicKey ?? "[REDACTED_SECRET]"}`,
+    `User ID: ${config.userId ? "configured" : "not set"}`,
     `Config file: ${configPath}`,
     `Privacy preset: ${inferPreset(policy)}`,
     "Capture:",

package/src/config.ts

@@ -8,6 +8,16 @@ import { createCapturePolicy, type EnvLike } from "./capture-policy.js";
 import { createPayloadLimits } from "./limits.js";
 import { createUsageOptions } from "./usage-options.js";
 
+const MAX_LANGFUSE_USER_ID_LENGTH = 200;
+
+function normalizeUserId(value: unknown): string | undefined {
+  if (typeof value !== "string") {
+    return undefined;
+  }
+  const trimmed = value.trim();
+  return trimmed ? trimmed.slice(0, MAX_LANGFUSE_USER_ID_LENGTH) : undefined;
+}
+
 export function loadConfigFromFile(path = CONFIG_PATH, env: EnvLike = process.env as EnvLike): Config | null {
   if (existsSync(path)) {
     try {
@@ -23,6 +33,7 @@ export function loadConfigFromFile(path = CONFIG_PATH, env: EnvLike = process.en
           publicKey: config.publicKey,
           secretKey: config.secretKey,
           host: config.host || DEFAULT_LANGFUSE_HOST,
+          userId: normalizeUserId(config.userId) ?? normalizeUserId(env.LANGFUSE_USER_ID),
           capturePolicy: createCapturePolicy(captureSource),
           limits: createPayloadLimits(env),
           usage: createUsageOptions(captureSource),
@@ -47,6 +58,7 @@ export function loadConfigFromEnv(env: EnvLike = process.env as EnvLike): Config
     publicKey,
     secretKey,
     host: env.LANGFUSE_BASE_URL || env.LANGFUSE_HOST || DEFAULT_LANGFUSE_HOST,
+    userId: normalizeUserId(env.LANGFUSE_USER_ID),
     capturePolicy: createCapturePolicy(env),
     limits: createPayloadLimits(env),
     usage: createUsageOptions(env),
@@ -108,10 +120,12 @@ async function collectConfigFromUI(ctx: any, reason: string): Promise<Config | n
   }
 
   const hostInput = (await ctx.ui.input("Langfuse host:", DEFAULT_LANGFUSE_HOST))?.trim();
+  const userId = normalizeUserId(await ctx.ui.input("Langfuse user ID (optional):", ""));
   return {
     publicKey,
     secretKey,
     host: hostInput || DEFAULT_LANGFUSE_HOST,
+    ...(userId ? { userId } : {}),
     capturePolicy: createCapturePolicy(),
   };
 }

package/src/handlers/agent.ts

@@ -5,6 +5,7 @@ import { shapePayload, truncate, extractFinalAssistant, extractAssistantOutput,
 import { closeDanglingObservations } from "./tool.js";
 import { applyCapturePolicy } from "../capture-policy.js";
 import { collectSourceMetadata } from "../source-metadata.js";
+import { startChildObservation } from "../observation.js";
 
 function stringMetadata(metadata: Record<string, unknown> | undefined): Record<string, string> | undefined {
   if (!metadata) {
@@ -70,6 +71,7 @@ export async function startAgentRun(event: Record<string, unknown>, ctx: any) {
           ...(state.currentModel ? { model: state.currentModel } : {}),
           ...(state.currentProvider ? { provider: state.currentProvider } : {}),
           sessionId: state.currentSessionId || undefined,
+          sessionLeafId: ctx?.sessionManager?.getLeafId?.() || undefined,
         },
       },
       capturePolicy,
@@ -84,11 +86,17 @@ export async function startAgentRun(event: Record<string, unknown>, ctx: any) {
       activeTools: new Map(),
       sourceMetadata,
       providerMetadataByRequest: new Map(),
+      attemptCount: 0,
+      systemStateChangeCount: 0,
+      cacheReadTokens: 0,
+      cacheWriteTokens: 0,
+      uncachedInputTokens: 0,
     };
 
     const root = rt.propagateAttributes(
       {
         sessionId: state.currentSessionId ? truncate(state.currentSessionId, 200) : undefined,
+        userId: state.config?.userId,
         traceName: "pi-agent",
         metadata: stringMetadata(captured.metadata),
       },
@@ -112,6 +120,82 @@ export async function startAgentRun(event: Record<string, unknown>, ctx: any) {
   }
 }
 
+export async function startAgentAttempt() {
+  const agent = state.agentState;
+  if (state.isTracingDisabled || !agent?.root) return;
+
+  if (agent.activeAttempt) {
+    agent.activeAttempt
+      .update({ level: "WARNING", statusMessage: "A new agent attempt started before the previous attempt ended" })
+      .end();
+  }
+
+  try {
+    agent.attemptCount++;
+    agent.activeAttempt = await startChildObservation({
+      parent: agent.root,
+      runtime: getRuntime,
+      name: "agent-attempt",
+      body: { metadata: { attemptIndex: agent.attemptCount } },
+      asType: "span",
+    });
+  } catch (error) {
+    console.warn("📊 Langfuse: Failed to start agent attempt", error);
+  }
+}
+
+export function finishAgentAttempt(event: Record<string, unknown> = {}) {
+  const agent = state.agentState;
+  if (!agent) return;
+  agent.lastAgentEndEvent = event;
+
+  closeDanglingObservations("Agent attempt ended before observation finalized");
+  if (agent.activeTurn) {
+    agent.activeTurn
+      .update({ level: "WARNING", statusMessage: "Agent attempt ended before turn finalized" })
+      .end();
+    agent.activeTurn = undefined;
+  }
+
+  if (!agent.activeAttempt) return;
+  try {
+    const lastAssistant = extractFinalAssistant(event.messages);
+    const captured = applyCapturePolicy(
+      { output: lastAssistant ? extractAssistantOutput(lastAssistant) : undefined },
+      getCapturePolicy(),
+    );
+    agent.activeAttempt.update({ output: captured.output }).end();
+  } catch (error) {
+    console.warn("📊 Langfuse: Failed to finish agent attempt", error);
+  } finally {
+    agent.activeAttempt = undefined;
+  }
+}
+
+export function cancelAgentRun(reason: string) {
+  const agent = state.agentState;
+  if (!agent?.root) {
+    resetRunState();
+    return;
+  }
+
+  closeDanglingObservations(reason);
+  if (agent.activeTurn) {
+    agent.activeTurn.update({ level: "WARNING", statusMessage: reason, metadata: { cancelled: true } }).end();
+    agent.activeTurn = undefined;
+  }
+  if (agent.activeAttempt) {
+    agent.activeAttempt.update({ level: "WARNING", statusMessage: reason, metadata: { cancelled: true } }).end();
+    agent.activeAttempt = undefined;
+  }
+  agent.root.update({
+    level: "WARNING",
+    statusMessage: reason,
+    metadata: { completed: false, cancelled: true },
+  }).end();
+  resetRunState();
+}
+
 /**
  * Records the effective system prompt on the root agent observation.
  *
@@ -161,7 +245,8 @@ export async function finishAgentRun(event: Record<string, unknown> = {}) {
     return;
   }
 
-  const lastAssistant = extractFinalAssistant(event.messages);
+  const finalEvent = Object.keys(event).length > 0 ? event : state.agentState.lastAgentEndEvent ?? {};
+  const lastAssistant = extractFinalAssistant(finalEvent.messages);
   const rawOutput = lastAssistant ? extractAssistantOutput(lastAssistant) : state.agentState.latestAssistantOutput;
   const captured = applyCapturePolicy(
     {
@@ -173,6 +258,19 @@ export async function finishAgentRun(event: Record<string, unknown> = {}) {
         model: state.currentModel || undefined,
         provider: state.currentProvider || undefined,
         totalTools: state.toolCallCount,
+        agentAttemptCount: state.agentState.attemptCount,
+        systemStateChangeCount: state.agentState.systemStateChangeCount,
+        promptStateHash: state.agentState.promptStateHash,
+        toolStateHash: state.agentState.toolStateHash,
+        activeToolCount: state.agentState.activeToolCount,
+        cacheReadTokens: state.agentState.cacheReadTokens,
+        cacheWriteTokens: state.agentState.cacheWriteTokens,
+        uncachedInputTokens: state.agentState.uncachedInputTokens,
+        cacheHitRatio:
+          state.agentState.cacheReadTokens + state.agentState.uncachedInputTokens > 0
+            ? state.agentState.cacheReadTokens /
+              (state.agentState.cacheReadTokens + state.agentState.uncachedInputTokens)
+            : undefined,
         ...computeEvaluationScores(),
       },
     },
@@ -181,6 +279,10 @@ export async function finishAgentRun(event: Record<string, unknown> = {}) {
   const scores = computeEvaluationScores();
 
   closeDanglingObservations("Agent run ended before observation finalized");
+  if (state.agentState.activeAttempt) {
+    state.agentState.activeAttempt.end();
+    state.agentState.activeAttempt = undefined;
+  }
 
   try {
     state.agentState.root