pi-langfuse 1.0.0 → 1.2.0

package/.trae/documents/optimize_langfuse_reporting.md

@@ -0,0 +1,68 @@
+# Langfuse 追踪逻辑优化方案
+
+## 1. 背景与目标
+根据之前的建议，我们需要进一步优化 Pi-Langfuse 扩展的上报逻辑，使得在 Langfuse 控制台中观察 Agent 执行过程时更加细致、层级分明且数据完备。
+本次优化包含 5 个核心点：引入 Turn 级别的 Span、精准追踪 TTFT（首字响应时间）、处理 Provider 请求异常、上报 System Prompt，以及完善 Session 的边缘生命周期和上下文压缩（Compact）处理。
+
+## 2. 现状分析
+- **层级结构**：目前 `llm-generation` 和 `tool` 是平级挂载在根 Trace 下，多轮对话时瀑布图非常扁平。
+- **TTFT**：尚未记录 `completionStartTime`，无法在 Langfuse 中直观看到首字耗时。
+- **异常闭环**：LLM Provider 发生 4xx/5xx HTTP 错误时，`message_end` 可能不触发，导致 Generation 无法正常闭合。
+- **上下文完备性**：`before_agent_start` 仅提取了 user prompt，缺少 System Prompt，不利于后期 Debug。
+- **Session 生命周期**：仅处理了 `session_start` 和 `session_shutdown`，忽略了 `/new`, `/resume`, `/fork`, `/compact` 等高级会话操作引发的状态混乱问题。
+
+## 3. 具体修改方案
+
+### 3.1 引入 Turn-Level Span (层级结构优化)
+将一轮对话（Turn）包装为一个 Span，使其成为 Generation 和 Tool 的父节点。
+- **`src/types.ts`**:
+  - 在 `AgentState` 接口中增加 `activeTurn?: LangfuseObservation` 字段。
+- **`src/handlers/turn.ts` (新建)**:
+  - 实现 `startTurnObservation(event)`：调用 `state.agentState.root.startObservation("turn", {...}, { asType: "span" })` 并赋值给 `activeTurn`。
+  - 实现 `finishTurnObservation(event)`：结束 `activeTurn` 并将其置空。
+- **`src/handlers/generation.ts` & `src/handlers/tool.ts`**:
+  - 在创建 `llm-generation` 和 `tool` 观察节点时，优先判断 `state.agentState.activeTurn` 是否存在，若存在则调用 `activeTurn.startObservation`，否则降级使用 `root.startObservation`。
+- **`index.ts`**:
+  - 注册 `turn_start` 事件，调用 `startTurnObservation`。
+  - 在现有的 `turn_end` 监听器中，除了处理 Fallback Generation，还需要调用 `finishTurnObservation`。
+
+### 3.2 精准追踪 TTFT (首字生成时间)
+记录模型流式输出第一块 Chunk 的时间，用于计算 TTFT。
+- **`src/types.ts`**:
+  - 在 `GenerationState` 中增加 `ttftRecorded?: boolean` 标志位。
+- **`src/handlers/generation.ts`**:
+  - 新增 `recordTTFT(event)` 函数：获取当前 `activeGeneration`，若 `ttftRecorded` 为 false/undefined，则调用 `generation.observation.update({ completionStartTime: new Date() })`，并标记 `ttftRecorded = true`。
+- **`index.ts`**:
+  - 在 `message_update` 事件监听器中，除了原有的提取逻辑外，新增调用 `recordTTFT(event)`。
+
+### 3.3 异常流闭环：Provider 请求级错误处理
+防止因 Provider HTTP 请求失败导致 Generation 永远处于开启状态。
+- **`src/handlers/generation.ts`**:
+  - 修改 `updateGenerationMetadata(event)` 函数。在提取完 metadata 后，检查 `metadata.status`。
+  - 如果 `status >= 400` 或者存在明确的 error 信息，立刻将该 Generation 的状态设为 `level: "ERROR"`，附加 `statusMessage`，并调用 `.end()` 提前闭合它，同时设置 `ended = true`。
+
+### 3.4 完善上下文信息：上报 System Prompt
+将 System Prompt 纳入 Trace 的元数据或输入中，方便回溯。
+- **`src/handlers/agent.ts`**:
+  - 修改 `startAgentRun(event, ctx)`。
+  - 增加 `const systemPrompt = await ctx.getSystemPrompt();`。
+  - 在 `root` Trace 的 `metadata` 中增加 `systemPrompt: truncate(systemPrompt, MAX_TOOL_PAYLOAD_LENGTH)`。
+
+### 3.5 Session 边缘生命周期与 Compact 处理
+完善 Pi 扩展 API 提供的各类会话切换与上下文管理事件。
+- **`index.ts`**:
+  - 增加 `session_before_switch` 和 `session_before_fork` 的事件监听。当触发这些事件时，表明当前会话即将被替换，调用 `closeDanglingObservations("Session switched or forked")` 提前闭合所有挂起的节点，并调用 `resetRunState()`。
+  - 增加 `session_compact` 事件监听。如果当前 `state.agentState.root` 存在，调用 `root.startObservation("session_compact", { level: "DEFAULT", statusMessage: "Context was compacted" }, { asType: "span" }).end()`，在 Trace 中记录下压缩动作发生的时机。
+
+## 4. 假设与决策
+- **决策**: Turn Span 的命名直接使用 `"turn"`，类型使用 `span`，这样在 Langfuse 的甘特图中能清晰地包裹住内部的 Generation 和 Tool。
+- **决策**: TTFT 使用系统当前时间 `new Date()`。尽管存在微小的事件传递延迟，但在 Node.js 环境下已足够精确。
+- **假设**: `ctx.getSystemPrompt()` 是异步方法，在 `before_agent_start` 和 `agent_start` 中可以通过 `await` 正常获取。
+- **决策**: 对于 Session Lifecycle 的处理，统一视作强行终止当前 Agent Run 的执行，因此复用 `closeDanglingObservations` 逻辑以保证不会出现悬空（Dangling）的 Trace/Span。
+
+## 5. 验收标准
+1. 在代码中实现上述 5 项修改，且不破坏现有 TypeScript 编译。
+2. 运行 Pi Agent 进行一次多轮对话（调用工具），能够在 Langfuse 观察到 `Trace -> Turn -> Generation/Tool` 的层级结构。
+3. Langfuse 的 Generation 详情中能够看到 `Time to First Token (TTFT)`。
+4. Trace 的 Metadata 中包含 `systemPrompt` 字段。
+5. 通过 `/new` 或 `/compact` 时，控制台不报挂起节点相关的警告。

package/.trae/documents/pi-langfuse-refactor.md

@@ -0,0 +1,78 @@
+# Pi-Langfuse 代码重构与拆分计划
+
+## 摘要 (Summary)
+将当前长达 1105 行的单文件 `index.ts` 重构并按功能职责拆分至 `src/` 目录下的多个模块中。保留根目录的 `index.ts` 作为 Pi 扩展的纯入口文件。此举将极大提升代码的可读性和可维护性。
+
+## 当前状态分析 (Current State Analysis)
+目前 `index.ts` 是一个单文件巨石（Monolith），包含了以下所有逻辑：
+1. 本地与环境变量配置读取
+2. TypeScript 接口声明（`Config`, `AgentState`, `LangfuseObservation` 等）
+3. 全局可变状态（如 `agentState`, `currentSessionId`, 各类计数器）
+4. Langfuse SDK 的懒加载与生命周期管理
+5. 数据格式化、截断与负载提取工具函数
+6. Agent、Generation 和 Tool 各自的观测事件处理器
+7. Pi 扩展命令注册与生命周期事件监听
+
+所有逻辑混合在一起，导致状态变更难以追踪，且由于闭包共享了大量模块级状态，修改代码容易引入隐藏的副作用。
+
+## 提议更改 (Proposed Changes)
+
+### 1. 更新项目配置
+- 修改 `tsconfig.json`：在 `include` 中增加 `"src/**/*.ts"`，以包含新的源码目录。
+- **依赖引用规范**：由于 `tsconfig.json` 配置了 `"moduleResolution": "NodeNext"`，所有内部文件导入必须显式带有 `.js` 后缀（例如 `import { state } from "./state.js"`）。
+
+### 2. 建立 `src/` 目录结构并拆分职责
+计划新建如下文件与目录：
+
+- **`src/types.ts`**
+  - **内容**：提取所有的 Interface 和 Type 定义。
+  - **包含**：`Config`, `LangfuseObservation`, `ObservationUpdate`, `LangfuseScoreClient`, `LangfuseRuntime`, `GenerationState`, `ToolState`, `AgentState`。
+
+- **`src/constants.ts`**
+  - **内容**：提取所有魔法数字和常量配置。
+  - **包含**：`EXT_DIR`, `CONFIG_PATH`, `DEFAULT_LANGFUSE_HOST`, `MAX_STRING_LENGTH` 等。
+
+- **`src/state.ts`**
+  - **内容**：统一管理所有全局可变状态。
+  - **实现方式**：将原来的 `let agentState`, `let currentSessionId` 等包裹在一个 `export const state = { ... }` 对象中，确保跨模块引用时状态一致。
+  - **包含函数**：`resetRunState`, `computeEvaluationScores`。
+
+- **`src/utils.ts`**
+  - **内容**：集中存放与状态无关的纯工具函数。
+  - **包含**：`truncate`, `tryParseJson`, `shapePayload`, `safeSerialize`，以及一系列解析函数（`extractTextContent`, `extractToolCalls`, `getToolCallId` 等）。
+
+- **`src/config.ts`**
+  - **内容**：负责配置读取与持久化，以及与 Pi UI 的交互提示。
+  - **包含**：`loadConfigFromFile`, `loadConfigFromEnv`, `saveConfig`, `ensureConfig`, `promptForConfig`。
+
+- **`src/langfuse.ts`**
+  - **内容**：Langfuse SDK 客户端封装。
+  - **包含**：`getRuntime` (单例模式加载 SDK)、`shutdownRuntime`、`sendScore`。
+
+- **`src/handlers/agent.ts`**
+  - **内容**：Agent 生命周期的事件逻辑。
+  - **包含**：`startAgentRun`, `finishAgentRun`, `updateTraceIO`。
+
+- **`src/handlers/generation.ts`**
+  - **内容**：模型生成 (Generation) 相关的生命周期逻辑。
+  - **包含**：`getOpenGeneration`, `startGeneration`, `updateGenerationMetadata`, `finishGenerationFromMessage`, `createFallbackGenerationFromTurn`。
+
+- **`src/handlers/tool.ts`**
+  - **内容**：工具调用 (Tool) 相关的逻辑。
+  - **包含**：`startToolObservation`, `finishToolObservation`, `closeDanglingObservations`。
+
+### 3. 精简根目录 `index.ts`
+- **内容**：将其转换为纯粹的事件路由中心。
+- **改动**：删除原有的业务实现，改为从 `src/` 各个模块导入必要的 handler。保留默认导出的扩展注册函数 `export default async function (pi: ExtensionAPI)`，并在其中通过 `pi.on` 和 `pi.registerCommand` 将事件委派给对应的处理函数。
+
+## 假设与决策 (Assumptions & Decisions)
+- **零行为变更**：本次重构仅做结构上的梳理，不修改任何核心逻辑、状态变更时机或事件 payload 的生成规则，从而保证与当前 Langfuse 的数据对接一致。
+- **状态管理**：采用共享的 `state` 单例对象替代原本的顶层变量，以最少的改动适配多文件架构。
+- **兼容性**：保留根目录的 `index.ts` 以兼容现有的 `package.json` 中的 `pi.extensions` 配置，无需修改发布行为。
+
+## 验证步骤 (Verification steps)
+1. 运行 `npm run typecheck` 确认代码拆分后无 TypeScript 编译及引入报错。
+2. 开启 Pi CLI (`pi "test prompt"`) 并挂载该本地扩展，验证：
+   - 配置初始化弹窗或环境变量读取是否正常工作。
+   - 工具调用、LLM 生成以及会话结束时的逻辑是否无异常抛出。
+3. 登录 Langfuse 控制面板，核对新产生的 trace 数据结构（agent、generation、tool observations 及打分数据）是否完整，未出现状态泄漏或丢失。

package/AGENTS.md

@@ -1,57 +1,48 @@
-# AGENTS.md
+# Pi Langfuse Extension - Agents & Architecture
 
 ## Project Overview
 
-This repository contains a Pi Coding Agent extension that forwards session activity to Langfuse for observability. The runtime is intentionally small:
+This repository contains a Pi Coding Agent extension that integrates with Langfuse to provide deep observability into agent sessions. By hooking into Pi's Extension API, it forwards telemetry data (traces, spans, and LLM generations) to Langfuse, enabling developers to monitor token usage, cost, tool execution success rates, and conversational context.
 
-- `index.ts`: extension entrypoint, event listeners, trace/span/generation lifecycle, and scoring
-- `package.json`: package metadata for Pi/NPM
-- `config.json`: local credentials file for manual development only, ignored by git
-- `.agents/skills/langfuse/SKILL.md`: local skill instructions for Langfuse-related workflows
+## Architecture & Event Mapping
 
-## How The Extension Works
+The extension leverages the `@earendil-works/pi-coding-agent` Extension API to intercept lifecycle events and maps them to Langfuse's hierarchical observability model: **Trace** -> **Span** / **Generation**.
 
-The extension listens to Pi lifecycle events and maps them into Langfuse objects:
+### 1. Trace Level (Agent Run)
+The root of the observability tree is a **Trace**, representing a single user prompt and the agent's complete execution to fulfill it.
 
-1. `session_start` captures a stable session id
-2. `before_agent_start` creates a trace for the user prompt
-3. `tool_call` / `tool_result` create and close tool spans
-4. `turn_end` records LLM usage/cost metadata as a generation
-5. `agent_end` finalizes the trace and publishes aggregate evaluation scores
-6. `session_shutdown` flushes any remaining state
+- `before_agent_start` / `agent_start`: Initializes the `pi-agent` **Trace**. Captures the initial prompt, working directory (`cwd`), and session ID.
+- `agent_end`: Finalizes the trace, captures the final assistant output, and submits evaluation scores (e.g., tool call count, success rate).
 
-The code is stateful by design. Changes to trace/session bookkeeping should be reviewed carefully because multiple Pi hooks share the same module-level state.
+### 2. Generation Level (LLM Calls)
+Every time the agent communicates with an LLM provider, a **Generation** is recorded to track token usage, costs, and latency.
 
-## Local Development
+- `before_provider_request`: Starts the `llm-generation` observation.
+- `after_provider_response`: Updates the generation with HTTP status and provider metadata.
+- `message_update`: Tracks streaming text (can be used to track Time-To-First-Token).
+- `message_end`: Ends the generation, extracting `usageDetails` (input/output tokens, cache metrics) and `costDetails`.
+- `turn_end`: Handles fallback generation logging if standard message events miss the completion.
 
-Recommended environment:
+### 3. Span Level (Tool Executions)
+When the LLM decides to use a registered tool (e.g., bash, file read), a **Span** is created under the root trace.
 
-- Node.js `>=22` as declared in `package.json`
-- A local `config.json` with Langfuse credentials
+- `tool_execution_start` / `tool_call`: Starts a `tool` span, logging the tool name and its input parameters (truncated for safety).
+- `tool_result` / `tool_execution_end`: Finalizes the tool span. If the tool fails, the span's level is marked as `ERROR` and the error message is recorded.
 
-Typical workflow:
+### 4. Session & Lifecycle Management
+- `session_start`: Captures the stable session ID from Pi (`ctx.sessionManager.getSessionFile()`).
+- `session_shutdown`: Cleans up and flushes dangling observations (e.g., if the user abruptly exits via `Ctrl+C`).
 
-```bash
-npm install
-pi "test prompt"
-```
+## Development & Testing
 
-There is currently no dedicated test suite or `npm` script. When changing behavior, validate by running Pi with the extension enabled and confirming traces, spans, generations, and scores appear in Langfuse.
+1. **Prerequisites**: Node.js `>=22` and a local `config.json` with Langfuse credentials (`LANGFUSE_PUBLIC_KEY`, `LANGFUSE_SECRET_KEY`, `LANGFUSE_HOST`).
+2. **Run**: 
+   ```bash
+   npm install
+   pi -e ./index.ts "your test prompt"
+   ```
+3. **Validation**: Open your Langfuse dashboard to verify that Traces, Generations, and Tool Spans are accurately grouped, and that token usage/costs are populated.
 
-## Editing Guidance
-
-- Prefer small, behavior-preserving changes in `index.ts`; most bugs here are lifecycle or observability-shape bugs rather than UI issues.
-- Keep credential material out of version control. `config.json` should stay local.
-- When logging tool input/output, be mindful of payload size and serialization failures.
-- If you add new Langfuse fields, verify them against current official docs because the SDK surface evolves.
-- Avoid hard-coding install-layout assumptions when importing SDK files unless Pi requires it.
-
-## Review Focus Areas
-
-When reviewing or extending this project, pay extra attention to:
-
-- Trace lifecycle consistency across `before_agent_start`, `agent_end`, and `session_shutdown`
-- Correct score attribution at trace level vs observation level
-- Defensive handling of unexpected event payload shapes
-- Truncation/redaction strategy for large tool payloads
-- Compatibility between the installed Langfuse SDK version and any manually declared local TypeScript interfaces
+## Design Constraints
+- **Statefulness**: The extension maintains module-level state (`state.ts`) because multiple Pi hooks share the same context across an agent run.
+- **Data Safety**: Large tool payloads and circular references are aggressively truncated/shaped to prevent serialization crashes and excessive network overhead.

package/README.md

@@ -1,6 +1,11 @@
 # pi-langfuse
 
-Langfuse observability extension for [Pi Coding Agent](https://github.com/earendil-works/pi-coding-agent). Sends traces to [Langfuse](https://langfuse.com) for monitoring tokens, costs, latency, and tool calls.
+[![npm version](https://img.shields.io/npm/v/pi-langfuse)](https://www.npmjs.com/package/pi-langfuse)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+[**English**](./README.md) | [**简体中文**](./README_CN.md)
+
+Langfuse observability extension for [Pi Coding Agent](https://github.com/earendil-works/pi-coding-agent). Sends complete Pi agent runs to [Langfuse](https://langfuse.com) so you can inspect the user prompt, root agent workflow, every LLM generation, every tool call, final assistant response, usage, cost, and health scores in one trace.
 
 ## Why Langfuse?
 
@@ -8,113 +13,319 @@ Langfuse provides open-source observability for LLM applications. This extension
 
 ## Features
 
-- **Hierarchical Tracing**: Maps user prompts to per-turn spans and nested tool executions for deep visibility.
-- **LLM Metadata**: Automatically records model name, provider, token usage, and API costs per turn.
-- **Tool Observability**: Detailed logs for every tool call, including arguments, results, and error states.
-- **Session Correlation**: Groups all prompts from the same Pi session into a single Langfuse session.
-- **Cost Tracking**: Records input/output/total costs in USD per generation.
-- **Token Usage**: Tracks input and output tokens per turn.
+- **Complete Agent Traces**: Creates one trace per user prompt with a root `agent` observation containing the prompt input and final assistant output.
+- **Per-Request Generations**: Records a separate `generation` observation for every provider request, including the actual provider payload instead of only the original prompt.
+- **Final Message Capture**: Uses finalized assistant messages for generation and root outputs, so Langfuse shows what the user actually saw in Pi.
+- **Tool Observability**: Creates Langfuse `tool` observations for every tool call, including arguments, results, and error states.
+- **Parallel Tool Safety**: Correlates tool observations by `toolCallId`, avoiding result mix-ups when Pi runs tools concurrently.
+- **Session Correlation**: Groups traces from the same Pi session under a shared Langfuse session ID.
+- **Cost and Token Tracking**: Records usage and cost details on each generation when Pi/provider payloads expose them.
+- **Evaluation Scores**: Automatically computes and sends tool success rates, error counts, and session health metrics.
+- **Defensive Payload Shaping**: Parses JSON-like strings when possible, limits object depth, and truncates large payloads before upload.
+
+## Highlights
+
+`pi-langfuse` is designed to make a Pi run readable as an agent workflow, not just a bag of logs:
+
+- The trace input/output mirrors the root `agent` observation, making the run understandable from the Langfuse trace list and detail view.
+- The first generation in a tool-using run can show the assistant's tool-call message, the tool observation shows execution I/O, and the follow-up generation shows the final natural-language answer.
+- Tool failures are marked on the tool observation and reflected in trace-level scores, while later generations still preserve the tool error result in their input history.
+- Shutdown and interrupted runs flush pending telemetry and mark unfinished observations as cancelled/warning instead of silently losing the trace.
 
-## Quick Install
+## Prerequisites
+
+- **Node.js** >= 22
+- **Pi Coding Agent** installed and configured
+- A **Langfuse** account ([cloud](https://cloud.langfuse.com) or self-hosted)
+
+## Installation
+
+### Option 1: Install via npm (recommended for users)
 
-### Via npm (recommended)
 ```bash
 pi install npm:pi-langfuse
 ```
 
+Pi will download the package and register it as an extension.
+
+### Option 2: Install from local source (recommended for development)
+
+```bash
+git clone <your-repo-url>
+cd pi-langfuse
+npm install
+```
+
+Then tell Pi to use it:
+
+```bash
+pi link /path/to/pi-langfuse
+```
+
+Or run Pi from the project directory — Pi auto-discovers extensions in the current directory's `package.json`.
+
 ## Configuration
 
-Get your keys from [Langfuse Cloud](https://cloud.langfuse.com) → Settings → API Keys.
+You need Langfuse API keys. Get them from **Langfuse Cloud** → **Settings** → **API Keys**.
 
-On first run, Pi will prompt in the CLI/TUI for:
+There are three ways to configure the extension:
 
-- `LANGFUSE_PUBLIC_KEY`
-- `LANGFUSE_SECRET_KEY`
-- `LANGFUSE_HOST` (optional, defaults to `https://cloud.langfuse.com`)
+### Method 1: Interactive setup (easiest)
 
-The extension stores this locally in its own `config.json`, which is ignored by git.
+Run any `pi` command with the extension loaded. On first run without configuration, Pi will prompt you in the CLI or TUI for:
 
-You can also preconfigure it with environment variables instead of using the interactive prompt.
+1. **Langfuse public key** — starts with `pk-lf-...`
+2. **Langfuse secret key** — starts with `sk-lf-...`
+3. **Langfuse host** — defaults to `https://cloud.langfuse.com`
 
-To rerun setup later, use:
+The extension saves these to a local `config.json` (ignored by git).
 
-```bash
+To re-run setup at any time:
+
+```
 /langfuse-setup
 ```
 
+### Method 2: Environment variables
+
+Set these before starting Pi:
+
+```bash
+export LANGFUSE_PUBLIC_KEY="pk-lf-xxxx"
+export LANGFUSE_SECRET_KEY="sk-lf-xxxx"
+export LANGFUSE_BASE_URL="https://cloud.langfuse.com"  # optional; LANGFUSE_HOST is also supported
+```
+
+The extension checks the local `config.json` first, then falls back to environment variables.
+
+### Method 3: Local config.json (development only)
+
+For local development, create a `config.json` in the project root:
+
+```json
+{
+  "publicKey": "pk-lf-xxxx",
+  "secretKey": "sk-lf-xxxx",
+  "host": "https://cloud.langfuse.com"
+}
+```
+
+> **⚠️ Security**: `config.json` is not tracked by git. Never commit API keys to version control.
+
 ## Usage
 
-### Run pi with tracing enabled
+### Basic usage
+
+Run Pi as usual — the extension auto-loads and traces every agent run:
 
 ```bash
-pi "your prompt"
+pi "Explain the architecture of Redis"
 ```
 
-Pi auto-loads the extension. All sessions will be traced to Langfuse.
+After the session ends, check your [Langfuse dashboard](https://cloud.langfuse.com) for the trace.
+
+### Verify the extension is loaded
+
+```bash
+pi list
+```
+
+You should see `pi-langfuse` in the list of installed packages.
+
+### Multiple sessions
+
+Each Pi session gets its own Langfuse session ID. Each user prompt within that Pi session becomes a separate Langfuse trace grouped under the same session.
+
+## Development Setup
+
+If you're contributing to this extension:
+
+```bash
+# Clone and install dependencies
+git clone <your-repo-url>
+cd pi-langfuse
+npm install
+
+# Type-check your changes
+npm run typecheck
+
+# Test with Pi
+pi "test prompt"
+```
+
+### Project structure
+
+```
+pi-langfuse/
+├── index.ts            # Extension entrypoint and core logic
+├── package.json        # Package metadata
+├── tsconfig.json       # TypeScript configuration
+├── config.json         # Local credentials (git-ignored)
+├── types/
+│   ├── pi-coding-agent.d.ts   # Pi extension API types
+│   └── node-shims.d.ts        # Node.js module shims
+├── .agents/
+│   └── skills/
+│       └── langfuse/
+│           └── SKILL.md       # Langfuse CLI skill for data queries
+├── AGENTS.md           # Developer guide (extended)
+├── README.md           # This file
+├── README_CN.md        # Chinese translation
+└── AGENTS_CN.md        # Developer guide (Chinese)
+```
+
+### Validation
+
+There is no dedicated test suite yet. To validate changes:
+
+1. Run `npm run typecheck` for TypeScript errors
+2. Start Pi with the extension enabled
+3. Run a few prompts
+4. Confirm traces, the root agent observation, tool observations, generations, and evaluation scores appear in your Langfuse project
 
 ## Trace Model
 
 ```
 Trace (name: "pi-agent")
 ├── Session ID: <pi-session-id>
-├── Metadata: model, provider, cwd
-└── Span (name: "tool:<name>")
-    └── Input/Output logs
-
-Generation (name: "llm-response")
-├── Model: MiniMax-M2.7
-├── Usage: input/output tokens
-└── Cost: input/output/total USD
+├── input:  user prompt, images/context summary when present
+├── output: final assistant response
+└── Agent observation (name: "pi-agent", type: agent)
+    ├── input:  current user prompt
+    ├── output: final assistant response
+    ├── Generation observation (name: "llm-generation", type: generation)
+    │   ├── input: provider request payload / message history
+    │   ├── output: finalized assistant message or tool-call message
+    │   ├── model, usageDetails, costDetails
+    │   └── metadata: provider/request details
+    └── Tool observation (name: "<tool-name>", type: tool)
+        ├── input: tool parameters
+        ├── output: tool result
+        └── metadata: toolCallId, isError
 ```
 
 ## What Gets Tracked
 
 ### Trace Level
-- `input` - User prompt
-- `output` - Assistant response
-- `sessionId` - Pi session identifier
-- `metadata` - Model, provider, cwd
+| Field | Description |
+|-------|-------------|
+| `input` | User prompt, with images/context summary when available |
+| `output` | Final assistant response shown in Pi |
+| `sessionId` | Pi session identifier |
+| `metadata.model` | Model identifier (e.g., "MiniMax-M2.7") |
+| `metadata.provider` | LLM provider name |
+| `metadata.cwd` | Working directory |
+
+### Agent Observation (Root Workflow)
+| Field | Description |
+|-------|-------------|
+| `type` | `agent` |
+| `name` | `pi-agent` |
+| `input` | Current user prompt payload |
+| `output` | Final assistant response |
+| `metadata.sessionId` | Pi session identifier |
+| `metadata.cwd` | Working directory |
+| `metadata.model` | Selected model when available |
+| `metadata.provider` | Provider when available |
+
+### Evaluation Scores (Trace Level)
+
+| Score Name | Type | Description |
+|------------|------|-------------|
+| `tool_call_count` | number | Total tool calls in session |
+| `turn_count` | number | Number of assistant turns |
+| `total_tool_errors` | number | Tools that returned errors |
+| `tool_success_rate` | float (0-1) | Ratio of successful tool calls |
+| `session_had_errors` | 0 or 1 | Whether any tool errored |
 
 ### Generation Observations (LLM Calls)
-- `model` - Model identifier (e.g., "MiniMax-M2.7")
-- `usage` - Token counts (input/output/total)
-- `costDetails` - Cost breakdown in USD
-
-### Span Observations (Tool Calls)
-- `name` - Tool name (e.g., "tool:bash")
-- `input` - Tool parameters (JSON)
-- `output` - Tool result
-- `metadata.isError` - Whether tool failed
+| Field | Description |
+|-------|-------------|
+| `type` | `generation` |
+| `name` | `llm-generation` |
+| `input` | Actual provider request payload / message history |
+| `output` | Finalized assistant message, including tool-call payloads for tool-calling turns |
+| `model` | Model identifier (e.g., "MiniMax-M2.7") |
+| `usageDetails.input` | Input token count |
+| `usageDetails.output` | Output token count |
+| `usageDetails.total` | Total token count |
+| `costDetails.total` | Total cost in USD |
+| `costDetails.input` | Input cost in USD |
+| `costDetails.output` | Output cost in USD |
+| `metadata.provider` | Provider name |
+| `metadata.requestId` | Provider/Pi request identifier when available |
+| `metadata.status` | HTTP/provider status when available |
+
+### Tool Observations
+| Field | Description |
+|-------|-------------|
+| `type` | `tool` |
+| `name` | Tool name (e.g., "bash", "read") |
+| `input` | Tool parameters |
+| `output` | Tool result, shaped and truncated for readability |
+| `metadata.toolCallId` | Stable Pi tool call identifier |
+| `metadata.isError` | Whether the tool failed |
+| `level` | `ERROR` for failed tool calls, otherwise `DEFAULT` |
+
+### Observation-Level Scores
+| Score Name | Description |
+|------------|-------------|
+| `tool_is_error` | Value 1 assigned to individual tool observations that errored |
 
 ## Langfuse Dashboard
 
 After running, check your Langfuse project for:
 
-1. **Traces** - All pi agent runs with I/O
-2. **Sessions** - Traces grouped by session ID
-3. **Observations** - Tool calls and LLM generations
-4. **Scores** - Evaluation metrics such as tool errors and success rate
-5. **Model Usage** - Usage breakdown by model
+1. **Traces** — All pi agent runs with I/O
+2. **Sessions** — Traces grouped by session ID
+3. **Observations** — Tool calls and LLM generations
+4. **Scores** — Evaluation metrics (tool errors, success rate, etc.)
+5. **Model Usage** — Usage breakdown by model
+
+You can also monitor your Langfuse data directly from the terminal using the built-in Langfuse skill:
+
+```
+/pi-langfuse-langfuse <your-query>
+```
 
 ## Troubleshooting
 
-**No traces appearing?**
-- Verify API keys are correct in `config.json`
-- Check Langfuse project is active
-- Ensure API keys have write permissions
+### No traces appearing?
+- Verify API keys are correct — run `/langfuse-setup` to re-configure
+- Check your Langfuse project is active and has write capacity
+- Ensure API keys have write permissions (not read-only)
+- Look for `📊 Langfuse:` log messages in the Pi output
+
+### Extension not loading?
+```bash
+pi list                      # Verify pi-langfuse is installed
+pi install npm:pi-langfuse   # Reinstall if missing
+```
+
+### "Missing config" message on startup?
+- The extension needs credentials. Use the interactive `/langfuse-setup` command
+- Or set `LANGFUSE_PUBLIC_KEY` and `LANGFUSE_SECRET_KEY` environment variables
 
-**Extension not loading?**
-- Run `pi list` to check installed packages
-- Try restarting pi
+### Model/cost not showing?
+- Not all providers expose cost information
+- Check the Langfuse traces API for raw observation data
+- The `model` field in generations comes from provider events, finalized assistant messages, `model_select`, or `ctx.model`
 
-**Model/cost not showing?**
-- Not all providers expose cost info
-- Check Langfuse traces API for raw observation data
+### API key errors?
+- Langfuse public keys start with `pk-lf-`, secret keys with `sk-lf-`
+- If self-hosting, verify your host URL is correct
 
 ## Dependencies
 
-- [langfuse](https://www.npmjs.com/package/langfuse) - Langfuse SDK
-- [@earendil-works/pi-coding-agent](https://www.npmjs.com/package/@earendil-works/pi-coding-agent) - Pi extension API
+- [@langfuse/tracing](https://www.npmjs.com/package/@langfuse/tracing) — Langfuse observation API for `agent`, `generation`, and `tool` traces
+- [@langfuse/otel](https://www.npmjs.com/package/@langfuse/otel) — OpenTelemetry span processor for exporting traces to Langfuse
+- [@langfuse/client](https://www.npmjs.com/package/@langfuse/client) — Langfuse API client used for scores
+- [@opentelemetry/sdk-node](https://www.npmjs.com/package/@opentelemetry/sdk-node) — Node OpenTelemetry SDK
+- [@earendil-works/pi-coding-agent](https://www.npmjs.com/package/@earendil-works/pi-coding-agent) — Pi extension API (peer dependency)
+
+## About Langfuse Skill
+
+This package includes a Langfuse CLI skill (at `.agents/skills/langfuse/`) that lets you query Langfuse data directly from Pi. Use it to look up traces, prompts, datasets, and scores without leaving the terminal. The skill is auto-registered when the extension is installed globally.
 
 ## License