pi-langfuse 1.4.1 → 1.4.2

package/.agents/skills/langfuse/references/user-feedback.md

@@ -1,88 +0,0 @@
----
-name: langfuse-user-feedback
-description: Wires up user feedback (thumbs up/down, ratings, comments) from an application's frontend to Langfuse scores. Use when user wants to capture end-user feedback, add ratings to traces, or connect user complaints to Langfuse.
----
-
-# User Feedback
-
-Tracing must already be set up — feedback is stored as scores on traces.
-
-Docs: https://langfuse.com/docs/observability/features/user-feedback
-
-## Workflow
-
-### 1. Determine What Feedback to Capture
-
-If the user has asked for something specific, go with that. Otherwise, look at the application and **present a few UX options** for how feedback could work, then ask the user which they prefer before implementing.
-
-Common UX patterns to suggest:
-
-| UX Pattern | Best for | How it works |
-|------------|----------|--------------|
-| Thumbs up/down | Chat apps, Q&A | Simple binary buttons next to each response |
-| Star rating (1–5) | Content generation, summaries | Star row or dropdown after each output |
-| "Was this helpful?" banner | Search, documentation assistants | Single yes/no prompt at the bottom of a response |
-| Regenerate / copy tracking | Any app with these actions | Implicit — log when users retry (negative signal) or copy output (positive signal) |
-| Free-text comment | Complex outputs, internal tools | Optional text field alongside a rating |
-| Report button | Any user-facing app | Flag icon to report bad/harmful responses |
-
-This table is not exhaustive — if the application suggests a different feedback pattern that fits better, propose that instead. Present 2–3 options that match the application's use case and ask the user which approach they'd like. This decision shapes everything downstream (score names, data types, frontend components), so it's important to align early.
-
-Feedback can be **explicit** (user rates via thumbs, stars, etc.) or **implicit** (derived from behavior like copying output, retrying, or escalating to support). Both are stored as scores. Explicit feedback requires the trace ID to reach the frontend; implicit feedback is logged server-side where the event already happens.
-
-### 2. Choose Score Names
-
-Name reflects the signal source, not what you hope it measures (e.g., `user-thumbs` not `response-quality` — a thumbs down doesn't tell you *what* was wrong). Avoid generic names like `feedback` or `score`.
-
-Rules:
-- Lowercase with hyphens
-- One consistent name per feedback type across the entire app
-- If capturing multiple signals, each gets its own distinct name
-
-### 3. Implement Score Creation
-
-**For implicit feedback (server-side):** Use `langfuse.create_score()` / `langfuse.score.create()` wherever the event is already handled in application code. Fetch SDK docs for current API: https://langfuse.com/docs/evaluation/evaluation-methods/scores-via-sdk
-
-**For explicit feedback (frontend):** Use `LangfuseWeb` in the browser. It uses the public key only — no secret key exposed.
-
-```typescript
-import { LangfuseWeb } from "langfuse";
-
-const langfuse = new LangfuseWeb({
-  publicKey: process.env.NEXT_PUBLIC_LANGFUSE_PUBLIC_KEY!,
-  baseUrl: process.env.NEXT_PUBLIC_LANGFUSE_HOST,
-});
-
-langfuse.score({
-  traceId,
-  name: "user-thumbs",
-  value: 1, // 1 = positive, 0 = negative
-  dataType: "BOOLEAN",
-  comment: optionalUserComment,
-});
-```
-
-The trace ID must be available in the frontend for this to work. For Vercel AI SDK, the non-obvious pattern is using `generateMessageId`:
-
-```typescript
-import { getActiveTraceId } from "@langfuse/tracing";
-
-// Inside route handler wrapped with observe()
-return result.toUIMessageStreamResponse({
-  generateMessageId: () => getActiveTraceId() || crypto.randomUUID(),
-});
-```
-
-### 4. Verify
-
-Trigger a feedback action and check the trace's Scores tab in Langfuse. Confirm the score name, value, and data type are correct.
-
-Point users to what they can do with feedback data: filter traces by low scores, use score analytics for trends, build annotation queues for team review.
-
-## Common Mistakes
-
-| Mistake | Problem | Fix |
-|---------|---------|-----|
-| Secret key in frontend code | Security risk | Use `LangfuseWeb` with public key only |
-| Missing `dataType` on boolean scores | Value `1` inferred as `NUMERIC` | Always pass `dataType: "BOOLEAN"` explicitly |
-| Inconsistent score names across the app | Can't aggregate or filter reliably | Pick one name per feedback type, use it everywhere |

package/AGENTS.md

@@ -1,48 +0,0 @@
-# Pi Langfuse Extension - Agents & Architecture
-
-## Project Overview
-
-This repository contains a Pi Coding Agent extension that integrates with Langfuse to provide deep observability into agent sessions. By hooking into Pi's Extension API, it forwards telemetry data (traces, spans, and LLM generations) to Langfuse, enabling developers to monitor token usage, cost, tool execution success rates, and conversational context.
-
-## Architecture & Event Mapping
-
-The extension leverages the `@earendil-works/pi-coding-agent` Extension API to intercept lifecycle events and maps them to Langfuse's hierarchical observability model: **Trace** -> **Span** / **Generation**.
-
-### 1. Trace Level (Agent Run)
-The root of the observability tree is a **Trace**, representing a single user prompt and the agent's complete execution to fulfill it.
-
-- `before_agent_start` / `agent_start`: Initializes the `pi-agent` **Trace**. Captures the initial prompt, working directory (`cwd`), and session ID.
-- `agent_end`: Finalizes the trace, captures the final assistant output, and submits evaluation scores (e.g., tool call count, success rate).
-
-### 2. Generation Level (LLM Calls)
-Every time the agent communicates with an LLM provider, a **Generation** is recorded to track token usage, costs, and latency.
-
-- `before_provider_request`: Starts the `llm-generation` observation.
-- `after_provider_response`: Updates the generation with HTTP status and provider metadata.
-- `message_update`: Tracks streaming text (can be used to track Time-To-First-Token).
-- `message_end`: Ends the generation, extracting `usageDetails` (input/output tokens, cache metrics) and `costDetails`.
-- `turn_end`: Handles fallback generation logging if standard message events miss the completion.
-
-### 3. Span Level (Tool Executions)
-When the LLM decides to use a registered tool (e.g., bash, file read), a **Span** is created under the root trace.
-
-- `tool_execution_start` / `tool_call`: Starts a `tool` span, logging the tool name and its input parameters (truncated for safety).
-- `tool_result` / `tool_execution_end`: Finalizes the tool span. If the tool fails, the span's level is marked as `ERROR` and the error message is recorded.
-
-### 4. Session & Lifecycle Management
-- `session_start`: Captures the stable session ID from Pi (`ctx.sessionManager.getSessionFile()`).
-- `session_shutdown`: Cleans up and flushes dangling observations (e.g., if the user abruptly exits via `Ctrl+C`).
-
-## Development & Testing
-
-1. **Prerequisites**: Node.js `>=22` and a local `config.json` with Langfuse credentials (`LANGFUSE_PUBLIC_KEY`, `LANGFUSE_SECRET_KEY`, `LANGFUSE_HOST`).
-2. **Run**: 
-   ```bash
-   npm install
-   pi -e ./index.ts "your test prompt"
-   ```
-3. **Validation**: Open your Langfuse dashboard to verify that Traces, Generations, and Tool Spans are accurately grouped, and that token usage/costs are populated.
-
-## Design Constraints
-- **Statefulness**: The extension maintains module-level state (`state.ts`) because multiple Pi hooks share the same context across an agent run.
-- **Data Safety**: Large tool payloads and circular references are aggressively truncated/shaped to prevent serialization crashes and excessive network overhead.

package/AGENTS_CN.md

@@ -1,57 +0,0 @@
-# AGENTS.md
-
-## 项目概述
-
-此仓库包含一个 Pi Coding Agent 扩展，将会话活动转发到 Langfuse 进行可观测性监控。运行时设计得尽可能精简：
-
-- `index.ts`：扩展入口点、事件监听器、跟踪/跨度/生成生命周期和评分
-- `package.json`：Pi/NPM 的包元数据
-- `config.json`：仅用于手动开发的本地凭据文件，被 git 忽略
-- `.agents/skills/langfuse/SKILL.md`：Langfuse 相关工作流的本地技能说明
-
-## 扩展工作原理
-
-该扩展监听 Pi 生命周期事件，并将它们映射到 Langfuse 对象：
-
-1. `session_start` 捕获稳定的会话 ID
-2. `before_agent_start` 为用户提示创建跟踪
-3. `tool_call` / `tool_result` 创建和关闭工具跨度
-4. `turn_end` 将 LLM 使用/成本元数据记录为生成
-5. `agent_end` 完成跟踪并发布聚合评估分数
-6. `session_shutdown` 刷新任何剩余状态
-
-代码设计为有状态的。对跟踪/会话簿记的更改应仔细审查，因为多个 Pi 钩子共享相同的模块级状态。
-
-## 本地开发
-
-推荐环境：
-
-- Node.js `>=22`（如 `package.json` 中声明）
-- 包含 Langfuse 凭据的本地 `config.json`
-
-典型工作流程：
-
-```bash
-npm install
-pi "test prompt"
-```
-
-目前没有专门的测试套件或 `npm` 脚本。更改行为时，通过启用扩展运行 Pi 并确认跟踪、跨度、生成和分数出现在 Langfuse 中来验证。
-
-## 编辑指南
-
-- 优先在 `index.ts` 中进行小的、保持行为的更改；这里的错误大多是生命周期或可观测性形状错误，而不是 UI 问题。
-- 将凭据材料排除在版本控制之外。`config.json` 应保持本地。
-- 记录工具输入/输出时，注意有效负载大小和序列化失败。
-- 如果添加新的 Langfuse 字段，请根据当前官方文档验证，因为 SDK 表面会演变。
-- 除非 Pi 要求，否则避免在导入 SDK 文件时硬编码安装布局假设。
-
-## 审查重点领域
-
-审查或扩展此项目时，请特别注意：
-
-- 跨 `before_agent_start`、`agent_end` 和 `session_shutdown` 的跟踪生命周期一致性
-- 跟踪级别与观察级别的正确分数归属
-- 对意外事件负载形状的防御性处理
-- 大型工具有效负载的截断/编辑策略
-- 已安装的 Langfuse SDK 版本与任何手动声明的本地 TypeScript 接口之间的兼容性