@narumitw/pi-langfuse 0.14.1

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 narumiruna
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,135 @@
+# 🪢 pi-langfuse — Langfuse Observability for Pi
+
+[![npm](https://img.shields.io/npm/v/@narumitw/pi-langfuse)](https://www.npmjs.com/package/@narumitw/pi-langfuse) [![Pi extension](https://img.shields.io/badge/Pi-extension-blue)](https://pi.dev) [![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](./LICENSE)
+
+`@narumitw/pi-langfuse` is a [Pi coding agent](https://pi.dev) extension that sends Pi LLM generations, agent runs, and tool executions to [Langfuse](https://langfuse.com/) through OpenTelemetry.
+
+## ✨ Features
+
+- Names each Langfuse trace `pi.trace` and creates a native `pi.agent` observation for the Pi agent run.
+- Adds a `pi.turn` span for every Pi turn, with generations and tools nested beneath it.
+- Records finalized assistant outputs without retaining intermediate provider request payloads.
+- Records provider, model, stop reason, token usage, and known non-zero reported cost.
+- Records normalized tool inputs, finalized outputs, duration, and failures as child spans.
+- Groups traces with Pi's session id.
+- Records provider HTTP status codes when Pi exposes them.
+- Reads Langfuse credentials and options only from a private `pi-langfuse.json` file.
+- Batches routine exports without delaying normal Pi agent completion.
+- Keeps its OpenTelemetry provider isolated so it coexists with other tracing extensions.
+- Redacts Langfuse public and secret keys from exported trace data.
+- Supports metadata-only tracing when content capture is disabled.
+
+## 📦 Install
+
+```bash
+pi install npm:@narumitw/pi-langfuse
+```
+
+Try the local workspace package:
+
+```bash
+pi -e ./extensions/pi-langfuse
+```
+
+The Langfuse v4 SDK requires Node.js 20 or newer.
+
+## ⚙️ Configuration
+
+Create or update the private config interactively from Pi:
+
+```text
+/langfuse init
+```
+
+The command prompts for the secret key, public key, and base URL in the same order Langfuse presents them. Leave either key blank to preserve its existing value when updating a valid config. Leave the base URL blank to use `https://us.cloud.langfuse.com`. The file is saved atomically with mode `0600`; restart Pi after saving. In print or JSON mode, edit the file manually because interactive input is unavailable.
+
+You can also create the file manually:
+
+```json
+{
+  "publicKey": "pk-lf-...",
+  "secretKey": "sk-lf-...",
+  "baseUrl": "https://us.cloud.langfuse.com",
+  "environment": "development",
+  "release": "local",
+  "captureContent": true
+}
+```
+
+`publicKey` and `secretKey` are required literal strings. Environment-variable and command interpolation are intentionally unsupported. `baseUrl` defaults to `https://us.cloud.langfuse.com`; regional and self-hosted HTTP or HTTPS endpoints are supported. Prefer HTTPS because HTTP sends Langfuse credentials and trace content without transport encryption.
+
+`environment` and `release` are optional Langfuse trace attributes. Set `captureContent` to `false` to trace timing, model, usage, cost, and status metadata without sending prompts, responses, or tool content.
+
+The extension automatically restricts an existing config file to mode `0600` and refuses to load credentials if that protection cannot be enforced. You can also set it explicitly:
+
+```bash
+chmod 600 ~/.pi/agent/pi-langfuse.json
+```
+
+Restart Pi after changing credentials, endpoint, environment, or release. The isolated OpenTelemetry tracer provider is initialized once per Pi process and selected only for Langfuse; it does not replace Pi's process-global provider or send Langfuse observations to another extension's exporter.
+
+## 🔭 What is traced
+
+Each `pi.trace` contains one `pi.agent` native `agent` observation, which contains:
+
+- a `pi.turn` native `span` for every Pi turn, including its index, stop reason, tool-result count, duration, and failure status;
+- a `pi.llm` native `generation` under the active turn for every provider request;
+- a `pi.tool.<tool-name>` native `tool` observation under the active turn for every tool execution;
+- the Pi session id, working directory, mode, provider, and model;
+- generation token usage and positive total cost when Pi reports a known price;
+- the concrete response model when Pi reports one, with a differing requested alias retained in metadata;
+- error levels and status messages for failed provider responses, model calls, and tools.
+
+Pi does not expose a post-transform provider payload event, so generation request bodies are intentionally omitted rather than risking capture of a payload that a later extension rewrites or redacts. The agent observation still records the user prompt, and assistant output is reconciled after message transformers. Tool input is captured after argument preparation and `tool_call` mutations, while tool output is captured after `tool_result` transformers.
+
+Images and embedded base64 data URIs are represented without their payloads, including provider data URLs. Every captured input or output has one cumulative 64 KiB serialized UTF-8 budget, bounded object/array traversal, and deterministic truncation markers. Langfuse credentials are masked again in the span processor before network export.
+
+Completed observations are exported in batches while Pi remains live. Normal `agent_end` handling never waits for Langfuse network I/O. Use `/langfuse flush` when you need to wait for completed exports; quit shutdown also drains the provider.
+
+Automatic retries or continuations that begin without a new user prompt are recorded as a new trace labeled `[automatic continuation]`, so provider activity is not lost on Pi versions without a final `agent_settled` extension event.
+
+## 💬 Command
+
+```text
+/langfuse status
+/langfuse flush
+/langfuse help
+/langfuse init
+```
+
+- `status` reports whether tracing is enabled, the endpoint, configuration source, and content-capture mode. It never displays credentials.
+- `flush` waits for all completed observations to export.
+- `help` displays command guidance.
+- `init` interactively creates or updates the private config without displaying existing credentials. Blank keys preserve valid existing values; a blank base URL uses the US cloud endpoint.
+
+## 🔐 Privacy
+
+With content capture enabled, traces can contain user prompts, model responses, tool arguments, and tool results. These may include source code, file contents, shell output, or other sensitive project data. Review your Langfuse retention and access controls before enabling this extension.
+
+The built-in mask specifically protects Langfuse credentials; it is not a general secret scanner. Set `"captureContent": false` in `pi-langfuse.json` when content must remain local.
+
+## 🗂️ Package layout
+
+```txt
+extensions/pi-langfuse/
+├── src/
+│   ├── langfuse.ts  # Pi lifecycle integration and slash command
+│   ├── tracing.ts   # Trace lifecycle and content bounding
+│   ├── runtime.ts   # Langfuse/OpenTelemetry runtime
+│   └── config.ts    # Private pi-langfuse.json loading and validation
+├── test/
+├── README.md
+├── LICENSE
+├── tsconfig.json
+└── package.json
+```
+
+Only `langfuse.ts` is a Pi entrypoint; the other source modules are internal.
+
+## 🔎 Keywords
+
+Pi extension, Pi coding agent, Langfuse, LLM observability, OpenTelemetry, tracing, generations, tool spans, token usage, AI agent monitoring.
+
+## 📄 License
+
+MIT. See [`LICENSE`](./LICENSE).

package/package.json

@@ -0,0 +1,49 @@
+{
+	"name": "@narumitw/pi-langfuse",
+	"version": "0.14.1",
+	"description": "Pi extension that traces LLM generations and tool activity to Langfuse.",
+	"type": "module",
+	"license": "MIT",
+	"private": false,
+	"keywords": [
+		"pi-package",
+		"pi-extension",
+		"pi",
+		"langfuse",
+		"observability",
+		"tracing"
+	],
+	"files": [
+		"src",
+		"README.md",
+		"LICENSE"
+	],
+	"pi": {
+		"extensions": [
+			"./src/langfuse.ts"
+		]
+	},
+	"scripts": {
+		"check": "biome check . && npm run typecheck",
+		"format": "biome check --write .",
+		"typecheck": "tsc --noEmit"
+	},
+	"dependencies": {
+		"@langfuse/otel": "^4.0.0",
+		"@langfuse/tracing": "^4.0.0",
+		"@opentelemetry/api": "^1.9.0",
+		"@opentelemetry/sdk-trace-base": "^2.9.0",
+		"@opentelemetry/sdk-trace-node": "^2.9.0"
+	},
+	"devDependencies": {
+		"@biomejs/biome": "2.5.3",
+		"@earendil-works/pi-coding-agent": "0.80.3",
+		"@types/node": "26.1.1",
+		"typescript": "6.0.3"
+	},
+	"repository": {
+		"type": "git",
+		"url": "https://github.com/narumiruna/pi-extensions",
+		"directory": "extensions/pi-langfuse"
+	}
+}

package/src/config.ts

@@ -0,0 +1,208 @@
+import { randomUUID } from "node:crypto";
+import { chmod, mkdir, readFile, rename, rm, stat, writeFile } from "node:fs/promises";
+import { homedir } from "node:os";
+import { dirname, join } from "node:path";
+
+const CONFIG_FILE_NAME = "pi-langfuse.json";
+export const DEFAULT_BASE_URL = "https://us.cloud.langfuse.com";
+
+export interface LangfuseConfig {
+	publicKey: string;
+	secretKey: string;
+	baseUrl: string;
+	environment?: string;
+	release?: string;
+	captureContent: boolean;
+}
+
+export type LangfuseConfigResult =
+	| { ok: true; config: LangfuseConfig; path: string; warnings: string[] }
+	| { ok: false; path: string; warnings: string[]; reason: string };
+
+export function langfuseConfigPath(): string {
+	return join(process.env.PI_CODING_AGENT_DIR ?? join(homedir(), ".pi", "agent"), CONFIG_FILE_NAME);
+}
+
+export async function writeLangfuseConfig(
+	config: LangfuseConfig,
+	path = langfuseConfigPath(),
+): Promise<LangfuseConfig> {
+	const normalized = normalizeLangfuseConfig(config);
+	if (!normalized.ok) throw new Error(normalized.reason);
+
+	await mkdir(dirname(path), { recursive: true, mode: 0o700 });
+	const tempPath = `${path}.${process.pid}.${randomUUID()}.tmp`;
+	try {
+		await writeFile(tempPath, `${JSON.stringify(normalized.config, null, 2)}\n`, {
+			encoding: "utf8",
+			mode: 0o600,
+		});
+		await chmod(tempPath, 0o600);
+		await rename(tempPath, path);
+		await chmod(path, 0o600);
+		return normalized.config;
+	} catch (error) {
+		await rm(tempPath, { force: true }).catch(() => undefined);
+		throw error;
+	}
+}
+
+export async function loadLangfuseConfig(
+	path = langfuseConfigPath(),
+): Promise<LangfuseConfigResult> {
+	const warnings: string[] = [];
+	const permissionFailure = await ensurePrivatePermissions(path, warnings);
+	if (permissionFailure) {
+		return {
+			ok: false,
+			path,
+			warnings,
+			reason: `Refusing to load credentials: ${permissionFailure}`,
+		};
+	}
+
+	let parsed: unknown;
+	try {
+		parsed = JSON.parse(await readFile(path, "utf8"));
+	} catch (error) {
+		if (isNodeError(error) && error.code === "ENOENT") {
+			return { ok: false, path, warnings, reason: `Configuration file not found: ${path}` };
+		}
+		return {
+			ok: false,
+			path,
+			warnings,
+			reason: `Failed to read ${path}: ${formatError(error)}`,
+		};
+	}
+
+	const normalized = normalizeLangfuseConfig(parsed);
+	if (!normalized.ok) return { ok: false, path, warnings, reason: normalized.reason };
+	return { ok: true, config: normalized.config, path, warnings };
+}
+
+export function normalizeLangfuseConfig(
+	value: unknown,
+): { ok: true; config: LangfuseConfig } | { ok: false; reason: string } {
+	if (!value || typeof value !== "object" || Array.isArray(value)) {
+		return { ok: false, reason: "pi-langfuse.json must contain a JSON object." };
+	}
+	const input = value as Record<string, unknown>;
+	const publicKey = normalizeString(input.publicKey);
+	const secretKey = normalizeString(input.secretKey);
+	if (!publicKey) {
+		return { ok: false, reason: "pi-langfuse.json requires a literal publicKey string." };
+	}
+	if (!secretKey) {
+		return { ok: false, reason: "pi-langfuse.json requires a literal secretKey string." };
+	}
+	if (isInterpolation(publicKey)) {
+		return {
+			ok: false,
+			reason:
+				"pi-langfuse.json publicKey must be literal; environment and command interpolation are not supported.",
+		};
+	}
+	if (isInterpolation(secretKey)) {
+		return {
+			ok: false,
+			reason:
+				"pi-langfuse.json secretKey must be literal; environment and command interpolation are not supported.",
+		};
+	}
+
+	const rawBaseUrl =
+		input.baseUrl === undefined ? DEFAULT_BASE_URL : normalizeString(input.baseUrl);
+	if (!rawBaseUrl) {
+		return { ok: false, reason: "pi-langfuse.json baseUrl must be a non-empty string." };
+	}
+	const baseUrl = normalizeBaseUrl(rawBaseUrl);
+	if (!baseUrl) {
+		return {
+			ok: false,
+			reason:
+				"pi-langfuse.json baseUrl must use HTTP or HTTPS without credentials, a query, or a fragment.",
+		};
+	}
+
+	if (input.captureContent !== undefined && typeof input.captureContent !== "boolean") {
+		return { ok: false, reason: "pi-langfuse.json captureContent must be a boolean." };
+	}
+	const environment = optionalString(input.environment, "environment");
+	if (!environment.ok) return environment;
+	const release = optionalString(input.release, "release");
+	if (!release.ok) return release;
+
+	return {
+		ok: true,
+		config: {
+			publicKey,
+			secretKey,
+			baseUrl,
+			...(environment.value ? { environment: environment.value } : {}),
+			...(release.value ? { release: release.value } : {}),
+			captureContent: input.captureContent ?? true,
+		},
+	};
+}
+
+async function ensurePrivatePermissions(
+	path: string,
+	warnings: string[],
+): Promise<string | undefined> {
+	try {
+		const file = await stat(path);
+		if ((file.mode & 0o777) !== 0o600) {
+			await chmod(path, 0o600);
+			const repaired = await stat(path);
+			if ((repaired.mode & 0o777) !== 0o600) {
+				throw new Error(`permissions remained ${(repaired.mode & 0o777).toString(8)}`);
+			}
+			warnings.push(`Restricted ${path} permissions to 0600.`);
+		}
+	} catch (error) {
+		if (isNodeError(error) && error.code === "ENOENT") return undefined;
+		const failure = `Failed to enforce 0600 permissions for ${path}: ${formatError(error)}`;
+		warnings.push(failure);
+		return failure;
+	}
+	return undefined;
+}
+
+function normalizeBaseUrl(value: string): string | undefined {
+	try {
+		const url = new URL(value);
+		if (url.protocol !== "http:" && url.protocol !== "https:") return undefined;
+		if (url.username || url.password || url.search || url.hash) return undefined;
+		return url.toString().replace(/\/+$/, "");
+	} catch {
+		return undefined;
+	}
+}
+
+function optionalString(
+	value: unknown,
+	name: string,
+): { ok: true; value?: string } | { ok: false; reason: string } {
+	if (value === undefined) return { ok: true };
+	const normalized = normalizeString(value);
+	return normalized
+		? { ok: true, value: normalized }
+		: { ok: false, reason: `pi-langfuse.json ${name} must be a non-empty string.` };
+}
+
+function normalizeString(value: unknown): string | undefined {
+	return typeof value === "string" && value.trim() ? value.trim() : undefined;
+}
+
+function isInterpolation(value: string): boolean {
+	return value.startsWith("$") || value.startsWith("!");
+}
+
+function isNodeError(error: unknown): error is NodeJS.ErrnoException {
+	return error instanceof Error && "code" in error;
+}
+
+function formatError(error: unknown): string {
+	return error instanceof Error ? error.message : String(error);
+}