pi-live-speed 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 aisensiy
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,46 @@
+# pi-live-speed
+
+**See how fast your model is generating, while it's still generating.**
+
+Live token speed in [Pi](https://pi.dev/)'s footer. No commands to run, no dashboard to open, no waiting for the response to finish.
+
+```text
+⚡ ~42.0 tok/s · ttft 2.3s · gen 4.1s
+```
+
+![pi-live-speed footer showing live token speed](docs/demo.gif)
+
+- **Live speed:** refreshes every 250 ms, even when the stream pauses.
+- **Live waiting time:** see how long you've been waiting for the first token.
+- **Final speed:** replaces the `~` estimate with provider-reported usage when the response ends.
+- **Stays out of the way:** uses Pi's status area without replacing the footer.
+
+## Install
+
+Requires Node.js 22.19+ and Pi. Tested with Pi 0.87.1.
+
+```bash
+pi install git:github.com/aisensiy/pi-live-speed
+```
+
+Restart Pi or run `/reload`, then send a message. If you previously installed `token-speed.ts`, disable or remove it first to avoid duplicate displays.
+
+The repository currently contains the **0.1.0 release candidate**, available through Git; it has not been published to npm. Live display, response logs and cancellation have been checked on the author's machine. Testing on a second machine is next.
+
+## Good to know
+
+`~` means an estimate during streaming. Final speed uses reported output tokens over the client-observed generation time, including stalls. It is not server-side decoding speed.
+
+Performance logging is enabled by default, without conversation text. See [logging and configuration](docs/configuration.md) to disable it, and [measurement details](docs/measurements.md) for the exact timing and log format.
+
+## Development
+
+```bash
+npm ci
+npm run check
+npm pack --dry-run
+```
+
+## License
+
+[MIT](LICENSE)

package/docs/configuration.md

@@ -0,0 +1,36 @@
+# Logging and configuration
+
+The footer works without configuration. This page covers optional performance history settings; changing them does not change the live display.
+
+## Settings
+
+Optional `~/.pi/agent/pi-live-speed.json` (or under `PI_CODING_AGENT_DIR`):
+
+```json
+{
+  "logging": true,
+  "logPath": "/absolute/path/to/pi-live-speed.jsonl"
+}
+```
+
+Default: logging enabled, writing `pi-live-speed.jsonl` inside Pi's agent directory. Set `logging` to `false` to retain live display without writing history. `logPath` must be absolute; omit it for the default. Reload after edits.
+
+Invalid configuration disables logging and displays a warning rather than silently using defaults. Write failures also show a warning; they do not interrupt the agent.
+
+## Privacy and retention
+
+Logs contain provider/model names, session identifiers and timestamps, which can still be sensitive. Prompts, answers, reasoning text, tool arguments and raw error messages are not saved by this plugin.
+
+Newly created log files use mode `0600`; existing permissions are unchanged. Logs grow until you archive or delete them: automatic retention is not implemented.
+
+See [measurement contract and schema](measurements.md) before comparing providers. Speed and availability do not measure answer correctness.
+
+## Legacy history
+
+The original `scripts/token-speed-stats.sh [log-path]` is retained for **legacy `token-speed.jsonl` only**. It requires Bash, jq, column, awk and GNU date. Do not use its unweighted averages or null handling to analyze the new schema.
+
+Existing legacy logs are not modified by this plugin.
+
+## Display compatibility
+
+Custom footers must render extension statuses; a narrow terminal can truncate them. If you previously loaded a personal `token-speed.ts` extension, disable or remove that copy before loading this package.

package/docs/measurements.md

@@ -0,0 +1,63 @@
+# Measurement contract (schema v1)
+
+## Scope and event boundaries
+
+One row measures one **assistant response observed through Pi's main extension event stream**, not a whole user prompt, tool batch, or HTTP request attempt.
+
+Pi 0.87.1 emits `turn_start` before preparing and streaming each response. `message_start` may arrive only after the provider starts streaming, and is also emitted for immediate errors. `message_end` closes the response before tool execution begins. A tool-following response starts a new turn.
+
+Source: [Pi v0.87.1 agent-loop.ts](https://github.com/earendil-works/pi/blob/v0.87.1/packages/agent/src/agent-loop.ts), particularly `runLoop` and `streamAssistantResponse`.
+
+We deliberately do not use provider request hooks as a primary clock: their public events do not carry a response/request correlation ID, and nested model calls or internal retries cannot safely be attributed to one assistant response. This first version favors an explicit client-observed wait over a misleading network-only TTFT.
+
+- Start: monotonic clock at `turn_start`; fallback to `message_start` if a host omits the turn event. `timingSource` records the distinction.
+- First content: first non-empty `text_delta`, `thinking_delta` or `toolcall_delta`. Start events and empty deltas do not count.
+- TTFT: first content minus start. Includes client preparation and transport, potentially provider-internal retries.
+- Generation seconds: `message_end` minus first content. Includes stream stalls and final usage/stream-finalization delays. Never subtracts stalls.
+- Final TPS: positive, finite provider `usage.output` divided by generation seconds. Output includes reasoning if the provider accounts for it there.
+- Live TPS: accumulated CJK units plus other UTF-16 units divided by four, divided by generation seconds. Fractional counts are accumulated, not rounded per chunk. This is a heuristic, not tokenization; always marked `~`.
+- Fewer than 50 ms of generation, missing positive output usage, or no content delta: TPS is `null`, never a fabricated zero. Zero usage may mean an unreported value, not necessarily zero work.
+
+Durations use `performance.now()` and survive wall-clock changes. Epoch timestamps use `Date.now()` for historical grouping. Client buffering, hidden reasoning and bursty delivery can distort measured speed; it is not the server's internal decoding throughput.
+
+## Lifecycle and coverage
+
+`error` and `aborted` terminal messages are logged even without content or positive usage. A response with no terminal message is recorded as `incomplete` at `agent_end`, session shutdown, session replacement or supersession. Timers are stopped on each completion and shutdown; no timer starts during module discovery or in headless sessions.
+
+Automatic retries that Pi exposes as separate turns get separate rows and response IDs. **Internal HTTP retries within a provider are not separate rows.** Calls by other extensions that do not emit main assistant events are outside this log. Abrupt process termination (for example SIGKILL) cannot flush an active response. This is not a complete HTTP request ledger, and its failure rate must be labeled as an observed assistant-response failure rate.
+
+We do not record raw error messages: they can contain credentials, URLs or conversation fragments. `status` and the controlled `stopReason` category are the first version's failure information; HTTP error classification and attempt correlation are deferred.
+
+## JSONL fields
+
+Each line is a standalone object. No prior line or process-global state is needed.
+
+| Field | Meaning |
+| --- | --- |
+| `schemaVersion` | `1` |
+| `scope` | `assistant-response` |
+| `sessionId`, `responseId` | Pi session identifier and a fresh response UUID |
+| `provider`, `model` | Terminal assistant identity, or initial context identity when no terminal message exists; nullable |
+| `timingSource` | `turn_start` or fallback `message_start` |
+| `startedAt`, `ts` | Start/end epoch milliseconds |
+| `elapsedSec`, `ttftSec`, `genSec` | Monotonic durations in seconds; unavailable TTFT/generation is null |
+| `outputTokens`, `inputTokens`, `cacheReadTokens`, `cacheWriteTokens` | Non-negative provider usage, or null if missing/invalid |
+| `tokenSource` | `provider-usage` when output is positive, otherwise `unavailable` |
+| `tps` | Final average, or null |
+| `unavailableReason` | Null when measurable; otherwise `no-content-delta`, `short-generation`, or `no-output-usage` |
+| `status` | `completed`, `error`, `aborted`, `incomplete` |
+| `stopReason` | Pi terminal reason or a lifecycle category such as `session-shutdown` |
+
+Raw prompts, deltas, reasoning, tool arguments, endpoint URLs and error text are never added to these records. Existing legacy history is not rewritten or mixed with schema v1 by default.
+
+## Comparing suppliers
+
+Separate successes, errors, user cancellations and incomplete samples. Report counts and missing-value coverage. For valid comparable samples, aggregate throughput as `sum(outputTokens) / sum(genSec)`, not the average of response TPS values. TTFT and speed percentiles are useful alongside throughput.
+
+Compare equivalent provider/model, timing source, input/cache and output length ranges. Thinking settings and task difficulty can still confound results; v1 does not record every such dimension and is not a controlled benchmark. Answer quality needs separate tests, task outcomes or human ratings.
+
+## Acceptance status
+
+The author's Pi 0.87.1 installation has been migrated and reloaded. Single-copy footer display and continuous updates were visually confirmed; recorded output usage matched real provider session messages. A manual cancellation produced an `aborted` record, and the footer timer stopped.
+
+Automated tests additionally cover waits and stalls without incoming chunks, lifecycle cleanup, configuration and log failures. Real narrow-width behavior and installation on a second machine remain unverified; the author will perform the latter before the release announcement.

package/package.json

@@ -0,0 +1,47 @@
+{
+  "name": "pi-live-speed",
+  "version": "0.1.0",
+  "description": "See token speed live in Pi's footer, while the model is still generating.",
+  "license": "MIT",
+  "type": "module",
+  "engines": {
+    "node": ">=22.19.0"
+  },
+  "scripts": {
+    "test": "vitest run",
+    "typecheck": "tsc --noEmit",
+    "check:load": "node scripts/check-load.mjs",
+    "check": "npm run typecheck && npm test && npm run check:load"
+  },
+  "keywords": [
+    "pi-package",
+    "tokens",
+    "performance"
+  ],
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/aisensiy/pi-live-speed.git"
+  },
+  "files": [
+    "src",
+    "scripts/token-speed-stats.sh",
+    "README.md",
+    "docs"
+  ],
+  "pi": {
+    "extensions": [
+      "./src/index.ts"
+    ],
+    "image": "./docs/screenshot.png",
+    "video": "./docs/demo.mp4"
+  },
+  "peerDependencies": {
+    "@earendil-works/pi-coding-agent": "*"
+  },
+  "devDependencies": {
+    "@earendil-works/pi-coding-agent": "^0.87.1",
+    "@types/node": "^26.6.3",
+    "typescript": "^7.0.2",
+    "vitest": "^5.0.2"
+  }
+}

package/scripts/token-speed-stats.sh

@@ -0,0 +1,28 @@
+#!/usr/bin/env bash
+# 汇总 token-speed.jsonl：按模型看速度分布，并列出最近 10 次
+LOG="${1:-$HOME/.pi/agent/token-speed.jsonl}"
+[ -f "$LOG" ] || { echo "没有日志：$LOG"; exit 1; }
+
+echo "=== 按模型汇总 ==="
+jq -rs '
+  group_by(.provider + "/" + .model)
+  | map({
+      model: .[0].provider + "/" + .[0].model,
+      runs: length,
+      ttft_avg: ((map(.ttftSec // 0) | add / length) * 100 | round / 100),
+      tps_avg: ((map(.tps) | add / length) * 10 | round / 10),
+      tps_min: (map(.tps) | min),
+      tps_max: (map(.tps) | max),
+      out_avg: ((map(.outputTokens) | add / length) | round)
+    })
+  | sort_by(-.runs)
+  | (["模型","次数","首字均值s","吞吐均值","吞吐最低","吞吐最高","输出均值"]
+     | @tsv),
+    (.[] | [.model, .runs, .ttft_avg, .tps_avg, .tps_min, .tps_max, .out_avg] | @tsv)
+' "$LOG" | column -t -s $'\t'
+
+echo
+echo "=== 最近 10 次 ==="
+tail -10 "$LOG" | jq -r '[.ts, .model, .ttftSec, .genSec, .outputTokens, .tps, .stopReason] | @tsv' \
+  | awk -F'\t' 'BEGIN{print "时间\t模型\t首字s\t生成s\ttoken\t吞吐\t结束原因"} {cmd="date -d @" int($1/1000) " +%H:%M:%S"; cmd | getline t; close(cmd); $1=t; print}' \
+  | column -t -s $'\t'

package/src/index.ts

@@ -0,0 +1,111 @@
+import { randomUUID } from "node:crypto";
+import { getAgentDir, type ExtensionAPI, type ExtensionContext } from "@earendil-works/pi-coding-agent";
+import { finalStatus, Meter, type Outcome, type Usage } from "./meter.ts";
+import { appendRecord, loadConfig, type Config } from "./storage.ts";
+
+const KEY = "pi-live-speed";
+const WARNING_KEY = "pi-live-speed-log";
+
+export default function liveSpeed(pi: ExtensionAPI): void {
+  let active: Meter | undefined;
+  let timer: ReturnType<typeof setInterval> | undefined;
+  let config: Config = { logging: false, logPath: "" };
+  let warned = false;
+
+  function warn(ctx: ExtensionContext, message: string): void {
+    if (warned) return;
+    warned = true;
+    // Do not expose filesystem errors: they may contain private paths or values.
+    ctx.ui.setStatus(WARNING_KEY, "⚠ live-speed log unavailable");
+    ctx.ui.notify(message, "warning");
+  }
+
+  function stopTimer(): void {
+    if (timer) clearInterval(timer);
+    timer = undefined;
+  }
+
+  function finish(ctx: ExtensionContext, status: Outcome, stopReason: string | null, usage?: Usage): void {
+    if (!active) return;
+    const record = active.finish(performance.now(), Date.now(), status, stopReason, usage);
+    active = undefined;
+    stopTimer();
+    ctx.ui.setStatus(KEY, finalStatus(record));
+    if (config.logging) {
+      try {
+        appendRecord(config.logPath, record);
+      } catch {
+        warn(ctx, "pi-live-speed could not append its performance log. Check the configured path and permissions.");
+      }
+    }
+  }
+
+  function start(ctx: ExtensionContext, source: "turn_start" | "message_start"): void {
+    if (active) finish(ctx, "incomplete", "superseded");
+    active = new Meter({
+      sessionId: ctx.sessionManager.getSessionId(),
+      responseId: randomUUID(),
+      provider: ctx.model?.provider ?? null,
+      model: ctx.model?.id ?? null,
+    }, performance.now(), Date.now(), source);
+    ctx.ui.setStatus(KEY, active.live(performance.now()));
+    if (ctx.hasUI) {
+      timer = setInterval(() => {
+        if (active) ctx.ui.setStatus(KEY, active.live(performance.now()));
+      }, 250);
+      timer.unref();
+    }
+  }
+
+  pi.on("session_start", (_event, ctx) => {
+    finish(ctx, "incomplete", "session-switch");
+    stopTimer();
+    warned = false;
+    ctx.ui.setStatus(KEY, undefined);
+    ctx.ui.setStatus(WARNING_KEY, undefined);
+    try {
+      config = loadConfig(getAgentDir());
+    } catch {
+      config = { logging: false, logPath: "" };
+      warn(ctx, "pi-live-speed config could not be read. Logging is disabled until a successful reload; check pi-live-speed.json.");
+    }
+  });
+
+  // A Pi turn is one assistant response plus any subsequent tools, not a whole user prompt.
+  // Starting here measures client-observed wait, including context preparation, not network-only TTFT.
+  pi.on("turn_start", (_event, ctx) => start(ctx, "turn_start"));
+  pi.on("message_start", (event, ctx) => {
+    if (event.message.role !== "assistant") return;
+    if (!active) start(ctx, "message_start");
+    if (active) {
+      active.identity.provider = event.message.provider;
+      active.identity.model = event.message.model;
+    }
+  });
+  pi.on("message_update", (event, ctx) => {
+    if (!active || event.message.role !== "assistant") return;
+    const delta = event.assistantMessageEvent;
+    if (delta.type === "text_delta" || delta.type === "thinking_delta" || delta.type === "toolcall_delta") {
+      active.delta(delta.delta, performance.now());
+      // Ticker handles steady rendering; headless clients still receive an updated status.
+      if (!timer) ctx.ui.setStatus(KEY, active.live(performance.now()));
+    }
+  });
+  pi.on("message_end", (event, ctx) => {
+    const message = event.message;
+    if (message.role !== "assistant") return;
+    if (active) {
+      active.identity.provider = message.provider;
+      active.identity.model = message.model;
+    }
+    const outcome = message.stopReason === "error" || message.stopReason === "aborted" ? message.stopReason : "completed";
+    finish(ctx, outcome, message.stopReason, message.usage);
+  });
+  pi.on("agent_end", (_event, ctx) => finish(ctx, "incomplete", "agent-end-without-message"));
+  pi.on("session_shutdown", (_event, ctx) => {
+    finish(ctx, "incomplete", "session-shutdown");
+    stopTimer();
+    ctx.ui.setStatus(KEY, undefined);
+    ctx.ui.setStatus(WARNING_KEY, undefined);
+  });
+}

package/src/meter.ts

@@ -0,0 +1,109 @@
+export interface Identity {
+  sessionId: string;
+  responseId: string;
+  provider: string | null;
+  model: string | null;
+}
+
+export interface Usage {
+  input?: number;
+  output?: number;
+  cacheRead?: number;
+  cacheWrite?: number;
+}
+
+export type Outcome = "completed" | "error" | "aborted" | "incomplete";
+
+export interface RecordV1 extends Identity {
+  schemaVersion: 1;
+  scope: "assistant-response";
+  timingSource: "turn_start" | "message_start";
+  startedAt: number;
+  ts: number;
+  elapsedSec: number;
+  ttftSec: number | null;
+  genSec: number | null;
+  outputTokens: number | null;
+  inputTokens: number | null;
+  cacheReadTokens: number | null;
+  cacheWriteTokens: number | null;
+  tokenSource: "provider-usage" | "unavailable";
+  tps: number | null;
+  unavailableReason: string | null;
+  status: Outcome;
+  stopReason: string | null;
+}
+
+const count = (n: number | undefined): number | null =>
+  n !== undefined && Number.isFinite(n) && n >= 0 ? n : null;
+const fmt = (n: number) => n >= 100 ? Math.round(n).toString() : n.toFixed(1);
+
+/** A response clock; all durations use injected monotonic milliseconds. Never retains content. */
+export class Meter {
+  private firstDelta: number | null = null;
+  private cjk = 0;
+  private other = 0;
+
+  constructor(
+    readonly identity: Identity,
+    readonly start: number,
+    readonly startedAt: number,
+    readonly timingSource: RecordV1["timingSource"],
+  ) {}
+
+  delta(text: string, now: number): void {
+    if (!text) return;
+    this.firstDelta ??= now;
+    // UTF-16 units keep the estimate invariant even when a surrogate pair is split across chunks.
+    for (let i = 0; i < text.length; i++) {
+      const code = text.charCodeAt(i);
+      if ((code >= 0x3000 && code <= 0x9fff) || (code >= 0xff00 && code <= 0xffef)) this.cjk++;
+      else this.other++;
+    }
+  }
+
+  live(now: number): string {
+    if (this.firstDelta === null) return `⏳ ttft ${fmt((now - this.start) / 1000)}s`;
+    const gen = (now - this.firstDelta) / 1000;
+    const speed = gen >= 0.05 ? `~${fmt((this.cjk + this.other / 4) / gen)}` : "--";
+    return `⚡ ${speed} tok/s · ttft ${fmt((this.firstDelta - this.start) / 1000)}s · gen ${fmt(gen)}s`;
+  }
+
+  finish(now: number, wall: number, status: Outcome, stopReason: string | null, usage?: Usage): RecordV1 {
+    const gen = this.firstDelta === null ? null : (now - this.firstDelta) / 1000;
+    const output = count(usage?.output);
+    let unavailableReason: string | null = null;
+    let tps: number | null = null;
+    if (gen === null) unavailableReason = "no-content-delta";
+    else if (gen < 0.05) unavailableReason = "short-generation";
+    else if (output === null || output === 0) unavailableReason = "no-output-usage";
+    else tps = output / gen;
+    return {
+      ...this.identity,
+      schemaVersion: 1,
+      scope: "assistant-response",
+      timingSource: this.timingSource,
+      startedAt: this.startedAt,
+      ts: wall,
+      elapsedSec: (now - this.start) / 1000,
+      ttftSec: this.firstDelta === null ? null : (this.firstDelta - this.start) / 1000,
+      genSec: gen,
+      outputTokens: output,
+      inputTokens: count(usage?.input),
+      cacheReadTokens: count(usage?.cacheRead),
+      cacheWriteTokens: count(usage?.cacheWrite),
+      tokenSource: output !== null && output > 0 ? "provider-usage" : "unavailable",
+      tps,
+      unavailableReason,
+      status,
+      stopReason,
+    };
+  }
+}
+
+export function finalStatus(record: RecordV1): string {
+  const outcome = record.status === "completed" ? "" : ` · ${record.status}`;
+  return `⚡ ${record.tps === null ? "--" : fmt(record.tps)} tok/s`
+    + ` · ttft ${record.ttftSec === null ? "--" : fmt(record.ttftSec)}s`
+    + ` · gen ${record.genSec === null ? "--" : fmt(record.genSec)}s${outcome}`;
+}

package/src/storage.ts

@@ -0,0 +1,37 @@
+import { appendFileSync, mkdirSync, readFileSync } from "node:fs";
+import { dirname, isAbsolute, join } from "node:path";
+import type { RecordV1 } from "./meter.ts";
+
+export interface Config {
+  logging: boolean;
+  logPath: string;
+}
+
+/** Invalid configuration disables logging rather than silently writing to an unintended location. */
+export function loadConfig(agentDir: string): Config {
+  const defaults: Config = { logging: true, logPath: join(agentDir, "pi-live-speed.jsonl") };
+  let text: string;
+  try {
+    text = readFileSync(join(agentDir, "pi-live-speed.json"), "utf8");
+  } catch (error) {
+    if ((error as NodeJS.ErrnoException).code === "ENOENT") return defaults;
+    throw error;
+  }
+  const value: unknown = JSON.parse(text);
+  if (!value || typeof value !== "object" || Array.isArray(value)) throw new Error("Invalid config");
+  const config = value as Record<string, unknown>;
+  if (Object.keys(config).some((key) => key !== "logging" && key !== "logPath")) throw new Error("Unknown setting");
+  if (config.logging !== undefined && typeof config.logging !== "boolean") throw new Error("Invalid logging setting");
+  if (config.logPath !== undefined && (typeof config.logPath !== "string" || !isAbsolute(config.logPath))) {
+    throw new Error("logPath must be absolute");
+  }
+  return {
+    logging: config.logging === undefined ? defaults.logging : config.logging as boolean,
+    logPath: config.logPath === undefined ? defaults.logPath : config.logPath as string,
+  };
+}
+
+export function appendRecord(path: string, record: RecordV1): void {
+  mkdirSync(dirname(path), { recursive: true, mode: 0o700 });
+  appendFileSync(path, JSON.stringify(record) + "\n", { encoding: "utf8", mode: 0o600 });
+}