pi-bedrouter 0.2.2 → 0.4.1

package/README.md

@@ -7,6 +7,7 @@ A [Pi](https://pi.dev) extension for [bedrouter](https://github.com/bmelton/bedr
 - **registers a `bedrouter` provider** in Pi whose models come straight from `bedrouter.json`: the `auto` aliases (the router decides everything) plus each rung of each ladder
 - **switches the session to `bedrouter/auto`** when the server is healthy, so nobody has to pick a model
 - **shows what served each request in the footer**, live: routed model vs requested, class and deciding signal, the classifier's note, and the session's running cost against what the requested model would have cost
+- **totals the whole session**: every request carries Pi's session id (`x-bedrouter-session`), so bedrouter can add up the session across conversation keys, compaction and sub-agent calls; `/bedrouter usage` shows spend, what the asked-for model would have cost, and the breakdown by route
 - **`/bedrouter`** for everything else: status, start/stop/restart, doctor and the per-rung entitlement probe, the savings report, the last N decisions, pi-agents fit notes
 
 ```
@@ -57,7 +58,8 @@ Example for a developer with a checkout:
 | `/bedrouter install` | `npm install` the dependency, or `npm run build` a checkout that has no `dist/` |
 | `/bedrouter doctor` | Which credential source resolved, expiry, the loaded ladder |
 | `/bedrouter probe` | One 1-token request per rung: which models this AWS account can actually invoke |
-| `/bedrouter report [--since t] [--json]` | Savings report over the decision log |
+| `/bedrouter usage` | This Pi session so far: requests, tokens, spend (models + classifier), what the asked-for model would have cost, saved/over, and a per-route table. Whole-session totals need bedrouter ≥ 0.3; against an older server it shows the current conversation and says so. `usage all` lists recent sessions on the server (`*` marks this one) |
+| `/bedrouter report [--since t] [--session key] [--json]` | Savings report over the decision log (all clients, all sessions; `--session` narrows it to one) |
 | `/bedrouter log [n]` | The last n routing decisions, one line each |
 | `/bedrouter models` | Re-read `bedrouter.json` and re-register the provider (after editing the ladder) |
 | `/bedrouter fitnotes` | Merge model notes into `~/.pi/agent/workflows.json` so the pi-agents planner defaults to `bedrouter/auto` and only pins premium rungs for planning/review |
@@ -65,7 +67,7 @@ Example for a developer with a checkout:
 
 ## How the footer works
 
-Bedrouter echoes every routing decision in response headers (`x-bedrouter-model`, `-requested`, `-class`, `-reason`, `-conversation`, `-classifier`). Pi hands extensions those headers in the `after_provider_response` event, so the status line updates the moment a response starts, before any tokens stream. When the turn ends, the extension asks bedrouter for the conversation's running totals (`GET /v1/conversations/:key`) and appends cost: spend so far, then the routing effect against what the same tokens would have cost on the model the client asked for: `saved $x (n%)`, `same as asked-for`, or `+$x over asked-for (routed up)` when the router chose a stronger rung. The classifier's one-off call is shown separately (`classifier $0.0004`) rather than counted against routing. `↑n` counts escalations in this conversation. The line clears when you switch to a non-bedrouter model. A background health check (every `healthPollS` seconds) keeps it honest between turns: if the server dies the line reads `bedrouter: DOWN` and, with `autoStart` on, the extension restarts it and says so.
+Bedrouter echoes every routing decision in response headers (`x-bedrouter-model`, `-requested`, `-class`, `-reason`, `-conversation`, `-classifier`). Pi hands extensions those headers in the `after_provider_response` event, so the status line updates the moment a response starts, before any tokens stream. When the turn ends, the extension asks bedrouter for the session's running totals (`GET /v1/sessions/:id`, keyed by the Pi session id the extension sends on every request as `x-bedrouter-session`; on a bedrouter older than 0.3 it falls back to the current conversation, `GET /v1/conversations/:key`) and appends cost: spend so far, then the routing effect against what the same tokens would have cost on the model the client asked for: `saved $x (n%)`, `same as asked-for`, or `+$x over asked-for (routed up)` when the router chose a stronger rung. The classifier's one-off call is shown separately (`classifier $0.0004`) rather than counted against routing. `↑n` counts escalations in this session. The line clears when you switch to a non-bedrouter model. A background health check (every `healthPollS` seconds) keeps it honest between turns: if the server dies the line reads `bedrouter: DOWN` and, with `autoStart` on, the extension restarts it and says so.
 
 Under a coding agent every request carries tools and a large system prompt, so you will see `execute` and `explore` decided by keywords or the classifier, then `sticky` for the rest of the session, `up:kw:explore` when an explicit design question moves the conversation up, and escalations after failures. `trivial` shows up for bare chat clients, not for Pi.
 

package/extensions/index.ts

@@ -6,19 +6,21 @@
 //  - optionally switches the session to bedrouter/auto so nobody has to pick a model
 //  - shows which model served each request in the footer, from bedrouter's x-bedrouter-* response headers,
 //    plus the running session cost against what the requested model would have cost
-//  - /bedrouter status|start|stop|restart|install|doctor|probe|report|log|models|fitnotes|config
+//  - tags every request with Pi's session id (x-bedrouter-session) so bedrouter can total the whole session, not just
+//    one conversation key; /bedrouter usage shows those totals
+//  - /bedrouter status|start|stop|restart|install|doctor|probe|usage|report|log|models|fitnotes|config
 import fs from "node:fs";
 import path from "node:path";
 import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent";
 import { loadSettings, saveSettings, settingsPath, agentDir, type Settings } from "../src/settings.js";
 import * as br from "../src/bedrouter.js";
 import { fitNotes, piModels } from "../src/models.js";
-import { downLine, fromHeaders, readyLine, restartingLine, statusLine, type LastDecision, type Paint } from "../src/footer.js";
+import { downLine, fromHeaders, readyLine, restartingLine, sessionsTable, statusLine, usageReport, type CostTotals, type LastDecision, type Paint } from "../src/footer.js";
 
 export default async function (pi: ExtensionAPI) {
   let settings = loadSettings();
   let last: LastDecision | null = null;
-  let stats: br.ConversationStats | null = null;
+  let stats: CostTotals | null = null;            // footer totals: the session when the server supports it, else the last conversation
   let registeredModelIds: string[] = [];
   let lastCtx: ExtensionContext | null = null; // most recent context, for the background poll to update the footer
   let serverUp: boolean | null = null;         // last known health; null = never checked
@@ -27,6 +29,15 @@ export default async function (pi: ExtensionAPI) {
   const ourConversations = new Set<string>();  // bedrouter conversation keys seen from this session
 
   const isOurs = (ctx: ExtensionContext) => ctx.model?.provider === settings.providerName;
+  const sessionId = (ctx: ExtensionContext) => { try { return ctx.sessionManager.getSessionId(); } catch { return null; } };
+  /** Whole-session totals from bedrouter (>= 0.3); falls back to the last conversation on older servers. */
+  async function refreshStats(ctx: ExtensionContext): Promise<{ session: br.SessionStats | null; conversation: br.ConversationStats | null }> {
+    const id = sessionId(ctx);
+    const session = id ? await br.session(settings, id) : null;
+    const conversation = !session && last?.conversation ? await br.conversation(settings, last.conversation) : null;
+    stats = session ?? conversation ?? stats;
+    return { session, conversation };
+  }
   const setStatus = (ctx: ExtensionContext, text: string | undefined) => { lastCtx = ctx; if (ctx.hasUI && settings.footer) ctx.ui.setStatus("bedrouter", text); };
   const paint = (ctx: ExtensionContext | null): Paint => (ctx?.hasUI && ctx.ui.theme ? (c, t) => ctx.ui.theme.fg(c, t) : (_c, t) => t);
   const readyText = (ctx: ExtensionContext | null = lastCtx) => statusLine(last, stats, paint(ctx));
@@ -134,6 +145,15 @@ export default async function (pi: ExtensionAPI) {
   });
 
   // ---- live routing display ------------------------------------------------------------------------------------
+  // Tag requests with Pi's session id: bedrouter's conversation key is derived from the system prompt and first user
+  // message, so one Pi session becomes several conversations (compaction, sub-agents, prompt changes) and trivial or
+  // pinned requests are not tracked at all. The session key is what /bedrouter usage and the footer total against.
+  pi.on("before_provider_headers", async (ev, ctx) => {
+    if (!isOurs(ctx)) return;
+    const id = sessionId(ctx);
+    if (id) ev.headers["x-bedrouter-session"] = id;
+  });
+
   pi.on("after_provider_response", async (ev, ctx) => {
     if (!isOurs(ctx)) return;
     const d = fromHeaders(ev.headers ?? {});
@@ -145,8 +165,8 @@ export default async function (pi: ExtensionAPI) {
   });
 
   pi.on("agent_end", async (_ev, ctx) => {
-    if (!isOurs(ctx) || !last?.conversation) return;
-    stats = await br.conversation(settings, last.conversation);
+    if (!isOurs(ctx) || !last) return;
+    await refreshStats(ctx);
     setStatus(ctx, statusLine(last, stats, paint(ctx)));
   });
 
@@ -182,9 +202,9 @@ export default async function (pi: ExtensionAPI) {
   });
 
   // ---- /bedrouter ------------------------------------------------------------------------------------------------
-  const SUB = ["status", "start", "stop", "restart", "install", "doctor", "probe", "report", "log", "models", "fitnotes", "config", "help"];
+  const SUB = ["status", "start", "stop", "restart", "install", "doctor", "probe", "usage", "report", "log", "models", "fitnotes", "config", "help"];
   pi.registerCommand("bedrouter", {
-    description: "bedrouter router: status | start | stop | restart | install | doctor | probe | report | log [n] | models | fitnotes | config",
+    description: "bedrouter router: status | start | stop | restart | install | doctor | probe | usage [all] | report | log [n] | models | fitnotes | config",
     getArgumentCompletions: (prefix) => SUB.filter((s) => s.startsWith(prefix.trim())).map((value) => ({ value, label: value })),
     handler: async (args, ctx) => {
       const [sub = "status", ...rest] = args.trim().split(/\s+/).filter(Boolean);
@@ -230,6 +250,20 @@ export default async function (pi: ExtensionAPI) {
           show(`bedrouter ${sub}`, r.out);
           break;
         }
+        case "usage": {
+          const h = await br.health(settings);
+          if (!h?.ok) { notify(ctx, `bedrouter is down (${br.baseUrl(settings)}); /bedrouter start`, "error"); break; }
+          const id = sessionId(ctx);
+          if (rest[0] === "all") { show("bedrouter usage: recent sessions on this server", sessionsTable(await br.recentSessions(settings), id ?? undefined)); break; }
+          const { session, conversation } = await refreshStats(ctx);
+          if (session) { show("bedrouter usage: this session", usageReport(session, { sessionId: id ?? undefined })); }
+          else if (conversation) {
+            // older bedrouter: no /v1/sessions; present the last conversation in the same shape
+            show("bedrouter usage: this conversation", usageReport({ ...conversation, errors: 0, conversations: 1, cacheReadTokens: 0, byRoute: conversation.routedModel ? { [`${conversation.requestedModel ?? "?"} -> ${conversation.routedModel}`]: { requests: conversation.requests, costUsd: conversation.costUsd, requestedCostUsd: conversation.requestedCostUsd, inputTokens: conversation.inputTokens, outputTokens: conversation.outputTokens } } : {}, firstTs: conversation.lastTs }, { sessionId: id ?? undefined, conversationOnly: true }));
+          } else show("bedrouter usage", isOurs(ctx) ? `nothing routed yet in this session (${id ?? "no session id"})` : `this session is not using bedrouter (model ${ctx.model?.provider}/${ctx.model?.id}); pick bedrouter/auto with /model`);
+          if (isOurs(ctx) && last) setStatus(ctx, statusLine(last, stats, paint(ctx)));
+          break;
+        }
         case "report": {
           if (!need() || !loc) break;
           const r = br.run(settings, loc, ["report", ...rest]);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-bedrouter",
-  "version": "0.2.2",
+  "version": "0.4.1",
   "description": "Pi extension for bedrouter: starts the cost-aware Bedrock model router, registers it as a provider, and shows which model served each request",
   "keywords": [
     "pi-package",
@@ -43,7 +43,7 @@
     "prepublishOnly": "npm test && npm run typecheck"
   },
   "dependencies": {
-    "bedrouter": "^0.1.0"
+    "bedrouter": "^0.4.1"
   },
   "peerDependencies": {
     "@earendil-works/pi-coding-agent": ">=0.85.0"

package/src/bedrouter.ts

@@ -8,6 +8,8 @@ import type { Settings } from "./settings.js";
 
 export type Health = { ok: boolean; region: string; pid: number; version: string; routing: boolean; classifier: string | null; uptimeS: number };
 export type ConversationStats = { key: string; requests: number; costUsd: number; requestedCostUsd: number; classifierCostUsd: number; inputTokens: number; outputTokens: number; escalations: number; class: string | null; routedModel: string | null; requestedModel: string | null; lastTs: string };
+/** Per-session totals from GET /v1/sessions/:key (bedrouter >= 0.3): every request this Pi session sent, across conversations. */
+export type SessionStats = { key: string; requests: number; errors: number; conversations: number; costUsd: number; requestedCostUsd: number; classifierCostUsd: number; inputTokens: number; outputTokens: number; cacheReadTokens: number; escalations: number; byRoute: Record<string, { requests: number; costUsd: number; requestedCostUsd: number; inputTokens: number; outputTokens: number }>; firstTs: string; lastTs: string };
 export type Rung = { alias: string; bedrockId: string; inputPerM: number; outputPerM: number };
 export type BedrouterConfig = { families: Record<string, Rung[]>; aliases?: Record<string, string>; routing?: { enabled?: boolean; classes?: Record<string, Record<string, string>>; classifier?: { enabled?: boolean; model?: string } } };
 
@@ -124,6 +126,8 @@ export const FALLBACK_CONFIG: BedrouterConfig = {
   routing: { classes: { anthropic: { trivial: "haiku", execute: "sonnet", explore: "opus" }, openai: { execute: "gpt-oss-20b", explore: "gpt-oss-120b" } } },
 };
 export const conversation = (s: Settings, key: string) => getJson<ConversationStats>(`${baseUrl(s)}/v1/conversations/${key}`);
+export const session = (s: Settings, key: string) => getJson<SessionStats>(`${baseUrl(s)}/v1/sessions/${encodeURIComponent(key)}`);
+export const recentSessions = async (s: Settings) => (await getJson<{ data: SessionStats[] }>(`${baseUrl(s)}/v1/sessions`))?.data ?? [];
 export const recentConversations = async (s: Settings) => (await getJson<{ data: ConversationStats[] }>(`${baseUrl(s)}/v1/conversations`))?.data ?? [];
 
 /** True when a conversation other than `ours` sent a request within `windowMs`: another client is using the server. */

package/src/footer.ts

@@ -1,5 +1,8 @@
 // Footer status text: what served the last request and how the session is doing on cost.
-import type { ConversationStats } from "./bedrouter.js";
+import type { ConversationStats, SessionStats } from "./bedrouter.js";
+
+/** The footer totals against a session (preferred, bedrouter >= 0.3) or a single conversation (older servers). */
+export type CostTotals = SessionStats | ConversationStats;
 
 export type LastDecision = { model: string; requested: string; cls: string; reason: string; conversation?: string; classifier?: string };
 
@@ -27,7 +30,7 @@ export function readyLine(paint: Paint = plain): string { return segment(`${pain
 export function downLine(hint: string, paint: Paint = plain): string { return segment(`${paint("error", "✗")} bedrouter down · ${hint}`, paint); }
 export function restartingLine(paint: Paint = plain): string { return segment(`${paint("warning", "↻")} bedrouter restarting…`, paint); }
 
-export function statusLine(d: LastDecision | null, c: ConversationStats | null, paint: Paint = plain): string {
+export function statusLine(d: LastDecision | null, c: CostTotals | null, paint: Paint = plain): string {
   if (!d) return readyLine(paint);
   const arrow = d.model === d.requested ? "=" : "≠";
   let s = `${paint("success", "✓")} bedrouter ⇄ ${d.model} ${arrow} ${d.requested}  ${d.cls}·${short(d.reason)}`;
@@ -46,3 +49,38 @@ export function statusLine(d: LastDecision | null, c: ConversationStats | null,
   }
   return segment(s, paint);
 }
+
+const usd4 = (n: number) => `$${n.toFixed(4)}`;
+const kTok = (n: number) => (n >= 1000 ? `${(n / 1000).toFixed(1)}k` : String(n));
+
+/** The `/bedrouter usage` body: this Pi session's spend so far, what the asked-for model would have cost, and where it went. */
+export function usageReport(st: SessionStats, opts: { sessionId?: string; conversationOnly?: boolean } = {}): string {
+  const spend = st.costUsd + st.classifierCostUsd;
+  const diff = st.requestedCostUsd - st.costUsd;
+  const pct = st.requestedCostUsd > 0 ? (diff / st.requestedCostUsd) * 100 : 0;
+  const mins = Math.max(0, Math.round((Date.parse(st.lastTs) - Date.parse(st.firstTs)) / 60_000));
+  const out: string[] = [];
+  out.push(`session   ${opts.sessionId ?? st.key}${opts.conversationOnly ? "  (current conversation only: this bedrouter predates per-session totals, upgrade it for whole-session numbers)" : ""}`);
+  out.push(`requests  ${st.requests}${st.errors ? ` (${st.errors} errors)` : ""}, conversations ${st.conversations}, escalations ${st.escalations}${mins ? `, over ${mins} min` : ""}`);
+  out.push(`tokens    in ${kTok(st.inputTokens)}  out ${kTok(st.outputTokens)}${st.cacheReadTokens ? `  cache-read ${kTok(st.cacheReadTokens)}` : ""}`);
+  out.push(`spend     ${usd4(spend)}${st.classifierCostUsd > 0 ? `  (models ${usd4(st.costUsd)} + classifier ${usd4(st.classifierCostUsd)})` : ""}`);
+  out.push(`asked-for ${usd4(st.requestedCostUsd)}  what the same tokens would have cost on the model Pi asked for`);
+  if (Math.abs(diff) < 1e-6) out.push(`routing   same as asked-for`);
+  else if (diff > 0) out.push(`routing   saved ${usd4(diff)} (${pct.toFixed(1)}%)`);
+  else out.push(`routing   +${usd4(-diff)} over asked-for (routed up: explore / escalations)`);
+  const routes = Object.entries(st.byRoute).sort((a, b) => b[1].costUsd - a[1].costUsd);
+  if (routes.length) {
+    out.push("");
+    out.push(`  ${"requested -> routed".padEnd(28)} ${"reqs".padStart(5)} ${"in tok".padStart(8)} ${"out tok".padStart(8)} ${"cost".padStart(10)} ${"asked-for".padStart(10)}`);
+    for (const [k, r] of routes) out.push(`  ${k.padEnd(28)} ${String(r.requests).padStart(5)} ${kTok(r.inputTokens).padStart(8)} ${kTok(r.outputTokens).padStart(8)} ${usd4(r.costUsd).padStart(10)} ${usd4(r.requestedCostUsd).padStart(10)}`);
+  }
+  return out.join("\n");
+}
+
+/** One line per session for `/bedrouter usage all`. */
+export function sessionsTable(list: SessionStats[], current?: string): string {
+  if (!list.length) return "(no sessions yet: nothing has sent x-bedrouter-session to this server)";
+  const rows = [`  ${"session".padEnd(38)} ${"reqs".padStart(5)} ${"spend".padStart(10)} ${"asked-for".padStart(10)} ${"saved".padStart(10)}  last`];
+  for (const st of list) rows.push(`${st.key === current ? "*" : " "} ${st.key.slice(0, 38).padEnd(38)} ${String(st.requests).padStart(5)} ${usd4(st.costUsd + st.classifierCostUsd).padStart(10)} ${usd4(st.requestedCostUsd).padStart(10)} ${usd4(st.requestedCostUsd - st.costUsd).padStart(10)}  ${st.lastTs.slice(0, 16).replace("T", " ")}`);
+  return rows.join("\n");
+}