@danypops/jittor 0.5.1 → 0.6.0

package/README.md

@@ -15,6 +15,8 @@ Initial telemetry providers:
 
 - ChatGPT-authenticated Codex subscription usage
 - OpenRouter API key usage, response accounting, and model pricing
+- Anthropic official per-response rate-limit headers (requests, tokens, input/output tokens, and optional Priority Tier buckets)
+- Google Vertex AI classified failure pressure (quota/auth/invalid-request/overload/transport), since Vertex has no documented remaining-budget header or personal polling endpoint
 
 Jittor follows the Papyrus daemon architecture: a supervised Bun service owns SQLite and provider polling; the native Pi extension uses an authenticated loopback client and applies model/thinking decisions.
 
@@ -31,11 +33,13 @@ The initial service scaffold is split into domain, ports, and adapters:
 
 SQLite runs in WAL mode with versioned migrations, JSON validation, bounded queries, chronological indexes, pruning, and checkpoints. The database follows `XDG_DATA_HOME`; private authentication state follows `XDG_STATE_HOME`; the daemon handle follows `XDG_RUNTIME_DIR`.
 
-Operations currently include `metrics.record`, `metrics.query`, `metrics.prune`, `context.assess`, and `service.checkpoint`.
+Operations currently include bounded metric recording/query/pruning, benchmark refresh/status/query, context assessment, routing control, telemetry polling, and service checkpointing. Every operation is exposed through the authenticated typed client; benchmark operations also have CLI parity.
 
-Provider adapters currently include official OpenRouter key/usage/model telemetry and an explicitly experimental Codex subscription adapter. The Codex adapter follows the pinned open-source CLI `/wham/usage` payload and `x-codex-*` response-header contracts, accepts additional metered limits, and fails closed on malformed windows or impossible percentages. File credentials must be explicitly configured and private (`0600`); Jittor reads only the access token and account ID, never refreshes credentials, and never logs or persists OAuth secrets.
+Provider adapters currently include official OpenRouter key/usage/model telemetry and an explicitly experimental Codex subscription adapter. The Codex adapter follows the pinned open-source CLI `/wham/usage` payload and `x-codex-*` response-header contracts, accepts additional metered limits, and fails closed on malformed windows or impossible percentages. File credentials must be explicitly configured and private (`0600`); Jittor reads only the access token and account ID, never refreshes credentials, and never logs or persists OAuth secrets. Anthropic has no personal-account polling endpoint (its Admin/Rate Limits API is documented as unavailable for individual accounts), so Jittor instead reads the official `anthropic-ratelimit-*` response headers Pi observes on every Messages API call and fails closed on schema drift the same way. Google Vertex AI has neither a personal polling endpoint nor a documented remaining-quota response header, so Jittor never fabricates a Vertex budget bar; it instead classifies Vertex's `google.rpc.Status` failure shape (quota, authentication, invalid-request, overload, transport, unknown) from Pi's bounded, content-free `errorMessage` and records only a bounded failure-count metric.
 
-The native Pi extension preflights input and every provider turn, applies model/thinking decisions, records response headers and finalized usage through the daemon, and blocks requests when required telemetry is unsafe. It follows Pi's current authenticated model/provider and synchronizes Pi's available models before every decision, so unavailable catalog routes are never selected. Its responsive integrated footer groups repository and model identity with cumulative usage, a color-coded context-window bar, and current-provider budget telemetry. Codex shows the active model's bounded quota as a draining remaining-budget bar with reset and freshness information. OpenRouter uses the same drain semantics when its official key telemetry exposes a configured limit and remaining balance; keys without a limit remain honest text-only spend and never receive a fabricated denominator. During Pi compaction, the context bar drains as an animation and reports elapsed time. Unknown and stale telemetry are marked explicitly. Run `/jittor` for detailed burn pressure, freshness, route state, and confirmed emergency-halt/override controls.
+The native Pi extension preflights input and every provider turn, applies model/thinking decisions, records response headers and finalized usage through the daemon, and blocks requests when required telemetry is unsafe. It follows Pi's current authenticated model/provider and synchronizes Pi's available models before every decision, so unavailable catalog routes are never selected. Its responsive integrated footer groups repository and model identity with cumulative usage, a color-coded context-window bar, and current-provider budget telemetry. Codex shows the active model's bounded quota as a draining remaining-budget bar with reset and freshness information. OpenRouter uses the same drain semantics when its official key telemetry exposes a configured limit and remaining balance; keys without a limit remain honest text-only spend and never receive a fabricated denominator. Anthropic shows the same drain semantics from its most-restrictive-in-effect token bucket, falling back to the request bucket when no token telemetry has been observed yet. During Pi compaction, the context bar drains against a learned median duration estimated from the last few completed compactions (bounded to the most recent 20 samples, requiring at least 3 before trusting it); until enough evidence exists it falls back to a fixed-rate cold-start heuristic and says so explicitly ("estimating") rather than claiming a time it does not know. Once learned, the status text reports both elapsed time and an approximate remaining time ("~Ns left"). A liveness dot blinks next to the bar once per render tick in either state so compaction never looks stalled. Run `jittor compaction estimate [--json]` to inspect the current estimate and its confidence directly. Unknown and stale telemetry are marked explicitly. Run `/jittor` for the consolidated Settings TUI (its default action), or `/jittor status` for detailed burn pressure, freshness, route state, and confirmed emergency-halt/override controls.
+
+Jittor currently registers no model-callable native tools, so Pi's native model `content` versus renderer `details` contract is explicitly not applicable. Daemon JSON, CLI `--json`, human CLI output, command notifications, panels, and the footer remain separate bounded channels. See [`docs/OUTPUT_CHANNELS.md`](docs/OUTPUT_CHANNELS.md) for the conformance matrix and the requirements that apply if a native tool is introduced later.
 
 Blocking always has a daemon-independent escape hatch. `/jittor off` immediately enters persisted monitor-only mode and never blocks provider requests. The informational footer is independently controlled with `/jittor footer on` and `/jittor footer off`, so showing status never enables enforcement. `/jittor on` only enables enforcement after telemetry polling and available-route synchronization succeed. Every fail-closed error includes these recovery commands plus the daemon restart command.
 
@@ -56,13 +60,33 @@ Jittor observes finalized Codex assistant errors through Pi's public message lif
 
 ### Settings
 
-Run `/jittor settings` for one keyboard-navigable TUI covering routing enforcement, the informational footer, Codex recovery, and all four token-budget thresholds. It uses explicit ON/OFF and configured/not-configured labels, remains bounded on narrow terminals, confirms weaker enforcement and recovery changes, and applies enforcement enablement through the same readiness checks as `/jittor on`. Existing non-TUI commands remain available for automation.
+`/jittor` is the settings and control command. Bare `/jittor` (or `/jittor settings`) opens one keyboard-navigable TUI covering routing enforcement, the informational footer, Codex recovery, and all four token-budget thresholds, with explicit ON/OFF and configured/not-configured labels, bounded rendering on narrow terminals, and confirmation for weaker enforcement/recovery changes. `/jittor status` shows the routing/pressure panel that used to be the bare command's default. Existing non-TUI subcommands (`benchmarks`, `outcome`, `recovery`, `on`/`off`, `footer on`/`off`, `context`) remain available for automation and are unchanged.
+
+### Usage and cost graphs
+
+`/usage` is its own top-level command, separate from `/jittor`. Bare `/usage` opens a colored Unicode cumulative graph with X/Y axes, per-provider/model series, and explicit **Hourly**, **Daily**, **Weekly**, **Monthly**, and **Quarterly** periods; `/usage cost` opens the same graph showing aggregated USD spend instead of tokens, reusing the `cost` metric already recorded content-free on every finalized Pi assistant message (no new instrumentation). Left/Right or Tab/Shift+Tab changes the time frame, `v` toggles between the token and cost views, and `r` refreshes.
+
+Series are colored with a categorical palette chosen to avoid this UI's own status colors ("success"/"warning"/"error" already mean something specific elsewhere in this panel, so reusing them for arbitrary model identity would make a model's bar segment look like a warning or a failure) and instead reuses the theme's syntax-highlighting roles, which are already tuned by theme authors to stay mutually distinguishable on screen — the same design problem as a categorical data palette. Once more series are active than there are hues, a series reuses a hue in bold rather than repeating an indistinguishable color. Multiple models active within the same cumulative time frame are rendered as one bar stacked by color, not separate bars.
+
+Token-budget thresholds are optional and must be configured by the user; Jittor never infers a token allowance from Codex or another provider's subscription percentage. Configure or clear one period with `/usage budget <hourly|daily|weekly|monthly|quarterly> <positive-tokens|off>`, and inspect all of them with `/usage budget`. A configured budget appears as a horizontal threshold on the cumulative graph with explicit remaining or **OVER BUDGET** state; the cost view does not yet support a budget threshold. These private settings persist in `$XDG_CONFIG_HOME/jittor/extension.json` (or `~/.config/jittor/extension.json`).
 
-### Usage graphs
+### Benchmark evidence
 
-Run `/jittor usage` for a colored Unicode cumulative token graph with X/Y axes, provider/model series, input/output/cache totals, refresh, and explicit **Hourly**, **Daily**, **Weekly**, and **Monthly** periods. Left/Right changes period and `r` refreshes. Usage is persisted by the daemon from finalized Pi assistant messages.
+Jittor can ingest bounded OpenRouter model metadata, p50 latency/throughput ordering, and versioned Artificial Analysis benchmark indices as provenance-bearing evidence without treating OpenRouter as model-scope authority. Enable online ingestion explicitly with `JITTOR_OPENROUTER_BENCHMARKS=1`; it is off by default. OpenRouter model metadata and operational ordering are public; benchmark-index ingestion additionally uses `OPENROUTER_API_KEY` from the supervised service environment without retaining it. Snapshots preserve the upstream publisher, normalized model identities, immutable retrieval revisions, source URLs, confidence, license terms, and explicit freshness deadlines. A malformed or oversized refresh leaves the last complete snapshot visible and records only a payload-safe failure state.
+
+Use the authenticated CLI channels independently:
+
+```text
+jittor benchmarks status [--json]
+jittor benchmarks refresh [--force] [--json]
+jittor benchmarks list --source openrouter-models [--model provider/model] [--dimension name] [--limit 1..500] [--json]
+```
 
-Token-budget thresholds are optional and must be configured by the user; Jittor never infers a token allowance from Codex or another provider's subscription percentage. Configure or clear one period with `/jittor usage budget <hourly|daily|weekly|monthly> <positive-tokens|off>`, and inspect all four with `/jittor usage budget`. A configured budget appears as a horizontal threshold on the cumulative graph with explicit remaining or **OVER BUDGET** state. These private settings persist in `$XDG_CONFIG_HOME/jittor/extension.json` (or `~/.config/jittor/extension.json`).
+Only complete snapshots are queryable. Query output reports both completeness and freshness. See [`docs/BENCHMARK_SOURCES.md`](docs/BENCHMARK_SOURCES.md) for source authority, provenance, conflict, and redistribution rules.
+
+Jittor separately records content-free local model observations from Pi's public lifecycle: TTFT, wall latency, output throughput, token/cache/cost efficiency, provider retries, tool-loop counts, failures, and task class derived only from bounded tool names. Prompts, responses, tool arguments/results, credentials, and project paths are never retained. `/jittor outcome accepted` or `/jittor outcome rejected` attaches explicit outcome evidence to the latest completed local run; runtime completion alone is not treated as quality success. Robust aggregates report sample size, median, p90, median absolute deviation, recency, and confidence without merging local observations into external benchmark facts.
+
+The ranking operation accepts an explicit bounded candidate set and never adds identities found only in evidence. It scores task quality, cost, latency, context, and local reliability with bounded user weights, budget-pressure adjustment, component confidence, freshness, provenance, and deterministic tie-breaking. Missing evidence remains unknown and lowers confidence. Run `/jittor benchmarks [coding|research|planning|general]` for the responsive recommendation panel. Because the released Pi extension API does not expose the exact `/scoped-models` set, the current adapter labels candidates `available-models`; the panel says **ADVISORY** and offers no selection action. Automatic route ordering is allowed only for `exact-session` authority and then narrows/reorders routes already present in the supplied candidate set.
 
 ### Context pressure
 
@@ -70,6 +94,26 @@ Papyrus emits content-free prompt-injection observations through Pi's shared ext
 
 Run `/jittor context` for the in-session summary, or `jittor context [--since <epoch-ms>] [--until <epoch-ms>] [--json]` through the authenticated daemon client. The assessment reports bounded average/p95/max injection, Rule/Task mix, unchanged rate, compaction frequency/duration/reasons, and between-compaction provider/cache facts. Repeated prompt content is not labeled billed waste: provider-reported input/cache usage and an injection-disabled control are required before making cost or compaction-causality claims.
 
+### CLI operations
+
+Every daemon operation is reachable from the CLI through the authenticated typed client only — no command reads the SQLite store or a provider adapter directly. Each command supports `--json` for stable machine output; without it, a purpose-built human presenter renders the same result, per [`docs/OUTPUT_CHANNELS.md`](docs/OUTPUT_CHANNELS.md).
+
+```text
+jittor metrics record --source <s> --scope <s> --metric <s> --value <number|null> --unit <unit> [--observed-at <ms>] [--attributes <json>] [--json]
+jittor metrics query [--source <s>] [--scope <s>] [--metric <s>] [--since <ms>] [--until <ms>] [--limit <n>] [--order asc|desc] [--json]
+jittor metrics prune --before <ms> [--json]
+jittor service checkpoint [--json]
+jittor telemetry poll [--json]
+jittor compaction estimate [--json]
+jittor router status|decide|pause|resume|clear-override [--json]
+jittor router override --route <provider/model@thinking> [--expires-at <ms>] [--json]
+jittor router current-route --route <provider/model@thinking> [--json]
+jittor router available-routes [--route <provider/model@thinking> ...] [--json]
+jittor op <operation> [--input <json>]
+```
+
+`jittor op` is a raw escape hatch restricted to the daemon's own `EXPECTED_OPERATION_NAMES`; it rejects an unrecognized operation name before ever reaching the daemon rather than forwarding it blindly. Human-readable metric listings and router status are bounded (at most 50 metric rows and 20 telemetry sources are printed; `--json` output is bounded independently by the daemon's own query and response-size limits). No command prints the daemon bearer token, a provider API key, or an OAuth credential; a daemon-unavailable error stays actionable ("install or start jittor.service") without ever including the token used to reach it.
+
 See [`docs/CALIBRATION.md`](docs/CALIBRATION.md) for thresholds and rollback, and [`docs/USAGE_PRIOR_ART.md`](docs/USAGE_PRIOR_ART.md) for the chart design research.
 
 ```bash

package/docs/BENCHMARK_SOURCES.md

@@ -0,0 +1,32 @@
+# Benchmark evidence sources
+
+Jittor treats benchmark metadata as evidence, not as an unrestricted model catalog. A model is eligible only when it is present in Pi's current scoped-model set; no external source may expand that scope.
+
+## Authority tiers
+
+1. Pi runtime scope and local Jittor observations.
+2. Creator APIs and model cards for identity, availability, capabilities, pricing, and terms.
+3. OpenRouter metadata for its own catalog, pricing, architecture, p50 latency/throughput ordering, and popularity. The authenticated OpenRouter benchmark endpoint retains Artificial Analysis as the upstream publisher rather than relabeling it as OpenRouter quality evidence.
+4. Reproducible independent suites such as SWE-bench Verified, Terminal-Bench, LiveCodeBench, and Aider Polyglot.
+5. Versioned operational measurements and preference datasets.
+
+Provider quality claims remain provider claims. OSINT is discovery-only unless corroborated by an accepted source.
+
+## Required provenance
+
+Every accepted observation identifies:
+
+- the exact provider/model/version and any explicit alias mapping;
+- one evidence dimension and unit;
+- publisher, source type, canonical URL, and immutable revision when available;
+- publication and retrieval timestamps;
+- benchmark dataset, harness, agent, prompting/edit format, effort, trials, and workload when applicable;
+- confidence and license/redistribution status.
+
+Missing evidence is unknown and reduces confidence; it is never converted to a zero score. Scores from incompatible dataset revisions, harnesses, agents, workloads, or effort settings are not merged.
+
+## Freshness and failure
+
+Mutable availability is short-lived, operational observations refresh hourly, price/capability metadata refreshes daily, and immutable benchmark runs do not expire. OpenRouter p50 operational evidence is stored as server-side rank because the Models API exposes ordering but not fabricated latency/throughput values. Fetches and retained observations are bounded. Schema drift or refresh failure preserves the last valid snapshot, marks its freshness honestly, and never replaces it with partially parsed data.
+
+Raw evidence with unknown or restrictive redistribution terms is not republished. Derived facts retain their citation and terms.

package/docs/OUTPUT_CHANNELS.md

@@ -0,0 +1,31 @@
+# Output-channel contracts
+
+Jittor does **not** register model-callable Pi tools. Pi's native `AgentToolResult.content` and `AgentToolResult.details` split is therefore explicitly **not applicable** to the current Jittor extension. Jittor contributes commands, panels, a footer, lifecycle hooks, and authenticated daemon operations only.
+
+`test/output-channels.test.ts` enforces this classification. Adding any `registerTool(...)` call requires a new versioned, bounded model-content and renderer-details contract before release.
+
+## Independent channels
+
+| Channel | Consumer | Contract |
+|---|---|---|
+| Daemon operation JSON | authenticated clients | Typed operation DTOs. Requests are capped at 1 MiB; serialized responses are capped at 4 MiB. Oversized responses fail with `{ "error": "response too large" }`. |
+| CLI `--json` | scripts and agents | Stable daemon DTO JSON. It does not contain human labels or parse TUI output. |
+| Human CLI output | terminal users | Purpose-built presenters such as `formatContextAssessment`; never parsed to recover machine state. |
+| `/jittor` command notifications | Pi users outside an interactive panel | Bounded textual projections of the same application state. |
+| Interactive panels | Pi TUI | Width-aware themed views with bounded telemetry rows and usage-series legends. Panels consume daemon DTOs, not CLI text. |
+| Integrated footer | Pi TUI | One width-bounded informational line. It remains independent from routing enforcement. |
+| Lifecycle/event hooks | Pi runtime | Typed control and telemetry events; they are not presentation output. |
+
+## Data safety
+
+- Metric identity fields and serialized attributes are bounded at ingress.
+- Credential-shaped attribute keys are rejected before persistence.
+- Provider adapters retain normalized telemetry only; raw OAuth tokens, API keys, response bodies, prompts, and project paths are excluded.
+- Human panels omit provider error payloads and sanitize external identity fields before rendering.
+- Status panels show at most 20 telemetry sources.
+- Usage panels show at most 20 provider/model legend rows and explicitly report omitted series.
+- Existing metric queries retain their row limits; the daemon response-byte cap is an independent final boundary.
+
+## Failure semantics
+
+Daemon operations return stable HTTP/JSON errors. The CLI maps operation failures to stderr and a non-zero exit code. Command and TUI adapters present actionable human messages without converting failures into machine-success DTOs. Since there are no native tools, native Pi tool-error rendering is not part of Jittor's current surface.

package/docs/PROVIDER_RESEARCH.md

@@ -10,6 +10,8 @@ Verified 2026-07-18. Contracts marked **official** are documented provider APIs.
 | OpenAI API key | Standard rate-limit headers and response usage | Exact token rate limits and API billing | Official, separate from subscription |
 | OpenRouter | `/api/v1/key`, response `usage`, `/generation`, `/models` | Exact per-request tokens/cost; key limit when configured; otherwise Jittor budget required | Official |
 | OpenRouter Analytics | `/api/v1/analytics/query` with management key | Historical aggregate spend/tokens | Official beta; not hot path |
+| Anthropic | `anthropic-ratelimit-*`/`anthropic-priority-*` response headers on every Messages API call | Exact requests/tokens/input-tokens/output-tokens remaining-vs-limit per response; no personal polling endpoint (Admin API is unavailable for individual accounts) | Official |
+| Google Vertex AI | No documented per-response rate-limit header or personal polling endpoint; errors carry a `google.rpc.Status` shape (`RESOURCE_EXHAUSTED`, `PERMISSION_DENIED`, `UNAVAILABLE`, ...) | No remaining-budget signal is available; Jittor classifies failure kind/transience only, as a bounded failure-count metric, never a fabricated fraction | No official hot-path budget telemetry exists |
 
 ## Codex subscription
 
@@ -157,6 +159,32 @@ Routing rules:
 
 OpenRouter's beta Analytics API accepts management keys and can aggregate `total_usage`, token metrics, cache hit rate, reasoning tokens, model, API key, and generation dimensions. It is useful for calibration and audits, not pre-request routing. The API is beta: discover metadata dynamically, parse count metrics as number or string, and honor truncation metadata.
 
+## Anthropic
+
+### Official per-response rate-limit headers
+
+Anthropic's Rate Limits API documentation states plainly that "the Admin API is unavailable for individual accounts," so there is no personal polling endpoint equivalent to OpenRouter's `/key` or even Codex's experimental `/wham/usage`. What Anthropic does document, on every Messages API response, is a fixed set of headers:
+
+```text
+retry-after
+anthropic-ratelimit-requests-limit / -remaining / -reset
+anthropic-ratelimit-tokens-limit / -remaining / -reset
+anthropic-ratelimit-input-tokens-limit / -remaining / -reset
+anthropic-ratelimit-output-tokens-limit / -remaining / -reset
+anthropic-priority-input-tokens-limit / -remaining / -reset   (Priority Tier only)
+anthropic-priority-output-tokens-limit / -remaining / -reset  (Priority Tier only)
+```
+
+`-reset` values are RFC 3339 timestamps. Anthropic's docs also state the `tokens` bucket headers always reflect "the most restrictive limit currently in effect," so Jittor treats `tokens` as the primary budget signal and falls back to `requests` only when no token telemetry has been observed. Jittor observes these headers from Pi's own `after_provider_response` event (the same mechanism used for Codex's `x-codex-*` headers) rather than daemon-side polling, because there is no standalone endpoint to poll. Schema drift (non-numeric limits, non-RFC-3339 resets, `remaining > limit`) fails closed with a user-visible notice, matching the Codex header-parsing contract.
+
+## Google Vertex AI
+
+### No official hot-path budget telemetry
+
+Unlike Anthropic and OpenRouter, Vertex does not document a per-response rate-limit or remaining-quota header for `generateContent`/Messages-compatible calls. Quota is configured and reported at the Google Cloud project/region level (Service Usage / Quota APIs, Cloud Console "Quotas & System Limits"), which is an account-configuration surface, not a response header Jittor could read before a request is throttled — the same class of limitation Amazon Bedrock has (see Papyrus doc `jittor-provider-survey-which-additional-apis-to-support-0wma`, Tier 2). Failures instead surface as a `google.rpc.Status` shape, `{error: {code, message, status, details[]}}`, with `status` one of the canonical gRPC codes (`RESOURCE_EXHAUSTED`, `PERMISSION_DENIED`, `UNAUTHENTICATED`, `UNAVAILABLE`, `DEADLINE_EXCEEDED`, `INVALID_ARGUMENT`, ...), sometimes with a `google.rpc.RetryInfo.retryDelay` or `google.rpc.QuotaFailure` detail.
+
+Jittor therefore does not fabricate a remaining-budget bar for Vertex. It classifies the bounded, content-free `errorMessage` string Pi already exposes for every provider (the same source `classifyCodexFailure` reads) into a failure kind and transience, and records only a bounded failure-count metric (`source: "google-vertex", scope: "failure", metric: <kind>, unit: "count"`) — never a `ratio` metric implying a known remaining fraction. This is an honest degradation: Jittor surfaces *that* and *what kind of* capacity/auth/request pressure Pi is seeing, without claiming to know how much budget remains.
+
 ## Normalized Jittor model
 
 ```ts
@@ -231,6 +259,16 @@ Required safeguards: hysteresis, cooldown, maximum delay, minimum telemetry fres
 - https://openrouter.ai/docs/cookbook/administration/analytics-cost-control
 - https://openrouter.ai/docs/guides/overview/models
 
+### Anthropic
+
+- https://platform.claude.com/docs/en/api/rate-limits (fetched 2026-07-21)
+- https://platform.claude.com/docs/en/manage-claude/rate-limits-api ("The Admin API is unavailable for individual accounts")
+
+### Google Vertex AI
+
+- Google Cloud/Gemini API 429 `RESOURCE_EXHAUSTED` error reports and `google.rpc.Status`/`QuotaFailure`/`RetryInfo` detail shapes, cross-checked across multiple live incident reports (fetched 2026-07-21); no official Vertex response header for remaining quota was found
+- Papyrus doc `jittor-provider-survey-which-additional-apis-to-support-0wma`, Tier 2 (Amazon Bedrock entry documents the same account-level-quota-not-header pattern)
+
 ### Pi
 
 - Pi extension lifecycle and model APIs: local `docs/extensions.md`

package/extension/src/benchmark-tui.ts

@@ -0,0 +1,103 @@
+import type { ExtensionCommandContext } from "@earendil-works/pi-coding-agent";
+import { matchesKey, truncateToWidth } from "@earendil-works/pi-tui";
+import {
+	BENCHMARK_TUI_MAX_CANDIDATES,
+	BENCHMARK_TUI_MAX_PROVENANCE_PER_CANDIDATE,
+	MODEL_RANKING_DEFAULT_CONTEXT_WEIGHT,
+	MODEL_RANKING_DEFAULT_COST_WEIGHT,
+	MODEL_RANKING_DEFAULT_LATENCY_WEIGHT,
+	MODEL_RANKING_DEFAULT_QUALITY_WEIGHT,
+	MODEL_RANKING_DEFAULT_RELIABILITY_WEIGHT,
+} from "../../src/constants.ts";
+import type { ModelTaskClass } from "../../src/domain/model-observation.ts";
+import type { ModelCandidate, ModelRankingResult, RankedModel, UtilityComponentName } from "../../src/domain/model-ranking.ts";
+
+export interface BenchmarkPanelClient {
+	call(operation: string, input: unknown): Promise<any>;
+}
+
+interface BenchmarkTheme {
+	fg(color: string, text: string): string;
+	bold(text: string): string;
+}
+
+type BenchmarkPanelAction = "refresh" | "close";
+
+const COMPONENT_LABELS: Record<UtilityComponentName, string> = { quality: "Q", cost: "$", latency: "L", context: "C", reliability: "R" };
+
+function componentText(item: RankedModel): string {
+	return item.components.map((component) => `${COMPONENT_LABELS[component.name]} ${component.score === null ? "?" : component.score.toFixed(3)}`).join(" · ");
+}
+
+function candidateLines(item: RankedModel, index: number, currentIdentity: string): string[] {
+	const current = item.identity.startsWith(`${currentIdentity}:`);
+	const localSamples = item.components.find((component) => component.name === "reliability")?.evidenceCount ?? 0;
+	const provenance = item.provenance.slice(0, BENCHMARK_TUI_MAX_PROVENANCE_PER_CANDIDATE).map((source) => `${source.sourceId}@${source.revision} ${source.freshness}`).join(" · ");
+	return [
+		` ${index + 1}. ${item.identity}${index === 0 ? "  recommended" : ""}${current ? "  current" : ""}`,
+		`    utility ${item.utility === null ? "?" : item.utility.toFixed(3)} · confidence ${(item.confidence * 100).toFixed(0)}% · ${componentText(item)}`,
+		`    local n=${localSamples}${provenance ? ` · ${provenance}` : " · no external provenance"}`,
+	];
+}
+
+export function renderBenchmarkView(result: ModelRankingResult, currentIdentity: string, width: number, theme: BenchmarkTheme): string[] {
+	const safeWidth = Math.max(1, width);
+	const shown = result.ranked.slice(0, BENCHMARK_TUI_MAX_CANDIDATES);
+	const currentIndex = result.ranked.findIndex((item) => item.identity.startsWith(`${currentIdentity}:`));
+	const recommended = result.ranked[0];
+	const reason = recommended && currentIndex > 0
+		? `Recommendation differs from current: ${recommended.identity} ranks #1; current ranks #${currentIndex + 1}.`
+		: recommended && currentIndex === 0 ? "Current model is the top recommendation." : "Current model is outside the ranked candidates.";
+	const lines = [
+		theme.fg("borderMuted", "─".repeat(safeWidth)),
+		theme.bold("Jittor Benchmark Recommendations"),
+		result.scopeAuthority === "exact-session" ? "Scope: exact session" : "Scope: available models · ADVISORY (exact session scope unavailable)",
+		`Task: ${result.taskClass} · evidence ${result.completeness}`,
+		reason,
+		...shown.flatMap((item, index) => candidateLines(item, index, currentIdentity)),
+		...(result.ranked.length > shown.length ? [` … ${result.ranked.length - shown.length} more candidates omitted`] : []),
+		...(result.scopeWarning ? [result.scopeWarning] : []),
+		theme.fg("dim", "r refresh · Esc close"),
+		theme.fg("borderMuted", "─".repeat(safeWidth)),
+	];
+	return lines.map((line) => truncateToWidth(line, safeWidth, "…"));
+}
+
+export async function showBenchmarkPanel(
+	ctx: ExtensionCommandContext,
+	client: BenchmarkPanelClient,
+	candidates: ModelCandidate[],
+	currentIdentity: string,
+	taskClass: ModelTaskClass,
+): Promise<void> {
+	for (;;) {
+		const result = await client.call("models.rank", {
+			candidates,
+			scopeAuthority: "available-models",
+			taskClass,
+			budgetPressure: 0,
+			weights: {
+				quality: MODEL_RANKING_DEFAULT_QUALITY_WEIGHT,
+				cost: MODEL_RANKING_DEFAULT_COST_WEIGHT,
+				latency: MODEL_RANKING_DEFAULT_LATENCY_WEIGHT,
+				context: MODEL_RANKING_DEFAULT_CONTEXT_WEIGHT,
+				reliability: MODEL_RANKING_DEFAULT_RELIABILITY_WEIGHT,
+			},
+			sourceIds: ["openrouter-models", "openrouter-artificial-analysis"],
+		}) as ModelRankingResult;
+		if (ctx.mode !== "tui") {
+			ctx.ui.notify(renderBenchmarkView(result, currentIdentity, 100, { fg: (_color, text) => text, bold: (text) => text }).join("\n"), "info");
+			return;
+		}
+		const action = await ctx.ui.custom<BenchmarkPanelAction>((_tui, theme, _keybindings, done) => ({
+			invalidate() {},
+			render(width: number): string[] { return renderBenchmarkView(result, currentIdentity, width, theme); },
+			handleInput(data: string): void {
+				if (matchesKey(data, "escape") || matchesKey(data, "ctrl+c")) done("close");
+				else if (data === "r") done("refresh");
+			},
+		}));
+		if (!action || action === "close") return;
+		await client.call("benchmark.refresh", { force: true });
+	}
+}

package/extension/src/footer.ts

@@ -6,6 +6,7 @@ import {
 	FOOTER_BAR_MIN_WIDTH,
 	FOOTER_CONTEXT_ACCENT_FRACTION,
 	FOOTER_CONTEXT_ERROR_FRACTION,
+	FOOTER_COMPACTION_BLINK_HALF_PERIOD_MS,
 	FOOTER_COMPACTION_DRAIN_STEP_MS,
 	FOOTER_CONTEXT_WARNING_FRACTION,
 	FOOTER_WIDE_TERMINAL_WIDTH,
@@ -65,6 +66,9 @@ export type ProviderBudget = {
 export interface CompactionProgress {
 	startedAt: number;
 	initialFraction: number;
+	/** Learned median duration from jittor-cli's `compaction.estimate`; absent/null means cold-start. */
+	estimatedMs?: number | null;
+	confidence?: "cold-start" | "learned";
 }
 
 interface UsageTotals {
@@ -133,12 +137,49 @@ function fillColor(fraction: number | null): FooterColor {
 	return "dim";
 }
 
+/**
+ * Once a learned median duration is available (see estimateCompactionDuration / the
+ * `compaction.estimate` daemon operation), the bar drains against that real estimate: fraction
+ * counts down linearly from 1 to 0 over estimatedMs. Until then — cold start, or the estimate
+ * fetch has not resolved yet — it falls back to draining from the context fill at a fixed rate,
+ * which is a liveness heuristic only and never labeled as a time estimate (see
+ * compactionStatusText, which is what actually communicates cold-start vs learned confidence).
+ */
 function compactionFraction(progress: CompactionProgress, width: number, now: number): number {
+	if (progress.confidence === "learned" && typeof progress.estimatedMs === "number" && progress.estimatedMs > 0) {
+		const elapsed = Math.max(0, now - progress.startedAt);
+		return Math.max(0, Math.min(1, 1 - (elapsed / progress.estimatedMs)));
+	}
 	const initialFilled = Math.round(Math.min(1, Math.max(0, progress.initialFraction)) * width);
 	const drained = Math.floor(Math.max(0, now - progress.startedAt) / FOOTER_COMPACTION_DRAIN_STEP_MS);
 	return Math.max(0, initialFilled - drained) / width;
 }
 
+/**
+ * Liveness blink independent of whether the drain bar reflects a learned estimate or the
+ * fixed-rate cold-start fallback: it does not claim to know how long compaction will take, only
+ * that it has not stalled. It toggles once per render tick so a single owned interval (installed
+ * in beginCompactionUi) drives both the drain and the blink — no extra timer is created here.
+ */
+export function compactionBlinkOn(startedAt: number, now: number, halfPeriodMs = FOOTER_COMPACTION_BLINK_HALF_PERIOD_MS): boolean {
+	const elapsed = Math.max(0, now - startedAt);
+	return Math.floor(elapsed / halfPeriodMs) % 2 === 0;
+}
+
+function compactionBlinkGlyph(progress: CompactionProgress, now: number): string {
+	return compactionBlinkOn(progress.startedAt, now) ? "●" : "○";
+}
+
+/** Communicates cold-start uncertainty distinctly from a learned approximate completion time. */
+function compactionStatusText(progress: CompactionProgress, now: number): string {
+	const elapsedSeconds = Math.floor(Math.max(0, now - progress.startedAt) / MILLISECONDS_PER_SECOND);
+	if (progress.confidence === "learned" && typeof progress.estimatedMs === "number" && progress.estimatedMs > 0) {
+		const remainingSeconds = Math.max(0, Math.ceil((progress.estimatedMs - (now - progress.startedAt)) / MILLISECONDS_PER_SECOND));
+		return `compact ${elapsedSeconds}s (~${remainingSeconds}s left)`;
+	}
+	return `compact ${elapsedSeconds}s (estimating)`;
+}
+
 function contextSegment(
 	context: FooterContext,
 	theme: FooterTheme,
@@ -150,8 +191,7 @@ function contextSegment(
 	const w = barWidth(width);
 	if (compaction) {
 		const fraction = compactionFraction(compaction, w, now);
-		const elapsedSeconds = Math.floor(Math.max(0, now - compaction.startedAt) / MILLISECONDS_PER_SECOND);
-		return `ctx ${theme.fg("accent", progressBar(fraction, w))} compact ${elapsedSeconds}s`;
+		return `ctx ${theme.fg("accent", progressBar(fraction, w))} ${compactionBlinkGlyph(compaction, now)} ${compactionStatusText(compaction, now)}`;
 	}
 	const usage = context.getContextUsage();
 	const window = usage?.contextWindow ?? context.model?.contextWindow ?? 0;
@@ -172,7 +212,7 @@ function minimalContextSegment(
 	const w = barWidth(width);
 	if (compaction) {
 		const fraction = compactionFraction(compaction, w, now);
-		return `ctx ${theme.fg("accent", progressBar(fraction, w))}`;
+		return `ctx ${theme.fg("accent", progressBar(fraction, w))} ${compactionBlinkGlyph(compaction, now)}`;
 	}
 	const percent = context.getContextUsage()?.percent;
 	const fraction = percent === null || percent === undefined ? null : percent / 100;