free-coding-models 0.5.58 → 0.5.59
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +34 -0
- package/changelog/v0.5.59.md +52 -0
- package/package.json +2 -2
- package/src/core/ping.js +7 -2
- package/src/core/provider-quota-fetchers.js +254 -2
- package/src/core/router-daemon.js +19 -3
- package/src/tui/app.js +5 -1
- package/src/tui/render-table.js +28 -2
- package/web/dist/assets/{index-4Jq00xKl.js → index-C_ZdUGrS.js} +2 -2
- package/web/dist/assets/{index-BkK1gdJN.css → index-wI9xrm0w.css} +1 -1
- package/web/dist/index.html +2 -2
- package/web/src/components/router/RouterView.jsx +34 -0
- package/web/src/components/router/RouterView.module.css +53 -0
package/README.md
CHANGED
|
@@ -517,6 +517,39 @@ Inspect or wipe it manually any time — it's plain JSON with `0600` perms.
|
|
|
517
517
|
|
|
518
518
|
---
|
|
519
519
|
|
|
520
|
+
## 📊 Live quota from response headers
|
|
521
|
+
|
|
522
|
+
Every chat-completion response carries rate-limit headers (`x-ratelimit-remaining-requests`, etc.). The daemon parses them passively on every routed request — **zero extra network requests, zero quota waste**.
|
|
523
|
+
|
|
524
|
+
- **8 header variants** supported: SambaNova, Mistral, generic `x-ratelimit-*`, the `ratelimit-*` (no-`x-`) variant some proxies use, and 3 daily / token / token-minute variants for Cerebras-style providers.
|
|
525
|
+
- **Pre-warmed by pings**: even health-check pings return headers, so quota is visible in the CLI before you ever route a real request through the daemon.
|
|
526
|
+
- **5-minute staleness**: snapshots older than 5 minutes are excluded, with the active `/api/v1/key` fetcher as fallback for idle providers.
|
|
527
|
+
- **Case-insensitive**: some proxies vary casing; we handle all of them.
|
|
528
|
+
- **Garbage-safe**: `limit: 0`, non-numeric values, or missing pairs return `null` (no crash).
|
|
529
|
+
|
|
530
|
+
### Where it shows up
|
|
531
|
+
|
|
532
|
+
| Surface | What you'll see |
|
|
533
|
+
|---------|-----------------|
|
|
534
|
+
| TUI footer | `📊 groq 78% · sambanova 41%` chip (top 5 most-depleted first) |
|
|
535
|
+
| Web Dashboard | New `Provider Quota` section with animated progress bars per provider |
|
|
536
|
+
| `/api/router/stats` | `quota: { providerKey: { remaining, limit, percent, source, lastUpdated, windowType } }` |
|
|
537
|
+
|
|
538
|
+
### Daemon `/stats.quota` shape
|
|
539
|
+
|
|
540
|
+
```json
|
|
541
|
+
{
|
|
542
|
+
"quota": {
|
|
543
|
+
"groq": { "remaining": 14, "limit": 30, "percent": 47, "windowType": "requests", "source": "header", "lastUpdated": 1753478400000 },
|
|
544
|
+
"sambanova": { "remaining": 1500, "limit": 14400, "percent": 10, "windowType": "day", "source": "header", "lastUpdated": 1753478400000 }
|
|
545
|
+
}
|
|
546
|
+
}
|
|
547
|
+
```
|
|
548
|
+
|
|
549
|
+
The `source` field is `"header"` when the value came from a passive response header, `"endpoint"` when it came from the active `/api/v1/key` fetcher. Use it in devtools to debug why a provider shows the value it does.
|
|
550
|
+
|
|
551
|
+
---
|
|
552
|
+
|
|
520
553
|
## π Pi Extension — FCM-Pi ⚠️ BETA
|
|
521
554
|
|
|
522
555
|
**FCM-Pi** is a native [Pi coding agent](https://pi.dev) extension that integrates `free-coding-models` directly into your Pi session. It stays silent by default, scans only when you run `/fcm`, and lets you explicitly hot-swap models mid-session.
|
|
@@ -730,6 +763,7 @@ See [`packages/fcm-agent-core/README.md`](./packages/fcm-agent-core/README.md) f
|
|
|
730
763
|
- **Mandatory self-update policy** — startup checks npm for a newer FCM and installs it automatically without a prompt. If the install fails twice in a row (offline, proxy, or permissions), FCM still starts but shows a red outdated-version warning until the user retries with `Shift+U` or runs the displayed install command.
|
|
731
764
|
- **Last release timestamp** — light pink footer shows `Last release: Mar 27, 2026, 09:42 PM` from npm so users know how fresh the data is
|
|
732
765
|
- **Persistent probe-cache (t1)** — every health probe result is cached to `~/.free-coding-models/probe-cache.json` for 24h. Warm starts render the full ranking in <500ms, only re-ping the models that are due (broken or TTL-expired). Broken models are auto-hidden across sessions — toggle visibility with **Shift+B**. See [Persistent probe cache](#-persistent-probe-cache) below for `--reprobe`, `--probe-ttl`, `--show-broken`.
|
|
766
|
+
- **Live quota from response headers (t2)** — every routed chat-completion response already carries `x-ratelimit-*` headers. The daemon parses them in 8 variants and exposes live per-provider quota on the TUI footer (`📊 groq 78% · sambanova 41%`) and in the Web Dashboard (`Provider Quota` section with animated progress bars). Zero extra network requests, zero quota waste. See [Live quota from headers](#-live-quota-from-response-headers) below.
|
|
733
767
|
|
|
734
768
|
---
|
|
735
769
|
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
# Changelog v0.5.59 - 2026-07-26
|
|
2
|
+
|
|
3
|
+
### Added
|
|
4
|
+
|
|
5
|
+
- 📊 **Live quota from response headers** (t2) — every chat-completion response already carries `x-ratelimit-*` headers. The daemon parses them passively on every routed request (and even on health-check pings), exposing **live** per-provider quota on the TUI footer and Web Dashboard. **Zero extra network requests, zero quota waste.**
|
|
6
|
+
|
|
7
|
+
- **8 header variants** supported in priority order:
|
|
8
|
+
- `x-ratelimit-remaining-requests` / `-limit-requests` (SambaNova-style)
|
|
9
|
+
- `x-ratelimit-remaining` / `-limit` (Mistral / generic)
|
|
10
|
+
- `ratelimit-remaining-requests` / `ratelimit-limit-requests` (proxy-stripped)
|
|
11
|
+
- `ratelimit-remaining` / `ratelimit-limit` (proxy-stripped generic)
|
|
12
|
+
- `x-ratelimit-remaining-requests-day` / `-limit-requests-day` (SambaNova daily)
|
|
13
|
+
- `x-ratelimit-remaining-day` / `-limit-day` (generic daily)
|
|
14
|
+
- `x-ratelimit-remaining-tokens` / `-limit-tokens` (Cerebras tokens)
|
|
15
|
+
- `x-ratelimit-remaining-tokens-minute` / `-limit-tokens-minute` (Cerebras token-minute)
|
|
16
|
+
- **First-match-wins priority** — when multiple pairs are present, the most-specific wins (e.g. daily variant beats generic).
|
|
17
|
+
- **Case-insensitive** — some proxies vary casing; we handle `X-RateLimit-Remaining` and `x-ratelimit-remaining` identically.
|
|
18
|
+
- **Garbage-safe** — `limit: 0`, non-numeric values, or missing pairs return `null` (no crash, no div-by-zero).
|
|
19
|
+
- **Staleness 5 min** — snapshots older than `STALENESS_MS` are excluded from `getAllQuotas()` so the consumer only sees fresh data. The existing active `/api/v1/key` fetcher stays as the fallback for idle providers.
|
|
20
|
+
- **Merge with active fetcher** — `getQuota(providerKey)` returns whichever is freshest, so quota is always live when traffic flows.
|
|
21
|
+
|
|
22
|
+
- 🛰️ **TUI footer chip** — `📊 groq 78% · sambanova 41%` (top 5 most-depleted first; icon escalation 📊 healthy → ⚠️ ≤ 25% → 🚨 ≤ 10%). Renders on the same footer line as the existing probe-cache chip.
|
|
23
|
+
|
|
24
|
+
- 🌐 **Web Dashboard `Provider Quota` section** — per-provider card with source icon (🛰️ passive header vs 🔌 active fetcher fallback), provider key, animated progress bar (green ≥ 25% / amber 10–25% / red ≤ 10%), and percent label. Auto-sorted most-depleted first. Updated every `/api/router/stats` poll (5 s).
|
|
25
|
+
|
|
26
|
+
- 📋 **Daemon `/api/router/stats` exposes `quota`** — `{ providerKey: { remaining, limit, percent, windowType, source, lastUpdated } }`. The `source` field tells you whether the value came from a passive header (`'header'`) or the active fetcher (`'endpoint'`) — useful for debugging why a provider shows the number it does.
|
|
27
|
+
|
|
28
|
+
- 🧪 **`processResponseHeaders(providerKey, headers)`** — drop-in hook for any future surface (Tauri Desktop, scripts, IDE plugins) that wants to feed the passive map.
|
|
29
|
+
|
|
30
|
+
### Changed
|
|
31
|
+
|
|
32
|
+
- 🩺 **Daemon reverse proxy** (`buildUpstreamMeta` in `src/core/router-daemon.js`) now calls `processResponseHeaders(providerKey, response.headers)` for every routed upstream response, both streaming and non-streaming. Zero new code paths, zero new dependencies.
|
|
33
|
+
|
|
34
|
+
- 📡 **Health-check pings** (`src/core/ping.js`) also call `processResponseHeaders(providerKey, resp.headers)`, so even the CLI-only flow (no daemon) shows live quota after the first ping cycle.
|
|
35
|
+
|
|
36
|
+
- 🔧 **Internal refactor** — the existing `extractQuotaPercent` in `src/core/ping.js` now uses the shared `extractQuota()` from `provider-quota-fetchers.js`. Same 8-variant parser, single source of truth, no behaviour change for existing callers.
|
|
37
|
+
|
|
38
|
+
### Maintenance
|
|
39
|
+
|
|
40
|
+
- 🧪 **+37 unit tests** (`test/passive-quota.test.js`, new file): HEADER_PAIRS structure, STALENESS_MS, all 8 header pairs parse correctly, robustness (missing fields, NaN, limit=0, negative limit, empty object, null/undefined headers), first-pair-wins priority, case-insensitivity, Fetch Headers object support, `processResponseHeaders` writes/overwrites/ignores bad providerKey, `getQuota` passive fresh/stale/custom maxAgeMs, `getAllQuotas` unions both stores / excludes stale, `formatQuotaStatus` healthy/low/critical + day window + stale/missing.
|
|
41
|
+
- 🧪 **632 → 671 tests passing** (`pnpm test`), **118 → 125 suites**.
|
|
42
|
+
- 🧹 `vite build` succeeds.
|
|
43
|
+
- 📖 README updated with a new **📊 Live quota from response headers** section (8 variants table, where it shows up, `/stats.quota` JSON shape).
|
|
44
|
+
|
|
45
|
+
### Inspiration
|
|
46
|
+
|
|
47
|
+
This implementation is informed by [`apmantza/pi-free`](https://github.com/apmantza/pi-free)'s `lib/quota-monitor.ts` (which itself credits us: *"Inspired by free-coding-models' `extractQuotaPercent`"*). The inspiration flows both ways — we extend their 6-pair design to 8 variants (adding the Cerebras tokens / token-minute pairs we already supported) and ship a Web Dashboard renderer that surfaces them as live progress bars instead of just CLI text.
|
|
48
|
+
|
|
49
|
+
### Files
|
|
50
|
+
|
|
51
|
+
- **Modified**: `src/core/provider-quota-fetchers.js` (+228 lines: new pure-logic section), `src/core/ping.js` (call `processResponseHeaders` on every ping), `src/core/router-daemon.js` (call on every routed response + expose `quota` in `/stats`), `src/tui/render-table.js` (new footer chip), `src/tui/app.js` (thread `quota` through), `web/src/components/router/RouterView.jsx` (new `Provider Quota` section), `web/src/components/router/RouterView.module.css` (7 new classes), `README.md`, `tasks/t2.md`, `package.json` (test script).
|
|
52
|
+
- **New**: `test/passive-quota.test.js` (340 lines, 37 tests), `changelog/v0.5.59.md`.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "free-coding-models",
|
|
3
|
-
"version": "0.5.
|
|
3
|
+
"version": "0.5.59",
|
|
4
4
|
"description": "Find the fastest coding LLM models in seconds \u2014 ping free models from multiple providers, pick the best one for OpenCode, Cursor, or any AI coding assistant.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"nvidia",
|
|
@@ -53,7 +53,7 @@
|
|
|
53
53
|
],
|
|
54
54
|
"scripts": {
|
|
55
55
|
"start": "node bin/free-coding-models.js",
|
|
56
|
-
"test": "node --test test/test.js test/fcm-agent-core.test.js test/patch-openclaw.test.js test/provider-metadata.test.js test/config-permission-hint.test.js test/probe-cache.test.js",
|
|
56
|
+
"test": "node --test test/test.js test/fcm-agent-core.test.js test/patch-openclaw.test.js test/provider-metadata.test.js test/config-permission-hint.test.js test/probe-cache.test.js test/passive-quota.test.js",
|
|
57
57
|
"prepack": "npm run build:web",
|
|
58
58
|
"dev": "node scripts/dev-web.mjs",
|
|
59
59
|
"dev:web": "node scripts/dev-web.mjs",
|
package/src/core/ping.js
CHANGED
|
@@ -41,7 +41,7 @@
|
|
|
41
41
|
*/
|
|
42
42
|
|
|
43
43
|
import { PING_TIMEOUT } from './constants.js'
|
|
44
|
-
import { fetchProviderQuota as _fetchProviderQuotaFromModule } from './provider-quota-fetchers.js'
|
|
44
|
+
import { fetchProviderQuota as _fetchProviderQuotaFromModule, extractQuota as _extractQuotaFromModule, processResponseHeaders as _processResponseHeadersFromModule } from './provider-quota-fetchers.js'
|
|
45
45
|
import { supportsUsagePercent } from './quota-capabilities.js'
|
|
46
46
|
|
|
47
47
|
const DISABLED_THINKING_RETRY_STATUSES = new Set([400, 422])
|
|
@@ -173,10 +173,15 @@ export async function ping(apiKey, modelId, providerKey, url) {
|
|
|
173
173
|
}
|
|
174
174
|
// 📖 Normalize all HTTP 2xx statuses to "200" so existing verdict/avg logic still works.
|
|
175
175
|
const code = resp.status >= 200 && resp.status < 300 ? '200' : String(resp.status)
|
|
176
|
+
// 📖 Passive quota tracker (t2): parse the response headers via the shared module.
|
|
177
|
+
// 📖 1) Write to the passive snapshot map so /stats + TUI footer see live quota.
|
|
178
|
+
// 📖 2) Return the percent number for the per-call consumer below.
|
|
179
|
+
const extracted = _extractQuotaFromModule(resp.headers)
|
|
180
|
+
_processResponseHeadersFromModule(providerKey, resp.headers)
|
|
176
181
|
return {
|
|
177
182
|
code,
|
|
178
183
|
ms: Math.round(performance.now() - t0),
|
|
179
|
-
quotaPercent:
|
|
184
|
+
quotaPercent: extracted ? extracted.percent : null,
|
|
180
185
|
}
|
|
181
186
|
} catch (err) {
|
|
182
187
|
const isTimeout = err.name === 'AbortError'
|
|
@@ -1,23 +1,42 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* @file lib/provider-quota-fetchers.js
|
|
3
|
-
* @description Provider endpoint quota pollers
|
|
3
|
+
* @description Provider endpoint quota pollers + passive rate-limit header parser.
|
|
4
4
|
*
|
|
5
|
-
*
|
|
5
|
+
* Active fetchers (existing):
|
|
6
6
|
* - openrouter: GET https://openrouter.ai/api/v1/key
|
|
7
7
|
* derives percent from limit_remaining/limit (with fallback field names)
|
|
8
8
|
* - siliconflow: GET https://api.siliconflow.cn/v1/user/info
|
|
9
9
|
* returns balance info; percent is null (no limit field to derive from)
|
|
10
10
|
*
|
|
11
|
+
* Passive tracker (t2):
|
|
12
|
+
* - Every chat-completion response carries rate-limit headers (x-ratelimit-*).
|
|
13
|
+
* - processResponseHeaders() parses those headers in 6 priority variants and
|
|
14
|
+
* writes to an in-memory map, kept fresh per `STALENESS_MS` (5 min default).
|
|
15
|
+
* - getQuota() merges the passive snapshot with the latest active fetch,
|
|
16
|
+
* returning whichever is freshest — so quota is *always* live when traffic
|
|
17
|
+
* flows, with the active fetcher as a safety net for idle periods.
|
|
18
|
+
* - Zero extra network requests: the headers are already on every response.
|
|
19
|
+
*
|
|
11
20
|
* Features:
|
|
12
21
|
* - TTL cache (default 60s) prevents hammering endpoints
|
|
13
22
|
* - Error backoff (default 15s) after failures
|
|
14
23
|
* - Injectable fetch + time for testing
|
|
15
24
|
* - API keys are never logged
|
|
25
|
+
* - Case-insensitive header parsing (some proxies vary casing)
|
|
16
26
|
*
|
|
17
27
|
* @exports parseOpenRouterResponse(data) → number|null
|
|
18
28
|
* @exports parseSiliconFlowResponse(data) → { balance, chargeBalance, totalBalance }|null
|
|
19
29
|
* @exports createProviderQuotaFetcher(options) → fetcher(providerKey, apiKey) → Promise<number|null>
|
|
20
30
|
* @exports fetchProviderQuota(providerKey, apiKey, options) → Promise<number|null>
|
|
31
|
+
* @exports extractQuota(headers) → { remaining, limit, percent, source, windowType }|null
|
|
32
|
+
* @exports processResponseHeaders(providerKey, headers, opts?) → boolean
|
|
33
|
+
* @exports getQuota(providerKey, opts?) → QuotaSnapshot|null
|
|
34
|
+
* @exports getAllQuotas(opts?) → ReadonlyMap<string, QuotaSnapshot>
|
|
35
|
+
* @exports formatQuotaStatus(providerKey, opts?) → string|undefined
|
|
36
|
+
* @exports resetPassiveQuota() — clear the in-memory passive map (tests)
|
|
37
|
+
* @exports HEADER_PAIRS — readonly array of [remainingKey, limitKey] pairs in priority order
|
|
38
|
+
* @exports STALENESS_MS — passive snapshots older than this are considered stale
|
|
39
|
+
* @exports QUOTA_WINDOW_LABELS — map of windowType → short label for tooltips
|
|
21
40
|
*/
|
|
22
41
|
|
|
23
42
|
// ─── Response parsers (pure, no I/O) ─────────────────────────────────────────
|
|
@@ -317,3 +336,236 @@ export async function fetchProviderQuota(providerKey, apiKey, options = {}) {
|
|
|
317
336
|
|
|
318
337
|
return pendingPromise
|
|
319
338
|
}
|
|
339
|
+
|
|
340
|
+
// ─── Passive rate-limit header tracker (t2) ───────────────────────────────────
|
|
341
|
+
|
|
342
|
+
/**
|
|
343
|
+
* 📖 STALENESS_MS: how long a passive snapshot stays "fresh" before we prefer
|
|
344
|
+
* 📖 the active fetcher result (or hide the chip entirely if neither is fresh).
|
|
345
|
+
* 📖 Mirrors pi-free's 5-minute window. Override per call via opts.now - opts.maxAgeMs.
|
|
346
|
+
*/
|
|
347
|
+
export const STALENESS_MS = 5 * 60 * 1000
|
|
348
|
+
|
|
349
|
+
/**
|
|
350
|
+
* 📖 HEADER_PAIRS: ordered list of [remainingKey, limitKey] pairs to try when
|
|
351
|
+
* 📖 parsing a response's rate-limit headers. First pair where both values parse
|
|
352
|
+
* 📖 as finite numbers AND limit > 0 wins. Order matters: most-specific (day,
|
|
353
|
+
* 📖 tokens) comes before generic (requests) where applicable.
|
|
354
|
+
*
|
|
355
|
+
* 📖 Provenance:
|
|
356
|
+
* 📖 - x-ratelimit-remaining-requests / x-ratelimit-limit-requests → SambaNova
|
|
357
|
+
* 📖 - x-ratelimit-remaining / x-ratelimit-limit → Mistral / generic
|
|
358
|
+
* 📖 - ratelimit-remaining-requests / ratelimit-limit-requests → proxies that strip 'x-' prefix
|
|
359
|
+
* 📖 - ratelimit-remaining / ratelimit-limit → same, generic
|
|
360
|
+
* 📖 - x-ratelimit-remaining-requests-day / x-ratelimit-limit-requests-day → SambaNova daily window
|
|
361
|
+
* 📖 - x-ratelimit-remaining-day / x-ratelimit-limit-day → generic daily
|
|
362
|
+
*/
|
|
363
|
+
export const HEADER_PAIRS = [
|
|
364
|
+
['x-ratelimit-remaining-requests', 'x-ratelimit-limit-requests'],
|
|
365
|
+
['x-ratelimit-remaining', 'x-ratelimit-limit'],
|
|
366
|
+
['ratelimit-remaining-requests', 'ratelimit-limit-requests'],
|
|
367
|
+
['ratelimit-remaining', 'ratelimit-limit'],
|
|
368
|
+
['x-ratelimit-remaining-requests-day', 'x-ratelimit-limit-requests-day'],
|
|
369
|
+
['x-ratelimit-remaining-day', 'x-ratelimit-limit-day'],
|
|
370
|
+
['x-ratelimit-remaining-tokens', 'x-ratelimit-limit-tokens'],
|
|
371
|
+
['x-ratelimit-remaining-tokens-minute', 'x-ratelimit-limit-tokens-minute'],
|
|
372
|
+
]
|
|
373
|
+
|
|
374
|
+
/**
|
|
375
|
+
* 📖 QUOTA_WINDOW_LABELS: short tooltip labels keyed by windowType suffix
|
|
376
|
+
* 📖 detected in the matched header pair name. Used by formatQuotaStatus to
|
|
377
|
+
* 📖 indicate whether the user is looking at a per-minute or per-day window.
|
|
378
|
+
*/
|
|
379
|
+
export const QUOTA_WINDOW_LABELS = {
|
|
380
|
+
day: 'day',
|
|
381
|
+
requests: 'min',
|
|
382
|
+
tokens: 'tok',
|
|
383
|
+
'tokens-minute': 'tok/min',
|
|
384
|
+
}
|
|
385
|
+
|
|
386
|
+
/**
|
|
387
|
+
* 📖 Internal: in-memory map of latest passive quota snapshot per provider.
|
|
388
|
+
* 📖 Keyed by providerKey; never persisted to disk (passive tracking is local-only).
|
|
389
|
+
*/
|
|
390
|
+
const _passiveQuota = new Map() // providerKey -> QuotaSnapshot
|
|
391
|
+
|
|
392
|
+
/**
|
|
393
|
+
* 📖 Case-insensitive header lookup. Accepts both Fetch `Headers` objects and
|
|
394
|
+
* 📖 plain object literals (some test doubles pass plain objects).
|
|
395
|
+
*/
|
|
396
|
+
function readHeader(headers, key) {
|
|
397
|
+
if (!headers) return null
|
|
398
|
+
if (typeof headers.get === 'function') {
|
|
399
|
+
return headers.get(key) ?? headers.get(key.toLowerCase()) ?? null
|
|
400
|
+
}
|
|
401
|
+
if (typeof headers === 'object') {
|
|
402
|
+
if (key in headers) return headers[key]
|
|
403
|
+
const lower = key.toLowerCase()
|
|
404
|
+
if (lower in headers) return headers[lower]
|
|
405
|
+
// 📖 Iterate as a last resort — some servers use unusual casings.
|
|
406
|
+
for (const k of Object.keys(headers)) {
|
|
407
|
+
if (k.toLowerCase() === lower) return headers[k]
|
|
408
|
+
}
|
|
409
|
+
}
|
|
410
|
+
return null
|
|
411
|
+
}
|
|
412
|
+
|
|
413
|
+
/**
|
|
414
|
+
* 📖 Parse rate-limit headers and extract a structured quota snapshot.
|
|
415
|
+
* 📖 Returns null when no header pair matches (caller decides to keep stale).
|
|
416
|
+
*
|
|
417
|
+
* @param {Headers | Record<string, string> | null | undefined} headers
|
|
418
|
+
* @returns {{ remaining: number, limit: number, percent: number, source: string, windowType: string } | null}
|
|
419
|
+
*/
|
|
420
|
+
export function extractQuota(headers) {
|
|
421
|
+
for (const [remainingKey, limitKey] of HEADER_PAIRS) {
|
|
422
|
+
const remainingRaw = readHeader(headers, remainingKey)
|
|
423
|
+
const limitRaw = readHeader(headers, limitKey)
|
|
424
|
+
if (remainingRaw == null || limitRaw == null) continue
|
|
425
|
+
const remaining = Number.parseFloat(remainingRaw)
|
|
426
|
+
const limit = Number.parseFloat(limitRaw)
|
|
427
|
+
if (!Number.isFinite(remaining) || !Number.isFinite(limit) || limit <= 0) continue
|
|
428
|
+
const percent = Math.max(0, Math.min(100, Math.round((remaining / limit) * 100)))
|
|
429
|
+
// 📖 Derive windowType from the matching pair's key suffix.
|
|
430
|
+
let windowType = 'requests'
|
|
431
|
+
if (remainingKey.endsWith('-day')) windowType = 'day'
|
|
432
|
+
else if (remainingKey.endsWith('-tokens-minute')) windowType = 'tokens-minute'
|
|
433
|
+
else if (remainingKey.endsWith('-tokens')) windowType = 'tokens'
|
|
434
|
+
return { remaining, limit, percent, source: remainingKey, windowType }
|
|
435
|
+
}
|
|
436
|
+
return null
|
|
437
|
+
}
|
|
438
|
+
|
|
439
|
+
/**
|
|
440
|
+
* 📖 Internal: build a QuotaSnapshot from extractQuota() output + timestamp.
|
|
441
|
+
*/
|
|
442
|
+
function makeSnapshot(extracted, source = 'header', now = Date.now()) {
|
|
443
|
+
return {
|
|
444
|
+
remaining: extracted.remaining,
|
|
445
|
+
limit: extracted.limit,
|
|
446
|
+
percent: extracted.percent,
|
|
447
|
+
windowType: extracted.windowType,
|
|
448
|
+
headerSource: extracted.source,
|
|
449
|
+
source, // 'header' (passive) or 'endpoint' (active fetcher)
|
|
450
|
+
lastUpdated: now,
|
|
451
|
+
}
|
|
452
|
+
}
|
|
453
|
+
|
|
454
|
+
/**
|
|
455
|
+
* 📖 processResponseHeaders: hook for the daemon reverse-proxy + ping responses.
|
|
456
|
+
* 📖 Parses the response headers, writes the snapshot to the passive map, and
|
|
457
|
+
* 📖 returns true if a snapshot was stored (so callers can decide to log).
|
|
458
|
+
*
|
|
459
|
+
* @param {string} providerKey
|
|
460
|
+
* @param {Headers | Record<string, string> | null | undefined} headers
|
|
461
|
+
* @param {object} [opts]
|
|
462
|
+
* @param {number} [opts.now=Date.now()]
|
|
463
|
+
* @returns {boolean} true if a snapshot was written
|
|
464
|
+
*/
|
|
465
|
+
export function processResponseHeaders(providerKey, headers, opts = {}) {
|
|
466
|
+
if (!providerKey || typeof providerKey !== 'string') return false
|
|
467
|
+
const now = opts.now ?? Date.now()
|
|
468
|
+
const extracted = extractQuota(headers)
|
|
469
|
+
if (!extracted) return false
|
|
470
|
+
_passiveQuota.set(providerKey, makeSnapshot(extracted, 'header', now))
|
|
471
|
+
return true
|
|
472
|
+
}
|
|
473
|
+
|
|
474
|
+
/**
|
|
475
|
+
* 📖 Internal: read the latest active-fetcher snapshot for a provider. The active
|
|
476
|
+
* 📖 fetcher uses a per-key Map of { value, expiresAt } entries; we synthesise a
|
|
477
|
+
* 📖 QuotaSnapshot from that. Returns null if the active cache is empty/expired.
|
|
478
|
+
*/
|
|
479
|
+
function getActiveSnapshot(providerKey, now = Date.now()) {
|
|
480
|
+
for (const [cacheKey, entry] of _defaultCache.entries()) {
|
|
481
|
+
if (!cacheKey.startsWith(`${providerKey}:`)) continue
|
|
482
|
+
if (!entry || typeof entry !== 'object') continue
|
|
483
|
+
if (typeof entry.expiresAt !== 'number' || entry.expiresAt <= now) continue
|
|
484
|
+
const value = entry.value
|
|
485
|
+
if (typeof value !== 'number' || !Number.isFinite(value)) continue
|
|
486
|
+
return makeSnapshot(
|
|
487
|
+
{ remaining: value, limit: 100, percent: value, windowType: 'unknown', source: 'active_fetcher' },
|
|
488
|
+
'endpoint',
|
|
489
|
+
// 📖 active fetcher stores wall-clock ms at fetch time; expose it so
|
|
490
|
+
// 📖 getQuota's freshest-wins logic works on real timestamps.
|
|
491
|
+
now,
|
|
492
|
+
)
|
|
493
|
+
}
|
|
494
|
+
return null
|
|
495
|
+
}
|
|
496
|
+
|
|
497
|
+
/**
|
|
498
|
+
* 📖 getQuota: merge passive + active snapshots, return the freshest.
|
|
499
|
+
* 📖 A snapshot is "stale" when older than STALENESS_MS. If both are stale,
|
|
500
|
+
* 📖 returns null (caller should hide the chip).
|
|
501
|
+
*
|
|
502
|
+
* @param {string} providerKey
|
|
503
|
+
* @param {object} [opts]
|
|
504
|
+
* @param {number} [opts.now=Date.now()]
|
|
505
|
+
* @param {number} [opts.maxAgeMs=STALENESS_MS]
|
|
506
|
+
* @returns {QuotaSnapshot | null}
|
|
507
|
+
*/
|
|
508
|
+
export function getQuota(providerKey, opts = {}) {
|
|
509
|
+
if (!providerKey) return null
|
|
510
|
+
const now = opts.now ?? Date.now()
|
|
511
|
+
const maxAgeMs = opts.maxAgeMs ?? STALENESS_MS
|
|
512
|
+
const passive = _passiveQuota.get(providerKey) || null
|
|
513
|
+
const active = getActiveSnapshot(providerKey, now)
|
|
514
|
+
|
|
515
|
+
// 📖 Drop stale snapshots.
|
|
516
|
+
const candidates = []
|
|
517
|
+
if (passive && now - passive.lastUpdated <= maxAgeMs) candidates.push(passive)
|
|
518
|
+
if (active && now - active.lastUpdated <= maxAgeMs) candidates.push(active)
|
|
519
|
+
|
|
520
|
+
if (candidates.length === 0) return null
|
|
521
|
+
// 📖 Freshest wins — tied timestamps prefer passive (it's the live signal).
|
|
522
|
+
return candidates.reduce((a, b) => (a.lastUpdated >= b.lastUpdated ? a : b))
|
|
523
|
+
}
|
|
524
|
+
|
|
525
|
+
/**
|
|
526
|
+
* 📖 getAllQuotas: snapshot of every provider we know about, merged passive+active.
|
|
527
|
+
* 📖 Stale entries (older than maxAgeMs) are excluded. Used by /stats and the TUI footer.
|
|
528
|
+
*
|
|
529
|
+
* @param {object} [opts]
|
|
530
|
+
* @returns {ReadonlyMap<string, QuotaSnapshot>}
|
|
531
|
+
*/
|
|
532
|
+
export function getAllQuotas(opts = {}) {
|
|
533
|
+
const now = opts.now ?? Date.now()
|
|
534
|
+
const maxAgeMs = opts.maxAgeMs ?? STALENESS_MS
|
|
535
|
+
const out = new Map()
|
|
536
|
+
// 📖 Union the keys from both passive and active stores so we don't miss a
|
|
537
|
+
// 📖 provider whose latest signal only exists in one.
|
|
538
|
+
const allKeys = new Set([..._passiveQuota.keys()])
|
|
539
|
+
for (const cacheKey of _defaultCache.keys()) {
|
|
540
|
+
const colon = cacheKey.indexOf(':')
|
|
541
|
+
if (colon > 0) allKeys.add(cacheKey.slice(0, colon))
|
|
542
|
+
}
|
|
543
|
+
for (const providerKey of allKeys) {
|
|
544
|
+
const q = getQuota(providerKey, { now, maxAgeMs })
|
|
545
|
+
if (q) out.set(providerKey, q)
|
|
546
|
+
}
|
|
547
|
+
return out
|
|
548
|
+
}
|
|
549
|
+
|
|
550
|
+
/**
|
|
551
|
+
* 📖 formatQuotaStatus: human-readable "⚠️ groq: 12/100 (12%) [day]" string.
|
|
552
|
+
* 📖 Returns undefined when the snapshot is missing or stale (caller hides the chip).
|
|
553
|
+
*
|
|
554
|
+
* @param {string} providerKey
|
|
555
|
+
* @param {object} [opts]
|
|
556
|
+
* @returns {string | undefined}
|
|
557
|
+
*/
|
|
558
|
+
export function formatQuotaStatus(providerKey, opts = {}) {
|
|
559
|
+
const snapshot = getQuota(providerKey, opts)
|
|
560
|
+
if (!snapshot) return undefined
|
|
561
|
+
const window = QUOTA_WINDOW_LABELS[snapshot.windowType] || snapshot.windowType
|
|
562
|
+
const icon = snapshot.percent <= 10 ? '🚨' : snapshot.percent <= 25 ? '⚠️ ' : '📊'
|
|
563
|
+
return `${icon} ${providerKey}: ${snapshot.remaining}/${snapshot.limit} (${snapshot.percent}%) [${window}]`
|
|
564
|
+
}
|
|
565
|
+
|
|
566
|
+
/**
|
|
567
|
+
* 📖 resetPassiveQuota: clear the in-memory passive map. Test-only utility.
|
|
568
|
+
*/
|
|
569
|
+
export function resetPassiveQuota() {
|
|
570
|
+
_passiveQuota.clear()
|
|
571
|
+
}
|
|
@@ -63,6 +63,11 @@ import {
|
|
|
63
63
|
isCacheFresh as isProbeCacheFresh,
|
|
64
64
|
pruneStaleEntries as pruneProbeCacheStaleEntries,
|
|
65
65
|
} from './probe-cache.js'
|
|
66
|
+
import {
|
|
67
|
+
processResponseHeaders as processPassiveQuotaHeaders,
|
|
68
|
+
getAllQuotas as getAllPassiveQuotas,
|
|
69
|
+
STALENESS_MS as PASSIVE_QUOTA_STALENESS_MS,
|
|
70
|
+
} from './provider-quota-fetchers.js'
|
|
66
71
|
|
|
67
72
|
export const ROUTER_DEFAULT_PORT = 19280
|
|
68
73
|
export const ROUTER_MAX_PORT = 19289
|
|
@@ -475,11 +480,16 @@ function serveWebStaticFile(res, pathname, requestId) {
|
|
|
475
480
|
serveStaticFromDist(res, candidate)
|
|
476
481
|
}
|
|
477
482
|
|
|
478
|
-
function buildUpstreamMeta(response, text = '') {
|
|
483
|
+
function buildUpstreamMeta(response, text = '', providerKey = '') {
|
|
479
484
|
// 📖 Keep quota diagnostics structural only: headers and retry timing are safe,
|
|
480
485
|
// 📖 while upstream response bodies stay out of logs and telemetry.
|
|
481
486
|
const rateLimitHeaders = extractRateLimitHeaders(response.headers)
|
|
482
487
|
const retryAfterMs = parseRetryAfterMs(rateLimitHeaders['retry-after'])
|
|
488
|
+
// 📖 Passive quota tracker (t2): every upstream response carries rate-limit
|
|
489
|
+
// 📖 headers — we parse them once here and write to the in-memory snapshot
|
|
490
|
+
// 📖 map. Zero extra network requests; works on providers with no quota
|
|
491
|
+
// 📖 endpoint. See src/core/provider-quota-fetchers.js for the 8 header pairs.
|
|
492
|
+
if (providerKey) processPassiveQuotaHeaders(providerKey, response.headers)
|
|
483
493
|
const quotaExhausted = response.status === 429
|
|
484
494
|
|| hasZeroRemainingQuota(rateLimitHeaders)
|
|
485
495
|
|| /\b(quota|rate[_ -]?limit|too many requests)\b/i.test(text || '')
|
|
@@ -1487,6 +1497,12 @@ class RouterRuntime {
|
|
|
1487
1497
|
// 📖 Surfaced so the Web Dashboard + CLI can show cache hit rate + how many
|
|
1488
1498
|
// 📖 broken models are currently hidden. Refreshed every /stats call.
|
|
1489
1499
|
probeCache: getProbeCacheStats(),
|
|
1500
|
+
// 📖 Passive quota (t2): latest known rate-limit headers per provider, keyed
|
|
1501
|
+
// 📖 by providerKey. Each entry is { remaining, limit, percent, source,
|
|
1502
|
+
// 📖 lastUpdated } — source can be 'header' (live) or 'endpoint' (active
|
|
1503
|
+
// 📖 fetcher fallback). Stale entries (older than PASSIVE_QUOTA_STALENESS_MS)
|
|
1504
|
+
// 📖 are excluded so the consumer only sees fresh data.
|
|
1505
|
+
quota: Object.fromEntries(getAllPassiveQuotas()),
|
|
1490
1506
|
}
|
|
1491
1507
|
}
|
|
1492
1508
|
|
|
@@ -2051,7 +2067,7 @@ class RouterRuntime {
|
|
|
2051
2067
|
clearTimeout(timeout)
|
|
2052
2068
|
const latencyMs = Math.round(performance.now() - started)
|
|
2053
2069
|
const text = await response.text()
|
|
2054
|
-
const upstreamMeta = buildUpstreamMeta(response, text)
|
|
2070
|
+
const upstreamMeta = buildUpstreamMeta(response, text, candidate.provider)
|
|
2055
2071
|
|
|
2056
2072
|
if (isLikelyHtmlResponse(response.headers, text)) {
|
|
2057
2073
|
this.markFailure(key, 'upstream_html_maintenance', 503, upstreamMeta)
|
|
@@ -2193,7 +2209,7 @@ class RouterRuntime {
|
|
|
2193
2209
|
})
|
|
2194
2210
|
clearTimeout(timeout)
|
|
2195
2211
|
const latencyMs = Math.round(performance.now() - started)
|
|
2196
|
-
const upstreamMeta = buildUpstreamMeta(response)
|
|
2212
|
+
const upstreamMeta = buildUpstreamMeta(response, '', candidate.provider)
|
|
2197
2213
|
if (isLikelyHtmlResponse(response.headers)) {
|
|
2198
2214
|
this.markFailure(key, 'upstream_html_maintenance', 503, upstreamMeta)
|
|
2199
2215
|
this.recordRouterError('upstream_html_maintenance', requestId, { model: key, status: response.status, stream: true })
|
package/src/tui/app.js
CHANGED
|
@@ -99,7 +99,7 @@ import { buildMergedModels } from '../core/model-merger.js'
|
|
|
99
99
|
import { loadOpenCodeConfig, saveOpenCodeConfig } from '../core/opencode-config.js'
|
|
100
100
|
import { usageForRow as _usageForRow } from '../core/usage-reader.js'
|
|
101
101
|
import { buildProviderModelTokenKey, loadTokenUsageByProviderModel } from '../core/token-usage-reader.js'
|
|
102
|
-
import { parseOpenRouterResponse, fetchProviderQuota as _fetchProviderQuotaFromModule } from '../core/provider-quota-fetchers.js'
|
|
102
|
+
import { parseOpenRouterResponse, fetchProviderQuota as _fetchProviderQuotaFromModule, getAllQuotas as _getAllPassiveQuotasFromModule } from '../core/provider-quota-fetchers.js'
|
|
103
103
|
import { isKnownQuotaTelemetry } from '../core/quota-capabilities.js'
|
|
104
104
|
import { ALT_ENTER, ALT_LEAVE, ALT_HOME, PING_TIMEOUT, PING_INTERVAL, FPS, COL_MODEL, COL_MS, CELL_W, FRAMES, TIER_CYCLE, VERDICT_CYCLE, HEALTH_CYCLE, SETTINGS_OVERLAY_BG, HELP_OVERLAY_BG, RECOMMEND_OVERLAY_BG, OVERLAY_PANEL_WIDTH, TABLE_HEADER_LINES, TABLE_FOOTER_LINES, TABLE_FIXED_LINES, WIDTH_WARNING_MIN_COLS, msCell, spinCell } from '../core/constants.js'
|
|
105
105
|
import { TIER_COLOR } from './tier-colors.js'
|
|
@@ -954,6 +954,10 @@ export async function runApp(cliArgs, config, startupOptions = {}) {
|
|
|
954
954
|
probeCacheMisses: state.probeCacheMisses || 0,
|
|
955
955
|
probeCacheBrokenHidden: state.probeCacheBrokenHidden || 0,
|
|
956
956
|
showBrokenMode: !!state.showBrokenMode,
|
|
957
|
+
// 📖 Passive quota (t2): live snapshots from response headers, populated
|
|
958
|
+
// 📖 by pings (every ping pre-warms quota for its provider). See
|
|
959
|
+
// 📖 src/core/provider-quota-fetchers.js for getAllPassiveQuotas().
|
|
960
|
+
quota: Object.fromEntries(_getAllPassiveQuotasFromModule()),
|
|
957
961
|
}
|
|
958
962
|
if (state.commandPaletteOpen) {
|
|
959
963
|
if (!state.commandPaletteFrozenTable) {
|
package/src/tui/render-table.js
CHANGED
|
@@ -155,6 +155,10 @@ export const PROVIDER_COLOR = new Proxy({}, {
|
|
|
155
155
|
* probeCacheMisses?: number, // t1: number of models that needed a live probe
|
|
156
156
|
* probeCacheBrokenHidden?: number, // t1: number of broken models auto-hidden this session
|
|
157
157
|
* showBrokenMode?: boolean, // t1: true when Shift+B has un-hidden broken models
|
|
158
|
+
* quota?: Record<string, { // t2: live quota from response headers
|
|
159
|
+
* remaining: number, limit: number, percent: number,
|
|
160
|
+
* source: 'header'|'endpoint', lastUpdated: number, windowType?: string,
|
|
161
|
+
* }>,
|
|
158
162
|
* }} opts
|
|
159
163
|
* @returns {string}
|
|
160
164
|
*/
|
|
@@ -205,6 +209,7 @@ export function renderTable({
|
|
|
205
209
|
probeCacheMisses = 0,
|
|
206
210
|
probeCacheBrokenHidden = 0,
|
|
207
211
|
showBrokenMode = false,
|
|
212
|
+
quota = {},
|
|
208
213
|
} = _) {
|
|
209
214
|
// 📖 Filter out hidden models for display
|
|
210
215
|
const visibleResults = results.filter(r => !r.hidden)
|
|
@@ -1219,8 +1224,27 @@ export function renderTable({
|
|
|
1219
1224
|
probeCacheLabel = chalk.bgRgb(40, 90, 140).rgb(220, 235, 255).bold(` ${cachedTxt}${brokenTxt} `)
|
|
1220
1225
|
}
|
|
1221
1226
|
|
|
1222
|
-
// 📖
|
|
1223
|
-
|
|
1227
|
+
// 📖 Passive quota chip (t2): `📊 groq 78% · sambanova 41%` (top-N depleted first).
|
|
1228
|
+
// 📖 Shows live rate-limit headers that the daemon / pings collected from upstream
|
|
1229
|
+
// 📖 responses. Zero extra network requests — see provider-quota-fetchers.js.
|
|
1230
|
+
// 📖 Cap at 5 providers to avoid footer bloat; pick most-depleted first.
|
|
1231
|
+
let quotaLabel = ''
|
|
1232
|
+
if (quota && typeof quota === 'object') {
|
|
1233
|
+
const entries = Object.entries(quota)
|
|
1234
|
+
.filter(([, s]) => s && typeof s.percent === 'number')
|
|
1235
|
+
.sort((a, b) => a[1].percent - b[1].percent) // 📖 most depleted first
|
|
1236
|
+
.slice(0, 5)
|
|
1237
|
+
if (entries.length > 0) {
|
|
1238
|
+
const parts = entries.map(([providerKey, snap]) => {
|
|
1239
|
+
const icon = snap.percent <= 10 ? '🚨' : snap.percent <= 25 ? '⚠️ ' : '📊'
|
|
1240
|
+
return `${icon} ${providerKey} ${snap.percent}%`
|
|
1241
|
+
})
|
|
1242
|
+
quotaLabel = chalk.bgRgb(60, 100, 60).rgb(220, 255, 220).bold(` ${parts.join(' · ')} `)
|
|
1243
|
+
}
|
|
1244
|
+
}
|
|
1245
|
+
|
|
1246
|
+
// 📖 Line 3: Speed Test + Global Benchmark + Probe + Probe-cache + Quota + Last release
|
|
1247
|
+
if (releaseLabel || speedTestLabel || globalBenchmarkLabel || probeLabel || probeCacheLabel || quotaLabel) {
|
|
1224
1248
|
const parts = [
|
|
1225
1249
|
{ text: ' ', key: null },
|
|
1226
1250
|
{ text: speedTestLabel, key: 'a' },
|
|
@@ -1230,6 +1254,8 @@ export function renderTable({
|
|
|
1230
1254
|
{ text: probeLabel, key: null },
|
|
1231
1255
|
{ text: probeCacheLabel ? ' ' : '', key: null },
|
|
1232
1256
|
{ text: probeCacheLabel, key: null },
|
|
1257
|
+
{ text: quotaLabel ? ' ' : '', key: null },
|
|
1258
|
+
{ text: quotaLabel, key: null },
|
|
1233
1259
|
{ text: ' ', key: null },
|
|
1234
1260
|
{ text: releaseLabel, key: null },
|
|
1235
1261
|
]
|