saros-proxy 0.11.0 → 1.1.0

package/README.md

@@ -52,44 +52,47 @@ Manage multiple API keys behind a single endpoint — OpenCode-Go **and** Comman
 
 ## Architecture
 
-```
-┌─────────────────────────────────────────────────────────────┐
-│                     Your App / OpenCode                     │
-│                                                             │
-│   ┌──────────┐   ┌──────────┐   ┌──────────┐                │
-│   │ Request 1│   │ Request 2│   │ Request 3│    ...         │
-│   └────┬─────┘   └────┬─────┘   └────┬─────┘                │
-└────────┼──────────────┼──────────────┼──────────────────────┘
-         │              │              │
-         └──────────────┼──────────────┘
-                        │
-                        ▼
-┌─────────────────────────────────────────────────────────────┐
-│                       Saros (:3000)                         │
-│                                                             │
-│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐       │
-│  │ Key Selector │─▶│   Circuit    │─▶│   Request   │       │
-│  │ Round-Robin  │  │   Breaker    │  │   Forwarder  │       │
-│  │ + Failover   │  │ Threshold +  │  │ + Timeout +  │       │
-│  │              │  │   Cooldown   │  │     SSE      │       │
-│  └──────────────┘  └──────────────┘  └──────┬───────┘       │
-└─────────────────────────────────────────────┼───────────────┘
-                                              │
-                                  ┌───────────┼───────────┐
-                                  ▼           ▼           ▼
-                           ┌──────────┐ ┌──────────┐ ┌──────────┐
-                           │  Key A   │ │  Key B   │ │  Key C   │
-                           │ sk-abc…  │ │ sk-def…  │ │ user_…   │
-                           └────┬─────┘ └────┬─────┘ └────┬─────┘
-                                │            │            │
-                                └────────────┼────────────┘
-                                             │
-                       ┌─────────────────────┴─────────────────────┐
-                       ▼                                           ▼
-              ┌───────────────────────┐                  ┌──────────────────────────┐
-              │      OpenCode-Go API  │                  │      CommandCode API     │
-              │    https://opencode.ai│                  │ https://api.commandcode.ai│
-              └───────────────────────┘                  └──────────────────────────┘
+```mermaid
+flowchart TB
+    subgraph client["Your App / OpenCode"]
+        direction LR
+        r1["Request 1"]
+        r2["Request 2"]
+        r3["Request 3"]
+        rest["..."]
+    end
+
+    subgraph saros["Saros :3000"]
+        direction LR
+        ks["Key Selector<br/>round-robin + failover"]
+        cb["Circuit Breaker<br/>threshold + cooldown"]
+        fw["Request Forwarder<br/>timeout + SSE"]
+        ks --> cb --> fw
+    end
+
+    subgraph keys["API keys"]
+        direction LR
+        ka["Key A<br/>sk-abc…"]
+        kb["Key B<br/>sk-def…"]
+        kc["Key C<br/>user_…"]
+    end
+
+    oc["OpenCode-Go API<br/>opencode.ai"]
+    cc["CommandCode API<br/>api.commandcode.ai"]
+
+    r1 --> ks
+    r2 --> ks
+    r3 --> ks
+    rest --> ks
+    fw --> ka
+    fw --> kb
+    fw --> kc
+    ka --> oc
+    kb --> oc
+    kc --> oc
+    ka --> cc
+    kb --> cc
+    kc --> cc
 ```
 
 ### How Request Distribution Works
@@ -98,13 +101,46 @@ Here's how Saros picks which API key to use:
 
 #### Usage-Weighted Rotation
 
-Saros polls the upstream usage API for each key (5h / weekly / monthly windows)
-and weights rotation by **remaining** quota. If key X is 30% used and key Y is
-70% used, 10 requests are distributed 7 → X and 3 → Y — proportional balancing,
-not hard cutoffs, so neither key absorbs the full load when the other runs low.
+Saros polls each key's quota endpoint and weights rotation by **remaining
+monthly** quota. If key X is 30% into its monthly allowance and key Y is 70%,
+10 requests are distributed 7 → X and 3 → Y — proportional balancing, not hard
+cutoffs, so neither key absorbs the full load when the other runs low.
+
+The number Saros compares is always a percentage of that key's **own provider
+monthly budget**, never absolute capacity:
+
+| provider | endpoint | what it reports |
+| --- | --- | --- |
+| OpenCode Go | `GET /zen/go/v1/usage` | `rolling` (5h) / `weekly` / `monthly` as percents |
+| CommandCode | `GET /alpha/billing/credits` | `fiveHour` and `weekly` as `used`/`cap` in credits, `credits.monthlyCredits` (remaining) and reset timestamps |
+
+CommandCode's payload has no explicit monthly cap. The parser **infers** it
+from the window caps (the 20% / 50% / 100% ladder is OpenCode Go's documented
+plan shape; for CommandCode the 14/35 ratio was measured against its
+dashboard): monthly cap = 2 × `weekly.cap`, falling back to 5 ×
+`fiveHour.cap` when the weekly window is absent. The candidate cap is only
+accepted when the credits agree with it — `credits.monthlyCredits <= cap` and
+`cap − monthlyCredits >= max(weekly.used, fiveHour.used)`; otherwise the
+fallback is tried and, if it also disagrees, the response is treated as
+unusable (the previous usage state is kept rather than branding a broken
+derivation as 0% usage). A payload without `monthlyCredits` reports the
+monthly metric as unavailable (`n/a` in `saros-proxy usage`). Both parsers
+emit the same 0-100 percent scale, so a mixed pool compares like with like.
+Two consequences worth knowing:
+
+- **Monthly is the weight; a spent window still excludes the key.** A key with
+  50% of the month left but a 5-hour or weekly window at 100% weighs 0 and is
+  not selected while another key has room — otherwise it would keep receiving
+  traffic and failing upstream. `/health`'s `usagePercent` is the monthly
+  number.
+- **Percent is all there is.** OpenCode Go publishes percents only — no counts,
+  no caps — so "remaining capacity" in tokens or dollars cannot be derived.
 
 - Refreshed in the background every 60s (stale data keeps working).
-- Without usage data, falls back to even round-robin.
+- Keys without usage data take the **median of the other keys' non-exhausted
+  weights**, not plain round-robin. Round-robin is only the fallback when no
+  key has usage data at all (or when every key is exhausted and a request must
+  share one).
 - A key whose account has no plan for its provider (403 `EntitlementError` /
   `upgrade_required` from the usage endpoint) is dropped from rotation without
   tripping the circuit breaker, and returns as soon as a later refresh sees it
@@ -119,9 +155,9 @@ object for non-streaming bodies — is booked against the key that served the
 request. Totals are persisted to `~/.config/saros/token-usage.json` (debounced
 atomic writes) and shown by `saros-proxy usage`.
 
-This is the only consumption signal available for providers without a
-queryable quota API (CommandCode): their keys report no percentages, but
-`saros-proxy usage` shows absolute token totals and request counts per key.
+This is the fallback signal: when a provider has no quota endpoint, or its usage
+call fails, absolute token totals and request counts per key are what
+`saros-proxy usage` can still show.
 
 #### Circuit Breaker
 
@@ -140,11 +176,11 @@ Saros tracks which keys are currently in use. A key handling a streaming request
 
 ### Prompt Install(for the lazy ones)
 
-Don't feel like reading through everything? Copy this prompt and paste it to your AI assistant. It'll read [`LLM_INSTRUCTIONS.md`](LLM_INSTRUCTIONS.md) and guide you through setup — no keys shared in chat.
+Don't feel like reading through everything? Copy this prompt and paste it to your AI assistant. It'll read [`docs/LLM_INSTRUCTIONS.md`](docs/LLM_INSTRUCTIONS.md) and guide you through setup — no keys shared in chat.
 
 ```
 Install Saros Proxy for my opencode. (https://github.com/vitorvilaca3011/saros-proxy)
-Follow the instructions on LLM_INSTRUCTIONS.md.
+Follow the instructions on docs/LLM_INSTRUCTIONS.md.
 ```
 
 ---
@@ -374,7 +410,7 @@ OPENCODE_GO_KEYS="account1:sk-xxx,cc:user_yyy"
 - The `/health` endpoint reports per-provider key counts under `providers`, and a `keys` array with each key's state: `enabled`, `entitled`, `consecutiveFailures`, `disabledAt`, `retryAfterMs`, and `lastError` (`status` + `reason`: `auth`, `quota`, `rate_limit`, `server`). The credential is never part of the response.
 - When no key can serve a request, the `503` body says which keys are out and why — `keys[]` with `reason`/`lastStatus`/`retryAfterMs`, plus a `causes[]` list (revoked key, exhausted quota, rate limit, missing plan, or a `@provider` pin with no usable key). The legacy `error` field (`All API keys are temporarily unavailable`) is still present.
 - OpenCode-Go requires a stable per-conversation session id (`x-opencode-session`, used for routing and prompt caching). Saros forwards a client-supplied value untouched and derives one from the conversation (model + first user turn) when the client sends none — harnesses that only attach the header on some request paths would otherwise get `400 MissingSessionID`.
-- Usage-based weighted rotation applies to providers with a queryable usage API (OpenCode-Go). CommandCode keys participate in rotation without usage data, but their token consumption is tracked per key and shown by `saros-proxy usage` (see [Token Accounting](#token-accounting)).
+- Usage-based weighted rotation uses each provider's queryable usage endpoint (OpenCode-Go `GET /zen/go/v1/usage`, CommandCode `GET /alpha/billing/credits`). Both contribute a monthly percentage of their own budget; token consumption stays tracked per key and shown by `saros-proxy usage` (see [Usage-Weighted Rotation](#usage-weighted-rotation) and [Token Accounting](#token-accounting)).
 
 #### Smoke test (key identification)
 
@@ -445,14 +481,21 @@ into the "no passphrase source" boot error — a locked keyring is not reported
 as an empty store.
 
 **systemd-creds recipe** (keeps the passphrase out of the unit file and the
-environment of the shell):
+environment of the shell). This is a **system** unit recipe — run the encryption
+as root, since the host key lives in `/var/lib/systemd/credential.secret`:
 
 ```bash
-# One-time: encrypt stdin into a host+user-bound credential.
-systemd-creds encrypt --name=saros-passphrase --with-key=host+user \
+# One-time, as root: encrypt stdin into a host-bound credential.
+# (--with-key=auto picks tpm2+host when the TPM is available, host otherwise.
+#  "host+user" is not a valid key type, and --user only exists from v256.)
+systemd-creds encrypt --name=saros-passphrase --with-key=auto \
   /dev/stdin /etc/saros/passphrase.cred
 ```
 
+For the `systemd --user` unit created by `saros-proxy autostart install`, use
+`Environment=`/`EnvironmentFile=` instead: user credentials need
+`systemd-creds --user`, which only exists from systemd v256 (this host runs 255).
+
 ```ini
 # /etc/systemd/system/saros-proxy.service
 [Service]
@@ -555,16 +598,48 @@ saros-proxy probe
 # Show per-key quota usage, token totals and most-used models
 saros-proxy usage
 
-# Install autostart (Windows: VBS script or Registry)
-saros-proxy autostart install [--port <port>] [--method vbs|registry|auto]
+# Install autostart (Linux: systemd --user; Windows: VBS script or Registry)
+saros-proxy autostart install [--port <port>] [--method systemd|vbs|registry|auto]
 
 # Uninstall autostart
-saros-proxy autostart uninstall [--method vbs|registry|auto]
+saros-proxy autostart uninstall [--method systemd|vbs|registry|auto]
 
 # Check autostart status
-saros-proxy autostart status [--method vbs|registry|auto]
+saros-proxy autostart status [--method systemd|vbs|registry|auto]
+```
+
+On Linux the default is a `systemd --user` unit at
+`~/.config/systemd/user/saros-proxy.service` (honouring `$XDG_CONFIG_HOME`), enabled
+with `systemctl --user enable` and restarted with `systemctl --user restart`. On
+Windows the default is AV-aware (`auto`): Registry when third-party antivirus is
+detected, otherwise the Startup-folder VBS.
+
+Manage the unit with systemd, not with the CLI: `saros-proxy stop` exits non-zero
+while the service still answers `/health`, because the CLI does not own the unit's
+process.
+
+```bash
+systemctl --user start|stop|restart|status saros-proxy.service
+journalctl --user -u saros-proxy -f
 ```
 
+The unit runs `serve` in the foreground: without a TTY it cannot prompt for a
+passphrase, and the user manager does not inherit the environment of the shell
+that ran `install`. If `config.yaml` holds `enc:v1:` envelopes, give the unit a
+source through a drop-in (`systemctl --user edit saros-proxy.service`):
+
+```ini
+[Service]
+Environment=SAROS_PASSPHRASE=your-passphrase
+```
+
+`EnvironmentFile=-%h/.config/saros/env` (mode 0600) keeps the value out of the
+unit file. A credential is stronger, but `Environment=` can only carry the
+credential *path* (`%d/<name>`), never its content — see the
+[systemd-creds recipe](#api-key-encryption), which wraps `serve` in
+`sh -c 'SAROS_PASSPHRASE="$(cat "$CREDENTIALS_DIRECTORY/<name>")" exec …'`.
+`systemd-creds` only gained a `--user` mode in v256.
+
 The `start` command automatically syncs model definitions to all enabled harnesses (see [Multi-Harness Integration](#multi-harness-integration-pi-oh-my-pi--dsh)).
 
 ### Model Sync Commands
@@ -716,7 +791,7 @@ Behavior:
 - Sync never creates a harness config: harnesses whose config file is missing are skipped.
 - Only `providers["saros-proxy"].models` is replaced; all other providers, fields, and settings are preserved, and a `<file>.backup` is created before the first write.
 
-The model list written to each harness comes from the live provider catalogs (opencode-go) enriched with models.dev metadata, merged with any CommandCode-only models — falling back to bundled models when offline — so every harness stays current alongside OpenCode.
+The model list written to each harness comes from the live provider catalogs (opencode-go) enriched with models.dev metadata, merged with any CommandCode-only models — falling back to the bundled `assets/models.json` when offline — so every harness stays current alongside OpenCode.
 
 > **Why a static model list?** The harnesses use a static `models` list on purpose. `pi` has no `/v1/models` discovery for custom providers, and `oh-my-pi`'s discovery cannot recover reasoning/context/output metadata for provider model IDs. `dsh` replaces only `providers["saros-proxy"].models` in its patch layer (`cordis.patch.yml`), which dsh rewrites at every boot — a static, explicitly-managed list is the only reliable source. Dynamic discovery would silently break thinking-level (`:max`/`:high`) role configs.
 
@@ -727,9 +802,9 @@ The model list written to each harness comes from the live provider catalogs (op
 Two kinds of upstream quota are visible in the provider documentation but are **not** modelled by Saros, on purpose — the proxy rotates keys and reasons about keys, not about what a single model or plan allows:
 
 - **Model-level free tiers (CommandCode).** `glyph-cluster:free` is capped at 100 requests/day, `beam:free` has no daily cap, and the `laguna-s-2.1-free` tiers cost no credits. Saros counts tokens per key (`token-usage.json`) and per model, but it does not refuse a request because a *model* reached its daily free quota — the upstream returns the error and the key's circuit breaker decides what happens next.
-- **Plan windows (OpenCode Go).** The Go plan meters each model against `5-hour — 20% of the monthly limit`, `weekly — 50%` and `monthly — 100%`. Those windows belong to the subscription: `/zen/go/v1/usage` reports what was consumed, and Saros surfaces it, but the proxy does not predict window exhaustion or re-route on it.
+- **Plan windows (OpenCode Go).** The Go plan meters each model against `5-hour — 20% of the monthly limit`, `weekly — 50%` and `monthly — 100%`. What was consumed is what feeds weighted rotation (see [Usage-Weighted Rotation](#usage-weighted-rotation)), but the proxy does not refuse a request because a model crossed a window: the upstream returns the error and that is what the client sees.
 
-Both are documented here so the behaviour is not mistaken for a bug. Requests through the alternate OpenAI/Anthropic surfaces (`/v1/responses`, `/v1/messages`) and through the decision endpoint (`/v1/systemone`) are forwarded and counted: streamed usage nested under `response.usage` and the `input_tokens`/`output_tokens` shape are booked like any other call.
+Both are documented here so the behaviour is not mistaken for a bug. Requests through the alternate OpenAI/Anthropic surfaces (`/v1/responses`, `/v1/messages`) and through the decision endpoint (`/v1/systemone`) are forwarded and counted: streamed usage nested under `response.usage` (Responses), the counts Anthropic splits between `message_start.message.usage` and `message_delta.usage` (Messages), and the `input_tokens`/`output_tokens` shape are booked like any other call — with one exception: Anthropic **cache** counters (`cache_read_input_tokens`, `cache_creation_input_tokens`) are not counted, so prompt-cached Messages traffic is under-booked on the prompt side.
 
 ---
 
@@ -796,6 +871,7 @@ Disable it with `SAROS_DEBUG_BUFFER=0`: the endpoint answers `404` and no entry
 | `ECONNREFUSED` on startup | Port already in use | Change port or kill the existing process |
 | `504 Gateway Timeout` | Upstream too slow | Increase `requestTimeoutMs` |
 | `All API keys are temporarily unavailable` | All keys disabled by circuit breaker | Wait for cooldown or check upstream status |
+| Upstream `403` with `error code: 1010` | Cloudflare in front of the provider rejected the `User-Agent` the client sent | Saros sets the provider's own identity (`cli`); if you see this, the request bypassed the proxy or a provider lost its `extraUpstreamHeaders` |
 | CORS error | Client origin not allowed | Add it to `allowedOrigins` in config |
 
 ---
@@ -829,6 +905,18 @@ npm run build
 
 Output goes to `dist/`.
 
+### Refresh the bundled model catalog
+
+`assets/models.json` is generated — do not edit it by hand. The generator merges models.dev metadata with the live opencode-go and CommandCode catalogs: a model that left upstream leaves the bundle, a model the upstream serves but models.dev does not list still enters with the live name and context window, and a field already in the file survives when models.dev omits it.
+
+```bash
+npm run models:refresh -- --dry-run      # print the diff, write nothing
+npm run models:refresh                   # rewrite assets/models.json
+npm run models:refresh -- --allow-shrink # accept removing more than half of the models
+```
+
+A failed fetch — or a 2xx with an empty catalog — aborts before the file is read, and a merge that would remove more than half of the current models aborts unless `--allow-shrink` is given. The write goes through a temp file and an atomic rename, so a crash never leaves a truncated catalog.
+
 ### Project structure
 
 ```
@@ -842,7 +930,8 @@ src/
   providers/            — Multi-provider key abstraction (KeyProvider, adapters, identifyKey)
   models-fetcher.ts     — Upstream model list fetching + caching
   models-sync.ts        — Auto-sync models from upstream to opencode config
-  models-catalog.ts     — Single source of truth: loads bundled models.json
+  models-catalog.ts     — Single source of truth: loads assets/models.json
+  models-refresh.ts     — Generates that catalog (npm run models:refresh)
   cli/
     setup.ts            — Interactive setup wizard
     daemon.ts           — Background process management