omnigateway 0.2.0 → 0.2.2

package/README.md

@@ -3,7 +3,7 @@
 One endpoint in front of the AI accounts you already pay for.
 
 OmniGateway is a self-hosted gateway that speaks the Anthropic and OpenAI APIs
-and answers them using your own Anthropic, OpenAI, and Kimi Coding
+and answers them using your own Anthropic, OpenAI, Kimi Coding, Kilo, and xAI
 subscriptions. Point any compatible client at it, ask for a model you defined,
 and the gateway picks an account that can serve it — falling back to another
 when one is rate-limited, expired, or out of quota.
@@ -29,7 +29,7 @@ omni start
   provider dispatch, preserve errors and non-tool-result content, and default off.
 - **Routes across your accounts.** Define a virtual model like `fast` or
   `smart` with several targets; the gateway ranks them by tier, health,
-  remaining quota, cost, and latency.
+  remaining quota, cost, latency, and current load.
 - **Fails over.** A rate-limited or broken account is skipped, its circuit
   breaker opens, and the next candidate is tried — before the response starts
   streaming.
@@ -45,12 +45,66 @@ omni start
 - **Ships an admin console and a CLI.** Both cover the same ground; use
   whichever suits the machine you are on.
 
+## How it is built
+
+A Bun workspace monorepo. The layering is deliberate and enforced by what each
+package is allowed to import: a provider-neutral core at the bottom, provider
+knowledge in one place, pure routing above that, and every side effect pushed up
+into the gateway process.
+
+```mermaid
+graph TD
+  subgraph frontends[Front ends]
+    gateway["apps/gateway<br/><i>Elysia server, dispatch, loops</i>"]
+    cli["apps/cli<br/><i>the omni binary</i>"]
+    dashboard["apps/dashboard<br/><i>React console</i>"]
+  end
+
+  control["@omni/control<br/><i>every operator action;<br/>no HTTP, argv or terminal</i>"]
+  router["@omni/router<br/><i>pure ranking;<br/>no I/O, no timers</i>"]
+  store["@omni/store<br/><i>SQLite + field encryption</i>"]
+  providers["@omni/providers<br/><i>adapters, wire codecs,<br/>catalog, HTTP client</i>"]
+  rtk["@omni/rtk<br/><i>tool-result filters</i>"]
+  ir["@omni/ir<br/><i>domain model — depends on nothing</i>"]
+
+  gateway --> control
+  gateway --> rtk
+  cli --> control
+  dashboard -. "types + catalog only" .-> providers
+  dashboard -. "types only" .-> store
+  control --> router
+  router --> store
+  router --> providers
+  store --> rtk
+  providers --> ir
+  rtk --> ir
+  store --> ir
+```
+
+Arrows read *depends on*, and the direction never reverses. Two rules do most of
+the work: `@omni/ir` is side-effect free and imports nothing, and `@omni/router`
+is a pure function — no network, no database, no timers. Everything that has to
+touch the world is pushed up into the gateway process.
+
+The dashboard's edges are dotted because they are type-level only.
+`@omni/providers/catalog` is deliberately kept import-free so model lists can be
+bundled into the browser without dragging in the HTTP client.
+
+The gateway and the CLI are two front ends over the same `@omni/control`
+functions. The CLI does not call the running server's API — it opens the same
+SQLite file directly, which is why it still works when the gateway is down.
+
+**[ARCHITECTURE.md](ARCHITECTURE.md)** goes a level deeper: a request traced end
+to end, how routing ranks and excludes accounts, the commit point that decides
+whether failover is still possible, the provider adapter shape, the database
+schema and its encryption boundary, and the background loops.
+
 ## Requirements
 
 - [Bun](https://bun.sh/) 1.4 or later. Bun is the runtime, not just the
   installer, so a Node-only machine cannot run OmniGateway.
 - A directory that persists, for the SQLite database.
-- At least one Anthropic, OpenAI, or Kimi Coding account to connect.
+- At least one Anthropic, OpenAI, Kimi Coding, Kilo, or xAI account to connect.
 
 ## Install
 
@@ -86,9 +140,18 @@ omni start                 # serves the API and the console on 127.0.0.1:9000
 Connect an account. The CLI prints a URL to open, and waits:
 
 ```bash
-omni connect anthropic     # or: openai, kimi
+omni connect anthropic     # or: openai, kimi, kilo, grok
 ```
 
+Every provider also takes a plain API key, if that is what you hold rather than
+a subscription. `custom` takes nothing else:
+
+```bash
+omni credentials add-key anthropic     # prompts for the key, or reads stdin
+```
+
+The console offers the same choice per provider on the Connect dialog.
+
 Define a virtual model your clients will ask for, seeding its pricing and
 capabilities from the built-in catalog:
 
@@ -97,6 +160,23 @@ omni models catalog                                  # what is available
 omni models put fast --from-catalog anthropic:claude-sonnet-5
 ```
 
+**A note on grok pricing.** xAI charges by request size: at or above 200K
+context the rate roughly doubles, and the higher rate applies to *every token
+in the request*, not just the tokens past the mark. A target holds one flat
+price, so the catalog carries xAI's sub-200K figures and long-context traffic
+is reported cheaper than it was billed. Catalog pricing is only the default a
+new target starts from — if you run grok at long context, edit the saved
+target's price to match the tier you are actually paying.
+
+**A note on `kilo-auto/*` pricing.** Kilo's `frontier`, `balanced`, and
+`efficient` routers choose an upstream model per request, and Kilo states no
+rate for them. The catalog records zero, which the router reads as *unpriced*
+and leaves out of its cost ranking — the same stored figure `kilo-auto/free`
+carries because it genuinely is free. So a `kilo-auto` target seeded from the
+catalog is not free, it is unranked: cost never counts for or against it. If you
+want one ranked against your other accounts, set a real `costPerMTok` on the
+saved target for the tier you expect it to land in.
+
 Mint a key for your client. **It is printed once and stored only as a hash:**
 
 ```bash
@@ -170,6 +250,10 @@ carries the header through but does not add one on your behalf.
 command takes `--json` for scripting, and `--root <path>` to manage an
 installation other than the default.
 
+`--root` names the installation, so an `OMNI_DB_PATH` exported in your shell is
+ignored alongside it and said so on stderr; that root's own `.env` still decides.
+Use `--db <path>` to point one command somewhere else.
+
 | | |
 | --- | --- |
 | `omni status` | the gateway, its accounts, and their quota, on one screen |
@@ -178,6 +262,7 @@ installation other than the default.
 | `omni logs` | recent requests as the gateway recorded them |
 | `omni console` | the gateway process's own output: boot, refreshes, quota, errors |
 | `omni usage` | spend and tokens, by provider, model, key, or day |
+| `omni quota` | provider quota per window: use, burn rate, and when it runs out |
 | `omni connect <provider>` | authorize an account from the terminal |
 | `omni credentials …` | list, show, enable, disable, retier, refresh, remove |
 | `omni models …` | list, show, put, remove, `dry-run`, `catalog` |
@@ -295,9 +380,11 @@ Worth knowing before you deploy it:
 
 ## Development
 
-Contributing, or running from a checkout? See [CLAUDE.md](CLAUDE.md) for the
-repository map, architectural boundaries, and conventions, and
-`docs/superpowers/specs/` for the design documents behind each feature.
+Contributing, or running from a checkout? See
+[ARCHITECTURE.md](ARCHITECTURE.md) for how the system fits together,
+[CLAUDE.md](CLAUDE.md) for the repository map, architectural boundaries, and
+conventions, and `docs/superpowers/specs/` for the design documents behind each
+feature.
 
 ```bash
 git clone https://github.com/harismawan/omnigateway.git