@convex-dev/ai-budget 0.0.2-alpha.3 → 0.0.2-alpha.5

package/README.md

@@ -25,10 +25,15 @@ full audit log you can replay later.
 |---|---|
 | **Usage & cost tracking** | Every request stored with messages, response, tokens, latency, and per-request cost. |
 | **Attribution** | Each call is attributed to a `userId` **and** to the Convex action that made it — auto-detected via `ctx.meta`, no manual tagging. Running totals per user and per action. |
-| **Spend & token limits** | Per-user daily / lifetime **spend** and **token** budgets, plus a requests-per-minute rate limit and a block switch. |
+| **Tagged budgets** | `user` and `action` are just built-in *dimensions* — add your own (team, project, customer, env, feature…) by passing `tags`, and cap any of them with `ai.tag("customer").setLimits(...)`. One request can be billed to several buckets at once. |
+| **Spend & token limits** | Per-bucket **daily / monthly / lifetime** spend and token budgets, plus a requests-per-minute rate limit, a max-concurrent cap, and a block switch. |
+| **Spend history** | Durable per-bucket **daily & monthly** rollups that survive request retention — real spend-over-time, "what did we spend last month," per user / action / tag. |
+| **Approaching-limit alerts** | Set `warnAtPct` (e.g. 0.8) and get an `onThreshold` callback before a cap is hit; `onLimitReached` fires when one blocks. |
+| **Manual credits/debits** | Comp a user or correct an overcharge with a signed adjustment; the change hits the live windows, the history, and an audit log. |
+| **Cache-aware cost** | Cached (prompt-cache-read) tokens are billed at a discount using the gateway's real cached-token count — not the full input rate. |
 | **Concurrency-safe caps** | A reserve-then-settle design makes admission a true atomic check — concurrent in-flight requests can't blow past the cap (a naive implementation overshoots ~40×). |
 | **Hard or soft** | Each limit either **blocks** (`hard`) or **allows-with-a-warning** (`soft`). |
-| **Per-feature budgets** | Cap or disable a whole action (e.g. `summarize`) independently of any user. |
+| **Per-feature budgets** | Cap or block a whole action (e.g. `summarize`) independently of any user. |
 | **Global killswitch** | A deployment-wide spend cap across all users and actions (sharded for throughput; enforced approximately). |
 | **One-time bumps** | "Approve another $X" at any level (user / action / global) without changing the standing cap — daily bumps are today-only, lifetime bumps permanent. |
 | **Model policy** | Allow/deny lists for models; an unknown/unpriced model **fails closed** (charged a conservative max, never $0). |
@@ -136,6 +141,7 @@ ai.chat(ctx, {
   messages?,         // [{ role, content }]
   model?,            // defaults to defaultModel
   action?,           // attribution name; defaults to the calling Convex action
+  tags?,             // extra dimensions: [{ dimension: "customer", value: "acme" }, …]
 }): Promise<{ text, requestId, costNanos, promptTokens, completionTokens, warnings }>
 ```
 
@@ -184,16 +190,21 @@ to the original. `lineage` walks the re-run chain in both directions.
 ai.users.setLimits(ctx, {
   userId,
   requestsPerMinute?,
+  maxConcurrent?,            // max in-flight requests at once
   dailySpendLimitNanos?,
+  monthlySpendLimitNanos?,   // calendar-month budget (UTC)
   lifetimeSpendLimitNanos?,
   dailyTokenLimit?,
+  monthlyTokenLimit?,
   lifetimeTokenLimit?,
-  enforcement?,       // "hard" (block, default) | "soft" (warn but allow)
-  blocked?,           // hard block on/off
+  warnAtPct?,                // e.g. 0.8 → fire onThreshold at 80% of a cap
+  enforcement?,             // "hard" (block, default) | "soft" (warn but allow)
+  blocked?,                 // hard block on/off
 })
 ai.users.delete(ctx, { userId })   // remove a user and all their request rows
 ```
 
+The same limit fields apply to `ai.actions.setLimits` and `ai.tag(d).setLimits`.
 Pass a field as `undefined` to clear that limit (unlimited).
 
 ### Per-action budgets
@@ -208,10 +219,84 @@ ai.actions.setLimits(ctx, {
   dailyTokenLimit?,
   lifetimeTokenLimit?,
   enforcement?,                  // "hard" | "soft"
-  disabled?,                     // kill switch for the whole feature
+  blocked?,                      // kill switch for the whole feature
 })
 ```
 
+### Tagged budgets (custom dimensions)
+
+`user` and `action` are the two built-in *dimensions*. To classify or budget
+along any other axis — team, project, tenant, customer, environment, feature —
+attach `tags` to a call and cap a value with `ai.tag(dimension)`:
+
+```ts
+// Bill this call to a user, an action (implicit), AND a customer + env.
+await ai.chat(ctx, {
+  prompt,
+  tags: [
+    { dimension: "customer", value: "acme" },
+    { dimension: "env", value: "prod" },
+  ],
+});
+
+// Cap the customer "acme" to $50/day — independent of any per-user cap.
+await ai.tag("customer").setLimits(ctx, {
+  value: "acme",
+  dailySpendLimitNanos: 50 * 1_000_000_000,
+});
+await ai.tag("customer").bump(ctx, { value: "acme", dailyNanos: 10 * 1_000_000_000 });
+await ai.tag("customer").list(ctx);                 // every customer's spend & caps
+await ai.tag("customer").get(ctx, { value: "acme" }); // one bucket
+```
+
+A single request is admitted only if it fits **every** bucket it touches (user,
+action, and each tag) — the same exact reserve-then-settle check runs per bucket.
+Uncapped buckets never serialize, so adding tags you don't cap is free at
+admission; their running totals still accrue for reporting. `ai.users.*` and
+`ai.actions.*` are simply sugar over `ai.tag("user")` / `ai.tag("action")`.
+
+Every dimension namespace (`users`, `actions`, `tag(d)`) shares the same methods:
+`list`, `get`, `setLimits`, `bump`, `adjust`, `history`, `adjustments`, `delete`.
+
+### Spend history (survives retention)
+
+Request rows are retained only briefly (see retention), but **durable per-bucket
+day/month rollups are not** — so charts and "what did we spend last month" keep
+working:
+
+```ts
+await ai.users.history(ctx, { userId, period: "month" });  // [{ stamp, spendNanos, tokens, requests }]
+await ai.tag("customer").history(ctx, { value: "acme", period: "day", limit: 30 });
+```
+
+### Approaching-limit alerts
+
+Set a threshold (per bucket via `warnAtPct`, or a deployment default) and get a
+callback before a cap is hit — plus one when a hard cap blocks:
+
+```ts
+await ai.global.setAlertDefaults(ctx, { warnAtPct: 0.8 });   // 80%, all buckets
+await ai.users.setLimits(ctx, { userId, warnAtPct: 0.9 });   // override per bucket
+
+new AIBudget(components.aiBudget, {
+  onThreshold:    ({ userId, messages }) => notify(userId, messages),  // approaching
+  onLimitReached: ({ userId, reason })   => notify(userId, reason),    // blocked
+  onSoftLimit:    ({ userId, messages }) => notify(userId, messages),  // soft-cap exceeded
+});
+```
+
+`chat()` also returns `notices` (approaching) alongside `warnings` (soft-exceeded).
+
+### Manual credits & debits
+
+Comp a user or correct an overcharge. Negative = credit, positive = extra charge;
+it adjusts the live day/month/lifetime windows, the history, and an audit log:
+
+```ts
+await ai.users.adjust(ctx, { userId, deltaNanos: -5 * 1_000_000_000, reason: "goodwill" });
+await ai.users.adjustments(ctx, { userId });   // the audit log
+```
+
 ### Global (deployment-wide) budget
 
 ```ts
@@ -249,10 +334,18 @@ common models (validated against OpenRouter's public pricing); override or add
 any model:
 
 ```ts
-ai.prices.set(ctx, { model, inputNanosPerMTok, outputNanosPerMTok })  // must be ≥ 0
+ai.prices.set(ctx, { model, inputNanosPerMTok, outputNanosPerMTok, cachedNanosPerMTok? })  // ≥ 0
 ai.prices.list(ctx)
 ```
 
+**Cached tokens & actual cost.** The gateway reports a real cached-token count
+(`usage.inputTokenDetails.cacheReadTokens`), so the cached slice of a prompt is
+billed at `cachedNanosPerMTok` (or a default 10%-of-input discount) instead of
+the full input rate. The gateway does **not** currently return a dollar cost, so
+cost is computed from tokens — but `finishRequest` accepts an authoritative
+`costNanos` and prefers it whenever present, so adopting a real gateway cost is a
+one-line change the day it's available.
+
 **Real prices from an API.** The gateway's `provider/model` ids match
 OpenRouter's, whose public models endpoint returns per-token pricing — so you can
 keep prices current from your own action (this is app code; the component just
@@ -277,9 +370,12 @@ flagged `unpricedModel: true` so you know to add a real price.
 ### Observability
 
 ```ts
-ai.users.list(ctx)                      // per-user spend today / total / tokens / limits
+ai.users.list(ctx)                      // per-user spend today / month / total / limits
 ai.actions.list(ctx)                    // per-action spend & totals
-ai.requests.list(ctx, { userId?, limit? })  // the audit log (blocked attempts included)
+ai.tag("customer").list(ctx)            // spend & caps for any custom dimension
+ai.users.history(ctx, { userId, period: "month" })   // durable spend-over-time
+ai.requests.list(ctx, { userId?, limit? })           // the audit log (blocked included)
+ai.requests.list(ctx, { dimension: "customer", value: "acme" })  // filter the log by any tag
 ```
 
 ---
@@ -305,15 +401,18 @@ same pre-spend total and all pass, so spend blows past the cap (measured at
    **exactly-once** (a terminal request is never re-folded), so a slow request
    that the reconciler already swept can't double-count when it finally returns.
 
-Reservations are only taken on entities that actually have a cap, so uncapped
-traffic never serializes. The `error.md` file documents the adversarial audits
-this design survived, with live repros.
-
-**One guarantee, all scopes.** Per-user, per-action, and global caps run through
-the *same* admission check — a request is admitted only when
-`committed + reserved + estimate ≤ cap` (bumps included). The only thing that
-differs is the holder: per-user and per-action reserve on a single document, an
-exact atomic check-and-reserve; the **global** killswitch is backed by a sharded
+Reservations are only taken on buckets that actually have a cap, so uncapped
+traffic never serializes — this is what makes arbitrary `tags` cheap: a request
+reserves on one row per *capped* dimension it carries, and nothing else. The
+`error.md` file documents the adversarial audits this design survived, with live
+repros.
+
+**One guarantee, all scopes.** Every per-bucket cap — user, action, or any
+custom tag dimension — and the global cap run through the *same* admission check:
+a request is admitted only when `committed + reserved + estimate ≤ cap` (bumps
+included) for **every** bucket it touches. The only thing that differs is the
+holder: each per-bucket cap reserves on a single document, an exact atomic
+check-and-reserve; the **global** killswitch is backed by a sharded
 counter for throughput, so its committed total is read as an eventually-consistent
 sum with no cross-request reservation. That makes the global cap **approximate** —
 it can overshoot by a bounded amount under a burst — the deliberate