@convex-dev/ai-budget 0.0.2-alpha.17 → 0.0.2-alpha.19

package/README.md

@@ -251,7 +251,10 @@ await ai.meter(ctx,
   });
 ```
 
-### `ai.decisions` — structured decisions (Jev)
+### `ai.decisions` — structured decisions (Jev) · **experimental**
+
+> **Experimental** — the Decisions endpoint is alpha on the gateway; this API may
+> change without a major version bump.
 
 Budget the gateway's Decisions endpoint ([Jev](https://docs.typesafe.ai)) — typed
 `choice` / `score` / `boolean` questions evaluated against a `state` — with the
@@ -280,7 +283,10 @@ defaults to `defaultEvalModel` (`"typesafe/jev-1.13"`). Requires
 `experimental_evaluate` — both imported lazily, so callers who don't use
 `decisions` are unaffected.
 
-### `ai.begin` / `ai.settle` — long async jobs (video)
+### `ai.begin` / `ai.settle` — long async jobs (video) · **experimental**
+
+> **Experimental** — video generation depends on alpha gateway features; the
+> async surface (`reserveTtlMs`, `registerWebhook`) may change without a major bump.
 
 A video job is submit → wait minutes → poll/webhook → done, spanning multiple
 Convex functions, so the synchronous `meter` bracket doesn't fit. Reserve with
@@ -432,8 +438,11 @@ await ai.users.adjust(ctx, { userId, deltaNanos: -5 * 1_000_000_000, reason: "go
 await ai.users.adjustments(ctx, { userId });   // the audit log
 ```
 
-Negative = credit, positive = extra charge; it adjusts the live day/month/lifetime
-windows, the history, and an audit log.
+Positive = an extra charge (adds to **gross** spend). Negative = a credit/refund,
+which accrues in a **separate** credits balance and never reduces gross spend — so
+gross spend and durable history stay consistent. Caps enforce on **net = gross −
+credits**, so a credit gives the bucket real headroom. Both are written to the
+audit log.
 
 ### Alerts
 
@@ -490,9 +499,12 @@ ai.requests.get(ctx, { requestId })                               // one request
 ### Global cap
 
 ```ts
+// enforcement: "approximate" (default, best-effort block) | "soft" (warn only).
+// Pass null to clear a field; only the fields you pass change.
 ai.global.setLimits(ctx, { dailySpendLimitNanos?, lifetimeSpendLimitNanos?, enforcement? })
-ai.global.status(ctx)   // { limits, spentTodayNanos, spentTotalNanos, … }
+ai.global.status(ctx)   // { dailySpendLimitNanos, lifetimeSpendLimitNanos, enforcement, spentTodayNanos, spentTotalNanos, … }
 ai.global.bump(ctx, { dailyNanos?, lifetimeNanos? })
+ai.global.setPolicy(ctx, { allowUnpricedModels?, storeContent? })  // deployment data/pricing policy
 ```
 
 A best-effort killswitch across everything. Backed by a sharded counter for
@@ -584,19 +596,29 @@ import { AIBudget } from "@convex-dev/ai-budget";
 const ai = new AIBudget(components.aiBudget);
 const http = httpRouter();
 
-ai.registerRoutes(http, {
-  // Gate it — the endpoint is public. Recommended: check the caller is an admin.
-  authorize: async (ctx) => (await ctx.auth.getUserIdentity())?.role === "admin",
-});
+// Recommended: token mode. Set AI_BUDGET_DASHBOARD_TOKEN in the deployment env;
+// open the dashboard with ?token=<it> once and the page keeps using it.
+ai.registerRoutes(http);
 
 export default http;
 ```
 
 It lives at `https://<deployment>.convex.site/aibudget` (override with `path`).
-**It is a public internet endpoint, so you must gate it**: pass `authorize`
-(return `true` to allow) or set `AI_BUDGET_DASHBOARD_TOKEN` (sent as
-`Authorization: Bearer …`). With neither, every route returns 401. Everything the
-page shows is backed by the component's own functions — nothing else to wire up.
+**It is a public internet endpoint, so you must gate it**, one of two ways:
+
+- **Token (recommended, works end-to-end):** set the `AI_BUDGET_DASHBOARD_TOKEN`
+  env var. Open `…/aibudget?token=<token>` once; the page strips it from the URL
+  and sends it as a bearer on every API call. `?token=` is accepted only on the
+  page navigation, and the compare is constant-time.
+- **`authorize(ctx, request)` callback:** must authenticate from something the
+  **browser sends on a top-level navigation** — a cookie or a header *you*
+  control — because a page load carries no `Authorization` bearer. `ctx.auth`
+  (deployment JWT) is `null` for the HTML page, so `authorize: (ctx) => (await
+  ctx.auth.getUserIdentity())?.role === "admin"` will 401 the page. Use it only
+  when you have your own session cookie to check.
+
+With neither configured, every route returns 401. Everything the page shows is
+backed by the component's own functions — nothing else to wire up.
 
 ---
 
@@ -641,6 +663,19 @@ more tokens or costs more than estimated, settlement records the real amount and
 the final total can exceed a hard cap by that request's estimation delta. The next
 admission sees the settled total and blocks until there is headroom again.
 
+**Throughput characteristics (know these before you turn on a global cap).**
+Reconciliation runs as small, independently-rescheduling phases, so folding,
+expiry, and retention can't stall each other and each drains its own backlog.
+Two shared-write hot spots remain, by design:
+- The **global cap** reads a sharded counter *inside* admission, which contends
+  with every fold that writes it — so a configured global cap adds contention on
+  the hot path. It's a deployment-wide killswitch, not a high-throughput per-call
+  limit; prefer per-bucket caps for the common case.
+- **Settlement writes every attributed bucket**, including the per-request
+  `action` bucket that every call shares. That single row is written by every
+  settle, so an extremely high single-action settle rate can lag totals. Spread
+  load across naturally-sharded dimensions (per user/customer) where you can.
+
 The `error.md` file documents the adversarial audits this design survived, with
 live repros.
 
@@ -732,6 +767,31 @@ metadata use their creation period and current capped buckets as a compatibility
 fallback. Exact historical hold ownership cannot be reconstructed if those caps
 changed before the upgrade. New requests always store explicit ownership.
 
+## Stability (v1)
+
+**Stable (frozen for 1.0)** — these keep backward compatibility within the 1.x line:
+`ai.chat`, `ai.meter`, `ai.begin`/`ai.settle`, `ai.languageModel`, and the admin
+namespaces `ai.users` / `ai.actions` / `ai.tag` / `ai.global` / `ai.models` /
+`ai.prices` / `ai.requests`, plus `ai.registerRoutes`. The stored schema is frozen;
+new fields will only be added as optional.
+
+**Experimental (may change without a major bump):** `ai.decisions` (the Jev /
+Decisions endpoint) and video generation (`begin`/`settle` with `reserveTtlMs` +
+`registerWebhook`). Both depend on gateway features that are still alpha.
+
+**Semantics worth knowing:**
+- **Spend caps** admit on an *estimate*, so a token-priced cap can be exceeded by
+  one request's estimate-vs-actual delta; pass `estimatedCostNanos` for an exact
+  reservation. The **global cap** is a best-effort killswitch (`enforcement:
+  "approximate"` | `"soft"`), not a to-the-dollar ceiling.
+- **Credits** (`ai.tag(d).adjust` / negative `deltaNanos`) accrue in a separate
+  balance and never reduce gross spend; caps enforce on **net = gross − credits**,
+  so a credit grants real headroom while spend history stays consistent.
+- **Deployment policy** (`ai.global.setPolicy`): `allowUnpricedModels: false`
+  rejects models with no configured price under hard enforcement (default charges
+  the conservative fallback); `storeContent: false` persists metadata but no
+  prompt/response content (for teams that want zero prompt retention).
+
 ## Development
 
 ```sh

package/dist/client/index.d.ts

@@ -495,6 +495,9 @@ export declare class AIBudget {
             lifetimeBumpNanos?: number | undefined;
             bumpDayStamp?: string | undefined;
             bumpMonthStamp?: string | undefined;
+            creditsNanos?: number | undefined;
+            creditsTodayNanos?: number | undefined;
+            creditsThisMonthNanos?: number | undefined;
             tokensToday?: number | undefined;
             monthStamp?: string | undefined;
             tokensThisMonth?: number | undefined;
@@ -536,6 +539,9 @@ export declare class AIBudget {
             lifetimeBumpNanos?: number | undefined;
             bumpDayStamp?: string | undefined;
             bumpMonthStamp?: string | undefined;
+            creditsNanos?: number | undefined;
+            creditsTodayNanos?: number | undefined;
+            creditsThisMonthNanos?: number | undefined;
             tokensToday?: number | undefined;
             monthStamp?: string | undefined;
             tokensThisMonth?: number | undefined;
@@ -632,6 +638,9 @@ export declare class AIBudget {
             lifetimeBumpNanos?: number | undefined;
             bumpDayStamp?: string | undefined;
             bumpMonthStamp?: string | undefined;
+            creditsNanos?: number | undefined;
+            creditsTodayNanos?: number | undefined;
+            creditsThisMonthNanos?: number | undefined;
             tokensToday?: number | undefined;
             monthStamp?: string | undefined;
             tokensThisMonth?: number | undefined;
@@ -673,6 +682,9 @@ export declare class AIBudget {
             lifetimeBumpNanos?: number | undefined;
             bumpDayStamp?: string | undefined;
             bumpMonthStamp?: string | undefined;
+            creditsNanos?: number | undefined;
+            creditsTodayNanos?: number | undefined;
+            creditsThisMonthNanos?: number | undefined;
             tokensToday?: number | undefined;
             monthStamp?: string | undefined;
             tokensThisMonth?: number | undefined;
@@ -768,6 +780,9 @@ export declare class AIBudget {
             lifetimeBumpNanos?: number | undefined;
             bumpDayStamp?: string | undefined;
             bumpMonthStamp?: string | undefined;
+            creditsNanos?: number | undefined;
+            creditsTodayNanos?: number | undefined;
+            creditsThisMonthNanos?: number | undefined;
             tokensToday?: number | undefined;
             monthStamp?: string | undefined;
             tokensThisMonth?: number | undefined;
@@ -809,6 +824,9 @@ export declare class AIBudget {
             lifetimeBumpNanos?: number | undefined;
             bumpDayStamp?: string | undefined;
             bumpMonthStamp?: string | undefined;
+            creditsNanos?: number | undefined;
+            creditsTodayNanos?: number | undefined;
+            creditsThisMonthNanos?: number | undefined;
             tokensToday?: number | undefined;
             monthStamp?: string | undefined;
             tokensThisMonth?: number | undefined;
@@ -886,17 +904,21 @@ export declare class AIBudget {
         status: (ctx: RunQueryCtx) => Promise<{
             dailySpendLimitNanos: number | null;
             lifetimeSpendLimitNanos: number | null;
-            enforcement: "hard" | "soft";
+            enforcement: "soft" | "approximate";
             spentTodayNanos: number;
             spentTotalNanos: number;
             retentionMs: number | null;
             defaultWarnAtPct: number | null;
         }>;
-        /** A killswitch spend cap across all users/actions (enforced approximately). */
+        /**
+         * A killswitch spend cap across all users/actions. Enforced
+         * **approximately** (bounded overshoot, no per-request reservation) — for
+         * an exact ceiling use a per-bucket cap. Pass `null` to clear a field.
+         */
         setLimits: (ctx: RunMutationCtx, args: {
-            dailySpendLimitNanos?: number;
-            lifetimeSpendLimitNanos?: number;
-            enforcement?: "hard" | "soft";
+            dailySpendLimitNanos?: number | null;
+            lifetimeSpendLimitNanos?: number | null;
+            enforcement?: "approximate" | "soft" | null;
         }) => Promise<null>;
         bump: (ctx: RunMutationCtx, args: {
             dailyNanos?: number;
@@ -910,6 +932,17 @@ export declare class AIBudget {
         setRetention: (ctx: RunMutationCtx, args: {
             retentionMs: number;
         }) => Promise<null>;
+        /**
+         * Deployment-wide data/pricing policy (only the fields you pass change):
+         * - `allowUnpricedModels: false` rejects models with no configured price
+         *   under hard enforcement (default true — charge the conservative fallback).
+         * - `storeContent: false` stops persisting prompt/response content on
+         *   request rows (default true).
+         */
+        setPolicy: (ctx: RunMutationCtx, args: {
+            allowUnpricedModels?: boolean;
+            storeContent?: boolean;
+        }) => Promise<null>;
     };
     /** Model allow/deny policy. */
     get models(): {

package/dist/client/index.js

@@ -39,10 +39,17 @@ function toTokenCount(x) {
 }
 function extractUsage(usage) {
     return {
-        // Cover AI SDK camelCase (v5/v7) AND raw OpenAI-compatible snake_case, so a
-        // `meter` caller passing a raw provider `usage` object still gets counts.
-        promptTokens: toTokenCount(usage?.inputTokens ?? usage?.promptTokens ?? usage?.prompt_tokens),
-        completionTokens: toTokenCount(usage?.outputTokens ?? usage?.completionTokens ?? usage?.completion_tokens),
+        // Cover AI SDK camelCase (v5/v7), raw OpenAI-compatible snake_case, AND raw
+        // Anthropic (`input_tokens`/`output_tokens`), so a `meter` caller passing any
+        // provider's raw `usage` object still gets counts.
+        promptTokens: toTokenCount(usage?.inputTokens ??
+            usage?.promptTokens ??
+            usage?.prompt_tokens ??
+            usage?.input_tokens),
+        completionTokens: toTokenCount(usage?.outputTokens ??
+            usage?.completionTokens ??
+            usage?.completion_tokens ??
+            usage?.output_tokens),
         // cached prompt tokens. The Convex gateway reports these at
         // `usage.inputTokenDetails.cacheReadTokens`; the other paths cover AI SDK v5
         // (`cachedInputTokens`) and raw OpenAI-compatible shapes.
@@ -50,7 +57,8 @@ function extractUsage(usage) {
             usage?.cachedInputTokens ??
             usage?.promptTokensDetails?.cachedTokens ??
             usage?.prompt_tokens_details?.cached_tokens ??
-            usage?.cached_tokens),
+            usage?.cached_tokens ??
+            usage?.cache_read_input_tokens),
     };
 }
 // The AI Gateway reports the authoritative dollar cost of each request.
@@ -72,6 +80,10 @@ function extractGatewayCostNanos(result) {
     return undefined;
 }
 const NANOS_PER_DOLLAR = 1e9;
+// Cap stored prompt/response content so one row can't approach Convex's 1 MiB
+// document limit (which would fail the call) or bloat the reconciler's scans.
+const MAX_STORED_CONTENT = 32 * 1024;
+const capContent = (s) => s.length > MAX_STORED_CONTENT ? s.slice(0, MAX_STORED_CONTENT) + "…[truncated]" : s;
 // Flatten an AI SDK prompt (roles + content parts) into simple storable messages.
 function simplifyPrompt(prompt) {
     if (!Array.isArray(prompt))
@@ -83,13 +95,22 @@ function simplifyPrompt(prompt) {
         }
         else if (Array.isArray(m.content)) {
             content = m.content
-                .map((part) => part?.type === "text" ? part.text : JSON.stringify(part))
+                .map((part) => {
+                if (part?.type === "text")
+                    return part.text ?? "";
+                // NEVER inline a base64 image/file part — a data URL or Uint8Array here
+                // becomes megabytes, pushing the stored row past the 1 MiB doc limit
+                // (failing the call) and turning a 200 KB image into a ~50k-token
+                // estimate. Store a compact placeholder.
+                const bytes = part?.data?.length ?? part?.image?.length ?? part?.data?.byteLength;
+                return `[${part?.type ?? "part"}${typeof bytes === "number" ? ` ${bytes}b` : ""}]`;
+            })
                 .join("");
         }
         else {
             content = JSON.stringify(m.content);
         }
-        return { role: String(m.role), content };
+        return { role: String(m.role), content: capContent(content) };
     });
 }
 function extractText(result) {
@@ -491,6 +512,21 @@ export class AIBudget {
                             async flush() {
                                 await settle();
                             },
+                            // A cancelled/aborted stream (client disconnect, AbortSignal, or
+                            // breaking out of `for await`) does NOT run `flush`. Without this
+                            // the request would sit pending until the 30-min reservation
+                            // expiry and be recorded as $0 though the provider charged for
+                            // what streamed. Settle here with the text so far, estimating the
+                            // completion tokens when the provider gave no usage.
+                            // `cancel` is a newer Streams-spec transformer hook not yet in
+                            // the TS DOM lib types; cast the literal so it compiles (runtimes
+                            // without it simply won't call it, and the reconciler backstops).
+                            async cancel(reason) {
+                                if (usage === undefined && text) {
+                                    usage = { outputTokens: Math.ceil(text.length / 4) };
+                                }
+                                await settle(`cancelled: ${String(reason)}`);
+                            },
                         }));
                         return { ...result, stream: tapped };
                     }
@@ -612,13 +648,25 @@ export class AIBudget {
         return {
             /** Limits + spend today/total. */
             status: (ctx) => ctx.runQuery(c.lib.getGlobalStatus, {}),
-            /** A killswitch spend cap across all users/actions (enforced approximately). */
+            /**
+             * A killswitch spend cap across all users/actions. Enforced
+             * **approximately** (bounded overshoot, no per-request reservation) — for
+             * an exact ceiling use a per-bucket cap. Pass `null` to clear a field.
+             */
             setLimits: (ctx, args) => ctx.runMutation(c.lib.setGlobalLimits, args),
             bump: (ctx, args) => ctx.runMutation(c.lib.bumpGlobal, args),
             /** Default approaching-limit alert threshold (fraction of a cap, e.g. 0.8). */
             setAlertDefaults: (ctx, args) => ctx.runMutation(c.lib.setAlertDefaults, args),
             /** Request-row retention window in ms (default 1h; 0 disables). */
             setRetention: (ctx, args) => ctx.runMutation(c.lib.setRetention, args),
+            /**
+             * Deployment-wide data/pricing policy (only the fields you pass change):
+             * - `allowUnpricedModels: false` rejects models with no configured price
+             *   under hard enforcement (default true — charge the conservative fallback).
+             * - `storeContent: false` stops persisting prompt/response content on
+             *   request rows (default true).
+             */
+            setPolicy: (ctx, args) => ctx.runMutation(c.lib.setDeploymentPolicy, args),
         };
     }
     /** Model allow/deny policy. */

package/dist/component/_generated/component.d.ts

@@ -66,7 +66,7 @@ export type ComponentApi<Name extends string | undefined = string | undefined> =
         getGlobalStatus: FunctionReference<"query", "internal", {}, {
             dailySpendLimitNanos: number | null;
             defaultWarnAtPct: number | null;
-            enforcement: "hard" | "soft";
+            enforcement: "approximate" | "soft";
             lifetimeSpendLimitNanos: number | null;
             retentionMs: number | null;
             spentTodayNanos: number;
@@ -116,10 +116,14 @@ export type ComponentApi<Name extends string | undefined = string | undefined> =
             value: string;
             warnAtPct?: number;
         }, null, Name>;
+        setDeploymentPolicy: FunctionReference<"mutation", "internal", {
+            allowUnpricedModels?: boolean;
+            storeContent?: boolean;
+        }, null, Name>;
         setGlobalLimits: FunctionReference<"mutation", "internal", {
-            dailySpendLimitNanos?: number;
-            enforcement?: "hard" | "soft";
-            lifetimeSpendLimitNanos?: number;
+            dailySpendLimitNanos?: number | null;
+            enforcement?: "approximate" | "soft" | null;
+            lifetimeSpendLimitNanos?: number | null;
         }, null, Name>;
         setModelPolicy: FunctionReference<"mutation", "internal", {
             mode: "open" | "allowlist" | "denylist";

package/dist/component/lib.d.ts

@@ -39,9 +39,15 @@ export declare const finishRequest: import("convex/server").RegisteredMutation<"
 export declare const foldTotals: import("convex/server").RegisteredMutation<"internal", {
     requestId: import("convex/values").GenericId<"requests">;
 }, Promise<null>>;
-export declare const reconcile: import("convex/server").RegisteredMutation<"internal", {}, Promise<{
+export declare const reconcile: import("convex/server").RegisteredMutation<"internal", {}, Promise<null>>;
+export declare const globalPhase: import("convex/server").RegisteredMutation<"internal", {}, Promise<null>>;
+export declare const foldPhase: import("convex/server").RegisteredMutation<"internal", {}, Promise<{
     folded: number;
+}>>;
+export declare const expirePhase: import("convex/server").RegisteredMutation<"internal", {}, Promise<{
     expired: number;
+}>>;
+export declare const retentionPhase: import("convex/server").RegisteredMutation<"internal", {}, Promise<{
     purged: number;
 }>>;
 export declare const setRetention: import("convex/server").RegisteredMutation<"public", {
@@ -237,6 +243,9 @@ export declare const listBuckets: import("convex/server").RegisteredQuery<"publi
     lifetimeBumpNanos?: number | undefined;
     bumpDayStamp?: string | undefined;
     bumpMonthStamp?: string | undefined;
+    creditsNanos?: number | undefined;
+    creditsTodayNanos?: number | undefined;
+    creditsThisMonthNanos?: number | undefined;
     tokensToday?: number | undefined;
     monthStamp?: string | undefined;
     tokensThisMonth?: number | undefined;
@@ -278,6 +287,9 @@ export declare const getBucket: import("convex/server").RegisteredQuery<"public"
     lifetimeBumpNanos?: number | undefined;
     bumpDayStamp?: string | undefined;
     bumpMonthStamp?: string | undefined;
+    creditsNanos?: number | undefined;
+    creditsTodayNanos?: number | undefined;
+    creditsThisMonthNanos?: number | undefined;
     tokensToday?: number | undefined;
     monthStamp?: string | undefined;
     tokensThisMonth?: number | undefined;
@@ -356,6 +368,10 @@ export declare const usageHistory: import("convex/server").RegisteredQuery<"publ
 export declare const setAlertDefaults: import("convex/server").RegisteredMutation<"public", {
     warnAtPct?: number | undefined;
 }, Promise<null>>;
+export declare const setDeploymentPolicy: import("convex/server").RegisteredMutation<"public", {
+    allowUnpricedModels?: boolean | undefined;
+    storeContent?: boolean | undefined;
+}, Promise<null>>;
 export declare const deleteBucket: import("convex/server").RegisteredMutation<"public", {
     dimension: string;
     value: string;
@@ -370,7 +386,7 @@ export declare const getModelPolicy: import("convex/server").RegisteredQuery<"pu
 export declare const getGlobalStatus: import("convex/server").RegisteredQuery<"public", {}, Promise<{
     dailySpendLimitNanos: number | null;
     lifetimeSpendLimitNanos: number | null;
-    enforcement: "hard" | "soft";
+    enforcement: "soft" | "approximate";
     spentTodayNanos: number;
     spentTotalNanos: number;
     retentionMs: number | null;
@@ -379,7 +395,7 @@ export declare const getGlobalStatus: import("convex/server").RegisteredQuery<"p
 export declare const setGlobalLimits: import("convex/server").RegisteredMutation<"public", {
     dailySpendLimitNanos?: number | null | undefined;
     lifetimeSpendLimitNanos?: number | null | undefined;
-    enforcement?: "hard" | "soft" | null | undefined;
+    enforcement?: "soft" | "approximate" | null | undefined;
 }, Promise<null>>;
 export declare const bumpGlobal: import("convex/server").RegisteredMutation<"public", {
     dailyNanos?: number | undefined;