@convex-dev/ai-budget 0.0.2-alpha.14 → 0.0.2-alpha.16

package/README.md

@@ -23,7 +23,7 @@ full audit log you can replay later.
 - [Setup](#setup)
 - [Quickstart](#quickstart)
 - [Concepts](#concepts) — dimensions, nanodollars, reserve→settle
-- [Generating text](#generating-text) — `chat`, `languageModel`, replay
+- [Generating text](#generating-text) — `chat`, `languageModel`, `meter`, `decisions`, replay
 - [Budgets & limits](#budgets--limits) — set caps, bumps, credits, alerts
 - [Monitoring](#monitoring) — totals, spend history, the request log
 - [Deployment-wide controls](#deployment-wide-controls) — global cap, model policy, pricing, retention
@@ -251,6 +251,35 @@ await ai.meter(ctx,
   });
 ```
 
+### `ai.decisions` — structured decisions (Jev)
+
+Budget the gateway's Decisions endpoint ([Jev](https://docs.typesafe.ai)) — typed
+`choice` / `score` / `boolean` questions evaluated against a `state` — with the
+same limits, audit log, cost tracking, and tags as `ai.chat`:
+
+```ts
+const { answers, costNanos } = await ai.decisions(ctx, {
+  state: { ticket: "Customer cannot sign in" },
+  questions: {
+    priority: {
+      type: "choice",
+      instructions: "Choose the response priority",
+      criteria: { urgent: "Respond now", normal: "Respond today" },
+    },
+    needsReview: { type: "boolean", instructions: "Does a human need to review this?" },
+  },
+  tags: [{ dimension: "team", value: "support" }],
+});
+answers.priority.choice;        // "urgent" | "normal"
+answers.needsReview.probability;
+```
+
+Sugar over `ai.meter`, so cost is the gateway's authoritative amount. Model
+defaults to `defaultEvalModel` (`"typesafe/jev-1.13"`). Requires
+`@convex-dev/ai-sdk-provider` ≥ 0.2.1 and an `ai` version with
+`experimental_evaluate` — both imported lazily, so callers who don't use
+`decisions` are unaffected.
+
 ### `ai.begin` / `ai.settle` — long async jobs (video)
 
 A video job is submit → wait minutes → poll/webhook → done, spanning multiple
@@ -282,6 +311,8 @@ ai.registerWebhook(http, {
 
 `begin` returns the admission result (it doesn't throw — check `allowed`).
 `settle` is idempotent (exactly-once server-side), so a retried webhook is safe.
+The webhook resolver receives the original, unread `Request`, allowing signature
+verification over the exact raw body; the parsed JSON comes from a clone.
 
 ### Replay
 
@@ -315,6 +346,25 @@ different dimensions. Each exposes:
 The identifier field is `userId` for `ai.users`, `name` for `ai.actions`, and
 `value` for `ai.tag(d)`. For example: `ai.tag("customer").setLimits(ctx, { value: "acme", … })`.
 
+### Request rate limits
+
+`requestsPerMinute` uses `@convex-dev/rate-limiter` 0.4.0 with a transactional
+**token bucket** for each user, action, or custom tag. A limit of 60 allows an
+initial burst of 60 requests and then refills at one request per second, up to
+60. Zero blocks all requests. Only admitted requests consume capacity: a budget,
+concurrency, or another bucket's rate rejection consumes none.
+
+This replaces the previous rolling 60-second request-log count. On upgrading,
+rate balances start full; historical requests are not imported. Budget balances,
+reservations, and spend history are preserved. Deploy the component update to
+mount its nested Rate Limiter component; applications do not register it separately.
+Changing a rate keeps its existing balance (clamped to the new capacity when
+checked); deleting and recreating a bucket starts a fresh balance.
+
+Rate limits remain transactional. Rate Limiter's asynchronous mode is not enabled
+here because simultaneous admissions must respect every configured bucket.
+Global spend accounting continues to use the existing sharded counter.
+
 ### Setting caps
 
 ```ts
@@ -549,21 +599,26 @@ this design). `ai-budget` instead **reserves then settles**:
    schedules a fold of the real cost into the totals, releasing the reservation —
    so a request is never orphaned mid-flight.
 3. **Reconcile.** A once-a-minute cron folds any stragglers and releases
-   reservations for requests that died before settling. Settlement is
-   **exactly-once** (a terminal request is never re-folded), so a slow request the
-   reconciler already swept can't double-count when it finally returns.
+   expired reservations using indexed deadlines. Expiry releases the hold but
+   leaves billing open: a late completion records its final charge once without
+   releasing the hold again. After content retention, unresolved requests retain
+   a small billing record with prompts and responses removed.
 
-Reservations are taken only on buckets that actually have a cap, so uncapped
-traffic never serializes — this is what makes arbitrary `tags` cheap: a request
-reserves on one row per *capped* dimension it carries, and nothing else.
+Requests record the exact bucket IDs and calendar windows they reserve.
+Settlement cannot release another request's hold, including across midnight or
+month boundaries. Admission reads separate policy documents; only capped buckets
+require reads of accounting state. Reporting updates still share bucket totals,
+but do not write the policy documents used by uncapped admission.
 
 **One admission rule, all scopes.** Every per-bucket cap — user, action, or any
 tag — runs through the *same* admission check: a request is admitted only when
 `committed + reserved + estimate ≤ cap` (bumps included) for **every** bucket it
 touches. Each per-bucket cap reserves on a single document, making concurrent
 admission atomic. The **global** killswitch is backed by a sharded counter for
-throughput, read as an eventually-consistent sum with no cross-request
-reservation, so concurrent global admissions can overshoot under a burst.
+throughput. Its sum is transactional but excludes unfinished and not-yet-folded
+requests and has no cross-request reservation, so global admissions can overshoot.
+Global usage is recorded even when limits are disabled. Historical usage omitted
+by older versions is not automatically reconstructed by this upgrade.
 
 Reservations are estimates, not provider-side maximum charges. If a response uses
 more tokens or costs more than estimated, settlement records the real amount and
@@ -651,6 +706,16 @@ handles rather than taking a dependency on either sibling component.
 
 ---
 
+
+## Accounting upgrade notes
+
+New request fields are optional for existing deployments. Policy documents are
+created on first admission or limit update; pending requests without deadlines
+are migrated in batches by reconciliation. Existing reservations without ownership
+metadata use their creation period and current capped buckets as a compatibility
+fallback. Exact historical hold ownership cannot be reconstructed if those caps
+changed before the upgrade. New requests always store explicit ownership.
+
 ## Development
 
 ```sh

package/dist/client/index.d.ts

@@ -39,6 +39,8 @@ export type SoftLimitInfo = BudgetEventInfo & {
 };
 export type AIBudgetOptions = {
     defaultModel?: string;
+    /** Default model for `decisions()` (the Decisions/"Jev" endpoint). */
+    defaultEvalModel?: string;
     /**
      * A *soft* limit was exceeded (request still allowed). Lets you surface budget
      * warnings even on the languageModel/Agent path where they can't be returned.
@@ -82,8 +84,19 @@ export type ChatResult = {
     /** Approaching-cap notices (empty unless a warnAtPct threshold was crossed). */
     notices: string[];
 };
+/** The tracked result of a `decisions()` call: budgeting metadata plus the
+ * structured answers from the Decisions ("Jev") endpoint. */
+export type DecisionResult = Omit<ChatResult, "text"> & {
+    /** Structured answers keyed by your question names (shape depends on each
+     * question type: `choice`, `score`, or `boolean`). */
+    answers: Record<string, any>;
+    /** The raw gateway response, including provider-specific fields (e.g.
+     * `confidence`) under `response.body`. */
+    response?: any;
+};
 /** Limits/controls settable on any budget bucket (user, action, or tag). */
 export type BucketLimits = {
+    /** Token-bucket refill per minute and burst capacity; 0 blocks all requests. */
     requestsPerMinute?: number;
     maxConcurrent?: number;
     dailySpendLimitNanos?: number;
@@ -106,6 +119,7 @@ export type BumpArgs = {
 export declare class AIBudget {
     component: AIBudgetApi;
     defaultModel: string;
+    defaultEvalModel: string;
     private onSoftLimit?;
     private onThreshold?;
     private onLimitReached?;
@@ -207,6 +221,46 @@ export declare class AIBudget {
         /** Extra attribution dimensions to bill/limit (team, customer, env, …). */
         tags?: Tag[];
     }): Promise<ChatResult>;
+    /**
+     * Budget a structured decision through the AI Gateway's Decisions ("Jev")
+     * endpoint — sugar over `meter`. Evaluates typed `questions` (choice / score /
+     * boolean) about the `state` you provide, with the same reserve→settle
+     * limits, audit log, cost tracking, and per-tag attribution as `chat`. Call
+     * from an action. `userId` defaults to the authenticated caller.
+     *
+     * Requires `@convex-dev/ai-sdk-provider` >= 0.2.1 and an `ai` version that
+     * exposes `experimental_evaluate` (AI SDK 7's evaluation interface); both are
+     * imported lazily, so consumers who never call `decisions()` are unaffected.
+     *
+     *   const { answers } = await ai.decisions(ctx, {
+     *     state: { ticket: "Customer cannot sign in" },
+     *     questions: {
+     *       priority: { type: "choice", instructions: "...", criteria: { urgent: "...", normal: "..." } },
+     *       needsReview: { type: "boolean", instructions: "..." },
+     *     },
+     *   });
+     *   answers.priority.choice; // "urgent" | "normal"
+     */
+    decisions(ctx: RunMutationCtx, args: {
+        /** The evaluation model. Defaults to `defaultEvalModel` ("typesafe/jev-1.13"). */
+        model?: string;
+        /** Context the questions are evaluated against (a string or an object). */
+        state: unknown;
+        /** Typed questions (choice / score / boolean) keyed by name. */
+        questions: Record<string, unknown>;
+        /** Whom to bill. Defaults to the authenticated user (ctx.auth). */
+        userId?: string;
+        /** Attribute spend to this action name. Defaults to the calling action. */
+        action?: string;
+        /** Extra attribution dimensions to bill/limit (team, customer, env, …). */
+        tags?: Tag[];
+        /** Reserve this exact amount (nanodollars) up front — the decision cost
+         * isn't known before the call, so a hard cap is only exact with this. */
+        estimatedCostNanos?: number;
+        rerunOf?: string;
+        /** Cancel the underlying request. */
+        abortSignal?: AbortSignal;
+    }): Promise<DecisionResult>;
     /**
      * An AI SDK LanguageModel that enforces limits and records usage/cost for
      * `userId` on every call. Drop it into `generateText`, `streamText`, or the
@@ -237,6 +291,14 @@ export declare class AIBudget {
                 dimension: string;
                 value: string;
             }[] | undefined;
+            heldBucketIds?: string[] | undefined;
+            reservationDay?: string | undefined;
+            reservationMonth?: string | undefined;
+            reservationReleased?: boolean | undefined;
+            reservationExpired?: boolean | undefined;
+            contentPurged?: boolean | undefined;
+            expiresAt?: number | undefined;
+            finishedAt?: number | undefined;
             estimatedNanos?: number | undefined;
             estimatedTokens?: number | undefined;
             unpricedModel?: boolean | undefined;
@@ -273,6 +335,14 @@ export declare class AIBudget {
                 dimension: string;
                 value: string;
             }[] | undefined;
+            heldBucketIds?: string[] | undefined;
+            reservationDay?: string | undefined;
+            reservationMonth?: string | undefined;
+            reservationReleased?: boolean | undefined;
+            reservationExpired?: boolean | undefined;
+            contentPurged?: boolean | undefined;
+            expiresAt?: number | undefined;
+            finishedAt?: number | undefined;
             estimatedNanos?: number | undefined;
             estimatedTokens?: number | undefined;
             unpricedModel?: boolean | undefined;
@@ -310,6 +380,14 @@ export declare class AIBudget {
                     dimension: string;
                     value: string;
                 }[] | undefined;
+                heldBucketIds?: string[] | undefined;
+                reservationDay?: string | undefined;
+                reservationMonth?: string | undefined;
+                reservationReleased?: boolean | undefined;
+                reservationExpired?: boolean | undefined;
+                contentPurged?: boolean | undefined;
+                expiresAt?: number | undefined;
+                finishedAt?: number | undefined;
                 estimatedNanos?: number | undefined;
                 estimatedTokens?: number | undefined;
                 unpricedModel?: boolean | undefined;
@@ -343,6 +421,14 @@ export declare class AIBudget {
                     dimension: string;
                     value: string;
                 }[] | undefined;
+                heldBucketIds?: string[] | undefined;
+                reservationDay?: string | undefined;
+                reservationMonth?: string | undefined;
+                reservationReleased?: boolean | undefined;
+                reservationExpired?: boolean | undefined;
+                contentPurged?: boolean | undefined;
+                expiresAt?: number | undefined;
+                finishedAt?: number | undefined;
                 estimatedNanos?: number | undefined;
                 estimatedTokens?: number | undefined;
                 unpricedModel?: boolean | undefined;

package/dist/client/index.js

@@ -115,12 +115,14 @@ function timingSafeEqual(a, b) {
 export class AIBudget {
     component;
     defaultModel;
+    defaultEvalModel;
     onSoftLimit;
     onThreshold;
     onLimitReached;
     constructor(component, options) {
         this.component = component;
         this.defaultModel = options?.defaultModel ?? "openai/gpt-4o-mini";
+        this.defaultEvalModel = options?.defaultEvalModel ?? "typesafe/jev-1.13";
         this.onSoftLimit = options?.onSoftLimit;
         this.onThreshold = options?.onThreshold;
         this.onLimitReached = options?.onLimitReached;
@@ -302,6 +304,72 @@ export class AIBudget {
             };
         });
     }
+    /**
+     * Budget a structured decision through the AI Gateway's Decisions ("Jev")
+     * endpoint — sugar over `meter`. Evaluates typed `questions` (choice / score /
+     * boolean) about the `state` you provide, with the same reserve→settle
+     * limits, audit log, cost tracking, and per-tag attribution as `chat`. Call
+     * from an action. `userId` defaults to the authenticated caller.
+     *
+     * Requires `@convex-dev/ai-sdk-provider` >= 0.2.1 and an `ai` version that
+     * exposes `experimental_evaluate` (AI SDK 7's evaluation interface); both are
+     * imported lazily, so consumers who never call `decisions()` are unaffected.
+     *
+     *   const { answers } = await ai.decisions(ctx, {
+     *     state: { ticket: "Customer cannot sign in" },
+     *     questions: {
+     *       priority: { type: "choice", instructions: "...", criteria: { urgent: "...", normal: "..." } },
+     *       needsReview: { type: "boolean", instructions: "..." },
+     *     },
+     *   });
+     *   answers.priority.choice; // "urgent" | "normal"
+     */
+    async decisions(ctx, args) {
+        const model = args.model ?? this.defaultEvalModel;
+        // `evaluate` is an experimental, version-gated export; import it lazily and
+        // untyped so consumers on an older `ai` (who never call this) aren't broken.
+        const evaluate = (await import("ai")).experimental_evaluate;
+        if (typeof evaluate !== "function") {
+            throw new Error("ai-budget: decisions() needs `experimental_evaluate` from the `ai` " +
+                "package (AI SDK 7's evaluation interface). Upgrade `ai` to a " +
+                "version that exports it.");
+        }
+        // Likewise, `evaluationModel` exists on @convex-dev/ai-sdk-provider >= 0.2.1.
+        const evaluationModel = convexGateway.evaluationModel;
+        if (typeof evaluationModel !== "function") {
+            throw new Error("ai-budget: decisions() needs `convexGateway.evaluationModel` from " +
+                "@convex-dev/ai-sdk-provider >= 0.2.1. Upgrade the provider.");
+        }
+        let decision;
+        const result = await this.meter(ctx, {
+            model,
+            // Store the structured request for audit/replay.
+            messages: [
+                {
+                    role: "user",
+                    content: JSON.stringify({ state: args.state, questions: args.questions }),
+                },
+            ],
+            userId: args.userId,
+            action: args.action,
+            tags: args.tags,
+            estimatedCostNanos: args.estimatedCostNanos,
+            rerunOf: args.rerunOf,
+        }, async () => {
+            decision = await evaluate({
+                model: evaluationModel(model),
+                state: args.state,
+                questions: args.questions,
+                ...(args.abortSignal ? { abortSignal: args.abortSignal } : {}),
+            });
+            return {
+                usage: decision?.usage,
+                costNanos: extractGatewayCostNanos(decision),
+            };
+        });
+        const { text: _text, ...tracking } = result;
+        return { ...tracking, answers: decision?.answers ?? {}, response: decision?.response };
+    }
     /**
      * An AI SDK LanguageModel that enforces limits and records usage/cost for
      * `userId` on every call. Drop it into `generateText`, `streamText`, or the
@@ -692,7 +760,7 @@ export class AIBudget {
             path,
             method: "POST",
             handler: httpActionGeneric(async (ctx, request) => {
-                const body = await request.json().catch(() => ({}));
+                const body = await request.clone().json().catch(() => ({}));
                 const settle = await opts.resolve(ctx, request, body);
                 if (!settle)
                     return new Response("ignored", { status: 202 });

package/dist/component/_generated/api.d.ts

@@ -33,5 +33,6 @@ export declare const api: FilterApi<typeof fullApi, FunctionReference<any, "publ
 export declare const internal: FilterApi<typeof fullApi, FunctionReference<any, "internal">>;
 export declare const components: {
     shardedCounter: import("@convex-dev/sharded-counter/_generated/component.js").ComponentApi<"shardedCounter">;
+    rateLimiter: import("@convex-dev/rate-limiter/_generated/component.js").ComponentApi<"rateLimiter">;
 };
 export {};

package/dist/component/convex.config.js

@@ -1,6 +1,8 @@
 import { defineComponent } from "convex/server";
 import shardedCounter from "@convex-dev/sharded-counter/convex.config";
+import rateLimiter from "@convex-dev/rate-limiter/convex.config";
 const component = defineComponent("aiBudget");
 // Global spend totals use a sharded counter for high write throughput.
 component.use(shardedCounter);
+component.use(rateLimiter);
 export default component;

package/dist/component/lib.d.ts

@@ -58,6 +58,14 @@ export declare const lineage: import("convex/server").RegisteredQuery<"public",
             dimension: string;
             value: string;
         }[] | undefined;
+        heldBucketIds?: import("convex/values").GenericId<"buckets">[] | undefined;
+        reservationDay?: string | undefined;
+        reservationMonth?: string | undefined;
+        reservationReleased?: boolean | undefined;
+        reservationExpired?: boolean | undefined;
+        contentPurged?: boolean | undefined;
+        expiresAt?: number | undefined;
+        finishedAt?: number | undefined;
         estimatedNanos?: number | undefined;
         estimatedTokens?: number | undefined;
         unpricedModel?: boolean | undefined;
@@ -89,6 +97,14 @@ export declare const lineage: import("convex/server").RegisteredQuery<"public",
             dimension: string;
             value: string;
         }[] | undefined;
+        heldBucketIds?: import("convex/values").GenericId<"buckets">[] | undefined;
+        reservationDay?: string | undefined;
+        reservationMonth?: string | undefined;
+        reservationReleased?: boolean | undefined;
+        reservationExpired?: boolean | undefined;
+        contentPurged?: boolean | undefined;
+        expiresAt?: number | undefined;
+        finishedAt?: number | undefined;
         estimatedNanos?: number | undefined;
         estimatedTokens?: number | undefined;
         unpricedModel?: boolean | undefined;
@@ -123,6 +139,14 @@ export declare const getRequest: import("convex/server").RegisteredQuery<"public
         dimension: string;
         value: string;
     }[] | undefined;
+    heldBucketIds?: import("convex/values").GenericId<"buckets">[] | undefined;
+    reservationDay?: string | undefined;
+    reservationMonth?: string | undefined;
+    reservationReleased?: boolean | undefined;
+    reservationExpired?: boolean | undefined;
+    contentPurged?: boolean | undefined;
+    expiresAt?: number | undefined;
+    finishedAt?: number | undefined;
     estimatedNanos?: number | undefined;
     estimatedTokens?: number | undefined;
     unpricedModel?: boolean | undefined;
@@ -159,6 +183,14 @@ export declare const listRequests: import("convex/server").RegisteredQuery<"publ
         dimension: string;
         value: string;
     }[] | undefined;
+    heldBucketIds?: import("convex/values").GenericId<"buckets">[] | undefined;
+    reservationDay?: string | undefined;
+    reservationMonth?: string | undefined;
+    reservationReleased?: boolean | undefined;
+    reservationExpired?: boolean | undefined;
+    contentPurged?: boolean | undefined;
+    expiresAt?: number | undefined;
+    finishedAt?: number | undefined;
     estimatedNanos?: number | undefined;
     estimatedTokens?: number | undefined;
     unpricedModel?: boolean | undefined;