zuplo 7.6.3 → 7.6.4

package/docs/ai-gateway/managing-apps.mdx

@@ -86,10 +86,11 @@ over—see [Dynamic model routing](./cookbooks/dynamic-model-routing.mdx).
 ## Setting budgets
 
 App budgets live on the **Budgets and Costs** policy in the app's chain. Open
-the app's **Policies** tab and configure it with daily or monthly limits for
-spending, tokens, or requests. Team and gateway limits apply to every app
-automatically, even when the app's chain doesn't include Budgets and Costs. See
-[Usage Limits](./usage-limits.mdx).
+the app's **Policies** tab and add budget rules capping spending, tokens, or
+requests over an hourly, daily, weekly, or monthly period—either for the app as
+a whole or per metadata value, so each caller gets its own budget. Team and
+gateway limits apply to every app automatically, even when the app's chain
+doesn't include Budgets and Costs. See [Usage Limits](./usage-limits.mdx).
 
 ## Editing an App
 

package/docs/ai-gateway/managing-teams.mdx

@@ -39,7 +39,7 @@ click **Create Sub-Team** in the team's header.
 
 After creating a team, use its tabs to configure it:
 
-- **Policies**—the team's [policy template](./policy-templates.mdx), the
+- **Policy Template**—the team's [policy template](./policy-templates.mdx), the
   starting policy chain for apps created in the team.
 - **Usage & Limits**—spending limits that apply to all of the team's apps
   combined. See [Usage Limits](./usage-limits.mdx).

package/docs/ai-gateway/usage-limits.mdx

@@ -1,9 +1,9 @@
 ---
-title: "Usage Limits & Thresholds"
+title: "Usage Limits & Budget Rules"
 sidebar_label: "Usage Limits"
 description:
-  Set hierarchical spending, token, and request limits at the gateway, team, and
-  app levels, with warning thresholds and quota fallbacks.
+  Set spending, token, and request budgets at the gateway, team, and app levels,
+  and give every metadata value its own separate budget using expression rules.
 ---
 
 The Zuplo AI Gateway provides hierarchical usage limits and budget controls to
@@ -13,7 +13,8 @@ gateway, team, and app levels.
 ## Budget Hierarchy
 
 Limits at every level apply together—a request is blocked when _any_ level's
-limit is exceeded, and the error names the level that blocked it:
+limit is exceeded, and the error names whether the whole application or the
+caller's own segment ran out:
 
 - **Gateway** - Limits across the Zuplo project (for example, $1,000/day),
   covering all teams and apps combined
@@ -24,28 +25,66 @@ limit is exceeded, and the error names the level that blocked it:
 
 ## Where limits are configured
 
-| Level   | Where                                                                             |
-| ------- | --------------------------------------------------------------------------------- |
-| Gateway | **Settings → Usage Limits**                                                       |
-| Team    | The team's **Usage & Limits** tab                                                 |
-| App     | The **Budgets and Costs** policy in the app's [policy chain](./policy-chains.mdx) |
+| Level   | Where                                                                             | Per-metadata budgets |
+| ------- | --------------------------------------------------------------------------------- | -------------------- |
+| Gateway | **Settings → Usage Limits**                                                       | No                   |
+| Team    | The team's **Usage & Limits** tab                                                 | No                   |
+| App     | The **Budgets and Costs** policy in the app's [policy chain](./policy-chains.mdx) | Yes                  |
 
-Each level supports the same three meters, each with independent daily and
-monthly settings:
+Gateway and team limits are inherited: they apply to every app beneath them,
+whether or not the app's chain includes Budgets and Costs. An app's own limits
+live on that policy, which is also the only place you can budget per metadata
+value.
 
-- **Budget** - spend in dollars
-- **Tokens** - input plus output tokens
-- **Requests** - request count
+:::note
+
+A team's **Policy Template** tab also carries a **Budgets and Costs** policy.
+That template seeds the policy chain of apps created in the team—it sets those
+apps' limits, not the team's own. Use **Usage & Limits** to cap the team.
+
+:::
+
+## How a budget rule works
+
+An app's budgets are a list of rules on the **Budgets and Costs** policy. Each
+rule answers two questions—what it budgets, and what the caps are.
+
+**Scope** decides who shares the budget:
+
+| Scope           | What it budgets                                             |
+| --------------- | ----------------------------------------------------------- |
+| **This app**    | One shared budget for all of the app's traffic              |
+| **By metadata** | A separate budget for every distinct value of an expression |
+
+A **By metadata** rule gives each caller its own allowance. With the expression
+`request.headers.get("x-user")` and a $5 daily cap, every distinct `x-user`
+value gets its own $5 per day—one user exhausting their budget doesn't affect
+anyone else. An app has a single shared budget, and can add several metadata
+rules alongside it.
+
+**Limit rows** set the caps. Each row is a meter, a period, an amount, and an
+action:
+
+| Field      | Values                                                            |
+| ---------- | ----------------------------------------------------------------- |
+| **Meter**  | Cost (dollars), Tokens (input plus output), Requests              |
+| **Period** | Hourly, Daily, Weekly, Monthly                                    |
+| **Limit**  | An absolute amount in the meter's own unit                        |
+| **Action** | **Block** rejects the request; **Warn** notifies without blocking |
+
+Add as many rows as you need to cap several meters and periods at once. A Warn
+row and a Block row on the same meter and period give you warn-then-block—warn
+at $80, block at $100. The warn amount has to be the lower of the two.
 
-For each meter and period you can set a **Limit** (usage at or above it gets
-blocked) and a **Warning** threshold (a percentage of the limit that triggers a
-warning notification without blocking). Leave a field empty to not enforce it.
+The **Overview** tab charts daily and monthly usage only. Hourly and weekly rows
+are enforced, but no meter on that tab tracks them.
 
 :::note
 
-Daily and monthly periods are anchored to your gateway, not the calendar: a
-daily period rolls over at the time of day (UTC) the gateway was created, and a
-monthly period runs anchor-day to anchor-day. Changing a limit mid-period
+Periods are anchored to your gateway, not the calendar. Every period rolls over
+relative to the time (UTC) the gateway was created, so a daily period resets at
+that hour rather than at midnight, and a monthly one on that day of the month.
+Teams and apps created later use the same anchor. Changing a limit mid-period
 doesn't reset the period's usage.
 
 :::
@@ -60,43 +99,102 @@ doesn't reset the period's usage.
 1. On the **Policies** tab, configure the **Budgets and Costs** policy (add it
    from **Add Policy** if the chain doesn't have it).
 
-1. Set daily or monthly limits for budget, tokens, or requests, and optional
-   warning thresholds.
+1. Select **Add rule**, choose the scope, and use **Add limit** to add rows for
+   the meters and periods you want to cap. For a **By metadata** rule, enter the
+   expression that identifies each caller.
 
-1. Save. The change applies within about a minute.
+1. Select **Apply**, then **Save changes** on the policy chain. The change
+   applies within about a minute.
 
 </Stepper>
 
-:::note
+Rules set on a parent team or on the gateway appear in the editor under
+**Inherited**. Edit those where they were created.
 
-Each period in the Budgets and Costs policy's options carries an `enabled` flag
-next to its number, and so does each warning threshold. A period enforces
-nothing until `enabled` is `true` _and_ `limit` is set—options that set only
-`limit` are silently inert. Watch for this when editing the policy's options
-directly, such as in `config/policies.json` or a
-[policy template](./policy-templates.mdx).
+### Budget expressions
 
-:::
+A **By metadata** rule identifies each caller with an expression evaluated per
+request. The editor validates it as you type. Available values include:
 
-If the app has a **quota fallback** model configured, an exceeded limit routes
-requests to that model instead of blocking with a `429`—see
-[Fallback Models](./fallback.mdx). The fallback's usage still counts toward the
-limits.
+- `request.headers.get("x-user")` and other request headers
+- `request.user.sub` and `request.user.data.<key>` from the authenticated API
+  key
+- `request.url`, `request.method`, `request.query`, `request.searchParams`, and
+  `request.params`
+- `context.contextId`, `context.requestId`, `context.route`, and
+  `context.custom`
 
-:::note{title="App, team, and gateway limits are configured separately"}
+The editor labels each expression by where its value comes from, because that
+decides how much the budget can be trusted. A **client-supplied** value—a header
+or query parameter—is attribution, not enforcement: a caller who changes it gets
+a fresh budget. Budget on `request.user.*`, labeled **verified**, when the
+segment itself has to hold, and keep a shared-budget rule alongside it as the
+hard cap.
 
-The Budgets and Costs policy configures and enforces an app's own limits. Team
-and gateway limits apply to every app automatically, whether or not the app's
-policy chain includes Budgets and Costs.
+:::caution
+
+Changing an expression starts a new budget. The old rule's usage stays with the
+old expression, so every value's accounting restarts from zero. Expressions are
+also case-sensitive: `get("X-User")` and `get("x-user")` read the same header
+but budget separately.
 
 :::
 
+To write rules by hand in `config/policies.json` or a
+[policy template](./policy-templates.mdx), see the
+[Budgets and Costs policy reference](../policies/ai-gateway-metering-v2-inbound.mdx).
+
+## Setting team and gateway limits
+
+Team and gateway limits cover everything beneath them, and apply to every app
+whether or not its chain includes Budgets and Costs.
+
+Both use the same editor as an app, with the same limit rows—meter, period,
+amount, and action. Open the team's **Usage & Limits** tab, or **Settings →
+Usage Limits** for the gateway, then **Add rule** and **Add limit**.
+
+Each level takes one shared budget covering everything beneath it: a team's caps
+all of its apps combined, and the gateway's caps the whole project. Rules set
+higher up appear under **Inherited**—edit those where they were created.
+
+## When a limit is exceeded
+
+If the app has a **quota fallback** model configured, an exceeded limit routes
+requests to that model instead of blocking—see
+[Fallback Models](./fallback.mdx), where the fallback model is selected. This
+applies to gateway and team limits as well as the app's own. The fallback's
+usage still counts toward the limits.
+
+Without a fallback, the request is rejected with `429 Too Many Requests` and a
+`budget` object naming what ran out:
+
+```json
+{
+  "type": "https://httpproblems.com/http-status/429",
+  "title": "Too Many Requests",
+  "detail": "Your daily cost budget has been exhausted (limit: 5).",
+  "budget": {
+    "scope": "dimension",
+    "dimensionedBy": "request.headers.get(\"x-user\")",
+    "ruleId": "3f2a9c1b",
+    "meter": "cost",
+    "window": "daily",
+    "limit": 5
+  }
+}
+```
+
+The `scope` tells the caller which budget they hit: `application` when a shared
+budget ran out, or `dimension` when their own segment did—two situations with
+different remedies. Quote the `ruleId` when raising a support request.
+
 :::caution{title="Budgets fail open by default"}
 
-The Budgets and Costs policy's `throwOnFailure` option controls how an error
-while checking the app's own limits is handled. It defaults to `false`, so the
-request continues through the gateway. Set it to `true` to reject the request
-instead. Team and gateway limit checks run independently of this policy.
+The Budgets and Costs policy's `throwOnFailure` option—**Fail closed when
+metering is unavailable** in the editor—controls how an error while checking the
+app's own limits is handled. It defaults to `false`, so the request continues
+through the gateway. Set it to `true` to reject the request instead. Team and
+gateway limit checks run independently of this policy.
 
 :::
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "zuplo",
-  "version": "7.6.3",
+  "version": "7.6.4",
   "type": "module",
   "description": "The official Zuplo CLI for local development and platform management",
   "homepage": "https://zuplo.com/docs/cli/overview",
@@ -32,9 +32,9 @@
     "zuplo": "zuplo.js"
   },
   "dependencies": {
-    "@zuplo/cli": "7.6.3",
-    "@zuplo/core": "7.6.3",
-    "@zuplo/runtime": "7.6.3",
-    "@zuplo/test": "7.6.3"
+    "@zuplo/cli": "7.6.4",
+    "@zuplo/core": "7.6.4",
+    "@zuplo/runtime": "7.6.4",
+    "@zuplo/test": "7.6.4"
   }
 }