zuplo 7.7.0 → 7.7.2

package/docs/ai-gateway/fallback.mdx

@@ -56,17 +56,21 @@ is configured, the primary model call runs unbounded.
 
 ## Quota fallback
 
-The quota fallback routes requests to an alternate, usually cheaper, model when
-an app, team, or gateway [usage limit](./usage-limits.mdx) is exceeded, rather
-than blocking the request with a `429`. This keeps an app available after an
-applicable budget, token, or request threshold is crossed, while shifting the
-overflow traffic to a lower-cost model.
-
-The **Fallback Model** policy supplies the quota fallback selection. Team and
-gateway limits can activate it whether or not the chain includes **Budgets and
-Costs**. Add Budgets and Costs when the app also needs its own limits, and place
-it after Fallback Model. If you leave the quota fallback empty, the gateway
-blocks requests with a `429` once an applicable limit is exceeded.
+For requests continuing to a provider, the quota fallback selects an alternate,
+usually cheaper, model when an app, team, or gateway
+[usage limit](./usage-limits.mdx) is exceeded, rather than blocking the request
+with a `429`. This keeps an app available after an applicable budget, token, or
+request threshold is crossed, while shifting the overflow traffic to a
+lower-cost model.
+
+The **Fallback Model** policy supplies the quota fallback selection. Add
+**Budgets and Costs** to set the app's own limits, and place it after Fallback
+Model. If you leave the quota fallback empty, the gateway blocks requests with a
+`429` once an applicable limit is exceeded.
+
+If a **Block** budget is exhausted, cache hits return `429` instead of the
+cached answer. Quota fallback doesn't apply to cache hits. See
+[Successful cache responses](./usage-limits.mdx#successful-cache-responses).
 
 :::tip
 

package/docs/ai-gateway/managing-teams.mdx

@@ -6,7 +6,7 @@ description:
 ---
 
 Teams in the Zuplo AI Gateway grant users access to apps. Teams are
-hierarchical, so access propagates to sub-teams only when the user also has at
+hierarchical, so access propagates to sub-teams only when the user also has
 permission to view the Zuplo project that contains the AI Gateway. Team or
 sub-team membership doesn't grant Zuplo project access. Teams are also where you
 define usage limits and [policy templates](./policy-templates.mdx).
@@ -41,9 +41,8 @@ After creating a team, use its tabs to configure it:
 
 - **Policy Template**—the team's [policy template](./policy-templates.mdx), the
   starting policy chain for apps created in the team.
-- **Usage & Limits**—the team's budgets, shared and per-expression, applying to
-  the team, its sub-teams, and their apps. See
-  [Usage Limits](./usage-limits.mdx).
+- **Usage & Limits**—shared budgets for the team, its sub-teams, and their apps.
+  See [Usage Limits](./usage-limits.mdx).
 - **Members**—who belongs to the team.
 
 ## Adding Team Members

package/docs/ai-gateway/policy-chains.mdx

@@ -21,12 +21,12 @@ The chain works in two layers:
   next production deploy.
 - **Each app selects from the menu.** An app's chain—which policies run, in what
   order, with what settings—is edited on the app's **Policies** tab in the
-  portal. Chain changes apply within about a minute, with no deploy.
+  portal. Chain changes apply after cached configuration refreshes, with no
+  deploy. See
+  [Propagation and accounting delays](./usage-limits.mdx#propagation-and-accounting-delays).
 
 No policy is required. An app whose chain is empty still serves requests: each
-request names its own model, and no app-specific model restrictions, budgets,
-guardrails, or caching apply. Team and gateway usage limits apply regardless of
-the app's policy chain.
+request names its own model, and no app-selected policies run.
 
 A request passes through whatever the app selected before the AI Gateway handler
 calls the provider:
@@ -36,6 +36,7 @@ Request
   → Model Filtering   ─┐
   → Budgets and Costs  ├─ the app's policy chain
   → Custom policy     ─┘
+  → Applicable budget checks
   → AI Gateway handler
   → LLM provider
 ```
@@ -62,15 +63,17 @@ remove that entry.
 - Entries run in the listed order, after the gateway's route-level policies and
   before the AI Gateway handler calls the provider.
 - A policy can answer the request itself—for example, a semantic cache hit or a
-  guardrail block. Later entries don't run in that case.
+  guardrail block. Later entries don't run in that case. Successful responses
+  still pass through automatic budget enforcement; error responses keep their
+  original status. See
+  [Successful cache responses](./usage-limits.mdx#successful-cache-responses).
 - An entry can be disabled to keep its configuration without running it.
   Disabled entries are still validated, so a disabled entry can't reference a
   policy that's no longer declared.
-- An app with an empty chain runs no app-selected policies; requests go straight
-  to the AI Gateway handler.
-- The chain runs on the request path. Policies that need the response—Semantic
-  Cache and the Akamai AI Firewall—register a response hook from their position
-  in the chain, so one entry covers both directions.
+- An app with an empty chain runs no app-selected policies. Applicable budget
+  checks still run before requests reach the AI Gateway handler.
+- Add Semantic Cache or the Akamai AI Firewall once. Each policy handles both
+  incoming requests and responses.
 - There's no allow-list of policy types. Any policy declared in `policies.json`
   is a valid chain entry, including a
   [custom code policy](./custom-policies.mdx). The only exclusions are the
@@ -125,18 +128,17 @@ Model Filtering → Fallback Model → Budgets and Costs → Semantic Cache
   never creates one. A chain with Fallback Model but no Model Filtering does
   nothing at all—the policy logs a warning and passes the request through
   unchanged. That's also why a fallback can never bypass the filter.
-- **Budgets and Costs** third: it configures the app's own limits and needs the
-  resolved selection so an exceeded app limit can activate the quota fallback.
+- **Budgets and Costs** third: use it to set the app's budget and record usage.
+  Place it after Fallback Model so an exhausted app budget can use the selected
+  quota fallback.
 - **Semantic Cache** after Budgets and Costs: cache hits still count toward
   request limits.
 
 Tracing policies work well at the end of the chain, so traces reflect the
 request the earlier policies produced.
 
-Team and gateway limits are enforced outside the app's policy chain. When one is
-exceeded, the app's quota fallback works whether or not its chain includes
-Budgets and Costs, as long as Model Filtering and Fallback Model supply a
-fallback selection.
+When a **Block** budget runs out, requests to a provider can use the app's quota
+fallback. Cache hits return `429` instead of using a fallback.
 
 ## Built-in policies
 
@@ -163,8 +165,9 @@ loader and the executor.
 Both take a single option, `cacheTtlSeconds`, which sets how long the gateway
 caches a loaded app configuration. The default is `10` seconds and `2` is the
 minimum—a lower value throws a configuration error, as does any other option
-key. New gateways scaffold the loader with `5`, so configuration changes take
-effect within about five seconds.
+key. New gateways scaffold the loader with `5`. This setting controls the
+configuration cache only; authentication caching and budget-decision refreshes
+also affect when a budget change takes effect.
 
 ## Authentication
 

package/docs/ai-gateway/policy-templates.mdx

@@ -34,11 +34,10 @@ Policies tab.
 
 :::
 
-Everything on the **Policy Template** tab is template content, budgets included:
-a **Budgets and Costs** entry in the template sets the default budget each new
-app starts with in its own chain, and doesn't enforce anything on the team. The
-team's own budgets—shared and per-expression—live on the team's **Usage &
-Limits** tab. See [Usage Limits](./usage-limits.mdx).
+New apps inherit their initial budget settings from their team's policy
+template. Configure these settings with **Budgets and Costs** in the template.
+To set a budget shared by all apps in the team, use **Usage & Limits**. See
+[Usage Limits](./usage-limits.mdx).
 
 ## Editing a template
 
@@ -50,10 +49,10 @@ Each policy in the template carries two checkboxes that control what an app
 starting from the template may do with that entry. Both start checked—a policy
 you add to a template stays locked until you uncheck the boxes:
 
-| Checkbox                                                | When checked                                                        |
-| ------------------------------------------------------- | ------------------------------------------------------------------- |
-| **Cannot edit** (`canEdit: false` in configuration)     | Apps can't change the entry's options—the template's settings apply |
-| **Cannot delete** (`canRemove: false` in configuration) | Apps can't remove the entry from their chain                        |
+| Checkbox          | When checked                                                        |
+| ----------------- | ------------------------------------------------------------------- |
+| **Cannot edit**   | Apps can't change the entry's options—the template's settings apply |
+| **Cannot delete** | Apps can't remove the entry from their chain                        |
 
 Locked entries appear in the app's chain with a lock icon and the note "Locked
 by the team policy template".

package/docs/ai-gateway/usage-limits.mdx

@@ -6,15 +6,16 @@ description:
   and give every metadata value its own separate budget using expression rules.
 ---
 
-The Zuplo AI Gateway provides hierarchical usage limits and budget controls to
-manage LLM spending across the gateway's Zuplo project. Limits can be set at the
-gateway, team, and app levels.
+Set spending, token, and request budgets for your apps, teams, and gateway to
+control how much they can use.
 
 ## Budget Hierarchy
 
-Limits at every level apply together. A request is blocked when the gateway, an
-ancestor team, or the app is over a **Block** limit. The error identifies
-whether a shared node budget or an expression-specific budget ran out:
+Limits at every level apply together. The gateway checks its own limits, parent
+team limits, and app limits. An exhausted **Block** limit rejects the request
+unless the normal provider path uses a configured
+[quota fallback](#when-a-limit-is-exceeded). **Warn** limits allow the request
+and report the warning. Budgets cover these levels:
 
 - **Gateway** - Limits across the Zuplo project (for example, $1,000/day),
   covering all teams, sub-teams, and apps combined
@@ -23,83 +24,76 @@ whether a shared node budget or an expression-specific budget ran out:
 - **Apps** - Per-app limits for granular control (for example, $10/day for a
   hackathon app)
 
-Budgets at different levels don't constrain each other's values, and app budgets
-under a team may add up to more than the team's own budget - the team's budget
-is the aggregate cap. For example, take a $100/month team containing a $50/month
-sub-team with two apps capped at $25/month and $40/month. When the apps have
-spent $40 and $10, the sub-team's aggregate reaches $50 and requests through
-**both** apps are blocked - including the app that is still under its own $25
-cap. Apps elsewhere in the $100/month team keep working until that team's own
-aggregate reaches $100.
-
-<Diagram direction="vertical" height="h-96">
-  <DiagramNode id="team" variant="green">
-    Team · spent $50 of $100
+For example, a team has a $50 monthly shared **Block** budget. App A spends $40
+and App B spends $10. Once that usage is reflected in the budget check, both
+apps have reached the team's budget. Without a quota fallback, requests through
+both apps receive `429 Too Many Requests`. An app outside this team doesn't
+share its $50 budget, but still has its own app, team, and gateway limits.
+
+<Diagram direction="vertical" height="h-80">
+  <DiagramNode id="gateway" variant="green">
+    Gateway · shared budget available
   </DiagramNode>
-  <DiagramNode id="subteam" variant="red">
-    Sub-team · spent $50 of $50
+  <DiagramNode id="team" variant="red">
+    Team · spent $50 of $50 this month
   </DiagramNode>
   <DiagramNode id="appa" variant="red">
-    App A · spent $10 of $25
+    App A · spent $40
   </DiagramNode>
   <DiagramNode id="appb" variant="red">
-    App B · spent $40 of $40
+    App B · spent $10
   </DiagramNode>
   <DiagramNode id="other" variant="green">
-    Other apps in the team
+    App outside the team
   </DiagramNode>
-  <DiagramEdge from="team" to="subteam" label="at its Block limit" />
-  <DiagramEdge from="team" to="other" label="still serving" />
-  <DiagramEdge from="subteam" to="appa" label="blocked — under its own cap" />
-  <DiagramEdge from="subteam" to="appb" label="blocked" />
+  <DiagramEdge from="gateway" to="team" label="team budget exhausted" />
+  <DiagramEdge
+    from="gateway"
+    to="other"
+    label="unaffected by this team budget"
+  />
+  <DiagramEdge from="team" to="appa" label="429 without quota fallback" />
+  <DiagramEdge from="team" to="appb" label="429 without quota fallback" />
 </Diagram>
 
+App budgets can add up to more than their team's budget. The team's budget caps
+the combined usage; increasing an app's budget doesn't increase its team's cap.
+The same rule applies to nested teams and the gateway root.
+
 ## Where limits are configured
 
-| Level   | Shared budget                                                                     | Per-expression budgets                                       |
-| ------- | --------------------------------------------------------------------------------- | ------------------------------------------------------------ |
-| Gateway | **Settings → Usage Limits**                                                       | Supported in configuration, but not authorable in the Portal |
-| Team    | The team's **Usage & Limits** tab                                                 | The team's **Usage & Limits** tab                            |
-| App     | The **Budgets and Costs** policy in the app's [policy chain](./policy-chains.mdx) | The **Budgets and Costs** policy                             |
+| Level   | Budgets                 | Where to configure                                                     |
+| ------- | ----------------------- | ---------------------------------------------------------------------- |
+| Gateway | Shared                  | **Settings → Usage Limits**                                            |
+| Team    | Shared                  | The team's **Usage & Limits** tab                                      |
+| App     | Shared and per-metadata | **Budgets and Costs** in the app's [policy chain](./policy-chains.mdx) |
 
-Gateway and team limits apply independently to every descendant app; they aren't
-copied into the app's configuration. Each node meters its own usage plus all
-descendant usage. A request is blocked if any node in its path is over a
-**Block** limit. An app's own limits live on the Budgets and Costs policy.
+Use **Budgets and Costs** to set an app's own budget.
 
 :::note
 
-Everything a team enforces lives on its **Usage & Limits** tab. The **Policy
-Template** tab is only the template: a Budgets and Costs entry there is copied
-into each new app as the app's own starting budget and doesn't enforce the
-team's budget.
+New apps inherit their initial budget settings from their
+[team policy template](./policy-templates.mdx). To limit the team's combined
+usage, set a budget on **Usage & Limits**.
 
 :::
 
 ## How a budget rule works
 
-Every node's budgets are a list of rules. Each rule answers two questions—what
-it budgets, and what the caps are.
+Each budget rule defines who shares the budget and how much they can use.
 
 **Scope** decides who shares the budget:
 
-| Scope                                      | What it budgets                                                          |
-| ------------------------------------------ | ------------------------------------------------------------------------ |
-| **Shared** (`budgetBy: "app"`)             | One shared budget for the gateway, team, or app and its descendants      |
-| **By metadata** (`budgetBy: "expression"`) | A separate budget at that node for every distinct value of an expression |
+| Scope           | What it budgets                                             |
+| --------------- | ----------------------------------------------------------- |
+| **Shared**      | One budget for the app, team, or gateway                    |
+| **By metadata** | A separate budget for every distinct value of an expression |
 
 A **By metadata** rule gives each distinct expression value its own allowance.
 With the expression `request.headers.get("x-user")` and a $5 daily cap, every
 distinct `x-user` value gets its own $5 per day—one user exhausting their budget
-doesn't affect anyone else. A node can combine a shared rule with several
-expression rules.
-
-Ancestor rules of both kinds apply to every request through the node's subtree.
-The difference is who gets blocked: a team's **Shared** rule blocks all of the
-team's traffic once the team's aggregate is over, while a team's **By metadata**
-rule meters each value across all of the team's apps combined and blocks only
-the exhausted value—the same `x-user` shares one budget whether it calls App A
-or App B.
+doesn't affect anyone else. An app can combine a shared budget with several
+per-metadata budgets.
 
 **Limit rows** set the caps. Each row is a meter, a period, an amount, and an
 action:
@@ -142,14 +136,14 @@ reset the period's usage. Custom anchors aren't configurable.
    the meters and periods you want to cap. For a **By metadata** rule, enter the
    expression that identifies each caller.
 
-1. Select **Apply**, then **Save changes** on the policy chain. The change
-   applies within about a minute.
+1. Select **Apply**, then **Save changes** on the policy chain. See
+   [Propagation and accounting delays](#propagation-and-accounting-delays) for
+   when the change takes effect.
 
 </Stepper>
 
-The app editor groups gateway and ancestor team rules under **Inherited**. Those
-rules remain stored and enforced at the gateway or team; edit them where they
-were created.
+The app editor shows team and gateway budgets under **Inherited**. To change
+one, open the team or gateway where you set it.
 
 ### Budget expressions
 
@@ -180,8 +174,8 @@ but budget separately.
 
 :::
 
-App rules are stored in `options.budgetRules` on the app's own Budgets and Costs
-policy entry. For the app rule schema, see the
+To configure budgets in JSON, use the `budgetRules` option in **Budgets and
+Costs**. For the available options, see the
 [Budgets and Costs policy reference](../policies/ai-gateway-metering-v2-inbound.mdx).
 
 This rule set budgets each user, each organization, and the app as a whole:
@@ -236,37 +230,29 @@ month, warning at $2,000, resetting on the first of the month.
 
 ## Setting team and gateway limits
 
-Team and gateway limits cover the node's own usage and all descendant usage.
-They apply to every descendant app whether or not its chain includes Budgets and
-Costs.
+A team budget covers the combined usage of its apps and sub-teams. A gateway
+budget covers all apps in the project.
 
-Team rules—shared and per-expression—are added on the team's **Usage & Limits**
-tab, using the same meter, period, amount, and action rows as the app editor.
+To add a shared team rule, select the team or sub-team in
+[Apps & Teams](https://portal.zuplo.com/+/account/project/ai/teams), then open
+**Usage & Limits**. To add a shared gateway rule, open **Settings → Usage
+Limits**. Both editors use the same meter, period, amount, and action rows as
+the app editor.
 
-To add a shared gateway rule, open **Settings → Usage Limits**. The gateway root
-supports per-expression rules in `metadata.budgetRules`, but the Portal doesn't
-author them. It displays a stored expression rule as read-only and lets you
-delete it.
-
-Each `budgetBy: "app"` rule is a shared budget for the node where the rule is
-stored. At the gateway or team level, `app` means that gateway or team node, not
-an app policy configuration. A team's usage includes its sub-teams and apps, and
-the gateway's usage includes the whole project.
-
-Gateway and team rules are stored in `metadata.budgetRules` on that node. The
-editor groups ancestor rules under **Inherited**, but each rule remains enforced
-at the node where it was created.
+The editor shows budgets from parent teams and the gateway under **Inherited**.
+To change an inherited budget, edit it on the team or gateway where you created
+it.
 
 ## When a limit is exceeded
 
-If the app has a **quota fallback** model configured, an exceeded limit routes
-requests to that model instead of blocking—see
+For a request continuing to a provider, an exhausted **Block** limit uses the
+app's configured **quota fallback** model instead of blocking. See
 [Fallback Models](./fallback.mdx), where the fallback model is selected. This
 applies to gateway and team limits as well as the app's own. The fallback's
 usage still counts toward the limits.
 
-Without a fallback, the request is rejected with `429 Too Many Requests` and a
-`budget` object naming what ran out:
+Without a quota fallback on that path, the request is rejected with
+`429 Too Many Requests` and a `budget` object naming what ran out:
 
 ```json
 {
@@ -285,20 +271,44 @@ Without a fallback, the request is rejected with `429 Too Many Requests` and a
 ```
 
 The `scope` tells the caller which budget they hit. It is `application` when the
-gateway, a team, or the app exhausts a shared node budget. It is `dimension`
-when a value exhausts an expression budget. Quote the `ruleId` when raising a
-support request.
+gateway, a team, or the app exhausts a shared budget. It is `dimension` when a
+value exhausts an expression budget. Quote the `ruleId` when raising a support
+request.
+
+### Successful cache responses
+
+Cached responses are subject to budget limits too. If a **Block** budget is
+exhausted, a cache hit returns `429 Too Many Requests` instead of the cached
+answer. Quota fallback doesn't apply to cache hits. **Warn** limits allow the
+cached answer through.
+
+Authentication failures and other error responses keep their original response;
+the budget check doesn't replace an authentication error with a budget error.
 
 :::caution{title="Budgets fail open by default"}
 
-The Budgets and Costs policy's `throwOnFailure` option—**Fail closed when
-metering is unavailable** in the editor—controls how an error while checking the
-app's own limits is handled. It defaults to `false`, so the request continues
-through the gateway. Set it to `true` to reject the request instead. Team and
-gateway limit checks run independently of this policy.
+If a team or gateway budget check fails, the request can continue. However, a
+failed check doesn't override a budget violation already detected: a cache hit
+still returns `429` if the app's budget check has found an exhausted **Block**
+limit.
+
+To reject requests when **Budgets and Costs** can't complete its own checks,
+enable **Fail closed when metering is unavailable** (`throwOnFailure`). This
+option is off by default and doesn't change how team or gateway check failures
+are handled.
 
 :::
 
+## Propagation and accounting delays
+
+Budget changes don't take effect immediately. After editing a budget or moving
+an app to another team, allow time for the saved settings and usage totals to
+update. The delay depends on your gateway's
+[cache settings](./policy-chains.mdx#configuration-executor).
+
+Requests already in progress and delays in updating usage totals can take
+spending past a budget before further requests are blocked.
+
 ## Monitoring Usage
 
 Each app, team, and the gateway show current usage against their limits:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "zuplo",
-  "version": "7.7.0",
+  "version": "7.7.2",
   "type": "module",
   "description": "The official Zuplo CLI for local development and platform management",
   "homepage": "https://zuplo.com/docs/cli/overview",
@@ -32,9 +32,9 @@
     "zuplo": "zuplo.js"
   },
   "dependencies": {
-    "@zuplo/cli": "7.7.0",
-    "@zuplo/core": "7.7.0",
-    "@zuplo/runtime": "7.7.0",
-    "@zuplo/test": "7.7.0"
+    "@zuplo/cli": "7.7.2",
+    "@zuplo/core": "7.7.2",
+    "@zuplo/runtime": "7.7.2",
+    "@zuplo/test": "7.7.2"
   }
 }