zuplo 6.73.29 → 6.73.31

package/docs/ai-gateway/integrations/ai-sdk.mdx

@@ -76,17 +76,24 @@ const { text } = await generateText({
 
 ### Google
 
+The `@ai-sdk/google` provider speaks Gemini's native protocol: it sends requests
+to `/models/{model}:generateContent` and authenticates with the `x-goog-api-key`
+header, neither of which the AI Gateway serves. Use the
+[OpenAI-compatible provider](https://ai-sdk.dev/providers/openai-compatible-providers)
+instead — the AI Gateway translates OpenAI-format requests to Google upstream.
+
 ```typescript
-import { createGoogleGenerativeAI } from "@ai-sdk/google";
+import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
 import { generateText } from "ai";
 
-const google = createGoogleGenerativeAI({
+const gateway = createOpenAICompatible({
+  name: "zuplo-ai-gateway",
   apiKey: process.env.ZUPLO_AI_GATEWAY_API_KEY,
   baseURL: "https://my-ai-gateway.zuplo.app/v1",
 });
 
 const { text } = await generateText({
-  model: google("gemini-2.5-flash"),
+  model: gateway("gemini-2.5-flash"),
   prompt: "Write a one-sentence bedtime story about a unicorn.",
 });
 ```

package/docs/articles/logging.mdx

@@ -113,3 +113,20 @@ request. If the property already exists, it will be overwritten.
 The log properties will be exported in a format native to the plugin you are
 using. For example, in the case of the OpenTelemetry plugin, the properties will
 be included in the `attributes` field of the log record.
+
+## Performance Under High Load
+
+Logging costs compute and memory on every request. That's rarely noticeable on a
+single response, but on an API running at a high request rate it shows up as
+lower peak throughput and higher tail latency.
+
+For high-traffic APIs, and before a [load test](./performance-testing.mdx):
+
+- Log the events you'd act on rather than everything that happens. Verbose
+  `debug` logging on a hot path is the usual culprit.
+- Keep custom log properties small. Identifiers cost far less than whole request
+  or response bodies.
+- Add logging plugins deliberately. Each one is another copy of every record.
+
+[OpenTelemetry tracing](./opentelemetry.mdx) carries a similar cost and supports
+[sampling](./opentelemetry.mdx#performance-and-sampling) to bring it down.

package/docs/articles/opentelemetry.mdx

@@ -60,6 +60,9 @@ export function runtimeInit(runtime: RuntimeExtensions) {
 }
 ```
 
+When you created your project, Zuplo added this plugin with a
+[sampling ratio](#sample-a-percentage-of-traces) already set.
+
 Deploy your project and open the **Observability** tab to see traces.
 
 ![Trace visualization](../../public/media/opentelemetry/image-1.png)
@@ -237,32 +240,45 @@ it consumes per request reduce the headroom available for the next one. On
 high-throughput APIs, that shows up as lower peak throughput and higher tail
 latency.
 
-For environments that need the most performance — high-RPS production traffic,
-latency-sensitive service-to-service calls, or a
-[load test](./performance-testing.mdx) — sample tracing down or turn it off.
+Sample traces on any API that runs at a high request rate — high-RPS production
+traffic, latency-sensitive service-to-service calls, or an API you're about to
+[load test](./performance-testing.mdx). A ratio of `0.1`, which keeps 10% of
+traces, is the standard starting point: enough traffic to see how the API
+behaves, without paying the export cost on every request. Tracing every request
+is a reasonable default during development and for APIs at moderate request
+rates.
 
 ### Sample a Percentage of Traces
 
-Set `sampling.headSampler.ratio` to export a fraction of traces. The ratio
-applies to every destination, including Zuplo's built-in storage:
+Set `sampling.headSampler.ratio` to the fraction of traces to keep. The ratio
+applies to every destination, including Zuplo's built-in storage. When you
+created your project, Zuplo set it up this way: every request traced in your
+working copy, 10% everywhere else.
 
-```ts title="zuplo.runtime.ts"
+```ts title="modules/zuplo.runtime.ts"
 import { OpenTelemetryPlugin } from "@zuplo/otel";
-import { RuntimeExtensions } from "@zuplo/runtime";
+import { environment, RuntimeExtensions } from "@zuplo/runtime";
 
 export function runtimeInit(runtime: RuntimeExtensions) {
+  const isWorkingCopy = environment.ZUPLO_ENVIRONMENT_STAGE === "working-copy";
+
   runtime.addPlugin(
     new OpenTelemetryPlugin({
       sampling: {
-        headSampler: {
-          ratio: 0.05, // Export 5% of traces
-        },
+        headSampler: { ratio: isWorkingCopy ? 1 : 0.1 },
       },
     }),
   );
 }
 ```
 
+Set `ratio` to a constant to use one value everywhere. Zuplo populates
+`ZUPLO_ENVIRONMENT_STAGE`, which reads `working-copy` in your working copy and
+`production`, `preview`, or `local` elsewhere, so local development
+(`zuplo dev`) samples at 10% along with deployed environments. See
+[system environment variables](./environment-variables.mdx#system-environment-variables)
+for the full list.
+
 By default, traces whose root span ends in an error are exported whatever the
 ratio, so errors stay debuggable on a heavily sampled API. Supplying your own
 `sampling.tailSampler` replaces that behavior.
@@ -273,6 +289,10 @@ request, so sampling reduces the cost of tracing rather than removing it.
 
 ### Disable Tracing
 
+Sampling leaves some per-request work in place. Where you need none of it —
+measuring the gateway's raw capacity, or squeezing the last of the headroom out
+of a high-RPS environment — turn tracing off instead.
+
 To remove the overhead entirely, don't register the plugin. Gate it on an
 [environment variable](./environment-variables.mdx) so you can control tracing
 per environment without changing code:
@@ -297,6 +317,14 @@ Where the plugin doesn't run, traces stop appearing in the **Observability** tab
 for that environment. Request analytics and logs come from a separate pipeline,
 so they aren't affected.
 
+### Account for Logging Too
+
+[Logging](./logging.mdx) costs compute and memory per request the same way
+tracing does. On a high-throughput API, log the events you'd act on rather than
+everything that happens, and keep custom log properties small. Sampling traces
+while logging verbosely on every request moves the cost rather than removing it.
+See [Performance Under High Load](./logging.mdx#performance-under-high-load).
+
 ## Logging
 
 The plugin can also export logs in OpenTelemetry format. Logs are sent to your

package/docs/articles/performance-testing.mdx

@@ -152,7 +152,7 @@ performing complex transformations.
 
 :::
 
-## Disable Tracing for Load Tests
+## Sample Tracing for Load Tests and High-Traffic APIs
 
 [OpenTelemetry tracing](./opentelemetry.mdx) instruments every request, opening
 spans for the request, each policy, the handler, and any `fetch` subrequests.
@@ -161,12 +161,42 @@ single response time, but the CPU and memory it consumes per request reduce the
 headroom available for the next one. Under sustained load, that shows up as
 lower peak throughput and higher tail latency.
 
-Zuplo recommends disabling OpenTelemetry before you run a load test. Otherwise
-you measure the gateway and the telemetry pipeline together, and the results
-understate the gateway's capacity.
+Zuplo recommends sampling tracing down before you run a load test, and on any
+API that serves a high request rate in production. A ratio of `0.1` keeps 10% of
+traces, enough to see how the API behaves without spending most of the gateway's
+headroom on telemetry:
 
-To disable tracing, remove the `OpenTelemetryPlugin` from `zuplo.runtime.ts`, or
-gate it on an [environment variable](./environment-variables.mdx) so you can
+```ts title="zuplo.runtime.ts"
+import { OpenTelemetryPlugin } from "@zuplo/otel";
+import { RuntimeExtensions } from "@zuplo/runtime";
+
+export function runtimeInit(runtime: RuntimeExtensions) {
+  runtime.addPlugin(
+    new OpenTelemetryPlugin({
+      sampling: {
+        headSampler: {
+          ratio: 0.1, // Keep 10% of traces
+        },
+      },
+    }),
+  );
+}
+```
+
+When you created your project, Zuplo set this ratio for everything outside the
+working copy, so check `modules/zuplo.runtime.ts` before you assume your test
+environment traces every request.
+
+Test with the ratio you plan to deploy. A pass that traces every request tells
+you little about an API that will run at 10%, and the reverse understates what
+tracing costs you. For the full sampling reference, see
+[Performance and Sampling](./opentelemetry.mdx#performance-and-sampling).
+
+### Turn Tracing Off to Measure Raw Capacity
+
+Sampling lowers the cost of tracing rather than removing it. To measure what the
+gateway alone can do, remove the `OpenTelemetryPlugin` from `zuplo.runtime.ts`,
+or gate it on an [environment variable](./environment-variables.mdx) so you can
 control it per environment without changing code:
 
 ```ts title="zuplo.runtime.ts"
@@ -181,24 +211,16 @@ export function runtimeInit(runtime: RuntimeExtensions) {
 ```
 
 Tracing runs only where you set `TRACING_ENABLED` to `true`, so leave the
-variable unset on the environment you test against.
-
-The same applies to any third-party logging or monitoring plugin you've added.
-Each one adds per-request work.
-
-:::tip
-
-Disable tracing in production too for environments that need the most
-performance, such as high-RPS traffic or latency-sensitive service-to-service
-calls. Where you want to keep some visibility, sample instead of disabling — see
-[Performance and Sampling](./opentelemetry.mdx#performance-and-sampling).
+variable unset on the environment you test against. Running one untraced pass
+next to your sampled pass tells you what telemetry costs at your target request
+rate.
 
-:::
+### Account for Logging Too
 
-If you plan to run tracing in production, run one test pass with the same
-configuration you'll deploy, including the same sampling ratio. Comparing that
-pass against an untraced one tells you what the telemetry costs you at your
-target request rate.
+The same applies to any logging or monitoring plugin you've added: each one adds
+per-request work that eats into the throughput you're trying to measure. Trim
+[logging](./logging.mdx) to the events you'd act on, and run the test with the
+same logging configuration you plan to deploy.
 
 ## Performance Testing Best Practices
 

package/docs/dev-portal/branding.mdx

@@ -0,0 +1,49 @@
+---
+title: Zuplo Branding
+sidebar_label: Zuplo Branding
+description:
+  Learn which plans show the "Powered by Zuplo" badge on the Developer Portal
+  and how to remove it.
+---
+
+Developer Portals on Zuplo's self-serve plans display a "Powered by Zuplo"
+badge. Whether the badge appears is controlled by your account's plan — it isn't
+a setting in `zudoku.config.tsx`.
+
+## Which plans show the badge
+
+| Plan          | "Powered by Zuplo" badge |
+| ------------- | ------------------------ |
+| Free          | Shown                    |
+| Pay-as-you-go | Shown                    |
+| Builder       | Shown                    |
+| Business      | Shown                    |
+| Enterprise    | Hidden                   |
+
+Upgrading from Free to a paid self-serve plan such as Builder doesn't remove the
+badge. Removing Zuplo branding is an Enterprise entitlement.
+
+## Remove the badge
+
+<EnterpriseFeature name="Developer Portal white-labeling" />
+
+To remove Zuplo branding from your Developer Portal, contact
+[sales@zuplo.com](mailto:sales@zuplo.com) or reach out to your account manager.
+Your account team enables the entitlement on your account — you don't need to
+change your project configuration.
+
+:::note
+
+The badge comes from your account's entitlements, not your portal configuration.
+There's no configuration option, environment variable, or Developer API endpoint
+that overrides it — editing your `zudoku.config.tsx` footer won't remove it.
+
+:::
+
+## Customize the rest of the footer
+
+The Zuplo badge is separate from the footer you control. On every plan you can
+set your own columns, social links, copyright notice, and logo — see
+[Footer Configuration](./zudoku/configuration/footer.mdx). To change colors,
+fonts, and the overall look of the portal, see
+[Colors and Theme](./zudoku/customization/colors-theme.mdx).

package/docs/policies/_index.md

@@ -5,6 +5,7 @@
 | set-query-params-inbound | Add or Set Query Parameters | Adds or sets query parameters on the incoming request. | api-gateway |
 | set-headers-inbound | Add or Set Request Headers | Adds or sets headers on the incoming request. | api-gateway |
 | ai-gateway-fallback-model-v2-inbound | AI Gateway Fallback Model (v2) | Adds failure and quota fallbacks to an existing AI Gateway model selection.  Place this policy after AI Gateway Model Filtering (v2). It never creates a model selection, so a misplaced policy cannot bypass filtering. | ai-gateway |
+| ai-gateway-metering-v2-inbound | AI Gateway Metering (v2) | Meters AI Gateway v2 usage and enforces options-driven usage limits.  The authentication policy must run before this policy so the app configuration id is available for meter storage and analytics. | ai-gateway |
 | ai-gateway-model-filtering-v2-inbound | AI Gateway Model Filtering (v2) | Matches AI Gateway requests against curated allow lists or open block lists, then stores the winning model reference for the route handler. | ai-gateway |
 | akamai-ai-firewall | Akamai AI Firewall | Akamai AI Firewall Inbound Policy | ai-gateway |
 | akamai-firewall-for-ai-outbound | Akamai Firewall for AI | Inspects each upstream response with Akamai's Firewall for AI detect API and replaces the response with a `403 Forbidden` if Akamai returns a `deny` rule. Useful behind AI-powered APIs to filter unsafe completions, sensitive data exposure, and toxic content before they reach the client.  The body, headers, URL, and query string sent to Akamai are configurable; by default only the response body is captured. Bodies are read from a clone so the client still receives the original. | api-gateway |

package/docs/policies/ai-gateway-metering-v2-inbound/doc.md

@@ -0,0 +1,121 @@
+# AI Gateway Metering (v2)
+
+`ai-gateway-metering-v2-inbound` records AI Gateway usage and enforces daily or
+monthly limits before the provider request runs. Declare the policy in
+`policies.json`, then add it to an application's `inboundPolicyChain`.
+
+## Example
+
+```json
+{
+  "name": "ai-gateway-metering-v2-inbound",
+  "options": {
+    "throwOnFailure": false,
+    "limits": {
+      "costs": {
+        "monthly": {
+          "enabled": true,
+          "limit": 100,
+          "warning": {
+            "enabled": true,
+            "threshold": 80
+          }
+        }
+      },
+      "requests": {
+        "daily": {
+          "enabled": true,
+          "limit": 1000
+        }
+      }
+    }
+  }
+}
+```
+
+The supported meters are `costs`, `tokens`, and `requests`. Each meter can have
+`daily` and `monthly` settings. A period is enforced only when `enabled` is
+`true` and `limit` is present. Usage equal to the limit is blocked.
+
+When `throwOnFailure` is `false`, metering failures pass the request through.
+Set it to `true` to fail the request instead.
+
+## Set limits from custom code
+
+Use a custom inbound policy to vary limits for individual requests. For example,
+the following policy gives callers with a `pro` plan a higher daily request
+limit:
+
+```typescript
+import {
+  AIGatewayMeteringV2InboundPolicy,
+  type ZuploContext,
+  type ZuploRequest,
+} from "@zuplo/runtime";
+
+export default function applyPlanLimit(
+  request: ZuploRequest,
+  context: ZuploContext
+) {
+  const dailyRequestLimit = request.user?.data.plan === "pro" ? 10_000 : 1_000;
+
+  AIGatewayMeteringV2InboundPolicy.setLimits(context, {
+    requests: {
+      daily: {
+        enabled: true,
+        limit: dailyRequestLimit,
+      },
+    },
+  });
+
+  return request;
+}
+```
+
+Register the module as a custom policy:
+
+```json
+{
+  "name": "plan-based-ai-limits",
+  "policyType": "custom-code-inbound",
+  "handler": {
+    "export": "default",
+    "module": "$import(./modules/plan-based-ai-limits)"
+  }
+}
+```
+
+Place it before metering in the application's inbound policy chain:
+
+```json
+{
+  "inboundPolicyChain": [
+    {
+      "name": "plan-based-ai-limits"
+    },
+    {
+      "name": "ai-gateway-metering-v2-inbound"
+    }
+  ]
+}
+```
+
+`setLimits` changes only the fields supplied for the current request. Other
+settings from the metering policy remain unchanged. For example, setting only
+`requests.daily.limit` preserves `requests.daily.enabled`, its warning settings,
+and all monthly limits. It does not update the policy configuration or affect
+later requests.
+
+The exported `AIGatewayMeteringV2LimitOverrides` interface can be used when a
+custom policy builds the override object separately. Its shape matches the
+`limits` option. These values are called overrides because they temporarily
+replace matching policy settings for the current request:
+
+- Choose a meter: `costs`, `tokens`, or `requests`.
+- Choose a period: `daily` or `monthly`.
+- Set one or more values: `enabled`, `limit`, or `warning`.
+
+If model filtering and fallback-model policies selected a `quotaFallback`, an
+exceeded limit activates that fallback instead of returning 429. The fallback
+request is still metered. Order the chain as model filtering, fallback model,
+metering, then policies that may short-circuit such as semantic cache.

package/docs/policies/ai-gateway-metering-v2-inbound/schema.json

@@ -0,0 +1,292 @@
+{
+  "$schema": "https://json-schema.org/draft-07/schema",
+  "$id": "https://cdn.zuplo.com/policies/runtime/schemas/ai-gateway-metering-v2-inbound.json",
+  "type": "object",
+  "title": "AI Gateway Metering (v2)",
+  "isDeprecated": false,
+  "isPaidAddOn": false,
+  "isEnterprise": false,
+  "isInternal": false,
+  "isBeta": true,
+  "isHidden": false,
+  "requiresAI": true,
+  "products": ["ai-gateway"],
+  "description": "Meters AI Gateway v2 usage and enforces options-driven usage limits.\n\nThe authentication policy must run before this policy so the app configuration id is available for meter storage and analytics.",
+  "deprecatedMessage": "",
+  "required": ["handler"],
+  "properties": {
+    "handler": {
+      "type": "object",
+      "default": {},
+      "required": ["export", "module", "options"],
+      "properties": {
+        "export": {
+          "const": "AIGatewayMeteringV2InboundPolicy",
+          "description": "The name of the exported type"
+        },
+        "module": {
+          "const": "$import(@zuplo/runtime)",
+          "description": "The module containing the policy"
+        },
+        "options": {
+          "x-zuplo-policy-type": "ai-gateway-metering-v2",
+          "title": "AIGatewayMeteringV2InboundPolicyOptions",
+          "type": "object",
+          "description": "Options for metering AI Gateway v2 usage and enforcing request-time limits.",
+          "additionalProperties": false,
+          "required": [],
+          "properties": {
+            "throwOnFailure": {
+              "type": "boolean",
+              "default": false,
+              "description": "Throw when the metering service is unavailable instead of failing open."
+            },
+            "limits": {
+              "type": "object",
+              "description": "Usage limits grouped by meter.",
+              "additionalProperties": false,
+              "properties": {
+                "costs": {
+                  "type": "object",
+                  "minProperties": 1,
+                  "additionalProperties": false,
+                  "properties": {
+                    "daily": {
+                      "type": "object",
+                      "additionalProperties": false,
+                      "required": ["enabled"],
+                      "properties": {
+                        "enabled": {
+                          "type": "boolean"
+                        },
+                        "limit": {
+                          "type": "number",
+                          "minimum": 0
+                        },
+                        "warning": {
+                          "type": "object",
+                          "additionalProperties": false,
+                          "required": ["enabled"],
+                          "properties": {
+                            "enabled": {
+                              "type": "boolean"
+                            },
+                            "threshold": {
+                              "type": "number",
+                              "minimum": 0,
+                              "maximum": 100,
+                              "description": "Percentage of the limit at which to emit a warning event."
+                            }
+                          }
+                        }
+                      }
+                    },
+                    "monthly": {
+                      "type": "object",
+                      "additionalProperties": false,
+                      "required": ["enabled"],
+                      "properties": {
+                        "enabled": {
+                          "type": "boolean"
+                        },
+                        "limit": {
+                          "type": "number",
+                          "minimum": 0
+                        },
+                        "warning": {
+                          "type": "object",
+                          "additionalProperties": false,
+                          "required": ["enabled"],
+                          "properties": {
+                            "enabled": {
+                              "type": "boolean"
+                            },
+                            "threshold": {
+                              "type": "number",
+                              "minimum": 0,
+                              "maximum": 100,
+                              "description": "Percentage of the limit at which to emit a warning event."
+                            }
+                          }
+                        }
+                      }
+                    }
+                  }
+                },
+                "tokens": {
+                  "type": "object",
+                  "minProperties": 1,
+                  "additionalProperties": false,
+                  "properties": {
+                    "daily": {
+                      "type": "object",
+                      "additionalProperties": false,
+                      "required": ["enabled"],
+                      "properties": {
+                        "enabled": {
+                          "type": "boolean"
+                        },
+                        "limit": {
+                          "type": "number",
+                          "minimum": 0
+                        },
+                        "warning": {
+                          "type": "object",
+                          "additionalProperties": false,
+                          "required": ["enabled"],
+                          "properties": {
+                            "enabled": {
+                              "type": "boolean"
+                            },
+                            "threshold": {
+                              "type": "number",
+                              "minimum": 0,
+                              "maximum": 100,
+                              "description": "Percentage of the limit at which to emit a warning event."
+                            }
+                          }
+                        }
+                      }
+                    },
+                    "monthly": {
+                      "type": "object",
+                      "additionalProperties": false,
+                      "required": ["enabled"],
+                      "properties": {
+                        "enabled": {
+                          "type": "boolean"
+                        },
+                        "limit": {
+                          "type": "number",
+                          "minimum": 0
+                        },
+                        "warning": {
+                          "type": "object",
+                          "additionalProperties": false,
+                          "required": ["enabled"],
+                          "properties": {
+                            "enabled": {
+                              "type": "boolean"
+                            },
+                            "threshold": {
+                              "type": "number",
+                              "minimum": 0,
+                              "maximum": 100,
+                              "description": "Percentage of the limit at which to emit a warning event."
+                            }
+                          }
+                        }
+                      }
+                    }
+                  }
+                },
+                "requests": {
+                  "type": "object",
+                  "minProperties": 1,
+                  "additionalProperties": false,
+                  "properties": {
+                    "daily": {
+                      "type": "object",
+                      "additionalProperties": false,
+                      "required": ["enabled"],
+                      "properties": {
+                        "enabled": {
+                          "type": "boolean"
+                        },
+                        "limit": {
+                          "type": "number",
+                          "minimum": 0
+                        },
+                        "warning": {
+                          "type": "object",
+                          "additionalProperties": false,
+                          "required": ["enabled"],
+                          "properties": {
+                            "enabled": {
+                              "type": "boolean"
+                            },
+                            "threshold": {
+                              "type": "number",
+                              "minimum": 0,
+                              "maximum": 100,
+                              "description": "Percentage of the limit at which to emit a warning event."
+                            }
+                          }
+                        }
+                      }
+                    },
+                    "monthly": {
+                      "type": "object",
+                      "additionalProperties": false,
+                      "required": ["enabled"],
+                      "properties": {
+                        "enabled": {
+                          "type": "boolean"
+                        },
+                        "limit": {
+                          "type": "number",
+                          "minimum": 0
+                        },
+                        "warning": {
+                          "type": "object",
+                          "additionalProperties": false,
+                          "required": ["enabled"],
+                          "properties": {
+                            "enabled": {
+                              "type": "boolean"
+                            },
+                            "threshold": {
+                              "type": "number",
+                              "minimum": 0,
+                              "maximum": 100,
+                              "description": "Percentage of the limit at which to emit a warning event."
+                            }
+                          }
+                        }
+                      }
+                    }
+                  }
+                }
+              }
+            }
+          }
+        }
+      },
+      "examples": [
+        {
+          "export": "AIGatewayMeteringV2InboundPolicy",
+          "module": "$import(@zuplo/runtime)",
+          "options": {
+            "limits": {
+              "costs": {
+                "daily": {
+                  "warning": {}
+                },
+                "monthly": {
+                  "warning": {}
+                }
+              },
+              "requests": {
+                "daily": {
+                  "warning": {}
+                },
+                "monthly": {
+                  "warning": {}
+                }
+              },
+              "tokens": {
+                "daily": {
+                  "warning": {}
+                },
+                "monthly": {
+                  "warning": {}
+                }
+              }
+            },
+            "throwOnFailure": false
+          }
+        }
+      ]
+    }
+  }
+}

package/docs/policies/ai-gateway-model-filtering-v2-inbound/doc.md

@@ -1,9 +1,73 @@
 # AI Gateway Model Filtering (v2) Policy
 
-Use this policy to control which models clients may select on AI Gateway routes.
-Model references use `providerName/model`; the first slash separates the
-Provider Name configured in the Zuplo Portal from the provider-specific model
-ID.
+Use this policy when an AI Gateway route must restrict which models clients may
+select. The policy is optional.
+
+Choose the setup that matches the route:
+
+- Omit Model Filtering to let each request select any available model. Every
+  request must provide `model` as `providerName/model`.
+- Use an `allowList` to expose a curated set of models and provide a default
+  model.
+- Use a `blockList` to permit available models except for specific exclusions.
+- Use a custom routing policy when model selection depends on request data or
+  application logic that an allow list or block list cannot express.
+
+`providerName` is the Provider Name configured in the Zuplo Portal. The text
+after the first slash is the provider-specific model ID, so model IDs may
+contain additional slashes.
+
+## Routing without Model Filtering
+
+When neither Model Filtering nor a custom routing policy selects a model, the AI
+Gateway handler reads the request body's `model` and uses it as the primary
+routing target:
+
+```json
+{
+  "model": "openai/gpt-4o-mini",
+  "messages": [{ "role": "user", "content": "Hello" }]
+}
+```
+
+The handler validates that:
+
+- `model` is a string in `providerName/model` form;
+- the Provider Name is configured;
+- the model is available for the route's capability;
+- the Provider Assignment has usable credentials.
+
+Missing, bare, non-string, or malformed model values receive an
+OpenAI-compatible 400 `invalid_request_error`. A validly formatted target that
+cannot be fulfilled is reported as a routing configuration error with a
+suggested fix.
+
+When Semantic Cache is attached, a cache hit also validates request-derived
+routing before returning the cached response. A removed, deprecated, or
+otherwise unavailable target therefore follows the same routing configuration
+error path instead of bypassing validation with a cached response. Native route
+requirements are checked too, so an Anthropic Messages cache hit cannot be
+served by an OpenAI-backed Provider Name.
+
+Attach Model Filtering only when the gateway must enforce model-selection rules.
+An attached policy must always have a valid, non-empty `models` configuration.
+
+### Responses management operations
+
+These Responses API operations do not have a request body:
+
+- `GET /v1/responses/:responseId`
+- `GET /v1/responses/:responseId/input_items`
+- `DELETE /v1/responses/:responseId`
+
+Because they cannot supply `model`, they require routing to be selected before
+the handler runs. Configure a `completions.allowList` in Model Filtering so its
+first entry supplies the default, or use a custom inbound policy that calls
+`AIGatewayModelRouting.set(context, { completions: "providerName/model" })`.
+Without preselected routing, the handler returns an OpenAI-compatible 400
+`invalid_request_error` with this configuration guidance.
+
+## Policy order
 
 Place Model Filtering before Fallback Model in the inbound policy chain:
 
@@ -13,7 +77,8 @@ Model Filtering -> Fallback Model -> AI Gateway handler
 
 Model Filtering accepts or rejects the request and creates the primary model
 selection. Fallback Model can then enrich that allowed selection without
-bypassing the filter.
+bypassing the filter. Fallback Model does not create a primary selection by
+itself.
 
 ## Options
 
@@ -25,9 +90,17 @@ chooses exactly one mode:
 - `blockList` leaves the catalog open except for named models. Every request
   must include `model`.
 
-Every list entry is a plain `providerName/model` string. Matching is
-case-insensitive, while configured casing is preserved for the upstream request.
-Route-target objects and fallback fields belong in the Fallback Model policy.
+Each configured list must contain at least one entry. A capability cannot define
+both `allowList` and `blockList`, and unsupported fields are rejected. Every
+list entry is a plain `providerName/model` string. Matching is case-insensitive,
+while configured casing is preserved for the upstream request. Route-target
+objects and fallback fields belong in the Fallback Model policy.
+
+Configure every capability served by a route using this policy. A route using
+`/v1/embeddings` needs `embeddings`; Chat Completions, Responses, and Anthropic
+Messages routes need `completions`. If the policy is attached but the active
+capability has no rules, the request receives a 403 response explaining which
+capability to add.
 
 ## Allow-list example
 
@@ -70,14 +143,30 @@ reported as warnings because they cannot match a request.
 
 ## Request behavior
 
-`/v1/embeddings` uses `embeddings`; Chat Completions, Responses, and Anthropic
-Messages use `completions`. Native routes also enforce their provider type:
-`/v1/responses` requires an OpenAI-backed Provider Name and `/v1/messages`
-requires an Anthropic-backed Provider Name.
+| Situation                                           | Result                                                 |
+| --------------------------------------------------- | ------------------------------------------------------ |
+| Allow list, request omits `model`                   | The first allow-list entry is selected.                |
+| Allow list, request names a listed model            | The matching configured entry is selected.             |
+| Allow list, request names an unlisted model         | 403 response listing the allowed models.               |
+| Block list, request omits `model`                   | 400 response asking for `providerName/model`.          |
+| Block list, request names a blocked model           | 403 response.                                          |
+| Either mode, request uses a bare or malformed model | 400 response explaining the required format.           |
+| Policy has no rules for the route capability        | 403 response explaining which capability to configure. |
+| Another inbound policy already selected routing     | Model Filtering leaves that selection unchanged.       |
 
-A bare model name receives a 400 response. An unlisted allow-list model or a
-blocked model receives a 403 response. If another policy has already selected a
-model, Model Filtering leaves that selection unchanged.
+Native routes also enforce provider type. `/v1/responses` requires a Provider
+Name backed by OpenAI, and `/v1/messages` requires a Provider Name backed by
+Anthropic. Provider Names may be custom labels; validation uses the provider
+type configured for that Provider Assignment.
+
+For example, this embedding request is evaluated against `models.embeddings`:
+
+```json
+{
+  "model": "openai/text-embedding-3-small",
+  "input": "Search text"
+}
+```
 
 ## Adding fallbacks
 
@@ -143,6 +232,18 @@ Attach the module as an ordinary inbound policy instead of Model Filtering:
 
 `AIGatewayModelRouting.get(context)` returns the sanitized, normalized routing
 for the current request and never returns credentials. The AI Gateway handler
-consumes a custom selection even when Model Filtering is not attached. Prefer
-one policy that creates the primary selection: Model Filtering preserves an
-existing selection, while a later custom policy can replace it.
+consumes a custom selection even when Model Filtering is not attached. If
+neither kind of policy creates a selection, the handler derives one from the
+request's required `providerName/model`.
+
+Policy order determines precedence:
+
+1. Routing selected before Model Filtering remains authoritative because Model
+   Filtering leaves an existing selection unchanged.
+2. Model Filtering creates routing when no earlier policy selected it.
+3. A custom policy placed after Model Filtering may deliberately replace that
+   selection.
+4. If no policy selects routing, the handler derives it from the request.
+
+Prefer one policy as the primary selector so the route's intent is easy to
+understand.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "zuplo",
-  "version": "6.73.29",
+  "version": "6.73.31",
   "type": "module",
   "description": "The programmable API Gateway",
   "author": "Zuplo, Inc.",
@@ -19,9 +19,9 @@
     "zuplo": "zuplo.js"
   },
   "dependencies": {
-    "@zuplo/cli": "6.73.29",
-    "@zuplo/core": "6.73.29",
-    "@zuplo/runtime": "6.73.29",
-    "@zuplo/test": "1.4.0"
+    "@zuplo/cli": "6.73.31",
+    "@zuplo/core": "6.73.31",
+    "@zuplo/runtime": "6.73.31",
+    "@zuplo/test": "1.4.4"
   }
 }