zuplo 6.73.16 → 6.73.18

package/docs/articles/opentelemetry.mdx

@@ -227,6 +227,76 @@ export function runtimeInit(runtime: RuntimeExtensions) {
 }
 ```
 
+## Performance and Sampling
+
+Tracing isn't free. The plugin instruments every request: it opens spans for the
+request, each policy, the handler, and any `fetch` subrequests, and holds them
+in memory until the request finishes. Spans are exported after the response is
+sent, so tracing adds little to any single response time, but the CPU and memory
+it consumes per request reduce the headroom available for the next one. On
+high-throughput APIs, that shows up as lower peak throughput and higher tail
+latency.
+
+For environments that need the most performance — high-RPS production traffic,
+latency-sensitive service-to-service calls, or a
+[load test](./performance-testing.mdx) — sample tracing down or turn it off.
+
+### Sample a Percentage of Traces
+
+Set `sampling.headSampler.ratio` to export a fraction of traces. The ratio
+applies to every destination, including Zuplo's built-in storage:
+
+```ts title="zuplo.runtime.ts"
+import { OpenTelemetryPlugin } from "@zuplo/otel";
+import { RuntimeExtensions } from "@zuplo/runtime";
+
+export function runtimeInit(runtime: RuntimeExtensions) {
+  runtime.addPlugin(
+    new OpenTelemetryPlugin({
+      sampling: {
+        headSampler: {
+          ratio: 0.05, // Export 5% of traces
+        },
+      },
+    }),
+  );
+}
+```
+
+By default, traces whose root span ends in an error are exported whatever the
+ratio, so errors stay debuggable on a heavily sampled API. Supplying your own
+`sampling.tailSampler` replaces that behavior.
+
+Sampling cuts the volume of trace data exported and stored, which lowers export
+overhead and backend cost. The per-request instrumentation still runs on every
+request, so sampling reduces the cost of tracing rather than removing it.
+
+### Disable Tracing
+
+To remove the overhead entirely, don't register the plugin. Gate it on an
+[environment variable](./environment-variables.mdx) so you can control tracing
+per environment without changing code:
+
+```ts title="zuplo.runtime.ts"
+import { OpenTelemetryPlugin } from "@zuplo/otel";
+import { RuntimeExtensions, environment } from "@zuplo/runtime";
+
+export function runtimeInit(runtime: RuntimeExtensions) {
+  if (environment.TRACING_ENABLED === "true") {
+    runtime.addPlugin(new OpenTelemetryPlugin());
+  }
+}
+```
+
+Set `TRACING_ENABLED` to `true` on the environments you want traced and leave it
+unset everywhere else. Tracing is off unless you opt in, so an environment built
+for maximum throughput — or for a load test — carries no tracing overhead until
+you ask for it.
+
+Where the plugin doesn't run, traces stop appearing in the **Observability** tab
+for that environment. Request analytics and logs come from a separate pipeline,
+so they aren't affected.
+
 ## Logging
 
 The plugin can also export logs in OpenTelemetry format. Logs are sent to your

package/docs/articles/performance-testing.mdx

@@ -152,6 +152,54 @@ performing complex transformations.
 
 :::
 
+## Disable Tracing for Load Tests
+
+[OpenTelemetry tracing](./opentelemetry.mdx) instruments every request, opening
+spans for the request, each policy, the handler, and any `fetch` subrequests.
+Spans are exported after the response is sent, so tracing adds little to any
+single response time, but the CPU and memory it consumes per request reduce the
+headroom available for the next one. Under sustained load, that shows up as
+lower peak throughput and higher tail latency.
+
+Zuplo recommends disabling OpenTelemetry before you run a load test. Otherwise
+you measure the gateway and the telemetry pipeline together, and the results
+understate the gateway's capacity.
+
+To disable tracing, remove the `OpenTelemetryPlugin` from `zuplo.runtime.ts`, or
+gate it on an [environment variable](./environment-variables.mdx) so you can
+control it per environment without changing code:
+
+```ts title="zuplo.runtime.ts"
+import { OpenTelemetryPlugin } from "@zuplo/otel";
+import { RuntimeExtensions, environment } from "@zuplo/runtime";
+
+export function runtimeInit(runtime: RuntimeExtensions) {
+  if (environment.TRACING_ENABLED === "true") {
+    runtime.addPlugin(new OpenTelemetryPlugin());
+  }
+}
+```
+
+Tracing runs only where you set `TRACING_ENABLED` to `true`, so leave the
+variable unset on the environment you test against.
+
+The same applies to any third-party logging or monitoring plugin you've added.
+Each one adds per-request work.
+
+:::tip
+
+Disable tracing in production too for environments that need the most
+performance, such as high-RPS traffic or latency-sensitive service-to-service
+calls. Where you want to keep some visibility, sample instead of disabling — see
+[Performance and Sampling](./opentelemetry.mdx#performance-and-sampling).
+
+:::
+
+If you plan to run tracing in production, run one test pass with the same
+configuration you'll deploy, including the same sampling ratio. Comparing that
+pass against an untraced one tells you what the telemetry costs you at your
+target request rate.
+
 ## Performance Testing Best Practices
 
 ### 1. Choose the Right Testing Tool

package/docs/articles/securing-the-gateway-with-client-mtls.mdx

@@ -15,6 +15,14 @@ the TLS handshake. Routes protected by the `mtls-auth-inbound` policy only allow
 clients that present a valid certificate chain anchored by a CA you uploaded to
 Zuplo.
 
+:::note
+
+Client mTLS is enforced at the Zuplo **API gateway** only. It is not available
+on the Zuplo developer portal — the portal cannot require client certificates
+from visitors.
+
+:::
+
 ## How client mTLS works
 
 When a client calls your Zuplo gateway:

package/docs/cli/dev.mdx

@@ -17,6 +17,17 @@ sidebar_label: dev
     "hidden": true,
     "normalize": true
   },
+  {
+    "name": "mtls-certs",
+    "type": "string",
+    "description": "Directory containing mTLS client certificates to load for local development (layout: <dir>/<cert-name>/tls.crt and tls.key). Only supported for managed-dedicated projects. If not provided, certificates are loaded from ./.zuplo-local/mtls when that directory exists.",
+    "required": false,
+    "deprecated": false,
+    "hidden": false,
+    "alias": [
+      "mtls-certificates-dir"
+    ]
+  },
   {
     "name": "start-editor",
     "type": "boolean",

package/docs/policies/_index.md

@@ -4,7 +4,8 @@
 | --- | --- | --- | --- |
 | set-query-params-inbound | Add or Set Query Parameters | Adds or sets query parameters on the incoming request. | api-gateway |
 | set-headers-inbound | Add or Set Request Headers | Adds or sets headers on the incoming request. | api-gateway |
-| ai-gateway-model-routing-v2-inbound | AI Gateway Model Routing (v2) | Matches AI Gateway requests against curated allow lists or open block lists, then stores the winning `AIGatewayRouteTarget` for the route handler. | ai-gateway |
+| ai-gateway-fallback-model-v2-inbound | AI Gateway Fallback Model (v2) | Adds failure and quota fallbacks to an existing AI Gateway model selection.  Place this policy after AI Gateway Model Filtering (v2). It never creates a model selection, so a misplaced policy cannot bypass filtering. | ai-gateway |
+| ai-gateway-model-filtering-v2-inbound | AI Gateway Model Filtering (v2) | Matches AI Gateway requests against curated allow lists or open block lists, then stores the winning model reference for the route handler. | ai-gateway |
 | akamai-ai-firewall | Akamai AI Firewall | Akamai AI Firewall Inbound Policy | ai-gateway |
 | akamai-firewall-for-ai-outbound | Akamai Firewall for AI | Inspects each upstream response with Akamai's Firewall for AI detect API and replaces the response with a `403 Forbidden` if Akamai returns a `deny` rule. Useful behind AI-powered APIs to filter unsafe completions, sensitive data exposure, and toxic content before they reach the client.  The body, headers, URL, and query string sent to Akamai are configurable; by default only the response body is captured. Bodies are read from a clone so the client still receives the original. | api-gateway |
 | akamai-firewall-for-ai-inbound | Akamai Firewall for AI | Inspects each incoming request with Akamai's Firewall for AI detect API and blocks the request if Akamai returns a `deny` rule. Useful in front of AI-powered APIs to filter prompt injection, jailbreaks, and other unsafe inputs before they reach the model.  The body, headers, URL, and query string sent to Akamai are configurable; by default only the request body is captured. Bodies are read from a clone so the upstream handler still sees the original. | api-gateway |

package/docs/policies/ai-gateway-fallback-model-v2-inbound/doc.md

@@ -0,0 +1,129 @@
+# AI Gateway Fallback Model (v2) Policy
+
+Use this policy to add resilience to a model selection created by Model
+Filtering or a custom routing policy. Place it after Model Filtering:
+
+```text
+Model Filtering -> Fallback Model -> AI Gateway handler
+```
+
+Fallback Model never creates a primary selection. If it runs without a prior
+selection, it logs a warning and leaves the request unchanged.
+
+## Options
+
+`models` must contain `completions`, `embeddings`, or both. Each configured
+capability must set at least one of:
+
+- `fallback`: the `providerName/model` attempted after a retryable error or
+  after the primary exceeds `fallbackTimeoutSeconds`.
+- `quotaFallback`: the `providerName/model` selected after a
+  usage-limit-exceeded signal. This path is independent of retry and timeout
+  fallback.
+
+`fallbackTimeoutSeconds` applies to every configured `fallback`. It defaults to
+60 seconds and accepts values from 1 through 300.
+
+## Cross-provider fallback example
+
+```json
+{
+  "name": "ai-gateway-fallback-model-v2-inbound",
+  "policyType": "ai-gateway-fallback-model-v2",
+  "handler": {
+    "export": "AIGatewayFallbackModelV2InboundPolicy",
+    "module": "$import(@zuplo/runtime)",
+    "options": {
+      "models": {
+        "completions": {
+          "fallback": "anthropic/claude-haiku-4-5",
+          "quotaFallback": "openai/gpt-4o-mini"
+        },
+        "embeddings": {
+          "fallback": "openai/text-embedding-3-small"
+        }
+      },
+      "fallbackTimeoutSeconds": 30
+    }
+  }
+}
+```
+
+This configuration preserves the primary model selected earlier in the chain.
+For completions, Anthropic becomes the retry/timeout backup and OpenAI becomes
+the independent quota fallback. The gateway validates every reference against
+the live provider catalog and resolves credentials when the selection is set.
+
+## Composition behavior
+
+Only capabilities already present in the current selection are modified.
+Capabilities not configured here are preserved, as are fields this policy does
+not override. A configured `fallback` replaces an existing backup and timeout; a
+configured `quotaFallback` replaces an existing quota fallback.
+
+Retry and timeout fallback is available for translated Chat Completions and
+Embeddings requests. Native `/v1/messages` and `/v1/responses` requests are
+passed through without retrying a backup. Quota fallback remains a separate,
+metering-driven path.
+
+## Write your own fallback policy
+
+Fallback Model also uses the public routing primitives, so custom code can
+choose fallbacks dynamically:
+
+- `AIGatewayModels.load(context)` returns the cached provider catalog, including
+  each model's capability, status, and per-token pricing.
+- `AIGatewayModelRouting.get(context)` returns the sanitized, normalized routing
+  selected earlier in the chain. It never returns provider credentials.
+- `AIGatewayModelRouting.set(context, routing)` validates every target, resolves
+  provider credentials internally, and replaces the complete stored selection.
+
+Because `set()` replaces rather than partially updates routing, a custom
+fallback policy must read, merge, and set. It should leave the request unchanged
+when no earlier policy created a selection:
+
+```typescript
+import {
+  AIGatewayModelRouting,
+  AIGatewayModels,
+  type ZuploContext,
+  type ZuploRequest,
+} from "@zuplo/runtime";
+
+export default async function addFallback(
+  request: ZuploRequest,
+  context: ZuploContext
+) {
+  const routing = AIGatewayModelRouting.get(context);
+  const current = routing?.completions;
+  if (!routing || !current) {
+    return request;
+  }
+
+  const providers = await AIGatewayModels.load(context);
+  const anthropic = providers.find(
+    (provider) => provider.providerName === "anthropic"
+  );
+  const fallback = anthropic?.models.find(
+    (candidate) =>
+      candidate.capability === "completions" && candidate.status === "active"
+  );
+  if (!fallback) {
+    return request;
+  }
+
+  await AIGatewayModelRouting.set(context, {
+    ...routing,
+    completions: {
+      ...(typeof current === "string" ? { main: current } : current),
+      backup: `anthropic/${fallback.model}`,
+      fallbackTimeoutSeconds: 30,
+    },
+  });
+  return request;
+}
+```
+
+Place this custom policy after Model Filtering, replacing Fallback Model in the
+inbound chain. Keeping the no-selection guard prevents a misordered custom
+policy from creating a primary selection and bypassing filtering.

package/docs/policies/ai-gateway-fallback-model-v2-inbound/intro.md

@@ -0,0 +1,4 @@
+AI Gateway Fallback Model (v2) adds retry, timeout, and quota fallbacks to an
+existing model selection. Place it after AI Gateway Model Filtering (v2). It
+preserves the primary selection and never creates one when the policy chain is
+misordered.

package/docs/policies/ai-gateway-fallback-model-v2-inbound/schema.json

@@ -0,0 +1,114 @@
+{
+  "$schema": "https://json-schema.org/draft-07/schema",
+  "$id": "https://cdn.zuplo.com/policies/runtime/schemas/ai-gateway-fallback-model-v2-inbound.json",
+  "type": "object",
+  "title": "AI Gateway Fallback Model (v2)",
+  "isDeprecated": false,
+  "isPaidAddOn": false,
+  "isEnterprise": false,
+  "isInternal": false,
+  "isBeta": true,
+  "isHidden": false,
+  "requiresAI": true,
+  "products": ["ai-gateway"],
+  "description": "Adds failure and quota fallbacks to an existing AI Gateway model selection.\n\nPlace this policy after AI Gateway Model Filtering (v2). It never creates a model selection, so a misplaced policy cannot bypass filtering.",
+  "deprecatedMessage": "",
+  "required": ["handler"],
+  "properties": {
+    "handler": {
+      "type": "object",
+      "default": {},
+      "required": ["export", "module", "options"],
+      "properties": {
+        "export": {
+          "const": "AIGatewayFallbackModelV2InboundPolicy",
+          "description": "The name of the exported type"
+        },
+        "module": {
+          "const": "$import(@zuplo/runtime)",
+          "description": "The module containing the policy"
+        },
+        "options": {
+          "x-zuplo-policy-type": "ai-gateway-fallback-model-v2",
+          "type": "object",
+          "title": "AIGatewayFallbackModelV2InboundPolicyOptions",
+          "description": "Options for adding failure and quota fallbacks to an existing AI Gateway model selection.",
+          "additionalProperties": false,
+          "required": ["models"],
+          "properties": {
+            "models": {
+              "type": "object",
+              "title": "Fallback Models",
+              "description": "Fallbacks grouped by AI Gateway capability.",
+              "additionalProperties": false,
+              "minProperties": 1,
+              "properties": {
+                "completions": {
+                  "title": "Completions Fallbacks",
+                  "type": "object",
+                  "additionalProperties": false,
+                  "minProperties": 1,
+                  "properties": {
+                    "fallback": {
+                      "title": "Fallback Model",
+                      "description": "The model attempted after a retryable error or timeout.",
+                      "type": "string",
+                      "pattern": "^[^/\\s]+/.+$"
+                    },
+                    "quotaFallback": {
+                      "title": "Quota Fallback Model",
+                      "description": "The model selected after a usage-limit-exceeded signal.",
+                      "type": "string",
+                      "pattern": "^[^/\\s]+/.+$"
+                    }
+                  }
+                },
+                "embeddings": {
+                  "title": "Embeddings Fallbacks",
+                  "type": "object",
+                  "additionalProperties": false,
+                  "minProperties": 1,
+                  "properties": {
+                    "fallback": {
+                      "title": "Fallback Model",
+                      "description": "The model attempted after a retryable error or timeout.",
+                      "type": "string",
+                      "pattern": "^[^/\\s]+/.+$"
+                    },
+                    "quotaFallback": {
+                      "title": "Quota Fallback Model",
+                      "description": "The model selected after a usage-limit-exceeded signal.",
+                      "type": "string",
+                      "pattern": "^[^/\\s]+/.+$"
+                    }
+                  }
+                }
+              }
+            },
+            "fallbackTimeoutSeconds": {
+              "type": "number",
+              "title": "Fallback Timeout Seconds",
+              "description": "How long a primary model may take before the failure fallback is attempted.",
+              "minimum": 1,
+              "maximum": 300,
+              "default": 60
+            }
+          }
+        }
+      },
+      "examples": [
+        {
+          "export": "AIGatewayFallbackModelV2InboundPolicy",
+          "module": "$import(@zuplo/runtime)",
+          "options": {
+            "fallbackTimeoutSeconds": 60,
+            "models": {
+              "completions": {},
+              "embeddings": {}
+            }
+          }
+        }
+      ]
+    }
+  }
+}

package/docs/policies/ai-gateway-model-filtering-v2-inbound/doc.md

@@ -0,0 +1,148 @@
+# AI Gateway Model Filtering (v2) Policy
+
+Use this policy to control which models clients may select on AI Gateway routes.
+Model references use `providerName/model`; the first slash separates the
+Provider Name configured in the Zuplo Portal from the provider-specific model
+ID.
+
+Place Model Filtering before Fallback Model in the inbound policy chain:
+
+```text
+Model Filtering -> Fallback Model -> AI Gateway handler
+```
+
+Model Filtering accepts or rejects the request and creates the primary model
+selection. Fallback Model can then enrich that allowed selection without
+bypassing the filter.
+
+## Options
+
+`models` must contain `completions`, `embeddings`, or both. Each capability
+chooses exactly one mode:
+
+- `allowList` creates a curated catalog. Only listed models are accepted, and
+  the first entry is used when a request omits `model`.
+- `blockList` leaves the catalog open except for named models. Every request
+  must include `model`.
+
+Every list entry is a plain `providerName/model` string. Matching is
+case-insensitive, while configured casing is preserved for the upstream request.
+Route-target objects and fallback fields belong in the Fallback Model policy.
+
+## Allow-list example
+
+```json
+{
+  "name": "ai-gateway-model-filtering-v2-inbound",
+  "policyType": "ai-gateway-model-filtering-v2",
+  "handler": {
+    "export": "AIGatewayModelFilteringV2InboundPolicy",
+    "module": "$import(@zuplo/runtime)",
+    "options": {
+      "models": {
+        "completions": {
+          "allowList": ["openai/gpt-4o-mini", "anthropic/claude-haiku-4-5"]
+        },
+        "embeddings": {
+          "allowList": ["openai/text-embedding-3-small"]
+        }
+      }
+    }
+  }
+}
+```
+
+## Block-list example
+
+```json
+{
+  "models": {
+    "completions": {
+      "blockList": ["openai/deprecated-model", "anthropic/deprecated-model"]
+    }
+  }
+}
+```
+
+In block-list mode, an unknown Provider Name is rejected even if the model is
+not listed. Entries whose Provider Names are absent from the live catalog are
+reported as warnings because they cannot match a request.
+
+## Request behavior
+
+`/v1/embeddings` uses `embeddings`; Chat Completions, Responses, and Anthropic
+Messages use `completions`. Native routes also enforce their provider type:
+`/v1/responses` requires an OpenAI-backed Provider Name and `/v1/messages`
+requires an Anthropic-backed Provider Name.
+
+A bare model name receives a 400 response. An unlisted allow-list model or a
+blocked model receives a 403 response. If another policy has already selected a
+model, Model Filtering leaves that selection unchanged.
+
+## Adding fallbacks
+
+Declare AI Gateway Fallback Model (v2) separately and place it immediately after
+this policy. Its `fallback` handles retryable errors and timeouts;
+`quotaFallback` handles usage-limit signals independently.
+
+## Write your own routing policy
+
+Everything Model Filtering does is built on two public primitives, so a custom
+inbound policy can replace it entirely. The policy's job is to choose one
+allowed target and store it:
+
+- `AIGatewayModels.load(context)` returns the cached provider catalog, including
+  each model's capability, status, and per-token pricing.
+- `AIGatewayModelRouting.set(context, routing)` validates the routing, resolves
+  provider credentials internally, and stores the selection that the AI Gateway
+  handler consumes.
+
+```typescript
+import {
+  AIGatewayModelRouting,
+  AIGatewayModels,
+  type ZuploContext,
+  type ZuploRequest,
+} from "@zuplo/runtime";
+
+export default async function selectModel(
+  request: ZuploRequest,
+  context: ZuploContext
+) {
+  const providers = await AIGatewayModels.load(context);
+  const openAI = providers.find(
+    (provider) => provider.providerName === "openai"
+  );
+  const model = openAI?.models.find(
+    (candidate) =>
+      candidate.capability === "completions" && candidate.status === "active"
+  );
+  if (!model) {
+    throw new Error("No active OpenAI completions model is available");
+  }
+
+  await AIGatewayModelRouting.set(context, {
+    completions: `openai/${model.model}`,
+  });
+  return request;
+}
+```
+
+Attach the module as an ordinary inbound policy instead of Model Filtering:
+
+```json
+{
+  "name": "my-model-filtering-inbound",
+  "policyType": "custom-code-inbound",
+  "handler": {
+    "export": "default",
+    "module": "$import(./modules/my-model-filtering)"
+  }
+}
+```
+
+`AIGatewayModelRouting.get(context)` returns the sanitized, normalized routing
+for the current request and never returns credentials. The AI Gateway handler
+consumes a custom selection even when Model Filtering is not attached. Prefer
+one policy that creates the primary selection: Model Filtering preserves an
+existing selection, while a later custom policy can replace it.

package/docs/policies/ai-gateway-model-filtering-v2-inbound/intro.md

@@ -0,0 +1,5 @@
+AI Gateway Model Filtering (v2) controls which `providerName/model` references
+clients may select. Use an `allowList` for a curated catalog with a default, or
+a `blockList` for an open catalog with explicit exclusions. Put this policy
+before AI Gateway Fallback Model (v2), which adds fallback behavior only after a
+primary model has passed filtering.

package/docs/policies/ai-gateway-model-filtering-v2-inbound/schema.json

@@ -0,0 +1,148 @@
+{
+  "$schema": "https://json-schema.org/draft-07/schema",
+  "$id": "https://cdn.zuplo.com/policies/runtime/schemas/ai-gateway-model-filtering-v2-inbound.json",
+  "type": "object",
+  "title": "AI Gateway Model Filtering (v2)",
+  "isDeprecated": false,
+  "isPaidAddOn": false,
+  "isEnterprise": false,
+  "isInternal": false,
+  "isBeta": true,
+  "isHidden": false,
+  "requiresAI": true,
+  "products": ["ai-gateway"],
+  "description": "Matches AI Gateway requests against curated allow lists or open block lists, then stores the winning model reference for the route handler.",
+  "deprecatedMessage": "",
+  "required": ["handler"],
+  "properties": {
+    "handler": {
+      "type": "object",
+      "default": {},
+      "required": ["export", "module", "options"],
+      "properties": {
+        "export": {
+          "const": "AIGatewayModelFilteringV2InboundPolicy",
+          "description": "The name of the exported type"
+        },
+        "module": {
+          "const": "$import(@zuplo/runtime)",
+          "description": "The module containing the policy"
+        },
+        "options": {
+          "x-zuplo-policy-type": "ai-gateway-model-filtering-v2",
+          "type": "object",
+          "title": "AIGatewayModelFilteringV2InboundPolicyOptions",
+          "description": "Options for allowing or blocking providerName/model references for each AI Gateway capability.",
+          "additionalProperties": false,
+          "required": ["models"],
+          "properties": {
+            "models": {
+              "type": "object",
+              "title": "Models",
+              "description": "Model filtering rules grouped by AI Gateway capability.",
+              "additionalProperties": false,
+              "minProperties": 1,
+              "properties": {
+                "completions": {
+                  "title": "Completions Models",
+                  "description": "Rules for chat completions, Responses, and Anthropic Messages requests.",
+                  "oneOf": [
+                    {
+                      "type": "object",
+                      "additionalProperties": false,
+                      "required": ["allowList"],
+                      "properties": {
+                        "allowList": {
+                          "type": "array",
+                          "title": "Allowed Models",
+                          "description": "Models clients may select; the first entry is the default when model is omitted.",
+                          "minItems": 1,
+                          "items": {
+                            "type": "string",
+                            "title": "Provider and Model",
+                            "description": "A model reference in providerName/model form, split on the first slash.",
+                            "pattern": "^[^/\\s]+/.+$"
+                          }
+                        }
+                      }
+                    },
+                    {
+                      "type": "object",
+                      "additionalProperties": false,
+                      "required": ["blockList"],
+                      "properties": {
+                        "blockList": {
+                          "type": "array",
+                          "title": "Blocked Models",
+                          "description": "Models rejected in open-but-filtered mode.",
+                          "minItems": 1,
+                          "items": {
+                            "type": "string",
+                            "title": "Provider and Model",
+                            "description": "A model reference in providerName/model form, split on the first slash.",
+                            "pattern": "^[^/\\s]+/.+$"
+                          }
+                        }
+                      }
+                    }
+                  ]
+                },
+                "embeddings": {
+                  "title": "Embeddings Models",
+                  "description": "Rules for embedding requests.",
+                  "oneOf": [
+                    {
+                      "type": "object",
+                      "additionalProperties": false,
+                      "required": ["allowList"],
+                      "properties": {
+                        "allowList": {
+                          "type": "array",
+                          "title": "Allowed Models",
+                          "minItems": 1,
+                          "items": {
+                            "type": "string",
+                            "title": "Provider and Model",
+                            "description": "A model reference in providerName/model form, split on the first slash.",
+                            "pattern": "^[^/\\s]+/.+$"
+                          }
+                        }
+                      }
+                    },
+                    {
+                      "type": "object",
+                      "additionalProperties": false,
+                      "required": ["blockList"],
+                      "properties": {
+                        "blockList": {
+                          "type": "array",
+                          "title": "Blocked Models",
+                          "minItems": 1,
+                          "items": {
+                            "type": "string",
+                            "title": "Provider and Model",
+                            "description": "A model reference in providerName/model form, split on the first slash.",
+                            "pattern": "^[^/\\s]+/.+$"
+                          }
+                        }
+                      }
+                    }
+                  ]
+                }
+              }
+            }
+          }
+        }
+      },
+      "examples": [
+        {
+          "export": "AIGatewayModelFilteringV2InboundPolicy",
+          "module": "$import(@zuplo/runtime)",
+          "options": {
+            "models": {}
+          }
+        }
+      ]
+    }
+  }
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "zuplo",
-  "version": "6.73.16",
+  "version": "6.73.18",
   "type": "module",
   "description": "The programmable API Gateway",
   "author": "Zuplo, Inc.",
@@ -19,9 +19,9 @@
     "zuplo": "zuplo.js"
   },
   "dependencies": {
-    "@zuplo/cli": "6.73.16",
-    "@zuplo/core": "6.73.16",
-    "@zuplo/runtime": "6.73.16",
+    "@zuplo/cli": "6.73.18",
+    "@zuplo/core": "6.73.18",
+    "@zuplo/runtime": "6.73.18",
     "@zuplo/test": "1.4.0"
   }
 }

package/docs/policies/ai-gateway-model-routing-v2-inbound/doc.md

@@ -1,165 +0,0 @@
-# AI Gateway Model Routing (v2) Policy
-
-The AI Gateway Model Routing (v2) policy applies declarative allow or block
-rules to AI requests. The Provider Name configured in the Zuplo Portal is its
-routing address: configured targets use `providerName/model` form and split on
-the first slash, so provider-specific model IDs can contain additional slashes.
-
-## Configuration
-
-Set a non-empty `models` object containing `completions`, `embeddings`, or both.
-Each capability uses exactly one mode:
-
-- `allowList` is curated mode. Only listed `main` models are accepted, entry 0
-  is the default when the request omits `model`, and entries can include
-  `backup`, `fallbackTimeoutSeconds`, and `quotaFallback`.
-- `blockList` is open-but-filtered mode. Any unblocked target can be requested,
-  but every request must include `model`; this mode has no defaults or
-  fallbacks.
-
-Because Responses management operations such as `GET /v1/responses/{id}` and
-`DELETE /v1/responses/{id}` have no request body, attach curated `allowList`
-rules (or a custom routing policy that supplies an OpenAI selection) to those
-operations. A `blockList` policy cannot select credentials for a bodyless
-request because that mode intentionally has no default.
-
-An allow-list entry can be a string shorthand or a full target object:
-
-```json
-{
-  "name": "ai-gateway-model-routing-v2-inbound",
-  "policyType": "ai-gateway-model-routing-v2",
-  "handler": {
-    "export": "AIGatewayModelRoutingV2InboundPolicy",
-    "module": "$import(@zuplo/runtime)",
-    "options": {
-      "models": {
-        "completions": {
-          "allowList": [
-            "openai/gpt-5",
-            {
-              "main": "anthropic/claude-sonnet-4-6",
-              "backup": "openai/gpt-5-mini",
-              "fallbackTimeoutSeconds": 30,
-              "quotaFallback": "anthropic/claude-haiku-4-5"
-            }
-          ]
-        },
-        "embeddings": {
-          "blockList": ["openai/legacy-embedding-model"]
-        }
-      }
-    }
-  }
-}
-```
-
-The first slash separates the Provider Name from the model. Model IDs may
-contain additional slashes, such as
-`my-fireworks/accounts/fireworks/models/llama-v3`. Provider Name and model
-matching is case-insensitive. In curated mode the configured casing is preserved
-when the request is sent upstream.
-
-## Request behavior
-
-Every non-empty request model must use `providerName/model`. A bare model name
-receives an actionable 400 response. When the model is omitted, curated mode
-uses allow-list entry zero; block-list mode rejects the request because it has
-no default.
-
-`/v1/embeddings` uses the `embeddings` rules; Chat Completions, Responses, and
-Anthropic Messages use `completions` rules. The first slash separates the
-Provider Name, so model IDs containing slashes remain intact after that
-separator.
-
-The pass-through routes are provider-native: `/v1/messages` accepts Anthropic
-targets and `/v1/responses` accepts OpenAI targets. A request that explicitly
-selects the wrong provider receives a 400 response. An incompatible entry-0
-default is a policy configuration error.
-
-## Fallback behavior
-
-When `backup` is configured, the gateway fails over after a retryable error or
-after `fallbackTimeoutSeconds` (60 seconds by default). The timeout must be from
-1 through 300 seconds. `quotaFallback` independently selects another model after
-a usage-limit-exceeded signal. Fallbacks do not chain through another allow-list
-entry.
-
-Retry and timeout backups apply to translated Chat Completions and Embeddings
-requests. Native `/v1/messages` and `/v1/responses` requests are passed through
-without retrying a `backup`; `AIGatewayModelRouting.set` still validates every
-configured target and credential eagerly. A same-native-provider `quotaFallback`
-can still serve those routes after a usage-limit-exceeded signal.
-
-## Write your own routing policy
-
-Everything this policy does is built on two public primitives, so a custom
-inbound policy can replace it entirely. The policy's job is to collapse its
-allow list to one winning target and store it; custom code makes the same call
-with whatever logic it wants:
-
-- `AIGatewayModels.load(context)` returns the cached provider catalog, including
-  each model's capability, status, and per-token pricing.
-- `AIGatewayModelRouting.set(context, routing)` validates the routing, resolves
-  provider credentials internally, and stores the selection that the AI Gateway
-  handler consumes.
-
-A per-capability target is the same `AIGatewayRouteTarget` shape used by
-`allowList` entries: a `"providerName/model"` string, or an object with `main`
-and optional `backup`, `fallbackTimeoutSeconds`, and `quotaFallback`.
-
-```typescript
-import {
-  AIGatewayModelRouting,
-  AIGatewayModels,
-  type ZuploContext,
-  type ZuploRequest,
-} from "@zuplo/runtime";
-
-export default async function routeModel(
-  request: ZuploRequest,
-  context: ZuploContext
-) {
-  const providers = await AIGatewayModels.load(context);
-  const openAI = providers.find(
-    (provider) => provider.providerName === "openai"
-  );
-  const model = openAI?.models.find(
-    (candidate) =>
-      candidate.capability === "completions" && candidate.status === "active"
-  );
-  if (!model) {
-    throw new Error("No active OpenAI completions model is available");
-  }
-  await AIGatewayModelRouting.set(context, {
-    completions: {
-      main: `openai/${model.model}`,
-      backup: "anthropic/claude-sonnet-4-6",
-      fallbackTimeoutSeconds: 30,
-    },
-  });
-  return request;
-}
-```
-
-Attach the module as an ordinary inbound policy on the AI Gateway route instead
-of (or before) the built-in model-routing policy:
-
-```json
-{
-  "name": "my-model-routing-inbound",
-  "policyType": "custom-code-inbound",
-  "handler": {
-    "export": "default",
-    "module": "$import(./modules/my-model-routing)"
-  }
-}
-```
-
-`AIGatewayModelRouting.get(context)` returns the sanitized, normalized routing
-set for the current request; it never returns credentials. The AI Gateway
-handler consumes this selection even when the built-in model-routing policy is
-not attached to the route. Prefer one routing policy per route. If you compose
-policies intentionally, a custom policy before the built-in policy wins because
-the built-in policy preserves an existing selection. A custom policy after it
-can replace an allowed selection, while a built-in rejection stops the chain.

package/docs/policies/ai-gateway-model-routing-v2-inbound/intro.md

@@ -1,18 +0,0 @@
-The AI Gateway Model Routing (v2) policy selects models with either a curated
-`allowList` or an open-but-filtered `blockList`. Allow-list entries accept the
-string shorthand `"providerName/model"` or an object with `main`, `backup`,
-`fallbackTimeoutSeconds`, and `quotaFallback`; entry zero is the default when a
-curated request omits `model`, while block-list mode requires every request to
-name a model.
-
-Request model references use `providerName/model`, split on the first slash,
-match provider and model case-insensitively, and preserve configured casing
-upstream. A non-empty request model without a Provider Name is rejected with an
-actionable 400. Bodyless Responses management operations need curated or custom
-routing because block-list mode has no default. Retry backups apply to
-translated Chat Completions and Embeddings, not native pass-through routes;
-quota fallbacks remain independent and fallbacks do not chain.
-
-Custom inbound policies can inspect `AIGatewayModels.load(context)` and call
-`AIGatewayModelRouting.set(context, routing)` to select the same target without
-attaching the built-in policy. This policy is in beta.

package/docs/policies/ai-gateway-model-routing-v2-inbound/schema.json

@@ -1,263 +0,0 @@
-{
-  "$schema": "https://json-schema.org/draft-07/schema",
-  "$id": "https://cdn.zuplo.com/policies/runtime/schemas/ai-gateway-model-routing-v2-inbound.json",
-  "type": "object",
-  "title": "AI Gateway Model Routing (v2)",
-  "isDeprecated": false,
-  "isPaidAddOn": false,
-  "isEnterprise": false,
-  "isInternal": false,
-  "isBeta": true,
-  "isHidden": false,
-  "requiresAI": true,
-  "products": ["ai-gateway"],
-  "description": "Matches AI Gateway requests against curated allow lists or open block lists, then stores the winning `AIGatewayRouteTarget` for the route handler.",
-  "deprecatedMessage": "",
-  "required": ["handler"],
-  "properties": {
-    "handler": {
-      "type": "object",
-      "default": {},
-      "required": ["export", "module", "options"],
-      "properties": {
-        "export": {
-          "const": "AIGatewayModelRoutingV2InboundPolicy",
-          "description": "The name of the exported type"
-        },
-        "module": {
-          "const": "$import(@zuplo/runtime)",
-          "description": "The module containing the policy"
-        },
-        "options": {
-          "type": "object",
-          "title": "AIGatewayModelRoutingV2InboundPolicyOptions",
-          "description": "Options for allowing or blocking providerName/model routes for each AI Gateway capability.",
-          "additionalProperties": false,
-          "required": ["models"],
-          "properties": {
-            "models": {
-              "type": "object",
-              "title": "Models",
-              "description": "Declarative model-routing rules, grouped by AI Gateway capability.",
-              "additionalProperties": false,
-              "minProperties": 1,
-              "properties": {
-                "completions": {
-                  "title": "Completions Models",
-                  "description": "Rules for chat completions, Responses, and Anthropic Messages requests.",
-                  "allOf": [
-                    {
-                      "title": "Capability Model Rules",
-                      "description": "Choose curated allow-list mode or open-but-filtered block-list mode.",
-                      "oneOf": [
-                        {
-                          "type": "object",
-                          "title": "Curated Model Rules",
-                          "description": "Allow only listed targets and use entry zero as the default.",
-                          "additionalProperties": false,
-                          "required": ["allowList"],
-                          "not": {
-                            "required": ["blockList"]
-                          },
-                          "properties": {
-                            "allowList": {
-                              "type": "array",
-                              "title": "Allowed Models",
-                              "description": "Curated targets clients may select; the first entry is the default when model is omitted.",
-                              "minItems": 1,
-                              "items": {
-                                "title": "Route Target",
-                                "description": "A providerName/model string shorthand or a full model route target.",
-                                "oneOf": [
-                                  {
-                                    "type": "string",
-                                    "title": "Provider and Model",
-                                    "description": "A model reference in providerName/model form, split on the first slash.",
-                                    "pattern": "^[^/\\s]+/.+$"
-                                  },
-                                  {
-                                    "type": "object",
-                                    "title": "Model Route Target",
-                                    "description": "A primary model and optional failure or quota fallbacks.",
-                                    "additionalProperties": false,
-                                    "required": ["main"],
-                                    "properties": {
-                                      "main": {
-                                        "type": "string",
-                                        "title": "Provider and Model",
-                                        "description": "A model reference in providerName/model form, split on the first slash.",
-                                        "pattern": "^[^/\\s]+/.+$"
-                                      },
-                                      "backup": {
-                                        "type": "string",
-                                        "title": "Provider and Model",
-                                        "description": "A model reference in providerName/model form, split on the first slash.",
-                                        "pattern": "^[^/\\s]+/.+$"
-                                      },
-                                      "fallbackTimeoutSeconds": {
-                                        "type": "number",
-                                        "title": "Fallback Timeout Seconds",
-                                        "description": "How long the main model may take before the backup is attempted.",
-                                        "minimum": 1,
-                                        "maximum": 300,
-                                        "default": 60
-                                      },
-                                      "quotaFallback": {
-                                        "type": "string",
-                                        "title": "Provider and Model",
-                                        "description": "A model reference in providerName/model form, split on the first slash.",
-                                        "pattern": "^[^/\\s]+/.+$"
-                                      }
-                                    }
-                                  }
-                                ]
-                              }
-                            }
-                          }
-                        },
-                        {
-                          "type": "object",
-                          "title": "Open-but-Filtered Model Rules",
-                          "description": "Allow any requested target except those explicitly blocked.",
-                          "additionalProperties": false,
-                          "required": ["blockList"],
-                          "not": {
-                            "required": ["allowList"]
-                          },
-                          "properties": {
-                            "blockList": {
-                              "type": "array",
-                              "title": "Blocked Models",
-                              "description": "Configured providerName/model targets rejected in open-but-filtered mode.",
-                              "minItems": 1,
-                              "items": {
-                                "type": "string",
-                                "title": "Provider and Model",
-                                "description": "A model reference in providerName/model form, split on the first slash.",
-                                "pattern": "^[^/\\s]+/.+$"
-                              }
-                            }
-                          }
-                        }
-                      ]
-                    }
-                  ]
-                },
-                "embeddings": {
-                  "title": "Embeddings Models",
-                  "description": "Rules for embedding requests.",
-                  "allOf": [
-                    {
-                      "title": "Capability Model Rules",
-                      "description": "Choose curated allow-list mode or open-but-filtered block-list mode.",
-                      "oneOf": [
-                        {
-                          "type": "object",
-                          "title": "Curated Model Rules",
-                          "description": "Allow only listed targets and use entry zero as the default.",
-                          "additionalProperties": false,
-                          "required": ["allowList"],
-                          "not": {
-                            "required": ["blockList"]
-                          },
-                          "properties": {
-                            "allowList": {
-                              "type": "array",
-                              "title": "Allowed Models",
-                              "description": "Curated targets clients may select; the first entry is the default when model is omitted.",
-                              "minItems": 1,
-                              "items": {
-                                "title": "Route Target",
-                                "description": "A providerName/model string shorthand or a full model route target.",
-                                "oneOf": [
-                                  {
-                                    "type": "string",
-                                    "title": "Provider and Model",
-                                    "description": "A model reference in providerName/model form, split on the first slash.",
-                                    "pattern": "^[^/\\s]+/.+$"
-                                  },
-                                  {
-                                    "type": "object",
-                                    "title": "Model Route Target",
-                                    "description": "A primary model and optional failure or quota fallbacks.",
-                                    "additionalProperties": false,
-                                    "required": ["main"],
-                                    "properties": {
-                                      "main": {
-                                        "type": "string",
-                                        "title": "Provider and Model",
-                                        "description": "A model reference in providerName/model form, split on the first slash.",
-                                        "pattern": "^[^/\\s]+/.+$"
-                                      },
-                                      "backup": {
-                                        "type": "string",
-                                        "title": "Provider and Model",
-                                        "description": "A model reference in providerName/model form, split on the first slash.",
-                                        "pattern": "^[^/\\s]+/.+$"
-                                      },
-                                      "fallbackTimeoutSeconds": {
-                                        "type": "number",
-                                        "title": "Fallback Timeout Seconds",
-                                        "description": "How long the main model may take before the backup is attempted.",
-                                        "minimum": 1,
-                                        "maximum": 300,
-                                        "default": 60
-                                      },
-                                      "quotaFallback": {
-                                        "type": "string",
-                                        "title": "Provider and Model",
-                                        "description": "A model reference in providerName/model form, split on the first slash.",
-                                        "pattern": "^[^/\\s]+/.+$"
-                                      }
-                                    }
-                                  }
-                                ]
-                              }
-                            }
-                          }
-                        },
-                        {
-                          "type": "object",
-                          "title": "Open-but-Filtered Model Rules",
-                          "description": "Allow any requested target except those explicitly blocked.",
-                          "additionalProperties": false,
-                          "required": ["blockList"],
-                          "not": {
-                            "required": ["allowList"]
-                          },
-                          "properties": {
-                            "blockList": {
-                              "type": "array",
-                              "title": "Blocked Models",
-                              "description": "Configured providerName/model targets rejected in open-but-filtered mode.",
-                              "minItems": 1,
-                              "items": {
-                                "type": "string",
-                                "title": "Provider and Model",
-                                "description": "A model reference in providerName/model form, split on the first slash.",
-                                "pattern": "^[^/\\s]+/.+$"
-                              }
-                            }
-                          }
-                        }
-                      ]
-                    }
-                  ]
-                }
-              }
-            }
-          }
-        }
-      },
-      "examples": [
-        {
-          "export": "AIGatewayModelRoutingV2InboundPolicy",
-          "module": "$import(@zuplo/runtime)",
-          "options": {
-            "models": {}
-          }
-        }
-      ]
-    }
-  }
-}