zuplo 7.7.8 → 7.7.9

package/docs/ai-gateway/azure-ai.mdx

@@ -0,0 +1,410 @@
+---
+title: Using Azure AI
+sidebar_label: Azure AI
+description:
+  Serve Azure OpenAI and Microsoft Foundry models through one provider
+  configuration. Map each deployment to the model it serves so the gateway
+  prices usage.
+---
+
+**Azure AI** serves models from your own Azure resource through one provider
+configuration—one resource endpoint and one resource key. Adding it gives your
+[apps](./apps.mdx) the models you have deployed, through the
+[Universal API](./universal-api.mdx), on your Azure subscription and Azure
+billing.
+
+One provider type covers both kinds of Azure resource:
+
+- An **Azure OpenAI** resource deploys OpenAI models and serves them on an
+  OpenAI-compatible API.
+- A **Microsoft Foundry** resource serves that same API for every Foundry model
+  family—Grok, DeepSeek, Llama, Mistral, Phi, Kimi and more—_and_ serves Claude
+  on the native Anthropic Messages API.
+
+Apps reference models as `providerName/model`, where `providerName` is the name
+you give the provider configuration. A provider named `azureai` serves
+`azureai/my-gpt`.
+
+## Azure serves deployments, not model names
+
+Of the ways Azure differs from other providers, this is the one that changes
+what you put in the `model` field.
+
+Other providers accept a published model ID. Azure resolves the `model` field
+against the **deployments** in your resource, and you choose deployment names
+when you create them. If you deploy `gpt-4.1-mini` under the name `my-gpt`, then
+`my-gpt` is the only name Azure accepts for it.
+
+The gateway forwards your deployment name to Azure unchanged, because it's the
+only address Azure answers to. What the gateway needs to know separately is
+which catalog model each deployment serves—that's what prices your usage.
+
+:::caution{title="An unmapped deployment is rejected"}
+
+The gateway prices a request by looking up the model name in its catalog. A
+deployment named after something outside the catalog, such as `my-gpt`, matches
+nothing—so the gateway returns **400** rather than serving a request it can't
+bill. Azure would have served it, and the usage would have recorded no cost.
+
+Map every deployment whose name differs from the model it serves. You do this
+when you add or edit the provider. The error names the deployment and lists the
+names that do work.
+
+:::
+
+Mapping is optional when it isn't needed. Deploy `gpt-4.1-mini` under the name
+`gpt-4.1-mini`—the Azure portal's own default—and it already matches the
+catalog.
+
+### Deployment name rules
+
+Azure accepts 2 to 64 characters of letters, digits, hyphens, underscores and
+periods, and won't accept a name ending in a period. Leading hyphens and
+periods, doubled periods, trailing hyphens and uppercase letters are all fine.
+
+Deployment names match case-insensitively, so a deployment named `MyGpt` answers
+to `mygpt`, and one named `my-gpt` answers to `MY-GPT`. Map each deployment once
+using whichever spelling you prefer; requests in any casing resolve to it.
+
+## How the gateway routes Azure AI models
+
+A Foundry resource serves two API formats on the same host with the same
+resource key:
+
+- An **OpenAI-compatible API** at `/openai/v1`, serving every non-Claude model.
+- The **native Anthropic Messages API** at `/anthropic/v1/messages`, serving the
+  Claude models.
+
+<Diagram height="h-64">
+  <DiagramNode id="app">Your app</DiagramNode>
+  <DiagramNode id="gateway" variant="zuplo">
+    AI Gateway
+  </DiagramNode>
+  <DiagramGroup id="azure" label="Your Azure resource">
+    <DiagramNode id="openai-surface" variant="blue">
+      OpenAI-compatible API
+    </DiagramNode>
+    <DiagramNode id="messages-surface" variant="green">
+      Anthropic Messages API
+    </DiagramNode>
+  </DiagramGroup>
+  <DiagramEdge from="app" to="gateway" label="Universal API" />
+  <DiagramEdge
+    from="gateway"
+    to="openai-surface"
+    label="OpenAI-compatible models"
+  />
+  <DiagramEdge from="gateway" to="messages-surface" label="Claude models" />
+</Diagram>
+
+The gateway's model catalog records which API serves each model, and routes
+every request accordingly. Your clients always call your app's URL and never see
+the Azure endpoint.
+
+Claude needs a Foundry resource, because an Azure OpenAI resource can't deploy
+Claude at all. Which endpoint host you configure doesn't affect this—a Foundry
+resource serves Claude on either host family the provider accepts.
+
+## Supported endpoints
+
+| Endpoint               | OpenAI-compatible models | Claude models |
+| ---------------------- | ------------------------ | ------------- |
+| `/v1/chat/completions` | ✅ Forwarded             | ✅ Translated |
+| `/v1/embeddings`       | ✅ Embedding models      | ❌            |
+| `/v1/messages`         | ❌                       | ✅ Native     |
+| `/v1/responses`        | ❌                       | ❌            |
+
+Streaming (`stream: true`) works on chat completions and messages.
+
+The catalog carries three embedding models—`text-embedding-3-small`,
+`text-embedding-3-large` and `text-embedding-ada-002`—so `/v1/embeddings` works
+once you deploy one of them and map your deployment to it.
+
+`/v1/responses` isn't served for Azure AI, and a Responses request fails with a
+`400` error before any upstream call. Azure does serve a Responses API on its
+own endpoint; the gateway doesn't route to it yet. Use `/v1/chat/completions`
+instead.
+
+Claude models work on `/v1/chat/completions` through the gateway's translation
+to the Messages API, which supports the core chat parameters: `messages`,
+`max_tokens`, `temperature`, `top_p`, `stop`, and `stream`. For tool use or
+other Anthropic-specific features, call
+[`/v1/messages`](#call-claude-models-on-the-messages-api) instead.
+
+This is the gateway's standard handling for Claude, not something specific to
+Azure—[Bedrock Mantle](./bedrock-mantle.mdx#supported-endpoints-by-model-family)
+serves Claude the same way. You get an ordinary `chat.completion` object back,
+with `choices` and `prompt_tokens`/`completion_tokens`. The one visible trace of
+the conversion is the response `id`, which keeps Anthropic's `msg_` prefix.
+Azure's OpenAI-compatible models are forwarded instead of translated, so their
+responses carry Azure's own fields, such as `content_filter_results`.
+
+## Before you begin
+
+You need:
+
+- An Azure subscription with an Azure OpenAI or Microsoft Foundry resource. Note
+  the resource name—it's the first label of the resource's endpoint host.
+- **At least one model deployed in that resource**, and its deployment name. The
+  gateway addresses deployments, so a resource with no deployments serves
+  nothing. Deploy models in the Azure portal under your resource's **Model
+  deployments**.
+- One of the resource's API keys, from the resource's **Keys and Endpoint** page
+  in the Azure portal.
+- An AI Gateway project in the Zuplo Portal.
+- An AI Gateway [app](./apps.mdx) to call the models from. The app page shows
+  the app's API URL, and its API key lives on the app's **API Key** tab.
+
+:::note
+
+Use a resource key, not a Microsoft Entra ID token. Azure accepts both on its
+own API, but the gateway stores long-lived resource keys, and the provider
+dialog rejects a pasted Entra token—those expire within hours.
+
+:::
+
+### Deploying Claude on Foundry
+
+Claude deployments carry extra requirements that Azure applies only to them, and
+the failures are easy to misread:
+
+- Azure asks for your **industry, organization name and country** when you
+  create an Anthropic-format deployment. Give a plain organization name;
+  punctuation in it has been enough to fail the deployment.
+- Azure validates none of it up front. A deployment can report success, spend a
+  few minutes provisioning, and then land in a **Failed** state with an
+  internal-error message. Check the deployment's provisioning state before you
+  configure the provider, and delete a failed deployment before reusing its
+  name.
+- If you deploy through the Azure REST API or a template rather than the portal,
+  use API version `2025-12-01` or later. Earlier versions silently ignore those
+  fields and then reject the request for not providing them.
+
+## Add the provider
+
+Adding or editing providers requires the **Edit** permission, granted to Zuplo
+account and project **Admins**—see
+[Managing Providers](./managing-providers.mdx).
+
+<Stepper>
+
+1. Open
+   [**Settings → AI Providers**](https://portal.zuplo.com/+/account/project/ai/settings/data-models)
+   in your AI Gateway project in the Zuplo Portal.
+
+1. Click the **Add Provider** button.
+
+1. In the **AI Provider** list, select **Azure AI** from the Default Providers
+   group.
+
+1. Review the **Provider Name**, which fills in as `azureai` (a second
+   configuration becomes `azureai-2`). You can replace it with your own name,
+   but only now—the name is permanent after creation, and it's the prefix in
+   every model reference: a provider named `azureai` serves `azureai/my-gpt`.
+
+1. In **Azure Resource Name**, enter your resource's name, such as
+   `my-resource`. Use the host family selector beside the field to pick
+   `openai.azure.com` or `services.ai.azure.com`, matching your resource's
+   endpoint. The gateway sends this provider's requests to
+   `https://<resource>.<host family>`.
+
+1. In **API Key**, paste one of the resource's keys.
+
+1. Select the models to enable, or click **Select All**. The picker shows
+   per-token prices, which the gateway uses to track cost per app. You can
+   change the selection later.
+
+1. Under **Deployment Names**, click **Add deployment** for each deployment
+   whose name differs from the model it serves. Enter the deployment name, then
+   pick the model it serves from the list. Skip this for deployments already
+   named after their model.
+
+1. Click **Create**.
+
+</Stepper>
+
+:::note
+
+Saving provider settings triggers an automatic production deployment of your
+gateway, because provider credentials are part of the deployed gateway. The
+change is live once the deployment completes.
+
+:::
+
+When you edit the provider later—see
+[Managing Providers](./managing-providers.mdx)—the **Azure Resource Name**, host
+family and deployment mappings all stay editable, and you can replace the API
+key. The **Provider Name** doesn't change.
+
+:::caution{title="Which endpoint form to enter"}
+
+Enter the resource name and pick a host family; don't paste a full URL. The
+provider accepts the `openai.azure.com` and `services.ai.azure.com` families
+only.
+
+A Foundry resource also has a `cognitiveservices.azure.com` address, and that's
+the one the Azure portal shows as the resource's endpoint. It isn't accepted
+here—use the resource name with one of the two families above instead. The
+resource is the same either way.
+
+:::
+
+## Verify the provider
+
+Once the deployment completes, send a chat completions request to your app's
+[Universal API](./universal-api.mdx) URL—shown at the top of the
+[app page](./apps.mdx)—with the app's API key as the bearer token and one of
+your deployments:
+
+```bash
+curl https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1/chat/completions \
+  -H "Authorization: Bearer $ZUPLO_APP_API_KEY" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "azureai/my-gpt",
+    "messages": [{ "role": "user", "content": "Say hi" }]
+  }'
+```
+
+The URL is a sample—replace it with your app's API URL plus
+`/v1/chat/completions`, set `ZUPLO_APP_API_KEY` to the app's API key, and
+substitute your own deployment name for `my-gpt`.
+
+A `200` response confirms the provider works. If the request fails immediately
+after you save the provider, the deployment may not have finished—retry before
+debugging further.
+
+## Call OpenAI-compatible models
+
+Use any OpenAI client with your app's URL plus `/v1` as the base URL and the
+app's API key:
+
+```ts
+import OpenAI from "openai";
+
+const client = new OpenAI({
+  apiKey: process.env.ZUPLO_APP_API_KEY,
+  baseURL:
+    "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1",
+});
+
+const response = await client.chat.completions.create({
+  model: "azureai/my-gpt",
+  messages: [{ role: "user", content: "Summarize this ticket." }],
+});
+```
+
+The same call works with a Claude deployment—the gateway translates it to the
+Messages API—within the [translation's parameter subset](#supported-endpoints).
+
+## Call Claude models on the Messages API
+
+Claude deployments on a Foundry resource serve the native
+[Anthropic Messages API](./universal-api.mdx#supported-endpoints) at
+`/v1/messages`. With the Anthropic SDK, set `baseURL` to the app's URL _without_
+`/v1`—the SDK appends `/v1/messages` itself—and pass the app's API key as
+`authToken`, not `apiKey`:
+
+```ts
+import Anthropic from "@anthropic-ai/sdk";
+
+const client = new Anthropic({
+  baseURL:
+    "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e",
+  authToken: process.env.ZUPLO_APP_API_KEY, // apiKey would send x-api-key, which the gateway ignores
+});
+
+const message = await client.messages.create({
+  model: "azureai/my-claude",
+  max_tokens: 1024,
+  messages: [{ role: "user", content: "Say hi" }],
+});
+```
+
+The gateway forwards the request body to Azure verbatim, so tool use, system
+prompts and streaming work as they do against Anthropic directly.
+
+`anthropic-beta` headers are the exception. The gateway forwards the
+long-context and prompt-caching betas and drops every other value, because Azure
+rejects an unrecognized beta by failing the whole request. A client asking for a
+different beta still gets an answer, without that feature.
+
+## What the model field reports in responses
+
+The `model` field in a response doesn't always say what you might expect, and
+what it says depends on the model you called.
+
+**Claude responses report your deployment name.** Every other response reports
+Azure's own model ID, which usually carries a version date the short name
+doesn't.
+
+| Endpoint               | Deployment `my-gpt` / `my-embed` / `my-claude` serves | `model` in the response   |
+| ---------------------- | ----------------------------------------------------- | ------------------------- |
+| `/v1/chat/completions` | `gpt-4.1-mini`                                        | `gpt-4.1-mini-2025-04-14` |
+| `/v1/embeddings`       | `text-embedding-3-small`                              | `text-embedding-3-small`  |
+| `/v1/messages`         | `claude-haiku-4-5`                                    | `my-claude`               |
+| `/v1/chat/completions` | `claude-haiku-4-5`                                    | `my-claude`               |
+
+A Claude model reports your deployment name on both endpoints—the native
+Messages API and the chat completions translation—because the gateway builds
+those responses itself and echoes back the model you asked for.
+
+One exception, if you stream Claude: the `message_start` event reports Azure's
+ID (`claude-haiku-4-5-20251001`) rather than your deployment name, because the
+gateway forwards each event as it arrives instead of rebuilding the response. So
+the same request reports two different values depending on whether you streamed
+it.
+
+Treat this field as informational either way. If you need to know which
+deployment served a request, use your own request metadata rather than parsing
+this.
+
+## Troubleshooting
+
+**A request fails with a deployment-not-found error.** The `model` value after
+the provider prefix must be a deployment name in your Azure resource, not a
+published model ID. Check the deployment list in the Azure portal under your
+resource's **Model deployments**, and confirm the deployment finished
+provisioning.
+
+**A request fails saying the deployment isn't mapped.** The deployment exists in
+Azure, but the gateway can't tell which model it serves, so it can't price the
+request. Edit the provider, add the deployment under **Deployment Names**, and
+pick its model. If the deployment is named after its model, selecting that model
+is enough. The error lists the names that currently work.
+
+**The dialog rejects your endpoint.** Enter the resource name, not a URL, and
+pick `openai.azure.com` or `services.ai.azure.com`. A Foundry resource's
+`cognitiveservices.azure.com` address isn't accepted—use the resource name with
+one of those two families. Resource names are 2 to 63 characters of lowercase
+letters, digits and hyphens, starting and ending with a letter or digit.
+
+**The dialog rejects your API key.** The gateway takes a resource key from the
+resource's **Keys and Endpoint** page. A Microsoft Entra ID access token is
+rejected—it expires within hours, so the connection would break the same day.
+
+**A Claude model returns an error on an Azure OpenAI resource.** Only Foundry
+resources deploy Claude. Confirm the deployment exists and is Anthropic-format,
+and see [Deploying Claude on Foundry](#deploying-claude-on-foundry) for the
+requirements Azure applies to those deployments.
+
+**A `400` error names `/v1/responses`.** The gateway doesn't serve the Responses
+API for Azure AI. Use `/v1/chat/completions`.
+
+**A model in the picker fails at request time.** The picker lists the full Azure
+catalog, but a model only works once you deploy it in your resource, and model
+availability varies by region and resource kind. Deploy the model, then map your
+deployment name to it.
+
+## Next steps
+
+- [AI Providers](./providers.mdx)—the capability matrix across every supported
+  provider.
+- [Universal API](./universal-api.mdx)—the endpoints every app serves and how
+  model references work.
+- [Managing Providers](./managing-providers.mdx)—edit models, keys, and the
+  endpoint, and understand when changes deploy.
+- [AI Gateway Apps](./apps.mdx)—create the apps that call your Azure-backed
+  models.
+- [Model Filtering policy](../policies/ai-gateway-model-filtering-v2-inbound.mdx)—control
+  which models each app can call.

package/docs/ai-gateway/managing-providers.mdx

@@ -53,9 +53,12 @@ To add a new AI provider to your Zuplo AI Gateway, follow these steps:
    [Zuplo Demo provider](./providers.mdx#zuplo-demo) asks for no API key—your
    gateway authenticates to the demo service itself,
    [Bedrock Mantle](./bedrock-mantle.mdx) additionally asks for an **AWS
-   Region** and accepts only long-term Bedrock API keys, and
+   Region** and accepts only long-term Bedrock API keys,
    [Vertex AI](./vertex-ai.mdx) asks for a **Location** and a **Google Cloud
-   Project ID** and takes a service account JSON key file instead of an API key.
+   Project ID** and takes a service account JSON key file instead of an API key,
+   and [Azure AI](./azure-ai.mdx) asks for an **Azure Resource Name** and a host
+   family, plus a mapping for any deployment named differently from the model it
+   serves.
 
 1. Select the model or models you want to use with this provider. The available
    models will depend on the selected provider. This can be changed later.
@@ -85,10 +88,11 @@ To modify an existing provider, open
 and click the **Edit** button next to the provider you want to modify.
 
 You can modify the API key and selected models for the provider—and, for
-[Bedrock Mantle](./bedrock-mantle.mdx), the AWS Region, or for
-[Vertex AI](./vertex-ai.mdx), the Location and Google Cloud Project ID. The
-**Provider Name** isn't editable: it's the routing address in every
-`providerName/model` reference, so renaming it would orphan each stored
+[Bedrock Mantle](./bedrock-mantle.mdx), the AWS Region, for
+[Vertex AI](./vertex-ai.mdx), the Location and Google Cloud Project ID, or for
+[Azure AI](./azure-ai.mdx), the Azure Resource Name, host family, and deployment
+mappings. The **Provider Name** isn't editable: it's the routing address in
+every `providerName/model` reference, so renaming it would orphan each stored
 reference to this provider. After making your changes, click **Save** to apply
 them.
 

package/docs/ai-gateway/overview.mdx

@@ -75,8 +75,9 @@ usage limits still apply.
 ### Multi-Provider Support
 
 Configure multiple LLM providers within a single AI Gateway project. Supported
-providers include OpenAI, Anthropic, Google, Mistral, xAI, Amazon Bedrock
-(through [Bedrock Mantle](./bedrock-mantle.mdx)), Google Cloud
+providers include OpenAI, Anthropic, Google, Mistral, xAI, Microsoft Azure
+(through [Azure AI](./azure-ai.mdx)), Amazon Bedrock (through
+[Bedrock Mantle](./bedrock-mantle.mdx)), Google Cloud
 ([Vertex AI](./vertex-ai.mdx)), and OpenAI-compatible custom providers. See
 [AI Providers](./providers.mdx) for the full list of providers and supported
 capabilities. Apps reference models as `providerName/model`—for example

package/docs/ai-gateway/providers.mdx

@@ -19,6 +19,8 @@ Zuplo currently supports the following AI providers:
 - Google
 - Mistral
 - xAI (Grok)
+- [Azure AI](./azure-ai.mdx)—Azure OpenAI and Microsoft Foundry resources,
+  serving Claude and every Foundry model family from your own Azure subscription
 - [Bedrock Mantle](./bedrock-mantle.mdx)—Amazon Bedrock's compatible-APIs
   endpoint, serving Claude models and models from many other vendors
 - [Vertex AI](./vertex-ai.mdx)—Google Cloud's managed model platform, serving
@@ -36,6 +38,7 @@ The following capabilities are supported across providers:
 | Google                     | ✅               | ✅         | ❌        | ❌       |
 | Mistral                    | ✅               | ✅         | ❌        | ❌       |
 | xAI                        | ✅               | ✅         | ❌        | ❌       |
+| Azure AI                   | ✅               | ✅         | ❌        | ✅       |
 | Bedrock Mantle             | ✅               | ❌         | ✅        | ✅       |
 | Vertex AI                  | ✅               | ✅         | ❌        | ✅       |
 | Zuplo Demo                 | ✅               | ❌         | ❌        | ❌       |
@@ -51,6 +54,12 @@ Messages (plus chat completions through translation), while its other models
 serve chat completions and—per model—Responses. See
 [Using Bedrock Mantle](./bedrock-mantle.mdx#supported-endpoints-by-model-family).
 
+Azure AI's Messages support needs a Microsoft Foundry resource, since Claude
+can't be deployed on an Azure OpenAI resource. Azure also addresses models by
+**deployment name** rather than by published model ID, so a deployment named
+differently from the model it serves needs mapping for the gateway to price it.
+See [Using Azure AI](./azure-ai.mdx#azure-serves-deployments-not-model-names).
+
 A custom provider must serve chat completions and embeddings under a `/v1` path
 segment on its API URL, and you enter that URL as an origin root—without the
 `/v1` suffix vendors usually publish. See
@@ -61,6 +70,32 @@ Apps reference a provider's models as `providerName/model`—for example
 name you give the provider configuration. See the
 [Universal API](./universal-api.mdx).
 
+## Azure AI
+
+**Azure AI** serves models from your own Azure resource. One provider
+configuration covers both kinds of resource, and which capabilities apply
+depends on the resource and the model:
+
+- An **Azure OpenAI** resource deploys OpenAI models and serves chat completions
+  and embeddings.
+- A **Microsoft Foundry** resource serves those plus every Foundry model
+  family—Grok, DeepSeek, Llama, Mistral, Phi, Kimi and more—and serves **Claude
+  models** on the native Anthropic Messages API, with chat completions through
+  the gateway's translation.
+
+Two things make Azure different from every other provider:
+
+- **It addresses deployments, not published model IDs.** You name deployments
+  when you create them in Azure, and that name is what goes in the `model`
+  field. A deployment whose name differs from the model it serves needs a
+  mapping, or the gateway can't price its usage.
+- **Its endpoint host embeds your resource name.** The provider dialog asks for
+  an **Azure Resource Name** and a host family rather than a full URL, and it
+  takes a resource key—not a Microsoft Entra ID token.
+
+For prerequisites, setup steps, code examples, and troubleshooting, see
+[Using Azure AI](./azure-ai.mdx).
+
 ## Bedrock Mantle
 
 **Bedrock Mantle** is Amazon Bedrock's compatible-APIs endpoint. One regional

package/docs/ai-gateway/universal-api.mdx

@@ -63,9 +63,9 @@ list—so clients that can't set a model still work.
 
 ## Supported endpoints
 
-| Endpoint               | Notes                                                                                                                               |
-| ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
-| `/v1/chat/completions` | Chat completions for every provider                                                                                                 |
-| `/v1/embeddings`       | Embeddings for every provider except Anthropic                                                                                      |
-| `/v1/responses`        | OpenAI Responses API: OpenAI, and [Bedrock Mantle](./bedrock-mantle.mdx) OpenAI-compatible models that serve it                     |
-| `/v1/messages`         | Anthropic Messages API: Anthropic, and the Claude models of [Bedrock Mantle](./bedrock-mantle.mdx) and [Vertex AI](./vertex-ai.mdx) |
+| Endpoint               | Notes                                                                                                                                                           |
+| ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `/v1/chat/completions` | Chat completions for every provider                                                                                                                             |
+| `/v1/embeddings`       | Embeddings for the providers marked in the [capability matrix](./providers.mdx#supported-providers)                                                             |
+| `/v1/responses`        | OpenAI Responses API: OpenAI, and [Bedrock Mantle](./bedrock-mantle.mdx) OpenAI-compatible models that serve it                                                 |
+| `/v1/messages`         | Anthropic Messages API: Anthropic, and the Claude models of [Azure AI](./azure-ai.mdx), [Bedrock Mantle](./bedrock-mantle.mdx) and [Vertex AI](./vertex-ai.mdx) |

package/docs/policies/_index.md

@@ -4,7 +4,7 @@
 | --- | --- | --- | --- |
 | set-query-params-inbound | Add or Set Query Parameters | Adds or sets query parameters on the incoming request. | api-gateway |
 | set-headers-inbound | Add or Set Request Headers | Adds or sets headers on the incoming request. | api-gateway |
-| ai-gateway-auth-v2-inbound | AI Gateway Authentication | Authenticates requests to an AI Gateway endpoint with application API keys.  Add this policy to an application's `inboundPolicyChain` to require a key for that app only, or place it on the route before the configuration executor to require a key for every application on the route. The policies that follow can read the authenticated application from `request.user` (`sub` is the application name, `data` its metadata), and the application's AI Gateway configuration takes effect for the request. Use `authHeader` and `authScheme` when clients send their app key somewhere other than the default `Authorization: Bearer` header.  When the matched route captures an `app_id` path parameter (platform catch-all `/:app_id/(.*)`), this policy also requires `configuration.id === request.params.app_id` and returns 403 on mismatch. | ai-gateway |
+| ai-gateway-auth-v2-inbound | AI Gateway Authentication | Authenticates requests to an AI Gateway endpoint with application API keys.  Add this policy to an application's `inboundPolicyChain` to require a key for that app only, or place it on the route before the configuration executor to require a key for every application on the route. The policies that follow can read the authenticated application from `request.user` (`sub` is the application name, `data` its metadata), and the application's AI Gateway configuration takes effect for the request. Use `authHeader` and `authScheme` when clients send their app key somewhere other than the default `Authorization: Bearer` header. Enable `credentialPassthrough`, with the application key in a separate header, to forward the caller's `Authorization` header to the primary provider as its credential.  When the matched route captures an `app_id` path parameter (platform catch-all `/:app_id/(.*)`), this policy also requires `configuration.id === request.params.app_id` and returns 403 on mismatch. | ai-gateway |
 | ai-gateway-configuration-executor-v2-inbound | AI Gateway Configuration Executor | Loads the app configuration for the request (when auth or the configuration loader has not already), runs the inbound policy chain from that configuration, and enforces limits inherited from parent teams or the gateway root.  Place this policy on AI Gateway routes after optional authentication and optional `ai-gateway-configuration-loader-v2-inbound`. When either of those already populated the app-configuration channel, this policy reuses it. Otherwise it loads the configuration with the route's `app_id` path parameter. Applications select from policies pre-declared by the gateway. Applications without a `inboundPolicyChain`, or with an empty chain, run no application-selected policies. Entry options replace the declaration's options as a complete object; omit them to inherit the declaration, including environment-backed credentials. Each occurrence receives a private deep copy of its entry options, so a policy mutating its options cannot corrupt the cached app configuration. | ai-gateway |
 | ai-gateway-configuration-loader-v2-inbound | AI Gateway Configuration Loader | Loads the AI Gateway app configuration for the request into the request-scoped channel and does nothing else.  Place this policy on AI Gateway routes before `ai-gateway-configuration-executor-v2-inbound` when you want configuration loading separated from chain execution. When `ai-gateway-auth-v2-inbound` already populated the channel, this policy reuses it. Otherwise it loads the configuration with the route's `app_id` path parameter.  If this policy is omitted, the configuration executor still loads configuration itself before running the application chain. | ai-gateway |
 | ai-gateway-fallback-model-v2-inbound | AI Gateway Fallback Model | Adds failure and quota fallbacks to an existing AI Gateway model selection.  Place this policy after AI Gateway Model Filtering. It never creates a model selection, so a misplaced policy cannot bypass filtering. | ai-gateway |

package/docs/policies/ai-gateway-auth-v2-inbound/doc.md

@@ -147,6 +147,64 @@ curl https://gateway.example.com/v1/chat/completions \
   }'
 ```
 
+## Pass through a caller's provider credential
+
+Set `credentialPassthrough` to `true` and configure a separate header for the
+Zuplo application key. For a bare key in `zp-gateway-api-key`, use these policy
+options:
+
+```json
+{
+  "credentialPassthrough": true,
+  "authHeader": "zp-gateway-api-key",
+  "authScheme": ""
+}
+```
+
+Send the provider credential in `Authorization`. `authHeader` and `authScheme`
+remain authoritative; sending a special header does not enable passthrough or
+override your configuration. The defaults remain `Authorization` and `Bearer`,
+so enabling passthrough without configuring a separate header is a configuration
+error.
+
+For example:
+
+```bash
+curl https://gateway.example.com/v1/chat/completions \
+  --header "zp-gateway-api-key: YOUR_ZUPLO_APP_KEY" \
+  --header "Authorization: Bearer YOUR_PROVIDER_CREDENTIAL" \
+  --header "Content-Type: application/json" \
+  --data '{
+    "model": "provider/model",
+    "messages": [{ "role": "user", "content": "Hello" }]
+  }'
+```
+
+After the Zuplo application key succeeds, the policy moves the provider
+credential into private request-scoped state and removes both authentication
+headers from the mutable inbound request. The primary provider receives the
+original `Authorization` value exactly as supplied, instead of its configured
+credential. Retries of the primary use that same caller credential. Error and
+quota fallbacks use their own configured credentials, never the caller's token.
+Fallbacks therefore require configured credentials and can incur charges on the
+gateway owner's provider account. On native create requests at `/v1/messages`
+and `/v1/responses`, a compatible, explicitly configured backup runs only after
+the primary exhausts upstream 429 retries. It uses the backup's configured
+credential while preserving the native response and stream format; these
+endpoints do not use timeout, transport-error, or 5xx fallback. Semantic-cache
+policies skip passthrough requests so a cache hit cannot bypass upstream
+authentication. ZuploDemo is excluded because it must continue to use the
+gateway's deployment credential.
+
+A missing, empty, or invalid dedicated key fails authentication; the gateway
+never retries with `Authorization` as the Zuplo key. A missing or empty
+`Authorization` also returns 401 in this mode.
+
+With `credentialPassthrough` omitted or `false`, existing authentication
+applies: `authHeader` and `authScheme` select the Zuplo key, and providers use
+their configured credentials. Moving the gateway key to an arbitrary custom
+header alone does not enable passthrough.
+
 ## Choose a cache duration
 
 `cacheTtlSeconds` controls how long an authentication result can be reused. The
@@ -163,6 +221,10 @@ A revoked key can continue to work until its cached result expires.
 - `cacheTtlSeconds`: Number of seconds to cache an authentication result.
   Defaults to `10` and must be at least `10`.
 - `authHeader`: Header containing the application key. Defaults to
-  `Authorization`.
-- `authScheme`: Scheme before the key. Defaults to `Bearer`. Use `""` for a
-  header containing the key without a scheme.
+  `Authorization`. With `credentialPassthrough`, use a different header, such as
+  `zp-gateway-api-key`.
+- `authScheme`: Scheme before the key. Defaults to `Bearer`. Use `""` when the
+  header contains only the key.
+- `credentialPassthrough`: Forward the caller's `Authorization` header to the
+  primary provider as its credential while Zuplo reads the application key from
+  `authHeader`. Defaults to `false`.

package/docs/policies/ai-gateway-auth-v2-inbound/schema.json

@@ -12,7 +12,7 @@
   "requiresAI": true,
   "policyType": "ai-gateway-auth-v2",
   "products": ["ai-gateway"],
-  "description": "Authenticates requests to an AI Gateway endpoint with application API keys.\n\nAdd this policy to an application's `inboundPolicyChain` to require a key for that app only, or place it on the route before the configuration executor to require a key for every application on the route. The policies that follow can read the authenticated application from `request.user` (`sub` is the application name, `data` its metadata), and the application's AI Gateway configuration takes effect for the request. Use `authHeader` and `authScheme` when clients send their app key somewhere other than the default `Authorization: Bearer` header.\n\nWhen the matched route captures an `app_id` path parameter (platform catch-all `/:app_id/(.*)`), this policy also requires `configuration.id === request.params.app_id` and returns 403 on mismatch.",
+  "description": "Authenticates requests to an AI Gateway endpoint with application API keys.\n\nAdd this policy to an application's `inboundPolicyChain` to require a key for that app only, or place it on the route before the configuration executor to require a key for every application on the route. The policies that follow can read the authenticated application from `request.user` (`sub` is the application name, `data` its metadata), and the application's AI Gateway configuration takes effect for the request. Use `authHeader` and `authScheme` when clients send their app key somewhere other than the default `Authorization: Bearer` header. Enable `credentialPassthrough`, with the application key in a separate header, to forward the caller's `Authorization` header to the primary provider as its credential.\n\nWhen the matched route captures an `app_id` path parameter (platform catch-all `/:app_id/(.*)`), this policy also requires `configuration.id === request.params.app_id` and returns 403 on mismatch.",
   "deprecatedMessage": "",
   "required": ["handler"],
   "properties": {
@@ -36,6 +36,25 @@
           "description": "The options for this policy.",
           "additionalProperties": false,
           "required": [],
+          "if": {
+            "required": ["credentialPassthrough"],
+            "properties": {
+              "credentialPassthrough": {
+                "const": true
+              }
+            }
+          },
+          "then": {
+            "required": ["authHeader"],
+            "properties": {
+              "authHeader": {
+                "minLength": 1,
+                "not": {
+                  "pattern": "^\\s*[Aa][Uu][Tt][Hh][Oo][Rr][Ii][Zz][Aa][Tt][Ii][Oo][Nn]\\s*$"
+                }
+              }
+            }
+          },
           "properties": {
             "cacheTtlSeconds": {
               "type": "number",
@@ -48,14 +67,19 @@
               "default": "Authorization",
               "x-show-example": false,
               "x-advanced": true,
-              "description": "The name of the header with the key."
+              "description": "The request header that carries the Zuplo application key. When credentialPassthrough is enabled, use a header other than Authorization, such as zp-gateway-api-key."
             },
             "authScheme": {
               "type": "string",
               "default": "Bearer",
               "x-show-example": false,
               "x-advanced": true,
-              "description": "The scheme used on the header."
+              "description": "The scheme that prefixes the key in the authHeader header, for example Bearer. Set to an empty string when the header contains only the key."
+            },
+            "credentialPassthrough": {
+              "type": "boolean",
+              "default": false,
+              "description": "Forward the caller's Authorization header to the primary provider as its credential. Zuplo reads the application key from authHeader, which must be a header other than Authorization."
             }
           }
         }
@@ -65,7 +89,8 @@
           "export": "AIGatewayAuthV2InboundPolicy",
           "module": "$import(@zuplo/runtime)",
           "options": {
-            "cacheTtlSeconds": 10
+            "cacheTtlSeconds": 10,
+            "credentialPassthrough": false
           }
         }
       ]

package/docs/policies/ai-gateway-configuration-executor-v2-inbound/doc.md

@@ -27,13 +27,38 @@ The gateway owner remains in control:
 | `inboundPolicyChain` contains entries  | Entries run in the listed order      |
 | An entry has `enabled: false`          | That entry is skipped                |
 
-If a policy returns a response, that response is sent immediately and later
-entries do not run. If a chain is invalid, the request fails closed with an
-error that identifies the entry to fix.
+If a policy returns a response, later entries do not run. Successful 2xx
+responses still receive the budget check described below before they are sent.
+Other responses, including authentication failures, remain unchanged. If a chain
+is invalid, the request fails closed with an error that identifies the entry to
+fix.
 
 This inbound executor does not run `outboundPolicyChain`. That field is reserved
 for an outbound configuration executor on the response pipeline.
 
+## Automatic budget enforcement
+
+The executor enforces budgets configured on the application, its parent teams,
+and the gateway. This does not require a Metering policy and applies even when
+the application's policy chain is empty. An application cannot override a shared
+budget with a higher limit of its own.
+
+Request logs indicate when automatic budget checks run, including when the
+application has no Metering policy. Warnings identify exceeded budgets, blocked
+requests, fallback decisions, and checks that could not be completed.
+
+When a budget blocks a provider request, the gateway uses the configured quota
+fallback model or returns `429 Too Many Requests`. Successful responses from
+policies, including Semantic Cache hits, are also subject to budgets. A blocked
+response returns `429 Too Many Requests` without using a fallback model.
+Warning-only budgets do not block responses. Authentication failures and other
+non-2xx policy responses are preserved.
+
+Place Metering before Semantic Cache to count cache hits toward request limits.
+Budget changes can take time to take effect, and concurrent requests can exceed
+a budget before further requests are blocked. If a budget check is unavailable,
+requests can proceed unless a budget has already been determined to block them.
+
 ## Build an AI Gateway from scratch
 
 ### 1. Declare the policies
@@ -171,8 +196,7 @@ chain:
 }
 ```
 
-Routes that list only the executor keep working — the executor loads
-configuration when the channel is empty.
+If you omit the loader, the executor loads the application configuration.
 
 Authentication is optional and placement controls its scope:
 
@@ -205,14 +229,19 @@ An application can inherit the options from `policies.json`. Include
     {
       "name": "ai-gateway-metering-v2-inbound",
       "options": {
-        "limits": {
-          "requests": {
-            "daily": {
-              "enabled": true,
-              "limit": 1000
-            }
+        "budgetRules": [
+          {
+            "budgetBy": "app",
+            "meters": [
+              {
+                "meter": "requests",
+                "period": "daily",
+                "value": 1000,
+                "action": "block"
+              }
+            ]
           }
-        }
+        ]
       }
     },
     {
@@ -347,10 +376,10 @@ canonical forms when writing application configuration:
 | A key requiring brackets    | `request.user.data["team-id"]`              |
 
 The `expression` field contains the complete expression. Do not wrap it in an
-interpolation marker such as `${...}`. The runtime resolves it for every request
-after the application's inbound policy chain finishes. Updating an application's
-configuration changes which value later requests select without rebuilding or
-redeploying the gateway.
+interpolation marker such as `${...}`. Expressions use values from the request
+and context, including values set by earlier policies. Place policies that set
+these values before Semantic Cache. You can change an expression in the
+application configuration without rebuilding or redeploying the gateway.
 
 JSON encoding and expression syntax are separate. When writing raw JSON, escape
 the double quotes required by a bracket segment:
@@ -393,10 +422,8 @@ The selectable data model is:
 | `request` | `url`, `method`, `headers`, `user.sub`, `user.data`, `query`, `searchParams`, `params`, and the scalar Fetch request metadata `bodyUsed`, `cache`, `credentials`, `destination`, `integrity`, `keepalive`, `mode`, `redirect`, `referrer`, and `referrerPolicy` |
 | `context` | `contextId`, `requestId`, `custom`, `route`, and `incomingRequestProperties`                                                                                                                                                                                    |
 
-`request.body`, `context.log`, policy invocation methods, and every other host
-object are outside the selectable data model. The evaluator receives a
-plain-data snapshot of the selected root. It never receives the live request or
-context object.
+Expressions cannot read the request body, call logging or policy methods, or
+access properties outside this table.
 
 Expressions have these limits:
 
@@ -413,9 +440,8 @@ Expressions have these limits:
   It does not traverse arrays, class instances, inherited properties, or the
   property names `__proto__`, `constructor`, and `prototype`.
 
-Expressions observe values after the application's inbound policy chain
-finishes. An authentication policy can populate `request.user`. A custom policy
-earlier in the chain can derive a value and place it in a request header:
+An authentication policy can populate `request.user`. A custom policy earlier in
+the chain can derive a value and place it in a request header:
 
 ```ts
 import { ZuploContext, ZuploRequest } from "@zuplo/runtime";
@@ -446,14 +472,12 @@ value to well-formed NFC Unicode and ignores it when it exceeds 256 UTF-8 bytes
 or contains control characters, U+2028, or U+2029. A missing or invalid value
 does not contribute to the rule for that request.
 
-The exact stored expression is the rule identity and analytics dimension name.
-Store its authored bytes unchanged. Different accepted spellings, such as single
-quotes and double quotes, identify different rules even when they select the
-same value.
+Use consistent spelling for each expression. Different spellings, such as single
+quotes and double quotes, identify different budget rules even when they select
+the same value.
 
-Malformed budget rules are logged and ignored individually. An unsupported
-expression is also logged and omitted from dimension capture and quota tokens;
-other valid rules continue to run.
+Invalid budget rules and unsupported expressions are logged and skipped. Other
+valid rules continue to apply.
 
 ## Write a custom policy for the chain
 

package/docs/policies/ai-gateway-fallback-model-v2-inbound/doc.md

@@ -70,9 +70,12 @@ not override. A configured `fallback` replaces an existing backup and timeout; a
 configured `quotaFallback` replaces an existing quota fallback.
 
 Retry and timeout fallback is available for translated Chat Completions and
-Embeddings requests. Native `/v1/messages` and `/v1/responses` requests are
-passed through without retrying a backup. Quota fallback remains a separate,
-metering-driven path.
+Embeddings requests. Native create requests at `/v1/messages` and
+`/v1/responses` use a compatible completions backup only after the primary
+exhausts its upstream 429 retries; they do not inherit `fallbackTimeoutSeconds`
+or transport/5xx fallback. Responses management operations do not retry a backup
+because stored response IDs belong to the provider that created them. Quota
+fallback remains a separate, metering-driven path.
 
 ## Write your own fallback policy
 

package/docs/policies/ai-gateway-metering-v2-inbound/doc.md

@@ -5,13 +5,19 @@ budgets before the provider request runs. It meters spend, tokens, and requests.
 Budget rules can cover the whole application or each distinct value of an
 expression.
 
-When a limit is exceeded, the policy activates the model selection's
-`quotaFallback` when AI Gateway Fallback Model supplied one. Otherwise it
-returns `429 Too Many Requests`.
+Application, team, and gateway budgets are enforced even when this policy is not
+included in the application's policy chain.
 
-Place Metering after Model Filtering and Fallback Model so an exceeded budget
-can activate the quota fallback. Put policies that may answer early, such as
-Semantic Cache, after Metering so cache hits still count toward request limits.
+Request logs indicate when automatic budget checks run and when a budget blocks
+a request or selects a fallback model.
+
+When a budget blocks a provider request, the gateway uses the configured quota
+fallback model or returns `429 Too Many Requests`. Cached responses are also
+subject to budgets: a blocked cache hit returns `429 Too Many Requests` without
+using a fallback model.
+
+Place Metering after Model Filtering and Fallback Model. Place it before
+Semantic Cache so cache hits count toward request limits.
 
 ## Example
 
@@ -62,19 +68,26 @@ value of its expression. An action of `"warn"` notifies without blocking. An
 action of `"block"` activates the configured quota fallback or returns
 `429 Too Many Requests` when usage reaches the value.
 
-> **Budgets fail open by default.** When `throwOnFailure` is `false`, a metering
-> service failure lets the request proceed unmetered and no limit is checked.
-> Set it to `true` to reject the request instead.
-
 ## Team limits
 
-Budgets configured on this application govern only this app. Limits configured
-on a parent team or the gateway root are enforced centrally after the
-application's policy chain, whether or not this policy appears in that chain. An
-inherited limit activates the selected model's quota fallback when available and
-otherwise returns `429 Too Many Requests`.
+Application budgets apply only to that application. Parent team and gateway
+budgets apply across their applications, even when an application does not
+include Metering. An application cannot override a shared budget with a higher
+limit of its own.
+
+For example, two applications that spend $40 and $10 exhaust their team's $50
+monthly budget. An application outside that team does not share its budget.
+
+Team policy templates provide defaults for application policies. To set a shared
+team budget, configure the budget on the team itself.
+
+## Budget availability
+
+Budget changes can take time to take effect. Usage is recorded asynchronously,
+so concurrent requests can exceed a budget before further requests are blocked.
 
-An application cannot disable inherited enforcement through its policy chain. If
-the central hierarchical check is unavailable, the request proceeds. The
-policy's `throwOnFailure` option controls failures while checking or recording
-the app's own limits; it does not change inherited-limit behavior.
+If a budget check is unavailable, requests can proceed unless a budget has
+already been determined to block them. The `throwOnFailure` option controls
+whether this policy rejects requests when its metering operations fail; it does
+not change how application, team, or gateway `budgetRules` handle unavailable
+checks.

package/docs/policies/ai-gateway-model-filtering-v2-inbound/doc.md

@@ -253,3 +253,58 @@ Policy order determines precedence:
 
 Prefer one policy as the primary selector so the route's intent is easy to
 understand.
+
+## Model discovery
+
+Authenticated `GET /v1/models` and `GET /v1/models/{providerName/model}` use the
+same app URL as inference. Select the inference endpoint with the optional
+`x-zuplo-models-endpoint` request header:
+
+| Value              | Eligible models                                                                                 | Response format |
+| ------------------ | ----------------------------------------------------------------------------------------------- | --------------- |
+| `chat-completions` | Active completion models supported by Chat Completions adapters, including existing translation | OpenAI          |
+| `messages`         | Active completion models supported by native Messages passthrough                               | Anthropic       |
+| `responses`        | Active completion models supported by the resolved Responses adapter                            | OpenAI          |
+| `embeddings`       | Active embedding models supported by the resolved embedding adapter                             | OpenAI          |
+
+An explicit header wins. Otherwise, the presence of `anthropic-version` selects
+`messages`; other requests default to `chat-completions`. Empty, unknown, and
+multiple selector values return an actionable 400. The selector affects
+discovery only. Embedding discovery is opt-in. Messages never translates to Chat
+Completions.
+
+Discovery applies the first Model Filtering policy's parsed rules, matching
+inference's selection precedence. A capability omitted from that policy lists no
+models. Without Model Filtering, discovery lists the eligible catalog. Inactive
+models, unavailable adapters, and assignments without locally resolvable
+credentials are excluded. List and retrieve use identical eligibility rules;
+unknown, filtered-out, and wrong-endpoint IDs return the same 404.
+
+IDs retain the exact model-name casing; only the provider assignment label is
+normalized. OpenAI entries contain `id`, `object`, `created`, and `owned_by`.
+`owned_by` means the provider assignment namespace, not the model's developer.
+Anthropic entries contain `id`, `type`, `display_name`, and `created_at`, with
+unknown required nullable metadata set to `null`. Unknown creation dates use the
+epoch; optional unknown metadata is omitted.
+
+OpenAI lists return every eligible model and ignore pagination parameters.
+Anthropic lists default to 20 entries, accept `limit` from 1–1000, and accept
+either `after_id` or `before_id` from a previous page. Follow `has_more` and
+`last_id` for forward pagination. Invalid cursors return 400; restart without a
+cursor.
+
+Discovery uses the existing 60-second catalog cache. An uncached lookup,
+including legacy fallback, has a two-second budget; failure returns a
+sanitized 503. A successfully loaded catalog with no eligible models returns an
+empty 200. Responses use `Cache-Control: private, no-store` and vary on both
+selector headers. Only GET is supported. POST, PUT, PATCH, DELETE, and HEAD
+return 405 with `Allow: GET`. OPTIONS is handled by the gateway’s existing CORS
+preflight handler before app routing.
+
+Authentication and general abuse protections still apply. Discovery skips prompt
+inspection, semantic caching, and inference-budget blocking, and records one
+baseline request. Eligibility describes the catalog, adapters, and declared
+filtering rules; it does not guarantee upstream availability, provider
+acceptance of credentials, or acceptance by prompt-dependent custom policies.
+Existing Chat Completions translation does not promise full feature parity. Keep
+a qualified model configured when a client's picker does not consume discovery.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "zuplo",
-  "version": "7.7.8",
+  "version": "7.7.9",
   "type": "module",
   "description": "The official Zuplo CLI for local development and platform management",
   "homepage": "https://zuplo.com/docs/cli/overview",
@@ -32,9 +32,9 @@
     "zuplo": "zuplo.js"
   },
   "dependencies": {
-    "@zuplo/cli": "7.7.8",
-    "@zuplo/core": "7.7.8",
-    "@zuplo/runtime": "7.7.8",
-    "@zuplo/test": "7.7.8"
+    "@zuplo/cli": "7.7.9",
+    "@zuplo/core": "7.7.9",
+    "@zuplo/runtime": "7.7.9",
+    "@zuplo/test": "7.7.9"
   }
 }