zuplo 7.7.8 → 7.7.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/docs/ai-gateway/azure-ai.mdx +466 -0
- package/docs/ai-gateway/bedrock-mantle.mdx +5 -1
- package/docs/ai-gateway/integrations/claude-code.mdx +166 -11
- package/docs/ai-gateway/managing-providers.mdx +10 -6
- package/docs/ai-gateway/overview.mdx +3 -2
- package/docs/ai-gateway/providers.mdx +39 -0
- package/docs/ai-gateway/universal-api.mdx +6 -6
- package/docs/ai-gateway/vertex-ai.mdx +6 -1
- package/docs/policies/_index.md +1 -1
- package/docs/policies/ai-gateway-auth-v2-inbound/doc.md +65 -3
- package/docs/policies/ai-gateway-auth-v2-inbound/schema.json +29 -4
- package/docs/policies/ai-gateway-configuration-executor-v2-inbound/doc.md +54 -30
- package/docs/policies/ai-gateway-fallback-model-v2-inbound/doc.md +6 -3
- package/docs/policies/ai-gateway-metering-v2-inbound/doc.md +32 -19
- package/docs/policies/ai-gateway-model-filtering-v2-inbound/doc.md +55 -0
- package/package.json +5 -5
|
@@ -10,6 +10,17 @@ You can configure [Claude Code](https://www.claude.com/product/claude-code) to
|
|
|
10
10
|
send its model requests through the Zuplo AI Gateway, so the gateway
|
|
11
11
|
authenticates, meters, and routes every Claude Code session.
|
|
12
12
|
|
|
13
|
+
The gateway can authenticate to Anthropic in two ways. Pick one before you
|
|
14
|
+
configure Claude Code:
|
|
15
|
+
|
|
16
|
+
| Approach | The gateway sends Anthropic | Claude Code sends the gateway |
|
|
17
|
+
| --------------------------------------------------------- | --------------------------------------------------------------------- | ------------------------------------------------------------------------- |
|
|
18
|
+
| [Configured provider key](#use-a-configured-provider-key) | The Anthropic API key saved on your provider | The app's API key as `ANTHROPIC_AUTH_TOKEN` |
|
|
19
|
+
| [Your Claude subscription](#use-your-claude-subscription) | The credential from your `claude.ai` login, such as a Claude Max plan | The app's API key in a `zp-gateway-api-key` header, plus your login as-is |
|
|
20
|
+
|
|
21
|
+
Both approaches share the same provider, team, and app setup, and the same model
|
|
22
|
+
configuration.
|
|
23
|
+
|
|
13
24
|
## Claude Code Setup
|
|
14
25
|
|
|
15
26
|
<Stepper>
|
|
@@ -19,12 +30,18 @@ authenticates, meters, and routes every Claude Code session.
|
|
|
19
30
|
|
|
20
31
|
2. [Set up a new team](../managing-teams.mdx)
|
|
21
32
|
|
|
22
|
-
3.
|
|
33
|
+
3. If you plan to use your Claude subscription, configure the team's
|
|
34
|
+
[API Key Authentication policy](#configure-the-api-key-authentication-policy)
|
|
35
|
+
for passthrough before you create the app
|
|
36
|
+
|
|
37
|
+
4. Create a [new app](../managing-apps.mdx) to use with Claude Code and assign
|
|
23
38
|
it to the team you created
|
|
24
39
|
|
|
25
|
-
|
|
40
|
+
5. Copy the **API URL** and **API Key** shown at the top of the app page
|
|
26
41
|
|
|
27
|
-
|
|
42
|
+
6. Add the gateway URL, the app's API key, and model mappings to
|
|
43
|
+
`.claude/settings.json`, following the section below for the approach you
|
|
44
|
+
chose
|
|
28
45
|
|
|
29
46
|
</Stepper>
|
|
30
47
|
|
|
@@ -45,7 +62,7 @@ The gateway routes every request using `providerName/model`. The provider prefix
|
|
|
45
62
|
picks which of your providers handles the request — a gateway can contain
|
|
46
63
|
several providers that serve Anthropic models.
|
|
47
64
|
|
|
48
|
-
###
|
|
65
|
+
### Where Claude Code reads its settings
|
|
49
66
|
|
|
50
67
|
You can configure Claude Code either globally or per project by editing the
|
|
51
68
|
settings file:
|
|
@@ -53,6 +70,23 @@ settings file:
|
|
|
53
70
|
- **Global:** `~/.claude/settings.json`
|
|
54
71
|
- **Project-specific:** `.claude/settings.json` in your project directory
|
|
55
72
|
|
|
73
|
+
Each approach below shows the `env` block to add to that file. Restart Claude
|
|
74
|
+
Code after saving it, and all your Claude Code LLM requests route through the AI
|
|
75
|
+
Gateway.
|
|
76
|
+
|
|
77
|
+
:::note
|
|
78
|
+
|
|
79
|
+
Claude Code appends `/v1/messages` to the base URL itself, so
|
|
80
|
+
`ANTHROPIC_BASE_URL` is the app's URL _without_ the `/v1` suffix.
|
|
81
|
+
|
|
82
|
+
:::
|
|
83
|
+
|
|
84
|
+
## Use a configured provider key
|
|
85
|
+
|
|
86
|
+
With this approach the gateway calls Anthropic with the API key you saved on the
|
|
87
|
+
[provider](../managing-providers.mdx). Claude Code presents only the app's API
|
|
88
|
+
key, and Anthropic bills the account that owns the provider key.
|
|
89
|
+
|
|
56
90
|
Add the following environment configuration to the settings file:
|
|
57
91
|
|
|
58
92
|
```json
|
|
@@ -70,17 +104,138 @@ Add the following environment configuration to the settings file:
|
|
|
70
104
|
}
|
|
71
105
|
```
|
|
72
106
|
|
|
73
|
-
|
|
107
|
+
`ANTHROPIC_AUTH_TOKEN` replaces whatever login Claude Code would otherwise use,
|
|
108
|
+
so this approach doesn't consume your Claude subscription.
|
|
74
109
|
|
|
75
|
-
|
|
76
|
-
`ANTHROPIC_BASE_URL` is the app's URL _without_ the `/v1` suffix.
|
|
110
|
+
## Use your Claude subscription
|
|
77
111
|
|
|
78
|
-
|
|
112
|
+
With this approach Claude Code stays signed in to `claude.ai` — for example with
|
|
113
|
+
a Claude Max plan — and the gateway forwards that login credential to Anthropic
|
|
114
|
+
as the request's provider credential. The app's API key travels in a separate
|
|
115
|
+
`zp-gateway-api-key` header, so the gateway still authenticates the app, applies
|
|
116
|
+
its policy chain, and meters its usage.
|
|
117
|
+
|
|
118
|
+
Two pieces of configuration make this work: the app's **API Key Authentication**
|
|
119
|
+
policy accepts the app key from the `zp-gateway-api-key` header and passes the
|
|
120
|
+
`Authorization` header through, and Claude Code sends both headers.
|
|
121
|
+
|
|
122
|
+
### Configure the API Key Authentication policy
|
|
123
|
+
|
|
124
|
+
The [API Key Authentication](../../policies/ai-gateway-auth-v2-inbound.mdx)
|
|
125
|
+
policy needs three options for passthrough:
|
|
126
|
+
|
|
127
|
+
| Option | Value | Effect |
|
|
128
|
+
| ----------------------- | -------------------- | -------------------------------------------------------- |
|
|
129
|
+
| `credentialPassthrough` | On | Forward the caller's `Authorization` header to Anthropic |
|
|
130
|
+
| `authHeader` | `zp-gateway-api-key` | Read the app's API key from this header instead |
|
|
131
|
+
| `authScheme` | Empty value | The header holds the bare key, with no `Bearer` prefix |
|
|
132
|
+
|
|
133
|
+
You can set these options on the team's
|
|
134
|
+
[policy template](../policy-templates.mdx), so every app created in the team
|
|
135
|
+
starts with passthrough enabled, or on a single app's **Policies** tab. The
|
|
136
|
+
steps below use the team template; the app's policy editor works the same way.
|
|
137
|
+
|
|
138
|
+
<Stepper>
|
|
139
|
+
|
|
140
|
+
1. Open the team in the Zuplo Portal and select its **Policy Template** tab.
|
|
141
|
+
|
|
142
|
+
2. Click the edit (pencil) icon on the **API Key Authentication** entry.
|
|
143
|
+
|
|
144
|
+
3. Turn on **credentialPassthrough**.
|
|
145
|
+
|
|
146
|
+
4. Set **authHeader** to `zp-gateway-api-key`.
|
|
147
|
+
|
|
148
|
+
5. Leave **authScheme** blank and click **Use an empty value for authScheme**.
|
|
149
|
+
The field then reports "Empty value set. The default (Bearer) is not
|
|
150
|
+
applied." Clearing the field alone isn't enough: a blank field omits the
|
|
151
|
+
option, so the `Bearer` default still applies and every request fails with
|
|
152
|
+
`401 Invalid Authorization Scheme`.
|
|
153
|
+
|
|
154
|
+
6. Click **Apply**, then **Save changes**.
|
|
155
|
+
|
|
156
|
+
</Stepper>
|
|
157
|
+
|
|
158
|
+
The **JSON** editor mode shows the equivalent options:
|
|
159
|
+
|
|
160
|
+
```json
|
|
161
|
+
{
|
|
162
|
+
"credentialPassthrough": true,
|
|
163
|
+
"authHeader": "zp-gateway-api-key",
|
|
164
|
+
"authScheme": ""
|
|
165
|
+
}
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
### The provider still needs an API key
|
|
169
|
+
|
|
170
|
+
The Portal requires an API key to save an Anthropic provider, and passthrough
|
|
171
|
+
requests never send it: the request to Anthropic carries the caller's credential
|
|
172
|
+
instead. Anything else that uses this provider's configured credential still
|
|
173
|
+
needs a real key. That includes apps on this gateway that don't use passthrough
|
|
174
|
+
and any
|
|
175
|
+
[Fallback Model](../../policies/ai-gateway-fallback-model-v2-inbound.mdx)
|
|
176
|
+
target, because fallback requests use the provider's configured credential
|
|
177
|
+
rather than the caller's. If nothing on the gateway uses the configured
|
|
178
|
+
credential, a placeholder value works.
|
|
179
|
+
|
|
180
|
+
The app's playground in the Portal sends the app key as a bearer token in
|
|
181
|
+
`Authorization`, so it can't call an app that uses passthrough. Test the app
|
|
182
|
+
from Claude Code instead, as described below.
|
|
183
|
+
|
|
184
|
+
### Configure Claude Code
|
|
185
|
+
|
|
186
|
+
Add the following environment configuration to the settings file. Don't set
|
|
187
|
+
`ANTHROPIC_AUTH_TOKEN` or `ANTHROPIC_API_KEY` here or in your shell: either one
|
|
188
|
+
replaces your `claude.ai` login, and the gateway would then forward that value
|
|
189
|
+
to Anthropic as the credential.
|
|
190
|
+
|
|
191
|
+
```json
|
|
192
|
+
{
|
|
193
|
+
"env": {
|
|
194
|
+
"ANTHROPIC_BASE_URL": "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e",
|
|
195
|
+
"ANTHROPIC_CUSTOM_HEADERS": "zp-gateway-api-key: <your-ai-gateway-app-api-key>",
|
|
196
|
+
"ANTHROPIC_MODEL": "anthropic/claude-sonnet-5",
|
|
197
|
+
"ANTHROPIC_SMALL_FAST_MODEL": "anthropic/claude-haiku-4-5",
|
|
198
|
+
"ANTHROPIC_DEFAULT_OPUS_MODEL": "anthropic/claude-opus-5",
|
|
199
|
+
"ANTHROPIC_DEFAULT_SONNET_MODEL": "anthropic/claude-sonnet-5",
|
|
200
|
+
"ANTHROPIC_DEFAULT_HAIKU_MODEL": "anthropic/claude-haiku-4-5",
|
|
201
|
+
"ANTHROPIC_DEFAULT_FABLE_MODEL": "anthropic/claude-fable-5"
|
|
202
|
+
}
|
|
203
|
+
}
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
`ANTHROPIC_CUSTOM_HEADERS` adds the `zp-gateway-api-key` header to every request
|
|
207
|
+
Claude Code makes, while Claude Code keeps sending its own `Authorization`
|
|
208
|
+
header from your login. The gateway consumes those incoming headers, uses the
|
|
209
|
+
login credential to authenticate the Anthropic request, and never sends your app
|
|
210
|
+
key to Anthropic.
|
|
211
|
+
|
|
212
|
+
### Verify the subscription setup
|
|
213
|
+
|
|
214
|
+
Restart Claude Code, then confirm which credential it's using and send one
|
|
215
|
+
request through the gateway:
|
|
216
|
+
|
|
217
|
+
```bash
|
|
218
|
+
claude auth status --text
|
|
219
|
+
claude -p "Reply with the single word OK"
|
|
220
|
+
```
|
|
221
|
+
|
|
222
|
+
The first command should report that you're signed in with your `claude.ai`
|
|
223
|
+
account rather than an API key or auth token. The second returns a short reply,
|
|
224
|
+
and the request appears in the app's usage in the Zuplo Portal. In an
|
|
225
|
+
interactive session, `/status` shows the same login information.
|
|
226
|
+
|
|
227
|
+
If the request fails:
|
|
79
228
|
|
|
80
|
-
|
|
81
|
-
|
|
229
|
+
- `401` with `Invalid Authorization Scheme` — `authScheme` still has its
|
|
230
|
+
`Bearer` default. Set it to an explicit empty value in the policy.
|
|
231
|
+
- `401` with `Header configured by options.authHeader is missing` — Claude Code
|
|
232
|
+
didn't send `zp-gateway-api-key`. Check `ANTHROPIC_CUSTOM_HEADERS`.
|
|
233
|
+
- `401` with `credentialPassthrough requires a non-empty Authorization header` —
|
|
234
|
+
Claude Code isn't signed in. Run `claude` and use `/login`.
|
|
235
|
+
- An authentication error from Anthropic — Anthropic rejected the forwarded
|
|
236
|
+
login credential. Sign in to Claude Code again.
|
|
82
237
|
|
|
83
|
-
|
|
238
|
+
## Configure every Claude Code model variable
|
|
84
239
|
|
|
85
240
|
The model ids in these examples reflect what was current when this page was
|
|
86
241
|
written, and ids differ across providers — a Bedrock provider serves different
|
|
@@ -53,9 +53,12 @@ To add a new AI provider to your Zuplo AI Gateway, follow these steps:
|
|
|
53
53
|
[Zuplo Demo provider](./providers.mdx#zuplo-demo) asks for no API key—your
|
|
54
54
|
gateway authenticates to the demo service itself,
|
|
55
55
|
[Bedrock Mantle](./bedrock-mantle.mdx) additionally asks for an **AWS
|
|
56
|
-
Region** and accepts only long-term Bedrock API keys,
|
|
56
|
+
Region** and accepts only long-term Bedrock API keys,
|
|
57
57
|
[Vertex AI](./vertex-ai.mdx) asks for a **Location** and a **Google Cloud
|
|
58
|
-
Project ID** and takes a service account JSON key file instead of an API key
|
|
58
|
+
Project ID** and takes a service account JSON key file instead of an API key,
|
|
59
|
+
and [Azure AI](./azure-ai.mdx) asks for an **Azure Resource Name** and a host
|
|
60
|
+
family, plus a mapping for any deployment named differently from the model it
|
|
61
|
+
serves.
|
|
59
62
|
|
|
60
63
|
1. Select the model or models you want to use with this provider. The available
|
|
61
64
|
models will depend on the selected provider. This can be changed later.
|
|
@@ -85,10 +88,11 @@ To modify an existing provider, open
|
|
|
85
88
|
and click the **Edit** button next to the provider you want to modify.
|
|
86
89
|
|
|
87
90
|
You can modify the API key and selected models for the provider—and, for
|
|
88
|
-
[Bedrock Mantle](./bedrock-mantle.mdx), the AWS Region,
|
|
89
|
-
[Vertex AI](./vertex-ai.mdx), the Location and Google Cloud Project ID
|
|
90
|
-
|
|
91
|
-
|
|
91
|
+
[Bedrock Mantle](./bedrock-mantle.mdx), the AWS Region, for
|
|
92
|
+
[Vertex AI](./vertex-ai.mdx), the Location and Google Cloud Project ID, or for
|
|
93
|
+
[Azure AI](./azure-ai.mdx), the Azure Resource Name, host family, and deployment
|
|
94
|
+
mappings. The **Provider Name** isn't editable: it's the routing address in
|
|
95
|
+
every `providerName/model` reference, so renaming it would orphan each stored
|
|
92
96
|
reference to this provider. After making your changes, click **Save** to apply
|
|
93
97
|
them.
|
|
94
98
|
|
|
@@ -75,8 +75,9 @@ usage limits still apply.
|
|
|
75
75
|
### Multi-Provider Support
|
|
76
76
|
|
|
77
77
|
Configure multiple LLM providers within a single AI Gateway project. Supported
|
|
78
|
-
providers include OpenAI, Anthropic, Google, Mistral, xAI,
|
|
79
|
-
(through [
|
|
78
|
+
providers include OpenAI, Anthropic, Google, Mistral, xAI, Microsoft Azure
|
|
79
|
+
(through [Azure AI](./azure-ai.mdx)), Amazon Bedrock (through
|
|
80
|
+
[Bedrock Mantle](./bedrock-mantle.mdx)), Google Cloud
|
|
80
81
|
([Vertex AI](./vertex-ai.mdx)), and OpenAI-compatible custom providers. See
|
|
81
82
|
[AI Providers](./providers.mdx) for the full list of providers and supported
|
|
82
83
|
capabilities. Apps reference models as `providerName/model`—for example
|
|
@@ -19,6 +19,8 @@ Zuplo currently supports the following AI providers:
|
|
|
19
19
|
- Google
|
|
20
20
|
- Mistral
|
|
21
21
|
- xAI (Grok)
|
|
22
|
+
- [Azure AI](./azure-ai.mdx)—Azure OpenAI and Microsoft Foundry resources,
|
|
23
|
+
serving Claude and every Foundry model family from your own Azure subscription
|
|
22
24
|
- [Bedrock Mantle](./bedrock-mantle.mdx)—Amazon Bedrock's compatible-APIs
|
|
23
25
|
endpoint, serving Claude models and models from many other vendors
|
|
24
26
|
- [Vertex AI](./vertex-ai.mdx)—Google Cloud's managed model platform, serving
|
|
@@ -36,6 +38,7 @@ The following capabilities are supported across providers:
|
|
|
36
38
|
| Google | ✅ | ✅ | ❌ | ❌ |
|
|
37
39
|
| Mistral | ✅ | ✅ | ❌ | ❌ |
|
|
38
40
|
| xAI | ✅ | ✅ | ❌ | ❌ |
|
|
41
|
+
| Azure AI | ✅ | ✅ | ✅ | ✅ |
|
|
39
42
|
| Bedrock Mantle | ✅ | ❌ | ✅ | ✅ |
|
|
40
43
|
| Vertex AI | ✅ | ✅ | ❌ | ✅ |
|
|
41
44
|
| Zuplo Demo | ✅ | ❌ | ❌ | ❌ |
|
|
@@ -51,6 +54,16 @@ Messages (plus chat completions through translation), while its other models
|
|
|
51
54
|
serve chat completions and—per model—Responses. See
|
|
52
55
|
[Using Bedrock Mantle](./bedrock-mantle.mdx#supported-endpoints-by-model-family).
|
|
53
56
|
|
|
57
|
+
Azure AI's capabilities split by model family, and within the OpenAI-compatible
|
|
58
|
+
family they split per deployed model: a chat model serves chat completions and—
|
|
59
|
+
per model and version—Responses, while an embedding model serves embeddings and
|
|
60
|
+
neither of the others. Claude serves Messages (plus chat completions through
|
|
61
|
+
translation) and needs a Microsoft Foundry resource, since Claude can't be
|
|
62
|
+
deployed on an Azure OpenAI resource. Azure also addresses models by
|
|
63
|
+
**deployment name** rather than by published model ID, so a deployment named
|
|
64
|
+
differently from the model it serves needs mapping for the gateway to price it.
|
|
65
|
+
See [Using Azure AI](./azure-ai.mdx#azure-serves-deployments-not-model-names).
|
|
66
|
+
|
|
54
67
|
A custom provider must serve chat completions and embeddings under a `/v1` path
|
|
55
68
|
segment on its API URL, and you enter that URL as an origin root—without the
|
|
56
69
|
`/v1` suffix vendors usually publish. See
|
|
@@ -61,6 +74,32 @@ Apps reference a provider's models as `providerName/model`—for example
|
|
|
61
74
|
name you give the provider configuration. See the
|
|
62
75
|
[Universal API](./universal-api.mdx).
|
|
63
76
|
|
|
77
|
+
## Azure AI
|
|
78
|
+
|
|
79
|
+
**Azure AI** serves models from your own Azure resource. One provider
|
|
80
|
+
configuration covers both kinds of resource, and which capabilities apply
|
|
81
|
+
depends on the resource and the model:
|
|
82
|
+
|
|
83
|
+
- An **Azure OpenAI** resource deploys OpenAI models and serves chat
|
|
84
|
+
completions, embeddings, and the Responses API.
|
|
85
|
+
- A **Microsoft Foundry** resource serves those plus every Foundry model
|
|
86
|
+
family—Grok, DeepSeek, Llama, Mistral, Phi, Kimi and more—and serves **Claude
|
|
87
|
+
models** on the native Anthropic Messages API, with chat completions through
|
|
88
|
+
the gateway's translation.
|
|
89
|
+
|
|
90
|
+
Two things make Azure different from every other provider:
|
|
91
|
+
|
|
92
|
+
- **It addresses deployments, not published model IDs.** You name deployments
|
|
93
|
+
when you create them in Azure, and that name is what goes in the `model`
|
|
94
|
+
field. A deployment whose name differs from the model it serves needs a
|
|
95
|
+
mapping, or the gateway can't price its usage.
|
|
96
|
+
- **Its endpoint host embeds your resource name.** The provider dialog asks for
|
|
97
|
+
an **Azure Resource Name** and a host family rather than a full URL, and it
|
|
98
|
+
takes a resource key—not a Microsoft Entra ID token.
|
|
99
|
+
|
|
100
|
+
For prerequisites, setup steps, code examples, and troubleshooting, see
|
|
101
|
+
[Using Azure AI](./azure-ai.mdx).
|
|
102
|
+
|
|
64
103
|
## Bedrock Mantle
|
|
65
104
|
|
|
66
105
|
**Bedrock Mantle** is Amazon Bedrock's compatible-APIs endpoint. One regional
|
|
@@ -63,9 +63,9 @@ list—so clients that can't set a model still work.
|
|
|
63
63
|
|
|
64
64
|
## Supported endpoints
|
|
65
65
|
|
|
66
|
-
| Endpoint | Notes
|
|
67
|
-
| ---------------------- |
|
|
68
|
-
| `/v1/chat/completions` | Chat completions for every provider
|
|
69
|
-
| `/v1/embeddings` | Embeddings for
|
|
70
|
-
| `/v1/responses` | OpenAI Responses API: OpenAI, and [Bedrock Mantle](./bedrock-mantle.mdx)
|
|
71
|
-
| `/v1/messages` | Anthropic Messages API: Anthropic, and the Claude models of [Bedrock Mantle](./bedrock-mantle.mdx) and [Vertex AI](./vertex-ai.mdx) |
|
|
66
|
+
| Endpoint | Notes |
|
|
67
|
+
| ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
68
|
+
| `/v1/chat/completions` | Chat completions for every provider |
|
|
69
|
+
| `/v1/embeddings` | Embeddings for the providers marked in the [capability matrix](./providers.mdx#supported-providers) |
|
|
70
|
+
| `/v1/responses` | OpenAI Responses API: OpenAI, and the OpenAI-compatible models that serve it on [Azure AI](./azure-ai.mdx) and [Bedrock Mantle](./bedrock-mantle.mdx) |
|
|
71
|
+
| `/v1/messages` | Anthropic Messages API: Anthropic, and the Claude models of [Azure AI](./azure-ai.mdx), [Bedrock Mantle](./bedrock-mantle.mdx) and [Vertex AI](./vertex-ai.mdx) |
|
|
@@ -129,7 +129,12 @@ send your traffic and your spend to someone else's Google Cloud project.
|
|
|
129
129
|
| `/v1/chat/completions` | ✅ Forwarded | ❌ |
|
|
130
130
|
| `/v1/embeddings` | ✅ Translated | ❌ |
|
|
131
131
|
| `/v1/responses` | ❌ | ❌ |
|
|
132
|
-
| `/v1/messages` | ❌ | ✅
|
|
132
|
+
| `/v1/messages` | ❌ | ✅ Forwarded |
|
|
133
|
+
|
|
134
|
+
**Forwarded** means the gateway sends your request on in the shape you wrote it.
|
|
135
|
+
**Translated** means it converts between two API shapes—for embeddings, from the
|
|
136
|
+
OpenAI shape onto Vertex's own embedding APIs, which is invisible from your
|
|
137
|
+
side.
|
|
133
138
|
|
|
134
139
|
Streaming (`stream: true`) works on chat completions and on Messages.
|
|
135
140
|
|
package/docs/policies/_index.md
CHANGED
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
| --- | --- | --- | --- |
|
|
5
5
|
| set-query-params-inbound | Add or Set Query Parameters | Adds or sets query parameters on the incoming request. | api-gateway |
|
|
6
6
|
| set-headers-inbound | Add or Set Request Headers | Adds or sets headers on the incoming request. | api-gateway |
|
|
7
|
-
| ai-gateway-auth-v2-inbound | AI Gateway Authentication | Authenticates requests to an AI Gateway endpoint with application API keys. Add this policy to an application's `inboundPolicyChain` to require a key for that app only, or place it on the route before the configuration executor to require a key for every application on the route. The policies that follow can read the authenticated application from `request.user` (`sub` is the application name, `data` its metadata), and the application's AI Gateway configuration takes effect for the request. Use `authHeader` and `authScheme` when clients send their app key somewhere other than the default `Authorization: Bearer` header. When the matched route captures an `app_id` path parameter (platform catch-all `/:app_id/(.*)`), this policy also requires `configuration.id === request.params.app_id` and returns 403 on mismatch. | ai-gateway |
|
|
7
|
+
| ai-gateway-auth-v2-inbound | AI Gateway Authentication | Authenticates requests to an AI Gateway endpoint with application API keys. Add this policy to an application's `inboundPolicyChain` to require a key for that app only, or place it on the route before the configuration executor to require a key for every application on the route. The policies that follow can read the authenticated application from `request.user` (`sub` is the application name, `data` its metadata), and the application's AI Gateway configuration takes effect for the request. Use `authHeader` and `authScheme` when clients send their app key somewhere other than the default `Authorization: Bearer` header. Enable `credentialPassthrough`, with the application key in a separate header, to forward the caller's `Authorization` header to the primary provider as its credential. When the matched route captures an `app_id` path parameter (platform catch-all `/:app_id/(.*)`), this policy also requires `configuration.id === request.params.app_id` and returns 403 on mismatch. | ai-gateway |
|
|
8
8
|
| ai-gateway-configuration-executor-v2-inbound | AI Gateway Configuration Executor | Loads the app configuration for the request (when auth or the configuration loader has not already), runs the inbound policy chain from that configuration, and enforces limits inherited from parent teams or the gateway root. Place this policy on AI Gateway routes after optional authentication and optional `ai-gateway-configuration-loader-v2-inbound`. When either of those already populated the app-configuration channel, this policy reuses it. Otherwise it loads the configuration with the route's `app_id` path parameter. Applications select from policies pre-declared by the gateway. Applications without a `inboundPolicyChain`, or with an empty chain, run no application-selected policies. Entry options replace the declaration's options as a complete object; omit them to inherit the declaration, including environment-backed credentials. Each occurrence receives a private deep copy of its entry options, so a policy mutating its options cannot corrupt the cached app configuration. | ai-gateway |
|
|
9
9
|
| ai-gateway-configuration-loader-v2-inbound | AI Gateway Configuration Loader | Loads the AI Gateway app configuration for the request into the request-scoped channel and does nothing else. Place this policy on AI Gateway routes before `ai-gateway-configuration-executor-v2-inbound` when you want configuration loading separated from chain execution. When `ai-gateway-auth-v2-inbound` already populated the channel, this policy reuses it. Otherwise it loads the configuration with the route's `app_id` path parameter. If this policy is omitted, the configuration executor still loads configuration itself before running the application chain. | ai-gateway |
|
|
10
10
|
| ai-gateway-fallback-model-v2-inbound | AI Gateway Fallback Model | Adds failure and quota fallbacks to an existing AI Gateway model selection. Place this policy after AI Gateway Model Filtering. It never creates a model selection, so a misplaced policy cannot bypass filtering. | ai-gateway |
|
|
@@ -147,6 +147,64 @@ curl https://gateway.example.com/v1/chat/completions \
|
|
|
147
147
|
}'
|
|
148
148
|
```
|
|
149
149
|
|
|
150
|
+
## Pass through a caller's provider credential
|
|
151
|
+
|
|
152
|
+
Set `credentialPassthrough` to `true` and configure a separate header for the
|
|
153
|
+
Zuplo application key. For a bare key in `zp-gateway-api-key`, use these policy
|
|
154
|
+
options:
|
|
155
|
+
|
|
156
|
+
```json
|
|
157
|
+
{
|
|
158
|
+
"credentialPassthrough": true,
|
|
159
|
+
"authHeader": "zp-gateway-api-key",
|
|
160
|
+
"authScheme": ""
|
|
161
|
+
}
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
Send the provider credential in `Authorization`. `authHeader` and `authScheme`
|
|
165
|
+
remain authoritative; sending a special header does not enable passthrough or
|
|
166
|
+
override your configuration. The defaults remain `Authorization` and `Bearer`,
|
|
167
|
+
so enabling passthrough without configuring a separate header is a configuration
|
|
168
|
+
error.
|
|
169
|
+
|
|
170
|
+
For example:
|
|
171
|
+
|
|
172
|
+
```bash
|
|
173
|
+
curl https://gateway.example.com/v1/chat/completions \
|
|
174
|
+
--header "zp-gateway-api-key: YOUR_ZUPLO_APP_KEY" \
|
|
175
|
+
--header "Authorization: Bearer YOUR_PROVIDER_CREDENTIAL" \
|
|
176
|
+
--header "Content-Type: application/json" \
|
|
177
|
+
--data '{
|
|
178
|
+
"model": "provider/model",
|
|
179
|
+
"messages": [{ "role": "user", "content": "Hello" }]
|
|
180
|
+
}'
|
|
181
|
+
```
|
|
182
|
+
|
|
183
|
+
After the Zuplo application key succeeds, the policy moves the provider
|
|
184
|
+
credential into private request-scoped state and removes both authentication
|
|
185
|
+
headers from the mutable inbound request. The primary provider receives the
|
|
186
|
+
original `Authorization` value exactly as supplied, instead of its configured
|
|
187
|
+
credential. Retries of the primary use that same caller credential. Error and
|
|
188
|
+
quota fallbacks use their own configured credentials, never the caller's token.
|
|
189
|
+
Fallbacks therefore require configured credentials and can incur charges on the
|
|
190
|
+
gateway owner's provider account. On native create requests at `/v1/messages`
|
|
191
|
+
and `/v1/responses`, a compatible, explicitly configured backup runs only after
|
|
192
|
+
the primary exhausts upstream 429 retries. It uses the backup's configured
|
|
193
|
+
credential while preserving the native response and stream format; these
|
|
194
|
+
endpoints do not use timeout, transport-error, or 5xx fallback. Semantic-cache
|
|
195
|
+
policies skip passthrough requests so a cache hit cannot bypass upstream
|
|
196
|
+
authentication. ZuploDemo is excluded because it must continue to use the
|
|
197
|
+
gateway's deployment credential.
|
|
198
|
+
|
|
199
|
+
A missing, empty, or invalid dedicated key fails authentication; the gateway
|
|
200
|
+
never retries with `Authorization` as the Zuplo key. A missing or empty
|
|
201
|
+
`Authorization` also returns 401 in this mode.
|
|
202
|
+
|
|
203
|
+
With `credentialPassthrough` omitted or `false`, existing authentication
|
|
204
|
+
applies: `authHeader` and `authScheme` select the Zuplo key, and providers use
|
|
205
|
+
their configured credentials. Moving the gateway key to an arbitrary custom
|
|
206
|
+
header alone does not enable passthrough.
|
|
207
|
+
|
|
150
208
|
## Choose a cache duration
|
|
151
209
|
|
|
152
210
|
`cacheTtlSeconds` controls how long an authentication result can be reused. The
|
|
@@ -163,6 +221,10 @@ A revoked key can continue to work until its cached result expires.
|
|
|
163
221
|
- `cacheTtlSeconds`: Number of seconds to cache an authentication result.
|
|
164
222
|
Defaults to `10` and must be at least `10`.
|
|
165
223
|
- `authHeader`: Header containing the application key. Defaults to
|
|
166
|
-
`Authorization`.
|
|
167
|
-
-
|
|
168
|
-
|
|
224
|
+
`Authorization`. With `credentialPassthrough`, use a different header, such as
|
|
225
|
+
`zp-gateway-api-key`.
|
|
226
|
+
- `authScheme`: Scheme before the key. Defaults to `Bearer`. Use `""` when the
|
|
227
|
+
header contains only the key.
|
|
228
|
+
- `credentialPassthrough`: Forward the caller's `Authorization` header to the
|
|
229
|
+
primary provider as its credential while Zuplo reads the application key from
|
|
230
|
+
`authHeader`. Defaults to `false`.
|
|
@@ -12,7 +12,7 @@
|
|
|
12
12
|
"requiresAI": true,
|
|
13
13
|
"policyType": "ai-gateway-auth-v2",
|
|
14
14
|
"products": ["ai-gateway"],
|
|
15
|
-
"description": "Authenticates requests to an AI Gateway endpoint with application API keys.\n\nAdd this policy to an application's `inboundPolicyChain` to require a key for that app only, or place it on the route before the configuration executor to require a key for every application on the route. The policies that follow can read the authenticated application from `request.user` (`sub` is the application name, `data` its metadata), and the application's AI Gateway configuration takes effect for the request. Use `authHeader` and `authScheme` when clients send their app key somewhere other than the default `Authorization: Bearer` header.\n\nWhen the matched route captures an `app_id` path parameter (platform catch-all `/:app_id/(.*)`), this policy also requires `configuration.id === request.params.app_id` and returns 403 on mismatch.",
|
|
15
|
+
"description": "Authenticates requests to an AI Gateway endpoint with application API keys.\n\nAdd this policy to an application's `inboundPolicyChain` to require a key for that app only, or place it on the route before the configuration executor to require a key for every application on the route. The policies that follow can read the authenticated application from `request.user` (`sub` is the application name, `data` its metadata), and the application's AI Gateway configuration takes effect for the request. Use `authHeader` and `authScheme` when clients send their app key somewhere other than the default `Authorization: Bearer` header. Enable `credentialPassthrough`, with the application key in a separate header, to forward the caller's `Authorization` header to the primary provider as its credential.\n\nWhen the matched route captures an `app_id` path parameter (platform catch-all `/:app_id/(.*)`), this policy also requires `configuration.id === request.params.app_id` and returns 403 on mismatch.",
|
|
16
16
|
"deprecatedMessage": "",
|
|
17
17
|
"required": ["handler"],
|
|
18
18
|
"properties": {
|
|
@@ -36,6 +36,25 @@
|
|
|
36
36
|
"description": "The options for this policy.",
|
|
37
37
|
"additionalProperties": false,
|
|
38
38
|
"required": [],
|
|
39
|
+
"if": {
|
|
40
|
+
"required": ["credentialPassthrough"],
|
|
41
|
+
"properties": {
|
|
42
|
+
"credentialPassthrough": {
|
|
43
|
+
"const": true
|
|
44
|
+
}
|
|
45
|
+
}
|
|
46
|
+
},
|
|
47
|
+
"then": {
|
|
48
|
+
"required": ["authHeader"],
|
|
49
|
+
"properties": {
|
|
50
|
+
"authHeader": {
|
|
51
|
+
"minLength": 1,
|
|
52
|
+
"not": {
|
|
53
|
+
"pattern": "^\\s*[Aa][Uu][Tt][Hh][Oo][Rr][Ii][Zz][Aa][Tt][Ii][Oo][Nn]\\s*$"
|
|
54
|
+
}
|
|
55
|
+
}
|
|
56
|
+
}
|
|
57
|
+
},
|
|
39
58
|
"properties": {
|
|
40
59
|
"cacheTtlSeconds": {
|
|
41
60
|
"type": "number",
|
|
@@ -48,14 +67,19 @@
|
|
|
48
67
|
"default": "Authorization",
|
|
49
68
|
"x-show-example": false,
|
|
50
69
|
"x-advanced": true,
|
|
51
|
-
"description": "The
|
|
70
|
+
"description": "The request header that carries the Zuplo application key. When credentialPassthrough is enabled, use a header other than Authorization, such as zp-gateway-api-key."
|
|
52
71
|
},
|
|
53
72
|
"authScheme": {
|
|
54
73
|
"type": "string",
|
|
55
74
|
"default": "Bearer",
|
|
56
75
|
"x-show-example": false,
|
|
57
76
|
"x-advanced": true,
|
|
58
|
-
"description": "The scheme
|
|
77
|
+
"description": "The scheme that prefixes the key in the authHeader header, for example Bearer. Set to an empty string when the header contains only the key."
|
|
78
|
+
},
|
|
79
|
+
"credentialPassthrough": {
|
|
80
|
+
"type": "boolean",
|
|
81
|
+
"default": false,
|
|
82
|
+
"description": "Forward the caller's Authorization header to the primary provider as its credential. Zuplo reads the application key from authHeader, which must be a header other than Authorization."
|
|
59
83
|
}
|
|
60
84
|
}
|
|
61
85
|
}
|
|
@@ -65,7 +89,8 @@
|
|
|
65
89
|
"export": "AIGatewayAuthV2InboundPolicy",
|
|
66
90
|
"module": "$import(@zuplo/runtime)",
|
|
67
91
|
"options": {
|
|
68
|
-
"cacheTtlSeconds": 10
|
|
92
|
+
"cacheTtlSeconds": 10,
|
|
93
|
+
"credentialPassthrough": false
|
|
69
94
|
}
|
|
70
95
|
}
|
|
71
96
|
]
|