zuplo 7.5.2 → 7.5.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -20,10 +20,10 @@ Each app has three things of its own:
20
20
  - **An API URL**—the endpoint the app's code calls, shown in full at the top of
21
21
  the app page. Expand it to see the ready-to-paste URL for each service the
22
22
  gateway mounts under the app—**Chat Completions** (`v1/chat/completions`),
23
- **Responses** (`v1/responses`, OpenAI only), and **Messages** (`v1/messages`,
24
- Anthropic only)—and copy the one your client needs. The gateway attributes
25
- requests made to this URL to the app, tracking usage independently per app.
26
- See the [Universal API](./universal-api.mdx).
23
+ **Responses** (`v1/responses`), and **Messages** (`v1/messages`)—and copy the
24
+ one your client needs. The [Universal API](./universal-api.mdx) lists which
25
+ providers serve each endpoint. The gateway attributes requests made to this
26
+ URL to the app, tracking usage independently per app.
27
27
  - **An API key**—authenticates the app's requests when the gateway runs the
28
28
  [authentication policy](./policy-chains.mdx#authentication).
29
29
  - **A [policy chain](./policy-chains.mdx)**—the ordered policies that run on the
@@ -0,0 +1,343 @@
1
+ ---
2
+ title: Using Bedrock Mantle
3
+ sidebar_label: Bedrock Mantle
4
+ description:
5
+ Serve Claude and models from many other vendors on Amazon Bedrock through one
6
+ provider configuration—one regional endpoint and one long-term Bedrock API
7
+ key. The gateway routes each model to the right Bedrock API.
8
+ ---
9
+
10
+ **Bedrock Mantle** is
11
+ [Amazon Bedrock's compatible-APIs endpoint](https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-mantle.html).
12
+ One regional endpoint (`https://bedrock-mantle.<region>.api.aws`) and one
13
+ long-term Bedrock API key serve models from many vendors—the Claude family,
14
+ OpenAI's GPT-5.x and GPT-OSS families, Gemma, Qwen, DeepSeek, Mistral, Grok,
15
+ Kimi, GLM, and more. Adding Bedrock Mantle as a provider gives your
16
+ [apps](./apps.mdx) all of them through the [Universal API](./universal-api.mdx)
17
+ with a single credential, on your AWS account and AWS billing.
18
+
19
+ Mantle is the part of Amazon Bedrock's catalog that AWS exposes over these
20
+ compatible APIs. A model that Bedrock offers only through its native
21
+ `InvokeModel` or `Converse` APIs isn't served by Mantle, so the gateway can't
22
+ route to it.
23
+
24
+ Apps reference the models as `providerName/model`, where `providerName` is the
25
+ name you give the provider configuration. A provider named `bedrockmantle`
26
+ serves `bedrockmantle/anthropic.claude-sonnet-5` and
27
+ `bedrockmantle/openai.gpt-oss-120b`.
28
+
29
+ ## How the gateway routes Mantle models
30
+
31
+ Mantle serves two API formats on the same host with the same API key:
32
+
33
+ - An **OpenAI-compatible API**, serving every non-Claude model with OpenAI-style
34
+ chat completions and—for the models that support it—the OpenAI Responses API.
35
+ - The **native Anthropic Messages API**, serving the Claude models.
36
+
37
+ <Diagram height="h-64">
38
+ <DiagramNode id="app">Your app</DiagramNode>
39
+ <DiagramNode id="gateway" variant="zuplo">
40
+ AI Gateway
41
+ </DiagramNode>
42
+ <DiagramGroup id="mantle" label="Bedrock Mantle (your region)">
43
+ <DiagramNode id="openai-surface" variant="blue">
44
+ OpenAI-compatible API
45
+ </DiagramNode>
46
+ <DiagramNode id="messages-surface" variant="green">
47
+ Anthropic Messages API
48
+ </DiagramNode>
49
+ </DiagramGroup>
50
+ <DiagramEdge from="app" to="gateway" label="Universal API" />
51
+ <DiagramEdge
52
+ from="gateway"
53
+ to="openai-surface"
54
+ label="OpenAI-compatible models"
55
+ />
56
+ <DiagramEdge from="gateway" to="messages-surface" label="Claude models" />
57
+ </Diagram>
58
+
59
+ The gateway's model catalog records which API serves each model, and the gateway
60
+ routes every request accordingly—your clients always call your app's URL and
61
+ never see the Mantle endpoint. This also covers a Mantle quirk: AWS serves some
62
+ models (the Gemma 4 and GPT-5.x families among them) on a different base path
63
+ (`/openai/v1` instead of `/v1`), documented per
64
+ [AWS model card](https://docs.aws.amazon.com/bedrock/latest/userguide/model-cards.html)—the
65
+ per-model reference pages in the AWS Bedrock docs. The gateway sends each model
66
+ to its documented path, so the model reference and your app's URL stay the same
67
+ either way.
68
+
69
+ ## Supported endpoints by model family
70
+
71
+ | Endpoint | OpenAI-compatible models | Claude models |
72
+ | ---------------------- | ---------------------------- | ------------- |
73
+ | `/v1/chat/completions` | ✅ Forwarded | ✅ Translated |
74
+ | `/v1/responses` | ✅ Models that serve it | ❌ |
75
+ | `/v1/messages` | ❌ | ✅ Native |
76
+ | `/v1/embeddings` | ❌ Mantle has no such models | ❌ |
77
+
78
+ Streaming (`stream: true`) works on all three serving endpoints.
79
+
80
+ On the OpenAI-compatible side, endpoint support is per model, decided by AWS:
81
+
82
+ - The GPT-5.x models (`openai.gpt-5.6-sol`, `openai.gpt-5.5`, and family) serve
83
+ the Responses API only. Sending one to `/v1/chat/completions` returns Mantle's
84
+ own error that the model doesn't support that API.
85
+ - The GPT-OSS models (`openai.gpt-oss-120b`, `openai.gpt-oss-20b`) serve both
86
+ chat completions and the Responses API.
87
+ - Most of the other models serve chat completions only, and answer
88
+ `/v1/responses` with an error.
89
+
90
+ The **APIs supported** row of a model's AWS model card is the reference. When a
91
+ model doesn't serve the requested API, the gateway forwards Mantle's error to
92
+ your client.
93
+
94
+ Claude models work on `/v1/chat/completions` through the gateway's translation
95
+ to the Messages API, which supports the core chat parameters: `messages`,
96
+ `max_tokens` (defaults to 1024 when omitted), `temperature`, `top_p`, `stop`,
97
+ and `stream`. Other OpenAI parameters—`tools`, `response_format`, penalties, and
98
+ the rest—are dropped before forwarding, and `n > 1` is rejected. For tool use or
99
+ other Anthropic-specific features with Claude models, call
100
+ [`/v1/messages`](#call-claude-models-on-the-messages-api) instead.
101
+
102
+ ## Before you begin
103
+
104
+ You need:
105
+
106
+ - An AWS account with access to Amazon Bedrock in the region you plan to use.
107
+ Model availability varies by region, and some models—the Claude Mythos models,
108
+ for example—require an access grant from AWS. Check each model's AWS model
109
+ card.
110
+ - A long-term Bedrock API key. In the AWS console, open **Bedrock → API keys**
111
+ and generate a long-term key—it starts with `ABSK`.
112
+ - An AI Gateway project in the Zuplo Portal.
113
+ - An AI Gateway [app](./apps.mdx) to call the models from. The app page shows
114
+ the app's API URL, and its API key lives on the app's **API Key** tab.
115
+
116
+ :::caution{title="Long-term API keys only"}
117
+
118
+ Bedrock issues both long-term and short-term API keys, and the gateway accepts
119
+ only long-term ones, which start with `ABSK`. A short-term key (prefixed
120
+ `bedrock-api-key-`) expires within 12 hours and is locked to the session that
121
+ minted it, so the provider dialog rejects it rather than letting the connection
122
+ break the same day.
123
+
124
+ :::
125
+
126
+ ## Add the provider
127
+
128
+ Adding or editing providers requires the **Edit** permission, granted to Zuplo
129
+ account and project **Admins**—see
130
+ [Managing Providers](./managing-providers.mdx).
131
+
132
+ <Stepper>
133
+
134
+ 1. Open
135
+ [**Settings → AI Providers**](https://portal.zuplo.com/+/account/project/ai/settings/data-models)
136
+ in your AI Gateway project in the Zuplo Portal.
137
+
138
+ 1. Click the **Add Provider** button.
139
+
140
+ 1. In the **AI Provider** list, select **Bedrock Mantle** from the Default
141
+ Providers group.
142
+
143
+ 1. Review the **Provider Name**, which fills in as `bedrockmantle` (a second
144
+ configuration becomes `bedrockmantle-2`). You can replace it with your own
145
+ name, but only now—the name is permanent after creation, and it's the prefix
146
+ in every model reference: a provider named `bedrockmantle` serves
147
+ `bedrockmantle/anthropic.claude-sonnet-5`.
148
+
149
+ 1. In **AWS Region**, enter the lowercase code of the AWS region you use Bedrock
150
+ in, such as `us-east-1`. The gateway sends this provider's requests to
151
+ `https://bedrock-mantle.<region>.api.aws`—there's no endpoint URL to enter.
152
+
153
+ 1. In **API Key**, paste your long-term Bedrock API key. The dialog rejects
154
+ short-term keys and any value that doesn't start with `ABSK`.
155
+
156
+ 1. Select the models to enable, or click **Select All**. The picker shows AWS's
157
+ per-token prices, which the gateway uses to track cost per app. You can
158
+ change the selection later.
159
+
160
+ 1. Click **Create**.
161
+
162
+ </Stepper>
163
+
164
+ :::note
165
+
166
+ Saving provider settings triggers an automatic production deployment of your
167
+ gateway, because provider credentials are part of the deployed gateway. The
168
+ change is live once the deployment completes.
169
+
170
+ :::
171
+
172
+ When you edit the provider later—see
173
+ [Managing Providers](./managing-providers.mdx)—the **AWS Region** field shows
174
+ the saved region and stays editable, and you can replace the API key. The
175
+ **Provider Name** doesn't change.
176
+
177
+ ## Verify the provider
178
+
179
+ Once the deployment completes, send a chat completions request to your app's
180
+ [Universal API](./universal-api.mdx) URL—shown at the top of the
181
+ [app page](./apps.mdx)—with the app's API key as the bearer token and one of the
182
+ models you enabled:
183
+
184
+ ```bash
185
+ curl https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1/chat/completions \
186
+ -H "Authorization: Bearer $ZUPLO_APP_API_KEY" \
187
+ -H "Content-Type: application/json" \
188
+ -d '{
189
+ "model": "bedrockmantle/openai.gpt-oss-120b",
190
+ "messages": [{ "role": "user", "content": "Say hi" }]
191
+ }'
192
+ ```
193
+
194
+ The URL is a sample—replace it with your app's API URL plus
195
+ `/v1/chat/completions`, set `ZUPLO_APP_API_KEY` to the app's API key, and
196
+ substitute any model you enabled for `bedrockmantle/openai.gpt-oss-120b`.
197
+
198
+ A `200` response confirms the provider works; the body carries
199
+ `"provider": "bedrock-mantle"`—the provider type, not the name you chose. If the
200
+ request fails immediately after you save the provider, the deployment may not
201
+ have finished—retry before debugging further.
202
+
203
+ ## Call OpenAI-compatible models
204
+
205
+ Use any OpenAI client with your app's URL plus `/v1` as the base URL and the
206
+ app's API key:
207
+
208
+ ```ts
209
+ import OpenAI from "openai";
210
+
211
+ const client = new OpenAI({
212
+ apiKey: process.env.ZUPLO_APP_API_KEY,
213
+ baseURL:
214
+ "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1",
215
+ });
216
+
217
+ const response = await client.chat.completions.create({
218
+ model: "bedrockmantle/openai.gpt-oss-120b",
219
+ messages: [{ role: "user", content: "Summarize this ticket." }],
220
+ });
221
+ ```
222
+
223
+ The same call works with a Claude model string—the gateway translates it to the
224
+ Messages API—within the
225
+ [translation's parameter subset](#supported-endpoints-by-model-family).
226
+
227
+ ### Use the Responses API
228
+
229
+ Models that serve the Responses API (the GPT-OSS and GPT-5.x families) work
230
+ through `/v1/responses`, including streaming:
231
+
232
+ ```ts
233
+ const response = await client.responses.create({
234
+ model: "bedrockmantle/openai.gpt-oss-20b",
235
+ input: "Write a haiku about container ships.",
236
+ max_output_tokens: 512,
237
+ });
238
+ ```
239
+
240
+ The GPT-OSS models reason before they answer, so give Responses calls a generous
241
+ `max_output_tokens`—a small cap can complete the response with an empty
242
+ `output`.
243
+
244
+ Setting `store: true` persists the response into Bedrock's session storage,
245
+ which is what makes the management operations—`GET /v1/responses/{id}`,
246
+ `DELETE /v1/responses/{id}`, and `GET /v1/responses/{id}/input_items`—useful.
247
+ Two Mantle-specific notes on them:
248
+
249
+ - The management operations carry no request body, so the gateway can't read a
250
+ model from the request. They work on apps whose
251
+ [policy chain](./policy-chains.mdx) includes a
252
+ [Model Filtering](../policies/ai-gateway-model-filtering-v2-inbound.mdx)
253
+ policy with an allow list (the gateway routes them with the first listed
254
+ model); without one they fail with a `400` error.
255
+ - Whether `DELETE` succeeds is a permission of your Bedrock API key (the
256
+ `bedrock-mantle:DeleteInference` action in AWS IAM—see
257
+ [AWS's Bedrock Mantle documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-mantle.html)).
258
+ When the key lacks it, the gateway forwards Mantle's `401` response.
259
+
260
+ ## Call Claude models on the Messages API
261
+
262
+ Claude models serve the native
263
+ [Anthropic Messages API](./universal-api.mdx#supported-endpoints) at
264
+ `/v1/messages`. With the Anthropic SDK, set `baseURL` to the app's URL _without_
265
+ `/v1`—the SDK appends `/v1/messages` itself—and pass the app's API key as
266
+ `authToken`, not `apiKey`:
267
+
268
+ ```ts
269
+ import Anthropic from "@anthropic-ai/sdk";
270
+
271
+ const client = new Anthropic({
272
+ baseURL:
273
+ "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e",
274
+ authToken: process.env.ZUPLO_APP_API_KEY, // apiKey would send x-api-key, which the gateway ignores
275
+ });
276
+
277
+ const message = await client.messages.create({
278
+ model: "bedrockmantle/anthropic.claude-sonnet-5",
279
+ max_tokens: 1024,
280
+ messages: [{ role: "user", content: "Say hi" }],
281
+ });
282
+ ```
283
+
284
+ The gateway forwards the request body to Mantle verbatim, so everything the
285
+ Messages API supports—tool use, system prompts, streaming—works as it does
286
+ against Anthropic directly. Client headers such as `anthropic-beta` pass
287
+ through, and `anthropic-version` defaults to `2023-06-01` when your client
288
+ doesn't send one.
289
+
290
+ ## Streaming details
291
+
292
+ Most clients never notice these, but if you parse the SSE stream yourself:
293
+
294
+ - Claude models on `/v1/chat/completions` stream gateway-translated
295
+ `chat.completion.chunk` events ending with `data: [DONE]`.
296
+ - OpenAI-compatible models stream exactly what Mantle sends. The GPT-OSS stream
297
+ ends on a usage chunk with no `[DONE]` sentinel, so don't wait for one.
298
+ - `/v1/messages` streams Anthropic's standard SSE events (`message_start`
299
+ through `message_stop`).
300
+
301
+ The gateway meters usage and cost on all of them, streamed or not.
302
+
303
+ ## Troubleshooting
304
+
305
+ **The dialog rejects your API key.** Only long-term Bedrock API keys work. A key
306
+ starting with `bedrock-api-key-` is short-term—it expires within 12 hours.
307
+ Generate a long-term key (starting with `ABSK`) in the AWS console under
308
+ **Bedrock → API keys**, and paste it without surrounding whitespace.
309
+
310
+ **A model returns `400` naming `/v1/messages` or `/v1/responses`.** The model's
311
+ family doesn't serve that endpoint: the gateway rejects an OpenAI-compatible
312
+ model on `/v1/messages`, or a Claude model on `/v1/responses`, before any
313
+ upstream call. The error names the provider type (`bedrock-mantle`), which
314
+ serves both APIs, so it can be confusing—support is per model family, per the
315
+ [endpoint table](#supported-endpoints-by-model-family).
316
+
317
+ **Mantle answers that the model doesn't support the API.** Endpoint support is
318
+ per model on the OpenAI-compatible side—the GPT-5.x models serve only
319
+ `/v1/responses`, and most other models serve only `/v1/chat/completions`. Check
320
+ the **APIs supported** row on the model's AWS model card.
321
+
322
+ **A model in the picker fails at request time.** The model picker lists Mantle's
323
+ full catalog, but availability is per region and some models are access-gated by
324
+ AWS. If a request fails upstream with a model-not-found or access error, check
325
+ the model's AWS model card for regional availability, and your AWS account's
326
+ model access for gated models such as the Claude Mythos family.
327
+
328
+ **Embeddings requests fail.** Mantle serves no embeddings models, so the model
329
+ picker lists none and `/v1/embeddings` has no Bedrock Mantle models to route to.
330
+ Use another provider for embeddings.
331
+
332
+ ## Next steps
333
+
334
+ - [AI Providers](./providers.mdx)—the capability matrix across every supported
335
+ provider.
336
+ - [Universal API](./universal-api.mdx)—the endpoints every app serves and how
337
+ model references work.
338
+ - [Managing Providers](./managing-providers.mdx)—edit models, keys, and the
339
+ region, and understand when changes deploy.
340
+ - [AI Gateway Apps](./apps.mdx)—create the apps that call your Mantle-backed
341
+ models.
342
+ - [Model Filtering policy](../policies/ai-gateway-model-filtering-v2-inbound.mdx)—control
343
+ which models each app can call.
@@ -34,7 +34,7 @@ example—won't work as a custom provider.
34
34
  Custom providers serve chat completions and embeddings only. The gateway's
35
35
  `/v1/responses` and `/v1/messages` endpoints don't accept custom-provider
36
36
  models: a `/v1/responses` request fails with a `400` error and the message
37
- `The /v1/responses endpoint is only supported by OpenAI provider. Current provider: <name>`.
37
+ `The /v1/responses endpoint is not supported by provider '<name>'`.
38
38
 
39
39
  :::
40
40
 
@@ -75,11 +75,12 @@ usage limits still apply.
75
75
  ### Multi-Provider Support
76
76
 
77
77
  Configure multiple LLM providers within a single AI Gateway project. Supported
78
- providers include OpenAI, Anthropic, Google, Mistral, xAI, and OpenAI-compatible
79
- custom providers. See [AI Providers](./providers.mdx) for the full list of
80
- providers and supported capabilities. Apps reference models as
81
- `providerName/model`—for example `openai/gpt-5-mini`—so a single app can use
82
- models from several providers.
78
+ providers include OpenAI, Anthropic, Google, Mistral, xAI, Amazon Bedrock
79
+ (through [Bedrock Mantle](./bedrock-mantle.mdx)), and OpenAI-compatible custom
80
+ providers. See [AI Providers](./providers.mdx) for the full list of providers
81
+ and supported capabilities. Apps reference models as `providerName/model`—for
82
+ example `openai/gpt-5-mini`—so a single app can use models from several
83
+ providers.
83
84
 
84
85
  ### Source-Controlled Gateway
85
86
 
@@ -51,7 +51,9 @@ To add a new AI provider to your Zuplo AI Gateway, follow these steps:
51
51
  create an API key for each provider see the
52
52
  [provider documentation](./providers.mdx). The
53
53
  [Zuplo Demo provider](./providers.mdx#zuplo-demo) asks for no API key—your
54
- gateway authenticates to the demo service itself.
54
+ gateway authenticates to the demo service itself, and
55
+ [Bedrock Mantle](./bedrock-mantle.mdx) additionally asks for an **AWS
56
+ Region** and accepts only long-term Bedrock API keys.
55
57
 
56
58
  1. Select the model or models you want to use with this provider. The available
57
59
  models will depend on the selected provider. This can be changed later.
@@ -80,8 +82,9 @@ To modify an existing provider, open
80
82
  [**Settings → AI Providers**](https://portal.zuplo.com/+/account/project/ai/settings/data-models)
81
83
  and click the **Edit** button next to the provider you want to modify.
82
84
 
83
- You can modify the API key and selected models for the provider. The **Provider
84
- Name** isn't editable: it's the routing address in every `providerName/model`
85
+ You can modify the API key and selected models for the provider—and, for
86
+ [Bedrock Mantle](./bedrock-mantle.mdx), the AWS Region. The **Provider Name**
87
+ isn't editable: it's the routing address in every `providerName/model`
85
88
  reference, so renaming it would orphan each stored reference to this provider.
86
89
  After making your changes, click **Save** to apply them.
87
90
 
@@ -19,8 +19,8 @@ Zuplo currently supports the following AI providers:
19
19
  - Google
20
20
  - Mistral
21
21
  - xAI (Grok)
22
- - [Bedrock Mantle](#bedrock-mantle)—Amazon Bedrock's compatible-APIs endpoint,
23
- serving both OpenAI-compatible and Claude models
22
+ - [Bedrock Mantle](./bedrock-mantle.mdx)—Amazon Bedrock's compatible-APIs
23
+ endpoint, serving Claude models and models from many other vendors
24
24
  - [Zuplo Demo](#zuplo-demo)—a free, keyless provider for trying the gateway
25
25
  - OpenAI-compatible [Custom Providers](./custom-providers.mdx) (such as Qwen,
26
26
  Kimi, etc)
@@ -34,7 +34,7 @@ The following capabilities are supported across providers:
34
34
  | Google | ✅ | ✅ | ❌ | ❌ |
35
35
  | Mistral | ✅ | ✅ | ❌ | ❌ |
36
36
  | xAI | ✅ | ✅ | ❌ | ❌ |
37
- | Bedrock Mantle | ✅ | | | ✅ |
37
+ | Bedrock Mantle | ✅ | | | ✅ |
38
38
  | Zuplo Demo | ✅ | ❌ | ❌ | ❌ |
39
39
  | OpenAI-compatible (Custom) | ✅ | ✅ | ❌ | ❌ |
40
40
 
@@ -43,6 +43,11 @@ the native Anthropic Messages API (`/v1/messages`). See the
43
43
  [Universal API](./universal-api.mdx#supported-endpoints) for the endpoint each
44
44
  capability maps to.
45
45
 
46
+ Bedrock Mantle's capabilities depend on the model family: Claude models serve
47
+ Messages (plus chat completions through translation), while its other models
48
+ serve chat completions and—per model—Responses. See
49
+ [Using Bedrock Mantle](./bedrock-mantle.mdx#supported-endpoints-by-model-family).
50
+
46
51
  A custom provider must serve chat completions and embeddings under a `/v1` path
47
52
  segment on its API URL, and you enter that URL as an origin root—without the
48
53
  `/v1` suffix vendors usually publish. See
@@ -55,28 +60,22 @@ name you give the provider configuration. See the
55
60
 
56
61
  ## Bedrock Mantle
57
62
 
58
- **Bedrock Mantle** is Amazon Bedrock's compatible-APIs endpoint. It serves two
59
- model families from one host and one credential, so which capabilities apply
60
- depends on the model you call:
63
+ **Bedrock Mantle** is Amazon Bedrock's compatible-APIs endpoint. One regional
64
+ endpoint and one long-term Bedrock API key serve two model families, and which
65
+ capabilities apply depends on the model you call:
61
66
 
62
- - **OpenAI-compatible models** (for example `openai.gpt-oss-120b`) serve chat
63
- completions and embeddings.
64
- - **Claude models** serve the native Anthropic Messages API, and chat
65
- completions through the gateway's translation.
67
+ - **Claude models** (for example `anthropic.claude-sonnet-5`) serve the native
68
+ Anthropic Messages API, and chat completions through the gateway's
69
+ translation.
70
+ - **OpenAI-compatible models**—everything else Mantle serves, from open-weight
71
+ models such as `openai.gpt-oss-120b` to frontier models such as
72
+ `openai.gpt-5.6-sol`—serve chat completions and, per model, the OpenAI
73
+ Responses API.
66
74
 
67
75
  Mantle endpoints are regional, so the provider dialog asks for an **AWS Region**
68
- instead of an endpoint URL and builds `https://bedrock-mantle.<region>.api.aws`
69
- from it. Enter a standard region code such as `us-east-1`.
70
-
71
- :::caution{title="Long-term API keys only"}
72
-
73
- Bedrock issues both long-term and short-term API keys, and the gateway accepts
74
- only long-term ones, which start with `ABSK`. Generate one in the AWS console
75
- under **Bedrock → API keys**. A short-term key (prefixed `bedrock-api-key-`)
76
- expires within 12 hours and is locked to the session that minted it, so the
77
- dialog rejects it rather than letting the connection break the same day.
78
-
79
- :::
76
+ instead of an endpoint URL, and it accepts only long-term Bedrock API keys,
77
+ which start with `ABSK`. For prerequisites, setup steps, code examples, and
78
+ troubleshooting, see [Using Bedrock Mantle](./bedrock-mantle.mdx).
80
79
 
81
80
  ## Zuplo Demo
82
81
 
@@ -63,9 +63,9 @@ list—so clients that can't set a model still work.
63
63
 
64
64
  ## Supported endpoints
65
65
 
66
- | Endpoint | Notes |
67
- | ---------------------- | ------------------------------------------------------ |
68
- | `/v1/chat/completions` | Chat completions for every provider |
69
- | `/v1/embeddings` | Embeddings for every provider except Anthropic |
70
- | `/v1/responses` | OpenAI Responses API, for OpenAI-backed providers |
71
- | `/v1/messages` | Anthropic Messages API, for Anthropic-backed providers |
66
+ | Endpoint | Notes |
67
+ | ---------------------- | --------------------------------------------------------------------------------------------------------------- |
68
+ | `/v1/chat/completions` | Chat completions for every provider |
69
+ | `/v1/embeddings` | Embeddings for every provider except Anthropic |
70
+ | `/v1/responses` | OpenAI Responses API: OpenAI, and [Bedrock Mantle](./bedrock-mantle.mdx) OpenAI-compatible models that serve it |
71
+ | `/v1/messages` | Anthropic Messages API: Anthropic, and [Bedrock Mantle](./bedrock-mantle.mdx) Claude models |
@@ -207,6 +207,7 @@ const config = {
207
207
  disablePlayground: false, // Disable the interactive API playground
208
208
  disableSidecar: false, // Disable the sidecar completely
209
209
  disableSecurity: true, // Disable security scheme display and playground auth (default)
210
+ disableMcpAuthInstructions: false, // Hide auth steps in the MCP server card
210
211
  showVersionSelect: "if-available", // Control version selector visibility
211
212
  expandAllTags: true, // Control initial expanded state of tag categories
212
213
  showInfoPage: true, // Always show the info page (unset = show only if a description is set)
@@ -229,6 +230,11 @@ Available options:
229
230
  - `disableSecurity`: Disable OpenAPI security scheme display (auth badges on operations, security
230
231
  schemes section on the info page, and the Authorize dialog in the playground). Disabled by default
231
232
  (`true`). Set to `false` to enable security scheme support
233
+ - `disableMcpAuthInstructions`: Hide the authentication instructions on
234
+ [MCP server](../guides/mcp-servers.md) endpoints. The MCP card normally derives a credential
235
+ header from the operation's security scheme and shows it in every install snippet. Set to `true`
236
+ to render the server as unauthenticated instead — no header snippets and no "replace
237
+ `YOUR_API_KEY`" steps
232
238
  - `showVersionSelect`: Control version selector visibility
233
239
  - `"if-available"`: Show version selector only when multiple versions exist (default)
234
240
  - `"always"`: Always show version selector (disabled if only one version)
@@ -260,6 +266,7 @@ const config = {
260
266
  disablePlayground: false, // Disable the interactive API playground
261
267
  disableSidecar: false, // Disable the sidecar completely
262
268
  disableSecurity: true, // Disable security scheme display and playground auth (default)
269
+ disableMcpAuthInstructions: false, // Hide auth steps in the MCP server card
263
270
  showVersionSelect: "if-available", // Control version selector visibility
264
271
  expandAllTags: false, // Control initial expanded state of tag categories
265
272
  showInfoPage: true, // Always show the info page (unset = show only if a description is set)
@@ -130,5 +130,14 @@ When detected, the operation page shows:
130
130
  The standard method badge, request body, parameters, and sidecar panels are hidden for MCP
131
131
  endpoints.
132
132
 
133
+ When the extension carries `security` and `securitySchemes`, the card also documents the credential
134
+ header and adds it to every install snippet. Set the
135
+ [`disableMcpAuthInstructions`](/dev-portal/zudoku/configuration/api-reference#options) API option to render the
136
+ server as unauthenticated instead.
137
+
133
138
  For a full walkthrough including Dev Portal configuration, see the
134
139
  [Documenting MCP Servers guide](/docs/dev-portal/documenting-mcp-servers).
140
+
141
+ If a document describes several MCP servers, mark it with
142
+ [`x-zudoku-type: mcp-catalog`](./x-zudoku-type) to render them as a searchable catalog instead of
143
+ individual operation pages.
@@ -0,0 +1,80 @@
1
+ ---
2
+ title: x-zudoku-type
3
+ sidebar_icon: layout-grid
4
+ ---
5
+
6
+ Use `x-zudoku-type` to change how Dev Portal renders an entire OpenAPI document, instead of the default
7
+ REST API view.
8
+
9
+ ## Location
10
+
11
+ The extension is added at the **Root Object** level — the outermost level of the OpenAPI
12
+ description.
13
+
14
+ | Option | Type | Description |
15
+ | --------------- | -------- | -------------------------------------- |
16
+ | `x-zudoku-type` | `string` | The renderer to use for this document. |
17
+
18
+ | Value | Renders as |
19
+ | --------------- | --------------------------------------------------------------- |
20
+ | `"mcp-catalog"` | A searchable, filterable catalog of the document's MCP servers. |
21
+
22
+ Values Dev Portal does not recognise are ignored with a build warning, and the document falls back to
23
+ the default API view. This keeps a description written for a newer version of Dev Portal building
24
+ against an older one.
25
+
26
+ ## `mcp-catalog`
27
+
28
+ A catalog document renders as a **single page** listing every operation marked with
29
+ [`x-mcp-server`](./x-mcp-server) as a card, in one grid that can be searched and filtered by tag.
30
+ Selecting a card opens the server's install instructions and its tool list.
31
+
32
+ Because the whole document becomes a catalog, a few things change:
33
+
34
+ - **Only MCP servers are rendered.** Operations without `x-mcp-server` are not shown, and the
35
+ document contributes no sidebar entries, tag pages, or schema page. If you also want to document
36
+ plain REST endpoints, put them in a separate OpenAPI document with its own `apis` entry.
37
+ - **Tags become filters** rather than pages or headings. Each tag appears as a filter chip and as a
38
+ badge on its servers' cards; the grid itself stays flat. Servers without a tag are filed under
39
+ "Other". The active filter is kept in the URL as `?tag=`, so a filtered view can be linked.
40
+ - **Only the latest version is rendered.** Catalog documents do not support version switching; a
41
+ versioned API marked as a catalog warns at build time.
42
+
43
+ The extension is read while your schema is processed, so it applies to `type: "file"` and
44
+ `type: "raw"` APIs. `type: "url"` schemas are fetched in the browser after routing is decided, so
45
+ the flag has no effect on them.
46
+
47
+ ## Example
48
+
49
+ ```yaml
50
+ openapi: 3.1.0
51
+ x-zudoku-type: mcp-catalog
52
+ info:
53
+ title: Employee MCP Servers
54
+ version: 1.0.0
55
+ servers:
56
+ - url: https://mcp.example.com
57
+ tags:
58
+ - name: CRM & Customer Operations
59
+ paths:
60
+ /v1/salesforce/sales-cloud/mcp:
61
+ post:
62
+ summary: Salesforce Sales Cloud
63
+ description: Read-only access to accounts, opportunities and contacts.
64
+ tags:
65
+ - CRM & Customer Operations
66
+ x-mcp-server:
67
+ name: salesforce-sales-cloud
68
+ version: 1.0.0
69
+ tools:
70
+ - name: searchAccounts
71
+ description: Find accounts by name, domain or owner.
72
+ ```
73
+
74
+ This renders one catalog page with a single `Salesforce Sales Cloud` card, reachable via a
75
+ `CRM & Customer Operations` filter chip. The card opens install snippets for Claude, ChatGPT,
76
+ Cursor, VS Code and Codex, alongside the server's tools.
77
+
78
+ ## Related
79
+
80
+ - [`x-mcp-server`](./x-mcp-server) — mark an individual operation as an MCP server
@@ -98,6 +98,7 @@
98
98
  | set-status-outbound | Set Status Code | Sets the status code on the on the outgoing response. | api-gateway |
99
99
  | set-upstream-api-key-inbound | Set Upstream API Key | Sets a single header on the incoming request, typically used to attach an API key for the upstream service. A more directed version of the `SetHeadersInboundPolicy` that defaults the header name to `Authorization` and is intended to be used with an `$env()` reference for the value. | api-gateway |
100
100
  | sleep-inbound | Sleep / Delay | Add a delay to the incoming request. Useful for testing. | api-gateway |
101
+ | ai-gateway-smart-router-inbound | Smart Router | Smart Router Inbound Policy for the AI Gateway | ai-gateway |
101
102
  | stripe-webhook-verification-inbound | Stripe Webhook Auth | The Stripe Webhook policy validates the authenticity of an incoming Stripe webhook. | api-gateway |
102
103
  | supabase-jwt-auth-inbound | Supabase JWT Auth | The Supabase JWT Authentication policy supports user JWT tokens created by Supabase. | api-gateway |
103
104
  | traffic-splitting-inbound | Traffic Splitting | Splits traffic randomly across a set of weighted base paths. On each request one base path is selected (weighted by `weight`) and written to the request custom context at `customOutputProperty`. Reference it from a later URL Rewrite `rewritePattern` or URL Forward `baseUrl`, e.g. `${context.custom.trafficSplitting.basePath}`. | api-gateway |
@@ -283,6 +283,178 @@ under `permissions` fail closed:
283
283
  Omit `permissions`, or either flag, when the portal does not need to record that
284
284
  constraint on the stored chain.
285
285
 
286
+ ## Configure expression budget rules
287
+
288
+ `budgetRules` in the application configuration can apply a budget to a value
289
+ selected from the request. Each expression rule requires one or more meter rows:
290
+
291
+ ```json
292
+ {
293
+ "budgetRules": [
294
+ {
295
+ "budgetBy": "expression",
296
+ "expression": "request.user.data[\"team-id\"]",
297
+ "meters": [
298
+ {
299
+ "meter": "requests",
300
+ "period": "daily",
301
+ "value": 10000,
302
+ "action": "warn"
303
+ },
304
+ {
305
+ "meter": "requests",
306
+ "period": "daily",
307
+ "value": 12000,
308
+ "action": "block"
309
+ }
310
+ ]
311
+ }
312
+ ]
313
+ }
314
+ ```
315
+
316
+ | Field | Supported values |
317
+ | ---------- | ---------------------------------------------------- |
318
+ | `budgetBy` | `expression` |
319
+ | `meter` | `cost`, `requests`, `tokens` |
320
+ | `period` | `hourly`, `daily`, `weekly`, `monthly` |
321
+ | `action` | `warn` records a warning; `block` returns HTTP `429` |
322
+ | `value` | A finite number greater than or equal to zero |
323
+
324
+ A rule cannot repeat the same meter, period, and action. When a meter and period
325
+ have both actions, the `warn` value must be less than the `block` value. The
326
+ policy does not treat entries whose `budgetBy` is `app` as expression budget
327
+ rules.
328
+
329
+ ### Supported expressions
330
+
331
+ Expressions read one string or safe integer from request-time data. Use these
332
+ canonical forms when writing application configuration:
333
+
334
+ | Source | Canonical expression example |
335
+ | --------------------------- | ------------------------------------------- |
336
+ | Request URL or method | `request.url`, `request.method` |
337
+ | One request header | `request.headers.get("x-customer-id")` |
338
+ | Authenticated subject | `request.user.sub` |
339
+ | User data | `request.user.data.profile.organizationId` |
340
+ | Query parameter | `request.query.customerId` |
341
+ | Query parameter (map form) | `request.searchParams.customerId` |
342
+ | Route parameter | `request.params.productId` |
343
+ | Custom context data | `context.custom.account.id` |
344
+ | Route data | `context.route.path` |
345
+ | Incoming request data | `context.incomingRequestProperties.country` |
346
+ | Request or context identity | `request.user.sub`, `context.requestId` |
347
+ | A key requiring brackets | `request.user.data["team-id"]` |
348
+
349
+ The `expression` field contains the complete expression. Do not wrap it in an
350
+ interpolation marker such as `${...}`. The runtime resolves it for every request
351
+ after the application's inbound policy chain finishes. Updating an application's
352
+ configuration changes which value later requests select without rebuilding or
353
+ redeploying the gateway.
354
+
355
+ JSON encoding and expression syntax are separate. When writing raw JSON, escape
356
+ the double quotes required by a bracket segment:
357
+
358
+ ```json
359
+ {
360
+ "expression": "request.user.data[\"team-id\"]"
361
+ }
362
+ ```
363
+
364
+ After a JSON parser decodes this value, the expression is
365
+ `request.user.data["team-id"]`; the backslashes are not part of its identity.
366
+ Code that creates application configuration should build an ordinary string and
367
+ let its JSON serializer handle the transport escaping:
368
+
369
+ ```ts
370
+ const expression = 'request.user.data["team-id"]';
371
+ const body = JSON.stringify({ expression });
372
+ ```
373
+
374
+ Header expressions are terminal. Header names are case-insensitive; use
375
+ lowercase in stored expressions. The parser accepts the canonical
376
+ `request.headers.get("content-type")` form and the equivalent
377
+ `request.headers.content-type` and `request.headers["content-type"]` forms. The
378
+ canonical form keeps header access visually distinct from an ordinary object
379
+ property.
380
+
381
+ A dot property starts with an ASCII letter, `_`, or `$`, followed by those
382
+ characters, ASCII digits, or hyphens. Canonical rendering uses quoted brackets
383
+ for keys containing `$` or `-`, or any other non-identifier character. For
384
+ example, `context.custom.team-id` is accepted and its canonical form is
385
+ `context.custom["team-id"]`. Single- and double-quoted bracket properties are
386
+ accepted. Whitespace immediately inside brackets is accepted but is not
387
+ canonical.
388
+
389
+ The selectable data model is:
390
+
391
+ | Root | Selectable properties |
392
+ | --------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
393
+ | `request` | `url`, `method`, `headers`, `user.sub`, `user.data`, `query`, `searchParams`, `params`, and the scalar Fetch request metadata `bodyUsed`, `cache`, `credentials`, `destination`, `integrity`, `keepalive`, `mode`, `redirect`, `referrer`, and `referrerPolicy` |
394
+ | `context` | `contextId`, `requestId`, `custom`, `route`, and `incomingRequestProperties` |
395
+
396
+ `request.body`, `context.log`, policy invocation methods, and every other host
397
+ object are outside the selectable data model. The evaluator receives a
398
+ plain-data snapshot of the selected root. It never receives the live request or
399
+ context object.
400
+
401
+ Expressions have these limits:
402
+
403
+ - The expression is at most 1,024 UTF-8 bytes and contains well-formed Unicode.
404
+ - Quoted keys are non-empty and do not contain double quotes, backslashes,
405
+ control characters, U+2028, or U+2029. String escapes are not supported.
406
+ - Canonical expressions contain no syntax whitespace. Spaces inside a quoted key
407
+ are part of the key.
408
+ - Wildcards, recursive descent, filters, numeric bracket indexes, calls other
409
+ than the terminal header `.get("name")`, function arguments, and roots other
410
+ than `request` and `context` are not supported. For example,
411
+ `request.url.substring(5, 1)` is not a supported expression.
412
+ - Traversal reads own properties from plain objects and keys from declared Maps.
413
+ It does not traverse arrays, class instances, inherited properties, or the
414
+ property names `__proto__`, `constructor`, and `prototype`.
415
+
416
+ Expressions observe values after the application's inbound policy chain
417
+ finishes. An authentication policy can populate `request.user`. A custom policy
418
+ earlier in the chain can derive a value and place it in a request header:
419
+
420
+ ```ts
421
+ import { ZuploContext, ZuploRequest } from "@zuplo/runtime";
422
+
423
+ export default function setBudgetDimension(
424
+ request: ZuploRequest,
425
+ context: ZuploContext
426
+ ): ZuploRequest {
427
+ const headers = new Headers(request.headers);
428
+ headers.set(
429
+ "x-budget-customer",
430
+ `${context.route.path}:${request.user?.sub ?? "anonymous"}`
431
+ );
432
+ return new ZuploRequest(request, { headers });
433
+ }
434
+ ```
435
+
436
+ The corresponding expression is `request.headers.get("x-budget-customer")`.
437
+ Configuration code should create that ordinary string and pass the enclosing
438
+ object to `JSON.stringify`; the serializer adds the JSON transport escaping.
439
+
440
+ The expression chooses one value; it does not concatenate values, run
441
+ JavaScript, or evaluate conditions. Compute composite values in an earlier
442
+ policy and store the result in a supported request or context property.
443
+
444
+ The selected value must be a string or a safe integer. The runtime converts the
445
+ value to well-formed NFC Unicode and ignores it when it exceeds 256 UTF-8 bytes
446
+ or contains control characters, U+2028, or U+2029. A missing or invalid value
447
+ does not contribute to the rule for that request.
448
+
449
+ The exact stored expression is the rule identity and analytics dimension name.
450
+ Store its authored bytes unchanged. Different accepted spellings, such as single
451
+ quotes and double quotes, identify different rules even when they select the
452
+ same value.
453
+
454
+ Malformed budget rules are logged and ignored individually. An unsupported
455
+ expression is also logged and omitted from dimension capture and quota tokens;
456
+ other valid rules continue to run.
457
+
286
458
  ## Write a custom policy for the chain
287
459
 
288
460
  A chain entry can run any declared custom policy. The policy uses the standard
@@ -155,10 +155,14 @@ reported as warnings because they cannot match a request.
155
155
  | Policy has no rules for the route capability | 403 response explaining which capability to configure. |
156
156
  | Another inbound policy already selected routing | Model Filtering leaves that selection unchanged. |
157
157
 
158
- Native routes also enforce provider type. `/v1/responses` requires a Provider
159
- Name backed by OpenAI, and `/v1/messages` requires a Provider Name backed by
160
- Anthropic. Provider Names may be custom labels; validation uses the provider
161
- type configured for that Provider Assignment.
158
+ Native routes also enforce the wire format. `/v1/responses` requires a selection
159
+ served on the OpenAI API, and `/v1/messages` one served on the Anthropic API. A
160
+ selection qualifies when its Provider Assignment is backed by that provider
161
+ type, or when the provider catalog declares the selected model's dialect as that
162
+ format — Bedrock Mantle serves its `openai`-dialect models on `/v1/responses`
163
+ and its `anthropic`-dialect models on `/v1/messages`. Provider Names may be
164
+ custom labels; validation never infers anything from the label or the model
165
+ name, and a model the catalog does not declare is rejected.
162
166
 
163
167
  For example, this embedding request is evaluated against `models.embeddings`:
164
168
 
@@ -0,0 +1,99 @@
1
+ # AI Gateway Smart Router
2
+
3
+ Use this policy to classify the last user prompt on Chat Completions, Responses,
4
+ and Anthropic Messages requests. It calls a dedicated AI Gateway application
5
+ (`/{classifierAppID}/v1/chat/completions`) and stores the result on
6
+ `AIGatewaySmartRouter` for later policies in the same request.
7
+
8
+ When `smartRoutingEnabled` is true, it overwrites completions routing from
9
+ `modelsByComplexity`. Place it **after** Model Filtering so an invalid client
10
+ model is still rejected before classification runs. The classified model then
11
+ replaces that selection.
12
+
13
+ Classification is optional. Timeouts, classifier errors, unreadable bodies, and
14
+ invalid options fail open: the original request is forwarded.
15
+
16
+ > **Loop prevention.** The classifier hop is an `invokeRoute` sub-request. The
17
+ > policy no-ops when `context.parentContext` is set, so the classifier app can
18
+ > share the same route chain without classifying its own request.
19
+
20
+ ## Required options
21
+
22
+ - `classifierAppID` — AI Gateway application id whose chat/completions route
23
+ runs the classifier.
24
+ - `classifierAppApiKey` — bearer token for that app. Use
25
+ `$env(CLASSIFIER_APP_API_KEY)`.
26
+ - `classifierModel` — `providerName/model` sent on the classifier request.
27
+
28
+ Omit `intents` and `classifierPrompt` to use the built-in taxonomy (code,
29
+ summarization, translation, qa, conversation, classification, creative_writing,
30
+ agentic, document_qa, other) and the built-in system prompt. Include
31
+ `{{intents}}` in a custom prompt to inject the configured intent list.
32
+
33
+ ## Example
34
+
35
+ ```json
36
+ {
37
+ "name": "ai-gateway-smart-router-inbound",
38
+ "policyType": "ai-gateway-smart-router",
39
+ "handler": {
40
+ "export": "AIGatewaySmartRouterInboundPolicy",
41
+ "module": "$import(@zuplo/runtime)",
42
+ "options": {
43
+ "classifierAppID": "$env(CLASSIFIER_APP_ID)",
44
+ "classifierAppApiKey": "$env(CLASSIFIER_APP_API_KEY)",
45
+ "classifierModel": "openai/gpt-4o-mini",
46
+ "smartRoutingEnabled": true,
47
+ "modelsByComplexity": {
48
+ "low": "openai/gpt-4o-mini",
49
+ "medium": "openai/gpt-4o",
50
+ "high": "openai/gpt-5"
51
+ }
52
+ }
53
+ }
54
+ }
55
+ ```
56
+
57
+ ## Policy order
58
+
59
+ ```text
60
+ Model Filtering -> Smart Router -> Fallback Model -> AI Gateway handler
61
+ ```
62
+
63
+ Smart Router always `set()`s routing when smart routing applies, even if
64
+ filtering already selected a model. Filtering skips when routing is already set,
65
+ so putting this policy first would also skip allow-list checks on the client's
66
+ original model.
67
+
68
+ ## Read the result from custom code
69
+
70
+ ```typescript
71
+ import { AIGatewaySmartRouter } from "@zuplo/runtime";
72
+
73
+ const result = AIGatewaySmartRouter.get(context);
74
+ if (result?.profile.intent === "code") {
75
+ // ...
76
+ }
77
+ ```
78
+
79
+ ## How prompt text is chosen
80
+
81
+ The policy reads the typed body from the route (`getFormat` / `getRequestBody`),
82
+ then takes the last real user text:
83
+
84
+ - `/v1/chat/completions` — OpenAI chat `messages[]`, even when the downstream
85
+ provider is Anthropic or Google (Zuplo translates below the policy chain).
86
+ - `/v1/responses` — OpenAI Responses `input`.
87
+ - `/v1/messages` — native Anthropic `messages[]`.
88
+
89
+ Tool follow-up turns (`tool` / `function` roles, `tool_result` blocks,
90
+ `function_call_output` items) are skipped so a prior user message is classified
91
+ instead. Embeddings and other non-AI paths are skipped.
92
+
93
+ ## Fail-open behavior
94
+
95
+ The policy never 500s the user request for an internal classifier problem.
96
+ Invalid options, classifier timeouts, empty classifier responses, and smart
97
+ routing catalog errors are logged and the original request continues. Chat
98
+ Completions, Responses, and Anthropic Messages are classified automatically.
99
+ Embeddings and other non-AI paths are skipped.
@@ -0,0 +1,4 @@
1
+ Classifies the last user prompt by calling a dedicated classifier AI Gateway
2
+ app, stores the result on `AIGatewaySmartRouter` for later policies, and
3
+ optionally routes completions by classified complexity. Classifier failures fail
4
+ open so the original request still reaches the model.
@@ -0,0 +1,204 @@
1
+ {
2
+ "$schema": "https://json-schema.org/draft-07/schema",
3
+ "$id": "https://cdn.zuplo.com/policies/runtime/schemas/ai-gateway-smart-router-inbound.json",
4
+ "type": "object",
5
+ "title": "Smart Router",
6
+ "isDeprecated": false,
7
+ "isPaidAddOn": false,
8
+ "isEnterprise": false,
9
+ "isInternal": false,
10
+ "isBeta": false,
11
+ "isHidden": false,
12
+ "requiresAI": true,
13
+ "policyType": "ai-gateway-smart-router",
14
+ "products": ["ai-gateway"],
15
+ "description": "Smart Router Inbound Policy for the AI Gateway",
16
+ "deprecatedMessage": "",
17
+ "required": ["handler"],
18
+ "properties": {
19
+ "handler": {
20
+ "type": "object",
21
+ "default": {},
22
+ "required": ["export", "module", "options"],
23
+ "properties": {
24
+ "export": {
25
+ "const": "AIGatewaySmartRouterInboundPolicy",
26
+ "description": "The name of the exported type"
27
+ },
28
+ "module": {
29
+ "const": "$import(@zuplo/runtime)",
30
+ "description": "The module containing the policy"
31
+ },
32
+ "options": {
33
+ "x-zuplo-policy-type": "ai-gateway-smart-router",
34
+ "type": "object",
35
+ "title": "AIGatewaySmartRouterInboundPolicyOptions",
36
+ "description": "Options for the Smart Router policy: classify the last user prompt with a dedicated classifier AI Gateway app, then optionally route by complexity.",
37
+ "additionalProperties": false,
38
+ "required": [
39
+ "classifierAppID",
40
+ "classifierAppApiKey",
41
+ "classifierModel"
42
+ ],
43
+ "examples": [
44
+ {
45
+ "classifierAppID": "$env(CLASSIFIER_APP_ID)",
46
+ "classifierAppApiKey": "$env(CLASSIFIER_APP_API_KEY)",
47
+ "classifierModel": "openai/gpt-4o-mini",
48
+ "smartRoutingEnabled": true,
49
+ "modelsByComplexity": {
50
+ "low": "openai/gpt-4o-mini",
51
+ "medium": "openai/gpt-4o",
52
+ "high": "openai/gpt-5"
53
+ }
54
+ }
55
+ ],
56
+ "properties": {
57
+ "classifierAppID": {
58
+ "type": "string",
59
+ "title": "Classifier App ID",
60
+ "description": "The AI Gateway application id whose `/v1/chat/completions` route runs the classifier. The policy calls `/{classifierAppID}/v1/chat/completions` via `context.invokeRoute`.",
61
+ "examples": ["config_1234"]
62
+ },
63
+ "classifierAppApiKey": {
64
+ "type": "string",
65
+ "title": "Classifier App API Key",
66
+ "description": "API key sent as `Authorization: Bearer` when invoking the classifier app.",
67
+ "examples": ["$env(CLASSIFIER_APP_API_KEY)"]
68
+ },
69
+ "classifierModel": {
70
+ "type": "string",
71
+ "title": "Classifier Model",
72
+ "description": "The `providerName/model` reference the classifier chat/completions request should use.",
73
+ "pattern": "^[^/\\s]+/.+$",
74
+ "examples": ["openai/gpt-4o-mini"]
75
+ },
76
+ "smartRoutingEnabled": {
77
+ "type": "boolean",
78
+ "title": "Smart Routing Enabled",
79
+ "description": "When true, apply model routing from `modelsByComplexity` when confidence is high enough and the intent is known. Classification still runs when false.",
80
+ "default": false
81
+ },
82
+ "modelsByComplexity": {
83
+ "type": "object",
84
+ "title": "Models By Complexity",
85
+ "description": "Optional `providerName/model` routing by classified complexity. Used only when `smartRoutingEnabled` is true.",
86
+ "additionalProperties": false,
87
+ "properties": {
88
+ "low": {
89
+ "type": "string",
90
+ "title": "Low Complexity Model",
91
+ "description": "Model used for prompts classified as low complexity.",
92
+ "pattern": "^[^/\\s]+/.+$",
93
+ "examples": ["openai/gpt-4o-mini"]
94
+ },
95
+ "medium": {
96
+ "type": "string",
97
+ "title": "Medium Complexity Model",
98
+ "description": "Model used for prompts classified as medium complexity.",
99
+ "pattern": "^[^/\\s]+/.+$",
100
+ "examples": ["openai/gpt-4o"]
101
+ },
102
+ "high": {
103
+ "type": "string",
104
+ "title": "High Complexity Model",
105
+ "description": "Model used for prompts classified as high complexity.",
106
+ "pattern": "^[^/\\s]+/.+$",
107
+ "examples": ["openai/gpt-5"]
108
+ }
109
+ }
110
+ },
111
+ "intents": {
112
+ "type": "array",
113
+ "title": "Intents",
114
+ "description": "Labels the classifier may assign. Used to build the JSON schema enum and injected into `classifierPrompt` at `{{intents}}`. Omit to use the built-in taxonomy (code, summarization, translation, qa, conversation, classification, creative\\_writing, agentic, document\\_qa, other).",
115
+ "minItems": 1,
116
+ "items": {
117
+ "type": "object",
118
+ "additionalProperties": false,
119
+ "required": ["id", "description"],
120
+ "properties": {
121
+ "id": {
122
+ "type": "string",
123
+ "title": "Intent ID",
124
+ "description": "Stable intent identifier written into the classifier JSON schema enum.",
125
+ "minLength": 1,
126
+ "examples": ["code"]
127
+ },
128
+ "description": {
129
+ "type": "string",
130
+ "title": "Intent Description",
131
+ "description": "Short description shown to the classifier for this intent.",
132
+ "examples": [
133
+ "Write, edit, refactor, debug, or review source code."
134
+ ]
135
+ }
136
+ }
137
+ }
138
+ },
139
+ "classifierPrompt": {
140
+ "title": "Classifier Prompt",
141
+ "description": "System prompt for the classifier. If it includes `{{intents}}`, that placeholder is replaced with the configured intent id/description list. A string or an array of lines (joined with newlines). Omit to use the built-in classifier prompt.",
142
+ "oneOf": [
143
+ {
144
+ "type": "string",
145
+ "minLength": 1
146
+ },
147
+ {
148
+ "type": "array",
149
+ "minItems": 1,
150
+ "items": {
151
+ "type": "string"
152
+ }
153
+ }
154
+ ]
155
+ },
156
+ "minConfidenceForRouting": {
157
+ "type": "number",
158
+ "title": "Minimum Confidence For Routing",
159
+ "description": "Minimum confidence (0–1) required before applying model routing. Unknown intents are capped strictly below this threshold.",
160
+ "minimum": 0,
161
+ "maximum": 1,
162
+ "default": 0.5,
163
+ "x-advanced": true
164
+ },
165
+ "classifierTimeoutMs": {
166
+ "type": "integer",
167
+ "title": "Classifier Timeout (ms)",
168
+ "description": "How long to wait for the classifier `invokeRoute` call before skipping classification and forwarding the original request.",
169
+ "minimum": 1,
170
+ "maximum": 120000,
171
+ "default": 8000,
172
+ "x-advanced": true
173
+ },
174
+ "maxPromptChars": {
175
+ "type": "integer",
176
+ "title": "Max Prompt Characters",
177
+ "description": "Maximum characters of user prompt sent to the classifier. Longer prompts are truncated.",
178
+ "minimum": 1,
179
+ "default": 8000,
180
+ "x-advanced": true
181
+ }
182
+ }
183
+ }
184
+ },
185
+ "examples": [
186
+ {
187
+ "export": "AIGatewaySmartRouterInboundPolicy",
188
+ "module": "$import(@zuplo/runtime)",
189
+ "options": {
190
+ "classifierAppID": "$env(CLASSIFIER_APP_ID)",
191
+ "classifierAppApiKey": "$env(CLASSIFIER_APP_API_KEY)",
192
+ "classifierModel": "openai/gpt-4o-mini",
193
+ "smartRoutingEnabled": true,
194
+ "modelsByComplexity": {
195
+ "low": "openai/gpt-4o-mini",
196
+ "medium": "openai/gpt-4o",
197
+ "high": "openai/gpt-5"
198
+ }
199
+ }
200
+ }
201
+ ]
202
+ }
203
+ }
204
+ }
@@ -54,8 +54,8 @@ The threshold is configurable via `minConfidence` (default `0.5`). Lowering it
54
54
  surfaces context-dependent entities without their context words. Raising it is
55
55
  not a precision filter: any recognizer whose base confidence falls below the
56
56
  threshold stops matching except near its context words, and most
57
- checksum-validated recognizers run at `0.85`, so high thresholds disable most
58
- of the catalog. To narrow detection, scope the `entities` list instead.
57
+ checksum-validated recognizers run at `0.85`, so high thresholds disable most of
58
+ the catalog. To narrow detection, scope the `entities` list instead.
59
59
 
60
60
  ## Custom patterns
61
61
 
@@ -68,20 +68,20 @@ JSON (for example `\\d` to match a digit).
68
68
  ## Allowed values
69
69
 
70
70
  Use `allowValues` to exempt specific values that would otherwise match — for
71
- example the Stripe test card number `4242 4242 4242 4242` in a payments
72
- sandbox, or a public support email address. Entries are compared against the
73
- exact matched text (case-sensitive, separators included), so list every
74
- written form you want to allow.
71
+ example the Stripe test card number `4242 4242 4242 4242` in a payments sandbox,
72
+ or a public support email address. Entries are compared against the exact
73
+ matched text (case-sensitive, separators included), so list every written form
74
+ you want to allow.
75
75
 
76
76
  ## Content types
77
77
 
78
78
  Only text-based bodies (JSON, XML, form-encoded, and `text/*`) are scanned;
79
79
  binary bodies pass through untouched. Server-sent event streams
80
- (`text/event-stream`) are excluded from the default allow-list: scanning
81
- buffers the whole body, which would hold back every event until the stream
82
- ends. Override the allow-list with the `contentTypes` option if you need to
83
- scan a different set of content types (including `text/event-stream`, if you
84
- accept the buffering).
80
+ (`text/event-stream`) are excluded from the default allow-list: scanning buffers
81
+ the whole body, which would hold back every event until the stream ends.
82
+ Override the allow-list with the `contentTypes` option if you need to scan a
83
+ different set of content types (including `text/event-stream`, if you accept the
84
+ buffering).
85
85
 
86
86
  ## Configuration
87
87
 
@@ -54,8 +54,8 @@ The threshold is configurable via `minConfidence` (default `0.5`). Lowering it
54
54
  surfaces context-dependent entities without their context words. Raising it is
55
55
  not a precision filter: any recognizer whose base confidence falls below the
56
56
  threshold stops matching except near its context words, and most
57
- checksum-validated recognizers run at `0.85`, so high thresholds disable most
58
- of the catalog. To narrow detection, scope the `entities` list instead.
57
+ checksum-validated recognizers run at `0.85`, so high thresholds disable most of
58
+ the catalog. To narrow detection, scope the `entities` list instead.
59
59
 
60
60
  ## Custom patterns
61
61
 
@@ -68,21 +68,21 @@ JSON (for example `\\d` to match a digit).
68
68
  ## Allowed values
69
69
 
70
70
  Use `allowValues` to exempt specific values that would otherwise match — for
71
- example the Stripe test card number `4242 4242 4242 4242` in a payments
72
- sandbox, or a public support email address. Entries are compared against the
73
- exact matched text (case-sensitive, separators included), so list every
74
- written form you want to allow.
71
+ example the Stripe test card number `4242 4242 4242 4242` in a payments sandbox,
72
+ or a public support email address. Entries are compared against the exact
73
+ matched text (case-sensitive, separators included), so list every written form
74
+ you want to allow.
75
75
 
76
76
  ## Content types
77
77
 
78
78
  Only text-based bodies (JSON, XML, form-encoded, and `text/*`) are scanned;
79
79
  binary bodies pass through untouched. Server-sent event streams
80
- (`text/event-stream`) are excluded from the default allow-list: scanning
81
- buffers the whole response, which would hold back every event until the
82
- stream ends — for an upstream SSE response, time-to-first-byte becomes the
83
- full stream duration. Override the allow-list with the `contentTypes` option
84
- if you need to scan a different set of content types (including
85
- `text/event-stream`, if you accept the buffering).
80
+ (`text/event-stream`) are excluded from the default allow-list: scanning buffers
81
+ the whole response, which would hold back every event until the stream ends —
82
+ for an upstream SSE response, time-to-first-byte becomes the full stream
83
+ duration. Override the allow-list with the `contentTypes` option if you need to
84
+ scan a different set of content types (including `text/event-stream`, if you
85
+ accept the buffering).
86
86
 
87
87
  ## Configuration
88
88
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "zuplo",
3
- "version": "7.5.2",
3
+ "version": "7.5.4",
4
4
  "type": "module",
5
5
  "description": "The programmable API Gateway",
6
6
  "author": "Zuplo, Inc.",
@@ -19,9 +19,9 @@
19
19
  "zuplo": "zuplo.js"
20
20
  },
21
21
  "dependencies": {
22
- "@zuplo/cli": "7.5.2",
23
- "@zuplo/core": "7.5.2",
24
- "@zuplo/runtime": "7.5.2",
25
- "@zuplo/test": "7.5.2"
22
+ "@zuplo/cli": "7.5.4",
23
+ "@zuplo/core": "7.5.4",
24
+ "@zuplo/runtime": "7.5.4",
25
+ "@zuplo/test": "7.5.4"
26
26
  }
27
27
  }