zuplo 7.1.9 → 7.1.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/docs/ai-gateway/apps.mdx +27 -10
- package/docs/ai-gateway/custom-policies.mdx +4 -4
- package/docs/ai-gateway/custom-providers.mdx +51 -5
- package/docs/ai-gateway/fallback.mdx +2 -2
- package/docs/ai-gateway/getting-started.mdx +51 -30
- package/docs/ai-gateway/integrations/ai-sdk.mdx +3 -3
- package/docs/ai-gateway/introduction.mdx +18 -5
- package/docs/ai-gateway/managing-apps.mdx +39 -24
- package/docs/ai-gateway/managing-providers.mdx +18 -3
- package/docs/ai-gateway/managing-teams.mdx +30 -23
- package/docs/ai-gateway/policies/akamai-ai-firewall.mdx +20 -19
- package/docs/ai-gateway/policies/authentication.mdx +1 -1
- package/docs/ai-gateway/policies/comet-opik-tracing.mdx +2 -2
- package/docs/ai-gateway/policies/galileo-tracing.mdx +2 -2
- package/docs/ai-gateway/policies/model-filtering.mdx +1 -1
- package/docs/ai-gateway/policy-chains.mdx +4 -3
- package/docs/ai-gateway/policy-templates.mdx +7 -5
- package/docs/ai-gateway/providers.mdx +40 -7
- package/docs/ai-gateway/source-control.mdx +5 -4
- package/docs/ai-gateway/teams.mdx +14 -9
- package/docs/ai-gateway/usage-limits.mdx +12 -12
- package/docs/articles/accounts/roles-and-permissions.mdx +40 -4
- package/docs/policies/akamai-ai-firewall/schema.json +9 -0
- package/package.json +5 -5
package/docs/ai-gateway/apps.mdx
CHANGED
|
@@ -9,13 +9,21 @@ description:
|
|
|
9
9
|
An app represents one caller of your AI Gateway—a service, an agent, or a
|
|
10
10
|
feature in a larger codebase. A support chatbot on your website is one app; the
|
|
11
11
|
batch job that summarizes tickets overnight is another. Each app belongs to a
|
|
12
|
-
[team](./teams.mdx)
|
|
12
|
+
[team](./teams.mdx), which controls dashboard access to the app. Access to AI
|
|
13
|
+
providers is governed separately by the user's Zuplo account and Zuplo project
|
|
14
|
+
roles. A Zuplo account contains one or more projects, and the AI Gateway is the
|
|
15
|
+
Zuplo project that contains these providers, teams, and apps. See
|
|
16
|
+
[Role Permissions](../articles/accounts/roles-and-permissions.mdx).
|
|
13
17
|
|
|
14
18
|
Each app has three things of its own:
|
|
15
19
|
|
|
16
|
-
- **An API URL**—the endpoint the app's code calls, shown at the top of
|
|
17
|
-
page.
|
|
18
|
-
|
|
20
|
+
- **An API URL**—the endpoint the app's code calls, shown in full at the top of
|
|
21
|
+
the app page. Expand it to see the ready-to-paste URL for each service the
|
|
22
|
+
gateway mounts under the app—**Chat Completions** (`v1/chat/completions`),
|
|
23
|
+
**Responses** (`v1/responses`, OpenAI only), and **Messages** (`v1/messages`,
|
|
24
|
+
Anthropic only)—and copy the one your client needs. The gateway attributes
|
|
25
|
+
requests made to this URL to the app, tracking usage independently per app.
|
|
26
|
+
See the [Universal API](./universal-api.mdx).
|
|
19
27
|
- **An API key**—authenticates the app's requests when the gateway runs the
|
|
20
28
|
[authentication policy](./policy-chains.mdx#authentication).
|
|
21
29
|
- **A [policy chain](./policy-chains.mdx)**—the ordered policies that run on the
|
|
@@ -32,15 +40,24 @@ validated key or from the `{app_id}` segment of the request URL, so an app
|
|
|
32
40
|
without authentication still tracks usage independently.
|
|
33
41
|
|
|
34
42
|
To find an app's API key, open the
|
|
35
|
-
[Apps](https://portal.zuplo.com/+/account/project/ai/apps) tab of your
|
|
36
|
-
Gateway project in the Zuplo Portal and select the app. The
|
|
37
|
-
|
|
43
|
+
[Apps & Teams](https://portal.zuplo.com/+/account/project/ai/apps) tab of your
|
|
44
|
+
AI Gateway project in the Zuplo Portal and select the app. The key lives on the
|
|
45
|
+
app's **API Key** tab.
|
|
46
|
+
|
|
47
|
+
:::caution{title="Rotate keys to revoke access"}
|
|
48
|
+
|
|
49
|
+
An issued app API key authenticates gateway traffic by the key itself. Changing
|
|
50
|
+
a user's dashboard permissions or removing them from a team doesn't invalidate
|
|
51
|
+
the key. To revoke access, rotate the app's API key and update authorized
|
|
52
|
+
clients with the new value.
|
|
53
|
+
|
|
54
|
+
:::
|
|
38
55
|
|
|
39
56
|
**Additional Resources**
|
|
40
57
|
|
|
41
|
-
- [Creating & Editing Apps](./managing-apps.mdx) - How to
|
|
42
|
-
|
|
58
|
+
- [Creating & Editing Apps](./managing-apps.mdx) - How to create, configure, and
|
|
59
|
+
delete apps.
|
|
43
60
|
- [Creating & Editing Teams](./managing-teams.mdx) - How to create and edit
|
|
44
61
|
teams.
|
|
45
62
|
- [Role Permissions](../articles/accounts/roles-and-permissions.mdx) - Details
|
|
46
|
-
on
|
|
63
|
+
on Zuplo account, Zuplo project, and AI Gateway team roles.
|
|
@@ -117,10 +117,10 @@ terms. By the end, one app on your gateway rejects a prompt containing
|
|
|
117
117
|
4. **Add the policy to an app's chain**
|
|
118
118
|
|
|
119
119
|
In the Zuplo Portal, open
|
|
120
|
-
[**Apps**](https://portal.zuplo.com/+/account/project/ai/apps),
|
|
121
|
-
app, and open its **Policies** tab. Click **Add
|
|
122
|
-
appears alongside the built-in policies. Add
|
|
123
|
-
chain it should run, and save.
|
|
120
|
+
[**Apps & Teams**](https://portal.zuplo.com/+/account/project/ai/apps),
|
|
121
|
+
select your app, and open its **Policies** tab. Click **Add
|
|
122
|
+
Policy**—`content-filter` now appears alongside the built-in policies. Add
|
|
123
|
+
it, drag it to where in the chain it should run, and save.
|
|
124
124
|
|
|
125
125
|
:::tip{title="Where in the chain?"}
|
|
126
126
|
|
|
@@ -8,14 +8,33 @@ description:
|
|
|
8
8
|
---
|
|
9
9
|
|
|
10
10
|
Zuplo's AI Gateway supports the addition of custom AI providers. This allows
|
|
11
|
-
users to route their AI Gateway Apps to self
|
|
11
|
+
users to route their AI Gateway Apps to self-hosted services and models,
|
|
12
12
|
providing an additional layer of control, security and governance when using
|
|
13
13
|
these models in production.
|
|
14
14
|
|
|
15
|
-
|
|
15
|
+
## Requirements
|
|
16
16
|
|
|
17
|
-
|
|
18
|
-
|
|
17
|
+
A custom provider must expose an OpenAI-compatible API. The gateway expects:
|
|
18
|
+
|
|
19
|
+
- `POST <api-url>/v1/chat/completions` for chat completion models, accepting and
|
|
20
|
+
returning OpenAI Chat Completions schemas
|
|
21
|
+
- `POST <api-url>/v1/embeddings` for embedding models, accepting and returning
|
|
22
|
+
OpenAI Embeddings schemas
|
|
23
|
+
|
|
24
|
+
The gateway authenticates each request with your provider API key as a bearer
|
|
25
|
+
token: `Authorization: Bearer <your-api-key>`.
|
|
26
|
+
|
|
27
|
+
The gateway appends these paths to the API URL you configure, so the provider
|
|
28
|
+
must serve `chat/completions` under a `/v1` path segment. A provider that serves
|
|
29
|
+
its OpenAI-compatible API at a path without `/v1`—directly off the origin, for
|
|
30
|
+
example—won't work as a custom provider.
|
|
31
|
+
|
|
32
|
+
:::note{title="Unsupported endpoints"}
|
|
33
|
+
|
|
34
|
+
Custom providers serve chat completions and embeddings only. The gateway's
|
|
35
|
+
`/v1/responses` and `/v1/messages` endpoints don't accept custom-provider
|
|
36
|
+
models: a `/v1/responses` request fails with a `400` error and the message
|
|
37
|
+
`The /v1/responses endpoint is only supported by OpenAI provider. Current provider: <name>`.
|
|
19
38
|
|
|
20
39
|
:::
|
|
21
40
|
|
|
@@ -41,7 +60,25 @@ To add a custom AI provider to your Zuplo AI Gateway, follow these steps:
|
|
|
41
60
|
`mistral`, `xai`, `moonshot`, `zuplo`, `zuplodemo`, `zuplo-demo`). The name
|
|
42
61
|
is permanent after creation.
|
|
43
62
|
|
|
44
|
-
1.
|
|
63
|
+
1. Enter the provider's **API URL** as its origin root—for example
|
|
64
|
+
`https://api.together.xyz`. The gateway appends `/v1/chat/completions` (and
|
|
65
|
+
`/v1/embeddings` for embedding models) to this value itself, so the URL must
|
|
66
|
+
not carry a `/v1` suffix.
|
|
67
|
+
|
|
68
|
+
:::caution{title="Leave the /v1 suffix off"}
|
|
69
|
+
|
|
70
|
+
Vendors usually publish a base URL that already ends in `/v1`. Pasting that
|
|
71
|
+
value makes the gateway call `/v1/v1/chat/completions`, and the provider
|
|
72
|
+
answers `404`. Strip the suffix:
|
|
73
|
+
|
|
74
|
+
:::
|
|
75
|
+
|
|
76
|
+
| Vendor | Published base URL | API URL to enter |
|
|
77
|
+
| ---------- | --------------------------------------- | ------------------------------------ |
|
|
78
|
+
| OpenRouter | `https://openrouter.ai/api/v1` | `https://openrouter.ai/api` |
|
|
79
|
+
| Together | `https://api.together.xyz/v1` | `https://api.together.xyz` |
|
|
80
|
+
| Groq | `https://api.groq.com/openai/v1` | `https://api.groq.com/openai` |
|
|
81
|
+
| Fireworks | `https://api.fireworks.ai/inference/v1` | `https://api.fireworks.ai/inference` |
|
|
45
82
|
|
|
46
83
|
1. Enter the API Key for the selected provider (if there is no API key required,
|
|
47
84
|
you can leave this blank).
|
|
@@ -54,6 +91,15 @@ To add a custom AI provider to your Zuplo AI Gateway, follow these steps:
|
|
|
54
91
|
|
|
55
92
|
</Stepper>
|
|
56
93
|
|
|
94
|
+
## Verify the provider
|
|
95
|
+
|
|
96
|
+
Saving a provider triggers a production deployment of your gateway, and the
|
|
97
|
+
portal doesn't test the provider connection itself. Once the deployment
|
|
98
|
+
completes, send a chat completions request through your app's
|
|
99
|
+
[Universal API](./universal-api.mdx) URL with one of the provider's models and
|
|
100
|
+
confirm a `200` response. If the provider answers `404`, check the API URL for a
|
|
101
|
+
stray `/v1` suffix.
|
|
102
|
+
|
|
57
103
|
## Modify, Update or Delete your Custom AI Provider
|
|
58
104
|
|
|
59
105
|
To modify, update, or delete an existing provider, open
|
|
@@ -27,8 +27,8 @@ models are referenced like any model, as `providerName/model`.
|
|
|
27
27
|
|
|
28
28
|
<Stepper>
|
|
29
29
|
|
|
30
|
-
1. Open the [Apps](https://portal.zuplo.com/+/account/project/ai/apps)
|
|
31
|
-
your AI Gateway project and select the app to edit.
|
|
30
|
+
1. Open the [Apps & Teams](https://portal.zuplo.com/+/account/project/ai/apps)
|
|
31
|
+
tab of your AI Gateway project and select the app to edit.
|
|
32
32
|
|
|
33
33
|
1. Select the **Policies** tab. If the chain doesn't have a **Fallback Model**
|
|
34
34
|
policy yet, click **Add Policy** and add it—placed directly after Model
|
|
@@ -10,11 +10,16 @@ description:
|
|
|
10
10
|
This guide walks you through setting up your first AI Gateway project, from
|
|
11
11
|
connecting a Git repository to making your first LLM request through Zuplo.
|
|
12
12
|
|
|
13
|
+
A Zuplo account contains your members and one or more projects. The AI Gateway
|
|
14
|
+
you create in this guide is a Zuplo project; its providers, teams, and apps all
|
|
15
|
+
belong to that project. See
|
|
16
|
+
[Role Permissions](../articles/accounts/roles-and-permissions.mdx) for how Zuplo
|
|
17
|
+
account, Zuplo project, and AI Gateway team roles control access.
|
|
18
|
+
|
|
13
19
|
## Prerequisites
|
|
14
20
|
|
|
15
21
|
- A Zuplo account (sign up free at [zuplo.com](https://zuplo.com))
|
|
16
|
-
-
|
|
17
|
-
guide uses GitHub)
|
|
22
|
+
- A Git provider account that can create repositories (this guide uses GitHub)
|
|
18
23
|
- An API key for at least one LLM provider (OpenAI, Anthropic, Google, Mistral,
|
|
19
24
|
xAI, etc.)—or none at all, if you start with the free
|
|
20
25
|
[Zuplo Demo provider](./providers.mdx#zuplo-demo)
|
|
@@ -96,21 +101,24 @@ connecting a Git repository to making your first LLM request through Zuplo.
|
|
|
96
101
|
Teams organize users and carry hierarchical budget controls and policy
|
|
97
102
|
templates. Even if you're starting solo, you need at least one team.
|
|
98
103
|
|
|
99
|
-
Open the **Teams** tab and click **
|
|
104
|
+
Open the **Apps & Teams** tab and click **New Team**. Name your team (for
|
|
100
105
|
example, "Root" or your company name), choose an icon, and click **Create
|
|
101
|
-
Team**.
|
|
102
|
-
|
|
103
|
-
|
|
106
|
+
Team**. This is your first team, so there's no parent picker yet—it appears
|
|
107
|
+
on later teams, letting you nest them. To set gateway-wide spending limits
|
|
108
|
+
across this Zuplo project, open the gateway's **Settings → Usage Limits**;
|
|
109
|
+
for team-specific limits, use the team's **Usage & Limits** tab. See
|
|
110
|
+
[Usage Limits](./usage-limits.mdx).
|
|
104
111
|
|
|
105
112
|
5. **Create an app**
|
|
106
113
|
|
|
107
|
-
Apps represent individual
|
|
108
|
-
app gets its own unique URL and API key.
|
|
114
|
+
Apps represent individual software applications or services that call the AI
|
|
115
|
+
Gateway. Each app gets its own unique URL and API key.
|
|
109
116
|
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
117
|
+
In the **Apps & Teams** tab, select the team you just created and click
|
|
118
|
+
**Create App** in the team's header. Give the app a descriptive name (for
|
|
119
|
+
example, "Tennis Chat") and confirm the team that owns it. That's the whole
|
|
120
|
+
dialog—models, budgets, and other behavior are configured on the app's policy
|
|
121
|
+
chain in the next step.
|
|
114
122
|
|
|
115
123
|
:::tip
|
|
116
124
|
|
|
@@ -135,8 +143,8 @@ connecting a Git repository to making your first LLM request through Zuplo.
|
|
|
135
143
|
:::caution{title="An unrestricted app can reach every model you configured"}
|
|
136
144
|
|
|
137
145
|
Without Model Filtering, an app may call any model available through the
|
|
138
|
-
providers
|
|
139
|
-
for any app that should be limited to a specific set.
|
|
146
|
+
providers configured for this Zuplo project, including expensive ones. Add
|
|
147
|
+
Model Filtering for any app that should be limited to a specific set.
|
|
140
148
|
|
|
141
149
|
:::
|
|
142
150
|
|
|
@@ -146,8 +154,9 @@ connecting a Git repository to making your first LLM request through Zuplo.
|
|
|
146
154
|
|
|
147
155
|
7. **Get the app's URL and API key**
|
|
148
156
|
|
|
149
|
-
The app
|
|
150
|
-
|
|
157
|
+
The app's **API URL** is shown in full at the top of the app page, and its
|
|
158
|
+
key lives on the app's **API Key** tab. The API URL is the gateway's hostname
|
|
159
|
+
plus the app's ID—for example
|
|
151
160
|
`https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e`.
|
|
152
161
|
Copy both rather than constructing them.
|
|
153
162
|
|
|
@@ -156,6 +165,14 @@ connecting a Git repository to making your first LLM request through Zuplo.
|
|
|
156
165
|
`https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1`
|
|
157
166
|
as a stand-in, and requests land on paths like `/v1/chat/completions`.
|
|
158
167
|
|
|
168
|
+
:::tip
|
|
169
|
+
|
|
170
|
+
Expand the API URL to get the exact endpoint for each service the gateway
|
|
171
|
+
mounts—**Chat Completions**, **Responses**, and **Messages**—and copy the one
|
|
172
|
+
your client needs instead of assembling it by hand.
|
|
173
|
+
|
|
174
|
+
:::
|
|
175
|
+
|
|
159
176
|
:::note
|
|
160
177
|
|
|
161
178
|
The gateway checks the key only after you add the
|
|
@@ -174,14 +191,16 @@ connecting a Git repository to making your first LLM request through Zuplo.
|
|
|
174
191
|
|
|
175
192
|
8. **Send your first request from the Playground**
|
|
176
193
|
|
|
177
|
-
|
|
178
|
-
key, so it exercises the same policy chain your code
|
|
179
|
-
up a client first.
|
|
194
|
+
Click **Playground** in the app's header to open it. It chats with the app's
|
|
195
|
+
own gateway URL and API key, so it exercises the same policy chain your code
|
|
196
|
+
will hit—without wiring up a client first. The button sits on every app tab,
|
|
197
|
+
so you can try a request while editing policies without losing your place.
|
|
180
198
|
|
|
181
199
|
Pick a model from the selector, optionally set a system prompt, and send a
|
|
182
|
-
message. The
|
|
183
|
-
it on your clipboard to run in a terminal or adapt it for your
|
|
184
|
-
any reply to inspect the request and the raw response.
|
|
200
|
+
message. The playground shows the cURL for the next request, and **Copy
|
|
201
|
+
cURL** puts it on your clipboard to run in a terminal or adapt it for your
|
|
202
|
+
client. Expand any reply to inspect the request and the raw response. Closing
|
|
203
|
+
the playground keeps the conversation, so you can reopen it and carry on.
|
|
185
204
|
|
|
186
205
|
</Stepper>
|
|
187
206
|
|
|
@@ -263,16 +282,18 @@ curl https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bc
|
|
|
263
282
|
}'
|
|
264
283
|
```
|
|
265
284
|
|
|
266
|
-
### Check your
|
|
285
|
+
### Check your usage
|
|
267
286
|
|
|
268
|
-
1. Open the
|
|
269
|
-
|
|
287
|
+
1. Open the
|
|
288
|
+
[**Apps & Teams**](https://portal.zuplo.com/+/account/project/ai/apps) tab of
|
|
289
|
+
your AI Gateway project
|
|
270
290
|
2. Click on your app
|
|
271
|
-
3.
|
|
272
|
-
-
|
|
273
|
-
|
|
274
|
-
-
|
|
275
|
-
|
|
291
|
+
3. The app's **Overview** tab shows:
|
|
292
|
+
- **Daily Usage** and **Monthly Usage**—spend, tokens, and requests, with
|
|
293
|
+
progress against any configured limits
|
|
294
|
+
- A **Metrics** frame with request count, token usage, and cost over time
|
|
295
|
+
4. Click **View in Analytics** on the Metrics frame for the full request-level
|
|
296
|
+
breakdown, already filtered to this app.
|
|
276
297
|
|
|
277
298
|
You should see your test request appear with token usage and performance
|
|
278
299
|
metrics.
|
|
@@ -43,9 +43,9 @@ the app you have configured in Zuplo.
|
|
|
43
43
|
|
|
44
44
|
Models are referenced as `providerName/model`, where `providerName` is the
|
|
45
45
|
provider name configured in your gateway. By default an app can reach any model
|
|
46
|
-
offered by the providers
|
|
47
|
-
|
|
48
|
-
|
|
46
|
+
offered by the providers configured for the Zuplo project. To limit it to a
|
|
47
|
+
curated set, add the [Model Filtering](../policies/model-filtering.mdx) policy
|
|
48
|
+
to the app—it applies separate rules to completions (`generateText`,
|
|
49
49
|
`streamText`) and embeddings (`embed`), and rejects a capability it doesn't
|
|
50
50
|
configure.
|
|
51
51
|
|
|
@@ -29,7 +29,7 @@ Google, Mistral, xAI, and more) dynamically without modifying app code.
|
|
|
29
29
|
Configure model access through the gateway rather than hard coding it into your
|
|
30
30
|
apps.
|
|
31
31
|
|
|
32
|
-
**Cost Control**: Set spending limits at
|
|
32
|
+
**Cost Control**: Set spending limits at gateway, team, and app levels with
|
|
33
33
|
hierarchical budgets that cascade down through your structure. Configure daily
|
|
34
34
|
and monthly thresholds with enforcement or warning notifications.
|
|
35
35
|
|
|
@@ -50,7 +50,7 @@ cached responses for similar prompts, reducing costs and improving response
|
|
|
50
50
|
times.
|
|
51
51
|
|
|
52
52
|
**Full Observability**: Real-time dashboards show request counts, token usage,
|
|
53
|
-
time-to-first-byte metrics, and spending patterns across your
|
|
53
|
+
time-to-first-byte metrics, and spending patterns across your gateway.
|
|
54
54
|
|
|
55
55
|
## How It Works
|
|
56
56
|
|
|
@@ -73,7 +73,7 @@ them.
|
|
|
73
73
|
|
|
74
74
|
### Multi-Provider Support
|
|
75
75
|
|
|
76
|
-
Configure multiple LLM providers within a single
|
|
76
|
+
Configure multiple LLM providers within a single AI Gateway project. Supported
|
|
77
77
|
providers include OpenAI, Anthropic, Google, Mistral, xAI, and OpenAI-compatible
|
|
78
78
|
custom providers. See [AI Providers](./providers.mdx) for the full list of
|
|
79
79
|
providers and supported capabilities. Apps reference models as
|
|
@@ -100,7 +100,7 @@ starting pipeline. See [Policy Chains](./policy-chains.mdx) and
|
|
|
100
100
|
Organize users into teams with hierarchical structures. Set budget limits at
|
|
101
101
|
each level that cascade down:
|
|
102
102
|
|
|
103
|
-
- **Gateway**:
|
|
103
|
+
- **Gateway**: Limits across the Zuplo project (for example, $1,000/day)
|
|
104
104
|
- **Teams**: Team-specific limits that can't exceed parent limits (for example,
|
|
105
105
|
$500/day for the Credit Team)
|
|
106
106
|
- **Apps**: Per-app limits for granular control
|
|
@@ -116,12 +116,25 @@ Each app gets its own:
|
|
|
116
116
|
- **Policy Chain**: Model access, budgets, caching, guardrails, and custom
|
|
117
117
|
policies, applied in the order the app chooses
|
|
118
118
|
|
|
119
|
+
### Zuplo accounts and projects
|
|
120
|
+
|
|
121
|
+
A **Zuplo account** is the top-level container for your members and projects.
|
|
122
|
+
Each **Zuplo project** belongs to one account, and an AI Gateway is a type of
|
|
123
|
+
Zuplo project. The providers, teams, and apps you configure for a gateway all
|
|
124
|
+
belong to that project.
|
|
125
|
+
|
|
126
|
+
Account roles can grant access across the projects in a Zuplo account, while
|
|
127
|
+
project roles grant access to one Zuplo project. AI Gateway team roles add
|
|
128
|
+
access to a team and its apps within that project. See
|
|
129
|
+
[Role Permissions](../articles/accounts/roles-and-permissions.mdx) for the exact
|
|
130
|
+
permissions each role grants.
|
|
131
|
+
|
|
119
132
|
## Use Cases
|
|
120
133
|
|
|
121
134
|
- **Multi-tenant AI Apps**: Enforce spending limits per customer or team
|
|
122
135
|
- **Agent Development**: Build AI agents that can switch providers without code
|
|
123
136
|
changes
|
|
124
|
-
- **Cost Management**: Control and monitor LLM spending across your
|
|
137
|
+
- **Cost Management**: Control and monitor LLM spending across your gateway
|
|
125
138
|
- **Security Compliance**: Ensure PII and prompt injection protection across all
|
|
126
139
|
LLM interactions
|
|
127
140
|
- **Performance**: Reduce costs and latency with semantic caching for common
|
|
@@ -11,22 +11,36 @@ feature in a larger codebase. Each app has its own API key, which the gateway
|
|
|
11
11
|
checks once you add the
|
|
12
12
|
[authentication policy](./policy-chains.mdx#authentication); usage is tracked
|
|
13
13
|
per app whether or not that policy applies. Each app belongs to a
|
|
14
|
-
[team](./teams.mdx)
|
|
14
|
+
[team](./teams.mdx), which controls who can access and manage the app. Provider
|
|
15
|
+
access is governed separately by the user's Zuplo account and Zuplo project
|
|
16
|
+
roles. A Zuplo account contains one or more projects; this AI Gateway and its
|
|
17
|
+
providers, teams, and apps belong to one Zuplo project. See
|
|
18
|
+
[Role Permissions](../articles/accounts/roles-and-permissions.mdx).
|
|
19
|
+
|
|
20
|
+
:::note{title="App permissions"}
|
|
21
|
+
|
|
22
|
+
A team **Admin** can create, edit, and delete apps owned by that team. A team
|
|
23
|
+
**Member** can access the team's apps but can't manage them. Both roles require
|
|
24
|
+
permission to view the Zuplo project that contains the AI Gateway.
|
|
25
|
+
|
|
26
|
+
:::
|
|
15
27
|
|
|
16
28
|
## Creating an App
|
|
17
29
|
|
|
18
30
|
<Stepper>
|
|
19
31
|
|
|
20
|
-
1. Open the [Apps](https://portal.zuplo.com/+/account/project/ai/apps)
|
|
21
|
-
your AI Gateway project in the Zuplo Portal.
|
|
32
|
+
1. Open the [Apps & Teams](https://portal.zuplo.com/+/account/project/ai/apps)
|
|
33
|
+
tab of your AI Gateway project in the Zuplo Portal.
|
|
22
34
|
|
|
23
|
-
1.
|
|
35
|
+
1. Select the team that will own the app, then click **Create App** in the
|
|
36
|
+
team's header. An app always belongs to a team, so the tree's own create
|
|
37
|
+
action is **New Team**.
|
|
24
38
|
|
|
25
|
-
1. Enter the name
|
|
39
|
+
1. Enter the **App name**.
|
|
26
40
|
|
|
27
|
-
1.
|
|
41
|
+
1. Confirm the **Team**—it's prefilled with the team you started from.
|
|
28
42
|
|
|
29
|
-
1. Click **Create**
|
|
43
|
+
1. Click **Create app**
|
|
30
44
|
|
|
31
45
|
</Stepper>
|
|
32
46
|
|
|
@@ -38,7 +52,7 @@ unrestricted until you add policies.
|
|
|
38
52
|
## Restricting models
|
|
39
53
|
|
|
40
54
|
By default an app isn't restricted to particular models: any model available
|
|
41
|
-
through the providers
|
|
55
|
+
through the providers configured for the Zuplo project is accepted. An
|
|
42
56
|
unrestricted app has no default model, so every request must name one as
|
|
43
57
|
`providerName/model`. A request with no model, or a model without that prefix,
|
|
44
58
|
gets a `400`.
|
|
@@ -57,13 +71,13 @@ and configure each capability (completions, embeddings) in one of two modes:
|
|
|
57
71
|
The two modes are mutually exclusive per capability: a capability takes an allow
|
|
58
72
|
list or a block list, never both.
|
|
59
73
|
|
|
60
|
-
| Request
|
|
61
|
-
|
|
|
62
|
-
| A model the rules don't allow
|
|
63
|
-
| No `model` named, with no allow list to supply a default
|
|
64
|
-
| A model with no `providerName/` prefix
|
|
65
|
-
| A capability the policy doesn't configure at all
|
|
66
|
-
| A provider name that isn't in the project's [provider catalog](./managing-providers.mdx) | `400` |
|
|
74
|
+
| Request | Response |
|
|
75
|
+
| ---------------------------------------------------------------------------------------------- | ----------------------------- |
|
|
76
|
+
| A model the rules don't allow | `403`, listing what's allowed |
|
|
77
|
+
| No `model` named, with no allow list to supply a default | `400` |
|
|
78
|
+
| A model with no `providerName/` prefix | `400` |
|
|
79
|
+
| A capability the policy doesn't configure at all | `403` |
|
|
80
|
+
| A provider name that isn't in the Zuplo project's [provider catalog](./managing-providers.mdx) | `400` |
|
|
67
81
|
|
|
68
82
|
When model selection needs logic instead of a list, a custom policy can take
|
|
69
83
|
over—see [Dynamic model routing](./cookbooks/dynamic-model-routing.mdx).
|
|
@@ -78,16 +92,17 @@ gateway's limits still apply. See [Usage Limits](./usage-limits.mdx).
|
|
|
78
92
|
## Editing an App
|
|
79
93
|
|
|
80
94
|
To edit an app, open the
|
|
81
|
-
[Apps](https://portal.zuplo.com/+/account/project/ai/apps) tab of your
|
|
82
|
-
Gateway project in the Zuplo Portal. Select the app you want to edit. The
|
|
83
|
-
behavior—models, budgets, caching, guardrails—is edited on the
|
|
84
|
-
the app's name lives on the **Settings** tab. Make your
|
|
85
|
-
**Save** button. Policy chain changes apply within about a
|
|
95
|
+
[Apps & Teams](https://portal.zuplo.com/+/account/project/ai/apps) tab of your
|
|
96
|
+
AI Gateway project in the Zuplo Portal. Select the app you want to edit. The
|
|
97
|
+
app's behavior—models, budgets, caching, guardrails—is edited on the
|
|
98
|
+
**Policies** tab; the app's name lives on the **Settings** tab. Make your
|
|
99
|
+
changes and click the **Save** button. Policy chain changes apply within about a
|
|
100
|
+
minute.
|
|
86
101
|
|
|
87
102
|
## Deleting an App
|
|
88
103
|
|
|
89
104
|
To delete an app, open the
|
|
90
|
-
[Apps](https://portal.zuplo.com/+/account/project/ai/apps) tab of your
|
|
91
|
-
Gateway project in the Zuplo Portal. Select the app you want to delete, open
|
|
92
|
-
**Settings** tab, and click the **Delete App** button. You will be prompted
|
|
93
|
-
confirm the deletion.
|
|
105
|
+
[Apps & Teams](https://portal.zuplo.com/+/account/project/ai/apps) tab of your
|
|
106
|
+
AI Gateway project in the Zuplo Portal. Select the app you want to delete, open
|
|
107
|
+
its **Settings** tab, and click the **Delete App** button. You will be prompted
|
|
108
|
+
to confirm the deletion.
|
|
@@ -10,6 +10,19 @@ Zuplo's AI Gateway supports integration with various
|
|
|
10
10
|
[AI providers](./providers.mdx), allowing you to leverage different models and
|
|
11
11
|
services for your apps.
|
|
12
12
|
|
|
13
|
+
:::note{title="Provider permissions"}
|
|
14
|
+
|
|
15
|
+
AI providers are configured for a Zuplo project. A Zuplo account contains one or
|
|
16
|
+
more projects: an account role can grant access across those projects, while a
|
|
17
|
+
project role grants access only to that project. Adding, editing, or deleting
|
|
18
|
+
providers and their models requires the **Edit** permission granted to Zuplo
|
|
19
|
+
account and Zuplo project **Admins**. A Zuplo account **Developer**, or a Zuplo
|
|
20
|
+
project **Developer** or **Member**, can view providers but can't manage them.
|
|
21
|
+
Team membership doesn't grant provider access. See
|
|
22
|
+
[Role Permissions](../articles/accounts/roles-and-permissions.mdx).
|
|
23
|
+
|
|
24
|
+
:::
|
|
25
|
+
|
|
13
26
|
## Adding a New AI Provider
|
|
14
27
|
|
|
15
28
|
To add a new AI provider to your Zuplo AI Gateway, follow these steps:
|
|
@@ -24,7 +37,9 @@ To add a new AI provider to your Zuplo AI Gateway, follow these steps:
|
|
|
24
37
|
|
|
25
38
|
1. Select the desired provider from the list of
|
|
26
39
|
[supported providers](./providers.mdx), or
|
|
27
|
-
[add a custom provider](./custom-providers.mdx).
|
|
40
|
+
[add a custom provider](./custom-providers.mdx). A custom provider's API URL
|
|
41
|
+
is its origin root—without the `/v1` suffix vendors usually publish—because
|
|
42
|
+
the gateway appends the operation path itself.
|
|
28
43
|
|
|
29
44
|
1. Review the **Provider Name**, which fills in automatically based on the
|
|
30
45
|
provider you selected (a second OpenAI configuration becomes `openai-2`). You
|
|
@@ -80,7 +95,7 @@ available fail with an error naming the model.
|
|
|
80
95
|
|
|
81
96
|
## Deleting an AI Provider
|
|
82
97
|
|
|
83
|
-
Providers that are no longer used within your project can be deleted. To
|
|
84
|
-
a provider, open
|
|
98
|
+
Providers that are no longer used within your Zuplo project can be deleted. To
|
|
99
|
+
delete a provider, open
|
|
85
100
|
[**Settings → AI Providers**](https://portal.zuplo.com/+/account/project/ai/settings/data-models)
|
|
86
101
|
and click the **Delete** button next to the provider you want to remove.
|
|
@@ -5,30 +5,38 @@ description:
|
|
|
5
5
|
and usage limits in your AI Gateway project.
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
-
Teams in the Zuplo AI Gateway
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
8
|
+
Teams in the Zuplo AI Gateway grant users access to apps. Teams are
|
|
9
|
+
hierarchical, so access propagates to sub-teams only when the user also has at
|
|
10
|
+
permission to view the Zuplo project that contains the AI Gateway. Team or
|
|
11
|
+
sub-team membership doesn't grant Zuplo project access. Teams are also where you
|
|
12
|
+
define usage limits and [policy templates](./policy-templates.mdx).
|
|
12
13
|
|
|
13
14
|
## Creating a Team
|
|
14
15
|
|
|
15
|
-
Creating a team requires
|
|
16
|
-
on the parent team for a sub-team.
|
|
17
|
-
Team** button.
|
|
16
|
+
Creating a team requires Admin rights on the Zuplo project or Zuplo account for
|
|
17
|
+
a top-level team, or on the parent team for a sub-team. As a convenience, the
|
|
18
|
+
Portal disables the **New Team** button when you don't have the required rights.
|
|
18
19
|
|
|
19
20
|
<Stepper>
|
|
20
21
|
|
|
21
|
-
1. Open the [Teams](https://portal.zuplo.com/+/account/project/ai/teams)
|
|
22
|
-
your AI Gateway project in the Zuplo Portal.
|
|
22
|
+
1. Open the [Apps & Teams](https://portal.zuplo.com/+/account/project/ai/teams)
|
|
23
|
+
tab of your AI Gateway project in the Zuplo Portal.
|
|
23
24
|
|
|
24
|
-
1. Click
|
|
25
|
+
1. Click **New Team** above the tree.
|
|
26
|
+
|
|
27
|
+
1. Choose where the team sits. Once the gateway has at least one team, the
|
|
28
|
+
dialog shows a parent picker above the name field; leave it on **No parent
|
|
29
|
+
team** for a top-level team, or pick a team to create a sub-team under it.
|
|
25
30
|
|
|
26
31
|
1. Enter the name of your team.
|
|
27
32
|
|
|
28
|
-
1. Click **Create**
|
|
33
|
+
1. Click **Create Team**
|
|
29
34
|
|
|
30
35
|
</Stepper>
|
|
31
36
|
|
|
37
|
+
You can also create a sub-team from the parent itself: select it in the tree and
|
|
38
|
+
click **Create Sub-Team** in the team's header.
|
|
39
|
+
|
|
32
40
|
After creating a team, use its tabs to configure it:
|
|
33
41
|
|
|
34
42
|
- **Policies**—the team's [policy template](./policy-templates.mdx), the
|
|
@@ -39,13 +47,14 @@ After creating a team, use its tabs to configure it:
|
|
|
39
47
|
|
|
40
48
|
## Adding Team Members
|
|
41
49
|
|
|
42
|
-
Team members are the users who belong to a team and can access
|
|
43
|
-
|
|
50
|
+
Team members are the users who belong to a team and can access its apps,
|
|
51
|
+
provided they also have permission to view the Zuplo project. AI provider access
|
|
52
|
+
is separate and depends on the user's Zuplo account or Zuplo project role.
|
|
44
53
|
|
|
45
54
|
<Stepper>
|
|
46
55
|
|
|
47
|
-
1. Open the [Teams](https://portal.zuplo.com/+/account/project/ai/teams)
|
|
48
|
-
your AI Gateway project in the Zuplo Portal.
|
|
56
|
+
1. Open the [Apps & Teams](https://portal.zuplo.com/+/account/project/ai/teams)
|
|
57
|
+
tab of your AI Gateway project in the Zuplo Portal.
|
|
49
58
|
|
|
50
59
|
1. Select the team you want to add members to.
|
|
51
60
|
|
|
@@ -54,20 +63,18 @@ and apps associated with that team.
|
|
|
54
63
|
1. Click **Add Member**.
|
|
55
64
|
|
|
56
65
|
1. Type the email address of the user you want to add. If the user is already a
|
|
57
|
-
member of your Zuplo account they
|
|
58
|
-
|
|
59
|
-
added to the team.
|
|
66
|
+
member of your Zuplo account, they are added directly to the team. Otherwise,
|
|
67
|
+
they are invited to join the Zuplo account and then added to the team.
|
|
60
68
|
|
|
61
69
|
1. Select the role for the user:
|
|
62
|
-
- **Member**: Can access
|
|
63
|
-
- **Admin**: Can manage team settings
|
|
64
|
-
providers and apps assigned to the team.
|
|
70
|
+
- **Member**: Can access apps owned by the team.
|
|
71
|
+
- **Admin**: Can manage the team's settings, members, and apps.
|
|
65
72
|
|
|
66
73
|
:::caution{title="The Member role requires RBAC"}
|
|
67
74
|
|
|
68
75
|
Role-based access control (RBAC) is an optional enterprise add-on on your
|
|
69
|
-
Zuplo account. Without it,
|
|
70
|
-
|
|
76
|
+
Zuplo account. Without it, every invited user is an **Admin**. For more
|
|
77
|
+
information see the
|
|
71
78
|
[document on roles and permissions](../articles/accounts/roles-and-permissions.mdx).
|
|
72
79
|
|
|
73
80
|
:::
|
|
@@ -54,16 +54,17 @@ To protect every new app in a team, add the policy to the team's
|
|
|
54
54
|
|
|
55
55
|
## Options
|
|
56
56
|
|
|
57
|
-
| Option | Default | What it does
|
|
58
|
-
| --------------------------------------- | ---------- |
|
|
59
|
-
| `configurationId` | — | Required. Your Akamai configuration
|
|
60
|
-
| `api-key` | — | Required. Note the hyphen—`apiKey` is rejected
|
|
61
|
-
| `applicationId` | — | Optional Akamai application identifier
|
|
62
|
-
| `endpoints` | All shapes | Which API shapes to inspect
|
|
63
|
-
| `onUnknownShape` | `deny` | What to do with a request shape the policy can't read
|
|
64
|
-
| `
|
|
65
|
-
| `streamingAccumulation.
|
|
66
|
-
| `streamingAccumulation.
|
|
57
|
+
| Option | Default | What it does |
|
|
58
|
+
| --------------------------------------- | ---------- | ----------------------------------------------------------------------------------------------------------------------- |
|
|
59
|
+
| `configurationId` | — | Required. Your Akamai configuration |
|
|
60
|
+
| `api-key` | — | Required. Note the hyphen—`apiKey` is rejected |
|
|
61
|
+
| `applicationId` | — | Optional Akamai application identifier |
|
|
62
|
+
| `endpoints` | All shapes | Which API shapes to inspect |
|
|
63
|
+
| `onUnknownShape` | `deny` | What to do with a request shape the policy can't read |
|
|
64
|
+
| `onError` | `block` | What to do when the Akamai detect call itself fails—`block` answers `502`; `allow` lets the request through uninspected |
|
|
65
|
+
| `streamingAccumulation.enabled` | `true` | Scan streaming completions. `false` stops scanning responses for streaming requests |
|
|
66
|
+
| `streamingAccumulation.eventsInterval` | `5` | How many streamed events to buffer between checks |
|
|
67
|
+
| `streamingAccumulation.checkIntervalMs` | none | Also check on a timer. Off unless set |
|
|
67
68
|
|
|
68
69
|
Unknown option keys are rejected.
|
|
69
70
|
|
|
@@ -91,18 +92,18 @@ Blocked requests are counted against the app and appear in its Dashboard—see
|
|
|
91
92
|
|
|
92
93
|
## Failure modes
|
|
93
94
|
|
|
94
|
-
| Situation | Result
|
|
95
|
-
| -------------------------------------------------------------- |
|
|
96
|
-
| Options are invalid or incomplete | Fails closed—request rejected
|
|
97
|
-
| The request body can't be read | Fails closed—request rejected
|
|
98
|
-
| An endpoint shape the policy can't inspect, such as embeddings | `400`, code `guardrail_uninspectable`
|
|
99
|
-
| Akamai is unreachable, or returns an error |
|
|
95
|
+
| Situation | Result |
|
|
96
|
+
| -------------------------------------------------------------- | ------------------------------------------------ |
|
|
97
|
+
| Options are invalid or incomplete | Fails closed—request rejected |
|
|
98
|
+
| The request body can't be read | Fails closed—request rejected |
|
|
99
|
+
| An endpoint shape the policy can't inspect, such as embeddings | `400`, code `guardrail_uninspectable` |
|
|
100
|
+
| Akamai is unreachable, or returns an error | Fails closed—`502`, code `guardrail_unavailable` |
|
|
100
101
|
|
|
101
102
|
:::caution
|
|
102
103
|
|
|
103
|
-
|
|
104
|
-
firewall while the gateway keeps returning `200`s. Monitor
|
|
105
|
-
sudden drop to zero doesn't go unnoticed.
|
|
104
|
+
With `onError: "allow"` the policy fails open instead: an expired Akamai key
|
|
105
|
+
silently disables the firewall while the gateway keeps returning `200`s. Monitor
|
|
106
|
+
blocked counts so a sudden drop to zero doesn't go unnoticed.
|
|
106
107
|
|
|
107
108
|
:::
|
|
108
109
|
|
|
@@ -41,8 +41,8 @@ app that should send traces:
|
|
|
41
41
|
|
|
42
42
|
<Stepper>
|
|
43
43
|
|
|
44
|
-
1. In your Comet account, create (or pick) a project and workspace and
|
|
45
|
-
an API key.
|
|
44
|
+
1. In your Comet account, create (or pick) a Comet project and workspace and
|
|
45
|
+
generate an API key.
|
|
46
46
|
|
|
47
47
|
2. In your gateway's repository, set the policy's settings and push:
|
|
48
48
|
|
|
@@ -64,7 +64,7 @@ For each request, the policy automatically captures:
|
|
|
64
64
|
1. ### Obtain Galileo Credentials
|
|
65
65
|
1. Sign up for a
|
|
66
66
|
[Galileo](https://galileo.ai?utm_source=zuplo&utm_medium=web) account
|
|
67
|
-
2. Create a new project in your Galileo dashboard
|
|
67
|
+
2. Create a new Galileo project in your Galileo dashboard
|
|
68
68
|
3. Generate an API key specifically for use with Zuplo
|
|
69
69
|
|
|
70
70
|
:::note
|
|
@@ -154,7 +154,7 @@ Common issues and solutions:
|
|
|
154
154
|
- **Authentication Errors**: Check that your Galileo API key is valid and has
|
|
155
155
|
proper permissions
|
|
156
156
|
- **Configuration Issues**: Ensure the policy declaration's environment
|
|
157
|
-
variables are set in your project settings
|
|
157
|
+
variables are set in your Zuplo project settings
|
|
158
158
|
|
|
159
159
|
## Additional Resources
|
|
160
160
|
|
|
@@ -10,7 +10,7 @@ description:
|
|
|
10
10
|
|
|
11
11
|
Model Filtering decides which models an app may use and creates the model
|
|
12
12
|
selection the rest of the chain works from. Without it, an app can reach any
|
|
13
|
-
model offered by the providers
|
|
13
|
+
model offered by the providers configured for the Zuplo project.
|
|
14
14
|
|
|
15
15
|
Configure each capability—completions, embeddings—in one of two modes. The modes
|
|
16
16
|
are mutually exclusive per capability.
|
|
@@ -136,9 +136,10 @@ programmatically before the chain runs: place your own route policy between the
|
|
|
136
136
|
loader and the executor.
|
|
137
137
|
|
|
138
138
|
Both take a single option, `cacheTtlSeconds`, which sets how long the gateway
|
|
139
|
-
caches a loaded app configuration. The default is `10` seconds
|
|
140
|
-
minimum
|
|
141
|
-
key.
|
|
139
|
+
caches a loaded app configuration. The default is `10` seconds and `2` is the
|
|
140
|
+
minimum—a lower value throws a configuration error, as does any other option
|
|
141
|
+
key. New gateways scaffold the loader with `5`, so configuration changes take
|
|
142
|
+
effect within about five seconds.
|
|
142
143
|
|
|
143
144
|
## Authentication
|
|
144
145
|
|
|
@@ -55,11 +55,13 @@ by the team policy template".
|
|
|
55
55
|
|
|
56
56
|
A sub-team can use its parent team's template instead of defining its own,
|
|
57
57
|
controlled by the **Inherit from parent** switch on the sub-team's Policies tab.
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
58
|
+
The switch drafts like the page's other edits and applies when you click **Save
|
|
59
|
+
changes**. While inheriting, the sub-team's template is read-only—it shows the
|
|
60
|
+
parent's effective template. Turn the switch off to give the sub-team its own
|
|
61
|
+
template: the editor starts from a copy of the parent's effective template,
|
|
62
|
+
which you then adjust. Turning inheritance back on and saving discards the
|
|
63
|
+
sub-team's own template in favor of the parent's—before you save, flip the
|
|
64
|
+
switch again to restore your draft.
|
|
63
65
|
|
|
64
66
|
## Next steps
|
|
65
67
|
|
|
@@ -19,6 +19,8 @@ Zuplo currently supports the following AI providers:
|
|
|
19
19
|
- Google
|
|
20
20
|
- Mistral
|
|
21
21
|
- xAI (Grok)
|
|
22
|
+
- [Bedrock Mantle](#bedrock-mantle)—Amazon Bedrock's compatible-APIs endpoint,
|
|
23
|
+
serving both OpenAI-compatible and Claude models
|
|
22
24
|
- [Zuplo Demo](#zuplo-demo)—a free, keyless provider for trying the gateway
|
|
23
25
|
- OpenAI-compatible [Custom Providers](./custom-providers.mdx) (such as Qwen,
|
|
24
26
|
Kimi, etc)
|
|
@@ -32,6 +34,7 @@ The following capabilities are supported across providers:
|
|
|
32
34
|
| Google | ✅ | ✅ | ❌ | ❌ |
|
|
33
35
|
| Mistral | ✅ | ✅ | ❌ | ❌ |
|
|
34
36
|
| xAI | ✅ | ✅ | ❌ | ❌ |
|
|
37
|
+
| Bedrock Mantle | ✅ | ✅ | ❌ | ✅ |
|
|
35
38
|
| Zuplo Demo | ✅ | ❌ | ❌ | ❌ |
|
|
36
39
|
| OpenAI-compatible (Custom) | ✅ | ✅ | ❌ | ❌ |
|
|
37
40
|
|
|
@@ -40,18 +43,48 @@ the native Anthropic Messages API (`/v1/messages`). See the
|
|
|
40
43
|
[Universal API](./universal-api.mdx#supported-endpoints) for the endpoint each
|
|
41
44
|
capability maps to.
|
|
42
45
|
|
|
46
|
+
A custom provider must serve chat completions and embeddings under a `/v1` path
|
|
47
|
+
segment on its API URL, and you enter that URL as an origin root—without the
|
|
48
|
+
`/v1` suffix vendors usually publish. See
|
|
49
|
+
[Custom Providers](./custom-providers.mdx#requirements) for the exact contract.
|
|
50
|
+
|
|
43
51
|
Apps reference a provider's models as `providerName/model`—for example
|
|
44
52
|
`openai/gpt-5-mini` or `anthropic/claude-sonnet-4-6`—where `providerName` is the
|
|
45
53
|
name you give the provider configuration. See the
|
|
46
54
|
[Universal API](./universal-api.mdx).
|
|
47
55
|
|
|
56
|
+
## Bedrock Mantle
|
|
57
|
+
|
|
58
|
+
**Bedrock Mantle** is Amazon Bedrock's compatible-APIs endpoint. It serves two
|
|
59
|
+
model families from one host and one credential, so which capabilities apply
|
|
60
|
+
depends on the model you call:
|
|
61
|
+
|
|
62
|
+
- **OpenAI-compatible models** (for example `openai.gpt-oss-120b`) serve chat
|
|
63
|
+
completions and embeddings.
|
|
64
|
+
- **Claude models** serve the native Anthropic Messages API, and chat
|
|
65
|
+
completions through the gateway's translation.
|
|
66
|
+
|
|
67
|
+
Mantle endpoints are regional, so the provider dialog asks for an **AWS Region**
|
|
68
|
+
instead of an endpoint URL and builds `https://bedrock-mantle.<region>.api.aws`
|
|
69
|
+
from it. Enter a standard region code such as `us-east-1`.
|
|
70
|
+
|
|
71
|
+
:::caution{title="Long-term API keys only"}
|
|
72
|
+
|
|
73
|
+
Bedrock issues both long-term and short-term API keys, and the gateway accepts
|
|
74
|
+
only long-term ones, which start with `ABSK`. Generate one in the AWS console
|
|
75
|
+
under **Bedrock → API keys**. A short-term key (prefixed `bedrock-api-key-`)
|
|
76
|
+
expires within 12 hours and is locked to the session that minted it, so the
|
|
77
|
+
dialog rejects it rather than letting the connection break the same day.
|
|
78
|
+
|
|
79
|
+
:::
|
|
80
|
+
|
|
48
81
|
## Zuplo Demo
|
|
49
82
|
|
|
50
83
|
**Zuplo Demo** is a free provider that Zuplo operates so you can try the AI
|
|
51
|
-
Gateway without
|
|
52
|
-
[Managing Providers](./managing-providers.mdx)—but skip the API
|
|
53
|
-
doesn't ask for one, because your gateway authenticates to the
|
|
54
|
-
itself.
|
|
84
|
+
Gateway without an account with an LLM provider. Add it like any other
|
|
85
|
+
provider—see [Managing Providers](./managing-providers.mdx)—but skip the API
|
|
86
|
+
key: the dialog doesn't ask for one, because your gateway authenticates to the
|
|
87
|
+
demo service itself.
|
|
55
88
|
|
|
56
89
|
The provider serves three demo persona models as chat completions, at no cost:
|
|
57
90
|
|
|
@@ -61,9 +94,9 @@ The provider serves three demo persona models as chat completions, at no cost:
|
|
|
61
94
|
|
|
62
95
|
:::caution{title="Not for production use"}
|
|
63
96
|
|
|
64
|
-
Zuplo Demo has daily
|
|
65
|
-
demos, but not for real traffic. Swap in a keyed provider such as
|
|
66
|
-
Anthropic before going live.
|
|
97
|
+
Zuplo Demo has daily limits for each Zuplo account—plenty for exploring the
|
|
98
|
+
gateway and demos, but not for real traffic. Swap in a keyed provider such as
|
|
99
|
+
OpenAI or Anthropic before going live.
|
|
67
100
|
|
|
68
101
|
:::
|
|
69
102
|
|
|
@@ -7,10 +7,11 @@ description:
|
|
|
7
7
|
changes require a deploy versus applying live.
|
|
8
8
|
---
|
|
9
9
|
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
TypeScript files you
|
|
10
|
+
An AI Gateway is a Zuplo project configured for AI traffic. Every AI Gateway
|
|
11
|
+
project is backed by a Git repository that you own. The repository contains the
|
|
12
|
+
gateway's routes and policy declarations, so gateway changes go through your
|
|
13
|
+
normal review workflow, and custom policies are plain TypeScript files you
|
|
14
|
+
commit alongside them.
|
|
14
15
|
|
|
15
16
|
## Connecting a repository
|
|
16
17
|
|
|
@@ -8,7 +8,7 @@ description:
|
|
|
8
8
|
|
|
9
9
|
Teams are used to manage access to AI Gateway apps, set usage limits, and
|
|
10
10
|
monitor activity. Each team can have multiple members and sub-teams. Use teams
|
|
11
|
-
to group users by department,
|
|
11
|
+
to group users by department, initiative, or any other logical grouping.
|
|
12
12
|
|
|
13
13
|
## Teams & Apps
|
|
14
14
|
|
|
@@ -22,14 +22,19 @@ that apply to all of the team's apps combined.
|
|
|
22
22
|
|
|
23
23
|
## Members
|
|
24
24
|
|
|
25
|
-
Members are the users who belong to a team.
|
|
26
|
-
|
|
25
|
+
Members are the users who belong to a team. Their permissions combine their role
|
|
26
|
+
in the Zuplo account, the Zuplo project that contains the AI Gateway, and the
|
|
27
|
+
team.
|
|
27
28
|
|
|
28
29
|
There are two roles at the team level:
|
|
29
30
|
|
|
30
|
-
- **Member**: Can access
|
|
31
|
-
- **Admin**: Can manage team settings
|
|
32
|
-
|
|
31
|
+
- **Member**: Can access apps owned by the team.
|
|
32
|
+
- **Admin**: Can manage the team's settings, members, and apps.
|
|
33
|
+
|
|
34
|
+
Team roles don't grant access to AI providers. Providers are configured for the
|
|
35
|
+
Zuplo project, and access to view or manage them depends on the user's Zuplo
|
|
36
|
+
account or Zuplo project role. A user also needs permission to view that project
|
|
37
|
+
before their team role grants access to the team's apps.
|
|
33
38
|
|
|
34
39
|
For more information on roles and permissions, see the
|
|
35
40
|
[document on roles and permissions](../articles/accounts/roles-and-permissions.mdx).
|
|
@@ -37,8 +42,8 @@ For more information on roles and permissions, see the
|
|
|
37
42
|
**Additional Resources**
|
|
38
43
|
|
|
39
44
|
- [Role Permissions](../articles/accounts/roles-and-permissions.mdx) - Details
|
|
40
|
-
on
|
|
45
|
+
on Zuplo account, Zuplo project, and AI Gateway team roles.
|
|
41
46
|
- [Creating & Editing Teams](./managing-teams.mdx) - How to create and edit
|
|
42
47
|
teams.
|
|
43
|
-
- [Creating & Editing Apps](./managing-apps.mdx) - How to
|
|
44
|
-
|
|
48
|
+
- [Creating & Editing Apps](./managing-apps.mdx) - How to create, configure, and
|
|
49
|
+
delete apps.
|
|
@@ -7,20 +7,20 @@ description:
|
|
|
7
7
|
---
|
|
8
8
|
|
|
9
9
|
The Zuplo AI Gateway provides hierarchical usage limits and budget controls to
|
|
10
|
-
manage LLM spending across
|
|
11
|
-
team, and app levels.
|
|
10
|
+
manage LLM spending across the gateway's Zuplo project. Limits can be set at the
|
|
11
|
+
gateway, team, and app levels.
|
|
12
12
|
|
|
13
13
|
## Budget Hierarchy
|
|
14
14
|
|
|
15
15
|
Limits at every level apply together—a request is blocked when _any_ level's
|
|
16
16
|
limit is exceeded, and the error names the level that blocked it:
|
|
17
17
|
|
|
18
|
-
- **Gateway** -
|
|
19
|
-
teams and apps combined
|
|
18
|
+
- **Gateway** - Limits across the Zuplo project (for example, $1,000/day),
|
|
19
|
+
covering all teams and apps combined
|
|
20
20
|
- **Teams** - Team-specific limits covering all of the team's apps combined (for
|
|
21
21
|
example, $500/day for the Engineering team)
|
|
22
22
|
- **Apps** - Per-app limits for granular control (for example, $10/day for a
|
|
23
|
-
hackathon
|
|
23
|
+
hackathon app)
|
|
24
24
|
|
|
25
25
|
## Where limits are configured
|
|
26
26
|
|
|
@@ -54,8 +54,8 @@ doesn't reset the period's usage.
|
|
|
54
54
|
|
|
55
55
|
<Stepper>
|
|
56
56
|
|
|
57
|
-
1. Open the [Apps](https://portal.zuplo.com/+/account/project/ai/apps)
|
|
58
|
-
select the app.
|
|
57
|
+
1. Open the [Apps & Teams](https://portal.zuplo.com/+/account/project/ai/apps)
|
|
58
|
+
tab and select the app.
|
|
59
59
|
|
|
60
60
|
1. On the **Policies** tab, configure the **Budgets and Costs** policy (add it
|
|
61
61
|
from **Add Policy** if the chain doesn't have it).
|
|
@@ -106,13 +106,13 @@ outage rejects the request rather than letting it escape the budget.
|
|
|
106
106
|
|
|
107
107
|
Each app, team, and the gateway show current usage against their limits:
|
|
108
108
|
|
|
109
|
-
1. Open the [Apps](https://portal.zuplo.com/+/account/project/ai/apps)
|
|
110
|
-
|
|
111
|
-
an app or team
|
|
109
|
+
1. Open the [Apps & Teams](https://portal.zuplo.com/+/account/project/ai/apps)
|
|
110
|
+
tab and select an app or team
|
|
112
111
|
2. The **Overview** tab shows daily and monthly usage—spend, tokens, and
|
|
113
112
|
requests—with progress against any configured limits
|
|
114
|
-
3.
|
|
115
|
-
|
|
113
|
+
3. The same tab's **Metrics** frame charts request count, token usage, and cost
|
|
114
|
+
over time; **View in Analytics** opens the full request-level breakdown
|
|
115
|
+
scoped to that app or team
|
|
116
116
|
|
|
117
117
|
## Related Resources
|
|
118
118
|
|
|
@@ -1,11 +1,25 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Role Permissions
|
|
3
|
+
sidebar_label: Role Permissions
|
|
4
|
+
description:
|
|
5
|
+
Compare account, project, and AI Gateway permissions for Admin, Developer,
|
|
6
|
+
Member, and team roles.
|
|
3
7
|
---
|
|
4
8
|
|
|
5
9
|
<EnterpriseFeature name="Role Based Access Control" />
|
|
6
10
|
|
|
7
|
-
|
|
8
|
-
|
|
11
|
+
:::note
|
|
12
|
+
|
|
13
|
+
Without the role-based access control (RBAC) enterprise add-on, every user
|
|
14
|
+
invited to an account is an **Admin**. The **Developer** and **Member**
|
|
15
|
+
distinctions below apply only when RBAC is enabled.
|
|
16
|
+
|
|
17
|
+
:::
|
|
18
|
+
|
|
19
|
+
A Zuplo account is the top-level container for members and projects. Account
|
|
20
|
+
roles grant permissions across that container and can also grant access to its
|
|
21
|
+
projects. A project belongs to one Zuplo account, and project roles grant access
|
|
22
|
+
only to that project.
|
|
9
23
|
|
|
10
24
|
The following roles are available at the account level:
|
|
11
25
|
|
|
@@ -35,7 +49,8 @@ The following roles are available at the project level:
|
|
|
35
49
|
|
|
36
50
|
## Account Role Permissions
|
|
37
51
|
|
|
38
|
-
The following table outlines the permissions
|
|
52
|
+
The following table outlines the permissions granted directly by each Zuplo
|
|
53
|
+
account role.
|
|
39
54
|
|
|
40
55
|
| Resource | Action | Admin | Developer | Member |
|
|
41
56
|
| -------------- | --------------- | ----- | --------- | ------ |
|
|
@@ -43,6 +58,12 @@ The following table outlines the permissions available to each account role.
|
|
|
43
58
|
| | View | ✅ | ✅ | ✅ |
|
|
44
59
|
| Projects | Edit | ✅ | ❌ | ❌ |
|
|
45
60
|
| | View | ✅ | ✅ | ❌ |
|
|
61
|
+
| AI Providers | Edit | ✅ | ❌ | ❌ |
|
|
62
|
+
| | View | ✅ | ✅ | ❌ |
|
|
63
|
+
| AI Teams | Manage | ✅ | ❌ | ❌ |
|
|
64
|
+
| | View | ✅ | ❌ | ❌ |
|
|
65
|
+
| AI Apps | Manage | ✅ | ❌ | ❌ |
|
|
66
|
+
| | View | ✅ | ❌ | ❌ |
|
|
46
67
|
| Custom Domains | Edit | ✅ | ❌ | ❌ |
|
|
47
68
|
| | View | ✅ | ✅ | ❌ |
|
|
48
69
|
| Tunnels | Edit | ✅ | ❌ | ❌ |
|
|
@@ -58,12 +79,19 @@ The following table outlines the permissions available to each account role.
|
|
|
58
79
|
|
|
59
80
|
## Project Role Permissions
|
|
60
81
|
|
|
61
|
-
The following table outlines the permissions
|
|
82
|
+
The following table outlines the permissions granted directly by each Zuplo
|
|
83
|
+
project role.
|
|
62
84
|
|
|
63
85
|
| Resource | Environment | Action | Admin | Developer | Member |
|
|
64
86
|
| --------------------- | ----------- | ------ | ----- | --------- | ------ |
|
|
65
87
|
| Project | | Edit | ✅ | ❌ | ❌ |
|
|
66
88
|
| | | View | ✅ | ✅ | ✅ |
|
|
89
|
+
| AI Providers | | Edit | ✅ | ❌ | ❌ |
|
|
90
|
+
| | | View | ✅ | ✅ | ✅ |
|
|
91
|
+
| AI Teams | | Manage | ✅ | ❌ | ❌ |
|
|
92
|
+
| | | View | ✅ | ❌ | ❌ |
|
|
93
|
+
| AI Apps | | Manage | ✅ | ❌ | ❌ |
|
|
94
|
+
| | | View | ✅ | ❌ | ❌ |
|
|
67
95
|
| Environment | Production | Edit | ✅ | ❌ | ❌ |
|
|
68
96
|
| | | View | ✅ | ✅ | ✅ |
|
|
69
97
|
| | | Deploy | ✅ | ❌ | ❌ |
|
|
@@ -106,3 +134,11 @@ The following table outlines the permissions available to each project role.
|
|
|
106
134
|
| | | View | ✅ | ✅ | ✅ |
|
|
107
135
|
| | Development | Edit | ✅ | ❌ | ❌ |
|
|
108
136
|
| | | View | ✅ | ✅ | ✅ |
|
|
137
|
+
|
|
138
|
+
The AI Teams and AI Apps rows show project-wide access granted directly by a
|
|
139
|
+
Zuplo account or Zuplo project role. A [team role](../../ai-gateway/teams.mdx)
|
|
140
|
+
can grant additional access to a specific team and its apps, but only when the
|
|
141
|
+
user also has permission to view the associated Zuplo project. A team **Admin**
|
|
142
|
+
can manage the team's settings, members, and apps, while a team **Member** can
|
|
143
|
+
access its apps. AI provider permissions come only from the user's Zuplo account
|
|
144
|
+
or Zuplo project role; team membership doesn't grant provider access.
|
|
@@ -77,6 +77,14 @@
|
|
|
77
77
|
}
|
|
78
78
|
},
|
|
79
79
|
"additionalProperties": false
|
|
80
|
+
},
|
|
81
|
+
"onError": {
|
|
82
|
+
"type": "string",
|
|
83
|
+
"title": "On Firewall Error",
|
|
84
|
+
"x-advanced": true,
|
|
85
|
+
"description": "What to do when the Akamai detect call itself fails (invalid API key, rate limit, outage). 'block' (the default) fails closed and returns a 502 so unverified content is never served; 'allow' fails open and lets the request through without inspection. An onError value in the AI Gateway app configuration takes precedence over this option.",
|
|
86
|
+
"enum": ["block", "allow"],
|
|
87
|
+
"default": "block"
|
|
80
88
|
}
|
|
81
89
|
},
|
|
82
90
|
"required": ["configurationId", "api-key"],
|
|
@@ -88,6 +96,7 @@
|
|
|
88
96
|
"export": "AkamaiAIFirewallInboundPolicy",
|
|
89
97
|
"module": "$import(@zuplo/runtime)",
|
|
90
98
|
"options": {
|
|
99
|
+
"onError": "block",
|
|
91
100
|
"streamingAccumulation": {
|
|
92
101
|
"enabled": true,
|
|
93
102
|
"eventsInterval": 5
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "zuplo",
|
|
3
|
-
"version": "7.1.
|
|
3
|
+
"version": "7.1.10",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "The programmable API Gateway",
|
|
6
6
|
"author": "Zuplo, Inc.",
|
|
@@ -19,9 +19,9 @@
|
|
|
19
19
|
"zuplo": "zuplo.js"
|
|
20
20
|
},
|
|
21
21
|
"dependencies": {
|
|
22
|
-
"@zuplo/cli": "7.1.
|
|
23
|
-
"@zuplo/core": "7.1.
|
|
24
|
-
"@zuplo/runtime": "7.1.
|
|
25
|
-
"@zuplo/test": "7.1.
|
|
22
|
+
"@zuplo/cli": "7.1.10",
|
|
23
|
+
"@zuplo/core": "7.1.10",
|
|
24
|
+
"@zuplo/runtime": "7.1.10",
|
|
25
|
+
"@zuplo/test": "7.1.10"
|
|
26
26
|
}
|
|
27
27
|
}
|