zuplo 7.1.0 → 7.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/docs/ai-gateway/apps.mdx +28 -10
- package/docs/ai-gateway/cookbooks/custom-fallback.mdx +96 -0
- package/docs/ai-gateway/cookbooks/dynamic-model-routing.mdx +119 -0
- package/docs/ai-gateway/custom-policies.mdx +213 -0
- package/docs/ai-gateway/custom-providers.mdx +10 -2
- package/docs/ai-gateway/fallback.mdx +48 -75
- package/docs/ai-gateway/getting-started.mdx +200 -132
- package/docs/ai-gateway/integrations/ai-sdk.mdx +71 -28
- package/docs/ai-gateway/integrations/claude-code.mdx +32 -8
- package/docs/ai-gateway/integrations/codex.mdx +28 -10
- package/docs/ai-gateway/integrations/goose.mdx +26 -19
- package/docs/ai-gateway/integrations/langchain.mdx +31 -15
- package/docs/ai-gateway/integrations/openai.mdx +28 -16
- package/docs/ai-gateway/introduction.mdx +73 -32
- package/docs/ai-gateway/managing-apps.mdx +62 -17
- package/docs/ai-gateway/managing-providers.mdx +23 -7
- package/docs/ai-gateway/managing-teams.mdx +29 -12
- package/docs/ai-gateway/policies/akamai-ai-firewall.mdx +92 -96
- package/docs/ai-gateway/policies/authentication.mdx +44 -0
- package/docs/ai-gateway/policies/comet-opik-tracing.mdx +40 -2
- package/docs/ai-gateway/policies/fallback-model.mdx +40 -0
- package/docs/ai-gateway/policies/galileo-tracing.mdx +36 -22
- package/docs/ai-gateway/policies/metering.mdx +47 -0
- package/docs/ai-gateway/policies/model-filtering.mdx +49 -0
- package/docs/ai-gateway/policies/overview.mdx +28 -0
- package/docs/ai-gateway/policies/semantic-cache.mdx +48 -0
- package/docs/ai-gateway/policy-chains.mdx +178 -0
- package/docs/ai-gateway/policy-templates.mdx +68 -0
- package/docs/ai-gateway/providers.mdx +23 -9
- package/docs/ai-gateway/source-control.mdx +88 -0
- package/docs/ai-gateway/teams.mdx +16 -10
- package/docs/ai-gateway/universal-api.mdx +39 -12
- package/docs/ai-gateway/usage-limits.mdx +87 -55
- package/docs/policies/cdn-cache-control-outbound/doc.md +13 -13
- package/package.json +5 -5
- package/docs/ai-gateway/guardrails.mdx +0 -65
|
@@ -1,6 +1,9 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Claude Code
|
|
3
3
|
sidebar_label: Claude Code
|
|
4
|
+
description:
|
|
5
|
+
Point Claude Code at an AI Gateway app so every Claude Code request is
|
|
6
|
+
authenticated, metered, and routed by the gateway.
|
|
4
7
|
---
|
|
5
8
|
|
|
6
9
|
The Zuplo AI Gateway supports the [Anthropic](https://docs.claude.com/en/home)
|
|
@@ -12,17 +15,17 @@ the AI Gateway.
|
|
|
12
15
|
|
|
13
16
|
<Stepper>
|
|
14
17
|
|
|
15
|
-
1. Create a [new provider](
|
|
18
|
+
1. Create a [new provider](../managing-providers.mdx) in the AI Gateway for
|
|
16
19
|
Anthropic
|
|
17
20
|
|
|
18
|
-
2. [Set up a new team](
|
|
21
|
+
2. [Set up a new team](../managing-teams.mdx)
|
|
19
22
|
|
|
20
|
-
3. Create a [new app](
|
|
21
|
-
|
|
23
|
+
3. Create a [new app](../managing-apps.mdx) to use with Claude Code and assign
|
|
24
|
+
it to the team you created
|
|
22
25
|
|
|
23
|
-
4. Copy the API
|
|
26
|
+
4. Copy the **API URL** and **API Key** shown at the top of the app page
|
|
24
27
|
|
|
25
|
-
5. Add the API key and
|
|
28
|
+
5. Add the API key and gateway URL to your environment, or Claude Code settings,
|
|
26
29
|
using either approach below
|
|
27
30
|
|
|
28
31
|
</Stepper>
|
|
@@ -31,7 +34,7 @@ the AI Gateway.
|
|
|
31
34
|
|
|
32
35
|
```
|
|
33
36
|
ANTHROPIC_AUTH_TOKEN=<your-ai-gateway-app-api-key>
|
|
34
|
-
ANTHROPIC_BASE_URL
|
|
37
|
+
ANTHROPIC_BASE_URL=https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e
|
|
35
38
|
```
|
|
36
39
|
|
|
37
40
|
### Using settings.json
|
|
@@ -40,10 +43,31 @@ ANTHROPIC_BASE_URL=<your-ai-gateway-url>
|
|
|
40
43
|
{
|
|
41
44
|
"env": {
|
|
42
45
|
"ANTHROPIC_AUTH_TOKEN": "<your-ai-gateway-app-api-key>",
|
|
43
|
-
"ANTHROPIC_BASE_URL": "
|
|
46
|
+
"ANTHROPIC_BASE_URL": "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e"
|
|
44
47
|
}
|
|
45
48
|
}
|
|
46
49
|
```
|
|
47
50
|
|
|
51
|
+
:::note
|
|
52
|
+
|
|
53
|
+
Claude Code appends `/v1/messages` to the base URL itself, so
|
|
54
|
+
`ANTHROPIC_BASE_URL` is the app's URL _without_ the `/v1` suffix.
|
|
55
|
+
|
|
56
|
+
:::
|
|
57
|
+
|
|
48
58
|
Restart Claude and it will switch to using your new AI Gateway configuration and
|
|
49
59
|
all your Claude Code LLM requests will route through the AI Gateway.
|
|
60
|
+
|
|
61
|
+
### Choosing a model
|
|
62
|
+
|
|
63
|
+
The gateway routes models named as `providerName/model`, where `providerName` is
|
|
64
|
+
the provider name configured in your gateway. Claude Code's default model names
|
|
65
|
+
don't carry that prefix, so set the model explicitly:
|
|
66
|
+
|
|
67
|
+
```
|
|
68
|
+
ANTHROPIC_MODEL=anthropic/claude-sonnet-4-5-20250929
|
|
69
|
+
ANTHROPIC_SMALL_FAST_MODEL=anthropic/claude-haiku-4-5
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
The app's [Model Filtering](../policies/model-filtering.mdx) policy controls
|
|
73
|
+
which models the app may use.
|
|
@@ -1,6 +1,9 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Codex
|
|
3
3
|
sidebar_label: Codex
|
|
4
|
+
description:
|
|
5
|
+
Configure the Codex CLI to send its requests through an AI Gateway app, either
|
|
6
|
+
by rerouting the built-in OpenAI provider or by adding a Zuplo provider.
|
|
4
7
|
---
|
|
5
8
|
|
|
6
9
|
[Codex](https://developers.openai.com/codex) is a coding agent developed by
|
|
@@ -15,15 +18,15 @@ first:
|
|
|
15
18
|
|
|
16
19
|
<Stepper>
|
|
17
20
|
|
|
18
|
-
1. Create a [new provider](
|
|
19
|
-
|
|
21
|
+
1. Create a [new provider](../managing-providers.mdx) in the AI Gateway for the
|
|
22
|
+
provider you want to use with Codex
|
|
20
23
|
|
|
21
|
-
2. [Set up a new team](
|
|
24
|
+
2. [Set up a new team](../managing-teams.mdx)
|
|
22
25
|
|
|
23
|
-
3. Create a [new app](
|
|
24
|
-
|
|
26
|
+
3. Create a [new app](../managing-apps.mdx) to use specifically with Codex and
|
|
27
|
+
assign it to the team you created
|
|
25
28
|
|
|
26
|
-
4. Copy the API
|
|
29
|
+
4. Copy the **API URL** and **API Key** shown at the top of the app page
|
|
27
30
|
|
|
28
31
|
</Stepper>
|
|
29
32
|
|
|
@@ -45,13 +48,14 @@ Configuration steps for both options are below:
|
|
|
45
48
|
|
|
46
49
|
Open the [Codex configuration](https://developers.openai.com/codex/local-config)
|
|
47
50
|
`~/.codex/config.toml` and modify the `model_providers.openai-chat-completions`
|
|
48
|
-
entry so that the `base_url` points to your
|
|
49
|
-
|
|
51
|
+
entry so that the `base_url` points to your app's API URL with `/v1` appended,
|
|
52
|
+
and set your `env_key` to the API Key of the app you created to use with Codex.
|
|
53
|
+
The app page in the Zuplo Portal shows the API URL, which ends in the app's ID.
|
|
50
54
|
|
|
51
55
|
```toml
|
|
52
56
|
[model_providers.openai-chat-completions]
|
|
53
57
|
name = "OpenAI using Chat Completions"
|
|
54
|
-
base_url = "https
|
|
58
|
+
base_url = "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1"
|
|
55
59
|
env_key = "ZUPLO_AI_GATEWAY_API_KEY"
|
|
56
60
|
```
|
|
57
61
|
|
|
@@ -66,7 +70,7 @@ the Codex `config.toml` file.
|
|
|
66
70
|
```toml
|
|
67
71
|
[model_providers.zuplo]
|
|
68
72
|
name = "Zuplo AI Gateway"
|
|
69
|
-
base_url = "https
|
|
73
|
+
base_url = "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1"
|
|
70
74
|
env_key = "ZUPLO_AI_GATEWAY_API_KEY"
|
|
71
75
|
```
|
|
72
76
|
|
|
@@ -76,3 +80,17 @@ available and you can switch Codex over to use it at any time by running:
|
|
|
76
80
|
```bash
|
|
77
81
|
codex --config model_provider="zuplo"
|
|
78
82
|
```
|
|
83
|
+
|
|
84
|
+
### Choosing a model
|
|
85
|
+
|
|
86
|
+
The app's [Model Filtering](../policies/model-filtering.mdx) policy controls
|
|
87
|
+
which models the app may use. Codex names the model as `providerName/model`,
|
|
88
|
+
where `providerName` is the provider name configured in your gateway:
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
codex --config model_provider="zuplo" --model "openai/gpt-5"
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
A request with no model, or a model without that prefix, gets a `400`. Adding
|
|
95
|
+
[Model Filtering](../policies/model-filtering.mdx) with an allow list also
|
|
96
|
+
supplies a default, letting requests omit the model.
|
|
@@ -1,6 +1,9 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: goose
|
|
3
3
|
sidebar_label: Goose
|
|
4
|
+
description:
|
|
5
|
+
Add your AI Gateway app as an OpenAI-compatible provider in the goose CLI or
|
|
6
|
+
desktop app so goose's requests route through the gateway.
|
|
4
7
|
---
|
|
5
8
|
|
|
6
9
|
[goose](https://block.github.io/goose/) is a local AI agent and CLI tool for
|
|
@@ -16,15 +19,15 @@ first:
|
|
|
16
19
|
|
|
17
20
|
<Stepper>
|
|
18
21
|
|
|
19
|
-
1. Create a [new provider](
|
|
20
|
-
|
|
22
|
+
1. Create a [new provider](../managing-providers.mdx) in the AI Gateway for the
|
|
23
|
+
provider you want to use with goose
|
|
21
24
|
|
|
22
|
-
2. [Set up a new team](
|
|
25
|
+
2. [Set up a new team](../managing-teams.mdx)
|
|
23
26
|
|
|
24
|
-
3. Create a [new app](
|
|
25
|
-
|
|
27
|
+
3. Create a [new app](../managing-apps.mdx) to use specifically with goose and
|
|
28
|
+
assign it to the team you created
|
|
26
29
|
|
|
27
|
-
4. Copy the API
|
|
30
|
+
4. Copy the **API URL** and **API Key** shown at the top of the app page
|
|
28
31
|
|
|
29
32
|
</Stepper>
|
|
30
33
|
|
|
@@ -47,20 +50,24 @@ configuration approaches depending on which version you choose.
|
|
|
47
50
|
4. Set the `OPENAI_API_KEY` to the API Key for the app you created for goose in
|
|
48
51
|
Zuplo
|
|
49
52
|
|
|
50
|
-
5. Set the `OPENAI_HOST` to
|
|
51
|
-
|
|
53
|
+
5. Set the `OPENAI_HOST` to your app's gateway URL _without_ the `/v1` suffix
|
|
54
|
+
(for example
|
|
55
|
+
`https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e`)—the
|
|
56
|
+
base path in the next step supplies it
|
|
52
57
|
|
|
53
58
|
6. The `OPENAI_BASE_PATH` may already be configured. If it's not, enter
|
|
54
59
|
`v1/chat/completions` as the value
|
|
55
60
|
|
|
56
|
-
7. Enter the model you want to use with goose
|
|
61
|
+
7. Enter the model you want to use with goose, as `providerName/model`—for
|
|
62
|
+
example `openai/gpt-5-mini`.
|
|
57
63
|
|
|
58
64
|
</Stepper>
|
|
59
65
|
|
|
60
66
|
:::note
|
|
61
67
|
|
|
62
|
-
The
|
|
63
|
-
|
|
68
|
+
The app's [Model Filtering](../policies/model-filtering.mdx) policy controls
|
|
69
|
+
which models the app may use, so the model you enter here must be one that
|
|
70
|
+
policy allows.
|
|
64
71
|
|
|
65
72
|
:::
|
|
66
73
|
|
|
@@ -70,12 +77,12 @@ the AI Gateway.
|
|
|
70
77
|
|
|
71
78
|
### Desktop App
|
|
72
79
|
|
|
73
|
-
If you are using the goose desktop
|
|
74
|
-
|
|
80
|
+
If you are using the goose desktop app you can add a Custom Provider for AI
|
|
81
|
+
Gateway by following these steps:
|
|
75
82
|
|
|
76
83
|
<Stepper>
|
|
77
84
|
|
|
78
|
-
1. Open the goose desktop
|
|
85
|
+
1. Open the goose desktop app
|
|
79
86
|
|
|
80
87
|
2. Click on **Settings**
|
|
81
88
|
|
|
@@ -88,17 +95,17 @@ AI Gateway by following these steps:
|
|
|
88
95
|
|
|
89
96
|
6. Enter a **Display Name** (for example Zuplo AI Gateway)
|
|
90
97
|
|
|
91
|
-
7. Set the **API URL** to
|
|
98
|
+
7. Set the **API URL** to your app's gateway URL
|
|
92
99
|
|
|
93
100
|
8. Set the **API Key** to the API key of the app you created for goose in Zuplo
|
|
94
101
|
|
|
95
|
-
9. Set the list of **Available Models** to the
|
|
96
|
-
|
|
97
|
-
gpt-5-nano)
|
|
102
|
+
9. Set the list of **Available Models** to the models the app's Model Filtering
|
|
103
|
+
policy allows, named as `providerName/model` (for example
|
|
104
|
+
`openai/gpt-5-mini`, `openai/gpt-5`, `openai/gpt-5-nano`)
|
|
98
105
|
|
|
99
106
|
10. Click on **Create Provider**
|
|
100
107
|
|
|
101
108
|
</Stepper>
|
|
102
109
|
|
|
103
110
|
The Zuplo AI Gateway and associated provider and models will now be available to
|
|
104
|
-
use with the goose desktop
|
|
111
|
+
use with the goose desktop app.
|
|
@@ -1,6 +1,9 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: LangChain
|
|
3
3
|
sidebar_label: LangChain SDK
|
|
4
|
+
description:
|
|
5
|
+
Point LangChain's ChatOpenAI integration at your AI Gateway app's URL so every
|
|
6
|
+
LangChain call is routed and metered by the gateway.
|
|
4
7
|
---
|
|
5
8
|
|
|
6
9
|
LangChain is a framework for developing applications that are powered by Large
|
|
@@ -10,20 +13,20 @@ integrates with hundreds of providers.
|
|
|
10
13
|
|
|
11
14
|
## Prerequisites
|
|
12
15
|
|
|
13
|
-
In order to use the AI Gateway with any LangChain powered
|
|
14
|
-
|
|
16
|
+
In order to use the AI Gateway with any LangChain powered app you will need to
|
|
17
|
+
complete these steps first:
|
|
15
18
|
|
|
16
19
|
<Stepper>
|
|
17
20
|
|
|
18
|
-
1. Create a [new provider](
|
|
19
|
-
|
|
21
|
+
1. Create a [new provider](../managing-providers.mdx) in the AI Gateway for the
|
|
22
|
+
provider you want to use with LangChain
|
|
20
23
|
|
|
21
|
-
2. [Set up a new team](
|
|
24
|
+
2. [Set up a new team](../managing-teams.mdx)
|
|
22
25
|
|
|
23
|
-
3. Create a [new app](
|
|
24
|
-
|
|
26
|
+
3. Create a [new app](../managing-apps.mdx) to use specifically with LangChain
|
|
27
|
+
and assign it to the team you created
|
|
25
28
|
|
|
26
|
-
4. Copy the API
|
|
29
|
+
4. Copy the **API URL** and **API Key** shown at the top of the app page
|
|
27
30
|
|
|
28
31
|
</Stepper>
|
|
29
32
|
|
|
@@ -46,21 +49,34 @@ def init_chat_model():
|
|
|
46
49
|
print("❌ Error: Please set your ZUPLO_AI_GATEWAY_API_KEY in a .env file")
|
|
47
50
|
exit(1)
|
|
48
51
|
|
|
49
|
-
# Check for custom BASE_URL - this is the
|
|
52
|
+
# Check for custom BASE_URL - this is the app's gateway URL from Zuplo
|
|
50
53
|
base_url = os.getenv("BASE_URL")
|
|
51
54
|
|
|
52
|
-
if base_url:
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
55
|
+
if not base_url:
|
|
56
|
+
print("❌ Error: Please set BASE_URL to your app's gateway URL")
|
|
57
|
+
exit(1)
|
|
58
|
+
|
|
59
|
+
return ChatOpenAI(
|
|
60
|
+
api_key=api_key, model="openai/gpt-5-mini", base_url=base_url
|
|
61
|
+
)
|
|
56
62
|
```
|
|
57
63
|
|
|
58
64
|
In the code above, checks are performed for two environment variables:
|
|
59
65
|
|
|
60
66
|
- `ZUPLO_AI_GATEWAY_API_KEY` - This is the API key of the app you have
|
|
61
67
|
configured to use with LangChain in Zuplo
|
|
62
|
-
- `BASE_URL` - This is
|
|
68
|
+
- `BASE_URL` - This is your app's API URL with `/v1` appended, for example
|
|
69
|
+
`https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1`.
|
|
70
|
+
The app page in the Zuplo Portal shows the API URL, which ends in the app's ID
|
|
63
71
|
|
|
64
72
|
Both of these values are passed to `ChatOpenAI` when it's instantiated,
|
|
65
73
|
switching the configuration from using default OpenAI APIs to using Zuplo's AI
|
|
66
|
-
Gateway
|
|
74
|
+
Gateway.
|
|
75
|
+
|
|
76
|
+
Always name the model as `providerName/model`, where `providerName` is the
|
|
77
|
+
provider name configured in your gateway. The gateway needs the prefix to know
|
|
78
|
+
which provider to route to, and a request with no model, or a model without that
|
|
79
|
+
prefix, gets a `400`. Adding the
|
|
80
|
+
[Model Filtering](../policies/model-filtering.mdx) policy restricts an app to
|
|
81
|
+
certain models, and an allow list also supplies a default so requests may omit
|
|
82
|
+
the model.
|
|
@@ -1,6 +1,9 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: OpenAI SDK
|
|
3
3
|
sidebar_label: OpenAI SDK
|
|
4
|
+
description:
|
|
5
|
+
Set the OpenAI SDK's baseURL to your AI Gateway app's URL to route chat
|
|
6
|
+
completions, embeddings, and responses requests through the gateway.
|
|
4
7
|
---
|
|
5
8
|
|
|
6
9
|
The
|
|
@@ -10,20 +13,20 @@ OpenAI compatible models from other providers in Node.js.
|
|
|
10
13
|
|
|
11
14
|
## Prerequisites
|
|
12
15
|
|
|
13
|
-
In order to use the AI Gateway with any OpenAI SDK powered
|
|
14
|
-
|
|
16
|
+
In order to use the AI Gateway with any OpenAI SDK powered app you will need to
|
|
17
|
+
complete these steps first:
|
|
15
18
|
|
|
16
19
|
<Stepper>
|
|
17
20
|
|
|
18
|
-
1. Create a [new provider](
|
|
21
|
+
1. Create a [new provider](../managing-providers.mdx) in the AI Gateway for
|
|
19
22
|
OpenAI
|
|
20
23
|
|
|
21
|
-
2. [Set up a new team](
|
|
24
|
+
2. [Set up a new team](../managing-teams.mdx)
|
|
22
25
|
|
|
23
|
-
3. Create a [new app](
|
|
24
|
-
|
|
26
|
+
3. Create a [new app](../managing-apps.mdx) to use specifically with the OpenAI
|
|
27
|
+
SDK and assign it to the team you created
|
|
25
28
|
|
|
26
|
-
4. Copy the API
|
|
29
|
+
4. Copy the **API URL** and **API Key** shown at the top of the app page
|
|
27
30
|
|
|
28
31
|
</Stepper>
|
|
29
32
|
|
|
@@ -32,23 +35,31 @@ need to complete these steps first:
|
|
|
32
35
|
To route all OpenAI SDK requests through Zuplo instead of directly to the OpenAI
|
|
33
36
|
API, you must set the API `baseUrl` in the SDK configuration.
|
|
34
37
|
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
to https://my-ai-gateway.zuplo.app/v1.
|
|
38
|
+
The app page in the Zuplo Portal shows each app's API URL. It ends in the app's
|
|
39
|
+
ID, so append `/v1` to it to get the base URL for your API client.
|
|
38
40
|
|
|
39
41
|
Additionally, change the value of `apiKey` to the API key of the app you have
|
|
40
42
|
configured in Zuplo
|
|
41
43
|
|
|
44
|
+
Always name the model as `providerName/model`, where `providerName` is the
|
|
45
|
+
provider name configured in your gateway. The gateway needs the prefix to know
|
|
46
|
+
which provider to route to, and a request with no model, or a model without that
|
|
47
|
+
prefix, gets a `400`. Adding the
|
|
48
|
+
[Model Filtering](../policies/model-filtering.mdx) policy restricts an app to
|
|
49
|
+
certain models, and an allow list also supplies a default so requests may omit
|
|
50
|
+
the model.
|
|
51
|
+
|
|
42
52
|
```typescript
|
|
43
53
|
import OpenAI from "openai";
|
|
44
54
|
|
|
45
55
|
const client = new OpenAI({
|
|
46
56
|
apiKey: process.env.ZUPLO_AI_GATEWAY_API_KEY,
|
|
47
|
-
baseURL:
|
|
57
|
+
baseURL:
|
|
58
|
+
"https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1",
|
|
48
59
|
});
|
|
49
60
|
|
|
50
61
|
const response = await client.chat.completions.create({
|
|
51
|
-
model: "gpt-
|
|
62
|
+
model: "openai/gpt-5-mini",
|
|
52
63
|
messages: [
|
|
53
64
|
{
|
|
54
65
|
role: "user",
|
|
@@ -65,8 +76,8 @@ APIs to using Zuplo's AI Gateway for all requests.
|
|
|
65
76
|
|
|
66
77
|
## Supported Endpoints
|
|
67
78
|
|
|
68
|
-
The AI Gateway supports all major OpenAI API endpoints through the
|
|
69
|
-
API:
|
|
79
|
+
The AI Gateway supports all major OpenAI API endpoints through the
|
|
80
|
+
[universal API](../universal-api.mdx):
|
|
70
81
|
|
|
71
82
|
- **Chat Completions** (`/v1/chat/completions`) - For conversational AI
|
|
72
83
|
interactions
|
|
@@ -84,11 +95,12 @@ import OpenAI from "openai";
|
|
|
84
95
|
|
|
85
96
|
const client = new OpenAI({
|
|
86
97
|
apiKey: process.env.ZUPLO_AI_GATEWAY_API_KEY,
|
|
87
|
-
baseURL:
|
|
98
|
+
baseURL:
|
|
99
|
+
"https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1",
|
|
88
100
|
});
|
|
89
101
|
|
|
90
102
|
const response = await client.responses.create({
|
|
91
|
-
model: "gpt-5",
|
|
103
|
+
model: "openai/gpt-5",
|
|
92
104
|
input: "Write a one-sentence bedtime story about a unicorn.",
|
|
93
105
|
});
|
|
94
106
|
|
|
@@ -1,33 +1,49 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Zuplo AI Gateway
|
|
3
3
|
sidebar_label: Introduction
|
|
4
|
+
description:
|
|
5
|
+
The Zuplo AI Gateway is a proxy between your apps and LLM providers, deployed
|
|
6
|
+
from your own Git repository, with per-app policy pipelines for cost control,
|
|
7
|
+
model access, guardrails, and observability.
|
|
4
8
|
---
|
|
5
9
|
|
|
6
10
|
Zuplo's AI Gateway acts as an intelligent proxy layer that sits between your
|
|
7
|
-
engineering team's
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
+
engineering team's apps and LLM providers like OpenAI, Anthropic, Google,
|
|
12
|
+
Mistral, and xAI. Instead of your apps communicating directly with these
|
|
13
|
+
providers, all requests flow through the Zuplo AI Gateway, which streams
|
|
14
|
+
responses while applying policies, controls, and monitoring.
|
|
15
|
+
|
|
16
|
+
Each piece of software that calls the gateway registers as an **app**—your
|
|
17
|
+
support chatbot is one app; an internal coding agent is another. A single AI
|
|
18
|
+
Gateway holds as many apps as you need, and you control exactly which policies
|
|
19
|
+
run for each one. Every app has its own ordered policy chain, so the
|
|
20
|
+
customer-facing chatbot can enforce budgets and guardrails while the internal
|
|
21
|
+
agent runs with none. Beyond the built-in policies, you can write your own in
|
|
22
|
+
TypeScript, and apps add them to their chains like any other policy. Chain
|
|
23
|
+
changes apply within about a minute, with no redeploy. See [Apps](./apps.mdx).
|
|
11
24
|
|
|
12
25
|
## Key Benefits
|
|
13
26
|
|
|
14
27
|
**Provider Independence**: Switch between LLM providers (OpenAI, Anthropic,
|
|
15
|
-
Google, Mistral, xAI, and more) dynamically without modifying
|
|
16
|
-
Configure
|
|
17
|
-
|
|
28
|
+
Google, Mistral, xAI, and more) dynamically without modifying app code.
|
|
29
|
+
Configure model access through the gateway rather than hard coding it into your
|
|
30
|
+
apps.
|
|
18
31
|
|
|
19
|
-
**Cost Control**: Set spending limits at organization, team, and
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
notifications.
|
|
32
|
+
**Cost Control**: Set spending limits at organization, team, and app levels with
|
|
33
|
+
hierarchical budgets that cascade down through your structure. Configure daily
|
|
34
|
+
and monthly thresholds with enforcement or warning notifications.
|
|
23
35
|
|
|
24
36
|
**Security & Compliance**: Apply guardrails to detect and block prompt injection
|
|
25
37
|
attempts and prevent PII leakage in both requests and responses through
|
|
26
38
|
integrated AI firewall policies.
|
|
27
39
|
|
|
28
|
-
**Self-Service Access**: Developers can create
|
|
29
|
-
|
|
30
|
-
|
|
40
|
+
**Self-Service Access**: Developers can create apps and access LLMs without
|
|
41
|
+
needing direct access to provider API keys. Administrators configure providers
|
|
42
|
+
once, and teams consume them securely.
|
|
43
|
+
|
|
44
|
+
**Extensibility**: Write custom policies in TypeScript—content filters, dynamic
|
|
45
|
+
model routing, plan-based limits—and let apps add them to their chains like any
|
|
46
|
+
built-in policy.
|
|
31
47
|
|
|
32
48
|
**Performance Optimization**: Enable semantic caching to identify and return
|
|
33
49
|
cached responses for similar prompts, reducing costs and improving response
|
|
@@ -38,12 +54,20 @@ time-to-first-byte metrics, and spending patterns across your organization.
|
|
|
38
54
|
|
|
39
55
|
## How It Works
|
|
40
56
|
|
|
41
|
-
Your
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
57
|
+
Your code sends its LLM requests to your AI Gateway instead of directly to a
|
|
58
|
+
provider. Each app has its own URL, API key, and policy chain, so one gateway
|
|
59
|
+
serves teams with completely different configurations.
|
|
60
|
+
|
|
61
|
+
Each request arrives at the URL of the app it belongs to. The gateway identifies
|
|
62
|
+
that app, runs its policy chain (model access, cost controls, security
|
|
63
|
+
guardrails), routes to the selected LLM provider, and streams the response back.
|
|
64
|
+
Throughout this process, the gateway captures metrics without exposing
|
|
65
|
+
underlying provider credentials.
|
|
66
|
+
|
|
67
|
+
Every control is opt-in. An app whose policy chain is empty proxies requests
|
|
68
|
+
immediately—each request names its own model, and no model restrictions,
|
|
69
|
+
budgets, guardrails, or caching apply until you add the policies that enforce
|
|
70
|
+
them.
|
|
47
71
|
|
|
48
72
|
## Core Features
|
|
49
73
|
|
|
@@ -52,32 +76,49 @@ credentials.
|
|
|
52
76
|
Configure multiple LLM providers within a single gateway project. Supported
|
|
53
77
|
providers include OpenAI, Anthropic, Google, Mistral, xAI, and OpenAI-compatible
|
|
54
78
|
custom providers. See [AI Providers](./providers.mdx) for the full list of
|
|
55
|
-
providers and supported capabilities.
|
|
56
|
-
|
|
79
|
+
providers and supported capabilities. Apps reference models as
|
|
80
|
+
`providerName/model`—for example `openai/gpt-5-mini`—so a single app can use
|
|
81
|
+
models from several providers.
|
|
82
|
+
|
|
83
|
+
### Source-Controlled Gateway
|
|
84
|
+
|
|
85
|
+
Every AI Gateway project connects to a Git repository that contains the
|
|
86
|
+
gateway's routes and policy declarations. The default branch is what production
|
|
87
|
+
runs, so gateway changes go through your normal review workflow. See
|
|
88
|
+
[Source Control](./source-control.mdx).
|
|
89
|
+
|
|
90
|
+
### Per-App Policy Chains
|
|
91
|
+
|
|
92
|
+
Each app runs its own ordered chain of policies—model filtering, fallback
|
|
93
|
+
models, budgets, semantic caching, guardrails, tracing, and any custom policies
|
|
94
|
+
declared in the repository. Team policy templates give new apps a consistent
|
|
95
|
+
starting pipeline. See [Policy Chains](./policy-chains.mdx) and
|
|
96
|
+
[Custom Policies](./custom-policies.mdx).
|
|
57
97
|
|
|
58
98
|
### Team Hierarchy & Budgets
|
|
59
99
|
|
|
60
100
|
Organize users into teams with hierarchical structures. Set budget limits at
|
|
61
101
|
each level that cascade down:
|
|
62
102
|
|
|
63
|
-
- **
|
|
64
|
-
- **
|
|
65
|
-
|
|
66
|
-
- **
|
|
103
|
+
- **Gateway**: Organization-wide limits (for example, $1,000/day)
|
|
104
|
+
- **Teams**: Team-specific limits that can't exceed parent limits (for example,
|
|
105
|
+
$500/day for the Credit Team)
|
|
106
|
+
- **Apps**: Per-app limits for granular control
|
|
107
|
+
|
|
108
|
+
See [Usage Limits](./usage-limits.mdx).
|
|
67
109
|
|
|
68
|
-
###
|
|
110
|
+
### App Configuration
|
|
69
111
|
|
|
70
|
-
Each
|
|
112
|
+
Each app gets its own:
|
|
71
113
|
|
|
72
114
|
- **Unique Gateway URL**: Single endpoint regardless of underlying provider
|
|
73
115
|
- **API Key**: Zuplo-managed key that never exposes provider credentials
|
|
74
|
-
- **
|
|
75
|
-
|
|
76
|
-
- **Semantic Caching**: Optional caching of similar prompts to reduce costs
|
|
116
|
+
- **Policy Chain**: Model access, budgets, caching, guardrails, and custom
|
|
117
|
+
policies, applied in the order the app chooses
|
|
77
118
|
|
|
78
119
|
## Use Cases
|
|
79
120
|
|
|
80
|
-
- **Multi-tenant AI
|
|
121
|
+
- **Multi-tenant AI Apps**: Enforce spending limits per customer or team
|
|
81
122
|
- **Agent Development**: Build AI agents that can switch providers without code
|
|
82
123
|
changes
|
|
83
124
|
- **Cost Management**: Control and monitor LLM spending across your organization
|