zuplo 7.1.0 → 7.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (36) hide show
  1. package/docs/ai-gateway/apps.mdx +28 -10
  2. package/docs/ai-gateway/cookbooks/custom-fallback.mdx +96 -0
  3. package/docs/ai-gateway/cookbooks/dynamic-model-routing.mdx +119 -0
  4. package/docs/ai-gateway/custom-policies.mdx +213 -0
  5. package/docs/ai-gateway/custom-providers.mdx +10 -2
  6. package/docs/ai-gateway/fallback.mdx +48 -75
  7. package/docs/ai-gateway/getting-started.mdx +200 -132
  8. package/docs/ai-gateway/integrations/ai-sdk.mdx +71 -28
  9. package/docs/ai-gateway/integrations/claude-code.mdx +32 -8
  10. package/docs/ai-gateway/integrations/codex.mdx +28 -10
  11. package/docs/ai-gateway/integrations/goose.mdx +26 -19
  12. package/docs/ai-gateway/integrations/langchain.mdx +31 -15
  13. package/docs/ai-gateway/integrations/openai.mdx +28 -16
  14. package/docs/ai-gateway/introduction.mdx +73 -32
  15. package/docs/ai-gateway/managing-apps.mdx +62 -17
  16. package/docs/ai-gateway/managing-providers.mdx +23 -7
  17. package/docs/ai-gateway/managing-teams.mdx +29 -12
  18. package/docs/ai-gateway/policies/akamai-ai-firewall.mdx +92 -96
  19. package/docs/ai-gateway/policies/authentication.mdx +44 -0
  20. package/docs/ai-gateway/policies/comet-opik-tracing.mdx +40 -2
  21. package/docs/ai-gateway/policies/fallback-model.mdx +40 -0
  22. package/docs/ai-gateway/policies/galileo-tracing.mdx +36 -22
  23. package/docs/ai-gateway/policies/metering.mdx +47 -0
  24. package/docs/ai-gateway/policies/model-filtering.mdx +49 -0
  25. package/docs/ai-gateway/policies/overview.mdx +28 -0
  26. package/docs/ai-gateway/policies/semantic-cache.mdx +48 -0
  27. package/docs/ai-gateway/policy-chains.mdx +178 -0
  28. package/docs/ai-gateway/policy-templates.mdx +68 -0
  29. package/docs/ai-gateway/providers.mdx +23 -9
  30. package/docs/ai-gateway/source-control.mdx +88 -0
  31. package/docs/ai-gateway/teams.mdx +16 -10
  32. package/docs/ai-gateway/universal-api.mdx +39 -12
  33. package/docs/ai-gateway/usage-limits.mdx +87 -55
  34. package/docs/policies/cdn-cache-control-outbound/doc.md +13 -13
  35. package/package.json +5 -5
  36. package/docs/ai-gateway/guardrails.mdx +0 -65
@@ -1,6 +1,9 @@
1
1
  ---
2
2
  title: Claude Code
3
3
  sidebar_label: Claude Code
4
+ description:
5
+ Point Claude Code at an AI Gateway app so every Claude Code request is
6
+ authenticated, metered, and routed by the gateway.
4
7
  ---
5
8
 
6
9
  The Zuplo AI Gateway supports the [Anthropic](https://docs.claude.com/en/home)
@@ -12,17 +15,17 @@ the AI Gateway.
12
15
 
13
16
  <Stepper>
14
17
 
15
- 1. Create a [new provider](/ai-gateway/managing-providers) in the AI Gateway for
18
+ 1. Create a [new provider](../managing-providers.mdx) in the AI Gateway for
16
19
  Anthropic
17
20
 
18
- 2. [Set up a new team](/ai-gateway/managing-teams)
21
+ 2. [Set up a new team](../managing-teams.mdx)
19
22
 
20
- 3. Create a [new app](/ai-gateway/managing-apps) to use with Claude Code and
21
- assign it to the team you created
23
+ 3. Create a [new app](../managing-apps.mdx) to use with Claude Code and assign
24
+ it to the team you created
22
25
 
23
- 4. Copy the API Key for the app you created, as well as the Gateway URL
26
+ 4. Copy the **API URL** and **API Key** shown at the top of the app page
24
27
 
25
- 5. Add the API key and Gateway URL to your environment, or Claude Code settings,
28
+ 5. Add the API key and gateway URL to your environment, or Claude Code settings,
26
29
  using either approach below
27
30
 
28
31
  </Stepper>
@@ -31,7 +34,7 @@ the AI Gateway.
31
34
 
32
35
  ```
33
36
  ANTHROPIC_AUTH_TOKEN=<your-ai-gateway-app-api-key>
34
- ANTHROPIC_BASE_URL=<your-ai-gateway-url>
37
+ ANTHROPIC_BASE_URL=https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e
35
38
  ```
36
39
 
37
40
  ### Using settings.json
@@ -40,10 +43,31 @@ ANTHROPIC_BASE_URL=<your-ai-gateway-url>
40
43
  {
41
44
  "env": {
42
45
  "ANTHROPIC_AUTH_TOKEN": "<your-ai-gateway-app-api-key>",
43
- "ANTHROPIC_BASE_URL": "<your-ai-gateway-url>"
46
+ "ANTHROPIC_BASE_URL": "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e"
44
47
  }
45
48
  }
46
49
  ```
47
50
 
51
+ :::note
52
+
53
+ Claude Code appends `/v1/messages` to the base URL itself, so
54
+ `ANTHROPIC_BASE_URL` is the app's URL _without_ the `/v1` suffix.
55
+
56
+ :::
57
+
48
58
  Restart Claude and it will switch to using your new AI Gateway configuration and
49
59
  all your Claude Code LLM requests will route through the AI Gateway.
60
+
61
+ ### Choosing a model
62
+
63
+ The gateway routes models named as `providerName/model`, where `providerName` is
64
+ the provider name configured in your gateway. Claude Code's default model names
65
+ don't carry that prefix, so set the model explicitly:
66
+
67
+ ```
68
+ ANTHROPIC_MODEL=anthropic/claude-sonnet-4-5-20250929
69
+ ANTHROPIC_SMALL_FAST_MODEL=anthropic/claude-haiku-4-5
70
+ ```
71
+
72
+ The app's [Model Filtering](../policies/model-filtering.mdx) policy controls
73
+ which models the app may use.
@@ -1,6 +1,9 @@
1
1
  ---
2
2
  title: Codex
3
3
  sidebar_label: Codex
4
+ description:
5
+ Configure the Codex CLI to send its requests through an AI Gateway app, either
6
+ by rerouting the built-in OpenAI provider or by adding a Zuplo provider.
4
7
  ---
5
8
 
6
9
  [Codex](https://developers.openai.com/codex) is a coding agent developed by
@@ -15,15 +18,15 @@ first:
15
18
 
16
19
  <Stepper>
17
20
 
18
- 1. Create a [new provider](/ai-gateway/managing-providers) in the AI Gateway for
19
- the provider you want to use with Codex
21
+ 1. Create a [new provider](../managing-providers.mdx) in the AI Gateway for the
22
+ provider you want to use with Codex
20
23
 
21
- 2. [Set up a new team](/ai-gateway/managing-teams)
24
+ 2. [Set up a new team](../managing-teams.mdx)
22
25
 
23
- 3. Create a [new app](/ai-gateway/managing-apps) to use specifically with Codex
24
- and assign it to the team you created
26
+ 3. Create a [new app](../managing-apps.mdx) to use specifically with Codex and
27
+ assign it to the team you created
25
28
 
26
- 4. Copy the API Key for the app you created, as well as the Gateway URL
29
+ 4. Copy the **API URL** and **API Key** shown at the top of the app page
27
30
 
28
31
  </Stepper>
29
32
 
@@ -45,13 +48,14 @@ Configuration steps for both options are below:
45
48
 
46
49
  Open the [Codex configuration](https://developers.openai.com/codex/local-config)
47
50
  `~/.codex/config.toml` and modify the `model_providers.openai-chat-completions`
48
- entry so that the `base_url` points to your AI Gateway URL, and your `env_key`
49
- is set to the API Key of the app you created to use with Codex.
51
+ entry so that the `base_url` points to your app's API URL with `/v1` appended,
52
+ and set your `env_key` to the API Key of the app you created to use with Codex.
53
+ The app page in the Zuplo Portal shows the API URL, which ends in the app's ID.
50
54
 
51
55
  ```toml
52
56
  [model_providers.openai-chat-completions]
53
57
  name = "OpenAI using Chat Completions"
54
- base_url = "https://<your-ai-gateway-url>/v1"
58
+ base_url = "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1"
55
59
  env_key = "ZUPLO_AI_GATEWAY_API_KEY"
56
60
  ```
57
61
 
@@ -66,7 +70,7 @@ the Codex `config.toml` file.
66
70
  ```toml
67
71
  [model_providers.zuplo]
68
72
  name = "Zuplo AI Gateway"
69
- base_url = "https://<your-ai-gateway-url>/v1"
73
+ base_url = "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1"
70
74
  env_key = "ZUPLO_AI_GATEWAY_API_KEY"
71
75
  ```
72
76
 
@@ -76,3 +80,17 @@ available and you can switch Codex over to use it at any time by running:
76
80
  ```bash
77
81
  codex --config model_provider="zuplo"
78
82
  ```
83
+
84
+ ### Choosing a model
85
+
86
+ The app's [Model Filtering](../policies/model-filtering.mdx) policy controls
87
+ which models the app may use. Codex names the model as `providerName/model`,
88
+ where `providerName` is the provider name configured in your gateway:
89
+
90
+ ```bash
91
+ codex --config model_provider="zuplo" --model "openai/gpt-5"
92
+ ```
93
+
94
+ A request with no model, or a model without that prefix, gets a `400`. Adding
95
+ [Model Filtering](../policies/model-filtering.mdx) with an allow list also
96
+ supplies a default, letting requests omit the model.
@@ -1,6 +1,9 @@
1
1
  ---
2
2
  title: goose
3
3
  sidebar_label: Goose
4
+ description:
5
+ Add your AI Gateway app as an OpenAI-compatible provider in the goose CLI or
6
+ desktop app so goose's requests route through the gateway.
4
7
  ---
5
8
 
6
9
  [goose](https://block.github.io/goose/) is a local AI agent and CLI tool for
@@ -16,15 +19,15 @@ first:
16
19
 
17
20
  <Stepper>
18
21
 
19
- 1. Create a [new provider](/ai-gateway/managing-providers) in the AI Gateway for
20
- the provider you want to use with goose
22
+ 1. Create a [new provider](../managing-providers.mdx) in the AI Gateway for the
23
+ provider you want to use with goose
21
24
 
22
- 2. [Set up a new team](/ai-gateway/managing-teams)
25
+ 2. [Set up a new team](../managing-teams.mdx)
23
26
 
24
- 3. Create a [new app](/ai-gateway/managing-apps) to use specifically with goose
25
- and assign it to the team you created
27
+ 3. Create a [new app](../managing-apps.mdx) to use specifically with goose and
28
+ assign it to the team you created
26
29
 
27
- 4. Copy the API Key for the app you created, as well as the Gateway URL
30
+ 4. Copy the **API URL** and **API Key** shown at the top of the app page
28
31
 
29
32
  </Stepper>
30
33
 
@@ -47,20 +50,24 @@ configuration approaches depending on which version you choose.
47
50
  4. Set the `OPENAI_API_KEY` to the API Key for the app you created for goose in
48
51
  Zuplo
49
52
 
50
- 5. Set the `OPENAI_HOST` to the URL of your AI Gateway using the Gateway URL
51
- from your project
53
+ 5. Set the `OPENAI_HOST` to your app's gateway URL _without_ the `/v1` suffix
54
+ (for example
55
+ `https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e`)—the
56
+ base path in the next step supplies it
52
57
 
53
58
  6. The `OPENAI_BASE_PATH` may already be configured. If it's not, enter
54
59
  `v1/chat/completions` as the value
55
60
 
56
- 7. Enter the model you want to use with goose.
61
+ 7. Enter the model you want to use with goose, as `providerName/model`—for
62
+ example `openai/gpt-5-mini`.
57
63
 
58
64
  </Stepper>
59
65
 
60
66
  :::note
61
67
 
62
- The provider model you want to use with goose must be selected in the provider
63
- you created in the Prerequisites step above
68
+ The app's [Model Filtering](../policies/model-filtering.mdx) policy controls
69
+ which models the app may use, so the model you enter here must be one that
70
+ policy allows.
64
71
 
65
72
  :::
66
73
 
@@ -70,12 +77,12 @@ the AI Gateway.
70
77
 
71
78
  ### Desktop App
72
79
 
73
- If you are using the goose desktop application you can add a Custom Provider for
74
- AI Gateway by following these steps:
80
+ If you are using the goose desktop app you can add a Custom Provider for AI
81
+ Gateway by following these steps:
75
82
 
76
83
  <Stepper>
77
84
 
78
- 1. Open the goose desktop application
85
+ 1. Open the goose desktop app
79
86
 
80
87
  2. Click on **Settings**
81
88
 
@@ -88,17 +95,17 @@ AI Gateway by following these steps:
88
95
 
89
96
  6. Enter a **Display Name** (for example Zuplo AI Gateway)
90
97
 
91
- 7. Set the **API URL** to the URL of your Zuplo AI Gateway
98
+ 7. Set the **API URL** to your app's gateway URL
92
99
 
93
100
  8. Set the **API Key** to the API key of the app you created for goose in Zuplo
94
101
 
95
- 9. Set the list of **Available Models** to the same list you selected when you
96
- set up the provider in the Prerequisites step (for example gpt-4o, gpt-5,
97
- gpt-5-nano)
102
+ 9. Set the list of **Available Models** to the models the app's Model Filtering
103
+ policy allows, named as `providerName/model` (for example
104
+ `openai/gpt-5-mini`, `openai/gpt-5`, `openai/gpt-5-nano`)
98
105
 
99
106
  10. Click on **Create Provider**
100
107
 
101
108
  </Stepper>
102
109
 
103
110
  The Zuplo AI Gateway and associated provider and models will now be available to
104
- use with the goose desktop application.
111
+ use with the goose desktop app.
@@ -1,6 +1,9 @@
1
1
  ---
2
2
  title: LangChain
3
3
  sidebar_label: LangChain SDK
4
+ description:
5
+ Point LangChain's ChatOpenAI integration at your AI Gateway app's URL so every
6
+ LangChain call is routed and metered by the gateway.
4
7
  ---
5
8
 
6
9
  LangChain is a framework for developing applications that are powered by Large
@@ -10,20 +13,20 @@ integrates with hundreds of providers.
10
13
 
11
14
  ## Prerequisites
12
15
 
13
- In order to use the AI Gateway with any LangChain powered application you will
14
- need to complete these steps first:
16
+ In order to use the AI Gateway with any LangChain powered app you will need to
17
+ complete these steps first:
15
18
 
16
19
  <Stepper>
17
20
 
18
- 1. Create a [new provider](/ai-gateway/managing-providers) in the AI Gateway for
19
- the provider you want to use with LangChain
21
+ 1. Create a [new provider](../managing-providers.mdx) in the AI Gateway for the
22
+ provider you want to use with LangChain
20
23
 
21
- 2. [Set up a new team](/ai-gateway/managing-teams)
24
+ 2. [Set up a new team](../managing-teams.mdx)
22
25
 
23
- 3. Create a [new app](/ai-gateway/managing-apps) to use specifically with
24
- LangChain and assign it to the team you created
26
+ 3. Create a [new app](../managing-apps.mdx) to use specifically with LangChain
27
+ and assign it to the team you created
25
28
 
26
- 4. Copy the API Key for the app you created, as well as the Gateway URL
29
+ 4. Copy the **API URL** and **API Key** shown at the top of the app page
27
30
 
28
31
  </Stepper>
29
32
 
@@ -46,21 +49,34 @@ def init_chat_model():
46
49
  print("❌ Error: Please set your ZUPLO_AI_GATEWAY_API_KEY in a .env file")
47
50
  exit(1)
48
51
 
49
- # Check for custom BASE_URL - this is the AI Gateway URL from Zuplo
52
+ # Check for custom BASE_URL - this is the app's gateway URL from Zuplo
50
53
  base_url = os.getenv("BASE_URL")
51
54
 
52
- if base_url:
53
- return ChatOpenAI(api_key=api_key, model="gpt-4o", base_url=base_url)
54
- else:
55
- return ChatOpenAI(api_key=api_key, model="gpt-4o")
55
+ if not base_url:
56
+ print("❌ Error: Please set BASE_URL to your app's gateway URL")
57
+ exit(1)
58
+
59
+ return ChatOpenAI(
60
+ api_key=api_key, model="openai/gpt-5-mini", base_url=base_url
61
+ )
56
62
  ```
57
63
 
58
64
  In the code above, checks are performed for two environment variables:
59
65
 
60
66
  - `ZUPLO_AI_GATEWAY_API_KEY` - This is the API key of the app you have
61
67
  configured to use with LangChain in Zuplo
62
- - `BASE_URL` - This is the Gateway URL of your AI Gateway project in Zuplo
68
+ - `BASE_URL` - This is your app's API URL with `/v1` appended, for example
69
+ `https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1`.
70
+ The app page in the Zuplo Portal shows the API URL, which ends in the app's ID
63
71
 
64
72
  Both of these values are passed to `ChatOpenAI` when it's instantiated,
65
73
  switching the configuration from using default OpenAI APIs to using Zuplo's AI
66
- Gateway for all `gpt-4o` requests that the LangChain SDK will make.
74
+ Gateway.
75
+
76
+ Always name the model as `providerName/model`, where `providerName` is the
77
+ provider name configured in your gateway. The gateway needs the prefix to know
78
+ which provider to route to, and a request with no model, or a model without that
79
+ prefix, gets a `400`. Adding the
80
+ [Model Filtering](../policies/model-filtering.mdx) policy restricts an app to
81
+ certain models, and an allow list also supplies a default so requests may omit
82
+ the model.
@@ -1,6 +1,9 @@
1
1
  ---
2
2
  title: OpenAI SDK
3
3
  sidebar_label: OpenAI SDK
4
+ description:
5
+ Set the OpenAI SDK's baseURL to your AI Gateway app's URL to route chat
6
+ completions, embeddings, and responses requests through the gateway.
4
7
  ---
5
8
 
6
9
  The
@@ -10,20 +13,20 @@ OpenAI compatible models from other providers in Node.js.
10
13
 
11
14
  ## Prerequisites
12
15
 
13
- In order to use the AI Gateway with any OpenAI SDK powered application you will
14
- need to complete these steps first:
16
+ In order to use the AI Gateway with any OpenAI SDK powered app you will need to
17
+ complete these steps first:
15
18
 
16
19
  <Stepper>
17
20
 
18
- 1. Create a [new provider](/ai-gateway/managing-providers) in the AI Gateway for
21
+ 1. Create a [new provider](../managing-providers.mdx) in the AI Gateway for
19
22
  OpenAI
20
23
 
21
- 2. [Set up a new team](/ai-gateway/managing-teams)
24
+ 2. [Set up a new team](../managing-teams.mdx)
22
25
 
23
- 3. Create a [new app](/ai-gateway/managing-apps) to use specifically with the
24
- OpenAI SDK and assign it to the team you created
26
+ 3. Create a [new app](../managing-apps.mdx) to use specifically with the OpenAI
27
+ SDK and assign it to the team you created
25
28
 
26
- 4. Copy the API Key for the app you created, as well as the Gateway URL
29
+ 4. Copy the **API URL** and **API Key** shown at the top of the app page
27
30
 
28
31
  </Stepper>
29
32
 
@@ -32,23 +35,31 @@ need to complete these steps first:
32
35
  To route all OpenAI SDK requests through Zuplo instead of directly to the OpenAI
33
36
  API, you must set the API `baseUrl` in the SDK configuration.
34
37
 
35
- For example, if your Zuplo application is hosted at
36
- https://my-ai-gateway.zuplo.app, you can change the base URL in your API client
37
- to https://my-ai-gateway.zuplo.app/v1.
38
+ The app page in the Zuplo Portal shows each app's API URL. It ends in the app's
39
+ ID, so append `/v1` to it to get the base URL for your API client.
38
40
 
39
41
  Additionally, change the value of `apiKey` to the API key of the app you have
40
42
  configured in Zuplo
41
43
 
44
+ Always name the model as `providerName/model`, where `providerName` is the
45
+ provider name configured in your gateway. The gateway needs the prefix to know
46
+ which provider to route to, and a request with no model, or a model without that
47
+ prefix, gets a `400`. Adding the
48
+ [Model Filtering](../policies/model-filtering.mdx) policy restricts an app to
49
+ certain models, and an allow list also supplies a default so requests may omit
50
+ the model.
51
+
42
52
  ```typescript
43
53
  import OpenAI from "openai";
44
54
 
45
55
  const client = new OpenAI({
46
56
  apiKey: process.env.ZUPLO_AI_GATEWAY_API_KEY,
47
- baseURL: "https://my-ai-gateway.zuplo.app/v1",
57
+ baseURL:
58
+ "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1",
48
59
  });
49
60
 
50
61
  const response = await client.chat.completions.create({
51
- model: "gpt-4",
62
+ model: "openai/gpt-5-mini",
52
63
  messages: [
53
64
  {
54
65
  role: "user",
@@ -65,8 +76,8 @@ APIs to using Zuplo's AI Gateway for all requests.
65
76
 
66
77
  ## Supported Endpoints
67
78
 
68
- The AI Gateway supports all major OpenAI API endpoints through the universal
69
- API:
79
+ The AI Gateway supports all major OpenAI API endpoints through the
80
+ [universal API](../universal-api.mdx):
70
81
 
71
82
  - **Chat Completions** (`/v1/chat/completions`) - For conversational AI
72
83
  interactions
@@ -84,11 +95,12 @@ import OpenAI from "openai";
84
95
 
85
96
  const client = new OpenAI({
86
97
  apiKey: process.env.ZUPLO_AI_GATEWAY_API_KEY,
87
- baseURL: "https://my-ai-gateway.zuplo.app/v1",
98
+ baseURL:
99
+ "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1",
88
100
  });
89
101
 
90
102
  const response = await client.responses.create({
91
- model: "gpt-5",
103
+ model: "openai/gpt-5",
92
104
  input: "Write a one-sentence bedtime story about a unicorn.",
93
105
  });
94
106
 
@@ -1,33 +1,49 @@
1
1
  ---
2
2
  title: Zuplo AI Gateway
3
3
  sidebar_label: Introduction
4
+ description:
5
+ The Zuplo AI Gateway is a proxy between your apps and LLM providers, deployed
6
+ from your own Git repository, with per-app policy pipelines for cost control,
7
+ model access, guardrails, and observability.
4
8
  ---
5
9
 
6
10
  Zuplo's AI Gateway acts as an intelligent proxy layer that sits between your
7
- engineering team's applications and LLM providers like OpenAI, Anthropic,
8
- Google, Mistral, and xAI. Instead of your applications communicating directly
9
- with these providers, all requests flow through the Zuplo AI Gateway, which
10
- streams responses while applying policies, controls, and monitoring.
11
+ engineering team's apps and LLM providers like OpenAI, Anthropic, Google,
12
+ Mistral, and xAI. Instead of your apps communicating directly with these
13
+ providers, all requests flow through the Zuplo AI Gateway, which streams
14
+ responses while applying policies, controls, and monitoring.
15
+
16
+ Each piece of software that calls the gateway registers as an **app**—your
17
+ support chatbot is one app; an internal coding agent is another. A single AI
18
+ Gateway holds as many apps as you need, and you control exactly which policies
19
+ run for each one. Every app has its own ordered policy chain, so the
20
+ customer-facing chatbot can enforce budgets and guardrails while the internal
21
+ agent runs with none. Beyond the built-in policies, you can write your own in
22
+ TypeScript, and apps add them to their chains like any other policy. Chain
23
+ changes apply within about a minute, with no redeploy. See [Apps](./apps.mdx).
11
24
 
12
25
  ## Key Benefits
13
26
 
14
27
  **Provider Independence**: Switch between LLM providers (OpenAI, Anthropic,
15
- Google, Mistral, xAI, and more) dynamically without modifying application code.
16
- Configure your provider choice through the gateway rather than hard coding it
17
- into your applications.
28
+ Google, Mistral, xAI, and more) dynamically without modifying app code.
29
+ Configure model access through the gateway rather than hard coding it into your
30
+ apps.
18
31
 
19
- **Cost Control**: Set spending limits at organization, team, and application
20
- levels with hierarchical budgets that cascade down through your structure.
21
- Configure daily and monthly thresholds with enforcement or warning
22
- notifications.
32
+ **Cost Control**: Set spending limits at organization, team, and app levels with
33
+ hierarchical budgets that cascade down through your structure. Configure daily
34
+ and monthly thresholds with enforcement or warning notifications.
23
35
 
24
36
  **Security & Compliance**: Apply guardrails to detect and block prompt injection
25
37
  attempts and prevent PII leakage in both requests and responses through
26
38
  integrated AI firewall policies.
27
39
 
28
- **Self-Service Access**: Developers can create applications and access LLMs
29
- without needing direct access to provider API keys. Administrators configure
30
- providers once, and teams consume them securely.
40
+ **Self-Service Access**: Developers can create apps and access LLMs without
41
+ needing direct access to provider API keys. Administrators configure providers
42
+ once, and teams consume them securely.
43
+
44
+ **Extensibility**: Write custom policies in TypeScript—content filters, dynamic
45
+ model routing, plan-based limits—and let apps add them to their chains like any
46
+ built-in policy.
31
47
 
32
48
  **Performance Optimization**: Enable semantic caching to identify and return
33
49
  cached responses for similar prompts, reducing costs and improving response
@@ -38,12 +54,20 @@ time-to-first-byte metrics, and spending patterns across your organization.
38
54
 
39
55
  ## How It Works
40
56
 
41
- Your applications send requests to the Zuplo AI Gateway URL using your Zuplo API
42
- key. The gateway authenticates the request, applies configured policies (cost
43
- controls, security guardrails), routes to the selected LLM provider, and streams
44
- the response back to your application. Throughout this process, the gateway
45
- captures metrics and enforces limits without exposing underlying provider
46
- credentials.
57
+ Your code sends its LLM requests to your AI Gateway instead of directly to a
58
+ provider. Each app has its own URL, API key, and policy chain, so one gateway
59
+ serves teams with completely different configurations.
60
+
61
+ Each request arrives at the URL of the app it belongs to. The gateway identifies
62
+ that app, runs its policy chain (model access, cost controls, security
63
+ guardrails), routes to the selected LLM provider, and streams the response back.
64
+ Throughout this process, the gateway captures metrics without exposing
65
+ underlying provider credentials.
66
+
67
+ Every control is opt-in. An app whose policy chain is empty proxies requests
68
+ immediately—each request names its own model, and no model restrictions,
69
+ budgets, guardrails, or caching apply until you add the policies that enforce
70
+ them.
47
71
 
48
72
  ## Core Features
49
73
 
@@ -52,32 +76,49 @@ credentials.
52
76
  Configure multiple LLM providers within a single gateway project. Supported
53
77
  providers include OpenAI, Anthropic, Google, Mistral, xAI, and OpenAI-compatible
54
78
  custom providers. See [AI Providers](./providers.mdx) for the full list of
55
- providers and supported capabilities. Select which models are available to your
56
- teams when configuring each provider.
79
+ providers and supported capabilities. Apps reference models as
80
+ `providerName/model`—for example `openai/gpt-5-mini`—so a single app can use
81
+ models from several providers.
82
+
83
+ ### Source-Controlled Gateway
84
+
85
+ Every AI Gateway project connects to a Git repository that contains the
86
+ gateway's routes and policy declarations. The default branch is what production
87
+ runs, so gateway changes go through your normal review workflow. See
88
+ [Source Control](./source-control.mdx).
89
+
90
+ ### Per-App Policy Chains
91
+
92
+ Each app runs its own ordered chain of policies—model filtering, fallback
93
+ models, budgets, semantic caching, guardrails, tracing, and any custom policies
94
+ declared in the repository. Team policy templates give new apps a consistent
95
+ starting pipeline. See [Policy Chains](./policy-chains.mdx) and
96
+ [Custom Policies](./custom-policies.mdx).
57
97
 
58
98
  ### Team Hierarchy & Budgets
59
99
 
60
100
  Organize users into teams with hierarchical structures. Set budget limits at
61
101
  each level that cascade down:
62
102
 
63
- - **Root Team**: Organization-wide limits (for example, $1,000/day)
64
- - **Sub-Teams**: Team-specific limits that can't exceed parent limits (for
65
- example, $500/day for the Credit Team)
66
- - **Applications**: Per-app limits for granular control
103
+ - **Gateway**: Organization-wide limits (for example, $1,000/day)
104
+ - **Teams**: Team-specific limits that can't exceed parent limits (for example,
105
+ $500/day for the Credit Team)
106
+ - **Apps**: Per-app limits for granular control
107
+
108
+ See [Usage Limits](./usage-limits.mdx).
67
109
 
68
- ### Application Configuration
110
+ ### App Configuration
69
111
 
70
- Each application gets its own:
112
+ Each app gets its own:
71
113
 
72
114
  - **Unique Gateway URL**: Single endpoint regardless of underlying provider
73
115
  - **API Key**: Zuplo-managed key that never exposes provider credentials
74
- - **Model Selection**: Choose specific models from configured providers
75
- - **Budget Thresholds**: Daily and monthly limits with enforcement or warnings
76
- - **Semantic Caching**: Optional caching of similar prompts to reduce costs
116
+ - **Policy Chain**: Model access, budgets, caching, guardrails, and custom
117
+ policies, applied in the order the app chooses
77
118
 
78
119
  ## Use Cases
79
120
 
80
- - **Multi-tenant AI Applications**: Enforce spending limits per customer or team
121
+ - **Multi-tenant AI Apps**: Enforce spending limits per customer or team
81
122
  - **Agent Development**: Build AI agents that can switch providers without code
82
123
  changes
83
124
  - **Cost Management**: Control and monitor LLM spending across your organization