zuplo 7.9.14 → 7.9.15
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/docs/ai-gateway/bedrock-runtime.mdx +1 -1
- package/docs/ai-gateway/custom-policies.mdx +1 -1
- package/docs/ai-gateway/fallback.mdx +1 -1
- package/docs/ai-gateway/integrations/ai-sdk.mdx +5 -5
- package/docs/ai-gateway/integrations/claude-code.mdx +6 -8
- package/docs/ai-gateway/integrations/codex.mdx +6 -5
- package/docs/ai-gateway/integrations/goose.mdx +3 -3
- package/docs/ai-gateway/integrations/langchain.mdx +1 -1
- package/docs/ai-gateway/integrations/openai.mdx +3 -3
- package/docs/ai-gateway/managing-providers.mdx +1 -1
- package/docs/ai-gateway/openrouter.mdx +14 -13
- package/docs/ai-gateway/overview.mdx +4 -2
- package/docs/ai-gateway/providers.mdx +2 -2
- package/docs/ai-gateway/source-control.mdx +61 -2
- package/docs/ai-gateway/universal-api.mdx +1 -1
- package/docs/ai-gateway/user-apps.mdx +5 -2
- package/docs/articles/accounts/audit-logs.mdx +3 -9
- package/docs/articles/environment-variables.mdx +56 -231
- package/docs/articles/graphql-support.mdx +59 -18
- package/docs/articles/local-development-env-variables.mdx +37 -8
- package/docs/articles/logging.mdx +21 -0
- package/docs/articles/opentelemetry.mdx +1 -1
- package/docs/cli/source-upgrade.mdx +39 -0
- package/docs/cli/source-upgrade.partial.mdx +11 -0
- package/docs/concepts/ai-gateway.mdx +1 -1
- package/docs/dev-portal/zudoku/guides/environment-variables.md +75 -0
- package/docs/dev-portal/zudoku-partials/guides/environment-variables.partial.md +74 -0
- package/docs/handlers/mcp-server.mdx +72 -0
- package/docs/mcp-server/overview.mdx +7 -0
- package/docs/mcp-server/tools.mdx +72 -0
- package/docs/policies/_index.md +1 -1
- package/docs/policies/ai-gateway-auth-inbound/doc.md +2 -2
- package/docs/policies/ai-gateway-configuration-executor-inbound/doc.md +3 -3
- package/docs/policies/ai-gateway-prompt-injection/doc.md +1 -2
- package/docs/policies/ai-gateway-prompt-injection/schema.json +2 -2
- package/docs/policies/ai-gateway-smart-router-inbound/doc.md +112 -29
- package/docs/policies/ai-gateway-smart-router-inbound/schema.json +109 -12
- package/docs/policies/internal-only-inbound/doc.md +206 -0
- package/docs/policies/internal-only-inbound/intro.md +10 -0
- package/docs/policies/internal-only-inbound/schema.json +75 -0
- package/docs/programmable-api/environment.mdx +220 -48
- package/docs/programmable-api/hooks.mdx +24 -0
- package/docs/programmable-api/mcp-sdk.mdx +22 -0
- package/package.json +5 -5
- package/docs/policies/ai-gateway-internal-only-inbound/doc.md +0 -102
- package/docs/policies/ai-gateway-internal-only-inbound/intro.md +0 -11
- package/docs/policies/ai-gateway-internal-only-inbound/schema.json +0 -50
|
@@ -97,3 +97,78 @@ in `vite-env.d.ts`. A helpful explanation can be found on
|
|
|
97
97
|
[this StackOverflow reply](https://stackoverflow.com/a/51114250).
|
|
98
98
|
|
|
99
99
|
:::
|
|
100
|
+
|
|
101
|
+
## Environment Variables in Zuplo
|
|
102
|
+
|
|
103
|
+
Set Dev Portal variables in the Zuplo Portal under **Settings** >
|
|
104
|
+
[**Environment Variables**](https://portal.zuplo.com/+/account/project/settings/environment-variables),
|
|
105
|
+
the same place you set gateway variables. For the Dev Portal:
|
|
106
|
+
|
|
107
|
+
- Only variables whose names start with `ZUPLO_PUBLIC_` or `ZUDOKU_PUBLIC_` are
|
|
108
|
+
available. Both prefixes work the same way.
|
|
109
|
+
- Only configuration variables are available. Zuplo never sends secret variables
|
|
110
|
+
to the Dev Portal build, even if their names have a public prefix.
|
|
111
|
+
- Values are read when the portal is built and are embedded in the published
|
|
112
|
+
site. Redeploy the environment after you change a value.
|
|
113
|
+
|
|
114
|
+
For how Dev Portal variables differ from gateway variables, see
|
|
115
|
+
[Configuring Environment Variables](/docs/articles/environment-variables#gateway-and-developer-portal-variables).
|
|
116
|
+
|
|
117
|
+
### Zuplo Build Variables
|
|
118
|
+
|
|
119
|
+
When Zuplo builds your Dev Portal, it sets the following variables. You don't
|
|
120
|
+
need to create them.
|
|
121
|
+
|
|
122
|
+
| Variable | Value |
|
|
123
|
+
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
124
|
+
| `ZUPLO_ENVIRONMENT_TYPE` | The environment being built: `PRODUCTION`, `PREVIEW`, or `WORKING_COPY`. |
|
|
125
|
+
| `ZUPLO_SERVER_URL` | The default URL of the API gateway deployed with this Dev Portal, for example `https://my-project-main-1234abc.zuplo.app`. Custom domains don't change this value. |
|
|
126
|
+
| `ZUPLO_BUILD_ID` | A unique ID (UUID) for the build. |
|
|
127
|
+
|
|
128
|
+
Read these variables with `import.meta.env` in your custom components and
|
|
129
|
+
plugins:
|
|
130
|
+
|
|
131
|
+
```tsx
|
|
132
|
+
export const EnvironmentBadge = () => (
|
|
133
|
+
<span>{import.meta.env.ZUPLO_ENVIRONMENT_TYPE}</span>
|
|
134
|
+
);
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
`ZUPLO_ENVIRONMENT_TYPE` in the Dev Portal build uses different values than the
|
|
138
|
+
`ZUPLO_ENVIRONMENT_TYPE` and `ZUPLO_ENVIRONMENT_STAGE` variables available to
|
|
139
|
+
your gateway. See
|
|
140
|
+
[System Environment Variables](/docs/programmable-api/environment#system-environment-variables)
|
|
141
|
+
for the gateway values.
|
|
142
|
+
|
|
143
|
+
### Check the Environment in `zudoku.config.tsx`
|
|
144
|
+
|
|
145
|
+
The configuration file runs twice: once in Node.js when the build starts, and
|
|
146
|
+
again as part of the built site, which renders your pages and `<head>` tags. In
|
|
147
|
+
the built site, `process.env` only resolves variables that start with
|
|
148
|
+
`ZUPLO_PUBLIC_` or `ZUDOKU_PUBLIC_`, so `process.env.ZUPLO_ENVIRONMENT_TYPE` is
|
|
149
|
+
`undefined` there. Read `import.meta.env` first and fall back to `process.env`
|
|
150
|
+
to get the same value in both places:
|
|
151
|
+
|
|
152
|
+
```tsx title="zudoku.config.tsx"
|
|
153
|
+
const environmentType =
|
|
154
|
+
import.meta.env?.ZUPLO_ENVIRONMENT_TYPE ?? process.env.ZUPLO_ENVIRONMENT_TYPE;
|
|
155
|
+
|
|
156
|
+
const isProduction = environmentType === "PRODUCTION";
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
The build logs a warning that `process.env.ZUPLO_ENVIRONMENT_TYPE` isn't
|
|
160
|
+
prefixed with `ZUPLO_PUBLIC_` or `ZUDOKU_PUBLIC_`. You can ignore it.
|
|
161
|
+
|
|
162
|
+
To avoid the warning, create a `ZUPLO_PUBLIC_` variable with a different value
|
|
163
|
+
for each environment instead. Both approaches include the value in the
|
|
164
|
+
client-side bundle, so use them only for values that aren't secret.
|
|
165
|
+
|
|
166
|
+
### Local Development
|
|
167
|
+
|
|
168
|
+
Locally, put `ZUPLO_PUBLIC_` and `ZUDOKU_PUBLIC_` variables in a `.env` file in
|
|
169
|
+
your Dev Portal folder, as described in [Local Env Files](#local-env-files).
|
|
170
|
+
Variables you set in the Zuplo Portal aren't available locally.
|
|
171
|
+
|
|
172
|
+
`zuplo dev` sets `ZUPLO_SERVER_URL` to your local gateway, for example
|
|
173
|
+
`http://localhost:9000`. `ZUPLO_ENVIRONMENT_TYPE` is never `PRODUCTION` locally,
|
|
174
|
+
so `isProduction` in the previous example is `false`.
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
## Environment Variables in Zuplo
|
|
2
|
+
|
|
3
|
+
Set Dev Portal variables in the Zuplo Portal under **Settings** >
|
|
4
|
+
[**Environment Variables**](https://portal.zuplo.com/+/account/project/settings/environment-variables),
|
|
5
|
+
the same place you set gateway variables. For the Dev Portal:
|
|
6
|
+
|
|
7
|
+
- Only variables whose names start with `ZUPLO_PUBLIC_` or `ZUDOKU_PUBLIC_` are
|
|
8
|
+
available. Both prefixes work the same way.
|
|
9
|
+
- Only configuration variables are available. Zuplo never sends secret variables
|
|
10
|
+
to the Dev Portal build, even if their names have a public prefix.
|
|
11
|
+
- Values are read when the portal is built and are embedded in the published
|
|
12
|
+
site. Redeploy the environment after you change a value.
|
|
13
|
+
|
|
14
|
+
For how Dev Portal variables differ from gateway variables, see
|
|
15
|
+
[Configuring Environment Variables](/docs/articles/environment-variables#gateway-and-developer-portal-variables).
|
|
16
|
+
|
|
17
|
+
### Zuplo Build Variables
|
|
18
|
+
|
|
19
|
+
When Zuplo builds your Dev Portal, it sets the following variables. You don't
|
|
20
|
+
need to create them.
|
|
21
|
+
|
|
22
|
+
| Variable | Value |
|
|
23
|
+
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
24
|
+
| `ZUPLO_ENVIRONMENT_TYPE` | The environment being built: `PRODUCTION`, `PREVIEW`, or `WORKING_COPY`. |
|
|
25
|
+
| `ZUPLO_SERVER_URL` | The default URL of the API gateway deployed with this Dev Portal, for example `https://my-project-main-1234abc.zuplo.app`. Custom domains don't change this value. |
|
|
26
|
+
| `ZUPLO_BUILD_ID` | A unique ID (UUID) for the build. |
|
|
27
|
+
|
|
28
|
+
Read these variables with `import.meta.env` in your custom components and
|
|
29
|
+
plugins:
|
|
30
|
+
|
|
31
|
+
```tsx
|
|
32
|
+
export const EnvironmentBadge = () => (
|
|
33
|
+
<span>{import.meta.env.ZUPLO_ENVIRONMENT_TYPE}</span>
|
|
34
|
+
);
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
`ZUPLO_ENVIRONMENT_TYPE` in the Dev Portal build uses different values than the
|
|
38
|
+
`ZUPLO_ENVIRONMENT_TYPE` and `ZUPLO_ENVIRONMENT_STAGE` variables available to
|
|
39
|
+
your gateway. See
|
|
40
|
+
[System Environment Variables](/docs/programmable-api/environment#system-environment-variables)
|
|
41
|
+
for the gateway values.
|
|
42
|
+
|
|
43
|
+
### Check the Environment in `zudoku.config.tsx`
|
|
44
|
+
|
|
45
|
+
The configuration file runs twice: once in Node.js when the build starts, and
|
|
46
|
+
again as part of the built site, which renders your pages and `<head>` tags. In
|
|
47
|
+
the built site, `process.env` only resolves variables that start with
|
|
48
|
+
`ZUPLO_PUBLIC_` or `ZUDOKU_PUBLIC_`, so `process.env.ZUPLO_ENVIRONMENT_TYPE` is
|
|
49
|
+
`undefined` there. Read `import.meta.env` first and fall back to `process.env`
|
|
50
|
+
to get the same value in both places:
|
|
51
|
+
|
|
52
|
+
```tsx title="zudoku.config.tsx"
|
|
53
|
+
const environmentType =
|
|
54
|
+
import.meta.env?.ZUPLO_ENVIRONMENT_TYPE ?? process.env.ZUPLO_ENVIRONMENT_TYPE;
|
|
55
|
+
|
|
56
|
+
const isProduction = environmentType === "PRODUCTION";
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
The build logs a warning that `process.env.ZUPLO_ENVIRONMENT_TYPE` isn't
|
|
60
|
+
prefixed with `ZUPLO_PUBLIC_` or `ZUDOKU_PUBLIC_`. You can ignore it.
|
|
61
|
+
|
|
62
|
+
To avoid the warning, create a `ZUPLO_PUBLIC_` variable with a different value
|
|
63
|
+
for each environment instead. Both approaches include the value in the
|
|
64
|
+
client-side bundle, so use them only for values that aren't secret.
|
|
65
|
+
|
|
66
|
+
### Local Development
|
|
67
|
+
|
|
68
|
+
Locally, put `ZUPLO_PUBLIC_` and `ZUDOKU_PUBLIC_` variables in a `.env` file in
|
|
69
|
+
your Dev Portal folder, as described in [Local Env Files](#local-env-files).
|
|
70
|
+
Variables you set in the Zuplo Portal aren't available locally.
|
|
71
|
+
|
|
72
|
+
`zuplo dev` sets `ZUPLO_SERVER_URL` to your local gateway, for example
|
|
73
|
+
`http://localhost:9000`. `ZUPLO_ENVIRONMENT_TYPE` is never `PRODUCTION` locally,
|
|
74
|
+
so `isProduction` in the previous example is `false`.
|
|
@@ -512,6 +512,78 @@ example, in Cursor:
|
|
|
512
512
|
}
|
|
513
513
|
```
|
|
514
514
|
|
|
515
|
+
## Keep REST routes private
|
|
516
|
+
|
|
517
|
+
Every operation an MCP server exposes is still a route on your gateway, and
|
|
518
|
+
clients can reach that route directly over HTTP. To make the routes reachable
|
|
519
|
+
only through the MCP server, add the
|
|
520
|
+
[Internal Only policy](../policies/internal-only-inbound.mdx) to every route the
|
|
521
|
+
server exposes.
|
|
522
|
+
|
|
523
|
+
The MCP Server handler calls each route with `context.invokeRoute`, which the
|
|
524
|
+
Internal Only policy accepts. The policy rejects every request that arrives over
|
|
525
|
+
the network. Set `status` to `404` so a direct call gets the same response as a
|
|
526
|
+
route that doesn't exist:
|
|
527
|
+
|
|
528
|
+
```json title="config/policies.json"
|
|
529
|
+
{
|
|
530
|
+
"name": "internal-only-inbound",
|
|
531
|
+
"policyType": "internal-only-inbound",
|
|
532
|
+
"handler": {
|
|
533
|
+
"export": "InternalOnlyInboundPolicy",
|
|
534
|
+
"module": "$import(@zuplo/runtime)",
|
|
535
|
+
"options": {
|
|
536
|
+
"status": 404
|
|
537
|
+
}
|
|
538
|
+
}
|
|
539
|
+
}
|
|
540
|
+
```
|
|
541
|
+
|
|
542
|
+
Put the policy first in the inbound policies of each route, so nothing else runs
|
|
543
|
+
for a rejected request:
|
|
544
|
+
|
|
545
|
+
```json title="config/routes.oas.json"
|
|
546
|
+
"/orders/{orderId}": {
|
|
547
|
+
"get": {
|
|
548
|
+
"operationId": "get-order",
|
|
549
|
+
"x-internal": true,
|
|
550
|
+
"parameters": [
|
|
551
|
+
{
|
|
552
|
+
"name": "orderId",
|
|
553
|
+
"in": "path",
|
|
554
|
+
"required": true,
|
|
555
|
+
"schema": { "type": "string" }
|
|
556
|
+
}
|
|
557
|
+
],
|
|
558
|
+
"x-zuplo-route": {
|
|
559
|
+
"corsPolicy": "none",
|
|
560
|
+
"handler": {
|
|
561
|
+
"export": "urlForwardHandler",
|
|
562
|
+
"module": "$import(@zuplo/runtime)",
|
|
563
|
+
"options": { "baseUrl": "https://orders.example.com" }
|
|
564
|
+
},
|
|
565
|
+
"policies": {
|
|
566
|
+
"inbound": ["internal-only-inbound"]
|
|
567
|
+
}
|
|
568
|
+
}
|
|
569
|
+
}
|
|
570
|
+
}
|
|
571
|
+
```
|
|
572
|
+
|
|
573
|
+
When you hide a route:
|
|
574
|
+
|
|
575
|
+
- **Authenticate callers on the MCP server route.** The Internal Only policy
|
|
576
|
+
checks where a request comes from, not who sent it. The MCP server forwards
|
|
577
|
+
the client's request headers to each route, so a route can keep its own
|
|
578
|
+
authentication policy after this one.
|
|
579
|
+
- **Set `corsPolicy` to `none`.** Preflight `OPTIONS` requests skip inbound
|
|
580
|
+
policies, so a route with a CORS policy still answers them.
|
|
581
|
+
- **Mark the route [`x-internal`](../articles/openapi.mdx#x-internal)** to keep
|
|
582
|
+
it out of your developer portal.
|
|
583
|
+
- **A custom [not-found handler](../programmable-api/not-found-handler.mdx)**
|
|
584
|
+
answers paths that don't exist with your own response, so a hidden route's
|
|
585
|
+
`404` no longer matches it.
|
|
586
|
+
|
|
515
587
|
## Testing
|
|
516
588
|
|
|
517
589
|
### MCP Inspector
|
|
@@ -69,6 +69,13 @@ Expose only the operations your clients need. Apply authentication and access
|
|
|
69
69
|
controls to those operations, use rate limits to control usage, and configure
|
|
70
70
|
[audit logging](../articles/audit-logging.mdx) to record calls.
|
|
71
71
|
|
|
72
|
+
Each operation the server exposes is also a REST route that clients can reach
|
|
73
|
+
directly over HTTP. To serve those operations only through the MCP server, add
|
|
74
|
+
the [Internal Only policy](../policies/internal-only-inbound.mdx) to each route,
|
|
75
|
+
with `"status": 404` so a direct call gets the same response as a route that
|
|
76
|
+
doesn't exist. See
|
|
77
|
+
[Keep REST routes private](../handlers/mcp-server.mdx#keep-rest-routes-private).
|
|
78
|
+
|
|
72
79
|
<span id="learn-more" />
|
|
73
80
|
|
|
74
81
|
## Next steps
|
|
@@ -364,6 +364,70 @@ MCP tool configuration error: invalid toolInputStyle "Flat" on operation "create
|
|
|
364
364
|
If your MCP server still uses the deprecated `files` option, set
|
|
365
365
|
`toolInputStyle` in the operation's `x-zuplo-mcp-tool` extension instead.
|
|
366
366
|
|
|
367
|
+
## Track tool calls with runtime hooks
|
|
368
|
+
|
|
369
|
+
When the MCP Server handler invokes an API route for a tool, the global runtime
|
|
370
|
+
hooks run for both the client request and the internal route invocation. Each
|
|
371
|
+
invocation has its own request and context:
|
|
372
|
+
|
|
373
|
+
| Value | Client request | Internal tool invocation |
|
|
374
|
+
| ----------------------- | ---------------------------------------- | ------------------------------------------------------------------------------- |
|
|
375
|
+
| Method and path | `POST /mcp`, or your configured MCP path | The API route's method and path, such as `GET /weather/current?location=Boston` |
|
|
376
|
+
| Body | The MCP JSON-RPC envelope | The tool arguments mapped to the API request body, if the route uses one |
|
|
377
|
+
| `context.route` | The MCP route | The tool's API route |
|
|
378
|
+
| `context.parentContext` | `undefined` | The context of the MCP request that invoked the route |
|
|
379
|
+
|
|
380
|
+
Path and query arguments become URL parameters, and declared header arguments
|
|
381
|
+
become request headers. A `GET` tool invocation has no request body.
|
|
382
|
+
|
|
383
|
+
Use
|
|
384
|
+
[`context.parentContext`](../programmable-api/zuplo-context.mdx#parentcontext)
|
|
385
|
+
to distinguish client requests from internal invocations. It identifies any
|
|
386
|
+
internal route invocation, including calls made with `context.invokeRoute` in
|
|
387
|
+
your own code.
|
|
388
|
+
|
|
389
|
+
To log only the API route invocations for MCP tools, add this global request
|
|
390
|
+
hook in `modules/zuplo.runtime.ts`:
|
|
391
|
+
|
|
392
|
+
```ts
|
|
393
|
+
import { RuntimeExtensions, ZuploMcpSdk } from "@zuplo/runtime";
|
|
394
|
+
|
|
395
|
+
export function runtimeInit(runtime: RuntimeExtensions) {
|
|
396
|
+
runtime.addRequestHook((request, context) => {
|
|
397
|
+
const toolCall = new ZuploMcpSdk(context).getRawCallToolRequest();
|
|
398
|
+
|
|
399
|
+
if (context.parentContext && toolCall) {
|
|
400
|
+
context.log.info({
|
|
401
|
+
toolName: toolCall.params.name,
|
|
402
|
+
method: request.method,
|
|
403
|
+
path: new URL(request.url).pathname,
|
|
404
|
+
requestId: context.requestId,
|
|
405
|
+
contextId: context.contextId,
|
|
406
|
+
});
|
|
407
|
+
}
|
|
408
|
+
|
|
409
|
+
return request;
|
|
410
|
+
});
|
|
411
|
+
}
|
|
412
|
+
```
|
|
413
|
+
|
|
414
|
+
This logs a tool's API route invocation without also counting the outer `/mcp`
|
|
415
|
+
request. To count only client requests instead, filter for
|
|
416
|
+
`context.parentContext === undefined`. The SDK helper reads the current context
|
|
417
|
+
and its immediate parent; see
|
|
418
|
+
[MCP metadata in runtime hooks](../programmable-api/mcp-sdk.mdx#mcp-metadata-in-runtime-hooks)
|
|
419
|
+
for timing and nested-call details.
|
|
420
|
+
|
|
421
|
+
By default, the internal invocation shares the parent's `requestId` and has its
|
|
422
|
+
own `contextId`. Use `requestId` to group related events and `contextId` to
|
|
423
|
+
identify each execution. See
|
|
424
|
+
[Request/Response Hooks](../programmable-api/hooks.mdx#internal-route-invocations)
|
|
425
|
+
for hook behavior and response logging.
|
|
426
|
+
|
|
427
|
+
This extra hook execution applies when a tool invokes a Zuplo API route.
|
|
428
|
+
[Proxying to an upstream MCP server](../mcp-gateway/code-config/mcp-proxy-handler.mdx)
|
|
429
|
+
doesn't create an internal API route invocation for each tool call.
|
|
430
|
+
|
|
367
431
|
## Testing MCP Tools
|
|
368
432
|
|
|
369
433
|
### List Available Tools
|
|
@@ -525,6 +589,14 @@ provided by the AI client _before_ your handler is ever called. Input validation
|
|
|
525
589
|
is an important part of MCP, so ensure you have strong validation in your
|
|
526
590
|
OpenAPI JSON schemas!
|
|
527
591
|
|
|
592
|
+
### Keep Tool Routes Private
|
|
593
|
+
|
|
594
|
+
A tool's route is also a REST endpoint that clients can reach directly. If AI
|
|
595
|
+
clients should reach the operation only through the MCP server, add the
|
|
596
|
+
[Internal Only policy](../policies/internal-only-inbound.mdx) to the route with
|
|
597
|
+
`"status": 404`. See
|
|
598
|
+
[Keep REST routes private](../handlers/mcp-server.mdx#keep-rest-routes-private).
|
|
599
|
+
|
|
528
600
|
### Custom Tools
|
|
529
601
|
|
|
530
602
|
For complex workflows that don't map 1:1 to a single API route, or require
|
package/docs/policies/_index.md
CHANGED
|
@@ -41,7 +41,6 @@
|
|
|
41
41
|
| data-loss-prevention-outbound | Data Loss Prevention | Scans the upstream response body for sensitive data — PII, secrets, and financial identifiers — using an extensible catalog of built-in recognizers plus any custom patterns, and takes a configurable action when a match is found. The action is one of `mask` (redact matches before returning the response), `block` (replace the response with a `422` listing the detected entity names only), or `log` (record a warning and return unchanged). Only text content types are inspected; binary bodies pass through untouched, and the body is read from a clone so the client still receives the original stream. | api-gateway |
|
|
42
42
|
| data-loss-prevention-inbound | Data Loss Prevention | Scans the incoming request body for sensitive data — PII, secrets, and financial identifiers — using an extensible catalog of built-in recognizers plus any custom patterns, and takes a configurable action when a match is found. The action is one of `mask` (redact matches before forwarding the request), `block` (reject with a `422` listing the detected entity names only), or `log` (record a warning and forward unchanged). Only text content types are inspected; binary bodies pass through untouched, and the body is read from a clone so the upstream still receives the original stream. | api-gateway |
|
|
43
43
|
| ai-gateway-dlp-inbound | Data Loss Prevention (DLP) | Data Loss Prevention (DLP) Inbound Policy for the AI Gateway | ai-gateway |
|
|
44
|
-
| ai-gateway-internal-only-inbound | Ensure Gateway Internal Invocation Only | Accepts only requests that this gateway made itself with `context.invokeRoute`, and rejects every request that arrived over the network with `403 Forbidden`. Use it in place of AI Gateway Authentication on an application that is only ever called by other routes and policies on the same gateway, such as the classifier application behind the Smart Router or Prompt Injection Protection policies. Those calls never leave the gateway, so an API key on them protects nothing. This policy asserts the one thing that matters: the request originated inside the deployment. The check reads `context.parentContext`, which the runtime sets only on the child context that `context.invokeRoute` creates. A client cannot supply it through a header, the URL, or the body. The policy has no options, makes no network calls, reads no body, and leaves `request.user` unset. | ai-gateway |
|
|
45
44
|
| firebase-jwt-inbound | Firebase JWT Auth | Authenticate users using Firebase issued JWT tokens. | api-gateway |
|
|
46
45
|
| formdata-to-json-inbound | Form Data to JSON | Converts form data in the incoming request to JSON. | api-gateway |
|
|
47
46
|
| ai-gateway-galileo-tracing-inbound | Galileo Tracing | Galileo Tracing Inbound Policy | ai-gateway |
|
|
@@ -52,6 +51,7 @@
|
|
|
52
51
|
| graphql-disable-introspection-inbound | GraphQL Disable Introspection | Policy that disables GraphQL introspection queries in production. Introspection allows clients to discover the schema, which can be a security risk as it exposes your entire API structure. | api-gateway |
|
|
53
52
|
| graphql-introspection-filter-outbound | GraphQL Introspection Filter | Filters GraphQL introspection responses to exclude specific types and fields. This policy intercepts GraphQL introspection query responses and removes configured types and fields from the schema. Useful for hiding internal types or sensitive fields from the public schema. | api-gateway |
|
|
54
53
|
| http-deprecation-outbound | HTTP Deprecation | Sets HTTP deprecation headers on the outgoing response following the IETF HTTP Deprecation Header standard. Supports the Deprecation, Sunset, and Link headers. | api-gateway |
|
|
54
|
+
| internal-only-inbound | Internal Only | Accepts only requests that this gateway made itself with `context.invokeRoute`, and rejects every request that arrived over the network. Use it on a route that only other routes, handlers, and policies on the same gateway call: a REST endpoint that an MCP server exposes as a tool, or the classifier application behind the Smart Router or Prompt Injection Protection policies. Those calls never leave the gateway, so an API key on them protects nothing. This policy asserts the one thing that matters: the request originated inside the deployment. The check reads `context.parentContext`, which the runtime sets only on the child context that `context.invokeRoute` creates. A client cannot supply it through a header, the URL, or the body. The policy makes no network calls, reads no body, and leaves `request.user` unset. A rejected request gets a `403 Forbidden` problem response with no `detail` by default, so the caller learns nothing about how the gateway works; the reason goes to the request log. Set `status` to answer with another client error, such as `404` to hide the route or `401`, `statusText` to change the status text, and `detail` to add a message. | api-gateway, ai-gateway |
|
|
55
55
|
| ip-address-restriction-inbound | IP Address Restriction | Restricts which client IP addresses can call a route using an allow list, a deny list, or both. Entries are individual IP addresses or CIDR ranges, in IPv4 or IPv6. | api-gateway |
|
|
56
56
|
| validate-json-schema-inbound | JSON Body Validation (deprecated) | Validates the body of an incoming request based on a JSON schema. | api-gateway |
|
|
57
57
|
| open-id-jwt-auth-inbound | JWT Auth | The Open ID JWT Authentication policy allows you to authenticate incoming requests using an Open ID compliant bearer token. | api-gateway |
|
|
@@ -93,8 +93,8 @@ configuration from the route's `app_id` path parameter instead.
|
|
|
93
93
|
> the application from the URL and accepts the request without an application
|
|
94
94
|
> key. Only leave authentication out when another control restricts access:
|
|
95
95
|
> network isolation, or the
|
|
96
|
-
> [
|
|
97
|
-
>
|
|
96
|
+
> [Internal Only](/docs/policies/internal-only-inbound) policy for an
|
|
97
|
+
> application that is only called from inside the gateway with
|
|
98
98
|
> `context.invokeRoute`. Team policy templates can include a locked auth entry
|
|
99
99
|
> so new applications require keys by default.
|
|
100
100
|
|
|
@@ -205,9 +205,9 @@ Authentication is optional and placement controls its scope:
|
|
|
205
205
|
key for every application on the route.
|
|
206
206
|
- **Internal applications** — for an application that is only called from inside
|
|
207
207
|
the gateway with `context.invokeRoute`, add
|
|
208
|
-
[
|
|
209
|
-
|
|
210
|
-
|
|
208
|
+
[Internal Only](/docs/policies/internal-only-inbound) to its
|
|
209
|
+
`inboundPolicyChain` instead of authentication. It rejects every request that
|
|
210
|
+
arrives over the network and needs no API key.
|
|
211
211
|
|
|
212
212
|
Use the same placement on each AI Gateway operation that should support
|
|
213
213
|
application-selected chains.
|
|
@@ -50,8 +50,7 @@ classification inherits the app's provider credentials, model routing,
|
|
|
50
50
|
fallbacks, and quotas, and provider keys stay on one app. In this mode:
|
|
51
51
|
|
|
52
52
|
- `apiKey` is optional. The invocation never leaves the gateway, so give the
|
|
53
|
-
classifier app the
|
|
54
|
-
[Ensure Gateway Internal Invocation Only](/docs/policies/ai-gateway-internal-only-inbound)
|
|
53
|
+
classifier app the [Internal Only](/docs/policies/internal-only-inbound)
|
|
55
54
|
policy — it accepts requests this gateway made itself and rejects everything
|
|
56
55
|
that arrives over the network — and omit `apiKey` entirely. Set it only when
|
|
57
56
|
the classifier app authenticates callers with API keys; it is then sent as
|
|
@@ -39,13 +39,13 @@
|
|
|
39
39
|
"apiKey": {
|
|
40
40
|
"type": "string",
|
|
41
41
|
"title": "API Key",
|
|
42
|
-
"description": "Credential used to run the classifier. Without `appId` this is the API key for the OpenAI-compatible service at `baseUrl`, and it is required. With `appId` it is optional: omit it when the classifier app runs the
|
|
42
|
+
"description": "Credential used to run the classifier. Without `appId` this is the API key for the OpenAI-compatible service at `baseUrl`, and it is required. With `appId` it is optional: omit it when the classifier app runs the Internal Only policy, which authorizes the in-process invocation itself, and set it only when that app authenticates callers with API keys, in which case it is sent as `Authorization: Bearer`.",
|
|
43
43
|
"examples": ["$env(OPENAI_API_KEY)"]
|
|
44
44
|
},
|
|
45
45
|
"appId": {
|
|
46
46
|
"type": "string",
|
|
47
47
|
"title": "Classifier App ID",
|
|
48
|
-
"description": "Run the classifier on another AI Gateway app instead of calling an external API directly. The gateway invokes that app in-process — no outbound HTTP hop — so the classifier inherits its provider credentials, model routing, fallbacks and quotas, and provider keys stay on one app. Give that app the
|
|
48
|
+
"description": "Run the classifier on another AI Gateway app instead of calling an external API directly. The gateway invokes that app in-process — no outbound HTTP hop — so the classifier inherits its provider credentials, model routing, fallbacks and quotas, and provider keys stay on one app. Give that app the Internal Only policy and `apiKey` is not needed at all. When set, `model` must name a routed model as `provider/model` and `baseUrl` must not be set. The target app must not itself run this policy: it would be asked to classify its own classification request, which is rejected as a configuration error rather than silently skipped. Omit to call `baseUrl` directly.",
|
|
49
49
|
"examples": ["$env(CLASSIFIER_APP_ID)"],
|
|
50
50
|
"x-show-example": false
|
|
51
51
|
},
|
|
@@ -23,16 +23,21 @@ agentic, document_qa, other) and the built-in classification prompt.
|
|
|
23
23
|
|
|
24
24
|
## Choose a classifier
|
|
25
25
|
|
|
26
|
-
| Classifier
|
|
27
|
-
|
|
|
28
|
-
| `classifier.app`
|
|
29
|
-
| `classifier.chatCompletions` | `apiKey`, `model` | You
|
|
30
|
-
| `classifier.jev` | `apiKey` | You
|
|
26
|
+
| Classifier | Set | Use it when |
|
|
27
|
+
| ----------------------------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
28
|
+
| `classifier.app` | `appId`, `model` | You want the classifier to use one of your providers, through an AI Gateway app in this project. The call stays in-process. Set `adapter: "systemOne"` for Jev and other System One models. |
|
|
29
|
+
| `classifier.chatCompletions` (deprecated) | `apiKey`, `model` | You need to call an OpenAI-compatible endpoint directly with its own key, without a second AI Gateway app. |
|
|
30
|
+
| `classifier.jev` (deprecated) | `apiKey` | You need to call Jev, from TypeSafe, directly with its own key. TypeSafe and OpenRouter both serve it. |
|
|
31
31
|
|
|
32
32
|
The three are mutually exclusive. Setting more than one is rejected with a
|
|
33
33
|
`ConfigurationError` rather than guessed at, because routing on a classifier you
|
|
34
34
|
didn't mean to use would look like it was working.
|
|
35
35
|
|
|
36
|
+
Use `classifier.app` for new configurations. The direct `chatCompletions` and
|
|
37
|
+
`jev` sections are deprecated: they bypass the classifier app's policy chain,
|
|
38
|
+
budgets, and analytics, and new classifier features are added only to
|
|
39
|
+
`classifier.app`.
|
|
40
|
+
|
|
36
41
|
### AI Gateway app
|
|
37
42
|
|
|
38
43
|
```json
|
|
@@ -44,8 +49,9 @@ didn't mean to use would look like it was working.
|
|
|
44
49
|
}
|
|
45
50
|
```
|
|
46
51
|
|
|
47
|
-
- `appId` — the AI Gateway app
|
|
48
|
-
|
|
52
|
+
- `appId` — the AI Gateway app that runs the classifier.
|
|
53
|
+
- `adapter` — optional: `chatCompletions` (default) or `systemOne`. Selects
|
|
54
|
+
`/v1/chat/completions` or `/v1/systemone` on that app.
|
|
49
55
|
- `model` — the `providerName/model` that app routes the classifier request to.
|
|
50
56
|
- `apiKey` — optional. See
|
|
51
57
|
[Authenticating the classifier app](#authenticating-the-classifier-app).
|
|
@@ -55,7 +61,35 @@ policy runs on. The classifier request never leaves the gateway, so that
|
|
|
55
61
|
application needs no API key checked at all — put the Ensure Gateway Internal
|
|
56
62
|
Invocation Only policy on it in place of API Key Authentication.
|
|
57
63
|
|
|
58
|
-
|
|
64
|
+
For Jev registered with your gateway, use the same app path with System One:
|
|
65
|
+
|
|
66
|
+
```json
|
|
67
|
+
"classifier": {
|
|
68
|
+
"app": {
|
|
69
|
+
"appId": "$env(CLASSIFIER_APP_ID)",
|
|
70
|
+
"adapter": "systemOne",
|
|
71
|
+
"model": "jev/jev-latest"
|
|
72
|
+
}
|
|
73
|
+
}
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
Replace `jev` with your configured provider name. The classifier app resolves
|
|
77
|
+
the provider assignment and owns upstream credentials, routing, DB pricing,
|
|
78
|
+
budget checks, token metering, and analytics. No direct provider key, base URL,
|
|
79
|
+
or pricing configuration is needed in Smart Router. Its result still includes
|
|
80
|
+
the classifier's usage and cost; it does not emit duplicate analytics under the
|
|
81
|
+
calling app.
|
|
82
|
+
|
|
83
|
+
Smart Router sends its intent and complexity questions to the app and interprets
|
|
84
|
+
the returned System One answers. Omit `classifierPrompt` for this adapter. Keep
|
|
85
|
+
Smart Router off the classifier app's own chain. System One is not a chat-shaped
|
|
86
|
+
request, so choose policies compatible with that endpoint; existing guardrails
|
|
87
|
+
that cannot inspect its shape still reject it according to their configuration.
|
|
88
|
+
|
|
89
|
+
Prefer this app configuration for registered gateway providers. The direct
|
|
90
|
+
sections below are deprecated and remain only for existing configurations.
|
|
91
|
+
|
|
92
|
+
### Chat Completions API (deprecated)
|
|
59
93
|
|
|
60
94
|
```json
|
|
61
95
|
"classifier": {
|
|
@@ -79,7 +113,7 @@ pointing `baseUrl` at the app's URL. `classifier.app` invokes the app
|
|
|
79
113
|
in-process, and it rejects a classifier app that would call back into this
|
|
80
114
|
policy.
|
|
81
115
|
|
|
82
|
-
### Jev
|
|
116
|
+
### Jev (deprecated)
|
|
83
117
|
|
|
84
118
|
```json
|
|
85
119
|
"classifier": {
|
|
@@ -125,23 +159,29 @@ routing applies only when Jev is sure of both. Jev writes no reasons, so
|
|
|
125
159
|
`profile.reasons` is empty.
|
|
126
160
|
|
|
127
161
|
Jev takes no system prompt, so setting `classifierPrompt` with `classifier.jev`
|
|
128
|
-
is rejected with a `ConfigurationError`.
|
|
162
|
+
is rejected with a `ConfigurationError`. The same restriction applies to
|
|
163
|
+
`classifier.app.adapter: "systemOne"`.
|
|
129
164
|
|
|
130
165
|
`chatCompletions` and `jev` call a service outside the gateway with the key you
|
|
131
166
|
configure, and send it the prompt. When that service answers with an error, the
|
|
132
167
|
policy logs the status and its most likely cause but not the response body,
|
|
133
168
|
which can echo the request back, prompt included.
|
|
134
169
|
|
|
170
|
+
To migrate a direct section, register the provider with the gateway, create a
|
|
171
|
+
classifier app for it, and replace the section with `classifier.app`. Use
|
|
172
|
+
`adapter: "systemOne"` to replace `classifier.jev`. Existing `classifier.app`
|
|
173
|
+
configurations continue to default to Chat Completions when `adapter` is
|
|
174
|
+
omitted. `systemOne` uses the existing Jev request and response contract; it
|
|
175
|
+
does not add support for arbitrary decision APIs.
|
|
176
|
+
|
|
135
177
|
## Authenticating the classifier app
|
|
136
178
|
|
|
137
179
|
The classifier call is made with `context.invokeRoute` and never leaves the
|
|
138
180
|
gateway, so there is no network hop to protect. Give the classifier application
|
|
139
|
-
the
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
`classifier.app.apiKey` — nothing is sent, and there is no key to rotate or
|
|
144
|
-
leak.
|
|
181
|
+
the [Internal Only](/docs/policies/internal-only-inbound) policy instead of AI
|
|
182
|
+
Gateway Authentication: it accepts requests this gateway made itself and rejects
|
|
183
|
+
everything that arrives over the network. Then omit `classifier.app.apiKey` —
|
|
184
|
+
nothing is sent, and there is no key to rotate or leak.
|
|
145
185
|
|
|
146
186
|
Set `classifier.app.apiKey` only when the classifier application authenticates
|
|
147
187
|
callers with API keys. It is sent as `Authorization: Bearer`, typically from
|
|
@@ -271,9 +311,9 @@ classifier calls a strict JSON-schema chat completion, so it can only return one
|
|
|
271
311
|
of your configured ids. The `description` values are only shown to the
|
|
272
312
|
classifier if your prompt includes `{{intents}}` (see below).
|
|
273
313
|
|
|
274
|
-
With `classifier.jev`, the `id` values
|
|
275
|
-
and every `description` is shown to
|
|
276
|
-
`{{intents}}` doesn't apply.
|
|
314
|
+
With `classifier.jev` or `classifier.app.adapter: "systemOne"`, the `id` values
|
|
315
|
+
are the options of Jev's intent question, and every `description` is shown to
|
|
316
|
+
Jev with its option. There is no prompt, so `{{intents}}` doesn't apply.
|
|
277
317
|
|
|
278
318
|
If the classifier returns an `id` outside this list, Smart Router keeps it as an
|
|
279
319
|
"unknown intent" and caps its confidence just below `minConfidenceForRouting`,
|
|
@@ -282,8 +322,10 @@ so it's never eligible for routing.
|
|
|
282
322
|
### Custom classifier prompt
|
|
283
323
|
|
|
284
324
|
`classifierPrompt` replaces the built-in classifier prompt for `classifier.app`
|
|
285
|
-
and `classifier.chatCompletions`.
|
|
286
|
-
|
|
325
|
+
(its default or `chatCompletions` adapter) and `classifier.chatCompletions`.
|
|
326
|
+
`classifier.app.adapter: "systemOne"` and `classifier.jev` take no prompt, so
|
|
327
|
+
combining `classifierPrompt` with either is a configuration error. It accepts
|
|
328
|
+
either a string or an array of lines:
|
|
287
329
|
|
|
288
330
|
```json
|
|
289
331
|
{
|
|
@@ -326,11 +368,23 @@ interface AIGatewaySmartRouterResult {
|
|
|
326
368
|
reasons: string[];
|
|
327
369
|
};
|
|
328
370
|
usage: {
|
|
329
|
-
|
|
330
|
-
|
|
371
|
+
inputTokens: number;
|
|
372
|
+
outputTokens: number;
|
|
331
373
|
totalTokens: number;
|
|
374
|
+
breakdown?: {
|
|
375
|
+
uncachedInputTokens: number;
|
|
376
|
+
cacheReadInputTokens?: number;
|
|
377
|
+
cacheWriteInput?: Array<{ tokens: number; cacheTtl?: "5m" | "1h" }>;
|
|
378
|
+
visibleOutputTokens: number;
|
|
379
|
+
reasoningOutputTokens?: number;
|
|
380
|
+
};
|
|
332
381
|
};
|
|
333
382
|
classifierModel: string;
|
|
383
|
+
cost?: {
|
|
384
|
+
costUsd: number;
|
|
385
|
+
source: "provider" | "catalog" | "unpriced";
|
|
386
|
+
accuracy: "exact" | "estimated" | "unpriced";
|
|
387
|
+
};
|
|
334
388
|
routing: { model?: string };
|
|
335
389
|
durationMs: number;
|
|
336
390
|
promptSource: "user" | "prior-user";
|
|
@@ -352,10 +406,11 @@ interface AIGatewaySmartRouterResult {
|
|
|
352
406
|
}
|
|
353
407
|
```
|
|
354
408
|
|
|
355
|
-
With `classifier.jev
|
|
356
|
-
in the host's naming:
|
|
357
|
-
|
|
358
|
-
returns as `
|
|
409
|
+
With `classifier.jev` or `classifier.app.adapter: "systemOne"`,
|
|
410
|
+
`classifierModel` is the versioned model that answered, in the host's naming:
|
|
411
|
+
`jev-1.13.0` on TypeSafe, or `typesafe/jev-1.13-20260917` on OpenRouter. `usage`
|
|
412
|
+
reports the input and output tokens that the Jev API returns as `inputTokens`
|
|
413
|
+
and `outputTokens`.
|
|
359
414
|
|
|
360
415
|
Read it from any policy placed after Smart Router in the chain. Common uses:
|
|
361
416
|
|
|
@@ -441,10 +496,11 @@ These misconfigurations throw a `ConfigurationError` instead, which fails the
|
|
|
441
496
|
request with a `500` that names the problem. Failing open on them would look
|
|
442
497
|
like smart routing quietly doing nothing:
|
|
443
498
|
|
|
444
|
-
- More than one section in `classifier
|
|
499
|
+
- More than one section in `classifier` (`app`, `chatCompletions`, and `jev`).
|
|
445
500
|
- A deprecated `classifierAppID`, `classifierModel`, or `classifierAppApiKey`
|
|
446
501
|
next to `classifier`.
|
|
447
|
-
- `classifierPrompt` with `classifier.
|
|
502
|
+
- `classifierPrompt` with `classifier.app.adapter: "systemOne"` or
|
|
503
|
+
`classifier.jev`.
|
|
448
504
|
- A `classifier.app.appId` that names this app.
|
|
449
505
|
- A classifier app that runs this policy or Akamai AI Firewall.
|
|
450
506
|
|
|
@@ -454,3 +510,30 @@ does not parse, so there is no prompt to classify, and letting it continue would
|
|
|
454
510
|
present your routing as in effect when it was not. The policy answers a
|
|
455
511
|
`400 ValidationException` in the AWS error envelope, naming the policy. Remove
|
|
456
512
|
the policy from that route to use Bedrock Runtime through it.
|
|
513
|
+
|
|
514
|
+
### Classifier cost reporting
|
|
515
|
+
|
|
516
|
+
Use `classifier.app` for DB-backed pricing, metering, analytics and budgets.
|
|
517
|
+
After classification, `AIGatewaySmartRouter.get(context)` exposes token usage
|
|
518
|
+
and cost from the invoked gateway's accounting headers, including its accounted
|
|
519
|
+
input, output, and total tokens and cost accuracy. Smart Router does not
|
|
520
|
+
recompute usage from the provider response for app classifiers. Missing or
|
|
521
|
+
invalid token headers produce zero counts; missing cost headers produce
|
|
522
|
+
`source: "unpriced"`. Responses from older gateways without cost accuracy retain
|
|
523
|
+
a conservative estimate for catalog costs.
|
|
524
|
+
|
|
525
|
+
Deprecated direct classifiers retain token usage and any validated
|
|
526
|
+
provider-reported cost in the Smart Router result, but do not look up DB prices,
|
|
527
|
+
send gateway analytics, or increment budget meters. If the provider does not
|
|
528
|
+
report a valid cost, the result is `unpriced`; its numeric zero does not mean
|
|
529
|
+
the call was free. No additional pricing configuration is supported for direct
|
|
530
|
+
calls.
|
|
531
|
+
|
|
532
|
+
App classifier results also include `usage.breakdown` when the gateway has
|
|
533
|
+
canonical accounting detail. It exposes uncached input, cache-read input,
|
|
534
|
+
cache-write input (with TTL when known), visible output, and reasoning output.
|
|
535
|
+
These disjoint categories are already included in `inputTokens` and
|
|
536
|
+
`outputTokens`; do not add them to the totals again. Unreported detail remains
|
|
537
|
+
absent, while an explicitly reported zero is preserved. Missing or malformed
|
|
538
|
+
breakdown headers do not discard aggregate token counts or cost. Smart Router
|
|
539
|
+
does not derive these categories from the provider body or infer cache TTLs.
|