zuplo 7.9.14 → 7.9.15

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (47) hide show
  1. package/docs/ai-gateway/bedrock-runtime.mdx +1 -1
  2. package/docs/ai-gateway/custom-policies.mdx +1 -1
  3. package/docs/ai-gateway/fallback.mdx +1 -1
  4. package/docs/ai-gateway/integrations/ai-sdk.mdx +5 -5
  5. package/docs/ai-gateway/integrations/claude-code.mdx +6 -8
  6. package/docs/ai-gateway/integrations/codex.mdx +6 -5
  7. package/docs/ai-gateway/integrations/goose.mdx +3 -3
  8. package/docs/ai-gateway/integrations/langchain.mdx +1 -1
  9. package/docs/ai-gateway/integrations/openai.mdx +3 -3
  10. package/docs/ai-gateway/managing-providers.mdx +1 -1
  11. package/docs/ai-gateway/openrouter.mdx +14 -13
  12. package/docs/ai-gateway/overview.mdx +4 -2
  13. package/docs/ai-gateway/providers.mdx +2 -2
  14. package/docs/ai-gateway/source-control.mdx +61 -2
  15. package/docs/ai-gateway/universal-api.mdx +1 -1
  16. package/docs/ai-gateway/user-apps.mdx +5 -2
  17. package/docs/articles/accounts/audit-logs.mdx +3 -9
  18. package/docs/articles/environment-variables.mdx +56 -231
  19. package/docs/articles/graphql-support.mdx +59 -18
  20. package/docs/articles/local-development-env-variables.mdx +37 -8
  21. package/docs/articles/logging.mdx +21 -0
  22. package/docs/articles/opentelemetry.mdx +1 -1
  23. package/docs/cli/source-upgrade.mdx +39 -0
  24. package/docs/cli/source-upgrade.partial.mdx +11 -0
  25. package/docs/concepts/ai-gateway.mdx +1 -1
  26. package/docs/dev-portal/zudoku/guides/environment-variables.md +75 -0
  27. package/docs/dev-portal/zudoku-partials/guides/environment-variables.partial.md +74 -0
  28. package/docs/handlers/mcp-server.mdx +72 -0
  29. package/docs/mcp-server/overview.mdx +7 -0
  30. package/docs/mcp-server/tools.mdx +72 -0
  31. package/docs/policies/_index.md +1 -1
  32. package/docs/policies/ai-gateway-auth-inbound/doc.md +2 -2
  33. package/docs/policies/ai-gateway-configuration-executor-inbound/doc.md +3 -3
  34. package/docs/policies/ai-gateway-prompt-injection/doc.md +1 -2
  35. package/docs/policies/ai-gateway-prompt-injection/schema.json +2 -2
  36. package/docs/policies/ai-gateway-smart-router-inbound/doc.md +112 -29
  37. package/docs/policies/ai-gateway-smart-router-inbound/schema.json +109 -12
  38. package/docs/policies/internal-only-inbound/doc.md +206 -0
  39. package/docs/policies/internal-only-inbound/intro.md +10 -0
  40. package/docs/policies/internal-only-inbound/schema.json +75 -0
  41. package/docs/programmable-api/environment.mdx +220 -48
  42. package/docs/programmable-api/hooks.mdx +24 -0
  43. package/docs/programmable-api/mcp-sdk.mdx +22 -0
  44. package/package.json +5 -5
  45. package/docs/policies/ai-gateway-internal-only-inbound/doc.md +0 -102
  46. package/docs/policies/ai-gateway-internal-only-inbound/intro.md +0 -11
  47. package/docs/policies/ai-gateway-internal-only-inbound/schema.json +0 -50
@@ -97,3 +97,78 @@ in `vite-env.d.ts`. A helpful explanation can be found on
97
97
  [this StackOverflow reply](https://stackoverflow.com/a/51114250).
98
98
 
99
99
  :::
100
+
101
+ ## Environment Variables in Zuplo
102
+
103
+ Set Dev Portal variables in the Zuplo Portal under **Settings** >
104
+ [**Environment Variables**](https://portal.zuplo.com/+/account/project/settings/environment-variables),
105
+ the same place you set gateway variables. For the Dev Portal:
106
+
107
+ - Only variables whose names start with `ZUPLO_PUBLIC_` or `ZUDOKU_PUBLIC_` are
108
+ available. Both prefixes work the same way.
109
+ - Only configuration variables are available. Zuplo never sends secret variables
110
+ to the Dev Portal build, even if their names have a public prefix.
111
+ - Values are read when the portal is built and are embedded in the published
112
+ site. Redeploy the environment after you change a value.
113
+
114
+ For how Dev Portal variables differ from gateway variables, see
115
+ [Configuring Environment Variables](/docs/articles/environment-variables#gateway-and-developer-portal-variables).
116
+
117
+ ### Zuplo Build Variables
118
+
119
+ When Zuplo builds your Dev Portal, it sets the following variables. You don't
120
+ need to create them.
121
+
122
+ | Variable | Value |
123
+ | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
124
+ | `ZUPLO_ENVIRONMENT_TYPE` | The environment being built: `PRODUCTION`, `PREVIEW`, or `WORKING_COPY`. |
125
+ | `ZUPLO_SERVER_URL` | The default URL of the API gateway deployed with this Dev Portal, for example `https://my-project-main-1234abc.zuplo.app`. Custom domains don't change this value. |
126
+ | `ZUPLO_BUILD_ID` | A unique ID (UUID) for the build. |
127
+
128
+ Read these variables with `import.meta.env` in your custom components and
129
+ plugins:
130
+
131
+ ```tsx
132
+ export const EnvironmentBadge = () => (
133
+ <span>{import.meta.env.ZUPLO_ENVIRONMENT_TYPE}</span>
134
+ );
135
+ ```
136
+
137
+ `ZUPLO_ENVIRONMENT_TYPE` in the Dev Portal build uses different values than the
138
+ `ZUPLO_ENVIRONMENT_TYPE` and `ZUPLO_ENVIRONMENT_STAGE` variables available to
139
+ your gateway. See
140
+ [System Environment Variables](/docs/programmable-api/environment#system-environment-variables)
141
+ for the gateway values.
142
+
143
+ ### Check the Environment in `zudoku.config.tsx`
144
+
145
+ The configuration file runs twice: once in Node.js when the build starts, and
146
+ again as part of the built site, which renders your pages and `<head>` tags. In
147
+ the built site, `process.env` only resolves variables that start with
148
+ `ZUPLO_PUBLIC_` or `ZUDOKU_PUBLIC_`, so `process.env.ZUPLO_ENVIRONMENT_TYPE` is
149
+ `undefined` there. Read `import.meta.env` first and fall back to `process.env`
150
+ to get the same value in both places:
151
+
152
+ ```tsx title="zudoku.config.tsx"
153
+ const environmentType =
154
+ import.meta.env?.ZUPLO_ENVIRONMENT_TYPE ?? process.env.ZUPLO_ENVIRONMENT_TYPE;
155
+
156
+ const isProduction = environmentType === "PRODUCTION";
157
+ ```
158
+
159
+ The build logs a warning that `process.env.ZUPLO_ENVIRONMENT_TYPE` isn't
160
+ prefixed with `ZUPLO_PUBLIC_` or `ZUDOKU_PUBLIC_`. You can ignore it.
161
+
162
+ To avoid the warning, create a `ZUPLO_PUBLIC_` variable with a different value
163
+ for each environment instead. Both approaches include the value in the
164
+ client-side bundle, so use them only for values that aren't secret.
165
+
166
+ ### Local Development
167
+
168
+ Locally, put `ZUPLO_PUBLIC_` and `ZUDOKU_PUBLIC_` variables in a `.env` file in
169
+ your Dev Portal folder, as described in [Local Env Files](#local-env-files).
170
+ Variables you set in the Zuplo Portal aren't available locally.
171
+
172
+ `zuplo dev` sets `ZUPLO_SERVER_URL` to your local gateway, for example
173
+ `http://localhost:9000`. `ZUPLO_ENVIRONMENT_TYPE` is never `PRODUCTION` locally,
174
+ so `isProduction` in the previous example is `false`.
@@ -0,0 +1,74 @@
1
+ ## Environment Variables in Zuplo
2
+
3
+ Set Dev Portal variables in the Zuplo Portal under **Settings** >
4
+ [**Environment Variables**](https://portal.zuplo.com/+/account/project/settings/environment-variables),
5
+ the same place you set gateway variables. For the Dev Portal:
6
+
7
+ - Only variables whose names start with `ZUPLO_PUBLIC_` or `ZUDOKU_PUBLIC_` are
8
+ available. Both prefixes work the same way.
9
+ - Only configuration variables are available. Zuplo never sends secret variables
10
+ to the Dev Portal build, even if their names have a public prefix.
11
+ - Values are read when the portal is built and are embedded in the published
12
+ site. Redeploy the environment after you change a value.
13
+
14
+ For how Dev Portal variables differ from gateway variables, see
15
+ [Configuring Environment Variables](/docs/articles/environment-variables#gateway-and-developer-portal-variables).
16
+
17
+ ### Zuplo Build Variables
18
+
19
+ When Zuplo builds your Dev Portal, it sets the following variables. You don't
20
+ need to create them.
21
+
22
+ | Variable | Value |
23
+ | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
24
+ | `ZUPLO_ENVIRONMENT_TYPE` | The environment being built: `PRODUCTION`, `PREVIEW`, or `WORKING_COPY`. |
25
+ | `ZUPLO_SERVER_URL` | The default URL of the API gateway deployed with this Dev Portal, for example `https://my-project-main-1234abc.zuplo.app`. Custom domains don't change this value. |
26
+ | `ZUPLO_BUILD_ID` | A unique ID (UUID) for the build. |
27
+
28
+ Read these variables with `import.meta.env` in your custom components and
29
+ plugins:
30
+
31
+ ```tsx
32
+ export const EnvironmentBadge = () => (
33
+ <span>{import.meta.env.ZUPLO_ENVIRONMENT_TYPE}</span>
34
+ );
35
+ ```
36
+
37
+ `ZUPLO_ENVIRONMENT_TYPE` in the Dev Portal build uses different values than the
38
+ `ZUPLO_ENVIRONMENT_TYPE` and `ZUPLO_ENVIRONMENT_STAGE` variables available to
39
+ your gateway. See
40
+ [System Environment Variables](/docs/programmable-api/environment#system-environment-variables)
41
+ for the gateway values.
42
+
43
+ ### Check the Environment in `zudoku.config.tsx`
44
+
45
+ The configuration file runs twice: once in Node.js when the build starts, and
46
+ again as part of the built site, which renders your pages and `<head>` tags. In
47
+ the built site, `process.env` only resolves variables that start with
48
+ `ZUPLO_PUBLIC_` or `ZUDOKU_PUBLIC_`, so `process.env.ZUPLO_ENVIRONMENT_TYPE` is
49
+ `undefined` there. Read `import.meta.env` first and fall back to `process.env`
50
+ to get the same value in both places:
51
+
52
+ ```tsx title="zudoku.config.tsx"
53
+ const environmentType =
54
+ import.meta.env?.ZUPLO_ENVIRONMENT_TYPE ?? process.env.ZUPLO_ENVIRONMENT_TYPE;
55
+
56
+ const isProduction = environmentType === "PRODUCTION";
57
+ ```
58
+
59
+ The build logs a warning that `process.env.ZUPLO_ENVIRONMENT_TYPE` isn't
60
+ prefixed with `ZUPLO_PUBLIC_` or `ZUDOKU_PUBLIC_`. You can ignore it.
61
+
62
+ To avoid the warning, create a `ZUPLO_PUBLIC_` variable with a different value
63
+ for each environment instead. Both approaches include the value in the
64
+ client-side bundle, so use them only for values that aren't secret.
65
+
66
+ ### Local Development
67
+
68
+ Locally, put `ZUPLO_PUBLIC_` and `ZUDOKU_PUBLIC_` variables in a `.env` file in
69
+ your Dev Portal folder, as described in [Local Env Files](#local-env-files).
70
+ Variables you set in the Zuplo Portal aren't available locally.
71
+
72
+ `zuplo dev` sets `ZUPLO_SERVER_URL` to your local gateway, for example
73
+ `http://localhost:9000`. `ZUPLO_ENVIRONMENT_TYPE` is never `PRODUCTION` locally,
74
+ so `isProduction` in the previous example is `false`.
@@ -512,6 +512,78 @@ example, in Cursor:
512
512
  }
513
513
  ```
514
514
 
515
+ ## Keep REST routes private
516
+
517
+ Every operation an MCP server exposes is still a route on your gateway, and
518
+ clients can reach that route directly over HTTP. To make the routes reachable
519
+ only through the MCP server, add the
520
+ [Internal Only policy](../policies/internal-only-inbound.mdx) to every route the
521
+ server exposes.
522
+
523
+ The MCP Server handler calls each route with `context.invokeRoute`, which the
524
+ Internal Only policy accepts. The policy rejects every request that arrives over
525
+ the network. Set `status` to `404` so a direct call gets the same response as a
526
+ route that doesn't exist:
527
+
528
+ ```json title="config/policies.json"
529
+ {
530
+ "name": "internal-only-inbound",
531
+ "policyType": "internal-only-inbound",
532
+ "handler": {
533
+ "export": "InternalOnlyInboundPolicy",
534
+ "module": "$import(@zuplo/runtime)",
535
+ "options": {
536
+ "status": 404
537
+ }
538
+ }
539
+ }
540
+ ```
541
+
542
+ Put the policy first in the inbound policies of each route, so nothing else runs
543
+ for a rejected request:
544
+
545
+ ```json title="config/routes.oas.json"
546
+ "/orders/{orderId}": {
547
+ "get": {
548
+ "operationId": "get-order",
549
+ "x-internal": true,
550
+ "parameters": [
551
+ {
552
+ "name": "orderId",
553
+ "in": "path",
554
+ "required": true,
555
+ "schema": { "type": "string" }
556
+ }
557
+ ],
558
+ "x-zuplo-route": {
559
+ "corsPolicy": "none",
560
+ "handler": {
561
+ "export": "urlForwardHandler",
562
+ "module": "$import(@zuplo/runtime)",
563
+ "options": { "baseUrl": "https://orders.example.com" }
564
+ },
565
+ "policies": {
566
+ "inbound": ["internal-only-inbound"]
567
+ }
568
+ }
569
+ }
570
+ }
571
+ ```
572
+
573
+ When you hide a route:
574
+
575
+ - **Authenticate callers on the MCP server route.** The Internal Only policy
576
+ checks where a request comes from, not who sent it. The MCP server forwards
577
+ the client's request headers to each route, so a route can keep its own
578
+ authentication policy after this one.
579
+ - **Set `corsPolicy` to `none`.** Preflight `OPTIONS` requests skip inbound
580
+ policies, so a route with a CORS policy still answers them.
581
+ - **Mark the route [`x-internal`](../articles/openapi.mdx#x-internal)** to keep
582
+ it out of your developer portal.
583
+ - **A custom [not-found handler](../programmable-api/not-found-handler.mdx)**
584
+ answers paths that don't exist with your own response, so a hidden route's
585
+ `404` no longer matches it.
586
+
515
587
  ## Testing
516
588
 
517
589
  ### MCP Inspector
@@ -69,6 +69,13 @@ Expose only the operations your clients need. Apply authentication and access
69
69
  controls to those operations, use rate limits to control usage, and configure
70
70
  [audit logging](../articles/audit-logging.mdx) to record calls.
71
71
 
72
+ Each operation the server exposes is also a REST route that clients can reach
73
+ directly over HTTP. To serve those operations only through the MCP server, add
74
+ the [Internal Only policy](../policies/internal-only-inbound.mdx) to each route,
75
+ with `"status": 404` so a direct call gets the same response as a route that
76
+ doesn't exist. See
77
+ [Keep REST routes private](../handlers/mcp-server.mdx#keep-rest-routes-private).
78
+
72
79
  <span id="learn-more" />
73
80
 
74
81
  ## Next steps
@@ -364,6 +364,70 @@ MCP tool configuration error: invalid toolInputStyle "Flat" on operation "create
364
364
  If your MCP server still uses the deprecated `files` option, set
365
365
  `toolInputStyle` in the operation's `x-zuplo-mcp-tool` extension instead.
366
366
 
367
+ ## Track tool calls with runtime hooks
368
+
369
+ When the MCP Server handler invokes an API route for a tool, the global runtime
370
+ hooks run for both the client request and the internal route invocation. Each
371
+ invocation has its own request and context:
372
+
373
+ | Value | Client request | Internal tool invocation |
374
+ | ----------------------- | ---------------------------------------- | ------------------------------------------------------------------------------- |
375
+ | Method and path | `POST /mcp`, or your configured MCP path | The API route's method and path, such as `GET /weather/current?location=Boston` |
376
+ | Body | The MCP JSON-RPC envelope | The tool arguments mapped to the API request body, if the route uses one |
377
+ | `context.route` | The MCP route | The tool's API route |
378
+ | `context.parentContext` | `undefined` | The context of the MCP request that invoked the route |
379
+
380
+ Path and query arguments become URL parameters, and declared header arguments
381
+ become request headers. A `GET` tool invocation has no request body.
382
+
383
+ Use
384
+ [`context.parentContext`](../programmable-api/zuplo-context.mdx#parentcontext)
385
+ to distinguish client requests from internal invocations. It identifies any
386
+ internal route invocation, including calls made with `context.invokeRoute` in
387
+ your own code.
388
+
389
+ To log only the API route invocations for MCP tools, add this global request
390
+ hook in `modules/zuplo.runtime.ts`:
391
+
392
+ ```ts
393
+ import { RuntimeExtensions, ZuploMcpSdk } from "@zuplo/runtime";
394
+
395
+ export function runtimeInit(runtime: RuntimeExtensions) {
396
+ runtime.addRequestHook((request, context) => {
397
+ const toolCall = new ZuploMcpSdk(context).getRawCallToolRequest();
398
+
399
+ if (context.parentContext && toolCall) {
400
+ context.log.info({
401
+ toolName: toolCall.params.name,
402
+ method: request.method,
403
+ path: new URL(request.url).pathname,
404
+ requestId: context.requestId,
405
+ contextId: context.contextId,
406
+ });
407
+ }
408
+
409
+ return request;
410
+ });
411
+ }
412
+ ```
413
+
414
+ This logs a tool's API route invocation without also counting the outer `/mcp`
415
+ request. To count only client requests instead, filter for
416
+ `context.parentContext === undefined`. The SDK helper reads the current context
417
+ and its immediate parent; see
418
+ [MCP metadata in runtime hooks](../programmable-api/mcp-sdk.mdx#mcp-metadata-in-runtime-hooks)
419
+ for timing and nested-call details.
420
+
421
+ By default, the internal invocation shares the parent's `requestId` and has its
422
+ own `contextId`. Use `requestId` to group related events and `contextId` to
423
+ identify each execution. See
424
+ [Request/Response Hooks](../programmable-api/hooks.mdx#internal-route-invocations)
425
+ for hook behavior and response logging.
426
+
427
+ This extra hook execution applies when a tool invokes a Zuplo API route.
428
+ [Proxying to an upstream MCP server](../mcp-gateway/code-config/mcp-proxy-handler.mdx)
429
+ doesn't create an internal API route invocation for each tool call.
430
+
367
431
  ## Testing MCP Tools
368
432
 
369
433
  ### List Available Tools
@@ -525,6 +589,14 @@ provided by the AI client _before_ your handler is ever called. Input validation
525
589
  is an important part of MCP, so ensure you have strong validation in your
526
590
  OpenAPI JSON schemas!
527
591
 
592
+ ### Keep Tool Routes Private
593
+
594
+ A tool's route is also a REST endpoint that clients can reach directly. If AI
595
+ clients should reach the operation only through the MCP server, add the
596
+ [Internal Only policy](../policies/internal-only-inbound.mdx) to the route with
597
+ `"status": 404`. See
598
+ [Keep REST routes private](../handlers/mcp-server.mdx#keep-rest-routes-private).
599
+
528
600
  ### Custom Tools
529
601
 
530
602
  For complex workflows that don't map 1:1 to a single API route, or require
@@ -41,7 +41,6 @@
41
41
  | data-loss-prevention-outbound | Data Loss Prevention | Scans the upstream response body for sensitive data — PII, secrets, and financial identifiers — using an extensible catalog of built-in recognizers plus any custom patterns, and takes a configurable action when a match is found. The action is one of `mask` (redact matches before returning the response), `block` (replace the response with a `422` listing the detected entity names only), or `log` (record a warning and return unchanged). Only text content types are inspected; binary bodies pass through untouched, and the body is read from a clone so the client still receives the original stream. | api-gateway |
42
42
  | data-loss-prevention-inbound | Data Loss Prevention | Scans the incoming request body for sensitive data — PII, secrets, and financial identifiers — using an extensible catalog of built-in recognizers plus any custom patterns, and takes a configurable action when a match is found. The action is one of `mask` (redact matches before forwarding the request), `block` (reject with a `422` listing the detected entity names only), or `log` (record a warning and forward unchanged). Only text content types are inspected; binary bodies pass through untouched, and the body is read from a clone so the upstream still receives the original stream. | api-gateway |
43
43
  | ai-gateway-dlp-inbound | Data Loss Prevention (DLP) | Data Loss Prevention (DLP) Inbound Policy for the AI Gateway | ai-gateway |
44
- | ai-gateway-internal-only-inbound | Ensure Gateway Internal Invocation Only | Accepts only requests that this gateway made itself with `context.invokeRoute`, and rejects every request that arrived over the network with `403 Forbidden`. Use it in place of AI Gateway Authentication on an application that is only ever called by other routes and policies on the same gateway, such as the classifier application behind the Smart Router or Prompt Injection Protection policies. Those calls never leave the gateway, so an API key on them protects nothing. This policy asserts the one thing that matters: the request originated inside the deployment. The check reads `context.parentContext`, which the runtime sets only on the child context that `context.invokeRoute` creates. A client cannot supply it through a header, the URL, or the body. The policy has no options, makes no network calls, reads no body, and leaves `request.user` unset. | ai-gateway |
45
44
  | firebase-jwt-inbound | Firebase JWT Auth | Authenticate users using Firebase issued JWT tokens. | api-gateway |
46
45
  | formdata-to-json-inbound | Form Data to JSON | Converts form data in the incoming request to JSON. | api-gateway |
47
46
  | ai-gateway-galileo-tracing-inbound | Galileo Tracing | Galileo Tracing Inbound Policy | ai-gateway |
@@ -52,6 +51,7 @@
52
51
  | graphql-disable-introspection-inbound | GraphQL Disable Introspection | Policy that disables GraphQL introspection queries in production. Introspection allows clients to discover the schema, which can be a security risk as it exposes your entire API structure. | api-gateway |
53
52
  | graphql-introspection-filter-outbound | GraphQL Introspection Filter | Filters GraphQL introspection responses to exclude specific types and fields. This policy intercepts GraphQL introspection query responses and removes configured types and fields from the schema. Useful for hiding internal types or sensitive fields from the public schema. | api-gateway |
54
53
  | http-deprecation-outbound | HTTP Deprecation | Sets HTTP deprecation headers on the outgoing response following the IETF HTTP Deprecation Header standard. Supports the Deprecation, Sunset, and Link headers. | api-gateway |
54
+ | internal-only-inbound | Internal Only | Accepts only requests that this gateway made itself with `context.invokeRoute`, and rejects every request that arrived over the network. Use it on a route that only other routes, handlers, and policies on the same gateway call: a REST endpoint that an MCP server exposes as a tool, or the classifier application behind the Smart Router or Prompt Injection Protection policies. Those calls never leave the gateway, so an API key on them protects nothing. This policy asserts the one thing that matters: the request originated inside the deployment. The check reads `context.parentContext`, which the runtime sets only on the child context that `context.invokeRoute` creates. A client cannot supply it through a header, the URL, or the body. The policy makes no network calls, reads no body, and leaves `request.user` unset. A rejected request gets a `403 Forbidden` problem response with no `detail` by default, so the caller learns nothing about how the gateway works; the reason goes to the request log. Set `status` to answer with another client error, such as `404` to hide the route or `401`, `statusText` to change the status text, and `detail` to add a message. | api-gateway, ai-gateway |
55
55
  | ip-address-restriction-inbound | IP Address Restriction | Restricts which client IP addresses can call a route using an allow list, a deny list, or both. Entries are individual IP addresses or CIDR ranges, in IPv4 or IPv6. | api-gateway |
56
56
  | validate-json-schema-inbound | JSON Body Validation (deprecated) | Validates the body of an incoming request based on a JSON schema. | api-gateway |
57
57
  | open-id-jwt-auth-inbound | JWT Auth | The Open ID JWT Authentication policy allows you to authenticate incoming requests using an Open ID compliant bearer token. | api-gateway |
@@ -93,8 +93,8 @@ configuration from the route's `app_id` path parameter instead.
93
93
  > the application from the URL and accepts the request without an application
94
94
  > key. Only leave authentication out when another control restricts access:
95
95
  > network isolation, or the
96
- > [Ensure Gateway Internal Invocation Only](/docs/policies/ai-gateway-internal-only-inbound)
97
- > policy for an application that is only called from inside the gateway with
96
+ > [Internal Only](/docs/policies/internal-only-inbound) policy for an
97
+ > application that is only called from inside the gateway with
98
98
  > `context.invokeRoute`. Team policy templates can include a locked auth entry
99
99
  > so new applications require keys by default.
100
100
 
@@ -205,9 +205,9 @@ Authentication is optional and placement controls its scope:
205
205
  key for every application on the route.
206
206
  - **Internal applications** — for an application that is only called from inside
207
207
  the gateway with `context.invokeRoute`, add
208
- [Ensure Gateway Internal Invocation Only](/docs/policies/ai-gateway-internal-only-inbound)
209
- to its `inboundPolicyChain` instead of authentication. It rejects every
210
- request that arrives over the network and needs no API key.
208
+ [Internal Only](/docs/policies/internal-only-inbound) to its
209
+ `inboundPolicyChain` instead of authentication. It rejects every request that
210
+ arrives over the network and needs no API key.
211
211
 
212
212
  Use the same placement on each AI Gateway operation that should support
213
213
  application-selected chains.
@@ -50,8 +50,7 @@ classification inherits the app's provider credentials, model routing,
50
50
  fallbacks, and quotas, and provider keys stay on one app. In this mode:
51
51
 
52
52
  - `apiKey` is optional. The invocation never leaves the gateway, so give the
53
- classifier app the
54
- [Ensure Gateway Internal Invocation Only](/docs/policies/ai-gateway-internal-only-inbound)
53
+ classifier app the [Internal Only](/docs/policies/internal-only-inbound)
55
54
  policy — it accepts requests this gateway made itself and rejects everything
56
55
  that arrives over the network — and omit `apiKey` entirely. Set it only when
57
56
  the classifier app authenticates callers with API keys; it is then sent as
@@ -39,13 +39,13 @@
39
39
  "apiKey": {
40
40
  "type": "string",
41
41
  "title": "API Key",
42
- "description": "Credential used to run the classifier. Without `appId` this is the API key for the OpenAI-compatible service at `baseUrl`, and it is required. With `appId` it is optional: omit it when the classifier app runs the Ensure Gateway Internal Invocation Only policy, which authorizes the in-process invocation itself, and set it only when that app authenticates callers with API keys, in which case it is sent as `Authorization: Bearer`.",
42
+ "description": "Credential used to run the classifier. Without `appId` this is the API key for the OpenAI-compatible service at `baseUrl`, and it is required. With `appId` it is optional: omit it when the classifier app runs the Internal Only policy, which authorizes the in-process invocation itself, and set it only when that app authenticates callers with API keys, in which case it is sent as `Authorization: Bearer`.",
43
43
  "examples": ["$env(OPENAI_API_KEY)"]
44
44
  },
45
45
  "appId": {
46
46
  "type": "string",
47
47
  "title": "Classifier App ID",
48
- "description": "Run the classifier on another AI Gateway app instead of calling an external API directly. The gateway invokes that app in-process — no outbound HTTP hop — so the classifier inherits its provider credentials, model routing, fallbacks and quotas, and provider keys stay on one app. Give that app the Ensure Gateway Internal Invocation Only policy and `apiKey` is not needed at all. When set, `model` must name a routed model as `provider/model` and `baseUrl` must not be set. The target app must not itself run this policy: it would be asked to classify its own classification request, which is rejected as a configuration error rather than silently skipped. Omit to call `baseUrl` directly.",
48
+ "description": "Run the classifier on another AI Gateway app instead of calling an external API directly. The gateway invokes that app in-process — no outbound HTTP hop — so the classifier inherits its provider credentials, model routing, fallbacks and quotas, and provider keys stay on one app. Give that app the Internal Only policy and `apiKey` is not needed at all. When set, `model` must name a routed model as `provider/model` and `baseUrl` must not be set. The target app must not itself run this policy: it would be asked to classify its own classification request, which is rejected as a configuration error rather than silently skipped. Omit to call `baseUrl` directly.",
49
49
  "examples": ["$env(CLASSIFIER_APP_ID)"],
50
50
  "x-show-example": false
51
51
  },
@@ -23,16 +23,21 @@ agentic, document_qa, other) and the built-in classification prompt.
23
23
 
24
24
  ## Choose a classifier
25
25
 
26
- | Classifier | Set | Use it when |
27
- | ---------------------------- | ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
28
- | `classifier.app` | `appId`, `model` | You want the classifier to use one of your providers, through an AI Gateway app in this project. The call stays in-process. |
29
- | `classifier.chatCompletions` | `apiKey`, `model` | You want to call an OpenAI-compatible model directly with its own key, without a second AI Gateway app. |
30
- | `classifier.jev` | `apiKey` | You want a model built for classification. Jev, from TypeSafe, answers typed questions instead of generating text. TypeSafe and OpenRouter both serve it. |
26
+ | Classifier | Set | Use it when |
27
+ | ----------------------------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
28
+ | `classifier.app` | `appId`, `model` | You want the classifier to use one of your providers, through an AI Gateway app in this project. The call stays in-process. Set `adapter: "systemOne"` for Jev and other System One models. |
29
+ | `classifier.chatCompletions` (deprecated) | `apiKey`, `model` | You need to call an OpenAI-compatible endpoint directly with its own key, without a second AI Gateway app. |
30
+ | `classifier.jev` (deprecated) | `apiKey` | You need to call Jev, from TypeSafe, directly with its own key. TypeSafe and OpenRouter both serve it. |
31
31
 
32
32
  The three are mutually exclusive. Setting more than one is rejected with a
33
33
  `ConfigurationError` rather than guessed at, because routing on a classifier you
34
34
  didn't mean to use would look like it was working.
35
35
 
36
+ Use `classifier.app` for new configurations. The direct `chatCompletions` and
37
+ `jev` sections are deprecated: they bypass the classifier app's policy chain,
38
+ budgets, and analytics, and new classifier features are added only to
39
+ `classifier.app`.
40
+
36
41
  ### AI Gateway app
37
42
 
38
43
  ```json
@@ -44,8 +49,9 @@ didn't mean to use would look like it was working.
44
49
  }
45
50
  ```
46
51
 
47
- - `appId` — the AI Gateway app whose Chat Completions endpoint runs the
48
- classifier.
52
+ - `appId` — the AI Gateway app that runs the classifier.
53
+ - `adapter` — optional: `chatCompletions` (default) or `systemOne`. Selects
54
+ `/v1/chat/completions` or `/v1/systemone` on that app.
49
55
  - `model` — the `providerName/model` that app routes the classifier request to.
50
56
  - `apiKey` — optional. See
51
57
  [Authenticating the classifier app](#authenticating-the-classifier-app).
@@ -55,7 +61,35 @@ policy runs on. The classifier request never leaves the gateway, so that
55
61
  application needs no API key checked at all — put the Ensure Gateway Internal
56
62
  Invocation Only policy on it in place of API Key Authentication.
57
63
 
58
- ### Chat Completions API
64
+ For Jev registered with your gateway, use the same app path with System One:
65
+
66
+ ```json
67
+ "classifier": {
68
+ "app": {
69
+ "appId": "$env(CLASSIFIER_APP_ID)",
70
+ "adapter": "systemOne",
71
+ "model": "jev/jev-latest"
72
+ }
73
+ }
74
+ ```
75
+
76
+ Replace `jev` with your configured provider name. The classifier app resolves
77
+ the provider assignment and owns upstream credentials, routing, DB pricing,
78
+ budget checks, token metering, and analytics. No direct provider key, base URL,
79
+ or pricing configuration is needed in Smart Router. Its result still includes
80
+ the classifier's usage and cost; it does not emit duplicate analytics under the
81
+ calling app.
82
+
83
+ Smart Router sends its intent and complexity questions to the app and interprets
84
+ the returned System One answers. Omit `classifierPrompt` for this adapter. Keep
85
+ Smart Router off the classifier app's own chain. System One is not a chat-shaped
86
+ request, so choose policies compatible with that endpoint; existing guardrails
87
+ that cannot inspect its shape still reject it according to their configuration.
88
+
89
+ Prefer this app configuration for registered gateway providers. The direct
90
+ sections below are deprecated and remain only for existing configurations.
91
+
92
+ ### Chat Completions API (deprecated)
59
93
 
60
94
  ```json
61
95
  "classifier": {
@@ -79,7 +113,7 @@ pointing `baseUrl` at the app's URL. `classifier.app` invokes the app
79
113
  in-process, and it rejects a classifier app that would call back into this
80
114
  policy.
81
115
 
82
- ### Jev
116
+ ### Jev (deprecated)
83
117
 
84
118
  ```json
85
119
  "classifier": {
@@ -125,23 +159,29 @@ routing applies only when Jev is sure of both. Jev writes no reasons, so
125
159
  `profile.reasons` is empty.
126
160
 
127
161
  Jev takes no system prompt, so setting `classifierPrompt` with `classifier.jev`
128
- is rejected with a `ConfigurationError`.
162
+ is rejected with a `ConfigurationError`. The same restriction applies to
163
+ `classifier.app.adapter: "systemOne"`.
129
164
 
130
165
  `chatCompletions` and `jev` call a service outside the gateway with the key you
131
166
  configure, and send it the prompt. When that service answers with an error, the
132
167
  policy logs the status and its most likely cause but not the response body,
133
168
  which can echo the request back, prompt included.
134
169
 
170
+ To migrate a direct section, register the provider with the gateway, create a
171
+ classifier app for it, and replace the section with `classifier.app`. Use
172
+ `adapter: "systemOne"` to replace `classifier.jev`. Existing `classifier.app`
173
+ configurations continue to default to Chat Completions when `adapter` is
174
+ omitted. `systemOne` uses the existing Jev request and response contract; it
175
+ does not add support for arbitrary decision APIs.
176
+
135
177
  ## Authenticating the classifier app
136
178
 
137
179
  The classifier call is made with `context.invokeRoute` and never leaves the
138
180
  gateway, so there is no network hop to protect. Give the classifier application
139
- the
140
- [Ensure Gateway Internal Invocation Only](/docs/policies/ai-gateway-internal-only-inbound)
141
- policy instead of AI Gateway Authentication: it accepts requests this gateway
142
- made itself and rejects everything that arrives over the network. Then omit
143
- `classifier.app.apiKey` — nothing is sent, and there is no key to rotate or
144
- leak.
181
+ the [Internal Only](/docs/policies/internal-only-inbound) policy instead of AI
182
+ Gateway Authentication: it accepts requests this gateway made itself and rejects
183
+ everything that arrives over the network. Then omit `classifier.app.apiKey` —
184
+ nothing is sent, and there is no key to rotate or leak.
145
185
 
146
186
  Set `classifier.app.apiKey` only when the classifier application authenticates
147
187
  callers with API keys. It is sent as `Authorization: Bearer`, typically from
@@ -271,9 +311,9 @@ classifier calls a strict JSON-schema chat completion, so it can only return one
271
311
  of your configured ids. The `description` values are only shown to the
272
312
  classifier if your prompt includes `{{intents}}` (see below).
273
313
 
274
- With `classifier.jev`, the `id` values are the options of Jev's intent question,
275
- and every `description` is shown to Jev with its option. There is no prompt, so
276
- `{{intents}}` doesn't apply.
314
+ With `classifier.jev` or `classifier.app.adapter: "systemOne"`, the `id` values
315
+ are the options of Jev's intent question, and every `description` is shown to
316
+ Jev with its option. There is no prompt, so `{{intents}}` doesn't apply.
277
317
 
278
318
  If the classifier returns an `id` outside this list, Smart Router keeps it as an
279
319
  "unknown intent" and caps its confidence just below `minConfidenceForRouting`,
@@ -282,8 +322,10 @@ so it's never eligible for routing.
282
322
  ### Custom classifier prompt
283
323
 
284
324
  `classifierPrompt` replaces the built-in classifier prompt for `classifier.app`
285
- and `classifier.chatCompletions`. Jev takes no prompt, so it can't be combined
286
- with `classifier.jev`. It accepts either a string or an array of lines:
325
+ (its default or `chatCompletions` adapter) and `classifier.chatCompletions`.
326
+ `classifier.app.adapter: "systemOne"` and `classifier.jev` take no prompt, so
327
+ combining `classifierPrompt` with either is a configuration error. It accepts
328
+ either a string or an array of lines:
287
329
 
288
330
  ```json
289
331
  {
@@ -326,11 +368,23 @@ interface AIGatewaySmartRouterResult {
326
368
  reasons: string[];
327
369
  };
328
370
  usage: {
329
- promptTokens: number;
330
- completionTokens: number;
371
+ inputTokens: number;
372
+ outputTokens: number;
331
373
  totalTokens: number;
374
+ breakdown?: {
375
+ uncachedInputTokens: number;
376
+ cacheReadInputTokens?: number;
377
+ cacheWriteInput?: Array<{ tokens: number; cacheTtl?: "5m" | "1h" }>;
378
+ visibleOutputTokens: number;
379
+ reasoningOutputTokens?: number;
380
+ };
332
381
  };
333
382
  classifierModel: string;
383
+ cost?: {
384
+ costUsd: number;
385
+ source: "provider" | "catalog" | "unpriced";
386
+ accuracy: "exact" | "estimated" | "unpriced";
387
+ };
334
388
  routing: { model?: string };
335
389
  durationMs: number;
336
390
  promptSource: "user" | "prior-user";
@@ -352,10 +406,11 @@ interface AIGatewaySmartRouterResult {
352
406
  }
353
407
  ```
354
408
 
355
- With `classifier.jev`, `classifierModel` is the versioned model that answered,
356
- in the host's naming: `jev-1.13.0` on TypeSafe, or `typesafe/jev-1.13-20260917`
357
- on OpenRouter. `usage` reports the input and output tokens that the Jev API
358
- returns as `promptTokens` and `completionTokens`.
409
+ With `classifier.jev` or `classifier.app.adapter: "systemOne"`,
410
+ `classifierModel` is the versioned model that answered, in the host's naming:
411
+ `jev-1.13.0` on TypeSafe, or `typesafe/jev-1.13-20260917` on OpenRouter. `usage`
412
+ reports the input and output tokens that the Jev API returns as `inputTokens`
413
+ and `outputTokens`.
359
414
 
360
415
  Read it from any policy placed after Smart Router in the chain. Common uses:
361
416
 
@@ -441,10 +496,11 @@ These misconfigurations throw a `ConfigurationError` instead, which fails the
441
496
  request with a `500` that names the problem. Failing open on them would look
442
497
  like smart routing quietly doing nothing:
443
498
 
444
- - More than one section in `classifier`.
499
+ - More than one section in `classifier` (`app`, `chatCompletions`, and `jev`).
445
500
  - A deprecated `classifierAppID`, `classifierModel`, or `classifierAppApiKey`
446
501
  next to `classifier`.
447
- - `classifierPrompt` with `classifier.jev`.
502
+ - `classifierPrompt` with `classifier.app.adapter: "systemOne"` or
503
+ `classifier.jev`.
448
504
  - A `classifier.app.appId` that names this app.
449
505
  - A classifier app that runs this policy or Akamai AI Firewall.
450
506
 
@@ -454,3 +510,30 @@ does not parse, so there is no prompt to classify, and letting it continue would
454
510
  present your routing as in effect when it was not. The policy answers a
455
511
  `400 ValidationException` in the AWS error envelope, naming the policy. Remove
456
512
  the policy from that route to use Bedrock Runtime through it.
513
+
514
+ ### Classifier cost reporting
515
+
516
+ Use `classifier.app` for DB-backed pricing, metering, analytics and budgets.
517
+ After classification, `AIGatewaySmartRouter.get(context)` exposes token usage
518
+ and cost from the invoked gateway's accounting headers, including its accounted
519
+ input, output, and total tokens and cost accuracy. Smart Router does not
520
+ recompute usage from the provider response for app classifiers. Missing or
521
+ invalid token headers produce zero counts; missing cost headers produce
522
+ `source: "unpriced"`. Responses from older gateways without cost accuracy retain
523
+ a conservative estimate for catalog costs.
524
+
525
+ Deprecated direct classifiers retain token usage and any validated
526
+ provider-reported cost in the Smart Router result, but do not look up DB prices,
527
+ send gateway analytics, or increment budget meters. If the provider does not
528
+ report a valid cost, the result is `unpriced`; its numeric zero does not mean
529
+ the call was free. No additional pricing configuration is supported for direct
530
+ calls.
531
+
532
+ App classifier results also include `usage.breakdown` when the gateway has
533
+ canonical accounting detail. It exposes uncached input, cache-read input,
534
+ cache-write input (with TTL when known), visible output, and reasoning output.
535
+ These disjoint categories are already included in `inputTokens` and
536
+ `outputTokens`; do not add them to the totals again. Unreported detail remains
537
+ absent, while an explicitly reported zero is preserved. Missing or malformed
538
+ breakdown headers do not discard aggregate token counts or cost. Smart Router
539
+ does not derive these categories from the provider body or infer cache TTLs.