ai-gateway-proxy 0.4.0 → 0.6.0

package/README.md

@@ -155,7 +155,7 @@ The proxy gets a credential on every request with `getGatewayAuthToken()`, in th
 | `beforeRawRequest` | none | `({ kind, path, modelId, request }) => unknown`. Runs on the calls that are not language model calls; see below. |
 | `afterRawResponse` | none | `({ kind, path, modelId, request, response }) => Response \| undefined`. Runs on the response to those calls; see below. |
 
-Each hook may also return a promise. `beforeRequest`, `afterResponse` and `onError` are typed for language model calls, and every hook runs only for requests with a body. Their `modelId` is the client's `ai-language-model-id` header, such as `openai/gpt-5`, since the call body does not name the model; it is `undefined` when the request has no such header. A `GET` or `HEAD` request, such as the provider's `GET /config` model list, and any other request whose body is missing or empty, such as a bodyless `DELETE`, is forwarded without a body and without running any hook, and the upstream status and body come back unchanged, except that a redirect is answered with a 502 as below. A request body that neither `beforeRequest` nor `beforeRawRequest` receives is sent upstream as the client sent it; once either hook runs, the body it returns or leaves in place is serialised again.
+Each hook may also return a promise. `beforeRequest`, `afterResponse` and `onError` are typed for language model calls, and every hook runs only for requests with a body. Their `modelId` is the client's `ai-language-model-id` header, such as `openai/gpt-5`, since the call body does not name the model; it is `undefined` when the request has no such header. A `GET` or `HEAD` request, such as the provider's `GET /config` model list, and any other request whose body is missing or empty, such as a bodyless `DELETE`, is forwarded without a body and without running any hook, and the upstream status and body come back unchanged, except that a redirect is answered with a 502 and a gateway `authentication_error` is retyped as below. A request body that neither `beforeRequest` nor `beforeRawRequest` receives is sent upstream as the client sent it; once either hook runs, the body it returns or leaves in place is serialised again.
 
 Path matching fails closed: the language model hooks run on every request with a body except one whose path relative to `baseUrl` is exactly one of the paths `@ai-sdk/gateway` uses for calls that are not language model calls:
 
@@ -230,7 +230,7 @@ createGatewayProxy({
 Both receive `kind` (`"embedding"`, `"image"`, `"video"`, `"speech"`, `"transcription"`, `"reranking"`, `"decision"` or `"batch"`), `path` (such as `video-model/start`), `modelId` (the client's `ai-model-id` header, which is the only place an embedding or image call names its model) and `request`, the parsed JSON body.
 
 - **`beforeRawRequest`:** a returned value replaces the body sent upstream; `undefined` or `null` sends it as is.
-- **`afterRawResponse`:** also receives `response`, the `Response` the proxy would send: the upstream status, `content-type` and the allowlisted headers below, with the upstream body on success (a video call's event stream included) and the `GatewayError` body described under `onError` on a failure. Return a `Response` to send that instead, or nothing to send `response`. The body can be read once, so read `response.clone()` when returning nothing; a hook that reads `response` itself and returns nothing, or returns anything but a `Response`, is a 500. When the hook sends another `Response`, throws or returns an invalid result, the proxy cancels `response`'s body unless the hook read it, piped it or built the returned `Response` on it, so the upstream request ends. It does not run on a redirect or when the upstream request fails.
+- **`afterRawResponse`:** also receives `response`, the `Response` the proxy would send: the upstream status, `content-type` and the allowlisted headers below, with the upstream body on success (a video call's event stream included) and the `GatewayError` body described under `onError` on a failure. Return a `Response` to send that instead, or nothing to send `response`. The body can be read once, so read `response.clone()` when returning nothing; a hook that reads `response` itself and returns nothing, or returns anything but a `Response`, is a failure like a throw. A failure on a successful `response` is a 424, since the gateway has already answered, and billed, the call and an AI SDK client retries a 5xx but not a 424; on an error `response` it is a 500. When the hook sends another `Response`, throws or returns an invalid result, the proxy cancels `response`'s body unless the hook read it, piped it or built the returned `Response` on it, so the upstream request ends. It does not run on a redirect or when the upstream request fails.
 
 ```typescript
 createGatewayProxy({
@@ -250,14 +250,14 @@ Apart from forwarding the gateway's own responses, the handler can answer with a
 | Status | Type | Message | When |
 | --- | --- | --- | --- |
 | 500 | `proxy_error` | `AI Gateway authentication not configured: set AI_GATEWAY_API_KEY or deploy on Vercel with OIDC enabled` | `getGatewayAuthToken()` returned `null`. The type is not `authentication_error`, which `@ai-sdk/gateway` replaces with advice about the client's own key. |
-| upstream, such as 401 | `proxy_error` | `AI Gateway rejected the proxy's credential: <message>` | The gateway answered a request with a body with an error typed `authentication_error`; see `onError` above. The other fields of the error are kept. |
+| upstream, such as 401 | `proxy_error` | `AI Gateway rejected the proxy's credential: <message>` | The gateway answered with an error typed `authentication_error`, for a call with a body or a bodyless one such as `GET /config`; see `onError` above. The other fields of the error are kept. |
 | 400 | `invalid_request_error` | `Invalid path` | The path is not a safe relative path under `baseUrl`. |
 | 400 | `invalid_request_error` | `Invalid JSON body` | A request other than `GET` or `HEAD` has a non-empty body that is not JSON. |
 | 400 | `invalid_request_error` | `Invalid request body: a language model call must be a JSON object` | The body is JSON but not an object, such as an array or `null`, on a path that runs `beforeRequest`, `afterResponse` or `onError`, and one of them is set. No hook runs and the gateway is not called. |
 | 499 | none | none | The client aborted the request (`request.signal`) before the handler returned. The signal is passed to the upstream `fetch`, so the abort also cancels the upstream request. |
-| 500 | `proxy_error` | `Error proxying request to AI Gateway` | Anything else that throws: `extractPath`, `beforeRequest`, `beforeRawRequest`, the upstream `fetch` or `afterRawResponse`, or an `afterRawResponse` that read the body and returned no `Response`. |
+| 500 | `proxy_error` | `Error proxying request to AI Gateway` | Anything else that throws: `extractPath`, `beforeRequest`, `beforeRawRequest`, the upstream `fetch`, or an `afterRawResponse` failure on an error response. |
 | 500 | `proxy_error` | `beforeRequest hook returned an invalid request: the result must be an object` | `beforeRequest` returned a value other than an object, `undefined` or `null`. The gateway is not called. |
-| 424 | `proxy_error` | `Error processing the AI Gateway response` | After the gateway answered a non-streamed call with `afterResponse` set: `afterResponse` threw, or the success body was not a JSON object. The status is one AI SDK clients do not retry, since the gateway has billed the call. |
+| 424 | `proxy_error` | `Error processing the AI Gateway response` | After the gateway answered a non-streamed call with `afterResponse` set: `afterResponse` threw, or the success body was not a JSON object. After a successful call with `afterRawResponse` set: the hook threw, returned anything but a `Response` or nothing, or read the body and returned no `Response`. The status is one AI SDK clients do not retry, since the gateway has billed the call. |
 | 424 | `proxy_error` | `afterResponse hook returned an invalid response: <problem>` | `afterResponse` on a non-streamed request returned a result that fails the result validation above. |
 | 500 | `proxy_error` | `No response body` | A successful streamed response arrived without a body. |
 | 502 | `proxy_error` | `Upstream redirected, the proxy does not follow redirects` | The gateway answered with a 3xx status. The proxy sets `redirect: "manual"`, runs no `onError` and forwards no `Location`. |

package/dist/index.cjs

@@ -515,6 +515,7 @@ const NULL_BODY_STATUS = new Set([
 const ANSWERED_CALL_FAILURE_STATUS = 424;
 // Successful bodies that are a sequence of values rather than one JSON value: event streams and JSON Lines.
 const STREAM_CONTENT_TYPE = /^\s*(?:text\/event-stream|application\/(?:x-ndjson|jsonl|jsonlines|x-jsonlines))\s*(?:;|$)/i;
+const JSON_CONTENT_TYPE = /^\s*application\/(?:[\w.+-]+\+)?json\s*(?:;|$)/i;
 /**
  * Extracts the path from the request using either a custom extractor or route params.
  */ async function extractPathFromRequest(request, context, options) {
@@ -652,11 +653,35 @@ function proxyFailureResponse(request, error) {
         if (isRedirect(res)) {
             return redirectRefusedResponse(res);
         }
-        return passThroughResponse(res);
+        return await bodylessUpstreamResponse(res);
     } catch (error) {
         return proxyFailureResponse(request, error);
     }
 }
+/**
+ * Passes a bodyless request's response through, except that a JSON error typed
+ * `authentication_error` is retyped as for a call with a body. Only JSON error bodies are read.
+ */ async function bodylessUpstreamResponse(res) {
+    const contentType = res.headers.get("content-type") ?? "application/json";
+    if (res.ok || !res.body || !JSON_CONTENT_TYPE.test(contentType)) {
+        return passThroughResponse(res);
+    }
+    const text = await res.text();
+    const parsed = parseJsonOrUndefined(text);
+    const body = isGatewayError(parsed) && parsed.error.type === "authentication_error" ? JSON.stringify(retypeProxyCredentialError(parsed)) : text;
+    return new Response(body, {
+        status: res.status,
+        statusText: res.statusText,
+        headers: responseHeaders(res, contentType)
+    });
+}
+function parseJsonOrUndefined(text) {
+    try {
+        return JSON.parse(text);
+    } catch  {
+        return undefined;
+    }
+}
 function isGatewayError(value) {
     if (typeof value !== "object" || value === null) {
         return false;
@@ -886,7 +911,20 @@ async function runAfterResponse(res, requestBody, afterResponse) {
 }
 /**
  * Runs afterRawResponse and returns the response to send.
- */ async function applyAfterRawResponse(afterRawResponse, context) {
+ * The gateway has answered, and billed, a successful call, so a failure there gets a status clients do not retry.
+ */ async function applyAfterRawResponse(afterRawResponse, context, signal) {
+    try {
+        return await runAfterRawResponse(afterRawResponse, context);
+    } catch (error) {
+        // The handler answers a client abort with 499, and a failed call costs nothing to retry.
+        if (signal.aborted || !context.response.ok) {
+            throw error;
+        }
+        console.error("AI Gateway proxy failed after the gateway answered:", error);
+        return proxyErrorResponse(ANSWERED_CALL_FAILURE_STATUS, "proxy_error", "Error processing the AI Gateway response");
+    }
+}
+async function runAfterRawResponse(afterRawResponse, context) {
     let result;
     try {
         result = await afterRawResponse(context);
@@ -1014,7 +1052,7 @@ async function runAfterResponse(res, requestBody, afterResponse) {
                     ...rawContext,
                     request: requestBody,
                     response
-                });
+                }, request.signal);
             }
             return response;
         } catch (error) {

package/dist/index.mjs

@@ -513,6 +513,7 @@ const NULL_BODY_STATUS = new Set([
 const ANSWERED_CALL_FAILURE_STATUS = 424;
 // Successful bodies that are a sequence of values rather than one JSON value: event streams and JSON Lines.
 const STREAM_CONTENT_TYPE = /^\s*(?:text\/event-stream|application\/(?:x-ndjson|jsonl|jsonlines|x-jsonlines))\s*(?:;|$)/i;
+const JSON_CONTENT_TYPE = /^\s*application\/(?:[\w.+-]+\+)?json\s*(?:;|$)/i;
 /**
  * Extracts the path from the request using either a custom extractor or route params.
  */ async function extractPathFromRequest(request, context, options) {
@@ -650,11 +651,35 @@ function proxyFailureResponse(request, error) {
         if (isRedirect(res)) {
             return redirectRefusedResponse(res);
         }
-        return passThroughResponse(res);
+        return await bodylessUpstreamResponse(res);
     } catch (error) {
         return proxyFailureResponse(request, error);
     }
 }
+/**
+ * Passes a bodyless request's response through, except that a JSON error typed
+ * `authentication_error` is retyped as for a call with a body. Only JSON error bodies are read.
+ */ async function bodylessUpstreamResponse(res) {
+    const contentType = res.headers.get("content-type") ?? "application/json";
+    if (res.ok || !res.body || !JSON_CONTENT_TYPE.test(contentType)) {
+        return passThroughResponse(res);
+    }
+    const text = await res.text();
+    const parsed = parseJsonOrUndefined(text);
+    const body = isGatewayError(parsed) && parsed.error.type === "authentication_error" ? JSON.stringify(retypeProxyCredentialError(parsed)) : text;
+    return new Response(body, {
+        status: res.status,
+        statusText: res.statusText,
+        headers: responseHeaders(res, contentType)
+    });
+}
+function parseJsonOrUndefined(text) {
+    try {
+        return JSON.parse(text);
+    } catch  {
+        return undefined;
+    }
+}
 function isGatewayError(value) {
     if (typeof value !== "object" || value === null) {
         return false;
@@ -884,7 +909,20 @@ async function runAfterResponse(res, requestBody, afterResponse) {
 }
 /**
  * Runs afterRawResponse and returns the response to send.
- */ async function applyAfterRawResponse(afterRawResponse, context) {
+ * The gateway has answered, and billed, a successful call, so a failure there gets a status clients do not retry.
+ */ async function applyAfterRawResponse(afterRawResponse, context, signal) {
+    try {
+        return await runAfterRawResponse(afterRawResponse, context);
+    } catch (error) {
+        // The handler answers a client abort with 499, and a failed call costs nothing to retry.
+        if (signal.aborted || !context.response.ok) {
+            throw error;
+        }
+        console.error("AI Gateway proxy failed after the gateway answered:", error);
+        return proxyErrorResponse(ANSWERED_CALL_FAILURE_STATUS, "proxy_error", "Error processing the AI Gateway response");
+    }
+}
+async function runAfterRawResponse(afterRawResponse, context) {
     let result;
     try {
         result = await afterRawResponse(context);
@@ -1012,7 +1050,7 @@ async function runAfterResponse(res, requestBody, afterResponse) {
                     ...rawContext,
                     request: requestBody,
                     response
-                });
+                }, request.signal);
             }
             return response;
         } catch (error) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ai-gateway-proxy",
-  "version": "0.4.0",
+  "version": "0.6.0",
   "type": "module",
   "sideEffects": false,
   "license": "MIT",