ai-gateway-proxy 0.2.0 → 0.4.0

package/README.md

@@ -171,7 +171,7 @@ The path, whether from `extractPath` or the route param, must stay relative: one
 
 ### `afterResponse`
 
-- **Non-streamed responses:** receives the gateway's JSON body as `response`, and the value it returns is sent to the client instead, with the gateway's `usage` when the result has none, as on a stream, since an AI SDK client reads usage from every result. `undefined` or `null` forwards the upstream body. Any other result is checked against `GatewayResponse` (see "Result validation" below), and one that fails is answered with a 500 naming the problem instead of being sent.
+- **Non-streamed responses:** receives the gateway's JSON body as `response`, and the value it returns is sent to the client instead, with the gateway's `usage` when the result has none, as on a stream, since an AI SDK client reads usage from every result. `undefined` or `null` forwards the upstream body. Any other result is checked against `GatewayResponse` (see "Result validation" below), and one that fails is answered with a 424 naming the problem instead of being sent. A hook that throws, or a gateway body that is not a JSON object, is a 424 too. The gateway has already answered, and billed, the call by then, and an AI SDK client retries a 5xx but not a 424, so the call is not paid for again.
 - **Streamed responses:** every event is forwarded to the client as it arrives. When the `finish` event comes, the hook receives the aggregated response, and the `finishReason`, `usage` and `providerMetadata` it returns replace those in the `finish` event; content changes have no effect because the content has already been sent. The aggregated response also carries `response`, the `id`, `modelId` and `timestamp` (the ISO string sent over the wire) of the stream's `response-metadata` parts, the latest value of each winning, so a hook can log the provider's response id or the model that answered; it is absent when no part carried them. If the hook throws, or returns a result that fails the check below, `undefined` or `null`, the error is logged and the original `finish` event is forwarded. If the stream ends, fails or is cancelled before its `finish` event, the hook runs once with the content received so far, `incomplete` set (`{ reason: "ended" }`, `{ reason: "error", error }` or `{ reason: "cancelled", cancelReason }`), `response.usage` `undefined` and `response.finishReason` `error` when the read failed or the stream sent an `error` part, `other` otherwise. A client that leaves counts as `cancelled` whether the framework cancels the response body or aborts `request.signal`, and is not logged. Nothing can be rewritten then, so its return value is ignored and a throw is logged. `response.usage` is typed `LanguageModelV4Usage | undefined`, so a hook that meters usage must handle `undefined`.
 - **Event streams and JSON Lines on other calls:** a successful `text/event-stream` or JSON Lines (`application/x-ndjson`, `application/jsonl`) response to a request without `ai-language-model-streaming: true`, such as the video model's `doGenerate` or the batch API's `doGetBatchResults`, is forwarded as is and the hook does not run, since it is not a language model response.
 - **No content:** a `204` or `205` response is forwarded with no body and no `content-type`, keeping the allowlisted headers, and the hook does not run, since there is no response to pass it.
@@ -187,11 +187,11 @@ A result is rejected when it is not an object or when a field is missing or has
 - `warnings`: absent or an array; `@ai-sdk/gateway` reads a missing one as none
 - `providerMetadata`: absent or an object
 
-`null` is accepted wherever a field may be absent, since a body that went through JSON cannot hold `undefined`. Other fields are not checked and are forwarded as returned. A rejected result is logged with `console.error` as `afterResponse hook returned an invalid response: <problem>`, and on a non-streamed request the client gets the same message as a 500 `proxy_error`.
+`null` is accepted wherever a field may be absent, since a body that went through JSON cannot hold `undefined`. Other fields are not checked and are forwarded as returned. A rejected result is logged with `console.error` as `afterResponse hook returned an invalid response: <problem>`, and on a non-streamed request the client gets the same message as a 424 `proxy_error`.
 
 ### `onError`
 
-Receives `{ request, error, status }`, where `error` is the gateway's JSON error body. When the body is not JSON of the form `{ error: { message } }`, such as an HTML page from a load balancer, `error` is `{ error: { message: "AI Gateway returned <status>: <excerpt>", type: "gateway_error" } }`, with the first 200 characters of the body as the excerpt. Return a replacement body to send it as JSON with the upstream status, or return a `Response` to send that as is. Returning `undefined` or `null`, as a hook that only logs does, forwards the error body unchanged, and so does a result that is not of the form `{ error: { message, type?, code? }, generationId? }` with a string `message`, a string or absent `type`, a string, number or absent `code` and a string or absent `generationId`, which an AI SDK client could not parse; the problem is logged. A hook that throws is logged and the error body is forwarded too, so the client keeps the upstream status and `retry-after` rather than a 500 it would retry. A `generationId` you return reaches the client, which appends it to the error message as `[<id>]`. Without `onError`, the error body is sent with the upstream status.
+Receives `{ request, error, status }`, where `error` is the gateway's JSON error body. When the body is not JSON of the form `{ error: { message } }`, such as an HTML page from a load balancer, `error` is `{ error: { message: "AI Gateway returned <status>: <excerpt>", type: "gateway_error" } }`, with the first 200 characters of the body as the excerpt. Return a replacement body to send it as JSON with the upstream status, or return a `Response` to send that as is. Returning `undefined` or `null`, as a hook that only logs does, forwards the error body unchanged, and so does a result that is not of the form `{ error: { message, type?, code? }, generationId? }` with a string `message`, a string or absent `type`, a string, number or absent `code` and a string or absent `generationId`, which an AI SDK client could not parse; the problem is logged. A hook that throws is logged and the error body is forwarded too, so the client keeps the upstream status and `retry-after` rather than a 500 it would retry. A `generationId` you return reaches the client, which appends it to the error message as `[<id>]`. Without `onError`, the error body is sent with the upstream status. An error typed `authentication_error` is about the proxy's own credential, since the client's never reaches the gateway, so the proxy retypes it as `proxy_error` and prefixes the message with `AI Gateway rejected the proxy's credential: ` before `onError` runs; `@ai-sdk/gateway` would otherwise replace the message with advice about the client's own key. `status` stays the gateway's, such as 401.
 
 ### Example: cap, meter and mask
 
@@ -250,13 +250,15 @@ Apart from forwarding the gateway's own responses, the handler can answer with a
 | Status | Type | Message | When |
 | --- | --- | --- | --- |
 | 500 | `proxy_error` | `AI Gateway authentication not configured: set AI_GATEWAY_API_KEY or deploy on Vercel with OIDC enabled` | `getGatewayAuthToken()` returned `null`. The type is not `authentication_error`, which `@ai-sdk/gateway` replaces with advice about the client's own key. |
+| upstream, such as 401 | `proxy_error` | `AI Gateway rejected the proxy's credential: <message>` | The gateway answered a request with a body with an error typed `authentication_error`; see `onError` above. The other fields of the error are kept. |
 | 400 | `invalid_request_error` | `Invalid path` | The path is not a safe relative path under `baseUrl`. |
 | 400 | `invalid_request_error` | `Invalid JSON body` | A request other than `GET` or `HEAD` has a non-empty body that is not JSON. |
 | 400 | `invalid_request_error` | `Invalid request body: a language model call must be a JSON object` | The body is JSON but not an object, such as an array or `null`, on a path that runs `beforeRequest`, `afterResponse` or `onError`, and one of them is set. No hook runs and the gateway is not called. |
 | 499 | none | none | The client aborted the request (`request.signal`) before the handler returned. The signal is passed to the upstream `fetch`, so the abort also cancels the upstream request. |
-| 500 | `proxy_error` | `Error proxying request to AI Gateway` | Anything else that throws: `extractPath`, `beforeRequest`, `beforeRawRequest`, the upstream `fetch`, a successful non-streamed response for `afterResponse` that is not a JSON object, `afterResponse` on a non-streamed request or `afterRawResponse`, or an `afterRawResponse` that read the body and returned no `Response`. |
+| 500 | `proxy_error` | `Error proxying request to AI Gateway` | Anything else that throws: `extractPath`, `beforeRequest`, `beforeRawRequest`, the upstream `fetch` or `afterRawResponse`, or an `afterRawResponse` that read the body and returned no `Response`. |
 | 500 | `proxy_error` | `beforeRequest hook returned an invalid request: the result must be an object` | `beforeRequest` returned a value other than an object, `undefined` or `null`. The gateway is not called. |
-| 500 | `proxy_error` | `afterResponse hook returned an invalid response: <problem>` | `afterResponse` on a non-streamed request returned a result that fails the result validation above. |
+| 424 | `proxy_error` | `Error processing the AI Gateway response` | After the gateway answered a non-streamed call with `afterResponse` set: `afterResponse` threw, or the success body was not a JSON object. The status is one AI SDK clients do not retry, since the gateway has billed the call. |
+| 424 | `proxy_error` | `afterResponse hook returned an invalid response: <problem>` | `afterResponse` on a non-streamed request returned a result that fails the result validation above. |
 | 500 | `proxy_error` | `No response body` | A successful streamed response arrived without a body. |
 | 502 | `proxy_error` | `Upstream redirected, the proxy does not follow redirects` | The gateway answered with a 3xx status. The proxy sets `redirect: "manual"`, runs no `onError` and forwards no `Location`. |
 

package/dist/index.cjs

@@ -511,6 +511,8 @@ const NULL_BODY_STATUS = new Set([
     204,
     205
 ]);
+// AI SDK clients retry 408, 409, 429 and any 5xx, which would pay for the answered call again.
+const ANSWERED_CALL_FAILURE_STATUS = 424;
 // Successful bodies that are a sequence of values rather than one JSON value: event streams and JSON Lines.
 const STREAM_CONTENT_TYPE = /^\s*(?:text\/event-stream|application\/(?:x-ndjson|jsonl|jsonlines|x-jsonlines))\s*(?:;|$)/i;
 /**
@@ -683,6 +685,23 @@ function isGatewayError(value) {
         }
     };
 }
+/**
+ * Retypes a gateway `authentication_error` as a `proxy_error` naming the proxy's credential, since
+ * the client's own credential never reaches the gateway and `@ai-sdk/gateway` would replace the
+ * message with advice about it.
+ */ function retypeProxyCredentialError(upstreamError) {
+    if (upstreamError.error.type !== "authentication_error") {
+        return upstreamError;
+    }
+    return {
+        ...upstreamError,
+        error: {
+            ...upstreamError.error,
+            message: `AI Gateway rejected the proxy's credential: ${upstreamError.error.message}`,
+            type: "proxy_error"
+        }
+    };
+}
 /**
  * Builds response headers with the given content type, if any, plus the allowlisted upstream headers.
  */ function responseHeaders(res, contentType) {
@@ -773,7 +792,20 @@ function isGatewayError(value) {
 }
 /**
  * Runs afterResponse on a non-streaming JSON response from the gateway.
- */ async function handleJsonResponse(res, requestBody, afterResponse) {
+ * The gateway has answered, and billed, the call, so a failure here gets a status clients do not retry.
+ */ async function handleJsonResponse(res, requestBody, afterResponse, signal) {
+    try {
+        return await runAfterResponse(res, requestBody, afterResponse);
+    } catch (error) {
+        // The handler answers a client abort with 499.
+        if (signal.aborted) {
+            throw error;
+        }
+        console.error("AI Gateway proxy failed after the gateway answered:", error);
+        return proxyErrorResponse(ANSWERED_CALL_FAILURE_STATUS, "proxy_error", "Error processing the AI Gateway response");
+    }
+}
+async function runAfterResponse(res, requestBody, afterResponse) {
     const parsed = await res.json();
     if (!isRecord(parsed)) {
         throw new TypeError("AI Gateway returned a successful body that is not a JSON object");
@@ -793,7 +825,7 @@ function isGatewayError(value) {
         if (problem) {
             const message = `afterResponse hook returned an invalid response: ${problem}`;
             console.error(message);
-            return proxyErrorResponse(500, "proxy_error", message);
+            return proxyErrorResponse(ANSWERED_CALL_FAILURE_STATUS, "proxy_error", message);
         }
         // As on a stream's finish event: the AI SDK reads usage from every generate result, so it cannot be stripped.
         responseBody = {
@@ -825,7 +857,7 @@ function isGatewayError(value) {
  * Builds the client response for a non-redirect upstream response, running the language model hooks given.
  */ async function forwardUpstreamResponse(res, requestBody, isStreaming, { afterResponse, onError }, signal) {
     if (!res.ok) {
-        const errorBody = await parseErrorBody(res);
+        const errorBody = retypeProxyCredentialError(await parseErrorBody(res));
         return await createErrorResponse(res, errorBody, requestBody, onError);
     }
     // A 204 or 205 has no body to parse or hand to afterResponse, and a Response with it cannot carry one.
@@ -843,7 +875,7 @@ function isGatewayError(value) {
     if (!afterResponse || STREAM_CONTENT_TYPE.test(res.headers.get("content-type") ?? "")) {
         return passThroughResponse(res);
     }
-    return await handleJsonResponse(res, requestBody, afterResponse);
+    return await handleJsonResponse(res, requestBody, afterResponse, signal);
 }
 /**
  * Cancels a body nothing will read, since an unread fetch body holds its upstream connection.

package/dist/index.d.cts

@@ -210,6 +210,9 @@ type CreateGatewayProxyOptions = {
      * non-string `generationId`) logs an error and sends the upstream error body instead, and so
      * does a hook that throws.
      *
+     * An `authentication_error` arrives retyped as `proxy_error`, its message prefixed with
+     * `AI Gateway rejected the proxy's credential: `, since only the proxy's credential reaches the gateway.
+     *
      * Not called for GET, HEAD or other requests without a body, which are forwarded as is.
      *
      * Not called when the path relative to `baseUrl` is exactly one of the paths `@ai-sdk/gateway`

package/dist/index.d.cts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.cts","sources":["../src/auth.ts","../src/types.ts","../src/proxy.ts","../src/stream.ts"],"mappings":";;AAAA,KAAK,gBAAgB;AACrB;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iBAAiB,mBAAmB,IAAI,OAAO,CAAC,gBAAgB;;ACRhE;AACA;AACA;AACA;AACA,KAAK,eAAe;AACpB,aAAa,sBAAsB;AACnC,kBAAkB,2BAA2B;AAC7C;AACA,WAAW,oBAAoB;AAC/B,cAAc,eAAe;AAC7B,uBAAuB,wBAAwB;AAC/C;AACA;AACA;AACA;AACA,eAAe,uBAAuB;AACtC;AACA;AACA;AACA;AACA,KAAK,uBAAuB;AAC5B;AACA;AACA;AACA;AACA;AACA;AACA;AACA,KAAK,gBAAgB;AACrB;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,KAAK,YAAY;AACjB;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,KAAK,aAAa;AAClB;AACA;AACA;AACA,KAAK,iBAAiB;AACtB,UAAU,aAAa;AACvB;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,KAAK,kBAAkB,GAAG,iBAAiB;AAC3C;AACA;AACA;AACA;AACA;AACA,cAAc,QAAQ;AACtB;AACA;AACA;AACA;AACA,KAAK,yBAAyB;AAC9B;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,cAAc,MAAM;AACpB;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,4BAA4B,OAAO,cAAc,OAAO;AACxD;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iBAAiB,0BAA0B;AAC3C;AACA;AACA,UAAU,0BAA0B,GAAG,OAAO,CAAC,0BAA0B;AACzE;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iBAAiB,0BAA0B;AAC3C;AACA;AACA,kBAAkB,eAAe;AACjC;AACA,qBAAqB,gBAAgB;AACrC,UAAU,eAAe,GAAG,OAAO,CAAC,eAAe;AACnD;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iBAAiB,0BAA0B;AAC3C;AACA;AACA,eAAe,YAAY;AAC3B;AACA,UAAU,YAAY,GAAG,QAAQ,GAAG,OAAO,CAAC,YAAY,GAAG,QAAQ;AACnE;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iCAAiC,iBAAiB;AAClD;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iCAAiC,kBAAkB,KAAK,QAAQ,eAAe,OAAO,CAAC,QAAQ;AAC/F;AACA;AACA;AACA;AACA;AACA;AACA,KAAK,oBAAoB,aAAa,OAAO;AAC7C,YAAY,OAAO,CAAC,MAAM;AAC1B,MAAM,OAAO,CAAC,QAAQ;AACtB;AACA;AACA;AACA,KAAK,wBAAwB,GAAG,oBAAoB;AACpD,SAAS,oBAAoB;AAC7B,UAAU,oBAAoB;AAC9B,SAAS,oBAAoB;AAC7B,YAAY,oBAAoB;AAChC,WAAW,oBAAoB;AAC/B;;AC7PA;AACA;AACA;AACA;AACA;AACA;AACA,cAAc,kBAAkB,kIAAkI,yBAAyB,KAAK,wBAAwB;;ACLxN;AACA;AACA;AACA;AACA;AACA;AACA;AACA,cAAc,uBAAuB;AACrC;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,kBAAkB,yBAAyB;AAC3C;AACA;AACA;AACA;AACA;AACA;AACA;AACA,mBAAmB,eAAe;AAClC;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iBAAiB,uBAAuB,OAAO,cAAc,CAAC,UAAU,gBAAgB,0BAA0B,iBAAiB,WAAW,CAAC,yBAAyB,6BAA6B,WAAW,GAAG,cAAc,CAAC,UAAU;;","names":[]}
+{"version":3,"file":"index.d.cts","sources":["../src/auth.ts","../src/types.ts","../src/proxy.ts","../src/stream.ts"],"mappings":";;AAAA,KAAK,gBAAgB;AACrB;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iBAAiB,mBAAmB,IAAI,OAAO,CAAC,gBAAgB;;ACRhE;AACA;AACA;AACA;AACA,KAAK,eAAe;AACpB,aAAa,sBAAsB;AACnC,kBAAkB,2BAA2B;AAC7C;AACA,WAAW,oBAAoB;AAC/B,cAAc,eAAe;AAC7B,uBAAuB,wBAAwB;AAC/C;AACA;AACA;AACA;AACA,eAAe,uBAAuB;AACtC;AACA;AACA;AACA;AACA,KAAK,uBAAuB;AAC5B;AACA;AACA;AACA;AACA;AACA;AACA;AACA,KAAK,gBAAgB;AACrB;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,KAAK,YAAY;AACjB;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,KAAK,aAAa;AAClB;AACA;AACA;AACA,KAAK,iBAAiB;AACtB,UAAU,aAAa;AACvB;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,KAAK,kBAAkB,GAAG,iBAAiB;AAC3C;AACA;AACA;AACA;AACA;AACA,cAAc,QAAQ;AACtB;AACA;AACA;AACA;AACA,KAAK,yBAAyB;AAC9B;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,cAAc,MAAM;AACpB;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,4BAA4B,OAAO,cAAc,OAAO;AACxD;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iBAAiB,0BAA0B;AAC3C;AACA;AACA,UAAU,0BAA0B,GAAG,OAAO,CAAC,0BAA0B;AACzE;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iBAAiB,0BAA0B;AAC3C;AACA;AACA,kBAAkB,eAAe;AACjC;AACA,qBAAqB,gBAAgB;AACrC,UAAU,eAAe,GAAG,OAAO,CAAC,eAAe;AACnD;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iBAAiB,0BAA0B;AAC3C;AACA;AACA,eAAe,YAAY;AAC3B;AACA,UAAU,YAAY,GAAG,QAAQ,GAAG,OAAO,CAAC,YAAY,GAAG,QAAQ;AACnE;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iCAAiC,iBAAiB;AAClD;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iCAAiC,kBAAkB,KAAK,QAAQ,eAAe,OAAO,CAAC,QAAQ;AAC/F;AACA;AACA;AACA;AACA;AACA;AACA,KAAK,oBAAoB,aAAa,OAAO;AAC7C,YAAY,OAAO,CAAC,MAAM;AAC1B,MAAM,OAAO,CAAC,QAAQ;AACtB;AACA;AACA;AACA,KAAK,wBAAwB,GAAG,oBAAoB;AACpD,SAAS,oBAAoB;AAC7B,UAAU,oBAAoB;AAC9B,SAAS,oBAAoB;AAC7B,YAAY,oBAAoB;AAChC,WAAW,oBAAoB;AAC/B;;AChQA;AACA;AACA;AACA;AACA;AACA;AACA,cAAc,kBAAkB,kIAAkI,yBAAyB,KAAK,wBAAwB;;ACLxN;AACA;AACA;AACA;AACA;AACA;AACA;AACA,cAAc,uBAAuB;AACrC;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,kBAAkB,yBAAyB;AAC3C;AACA;AACA;AACA;AACA;AACA;AACA;AACA,mBAAmB,eAAe;AAClC;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iBAAiB,uBAAuB,OAAO,cAAc,CAAC,UAAU,gBAAgB,0BAA0B,iBAAiB,WAAW,CAAC,yBAAyB,6BAA6B,WAAW,GAAG,cAAc,CAAC,UAAU;;","names":[]}

package/dist/index.d.ts

@@ -210,6 +210,9 @@ type CreateGatewayProxyOptions = {
      * non-string `generationId`) logs an error and sends the upstream error body instead, and so
      * does a hook that throws.
      *
+     * An `authentication_error` arrives retyped as `proxy_error`, its message prefixed with
+     * `AI Gateway rejected the proxy's credential: `, since only the proxy's credential reaches the gateway.
+     *
      * Not called for GET, HEAD or other requests without a body, which are forwarded as is.
      *
      * Not called when the path relative to `baseUrl` is exactly one of the paths `@ai-sdk/gateway`

package/dist/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sources":["../src/auth.ts","../src/types.ts","../src/proxy.ts","../src/stream.ts"],"mappings":";;AAAA,KAAK,gBAAgB;AACrB;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iBAAiB,mBAAmB,IAAI,OAAO,CAAC,gBAAgB;;ACRhE;AACA;AACA;AACA;AACA,KAAK,eAAe;AACpB,aAAa,sBAAsB;AACnC,kBAAkB,2BAA2B;AAC7C;AACA,WAAW,oBAAoB;AAC/B,cAAc,eAAe;AAC7B,uBAAuB,wBAAwB;AAC/C;AACA;AACA;AACA;AACA,eAAe,uBAAuB;AACtC;AACA;AACA;AACA;AACA,KAAK,uBAAuB;AAC5B;AACA;AACA;AACA;AACA;AACA;AACA;AACA,KAAK,gBAAgB;AACrB;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,KAAK,YAAY;AACjB;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,KAAK,aAAa;AAClB;AACA;AACA;AACA,KAAK,iBAAiB;AACtB,UAAU,aAAa;AACvB;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,KAAK,kBAAkB,GAAG,iBAAiB;AAC3C;AACA;AACA;AACA;AACA;AACA,cAAc,QAAQ;AACtB;AACA;AACA;AACA;AACA,KAAK,yBAAyB;AAC9B;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,cAAc,MAAM;AACpB;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,4BAA4B,OAAO,cAAc,OAAO;AACxD;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iBAAiB,0BAA0B;AAC3C;AACA;AACA,UAAU,0BAA0B,GAAG,OAAO,CAAC,0BAA0B;AACzE;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iBAAiB,0BAA0B;AAC3C;AACA;AACA,kBAAkB,eAAe;AACjC;AACA,qBAAqB,gBAAgB;AACrC,UAAU,eAAe,GAAG,OAAO,CAAC,eAAe;AACnD;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iBAAiB,0BAA0B;AAC3C;AACA;AACA,eAAe,YAAY;AAC3B;AACA,UAAU,YAAY,GAAG,QAAQ,GAAG,OAAO,CAAC,YAAY,GAAG,QAAQ;AACnE;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iCAAiC,iBAAiB;AAClD;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iCAAiC,kBAAkB,KAAK,QAAQ,eAAe,OAAO,CAAC,QAAQ;AAC/F;AACA;AACA;AACA;AACA;AACA;AACA,KAAK,oBAAoB,aAAa,OAAO;AAC7C,YAAY,OAAO,CAAC,MAAM;AAC1B,MAAM,OAAO,CAAC,QAAQ;AACtB;AACA;AACA;AACA,KAAK,wBAAwB,GAAG,oBAAoB;AACpD,SAAS,oBAAoB;AAC7B,UAAU,oBAAoB;AAC9B,SAAS,oBAAoB;AAC7B,YAAY,oBAAoB;AAChC,WAAW,oBAAoB;AAC/B;;AC7PA;AACA;AACA;AACA;AACA;AACA;AACA,cAAc,kBAAkB,kIAAkI,yBAAyB,KAAK,wBAAwB;;ACLxN;AACA;AACA;AACA;AACA;AACA;AACA;AACA,cAAc,uBAAuB;AACrC;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,kBAAkB,yBAAyB;AAC3C;AACA;AACA;AACA;AACA;AACA;AACA;AACA,mBAAmB,eAAe;AAClC;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iBAAiB,uBAAuB,OAAO,cAAc,CAAC,UAAU,gBAAgB,0BAA0B,iBAAiB,WAAW,CAAC,yBAAyB,6BAA6B,WAAW,GAAG,cAAc,CAAC,UAAU;;","names":[]}
+{"version":3,"file":"index.d.ts","sources":["../src/auth.ts","../src/types.ts","../src/proxy.ts","../src/stream.ts"],"mappings":";;AAAA,KAAK,gBAAgB;AACrB;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iBAAiB,mBAAmB,IAAI,OAAO,CAAC,gBAAgB;;ACRhE;AACA;AACA;AACA;AACA,KAAK,eAAe;AACpB,aAAa,sBAAsB;AACnC,kBAAkB,2BAA2B;AAC7C;AACA,WAAW,oBAAoB;AAC/B,cAAc,eAAe;AAC7B,uBAAuB,wBAAwB;AAC/C;AACA;AACA;AACA;AACA,eAAe,uBAAuB;AACtC;AACA;AACA;AACA;AACA,KAAK,uBAAuB;AAC5B;AACA;AACA;AACA;AACA;AACA;AACA;AACA,KAAK,gBAAgB;AACrB;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,KAAK,YAAY;AACjB;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,KAAK,aAAa;AAClB;AACA;AACA;AACA,KAAK,iBAAiB;AACtB,UAAU,aAAa;AACvB;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,KAAK,kBAAkB,GAAG,iBAAiB;AAC3C;AACA;AACA;AACA;AACA;AACA,cAAc,QAAQ;AACtB;AACA;AACA;AACA;AACA,KAAK,yBAAyB;AAC9B;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,cAAc,MAAM;AACpB;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,4BAA4B,OAAO,cAAc,OAAO;AACxD;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iBAAiB,0BAA0B;AAC3C;AACA;AACA,UAAU,0BAA0B,GAAG,OAAO,CAAC,0BAA0B;AACzE;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iBAAiB,0BAA0B;AAC3C;AACA;AACA,kBAAkB,eAAe;AACjC;AACA,qBAAqB,gBAAgB;AACrC,UAAU,eAAe,GAAG,OAAO,CAAC,eAAe;AACnD;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iBAAiB,0BAA0B;AAC3C;AACA;AACA,eAAe,YAAY;AAC3B;AACA,UAAU,YAAY,GAAG,QAAQ,GAAG,OAAO,CAAC,YAAY,GAAG,QAAQ;AACnE;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iCAAiC,iBAAiB;AAClD;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iCAAiC,kBAAkB,KAAK,QAAQ,eAAe,OAAO,CAAC,QAAQ;AAC/F;AACA;AACA;AACA;AACA;AACA;AACA,KAAK,oBAAoB,aAAa,OAAO;AAC7C,YAAY,OAAO,CAAC,MAAM;AAC1B,MAAM,OAAO,CAAC,QAAQ;AACtB;AACA;AACA;AACA,KAAK,wBAAwB,GAAG,oBAAoB;AACpD,SAAS,oBAAoB;AAC7B,UAAU,oBAAoB;AAC9B,SAAS,oBAAoB;AAC7B,YAAY,oBAAoB;AAChC,WAAW,oBAAoB;AAC/B;;AChQA;AACA;AACA;AACA;AACA;AACA;AACA,cAAc,kBAAkB,kIAAkI,yBAAyB,KAAK,wBAAwB;;ACLxN;AACA;AACA;AACA;AACA;AACA;AACA;AACA,cAAc,uBAAuB;AACrC;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,kBAAkB,yBAAyB;AAC3C;AACA;AACA;AACA;AACA;AACA;AACA;AACA,mBAAmB,eAAe;AAClC;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA,iBAAiB,uBAAuB,OAAO,cAAc,CAAC,UAAU,gBAAgB,0BAA0B,iBAAiB,WAAW,CAAC,yBAAyB,6BAA6B,WAAW,GAAG,cAAc,CAAC,UAAU;;","names":[]}

package/dist/index.mjs

@@ -509,6 +509,8 @@ const NULL_BODY_STATUS = new Set([
     204,
     205
 ]);
+// AI SDK clients retry 408, 409, 429 and any 5xx, which would pay for the answered call again.
+const ANSWERED_CALL_FAILURE_STATUS = 424;
 // Successful bodies that are a sequence of values rather than one JSON value: event streams and JSON Lines.
 const STREAM_CONTENT_TYPE = /^\s*(?:text\/event-stream|application\/(?:x-ndjson|jsonl|jsonlines|x-jsonlines))\s*(?:;|$)/i;
 /**
@@ -681,6 +683,23 @@ function isGatewayError(value) {
         }
     };
 }
+/**
+ * Retypes a gateway `authentication_error` as a `proxy_error` naming the proxy's credential, since
+ * the client's own credential never reaches the gateway and `@ai-sdk/gateway` would replace the
+ * message with advice about it.
+ */ function retypeProxyCredentialError(upstreamError) {
+    if (upstreamError.error.type !== "authentication_error") {
+        return upstreamError;
+    }
+    return {
+        ...upstreamError,
+        error: {
+            ...upstreamError.error,
+            message: `AI Gateway rejected the proxy's credential: ${upstreamError.error.message}`,
+            type: "proxy_error"
+        }
+    };
+}
 /**
  * Builds response headers with the given content type, if any, plus the allowlisted upstream headers.
  */ function responseHeaders(res, contentType) {
@@ -771,7 +790,20 @@ function isGatewayError(value) {
 }
 /**
  * Runs afterResponse on a non-streaming JSON response from the gateway.
- */ async function handleJsonResponse(res, requestBody, afterResponse) {
+ * The gateway has answered, and billed, the call, so a failure here gets a status clients do not retry.
+ */ async function handleJsonResponse(res, requestBody, afterResponse, signal) {
+    try {
+        return await runAfterResponse(res, requestBody, afterResponse);
+    } catch (error) {
+        // The handler answers a client abort with 499.
+        if (signal.aborted) {
+            throw error;
+        }
+        console.error("AI Gateway proxy failed after the gateway answered:", error);
+        return proxyErrorResponse(ANSWERED_CALL_FAILURE_STATUS, "proxy_error", "Error processing the AI Gateway response");
+    }
+}
+async function runAfterResponse(res, requestBody, afterResponse) {
     const parsed = await res.json();
     if (!isRecord(parsed)) {
         throw new TypeError("AI Gateway returned a successful body that is not a JSON object");
@@ -791,7 +823,7 @@ function isGatewayError(value) {
         if (problem) {
             const message = `afterResponse hook returned an invalid response: ${problem}`;
             console.error(message);
-            return proxyErrorResponse(500, "proxy_error", message);
+            return proxyErrorResponse(ANSWERED_CALL_FAILURE_STATUS, "proxy_error", message);
         }
         // As on a stream's finish event: the AI SDK reads usage from every generate result, so it cannot be stripped.
         responseBody = {
@@ -823,7 +855,7 @@ function isGatewayError(value) {
  * Builds the client response for a non-redirect upstream response, running the language model hooks given.
  */ async function forwardUpstreamResponse(res, requestBody, isStreaming, { afterResponse, onError }, signal) {
     if (!res.ok) {
-        const errorBody = await parseErrorBody(res);
+        const errorBody = retypeProxyCredentialError(await parseErrorBody(res));
         return await createErrorResponse(res, errorBody, requestBody, onError);
     }
     // A 204 or 205 has no body to parse or hand to afterResponse, and a Response with it cannot carry one.
@@ -841,7 +873,7 @@ function isGatewayError(value) {
     if (!afterResponse || STREAM_CONTENT_TYPE.test(res.headers.get("content-type") ?? "")) {
         return passThroughResponse(res);
     }
-    return await handleJsonResponse(res, requestBody, afterResponse);
+    return await handleJsonResponse(res, requestBody, afterResponse, signal);
 }
 /**
  * Cancels a body nothing will read, since an unread fetch body holds its upstream connection.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ai-gateway-proxy",
-  "version": "0.2.0",
+  "version": "0.4.0",
   "type": "module",
   "sideEffects": false,
   "license": "MIT",