ai-retry 0.7.0 → 0.9.0

package/README.md

@@ -95,29 +95,37 @@ console.log(result.embedding);
 
 The objects passed to the `retries` are called retryables and control the retry behavior. We can distinguish between two types of retryables:
 
-- **Static retryables** are simply models instances (language or embedding) that will always be used when an error occurs. This is also called a fallback model.
+- **Static retryables** are simply models instances (language or embedding) that will always be used when an error occurs. They are also called fallback models.
 - **Dynamic retryables** are functions that receive the current attempt context (error/result and previous attempts) and decide whether to retry with a different model based on custom logic.
 
-You can think of `retries` as a big `if-else` block, where each dynamic retryable is an `if` condition that can match a certain error/result condition, and static retryables are the `else` branches that match all other conditions. The analogy is not perfect, because the order of retryables matters because `retries` are evaluated in order until one matches:
+You can think of the `retries` array as a big `if-else` block, where each dynamic retryable is an `if` branch that can match a certain error/result condition, and static retryables are the `else` branches that match all other conditions. The analogy is not perfect, because the order of retryables matters because `retries` are evaluated in order until one matches:
 
 ```typescript
-import { openai } from '@ai-sdk/openai';
 import { generateText, streamText } from 'ai';
 import { createRetryable } from 'ai-retry';
 
 const retryableModel = createRetryable({
   // Base model
-  model: openai('gpt-4-mini'),
-  // Retryables are evaluated in order
+  model: openai('gpt-4'),
+  // Retryables are evaluated top-down in order
   retries: [
-    // Dynamic retryable that matches only certain errors
+    // Dynamic retryables act like if-branches:
+    // If error.code == 429 (too many requests) happens, retry with this model
     (context) => {
       return context.current.error.statusCode === 429 
-        ? { model: openai('gpt-3.5-turbo') }  // Retry with this model
-        : undefined;                          // Skip to next retryable
+        ? { model: azure('gpt-4-mini') }   // Retry 
+        : undefined;                       // Skip
     },
 
-    // Static retryable that always matches (fallback)
+    // If error.message ~= "service overloaded", retry with this model
+    (context) => {
+      return context.current.error.message.includes("service overloaded") 
+        ? { model: azure('gpt-4-mini') }   // Retry 
+        : undefined;                       // Skip
+    },
+
+    // Static retryables act like else branches:
+    // Else, always fallback to this model
     anthropic('claude-3-haiku-20240307'),
     // Same as:
     // { model: anthropic('claude-3-haiku-20240307'), maxAttempts: 1 }
@@ -125,7 +133,7 @@ const retryableModel = createRetryable({
 });
 ```
 
-In this example, if the base model fails with a 429 error, it will retry with `gpt-4`. In any other error case, it will fallback to `gpt-3.5-turbo`. If the order would be reversed, the static retryable would catch all errors first, and the dynamic retryable would never be reached.
+In this example, if the base model fails with code 429 or a service overloaded error, it will retry with `gpt-4-mini` on Azure. In any other error case, it will fallback to `claude-3-haiku-20240307` on Anthropic. If the order would be reversed, the static retryable would catch all errors first, and the dynamic retryable would never be reached.
 
 #### Fallbacks
 
@@ -243,12 +251,16 @@ Errors are tracked per unique model (provider + modelId). That means on the firs
 
 There are several built-in dynamic retryables available for common use cases:
 
+> [!TIP]
+> You are missing a retryable for your use case? [Open an issue](https://github.com/zirkelc/ai-retry/issues/new) and let's discuss it!
+
 - [`contentFilterTriggered`](./src/retryables/content-filter-triggered.ts): Content filter was triggered based on the prompt or completion.
 - [`requestTimeout`](./src/retryables/request-timeout.ts): Request timeout occurred.
 - [`requestNotRetryable`](./src/retryables/request-not-retryable.ts): Request failed with a non-retryable error.
 - [`retryAfterDelay`](./src/retryables/retry-after-delay.ts): Retry with delay and exponential backoff and respect `retry-after` headers.
 - [`serviceOverloaded`](./src/retryables/service-overloaded.ts): Response with status code 529 (service overloaded).
   - Use this retryable to handle Anthropic's overloaded errors.
+- [`serviceUnavailable`](./src/retryables/service-unavailable.ts): Response with status code 503 (service unavailable).
 
 #### Content Filter
 
@@ -275,20 +287,26 @@ Handle timeouts by switching to potentially faster models.
 > [!NOTE] 
 > You need to use an `abortSignal` with a timeout on your request. 
 
+When a request times out, the `requestTimeout` retryable will automatically create a fresh abort signal for the retry attempt. This prevents the retry from immediately failing due to the already-aborted signal from the original request. If you do not provide a `timeout` value, a default of 60 seconds is used for the retry attempt.
+
 ```typescript
 import { requestTimeout } from 'ai-retry/retryables';
 
 const retryableModel = createRetryable({
   model: azure('gpt-4'),
   retries: [
-    requestTimeout(azure('gpt-4-mini')), // Use faster model on timeout
+    // Defaults to 60 seconds timeout for the retry attempt
+    requestTimeout(azure('gpt-4-mini')), 
+    
+    // Or specify a custom timeout for the retry attempt
+    requestTimeout(azure('gpt-4-mini'), { timeout: 30_000 }),
   ],
 });
 
 const result = await generateText({
   model: retryableModel,
   prompt: 'Write a vegetarian lasagna recipe for 4 people.',
-  abortSignal: AbortSignal.timeout(60_000), 
+  abortSignal: AbortSignal.timeout(60_000), // Original request timeout
 });
 ```
 
@@ -310,6 +328,21 @@ const retryableModel = createRetryable({
 });
 ```
 
+#### Service Unavailable
+
+Handle service unavailable errors (status code 503) by switching to a different provider.
+
+```typescript
+import { serviceUnavailable } from 'ai-retry/retryables';
+
+const retryableModel = createRetryable({
+  model: azure('gpt-4'),
+  retries: [
+    serviceUnavailable(openai('gpt-4')), // Switch to OpenAI if Azure is unavailable
+  ],
+});
+```
+
 #### Request Not Retryable
 
 Handle cases where the base model fails with a non-retryable error.
@@ -343,10 +376,10 @@ const retryableModel = createRetryable({
   model: openai('gpt-4'), // Base model
   retries: [
     // Retry base model 3 times with fixed 2s delay
-    retryAfterDelay({ delay: 2000, maxAttempts: 3 }),
+    retryAfterDelay({ delay: 2_000, maxAttempts: 3 }),
 
     // Or retry with exponential backoff (2s, 4s, 8s)
-    retryAfterDelay({ delay: 2000, backoffFactor: 2, maxAttempts: 3 }),
+    retryAfterDelay({ delay: 2_000, backoffFactor: 2, maxAttempts: 3 }),
 
     // Or retry only if the response contains a retry-after header
     retryAfterDelay({ maxAttempts: 3 }),
@@ -367,10 +400,10 @@ const retryableModel = createRetryable({
   model: openai('gpt-4'),
   retries: [
     // Retry model 3 times with fixed 2s delay
-    { model: openai('gpt-4'), delay: 2000, maxAttempts: 3 },
+    { model: openai('gpt-4'), delay: 2_000, maxAttempts: 3 },
 
     // Or retry with exponential backoff (2s, 4s, 8s)
-    { model: openai('gpt-4'), delay: 2000, backoffFactor: 2, maxAttempts: 3 },
+    { model: openai('gpt-4'), delay: 2_000, backoffFactor: 2, maxAttempts: 3 },
   ],
 });
 
@@ -395,6 +428,30 @@ const retryableModel = createRetryable({
   ],
 });
 ```
+#### Timeouts
+
+When a retry specifies a `timeout` value, a fresh `AbortSignal.timeout()` is created for that retry attempt, replacing any existing abort signal. This is essential when retrying after timeout errors, as the original abort signal would already be in an aborted state.
+
+```typescript
+const retryableModel = createRetryable({
+  model: openai('gpt-4'),
+  retries: [
+    // Provide a fresh 30 second timeout for the retry
+    { 
+      model: openai('gpt-3.5-turbo'), 
+      timeout: 30_000 
+    },
+  ],
+});
+
+// Even if the original request times out, the retry gets a fresh signal
+const result = await generateText({
+  model: retryableModel,
+  prompt: 'Write a story',
+  // Original request timeout
+  abortSignal: AbortSignal.timeout(60_000), 
+});
+```
 
 #### Max Attempts
 
@@ -509,7 +566,7 @@ type Retryable = (
 
 #### `Retry`
 
-A `Retry` specifies the model to retry and optional settings like `maxAttempts`, `delay`, `backoffFactor`, and `providerOptions`.
+A `Retry` specifies the model to retry and optional settings like `maxAttempts`, `delay`, `backoffFactor`, `timeout`, and `providerOptions`.
 
 ```typescript
 interface Retry {
@@ -517,6 +574,7 @@ interface Retry {
   maxAttempts?: number;      // Maximum retry attempts per model (default: 1)
   delay?: number;            // Delay in milliseconds before retrying
   backoffFactor?: number;    // Multiplier for exponential backoff
+  timeout?: number;          // Timeout in milliseconds for the retry attempt
   providerOptions?: ProviderOptions; // Provider-specific options for the retry
 }
 ```
@@ -526,6 +584,7 @@ interface Retry {
 - `maxAttempts`: Maximum number of times this model can be retried. Default is 1.
 - `delay`: Delay in milliseconds to wait before retrying. The delay respects abort signals from the request.
 - `backoffFactor`: Multiplier for exponential backoff (`delay × backoffFactor^attempt`). If not provided, uses fixed delay.
+- `timeout`: Timeout in milliseconds for creating a fresh `AbortSignal.timeout()` for the retry attempt. This replaces any existing abort signal.
 - `providerOptions`: Provider-specific options that override the original request's provider options during retry attempts.
 
 #### `RetryContext`

package/dist/index.d.ts

@@ -1,4 +1,4 @@
-import { a as LanguageModelV2Generate, c as Retry, d as RetryErrorAttempt, f as RetryResultAttempt, h as RetryableOptions, i as LanguageModelV2, l as RetryAttempt, m as RetryableModelOptions, n as EmbeddingModelV2CallOptions, o as LanguageModelV2Stream, p as Retryable, r as EmbeddingModelV2Embed, s as Retries, t as EmbeddingModelV2, u as RetryContext } from "./types-CfE400mD.js";
+import { a as LanguageModelV2Generate, c as Retry, d as RetryErrorAttempt, f as RetryResultAttempt, h as RetryableOptions, i as LanguageModelV2, l as RetryAttempt, m as RetryableModelOptions, n as EmbeddingModelV2CallOptions, o as LanguageModelV2Stream, p as Retryable, r as EmbeddingModelV2Embed, s as Retries, t as EmbeddingModelV2, u as RetryContext } from "./types-BuPozWMn.js";
 import * as _ai_sdk_provider0 from "@ai-sdk/provider";
 import { LanguageModelV2 as LanguageModelV2$1, LanguageModelV2StreamPart } from "@ai-sdk/provider";
 

package/dist/index.js

@@ -201,7 +201,8 @@ var RetryableEmbeddingModel = class {
 			fn: async (currentRetry) => {
 				const callOptions = {
 					...options,
-					providerOptions: currentRetry?.providerOptions ?? options.providerOptions
+					providerOptions: currentRetry?.providerOptions ?? options.providerOptions,
+					abortSignal: currentRetry?.timeout ? AbortSignal.timeout(currentRetry.timeout) : options.abortSignal
 				};
 				return this.currentModel.doEmbed(callOptions);
 			},
@@ -374,7 +375,8 @@ var RetryableLanguageModel = class {
 			fn: async (currentRetry) => {
 				const callOptions = {
 					...options,
-					providerOptions: currentRetry?.providerOptions ?? options.providerOptions
+					providerOptions: currentRetry?.providerOptions ?? options.providerOptions,
+					abortSignal: currentRetry?.timeout ? AbortSignal.timeout(currentRetry.timeout) : options.abortSignal
 				};
 				return this.currentModel.doGenerate(callOptions);
 			},
@@ -394,7 +396,8 @@ var RetryableLanguageModel = class {
 			fn: async (currentRetry) => {
 				const callOptions = {
 					...options,
-					providerOptions: currentRetry?.providerOptions ?? options.providerOptions
+					providerOptions: currentRetry?.providerOptions ?? options.providerOptions,
+					abortSignal: currentRetry?.timeout ? AbortSignal.timeout(currentRetry.timeout) : options.abortSignal
 				};
 				return this.currentModel.doStream(callOptions);
 			},
@@ -456,7 +459,8 @@ var RetryableLanguageModel = class {
 					fn: async () => {
 						const callOptions = {
 							...options,
-							providerOptions: retryModel.providerOptions ?? options.providerOptions
+							providerOptions: retryModel.providerOptions ?? options.providerOptions,
+							abortSignal: retryModel.timeout ? AbortSignal.timeout(retryModel.timeout) : options.abortSignal
 						};
 						return this.currentModel.doStream(callOptions);
 					},

package/dist/retryables/index.d.ts

@@ -1,4 +1,4 @@
-import { h as RetryableOptions, i as LanguageModelV2, p as Retryable, t as EmbeddingModelV2 } from "../types-CfE400mD.js";
+import { h as RetryableOptions, i as LanguageModelV2, p as Retryable, t as EmbeddingModelV2 } from "../types-BuPozWMn.js";
 
 //#region src/retryables/content-filter-triggered.d.ts
 
@@ -17,7 +17,7 @@ declare function requestNotRetryable<MODEL extends LanguageModelV2 | EmbeddingMo
 /**
  * Fallback to a different model after a timeout/abort error.
  * Use in combination with the `abortSignal` option.
- * Works with both `LanguageModelV2` and `EmbeddingModelV2`.
+ * If no timeout is specified, a default of 60 seconds is used.
  */
 declare function requestTimeout<MODEL extends LanguageModelV2 | EmbeddingModelV2>(model: MODEL, options?: RetryableOptions<MODEL>): Retryable<MODEL>;
 //#endregion
@@ -39,4 +39,11 @@ declare function retryAfterDelay<MODEL extends LanguageModelV2 | EmbeddingModelV
  */
 declare function serviceOverloaded<MODEL extends LanguageModelV2 | EmbeddingModelV2>(model: MODEL, options?: RetryableOptions<MODEL>): Retryable<MODEL>;
 //#endregion
-export { contentFilterTriggered, requestNotRetryable, requestTimeout, retryAfterDelay, serviceOverloaded };
+//#region src/retryables/service-unavailable.d.ts
+/**
+ * Fallback to a different model if the provider returns a service unavailable error.
+ * This retryable handles HTTP status code 503 (Service Unavailable).
+ */
+declare function serviceUnavailable<MODEL extends LanguageModelV2 | EmbeddingModelV2>(model: MODEL, options?: RetryableOptions<MODEL>): Retryable<MODEL>;
+//#endregion
+export { contentFilterTriggered, requestNotRetryable, requestTimeout, retryAfterDelay, serviceOverloaded, serviceUnavailable };

package/dist/retryables/index.js

@@ -51,18 +51,20 @@ function requestNotRetryable(model, options) {
 /**
 * Fallback to a different model after a timeout/abort error.
 * Use in combination with the `abortSignal` option.
-* Works with both `LanguageModelV2` and `EmbeddingModelV2`.
+* If no timeout is specified, a default of 60 seconds is used.
 */
 function requestTimeout(model, options) {
 	return (context) => {
 		const { current } = context;
 		if (isErrorAttempt(current)) {
 			/**
-			* Fallback to the specified model after all retries are exhausted.
+			* Fallback to the specified model after a timeout/abort error.
+			* Provides a fresh timeout signal for the retry attempt.
 			*/
 			if (isAbortError(current.error)) return {
 				model,
 				maxAttempts: 1,
+				timeout: options?.timeout ?? 6e4,
 				...options
 			};
 		}
@@ -150,4 +152,24 @@ function serviceOverloaded(model, options) {
 }
 
 //#endregion
-export { contentFilterTriggered, requestNotRetryable, requestTimeout, retryAfterDelay, serviceOverloaded };
+//#region src/retryables/service-unavailable.ts
+/**
+* Fallback to a different model if the provider returns a service unavailable error.
+* This retryable handles HTTP status code 503 (Service Unavailable).
+*/
+function serviceUnavailable(model, options) {
+	return (context) => {
+		const { current } = context;
+		if (isErrorAttempt(current)) {
+			const { error } = current;
+			if (APICallError.isInstance(error) && error.statusCode === 503) return {
+				model,
+				maxAttempts: 1,
+				...options
+			};
+		}
+	};
+}
+
+//#endregion
+export { contentFilterTriggered, requestNotRetryable, requestTimeout, retryAfterDelay, serviceOverloaded, serviceUnavailable };

package/dist/{types-CfE400mD.d.ts → types-BuPozWMn.d.ts}

@@ -56,6 +56,7 @@ type Retry<MODEL extends LanguageModelV2$1 | EmbeddingModelV2$1> = {
   delay?: number;
   backoffFactor?: number;
   providerOptions?: ProviderOptions;
+  timeout?: number;
 };
 /**
  * A function that determines whether to retry with a different model based on the current attempt and all previous attempts.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ai-retry",
-  "version": "0.7.0",
+  "version": "0.9.0",
   "description": "AI SDK Retry",
   "main": "./dist/index.js",
   "module": "./dist/index.js",