ai-retry 0.2.0 → 0.4.0

package/README.md

@@ -171,6 +171,39 @@ const retryable = createRetryable({
 });
 ```
 
+#### Retry After Delay
+
+Handle retryable errors with delays and respect `retry-after` headers from rate-limited responses. This is useful for handling 429 (Too Many Requests) and 503 (Service Unavailable) errors.
+
+> [!NOTE] 
+> If the response contains a [`retry-after`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Retry-After) header, it will be prioritized over the configured delay.
+
+
+```typescript
+import { retryAfterDelay } from 'ai-retry/retryables';
+
+const retryableModel = createRetryable({
+  model: openai('gpt-4'), // Base model
+  retries: [
+    // Retry base model 3 times with fixed 2s delay
+    retryAfterDelay({ delay: 2000, maxAttempts: 3 }),
+
+    // Or retry with exponential backoff (2s, 4s, 8s)
+    retryAfterDelay({ delay: 2000, backoffFactor: 2, maxAttempts: 3 }),
+    
+    // Or switch to a different model after delay
+    retryAfterDelay(openai('gpt-4-mini'), { delay: 1000 }),
+  ],
+});
+```
+
+**Options:**
+- `delay` (required): Delay in milliseconds before retrying
+- `backoffFactor` (optional): Multiplier for exponential backoff (delay × backoffFactor^attempt). If not provided, uses fixed delay.
+- `maxAttempts` (optional): Maximum number of retry attempts for this model
+
+By default, if a [`retry-after-ms`](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/provisioned-get-started#what-should--i-do-when-i-receive-a-429-response) or `retry-after` header is present in the response, it will be prioritized over the configured delay. The delay from the header will be capped at 60 seconds for safety. If no headers are present, the configured delay or exponential backoff will be used.
+
 #### Fallbacks
 
 If you always want to fallback to a different model on any error, you can simply provide a list of models.
@@ -247,6 +280,71 @@ try {
 }
 ```
 
+### Options
+
+#### Retry Delays
+
+You can add delays before retrying to handle rate limiting or give services time to recover. The delay respects abort signals, so requests can still be cancelled during the delay period.
+
+```typescript
+const retryableModel = createRetryable({
+  model: openai('gpt-4'),
+  retries: [
+    // Wait 1 second before retrying
+    () => ({ 
+      model: openai('gpt-4'), 
+      delay: 1_000 
+    }),
+    // Wait 2 seconds before trying a different provider
+    () => ({ 
+      model: anthropic('claude-3-haiku-20240307'), 
+      delay: 2_000 
+    }),
+  ],
+});
+
+const result = await generateText({
+  model: retryableModel,
+  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
+  // Will be respected during delays
+  abortSignal: AbortSignal.timeout(60_000), 
+});
+```
+
+You can also use delays with built-in retryables:
+
+```typescript
+import { serviceOverloaded } from 'ai-retry/retryables';
+
+const retryableModel = createRetryable({
+  model: openai('gpt-4'),
+  retries: [
+    // Wait 5 seconds before retrying on service overload
+    serviceOverloaded(openai('gpt-4'), { maxAttempts: 3, delay: 5_000 }),
+  ],
+});
+```
+
+#### Max Attempts
+
+By default, each retryable will only attempt to retry once per model to avoid infinite loops. You can customize this behavior by returning a `maxAttempts` value from your retryable function. Note that the initial request with the base model is counted as the first attempt.
+
+```typescript
+const retryableModel = createRetryable({
+  model: openai('gpt-4'),
+  retries: [
+    // Try this once
+    anthropic('claude-3-haiku-20240307'), 
+    // Try this one more time (initial + 1 retry)
+    () => ({ model: openai('gpt-4'), maxAttempts: 2 }), 
+    // Already tried, won't be retried again
+    anthropic('claude-3-haiku-20240307') 
+  ],
+});
+```
+
+The attempts are counted per unique model (provider + modelId). That means if multiple retryables return the same model, it won't be retried again once the `maxAttempts` is reached.
+
 #### Logging
 
 You can use the following callbacks to log retry attempts and errors:
@@ -285,11 +383,10 @@ There are several built-in retryables:
 - [`contentFilterTriggered`](./src/retryables/content-filter-triggered.ts): Content filter was triggered based on the prompt or completion.
 - [`requestTimeout`](./src/retryables/request-timeout.ts): Request timeout occurred.
 - [`requestNotRetryable`](./src/retryables/request-not-retryable.ts): Request failed with a non-retryable error.
+- [`retryAfterDelay`](./src/retryables/retry-after-delay.ts): Retry with exponential backoff and respect `retry-after` headers for rate limiting.
 - [`serviceOverloaded`](./src/retryables/service-overloaded.ts): Response with status code 529 (service overloaded).
   - Use this retryable to handle Anthropic's overloaded errors.
 
-By default, each retryable will only attempt to retry once per model to avoid infinite loops. You can customize this behavior by returning a `maxAttempts` value from your retryable function.
-
 ### API Reference
 
 #### `createRetryable(options: RetryableModelOptions): LanguageModelV2 | EmbeddingModelV2`
@@ -318,16 +415,20 @@ type Retryable = (
 
 #### `RetryModel`
 
-A `RetryModel` specifies the model to retry and an optional `maxAttempts` to limit how many times this model can be retried.
-By default, each retryable will only attempt to retry once per model. This can be customized by setting the `maxAttempts` property.
+A `RetryModel` specifies the model to retry and optional settings like `maxAttempts` and `delay`.
 
 ```typescript
 interface RetryModel {
   model: LanguageModelV2 | EmbeddingModelV2;
-  maxAttempts?: number;
+  maxAttempts?: number; // Maximum retry attempts per model (default: 1)
+  delay?: number;       // Delay in milliseconds before retrying
 }
 ```
 
+**Options:**
+- `maxAttempts`: Maximum number of times this model can be retried. Default is 1.
+- `delay`: Delay in milliseconds to wait before retrying. Useful for rate limiting or giving services time to recover. The delay respects abort signals from the request.
+
 #### `RetryContext`
 
 The `RetryContext` object contains information about the current attempt and all previous attempts.

package/dist/index.d.ts

@@ -1,4 +1,4 @@
-import { EmbeddingModelV2, LanguageModelV2, RetryableModelOptions } from "./types-CHhEGL5x.js";
+import { EmbeddingModelV2, EmbeddingModelV2CallOptions, EmbeddingModelV2Embed, LanguageModelV2, LanguageModelV2Generate, LanguageModelV2Stream, Retries, RetryAttempt, RetryContext, RetryErrorAttempt, RetryModel, RetryResultAttempt, Retryable, RetryableModelOptions } from "./types-BrJaHkFh.js";
 
 //#region src/create-retryable-model.d.ts
 declare function createRetryable<MODEL extends LanguageModelV2>(options: RetryableModelOptions<MODEL>): LanguageModelV2;
@@ -10,4 +10,4 @@ declare function createRetryable<MODEL extends EmbeddingModelV2>(options: Retrya
  */
 declare const getModelKey: (model: LanguageModelV2 | EmbeddingModelV2) => string;
 //#endregion
-export { createRetryable, getModelKey };
+export { EmbeddingModelV2, EmbeddingModelV2CallOptions, EmbeddingModelV2Embed, LanguageModelV2, LanguageModelV2Generate, LanguageModelV2Stream, Retries, RetryAttempt, RetryContext, RetryErrorAttempt, RetryModel, RetryResultAttempt, Retryable, RetryableModelOptions, createRetryable, getModelKey };

package/dist/index.js

@@ -1,17 +1,8 @@
-import { isErrorAttempt, isGenerateResult, isResultAttempt, isStreamContentPart } from "./utils-lRsC105f.js";
-import "@ai-sdk/provider-utils";
-import { RetryError } from "ai";
+import { getModelKey, isErrorAttempt, isGenerateResult, isResultAttempt, isStreamContentPart } from "./utils-DNoBKkQe.js";
+import { delay } from "@ai-sdk/provider-utils";
 import { getErrorMessage } from "@ai-sdk/provider";
+import { RetryError } from "ai";
 
-//#region src/get-model-key.ts
-/**
-* Generate a unique key for a LanguageModelV2 instance.
-*/
-const getModelKey = (model) => {
-	return `${model.provider}/${model.modelId}`;
-};
-
-//#endregion
 //#region src/find-retry-model.ts
 /**
 * Find the next model to retry with based on the retry context
@@ -44,7 +35,7 @@ async function findRetryModel(retries, context) {
 			/**
 			* Check if the model can still be retried based on maxAttempts
 			*/
-			if (retryAttempts.length < maxAttempts) return retryModel.model;
+			if (retryAttempts.length < maxAttempts) return retryModel;
 		}
 	}
 }
@@ -126,9 +117,10 @@ var RetryableEmbeddingModel = class {
 					attempts
 				};
 			} catch (error) {
-				const { nextModel, attempt } = await this.handleError(error, attempts);
+				const { retryModel, attempt } = await this.handleError(error, attempts);
 				attempts.push(attempt);
-				this.currentModel = nextModel;
+				if (retryModel.delay) await delay(retryModel.delay, { abortSignal: input.abortSignal });
+				this.currentModel = retryModel.model;
 			}
 		}
 	}
@@ -150,17 +142,17 @@ var RetryableEmbeddingModel = class {
 			attempts: updatedAttempts
 		};
 		this.options.onError?.(context);
-		const nextModel = await findRetryModel(this.options.retries, context);
+		const retryModel = await findRetryModel(this.options.retries, context);
 		/**
 		* Handler didn't return any models to try next, rethrow the error.
 		* If we retried the request, wrap the error into a `RetryError` for better visibility.
 		*/
-		if (!nextModel) {
+		if (!retryModel) {
 			if (updatedAttempts.length > 1) throw prepareRetryError(error, updatedAttempts);
 			throw error;
 		}
 		return {
-			nextModel,
+			retryModel,
 			attempt: errorAttempt
 		};
 	}
@@ -169,7 +161,10 @@ var RetryableEmbeddingModel = class {
 		* Always start with the original model
 		*/
 		this.currentModel = this.baseModel;
-		const { result } = await this.withRetry({ fn: async () => await this.currentModel.doEmbed(options) });
+		const { result } = await this.withRetry({
+			fn: async () => await this.currentModel.doEmbed(options),
+			abortSignal: options.abortSignal
+		});
 		return result;
 	}
 };
@@ -236,10 +231,11 @@ var RetryableLanguageModel = class {
 				* Check if the result should trigger a retry (only for generate results, not streams)
 				*/
 				if (isGenerateResult(result)) {
-					const { nextModel, attempt } = await this.handleResult(result, attempts);
+					const { retryModel, attempt } = await this.handleResult(result, attempts);
 					attempts.push(attempt);
-					if (nextModel) {
-						this.currentModel = nextModel;
+					if (retryModel) {
+						if (retryModel.delay) await delay(retryModel.delay, { abortSignal: input.abortSignal });
+						this.currentModel = retryModel.model;
 						/**
 						* Continue to the next iteration to retry
 						*/
@@ -251,9 +247,10 @@ var RetryableLanguageModel = class {
 					attempts
 				};
 			} catch (error) {
-				const { nextModel, attempt } = await this.handleError(error, attempts);
+				const { retryModel, attempt } = await this.handleError(error, attempts);
 				attempts.push(attempt);
-				this.currentModel = nextModel;
+				if (retryModel.delay) await delay(retryModel.delay, { abortSignal: input.abortSignal });
+				this.currentModel = retryModel.model;
 			}
 		}
 	}
@@ -275,7 +272,7 @@ var RetryableLanguageModel = class {
 			attempts: updatedAttempts
 		};
 		return {
-			nextModel: await findRetryModel(this.options.retries, context),
+			retryModel: await findRetryModel(this.options.retries, context),
 			attempt: resultAttempt
 		};
 	}
@@ -297,17 +294,17 @@ var RetryableLanguageModel = class {
 			attempts: updatedAttempts
 		};
 		this.options.onError?.(context);
-		const nextModel = await findRetryModel(this.options.retries, context);
+		const retryModel = await findRetryModel(this.options.retries, context);
 		/**
 		* Handler didn't return any models to try next, rethrow the error.
 		* If we retried the request, wrap the error into a `RetryError` for better visibility.
 		*/
-		if (!nextModel) {
+		if (!retryModel) {
 			if (updatedAttempts.length > 1) throw prepareRetryError(error, updatedAttempts);
 			throw error;
 		}
 		return {
-			nextModel,
+			retryModel,
 			attempt: errorAttempt
 		};
 	}
@@ -316,7 +313,10 @@ var RetryableLanguageModel = class {
 		* Always start with the original model
 		*/
 		this.currentModel = this.baseModel;
-		const { result } = await this.withRetry({ fn: async () => await this.currentModel.doGenerate(options) });
+		const { result } = await this.withRetry({
+			fn: async () => await this.currentModel.doGenerate(options),
+			abortSignal: options.abortSignal
+		});
 		return result;
 	}
 	async doStream(options) {
@@ -327,7 +327,10 @@ var RetryableLanguageModel = class {
 		/**
 		* Perform the initial call to doStream with retry logic to handle errors before any data is streamed.
 		*/
-		let { result, attempts } = await this.withRetry({ fn: async () => await this.currentModel.doStream(options) });
+		let { result, attempts } = await this.withRetry({
+			fn: async () => await this.currentModel.doStream(options),
+			abortSignal: options.abortSignal
+		});
 		/**
 		* Wrap the original stream to handle retries if an error occurs during streaming.
 		*/
@@ -362,19 +365,21 @@ var RetryableLanguageModel = class {
 				* Check if the error from the stream can be retried.
 				* Otherwise it will rethrow the error.
 				*/
-				const { nextModel, attempt } = await this.handleError(error, attempts);
-				this.currentModel = nextModel;
+				const { retryModel, attempt } = await this.handleError(error, attempts);
 				/**
 				* Save the attempt
 				*/
 				attempts.push(attempt);
+				if (retryModel.delay) await delay(retryModel.delay, { abortSignal: options.abortSignal });
+				this.currentModel = retryModel.model;
 				/**
 				* Retry the request by calling doStream again.
 				* This will create a new stream.
 				*/
 				const retriedResult = await this.withRetry({
 					fn: async () => await this.currentModel.doStream(options),
-					attempts
+					attempts,
+					abortSignal: options.abortSignal
 				});
 				/**
 				* Cancel the previous reader and stream if we are retrying

package/dist/retryables/index.d.ts

@@ -1,4 +1,4 @@
-import { EmbeddingModelV2, LanguageModelV2, RetryModel, Retryable } from "../types-CHhEGL5x.js";
+import { EmbeddingModelV2, LanguageModelV2, RetryModel, Retryable } from "../types-BrJaHkFh.js";
 
 //#region src/retryables/content-filter-triggered.d.ts
 
@@ -21,6 +21,19 @@ declare function requestNotRetryable<MODEL extends LanguageModelV2 | EmbeddingMo
  */
 declare function requestTimeout<MODEL extends LanguageModelV2 | EmbeddingModelV2>(model: MODEL, options?: Omit<RetryModel<MODEL>, 'model'>): Retryable<MODEL>;
 //#endregion
+//#region src/retryables/retry-after-delay.d.ts
+type RetryAfterDelayOptions<MODEL extends LanguageModelV2 | EmbeddingModelV2> = Omit<RetryModel<MODEL>, 'model' | 'delay'> & {
+  delay: number;
+  backoffFactor?: number;
+};
+/**
+ * Retry with the same or a different model if the error is retryable with a delay.
+ * Uses the `Retry-After` or `Retry-After-Ms` headers if present.
+ * Otherwise uses the specified `delay`, or exponential backoff if `backoffFactor` is provided.
+ */
+declare function retryAfterDelay<MODEL extends LanguageModelV2 | EmbeddingModelV2>(model: MODEL, options?: RetryAfterDelayOptions<MODEL>): Retryable<MODEL>;
+declare function retryAfterDelay<MODEL extends LanguageModelV2 | EmbeddingModelV2>(options: RetryAfterDelayOptions<MODEL>): Retryable<MODEL>;
+//#endregion
 //#region src/retryables/service-overloaded.d.ts
 /**
  * Fallback to a different model if the provider returns an overloaded error.
@@ -31,4 +44,4 @@ declare function requestTimeout<MODEL extends LanguageModelV2 | EmbeddingModelV2
  */
 declare function serviceOverloaded<MODEL extends LanguageModelV2 | EmbeddingModelV2>(model: MODEL, options?: Omit<RetryModel<MODEL>, 'model'>): Retryable<MODEL>;
 //#endregion
-export { contentFilterTriggered, requestNotRetryable, requestTimeout, serviceOverloaded };
+export { contentFilterTriggered, requestNotRetryable, requestTimeout, retryAfterDelay, serviceOverloaded };

package/dist/retryables/index.js

@@ -1,4 +1,4 @@
-import { isErrorAttempt, isObject, isResultAttempt, isString } from "../utils-lRsC105f.js";
+import { getModelKey, isErrorAttempt, isObject, isResultAttempt, isString } from "../utils-DNoBKkQe.js";
 import { isAbortError } from "@ai-sdk/provider-utils";
 import { APICallError } from "ai";
 
@@ -69,6 +69,68 @@ function requestTimeout(model, options) {
 	};
 }
 
+//#endregion
+//#region src/calculate-exponential-backoff.ts
+/**
+* Calculates the exponential backoff delay.
+*/
+function calculateExponentialBackoff(baseDelay, backoffFactor, attempts) {
+	return baseDelay * backoffFactor ** attempts;
+}
+
+//#endregion
+//#region src/parse-retry-headers.ts
+function parseRetryHeaders(headers) {
+	if (!headers) return null;
+	const retryAfterMs = headers["retry-after-ms"];
+	if (retryAfterMs) {
+		const delayMs = Number.parseFloat(retryAfterMs);
+		if (!Number.isNaN(delayMs) && delayMs >= 0) return delayMs;
+	}
+	const retryAfter = headers["retry-after"];
+	if (retryAfter) {
+		const seconds = Number.parseFloat(retryAfter);
+		if (!Number.isNaN(seconds)) return seconds * 1e3;
+		const date = Date.parse(retryAfter);
+		if (!Number.isNaN(date)) return Math.max(0, date - Date.now());
+	}
+	return null;
+}
+
+//#endregion
+//#region src/retryables/retry-after-delay.ts
+const MAX_RETRY_AFTER_MS = 6e4;
+function retryAfterDelay(modelOrOptions, options) {
+	const model = modelOrOptions && "delay" in modelOrOptions ? void 0 : modelOrOptions;
+	const opts = modelOrOptions && "delay" in modelOrOptions ? modelOrOptions : options;
+	if (!opts?.delay) throw new Error("retryAfterDelay: delay is required");
+	const delay$1 = opts.delay;
+	const backoffFactor = Math.max(opts.backoffFactor ?? 1, 1);
+	return (context) => {
+		const { current, attempts } = context;
+		if (isErrorAttempt(current)) {
+			const { error } = current;
+			if (APICallError.isInstance(error) && error.isRetryable === true) {
+				const targetModel = model ?? current.model;
+				const modelKey = getModelKey(targetModel);
+				const modelAttempts = attempts.filter((a) => getModelKey(a.model) === modelKey);
+				const headerDelay = parseRetryHeaders(error.responseHeaders);
+				if (headerDelay !== null) return {
+					model: targetModel,
+					delay: Math.min(headerDelay, MAX_RETRY_AFTER_MS),
+					maxAttempts: opts.maxAttempts
+				};
+				const calculatedDelay = calculateExponentialBackoff(delay$1, backoffFactor, modelAttempts.length);
+				return {
+					model: targetModel,
+					delay: calculatedDelay,
+					maxAttempts: opts.maxAttempts
+				};
+			}
+		}
+	};
+}
+
 //#endregion
 //#region src/retryables/service-overloaded.ts
 /**
@@ -100,4 +162,4 @@ function serviceOverloaded(model, options) {
 }
 
 //#endregion
-export { contentFilterTriggered, requestNotRetryable, requestTimeout, serviceOverloaded };
+export { contentFilterTriggered, requestNotRetryable, requestTimeout, retryAfterDelay, serviceOverloaded };

package/dist/{types-CHhEGL5x.d.ts → types-BrJaHkFh.d.ts}

@@ -52,6 +52,7 @@ type RetryAttempt<MODEL extends LanguageModelV2 | EmbeddingModelV2$1> = RetryErr
 type RetryModel<MODEL extends LanguageModelV2 | EmbeddingModelV2$1> = {
   model: MODEL;
   maxAttempts?: number;
+  delay?: number;
 };
 /**
  * A function that determines whether to retry with a different model based on the current attempt and all previous attempts.
@@ -59,5 +60,8 @@ type RetryModel<MODEL extends LanguageModelV2 | EmbeddingModelV2$1> = {
 type Retryable<MODEL extends LanguageModelV2 | EmbeddingModelV2$1> = (context: RetryContext<MODEL>) => RetryModel<MODEL> | Promise<RetryModel<MODEL>> | undefined;
 type Retries<MODEL extends LanguageModelV2 | EmbeddingModelV2$1> = Array<Retryable<MODEL> | MODEL>;
 type LanguageModelV2Generate = Awaited<ReturnType<LanguageModelV2['doGenerate']>>;
+type LanguageModelV2Stream = Awaited<ReturnType<LanguageModelV2['doStream']>>;
+type EmbeddingModelV2CallOptions<VALUE> = Parameters<EmbeddingModelV2$1<VALUE>['doEmbed']>[0];
+type EmbeddingModelV2Embed<VALUE> = Awaited<ReturnType<EmbeddingModelV2$1<VALUE>['doEmbed']>>;
 //#endregion
-export { EmbeddingModelV2$1 as EmbeddingModelV2, type LanguageModelV2, RetryModel, Retryable, RetryableModelOptions };
+export { EmbeddingModelV2$1 as EmbeddingModelV2, EmbeddingModelV2CallOptions, EmbeddingModelV2Embed, type LanguageModelV2, LanguageModelV2Generate, LanguageModelV2Stream, Retries, RetryAttempt, RetryContext, RetryErrorAttempt, RetryModel, RetryResultAttempt, Retryable, RetryableModelOptions };

package/dist/{utils-lRsC105f.js → utils-DNoBKkQe.js}

@@ -1,3 +1,12 @@
+//#region src/get-model-key.ts
+/**
+* Generate a unique key for a LanguageModelV2 instance.
+*/
+const getModelKey = (model) => {
+	return `${model.provider}/${model.modelId}`;
+};
+
+//#endregion
 //#region src/utils.ts
 const isObject = (value) => typeof value === "object" && value !== null;
 const isString = (value) => typeof value === "string";
@@ -24,4 +33,4 @@ const isStreamContentPart = (part) => {
 };
 
 //#endregion
-export { isErrorAttempt, isGenerateResult, isObject, isResultAttempt, isStreamContentPart, isString };
+export { getModelKey, isErrorAttempt, isGenerateResult, isObject, isResultAttempt, isStreamContentPart, isString };

package/package.json

@@ -1,8 +1,7 @@
 {
   "name": "ai-retry",
-  "version": "0.2.0",
+  "version": "0.4.0",
   "description": "AI SDK Retry",
-  "packageManager": "pnpm@9.0.0",
   "main": "./dist/index.js",
   "module": "./dist/index.js",
   "types": "./dist/index.d.ts",
@@ -18,14 +17,6 @@
   "publishConfig": {
     "access": "public"
   },
-  "scripts": {
-    "prepublishOnly": "pnpm build",
-    "publish:alpha": "pnpm version prerelease --preid alpha && pnpm publish --tag alpha",
-    "build": "tsdown",
-    "test": "vitest",
-    "lint": "biome check . --write",
-    "prepare": "husky"
-  },
   "keywords": [
     "ai",
     "ai-sdk",
@@ -64,5 +55,11 @@
   "dependencies": {
     "@ai-sdk/provider": "^2.0.0",
     "@ai-sdk/provider-utils": "^3.0.9"
+  },
+  "scripts": {
+    "publish:alpha": "pnpm version prerelease --preid alpha && pnpm publish --tag alpha",
+    "build": "tsdown",
+    "test": "vitest",
+    "lint": "biome check . --write"
   }
-}
+}