@splashcodex/api-key-manager 1.0.0 → 3.0.0

package/README.md

@@ -1,14 +1,24 @@
 # @splashcodex/api-key-manager
 
-A robust, universal API Key Rotation and Management system designed for high-availability applications using rate-limited APIs (like Google Gemini).
+> Universal API Key Rotation System with Resilience, Load Balancing & AI Gateway Features
+
+[![npm version](https://img.shields.io/npm/v/@splashcodex/api-key-manager)](https://www.npmjs.com/package/@splashcodex/api-key-manager)
 
 ## Features
 
--   **🔄 Automatic Key Rotation**: Seamlessly switches to the next available key upon exhaustion.
--   **🔌 Circuit Breaker Pattern**: Automatically "opens" the circuit for keys that return 429s or 500s, preventing wasted requests.
--   **💾 Persistence**: Remembers key states (failures, cooldowns) across restarts using local storage.
--   **🧠 Smart Error Classification**: Distinguishes between transient errors (retryable), quota errors (cooldown), and auth errors (dead).
--   **⏱️ Jittered Exponential Backoff**: Prevents thundering herd problems during retries.
+- **Circuit Breaker** — Keys transition through `CLOSED → OPEN → HALF_OPEN → DEAD`
+- **Error Classification** — Automatic detection of 429 (Quota), 403 (Auth), 5xx (Transient), Timeout, Safety blocks
+- **Pluggable Strategies** — `StandardStrategy`, `WeightedStrategy`, `LatencyStrategy`
+- **`execute()` Wrapper** — Single method: get key → call → latency → retry → fallback
+- **Event Emitter** — Typed lifecycle hooks for monitoring & alerting
+- **Auto-Retry with Backoff** — Built-in retry loop with exponential backoff + jitter
+- **Request Timeout** — `AbortController`-based timeout per attempt
+- **Fallback Function** — Graceful degradation when all keys fail
+- **Provider Tagging** — Multi-provider routing (`openai`, `gemini`, etc.)
+- **Health Checks** — Periodic key validation and auto-recovery
+- **Bulkhead / Concurrency** — Limits concurrent `execute()` calls
+- **State Persistence** — Survives restarts via pluggable storage
+- **100% Backward Compatible** — v1.x and v2.x code works without changes
 
 ## Installation
 
@@ -16,65 +26,201 @@ A robust, universal API Key Rotation and Management system designed for high-ava
 npm install @splashcodex/api-key-manager
 ```
 
-## Usage
-
-### 1. Initialize
+## Quick Start
 
 ```typescript
 import { ApiKeyManager } from '@splashcodex/api-key-manager';
 
-// Initialize with a pool of keys
-const apiKeys = [
-  "AIzaSy...",
-  "AIzaSy...",
-  "AIzaSy..."
-];
+// Simple (v1/v2 compatible)
+const manager = new ApiKeyManager(['key1', 'key2', 'key3']);
+const key = manager.getKey();
+manager.markSuccess(key!);
 
-const manager = new ApiKeyManager(apiKeys);
+// v3 — Full power
+const result = await manager.execute(
+  (key) => fetch(`https://api.example.com?key=${key}`),
+  { maxRetries: 3, timeoutMs: 5000 }
+);
 ```
 
-### 2. Get a Key
+## v3.0 — execute() Wrapper
+
+The star feature. Wraps the entire lifecycle into one method:
 
 ```typescript
-const key = manager.getKey();
+const manager = new ApiKeyManager(keys, {
+  storage: localStorage,
+  strategy: new WeightedStrategy(),
+  fallbackFn: () => cachedResponse,
+  concurrency: 10
+});
+
+const result = await manager.execute(
+  async (key, signal) => {
+    const res = await fetch(url, { headers: { 'x-api-key': key }, signal });
+    return res.json();
+  },
+  { maxRetries: 3, timeoutMs: 10000 }
+);
+// Handles: key selection → timeout → retry → fallback → latency tracking
+```
 
-if (!key) {
-  throw new Error("All API keys are exhausted or cooling down.");
-}
+## Event Emitter
+
+Monitor every state change:
 
-// Use the key with your API client
-const client = new GoogleGenerativeAI(key);
+```typescript
+manager.on('keyDead', (key) => alertTeam(`Key ${key} permanently dead`));
+manager.on('circuitOpen', (key) => metrics.increment('circuit_opens'));
+manager.on('keyRecovered', (key) => log(`Key ${key} recovered`));
+manager.on('retry', (key, attempt, delay) => log(`Retry #${attempt} in ${delay}ms`));
+manager.on('fallback', (reason) => log(`Fallback triggered: ${reason}`));
+manager.on('allKeysExhausted', () => alert('No healthy keys!'));
+manager.on('bulkheadRejected', () => metrics.increment('rejected'));
+manager.on('healthCheckPassed', (key) => log(`${key} healthy`));
+manager.on('healthCheckFailed', (key, err) => log(`${key} unhealthy`));
 ```
 
-### 3. Report Results (The Feedback Loop)
+## Load Balancing Strategies
 
-Crucial Step: You must report success or failure back to the manager so it can update the circuit state.
+### Weighted (Cost Optimization)
 
 ```typescript
-try {
-  const response = await client.generateContent(prompt);
+import { ApiKeyManager, WeightedStrategy } from '@splashcodex/api-key-manager';
+
+const manager = new ApiKeyManager(
+  [
+    { key: 'free-key-1', weight: 1.0 },
+    { key: 'free-key-2', weight: 1.0 },
+    { key: 'paid-backup', weight: 0.1 },
+  ],
+  { strategy: new WeightedStrategy() }
+);
+```
+
+### Latency (Performance)
+
+```typescript
+import { ApiKeyManager, LatencyStrategy } from '@splashcodex/api-key-manager';
+
+const manager = new ApiKeyManager(keys, { strategy: new LatencyStrategy() });
+// After execute(), latency is tracked automatically
+```
 
-  // ✅ REPORT SUCCESS
-  manager.markSuccess(key);
+## Provider Tagging
 
-  return response;
+Route requests to specific providers:
+
+```typescript
+const manager = new ApiKeyManager([
+  { key: 'sk-openai-1', weight: 1.0, provider: 'openai' },
+  { key: 'sk-openai-2', weight: 1.0, provider: 'openai' },
+  { key: 'AIza-gemini',  weight: 0.5, provider: 'gemini' },
+]);
+
+const openaiKey = manager.getKeyByProvider('openai');
+const geminiKey = manager.getKeyByProvider('gemini');
+```
+
+## Health Checks
+
+Proactively detect recovered keys:
+
+```typescript
+manager.setHealthCheck(async (key) => {
+  const res = await fetch(`https://api.openai.com/v1/models`, {
+    headers: { Authorization: `Bearer ${key}` }
+  });
+  return res.ok;
+});
+
+manager.startHealthChecks(60_000); // Check every 60 seconds
+// manager.stopHealthChecks();     // Stop when done
+```
+
+## Error Handling
+
+```typescript
+try {
+  const result = await callApi(key);
+  manager.markSuccess(key, duration);
 } catch (error) {
-  // ❌ REPORT FAILURE
-  // The manager will automatically classify the error (Quota vs Auth vs Transient)
   const classification = manager.classifyError(error);
   manager.markFailed(key, classification);
 
-  throw error; // Re-throw or handle accordingly
+  if (classification.retryable) {
+    const delay = manager.calculateBackoff(attempt);
+    await sleep(delay);
+  }
 }
 ```
 
-## Error Classification
+## API Reference
+
+### Constructor
+
+```typescript
+// Legacy (v1/v2)
+new ApiKeyManager(keys, storage?, strategy?)
+
+// v3 Options
+new ApiKeyManager(keys, {
+  storage?,      // Pluggable storage { getItem, setItem }
+  strategy?,     // LoadBalancingStrategy instance
+  fallbackFn?,   // () => any — called when all keys exhausted
+  concurrency?,  // Max concurrent execute() calls
+})
+```
 
-The `classifyError` method automatically detects:
--   **429 / Quota**: Marks key as 'OPEN' for a cooldown period (default 5 mins).
--   **403 / Auth**: Marks key as 'DEAD' permanently.
--   **500 / Transient**: Marks key as 'OPEN' for a short cooldown (default 1 min).
--   **FinishReason: SAFETY/RECITATION**: specific to Gemini, does not penalize the key.
+### Methods
+
+| Method | Description |
+|--------|-------------|
+| `getKey()` | Returns best available key via strategy |
+| `getKeyByProvider(provider)` | Get key filtered by provider tag |
+| `markSuccess(key, durationMs?)` | Report success + optional latency |
+| `markFailed(key, classification)` | Report failure with error type |
+| `classifyError(error, finishReason?)` | Classify an error automatically |
+| `execute(fn, options?)` | Full lifecycle wrapper with retry/timeout |
+| `calculateBackoff(attempt)` | Get backoff delay with jitter |
+| `getStats()` | Get pool health statistics |
+| `getKeyCount()` | Count of non-DEAD keys |
+| `setHealthCheck(fn)` | Set health check function |
+| `startHealthChecks(ms)` | Start periodic health checks |
+| `stopHealthChecks()` | Stop health checks |
+
+### Events
+
+| Event | Payload | Trigger |
+|-------|---------|---------|
+| `keyDead` | `key: string` | Key marked as permanently dead |
+| `circuitOpen` | `key: string` | Key circuit opened (cooldown) |
+| `circuitHalfOpen` | `key: string` | Key entering test phase |
+| `keyRecovered` | `key: string` | Key recovered from failure |
+| `fallback` | `reason: string` | Fallback function invoked |
+| `allKeysExhausted` | — | All keys dead, no fallback |
+| `retry` | `key, attempt, delayMs` | Retry attempt starting |
+| `executeSuccess` | `key, durationMs` | execute() completed successfully |
+| `executeFailed` | `key, error` | execute() attempt failed |
+| `bulkheadRejected` | — | Concurrency limit reached |
+| `healthCheckPassed` | `key: string` | Health check succeeded |
+| `healthCheckFailed` | `key, error` | Health check failed |
+
+### Custom Errors
+
+| Error | When |
+|-------|------|
+| `TimeoutError` | Request exceeded `timeoutMs` |
+| `BulkheadRejectionError` | Concurrency limit exceeded |
+| `AllKeysExhaustedError` | All keys dead, no fallback |
+
+### Strategies
+
+| Strategy | Algorithm | Best For |
+|----------|-----------|----------|
+| `StandardStrategy` | Least Failures → LRU | General use |
+| `WeightedStrategy` | Probabilistic by weight | Cost optimization |
+| `LatencyStrategy` | Lowest avg latency | Performance |
 
 ## License
 

package/dist/index.d.ts

@@ -1,8 +1,11 @@
 /**
- * Universal ApiKeyManager v2.0
- * Implements: Rotation, Circuit Breaker, Persistence, Exponential Backoff
+ * Universal ApiKeyManager v3.0
+ * Implements: Rotation, Circuit Breaker, Persistence, Exponential Backoff, Strategies,
+ *             Event Emitter, Fallback, execute(), Timeout, Auto-Retry, Provider Tags,
+ *             Health Checks, Bulkhead/Concurrency
  * Gemini-Specific: finishReason handling, Safety blocks, RECITATION detection
  */
+import { EventEmitter } from 'events';
 export interface KeyState {
     key: string;
     failCount: number;
@@ -14,8 +17,13 @@ export interface KeyState {
     totalRequests: number;
     halfOpenTestTime: number | null;
     customCooldown: number | null;
+    weight: number;
+    averageLatency: number;
+    totalLatency: number;
+    latencySamples: number;
+    provider: string;
 }
-export type ErrorType = 'QUOTA' | 'TRANSIENT' | 'AUTH' | 'BAD_REQUEST' | 'SAFETY' | 'RECITATION' | 'UNKNOWN';
+export type ErrorType = 'QUOTA' | 'TRANSIENT' | 'AUTH' | 'BAD_REQUEST' | 'SAFETY' | 'RECITATION' | 'TIMEOUT' | 'UNKNOWN';
 export interface ErrorClassification {
     type: ErrorType;
     retryable: boolean;
@@ -29,55 +37,138 @@ export interface ApiKeyManagerStats {
     cooling: number;
     dead: number;
 }
-export declare class ApiKeyManager {
+export interface ExecuteOptions {
+    timeoutMs?: number;
+    maxRetries?: number;
+    finishReason?: string;
+}
+export interface ApiKeyManagerOptions {
+    storage?: any;
+    strategy?: LoadBalancingStrategy;
+    fallbackFn?: () => any;
+    concurrency?: number;
+}
+export interface ApiKeyManagerEventMap {
+    keyDead: (key: string) => void;
+    circuitOpen: (key: string) => void;
+    circuitHalfOpen: (key: string) => void;
+    keyRecovered: (key: string) => void;
+    fallback: (reason: string) => void;
+    allKeysExhausted: () => void;
+    retry: (key: string, attempt: number, delayMs: number) => void;
+    healthCheckFailed: (key: string, error: any) => void;
+    healthCheckPassed: (key: string) => void;
+    executeSuccess: (key: string, durationMs: number) => void;
+    executeFailed: (key: string, error: any) => void;
+    bulkheadRejected: () => void;
+}
+export declare class TimeoutError extends Error {
+    constructor(ms: number);
+}
+export declare class BulkheadRejectionError extends Error {
+    constructor();
+}
+export declare class AllKeysExhaustedError extends Error {
+    constructor();
+}
+/**
+ * Strategy Interface for selecting the next key
+ */
+export interface LoadBalancingStrategy {
+    next(candidates: KeyState[]): KeyState | null;
+}
+/**
+ * Standard Strategy: Least Failed > Least Recently Used
+ */
+export declare class StandardStrategy implements LoadBalancingStrategy {
+    next(candidates: KeyState[]): KeyState | null;
+}
+/**
+ * Weighted Strategy: Probabilistic selection based on weight
+ * Higher weight = Higher chance of selection
+ */
+export declare class WeightedStrategy implements LoadBalancingStrategy {
+    next(candidates: KeyState[]): KeyState | null;
+}
+/**
+ * Latency Strategy: Pick lowest average latency
+ */
+export declare class LatencyStrategy implements LoadBalancingStrategy {
+    next(candidates: KeyState[]): KeyState | null;
+}
+export declare class ApiKeyManager extends EventEmitter {
     private keys;
     private storageKey;
     private storage;
-    constructor(initialKeys: string[], storage?: any);
+    private strategy;
+    private fallbackFn?;
+    private maxConcurrency;
+    private activeCalls;
+    private healthCheckFn?;
+    private healthCheckInterval?;
     /**
-     * CLASSIFIES an error to determine handling strategy
+     * Constructor supports both legacy positional args and new options object.
+     *
+     * @example Legacy (v1/v2 — still works):
+     *   new ApiKeyManager(['key1', 'key2'], storage, strategy)
+     *
+     * @example New (v3):
+     *   new ApiKeyManager(keys, { storage, strategy, fallbackFn, concurrency })
      */
-    classifyError(error: any, finishReason?: string): ErrorClassification;
+    constructor(initialKeys: string[] | {
+        key: string;
+        weight?: number;
+        provider?: string;
+    }[], storageOrOptions?: any | ApiKeyManagerOptions, strategy?: LoadBalancingStrategy);
     /**
-     * Parses Retry-After header from error response
+     * CLASSIFIES an error to determine handling strategy
      */
+    classifyError(error: any, finishReason?: string): ErrorClassification;
     private parseRetryAfter;
-    /**
-     * HEALTH CHECK
-     * Determines if a key is usable based on Circuit Breaker logic
-     */
     private isOnCooldown;
-    /**
-     * CORE ROTATION LOGIC
-     * Returns the best available key
-     */
     getKey(): string | null;
     /**
-     * Get count of healthy (non-DEAD) keys
+     * Get a key filtered by provider tag
      */
+    getKeyByProvider(provider: string): string | null;
     getKeyCount(): number;
     /**
-     * FEEDBACK LOOP: Success
+     * Mark success AND update latency stats
+     * @param durationMs Duration of the request in milliseconds
      */
-    markSuccess(key: string): void;
+    markSuccess(key: string, durationMs?: number): void;
+    markFailed(key: string, classification: ErrorClassification): void;
+    markFailedLegacy(key: string, isQuota?: boolean): void;
+    calculateBackoff(attempt: number): number;
+    getStats(): ApiKeyManagerStats;
+    _getKeys(): KeyState[];
     /**
-     * FEEDBACK LOOP: Failure
-     * Enhanced with error classification
+     * Wraps the entire API call lifecycle into a single method.
+     *
+     * @example
+     *   const result = await manager.execute(
+     *     (key) => fetch(`https://api.example.com?key=${key}`),
+     *     { maxRetries: 3, timeoutMs: 5000 }
+     *   );
      */
-    markFailed(key: string, classification: ErrorClassification): void;
+    execute<T>(fn: (key: string, signal?: AbortSignal) => Promise<T>, options?: ExecuteOptions): Promise<T>;
+    private _executeWithRetry;
+    private _executeWithTimeout;
+    private _sleep;
     /**
-     * Legacy markFailed for backward compatibility
+     * Set a health check function that tests if a key is operational
      */
-    markFailedLegacy(key: string, isQuota?: boolean): void;
+    setHealthCheck(fn: (key: string) => Promise<boolean>): void;
     /**
-     * Calculate backoff delay with jitter
+     * Start periodic health checks
+     * @param intervalMs How often to run health checks (default: 60s)
      */
-    calculateBackoff(attempt: number): number;
+    startHealthChecks(intervalMs?: number): void;
     /**
-     * Get health statistics
+     * Stop periodic health checks
      */
-    getStats(): ApiKeyManagerStats;
-    _getKeys(): KeyState[];
+    stopHealthChecks(): void;
+    private _runHealthChecks;
     private saveState;
     private loadState;
 }