ai-resilience-gateway 1.0.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Vasil Tomov
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,205 @@
+# ai-resilience-gateway
+
+Resilient LLM chat completions with **provider fallback**, **retries**, and **timeouts**.
+Zero runtime dependencies — a TypeScript library for learning production LLM client patterns and showcasing them in a portfolio.
+
+When your primary provider rate-limits you or goes down, the gateway falls through to the next one so a single hiccup doesn’t take down your feature.
+
+```
+Groq ──fails──▶ Ollama (local)
+  │                  │
+ retries+backoff  retries+backoff
+```
+
+Presets also include `openrouter()` if you set `OPENROUTER_API_KEY`. You can order any providers you want, for example `[groq(), openrouter(), ollama()]`.
+
+## Why
+
+Most “call an LLM” code is a single `fetch` with no retry, no timeout, and no plan B:
+
+- **429** during traffic spikes → the feature just errors out
+- a **provider outage** → downtime for anything LLM-backed
+- a **hung request** with no timeout → work piles up until the process struggles
+
+This library handles all three:
+
+- exponential backoff on retryable failures (`429` / `5xx` / network / timeout)
+- a hard per-request timeout via `AbortController`
+- ordered fallback across any number of OpenAI-compatible providers
+
+## Requirements
+
+- **Node 18+** (uses the global `fetch` API)
+
+## Setup
+
+```sh
+git clone https://github.com/vtomov90/AI-resilience-gateway.git
+cd AI-resilience-gateway
+npm install
+```
+
+## Usage
+
+Install the package:
+
+```sh
+npm install ai-resilience-gateway
+```
+
+
+Then use it:
+
+
+```typescript
+import { LLMCascade, groq, ollama } from "ai-resilience-gateway";
+
+
+const llm = new LLMCascade([groq(), ollama()], {
+    retries: 2,
+    timeoutMs: 30_000,
+    onFallback: (provider, err) =>
+        console.warn(`[gateway] ${provider} failed:`, err.message),
+});
+
+const result = await llm.complete([
+  { role: "system", content: "You are a concise assistant." },
+  { role: "user", content: "Explain retries in one sentence." },
+]);
+
+
+console.log(result.text);       // model reply
+console.log(result.provider);   // "groq" or "ollama" — who answered
+console.log(result.latencyMs);  // e.g. 266
+```
+
+
+### Provider presets
+
+Built-in factories (they just return a `Provider` object):
+
+| Preset | Default model | API key |
+|--------|---------------|---------|
+| `groq()` | `openai/gpt-oss-20b` | `GROQ_API_KEY` |
+| `openrouter()` | `meta-llama/llama-3.3-70b-instruct` | `OPENROUTER_API_KEY` (optional) |
+| `ollama()` | `llama3.1` | none (local) |
+
+Override model or key when needed:
+
+```typescript
+groq("openai/gpt-oss-20b", process.env.GROQ_API_KEY);
+openrouter("meta-llama/llama-3.3-70b-instruct", process.env.OPENROUTER_API_KEY);
+ollama("llama3.1", "http://localhost:11434");
+```
+
+### Any OpenAI-compatible endpoint
+
+```typescript
+const custom = {
+  name: "together",
+  url: "https://api.together.xyz/v1/chat/completions",
+  model: "some-model-id",
+  apiKey: process.env.TOGETHER_API_KEY,
+};
+
+const llm = new LLMCascade([custom, ollama()]);
+```
+
+## API
+
+### `new LLMCascade(providers, options?)`
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `timeoutMs` | `30000` | Per-request timeout; abort is treated as retryable |
+| `retries` | `2` | Attempts per provider (`1` = no retry) |
+| `backoffMs` | `500` | Base backoff; doubles each retry (`500` → `1000` → …) |
+| `onFallback` | — | `(provider, error) => void` when moving past a provider |
+
+### `cascade.complete(messages, options?)`
+
+`options`: `{ temperature?, maxTokens?, signal? }`
+
+Returns:
+
+```typescript
+{
+  text: string;
+  provider: string;
+  model: string;
+  latencyMs: number;
+  usage?: { promptTokens?: number; completionTokens?: number };
+}
+```
+
+Throws `AllProvidersFailedError` (with a `.causes` map of `provider → Error`) only when every provider is exhausted.
+
+## Retry policy
+
+**Retryable** (same provider, with backoff):
+
+- HTTP `408`, `429`, `500`, `502`, `503`, `504`
+- network errors and timeouts
+
+**Non-retryable** (fall through to the next provider immediately):
+
+- typical `4xx` like `401`, `403`, `404` — no point hammering a bad API key or unknown URL
+
+## Example CLI
+
+Create a local `.env` (do **not** commit it):
+
+```env
+GROQ_API_KEY=gsk_your_key_here
+OPENROUTER_API_KEY=
+OLLAMA_MODEL=llama3.1
+OLLAMA_BASE_URL=http://localhost:11434
+```
+
+
+
+Run:
+
+```sh
+npm run example -- "Say hello in one sentence."
+```
+
+The example tries Groq first (if `GROQ_API_KEY` is set), then Ollama as a local fallback.
+
+> Groq model ids change over time and can depend on your account plan. If you get `model_not_found`, list models for your key (`GET https://api.groq.com/openai/v1/models`) and pass an available chat model into `groq("your-model-id")`.
+
+## Development
+
+```sh
+npm test          # Vitest — fetch fully mocked, no network / no API keys
+npm run test:watch
+npm run typecheck
+npm run build
+```
+
+### Project layout
+
+```
+src/
+  types.ts       # ChatMessage, Provider, options, result types
+  errors.ts      # HttpError, AllProvidersFailedError
+  retry.ts       # isRetryable, backoff helpers
+  cascade.ts     # LLMCascade (request → retries → fallback)
+  providers.ts   # groq / openrouter / ollama presets
+  index.ts       # public exports
+examples/
+  ask.ts         # live demo CLI
+tests/           # unit tests with mocked fetch
+```
+
+## What this project demonstrates
+
+- OpenAI-compatible chat completions (`choices[0].message.content`)
+- `fetch` + `AbortController` for timeouts and external abort
+- Retry vs fallback as separate layers
+- Typed custom errors for debugging multi-provider failure
+- Vitest with mocked `fetch` and fake timers
+
+## License
+
+MIT

package/dist/cascade.d.ts

@@ -0,0 +1,10 @@
+import type { CascadeOptions, ChatMessage, CompletionOptions, CompletionResult, Provider } from "./types.js";
+export declare class LLMCascade {
+    private readonly providers;
+    private readonly opts;
+    constructor(providers: Provider[], options?: CascadeOptions);
+    complete(messages: ChatMessage[], options?: CompletionOptions): Promise<CompletionResult>;
+    private completeWithRetries;
+    private request;
+}
+//# sourceMappingURL=cascade.d.ts.map

package/dist/cascade.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"cascade.d.ts","sourceRoot":"","sources":["../src/cascade.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,EACV,cAAc,EACd,WAAW,EACX,iBAAiB,EACjB,gBAAgB,EAChB,QAAQ,EACT,MAAM,YAAY,CAAC;AAEpB,qBAAa,UAAU;IACnB,OAAO,CAAC,QAAQ,CAAC,SAAS,CAAa;IACvC,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAoF;gBAE7F,SAAS,EAAE,QAAQ,EAAE,EAAE,OAAO,GAAE,cAAmB;IAczD,QAAQ,CACV,QAAQ,EAAE,WAAW,EAAE,EACvB,OAAO,GAAE,iBAAsB,GAChC,OAAO,CAAE,gBAAgB,CAAC;YAgBf,mBAAmB;YAsBnB,OAAO;CAqExB"}

package/dist/cascade.js

@@ -0,0 +1,95 @@
+import { AllProvidersFailedError, HttpError } from "./errors.js";
+import { isRetryable, sleep } from "./retry.js";
+export class LLMCascade {
+    providers;
+    opts;
+    constructor(providers, options = {}) {
+        if (providers.length === 0) {
+            throw new Error("LLMCascade requires at least one provider");
+        }
+        this.providers = providers;
+        this.opts = {
+            timeoutMs: options.timeoutMs ?? 30_000,
+            retries: options.retries ?? 2,
+            backoffMs: options.backoffMs ?? 500,
+            onFallback: options.onFallback,
+        };
+    }
+    async complete(messages, options = {}) {
+        const causes = {};
+        for (const provider of this.providers) {
+            try {
+                return await this.completeWithRetries(provider, messages, options);
+            }
+            catch (err) {
+                const error = err instanceof Error ? err : new Error(String(err));
+                causes[provider.name] = error;
+                this.opts.onFallback?.(provider.name, error);
+            }
+        }
+        throw new AllProvidersFailedError(causes);
+    }
+    async completeWithRetries(provider, messages, options) {
+        let lastError = new Error("unreachable");
+        for (let attempt = 0; attempt < this.opts.retries; attempt++) {
+            if (attempt > 0) {
+                await sleep(this.opts.backoffMs * 2 ** (attempt - 1));
+            }
+            try {
+                return await this.request(provider, messages, options);
+            }
+            catch (err) {
+                lastError = err instanceof Error ? err : new Error(String(err));
+                if (!isRetryable(lastError))
+                    break;
+            }
+        }
+        throw lastError;
+    }
+    async request(provider, messages, options) {
+        const controller = new AbortController();
+        const timer = setTimeout(() => controller.abort(new Error(`timeout after ${this.opts.timeoutMs}ms`)), this.opts.timeoutMs);
+        options.signal?.addEventListener("abort", () => controller.abort(options.signal.reason), { once: true });
+        const started = Date.now();
+        try {
+            const res = await fetch(provider.url, {
+                method: "POST",
+                signal: controller.signal,
+                headers: {
+                    "Content-Type": "application/json",
+                    ...(provider.apiKey ? { Authorization: `Bearer ${provider.apiKey}` } : {}),
+                    ...provider.headers,
+                },
+                body: JSON.stringify({
+                    model: provider.model,
+                    messages,
+                    temperature: options.temperature ?? 0.7,
+                    ...(options.maxTokens ? { max_tokens: options.maxTokens } : {})
+                }),
+            });
+            if (!res.ok) {
+                const body = await res.text().catch(() => "");
+                throw new HttpError(res.status, `${provider.name} HTTP ${res.status}: ${body.slice(0, 200)}`);
+            }
+            const data = (await res.json());
+            const text = data.choices?.[0]?.message?.content;
+            if (typeof text !== "string") {
+                throw new HttpError(502, `${provider.name} returned no completion text`);
+            }
+            return {
+                text,
+                provider: provider.name,
+                model: provider.model,
+                latencyMs: Date.now() - started,
+                usage: {
+                    promptTokens: data.usage?.prompt_tokens,
+                    completionTokens: data.usage?.completion_tokens,
+                }
+            };
+        }
+        finally {
+            clearTimeout(timer);
+        }
+    }
+}
+//# sourceMappingURL=cascade.js.map

package/dist/cascade.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"cascade.js","sourceRoot":"","sources":["../src/cascade.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,uBAAuB,EAAE,SAAS,EAAE,MAAM,aAAa,CAAC;AACjE,OAAO,EAAE,WAAW,EAAE,KAAK,EAAE,MAAM,YAAY,CAAC;AAShD,MAAM,OAAO,UAAU;IACF,SAAS,CAAa;IACtB,IAAI,CAAoF;IAEzG,YAAY,SAAqB,EAAE,UAA0B,EAAE;QAC3D,IAAI,SAAS,CAAC,MAAM,KAAK,CAAC,EAAC,CAAC;YACxB,MAAM,IAAI,KAAK,CAAC,2CAA2C,CAAC,CAAC;QACjE,CAAC;QAED,IAAI,CAAC,SAAS,GAAG,SAAS,CAAA;QAC1B,IAAI,CAAC,IAAI,GAAG;YACR,SAAS,EAAE,OAAO,CAAC,SAAS,IAAI,MAAM;YACtC,OAAO,EAAE,OAAO,CAAC,OAAO,IAAI,CAAC;YAC7B,SAAS,EAAE,OAAO,CAAC,SAAS,IAAI,GAAG;YACnC,UAAU,EAAE,OAAO,CAAC,UAAU;SACjC,CAAC;IACN,CAAC;IAED,KAAK,CAAC,QAAQ,CACV,QAAuB,EACvB,UAA6B,EAAE;QAE/B,MAAM,MAAM,GAA0B,EAAE,CAAC;QAEzC,KAAK,MAAM,QAAQ,IAAI,IAAI,CAAC,SAAS,EAAE,CAAC;YACpC,IAAG,CAAC;gBACA,OAAO,MAAM,IAAI,CAAC,mBAAmB,CAAC,QAAQ,EAAE,QAAQ,EAAE,OAAO,CAAC,CAAC;YACvE,CAAC;YAAA,OAAO,GAAG,EAAC,CAAC;gBACT,MAAM,KAAK,GAAG,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,KAAK,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC;gBAClE,MAAM,CAAC,QAAQ,CAAC,IAAI,CAAC,GAAG,KAAK,CAAC;gBAC9B,IAAI,CAAC,IAAI,CAAC,UAAU,EAAE,CAAC,QAAQ,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC;YACjD,CAAC;QACL,CAAC;QAED,MAAM,IAAI,uBAAuB,CAAC,MAAM,CAAC,CAAC;IAC9C,CAAC;IAEO,KAAK,CAAC,mBAAmB,CAC7B,QAAkB,EAClB,QAAuB,EACvB,OAA0B;QAE1B,IAAI,SAAS,GAAU,IAAI,KAAK,CAAC,aAAa,CAAC,CAAA;QAE/C,KAAK,IAAI,OAAO,GAAG,CAAC,EAAE,OAAO,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,EAAE,OAAO,EAAE,EAAC,CAAC;YAC1D,IAAI,OAAO,GAAG,CAAC,EAAC,CAAC;gBACb,MAAM,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,SAAS,GAAG,CAAC,IAAI,CAAC,OAAO,GAAG,CAAC,CAAC,CAAC,CAAC;YAC1D,CAAC;YACD,IAAG,CAAC;gBACA,OAAO,MAAM,IAAI,CAAC,OAAO,CAAC,QAAQ,EAAE,QAAQ,EAAE,OAAO,CAAC,CAAC;YAC3D,CAAC;YAAA,OAAM,GAAG,EAAC,CAAC;gBACR,SAAS,GAAG,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,KAAK,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC;gBAChE,IAAI,CAAC,WAAW,CAAC,SAAS,CAAC;oBAAE,MAAM;YACvC,CAAC;QACL,CAAC;QAED,MAAM,SAAS,CAAC;IACpB,CAAC;IAEO,KAAK,CAAC,OAAO,CACjB,QAAkB,EAClB,QAAuB,EACvB,OAA0B;QAE1B,MAAM,UAAU,GAAG,IAAI,eAAe,EAAE,CAAC;QACzC,MAAM,KAAK,GAAG,UAAU,CAAC,GAAG,EAAE,CAAC,UAAU,CAAC,KAAK,CAAC,IAAI,KAAK,CAAC,iBAAiB,IAAI,CAAC,IAAI,CAAC,SAAS,IAAI,CAAC,CAAC,EAAE,IAAI,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;QAE3H,OAAO,CAAC,MAAM,EAAE,gBAAgB,CAC5B,OAAO,EACP,GAAG,EAAE,CAAC,UAAU,CAAC,KAAK,CAAC,OAAO,CAAC,MAAO,CAAC,MAAM,CAAC,EAC9C,EAAE,IAAI,EAAE,IAAI,EAAE,CACjB,CAAC;QAGF,MAAM,OAAO,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;QAC3B,IAAG,CAAC;YACA,MAAM,GAAG,GAAG,MAAM,KAAK,CAAE,QAAQ,CAAC,GAAG,EAAC;gBAClC,MAAM,EAAE,MAAM;gBACd,MAAM,EAAE,UAAU,CAAC,MAAM;gBACzB,OAAO,EAAE;oBACL,cAAc,EAAE,kBAAkB;oBAClC,GAAG,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC,CAAC,EAAG,aAAa,EAAE,UAAU,QAAQ,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBAC3E,GAAG,QAAQ,CAAC,OAAO;iBACtB;gBACD,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC;oBACjB,KAAK,EAAE,QAAQ,CAAC,KAAK;oBACrB,QAAQ;oBACR,WAAW,EAAE,OAAO,CAAC,WAAW,IAAI,GAAG;oBACvC,GAAG,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC,CAAC,EAAE,UAAU,EAAE,OAAO,CAAC,SAAS,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;iBAClE,CAAC;aACL,CAAC,CAAC;YAEH,IAAG,CAAC,GAAG,CAAC,EAAE,EAAC,CAAC;gBACR,MAAM,IAAI,GAAG,MAAM,GAAG,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,EAAE,CAAC,CAAC;gBAC9C,MAAM,IAAI,SAAS,CACf,GAAG,CAAC,MAAM,EACV,GAAG,QAAQ,CAAC,IAAI,SAAS,GAAG,CAAC,MAAM,KAAK,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE,CAC/D,CAAA;YACL,CAAC;YAED,MAAM,IAAI,GAAG,CAAC,MAAM,GAAG,CAAC,IAAI,EAAE,CAG7B,CAAA;YAGD,MAAM,IAAI,GAAG,IAAI,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,OAAO,CAAC;YACjD,IAAI,OAAO,IAAI,KAAK,QAAQ,EAAC,CAAC;gBAC1B,MAAM,IAAI,SAAS,CAAC,GAAG,EAAE,GAAG,QAAQ,CAAC,IAAI,8BAA8B,CAAC,CAAC;YAC7E,CAAC;YAED,OAAO;gBACH,IAAI;gBACJ,QAAQ,EAAE,QAAQ,CAAC,IAAI;gBACvB,KAAK,EAAE,QAAQ,CAAC,KAAK;gBACrB,SAAS,EAAE,IAAI,CAAC,GAAG,EAAE,GAAG,OAAO;gBAC/B,KAAK,EAAE;oBACH,YAAY,EAAE,IAAI,CAAC,KAAK,EAAE,aAAa;oBACvC,gBAAgB,EAAE,IAAI,CAAC,KAAK,EAAE,iBAAiB;iBAClD;aACJ,CAAA;QACL,CAAC;gBAAQ,CAAC;YACN,YAAY,CAAC,KAAK,CAAC,CAAA;QACvB,CAAC;IAEL,CAAC;CAGJ"}

package/dist/errors.d.ts

@@ -0,0 +1,9 @@
+export declare class HttpError extends Error {
+    readonly status: number;
+    constructor(status: number, message: string);
+}
+export declare class AllProvidersFailedError extends Error {
+    readonly causes: Record<string, Error>;
+    constructor(causes: Record<string, Error>);
+}
+//# sourceMappingURL=errors.d.ts.map

package/dist/errors.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"errors.d.ts","sourceRoot":"","sources":["../src/errors.ts"],"names":[],"mappings":"AAAA,qBAAa,SAAU,SAAQ,KAAK;aAEZ,MAAM,EAAE,MAAM;gBAAd,MAAM,EAAE,MAAM,EAC9B,OAAO,EAAC,MAAM;CAKrB;AAGD,qBAAa,uBAAwB,SAAQ,KAAK;aAE1B,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,KAAK,CAAC;gBAA7B,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,KAAK,CAAC;CAUpD"}

package/dist/errors.js

@@ -0,0 +1,19 @@
+export class HttpError extends Error {
+    status;
+    constructor(status, message) {
+        super(message);
+        this.status = status;
+        this.name = "HttpError";
+    }
+}
+export class AllProvidersFailedError extends Error {
+    causes;
+    constructor(causes) {
+        super("All LLM providers failed: " +
+            Object.entries(causes)
+                .map(([name, err]) => `${name} (${err.message})`));
+        this.causes = causes;
+        this.name = "AllProvidersFailedError";
+    }
+}
+//# sourceMappingURL=errors.js.map

package/dist/errors.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"errors.js","sourceRoot":"","sources":["../src/errors.ts"],"names":[],"mappings":"AAAA,MAAM,OAAO,SAAU,SAAQ,KAAK;IAEZ;IADpB,YACoB,MAAc,EAC9B,OAAc;QAEd,KAAK,CAAC,OAAO,CAAC,CAAC;QAHC,WAAM,GAAN,MAAM,CAAQ;QAI9B,IAAI,CAAC,IAAI,GAAG,WAAW,CAAC;IAC5B,CAAC;CACJ;AAGD,MAAM,OAAO,uBAAwB,SAAQ,KAAK;IAE1B;IADpB,YACoB,MAA6B;QAE7C,KAAK,CACD,4BAA4B;YACxB,MAAM,CAAC,OAAO,CAAC,MAAM,CAAC;iBACjB,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE,GAAG,CAAC,EAAE,EAAE,CAAC,GAAG,IAAI,KAAK,GAAG,CAAC,OAAO,GAAG,CAAC,CAC5D,CAAC;QANc,WAAM,GAAN,MAAM,CAAuB;QAQ7C,IAAI,CAAC,IAAI,GAAG,yBAAyB,CAAC;IAC1C,CAAC;CACJ"}

package/dist/index.d.ts

@@ -0,0 +1,6 @@
+export { LLMCascade } from "./cascade.js";
+export { AllProvidersFailedError, HttpError } from "./errors.js";
+export { groq, ollama, openrouter } from "./providers.js";
+export { isRetryable, RETRYABLE_STATUS, sleep } from "./retry.js";
+export type { CascadeOptions, ChatMessage, CompletionOptions, CompletionResult, Provider, } from "./types.js";
+//# sourceMappingURL=index.d.ts.map

package/dist/index.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,MAAM,cAAc,CAAC;AAC1C,OAAO,EAAE,uBAAuB,EAAE,SAAS,EAAE,MAAM,aAAa,CAAC;AACjE,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,UAAU,EAAE,MAAM,gBAAgB,CAAC;AAC1D,OAAO,EAAE,WAAW,EAAE,gBAAgB,EAAE,KAAK,EAAE,MAAM,YAAY,CAAC;AAClE,YAAY,EACV,cAAc,EACd,WAAW,EACX,iBAAiB,EACjB,gBAAgB,EAChB,QAAQ,GACT,MAAM,YAAY,CAAC"}

package/dist/index.js

@@ -0,0 +1,5 @@
+export { LLMCascade } from "./cascade.js";
+export { AllProvidersFailedError, HttpError } from "./errors.js";
+export { groq, ollama, openrouter } from "./providers.js";
+export { isRetryable, RETRYABLE_STATUS, sleep } from "./retry.js";
+//# sourceMappingURL=index.js.map

package/dist/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,MAAM,cAAc,CAAC;AAC1C,OAAO,EAAE,uBAAuB,EAAE,SAAS,EAAE,MAAM,aAAa,CAAC;AACjE,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,UAAU,EAAE,MAAM,gBAAgB,CAAC;AAC1D,OAAO,EAAE,WAAW,EAAE,gBAAgB,EAAE,KAAK,EAAE,MAAM,YAAY,CAAC"}

package/dist/providers.d.ts

@@ -0,0 +1,5 @@
+import type { Provider } from "./types.js";
+export declare const groq: (model?: string, apiKey?: string | undefined) => Provider;
+export declare const openrouter: (model?: string, apiKey?: string | undefined) => Provider;
+export declare const ollama: (model?: string, baseUrl?: string) => Provider;
+//# sourceMappingURL=providers.d.ts.map

package/dist/providers.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"providers.d.ts","sourceRoot":"","sources":["../src/providers.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,YAAY,CAAC;AAE3C,eAAO,MAAM,IAAI,GACb,cAA4B,EAC5B,2BAAiC,KAClC,QAKD,CAAA;AAEF,eAAO,MAAM,UAAU,GACnB,cAA2C,EAC3C,2BAAuC,KACxC,QASD,CAAA;AAEF,eAAO,MAAM,MAAM,GACf,cAAkB,EAClB,gBAAkC,KACnC,QAID,CAAA"}

package/dist/providers.js

@@ -0,0 +1,22 @@
+export const groq = (model = "openai/gpt-oss-20b", apiKey = process.env.GROQ_API_KEY) => ({
+    name: "groq",
+    url: "https://api.groq.com/openai/v1/chat/completions",
+    model,
+    apiKey
+});
+export const openrouter = (model = "meta-llama/llama-3.3-70b-instruct", apiKey = process.env.OPENROUTER_API_KEY) => ({
+    name: "openrouter",
+    url: "https://openrouter.ai/api/v1/chat/completions",
+    model,
+    apiKey,
+    headers: {
+        "HTTP-Referer": "https://github.com/ai-resilience-gateway",
+        "X-Title": "ai-resilience-gateway",
+    }
+});
+export const ollama = (model = "llama3.1", baseUrl = "http://localhost:11434") => ({
+    name: "ollama",
+    url: `${baseUrl}/v1/chat/completions`,
+    model,
+});
+//# sourceMappingURL=providers.js.map

package/dist/providers.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"providers.js","sourceRoot":"","sources":["../src/providers.ts"],"names":[],"mappings":"AAEA,MAAM,CAAC,MAAM,IAAI,GAAG,CAChB,KAAK,GAAG,oBAAoB,EAC5B,MAAM,GAAG,OAAO,CAAC,GAAG,CAAC,YAAY,EACzB,EAAE,CAAC,CAAC;IACZ,IAAI,EAAE,MAAM;IACZ,GAAG,EAAG,iDAAiD;IACvD,KAAK;IACL,MAAM;CACT,CAAC,CAAA;AAEF,MAAM,CAAC,MAAM,UAAU,GAAG,CACtB,KAAK,GAAG,mCAAmC,EAC3C,MAAM,GAAG,OAAO,CAAC,GAAG,CAAC,kBAAkB,EAC/B,EAAE,CAAC,CAAC;IACZ,IAAI,EAAG,YAAY;IACnB,GAAG,EAAE,+CAA+C;IACpD,KAAK;IACL,MAAM;IACN,OAAO,EAAE;QACL,cAAc,EAAE,0CAA0C;QAC1D,SAAS,EAAE,uBAAuB;KACrC;CACJ,CAAC,CAAA;AAEF,MAAM,CAAC,MAAM,MAAM,GAAG,CAClB,KAAK,GAAG,UAAU,EAClB,OAAO,GAAG,wBAAwB,EAC1B,EAAE,CAAC,CAAC;IACZ,IAAI,EAAE,QAAQ;IACd,GAAG,EAAE,GAAG,OAAO,sBAAsB;IACrC,KAAK;CACR,CAAC,CAAA"}

package/dist/retry.d.ts

@@ -0,0 +1,28 @@
+/**
+ * A set of HTTP status codes that indicate temporary or transient failures.
+ *
+ * If a provider fails with one of these statuses, it means the issue might resolve
+ * if we retry the request (e.g., rate limits cooling down or server recovering).
+ * Non-listed statuses (like 401 Unauthorized or 404 Not Found) are permanent and
+ * should fail immediately without retrying on the same provider.
+ */
+export declare const RETRYABLE_STATUS: Set<number>;
+/**
+ * Pauses execution for a specified duration.
+ *
+ * Uses a Promise wrapping `setTimeout` (forming a closure over `ms`) so it can
+ * be paused using `await`.
+ *
+ * @param ms - The number of milliseconds to pause execution.
+ * @returns A Promise that resolves after the specified delay.
+ 
+*/
+export declare const sleep: (ms: number) => Promise<unknown>;
+/**
+ * Determines whether a failed request should be retried on the same provider.
+ *
+ * @param err - The Error caught during a provider request attempt.
+ * @returns `true` if the failure is temporary and safe to retry; `false` otherwise.
+ */
+export declare function isRetryable(err: Error): boolean;
+//# sourceMappingURL=retry.d.ts.map

package/dist/retry.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"retry.d.ts","sourceRoot":"","sources":["../src/retry.ts"],"names":[],"mappings":"AAGA;;;;;;;GAOG;AAEH,eAAO,MAAM,gBAAgB,aAA2C,CAAC;AAIzE;;;;;;;;;EASE;AAEF,eAAO,MAAM,KAAK,GAAI,IAAI,MAAM,qBAA0C,CAAC;AAG3E;;;;;GAKG;AAEH,wBAAgB,WAAW,CAAC,GAAG,EAAE,KAAK,GAAG,OAAO,CAG/C"}

package/dist/retry.js

@@ -0,0 +1,33 @@
+import { HttpError } from "./errors.js";
+/**
+ * A set of HTTP status codes that indicate temporary or transient failures.
+ *
+ * If a provider fails with one of these statuses, it means the issue might resolve
+ * if we retry the request (e.g., rate limits cooling down or server recovering).
+ * Non-listed statuses (like 401 Unauthorized or 404 Not Found) are permanent and
+ * should fail immediately without retrying on the same provider.
+ */
+export const RETRYABLE_STATUS = new Set([408, 429, 500, 502, 503, 504]);
+/**
+ * Pauses execution for a specified duration.
+ *
+ * Uses a Promise wrapping `setTimeout` (forming a closure over `ms`) so it can
+ * be paused using `await`.
+ *
+ * @param ms - The number of milliseconds to pause execution.
+ * @returns A Promise that resolves after the specified delay.
+ 
+*/
+export const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
+/**
+ * Determines whether a failed request should be retried on the same provider.
+ *
+ * @param err - The Error caught during a provider request attempt.
+ * @returns `true` if the failure is temporary and safe to retry; `false` otherwise.
+ */
+export function isRetryable(err) {
+    if (err instanceof HttpError)
+        return RETRYABLE_STATUS.has(err.status);
+    return true;
+}
+//# sourceMappingURL=retry.js.map

package/dist/retry.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"retry.js","sourceRoot":"","sources":["../src/retry.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,SAAS,EAAE,MAAM,aAAa,CAAC;AAGxC;;;;;;;GAOG;AAEH,MAAM,CAAC,MAAM,gBAAgB,GAAG,IAAI,GAAG,CAAE,CAAC,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,CAAC,CAAC,CAAC;AAIzE;;;;;;;;;EASE;AAEF,MAAM,CAAC,MAAM,KAAK,GAAG,CAAC,EAAU,EAAE,EAAE,CAAC,IAAI,OAAO,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,UAAU,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC;AAG3E;;;;;GAKG;AAEH,MAAM,UAAU,WAAW,CAAC,GAAU;IAClC,IAAI,GAAG,YAAY,SAAS;QAAE,OAAO,gBAAgB,CAAC,GAAG,CAAC,GAAG,CAAC,MAAM,CAAC,CAAA;IACrE,OAAO,IAAI,CAAC;AAChB,CAAC"}

package/dist/types.d.ts

@@ -0,0 +1,33 @@
+export interface ChatMessage {
+    role: "system" | "user" | "assistant";
+    content: string;
+}
+export interface Provider {
+    name: string;
+    url: string;
+    model: string;
+    apiKey?: string;
+    headers?: Record<string, string>;
+}
+export interface CascadeOptions {
+    timeoutMs?: number;
+    retries?: number;
+    backoffMs?: number;
+    onFallback?: (provider: string, error: Error) => void;
+}
+export interface CompletionOptions {
+    temperature?: number;
+    maxTokens?: number;
+    signal?: AbortSignal;
+}
+export interface CompletionResult {
+    text: string;
+    provider: string;
+    model: string;
+    latencyMs: number;
+    usage?: {
+        promptTokens?: number;
+        completionTokens?: number;
+    };
+}
+//# sourceMappingURL=types.d.ts.map

package/dist/types.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"types.d.ts","sourceRoot":"","sources":["../src/types.ts"],"names":[],"mappings":"AAAA,MAAM,WAAW,WAAW;IACxB,IAAI,EAAE,QAAQ,GAAG,MAAM,GAAG,WAAW,CAAC;IACtC,OAAO,EAAE,MAAM,CAAC;CACnB;AAED,MAAM,WAAW,QAAQ;IACrB,IAAI,EAAE,MAAM,CAAC;IACb,GAAG,EAAE,MAAM,CAAC;IACZ,KAAK,EAAE,MAAM,CAAC;IACd,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,OAAO,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;CACpC;AAED,MAAM,WAAW,cAAc;IAC3B,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,UAAU,CAAC,EAAE,CAAC,QAAQ,EAAE,MAAM,EAAE,KAAK,EAAE,KAAK,KAAK,IAAI,CAAC;CACzD;AAED,MAAM,WAAW,iBAAiB;IAChC,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,MAAM,CAAC,EAAE,WAAW,CAAC;CACtB;AAED,MAAM,WAAW,gBAAgB;IAC7B,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,MAAM,CAAC;IACjB,KAAK,EAAE,MAAM,CAAC;IACd,SAAS,EAAE,MAAM,CAAC;IAClB,KAAK,CAAC,EAAE;QAAE,YAAY,CAAC,EAAE,MAAM,CAAC;QAAC,gBAAgB,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;CAEhE"}

package/dist/types.js

@@ -0,0 +1,2 @@
+export {};
+//# sourceMappingURL=types.js.map

package/dist/types.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"types.js","sourceRoot":"","sources":["../src/types.ts"],"names":[],"mappings":""}

package/package.json

@@ -0,0 +1,56 @@
+{
+  "name": "ai-resilience-gateway",
+  "version": "1.0.0",
+  "description": "Resilient LLM chat completions with provider fallback, retries, and timeouts. Zero runtime dependencies.",
+  "type": "module",
+  "main": "./dist/index.js",
+  "types": "./dist/index.d.ts",
+  "exports": {
+    ".": {
+      "types": "./dist/index.d.ts",
+      "import": "./dist/index.js"
+    }
+  },
+  "files": [
+    "dist"
+  ],
+  "scripts": {
+    "build": "tsc -p tsconfig.build.json",
+    "typecheck": "tsc --noEmit -p tsconfig.json",
+    "test": "vitest run",
+    "test:watch": "vitest",
+    "example": "tsx examples/ask.ts",
+    "prepublishOnly": "npm test && npm run build"
+  },
+  "engines": {
+    "node": ">=18"
+  },
+  "keywords": [
+    "llm",
+    "openai-compatible",
+    "groq",
+    "openrouter",
+    "ollama",
+    "fallback",
+    "retry",
+    "typescript"
+  ],
+  "author": "Vasil Tomov",
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/vtomov90/AI-resilience-gateway.git"
+  },
+  "bugs": {
+    "url": "https://github.com/vtomov90/AI-resilience-gateway/issues"
+  },
+  "homepage": "https://github.com/vtomov90/AI-resilience-gateway#readme",
+
+  "devDependencies": {
+    "@types/node": "^26.6.1",
+    "dotenv": "^17.4.2",
+    "tsx": "^4.23.13",
+    "typescript": "^5.7.2",
+    "vitest": "^4.1.11"
+  }
+}