@imenam/unified-llm-client 2.2.0 → 2.3.1

package/README.md

@@ -1,359 +1,383 @@
-# @imenam/unified-llm-client
-
-A unified interface for multiple LLM providers — OpenAI, Gemini, Anthropic, and Qwen — with built-in support for tool-calling, streaming, multimodal inputs, audio generation, and a proxy mode for API key isolation.
-
-## Features
-
-- **Unified API** — one interface, four providers
-- **Tool-calling** — automatic execution loop with structured results
-- **Streaming** — chunk-by-chunk responses via callbacks
-- **Multimodal** — image and audio file inputs
-- **Audio generation** — Text-to-Speech (Gemini)
-- **Proxy mode** — delegate LLM calls to a server, no API keys on the client
-- **API key resolver** — dynamically resolve API keys per request from a user-provided context
-- **Cancellation** — native `AbortSignal` support
-- **Usage tracking** — token counts and estimated costs
-- **Full TypeScript support** — exported types and `.d.ts` declarations
-
-## Installation
-
-```bash
-npm install @imenam/unified-llm-client
-```
-
-## Supported Providers
-
-| Provider  | Chat | Streaming | Tool-calling | Multimodal | Audio TTS |
-|-----------|:----:|:---------:|:------------:|:----------:|:---------:|
-| OpenAI    | ✅   | ✅        | ✅           | ✅         | ❌        |
-| Gemini    | ✅   | ✅        | ✅           | ✅         | ✅        |
-| Anthropic | ✅   | ✅        | ✅           | ✅         | ❌        |
-| Qwen      | ✅   | ✅        | ✅           | ✅         | ❌        |
-
-## Quick Start
-
-Set your API keys in a `.env` file:
-
-```env
-OPENAI_API_KEY=sk-...
-GOOGLE_API_KEY=...
-ANTHROPIC_API_KEY=sk-ant-...
-ALIBABA_CLOUD_API_KEY=...   # for Qwen
-```
-
-```typescript
-import { UnifiedLLMClient, MODELS } from "@imenam/unified-llm-client";
-
-const client = new UnifiedLLMClient(MODELS.GPT_4O_MINI);
-
-const response = await client.chat({
-  message: "What is the capital of France?",
-});
-
-console.log(response.content); // "The capital of France is Paris."
-```
-
-## Available Models
-
-```typescript
-import { MODELS } from "@imenam/unified-llm-client";
-
-// OpenAI
-MODELS.GPT_4O_MINI
-MODELS.GPT_5
-
-// Gemini
-MODELS.GEMINI_FLASH_2_5
-MODELS.GEMINI_2_5_PRO
-MODELS.GEMINI_3_FLASH
-MODELS.GEMINI_3_PRO
-MODELS.GEMINI_3_1_PRO
-
-// Anthropic
-MODELS.CLAUDE_SONNET_4_6
-MODELS.CLAUDE_OPUS_4_6
-
-// Qwen
-MODELS.QWEN_VL_3
-MODELS.QWEN_VL_FLASH
-MODELS.QWEN_VL_3_FLASH
-```
-
-You can also pass a custom model:
-
-```typescript
-const client = new UnifiedLLMClient({
-  provider: "openai",
-  modelName: "gpt-4o",
-  costsPer1MTokens: { prompt: 2.5, output: 10 },
-  currency: "USD",
-});
-```
-
-## Tool-calling
-
-Define tools with an `executor` function — the client will call them automatically and loop until the LLM produces a final text response.
-
-```typescript
-import { UnifiedLLMClient, MODELS } from "@imenam/unified-llm-client";
-import type { ToolDefinition } from "@imenam/unified-llm-client";
-
-const tools: ToolDefinition[] = [
-  {
-    name: "get_weather",
-    description: "Get the current weather for a city",
-    parameters: {
-      type: "object",
-      properties: {
-        city: { type: "string", description: "City name" },
-      },
-      required: ["city"],
-    },
-    executor: async ({ city }) => {
-      return `The weather in ${city} is sunny and 22°C.`;
-    },
-  },
-];
-
-const client = new UnifiedLLMClient(MODELS.GEMINI_FLASH_2_5);
-
-const response = await client.chat({
-  message: "What's the weather like in Paris?",
-  tools,
-});
-
-console.log(response.content);
-```
-
-## Streaming
-
-```typescript
-const client = new UnifiedLLMClient(MODELS.GPT_4O_MINI);
-
-await client.chat({
-  message: "Tell me a short story.",
-  stream: true,
-  onChunk: (chunk) => {
-    process.stdout.write(chunk.contentChunk ?? "");
-  },
-  onFinalResponse: (final) => {
-    console.log("\n\nTotal tokens:", final.usage?.totalTokens);
-  },
-});
-```
-
-## Multimodal Inputs
-
-Attach images or audio files directly to a message:
-
-```typescript
-const response = await client.chat({
-  message: {
-    role: "user",
-    content: "What do you see in this image?",
-    attachedFile: "./photo.jpg",
-  },
-});
-```
-
-Multiple files are also supported:
-
-```typescript
-await client.chat({
-  message: {
-    role: "user",
-    content: "Compare these two images.",
-    attachedFile: ["./image1.jpg", "./image2.png"],
-  },
-});
-```
-
-## Audio Generation (Gemini)
-
-```typescript
-import { UnifiedLLMClient, MODELS } from "@imenam/unified-llm-client";
-
-const client = new UnifiedLLMClient(MODELS.GEMINI_FLASH_2_5);
-
-const audio = await client.generateAudio({
-  text: "Hello, this is a test of text-to-speech.",
-  outputPath: "./output.wav",
-  voiceConfig: { voiceName: "Kore" },
-});
-```
-
-## Proxy Mode
-
-Run an HTTP server that holds your API keys server-side, and connect clients without any keys:
-
-```typescript
-import { ProxyServer } from "@imenam/unified-llm-client";
-
-// Server side — API keys are read from process.env
-const server = new ProxyServer({ port: 3000, apiKey: "my-secret" });
-await server.start();
-```
-
-```typescript
-// Client side — no API keys needed
-import { UnifiedLLMClient, MODELS } from "@imenam/unified-llm-client";
-
-const client = new UnifiedLLMClient(MODELS.GEMINI_FLASH_2_5, {
-  proxyUrl: "http://localhost:3000",
-  proxyApiKey: "my-secret",
-});
-
-// The chat() API is identical — streaming and file attachments are fully supported
-const response = await client.chat({ message: "Hello!" });
-```
-
-The proxy server also supports an `apiKeyResolver` for per-request dynamic key resolution (e.g. multi-tenant scenarios):
-
-```typescript
-import { ProxyServer } from "@imenam/unified-llm-client";
-import type { ApiKeyResolver } from "@imenam/unified-llm-client";
-
-const resolver: ApiKeyResolver<{ userId: string }> = async (ctx) =>
-  (await userKeyStore.get(ctx.userId)) ?? false;
-
-const server = new ProxyServer({ port: 3000, apiKey: "my-secret", apiKeyResolver: resolver });
-await server.start();
-```
-
-```typescript
-// Client passes the context — server resolves the key
-const client = new UnifiedLLMClient(MODELS.GPT_4O_MINI, {
-  proxyUrl: "http://localhost:3000",
-  proxyApiKey: "my-secret",
-});
-
-const response = await client.chat({
-  message: "Hello!",
-  apiKeyContext: { userId: "user-123" },
-});
-```
-
-## API Key Resolver
-
-Dynamically resolve which API key to use for each request based on a custom context (e.g. per-user key, request metadata):
-
-```typescript
-import { UnifiedLLMClient, MODELS } from "@imenam/unified-llm-client";
-import type { ApiKeyResolver } from "@imenam/unified-llm-client";
-
-const resolver: ApiKeyResolver<{ userId: string }> = async (ctx) => {
-  const key = await userKeyStore.get(ctx.userId);
-  return key ?? false; // return false to deny access
-};
-
-const client = new UnifiedLLMClient(MODELS.GPT_4O_MINI, {
-  apiKeyResolver: resolver,
-});
-
-const response = await client.chat({
-  message: "Hello!",
-  apiKeyContext: { userId: "user-123" },
-});
-```
-
-You can also set a global default resolver for all instances:
-
-```typescript
-UnifiedLLMClient.setDefaultApiKeyResolver((ctx) => {
-  return ctx.apiKey ?? false;
-});
-```
-
-> The resolver is called on every `chat()` call. Returning `false` throws an access-denied error.
-> The client-side `apiKeyResolver` option is for direct mode only. In proxy mode, the resolver is defined on the `ProxyServer` — the client passes `apiKeyContext` and the server resolves the key.
-
-## Cancellation
-
-Pass an `AbortSignal` to cancel a request mid-flight:
-
-```typescript
-const ac = new AbortController();
-setTimeout(() => ac.abort(), 800);
-
-try {
-  await client.chat({
-    message: "Write a very long story...",
-    stream: true,
-    signal: ac.signal,
-    onChunk: (chunk) => process.stdout.write(chunk.contentChunk ?? ""),
-  });
-} catch (e) {
-  if (e instanceof Error && e.name === "AbortError") {
-    console.log("Request cancelled.");
-  }
-}
-```
-
-## Logging
-
-Pass a `logger` callback to capture structured logs for every request:
-
-```typescript
-const client = new UnifiedLLMClient(MODELS.GPT_4O_MINI, {
-  logger: (log) => {
-    console.log(`[${log.provider}] ${log.model} — ${log.usage.totalTokens} tokens`);
-  },
-});
-```
-
-Set a global default logger applied to all instances:
-
-```typescript
-UnifiedLLMClient.setDefaultLogger((log) => {
-  console.log(`[${log.provider}] ${log.model} — ${log.usage.totalTokens} tokens`);
-});
-```
-
-## API Reference
-
-### `new UnifiedLLMClient(model, options?)`
-
-| Parameter | Type | Description |
-|-----------|------|-------------|
-| `model` | `Model` | Model object from `MODELS` or a custom `{ provider, modelName, costsPer1MTokens, currency }` |
-| `options.logger` | `(log: UnifiedLog) => void` | Optional logging callback |
-| `options.proxyUrl` | `string` | Proxy server URL — activates proxy mode |
-| `options.proxyApiKey` | `string` | Key sent in the `x-api-key` header (proxy mode) |
-| `options.apiKey` | `string` | Static API key passed directly to the provider (overrides env var) |
-| `options.apiKeyResolver` | `ApiKeyResolver` | Dynamic per-request key resolver (direct mode only — in proxy mode, set this on `ProxyServer`) |
-
-### `client.chat(options)`
-
-| Option | Type | Default | Description |
-|--------|------|---------|-------------|
-| `message` | `string \| ChatMessage \| ChatMessage[]` | — | User message or full conversation history |
-| `tools` | `ToolDefinition[]` | `[]` | Tools available to the LLM |
-| `toolChoice` | `"auto" \| "none" \| string` | `"auto"` | Tool selection strategy |
-| `stream` | `boolean` | `false` | Enable streaming |
-| `onChunk` | `(chunk: StreamedChatChunk) => void` | — | Streaming chunk callback |
-| `onFinalResponse` | `(res: ChatResponse) => void` | — | Final response callback |
-| `onLog` | `(log: UnifiedLog) => void` | — | Per-call log callback |
-| `config` | `ChatOptionalConfig` | — | Advanced options: `thinkingBudget`, `metadata` |
-| `signal` | `AbortSignal` | — | Cancellation signal |
-| `apiKeyContext` | `unknown` | — | Context passed to `apiKeyResolver` to resolve the API key |
-
-### `ChatResponse`
-
-```typescript
-interface ChatResponse {
-  content: string | null;
-  toolCalls?: ToolCall[];
-  usage?: {
-    promptTokens: number;
-    completionTokens: number;
-    totalTokens: number;
-  };
-  finishReason?: string;
-  requestId?: string;
-  raw: unknown;
-}
-```
-
-## License
-
-MIT
+# @imenam/unified-llm-client
+
+A unified interface for multiple LLM providers — OpenAI, Gemini, Anthropic, and Qwen — with built-in support for tool-calling, streaming, multimodal inputs, audio generation, and a proxy mode for API key isolation.
+
+## Features
+
+- **Unified API** — one interface, four providers
+- **Tool-calling** — automatic execution loop with structured results
+- **Streaming** — chunk-by-chunk responses via callbacks
+- **Multimodal** — image and audio file inputs
+- **Audio generation** — Text-to-Speech (Gemini)
+- **Proxy mode** — delegate LLM calls to a server, no API keys on the client
+- **API key resolver** — dynamically resolve API keys per request from a user-provided context
+- **Cancellation** — native `AbortSignal` support
+- **Usage tracking** — token counts and estimated costs
+- **Full TypeScript support** — exported types and `.d.ts` declarations
+
+## Installation
+
+```bash
+npm install @imenam/unified-llm-client
+```
+
+## Supported Providers
+
+| Provider  | Chat | Streaming | Tool-calling | Multimodal | Audio TTS |
+|-----------|:----:|:---------:|:------------:|:----------:|:---------:|
+| OpenAI    | ✅   | ✅        | ✅           | ✅         | ❌        |
+| Gemini    | ✅   | ✅        | ✅           | ✅         | ✅        |
+| Anthropic | ✅   | ✅        | ✅           | ✅         | ❌        |
+| Qwen      | ✅   | ✅        | ✅           | ✅         | ❌        |
+| OpenRouter | ✅  | ✅        | ✅           | ✅         | ❌        |
+
+## Quick Start
+
+Set your API keys in a `.env` file:
+
+```env
+OPENAI_API_KEY=sk-...
+GOOGLE_API_KEY=...
+ANTHROPIC_API_KEY=sk-ant-...
+ALIBABA_CLOUD_API_KEY=...   # for Qwen
+OPEN_ROUTER_API_KEY=sk-or-... # for OpenRouter (OPENROUTER_API_KEY is also read)
+```
+
+```typescript
+import { UnifiedLLMClient, MODELS } from "@imenam/unified-llm-client";
+
+const client = new UnifiedLLMClient(MODELS.GPT_4O_MINI);
+
+const response = await client.chat({
+  message: "What is the capital of France?",
+});
+
+console.log(response.content); // "The capital of France is Paris."
+```
+
+## Available Models
+
+```typescript
+import { MODELS } from "@imenam/unified-llm-client";
+
+// OpenAI
+MODELS.GPT_4O_MINI
+MODELS.GPT_5
+
+// Gemini
+MODELS.GEMINI_FLASH_2_5
+MODELS.GEMINI_2_5_PRO
+MODELS.GEMINI_3_FLASH
+MODELS.GEMINI_3_PRO
+MODELS.GEMINI_3_1_PRO
+
+// Anthropic
+MODELS.CLAUDE_SONNET_4_6
+MODELS.CLAUDE_OPUS_4_6
+
+// Qwen
+MODELS.QWEN_VL_3
+MODELS.QWEN_VL_FLASH
+MODELS.QWEN_VL_3_FLASH
+```
+
+You can also pass a custom model:
+
+```typescript
+const client = new UnifiedLLMClient({
+  provider: "openai",
+  modelName: "gpt-4o",
+  costsPer1MTokens: { prompt: 2.5, output: 10 },
+  currency: "USD",
+});
+```
+
+## OpenRouter
+
+OpenRouter gives access to every model of its catalogue with a single key. Build the
+model from its OpenRouter id:
+
+```typescript
+import { UnifiedLLMClient, openRouterModel } from "@imenam/unified-llm-client";
+
+const client = new UnifiedLLMClient(openRouterModel("google/gemini-3-flash-preview"));
+```
+
+- Images are sent as `image_url`, PDFs as `file` (the model reads them natively when it can),
+  `.mp3` / `.wav` as `input_audio`.
+- `response.usage.cost` holds the real cost of the call in USD, as reported by OpenRouter.
+- `config.thinkingBudget` (> 0) is sent as `reasoning.max_tokens`.
+- A failed generation throws an error that contains the model's native finish reason,
+  e.g. `OpenRouter: generation failed (MALFORMED_FUNCTION_CALL)`.
+- Multi-turn tool calls: copy `response.reasoningDetails` onto the assistant message you
+  send back (`{ role: "assistant", tool_calls, reasoningDetails }`). Reasoning models such as
+  Gemini 3 reject the next turn otherwise.
+- Optional headers: `OPENROUTER_APP_URL` (`HTTP-Referer`) and `OPENROUTER_APP_NAME` (`X-Title`).
+
+## Tool-calling
+
+Define tools with an `executor` function — the client will call them automatically and loop until the LLM produces a final text response.
+
+```typescript
+import { UnifiedLLMClient, MODELS } from "@imenam/unified-llm-client";
+import type { ToolDefinition } from "@imenam/unified-llm-client";
+
+const tools: ToolDefinition[] = [
+  {
+    name: "get_weather",
+    description: "Get the current weather for a city",
+    parameters: {
+      type: "object",
+      properties: {
+        city: { type: "string", description: "City name" },
+      },
+      required: ["city"],
+    },
+    executor: async ({ city }) => {
+      return `The weather in ${city} is sunny and 22°C.`;
+    },
+  },
+];
+
+const client = new UnifiedLLMClient(MODELS.GEMINI_FLASH_2_5);
+
+const response = await client.chat({
+  message: "What's the weather like in Paris?",
+  tools,
+});
+
+console.log(response.content);
+```
+
+## Streaming
+
+```typescript
+const client = new UnifiedLLMClient(MODELS.GPT_4O_MINI);
+
+await client.chat({
+  message: "Tell me a short story.",
+  stream: true,
+  onChunk: (chunk) => {
+    process.stdout.write(chunk.contentChunk ?? "");
+  },
+  onFinalResponse: (final) => {
+    console.log("\n\nTotal tokens:", final.usage?.totalTokens);
+  },
+});
+```
+
+## Multimodal Inputs
+
+Attach images or audio files directly to a message:
+
+```typescript
+const response = await client.chat({
+  message: {
+    role: "user",
+    content: "What do you see in this image?",
+    attachedFile: "./photo.jpg",
+  },
+});
+```
+
+Multiple files are also supported:
+
+```typescript
+await client.chat({
+  message: {
+    role: "user",
+    content: "Compare these two images.",
+    attachedFile: ["./image1.jpg", "./image2.png"],
+  },
+});
+```
+
+## Audio Generation (Gemini)
+
+```typescript
+import { UnifiedLLMClient, MODELS } from "@imenam/unified-llm-client";
+
+const client = new UnifiedLLMClient(MODELS.GEMINI_FLASH_2_5);
+
+const audio = await client.generateAudio({
+  text: "Hello, this is a test of text-to-speech.",
+  outputPath: "./output.wav",
+  voiceConfig: { voiceName: "Kore" },
+});
+```
+
+## Proxy Mode
+
+Run an HTTP server that holds your API keys server-side, and connect clients without any keys:
+
+```typescript
+import { ProxyServer } from "@imenam/unified-llm-client";
+
+// Server side — API keys are read from process.env
+const server = new ProxyServer({ port: 3000, apiKey: "my-secret" });
+await server.start();
+```
+
+```typescript
+// Client side — no API keys needed
+import { UnifiedLLMClient, MODELS } from "@imenam/unified-llm-client";
+
+const client = new UnifiedLLMClient(MODELS.GEMINI_FLASH_2_5, {
+  proxyUrl: "http://localhost:3000",
+  proxyApiKey: "my-secret",
+});
+
+// The chat() API is identical — streaming and file attachments are fully supported
+const response = await client.chat({ message: "Hello!" });
+```
+
+The proxy server also supports an `apiKeyResolver` for per-request dynamic key resolution (e.g. multi-tenant scenarios):
+
+```typescript
+import { ProxyServer } from "@imenam/unified-llm-client";
+import type { ApiKeyResolver } from "@imenam/unified-llm-client";
+
+const resolver: ApiKeyResolver<{ userId: string }> = async (ctx) =>
+  (await userKeyStore.get(ctx.userId)) ?? false;
+
+const server = new ProxyServer({ port: 3000, apiKey: "my-secret", apiKeyResolver: resolver });
+await server.start();
+```
+
+```typescript
+// Client passes the context — server resolves the key
+const client = new UnifiedLLMClient(MODELS.GPT_4O_MINI, {
+  proxyUrl: "http://localhost:3000",
+  proxyApiKey: "my-secret",
+});
+
+const response = await client.chat({
+  message: "Hello!",
+  apiKeyContext: { userId: "user-123" },
+});
+```
+
+## API Key Resolver
+
+Dynamically resolve which API key to use for each request based on a custom context (e.g. per-user key, request metadata):
+
+```typescript
+import { UnifiedLLMClient, MODELS } from "@imenam/unified-llm-client";
+import type { ApiKeyResolver } from "@imenam/unified-llm-client";
+
+const resolver: ApiKeyResolver<{ userId: string }> = async (ctx) => {
+  const key = await userKeyStore.get(ctx.userId);
+  return key ?? false; // return false to deny access
+};
+
+const client = new UnifiedLLMClient(MODELS.GPT_4O_MINI, {
+  apiKeyResolver: resolver,
+});
+
+const response = await client.chat({
+  message: "Hello!",
+  apiKeyContext: { userId: "user-123" },
+});
+```
+
+You can also set a global default resolver for all instances:
+
+```typescript
+UnifiedLLMClient.setDefaultApiKeyResolver((ctx) => {
+  return ctx.apiKey ?? false;
+});
+```
+
+> The resolver is called on every `chat()` call. Returning `false` throws an access-denied error.
+> The client-side `apiKeyResolver` option is for direct mode only. In proxy mode, the resolver is defined on the `ProxyServer` — the client passes `apiKeyContext` and the server resolves the key.
+
+## Cancellation
+
+Pass an `AbortSignal` to cancel a request mid-flight:
+
+```typescript
+const ac = new AbortController();
+setTimeout(() => ac.abort(), 800);
+
+try {
+  await client.chat({
+    message: "Write a very long story...",
+    stream: true,
+    signal: ac.signal,
+    onChunk: (chunk) => process.stdout.write(chunk.contentChunk ?? ""),
+  });
+} catch (e) {
+  if (e instanceof Error && e.name === "AbortError") {
+    console.log("Request cancelled.");
+  }
+}
+```
+
+## Logging
+
+Pass a `logger` callback to capture structured logs for every request:
+
+```typescript
+const client = new UnifiedLLMClient(MODELS.GPT_4O_MINI, {
+  logger: (log) => {
+    console.log(`[${log.provider}] ${log.model} — ${log.usage.totalTokens} tokens`);
+  },
+});
+```
+
+Set a global default logger applied to all instances:
+
+```typescript
+UnifiedLLMClient.setDefaultLogger((log) => {
+  console.log(`[${log.provider}] ${log.model} — ${log.usage.totalTokens} tokens`);
+});
+```
+
+## API Reference
+
+### `new UnifiedLLMClient(model, options?)`
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `model` | `Model` | Model object from `MODELS` or a custom `{ provider, modelName, costsPer1MTokens, currency }` |
+| `options.logger` | `(log: UnifiedLog) => void` | Optional logging callback |
+| `options.proxyUrl` | `string` | Proxy server URL — activates proxy mode |
+| `options.proxyApiKey` | `string` | Key sent in the `x-api-key` header (proxy mode) |
+| `options.apiKey` | `string` | Static API key passed directly to the provider (overrides env var) |
+| `options.apiKeyResolver` | `ApiKeyResolver` | Dynamic per-request key resolver (direct mode only — in proxy mode, set this on `ProxyServer`) |
+
+### `client.chat(options)`
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `message` | `string \| ChatMessage \| ChatMessage[]` | — | User message or full conversation history |
+| `tools` | `ToolDefinition[]` | `[]` | Tools available to the LLM |
+| `toolChoice` | `"auto" \| "none" \| string` | `"auto"` | Tool selection strategy |
+| `stream` | `boolean` | `false` | Enable streaming |
+| `onChunk` | `(chunk: StreamedChatChunk) => void` | — | Streaming chunk callback |
+| `onFinalResponse` | `(res: ChatResponse) => void` | — | Final response callback |
+| `onLog` | `(log: UnifiedLog) => void` | — | Per-call log callback |
+| `config` | `ChatOptionalConfig` | — | Advanced options: `thinkingBudget`, `metadata` |
+| `signal` | `AbortSignal` | — | Cancellation signal |
+| `apiKeyContext` | `unknown` | — | Context passed to `apiKeyResolver` to resolve the API key |
+
+### `ChatResponse`
+
+```typescript
+interface ChatResponse {
+  content: string | null;
+  toolCalls?: ToolCall[];
+  usage?: {
+    promptTokens: number;
+    completionTokens: number;
+    totalTokens: number;
+  };
+  finishReason?: string;
+  requestId?: string;
+  raw: unknown;
+}
+```
+
+## License
+
+MIT