@pkent/aigateway 1.0.0 → 1.2.0

package/README.md

@@ -2,8 +2,8 @@
 
 A provider-neutral LLM client library. One class routes chat / vision /
 streaming requests to OpenAI, Anthropic, Qwen (Alibaba DashScope), GLM (Zhipu
-AI), or OpenRouter based on the model name, and normalizes every response into a
-single shape.
+AI), OpenRouter, or an OpenAI-compatible gateway (`aibroker`) based on the model
+name, and normalizes every response into a single shape.
 
 No server, no `.env` — all configuration is passed into the constructor.
 
@@ -49,8 +49,9 @@ new AIGateway(model, key, options?)
 
 | Option      | Type   | Applies to      | Default          | Description |
 |-------------|--------|-----------------|------------------|-------------|
-| `baseURL`   | string | all             | provider default | Override the upstream endpoint (regional endpoints, proxies, self-hosted). |
+| `baseURL`   | string | all             | provider default | Override the upstream endpoint (regional endpoints, proxies, self-hosted). **Required** for `aibroker` (no default). |
 | `maxTokens` | number | all             | unset (see below)| Default max output tokens; a per-call `maxTokens` overrides it. |
+| `timeout`   | number | all             | unset (SDK default) | Default per-call request timeout in ms; a per-call `timeout` overrides it. |
 | `referer`   | string | OpenRouter only | omitted          | Sent as the `HTTP-Referer` attribution header. |
 | `title`     | string | OpenRouter only | omitted          | Sent as the `X-Title` attribution header. |
 | `client`    | object | advanced        | —                | Inject a pre-built SDK client (or a compatible fake for testing). When set, `key`/`baseURL`/`referer`/`title` are not used to build a client. |
@@ -70,7 +71,8 @@ AIGateway.providers()
 //   { id: 'anthropic',  prefix: 'anthropic/' },
 //   { id: 'qwen',       prefix: 'qwen/' },
 //   { id: 'glm',        prefix: 'glm/' },
-//   { id: 'openai',     prefix: 'openai/' }
+//   { id: 'openai',     prefix: 'openai/' },
+//   { id: 'aibroker',   prefix: 'aibroker/' }
 // ]
 ```
 
@@ -81,8 +83,9 @@ AIGateway.providers()
 
 `vision` is `chat` with image content blocks in `messages`. Both return the
 [v2 response shape](#response-shape). `options` may include `temperature`,
-`maxTokens`, and `responseFormat` (mapped to OpenAI-compatible `response_format`;
-ignored by Anthropic).
+`maxTokens`, `responseFormat`, `signal` (an `AbortSignal` to cancel the
+request), and `timeout` (ms). (`responseFormat` is mapped to OpenAI-compatible
+`response_format`; ignored by Anthropic.)
 
 ```js
 const g = new AIGateway('openai/gpt-4o', OPENAI_KEY);
@@ -118,6 +121,9 @@ If the upstream errors, the iterator throws and `.final` rejects with the same
 error. You may await `.final` without iterating (deltas buffer in memory), or
 iterate without awaiting `.final`.
 
+Accepts the same `signal`/`timeout` options; aborting the signal rejects both
+the iterator and `.final`.
+
 ## Model routing
 
 Every model is addressed as `<provider>/<model>`. The leading provider segment
@@ -132,10 +138,25 @@ catch-all — an unknown or missing segment throws `AIGatewayError`
 | `qwen/<model>`                 | `qwen`       | `<model>`      | OpenAI-compatible. Default base URL `https://dashscope-intl.aliyuncs.com/compatible-mode/v1`. |
 | `glm/<model>`                  | `glm`        | `<model>`      | OpenAI-compatible. Default base URL `https://open.bigmodel.cn/api/paas/v4`. |
 | `openrouter/<vendor>/<model>`  | `openrouter` | `<vendor>/<model>` | Only the `openrouter/` segment is stripped, leaving OpenRouter's native id. Default base URL `https://openrouter.ai/api/v1`. |
+| `aibroker/<remainder>`         | `aibroker`   | `<remainder>`  | OpenAI-compatible gateway meta-provider. Only the `aibroker/` segment is stripped; the remainder is forwarded verbatim and the gateway does its own routing. **Requires** `baseURL` (no default) — the constructor throws `AIGatewayError` (`code: 'missing_base_url'`) without it. |
 
 OpenRouter is opt-in: send `openrouter/anthropic/claude-opus-4.8` to route
 through OpenRouter, or `anthropic/claude-opus-4.8` to hit Anthropic directly.
 
+`aibroker` is a gateway meta-provider: it forwards to *any* OpenAI-compatible
+gateway you point it at, so it embeds no host and **requires** a `baseURL`. Only
+the leading `aibroker/` segment is stripped — everything after it is sent
+upstream as the model id, letting the gateway do its own routing
+(`aibroker/openai/chatgpt-5.5` → `openai/chatgpt-5.5`,
+`aibroker/openrouter/openai/chatgpt-5.5` → `openrouter/openai/chatgpt-5.5`). The
+key is passed straight through as the Bearer for the gateway.
+
+```js
+const g = new AIGateway('aibroker/openai/chatgpt-5.5', GATEWAY_TOKEN, {
+  baseURL: 'https://your-gateway/v1',
+});
+```
+
 **Adding a provider:** drop a module into [src/providers/](src/providers/)
 exporting `{ id, prefix, matches, create }` (where `matches` tests the
 `<id>/` segment) and register it in
@@ -206,6 +227,32 @@ behavior:
 Supply `maxTokens` (in the constructor or per call) to apply a cap to every
 provider. A per-call value overrides the constructor default.
 
+## Cancellation & timeouts
+
+Pass an `AbortSignal` as `signal` to cancel an in-flight request; pass
+`timeout` (ms) to bound it. Both are forwarded to the underlying provider
+SDK's request options and apply to `chat`, `vision`, and `stream`.
+
+```js
+const controller = new AbortController();
+const res = g.chat(messages, { signal: controller.signal });
+// ...later:
+controller.abort();   // res rejects with the SDK's APIUserAbortError
+```
+
+For a hard wall-clock cap that also honors an external cancel, combine a
+timeout signal with your own:
+
+```js
+const signal = AbortSignal.any([AbortSignal.timeout(120_000), runSignal]);
+const res = await g.chat(messages, { signal });
+```
+
+The `timeout` option is a per-attempt connection timeout and may be retried
+by the SDK; prefer a combined `signal` (as above) when you need a guaranteed
+wall-clock bound. Aborting a stream rejects both the async iterator and the
+`.final` promise.
+
 ## Errors
 
 - **Input errors** (invalid model/key/messages) throw `AIGatewayError` with a
@@ -214,6 +261,9 @@ provider. A per-call value overrides the constructor default.
   `invalid_message_content`).
 - **Upstream/provider errors** propagate as the underlying SDK error — `chat` /
   `vision` reject; `stream` throws on the iterator and rejects `.final`.
+- An aborted request rejects with the SDK's `APIUserAbortError`; a timed-out
+  request with `APIConnectionTimeoutError`. Like other provider errors, these
+  propagate unchanged (not wrapped in `AIGatewayError`).
 
 ```js
 import AIGateway, { AIGatewayError } from '@pkent/aigateway';

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@pkent/aigateway",
-  "version": "1.0.0",
+  "version": "1.2.0",
   "description": "A provider-neutral LLM client library for OpenAI, Anthropic, Qwen, GLM, and OpenRouter",
   "main": "src/index.js",
   "type": "module",
@@ -9,7 +9,7 @@
   },
   "scripts": {
     "test": "node --test",
-    "prepublishOnly": "node --test test/aigateway.test.js test/cache-hints.test.js test/usage-normalization.test.js"
+    "prepublishOnly": "node --test test/aigateway.test.js test/cache-hints.test.js test/usage-normalization.test.js test/cancellation.test.js test/aibroker.test.js"
   },
   "files": [
     "src",

package/src/AIGateway.js

@@ -27,7 +27,7 @@ export class AIGateway {
       );
     }
 
-    const { baseURL, maxTokens, referer, title, client } = options;
+    const { baseURL, maxTokens, timeout, referer, title, client } = options;
 
     if (!client && (typeof key !== 'string' || key.trim() === '')) {
       throw new AIGatewayError('Missing or invalid API key', { code: 'invalid_api_key' });
@@ -36,7 +36,7 @@ export class AIGateway {
     this.#model = model;
     this.#providerId = entry.id;
     this.#provider = entry.create({ apiKey: key, baseURL, referer, title, client });
-    this.#defaults = { maxTokens };
+    this.#defaults = { maxTokens, timeout };
   }
 
   get model() {
@@ -57,6 +57,8 @@ export class AIGateway {
       maxTokens: callOptions.maxTokens ?? this.#defaults.maxTokens,
       temperature: callOptions.temperature,
       responseFormat: callOptions.responseFormat,
+      signal: callOptions.signal,
+      timeout: callOptions.timeout ?? this.#defaults.timeout,
     };
   }
 

package/src/providers/aibroker.js

@@ -0,0 +1,22 @@
+import { createOpenAICompatibleProvider } from './openaiCompatible.js';
+import { AIGatewayError } from '../errors.js';
+
+// "aibroker/<provider>/<model>" routes to an OpenAI-compatible gateway. The
+// "aibroker/" segment is stripped (existing stripProviderPrefix) and the rest is
+// forwarded verbatim as the model — the gateway does its own routing. The
+// caller MUST supply the gateway URL via options.baseURL; there is no default.
+export default {
+  id: 'aibroker',
+  prefix: 'aibroker/',
+  matches: model => typeof model === 'string' && model.startsWith('aibroker/'),
+  create({ apiKey, baseURL, client }) {
+    if (!client && !baseURL) {
+      throw new AIGatewayError(
+        'The "aibroker" provider requires a baseURL (the gateway URL), e.g. ' +
+          'new AIGateway(model, key, { baseURL: "https://your-gateway/v1" })',
+        { code: 'missing_base_url' },
+      );
+    }
+    return createOpenAICompatibleProvider({ id: 'aibroker', apiKey, baseURL, client });
+  },
+};

package/src/providers/anthropic.js

@@ -1,6 +1,7 @@
 import Anthropic from '@anthropic-ai/sdk';
 import { convertMessagesForAnthropic } from './messageTransforms.js';
 import { stripProviderPrefix } from './modelId.js';
+import { buildRequestOptions } from './requestOptions.js';
 import {
   buildV2Response,
   extractAnthropicUsage,
@@ -40,15 +41,18 @@ function createProvider({ apiKey, baseURL, client }) {
     id: 'anthropic',
 
     async chat(model, messages, options = {}) {
-      return normalize(await anthropic.messages.create(buildRequest(model, messages, options)), model);
+      return normalize(await anthropic.messages.create(buildRequest(model, messages, options), buildRequestOptions(options)), model);
     },
 
     async vision(model, messages, options = {}) {
-      return normalize(await anthropic.messages.create(buildRequest(model, messages, options)), model);
+      return normalize(await anthropic.messages.create(buildRequest(model, messages, options), buildRequestOptions(options)), model);
     },
 
     async stream(model, messages, options = {}, { emitDelta }) {
-      const upstream = await anthropic.messages.create({ ...buildRequest(model, messages, options), stream: true });
+      const upstream = await anthropic.messages.create(
+        { ...buildRequest(model, messages, options), stream: true },
+        buildRequestOptions(options),
+      );
 
       let streamId = `anthropic_${Date.now()}`;
       const created = Math.floor(Date.now() / 1000);

package/src/providers/openaiCompatible.js

@@ -1,6 +1,7 @@
 import OpenAI from 'openai';
 import { stripCacheHintsFromMessages } from './messageTransforms.js';
 import { stripProviderPrefix } from './modelId.js';
+import { buildRequestOptions } from './requestOptions.js';
 import {
   buildV2Response,
   extractOpenAIUsage,
@@ -53,15 +54,24 @@ export function createOpenAICompatibleProvider({
     id,
 
     async chat(model, messages, options = {}) {
-      return normalize(await openai.chat.completions.create(buildRequest(model, messages, options)), model);
+      return normalize(
+        await openai.chat.completions.create(buildRequest(model, messages, options), buildRequestOptions(options)),
+        model,
+      );
     },
 
     async vision(model, messages, options = {}) {
-      return normalize(await openai.chat.completions.create(buildRequest(model, messages, options)), model);
+      return normalize(
+        await openai.chat.completions.create(buildRequest(model, messages, options), buildRequestOptions(options)),
+        model,
+      );
     },
 
     async stream(model, messages, options = {}, { emitDelta }) {
-      const upstream = await openai.chat.completions.create(buildRequest(model, messages, options, { stream: true }));
+      const upstream = await openai.chat.completions.create(
+        buildRequest(model, messages, options, { stream: true }),
+        buildRequestOptions(options),
+      );
 
       let streamId = `${id}_${Date.now()}`;
       let created = Math.floor(Date.now() / 1000);

package/src/providers/registry.js

@@ -3,10 +3,11 @@ import anthropic from './anthropic.js';
 import qwen from './qwen.js';
 import glm from './glm.js';
 import openai from './openai.js';
+import aibroker from './aibroker.js';
 
 // Every model is addressed as "<provider>/<model>". The provider segments are
 // disjoint, so resolution order does not matter; there is no catch-all.
-const ENTRIES = [openrouter, anthropic, qwen, glm, openai];
+const ENTRIES = [openrouter, anthropic, qwen, glm, openai, aibroker];
 
 export function resolveProviderEntry(model) {
   for (const entry of ENTRIES) {

package/src/providers/requestOptions.js

@@ -0,0 +1,14 @@
+// Maps the provider-neutral transport options onto the SDK's per-call
+// "request options" object — the SECOND argument to anthropic.messages.create /
+// openai.chat.completions.create. Both SDKs accept { signal, timeout } there.
+//
+// These are transport concerns and must NEVER be placed in the request body.
+// Each field is included only when set (mirroring the conditional-spread style
+// used for body fields), so an absent signal/timeout leaves SDK defaults intact
+// and an omitted-options call sends an empty {} — a no-op for both SDKs.
+export function buildRequestOptions({ signal, timeout } = {}) {
+  return {
+    ...(signal != null && { signal }),
+    ...(timeout != null && { timeout }),
+  };
+}