@x12i/ai-providers-router 4.8.5 → 4.8.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +558 -317
- package/dist/factory.js +14 -29
- package/dist/index.d.ts +2 -2
- package/dist/index.js +2 -2
- package/dist/logger.d.ts +41 -48
- package/dist/logger.js +112 -149
- package/dist/router/Router.d.ts +3 -1
- package/dist/router/Router.js +37 -40
- package/dist/router/RouterTypes.d.ts +15 -2
- package/dist/router/RouterWrapper.d.ts +1 -0
- package/dist/router/RouterWrapper.js +25 -8
- package/dist/utils/openrouterEnv.d.ts +18 -0
- package/dist/utils/openrouterEnv.js +28 -0
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -1,24 +1,50 @@
|
|
|
1
1
|
# @x12i/ai-providers-router
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Unified **LLM provider router** for Node.js. Routes requests to installed provider packages using the **ProviderModule** architecture from [`@x12i/ai-provider-interface`](https://www.npmjs.com/package/@x12i/ai-provider-interface).
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
- **OpenRouter Mode**: Access 353+ models from 67 providers using catalog-driven routing
|
|
7
|
-
- Chooses a provider/model (and optionally a fallback chain)
|
|
8
|
-
- Loads ProviderModules from installed provider packages (lazy import)
|
|
9
|
-
- Uses router-side adapters to convert requests to ProviderSDKCallSpec
|
|
10
|
-
- Executes via ProviderModule.execute() / stream() / submitBatch()
|
|
11
|
-
- Parses responses using router-side adapters
|
|
12
|
-
- Returns standardized responses with lossless rawResponse
|
|
5
|
+
**Highlights**
|
|
13
6
|
|
|
14
|
-
|
|
7
|
+
- **Multi-provider routing** — OpenAI, Grok, and more via lazy-loaded provider packages
|
|
8
|
+
- **OpenRouter mode** — Access 350+ models from 60+ providers through one API key
|
|
9
|
+
- **Sync, stream, and batch** — Gated by each provider's declared capabilities
|
|
10
|
+
- **Fallback chains** — Automatic provider/model failover with full attempt traces
|
|
11
|
+
- **Structured diagnostics** — Usage, cost, timing, and ordered `metadata.attempts[]`
|
|
12
|
+
- **Reasoning support** — Cross-vendor effort, visibility, and encrypted trace handling
|
|
13
|
+
- **ERC 2.0** — Zero-config initialization from environment variables
|
|
14
|
+
- **Structured logging** — Powered by [`@x12i/logxer`](https://www.npmjs.com/package/@x12i/logxer)
|
|
15
|
+
|
|
16
|
+
> This router **never installs** provider packages at runtime. You must install the packages you intend to use.
|
|
15
17
|
|
|
16
|
-
|
|
17
|
-
- **Router Adapters**: Router-side adapters convert router requests to ProviderSDKCallSpec and parse responses
|
|
18
|
-
- **Capability Gating**: Router gates execution by `provider.capabilities.modes.sync/stream/batch` (ProviderModule is source of truth)
|
|
19
|
-
- **Execution Semantics**: Router owns execution semantics (timeoutMs, retries, idempotencyKey, signal)
|
|
18
|
+
---
|
|
20
19
|
|
|
21
|
-
|
|
20
|
+
## Table of contents
|
|
21
|
+
|
|
22
|
+
- [Install](#install)
|
|
23
|
+
- [Quick start](#quick-start)
|
|
24
|
+
- [Architecture](#architecture)
|
|
25
|
+
- [Provider IDs](#provider-ids)
|
|
26
|
+
- [OpenRouter mode](#openrouter-mode)
|
|
27
|
+
- [Configuration](#configuration)
|
|
28
|
+
- [Logging](#logging)
|
|
29
|
+
- [API usage](#api-usage)
|
|
30
|
+
- [Sync](#sync-call)
|
|
31
|
+
- [Streaming](#streaming-call)
|
|
32
|
+
- [Batch](#batch-requests)
|
|
33
|
+
- [Request and response types](#request-and-response-types)
|
|
34
|
+
- [Trace diagnostics](#trace-diagnostics)
|
|
35
|
+
- [Reasoning](#reasoning)
|
|
36
|
+
- [Fallback chains](#fallback-chains)
|
|
37
|
+
- [Interceptors](#interceptors)
|
|
38
|
+
- [Health checks](#health-checks)
|
|
39
|
+
- [AIGateway](#aigateway)
|
|
40
|
+
- [Response normalization and cost](#response-normalization-and-cost)
|
|
41
|
+
- [Error types](#error-types)
|
|
42
|
+
- [Manual setup (advanced)](#manual-setup-advanced)
|
|
43
|
+
- [Public API exports](#public-api-exports)
|
|
44
|
+
- [Provider packages](#provider-packages)
|
|
45
|
+
- [Development and testing](#development-and-testing)
|
|
46
|
+
- [Related documentation](#related-documentation)
|
|
47
|
+
- [License](#license)
|
|
22
48
|
|
|
23
49
|
---
|
|
24
50
|
|
|
@@ -28,470 +54,685 @@ This router:
|
|
|
28
54
|
npm i @x12i/ai-providers-router
|
|
29
55
|
```
|
|
30
56
|
|
|
31
|
-
|
|
57
|
+
**Bundled provider packages** (included as dependencies):
|
|
58
|
+
|
|
59
|
+
- `@x12i/ai-provider-openai` — OpenAI and OpenRouter-compatible APIs
|
|
60
|
+
- `@x12i/ai-provider-grok` — Grok / xAI
|
|
61
|
+
|
|
62
|
+
**Optional provider packages** (install when you need direct access):
|
|
32
63
|
|
|
33
64
|
```bash
|
|
34
|
-
npm i @x12i/ai-provider-openai
|
|
35
65
|
npm i @x12i/ai-provider-anthropic
|
|
36
66
|
npm i @x12i/ai-provider-google
|
|
37
|
-
npm i @x12i/ai-provider-xai
|
|
38
67
|
npm i @x12i/ai-provider-groq
|
|
68
|
+
# ... other @x12i/ai-provider-* packages
|
|
39
69
|
```
|
|
40
70
|
|
|
41
|
-
|
|
71
|
+
For **OpenRouter mode**, only `@x12i/ai-provider-openai` is required to reach models from many vendors through OpenRouter's unified API.
|
|
42
72
|
|
|
43
73
|
---
|
|
44
74
|
|
|
45
|
-
##
|
|
75
|
+
## Quick start
|
|
46
76
|
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
* `anthropic` → Claude
|
|
50
|
-
* `google` → Gemini
|
|
51
|
-
* `xai` → Grok (xAI)
|
|
52
|
-
* `groq` → GroqCloud (Llama/Mixtral/OSS models)
|
|
53
|
-
* `kimi` → Moonshot/Kimi (if installed)
|
|
77
|
+
```ts
|
|
78
|
+
import { createRouter, type AIRouterRequest, type AIResponse } from '@x12i/ai-providers-router';
|
|
54
79
|
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
80
|
+
const router = await createRouter();
|
|
81
|
+
|
|
82
|
+
const req: AIRouterRequest = {
|
|
83
|
+
request: {
|
|
84
|
+
messages: [{ role: 'user', content: 'Write 3 bullets about routers.' }],
|
|
85
|
+
config: { model: 'gpt-4o-mini', maxTokens: 200 },
|
|
86
|
+
},
|
|
87
|
+
provider: 'openai',
|
|
88
|
+
mode: 'sync',
|
|
89
|
+
};
|
|
90
|
+
|
|
91
|
+
const res: AIResponse = await router.invoke(req);
|
|
92
|
+
console.log(res.outputText);
|
|
93
|
+
console.log(res.usage);
|
|
94
|
+
console.log(res.rawResponse); // always present — lossless provider payload
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
Set provider API keys in your environment (see [Configuration](#configuration)). With no arguments, `createRouter()` auto-discovers settings via ERC.
|
|
98
|
+
|
|
99
|
+
---
|
|
100
|
+
|
|
101
|
+
## Architecture
|
|
102
|
+
|
|
103
|
+
```
|
|
104
|
+
AIRouterRequest
|
|
105
|
+
→ request interceptors (e.g. OpenRouter routing)
|
|
106
|
+
→ ProviderModule (from installed @x12i/ai-provider-* package)
|
|
107
|
+
→ router-side adapter (request → ProviderSDKCallSpec)
|
|
108
|
+
→ provider.execute() | stream() | submitBatch()
|
|
109
|
+
→ router-side adapter (ProviderSDKExecResult → AIResponse)
|
|
110
|
+
→ response interceptors
|
|
111
|
+
→ AIResponse (with lossless rawResponse)
|
|
112
|
+
```
|
|
59
113
|
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
114
|
+
| Layer | Role |
|
|
115
|
+
|-------|------|
|
|
116
|
+
| **ProviderModule** | Provider packages implement `@x12i/ai-provider-interface` |
|
|
117
|
+
| **Router adapters** | Convert router requests to `ProviderSDKCallSpec` and parse responses |
|
|
118
|
+
| **Capability gating** | Router checks `provider.capabilities.modes.sync/stream/batch` |
|
|
119
|
+
| **Execution semantics** | Router owns `timeoutMs`, retries, `idempotencyKey`, `AbortSignal` |
|
|
64
120
|
|
|
65
121
|
---
|
|
66
122
|
|
|
67
|
-
##
|
|
123
|
+
## Provider IDs
|
|
124
|
+
|
|
125
|
+
**Direct providers** (require matching `@x12i/ai-provider-*` package and API key):
|
|
68
126
|
|
|
69
|
-
|
|
127
|
+
| ID | Vendor |
|
|
128
|
+
|----|--------|
|
|
129
|
+
| `openai` | OpenAI |
|
|
130
|
+
| `grok` | Grok / xAI |
|
|
131
|
+
| `anthropic` | Claude |
|
|
132
|
+
| `google` | Gemini |
|
|
133
|
+
| `groq` | GroqCloud |
|
|
70
134
|
|
|
71
|
-
|
|
135
|
+
**OpenRouter** (unified gateway):
|
|
72
136
|
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
- **Provider Aliases**: Supports vendor mappings (e.g., `xai` models route to `grok` provider)
|
|
78
|
-
- **Model Name Mapping**: Automatically converts provider + model to OpenRouter format (e.g., `provider: "openai"` + `model: "gpt-4o"` → `"openai/gpt-4o"`)
|
|
79
|
-
- **Access any OpenRouter model**: Call models even without direct provider packages (e.g., `"meta-llama/llama-3-70b-instruct"`)
|
|
80
|
-
- **Unified Reasoning API**: Cross-vendor reasoning support with effort control and visibility options (see [Reasoning Integration](./docs/reasoning-integration.md))
|
|
81
|
-
- **No ai-io-normalizer**: OpenRouter responses are parsed directly (faster, simpler)
|
|
137
|
+
| ID | Role |
|
|
138
|
+
|----|------|
|
|
139
|
+
| `openrouter` | Explicit OpenRouter transport |
|
|
140
|
+
| Any vendor ID | Routed through OpenRouter when preferred (`USE_OPENROUTER=true`, default) or as fallback when no direct key |
|
|
82
141
|
|
|
83
|
-
|
|
142
|
+
> **Grok ≠ Groq** — Grok is xAI (`grok` / `xai`). Groq is GroqCloud (`groq`).
|
|
143
|
+
|
|
144
|
+
---
|
|
84
145
|
|
|
85
|
-
|
|
146
|
+
## OpenRouter mode
|
|
86
147
|
|
|
87
|
-
|
|
148
|
+
OpenRouter is a unified API gateway. With an `OPENROUTER_API_KEY`, the router can reach models from many vendors through one key — using familiar provider names (`openai`, `grok`, `anthropic`, …) and automatic model mapping (e.g. `openai` + `gpt-4o` → `openai/gpt-4o`).
|
|
149
|
+
|
|
150
|
+
### Enable OpenRouter
|
|
151
|
+
|
|
152
|
+
Set an API key (canonical name preferred):
|
|
88
153
|
|
|
89
154
|
```bash
|
|
90
|
-
export
|
|
155
|
+
export OPENROUTER_API_KEY=sk-or-your-key-here
|
|
156
|
+
# Legacy alias also supported:
|
|
157
|
+
# export OPEN_ROUTER_KEY=sk-or-your-key-here
|
|
91
158
|
```
|
|
92
159
|
|
|
93
|
-
|
|
160
|
+
Optional ranking headers:
|
|
94
161
|
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
162
|
+
```bash
|
|
163
|
+
export OPENROUTER_HTTP_REFERER=https://your-site.com # legacy: OPEN_ROUTER_HTTP_REFERER
|
|
164
|
+
export OPENROUTER_X_TITLE=Your Site Name # legacy: OPEN_ROUTER_X_TITLE
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
### `USE_OPENROUTER` — prefer vs fallback
|
|
168
|
+
|
|
169
|
+
`USE_OPENROUTER` does **not** turn OpenRouter on or off. The router always registers OpenRouter when a key is present. This flag controls **whether OpenRouter is preferred over direct provider keys**.
|
|
98
170
|
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
171
|
+
| `USE_OPENROUTER` | `OPENROUTER_API_KEY` | Direct provider key (e.g. `OPENAI_API_KEY`) | What happens |
|
|
172
|
+
|------------------|----------------------|---------------------------------------------|--------------|
|
|
173
|
+
| unset or `true` *(default)* | set | set or unset | **Prefer OpenRouter** — all vendor calls route through OpenRouter, even when a direct key exists |
|
|
174
|
+
| unset or `true` | set | not set | Route through OpenRouter |
|
|
175
|
+
| `false` | set | set for requested vendor | **Direct provider** — use the vendor's own key/API |
|
|
176
|
+
| `false` | set | not set for requested vendor | **OpenRouter fallback** — e.g. request `anthropic` with no `ANTHROPIC_API_KEY` still works via OpenRouter |
|
|
104
177
|
|
|
105
|
-
**
|
|
178
|
+
**Default:** prefer OpenRouter whenever `OPENROUTER_API_KEY` is set (`USE_OPENROUTER` defaults to `true`).
|
|
179
|
+
|
|
180
|
+
To use direct provider keys when available, while keeping OpenRouter as fallback for vendors without keys:
|
|
106
181
|
|
|
107
182
|
```bash
|
|
108
183
|
export USE_OPENROUTER=false
|
|
184
|
+
export OPENROUTER_API_KEY=sk-or-...
|
|
185
|
+
export OPENAI_API_KEY=sk-... # openai requests → direct OpenAI
|
|
186
|
+
# no ANTHROPIC_API_KEY # anthropic requests → OpenRouter fallback
|
|
109
187
|
```
|
|
110
188
|
|
|
111
|
-
|
|
189
|
+
Programmatic override:
|
|
112
190
|
|
|
113
|
-
|
|
191
|
+
```ts
|
|
192
|
+
const router = await createRouter({
|
|
193
|
+
useOpenRouter: false, // direct when keys exist; OpenRouter fallback otherwise
|
|
194
|
+
});
|
|
195
|
+
```
|
|
114
196
|
|
|
115
|
-
|
|
116
|
-
1. ✅ Check that `OPEN_ROUTER_KEY` is set: `echo $OPEN_ROUTER_KEY`
|
|
117
|
-
2. ✅ Verify the key is valid (not empty, doesn't start with "ENV.")
|
|
118
|
-
3. ✅ Ensure `config.provider` is specified in your request (e.g., `config: { provider: "openai", model: "gpt-4o" }`)
|
|
119
|
-
4. ✅ The OpenRouter adapter is always registered - no additional setup needed
|
|
197
|
+
### Behavior summary
|
|
120
198
|
|
|
121
|
-
|
|
199
|
+
- **OpenRouter is always available** when `OPENROUTER_API_KEY` is set — used as the default transport or as fallback
|
|
200
|
+
- **`USE_OPENROUTER=true` (default):** routes through OpenRouter even if `OPENAI_API_KEY`, `GROK_API_KEY`, etc. are also set; direct provider packages are not auto-registered (avoids singleton config conflicts)
|
|
201
|
+
- **`USE_OPENROUTER=false`:** auto-registers direct providers when their API keys exist; OpenRouter handles any vendor without a direct key
|
|
202
|
+
- Works with `createRouter()` and `new LLMProviderRouter()` — auto-registration on first call
|
|
203
|
+
- Provider names stay the same in your code; the router handles transport selection internally
|
|
204
|
+
- Catalog data (`.metadata/openrouter_catalog_with_vendor_mapping.json`) drives model validation and provider inference
|
|
205
|
+
- Responses on the OpenRouter path are parsed directly from OpenAI-compatible formats (no `ai-io-normalizer`)
|
|
122
206
|
|
|
123
|
-
###
|
|
207
|
+
### Examples
|
|
124
208
|
|
|
125
|
-
**
|
|
209
|
+
**Same provider name, OpenRouter underneath:**
|
|
126
210
|
|
|
127
211
|
```ts
|
|
128
|
-
const router = await createRouter();
|
|
129
|
-
|
|
130
|
-
// Works exactly the same whether OpenRouter mode is on or off
|
|
131
212
|
const req: AIRouterRequest = {
|
|
132
|
-
request: {
|
|
133
|
-
|
|
134
|
-
config: { model: 'gpt-4o' },
|
|
135
|
-
},
|
|
136
|
-
provider: 'openai', // Still use "openai" - router handles routing
|
|
213
|
+
request: { messages: [{ role: 'user', content: 'Hello!' }], config: { model: 'gpt-4o' } },
|
|
214
|
+
provider: 'openai',
|
|
137
215
|
mode: 'sync',
|
|
138
216
|
};
|
|
139
|
-
|
|
140
|
-
const res = await router.invoke(req);
|
|
141
|
-
// Model automatically mapped to "openai/gpt-4o" when using OpenRouter
|
|
217
|
+
await router.invoke(req);
|
|
142
218
|
```
|
|
143
219
|
|
|
144
|
-
**
|
|
220
|
+
**OpenRouter model format directly:**
|
|
145
221
|
|
|
146
222
|
```ts
|
|
147
|
-
// Router infers provider from model name
|
|
148
223
|
const req: AIRouterRequest = {
|
|
149
224
|
request: {
|
|
150
225
|
messages: [{ role: 'user', content: 'Hello!' }],
|
|
151
|
-
config: { model: '
|
|
226
|
+
config: { model: 'anthropic/claude-3-opus' },
|
|
152
227
|
},
|
|
153
|
-
|
|
228
|
+
provider: 'openrouter',
|
|
154
229
|
mode: 'sync',
|
|
155
230
|
};
|
|
156
|
-
|
|
157
|
-
const res = await router.invoke(req);
|
|
231
|
+
await router.invoke(req);
|
|
158
232
|
```
|
|
159
233
|
|
|
160
|
-
**
|
|
234
|
+
**Provider inference from model name** (no `provider` field):
|
|
161
235
|
|
|
162
236
|
```ts
|
|
163
|
-
// Call any OpenRouter-supported model using OpenRouter's format
|
|
164
237
|
const req: AIRouterRequest = {
|
|
165
|
-
request: {
|
|
166
|
-
messages: [{ role: 'user', content: 'Hello!' }],
|
|
167
|
-
config: { model: 'anthropic/claude-3-opus' }, // Direct OpenRouter format
|
|
168
|
-
},
|
|
169
|
-
provider: 'openrouter', // Use "openrouter" provider
|
|
238
|
+
request: { messages: [{ role: 'user', content: 'Hello!' }], config: { model: 'gpt-4o' } },
|
|
170
239
|
mode: 'sync',
|
|
171
240
|
};
|
|
172
|
-
|
|
173
|
-
const res = await router.invoke(req);
|
|
241
|
+
await router.invoke(req); // infers openai
|
|
174
242
|
```
|
|
175
243
|
|
|
176
|
-
|
|
244
|
+
### Troubleshooting
|
|
177
245
|
|
|
178
|
-
|
|
179
|
-
// Access Meta Llama models without installing @x12i/ai-provider-meta
|
|
180
|
-
const req: AIRouterRequest = {
|
|
181
|
-
request: {
|
|
182
|
-
messages: [{ role: 'user', content: 'Hello!' }],
|
|
183
|
-
config: { model: 'meta-llama/llama-3-70b-instruct' },
|
|
184
|
-
},
|
|
185
|
-
provider: 'openrouter',
|
|
186
|
-
mode: 'sync',
|
|
187
|
-
};
|
|
246
|
+
If you see *"No provider specified and no providers registered"*:
|
|
188
247
|
|
|
189
|
-
|
|
190
|
-
|
|
248
|
+
1. Confirm `OPENROUTER_API_KEY` (or `OPEN_ROUTER_KEY`) is set and non-empty
|
|
249
|
+
2. Ensure the key does not start with `ENV.` (unresolved placeholder)
|
|
250
|
+
3. Set `config.provider` in the request (e.g. `{ provider: 'openai', model: 'gpt-4o' }`)
|
|
251
|
+
4. The OpenRouter adapter is always registered — no extra setup required
|
|
191
252
|
|
|
192
|
-
|
|
253
|
+
See also [debugging guide](./docs/debugging-no-provider-error.md).
|
|
254
|
+
|
|
255
|
+
---
|
|
256
|
+
|
|
257
|
+
## Configuration
|
|
258
|
+
|
|
259
|
+
### Zero-config (`createRouter`)
|
|
193
260
|
|
|
194
261
|
```ts
|
|
195
|
-
|
|
196
|
-
const claudeReq = { request: { messages: [{ role: 'user', content: 'Hello!' }], config: { model: 'claude-3-opus' } }, provider: 'anthropic', mode: 'sync' };
|
|
262
|
+
import { createRouter } from '@x12i/ai-providers-router';
|
|
197
263
|
|
|
198
|
-
//
|
|
199
|
-
|
|
264
|
+
const router = await createRouter(); // reads process.env
|
|
265
|
+
```
|
|
200
266
|
|
|
201
|
-
|
|
202
|
-
const groqReq = { request: { messages: [{ role: 'user', content: 'Hello!' }], config: { model: 'llama-3-70b-8192' } }, provider: 'groq', mode: 'sync' };
|
|
267
|
+
### Programmatic (advanced mode)
|
|
203
268
|
|
|
204
|
-
|
|
205
|
-
const
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
269
|
+
```ts
|
|
270
|
+
const router = await createRouter({
|
|
271
|
+
logLevel: 'info',
|
|
272
|
+
verbose: false,
|
|
273
|
+
timeoutMs: 60_000,
|
|
274
|
+
useOpenRouter: true, // default: prefer OpenRouter when OPENROUTER_API_KEY is set
|
|
275
|
+
fallbackChain: [{ provider: 'openai', model: 'gpt-4o-mini' }, { provider: 'grok', model: 'grok-2' }],
|
|
276
|
+
openrouter: { apiKey: 'sk-or-...', httpReferer: 'https://example.com', xTitle: 'My App' },
|
|
277
|
+
usageTracker: {
|
|
278
|
+
recordRequest(e) {
|
|
279
|
+
// provider, timestamp, duration, tokens, cost, success
|
|
280
|
+
},
|
|
281
|
+
},
|
|
282
|
+
providerConfigs: {
|
|
283
|
+
openai: { apiKey: 'sk-...', baseURL: 'https://api.openai.com/v1' },
|
|
284
|
+
grok: { apiKey: 'xai-...' },
|
|
285
|
+
},
|
|
286
|
+
});
|
|
210
287
|
```
|
|
211
288
|
|
|
212
|
-
|
|
289
|
+
Passing any explicit config object to `createRouter(config)` overrides zero-config env discovery for that call.
|
|
290
|
+
|
|
291
|
+
### Environment variables
|
|
292
|
+
|
|
293
|
+
| Variable | Default | Description |
|
|
294
|
+
|----------|---------|-------------|
|
|
295
|
+
| **Router** | | |
|
|
296
|
+
| `AI_PROVIDER_ROUTER_LOGS_LEVEL` | *(see logging)* | Canonical log threshold via logxer (`error`, `warn`, `info`, `debug`, `verbose`, `off`) |
|
|
297
|
+
| `AI_PROVIDER_ROUTER_LOG_LEVEL` | `info` | Legacy alias for log level (used when `_LOGS_LEVEL` is unset) |
|
|
298
|
+
| `AI_PROVIDER_ROUTER_VERBOSE` | `false` | Log full AI request/response payloads (sanitized) |
|
|
299
|
+
| `AI_PROVIDER_ROUTER_TIMEOUT_MS` | `60000` | Default operation timeout (ms) |
|
|
300
|
+
| **OpenAI** | | |
|
|
301
|
+
| `OPENAI_API_KEY` | — | Required for direct OpenAI calls |
|
|
302
|
+
| `OPENAI_API_BASE` | — | Custom API base URL |
|
|
303
|
+
| `OPENAI_ORGANIZATION` | — | Organization ID |
|
|
304
|
+
| **Grok / xAI** | | |
|
|
305
|
+
| `GROK_API_KEY` | — | Required for direct Grok calls |
|
|
306
|
+
| `XAI_API_BASE` | — | Custom xAI base URL |
|
|
307
|
+
| **OpenRouter** | | |
|
|
308
|
+
| `OPENROUTER_API_KEY` | — | Enables OpenRouter (always registered when set) |
|
|
309
|
+
| `OPEN_ROUTER_KEY` | — | Legacy alias for `OPENROUTER_API_KEY` |
|
|
310
|
+
| `USE_OPENROUTER` | `true` | Prefer OpenRouter over direct keys when OR key is set; set `false` to use direct providers when keys exist (OpenRouter remains fallback) |
|
|
311
|
+
| `OPENROUTER_HTTP_REFERER` | — | Optional ranking header |
|
|
312
|
+
| `OPENROUTER_X_TITLE` | — | Optional ranking header |
|
|
313
|
+
| **Other providers** | | |
|
|
314
|
+
| `ANTHROPIC_API_KEY`, `GOOGLE_API_KEY`, `GROQ_API_KEY`, … | — | Used when those providers are installed |
|
|
315
|
+
|
|
316
|
+
Full reference: [Environment variables](./docs/environment-variables.md) · [Configuration guide](./docs/CONFIGURATION_GUIDE.md)
|
|
213
317
|
|
|
214
|
-
|
|
215
|
-
- Preserves the original provider name (e.g., `"openai"`, `"grok"`) in `request.config.provider`
|
|
216
|
-
- Routes the request to `"openrouter"` provider
|
|
217
|
-
- Infers provider from model name if not specified
|
|
318
|
+
---
|
|
218
319
|
|
|
219
|
-
|
|
220
|
-
- Reads the original provider from `request.config.provider`
|
|
221
|
-
- Maps model names: `"gpt-4o"` + `provider: "openai"` → `"openai/gpt-4o"`
|
|
222
|
-
- Handles models already in OpenRouter format (with `/`) as-is
|
|
320
|
+
## Logging
|
|
223
321
|
|
|
224
|
-
|
|
225
|
-
- **Chat Completions**: Extracts `choices[0].message.content` for text
|
|
226
|
-
- **Responses API (v1)**: Handles `output` array with text and encrypted reasoning items
|
|
227
|
-
- Extracts `usage` for token counts from both formats
|
|
228
|
-
- Adds `status: 'completed'` for compatibility
|
|
322
|
+
The router uses [`@x12i/logxer`](https://www.npmjs.com/package/@x12i/logxer) for structured, package-scoped logging.
|
|
229
323
|
|
|
230
|
-
|
|
324
|
+
**Package prefix:** `AI_PROVIDER_ROUTER`
|
|
231
325
|
|
|
232
|
-
|
|
326
|
+
```bash
|
|
327
|
+
# Canonical (preferred)
|
|
328
|
+
AI_PROVIDER_ROUTER_LOGS_LEVEL=info
|
|
233
329
|
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
- **Vendor Mapping**: Maps vendor IDs to provider slugs (e.g., `xai` → `grok`)
|
|
237
|
-
- **Fallback Patterns**: Uses legacy pattern matching when catalog data is unavailable:
|
|
330
|
+
# Legacy (still supported when _LOGS_LEVEL is unset)
|
|
331
|
+
AI_PROVIDER_ROUTER_LOG_LEVEL=info
|
|
238
332
|
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
- `gemini-*`, `google/*` → `"google"`
|
|
243
|
-
- `llama-*`, `meta-llama/*` → `"meta"`
|
|
244
|
-
- Default → `"openai"` (most common case)
|
|
333
|
+
# Log full AI request/response payloads (router-specific, separate from log level)
|
|
334
|
+
AI_PROVIDER_ROUTER_VERBOSE=true
|
|
335
|
+
```
|
|
245
336
|
|
|
246
|
-
|
|
337
|
+
**Log levels:** `error` · `warn` · `info` · `debug` · `verbose` · `off`
|
|
247
338
|
|
|
248
|
-
|
|
339
|
+
When neither `_LOGS_LEVEL` nor `_LOG_LEVEL` is set, no `logLevel` / `logging` is passed, and the logxer registry has no entry, the router defaults to **`info`** (not logxer's package-only default of `warn`). `createRouter()` loads `LOGXER_PACKAGE_LEVELS` / `LOGXER_PACKAGE_LOGS_DEFAULT` via `applyPackageLogLevelsFromEnv()` after `.env`.
|
|
249
340
|
|
|
250
|
-
|
|
251
|
-
- **Alias Resolution**: Automatically resolves model aliases to canonical OpenRouter IDs
|
|
252
|
-
- **Capability Checking**: Validates model parameters against supported capabilities
|
|
253
|
-
- **Graceful Fallbacks**: Falls back to legacy logic if catalog loading fails
|
|
254
|
-
- **Format Support**: Handles both OpenAI Chat Completions and Responses API v1 formats
|
|
255
|
-
- **Encrypted Reasoning**: Processes encrypted reasoning traces (model thinking is privacy-protected)
|
|
256
|
-
- **Reasoning Parameter Support**: Enables reasoning effort levels for compatible models
|
|
341
|
+
**Host apps (logxer ≥ 4.5)** — provider packages (`@x12i/ai-provider-openai`, etc.) are **not** on the logxer 4.5 stack. Stack/registry options apply **only** to this router's logs (`AI_PROVIDER_ROUTER`). Configure your other libraries from `@x12i/logxer` in the host; pass the same `StackLoggingOptions` into `createRouter` when you want one object for the whole app:
|
|
257
342
|
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
343
|
+
```ts
|
|
344
|
+
import { configurePackageLogLevels, type StackLoggingOptions } from '@x12i/logxer';
|
|
345
|
+
import { createRouter, ROUTER_LOG_ENV_PREFIX } from '@x12i/ai-providers-router';
|
|
346
|
+
|
|
347
|
+
configurePackageLogLevels({
|
|
348
|
+
default: 'warn',
|
|
349
|
+
levels: {
|
|
350
|
+
MY_GATEWAY: 'info',
|
|
351
|
+
[ROUTER_LOG_ENV_PREFIX]: 'debug',
|
|
352
|
+
},
|
|
353
|
+
});
|
|
354
|
+
|
|
355
|
+
const logging: StackLoggingOptions = {
|
|
356
|
+
packageLevels: { [ROUTER_LOG_ENV_PREFIX]: 'debug' },
|
|
357
|
+
};
|
|
263
358
|
|
|
264
|
-
|
|
359
|
+
const router = await createRouter({ logging, verbose: true });
|
|
360
|
+
```
|
|
265
361
|
|
|
266
|
-
|
|
362
|
+
Bulk env for this package (loaded by `createRouter()` after `.env`):
|
|
267
363
|
|
|
268
364
|
```bash
|
|
269
|
-
|
|
270
|
-
|
|
365
|
+
LOGXER_PACKAGE_LEVELS=AI_PROVIDER_ROUTER:info
|
|
366
|
+
AI_PROVIDER_ROUTER_LOGS_LEVEL=error # wins over bulk for this prefix only
|
|
271
367
|
```
|
|
272
368
|
|
|
273
|
-
|
|
369
|
+
**Programmatic (router only):**
|
|
370
|
+
|
|
371
|
+
```ts
|
|
372
|
+
const router = await createRouter({ logLevel: 'debug', verbose: true });
|
|
373
|
+
const router2 = await createRouter({ logger: createLogger({ level: 'info', verbose: false }) });
|
|
374
|
+
```
|
|
375
|
+
|
|
376
|
+
Verbose mode logs sanitized AI request/response payloads. Cross-cutting sinks (console, file, format) are configured in the **host** via `@x12i/logxer` — not via provider packages.
|
|
274
377
|
|
|
275
378
|
---
|
|
276
379
|
|
|
277
|
-
##
|
|
380
|
+
## API usage
|
|
278
381
|
|
|
279
|
-
|
|
382
|
+
### Sync call
|
|
280
383
|
|
|
281
384
|
```ts
|
|
282
|
-
import { createRouter } from '@x12i/ai-providers-router';
|
|
385
|
+
import { createRouter, type AIRouterRequest, type AIResponse } from '@x12i/ai-providers-router';
|
|
283
386
|
|
|
284
387
|
const router = await createRouter();
|
|
388
|
+
|
|
389
|
+
const req: AIRouterRequest = {
|
|
390
|
+
request: {
|
|
391
|
+
inputData: 'Write 3 bullets about routers.',
|
|
392
|
+
config: { model: 'gpt-4o-mini', maxTokens: 200, temperature: 0.7 },
|
|
393
|
+
},
|
|
394
|
+
provider: 'openai',
|
|
395
|
+
mode: 'sync',
|
|
396
|
+
exec: {
|
|
397
|
+
timeoutMs: 60_000,
|
|
398
|
+
idempotencyKey: 'optional-key',
|
|
399
|
+
signal: abortController.signal,
|
|
400
|
+
},
|
|
401
|
+
};
|
|
402
|
+
|
|
403
|
+
const res: AIResponse = await router.invoke(req);
|
|
285
404
|
```
|
|
286
405
|
|
|
287
|
-
|
|
406
|
+
### Streaming call
|
|
288
407
|
|
|
289
408
|
```ts
|
|
290
|
-
const
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
|
|
409
|
+
const streamReq: AIRouterRequest = { ...req, mode: 'stream' };
|
|
410
|
+
|
|
411
|
+
for await (const ev of router.stream(streamReq)) {
|
|
412
|
+
switch (ev.type) {
|
|
413
|
+
case 'provider_raw':
|
|
414
|
+
console.log('Raw:', ev.raw);
|
|
415
|
+
break;
|
|
416
|
+
case 'output_text_delta':
|
|
417
|
+
process.stdout.write(ev.delta);
|
|
418
|
+
break;
|
|
419
|
+
case 'reasoning_summary_delta':
|
|
420
|
+
case 'reasoning_trace_delta':
|
|
421
|
+
// reasoning stream chunks
|
|
422
|
+
break;
|
|
423
|
+
case 'completed':
|
|
424
|
+
console.log('Final:', ev.response.outputText);
|
|
425
|
+
break;
|
|
426
|
+
case 'error':
|
|
427
|
+
console.error(ev.error);
|
|
428
|
+
break;
|
|
429
|
+
}
|
|
430
|
+
}
|
|
431
|
+
```
|
|
432
|
+
|
|
433
|
+
### Batch requests
|
|
434
|
+
|
|
435
|
+
Batch is available only when `provider.capabilities.modes.batch === true`:
|
|
436
|
+
|
|
437
|
+
```ts
|
|
438
|
+
const items = [
|
|
439
|
+
{ request: { inputData: 'First', config: { model: 'gpt-4o-mini' } } },
|
|
440
|
+
{ request: { inputData: 'Second', config: { model: 'gpt-4o-mini' } } },
|
|
441
|
+
];
|
|
442
|
+
|
|
443
|
+
const batchResult = await router.createBatch('openai', items, {
|
|
444
|
+
timeoutMs: 120_000,
|
|
445
|
+
idempotencyKey: 'batch-1',
|
|
297
446
|
});
|
|
447
|
+
|
|
448
|
+
console.log(batchResult.items);
|
|
449
|
+
console.log(batchResult.rawBatch);
|
|
298
450
|
```
|
|
299
451
|
|
|
300
|
-
|
|
452
|
+
### Request and response types
|
|
301
453
|
|
|
302
|
-
|
|
454
|
+
| Type | Purpose |
|
|
455
|
+
|------|---------|
|
|
456
|
+
| `AIRouterRequest` | Router input (`request`, `provider`, `mode`, `exec`) |
|
|
457
|
+
| `AIResponse` | Sync output (`outputText`, `rawResponse`, `usage`, `reasoning`, `metadata`) |
|
|
458
|
+
| `AIStreamEvent` | Streaming events (`output_text_delta`, `completed`, `error`, …) |
|
|
459
|
+
| `AIBatchResponse` | Batch results |
|
|
460
|
+
| `RouterConfig` | Router-level settings |
|
|
461
|
+
| `ProviderModelRef` | `{ provider?, engine?, model? }` for fallback chains |
|
|
303
462
|
|
|
304
|
-
|
|
463
|
+
### Trace diagnostics
|
|
305
464
|
|
|
306
|
-
|
|
307
|
-
* `AIResponse` (sync output) - includes unified reasoning response
|
|
308
|
-
* `AIStreamEvent` (streaming output) - includes reasoning streaming events
|
|
309
|
-
* `AIBatchResponse` (batch output)
|
|
465
|
+
Every `AIResponse` includes stable, provider-agnostic diagnostics in `metadata`:
|
|
310
466
|
|
|
311
|
-
|
|
467
|
+
| Field | Description |
|
|
468
|
+
|-------|-------------|
|
|
469
|
+
| `metadata.provider` | Final provider used |
|
|
470
|
+
| `metadata.modelUsed` | Actual model that served the response |
|
|
471
|
+
| `metadata.costUsd` / `metadata.cost` | USD cost when reported (e.g. OpenRouter `usage.cost`) |
|
|
472
|
+
| `metadata.costStatus` | `'priced'` or `'unpriced'` |
|
|
473
|
+
| `metadata.maxTokensRequested` | Effective generation cap |
|
|
474
|
+
| `metadata.requestIds` | `{ routerRequestId, providerRequestId?, openrouterRequestId? }` |
|
|
475
|
+
| `metadata.timing` | `{ startedAt, endedAt, durationMs }` |
|
|
476
|
+
| `metadata.latencyMs` | Alias for `timing.durationMs` |
|
|
477
|
+
| `metadata.attempts[]` | Ordered retry + fallback trace |
|
|
478
|
+
| `response.output.parsed` | Structured fields when `outputContract` is set |
|
|
312
479
|
|
|
313
|
-
|
|
480
|
+
### Reasoning
|
|
314
481
|
|
|
315
|
-
|
|
316
|
-
- `response.metadata` (keys when known):
|
|
317
|
-
- `metadata.provider`: final provider used for the successful call (or last attempt)
|
|
318
|
-
- `metadata.modelUsed`: the actual model that served the response
|
|
319
|
-
- `metadata.maxTokensRequested`: final effective generation cap applied (if determinable)
|
|
320
|
-
- `metadata.costUsd` / `metadata.cost`: normalized USD cost when the provider reports it (e.g. OpenRouter `usage.cost`)
|
|
321
|
-
- `metadata.costStatus`: `'priced'` when `costUsd` is set; `'unpriced'` when usage exists but no cost was returned
|
|
322
|
-
- `response.output.parsed`: structured fields when `outputContract` is on the request (markdown sections → camelCase keys)
|
|
323
|
-
- `metadata.requestIds`: `{ routerRequestId, providerRequestId?, openrouterRequestId? }`
|
|
324
|
-
- `metadata.timing`: `{ startedAt, endedAt, durationMs }` (provider-call timing)
|
|
325
|
-
- `metadata.latencyMs`: alias for `metadata.timing.durationMs`
|
|
326
|
-
- `metadata.attempts[]`: ordered attempts across retries + fallbacks (authoritative execution trace)
|
|
482
|
+
Request unified reasoning controls via `request.config.reasoning`:
|
|
327
483
|
|
|
328
484
|
```ts
|
|
329
|
-
import type { AIRouterRequest, AIResponse } from '@x12i/ai-providers-router';
|
|
330
|
-
|
|
331
|
-
// Request reasoning with extended effort levels
|
|
332
485
|
config: {
|
|
333
486
|
reasoning: {
|
|
334
|
-
effort: 'high',
|
|
335
|
-
maxTokens: 2000,
|
|
336
|
-
visibility: 'trace',
|
|
337
|
-
onUnsupported: 'downgrade'
|
|
338
|
-
}
|
|
487
|
+
effort: 'high', // low | medium | high | xhigh (xhigh → high)
|
|
488
|
+
maxTokens: 2000, // Anthropic/Gemini max_tokens mode
|
|
489
|
+
visibility: 'trace', // none | summary | trace (best-effort)
|
|
490
|
+
onUnsupported: 'downgrade', // downgrade | error | ignore
|
|
491
|
+
},
|
|
339
492
|
}
|
|
340
|
-
|
|
341
|
-
// Access unified reasoning response
|
|
342
|
-
response.reasoning.artifacts.encrypted // Encrypted reasoning traces
|
|
343
|
-
response.reasoning.applied.effort // What was actually applied (may differ from requested)
|
|
344
|
-
response.reasoning.applied.visibility // What visibility was actually returned
|
|
345
|
-
response.reasoning.availability // Model capability flags
|
|
346
|
-
response.reasoning.warnings // Any downgrade/normalization warnings
|
|
347
493
|
```
|
|
348
494
|
|
|
349
|
-
|
|
350
|
-
- ✅ **Effort Control**: `low`, `medium`, `high`, `xhigh` (xhigh auto-normalized to high)
|
|
351
|
-
- ✅ **Max Tokens Control**: Direct `maxTokens` budget for Anthropic/Gemini models
|
|
352
|
-
- ✅ **Encrypted Traces**: Access encrypted reasoning artifacts (ciphertext not decryptable by user; only metadata/prefix logged)
|
|
353
|
-
- ✅ **Summary Visibility**: Human-readable reasoning summary (best-effort; returned only if provider returns `reasoning_details` with `reasoning.summary`; otherwise downgraded with warning)
|
|
354
|
-
- ✅ **Trace Visibility**: Encrypted or readable reasoning traces (best-effort; satisfied by either `reasoning.encrypted` artifacts or `reasoning.text` chunks; downgraded if not available)
|
|
355
|
-
- ✅ **Model Detection**: Automatic detection of reasoning-capable models via JSON registry (cross-vendor support)
|
|
356
|
-
- ✅ **Extended Support**: Works with OpenAI o-series models (o1, o3, o4 series - 10+ models), xAI Grok models, Anthropic Claude reasoning models, and Google Gemini reasoning models
|
|
495
|
+
Response fields: `response.reasoning.applied`, `response.reasoning.artifacts`, `response.reasoning.warnings`.
|
|
357
496
|
|
|
358
|
-
|
|
359
|
-
- **OpenAI o-series** (`openai/o*` pattern): `openai/o1`, `openai/o1-pro`, `openai/o3`, `openai/o3-mini`, `openai/o3-pro`, `openai/o3-deep-research`, `openai/o3-mini-high`, `openai/o4-mini`, `openai/o4-mini-deep-research`, `openai/o4-mini-high`
|
|
360
|
-
- **xAI Grok** (`x-ai/grok*` pattern): `x-ai/grok-4.1-fast` and other reasoning-enabled Grok models
|
|
361
|
-
- **Anthropic Claude** (`anthropic/claude*` pattern): Reasoning-enabled Claude models (uses `max_tokens` mode)
|
|
362
|
-
- **Google Gemini** (`google/gemini*` pattern): Reasoning-enabled Gemini models (uses `max_tokens` mode)
|
|
497
|
+
Supported models are tracked in `.metadata/reasoning-support.json`.
|
|
363
498
|
|
|
364
|
-
|
|
499
|
+
- [Reasoning integration guide](./docs/reasoning-integration.md)
|
|
500
|
+
- [Supported models](./docs/reasoning-supported-models.md)
|
|
365
501
|
|
|
366
|
-
|
|
502
|
+
### Fallback chains
|
|
367
503
|
|
|
368
|
-
|
|
504
|
+
On failure, the router tries the next candidate in order. Attempts are recorded in `metadata.attempts[]`. On exhaustion, throws `FallbackExhaustedError`.
|
|
369
505
|
|
|
370
|
-
|
|
506
|
+
**Router-level default chain:**
|
|
371
507
|
|
|
372
508
|
```ts
|
|
373
|
-
|
|
509
|
+
const router = await createRouter({
|
|
510
|
+
fallbackChain: [
|
|
511
|
+
{ provider: 'openai', model: 'gpt-4o' },
|
|
512
|
+
{ provider: 'grok', model: 'grok-2' },
|
|
513
|
+
],
|
|
514
|
+
});
|
|
515
|
+
```
|
|
374
516
|
|
|
375
|
-
|
|
517
|
+
**Per-request chain** (in `request.config`):
|
|
376
518
|
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
|
|
380
|
-
|
|
381
|
-
|
|
382
|
-
|
|
383
|
-
model: '
|
|
384
|
-
|
|
385
|
-
|
|
386
|
-
|
|
387
|
-
mode: 'sync',
|
|
388
|
-
exec: {
|
|
389
|
-
timeoutMs: 60000, // Optional: override default timeout
|
|
390
|
-
idempotencyKey: 'optional-key', // Optional: for idempotent requests
|
|
519
|
+
```ts
|
|
520
|
+
request: {
|
|
521
|
+
config: {
|
|
522
|
+
model: 'gpt-4o',
|
|
523
|
+
fallbackChain: [
|
|
524
|
+
{ provider: 'openai', model: 'gpt-4o-mini' },
|
|
525
|
+
{ engine: 'grok', model: 'grok-2' }, // engine is alias for provider
|
|
526
|
+
],
|
|
527
|
+
// Legacy: provider-only fallback (same model)
|
|
528
|
+
// fallbackProviders: ['grok', 'openai'],
|
|
391
529
|
},
|
|
392
|
-
}
|
|
530
|
+
},
|
|
531
|
+
```
|
|
393
532
|
|
|
394
|
-
|
|
533
|
+
Precedence: `request.config.fallbackChain` → `request.config.fallbackEngines` → `router.fallbackChain` → `request.config.fallbackProviders`.
|
|
534
|
+
|
|
535
|
+
### Interceptors
|
|
536
|
+
|
|
537
|
+
```ts
|
|
538
|
+
router.addRequestInterceptor(async (req, provider) => {
|
|
539
|
+
// mutate or replace request before execution
|
|
540
|
+
return req;
|
|
541
|
+
});
|
|
395
542
|
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
|
|
543
|
+
router.addResponseInterceptor(async (res, provider) => {
|
|
544
|
+
// mutate or replace response after execution
|
|
545
|
+
return res;
|
|
546
|
+
});
|
|
399
547
|
```
|
|
400
548
|
|
|
549
|
+
OpenRouter registers a request interceptor when `USE_OPENROUTER=true` (default) to route vendor calls through OpenRouter while preserving the original provider name for model mapping. When `USE_OPENROUTER=false`, the interceptor is skipped; `resolveProviderName` uses direct providers when registered and falls back to OpenRouter otherwise.
|
|
550
|
+
|
|
551
|
+
### Health checks
|
|
552
|
+
|
|
553
|
+
```ts
|
|
554
|
+
const result = await router.checkHealth('openai');
|
|
555
|
+
// { provider: 'openai', healthy: true, latencyMs: 1234 }
|
|
556
|
+
// or { provider: 'openai', healthy: false, latencyMs: 5000, error: '...' }
|
|
557
|
+
```
|
|
558
|
+
|
|
559
|
+
Runs a minimal sync invoke with a 5 s timeout.
|
|
560
|
+
|
|
401
561
|
---
|
|
402
562
|
|
|
403
|
-
##
|
|
563
|
+
## AIGateway
|
|
564
|
+
|
|
565
|
+
Thin wrapper around the router for gateway-style requests (instructions + inputData):
|
|
404
566
|
|
|
405
567
|
```ts
|
|
406
|
-
|
|
407
|
-
...req,
|
|
408
|
-
mode: 'stream',
|
|
409
|
-
};
|
|
568
|
+
import { AIGateway, createRouter } from '@x12i/ai-providers-router';
|
|
410
569
|
|
|
411
|
-
|
|
412
|
-
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
|
|
418
|
-
|
|
419
|
-
// Final response
|
|
420
|
-
console.log('Final:', ev.response.outputText);
|
|
421
|
-
} else if (ev.type === 'error') {
|
|
422
|
-
console.error('Error:', ev.error);
|
|
423
|
-
}
|
|
424
|
-
}
|
|
570
|
+
const gateway = new AIGateway(await createRouter());
|
|
571
|
+
|
|
572
|
+
const response = await gateway.invoke({
|
|
573
|
+
instructions: 'You are a helpful assistant.',
|
|
574
|
+
inputData: 'Explain routers in one sentence.',
|
|
575
|
+
config: { provider: 'openai', model: 'gpt-4o-mini' },
|
|
576
|
+
mode: 'sync',
|
|
577
|
+
});
|
|
425
578
|
```
|
|
426
579
|
|
|
580
|
+
Also accepts full `AIRouterRequest` shapes (`{ request, provider, mode }`) and unwraps them automatically.
|
|
581
|
+
|
|
582
|
+
Optional strict provider/model pinning: set `config.enforceProviderModel: true` to throw on mismatch instead of silently switching.
|
|
583
|
+
|
|
427
584
|
---
|
|
428
585
|
|
|
429
|
-
##
|
|
586
|
+
## Response normalization and cost
|
|
430
587
|
|
|
431
|
-
|
|
588
|
+
Exported helpers for downstream activity persistence and output contracts:
|
|
432
589
|
|
|
433
590
|
```ts
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
|
|
437
|
-
|
|
591
|
+
import {
|
|
592
|
+
applyResponseNormalization,
|
|
593
|
+
resolveCostReporting,
|
|
594
|
+
extractCostUsdFromRouterResponse,
|
|
595
|
+
extractCostUsdFromProviderUsage,
|
|
596
|
+
enrichParsedForOutputContract,
|
|
597
|
+
resolveOutputContractFieldKeys,
|
|
598
|
+
parseMarkdownSectionsFromContent,
|
|
599
|
+
} from '@x12i/ai-providers-router';
|
|
600
|
+
```
|
|
438
601
|
|
|
439
|
-
|
|
440
|
-
|
|
441
|
-
|
|
442
|
-
|
|
602
|
+
- **Cost** — Normalizes OpenRouter and provider usage into `metadata.costUsd` / `costStatus`
|
|
603
|
+
- **Output contract** — When `outputContract` is on the request, markdown sections map to camelCase keys in `output.parsed`
|
|
604
|
+
|
|
605
|
+
See [normalization field support](./docs/normalization-field-support.md).
|
|
606
|
+
|
|
607
|
+
---
|
|
443
608
|
|
|
444
|
-
|
|
445
|
-
|
|
609
|
+
## Error types
|
|
610
|
+
|
|
611
|
+
| Error | When |
|
|
612
|
+
|-------|------|
|
|
613
|
+
| `ProviderNotFoundError` | Requested provider is not registered |
|
|
614
|
+
| `ProviderNotInstalledError` | Provider package not installed (includes `npm install` hint) |
|
|
615
|
+
| `ProviderTimeoutError` | Request exceeded `timeoutMs` (`code: 'ETIMEDOUT'`) |
|
|
616
|
+
| `FallbackExhaustedError` | All fallback candidates failed; check `.attempts[]` |
|
|
617
|
+
|
|
618
|
+
On partial provider failures, `FallbackExhaustedError` may carry a router-shaped partial payload for gateway extraction (`PartialRouterPayload`).
|
|
619
|
+
|
|
620
|
+
---
|
|
621
|
+
|
|
622
|
+
## Manual setup (advanced)
|
|
623
|
+
|
|
624
|
+
For full control without `createRouter()`:
|
|
625
|
+
|
|
626
|
+
```ts
|
|
627
|
+
import { LLMProviderRouter } from '@x12i/ai-providers-router';
|
|
628
|
+
import * as openaiModule from '@x12i/ai-provider-openai';
|
|
629
|
+
|
|
630
|
+
const router = new LLMProviderRouter({ logLevel: 'info', timeoutMs: 60_000 });
|
|
631
|
+
|
|
632
|
+
router.configureProvider('openai', { apiKey: process.env.OPENAI_API_KEY! });
|
|
633
|
+
router.registerProvider(openaiModule, 'initializeClient');
|
|
634
|
+
|
|
635
|
+
const providers = router.listProviders(); // ['openai']
|
|
636
|
+
const registry = router.getProviderRegistry();
|
|
637
|
+
const adapters = router.getAdapterRegistry();
|
|
446
638
|
```
|
|
447
639
|
|
|
448
|
-
**
|
|
640
|
+
Providers are **auto-registered on first invoke** when matching API keys are in the environment. When `USE_OPENROUTER=true` (default) and `OPENROUTER_API_KEY` is set, direct providers are skipped in favor of OpenRouter. With `USE_OPENROUTER=false`, both direct providers and OpenRouter can be registered simultaneously.
|
|
641
|
+
|
|
642
|
+
Legacy config file support:
|
|
643
|
+
|
|
644
|
+
```ts
|
|
645
|
+
import { createRouterFromConfig } from '@x12i/ai-providers-router';
|
|
646
|
+
const router = await createRouterFromConfig('./router-config.json');
|
|
647
|
+
```
|
|
449
648
|
|
|
450
649
|
---
|
|
451
650
|
|
|
452
|
-
##
|
|
453
|
-
|
|
454
|
-
|
|
455
|
-
|
|
456
|
-
|
|
457
|
-
|
|
458
|
-
|
|
459
|
-
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
|
|
463
|
-
|
|
464
|
-
|
|
465
|
-
|
|
466
|
-
|
|
467
|
-
|
|
468
|
-
|
|
469
|
-
|
|
470
|
-
|
|
651
|
+
## Public API exports
|
|
652
|
+
|
|
653
|
+
```ts
|
|
654
|
+
// Router
|
|
655
|
+
export { LLMProviderRouter, createRouter, createRouterFromConfig }
|
|
656
|
+
|
|
657
|
+
// Types
|
|
658
|
+
export type { RouterConfig, AIRouterRequest, AIResponse, AIStreamEvent,
|
|
659
|
+
AIBatchResponse, AIBatchRequestItem, NormalizedRouterOutput, ProviderModelRef,
|
|
660
|
+
HealthCheckResult, ProviderId, CreateRouterConfig }
|
|
661
|
+
|
|
662
|
+
// Errors
|
|
663
|
+
export { ProviderNotFoundError, FallbackExhaustedError,
|
|
664
|
+
ProviderNotInstalledError, ProviderTimeoutError }
|
|
665
|
+
export type { FallbackAttempt, PartialRouterPayload }
|
|
666
|
+
|
|
667
|
+
// Interceptors
|
|
668
|
+
export type { RequestInterceptor, ResponseInterceptor }
|
|
669
|
+
|
|
670
|
+
// Logger
|
|
671
|
+
export { Logger, getLogger, createLogger }
|
|
672
|
+
export type { LogLevel, LoggerConfig }
|
|
673
|
+
|
|
674
|
+
// Gateway
|
|
675
|
+
export { AIGateway }
|
|
676
|
+
export type { EnhancedLLMResponse }
|
|
677
|
+
|
|
678
|
+
// Normalization
|
|
679
|
+
export { applyResponseNormalization, resolveCostReporting,
|
|
680
|
+
extractCostUsdFromRouterResponse, extractCostUsdFromProviderUsage,
|
|
681
|
+
hasNonZeroTokenUsage, enrichParsedForOutputContract,
|
|
682
|
+
resolveOutputContractFieldKeys, contractSpecToFieldKeys,
|
|
683
|
+
parseMarkdownSectionsFromContent }
|
|
684
|
+
export type { ActivityCostStatus, ResolvedCostReporting }
|
|
685
|
+
|
|
686
|
+
// Registries and adapters (advanced)
|
|
687
|
+
export { ProviderRegistry, AdapterRegistry, OpenAIAdapter, GrokAdapter }
|
|
688
|
+
```
|
|
471
689
|
|
|
472
690
|
---
|
|
473
691
|
|
|
474
|
-
## Provider packages
|
|
692
|
+
## Provider packages
|
|
693
|
+
|
|
694
|
+
| Provider ID | Package | API key env |
|
|
695
|
+
|-------------|---------|-------------|
|
|
696
|
+
| `openai` | `@x12i/ai-provider-openai` | `OPENAI_API_KEY` |
|
|
697
|
+
| `grok` | `@x12i/ai-provider-grok` | `GROK_API_KEY` |
|
|
698
|
+
| `anthropic` | `@x12i/ai-provider-anthropic` | `ANTHROPIC_API_KEY` |
|
|
699
|
+
| `google` | `@x12i/ai-provider-google` | `GOOGLE_API_KEY` |
|
|
700
|
+
| `groq` | `@x12i/ai-provider-groq` | `GROQ_API_KEY` |
|
|
701
|
+
| OpenRouter mode | `@x12i/ai-provider-openai` (bundled) | `OPENROUTER_API_KEY` |
|
|
475
702
|
|
|
476
|
-
|
|
703
|
+
Missing packages produce a clear `ProviderNotInstalledError` with install instructions.
|
|
477
704
|
|
|
478
|
-
|
|
705
|
+
---
|
|
706
|
+
|
|
707
|
+
## Development and testing
|
|
708
|
+
|
|
709
|
+
```bash
|
|
710
|
+
npm run build # compile TypeScript
|
|
711
|
+
npm test # build + run all .tests/**/*.test.js
|
|
712
|
+
npm run test:openai # live OpenAI call (requires OPENAI_API_KEY)
|
|
713
|
+
npm run test:openrouter
|
|
714
|
+
npm run test:reasoning
|
|
715
|
+
npm run erc:verify # ERC manifest verification
|
|
716
|
+
```
|
|
479
717
|
|
|
480
|
-
**
|
|
481
|
-
- All major providers: OpenAI, Anthropic, Google, xAI (Grok), Groq, Meta, Mistral, Cohere, etc.
|
|
482
|
-
- 67 total providers from the OpenRouter catalog
|
|
483
|
-
- 353 models with full capability support
|
|
718
|
+
Requires **Node.js ≥ 18**.
|
|
484
719
|
|
|
485
|
-
|
|
720
|
+
---
|
|
486
721
|
|
|
487
|
-
|
|
488
|
-
* Provider `grok` requires `@x12i/ai-provider-grok`
|
|
489
|
-
* **OpenRouter mode**: Only requires `@x12i/ai-provider-openai` to access all OpenRouter-supported models
|
|
722
|
+
## Related documentation
|
|
490
723
|
|
|
491
|
-
|
|
724
|
+
| Document | Topic |
|
|
725
|
+
|----------|-------|
|
|
726
|
+
| [Configuration guide](./docs/CONFIGURATION_GUIDE.md) | Full request/config reference |
|
|
727
|
+
| [Environment variables](./docs/environment-variables.md) | Complete env var list |
|
|
728
|
+
| [Reasoning integration](./docs/reasoning-integration.md) | Reasoning API details |
|
|
729
|
+
| [Reasoning supported models](./docs/reasoning-supported-models.md) | Model registry |
|
|
730
|
+
| [Request/response flow](./docs/request-response-flow.md) | Internal flow |
|
|
731
|
+
| [Debugging no-provider error](./docs/debugging-no-provider-error.md) | OpenRouter troubleshooting |
|
|
732
|
+
| [Normalization fields](./docs/normalization-field-support.md) | Output contract and cost |
|
|
492
733
|
|
|
493
734
|
---
|
|
494
735
|
|
|
495
736
|
## License
|
|
496
737
|
|
|
497
|
-
|
|
738
|
+
MIT
|