@starci/ai-router 0.7.0 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +14 -0
- package/README.md +15 -0
- package/dist/ai-router.service.d.ts +3 -0
- package/dist/ai-router.service.js +4 -0
- package/dist/gateway/index.d.ts +2 -0
- package/dist/gateway/index.js +1 -0
- package/dist/known-models.d.ts +81 -0
- package/dist/known-models.js +142 -0
- package/dist/router.js +2 -0
- package/dist/types.d.ts +3 -0
- package/package.json +3 -3
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,19 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.8.0 - 2026-11-16
|
|
4
|
+
|
|
5
|
+
The router knows the current Claude 5 and OpenAI models by id, offers OpenRouter's catalog at run time, lists what a caller may choose, and routes `auto-*` profiles by effort tier within the providers that have keys. Additive: nothing declared before changes.
|
|
6
|
+
|
|
7
|
+
- Added: `KNOWN_MODELS` and `knownModelConfigs("anthropic" | "openai")`: `claude-fable-5-2`, `claude-opus-5-5`, `claude-sonnet-5-5`, `claude-haiku-5-5`, `gpt-6-astra`, `gpt-6.1-sol`, `gpt-6.1-terra`, `gpt-6-luna`, each with chat, streaming, json, tools and vision and a 1M (Claude) or 1.05M (OpenAI) token window. Sources, read 2026-11-16: https://platform.claude.com/docs/en/models/overview (ids, window, image input, tool use, prices, cache-read share) and https://developers.openai.com/api/docs/models (ids, window, image input, function calling). Left out because the pages do not state it: Haiku 5.5's price (the page says "From $0.10 / $0.50"), every Claude cache-write price, and OpenAI prices. Opus 5.5 and Sonnet 5.5 carry `forcedToolUse: false`.
|
|
8
|
+
- Added: `fetchOpenRouterModels({ baseUrl?, apiKey?, fetch?, ttlMs?, now? })` reads OpenRouter's `GET /api/v1/models` (https://openrouter.ai/docs/api-reference/list-available-models), cached per base URL (one hour by default), into `ModelConfig`s: the id sent is `<vendor>/<model>`, the name a caller uses is `openrouter/<vendor>/<model>` (an alias); vision from `architecture.input_modalities`, tools from `supported_parameters`, the window from `context_length`, each left out when the entry omits it; prices are not read. `clearOpenRouterCatalogCache()`.
|
|
9
|
+
- Added: `defaultProviderSpecs({ openRouterModels? })` for `poolFromEnv`: Anthropic, OpenAI and OpenRouter, each present only when its key is in env (`ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `OPENROUTER_API_KEY`); without a key the provider and its models are absent and nothing fails at boot.
|
|
10
|
+
- Added: `availableModels()` on the router and `AiRouterService` (and `availableModels(config)`): `{ id, provider, label, vision, tools, contextTokens? }` per chat model per provider in the pool, so an app lists choices. `AvailableModel`.
|
|
11
|
+
- Added: `withAutoProfiles(config)` adds `auto-low`, `auto-medium`, `auto-high`, `auto-max` and `auto` (medium): each starts at its effort tier, then the tiers below, then those above, over the known models the pool serves, so it never reaches a provider without a key. `EFFORT_TIERS`, `EffortTier`. The tier of a model is this package's ranking from the vendors' descriptions and prices, not a vendor claim. A profile the app declares is kept.
|
|
12
|
+
|
|
13
|
+
### Migration from 0.7
|
|
14
|
+
|
|
15
|
+
- Code that implements `AiRouter` adds `availableModels`.
|
|
16
|
+
|
|
3
17
|
## 0.7.0 - 2026-11-13
|
|
4
18
|
|
|
5
19
|
One virtual key, two protocols: the gateway speaks Anthropic Messages beside OpenAI, a LangChain app asks for the client
|
package/README.md
CHANGED
|
@@ -95,6 +95,21 @@ A virtual key carries the profiles it may call and, optionally, an allowlist of
|
|
|
95
95
|
default). A model it names that is neither is refused 403 `model_not_allowed`, never remapped; `/v1/models` lists only
|
|
96
96
|
what it may call; its quotas count every call whatever the model. See [key scopes](docs/design.md#key-scopes).
|
|
97
97
|
|
|
98
|
+
## Known models, OpenRouter and `auto`
|
|
99
|
+
|
|
100
|
+
The router knows the Claude 5 ids (`claude-fable-5-2`, `claude-opus-5-5`, `claude-sonnet-5-5`, `claude-haiku-5-5`) and the OpenAI GPT-6 ids (`gpt-6-astra`, `gpt-6.1-sol`, `gpt-6.1-terra`, `gpt-6-luna`) with the facts their vendors state (the sources and what is left out are in the 0.8.0 changelog entry). OpenRouter's catalog is read at run time. A provider is in the pool only when its key is in env, so a deployment with one key lists one provider's models and nothing fails at boot.
|
|
101
|
+
|
|
102
|
+
```ts
|
|
103
|
+
import { availableModels, createRouter, defaultProviderSpecs, fetchOpenRouterModels, poolFromEnv, withAutoProfiles } from "@starci/ai-router/gateway"
|
|
104
|
+
|
|
105
|
+
const openRouterModels = process.env.OPENROUTER_API_KEY ? await fetchOpenRouterModels({ apiKey: process.env.OPENROUTER_API_KEY }) : undefined
|
|
106
|
+
const router = createRouter(withAutoProfiles(poolFromEnv({ providers: defaultProviderSpecs({ openRouterModels }) })))
|
|
107
|
+
|
|
108
|
+
router.availableModels() // [{ id: "claude-opus-5-5", provider: "anthropic", label: "Claude Opus 5.5", vision: true, tools: true, contextTokens: 1000000 }, ...]
|
|
109
|
+
await router.chat({ messages }, { profile: "auto-high" }) // opus first, then the tiers below, then above; only providers with keys
|
|
110
|
+
await router.chat({ model: "openrouter/vendor/model", messages }) // sent to OpenRouter as vendor/model
|
|
111
|
+
```
|
|
112
|
+
|
|
98
113
|
## Run it as a gateway
|
|
99
114
|
|
|
100
115
|
One process (one container) holds the pool, the virtual keys and the admin API. The package declares no `bin` (the
|
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
import type { AvailableModel } from "./known-models.js";
|
|
1
2
|
import type { OnApplicationShutdown } from "@nestjs/common";
|
|
2
3
|
import type { AiRouter, AnthropicMessageResult, AnthropicMessagesRequest, AnthropicStream, CallOptions, ChatCompletion, ChatCompletionRequest, ChatStream, EmbeddingsRequest, EmbeddingsResponse, HealthSnapshot, ResetTarget, RouterCatalog, RouterEvents, StatsSnapshot } from "./types.js";
|
|
3
4
|
import type { AiRouterHandlers } from "./ai-router.options.js";
|
|
@@ -30,6 +31,8 @@ export declare class AiRouterService<Profiles extends string = string> implement
|
|
|
30
31
|
reset(target?: ResetTarget): Promise<void>;
|
|
31
32
|
/** Lists the pool's profiles and models by name. */
|
|
32
33
|
catalog(): RouterCatalog;
|
|
34
|
+
/** The chat models a caller may choose, from the providers whose keys are configured. */
|
|
35
|
+
availableModels(): AvailableModel[];
|
|
33
36
|
/** Aborts every call still in flight through this service and releases its handlers. */
|
|
34
37
|
onApplicationShutdown(): void;
|
|
35
38
|
}
|
|
@@ -70,6 +70,10 @@ export class AiRouterService {
|
|
|
70
70
|
catalog() {
|
|
71
71
|
return this.router.catalog();
|
|
72
72
|
}
|
|
73
|
+
/** The chat models a caller may choose, from the providers whose keys are configured. */
|
|
74
|
+
availableModels() {
|
|
75
|
+
return this.router.availableModels();
|
|
76
|
+
}
|
|
73
77
|
/** Aborts every call still in flight through this service and releases its handlers. */
|
|
74
78
|
onApplicationShutdown() {
|
|
75
79
|
this.#shutdown.abort();
|
package/dist/gateway/index.d.ts
CHANGED
|
@@ -5,6 +5,8 @@ export { classifyFailure, readProviderError } from "../classify.js";
|
|
|
5
5
|
export type { Classified, ProviderErrorInfo } from "../classify.js";
|
|
6
6
|
export { keysFromEnv, poolFromEnv, providersFromEnv, virtualKeyFromEnv } from "../env.js";
|
|
7
7
|
export type { EnvPoolSpec, EnvProviderSpec, EnvSource, VirtualKeyEnvSpec } from "../env.js";
|
|
8
|
+
export { availableModels, clearOpenRouterCatalogCache, defaultProviderSpecs, EFFORT_TIERS, fetchOpenRouterModels, KNOWN_MODELS, knownModelConfigs, withAutoProfiles } from "../known-models.js";
|
|
9
|
+
export type { AvailableModel, DefaultProvidersOptions, EffortTier, KnownModel, OpenRouterCatalogOptions } from "../known-models.js";
|
|
8
10
|
export { PROVIDER_PRESETS } from "../providers.js";
|
|
9
11
|
export type { Protocol, ProviderPreset, QuotaWindow } from "../providers.js";
|
|
10
12
|
export { openSqliteStateStore, redisStateStore, STATE_STORE_SCHEMA } from "../state-stores.js";
|
package/dist/gateway/index.js
CHANGED
|
@@ -2,6 +2,7 @@ export { AI_ROUTER_ERROR_CODES, AiRouterError } from "../errors.js";
|
|
|
2
2
|
export { createRouter } from "../router.js";
|
|
3
3
|
export { classifyFailure, readProviderError } from "../classify.js";
|
|
4
4
|
export { keysFromEnv, poolFromEnv, providersFromEnv, virtualKeyFromEnv } from "../env.js";
|
|
5
|
+
export { availableModels, clearOpenRouterCatalogCache, defaultProviderSpecs, EFFORT_TIERS, fetchOpenRouterModels, KNOWN_MODELS, knownModelConfigs, withAutoProfiles } from "../known-models.js";
|
|
5
6
|
export { PROVIDER_PRESETS } from "../providers.js";
|
|
6
7
|
export { openSqliteStateStore, redisStateStore, STATE_STORE_SCHEMA } from "../state-stores.js";
|
|
7
8
|
export { EMPTY_RECORD, MemoryStateStore, parseHealthRecord, toHealthRecord } from "../store.js";
|
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
import type { EnvProviderSpec } from "./env.js";
|
|
2
|
+
import type { FetchLike, ModelConfig, ModelCost, PoolConfig } from "./types.js";
|
|
3
|
+
/**
|
|
4
|
+
* The models the router knows by id, with the facts their vendors state. A fact a vendor's page does not state is left
|
|
5
|
+
* out, never guessed. Sources, read 2026-11-16:
|
|
6
|
+
*
|
|
7
|
+
* - Anthropic, https://platform.claude.com/docs/en/models/overview: ids, 1M-token context window, text and image input,
|
|
8
|
+
* tool use, price per million tokens (Fable 5.2 8 and 40, Opus 5.5 4 and 20, Sonnet 5.5 2 and 10) and the cache-read share
|
|
9
|
+
* of the input price (2.5% on Fable 5.2, 5% on Opus 5.5 and Sonnet 5.5). Haiku 5.5 is priced "From $0.10 / $0.50", so it
|
|
10
|
+
* carries no cost, and the page states no cache write price, so none is declared.
|
|
11
|
+
* - OpenAI, https://developers.openai.com/api/docs/models: ids, 1.05M-token context window, image input and function
|
|
12
|
+
* calling for the latest models. Their price is not declared here: the router prices what a deployment sets.
|
|
13
|
+
*
|
|
14
|
+
* `tier` is this package's own ranking by the vendors' descriptions and prices ("most demanding work" is `max`,
|
|
15
|
+
* "high-volume tasks" is `low`); it orders the `auto-*` profiles and states nothing a vendor claims.
|
|
16
|
+
*/
|
|
17
|
+
export type EffortTier = "low" | "medium" | "high" | "max";
|
|
18
|
+
export declare const EFFORT_TIERS: readonly EffortTier[];
|
|
19
|
+
export interface KnownModel {
|
|
20
|
+
readonly id: string;
|
|
21
|
+
readonly kind: "anthropic" | "openai";
|
|
22
|
+
readonly label: string;
|
|
23
|
+
readonly tier: EffortTier;
|
|
24
|
+
readonly vision: boolean;
|
|
25
|
+
readonly tools: boolean;
|
|
26
|
+
readonly contextWindow: number;
|
|
27
|
+
readonly cost?: ModelCost;
|
|
28
|
+
}
|
|
29
|
+
export declare const KNOWN_MODELS: readonly KnownModel[];
|
|
30
|
+
/** The known models of one provider kind as the `ModelConfig`s of a provider. */
|
|
31
|
+
export declare function knownModelConfigs(kind: "anthropic" | "openai"): ModelConfig[];
|
|
32
|
+
export interface OpenRouterCatalogOptions {
|
|
33
|
+
/** Default `https://openrouter.ai/api/v1`. */
|
|
34
|
+
readonly baseUrl?: string;
|
|
35
|
+
/** Sent as a bearer token when given; the endpoint's auth is optional (https://openrouter.ai/docs/api-reference/list-available-models). */
|
|
36
|
+
readonly apiKey?: string;
|
|
37
|
+
readonly fetch?: FetchLike;
|
|
38
|
+
/** How long a fetched catalog is reused. Default one hour; 0 never caches. */
|
|
39
|
+
readonly ttlMs?: number;
|
|
40
|
+
readonly now?: () => number;
|
|
41
|
+
}
|
|
42
|
+
/**
|
|
43
|
+
* OpenRouter's catalog as `ModelConfig`s: the id sent to OpenRouter is `<vendor>/<model>` and the name a caller uses is
|
|
44
|
+
* `openrouter/<vendor>/<model>` (an alias). The endpoint answers text-output models by default. Vision comes from
|
|
45
|
+
* `architecture.input_modalities` holding `image`, tools from `supported_parameters` holding `tools`, the window from
|
|
46
|
+
* `context_length`; an entry that omits a field leaves that fact out. Prices are not read: the page does not state their
|
|
47
|
+
* unit. The answer is cached per base URL for `ttlMs`.
|
|
48
|
+
*/
|
|
49
|
+
export declare function fetchOpenRouterModels(options?: OpenRouterCatalogOptions): Promise<ModelConfig[]>;
|
|
50
|
+
/** Forgets every cached OpenRouter catalog (tests, or a deployment that wants a refresh now). */
|
|
51
|
+
export declare function clearOpenRouterCatalogCache(): void;
|
|
52
|
+
export interface DefaultProvidersOptions {
|
|
53
|
+
/** Models for the `openrouter` provider, from `fetchOpenRouterModels`; without them it is left out (nothing to route). */
|
|
54
|
+
readonly openRouterModels?: readonly ModelConfig[];
|
|
55
|
+
}
|
|
56
|
+
/**
|
|
57
|
+
* Provider specs for `poolFromEnv`: Anthropic and OpenAI with the known models, and OpenRouter with the models given.
|
|
58
|
+
* Each is present only when its key is in env (`ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `OPENROUTER_API_KEY`): without a
|
|
59
|
+
* key the provider and its models are absent from the catalog, and nothing fails at boot.
|
|
60
|
+
*/
|
|
61
|
+
export declare function defaultProviderSpecs(options?: DefaultProvidersOptions): EnvProviderSpec[];
|
|
62
|
+
/** One model a caller may choose, from the providers whose keys are configured. */
|
|
63
|
+
export interface AvailableModel {
|
|
64
|
+
/** The name to ask for: the model id, or `openrouter/<vendor>/<model>` on OpenRouter. */
|
|
65
|
+
readonly id: string;
|
|
66
|
+
/** The id of the provider serving it. */
|
|
67
|
+
readonly provider: string;
|
|
68
|
+
readonly label: string;
|
|
69
|
+
readonly vision: boolean;
|
|
70
|
+
readonly tools: boolean;
|
|
71
|
+
readonly contextTokens?: number;
|
|
72
|
+
}
|
|
73
|
+
/** The chat models of a pool, one per provider serving them; a pool built from env holds only providers with keys. */
|
|
74
|
+
export declare function availableModels(config: PoolConfig): AvailableModel[];
|
|
75
|
+
/**
|
|
76
|
+
* The pool with `auto-low`, `auto-medium`, `auto-high`, `auto-max` and `auto` (the medium one) profiles added: each tries
|
|
77
|
+
* the known models the pool serves, starting at its own effort tier, then the tiers below it from the nearest down, then
|
|
78
|
+
* those above from the nearest up. A profile with no known model to try is not added, and a profile the app declares is
|
|
79
|
+
* kept as it is. The pool holds only providers with keys, so `auto` never leaves them.
|
|
80
|
+
*/
|
|
81
|
+
export declare function withAutoProfiles<P extends string>(config: PoolConfig<P>): PoolConfig;
|
|
@@ -0,0 +1,142 @@
|
|
|
1
|
+
import { matchesOf } from "./config.js";
|
|
2
|
+
import { AiRouterError } from "./errors.js";
|
|
3
|
+
export const EFFORT_TIERS = ["low", "medium", "high", "max"];
|
|
4
|
+
const CLAUDE_CONTEXT = 1_000_000;
|
|
5
|
+
const OPENAI_CONTEXT = 1_050_000;
|
|
6
|
+
export const KNOWN_MODELS = Object.freeze([
|
|
7
|
+
{ id: "claude-fable-5-2", kind: "anthropic", label: "Claude Fable 5.2", tier: "max", vision: true, tools: true, contextWindow: CLAUDE_CONTEXT, cost: { input: 8, output: 40, cacheRead: 0.2 } },
|
|
8
|
+
{ id: "claude-opus-5-5", kind: "anthropic", label: "Claude Opus 5.5", tier: "high", vision: true, tools: true, contextWindow: CLAUDE_CONTEXT, cost: { input: 4, output: 20, cacheRead: 0.2 } },
|
|
9
|
+
{ id: "claude-sonnet-5-5", kind: "anthropic", label: "Claude Sonnet 5.5", tier: "medium", vision: true, tools: true, contextWindow: CLAUDE_CONTEXT, cost: { input: 2, output: 10, cacheRead: 0.1 } },
|
|
10
|
+
{ id: "claude-haiku-5-5", kind: "anthropic", label: "Claude Haiku 5.5", tier: "low", vision: true, tools: true, contextWindow: CLAUDE_CONTEXT },
|
|
11
|
+
{ id: "gpt-6-astra", kind: "openai", label: "GPT-6 Astra", tier: "max", vision: true, tools: true, contextWindow: OPENAI_CONTEXT },
|
|
12
|
+
{ id: "gpt-6.1-sol", kind: "openai", label: "GPT-6.1 Sol", tier: "high", vision: true, tools: true, contextWindow: OPENAI_CONTEXT },
|
|
13
|
+
{ id: "gpt-6.1-terra", kind: "openai", label: "GPT-6.1 Terra", tier: "medium", vision: true, tools: true, contextWindow: OPENAI_CONTEXT },
|
|
14
|
+
{ id: "gpt-6-luna", kind: "openai", label: "GPT-6 Luna", tier: "low", vision: true, tools: true, contextWindow: OPENAI_CONTEXT },
|
|
15
|
+
]);
|
|
16
|
+
function capabilitiesOf(model) {
|
|
17
|
+
return ["chat", "streaming", "json", ...(model.tools ? ["tools"] : []), ...(model.vision ? ["vision"] : [])];
|
|
18
|
+
}
|
|
19
|
+
/** The known models of one provider kind as the `ModelConfig`s of a provider. */
|
|
20
|
+
export function knownModelConfigs(kind) {
|
|
21
|
+
return KNOWN_MODELS.filter((model) => model.kind === kind).map((model) => ({
|
|
22
|
+
id: model.id,
|
|
23
|
+
contextWindow: model.contextWindow,
|
|
24
|
+
capabilities: capabilitiesOf(model),
|
|
25
|
+
...(model.cost === undefined ? {} : { cost: model.cost }),
|
|
26
|
+
// Claude Opus 5.5 and Sonnet 5.5 refuse a forced tool choice (docs/design.md, "A model's own request shape").
|
|
27
|
+
...(model.id === "claude-opus-5-5" || model.id === "claude-sonnet-5-5" ? { forcedToolUse: false } : {}),
|
|
28
|
+
}));
|
|
29
|
+
}
|
|
30
|
+
function strings(value) {
|
|
31
|
+
return Array.isArray(value) ? value.filter((item) => typeof item === "string") : [];
|
|
32
|
+
}
|
|
33
|
+
const cache = new Map();
|
|
34
|
+
/**
|
|
35
|
+
* OpenRouter's catalog as `ModelConfig`s: the id sent to OpenRouter is `<vendor>/<model>` and the name a caller uses is
|
|
36
|
+
* `openrouter/<vendor>/<model>` (an alias). The endpoint answers text-output models by default. Vision comes from
|
|
37
|
+
* `architecture.input_modalities` holding `image`, tools from `supported_parameters` holding `tools`, the window from
|
|
38
|
+
* `context_length`; an entry that omits a field leaves that fact out. Prices are not read: the page does not state their
|
|
39
|
+
* unit. The answer is cached per base URL for `ttlMs`.
|
|
40
|
+
*/
|
|
41
|
+
export async function fetchOpenRouterModels(options = {}) {
|
|
42
|
+
const baseUrl = (options.baseUrl ?? "https://openrouter.ai/api/v1").replace(/\/+$/, "");
|
|
43
|
+
const now = options.now ?? Date.now;
|
|
44
|
+
const ttl = options.ttlMs ?? 3_600_000;
|
|
45
|
+
const hit = cache.get(baseUrl);
|
|
46
|
+
if (hit !== undefined && ttl > 0 && now() - hit.at < ttl)
|
|
47
|
+
return hit.models;
|
|
48
|
+
const doFetch = options.fetch ?? ((input, init) => fetch(input, init));
|
|
49
|
+
const response = await doFetch(`${baseUrl}/models`, { method: "GET", headers: options.apiKey === undefined ? {} : { authorization: `Bearer ${options.apiKey}` } });
|
|
50
|
+
if (!response.ok)
|
|
51
|
+
throw new AiRouterError({ errorClass: "config", message: `OpenRouter catalog answered ${String(response.status)}` });
|
|
52
|
+
const body = (await response.json());
|
|
53
|
+
const models = [];
|
|
54
|
+
for (const entry of Array.isArray(body.data) ? body.data : []) {
|
|
55
|
+
if (typeof entry.id !== "string" || entry.id === "")
|
|
56
|
+
continue;
|
|
57
|
+
const outputs = strings(entry.architecture?.output_modalities);
|
|
58
|
+
if (outputs.length > 0 && !outputs.includes("text"))
|
|
59
|
+
continue;
|
|
60
|
+
models.push({
|
|
61
|
+
id: entry.id,
|
|
62
|
+
aliases: [`openrouter/${entry.id}`],
|
|
63
|
+
capabilities: capabilitiesOf({ tools: strings(entry.supported_parameters).includes("tools"), vision: strings(entry.architecture?.input_modalities).includes("image") }),
|
|
64
|
+
...(typeof entry.context_length === "number" && entry.context_length > 0 ? { contextWindow: entry.context_length } : {}),
|
|
65
|
+
});
|
|
66
|
+
}
|
|
67
|
+
cache.set(baseUrl, { at: now(), models });
|
|
68
|
+
return models;
|
|
69
|
+
}
|
|
70
|
+
/** Forgets every cached OpenRouter catalog (tests, or a deployment that wants a refresh now). */
|
|
71
|
+
export function clearOpenRouterCatalogCache() {
|
|
72
|
+
cache.clear();
|
|
73
|
+
}
|
|
74
|
+
/**
|
|
75
|
+
* Provider specs for `poolFromEnv`: Anthropic and OpenAI with the known models, and OpenRouter with the models given.
|
|
76
|
+
* Each is present only when its key is in env (`ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `OPENROUTER_API_KEY`): without a
|
|
77
|
+
* key the provider and its models are absent from the catalog, and nothing fails at boot.
|
|
78
|
+
*/
|
|
79
|
+
export function defaultProviderSpecs(options = {}) {
|
|
80
|
+
const specs = [
|
|
81
|
+
{ kind: "anthropic", models: knownModelConfigs("anthropic") },
|
|
82
|
+
{ kind: "openai", models: knownModelConfigs("openai") },
|
|
83
|
+
];
|
|
84
|
+
if (options.openRouterModels !== undefined)
|
|
85
|
+
specs.push({ kind: "openrouter", models: options.openRouterModels });
|
|
86
|
+
return specs;
|
|
87
|
+
}
|
|
88
|
+
/** The chat models of a pool, one per provider serving them; a pool built from env holds only providers with keys. */
|
|
89
|
+
export function availableModels(config) {
|
|
90
|
+
const out = [];
|
|
91
|
+
for (const provider of config.providers) {
|
|
92
|
+
const kind = provider.kind ?? "openai-compatible";
|
|
93
|
+
for (const model of provider.models) {
|
|
94
|
+
const capabilities = model.capabilities ?? ["chat", "streaming", "tools", "json"];
|
|
95
|
+
if (!capabilities.includes("chat"))
|
|
96
|
+
continue;
|
|
97
|
+
out.push({
|
|
98
|
+
id: kind === "openrouter" ? `openrouter/${model.id.replace(/^openrouter\//, "")}` : model.id,
|
|
99
|
+
provider: provider.id,
|
|
100
|
+
label: KNOWN_MODELS.find((known) => known.id === model.id)?.label ?? model.id,
|
|
101
|
+
vision: capabilities.includes("vision"),
|
|
102
|
+
tools: capabilities.includes("tools"),
|
|
103
|
+
...(model.contextWindow === undefined ? {} : { contextTokens: model.contextWindow }),
|
|
104
|
+
});
|
|
105
|
+
}
|
|
106
|
+
}
|
|
107
|
+
return out;
|
|
108
|
+
}
|
|
109
|
+
/**
|
|
110
|
+
* The pool with `auto-low`, `auto-medium`, `auto-high`, `auto-max` and `auto` (the medium one) profiles added: each tries
|
|
111
|
+
* the known models the pool serves, starting at its own effort tier, then the tiers below it from the nearest down, then
|
|
112
|
+
* those above from the nearest up. A profile with no known model to try is not added, and a profile the app declares is
|
|
113
|
+
* kept as it is. The pool holds only providers with keys, so `auto` never leaves them.
|
|
114
|
+
*/
|
|
115
|
+
export function withAutoProfiles(config) {
|
|
116
|
+
const served = KNOWN_MODELS.filter((model) => matchesOf(config, { model: model.id }).length > 0);
|
|
117
|
+
const profiles = { ...(config.profiles ?? {}) };
|
|
118
|
+
const add = (name, tier) => {
|
|
119
|
+
if (name in profiles)
|
|
120
|
+
return;
|
|
121
|
+
const at = EFFORT_TIERS.indexOf(tier);
|
|
122
|
+
const distance = (model) => {
|
|
123
|
+
const gap = EFFORT_TIERS.indexOf(model.tier) - at;
|
|
124
|
+
return gap <= 0 ? -gap : 10 + gap;
|
|
125
|
+
};
|
|
126
|
+
const tiers = [];
|
|
127
|
+
let current;
|
|
128
|
+
for (const model of [...served].sort((a, b) => distance(a) - distance(b))) {
|
|
129
|
+
if (model.tier === current)
|
|
130
|
+
tiers[tiers.length - 1]?.push({ model: model.id });
|
|
131
|
+
else
|
|
132
|
+
tiers.push([{ model: model.id }]);
|
|
133
|
+
current = model.tier;
|
|
134
|
+
}
|
|
135
|
+
if (tiers.length > 0)
|
|
136
|
+
profiles[name] = { candidates: tiers };
|
|
137
|
+
};
|
|
138
|
+
for (const tier of EFFORT_TIERS)
|
|
139
|
+
add(`auto-${tier}`, tier);
|
|
140
|
+
add("auto", "medium");
|
|
141
|
+
return { ...config, profiles };
|
|
142
|
+
}
|
package/dist/router.js
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
import { isPlainRecord } from "@starci/common";
|
|
2
2
|
import { AttemptError } from "./attempt.js";
|
|
3
3
|
import { classifyFailure } from "./classify.js";
|
|
4
|
+
import { availableModels } from "./known-models.js";
|
|
4
5
|
import { buildCatalog, resolvePolicy, resolveSettings, validateConfig } from "./config.js";
|
|
5
6
|
import { AiRouterError, isRetryable, scrub } from "./errors.js";
|
|
6
7
|
import { Emitter } from "./events.js";
|
|
@@ -646,6 +647,7 @@ export function createRouter(config) {
|
|
|
646
647
|
return { ...value, routing };
|
|
647
648
|
},
|
|
648
649
|
catalog: () => catalog,
|
|
650
|
+
availableModels: () => availableModels(config),
|
|
649
651
|
health: () => state.health(),
|
|
650
652
|
stats: () => state.stats(),
|
|
651
653
|
on: (event, handler) => emitter.on(event, handler),
|
package/dist/types.d.ts
CHANGED
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
import type { AvailableModel } from "./known-models.js";
|
|
1
2
|
import type { StateStore } from "./store.js";
|
|
2
3
|
/** A value or a promise of it: store adapters and key resolvers may answer either way. */
|
|
3
4
|
export type MaybePromise<T> = T | Promise<T>;
|
|
@@ -621,6 +622,8 @@ export interface AiRouter<Profiles extends string = string> {
|
|
|
621
622
|
reset(target?: ResetTarget): Promise<void>;
|
|
622
623
|
/** Lists the pool's profiles and models by name, with no key: what a caller may ask for. */
|
|
623
624
|
catalog(): RouterCatalog;
|
|
625
|
+
/** The chat models a caller may choose, one per provider serving them: only providers in the pool, which has the ones with keys. */
|
|
626
|
+
availableModels(): AvailableModel[];
|
|
624
627
|
}
|
|
625
628
|
/** One model of the pool as the catalog lists it: one entry per configured id, merged across the providers serving it. */
|
|
626
629
|
export interface CatalogModel {
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@starci/ai-router",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.8.0",
|
|
4
4
|
"description": "A router for chat and embeddings calls across many providers (Anthropic, OpenRouter, Qwen, DeepSeek, OpenAI, Gemini, Groq and any OpenAI-compatible endpoint), each with an array of keys, behind one virtual key: a gateway that speaks the OpenAI and the Anthropic Messages protocols with virtual keys, quotas and usage, LangChain clients over it (@starci/ai-router/langchain), a NestJS module (AiRouterModule, AiRouterService, AiRouterGatewayModule), and createRouter for plain Node, with health-aware selection, a persisted key blacklist for spent quotas and refused secrets, circuit breakers, rate-limit awareness, failover across providers and models by profile, prompt-cache breakpoints and cache-priced usage, pinned embeddings, config from env and redacted telemetry.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"ai",
|
|
@@ -138,6 +138,6 @@
|
|
|
138
138
|
"optional": "true"
|
|
139
139
|
}
|
|
140
140
|
},
|
|
141
|
-
"gitHead": "
|
|
142
|
-
"starciSource": "
|
|
141
|
+
"gitHead": "a93642ee8010c914909df92f65ecf644493037ce",
|
|
142
|
+
"starciSource": "10805e00e49adcc5d1ccdccc73762ae88c16b41d02037239a0d475e1ded1afdb"
|
|
143
143
|
}
|