@tanstack/ai-llmgateway 0.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +94 -0
- package/dist/esm/adapters/summarize.d.ts +51 -0
- package/dist/esm/adapters/summarize.js +55 -0
- package/dist/esm/adapters/summarize.js.map +1 -0
- package/dist/esm/adapters/text.d.ts +64 -0
- package/dist/esm/adapters/text.js +67 -0
- package/dist/esm/adapters/text.js.map +1 -0
- package/dist/esm/index.d.ts +14 -0
- package/dist/esm/index.js +5 -0
- package/dist/esm/message-types.d.ts +116 -0
- package/dist/esm/model-meta.d.ts +374 -0
- package/dist/esm/model-meta.js +363 -0
- package/dist/esm/model-meta.js.map +1 -0
- package/dist/esm/text/text-provider-options.d.ts +97 -0
- package/dist/esm/utils/client.d.ts +16 -0
- package/dist/esm/utils/client.js +29 -0
- package/dist/esm/utils/client.js.map +1 -0
- package/package.json +74 -0
- package/src/adapters/summarize.ts +79 -0
- package/src/adapters/text.ts +119 -0
- package/src/index.ts +52 -0
- package/src/message-types.ts +130 -0
- package/src/model-meta.ts +516 -0
- package/src/text/text-provider-options.ts +128 -0
- package/src/utils/client.ts +36 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2025 Tanner Linsley
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
# @tanstack/ai-llmgateway
|
|
2
|
+
|
|
3
|
+
[LLM Gateway](https://llmgateway.io) adapter for TanStack AI — one OpenAI-compatible endpoint that routes chat, tool calling, and structured outputs to hundreds of models across many providers.
|
|
4
|
+
|
|
5
|
+
## Installation
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
npm install @tanstack/ai-llmgateway
|
|
9
|
+
# or
|
|
10
|
+
pnpm add @tanstack/ai-llmgateway
|
|
11
|
+
# or
|
|
12
|
+
yarn add @tanstack/ai-llmgateway
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
## Setup
|
|
16
|
+
|
|
17
|
+
Get your API key from the [LLM Gateway dashboard](https://llmgateway.io) and set it as an environment variable:
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
export LLM_GATEWAY_API_KEY="llmgtwy_..."
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
## Usage
|
|
24
|
+
|
|
25
|
+
### Text/Chat Adapter
|
|
26
|
+
|
|
27
|
+
```typescript
|
|
28
|
+
import { llmGatewayText } from '@tanstack/ai-llmgateway'
|
|
29
|
+
import { chat } from '@tanstack/ai'
|
|
30
|
+
|
|
31
|
+
const stream = chat({
|
|
32
|
+
adapter: llmGatewayText('gpt-5.6-terra'),
|
|
33
|
+
messages: [
|
|
34
|
+
{ role: 'user', content: 'Explain quantum computing in simple terms' },
|
|
35
|
+
],
|
|
36
|
+
})
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
### With Explicit API Key
|
|
40
|
+
|
|
41
|
+
```typescript
|
|
42
|
+
import { createLLMGatewayText } from '@tanstack/ai-llmgateway'
|
|
43
|
+
|
|
44
|
+
const adapter = createLLMGatewayText('gpt-5.6-terra', 'llmgtwy_api_key')
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
### Self-Hosted Gateways
|
|
48
|
+
|
|
49
|
+
LLM Gateway is open source and self-hostable — point `baseURL` at your own deployment:
|
|
50
|
+
|
|
51
|
+
```typescript
|
|
52
|
+
import { createLLMGatewayText } from '@tanstack/ai-llmgateway'
|
|
53
|
+
|
|
54
|
+
const adapter = createLLMGatewayText('gpt-5.6-terra', 'llmgtwy_api_key', {
|
|
55
|
+
baseURL: 'https://gateway.example.com/v1',
|
|
56
|
+
})
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
## Models
|
|
60
|
+
|
|
61
|
+
Any model listed on [llmgateway.io/models](https://llmgateway.io/models) works — pass its id as the model name. A curated set of flagship models additionally carries per-model type metadata (input modalities, provider options) with autocomplete, including `gpt-5.6-terra`, `claude-sonnet-5`, `gemini-pro-latest`, `kimi-k3`, `glm-5.2`, `deepseek-v4-pro`, and more (see `LLMGATEWAY_CHAT_MODELS`).
|
|
62
|
+
|
|
63
|
+
Model ids accept an optional `provider/` prefix to pin routing to a specific provider:
|
|
64
|
+
|
|
65
|
+
```typescript
|
|
66
|
+
llmGatewayText('kimi-k3') // gateway picks the best available provider
|
|
67
|
+
llmGatewayText('moonshot/kimi-k3') // always routed to Moonshot
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
## Features
|
|
71
|
+
|
|
72
|
+
- ✅ Streaming chat completions
|
|
73
|
+
- ✅ Structured output (JSON Schema)
|
|
74
|
+
- ✅ Function/tool calling
|
|
75
|
+
- ✅ Multimodal input (text + images for vision models)
|
|
76
|
+
- ✅ Reasoning output (`reasoning_content` deltas from reasoning models)
|
|
77
|
+
- ✅ Summarization (`llmGatewaySummarize`)
|
|
78
|
+
|
|
79
|
+
## Tree-Shakeable Adapters
|
|
80
|
+
|
|
81
|
+
This package uses tree-shakeable adapters, so you only import what you need:
|
|
82
|
+
|
|
83
|
+
```typescript
|
|
84
|
+
// Text/chat only
|
|
85
|
+
import { llmGatewayText } from '@tanstack/ai-llmgateway'
|
|
86
|
+
|
|
87
|
+
// Summarization only
|
|
88
|
+
import { llmGatewaySummarize } from '@tanstack/ai-llmgateway'
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
## Documentation
|
|
92
|
+
|
|
93
|
+
- [TanStack AI Documentation](https://tanstack.com/ai)
|
|
94
|
+
- [LLM Gateway Documentation](https://docs.llmgateway.io)
|
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
import { ChatStreamSummarizeAdapter, InferTextProviderOptions } from '@tanstack/ai/adapters';
|
|
2
|
+
import { LLMGatewayTextAdapter } from './text.js';
|
|
3
|
+
import { LLMGatewayModelId } from '../model-meta.js';
|
|
4
|
+
import { LLMGatewayClientConfig } from '../utils/client.js';
|
|
5
|
+
/**
|
|
6
|
+
* Configuration for LLM Gateway summarize adapter
|
|
7
|
+
*/
|
|
8
|
+
export interface LLMGatewaySummarizeConfig extends LLMGatewayClientConfig {
|
|
9
|
+
}
|
|
10
|
+
/** Model type for LLM Gateway summarization */
|
|
11
|
+
export type LLMGatewaySummarizeModel = LLMGatewayModelId;
|
|
12
|
+
/**
|
|
13
|
+
* Creates an LLM Gateway summarize adapter with explicit API key.
|
|
14
|
+
* Type resolution happens here at the call site.
|
|
15
|
+
*
|
|
16
|
+
* @param model - The model id (e.g., 'gpt-5.6-terra')
|
|
17
|
+
* @param apiKey - Your LLM Gateway API key
|
|
18
|
+
* @param config - Optional additional configuration
|
|
19
|
+
* @returns Configured LLM Gateway summarize adapter instance with resolved types
|
|
20
|
+
*
|
|
21
|
+
* @example
|
|
22
|
+
* ```typescript
|
|
23
|
+
* const adapter = createLLMGatewaySummarize('gpt-5.6-terra', "llmgtwy_...");
|
|
24
|
+
* ```
|
|
25
|
+
*/
|
|
26
|
+
export declare function createLLMGatewaySummarize<TModel extends LLMGatewaySummarizeModel>(model: TModel, apiKey: string, config?: Omit<LLMGatewaySummarizeConfig, 'apiKey'>): ChatStreamSummarizeAdapter<TModel, InferTextProviderOptions<LLMGatewayTextAdapter<TModel>>>;
|
|
27
|
+
/**
|
|
28
|
+
* Creates an LLM Gateway summarize adapter with automatic API key detection
|
|
29
|
+
* from environment variables. Type resolution happens here at the call site.
|
|
30
|
+
*
|
|
31
|
+
* Looks for `LLM_GATEWAY_API_KEY` in:
|
|
32
|
+
* - `process.env` (Node.js)
|
|
33
|
+
* - `window.env` (Browser with injected env)
|
|
34
|
+
*
|
|
35
|
+
* @param model - The model id (e.g., 'gpt-5.6-terra')
|
|
36
|
+
* @param config - Optional configuration (excluding apiKey which is auto-detected)
|
|
37
|
+
* @returns Configured LLM Gateway summarize adapter instance with resolved types
|
|
38
|
+
* @throws Error if LLM_GATEWAY_API_KEY is not found in environment
|
|
39
|
+
*
|
|
40
|
+
* @example
|
|
41
|
+
* ```typescript
|
|
42
|
+
* // Automatically uses LLM_GATEWAY_API_KEY from environment
|
|
43
|
+
* const adapter = llmGatewaySummarize('gpt-5.6-terra');
|
|
44
|
+
*
|
|
45
|
+
* await summarize({
|
|
46
|
+
* adapter,
|
|
47
|
+
* text: "Long article text..."
|
|
48
|
+
* });
|
|
49
|
+
* ```
|
|
50
|
+
*/
|
|
51
|
+
export declare function llmGatewaySummarize<TModel extends LLMGatewaySummarizeModel>(model: TModel, config?: Omit<LLMGatewaySummarizeConfig, 'apiKey'>): ChatStreamSummarizeAdapter<TModel, InferTextProviderOptions<LLMGatewayTextAdapter<TModel>>>;
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
import { getLLMGatewayApiKeyFromEnv } from "../utils/client.js";
|
|
2
|
+
import { LLMGatewayTextAdapter } from "./text.js";
|
|
3
|
+
import { ChatStreamSummarizeAdapter } from "@tanstack/ai/adapters";
|
|
4
|
+
//#region src/adapters/summarize.ts
|
|
5
|
+
/**
|
|
6
|
+
* Creates an LLM Gateway summarize adapter with explicit API key.
|
|
7
|
+
* Type resolution happens here at the call site.
|
|
8
|
+
*
|
|
9
|
+
* @param model - The model id (e.g., 'gpt-5.6-terra')
|
|
10
|
+
* @param apiKey - Your LLM Gateway API key
|
|
11
|
+
* @param config - Optional additional configuration
|
|
12
|
+
* @returns Configured LLM Gateway summarize adapter instance with resolved types
|
|
13
|
+
*
|
|
14
|
+
* @example
|
|
15
|
+
* ```typescript
|
|
16
|
+
* const adapter = createLLMGatewaySummarize('gpt-5.6-terra', "llmgtwy_...");
|
|
17
|
+
* ```
|
|
18
|
+
*/
|
|
19
|
+
function createLLMGatewaySummarize(model, apiKey, config) {
|
|
20
|
+
return new ChatStreamSummarizeAdapter(new LLMGatewayTextAdapter({
|
|
21
|
+
apiKey,
|
|
22
|
+
...config
|
|
23
|
+
}, model), model, "llmgateway");
|
|
24
|
+
}
|
|
25
|
+
/**
|
|
26
|
+
* Creates an LLM Gateway summarize adapter with automatic API key detection
|
|
27
|
+
* from environment variables. Type resolution happens here at the call site.
|
|
28
|
+
*
|
|
29
|
+
* Looks for `LLM_GATEWAY_API_KEY` in:
|
|
30
|
+
* - `process.env` (Node.js)
|
|
31
|
+
* - `window.env` (Browser with injected env)
|
|
32
|
+
*
|
|
33
|
+
* @param model - The model id (e.g., 'gpt-5.6-terra')
|
|
34
|
+
* @param config - Optional configuration (excluding apiKey which is auto-detected)
|
|
35
|
+
* @returns Configured LLM Gateway summarize adapter instance with resolved types
|
|
36
|
+
* @throws Error if LLM_GATEWAY_API_KEY is not found in environment
|
|
37
|
+
*
|
|
38
|
+
* @example
|
|
39
|
+
* ```typescript
|
|
40
|
+
* // Automatically uses LLM_GATEWAY_API_KEY from environment
|
|
41
|
+
* const adapter = llmGatewaySummarize('gpt-5.6-terra');
|
|
42
|
+
*
|
|
43
|
+
* await summarize({
|
|
44
|
+
* adapter,
|
|
45
|
+
* text: "Long article text..."
|
|
46
|
+
* });
|
|
47
|
+
* ```
|
|
48
|
+
*/
|
|
49
|
+
function llmGatewaySummarize(model, config) {
|
|
50
|
+
return createLLMGatewaySummarize(model, getLLMGatewayApiKeyFromEnv(), config);
|
|
51
|
+
}
|
|
52
|
+
//#endregion
|
|
53
|
+
export { createLLMGatewaySummarize, llmGatewaySummarize };
|
|
54
|
+
|
|
55
|
+
//# sourceMappingURL=summarize.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"summarize.js","names":[],"sources":["../../../src/adapters/summarize.ts"],"sourcesContent":["import { ChatStreamSummarizeAdapter } from '@tanstack/ai/adapters'\nimport { getLLMGatewayApiKeyFromEnv } from '../utils/client'\nimport { LLMGatewayTextAdapter } from './text'\nimport type { InferTextProviderOptions } from '@tanstack/ai/adapters'\nimport type { LLMGatewayModelId } from '../model-meta'\nimport type { LLMGatewayClientConfig } from '../utils/client'\n\n/**\n * Configuration for LLM Gateway summarize adapter\n */\nexport interface LLMGatewaySummarizeConfig extends LLMGatewayClientConfig {}\n\n/** Model type for LLM Gateway summarization */\nexport type LLMGatewaySummarizeModel = LLMGatewayModelId\n\n/**\n * Creates an LLM Gateway summarize adapter with explicit API key.\n * Type resolution happens here at the call site.\n *\n * @param model - The model id (e.g., 'gpt-5.6-terra')\n * @param apiKey - Your LLM Gateway API key\n * @param config - Optional additional configuration\n * @returns Configured LLM Gateway summarize adapter instance with resolved types\n *\n * @example\n * ```typescript\n * const adapter = createLLMGatewaySummarize('gpt-5.6-terra', \"llmgtwy_...\");\n * ```\n */\nexport function createLLMGatewaySummarize<\n TModel extends LLMGatewaySummarizeModel,\n>(\n model: TModel,\n apiKey: string,\n config?: Omit<LLMGatewaySummarizeConfig, 'apiKey'>,\n): ChatStreamSummarizeAdapter<\n TModel,\n InferTextProviderOptions<LLMGatewayTextAdapter<TModel>>\n> {\n return new ChatStreamSummarizeAdapter(\n new LLMGatewayTextAdapter({ apiKey, ...config }, model),\n model,\n 'llmgateway',\n )\n}\n\n/**\n * Creates an LLM Gateway summarize adapter with automatic API key detection\n * from environment variables. Type resolution happens here at the call site.\n *\n * Looks for `LLM_GATEWAY_API_KEY` in:\n * - `process.env` (Node.js)\n * - `window.env` (Browser with injected env)\n *\n * @param model - The model id (e.g., 'gpt-5.6-terra')\n * @param config - Optional configuration (excluding apiKey which is auto-detected)\n * @returns Configured LLM Gateway summarize adapter instance with resolved types\n * @throws Error if LLM_GATEWAY_API_KEY is not found in environment\n *\n * @example\n * ```typescript\n * // Automatically uses LLM_GATEWAY_API_KEY from environment\n * const adapter = llmGatewaySummarize('gpt-5.6-terra');\n *\n * await summarize({\n * adapter,\n * text: \"Long article text...\"\n * });\n * ```\n */\nexport function llmGatewaySummarize<TModel extends LLMGatewaySummarizeModel>(\n model: TModel,\n config?: Omit<LLMGatewaySummarizeConfig, 'apiKey'>,\n): ChatStreamSummarizeAdapter<\n TModel,\n InferTextProviderOptions<LLMGatewayTextAdapter<TModel>>\n> {\n return createLLMGatewaySummarize(model, getLLMGatewayApiKeyFromEnv(), config)\n}\n"],"mappings":";;;;;;;;;;;;;;;;;;AA6BA,SAAgB,0BAGd,OACA,QACA,QAIA;CACA,OAAO,IAAI,2BACT,IAAI,sBAAsB;EAAE;EAAQ,GAAG;CAAO,GAAG,KAAK,GACtD,OACA,YACF;AACF;;;;;;;;;;;;;;;;;;;;;;;;;AA0BA,SAAgB,oBACd,OACA,QAIA;CACA,OAAO,0BAA0B,OAAO,2BAA2B,GAAG,MAAM;AAC9E"}
|
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
import { default as OpenAI } from 'openai';
|
|
2
|
+
import { OpenAIBaseChatCompletionsTextAdapter } from '@tanstack/openai-base';
|
|
3
|
+
import { Modality } from '@tanstack/ai';
|
|
4
|
+
import { LLMGatewayChatModelToolCapabilitiesByName, LLMGatewayModelId, ResolveInputModalities, ResolveProviderOptions } from '../model-meta.js';
|
|
5
|
+
import { LLMGatewayMessageMetadataByModality } from '../message-types.js';
|
|
6
|
+
import { LLMGatewayClientConfig } from '../utils/client.js';
|
|
7
|
+
type ResolveToolCapabilities<TModel extends string> = TModel extends keyof LLMGatewayChatModelToolCapabilitiesByName ? NonNullable<LLMGatewayChatModelToolCapabilitiesByName[TModel]> : readonly [];
|
|
8
|
+
/**
|
|
9
|
+
* Configuration for LLM Gateway text adapter
|
|
10
|
+
*/
|
|
11
|
+
export interface LLMGatewayTextConfig extends LLMGatewayClientConfig {
|
|
12
|
+
}
|
|
13
|
+
/**
|
|
14
|
+
* Re-export of the public provider options type
|
|
15
|
+
*/
|
|
16
|
+
export type { ExternalTextProviderOptions as LLMGatewayTextProviderOptions } from '../text/text-provider-options.js';
|
|
17
|
+
/**
|
|
18
|
+
* LLM Gateway Text (Chat) Adapter
|
|
19
|
+
*
|
|
20
|
+
* Tree-shakeable adapter for LLM Gateway chat/text completion. LLM Gateway
|
|
21
|
+
* exposes one OpenAI-compatible Chat Completions endpoint that routes to
|
|
22
|
+
* hundreds of models across many providers, so the adapter drives it with
|
|
23
|
+
* the OpenAI SDK via a `baseURL` override (the same pattern as `ai-grok`
|
|
24
|
+
* and `ai-groq`).
|
|
25
|
+
*
|
|
26
|
+
* Model ids are open-ended: curated ids get per-model type metadata, and
|
|
27
|
+
* any other id from https://llmgateway.io/models works with text-only
|
|
28
|
+
* defaults. A `provider/model` id (e.g. `openai/gpt-5.5`) pins routing to
|
|
29
|
+
* that provider; a bare id lets the gateway pick.
|
|
30
|
+
*/
|
|
31
|
+
export declare class LLMGatewayTextAdapter<TModel extends LLMGatewayModelId, TProviderOptions extends Record<string, any> = ResolveProviderOptions<TModel>, TInputModalities extends ReadonlyArray<Modality> = ResolveInputModalities<TModel>, TToolCapabilities extends ReadonlyArray<string> = ResolveToolCapabilities<TModel>> extends OpenAIBaseChatCompletionsTextAdapter<TModel, TProviderOptions, TInputModalities, LLMGatewayMessageMetadataByModality, TToolCapabilities> {
|
|
32
|
+
readonly kind: "text";
|
|
33
|
+
readonly name: "llmgateway";
|
|
34
|
+
constructor(config: LLMGatewayTextConfig, model: TModel);
|
|
35
|
+
/**
|
|
36
|
+
* Surfaces reasoning deltas during streaming. LLM Gateway normalizes
|
|
37
|
+
* upstream reasoning output to `delta.reasoning_content` on the OpenAI
|
|
38
|
+
* Chat Completions wire format (the DeepSeek-style field most
|
|
39
|
+
* OpenAI-compatible providers emit); some routed providers emit
|
|
40
|
+
* `delta.reasoning` instead, so both are read.
|
|
41
|
+
*/
|
|
42
|
+
protected extractReasoning(chunk: OpenAI.Chat.Completions.ChatCompletionChunk): {
|
|
43
|
+
text: string;
|
|
44
|
+
} | undefined;
|
|
45
|
+
}
|
|
46
|
+
/**
|
|
47
|
+
* Creates an LLM Gateway text adapter with explicit API key.
|
|
48
|
+
*
|
|
49
|
+
* @example
|
|
50
|
+
* ```typescript
|
|
51
|
+
* const adapter = createLLMGatewayText('gpt-5.6-terra', "llmgtwy_...");
|
|
52
|
+
* ```
|
|
53
|
+
*/
|
|
54
|
+
export declare function createLLMGatewayText<TModel extends LLMGatewayModelId>(model: TModel, apiKey: string, config?: Omit<LLMGatewayTextConfig, 'apiKey'>): LLMGatewayTextAdapter<TModel>;
|
|
55
|
+
/**
|
|
56
|
+
* Creates an LLM Gateway text adapter with API key from
|
|
57
|
+
* `LLM_GATEWAY_API_KEY`.
|
|
58
|
+
*
|
|
59
|
+
* @example
|
|
60
|
+
* ```typescript
|
|
61
|
+
* const adapter = llmGatewayText('gpt-5.6-terra');
|
|
62
|
+
* ```
|
|
63
|
+
*/
|
|
64
|
+
export declare function llmGatewayText<TModel extends LLMGatewayModelId>(model: TModel, config?: Omit<LLMGatewayTextConfig, 'apiKey'>): LLMGatewayTextAdapter<TModel>;
|
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
import { getLLMGatewayApiKeyFromEnv, withLLMGatewayDefaults } from "../utils/client.js";
|
|
2
|
+
import OpenAI from "openai";
|
|
3
|
+
import { OpenAIBaseChatCompletionsTextAdapter } from "@tanstack/openai-base";
|
|
4
|
+
//#region src/adapters/text.ts
|
|
5
|
+
/**
|
|
6
|
+
* LLM Gateway Text (Chat) Adapter
|
|
7
|
+
*
|
|
8
|
+
* Tree-shakeable adapter for LLM Gateway chat/text completion. LLM Gateway
|
|
9
|
+
* exposes one OpenAI-compatible Chat Completions endpoint that routes to
|
|
10
|
+
* hundreds of models across many providers, so the adapter drives it with
|
|
11
|
+
* the OpenAI SDK via a `baseURL` override (the same pattern as `ai-grok`
|
|
12
|
+
* and `ai-groq`).
|
|
13
|
+
*
|
|
14
|
+
* Model ids are open-ended: curated ids get per-model type metadata, and
|
|
15
|
+
* any other id from https://llmgateway.io/models works with text-only
|
|
16
|
+
* defaults. A `provider/model` id (e.g. `openai/gpt-5.5`) pins routing to
|
|
17
|
+
* that provider; a bare id lets the gateway pick.
|
|
18
|
+
*/
|
|
19
|
+
var LLMGatewayTextAdapter = class extends OpenAIBaseChatCompletionsTextAdapter {
|
|
20
|
+
kind = "text";
|
|
21
|
+
name = "llmgateway";
|
|
22
|
+
constructor(config, model) {
|
|
23
|
+
super(model, "llmgateway", new OpenAI(withLLMGatewayDefaults(config)));
|
|
24
|
+
}
|
|
25
|
+
/**
|
|
26
|
+
* Surfaces reasoning deltas during streaming. LLM Gateway normalizes
|
|
27
|
+
* upstream reasoning output to `delta.reasoning_content` on the OpenAI
|
|
28
|
+
* Chat Completions wire format (the DeepSeek-style field most
|
|
29
|
+
* OpenAI-compatible providers emit); some routed providers emit
|
|
30
|
+
* `delta.reasoning` instead, so both are read.
|
|
31
|
+
*/
|
|
32
|
+
extractReasoning(chunk) {
|
|
33
|
+
const delta = chunk.choices[0]?.delta;
|
|
34
|
+
const raw = delta?.reasoning_content ?? delta?.reasoning;
|
|
35
|
+
if (typeof raw === "string" && raw.length > 0) return { text: raw };
|
|
36
|
+
}
|
|
37
|
+
};
|
|
38
|
+
/**
|
|
39
|
+
* Creates an LLM Gateway text adapter with explicit API key.
|
|
40
|
+
*
|
|
41
|
+
* @example
|
|
42
|
+
* ```typescript
|
|
43
|
+
* const adapter = createLLMGatewayText('gpt-5.6-terra', "llmgtwy_...");
|
|
44
|
+
* ```
|
|
45
|
+
*/
|
|
46
|
+
function createLLMGatewayText(model, apiKey, config) {
|
|
47
|
+
return new LLMGatewayTextAdapter({
|
|
48
|
+
apiKey,
|
|
49
|
+
...config
|
|
50
|
+
}, model);
|
|
51
|
+
}
|
|
52
|
+
/**
|
|
53
|
+
* Creates an LLM Gateway text adapter with API key from
|
|
54
|
+
* `LLM_GATEWAY_API_KEY`.
|
|
55
|
+
*
|
|
56
|
+
* @example
|
|
57
|
+
* ```typescript
|
|
58
|
+
* const adapter = llmGatewayText('gpt-5.6-terra');
|
|
59
|
+
* ```
|
|
60
|
+
*/
|
|
61
|
+
function llmGatewayText(model, config) {
|
|
62
|
+
return createLLMGatewayText(model, getLLMGatewayApiKeyFromEnv(), config);
|
|
63
|
+
}
|
|
64
|
+
//#endregion
|
|
65
|
+
export { LLMGatewayTextAdapter, createLLMGatewayText, llmGatewayText };
|
|
66
|
+
|
|
67
|
+
//# sourceMappingURL=text.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"text.js","names":[],"sources":["../../../src/adapters/text.ts"],"sourcesContent":["import OpenAI from 'openai'\nimport { OpenAIBaseChatCompletionsTextAdapter } from '@tanstack/openai-base'\nimport {\n getLLMGatewayApiKeyFromEnv,\n withLLMGatewayDefaults,\n} from '../utils/client'\nimport type { Modality } from '@tanstack/ai'\nimport type {\n LLMGatewayChatModelToolCapabilitiesByName,\n LLMGatewayModelId,\n ResolveInputModalities,\n ResolveProviderOptions,\n} from '../model-meta'\nimport type { LLMGatewayMessageMetadataByModality } from '../message-types'\nimport type { LLMGatewayClientConfig } from '../utils/client'\n\ntype ResolveToolCapabilities<TModel extends string> =\n TModel extends keyof LLMGatewayChatModelToolCapabilitiesByName\n ? NonNullable<LLMGatewayChatModelToolCapabilitiesByName[TModel]>\n : readonly []\n\n/**\n * Configuration for LLM Gateway text adapter\n */\nexport interface LLMGatewayTextConfig extends LLMGatewayClientConfig {}\n\n/**\n * Re-export of the public provider options type\n */\nexport type { ExternalTextProviderOptions as LLMGatewayTextProviderOptions } from '../text/text-provider-options'\n\n/**\n * LLM Gateway Text (Chat) Adapter\n *\n * Tree-shakeable adapter for LLM Gateway chat/text completion. LLM Gateway\n * exposes one OpenAI-compatible Chat Completions endpoint that routes to\n * hundreds of models across many providers, so the adapter drives it with\n * the OpenAI SDK via a `baseURL` override (the same pattern as `ai-grok`\n * and `ai-groq`).\n *\n * Model ids are open-ended: curated ids get per-model type metadata, and\n * any other id from https://llmgateway.io/models works with text-only\n * defaults. A `provider/model` id (e.g. `openai/gpt-5.5`) pins routing to\n * that provider; a bare id lets the gateway pick.\n */\nexport class LLMGatewayTextAdapter<\n TModel extends LLMGatewayModelId,\n TProviderOptions extends Record<string, any> = ResolveProviderOptions<TModel>,\n TInputModalities extends ReadonlyArray<Modality> =\n ResolveInputModalities<TModel>,\n TToolCapabilities extends ReadonlyArray<string> =\n ResolveToolCapabilities<TModel>,\n> extends OpenAIBaseChatCompletionsTextAdapter<\n TModel,\n TProviderOptions,\n TInputModalities,\n LLMGatewayMessageMetadataByModality,\n TToolCapabilities\n> {\n override readonly kind = 'text' as const\n override readonly name = 'llmgateway' as const\n\n constructor(config: LLMGatewayTextConfig, model: TModel) {\n super(model, 'llmgateway', new OpenAI(withLLMGatewayDefaults(config)))\n }\n\n /**\n * Surfaces reasoning deltas during streaming. LLM Gateway normalizes\n * upstream reasoning output to `delta.reasoning_content` on the OpenAI\n * Chat Completions wire format (the DeepSeek-style field most\n * OpenAI-compatible providers emit); some routed providers emit\n * `delta.reasoning` instead, so both are read.\n */\n protected override extractReasoning(\n chunk: OpenAI.Chat.Completions.ChatCompletionChunk,\n ): { text: string } | undefined {\n const delta = chunk.choices[0]?.delta as\n | { reasoning?: unknown; reasoning_content?: unknown }\n | undefined\n const raw = delta?.reasoning_content ?? delta?.reasoning\n if (typeof raw === 'string' && raw.length > 0) {\n return { text: raw }\n }\n return undefined\n }\n}\n\n/**\n * Creates an LLM Gateway text adapter with explicit API key.\n *\n * @example\n * ```typescript\n * const adapter = createLLMGatewayText('gpt-5.6-terra', \"llmgtwy_...\");\n * ```\n */\nexport function createLLMGatewayText<TModel extends LLMGatewayModelId>(\n model: TModel,\n apiKey: string,\n config?: Omit<LLMGatewayTextConfig, 'apiKey'>,\n): LLMGatewayTextAdapter<TModel> {\n return new LLMGatewayTextAdapter({ apiKey, ...config }, model)\n}\n\n/**\n * Creates an LLM Gateway text adapter with API key from\n * `LLM_GATEWAY_API_KEY`.\n *\n * @example\n * ```typescript\n * const adapter = llmGatewayText('gpt-5.6-terra');\n * ```\n */\nexport function llmGatewayText<TModel extends LLMGatewayModelId>(\n model: TModel,\n config?: Omit<LLMGatewayTextConfig, 'apiKey'>,\n): LLMGatewayTextAdapter<TModel> {\n const apiKey = getLLMGatewayApiKeyFromEnv()\n return createLLMGatewayText(model, apiKey, config)\n}\n"],"mappings":";;;;;;;;;;;;;;;;;;AA6CA,IAAa,wBAAb,cAOU,qCAMR;CACA,OAAyB;CACzB,OAAyB;CAEzB,YAAY,QAA8B,OAAe;EACvD,MAAM,OAAO,cAAc,IAAI,OAAO,uBAAuB,MAAM,CAAC,CAAC;CACvE;;;;;;;;CASA,iBACE,OAC8B;EAC9B,MAAM,QAAQ,MAAM,QAAQ,EAAE,EAAE;EAGhC,MAAM,MAAM,OAAO,qBAAqB,OAAO;EAC/C,IAAI,OAAO,QAAQ,YAAY,IAAI,SAAS,GAC1C,OAAO,EAAE,MAAM,IAAI;CAGvB;AACF;;;;;;;;;AAUA,SAAgB,qBACd,OACA,QACA,QAC+B;CAC/B,OAAO,IAAI,sBAAsB;EAAE;EAAQ,GAAG;CAAO,GAAG,KAAK;AAC/D;;;;;;;;;;AAWA,SAAgB,eACd,OACA,QAC+B;CAE/B,OAAO,qBAAqB,OADb,2BACoB,GAAQ,MAAM;AACnD"}
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @module @tanstack/ai-llmgateway
|
|
3
|
+
*
|
|
4
|
+
* LLM Gateway provider adapter for TanStack AI.
|
|
5
|
+
* Provides tree-shakeable adapters for LLM Gateway's OpenAI-compatible Chat
|
|
6
|
+
* Completions API, which routes one endpoint to hundreds of models across
|
|
7
|
+
* many providers.
|
|
8
|
+
*/
|
|
9
|
+
export { LLMGatewayTextAdapter, createLLMGatewayText, llmGatewayText, type LLMGatewayTextConfig, type LLMGatewayTextProviderOptions, } from './adapters/text.js';
|
|
10
|
+
export { createLLMGatewaySummarize, llmGatewaySummarize, type LLMGatewaySummarizeConfig, type LLMGatewaySummarizeModel, } from './adapters/summarize.js';
|
|
11
|
+
export type { LLMGatewayChatModelProviderOptionsByName, LLMGatewayChatModelToolCapabilitiesByName, LLMGatewayModelInputModalitiesByName, ResolveProviderOptions, ResolveInputModalities, LLMGatewayChatModels, LLMGatewayModelId, } from './model-meta.js';
|
|
12
|
+
export { LLMGATEWAY_CHAT_MODELS } from './model-meta.js';
|
|
13
|
+
export type { LLMGatewayTextMetadata, LLMGatewayImageMetadata, LLMGatewayAudioMetadata, LLMGatewayVideoMetadata, LLMGatewayDocumentMetadata, LLMGatewayMessageMetadataByModality, } from './message-types.js';
|
|
14
|
+
export { getLLMGatewayApiKeyFromEnv, withLLMGatewayDefaults, type LLMGatewayClientConfig, } from './utils/client.js';
|
|
@@ -0,0 +1,5 @@
|
|
|
1
|
+
import { getLLMGatewayApiKeyFromEnv, withLLMGatewayDefaults } from "./utils/client.js";
|
|
2
|
+
import { LLMGatewayTextAdapter, createLLMGatewayText, llmGatewayText } from "./adapters/text.js";
|
|
3
|
+
import { createLLMGatewaySummarize, llmGatewaySummarize } from "./adapters/summarize.js";
|
|
4
|
+
import { LLMGATEWAY_CHAT_MODELS } from "./model-meta.js";
|
|
5
|
+
export { LLMGATEWAY_CHAT_MODELS, LLMGatewayTextAdapter, createLLMGatewaySummarize, createLLMGatewayText, getLLMGatewayApiKeyFromEnv, llmGatewaySummarize, llmGatewayText, withLLMGatewayDefaults };
|
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* LLM Gateway-specific message types for the Chat Completions API.
|
|
3
|
+
*
|
|
4
|
+
* LLM Gateway's wire format is OpenAI Chat Completions — the gateway
|
|
5
|
+
* translates it to each routed provider's native format server-side. These
|
|
6
|
+
* type definitions describe that wire shape directly; the adapter drives
|
|
7
|
+
* the endpoint with the OpenAI SDK pointed at the gateway's base URL.
|
|
8
|
+
*
|
|
9
|
+
* @see https://docs.llmgateway.io
|
|
10
|
+
*/
|
|
11
|
+
export interface ChatCompletionNamedToolChoice {
|
|
12
|
+
/** Always `function` for a named tool choice. */
|
|
13
|
+
type: 'function';
|
|
14
|
+
function: {
|
|
15
|
+
/** The name of the function to call. */
|
|
16
|
+
name: string;
|
|
17
|
+
};
|
|
18
|
+
}
|
|
19
|
+
/**
|
|
20
|
+
* Controls which (if any) tool is called by the model.
|
|
21
|
+
*
|
|
22
|
+
* - `none` — the model will not call any tool and instead generates a message
|
|
23
|
+
* - `auto` — the model can pick between generating a message or calling tools
|
|
24
|
+
* - `required` — the model must call one or more tools
|
|
25
|
+
* - Named tool choice — forces the model to call a specific tool
|
|
26
|
+
*/
|
|
27
|
+
export type ChatCompletionToolChoiceOption = 'none' | 'auto' | 'required' | ChatCompletionNamedToolChoice;
|
|
28
|
+
export interface ResponseFormatText {
|
|
29
|
+
/** The type of response format being defined. Always `text`. */
|
|
30
|
+
type: 'text';
|
|
31
|
+
}
|
|
32
|
+
export interface ResponseFormatJsonSchemaJsonSchema {
|
|
33
|
+
/**
|
|
34
|
+
* The name of the response format. Must be a-z, A-Z, 0-9, or contain
|
|
35
|
+
* underscores and dashes, with a maximum length of 64.
|
|
36
|
+
*/
|
|
37
|
+
name: string;
|
|
38
|
+
/**
|
|
39
|
+
* A description of what the response format is for, used by the model to
|
|
40
|
+
* determine how to respond in the format.
|
|
41
|
+
*/
|
|
42
|
+
description?: string;
|
|
43
|
+
/**
|
|
44
|
+
* The schema for the response format, described as a JSON Schema object.
|
|
45
|
+
* @see https://json-schema.org/
|
|
46
|
+
*/
|
|
47
|
+
schema?: {
|
|
48
|
+
[key: string]: unknown;
|
|
49
|
+
};
|
|
50
|
+
/**
|
|
51
|
+
* Whether to enable strict schema adherence when generating the output. If
|
|
52
|
+
* set to true, the model will always follow the exact schema defined in the
|
|
53
|
+
* `schema` field. Only a subset of JSON Schema is supported when `strict`
|
|
54
|
+
* is `true`.
|
|
55
|
+
*/
|
|
56
|
+
strict?: boolean | null;
|
|
57
|
+
}
|
|
58
|
+
export interface ResponseFormatJsonSchema {
|
|
59
|
+
/** Structured Outputs configuration options, including a JSON Schema. */
|
|
60
|
+
json_schema: ResponseFormatJsonSchemaJsonSchema;
|
|
61
|
+
/** The type of response format being defined. Always `json_schema`. */
|
|
62
|
+
type: 'json_schema';
|
|
63
|
+
}
|
|
64
|
+
export interface ResponseFormatJsonObject {
|
|
65
|
+
/** The type of response format being defined. Always `json_object`. */
|
|
66
|
+
type: 'json_object';
|
|
67
|
+
}
|
|
68
|
+
/**
|
|
69
|
+
* Metadata for LLM Gateway document content parts.
|
|
70
|
+
*/
|
|
71
|
+
export interface LLMGatewayDocumentMetadata {
|
|
72
|
+
}
|
|
73
|
+
/**
|
|
74
|
+
* Metadata for LLM Gateway text content parts.
|
|
75
|
+
* Currently no specific metadata options for text.
|
|
76
|
+
*/
|
|
77
|
+
export interface LLMGatewayTextMetadata {
|
|
78
|
+
}
|
|
79
|
+
/**
|
|
80
|
+
* Metadata for LLM Gateway image content parts.
|
|
81
|
+
* Controls how the model processes and analyzes images.
|
|
82
|
+
*/
|
|
83
|
+
export interface LLMGatewayImageMetadata {
|
|
84
|
+
/**
|
|
85
|
+
* Specifies the detail level of the image.
|
|
86
|
+
* - 'auto': Let the model decide based on image size and content
|
|
87
|
+
* - 'low': Use low resolution processing (faster, cheaper, less detail)
|
|
88
|
+
* - 'high': Use high resolution processing (slower, more expensive, more detail)
|
|
89
|
+
*
|
|
90
|
+
* @default 'auto'
|
|
91
|
+
*/
|
|
92
|
+
detail?: 'auto' | 'low' | 'high';
|
|
93
|
+
}
|
|
94
|
+
/**
|
|
95
|
+
* Metadata for LLM Gateway audio content parts.
|
|
96
|
+
* Note: audio input support depends on the routed model.
|
|
97
|
+
*/
|
|
98
|
+
export interface LLMGatewayAudioMetadata {
|
|
99
|
+
}
|
|
100
|
+
/**
|
|
101
|
+
* Metadata for LLM Gateway video content parts.
|
|
102
|
+
* Note: video input support depends on the routed model.
|
|
103
|
+
*/
|
|
104
|
+
export interface LLMGatewayVideoMetadata {
|
|
105
|
+
}
|
|
106
|
+
/**
|
|
107
|
+
* Map of modality types to their LLM Gateway-specific metadata types.
|
|
108
|
+
* Used for type inference when constructing multimodal messages.
|
|
109
|
+
*/
|
|
110
|
+
export interface LLMGatewayMessageMetadataByModality {
|
|
111
|
+
text: LLMGatewayTextMetadata;
|
|
112
|
+
image: LLMGatewayImageMetadata;
|
|
113
|
+
audio: LLMGatewayAudioMetadata;
|
|
114
|
+
video: LLMGatewayVideoMetadata;
|
|
115
|
+
document: LLMGatewayDocumentMetadata;
|
|
116
|
+
}
|