@kreuzberg/liter-llm-node 1.5.1 → 1.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md
CHANGED
|
@@ -1,139 +1,113 @@
|
|
|
1
1
|
# TypeScript (Node.js)
|
|
2
2
|
|
|
3
|
-
<div
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
</a>
|
|
57
|
-
<a href="https://hex.pm/packages/liter_llm">
|
|
58
|
-
<img src="https://img.shields.io/hexpm/v/liter_llm?label=Elixir&color=007ec6" alt="Elixir" />
|
|
59
|
-
</a>
|
|
60
|
-
<a href="https://github.com/kreuzberg-dev/liter-llm/pkgs/container/liter-llm">
|
|
61
|
-
<img
|
|
62
|
-
src="https://img.shields.io/badge/Docker-007ec6?logo=docker&logoColor=white"
|
|
63
|
-
alt="Docker"
|
|
64
|
-
/>
|
|
65
|
-
</a>
|
|
66
|
-
<a href="https://github.com/kreuzberg-dev/homebrew-tap/blob/main/Formula/liter-llm.rb">
|
|
67
|
-
<img
|
|
68
|
-
src="https://img.shields.io/badge/Homebrew-007ec6?logo=homebrew&logoColor=white"
|
|
69
|
-
alt="Homebrew"
|
|
70
|
-
/>
|
|
71
|
-
</a>
|
|
72
|
-
<a href="https://github.com/kreuzberg-dev/liter-llm/tree/main/crates/liter-llm-ffi">
|
|
73
|
-
<img src="https://img.shields.io/badge/C-FFI-007ec6" alt="C FFI" />
|
|
74
|
-
</a>
|
|
75
|
-
|
|
76
|
-
<!-- Project Info -->
|
|
77
|
-
<a href="https://github.com/kreuzberg-dev/liter-llm/blob/main/LICENSE">
|
|
78
|
-
<img src="https://img.shields.io/badge/License-MIT-007ec6" alt="License" />
|
|
79
|
-
</a>
|
|
80
|
-
<a href="https://docs.liter-llm.kreuzberg.dev">
|
|
81
|
-
<img src="https://img.shields.io/badge/docs-kreuzberg.dev-007ec6" alt="Docs" />
|
|
82
|
-
</a>
|
|
3
|
+
<div align="center" style="display: flex; flex-wrap: wrap; gap: 8px; justify-content: center; margin: 20px 0">
|
|
4
|
+
<!-- Built with -->
|
|
5
|
+
<a href="https://github.com/kreuzberg-dev/alef">
|
|
6
|
+
<img src="https://img.shields.io/badge/Bindings-alef%20%D7%90-007ec6" alt="Bindings" />
|
|
7
|
+
</a>
|
|
8
|
+
<!-- Language Bindings -->
|
|
9
|
+
<a href="https://crates.io/crates/liter-llm">
|
|
10
|
+
<img src="https://img.shields.io/crates/v/liter-llm?label=Rust&color=007ec6" alt="Rust" />
|
|
11
|
+
</a>
|
|
12
|
+
<a href="https://pypi.org/project/liter-llm/">
|
|
13
|
+
<img src="https://img.shields.io/pypi/v/liter-llm?label=Python&color=007ec6" alt="Python" />
|
|
14
|
+
</a>
|
|
15
|
+
<a href="https://www.npmjs.com/package/@kreuzberg/liter-llm">
|
|
16
|
+
<img src="https://img.shields.io/npm/v/@kreuzberg/liter-llm?label=Node.js&color=007ec6" alt="Node.js" />
|
|
17
|
+
</a>
|
|
18
|
+
<a href="https://www.npmjs.com/package/@kreuzberg/liter-llm-wasm">
|
|
19
|
+
<img src="https://img.shields.io/npm/v/@kreuzberg/liter-llm-wasm?label=WASM&color=007ec6" alt="WASM" />
|
|
20
|
+
</a>
|
|
21
|
+
<a href="https://central.sonatype.com/artifact/dev.kreuzberg/liter-llm">
|
|
22
|
+
<img src="https://img.shields.io/maven-central/v/dev.kreuzberg/liter-llm?label=Java&color=007ec6" alt="Java" />
|
|
23
|
+
</a>
|
|
24
|
+
<a href="https://github.com/kreuzberg-dev/liter-llm/tree/main/packages/go">
|
|
25
|
+
<img src="https://img.shields.io/github/v/tag/kreuzberg-dev/liter-llm?label=Go&color=007ec6" alt="Go" />
|
|
26
|
+
</a>
|
|
27
|
+
<a href="https://www.nuget.org/packages/LiterLlm">
|
|
28
|
+
<img src="https://img.shields.io/nuget/v/LiterLlm?label=C%23&color=007ec6" alt="C#" />
|
|
29
|
+
</a>
|
|
30
|
+
<a href="https://packagist.org/packages/kreuzberg/liter-llm">
|
|
31
|
+
<img src="https://img.shields.io/packagist/v/kreuzberg/liter-llm?label=PHP&color=007ec6" alt="PHP" />
|
|
32
|
+
</a>
|
|
33
|
+
<a href="https://rubygems.org/gems/liter_llm">
|
|
34
|
+
<img src="https://img.shields.io/gem/v/liter_llm?label=Ruby&color=007ec6" alt="Ruby" />
|
|
35
|
+
</a>
|
|
36
|
+
<a href="https://hex.pm/packages/liter_llm">
|
|
37
|
+
<img src="https://img.shields.io/hexpm/v/liter_llm?label=Elixir&color=007ec6" alt="Elixir" />
|
|
38
|
+
</a>
|
|
39
|
+
<a href="https://github.com/kreuzberg-dev/liter-llm/pkgs/container/liter-llm">
|
|
40
|
+
<img src="https://img.shields.io/badge/Docker-007ec6?logo=docker&logoColor=white" alt="Docker" />
|
|
41
|
+
</a>
|
|
42
|
+
<a href="https://github.com/kreuzberg-dev/homebrew-tap/blob/main/Formula/liter-llm.rb">
|
|
43
|
+
<img src="https://img.shields.io/badge/Homebrew-007ec6?logo=homebrew&logoColor=white" alt="Homebrew" />
|
|
44
|
+
</a>
|
|
45
|
+
<a href="https://github.com/kreuzberg-dev/liter-llm/tree/main/crates/liter-llm-ffi">
|
|
46
|
+
<img src="https://img.shields.io/badge/C-FFI-007ec6" alt="C FFI" />
|
|
47
|
+
</a>
|
|
48
|
+
|
|
49
|
+
<!-- Project Info -->
|
|
50
|
+
<a href="https://github.com/kreuzberg-dev/liter-llm/blob/main/LICENSE">
|
|
51
|
+
<img src="https://img.shields.io/badge/License-MIT-007ec6" alt="License" />
|
|
52
|
+
</a>
|
|
53
|
+
<a href="https://docs.liter-llm.kreuzberg.dev">
|
|
54
|
+
<img src="https://img.shields.io/badge/Docs-liter--llm-007ec6" alt="Docs" />
|
|
55
|
+
</a>
|
|
83
56
|
</div>
|
|
84
|
-
<div align="center" style="margin:
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
</picture>
|
|
57
|
+
<div align="center" style="margin: 24px 0 0">
|
|
58
|
+
<a href="https://kreuzberg.dev">
|
|
59
|
+
<img
|
|
60
|
+
alt="kreuzberg.dev"
|
|
61
|
+
src="https://github.com/user-attachments/assets/1b6c6ad7-3b6d-4171-b1c9-f2026cc9deb8"
|
|
62
|
+
/>
|
|
63
|
+
</a>
|
|
92
64
|
</div>
|
|
93
|
-
<div align="center" style="
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
65
|
+
<div align="center" style="display: flex; flex-wrap: wrap; gap: 12px; justify-content: center; margin: 28px 0 24px">
|
|
66
|
+
<a href="https://discord.gg/xt9WY3GnKR">
|
|
67
|
+
<img
|
|
68
|
+
height="22"
|
|
69
|
+
src="https://img.shields.io/badge/Discord-Chat-007ec6?logo=discord&logoColor=white"
|
|
70
|
+
alt="Join Discord"
|
|
71
|
+
/>
|
|
72
|
+
</a>
|
|
101
73
|
</div>
|
|
102
74
|
|
|
103
|
-
Universal LLM API client for TypeScript and Node.js. Access 143
|
|
75
|
+
Universal LLM API client for TypeScript and Node.js. Access 143 LLM providers through a single interface with native NAPI-RS bindings, async/await, streaming, tool calling, and full TypeScript type definitions.
|
|
76
|
+
|
|
77
|
+
## What This Package Provides
|
|
78
|
+
|
|
79
|
+
- **One provider surface** — chat, streaming, embeddings, images, audio, search, OCR, tools, and structured output across the provider registry.
|
|
80
|
+
- **Provider/model routing** — call models with the `provider/model` convention and keep provider-specific request code out of application paths.
|
|
81
|
+
- **Production controls** — retries, fallback, rate limits, cache layers, budgets, health checks, OpenTelemetry spans, and redacted secrets.
|
|
82
|
+
- **Same core as every binding** — Rust, Python, Node.js, Go, Java, PHP, Ruby, .NET, Elixir, WASM, Kotlin Android, Swift, Dart, Zig, and C FFI use the same Rust implementation.
|
|
83
|
+
- **Node-first TypeScript API** — NAPI-RS package with typed requests/responses and async iterables for streaming.
|
|
104
84
|
|
|
105
85
|
## Installation
|
|
106
86
|
|
|
107
87
|
### Package Installation
|
|
108
88
|
|
|
109
|
-
|
|
110
89
|
Install via one of the supported package managers:
|
|
111
90
|
|
|
112
|
-
|
|
113
91
|
**npm:**
|
|
114
92
|
|
|
115
93
|
```bash
|
|
116
94
|
npm install @kreuzberg/liter-llm
|
|
117
95
|
```
|
|
118
96
|
|
|
119
|
-
|
|
120
97
|
**pnpm:**
|
|
121
98
|
|
|
122
99
|
```bash
|
|
123
100
|
pnpm add @kreuzberg/liter-llm
|
|
124
101
|
```
|
|
125
102
|
|
|
126
|
-
|
|
127
103
|
**yarn:**
|
|
128
104
|
|
|
129
105
|
```bash
|
|
130
106
|
yarn add @kreuzberg/liter-llm
|
|
131
107
|
```
|
|
132
108
|
|
|
133
|
-
|
|
134
109
|
### System Requirements
|
|
135
110
|
|
|
136
|
-
|
|
137
111
|
- **Node.js 22+** required (NAPI-RS native bindings)
|
|
138
112
|
- API keys via environment variables (e.g. `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`)
|
|
139
113
|
|
|
@@ -145,7 +119,6 @@ Pre-built binaries available for:
|
|
|
145
119
|
- Linux (x64)
|
|
146
120
|
- Windows (x64)
|
|
147
121
|
|
|
148
|
-
|
|
149
122
|
## Quick Start
|
|
150
123
|
|
|
151
124
|
### Basic Chat
|
|
@@ -165,7 +138,6 @@ console.log(response.choices[0].message.content);
|
|
|
165
138
|
|
|
166
139
|
### Common Use Cases
|
|
167
140
|
|
|
168
|
-
|
|
169
141
|
#### Streaming Responses
|
|
170
142
|
|
|
171
143
|
Stream tokens in real time:
|
|
@@ -179,13 +151,12 @@ const chunks = await client.chatStream({
|
|
|
179
151
|
messages: [{ role: "user", content: "Tell me a story" }],
|
|
180
152
|
});
|
|
181
153
|
|
|
182
|
-
for (const chunk of chunks) {
|
|
154
|
+
for await (const chunk of chunks) {
|
|
183
155
|
process.stdout.write(chunk.choices?.[0]?.delta?.content ?? "");
|
|
184
156
|
}
|
|
185
157
|
console.log();
|
|
186
158
|
```
|
|
187
159
|
|
|
188
|
-
|
|
189
160
|
#### Tool Calling
|
|
190
161
|
|
|
191
162
|
Define and invoke tools:
|
|
@@ -219,13 +190,11 @@ for (const call of response.choices[0]?.message?.toolCalls ?? []) {
|
|
|
219
190
|
}
|
|
220
191
|
```
|
|
221
192
|
|
|
222
|
-
|
|
223
193
|
### Next Steps
|
|
224
194
|
|
|
225
195
|
- **[Provider Registry](https://github.com/kreuzberg-dev/liter-llm/blob/main/schemas/providers.json)** - Full list of supported providers
|
|
226
196
|
- **[GitHub Repository](https://github.com/kreuzberg-dev/liter-llm)** - Source, issues, and discussions
|
|
227
197
|
|
|
228
|
-
|
|
229
198
|
## NAPI-RS Implementation Details
|
|
230
199
|
|
|
231
200
|
### Native Performance
|
|
@@ -249,10 +218,9 @@ This binding uses NAPI-RS to provide native Node.js bindings with:
|
|
|
249
218
|
- Streaming buffers are released as soon as each chunk is consumed
|
|
250
219
|
- Provider registry is compiled into the binary — no runtime disk access
|
|
251
220
|
|
|
252
|
-
|
|
253
221
|
## Features
|
|
254
222
|
|
|
255
|
-
### Supported Providers (143
|
|
223
|
+
### Supported Providers (143)
|
|
256
224
|
|
|
257
225
|
Route to any provider using the `provider/model` prefix convention:
|
|
258
226
|
|
|
@@ -272,15 +240,11 @@ Route to any provider using the `provider/model` prefix convention:
|
|
|
272
240
|
|
|
273
241
|
### Key Capabilities
|
|
274
242
|
|
|
275
|
-
- **Provider Routing** -- Single client for 143
|
|
243
|
+
- **Provider Routing** -- Single client for 143 LLM providers via `provider/model` prefix
|
|
276
244
|
- **Local LLMs** — Connect to locally-hosted models via Ollama, LM Studio, vLLM, llama.cpp, and other local inference servers
|
|
277
245
|
- **Unified API** -- Consistent `chat`, `chat_stream`, `embeddings`, `list_models` interface
|
|
278
|
-
|
|
279
246
|
- **Streaming** -- Real-time token streaming via `chat_stream`
|
|
280
|
-
|
|
281
|
-
|
|
282
247
|
- **Tool Calling** -- Function calling and tool use across all supporting providers
|
|
283
|
-
|
|
284
248
|
- **Type Safe** -- Schema-driven types compiled from JSON schemas
|
|
285
249
|
- **Secure** -- API keys never logged or serialized, managed via environment variables
|
|
286
250
|
- **Observability** -- Built-in [OpenTelemetry](https://opentelemetry.io/docs/specs/semconv/gen-ai/) with GenAI semantic conventions
|
|
@@ -295,10 +259,9 @@ Built on a compiled Rust core for speed and safety:
|
|
|
295
259
|
- **Zero-copy streaming** with SSE and AWS EventStream support
|
|
296
260
|
- **API keys** wrapped in secure memory, zeroed on drop
|
|
297
261
|
|
|
298
|
-
|
|
299
262
|
## Provider Routing
|
|
300
263
|
|
|
301
|
-
Route to 143
|
|
264
|
+
Route to 143 providers using the `provider/model` prefix convention:
|
|
302
265
|
|
|
303
266
|
```text
|
|
304
267
|
openai/gpt-4o
|
|
@@ -309,7 +272,6 @@ mistral/mistral-large-latest
|
|
|
309
272
|
|
|
310
273
|
See the [provider registry](https://github.com/kreuzberg-dev/liter-llm/blob/main/schemas/providers.json) for the full list.
|
|
311
274
|
|
|
312
|
-
|
|
313
275
|
## Proxy Server
|
|
314
276
|
|
|
315
277
|
liter-llm also ships as an OpenAI-compatible proxy server with Docker support:
|
|
@@ -318,7 +280,7 @@ liter-llm also ships as an OpenAI-compatible proxy server with Docker support:
|
|
|
318
280
|
docker run -p 4000:4000 -e LITER_LLM_MASTER_KEY=sk-your-key ghcr.io/kreuzberg-dev/liter-llm
|
|
319
281
|
```
|
|
320
282
|
|
|
321
|
-
See the [proxy server documentation](https://docs.liter-llm.kreuzberg.dev/server/proxy/) for configuration, CLI usage, and MCP integration.
|
|
283
|
+
See the [proxy server documentation](https://docs.liter-llm.kreuzberg.dev/server/proxy-server/) for configuration, CLI usage, and MCP integration.
|
|
322
284
|
|
|
323
285
|
## Documentation
|
|
324
286
|
|
|
@@ -332,8 +294,9 @@ See the [proxy server documentation](https://docs.liter-llm.kreuzberg.dev/server
|
|
|
332
294
|
- [Kreuzberg Cloud](https://github.com/kreuzberg-dev/kreuzberg-cloud) — managed extraction API with SDKs, dashboards, and observability.
|
|
333
295
|
- [kreuzcrawl](https://github.com/kreuzberg-dev/kreuzcrawl) — web crawling and scraping with HTML→Markdown and headless-Chrome fallback.
|
|
334
296
|
- [html-to-markdown](https://github.com/kreuzberg-dev/html-to-markdown) — fast, lossless HTML→Markdown engine.
|
|
297
|
+
- [liter-llm](https://github.com/kreuzberg-dev/liter-llm) — universal LLM API client with native bindings for 14 languages and 143 providers.
|
|
335
298
|
- [tree-sitter-language-pack](https://github.com/kreuzberg-dev/tree-sitter-language-pack) — tree-sitter grammars and code-intelligence primitives.
|
|
336
|
-
- [alef](https://github.com/kreuzberg-dev/alef) — the polyglot binding generator that produces
|
|
299
|
+
- [alef](https://github.com/kreuzberg-dev/alef) — the polyglot binding generator that produces every per-language binding across the 5 polyglot repos.
|
|
337
300
|
- [Discord](https://discord.gg/xt9WY3GnKR) — community, roadmap, announcements.
|
|
338
301
|
|
|
339
302
|
## Contributing
|