@kreuzberg/liter-llm-node 1.4.0-rc.26 → 1.4.0-rc.28

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (9) hide show
  1. package/LICENSE +7 -0
  2. package/README.md +111 -81
  3. package/index.d.ts +1240 -562
  4. package/index.js +619 -0
  5. package/package.json +46 -55
  6. package/liter-llm-node.darwin-arm64.node +0 -0
  7. package/liter-llm-node.linux-arm64-gnu.node +0 -0
  8. package/liter-llm-node.linux-x64-gnu.node +0 -0
  9. package/liter-llm-node.win32-x64-msvc.node +0 -0
package/LICENSE ADDED
@@ -0,0 +1,7 @@
1
+ Copyright 2026 Na'aman Hirschfeld
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
4
+
5
+ The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
6
+
7
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
package/README.md CHANGED
@@ -1,102 +1,140 @@
1
1
  # TypeScript (Node.js)
2
2
 
3
- <div align="center" style="display: flex; flex-wrap: wrap; gap: 8px; justify-content: center; margin: 20px 0;">
3
+ <div
4
+ align="center"
5
+ style="display: flex; flex-wrap: wrap; gap: 8px; justify-content: center; margin: 20px 0"
6
+ >
7
+ <!-- Built with -->
8
+ <a href="https://github.com/kreuzberg-dev/alef">
9
+ <img src="https://img.shields.io/badge/Bindings-alef%20%D7%90-007ec6" alt="Bindings" />
10
+ </a>
4
11
  <!-- Language Bindings -->
5
12
  <a href="https://crates.io/crates/liter-llm">
6
- <img src="https://img.shields.io/crates/v/liter-llm?label=Rust&color=007ec6" alt="Rust">
13
+ <img src="https://img.shields.io/crates/v/liter-llm?label=Rust&color=007ec6" alt="Rust" />
7
14
  </a>
8
15
  <a href="https://pypi.org/project/liter-llm/">
9
- <img src="https://img.shields.io/pypi/v/liter-llm?label=Python&color=007ec6" alt="Python">
16
+ <img src="https://img.shields.io/pypi/v/liter-llm?label=Python&color=007ec6" alt="Python" />
10
17
  </a>
11
18
  <a href="https://www.npmjs.com/package/@kreuzberg/liter-llm">
12
- <img src="https://img.shields.io/npm/v/@kreuzberg/liter-llm?label=Node.js&color=007ec6" alt="Node.js">
19
+ <img
20
+ src="https://img.shields.io/npm/v/@kreuzberg/liter-llm?label=Node.js&color=007ec6"
21
+ alt="Node.js"
22
+ />
13
23
  </a>
14
24
  <a href="https://www.npmjs.com/package/@kreuzberg/liter-llm-wasm">
15
- <img src="https://img.shields.io/npm/v/@kreuzberg/liter-llm-wasm?label=WASM&color=007ec6" alt="WASM">
25
+ <img
26
+ src="https://img.shields.io/npm/v/@kreuzberg/liter-llm-wasm?label=WASM&color=007ec6"
27
+ alt="WASM"
28
+ />
16
29
  </a>
17
30
  <a href="https://central.sonatype.com/artifact/dev.kreuzberg/liter-llm">
18
- <img src="https://img.shields.io/maven-central/v/dev.kreuzberg/liter-llm?label=Java&color=007ec6" alt="Java">
31
+ <img
32
+ src="https://img.shields.io/maven-central/v/dev.kreuzberg/liter-llm?label=Java&color=007ec6"
33
+ alt="Java"
34
+ />
19
35
  </a>
20
36
  <a href="https://github.com/kreuzberg-dev/liter-llm/tree/main/packages/go">
21
- <img src="https://img.shields.io/github/v/tag/kreuzberg-dev/liter-llm?label=Go&color=007ec6" alt="Go">
37
+ <img
38
+ src="https://img.shields.io/github/v/tag/kreuzberg-dev/liter-llm?label=Go&color=007ec6"
39
+ alt="Go"
40
+ />
22
41
  </a>
23
42
  <a href="https://www.nuget.org/packages/LiterLlm">
24
- <img src="https://img.shields.io/nuget/v/LiterLlm?label=C%23&color=007ec6" alt="C#">
43
+ <img src="https://img.shields.io/nuget/v/LiterLlm?label=C%23&color=007ec6" alt="C#" />
25
44
  </a>
26
45
  <a href="https://packagist.org/packages/kreuzberg/liter-llm">
27
- <img src="https://img.shields.io/packagist/v/kreuzberg/liter-llm?label=PHP&color=007ec6" alt="PHP">
46
+ <img
47
+ src="https://img.shields.io/packagist/v/kreuzberg/liter-llm?label=PHP&color=007ec6"
48
+ alt="PHP"
49
+ />
28
50
  </a>
29
51
  <a href="https://rubygems.org/gems/liter_llm">
30
- <img src="https://img.shields.io/gem/v/liter_llm?label=Ruby&color=007ec6" alt="Ruby">
52
+ <img src="https://img.shields.io/gem/v/liter_llm?label=Ruby&color=007ec6" alt="Ruby" />
31
53
  </a>
32
54
  <a href="https://hex.pm/packages/liter_llm">
33
- <img src="https://img.shields.io/hexpm/v/liter_llm?label=Elixir&color=007ec6" alt="Elixir">
55
+ <img src="https://img.shields.io/hexpm/v/liter_llm?label=Elixir&color=007ec6" alt="Elixir" />
34
56
  </a>
35
57
  <a href="https://github.com/kreuzberg-dev/liter-llm/pkgs/container/liter-llm">
36
- <img src="https://img.shields.io/badge/Docker-007ec6?logo=docker&logoColor=white" alt="Docker">
58
+ <img
59
+ src="https://img.shields.io/badge/Docker-007ec6?logo=docker&logoColor=white"
60
+ alt="Docker"
61
+ />
37
62
  </a>
38
63
  <a href="https://github.com/kreuzberg-dev/homebrew-tap/blob/main/Formula/liter-llm.rb">
39
- <img src="https://img.shields.io/badge/Homebrew-007ec6?logo=homebrew&logoColor=white" alt="Homebrew">
64
+ <img
65
+ src="https://img.shields.io/badge/Homebrew-007ec6?logo=homebrew&logoColor=white"
66
+ alt="Homebrew"
67
+ />
40
68
  </a>
41
69
  <a href="https://github.com/kreuzberg-dev/liter-llm/tree/main/crates/liter-llm-ffi">
42
- <img src="https://img.shields.io/badge/C-FFI-007ec6" alt="C FFI">
70
+ <img src="https://img.shields.io/badge/C-FFI-007ec6" alt="C FFI" />
43
71
  </a>
44
72
 
45
73
  <!-- Project Info -->
46
74
  <a href="https://github.com/kreuzberg-dev/liter-llm/blob/main/LICENSE">
47
- <img src="https://img.shields.io/badge/License-MIT-007ec6" alt="License">
75
+ <img src="https://img.shields.io/badge/License-MIT-007ec6" alt="License" />
48
76
  </a>
49
77
  <a href="https://docs.liter-llm.kreuzberg.dev">
50
- <img src="https://img.shields.io/badge/docs-kreuzberg.dev-007ec6" alt="Docs">
78
+ <img src="https://img.shields.io/badge/Docs-liter--llm-007ec6" alt="Docs" />
51
79
  </a>
52
80
  </div>
53
-
54
- <div align="center" style="margin: 20px 0;">
55
- <picture>
56
- <img width="100%" alt="kreuzberg.dev" src="https://github.com/user-attachments/assets/1b6c6ad7-3b6d-4171-b1c9-f2026cc9deb8" />
57
- </picture>
81
+ <div align="center" style="margin: 24px 0 0">
82
+ <a href="https://kreuzberg.dev">
83
+ <img
84
+ alt="kreuzberg.dev"
85
+ src="https://github.com/user-attachments/assets/1b6c6ad7-3b6d-4171-b1c9-f2026cc9deb8"
86
+ />
87
+ </a>
58
88
  </div>
59
-
60
- <div align="center" style="margin-bottom: 20px;">
89
+ <div
90
+ align="center"
91
+ style="display: flex; flex-wrap: wrap; gap: 12px; justify-content: center; margin: 28px 0 24px"
92
+ >
61
93
  <a href="https://discord.gg/xt9WY3GnKR">
62
- <img height="22" src="https://img.shields.io/badge/Discord-Join%20our%20community-7289da?logo=discord&logoColor=white" alt="Discord">
94
+ <img
95
+ height="22"
96
+ src="https://img.shields.io/badge/Discord-Chat-007ec6?logo=discord&logoColor=white"
97
+ alt="Join Discord"
98
+ />
63
99
  </a>
64
100
  </div>
65
101
 
66
102
  Universal LLM API client for TypeScript and Node.js. Access 143+ LLM providers through a single interface with native NAPI-RS bindings, async/await, streaming, tool calling, and full TypeScript type definitions.
67
103
 
104
+ ## What This Package Provides
105
+
106
+ - **One provider surface** — chat, streaming, embeddings, images, audio, search, OCR, tools, and structured output across the provider registry.
107
+ - **Provider/model routing** — call models with the `provider/model` convention and keep provider-specific request code out of application paths.
108
+ - **Production controls** — retries, fallback, rate limits, cache layers, budgets, health checks, OpenTelemetry spans, and redacted secrets.
109
+ - **Same core as every binding** — Rust, Python, Node.js, Go, Java, PHP, Ruby, .NET, Elixir, WASM, Kotlin Android, Swift, Dart, Zig, and C FFI use the same Rust implementation.
110
+ - **Node-first TypeScript API** — NAPI-RS package with typed requests/responses and async iterables for streaming.
111
+
68
112
  ## Installation
69
113
 
70
114
  ### Package Installation
71
115
 
72
-
73
116
  Install via one of the supported package managers:
74
117
 
75
-
76
118
  **npm:**
77
119
 
78
120
  ```bash
79
121
  npm install @kreuzberg/liter-llm
80
122
  ```
81
123
 
82
-
83
124
  **pnpm:**
84
125
 
85
126
  ```bash
86
127
  pnpm add @kreuzberg/liter-llm
87
128
  ```
88
129
 
89
-
90
130
  **yarn:**
91
131
 
92
132
  ```bash
93
133
  yarn add @kreuzberg/liter-llm
94
134
  ```
95
135
 
96
-
97
136
  ### System Requirements
98
137
 
99
-
100
138
  - **Node.js 22+** required (NAPI-RS native bindings)
101
139
  - API keys via environment variables (e.g. `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`)
102
140
 
@@ -108,7 +146,6 @@ Pre-built binaries available for:
108
146
  - Linux (x64)
109
147
  - Windows (x64)
110
148
 
111
-
112
149
  ## Quick Start
113
150
 
114
151
  ### Basic Chat
@@ -116,9 +153,9 @@ Pre-built binaries available for:
116
153
  Send a message to any provider using the `provider/model` prefix:
117
154
 
118
155
  ```typescript
119
- import { LlmClient } from "@kreuzberg/liter-llm";
156
+ import { createClient } from "@kreuzberg/liter-llm-node";
120
157
 
121
- const client = new LlmClient({ apiKey: process.env.OPENAI_API_KEY! });
158
+ const client = createClient(process.env.OPENAI_API_KEY!);
122
159
  const response = await client.chat({
123
160
  model: "openai/gpt-4o",
124
161
  messages: [{ role: "user", content: "Hello!" }],
@@ -128,57 +165,51 @@ console.log(response.choices[0].message.content);
128
165
 
129
166
  ### Common Use Cases
130
167
 
131
-
132
168
  #### Streaming Responses
133
169
 
134
170
  Stream tokens in real time:
135
171
 
136
172
  ```typescript
137
- import { LlmClient } from "@kreuzberg/liter-llm";
173
+ import { createClient } from "@kreuzberg/liter-llm-node";
138
174
 
139
- const client = new LlmClient({ apiKey: process.env.OPENAI_API_KEY! });
175
+ const client = createClient(process.env.OPENAI_API_KEY!);
140
176
  const chunks = await client.chatStream({
141
177
  model: "openai/gpt-4o",
142
178
  messages: [{ role: "user", content: "Tell me a story" }],
143
179
  });
144
180
 
145
181
  for (const chunk of chunks) {
146
- process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
182
+ process.stdout.write(chunk.choices?.[0]?.delta?.content ?? "");
147
183
  }
148
184
  console.log();
149
185
  ```
150
186
 
151
-
152
187
  #### Tool Calling
153
188
 
154
189
  Define and invoke tools:
155
190
 
156
191
  ```typescript
157
- import { LlmClient } from "@kreuzberg/liter-llm";
158
-
159
- const client = new LlmClient({ apiKey: process.env.OPENAI_API_KEY! });
160
-
161
- const tools = [
162
- {
163
- type: "function" as const,
164
- function: {
165
- name: "get_weather",
166
- description: "Get the current weather for a location",
167
- parameters: {
168
- type: "object",
169
- properties: {
170
- location: { type: "string", description: "City name" },
171
- },
172
- required: ["location"],
173
- },
174
- },
175
- },
176
- ];
192
+ import { createClient, ToolType } from "@kreuzberg/liter-llm-node";
193
+
194
+ const client = createClient(process.env.OPENAI_API_KEY!);
177
195
 
178
196
  const response = await client.chat({
179
197
  model: "openai/gpt-4o",
180
198
  messages: [{ role: "user", content: "What is the weather in Berlin?" }],
181
- tools,
199
+ tools: [
200
+ {
201
+ toolType: ToolType.Function,
202
+ function: {
203
+ name: "get_weather",
204
+ description: "Get the current weather for a location",
205
+ parameters: {
206
+ type: "object",
207
+ properties: { location: { type: "string" } },
208
+ required: ["location"],
209
+ },
210
+ },
211
+ },
212
+ ],
182
213
  });
183
214
 
184
215
  for (const call of response.choices[0]?.message?.toolCalls ?? []) {
@@ -186,13 +217,11 @@ for (const call of response.choices[0]?.message?.toolCalls ?? []) {
186
217
  }
187
218
  ```
188
219
 
189
-
190
220
  ### Next Steps
191
221
 
192
222
  - **[Provider Registry](https://github.com/kreuzberg-dev/liter-llm/blob/main/schemas/providers.json)** - Full list of supported providers
193
223
  - **[GitHub Repository](https://github.com/kreuzberg-dev/liter-llm)** - Source, issues, and discussions
194
224
 
195
-
196
225
  ## NAPI-RS Implementation Details
197
226
 
198
227
  ### Native Performance
@@ -216,24 +245,23 @@ This binding uses NAPI-RS to provide native Node.js bindings with:
216
245
  - Streaming buffers are released as soon as each chunk is consumed
217
246
  - Provider registry is compiled into the binary — no runtime disk access
218
247
 
219
-
220
248
  ## Features
221
249
 
222
250
  ### Supported Providers (143+)
223
251
 
224
252
  Route to any provider using the `provider/model` prefix convention:
225
253
 
226
- | Provider | Example Model |
227
- |----------|--------------|
228
- | **OpenAI** | `openai/gpt-4o`, `openai/gpt-4o-mini` |
229
- | **Anthropic** | `anthropic/claude-3-5-sonnet-20241022` |
230
- | **Groq** | `groq/llama-3.1-70b-versatile` |
231
- | **Mistral** | `mistral/mistral-large-latest` |
232
- | **Cohere** | `cohere/command-r-plus` |
233
- | **Together AI** | `together/meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo` |
234
- | **Fireworks** | `fireworks/accounts/fireworks/models/llama-v3p1-70b-instruct` |
235
- | **Google Vertex** | `vertexai/gemini-1.5-pro` |
236
- | **Amazon Bedrock** | `bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0` |
254
+ | Provider | Example Model |
255
+ | ------------------ | ------------------------------------------------------------- |
256
+ | **OpenAI** | `openai/gpt-4o`, `openai/gpt-4o-mini` |
257
+ | **Anthropic** | `anthropic/claude-3-5-sonnet-20241022` |
258
+ | **Groq** | `groq/llama-3.1-70b-versatile` |
259
+ | **Mistral** | `mistral/mistral-large-latest` |
260
+ | **Cohere** | `cohere/command-r-plus` |
261
+ | **Together AI** | `together/meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo` |
262
+ | **Fireworks** | `fireworks/accounts/fireworks/models/llama-v3p1-70b-instruct` |
263
+ | **Google Vertex** | `vertexai/gemini-1.5-pro` |
264
+ | **Amazon Bedrock** | `bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0` |
237
265
 
238
266
  **[Complete Provider List](https://github.com/kreuzberg-dev/liter-llm/blob/main/schemas/providers.json)**
239
267
 
@@ -242,12 +270,8 @@ Route to any provider using the `provider/model` prefix convention:
242
270
  - **Provider Routing** -- Single client for 143+ LLM providers via `provider/model` prefix
243
271
  - **Local LLMs** — Connect to locally-hosted models via Ollama, LM Studio, vLLM, llama.cpp, and other local inference servers
244
272
  - **Unified API** -- Consistent `chat`, `chat_stream`, `embeddings`, `list_models` interface
245
-
246
273
  - **Streaming** -- Real-time token streaming via `chat_stream`
247
-
248
-
249
274
  - **Tool Calling** -- Function calling and tool use across all supporting providers
250
-
251
275
  - **Type Safe** -- Schema-driven types compiled from JSON schemas
252
276
  - **Secure** -- API keys never logged or serialized, managed via environment variables
253
277
  - **Observability** -- Built-in [OpenTelemetry](https://opentelemetry.io/docs/specs/semconv/gen-ai/) with GenAI semantic conventions
@@ -262,7 +286,6 @@ Built on a compiled Rust core for speed and safety:
262
286
  - **Zero-copy streaming** with SSE and AWS EventStream support
263
287
  - **API keys** wrapped in secure memory, zeroed on drop
264
288
 
265
-
266
289
  ## Provider Routing
267
290
 
268
291
  Route to 143+ providers using the `provider/model` prefix convention:
@@ -276,7 +299,6 @@ mistral/mistral-large-latest
276
299
 
277
300
  See the [provider registry](https://github.com/kreuzberg-dev/liter-llm/blob/main/schemas/providers.json) for the full list.
278
301
 
279
-
280
302
  ## Proxy Server
281
303
 
282
304
  liter-llm also ships as an OpenAI-compatible proxy server with Docker support:
@@ -285,7 +307,7 @@ liter-llm also ships as an OpenAI-compatible proxy server with Docker support:
285
307
  docker run -p 4000:4000 -e LITER_LLM_MASTER_KEY=sk-your-key ghcr.io/kreuzberg-dev/liter-llm
286
308
  ```
287
309
 
288
- See the [proxy server documentation](https://docs.liter-llm.kreuzberg.dev/server/proxy/) for configuration, CLI usage, and MCP integration.
310
+ See the [proxy server documentation](https://docs.liter-llm.kreuzberg.dev/server/proxy-server/) for configuration, CLI usage, and MCP integration.
289
311
 
290
312
  ## Documentation
291
313
 
@@ -293,7 +315,15 @@ See the [proxy server documentation](https://docs.liter-llm.kreuzberg.dev/server
293
315
  - **[GitHub Repository](https://github.com/kreuzberg-dev/liter-llm)** -- Source, issues, and discussions
294
316
  - **[Provider Registry](https://github.com/kreuzberg-dev/liter-llm/blob/main/schemas/providers.json)** -- 143 supported providers
295
317
 
296
- Part of [kreuzberg.dev](https://kreuzberg.dev).
318
+ ## Part of Kreuzberg.dev
319
+
320
+ - [Kreuzberg](https://github.com/kreuzberg-dev/kreuzberg) — document intelligence: text, tables, metadata from 90+ formats with optional OCR.
321
+ - [Kreuzberg Cloud](https://github.com/kreuzberg-dev/kreuzberg-cloud) — managed extraction API with SDKs, dashboards, and observability.
322
+ - [kreuzcrawl](https://github.com/kreuzberg-dev/kreuzcrawl) — web crawling and scraping with HTML→Markdown and headless-Chrome fallback.
323
+ - [html-to-markdown](https://github.com/kreuzberg-dev/html-to-markdown) — fast, lossless HTML→Markdown engine.
324
+ - [tree-sitter-language-pack](https://github.com/kreuzberg-dev/tree-sitter-language-pack) — tree-sitter grammars and code-intelligence primitives.
325
+ - [alef](https://github.com/kreuzberg-dev/alef) — the polyglot binding generator that produces this README and all per-language bindings.
326
+ - [Discord](https://discord.gg/xt9WY3GnKR) — community, roadmap, announcements.
297
327
 
298
328
  ## Contributing
299
329