@kreuzberg/liter-llm-node 1.4.1 → 1.5.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (7) hide show
  1. package/README.md +116 -78
  2. package/index.d.ts +360 -386
  3. package/liter-llm-node.darwin-arm64.node +0 -0
  4. package/liter-llm-node.linux-arm64-gnu.node +0 -0
  5. package/liter-llm-node.linux-x64-gnu.node +0 -0
  6. package/liter-llm-node.win32-x64-msvc.node +0 -0
  7. package/package.json +10 -10
package/README.md CHANGED
@@ -1,113 +1,139 @@
1
1
  # TypeScript (Node.js)
2
2
 
3
- <div align="center" style="display: flex; flex-wrap: wrap; gap: 8px; justify-content: center; margin: 20px 0">
4
- <!-- Built with -->
5
- <a href="https://github.com/kreuzberg-dev/alef">
6
- <img src="https://img.shields.io/badge/Bindings-alef%20%D7%90-007ec6" alt="Bindings" />
7
- </a>
8
- <!-- Language Bindings -->
9
- <a href="https://crates.io/crates/liter-llm">
10
- <img src="https://img.shields.io/crates/v/liter-llm?label=Rust&color=007ec6" alt="Rust" />
11
- </a>
12
- <a href="https://pypi.org/project/liter-llm/">
13
- <img src="https://img.shields.io/pypi/v/liter-llm?label=Python&color=007ec6" alt="Python" />
14
- </a>
15
- <a href="https://www.npmjs.com/package/@kreuzberg/liter-llm">
16
- <img src="https://img.shields.io/npm/v/@kreuzberg/liter-llm?label=Node.js&color=007ec6" alt="Node.js" />
17
- </a>
18
- <a href="https://www.npmjs.com/package/@kreuzberg/liter-llm-wasm">
19
- <img src="https://img.shields.io/npm/v/@kreuzberg/liter-llm-wasm?label=WASM&color=007ec6" alt="WASM" />
20
- </a>
21
- <a href="https://central.sonatype.com/artifact/dev.kreuzberg/liter-llm">
22
- <img src="https://img.shields.io/maven-central/v/dev.kreuzberg/liter-llm?label=Java&color=007ec6" alt="Java" />
23
- </a>
24
- <a href="https://github.com/kreuzberg-dev/liter-llm/tree/main/packages/go">
25
- <img src="https://img.shields.io/github/v/tag/kreuzberg-dev/liter-llm?label=Go&color=007ec6" alt="Go" />
26
- </a>
27
- <a href="https://www.nuget.org/packages/LiterLlm">
28
- <img src="https://img.shields.io/nuget/v/LiterLlm?label=C%23&color=007ec6" alt="C#" />
29
- </a>
30
- <a href="https://packagist.org/packages/kreuzberg/liter-llm">
31
- <img src="https://img.shields.io/packagist/v/kreuzberg/liter-llm?label=PHP&color=007ec6" alt="PHP" />
32
- </a>
33
- <a href="https://rubygems.org/gems/liter_llm">
34
- <img src="https://img.shields.io/gem/v/liter_llm?label=Ruby&color=007ec6" alt="Ruby" />
35
- </a>
36
- <a href="https://hex.pm/packages/liter_llm">
37
- <img src="https://img.shields.io/hexpm/v/liter_llm?label=Elixir&color=007ec6" alt="Elixir" />
38
- </a>
39
- <a href="https://github.com/kreuzberg-dev/liter-llm/pkgs/container/liter-llm">
40
- <img src="https://img.shields.io/badge/Docker-007ec6?logo=docker&logoColor=white" alt="Docker" />
41
- </a>
42
- <a href="https://github.com/kreuzberg-dev/homebrew-tap/blob/main/Formula/liter-llm.rb">
43
- <img src="https://img.shields.io/badge/Homebrew-007ec6?logo=homebrew&logoColor=white" alt="Homebrew" />
44
- </a>
45
- <a href="https://github.com/kreuzberg-dev/liter-llm/tree/main/crates/liter-llm-ffi">
46
- <img src="https://img.shields.io/badge/C-FFI-007ec6" alt="C FFI" />
47
- </a>
48
-
49
- <!-- Project Info -->
50
- <a href="https://github.com/kreuzberg-dev/liter-llm/blob/main/LICENSE">
51
- <img src="https://img.shields.io/badge/License-MIT-007ec6" alt="License" />
52
- </a>
53
- <a href="https://docs.liter-llm.kreuzberg.dev">
54
- <img src="https://img.shields.io/badge/Docs-liter--llm-007ec6" alt="Docs" />
55
- </a>
3
+ <div
4
+ align="center"
5
+ style="display: flex; flex-wrap: wrap; gap: 8px; justify-content: center; margin: 20px 0"
6
+ >
7
+ <!-- Built with -->
8
+ <a href="https://github.com/kreuzberg-dev/alef">
9
+ <img
10
+ src="https://img.shields.io/badge/built%20with-alef%20%D7%90-007ec6"
11
+ alt="Built with alef"
12
+ />
13
+ </a>
14
+ <!-- Language Bindings -->
15
+ <a href="https://crates.io/crates/liter-llm">
16
+ <img src="https://img.shields.io/crates/v/liter-llm?label=Rust&color=007ec6" alt="Rust" />
17
+ </a>
18
+ <a href="https://pypi.org/project/liter-llm/">
19
+ <img src="https://img.shields.io/pypi/v/liter-llm?label=Python&color=007ec6" alt="Python" />
20
+ </a>
21
+ <a href="https://www.npmjs.com/package/@kreuzberg/liter-llm">
22
+ <img
23
+ src="https://img.shields.io/npm/v/@kreuzberg/liter-llm?label=Node.js&color=007ec6"
24
+ alt="Node.js"
25
+ />
26
+ </a>
27
+ <a href="https://www.npmjs.com/package/@kreuzberg/liter-llm-wasm">
28
+ <img
29
+ src="https://img.shields.io/npm/v/@kreuzberg/liter-llm-wasm?label=WASM&color=007ec6"
30
+ alt="WASM"
31
+ />
32
+ </a>
33
+ <a href="https://central.sonatype.com/artifact/dev.kreuzberg/liter-llm">
34
+ <img
35
+ src="https://img.shields.io/maven-central/v/dev.kreuzberg/liter-llm?label=Java&color=007ec6"
36
+ alt="Java"
37
+ />
38
+ </a>
39
+ <a href="https://github.com/kreuzberg-dev/liter-llm/tree/main/packages/go">
40
+ <img
41
+ src="https://img.shields.io/github/v/tag/kreuzberg-dev/liter-llm?label=Go&color=007ec6"
42
+ alt="Go"
43
+ />
44
+ </a>
45
+ <a href="https://www.nuget.org/packages/LiterLlm">
46
+ <img src="https://img.shields.io/nuget/v/LiterLlm?label=C%23&color=007ec6" alt="C#" />
47
+ </a>
48
+ <a href="https://packagist.org/packages/kreuzberg/liter-llm">
49
+ <img
50
+ src="https://img.shields.io/packagist/v/kreuzberg/liter-llm?label=PHP&color=007ec6"
51
+ alt="PHP"
52
+ />
53
+ </a>
54
+ <a href="https://rubygems.org/gems/liter_llm">
55
+ <img src="https://img.shields.io/gem/v/liter_llm?label=Ruby&color=007ec6" alt="Ruby" />
56
+ </a>
57
+ <a href="https://hex.pm/packages/liter_llm">
58
+ <img src="https://img.shields.io/hexpm/v/liter_llm?label=Elixir&color=007ec6" alt="Elixir" />
59
+ </a>
60
+ <a href="https://github.com/kreuzberg-dev/liter-llm/pkgs/container/liter-llm">
61
+ <img
62
+ src="https://img.shields.io/badge/Docker-007ec6?logo=docker&logoColor=white"
63
+ alt="Docker"
64
+ />
65
+ </a>
66
+ <a href="https://github.com/kreuzberg-dev/homebrew-tap/blob/main/Formula/liter-llm.rb">
67
+ <img
68
+ src="https://img.shields.io/badge/Homebrew-007ec6?logo=homebrew&logoColor=white"
69
+ alt="Homebrew"
70
+ />
71
+ </a>
72
+ <a href="https://github.com/kreuzberg-dev/liter-llm/tree/main/crates/liter-llm-ffi">
73
+ <img src="https://img.shields.io/badge/C-FFI-007ec6" alt="C FFI" />
74
+ </a>
75
+
76
+ <!-- Project Info -->
77
+ <a href="https://github.com/kreuzberg-dev/liter-llm/blob/main/LICENSE">
78
+ <img src="https://img.shields.io/badge/License-MIT-007ec6" alt="License" />
79
+ </a>
80
+ <a href="https://docs.liter-llm.kreuzberg.dev">
81
+ <img src="https://img.shields.io/badge/docs-kreuzberg.dev-007ec6" alt="Docs" />
82
+ </a>
56
83
  </div>
57
- <div align="center" style="margin: 24px 0 0">
58
- <a href="https://kreuzberg.dev">
59
- <img
60
- alt="kreuzberg.dev"
61
- src="https://github.com/user-attachments/assets/1b6c6ad7-3b6d-4171-b1c9-f2026cc9deb8"
62
- />
63
- </a>
84
+ <div align="center" style="margin: 20px 0">
85
+ <picture>
86
+ <img
87
+ width="100%"
88
+ alt="kreuzberg.dev"
89
+ src="https://github.com/user-attachments/assets/1b6c6ad7-3b6d-4171-b1c9-f2026cc9deb8"
90
+ />
91
+ </picture>
64
92
  </div>
65
- <div align="center" style="display: flex; flex-wrap: wrap; gap: 12px; justify-content: center; margin: 28px 0 24px">
66
- <a href="https://discord.gg/xt9WY3GnKR">
67
- <img
68
- height="22"
69
- src="https://img.shields.io/badge/Discord-Chat-007ec6?logo=discord&logoColor=white"
70
- alt="Join Discord"
71
- />
72
- </a>
93
+ <div align="center" style="margin-bottom: 20px">
94
+ <a href="https://discord.gg/xt9WY3GnKR">
95
+ <img
96
+ height="22"
97
+ src="https://img.shields.io/badge/Discord-Join%20our%20community-7289da?logo=discord&logoColor=white"
98
+ alt="Discord"
99
+ />
100
+ </a>
73
101
  </div>
74
102
 
75
103
  Universal LLM API client for TypeScript and Node.js. Access 143+ LLM providers through a single interface with native NAPI-RS bindings, async/await, streaming, tool calling, and full TypeScript type definitions.
76
104
 
77
- ## What This Package Provides
78
-
79
- - **One provider surface** — chat, streaming, embeddings, images, audio, search, OCR, tools, and structured output across the provider registry.
80
- - **Provider/model routing** — call models with the `provider/model` convention and keep provider-specific request code out of application paths.
81
- - **Production controls** — retries, fallback, rate limits, cache layers, budgets, health checks, OpenTelemetry spans, and redacted secrets.
82
- - **Same core as every binding** — Rust, Python, Node.js, Go, Java, PHP, Ruby, .NET, Elixir, WASM, Kotlin Android, Swift, Dart, Zig, and C FFI use the same Rust implementation.
83
- - **Node-first TypeScript API** — NAPI-RS package with typed requests/responses and async iterables for streaming.
84
-
85
105
  ## Installation
86
106
 
87
107
  ### Package Installation
88
108
 
109
+
89
110
  Install via one of the supported package managers:
90
111
 
112
+
91
113
  **npm:**
92
114
 
93
115
  ```bash
94
116
  npm install @kreuzberg/liter-llm
95
117
  ```
96
118
 
119
+
97
120
  **pnpm:**
98
121
 
99
122
  ```bash
100
123
  pnpm add @kreuzberg/liter-llm
101
124
  ```
102
125
 
126
+
103
127
  **yarn:**
104
128
 
105
129
  ```bash
106
130
  yarn add @kreuzberg/liter-llm
107
131
  ```
108
132
 
133
+
109
134
  ### System Requirements
110
135
 
136
+
111
137
  - **Node.js 22+** required (NAPI-RS native bindings)
112
138
  - API keys via environment variables (e.g. `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`)
113
139
 
@@ -119,6 +145,7 @@ Pre-built binaries available for:
119
145
  - Linux (x64)
120
146
  - Windows (x64)
121
147
 
148
+
122
149
  ## Quick Start
123
150
 
124
151
  ### Basic Chat
@@ -138,6 +165,7 @@ console.log(response.choices[0].message.content);
138
165
 
139
166
  ### Common Use Cases
140
167
 
168
+
141
169
  #### Streaming Responses
142
170
 
143
171
  Stream tokens in real time:
@@ -157,6 +185,7 @@ for (const chunk of chunks) {
157
185
  console.log();
158
186
  ```
159
187
 
188
+
160
189
  #### Tool Calling
161
190
 
162
191
  Define and invoke tools:
@@ -190,11 +219,13 @@ for (const call of response.choices[0]?.message?.toolCalls ?? []) {
190
219
  }
191
220
  ```
192
221
 
222
+
193
223
  ### Next Steps
194
224
 
195
225
  - **[Provider Registry](https://github.com/kreuzberg-dev/liter-llm/blob/main/schemas/providers.json)** - Full list of supported providers
196
226
  - **[GitHub Repository](https://github.com/kreuzberg-dev/liter-llm)** - Source, issues, and discussions
197
227
 
228
+
198
229
  ## NAPI-RS Implementation Details
199
230
 
200
231
  ### Native Performance
@@ -218,6 +249,7 @@ This binding uses NAPI-RS to provide native Node.js bindings with:
218
249
  - Streaming buffers are released as soon as each chunk is consumed
219
250
  - Provider registry is compiled into the binary — no runtime disk access
220
251
 
252
+
221
253
  ## Features
222
254
 
223
255
  ### Supported Providers (143+)
@@ -243,8 +275,12 @@ Route to any provider using the `provider/model` prefix convention:
243
275
  - **Provider Routing** -- Single client for 143+ LLM providers via `provider/model` prefix
244
276
  - **Local LLMs** — Connect to locally-hosted models via Ollama, LM Studio, vLLM, llama.cpp, and other local inference servers
245
277
  - **Unified API** -- Consistent `chat`, `chat_stream`, `embeddings`, `list_models` interface
278
+
246
279
  - **Streaming** -- Real-time token streaming via `chat_stream`
280
+
281
+
247
282
  - **Tool Calling** -- Function calling and tool use across all supporting providers
283
+
248
284
  - **Type Safe** -- Schema-driven types compiled from JSON schemas
249
285
  - **Secure** -- API keys never logged or serialized, managed via environment variables
250
286
  - **Observability** -- Built-in [OpenTelemetry](https://opentelemetry.io/docs/specs/semconv/gen-ai/) with GenAI semantic conventions
@@ -259,6 +295,7 @@ Built on a compiled Rust core for speed and safety:
259
295
  - **Zero-copy streaming** with SSE and AWS EventStream support
260
296
  - **API keys** wrapped in secure memory, zeroed on drop
261
297
 
298
+
262
299
  ## Provider Routing
263
300
 
264
301
  Route to 143+ providers using the `provider/model` prefix convention:
@@ -272,6 +309,7 @@ mistral/mistral-large-latest
272
309
 
273
310
  See the [provider registry](https://github.com/kreuzberg-dev/liter-llm/blob/main/schemas/providers.json) for the full list.
274
311
 
312
+
275
313
  ## Proxy Server
276
314
 
277
315
  liter-llm also ships as an OpenAI-compatible proxy server with Docker support:
@@ -280,7 +318,7 @@ liter-llm also ships as an OpenAI-compatible proxy server with Docker support:
280
318
  docker run -p 4000:4000 -e LITER_LLM_MASTER_KEY=sk-your-key ghcr.io/kreuzberg-dev/liter-llm
281
319
  ```
282
320
 
283
- See the [proxy server documentation](https://docs.liter-llm.kreuzberg.dev/server/proxy-server/) for configuration, CLI usage, and MCP integration.
321
+ See the [proxy server documentation](https://docs.liter-llm.kreuzberg.dev/server/proxy/) for configuration, CLI usage, and MCP integration.
284
322
 
285
323
  ## Documentation
286
324
 
@@ -290,7 +328,7 @@ See the [proxy server documentation](https://docs.liter-llm.kreuzberg.dev/server
290
328
 
291
329
  ## Part of Kreuzberg.dev
292
330
 
293
- - [Kreuzberg](https://github.com/kreuzberg-dev/kreuzberg) — document intelligence: text, tables, metadata from 90+ formats with optional OCR.
331
+ - [Kreuzberg](https://github.com/kreuzberg-dev/kreuzberg) — document intelligence: text, tables, metadata from 91+ formats with optional OCR.
294
332
  - [Kreuzberg Cloud](https://github.com/kreuzberg-dev/kreuzberg-cloud) — managed extraction API with SDKs, dashboards, and observability.
295
333
  - [kreuzcrawl](https://github.com/kreuzberg-dev/kreuzcrawl) — web crawling and scraping with HTML→Markdown and headless-Chrome fallback.
296
334
  - [html-to-markdown](https://github.com/kreuzberg-dev/html-to-markdown) — fast, lossless HTML→Markdown engine.