liter_llm 1.5.1 → 1.6.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +83 -117
- data/ext/liter_llm_rb/native/Cargo.lock +396 -138
- data/ext/liter_llm_rb/native/Cargo.toml +8 -2
- data/ext/liter_llm_rb/src/lib.rs +627 -1
- data/lib/liter_llm/native.rb +1 -1
- data/lib/liter_llm/version.rb +2 -2
- data/lib/liter_llm.rb +1 -1
- data/sig/types.rbs +54 -1
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 7f9f444babeb7b9e9c37a44ab4e54f269626c1f0771d10320bd47d92d752c330
|
|
4
|
+
data.tar.gz: 9a1e93e18637b22847a7fcd944ac859194679b6d89a81a578e8767aeaf3baf00
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 8896d0498ce47f8f5a195b5302067dd968dcfd3fd44066eeee1af95ef1f2c524a2f3456e05211cbace7b47401fd18f480fab8fc2bc15ea8e50a7ab9d1727af24
|
|
7
|
+
data.tar.gz: d80c2ee6c0acca632e7800aac7ccfc062c4fa4f58f316ba38207c8d4426abf121a5af7a51e82b85abe5bec3ea69b45119e67f19e9bd8b571fee312cb369f13dd
|
data/README.md
CHANGED
|
@@ -1,136 +1,110 @@
|
|
|
1
1
|
# Ruby
|
|
2
2
|
|
|
3
|
-
<div
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
</a>
|
|
57
|
-
<a href="https://hex.pm/packages/liter_llm">
|
|
58
|
-
<img src="https://img.shields.io/hexpm/v/liter_llm?label=Elixir&color=007ec6" alt="Elixir" />
|
|
59
|
-
</a>
|
|
60
|
-
<a href="https://github.com/kreuzberg-dev/liter-llm/pkgs/container/liter-llm">
|
|
61
|
-
<img
|
|
62
|
-
src="https://img.shields.io/badge/Docker-007ec6?logo=docker&logoColor=white"
|
|
63
|
-
alt="Docker"
|
|
64
|
-
/>
|
|
65
|
-
</a>
|
|
66
|
-
<a href="https://github.com/kreuzberg-dev/homebrew-tap/blob/main/Formula/liter-llm.rb">
|
|
67
|
-
<img
|
|
68
|
-
src="https://img.shields.io/badge/Homebrew-007ec6?logo=homebrew&logoColor=white"
|
|
69
|
-
alt="Homebrew"
|
|
70
|
-
/>
|
|
71
|
-
</a>
|
|
72
|
-
<a href="https://github.com/kreuzberg-dev/liter-llm/tree/main/crates/liter-llm-ffi">
|
|
73
|
-
<img src="https://img.shields.io/badge/C-FFI-007ec6" alt="C FFI" />
|
|
74
|
-
</a>
|
|
75
|
-
|
|
76
|
-
<!-- Project Info -->
|
|
77
|
-
<a href="https://github.com/kreuzberg-dev/liter-llm/blob/main/LICENSE">
|
|
78
|
-
<img src="https://img.shields.io/badge/License-MIT-007ec6" alt="License" />
|
|
79
|
-
</a>
|
|
80
|
-
<a href="https://docs.liter-llm.kreuzberg.dev">
|
|
81
|
-
<img src="https://img.shields.io/badge/docs-kreuzberg.dev-007ec6" alt="Docs" />
|
|
82
|
-
</a>
|
|
3
|
+
<div align="center" style="display: flex; flex-wrap: wrap; gap: 8px; justify-content: center; margin: 20px 0">
|
|
4
|
+
<!-- Built with -->
|
|
5
|
+
<a href="https://github.com/kreuzberg-dev/alef">
|
|
6
|
+
<img src="https://img.shields.io/badge/Bindings-alef%20%D7%90-007ec6" alt="Bindings" />
|
|
7
|
+
</a>
|
|
8
|
+
<!-- Language Bindings -->
|
|
9
|
+
<a href="https://crates.io/crates/liter-llm">
|
|
10
|
+
<img src="https://img.shields.io/crates/v/liter-llm?label=Rust&color=007ec6" alt="Rust" />
|
|
11
|
+
</a>
|
|
12
|
+
<a href="https://pypi.org/project/liter-llm/">
|
|
13
|
+
<img src="https://img.shields.io/pypi/v/liter-llm?label=Python&color=007ec6" alt="Python" />
|
|
14
|
+
</a>
|
|
15
|
+
<a href="https://www.npmjs.com/package/@kreuzberg/liter-llm-node">
|
|
16
|
+
<img src="https://img.shields.io/npm/v/@kreuzberg/liter-llm-node?label=Node.js&color=007ec6" alt="Node.js" />
|
|
17
|
+
</a>
|
|
18
|
+
<a href="https://www.npmjs.com/package/@kreuzberg/liter-llm-wasm">
|
|
19
|
+
<img src="https://img.shields.io/npm/v/@kreuzberg/liter-llm-wasm?label=WASM&color=007ec6" alt="WASM" />
|
|
20
|
+
</a>
|
|
21
|
+
<a href="https://central.sonatype.com/artifact/dev.kreuzberg.literllm/liter-llm">
|
|
22
|
+
<img src="https://img.shields.io/maven-central/v/dev.kreuzberg.literllm/liter-llm?label=Java&color=007ec6" alt="Java" />
|
|
23
|
+
</a>
|
|
24
|
+
<a href="https://github.com/kreuzberg-dev/liter-llm/tree/main/packages/go">
|
|
25
|
+
<img src="https://img.shields.io/github/v/tag/kreuzberg-dev/liter-llm?label=Go&color=007ec6" alt="Go" />
|
|
26
|
+
</a>
|
|
27
|
+
<a href="https://www.nuget.org/packages/LiterLlm">
|
|
28
|
+
<img src="https://img.shields.io/nuget/v/LiterLlm?label=C%23&color=007ec6" alt="C#" />
|
|
29
|
+
</a>
|
|
30
|
+
<a href="https://packagist.org/packages/kreuzberg-dev/liter-llm">
|
|
31
|
+
<img src="https://img.shields.io/packagist/v/kreuzberg-dev/liter-llm?label=PHP&color=007ec6" alt="PHP" />
|
|
32
|
+
</a>
|
|
33
|
+
<a href="https://rubygems.org/gems/liter_llm">
|
|
34
|
+
<img src="https://img.shields.io/gem/v/liter_llm?label=Ruby&color=007ec6" alt="Ruby" />
|
|
35
|
+
</a>
|
|
36
|
+
<a href="https://hex.pm/packages/liter_llm">
|
|
37
|
+
<img src="https://img.shields.io/hexpm/v/liter_llm?label=Elixir&color=007ec6" alt="Elixir" />
|
|
38
|
+
</a>
|
|
39
|
+
<a href="https://github.com/kreuzberg-dev/liter-llm/pkgs/container/liter-llm">
|
|
40
|
+
<img src="https://img.shields.io/badge/Docker-007ec6?logo=docker&logoColor=white" alt="Docker" />
|
|
41
|
+
</a>
|
|
42
|
+
<a href="https://github.com/kreuzberg-dev/homebrew-tap/blob/main/Formula/liter-llm.rb">
|
|
43
|
+
<img src="https://img.shields.io/badge/Homebrew-007ec6?logo=homebrew&logoColor=white" alt="Homebrew" />
|
|
44
|
+
</a>
|
|
45
|
+
<a href="https://github.com/kreuzberg-dev/liter-llm/tree/main/crates/liter-llm-ffi">
|
|
46
|
+
<img src="https://img.shields.io/badge/C-FFI-007ec6" alt="C FFI" />
|
|
47
|
+
</a>
|
|
48
|
+
|
|
49
|
+
<!-- Project Info -->
|
|
50
|
+
<a href="https://github.com/kreuzberg-dev/liter-llm/blob/main/LICENSE">
|
|
51
|
+
<img src="https://img.shields.io/badge/License-MIT-007ec6" alt="License" />
|
|
52
|
+
</a>
|
|
53
|
+
<a href="https://docs.liter-llm.kreuzberg.dev">
|
|
54
|
+
<img src="https://img.shields.io/badge/Docs-liter--llm-007ec6" alt="Docs" />
|
|
55
|
+
</a>
|
|
83
56
|
</div>
|
|
84
|
-
<div align="center" style="margin:
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
</picture>
|
|
57
|
+
<div align="center" style="margin: 24px 0 0">
|
|
58
|
+
<a href="https://kreuzberg.dev">
|
|
59
|
+
<img
|
|
60
|
+
alt="kreuzberg.dev"
|
|
61
|
+
src="https://github.com/user-attachments/assets/1b6c6ad7-3b6d-4171-b1c9-f2026cc9deb8"
|
|
62
|
+
/>
|
|
63
|
+
</a>
|
|
92
64
|
</div>
|
|
93
|
-
<div align="center" style="
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
65
|
+
<div align="center" style="display: flex; flex-wrap: wrap; gap: 12px; justify-content: center; margin: 28px 0 24px">
|
|
66
|
+
<a href="https://discord.gg/xt9WY3GnKR">
|
|
67
|
+
<img
|
|
68
|
+
height="22"
|
|
69
|
+
src="https://img.shields.io/badge/Discord-Chat-007ec6?logo=discord&logoColor=white"
|
|
70
|
+
alt="Join Discord"
|
|
71
|
+
/>
|
|
72
|
+
</a>
|
|
101
73
|
</div>
|
|
102
74
|
|
|
103
|
-
Universal LLM API client for Ruby. Access 143
|
|
75
|
+
Universal LLM API client for Ruby. Access 143 LLM providers through a single interface with idiomatic Ruby API and native performance.
|
|
76
|
+
|
|
77
|
+
## What This Package Provides
|
|
78
|
+
|
|
79
|
+
- **One provider surface** — chat, streaming, embeddings, images, audio, search, OCR, tools, and structured output across the provider registry.
|
|
80
|
+
- **Provider/model routing** — call models with the `provider/model` convention and keep provider-specific request code out of application paths.
|
|
81
|
+
- **Production controls** — retries, fallback, rate limits, cache layers, budgets, health checks, OpenTelemetry spans, and redacted secrets.
|
|
82
|
+
- **Same core as every binding** — Rust, Python, Node.js, Go, Java, PHP, Ruby, .NET, Elixir, WASM, Kotlin Android, Swift, Dart, Zig, and C FFI use the same Rust implementation.
|
|
83
|
+
- **Ruby package** — native extension with idiomatic Ruby request and response objects.
|
|
104
84
|
|
|
105
85
|
## Installation
|
|
106
86
|
|
|
107
87
|
### Package Installation
|
|
108
88
|
|
|
109
|
-
|
|
110
89
|
Install via one of the supported package managers:
|
|
111
90
|
|
|
112
|
-
|
|
113
91
|
**gem:**
|
|
114
92
|
|
|
115
93
|
```bash
|
|
116
94
|
gem install liter_llm
|
|
117
95
|
```
|
|
118
96
|
|
|
119
|
-
|
|
120
97
|
**Bundler:**
|
|
121
98
|
|
|
122
99
|
```ruby
|
|
123
100
|
gem 'liter_llm'
|
|
124
101
|
```
|
|
125
102
|
|
|
126
|
-
|
|
127
103
|
### System Requirements
|
|
128
104
|
|
|
129
|
-
|
|
130
105
|
- **Ruby 3.2+** required
|
|
131
106
|
- API keys via environment variables (e.g. `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`)
|
|
132
107
|
|
|
133
|
-
|
|
134
108
|
## Quick Start
|
|
135
109
|
|
|
136
110
|
### Basic Chat
|
|
@@ -156,7 +130,6 @@ puts result.choices[0].message.content
|
|
|
156
130
|
|
|
157
131
|
### Common Use Cases
|
|
158
132
|
|
|
159
|
-
|
|
160
133
|
#### Streaming Responses
|
|
161
134
|
|
|
162
135
|
Stream tokens in real time:
|
|
@@ -181,16 +154,14 @@ end
|
|
|
181
154
|
puts
|
|
182
155
|
```
|
|
183
156
|
|
|
184
|
-
|
|
185
157
|
### Next Steps
|
|
186
158
|
|
|
187
159
|
- **[Provider Registry](https://github.com/kreuzberg-dev/liter-llm/blob/main/schemas/providers.json)** - Full list of supported providers
|
|
188
160
|
- **[GitHub Repository](https://github.com/kreuzberg-dev/liter-llm)** - Source, issues, and discussions
|
|
189
161
|
|
|
190
|
-
|
|
191
162
|
## Features
|
|
192
163
|
|
|
193
|
-
### Supported Providers (143
|
|
164
|
+
### Supported Providers (143)
|
|
194
165
|
|
|
195
166
|
Route to any provider using the `provider/model` prefix convention:
|
|
196
167
|
|
|
@@ -210,15 +181,11 @@ Route to any provider using the `provider/model` prefix convention:
|
|
|
210
181
|
|
|
211
182
|
### Key Capabilities
|
|
212
183
|
|
|
213
|
-
- **Provider Routing** -- Single client for 143
|
|
184
|
+
- **Provider Routing** -- Single client for 143 LLM providers via `provider/model` prefix
|
|
214
185
|
- **Local LLMs** — Connect to locally-hosted models via Ollama, LM Studio, vLLM, llama.cpp, and other local inference servers
|
|
215
186
|
- **Unified API** -- Consistent `chat`, `chat_stream`, `embeddings`, `list_models` interface
|
|
216
|
-
|
|
217
187
|
- **Streaming** -- Real-time token streaming via `chat_stream`
|
|
218
|
-
|
|
219
|
-
|
|
220
188
|
- **Tool Calling** -- Function calling and tool use across all supporting providers
|
|
221
|
-
|
|
222
189
|
- **Type Safe** -- Schema-driven types compiled from JSON schemas
|
|
223
190
|
- **Secure** -- API keys never logged or serialized, managed via environment variables
|
|
224
191
|
- **Observability** -- Built-in [OpenTelemetry](https://opentelemetry.io/docs/specs/semconv/gen-ai/) with GenAI semantic conventions
|
|
@@ -233,10 +200,9 @@ Built on a compiled Rust core for speed and safety:
|
|
|
233
200
|
- **Zero-copy streaming** with SSE and AWS EventStream support
|
|
234
201
|
- **API keys** wrapped in secure memory, zeroed on drop
|
|
235
202
|
|
|
236
|
-
|
|
237
203
|
## Provider Routing
|
|
238
204
|
|
|
239
|
-
Route to 143
|
|
205
|
+
Route to 143 providers using the `provider/model` prefix convention:
|
|
240
206
|
|
|
241
207
|
```text
|
|
242
208
|
openai/gpt-4o
|
|
@@ -247,7 +213,6 @@ mistral/mistral-large-latest
|
|
|
247
213
|
|
|
248
214
|
See the [provider registry](https://github.com/kreuzberg-dev/liter-llm/blob/main/schemas/providers.json) for the full list.
|
|
249
215
|
|
|
250
|
-
|
|
251
216
|
## Proxy Server
|
|
252
217
|
|
|
253
218
|
liter-llm also ships as an OpenAI-compatible proxy server with Docker support:
|
|
@@ -256,7 +221,7 @@ liter-llm also ships as an OpenAI-compatible proxy server with Docker support:
|
|
|
256
221
|
docker run -p 4000:4000 -e LITER_LLM_MASTER_KEY=sk-your-key ghcr.io/kreuzberg-dev/liter-llm
|
|
257
222
|
```
|
|
258
223
|
|
|
259
|
-
See the [proxy server documentation](https://docs.liter-llm.kreuzberg.dev/server/proxy/) for configuration, CLI usage, and MCP integration.
|
|
224
|
+
See the [proxy server documentation](https://docs.liter-llm.kreuzberg.dev/server/proxy-server/) for configuration, CLI usage, and MCP integration.
|
|
260
225
|
|
|
261
226
|
## Documentation
|
|
262
227
|
|
|
@@ -270,8 +235,9 @@ See the [proxy server documentation](https://docs.liter-llm.kreuzberg.dev/server
|
|
|
270
235
|
- [Kreuzberg Cloud](https://github.com/kreuzberg-dev/kreuzberg-cloud) — managed extraction API with SDKs, dashboards, and observability.
|
|
271
236
|
- [kreuzcrawl](https://github.com/kreuzberg-dev/kreuzcrawl) — web crawling and scraping with HTML→Markdown and headless-Chrome fallback.
|
|
272
237
|
- [html-to-markdown](https://github.com/kreuzberg-dev/html-to-markdown) — fast, lossless HTML→Markdown engine.
|
|
238
|
+
- [liter-llm](https://github.com/kreuzberg-dev/liter-llm) — universal LLM API client with native bindings for 14 languages and 143 providers.
|
|
273
239
|
- [tree-sitter-language-pack](https://github.com/kreuzberg-dev/tree-sitter-language-pack) — tree-sitter grammars and code-intelligence primitives.
|
|
274
|
-
- [alef](https://github.com/kreuzberg-dev/alef) — the polyglot binding generator that produces
|
|
240
|
+
- [alef](https://github.com/kreuzberg-dev/alef) — the polyglot binding generator that produces every per-language binding across the 5 polyglot repos.
|
|
275
241
|
- [Discord](https://discord.gg/xt9WY3GnKR) — community, roadmap, announcements.
|
|
276
242
|
|
|
277
243
|
## Contributing
|