@xberg-io/liter-llm 0.0.1 → 1.9.0-rc.2

package/LICENSE

@@ -0,0 +1,7 @@
+Copyright 2026 Kreuzberg, Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

package/README.md

@@ -1,3 +1,327 @@
-# @xberg-io/liter-llm
+<p align="center">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" srcset="https://cdn.jsdelivr.net/gh/xberg-io/assets@v1/banner/readme-banner-dark.svg">
+    <img alt="Xberg" width="420" src="https://cdn.jsdelivr.net/gh/xberg-io/assets@v1/banner/readme-banner-light.svg">
+  </picture>
+</p>
 
-Placeholder release. See https://github.com/xberg-io/liter-llm
+# TypeScript (Node.js)
+
+<div align="center" style="display: flex; flex-wrap: wrap; gap: 8px; justify-content: center; margin: 20px 0">
+	<!-- Built with -->
+	<a href="https://github.com/xberg-io/alef">
+		<img src="https://img.shields.io/badge/Bindings-alef%20%D7%90-007ec6" alt="Bindings" />
+	</a>
+	<!-- Language Bindings -->
+	<a href="https://crates.io/crates/liter-llm">
+		<img src="https://img.shields.io/crates/v/liter-llm?label=Rust&color=007ec6" alt="Rust" />
+	</a>
+	<a href="https://pypi.org/project/liter-llm/">
+		<img src="https://img.shields.io/pypi/v/liter-llm?label=Python&color=007ec6" alt="Python" />
+	</a>
+	<a href="https://www.npmjs.com/package/@xberg-io/liter-llm">
+		<img src="https://img.shields.io/npm/v/@xberg-io/liter-llm?label=Node.js&color=007ec6" alt="Node.js" />
+	</a>
+	<a href="https://www.npmjs.com/package/@xberg-io/liter-llm-wasm">
+		<img src="https://img.shields.io/npm/v/@xberg-io/liter-llm-wasm?label=WASM&color=007ec6" alt="WASM" />
+	</a>
+	<a href="https://central.sonatype.com/artifact/io.xberg.literllm/liter-llm">
+		<img src="https://img.shields.io/maven-central/v/io.xberg.literllm/liter-llm?label=Java&color=007ec6" alt="Java" />
+	</a>
+	<a href="https://github.com/xberg-io/liter-llm/tree/main/packages/go">
+		<img src="https://img.shields.io/github/v/tag/xberg-io/liter-llm?label=Go&color=007ec6" alt="Go" />
+	</a>
+	<a href="https://www.nuget.org/packages/LiterLlm">
+		<img src="https://img.shields.io/nuget/v/LiterLlm?label=C%23&color=007ec6" alt="C#" />
+	</a>
+	<a href="https://packagist.org/packages/xberg-io/liter-llm">
+		<img src="https://img.shields.io/packagist/v/xberg-io/liter-llm?label=PHP&color=007ec6" alt="PHP" />
+	</a>
+	<a href="https://rubygems.org/gems/liter_llm">
+		<img src="https://img.shields.io/gem/v/liter_llm?label=Ruby&color=007ec6" alt="Ruby" />
+	</a>
+	<a href="https://hex.pm/packages/liter_llm">
+		<img src="https://img.shields.io/hexpm/v/liter_llm?label=Elixir&color=007ec6" alt="Elixir" />
+	</a>
+	<a href="https://github.com/xberg-io/liter-llm/pkgs/container/liter-llm">
+		<img src="https://img.shields.io/badge/Docker-007ec6?logo=docker&logoColor=white" alt="Docker" />
+	</a>
+	<a href="https://github.com/xberg-io/homebrew-tap/blob/main/Formula/liter-llm.rb">
+		<img src="https://img.shields.io/badge/Homebrew-007ec6?logo=homebrew&logoColor=white" alt="Homebrew" />
+	</a>
+	<a href="https://github.com/xberg-io/liter-llm/tree/main/crates/liter-llm-ffi">
+		<img src="https://img.shields.io/badge/C-FFI-007ec6" alt="C FFI" />
+	</a>
+
+	<!-- Project Info -->
+	<a href="https://github.com/xberg-io/liter-llm/blob/main/LICENSE">
+		<img src="https://img.shields.io/badge/License-MIT-007ec6" alt="License" />
+	</a>
+	<a href="https://docs.liter-llm.xberg.io">
+		<img src="https://img.shields.io/badge/Docs-liter--llm-007ec6" alt="Docs" />
+	</a>
+</div>
+<div align="center" style="margin: 24px 0 0">
+	<a href="https://xberg.io">
+		<img
+			alt="xberg.io"
+			src="https://github.com/user-attachments/assets/1b6c6ad7-3b6d-4171-b1c9-f2026cc9deb8"
+		/>
+	</a>
+</div>
+<div align="center" style="display: flex; flex-wrap: wrap; gap: 12px; justify-content: center; margin: 28px 0 24px">
+	<a href="https://discord.gg/xt9WY3GnKR">
+		<img
+			height="22"
+			src="https://img.shields.io/badge/Discord-Chat-007ec6?logo=discord&logoColor=white"
+			alt="Join Discord"
+		/>
+	</a>
+</div>
+
+Universal LLM API client for TypeScript and Node.js. Access 143 LLM providers through a single interface with native NAPI-RS bindings, async/await, streaming, tool calling, and full TypeScript type definitions.
+
+## What This Package Provides
+
+- **One provider surface** — chat, streaming, embeddings, images, audio, search, OCR, tools, and structured output across the provider registry.
+- **Provider/model routing** — call models with the `provider/model` convention and keep provider-specific request code out of application paths.
+- **Production controls** — retries, fallback, rate limits, cache layers, budgets, health checks, OpenTelemetry spans, and redacted secrets.
+- **Same core as every binding** — Rust, Python, Node.js, Go, Java, PHP, Ruby, .NET, Elixir, WASM, Kotlin Android, Swift, Dart, Zig, and C FFI use the same Rust implementation.
+- **Node-first TypeScript API** — NAPI-RS package with typed requests/responses and async iterables for streaming.
+
+## Installation
+
+### Package Installation
+
+Install via one of the supported package managers:
+
+**npm:**
+
+```bash
+npm install @xberg-io/liter-llm
+```
+
+**pnpm:**
+
+```bash
+pnpm add @xberg-io/liter-llm
+```
+
+**yarn:**
+
+```bash
+yarn add @xberg-io/liter-llm
+```
+
+### System Requirements
+
+- **Node.js 22+** required (NAPI-RS native bindings)
+- API keys via environment variables (e.g. `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`)
+
+### Platform Support
+
+Pre-built binaries available for:
+
+- macOS (arm64, x64)
+- Linux (x64)
+- Windows (x64)
+
+## Quick Start
+
+### Basic Chat
+
+Send a message to any provider using the `provider/model` prefix:
+
+```typescript
+import { createClient } from "@xberg-io/liter-llm";
+
+const client = createClient(process.env.OPENAI_API_KEY!);
+const response = await client.chat({
+  model: "openai/gpt-4o",
+  messages: [{ role: "user", content: "Hello!" }],
+});
+console.log(response.choices[0].message.content);
+```
+
+### Common Use Cases
+
+#### Streaming Responses
+
+Stream tokens in real time:
+
+```typescript
+import { createClient } from "@xberg-io/liter-llm";
+
+const client = createClient(process.env.OPENAI_API_KEY!);
+const chunks = await client.chatStream({
+  model: "openai/gpt-4o",
+  messages: [{ role: "user", content: "Tell me a story" }],
+});
+
+for await (const chunk of chunks) {
+  process.stdout.write(chunk.choices?.[0]?.delta?.content ?? "");
+}
+console.log();
+```
+
+#### Tool Calling
+
+Define and invoke tools:
+
+```typescript
+import { createClient, ToolType } from "@xberg-io/liter-llm";
+
+const client = createClient(process.env.OPENAI_API_KEY!);
+
+const response = await client.chat({
+  model: "openai/gpt-4o",
+  messages: [{ role: "user", content: "What is the weather in Berlin?" }],
+  tools: [
+    {
+      toolType: ToolType.Function,
+      function: {
+        name: "get_weather",
+        description: "Get the current weather for a location",
+        parameters: {
+          type: "object",
+          properties: { location: { type: "string" } },
+          required: ["location"],
+        },
+      },
+    },
+  ],
+});
+
+for (const call of response.choices[0]?.message?.toolCalls ?? []) {
+  console.log(`Tool: ${call.function.name}, Args: ${call.function.arguments}`);
+}
+```
+
+### Next Steps
+
+- **[Provider Registry](https://github.com/xberg-io/liter-llm/blob/main/schemas/providers.json)** - Full list of supported providers
+- **[GitHub Repository](https://github.com/xberg-io/liter-llm)** - Source, issues, and discussions
+
+## NAPI-RS Implementation Details
+
+### Native Performance
+
+This binding uses NAPI-RS to provide native Node.js bindings with:
+
+- **Zero-copy data transfer** between JavaScript and Rust layers
+- **Async by default** — all LLM calls return Promises backed by Tokio
+- **Binary-compatible** pre-built native modules across platforms
+- **TypeScript definitions** generated automatically from Rust types
+
+### Threading Model
+
+- LLM calls are non-blocking — Tokio async runtime handles concurrency
+- Streaming responses use Node.js async iterators backed by Tokio streams
+- CPU-bound work runs in `spawn_blocking` to avoid blocking the event loop
+
+### Memory Management
+
+- API keys are wrapped in `secrecy::SecretString` and never logged
+- Streaming buffers are released as soon as each chunk is consumed
+- Provider registry is compiled into the binary — no runtime disk access
+
+## Features
+
+### Supported Providers (143)
+
+Route to any provider using the `provider/model` prefix convention:
+
+| Provider           | Example Model                                                 |
+| ------------------ | ------------------------------------------------------------- |
+| **OpenAI**         | `openai/gpt-4o`, `openai/gpt-4o-mini`                         |
+| **Anthropic**      | `anthropic/claude-3-5-sonnet-20241022`                        |
+| **Groq**           | `groq/llama-3.1-70b-versatile`                                |
+| **Mistral**        | `mistral/mistral-large-latest`                                |
+| **Cohere**         | `cohere/command-r-plus`                                       |
+| **Together AI**    | `together/meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo`       |
+| **Fireworks**      | `fireworks/accounts/fireworks/models/llama-v3p1-70b-instruct` |
+| **Google Vertex**  | `vertexai/gemini-1.5-pro`                                     |
+| **Amazon Bedrock** | `bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0`           |
+
+**[Complete Provider List](https://github.com/xberg-io/liter-llm/blob/main/schemas/providers.json)**
+
+### Key Capabilities
+
+- **Provider Routing** -- Single client for 143 LLM providers via `provider/model` prefix
+- **Local LLMs** — Connect to locally-hosted models via Ollama, LM Studio, vLLM, llama.cpp, and other local inference servers
+- **Unified API** -- Consistent `chat`, `chat_stream`, `embeddings`, `list_models` interface
+- **Streaming** -- Real-time token streaming via `chat_stream`
+- **Tool Calling** -- Function calling and tool use across all supporting providers
+- **Type Safe** -- Schema-driven types compiled from JSON schemas
+- **Secure** -- API keys never logged or serialized, managed via environment variables
+- **Observability** -- Built-in [OpenTelemetry](https://opentelemetry.io/docs/specs/semconv/gen-ai/) with GenAI semantic conventions
+- **Error Handling** -- Structured errors with provider context and retry hints
+
+### Performance
+
+Built on a compiled Rust core for speed and safety:
+
+- **Provider resolution** at client construction -- zero per-request overhead
+- **Configurable timeouts** and connection pooling
+- **Zero-copy streaming** with SSE and AWS EventStream support
+- **API keys** wrapped in secure memory, zeroed on drop
+
+## Provider Routing
+
+Route to 143 providers using the `provider/model` prefix convention:
+
+```text
+openai/gpt-4o
+anthropic/claude-3-5-sonnet-20241022
+groq/llama-3.1-70b-versatile
+mistral/mistral-large-latest
+```
+
+See the [provider registry](https://github.com/xberg-io/liter-llm/blob/main/schemas/providers.json) for the full list.
+
+## Proxy, MCP Server & Plugin
+
+<details>
+<summary><strong>Run the OpenAI-compatible proxy or the MCP server</strong></summary>
+
+Beyond the SDK, the `liter-llm` CLI ships an OpenAI-compatible proxy and a Model Context Protocol (MCP) server:
+
+```bash
+brew install xberg-io/tap/liter-llm   # or: cargo install liter-llm-cli
+liter-llm api --config liter-llm-proxy.toml   # OpenAI-compatible proxy
+liter-llm mcp --transport stdio               # MCP tool server
+
+# or run the proxy without installing:
+docker run -p 4000:4000 -e LITER_LLM_MASTER_KEY=sk-your-key ghcr.io/xberg-io/liter-llm
+```
+
+To use the MCP server inside a coding agent, install the **liter-llm plugin** from the [`xberg-io/plugins`](https://github.com/xberg-io/plugins) marketplace — it auto-registers the server. See the [MCP server](https://docs.liter-llm.xberg.io/server/mcp-server/) and [proxy server](https://docs.liter-llm.xberg.io/server/proxy-server/) guides for configuration, CLI usage, and agent integration.
+
+</details>
+
+## Documentation
+
+- **[Documentation](https://docs.liter-llm.xberg.io)** -- Full docs and API reference
+- **[GitHub Repository](https://github.com/xberg-io/liter-llm)** -- Source, issues, and discussions
+- **[Provider Registry](https://github.com/xberg-io/liter-llm/blob/main/schemas/providers.json)** -- 143 supported providers
+
+## Part of Xberg.io
+
+- [Xberg](https://github.com/xberg-io/xberg) — document intelligence: text, tables, metadata from 91+ formats with optional OCR.
+- [Xberg Enterprise](https://github.com/xberg-io/xberg-enterprise) — managed extraction API with SDKs, dashboards, and observability.
+- [crawlberg](https://github.com/xberg-io/crawlberg) — web crawling and scraping with HTML→Markdown and headless-Chrome fallback.
+- [html-to-markdown](https://github.com/xberg-io/html-to-markdown) — fast, lossless HTML→Markdown engine.
+- [liter-llm](https://github.com/xberg-io/liter-llm) — universal LLM API client with native bindings for 14 languages and 143 providers.
+- [tree-sitter-language-pack](https://github.com/xberg-io/tree-sitter-language-pack) — tree-sitter grammars and code-intelligence primitives.
+- [alef](https://github.com/xberg-io/alef) — the polyglot binding generator that produces every per-language binding across the 5 polyglot repos.
+- [Discord](https://discord.gg/xt9WY3GnKR) — community, roadmap, announcements.
+
+## Contributing
+
+Contributions are welcome! See [CONTRIBUTING.md](https://github.com/xberg-io/liter-llm/blob/main/CONTRIBUTING.md) for guidelines.
+
+Join our [Discord community](https://discord.gg/xt9WY3GnKR) for questions and discussion.
+
+## License
+
+MIT -- see [LICENSE](https://github.com/xberg-io/liter-llm/blob/main/LICENSE) for details.