@xberg-io/opencode-liter-llm 2.2.3 → 2.3.0

package/.pi/skills/liter-llm/SKILL.md

@@ -0,0 +1,208 @@
+---
+description: Universal LLM API client for 174 providers with native bindings for 14 languages. Use when writing code that calls LLM APIs via liter-llm in Python, TypeScript, Rust, Go, Java, C#, Ruby, PHP, Elixir, WASM, or C, when running the OpenAI-compatible proxy, or when calling LLMs through the MCP server. Covers chat, streaming, tool calling, embeddings, image generation, speech, transcription, moderation, web search, OCR, reranking, provider routing, middleware, and configuration.
+license: MIT
+metadata:
+    author: xberg-io
+    version: "0.1.0"
+    repository: https://github.com/xberg-io/liter-llm
+name: liter-llm
+---
+
+<!--
+AI-RULEZ :: GENERATED FILE — DO NOT EDIT
+Content-Hash: blake3:d6c306e983d2305d8037cb8f4e973ff21eeb0e3ed4a5ac342c4c7dbfae965836
+Source-Hash: blake3:88c87867f5079f6c8fcb582aee913cca4f6feefe541c5361d2307a9b45233f97
+Schema-Version: v1
+-->
+
+# Liter-LLM Universal LLM Client
+
+Liter-LLM is a universal LLM API client with a Rust core and native bindings for
+Python, TypeScript/Node.js, Go, Java, C#, Ruby, PHP, Elixir, WebAssembly, and C
+(FFI). One unified interface reaches 174 providers (OpenAI, Anthropic, Google
+Gemini, Groq, Mistral, Cohere, AWS Bedrock, Azure, and many more).
+
+## Capability map
+
+- **Modalities** — chat completions, streaming, tool/function calling,
+  structured outputs, embeddings, image generation, audio (speech,
+  transcription), moderation, web search, OCR, reranking, batch,
+  and file operations.
+- **Provider routing** — `provider/model` prefix selects the backend
+  (`openai/gpt-4o`, `anthropic/claude-sonnet-4-20250514`,
+  `gemini/gemini-2.0-flash`). Set a full `provider/model` as `model_hint` to choose the client provider.
+- **Middleware** — response caching, rate limiting (RPM/TPM), cost tracking,
+  budget enforcement (`hard` / `soft`), fallback chains, circuit-breaker
+  cooldown, and background health checks.
+- **Search providers (12)** and **OCR providers (4)** — first-class `search`
+  and `ocr` methods routed by the same prefix convention.
+- **Proxy server** — `liter-llm api` exposes an OpenAI-compatible gateway with
+  22 REST endpoints, virtual API keys, budgets, and SSE streaming.
+- **MCP server** — `liter-llm mcp` exposes 22 tools mirroring the proxy
+  endpoints, for MCP-compatible clients (Claude Code, Claude Desktop).
+- **14 language bindings** — same surface, language-native naming (snake_case
+  for Python/Rust/Ruby/Go/Elixir/PHP, camelCase for TS/Node/WASM/C#/Java).
+
+## When to use the MCP server vs the SDK vs the proxy
+
+| Use the … | When you want to … | Entry point |
+|-----------|--------------------|-------------|
+| **MCP server** | Let the agent call LLM APIs directly as tools, with no glue code | `liter-llm mcp --transport stdio` (this plugin auto-registers it) |
+| **SDK / binding** | Write application code that calls LLMs in a specific language | `pip install liter-llm`, `cargo add liter-llm`, etc. |
+| **Proxy** | Give many apps/teams a shared OpenAI-compatible endpoint with keys, budgets, and rate limits | `liter-llm api --config liter-llm-proxy.toml` |
+
+Rule of thumb: reach for the MCP server inside an agent session; the SDK when
+building software; the proxy when centralizing access for multiple consumers.
+
+## Installation
+
+### CLI (proxy + MCP server)
+
+```bash
+# Homebrew (macOS / Linux)
+brew install xberg-io/tap/liter-llm
+
+# or run it without a persistent install (the CLI proxy package self-installs the binary)
+pnpm dlx @xberg-io/liter-llm-cli@2.2.3 --help
+
+# or download a prebuilt binary from the latest GitHub release:
+#   https://github.com/xberg-io/liter-llm/releases/latest
+
+# or build from source
+cargo install liter-llm-cli
+
+# or Docker (35MB image)
+docker pull ghcr.io/xberg-io/liter-llm
+```
+
+### Language bindings
+
+| Language | Install |
+|----------|---------|
+| Python | `pip install liter-llm` |
+| Node.js | `pnpm add @xberg-io/liter-llm` |
+| Rust | `cargo add liter-llm` |
+| Go | `go get github.com/xberg-io/liter-llm/packages/go/v2` |
+| Ruby | `gem install liter_llm` |
+| PHP | `composer require xberg-io/liter-llm` |
+| C# | `dotnet add package XbergIo.LiterLlm` |
+| WASM | `pnpm add @xberg-io/liter-llm-wasm` |
+
+## Quick start (Python, async)
+
+```python
+import asyncio
+import os
+from liter_llm import create_client
+from liter_llm._internal_bindings import ChatCompletionRequest
+
+async def main() -> None:
+    client = create_client(api_key=os.environ["OPENAI_API_KEY"])
+    request = ChatCompletionRequest.from_json(
+        '{"model":"openai/gpt-4o","messages":[{"role":"user","content":"Hello!"}]}'
+    )
+    response = await client.chat(request)
+    print(response.choices[0].message.content)
+
+asyncio.run(main())
+```
+
+## Provider routing
+
+The prefix before `/` in the request's `model` selects the provider:
+
+```python
+ChatCompletionRequest.from_json('{"model":"openai/gpt-4o","messages":[...]}')
+ChatCompletionRequest.from_json('{"model":"anthropic/claude-sonnet-4-20250514","messages":[...]}')
+ChatCompletionRequest.from_json('{"model":"gemini/gemini-2.0-flash","messages":[...]}')
+ChatCompletionRequest.from_json('{"model":"groq/llama3-70b","messages":[...]}')
+```
+
+Client construction resolves credentials for its selected provider from environment variables: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`,
+`GEMINI_API_KEY`, `GROQ_API_KEY`, `MISTRAL_API_KEY`, `COHERE_API_KEY`, and AWS
+credentials for Bedrock. Create a separate client for each provider with matching
+`model_hint` and credentials. Request prefixes change routing but reuse the client
+API key; they do not select a new environment key per request.
+
+## Configuration
+
+Pass basic options to `create_client(...)` (`api_key`, `base_url`,
+`timeout_secs`, `max_retries`, `model_hint`), or set the full surface in a
+`liter-llm.toml` (SDK) / `liter-llm-proxy.toml` (proxy) — both auto-discover
+from the cwd upward.
+
+```toml
+# liter-llm.toml
+api_key = "${OPENAI_API_KEY}"
+model_hint = "openai/gpt-4o"
+timeout_secs = 120
+max_retries = 5
+
+[cache]
+max_entries = 512
+ttl_seconds = 600
+
+[budget]
+global_limit = 50.0
+enforcement = "hard"
+
+[rate_limit]
+rpm = 60
+tpm = 100000
+```
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `api_key` | required | Provider key, wrapped in `SecretString` (never logged). |
+| `base_url` | from registry | Override the provider base URL. |
+| `model_hint` | none | Full provider/model used to select the client provider and credentials. |
+| `timeout_secs` | 60 | Request timeout in seconds. |
+| `max_retries` | 3 | Retries on 429/5xx with exponential backoff. |
+| `cache` | none | `max_entries`, `ttl_seconds`. |
+| `budget` | none | `global_limit`, `model_limits`, `enforcement`. |
+| `rate_limit` | none | `rpm`, `tpm`. |
+| `cost_tracking` | false | Per-request cost tracking. |
+| `tracing` | false | OpenTelemetry spans. |
+
+## Proxy server
+
+```bash
+liter-llm api --config liter-llm-proxy.toml
+```
+
+22 OpenAI-compatible endpoints, model routing by prefix, virtual API keys,
+per-key RPM/TPM limits, cost tracking, budget enforcement, response caching, SSE
+streaming, and an OpenAPI 3.1 spec at `/openapi.json`.
+
+## MCP server
+
+```bash
+liter-llm mcp --transport stdio   # for Claude Code / Claude Desktop
+liter-llm mcp --transport http --port 3001
+```
+
+This plugin auto-registers the stdio server via `scripts/mcp-launch.sh`. The 22
+tools mirror the proxy endpoints: `chat`, `embed`, `generate_image`, `speech`,
+`transcribe`, `moderate`, `rerank`, `search`, `ocr`, `list_models`; file ops
+(`create_file`, `list_files`, `retrieve_file`, `delete_file`, `file_content`);
+batch ops (`create_batch`, `list_batches`, `retrieve_batch`, `cancel_batch`);
+and Responses API (`create_response`, `retrieve_response`, `cancel_response`).
+
+## Common pitfalls
+
+1. **Provider routing** — use `"provider/model"` to select a request backend. Bare
+   model names fall back to the configured client provider (OpenAI by default).
+2. **API keys are SecretString** — never logged or serialized. Read from env
+   vars; never hardcode.
+3. **Python methods are async** — `await` inside an async context.
+4. **Naming conventions differ** — camelCase for TS/Node/WASM/C#/Java,
+   snake_case elsewhere.
+5. **Streaming chunks may have null content** — null-check
+   `chunk.choices[0].delta.content` before use.
+6. **Budget modes** — `hard` rejects over-budget requests; `soft` logs and
+   allows.
+
+## Additional resources
+
+- Upstream docs: <https://docs.liter-llm.xberg.io>
+- GitHub: <https://github.com/xberg-io/liter-llm>

package/.pi/skills/liter-llm/index.md

@@ -0,0 +1,10 @@
+<!--
+AI-RULEZ :: GENERATED FILE — DO NOT EDIT
+Content-Hash: blake3:60957d0aa8465f2855ee8ec0ad10f311a31f8a64822760eb1bfbe7d1a559255b
+Source-Hash: blake3:88c87867f5079f6c8fcb582aee913cca4f6feefe541c5361d2307a9b45233f97
+Schema-Version: v1
+-->
+
+# Concepts
+
+* [Liter Llm](SKILL.md) - Universal LLM API client for 174 providers with native bindings for 14 languages. Use when writing code that calls LLM APIs via liter-llm in Python, TypeScript, Rust, Go, Java, C#, Ruby, PHP, Elixir, WASM, or C, when running the OpenAI-compatible proxy, or when calling LLMs through the MCP server. Covers chat, streaming, tool calling, embeddings, image generation, speech, transcription, moderation, web search, OCR, reranking, provider routing, middleware, and configuration.

package/.pi/skills/liter-llm-calling-llms/SKILL.md

@@ -0,0 +1,75 @@
+---
+description: Use when sending chat completions through liter-llm and routing to a specific provider via the `provider/model` prefix. Covers the chat call shape, provider routing, model_hint, message roles, and error categories.
+name: liter-llm-calling-llms
+---
+
+<!--
+AI-RULEZ :: GENERATED FILE — DO NOT EDIT
+Content-Hash: blake3:d45c8ad7f731f0e0bc91646e677fd7c89b6af31e38944bff4302e7e242a12906
+Source-Hash: blake3:88c87867f5079f6c8fcb582aee913cca4f6feefe541c5361d2307a9b45233f97
+Schema-Version: v1
+-->
+
+# Calling LLMs
+
+Build a `ChatCompletionRequest` and send it with `client.chat(request)`. Create
+the client with `create_client(...)`. The model string is `provider/model`; the
+prefix selects the backend.
+
+```python
+import asyncio, json, os
+from liter_llm import create_client
+from liter_llm._internal_bindings import ChatCompletionRequest
+
+async def main() -> None:
+    client = create_client(api_key=os.environ["OPENAI_API_KEY"])
+    request = ChatCompletionRequest.from_json(json.dumps({
+        "model": "openai/gpt-4o",
+        "messages": [
+            {"role": "system", "content": "You are concise."},
+            {"role": "user", "content": "Name three Rust crates for HTTP."},
+        ],
+    }))
+    response = await client.chat(request)
+    print(response.choices[0].message.content)
+
+asyncio.run(main())
+```
+
+## Provider routing
+
+The model string's prefix selects the provider; build a request per backend:
+
+```python
+ChatCompletionRequest.from_json('{"model":"anthropic/claude-sonnet-4-20250514","messages":[...]}')
+ChatCompletionRequest.from_json('{"model":"gemini/gemini-2.0-flash","messages":[...]}')
+ChatCompletionRequest.from_json('{"model":"groq/llama3-70b","messages":[...]}')
+ChatCompletionRequest.from_json('{"model":"mistral/mistral-large-latest","messages":[...]}')
+ChatCompletionRequest.from_json('{"model":"bedrock/anthropic.claude-v2","messages":[...]}')
+```
+
+Set a full `provider/model` as `model_hint` at construction to select the client
+provider and credential source. Requests without a recognized prefix fall back to
+that client provider:
+
+```python
+client = create_client(api_key=os.environ["OPENAI_API_KEY"], model_hint="openai/gpt-4o")
+# the request model can now omit the provider prefix:
+request = ChatCompletionRequest.from_json('{"model":"gpt-4o","messages":[...]}')
+await client.chat(request)  # routes to OpenAI
+```
+
+## Notes
+
+- Keys come from env vars (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, …); never
+  hardcode them.
+- Without a recognized prefix, requests fall back to the client's configured provider;
+  a client without `model_hint` defaults to OpenAI.
+- Environment credentials are selected when constructing the client, not independently
+  for every request. Use a separate client with matching `model_hint` and credentials
+  for each provider. A prefix changes routing but does not switch the client's API key.
+- Python errors are typed exceptions exported from `liter_llm`:
+  `AuthenticationError`, `RateLimitedError`, `BadRequestError`,
+  `ContextWindowExceededError`, `ContentPolicyError`, `NotFoundError`,
+  `ServerError`, `ServiceUnavailableError`, `LiterLlmTimeoutError`,
+  `BudgetExceededError` — all subclasses of `LiterLlmError`.

package/.pi/skills/liter-llm-calling-llms/index.md

@@ -0,0 +1,10 @@
+<!--
+AI-RULEZ :: GENERATED FILE — DO NOT EDIT
+Content-Hash: blake3:9f87cef1b6478cba89e995bc6d58c203c63ad6668e495f5cda9aeca30caf20f0
+Source-Hash: blake3:88c87867f5079f6c8fcb582aee913cca4f6feefe541c5361d2307a9b45233f97
+Schema-Version: v1
+-->
+
+# Concepts
+
+* [Calling Llms](SKILL.md) - Use when sending chat completions through liter-llm and routing to a specific provider via the `provider/model` prefix. Covers the chat call shape, provider routing, model_hint, message roles, and error categories.

package/.pi/skills/liter-llm-embeddings-and-search/SKILL.md

@@ -0,0 +1,81 @@
+---
+description: Use when generating embeddings, calling the 12 web-search providers, or running OCR over documents with the 4 OCR providers through liter-llm. Covers embed, search, and ocr methods plus reranking.
+name: liter-llm-embeddings-and-search
+---
+
+<!--
+AI-RULEZ :: GENERATED FILE — DO NOT EDIT
+Content-Hash: blake3:b5eb4ea6ac25a0cf5bda7f68424924e9760e43055ad34b314f5030588a6c3fc8
+Source-Hash: blake3:88c87867f5079f6c8fcb582aee913cca4f6feefe541c5361d2307a9b45233f97
+Schema-Version: v1
+-->
+
+# Embeddings and Search
+
+liter-llm exposes embeddings, web search (12 providers), OCR (4 providers), and
+reranking through the same `provider/model` routing convention.
+
+## Embeddings
+
+```python
+import asyncio, os
+from liter_llm import create_client
+from liter_llm._internal_bindings import EmbeddingRequest
+
+async def main() -> None:
+    client = create_client(api_key=os.environ["OPENAI_API_KEY"])
+    request = EmbeddingRequest.from_json(
+        '{"model":"openai/text-embedding-3-small","input":["first document","second document"]}'
+    )
+    response = await client.embed(request)
+    for item in response.data:
+        print(len(item.embedding))
+
+asyncio.run(main())
+```
+
+Many embedding models support dimension selection and base64 output; set
+`dimensions` / `encoding_format` in the request where the provider allows it.
+
+## Web search (12 providers)
+
+```python
+from liter_llm._internal_bindings import SearchRequest
+
+client = create_client(api_key=os.environ["BRAVE_API_KEY"])
+request = SearchRequest.from_json(
+    '{"model":"brave/web-search","query":"What is the Rust programming language?","max_results":5}'
+)
+response = await client.search(request)
+for result in response.results:
+    print(result.title, result.url)
+```
+
+## OCR (4 providers)
+
+```python
+from liter_llm._internal_bindings import OcrRequest
+
+client = create_client(api_key=os.environ["MISTRAL_API_KEY"])
+request = OcrRequest.from_json(
+    '{"model":"mistral/mistral-ocr-latest",'
+    '"document":{"type":"document_url","url":"https://example.com/invoice.pdf"}}'
+)
+response = await client.ocr(request)
+for page in response.pages:
+    print(page.index, page.markdown[:100])
+```
+
+## Reranking
+
+Build a `RerankRequest` (model, query, documents) and call `client.rerank(request)`
+to score and order candidate documents against a query for retrieval pipelines —
+combine it with `embed` for hybrid retrieval. Each result carries `index` and
+`relevance_score`. Routing follows the same `provider/model` convention.
+
+## Notes
+
+- Search and OCR providers each need their own API key (e.g. `BRAVE_API_KEY`,
+  `MISTRAL_API_KEY`); read them from env vars.
+- See the upstream provider reference for the full list of the 12 search and 4
+  OCR backends and their model identifiers.

package/.pi/skills/liter-llm-embeddings-and-search/index.md

@@ -0,0 +1,10 @@
+<!--
+AI-RULEZ :: GENERATED FILE — DO NOT EDIT
+Content-Hash: blake3:06bd68d2dd91f5a85234a7fa75f4e8f90c5bed83ff6200cf601fc12814c38ddb
+Source-Hash: blake3:88c87867f5079f6c8fcb582aee913cca4f6feefe541c5361d2307a9b45233f97
+Schema-Version: v1
+-->
+
+# Concepts
+
+* [Embeddings And Search](SKILL.md) - Use when generating embeddings, calling the 12 web-search providers, or running OCR over documents with the 4 OCR providers through liter-llm. Covers embed, search, and ocr methods plus reranking.

package/.pi/skills/liter-llm-running-the-proxy/SKILL.md

@@ -0,0 +1,74 @@
+---
+description: Use when running the `liter-llm api` OpenAI-compatible gateway — virtual keys, per-key rate limits, budgets, cost tracking, and model routing. Covers the TOML config and the 22 REST endpoints.
+name: liter-llm-running-the-proxy
+---
+
+<!--
+AI-RULEZ :: GENERATED FILE — DO NOT EDIT
+Content-Hash: blake3:b0a0178eae0e6882ae032983d8927142b6db16add6ab9cab391538db2f202eb4
+Source-Hash: blake3:88c87867f5079f6c8fcb582aee913cca4f6feefe541c5361d2307a9b45233f97
+Schema-Version: v1
+-->
+
+# Running the Proxy
+
+`liter-llm api` is a drop-in OpenAI-compatible gateway: 22 REST endpoints that
+route to 174 providers, with multi-tenant virtual keys, rate limits, budgets,
+and cost tracking.
+
+## Start it
+
+```bash
+liter-llm api --config liter-llm-proxy.toml
+```
+
+The proxy auto-discovers `liter-llm-proxy.toml` in the current directory.
+
+## Configuration
+
+```toml
+[server]
+host = "0.0.0.0"
+port = 4000
+
+[general]
+master_key = "${LITER_LLM_MASTER_KEY}"
+
+# Each [[models]] entry maps a routable name to a provider/model and its key.
+[[models]]
+name = "gpt-4o"
+provider_model = "openai/gpt-4o"
+api_key = "${OPENAI_API_KEY}"
+
+[[models]]
+name = "claude-sonnet"
+provider_model = "anthropic/claude-sonnet-4-20250514"
+api_key = "${ANTHROPIC_API_KEY}"
+
+# Virtual keys scope which configured model names a caller may use.
+[[keys]]
+key = "sk-team-frontend"
+models = ["gpt-4o", "claude-sonnet"]
+rpm = 60
+tpm = 100000
+budget_limit = 50.0
+```
+
+`${ENV_VAR}` interpolation keeps secrets out of the file.
+
+## Call it like OpenAI
+
+```bash
+curl http://localhost:4000/v1/chat/completions \
+  -H "Authorization: Bearer sk-team-frontend" \
+  -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}]}'
+```
+
+## Notes
+
+- Virtual keys scope which models a caller may use and carry their own RPM/TPM
+  and budget limits; the `master_key` administers them.
+- The OpenAPI 3.1 spec is served at `/openapi.json`.
+- Endpoints cover chat, embeddings, images, audio, moderations, files, batches,
+  responses, and model listing.
+- Docker: `docker run -p 4000:4000 -e LITER_LLM_MASTER_KEY=sk-key ghcr.io/xberg-io/liter-llm`.

package/.pi/skills/liter-llm-running-the-proxy/index.md

@@ -0,0 +1,10 @@
+<!--
+AI-RULEZ :: GENERATED FILE — DO NOT EDIT
+Content-Hash: blake3:157fa991ddf536a4cd3c9413a6bf2f44b25762efcb64ad3f7732731e4cb2f14e
+Source-Hash: blake3:88c87867f5079f6c8fcb582aee913cca4f6feefe541c5361d2307a9b45233f97
+Schema-Version: v1
+-->
+
+# Concepts
+
+* [Running The Proxy](SKILL.md) - Use when running the `liter-llm api` OpenAI-compatible gateway — virtual keys, per-key rate limits, budgets, cost tracking, and model routing. Covers the TOML config and the 22 REST endpoints.

package/.pi/skills/liter-llm-streaming-responses/SKILL.md

@@ -0,0 +1,63 @@
+---
+description: Use when streaming tokens incrementally from an LLM via liter-llm over SSE or async iterators. Covers chat_stream, delta handling, and null-content chunks.
+name: liter-llm-streaming-responses
+---
+
+<!--
+AI-RULEZ :: GENERATED FILE — DO NOT EDIT
+Content-Hash: blake3:6de558b3f33f35e9e7ef76dc64d4ebd199a8a831dc28e306698f18fc16933b4e
+Source-Hash: blake3:88c87867f5079f6c8fcb582aee913cca4f6feefe541c5361d2307a9b45233f97
+Schema-Version: v1
+-->
+
+# Streaming Responses
+
+Use `chat_stream(...)` to receive tokens as they are produced instead of waiting
+for the full completion. The proxy streams over SSE; bindings expose async
+iterators.
+
+## Python
+
+```python
+import asyncio, os
+from liter_llm import create_client
+from liter_llm._internal_bindings import ChatCompletionRequest
+
+async def main() -> None:
+    client = create_client(api_key=os.environ["OPENAI_API_KEY"])
+    request = ChatCompletionRequest.from_json(
+        '{"model":"openai/gpt-4o","messages":[{"role":"user","content":"Tell me a story"}],"stream":true}'
+    )
+    async for chunk in client.chat_stream(request):
+        if chunk.choices and chunk.choices[0].delta.content:
+            print(chunk.choices[0].delta.content, end="", flush=True)
+    print()
+
+asyncio.run(main())
+```
+
+## TypeScript
+
+```typescript
+import { createClient } from "@xberg-io/liter-llm";
+
+const client = createClient(process.env.OPENAI_API_KEY!);
+const chunks = await client.chatStream({
+  model: "openai/gpt-4o",
+  messages: [{ role: "user", content: "Tell me a story" }],
+});
+for await (const chunk of chunks) {
+  process.stdout.write(chunk.choices?.[0]?.delta?.content ?? "");
+}
+```
+
+## Notes
+
+- The first and last chunks often carry null content. Always null-check
+  `chunk.choices[0].delta.content` (Python) or
+  `chunk.choices[0]?.delta?.content` (TypeScript) before using it.
+- Tool-call deltas arrive in `delta.tool_calls` (Python) /
+  `delta.toolCalls` (TypeScript); accumulate `function.arguments` fragments
+  across chunks before parsing.
+- Through the proxy, request streaming with `"stream": true` on
+  `/v1/chat/completions`.

package/.pi/skills/liter-llm-streaming-responses/index.md

@@ -0,0 +1,10 @@
+<!--
+AI-RULEZ :: GENERATED FILE — DO NOT EDIT
+Content-Hash: blake3:614ded1eb401cb2a27434f2b901aa4cab62f6be4964c0882783e21592ebb347e
+Source-Hash: blake3:88c87867f5079f6c8fcb582aee913cca4f6feefe541c5361d2307a9b45233f97
+Schema-Version: v1
+-->
+
+# Concepts
+
+* [Streaming Responses](SKILL.md) - Use when streaming tokens incrementally from an LLM via liter-llm over SSE or async iterators. Covers chat_stream, delta handling, and null-content chunks.

package/.pi/skills/liter-llm-tool-calling/SKILL.md

@@ -0,0 +1,77 @@
+---
+description: Use when defining functions/tools for an LLM to call through liter-llm, or requesting structured JSON outputs. Covers tool schemas, tool_calls handling, and response formats.
+name: liter-llm-tool-calling
+---
+
+<!--
+AI-RULEZ :: GENERATED FILE — DO NOT EDIT
+Content-Hash: blake3:f26049693a8fa3229f675eefc4097cfa0aeb669a5744967a7ab3614009a80310
+Source-Hash: blake3:88c87867f5079f6c8fcb582aee913cca4f6feefe541c5361d2307a9b45233f97
+Schema-Version: v1
+-->
+
+# Tool Calling
+
+Pass a `tools` array of function definitions; the model may respond with
+`tool_calls` instead of (or alongside) text. Execute the named function and feed
+the result back as a `tool` message.
+
+## Python
+
+```python
+import asyncio, json, os
+from liter_llm import create_client
+from liter_llm._internal_bindings import ChatCompletionRequest
+
+payload = {
+    "model": "openai/gpt-4o",
+    "messages": [{"role": "user", "content": "What is the weather in Berlin?"}],
+    "tools": [
+        {
+            "type": "function",
+            "function": {
+                "name": "get_weather",
+                "description": "Get the current weather for a location",
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        "location": {"type": "string", "description": "City name"},
+                    },
+                    "required": ["location"],
+                },
+            },
+        }
+    ],
+    "tool_choice": "auto",
+}
+
+async def main() -> None:
+    client = create_client(api_key=os.environ["OPENAI_API_KEY"])
+    request = ChatCompletionRequest.from_json(json.dumps(payload))
+    response = await client.chat(request)
+    for call in response.choices[0].message.tool_calls or []:
+        print(call.function.name, call.function.arguments)  # arguments is a JSON string
+
+asyncio.run(main())
+```
+
+## Structured outputs
+
+Request strict JSON with `response_format`:
+
+```python
+request = ChatCompletionRequest.from_json(json.dumps({
+    "model": "openai/gpt-4o",
+    "messages": [{"role": "user", "content": "Extract name and age as JSON."}],
+    "response_format": {"type": "json_object"},
+}))
+response = await client.chat(request)
+```
+
+## Notes
+
+- `function.arguments` is a JSON **string** — parse it before use.
+- Append each tool result as a message with `role="tool"` and the matching
+  `tool_call_id`, then call `chat` again to let the model continue.
+- Tool support and JSON-mode availability vary by provider; check the provider
+  reference if a model ignores `tools`.

package/.pi/skills/liter-llm-tool-calling/index.md

@@ -0,0 +1,10 @@
+<!--
+AI-RULEZ :: GENERATED FILE — DO NOT EDIT
+Content-Hash: blake3:27b7d413268994b2de57be9cce2eba3277c235244cf4dd3d8018746f2b98eeed
+Source-Hash: blake3:88c87867f5079f6c8fcb582aee913cca4f6feefe541c5361d2307a9b45233f97
+Schema-Version: v1
+-->
+
+# Concepts
+
+* [Tool Calling](SKILL.md) - Use when defining functions/tools for an LLM to call through liter-llm, or requesting structured JSON outputs. Covers tool schemas, tool_calls handling, and response formats.