llmshim 0.3.0__tar.gz → 0.3.2__tar.gz

{llmshim-0.3.0 → llmshim-0.3.2}/CLAUDE.md

@@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 ## What is llmshim
 
-A pure Rust LLM API translation layer. Takes OpenAI-format JSON requests, translates them to provider-native formats (and back), with zero infrastructure requirements. Supports OpenAI (Responses API), Anthropic, Google Gemini, xAI, and OpenRouter (an OpenAI Chat Completions-compatible aggregator). Includes an interactive CLI chat with streaming, reasoning, and mid-conversation model switching.
+A pure Rust LLM API translation layer. Takes OpenAI-format JSON requests, translates them to provider-native formats (and back), with zero infrastructure requirements. Supports OpenAI (Responses API), Anthropic, Google Gemini, xAI, OpenRouter (an OpenAI Chat Completions-compatible aggregator), and self-hosted **vLLM** / **SGLang** servers (OpenAI Chat Completions-compatible, local or remote). Includes an interactive CLI chat with streaming, reasoning, and mid-conversation model switching.
 
 **Published on crates.io as `llmshim`** — https://crates.io/crates/llmshim
 
@@ -15,8 +15,9 @@ This is a public crate on crates.io. Do NOT make breaking changes to `pub` items
 - **OpenAI:** `gpt-5.6-sol`, `gpt-5.6-terra`, `gpt-5.6-luna`, `gpt-5.5`, `gpt-5.5-pro`, `gpt-5.4`, `gpt-5.4-pro`, `gpt-5.4-mini`, `gpt-5.4-nano`
 - **Anthropic:** `claude-opus-5`, `claude-opus-4-8`, `claude-sonnet-5`, `claude-opus-4-7`, `claude-opus-4-6`, `claude-sonnet-4-6`, `claude-haiku-4-5-20251001`
 - **Gemini:** `gemini-3.5-flash`, `gemini-3.1-pro-preview`, `gemini-3-flash-preview`
-- **xAI:** `grok-4.5`, `grok-4.3`, `grok-4.20-multi-agent-beta-0309`, `grok-4.20-beta-0309-reasoning`, `grok-4.20-beta-0309-non-reasoning`
+- **xAI:** `grok-4.6`, `grok-4.5`, `grok-4.3`, `grok-4.20-multi-agent-beta-0309`, `grok-4.20-beta-0309-reasoning`, `grok-4.20-beta-0309-non-reasoning`
 - **OpenRouter:** not enumerated (huge/dynamic catalog) — any `openrouter/<vendor>/<model>` slug routes through, e.g. `openrouter/anthropic/claude-sonnet-4.5`.
+- **vLLM / SGLang:** not enumerated (self-hosted) — any `vllm/<served-model>` or `sglang/<served-model>` routes through to the configured server, e.g. `sglang/Qwen/Qwen3.6-35B-A3B-FP8`.
 
 ## Build & Test
 
@@ -31,13 +32,15 @@ cargo run                                            # interactive CLI chat
 cargo run --features proxy -- proxy                  # proxy server on :3000
 ```
 
-API keys: `~/.llmshim/config.toml` (via `llmshim configure`) or env vars `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, `XAI_API_KEY`, `OPENROUTER_API_KEY`. Precedence: env vars > config file.
+API keys: `~/.llmshim/config.toml` (via `llmshim configure`) or env vars `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, `XAI_API_KEY`, `OPENROUTER_API_KEY`. Precedence: env vars > config file. Self-hosted servers are configured by **base URL** instead of a key: `VLLM_BASE_URL` / `SGLANG_BASE_URL` (each with an optional `VLLM_API_KEY` / `SGLANG_API_KEY`); the provider registers only when its base URL is set. Local vs remote is just the URL value.
 
 ## Architecture
 
 ### Value-based transforms, no canonical struct
 
-Requests flow as `serde_json::Value`. Each provider's transform takes raw JSON and maps only what it understands. Provider-specific features use `x-anthropic`, `x-gemini`, `x-openrouter` namespaces.
+Requests flow as `serde_json::Value`. Each provider's transform takes raw JSON and maps only what it understands. Provider-specific features use `x-anthropic`, `x-gemini`, `x-openrouter`, `x-vllm`, `x-sglang` namespaces.
+
+**Self-hosted passthrough providers (vLLM / SGLang).** `src/providers/openai_compat.rs` is one generic OpenAI Chat Completions passthrough backing both `vllm` and `sglang` (`OpenAiCompatible::new(name, base_url, api_key: Option)`, registered per env base URL). Two things differ from the hosted providers: the **base URL is configuration** (local `http://localhost:8000/v1` vs remote `https://host/v1`), and **auth is optional** (self-hosted servers are unauthenticated unless launched with `--api-key`, so the `Authorization` header is sent only when a key is set). Passthrough transforms; `reasoning`/`reasoning_content` normalized to `reasoning_content` (vLLM is migrating the field name); `reasoning_effort` forwarded as-is (honored per-model, not clamped); server-specific params go under `x-<name>` (`chat_template_kwargs`, `separate_reasoning`, `guided_json`, `top_k`, …). Note: reasoning/tool parsing are **launch-time server flags** (`--reasoning-parser`, `--tool-call-parser`), so a request only gets that behavior if the server was started for it — llmshim can't enable it per request.
 
 **OpenRouter is the one passthrough provider.** Every other provider translates the OpenAI-format input *away* to a native dialect; OpenRouter (`src/providers/openrouter.rs`) *is* OpenAI Chat Completions, so its transforms are near-identity — messages, tools, vision (`image_url`), and `response_format` are forwarded unchanged; `reasoning_effort` maps 1:1 to OpenRouter's `reasoning:{effort}` (its effort vocabulary is a superset, so no clamping); `message.reasoning` is normalized to `reasoning_content` on responses. OpenRouter models are **not enumerated** in `src/models.rs` (the catalog is huge and dynamic) — any `openrouter/<vendor>/<model>` slug routes through. `x-openrouter` carries OpenRouter-only controls (`provider`, `models`, `transforms`, `route`, native `reasoning`; plus `http_referer`/`x_title` which become headers). The `middle-out` transform is disabled by default for faithful passthrough. Uses `image_url` (Chat Completions) vision via `vision::to_openai_chat`.
 
@@ -57,7 +60,7 @@ Every provider implements: `transform_request`, `transform_response`, `transform
 
 ### Router (`src/router.rs`)
 
-Parses `"provider/model"` strings by splitting on the **first** `/` only, so an OpenRouter slug's internal slash survives (`openrouter/anthropic/claude-sonnet-4.5` → provider `openrouter`, model `anthropic/claude-sonnet-4.5`). Auto-infers provider from prefix (`gpt*`/`o*` → openai, `claude*` → anthropic, `gemini*` → gemini, `grok*` → xai); **OpenRouter has no prefix inference** — its slugs collide with everyone's, so it must be addressed explicitly as `openrouter/…`. Supports aliases. `Router::from_env()` reads API key env vars.
+Parses `"provider/model"` strings by splitting on the **first** `/` only, so an OpenRouter slug's internal slash survives (`openrouter/anthropic/claude-sonnet-4.5` → provider `openrouter`, model `anthropic/claude-sonnet-4.5`). Auto-infers provider from prefix (`gpt*`/`o*` → openai, `claude*` → anthropic, `gemini*` → gemini, `grok*` → xai); **OpenRouter, vLLM, and SGLang have no prefix inference** — their slugs collide with everyone's, so address them explicitly (`openrouter/…`, `vllm/…`, `sglang/…`); the first-slash split also preserves HF-style served-model slugs (`vllm/meta-llama/Llama-3.1-8B-Instruct`). Supports aliases. `Router::from_env()` reads API-key env vars, plus `VLLM_BASE_URL` / `SGLANG_BASE_URL` (+ optional `*_API_KEY`) for the self-hosted providers.
 
 ### HTTP Client (`src/client.rs`)
 
@@ -94,6 +97,7 @@ llmshim accepts tools in OpenAI Chat Completions format (nested `function` objec
 - **Anthropic:** Tools translated to `{"name": ..., "description": ..., "input_schema": ...}` format. Tool results translated to Anthropic's `tool_result` content blocks.
 - **xAI:** Same flat format as OpenAI Responses API — `translate_tools()` flattens nested format.
 - **OpenRouter:** No translation — it accepts the Chat Completions nested `{"type":"function","function":{…}}` format directly, so `tools`/`tool_choice`/`tool_calls` pass through unchanged.
+- **vLLM / SGLang:** Same as OpenRouter — Chat Completions nested tool format passes through unchanged (the server must be launched with `--tool-call-parser` / `--enable-auto-tool-choice` for tool calls to be parsed).
 - **Gemini:** Tools wrapped in `functionDeclarations`. Tool results translated to `functionResponse` format.
 
 ### CLI (`src/main.rs`)

{llmshim-0.3.0 → llmshim-0.3.2}/Cargo.lock

@@ -958,7 +958,7 @@ checksum = "6373607a59f0be73a39b6fe456b8192fcc3585f602af20751600e974dd455e77"
 
 [[package]]
 name = "llmshim"
-version = "0.3.0"
+version = "0.3.2"
 dependencies = [
  "async-stream",
  "async-trait",

{llmshim-0.3.0 → llmshim-0.3.2}/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "llmshim"
-version = "0.3.0"
+version = "0.3.2"
 edition = "2021"
 description = "Blazing fast LLM API translation layer in pure Rust"
 license = "MIT OR Apache-2.0"

{llmshim-0.3.0 → llmshim-0.3.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: llmshim
-Version: 0.3.0
+Version: 0.3.2
 Classifier: Development Status :: 4 - Beta
 Classifier: Intended Audience :: Developers
 Classifier: License :: OSI Approved :: MIT License
@@ -223,7 +223,7 @@ billed provider calls; run it only when you deliberately want to hit real APIs.
 | OpenAI | `gpt-5.5`, `gpt-5.5-pro`, `gpt-5.4`, `gpt-5.4-pro`, `gpt-5.4-mini`, `gpt-5.4-nano` |
 | Anthropic | `claude-opus-5`, `claude-opus-4-8`, `claude-sonnet-5`, `claude-opus-4-7`, `claude-opus-4-6`, `claude-sonnet-4-6`, `claude-haiku-4-5-20251001` |
 | Gemini | `gemini-3.5-flash`, `gemini-3.1-pro-preview`, `gemini-3-flash-preview` |
-| xAI | `grok-4.5`, `grok-4.3`, `grok-4.20-multi-agent-beta-0309`, `grok-4.20-beta-0309-reasoning`, `grok-4.20-beta-0309-non-reasoning` |
+| xAI | `grok-4.6`, `grok-4.5`, `grok-4.3`, `grok-4.20-multi-agent-beta-0309`, `grok-4.20-beta-0309-reasoning`, `grok-4.20-beta-0309-non-reasoning` |
 
 Call `llmshim.models()` for the live list filtered to your configured providers.
 

{llmshim-0.3.0 → llmshim-0.3.2}/README.md

@@ -1,6 +1,6 @@
 # llmshim
 
-A blazing-fast LLM API translation layer written in **pure Rust**. One request format, every provider — OpenAI, Anthropic, Google Gemini, xAI, and OpenRouter.
+A blazing-fast LLM API translation layer written in **pure Rust**. One request format, every provider — OpenAI, Anthropic, Google Gemini, xAI, OpenRouter, and self-hosted vLLM / SGLang.
 
 Send an OpenAI-style request, pick any model, and llmshim translates it to that provider's native API (and translates the response back). Switch providers by changing one string.
 
@@ -56,6 +56,18 @@ OpenRouter is OpenAI Chat Completions-compatible, so tools, vision, streaming,
 and `reasoning_effort` all pass through; OpenRouter-only controls (provider
 routing, model fallbacks, transforms) go under an `x-openrouter` key.
 
+Point at a **self-hosted vLLM or SGLang** server (local or remote) by setting its
+base URL — no key needed unless the server was launched with one:
+
+```bash
+export SGLANG_BASE_URL=http://localhost:30000/v1   # or https://your-host/v1
+export VLLM_BASE_URL=http://localhost:8000/v1
+```
+
+Then address the served model as `sglang/<served-model>` or `vllm/<served-model>`
+(e.g. `sglang/Qwen/Qwen3.6-35B-A3B-FP8`). Server-specific knobs go under
+`x-vllm` / `x-sglang`.
+
 Or persist them to the config file (used by all three surfaces):
 
 ```bash
@@ -353,7 +365,7 @@ Standard library only. Full docs: [`clients/ruby/README.md`](clients/ruby/README
 | **OpenAI** | `gpt-5.6-sol`, `gpt-5.6-terra`, `gpt-5.6-luna`, `gpt-5.5`, `gpt-5.5-pro`, `gpt-5.4`, `gpt-5.4-pro`, `gpt-5.4-mini`, `gpt-5.4-nano` | Yes (summaries) |
 | **Anthropic** | `claude-opus-5`, `claude-opus-4-8`, `claude-sonnet-5`, `claude-opus-4-7`, `claude-opus-4-6`, `claude-sonnet-4-6`, `claude-haiku-4-5-20251001` | Yes (full thinking) |
 | **Google Gemini** | `gemini-3.5-flash`, `gemini-3.1-pro-preview`, `gemini-3-flash-preview` | Yes (thought summaries) |
-| **xAI** | `grok-4.5`, `grok-4.3`, `grok-4.20-multi-agent-beta-0309`, `grok-4.20-beta-0309-reasoning`, `grok-4.20-beta-0309-non-reasoning` | No (hidden) |
+| **xAI** | `grok-4.6`, `grok-4.5`, `grok-4.3`, `grok-4.20-multi-agent-beta-0309`, `grok-4.20-beta-0309-reasoning`, `grok-4.20-beta-0309-non-reasoning` | No (hidden) |
 
 Use a bare model name (auto-detected by prefix) or an explicit `provider/model` string.
 

{llmshim-0.3.0 → llmshim-0.3.2}/docs/src/guides/reasoning.md

@@ -134,7 +134,7 @@ through `x-gemini.thinkingConfig`.
 
 xAI receives the nested native shape `reasoning: {effort}`:
 
-| unified | grok-4.3 | grok-4.5 | grok-4.20-\*-reasoning / -non-reasoning |
+| unified | grok-4.3 | grok-4.5 / grok-4.6 | grok-4.20-\*-reasoning / -non-reasoning |
 |---|---|---|---|
 | `none` | `none` | **`low`** | omitted |
 | `low` | `low` | `low` | omitted |
@@ -143,7 +143,7 @@ xAI receives the nested native shape `reasoning: {effort}`:
 | `xhigh` | `xhigh` | `xhigh` | omitted |
 | `max` | **`xhigh`** | **`xhigh`** | omitted |
 
-grok-4.5 cannot disable reasoning, so `none` clamps to `low`. grok-4.20 models
+grok-4.5 and grok-4.6 cannot disable reasoning, so `none` clamps to `low`. grok-4.20 models
 are name-locked: reasoning on or off is encoded in the model name, and the API
 rejects any reasoning parameter. llmshim therefore omits it for that family.
 

{llmshim-0.3.0 → llmshim-0.3.2}/docs/src/reference/configuration.md

@@ -13,6 +13,8 @@ set. Therefore **environment variables take precedence over the file**.
 | Google Gemini | `GEMINI_API_KEY` | `keys.gemini` |
 | xAI | `XAI_API_KEY` | `keys.xai` |
 | OpenRouter | `OPENROUTER_API_KEY` | `keys.openrouter` |
+| vLLM (self-hosted) | `VLLM_BASE_URL` (+ optional `VLLM_API_KEY`) | — (env only) |
+| SGLang (self-hosted) | `SGLANG_BASE_URL` (+ optional `SGLANG_API_KEY`) | — (env only) |
 
 The config file shape is:
 

{llmshim-0.3.0 → llmshim-0.3.2}/docs/src/reference/models.md

@@ -56,6 +56,7 @@ this table in applications.
 
 | ID | Display name |
 |---|---|
+| `xai/grok-4.6` | Grok 4.6 |
 | `xai/grok-4.5` | Grok 4.5 |
 | `xai/grok-4.3` | Grok 4.3 |
 | `xai/grok-4.20-multi-agent-beta-0309` | Grok 4.20 Multi-Agent |
@@ -126,6 +127,8 @@ the slug's internal slash and `:variant` suffix are preserved. Because its
 slugs collide with other providers' prefixes, OpenRouter has no bare-model
 inference — always address it explicitly as `openrouter/…`.
 
+**Self-hosted vLLM / SGLang** are likewise not enumerated. Set `VLLM_BASE_URL` or `SGLANG_BASE_URL` (with an optional `*_API_KEY`) and address the served model as `vllm/<served-model>` or `sglang/<served-model>` — e.g. `sglang/Qwen/Qwen3.6-35B-A3B-FP8`. Local vs remote is just the base-URL value.
+
 Rust applications can also define one-level Router aliases with
 `Router::alias`. Those aliases are not part of the static registry and are not
 configured by the stock CLI or proxy. See [Models and the Router](../concepts/routing.md).

{llmshim-0.3.0 → llmshim-0.3.2}/docs/src/reference/providers.md

@@ -10,6 +10,8 @@ different native API and translates only the fields that API understands.
 | Google Gemini | `generateContent` / `streamGenerateContent` | `gemini*` | `x-gemini` |
 | xAI | Responses API | `grok*` | none |
 | OpenRouter | Chat Completions (aggregator) | none — address as `openrouter/<vendor>/<model>` | `x-openrouter` |
+| vLLM | Chat Completions (self-hosted, `VLLM_BASE_URL`) | none — address as `vllm/<served-model>` | `x-vllm` |
+| SGLang | Chat Completions (self-hosted, `SGLANG_BASE_URL`) | none — address as `sglang/<served-model>` | `x-sglang` |
 
 An explicit address such as `anthropic/claude-sonnet-5` avoids inference.
 The named provider must be registered in the Router—that normally means its
@@ -24,6 +26,7 @@ API key is configured.
 | Gemini | Messages become `contents`; tools use `functionDeclarations`, `functionCall`, and `functionResponse` | Base64 images become `inline_data`, but a remote image URL becomes a text placeholder because Gemini cannot consume it directly; `x-gemini.thinkingConfig` replaces mapped thinking configuration |
 | xAI | System/developer text becomes Responses `instructions`; tools are flattened like OpenAI Responses | Unified reasoning becomes `reasoning: {effort}` where the model accepts it; grok-4.20 reasoning is encoded in the model name; there is no `x-xai` namespace |
 | OpenRouter | Passthrough — messages, tools, `image_url` vision, and `response_format` are already Chat Completions and forwarded unchanged | `reasoning_effort` maps 1:1 to OpenRouter's `reasoning:{effort}` (superset vocabulary, no clamping); `message.reasoning` is normalized to `reasoning_content`; the `middle-out` transform is disabled by default; `x-openrouter` carries `provider`/`models`/`transforms`/`route`/native `reasoning` (and `http_referer`/`x_title` headers) |
+| vLLM / SGLang | Passthrough to a self-hosted server — configured by base URL (local or remote), auth optional | `reasoning`/`reasoning_content` normalized to `reasoning_content`; `reasoning_effort` forwarded (honored per-model); server-specific params (`chat_template_kwargs`, `guided_json`, `top_k`, `separate_reasoning`, …) go under `x-vllm`/`x-sglang`. Reasoning/tool parsing depend on the server's launch flags (`--reasoning-parser`, `--tool-call-parser`) |
 
 OpenAI and xAI do not receive `temperature`, `top_p`, `top_k`, or `stop` from
 the portable top-level request. Anthropic and Gemini do. This is the

{llmshim-0.3.0 → llmshim-0.3.2}/docs/src/reference/request-fields.md

@@ -30,6 +30,7 @@ renamed or reshaped for the selected provider; unsupported controls are omitted.
 | `x-anthropic` | object | Native passthrough/control | Anthropic body fields plus `extra_betas` and `disable_1m_context` controls |
 | `x-gemini` | object | Native passthrough | Gemini body fields; `thinkingConfig` goes under `generationConfig` |
 | `x-openrouter` | object | Native passthrough/control | OpenRouter body fields (`provider`, `models`, `transforms`, `route`, native `reasoning`); `http_referer`/`x_title` become request headers |
+| `x-vllm` / `x-sglang` | object | Native passthrough | Server-specific body params for a self-hosted vLLM/SGLang server (`chat_template_kwargs`, `separate_reasoning`, `guided_json`, `top_k`, `min_p`, …) |
 
 There is no `x-xai` namespace. OpenAI additionally recognizes `store`,
 `prompt_cache_key`, `prompt_cache_retention`, `safety_identifier`, and `speed`.

{llmshim-0.3.0 → llmshim-0.3.2}/src/main.rs

@@ -25,6 +25,7 @@ const MODELS: &[(&str, &str)] = &[
     ("gemini/gemini-3.5-flash", "Gemini 3.5 Flash"),
     ("gemini/gemini-3.1-pro-preview", "Gemini 3.1 Pro"),
     ("gemini/gemini-3-flash-preview", "Gemini 3 Flash"),
+    ("xai/grok-4.6", "Grok 4.6"),
     ("xai/grok-4.5", "Grok 4.5"),
     ("xai/grok-4.3", "Grok 4.3"),
     (

{llmshim-0.3.0 → llmshim-0.3.2}/src/models.rs

@@ -352,6 +352,15 @@ pub const MODELS: &[ModelInfo] = &[
         max_output_tokens: Some(65_536),
         capabilities: CAPS_STD,
     },
+    ModelInfo {
+        id: "xai/grok-4.6",
+        provider: "xai",
+        name: "grok-4.6",
+        label: "Grok 4.6",
+        context_window_tokens: Some(500_000),
+        max_output_tokens: None,
+        capabilities: CAPS_XAI,
+    },
     ModelInfo {
         id: "xai/grok-4.5",
         provider: "xai",

{llmshim-0.3.0 → llmshim-0.3.2}/src/providers/mod.rs

@@ -1,5 +1,6 @@
 pub mod anthropic;
 pub mod gemini;
 pub mod openai;
+pub mod openai_compat;
 pub mod openrouter;
 pub mod xai;

llmshim-0.3.2/src/providers/openai_compat.rs

@@ -0,0 +1,196 @@
+use crate::error::{Result, ShimError};
+use crate::provider::{Provider, ProviderRequest};
+use crate::vision;
+use serde_json::{json, Value};
+
+/// A generic OpenAI Chat Completions-compatible provider for **self-hosted**
+/// inference servers — vLLM and SGLang. Like OpenRouter it's a passthrough
+/// (messages, tools, `image_url` vision, and `response_format` are already in
+/// the target shape), but two things differ from a hosted aggregator:
+///
+/// - **The base URL is configuration**, not a constant — that's what "local vs
+///   remote" means (`http://localhost:8000/v1` vs `https://host/v1`).
+/// - **Auth is optional** — these servers accept unauthenticated requests unless
+///   launched with `--api-key`, so the `Authorization` header is sent only when
+///   a key is configured.
+///
+/// `name` (e.g. `"vllm"` / `"sglang"`) is both the provider key and the
+/// extension namespace: server-specific params (`chat_template_kwargs`,
+/// `separate_reasoning`, `guided_json`, `top_k`, …) go under `x-<name>`.
+pub struct OpenAiCompatible {
+    pub name: String,
+    pub base_url: String,
+    pub api_key: Option<String>,
+}
+
+impl OpenAiCompatible {
+    pub fn new(
+        name: impl Into<String>,
+        base_url: impl Into<String>,
+        api_key: Option<String>,
+    ) -> Self {
+        Self {
+            name: name.into(),
+            base_url: base_url.into(),
+            api_key,
+        }
+    }
+}
+
+/// Strip llmshim-normalized / foreign-provider fields and normalize content
+/// blocks to Chat Completions form. Messages, `tool_calls`, and `role: "tool"`
+/// stay in Chat Completions shape (the target format).
+fn sanitize_messages(messages: &[Value]) -> Vec<Value> {
+    messages
+        .iter()
+        .map(|msg| {
+            let mut out = msg.clone();
+            if let Some(obj) = out.as_object_mut() {
+                obj.remove("reasoning_content"); // regenerated server-side; don't echo back
+                obj.remove("reasoning_signature"); // opaque Anthropic token — never forward
+                obj.remove("redacted_reasoning_content"); // opaque Anthropic token — never forward
+                obj.remove("annotations");
+                obj.remove("refusal");
+            }
+            if let Some(content) = out.get("content").cloned() {
+                if content.is_array() {
+                    let translated =
+                        vision::translate_content_blocks(&content, vision::to_openai_chat);
+                    out["content"] = vision::text_blocks_to_chat(&translated);
+                }
+            }
+            out
+        })
+        .collect()
+}
+
+/// Copy OpenRouter/vLLM/SGLang's `reasoning` field into llmshim's
+/// `reasoning_content` convention if the latter isn't already present. (vLLM is
+/// migrating `reasoning_content` → `reasoning`; SGLang uses `reasoning_content`.)
+fn normalize_reasoning(obj: &mut serde_json::Map<String, Value>) {
+    if obj.contains_key("reasoning_content") {
+        return;
+    }
+    if let Some(r) = obj
+        .get("reasoning")
+        .and_then(|r| r.as_str())
+        .filter(|s| !s.is_empty())
+        .map(str::to_string)
+    {
+        obj.insert("reasoning_content".to_string(), json!(r));
+    }
+}
+
+impl Provider for OpenAiCompatible {
+    fn name(&self) -> &str {
+        &self.name
+    }
+
+    fn transform_request(&self, model: &str, request: &Value) -> Result<ProviderRequest> {
+        let obj = request.as_object().ok_or(ShimError::MissingModel)?;
+        let messages = obj
+            .get("messages")
+            .and_then(|m| m.as_array())
+            .ok_or(ShimError::MissingModel)?;
+
+        let mut body = json!({
+            "model": model,
+            "messages": sanitize_messages(messages),
+        });
+        let body_obj = body.as_object_mut().unwrap();
+
+        // Standard Chat Completions params (plus reasoning_effort, which vLLM and
+        // some SGLang models honor natively) — forwarded unchanged.
+        for key in [
+            "max_tokens",
+            "max_completion_tokens",
+            "temperature",
+            "top_p",
+            "frequency_penalty",
+            "presence_penalty",
+            "stop",
+            "seed",
+            "stream",
+            "stream_options",
+            "tools",
+            "tool_choice",
+            "parallel_tool_calls",
+            "response_format",
+            "logprobs",
+            "top_logprobs",
+            "n",
+            "reasoning_effort",
+        ] {
+            if let Some(v) = obj.get(key) {
+                body_obj.insert(key.to_string(), v.clone());
+            }
+        }
+
+        // x-<name> namespace: server-specific params (sampling knobs like top_k /
+        // min_p, guided_json / regex / ebnf, chat_template_kwargs,
+        // separate_reasoning, …) are copied straight into the body.
+        let ns = format!("x-{}", self.name);
+        if let Some(ext) = obj.get(&ns).and_then(|e| e.as_object()) {
+            for (k, v) in ext {
+                body_obj.insert(k.clone(), v.clone());
+            }
+        }
+
+        let mut headers = vec![("Content-Type".to_string(), "application/json".to_string())];
+        // Auth is optional — self-hosted servers are unauthenticated unless
+        // launched with --api-key.
+        if let Some(key) = &self.api_key {
+            if !key.is_empty() {
+                headers.push(("Authorization".to_string(), format!("Bearer {key}")));
+            }
+        }
+
+        let url = format!("{}/chat/completions", self.base_url.trim_end_matches('/'));
+        Ok(ProviderRequest { url, headers, body })
+    }
+
+    fn transform_response(&self, _model: &str, mut response: Value) -> Result<Value> {
+        if let Some(err) = response.get("error") {
+            if !err.is_null() {
+                let message = err
+                    .get("message")
+                    .and_then(|m| m.as_str())
+                    .unwrap_or("unknown error")
+                    .to_string();
+                let status = err.get("code").and_then(|c| c.as_u64()).unwrap_or(400) as u16;
+                return Err(ShimError::ProviderError {
+                    status,
+                    body: message,
+                });
+            }
+        }
+
+        // Already Chat Completions-shaped. Normalize the reasoning field name.
+        if let Some(choices) = response.get_mut("choices").and_then(|c| c.as_array_mut()) {
+            for choice in choices {
+                if let Some(msg) = choice.get_mut("message").and_then(|m| m.as_object_mut()) {
+                    normalize_reasoning(msg);
+                }
+            }
+        }
+
+        Ok(response)
+    }
+
+    fn transform_stream_chunk(&self, _model: &str, chunk: &str) -> Result<Option<String>> {
+        let mut parsed: Value = match serde_json::from_str(chunk) {
+            Ok(v) => v,
+            Err(_) => return Ok(None),
+        };
+
+        if let Some(choices) = parsed.get_mut("choices").and_then(|c| c.as_array_mut()) {
+            for choice in choices {
+                if let Some(delta) = choice.get_mut("delta").and_then(|d| d.as_object_mut()) {
+                    normalize_reasoning(delta);
+                }
+            }
+        }
+
+        Ok(Some(serde_json::to_string(&parsed)?))
+    }
+}

{llmshim-0.3.0 → llmshim-0.3.2}/src/providers/xai.rs

@@ -189,11 +189,14 @@ fn is_reasoning_name_locked(model: &str) -> bool {
     model.to_lowercase().contains("4.20")
 }
 
-/// grok-4.5 cannot disable reasoning: `effort: "none"` -> 400 ("does not
-/// support `reasoning_effort` value `none`", verified live). Unified effort
-/// "none" clamps to "low". grok-4.3 and the fast pair DO accept "none".
+/// grok-4.5 / grok-4.6 cannot disable reasoning: `effort: "none"` -> 400 ("does
+/// not support `reasoning_effort` value `none`", verified live). Unified effort
+/// "none" clamps to "low". grok-4.3 DOES accept "none".
 fn reasoning_cannot_disable(model: &str) -> bool {
-    model.to_lowercase().contains("4.5")
+    let m = model.to_lowercase();
+    // grok-4.5 and grok-4.6 both 400 on `reasoning_effort: "none"` (verified
+    // live); unified "none" clamps to "low" for them. grok-4.3 accepts "none".
+    m.contains("4.5") || m.contains("4.6")
 }
 
 impl Provider for Xai {

{llmshim-0.3.0 → llmshim-0.3.2}/src/router.rs

@@ -3,6 +3,7 @@ use crate::provider::Provider;
 use crate::providers::anthropic::Anthropic;
 use crate::providers::gemini::Gemini;
 use crate::providers::openai::OpenAi;
+use crate::providers::openai_compat::OpenAiCompatible;
 use crate::providers::openrouter::OpenRouter;
 use crate::providers::xai::Xai;
 use std::collections::HashMap;
@@ -97,6 +98,22 @@ impl Router {
         if let Ok(key) = std::env::var("OPENROUTER_API_KEY") {
             router = router.register("openrouter", Box::new(OpenRouter::new(key)));
         }
+        // Self-hosted OpenAI-compatible servers: the base URL is the config
+        // (local vs remote); the API key is optional. Registered only when the
+        // base URL is set. Address as `vllm/<served-model>` / `sglang/<served-model>`.
+        if let Ok(base) = std::env::var("VLLM_BASE_URL") {
+            let key = std::env::var("VLLM_API_KEY").ok().filter(|k| !k.is_empty());
+            router = router.register("vllm", Box::new(OpenAiCompatible::new("vllm", base, key)));
+        }
+        if let Ok(base) = std::env::var("SGLANG_BASE_URL") {
+            let key = std::env::var("SGLANG_API_KEY")
+                .ok()
+                .filter(|k| !k.is_empty());
+            router = router.register(
+                "sglang",
+                Box::new(OpenAiCompatible::new("sglang", base, key)),
+            );
+        }
 
         router
     }

{llmshim-0.3.0 → llmshim-0.3.2}/src/vision.rs

@@ -257,3 +257,27 @@ pub fn text_blocks_to_openai(content: &Value) -> Value {
         _ => content.clone(),
     }
 }
+
+/// Translate text blocks to OpenAI Chat Completions format ("text" instead of
+/// the Responses API's "input_text") — for OpenAI-compatible targets like
+/// OpenRouter, vLLM, and SGLang.
+pub fn text_blocks_to_chat(content: &Value) -> Value {
+    match content {
+        Value::Array(blocks) => {
+            let translated: Vec<Value> = blocks
+                .iter()
+                .map(|block| {
+                    if block.get("type").and_then(|t| t.as_str()) == Some("input_text") {
+                        let mut out = block.clone();
+                        out["type"] = Value::String("text".into());
+                        out
+                    } else {
+                        block.clone()
+                    }
+                })
+                .collect();
+            Value::Array(translated)
+        }
+        _ => content.clone(),
+    }
+}

llmshim-0.3.2/tests/integration_sglang.rs

@@ -0,0 +1,99 @@
+/// Integration tests for a self-hosted SGLang (or any OpenAI-compatible) server.
+/// Point SGLANG_BASE_URL at a running server and run:
+///   SGLANG_BASE_URL=http://host:30000/v1 SGLANG_TEST_MODEL=<served-name> \
+///     cargo test --test integration_sglang -- --ignored --nocapture
+/// SGLANG_API_KEY is optional (only if the server was launched with --api-key).
+use serde_json::json;
+
+fn router() -> llmshim::router::Router {
+    llmshim::router::Router::from_env()
+}
+
+fn model() -> Option<String> {
+    let m = std::env::var("SGLANG_TEST_MODEL").ok()?;
+    Some(format!("sglang/{m}"))
+}
+
+#[tokio::test]
+#[ignore]
+async fn sglang_basic_completion() {
+    if std::env::var("SGLANG_BASE_URL").is_err() {
+        return;
+    }
+    let Some(model) = model() else {
+        eprintln!("set SGLANG_TEST_MODEL");
+        return;
+    };
+    let router = router();
+    let req = json!({
+        "model": model,
+        "messages": [{"role": "user", "content": "In one short sentence, what is Rust?"}],
+        "max_tokens": 512,
+    });
+    let resp = llmshim::completion(&router, &req).await.unwrap();
+    assert_eq!(resp["object"], "chat.completion");
+    let content = resp["choices"][0]["message"]["content"]
+        .as_str()
+        .unwrap_or("");
+    assert!(!content.is_empty(), "expected a response, got: {resp}");
+    println!("model={} | said: {content}", resp["model"]);
+}
+
+#[tokio::test]
+#[ignore]
+async fn sglang_reasoning() {
+    if std::env::var("SGLANG_BASE_URL").is_err() {
+        return;
+    }
+    let Some(model) = model() else { return };
+    let router = router();
+    let req = json!({
+        "model": model,
+        "messages": [{"role": "user", "content": "What is 17 * 24? Reason step by step, then give the answer."}],
+        "max_tokens": 2000,
+        "reasoning_effort": "high",
+    });
+    let resp = llmshim::completion(&router, &req).await.unwrap();
+    let msg = &resp["choices"][0]["message"];
+    let reasoning = msg["reasoning_content"].as_str().unwrap_or("");
+    let content = msg["content"].as_str().unwrap_or("");
+    assert!(
+        !reasoning.is_empty() || !content.is_empty(),
+        "expected reasoning or answer, got: {resp}"
+    );
+    println!(
+        "reasoning chars={}, answer chars={}",
+        reasoning.len(),
+        content.len()
+    );
+}
+
+#[tokio::test]
+#[ignore]
+async fn sglang_tool_call() {
+    if std::env::var("SGLANG_BASE_URL").is_err() {
+        return;
+    }
+    let Some(model) = model() else { return };
+    let router = router();
+    let req = json!({
+        "model": model,
+        "messages": [{"role": "user", "content": "What's the weather in Tokyo? Use the get_weather tool."}],
+        "max_tokens": 512,
+        "tools": [{
+            "type": "function",
+            "function": {
+                "name": "get_weather",
+                "description": "Get the weather for a city",
+                "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}
+            }
+        }]
+    });
+    let resp = llmshim::completion(&router, &req).await.unwrap();
+    let tool_calls = resp["choices"][0]["message"].get("tool_calls");
+    assert!(tool_calls.is_some(), "expected a tool call, got: {resp}");
+    println!(
+        "tool call ok: {}",
+        tool_calls.unwrap()[0]["function"]["name"]
+    );
+}