llmshim 0.3.3 → 0.3.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 373860a58de51c825431e5190b6799d910233273012847b371855bbbce04c432
-  data.tar.gz: 4a6289b522e14e4cdc94d55666e8bb379ef04924f48dccd4e3f1d6c0ae27aa07
+  metadata.gz: dc7d1f66fec1e50c72a93b0470ef6df88d0b61be7fbc3828972280cda625caa1
+  data.tar.gz: 614251f70adf8b39377307c04698fa09db451266006e590628e18b8c8a4fad9c
 SHA512:
-  metadata.gz: e2a5c5ce9b3b27ca8d95511a9acf903c2c00cacc329e5bb765d1479f27f907fa421c7b8f31cd3e355f7e7e3a69708b6b7c88ecf778cca5cfa9e2d8c340fa4f75
-  data.tar.gz: 8ed01b6ffc12eaf18059a0264c0943bb0959721bf1fdb5793cb8fe3473de88d077b453329170684e0aacef7f0e5a4f4ec3c24bc4c1ccf51eb9aab6d6525aaa9d
+  metadata.gz: 68b3fcc22b3371f110a6c5dcd123bbedb4e673345fd9ec7ff4b919eb374dffbec070cc55913fa82cf8ab9d4afbe7d445f99a926700465147c521057f5689039d
+  data.tar.gz: e0bf20f098bf17496665d105abd69d79bff92513dc11141c64fadef4a753e4f9a57fc9ac41e9347a81490eeaa48b54bfe520160ead394c4559969b5104b0def1

data/README.md

@@ -2,7 +2,8 @@
 
 A thin, dependency-free Ruby client for the [llmshim](https://github.com/sanjay920/llmshim)
 multi-provider LLM proxy. Send OpenAI-style chat requests to a running llmshim
-proxy and let it translate to OpenAI, Anthropic, Google Gemini, or xAI.
+proxy and let it translate to OpenAI, Anthropic, Google Gemini, xAI, OpenRouter,
+vLLM, or SGLang.
 
 This gem talks to a proxy over plain HTTP — it does **not** spawn the Rust
 binary. Start the proxy separately (`llmshim proxy`, default `http://localhost:3000`).
@@ -38,10 +39,22 @@ gem install ./llmshim-*.gem
 llmshim proxy            # listens on 0.0.0.0:3000 by default
 ```
 
-Configure provider API keys for the proxy via `llmshim configure` or the
-`OPENAI_API_KEY` / `ANTHROPIC_API_KEY` / `GEMINI_API_KEY` / `XAI_API_KEY`
+Configure provider credentials for the proxy via `llmshim configure` or
 environment variables. The Ruby client never sees your keys — the proxy holds them.
 
+| Provider   | Model string form                       | Env vars                              |
+| ---------- | --------------------------------------- | ------------------------------------- |
+| OpenAI     | `openai/gpt-5.6-sol`                    | `OPENAI_API_KEY`                      |
+| Anthropic  | `anthropic/claude-sonnet-5`            | `ANTHROPIC_API_KEY`                   |
+| Gemini     | `gemini/gemini-3.5-flash`              | `GEMINI_API_KEY`                      |
+| xAI        | `xai/grok-4.5`                         | `XAI_API_KEY`                         |
+| OpenRouter | `openrouter/anthropic/claude-sonnet-4.5` | `OPENROUTER_API_KEY`                |
+| vLLM       | `vllm/<served-model>`                  | `VLLM_BASE_URL` (+ optional `VLLM_API_KEY`)   |
+| SGLang     | `sglang/<served-model>`                | `SGLANG_BASE_URL` (+ optional `SGLANG_API_KEY`) |
+
+vLLM and SGLang are self-hosted (local or remote) OpenAI-compatible servers;
+point the proxy at them with `VLLM_BASE_URL` / `SGLANG_BASE_URL`.
+
 ## Quickstart
 
 ```ruby
@@ -49,10 +62,12 @@ require "llmshim"
 
 client = Llmshim::Client.new(base_url: "http://localhost:3000")
 
-resp = client.chat(model: "claude-sonnet-4-6", messages: "What is Rust?")
+resp = client.chat(model: "anthropic/claude-sonnet-5", messages: "What is Rust?")
 puts resp.content                 # => "Rust is a systems programming language..."
 puts resp.provider                # => "anthropic"
+puts resp.latency_ms              # => 1234 (round-trip latency in ms)
 puts resp.usage.total_tokens
+puts resp.usage.reasoning_tokens  # thinking tokens billed (nil if none)
 ```
 
 `messages:` accepts a single string (treated as one user message) or an array
@@ -71,6 +86,12 @@ resp = client.chat(
 )
 ```
 
+### Streaming vs. non-streaming
+
+`chat` is non-streaming and returns a single `Llmshim::ChatResponse`. Passing
+`stream: true` to `chat` raises `ArgumentError` — the proxy would emit SSE that
+`chat` cannot parse. Use `stream` (below) for token-by-token output.
+
 ### Module-level convenience
 
 A shared default client (base URL from `LLMSHIM_BASE_URL`, else `http://localhost:3000`):
@@ -89,7 +110,7 @@ puts resp.content
 `done`, `error`.
 
 ```ruby
-client.stream(model: "claude-sonnet-4-6", messages: "Write a haiku") do |event|
+client.stream(model: "anthropic/claude-sonnet-5", messages: "Write a haiku") do |event|
   case event.type
   when "reasoning" then print event.text   # thinking tokens
   when "content"   then print event.text   # answer tokens
@@ -114,7 +135,7 @@ text = events.select(&:content?).map(&:text).join
 
 ```ruby
 resp = client.chat(
-  model: "anthropic/claude-sonnet-4-6",
+  model: "anthropic/claude-sonnet-5",
   messages: "What's the weather in SF?",
   tools: [
     { type: "function",
@@ -123,10 +144,10 @@ resp = client.chat(
                                 properties: { city: { type: "string" } } } } }
   ],
   tool_choice: "auto",
-  # Raw provider-specific JSON merged into the underlying request:
-  provider_config: { thinking: { type: "adaptive" } },
+  # Provider-specific controls, namespaced under x-<provider> (see below):
+  provider_config: { "x-anthropic" => { thinking: { type: "enabled", budget_tokens: 4000 } } },
   # Try these models if the primary fails with a retryable error:
-  fallback: ["openai/gpt-5.5", "gemini/gemini-3-flash-preview"]
+  fallback: ["openai/gpt-5.6-sol", "gemini/gemini-3.5-flash"]
 )
 
 resp.message.tool_calls.each do |tc|
@@ -136,7 +157,63 @@ end
 
 `tools` and `tool_choice` are folded into `provider_config` (passed straight
 through to the provider). `max_tokens`, `temperature`, `top_p`, `top_k`,
-`stop`, and `reasoning_effort` are folded into the request `config`.
+`stop`, `reasoning_effort`, and `reasoning_mode` are folded into the request
+`config`.
+
+### Provider passthrough (`provider_config`)
+
+`provider_config` merges at the **request root**, so native provider controls
+must be namespaced under an `x-<provider>` key — a bare `thinking:` at the top
+level is ignored. Use the namespace matching the target provider:
+
+| Provider   | Namespace       |
+| ---------- | --------------- |
+| OpenAI     | `x-openai`      |
+| Anthropic  | `x-anthropic`   |
+| Gemini     | `x-gemini`      |
+| OpenRouter | `x-openrouter`  |
+| vLLM       | `x-vllm`        |
+| SGLang     | `x-sglang`      |
+
+```ruby
+# Anthropic extended thinking:
+provider_config: { "x-anthropic" => { thinking: { type: "enabled", budget_tokens: 4000 } } }
+
+# OpenRouter provider routing (also accepts `models`, `transforms`):
+provider_config: { "x-openrouter" => { provider: { sort: "throughput" } } }
+```
+
+`provider_config` also carries a few root-level keys the proxy understands
+directly (not namespaced):
+
+- `tools` / `tool_choice` — folded in for you from the `tools:`/`tool_choice:` kwargs.
+- `response_format` — e.g. `{ type: "json_object" }` for structured output.
+- `reasoning_summary` — request a summary of the model's reasoning.
+
+```ruby
+provider_config: {
+  response_format: { type: "json_object" },
+  reasoning_summary: "auto"
+}
+```
+
+### Reasoning controls
+
+Two provider-agnostic knobs live in `config` (pass them as top-level kwargs):
+
+- `reasoning_effort` — one of `none`, `low`, `medium`, `high`, `xhigh`, `max`.
+  Clamped to the nearest tier the target model supports.
+- `reasoning_mode` — `standard` or `pro`. `pro` is native on OpenAI gpt-5.6/-pro
+  models and emulated as a one-tier effort bump elsewhere.
+
+```ruby
+resp = client.chat(
+  model: "openai/gpt-5.6-sol",
+  messages: "Prove that sqrt(2) is irrational.",
+  reasoning_effort: "high",
+  reasoning_mode: "pro"
+)
+```
 
 ## Models and health
 

data/lib/llmshim/client.rb

@@ -43,7 +43,15 @@ module Llmshim
     # @param opts [Hash] see #build_body (max_tokens, temperature, top_p, top_k,
     #   stop, reasoning_effort, config, provider_config, tools, tool_choice, fallback)
     # @return [Llmshim::ChatResponse]
+    # @raise [ArgumentError] if +stream: true+ is passed — /v1/chat would emit
+    #   SSE, which this method cannot parse. Use #stream for streaming.
     def chat(model:, messages:, **opts)
+      if opts[:stream]
+        raise ArgumentError,
+              "chat() returns a single ChatResponse and cannot stream. " \
+              "Use #stream (or Llmshim.stream) for streaming responses."
+      end
+
       body = build_body(model, messages, opts)
       hash = post_json("/v1/chat", body)
       ChatResponse.from_hash(hash)

data/lib/llmshim/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Llmshim
-  VERSION = "0.3.3"
+  VERSION = "0.3.4"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: llmshim
 version: !ruby/object:Gem::Version
-  version: 0.3.3
+  version: 0.3.4
 platform: ruby
 authors:
 - Sanjay Nadhavajhala
@@ -55,8 +55,8 @@ dependencies:
 description: |
   A thin, dependency-free HTTP client for a running llmshim proxy. Send
   OpenAI-style chat requests and let llmshim translate to OpenAI, Anthropic,
-  Google Gemini, or xAI. Supports non-streaming and SSE streaming, model
-  listing, and health checks. Standard library only.
+  Google Gemini, xAI, OpenRouter, vLLM, or SGLang. Supports non-streaming and
+  SSE streaming, model listing, and health checks. Standard library only.
 email:
 - sanjay@f2.ai
 executables: []