llmshim 0.3.5__tar.gz → 0.3.6__tar.gz

{llmshim-0.3.5 → llmshim-0.3.6}/CLAUDE.md

@@ -58,12 +58,23 @@ llmshim::completion(router, request)
 
 Every provider implements: `transform_request`, `transform_response`, `transform_stream_chunk`.
 
+Non-streaming OpenAI/xAI Responses, Gemini and Anthropic transformations require
+supported terminal metadata. Do not turn absent or unrecognized status into
+`finish_reason: "stop"`. Rejected metadata returns a fixed 502 provider error
+without copying provider-supplied status/content into the diagnostic. Tests
+live in `tests/support/completion_status.rs`, included by `unit_openai`.
+Streaming status handling is separate and is not changed by this policy.
+
 ### Router (`src/router.rs`)
 
 Parses `"provider/model"` strings by splitting on the **first** `/` only, so an OpenRouter slug's internal slash survives (`openrouter/anthropic/claude-sonnet-4.5` → provider `openrouter`, model `anthropic/claude-sonnet-4.5`). Auto-infers provider from prefix (`gpt*`/`o*` → openai, `claude*` → anthropic, `gemini*` → gemini, `grok*` → xai); **OpenRouter, vLLM, and SGLang have no prefix inference** — their slugs collide with everyone's, so address them explicitly (`openrouter/…`, `vllm/…`, `sglang/…`); the first-slash split also preserves HF-style served-model slugs (`vllm/meta-llama/Llama-3.1-8B-Instruct`). Supports aliases. `Router::from_env()` reads API-key env vars, plus `VLLM_BASE_URL` / `SGLANG_BASE_URL` (+ optional `*_API_KEY`) for the self-hosted providers.
 
 ### HTTP Client (`src/client.rs`)
 
+Automatic redirects are disabled on the shared client. Keep prompts and
+provider-specific credential headers at the configured endpoint; 3xx responses
+remain provider errors. Callers must configure the final URL directly.
+
 `ShimClient` with shared connection pool (`LazyLock`), HTTP/2, gzip/brotli/zstd compression, TCP keepalive + nodelay. Automatic retry (3 attempts by default) on transport errors and 429/500/502/503/504/529 status codes. This is the **reactive** layer: on a retryable *response* it honors the server's `Retry-After` header (integer seconds or HTTP-date) and provider reset hints (OpenAI `x-ratelimit-reset-*`, Anthropic `anthropic-ratelimit-*-reset`), clamped to a cap and nudged with a little jitter; when there's no server hint (or a transport error) it falls back to full-jitter exponential backoff (uniform in `[0, min(cap, base·2^attempt)]`) to avoid a thundering herd. Tunable via `LLMSHIM_MAX_RETRIES` and `LLMSHIM_MAX_BACKOFF_SECS`. `warmup()` pre-establishes TCP+TLS connections. `SseStream` buffers bytes, extracts `data:` lines, routes through provider's `transform_stream_chunk`.
 
 ### Fallback chains (`src/fallback.rs`)

{llmshim-0.3.5 → llmshim-0.3.6}/Cargo.lock

@@ -958,7 +958,7 @@ checksum = "6373607a59f0be73a39b6fe456b8192fcc3585f602af20751600e974dd455e77"
 
 [[package]]
 name = "llmshim"
-version = "0.3.5"
+version = "0.3.6"
 dependencies = [
  "async-stream",
  "async-trait",

{llmshim-0.3.5 → llmshim-0.3.6}/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "llmshim"
-version = "0.3.5"
+version = "0.3.6"
 edition = "2021"
 description = "Blazing fast LLM API translation layer in pure Rust"
 license = "MIT OR Apache-2.0"

{llmshim-0.3.5 → llmshim-0.3.6}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: llmshim
-Version: 0.3.5
+Version: 0.3.6
 Classifier: Development Status :: 4 - Beta
 Classifier: Intended Audience :: Developers
 Classifier: License :: OSI Approved :: MIT License

{llmshim-0.3.5 → llmshim-0.3.6}/README.md

@@ -76,8 +76,23 @@ llmshim configure          # interactive prompt
 
 ---
 
+## Endpoint redirects
+
+The shared HTTP client does not follow redirects. Configure the final API URL
+directly: a 3xx response is returned as a provider error instead of forwarding
+the prompt and provider-specific credential headers to another endpoint.
+This applies to both streaming and non-streaming requests.
+
 ## Use it from Rust
 
+Non-streaming OpenAI Responses, xAI Responses, Gemini, and Anthropic results
+require supported terminal metadata. Missing, malformed, or unsupported
+statuses return a `ProviderError` (502) with a fixed diagnostic instead of
+being interpreted as successful completion. Known completion, output-limit,
+filtering and Anthropic tool-call mappings remain available. Additional native
+reasons require an explicit mapping; streaming transforms are unchanged.
+Provider mocks should include the native terminal status/stop reason.
+
 ```bash
 cargo add llmshim tokio serde_json
 ```

{llmshim-0.3.5 → llmshim-0.3.6}/docs/src/reference/errors.md

@@ -23,6 +23,18 @@ failure can occur after streaming begins.
 
 ## What the shared client retries
 
+Provider requests do not follow HTTP redirects. A `3xx` response returns a
+`ProviderError` with that status; configure the final endpoint URL directly.
+This keeps prompts and provider authentication headers at the configured
+destination for both completion and streaming requests.
+
+Non-streaming native normalizers require supported terminal metadata. Missing,
+malformed, nonterminal or unrecognized status/finish-reason values produce a
+`ProviderError` with status `502` and a fixed diagnostic, rather than labeling
+partial text as a successful `stop`. The diagnostic excludes response text
+and the provider-supplied reason. Known completion, length, filtering and tool
+mappings remain supported. Streaming transformations are unchanged.
+
 The shared provider client automatically retries:
 
 - transport connect, timeout, request, and body failures;

{llmshim-0.3.5 → llmshim-0.3.6}/src/client.rs

@@ -65,6 +65,9 @@ impl ShimClient {
     pub fn new() -> Self {
         Self {
             http: Client::builder()
+                // Prompts and custom provider credentials belong only at the
+                // configured endpoint, never an HTTP Location target.
+                .redirect(reqwest::redirect::Policy::none())
                 .pool_idle_timeout(Duration::from_secs(90))
                 .pool_max_idle_per_host(4)
                 .tcp_keepalive(Duration::from_secs(30))
@@ -670,6 +673,49 @@ mod tests {
 
     // --- Integration (mockito, local only — no provider API calls) ----------
 
+    #[tokio::test]
+    async fn redirects_do_not_forward_prompts_or_credentials() {
+        for status in [301, 302, 303, 307, 308] {
+            for same_origin in [false, true] {
+                let mut origin = mockito::Server::new_async().await;
+                let mut other = mockito::Server::new_async().await;
+                let destination = if same_origin { &mut origin } else { &mut other };
+                let location = format!("{}/moved", destination.url());
+                let forwarded = destination
+                    .mock(if status <= 303 { "GET" } else { "POST" }, "/moved")
+                    .with_status(200)
+                    .with_body("unexpected forwarding")
+                    .expect(0)
+                    .create_async()
+                    .await;
+                let redirect = origin
+                    .mock("POST", "/v1/chat/completions")
+                    .match_header("x-api-key", "test-secret")
+                    .match_body(mockito::Matcher::Json(serde_json::json!({
+                        "messages": [{"role": "user", "content": "private transcript"}]
+                    })))
+                    .with_status(status)
+                    .with_header("location", &location)
+                    .expect(1)
+                    .create_async()
+                    .await;
+                let request = ProviderRequest {
+                    url: format!("{}/v1/chat/completions", origin.url()),
+                    headers: vec![("x-api-key".into(), "test-secret".into())],
+                    body: serde_json::json!({
+                        "messages": [{"role": "user", "content": "private transcript"}]
+                    }),
+                };
+                let result = ShimClient::new().send(&request).await;
+                redirect.assert_async().await;
+                forwarded.assert_async().await;
+                assert!(
+                    matches!(result, Err(ShimError::ProviderError { status: actual, .. }) if actual == status as u16)
+                );
+            }
+        }
+    }
+
     #[tokio::test]
     async fn honors_retry_after_then_succeeds() {
         let mut server = mockito::Server::new_async().await;

{llmshim-0.3.5 → llmshim-0.3.6}/src/providers/anthropic.rs

@@ -381,7 +381,7 @@ fn translate_tool_choice(tc: &Value) -> Option<Value> {
 
 // -- Response transformation helpers --
 
-fn transform_response_to_openai(model: &str, resp: &Value) -> Value {
+fn transform_response_to_openai(model: &str, resp: &Value) -> Result<Value> {
     let content_blocks = resp
         .get("content")
         .and_then(|c| c.as_array())
@@ -438,16 +438,17 @@ fn transform_response_to_openai(model: &str, resp: &Value) -> Value {
         json!(text_parts.join(""))
     };
 
-    let stop_reason = resp
-        .get("stop_reason")
-        .and_then(|r| r.as_str())
-        .map(|r| match r {
-            "end_turn" => "stop",
-            "max_tokens" => "length",
-            "tool_use" => "tool_calls",
-            other => other,
-        })
-        .unwrap_or("stop");
+    let stop_reason = match resp.get("stop_reason").and_then(Value::as_str) {
+        Some("end_turn" | "stop_sequence") => "stop",
+        Some("max_tokens") => "length",
+        Some("tool_use") => "tool_calls",
+        _ => {
+            return Err(ShimError::ProviderError {
+                status: 502,
+                body: "Anthropic response has no supported terminal stop reason".into(),
+            })
+        }
+    };
 
     let usage = resp.get("usage").cloned().unwrap_or(json!({}));
     let normalized_usage = normalized_anthropic_usage(&usage);
@@ -472,7 +473,7 @@ fn transform_response_to_openai(model: &str, resp: &Value) -> Value {
         message["redacted_reasoning_content"] = json!(data);
     }
 
-    json!({
+    Ok(json!({
         "id": resp.get("id").cloned().unwrap_or(json!("")),
         "object": "chat.completion",
         "model": model,
@@ -482,7 +483,7 @@ fn transform_response_to_openai(model: &str, resp: &Value) -> Value {
             "finish_reason": stop_reason,
         }],
         "usage": normalized_usage
-    })
+    }))
 }
 
 /// Normalize a unified reasoning effort (`none|low|medium|high|xhigh|max`,
@@ -770,7 +771,7 @@ impl Provider for Anthropic {
             });
         }
 
-        Ok(transform_response_to_openai(model, &response))
+        transform_response_to_openai(model, &response)
     }
 
     fn transform_stream_chunk(&self, model: &str, chunk: &str) -> Result<Option<String>> {

{llmshim-0.3.5 → llmshim-0.3.6}/src/providers/gemini.rs

@@ -540,16 +540,17 @@ fn transform_response_to_openai(model: &str, resp: &Value) -> Result<Value> {
         json!(text_parts.join(""))
     };
 
-    let finish_reason = candidate
-        .get("finishReason")
-        .and_then(|f| f.as_str())
-        .map(|f| match f {
-            "STOP" => "stop",
-            "MAX_TOKENS" => "length",
-            "SAFETY" => "content_filter",
-            _ => "stop",
-        })
-        .unwrap_or("stop");
+    let finish_reason = match candidate.get("finishReason").and_then(Value::as_str) {
+        Some("STOP") => "stop",
+        Some("MAX_TOKENS") => "length",
+        Some("SAFETY") => "content_filter",
+        _ => {
+            return Err(ShimError::ProviderError {
+                status: 502,
+                body: "Gemini response has no supported terminal finish reason".into(),
+            })
+        }
+    };
 
     let usage = resp.get("usageMetadata").cloned().unwrap_or(json!({}));
 

{llmshim-0.3.5 → llmshim-0.3.6}/src/providers/openai.rs

@@ -539,14 +539,15 @@ impl Provider for OpenAi {
             message["reasoning_content"] = json!(reasoning);
         }
 
-        let status = response
-            .get("status")
-            .and_then(|s| s.as_str())
-            .unwrap_or("completed");
-        let finish_reason = match status {
-            "completed" => "stop",
-            "incomplete" => "length",
-            _ => "stop",
+        let finish_reason = match response.get("status").and_then(Value::as_str) {
+            Some("completed") => "stop",
+            Some("incomplete") => "length",
+            _ => {
+                return Err(ShimError::ProviderError {
+                    status: 502,
+                    body: "OpenAI response has no supported terminal status".into(),
+                })
+            }
         };
 
         let usage = response.get("usage").cloned().unwrap_or(json!({}));

{llmshim-0.3.5 → llmshim-0.3.6}/src/providers/xai.rs

@@ -379,14 +379,15 @@ impl Provider for Xai {
             message["reasoning_content"] = json!(reasoning);
         }
 
-        let status = response
-            .get("status")
-            .and_then(|s| s.as_str())
-            .unwrap_or("completed");
-        let finish_reason = match status {
-            "completed" => "stop",
-            "incomplete" => "length",
-            _ => "stop",
+        let finish_reason = match response.get("status").and_then(Value::as_str) {
+            Some("completed") => "stop",
+            Some("incomplete") => "length",
+            _ => {
+                return Err(ShimError::ProviderError {
+                    status: 502,
+                    body: "xAI response has no supported terminal status".into(),
+                });
+            }
         };
 
         let usage = response.get("usage").cloned().unwrap_or(json!({}));

llmshim-0.3.6/tests/support/completion_status.rs

@@ -0,0 +1,124 @@
+use llmshim::provider::Provider;
+use llmshim::providers::{anthropic::Anthropic, gemini::Gemini, openai::OpenAi, xai::Xai};
+use serde_json::{json, Value};
+
+fn check(provider: &dyn Provider, template: Value, pointer: &str, bad: &[Value]) {
+    for value in bad {
+        let mut response = template.clone();
+        *response.pointer_mut(pointer).unwrap() = value.clone();
+        let error = provider
+            .transform_response("test-model", response)
+            .unwrap_err();
+        assert!(!error.to_string().contains("private"));
+    }
+    let mut response = template;
+    let (parent, key) = pointer.rsplit_once('/').unwrap();
+    response
+        .pointer_mut(parent)
+        .unwrap()
+        .as_object_mut()
+        .unwrap()
+        .remove(key);
+    assert!(provider.transform_response("test-model", response).is_err());
+}
+
+#[test]
+fn openai_nonterminal_or_missing_status_cannot_become_stop() {
+    check_responses_api(&OpenAi::new("unused".into()));
+}
+
+#[test]
+fn xai_nonterminal_or_missing_status_cannot_become_stop() {
+    check_responses_api(&Xai::new("unused".into()));
+}
+
+fn check_responses_api(provider: &dyn Provider) {
+    let response = json!({"status":"completed","output":[{"type":"message","content":[{"type":"output_text","text":"private partial text"}]}]});
+    check(
+        provider,
+        response.clone(),
+        "/status",
+        &[
+            json!("failed"),
+            json!("cancelled"),
+            json!("queued"),
+            json!("in_progress"),
+            json!("private-unknown"),
+            Value::Null,
+            json!(42),
+        ],
+    );
+    for (status, expected) in [("completed", "stop"), ("incomplete", "length")] {
+        let mut value = response.clone();
+        value["status"] = json!(status);
+        assert_eq!(
+            provider.transform_response("test-model", value).unwrap()["choices"][0]
+                ["finish_reason"],
+            expected
+        );
+    }
+}
+
+#[test]
+fn gemini_unknown_or_missing_reason_cannot_become_stop() {
+    let provider = Gemini::new("unused".into());
+    let response = json!({"candidates":[{"finishReason":"STOP","content":{"parts":[{"text":"private partial text"}]}}]});
+    check(
+        &provider,
+        response.clone(),
+        "/candidates/0/finishReason",
+        &[
+            json!("RECITATION"),
+            json!("MALFORMED_FUNCTION_CALL"),
+            json!("OTHER"),
+            json!("private-unknown"),
+            Value::Null,
+            json!(42),
+        ],
+    );
+    for (reason, expected) in [
+        ("STOP", "stop"),
+        ("MAX_TOKENS", "length"),
+        ("SAFETY", "content_filter"),
+    ] {
+        let mut value = response.clone();
+        value["candidates"][0]["finishReason"] = json!(reason);
+        assert_eq!(
+            provider.transform_response("test-model", value).unwrap()["choices"][0]
+                ["finish_reason"],
+            expected
+        );
+    }
+}
+
+#[test]
+fn anthropic_missing_or_unknown_reason_cannot_become_stop() {
+    let provider = Anthropic::new("unused".into());
+    let response =
+        json!({"stop_reason":"end_turn","content":[{"type":"text","text":"private partial text"}]});
+    check(
+        &provider,
+        response.clone(),
+        "/stop_reason",
+        &[
+            json!("pause_turn"),
+            json!("private-unknown"),
+            Value::Null,
+            json!(42),
+        ],
+    );
+    for (reason, expected) in [
+        ("end_turn", "stop"),
+        ("stop_sequence", "stop"),
+        ("max_tokens", "length"),
+        ("tool_use", "tool_calls"),
+    ] {
+        let mut value = response.clone();
+        value["stop_reason"] = json!(reason);
+        assert_eq!(
+            provider.transform_response("test-model", value).unwrap()["choices"][0]
+                ["finish_reason"],
+            expected
+        );
+    }
+}

{llmshim-0.3.5 → llmshim-0.3.6}/tests/unit_openai.rs

@@ -2,6 +2,9 @@ use llmshim::provider::Provider;
 use llmshim::providers::openai::OpenAi;
 use serde_json::{json, Value};
 
+#[path = "support/completion_status.rs"]
+mod completion_status;
+
 fn provider() -> OpenAi {
     OpenAi::new("test-key-123".into())
 }