llmshim 0.6.0__tar.gz → 0.7.1__tar.gz

{llmshim-0.6.0 → llmshim-0.7.1}/CLAUDE.md

@@ -228,8 +228,16 @@ own: sliding failure window, open state, and a single half-open probe admitted
 after the cooldown. Config: `LLMSHIM_BREAKER_WINDOW_SECS` (60),
 `LLMSHIM_BREAKER_TRIP_THRESHOLD` (3; `0` disables), `LLMSHIM_BREAKER_COOLDOWN_SECS` (30).
 
-The breaker hangs on the `Router` (`Router::breaker()` / `with_breaker`). Every
-dispatch path *observes* outcomes so health accrues from ordinary traffic; only
+The breaker hangs on the `Router` (`Router::breaker()` / `with_breaker`), but
+the *counting* happens in `ShimClient`: `ShimClient::with_breaker` attaches one,
+and `completion` / `stream` / `stream_owned` observe their final result exactly
+once. The top-level entry points bind the router's breaker to the shared client
+per call (`lib.rs::bound_client`), so `llmshim::completion`, `stream`,
+`completion_with_fallback` and a caller that resolves its own provider and dials
+`ShimClient` directly all feed the same breaker — the last one only if it opted
+in with `ShimClient::new().with_breaker(router.breaker().clone())`; a bare
+`ShimClient::new()` reports to nobody. Do not add a second `.observe` around a
+client call: one call, one observation (`tests/unit_client_breaker.rs`). Only
 `fallback.rs` *refuses*, and it checks before every attempt rather than once per
 chain entry — the attempt that opens a circuit is usually the chain's own, so a
 per-entry check would still retry into a target it just watched die. A single-target call is still dispatched:

{llmshim-0.6.0 → llmshim-0.7.1}/Cargo.lock

@@ -1180,7 +1180,7 @@ checksum = "6373607a59f0be73a39b6fe456b8192fcc3585f602af20751600e974dd455e77"
 
 [[package]]
 name = "llmshim"
-version = "0.6.0"
+version = "0.7.1"
 dependencies = [
  "async-stream",
  "async-trait",

{llmshim-0.6.0 → llmshim-0.7.1}/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "llmshim"
-version = "0.6.0"
+version = "0.7.1"
 edition = "2021"
 description = "Blazing fast LLM API translation layer in pure Rust"
 license = "MIT OR Apache-2.0"

{llmshim-0.6.0 → llmshim-0.7.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: llmshim
-Version: 0.6.0
+Version: 0.7.1
 Classifier: Development Status :: 4 - Beta
 Classifier: Intended Audience :: Developers
 Classifier: License :: OSI Approved :: MIT License

{llmshim-0.6.0 → llmshim-0.7.1}/docs/src/guides/caching.md

@@ -22,7 +22,14 @@ llmshim translates the declaration into the provider's caching mechanism.
 }
 ```
 
-`upto_message` is a zero-based index into the supplied messages. Labels are
+`upto_message` is a zero-based index into the `messages` array exactly as you
+sent it — system and developer messages count, and each `role: "tool"` result
+counts as its own entry — not into the provider-native array (Anthropic hoists
+the system prompt out, so native numbering differs). A caller whose own message
+model expands into more wire messages than it holds must remap before copying
+an index across. On Anthropic an index past the end is rejected with a 400,
+and one that lands too early silently caches less than intended; on every
+other provider segments are parsed but neither checked nor placed. Labels are
 informational and are never used to guess prompt semantics. Keep stable sections
 before volatile sections. The proxy accepts the same top-level `x-cache` field;
 Python and Ruby provide a `cache=`/`cache:` keyword, Go has `ChatRequest.Cache`,

{llmshim-0.6.0 → llmshim-0.7.1}/docs/src/proxy/scaling.md

@@ -106,11 +106,18 @@ buckets. A gateway key's identity may carry `budget_usd` and an optional
 {"sk-example": {"tenant": "acme", "tier": 1, "budget_usd": 100, "budget_window_secs": 86400}}
 ```
 
-Cost is only knowable after a response, so the cap is checked before dispatch
-and charged after: one in-flight request can overshoot.
-
-A response the catalog cannot price is **not** charged — recording zero would let
-an unpriced model run forever under a budget. So that a cap cannot silently stop
+Cost is only knowable after a response, so the cap is checked before dispatch and
+charged after. Everything admitted between the last charge and the next check
+passes, so the overshoot bound is **admitted concurrency × the most expensive
+request**, multiplied again across replicas that have not yet shared their
+ledger. Size a cap with that headroom in mind rather than as a hard ceiling.
+
+A response the catalog cannot price at all is **not** charged — recording zero
+would let an unpriced model run forever under a budget. A model that prices only
+*some* token classes is charged at its highest published rate for the rest, so a
+partial price bounds the charge from above instead of voiding it: 2,537 of the
+7,461 priced models in the catalog publish no `cache_read` rate, and voiding
+those would have reopened this same hole one layer down. So that a cap cannot silently stop
 binding, a request whose target has **no catalog price is refused before it runs**
 when a budget is set:
 

{llmshim-0.6.0 → llmshim-0.7.1}/src/cache.rs

@@ -10,20 +10,75 @@ use std::collections::BTreeMap;
 
 pub const ANTHROPIC_BREAKPOINT_LIMIT: usize = 4;
 
+/// How long the caller expects a prefix to stay byte-stable. Only the caller
+/// knows; llmshim never infers it.
 #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
 #[serde(rename_all = "lowercase")]
 pub enum Stability {
+    /// Stable across sessions — a one-hour breakpoint on Anthropic.
     Static,
+    /// Stable for this session — a five-minute breakpoint on Anthropic.
     Session,
+    /// The volatile tail. Places no marker; it exists so a caller can name
+    /// where stability ends.
     Turn,
 }
+
+/// One caller-declared stability boundary: every message up to and including
+/// `upto_message` is expected to stay byte-stable for as long as `stability`
+/// says.
+///
+/// **`upto_message` indexes the request's own `messages` array, exactly as the
+/// caller sent it.** Zero-based, and every entry counts — `system` and
+/// `developer` messages included, and each `role: "tool"` result as its own
+/// entry — because the annotation is applied before any adapter hoists the
+/// system prompt out or reshapes tool results into native turns. It is *not*
+/// an index into the provider-native array Anthropic receives, where the
+/// system message is gone and the numbering has shifted.
+///
+/// A caller whose own message model is richer than the wire's — one entry that
+/// expands into a leading system message plus one wire message per tool
+/// result, say — must remap to the index of the wire message it actually sent.
+/// Copied across unchanged, the boundary lands on the wrong message: too far
+/// and the request is rejected (`400 invalid x-cache: segment message index is
+/// out of range`); too near and less of the prefix is cached than was hashed,
+/// with nothing to say so.
+///
+/// llmshim's own insertions never move this index. A managed-output
+/// instruction merges into an existing system message, and when it has to
+/// prepend one it renumbers every segment itself (`shim.rs`,
+/// `prepend_instruction`).
+///
+/// Honoured on the Anthropic Messages wire only, where it becomes a
+/// `cache_control` breakpoint on the last block of that message (or the last
+/// tool call, or the tool result itself) — unless that last block is a
+/// thinking block, in which case the segment is skipped without a marker.
+/// The bounds check is Anthropic-only too: on every other wire the policy is
+/// still parsed (malformed `x-cache` fails everywhere), but a segment's index
+/// is neither checked nor placed — an out-of-range index there is silently
+/// ignored, not rejected. `label` is the caller's own tag; llmshim never
+/// reads it.
 #[derive(Debug, Clone, Serialize, Deserialize)]
 pub struct CacheSegment {
+    /// Index into the request's `messages` as sent — see the type-level note.
     pub upto_message: usize,
     #[serde(default)]
     pub label: String,
     pub stability: Stability,
 }
+
+/// The caller's `x-cache` annotation: stability boundaries plus an optional
+/// cache identity. Read once per request and never forwarded to a provider.
+///
+/// The two halves land on different wires. `segments` are Anthropic-only —
+/// see [`CacheSegment`] for the index convention and the no-op rule elsewhere.
+/// `key` is not: on the OpenAI Responses wire it becomes `prompt_cache_key`,
+/// and is ignored on the others. A request may carry both; each wire takes the
+/// half it can use.
+///
+/// Explicit segments supersede any request-level `cache_control` the caller
+/// placed, and Anthropic's four-breakpoint budget is spent from the end of the
+/// request backwards, so the boundaries nearest the tail are the ones kept.
 #[derive(Debug, Clone, Default, Serialize, Deserialize)]
 pub struct CachePolicy {
     #[serde(default)]

{llmshim-0.6.0 → llmshim-0.7.1}/src/client.rs

@@ -1,3 +1,4 @@
+use crate::breaker::ProviderBreaker;
 use crate::error::{Result, ShimError};
 use crate::provider::{Provider, ProviderRequest};
 use bytes::Bytes;
@@ -7,6 +8,7 @@ use futures::{Stream, StreamExt};
 use reqwest::header::HeaderMap;
 use reqwest::Client;
 use std::pin::Pin;
+use std::sync::Arc;
 use std::time::Duration;
 
 /// Retry bounds, resolved once from the environment (with defaults) at
@@ -55,6 +57,9 @@ fn env_parse<T: std::str::FromStr>(key: &str) -> Option<T> {
 pub struct ShimClient {
     http: Client,
     retry: RetryConfig,
+    /// Provider health, fed by every dispatch this client makes. `None` means
+    /// this client reports to nobody — see [`ShimClient::with_breaker`].
+    breaker: Option<Arc<ProviderBreaker>>,
 }
 
 impl Default for ShimClient {
@@ -77,6 +82,36 @@ impl ShimClient {
                 .build()
                 .expect("failed to build HTTP client"),
             retry: RetryConfig::from_env(),
+            breaker: None,
+        }
+    }
+
+    /// Report every dispatch's outcome to `breaker`.
+    ///
+    /// The breaker lives on the [`Router`](crate::router::Router), so a caller
+    /// that resolves a provider itself and comes straight here has to hand it
+    /// over: `ShimClient::new().with_breaker(router.breaker().clone())`. The
+    /// crate's own entry points (`llmshim::completion`, `stream`,
+    /// `completion_with_fallback`) bind the router's breaker this way, so this
+    /// is the one place a dispatch is counted — whichever door it came in by.
+    ///
+    /// Cheap: the HTTP connection pool is shared by clone, so binding a breaker
+    /// per call costs an `Arc` clone, not a new pool.
+    pub fn with_breaker(mut self, breaker: Arc<ProviderBreaker>) -> Self {
+        self.breaker = Some(breaker);
+        self
+    }
+
+    /// Record one dispatch against the attached breaker, if any. Called once
+    /// per public entry point on its *final* result — after transport retries
+    /// and any output-contract repair — so one caller-visible call is one
+    /// observation, never one per attempt.
+    ///
+    /// Takes the projected outcome rather than the result itself so a stream's
+    /// non-`Sync` body is never borrowed across the await.
+    async fn observe(&self, provider: &dyn Provider, outcome: std::result::Result<(), &ShimError>) {
+        if let Some(breaker) = &self.breaker {
+            breaker.observe(provider.name(), outcome).await;
         }
     }
 
@@ -167,6 +202,17 @@ impl ShimClient {
         provider: &dyn Provider,
         model: &str,
         request: &serde_json::Value,
+    ) -> Result<serde_json::Value> {
+        let result = self.completion_unobserved(provider, model, request).await;
+        self.observe(provider, result.as_ref().map(|_| ())).await;
+        result
+    }
+
+    async fn completion_unobserved(
+        &self,
+        provider: &dyn Provider,
+        model: &str,
+        request: &serde_json::Value,
     ) -> Result<serde_json::Value> {
         let plan = crate::shim::Plan::new(
             provider.name(),
@@ -232,6 +278,19 @@ impl ShimClient {
         provider: &dyn Provider,
         model: &str,
         request: &serde_json::Value,
+    ) -> Result<Pin<Box<dyn Stream<Item = Result<String>> + Send>>> {
+        // A stream's health verdict is whether it opened; per-chunk failures
+        // are the transport's business, not the breaker's.
+        let opened = self.stream_unobserved(provider, model, request).await;
+        self.observe(provider, opened.as_ref().map(|_| ())).await;
+        opened
+    }
+
+    async fn stream_unobserved(
+        &self,
+        provider: &dyn Provider,
+        model: &str,
+        request: &serde_json::Value,
     ) -> Result<Pin<Box<dyn Stream<Item = Result<String>> + Send>>> {
         let plan = crate::shim::Plan::new(
             provider.name(),
@@ -278,7 +337,23 @@ impl ShimClient {
     /// and keepalives while validation and a possible repair are in progress.
     pub async fn stream_owned(
         &self,
-        provider: std::sync::Arc<dyn Provider>,
+        provider: Arc<dyn Provider>,
+        model: &str,
+        request: &serde_json::Value,
+    ) -> Result<Pin<Box<dyn Stream<Item = Result<String>> + Send>>> {
+        // Observed on the open only. A buffered plan's repair re-opens inside
+        // the returned stream; that second dial is not a separate verdict.
+        let opened = self
+            .stream_owned_unobserved(provider.clone(), model, request)
+            .await;
+        self.observe(provider.as_ref(), opened.as_ref().map(|_| ()))
+            .await;
+        opened
+    }
+
+    async fn stream_owned_unobserved(
+        &self,
+        provider: Arc<dyn Provider>,
         model: &str,
         request: &serde_json::Value,
     ) -> Result<Pin<Box<dyn Stream<Item = Result<String>> + Send>>> {

{llmshim-0.6.0 → llmshim-0.7.1}/src/cost.rs

@@ -29,6 +29,27 @@ fn charge(tokens: u64, rate: Option<f64>) -> Option<f64> {
     Some(tokens as f64 * rate? / PER_MILLION)
 }
 
+/// The highest rate this model publishes, used where a class has none of its own.
+///
+/// Catalogs price partially and often: **2,537 of 7,461 priced models in the
+/// vendored snapshot carry no `cache_read` rate and 5,892 no `cache_write`**,
+/// including `gpt-5-pro` and `o3-pro`, priced `{input, output}` only. Returning
+/// `None` for those made the whole response unpriceable the moment it reported a
+/// cached token — and a discarded charge is a spend cap that silently stops
+/// binding, which is the failure this crate exists to prevent.
+///
+/// Substituting the highest published rate can only over-estimate, never under.
+/// For a cap that is the safe direction: it spends a budget slightly early, where
+/// under-estimating spends it forever.
+fn highest_published_rate(cost: &Cost) -> Option<f64> {
+    [cost.input, cost.output, cost.cache_read, cost.cache_write]
+        .into_iter()
+        .flatten()
+        .fold(None, |best: Option<f64>, rate| {
+            Some(best.map_or(rate, |b| b.max(rate)))
+        })
+}
+
 /// Price one normalized usage object.
 ///
 /// Input is charged on `uncached_input_tokens` — the prompt minus whatever
@@ -53,11 +74,17 @@ pub fn price(usage: &Value, cost: &Cost) -> Option<f64> {
         return None;
     }
 
+    // A class the model does not price falls back to its highest published rate
+    // rather than voiding the total. See `highest_published_rate`: the result is
+    // an upper bound, which is the only safe direction for a budget.
+    let ceiling = highest_published_rate(cost);
+    let rate_for = |own: Option<f64>| own.or(ceiling);
+
     Some(
-        charge(input, cost.input)?
-            + charge(count(usage, "completion_tokens"), cost.output)?
-            + charge(cache_read, cost.cache_read)?
-            + charge(cache_write, cost.cache_write)?,
+        charge(input, rate_for(cost.input))?
+            + charge(count(usage, "completion_tokens"), rate_for(cost.output))?
+            + charge(cache_read, rate_for(cost.cache_read))?
+            + charge(cache_write, rate_for(cost.cache_write))?,
     )
 }
 
@@ -221,14 +248,21 @@ mod tests {
     }
 
     #[test]
-    fn a_missing_cache_rate_poisons_the_total_only_when_it_is_used() {
+    fn a_missing_cache_rate_is_charged_at_the_models_highest_rate() {
+        // This replaced an assertion that a used-but-unpriced class voids the
+        // total. That instinct was right — never under-report — but the remedy
+        // was wrong: `None` is discarded by `SpendCap::record`, so a spend cap
+        // silently stopped binding for the 2,537 catalogued models that publish
+        // no `cache_read` rate. An upper bound honours the same principle
+        // without the hole.
         let partial = Cost {
             input: Some(3.0),
             output: Some(15.0),
             cache_read: None,
             cache_write: None,
         };
-        // No cache tokens: the missing rates are irrelevant.
+
+        // No cache tokens: the missing rates never come into play.
         close(
             price(
                 &json!({"uncached_input_tokens": 1_000_000, "completion_tokens": 0, "cache_read_tokens": 0}),
@@ -236,14 +270,26 @@ mod tests {
             ),
             3.0,
         );
-        // Cache tokens actually used, with no rate to charge them at.
-        assert_eq!(
+
+        // Cache tokens used with no rate of their own: charged at 15.0, the
+        // highest rate this model publishes — an over-estimate, never an under.
+        close(
             price(
-                &json!({"uncached_input_tokens": 1_000_000, "cache_read_tokens": 1}),
+                &json!({"uncached_input_tokens": 1_000_000, "cache_read_tokens": 1_000_000}),
                 &partial,
             ),
-            None,
-            "an unpriced bucket that was used must not round down to a partial sum"
+            18.0,
+        );
+
+        // The bound must never fall below what a complete price would charge.
+        let complete = Cost {
+            cache_read: Some(0.3),
+            ..partial
+        };
+        let usage = json!({"uncached_input_tokens": 1_000_000, "cache_read_tokens": 1_000_000});
+        assert!(
+            price(&usage, &partial).unwrap() >= price(&usage, &complete).unwrap(),
+            "a fallback rate must bound the real one from above, or a cap under-charges"
         );
     }
 

{llmshim-0.6.0 → llmshim-0.7.1}/src/fallback.rs

@@ -1,7 +1,6 @@
 use crate::error::{Result, ShimError};
 use crate::log::{LogEntry, Logger, RequestTimer};
 use crate::router::Router;
-use crate::SHARED_CLIENT;
 use serde_json::Value;
 use std::time::Duration;
 
@@ -75,7 +74,9 @@ pub async fn completion_with_fallback(
     };
 
     let mut errors: Vec<String> = Vec::new();
-    let client = &*SHARED_CLIENT;
+    // Every attempt below is counted by the client against this router's
+    // breaker; the loop only asks `admit` before dialling.
+    let client = crate::bound_client(router);
 
     for model_str in &models {
         // Build request with this model. A named route expands to its model and
@@ -118,10 +119,6 @@ pub async fn completion_with_fallback(
             // Keep OAuth preparation, SSE-only providers, reasoning provenance,
             // and tool normalization identical to an ordinary completion.
             let outcome = client.completion(provider, &model, &req).await;
-            router
-                .breaker()
-                .observe(provider.name(), outcome.as_ref().map(|_| ()))
-                .await;
             match outcome {
                 Ok(result) => {
                     if let Some(logger) = logger {

{llmshim-0.6.0 → llmshim-0.7.1}/src/lib.rs

@@ -79,16 +79,13 @@ pub async fn completion_with_logger(
         .ok_or(error::ShimError::MissingModel)?;
 
     let (provider, model) = router.resolve(model_str)?;
-    let client = &*SHARED_CLIENT;
+    let client = bound_client(router);
     let timer = RequestTimer::start();
 
     // Ordinary traffic feeds provider health too, so a chain's first fallback
     // decision is not the first thing that ever noticed a provider is down.
+    // The client does the counting; see `ShimClient::with_breaker`.
     let result = client.completion(provider, &model, request).await;
-    router
-        .breaker()
-        .observe(provider.name(), result.as_ref().map(|_| ()))
-        .await;
 
     match result {
         Ok(resp) => {
@@ -132,13 +129,13 @@ pub async fn stream(
     // Observed but not gated: a single-target call has no alternative, so
     // refusing here would only convert an upstream failure into a local one.
     // The breaker refuses where there is somewhere else to go — `fallback.rs`.
-    let client = &*SHARED_CLIENT;
-    let opened = client.stream_owned(provider.clone(), &model, request).await;
-    // A stream's health verdict is whether it opened; per-chunk failures are
-    // the transport's business, not the breaker's.
-    router
-        .breaker()
-        .observe(provider.name(), opened.as_ref().map(|_| ()))
-        .await;
-    opened
+    bound_client(router)
+        .stream_owned(provider, &model, request)
+        .await
+}
+
+/// The shared HTTP client, reporting to this router's breaker. The pool is
+/// shared by clone; only the breaker handle is per call.
+pub(crate) fn bound_client(router: &Router) -> ShimClient {
+    SHARED_CLIENT.clone().with_breaker(router.breaker().clone())
 }

{llmshim-0.6.0 → llmshim-0.7.1}/src/providers/anthropic.rs

@@ -116,6 +116,17 @@ fn extract_system_message(
     (system, rest)
 }
 
+/// One Chat Completions message in, one Anthropic message out — a `role:
+/// "tool"` result included, which becomes its own `user` message even when it
+/// sits beside another.
+///
+/// Same-role adjacency is deliberately left alone. The Messages API accepts
+/// it: its reference states that consecutive `user` or `assistant` turns in a
+/// request are combined into a single turn server-side (platform.claude.com,
+/// Messages API, `messages` parameter), and parallel tool results already
+/// reach it here as back-to-back `user` messages. Merging locally would only
+/// destroy message boundaries a caller may key on. Gemini is the one wire in
+/// this crate that rejects adjacency, and it merges in its own adapter.
 fn transform_messages(messages: &[Value]) -> Vec<Value> {
     messages
         .iter()

{llmshim-0.6.0 → llmshim-0.7.1}/src/providers/gemini.rs

@@ -114,6 +114,12 @@ fn transform_messages(messages: &[Value]) -> (Option<Value>, Vec<Value>) {
     (system_instruction, contents)
 }
 
+/// Gemini's own repair, not a shared one. This is the only wire in the crate
+/// known to reject adjacent same-role turns, so it is the only adapter that
+/// folds them together. Every other adapter passes adjacency through — each
+/// says why on its own message pass — because a merge destroys message
+/// boundaries a caller may depend on, and only a wire that would otherwise
+/// fail the request earns that.
 fn merge_same_role(turns: Vec<Value>) -> Vec<Value> {
     let mut merged: Vec<Value> = Vec::new();
     for turn in turns {

{llmshim-0.6.0 → llmshim-0.7.1}/src/providers/openai.rs

@@ -46,6 +46,13 @@ fn strip_cache_control(value: &mut Value) {
 /// - Assistant messages with tool_calls → split into the assistant message +
 ///   separate `function_call` items
 /// - `role: "tool"` messages → `function_call_output` items
+///
+/// Same-role adjacency passes through. `input` is a flat item list, and the
+/// Responses reference documents roles and their precedence but no ordering or
+/// alternation rule (developers.openai.com, Responses API, `input`); this
+/// adapter already emits several `function_call_output` items in a row for
+/// parallel tool calls. Two adjacent assistant messages stay two items. The
+/// ChatGPT adapter goes through this same translator and inherits the stance.
 fn sanitize_messages(messages: &[Value]) -> Vec<Value> {
     let mut result = Vec::new();
     for msg in messages {

{llmshim-0.6.0 → llmshim-0.7.1}/src/providers/openai_compat.rs

@@ -40,6 +40,16 @@ impl OpenAiCompatible {
 /// Strip llmshim-normalized / foreign-provider fields and normalize content
 /// blocks to Chat Completions form. Messages, `tool_calls`, and `role: "tool"`
 /// stay in Chat Completions shape (the target format).
+///
+/// Same-role adjacency passes through unchanged, and here the answer is
+/// genuinely the served model's. vLLM and SGLang render `messages` through the
+/// tokenizer's Jinja chat template (or `--chat-template`), so acceptance is a
+/// property of that template: most current ones accept adjacent turns, some
+/// older ones raise — Mistral-7B-Instruct-v0.1's template errors with
+/// "conversation roles must alternate user/assistant/user/assistant/...".
+/// llmshim cannot see the template, so it does not merge; a strict template's
+/// rejection surfaces as the server's own 400. No `x-vllm` / `x-sglang`
+/// parameter changes this.
 fn sanitize_messages(messages: &[Value]) -> Vec<Value> {
     messages
         .iter()

{llmshim-0.6.0 → llmshim-0.7.1}/src/providers/openrouter.rs

@@ -61,6 +61,14 @@ fn normalize_openrouter_effort(effort: &str, pro: bool) -> &'static str {
 /// llmshim-normalized / foreign-provider fields so multi-model conversations
 /// don't leak them, and normalize vision blocks to OpenAI form. Messages,
 /// `tool_calls`, and `role: "tool"` all stay in Chat Completions shape.
+///
+/// Same-role adjacency passes through unchanged. OpenRouter is an aggregator:
+/// its own API is Chat Completions and documents no alternation rule
+/// (openrouter.ai/docs, API reference and parameters). Whether the vendor
+/// behind a given slug rejects adjacent turns is unknown from here and is
+/// OpenRouter's to reconcile; a faithful passthrough does not pre-empt it by
+/// merging, which would destroy message boundaries for the vendors that
+/// accept them.
 fn sanitize_messages(messages: &[Value]) -> Vec<Value> {
     messages
         .iter()

{llmshim-0.6.0 → llmshim-0.7.1}/src/providers/xai.rs

@@ -29,6 +29,13 @@ impl Xai {
 /// - Assistant messages with tool_calls → split into the assistant message +
 ///   separate `function_call` items
 /// - `role: "tool"` messages → `function_call_output` items
+///
+/// Same-role adjacency passes through unchanged. xAI's API is OpenAI
+/// Responses-shaped and its documentation (docs.x.ai, chat guide and API
+/// reference) states no ordering or alternation rule — but silence is not a
+/// verified acceptance, and this has not been checked live. Nothing is merged
+/// here because merging would destroy message boundaries a caller may depend
+/// on; if xAI ever rejects adjacency the rejection arrives as its own 400.
 fn sanitize_messages(messages: &[Value]) -> Vec<Value> {
     let mut result = Vec::new();
     for msg in messages {

{llmshim-0.6.0 → llmshim-0.7.1}/src/proxy/types.rs

@@ -119,7 +119,12 @@ pub struct Usage {
     pub cache_read_tokens: u64,
     pub cache_write_tokens: u64,
     /// USD charged for this response. `null` means the catalog carries no price
-    /// for the model — it never means free. See `llmshim::cost`.
+    /// for the model at all — it never means free.
+    ///
+    /// Where a model prices some token classes and not others, the unpriced ones
+    /// are charged at its highest published rate, so this is an **upper bound**
+    /// rather than `null`. Under-reporting would let a spend cap stop binding;
+    /// over-reporting merely spends a budget slightly early. See `llmshim::cost`.
     pub cost_usd: Option<f64>,
 }