llmshim 0.7.0__tar.gz → 0.7.1__tar.gz

{llmshim-0.7.0 → llmshim-0.7.1}/CLAUDE.md

@@ -228,8 +228,16 @@ own: sliding failure window, open state, and a single half-open probe admitted
 after the cooldown. Config: `LLMSHIM_BREAKER_WINDOW_SECS` (60),
 `LLMSHIM_BREAKER_TRIP_THRESHOLD` (3; `0` disables), `LLMSHIM_BREAKER_COOLDOWN_SECS` (30).
 
-The breaker hangs on the `Router` (`Router::breaker()` / `with_breaker`). Every
-dispatch path *observes* outcomes so health accrues from ordinary traffic; only
+The breaker hangs on the `Router` (`Router::breaker()` / `with_breaker`), but
+the *counting* happens in `ShimClient`: `ShimClient::with_breaker` attaches one,
+and `completion` / `stream` / `stream_owned` observe their final result exactly
+once. The top-level entry points bind the router's breaker to the shared client
+per call (`lib.rs::bound_client`), so `llmshim::completion`, `stream`,
+`completion_with_fallback` and a caller that resolves its own provider and dials
+`ShimClient` directly all feed the same breaker — the last one only if it opted
+in with `ShimClient::new().with_breaker(router.breaker().clone())`; a bare
+`ShimClient::new()` reports to nobody. Do not add a second `.observe` around a
+client call: one call, one observation (`tests/unit_client_breaker.rs`). Only
 `fallback.rs` *refuses*, and it checks before every attempt rather than once per
 chain entry — the attempt that opens a circuit is usually the chain's own, so a
 per-entry check would still retry into a target it just watched die. A single-target call is still dispatched:

{llmshim-0.7.0 → llmshim-0.7.1}/Cargo.lock

@@ -1180,7 +1180,7 @@ checksum = "6373607a59f0be73a39b6fe456b8192fcc3585f602af20751600e974dd455e77"
 
 [[package]]
 name = "llmshim"
-version = "0.7.0"
+version = "0.7.1"
 dependencies = [
  "async-stream",
  "async-trait",

{llmshim-0.7.0 → llmshim-0.7.1}/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "llmshim"
-version = "0.7.0"
+version = "0.7.1"
 edition = "2021"
 description = "Blazing fast LLM API translation layer in pure Rust"
 license = "MIT OR Apache-2.0"

{llmshim-0.7.0 → llmshim-0.7.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: llmshim
-Version: 0.7.0
+Version: 0.7.1
 Classifier: Development Status :: 4 - Beta
 Classifier: Intended Audience :: Developers
 Classifier: License :: OSI Approved :: MIT License

{llmshim-0.7.0 → llmshim-0.7.1}/docs/src/guides/caching.md

@@ -22,7 +22,14 @@ llmshim translates the declaration into the provider's caching mechanism.
 }
 ```
 
-`upto_message` is a zero-based index into the supplied messages. Labels are
+`upto_message` is a zero-based index into the `messages` array exactly as you
+sent it — system and developer messages count, and each `role: "tool"` result
+counts as its own entry — not into the provider-native array (Anthropic hoists
+the system prompt out, so native numbering differs). A caller whose own message
+model expands into more wire messages than it holds must remap before copying
+an index across. On Anthropic an index past the end is rejected with a 400,
+and one that lands too early silently caches less than intended; on every
+other provider segments are parsed but neither checked nor placed. Labels are
 informational and are never used to guess prompt semantics. Keep stable sections
 before volatile sections. The proxy accepts the same top-level `x-cache` field;
 Python and Ruby provide a `cache=`/`cache:` keyword, Go has `ChatRequest.Cache`,

{llmshim-0.7.0 → llmshim-0.7.1}/src/cache.rs

@@ -10,20 +10,75 @@ use std::collections::BTreeMap;
 
 pub const ANTHROPIC_BREAKPOINT_LIMIT: usize = 4;
 
+/// How long the caller expects a prefix to stay byte-stable. Only the caller
+/// knows; llmshim never infers it.
 #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
 #[serde(rename_all = "lowercase")]
 pub enum Stability {
+    /// Stable across sessions — a one-hour breakpoint on Anthropic.
     Static,
+    /// Stable for this session — a five-minute breakpoint on Anthropic.
     Session,
+    /// The volatile tail. Places no marker; it exists so a caller can name
+    /// where stability ends.
     Turn,
 }
+
+/// One caller-declared stability boundary: every message up to and including
+/// `upto_message` is expected to stay byte-stable for as long as `stability`
+/// says.
+///
+/// **`upto_message` indexes the request's own `messages` array, exactly as the
+/// caller sent it.** Zero-based, and every entry counts — `system` and
+/// `developer` messages included, and each `role: "tool"` result as its own
+/// entry — because the annotation is applied before any adapter hoists the
+/// system prompt out or reshapes tool results into native turns. It is *not*
+/// an index into the provider-native array Anthropic receives, where the
+/// system message is gone and the numbering has shifted.
+///
+/// A caller whose own message model is richer than the wire's — one entry that
+/// expands into a leading system message plus one wire message per tool
+/// result, say — must remap to the index of the wire message it actually sent.
+/// Copied across unchanged, the boundary lands on the wrong message: too far
+/// and the request is rejected (`400 invalid x-cache: segment message index is
+/// out of range`); too near and less of the prefix is cached than was hashed,
+/// with nothing to say so.
+///
+/// llmshim's own insertions never move this index. A managed-output
+/// instruction merges into an existing system message, and when it has to
+/// prepend one it renumbers every segment itself (`shim.rs`,
+/// `prepend_instruction`).
+///
+/// Honoured on the Anthropic Messages wire only, where it becomes a
+/// `cache_control` breakpoint on the last block of that message (or the last
+/// tool call, or the tool result itself) — unless that last block is a
+/// thinking block, in which case the segment is skipped without a marker.
+/// The bounds check is Anthropic-only too: on every other wire the policy is
+/// still parsed (malformed `x-cache` fails everywhere), but a segment's index
+/// is neither checked nor placed — an out-of-range index there is silently
+/// ignored, not rejected. `label` is the caller's own tag; llmshim never
+/// reads it.
 #[derive(Debug, Clone, Serialize, Deserialize)]
 pub struct CacheSegment {
+    /// Index into the request's `messages` as sent — see the type-level note.
     pub upto_message: usize,
     #[serde(default)]
     pub label: String,
     pub stability: Stability,
 }
+
+/// The caller's `x-cache` annotation: stability boundaries plus an optional
+/// cache identity. Read once per request and never forwarded to a provider.
+///
+/// The two halves land on different wires. `segments` are Anthropic-only —
+/// see [`CacheSegment`] for the index convention and the no-op rule elsewhere.
+/// `key` is not: on the OpenAI Responses wire it becomes `prompt_cache_key`,
+/// and is ignored on the others. A request may carry both; each wire takes the
+/// half it can use.
+///
+/// Explicit segments supersede any request-level `cache_control` the caller
+/// placed, and Anthropic's four-breakpoint budget is spent from the end of the
+/// request backwards, so the boundaries nearest the tail are the ones kept.
 #[derive(Debug, Clone, Default, Serialize, Deserialize)]
 pub struct CachePolicy {
     #[serde(default)]

{llmshim-0.7.0 → llmshim-0.7.1}/src/client.rs

@@ -1,3 +1,4 @@
+use crate::breaker::ProviderBreaker;
 use crate::error::{Result, ShimError};
 use crate::provider::{Provider, ProviderRequest};
 use bytes::Bytes;
@@ -7,6 +8,7 @@ use futures::{Stream, StreamExt};
 use reqwest::header::HeaderMap;
 use reqwest::Client;
 use std::pin::Pin;
+use std::sync::Arc;
 use std::time::Duration;
 
 /// Retry bounds, resolved once from the environment (with defaults) at
@@ -55,6 +57,9 @@ fn env_parse<T: std::str::FromStr>(key: &str) -> Option<T> {
 pub struct ShimClient {
     http: Client,
     retry: RetryConfig,
+    /// Provider health, fed by every dispatch this client makes. `None` means
+    /// this client reports to nobody — see [`ShimClient::with_breaker`].
+    breaker: Option<Arc<ProviderBreaker>>,
 }
 
 impl Default for ShimClient {
@@ -77,6 +82,36 @@ impl ShimClient {
                 .build()
                 .expect("failed to build HTTP client"),
             retry: RetryConfig::from_env(),
+            breaker: None,
+        }
+    }
+
+    /// Report every dispatch's outcome to `breaker`.
+    ///
+    /// The breaker lives on the [`Router`](crate::router::Router), so a caller
+    /// that resolves a provider itself and comes straight here has to hand it
+    /// over: `ShimClient::new().with_breaker(router.breaker().clone())`. The
+    /// crate's own entry points (`llmshim::completion`, `stream`,
+    /// `completion_with_fallback`) bind the router's breaker this way, so this
+    /// is the one place a dispatch is counted — whichever door it came in by.
+    ///
+    /// Cheap: the HTTP connection pool is shared by clone, so binding a breaker
+    /// per call costs an `Arc` clone, not a new pool.
+    pub fn with_breaker(mut self, breaker: Arc<ProviderBreaker>) -> Self {
+        self.breaker = Some(breaker);
+        self
+    }
+
+    /// Record one dispatch against the attached breaker, if any. Called once
+    /// per public entry point on its *final* result — after transport retries
+    /// and any output-contract repair — so one caller-visible call is one
+    /// observation, never one per attempt.
+    ///
+    /// Takes the projected outcome rather than the result itself so a stream's
+    /// non-`Sync` body is never borrowed across the await.
+    async fn observe(&self, provider: &dyn Provider, outcome: std::result::Result<(), &ShimError>) {
+        if let Some(breaker) = &self.breaker {
+            breaker.observe(provider.name(), outcome).await;
         }
     }
 
@@ -167,6 +202,17 @@ impl ShimClient {
         provider: &dyn Provider,
         model: &str,
         request: &serde_json::Value,
+    ) -> Result<serde_json::Value> {
+        let result = self.completion_unobserved(provider, model, request).await;
+        self.observe(provider, result.as_ref().map(|_| ())).await;
+        result
+    }
+
+    async fn completion_unobserved(
+        &self,
+        provider: &dyn Provider,
+        model: &str,
+        request: &serde_json::Value,
     ) -> Result<serde_json::Value> {
         let plan = crate::shim::Plan::new(
             provider.name(),
@@ -232,6 +278,19 @@ impl ShimClient {
         provider: &dyn Provider,
         model: &str,
         request: &serde_json::Value,
+    ) -> Result<Pin<Box<dyn Stream<Item = Result<String>> + Send>>> {
+        // A stream's health verdict is whether it opened; per-chunk failures
+        // are the transport's business, not the breaker's.
+        let opened = self.stream_unobserved(provider, model, request).await;
+        self.observe(provider, opened.as_ref().map(|_| ())).await;
+        opened
+    }
+
+    async fn stream_unobserved(
+        &self,
+        provider: &dyn Provider,
+        model: &str,
+        request: &serde_json::Value,
     ) -> Result<Pin<Box<dyn Stream<Item = Result<String>> + Send>>> {
         let plan = crate::shim::Plan::new(
             provider.name(),
@@ -278,7 +337,23 @@ impl ShimClient {
     /// and keepalives while validation and a possible repair are in progress.
     pub async fn stream_owned(
         &self,
-        provider: std::sync::Arc<dyn Provider>,
+        provider: Arc<dyn Provider>,
+        model: &str,
+        request: &serde_json::Value,
+    ) -> Result<Pin<Box<dyn Stream<Item = Result<String>> + Send>>> {
+        // Observed on the open only. A buffered plan's repair re-opens inside
+        // the returned stream; that second dial is not a separate verdict.
+        let opened = self
+            .stream_owned_unobserved(provider.clone(), model, request)
+            .await;
+        self.observe(provider.as_ref(), opened.as_ref().map(|_| ()))
+            .await;
+        opened
+    }
+
+    async fn stream_owned_unobserved(
+        &self,
+        provider: Arc<dyn Provider>,
         model: &str,
         request: &serde_json::Value,
     ) -> Result<Pin<Box<dyn Stream<Item = Result<String>> + Send>>> {

{llmshim-0.7.0 → llmshim-0.7.1}/src/fallback.rs

@@ -1,7 +1,6 @@
 use crate::error::{Result, ShimError};
 use crate::log::{LogEntry, Logger, RequestTimer};
 use crate::router::Router;
-use crate::SHARED_CLIENT;
 use serde_json::Value;
 use std::time::Duration;
 
@@ -75,7 +74,9 @@ pub async fn completion_with_fallback(
     };
 
     let mut errors: Vec<String> = Vec::new();
-    let client = &*SHARED_CLIENT;
+    // Every attempt below is counted by the client against this router's
+    // breaker; the loop only asks `admit` before dialling.
+    let client = crate::bound_client(router);
 
     for model_str in &models {
         // Build request with this model. A named route expands to its model and
@@ -118,10 +119,6 @@ pub async fn completion_with_fallback(
             // Keep OAuth preparation, SSE-only providers, reasoning provenance,
             // and tool normalization identical to an ordinary completion.
             let outcome = client.completion(provider, &model, &req).await;
-            router
-                .breaker()
-                .observe(provider.name(), outcome.as_ref().map(|_| ()))
-                .await;
             match outcome {
                 Ok(result) => {
                     if let Some(logger) = logger {

{llmshim-0.7.0 → llmshim-0.7.1}/src/lib.rs

@@ -79,16 +79,13 @@ pub async fn completion_with_logger(
         .ok_or(error::ShimError::MissingModel)?;
 
     let (provider, model) = router.resolve(model_str)?;
-    let client = &*SHARED_CLIENT;
+    let client = bound_client(router);
     let timer = RequestTimer::start();
 
     // Ordinary traffic feeds provider health too, so a chain's first fallback
     // decision is not the first thing that ever noticed a provider is down.
+    // The client does the counting; see `ShimClient::with_breaker`.
     let result = client.completion(provider, &model, request).await;
-    router
-        .breaker()
-        .observe(provider.name(), result.as_ref().map(|_| ()))
-        .await;
 
     match result {
         Ok(resp) => {
@@ -132,13 +129,13 @@ pub async fn stream(
     // Observed but not gated: a single-target call has no alternative, so
     // refusing here would only convert an upstream failure into a local one.
     // The breaker refuses where there is somewhere else to go — `fallback.rs`.
-    let client = &*SHARED_CLIENT;
-    let opened = client.stream_owned(provider.clone(), &model, request).await;
-    // A stream's health verdict is whether it opened; per-chunk failures are
-    // the transport's business, not the breaker's.
-    router
-        .breaker()
-        .observe(provider.name(), opened.as_ref().map(|_| ()))
-        .await;
-    opened
+    bound_client(router)
+        .stream_owned(provider, &model, request)
+        .await
+}
+
+/// The shared HTTP client, reporting to this router's breaker. The pool is
+/// shared by clone; only the breaker handle is per call.
+pub(crate) fn bound_client(router: &Router) -> ShimClient {
+    SHARED_CLIENT.clone().with_breaker(router.breaker().clone())
 }

{llmshim-0.7.0 → llmshim-0.7.1}/src/providers/anthropic.rs

@@ -116,6 +116,17 @@ fn extract_system_message(
     (system, rest)
 }
 
+/// One Chat Completions message in, one Anthropic message out — a `role:
+/// "tool"` result included, which becomes its own `user` message even when it
+/// sits beside another.
+///
+/// Same-role adjacency is deliberately left alone. The Messages API accepts
+/// it: its reference states that consecutive `user` or `assistant` turns in a
+/// request are combined into a single turn server-side (platform.claude.com,
+/// Messages API, `messages` parameter), and parallel tool results already
+/// reach it here as back-to-back `user` messages. Merging locally would only
+/// destroy message boundaries a caller may key on. Gemini is the one wire in
+/// this crate that rejects adjacency, and it merges in its own adapter.
 fn transform_messages(messages: &[Value]) -> Vec<Value> {
     messages
         .iter()

{llmshim-0.7.0 → llmshim-0.7.1}/src/providers/gemini.rs

@@ -114,6 +114,12 @@ fn transform_messages(messages: &[Value]) -> (Option<Value>, Vec<Value>) {
     (system_instruction, contents)
 }
 
+/// Gemini's own repair, not a shared one. This is the only wire in the crate
+/// known to reject adjacent same-role turns, so it is the only adapter that
+/// folds them together. Every other adapter passes adjacency through — each
+/// says why on its own message pass — because a merge destroys message
+/// boundaries a caller may depend on, and only a wire that would otherwise
+/// fail the request earns that.
 fn merge_same_role(turns: Vec<Value>) -> Vec<Value> {
     let mut merged: Vec<Value> = Vec::new();
     for turn in turns {

{llmshim-0.7.0 → llmshim-0.7.1}/src/providers/openai.rs

@@ -46,6 +46,13 @@ fn strip_cache_control(value: &mut Value) {
 /// - Assistant messages with tool_calls → split into the assistant message +
 ///   separate `function_call` items
 /// - `role: "tool"` messages → `function_call_output` items
+///
+/// Same-role adjacency passes through. `input` is a flat item list, and the
+/// Responses reference documents roles and their precedence but no ordering or
+/// alternation rule (developers.openai.com, Responses API, `input`); this
+/// adapter already emits several `function_call_output` items in a row for
+/// parallel tool calls. Two adjacent assistant messages stay two items. The
+/// ChatGPT adapter goes through this same translator and inherits the stance.
 fn sanitize_messages(messages: &[Value]) -> Vec<Value> {
     let mut result = Vec::new();
     for msg in messages {

{llmshim-0.7.0 → llmshim-0.7.1}/src/providers/openai_compat.rs

@@ -40,6 +40,16 @@ impl OpenAiCompatible {
 /// Strip llmshim-normalized / foreign-provider fields and normalize content
 /// blocks to Chat Completions form. Messages, `tool_calls`, and `role: "tool"`
 /// stay in Chat Completions shape (the target format).
+///
+/// Same-role adjacency passes through unchanged, and here the answer is
+/// genuinely the served model's. vLLM and SGLang render `messages` through the
+/// tokenizer's Jinja chat template (or `--chat-template`), so acceptance is a
+/// property of that template: most current ones accept adjacent turns, some
+/// older ones raise — Mistral-7B-Instruct-v0.1's template errors with
+/// "conversation roles must alternate user/assistant/user/assistant/...".
+/// llmshim cannot see the template, so it does not merge; a strict template's
+/// rejection surfaces as the server's own 400. No `x-vllm` / `x-sglang`
+/// parameter changes this.
 fn sanitize_messages(messages: &[Value]) -> Vec<Value> {
     messages
         .iter()

{llmshim-0.7.0 → llmshim-0.7.1}/src/providers/openrouter.rs

@@ -61,6 +61,14 @@ fn normalize_openrouter_effort(effort: &str, pro: bool) -> &'static str {
 /// llmshim-normalized / foreign-provider fields so multi-model conversations
 /// don't leak them, and normalize vision blocks to OpenAI form. Messages,
 /// `tool_calls`, and `role: "tool"` all stay in Chat Completions shape.
+///
+/// Same-role adjacency passes through unchanged. OpenRouter is an aggregator:
+/// its own API is Chat Completions and documents no alternation rule
+/// (openrouter.ai/docs, API reference and parameters). Whether the vendor
+/// behind a given slug rejects adjacent turns is unknown from here and is
+/// OpenRouter's to reconcile; a faithful passthrough does not pre-empt it by
+/// merging, which would destroy message boundaries for the vendors that
+/// accept them.
 fn sanitize_messages(messages: &[Value]) -> Vec<Value> {
     messages
         .iter()

{llmshim-0.7.0 → llmshim-0.7.1}/src/providers/xai.rs

@@ -29,6 +29,13 @@ impl Xai {
 /// - Assistant messages with tool_calls → split into the assistant message +
 ///   separate `function_call` items
 /// - `role: "tool"` messages → `function_call_output` items
+///
+/// Same-role adjacency passes through unchanged. xAI's API is OpenAI
+/// Responses-shaped and its documentation (docs.x.ai, chat guide and API
+/// reference) states no ordering or alternation rule — but silence is not a
+/// verified acceptance, and this has not been checked live. Nothing is merged
+/// here because merging would destroy message boundaries a caller may depend
+/// on; if xAI ever rejects adjacency the rejection arrives as its own 400.
 fn sanitize_messages(messages: &[Value]) -> Vec<Value> {
     let mut result = Vec::new();
     for msg in messages {

llmshim-0.7.1/tests/unit_client_breaker.rs

@@ -0,0 +1,118 @@
+//! Provider health is counted where the dispatch happens — in `ShimClient` —
+//! so a caller that resolves its own provider and comes straight to the client
+//! feeds the same breaker as the crate's top-level entry points, and one
+//! caller-visible call is one observation whichever door it came in by.
+use llmshim::breaker::{BreakerConfig, ProviderBreaker};
+use llmshim::client::ShimClient;
+use llmshim::providers::openai_compat::OpenAiCompatible;
+use llmshim::router::Router;
+use serde_json::json;
+use std::sync::Arc;
+use std::time::Duration;
+
+/// Trips on the second eligible failure. A path that observed one call twice
+/// would open the circuit on that single call; a path that never observed
+/// would never open it at all.
+fn breaker() -> Arc<ProviderBreaker> {
+    Arc::new(ProviderBreaker::with_config(BreakerConfig {
+        window: Duration::from_secs(60),
+        trip_threshold: 2,
+        cooldown: Duration::from_secs(300),
+    }))
+}
+
+/// A 503 is both health-counting and transport-retryable, so the client dials
+/// it several times per call. `Retry-After: 0` keeps those retries instant.
+async fn dead(server: &mut mockito::ServerGuard) -> mockito::Mock {
+    server
+        .mock("POST", "/chat/completions")
+        .with_status(503)
+        .with_header("retry-after", "0")
+        .with_body("upstream down")
+        .expect_at_least(1)
+        .create_async()
+        .await
+}
+
+fn ok_body() -> String {
+    json!({
+        "id": "r", "model": "m",
+        "choices": [{"message": {"role": "assistant", "content": "ok"}, "finish_reason": "stop"}],
+        "usage": {}
+    })
+    .to_string()
+}
+
+#[tokio::test]
+async fn a_direct_client_counts_each_call_once_and_a_success_closes_the_circuit() {
+    let mut server = mockito::Server::new_async().await;
+    let down = dead(&mut server).await;
+    let provider = OpenAiCompatible::new("local", server.url(), None);
+    let health = breaker();
+    let client = ShimClient::new().with_breaker(health.clone());
+    let request = json!({"model": "m", "messages": [{"role": "user", "content": "hi"}]});
+
+    assert!(client.completion(&provider, "m", &request).await.is_err());
+    assert!(
+        !health.should_skip("local"),
+        "the client's own transport retries are one dispatch, not several"
+    );
+    assert!(client.completion(&provider, "m", &request).await.is_err());
+    assert!(
+        health.should_skip("local"),
+        "the second failed call must be the one that opens the circuit"
+    );
+
+    down.remove_async().await;
+    let up = server
+        .mock("POST", "/chat/completions")
+        .with_body(ok_body())
+        .expect(1)
+        .create_async()
+        .await;
+    // A direct call is observed but never gated: with no alternative target,
+    // refusing would only turn an upstream failure into a local one.
+    assert!(client.completion(&provider, "m", &request).await.is_ok());
+    assert!(!health.should_skip("local"), "success closes it again");
+    up.assert_async().await;
+}
+
+#[tokio::test]
+async fn the_top_level_entry_points_count_through_the_client_and_not_again() {
+    let mut server = mockito::Server::new_async().await;
+    let _down = dead(&mut server).await;
+    let health = breaker();
+    let router = Router::new()
+        .register(
+            "local",
+            Box::new(OpenAiCompatible::new("local", server.url(), None)),
+        )
+        .with_breaker(health.clone());
+    let request = json!({"model": "local/m", "messages": [{"role": "user", "content": "hi"}]});
+
+    assert!(llmshim::completion(&router, &request).await.is_err());
+    assert!(
+        !health.should_skip("local"),
+        "one top-level completion observed twice would already have tripped"
+    );
+    assert!(llmshim::completion(&router, &request).await.is_err());
+    assert!(health.should_skip("local"));
+
+    // Same contract for the streaming door: the verdict is whether it opened.
+    let health = breaker();
+    let router = Router::new()
+        .register(
+            "local",
+            Box::new(OpenAiCompatible::new("local", server.url(), None)),
+        )
+        .with_breaker(health.clone());
+    let mut streaming = request.clone();
+    streaming["stream"] = json!(true);
+    assert!(llmshim::stream(&router, &streaming).await.is_err());
+    assert!(
+        !health.should_skip("local"),
+        "one failed stream open is one observation"
+    );
+    assert!(llmshim::stream(&router, &streaming).await.is_err());
+    assert!(health.should_skip("local"));
+}

{llmshim-0.7.0 → llmshim-0.7.1}/tests/unit_provider_contracts.rs

@@ -217,3 +217,56 @@ fn every_provider_filters_reasoning_and_validates_tools_before_serialization() {
         }
     }
 }
+
+/// Two same-role messages side by side — what a harness produces when it
+/// compacts a window and the summary lands beside a kept assistant turn. Only
+/// a wire that rejects adjacency may fold them (Gemini does); everywhere else
+/// the boundary is the caller's and must survive the transform. The inventory
+/// check above means a new adapter has to declare which side it is on.
+#[test]
+fn same_role_adjacency_is_merged_only_where_the_wire_rejects_it() {
+    let dir = tempfile::tempdir().unwrap();
+    let auth = ChatGptAuth::new(dir.path().join("auth.json"));
+    std::fs::write(auth.auth_path(),json!({"access_token":"test-access","refresh_token":"test-refresh","account_id":"test-account","expires_at":chrono::Utc::now().timestamp()+3600}).to_string()).unwrap();
+    let request = json!({"messages":[
+        {"role":"user","content":"question"},
+        {"role":"assistant","content":"first"},
+        {"role":"assistant","content":"second"},
+        {"role":"user","content":"continue"}]});
+
+    for (name, model, provider) in providers(&auth) {
+        let wire = provider.replay_target(model).wire;
+        let body = provider.transform_request(model, &request).unwrap().body;
+        let (turns, assistant) = match wire {
+            WireFormat::GoogleGenerateContent => (&body["contents"], "model"),
+            WireFormat::OpenAiResponses => (&body["input"], "assistant"),
+            WireFormat::AnthropicMessages | WireFormat::OpenAiChat => {
+                (&body["messages"], "assistant")
+            }
+        };
+        let assistant_turns: Vec<_> = turns
+            .as_array()
+            .unwrap()
+            .iter()
+            .filter(|turn| turn["role"] == assistant)
+            .collect();
+        let text = body.to_string();
+        assert!(
+            text.contains("first") && text.contains("second"),
+            "{name} lost a message's content"
+        );
+        if wire == WireFormat::GoogleGenerateContent {
+            assert_eq!(
+                assistant_turns.len(),
+                1,
+                "{name} must fold adjacent turns; its wire rejects them"
+            );
+        } else {
+            assert_eq!(
+                assistant_turns.len(),
+                2,
+                "{name} must keep the caller's message boundaries; its wire accepts them"
+            );
+        }
+    }
+}