jevper 0.2.2__tar.gz → 0.2.3__tar.gz

{jevper-0.2.2 → jevper-0.2.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: jevper
-Version: 0.2.2
+Version: 0.2.3
 Summary: The Jev (System One) interface over any OpenAI-compatible model: noul, choice and score questions in one call
 Project-URL: Homepage, https://github.com/zhulinchng/jevper
 Project-URL: Repository, https://github.com/zhulinchng/jevper
@@ -249,6 +249,7 @@ LLM_MODEL=gpt-5.6-terra OPENAI_API_KEY=... pytest -q tests/test_live.py
 
 - [docs/api.md](https://github.com/zhulinchng/jevper/blob/main/docs/api.md) — constructor and `system_one` parameters, answer/usage/debug shapes, errors
 - [docs/methods.md](https://github.com/zhulinchng/jevper/blob/main/docs/methods.md) — the four methods, request bodies, readout rules, surface selection
+- [docs/local-servers.md](https://github.com/zhulinchng/jevper/blob/main/docs/local-servers.md) — ollama, llama.cpp, vLLM and SGLang: what to pass, turning thinking off, what fits a small GPU
 - [docs/reasoning.md](https://github.com/zhulinchng/jevper/blob/main/docs/reasoning.md) — native vs two-step reasoning, traces, encrypted content
 - [docs/few-shot.md](https://github.com/zhulinchng/jevper/blob/main/docs/few-shot.md) — example levels, precedence, rendering, structured examples
 - [docs/internals.md](https://github.com/zhulinchng/jevper/blob/main/docs/internals.md) — module map, call flow, concurrency, retries, testing

{jevper-0.2.2 → jevper-0.2.3}/README.md

@@ -220,6 +220,7 @@ LLM_MODEL=gpt-5.6-terra OPENAI_API_KEY=... pytest -q tests/test_live.py
 
 - [docs/api.md](https://github.com/zhulinchng/jevper/blob/main/docs/api.md) — constructor and `system_one` parameters, answer/usage/debug shapes, errors
 - [docs/methods.md](https://github.com/zhulinchng/jevper/blob/main/docs/methods.md) — the four methods, request bodies, readout rules, surface selection
+- [docs/local-servers.md](https://github.com/zhulinchng/jevper/blob/main/docs/local-servers.md) — ollama, llama.cpp, vLLM and SGLang: what to pass, turning thinking off, what fits a small GPU
 - [docs/reasoning.md](https://github.com/zhulinchng/jevper/blob/main/docs/reasoning.md) — native vs two-step reasoning, traces, encrypted content
 - [docs/few-shot.md](https://github.com/zhulinchng/jevper/blob/main/docs/few-shot.md) — example levels, precedence, rendering, structured examples
 - [docs/internals.md](https://github.com/zhulinchng/jevper/blob/main/docs/internals.md) — module map, call flow, concurrency, retries, testing

{jevper-0.2.2 → jevper-0.2.3}/docs/api.md

@@ -26,7 +26,7 @@ caller (`close()` only shuts down jevper's own thread pool).
 | --- | --- | --- |
 | `model` | required | Model id sent with every call; overridable per `system_one` call |
 | `method` | `"auto"` | `"auto"`, `"logprobs"`, `"grammar"`, `"structured"` or `"discrete"`; `"auto"` resolves per model and surface — see [methods.md](methods.md#auto) |
-| `api` | `"auto"` | `"auto"`, `"chat_completions"` or `"responses"` |
+| `api` | `"auto"` | `"auto"`, `"chat_completions"` or `"responses"`; `"auto"` prefers `responses` and falls back to `chat_completions` when the server answers 404 for that route — see [methods.md](methods.md#auto) |
 | `reasoning` | `None` | A `ReasoningConfig`; `None` disables reasoning entirely |
 | `examples` | `()` | Default few-shot examples: a sequence for all questions, or a mapping keyed by question id |
 | `structured_outputs` | `True` | Send a strict `json_schema` response format; `False` falls back to `{"type": "json_object"}` with the schema left in the prompt |
@@ -250,6 +250,17 @@ Catch the public `LabelReadoutError`. `capability=True` from a rejection is reme
 read out of an answer that carried no usable logprobs needs a second one, because a single anomalous response is
 not evidence about the provider — [`internals.md`](internals.md#invariants) has the rule.
 
+The surface has its own private verdict. An `openai` client object exposes `responses.create` whether or not the
+server behind it implements the route — every local server (ollama, llama.cpp, SGLang, vLLM without the route)
+does not — so under `api="auto"` a 404 that does not name the model is read as *this server has no Responses
+route*: the call is re-issued on `chat_completions` and the verdict is remembered for the rest of the client's
+life. A 404 that names the model is the model, and `api="responses"` asked for explicitly is never overridden.
+
+A surface that cannot deliver a distribution — it answered without logprobs, or refused the logprob fields —
+is the same kind of verdict: under `api="auto"` the readout moves to the other surface once, and the surface
+that failed is marked so later calls for that model start where the distribution is. `reasoning="native"` keeps
+the surface it implies, and a provider failure that survived its retries moves nothing.
+
 ## Constants
 
 `Method`, `MethodSelection` and `Api` are `Literal` aliases; the runtime tuples are `jevper.client.METHODS`

{jevper-0.2.2 → jevper-0.2.3}/docs/internals.md

@@ -174,6 +174,17 @@ Worth keeping when editing:
   decides whether to retry. A number the provider did not report stays missing (`TokenLogprob.logprob` is
   `float | None`) rather than defaulting to `0.0`, which would read as certainty, and a non-finite logprob
   raises instead of propagating a `nan` distribution into `confidence` and `score`.
+- The first token a label readout reads is the answer's, not the completion's. When the provider separates its
+  reasoning from the answer *and* the answer text is exactly the tail of the token stream, the answer's own
+  tokens start at that offset (`methods._answer_tokens`); anything else — no trace, no exact tail — is read as
+  it arrives, so a stream that cannot be anchored is reported rather than guessed at.
+- `method="auto"` remembers two kinds of verdict per client, and neither changes a pinned method or a pinned
+  surface: which method works per `(model, surface)`, and which surfaces answer 404. A surface that cannot
+  deliver a distribution — it withheld logprobs, or refused the fields outright — is worth one request on the
+  other surface, and is then marked so later calls start there; the mark is per model, survives the call, and
+  is cleared by a distribution arriving on that surface. The move happens only while the reasoning plan
+  survives it (`native` reasoning exists only on Responses) and never for a provider failure that outlived its
+  retries, which says nothing about the surface.
 - An empty assistant turn is never sent. Two-step analysis output that is blank falls back to the call's
   reasoning text, and if there is none the answer call goes from the question block straight to the cue.
 - Content is rendered with `allow_nan=False`: `NaN`/`Infinity` are not valid JSON, so they raise `JevperError`

jevper-0.2.3/docs/local-servers.md

@@ -0,0 +1,94 @@
+# Local servers
+
+jevper talks to anything that speaks the OpenAI wire format, so a local server is the same client with a
+different `base_url`. Three things differ between servers, and all three are decided by what you pass rather
+than by the library: **which surface carries logprobs**, **how thinking is turned off**, and **which request
+fields the server quietly ignores**. This page is the short version, checked against ollama 0.34, llama.cpp
+b11139, vLLM 0.30 and SGLang 0.5.20 serving `Qwen3.5-9B` at 4-bit on one 12 GB card.
+
+```python
+from openai import OpenAI
+from jevper import Choice, SystemOneClient
+
+client = SystemOneClient(
+    OpenAI(base_url="http://127.0.0.1:11434/v1", api_key="local"),
+    model="qwen3.5:9b",
+    extra_body={"reasoning_effort": "none"},  # thinking off — see below
+)
+
+response = client.system_one(
+    state="I was charged twice for the same subscription this month.",
+    questions={
+        "intent": Choice(
+            criteria={
+                "billing": "money, invoices, refunds, charges",
+                "technical": "errors, crashes, login or performance problems",
+                "sales": "pricing, plans, purchasing, upgrades",
+            }
+        )
+    },
+)
+
+response.answers["intent"].probabilities  # a real distribution, read from the server's logprobs
+```
+
+## What to pass per server
+
+| Server | `base_url` | `model` | Thinking off | Notes |
+| --- | --- | --- | --- | --- |
+| ollama | `http://127.0.0.1:11434/v1` | the tag you pulled, e.g. `qwen3.5:9b` | `extra_body={"reasoning_effort": "none"}` | Chat Completions carries logprobs; the Responses route exists but returns an empty logprob list |
+| llama.cpp | `http://127.0.0.1:8080/v1` | the `--alias` value | `extra_body={"chat_template_kwargs": {"enable_thinking": False}}` | serve with `--jinja` for the model's own template; `grammar` is llama.cpp-only |
+| vLLM | `http://127.0.0.1:8000/v1` | the `--served-model-name` value | `extra_body={"chat_template_kwargs": {"enable_thinking": False}}` | serve with `--reasoning-parser qwen3`; `top_logprobs` is capped by `--max-logprobs` (20) |
+| SGLang | `http://127.0.0.1:30000/v1` | the `--served-model-name` value | `extra_body={"chat_template_kwargs": {"enable_thinking": False}}` | serve with `--reasoning-parser qwen3`; its Responses route needs `top_logprobs` sent explicitly, which jevper always does |
+
+`api="auto"` (the default) works against all four: it prefers the Responses surface, and when that route is
+missing — or answers without carrying logprobs through — it re-asks on Chat Completions and remembers the
+verdict. Passing `api="chat_completions"` skips the discovery entirely.
+
+## Thinking is the one decision you must make
+
+`logprobs`, `grammar` and `discrete` all read a **one-token answer**, and every one of these servers reports
+logprobs for *every* generated token. With thinking on, that is the first token of the reasoning, not the
+label: the readout raises `LabelReadoutError`, spends one corrective retry, and raises again. Turn thinking
+off for classification work — it costs a whole reasoning pass to choose one letter.
+
+| Server | Request field that turns it off | Also available |
+| --- | --- | --- |
+| ollama | `reasoning_effort: "none"` | `OLLAMA_CONTEXT_LENGTH`, per-model `think` on the native API |
+| llama.cpp | `chat_template_kwargs: {"enable_thinking": false}` | `reasoning_budget: 0`; `--reasoning-format deepseek` splits the trace into `reasoning_content` |
+| vLLM | `chat_template_kwargs: {"enable_thinking": false}` | `reasoning_effort` |
+| SGLang | `chat_template_kwargs: {"enable_thinking": false}` | `reasoning_effort` |
+
+If you keep thinking on, `structured` and `discrete` are unaffected — they read the answer text, and the
+trace lands in `response.reasoning` (`message.reasoning` on ollama, `reasoning_content` on the others;
+jevper reads all three). A label readout also survives it when the server separates the trace *and* the token
+stream ends exactly with the answer text: jevper anchors on that tail and reads the answer's own first token.
+That anchor is deliberately strict — anything else is reported rather than guessed.
+
+## What each server ignores
+
+Unknown fields are accepted and dropped by all four, so a field that does not apply is not an error:
+
+- `grammar` (jevper's `method="grammar"`) is a llama.cpp convention. ollama, vLLM and SGLang ignore it, so
+  the model answers unconstrained and the label readout reports a non-label first token instead of a grammar
+  failure. Use `logprobs` or `structured` there.
+- `strict: true` inside `json_schema` is ignored by ollama and honoured by vLLM and SGLang.
+- `reasoning_effort` is a no-op on llama.cpp (it has no such field) and is accepted by the others.
+
+## Sizing a 12 GB card
+
+A 9B model at 4-bit is 5.5-8.5 GB of weights, which leaves room for a small KV cache but not much else. What
+worked here, one server at a time (they cannot share the card):
+
+| Server | Weights | Serve flags that fit |
+| --- | --- | --- |
+| ollama | `qwen3.5:9b` (Q4_K_M, 6.1 GiB) | `OLLAMA_CONTEXT_LENGTH=4096`, `OLLAMA_MAX_LOADED_MODELS=1`, `OLLAMA_NUM_PARALLEL=1` |
+| llama.cpp | Q4_K_M GGUF (5.75 GiB) | `-ngl 99 --ctx-size 4096 -np 1 --jinja` |
+| vLLM | 4-bit compressed-tensors AWQ (8.45 GiB) | `--language-model-only --gpu-memory-utilization 0.80 --max-model-len 1024 --max-num-seqs 1 --enforce-eager` — dropping the vision tower is what makes it fit: without it the engine dies at init, with it the server came up in 43 s and gave 1.88 GiB of KV cache |
+| SGLang | same 8.45 GiB checkpoint | `--mem-fraction-static 0.85 --context-length 4096 --attention-backend triton --sampling-backend pytorch --disable-cuda-graph` — the weights alone exceed 0.80 of the card (SGLang's own guard says so), and FlashInfer's JIT cannot build here, so SGLang's Triton kernels are required; unlike vLLM it has no `--language-model-only` for this architecture, so the vision tower cannot be dropped |
+
+Neither builder sends `max_tokens`, so a chatty model can generate far more than the answer needs. Bound it
+per call with `extra_body={"max_tokens": 512}` — the one field worth setting on a small card. With thinking
+left on, that cap is spent on the reasoning first: ollama answered a 512-token cap with an empty `content` and
+`finish_reason: "length"`, which reaches the caller as a malformed answer. Disable thinking *and* bound the
+output.

{jevper-0.2.2 → jevper-0.2.3}/docs/methods.md

@@ -86,8 +86,10 @@ Provider support, as of this release — check your provider's docs, since this
 | Gemini native API | yes | not reachable through an OpenAI-compatible client |
 | DeepSeek | yes | `top_logprobs` up to 20 |
 | Together | yes | send `top_logprobs` for alternatives; `logprobs: 1` alone returns the sampled token |
-| llama.cpp, vLLM | yes | vLLM caps `top_logprobs` at its own `--max-logprobs` |
-| Ollama | partial | local builds since Nov 2025 return logprobs; Ollama Cloud and older builds do not, and the compatibility page still lists Logprobs as unsupported |
+| llama.cpp | yes | Chat Completions only: its `/v1/responses` shim rejects the logprob fields (`400 top_logprobs requires logprobs to be set to true`), so `auto` re-asks on Chat Completions |
+| vLLM | yes | caps `top_logprobs` at its own `--max-logprobs` (20 by default); `/v1/responses` carries them through `include` |
+| SGLang | yes | its `/v1/responses` needs `top_logprobs` sent explicitly (it defaults to 0) — jevper always sends it |
+| Ollama | partial | local builds since Nov 2025 return logprobs on Chat Completions; its `/v1/responses` returns an empty logprob list, so `auto` re-asks on Chat Completions. Ollama Cloud and older builds report none at all |
 | OpenRouter | per model | it routes by price and, by default, sends your request to an endpoint that may ignore `logprobs` — the answer comes back with none, which `auto` reads as "no logprobs" and falls back on. Add `extra_body={"provider": {"require_parameters": True}}` to route only to endpoints that support every field you send. Its Responses API rejects the logprob includable outright (`400 Invalid option: expected one of …` at `path: ["include", 0]`), so `api="auto"` there always resolves to `structured` |
 | everything else | unknown | reasoning models and thin compatibility layers are the ones that say no |
 
@@ -105,12 +107,33 @@ flowchart TD
     E -->|"no"| F{"client has responses.create"}
     F -->|"yes"| C
     F -->|"no"| D
+    C -->|"404 that does not name the model"| D
 ```
 
 `api="auto"` (the default) prefers the Responses surface because it carries native reasoning and encrypted
 content, except for `grammar`, which only Chat Completions can carry. A missing attribute raises
 `ClientCapabilityError` naming the surface to pass explicitly.
 
+A client object cannot tell you whether the *server* implements the route: `openai.OpenAI` exposes
+`responses.create` either way, so a server that does not implement it answers 404 for that call. Under `auto`
+that 404 is read as "no Responses surface here" — unless the error names the model, which would fail the same
+way on either surface — and the call is re-issued on `chat_completions` and remembered for the rest of the
+client's life. An explicit `api="responses"` is a decision, not a preference: its 404 reaches you unchanged.
+
+The preference has one exception, and it is about the readout rather than the surface. A server can implement
+the Responses route and still not carry logprobs through it: ollama answers it with an empty logprob list,
+llama.cpp refuses the logprob fields there outright (`400 top_logprobs requires logprobs to be set to true`),
+and OpenAI's Responses logprobs hold the sampled token with no alternatives. When the label readout cannot
+produce a distribution on the chosen surface — for any reason other than a provider failure that survived its
+retries — `auto` re-asks on the other surface, marks the one that failed so later calls start where the
+distribution is, and keeps that mark for the model. Two things stop the move: `reasoning="native"`, because
+native reasoning is the reason to prefer Responses and switching would silently turn it into a two-step pass,
+and a server whose other route is missing or already known to be missing. A distribution arriving later on a
+marked surface clears the mark: the verdict moves the readout, it does not condemn the surface.
+
+For the four servers this was checked against — what to pass, how to turn thinking off, and what fits a 12 GB
+card — see [local-servers.md](local-servers.md).
+
 Request fields per surface:
 
 | | Chat Completions | Responses |
@@ -133,7 +156,13 @@ allowed). On the Responses surface the logprobs arrive through `include=["messag
 
 Readout:
 
-1. The first non-whitespace token of the answer must be a label (compared case-insensitively).
+1. The first non-whitespace token of the answer must be a label (compared case-insensitively). A reasoning
+   server reports logprobs for every generated token — vLLM, SGLang and ollama include the thinking span while
+   `message.content` holds only the answer — so the answer's own tokens are located first, by matching the
+   answer text against the tail of the token stream. The match is strict: without a separated trace, or without
+   an exact tail, nothing is skipped and the stream is read as it arrives, so a mismatch is an error rather
+   than a guess. Disabling thinking is still the better deployment — a one-token answer does not need a
+   reasoning pass.
 2. Its logprob is taken from the token itself, and the rest of the distribution from the token's
    `top_logprobs` entries. A label the provider did not report gets probability exactly `0.0` and is listed in
    `debug["labels_missing"]`. A provider that reports `logprob: null` — some OpenAI-compatible servers do — is

{jevper-0.2.2 → jevper-0.2.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "jevper"
-version = "0.2.2"
+version = "0.2.3"
 description = "The Jev (System One) interface over any OpenAI-compatible model: noul, choice and score questions in one call"
 readme = "README.md"
 requires-python = ">=3.10"

{jevper-0.2.2 → jevper-0.2.3}/src/jevper/__init__.py

@@ -42,7 +42,7 @@ from .types import (
     Usage,
 )
 
-__version__ = "0.2.2"
+__version__ = "0.2.3"
 
 __all__ = [
     "Answer",

{jevper-0.2.2 → jevper-0.2.3}/src/jevper/client.py

@@ -26,6 +26,7 @@ from .errors import (
     MalformedAnswerError,
     ProviderError,
     _LogprobsUnavailable,
+    _SurfaceUnavailable,
 )
 from .labels import MAX_LABEL_OPTIONS, labels_for
 from .normalize import (
@@ -55,10 +56,12 @@ from .reasoning import (
     resolve_reasoning_mode,
 )
 from .transport import (
+    SURFACES,
     CallResult,
     CallSpec,
     Surface,
     Transport,
+    _has_attribute,
     make_transport,
     select_surface,
 )
@@ -100,6 +103,11 @@ _VALUE_MARKERS = ("must be between", "out of range", "maximum", "max_logprobs",
 # expected one of "file_search_call.results"|...`` for ``path: ["include", 0]``. This vocabulary is
 # only read next to ``include``, where it is the refusal of the field rather than of a value.
 _INCLUDE_REJECTION_MARKERS = ("invalid option", "invalid_value", "expected one of")
+# A 404 that names the model *and* talks about a model is about the model: the other surface would
+# answer the same way, so switching would only hide the real problem. Both halves are required —
+# an unrelated message can easily contain a short model name, and a server that does not implement
+# the Responses route answers 404 there with a message about the route.
+_MODEL_404_MARKERS = ("model", "no such", "not exist")
 _TRANSIENT_NAME_MARKERS = ("Connection", "Timeout")
 _TRANSIENT_TRANSPORT_CLASSES = frozenset({"TransportError", "TimeoutException"})
 METHODS: tuple[Method, ...] = ("logprobs", "grammar", "structured", "discrete")
@@ -175,7 +183,7 @@ def _require_top_logprobs(method: Method, top_logprobs: int) -> None:
         )
 
 
-def _logprob_evidence(exc: BaseException) -> str:
+def _error_evidence(exc: BaseException) -> str:
     """Everything the provider said about the failure: its message, and the param/code it named."""
     parts = [str(exc)]
     for attr in ("param", "code"):
@@ -185,6 +193,20 @@ def _logprob_evidence(exc: BaseException) -> str:
     return " ".join(parts).lower()
 
 
+def _route_missing(exc: BaseException, *, surface: Surface, model: str) -> bool:
+    """The server has no route for this surface, rather than a problem with what was sent.
+
+    An OpenAI-compatible server that does not implement a surface's route answers 404 for it, and the
+    ``openai`` client object exposes ``responses.create`` either way. A 404 that names the model is
+    about the model — the same error would come back from the other surface — so it is left alone.
+    """
+    if _status_code(exc) != 404:
+        return False
+    evidence = _error_evidence(exc)
+    names_the_model = model.lower() in evidence
+    return not (names_the_model and any(marker in evidence for marker in _MODEL_404_MARKERS))
+
+
 def _include_refused(evidence: str) -> bool:
     """The provider refused the Responses surface's logprob carrier, the ``include`` entry.
 
@@ -208,7 +230,7 @@ def _logprobs_rejected(exc: BaseException) -> bool:
     """
     if _status_code(exc) not in _LOGPROB_REJECTION_STATUS_CODES:
         return False
-    evidence = _logprob_evidence(exc)
+    evidence = _error_evidence(exc)
     return "logprob" in evidence or _include_refused(evidence)
 
 
@@ -222,7 +244,7 @@ def _logprobs_unsupported(exc: BaseException) -> bool:
     """
     if not _logprobs_rejected(exc):
         return False
-    evidence = _logprob_evidence(exc)
+    evidence = _error_evidence(exc)
     if any(marker in evidence for marker in _VALUE_MARKERS):
         return False
     if _include_refused(evidence):
@@ -231,12 +253,23 @@ def _logprobs_unsupported(exc: BaseException) -> bool:
 
 
 def _dump_model(obj: Any) -> Any:
+    """The provider object as plain data for ``debug``, without the provider's typing noise.
+
+    A server can put a value the SDK's model does not expect in a field it still has to serialize —
+    SGLang returns a list for ``metadata``, which the SDK types as a string — and pydantic answers
+    every ``model_dump`` with a ``PydanticSerializationUnexpectedValue`` warning. The value survives
+    the dump either way, and the dump exists for ``debug``, so the warning is noise the caller never
+    asked for; ``warnings=False`` keeps it out of their logs without touching global warning state.
+    """
     dump = getattr(obj, "model_dump", None)
     if callable(dump):
         try:
-            return dump(mode="json")
-        except TypeError:  # pragma: no cover - non-pydantic objects with a model_dump()
-            return dump()
+            return dump(mode="json", warnings=False)
+        except TypeError:  # pragma: no cover - a model_dump() without those keywords
+            try:
+                return dump(mode="json")
+            except TypeError:
+                return dump()
     return obj
 
 
@@ -254,6 +287,8 @@ class _CallContext:
     model: str
     method: Method
     mode: str
+    reasoning: ReasoningConfig | None
+    """The effective config, kept so a surface switch can re-derive ``mode`` for the new surface."""
     temperature: float | None
     examples: Examples
     state_messages: tuple[dict[str, str], ...]
@@ -262,6 +297,9 @@ class _CallContext:
     auto: bool = False
     """``method="auto"``: choose per question, and fall back when logprobs are unavailable. ``method``
     then holds the resolved default rather than what the caller asked for."""
+    api_auto: bool = False
+    """``api="auto"``: the surface was chosen here, so this call may re-choose it. An explicit
+    ``api="responses"`` is a decision: its 404 belongs to the caller, not to a fallback."""
 
 
 @dataclass
@@ -389,6 +427,8 @@ class _BaseClient:
         # What method="auto" has learned about this provider, per (model, surface).
         self._auto_methods: dict[tuple[str, Surface], Method] = {}
         self._auto_misses: dict[tuple[str, Surface], int] = {}
+        # Surfaces this server has answered 404 for, learned by trying them once each.
+        self._missing_surfaces: set[Surface] = set()
         self._auto_lock = threading.Lock()
 
     # -- shared helpers ----------------------------------------------------------------
@@ -430,9 +470,84 @@ class _BaseClient:
             return misses
 
     def _note_logprobs_present(self, model: str, surface: Surface) -> None:
-        """A readable distribution retires the absences counted so far."""
+        """A readable distribution retires the absences counted so far, and any verdict against it."""
         with self._auto_lock:
             self._auto_misses.pop((model, surface), None)
+            # A distribution that arrived here proves this surface can carry one, so a verdict that
+            # moved the readout away from it is stale.
+            if self._auto_methods.get((model, surface)) == FALLBACK_METHOD:
+                self._auto_methods.pop((model, surface), None)
+
+    def _surface_missing(self, surface: Surface) -> bool:
+        """Whether this server has already answered 404 for a surface's route."""
+        with self._auto_lock:
+            return surface in self._missing_surfaces
+
+    def _remember_surface_missing(self, surface: Surface) -> None:
+        """Remember that this server has no route for a surface, for the rest of this client's life."""
+        with self._auto_lock:
+            self._missing_surfaces.add(surface)
+
+    def _logprobs_absent_here(self, model: str, surface: Surface) -> bool:
+        """Whether the label readout is already known to have no future on this surface."""
+        with self._auto_lock:
+            return self._auto_methods.get((model, surface), AUTO_METHOD) == FALLBACK_METHOD
+
+    def _logprob_surface_alternative(
+        self, surface: Surface, reasoning: ReasoningConfig | None
+    ) -> Surface | None:
+        """The other surface to try for a label readout, when the move is free of side effects.
+
+        ``native`` reasoning exists only on the Responses surface, so a caller who asked for it — or
+        whose ``mode="auto"`` resolved to it — keeps the surface that choice implies. Otherwise the
+        move costs nothing but a request, and it is worth one: a server can implement a surface
+        without carrying logprobs through it — ollama answers the Responses route with an empty
+        logprob list, llama.cpp rejects the logprob fields there outright, and OpenAI's Responses
+        logprobs hold the sampled token and no alternatives — while Chat Completions on the same
+        server carries a full distribution.
+        """
+        other: Surface = "chat_completions" if surface == "responses" else "responses"
+        if self._surface_missing(other) or resolve_reasoning_mode(reasoning, surface) == "native":
+            return None
+        return other if _has_attribute(self.client, f"{SURFACES[other][2]}.create") else None
+
+    def _note_surface_absence(
+        self, model: str, surface: Surface, exc: _LogprobsUnavailable
+    ) -> None:
+        """Count one absence for this surface, remembering the verdict once it is a pattern."""
+        if exc.capability and (
+            exc.evidence == "provider"
+            or self._note_absent_logprobs(model, surface) >= AUTO_ABSENCES_BEFORE_REMEMBERING
+        ):
+            self._remember_logprobs_unavailable(model, surface)
+
+    def _remember_responses_missing(self) -> None:
+        """Remember that this server has no Responses route, for the rest of this client's life."""
+        with self._auto_lock:
+            self._responses_missing = True
+
+    def _switch_surface(self, context: _CallContext, surface: Surface) -> None:
+        """Rebuild the transport and the reasoning plan for another surface, in place.
+
+        ``_assemble`` reports the surface from the context it was handed, and the driver reads the
+        transport per call, so the switch has to be visible on the object they already hold.
+        """
+        context.transport = make_transport(
+            self.client,
+            surface,
+            structured_outputs=self.structured_outputs,
+            extra_body=self.extra_body,
+            extra_headers=self.extra_headers,
+        )
+        if context.auto:
+            # The method verdict is keyed by surface, so the surface that just changed has its own:
+            # coming back to one that is known to withhold logprobs must not ask for them again.
+            context.method = self._auto_method(context.model, surface)
+        context.mode = resolve_reasoning_mode(context.reasoning, surface)
+        context.answer_reasoning = context.reasoning if context.mode == "native" else None
+        context.analysis_reasoning = (
+            context.reasoning if context.mode == "two_step" and surface == "responses" else None
+        )
 
     def _call_failure(
         self, exc: Exception, spec: CallSpec, context: _CallContext, log: _CallLog
@@ -459,6 +574,9 @@ class _BaseClient:
                 return _LogprobsUnavailable(
                     f"the provider failed every attempt at the logprob request ({failure})",
                     capability=False,
+                    # Not evidence about the surface: another surface would have failed too, so this
+                    # must not move the label readout anywhere.
+                    evidence="transient",
                 )
         if isinstance(exc, ProviderError):
             # The transport already built the right error — an embedded provider failure — so keep
@@ -503,18 +621,30 @@ class _BaseClient:
         if requested not in METHOD_SELECTIONS:
             raise JevperError(f"method must be one of {METHOD_SELECTIONS!r}, got {requested!r}")
         auto = requested == "auto"
+        api_auto = (api or self.api) == "auto"
+        effective_model = model or self.model
+        effective_reasoning = reasoning if reasoning is not None else self.reasoning
         # The surface is picked for the method auto tries first; the fallback runs on either surface.
         surface = select_surface(self.client, api or self.api, AUTO_METHOD if auto else requested)
+        if api_auto and self._surface_missing(surface):
+            # This server answered 404 for the route before: do not pay for the discovery again.
+            surface = "chat_completions" if surface == "responses" else "responses"
+        elif api_auto and auto and self._logprobs_absent_here(effective_model, surface):
+            alternative = self._logprob_surface_alternative(surface, effective_reasoning)
+            if alternative is not None and not self._logprobs_absent_here(
+                effective_model, alternative
+            ):
+                # The label readout has no future on this surface and the other one is unmarked:
+                # start there instead of paying for the same discovery on every call.
+                surface = alternative
         if requested == "grammar":
             methods.require_grammar_surface(surface)
-        effective_model = model or self.model
         effective_method = self._auto_method(effective_model, surface) if auto else requested
         if not auto:
             # auto never reaches the label-readout cap: a wide Choice is answered in JSON.
             for question_id, question in parsed.items():
                 methods.require_label_readout(effective_method, question, question_id)
             _require_top_logprobs(effective_method, self.top_logprobs)
-        effective_reasoning = reasoning if reasoning is not None else self.reasoning
         mode = resolve_reasoning_mode(effective_reasoning, surface)
         context = _CallContext(
             transport=make_transport(
@@ -527,6 +657,7 @@ class _BaseClient:
             model=effective_model,
             method=effective_method,
             mode=mode,
+            reasoning=effective_reasoning,
             temperature=temperature if temperature is not None else self.temperature,
             examples=examples,
             state_messages=state_messages,
@@ -535,6 +666,7 @@ class _BaseClient:
                 effective_reasoning if mode == "two_step" and surface == "responses" else None
             ),
             auto=auto,
+            api_auto=api_auto,
         )
         return context, parsed
 
@@ -580,16 +712,40 @@ class _BaseClient:
             except _LogprobsUnavailable as exc:
                 if not context.auto or fell_back:
                     raise
+                if exc.evidence != "transient" and context.api_auto:
+                    alternative = self._logprob_surface_alternative(
+                        context.transport.surface, context.reasoning
+                    )
+                    if alternative is not None and not self._logprobs_absent_here(
+                        context.model, alternative
+                    ):
+                        # The verdict is about this surface — it answered without logprobs, or refused
+                        # the fields outright, as llama.cpp's Responses shim does. Mark it, so later
+                        # calls start where the distribution is, and answer this question there.
+                        self._remember_logprobs_unavailable(context.model, context.transport.surface)
+                        retry_reasons.append(
+                            f"{exc} — retrying the label readout on api={alternative!r}"
+                        )
+                        self._switch_surface(context, alternative)
+                        continue
                 fell_back = True
                 retry_reasons.append(f"{exc} — answering with method={FALLBACK_METHOD!r}")
-                if exc.capability and (
-                    exc.evidence == "provider"
-                    or self._note_absent_logprobs(context.model, context.transport.surface)
-                    >= AUTO_ABSENCES_BEFORE_REMEMBERING
-                ):
-                    self._remember_logprobs_unavailable(context.model, context.transport.surface)
+                self._note_surface_absence(context.model, context.transport.surface, exc)
                 # Questions that have not started yet take the fallback without paying for it.
                 context.method = FALLBACK_METHOD
+            except _SurfaceUnavailable as exc:
+                if not context.api_auto:
+                    raise
+                self._remember_surface_missing(context.transport.surface)
+                other: Surface = (
+                    "chat_completions" if context.transport.surface == "responses" else "responses"
+                )
+                if self._surface_missing(other) or not _has_attribute(
+                    self.client, f"{SURFACES[other][2]}.create"
+                ):
+                    raise  # nowhere left to go: this 404 is the answer
+                retry_reasons.append(f"{exc} — answering on api={other!r}")
+                self._switch_surface(context, other)
 
     def _answer_steps(
         self,
@@ -814,7 +970,7 @@ class SystemOneClient(_BaseClient):
             while True:
                 try:
                     result = self._call(log, question_id, spec, context)
-                except _LogprobsUnavailable as exc:
+                except (_LogprobsUnavailable, _SurfaceUnavailable) as exc:
                     # Hand the verdict back to the generator: it owns the fallback decision.
                     spec = steps.throw(exc)
                 else:
@@ -835,6 +991,15 @@ class SystemOneClient(_BaseClient):
                 ):
                     if isinstance(exc, ClientCapabilityError):
                         raise  # the client cannot read this surface; another attempt cannot help
+                    if context.api_auto and _route_missing(
+                        exc, surface=context.transport.surface, model=context.model
+                    ):
+                        # A server with no such route: hand the verdict to the generator, which owns
+                        # the fallback decision, exactly as with an unavailable logprob readout.
+                        raise _SurfaceUnavailable(
+                            f"the server has no {context.transport.surface!r} route: {exc}",
+                            surface=context.transport.surface,
+                        ) from exc
                     failure = self._call_failure(exc, spec, context, log)
                     # An embedded provider error is the caught exception itself; raising it `from`
                     # itself would print as its own cause.
@@ -922,7 +1087,7 @@ class AsyncSystemOneClient(_BaseClient):
             while True:
                 try:
                     result = await self._call(log, question_id, spec, context)
-                except _LogprobsUnavailable as exc:
+                except (_LogprobsUnavailable, _SurfaceUnavailable) as exc:
                     # Hand the verdict back to the generator: it owns the fallback decision.
                     spec = steps.throw(exc)
                 else:
@@ -943,6 +1108,15 @@ class AsyncSystemOneClient(_BaseClient):
                 ):
                     if isinstance(exc, ClientCapabilityError):
                         raise  # the client cannot read this surface; another attempt cannot help
+                    if context.api_auto and _route_missing(
+                        exc, surface=context.transport.surface, model=context.model
+                    ):
+                        # A server with no such route: hand the verdict to the generator, which owns
+                        # the fallback decision, exactly as with an unavailable logprob readout.
+                        raise _SurfaceUnavailable(
+                            f"the server has no {context.transport.surface!r} route: {exc}",
+                            surface=context.transport.surface,
+                        ) from exc
                     failure = self._call_failure(exc, spec, context, log)
                     # An embedded provider error is the caught exception itself; raising it `from`
                     # itself would print as its own cause.

{jevper-0.2.2 → jevper-0.2.3}/src/jevper/errors.py

@@ -56,6 +56,20 @@ class MalformedAnswerError(JevperError):
     """The model's answer had an unusable shape after corrective retries."""
 
 
+class _SurfaceUnavailable(JevperError):
+    """The server does not implement the surface ``api="auto"`` picked.
+
+    An ``openai`` client object always exposes ``chat.completions.create`` *and* ``responses.create``, so
+    client-side detection cannot tell a server that implements the Responses API from one that answers 404
+    for the route. The verdict travels to the fallback the way ``_LogprobsUnavailable`` does, and is
+    remembered for the rest of the client's life.
+    """
+
+    def __init__(self, message: str, *, surface: str) -> None:
+        super().__init__(message)
+        self.surface = surface
+
+
 class ProviderError(JevperError):
     """A provider call failed after transient retries were exhausted.
 

{jevper-0.2.2 → jevper-0.2.3}/src/jevper/methods.py

@@ -183,8 +183,34 @@ def softmax_over_labels(values: Mapping[str, float]) -> dict[str, float]:
     return {key: weights.get(key, 0.0) / total for key in values}
 
 
+def _answer_tokens(result: CallResult) -> tuple[TokenLogprob, ...]:
+    """The tokens of the answer itself, when the provider separated its reasoning from the answer.
+
+    A reasoning server reports logprobs for every generated token — vLLM, SGLang and ollama include
+    the thinking span, while ``message.content`` holds only the answer — so the first token of the
+    stream is the first token of the *thinking*, which is never a label. The answer text is the
+    anchor: when it is exactly the tail of the token stream, the token covering its first character
+    is the answer's first token. The check is strict on purpose — no separated trace, or no exact
+    tail, means nothing is skipped and the stream is read as it arrives.
+    """
+    tokens = result.token_logprobs
+    content = result.text
+    if not result.reasoning or not content.strip():
+        return tokens
+    stream = "".join(token.token for token in tokens)
+    if len(content) >= len(stream) or not stream.endswith(content):
+        return tokens
+    start = len(stream) - len(content)
+    offset = 0
+    for index, token in enumerate(tokens):
+        offset += len(token.token)
+        if offset > start:
+            return tokens[index:]
+    return tokens
+
+
 def first_answer_token(result: CallResult, labels: Sequence[str], *, method: Method) -> TokenLogprob:
-    """The first non-whitespace token, which must be one of the labels."""
+    """The first non-whitespace token of the answer, which must be one of the labels."""
     if not result.token_logprobs:
         raise _LogprobsUnavailable(
             f"no logprobs returned for the answer token (method={method!r}); this provider does not "
@@ -192,7 +218,7 @@ def first_answer_token(result: CallResult, labels: Sequence[str], *, method: Met
             f"method='discrete' for one label, neither of which needs logprobs",
             evidence="readout",
         )
-    for token in result.token_logprobs:
+    for token in _answer_tokens(result):
         if not token.token.strip():
             continue
         if token.token.strip().upper() in labels:

{jevper-0.2.2 → jevper-0.2.3}/tests/fakes.py

@@ -21,18 +21,21 @@ def chat_body(
     *,
     content: str,
     logprobs: Sequence[tuple[str, float]] | None = None,
+    alternatives: Sequence[tuple[str, float]] | None = None,
     reasoning: str | None = None,
     input_tokens: int | None = 10,
     output_tokens: int | None = 3,
     reasoning_tokens: int | None = 0,
 ) -> dict[str, Any]:
     """A ``chat.completion`` body. ``logprobs`` is the generated token stream for the answer, first
-    entry first; the answer token's ``top_logprobs`` is the whole sequence."""
+    entry first; the answer token's ``top_logprobs`` is the whole sequence unless ``alternatives``
+    says otherwise — a stream that carries reasoning tokens needs the distribution spelled out."""
     message: dict[str, Any] = {"role": "assistant", "content": content}
     if reasoning is not None:
         message["reasoning_content"] = reasoning
     choice: dict[str, Any] = {"index": 0, "finish_reason": "stop", "message": message}
     if logprobs is not None:
+        distribution = logprobs if alternatives is None else alternatives
         choice["logprobs"] = {
             "content": [
                 {
@@ -41,7 +44,7 @@ def chat_body(
                     "bytes": list(token.encode()),
                     "top_logprobs": [
                         {"token": name, "logprob": value, "bytes": list(name.encode())}
-                        for name, value in logprobs
+                        for name, value in distribution
                     ],
                 }
                 for token, logprob in logprobs

{jevper-0.2.2 → jevper-0.2.3}/tests/test_auto_method.py

@@ -266,6 +266,111 @@ def test_auto_falls_back_when_a_provider_refuses_the_include_path(stub_server):
     assert stub.bodies("/responses")[2].get("include") is None
 
 
+def test_a_rejection_moves_the_readout_to_the_other_surface(stub_server):
+    """llama.cpp's Responses shim refuses the logprob fields; Chat Completions carries them.
+
+    A refusal is about the surface that made it — the same question on the other surface is worth one
+    request, because that is where the distribution is. llama.cpp answers
+    ``400 top_logprobs requires logprobs to be set to true`` on ``/v1/responses`` and a full
+    distribution on ``/v1/chat/completions``.
+    """
+    stub = stub_server(
+        chat=lambda _: (200, chat_body(content="A", logprobs=CHOICE_LOGS)),
+        responses=lambda body: (400, INCLUDE_REJECTION)
+        if body.get("include")
+        else (200, responses_body(text=STRUCTURED)),
+    )
+    client = SystemOneClient(openai_client(stub), model="stub")  # api="auto" prefers responses
+
+    response = client.system_one(state="s", questions={"q": Choice(criteria=CRITERIA)})
+
+    assert response.debug["api"] == "chat_completions"
+    assert response.debug["methods"] == {"q": "logprobs"}
+    assert stub.paths == ["/v1/responses", "/v1/chat/completions"]
+    assert response.answers["q"].probabilities["billing"] == pytest.approx(0.884873983, abs=1e-9)
+
+
+def test_a_surface_that_delivers_again_is_not_written_off(stub_server):
+    """The verdict moves the readout away; it does not condemn the surface. A distribution clears it."""
+    calls = {"probes": 0}
+
+    def responses_script(body):
+        if body.get("include") or body.get("top_logprobs"):
+            calls["probes"] += 1
+            if calls["probes"] == 1:
+                return 200, responses_body(text="A")  # answered, carrying no logprobs
+            return 200, responses_body(text="A", logprobs=CHOICE_LOGS)  # this time it does
+        return 200, responses_body(text=STRUCTURED)
+
+    stub = stub_server(
+        chat=lambda _: (200, chat_body(content="A", logprobs=CHOICE_LOGS)),
+        responses=responses_script,
+    )
+    client = SystemOneClient(openai_client(stub), model="stub")
+    question = {"q": Choice(criteria=CRITERIA)}
+
+    client.system_one(state="s", questions=question)  # responses withholds: the readout moves to chat
+    assert stub.paths == ["/v1/responses", "/v1/chat/completions"]
+
+    # A pinned call proves the surface can carry a distribution after all, which clears the verdict.
+    client.system_one(state="s", questions=question, api="responses", method="logprobs")
+    assert stub.paths[-1] == "/v1/responses"
+
+    # So the next auto call opens there again, rather than staying away for the client's life — and
+    # it needs only that one request, because the surface now carries the distribution.
+    client.system_one(state="s", questions=question)
+    assert stub.paths[-1] == "/v1/responses"
+    assert stub.paths.count("/v1/responses") == 3
+    assert stub.paths.count("/v1/chat/completions") == 1
+
+
+def test_a_refusal_is_remembered_even_when_the_surface_cannot_move(stub_server):
+    """Native reasoning keeps the surface, so the only verdict left is the provider's own."""
+
+    def script(body):
+        # Native reasoning also sends an `include` (for encrypted content); only the logprob
+        # includable is refused here, which is what the real rejection names.
+        if "message.output_text.logprobs" in (body.get("include") or []):
+            return 400, INCLUDE_REJECTION
+        return 200, responses_body(text=STRUCTURED)
+
+    stub = stub_server(responses=script)
+    client = SystemOneClient(
+        openai_client(stub),
+        model="stub",
+        reasoning=ReasoningConfig(mode="native", effort="low"),
+    )
+    question = {"q": Choice(criteria=CRITERIA)}
+
+    client.system_one(state="s", questions=question)
+    client.system_one(state="s", questions=question)
+
+    # The refusal names the surface's carrier, so it is a capability verdict: no second probe.
+    assert "message.output_text.logprobs" not in (
+        stub.bodies("/responses")[2].get("include") or []
+    )
+    assert stub.paths.count("/v1/responses") == 3  # probe, structured, structured
+
+
+def test_coming_back_to_a_marked_surface_does_not_ask_for_logprobs_again(stub_server):
+    """The method verdict is keyed by surface: returning to one that withheld logprobs stays in JSON."""
+
+    def responses_script(body):
+        if body.get("include") or body.get("top_logprobs"):
+            return 200, responses_body(text="A")  # answered, carrying no logprobs
+        return 200, responses_body(text=STRUCTURED)
+
+    stub = stub_server(responses=responses_script)  # no chat route: the move cannot land
+    client = SystemOneClient(openai_client(stub), model="stub")
+
+    response = client.system_one(state="s", questions={"q": Choice(criteria=CRITERIA)})
+
+    # responses withholds → chat is tried and 404s → back on responses, answered in JSON, not probed.
+    assert stub.paths == ["/v1/responses", "/v1/chat/completions", "/v1/responses"]
+    assert response.debug["api"] == "responses"
+    assert response.answers["q"].choice == "billing"
+
+
 def test_serial_questions_share_the_discovery(stub_server):
     """A question that has not started yet takes the verdict: one probe, not one per question."""
 
@@ -498,3 +603,144 @@ def test_a_readable_distribution_retires_earlier_absences(stub_server):
 
     # Two absences separated by a readable distribution are still only one absence in a row.
     assert stub.bodies("/chat/completions")[5]["logprobs"] is True
+
+
+def test_auto_switches_to_chat_when_the_server_has_no_responses_route(stub_server):
+    """A server with no ``/v1/responses`` route: the SDK object exposes ``responses.create`` regardless.
+
+    The stub answers 404 for a path it has no script for, exactly as such a server does, so the default
+    ``api="auto"`` must notice and answer on the surface the server actually implements.
+    """
+    stub = stub_server(chat=lambda _: (200, chat_body(content="A", logprobs=CHOICE_LOGS)))
+    client = SystemOneClient(openai_client(stub), model="stub")  # api="auto" prefers responses
+
+    response = client.system_one(state="s", questions={"q": Choice(criteria=CRITERIA)})
+
+    assert stub.paths == ["/v1/responses", "/v1/chat/completions"]
+    assert response.debug["api"] == "chat_completions"
+    assert response.answers["q"].choice == "billing"
+    assert response.usage.n_retries == 0  # a surface switch is not a transient retry
+
+
+def test_the_missing_responses_surface_is_remembered(stub_server):
+    """One 404 is enough: later calls on the same client go straight to the surface that works."""
+    stub = stub_server(chat=lambda _: (200, chat_body(content="A", logprobs=CHOICE_LOGS)))
+    client = SystemOneClient(openai_client(stub), model="stub")
+
+    client.system_one(state="s", questions={"q": Choice(criteria=CRITERIA)})
+    client.system_one(state="s", questions={"q": Choice(criteria=CRITERIA)})
+
+    assert stub.paths.count("/v1/responses") == 1
+    assert stub.paths.count("/v1/chat/completions") == 2
+
+
+def test_a_404_that_names_the_model_does_not_switch_surface(stub_server):
+    """The model is missing on either surface, so the provider's own error must reach the caller."""
+
+    def script(_):
+        return 404, {"error": {"message": "The model 'stub' does not exist", "code": "model_not_found"}}
+
+    stub = stub_server(responses=script)
+    client = SystemOneClient(openai_client(stub), model="stub")
+
+    with pytest.raises(ProviderError) as error:
+        client.system_one(state="s", questions={"q": Choice(criteria=CRITERIA)})
+
+    assert "does not exist" in str(error.value)
+    assert stub.paths == ["/v1/responses"]
+
+
+def test_an_explicit_responses_surface_reports_the_404(stub_server):
+    """``api="responses"`` is a decision, not a preference: it is never silently overridden."""
+    stub = stub_server(chat=lambda _: (200, chat_body(content="A", logprobs=CHOICE_LOGS)))
+    client = SystemOneClient(openai_client(stub), model="stub", api="responses")
+
+    with pytest.raises(ProviderError):
+        client.system_one(state="s", questions={"q": Choice(criteria=CRITERIA)})
+
+    assert stub.paths == ["/v1/responses"]
+
+
+def test_switching_surface_re_derives_the_reasoning_mode(stub_server):
+    """``mode="auto"`` is native on Responses and two-step on Chat: the switch must follow the surface."""
+    stub = stub_server(chat=lambda _: (200, chat_body(content="A", logprobs=CHOICE_LOGS)))
+    client = SystemOneClient(
+        openai_client(stub),
+        model="stub",
+        reasoning=ReasoningConfig(effort="medium", mode="auto"),
+    )
+
+    response = client.system_one(state="s", questions={"q": Choice(criteria=CRITERIA)})
+
+    assert stub.paths[0] == "/v1/responses"  # the 404 that teaches the verdict
+    assert response.debug["api"] == "chat_completions"
+    assert response.debug["reasoning_mode"] == "two_step"
+    sent = stub.bodies("/chat/completions")
+    assert len(sent) == 2  # analysis pass, then the answer pass
+    assert all("reasoning_effort" not in body for body in sent)  # two-step asks for no native reasoning
+    assert response.answers["q"].choice == "billing"
+
+
+def test_auto_answers_from_chat_when_the_responses_surface_carries_no_logprobs(stub_server):
+    """ollama and llama.cpp implement ``/v1/responses`` with an empty logprob list; chat has them.
+
+    The surface preference exists for native reasoning, but a label readout with nothing to read is
+    worth one request on the other surface — which on the same server carries a full distribution.
+    """
+    stub = stub_server(
+        chat=lambda _: (200, chat_body(content="A", logprobs=CHOICE_LOGS)),
+        responses=lambda _: (200, responses_body(text="A")),
+    )
+    client = SystemOneClient(openai_client(stub), model="stub")
+
+    response = client.system_one(state="s", questions={"q": Choice(criteria=CRITERIA)})
+
+    assert response.debug["api"] == "chat_completions"
+    assert response.debug["methods"] == {"q": "logprobs"}
+    assert response.answers["q"].probabilities["billing"] == pytest.approx(0.884873983, abs=1e-9)
+    assert stub.paths == ["/v1/responses", "/v1/chat/completions"]
+
+
+def test_native_reasoning_keeps_the_responses_surface(stub_server):
+    """Switching surfaces would silently turn native reasoning into a two-step pass: it stays put."""
+    stub = stub_server(
+        chat=lambda _: (200, chat_body(content="A", logprobs=CHOICE_LOGS)),
+        responses=lambda _: (200, responses_body(text=STRUCTURED)),
+    )
+    client = SystemOneClient(
+        openai_client(stub), model="stub", reasoning=ReasoningConfig(mode="native", effort="low")
+    )
+
+    response = client.system_one(state="s", questions={"q": Choice(criteria=CRITERIA)})
+
+    assert response.debug["api"] == "responses"
+    assert response.debug["reasoning_mode"] == "native"
+    assert response.debug["methods"] == {"q": "structured"}
+    assert stub.paths == ["/v1/responses", "/v1/responses"]
+    assert response.answers["q"].choice == "billing"
+
+
+def test_the_surface_verdict_is_remembered_from_the_first_call(stub_server):
+    """One surface without logprobs is enough to stop opening there: the next call starts elsewhere."""
+    stub = stub_server(
+        chat=lambda _: (200, chat_body(content="A", logprobs=CHOICE_LOGS)),
+        responses=lambda _: (200, responses_body(text="A")),
+    )
+    client = SystemOneClient(openai_client(stub), model="stub")
+
+    for _ in range(3):
+        client.system_one(state="s", questions={"q": Choice(criteria=CRITERIA)})
+
+    assert stub.paths.count("/v1/responses") == 1  # the discovery, paid once
+    assert stub.paths.count("/v1/chat/completions") == 3
+
+
+def test_async_auto_switches_surface_too(stub_server):
+    stub = stub_server(chat=lambda _: (200, chat_body(content="A", logprobs=CHOICE_LOGS)))
+    client = AsyncSystemOneClient(async_openai_client(stub), model="stub")
+
+    response = asyncio.run(client.system_one(state="s", questions={"q": Choice(criteria=CRITERIA)}))
+
+    assert stub.paths == ["/v1/responses", "/v1/chat/completions"]
+    assert response.debug["api"] == "chat_completions"
+    assert response.answers["q"].choice == "billing"

{jevper-0.2.2 → jevper-0.2.3}/tests/test_client_behaviour.py

@@ -6,6 +6,7 @@ import asyncio
 import json
 import threading
 import time
+import warnings
 
 import pytest
 from fakes import (
@@ -795,3 +796,26 @@ def test_a_per_call_method_cannot_ask_for_one_alternative(stub_server):
 
     assert "at least 2" in str(error.value)
     assert stub.requests == []
+
+
+def test_a_provider_field_the_sdk_types_differently_is_dumped_without_noise(stub_server):
+    """SGLang returns a list for ``metadata`` where the SDK declares a string.
+
+    jevper dumps the provider object for ``debug``, so without asking pydantic to keep quiet every
+    call against such a server prints a ``PydanticSerializationUnexpectedValue`` warning the caller
+    never asked for. The value survives the dump either way: only the noise is removed.
+    """
+    body = chat_body(content="A", logprobs=CHOICE_LOGS)
+    body["metadata"] = [{"version": "default", "start": 0, "end": 4}]
+    stub = stub_server(chat=lambda _: (200, body))
+    client = SystemOneClient(openai_client(stub), model="stub", api="chat_completions")
+
+    with warnings.catch_warnings(record=True) as caught:
+        warnings.simplefilter("always")
+        response = client.system_one(state="s", questions={"q": Choice(criteria=CRITERIA)})
+
+    assert [item for item in caught if "PydanticSerializationUnexpectedValue" in str(item.message)] == []
+    assert response.debug["llm_attempts"][-1]["response"]["metadata"] == [
+        {"version": "default", "start": 0, "end": 4}
+    ]
+    assert response.answers["q"].choice == "billing"

{jevper-0.2.2 → jevper-0.2.3}/tests/test_logprobs_methods.py

@@ -119,6 +119,103 @@ def test_non_label_answer_is_retried_then_raises(stub_server):
     assert "A, B" in requests[1]["messages"][-1]["content"]
 
 
+def test_a_separated_reasoning_trace_does_not_hide_the_answer_token(stub_server):
+    """A reasoning server reports logprobs for every generated token, the thinking span included.
+
+    vLLM, SGLang and ollama all do this while separating the trace into ``reasoning_content``, so the
+    first token of the stream is the first token of the *thinking*. The answer text is the anchor: the
+    token that starts its exact tail is the answer's first token, and the distribution is read there.
+    """
+    stream = [
+        ("The", -0.1),
+        (" user", -0.2),
+        (" was", -0.3),
+        (" charged", -0.4),
+        (" twice", -0.5),
+        (".", -0.6),
+        ("A", -0.12),
+    ]
+    body = chat_body(
+        content="A",
+        logprobs=stream,
+        alternatives=[("A", -0.12), ("B", -2.47), ("C", -3.48)],
+        reasoning="The user was charged twice, so this is about money.",
+    )
+    stub = stub_server(chat=lambda _: (200, body))
+    client = SystemOneClient(openai_client(stub), model="stub", api="chat_completions", method="logprobs")
+
+    response = client.system_one(state="s", questions={"q": Choice(criteria={"billing": None, "technical": None, "sales": None})})
+
+    answer = response.answers["q"]
+    assert answer.choice == "billing"
+    assert answer.probabilities["billing"] == pytest.approx(0.884873983, abs=1e-9)
+    assert response.usage.n_calls == 1  # the readout needed no corrective retry
+    assert response.debug["retry_reasons"] == []
+
+
+def test_the_answer_is_the_tail_of_the_stream_not_its_first_mention(stub_server):
+    """The trace weighs the options by name; only the last occurrence of the answer text is the answer."""
+    body = chat_body(
+        content="B",
+        logprobs=[
+            ("The", -0.2),
+            (" user", -0.3),
+            (" paid", -0.4),
+            (" twice", -0.5),
+            (",", -0.6),
+            (" so", -0.7),
+            ("A", -0.8),
+            (" is", -0.9),
+            (" wrong", -1.0),
+            (" and", -1.1),
+            (" B", -1.2),
+            (" fits", -1.3),
+            (".", -1.4),
+            ("B", -0.05),
+        ],
+        alternatives=[("B", -0.05), ("A", -3.0), ("C", -4.0)],
+        reasoning="The user paid twice, so A is wrong and B fits.",
+    )
+    stub = stub_server(chat=lambda _: (200, body))
+    client = SystemOneClient(openai_client(stub), model="stub", api="chat_completions", method="logprobs")
+
+    response = client.system_one(state="s", questions={"q": Choice(criteria={"billing": None, "technical": None, "sales": None})})
+
+    answer = response.answers["q"]
+    assert answer.choice == "technical"  # B, not the A the trace rules out
+    # The distribution is the final token's: the trace's mention of B carries logprob -1.2 there.
+    assert answer.probabilities["technical"] == pytest.approx(0.933188894, abs=1e-9)
+
+
+def test_a_stream_that_does_not_end_with_the_answer_is_read_as_it_arrives(stub_server):
+    """The anchor is exact on purpose: a trailing token after the answer means nothing is skipped.
+
+    Some servers append an end-of-turn token to the logprobs; guessing where the answer ends there
+    would be a guess, so the stream is read as it arrives and the first token is reported as it is.
+    """
+    body = chat_body(
+        content="A",
+        logprobs=[
+            ("The", -0.2),
+            (" user", -0.3),
+            (" paid", -0.4),
+            (" twice", -0.5),
+            (".", -0.6),
+            ("A", -0.12),
+            ("<|im_end|>", -0.01),
+        ],
+        alternatives=[("A", -0.12), ("B", -2.47), ("C", -3.48)],
+        reasoning="The user paid twice.",
+    )
+    stub = stub_server(chat=lambda _: (200, body))
+    client = SystemOneClient(openai_client(stub), model="stub", api="chat_completions", method="logprobs")
+
+    with pytest.raises(LabelReadoutError) as error:
+        client.system_one(state="s", questions={"q": Choice(criteria={"billing": None, "technical": None, "sales": None})})
+
+    assert "first non-whitespace token 'The'" in str(error.value)
+
+
 def test_corrective_retry_succeeds_and_records_reason(stub_server):
     good = chat_body(content="B", logprobs=[("B", -0.05), ("A", -3.0)])
     bad = chat_body(content="maybe", logprobs=[("maybe", -0.1), ("A", -2.0), ("B", -2.5)])