decidr 0.1.0__tar.gz

decidr-0.1.0/.gitignore

@@ -0,0 +1,7 @@
+__pycache__/
+*.py[cod]
+.venv/
+dist/
+build/
+*.egg-info/
+.pytest_cache/

decidr-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Anmol Sharma
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

decidr-0.1.0/PKG-INFO

@@ -0,0 +1,150 @@
+Metadata-Version: 2.5
+Name: decidr
+Version: 0.1.0
+Summary: Typed decisions from local LLMs in one forward pass, via Ollama
+Project-URL: Homepage, https://github.com/devanmolsharma/decidr
+Project-URL: Issues, https://github.com/devanmolsharma/decidr/issues
+License: MIT
+License-File: LICENSE
+Keywords: classification,decisions,llm,logprobs,ollama,routing
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# decidr
+
+**Typed decisions from local LLMs, in one forward pass.**
+
+Most decisions an application asks an LLM to make are small: *route this ticket*, *is this evidence sufficient*, *how angry is this customer*. A chat model can answer them, but it generates a sentence, or JSON, which your code then parses back into an `if` statement.
+
+`decidr` skips that. It gives the model your options, runs **one forward pass**, and reads the probability of each option directly out of the model's own logits. No answer sentence. No JSON to repair. No decoding loop.
+
+```python
+from decidr import Client
+
+client = Client(model="qwen3.5:4b")
+
+decision = client.decide({
+    "id": "route-1",
+    "state": "Customer cannot access an account after a password reset. The reset email never arrived.",
+    "question": "Which queue should handle this request?",
+    "options": [
+        {"id": "access",  "description": "Account access and authentication support."},
+        {"id": "billing", "description": "Billing and payment support."},
+        {"id": "sales",   "description": "Sales and product evaluation."},
+    ],
+})
+
+decision.choice          # 'access'
+decision.confidence      # 0.9999
+decision.probabilities   # {'access': 0.9999, 'billing': 0.0001, 'sales': 0.0}
+decision.is_reliable()   # True
+```
+
+Works with any model you already have in [Ollama](https://ollama.com). No fine-tuning, no extra runtime, no separate model to download.
+
+## Install
+
+```bash
+pip install decidr
+```
+
+Requires Python 3.10+ and a running Ollama. Zero dependencies — it's stdlib `urllib` and `math`.
+
+## How it works
+
+Three things, in order:
+
+**1. Options become letters.** Each option is presented to the model as `A`, `B`, `C`… rather than by its own name. This is not cosmetic: real labels like `billing` or `SYS_OUTAGE` are usually *several* tokens, and you cannot read a single-token probability for a multi-token string. Letters are single tokens in every vocabulary. The meaning lives in the descriptions, which is where the model actually reads it.
+
+**2. One forward pass, no generation.** The prompt ends where the answer begins, and generation is capped at a single token. Reasoning mode is explicitly disabled — on a hybrid-reasoning model, a `<think>` preamble would put thinking tokens in the answer slot, and the next token would stop being the decision.
+
+**3. The scores are read, not sampled.** Instead of taking whichever letter the model emitted, `decidr` reads the log probability of *every* option letter and normalizes over them. You get a distribution, not just a pick — so you can threshold on confidence, route ambiguous cases to a human, or log calibration over time.
+
+## Two modes, picked automatically
+
+How completely `decidr` can read those scores depends on your Ollama build. It probes once per process and tells you which mode you're in via `decision.mode`.
+
+| Mode | When | Behavior |
+|---|---|---|
+| `ranked` | **Stock Ollama** (what you have today) | Falls back to `top_logprobs` (capped at 20 by the API). Options whose letters don't surface in that window are reported in `decision.unscored`. |
+| `exact` | Ollama with [`logprob_tokens`](https://github.com/ollama/ollama/pull/18580) | Every option's probability is read directly from the full distribution, regardless of rank. |
+
+**Why this distinction exists:** stock Ollama can only tell you about tokens that rank in the model's top 20 guesses. For a 3-option decision, the letters almost always make that cut and `ranked` is fine. For a 12-option decision, they often don't:
+
+```
+12 options, same model, same prompt:
+  exact    ->  scored 12/12   reliable=True
+  ranked   ->  scored 11/12   reliable=False   unscored: ['cat9']
+```
+
+`decidr` never invents a number for an option it couldn't measure. It reports it as unscored, `is_reliable()` returns `False`, and the remaining probabilities are normalized over what was actually observed. A fabricated floor value would be indistinguishable from a real measurement, which is the one thing a probability API must never do.
+
+### Getting `exact` mode
+
+`logprob_tokens` is a change I've proposed upstream to Ollama — **it is not merged yet**:
+
+- Issue: [ollama/ollama#18579](https://github.com/ollama/ollama/issues/18579)
+- PR: [ollama/ollama#18580](https://github.com/ollama/ollama/pull/18580)
+
+Until it lands (if it lands), `decidr` works today in `ranked` mode against stock Ollama. Nothing here depends on that PR being accepted — `exact` mode is an upgrade, not a requirement. If you want it now, you can build from [the branch](https://github.com/devanmolsharma/ollama/tree/classification-logprobs).
+
+## API
+
+### `Client(model, host=..., timeout=..., temperature=..., force_mode=None)`
+
+- `model` — any Ollama model name, e.g. `"qwen3.5:4b"`.
+- `host` — defaults to `http://127.0.0.1:11434`.
+- `temperature` — applied to the softmax over option logprobs, **not** to sampling. Higher values flatten confidence. Use this to calibrate against a labeled set (see below).
+- `force_mode` — skip the capability probe and pin `"exact"` or `"ranked"`.
+
+### `client.decide(row) -> Decision`
+
+`row` needs `id`, `state`, `question`, and 2–16 `options`, each with an `id` and a `description`. `state` may be a string, dict, or list. Malformed rows raise `DecisionError` naming the problem rather than silently producing a confident wrong answer.
+
+### `Decision`
+
+| Field | |
+|---|---|
+| `choice` | option id with the highest probability |
+| `confidence` | probability of `choice` |
+| `probabilities` | option id → probability, sums to 1 **over scored options** |
+| `logprobs` | raw log probabilities before normalizing |
+| `mode` | `"exact"` or `"ranked"` |
+| `unscored` | options the server could not report |
+| `is_reliable()` | `False` when anything went unscored |
+| `raw_answer` | the letter the model actually emitted |
+
+`client.decide_all(rows)` runs a list sequentially.
+
+## Calibration, honestly
+
+The probabilities are **conditional on the option set you supplied** and are *not* calibrated out of the box. A raw instruction-tuned model's logits are not a calibrated probability distribution — they're systematically overconfident, and binary yes/no framings in particular carry a strong prior-driven skew.
+
+What that means in practice: `choice` is usually trustworthy; `confidence` is directional, not a true probability. Before thresholding on it (e.g. "auto-route above 0.9"), fit `temperature` against a labeled sample from your own workload and measure whether the numbers track observed accuracy. Purpose-built decision models are trained against proper scoring rules precisely because this step matters.
+
+If you need calibrated confidence out of the box more than you need "runs on anything you already have," use a trained decision model instead. `decidr`'s bet is that zero setup is worth more for most use cases, and that you should be told plainly where the limits are.
+
+## What this is not
+
+- **Not a new model.** It's a way of reading models you already run.
+- **Not a fine-tune.** Nothing is trained; taxonomies change per request.
+- **Not novel.** Reading option logits in a single pass is an established technique — [TypeSafe's Jev](https://www.typesafe.ai), [Laya](https://huggingface.co/convaiinnovations/laya), and [SemIf](https://github.com/TheoLeeCJ/SemIf) all do versions of it, and SemIf in particular has done far more rigorous benchmarking and calibration work than this has. `decidr`'s only claim is a narrow one: it's the smallest possible version that runs against an Ollama you already have, with zero dependencies and no model downloads.
+
+## Limitations
+
+- Sequential — no batching yet. One decision, one request.
+- 2–16 options (letters `A`–`P`).
+- `ranked` mode's completeness degrades as option count grows.
+- Probabilities uncalibrated by default (see above).
+- Tested against `qwen3.5:4b`. Small models (<1B) often won't treat a bare letter as a plausible next token; `decidr` raises a clear error rather than returning noise if none of the letters are scored.
+
+## License
+
+MIT

decidr-0.1.0/README.md

@@ -0,0 +1,131 @@
+# decidr
+
+**Typed decisions from local LLMs, in one forward pass.**
+
+Most decisions an application asks an LLM to make are small: *route this ticket*, *is this evidence sufficient*, *how angry is this customer*. A chat model can answer them, but it generates a sentence, or JSON, which your code then parses back into an `if` statement.
+
+`decidr` skips that. It gives the model your options, runs **one forward pass**, and reads the probability of each option directly out of the model's own logits. No answer sentence. No JSON to repair. No decoding loop.
+
+```python
+from decidr import Client
+
+client = Client(model="qwen3.5:4b")
+
+decision = client.decide({
+    "id": "route-1",
+    "state": "Customer cannot access an account after a password reset. The reset email never arrived.",
+    "question": "Which queue should handle this request?",
+    "options": [
+        {"id": "access",  "description": "Account access and authentication support."},
+        {"id": "billing", "description": "Billing and payment support."},
+        {"id": "sales",   "description": "Sales and product evaluation."},
+    ],
+})
+
+decision.choice          # 'access'
+decision.confidence      # 0.9999
+decision.probabilities   # {'access': 0.9999, 'billing': 0.0001, 'sales': 0.0}
+decision.is_reliable()   # True
+```
+
+Works with any model you already have in [Ollama](https://ollama.com). No fine-tuning, no extra runtime, no separate model to download.
+
+## Install
+
+```bash
+pip install decidr
+```
+
+Requires Python 3.10+ and a running Ollama. Zero dependencies — it's stdlib `urllib` and `math`.
+
+## How it works
+
+Three things, in order:
+
+**1. Options become letters.** Each option is presented to the model as `A`, `B`, `C`… rather than by its own name. This is not cosmetic: real labels like `billing` or `SYS_OUTAGE` are usually *several* tokens, and you cannot read a single-token probability for a multi-token string. Letters are single tokens in every vocabulary. The meaning lives in the descriptions, which is where the model actually reads it.
+
+**2. One forward pass, no generation.** The prompt ends where the answer begins, and generation is capped at a single token. Reasoning mode is explicitly disabled — on a hybrid-reasoning model, a `<think>` preamble would put thinking tokens in the answer slot, and the next token would stop being the decision.
+
+**3. The scores are read, not sampled.** Instead of taking whichever letter the model emitted, `decidr` reads the log probability of *every* option letter and normalizes over them. You get a distribution, not just a pick — so you can threshold on confidence, route ambiguous cases to a human, or log calibration over time.
+
+## Two modes, picked automatically
+
+How completely `decidr` can read those scores depends on your Ollama build. It probes once per process and tells you which mode you're in via `decision.mode`.
+
+| Mode | When | Behavior |
+|---|---|---|
+| `ranked` | **Stock Ollama** (what you have today) | Falls back to `top_logprobs` (capped at 20 by the API). Options whose letters don't surface in that window are reported in `decision.unscored`. |
+| `exact` | Ollama with [`logprob_tokens`](https://github.com/ollama/ollama/pull/18580) | Every option's probability is read directly from the full distribution, regardless of rank. |
+
+**Why this distinction exists:** stock Ollama can only tell you about tokens that rank in the model's top 20 guesses. For a 3-option decision, the letters almost always make that cut and `ranked` is fine. For a 12-option decision, they often don't:
+
+```
+12 options, same model, same prompt:
+  exact    ->  scored 12/12   reliable=True
+  ranked   ->  scored 11/12   reliable=False   unscored: ['cat9']
+```
+
+`decidr` never invents a number for an option it couldn't measure. It reports it as unscored, `is_reliable()` returns `False`, and the remaining probabilities are normalized over what was actually observed. A fabricated floor value would be indistinguishable from a real measurement, which is the one thing a probability API must never do.
+
+### Getting `exact` mode
+
+`logprob_tokens` is a change I've proposed upstream to Ollama — **it is not merged yet**:
+
+- Issue: [ollama/ollama#18579](https://github.com/ollama/ollama/issues/18579)
+- PR: [ollama/ollama#18580](https://github.com/ollama/ollama/pull/18580)
+
+Until it lands (if it lands), `decidr` works today in `ranked` mode against stock Ollama. Nothing here depends on that PR being accepted — `exact` mode is an upgrade, not a requirement. If you want it now, you can build from [the branch](https://github.com/devanmolsharma/ollama/tree/classification-logprobs).
+
+## API
+
+### `Client(model, host=..., timeout=..., temperature=..., force_mode=None)`
+
+- `model` — any Ollama model name, e.g. `"qwen3.5:4b"`.
+- `host` — defaults to `http://127.0.0.1:11434`.
+- `temperature` — applied to the softmax over option logprobs, **not** to sampling. Higher values flatten confidence. Use this to calibrate against a labeled set (see below).
+- `force_mode` — skip the capability probe and pin `"exact"` or `"ranked"`.
+
+### `client.decide(row) -> Decision`
+
+`row` needs `id`, `state`, `question`, and 2–16 `options`, each with an `id` and a `description`. `state` may be a string, dict, or list. Malformed rows raise `DecisionError` naming the problem rather than silently producing a confident wrong answer.
+
+### `Decision`
+
+| Field | |
+|---|---|
+| `choice` | option id with the highest probability |
+| `confidence` | probability of `choice` |
+| `probabilities` | option id → probability, sums to 1 **over scored options** |
+| `logprobs` | raw log probabilities before normalizing |
+| `mode` | `"exact"` or `"ranked"` |
+| `unscored` | options the server could not report |
+| `is_reliable()` | `False` when anything went unscored |
+| `raw_answer` | the letter the model actually emitted |
+
+`client.decide_all(rows)` runs a list sequentially.
+
+## Calibration, honestly
+
+The probabilities are **conditional on the option set you supplied** and are *not* calibrated out of the box. A raw instruction-tuned model's logits are not a calibrated probability distribution — they're systematically overconfident, and binary yes/no framings in particular carry a strong prior-driven skew.
+
+What that means in practice: `choice` is usually trustworthy; `confidence` is directional, not a true probability. Before thresholding on it (e.g. "auto-route above 0.9"), fit `temperature` against a labeled sample from your own workload and measure whether the numbers track observed accuracy. Purpose-built decision models are trained against proper scoring rules precisely because this step matters.
+
+If you need calibrated confidence out of the box more than you need "runs on anything you already have," use a trained decision model instead. `decidr`'s bet is that zero setup is worth more for most use cases, and that you should be told plainly where the limits are.
+
+## What this is not
+
+- **Not a new model.** It's a way of reading models you already run.
+- **Not a fine-tune.** Nothing is trained; taxonomies change per request.
+- **Not novel.** Reading option logits in a single pass is an established technique — [TypeSafe's Jev](https://www.typesafe.ai), [Laya](https://huggingface.co/convaiinnovations/laya), and [SemIf](https://github.com/TheoLeeCJ/SemIf) all do versions of it, and SemIf in particular has done far more rigorous benchmarking and calibration work than this has. `decidr`'s only claim is a narrow one: it's the smallest possible version that runs against an Ollama you already have, with zero dependencies and no model downloads.
+
+## Limitations
+
+- Sequential — no batching yet. One decision, one request.
+- 2–16 options (letters `A`–`P`).
+- `ranked` mode's completeness degrades as option count grows.
+- Probabilities uncalibrated by default (see above).
+- Tested against `qwen3.5:4b`. Small models (<1B) often won't treat a bare letter as a plausible next token; `decidr` raises a clear error rather than returning noise if none of the letters are scored.
+
+## License
+
+MIT

decidr-0.1.0/examples/triage.py

@@ -0,0 +1,76 @@
+"""Route support tickets, score evidence, and read tone -- three decisions,
+one forward pass each, against whatever model you already have in Ollama.
+
+    python examples/triage.py [model] [host]
+"""
+
+import sys
+import time
+
+from decidr import Client, DecisionError
+
+ROWS = [
+    {
+        "id": "route",
+        "state": "Customer cannot access an account after a password reset. The reset email never arrived.",
+        "question": "Which queue should handle this request?",
+        "options": [
+            {"id": "access", "description": "Account access and authentication support."},
+            {"id": "billing", "description": "Billing and payment support."},
+            {"id": "sales", "description": "Sales and product evaluation."},
+        ],
+    },
+    {
+        "id": "evidence",
+        "state": "The deployment completed at 14:02 UTC. Health checks passed in all three zones. No rollback was initiated.",
+        "question": "Is there evidence that the deployment succeeded?",
+        "options": [
+            {"id": "yes", "description": "The deployment succeeded."},
+            {"id": "no", "description": "The deployment did not succeed."},
+            {"id": "insufficient", "description": "The evidence is insufficient to decide."},
+        ],
+    },
+    {
+        "id": "tone",
+        "state": "This is the THIRD time you've charged me twice. I want a refund NOW or I'm calling my bank.",
+        "question": "How frustrated is the customer?",
+        "options": [
+            {"id": "calm", "description": "Calm and neutral."},
+            {"id": "annoyed", "description": "Mildly annoyed."},
+            {"id": "angry", "description": "Very angry."},
+        ],
+    },
+]
+
+
+def main() -> int:
+    model = sys.argv[1] if len(sys.argv) > 1 else "qwen3.5:4b"
+    host = sys.argv[2] if len(sys.argv) > 2 else "http://127.0.0.1:11434"
+
+    client = Client(model=model, host=host)
+    try:
+        mode = "exact" if client.supports_exact() else "ranked"
+    except DecisionError as e:
+        print(f"could not reach Ollama: {e}")
+        return 1
+
+    print(f"model={model}  mode={mode}\n")
+    for row in ROWS:
+        started = time.perf_counter()
+        try:
+            d = client.decide(row)
+        except DecisionError as e:
+            print(f"{row['id']:10s} error: {e}")
+            continue
+        ms = round((time.perf_counter() - started) * 1000)
+
+        ranked = sorted(d.probabilities.items(), key=lambda kv: -kv[1])
+        spread = "  ".join(f"{k}={v:.3f}" for k, v in ranked)
+        print(f"{row['id']:10s} {d.choice:13s} {d.confidence:6.1%}  {ms:5d}ms   {spread}")
+        if not d.is_reliable():
+            print(f"{'':10s} unscored (outside the top-20 window): {d.unscored}")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

decidr-0.1.0/pyproject.toml

@@ -0,0 +1,33 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "decidr"
+version = "0.1.0"
+description = "Typed decisions from local LLMs in one forward pass, via Ollama"
+readme = "README.md"
+requires-python = ">=3.10"
+license = { text = "MIT" }
+keywords = ["llm", "ollama", "classification", "routing", "logprobs", "decisions"]
+classifiers = [
+  "Development Status :: 3 - Alpha",
+  "Intended Audience :: Developers",
+  "License :: OSI Approved :: MIT License",
+  "Programming Language :: Python :: 3",
+  "Topic :: Scientific/Engineering :: Artificial Intelligence",
+]
+dependencies = []
+
+[project.optional-dependencies]
+dev = ["pytest>=7"]
+
+[project.urls]
+Homepage = "https://github.com/devanmolsharma/decidr"
+Issues = "https://github.com/devanmolsharma/decidr/issues"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/decidr"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

decidr-0.1.0/src/decidr/__init__.py

@@ -0,0 +1,22 @@
+"""decidr -- typed decisions from local LLMs, in one forward pass."""
+
+from .core import (
+    LETTERS,
+    Client,
+    Decision,
+    DecisionError,
+    build_messages,
+    softmax,
+    validate_row,
+)
+
+__all__ = [
+    "Client",
+    "Decision",
+    "DecisionError",
+    "LETTERS",
+    "build_messages",
+    "softmax",
+    "validate_row",
+]
+__version__ = "0.1.0"

decidr-0.1.0/src/decidr/core.py

@@ -0,0 +1,256 @@
+"""Typed semantic decisions against any local Ollama model, in one forward pass.
+
+Send state + question + options, get probabilities back. No answer sentence, no
+JSON to repair, no decoding loop -- the answer is read out of the option tokens'
+log probabilities at a single position.
+
+Rows use the same shape other single-pass decision engines accept, so they port
+between implementations:
+
+    {"id": "route-1",
+     "state": "Customer cannot access an account after a password reset.",
+     "question": "Which queue should handle this request?",
+     "options": [{"id": "access",  "description": "Account access support."},
+                 {"id": "billing", "description": "Billing support."}]}
+
+Options are presented to the model as letters (A, B, C...) rather than as their
+own names. Letters are single tokens in every vocabulary, which sidesteps the
+fact that most real labels ("billing", "SYS_OUTAGE") are not -- the meaning
+lives in the descriptions, where the model can actually read it.
+
+Two ways of reading the letters' scores, picked automatically:
+
+  exact   - the server supports `logprob_tokens` (ollama/ollama#18579), so every
+            option's logprob is read directly from the full distribution
+            regardless of rank.
+  ranked  - stock Ollama: fall back to `top_logprobs` (max 20) and match the
+            letters that appear. Options outside that window are reported as
+            unscored rather than guessed at.
+"""
+
+from __future__ import annotations
+
+import json
+import math
+import urllib.error
+import urllib.request
+from dataclasses import dataclass, field
+from typing import Any, Literal
+
+LETTERS = "ABCDEFGHIJKLMNOP"
+
+SYSTEM = (
+    "Apply the supplied criterion to the supplied evidence. Choose exactly one listed option. "
+    "Respond with only its uppercase letter, with no explanation or reasoning."
+)
+
+DEFAULT_HOST = "http://127.0.0.1:11434"
+
+
+class DecisionError(ValueError):
+    """Bad input row, or a server that could not answer it."""
+
+
+@dataclass
+class Decision:
+    id: str
+    choice: str                       # option id with the highest probability
+    probabilities: dict[str, float]   # option id -> probability, sums to 1 over scored options
+    logprobs: dict[str, float]        # option id -> raw logprob, before normalizing
+    mode: Literal["exact", "ranked"]  # how the scores were read; see module docstring
+    unscored: list[str] = field(default_factory=list)  # options the server could not report
+    raw_answer: str | None = None     # the letter the model actually emitted
+
+    @property
+    def confidence(self) -> float:
+        return self.probabilities.get(self.choice, 0.0)
+
+    def is_reliable(self) -> bool:
+        """False when some option went unscored, so the distribution is incomplete
+        and `probabilities` is normalized over a subset of what was asked."""
+        return not self.unscored
+
+
+def validate_row(row: dict) -> None:
+    """Reject a malformed row up front, with a message naming what's wrong --
+    a bad row should fail here, not produce a confident-looking wrong answer."""
+    required = {"id", "state", "question", "options"}
+    if not required <= row.keys():
+        raise DecisionError(f"row is missing fields: {sorted(required - row.keys())}")
+    if not all(isinstance(row[k], str) and row[k] for k in ("id", "question")):
+        raise DecisionError("id and question must be nonempty strings")
+    state = row["state"]
+    if not isinstance(state, (str, dict, list)) or not state:
+        raise DecisionError("state must be a nonempty string, object, or array")
+    try:
+        json.dumps(state, ensure_ascii=False, allow_nan=False)
+    except (TypeError, ValueError) as e:
+        raise DecisionError("state must be finite JSON-compatible data") from e
+    options = row["options"]
+    if not isinstance(options, list) or not 2 <= len(options) <= len(LETTERS):
+        raise DecisionError(f"options must contain 2-{len(LETTERS)} entries")
+    ids = []
+    for opt in options:
+        if not isinstance(opt, dict) or not isinstance(opt.get("id"), str) or not isinstance(opt.get("description"), str):
+            raise DecisionError("each option needs string id and description fields")
+        ids.append(opt["id"])
+    if len(ids) != len(set(ids)):
+        raise DecisionError("option ids must be unique")
+
+
+def build_messages(row: dict) -> list[dict]:
+    payload = {
+        "evidence": row["state"],
+        "criterion": row["question"],
+        "options": [
+            {"letter": LETTERS[i], "description": opt["description"]}
+            for i, opt in enumerate(row["options"])
+        ],
+    }
+    return [
+        {"role": "system", "content": SYSTEM},
+        {"role": "user", "content": json.dumps(payload, ensure_ascii=False)},
+    ]
+
+
+def softmax(logprobs: list[float], temperature: float = 1.0) -> list[float]:
+    """Normalize over the supplied options only. The result is conditional on
+    this option set -- it says nothing about tokens outside it."""
+    if not logprobs:
+        return []
+    scaled = [lp / temperature for lp in logprobs]
+    top = max(scaled)
+    exp = [math.exp(v - top) for v in scaled]
+    total = sum(exp)
+    return [v / total for v in exp] if total else [1.0 / len(exp)] * len(exp)
+
+
+class Client:
+    """Talks to one Ollama server. Probes once for `logprob_tokens` support and
+    reuses the answer, so the capability check costs one request per process."""
+
+    def __init__(self, model: str, host: str = DEFAULT_HOST, timeout: float = 120.0,
+                 temperature: float = 1.0, force_mode: Literal["exact", "ranked"] | None = None):
+        self.model = model
+        self.host = host.rstrip("/")
+        self.timeout = timeout
+        self.temperature = temperature
+        self._supports_exact: bool | None = {"exact": True, "ranked": False}.get(force_mode or "")
+        if force_mode is None:
+            self._supports_exact = None
+
+    # ---- transport ---------------------------------------------------------
+
+    def _post(self, path: str, body: dict) -> dict:
+        req = urllib.request.Request(
+            f"{self.host}{path}",
+            data=json.dumps(body).encode(),
+            headers={"Content-Type": "application/json"},
+        )
+        try:
+            with urllib.request.urlopen(req, timeout=self.timeout) as resp:
+                return json.loads(resp.read())
+        except urllib.error.HTTPError as e:
+            detail = e.read().decode(errors="replace")
+            try:
+                detail = json.loads(detail).get("error", detail)
+            except json.JSONDecodeError:
+                pass
+            raise DecisionError(f"{path} failed ({e.code}): {detail}") from e
+        except urllib.error.URLError as e:
+            raise DecisionError(f"cannot reach Ollama at {self.host}: {e.reason}") from e
+
+    def _chat(self, messages: list[dict], letters: list[str]) -> dict:
+        body: dict[str, Any] = {
+            "model": self.model,
+            "messages": messages,
+            "stream": False,
+            # A reasoning preamble would put thinking tokens in the answer slot,
+            # so the very next token stops being the decision.
+            "think": False,
+            "options": {"num_predict": 1, "temperature": 0},
+            "logprobs": True,
+            "top_logprobs": 20,
+        }
+        if self.supports_exact():
+            body["logprob_tokens"] = letters
+        return self._post("/api/chat", body)
+
+    def supports_exact(self) -> bool:
+        """True when the server accepts `logprob_tokens`. Probed once with a
+        throwaway request; a server that rejects the field answers the probe
+        with a 400, which is how we detect stock Ollama."""
+        if self._supports_exact is None:
+            probe = {
+                "model": self.model,
+                "messages": [{"role": "user", "content": "hi"}],
+                "stream": False,
+                "think": False,
+                "options": {"num_predict": 1},
+                "logprob_tokens": ["A"],
+            }
+            try:
+                resp = self._post("/api/chat", probe)
+                got = resp.get("logprobs") or [{}]
+                self._supports_exact = bool(got[0].get("requested_logprobs"))
+            except DecisionError:
+                self._supports_exact = False
+        return self._supports_exact
+
+    # ---- decisions ---------------------------------------------------------
+
+    def decide(self, row: dict) -> Decision:
+        validate_row(row)
+        options = row["options"]
+        letters = [LETTERS[i] for i in range(len(options))]
+        resp = self._chat(build_messages(row), letters)
+
+        entries = resp.get("logprobs") or []
+        if not entries:
+            raise DecisionError(
+                f"server returned no logprobs for row {row['id']!r}; the model may not "
+                "support them, or `logprobs` was rejected"
+            )
+        first = entries[0]
+        mode: Literal["exact", "ranked"] = "exact" if first.get("requested_logprobs") else "ranked"
+
+        # Both paths yield {token: logprob}; exact covers every letter, ranked
+        # only those that surfaced in the top-20 window.
+        if mode == "exact":
+            found = {e["token"]: e["logprob"] for e in first["requested_logprobs"]}
+        else:
+            found = {}
+            if first.get("token"):
+                found[first["token"].strip()] = first.get("logprob", 0.0)
+            for alt in first.get("top_logprobs", []):
+                found.setdefault(alt["token"].strip(), alt["logprob"])
+
+        logprobs: dict[str, float] = {}
+        unscored: list[str] = []
+        for letter, opt in zip(letters, options):
+            if letter in found:
+                logprobs[opt["id"]] = found[letter]
+            else:
+                unscored.append(opt["id"])
+
+        if not logprobs:
+            raise DecisionError(
+                f"row {row['id']!r}: none of the option letters {letters} were scored. "
+                "The model did not treat a bare letter as a plausible next token here."
+            )
+
+        ids = list(logprobs)
+        probs = softmax([logprobs[i] for i in ids], self.temperature)
+        probabilities = dict(zip(ids, probs))
+        return Decision(
+            id=row["id"],
+            choice=max(probabilities, key=probabilities.get),
+            probabilities=probabilities,
+            logprobs=logprobs,
+            mode=mode,
+            unscored=unscored,
+            raw_answer=(resp.get("message") or {}).get("content"),
+        )
+
+    def decide_all(self, rows: list[dict]) -> list[Decision]:
+        return [self.decide(row) for row in rows]

decidr-0.1.0/tests/test_core.py

@@ -0,0 +1,163 @@
+import json
+import math
+
+import pytest
+
+from decidr import Client, DecisionError, build_messages, softmax, validate_row
+
+ROW = {
+    "id": "route-1",
+    "state": "Customer cannot access an account after a password reset.",
+    "question": "Which queue should handle this request?",
+    "options": [
+        {"id": "access", "description": "Account access support."},
+        {"id": "billing", "description": "Billing support."},
+    ],
+}
+
+
+def test_validate_accepts_a_good_row():
+    validate_row(ROW)
+
+
+@pytest.mark.parametrize(
+    "mutate, expected",
+    [
+        (lambda r: r.pop("question"), "missing fields"),
+        (lambda r: r.update(question=""), "nonempty strings"),
+        (lambda r: r.update(state=""), "state must be"),
+        (lambda r: r.update(options=r["options"][:1]), "options must contain"),
+        (lambda r: r.update(options=[{"id": "a"}, {"id": "b"}]), "id and description"),
+        (lambda r: r.update(options=[{"id": "x", "description": "d"}] * 2), "unique"),
+    ],
+)
+def test_validate_rejects_bad_rows(mutate, expected):
+    row = json.loads(json.dumps(ROW))
+    mutate(row)
+    with pytest.raises(DecisionError, match=expected):
+        validate_row(row)
+
+
+def test_options_are_presented_as_letters():
+    # Real labels are usually multi-token; letters always aren't. The meaning
+    # has to survive in the descriptions instead.
+    messages = build_messages(ROW)
+    payload = json.loads(messages[-1]["content"])
+    assert [o["letter"] for o in payload["options"]] == ["A", "B"]
+    assert payload["options"][0]["description"] == "Account access support."
+    # Only letter + description reach the model; the option ids stay client-side,
+    # so an id like "SYS_OUTAGE" never has to tokenize cleanly.
+    assert all(set(o) == {"letter", "description"} for o in payload["options"])
+
+
+def test_softmax_normalizes_over_supplied_options_only():
+    probs = softmax([-0.1, -2.0, -8.0])
+    assert math.isclose(sum(probs), 1.0)
+    assert probs[0] > probs[1] > probs[2]
+
+
+def test_softmax_handles_degenerate_input():
+    assert softmax([]) == []
+    equal = softmax([-1.0, -1.0])
+    assert math.isclose(equal[0], 0.5)
+
+
+def test_temperature_flattens_the_distribution():
+    sharp = softmax([-0.1, -3.0], temperature=1.0)
+    flat = softmax([-0.1, -3.0], temperature=5.0)
+    assert flat[0] < sharp[0]  # higher temperature is less confident
+
+
+class FakeClient(Client):
+    """Client with the transport stubbed, so decision parsing is tested without
+    a live server."""
+
+    def __init__(self, response, supports_exact):
+        super().__init__(model="test", force_mode="exact" if supports_exact else "ranked")
+        self._response = response
+
+    def _post(self, path, body):
+        self._last_body = body
+        return self._response
+
+
+def test_exact_mode_reads_requested_logprobs():
+    client = FakeClient(
+        {
+            "message": {"content": "A"},
+            "logprobs": [
+                {
+                    "token": "A",
+                    "logprob": -0.05,
+                    "requested_logprobs": [
+                        {"token": "A", "logprob": -0.05},
+                        {"token": "B", "logprob": -9.0},
+                    ],
+                }
+            ],
+        },
+        supports_exact=True,
+    )
+    d = client.decide(ROW)
+    assert d.mode == "exact"
+    assert d.choice == "access"
+    assert d.is_reliable()
+    assert d.probabilities["access"] > 0.99
+    assert client._last_body["logprob_tokens"] == ["A", "B"]
+
+
+def test_ranked_mode_falls_back_to_top_logprobs():
+    client = FakeClient(
+        {
+            "message": {"content": "B"},
+            "logprobs": [
+                {
+                    "token": "B",
+                    "logprob": -0.2,
+                    "top_logprobs": [
+                        {"token": "B", "logprob": -0.2},
+                        {"token": "A", "logprob": -1.9},
+                    ],
+                }
+            ],
+        },
+        supports_exact=False,
+    )
+    d = client.decide(ROW)
+    assert d.mode == "ranked"
+    assert d.choice == "billing"
+    assert "logprob_tokens" not in client._last_body
+
+
+def test_options_outside_the_window_are_reported_not_guessed():
+    # Only A came back. B must be surfaced as unscored rather than assigned
+    # some floor value that would look like a real measurement.
+    client = FakeClient(
+        {
+            "message": {"content": "A"},
+            "logprobs": [{"token": "A", "logprob": -0.01, "top_logprobs": [{"token": "A", "logprob": -0.01}]}],
+        },
+        supports_exact=False,
+    )
+    d = client.decide(ROW)
+    assert d.unscored == ["billing"]
+    assert not d.is_reliable()
+    assert math.isclose(d.probabilities["access"], 1.0)  # normalized over what was scored
+
+
+def test_no_scored_options_is_an_error_not_a_coin_flip():
+    client = FakeClient(
+        {
+            "message": {"content": "hello"},
+            "logprobs": [{"token": "hello", "logprob": -0.5, "top_logprobs": [{"token": "hi", "logprob": -1.0}]}],
+        },
+        supports_exact=False,
+    )
+    with pytest.raises(DecisionError, match="none of the option letters"):
+        client.decide(ROW)
+
+
+def test_missing_logprobs_is_an_error():
+    client = FakeClient({"message": {"content": "A"}}, supports_exact=False)
+    with pytest.raises(DecisionError, match="no logprobs"):
+        client.decide(ROW)