onejev 0.1.1__py3-none-any.whl

onejev-0.1.1.dist-info/METADATA

@@ -0,0 +1,199 @@
+Metadata-Version: 2.4
+Name: onejev
+Version: 0.1.1
+Summary: Open System One decision model: typed, calibrated decisions behind a TypeSafe-compatible API
+License: Apache-2.0
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: transformers>=5.0
+Requires-Dist: pillow>=10
+Requires-Dist: jinja2>=3.1
+Requires-Dist: pydantic>=2.0
+Requires-Dist: fastapi>=0.110
+Requires-Dist: uvicorn>=0.30
+Provides-Extra: torch
+Requires-Dist: torch>=2.6; extra == "torch"
+Requires-Dist: torchvision>=0.21; extra == "torch"
+Requires-Dist: accelerate>=1.0; extra == "torch"
+Requires-Dist: safetensors>=0.4; extra == "torch"
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+Requires-Dist: httpx>=0.27; extra == "dev"
+Requires-Dist: typesafe-sdk>=0.6; extra == "dev"
+Provides-Extra: train
+Requires-Dist: torch>=2.6; extra == "train"
+Requires-Dist: torchvision>=0.21; extra == "train"
+Requires-Dist: accelerate>=1.0; extra == "train"
+Requires-Dist: safetensors>=0.4; extra == "train"
+Requires-Dist: peft>=0.13; extra == "train"
+Requires-Dist: pyyaml>=6; extra == "train"
+Requires-Dist: pyarrow>=14; extra == "train"
+Dynamic: license-file
+
+<p align="center">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" srcset="assets/banner-dark.svg">
+    <img alt="OneJev, a Multimodal System One Decision Model" src="assets/banner.svg" width="100%">
+  </picture>
+</p>
+
+<p align="center"><b>English</b> · <a href="assets/README_zh.md">简体中文</a> · <a href="assets/README_ja.md">日本語</a></p>
+
+<p align="center">
+  <a href="https://huggingface.co/collections/OmniJev/onejev"><img alt="Hugging Face" src="https://img.shields.io/badge/Hugging_Face-OneJev-FFD21E?style=flat-square&logo=huggingface&logoColor=black"></a>
+  <a href="https://huggingface.co/spaces/BradNLP/OneJev"><img alt="Demo" src="https://img.shields.io/badge/Demo-Try_it-D45BB6?style=flat-square&logo=gradio&logoColor=white"></a>
+  <a href="https://github.com/OmniJev/OneJev"><img alt="GitHub" src="https://img.shields.io/badge/GitHub-OmniJev%2FOneJev-181717?style=flat-square&logo=github&logoColor=white"></a>
+  <a href="https://omnijev.github.io/OneJev/"><img alt="Website" src="https://img.shields.io/badge/Website-OneJev-0A84FF?style=flat-square&logo=googlechrome&logoColor=white"></a>
+  <a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/License-Apache_2.0-D22128?style=flat-square&logo=apache&logoColor=white"></a>
+</p>
+<p align="center">
+  <a href="https://huggingface.co/datasets/OmniJev/OneJev-Data"><img alt="Data" src="https://img.shields.io/badge/Data-OneJev--Data-FF9D00?style=flat-square&logo=huggingface&logoColor=white"></a>
+  <a href="examples/request.json"><img alt="API" src="https://img.shields.io/badge/API-System_One-009688?style=flat-square&logo=fastapi&logoColor=white"></a>
+  <a href="https://www.python.org"><img alt="Python" src="https://img.shields.io/badge/Python-3.10%2B-3776AB?style=flat-square&logo=python&logoColor=white"></a>
+  <a href="https://pytorch.org"><img alt="PyTorch" src="https://img.shields.io/badge/PyTorch-2.6%2B-EE4C2C?style=flat-square&logo=pytorch&logoColor=white"></a>
+  <a href="https://github.com/huggingface/transformers"><img alt="Transformers" src="https://img.shields.io/badge/Transformers-5%2B-FFD21E?style=flat-square&logo=huggingface&logoColor=black"></a>
+</p>
+
+OneJev is a multimodal System One decision model. It returns calibrated probabilities for typed questions about
+screenshots, photos, videos and text in a single forward pass. Available in 0.8B, 4B, 9B and 27B.
+
+## Results
+
+<p align="center">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" srcset="assets/results-dark.svg">
+    <img alt="Block bar charts: the four OneJev sizes against Jev 1.13, Jev-Omni 12B and Qwen3.8-27B thinking on the OneJev test set, DecisionBench hard, TypeSafe and MMStar" src="assets/results.svg" width="100%">
+  </picture>
+</p>
+
+<p align="center">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" srcset="assets/table-dark.svg">
+    <img alt="Results table, best score in each row in magenta, second best in light pink" src="assets/table.svg" width="100%">
+  </picture>
+</p>
+
+Accuracy (%). The OneJev test set is held out from OneJev training. Jev 1.13 uses published text-only scores;
+Jev-Omni and Qwen3.8-27B thinking were evaluated by us.
+
+<p align="center">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" srcset="assets/speed-dark.svg">
+    <img alt="Latency on one H200 for one 1280x720 screenshot: OneJev-0.8B 31 ms for 1 question, 51 ms for 10, 5.1 ms per question; OneJev-4B 64 ms for 1 question, 104 ms for 10, 10.4 ms per question; OneJev-9B 81 ms for 1 question, 131 ms for 10, 13.1 ms per question; OneJev-27B 189 ms for 1 question, 324 ms for 10, 32.4 ms per question" src="assets/speed.svg" width="100%">
+  </picture>
+</p>
+
+## Quick start
+
+Choose one backend, then run the Python example below.
+
+### Option A: PyTorch
+
+For NVIDIA GPUs. Supports text, images and video.
+
+```bash
+pip install "qev[torch] @ git+https://github.com/OmniJev/OneJev.git"
+qev serve --model OmniJev/OneJev-4B --port 8000
+```
+
+### Option B: llama.cpp
+
+For GGUF models. Supports text and images; use PyTorch for video. Install
+[llama.cpp](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md) first (`brew install llama.cpp` on macOS).
+
+```bash
+pip install git+https://github.com/OmniJev/OneJev.git
+qev serve --gguf mradermacher/OneJev-4B-GGUF:Q8_0 --port 8000
+```
+
+### Send a request
+
+Both backends serve the same API at `http://localhost:8000`. In another terminal, run this example with your own
+`screenshot.png`:
+
+```python
+from qev import Client, Choice, Noul, Score
+from qev.media import data_uri
+
+r = Client("http://localhost:8000").system_one(
+    state={"task": "Pay the open invoice from ACME", "screen": "<image:1>"},
+    media=[{"type": "image", "data": data_uri("screenshot.png")}],
+    questions={"done": Noul("The invoice has been paid"),
+               "next": Choice("What should the agent do next?",
+                              {"click": "click an element", "type": "type text", "scroll": "scroll", "stop": "stop"}),
+               "progress": Score("How far along is the task?", ["not started", "halfway", "almost done", "done"])},
+)
+r.answers["done"].noul                  # probability of yes
+r.answers["next"].probabilities         # one probability per option
+r.answers["progress"].score             # expected level
+```
+
+The API is compatible with TypeSafe System One. More examples: [video](examples/video.py),
+[curl](examples/curl.sh), [official TypeSafe SDK](examples/official_sdk.py).
+
+## Training
+
+```bash
+git clone https://github.com/OmniJev/OneJev.git
+cd OneJev
+pip install -e ".[train]"
+
+# Choose A or B. A is enabled below.
+# A. Demo: 100 examples with images (23 MB)
+hf download OmniJev/OneJev-Data sample/sample-100.parquet --repo-type dataset --local-dir data/onejev
+python -m train.unpack data/onejev --sample
+
+# B. Full dataset: 94,707 examples (17.7 GB). Uncomment these two lines instead of A.
+# hf download OmniJev/OneJev-Data --repo-type dataset --include "data/*.parquet" --local-dir data/onejev
+# python -m train.unpack data/onejev
+```
+
+Either option creates `data/onejev/train.jsonl` and extracts images and video frames into `data/onejev/media/`.
+
+[OneJev-Data](https://huggingface.co/datasets/OmniJev/OneJev-Data) releases 94,707 of the original 99,193
+training questions; the remaining sources do not permit redistribution.
+
+### Fine-tune
+
+The four models fine-tune Qwen3.5-0.8B, Qwen3.5-4B, Qwen3.5-9B and Qwen3.8-27B for one epoch with the vision tower
+frozen. The configs use a learning rate of `5e-6`, a 16,384-token limit, and cross-entropy plus Brier loss over answer
+probabilities. To train the 4B model on four GPUs:
+
+```bash
+torchrun --nproc-per-node 4 -m train.sft --config train/configs/onejev_4b_full.yaml
+```
+
+Configs: [0.8B](train/configs/onejev_08b_full.yaml) · [4B](train/configs/onejev_4b_full.yaml) ·
+[9B](train/configs/onejev_9b_full.yaml) · [27B](train/configs/onejev_27b_full.yaml).
+Set `--nproc-per-node` to your GPU count; the 9B and 27B configs enable FSDP to shard model and optimizer state.
+For your own data, change `train` and `media_root` in the config.
+
+The final 4B checkpoint is saved to `train/runs/onejev_4b/final`. Start it with:
+
+```bash
+qev serve --model train/runs/onejev_4b/final --port 8000
+```
+
+## Citation
+
+```bibtex
+@misc{onejev2026,
+  title        = {{OneJev}: A Multimodal System One Decision Model},
+  author       = {{OmniJev Team}},
+  year         = {2026},
+  howpublished = {\url{https://github.com/OmniJev/OneJev}}
+}
+```
+
+## Friendly Links
+
+- [LINUX DO](https://linux.do)
+- [Jev](https://typesafe.ai)
+- [Awesome JEV](https://github.com/OmniJev/awesome-jev-gallery)
+- [Awesome JEV Website](https://omnijev.github.io/awesome-jev-gallery/)
+- [PlayJev](https://github.com/OmniJev/PlayJev)
+
+## License
+
+[Apache 2.0](LICENSE)

onejev-0.1.1.dist-info/RECORD

@@ -0,0 +1,20 @@
+onejev-0.1.1.dist-info/licenses/LICENSE,sha256=z8d0m5b2O9McPEK1xHG_dWgUBT6EfBDz6wA0F7xSPTA,11358
+qev/__init__.py,sha256=bpWa-RBdgR_BEz727x0GV1w2botOMfHqWUlWjKMkNtA,145
+qev/answers.py,sha256=CbBtuiWMgKapr9qlqs_Q91I_nSoIentkAsoDVT6m-6Y,2542
+qev/calibrate.py,sha256=P6YBOoHH55pwKvzcwOvQib7nBHWQwYzERwiFD1rtoe4,4245
+qev/cli.py,sha256=Iy963SM978vcqwUuqqaR3QPxwvDhqpV7JW1qurg4Avs,10152
+qev/client.py,sha256=VgygD37P_ucuZJCv3WmjTPT1Wm4XZaBJPUvuv4trQng,4417
+qev/engine.py,sha256=-WZcaHFIGNvqjtwegNCw2Gc3U33WBv9Tz_N6r7hNa_Q,22720
+qev/fp8.py,sha256=5mZjUUcSdHnbuB9FxGmhOwkHif855gpyh38rbQ4njek,5996
+qev/llamacpp.py,sha256=eYK4B_HO_WI00mDn6urlv91egEQXJR8DJa35AeFDidk,14574
+qev/media.py,sha256=H27HGwOkL39IRhJgWA2KrMH6b_ZKPGhMsjux1dE6qMA,6676
+qev/mm_engine.py,sha256=KwOykO6TKMql4jvqjZK229j_9hFSMFu-DkRa1NHnoTI,38656
+qev/prompt.py,sha256=F9-uCeTf9Hy4zoDhcuuhFvGIuCtwwnd70U_ZuKcWgN4,6421
+qev/schema.py,sha256=vOS301OsEVqeUK7yhJdd3u2aO7RS4_2SilVmaHBbwvM,3050
+qev/server.py,sha256=Ud2q_rxncoauv_Nw7gdAuu-fkCd4syu42hsGH_qWTdI,3954
+qev/playground/index.html,sha256=dV8iRSoS5WzjOvC8KEF1OcwG9oOHN6RpAqGy7W-5xGE,6483
+onejev-0.1.1.dist-info/METADATA,sha256=S5YLIB9p0S8hkw4F9wm1hOw13tIOeVY9c3VkMKlrbUk,8941
+onejev-0.1.1.dist-info/WHEEL,sha256=YVMoNqKzERt-wjUZwJ33xBGAwnFl-4cqbYkTtWa4itE,91
+onejev-0.1.1.dist-info/entry_points.txt,sha256=nu0tCD0-9sRclFCluddGkyPu1kZzYo5e1VGvRP-nqEg,37
+onejev-0.1.1.dist-info/top_level.txt,sha256=VwtV-FGA2ITJe4PdPSpCWK5c6h6d6VYl1DnfVrH2XTE,4
+onejev-0.1.1.dist-info/RECORD,,

onejev-0.1.1.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: setuptools (84.0.0)
+Root-Is-Purelib: true
+Tag: py3-none-any
+

onejev-0.1.1.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+qev = qev.cli:main

onejev-0.1.1.dist-info/licenses/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

onejev-0.1.1.dist-info/top_level.txt

@@ -0,0 +1 @@
+qev

qev/__init__.py

@@ -0,0 +1,4 @@
+"""Qev: open System One decision model and TypeSafe-compatible server."""
+__version__ = "0.1.1"
+
+from .client import Choice, Client, Noul, Score

qev/answers.py

@@ -0,0 +1,74 @@
+"""Answer assembly: probabilities to typed answers, with TypeSafe's confidence formulas.
+
+The two confidence functions are ported from typesafe-ai/system-one-adapter-python
+(MIT License, Copyright (c) 2026 TypeSafe AI) and verified against Jev's published answers
+(mean absolute error 0.013 for Choice, 0.005 for Score, within two-decimal rounding).
+"""
+from __future__ import annotations
+
+from typing import Any, Sequence
+
+from .schema import ChoiceAnswer, NoulAnswer, ScoreAnswer
+
+
+def _normalize(probs: Sequence[float]) -> list[float]:
+    total = float(sum(probs))
+    if total <= 0:
+        return [1.0 / len(probs)] * len(probs)
+    return [float(p) / total for p in probs]
+
+
+def choice_confidence(probs: Sequence[float]) -> float:
+    """Peak probability rescaled from uniform (0) to certainty (1)."""
+    if len(probs) == 1:
+        return 1.0
+    p = _normalize(probs)
+    uniform = 1.0 / len(p)
+    return (max(p) - uniform) / (1.0 - uniform)
+
+
+def score_confidence(probs: Sequence[float]) -> float:
+    """One minus the probability-weighted mean absolute deviation from the modal level,
+    normalised by the mean absolute deviation of the uniform distribution over the levels."""
+    if len(probs) == 1:
+        return 1.0
+    p = _normalize(probs)
+    k = len(p)
+    mode = max(range(k), key=p.__getitem__)
+    distance = sum(pi * abs(i - mode) for i, pi in enumerate(p))
+    center = (k - 1) / 2
+    uniform_mad = sum(abs(i - center) for i in range(k)) / k
+    return max(0.0, 1.0 - distance / uniform_mad)
+
+
+def expected_score(probs: Sequence[float]) -> float:
+    p = _normalize(probs)
+    return float(sum(i * pi for i, pi in enumerate(p)))
+
+
+def _round(x: float, nd: int = 6) -> float:
+    return float(round(x, nd))
+
+
+def make_noul(p_yes: float) -> NoulAnswer:
+    return NoulAnswer(noul=_round(min(1.0, max(0.0, float(p_yes)))))
+
+
+def make_choice(options: Sequence[str], probs: Sequence[float]) -> ChoiceAnswer:
+    p = _normalize(probs)
+    best = max(range(len(p)), key=p.__getitem__)
+    return ChoiceAnswer(
+        choice=options[best],
+        probabilities={o: _round(pi) for o, pi in zip(options, p)},
+        confidence=_round(choice_confidence(p)),
+    )
+
+
+def make_score(levels: Sequence[Any], probs: Sequence[float]) -> ScoreAnswer:
+    p = _normalize(probs)
+    return ScoreAnswer(
+        score=_round(expected_score(p)),
+        legend={str(i): lvl for i, lvl in enumerate(levels)},
+        probabilities={str(i): _round(pi) for i, pi in enumerate(p)},
+        confidence=_round(score_confidence(p)),
+    )

qev/calibrate.py

@@ -0,0 +1,114 @@
+"""Calibration: temperatures per (primitive, option count), fitting, and probability metrics."""
+from __future__ import annotations
+
+import json
+import logging
+import math
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Sequence
+
+
+@dataclass
+class Calibration:
+    """Temperature table. Keys are "kind" or "kind:K". Missing keys fall back to the primitive-level
+    temperature, then to 1.0."""
+
+    temperatures: dict[str, float] = field(default_factory=dict)
+    source: str = "none"
+
+    def temperature(self, kind: str, k: int) -> float:
+        return self.temperatures.get(f"{kind}:{k}", self.temperatures.get(kind, 1.0))
+
+    @classmethod
+    def load(cls, path: str | Path) -> "Calibration":
+        data = json.loads(Path(path).read_text())
+        return cls(temperatures={k: float(v) for k, v in data.get("temperatures", {}).items()}, source=str(path))
+
+    def save(self, path: str | Path) -> None:
+        from .prompt import PROMPT_VERSION
+
+        Path(path).write_text(json.dumps({"temperatures": self.temperatures, "source": self.source,
+                                          "prompt_version": PROMPT_VERSION}, indent=2))
+
+
+def softmax(logits: Sequence[float], temperature: float = 1.0) -> list[float]:
+    z = [float(x) / temperature for x in logits]
+    m = max(z)
+    w = [math.exp(x - m) for x in z]
+    s = sum(w)
+    return [x / s for x in w]
+
+
+def fit_temperature(logits: Sequence[Sequence[float]], targets: Sequence[Sequence[float]],
+                    lo: float = 0.05, hi: float = 50.0, iters: int = 60) -> float:
+    """Fit one scalar temperature by minimising cross-entropy against (possibly soft) targets.
+    Golden-section search on log T; no torch dependency so it runs anywhere."""
+    def nll(t: float) -> float:
+        total = 0.0
+        for z, y in zip(logits, targets):
+            p = softmax(z, t)
+            total -= sum(yi * math.log(max(pi, 1e-12)) for yi, pi in zip(y, p))
+        return total / max(len(logits), 1)
+
+    a, b = math.log(lo), math.log(hi)
+    gr = (math.sqrt(5) - 1) / 2
+    c, d = b - gr * (b - a), a + gr * (b - a)
+    fc, fd = nll(math.exp(c)), nll(math.exp(d))
+    for _ in range(iters):
+        if fc < fd:
+            b, d, fd = d, c, fc
+            c = b - gr * (b - a)
+            fc = nll(math.exp(c))
+        else:
+            a, c, fc = c, d, fd
+            d = a + gr * (b - a)
+            fd = nll(math.exp(d))
+    t = math.exp((a + b) / 2)
+    if t >= hi * 0.99 or t <= lo * 1.01:
+        logging.getLogger("qev.calibrate").warning(
+            "temperature fit hit the %s bound (T=%.4f) on %d rows; the slot logits likely carry no signal for this group",
+            "upper" if t >= hi * 0.99 else "lower", t, len(logits))
+    return t
+
+
+def ece(confidences: Sequence[float], correct: Sequence[float], n_bins: int = 15) -> float:
+    n = len(confidences)
+    if n == 0:
+        return 0.0
+    total = 0.0
+    for b in range(n_bins):
+        lo, hi = b / n_bins, (b + 1) / n_bins
+        idx = [i for i, c in enumerate(confidences) if (lo < c <= hi) or (b == 0 and c <= lo)]
+        if idx:
+            avg_conf = sum(confidences[i] for i in idx) / len(idx)
+            avg_acc = sum(correct[i] for i in idx) / len(idx)
+            total += len(idx) / n * abs(avg_conf - avg_acc)
+    return total
+
+
+def adaptive_ece(confidences: Sequence[float], correct: Sequence[float], n_bins: int = 15) -> float:
+    n = len(confidences)
+    if n == 0:
+        return 0.0
+    order = sorted(range(n), key=lambda i: confidences[i])
+    total = 0.0
+    size = max(1, n // n_bins)
+    for start in range(0, n, size):
+        idx = order[start:start + size]
+        avg_conf = sum(confidences[i] for i in idx) / len(idx)
+        avg_acc = sum(correct[i] for i in idx) / len(idx)
+        total += len(idx) / n * abs(avg_conf - avg_acc)
+    return total
+
+
+def brier(probs: Sequence[float], target: Sequence[float]) -> float:
+    return sum((p - t) ** 2 for p, t in zip(probs, target))
+
+
+def nll(probs: Sequence[float], target: Sequence[float]) -> float:
+    return -sum(t * math.log(max(p, 1e-12)) for p, t in zip(probs, target))
+
+
+def tvd(p: Sequence[float], q: Sequence[float]) -> float:
+    return 0.5 * sum(abs(a - b) for a, b in zip(p, q))