qarar 1.0.2__tar.gz

qarar-1.0.2/PKG-INFO

@@ -0,0 +1,179 @@
+Metadata-Version: 2.4
+Name: qarar
+Version: 1.0.2
+Summary: QARAR: Arabic-first typed-decision models (calibrated distributions over supplied options).
+Author: NOMEDA
+License-Expression: Apache-2.0
+Project-URL: Homepage, https://huggingface.co/nomeda-lab/nomeda-decision-v0.1-beta
+Keywords: arabic,nlp,typed-decisions,calibration,probability-estimation
+Classifier: Programming Language :: Python :: 3
+Classifier: Natural Language :: Arabic
+Classifier: Natural Language :: English
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: torch>=2.2
+Requires-Dist: transformers>=4.44
+Requires-Dist: safetensors>=0.4
+Requires-Dist: huggingface_hub>=0.23
+Requires-Dist: numpy>=1.24
+Requires-Dist: PyYAML
+Provides-Extra: data
+Requires-Dist: datasets>=2.18; extra == "data"
+Requires-Dist: pandas>=2.0; extra == "data"
+Requires-Dist: pyarrow>=14; extra == "data"
+Provides-Extra: train
+Requires-Dist: sentence-transformers>=3.0; extra == "train"
+Requires-Dist: regex; extra == "train"
+Requires-Dist: pydantic>=2.5; extra == "train"
+Provides-Extra: dev
+Requires-Dist: pytest; extra == "dev"
+
+# NOMEDA-Decision v1
+
+Arabic-first **typed-decision** model. Given one *state* it answers **many independent
+questions** by returning a calibrated probability distribution over the **supplied
+options** — it never generates text.
+
+## Install
+
+```bash
+pip install qarar
+```
+
+```python
+from qarar import load          # published alias; `from nomeda import load` also works
+m = load("nomeda-lab/qarar")
+```
+
+Question types share one scoring path:
+
+| type | meaning | options |
+|---|---|---|
+| `choice` | pick one named option | K named options |
+| `score` | rank/level a target | K ordered options (+ `rank_emb`) |
+| `noul` | yes / no / unknown | 2 options (`false`, `true`) |
+
+- Encoder: `silma-ai/silma-embedding-matryoshka-v0.1` (Arabetv02 backbone, D=768, mean
+  pooling), loaded via `AutoModel`/`AutoTokenizer` (no `trust_remote_code`).
+- Non-interference is **exact**: each question cross-attends only to the shared state,
+  never to another question (block-diagonal attention).
+- Encoder calls per request ≤ 2: state once, all question/option texts in one batched pass.
+
+See `docs/MODEL_REPORT.md` for the architecture, measured parameter budget, latency and
+readiness. Diagram: `docs/nomeda_decision_architecture.drawio`.
+
+## Layout
+
+```
+nomeda/            library
+  config.py        NomedaConfig
+  encoder.py       SilmaEncoder (mean pooling, chunking, LRU cache)
+  blocks.py        DecisionBlock / OptionStateBlock / AttnPool / CompatibilityScorer
+  model.py         NomedaModel + param_report()
+  runtime.py       NomedaRuntime.predict / predict_batch, validate_questions
+  loss.py          proper-scoring losses
+  calibrate.py     temperature scaling, abstention thresholds, ECE
+  data/            loader, batching/augmentation, family_map.yaml
+scripts/           audit_data, train, calibrate, evaluate, export, bench_latency
+tests/             invariance, runtime, leakage, pooling-parity, determinism
+configs/           default.yaml
+docs/              MODEL_REPORT, DATA_AUDIT, VERIFIED, OPEN_QUESTIONS, RESULTS
+```
+
+## Install
+
+```bash
+pip install nomeda                       # from PyPI (v1.0.1+)
+# or, for development, from the repository root:
+pip install -e .
+# with the data-prep and training extras:
+pip install -e ".[data,train,dev]"
+```
+
+`pip install nomeda` (or `pip install -e .`) makes `import nomeda` work from any
+directory.
+
+## Usage
+
+**Recommended — load the published weights (calibration included):**
+
+```python
+from nomeda import load
+
+nm = load("nomeda-lab/nomeda-decision-v0.1-beta")   # or a local export directory
+out = nm.predict(
+    state="أنا اتخصم مني مرتين على فاتورة مارس، عايز أرجّع الفلوس النهاردة.",
+    questions={
+        "intent": {"type": "choice", "instructions": "ماذا يريد العميل؟",
+                   "criteria": {"refund": "استرداد المال",
+                                "technical_help": "مشكلة تقنية",
+                                "cancellation": "إلغاء الخدمة"}},
+        "frustration": {"type": "score", "instructions": "ما مدى انزعاج العميل؟",
+                        "criteria": ["هادئ", "قلق", "منزعج", "غاضب جدًا"]},
+        "refund_requested": {"type": "noul", "instructions": "هل يطلب العميل استرداد أمواله؟"},
+    },
+)
+
+out["answers"]["intent"]["choice"]           # -> "refund"
+out["answers"]["intent"]["probabilities"]    # -> {"refund": 0.555, ...}
+out["answers"]["refund_requested"]["noul"]   # -> 0.608
+```
+
+`load()` fetches `temperatures.json` and `thresholds.json` alongside the weights,
+so predictions are calibrated and abstention works. **This is what you want
+almost always.**
+
+**Manual construction** (when you have the weights and calibration already and
+want full control):
+
+```python
+import json
+from nomeda import NomedaConfig, NomedaModel
+from nomeda.runtime import NomedaRuntime
+
+model = NomedaModel(NomedaConfig(), device="cpu")        # or device="cuda"
+
+# temperatures/thresholds are REAL files, not placeholders. Load them:
+temperatures = json.load(open("temperatures.json"))       # {type: {bucket: T}}
+thresholds   = json.load(open("thresholds.json"))         # {type: p_min}
+
+rt = NomedaRuntime(model, temperatures=temperatures,
+                   thresholds=thresholds, device="cpu")
+rt.predict(state, questions)
+```
+
+Without calibration pass `temperatures={}, thresholds={}` (identity temperature,
+no abstention) — never `...`, which raises a `TypeError`.
+
+> **Naming.** The classes were `Nomed*` in the v0.1-beta release. `NomedConfig`,
+> `NomedModel`, `NomedRuntime` and `Nomed` remain available as aliases, so existing
+> code keeps working; new code should use the `Nomeda*` names.
+
+## Pipeline
+
+```bash
+DATA=/path/to/data/final_training        # train.jsonl, validation.jsonl, test.jsonl
+python scripts/train.py       --data-dir $DATA --out checkpoints/run
+python scripts/calibrate.py   --data-dir $DATA --checkpoint checkpoints/run/checkpoint.pt --out artifacts
+python scripts/evaluate.py    --data-dir $DATA --checkpoint checkpoints/run/checkpoint.pt --artifacts artifacts --out docs/RESULTS.md
+python scripts/export.py      --checkpoint checkpoints/run/checkpoint.pt --artifacts artifacts --out dist/nomeda-decision
+```
+
+## Tests
+
+```bash
+PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests -m "not slow"   # fast suite
+PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests -m "slow"       # real-encoder parity
+```
+
+## Kaggle
+
+`kaggle/nomeda_decision_kaggle.ipynb` runs the whole frozen-encoder pipeline (feature
+cache → baselines → training → calibration → test → export/handoff) on a Kaggle GPU.
+Upload the repo as a zip dataset and the `*.jsonl` data as another dataset, attach both,
+enable GPU + Internet, and run the cells top to bottom. See the notebook's first cell.
+
+## License
+
+Code: see `LICENSES.yaml` / repository. Encoder `silma-embedding-matryoshka-v0.1` is
+Apache-2.0 (model card; no standalone LICENSE file upstream).

qarar-1.0.2/README.md

@@ -0,0 +1,149 @@
+# NOMEDA-Decision v1
+
+Arabic-first **typed-decision** model. Given one *state* it answers **many independent
+questions** by returning a calibrated probability distribution over the **supplied
+options** — it never generates text.
+
+## Install
+
+```bash
+pip install qarar
+```
+
+```python
+from qarar import load          # published alias; `from nomeda import load` also works
+m = load("nomeda-lab/qarar")
+```
+
+Question types share one scoring path:
+
+| type | meaning | options |
+|---|---|---|
+| `choice` | pick one named option | K named options |
+| `score` | rank/level a target | K ordered options (+ `rank_emb`) |
+| `noul` | yes / no / unknown | 2 options (`false`, `true`) |
+
+- Encoder: `silma-ai/silma-embedding-matryoshka-v0.1` (Arabetv02 backbone, D=768, mean
+  pooling), loaded via `AutoModel`/`AutoTokenizer` (no `trust_remote_code`).
+- Non-interference is **exact**: each question cross-attends only to the shared state,
+  never to another question (block-diagonal attention).
+- Encoder calls per request ≤ 2: state once, all question/option texts in one batched pass.
+
+See `docs/MODEL_REPORT.md` for the architecture, measured parameter budget, latency and
+readiness. Diagram: `docs/nomeda_decision_architecture.drawio`.
+
+## Layout
+
+```
+nomeda/            library
+  config.py        NomedaConfig
+  encoder.py       SilmaEncoder (mean pooling, chunking, LRU cache)
+  blocks.py        DecisionBlock / OptionStateBlock / AttnPool / CompatibilityScorer
+  model.py         NomedaModel + param_report()
+  runtime.py       NomedaRuntime.predict / predict_batch, validate_questions
+  loss.py          proper-scoring losses
+  calibrate.py     temperature scaling, abstention thresholds, ECE
+  data/            loader, batching/augmentation, family_map.yaml
+scripts/           audit_data, train, calibrate, evaluate, export, bench_latency
+tests/             invariance, runtime, leakage, pooling-parity, determinism
+configs/           default.yaml
+docs/              MODEL_REPORT, DATA_AUDIT, VERIFIED, OPEN_QUESTIONS, RESULTS
+```
+
+## Install
+
+```bash
+pip install nomeda                       # from PyPI (v1.0.1+)
+# or, for development, from the repository root:
+pip install -e .
+# with the data-prep and training extras:
+pip install -e ".[data,train,dev]"
+```
+
+`pip install nomeda` (or `pip install -e .`) makes `import nomeda` work from any
+directory.
+
+## Usage
+
+**Recommended — load the published weights (calibration included):**
+
+```python
+from nomeda import load
+
+nm = load("nomeda-lab/nomeda-decision-v0.1-beta")   # or a local export directory
+out = nm.predict(
+    state="أنا اتخصم مني مرتين على فاتورة مارس، عايز أرجّع الفلوس النهاردة.",
+    questions={
+        "intent": {"type": "choice", "instructions": "ماذا يريد العميل؟",
+                   "criteria": {"refund": "استرداد المال",
+                                "technical_help": "مشكلة تقنية",
+                                "cancellation": "إلغاء الخدمة"}},
+        "frustration": {"type": "score", "instructions": "ما مدى انزعاج العميل؟",
+                        "criteria": ["هادئ", "قلق", "منزعج", "غاضب جدًا"]},
+        "refund_requested": {"type": "noul", "instructions": "هل يطلب العميل استرداد أمواله؟"},
+    },
+)
+
+out["answers"]["intent"]["choice"]           # -> "refund"
+out["answers"]["intent"]["probabilities"]    # -> {"refund": 0.555, ...}
+out["answers"]["refund_requested"]["noul"]   # -> 0.608
+```
+
+`load()` fetches `temperatures.json` and `thresholds.json` alongside the weights,
+so predictions are calibrated and abstention works. **This is what you want
+almost always.**
+
+**Manual construction** (when you have the weights and calibration already and
+want full control):
+
+```python
+import json
+from nomeda import NomedaConfig, NomedaModel
+from nomeda.runtime import NomedaRuntime
+
+model = NomedaModel(NomedaConfig(), device="cpu")        # or device="cuda"
+
+# temperatures/thresholds are REAL files, not placeholders. Load them:
+temperatures = json.load(open("temperatures.json"))       # {type: {bucket: T}}
+thresholds   = json.load(open("thresholds.json"))         # {type: p_min}
+
+rt = NomedaRuntime(model, temperatures=temperatures,
+                   thresholds=thresholds, device="cpu")
+rt.predict(state, questions)
+```
+
+Without calibration pass `temperatures={}, thresholds={}` (identity temperature,
+no abstention) — never `...`, which raises a `TypeError`.
+
+> **Naming.** The classes were `Nomed*` in the v0.1-beta release. `NomedConfig`,
+> `NomedModel`, `NomedRuntime` and `Nomed` remain available as aliases, so existing
+> code keeps working; new code should use the `Nomeda*` names.
+
+## Pipeline
+
+```bash
+DATA=/path/to/data/final_training        # train.jsonl, validation.jsonl, test.jsonl
+python scripts/train.py       --data-dir $DATA --out checkpoints/run
+python scripts/calibrate.py   --data-dir $DATA --checkpoint checkpoints/run/checkpoint.pt --out artifacts
+python scripts/evaluate.py    --data-dir $DATA --checkpoint checkpoints/run/checkpoint.pt --artifacts artifacts --out docs/RESULTS.md
+python scripts/export.py      --checkpoint checkpoints/run/checkpoint.pt --artifacts artifacts --out dist/nomeda-decision
+```
+
+## Tests
+
+```bash
+PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests -m "not slow"   # fast suite
+PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests -m "slow"       # real-encoder parity
+```
+
+## Kaggle
+
+`kaggle/nomeda_decision_kaggle.ipynb` runs the whole frozen-encoder pipeline (feature
+cache → baselines → training → calibration → test → export/handoff) on a Kaggle GPU.
+Upload the repo as a zip dataset and the `*.jsonl` data as another dataset, attach both,
+enable GPU + Internet, and run the cells top to bottom. See the notebook's first cell.
+
+## License
+
+Code: see `LICENSES.yaml` / repository. Encoder `silma-embedding-matryoshka-v0.1` is
+Apache-2.0 (model card; no standalone LICENSE file upstream).

qarar-1.0.2/nomeda/__init__.py

@@ -0,0 +1,71 @@
+"""NOMEDA-Decision v1: one Arabic state -> many independent typed decisions."""
+from __future__ import annotations
+
+import json
+from pathlib import Path
+
+from .config import (NomedaConfig, NomedConfig, DEFAULT_CONFIG, k_bucket,
+                     K_BUCKETS, K_BUCKET_NAMES)
+from .model import NomedaModel, NomedModel
+from .runtime import NomedaRuntime, NomedRuntime, validate_questions
+
+__version__ = "1.0.0"
+__all__ = ["load", "Nomeda", "Nomed",
+           "NomedaConfig", "NomedaModel", "NomedaRuntime",
+           "NomedConfig", "NomedModel", "NomedRuntime",
+           "validate_questions", "k_bucket"]
+
+
+class Nomeda:
+    """Loaded model + calibration; ``predict`` mirrors the Laya output shape."""
+
+    def __init__(self, model: NomedaModel, temperatures: dict | None = None,
+                 thresholds: dict | None = None, device: str = "cpu"):
+        self.model = model
+        self.encoder_id = model.cfg.encoder_id
+        self.encoder_revision = model.cfg.encoder_revision
+        self.runtime = NomedaRuntime(model, temperatures or {}, thresholds or {}, device=device)
+
+    def predict(self, state, questions):
+        return self.runtime.predict(state, questions)
+
+    def predict_batch(self, states, questions):
+        return self.runtime.predict_batch(states, questions)
+
+    def param_report(self) -> dict:
+        return self.model.param_report()
+
+
+def load(path_or_hub_id: str, device: str = "auto", tiny: bool = False) -> Nomeda:
+    """Load a NOMEDA-Decision model from a local export directory or a Hub id."""
+    if device == "auto":
+        import torch
+        device = "cuda" if torch.cuda.is_available() else "cpu"
+    if Path(path_or_hub_id).is_dir():
+        root = Path(path_or_hub_id)
+        raw = json.loads((root / "config.json").read_text(encoding="utf-8"))
+        cfg = NomedaConfig(**{k: v for k, v in raw.items() if k in NomedaConfig.__dataclass_fields__})
+        enc_dir = root / "encoder"
+        model = NomedaModel(cfg, device=device, tiny=tiny)
+        if enc_dir.exists():
+            from transformers import AutoModel, AutoTokenizer
+            model.encoder.tok = AutoTokenizer.from_pretrained(enc_dir)
+            model.encoder.model = AutoModel.from_pretrained(enc_dir).to(device)
+        head = root / "decision_head.safetensors"
+        if head.exists():
+            from safetensors.torch import load_file
+            state = load_file(str(head))
+            model.load_state_dict(state, strict=False)
+        temps = json.loads((root / "temperatures.json").read_text(encoding="utf-8")) if (root / "temperatures.json").exists() else {}
+        thr = json.loads((root / "thresholds.json").read_text(encoding="utf-8")) if (root / "thresholds.json").exists() else {}
+        model.eval()
+        model.to(device)
+        return Nomeda(model, temps, thr, device=device)
+    # Hub id
+    from huggingface_hub import snapshot_download
+    d = snapshot_download(repo_id=path_or_hub_id)
+    return load(d, device=device, tiny=tiny)
+
+
+# ---- backward-compatible alias (published as `Nomed` in v0.1-beta) --------
+Nomed = Nomeda

qarar-1.0.2/nomeda/blocks.py

@@ -0,0 +1,112 @@
+"""Decision blocks, option-state block and attention pooling (§3.4–3.6)."""
+from __future__ import annotations
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+
+class MultiheadAttention(nn.Module):
+    def __init__(self, d: int, n_heads: int, dropout: float = 0.0):
+        super().__init__()
+        assert d % n_heads == 0
+        self.d, self.h = d, n_heads
+        self.dh = d // n_heads
+        self.q = nn.Linear(d, d)
+        self.k = nn.Linear(d, d)
+        self.v = nn.Linear(d, d)
+        self.o = nn.Linear(d, d)
+        self.dropout = dropout
+
+    def forward(self, q, k, v, q_mask=None, k_mask=None):
+        B, Lq, _ = q.shape
+        Lk = k.shape[1]
+        Q = self.q(q).view(B, Lq, self.h, self.dh).transpose(1, 2)
+        K = self.k(k).view(B, Lk, self.h, self.dh).transpose(1, 2)
+        V = self.v(v).view(B, Lk, self.h, self.dh).transpose(1, 2)
+        attn_mask = None
+        if k_mask is not None:
+            attn_mask = torch.zeros(B, 1, 1, Lk, device=q.device, dtype=Q.dtype)
+            attn_mask = attn_mask.masked_fill(~k_mask[:, None, None, :].bool(), -1e4)
+        out = F.scaled_dot_product_attention(
+            Q, K, V, attn_mask=attn_mask,
+            dropout_p=self.dropout if self.training else 0.0)
+        out = out.transpose(1, 2).reshape(B, Lq, self.d)
+        return self.o(out)
+
+
+class FFN(nn.Module):
+    def __init__(self, d: int, mult: int, dropout: float):
+        super().__init__()
+        self.net = nn.Sequential(nn.Linear(d, d * mult), nn.GELU(), nn.Dropout(dropout), nn.Linear(d * mult, d))
+
+    def forward(self, x):
+        return self.net(x)
+
+
+class DecisionBlock(nn.Module):
+    """Self-attn within a question -> cross-attn to its own state -> FFN (pre-norm)."""
+
+    def __init__(self, d, n_heads, mult, dropout):
+        super().__init__()
+        self.ln1, self.ln2, self.ln3 = nn.LayerNorm(d), nn.LayerNorm(d), nn.LayerNorm(d)
+        self.self_attn = MultiheadAttention(d, n_heads, dropout)
+        self.cross_attn = MultiheadAttention(d, n_heads, dropout)
+        self.ffn = FFN(d, mult, dropout)
+
+    def forward(self, Q, q_mask, S_pad, S_mask, state_idx):
+        Q = Q + self.self_attn(self.ln1(Q), self.ln1(Q), self.ln1(Q), q_mask=q_mask, k_mask=q_mask)
+        # cross-attention grouped by state: gather each question's own state (vectorised).
+        # Equivalent to a per-state loop but without one kernel launch + sync per state.
+        x = self.ln2(Q)
+        k = S_pad.index_select(0, state_idx)
+        kmask = S_mask.index_select(0, state_idx)
+        Q = Q + self.cross_attn(x, k, k, k_mask=kmask)
+        Q = Q + self.ffn(self.ln3(Q))
+        return Q
+
+
+class OptionStateBlock(nn.Module):
+    def __init__(self, d, n_heads, mult, dropout):
+        super().__init__()
+        self.ln1, self.ln2 = nn.LayerNorm(d), nn.LayerNorm(d)
+        self.cross_attn = MultiheadAttention(d, n_heads, dropout)
+        self.ffn = FFN(d, mult, dropout)
+
+    def forward(self, E, S_pad, S_mask, state_idx):
+        x = self.ln1(E).unsqueeze(1)  # [Nopt,1,d]
+        k = S_pad.index_select(0, state_idx)
+        kmask = S_mask.index_select(0, state_idx)
+        out = self.cross_attn(x, k, k, k_mask=kmask)
+        E = E + out.squeeze(1)
+        E = E + self.ffn(self.ln2(E))
+        return E
+
+
+class AttnPool(nn.Module):
+    def __init__(self, d):
+        super().__init__()
+        self.query = nn.Parameter(torch.randn(d) * 0.02)
+        self.ln = nn.LayerNorm(d)
+
+    def forward(self, Q, mask):
+        attn = (Q @ self.query) / (Q.shape[-1] ** 0.5)  # [B,L]
+        attn = attn.masked_fill(~mask.bool(), -1e4)
+        p = torch.softmax(attn, dim=-1).unsqueeze(1)
+        pooled = torch.bmm(p, Q).squeeze(1)
+        return self.ln(pooled)
+
+
+class CompatibilityScorer(nn.Module):
+    def __init__(self, d, dropout):
+        super().__init__()
+        self.ln = nn.LayerNorm(4 * d)
+        self.w1 = nn.Linear(4 * d, d)
+        self.w2 = nn.Linear(d, 1)
+        self.dropout = nn.Dropout(dropout)
+
+    def forward(self, q, E):
+        h = torch.cat([q, E, q * E, (q - E).abs()], dim=-1)
+        h = self.ln(h)
+        h = self.dropout(F.gelu(self.w1(h)))
+        return self.w2(h).squeeze(-1)

qarar-1.0.2/nomeda/calibrate.py

@@ -0,0 +1,165 @@
+"""Calibration: temperatures per (type, K-bucket) and abstention thresholds (§6)."""
+from __future__ import annotations
+
+import hashlib
+import json
+import math
+from collections import defaultdict
+from typing import Dict, List, Optional, Tuple
+
+import torch
+import torch.nn.functional as F
+
+from .config import k_bucket
+
+
+def ece(probs: List[float], correct: List[int], n_bins: int = 15) -> float:
+    """Expected calibration error, confidence = max(p), equal-width bins (Laya's definition)."""
+    if not probs:
+        return 0.0
+    total = len(probs)
+    out = 0.0
+    for b in range(n_bins):
+        lo, hi = b / n_bins, (b + 1) / n_bins
+        idx = [i for i, p in enumerate(probs) if (p > lo or (b == 0 and p >= lo)) and p <= hi]
+        if not idx:
+            continue
+        conf = sum(probs[i] for i in idx) / len(idx)
+        acc = sum(correct[i] for i in idx) / len(idx)
+        out += (len(idx) / total) * abs(conf - acc)
+    return out
+
+
+def fit_temperature(logits_list, targets_list, clamp: Tuple[float, float] = (0.5, 5.0)) -> float:
+    """Minimise NLL over a single temperature; logits_list holds per-item 1-D tensors
+    (K may vary between items)."""
+    logT = torch.zeros(1, requires_grad=True)
+    opt = torch.optim.LBFGS([logT], lr=0.1, max_iter=50)
+    tg = [int(t) for t in targets_list]
+
+    def closure():
+        opt.zero_grad()
+        T = torch.exp(logT).clamp(*clamp)
+        loss = 0.0
+        for l, t in zip(logits_list, tg):
+            loss = loss + F.cross_entropy((l / T).unsqueeze(0).float(), torch.tensor([t]))
+        loss = loss / max(1, len(logits_list))
+        loss.backward()
+        return loss
+
+    opt.step(closure)
+    with torch.no_grad():
+        return float(torch.exp(logT).clamp(*clamp))
+
+
+def fit_temperatures(buckets: Dict[Tuple[str, str], Tuple[list, list]],
+                     min_items: int = 200) -> dict:
+    """buckets: {(type, K-bucket): ([logits_1d...], [targets...])} -> temperatures.json dict."""
+    out: Dict[str, Dict[str, float]] = {}
+    per_type: Dict[str, Tuple[list, list]] = {}
+    for (qtype, kb), (logits, targets) in buckets.items():
+        per_type.setdefault(qtype, ([], []))
+        per_type[qtype][0].extend(logits)
+        per_type[qtype][1].extend(targets)
+        if len(logits) < min_items:
+            continue
+        out.setdefault(qtype, {})[kb] = fit_temperature(logits, targets)
+    for qtype, (ls, ts) in per_type.items():
+        out.setdefault(qtype, {})["default"] = fit_temperature(ls, ts)
+    return out
+
+
+def fit_abstention(conf: List[float], correct: List[int], target_acc: float = 0.95) -> float:
+    """Smallest threshold so answered questions reach accuracy >= target_acc."""
+    pairs = sorted(zip(conf, correct), reverse=True)
+    if not pairs:
+        return 0.0
+    best_t = 1.0
+    n_ok = 0
+    for i, (c, ok) in enumerate(pairs):
+        n_ok += ok
+        acc = n_ok / (i + 1)
+        if acc >= target_acc:
+            best_t = c
+    return float(best_t)
+
+
+def coverage(conf: List[float], threshold: float) -> float:
+    if not conf:
+        return 0.0
+    return sum(1 for c in conf if c >= threshold) / len(conf)
+
+
+def save(path: str, obj: dict) -> None:
+    with open(path, "w", encoding="utf-8") as f:
+        json.dump(obj, f, ensure_ascii=False, indent=2)
+
+
+# ---- fair calibrated comparison (cal_cv, §0.4) ---------------------------
+def group_of(record_id, view_group) -> str:
+    """Fold-group id: view_group and V2 en↔ar pairs stay together."""
+    if view_group:
+        return f"vg:{view_group}"
+    rid = str(record_id)
+    if ":en:" in rid or ":ar:" in rid:
+        return "pair:" + rid.replace(":en:", ":*:").replace(":ar:", ":*:")
+    return f"solo:{rid}"
+
+
+def _fold_of(group_id: str, folds: int) -> int:
+    h = int(hashlib.sha1(str(group_id).encode()).hexdigest()[:8], 16)
+    return h % folds
+
+
+def _fit_temps_from_items(items: List[dict], min_items: int,
+                          clamp: Tuple[float, float] = (0.5, 5.0)) -> dict:
+    buckets: Dict[Tuple[str, str], Tuple[list, list]] = defaultdict(lambda: ([], []))
+    per_type: Dict[str, Tuple[list, list]] = defaultdict(lambda: ([], []))
+    for it in items:
+        kb = it["kb"]
+        buckets[(it["type"], kb)][0].append(it["logits"])
+        buckets[(it["type"], kb)][1].append(it["idx"])
+        per_type[it["type"]][0].append(it["logits"])
+        per_type[it["type"]][1].append(it["idx"])
+    out: Dict[str, dict] = {}
+    for (qt, kb), (L, T) in buckets.items():
+        if len(L) >= min_items:
+            out.setdefault(qt, {})[kb] = fit_temperature(L, T, clamp)
+    for qt, (L, T) in per_type.items():
+        out.setdefault(qt, {})["default"] = fit_temperature(L, T, clamp)
+    return out
+
+
+def _score_items(items: List[dict], temps: dict) -> Tuple[float, float, List[float], List[int]]:
+    nll = 0.0
+    confs, oks = [], []
+    for it in items:
+        T = float(temps.get(it["type"], {}).get(it["kb"], temps.get(it["type"], {}).get("default", 1.0)))
+        p = torch.softmax(it["logits"] / T, dim=-1)
+        nll += -math.log(max(float(p[it["idx"]]), 1e-12))
+        confs.append(float(p.max()))
+        oks.append(int(int(torch.argmax(p)) == it["idx"]))
+    return nll / max(1, len(items)), ece(confs, oks), confs, oks
+
+
+def cal_cv_from_rows(rows: List[dict], folds: int = 2, min_items: int = 100) -> dict:
+    """Fit on fold A / score on B then swap; average NLL and ECE (§0.4)."""
+    items = []
+    for r in rows:
+        if "logits" not in r or "labels" not in r:
+            continue
+        items.append({
+            "group": group_of(r.get("record_id"), r.get("view_group")),
+            "type": r["type"], "kb": k_bucket(len(r["labels"])),
+            "logits": torch.tensor(r["logits"]), "idx": r["labels"].index(r["gold"]),
+        })
+    fold_of = [_fold_of(it["group"], folds) for it in items]
+    nlls, eces = [], []
+    for f in range(folds):
+        fit = [it for it, ff in zip(items, fold_of) if ff != f]
+        ev = [it for it, ff in zip(items, fold_of) if ff == f]
+        temps = _fit_temps_from_items(fit, min_items)
+        n, e, _, _ = _score_items(ev, temps)
+        nlls.append(n); eces.append(e)
+    return {"nll_cal_cv": round(sum(nlls) / folds, 4),
+            "ece_cal_cv": round(sum(eces) / folds, 4)}