decido-ai 1.0.0__tar.gz

decido_ai-1.0.0/PKG-INFO

@@ -0,0 +1,62 @@
+Metadata-Version: 2.4
+Name: decido-ai
+Version: 1.0.0
+Summary: Calibrated decisions (yes/no, pick-one, ordered scale) from a document — Decido models from the Hugging Face Hub, on a laptop CPU or a GPU.
+License: Apache-2.0
+Project-URL: Homepage, https://huggingface.co/decido-ai
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: numpy>=1.24
+Requires-Dist: tokenizers>=0.20
+Requires-Dist: huggingface_hub>=0.25
+Provides-Extra: gpu
+Requires-Dist: torch>=2.4; extra == "gpu"
+Requires-Dist: transformers>=5.17; extra == "gpu"
+Requires-Dist: accelerate; extra == "gpu"
+Provides-Extra: xl
+Requires-Dist: torch>=2.6; extra == "xl"
+Requires-Dist: transformers>=5.22; extra == "xl"
+Requires-Dist: accelerate; extra == "xl"
+Requires-Dist: peft>=0.21; extra == "xl"
+Requires-Dist: triton>=3.2; extra == "xl"
+Requires-Dist: safetensors; extra == "xl"
+Provides-Extra: pro
+Requires-Dist: vllm==0.31.0; extra == "pro"
+Requires-Dist: transformers>=5.17; extra == "pro"
+
+# decido
+
+**Calibrated decisions from a document — on a laptop CPU or a GPU.** Give Decido a document and questions (yes/no, pick-one,
+or an ordered scale) and get a probability for every answer. This package runs the [Decido 1 models](https://huggingface.co/decido-ai)
+from the Hugging Face Hub.
+
+```bash
+pip install decido-ai
+decido build-runtime      # only on Linux / Intel Macs (one-time, needs cmake + a C++ compiler); Apple Silicon Macs: skip
+```
+```python
+from decido import Decido
+
+m = Decido("decido-ai/decido-1-fast-1.7b")      # or decido-1-26b-a4b (most accurate on CPU), decido-1-ultra-0.6b (fastest)
+m.decide("Customer: I was charged twice for order 4821. Please refund one of the charges today.",
+         {"refund": "Is the customer asking for a refund?",
+          "team": {"q": "Which team should handle this?", "options": ["billing", "shipping", "tech"]},
+          "urgency": {"q": "How urgent is it?", "scale": ["low", "normal", "urgent"]}})
+```
+```bash
+decido models                                   # the family, with scores and latency
+decido run decido-ai/decido-1-fast-1.7b "Customer: charged twice" -q refund="Is a refund requested?"
+decido serve decido-ai/decido-1-26b-a4b         # Jev-compatible HTTP API on http://127.0.0.1:8080/v1/systemone
+```
+
+| model | runs on | six-group score (Jev 1.13 = 83.85) | latency (M4 Pro CPU, 1k-token document, 5 questions) |
+|---|---|---|---|
+| [decido-1-26b-a4b](https://huggingface.co/decido-ai/decido-1-26b-a4b) | laptop CPU (~30 GB RAM) · GPU | 85.38 | 4.1 s |
+| [decido-1-fast-1.7b](https://huggingface.co/decido-ai/decido-1-fast-1.7b) | laptop CPU | 73.01 | 0.64 s |
+| [decido-1-ultra-0.6b](https://huggingface.co/decido-ai/decido-1-ultra-0.6b) | laptop CPU | 62.16 | 0.41 s |
+| [decido-1-think-26b-a4b](https://huggingface.co/decido-ai/decido-1-think-26b-a4b) | GPU · laptop CPU (slow) | 88.71 (GPU) | see card |
+| [decido-1-xl](https://huggingface.co/decido-ai/decido-1-xl) | 4× A100 80 GB | 87.25 (GPU) | — |
+| [decido-1-pro](https://huggingface.co/decido-ai/decido-1-pro) | GPU | 89.57 (GPU) | — |
+
+GPU extras: `pip install "decido-ai[gpu]"` (transformers), `"decido-ai[xl]"`, `"decido-ai[pro]"` (vLLM). Results, calibration,
+disclosures and limitations are on each model card. License: Apache-2.0; the CPU runtime links llama.cpp (MIT).

decido_ai-1.0.0/README.md

@@ -0,0 +1,36 @@
+# decido
+
+**Calibrated decisions from a document — on a laptop CPU or a GPU.** Give Decido a document and questions (yes/no, pick-one,
+or an ordered scale) and get a probability for every answer. This package runs the [Decido 1 models](https://huggingface.co/decido-ai)
+from the Hugging Face Hub.
+
+```bash
+pip install decido-ai
+decido build-runtime      # only on Linux / Intel Macs (one-time, needs cmake + a C++ compiler); Apple Silicon Macs: skip
+```
+```python
+from decido import Decido
+
+m = Decido("decido-ai/decido-1-fast-1.7b")      # or decido-1-26b-a4b (most accurate on CPU), decido-1-ultra-0.6b (fastest)
+m.decide("Customer: I was charged twice for order 4821. Please refund one of the charges today.",
+         {"refund": "Is the customer asking for a refund?",
+          "team": {"q": "Which team should handle this?", "options": ["billing", "shipping", "tech"]},
+          "urgency": {"q": "How urgent is it?", "scale": ["low", "normal", "urgent"]}})
+```
+```bash
+decido models                                   # the family, with scores and latency
+decido run decido-ai/decido-1-fast-1.7b "Customer: charged twice" -q refund="Is a refund requested?"
+decido serve decido-ai/decido-1-26b-a4b         # Jev-compatible HTTP API on http://127.0.0.1:8080/v1/systemone
+```
+
+| model | runs on | six-group score (Jev 1.13 = 83.85) | latency (M4 Pro CPU, 1k-token document, 5 questions) |
+|---|---|---|---|
+| [decido-1-26b-a4b](https://huggingface.co/decido-ai/decido-1-26b-a4b) | laptop CPU (~30 GB RAM) · GPU | 85.38 | 4.1 s |
+| [decido-1-fast-1.7b](https://huggingface.co/decido-ai/decido-1-fast-1.7b) | laptop CPU | 73.01 | 0.64 s |
+| [decido-1-ultra-0.6b](https://huggingface.co/decido-ai/decido-1-ultra-0.6b) | laptop CPU | 62.16 | 0.41 s |
+| [decido-1-think-26b-a4b](https://huggingface.co/decido-ai/decido-1-think-26b-a4b) | GPU · laptop CPU (slow) | 88.71 (GPU) | see card |
+| [decido-1-xl](https://huggingface.co/decido-ai/decido-1-xl) | 4× A100 80 GB | 87.25 (GPU) | — |
+| [decido-1-pro](https://huggingface.co/decido-ai/decido-1-pro) | GPU | 89.57 (GPU) | — |
+
+GPU extras: `pip install "decido-ai[gpu]"` (transformers), `"decido-ai[xl]"`, `"decido-ai[pro]"` (vLLM). Results, calibration,
+disclosures and limitations are on each model card. License: Apache-2.0; the CPU runtime links llama.cpp (MIT).

decido_ai-1.0.0/pyproject.toml

@@ -0,0 +1,29 @@
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "decido-ai"
+version = "1.0.0"
+description = "Calibrated decisions (yes/no, pick-one, ordered scale) from a document — Decido models from the Hugging Face Hub, on a laptop CPU or a GPU."
+readme = "README.md"
+requires-python = ">=3.9"
+license = {text = "Apache-2.0"}
+dependencies = ["numpy>=1.24", "tokenizers>=0.20", "huggingface_hub>=0.25"]
+
+[project.optional-dependencies]
+gpu = ["torch>=2.4", "transformers>=5.17", "accelerate"]
+xl = ["torch>=2.6", "transformers>=5.22", "accelerate", "peft>=0.21", "triton>=3.2", "safetensors"]
+pro = ["vllm==0.31.0", "transformers>=5.17"]
+
+[project.scripts]
+decido = "decido.cli:main"
+
+[project.urls]
+Homepage = "https://huggingface.co/decido-ai"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+decido = ["_bin/*", "csrc/*.cpp"]

decido_ai-1.0.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

decido_ai-1.0.0/src/decido/__init__.py

@@ -0,0 +1,5 @@
+"""Decido: calibrated decisions (yes/no, pick-one, ordered scale) from a document. https://huggingface.co/decido-ai"""
+from decido.api import Decido
+
+__version__ = "1.0.0"
+__all__ = ["Decido", "__version__"]

decido_ai-1.0.0/src/decido/_vendor/ds/derive.py

@@ -0,0 +1,42 @@
+"""Jev 1.13 wire answers + derived fields (formulas from research/jev-teardown.md §4), computed unrounded."""
+
+from __future__ import annotations
+
+import numpy as np
+
+
+def choice_confidence(p) -> float:
+    n = len(p)
+    if n <= 1:
+        return 1.0
+    return float((n * max(p) - 1) / (n - 1))
+
+
+def score_confidence(p) -> float:
+    p = np.asarray(p, dtype=float)
+    n = len(p)
+    if n <= 1:
+        return 1.0
+    m = int(np.argmax(p))
+    idx = np.arange(n)
+    mad_unif = float(np.mean(np.abs(idx - (n - 1) / 2)))
+    return float(max(0.0, 1.0 - float((p * np.abs(idx - m)).sum()) / mad_unif))
+
+
+def wire_answer(q: dict, p, round_dp: int | None = None) -> dict:
+    """p is aligned with eval.parity.common.option_keys(q)."""
+    p = np.asarray(p, dtype=float)
+    r = (lambda x: round(float(x), round_dp)) if round_dp is not None else float
+    t = q["type"]
+    if t == "noul":
+        return {"type": "noul", "noul": r(p[1])}
+    if t == "choice":
+        keys = list(q["criteria"].keys())
+        return {"type": "choice", "choice": keys[int(np.argmax(p))],
+                "probabilities": {k: r(v) for k, v in zip(keys, p)}, "confidence": r(choice_confidence(p))}
+    if t == "score":
+        levels = list(q["criteria"])
+        return {"type": "score", "score": r(float((np.arange(len(p)) * p).sum())),
+                "legend": {str(i): levels[i] for i in range(len(levels))},
+                "probabilities": {str(i): r(v) for i, v in enumerate(p)}, "confidence": r(score_confidence(p))}
+    raise ValueError(t)

decido_ai-1.0.0/src/decido/_vendor/ds/predictor_v4.py

@@ -0,0 +1,85 @@
+"""DeepSeek-V4-Flash predictor for the parity harness and gold dev (GPU, HF eager forward via verdict.v4).
+
+    VERDICT_ALLOW_GPU=1 python -m eval.parity.run --allow-gpu --batch-size 2048 \
+        --predictor verdict.predictor_v4:VerdictV4 --predictor-kw adapter=runs/train/dsD/frac0.6-stepN
+
+Every question is scored as a whole prompt (state prefix + one question [+ first label symbol for >26 options]); V4's
+compressor cache is stateful, so there is no prefix reuse (cost = full re-prefill per question). Batches are
+length-sorted under a padded-token and a padded B*L^2 budget. The LoRA adapter stays unmerged (peft), FP4 experts stay
+packed. Throughput (prompts, tokens, tok/s) is appended to `stats` after every batch call.
+"""
+
+from __future__ import annotations
+
+import json
+import time
+
+from decido._vendor.ds.readout import Readout
+from decido._vendor.ds.template import ChatFormat
+
+
+class VerdictV4:
+    def __init__(self, model: str = "deepseek-ai/DeepSeek-V4-Flash-0731", adapter: str | None = None, perm: int = 1,
+                 temps=None, max_prompt_tokens: int = 8192, max_tokens: int = 32768, max_s2: int = 16_000_000,
+                 attn: str = "band", on_overflow: str = "truncate", pipeline_mb: int = 8):
+        import torch
+        from decido._vendor.ds.v4.loader import load_v4
+        self.model, self.tok, self.info = load_v4(model, log=print)
+        if attn == "band":
+            from decido._vendor.ds.v4.attention import register
+            register(self.model)
+        if adapter:
+            from peft import PeftModel
+            from decido._vendor.ds.template import check_model_template
+            check_model_template(adapter, "verdict-ds-t1")
+            self.model = PeftModel.from_pretrained(self.model, adapter, is_trainable=False)
+        self.model.eval()
+        torch.set_grad_enabled(False)
+        if isinstance(temps, str) and temps:
+            temps = json.load(open(temps))
+        self.cf = ChatFormat(self.tok, family="deepseek_v4", version="verdict-ds-t1")
+        self.readout = Readout(self.cf, perm=perm, temps=temps or None, max_prompt_tokens=max_prompt_tokens,
+                               on_overflow=on_overflow)
+        self.kw = {"max_tokens": max_tokens, "max_s2": max_s2, "pad_id": self.tok.pad_token_id or 0,
+                   "pipeline_mb": int(pipeline_mb)}
+        self.stats = []
+
+    def _base(self):
+        m = self.model
+        return m.get_base_model() if hasattr(m, "get_base_model") else m
+
+    def predict_batch(self, requests):
+        from decido._vendor.ds.v4.score import score_plans
+        plans, out_idx, outs = [], [], [None] * len(requests)
+        for i, r in enumerate(requests):
+            try:
+                plans.append(self.readout.plan(r))
+                out_idx.append(i)
+            except ValueError as e:
+                outs[i] = {"__error__": str(e)[:200]}
+        t0 = time.time()
+        rec = {}
+        res = score_plans(self._base(), self.readout, plans, log=lambda m: rec.setdefault("log", m), **self.kw)
+        for i, pl, r in zip(out_idx, plans, res):
+            raw = r["answers"]
+            from decido._vendor.ds.derive import wire_answer
+            import numpy as np
+            resp = {"answers": {qid: wire_answer(pl.request["questions"][qid], np.asarray(p)) for qid, p in raw.items()}}
+            if "_logp" in r:
+                resp["_logp"] = r["_logp"]
+            if pl.truncated_tokens:
+                resp["truncated_state_tokens"] = pl.truncated_tokens
+            outs[i] = resp
+        self.stats.append({"requests": len(requests), "s": round(time.time() - t0, 2), **rec})
+        return outs
+
+    def predict(self, request):
+        return self.predict_batch([request])[0]
+
+    def predict_raw(self, requests):
+        """Raw (temperature-applied) distributions + pre-temperature log-probs, for gold dev / calib."""
+        from decido._vendor.ds.v4.score import score_plans
+        import os
+        os.environ["VERDICT_DUMP_DIST"] = "1"
+        plans = [self.readout.plan(r) for r in requests]
+        return score_plans(self._base(), self.readout, plans, **self.kw)

decido_ai-1.0.0/src/decido/_vendor/ds/readout.py

@@ -0,0 +1,175 @@
+"""Backend-agnostic verbalizer readout: request -> token queries -> calibrated distributions.
+
+A backend only has to answer two kinds of query on (prefix_ids + suffix_ids [+ extra_ids]):
+  "log-probabilities of these token ids at the next position".
+Everything else (labels, variants, permutation averaging, temperature, wire format) lives here so the
+vLLM, HF and llama.cpp paths produce the same numbers up to backend numerics.
+"""
+
+from __future__ import annotations
+
+import math
+import os
+from dataclasses import dataclass, field
+
+import numpy as np
+
+from decido._vendor.ds.derive import wire_answer
+from decido._vendor.ds.template import LabelTrie
+
+
+def _lse(xs):
+    m = max(xs)
+    if m == -math.inf:
+        return -math.inf
+    return m + math.log(sum(math.exp(x - m) for x in xs))
+
+
+def request_key(request: dict) -> str:
+    import hashlib
+    import json
+    return hashlib.sha256(json.dumps({"state": request["state"], "questions": request["questions"]}, sort_keys=True,
+                                     ensure_ascii=False).encode()).hexdigest()[:32]
+
+
+def cyclic_orders(k: int, m: int) -> list[list[int]]:
+    m = max(1, min(m, k))
+    shifts = sorted({round(j * k / m) % k for j in range(m)})
+    return [[(i + s) % k for i in range(k)] for s in shifts]
+
+
+@dataclass
+class Query:
+    qid: str
+    order: list[int]
+    suffix_ids: list[int]
+    trie: LabelTrie
+    step1: dict = field(default_factory=dict)            # token id -> logprob
+    step2: dict = field(default_factory=dict)            # first symbol -> {token id -> logprob}
+
+
+@dataclass
+class Plan:
+    request: dict
+    prefix_ids: list[int]
+    queries: list[Query]
+    truncated_tokens: int = 0
+
+
+class Readout:
+    def __init__(self, chat_format, perm: int = 1, temps: dict | None = None, max_prompt_tokens: int = 16000,
+                 on_overflow: str = "error"):
+        self.cf = chat_format
+        self.tok = chat_format.tok
+        self.perm = int(perm)
+        self.temps = temps or {}
+        self.max_prompt_tokens = max_prompt_tokens
+        self.on_overflow = on_overflow       # "error" (Jev-like max_tokens_exceeded) or "truncate" (flagged)
+        self._trie_cache = {}
+        self._enc_cache = {}
+
+    def trie(self, labels: list[str]) -> LabelTrie:
+        key = tuple(labels)
+        if key not in self._trie_cache:
+            self._trie_cache[key] = LabelTrie(self.tok, labels)
+        return self._trie_cache[key]
+
+    def _enc(self, text: str) -> list[int]:
+        ids = self._enc_cache.get(text)
+        if ids is None:
+            ids = self.cf.encode(text)
+            if len(self._enc_cache) < 200000:
+                self._enc_cache[text] = ids
+        return ids
+
+    @staticmethod
+    def validate(request: dict):
+        """Jev request rules (research/jev-teardown.md §2.2): choice 2-255 options, score 2-10 levels."""
+        qs = request.get("questions")
+        if not isinstance(qs, dict) or not qs:
+            raise ValueError("invalid_request: questions must be a non-empty map")
+        for qid, q in qs.items():
+            t = q.get("type")
+            if t not in ("noul", "choice", "score"):
+                raise ValueError(f"invalid_request: question {qid!r} has unsupported type {t!r}")
+            if q.get("instructions") in (None, ""):
+                raise ValueError(f"invalid_request: question {qid!r} needs instructions")
+            if t == "choice" and not (isinstance(q.get("criteria"), dict) and 2 <= len(q["criteria"]) <= 255):
+                raise ValueError(f"invalid_request: choice {qid!r} needs 2-255 options")
+            if t == "score" and not (isinstance(q.get("criteria"), list) and 2 <= len(q["criteria"]) <= 10):
+                raise ValueError(f"invalid_request: score {qid!r} needs 2-10 levels")
+
+    def plan(self, request: dict, perm: int | None = None) -> Plan:
+        self.validate(request)
+        perm = self.perm if perm is None else perm
+        prefix_ids = self.cf.encode(self.cf.prefix_text(request["state"]))
+        queries = []
+        max_suffix = 0
+        for qid, q in request["questions"].items():
+            orders = [None]
+            if q["type"] == "choice" and perm > 1:
+                orders = cyclic_orders(len(q["criteria"]), perm)
+            for order in orders:
+                text, labels, order_used = self.cf.qblock(q, order)
+                sids = self._enc(self.cf.suffix_text(text))
+                max_suffix = max(max_suffix, len(sids))
+                queries.append(Query(qid, order_used, sids, self.trie(labels)))
+        budget = self.max_prompt_tokens - max_suffix - 4
+        truncated = 0
+        if len(prefix_ids) > budget:
+            if self.on_overflow != "truncate":
+                raise ValueError(f"max_tokens_exceeded: state+question needs {len(prefix_ids) + max_suffix} tokens, "
+                                 f"limit {self.max_prompt_tokens}")
+            closing = self.cf.encode("\n</document>\n\n")  # keep the head of the state; flagged in the response
+            truncated = len(prefix_ids) - max(budget - len(closing), 16) - len(closing)
+            prefix_ids = prefix_ids[:max(budget - len(closing), 16)] + closing
+        pl = Plan(request, prefix_ids, queries)
+        pl.truncated_tokens = truncated
+        return pl
+
+    # ---------------------------------------------------------------- combination
+
+    def label_logprobs(self, qu: Query) -> np.ndarray:
+        tr = qu.trie
+        if not tr.two_step:
+            s = [_lse([qu.step1.get(t, -math.inf) for t in tr.first_variants[sym]]) for sym in tr.labels]
+        else:
+            s = []
+            for lab in tr.labels:
+                a = _lse([qu.step1.get(t, -math.inf) for t in tr.first_variants[lab[0]]])
+                b = _lse([qu.step2.get(lab[0], {}).get(t, -math.inf) for t in tr.second_variants[lab[1]]])
+                s.append(a + b)
+        s = np.array(s, dtype=float)
+        if not np.isfinite(s).any():
+            s = np.zeros_like(s)
+        s = np.where(np.isfinite(s), s, -1e4)
+        return s - (s.max() + np.log(np.exp(s - s.max()).sum()))
+
+    def combine(self, plan: Plan, round_dp: int | None = None, raw: bool = False) -> dict:
+        req = plan.request
+        by_q: dict[str, list[np.ndarray]] = {}
+        for qu in plan.queries:
+            lp_disp = self.label_logprobs(qu)                   # display positions
+            lp = np.empty_like(lp_disp)
+            lp[np.array(qu.order)] = lp_disp                     # -> option_keys order
+            by_q.setdefault(qu.qid, []).append(lp)
+        answers, logp = {}, {}
+        for qid, q in req["questions"].items():
+            z = np.mean(by_q[qid], axis=0)
+            logp[qid] = z.tolist()
+            T = float(self.temps.get(q["type"], 1.0))
+            z = z / T
+            p = np.exp(z - z.max())
+            p /= p.sum()
+            answers[qid] = p.tolist() if raw else wire_answer(q, p, round_dp)
+        out = {"answers": answers}
+        if os.environ.get("VERDICT_DUMP_DIST") == "1":   # eval/parity/run.py --dump-dist: pre-temperature log-probs
+            out["_logp"] = logp
+        cache = os.environ.get("VERDICT_LOGP_CACHE")
+        if cache:   # request-content-keyed log-probs (verdict.replay rebuilds ensembles for any request, incl. flips)
+            import json
+            with open(cache, "a") as fh:
+                fh.write(json.dumps({"h": request_key(req), "logp": logp}) + "\n")
+        if plan.truncated_tokens:
+            out["truncated_state_tokens"] = plan.truncated_tokens
+        return out