duplexjev 0.1.0__tar.gz → 0.2.0__tar.gz

{duplexjev-0.1.0/duplexjev.egg-info → duplexjev-0.2.0}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.4
 Name: duplexjev
-Version: 0.1.0
-Summary: Batched typed speech decisions without decoding: one forward pass answers many closed-set questions about many calls.
+Version: 0.2.0
+Summary: Jev-style typed decisions about speech with zero decode steps: one forward pass answers many option groups about many clips.
 Author: Adventists.ai
 License: Apache-2.0
 Project-URL: Homepage, https://adventists-ai.github.io/duplexjev/
@@ -41,28 +41,31 @@ Dynamic: license-file
 
 # duplexjev
 
-Batched typed speech decisions without decoding. Every question — *has the user finished? which filler? which intent?*
-— is answered as a probability over its options from a **single forward pass**, with no ASR or text decoding. Many
-questions about many calls share one pass, with exact prefix sharing.
+Jev-style typed decisions about speech with **zero decode steps**. Give it audio and one or more option groups
+(*has the user finished? which filler? which intent? speaker gender?*); every group comes back as a probability over
+its options from a single forward pass. Many groups about many calls share one pass, with exact prefix sharing.
 
 ```bash
-pip install duplexjev            # text models
-pip install "duplexjev[all]"     # + speech checkpoints and the HTTP server
+pip install "duplexjev[speech]"      # add [all] for the HTTP server
 ```
 
 ```python
 from duplexjev import Decider, Question
 
-d = Decider.from_pretrained("Qwen/Qwen3-8B")        # any causal LM; or a speech checkpoint (see below)
-qs = [Question("turn", "Has the user finished the turn?", ["finished", "not finished"]),
-      Question("intent", "What does the user want?", ["climate", "media", "navigation", "phone"])]
-d.decide(["Turn on the air conditioning", "Navigate to the"], qs)
-# [{'turn': {'answer': 'finished', 'confidence': 0.97, 'probs': {...}}, 'intent': {...}}, {...}]
-```
+d = Decider.from_pretrained("fixie-ai/ultravox-v0_6-qwen-3-32b")   # or a DuplexJev adapter
+groups = [Question("turn", "Has the user finished the turn?", ["finished", "not finished"]),
+          Question("gender", "Speaker gender", ["female", "male"])]
+
+d.decide("call.wav", groups)
+# {'turn': {'answer': 'finished', 'confidence': 0.97, 'probs': {...}}, 'gender': {...}}
 
-Speech checkpoints (Ultravox format, including the DuplexJev adapters) take audio directly:
-`Decider.from_pretrained("fixie-ai/ultravox-v0_6-qwen-3-32b").decide(["call1.wav", "call2.wav"], qs)`.
+# many clips: each option group names its clip(s) with audio=<id> | [ids] | "*"
+d.decide_batch({"car1": "a.wav", "car2": "b.wav"},
+               [Question("turn", "Has the user finished the turn?", ["finished", "not finished"], audio="*"),
+                Question("gender", "Speaker gender", ["female", "male"], audio="car2")])
+# {'car1': {'turn': ...}, 'car2': {'turn': ..., 'gender': ...}}
+```
 
-Serve with tick batching: `duplexjev serve --model <id> --tick-ms 160`.
+Serve with tick batching: `duplexjev serve --model <checkpoint> --tick-ms 160`.
 
 Project: https://github.com/adventists-ai/duplexjev · License: Apache-2.0

duplexjev-0.2.0/PYPI.md

@@ -0,0 +1,30 @@
+# duplexjev
+
+Jev-style typed decisions about speech with **zero decode steps**. Give it audio and one or more option groups
+(*has the user finished? which filler? which intent? speaker gender?*); every group comes back as a probability over
+its options from a single forward pass. Many groups about many calls share one pass, with exact prefix sharing.
+
+```bash
+pip install "duplexjev[speech]"      # add [all] for the HTTP server
+```
+
+```python
+from duplexjev import Decider, Question
+
+d = Decider.from_pretrained("fixie-ai/ultravox-v0_6-qwen-3-32b")   # or a DuplexJev adapter
+groups = [Question("turn", "Has the user finished the turn?", ["finished", "not finished"]),
+          Question("gender", "Speaker gender", ["female", "male"])]
+
+d.decide("call.wav", groups)
+# {'turn': {'answer': 'finished', 'confidence': 0.97, 'probs': {...}}, 'gender': {...}}
+
+# many clips: each option group names its clip(s) with audio=<id> | [ids] | "*"
+d.decide_batch({"car1": "a.wav", "car2": "b.wav"},
+               [Question("turn", "Has the user finished the turn?", ["finished", "not finished"], audio="*"),
+                Question("gender", "Speaker gender", ["female", "male"], audio="car2")])
+# {'car1': {'turn': ...}, 'car2': {'turn': ..., 'gender': ...}}
+```
+
+Serve with tick batching: `duplexjev serve --model <checkpoint> --tick-ms 160`.
+
+Project: https://github.com/adventists-ai/duplexjev · License: Apache-2.0

{duplexjev-0.1.0 → duplexjev-0.2.0}/README.md

@@ -52,7 +52,7 @@ state + N typed questions ──► frozen LLM (Qwen3-32B), ONE forward pass
 
 ## 2. News
 
-- **2026-09** — Paper submitted to ICASSP 2027. Released the `duplexjev` package (batched decider, tick server), research code, latency benchmarks and project page.
+- **2026-09** — Paper submitted to ICASSP 2027. Released the [`duplexjev`](https://pypi.org/project/duplexjev/) package on PyPI (batched decider, tick server), research code, latency benchmarks and project page.
 - **2026-10-10 (planned)** — Speech-to-Decision commercial API.
 - **2026-10-15 (planned)** — Open-source inference pipeline and model weights.
 
@@ -73,45 +73,55 @@ are named `DuplexJev-<variant>-<ASR encoder>-<LLM>`, so adapters for other model
 ## 4. Quick start
 
 ```bash
-pip install "duplexjev[all] @ git+https://github.com/adventists-ai/duplexjev.git"
+pip install "duplexjev[speech]"      # add [all] for the HTTP server
 ```
 
-**Any open LLM, transcripts in** (no speech model needed):
+**One clip, several option groups** (the default):
 
 ```python
 from duplexjev import Decider, Question
 
-d = Decider.from_pretrained("Qwen/Qwen3-8B")
-qs = [Question("turn", "Has the user finished the turn?", ["finished", "not finished"]),
-      Question("intent", "What does the user want?", ["climate", "media", "navigation", "phone"])]
-d.decide(["Turn on the air conditioning", "Navigate to the"], qs)
-# [{'turn': {'answer': 'finished', 'confidence': 0.97, 'probs': {...}}, 'intent': {...}}, {...}]
+d = Decider.from_pretrained("fixie-ai/ultravox-v0_6-qwen-3-32b")   # DuplexJev adapters load the same way
+groups = [Question("turn", "Has the user finished the turn?", ["finished", "not finished"]),
+          Question("filler", "Which filler fits?", ["Sure —", "One moment —", "(stay silent)"]),
+          Question("gender", "Speaker gender", ["female", "male"])]
+
+d.decide("call_017.wav", groups)
+# {'turn': {'answer': 'finished', 'confidence': 0.97, 'probs': {...}}, 'filler': {...}, 'gender': {...}}
 ```
 
-**Speech checkpoints, audio in** (Ultravox format; the DuplexJev adapters load the same way):
+**Many clips, many option groups** (advanced): each group names the clip(s) it is about with `audio=<id>`, a list
+of ids, or `"*"` for every clip. Everything runs in one batched pass.
 
 ```python
-d = Decider.from_pretrained("fixie-ai/ultravox-v0_6-qwen-3-32b")
-d.decide(["call_017.wav", "call_018.wav"], qs)          # one pass for all calls and all questions
+d.decide_batch(
+    {"car1": "a.wav", "car2": "b.wav", "car3": "c.wav"},
+    [Question("turn", "Has the user finished the turn?", ["finished", "not finished"], audio="*"),
+     Question("gender", "Speaker gender", ["female", "male"], audio=["car2", "car3"]),
+     Question("human", "Does the user need a human agent?", ["yes", "no"], audio="car1")],
+    context={"car1": "Driver asked to call home twice."})
+# {'car1': {'turn': ..., 'human': ...}, 'car2': {'turn': ..., 'gender': ...}, 'car3': {...}}
 ```
 
 **Batched server:** every request that arrives within one tick is answered in one pass.
 
 ```bash
 duplexjev serve --model fixie-ai/ultravox-v0_6-qwen-3-32b --tick-ms 160
-curl -s localhost:8000/v1/decide -H 'content-type: application/json' \
-  -d '{"text": "Call my", "questions": [{"id": "turn", "text": "Has the user finished?", "options": ["finished", "not finished"]}]}'
+# POST /v1/decide        {"audio_b64": ..., "questions": [...]}
+# POST /v1/decide_batch  {"audios": {"car1": ..., "car2": ...}, "questions": [{..., "audio": "car1"}]}
 ```
 
+Command line: `duplexjev decide --model M call.wav --q "turn|Has the user finished?|finished,not finished"`.
+
 What the package guarantees (unit tests in `tests/`, checked on Qwen3-32B and Ultravox v0.6 on an A800):
 
 - **0 decode steps.** Each answer is the next-token softmax over its option letters; options appear under permuted letters (`n_perm` averages several orders).
-- **Exact prefix sharing** (`mode="packed"`, default): the context and audio of an item are encoded once and all its questions are packed into one row under a block-diagonal mask. In fp32 on Qwen3-32B, packed and one-row-per-question differ by at most 2e-5; in bf16 the mean difference is 0.001–0.002, with at most 1/100 answers changed.
-- **Batch invariance.** An item's answers do not depend on which other items share the pass (fp32: max difference 1e-5). For speech this requires aligning every clip to whole audio tokens, which the package does (batched Ultravox inference otherwise leaks padding into the last audio token of shorter clips). In bf16, GPU kernels depend on batch shape, so running an item alone vs. in a batch changed 1/100 answers (text and speech).
-- **Checked accuracy.** qa100 through the package: 0.88–0.90 with Ultravox v0.6 (audio), 0.92 with Qwen3-32B (transcript).
+- **Exact prefix sharing** (`mode="packed"`, default): the context and audio of an item are encoded once and all its questions are packed into one row under a block-diagonal mask. The packing engine, checked in fp32 on Qwen3-32B, matches one-row-per-question to within 2e-5; in bf16 the mean difference is 0.001–0.002, with at most 1/100 answers changed.
+- **Batch invariance.** An item's answers do not depend on which other items share the pass (fp32: max difference 1e-5). For speech this requires aligning every clip to whole audio tokens, which the package does (batched Ultravox inference otherwise leaks padding into the last audio token of shorter clips). In bf16, GPU kernels depend on batch shape, so running an item alone vs. in a batch changed 1/100 answers.
+- **Checked accuracy.** qa100 through the package: 0.88–0.90 with Ultravox v0.6.
 - **Speed (current).** Plain PyTorch, one A800, bf16: 8 questions for 64 calls in 9 s (about 140 ms per call); 48 concurrent server requests are answered in one tick. The paper's latency numbers use a vLLM engine; a vLLM backend for the package is on the roadmap.
 
-Speech checkpoints need transformers 4.51–4.55 (installed by the `speech` extra); text models also work with newer versions.
+Speech checkpoints in Ultravox format need transformers 4.51–4.55, which the `speech` extra installs.
 
 More in [`examples/`](examples): tick-batch timing, server client, speech quick start.
 

duplexjev-0.2.0/duplexjev/__init__.py

@@ -0,0 +1,14 @@
+"""DuplexJev: Jev-style typed decisions about speech, with zero decode steps.
+
+    from duplexjev import Decider, Question
+    d = Decider.from_pretrained("adventists-ai/DuplexJev-A-Qwen3-ASR-0.6B-Qwen3-32B")
+    d.decide("call.wav", [Question("turn", "Has the user finished?", ["finished", "not finished"])])
+    d.decide_batch({"car1": "a.wav", "car2": "b.wav"},
+                   [Question("turn", "Has the user finished?", ["finished", "not finished"], audio="*"),
+                    Question("gender", "Speaker gender", ["female", "male"], audio="car2")])
+"""
+from .decider import Decider, load_audio
+from .question import Question
+
+__all__ = ["Decider", "Question", "load_audio"]
+__version__ = "0.2.0"

duplexjev-0.2.0/duplexjev/cli.py

@@ -0,0 +1,84 @@
+"""Command line.
+
+    duplexjev decide --model M call.wav --q "turn|Has the user finished?|finished,not finished"
+    duplexjev batch  --model M --audio car1=a.wav --audio car2=b.wav --questions groups.json
+    duplexjev serve  --model M --tick-ms 160
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+
+from .question import Question
+
+
+def _questions(args, need_audio=False) -> list[Question]:
+    qs = []
+    if args.questions:
+        with open(args.questions, encoding="utf-8") as f:
+            qs += [Question.from_dict(q) for q in json.load(f)]
+    for spec in args.q or []:
+        # "id|question text|opt1,opt2,..." or, for batch, "clip|id|question text|opt1,opt2,..."
+        parts = spec.split("|")
+        clip = parts.pop(0) if need_audio and len(parts) == 4 else None
+        qid, text, opts = parts[0], parts[1], parts[2]
+        qs.append(Question(qid, text, [o.strip() for o in opts.split(",")], lang=args.lang or "en", audio=clip))
+    if not qs:
+        sys.exit("give --questions FILE.json or at least one --q")
+    return qs
+
+
+def main(argv=None):
+    ap = argparse.ArgumentParser(prog="duplexjev")
+    sub = ap.add_subparsers(dest="cmd", required=True)
+
+    common = argparse.ArgumentParser(add_help=False)
+    common.add_argument("--model", required=True, help="speech checkpoint (Ultravox format), HF id or path")
+    common.add_argument("--text-model", help="override the LLM the checkpoint points to (e.g. a local path)")
+    common.add_argument("--audio-model", help="override the encoder the checkpoint points to")
+    common.add_argument("--device")
+
+    q = argparse.ArgumentParser(add_help=False)
+    q.add_argument("--questions", help="JSON list of option groups {id, text, options, lang[, audio]}")
+    q.add_argument("--q", action="append", help="inline option group 'id|text|opt1,opt2' (batch: 'clip|id|text|opts')")
+    q.add_argument("--lang", choices=["en", "zh"])
+    q.add_argument("--mode", default="packed", choices=["packed", "batch"])
+    q.add_argument("--n-perm", type=int, default=1)
+
+    d = sub.add_parser("decide", parents=[common, q], help="option groups about one audio clip")
+    d.add_argument("audio")
+    d.add_argument("--context")
+
+    b = sub.add_parser("batch", parents=[common, q], help="option groups about many clips (groups name their clip)")
+    b.add_argument("--audio", action="append", required=True, help="clip as id=path (repeatable)")
+
+    s = sub.add_parser("serve", parents=[common], help="HTTP server; batches all requests of each tick")
+    s.add_argument("--host", default="0.0.0.0")
+    s.add_argument("--port", type=int, default=8000)
+    s.add_argument("--tick-ms", type=float, default=160.0)
+    s.add_argument("--max-items", type=int)
+    s.add_argument("--max-tokens", type=int)
+
+    args = ap.parse_args(argv)
+    from .decider import Decider
+
+    dec = Decider.from_pretrained(args.model, device=args.device, text_model=args.text_model, audio_model=args.audio_model)
+    if args.cmd == "decide":
+        res = dec.decide(args.audio, _questions(args), context=args.context, lang=args.lang, mode=args.mode, n_perm=args.n_perm)
+        print(json.dumps({"audio": args.audio, "answers": res, "pass": dec.last_stats}, ensure_ascii=False, indent=1))
+    elif args.cmd == "batch":
+        audios = dict(a.split("=", 1) for a in args.audio)
+        res = dec.decide_batch(audios, _questions(args, need_audio=True), lang=args.lang, mode=args.mode, n_perm=args.n_perm)
+        print(json.dumps({"answers": res, "pass": dec.last_stats}, ensure_ascii=False, indent=1))
+    else:
+        import uvicorn
+
+        from .server import create_app
+
+        uvicorn.run(create_app(dec, tick_ms=args.tick_ms, max_items=args.max_items, max_tokens=args.max_tokens),
+                    host=args.host, port=args.port)
+
+
+if __name__ == "__main__":
+    main()

{duplexjev-0.1.0 → duplexjev-0.2.0}/duplexjev/decider.py

@@ -1,4 +1,8 @@
-"""Batched typed decisions from one forward pass, with exact prefix sharing.
+"""Typed decisions about speech, answered from one forward pass with exact prefix sharing.
+
+Input is audio plus one or more Jev-style option groups (``Question``); output is a probability for every option.
+``Decider.decide`` answers the option groups for one clip; ``Decider.decide_batch`` answers many option groups about
+many clips at once, each group naming the clip(s) it is about.
 
 Two layouts give the same answers (up to floating-point noise):
 
@@ -25,8 +29,8 @@ SAMPLE_RATE = 16000
 _SENTINEL = "⁣QUESTION⁣"
 
 PROMPTS = {
-    "en": {"context": "Context:\n{c}", "transcript": "The user said: {t}", "audio": "The user said: <|audio|>"},
-    "zh": {"context": "上下文：\n{c}", "transcript": "用户说：{t}", "audio": "用户说：<|audio|>"},
+    "en": {"context": "Context:\n{c}", "audio": "The user said: <|audio|>"},
+    "zh": {"context": "上下文：\n{c}", "audio": "用户说：<|audio|>"},
 }
 
 
@@ -53,32 +57,23 @@ def load_audio(x: Any) -> np.ndarray:
     return a
 
 
-def as_item(x: Any) -> dict:
-    """An item is a dict with any of: ``audio``, ``text`` (transcript), ``context``, ``questions``, ``lang``."""
-    if isinstance(x, dict):
-        return x
-    if isinstance(x, str):
-        return {"text": x}
-    return {"audio": x}
-
-
 # ---------------------------------------------------------------------------------------------------------- engine
 class Decider:
-    """Answer typed closed-set questions about speech (or text) with zero decode steps.
+    """Answer Jev-style option groups about speech with zero decode steps.
 
-    Create with :meth:`from_pretrained`. The same object serves text-only LLMs (inputs are transcripts) and
-    Ultravox-format speech checkpoints, including the DuplexJev adapters.
+    Create with :meth:`from_pretrained` from a speech checkpoint in Ultravox format: the DuplexJev adapters
+    (ASR encoder + trained connector + frozen LLM) or the released Ultravox models.
     """
 
-    def __init__(self, model, tokenizer, processor=None, *, device=None, is_audio: bool = False):
+    def __init__(self, model, processor, *, device=None):
         self.model = model.eval()
-        self.tok = tokenizer
         self.processor = processor
-        self.is_audio = is_audio
+        self.tok = processor.tokenizer
+        tokenizer = self.tok
         if device is None or str(device) == "auto":  # sharded model: inputs go where the embeddings live
-            device = model.get_input_embeddings().weight.device if not is_audio else model.language_model.get_input_embeddings().weight.device
+            device = model.language_model.get_input_embeddings().weight.device
         self.device = torch.device(device)
-        self.lm = model.language_model if is_audio else model
+        self.lm = model.language_model
         self.base = getattr(self.lm, self.lm.base_model_prefix)
         self.head = self.lm.get_output_embeddings()
         self.pad_id = tokenizer.pad_token_id if tokenizer.pad_token_id is not None else tokenizer.eos_token_id
@@ -98,7 +93,7 @@ class Decider:
         audio_model: str | None = None,
         **kwargs,
     ) -> "Decider":
-        """Load a text LLM (any causal LM on the Hugging Face hub) or an Ultravox-format speech checkpoint.
+        """Load a speech checkpoint (Ultravox format: encoder + connector, pointing to its frozen LLM).
 
         ``device="auto"`` shards the model over all visible GPUs (needs ``accelerate``).
         ``text_model`` / ``audio_model`` override the LLM and encoder a speech checkpoint points to (e.g. a local
@@ -113,7 +108,13 @@ class Decider:
             dtype = torch.bfloat16 if str(device).startswith("cuda") or device == "auto" else torch.float32
         dmap = "auto" if device == "auto" else {"": device}
         cfg = transformers.AutoConfig.from_pretrained(name_or_path, trust_remote_code=True)
-        if getattr(cfg, "model_type", "") == "ultravox":
+        if getattr(cfg, "model_type", "") != "ultravox":
+            raise ValueError(
+                f"{name_or_path!r} is not a speech checkpoint. duplexjev answers questions about audio and needs an "
+                "encoder + connector checkpoint in Ultravox format, e.g. adventists-ai/DuplexJev-A-Qwen3-ASR-0.6B-Qwen3-32B "
+                "or fixie-ai/ultravox-v0_6-qwen-3-32b."
+            )
+        if True:
             if dkw == "dtype":
                 import warnings
 
@@ -140,13 +141,7 @@ class Decider:
                 processor.audio_processor = transformers.WhisperProcessor(
                     feature_extractor=ap, tokenizer=processor.tokenizer
                 )
-            return cls(model, processor.tokenizer, processor, device=None if device == "auto" else device, is_audio=True)
-        tok = transformers.AutoTokenizer.from_pretrained(name_or_path, **{k: v for k, v in kwargs.items() if k == "revision"})
-        if device == "auto":
-            model = transformers.AutoModelForCausalLM.from_pretrained(name_or_path, device_map="auto", **{dkw: dtype}, **kwargs)
-        else:
-            model = transformers.AutoModelForCausalLM.from_pretrained(name_or_path, **{dkw: dtype}, **kwargs).to(device)
-        return cls(model, tok, device=None if device == "auto" else device, is_audio=False)
+            return cls(model, processor, device=None if device == "auto" else device)
 
     # ------------------------------------------------------------------ prompt pieces
     def _head_tail(self, context: str | None, content: str, lang: str) -> tuple[str, str]:
@@ -192,29 +187,12 @@ class Decider:
         return np.concatenate([np.zeros(r, dtype=np.float32), a]) if r else a
 
     def _encode_prefix(self, item: dict, lang: str) -> dict:
-        audio = item.get("audio")
-        text = item.get("text")
-        if audio is not None and not self.is_audio:
-            raise ValueError("this model is text-only; pass `text` (a transcript) instead of `audio`")
-        if audio is not None:
-            content = PROMPTS[lang]["audio"]
-            if text:
-                content = PROMPTS[lang]["transcript"].format(t=text) + "\n" + content
-        elif text is not None:
-            content = PROMPTS[lang]["transcript"].format(t=text)
-        else:
-            raise ValueError("each item needs `audio` or `text`")
-        head, tail = self._head_tail(item.get("context"), content, lang)
-        if audio is not None:
-            p = self.processor(text=head, audio=self._align(load_audio(audio)), sampling_rate=SAMPLE_RATE, return_tensors="pt")
-            ids = p["input_ids"][0].tolist()
-            aud = {k: p[k] for k in ("audio_values", "audio_lens", "audio_token_len", "audio_token_start_idx", "audio_batch_size") if k in p}
-            if "audio_batch_size" not in aud:
-                aud["audio_batch_size"] = torch.tensor([p["audio_values"].shape[0]])
-        else:
-            ids = self.tok(head, add_special_tokens=False)["input_ids"]
-            aud = None
-        return {"ids": ids, "audio": aud, "tail": tail}
+        head, tail = self._head_tail(item.get("context"), PROMPTS[lang]["audio"], lang)
+        p = self.processor(text=head, audio=self._align(load_audio(item["audio"])), sampling_rate=SAMPLE_RATE, return_tensors="pt")
+        aud = {k: p[k] for k in ("audio_values", "audio_lens", "audio_token_len", "audio_token_start_idx", "audio_batch_size") if k in p}
+        if "audio_batch_size" not in aud:
+            aud["audio_batch_size"] = torch.tensor([p["audio_values"].shape[0]])
+        return {"ids": p["input_ids"][0].tolist(), "audio": aud, "tail": tail}
 
     def _suffixes(self, qs: list[Question], tail: str, n_perm: int, seed: int):
         """Token ids of every (question, permutation) suffix, with the letter ids to read."""
@@ -228,7 +206,7 @@ class Decider:
         return out
 
     # ------------------------------------------------------------------ collation
-    def _collate(self, rows: list[tuple[list[int], dict | None]]) -> dict:
+    def _collate(self, rows: list[tuple[list[int], dict]]) -> dict:
         """Left-pad token rows; stack audio (right-padded) and shift audio start indices by the left padding."""
         L = max(len(r[0]) for r in rows)
         ids = torch.full((len(rows), L), self.pad_id, dtype=torch.long)
@@ -237,11 +215,9 @@ class Decider:
             ids[i, L - len(r):] = torch.tensor(r)
             mask[i, L - len(r):] = 1
         batch = {"input_ids": ids, "attention_mask": mask}
-        if any(a is not None for _, a in rows):
+        if True:
             vals, lens, tlen, start, bs = [], [], [], [], []
             for r, a in rows:
-                if a is None:
-                    raise ValueError("cannot mix items with and without audio in one call")
                 shift = L - len(r)
                 vals += list(a["audio_values"])
                 lens.append(a["audio_lens"])
@@ -268,46 +244,104 @@ class Decider:
         return z.float()
 
     # ------------------------------------------------------------------ public API
-    @torch.no_grad()
     def decide(
         self,
-        items: Iterable[Any],
-        questions: Sequence[Question | dict] | None = None,
+        audio: Any,
+        questions: Question | dict | Sequence[Question | dict],
+        *,
+        context: str | None = None,
+        lang: str | None = None,
+        **options,
+    ) -> dict:
+        """Answer one or more option groups about one audio clip.
+
+        Args:
+            audio: a file path, a 16 kHz float array, or an ``(array, sample_rate)`` tuple.
+            questions: one ``Question`` (option group) or a list of them.
+            context: optional dialogue context or state, shared by all questions.
+            lang: language of the fixed prompt words (``"en"``/``"zh"``); defaults to the first question's.
+            **options: ``mode``, ``n_perm``, ``seed`` (see :meth:`decide_batch`).
+
+        Returns:
+            ``{question_id: {"answer": option, "confidence": p, "probs": {option: p}}}``.
+        """
+        qs = [questions] if isinstance(questions, (Question, dict)) else list(questions)
+        qs = [q.for_audio(None) if isinstance(q, Question) else q for q in qs]
+        item = {"audio": audio, "questions": qs, "context": context, "lang": lang}
+        return self._decide_items([item], **options)[0]
+
+    def decide_batch(
+        self,
+        audios: dict | Sequence[Any],
+        questions: Sequence[Question | dict],
         *,
+        context: dict | Sequence[str | None] | None = None,
+        lang: str | None = None,
         mode: str = "packed",
         n_perm: int = 1,
         seed: int = 0,
-        lang: str | None = None,
         max_items: int | None = None,
         max_tokens: int | None = None,
-    ) -> list[dict]:
-        """Answer every question about every item in one batched pass.
+    ) -> dict:
+        """Answer many option groups about many clips in one batched pass.
+
+        Every option group names the clip(s) it is about with ``Question(..., audio=<id>)``, where ``<id>`` is a key
+        of ``audios`` (or an index if ``audios`` is a list), a list of ids, or ``"*"`` for every clip.
 
         Args:
-            items: audio arrays / file paths, transcript strings, or dicts with ``audio`` and/or ``text``, optional
-                ``context`` and optional per-item ``questions``.
-            questions: questions asked of every item (unless an item brings its own).
+            audios: ``{id: audio}`` or a list of audio (ids are then 0, 1, ...). Audio as in :meth:`decide`.
+            questions: option groups, each with ``audio=`` set.
+            context: optional context per clip (``{id: str}`` or a list aligned with ``audios``).
             mode: ``"packed"`` (prefix sharing, default) or ``"batch"`` (one row per question; reference).
-            n_perm: average over this many option orders (reduces position bias; costs more suffix tokens).
+            n_perm: average over this many option orders (reduces position bias; costs more tokens).
             seed: selects the option orders.
-            lang: language of the fixed prompt words; defaults to each item's ``lang`` or the first question's.
-            max_items: split into forward passes of at most this many items.
-            max_tokens: split so that each pass holds at most this many prompt tokens (prefixes + all suffixes).
-                Bounds activation memory; an item larger than the budget still runs, alone.
+            max_items / max_tokens: split into several forward passes of bounded size.
 
         Returns:
-            one dict per item: ``{question_id: {"answer": option, "confidence": p, "probs": {option: p}}}``.
+            ``{audio_id: {question_id: {"answer", "confidence", "probs"}}}`` for every clip that was asked something.
         """
-        items = [as_item(x) for x in items]
+        amap = dict(audios) if isinstance(audios, dict) else dict(enumerate(audios))
+        if not amap:
+            return {}
+        if context is None:
+            cmap = {}
+        elif isinstance(context, dict):
+            cmap = context
+        else:
+            cmap = dict(zip(amap, context))
+        per = {k: [] for k in amap}
+        for q in as_questions(questions, unique=False):
+            if q.audio is None:
+                raise ValueError(f"question {q.id!r}: set audio=<clip id> (or '*' for all clips) in decide_batch")
+            targets = list(amap) if q.audio == "*" else (q.audio if isinstance(q.audio, (list, tuple)) else [q.audio])
+            for t in targets:
+                if t not in amap:
+                    raise ValueError(f"question {q.id!r} refers to unknown audio {t!r}; known: {list(amap)}")
+                per[t].append(q)
+        items, ids = [], []
+        for k, qs in per.items():
+            if qs:
+                seen = [q.id for q in qs]
+                if len(set(seen)) != len(seen):
+                    raise ValueError(f"audio {k!r}: duplicate question ids {seen}")
+                items.append({"audio": amap[k], "questions": qs, "context": cmap.get(k), "lang": lang})
+                ids.append(k)
+        res = self._decide_items(items, mode=mode, n_perm=n_perm, seed=seed, max_items=max_items, max_tokens=max_tokens)
+        return dict(zip(ids, res))
+
+    @torch.no_grad()
+    def _decide_items(self, items, *, mode="packed", n_perm=1, seed=0, max_items=None, max_tokens=None) -> list[dict]:
         if not items:
             return []
         t0 = time.perf_counter()
         plans = []
         for it in items:
-            qs = as_questions(it.get("questions") or questions or [])
+            qs = as_questions(it.get("questions") or [])
             if not qs:
                 raise ValueError("no questions given")
-            lg = lang or it.get("lang") or qs[0].lang
+            if it.get("audio") is None:
+                raise ValueError("every item needs audio")
+            lg = it.get("lang") or qs[0].lang
             pre = self._encode_prefix(it, lg)
             plans.append({"qs": qs, "pre": pre, "sufs": self._suffixes(qs, pre["tail"], n_perm, seed)})
         t_enc = time.perf_counter()
@@ -315,18 +349,16 @@ class Decider:
             raise ValueError("mode must be 'packed' or 'batch'")
         run = self._run_packed if mode == "packed" else self._run_batch
         n_pass = 0
-        for with_audio in (True, False):  # speech and text-only items go in separate passes
-            group = [p for p in plans if (p["pre"]["audio"] is not None) == with_audio]
-            for chunk in _chunks(group, max_items, max_tokens, packed=mode == "packed"):
-                run(chunk)
-                n_pass += 1
+        for chunk in _chunks(plans, max_items, max_tokens, packed=mode == "packed"):
+            run(chunk)
+            n_pass += 1
         if self.device.type == "cuda":
             torch.cuda.synchronize(self.device)
         t_fwd = time.perf_counter()
         results = [self._collect(p) for p in plans]
         self.last_stats = {
             "mode": mode,
-            "items": len(items),
+            "clips": len(items),
             "questions": sum(len(p["qs"]) for p in plans),
             "prefix_tokens": sum(len(p["pre"]["ids"]) for p in plans),
             "suffix_tokens": sum(len(s["ids"]) for p in plans for s in p["sufs"]),
@@ -344,26 +376,17 @@ class Decider:
                 rows.append((p["pre"]["ids"] + s["ids"], p["pre"]["audio"]))
                 where.append((pi, si))
         batch = self._collate(rows)
-        if self.is_audio:  # Ultravox merges audio inside its own forward; read last-position logits
-            last = self.model(**batch, use_cache=False, logits_to_keep=1).logits[:, -1].float()
-            for r, (pi, si) in enumerate(where):
-                s = plans[pi]["sufs"][si]
-                s["z"] = last[r, torch.tensor(s["letters"], device=last.device)]
-            return
-        h = self.base(input_ids=batch["input_ids"], attention_mask=batch["attention_mask"], use_cache=False)
-        h = h.last_hidden_state[:, -1]
+        # Ultravox merges the audio inside its own forward; read the last-position logits
+        last = self.model(**batch, use_cache=False, logits_to_keep=1).logits[:, -1].float()
         for r, (pi, si) in enumerate(where):
             s = plans[pi]["sufs"][si]
-            s["z"] = self._letter_logits(h[r], s["letters"])
+            s["z"] = last[r, torch.tensor(s["letters"], device=last.device)]
 
     def _run_packed(self, plans):
         # 1) prefill all prefixes together (left-padded); keep the KV cache
         batch = self._collate([(p["pre"]["ids"], p["pre"]["audio"]) for p in plans])
         P = batch["input_ids"].shape[1]
-        if self.is_audio:
-            out = self.model(**batch, use_cache=True, logits_to_keep=1)
-        else:
-            out = self.base(input_ids=batch["input_ids"], attention_mask=batch["attention_mask"], use_cache=True)
+        out = self.model(**batch, use_cache=True, logits_to_keep=1)
         cache = out.past_key_values
         # 2) all suffixes of an item in one row; block-diagonal causal mask over the suffix part
         B = len(plans)